"I can allow that," I conceded. "But any other poking around and I'm going to have to kick your ass. Or anyone else's ass. I am not a fan of having my brain examined."
Other options:
no subject
"I can allow that," I conceded. "But any other poking around and I'm going to have to kick your ass. Or anyone else's ass. I am not a fan of having my brain examined."