You have two ways of handling it, If you say he's a real PIA and you dont want or need him as a custie, then you have to just drop him and if he asks, say you dont need the grief,.
Or-- You need to look beyond the PIA principle and try to understand why he acts the way you describe, coming out with a notepad and a list of questions is NOT normal behaviour but maybe he is not "normally wired" If you can start to understand why he acts the way he does, (mayby autistic,controlling, OCD,Bi polar or other mental issues) ,where he needs a structure and order in life to avoid cracking up completley, Understanding these issues You should then be able to not let the eccentricties his personallity traits show effect your working relationship.
I am not suggesting you act as he phscotherapist merely try to see thing from his perspective could give you a little room for tolerating him before sacking him off.