People who ask about "Were you also big when you were young"

I don't understand that question, makes me so uncomfortable that the first thing these people think of when they see my dick is "I wonder if he was also big back when he was 13?"

That's borderline pedophilic in my opinion, or atleast really weird of a question to ask.

Am I the only one? And if you're that person that does this, why? Is it curiosity or some sort of kink?