I’m not sure what you say to people, (if they have already categorized your trade), will make any difference to their views. I think if you need to say something then get it wrong, can make it a lot worse, especially if the comments are transparent.
Sometimes a lot of explanation or over reaction can reveal something about the person being uncomfortable with themselves, and any defensive statements could simply compound the matter.
People will always judge other people, its human nature and you wont stop it.
I would suggest that "less is more" in these issues and an all knowing smile can have a real effect and if you feel you have to say something, keep it short and believable. The personal betterment statements rarely convinces anybody unless they were already involved with you before you started.
Anything else and I would be wondering who am I trying to convince them or me.
Just my views.