No problem
In the USA there is no problem in asking a man if he is a Mason. We wear rings, belt buckles, baseball caps, etc. I never leave the house, without at least one piece of 'bling' !! I personally sponsored the legislation, to get masonic vehicle license plates in Kentucky.
In some countries, like France, masonry is very discreet, and most masons are encouraged to keep their masonic membership very low-profile. When I lived in Paris (1986-1987), I took the brothers advice, and kept my bling at home.
In Saudi Arabia, masonry is against the law. I put all my masonic items away, and did not acknowledge membership to anyone. If asked if I was a mason, I declined to answer, or pretended that I did not know what they were talking about.
Masonry is now legal in Russia, I attended a lodge in Moscow in 2006, which was an amazing experience. Some of the men who were in lodge with me, risked prison, to keep the Craft alive during the communist times.
I now live in Iraq. Masonry operates openly there, the last lodge was shut down in 1958, when the British military lodges departed. Masonry was totally illegal under the Saddam Hussein government. I am honored to have a part in restoring Masonry to a great country.