Well, Panda, let me say first that I agree with you: there is nothing offensive about our fraternity. But that, of course, is our opinion. There are many reasons that people object to Freemasonry, and a comprehensive list of them would fill several volumes. Let's look at three of the ones I have encountered most frequently:
1. Freemasonry is a fraternity for religious men but we do not, in most jurisdictions, specify what religion (or spirituality) a man must hold to become a member. Because we pray to a 'non-specific' God (the Great Architect of the Universe), some people are offended. They believe that their God is the only true God and that to deny that by refusing to pray in his name is blasphemy.
2. Freemasonry has secrets. We have secrets that we keep even from the leaders and administrations of the various faiths out there. There are those who believe that because we have secrets, we could be up to anything, and that anyone who could be up to anything has got to be up to something evil.
3. This one is more specific to Catholicism: Because some of our members, in the distant past, were politically active and advocated the overthrow of the royal houses in Europe (and European royalty were functionaries of the Church as 'Defenders of the Faith"), the Church was directly threatened by Freemasons (note I did not say 'Freemasonry', but 'Freemasons'). Because of this threat to the Church, along with other reasons (two of which are above), several Popes have issued proclamations (called Papal Bulls) which forbid Catholics from becoming Masons.
So there are three examples, out of hundreds, why 'church going people' are taught to be anti-Masonic.