Well, that's not always the case. If it's purely a social relationship, then what you say is valid, but that changes if the person has any power over you. If the person is your boss, it can affect your employment and promotion opportunities. Similar situations exist for police, judges, elected representatives, teachers, etc.
Yeah, it's a different story when you're talking about those types of people. One thing I find to be most effective is to only speak to them when they ask a question. I refuse to answer any religious questions unless I know for sure they would be okay with my answer. If I don't know the person, then he/she will not know me. In fact, I'm more open with my views on the internet than I am in person.
Speaking of the boss, when I was hired for my current job, the president of the company asked me what church I went to. This immediately threw up a red flag on the situation. I knew if I said I didn't go to church, then he would try to promote the church he goes to, so I didn't want to go that route. I knew if I said I wasn't religious, then he would try to question my morality and second guess my ability to perform in a trustworthy manner. I was practically begging for a job at the time, so I had to pull a slick move. I thought of the very last church I've actually been to in the past and gave him that name. He then says, "Oh yeah. That's a nice small church. I've been there once. Has the size changed any?" - [I KNEW IT! He only goes for bigger churches!] - I reply with, "Nope, it's still as small as ever." - LOL... and so I got the job. Hey, I didn't lie. I really went to that church... ONCE... to help them move some furniture.