When I was in the US, I said "dammit" all the time. I am not religious and don't consider it all that bad.
Sometimes I said the f-word too, but only when I was really upset.
Since coming to Japan, I have only had the occasion to use やがる once, and that was as a joke.