Now I'm going to preface this post with a little disclaimer. I am by no means a lawyer, don't really know any and you should check anything I write here with a local authority. It's a bit of a can of worms is the old subject of model releases and the rules change according to which part of the world you live in.
As a general rule though you don't need permission to take somebody's photo in a public place. It's not the taking of the photo that is the problem - it's what you do with the picture that can get you into hot water.
Take this picture of a little girl taking a break at a festival in Japan. There's no problem if I publish this picture in a magazine with a caption that explains what's happening. But if this picture runs on the front cover of the magazine it could be considered advertising and I would most likely need a model release for that.
Likewise if I was to use it in a billboard which said that this little girl loves the taste of Coke I would get into trouble. Or at least the person who published it would get into trouble. Remember the taking of the photograph is not the issue when you are in a public place, the publishing of it is. So if somebody uses it to advertise something and they know it doesn't have a release (because you the photographer told them) then they would be in trouble not you.
So do I carry a whole bunch of model releases around with me? You bet. Do I ever use them? Hardly ever. I mostly work in the editorial sector of the industry - books, magazines, newspapers. So I hardly ever need a release. If I think I have a cover shot I often give my subject a release form to fill out and promise to send them copies of the pictures and the cover of the magazine if I make it. Otherwise those pieces of paper stay in my bag.