The few places that I know that have fare-free public transit are probably not indicative of what it would be like to do this in a big city. Every once in a while here in the Bay Area, the transit folks join into a "Save The Air" push by making transit free. The last time they did this it turned into a bunch of problems stemming from the notion that a free seat on a moving bus or train can seem like an invitation to spend the day there. Groups of kids and homeless became fixtures on BART to the point where lots of people complained; petty crime jumped for that day, too.
It's not clear whether this would get old after the first week or even month.
One thing that article didn't mention was the additional cost of collecting money on busses: if people could just board (by using both the front and rear doors!) and sit down, the bus can leave the curb sooner. A LOT of time is spent on people putting the right fare into the box, etc. And if the bus misses the light, more people can try to get on the bus, causing cyclic behavior. I've spent three light cycles at busy corners before due to this phenomeonon.
/jordan