That's how I understand it too. Greece loses an enormous amount of its potential tax take through exemptions on things such as income from shipping (bear in mind the Greek shipping industry is the largest in the world), capital gains, sale of shares traded on the Athens stock exchange and dividends received from Greek companies. There is also endemic tax evasion among the middle and upper classes. The fact that the lower-paid public sector workers, who actually do pay their taxes, are going to bear the brunt of the austerity measures is the primary source of the anger.
Another interesting fact I read recently is that Greek military spending is more than four times the EU average, so they could certainly take a big chunk off their deficit by reducing that. But instead of insisting that they do so, France and Germany are insisting that Greece buy its arms from them:
http://www.reuters.com/article/idUSTRE62M1Q520100323