To add to that speculation, since the 60s there has also been an ethos of personal fulfillment and improvement. The exercise/gym movement seems to have benefited from this (also from the decline of more manufacturing jobs that may have offered more of an on the job workout). There is a new obligation to refine both the body and mind -- and a huge range of services that each tout themselves as an integral part of your personal improvement project. Then, of course, there is also the corresponding obligation parents have to schedule personal improvement/college resume building activities for their kids.
For many people, I am sure, these activities -- while not income-generating -- are still experienced as work. Hence the self-reporting of more work stress and less leisure time.
-WD