Affording a $500k house isn't such a big deal. But having the downpayment and credit score typically is. IMHO, the $500k purchases are probably being made by people trading up. A mortgage costs about $500/$100k/month these days, and if you can make that "median income" of $43k, you'll get just about 100% of your Federal (and state, if you live in one of those) Income Tax burden back at the end of the year if you have a $400k mortgage. So a $500k house in that scenario is sorta like a $1400/mo rent. Of course, these days, a straight-up fixed 30yr $400k mortgage is for suckers, so you'd more likely find $250k of a 5/1 and $150k of an interest-only HELC and then you're down around $1k/mo net.
/jordan