Well, the housing prices were only one aspect of the process.
It started with a presumption that prices would go up forever. This justified the reliance on sub-prime mortgages, they just assumed they could continually refinance.
Then those mortgages were packaged into exotic financial instruments, like credit default swaps, and were sold and borrowed against.
The trading of those instruments bolstered the stock market causing people to move money from safe, low yield savings into the market.
Thus anyone who decided to invest based on the promise of large mutual fund returns instead of leaving money in the bank or buying CDs, really did have actual wealth disapear when everything crashed.
Again, that's just one simple aspect of the monstrosity.
The World Economic Forum estimated that 40% of worldwide wealth was lost in the collapse:
http://www.telegraph.co.uk/finance/...risis-has-destroyed-40pc-of-world-wealth.html
But like you said, some of that was imaginary in the first place. It's all a mess.