Sony/Philips hit on a pretty good form factor in developing the orginal CD. It's survived three subsequent generations of data formats. Smaller disks like the mini-CD & miniDVD±R give it versatility.
This form factor has given technology developers a standard canvass on which to make each new generation. And the great thing is that it's comparatively trival to make them backward compatible as each new format is introduced. Any obstacles to that so far have been because of IP issues not technology.
Flash memory will always be more expensive than disc-based. The latest manufacturing processes require investments of billions (yeah, with a B) of dollars to establish, the process takes much longer and is more material intensive. On the other hand even the most expensive disc based production can be done with millions of dollars investment, can be done almost anywhere and for literally pennies a disc in volume.
You talk of cheap, high capacity flash memory as the medium. Consider that today an 8GB flash drive costs only, lets say, $100. Well that's before you put the content on it to the tune of no less than $20.
Hmmm... Spiderman 3 on DVD or BluRay for $20, or a 480p (DVD quality) version on an 8GB flash for $120? And we're not even talking about HD yet!
As for streaming or downloads replacing physical media? For too many people it doesn't add up. As much as you might wish otherwise, most consumers do not worship at the altar of their PC or media center. It's still to complicated for the average soccer mom to deal with this crap. There are millions of people with digital cameras that take them to Wal-Mart, have their pictures printed, and hopefully burned to a CD or DVD, and then they erase the card. Never putting them in the computer... Ya think they're gonna go without a disc for their movies? Not anytime soon.
Plus with portable DVD's and players in mini-vans, etc... a hardware media like the disc has a long life.
I think if there isn't another HD format introduction (and I think the public won't tolerate another for a long time), those disc formats will have a long life... iTunes IMO is an anomoly because unlike movies, music can be divied up into nice little 2 or 3MB nuggets that sell for pennies and are easy to download. That ain't gonna happen with blockbuster Hollywood titles.
JMO.
striderx
P.S. - I'm not saying the Blu-Ray or HD-DVD are the end-all be-all of media, just that a cheap physical media of the established disc form factor, not flash or streaming, will remain a major if not dominat player for a long time.