It's a very complex problem and that cannot be fixed as easily as one would wish, but Bethesda have said they will be able to resolve the issue in a later patch. In the meantime they have issued the following recommendations:
- Disabling automatic saves seems to help, especially with large saves
- Free up some space on your hard drive, make sure you have at least 2 gigabytes for caching
- Use the "wait" feature in the game for at least 30 consecutive days if you want the game to "forget" game elements such as item and character location and position, places you've visited and so forth.