Ignore what everyone just said. Apple is still giving out the SHSHs for iOS 4.0 on iPod touch 3G, so
you can still downgrade to 4.0 (and maintain the capability to downgrade to 4.0). They'll probably stop when iOS 4.1 is released.
Put your device into DFU Mode. Plug it into your computer, and turn it off. Press and hold the Sleep and Home buttons for ten seconds, then release the Sleep button (keep the Home button down). The iPod's screen should be off and your computer should recognize an "Apple Mobile Device (DFU Mode)."
Open up iTunes and Option-Click (Mac) or Shift-Click (Windows) to restore to
iOS 4.0.
Don't forget to save your SHSHs so you can downgrade in the future (instructions available on
the jailbreaking guide I'm maintaining *shameless plug*), and good luck!