The original Xbox has had a cold boot software hack since...
checks notes...
July 2003. It was literally hacked before I was born.
Or if you mean "custom firmware" in a literal sense (flashing the BIOS of the motherboard itself), this has been known to be possible on most Xboxes for a similar amount of time. Recently, even the 1.6 model's flash ROM inside the proprietary Xyclops IC has been discovered to be flashable through its TTL interface:
https://www.xbox-scene.info/forums/topic/804-xyclomod-flash-the-on-board-bios-on-v16/