Most modern stuff the games run at a low privilege level to prevent things like this, hence why so many things these days rely on updates, menus, menu embedded functions and repair modes.
Classically consoles were more like real mode DOS in that multitasking was not really a thing. In which case "insert code however you will, jump to it" is the story. Save memory for obvious reasons (be it wholesale or overflow based -- carries on reading until a 00 is hit because that is how C works and edit the names of a character to not have a 00 and fun ensues) , network functionality, anything that takes external input (QR codes, audio sampling, level data) and usually for fun in more recent years serious abstraction with internal game logic (you control what is in your inventory, if a crash causes an execute to happen in inventory space then with a bit of creativity* on your inventory management you can do things there. Jump to it gets more fun, can involve inducing errors if they have return to given position, can involve more hardware fun**.
*the same thing that leads into return oriented programming/ROP, something that will rise back up with the privilege levels thing where execution is blocked from certain memory areas, where it is not necessarily every individual instruction you want but something that ultimately does the task then coming into play.
**hardware fun usually initially and today increasingly the order of the day as hardware protections are less known/developed than software methods, and hard to defend against even if you do know what goes (security is far less sexy than anything else really as it is apparently only needed when it has failed if you are funding things).
Written documents are few and far between really, and are more likely to be general embedded security or computer security in general (
https://www.schneier.com/books/ being the jumping off point for most in that world). Some of the game console forensics might have a tiny bit (though forensics is the bigger concern and most just use the same exploits that the people investigated likely used***), and some of the people doing PC exploits tend to have what consoles will have 5-10 years later (the Switch for instance gained ASLR, something available on Windows XP).
***Mr organised crime member is not going to pay someone to 0 day a console like they might recruit some student of radar jamming but they would be smart enough to realise police only collected laptops and phones when they kicked down the door, the little xbox which just so happens to be a full linux machine (or at least some nice encrypted comms on a game, though many games explicitly dodge this for just this reason) if you push the right button then got left untouched until someone said hold on a second.
To that end you would probably be better off with hacker conference talks. Most of the ones you will see (likely the wii and PS3 ones) will have been from the C3 conferences (usually titled console hacking [year], though I like the ultimate series of talks on various older consoles too)
https://xboxdevwiki.net/17_Mistakes_Microsoft_Made_in_the_Xbox_Security_System is before most recordings but a good one.
Sorry for the browser killer
Not of that line but also really good, though the "nothing interesting there" bit quite famously came back to be the basis for RGH series of hacks.
https://www.youtube.com/c/mediacccde/playlists
Hacker conferences in general will deal all the time in embedded security (game consoles being technically embedded devices) and other interesting things if you are heading down this path so pick any you like (defcon, black hat and c3 being the big three, derbycon, shmoocon, and dozens of others being held and having good stuff too). Don't think I have seen anything on security really at GDC and other game dev conferences other than a few anecdotes here and there/this is how we did it some 20 years later such as
Though there are some great talks at such things
https://www.gdcvault.com/play/1023470/contactUs and its sequel being good stuff if you are on a site like this.
Other videos I like that get you in the mindset needed
