As above this is a bit beyond your basic infinite health/ammo/gold type cheat, though I could see someone going in cold to this all and getting somewhere.
There are possibly three main approaches
1) Teleport
2) Disable collision
3) Super speed which is arguably a combination of both
1) is probably the easiest to make. Somewhere in the game, and findable by basic move, search for change, move, search for change...
It is also what might be happening when you do say emotes next to a wall or spin around next to one, or other things you might see in speedruns.
Anyway once you have the game's internal location (presumably X,Y,Z for a 3d again and maybe same again for direction, 2d if it is actual 2d and not 3d from a projection then maybe just X and Y) you make a cheat that adds a suitable amount (varies with game and what you are doing) to the value to effectively transport you a suitable distance to either phase through the wall, past a guard or similar. You probably will be seeing float values as noted above for this as well, though floats are not so bad when all is said and done (though potentially more tricky for cheat engines that don't always deal with them).
Moon jump if it is a game barrier of sorts might also be a viable option here -- either change the stats, change the gravity or if it is a game with double jump there will be a little flag somewhere noting whether a double jump/air jump has happened (or how many if it is a game where multiple are allowed). This can be reasonably easy to find, even more so if you have savestates, and if you are inclined to play with assembly then possibly even easier still (though that would require learning a bit of assembly. Short version whatever changes the OAM will also necessarily have come from the same code that handles the rest). Related is also swimming in air if there is a swimming mechanic -- chances are there will also be an in water flag, find this.
2) I will spare the level editing approach to this -- games based on 3d worlds, and some 2d ones too, will often have an entirely separate data section that mirrors the simple graphical that constitutes the level layout but also say track data or anything that is not included in simple models in a 3d world. Mario Kart KCL format is probably a good jumping off point for that
http://wiki.tockdom.com/w/index.php?title=KCL_(File_Format) . You could edit this or part thereof in RAM but few really do, though if it is a workaround for a soft lock then different matter entirely.
Anyway somewhere will be a if collision with level happens then stop player from moving type instruction. As the 3ds binary is in RAM you might be able to disable this with such a thing.
3) If collision then stop player from moving is normally only an interrupt level event rather than a constant check. Make the player move fast enough and as anybody that has ever fallen through the world on a game will tell you the walls are only tissue thin. Now Mario 64 as ancient as that now is would be noted as doing it 4 times a frame* so it might have to be something considerable, and again to get back through, but play it as you will.
*choice video