the latest nightlies are diffrent from the old 2.XX builds, firmware is now stored in 3 diffrent.bins in /rxTools/firm and it stores its Bot file as code.bin in /rxTools/sys
i'd suggest atleast setting up ctrbootmanager as the boot.3dsx and to have rxTools a default boot option in it. (doing this would still let you access Homebrew w/ installing HBLauncher CIA and Hexediting the payload to load a renamed Homebrew launcher 3dsx for a streamlined experince (you can find a guide here to set that up
http://gbatemp.net/threads/tutorial-more-or-less-catch-all-tutorial.407080/ ) and to setup the payload for homebrew, grab from here
http://smealum.github.io/3ds/#prepare the 10.5.0-30 (your region) otherapp payload, and rename it "hblauncherloader_otherapp_payload_OLD-10-5-0-30-USA.bin" ,
Next grab
https://smealum.github.io/ninjhax2/boot.3dsx and rename it anything with 4 letters/numbers and save it to your SDCard's Root directory,
open the Otherapp payload you downloaded and renamed in anyhexeditor (i suggest HxD) and look for the string "boot.3dsx" and change that "boot" to what you named your Homebrew launcher's boot.3dsx
That will let you still access Homebrew from 10.5 Emunand while having a semi-coldboot rxTools and you can leave entries in ctrbootmanager for EmuNAND9Tool for dealing w/ backingup your SysNAND/EmuNAND/manipulating it etc. BUT to update your EmuNAND to 10.5 you can either use safesysupdateer or from System Settings but MAKE SURE when you open System settinsg it Shows RX-E 10.1 etc if you dont see RX-E in that version string, DO NOT Update! (that will indicate your in Sysnand!)