Up-to-date emunand would be nice, but literally the only thing I (personally) can't do without it so far is install the Binding of Isaac update, and from what I've heard that makes the game less stable anyway.
Same here for the Issac update/soon to be upcoming DLC. Strangely enough, i've only had the "music stopped" bug happen twice, and never any other issues that in all the hours i've played this. Beaten the game a few time, beat a couple challenges. No other bugs occurred.