huh, I never tested it with only importing one set (I only tested by importing two)Think I found the problem, apparently if you only have one badge sets, the set will show up in your Badges, and the set will say it contains the right amount of badges, except it'll actually be empty once you switch to it.
I made myself a second set of badges and added it to a fresh BadgeData.dat along with my first set, and both sets show up properly, and they contain the right badges as they should.