[Devs] Automated crash reports for vulnerability discovery?

Discussion in '3DS - Flashcards & Custom Firmwares' started by DrDaxxy, Jul 24, 2016.

  1. DrDaxxy
    OP

    DrDaxxy Member

    Newcomer
    13
    10
    Jan 24, 2015
    Gambia, The
    Dunno where to put this since it's more of a general console hacking idea than anything totally 3DS-specific, and I guess the right people would better be reached on IRC instead of here, but I don't feel like going there right now so here goes:

    Say we already have code execution on a normally locked-down platform, like the 3DS. And we're developing system software (like a CFW) that's destined to be used by the general public, not just privately shared among a few console hackers.

    Why not put in an opt-in feature to inject exception handlers everywhere we can and automatically send crash dumps to us? Not for catching bugs in homebrew - but among the countless CFW/homebrew/etc. end users, surely some of them have gotten random crashes due to browser or savegame corruption bugs that could be exploited to gain userland code execution. And hey, we might even hit the jackpot and have someone get a kernel panic, save a dump to disk, then submit it on next boot.

    Of course, on the 3DS it's not that big of a deal anymore since we have no shortage of very useful exploits, but I shouldn't have to tell you it's always good to have alternatives (or spares for when something we use right now gets patched).

    Possible problems I see with this: For one, it would require a bunch of effort. There's also the same problems you get with regular error reporting: generates lots of data/traffic, and that data may contain sensitive private information. That said, we should be able to keep traffic down at least a little by shipping an updated blacklist of already investigated bugs that we don't want to get more reports of, and we should of course let users know that what kind of data these reports may contain when asking for their permission to send them.

    So, can anyone tell me whether this is a terrible idea and why? ^_^
     
  2. NintenDavid

    NintenDavid GBAtemp Fan

    Member
    447
    56
    May 25, 2016
    United States
    Soobway
    Seems awesome, but it might take a while to develop something like that.
     
  3. pokemoner2500

    pokemoner2500 GBAtemp Advanced Fan

    Member
    846
    276
    Aug 14, 2013
    United States
    Doesn't LUMA (Dev version) already have something like this?
     
  4. NintenDavid

    NintenDavid GBAtemp Fan

    Member
    447
    56
    May 25, 2016
    United States
    Soobway
    I don't think so, but I never use dev version.
     
  5. shinyquagsire23

    shinyquagsire23 SALT/Sm4sh Leak Guy

    Member
    1,962
    3,231
    Nov 18, 2012
    United States
    Las Vegas
    This would probably just end with a bunch of people looking for crashes and finding a bunch of useless ones. Anyone capable of finding exploitable crashes probably also is able to know if it's exploitable, and is possibly able to actually exploit it.
     
    MarcusD likes this.
  6. DrDaxxy
    OP

    DrDaxxy Member

    Newcomer
    13
    10
    Jan 24, 2015
    Gambia, The
    Sort of. It can take crash dumps. It doesn't have functionality for automatically posting them on the Internet. The feature's intended for homebrew developers debugging their own code on their own hardware, not detecting bugs in Nintendo software or commercial games.
    Of course, that's already exactly the low-level functionality one would need to implement this; the rest is fairly easy.

    For what it's worth, my intention isn't to have users actively look for bugs, but to collect data from a massive amount of them (as with regular crash reporters included in popular software). Sure, this doesn't make examining crashes any easier, but it does show you where to look.

    As per Sturgeon's law, I'd expect 90% of (raw) reports to be garbage caused by either user error, unstable hax and two dozen boring issues in the browser and some popular games. Filtering them is a challenge. In addition to the aforementioned blacklist I'd suggest ratelimiting (if one person gets 50 crashes a day, it's probably nothing interesting), ignoring what can be identified as homebrew, and ignoring (or at least flagging) patched code. Happy to hear other ideas.

    That said: Let's assume we build this and a large enough number of people use it but not a single interesting crash is submitted. That could still be useful. A game with an abnormally high crash rate (in relation to global playtime - obviously you'll get more reports about Pokémon than Face Racers: Photo Finish) is probably rather poorly written and I'd suspect you'd have better than average chances of finding some silly buffer overflow in the savegame handling code.
     
  7. NintenDavid

    NintenDavid GBAtemp Fan

    Member
    447
    56
    May 25, 2016
    United States
    Soobway
    Now I think about this, it seems awesome, but who would we send it to?
     
  8. DrDaxxy
    OP

    DrDaxxy Member

    Newcomer
    13
    10
    Jan 24, 2015
    Gambia, The
    Someone would have to run a service (with HTTP API or whatever) for consoles to send data to. Could be me, could be a CFW author, could be someone else entirely. Or multiple people, sharing their data, so that if someone's service goes down (whether intentionally or because of technical issues) things keep working.

    ...I'm not sure why I ever suggested uploading full dumps, that's totally impractical. Keeping it to stack traces (and other similarly small data), this should be cheap enough to host too. Unless I seriously underestimate the amount of CFW users (who would turn this on). Or the bugginess of the average 3DS game/modded system ;)

    Also mostly solves the privacy concerns. It might even be okay to publicly post the data. Admittedly that would also allow Nintendo to access it; then again, they could just build error reporting into the firmware and get reports from most if not all consoles on their own.
     
    Last edited by DrDaxxy, Jul 24, 2016