Hacking Statistics regarding compatibility

[cenotaph]

Right, yesterday night I wrote a (somewhat ugly) perl hack to mine data from the compatibility lists over at wiki.scorpei.com in order to calculate statistics as to the compatibility of the various flash kits which have compatibility lists over there. I've been somewhat curious as to the average compatibility of them, and I figured someone else might be interested in this too, so I decided I'd share my findings. These numbers are obviously flawed though as they're based on user reports, but they give a general idea.

- DS-Xtreme
Total amount of roms as reported by the compatibility list: 813
Amount of roms that
* run perfectly: 504
* run with flaws: 47 (0003, 0013, 0015, 0017, 0023, 0060, 0062, 0064, 0073, 0076, 0088, 0099, 0102, 0140, 0142, 0146, 0149, 0238, 0287, 0320, 0333, 0361, 0395, 0406, 0457, 0459, 0491, 0506, 0530, 0580, 0591, 0609, 0622, 0630, 0633, 0640, 0644, 0651, 0667, 0676, 0699, 0705, 0717, 0735, 0754, 0773, 0803).
* do not run: 4 (0193, 0223, 0390, 0799).
* are untested: 258
Of the 555 roms that have been tested, 551 were found to be compatible and 4 were found non-compatible.
That's an average compatibility of 99.28%.

- NinjaPass X9
Total amount of roms as reported by the compatibility list: 813
Amount of roms that
* run perfectly: 743
* run with flaws: 42 (0013, 0015, 0017, 0022, 0025, 0029, 0037, 0056, 0063, 0073, 0108, 0115, 0118, 0120, 0147, 0160, 0168, 0195, 0201, 0205, 0210, 0228, 0284, 0285, 0320, 0333, 0368, 0371, 0434, 0442, 0459, 0472, 0479, 0491, 0580, 0595, 0626, 0676, 0696, 0707, 0767, 0772).
* do not run: 24 (0003, 0273, 0306, 0367, 0379, 0403, 0431, 0437, 0455, 0458, 0478, 0485, 0497, 0517, 0521, 0545, 0551, 0565, 0570, 0576, 0577, 0673, 0700, 0735).
* are untested: 4
Of the 809 roms that have been tested, 785 were found to be compatible and 24 were found non-compatible.
That's an average compatibility of 97.03%.

- Supercard DS
Total amount of roms as reported by the compatibility list: 810
Amount of roms that
* run perfectly: 754
* run with flaws: 52 (0023, 0038, 0053, 0081, 0113, 0134, 0146, 0191, 0231, 0235, 0247, 0271, 0274, 0278, 0281, 0299, 0316, 0317, 0318, 0351, 0357, 0364, 0369, 0372, 0380, 0390, 0397, 0401, 0405, 0445, 0460, 0509, 0522, 0572, 0599, 0610, 0611, 0612, 0627, 0628, 0630, 0640, 0649, 0656, 0664, 0668, 0698, 0704, 0709, 0720, 0766, 0768).
* do not run: 0
* are untested: 4
Of the 806 roms that have been tested, 806 were found to be compatible and 0 were found non-compatible.
That's an average compatibility of 100.00%.

- R4
Total amount of roms as reported by the compatibility list: 813
Amount of roms that
* run perfectly: 195
* run with flaws: 6 (0022, 0201, 0223, 0333, 0636, 0735).
* do not run: 0
* are untested: 612
Of the 201 roms that have been tested, 201 were found to be compatible and 0 were found non-compatible.
That's an average compatibility of 100.00%.

- M3 Simply:
Total amount of roms as reported by the compatibility list: 788
Amount of roms that
* run perfectly: 68
* run with flaws: 1 (0735).
* do not run: 1
* are untested: 718
Of the 70 roms that have been tested, 69 was found to be compatible and 1 were found non-compatible.
That's an average compatibility of 98.57%.

- EZ-Flash V
Total amount of roms as reported by the compatibility list: 809
Amount of roms that
* run perfectly: 81
* run with flaws: 0
* do not run: 0
* are untested: 728
Of the 81 roms that have been tested, 81 were found to be compatible and 0 were found non-compatible.
That's an average compatibility of 100.00%.

More might be added whenever I feel like making the script able to mine other compatibility lists.

 

[Devante]

This is very nice.

Thanks for the work.

 

[ambitous21]

how about let the percentage be

total minus doesnt run and flaws
---------------------------------------
total


that'll let us distinguish which one is the real top cart

 

[Destructobot]

It would be more useful to show both. I'm thinking something like this:

XX.X% of games are compatible.
XX.X% of games run perfectly.


It will still be wildly innacurate as long as carts with 800+ games tested are being compared to carts with 80 games tested.

 

[Katalyst]

This would really be useful if it was uploaded to a server and the script ran in real time. Mining the data from the compat. lists every so often and updating itself automatically.

 

[krdvmus]

hmm?

What's the one that didn't run on M3 Simply?

Unlike DS-X, others have micro SD card compatibility issue, which affect the game compatibility.

I mean, depending on micro SD cards, your result could be different especially "run with flaws" part.

 

[Destructobot]

A bad dump of the Rockman Leo game. The proper dump has been confirmed as working.

 

[cenotaph]

This would really be useful if it was uploaded to a server and the script ran in real time. Mining the data from the compat. lists every so often and updating itself automatically.

I was thinking I'd do this once I've cleaned up the code and added support for mining data from various other compatibility lists. Not sure when it'll happen though.