ROM Hack sky3ds diskwriter python clone (for linux & osx)

[NoSmokingBandit]

You don't have a mac to test this on, but I've come up with a slightly less hack-y way to get disk size on a mac. It will only work on a mac since it relies on diskutil, but thats easy enough to work around.

This is what I have. I can clean it up a little, I just wanted to make sure it worked.

import sys, subprocess
import plistlib
 
if sys.platform == 'darwin':
   
    list_command = "diskutil list -plist"       
   
    try:
        output = subprocess.check_output(list_command, shell=True, stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as err:
        sys.exit("I don't even know. Just. I don't know. %s" % err)
   
    list_plist = plistlib.readPlistFromString(output)
   
    disk_list =  list_plist["WholeDisks"]
   
   
    print disk_list
    chosen_disk = raw_input("Choose a disk: ")
   
    size_command = "diskutil info -plist %s" % chosen_disk
    try:
        output = subprocess.check_output(size_command, shell=True, stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as err:
        sys.exit("OH NO! The chosen disk could not be found. %s" % err )
   
    size_plist = plistlib.readPlistFromString(output)
   
    disk_size = size_plist["TotalSize"]
   
    print "%s's size is %s" % (chosen_disk, disk_size)
       
else:
    print "not a mac, doofus"
    #do whatever works for other platforms here

And this is the output for my 4gb usb drive:

$ python /Users/Me/Desktop/disklist.py
['disk0', 'disk1', 'disk2']
Choose a disk: disk2
disk2's size is 4004511744

and an 80gb hdd:

$ python /Users/Me/Desktop/disklist.py
['disk0', 'disk1', 'disk2']
Choose a disk: disk0
disk0's size is 79998918144

When I attach this to a gui (which I've worked on a little) I'll have the disk ID and size returned together to make a dropdown menu the use can pick from.

For the cli part I'd probably have it return a tuple like ('disk0', 79998918144) for every disk in the system, but its easy enough to tweak.

According to the docs, plistutil is in every python installation, regardless of platform, but you shouldtry it out. I can't be arsed to reboot into windows to see for myself.

 

[Cliemacfr]

still :

Exception: 0 byte disk?!?

Update : I tried NoSmokingBandIt's code.
It works on osx but it's python2, not 3. It succeded in giving me the size of the SDCard.

 

[lukas_2511]

[...]
When I attach this to a gui (which I've worked on a little) I'll have the disk ID and size returned together to make a dropdown menu the use can pick from.

For the cli part I'd probably have it return a tuple like ('disk0', 79998918144) for every disk in the system, but its easy enough to tweak.

According to the docs, plistutil is in every python installation, regardless of platform, but you shouldtry it out. I can't be arsed to reboot into windows to see for myself.

Yea thanks for the example code, I kinda wanted to go without using calls to other processes, but this would be a solution if I can't find any other way...

Is diskutil output considered stable over different osx versions? I don't want something that only works on YO!smite...

 

[NoSmokingBandit]

I don't have anything older than 10.9 so I can confirm it works on 10.10 and 10.9. I'm 99% sure it will work from 10.3 and up. 10.5 was the last major update for diskutil, but it should still have the necessary functions in 10.3 and 10.4 as well.

still :

Exception: 0 byte disk?!?

Update : I tried NoSmokingBandIt's code.
It works on osx but it's python2, not 3. It succeded in giving me the size of the SDCard.

People use python 3?

For osx distribution (and even for my own use) I'd run it through py2app and we wouldn't need to worry about anything.


Anyway, this needs polish and I just used dummy data to fill it in, but this is what I've put together thus far. I don't like the rom list table, but I can't find a better way to do it.

 

[Cliemacfr]

Well done NoSmokingBandit !!! looks good and promising !!
I'm on yosemite also. If you need help testing, tell me.

I used pytyhon3 because sky3ds.py uses python3 ! I think... but python2 is installed by default on osx.

 

[NoSmokingBandit]

Iirc osx comes with 2.6.something, but I've updated to 2.7.9. I'll change my stuff over to python 3. There's very little in my code that needs to be changed to work with python 3, so I'll just run it through 2to3 when I'm done.

This weekend I'll probably start combining the two halves. I might have to rework some of your code, Lukas, but I'll check in with you about that.

It looks like I'll be able to completely ignore the sky3ds.py file since all of the functional parts are in the included docs. That means we can keep cli capability and the gui without much effort.

 

[lukas_2511]

Iirc osx comes with 2.6.something, but I've updated to 2.7.9. I'll change my stuff over to python 3. There's very little in my code that needs to be changed to work with python 3, so I'll just run it through 2to3 when I'm done.

This weekend I'll probably start combining the two halves. I might have to rework some of your code, Lukas, but I'll check in with you about that.

It looks like I'll be able to completely ignore the sky3ds.py file since all of the functional parts are in the included docs. That means we can keep cli capability and the gui without much effort.

The class should be ready for implementing in a gui, I made it this way so it can have many interfaces, mostly because I'm really bad at writing user-interfaces

In other news: I added (experimental) python2 support. I played around with it a bit, everything seems to be working fine.

Edit: it will probably have unicode problems, like basically every python2 thing...

 

[NoSmokingBandit]

The only thing I saw that I might need to adjust is little things like the progress bar. I'm not familiar enough with the text progress bars to know 100%, but I might need to add a few lines to report progress back to the gui progress bar. Should be minor, and that is a very low priority, but I'll make sure you check everything out before I finalize any changes.

 

[lukas_2511]

The only thing I saw that I might need to adjust is little things like the progress bar. I'm not familiar enough with the text progress bars to know 100%, but I might need to add a few lines to report progress back to the gui progress bar. Should be minor, and that is a very low priority, but I'll make sure you check everything out before I finalize any changes.

Oh, yea, you are right, the progressbar thing. If you wait a few days I may do something about that on the weekend. Already have an idea for that one.

Edit: Want to split it into two functions, one that does the work and gets forked to the background, and another function that returns the status, so you can poll for the status.

 

[NoSmokingBandit]

The way progress bars work in tk are actually pretty easy. I can define the max value and update in steps just like you do with the text progress bar. I have a comment block in disk.py for when I get to it. Should be minimal changes.

"""
Saved for gui progress bar update.
something like....
gui.progress_bar.step(written)
Set it up like:
self.progress_bar = ttk.Progressbar(parent, maximum = rom_size, length = 150)
Make sure to check every method for a progress bar. Probs in add rom, backup rom only.
 
"""

Just my notes to myself.

I'd change this section:

        while written < rom_size:
            chunk = romfp.read(1024*1024)
 
            self.diskfp.write(chunk)
            os.fsync(self.diskfp)
 
            written = written + len(chunk)
            try:
                if not silent:
                    progress.update(written)
            except:
                pass
#start changes
            try:
                gui.progressbar.step(written)
            except:
                pass
#end changes
 
        try:
            if not silent:
                progress.finish()
        except:
            pass

I hate to add another try/except, but its the easiest way to maintain dual functions for that method.

I'm not super familiar with Python (I only started casually learning a few months ago), but is there a way to set up a default value if something isn't passed. I know in javascript I can do something like this:

function this_is_a_function(var1){
    this.var1 = var1 || 'default'
    }

Then if I just call this_is_a_function() without passing anything as var1 it automatically assigns var1 = 'default' (or whatever I want it to be).

If that can be done, maybe set up a var gui that defaults to False unless I pass gui = True when I call your methods via the gui. Then I'll put in a simple

if gui == True:
    do the stuff I need for the gui.

 

[lukas_2511]

I hate to add another try/except, but its the easiest way to maintain dual functions for that method.

That isn't even needed!

I added some code that allows to give write_rom and dump_rom an additional parameters with a ProgressBar-like object. So all you need to do is build a little wrapper-class around the gui progressbar, and give that to the function, and it will be used instead of the cli progressbar.

 

[NoSmokingBandit]

I haven't checked your git uploads in a while. I'll look into it. I'm still finishing up little details in the UI.

 

[lukas_2511]

I haven't checked your git uploads in a while. I'll look into it. I'm still finishing up little details in the UI.

Well, I've done that commit a few seconds before the reply

 

[NoSmokingBandit]

So I've just started with a new sd card to test every function while I'll make sure the gui is passing the correct info along and I'm already a bit stuck. It seems that I can't open my disk at all without root permission.

python sky3ds.py -d /dev/disk2 -l
Traceback (most recent call last):
  File "sky3ds.py", line 55, in <module>
    disk = disk.Sky3DS_Disk(args.disk)
  File "/Users/Me/Downloads/sky3ds.py/sky3ds/disk.py", line 34, in __init__
    raise Exception("Couldn't open disk, can't continue.")
Exception: Couldn't open disk, can't continue.

Running 'sudo python' makes it work. OSX doesn't seem to let you open anything in /dev.
Since there is no way to gain root via gui (you'd have to close then relaunch), I'm kind of in a pickle here.


edit:


....and GOD DAMNIT. I hit ctrl-v instead of ctrl-c to copy the ENTIRE gui script and accidentally overwrote it with something else entirely. No big deal, I'll just undo it, right? NOT TODAY MOTHERFUCKER. Brackets just stopped undoing things for no good reason. Now I have nothing.

If we can get the root permission thing sorted out I might rewrite everything, but right now I'm just pissed.

 

[lukas_2511]

....and GOD DAMNIT. I hit ctrl-v instead of ctrl-c to copy the ENTIRE gui script and accidentally overwrote it with something else entirely. No big deal, I'll just undo it, right? NOT TODAY MOTHERFUCKER. Brackets just stopped undoing things for no good reason. Now I have nothing.

If we can get the root permission thing sorted out I might rewrite everything, but right now I'm just pissed.

meh, happens to the best, but they use git and just restore the last commit

for the root stuff maybe take a look at unetbootin, they are doing something to get root privs, maybe you can copy that

 

[NoSmokingBandit]

I always put my working files in google drive or dropbox. Always. Except this time. I'm so mad at myself.

FYI, I actually just found this neat little bit of code:

import os
import sys
 
euid = os.geteuid()
if euid != 0:
    args = ['sudo', sys.executable] + sys.argv + [os.environ]
    os.execlpe('sudo', *args)

That make everything after it run as root.

I put that at the beginning of sky3ds.py, ran as my normal user and everything worked! It won't work with windows, but its easy enough to figure out if the platform is windows and just ignore it. This is my output with that added:

cd /Users/Me/Downloads/sky3ds.py
python sky3ds.py -d '/dev/disk2' -l
| Slot | Start | Size  | Type  | Code      | Title                  | Save Crypto  |
|    0 | 32 MB | 512 MB | Card1 | CTR-P-ABEE | New Super Mario Bros 2 |        <6.x |
 
Disk Size: 4032 MB | Free space: 3488 MB | Largest free continous space: 3488 MB

edit:
that prints lined up, the forum messes up the formatting.

 

[lukas_2511]

That make everything after it run as root.

I put that at the beginning of sky3ds.py, ran as my normal user and everything worked! It won't work with windows, but its easy enough to figure out if the platform is windows and just ignore it. This is my output with that added

That'll probably cause problems if you want to package it as a GUI (as sudo asks for a password on cli).
If you have to open it from Terminal anyway i would suggest replacing that with just a message "hey please run me with sudo!1" or something like that, instead of doing it automatically, i hate it when software does this by itself.

 

[NoSmokingBandit]

That code works for the gui as well, Its just harder to explain what is actually going on with a screenshot. Its a little wonky with how it relaunches the process, but I'll iron it out.

 

[lukas_2511]

I've added another feature to this tool.

It should now be able to import headers (Card ID + Unique ID) from .3dz files.

It checks for the ending (and also if there are any headers availabe to begin with), so if you don't want to import headers just rename it to .3ds.

Feature is currently untested, i'll test it soon.

 

[lukas_2511]

Added another (similar) feature: If you create a header.bin next to template.txt it reads the bytes from there. Structure is 0x40 bytes unique id followed by 0x4 bytes card id.