Homebrew RELEASE Mario Maker level browser for tinfoil

[quicksilver501]

Hi all, some time ago I wanted to download courses from tinfoi.io, but kept downloading the same levels because there is no easy way to distinguish downloaded courses from new ones. So I made this little python gui to keep track of downloaded courses and batch download them, showing a preview of the level.

Just run "gui. py" and follow the instructions in the readme.


https://github.com/mercury501/ductile_mm_lv_browser/releases



Please keep in mind that I made this tool for myself primarily, then thought somebody else could use it. There may be bugs and shortcomings, maybe they'll be fixed one day, maybe they won't. Open an issue on github or, if you can, fork it.

 

[blawar]

While I appreciate the effort, you should not be scraping the site, let alone publishing a public scraper. I will have to kill those endpoints now.

 

[quicksilver501]

Well, if it bothers you I won't go publicly against your will. I'll take it down anyway.

 

[hippy dave]

While I appreciate the effort, you should not be scraping the site, let alone publishing a public scraper. I will have to kill those endpoints now.

So it's cool for you to build this database from other people's work without their consent, but not for others to access it?

 

[Mthodmn101]

So it's cool for you to build this database from other people's work without their consent, but not for others to access it?

dont people upload courses in mario maker so others will play them?

 

[hippy dave]

dont people upload courses in mario maker so others will play them?

Yes, not sure how that affects my point.
E: more specifically, I'm not complaining about him building his database from other people's work, I'm saying that leaves him in no position to be prissy about how people access his database.

 

[OrGoN3]

dont people upload courses in mario maker so others will play them?

The uploader isn't necessarily the IP owner. Seeing as you need to hack your console to download your course, I'd assume that <10% are from the content creator, if any.

 

[blawar]

Well, if it bothers you I won't go publicly against your will. I'll take it down anyway.

Just scrape Cloudflare. My main rub is that you scraped the main site which is already under high load. You also do not need to scrape the html, that just drives up load. there is an api endpoint for that info: https://tinfoil.media/Api/mml

replace all of the tinfoil.io references to tinfoil.media which is the caching server.

--------------------- MERGED ---------------------------

So it's cool for you to build this database from other people's work without their consent, but not for others to access it?

The data is public for all to access it. The problem is running a scraper on the main server (not the caching server).

edit: that website serves ~3 million requests per day, some of them scrapers which makes no sense since I publish most of the data on github. If users really want the mario marker levels, i can publish that data to github as well, someone just needs to ask

edit2: those zip files are not stored on the disk, they are created on the fly because that url was only meant to be hit by cloudflare to refresh cache. Scraping that endpoint alone drives up load significantly.

 

[quicksilver501]

Well,

Just scrape Cloudflare. My main rub is that you scraped the main site which is already under high load. You also do not need to scrape the html, that just drives up load. there is an api endpoint for that info: https://tinfoil.media/Api/mml

replace all of the tinfoil.io references to tinfoil.media which is the caching server.

--------------------- MERGED ---------------------------



The data is public for all to access it. The problem is running a scraper on the main server (not the caching server).

edit: that website serves ~3 million requests per day, some of them scrapers which makes no sense since I publish most of the data on github. If users really want the mario marker levels, i can publish that data to github as well, someone just needs to ask

Well, in my research I never came across this tinfoil.media caching server, or I'd have used that.

As for the level data, I'd be glad if you posted it on github as well, maybe I can change the program to use that data instead of scraping (if anybody cares).

I indeed asked for a way to store downloaded courses, but not for the raw data.

If there's demand for something like this i'd be glad to whip up another quick gui (maybe using tinfoil.media/Api/mml or the github), but if there's not I'll pass momentarily. I just wanted a feature, and implemented it myself.

So, to be clear, as a temporary solution I can reup the repo replacing tinfoil.io with .media?

 

[blawar]

Well,


Well, in my research I never came across this tinfoil.media caching server, or I'd have used that.

As for the level data, I'd be glad if you posted it on github as well, maybe I can change the program to use that data instead of scraping (if anybody cares).

I indeed asked for a way to store downloaded courses, but not for the raw data.

If there's demand for something like this i'd be glad to whip up another quick gui (maybe using tinfoil.media/Api/mml or the github), but if there's not I'll pass momentarily. I just wanted a feature, and implemented it myself.

So, to be clear, as a temporary solution I can reup the repo replacing tinfoil.io with .media?

yeah just replace all tinfoil.io references with tinfoil.media and it should be good enough for now, though the web pages wont be cached. Ideally you want to update it to not scrape any html, and use that mml endpoint I gave you above.

if you go to the website and hover over the asset links, you will see they all link to tinfoil.media not tinfoil.io

 

[Datahead]

If there's demand for something like this i'd be glad to whip up another quick gui (maybe using tinfoil.media/Api/mml or the github), but if there's not I'll pass momentarily. I just wanted a feature, and implemented it myself.

Sounds like a great feature to me!

 

[Sonikkua]

I just wish people would upload weekly top courses as a save file like they used to :-/

would love getting to my day off every week and grabbing a save with that weeks top levels in all categories. Was damned convenient

 

[quicksilver501]

Very well then, in a few hours I will try it with tinfoil.media,and in a few days I'll implement it using the api. Stay tuned!

 

[hippy dave]

Glad to see a solution reached

 

[quicksilver501]

Update:

I'm now working on using the api directly, and a solution to cache just the right amount of icons or displaying them directly from the web, thus removing the need to download ALL of them at the start.

 

[quicksilver501]

v0.1 Now it works using blawar's api link, downloads icons as needed, while browsing, and deletes them if you download/delete the level from list. Feedback appreciated! This is my first "Release" so be gentle.


(Don't know if I can "UP" the thread, but felt a pre-release warranted one. Sorry if not allowed!)