Homebrew RELEASE Mario Maker level browser for tinfoil

quicksilver501

Member
OP
Newcomer
Joined
Jul 1, 2019
Messages
13
Trophies
0
Age
30
XP
164
Country
Italy
Hi all, some time ago I wanted to download courses from tinfoi.io, but kept downloading the same levels because there is no easy way to distinguish downloaded courses from new ones. So I made this little python gui to keep track of downloaded courses and batch download them, showing a preview of the level.

Just run "gui. py" and follow the instructions in the readme.


https://github.com/mercury501/ductile_mm_lv_browser/releases



Please keep in mind that I made this tool for myself primarily, then thought somebody else could use it. There may be bugs and shortcomings, maybe they'll be fixed one day, maybe they won't. Open an issue on github or, if you can, fork it.
 
Last edited by quicksilver501,
  • Like
Reactions: YueYongMeng

blawar

Developer
Developer
Joined
Nov 21, 2016
Messages
1,709
Trophies
1
Age
39
XP
4,293
Country
United States
While I appreciate the effort, you should not be scraping the site, let alone publishing a public scraper. I will have to kill those endpoints now.
 

hippy dave

BBMB
Member
Joined
Apr 30, 2012
Messages
8,790
Trophies
2
XP
17,272
Country
United Kingdom
dont people upload courses in mario maker so others will play them?
Yes, not sure how that affects my point.
E: more specifically, I'm not complaining about him building his database from other people's work, I'm saying that leaves him in no position to be prissy about how people access his database.
 
Last edited by hippy dave,

blawar

Developer
Developer
Joined
Nov 21, 2016
Messages
1,709
Trophies
1
Age
39
XP
4,293
Country
United States
Well, if it bothers you I won't go publicly against your will. I'll take it down anyway.

Just scrape Cloudflare. My main rub is that you scraped the main site which is already under high load. You also do not need to scrape the html, that just drives up load. there is an api endpoint for that info: https://tinfoil.media/Api/mml

replace all of the tinfoil.io references to tinfoil.media which is the caching server.

--------------------- MERGED ---------------------------

So it's cool for you to build this database from other people's work without their consent, but not for others to access it?

The data is public for all to access it. The problem is running a scraper on the main server (not the caching server).

edit: that website serves ~3 million requests per day, some of them scrapers which makes no sense since I publish most of the data on github. If users really want the mario marker levels, i can publish that data to github as well, someone just needs to ask :)

edit2: those zip files are not stored on the disk, they are created on the fly because that url was only meant to be hit by cloudflare to refresh cache. Scraping that endpoint alone drives up load significantly.
 
Last edited by blawar,

quicksilver501

Member
OP
Newcomer
Joined
Jul 1, 2019
Messages
13
Trophies
0
Age
30
XP
164
Country
Italy
Well,
Just scrape Cloudflare. My main rub is that you scraped the main site which is already under high load. You also do not need to scrape the html, that just drives up load. there is an api endpoint for that info: https://tinfoil.media/Api/mml

replace all of the tinfoil.io references to tinfoil.media which is the caching server.

--------------------- MERGED ---------------------------



The data is public for all to access it. The problem is running a scraper on the main server (not the caching server).

edit: that website serves ~3 million requests per day, some of them scrapers which makes no sense since I publish most of the data on github. If users really want the mario marker levels, i can publish that data to github as well, someone just needs to ask :)

Well, in my research I never came across this tinfoil.media caching server, or I'd have used that.

As for the level data, I'd be glad if you posted it on github as well, maybe I can change the program to use that data instead of scraping (if anybody cares).

I indeed asked for a way to store downloaded courses, but not for the raw data.

If there's demand for something like this i'd be glad to whip up another quick gui (maybe using tinfoil.media/Api/mml or the github), but if there's not I'll pass momentarily. I just wanted a feature, and implemented it myself.

So, to be clear, as a temporary solution I can reup the repo replacing tinfoil.io with .media?
 
Last edited by quicksilver501,

blawar

Developer
Developer
Joined
Nov 21, 2016
Messages
1,709
Trophies
1
Age
39
XP
4,293
Country
United States
Well,


Well, in my research I never came across this tinfoil.media caching server, or I'd have used that.

As for the level data, I'd be glad if you posted it on github as well, maybe I can change the program to use that data instead of scraping (if anybody cares).

I indeed asked for a way to store downloaded courses, but not for the raw data.

If there's demand for something like this i'd be glad to whip up another quick gui (maybe using tinfoil.media/Api/mml or the github), but if there's not I'll pass momentarily. I just wanted a feature, and implemented it myself.

So, to be clear, as a temporary solution I can reup the repo replacing tinfoil.io with .media?

yeah just replace all tinfoil.io references with tinfoil.media and it should be good enough for now, though the web pages wont be cached. Ideally you want to update it to not scrape any html, and use that mml endpoint I gave you above.

if you go to the website and hover over the asset links, you will see they all link to tinfoil.media not tinfoil.io
 

Sonikkua

Well-Known Member
Member
Joined
Mar 26, 2011
Messages
116
Trophies
1
XP
723
Country
United States
I just wish people would upload weekly top courses as a save file like they used to :-/

would love getting to my day off every week and grabbing a save with that weeks top levels in all categories. Was damned convenient
 

quicksilver501

Member
OP
Newcomer
Joined
Jul 1, 2019
Messages
13
Trophies
0
Age
30
XP
164
Country
Italy
Update:

I'm now working on using the api directly, and a solution to cache just the right amount of icons or displaying them directly from the web, thus removing the need to download ALL of them at the start.
 

quicksilver501

Member
OP
Newcomer
Joined
Jul 1, 2019
Messages
13
Trophies
0
Age
30
XP
164
Country
Italy
v0.1 Now it works using blawar's api link, downloads icons as needed, while browsing, and deletes them if you download/delete the level from list. Feedback appreciated! This is my first "Release" so be gentle.


(Don't know if I can "UP" the thread, but felt a pre-release warranted one. Sorry if not allowed!)
 

You may also like...

General chit-chat
Help Users
  • No one is chatting at the moment.
  • captainbob321 @ captainbob321:
    Hello!
  • captainbob321 @ captainbob321:
    What's so Funny, @M4x1mumReZ
    ?
    +1
  • M4x1mumReZ @ M4x1mumReZ:
    Welcome new user
  • trepp0 @ trepp0:
    Just got the notice that school is being released 2 hours early cause of the winter storm
  • trepp0 @ trepp0:
    lets go
  • Psionic Roshambo @ Psionic Roshambo:
    Better than 3 hours late from a lockdown lol
    +1
  • FAST6191 @ FAST6191:
    While I know you mean in case of someone confusing schooling establishment from shooting establishment (many of the same letters) I am still going to read that as lockdown (also known as lock in) for a pub
  • FAST6191 @ FAST6191:
    where if you were in the club you could be invited to be there as a guest of the landlords after things are supposed to stop being served
  • FAST6191 @ FAST6191:
    Have technically done that in a school but it was the sports centre associated with the school more than the school itself
  • FAST6191 @ FAST6191:
    being drunk in school was either because drunk on playing field or could not be arsed with one particular Thursday afternoon so went and had some beers and played some games instead before returning for an ill advised last lesson/period
  • K3N1 @ K3N1:
    Being drunk and high in school was cool
  • K3N1 @ K3N1:
    It was high school
  • The Real Jdbye @ The Real Jdbye:
    my power cut out twice
  • The Real Jdbye @ The Real Jdbye:
    my pi running octoprint wiped all my plugins and i cba to fix it
  • Psionic Roshambo @ Psionic Roshambo:
    No pi for you lol
  • Sonic Angel Knight @ Sonic Angel Knight:
    Psi, you didn't spell your name right
  • Sonic Angel Knight @ Sonic Angel Knight:
    It's "Psionic" not "PI"
  • Psionic Roshambo @ Psionic Roshambo:
    So R Kelly is probably pissing himself right now
  • K3N1 @ K3N1:
    R Kelly knows how to piss on himself?
    Psionic Roshambo @ Psionic Roshambo: Lol