Misc Fast TitleDB Lookup Tool

Slluxx

GBATemp Mayor
OP
Developer
Joined
Jul 17, 2019
Messages
344
Trophies
0
XP
1,050
Country
Germany
@masagrator

I changed the text in the advanced filter to "Show entries with NULL value".
Ive also modified the searching. I personally think its too loose because it finds a lot of other junk too but please test for yourself.

Every string in the input filed is split by whitespace into an array, a regex is made out of each item and then all games are looped through and tested against each regex. This means searching just for "doom" will give all games with doom in their name, searching for "doom eternal" will give all games with either the word "doom" or "eternal" in its name (also if the search just a portion of the name) and thus results in much more found entries instead of "narrowing it down".

JavaScript:
var inputList = this.search.value.toLowerCase().trim().split(' ');
var regexList = [];
inputList.forEach((el) => {
  regexList.push(new RegExp(el));
});

var matchingGames = [];
for (var game in this.allItemsFromJson) {
  var filterVal = this.allItemsFromJson[game][filter];
  var isMatch = regexList.some((rx) =>
    rx.test(filterVal.toString().toLowerCase())
  );
  if (isMatch) matchingGames.push(this.allItemsFromJson[game]);
}

Not sure if someone here knows some javascript too and can come up with an even better way of searching titles.
I could just loop through the regex array and test each game against the first element, save the matches and testit against the second element and so on but nested for loops should be avoided and it would mean it would loop through all games * the number of words in your search. not a great solution.

The easiest solution would be to go back to the old way but exclude copyright/trademark/... characters from the regex. to do that, i would need to create and maintain a list of characters to ignore. Doing that for every language is not smart unless some genius has implemented something like that into regex
 
Last edited by Slluxx,

masagrator

The patches guy
Member
Joined
Oct 14, 2018
Messages
4,987
Trophies
2
XP
8,694
Country
Poland
I could just loop through the regex array and test each game against the first element, save the matches and testit against the second element and so on but nested for loops should be avoided and it would mean it would loop through all games * the number of words in your search. not a great solution
Just avoid creating new array outside of results.
regex allows for searching multiple words without any order at the same time. This won't work if word is a part of bigger alphanumerical word. "doom" won't be found in "dooms"
IMG_20220305_183348.jpg

This will return only 1 result if all words matches.

Or if it's faster, just check in loop if all words exist in string one by one without regex, if one fails return nothing. No new array needed.
 
Last edited by masagrator,

Slluxx

GBATemp Mayor
OP
Developer
Joined
Jul 17, 2019
Messages
344
Trophies
0
XP
1,050
Country
Germany
Just avoid creating new array outside of results.
regex allows for searching multiple words without any order at the same time. This won't work if word is a part of bigger alphanumerical word. "doom" won't be found in "dooms"
View attachment 300507

This will return only 1 result if all words matches.

Or if it's faster, just check in loop if all words exist in string one by one without regex, if one fails return nothing. No new array needed.

Maybe i am just blind but how did you get that to match?
Unbenannt.PNG


It doesnt match a single thing for me. I understand how this regex is supposed to work, it just doesnt yield the result.

I am planning something with that "new array" though. i was aware its technically not needed, i just didnt know how to exclude the trademark/copyright characters from it. If i get this regex to work, ill add a pretty neat new feature to the site that you didnt thought you needed till you see it :)
 

masagrator

The patches guy
Member
Joined
Oct 14, 2018
Messages
4,987
Trophies
2
XP
8,694
Country
Poland
It doesnt match a single thing for me. I understand how this regex is supposed to work, it just doesnt yield the result.
You forgot about ^ at the beginning.
And don't use multiple lines. regex then needs different approach and for no reason since no title has more than 1 line

Edit: Checked this on the same site you're testing and it seems using javascript browser side it returns 0 results, but changing it to PCRE server side return results. :V
 
Last edited by masagrator,

Slluxx

GBATemp Mayor
OP
Developer
Joined
Jul 17, 2019
Messages
344
Trophies
0
XP
1,050
Country
Germany
You forgot about ^ at the beginning.
And don't use multiple lines. regex then needs different approach and for no reason since no title has more than 1 line
i tested with and without it. I can use multiple lines with this tool. each line gets treated as its own to search through. its simply a help to try multiple matches at once so you dont always have to put text in, check if it works, remove and put something else in.

As you can see, it still doesnt match
Unbenannt.PNG
 

masagrator

The patches guy
Member
Joined
Oct 14, 2018
Messages
4,987
Trophies
2
XP
8,694
Country
Poland
Issue with this is that it doesn't work correctly with japanese titles that tend to put subtitle in the same "word" as last original title word.
For example
英雄伝説 閃の軌跡
英雄伝説 閃の軌跡II
英雄伝説 閃の軌跡IV -THE END OF SAGA-


writing 英雄伝説 閃の軌跡 will work only with first title.

By removing \b you can fix this issue.
So yeah, you must allow that. :P

Code:
^(?=.*(doom))(?=.*?(eternal)).*
 
  • Like
Reactions: Slluxx

mspy

Well-Known Member
Member
Joined
Jul 29, 2018
Messages
298
Trophies
0
XP
1,444
Country
Brazil
@Slluxx

I don't know if you notice but TitleDB Lookup isn't reporting updates for DLC which like regular game updates can also have different versions.
Maybe it has something to do with this issue https://github.com/blawar/titledb/issues/16

Also I believe there's an easy way to grab or generate an universal small file containing the data from all the different languages files put together instead of having us select which language file we wish to search at.
 

masagrator

The patches guy
Member
Joined
Oct 14, 2018
Messages
4,987
Trophies
2
XP
8,694
Country
Poland
@Slluxx

I don't know if you notice but TitleDB Lookup isn't reporting updates for DLC which like regular game updates can also have different versions.
Maybe it has something to do with this issue https://github.com/blawar/titledb/issues/16

Also I believe there's an easy way to grab or generate an universal small file containing the data from all the different languages files put together instead of having us select which language file we wish to search at.
There isn't because titledb repo doesn't have one. Site is downloading data user-side, nothing is stored on server.
You would need to use a server or another github repo that would be responsible for making such a file with every new update in titledb repo.
 
  • Like
Reactions: Slluxx

mspy

Well-Known Member
Member
Joined
Jul 29, 2018
Messages
298
Trophies
0
XP
1,444
Country
Brazil
There isn't because titledb repo doesn't have one. Site is downloading data user-side, nothing is stored on server.
You would need to use a server or another github repo that would be responsible for making such a file with every new update in titledb repo.
So how does https://github.com/giwty/switch-library-manager manages to just grab a 68MB file containing all the data that TitleDB Lookup appears to show, images and everything, I dunno man...
 

Slluxx

GBATemp Mayor
OP
Developer
Joined
Jul 17, 2019
Messages
344
Trophies
0
XP
1,050
Country
Germany
So how does https://github.com/giwty/switch-library-manager manages to just grab a 68MB file containing all the data that TitleDB Lookup appears to show, images and everything, I dunno man...

Because SLM grabs temporary files from https://tinfoil.media/repo/db instead of the github repository.

You can see it here: https://github.com/giwty/switch-library-manager/blob/master/settings/settings.go#L18
And blawars comment about this here: https://github.com/blawar/titledb/issues/16#issuecomment-1126731937

Also I believe there's an easy way to grab or generate an universal small file containing the data from all the different languages files put together instead of having us select which language file we wish to search at.

Also not going to happen because that would mean you need to download about a gigabyte of data before doing anything.
Even the best gaming pc around will not be able to handle that much json efficiently. It would most likely just crash the tab/browser/pc before you can even search for something. It is absolutly impossible that SLM downloads a ~70 mb file with all data. go checkout the github repository and look at the language json files. You will see that they are around 150 - 200 mb each (iirc) so the data that SLM grabs must miss a lot of information

TLDR; there is no such thing as a complete "universal small file".
 
Last edited by Slluxx,

mspy

Well-Known Member
Member
Joined
Jul 29, 2018
Messages
298
Trophies
0
XP
1,444
Country
Brazil
Because SLM grabs temporary files from https://tinfoil.media/repo/db instead of the github repository.

You can see it here: https://github.com/giwty/switch-library-manager/blob/master/settings/settings.go#L18
And blawars comment about this here: https://github.com/blawar/titledb/issues/16#issuecomment-1126731937



Also not going to happen because that would mean you need to download about a gigabyte of data before doing anything.
Even the best gaming pc around will not be able to handle that much json efficiently. It would most likely just crash the tab/browser/pc before you can even search for something. It is absolutly impossible that SLM downloads a ~70 mb file with all data. go checkout the github repository and look at the language json files. You will see that they are around 150 - 200 mb each (iirc) so the data that SLM grabs must miss a lot of information

TLDR; there is no such thing as a complete "universal small file".
I may be wrong on this but I think the 'different' language files on github contain many duplicate information between each other hence the file SLM uses would be all you need, I mean that I never notice any missing information from it.
I wonder if that temp file that SLM grabs is hard to generate.

Anyway, TitleDB Lookup doesn't seem to be reporting DLC updates, a problem that is discussed on the github issue you linked.
 

Slluxx

GBATemp Mayor
OP
Developer
Joined
Jul 17, 2019
Messages
344
Trophies
0
XP
1,050
Country
Germany
I may be wrong on this but I think the 'different' language files on github contain many duplicate information between each other hence the file SLM uses would be all you need, I mean that I never notice any missing information from it.
I wonder if that temp file that SLM grabs is hard to generate.

Anyway, TitleDB Lookup doesn't seem to be reporting DLC updates, a problem that is discussed on the github issue you linked.

different languages can (and do) contain different data. Yeah, most of the data is duplicate but i cant just ignore the data that is in fact different. Tbh im not sure why its such an issue to klick on the language dropdown and choose one.

Its not on my todo list to fix updates for dlcs, unless its requested so much that i cant ignore it.
The switch (hacking) scene is kind of dead and i have other projects that need my attention
 
General chit-chat
Help Users
  • No one is chatting at the moment.
    NeoGaming @ NeoGaming: i'm back