Homebrew RELEASE Twili - Homebrew Debug Monitor

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
This one's been in the works for a bit of a while. Presenting Twili, my homebrew debug monitor for the Nintendo Switch:

Twili - Homebrew Debug Monitor (now with GDB stub)
[GitHub] [Latest Release]


What is it?: It's a debug monitor/development tool/nx-hbloader replacement.

What does it let me do?:
  • It lets you send and launch homebrew over USB (or TCP, I don't judge).
  • If the homebrew is compatible, it lets you have stdin/stdout/stderr over USB, too, through the command line tool. This allows for a very fast development cycle.
  • If you don't need your homebrew to be an applet (you're developing a background sysmodule), it can reduce iteration time even farther by preventing fatal errors when your sysmodule inevitably crashes.
  • You can use GDB to debug your homebrew (applets, sysmodules, etc.) or even official software.
  • When something crashes, you can generate an ELF core dump that you can load in a real man's debugger (aarch64-linux-gnu-elf) to help you figure out what went wrong instead of having to turn your console off, remove the SD card, find the crash log, and use addr2line to turn it into something meaningful.
  • Did I mention that you can core dump any process, and not just homebrew?
  • It lets you query overall RAM usage on your console.
  • It lets you get a list of all the processes running on your console, and...
  • It lets you kill them, too.
  • If low-level debugging is your style, it lets you debug them too. As of 1.2.0, there's even a GDB stub.
How do I use Twili's standard IO?: If you're using libtransistor, you get it for free. If you're using libnx, you can link against libtwili (found in my twili-libnx repo). If you launch your application through twib run, stdout/stderr will go to your console and stdin will come from that console. If you launch any other way, stdout/stderr will go to twibd logs and stdin will block.

How is it different from nx-hbloader/nxlink?:
Aside from having way more features, it runs each homebrew in its own process, which is a clean slate except for a small stub providing HBABI keys. This means that homebrew can manage its own memory and doesn't have to worry about cleaning up entirely behind itself for the next application.

Does it have a GDB stub?: As of 1.2.0, yes! See the README on the github repo for instructions.

What firmware versions does it work on?: To my knowledge, all of them. 1.0.0 might behave a bit differently regarding sysmodule crash fatal errors.

What CFWs does it work on?: The only CFW that I can confirm functionality on and actively support is Atmosphère, and only somewhat recent versions of it (0.10.2+). Twili has been tested on Atmosphère 0.10.2. Twili might work on other CFWs with some tweaks. Don't run Twili on piracy firmware because you'll make me very sad. :(

What should I know before I install it?: It replaces nx-hbloader. That means it replaces hbl.nsp. If you don't like Twili and want to remove it, don't forget to put back nx-hbloader's hbl.nsp.

How do I install it?: See the installation instructions on the GitHub repo.

Where do I download it?: Releases page on the GitHub repo.

What is the most recent version?: The most recent version is Twili 1.3.0. Also update Twib on each release, since bugfixes are included.

Where is the source code?: https://github.com/misson20000/twili

How is it licensed?: Twili is licensed under the GPLv3.

Where do I report bugs?: If you're not 100% convinced it's a bug, it's best to bring it up either here or on Discord first. You can find me in the ReSwitched discord. If you're convinced it's a bug, I keep track of those on GitHub.

Where do I get support?: Either here on on Discord. You can find me in the ReSwitched discord.
 
Last edited by misson20000,

Cyan

GBATemp's lurking knight
Former Staff
Joined
Oct 27, 2002
Messages
23,436
Trophies
2
Age
43
Location
Engine room, learning
XP
14,574
Country
France
thank you for this tool.
as you need 5 posts to get link rights, here is a direct link : https://github.com/misson20000/twili


can we get stderr/stdout over tcp too or it's usb only ?
I guess I need to install the zadig/libusbk driver. I didn't install that yet, that's why I'd prefer tcp for now but I'd understand if it's not a good idea as it can't log if network is not init in the homebrew? or it can log over tcp even before launching the homebrew to debug?

you said we need to replace hbloader.nsp, can't we have both ? can't we set that nsp to another titleID using the loader.ini file? I guess it would require an update to atmosphere to support more button combo or more titleID in the ini.

If I need a simple stdout logger, it's nxlink I should use? I don't know all the existing tools yet.
 
Last edited by Cyan,

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
can we get stderr/stdout over tcp too or it's usb only?
Everything you can do over USB you can do over TCP too, but in my experience, the USB bridge is more stable. When I say that the homebrew needs to be compatible, I mean that it needs to be written to send stdio over to Twili's IPC service, which hides the details of USB or TCP from the application. I've done this for libtransistor, and it probably wouldn't be too hard to do for libnx.

you said we need to replace hbloader.nsp, can't we have both ? can't we set that nsp to another titleID using the loader.ini file? I guess it would require an update to atmosphere to support more button combo or more titleID in the ini.
If you're not interested in launching applets through Twili, you can use it without replacing HBL.

If I need a simple stdout logger, it's nxlink I should use? I don't know all the existing tools yet.
If you're looking for a just simple stdout logger and don't mind that the application has to make its own network connection, that logging may not necessarily be entirely synchronous (log messages may get queued for a short while, which is not good if you're debugging crashes), or that your application is living in a process that has been/will be used by other homebrew applications, nxlink is probably a better tool for you. If you're looking for separate stdout/stderr streams and stdin support, you want your homebrew to be explicitly blocked until you're sure log messages have been delivered (useful for crash debugging), you want your own process (to run in the background), or your network is configured in such a way that makes nxlink impractical, Twili is the better tool.
 
Last edited by misson20000,
  • Like
Reactions: Cyan

ali.ee

Member
Newcomer
Joined
May 1, 2009
Messages
15
Trophies
0
XP
245
Country
Aside from the obvious usage of debugging, is this also the first step for a cheat engine running on switch itself?:unsure:
 

Reisyukaku

Onii-sama~
Developer
Joined
Feb 11, 2014
Messages
1,534
Trophies
0
Website
reisyukaku.org
XP
5,369
Country
United States
>What CFWs does it work on?:
>The only CFW that I acknowledge and support..
That doesnt even answer the question though... I'll add this to my cringe comp though.
f0Iu4Bh.png
 
Last edited by Reisyukaku,

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
Aside from the obvious usage of debugging, is this also the first step for a cheat engine running on switch itself?:unsure:
No, not really. This is a development tool intended to be used with the corresponding desktop tool (twib). Using debug SVCs to poke process's memory spaces isn't new (it's what everyone else does), and this particular usage of them is geared specifically towards remote debugging.
 
  • Like
Reactions: ali.ee

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
I've released Twili 1.0.1, including some fixes for recent versions of Atmosphère and an issue that would prevent games from launching on 5.0.0+.

Both the Twili sysmodule and twib/twibd have been updated. Old versions of twib/twibd should remain compatible, but I recommend you update them.

I've also published a library allowing libnx applications to use Twili's stdio. You can find it and its usage instructions on GitHub at misson20000/twili-libnx.

Additions
  • Added `PrintDebugInfo` command
  • AppletHost has more SVC permissions now
Changes
  • Twili is now distributed as exefs.nsp
  • We use the new npdm-json format now.
  • Twib should no longer be quite as `exit()` happy
  • USBBridge catches `std::bad_alloc` and returns `LIBTRANSISTOR_ERR_OUT_OF_MEMORY` responses instead of blowing up.
  • Use upstream switch-tools (don't need my elf2kip fork anymore)
Fixes
  • Managed processes now register service access control (rip sm:h on new versions of Atmosphere)
  • AppletTracker now clears created process queue when control applet is lost
  • Use applet memory instead of application memory so games can launch on 5.0.0+ with Twili running in background.
Notable Known Issues
  • #55: Crash with 2000-000 while sitting on home menu: This crash doesn't seem to happen when twibd is running and is connected.
  • #56: LibusbK (Windows) driver association is bad: Will be addressed in the next release.
 

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
Twili 1.2.0 is out, notably including a GDB stub (finally). Usage instructions can still be found in the README.

Changelog since 1.0.1
  • Added gdb stub (linux and osx only for now)
  • Added push and pull commands
  • Added launch command for pm:shell#LaunchProcess (won't launch games, but will launch sysmodules)
  • Added libusbK-based contrib driver package
  • Added buffering for twib pipes (speeds up throughput on small writes from ~315 KiB/s to ~2.35 MiB/s)
  • Made twib pipes faster (max speed I've been able to measure is ~20 MiB/s throughput)
  • Changed process launch to store executable on SD card instead of in memory (allows larger applications and fixes romfs for libnx)
  • Added configuration file
  • Fixed NextLoadPath for non-hbmenu (like RetroArch)
  • Fixed for Stratosphere compiled with newer libnx
  • Fixed thread handles for libnx
  • Fixed IAppletController
  • Fixed applet launch when first try fails
  • Fixed broken device ID calculation for TCP backend
  • Fixed 2000-0000 fatals
  • Fixed race condition in initial device probing (twib list-devices could report before scan was complete)
  • Refactored request handling
  • Refactored twib event loops and platform code
  • Reduced default buffer size (drastically improves memory usage: ~29 MiB -> 9 MiB)
  • Made twibd exit cleanly on SIGINT
  • Made twibd stop complaining about thread termination
  • Various improvements for Windows
 

Blackspoon

Active Member
Newcomer
Joined
Feb 19, 2016
Messages
31
Trophies
0
Age
29
Location
next to the blackfork
XP
132
Country
Gambia, The
I cant get it work on 8.1.0 Atmos 0.9.3.

When I try to load hbl.nsp the console will do nothing.

twilibd will not recognize the console.

When I switch the usb_bridge and the usb log off it will boot into hbmenu and let me connect over tcp. But twilib still says that no device detected altough connect-tcp returned "OK".

Does twili support atmos 0.9.3 or have I go to 0.8.3 instead?
 

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
When I switch the usb_bridge and the usb log off it will boot into hbmenu and let me connect over tcp. But twilib still says that no device detected altough connect-tcp returned "OK".

Does twili support atmos 0.9.3 or have I go to 0.8.3 instead?

It's supposed to work on latest Atmosphere version.

I noticed it doesn't support breakpoints yet. Is that a feature you're looking to add if possible?

Software breakpoints are supported- overwriting the target instruction with a break instruction and putting it back when we need to continue over it. Watchpoints are not supported. Comex has done some work on hardware breakpoints and watchpoints.
 

wiitendo84

Well-Known Member
Member
Joined
Jan 3, 2014
Messages
1,322
Trophies
1
Age
37
Location
New Albany, Indiana
XP
3,072
Country
United States
It's supposed to work on latest Atmosphere version.



Software breakpoints are supported- overwriting the target instruction with a break instruction and putting it back when we need to continue over it. Watchpoints are not supported. Comex has done some work on hardware breakpoints and watchpoints.
Essentially the type I was wondering about would be setting a breakpoint on a address in a game to see what's writing to it so you could essentially nop it or stop it from subtracting or adding to it.
 

skiddd

New Member
Newbie
Joined
Oct 26, 2019
Messages
3
Trophies
0
Age
41
XP
51
Country
United States
I am trying to set things up on a 9.1.0 switch with Atmosphere 0.10.1.

I am on Windows 7 64-bit and have successfully installed the USB driver.
However, in my device manager I could only see one "TransistorUSB" device.
Nothing else is in Other Devices or devices without drivers installed.

I followed the Switch side installation and replaced hbl.nsp. Now when I try to
enter the Album section of the Switch it just shows me a blank gray page with
"Continue A" and nothing else.

Ran twibd_win64.exe and allowed it in the firewall and then tried twib_win64 list-devices.
The Switch was not detected :(

Any hints on how to get this working on a Windows machine?

B.R.
 

misson20000

Member
OP
Newcomer
Joined
Nov 7, 2018
Messages
7
Trophies
0
Location
Near Seattle
Website
xenotoad.net
XP
187
Country
United States
Twili 1.3.0 is out. See GitHub for detailed changelog and release notes. This release fixes a lot of bugs and improves quality of life.

Any hints on how to get this working on a Windows machine?

Sorry, the Windows situation is quite a struggle right now. Ping me in the ReSwitched discord sometime and we can see about getting it working?
 
General chit-chat
Help Users
    Psionic Roshambo @ Psionic Roshambo: Grand Theft Auto V on mobil looks awesome