If you can use the browser, I suggest that you use the new browserhax (up to 11.0.0-33!) in the meantime.

Try out Ubuntu 15.04: http://old-releases.ubuntu.com/releases/15.04/

This requires a New 3DS. The demo version doesn't work and is patched. If you have 1.1.3 or newer delete/disable the update data.
officially called smashbroshax sometimes smashax
Here is a new tool I call smashbroshax-helper. It is a graphical interface for the exploit which simplifies most of the process of broadcasting the packet. It requires almost no setup outside of creating a bootable Linux USB/DVD.

Important notes:

• This does not work on Fedora or Red Hat-based distributions because aircrack-ng needs to be compiled on it (and I can't figure out how to install the needed things).
  
• Don't use a virtual machine!
• This probably will never work on Old 3DS.

Download smashbroshax-helper beta

It is recommended that you use a live Ubuntu 15.04 image. Versions 15.10+ have issues. http://old-releases.ubuntu.com/releases/15.04/

1. Create a bootable USB/DVD with a Linux distribution (there are various guides online).
2. If possible, get a second USB/SD card/storage device, download and save the above .zip to it.
   • Don't extract the contents of it to the USB device, as it might cause problems. Just save the .zip file to it.
   • If you can't do this, you'll have to connect to the Internet while in Linux to download it.
   • If using a bootable USB, make sure you can use two ports at a time. Don't take out the Linux USB while it is being used!
3. Restart your computer and run the bootable USB/DVD you created.
4. Extract the contents of the .zip to the Desktop.
5. Open the smashbroshax-helper folder and double click "smashbroshax.sh".
6. Follow the on-screen prompts.
7. If everything goes well, you should now have homebrew!

Video demonstration, from boot to shutdown:

from https://gbatemp.net/threads/tutorial-using-smashhax-with-linux.397194/page-9#post-5842512

---

If you would like the full tutorial, involving terminal usage and compiling:

Spoiler: Everything

To reverse the changes to your wireless interface and remove issues connecting to networks after, rebooting your system or changing it to "Managed" instead of "monitor" should fix it. How to do that is in the second to last section.

---

@Cydget made a script that condenses most of this into a script. I have not tried it myself yet but it seems to work for others.

Spoiler

So, I made a little script for this. If anybody wants it, then unzip this file and read the readme. And yes, I like to pipe things. http://www.mediafire.com/download/oulnubnzkk9g3i0/smashhaxEZ.zip

---

Requirements

• Any Linux distribution should do (this has only been tested with Debian-based distributions). Windows and OS X users should wait or find a method for now, sorry!
  • Please do not use Linux in a virtual machine, it likely won't give direct access to your wireless card. Dual boot or use a live USB/disk.
  • The recommended distro to use is Ubuntu 15.04 (link to Ubuntu MATE 15.04).
• A Wi-Fi-capable wireless card.
• Super Smash Bros. for Nintendo 3DS Full or Demo.
• New 3DS. This does not work on Old 3DS.
• Patience. The hax is sort-of unreliable so your game will most likely crash a few times.
• Recommended: Another device to access the internet (phone, tablet, computer, console).

Preparing

• Install these packages using your package manager. For example, "apt-get" for Debian-based distributions (including Ubuntu).

  openssl libssl-dev libnl-genl-3-200 libnl-genl-3-dev libnl-3-200 libnl-3-dev pkg-config

• Find your wireless card's interface by opening a terminal and using the command "ip link". It would be something like wlan0 or wlp3s0.

  ian@ian-VPCEG34FX:~/Desktop/aircrack-ng-1.2-rc2/src$ ip link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
      link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
  3: wlan0: <NO-CARRIER,BROADCAST,ALLMULTI,PROMISC,NOTRAILERS,UP,LOWER_UP> mtu 1500 qdisc mq state DORMANT mode DORMANT group default qlen 1000
      link/ieee802.11/radiotap xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff

• You probably already have it, but get the homebrew starter kit and place it on your SD card.
• Download the smashhax .pcap files from the releases section of the smashbroshax repository.
• Determine the .pcap file to use. This should be straightforward using the file names.

  The built beacon-hax pcaps are located under "pcap_out/". In the filenames, "vXYZ" means game-version "vX.Y.Z". Full-game filenames for USA include "gameusa", while the other regions filenames include "gameother".

• Get the "Otherapp payload" from the Homebrew Launcher site and save it to "smashpayload.bin" at the root of your SD card.
  
• Download and extract aircrack-ng's latest release source.
• Save aireplay-ng.patch from the smashbroshax repository in the folder "aircrack-ng-1.2-rc2".

Compiling aircrack-ng/aireplay-ng

• Open a terminal and go to the "aircrack-ng-1.2-rc2" folder.
• Use the command "patch src/aireplay-ng.c < aireplay-ng.patch". If you get the following then it has succeeded.

  patching file src/aireplay-ng.c
  Hunk #1 succeeded at 560 (offset 1 line).
  Hunk #2 succeeded at 573 (offset 1 line).

• Run "make" and wait. The program should be compiled with the patch now. If you get the following as the last line then it has succeeded.

  make[1]: Leaving directory '/path/to/aircrack-ng-1.2-rc2/src'

The moment of truth

• Run these 3 terminal commands in order, using the wireless interface (from "ip link") you found earlier.
  
  sudo ifconfig wireless_interface down
  sudo iwconfig wireless_interface mode monitor
  sudo ifconfig wireless_interface up
  sudo iwconfig wireless_interface channel 6​
  (the last line was suggested by @difool. and might make the payload trigger faster)
• Enter the "src" folder in your terminal.
• Run the following command to start broadcasting the packet: "sudo ./aireplay-ng --interactive -r /path/to/smashbros_version_beaconhax.pcap -h 59:ee:3f:2a:37:e0 -x 10 wireless_interface"
  Use the .pcap file and wireless interface you found out earlier.
• On the 3DS system, start the game, then choose Smash and Group. Wait for the magic to happen.

Encountering errors? Something confusing?

• Does running aireplay-ng end with "End of file"? You might be running your installed version of aircrack-ng. Don't forget the ./ for "sudo ./aireplay-ng ..."!
• Please tell me the distribution you are using and the error you've encountered. This will help me fix your issue faster.
• If you don't get something, don't hesitate to point it out! I want to help anyone I can.
• Did you spot an inaccuracy or mistake I made? It would be great if you can tell me that too.
• This was before the Otherapp payload selector was added to the HBL site. This is kept here for legacy reasons or something.
  Spoiler

  • Determine what file you need to get from the Homebrew Launcher Payload section:
    
    Spoiler: Homebrew Launcher Payload

    With the release builds, the hax loads the payload from SD "/smashpayload.bin". This should contain the hb-launcher(https://smealum.github.io/3ds/) otherapp payload. Until there's a proper otherapp payload selector on the hb-launcher site, the payload can be downloaded from the following URL(see also https://github.com/smealum/sploit_installer):
    

    • "https://smealum.github.io/ninjhax2/Pvl9iD2Im5/otherapp/{PAYLOADNAME}.bin" Where {PAYLOADNAME} is: "FIRMVER_REGION_MENUVER_MSETVER".

    FIRMVER values(without quotes):
    

    • "POST5" = non-New3DS
    • "N3DS" = New3DS

    REGION values(without quotes):
    

    • "U" = USA
    • "E" = EUR
    • "J" = JPN

    MENUVER values(without quotes):
    

    • "11272": Non-JPN, system-version v9.0.
    • "12288": System-version v9.2.
    • "13330": System-version v9.3.
    • "14336": System-version v9.4.
    • "15360": System-version v9.5.
    • "16404": System-version v9.6.
    • "17415": System-version v9.7.
    • "20480_usa": USA, system-versions v9.9-v10.0.
    • "19456": Non-USA, system-versions v9.8-v10.0.

    MSETVER values(without quotes):
    

    • "8203": System-versions below v9.6.
    • "9221": System-versions starting with v9.6.

    For example, the payload URL for New3DS USA 9.9.0-X - 10.0.0-X is:https://smealum.github.io/ninjhax2/Pvl9iD2Im5/otherapp/N3DS_U_20480_usa_9221.bin
    The end result is a file named "smashpayload.bin" at the root of your SD card.

You are allowed to reproduce/reprint this tutorial, as long as a link back to this page (https://gbatemp.net/threads/tutorial-using-smashhax-with-linux.397194/) is included.

 

[cultopi]

This what you want?

Network controller: Intel Corporation Centrino Wireless-N 1000 [Condor Peak]

yours is a laptop built-in?

 

[ihaveahax]

yours is a laptop built-in?

yeah, which was shown in a video I made (don't think I linked to it in the post).

 

[Deleted User]

On windows i get stuck on step 4 at the github page. Can someone make a windows tutorial?

 

[cultopi]

yeah, which was shown in a video I made (don't think I linked to it in the post).

got this working with linksys WUSB54GC v1 usb adapter (had to dig deep in my storage room). For a full list of wifi adapters that have a better chance with smashhax, visit http://www.aircrack-ng.org/doku.php?id=compatibility_drivers
Confirmed 9.1J working but not my 8.1J (was hoping it would). I really hate this method, avoid this method if you have other choices.

 

[ihaveahax]

got this working with linksys WUSB54GC v1 usb adapter (had to dig deep in my storage room). For a full list of wifi adapters that have a better chance with smashhax, visit http://www.aircrack-ng.org/doku.php?id=compatibility_drivers
Confirmed 9.1J working but not my 8.1J (was hoping it would). I really hate this method, avoid this method if you have other choices.

This is based on Ninjhax 2, which only works on 9.0+ unfortunately. smashhax is more like a proof of concept, but I know curious people wanted to do it, so I made this tutorial.

 

[Favna]

well a friend will be lending him his physical ocarina of time on which i can install oot3dhax using ironhax on my o3ds (that is still on 9.9)... after which I can use that to install oot3dhax on my eshop version of ocarina of time on my new3ds so I guess i'm all good and i'll be leaving smashhax for what it is :\

 

[difool.]

Better work with this command line:
sudo ifconfig wireless_interface down
sudo iwconfig wireless_interface mode monitor
sudo ifconfig wireless_interface up
sudo iwconfig wireless_interface channel 6 #This line

 

[LordDaemon]

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 08:00:27:8f:e1:8b brd ff:ff:ff:ff:ff:ff

 

[ihaveahax]

sudo iwconfig wireless_interface channel 6 #This line

I might add this to the post soon, but what does it do?

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 08:00:27:8f:e1:8b brd ff:ff:ff:ff:ff:ff

what is going on?

 

[LordDaemon]

I might add this to the post soon, but what does it do?

what is going on?

it doesn't appear mlan0

 

[ihaveahax]

it doesn't appear mlan0

it's not always wlan0. enp0s3 might be ethernet. Do you have a wireless card physically installed?

 

[LordDaemon]

mmm idk really xd

 

[coreycubed]

With browserhax closing today, figured I'd give this a shot. Ran into the same issue as Favna with the first wireless card I tried, but found success with another. However, I'm still not able to trigger smashbroshax.

Console: N3DS running 10.1.0-27U
Using the Smash Bros. 3DS demo with smashbros_usademo_beaconhax.pcap. I downloaded N3DS_U_21504_usa_9221.bin from the Homebrew Launcher page by inputting 10.1.0-27U and selecting otherapp. It's on the root of my SD card as smashpayload.bin along with the 2.5 hax payload.

When I eventually get the payload to work, it just says "An error has occurred, forcing the software to close. The system will now restart." then reboots to the Home Menu. Doesn't look like it's about to trigger anything else. Any tips would be appreciated. I'll try to update this post with more details if I find out anything else.

Edit: In case someone else stumbles across this post looking for a workaround - browserhax reportedly still works on 10.1 for N3DS ONLY, but you need to change your DNS to 107.211.140.165 and 107.211.140.065 first. Power down your 3DS and then power it back on and IMMEDIATELY go to the browser. Browserhax should trigger and you can install themehax with the 2.5 payload. Then you can forget about browserhax

 

[ihaveahax]

When I eventually get the payload to work, it just says "The application has encountered an error and needs to close" then returns to the Home Menu. Doesn't look like it's about to trigger anything else. Any tips would be appreciated. I'll try to update this post with more details if I find out anything else.

This is expected honestly. The hax is not that reliable. I managed to get it working once using the same setup and all. If you get it working, though, it would be nice if you told us.

 

[coreycubed]

This is expected honestly. The hax is not that reliable. I managed to get it working once using the same setup and all. If you get it working, though, it would be nice if you told us.

Hey, I can try this a couple dozen times if that's what it takes. I did just now set my channel to 6 which helped trigger the payload a lot faster, from what I could tell.

Edit: Been trying this all evening and so far no luck

 

[Acryt]

I've also got it triggering the payload within seconds, but it just freezes up. Tried like 15 times. Heh. Dunno if it will ever even work. Took a lot of time setting up with Ubuntu to do this..

Crappy thing is all it will take is once is to get themehax on it and never have to do it again, but its not cooperating..

Tried an hour straight. Nothing, never once loaded correctly. Had the payload distorting the screen and such nearly instantly everytime, but nada.

 

[difool.]

This command line force the canal 6 Wi-Fi use. It's the canal Wi-Fi used to generate de pcap file by "yellow8", he speak about that into the Readme Github.

SmashBrosh crashed more fast when I used this command. I have the demo, and the 30 launch limitation wasn't enought for me to start the homebrew Launcher. Only screen BUG.

This work good for this man on Youtube: "New 3DS Smashhax installieren (Tutorial) [Deutsch|HD]"

 

[ihaveahax]

This command line force the canal 6 Wi-Fi use. It's the canal Wi-Fi used to generate de pcap file by "yellow8", he speak about that into the Readme Github.

SmashBrosh crashed more fast when I used this command. I have the demo, and the 30 launch limitation wasn't enought for me to start the homebrew Launcher. Only screen BUG.

I'll go try this as soon as I can (and hopefully be recording at the same time). I have the full game.

 

[coreycubed]

I'll go try this as soon as I can (and hopefully be recording at the same time). I have the full game.

I have the full game too but didn't want to delete my update data. Latest version of Smash Bros. for 3DS is 1.1.2 which isn't supported by smashbroshax yet, so you'd need to remove the data and go back to 1.0.0 to test it out. I won't be able to try that until later today.

Edit: Not to hijack a smashbroshax thread, but there's a workaround for browserhax (see my post above) on 10.1 N3DS which I'm planning to use instead. Still keen on seeing smashbroshax work semi-reliably, as we always need more payload vectors, but I'd recommend anyone looking to reapply hax use that first and then come back to tinker with smashbroshax

 

[LoneFlo]

Could this work with le last payload ? Just crashes again and again for now.

EDIT: Doesn't work. I used 2.1 payload.