If you can use the browser, I suggest that you use the new browserhax (up to 11.0.0-33!) in the meantime.

Try out Ubuntu 15.04: http://old-releases.ubuntu.com/releases/15.04/

This requires a New 3DS. The demo version doesn't work and is patched. If you have 1.1.3 or newer delete/disable the update data.
officially called smashbroshax sometimes smashax
Here is a new tool I call smashbroshax-helper. It is a graphical interface for the exploit which simplifies most of the process of broadcasting the packet. It requires almost no setup outside of creating a bootable Linux USB/DVD.

Important notes:

• This does not work on Fedora or Red Hat-based distributions because aircrack-ng needs to be compiled on it (and I can't figure out how to install the needed things).
  
• Don't use a virtual machine!
• This probably will never work on Old 3DS.

Download smashbroshax-helper beta

It is recommended that you use a live Ubuntu 15.04 image. Versions 15.10+ have issues. http://old-releases.ubuntu.com/releases/15.04/

1. Create a bootable USB/DVD with a Linux distribution (there are various guides online).
2. If possible, get a second USB/SD card/storage device, download and save the above .zip to it.
   • Don't extract the contents of it to the USB device, as it might cause problems. Just save the .zip file to it.
   • If you can't do this, you'll have to connect to the Internet while in Linux to download it.
   • If using a bootable USB, make sure you can use two ports at a time. Don't take out the Linux USB while it is being used!
3. Restart your computer and run the bootable USB/DVD you created.
4. Extract the contents of the .zip to the Desktop.
5. Open the smashbroshax-helper folder and double click "smashbroshax.sh".
6. Follow the on-screen prompts.
7. If everything goes well, you should now have homebrew!

Video demonstration, from boot to shutdown:

from https://gbatemp.net/threads/tutorial-using-smashhax-with-linux.397194/page-9#post-5842512

---

If you would like the full tutorial, involving terminal usage and compiling:

Spoiler: Everything

To reverse the changes to your wireless interface and remove issues connecting to networks after, rebooting your system or changing it to "Managed" instead of "monitor" should fix it. How to do that is in the second to last section.

---

@Cydget made a script that condenses most of this into a script. I have not tried it myself yet but it seems to work for others.

Spoiler

So, I made a little script for this. If anybody wants it, then unzip this file and read the readme. And yes, I like to pipe things. http://www.mediafire.com/download/oulnubnzkk9g3i0/smashhaxEZ.zip

---

Requirements

• Any Linux distribution should do (this has only been tested with Debian-based distributions). Windows and OS X users should wait or find a method for now, sorry!
  • Please do not use Linux in a virtual machine, it likely won't give direct access to your wireless card. Dual boot or use a live USB/disk.
  • The recommended distro to use is Ubuntu 15.04 (link to Ubuntu MATE 15.04).
• A Wi-Fi-capable wireless card.
• Super Smash Bros. for Nintendo 3DS Full or Demo.
• New 3DS. This does not work on Old 3DS.
• Patience. The hax is sort-of unreliable so your game will most likely crash a few times.
• Recommended: Another device to access the internet (phone, tablet, computer, console).

Preparing

• Install these packages using your package manager. For example, "apt-get" for Debian-based distributions (including Ubuntu).

  openssl libssl-dev libnl-genl-3-200 libnl-genl-3-dev libnl-3-200 libnl-3-dev pkg-config

• Find your wireless card's interface by opening a terminal and using the command "ip link". It would be something like wlan0 or wlp3s0.

  ian@ian-VPCEG34FX:~/Desktop/aircrack-ng-1.2-rc2/src$ ip link
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
      link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
  3: wlan0: <NO-CARRIER,BROADCAST,ALLMULTI,PROMISC,NOTRAILERS,UP,LOWER_UP> mtu 1500 qdisc mq state DORMANT mode DORMANT group default qlen 1000
      link/ieee802.11/radiotap xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff

• You probably already have it, but get the homebrew starter kit and place it on your SD card.
• Download the smashhax .pcap files from the releases section of the smashbroshax repository.
• Determine the .pcap file to use. This should be straightforward using the file names.

  The built beacon-hax pcaps are located under "pcap_out/". In the filenames, "vXYZ" means game-version "vX.Y.Z". Full-game filenames for USA include "gameusa", while the other regions filenames include "gameother".

• Get the "Otherapp payload" from the Homebrew Launcher site and save it to "smashpayload.bin" at the root of your SD card.
  
• Download and extract aircrack-ng's latest release source.
• Save aireplay-ng.patch from the smashbroshax repository in the folder "aircrack-ng-1.2-rc2".

Compiling aircrack-ng/aireplay-ng

• Open a terminal and go to the "aircrack-ng-1.2-rc2" folder.
• Use the command "patch src/aireplay-ng.c < aireplay-ng.patch". If you get the following then it has succeeded.

  patching file src/aireplay-ng.c
  Hunk #1 succeeded at 560 (offset 1 line).
  Hunk #2 succeeded at 573 (offset 1 line).

• Run "make" and wait. The program should be compiled with the patch now. If you get the following as the last line then it has succeeded.

  make[1]: Leaving directory '/path/to/aircrack-ng-1.2-rc2/src'

The moment of truth

• Run these 3 terminal commands in order, using the wireless interface (from "ip link") you found earlier.
  
  sudo ifconfig wireless_interface down
  sudo iwconfig wireless_interface mode monitor
  sudo ifconfig wireless_interface up
  sudo iwconfig wireless_interface channel 6​
  (the last line was suggested by @difool. and might make the payload trigger faster)
• Enter the "src" folder in your terminal.
• Run the following command to start broadcasting the packet: "sudo ./aireplay-ng --interactive -r /path/to/smashbros_version_beaconhax.pcap -h 59:ee:3f:2a:37:e0 -x 10 wireless_interface"
  Use the .pcap file and wireless interface you found out earlier.
• On the 3DS system, start the game, then choose Smash and Group. Wait for the magic to happen.

Encountering errors? Something confusing?

• Does running aireplay-ng end with "End of file"? You might be running your installed version of aircrack-ng. Don't forget the ./ for "sudo ./aireplay-ng ..."!
• Please tell me the distribution you are using and the error you've encountered. This will help me fix your issue faster.
• If you don't get something, don't hesitate to point it out! I want to help anyone I can.
• Did you spot an inaccuracy or mistake I made? It would be great if you can tell me that too.
• This was before the Otherapp payload selector was added to the HBL site. This is kept here for legacy reasons or something.
  Spoiler

  • Determine what file you need to get from the Homebrew Launcher Payload section:
    
    Spoiler: Homebrew Launcher Payload

    With the release builds, the hax loads the payload from SD "/smashpayload.bin". This should contain the hb-launcher(https://smealum.github.io/3ds/) otherapp payload. Until there's a proper otherapp payload selector on the hb-launcher site, the payload can be downloaded from the following URL(see also https://github.com/smealum/sploit_installer):
    

    • "https://smealum.github.io/ninjhax2/Pvl9iD2Im5/otherapp/{PAYLOADNAME}.bin" Where {PAYLOADNAME} is: "FIRMVER_REGION_MENUVER_MSETVER".

    FIRMVER values(without quotes):
    

    • "POST5" = non-New3DS
    • "N3DS" = New3DS

    REGION values(without quotes):
    

    • "U" = USA
    • "E" = EUR
    • "J" = JPN

    MENUVER values(without quotes):
    

    • "11272": Non-JPN, system-version v9.0.
    • "12288": System-version v9.2.
    • "13330": System-version v9.3.
    • "14336": System-version v9.4.
    • "15360": System-version v9.5.
    • "16404": System-version v9.6.
    • "17415": System-version v9.7.
    • "20480_usa": USA, system-versions v9.9-v10.0.
    • "19456": Non-USA, system-versions v9.8-v10.0.

    MSETVER values(without quotes):
    

    • "8203": System-versions below v9.6.
    • "9221": System-versions starting with v9.6.

    For example, the payload URL for New3DS USA 9.9.0-X - 10.0.0-X is:https://smealum.github.io/ninjhax2/Pvl9iD2Im5/otherapp/N3DS_U_20480_usa_9221.bin
    The end result is a file named "smashpayload.bin" at the root of your SD card.

You are allowed to reproduce/reprint this tutorial, as long as a link back to this page (https://gbatemp.net/threads/tutorial-using-smashhax-with-linux.397194/) is included.

 

[Metalchilla]

so i have a n3ds with firmware 11.0 and using the 10.7 payload. my smash is version 1.0.1 and every time i run the .sh and it broadcasts i go into group but it freezes and goes to the home menu and has to restart. I tried both 1.0.0 and 1.0.2 when it told me to pick a version and none work.

 

[greatatlas]

I'm in the same boat, mostly. I'm running Ubuntu 14.04, firmware 11.0.0.33U, and I've got the custom compiled PCAP for 1.0.1. The game will crash, but it will never load the payload. That says to me that the PCAP works and there's an issue with the payload. I just downloaded the newest starter kit and the otherapp payload for 10.7.0.32U. Am I doing something wrong?

 

[Guriam]

I'm in the same boat, mostly. I'm running Ubuntu 14.04, firmware 11.0.0.33U, and I've got the custom compiled PCAP for 1.0.1. The game will crash, but it will never load the payload. That says to me that the PCAP works and there's an issue with the payload. I just downloaded the newest starter kit and the otherapp payload for 10.7.0.32U. Am I doing something wrong?

Many people are getting the same problem, me as well. The game crashes but returns to the main menu.

 

[rtu96]

Many people are getting the same problem, me as well. The game crashes but returns to the main menu.

Same here. I'm somewhat familiar with Linux because of my current college class. Hopefully someone with more experience can chime in to this issue.

 

[000tin]

So.. Does this work on 11.0?

 

[Creat1ve]

Worked for me, thank you!

 

[Jay Clay]

So this is the only way you can get smashhax running on the n3ds right?, I tried with a couple of distros, ubuntu 14.04 and ubuntu mate 16.04, I don't use virtual machine, and I was able to install efi tools before running the script, it seems like using a macbook the wlan will be like "wlp2s0", but everytime I try to run the scrip .sh there is a ioctl error on the terminal, my macbook is mid 2012 and it seems like my wifi is not supported in monitor mode, yeah I am pretty sure that is my problem, in terminal I used the command sudo iwfconfig wlp2s0 mode monitor and the same thing, mode is not supported, unfortunately I don't have another pc or laptop to try.

 

[ThatNintendoNerd]

I cannot get smashhax to work for the life of me.

Spoiler

This is my third attempt as of now with no success rate, I haven't even had a crash on it yet

What I used:

• Ubuntu MATE 15.10
• Smash 3DS ver 1.1.0
  
• smashhax-helper, wireless interface - wlp2s0
  
• otherapp payload for 10.7 (Since I heard 10.7 works for 11.0) named smashpayload.bin
• New 3DS XL 11.0 SysNAND (Nothing else)

Smash is my only Homebrew-able game. I gave up after 10K packets. So any ideas or should I just buy a pre-owned Cubic Ninja?

 

[000tin]

I can't run the smashbroshax.sh file.
It opens in gedit by default when I try to run it in other applications it doesn't work..
Am I supposed to run it in terminal?
I get two errors and it stops
Line 12: cd: resources: no such file or directory
Line 13: gksudo: command not found

 

[Guriam]

I thought it was because of the wireless interface being wlp2s0, but in mint is wlan0 and I already tried on another computer. Still the same. Well I guess it's waiting for another freehax, or give 40$ to oothax.

 

[Darukeru]

I thought it was because of the wireless interface being wlp2s0, but in mint is wlan0 and I already tried on another computer. Still the same. Well I guess it's waiting for another freehax, or give 40$ to oothax.

Smashhax works for me on 11.0 using a 1.0 cartridge, but I didn't used the .sh file.
I followed a different YouTube tutorial where you've to compile some stuff and start sending packets using a terminal command.
And my wireless interface also is wlp1s0 whatever; I used Ubuntu 16.04 installed, not virtualized or booted from a USB.

 

[Metalchilla]

I can't run the smashbroshax.sh file.
It opens in gedit by default when I try to run it in other applications it doesn't work..
Am I supposed to run it in terminal?
I get two errors and it stops
Line 12: cd: resources: no such file or directory
Line 13: gksudo: command not found

you probably have an old version of linux. i would recommend ubuntu mate 15.04. i had the same problem

 

[Drek_blood]

I have the digital version of the game on 1.0.1. My New 3DS is on 10.7E and I can't make it work properly. The wireless interface is ok as my console reboots every time I run the script, but I can't get the payload to run. Right now I have no other option to get homebrew back to my New 3DS, right?

 

[Guriam]

Smashhax works for me on 11.0 using a 1.0 cartridge, but I didn't used the .sh file.
I followed a different YouTube tutorial where you've to compile some stuff and start sending packets using a terminal command.
And my wireless interface also is wlp1s0 whatever; I used Ubuntu 16.04 installed, not virtualized or booted from a USB.

Already did that mate. Still no luck tho.

 

[000tin]

you probably have an old version of linux. i would recommend ubuntu mate 15.04. i had the same problem

thanks, i'll try that.
op recommended 14.04. though..

 

[Metalchilla]

thanks, i'll try that.
op recommended 14.04. though..

i got linux 14.04 (i didnt get ubuntu mate 14.04) and had the same problem as you.
it kept opening in gedit and once i got the latest version of ubuntu mate it didnt open in gedit but instead gave me the option to run in terminal, cancel, or just run in general

--------------------- MERGED ---------------------------

I have the digital version of the game on 1.0.1. My New 3DS is on 10.7E and I can't make it work properly. The wireless interface is ok as my console reboots every time I run the script, but I can't get the payload to run. Right now I have no other option to get homebrew back to my New 3DS, right?

well there is a new exploit coming out on friday and the ninjhax team is almost finished with ninjhax for 11.0 so you can expect that.
also im not sure smashhax-helper works with 1.0.1. you can always do smashhax the hard way but as i said its hard.

 

[000tin]

i got linux 14.04 (i didnt get ubuntu mate 14.04) and had the same problem as you.
it kept opening in gedit and once i got the latest version of ubuntu mate it didnt open in gedit but instead gave me the option to run in terminal, cancel, or just run in general

Yes It's running now., but I get an error message and don't even get to beacon.

I can select 2 interfaces: lo or eth0, I tried both
Eshop smash is on 1.0.1 by default when you download it, so I chose 1.0.0

The terminal window opens but doesn't say ' use this package?'
I get the error 'ioctl(SIOCIWMODE) failed: Operation not supported'

--------------------- MERGED ---------------------------

I guess I'll just wait for plutoos exploit on friday

 

[lulu66000]

his works on old 3ds ?

 

[000tin]

his works on old 3ds ?

no. this requires a new 3ds

 

[Quin452]

I'm going to need to edit the program a bit later, but for now edit the script to replace "gksudo" with "sudo".

Hi - I was getting the same issue, and changed it to sudo. Now the error is

<file path and name>.sh line 12: cd: resources: No such file or directory
sudo: ./lua53-32: command not found

EDIT: I tried Ubuntu Mate, and it works fine... or loads fine. The packets stop loading at 73 packets (19 pps).

EDIT 2: The "packet 73" issue is with the wireless card, so I've bought a new USB dongle (£15). I can send packets now, and even begin to connect, but "an error has occurred" and the N3DS is restarted. So I need help with this now :-D

EDIT 3: I've tried using the 1.0.1 version pcap (took ages to find), but still the same error. I think I've tried everything solution offered, although limited. Any ideas what could be causing the problem?