AI Hardware and Video Games: Is there a Symbiotic Future?
After investing billions in artificial intelligence (AI) technology, tech companies aren't tired of touting its prowess to the general public. "We should stop training radiologists now. It’s just completely obvious that within five years, [AI] is going to do better than radiologists," said one prominent academic... in 2016. A tech company just laid off 4000 employees due to “intelligence tool capabilities compounding faster every week”. “Within the next year, I believe the majority of companies will reach the same conclusion and make similar structural changes,” predicted the CEO, who isn't unfamiliar with poor decision-making for his companies. Monetary incentives be damned, they are betting that it is the future; more specifically, it is our collective future, so we have to live and breathe AI.
This push for the technology in recent years is mostly driven by large language models (LLMs), which are built upon transformer architecture, a specific technique within the broader field of AI, following the ChatGPT phenomenon of late 2022. Of course, we can't be content with having AI as software; we have to house it in its own hardware, often inspired by science fiction tales that warned of rogue AI. But these are just tales! If we can't have flying cars in the 21st century, at least let us have virtual (not imaginary, mind you!) companions; and let's trap them in a box while we're at it!
Prominent examples emerged in 2024, with devices such as the Humane Ai Pin and Rabbit R1 making the front page of tech websites. Despite early "hiccups" (Humane discontinued its first and only device in less than a year), AI hardware endeavours persisted. The Rabbit R1 has received multiple updates since its launch to address initial complaints, and other companies brought forth their own devices. This year's CES and MWC were not lacking in upcoming (or conceptual) AI hardware, and even major players like OpenAI and Apple (reportedly) have AI hardware of their own in the works. If AI hardware is the next big thing, according to the tech industry, then what can they do for, if anything, for video games games?
To date, there hasn't been a dedicated AI device for gaming purposes, but there are indications of what these could do, and existing AI hardware might have gaming use cases (right?). So I decided to take a look by trying out some of the different AI hardware that are available on the market.
H(A)i, hardware
When it comes to AI devices, they come in all shapes and forms. Companies are seemingly throwing AI at everything to see what sticks. Heck, even retro handhelds from Anbernic have an AI assistant. But here we are interested in dedicated AI hardware, not something existing that has the abbreviation slapped onto it.
With a few exceptions, dedicated AI devices have mostly tended to adopt the form of wearables (pieces of tech that you wear on your body). As companies are still trying to crack the code on the form factor that works best, the market is currently buzzing with devices of all formats, some more creative than others. Essentially, the hardware is receiving input in similar ways you would query an LLM chatbot and output corresponding responses. With LLMs being multimodal (i.e., accepting input of different formats), this extends beyond text and speech to include pictures and videos.
As such, AI hardware needs to replicate these input capabilities, but not all devices have the same capabilities. The ones I've tested can generally be grouped into three categories: screenless wearable, smart glasses and the wildcard. So let's take a look what they can do in terms of gaming.
I have no screen, and I must chat
In their simplest, least conspicuous form, AI hardware are wearable devices. The Looki L1, Plaud and Omi fit in this category. Basically, you can speak to them, and they will answer back. But how they function differs.
The Looki L1 is advertised as a lifelogger, as, thanks to its integrated camera, it can record videos and take pictures, and even automatically make short vlogs of your daily activities. Using it, I feel pretty self-conscious with the outward-facing camera and its conspicuous design. On hikes and solo sport activities it can be acceptable to use, but in the general public, I preemptively think that it will raise privacy concerns from those around me, so I don’t wear it out.
Plaud has a couple of devices out, and they function similarly. They are note-taking and transcription devices first, which record conversations and autio notes, transcribe them, and you can view them on the companion app. It doesn't induce privacy anxiety as the Looki L1, since it doesn't pack a camera. Plaud’s companion app offers further AI functions such as automated summaries and querying the AI assistant on past conversations.
The Omi sits somewhere between the Looki L1 and the Plaud. It doesn't have a camera but has a microphone that you can talk to. Like the L1, it creates a summary of your daily activity by listening in. And like the Plaud, it can transcribe audio memos and conversations. The companion app now has an “app store” of sorts with apps that can give the Omi device specific roles and provide corresponding answers.
None of these devices is directly marketed as gaming-oriented. I've found myself using them mostly to take hands-free notes when working on gaming reviews and articles. For the L1 and Omi, I've tried asking walkthrough assistance and strategy advice for video games. The Omi doesn’t seem to be very good at catching what the user is actually saying from the get-go but does process it better given some more time, while, the L1 doesn't do a great job at identifying games with its camera, so vocal commands work best with it and these provide quicker responses than with the Omi.
Of the three, the Looki L1 has served me most, as it packs a speaker and you can query it and get relatively fast replies, without having to rely on a phone. There's a missed opportunity for Omi to have included a speaker. I'm quite fond of its minimalist and more privacy-focused design (it can't be used as a spy cam). The reliance on the companion app is a hassle and the audio that it transcribes can be unreliable, making it challenging to be a real companion.
Looki L1 vs Omi responses to the same query
How not to be a glasshole
AI glasses are an emerging category that has been a personal interest. Formely popularised as smart glasses, companies such as Xreal and Rokid changed the narrative around AR from glassholes to handheld gaming companions. These work as portable displays that mirror your handheld’s screen. Nowadays, the attention has shifted mostly to AI glasses. These don't focus on providing large virtual displays to mirror your source device, but are more geared towards packing AI capabilities in the form factor of glasses.
This approach isn't new in itself. At least since 2019, there were audio smart glasses with tiny speakers embedded in the arms that allowed you to activate your smartphone's assistant. The likes of the Meta Ray-Ban have popularised this form factor, but with Meta's more than dubious track record around privacy, you might want to consider other options out there.
The two alternatives that I've tried are more privacy-focused but do so with different approaches. The first one is the Even Realities G1. These are, by far, the sleekest and least inconspicuous smart glasses that I've ever tried. It has integrated displays, a microphone and touch panels; importantly, it doesn't house any cameras and, as such, makes for a less intrusive device. Nothing fancy with the displays either. There are monochromatic green displays on each lens which mostly mirror phone notifications, show a map and display answers when its proprietary AI assistant, Even AI, is queried.
There's a voice memo option, which I use in a similar way to the Looki L1, that is, to take notes of my reviews, and I can also query the Even assistant for gaming advice. As there's no speaker in the G1, the answers are displayed on the glasses themselves. It's handy, but built-in speakers would have made it more versatile. Even Realities also allows developers to access its SDK to create custom apps for the glasses. So, technically speaking, there could be gaming-focused apps; although none have made it into these AI glasses’ main software. So far, asking Even AI gaming-related questions and mirroring notifications from phone apps such as Discord can be handy out-of-the-box “gaming” uses for the G1.
Brilliant Lab's Frame and the upcoming Halo take a whole different approach to privacy. The company's devices and software are fully open source. They include a single display, a microphone and a tiny camera. While the latter can lead to some privacy concerns, it is not very conspicuous, and all data is processed locally on your paired smartphone and not stored elsewhere or used by the company.
Unlike most mainstream AI hardware, Brilliant Lab's products are oriented towards developers. They can be used as is to chat with the AI assistant via voice or to give you insights on what you're seeing. But the true purpose of their devices is to leverage the developer community to tinker with them and make software. Think of their products like the Raspberry Pi of AI hardware. There's a growing number of projects on Brilliant Lab's Discord channel, if you have the technical skills, you can even make your own gaming software for it. There’s even a Dino Game for it! Brilliant Labs' upcoming device, Halo, might make such projects easier with its "vibe coding" capabilities. The idea is that you would chat with the AI assistant to describe an app you'd like to build, and the assistant will build it for you (for $19.99/month). We still have to see it in the works to make a judgment (Halo ships in the coming months) and how well gaming apps work on the device.
The wildcard’s redemption arc
You might have heard of the Rabbit r1 from CES 2024 or you might know it from the ensuing backlash. Frankly, it is impressive to see that the company has persisted with its AI hardware efforts, especially when other companies like Humane called it quits much earlier. In attempting a redemption arc, rabbit has released major updates to address the initial feedback while adding new features to the r1.
This dedicated AI hardware that has a screen, camera, speaker, microphone, and a scroll wheel for navigation and interaction. It is the most full-fledged AI hardware I've tried and packs a unique look. It functions like a phone without trying to replace your phone. It's something of a wildcard that has been trying to find its footing and might just be on the way to find sure ground.
With rabbitOS 2, the R1 experience radically changed. It updated its UI and UX, and, notably, introduced Creations. The latter leverageds vibe coding to allow users to create their own apps and even opened up a dedicated store for users to share their projects. This alone makes the r1 the AI device with the most gaming-oriented uses from the devices. There's a Tetris clone, an NES emulator (that lacks any instructions regarding how to load ROMs onto it), a Silksong map (in German) and many more. R1 Creations not only makes the device a gaming companion more than most AI hardware, but also turns the R1 into a standalone gaming device with casual games.
But (of course there’s a ‘but’, and this is a major one) making apps through Creations requires a subscription. You do get 3 free creations, but then you are locked out from creating apps unless you subscribe for $70/month (billed yearly) for 30 tasks per month or $30 for one single month with 3 tasks. It should be noted that once you have conversed with R1 about one Creation, you can't edit the output. If you want anything different about the look or functioning, you'll have to use up another task and create a new app (I learnt that the hard way).
The R1 feels like a fun hardware to test the capabilities of AI, and I think that the device would have had more success if it was made more open to developers, akin Brilliant Lab's approach, and less gated by subscriptions. Rabbit is gearing up to announce its next device, so let's hope it takes the lessons learnt from the R1 constructively.
Do you need an AI hardware as a gaming companion?
The short answer is: no. Now for the longer answer: I'm generally open to new technologies and look forward to testing them. While I've found some practical uses for some AI hardware, they don't really make that great of gaming companions. The R1 can have some actually practical video game uses, but the rest don't do much more than what you can already do with your phone and an internet connection.
I’m actually quite surprised that there hasn’t been an AI hardware that is directly marketed to gamers, given how companies are trying to see what sticks. After trying the devices mentioned in this article, I have some thoughts on what would make for a convincing AI gaming companion. It can bear any form factor, but a speaker or on-device screen would be required to receive responses without having to rely on a phone. It should remain functional even offline, and this could address privacy issues. Vibe coding your own apps, such as walkthroughs or guides for specific games, could be interesting, but the companion has to provide factual assistance for individual games. The immediate output from these devices aren’t instantly verifiable, so you’ll have to take the output at face value, while being cognisant of the “hallucination” phenomenon of AI models. Moreover, locking vibe coding behind subscription doesn’t make for an attractive proposition. A camera that recognises on-screen games, in-game location and progress to provide contextual advice and insights would make for a "wow factor", but I'm not sure how realistic this is for dedicated hardware in the near future. Others might want an assistant that autobattles or handles repetitive tasks, and some of the r1’s features provide an inkling of this possibility. We've recently seen an offline AI that translates games that haven't been localised, and this feature would lend itself well to AI glasses.
Of course, such hardware would be contingent on being adopted by gamers. Given the backlash on AI generated content in the industry, even against popular developers like Larian, it might be an uphill battle. The environmental impact of the technology, and ongoing debates on the veracity of an AI bubble further put into question the sustainability of such hardware. However, the AI hardware we're thinking of here is directly assisting players, instead of being used to make marketable games and take over jobs from humans. The training data would also be a concern, unless it is developed in partnership with games websites.
With the rise of vibe coding, I think that we will see more gaming-oriented AI apps, which could be ported or even developed specifically for AI hardware. Whether they evolve to become a symbiotic accessory for video games is an open question, but if current AI devices are any indication, most won’t have practical uses for the general gamer in the near future.
