Riz Virk is an entrepreneur, angel investor, bestselling author and founder of Play labs @ MIT.
This discussion made me recall the first time I really through semi-seriously about BCIs. It was while watching the 80s movie Firefox, starring Clint Eastwood as a spy/pilot who is tasked with stealing a super-secret jet from the Soviet Union. What made the jet special was that it could respond to the mental commands of the pilot; the idea was that if you could cut off even a second (or perhaps s milliseconds) of response time, the fighter jet would have an advantage over NATO’s fighter jets, which still used handheld controls. The one catch when Eastwood finally steals the jet is that it won’t respond to his commands — unless he thinks in Russian!
While we’re not at Matrix or Firefox level BCI’s just yet, there were presentations from a number of interesting companies which helped shed light on the current state of the industry and methods.
The first thing I noticed was that all of these companies were working on non-invasive neural interfaces. This is in contrast to Neuralink, company funded by Elon Musk which is creating a chip that can be implanted into the brain by drilling a small hole into the skull. This much vaunted chip promises magical properties — including stimulating the brain’s pleasure centers and increasing human intelligence with AI. The first versions, though will be targeted towards those with brain disorders. Personally, I don’t know of any gamers who are ready to drill holes into their heads at this time, but that may change in the future.
Figure 1: In the Matrix, humans could plug into Simulations
More importantly, unlike in the Matrix, which had a two-way connection, these companies were mostly focused on getting input from the player in gaming through a brain interface, though one of them also was interested in having the user have a more immersive realistic experience.
There were six companies doing interesting things in the field of Neuro linking technology that presented at the virtual conference:
Valve — The first presentation was by Mike Abinder, an experimental psychologist associated with the University of Washington, who has been working at Valve on some mysterious thing related to BCI and gaming. He basically gave an overview of how BCI’s could help in developing and improving gameplay experiences. While Abinder was coy about what Valve was actually doing in this space, he did make some good arguments about real-time data that could be gathered on the player’s experience and emotions while playing, which could lead to a whole new level of adaptive gameplay. He also gave an overview of “what we can measure today” — which was mostly EEGs in different permutations — his technical terminology was that we can measure “EEG correlates of Task/Engagement and Mental Workload in Vigilance, Learning and Memory Tasks”. What I got out of it was that the basic idea is to measure the EEG signals coming from the brain while the player is doing different things — competitive play, strategy games, etc., measuring things such as “valence, arousal and dominance”.
Figure 2: The new EEG device from Brainattach that is touted as Biotactical Gameware
Figure 3: The fully immersive haptic Teslasuit provides a full immersion
Teslasuit — The most fun presentation on the first day came from Dmitri Mikhalchuk of Teslasuit, company that is technically not a BCI at all — unless you replace the word “brain” with “body”. for a BBI. The Teslasuit is a fully immersive haptic suit that, when combined with a VR headset, is meant to not only immerse you in a world, but let you feel the gameplay through stimulating different parts of the body. The Teslasuit, which debuted and made waves at CES in 2018 and 2019, looks more like something straight out of Ready Player One. The way that it works is that it emits electric impulses to different parts of your body, and uses and advanced motion capture system and biometrics to see what you are doing as input. As far as I could tell, it wasn’t using EEGs or brain interfaces strictly, but the field seems to merge with biometric feedback.
During the second session, there were presentations from the following three companies, which shed even more light on the state of the industry and what it will take to get more adoption of BCI’s.
Figure 4: Getting input for gaming today (controller) vs the future, Brain and Biometrics, via Erin Reynolds of Flying Mollusk
Figure 5: The evolution of Hardware for BCI’s according to Neurable’s Adam Molnar
Neurable — next cam Adam Molnar from Neurable, a company which has had impressive demos of being able to move objects in VR through thought alone. Neurable, which grew out of research at the University of Michigan, is a Boston based company that is bringing neural interfaces to VR worlds with modified VR headsets that can detect EEG signals. That’s not all they use — they also use eye tracking information and other information, though the focus is on different EEG signal processing. Adam gave an overview of the history of BCI for gaming and the road left to travel. This included devices like the Star Wars Force Trainer from Mattel, which let you feel like you were actually using the Force.
Figure 6: Adam Molnar talked about the history of BCI in games, including the Star Wars Force Trainer!
Figure 7: The Brink Bionics Glove senses movements and gives gamers an edge!
Figure 8: Erik Lloyd talked about the state of BCI today vs. the future
Erik also talked about the evolution of neural technology in gaming and made the point that today we are in phase I — where a stimulus is presented by the game to the user, and then we measure the result ( or the reaction of the user), and then map it to some known reactions. He calls this “dependent” BCI and that there will eventually be “independent” BCI which wont’ require presenting a scenario to the user (as in VR) but will allow machines to directly pick up on any of our thoughts and we can use those thoughts to control the machines.
Erik’s distinction between “dependent” and “independent” BCI was perhaps a good place to end the discussion of the state of the industry today. You’ll notice I watched the conference not in VR or AR or through a BCI, but simply on Youtube.
Today’s BCIs use a combination of biofeedback (heart rate, skin/stress response), affective computing (emotions based on facial recognition), EEG signals detection (and there is a lot of research on different ways to interpret these signals), eye tracking (for intention inside VR), motion capture (for the teslasuit), and electrical impulses to simulate touch. While the state of the industry is no longer research lab only, it still has a long way to go to directly control any game from your mind. It may not be long that the impact of these technologies goes way beyond gaming — who knows there may be a Firefox like fighter jet on the way!