What happens when you bring neuroscientists, hackers, and artists together for a weekend? Apparently, something that shouldn’t be possible.
I was brought in to help with Project 1 at Hacktionpotential, at the Paris Brain Institute (ICM), after the organizers discovered their planned EEG setup only provided offline data through the cloud, which is not ideal for a real-time BCI hackathon. Fortunately, I had just received the Mentalab Explore Pro on loan as an upgrade to my original Explore, which I’ve was beta-testing at the time, and have used for years in live EEG performances.
Those performances had become a bit stressfull as my setup had started showing connection jitter then several mid-show dropouts. I’d grown discouraged as I couldn’t trust the setup on stage. I blamed the EEG, as we went through several OpenBCI sets over the years with similar issues. However, it turned out the problem was my aging laptop slipping into power-saving mode when on battery. I only figured this out at the last moment – just before the jury event on the hackathon.
By the time of the hackathon I had a new MacBook as well. Because I remembered that the Mentalab acquisition software didn’t run on Mac, I brought my old Windows laptop as the acquisition machine, to stream via LabStreamLayer (LSL) to the Mac, hopefully relieving it from too much CPU strain, and preventing disconnection. Later did the people of Mentalab tell me that in fact, the software for the Mentalab Pro now does run on Mac with their new version! That’s a game-changer for me, as I can now develop my EEG-controlled music-setup in a much more straightforward and more portable manner (I am still carrying a modular synthesizer and light-setup about, but every kilo counts).
Saturday: Introducing EEG
Back to the hackathon. The event had 45 participants across three projects, one working with live human EEG, the others with pre-recorded mouse data. To get them on their way, I suggested an introduction and demonstration of EEG, so the organizers booked a room and off we went. I didn’t (want to) bring slides, so it stayed a nice workshop feel. We explored the signal together using the Explore Pro for the first time. Technically, the setup was intuitive and nearly identical to the previous version and everything was running within minutes. Then we explored artefacts, alpha waves, frequency analysis, and real-time paradigms like imagined movement and alpha lateralization.
After the demonstrations and discussion, I suggested to make sure that everyone could read the EEG stream via LabStreamLayer (LSL) so they were sure to be able to build a game compatible with any EEG system, and not depend on a vendor API. Also, they should be able to use existing open-source tools to playback EEG data, which would help development and not require a live person to be connected through the day/night. Quickly, after plugging in a router from the organizers, we confirmed the stream was readable across the room – I love LSL. Then the groups suggested to do some real recordings with imagined movement to train their model (they were far quicker on the uptake than I expected, reading papers during the discussion). We connected a second test person, and figured out we could manually provide markers in the data and LSL stream with the Mentalab software, which we did to identify periods of imagined movement (left vs right hand). I gave no instructions on how to do any of it, and they didn’t expect me to: they found all Python packages, tools and algorithms by themselves. Honestly, I was very impressed by how the groups had organized themselves and how easily they took on different roles and collaborated: EEG analysis, machine learning, game design, literature research, presentation, visual design. Still, I thought: there’s no way this is going to work with less than a day left.
Sunday: I Was Wrong
I was wrong. The next afternoon I returned to provide the EEG stream for the final build session. There were three working game prototypes, each using a different EEG system. The Mentalab group had built an EEG-controlled tractor navigating a maze. The calibration, the timing, the control, it was pretty close to what you’d expect from any BCI lab.
Then it was time to prepare for the presentation and demonstration to the jury. They wanted to calibrate and train the model right before, so we connected the final EEG subject and started recording, before plugging the amplifier back in the charger to ensure a full battery after a day of work. Then, just before going on state, the Explore Pro lit up red and refused to stream. Really? The battery? After freaking out a bit, I looked online for the manual, and after skimming it turns out the device automatically stores data internally. Once the memory is full, it won’t record, and will indicate this by a red light at startup. Luckily, a button-press combination clears the memory so we were back in the game. However, after several minutes the familiar jitter appeared, showing connections problems with signal dropping, catching up, then dropping again. My heart sank. Then I remembered a suggestion months ago that it might be the power-saving mode on battery, which I never got to test. I ran to the other side of the auditorium, grabbed my charger, ran back, plugged in and the jitter stopped. It was always my computer. Never the EEG…
The Presentation
The group finally delivered a very impressive presentation, with scientific context, explanation of the BCI methodology, and all with a clear collaborative and open-science attitude. Then the demo. It went great, but after a while it was clear that the EEG-control didn’t work as well as during rehearsal… Perhaps the training set was insufficient, but there was no time to redo it. Typical. But they took it in stride and seemed genuinly happy with the time they spent together, and what they achieved.
Events like this Hacktion Potential are exactly what the BCI community needs: diverse people, focus, fun, and just enough chaos to build things that can kickstart people’s interest and learning process. It was also a test for me, and gave me renewed enthousiasms to do my EEG-music performance, knowing that the Mentalab Explore Pro will be a reliable, lab-quality EEG in the field. I do recommend to read the manual.