My final project during my summer at OpenBCI was an EMG-Controlled Tetris Game using the OpenBCI Cyton and UltraCortex Headset. During my last two weeks, I was excited to put together a project completely from ideation to final product. Tetris was a great de-stressing game for me in high school, so I thought it would be a fun idea to work with the different capabilities of the Cyton to be able to play it completely hands-free.
In terms of controls, I decided to use the Cyton’s accelerometer to detect a tilting movement in the left, right, and down direction to move the piece left, right, and dropping it, respectively. Using the pyAutoGui library, I was able to map each of these processes to the corresponding keyboard click. I was excited to create a demo that utilized the accelerometer, since many of the projects and tutorials in the past did not highlight this feature. To rotate a piece, I decided to use eye blinks. We can detect the eye blinks using the flat electrodes on the forehead with the UltraCortex.
I enjoyed learning about the whole pipeline from UDP output to processing the data and coding that all on my own. With this project, I learned that it was really important for me to keep track of what the data looked like at each point along the way, from in the GUI to what was being sent over UDP to what was being processed with the final movements, since it was a multi-stepped process. I also learned a lot about different industry-standard networking protocols, like LSL and OSC, before finally settling on UDP. I learned how to create sockets and listeners, and how that entire framework worked in conjunction with the data output from the GUI. From there, I was able to write the code to detect three-directional movement and blinks, and then mapped each control to the appropriate keyboard key needed to play Tetris (up, left, right, space bar, etc.)
Check out a demo of the game here! Follow the tutorial I made on the examples page of the OpenBCI docs to recreate the game yourself!
In the future, a cool feature would be to incorporate the Concentration Widget to also change the speed of the game depending on how focused the user is, further intensifying the Tetris flow that is commonly associated with the game.
Towards the end of my Tetris project, I was also able to dive a little deeper into the different algorithms that exist to detect blinks. I began to do some preliminary processing work to detect blinks in BrainFlow following papers like this, but the different research that went into blink detection became a continued interest of mine after this project. It was interesting how something that seemed so distinct in the first two channels of the GUI time series had so much research going on behind it to detect. While I did get to play around with some preliminary methods of detecting these peaks with a sample window and detecting a jump over a mean threshold, I am interested in continuing to learn more about different blink detection algorithms and implementing them to see what they reveal about the way that I was processing the data.
You can learn more about other projects I've done and my summer experience at OpenBCI here.