The system consists of an intuitive way in which a locomotion conveying device such as a robot can be controlled by virtually no physically at all.
Such a setup would not only facilitate below neck paralyzed patients to control hardware such as electronic wheelchairs or robotic arms, but also enable any fully abled person to control systems telepathically without lifting a finger. In the long run, such a system can be coupled with existing manual control systems to make them more efficient and safe. Neurofeedback to the individual in the form of visual or haptic stimulus would be given so that they can control their brainwave output more easily and enable a more efficient control system.
Summary of Contributions:
2. BCI control system
The control layout employed in this paper consists of four main features: The BCI, the signal processing system, the control output system and the visual Neurofeelback system.
The BCI used here, is the NeuroSky MindWave Brainwave Headset. It extracts the raw EEG data from the scalp in an noninvasive manner and transmits it via Bluetooth to the processing unit. The device consists of a headset, an ear clip, and a sensor arm. The headsets reference and ground electrodes are on the ear clip and the EEG electrode is on the sensor arm, resting on the forehead above the eye.
The most unique aspect on this particular BCI as compared to others available in the market is that it incorporates dry-sensors for its electrodes without the need for any conductive gels to be applied on the skin and was the paramount criteria for the selection of this equipment over others.
The single sensor on FP1 site (above left eyebrow) provides a high degree of freedom; it can measure multiple mental states simultaneously, and its close proximity to the eye enable it to detect Electromyography (EMG) signals created by eye blinks.
This enables us to process various variables that can be formulated into a control algorithm in the processing unit from which an efficient control system can be derived.
2.1 Communication of BCI
The Neurosky Mindwave BCI sends brainwave and debugging information wirelessly through Bluetooth communication. The device creates a virtual communications port (COM port) and sends data serially by serial over Bluetooth communication protocol.
The Think Gear Communications Driver (TGCD) is a device driver with a simple API that allows
communication between an Application on a computer and a Think Gear chip (the ASIC chip embedded inside the Mindwave headset).It is available as a .dll Using the documentation provided by Neurosky along with the driver it is possible to create standalone applications on a computer or mobile platforms.
2.2 Hardware-Application Interface and Debugging
The use of LabVIEW graphical programming environment enables for a platform for easy development of complex features along with error handling capabilities as well as to ability to build an intuitive and interactive interface with the user.
LabVIEW communicates with the BCI using the TGCD driver with the ability to receive data pertaining to signal strength as well as error information from the headset and display it on the computer screen.
Fig 2.2 shows simple error handling VI’s to stop the program and output error information to the screen in the case of an error before or during execution of the program.
This enables for easy and fast debugging, allowing for a user friendly interface.
Fig2.1 Error Handling in the VI
Processing Raw EEG
LabVIEW is graphical programming environment. Programs in IDE are called Virtual Instruments (VIs), consists of a Block Diagram (BD) and a Front Panel (FP). A BD provides a graphical code development environment whereas a FP allows the user to interact with a VI. It provides an efficient and easy-to-use environment for code development especially when the user needs to interact with the program and visualize the results. Unlike text-based programming languages like C which follow a control flow execution model, the environment of programming follows a dataflow execution model.
After an extensive and exceedingly informative session with Dr. Praveen Kumar of Tranquil Minds Psychiatric clinic of SIMHANS (Samhita Institute of Mental Health and Neuro Sciences Pvt. Ltd.) in Hyderabad, one comes to understand that Scalp EEG picked up from sensors placed on the forehead is in actuality a summation of brainwaves of different frequencies and artifacts such muscle activity, eye blinks, pulse signals and line noise. Extracting useful artifacts from raw EEG signal and transforming it into a useful control output is done by processing it in LabVEW. The FP provides for a useful tool for providing Neurofeedback to the user.
The input raw EEG as mentioned above also includes the EOG, which basically can be termed as wavelets with larger amplitudes due to muscle related movements in the ocular region. Since the acquisition is done in real time, thresholding the amplitudes pertaining to the eye blinks and counting the time intervals between consecutive eye blinks has worked efficiently in counting two and three consecutive eye blinks and also eliminating the effect of normal eye blinks. The following part of the block diagram describes the analysis of raw data, thresholding, and outputting the number of blinks on visual feedback which is further utilized in controlling the device.
Blinking causes a spike in the EEG of appreciable magnitude as compared to base signal without the blink artifact. This enables us to detect eye blinks by thresholding the EEG signal.
Also, the time difference between consecutive eye blinks can also be found out with simple programming instructions involving timed looping statements and shift registers to send data between loops in LabVIEW. A clear momentary spike can be seen in fig 3.1 where a blink occurred as opposed to a clear EEG waveform with no physical movements occurring.
Fig3.1 blink occurring in the second waveform chart.
When a single blink of magnitude more than that set by a certain threshold occurs, a counter starts counting the number of milliseconds before another blink occurs. If the blinks occur within a certain time limit, it is counted as a double blink. In a similar manner, triple blinks are also detected.
Experimentally, it has been found that voluntary blinking produces a deflection of magnitude 400 micro volts and above, thus this is used as a threshold for blink detection.
Fig3.2 Thresholding blink intensity
Therefore, essentially, only forced voluntary blinks are detected and natural blinks are overlooked. Fig 3.3 shows the blink detecting code for double and triple blinks.
Fig3.3 Double and triple blink detection VI
3.2 Attention Level Separation and Mapping
As mentioned above, raw EEG data is essentially a summation of various brain wave oscillations and artifacts. Some oscillations of notable importance are as follows:
1. Alpha waves: Frequency ranging from 3- 13 Hz and corresponds to meditation levels in the brain activity.
2. Beta waves: Frequencies ranging from 12 – 30 Hz and these beta states are the states corresponding to normal waking consciousness.
3. Theta waves: 4 hz- 7 Hz.
4. Delta waves: Frequencies less than 3 Hz.
The beta wave in particular can be attributed to the “concentration” part of the brain activity and this is what is being used to control the output magnitude. The beta wave magnitude is deduced by first building a waveform 450 samples long by inserting samples sent by the BCI point-by point into an array and then using an elliptic bandpass filter and the power spectrums are respectively calculated after using FFT on the input signal as shown in fig.3.4. Similarly meditation which corresponds to alpha wave intensity can also be deduced. The power spectrum for alpha wave is shown in fig 3.5
Fig3.4 Beta wave intensity extraction
Fig3.5 power spectrum output for Alpha wave(meditation)
The power spectrum is put on a 0 to 100 numeric scale. This scaling is then converted to a 0 to 255 (8 bit resolution) scale since the control output magnitude output needs an 8 bit input to function properly. This is done by the equation 3.1
(x - in_min)*(out_max - out_min / in_max - in_min)+out_min (3.1)
Where x is the value to be scaled, in_min and out_min are the minimum input and output values respectively and in_max and out_max Are the maximum input and output values respectively.
On the 0-100 scale, a beta level reading of 40-60 is considered normal. Control output should only be sent to the actuators for elated values of attention or concentration, thus a numeric threshold is compared with the attention level, above which, the final control output would be sent.
3.3 Nurofeedback through LabVIEW FrontPanel
Neurofeedback, also called biofeedback, is a treatment where EEG equipment is used to present real-time records of brain activity. This enables to training yourself, for instance to become more concentrated or relaxed, by altering your brain wave patterns over time. Feedback about your concentration level is for instance given as a graph, illustrated in fig 3.6
Fig3.6 NeuroFeedback using a blue bar graph
The subjects task is usually to keep the bar above a certain threshold, indicated here by the horizontal black line. Typically one training sessions like this last for about 45-60 minutes and needs to be repeated to get the wanted effect, variating from 25-50 times depending on the patient (Heinrich, Gevensleven, & Strehl,
LabVIEW FP provides for an excellent way to implement neurofeedback. Along with attention and meditation levels, EOG reading, raw EEG reading along with the control output selection can be outputted in a user friendly way. Fig 3.7 shows the final LabVIEW FP implemented.
Fig3.7 NeuroFeedback using a LabVIEW FrontPanel
Control Output Generation
The multiple blink information and attention magnitude data is used for the control output. The system is set up to control motion in a 2-D axis where double blinks are used to cycle between forward, left and right. Triple blink is for stoping the output. A integer is assigned for each direction. The magnitude of motion is decided by the attention levels in the currently selected direction.
The direction and magnitude information in concatenated into a string with the first character pertaining to the direction and three characters following pertaining to the magnitude of movement in 8 bit resolution. This information is sent to a Arduino microcontroller board serially. The code fragment can be seen in fig 4.1.
Fig3.7 Send concatenated string to Arduino
The arduino to which the VI is sending information to serially is connected to a RF transmitter that transmits the concatenated string to the robot to be controlled. The code running on the robot separated the two parts of the string and using switch-case statements is able to drive the robot in a differential drive manner. Allowing for easy wireless control.
We would like to thank our friends and family for helping us fund this project and supporting us along the way. We would also like to thank Dr. Praveen Kumar of Tranquil Minds Psychiatric clinic of SIMHANS (Samhita Institute of Mental Health and Neuro Sciences Pvt. Ltd.) for taking the time and explaining us the various aspects on a BCI and also introducing us to the possibility of this project.
 Heinrich, H., Gevensleven, H., & Strehl, U. (2007). Annotation: Neurofeedback -train your brain to train behaviour. Journal of Child Psychology and Psychiatry,48:1, 3–16.
 Larsen, E. A. (2010). Playstation Controlled by Brain Waves (Tech. Rep.). Trondheim.
 Mitsar. (2008). BIOFEEDBACK TRAINER MITSAR-BFB. pdf available at http://www.mitsar-medical.com/download/promo/Mitsar-BFB 2008.pdf.
 Mitsar. (2010). Mitsar Co. Ltd. Medical equipment for functional diagnostics and neurofeedback. Retrieved from http://www.mitsar-medical.com/.
 Neurosky. (n.d.). Neurosky. Available from http://www.neurosky.com/
 NeuroSky, I. A. (2010). ThinkGear Socket Protocol (Tech. Rep.). Available from http://www.neurosky.com
 Reuderink, B. (2008b). Games and brain-computer interfaces: The state of the art. WP2
 Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wolpaw,
J. R. (2004, June). BCI2000: a general-purpose braincomputer
interface (BCI) system. (Vol. 51) (No. 6). Available from
 SmartBrain. (n.d.). Biofeedback. Retrieved december 2010, from
 S. Delsanto, F. Lamberti and B. Montrucchio, "Automatic ocular artifact rejection based on independent component analysis and eyeblink detection," IEEE EMBS Conference on Neural Engineering, pp. 309-312, 2003.
 I.Daubechies, Ten Lectures on Wavelets, New York: SIAM, 1992.
 V. J. Samar, A. Bopardikar, R. Rao, and K. Swartz, "Wavelet analysis of neuroelectric waveforms: a conceptual tutorial," Brain and language, vol. 66, pp. 7- 60, Jan 1999.
 V. Krishnaveni, S. Jayaraman, L. Anitha and K. Ramadoss, "Removal of ocular artifacts from EEG using adaptive thresholding of wavelet coefficients," J. Neural Eng., vol. 3, pp. 338-346, Dec. 2006.
 Croft RJ, Barry RJ “Removal of ocular artifact from the EEG: a review” Clinical Neurophysiology, 30(1), pp 5-19, 2000
.  S.Venkata Ramanan, J.S.Sahambi, N.V.Kalpakam , “A Novel Wavelet Based Technique for Detection and De-Noising of Ocular Artifact in Normal and Epileptic Electroencephalogram” BICS 2004.
 Micheal, Arduino Cookbook, 3rd ed., vol. 2, pp.68-73.