12-05-2013 08:31 AM
Hello, i want to know how to implement template matching for building a eye controlled wheelchair? i am capturing the image of the eye using webcam and then want to turn the wheelchair to left, right forward etc based on my eye movement to the left, right, up. how do i implement this?
12-05-2013 09:11 AM
12-05-2013 09:52 AM
12-05-2013 10:03 AM - edited 12-05-2013 10:11 AM
Okay,Coming to vision part i can try, for arduino there is dedicated toolkit available for that and try in forum.
-First, did you try pattern matching example that comes along with labview?
Go to Help>>Find Examples>>Search for Pattern.
And I suggest you check out this idea: https://decibel.ni.com/content/docs/DOC-25399
12-05-2013 01:44 PM
Hello,
you could also give a go with OpenCV's feature detection algorithms (specifically Haar classifiers for eye detection). There is an example which uses a dll that is called in Labview here:
Best regards,
K
01-06-2014 09:09 PM
hello udka !!
i have implemented pattern matching. now how to proceed?? pls help
01-07-2014 01:37 AM
Hello,
In my opinion, you will have difficulties in (roboustly, i.e. scaling, illumination invariant, etc..) detecting the eyes using only raw pattern matching. Anyway, when you detect the eyes, you would need to detect the iris also to track its movement. When you narrow the ROI to you eyes, you could probably use circle/ellipse detection.
If you use the source code provided in one of the previous posts (Haar eye classifiers) you can implement for example Hough circle detection in openCV directly and find the circles inside the detected eye ROI. These can be passed to Labview for further analysis.
You can also try using optical flow for iris tracking or the new mean shift algorithm tracking (Vision Module 2013).
Also, do you have the controller part (and the power part - the part that will actually drive the wheelchair) already figured out?
You will have more control if you implement the code on Arduino in C++ instead of LIFA (that is just my opinion). There is a great (free) development tool called VisualMicro that can be integrated in the Visual Studio (with intellisense included).
For the actual data communication you can go with wireless or direct cable communication (USART). Since the wheelchair is mobile, you will need to have a PC (laptop) attached to it right?In that case, the easiest way is to use serial communication. I would structure the commands for example (for a stepper motor):
[direction] [motor speed] [number of steps] [\n] [CRC]
where the [CRC] is a simple data check (cyclic redundancy check) to see if the received commad (on the microcontroller side) is equal to the sent command (on the PC side). Also, you would probably need to have a gesture to stop sending the commands to the microcontroller, so that the user can stop (for example, closing the eyes for a couple of seconds stops/starts the control mode, along perhaps with some sound signal - can be controled with the same microcontroller).
01-07-2014 08:13 AM
hello ,
that was very informative. I am new to Labview. I have been learning it from the online tutorials.
This is my final year project and We are supposed to submit it by march 1.
Could you help me out with the image processing part? any tutorials on how to implement the haar eye detection etc..
How do we interface the labview and arduino? U said Arduino programming in C++ is better than LIFA. How do i send the corresponding signals to the arduino board without using LIFA?
we will need to have PC connected to the wheelchair to send the signals. Would you help me with the serial communication part?
how can i implement the last part(that you had mentioned in your reply), to stop sending information to the microcontroller??
01-07-2014 10:30 PM
01-09-2014 04:47 AM
Hello,
I cannot do the project for you, but here is some help if you find it useful:
regarding the Vision part, try the link provided in the previous post(s). When you narrow the ROI to the eyes only, find the iris (search for circular objects or threshold in hsv space or .... - a lot of solutions, google them...). You would also need to think about how to interpret the iris motion (head movement???).
Also if no eyes are detected, the function (in the link) returns an error (or rather a value that means no eyes were detected, i think - cannot remember exactly). Use this "to shut-down" your system - for example, if no eyes are detected for 5 seconds stop the eye controlled movement). For turning on the system you could use blink sequence or something similar...?
Regarding the communication part, here is a sample C++ code (with comments) that should work with Arduino (below is also the Labview code that should work with it):
const uint32_t baudRate = 9600; // or whatever supported baud rate const uint16_t crc_init = 0xFFFF; // crc initialization uint16_t crc_received; // received crc // setup void setup() { // init usart Serial.begin(baudRate); // clear input buffer while(Serial.available() > 0) { Serial.read(); } } // main loop void loop() { // check if Rx buffer is available if(Serial.available() > 0) { // incoming byte (char) over usart char incomingByte; // read until end line '\n' is received while(1) { // read one byte form usart incomingByte = (char) Serial.read(); usart_string += incomingByte; if(incomingByte == '\n') break; } // initialize CRC with 0xFFFF uint16_t crc = crc_init; // uint8_t pointer to string begining to calculate CRC uint8_t* pD = (uint8_t*) &serial_string[0]; // calculate CRC for(int16_t i=0;i<serial_string.length();i++,pD++) { crc = crc_16_update(crc,pD); } // read two additional bytes from serial port - THE CRC THAT IS SENT FROM PC pD = (uint8_t*) &crc_received; // first byte (*pD) = (uint8_t) Serial.read(); crc = crc_16_update(crc,pD); pD++; // second byte (*pD) = (uint8_t) Serial.read(); crc = crc_16_update(crc,pD); // IF CRC IS OKAY if(crc == 0) { // DO SOMETHING!!! } // ELSE HANDLE ERROR else { // handle error } } // a short delay for processor delay(100); } // crc calculation update (YOU CAN DO FASTER CALCULATION BY USING LOOKUP TABLES, BUT MORE MEMORY IS USED) uint16_t crc_16_update(uint16_t crc, uint8_t *data) { crc = crc ^ *data; for(int16_t i=0;i<8;i++) { if(crc & 0001) crc = (crc >> 1) ^ 0xA001; else crc = (crc >> 1); } return crc; }
and LV code:
Both codes are attached (Labview code is saved for 2013).
I hope this gets you started.
Best regards,
K