Machine Vision

cancel
Showing results for 
Search instead for 
Did you mean: 

eye controlled wheelchair

Hello, i want to know how to implement template matching for building a eye controlled wheelchair? i am capturing the image of the eye using webcam and then want to turn the wheelchair to left, right forward etc based on my eye movement to the left, right, up. how do i implement this?

0 Kudos
Message 1 of 17
(7,738 Views)
Hi dheek,
Did you check this before posting??
http://forums.ni.com/t5/Machine-Vision/Eye-controlled-wheelchair/m-p/2531898
Thanks
uday
0 Kudos
Message 2 of 17
(7,736 Views)
yes. I did check. I am having problems in implementing the pattern matching using vision development module.
i am having doubts about what blocks to wire to the vision development module after the pattern is saved.
how do i give controls to the arduino board to make the wheelchaor turn to a specific direction?
0 Kudos
Message 3 of 17
(7,734 Views)

Okay,Coming to vision part i can try, for arduino there is dedicated toolkit available for that and try in forum.
-First, did you try pattern matching example that comes along with labview?
Go to Help>>Find Examples>>Search for Pattern.

 

And I suggest you check out this idea: https://decibel.ni.com/content/docs/DOC-25399

Thanks
uday
Message 4 of 17
(7,734 Views)

Hello,

 

you could also give a go with OpenCV's feature detection algorithms (specifically Haar classifiers for eye detection). There is an example which uses a dll that is called in Labview here:

 

https://decibel.ni.com/content/blogs/kl3m3n/2013/10/08/real-time-face-and-eye-detection-in-labview-u...

 

Best regards,

K

 

 


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
0 Kudos
Message 5 of 17
(7,729 Views)

hello udka !! 

 

i have implemented pattern matching. now how to proceed?? pls helppattern match diag.png

 

0 Kudos
Message 6 of 17
(7,606 Views)

Hello,

 

In my opinion, you will have difficulties in (roboustly, i.e. scaling, illumination invariant, etc..) detecting the eyes using only raw pattern matching. Anyway, when you detect the eyes, you would need to detect the iris also to track its movement. When you narrow the ROI to you eyes, you could probably use circle/ellipse detection.

If you use the source code provided in one of the previous posts (Haar eye classifiers) you can implement for example Hough circle detection in openCV directly and find the circles inside the detected eye ROI. These can be passed to Labview for further analysis.

You can also try using optical flow for iris tracking or the new mean shift algorithm tracking (Vision Module 2013).

 

Also, do you have the controller part (and the power part - the part that will actually drive the wheelchair) already figured out?


You will have more control if you implement the code on Arduino in C++ instead of LIFA (that is just my opinion). There is a great (free) development tool called VisualMicro that can be integrated in the Visual Studio (with intellisense included).

 

For the actual data communication you can go with wireless or direct cable communication (USART). Since the wheelchair is mobile, you will need to have a PC (laptop) attached to it right?In that case, the easiest way is to use serial communication. I would structure the commands for example (for a stepper motor):

 

[direction] [motor speed] [number of steps] [\n] [CRC]

 

where the [CRC] is a simple data check (cyclic redundancy check) to see if the received commad (on the microcontroller side) is equal to the sent command (on the PC side). Also, you would probably need to have a gesture to stop sending the commands to the microcontroller, so that the user can stop (for example, closing the eyes for a couple of seconds stops/starts the control mode, along perhaps with some sound signal - can be controled with the same microcontroller).


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
0 Kudos
Message 7 of 17
(7,593 Views)

hello ,
that was very informative. I am new to Labview. I have been learning it from the online tutorials.
This is my final year project and We are supposed to submit it by march 1.

 

 

Could you help me out with the image processing part? any tutorials on how to implement the haar eye detection etc.. 

 How do we interface the labview and arduino? U said Arduino programming in C++ is better than LIFA. How do i send the corresponding signals to the arduino board without using LIFA? 

we will need to have PC connected to the wheelchair to send the signals. Would you help me with the serial communication part?

how can i implement the last part(that you had mentioned in your reply), to stop sending information to the microcontroller??

0 Kudos
Message 8 of 17
(7,577 Views)
For the image processing part, haar eye detection did you check his link in previous post?
-klemen has implemented the same and you can find the info here https://decibel.ni.com/content/blogs/kl3m3n/2013/10/08/real-time-face-and-eye-detection-in-labview-u...
Thanks
uday
0 Kudos
Message 9 of 17
(7,553 Views)

Hello,

 

I cannot do the project for you, but here is some help if you find it useful:

 

regarding the Vision part, try the link provided in the previous post(s). When you narrow the ROI to the eyes only, find the iris (search for circular objects or threshold in hsv space or .... - a lot of solutions, google them...). You would also need to think about how to interpret the iris motion (head movement???).

Also if no eyes are detected, the function (in the link) returns an error (or rather a value that means no eyes were detected, i think - cannot remember exactly). Use this "to shut-down" your system - for example, if no eyes are detected for 5 seconds stop the eye controlled movement). For turning on the system you could use blink sequence or something similar...?

 

Regarding the communication part, here is a sample C++ code (with comments) that should work with Arduino (below is also the Labview code that should work with it):

 

const uint32_t 	baudRate = 9600; 	// or whatever supported baud rate
const uint16_t 	crc_init = 0xFFFF; 	// crc initialization 

uint16_t 		crc_received;       // received crc

// setup
void setup() {
	// init usart
	Serial.begin(baudRate);
	// clear input buffer
	while(Serial.available() > 0) {
		Serial.read();
	}
}

// main loop
void loop() {
	// check if Rx buffer is available
	if(Serial.available() > 0) {
		// incoming byte (char) over usart
		char incomingByte;
		// read until end line '\n' is received
		while(1) {
			// read one byte form usart
			incomingByte = (char) Serial.read();
			usart_string += incomingByte;
			if(incomingByte == '\n')
				break;
		}
		// initialize CRC with 0xFFFF
		uint16_t crc = crc_init;
		// uint8_t pointer to string begining to calculate CRC
		uint8_t* pD = (uint8_t*) &serial_string[0];
		// calculate CRC
		for(int16_t i=0;i<serial_string.length();i++,pD++) {
			crc = crc_16_update(crc,pD);
		}
		// read two additional bytes from serial port - THE CRC THAT IS SENT FROM PC
		pD = (uint8_t*) &crc_received;
		// first byte
		(*pD) = (uint8_t) Serial.read();
		crc = crc_16_update(crc,pD); 
		pD++;
		// second byte
		(*pD) = (uint8_t) Serial.read();
		crc = crc_16_update(crc,pD); 
		
		// IF CRC IS OKAY
		if(crc == 0) {
			// DO SOMETHING!!!
		}
		// ELSE HANDLE ERROR
		else {
			// handle error
		}		
	}
	// a short delay for processor
	delay(100);
}

// crc calculation update (YOU CAN DO FASTER CALCULATION BY USING LOOKUP TABLES, BUT MORE MEMORY IS USED)
uint16_t crc_16_update(uint16_t crc, uint8_t *data) {		
	crc = crc ^ *data;
	for(int16_t i=0;i<8;i++) {
		if(crc & 0001)
			crc = (crc >> 1) ^ 0xA001;
		else
			crc = (crc >> 1);
	}
	return crc;
}

 

and LV code:

 

La_BD.png

 

Both codes are attached (Labview code is saved for 2013).

 

I hope this gets you started.

 

Best regards,

K


https://decibel.ni.com/content/blogs/kl3m3n



"Kudos: Users may give one another Kudos on the forums for posts that they found particularly helpful or insightful."
Download All
Message 10 of 17
(7,496 Views)