General Academic Projects

cancel
Showing results for 
Search instead for 
Did you mean: 

The bionic mechanical arm system based on Kinect and LABVIEW

Contact Information

University:

Zhe Jiang University

Team Members (with year of graduation):

Zhang Ning               2015

Xu Qinhang              2015

Chen Zhe                  2015

Faculty Advisers:            

Prof. Ma Yongchang

Email Address:

supermancanfly@foxmail.com

Submission Language:

English

Project Information

Description:

This project is a bionic mechanical arm system which can move as the operator’s arm moves. Furthermore, it can also recognize a specified object (appointed by operators in advance) in two ways:real time and by memory.

Products:

NI hardware and software:

Labview 2013

Visa

Vision tool kit

Robotic tool kit

Other hardware:

Kinect

Mechanical arm

Servo control board(MyRIO)

The challenge:

1 Monitors and captures the operator's arm postures by kinect and then make the arm itself move as the postures by the function of labview(inverse kinematic).

2 Recognize a specified object and move it to a specified place.

3.Keep a series of movement in memory and move as the movements sometime.

4.Get the 3D depth of field for grasping things.

The solution:

l how does the project work:

The whole procedure can be described as follows:

1.png

2.png

① recognition with kinect and vision

We use kinect to recognize the skeletal coordinates and object’s position. our mechanical arm by the data of arm joint got from kinect.Kinect for Windows SDK Beta has two important parts:NUI API and Audio API. NUI API obtain data from sensor to control Kinect device。It contains an visit interface connected to computer and two other ones connected to color images and depth data.with which it is capable to track skeleton especially the arm skeletal data. The 3D coordinates is easily got via the Joints attribute in Skeleton. The X,Y,Z position in kinect is somehow a little bit different with ordinary ones. We define the left direction of kinect to be X positive , the upside direction to be the Y positive, to forward direction to be the Z positive.

3.png

For the object, Kinect captures the video stream as a common camera and transmit to NI VISION toolkit ,in which we can teach the VI to know what the object is. Then we can find the same object next.Once we lock the aim ,we can get the X、Y coordinates of it.The important step is that we will get the depth of field from Kinect independently.The depth of field is the Z coordinate.As we know the three coordinates ,VI lead the arm to grasp the object.We can grab the aim more accurately in this way.

4.png

② serial data transmit

Serial data from Kinect can be transmitted to labview on PC with TCPIP protocol and the one from labview to mechanical with labview VISA.

③ analyse and control with labview

Now we describe the structure and procedure of the VI.

First of all, we transmit the data to labview from kinect .

Secondly, we need make sure the mechanical arm can move the specified positon.this is a robotic problem we are naturally recommended to use the robotic tool offered by NI. Here we mainly make use of robotic tool kit to solve inverse kinematic :Calculate the angles of servos in mechanical arm. Whether we need to analyse the angles in gesture track or object capture,we can analyse in the same way: define the bottom of mechanical arm to be the coordinate origin then calculate the relative XYZ coordinates which can be converted into angels via the robotic tool kit.  Meanwhile,we can display the whole procedure in 3D of the mechanical arm in labview.

Thirdly ,to make sure the movement be smooth,we design two modules to realize functions of  moving average and pd feedback.

Fourthly, get the object’s position. Primarily we tell the labview which object is the target by draw a regional in the frame. learned the object’s characteristics,labview could recognize and track it when the object moves away. In the meantime , labview calculates the value of XYZ coordinates of the object which is used in robotic tool subsequently.

At last, to memorize a specific movement we storage the component of the movement, “a series of movements” in other words, into a register( a text book). Once we want the system to perform as a specific movement, the system can move as what is stored in the register , whatever the time it need perform is.

5.png

④ control the mechanical arm

Given the angles and the fact that servos are driven by character string with format as #3P%d #4 P%d #5 P%d #13 P%d #14 P%d #15 P%d where the number after”#” stands for the servo number and the “%d” means the width of pwm wave which varies from 500 to 2500 . we use a system of conservation formulas to change angles to character string.

6.png

l The benefits using labview and NI tool

Labview:       Help build up the whole system in a rapid way;

               Specific surface makes every module and data flow visual and changeable easily.

Robotic tool kit:  Make the inverse kinematic problems easier.

                 Display the simulation of mechanical arm in 3D window.

Vision tool kit: Teach the labview learn the characteristic quickly and exactly;

                 Get the position of the specified object swiftly and accurately.

MyRIO:  Make the control easier.

l Images with caption\video

Seeing in attachments

l VI code

Seeing in attachments

l Level of completion

Fully functional

l Time to build

Two Month

l Additional revisions

Seeing in attachments

Comments
trogen
Member
Member
on

could you pleasse provide me the  vi code of your project?

dynamicsuperman
Member
Member
on

Sure :),are you an officer of NI or student ?

drummer125
Member
Member
on

Could you provide me the vi code of your project? please, i am a student.

selta
Member
Member
on

it´s possible to provide me the vi code ? I´m a student.:D

allenroboticwor
Member
Member
on

I want to see the code and the connection from the arm to the myRIO device.

Sara_snail
Member
Member
on

I WOULD LIKE TO SEE YOUR CODE, CAN YOU SEND THAT TO saravanaprashanth@outlook.com

joanacristina
Member
Member
on

could you pleasse provide me the  vi code of your project? joana.cristina_s@hotmail.com

Nacef
Member
Member
on

i'm working in the same project in my university , can you send me the vi please?

Contributors