Zhe Jiang University
Team Members (with year of graduation):
Zhang Ning 2015
Xu Qinhang 2015
Chen Zhe 2015
Prof. Ma Yongchang
This project is a bionic mechanical arm system which can move as the operator’s arm moves. Furthermore, it can also recognize a specified object (appointed by operators in advance) in two ways:real time and by memory.
NI hardware and software:
Vision tool kit
Robotic tool kit
Servo control board(MyRIO)
1 Monitors and captures the operator's arm postures by kinect and then make the arm itself move as the postures by the function of labview(inverse kinematic).
2 Recognize a specified object and move it to a specified place.
3.Keep a series of movement in memory and move as the movements sometime.
4.Get the 3D depth of field for grasping things.
l how does the project work:
The whole procedure can be described as follows:
① recognition with kinect and vision
We use kinect to recognize the skeletal coordinates and object’s position. our mechanical arm by the data of arm joint got from kinect.Kinect for Windows SDK Beta has two important parts:NUI API and Audio API. NUI API obtain data from sensor to control Kinect device。It contains an visit interface connected to computer and two other ones connected to color images and depth data.with which it is capable to track skeleton especially the arm skeletal data. The 3D coordinates is easily got via the Joints attribute in Skeleton. The X,Y,Z position in kinect is somehow a little bit different with ordinary ones. We define the left direction of kinect to be X positive , the upside direction to be the Y positive, to forward direction to be the Z positive.
For the object, Kinect captures the video stream as a common camera and transmit to NI VISION toolkit ,in which we can teach the VI to know what the object is. Then we can find the same object next.Once we lock the aim ,we can get the X、Y coordinates of it.The important step is that we will get the depth of field from Kinect independently.The depth of field is the Z coordinate.As we know the three coordinates ,VI lead the arm to grasp the object.We can grab the aim more accurately in this way.
② serial data transmit
Serial data from Kinect can be transmitted to labview on PC with TCPIP protocol and the one from labview to mechanical with labview VISA.
③ analyse and control with labview
Now we describe the structure and procedure of the VI.
First of all, we transmit the data to labview from kinect .
Secondly, we need make sure the mechanical arm can move the specified positon.this is a robotic problem we are naturally recommended to use the robotic tool offered by NI. Here we mainly make use of robotic tool kit to solve inverse kinematic :Calculate the angles of servos in mechanical arm. Whether we need to analyse the angles in gesture track or object capture,we can analyse in the same way: define the bottom of mechanical arm to be the coordinate origin then calculate the relative XYZ coordinates which can be converted into angels via the robotic tool kit. Meanwhile,we can display the whole procedure in 3D of the mechanical arm in labview.
Thirdly ,to make sure the movement be smooth,we design two modules to realize functions of moving average and pd feedback.
Fourthly, get the object’s position. Primarily we tell the labview which object is the target by draw a regional in the frame. learned the object’s characteristics,labview could recognize and track it when the object moves away. In the meantime , labview calculates the value of XYZ coordinates of the object which is used in robotic tool subsequently.
At last, to memorize a specific movement we storage the component of the movement, “a series of movements” in other words, into a register( a text book). Once we want the system to perform as a specific movement, the system can move as what is stored in the register , whatever the time it need perform is.
④ control the mechanical arm
Given the angles and the fact that servos are driven by character string with format as #3P%d #4 P%d #5 P%d #13 P%d #14 P%d #15 P%d where the number after”#” stands for the servo number and the “%d” means the width of pwm wave which varies from 500 to 2500 . we use a system of conservation formulas to change angles to character string.
l The benefits using labview and NI tool
Labview: Help build up the whole system in a rapid way;
Specific surface makes every module and data flow visual and changeable easily.
Robotic tool kit: Make the inverse kinematic problems easier.
Display the simulation of mechanical arm in 3D window.
Vision tool kit: Teach the labview learn the characteristic quickly and exactly;
Get the position of the specified object swiftly and accurately.
MyRIO: Make the control easier.
l Images with caption\video
Seeing in attachments
l VI code
Seeing in attachments
l Level of completion
l Time to build
l Additional revisions
Seeing in attachments