myWingman: Gesture Controlled Drone

[SarahMC]

myWingMan is a quadrotor drone, which is controlled by the movement of your hands. This novel human-machine-interface feels like you have the telekinetic capabilities of a Jedi knight! The entire system was built with low-cost, commercial tech. Although these unrelated technologies were never intended to work together, LabVIEW allowed me to combine into a single, coherent system.

Project Summary.

myWingman is a Parrot AR Drone controlled with a Leap Motion sensor, with LabVIEW providing the interface between the two. The Leap sensor detects hand movements, and this then gets translated by LabVIEW into movement by the drone. Along with the direct video feed from the drone to the LabVIEW front panel, this is a completely new and innovative way to fly a drone.

 The Hardware.

Unlike many innovative drone projects, the hardware used here is all widely available and affordable:

Parrot AR Drone 2.0- I chose to use this drone due to it already having an API built within LabVIEW, which would make the integration of hardware and software cheaper. This drone is used within a lot of maker projects, as it gives its own wifi connection, which makes it very easy to run custom code on, and is also incredibly easy to fly and control.

Parrot Website - £229- https://www.parrot.com/uk/drones/parrot-ARDrone-20-elite-%C3%A9dition#parrot-ardrone-20-elite-editio...

Leap Motion Sensor- This innovative piece of technology is able to plug into your computer to allow you to control your applications using hand movements. It is also becoming widely used as a way to control other pieces of hardware, and it is easy to interface with software such as LabVIEW

Leap Website - £55- https://store-eur.leapmotion.com/products/leap-motion-controller

A laptop is also required to ensure portability. 

The Software.

• LabVIEW 2016 (or above)
• AR Drone Toolkit - Although there was a readily available toolkit within VI Package Manager, this did not have the functionality I needed in terms of the video. Therefore I edited the existing API to create my own. Details on how to install this are located in the 'Steps' section
• Leap Motion Toolkit - Available from VI Package Manager

 

Steps to Run Project.

1. Ensure LabVIEW 2016 (32 bit) is installed

2. Install the Leap Motion interface for LabVIEW https://www.labviewmakerhub.com/doku.php?id=libraries:leap:start

3.

• a) Download the attachment AR Drone Toolkit
• b) Navigate to the following path- C:\Program Files (x86)\National Instruments\LabVIEW 2016\vi.lib
• c) Create a folder called LVH if it does not already exist
• d) Unzip the AR Drone Toolkit into the LVH folder

4. Install Leap drivers from https://www.leapmotion.com/setup/desktop/windows

5. Plug Leap device into computer and test using Leap software.

6. Download Drone Project.zip, and unzip onto a local drive on your computer- for example in My Documents

7. Open the Drone project.lvproj file

8. Within the LabVIEW project open Quadrotor.vi and Video.vi

9. Turn on drone and connect to the drone's wifi from the laptop

10. Press run on Quadrotor. vi, then press run on Video.vi

11. Have fun flying your drone!

Future Innovations.

I would really like to implement more "animations" (super-cool flight maneuvers), as well as the ability to record the video streams from the Drones built-in camera. Both of these features will be easy to implement in LabVIEW, I just need to find the time!

I would also hope to give the drone an 'autonomous mode', with a tracking feature. Using the video streams from the Drone camera, I would be able to teach it to recognise a certain shape, pattern or person and then use PID control keep the drone at a defined distance from the object of interest. Again all of this machine vision and control functionality is available in LabVIEW.

It would be like have a little ariel buddy, following you around! Combined with the video record functionality, this could be a very interesting tool... imagine it following you down a ski slope, recording your awesome run!?

About the Developer.

My name is Sarah, and I am an Applications Engineering Intern at National Instruments. I am currently studying for a degree in Systems Engineering at Loughborough University. I have always enjoyed finding out about cool tech projects that people have been working on, and through the use of National Instruments tools and support, have been able to create my own!