Home Community Example Programs Example Code Document

Example Code

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
ASK A QUESTION

Integrating Android Devices with NI Vision Software

by: Community Team
‎08-10-2012 10:26 AM
Last Edited by: Community Team ‎12-03-2019 02:06 AM

Code and Documents

Attachment

Overview


This example program will demonstrate how it is possible to integrate your Android Device into a Vision System using NI Vision and LabVIEW. The example includes: 1. Interfacing your Android camera with NI-IMAQdx and LabVIEW, 2. Acquiring and Processing images in real time from your Android device, 3. Broadcasting your processed images as an MJPEG stream, 4. Receiving and displaying that MJPEG stream on your Android device.

Project Outline

General Requirements

The following application was tested using a Samsung Galaxy S2. It is not confirmed to work on any other Android device (but there is no immediate reason it shouldn't). The example requires the following:

  • LabVIEW 2011 SP1
  • Vision Acquisition Software (including NI-IMAQdx) 2011
  • Vision Development Module 2011
  • .NET 3.5

Third Party Software

There are also a series of third party tools used which will require installation on both your Android device and your Windows PC (nothing is tested in Linux and is unlikely to work, but individual components could be replaced by their equivalent components in other OSs to adapt the system).

On your Android Device you will need:

On your PC you will need (in addition to the software mentioned above):

The C# component to this application was started by Ragheed Al-Tayeb in his article Motion JPEG Streaming Server on The Code Project.

The C# component was modified to help prevent Orphaned Sockets and to include an interface for LabVIEW. The NI Vision Image to .NET Bimap conversion algorithm is based on the approach used by GriffinRU in From VisionImage to BITMAP (.NET).

System Architecture

The processing involved in the application is arbitrary and may be replaced by you own application.

The structure of the system is displayed in the image below:

Integrating Your Android Camera

There are many apps capable of broadcasting the output from the camera on your Android phone as an MJPEG stream. One such app is the IP Webcam app by PAS. 

IP Webcam

This application is capable of broadcasting in the background, on startup, with a password, and with varied quality and resolution. Download this application and, once you're finished configuring the broadcast, select Start server.

Your camera is now broadcasting an MJPEG stream via its IP Address. You will need to be connected to Wifi to have an IP Address which is visible to another machine. Most cell phone providers will not provide you with the capabilities to broadcast over their mobile networks (data usage rates would apply if you figured out how to do it). The address of your camera will show up on the bottom of your video feed. You can type this address into a web browser to access the camera.

In order for the camera to show up in Measurment and Automation Explorer, you'll need to interface with the MJPEG stream. Although there are other methods of interfacing with an MJPEG stream, one very simple method is to leverage Virtual Webcam software which emulates the functionality of a USB webcam. One such piece of software is the IP Camera Adapter. Once this software is installed, it will be capable of emulating your MJPEG stream as a DirectShow device. This will allow the IMAQdx drivers to use the camera and, therefore, allow you to interface the camera with LabVIEW and NI Vision.

IP Camera Adapter

You can download the IP Camera Adapter from here. Once installed, you can navigate the the Configure IP Camera Adapter shortcut in your start menu. You will see a screen like this:

You can use this screen to configure the location of the camera feed, username, password and resolution. Upon clicking OK you will be prompted that you must restart any applications using the camera for the changes to take effect. Restart MAX and you will see the camera appear under the IMAQdx part of the Hardware tree. You are now seeing a live feed from your Android phone to your computer.

Processing

The processing step is largely arbitrary. We perform the following steps:

  1. Extra the Intensity Color Plane
  2. Perform an Edge Detection Filter
  3. Threshold
  4. Particle Filter
  5. Particle Analysis
  6. Overlay Bounding Boxes
  7. Merge Overlay (so our external device can see it)

MJPEG Server

The code here is very simple and leverages the previous referenced example code. The C# solution (VS2010) is included in case bugs are found or modifications should be made. The VI package includes the following functions:

  • Dispose MJPEG Server - Closes the server
  • Init MJPEG Server - Starts the server on the designated port
  • Is MJPEG Server Running - Checks to see if the server is running
  • Send Image to MJPEG Server - Sends an image to the server for broadcasting

Additionally two utility functions are included:

  • IMAQ Image To NET Bitmap - Converts an IMAQ Image to a .NET Bitmap
  • Working Bitmap FGV - A functional global variable maintaining the previous Bitmap references for disposal

The example code shows how all of this fits together, but it follows the typical Open, Act, Close paradigm.

Reading From Android

The MJPEG Viewer application we installed at the beginning is now going to come in handy! I have had problems with using the function natively so I recommend opening the URL in the browser and then selecting the app. You will then see your post processed image. This has also been tested in Google Chrome (although it is just a still frame then), and in VLC Media Player. You can open simultaneous connections on multiple devices but it will likely reduce performance unless your host is very capable.

Conclusions and Future Work

This application took a little longer to make than this article took to write. LabVIEW is an incredible platform for prototyping and there are many great tools out there which can be integrated to make it even better. This application was developed in 4 hours and on a Friday so I provide no guarantees that it will be without bugs, but it should show the basic principle involved with minimal effort. The IP Camera application has many many more features which could be integrated with the application should the user wish it. It provides controls which could be sent to the camera (turn on the light, change focus, etc) which might be of interest to the user (see the screenshot above).

 

Happy Processing!

 

Example code from the Example Code Exchange in the NI Community is licensed with the MIT license.

0 Kudos
Contributors