Deep Learning Object Detection using Vision Development Module for LabVIEW

[thuyanhl]

Overview

 This example demonstrates the use of the Model Importer API in the Vision Development Module to perform Object detection feature for Defect Inspection application using Deep Learning.

Description

The example uses a pre-trained model – SSD_MobilenetV1 which is trained in TensorFlow. This model is loaded using the Model Importer VI to detect defects in the images. 

The example has two controls:  

• Select Image Control to browse through different images.  
• Minimum Score Threshold input, which determines which defects to overlay on the image display.

Hardware and Software Requirements

• LabVIEW Full Development System 64-bit 2018 or later 
• Vision Development Module 2018 or later

Steps to Implement or Execute Code

• Run the Deep Learning Object Detection.VI
• Choose a specific image using the Select Image slider control
• Observe the bounding box values for the detected defects and the Score Threshold in the Detected Defects array. This array shows every defect in the selected image.
• Modify the Minimum Score Threshold to select which bounding box to overlay.

Additional Information or References

• Deep Learning VIs: http://zone.ni.com/reference/en-XX/help/370281AE-01/TOC75.htm
• Deep Learning Introduction: http://zone.ni.com/reference/en-XX/help/370281AE-01/nivisionconcepts/deeplearning_introduction/
• Deep Learning FAQs: http://zone.ni.com/reference/en-XX/help/370281AE-01/nivisionconcepts/deeplearning_faq/

Front Panel: 

[D*]

If I read this correctly, the heavy lifting (training, configuration, etc) is done first in Python, then when we want to run the model, we can use this set of .vis to reload that model and classify images.  Is that correct?

To classify is NI Vision running python to do this or is tensorflow being called directly somehow?  Is this any better than trying to do the classification with some labview<->python bridge?

What are the Python, tensorflow, etc versions needed? 

[Abhishekns]

Dear D*,

Yes, tensorflow is used to do the heavy lifting and Labview VIs do the model loading and inference.

The supported tensorflow version is 1.4.  Python is not being used here for execution.

For deployment scenarios labvew<-->python bridge doesn't make life easier for the end user.

 Python version would be whichever you use for your development.

[m_curtin]

Is the code for generating the tensorflow model available along with the training data set in order to help us setup & train our own models?

[AlexZhang]

---

@thuyanhl  已写:

Overview

 This example demonstrates the use of the Model Importer API in the Vision Development Module to perform Object detection feature for Defect Inspection application using Deep Learning.

Description

The example uses a pre-trained model – SSD_MobilenetV1 which is trained in TensorFlow. This model is loaded using the Model Importer VI to detect defects in the images. 

The example has two controls:  

• Select Image Control to browse through different images.  
• Minimum Score Threshold input, which determines which defects to overlay on the image display.

Hardware and Software Requirements

• LabVIEW Full Development System 64-bit 2018 or later 
• Vision Development Module 2018 or later

Steps to Implement or Execute Code

• Run the Deep Learning Object Detection.VI
• Choose a specific image using the Select Image slider control
• Observe the bounding box values for the detected defects and the Score Threshold in the Detected Defects array. This array shows every defect in the selected image.
• Modify the Minimum Score Threshold to select which bounding box to overlay.

Additional Information or References

• Deep Learning VIs: http://zone.ni.com/reference/en-XX/help/370281AE-01/TOC75.htm
• Deep Learning Introduction: http://zone.ni.com/reference/en-XX/help/370281AE-01/nivisionconcepts/deeplearning_introduction/
• Deep Learning FAQs: http://zone.ni.com/reference/en-XX/help/370281AE-01/nivisionconcepts/deeplearning_faq/

Front Panel: 

---

Would you please post out the python code for training the tensorflow model and saving the model for this project?

[M.Hojo]

As mentioned in the previous two posts, we are also very much interested in testing out the python code for generating the tensorflow model and the training data set for setting up and train our own models. Could someone please share the code and the training data set with us?

[Abhishekns]

Tensorflow model zoo being open source the changes may happen in the object detection model's topology and older published models may get out-dated. A good idea would be to follow object_detection/Readme.md and the links mentioned in there. eg. training_pets. 

[X.]

This page is pretty symptomatic of the abysmal level of documentation in LabVIEW nowadays. 

You might think: well, this is just a page to post an example of how to use TensorFlow in LabVIEW, and links are posted for further information.

Think again.

Link 1: list of Deep Learning VIs linking to the terse Help file for each VI (terse as "nothing to be gained from it").

Link 2: Half a page description of the Vision Development Module (it's used to handle images?)

Link 3: Half a page FAQ

If you know TensorFlow, you don't need LabVIEW to use it and in fact you will want to stay away from it in all likelihood.

If you don't know TensorFlow, well, I have really no clue as to what you will gain from this series of VIs.

This is not specific to Deep Learning. In fact, the whole Vision Development module is exactly as undocumented.

It is as if NI had concluded they did not need to bother with documentation (while still charging an arm and a leg for their software).

Sad state of affair, indeed. Luckily, open source software is thriving. A coincidence?

[likangyu]

Hello, thank you very much for your sharing about deep learning object detection. It is very exciting. I I noticed when you used "IMDQ DL Model Detect Objects ", your Input Node Name is "image_tensor", it has only one input node. However, the network I used have two input node, including "image_tensor "and " image_shape_tensor". But the Input Node Name should be a string. How should I modify my labview code? Thanks again！ If it is convenient, could you leave your email address to communicate?

[Esakki]

Hi NI Team,

I'm working on Custom labeled Object detection. so I need to create own TensorFlow frozen graph with a version<1.4. I can able to create a frozen graph with the latest version, But that doesn't support in LabVIEW. If I try with Tensorflow 1.4, I am getting an error. Please help me to create a frozen graph with Tensorflow 1.4.