From Friday, April 19th (11:00 PM CDT) through Saturday, April 20th (2:00 PM CDT), 2024, ni.com will undergo system upgrades that may result in temporary service interruption.

We appreciate your patience as we improve our online experience.

LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Creating a .dll in Visual Studio 2015 to pass an image through some OPENCV filters in Labview?

I need to apply a gaussian blur, adaptive threshold, and median blur all in real time.  

 

I looked at this:

 

http://www.ni.com/white-paper/3056/en/

 

which I can open in visual studio and modify parameters and re-export the .dll to use with the accompanying VI, but since I need to pass an image that means I'll have to send either a cluster or array, right?

 

So I looked at the example "External Code (DLL) Execution" VI and looked at the "2-D Array" code, but the source code it shows is the source of the function that squares it, and not how the "ARRAY2D" header (?) works. I think. I'm very unfamiliar with C++.

 

 

Does anyone have any sugestions on other resources? Currently I'm using the "Python Integration Toolkit" and calling python to call opencv, simply because I could figure it out relatively easily. But that drops the camera fps by about 50%.

 

Any help / teaching would be greatly appreciated! 

 

For reference, here is my entire Python ".py" file:

 

 

import numpy as np
import cv2

    
def send_photox2(image):
    im2 = np.array(image, dtype = np.uint8)
    im2 = cv2.GaussianBlur(im2, (5, 5), 0 )
    im2 = cv2.adaptiveThreshold(im2,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,cv2.THRESH_BINARY,25,-1)
    im2 = cv2.medianBlur(im2, 7)
    return im2, image

Changing it to only return im2 instead of both didn't affect fps, and removing the processing only added about 1fps regardless of framerate. (We alter the resolution a lot which makes our framerate range from 20fps to 1500fps.)

 

Thanks!

0 Kudos
Message 1 of 6
(7,105 Views)

I've had some luck, and hav figured out how to pass arrays through my C++ code!

 

However, I don't see how to convert the pixel arrays from "IMAQ Imagetoarray" into something that the OPENCV filters can process. Any input?

0 Kudos
Message 2 of 6
(7,073 Views)

Hi idmb,

 

Have you tried to use some of the examples for this?

Also check this out:

 

NI Vision OpenCV Support - February 2016
http://digital.ni.com/manuals.nsf/websearch/392ECD0BC63AB87E86257F72006F7833

 

LabVIEW Datatypes to .NET reference - Discussion Forums
http://forums.ni.com/t5/LabVIEW/LabVIEW-Datatypes-to-NET-reference/td-p/2551719

 

OpenCV 2.4.6 Wrapper for LabVIEW - YouTube
https://www.youtube.com/watch?v=aahlv6p3z2E

 

Regards,

Daniel C.

Message 3 of 6
(7,050 Views)

Thanks I've had some luck with those! I'm now stuck on how to return an array of varying size from C++ to Labview?

It appears the array I am making in C++ (or rather, an OPENCV "Vector") is being cleared from Memory upon execution of the .dll?

 

I was looking at this:

 

https://decibel.ni.com/content/docs/DOC-9091#Special_Case_Dereferencing_Arrays

 

and it seems to be returning different values (things like "1.27946E-307" and "-1.45682E+144") whilst the code in my .dll right now is just:

 

 

#include "math.h"
#include <opencv2/opencv.hpp>		


#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
using namespace std;
using namespace cv;



extern "C" __declspec(dllexport) double * __cdecl Reimage(int *myarray, int array_length_row, int array_length_col);




__declspec(dllexport) double * __cdecl Reimage(int *myarray, int array_length_row, int array_length_col)
{
	vector<double> xy;
	xy.push_back(250);
	xy.push_back(155);
	return xy.data(); 
	
}

so I would expect to get back a 1d array with values 250 and 155 in the 0 and 1 spots respectively?

 

 

In the end my goal for the code will be something like 

Mat m(array_length_col, array_length_row, CV_8U, myarray);
	
	GaussianBlur(m, m, Size(5, 5),0,0,0);
	adaptiveThreshold(m, m, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 25, -1);
	medianBlur(m, m, 7);
	vector<vector<Point> > cnt;
	vector<double> xy;	
	findContours(m, cnt, CV_RETR_TREE, CHAIN_APPROX_SIMPLE, Point(0,0));
	//findContours(m, cnt, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
	for (int i = 0; i < cnt.size(); i++)
	{
		RotatedRect mar = minAreaRect(cnt[i]);
		xy.push_back(mar.center.x);
		xy.push_back(mar.center.y);
		
	}
return xy.data();

 

so as you can see the size of xy will change for every image. Is it possible to create the array in the .dll and output it to Labview before clearing it from memory?

 

Thanks for the help!

0 Kudos
Message 4 of 6
(7,041 Views)

@idmb wrote:

  

#include "math.h"
#include <opencv2/opencv.hpp>		


#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
using namespace std;
using namespace cv;



extern "C" __declspec(dllexport) double * __cdecl Reimage(int *myarray, int array_length_row, int array_length_col);




__declspec(dllexport) double * __cdecl Reimage(int *myarray, int array_length_row, int array_length_col)
{
	vector<double> xy;
	xy.push_back(250);
	xy.push_back(155);
	return xy.data(); 
	
}

so I would expect to get back a 1d array with values 250 and 155 in the 0 and 1 spots respectively?


You have two problems with this:

 

1) You can't return an array as function return value to LabVIEW. LabVIEW has no way to know how large the array is and therefore couldn't possibly convert it into a LabVIEW array (which is distinctively different than a C array data pointer).

 

2) You create the standard template variable xy on the stack. You do not create a reference to it but the entire variable is on the stack. The automatic memory management of standard template variables makes sure that the variable is deallocated as soon as it goes out of program scope, which in this case is at the moment the return statement is executed. This means you are returning a pointer to a memory area that is not valid anymore as soon as the function returns to the caller. But avoiding the auto cleanup by either using variable references with explicit allocation of the variable through new, would only seemingly solve the problem. You would get the array properly back into LabVIEW but you would leak memory with every call of this function.

 

So even if you go around the limitation of LabVIEW to not allow to return C data array pointers as function value by treating it as a pointer sized variable and trying to index the pointer manually on your LabVIEW diagram, you are indexing a pointer that is not valid anymore and most likely used for other things already. The standard C++ template library has a tendency to inline the storage of its variable up to a certain amount so it is possible that the pointer you return is actually directly pointing into the stack rather than some dynamically allocated memory, and that stack is of course overwritten and reused umptien times at the point where your LabVIEW code tries to reference it. If you would create a bigger array (or initialize the array to be of some bigger size, the standard C++ template would likely use a dynamically allocated pointer inside the variable but at the moment the return statement is executed the variable will be cleaned up and free that pointer too. The contents in its memory may survive long enough to be able to get read by your LabVIEW code, but you are still accessing invalid memory that way and sooner or later you would run into big trouble anyways.

 

Interfacing LabVIEW to C code is not easy, interfacing it to C++ and especially standard template code can be a very serious challenge, since the memory management of both assume very different paradigmas, that you have to properly translate between the two in you C++ code.

 

To make this work, the simplest way would be to change the function to return a LabVIEW array handle, then inside the function allocate such a handle through a LabVIEW memory manager function such as NumericArrayResize(), copy the contents from the standard template vector into it (and update the length in the handle) and then return from the code.

 

Another approach would be to return the array in an array pointer argument. This only works easily if you know beforehand how large the array will be and can allocated the array on the LabVIEW diagram and pass the C data pointer as argument to the function. Then you copy the content of the vector into the array pointer before returning to the caller and having the vector auto destroyed by the C++ compiler.

 

In terms of OpenCV you can optimize this to avoid an extra memory copy of the image data but you have to know beforehand how big the returned picture will get. Then you can allocate a properly sized array data on the diagram, pass its pointer to the C function, which uses it to wrap an OpenCV image datatype around it, then you can pass the OpenCV image variable to just about any OpenCV function that requires this and clean up the wrapped image datatype before returning and the data is already in the LabVIEW array on return from the C function.

 

That is the most efficient way to deal with OpenCV images in LabVIEW. But as said, it requires you to know beforehand what the size of the image will be so you can allocate the array properly in LabVIEW before calling the function. If you can't know the size beforehand, the only way to deal with this is to do the OpenCV stuff in your C function, then determine the needed size, allocate a LabVIEW array handle through the proper LabVIEW memory manager functions and then copy the OpenCV image content into the LabVIEW array handle and return that handle to LabVIEW.

Rolf Kalbermatter
My Blog
0 Kudos
Message 5 of 6
(7,032 Views)


That is the most efficient way to deal with OpenCV images in LabVIEW. But as said, it requires you to know beforehand what the size of the image will be so you can allocate the array properly in LabVIEW before calling the function. If you can't know the size beforehand, the only way to deal with this is to do the OpenCV stuff in your C function, then determine the needed size, allocate a LabVIEW array handle through the proper LabVIEW memory manager functions and then copy the OpenCV image content into the LabVIEW array handle and return that handle to LabVIEW.

Wow! Thanks so much for all the insight.

 

The image input to the .dll will change in size different times we run the VI as sometimes we'll be using 1920x1200 for the whole image in full detail at a lower FPS, but other times we will use it for very small regions that may only be a few thousand pixels. The image size will stay the same for the duration of the VI so it could just get the size from the first image... But I don't think that solves the issue that what we're looking for in the C++ code is to detect peaks and give back their coordinates, and since every image will have a different number of peaks we can't possibly know how many will be returned.

 

It seems like it would be inefficient to process the image and find the number of peaks in a .dll, return the array to labview ("myarray" in my code),  then use the number of peaks to initialize an array for the x and y of their positions and send the image array back to C++ to find them?

 

 

0 Kudos
Message 6 of 6
(7,002 Views)