Is it possible to pass an image to the Python Node as an image and not as a cluster of data(as a pixel array)?

[Felix549]

Hello NI community, I am new to LabView so I had this question:

Is it possible to pass an image directly into the Python script(in a Python Node) without passing it as a pixel array and after the processing, return the image as an ‘Image’ and not a cluster of data of some kind? (this cluster would include the pixel array as well, among other things) 

For context, I have to do some standard image processing using OpenCV on an image and I was wondering if it is possible to pass the image directly without breaking it into a pixel array using a function such as ‘Read JPEG file.vi ’ or ‘IMAQ ReadFile.vi ’.

This is the program where I am passing an image as a cluster of data. The orange arrow shows that the return type is a data cluster consisting of 6 different elements that includes the pixel array and some other data such as pixel resolution, etc.

HOWEVER! If it is indeed not possible to pass an image as just an image, my next question would be:

What is the best way to pass an image into a Python node and which function to use (is IMAQ ReadFile.vi  the most suitable function?)

Also, is it possible to exactly recreate the original image by converting the data cluster(that includes the pixel array) into a NumPy array? Is the conversion process accurate and feasible or is it too low level and error prone to deal with the pixel array? especially if I end up doing it multiple times with different Python nodes?

I hope I have described my problem sufficiently. Please let me know if any other detail would be required.

Thanks!

[altenbach]

You can look inside "flatten image" and "unflatten image" to see how things are converted back and forth. You can either use these function or implement equivalent processing in python.

(Also note that you should use a path constant, not a string constant for the path. This will make it OS independent).

[Felix549]

So are you suggesting that I should first flatten the image, pass it as a string to the Python node and then return as a string from the Python node(after processing in Python script) and then unflatten it once it is returned from the Python Node into LV ??

And within the Python script, is it feasible enough to convert the flattened image string back into the original image(or a NumPy array) and then once again as a flattened string in order to return it? If yes, is it practical to handle the image as a NumPy array without losing one's mind?

Or am I mistaken somewhere in understanding the suggested approach?

[rolfk · Accepted solution]

Almost everything is possible with enough effort! Is it easy? Nope!

An image is not a standard storage format. There are virtually thousands of possible formats. Each graphic library tends to have its own specific internal image format and then there are the streamed image formats to store files to disk or transfer them over a network connection. None of them are 1 to 1 compatible!

Python OpenCV uses numpy arrays to store pictures. That is a specific memory format created by the C library that implements the Python numpy library. LabVIEW uses a number of its own memory formats, one being the IMAQ view image and the other being a proprietary format resembling the Macintosh PICT format. In order to be able to stream the second, there is the Flattened format which puts the various elements of a picture into defined places.

The internal LabVIEW format is private, there is no other code that could directly interpret it outside of LabVIEW so trying to pass a pointer to it to your Python code would be pretty useless. The numpy format is alien to LabVIEW and while you could convert a LabVIEW picture into a numpy compatible memory structure in LabVIEW, this would be a lot of work to do.

When crossing environments you need to somehow adapt the formats. Either Python needs to be able to understand the LabVIEW picture directly (technically pretty unfeasible since that format is not documented), or creating a numpy compatible array in LabVIEW (technically possible but really a lot of work to do), or using an intermediate form such as the Flattened image format.

Of course you could avoid at least some of the conversions by passing the path to the image into Python instead of the image.

[Felix549]

Thank you for the explanation. Having read your answer, I am thinking of an approach that avoids conversion while passing the image to the Python node. Please let me know if you'd suggest this approach:

Instead of passing the image data to the Python node, I pass the path of the image. In the Python script, I use the path and read the image using OpenCV, perform the processing and once the processing is done, convert the image back to a suitable array format that before returning it to LabView. Once outside of the Python node, use a function such as IMAQ Write File 2 VI or Write JPEG file to convert the image array back to an image.

Do you think this approach would work and feasible enough? Can it be further improved? 

Thanks!

[rolfk]

I understand your approach of passing in the path (it's basically what I suggested), but why oh why do you want to pass the image data back when all you want to do is to save the pic to a file anyhow? Why not write it in Python too?

[Felix549]

This is part of a bigger application made in LabView, that's why I am inclined towards returning the image data out of the Python node. That resulting image would be required by other functions in the application. But still, what you are saying still holds. I can still just write the image within the Python node. The other functions can read from that saved file (as long as it is synchronous).

However, I wonder the reason why you dislike the idea of returning the image as an array is because it's too much detail and therefore prone to error. The image is better never converted to an array in the first place. Is that the reason? I'm curious.

[rolfk]

I would probably try to pass it as an numpy array from and to LabVIEW but that is not something that can be done just in LabVIEW itself, you would need to write some C code for that, and no not a little hello world program but quite a bit more involved. And I'm not going to try to explain how many hoops and loops you would have to jump through to get that working. 😁