LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Creating EXTERNAL data value references

Solved!
Go to solution

A common practice to reduce the memory footprint of a program, is to pass data value references, instead of full blown copies ( Storing Data and Reducing Data Copies with Data Value References ). However when interfacing with external libraries, the well documented interfaces all rely on either performing a deep copy from/to LabVIEW arrays. Although these then can be referenced, depending on the external library one might end up with a full copy of the data.

 

The LabVIEW documentation for data value references does mention in passing, that there also are External Data Value References:

 

External data value references are similar to data value references except they work directly with data owned by components external to LabVIEW, like the direct memory access (DMA) buffer of a device driver that controls a data-streaming device. You can work with this external data without incurring a copy to bring the data into LabVIEW. If a device driver supports an external data value reference, you can interact with the external data similarly to the way you interact with data value references. You also can use the Advanced TDMS VIs and functions to interact with external data value references.

 

This description very much matches my particular use case: I have a library, which interfaces with a driver that does (very bulky, several GB) DMA transfers.

 

Unfortunately I was unable to find any documentation on how to create external data value references. The best thing I could up was creating a data value reference constant, which can be set to "External" in its properties. However that alone is pretty much useless.

edvr.png

There's exactly one documented standard VI which makes use of EDVRs, that is the the "IMAQ ImageToEDVR" VI. which is polymorphic and dispatches into the various data types.imaq_edvr.png

 

All of the polymorphics call into a function "LV_ImageToEDVR" of an IMAQ external library.edvr_call.png

 

 

Whatever is pointed to by the "edvr" parameter is some kind of descriptor buffer, parts of which must be populated by the callee; I assume, at the least somewhere the pointer to the actual data, its type and size must be stored in that.

 

But the exact layout of whatever structure this is, I found no documentation for. I actually did try to figure it out, by writing my own external library which simply dumped 256 bytes starting at where the "edvr" parameter points to, and also did this with a regular data value reference. But in neither case within those first 256 bytes there was a value to be found, which would match the address of the array referenced by that very data value reference. However I can clearly discern some form of counter and handle values.

 

So what I want to know right now is, how to create external data value references, which means I need to know the structure of whatever is pointed to by what's passed to a function parameter a EDVR is connected to. And I presume, because whatever this structure contains has no pointers, there must be some LabVIEW function be called to actually register a EDVR to get the (presumably) handle values, I could see in my memory dumps.

 

Can anyone help with that?

0 Kudos
Message 1 of 5
(4,587 Views)

Found some info here. It looks like the driver has to supply the EDVR. I guess thats why IMAQ has it. 

 

Some more discussion on LAVA.

Certified LabVIEW Architect
0 Kudos
Message 2 of 5
(4,566 Views)
Solution
Accepted by topic author ag-huber-luebeck

Another place that supposedly uses External DVRs is the GPU Analysis Toolkit.

 

Take a look at this document here and the ZIP file it contains. It contains a document and some C examples with additional not fully official LabVIEW header files that declare the needed function prototypes to do this. Expect this to be a very different C programming level than your average standard LabVIEW DLL wrapper.  I don't think it will show you how to do custom datatype EDVRs, that part is even more secret, but for what you describe arrays of LabVIEW skalar numerics should be more than enough.

Rolf Kalbermatter
My Blog
Message 3 of 5
(4,550 Views)

The answer given by @rolfk pointed me into the right direction. I'll want to make this a longer writeup. The gist is, that once I knew where and what to look for I found all the relevant internal APIs and got the whole EDVR stuff working. I had to chuckle a bit about this:

 

Expect this to be a very different C programming level than your average standard LabVIEW DLL wrapper.

After all the driver (kernel side, DMA engine and all) I'm interfacing with was written by me anyway. I'd consider C to be the one programming language I fully mastered (knowing all its quirks, know how the popular compilers align structs, etc. BTW, the way the LabVIEW extcode.h headers try to pin down structure element alignments isn't particularly robust; enabling packing still allows compilers some leeway if they see the need to satisfy architecture alignment rules).

 

Anyway, the EDVR interface has all the things I was dearly looking for. Including the notion of actually getting notice when all branches of a G code dataflow terminated and some externally allocated resource required deallocation.

 

That's something that always irked me in the bindings I've implemented so far: There was no way to release system handles allocated inside a DLL in case execution is suddenly stopped (user hit stop execution button, or some automatic error handler triggered).

Message 4 of 5
(4,504 Views)

@ag-huber-luebeck wrote:

 

After all the driver (kernel side, DMA engine and all) I'm interfacing with was written by me anyway. I'd consider C to be the one programming language I fully mastered (knowing all its quirks, know how the popular compilers align structs, etc. BTW, the way the LabVIEW extcode.h headers try to pin down structure element alignments isn't particularly robust; enabling packing still allows compilers some leeway if they see the need to satisfy architecture alignment rules).

Well the average user posting here about such problems has often just enough knowledge to startup the Visual Whatever IDE and hit the Build menu and hope everything works. They still expect to be able to interface to low level device drivers and of course use the most performant asynchronous APIs to use their specific hardware, because after all the hardware provides this capability and LabVIEW is super easy to use! Your expertise is obviously very different but that is very seldom.

 

That's something that always irked me in the bindings I've implemented so far: There was no way to release system handles allocated inside a DLL in case execution is suddenly stopped (user hit stop execution button, or some automatic error handler triggered).


There is another way using UserDataRefnums which happens to be "documented" in those headers too and which I used in Lua for LabVIEW. Or you can use directly the RTSetCleanupProc() facility to register a cleanup procedure for whatever pointer value you fancy and when it should be called. This is the facility that LabVIEW also uses for all its refnums, and that is under the hood also used by the UserDataRefnums and the EDVRs too, but LabVIEW only uses the kCleanOnIdle setting exclusively with the exception of VISA Refnums where you can configure LabVIEW to never autocleanup VISA Refnums. For debugging purposes I allow in Lua for LabVIEW to create refnums without autocleanup facility but recommend to not set that boolean input to true on the Create function.

 

As to the alignment magic used in the LabVIEW headers, it's not universal but works for the platforms that LabVIEW compiles on and for user space external libraries on those platforms. And it basically only uses custom alignment anymore for Windows 32 bit compilation, and MacOS, leaving the default alignment for all other platforms. That used to be different in old days with LabVIEW also running on Mac Classic, Mac PPC (both Classic and OSX), Sun Solaris, HP Unix, and in internal tries even for never released platforms like Mac Copland, Next, and a few more.

Trying to use that alignment magic in any way in kernel space on any platform is bound to give unexpected results.

Rolf Kalbermatter
My Blog
Message 5 of 5
(4,497 Views)