.NET Invoke does not update array passed to it - Makes new!

[blublubBLUB]

Hello everyone! This is my first post on these forums. I am having a problem that is stumping me.

It is as follows:

In my code I am invoking a function located in a .NET 2.0 assembly. The only documentation I have says the following:

"int ReadHPTDCHit(HPTDCHit buffer)
Copy TDC data into buffer. The number of data words that were read is returned as an integer. All available data up to the size of the buffer is read."

However, it would appear that when I get my HPTDCHit buffer back from the invoke node, it has not used the array I supplied it with. Memory concuption increases by every call of ReadHPTDCHit. To my limited knowledge, it seems that the array I supply is no longer handled neither by Labview, nor the DLL. 

I have attached a picture showing a minimum working example of my problem. It is analogous to what my main program is doing. To get it to work, I needed some more "fluff" around the problematic part to properly start and stop my hardware. This can be ignored.

Explanation of the VI: A 1000-element array of HPTDCHit .NET references is created. This will the the buffer that the function requires. It is passed to the function repeatedly, and the return from the function is reused. One should imagine that the original array is forever reused, but what I experience is that a new array is returned from the invoke node on every call. The original array is lost to the wind, but the memory is still allocated.

A possible solution is of course to feed the original array into a "close reference" function on every loop iteration, but that would seriously degrade the performance of the readout. A quick test shows that dereferencing 1000 .NET references takes 2.4ms. The amount of data I am expecting (a few MHz) could easily overflow with such a delay. At 2 MHz, 4800 HPTDCHit  objects is generated every 2.4ms.

Any suggestions on how to solve this would be greatly appreciated.

[tyk007]

---

@blublubBLUB wrote:

Hello everyone! This is my first post on these forums. I am having a problem that is stumping me.

It is as follows:

In my code I am invoking a function located in a .NET 2.0 assembly. The only documentation I have says the following:

"int ReadHPTDCHit(HPTDCHit buffer)
Copy TDC data into buffer. The number of data words that were read is returned as an integer. All available data up to the size of the buffer is read."

However, it would appear that when I get my HPTDCHit buffer back from the invoke node, it has not used the array I supplied it with. Memory concuption increases by every call of ReadHPTDCHit. To my limited knowledge, it seems that the array I supply is no longer handled neither by Labview, nor the DLL. 

I have attached a picture showing a minimum working example of my problem. It is analogous to what my main program is doing. To get it to work, I needed some more "fluff" around the problematic part to properly start and stop my hardware. This can be ignored.

Explanation of the VI: A 1000-element array of HPTDCHit .NET references is created. This will the the buffer that the function requires. It is passed to the function repeatedly, and the return from the function is reused. One should imagine that the original array is forever reused, but what I experience is that a new array is returned from the invoke node on every call. The original array is lost to the wind, but the memory is still allocated.

A possible solution is of course to feed the original array into a "close reference" function on every loop iteration, but that would seriously degrade the performance of the readout. A quick test shows that dereferencing 1000 .NET references takes 2.4ms. The amount of data I am expecting (a few MHz) could easily overflow with such a delay. At 2 MHz, 4800 HPTDCHit  objects is generated every 2.4ms.

Any suggestions on how to solve this would be greatly appreciated.

---

I assume that the method signature you have written down is not correct since the parameter does not indicate an array (the Array type created would be indicated as HPTDCHit[] - not the square brackets). Clarify this first to make certain that the interface is correct.

One tip I can share is that any .NET objects created by LabVIEW are pinned by LabVIEW until you close references or the top-level VI in your hierarchy ends (at which point references are automatically freed) - your dll call won't be able to free or re-allocate the memory assigned to the array but it might (and probably does in this case) re-allocate what each element of the array contains.

And some source code would help; any would help us to assist you further.

[tyk007]

Forum access appears to be up again but the private message you sent me has gone missing. For future searches let's continue in this thread.

[blublubBLUB]

Yes, the private message has disappeared from my sent items folder as well.

Okey, the jist of the message was:

The missing square brackets in the documentation (int ReadHPTDCHit(HPTDCHit buffer) instead of int ReadHPTDCHit(HPTDCHit[] buffer)) is likely an errata, as the single code example the manufacturer of the .dll supplies us with says:

HPTDCHit[] buffer = new HPTDCHit[2000];
int count = manager.ReadHPTDCHit(buffer); 

So we know that it is meant to take an array, which is also what the invoke node accepts.

Regarding your possible explanation:

"... .NET objects created by LabVIEW are pinned by LabVIEW until you close references or the top-level VI in your hierarchy ends (at which point references are automatically freed) - your dll call won't be able to free or re-allocate the memory assigned to the array but it might (and probably does in this case) re-allocate what each element of the array contains."

This sounds to me like the likely explanation. If so it would be a huge drawback to use labview for high speed acquisition when the data array to be filled consists of .NET object references.

I do not have the source code of the .DLL, as it was written by a different company, but I can link the user manual for the hardware in question. Not sure what good it can do, but hey. Programming interface description from page 15 and out.

http://www.cronologic.de/products/time_measurement/tdc/hptdc/HPTDC_description.pdf

[tyk007]

Regarding your possible explanation:

"... .NET objects created by LabVIEW are pinned by LabVIEW until you close references or the top-level VI in your hierarchy ends (at which point references are automatically freed) - your dll call won't be able to free or re-allocate the memory assigned to the array but it might (and probably does in this case) re-allocate what each element of the array contains."

This sounds to me like the likely explanation. If so it would be a huge drawback to use labview for high speed acquisition when the data array to be filled consists of .NET object references. 

---

There is going to be overhead with using LabVIEW as the "middle-man" so to speak. LabVIEW needs to work with the CLR COM Interop in order to create objects, call methods or set / get data types that it can understand. 

One thing I should have clarified is - there is memory associated with an array that consists of what the array looks like and it's size and then there is the memory allocated for each of it's array indexes (slots) that can be changed. Two different memory sets. Generally the first is fixed - you can't change the size of an array once it has been made or what types it is supposed to store but you can change what each index points to - or the properties of the object that the index points to. Looking at the manual it would seem that the latter is what is occuring; hence why you create the objects at the indexes first.

It sounds like it works this way - initially that buffer has an array with indexes that each point to a new instance of the HPTDCHit object. Presumably once the acquisition has taken place the objects inside that array have been updated ie. the properties of the HPTDCHit objects at some or all indexes is updated. Note that the actual array hasn't changed at all - it has the same size and each index still has the same reference for the same object. It is the object internals themselves that have changed. This implies that getting this working in LabVIEW might be possible; though whether the overhead is adequate is another issue. The longer-term best solution thought might be to wrap the calls up into another .NET dll that has a simpler interface for LabVIEW.

There is a chance that the dll is not obfuscated; ie. it can be decompiled into readable C#. You could try attaching it and I can give that a go (you might need to zip the dll etc. to get past any blocking) along with your source code to go with it. This would help confirm my theory on this.

[blublubBLUB]

I have lots of experience working on more "proper" programming languages such as C and C++, so what you are saying about how arrays work resonates with me. I believe you are correct in your assumption that the DLL would simply update the values of the HPTDCHit objects already in memory, that's how it is normally done. IT's not too problematic that we could not find a solution, as the DLL has another function with much the same functionality as the one that is causing problems. This function also takes an array and updates the values, but this array consist of simple uint_64's, and is much much faster and I do not have to deal with deallocations.

[tyk007]

---

@blublubBLUB wrote:

I have lots of experience working on more "proper" programming languages such as C and C++, so what you are saying about how arrays work resonates with me. I believe you are correct in your assumption that the DLL would simply update the values of the HPTDCHit objects already in memory, that's how it is normally done. IT's not too problematic that we could not find a solution, as the DLL has another function with much the same functionality as the one that is causing problems. This function also takes an array and updates the values, but this array consist of simple uint_64's, and is much much faster and I do not have to deal with deallocations.

---

Great. Post back if you are able to solve your issue or have any other questions.