fatal internal error 0x0037C03D: "MemoryManager.cpp", line 1318 with Array Handle Pointer dll

[__dmtrs__]

Hello everyone,

I need to call some VIs after having exported them as dlls and I am facing the fatal error mentioned above when I try to get the numerical array resulting from the VIs' execution. :

I made a simple VI to reproduce the problem but I still can't get my head around it.Here's the snippet of the demo:

vi snippet

I was thinking of using the Array Handle Pointer since the array dimension may vary with execution time.

Any help is really appreciated!

Thanks,

Dimitris.

---

This is the python code that should get the array after the call:

from ctypes import *

path = 'C:\\Lavoro\\DPK\\Workspace\\dpk_test\\prova_dll\\python\\venv\\Lib\\array.dll'

lib = CDLL(path)


class DoubleArrayBase(Structure):
    _fields_ = [
        ('dimSize', c_uint32),
        ('NumericControl', (c_double * 1))
    ]


array_out = pointer(pointer(DoubleArrayBase()))

lib_array = lib.getArray

lib_array(byref(array_out))

print(array_out)

---

This is the dll's '.h':

#include "extcode.h"
#pragma pack(push)
#pragma pack(1)

#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
	int32_t dimSize;
	double NumericControl[1];
} DoubleArrayBase;
typedef DoubleArrayBase **DoubleArray;

/*!
 * getArray
 */
void __cdecl getArray(DoubleArray *array);

MgErr __cdecl LVDLLStatus(char *errStr, int errStrLen, void *module);

/*
* Memory Allocation/Resize/Deallocation APIs for type 'DoubleArray'
*/
DoubleArray __cdecl AllocateDoubleArray (int32 elmtCount);
MgErr __cdecl ResizeDoubleArray (DoubleArray *hdlPtr, int32 elmtCount);
MgErr __cdecl DeAllocateDoubleArray (DoubleArray *hdlPtr);

#ifdef __cplusplus
} // extern "C"
#endif

#pragma pack(pop)

---

[rolfk]

If I had a guess I would think that the handle you pass into the function is not properly zeroed.

void __cdecl getArray(DoubleArray *array);

Here before calling this function you need to do something like this in C:

DoubleArray array = NULL;
/* If the handle is not properly zeroed out, the LabVIEW DLL will interprete it as an already allocated handle and attempt to resize it to the needed size, if it is NULLed then the DLL will allocate a new array handle and return it. */

getArray(&array);

/* Read the content of the array, do NOT attempt to read beyond the end as indicated by dimSize, and even less to write beyond that value! */
double *ptr = malloc(sizeof(double) * (*array)->dimSize);
memcpy(ptr, (*array)->NumericControl, sizeof(double) * (*array)->dimSize);

/* Deallocate the array to not create a memory leak
MgErr err = DeAllocateDoubleArray(&array);


free(ptr);

So I guess you will need to do something like this, but not sure about the actual Python syntax:

array_out = pointer(pointer(DoubleArrayBase())) = null

Oops! This from the Python manual would indicate that you can't do it with the pointer keyword at all!

Note that ctypes does not have OOR (original object return), it constructs a new, equivalent object each time you retrieve an attribute:

This won't work with LabVIEW handles as Python has absolutely no way of correctly creating pointer objects that match LabVIEW data handle internas unless there is a way in Python to change this definition in your code:

class DoubleArrayBase(Structure):
    _fields_ = [
        ('dimSize', c_uint32),
        ('NumericControl', (c_double * 1))
    ]

to somethig similar to this:

class DoubleArrayBase(Structure):
    _fields_ = [
        ('dimSize', c_uint32),
        ('NumericControl', (c_double * dimSize))
    ]

[__dmtrs__]

Hi Rolf,

thanks for the tips!

I tried following your suggestions but I still cannot get it work.

This is where the combination of your help and some stackoverflow.com led me:

---

from ctypes import *

path = '..\\array.dll'

lib = CDLL(path)
elmtCount = c_int32(10)

dimSize = 0
# need to initialize otherwise I get --> NameError: name 'dimSize' is not defined


# update structure definition
def updateStructure(self, field_name, field_type):
    fields_copy = self._fields_
    found = False
    for idx, itm in enumerate(fields_copy):
        # find matching field name
        i = list(itm)
        print(i[0])
        if i[0] == field_name:
            i[1] = field_type
            fields_copy[idx] = tuple(i)  # update with the new type
            found = True
            break
        if not found:
            print('element not found')
            # return False

    class TmpStructure(Structure):
        _pack_ = self._pack_
        _fields_ = self._fields_
    return TmpStructure


class DoubleArrayBase(Structure):
    _fields_ = [
        ('dimSize', c_uint32),
        ('NumericControl', (c_double * dimSize))
    ]

    def __init__(self, dimSize):
        self.dimSize = dimSize
        self.NumericControl = (c_double * dimSize)()


# dll methods
getArray = lib.getArray

#solution A
# array_out = byref(byref(updateStructure(DoubleArrayBase, 'NumericControl', (c_double * 10))))
#solution B
array_out = pointer(pointer(DoubleArrayBase(10)))
# get array
getArray(byref(array_out))
print(array_out.NumericControl[1])

---

Solution A turns the ctype structure in a not valid 'ctypes.PyCStructType'

Solution B manages to update the NumericControl, but I still get the fatal error message.

At this point I was wondering if there should be any call to the *.h methods:

/*
* Memory Allocation/Resize/Deallocation APIs for type 'DoubleArray'
*/
DoubleArray __cdecl AllocateDoubleArray (int32 elmtCount);
MgErr __cdecl ResizeDoubleArray (DoubleArray *hdlPtr, int32 elmtCount);
MgErr __cdecl DeAllocateDoubleArray (DoubleArray *hdlPtr);

I tried using them following as much as possible what you suggested in your C code, but nothing's new.

Thanks anyway for the support!

Dimitris

[rolfk · Accepted solution]

I think trying to manage the LabVIEW handle in Python is going to be an almost impossible feat. ctypes is pretty good but can't do everything possible in C land also partly due to certain restrictions of Python itself (such as the no OOR support).

The most flexible way will be to implement the buffer as a C array pointer in the DLL with an addtional integer parameter that tells the DLL how big the allocated buffer is, so that the DLL won't write over the end of that buffer. This will be easy to implement with ctypes.

The limitation that you may not know ahead how big the buffer needs to be can be worked around by adding a function that tells you the required size for the parameters in question and use that to allocate the array in the correct size.

LabVIEW handles are a great way to manage variable sized data buffers in environments that support such things as a first class data type but environments like Python, Java and even .Net have really a different approach in that regard. They try to shield users away from things like memory allocations and their method is very different than what LabVIEW does to reach the same goal. Environments like C (or C++) with the low level capabilities of working almost on the bare metal CPU itself can do any of these (managed) contracts as long as you know how they work but that is a different story when working on high level languages like Python, Java, .Net and LabVIEW when you try to interface to the managment contract of one of the others.

Expanding ctypes support in Python to also support LabVIEW datatypes specifically would be one possibility but not a very likely one to occur. Adding a binary Python module for LabVIEW support similar to ctypes called for instance lvtypes would be another one, but the demand for either of those would be likely very low and the effort to implement it fairly high. As binary modules are written in C one can do any possible memory manupulation that is allowed in protected mode to bridge the different paradigma of LabVIEW and Python managed memory.

[__dmtrs__]

Thanks for the support Rolf,

I actually started using the Array Data Pointer (worked at the first run), but I was trying to get rid of the array size declaration I had to do in python before getting the array.

As you said, it seams to be quite an impossible feat! I'll stick with the basics and drop the Array Handle Pointer!

Thanks again,

Dimitris