NumericArrayResize CIN function bug (feature)

[Tomi_Maila]

CIN function NumericArrayResize doesn't resize the array if you set the sizes fields of the array to the correct values prior to calling the function. This causes LabVIEW to crash since not enough space is allocated for the numeric array. I assume that instead of really checking if enough space is allocated for the array, NumericArrayResizes relies on the size fields to check if the array is already allocated correct amount of memory.

In my case there is an easy work around, I just set the size fields of the array after calling numeric array resize. This bug (feature) may be a major problem in the following case however. Assume that you allocate memory with DSNewHandle. The memory is uninitialized and may contain anything. You later resize the Handle with NumericArrayResize. Since the memory content of the handle may be anything, there is a small chance that the memory element in the dimsizes fields happen to define an array of the size you would like your numeric array to be. In this case the array is no resized and memory leaks will occur.

I suggest that NI will either 1) fix the bug (feature) or 2) stress the issue in the documentation. The documentation should state that NumericArrayResize should always be used to create handles to numeric arrays instead of any other function. The documentation should also state that user should never set the size fields prior to resizing the handle with NumericArrayResize. Neither of these two things were present in the August 2003 version of "Using External Code in LabVIEW". This was the newest I found.

[shahina]

Hello Tomi,

Thank you for the feedback. If you have a simple code that illustrates this problem, could you please post it here?

Thanks and best regards,
Shakhina P.
Applications Engineer
National Instruments

[Tomi_Maila]

If I remeber correctly, the problem is illustrated by the code below. The example is for 1D array, but I encountered it with multidimensional arrays. The problem occurs if the array dimension sizes are set before the call to NumericArrayResize as it seems NumericArrayResize relies on the information on the dimSize or dimSizes parameter to find out how large the array is currently. This behaviour is ok, if it would just be documted.

#include "extcode.h"

/* Typedefs */

typedef struct {
    int32 dimSize;
    int32 elt[1];
    } Array;
typedef Array **ArrayHdl;


long ResizeArray(ArrayHdl arg1,const long newsize)
{
    /* This fails, array is not resized by NumericArrayResize. No error is returned either. */
    (*arg1)->dimSize=newsize;
    NumericArrayResize(iL,1,&arg1,newsize);
    return noErr;
}

long ResizeArray(ArrayHdl arg1,const long newsize)
{
    /* this works */
    NumericArrayResize(iL,1,&arg1,newsize);
    (*arg1)->dimSize=newsize;
    return noErr;
}

[rolfk]

---

@Tomi M wrote:
If I remeber correctly, the problem is illustrated by the code below. The example is for 1D array, but I encountered it with multidimensional arrays. The problem occurs if the array dimension sizes are set before the call to NumericArrayResize as it seems NumericArrayResize relies on the information on the dimSize or dimSizes parameter to find out how large the array is currently. This behaviour is ok, if it would just be documted.

#include "extcode.h"

/* Typedefs */

typedef struct {
    int32 dimSize;
    int32 elt[1];
    } Array;
typedef Array **ArrayHdl;


long ResizeArray(ArrayHdl arg1,const long newsize)
{
    /* This fails, array is not resized by NumericArrayResize. No error is returned either. */
    (*arg1)->dimSize=newsize;
    NumericArrayResize(iL,1,&arg1,newsize);
    return noErr;
}

long ResizeArray(ArrayHdl arg1,const long newsize)
{
    /* this works */
    NumericArrayResize(iL,1,&arg1,newsize);
    (*arg1)->dimSize=newsize;
    return noErr;
}

---

This is not really a bug but a performance optimization.

NumericArrayResize got at around version 6.0 more "smart" in a few ways. I agree that it is badly documented but I can't really consider it a bug. Besides the optimization you found it will also only copy as many data from the old handle to a newly created (larger) handle as the dim size indicates. Before it copied as much data as the handle was originally long even though it may be a 1MB array where only the first value has been filled in yet.

If you ever allocate a handle that you do want to pass to other functions without dealing with it directly yourself you should Use DSNewHClr() which clears the handle all to 0 or at least set the dim size(s) explicitedly to 0. But in your case you do not even need to use any DSNewHandle function.

NumericArrayResize if passed a reference to a NULL value will allocate that handle too. That is why the handle parameter is actually a referenced handle instead of just a handle. This has been all the time so as far as I know but I think it is not well documented.

MgErr err;
UHandle h = NULL;

err = NumericArrayResize(iL,1,&h,newsize);
(*h)->dimSize=newsize;

As to intializing an array size you never should do that before resizing that array. That is a common practice and especially true for LabVIEW.

NumericArrayResize will intialize additional memory to 0. However if you happen to have an array of handles beware that if you make the array shorter you first need to deallocate the non used handles before resizing the array. Otherwise you create a memory leak.

This function also returns a LabVIEW manager error and it is definitely a good idea to check that one too after a call to it.

Rolf Kalbermatter

[Tomi_Maila]

Thanks Rolf,

Could NI just please consider documenting all the exceptions in the behaviour of the LabVIEW C interface.

I was trying write very optimized code when I encountered the issue. In the case of my code it would have been more efficient to set the dim sizes before the call to ResizeNumericArray and not after, I won't go into detail why. It was quite hard to find the problem that made LabVIEW to crash, since the code I wrote was literally correct according to LabVIEW documentation. The array I passed to the function was never NULL, so there always was an valid array handle present before I tried to modify the dimsizes.
  
Tomi

[rolfk]

---

@Tomi M wrote:
Thanks Rolf,

Could NI just please consider documenting all the exceptions in the behaviour of the LabVIEW C interface.

I was trying write very optimized code when I encountered the issue. In the case of my code it would have been more efficient to set the dim sizes before the call to ResizeNumericArray and not after, I won't go into detail why. It was quite hard to find the problem that made LabVIEW to crash, since the code I wrote was literally correct according to LabVIEW documentation. The array I passed to the function was never NULL, so there always was an valid array handle present before I tried to modify the dimsizes.

---

Consider the C API of LabVIEW as some sort of legacy remainder. They can't get rid of it for compatibility reasons but are not spending much time into it either. If they could they would simply drop it. Solves the trouble of documenting it, and even more troublesome supporting it to people outside of the company.

Feel free to put a product suggestion into the system. They do occasionally do some stuff on that. But be aware that support for the C API is very low on the priority list. Starting up a symbolic debugger and checking out the trouble is light years faster than waiting for an improved C API or documentation thereof.

Rolf Kalbermatter

Message Edited by rolfk on 07-24-2006 10:31 PM

[Tomi_Maila]

If what you say Rolf is true, I must say that it's kind of weird. The C interface, even though not used by most of LabVIEW developers, allows those few developers using it to build outstanding LabVIEW products. Integrating C and LabVIEW can bring valauable missing functionality into LabVIEW. Those using this integration can develop LabVIEW toolkits that can extend the capabilities of LabVIEW and in such a way bring LabVIEW more customers that would otherwise choose some other solution for their problem.

This is a bit off topic, but I have been quite disapointed in the way NI is developing some tools only for internal use and never even plan to release those. This gives NI a strategic advantage over other LabVIEW developers. NI can develop LabVIEW toolkit products that other LabVIEW developers cannot, since they don't have access to the same technology. For example developers outside NI cannot develop VIs similar to "Match Regular Expression" or "Storage" VIs since NI doesn't give access to technology used to develop these features anybody outside NI. Since LabVIEW is for quite a small niche market, NI would benefit if there were better products available that would utilize LabVIEW. A more open LabVIEW would bring more developers, more products and more users in the end. I have wondered if NI is using it's near a monopoly position in data-acquisition software development environment according to U.S. and E.U. legistlation. Microsoft has been sued multiple times over similar kind of protectionism in Windows.

[rolfk]

---

@Tomi M wrote:
If what you say Rolf is true, I must say that it's kind of weird. The C interface, even though not used by most of LabVIEW developers, allows those few developers using it to build outstanding LabVIEW products. Integrating C and LabVIEW can bring valauable missing functionality into LabVIEW. Those using this integration can develop LabVIEW toolkits that can extend the capabilities of LabVIEW and in such a way bring LabVIEW more customers that would otherwise choose some other solution for their problem.

This is a bit off topic, but I have been quite disapointed in the way NI is developing some tools only for internal use and never even plan to release those. This gives NI a strategic advantage over other LabVIEW developers. NI can develop LabVIEW toolkit products that other LabVIEW developers cannot, since they don't have access to the same technology. For example developers outside NI cannot develop VIs similar to "Match Regular Expression" or "Storage" VIs since NI doesn't give access to technology used to develop these features anybody outside NI. Since LabVIEW is for quite a small niche market, NI would benefit if there were better products available that would utilize LabVIEW. A more open LabVIEW would bring more developers, more products and more users in the end. I have wondered if NI is using it's near a monopoly position in data-acquisition software development environment according to U.S. and E.U. legistlation. Microsoft has been sued multiple times over similar kind of protectionism in Windows.

---

I think you are pointing out the wrong examples. Match Regular Expression is a built in node of LabVIEW. It does use internally the PCRE library and you are free to do so too. The only thing you can't do is automatic type adaption and resizing and to the best of my knowledge that are things you can only add in the source code of LabVIEW itself. Allowing to do that for external users would mean opening up almost the entire object interface of LabVIEW, documenting a few hundred extra APIs and in general publishing the LabVIEW source code may be simpler solution. That the last isn't going to happen soon is probably clear to anybody.

There is a small chance that these things are based on and possible with XNodes, I haven't checked myself and in that case there have been people investigating how they could be used. Unfortunately LabVIEW 8 does protect this better ;-(
And I think it is not so much about not wanting others to be able to use such technologies than the requirement to produce decent documentation for them and train some people in supporting them, before making such features available. I know for a fact that there are many that would like those features, very few that would accept to overlook limited documentation for them and just about as few that would want to pay for them extra even with outstanding docs.

So it does also get down to the question how much can it bring NI to make this available and I mean here hard dollars not just some improved user satisfaction.

You can try to bring this to the attention of the EU legislative. but I'm pretty sure they do not consider NI a big enough threat to a free market that they will even consider doing anything at the moment. The market is by all standards still quite small and NI is by far not the only player in it.

Rolf Kalbermatter

[rolfk]

---

@Tomi M wrote:
If what you say Rolf is true, I must say that it's kind of weird. The C interface, even though not used by most of LabVIEW developers, allows those few developers using it to build outstanding LabVIEW products. Integrating C and LabVIEW can bring valauable missing functionality into LabVIEW. Those using this integration can develop LabVIEW toolkits that can extend the capabilities of LabVIEW and in such a way bring LabVIEW more customers that would otherwise choose some other solution for their problem.

---

Well it is used by extremely few people, and not because it is badly documented (it is actually quite well documented) but because most people that would know how to use it would not start to use LabVIEW, and those that use LabVIEW do not care to learn the exponential bigger complications of C programming in comparison to LabVIEW.

As to developing Toolkits for LabVIEW you will quite soon find out, that this is a market you can't really live from.
A lot do want to get additional functions for free, expect outstanding documentation nevertheless, and consequently a perfect one for something they buy and will usually prefer a solution from NI before any other unless it is very outstanding and special. And then you should not forget about all those with the "Not invented here" syndrome.

So you will be spending lots of time for something you can not easily get enough money with to pay your development costs not to speak about your support. That is not so much NIs fault (well you could ask them not to sell any Toolkits to increase your chances) but just how things are.

Rolf Kalbermatter

[Tomi_Maila]

The examples I provided are LabVIEW 8 Xnodes, the technology which allows automatic generation of the node at the time of dropping the node code on the block diagram. I didn't want to mention the name of the technology, so that NI wouldn't remove this thread. I have requested that I could use Xnodes to be able to build file I/O VIs that would adapt to type and dimension of arrays, but NI head office refues to give the technology out to anybody. It's a pity. I understand that the technology is not ready and not yet documented but still it gives the NI competetive advantage to provide toolkits that other companies cannot provide. Currently I use scripting to automatically generate polymorphic VIs with hundreds of subVIs that call the above mentioned C code (NumericArrayResize and all) to do the actual I/O.

Message Edited by Tomi M on 07-25-2006 12:47 AM