DIG_BLOCK_IN memory usage. for 6533 PCMCIA card

[Buffning]

I'm trying to take in large amounts of data (100mB) very quickly (>100kHz) using a 6533 PCMCIA card and c++.  The card/computer can't keep up with the speed if the data is written to a file during input.  THerefore, I'm trying to keep all the data in RAM until it is done transferring data and then save it to the harddrive.  I have >200 MB of RAM free (should be enough?), yet it DIG_BLOCK_IN is giving me error 10444, a memory full error when I pass it an array of only 32 MB.  The error doesn't come when I allocate the memory, but many lines of code later when I call DIG_BLOCK_IN.  Does DIG_BLOCK_iN allocate more memory?  How much more?  Or is there a memory limit to the function?  The documentation makes it seem that you can pass any array of length less than 2^32-1 or the size of  count. 

[Buffning]

It gives out of memory error when I pass DB_BLOCK_IN a count value greater than 2^25 despite the documentation showing

"count is the number of items (for example, 8-bit items for a group of size 1, 16-bit items for a group of size 2, and 32-bit items for a group of size 4) to be transferred to the area of memory specified by buffer from the group indicated by group.

Range: 2 through 2³² - 1"

Is there a lower range on count than shown in the documentation?

[Travis_W]

Hi Buffning,

What operating system are you working with?  Different versions of Windows can have differnet limitations with memory allocation.

Are you using a double-buffered acquisition?  This post describes how this can be a possible solution to the issue, and this post mentions the location of a good example.

If this doesn't help out, please let me know, as I want to make sure that you get up and running.

Have a great day!
Travis W

Message Edited by Travis W on 08-15-2006 11:19 PM

[Buffning]

Hello,

I'm working on windows xp pro.  I have over 200 MB of ram free when I try to allocate the 64 MB.  I tried to find a limit on the amount of memory that it allowed me to allocate in C++ and it let me allocate and work with 300MB of Ram, which I would assume isn't all physical ram, like Block_In seems to require.  It seems though that it would give a pagelock error if it couldn't get the memory to the physical ram and not a error 10444.  I allocate all of the memory it should need before calling Block_in. 

I tried double buffering, but it wouldn't run fast enough to keep up with the speeds I need.  Actually, it wouldn't even run at 10KHz, which seems very slow.

My code is based on the example you mentioned.  I simply changed some variables to create a larger buffer.

[Travis_W]

Hi Buffning,

After looking into this issue more, I've found varying information about what Windows XP can and can't do in terms of buffer allocation with this function.  Can you try changing the size of your buffer to find out at what point this error occurs?  Is it exactly 32MB?

Also, do you have access to a Windows 98 machine?  I'm interested to see if this application will run any differently on another OS as per this post.

Have a great day!
Travis W

[Buffning]

The error occurs when I pass ulCount >= 2^25 to Block_In.  It doesn't seem to be effected by the size of the buffer I create or pass to Block_In. I don't have access to any other OS systems unfortunately. 

Thanks a lot for your help,
Mark

[Travis_W]

Hi Mark,

It seems that this may be a limitation of your particular computer.  Do you have any extra RAM that you could borrow from another machine to see if the 2^25 limit can be extended?

Also, stepping back to your original code where you were calling DIG_BLOCK_IN with less data, I'm curious why streaming the data to disk is too slow at acquisition rates as low as 100k.  Even reading all 32 lines this should only be 4 Bytes * 100kHz = 400kB/s.  Most laptops should be able to write to disk at least at a few MB/s.  How are you writing to file?  Are you using binary files or ASCII or other?  How many data points were you attempting to write each time?  Were you opening the file each time you attempted to write, or did you leave the file open and simply add data each iteration of your acquisition loop?

Thanks for working with me Mark.

Have a great day!
Travis W

[Buffning]

I don't, unfortunately, have any more ram I can add to try it.  The reason why the older version, where I take in less data with block_in, doesn't work must be a limitation with the card and not with the writing to disk because as you say it should write to the disk fast enough.  To save to the disk, I created a pointer to the file by mapping a view of the file and filled in the file through the pointer as I got the data, which means the file was binary and stayed open the entire write process, it would write as many as the buffer, I guess, to the file at a time (not sure how it handles that part).  I tried keeping everything in RAM and it still wasn't fast enough.  I've been talking to tech support and it seems like the card (6533PCMCIA card) is limited to less than 10KHz.

I'm still confused as to why 2^25 is a limit for the count value in Block_In, but I my card doesn't seem to be up to my specifications anyways, so it's not too important.

Thanks for the help,
Mark