Binary write to file slowing down my program

[dre99gsx]

I have noticed this more on some types of computers, but I have two threads going.  One that captures 10MB images and queues them, and another thread that dequeues them and appends to a binary file.

If I disable the write to file, program chugs along well.  Once enabled, I noticed that the image capturing begins to act sluggishly, almost in sync with the writing to hard drive.  I have tried changing the write loop priority to the lowest as well.

It isn't holding up at the queue/dequeue step either.  Any pointers on what to try?  I'm almost certain that there is a memory sharing issue where one process is writing to 10MB image to memory, while the other is reading a 10MB image from memory, and this is tying things up.

[crossrulz]

Are you limiting the size of the queue?  How are you writing the file?  Where are the images coming from?  Can you share any code?

[dre99gsx]

Tough to pull code out for sharing, but it is very simply written. 

No queue limit setup.  As I queue, it empties the queue and writes quick enough to keep the queue at 1 image max.  This is a Firewire camera, thus its throughput is about 300ms per image.  I timed the write to disk at about 200-300ms per image as well (Dell T7400).  I am writing it U16 direct to a binary.  Nothing special done there.  I open a file handle before hand, so no opening/closing of handles.

I've disabled certain portions of my code to diagnose.  Seems that it has no problem running at full speed, capturing images, queuing them, dequeing and preparing the array for writing into a binary, only when I temporarily disable the single call to "Write Binary" intrinsic Labview function.  

[nathand]

You mentioned that you tried lowering the priority of the writing loop. Is it running in its own VI? Have you tried moving it to a different execution subsystem? Are you using timed loops, or normal while loops?

[dre99gsx]

Did some more investigation.   Nathand, you may be on the right track.

No timed loops.  Normal while loops.  

The main VI which hosts both of these loops is setup as User Interface (get images from camera subvi in a while loop, and write image to file subvi in another while loop).  The vi which Dequeus and writes to file was set up as "same as caller".  I toyed around with the execution and set it up as "standard".  It seems to now have shifted the priority to the image capture subvi, where at the end of a few snapshots, my write-to-file is still dequeuing the images.  This is preferred, but interesting how subtle this change is.

[nathand]

---

dre99gsx wrote:

The main VI which hosts both of these loops is setup as User Interface

---

There's the problem, I think. The user interface subsystem is special - it's a single thread. This is required for certain things (interactions with the operating system and user interface, some external DLLs and ActiveX components) that require a single thread. In general, don't run your code in the user interface system unless you have a specific reason to run it there. As an example, property nodes that refer to front-panel items cause a switch to the user interface thread, so if you have VI that uses a lot of references to update the front panel, that VI should run in the UI system. In your case, if you switch your main VI to Standard and leave the subVIs set as Same as Caller, I think you'll see a similar speed improvement. There's more information about LabVIEW threading here, although that information is slightly out of date; if I remember correctly (couldn't find the reference in a quick search) LabVIEW now allocates as many threads per system as there are processor cores in your CPU, but not less than 4 per execution system with the exception of the user interface.

[dre99gsx]

Gotcha.  I do have a reason to run the Main vi as User Interface.  It interacts with Excel ActiveX, which didn't function properly unless I forced the Main vi as User Interface.  Basically, I changed execution from sameascaller to userinterface to attempt to alleviate "The application called an interface that was marshalled for a different thread" when loading pages from Excel. 

Thanks!