speed problem saving image series

[Neutronensturm]

I have a .NET program that has to save an array of am saving an array CWIMAQImage images to disk at high speed (the array will be fed by a cameraLink camera). The images are 1344x1024x16bits large. Even arrays as "small" as 20 images can't be saved continuously - the saving seems to stop every now and then. I can't understand this, as I have a P4 3,2 GHz with 4 GB Ram (XP Pro). This can't be a system cache problem, can it? I've already tried isolating the whole thing (separate hdd, switching off windows indexing service and virus scanner for the disk) - the problem stays.

Here a few snippets of my c#-Code:

            object missing = System.Reflection.Missing.Value; // ILDASM tells us, that with VBs optional arguments the CLR creates an object of type System.Reflection.Missing on the stack - C# doesn't have optional arguments...

            // load test images from disk into memory
            CWIMAQImage[] sourceImages = new CWIMAQImage[filesToConvert.Length];
            for (int i = 0; i < filesToConvert.Length; i++)
            {
                sourceImages[i] = axCWIMAQVision1.CreateCWIMAQImage();
                sourceImages[i].Type = CWIMAQImageTypes.cwimaqImageTypeI16;
                axCWIMAQVision1.ReadImage(sourceImages[i], filesToConvert[i], missing);
            }

             // save images to disk
                CWIMAQTIFFFileOptions to = axCWIMAQVision1.CreateCWIMAQTIFFFileOptions();
                to.CompressionMethod = CWIMAQTIFFCompressionMethods.cwimaqTIFFCompressionNone;
                for (int i = 0; i < filesToConvert.Length; i++)
                {
                    targetFileName = Path.GetFileNameWithoutExtension(filesToConvert[i]) + ".tif";
                    axCWIMAQVision1.WriteTIFFFile(sourceImages[i], targetDirectory +  targetFileName, to, missing);
                    backgroundWorker1.ReportProgress((i + 1) * 100 / filesToConvert.Length);
                }

What can I do to solve this? Or is there a way of direct streaming data to the disk?

Thx for your help

Fabian

[Steffen Hoog]

Fabian,

did you read this knowledgebase, already?
http://zone.ni.com/devzone/conceptd.nsf/webmain/60BEEED3818E7E808625705B000511E6

This explains streaming to disk very well. Where are the bottlenecks, what are the options, etc.

Regards,
Steffen

[Neutronensturm]

Hi Steffen,

yes, I know this article. And in the meanwhile I had the chance to try my program on a machine with a two disk SATA2 raid system. It doesn't stop as often, but it does. I did another test on my machine: I read in all the tiff files into an array of byte arrays via a binary stream. Then I streamed the byte arrays back to Disk - almost no performance problem. Not only that it's much faster (about 50ms instead of 90 ms per image), there's only a slight jitter. I cannot say what IMAQ does, but it is not as efficient.

Other software products save their images in huge stack files (multi layer tiff) to avoid the opening and closing of small files. Is there a possibility to do that with IMAQ?

Thx

Fabian

[AZimmer]

I was interested in this problem as well and am also aware of other software that does stack TIFF's together.

  There is also a previous message on the message board that says this may speed up saving images (see: How do i save STACKS of images (e.g. TIFF) as a SINGLE file in IMAQ? 05-06-2001 02:28 AM).
So I thought I would try this, I have a 2 Disk SATA RAID 0 configuration and am able to save 1.4MB 8 bit images (from a cameralink camera) at 28 frames per second, if only this application is running.  If I add a  firewire camera that is capable of collecting 48 frames per second (200*1360 images), the first application slows down to 24 frames per second and the second 42 framers per second (the second camera is saving images to a second 2 disk SATA RAID 0 array ).

  So I thought I would try stacking the images into an array, by converting them into arrays, adding the arrays then converting the added arrays back to a TIFF.

This just slowed the image acquisition rates down a lot or a little depending on how many images I stuffed into one image.  It seems that I am running pretty close to the max frame rates (24 of 28 and 42 of 48) so the loss is unavoidable. 

So, my observations are, it is possible to stuff multiple images into one TIFF by converting the image to an array, adding arrays then converting them back, but it doesn't speed things up.  I have attached the code for the firewire camera with the TIFF bundler, maybe something is slow with my code. 
Andre