How do i store high bandwidth imaging data to disk in "real time"?

[novasecure]

When developing a large area line-scan application, the bandwidth requirements becomes large. For example, an application using a 12 bit 12K pixel line camera with 23,5 KHz, the bandwidth will be approximately 400 MB/s. In large area applications the on-board and pc internal memory will be filled fast, so we need to store data to disk while scanning.

A fast SATA disk today have a rated bandwidth of 150 MB/S, so it seems impossible to utilize the 23,5 Khz with a commercial computer.

1. What is the most cost optimal solution of getting around this problem?

2. If we reduce the Khz to ¼ (bandwith becomes 100MB/s) will the National Instruments cameralink cards installed in a commercial computer with XP be able to store the data to disk at this rate without problems?

[Brent_R]

Calculating data rates can be tricky.  The Camera Link bus packs two 12 bit pixels into 3 bytes, thus on the cable, your data rate would be about 420MB/s.  In PC memory each pixel occupies a full 16 bit word and the data rate is effectively 560MB/s.  This is the data rate you would need to write to disk.

Hard drive specifications can be confusing too.  The 150MB/s data rate quoted below is the theoretical interface bandwidth, for a SATA1 connection, between the disk controller and the hard drive cache.  The actual interface rate rarely exceeds 120MB/s.  This is not a limitation because the real bottleneck occurs in writing data from cache to the physical platter.  A typical 7200rpm SATA drive is good for 60-70MB/s.

All is not lost.  There are inexpensive RAID controllers available that allow systems to combine the bandwidth of multiple drives.  Reading and writing rates scale roughly linearly as more drives are added.  We've done testing here with arrays up to 12 drives and have seen reading and writing rates over 600MB/s.  I've listed our test configuration below as a starting point and have included a link to a set of VIs I wrote for high speed image streaming.

Intel 975 motherboard
One Stop 12 drive chassis w/ PCIe x8 connection
Highpoint 2340 controller
12x Seagate 7200.10

ftp.ni.com/outgoing/IMAQ_WinApi_Streaming.zip

Thanks,
Brent