IMAQdx Grab images using buffer in memory

[BruceAmmons]

I think the simplest solution is to create your own buffer.  Use a relatively small buffer for acquisition (a few seconds), and rapidly copy the images as they are acquired into your buffer.  It may speed things up to preallocate the size of each image in your buffer.

Bruce

[Baturin]

Bruce, thank you for the suggestions.

I think it should work fine.

[Orbital]

I'm trying to set up something similar and after reading Bruce's buffer suggestion I found this post which might be useful to you also:

http://forums.ni.com/t5/LabVIEW/queueing-the-IMAQ-image-data-type/m-p/1110228#M490517

I'm going to try implementing that idea soon.

[nemi]

---

@Baturin wrote:

Such data stream is quite fast and it is not possible to just save it to HDD. Although I will soon experiment with RAID, 

---

On the high speed storage front, Typical hard drives max out at about ~120 Mbit/s for sequential read/writes.

RAID 0 would double that but most (cheap) RAID solutions are software and would eat up a few % of your CPU.

RAID 5 or other parity based raids can really eat into your CPU and you should look at a true Hardwar RAID 5 solution or you may miss frames.

I don't think you need to go that far myself as you commented you were only going to capture a few minutes of video.

For your application I would strong recommend an SSD (solid state disk) . 240GB models are now ~$150 or less and can do sustained sequential  write speeds of over 300 Mbit/s , (usually 400+  check the specs on the one your are buying). Transfer speeds will go down once you fill more than ~80% of the drive, so make sure you over provision capacity.

Also make sure you plug the modern SSD into a modern SATA 3 (600 Mbit/s) interface. SATA 2 or 1 will limit you to 300 and 150 Mbit/s respectively.

Other thoughts:

Again you can RAID 0 a pair of SSD. I have 2x128GB models in RAID 0 and hit peak reads of over 900 Mbit/s.

Use the SSd in this application as your video capture scratch space, have no other files or OS on it so nothing else is interrupting with I/O requests.

Copy your video off to a HD after capture.

[Baturin]

---

@nemi wrote:

---

@Baturin wrote:

Such data stream is quite fast and it is not possible to just save it to HDD. Although I will soon experiment with RAID, 

---

On the high speed storage front,

---

---

Thank you for your comments. Actually, before starting a project we have approximately same ideas and calculations.

We start from the simplest - RAID0 with two drives on integrated Intel raid controller.  Really, combining with 16G memory and LabVIEW x64 everything is already working smoothly. Couple of minutes record is ok and CPU load (there is i7 with 8 virtual cores) seems to be almost negligible without video encoding.

Really we don't need to write such big amount of data onto disks. The picture is quite static, so any compression, even lossless allows several times reduction in file size. I am using free utvideo codec which can be used both in x32 and x64 (formerly we have used huffyuv, which is quite fast) but is doesn't work in x64. utvideo is also fast enough and doesn't make things worse. File size reduce considerably and overall processing speed increases a little. This costs about 10-15% on several cores, which is ok for us.

So, we are already happy even without SSD raids and using custom buffer. The software is only for inhouse utility usage, so the amount of work for its polishing should be limited according to famous 80/20 rule :).

[CAPTIC_LA]

LV_Warrior, I found that post because i noticed the same problem as you.

Il also use "Overwrite mode = Fail" and Get Image with "Buffer Number" and I get the error "Requested buffer number is unavailable" juste after half the buffers are used.

Your attached code is behaving the same way with my hardware.

Dis you have any interresting answer to your questions or did you find out a way of avoiding such situation?

Thanks

Lionel

[LV_Warrior]

Thank you Lionel. Pleased to see I am not alone in the observation the grabbing crashes at half of the buffers.

I did not really solve it. What I do now is:

• first allocate 2000 buffers to be used in combination with a que.
• Then I start grabbing, send each frame to it's allocated buffer, and put a reference pointing to that buffer into the que.
• I start a parallel process "processing" these frames after receiving their reference from the que
• So far that "processing" has been taking the average value of the frame and storing the frame on SSD
• In the main loop I count till 2000
• And then? here it comes: THAN I STOP STREAMING THE FRAMES,

                                                FINISH PROCESSING THE LAST QUED REFERENCES

                                                AND RESTART THE GRABBING LOOP FOR THE NEXT 2000 FRAMES

So NO I do not have the desired continuous streaming. This approach shifts the load away from the funny grabbing algorithm to more comprehansible structures. The next bottleneck when processing is slower than grabbing is the memory space available in your computer. As it is not enough anyway I decided I might just as well take a break at 2000.

Using SSD, I can now "process" frames of 640x480 pixels x 8bits at 1400FPS

Best regards, LV_Warrior

[CAPTIC_LA]

Ouch! 

Thanks LV_Warrior for your quick reply...

Unfortunately i guess i will have to find a "workaround" also for my needs... not a good point for NI!

[LV_Warrior]

I also believe this grabbing block is insufficiently documented.

How the inputs and outputs behave exactly is not decribed.

Why can't we just access the values of the read and the write pointer ?

What are the attributes for and why do we have so few ?

The low level grab example is a poor demonstration of the possibilities that ought to be covered by the grabbing block.

I have been struggling with continuous grabbing mode for more than a year now.

It is stunning to see on this forum that there is still no news. This thing is smelling fishy.

[BlueCheese]

I'm sorry you find the documentation lacking. We are working on improving both some of the documentation in the acquisition concepts manual as well as adding some new features that make it both easier to use and more efficient. I am curious to know why you find the examples a poor demonstration of the grabbing functionality.

You have many different (vague) questions listed here. If you have specific ones I can certainly try to answer them here.

To address some ones in general:

-The acquisition model IMAQdx currently uses relies on internal buffers (of which you can control how many there are) that the camera acquires into. You then use the Get Image (or similar higher-level functions) to fetch that data into an image reference. This decodes the image data as needed into the format the Vision libraries use.

-The acqusition model of the internal buffers if FIFO-based. the driver attempts to keep 50% of the buffers are queued to the hardware at all times, and buffers that have been re-queued can no longer be retrieved until they are filled again.

-If you want to access the image content directly after this step, you can use the EDVR set of functions in the Vision palette to access the raw image array data behind the image datatype

Eric