Simultaneous Image Acquistion from RT CompactRiO Target

[bilaln]

We have a 2-channel AF-1502 MoviMed FPGA module for RT image acquisition. We have two cameras, each connected to one of the MoviMed channels. We would like to acquire images from each both cameras simultaneously (for stereovision purposes). However, the MoviMed FPGA can only acquire from one channel  and output one image at a time.

To work around this, we have tried to write a program that simulates simultaneous capture by quickly switching between the channels after every iteration. However we get time-out errors a few seconds after starting the program.

With this in mind, we have several questions:

1. Is the time out due to memory, i.e. the FPGA's RAM is not clearing when we switch between channels?

2. When acquiring from different sources, the way to change the input channel on the multiplexer is via an I2C message, which takes time to send and time to execute. Will this limit our frame rate drastically? We would like at least 10 FPS.

3. Instead of channel switching, the other option would be to purchase a second MoviMed module for true simultaneous acquisition. In this case, will the Compact RIO be able to handle the processing? Our system consists of: cRIO-9025 controller, NI-9104 chassis, and AF-1502 MoviMed framegrabber. We perform the image processing routines on the host PC via shared variable architecture.

4. Is there anyway to pass Session data from the image capture VI using an I/O shared variable and have the host machine configure the device using this method? If so, how do we choose the alias?

[BlueCheese]

Here's my ideas below in bold.

---

@bilaln wrote:

We have a 2-channel AF-1502 MoviMed FPGA module for RT image acquisition. We have two cameras, each connected to one of the MoviMed channels. We would like to acquire images from each both cameras simultaneously (for stereovision purposes). However, the MoviMed FPGA can only acquire from one channel  and output one image at a time.

To work around this, we have tried to write a program that simulates simultaneous capture by quickly switching between the channels after every iteration. However we get time-out errors a few seconds after starting the program.

With this in mind, we have several questions:

1. Is the time out due to memory, i.e. the FPGA's RAM is not clearing when we switch between channels?

Unknown - This would likely be something inside of MoviMed's IP to figure out...

2. When acquiring from different sources, the way to change the input channel on the multiplexer is via an I2C message, which takes time to send and time to execute. Will this limit our frame rate drastically? We would like at least 10 FPS.

I imagine much of the delay is locking on to the analog signal... You'd have to confirm with MoviMed where the delay is and how long it can take as it is dependent on their hardware's design/

3. Instead of channel switching, the other option would be to purchase a second MoviMed module for true simultaneous acquisition. In this case, will the Compact RIO be able to handle the processing? Our system consists of: cRIO-9025 controller, NI-9104 chassis, and AF-1502 MoviMed framegrabber. We perform the image processing routines on the host PC via shared variable architecture.

I believe the limit of the MoviMed module is the backplane bandwidth back to the cRIO controller. With a second one you would likely have half the max frame rate of each device (just a guess). Additionally, you have to consider the overhead of sending that data over the network. I do not know offhand if this will be significant for your use case.

If the data ia alsways going straight to the network over shared variables, why not consider IP cameras attached to the network rather than the cRIO?

4. Is there anyway to pass Session data from the image capture VI using an I/O shared variable and have the host machine configure the device using this method? If so, how do we choose the alias?

My hunch would be no, unless you simply talked too the FPGA refence directly over the network rather than from the controller. But this would be dependent on the driver API MoviMed provides.

---

[Alvin6688]

I'm the OP's partner on the project. Thanks for the reply BlueCheese. Just wanted to provide some follow-up info.

Our setup:
Detector: two cameras, several cm apart
FPGA: 2-channel AF-1502 MoviMed (BNC connector)
Controller: cRIO-9025 (best available)
Chassis: NI-9104 (may need upgrade?)

Problem description:
After getting time-out errors a few seconds after starting our switch program, we guessed it to be a speed problem, so we added delays of up to 2000ms between each iteration to slow down the switching. Yet we still received the time-out error. We then noticed that the program could run for a longer duration (before timing out) the first time we run it; but as we successively re-ran the program, the duration before time-out became progressively worse. Thus we guessed the problem to be a memory (RAM?) issue. However when we monitored the CPU usage, it was never any higher than the usual single-channel acquisition (~75%).

Very strange, no? If anyone as any idea what's going on, please let us know. We've sent the questions to Movimed as well, so hopefully we get a response soon.

We also bought a second MoviMed module so when it comes in we'll try having them both run at the same time. Hopefully that will work, and we won't need to upgrade the chassis.

[Cody C]

Good Afternoon,

Do you have any updates from your correspondence with MoviMed?  I think their answers to some of your questions will be very helpful considering the extent this issue deals with their module.  I hope this helps and please keep us updated!

-Cody C

[Greg_S-Video]

Hello Alvin6688

Would you consider posting your code for this application?  I would very much like to see this issue resolved on your end!  Perhaps you could create a VI where you can reproduce this issue on the smallest possible level?  That would greatly assist the community in helping you troubleshoot your issue.

Thank you for choosing National Instruments!

[cagell]

Hi All,

I am an Applications Engineer at MoviMED and would like to post the response to the issues that Bilal and Alvin had in the past with the simultaneous acquisition with multiple framegrabbers on the cRIO platform. We followed up using private email threads but I thought it would be worth posting the findings here so that if anybody else is in need of this application they have a reference to look at.

The AF-150x drivers are originally thought to be used with a single AF-150x per system. The mechanism the driver uses is descrived in the user manual but basically the driver has two parts

a) FPGA driver : Collects the image at the low level and sends it to a DMA FIFO

b) RT driver : reads the DMA FIFO and puts the image together

It turns out I had to modify the driver to make the system work with multiple framegrabbers. That is because now two low level loops are needed to acquire from two different sources, so two sets of signals and FIFOs are needed, so the configuration and image acquisition VI's now need to be case-structure-split depending on the slot. Looks like something easy but all in all I had to modify a handful of VIs (14) way more than expected.

I wrote an AppNote for future users. You can see the front panel and code of the resulting example at the end of the code.

The code itself (packaged in a nice project) is available, just send me an email over (info@movimed.com) or a message through my user. In order to have the code working though you will need:

- LabVIEW

- NI Vision

- LabVIEW FPGA

- LabVIEW RT

- The AF-1501 driver

To close this post let me answer really quickly several questions that came up in this thread: (My answers are in red)

We have a 2-channel AF-1502 MoviMed FPGA module for RT image acquisition. We have two cameras, each connected to one of the MoviMed channels. We would like to acquire images from each both cameras simultaneously (for stereovision purposes). However, the MoviMed FPGA can only acquire from one channel  and output one image at a time.

To work around this, we have tried to write a program that simulates simultaneous capture by quickly switching between the channels after every iteration. However we get time-out errors a few seconds after starting the program.

With this in mind, we have several questions:

1. Is the time out due to memory, i.e. the FPGA's RAM is not clearing when we switch between channels?

Unknown - This would likely be something inside of MoviMed's IP to figure out...

The internal FPGA on the AF-1501 needs to stop acquisition and change to configuration mode when that happens. In the middle of an acquisition you will need transition time to do that. Additionally the change of configuration results in the sending of an I2C message so that makes the waiting time longer.

2. When acquiring from different sources, the way to change the input channel on the multiplexer is via an I2C message, which takes time to send and time to execute. Will this limit our frame rate drastically? We would like at least 10 FPS.

I imagine much of the delay is locking on to the analog signal... You'd have to confirm with MoviMed where the delay is and how long it can take as it is dependent on their hardware's design/

The I2C message takes at least 2 images to be sent (at 30 fps). This means that effectively your framerate will be divided by 6 (see below). Knowing that the maximum framerate you can get from the analog source is 30fps, your maximum framerate is 5fps using this technique.

The division by six comes from this. Assume A is image from channel0 and B is an image from channel 1. T is transitional (wasted) image. The sequence is the following

A T T B T T A T T B ....

The frame rate will be calculated from a frame of A to the next (or B), hence it will be divided by 6.

3. Instead of channel switching, the other option would be to purchase a second MoviMed module for true simultaneous acquisition. In this case, will the Compact RIO be able to handle the processing? Our system consists of: cRIO-9025 controller, NI-9104 chassis, and AF-1502 MoviMed framegrabber. We perform the image processing routines on the host PC via shared variable architecture.

I believe the limit of the MoviMed module is the backplane bandwidth back to the cRIO controller. With a second one you would likely have half the max frame rate of each device (just a guess). Additionally, you have to consider the overhead of sending that data over the network. I do not know offhand if this will be significant for your use case.

Correct, additional AF-150x lead to division of the framerate due to resource sharing (backplane). Still for a 320x240 image 8 bits I was able to send two images over at 11fps each on a NI-9073. Better hardware will allow higher compilation speeds and hence higher framerate.

If the data ia alsways going straight to the network over shared variables, why not consider IP cameras attached to the network rather than the cRIO?

IP camera might be an option but the cRIO allows you to control other gear you cannot control with the IP camera. But it is a valid approach.

4. Is there anyway to pass Session data from the image capture VI using an I/O shared variable and have the host machine configure the device using this method? If so, how do we choose the alias?

My hunch would be no, unless you simply talked too the FPGA refence directly over the network rather than from the controller. But this would be dependent on the driver API MoviMed provides.

The AF-1501 session is an FPGA session and it cannot be passed through a shared variable. Configuration needs to be changed within the controller.

Something that can be done to see an image from a remote PC is using the AF-150x driver at the host PC level instead of the RT level, that makes the code execute in your computer and it saves you a step. This is a trick I do not believe works for an executable but it definately works for development.

Any other technical questions on the AF-150x can be sent to MoviMED using the info@movimed.com email address.

I hope this helps!!!