latency of FPGA imaging processing

[quanwen]

Hi,

I have been writing codes for fast real time imaging processing using Labview FPGA 2014 as well as the vision development module. The idea is to (1) acquire the image, (2) transfer the image from host to fpga target via FIFO,  (3) process the pixel data through thresholding and morphological operation, and (4) finally find the centroid of the object in the image. (3) and (4) can be implemented via fpga VIs provided by the vision development module.

My question is: For image size (1024x768), what is the typical latency to process a single frame via FPGA? From some of the documents I read from the NI website, the time delay would be ~ 10 ms, which is more than I would expect. If I increase the single clock cycle frequency to 200~300 MHz, would this work?    

Thanks a lot for your help.

Best,

Quan

[ChristopheC]

Hi Quan,

In the current shipping version (14.0 or 14.5), this information is available in the help file for each FPGA VI and you have to manually add that up, based on the VIs you use.
In the upcoming 2015 release, which should be available for download from the National Instruments website very soon, this information will be available from the Vision Assistant Performance Meter. (Tools>>Perfomance Meter)

Hope this helps.

Christophe

[quanwen]

Hi Christophe, Thank you for your reply. This is very helpful. Given these information, and in order to reduce the overall latency, is it feasible to perform preprocessing (threshold, morphology and centroid) on the pixel stream during the image acquisition phase? I am using a 10 Tap setting, and it is no clear to me how to manipulate all the 10 pixels within single clock cycle (100 MHz)? Another naive question, how to change the pixel bus data type from U1 to U8 in fpga VI? I did not find VIs in the corresponding panel. Thanks again! quan

[ChristopheC]

What is your hardware? Given the fact that you're mentioning 10-tap and 100MHz, I assume that you're using a NI1473R with a 10-tap camera.

While the API only currently support 1 pixel processing, it is possible to paralellize the FPGA code to process 10 pixels at once.

We ship an example that you should be able to modify to work with 10-taps by duplicating and paralelizing some code.

C:\Program Files (x86)\National Instruments\LabVIEW 2014\examples\Vision FPGA\Image Processing with Vision Assistant on FPGA\Image Processing with Vision Assistant on FPGA.lvproj

Take a look at the FPGA VI: C:\Program Files (x86)\National Instruments\LabVIEW 2014\examples\Vision FPGA\Image Processing with Vision Assistant on FPGA\Image Processing with Vision Assistant on 1473R (FPGA).vi

I think our Product Support Engineer already helped another customer with a similar use case. I'll ask him to take a look at this post and see if he already has an example he can post.

> Another naive question, how to change the pixel bus data type from U1 to U8 in fpga VI?

Yes, you will need that if you want to do some binary morphology prior to the centroid.

In Vision Assistant, you will find the Conversion function in the Grayscale palette (just above the centroid).

In LabVIEW, it's in the Vision>>Image Management palette. It the Cast VI (IMAQ FPGA Cast.vi)

For your processing, I would recommend using Vision Assistant (or the Vision Assistant Express VI) that exposes most of the FPGA functions available in the library, and can generate the LabVIEW FPGA code for you. It handles all the 4-wire handshaking and feedback nodes for you, that can be tricky to wire in some cases.

Hope this helps.

Best regards,

Christophe

[kewei.qian]

Hi Quan, The image is acquired and transfered to FPGA directly through Cameralink port. In my opinion, As the maximum chipset operating frequency is 85MHz , the frequency of FPGA that is 100MHz is enough. It doesn't waste time in tranfering data. The difficult point is how to manipulate all the 10 pixels within single clock cycle (100 MHz). Maybe we should parallel computing pixels based on some algorithm. Is the U1 data type you mentioned binary data after threshold? I think you can map 0 to 0 and 1 to 255 , so that you can change it to U8. Kewei

[quanwen]

Hi Christophe,

Thanks a lot for your quick reponse.

> What is your hardware? Given the fact that you're mentioning 10-tap and 100MHz, I assume that you're using a NI1473R with a 10-tap camera.

You are right. I am using a PCIe-1473R with a 10-taps camera link camera. I will look into the example you mentioned. Just as Kewei posted below, the challenge is in processing all 10 pixels at once during a single time cycle. I will keep you updated about the working progress.

>I think our Product Support Engineer already helped another customer with a similar use case. I'll ask him to take a look at this post and see if he already has an example he can post

Awesome!  that will be really helpful.

>Yes, you will need that if you want to do some binary morphology prior to the centroid.

In Vision Assistant, you will find the Conversion function in the Grayscale palette (just above the centroid).

In LabVIEW, it's in the Vision>>Image Management palette. It the Cast VI (IMAQ FPGA Cast.vi)

Got it! 

Cheers,

Quan

[ChristopheC]

What is the bit depth of your camera? If it's 8 bit, you can try to change the instance of the VI that returns the pixel data to 10 Tap 8 bit, use 10 local FIFOs, and duplicate the processing 10 times in parallel. That would be the way to processing 10 pixels in a clock cycle, since the API only supports 1 tap for now.

You will then need to pack the pixels to send them back to the host. I emailed the PSE, and he will hopefully answer tomorrow.

Christophe

[quanwen]

>What is the bit depth of your camera? If it's 8 bit, you can try to change the instance of the VI that returns the pixel data to 10 Tap 8 bit, use 10 local FIFOs, and duplicate the >processing 10 times in parallel. That would be the way to processing 10 pixels in a clock cycle, since the API only supports 1 tap for now.

>You will then need to pack the pixels to send them back to the host. I emailed the PSE, and he will hopefully answer tomorrow.

Thank you for your suggestions! Indeed, it's a 10-tap 8 bit camera. I will try to implement it!

Best,

Quan

[altran]

Hi Quan,

Could you direct me to the NI document that mentions the 10ms delay for frame latency?

Thanks.