LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Access compressed live video bitstream

Solved!
Go to solution

Yes, that particular webcam has on-board h.264 video compression, and it should be communicating that compressed data to the computer somehow. The h.264 encoded bitstream between the camera and the DirectShow input filter, or whereever that data goes to, is precisely what I want to access.

My application is not contained in one computer. I need that compressed bitstream since I'll transmit it through a bandlimited communication channel (it is a different channel that we constructed with special transmitter and receivers, not the one between the camera and computer). Since it is a bandlimited channel, my max data rate will be limited and I need to squeeze as many information as possible per second, and compression makes my job much easier in that sense.

0 Kudos
Message 11 of 16
(1,524 Views)

Well I'm starting to understand what you want to do, but that is not gonna work like that. The Logitec camera supports H.264 compression over the USB link. But!!! The H.264 data stream on the USB link is embedded in the USB protocol. There is no easy way to get at that data directly. The Logitec driver translates this data stream back into a DirectShow source stream, which is basically a series of image frames, one frame per image. The DirectShow input filter is responsible for translating the data stream (in your case the supposed H.264 compressed data) into a format that can be processed further in DirectShow, for video this is basically a stream of images which might or might not use some form of bitmap compression, but not a byte stream compression. IMAQdx is specifically an image acquisition interface and therefore always delivers image frames (and requires them from any frame grabber it can access). It is specifically NOT a streaming interface that could be used to retrieve the (compressed) bytestream from a camera directly and transfer it as is to some other place.

 

In order to access the H.264 data stream you would have to replace the Logitec DirectShow driver with a different driver that takes the data directly from the USB bus and sends them then further to your supposed final application through your bandwidth limited channel. While the latest USB camera profile 1.5 does support H.264 data there is absolutely no guarantee that Logitec uses this standardized USB profile rather than their own proprietary protocol. Even if they used the standardized USB profile it is unlikely that you will find a USB driver that you can let connect to the camera and directly retrieve the H.264 bytestream from it. Basically your use of a USB camera makes this an almost impossible exercise. If you had used an IP camera with H.264 support or similar, you have the compressed bytestream directly on the network wire and then only need an according driver that can connect to this stream and deliver it at the end point to your final application. Here one of the IP camera to DirectShow input filters can be used.

Rolf Kalbermatter
My Blog
Message 12 of 16
(1,521 Views)

Rolf, thank you so much for your continuous replies, they've been very helpful. I was naively hoping that IMAQdx gets h.264 bitstream from the webcam and takes care of the decoding itself, and maybe I could access that data stream before decoding. Now I see better what is going on with DirectShow and Labview.

I've seen online that people had access to h.264 bitstream from a USB webcam (please see examples below), but they use low-level frameworks instead of Labview. They are mostly (if not all) in Linux and using Video4Linux (v4l), FFmpeg with GStreamer. I am not familiar with these, and have no idea how to encorporate these or the data I got from these (if I can get any) in Labview. Do you have any information on how to combine these with Labview?

 

Examples:

https://stackoverflow.com/questions/25086843/how-to-access-the-h264-stream-from-webcams-that-feature...

https://wiki.matthiasbock.net/index.php/Logitech_C920,_streaming_H.264

 

0 Kudos
Message 13 of 16
(1,511 Views)

Well V4L is on Linux an equivalent to DirectShow on Windows. A framework to access video sources and present them to the user. But on Linux it is open source and written in C(++). gstreamer is another framework/application that can handle video and image acquisition through its own plugins (and of course also has a plugin to access V4L sources). As this is all open source someone who really knows can hack into this and make things work differently than how it was initially meant. No such option exists for DirectShow.

 

Your best bet would be in your first link. Apparently H.264 capable DirectShow input filters are supposed to provide a separate additional IPin that applications in the know can connect to to directly receive the compressed bitstream. IMAQdx does not do that since it does not have any benefit from accessing a compressed bitstream that it then has to decompress anyway using a patent encumbered algorithm in order to turn it into the IMAQ image stream that everything in IMAQ works with. 

So you would have to find a DirectShow interface library that can connect to this IPin and directly hand you the data in its binary form. Since DirectShow is COM based, you can not use the ActiveX functionality in LabVIEW to access it directly. COM is one of the building blocks of ActiveX but not sufficient to be ActiveX. Maybe there exists a .Net component that exposes the DirectShow interfaces in some way but I would guess that it only exposes a subset of the whole DirectShow interface. Personally if I needed to do this I would "simply" write a DLL in C(++) and do all the COM stuff in there to access the special IPin and then interface that DLL to LabVIEW. But that is a several man week project, also because I never really did more than dabble a bit with DirectShow directly and would need to learn a few things first.

And once you have that working you have at the other end a H.264 compressed bitstream that you somehow have to decode again. Here you will have the same problem really, but in reverse. You need to plug somewhere into a H.264 decoder. If it is a Windows computer you would have to access a direct feedthrough DirectShow input filter that you can hand the bitstream and that then passes that bitstream without any modifications to the H.264 decoder DirectShow filter indicating that they need to treat it as H.264 datastream. It's all possible, DirectShow is EXTREMELY flexible and designed to just about handle everything from juggling with 10 balls and at the same time also recite Shakespeare, to running your entire household, but to convince it to do that is an art in itself. Smiley Very Happy

 

Trying to use the infrastructure from gstreamer, mplayer, vlc and similar on Windows is likely an even more complex project that would probably take several man months.

Rolf Kalbermatter
My Blog
0 Kudos
Message 14 of 16
(1,504 Views)
wrote:

 

It's all possible, DirectShow is EXTREMELY flexible and designed to just about handle everything from juggling with 10 balls and at the same time also recite Shakespeare, to running your entire household, but to convince it to do that is an art in itself. Smiley Very Happy

 

Trying to use the infrastructure from gstreamer, mplayer, vlc and similar on Windows is likely an even more complex project that would probably take several man months.

This is exactly what I was afraid of Smiley Sad I don't know how to make DirectShow crawl, let alone juggling, reciting Shakespeare, running my household and apparently, saving the world Smiley LOL

Do you know any other "simple" video codec I can use (on-the-fly) in Labview? As I already have access to frames thanks to the mighty IMAQ, can I somehow feed those frames into a .vi or module in Labview that would apply video compression on-the fly and spit out the compressed bitstream for me to transmit? I have some overhead at the receiver, so I can use that time for this inefficient method to generate compressed bitstream at the transmitter end. I am aware that decoding them at the receiver would add some more overhead, but I'd much rather doing so instead of spending months on getting DirectShow to work as I intended. 

0 Kudos
Message 15 of 16
(1,491 Views)
Solution
Accepted by topic author gt77

Update:

I decided to use FFmpeg in Labview in order to access and manipulate raw h.264 packets using this library: https://forums.ni.com/t5/Example-Program-Drafts/FFMPEG-scripting-in-LabVIEW-NET/ta-p/3515713

I contacted the developer and got the confirmation that it would do what I intended to do, and I'll update this thread once (or if) I get it to work.

I am still open to other less-efficient codecs that are easier to use in Labview directly via .vi's or Labview libraries.

0 Kudos
Message 16 of 16
(1,477 Views)