DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

[GriffinRU]

Inspired from LabVIEW portal - reference link

Library and examples provided as is, minor bugs (aka. features) included but still can be useful for someone who needs to acquire, compress and edit video/audio streams from various sources, directshow compatible devices and internet streams.

Works with GigE cameras if manufacturer provides directshow compatible filter. (Tested with Basler family).

DirectShow has a steep learning curve but once you get familiar with GraphEdit then samples from internet and examples from below should be sufficient to create sophisticated applications.

To back check your LabVIEW creating add

Wire it to your FilterGraph and open in GraphEdit program for inspection and debugging.

Requires:

Microsoft Windows SDK - download

Directshow.net library - download

DirectX.Capture class library - download

References:

MSDN DirectShow documetation

DirectShow Debugging/ConceptProofing Tools:

GraphEdit (Microsoft Windows SDK)

Alphabetical and Numerically Logical String Array Sort for user menu facelift

BEFORE Opening any vi make sure you have in the same directory:

DirectShowLib-2005.dll

DShowNET.dll

DirectX.Capture.dll

logicalstringarraysort.llb

Great links:

H.264 CUDA Encoder directshow filter -> Examples Added 02/16/2013

Directshow virtual video capture source - aka printscreen

Other articles from Maxim Kartavenkov

02/16/2013 Update

H264Encoder requires:

• BaseClassesNET.dll
• H264Encoder.dll

I went with wrapper mode implementation. Check Maxim Kartavenkov forum for details.

H264 CUDA Encoder directshow filter example can be done entirely in BaseClassesNET.dll. Which might be done one day...

No error checking on CUDA capabilities.

Use with Nvidia GPU's only

Examples:

• Capture Video and H264 CUDA.vi
• Render Media Stream.vi

Updates to Library:

• Add H264 CUDA Filter.vi
• Create AviMux.vi - overrite file option added
• Enum Pin Media Type.vi
• Enum PinMediaTypes.vi
• Enum Pins.vi
• Free PinInfo.vi
• H264 CUDA Get Filter Options.vi
• H264 CUDA Set Filter Options.vi
• H264 Object Wrapper.vi
• H264 Settings.ctl
• Media Mode.ctl
• Media State.ctl
• MediaFormatType to String.vi
• MediaSubType to String.vi
• MediaType to String.vi
• AMMediaType to String.vi
• Picture Snap (32-bit RGB Image).vi - updated auto init
• Picture Snap (DIB Image).vi - updated auto init
• Set Sample Grabber.vi
• WaveFormatEx.ctl
• Capture.vi - updated to reflect above changes
• Minor tweaks

02/07/2013 Update

Examples:

• Acquire Video.vi
• Acquire Audio.vi
• Acquire Video plus Sample Grabber.vi
• Capture Video.vi
• Capture Video with Preview.vi
• Capture Video with Preview and Sample Grabber.vi

Updates to Library:

• Release Object.vi
• Init Capture Preview.vi
• Configure Capture plus Preview.vi
• Configure Video with Sample Grabber.vi
• Media Control - StartStopPause.vi

Updates to Capture.vi

• Preview in Capture Mode
• Proper Objects release upon start/stop/exit

Update 10/08/2014:

DShow - JPEG dump example.

-Artur

[Sergiv]

Thanks for the reply and the example reference. I tried - works fine. But from my link (rtsp: //@192.168.1.10/user=admin_password=DuTIbxbU_channel=1_stream=0.sdp? Real_stream) does not work. I know that szhatkie video stream H.264. Maybe because of this, my video does not appear?

As far rtsp streams, you can use 3rd party rtsp filter or directshow filter provided by camera manufacture.

I can not translate it. that means a third party?

this screen shot frommy computer

[Sergiv]

Спасибо за помощь. Я сам попробую, покажу что получится или не получится.

Thanks for the help. I did try, I'll show what happens or does not happen.

[GriffinRU]

Hi Sergiv,

I will reply on russian later. Provided example won't work for rtsp stream.

Example of 3rd party directshow filter: http://sourceforge.net/projects/rtspdirectshow/

-Artur

[GriffinRU]

Так, давайте по порядку.

Какая стоит задача?

• показывать видео с возможностью сохранения индивидуальных кадров
• только сохранять кадры
• проводить обработку видео потока, сохранять индивидуальные кадры в зависимости от обработки...
• сохранять видео и индивидуальные кадры...
• с каким-то интервалом сохранять кадры
• ...

Определившись с задачей, выявим наиболее оптимальный способ программирования, DirectShow, VLC, GStreamer или все в LabVIEW.

Думаю вот так, а найти универсальное решение для rtsp стрима конечно можно, но и с программированием придется попотеть. Также практически все сетевые камеры позволяют получить JPEG стрим напрямую через HTTP запрос-ответ.

-Артур

[Sergiv]

Задача первая

• показывать видео с возможностью сохранения индивидуальных кадров

В принципе, вы мне помогли в соседней теме про VLC. Думаю, это наиболее оптималный для меня вариант.

https://decibel.ni.com/content/docs/DOC-24114#comment-37089

Также практически все сетевые камеры позволяют получить JPEG стрим напрямую через HTTP запрос-ответ.

У меня камера, к которой я IP адрес не сразу-то нашёл ) уже не говоря, про стрим JPEG.

http://www.labviewportal.eu/viewtopic.php?f=22&t=7379 - тут писал. вы, кстати, мне уже на лабвьюпортале помогали с веб камерой как-то.

[GriffinRU]

Сергей, тогда считаем тему закрытой, а по IP камере отвечу в другом форуме в вашей теме.

-Артур

[Sergiv]

Да, спасибо !

[sqrt(-1)]

Very very very useful library.

I've got a question. Is there a way to view the list of video mode supported by a camera? and a mode to set a specific video mode?

Thank you for help

[GriffinRU]

Thank you,

Check DirectShow Capture.vi inside DirectShow SDK.llb.

Run vi -> select video device -> check property pages under Options menu item...

-Artur

[sqrt(-1)]

Thank you. As you suggested me, I've found the way to view and set video mode of a camera.

I tried to grab video with different cameras and this library works very well.

I'm a newbie with directShow.dll and I would have another question. I modified the example Acquire Video plus Sample grabber.VI in order to continuously snap the image of the camera and convert image data to a byte array (I used .NET memory stream to read byte array).

In my example I use a picture box only. This picture box .NET reference is used for SetRenderingWindow.VI and SampleGrabberSnap.VI. With respect to the standard example, I removed the Picture box used for snap and I directly connected the picturebox used for rendering window to the SampleGrabberSnap.VI. It works but is it a correct method?

Morevoer, is there a way to grab camera without using rendering window. I'd like to directly convert acquired image without the rendering window.

Thanks a lot

[GriffinRU]

Hi,

I am confused with your question...

"Acquire Video plus Sample Grabber.vi" was snapping images continuously with 250ms delay, no modification required.

Now is it possible to get image from iether picture boxes, sure, but kinda backwords

Yes it is possible to get image data from sample grabber without rendering.

You can always attach your vi, so it would be clear how to help.

-Artur

[sqrt(-1)]

Thanks for your help

Here are screenshots of my test VI

I'd like to continuosly read byte array of acquired images without rendering and without using picturebox objects. Now, as you can see in my test VI, I continuosly snap and then I read image data (byte array) of the picture box.

[sqrt(-1)]

Yes, you're right. I would need a jpeg stream and I'd like to remove unnecessary code. Unfortunalety, I'm a beginner with DirectShow.

Goal of my application is to perform a jpeg stream and send streaming image data to a remote application.

Thank you again for your help

[GriffinRU]

If all what you need is jpeg stream, you do not need most of the code.

By the way converting from binary stream to jpeg would take some CPU do not expect high throughput with large frame size. If camera suports jpeg stream directly that would be a bonus but framerate probably would drop.

I think you should be able to create a Filter -> add Camera -> set Mode -> Add Sample Grabber -> Run -> Get Data from Sample Grabber -> Stop or continue.

I will try to add example over the weekend.

What is the primary goal for your application?

-Artur

[sqrt(-1)]

Yes, you're right. I would need a jpeg stream and I'd like to remove unnecessary code. Unfortunalety, I'm a beginner with DirectShow.

Goal of my application is to perform a jpeg stream and send streaming image data to a remote application.

Thank you again for your help

[GriffinRU]

Example attached,

-Artur

[sqrt(-1)]

Great!

Thank you again and again. It works fine!!!!!!

[GriffinRU]

Good,

I didn't spent time in optimizing bitmap to jpeg conversion, if you can handle bitmap on your own in your webserver code you should be able to gain more performance. I.e. converting bitmap to jpeg with no reason is wasteful, I would rather convert upon request and share just bitmap pointer.

Otherwise I am glad it worked for you,

-Artur

P.S. Note: If you do not perform any data manipulation with image, then streaming DShow device as jpeg stream/image can be done more effectively with other 3rd party applications like VLC, FFMPEG, GStreamer. (unless dshow device doesn't work properly with mentioned products)

[sqrt(-1)]

Thank you for your suggestions and for your time.

I know that jpeg conversion is the most critical part of my application (from the point of view of performance) but, unfortunately, there is no way to avoid to use it.

[daniel0803]

Nice to meet you GriffinRU

I'm using Image Capture board called "UFG-07 HS"

I wanna switching to "Input Source Type" on LabVIEW VI(ex. Click the boolean)

Can I do it?

Then, Please Tell me the solution and advice.

Thanks.

[GriffinRU]

Probably,

Can you open this property page from LabVIEW? (DirectShow - Pin Property Page.vi)

If yes, I assume some parameters can be passed programmatically, if driver developer didn't expose them, you still should be able to change parameters with windows API tools.

-Artur

[daniel0803]

I Can open Pin property page.

But I don't know how to change parameters(Input Source Type).

I'm beginner developer, so I can't understand your answer well.

Please show me the LabVIEW example(on Blockdiagram).

Thanks.

[neohimura]

hi,

Amazing work with the library. Unfortunately I'm new to DirectShow and I'm kinda lost now.

I'm developing an application similar to a 2-way video chat program via internet (similar to skype). Both user application will have a preview of their own webcam and the feed from the other user which is received via internet.

Due to this, I would like to minimize the bandwidth usage by compressing the webcam video feed on-the-fly before sending it via internet. I believe your library is my best bet to develop this in LabVIEW. I wont be able to use the H264 CUDA library because I'm using a laptop with integrated Intel HD Graphics 4200 GPU. Maybe I can use other x264 codecs which utilizes my GPU.

May I know which one of the example you provided should I be focusing on?

Please point me in the right direction.

Thank you.

[GriffinRU]

Hi,

At this point I can see couple ways to solve your problem:

1. contact the customer support and ask for code sample
2. explore *.inf file on possibility to set custom defaults...
3. explore Windows API on how to access controls from windows handle...
4. Google-it - if someone already developed a code you are looking for...
5. buy capture card for me and I provide example on how to... (over pm)

-Artur

[GriffinRU]

Hi,

Thank you for the comment,

I think this library is wrong option for developing 2-way video chat, it can be done but not in LabVIEW and not with LabVIEW tools. You can find directshow filter which provides such service for webcams and use LabVIEW to integrate in your custom application.

Now, if you still planning to work inside LabVIEW, than the best option for you would be FFmpeg library and based on your programming skills you can do that over DLLs or calling static executables.

Assuming, you are absolutely familiar with real-time audio/video streaming, if not -> the best you would end-up 1-5 seconds delayed video with audio all over the time window.

Again, compressing content, is important but time accuracy and real-time reliability is what makes your application standing out or being lost time effort.

Familiarity with network protocols - is the "must" key to have here.

-Artur

P.S. I am willing to help and provide as much help as time permits, but it is a complex task...

[neohimura]

Hi,

Thank you for your reply. I found a video conferencing API called iConf.NET at http://avspeed.com/. The developer version requires purchase but I don't mind paying for it.

How can I integrate this API into LabVIEW? Or do you have other API/SDK suggestion?

I'm not familiar with FFmpeg and DLL intergration. I was able to run your example "DirectShow Capture.vi" with x264vfw video compressing and AAC audio compressing and the resulting AVi file video and audio was sychnonized.

If it is possible and you don't mind helping, I want to do the whole aquiring video/audio and streaming process in LabVIEW. But if it is a big task, I do not mind using any API/SDK/filter that provides this service and integrate it in LabVIEW.

Please advise.

Thank you very much for your help.

[GriffinRU]

Hi,

I am not familiar with iConf.NET, but implementation in LabVIEW should be identical to C#.

I meant choppy video and audio over the Internet , not locally at PC.

-Artur

[neohimura]

Hi,

Is it possible for me to get the compressed video feed from your SDK so that I can send it on-the-fly via LabVIEW udp or tcp connection?

Or is there a way for me to compress the video feed i get from IMAQdx Grab on-the-fly using x265 or other encoders and send it via udp or tcp?

Thank you.

[GriffinRU]

Hi,

You should be able to accomplish that with my SDK but you need to try it first with GraphEdit, there are "rtsp" read and write filters as well as other 3rd party filters which you can download. Once you figure out how to do that with GraphEdit - switch to LabVIEW.

As far as your second question the best option would be to check my examples with RTSP over TCP and FFmpeg scripting, or VLC scripting.

-Artur

[FrankvdHulst]

Hi,

I have a Conexant CX25858 decoder card, but is not on my development PC. I'm not allowed to get the microsoft SDK on this pc and I'm not experienced enough with the directshow developedment to see the direct problem.

The problem I currently have is a black screen on this conexant CX25858, but the software work on my development PC with my webcam. The only difference I see is the signal of my webcam is RBG24 and the signal from the CX25858 is YUY2. Another reason could be not correctly installed codecs on the PC with the CX25858.

Cheers,

Frank

[GriffinRU]

Hi,

Add "color space" filter which convert yuy2 to rgb.

-Artur

[FrankvdHulst]

Thank you Artur.

I currently have it working in Graphedit (I managed to convince people to install MS SDK on the PC with CX25858). But I have no clue how to add the Crossbar into Labview. Is there an example or maybe somebody who could give instructions?

Kind Regards,

Frank

[GriffinRU]

Hi,

You can add filter by name, obtain name from Filtergraph or try to enumerate dshow filters.

-Artur

[FrankvdHulst]

Hi,

Artur thank you again for your quick response. It was maybe to quick to post my question here, but I solved it. I add it in the same way as WMD capture device (DS - Add Source Filter for Moniker.vi), but instead I search first for Crossbars with AKMSCrossbar. I have it working right now except for the release of the object which it fails to do correctly, but I hope to find the solution soon to that part.

Cheers,

Frank

[GriffinRU]

Hi,

Glad you have it woking out, the key for safe release is following the order, i.e. You shouldn't be able to release an object if something/someone is still attached, i.e. Pulling resources.

Make sure you release/discard object in proper order and you should be succesfull in releasing top calling object. Sometimes you need to give it some time...i. Terms of wait function

-Artur

Labview speaking...

[FrankvdHulst]

Hi,

But I guess safe release is in the same order as you did in your examples, but this is still giving me failure at releasecomobject() or should I first release the capture device in some kind?

Cheers,

Frank

[GriffinRU]

Hi Frank,

Check with GraphEdit before releasing your object which filters are still in memory, if you find the ones which you never called than you should release them as well. (dshow would try to complete the chain of mismatched filter pins properties, if it can, by placing additional filters to match them.)

-Artur

[FrankvdHulst]

But I can just release the filtergraph can't I. There is no need to release every filter?

Further I can't get my capture model working. I have delete every filter in my filtergraph and afterwards rebuild it again including to set the video user format. If I check the rebuild fitler graph it is build according how I want it and which should work, however the image is stopped(preview in picturebox) but the it is recording and I can open afterwards the file in VLC. Should I release the image from the picturebox too?

[FrankvdHulst]

To add on my previous comment. If I only use one camera it works fine, but as soon as I try to capture and preview two cameras the preview is not running anymore, however it is recording. I still use the CX258585 and open it on multiple filtergraphs, so I can assign every preview in a different picturebox.

[GriffinRU]

You can release filtergraph, but you need to make sure that none of the resources are trapped either by LabVIEW or by dshow filter.

Can you create in Filtergraph exactly what you want?

If yes, than you just have an error or misunderstanding how to create rendering window.

And you do not need to release picturebox, but an image bonded to picturebox "yes".

You should be able to create multiple filtergraphs and run them at the same time at different rendering windows with no problems. Make sure you do not share pointers by any means...

-Artur

[viScience]

Hello - I have an application that needs to acquire tirggered images from 2 Bassler GiGe 2048 line scan cameras.  The frame rate will be 12 fps x 2 = 24 fps aggregate into one PC.  Currently it looks like I can do TIF compression in realtime on one camera but I am leary about 2 cameras in realtime. Could you recommend or sell an optimized PC and GiGe interface that can stream directly to disk without having to pass the data through LabVIEW first.  During the day the system will run continuously storing images.  At night the images will be post processed and a IMB barcode will be read and then the images will probably be recompressed using something like JBIG that segregates images into text and image regions since we will want super high compression (we need to archives 2 years of images) but still want to be able to read the lettering on USPS letters.  I would like to buy as much as possible off the shelf, do you have any suggestions?

[GriffinRU]

Hello,
If you do not want to pass data through LabVIEW -> Use FFmpeg with DShow filter from Basler Pylon software. With FFmpeg you should be able to collect data from both cameras and compress them on-the-fly (probably 8+ core system with 8+ GB - would run under 35% CPU with preview or less without). The only trick would be how to deal with triggering...
We can discuss more in-detail over the PM.
-Artur

[FM_AM]

i am new to Labview SDKs. kindly update me if i want to run DirectShow.Net labview SDK on windows 7 what files i want to download and in which sequence. Kindly elobrate

regards.

[GriffinRU]

BEFORE Opening any vi make sure you have in the same directory:  DirectShowLib-2005.dll  DShowNET.dll  DirectX.Capture.dll  logicalstringarraysort.llb

[FM_AM]

Can u please tell me what files to downlaod.do i need tobdownload SDK for windows 7?

Sent from Yahoo Mail on Android

On Wed, Jun 8, 2016 at 5:23 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

BEFORE Opening any vi make sure you have in the same directory:  DirectShowLib-2005.dll  DShowNET.dll  DirectX.Capture.dll  logicalstringarraysort.llb

Reply to this email to respond to GriffinRU's comment.

[FM_AM]

My dear friend i am not clear from post how to run this sdk.Kindly elobrate me which files do i need to download from forum to use this utility.. 

Sent from Yahoo Mail on Android

On Wed, Jun 8, 2016 at 5:31 PM, Electrical Engineer<electricalengineerislamabad@yahoo.com> wrote: Can u please tell me what files to downlaod.do i need tobdownload SDK for windows 7?

Sent from Yahoo Mail on Android

On Wed, Jun 8, 2016 at 5:23 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

BEFORE Opening any vi make sure you have in the same directory:  DirectShowLib-2005.dll  DShowNET.dll  DirectX.Capture.dll  logicalstringarraysort.llb

Reply to this email to respond to GriffinRU's comment.

[GriffinRU]

Hi,

You need to download and install:

Microsoft Windows SDK (if you plan to use GraphEdit)

Directshow.net library

DirectX.Capture class library

LabVIEW code:

DirectShow SDK.llb

DirectShow Examples.llb

-Artur

[FM_AM]

I have downloaded the mentioned libraries but there is no installer in source files.

Is there anyother software required  such as Visual studio or .netframework for installing.. i have attached pics of both source folders downloaded from net. there is no installer in any one of sub folders eitherSent from Yahoo Mail on Android

On Tue, Jun 14, 2016 at 5:32 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

You need to download and install:

Microsoft Windows SDK (if you plan to use GraphEdit)

Directshow.net library

DirectX.Capture class library

LabVIEW code:

DirectShow SDK.llb

DirectShow Examples.llb

-Artur

Reply to this email to respond to GriffinRU's comment.

[GriffinRU]

Ok,

Have you installed Windows SDK?

Can you run GraphEdit?

DirectShow.net and DirectX.Capture class libraries need to be downloaded and only "DLL" from "lib" folder is used, need to be in the same folder as LabVIEW llb's. No special installer required.

DirectShow.net and DirectX.Capture class libraries are nice to have for documenting and studying, only compiled DLL used by LabVIEW SDK.

-Artur

[FM_AM]

 i havent downloaded windows SDK as you mentioned earlier that this is mandatory only for graph edit option. I would like to manipulate with live vedio frames using example vi's as a starting point. I will keep u updated with learning curve and will like to seek guidance from time to time

Sent from Yahoo Mail on Android

On Wed, Jun 15, 2016 at 7:09 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Ok,   Have you installed Windows SDK?  Can you run GraphEdit?     DirectShow.net and DirectX.Capture class libraries need to be downloaded and only "DLL" from "lib" folder is used, need to be in the same folder as LabVIEW llb's. No special installer required.     DirectShow.net and DirectX.Capture class libraries are nice to have for documenting and studiyng, only compiled DLL used by LabVIEW SDK.     -Artur

Reply to this email to respond to GriffinRU's comment.