Example Code

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

Code and Documents

Attachment

Inspired from LabVIEW portal - reference link

 

Library and examples provided as is, minor bugs (aka. features) included but still can be useful for someone who needs to acquire, compress and edit video/audio streams from various sources, directshow compatible devices and internet streams.

Works with GigE cameras if manufacturer provides directshow compatible filter. (Tested with Basler family).

DirectShow has a steep learning curve but once you get familiar with GraphEdit then samples from internet and examples from below should be sufficient to create sophisticated applications.

 

To back check your LabVIEW creating add

DsROTEntry.png

Wire it to your FilterGraph and open in GraphEdit program for inspection and debugging.

 

Requires:

Microsoft Windows SDK - download

Directshow.net library - download

DirectX.Capture class library - download

References:

MSDN DirectShow documetation

DirectShow Debugging/ConceptProofing Tools:

GraphEdit (Microsoft Windows SDK)

 

Alphabetical and Numerically Logical String Array Sort for user menu facelift

 

BEFORE Opening any vi make sure you have in the same directory:

DirectShowLib-2005.dll

DShowNET.dll

DirectX.Capture.dll

logicalstringarraysort.llb

 

Great links:

 

H.264 CUDA Encoder directshow filter -> Examples Added 02/16/2013

Directshow virtual video capture source - aka printscreen

Other articles from Maxim Kartavenkov

 

02/16/2013 Update

 

H264Encoder requires:

  • BaseClassesNET.dll
  • H264Encoder.dll

 

I went with wrapper mode implementation. Check Maxim Kartavenkov forum for details.

H264 CUDA Encoder directshow filter example can be done entirely in BaseClassesNET.dll. Which might be done one day...

No error checking on CUDA capabilities.

Use with Nvidia GPU's only

H264.png

Examples:

  • Capture Video and H264 CUDA.vi
  • Render Media Stream.vi

 

Updates to Library:

  • Add H264 CUDA Filter.vi
  • Create AviMux.vi - overrite file option added
  • Enum Pin Media Type.vi
  • Enum PinMediaTypes.vi
  • Enum Pins.vi
  • Free PinInfo.vi
  • H264 CUDA Get Filter Options.vi
  • H264 CUDA Set Filter Options.vi
  • H264 Object Wrapper.vi
  • H264 Settings.ctl
  • Media Mode.ctl
  • Media State.ctl
  • MediaFormatType to String.vi
  • MediaSubType to String.vi
  • MediaType to String.vi
  • AMMediaType to String.vi
  • Picture Snap (32-bit RGB Image).vi - updated auto init
  • Picture Snap (DIB Image).vi - updated auto init
  • Set Sample Grabber.vi
  • WaveFormatEx.ctl
  • Capture.vi - updated to reflect above changes
  • Minor tweaks

 

02/07/2013 Update

 

 

Examples:

 

  • Acquire Video.vi
  • Acquire Audio.vi
  • Acquire Video plus Sample Grabber.vi
  • Capture Video.vi
  • Capture Video with Preview.vi
  • Capture Video with Preview and Sample Grabber.vi

Acquire Video.png

Acquire Audio.png

Acquire Video plus Sample Grabber.png

Capture Video.png

Capture Video with Preview.png

Capture Video with Preview and Sample Grabber.png

 

 

Updates to Library:

  • Release Object.vi
  • Init Capture Preview.vi
  • Configure Capture plus Preview.vi
  • Configure Video with Sample Grabber.vi
  • Media Control - StartStopPause.vi

 

 

 

Updates to Capture.vi

  • Preview in Capture Mode
  • Proper Objects release upon start/stop/exit

DS Capture.png

Update 10/08/2014:

DShow - JPEG dump example.

 

-Artur

 

Example code from the Example Code Exchange in the NI Community is licensed with the MIT license.

Comments
FM_AM
Member
Member
on

I want to run render media stream.vi from directshow examples folder. I want to input the vedio on LAN. How to setup the source parameter for that vi. Pic attached.. and also kindly summarize what will this vi do with input vedio stream

Sent from Yahoo Mail on Android

On Wed, Jun 15, 2016 at 11:39 PM, Electrical Engineer<electricalengineerislamabad@yahoo.com> wrote:  i havent downloaded windows SDK as you mentioned earlier that this is mandatory only for graph edit option. I would like to manipulate with live vedio frames using example vi's as a starting point. I will keep u updated with learning curve and will like to seek guidance from time to time

Sent from Yahoo Mail on Android

On Wed, Jun 15, 2016 at 7:09 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Ok,   Have you installed Windows SDK?  Can you run GraphEdit?     DirectShow.net and DirectX.Capture class libraries need to be downloaded and only "DLL" from "lib" folder is used, need to be in the same folder as LabVIEW llb's. No special installer required.     DirectShow.net and DirectX.Capture class libraries are nice to have for documenting and studiyng, only compiled DLL used by LabVIEW SDK.     -Artur

Reply to this email to respond to GriffinRU's comment.

GriffinRU
Member
Member
on

Hi,

You should "google" - WM ASF Reader and supported media formats.

Render Media Stream.vi - uses WM ASF Reader filter to load stream and exposes available streams to pins. Parsing pins allows you to select particular stream for use with directshow filtergraph.

If you are looking for rtsp stream render - this is wrong vi, you should search for rtsp filter...

-Artur

FM_AM
Member
Member
on

 i have read the google content about .asf files. As far as i understand .asf files contain .wma audio files and .wmv vedio files.  As a beginning point i want to run the" Render Media Stream.vi" to see whether its working with compatible file formats or not. So can u kindly send me the link from where i can get the ASF Reader supported file or mail me a small compatible file for this "vi". Suppose i got the required file from you ,  i have to set the path only in source field or do something else? Can u please share a snapshot of this running vi?

Sent from Yahoo Mail on Android

On Tue, Jun 21, 2016 at 5:20 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

You should "google" - WM ASF Reader and supported media formats.

Render Media Stream.vi - uses WM ASF Reader filter to load stream and exposes available streams to pins. Parsing pins allows you to select particular stream for use with directshow filtergraph.

If you are looking for rtsp stream render - this wrong vi, you should serach for rtsp filter...

-Artur

Reply to this email to respond to GriffinRU's comment.

FM_AM
Member
Member
on

Sent from Yahoo Mail on Android

GriffinRU
Member
Member
on

Hi,

Search for mms stream and try it. If you can play it with Windows Media Player you should be able to play it with LabVIEW.

As fall back try .asf file (you can create one with video editing application ffmpeg, vlc, gstreamer...

-Artur

FM_AM
Member
Member
on

Can u send me a mms streaming file? Its not getting downloaded from web

Sent from Yahoo Mail on Android

On Wed, Jun 22, 2016 at 6:13 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

Search for mms stream and try it. If you can play it with Windows Media Player you should be able to play it with LabVIEW.

As fall back try .asf file (you can create one with video editing application ffmpeg, vlc, gstreamer...

-Artur

Reply to this email to respond to GriffinRU's comment.

FM_AM
Member
Member
on

I have tried. Neither windows player nor vlc nor labview is playing mms stream from net. Actually we intend  to play rtsp streaming vedio over lan and observe the real time delays. Is there any vi in this sdk which can run real time vedio over lan .?is there any requiremnt of graphics card

Sent from Yahoo Mail on Android

On Thu, Jun 23, 2016 at 9:15 AM, Electrical Engineer<electricalengineerislamabad@yahoo.com> wrote: Can u send me a mms streaming file? Its not getting downloaded from web

Sent from Yahoo Mail on Android

On Wed, Jun 22, 2016 at 6:13 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

Search for mms stream and try it. If you can play it with Windows Media Player you should be able to play it with LabVIEW.

As fall back try .asf file (you can create one with video editing application ffmpeg, vlc, gstreamer...

-Artur

Reply to this email to respond to GriffinRU's comment.

FM_AM
Member
Member
on

Am i playing correct vi for stream playing? I mean to ask " Render media stream.vi "is correct vi to play streaming vedio? I am playing it from SDK example folder..should i try any other vi.

Sent from Yahoo Mail on Android

On Thu, Jun 23, 2016 at 6:59 PM, Electrical Engineer<electricalengineerislamabad@yahoo.com> wrote: I have tried. Neither windows player nor vlc nor labview is playing mms stream from net. Actually we intend  to play rtsp streaming vedio over lan and observe the real time delays. Is there any vi in this sdk which can run real time vedio over lan .?is there any requiremnt of graphics card

Sent from Yahoo Mail on Android

On Thu, Jun 23, 2016 at 9:15 AM, Electrical Engineer<electricalengineerislamabad@yahoo.com> wrote: Can u send me a mms streaming file? Its not getting downloaded from web

Sent from Yahoo Mail on Android

On Wed, Jun 22, 2016 at 6:13 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

Search for mms stream and try it. If you can play it with Windows Media Player you should be able to play it with LabVIEW.

As fall back try .asf file (you can create one with video editing application ffmpeg, vlc, gstreamer...

-Artur

Reply to this email to respond to GriffinRU's comment.

GriffinRU
Member
Member
on

Hi,

Try this: mms://stream.lds.org/webcast

from google search... https://tech.lds.org/forum/viewtopic.php?t=16392

-Artur

P.S.

Start working with Microsoft SDK, GraphEdit must be your first tool -> LabVIEW last!

FM_AM
Member
Member
on

Not working still. May be logicalstringarraysort.llb is missing as i cant find it in any downloaded folder mentioned. Also do Cuda filter requires graphics card?

Sent from Yahoo Mail on Android

On Fri, Jun 24, 2016 at 6:06 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

Try this: mms://stream.lds.org/webcast

from google search... https://tech.lds.org/forum/viewtopic.php?t=16392

-Artur

Reply to this email to respond to GriffinRU's comment.

GriffinRU
Member
Member
on

Hi,

Wow, you jumping all over the place...

mms stream should work with "render media stream.vi".

logicalstringarraysort.llb used in different example, targeted primary for directshow devices.

You should start with what you would like to do first, try it with GraphEdit second and than look for examples here.

No magic here, and to use given examples, familiarity with concept is required and additional libraries need to be downloaded/installed.

-Artur

FM_AM
Member
Member
on

I will try to run render media stream with given mms stream and let you know. Can this happen that this vi requires WindowsSDK? I am using windows 7. is SDK operating system specific?

Sent from Yahoo Mail on Android

On Mon, Jun 27, 2016 at 5:11 PM, GriffinRU<web.community@ni.com> wrote:

|

|

NI Community

|

DirectShow .NET LabVIEW SDK (Video, Audio, IP streams)

new comment by GriffinRU View all comments on this document

Hi,

Wow, you jumping all over the place...

mms stream should work with "render media stream.vi".

logicalstringarraysort.llb used in different example, targeted primary for directshow devices.

You should start with what you would like to do first, try it with GraphEdit second and than look for examples here.

No magic here, and to use given examples, familiarity with concept is required and additional libraries need to be downloaded/installed.

-Artur

Reply to this email to respond to GriffinRU's comment.

GriffinRU
Member
Member
on

no comments...

alvaroaldama92
Member
Member
on

Hello,

I find that the library is a great job.

Iam pretty new. I'm trying to use the example "DirectShow - Example - Sample Acquire Video plus Grabber.vi". On the computer with a USB webcam (logitech C270, Windows 10 and LabView 2014) it works properly. But I try to run on a tablet with Windows 10 (https://www.pccomponentes.com/energy-system-energy-tablet-8---windows-lego-edition) and I can see the image of the front camera, but it does not grab me (I get a picture in black, attached capture).

I noticed that the cameras of the tablet have a NV12 format camera and computer has a RGB format. In the case of I only need to transform from one to another, what would you add and where?

P.S .: If I connect the webcam on the tablet, it also works

Thanks in advanvance.

A greetingphoto_2016-09-13_12-50-01.jpg

GriffinRU
Member
Member
on

Hi, Thank you.

I think colorspace filter in front of samplegrabber filter would do the job, but would be nice to check with GraphEdit program but you are on win10 tablet... Good luck

I will be back in ofiice on Friday and check if i can re-produce your case on my PC.

-Artur

alvaroaldama92
Member
Member
on

Thank you very much for answering so quickly.

My main problem is that I do not know how the library works and although I tried to find information, I still do not learn anything. I do not know how to connect terminals (if it had worked, it would have been lucky only).

I created a filter just before the filter "Grabber". But nothing changes. Surely it is not well done .

(I attached a screenshot of where I put the filter)

Thanks again.

Greetings

Captura3.PNG

GriffinRU
Member
Member
on

Hi,

No, not that simple but I am glad your are trying. I think you will do fine with getting how dshow works, just couple examples here and there a little bit MSDN reading and you will be good as it can be.

I will try to re-produce your case later today and hopefully provide you with solution either how to deal with NV12 or how to convert it to RGB at the moment I thought that in frame grabber it should be RGB buffer but let me check.

As far as how it should be done, with your trial:

You need to add filtergraph (you did)

You need to connect input and output pins to other filters.

It is clear when you play with GraphEdit program, but inside LabVIEW requires extra steps, you could ask it to connect filters for you programmatically but without checking entire chain you might end up with interesting configuration.

-Artur

GriffinRU
Member
Member
on

Quick testing not so promising at the moment, but while it is possible to go from NV12 to RGB (with performance hit, re-order bit array before passing it to bitmap), but what other formats available on your Windows 10 tablet, should be others like I420, RGB24, BGR...

-Artur

alvaroaldama92
Member
Member
on

Hello, I was trying to check the availables formats on the tablet, but I do not how to do that. It can be made with the program that you mention or at the windows configuration?

alvaroaldama92
Member
Member
on

Hello,

I am checking the camera sensor and it is an Unicam ov2680.

photo_2016-09-20_09-15-14.jpg

Checking at the Internet the valid formats, it said that the supported valid format is RAW 10-bit RGB

sdf.PNG

How is that posible?

GriffinRU
Member
Member
on

DirectShow - Get Video Capabilities.png

alvaroaldama92
Member
Member
on

I could not find the "enum filters" vi so I creted this similar vi:

1.jpg

And the results that I get :

Cddaptura.PNG

GriffinRU
Member
Member
on

Can you share the array contents? Compression types...

-Artur

alvaroaldama92
Member
Member
on

Sorry for not answering until today, but I have been a little busy.

0.PNG

1.PNG

2.PNG

3.PNG

4.PNG

5.PNG

6.PNG

7.PNG

8.PNG

9.PNG

10.PNG

11.PNG

12.PNG

13.PNG

GriffinRU
Member
Member
on

Hi,

No problem,

Looks strange that you do not have 720x1280 in YUY2 format, but try to use that format and report back.

-Artur

P.S. You can select format in similar fashion or check how I've done it in top level vi under "Video" case.

alvaroaldama92
Member
Member
on

Sorry but I am trying to find where I have to select the format, but I do not know where.

Thanks

GriffinRU
Member
Member
on

DirectShow-SetNativeFormat.png

alvaroaldama92
Member
Member
on

Thank you very much for all the help, finally works.

Without your help it would not have been possible.

A greeting

GriffinRU
Member
Member
on

You welcome,

Good to know Win10 tablets are compatible...

-Artur

russellb
Member
Member
on

Hello,

I saw that you have the DirectShow JPEG Dump.vi that creates a JPEG string. However how do i then write this string to a JPG file? 

Thank you for your help,

Russell

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
russellb
Member
Member
on

I was able to get the JPEG string to save to a file.  One of my coworkers pointed me to the problem. There are null characters at the end of the JPEG string because the string buffer size. When we delete these characters at the end then we were able to write to a binary file and the image looked correct.

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
GriffinRU
Member
Member
on

Bingo ... "prepend array or string size?(T)" boolean

I am glad you solved it.

-Artur

russellb
Member
Member
on

Thanks I had to set the prepend array or string size boolean to false as well.

Is there way with your VIs to set the input source type (i.e DVI, RGB) as well as all of the resolution and image settings? 

Also do you know why the picture control might be show a black image (i.e. not the captured image), but the number of dropped fames is zero and the non dropped frames is increasing?

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
GriffinRU
Member
Member
on

Have you looked into TopLevel VI - DirectShow Capture.vi?

Input source - from DirectShow - Enumerate Video Devices.vi.

-Artur

russellb
Member
Member
on

Thanks I was looking at one of the examples. 

However I am still not getting an image in the picture control display.  I can see that frames are being acquired, but they are not being shown in the display. I am trying to acquire from a DVI framegrabber that uses RGB video format.  I have the framegrabber vendor's application that also uses DirectShow and the video is displayed properly.  Since they are using DirectShow too I can see that the settings are same between your VI and their application.  I actually built your VI into an executable since the computer framegrabber is in does not have LabVIEW installed.

Any thoughts on why the image is not being displayed in the DirectShow Capture.vi?

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
GriffinRU
Member
Member
on

Can you attache your vi?

russellb
Member
Member
on

Unfortunately this comment section does not allow me to attach the VI. But I am using your DirectShow Capture.vi with no changes all I did was build it into an executable. Here is a picture of what I see when I try to run it:DirectShowCapture.png

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
GriffinRU
Member
Member
on

Hard to tell without the code...

I would start debugging with FilterGraph, by loading one of simpler examples and recalling it from GraphEdit.

-Artur

russellb
Member
Member
on

I think the problem might be related to the video frame format and not the actual software program as a whole.  If I use a PCIe frame grabber card with video frame format of RGB the image is not displayed. But I used PCIe frame grabber with video frame format of YUY2 and the image is displayed properly.  My built in webcam uses MJPG and YUY2 and it also works properly.  

So my question is there any reason the RGB video frame format might not be working? 

Thank you,

Russell

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
GriffinRU
Member
Member
on

Hi,

If you can get the frame properly with framegrabber than it is something wrong with videorenderer filter.

If you cannot get frame - try to add colorspace filter and try again, given that should be tried first with graphpgedit and then in labview.

If problem with videorenderer filter - you can try other renderer filters.

If it is not clear - let me know i will try to build labview code...

- Artur

GriffinRU
Member
Member
on

Hi, Have you solved your problem?

Attached screen shot of Color Space converter, which should be placed automatically between pins if they are not compatible but can be converted with color space converter. Video renederer accepts RGB32, so RGB, RGB24, BGR and others should be upscaled to RGB32...

ColorSpace.png

russellb
Member
Member
on

Hello,

I haven't had a chance to try your suggestion yet. Unfortunately I probably won't have a chance until Thursday. I will let you know what I find out.

Thank you for your help,

Russell

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
russellb
Member
Member
on

Hello,

I tried to use the GraphEdit tool, but for unfortunately it didn't install with the Windows Standalone SDK for Windows 10.  However I did check the Video Capture Pin settings and it is configured for RGB32, see attached image.   Unfortunately I have this frame grabber on loan from the vendor and I will need to send it back soon so I won't be able to do anymore testing.  I think I will most likely go with another framegrabber because it worked in LabVIEW with no problems. I really appreciate your help. Stream Format.png

Director of Engineering
G Systems, www.gsystems.com
Certified LabVIEW Architect
Certified LabVIEW Embedded Systems Developer
Certified Professional Instructor
GCentral
GriffinRU
Member
Member
on

Ok, sorry wasn't able to help you out fast enough.

-Artur

bseguin
Active Participant
Active Participant
on

OK all this is wonderful, but completely useless if i cannot manage my resolution of the Camera. Is there a way to not use the Popup to configure the camera and use some constant? I want to use the Maximum resolution per example

Thanks for any help.

Benoit

GriffinRU
Member
Member
on

Have you checked top-level vi inside llb?

You should be able to do that with pop-up dialogs or programmatically.

Examples designed to be simple

-Artur

bseguin
Active Participant
Active Participant
on

tjhis is what I've tried so far without out success.

Example aquire video plus frame graber.bmpThanks

Benoit

bseguin
Active Participant
Active Participant
on

Oh I tried other method even with the popup and the only message I got is: This format is not currently available. but My device support full HD I verified with other software.

Please any help on how to make this driver working with a custom resolution will be appreciated.

Thanks

 

Benoit

bim4
Member
Member
on

Hi,

 

Could someone indicate what license to use for DirectShow SDK.llb ?

I couldn't find any license/IP info regarding this llb.

 

Thanks,

 

Cedric

 

LM3
Member
Member
on

Dear GriffinRU,

 

This is a job well done. Can you comment on whether or not I should expect stability if using multiple graphs with multiple cameras? I am especially interested in being able to store grabbed images as .bmps and have found that if I put the grab and store in a loop using multiple cameras, I eventually get a 1172 error for a .net assembly issue for either "Bitmap region is already locked." or "Object in Use". The error appears to be happening within the DirectShow - SampleGrabber Snap (Buffer).vi only if I call the following code on the same snapped image box. 

store bmp.png

Please let me know if you have any advice or need more information. Thank you for a very helpful project.

Contributors