Controling other Aplication GUI objects

Brandon.Baxter · ‎05-17-2017

For the purpose of general test automation I am looking for a robust low level approach to controlling external GUIs. In the past I have used LabVIEW to call Autoit applications. Using the scripting language to abstract GUI control worked to a large degree but when dealing with GUI objects not supported by Autoit such as WindowsForms10 checkboxes I didn't have a good solution. I don't really know what Autoit does under the hood but I was wondering if there was any way to code something similar from the ground up in LabVIEW with .Net or something? Ideally where I would like to start if controlling a WindowsForms10 checkboxes on a C# GUI to do things like Check the box, Uncheck the box, Check if the box is checked to get the basics am move forward from there but I could really use a nudge in the right direction.

t@nim@t0R · ‎05-18-2017

Brandon.Baxter,

Please refer to the forum post linked below (the solution) as it has a good description of how to interact with WindowsForms using a custom made script instead of Autoit.

https://forums.ni.com/t5/LabVIEW/How-do-I-convert-a-C-executable-to-a-LabVIEW-executable/td-p/832192

Let me know how it goes and what challenges you run into.

Brandon.Baxter · ‎05-22-2017

t@nim@t0R thank you for the link, I have read it thoroughly over the weekend. From my understanding the post covers creating .NET assemblies to give LabVIEW extra functionality and it covers controlling a GUI with Autoit. In this post the solution for controlling the external application GUI was to invoke Autoit scripting in LabVIEW through the ActiveX interface. I was hoping to control an external application directly through LabVIEW with low level control through .NET .This is a solution that I have used in the past but I am running into controls that are not supported by Autoit.

t@nim@t0R · ‎05-30-2017

So are you looking for resources on how to use .NET in LabVIEW?

I have included a few resources on using .NET libraries in LabVIEW:

Requirements for Using .NET Assemblies in LabVIEW

https://www.ni.com/docs/en-US/bundle/labview/page/requirements-for-using-net-assemblies-in-labview.h...

Calling .NET assemblies from LabVIEW

https://forums.ni.com/t5/Developer-Center-Resources/Calling-NET-Assemblies-from-LabVIEW/ta-p/3523180

Let me know if these are what you need.

Brandon.Baxter · ‎05-30-2017

Thank you for the references, I have dabbled in .NET before but the documentation dives you a mush bigger picture of the functionality, and I really appreciate it ;). I have found a collection native windows libraries for automating UIs that I can call through .net (most notably UIAutomationClient.dll). I feel like this could be the solution to the OP because it gives access to native low level accessibility and automation code. That being said I am running into some issues putting the solution into practice.

Below is some code I wrote to control pressing the "1" button on the windows calculator in C#:

private void StartAutomationCalc()
{
    //Get the Root Automation Element, i.e. Desktop
    AutomationElement desktopObject = AutomationElement.RootElement;

    Automation.Condition CalculatorCondition = new PropertyCondition(AutomationElement.NameProperty, "Calculator");
    Automation.Condition ButtonOneCondition = new PropertyCondition(AutomationElement.AutomationIdProperty, "131");

    AutomationElement CalculatorWindow = desktopObject.FindFirst(TreeScope.Children, CalculatorCondition);

    AutomationElement ButtonOneElement = CalculatorWindow.FindFirst(TreeScope.Descendants, ButtonOneCondition);
    InvokePattern ButtonOneInvoke = ButtonOneElement.GetCurrentPattern(InvokePattern.Pattern) as InvokePattern;
    ButtonOneInvoke.Invoke();
}

I did this in C# as a sanity check before I started because the documentation for the functionality is on MSDN.

Essentially what I am doing is navigating the Windows Automation Element Tree to find the calculator 1 button, isolate its Invoke Method, and calling the method to "press" the button

I have recreated the code in a LabVIEW VI but I am having trouble with the typecasting the Generic object returned by the "GetCurrentPattern" method to an InvokePattern Object. Attatched is my progress any help would be appreciated. I think I should use the "to more specific class" primitive but I'm not sure how.

LabVIEW