LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Windows UIAutomation mit LabVIEW Applikationen

Solved!
Go to solution

Hi,

 

I'm using Windows 10 with a touch screen (no keyboard).

.NET controls are part of the UI Automation of Windows - therefore the touch keyboard pops up as soon as they get the focus. (at least with .NET framework which is not too old)

 

Unfortunately this does not work with LabVIEW controls. I tried to register them to the UIAutomation but it seems that they aren't derived from a WIN32 control (see attached VI). Using the display coordinates (not the front panel xy) of the "LV Numeric" control, the System.Windows.Automation.AutomationElement.FromPoint() method does not find it. 

 

I'm wondering if someone of you guys found a solution for the LV touch keyboard feature lack?

 

Following is not an acceptable solution:

- self made keyboard

- osk.exe

- click on the keyboard icon in the task list every time I need the keyboard

 

Cheers,

Jonas

 

 

 

 

0 Kudos
Message 1 of 10
(3,905 Views)
Solution
Accepted by topic author zuegerj

I'd split the problem up into two problems. I did minor proof-of-principle testing, it might help to get you started.

 

1) detect when the touch keyboard should appear.

2) show the touch keyboard.

3) hide the touch keyboard.

 

1) seems fairly easy with an event structure or callback event. Not sure if it is that easy. For instance, clicking the control doesn't mean it actually got key focus. So I think it should be a combination of catching key down event, and checking if the control has key focus.

 

2) Starting "C:\Program Files\Common Files\Microsoft Shared\Ink\TabTip.exe" works for me. I've read online that it doesn't work for a lot of people, that they actually have to simulate a mouse down on the TapTip window. You have to start it with a .NET process.start method. Starting with the command line doesn't work, as LabVIEW looses focus. You'll find Process in the System assembly, under System.Diagnostics.

 

3) So on some event (enter key, or something else getting focus), you'd have to kill the process. Not in the attachment.

 

Touch Keyboard in LabVIEW.png

0 Kudos
Message 2 of 10
(3,859 Views)

I found some solutions with tabtip.exe before. 

In newer versions of Windows 10 tabtip.exe does no longer open the keyboard. It just starts the service for it.

 

As far as I know Microsoft doesn't want to allow the client to open the keyboard by a command because of strange things can happen and the user thinks Windows has a bug. That's why they decided to manage it by their own UIAutomation concept.

 

Unfortunately LabVIEW didn't made their controls  an EditControlType. They made all controls a WindowControlType. Therefore Windows UIAutomation does not open the keyboard - it does not know if it is a control or an indicator.

 

I guess that there's no other solution than LabVIEW make a new version and change the control types.

Please let me know if I'm wrong and you found a solution anyways.

 

Cheers,

Jonas 

0 Kudos
Message 3 of 10
(3,844 Views)

@zuegerj wrote:

Unfortunately LabVIEW didn't made their controls  an EditControlType. 


And that makes sense, as LabVIEW has to look the same on Windows, Linux and MacOS. Not sure how that works for Windows controls...

 


@zuegerj wrote:

 

They made all controls a WindowControlType. Therefore Windows UIAutomation does not open the keyboard - it does not know if it is a control or an indicator.


Maybe Windows controls become WindowControlType, I haven't checked recently. Most LabVIEW controls have nothing to do with anything in Windows. The controls are simply graphics drawn on a Mesa3D\OpenGL canvas. 

 


@zuegerj wrote:

I found some solutions with tabtip.exe before. 

In newer versions of Windows 10 tabtip.exe does no longer open the keyboard. It just starts the service for it.


Then I wander why it works on my newer version of Windows. Pretty sure I'm up to date.

 

There are several tricks online to make it show anyway, by sending windows messages to the process.

0 Kudos
Message 4 of 10
(3,835 Views)

Im on Windows 10, version 1703 and it works for me, but it looks like it doesnt work for all - see the link below with one possible solution.

 

https://answers.microsoft.com/en-us/windows/forum/windows_10-performance/tabtipexe-can-not-work-from...



 


 

0 Kudos
Message 5 of 10
(3,829 Views)

Maybe MS fixed it? So it works again for newer versions?

 

What I think is happening is that some people try to invoke the keyboard while Windows is not configured to show it. That's not the situation: Windows is configured to use the keyboard, but LabVIEW doesn't signal Windows to show it. It could be related to the Windows version, but I wouldn't be surprised if it always just worked with the right Windows settings.

0 Kudos
Message 6 of 10
(3,825 Views)

Your're right. It works on my other computer. May be Microsoft fixed it. tabtip.exe might work.

However, there's no specification how to use tabtip.exe. So it's kind of unsecure if it will work on different Windows 10 platforms or not. That's why it can't be used in production in my case. 

 

0 Kudos
Message 7 of 10
(3,811 Views)

@zuegerj wrote:

Your're right. It works on my other computer. May be Microsoft fixed it. tabtip.exe might work.

However, there's no specification how to use tabtip.exe. So it's kind of unsecure if it will work on different Windows 10 platforms or not. That's why it can't be used in production in my case.


Ok. I'm not sure if there are other solutions. BTW, MS MFC applications seem to have the same issue. Really puzzling how MS ignores this...

 

The lack of documentation is indeed a problem. One thing that is not clear to me is how to get a numeric keyboard...

 

A totally different solution, maybe completely unrealistic and untested, is to use .NET FP controls instead of the LabVIEW controls. The .NET controls could be wrapped in a class, so you insert the ref in the init of the class\object and let the class handle things. But this will be a complete rewrite, and that could be a lot of work depending on the application and level of abstraction in it.

0 Kudos
Message 8 of 10
(3,799 Views)

 

A totally different solution, maybe completely unrealistic and untested, is to use .NET FP controls instead of the LabVIEW controls. The .NET controls could be wrapped in a class, so you insert the ref in the init of the class\object and let the class handle things. But this will be a complete rewrite, and that could be a lot of work depending on the application and level of abstraction in it.


Yes, that's actually the solution I implemented instead of the tabtip.exe. The reason for this decision is that .NET controls are linked to the .NET framework version running on the client. The UIAutomation is part of the .NET framework too. Therefore, once Microsoft changes some functionality of the UIAutomation look and feel we just have to upgrade the .NET framework and the LabVIEW UI will work like Microsoft thinks it should - without re-building it.

 

Cheers,

Jonas

0 Kudos
Message 9 of 10
(3,775 Views)

Hi Jonas,

 

I am struggling with the same problem as you with how to show the TabTip.exe automatically when my string or numeric controls have focus.Would you be able to share with me an example of how you overcame this issue using .NET controls instead? I assume there is no programming involved if you use .NET versions of string control etc. and windows will detect the key focus and display the tabtip automatically.

 

Also do you know if it is possible to customize the look of the .NET controls such as text font,size and colors.

 

Thanks

0 Kudos
Message 10 of 10
(3,555 Views)