NI Home > Community > NI Discussion Forums

BreakPoint

Showing results for 
Search instead for 
Do you mean 
Reply
Active Participant
Andrey_Dmitriev
Posts: 1,540

Santa Claus - wish list.

Yes, I have to wait till next year, I know. But anyway,

 

Dear Santa!

My name is Andrey Dmitriev

and I want a Multithreaded Application Builder in LabVIEW!

PLEASE!!!

 

 

lv_multi.png

 

Thanks, Santa, if you can hear me,

 

Andrey.

 

Active Participant
Brad
Posts: 787
0 Kudos

Re: Santa Claus - wish list.

[ Edited ]

You could try adding "allowmultipleinstances=True" to the top of your labVIEW INI. It launched multiple instances for me, but no guarantee on how everything will work.

 

Merry Christmas,
Santa

Knight of NI
Knight of NI
Ben
Posts: 16,149
0 Kudos

Re: Santa Claus - wish list.


Brad wrote:

You could try adding "allowmultipleinstances=True" to the top of your labVIEW INI. It launched multiple instances for me, but no guarantee on how everything will work.


 

Brad,

 

Please help me undestand this.

 

I thought that switch allowed multiple LV instances to be open (we used to have to copy and rename LabVIEW.exe to do that).

 

I believe Andrey is asking the application builder use more than one core when building applcations.

 

Does your switch now allow that?

 

Ben

 

Ben Rayner
Who is now transitioning to John Galt.. just building Rayner's Ridge
Rayner's Ridge is under construction
Active Participant
Brad
Posts: 787
0 Kudos

Re: Santa Claus - wish list.

I wouldn't think using the multipleinstance flag behaves any diffferently than launching multiple LV exe's with different names (I thought from his screenshot that's what he wanted). I tried using app builder and with my 4 core machine (with hyperthreading), it looks like building 4 projects at the same time was using multiple threads (See attached screenshot).

 

Anyway, hope this helps, if not, maybe the real santa can help,

Brad

Active Participant
Andrey_Dmitriev
Posts: 1,540
0 Kudos

Re: Santa Claus - wish list.


Ben wrote:

I believe Andrey is asking the application builder use more than one core when building applcations.

 

Does your switch now allow that?

 



Exactly. I have large application, it contains over 60 plugins (which are libraries). Full rebuild needed rarely, but take more than half hour. So, I can start 6 LabVIEW copies, and each one will build 10 libraries. As result build done in 5-7 minutes. And each one uses only 8 percent CPU. 

Active Participant
Andrey_Dmitriev
Posts: 1,540
0 Kudos

Re: Santa Claus - wish list.


Brad wrote:

I wouldn't think using the multipleinstance flag behaves any diffferently than launching multiple LV exe's with different names (I thought from his screenshot that's what he wanted). I tried using app builder and with my 4 core machine (with hyperthreading), it looks like building 4 projects at the same time was using multiple threads (See attached screenshot).

 

Anyway, hope this helps, if not, maybe the real santa can help,

Brad


Yes, In general its the same as start executables under different names (the only difference - that in Task Manager they named all the same - LabVIEW.exe).

 

What I want is to increase CPU usage within single LabVIEW instance during build. Or, alternatively, to have possibilities to start builds programmatically in different threads and fully parallel (right now also impossible).

 

In additional - application builder called programmatically with BuildTargetBuildSpecification.vi seems to be have resources leak - after 50-60 builds it stops working (not enough memory).

Active Participant
Andrey_Dmitriev
Posts: 1,540
0 Kudos

Re: Santa Claus - wish list.


Andrey Dmitriev wrote:
In additional - application builder called programmatically with BuildTargetBuildSpecification.vi seems to be have resources leak - after 50-60 builds it stops working (not enough memory).

The reason probably here: after each call BuildTargetBuildSpecification.vi it make copy of nimxsoffline.exe in memory, so my task manager during build looks like that:

 

lv_multi3.png

Knight of NI
altenbach
Posts: 27,598
0 Kudos

Re: Santa Claus - wish list.

Well, the correct place for this would be the LabVIEW Idea Exchange. :smileyvery-happy:

 

NI already has compile farms and cloud options for FPGA compiles. How about being able to submit build jobs to the cloud?? Why not?


LabVIEW Champion . Do more with less code and in less time .

Active Participant
Andrey_Dmitriev
Posts: 1,540

Re: Santa Claus - wish list.


altenbach wrote:

Well, the correct place for this would be the LabVIEW Idea Exchange. :smileyvery-happy:

 

NI already has compile farms and cloud options for FPGA compiles. How about being able to submit build jobs to the cloud?? Why not?


Thanks, Christian,

 

It is already in LabVIEW Idea Exchange: Multi-Core LabVIEW Compiles, but not too much Kudos... Santa is my last hope.

 

 

Member
Rob_Calhoun
Posts: 124

Re: Santa Claus - wish list.

We have implemented this in our build system and I wanted to share a few implementation details. This is all on Windows.

 

First, limitations: this will not multi-thread the builder itself. Rather, it allows multiple single-threaded build processes to run in parallel. For us this is still very useful because we have several different products to build (different apps, conditional symbols, etc) and building them in parallel saves time. Your mileage may vary.

 

Hardware Setup

Use a virtual machine for building! This is an essential part of ensuring that today's build can be re-built on the same environment in the future. (There is nothing more frustrating than having to change one tiny part of your own code in a build from a year ago and discovering that environment changes beyond your control have introduced baffling bugs. Allocate one core per simultaneous build desired.

 

Use Windows 2008 R2 (x64) as a build platform, if possible. National Instruments finally supports this platform, and it is a good, stable platform for builds. In any event, use a 64-bit OS even when building the 32-bit version of LabView, as it avoids an issue where NIDAQ 9.x fails to load on a 32-bit machine with 4+ GB of ram (due to PAE issues), and provides a cleaner address space for the 32-bit app. (LabView uses absurd amounts of memory while building and the more memory on the machine, the better.)

 

Source Code Control

You absolutely must use a source code control system. We use subversion because it handles binary files competently. Your builder should automatically tag the code it built. (This costs nothing.)

 

LabView Setup
Install the desired version of LabView and the DAQmx drivers, if needed. Run LabView.

 

Turn off all crash reporting features. Disable ni.com update. Skip geting started window on launch. Disable automatic saving for recovery. These can throw up a dialog box, rendering your build process dead in the water.

 

Turn off auto-save; set number of undo steps to 1. Set load palettes when needed. These waste of memory and time.

 

Turn off the web server. Turn off the VI server. There is no way to have different instances use different ports, so only one instance will be able to bind to it. You must use a file or other mechanism to talk to the instance. (Well, if you really want to use the VI Server for control, you can make copies of LabView.exe and set the port individually, but now you've got multiple LabView apps to manage.)

 

Quit LabView and edit the labview.ini file. Set:

 

allowMultipleInstances=True

 

Now you can run more than one copy of LabView at a time. You probably don't want this on your development machine; it's too easy to launch more than one instance by mistake.

 

Make a project for your builder and identify your builder VI. Test that this can be launched from the command line (from the appropriate directory) using \Full\path\to\LabView.exe mybuilderproject.lvproj builder.vi. Test that everything works while you still have a user interface to work with.

 

Create one new user for each build process. We call ours "BuildBot1" etc. There are two reasons for doing this:

 

  • LabView will only use per-user code-signing certificates. Per-machine code-signing certificates will not work; each build user must have the certificate in its own store.
  • LabView caches compiled object code in the VIObjCache directory, which is stored on a per-user basis. While LabView (somewhat remarkably) appears to work fine when two instances are pointing to the same VIObjCache (exception: clearing the object cache will fail) this seems like setting the stage for weird, untraceable failures in the future. Best to avoid the problem entirely by giving each build its own private cache.

Log in as the the build process user. Do whatever you need to do to give access to the source code repository. Our build does a fresh svn export for each build, and commits the build product to a different build products repository. (Some people are against sticking executables in a source code control system, but if you commit them, you can always get an old release when you need it and, more importantly, there is a traceable record of the the bits: well worth the disk space in my opinion.) We use the same svn user for all build processes.

 

Install the code signing certificate for the current user. As noted above, LabView code signing certificates will only work on a per-user basis.

Test that everything works under the builder user. The code signing stuff is particularly vexing, because if it is wrong it will tank at the very end of the build process. Repeat for each user (arduous, but you only need to do it once.)

 

Service Setup
The builder instances are run as a Windows service using a service runner application. We use FireDaemon; there are others available. Note that Microsoft has not allowed services to have a user interface since Vista, which is why you want to get the kinks out in advance.

 

Set up each builder as a Windows service using FireDaemon. Set the working directory to the directory containing the LabView project, and set the arguments to "ProjectName.lvproj ViName.vi".

 

We make each builder identical and then have them pull buildspecs out of a common directory. To avoid resource contention, the builders are only allowed to scan the directory during non-overlapping 10 second periods each minute. (Locking the file would be another option, this was just easy and it seemed reasonable to stagger the builds a bit anyway.)


It's fun when you get this all working. Here's a snapshot of the Windows task manager after kicking off the build process:

 

buildbot.png

 

Set the builder application to quit after the build is finished. LabView uses a lot of memory during the build, and because LabView's deallocations are "lazy", quitting LabView is the only-sure fire way to reclaim memory on the server.

Snapshot your virtual machine after building the release build(s) so that the environment can be re-created when needed.

 

Continuous Integration

One could drive the whole thing from a continuous integration server like Hudson or Bamboo, but we haven't done that yet. Automated pre- and post-build testing is a topic for another day.

 

-Rob Calhoun