I was really hoping this thread had solved the problem that's been plaguing me for the past few days. It certainly changed the behavior I'm getting from LabVIEW.
Instead of resulting in a fatal error, building my project now causes the screen to flash rapidly at several points in the process, and then when the progress bar reaches the end, the build progress dialog disappears without waiting for me to click Done. Checking the build directory reveals no .exe file, and meanwhile LabVIEW has become unresponsive and has to be terminated.
Any idea what's going on? I've tried values of 65536, 65535 and 20000 with the same result. With the default value of 10000 I got the same fatal error described in the OP, which is why I was hopeful, but it's not fixed.
-- Abe Thurtell
I was having the same problem in WIN 7 LabVIEW 2016 64bit. I was trying to build an executable with more than 10000 VIs and labview kept crashing on me.
Changing the GDI limit to 20000, restarting my PC and the build the application again solved my problem.