LabVIEW
FPGA Compile Cloud Service End-of-Life: Feedback and Transition
I don't have experience with the R series hardware, only cRIO and sbRIO. But yes as you get closer to 100% utilization of the FPGA fabric, expect compile times to increase at least exponentially. Also a lot more memory is needed to closer you get to 100%. Hearing your experience I am starting to wonder if trying to separate things into two cards would not be much more cost effective in the long run. A thumb of rule I maintain is that anything above 95% is going to ask for problems that are going to be very painful in the long run.
And much of the speed gain in the compile farm (and cloud) is due to the fact that it uses the Linux version of the compiler. It's considerably faster and more efficient than the Windows port of that compiler. That 64GB seems not enough for those extreme compiles kind of surprises me but I was finding out in the early days of FPGA (LabVIEW 7.1 days) that it made a huge difference if the PC had 8GB or 16 GB of memory and that was for the old FPGAs in the first cRIOs.
@pwj wrote:
We do not have other option now since NI is shutting down this service. Some things can't be arranged at will and if we include company processes into it then we may not be ready by the proposed day by NI, and it will impact projects delivery. I have to work close with IT to ramp it up.
Definitely sounds like you're in a pickle- I'm very sorry to hear that, and I can't offer a ton of advice, other than to say that I got started in Linux around a year or two ago and it's been MUCH smoother sailing than I'd thought originally.
If you're setting up a compile server and don't have existing infrastructure/expertise, I'd advise looking into Proxmox. It runs on bare metal as a VM server, and you can install multiple other VM's onto the machine. Proxmox is based on Debian long-term stable and, in my experience has been 100% rock solid stable.
You can install another Linux VM (or ten!) into Proxmox, use snapshots to roll back to previous versions, try new installs, etc. Proxmox adds very little overhead and your VM's will run basically just as fast as they would on bare metal and will be much easier to back up and manage.
You can also add Windows VM's if you need some Windows-specific features, and can fire them up or idle them as needed.
Best part is, Proxmox is open source and is free. You can get Enterprise features if you want but it's certainly not required.
Anyway- if you already have IT handling this, then please defer to them. I just can comment as an ME-turned-LV-dev who's been dabbling in Linux. The Proxmox/VM route is very nice, and I think it would be a great way to spin up a brand new Linux compile server from scratch.
(My company has, until very recently, been Windows only, and we went with Proxmox+Linux VM's for our own hosted GitLab server and it's been VERY smooth.)
@Intaris wrote:
This is not a serious proposal, surely. Doubling of hardware cost (not to mention the refactoring of FPGA code to allow this) just to get around a compile tool-chain issue? Add into this the idea of having to modify existing systems to offer bug-fixes and updates. Nightmare.
Not so much to get around the compile toolchain issue as the fact that only ~10% of the compiles succeed, independent if it is local or in the cloud. That together with a 3-7 hour compile time is in itself already a huge liability. Add to that that every bugfix or even seemingly harmless improvement or optimization has a serious chance to make the compile success approach 0%. That's not a stable long term plan. And long term it seems to be as the compiles are still regularly happening or we would not even talk about this here.
As to the solution with Proxmox you need to watch out with different Linux distributions. The supported Linux distributions for the 2021.1 Vivado toolchain (the latest NI uses in the current FPGA Module) is not to much out of date with support for RedHat and CentOS up to 8.3, SUSE 15.4 and Ubuntu 20.04 but also not state of the art anymore. Expect some problems with installation in newer Linux versions, or different distributions but for a seasoned Linux user it is doable.
ISE 14.7 is another story which requires officially Redhat 6.5 which is for more than a year out of even extended (paid) maintenance. Not something you want to have attached to your network in any way when you are not the only person who has access to that network.
Even lower utilization can sometimes fail with timing violations for no obvious reason. Probably Heisenberg playing a joke there. But 10% success sounds awful. I would dread every time I need to make even the slightest change to that design. And considering what engineering time costs, you can buy even a high end R series card after a few such design changes. Another approach might be to design some core parts directly in VHDL (or Verilog) for optimization.
I'm not saying that is the solution and I would also prefer if we could just continue with the Cloud Compiler, as that would be the way of least resistance in our case too. I'm still in the process of talking with IT about getting the necessary virtual servers setup and it is anything but easy, since they are very much against installing any but the latest and greatest OS versions. So likely will have to do some serious detective work about getting those compilers installed anyways.
@pwj wrote:
Unfortunatelly we are too an almost Windows only company, that is why it is so painfull.
Only linux that we have nearby us is a ni linux on the PXI systems. And those can't be even connected to our network since they do not comply with IT regulations.
I know that someone, somewhere is using some linux distribution that IT has accepted but right now it is a gossip to me and I need to verify what exactly it is about.
I wish I could offer you some help on the compliance side, but unfortunately my advice will have to stop at "it wasn't terribly hard to implement, and I feel your pain".
Only because I don't have any additional advice to give xD
That said- there are several Linux distros that are meant for this sort of thing. If you can't get a dedicated server to run Proxmox, then you've probably got some other VM host out there that maybe you could get Debian LTS on (assuming that works with NI, I don't know).
Good luck 😞
