JDP Science Tools
Notes on Memory optimization
This is a copy of the final reply on this private conversation in 2021. It summarizes the work done on minimizing copying of large messages in Messenger Library. I'm copying it here so it is public:
Here's some notes. I'm quite happy with things, as I was able to get zero-copy for the "Event Notifications" in Messenger Library. I've been testing by passing 400Mb arrays from one Messenger-Library "actor" to another. This is much better than it used to be. What I had to do was:
A) my Message Objects hold data in Variants, and I needed to Swap in to the Variant to avoid a copy:
B) Because I usually just wire a non-variant to a Variant input, and that implicit conversion causes a copy, I wrote a VIM that puts in a To-Variant node, which doesn't do a copy:
C) The Notification system is written for arrays of addresses, as there can be multiple Observers of each Notification. Doing Send over an array in a For Loop leads to N copies. The get the N-1 copies (zero for one Observer) required looping over N-1 addresses, then doing the last address differently, like this:
D) To go through a User Event without two copies, I used DVRs, with the EventDVRMessenger described before. This requires a minor change to my DEV Message-Handling Template: just a Free-DVR node:
E) I had already been using Swap to get my Variants out of my Message Objects, but now I also use a new VIM that Swaps the data after conversion from a Variant.
By supplying an unbundle, from my main data-storing shift register, to the "Contents Data Type" input, this avoids the extra copy on the rebundle, as seen in this shot of the whole actor:
On the right is the initialization code to Launch a "Big Data Producer" actor, and to register for the Notifications holding the 400Mb arrays that Big Data produces.
On the left is the event case handling the Notifications (same case handle async replies, which i also tested to be zero-copy).
The time to Write, Send, Receive, and Read is less than 1ms for a 400Mb message. However, once one has processed a couple of messages, LabVIEW starts to free previous memory allocations, and apparently a 400Mb allocation takes about 45ms to free. From the Windows Task Manager, as well as DETT, i can see that three 400Mb allocations exist at any one time (I'm running 32-bit LabVIEW 2017).
Also found I had to prevent copies of an added array element by adding an empty element first then swapping in. Looks like this (done as a Malleable VI):
