Dear Matt,
I was aware of the /3GB and I tryed time ago with LV 7 and XP but without being able to allocate a larger image anyway...
I'll try again and for sure 4GB under Vista 64 is better then anything !!! Do you know if "Image Create.vi" (a VI included in the Vision package) is also able to allocate up to 4GB ?
Concerning Memory defrag I started using some tools available in the marlet (there are good ones for memory analyzing but no one is capable of defragging the memory really)... then a coupple of years ago I wrote a small application in Labview based on 2 free tools provided on the web (those are free tools for developpers provided from Microsoft)...
One is Listdlls.exe which is a command line capable of listing all running DLLs (associated with any running application) and it produces a memory map of the DLLs allocation in memory which allow you to understand how memory is fragmented.
The other one is also a command line tool named Rebase.exe which allow you to a DLL to a different address in memory.
In fact once rebased a DLL will generally load automatically starting at the new address instead of loading in the middle of the memory or at the original address... but not always as some rare system DLLs will anyway try to load always at the same position (as for Asian Language Pack on Chinese/Japanese OS). Furthermore some DLLs need to be loaded to a specific address.
I wrote a program in order to automatically create a memory map of all running DLLs (including system ones) using Listdlls.exe and automatically rebasing them after the first 128MB in memory. Rebasing system/running DLLs is generally more difficult (because of the required access policy and because you must plan a replacement only after next reboot) ... my software take care of this.
Furthermore rebasing running/system DLLs is also dangerous and can crash your system... after many test I limited the rebasing to the memory addresses included between 256MB and 1.6GB... I never have problems rebasing all DLLs in these range (including system ones) under XP 32bit SP2.
I also move DLLs to free space only (not on already occupyed space). Furthermore I never allocate DLLs in the first 128MB (as you'll crash your system probably). In this way I was capable of freing a lot of memory (generally about 15-20 DLLs are reallocated and memory between is free between 300 and 1.6)... something more can be done manually on the higher memory addresses but it take times and must be checked always. In automatic mode my software is capable of freing memory in 2 seconds (but you need a reboot).
Backing up original DLLs is also suggested in order to be able to restore them in case of any problem. I implemented an automatic backup and a restore option for this. A manual mode allow me to manually move DLLs in memory (but it's more dangerous and never allow my customer to do it).
Generally after rebased the memory stay defragmented until you install a new OS patch (which replace old DLLs).
Then you must be aware then in order to mantain the memory clean when running your application in Labview you should preallocate a large memory block when starting your application preventing labview from allocating resources in your free memory (it must use memory outside the 300-1600MB range). This is another problem... I found it hard to solve and my application is doing a good job but not perfect for this.
p.s. Labview and LV Runtimes are major cause of memory fragmentation as they load just in the middle of the system memory... these are the first DLLs ro be rebased !!!