I use a Diadem script to load files sequentially into the data portal and process them one by one. After the processing I save them, clear the data portal and load the next file etc. Despite clearing the Data Portal, Diadem appears to 'hang on' to the memory with each file incrementing the memory usage as shown here.
Is there some way I can 'reset' the memory after clearing the data portal? At the moment I appear to have memory issues if I process many files.
Solved! Go to Solution.
Can you add some information.
In fact, we are faced with Diadem crashing without warning and we suspect memory issues.
Thanks for the support.
The DataFileLoadSel command 'locks' the source folder. Even when the processing is finished, and I clear the data portal, it is impossible to delete the original data files or their folders until I close Diadem. At the same time, the memory usage drops back to the normal baseline level. The command DataFileLoad does not do this. Expected behavior for DataFileLoadSel?
If the files are still locked when portal is cleared something went wrong.
So after data.root.clear all files should be deletable.
Could you please try the following:
Got to: Settings->DIAdem Settings->Navigator->Loading Behavior
Modify "Load bulk data on modification of channel data" to "Always load bulk data"
Maybe it is easier for you to set
BulkDataLoadingMode = eBulkDataLoadingImmediately ChnCharacteristicsUpdate= "Automatic"
at the beginning of your script. Does the effect remain?
P.S.: Are you capable to reproduce the behavior that after the file is no longer deletable after data.rot.clear was called and provide a script and files to us? We would be very interested because this is potentially causing your memory issue.
"Always load bulk data" does not solve the issue. The files are still locked and the memory usage stays high even when I clear the data portal.
Another point, if at this stage, I try to open a new file (script), Diadem immediately crashes with this error message.
This behavior is repeatable for the memory, file locking and crashing. I would need approval to share data files with you....
This is the method I came up with and it works pretty well:
Dim File, iCount, Directory '-- Set Directory Path Directory = "C:\_Processed Data" '-- Load All TDM Files from Directory File = DirListGet(Directory, "*.TDM", "filename", "FullFilenames") For iCount = LBound(File) To UBound(File) Call DataFileLoad(File(iCount)) '-- Calculations Call ChnIntegrate("/Time","/Pelvis RY","/Pelvis Rotation RY","TrapezoidalRule",0,0,0) '-- Save and Clear Call DataFileSave(Directory&"\"&Data.Root.Name&".TDM","TDM") Call Data.Root.Clear() Next
Thanks for the response. I normally use a method similar to yours and it works ok. However, if I use DataFileLoadSel instead of DataFileLoad, I get the memory issues described. I could use DataFileLoad, but then I would load many channels which I do not need and end up with a very large file.
CORRECTION : Diadem 2015 NOT 2017
Please note that I am using Diadem 2015 sp2, not Diadem 2017 as I have written initially.
Apologies to all.