08-09-2017 04:11 AM
Hello,
when I'm loading multiple files with:
DataFileLoad(sFileToRead,"DEWESoft7","Register")
into the database, I'm getting this error:
The file is not corrupt, I can Import it for itself.
If the error occurs, then I can't use DIADEM anymore and have to restart it. If i don't i get an additional application error "not enough space"...
When a file is imported by my script, specific channel values (31 each file) are picked and copied to a constant channel group and after that the original file data is removed. So in sum there are 25327 values imported until the programm crashes, so I don't assume this is an database size error.
Regards
08-09-2017 04:53 AM - edited 08-09-2017 04:54 AM
Can you check the amount of used memory. (Task Manager)
You are using "register" instead of Load.
Maybe the plugin uses lots of memory or has a memory leak.
Data loaded into DIAdem is swapped to file when memory is running short.
If you only have to access part of the file it might be useful to switch to
DataFileLoadSel
using index or name of the channels or groups.
08-10-2017 02:53 AM - edited 08-10-2017 02:55 AM
Indeed the Memory is filling up but not exceeding although I remove the channel group which represents the reading data and I dispose the DataFileLoad-object, in addition the type "Import Action" doesn't matter:
There is no dump file...
When I'm using DataFileLoadSel, some of the channel values or properties that I get by calling
Data.Root.ChannelGroups(groupIndex).Channels.Item(channelName).Properties.Item(channelProperty).Value
surprisingly can't be read out....??? (Although I can see the values in the data portal)
08-10-2017 03:21 AM
Btw. when I'm importing all files manually in the navigator, about 700 files can be read out, then exactly the same error occurs - the plugin can't be found anymore / the plugin-selector doesn't show it anymore and the DIAdem memory is corrupt and the program is unstable...
08-10-2017 03:27 AM - edited 08-10-2017 03:32 AM
So I assume the plugin seems to have memory leak.
Can you give me the DIAdem version you are using.
From your post I assume that you load thousands of files.
Would it be possible to splt the job into multiple jobs and restart DIAdem between.
Maybe you can use a DIAdem `Worker` (see DIAdem help) to do the chunks.
P.S.: Can you provide an example file
08-10-2017 03:37 AM
plugin version is 1.2.0.6511
08-10-2017 09:13 AM - edited 08-10-2017 09:18 AM
My actual workaround: exporting the data files in DEWESoft7 to dat/r32-file, then reading the DIAdem files with the (same) script - works fine, no memory Problems and 10 times faster (2 minutes instead of 20 minutes for 1000 files)