Threaded Loading into the Data Portal so I can Work While Data is Loading

[DieselPower]

Title says it all.  I don't like waiting while files load.  Can this process be threaded?

[AndreasH_Ha]

Hello,

let me double check whether I understand your use case :

When you load data the groups and channels should be listed in the data portal, but you want want to wait until teh data is loaded right ?

If so, did you try "register loading" and what is missing to cover your use case ?

Andreas

[DieselPower]

If I drop a 30 MB file into the data portal it might take 1 minute to load.  If I drop ten 30 MB files into the data portal at once, it might take 5 minutes to load.  I would like to be able to continue to use DIAdem instead of waiting until all of the files load even at the sacrifice of additional time to load all of the files.  That majority of my use of DIAdem involves loading data files into a report template for PDF export.  I can't do this until all of the files are loaded.  If file loading could be passed of as a thread I could still use other features of DIAdem on the files as they become available.

I have not tried "Register Loading."  I just looked it up and I will try that, although I think my idea might have merit.  I just need to so what I lose from doing "Registered Loading."

Thanks!

[AndreasH_Ha]

"Register Load" should do what you excpect to happen. It only loads the header data and reads the bulkd data from disk when needed.

If you could please give it a try and let me know whether it works as expected I'd appreciate it.

Thanks

[AndreasH_Ha]

One thing I'd like to double check : You wrote that a 30 MB files takes a minute to load. That's pretty long.

Can you please describe what version of DIAdem you use and which file format ?

Thanks

[DieselPower]

It took 80s to load a 25 MB file just now. 

DIAdem 11.1 SP1

.txt files with 1 header and 1 unit row

Intel Xeon 2.53 GHz

8 GB ram

Windows 7 64-bit

[AndreasH_Ha]

OK, sounds to me like the problem is the performance when loading the text file. Would you mind to post an excerpt of the file with the first lines ?

How to you load the file ? With a DataPlugin ? Or the ASCII-Import Wizard ?

Thanks

[DieselPower]

Data plugin.

How important is it for you to see the first lines?  Lets say I have 40 columns of data.  The first row is the column header.  The second row is the unit.  All subsequent rows are data.  This means the first two rows are strings no more than 20 characters in length or so.  The data ranges from 0-3000 with no more than 5 decimal places, but mostly 0 or 1 decimal places.

[AndreasH_Ha]

OK, that describes most of I wanted to see. One last question : How many columns does the file have typically.

And please let me knwo how you load the data into DIAdem

Thanks

[DieselPower]

40-60 on average.  I drop files from explorer into the data portal and it uses a data plugin for text files called With Units.

[AndreasH_Ha]

Can you tell more about the DataPlugin you called "with Units". Is this something you developed?