Splitting Up A Large Data File

[stfung]

Hello:

I have a large .dat file which contains several groups of data. I tried the import wizard, but it is only able to parse all the data into the channels (columns). How would I create a data plugin which is able to break up a large file into several groups of data?

Example Structure:

Comments and header information

Group info

Group info

Group info

Channels

Data....

Group info

Group info

Group info

Channels

Data....

and it repeats.

My goal for having it in separate groups is to be able to import each group of data as sheets in an excel file. 

[AndreasH_Ha]

Hello stfung,

you didn't provide any details about the file format itself. Given that you mentioned thte import wizard, I assume its a text file. The structure you described requires a data plugin based on some manual coding. If you could provide an example file, I could try to create the plugin for you. Please also let me know teh DIAdem version you are using.

Andreas

[stfung]

Hi Andreas,

Yes, it's a data file in the form of text. I have attached a stripped down example of the data. The actual file is probably 50 times larger, but I figured a smaller chunk of it would be easier to test on. 

I would like for each module of data to become separate groups of data, so it would be easier to export each module into separate excel worksheets. The channels are the three data columns. 

I was thinking of writing some code in another scripting language, but I wanted to check on this forum in case somebody could assist me with a DIAdem data plugin. I am using DIAdem 2012 on a 64-bit machine. 

Thank you,

Stephanie

[AndreasH_Ha · Accepted solution]

Hello stfung,

Please find attached a draft for the DataPlugin. Download the "URI" file to your computer and then double click it. This will install a Dataplugin called "ModTest-TextFile"

Most of the Plugin deals with handling the meta-data. reading the signal data into groups is the smallest part.

If you are interested this part of the script is in a Function called "ProcessSignals(oFile,oGroup)"

Please let me know whether the plugin works for you.

Andreas

[stfung]

Hello Andreas,

It looks like it works! Thank you for the data plugin, I'm not that familiar with VB so this was extremely helpful. The metadata was indeed the part I was having the most difficulty with handling. 

Stephanie

[AndreasH_Ha]

Hello Stephanie,

glad to hear that it works for you. I just re-read your initial question and saw that you plan to export to Excel. Is this for analysis and report or just to make the data available for others ?

Andreas