Modify dataplugin for HST file

[levan8421]

Perfect.

I also just figured it out by using script recording 🙂

[levan8421]

Hi Brad,

The most surprising about this DataFileLoadRed method is that it does not need to load the data yet can be able to index the dataset thus it can select the right range of data based on selected index range. 

I would like to get a certain amount of latest data, say 1,000,000 data points from now. If using DataFileLoadRed, I still have to load the dataset to find the length for the end index; from that I manually subtract to 1,000,000 to get the start index. Is there any method that can figure out the length of dataset with loading the data?

Thank you.

[levan8421]

I tried this method for different data sets that I have not touched before. I recognize one thing: even using DataFileLoadSel, the first time always takes a lot of time to load the data (I guess it is the same as using DataFileLoad). After that, if I remove the data set and use DataFileLoadSel to load data again, it is dramatically faster. I guess in the first run, DIADem "records" the index of values of the data set thus in next run, it does not need to do that anymore. If I restart DIADem, it happens again: first run takes long, and next runs are much faster. 

I tried to use DataFinder to index the data, but it seems just to index the channels, not the values. Thus the time issue is still unsolved. 

I hope there is more convenient way (or a workaround) to deal with this issue. Currently I have about 50 data channels to load (could be up to ~300); each channel has data for ~ 3 years but I only need data of 10 minutes of yesterday. It will take a total at least 1.5 hour just to load data. 

Any suggestion is greatly appreciated.

[Brad_Turpin · Accepted solution]

Hi levan,

Depending on how the DataPlugin you're using is programmed, it might be helpful to "Register Load" the data set first to determine the total number of values.  Here's a script that register loads just the first channel from the first group of the desired data file, finds its length (Size), then reduced loads just the last 1000 data points for all channels of that file:

Call Data.Root.Clear
Call DataFileLoadSel(DataFilePath, "", "[1]/[1]", "Register")
Set Channel = Data.Root.ChannelGroups(1).Channels(1)
SectionEnd = Channel.Size
SectionBegin = SectionEnd - 1000
IF SectionEnd < 1 THEN SectionEnd = 1
Call Data.Root.Clear
Call DataFileLoadRed(DataFilePath, "", "", "IntervalWidth", 1, 1, SectionBegin, SectionEnd)

Brad Turpin

DIAdem Product Support Engineer

National Instruments