DIAdem CSV Custom DataPlugin

[CarlosCazares]

I am fairly new to using DIAdem and I am trying to configure a DataPlugin for my .csv files when I import them into DIAdem. I've attached an example file of what all of my files are standardized too, with the addition or subtraction of channel columns.

The way our data acquisition is currently configured, my Channel Name and Units are all in one cell. So for example, my first channel is labelled "Time (sec)", so my Channel Name would be "Time" and my Units for that channel would be "sec" or "seconds". This format is standard across all my channels with varying names and units.

I am trying to figure a way out to essentially designate that row to be my Channel Name and Units, but then go into each one and extract the Channel Name and Units for that column to assign it in the metadata when I import these files. I have tried using the DataPlugin Wizard, but it's very limited past choosing how to segment the data and what each row designation should be. I'm assuming I could use a script to help out, but not entirely sure how that would work with importing data or using the data finder.

To clarify, I'd like to achieve this on the DataFinder side and not just have a script run and then user select what data to import. By achieving this on the DataFinder side, any time new data is added to my server it will pull all that metadata and I can search using those properties.

[AndreasK · Accepted solution]

I assume if you have to extract more information than the wizard can extract you need to write a VBS Plugin on your own.

DIAdem Help contains some examples on how to use the script API.

I created some lines to help you get started reading your file.

Test Request ID,12345,,,,,,,,,,
Test Name,Example,,,,,,,,,,
... More Content
,,,,,,,,,,,
Time (sec),P1 Low (0-100 psig),P1 High (0-500 psig),P2 Low (0-25 in-wc),P2 Mid-Low (0-250 in-wc),P2 Mid-High (0-100 psig),P2 High (0-500 psig),dP (0-500 psig),Q Low (0-3271 scfh),Q Mid-Low (0-13086 scfh),Q Mid-High (0-100018 scfh),Q High (0-750215 scfh)
0,104.994993,299.950835,26.250268,262.513648,63.996466,64.026311,235.715353,34.25113,-2.530559,-17.556872,84.747909
0.201,104.990516,299.941905,26.25212,262.529405,63.991906,64.027585,235.667056,34.46066,-3.345261,-20.114207,34.181559

Looking at a file following jobs show up

• Is it one of my CSV files?
  If not just leave
• What to extract from header?
  Read whats needed and prut it on the properties
• Where do the channel names start and which channels do exist?
  I just defined that "Time (sec)" is the marker to figure out that here the channels start. You will potentially implement it slightly different.
• Extract the unit from the channelnames.
  I have choosen a regular expression. Might be ugly but it is efficient and a lot of help can be found in the net to write regular expressions.

Option Explicit

Sub ReadStore(File)
  File.Formatter.Delimiters = ","
  File.Formatter.LineFeeds = File.GetLineFeed()

  dim strVal : strVal = File.GetNextStringValue(eString)
  if 0 <> strcomp(strVal, "Test Request ID") then
    ' None of mine CSV files
    RaiseError
  end if

  Dim ChannelGroup : Set ChannelGroup = Root.ChannelGroups.Add("group")
  ' add some properties
  ChannelGroup.Properties.Add strVal, File.GetNextStringValue(eString)

  ' find begin of channels
  dim firstChannelName : firstChannelName = ""
  do while file.SkipLine
    strVal = File.GetNextStringValue(eString)
    if 0 = strcomp(strVal, "Time (sec)") then
      firstChannelName = strVal
      Exit Do
    end if
  loop
  
  if "" = firstChannelName then
    RaiseError "No channels found"
  end if

  'collect channelnames and units to be added later on
  dim channelCount : channelCount = 0
  dim channelNames : channelNames = Array()
  dim channelUnits : channelUnits = Array()
  
  dim re : set re = CreateObject("VBScript.RegExp")
  '  "any char but (" --- "(" --- "any char but )" --- ")"
  re.Pattern = "^([^\(]+)" & "\(" & "([^\)]+)"& "\)"

  dim currChannelName : currChannelName = firstChannelName
  do
    ' execute regexp. This example does not check if successful
    dim matches : set matches = re.Execute(currChannelName)
    
    channelCount = channelCount + 1
    dim realChannelName : realChannelName = trim(matches(0).SubMatches(0))
    dim unitString : unitString = trim(matches(0).SubMatches(1))
    redim preserve channelNames(channelCount) ' grow array
    channelNames(channelCount) = realChannelName
    redim preserve channelUnits(channelCount) ' grow array
    channelUnits(channelCount) = unitString
    
    ' pick next string from the line
    currChannelName = File.GetNextStringValue(eString)
  loop until isempty(currChannelName)

  ' Now we now the amount of channels and can add them
  File.SkipLine ' to reach start of raw data
  Dim Block : Set Block = File.GetStringBlock()
  dim channelIndex : for channelIndex = 1 to channelCount
    Dim DirectAccessChannel : Set DirectAccessChannel = Block.Channels.Add(channelNames(channelIndex), eR64)
    dim channelObj : set channelObj = ChannelGroup.Channels.AddDirectAccessChannel(DirectAccessChannel)
    channelObj.Properties("unit_string").Value = channelUnits(channelIndex)
  Next
End Sub

I attached vbs plugin as uri file. !!! Import it using DIAdem. Do not double click it. Else you will only see it crypted.

Or just create a new one and copy the code inside of it.

Hope this helps

Andreas