Lookup Table in Calculator

[RussellSenior]

Hello, I would like to implement a lookup table function in the calculator.  I have a channel of data labled Lambda, and I want to tranform it to a channel called CO.  I have two other channels with the relationship between Lambda and CO.  This is almost trivial to do in VB, but I need it to be as fast as possible and doing it in a VB loop is usually not fast.  

I can make the relationship channels as high fidelity as needed.  

I believe I want to use the CHD function in the calculator, with the y channel of my relationship as the input.  Where I'm stuck is trying to find the index of the xvalue of the relationship to put into the CHD function. 

I've tried to use the ChnFind Command in the calculator, but I cannot get it to work. I suspect it's an error with quotes, but I i've tried every possible combination I can think of. 

Any help would be appreciated. 

[Brad_Turpin]

Hi Russell,

What type of transform are you talking about.  Is it a simple algebraic combination of your two input channels to create your output channel?  It is very rare that you would use ChD or ChnFind in a Channel Calculator expression, in fact I would advise to always choose to leave those outside the expression and use global variables in the expression instead.  You do need to fool with indexing in a Channel Calculator expression.  You just describe the transform logic by referencing the entire channel, then the Channel Calculator automatically applies that transform to all values of all channels.

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[RussellSenior]

The transform is sort of algebraic, but it's an 8th order polynomial, which is what I'm currently using. Unfortunatley, this is leading to too much error near the bounds of the independant variable.   So, since the transfer function is not great for fitting a line to, I wanted to implement a lookup function.  The issue is that I need this to be fast, since the algorithm this is a part of will do this transformation a hundred times or so.

I've attached my transfer function, along with arbitrary inputs and the expected outputs that I estimated and keyed in by hand. So, what I need is a really fast way to do this:

For each value of the input

-find the index of the input on the independant variable of the transfer function.

-take that index and output that value of the dependant variable. 

Fairly simple, I think, but I just need it to be fast. I can do a linear map to increase the fidelity so no interpolation is needed, but I'm still not sure how transfer from the x_axis to the y_axis in a calculator. 

What do you think about doing this peicewise?  Maybe split my transfer function into two, or three sections that can be described by simple algebra, and use logic in the calculator. 

[Brad_Turpin]

Hi Russell,

See if this approach is fast enough for you.  I chose to improve the resolution of your transfer lookup channels by a factor of 1000 to avoid linear interpolation (as you mentioned).  Then all you have to do is use the PNo() function to find the closest match, then use the row that PNo() returns to look up the correct Y value.

Call Data.Root.Clear
Call DataFileLoad(DataFilePath, "TDM")
Set TransGroup = Data.Root.ChannelGroups("Transfer Function Test")
Set TransXChannel = TransGroup.Channels("Transfer Function_x_axis")
Set TransYChannel = TransGroup.Channels("Transfer Function_y_axis")
Set DataXChannel = TransGroup.Channels("Input")
Set DataYChannel = TransGroup.Channels.Add("Output", DataTypeChnFloat64)
Set TrnewXChannel = TransGroup.Channels.Add("Trnewfer Function_x_axis", DataTypeChnFloat64)
Set TrnewYChannel = TransGroup.Channels.Add("Trnewfer Function_y_axis", DataTypeChnFloat64)
Call ChnLinGen(TrnewXChannel, CMin(TransXChannel), CMax(TransXChannel), 1000*TransXChannel.Size)
Call ChnMapLinCalc(TransXChannel, TransYChannel, TrnewXChannel, TrnewYChannel, 0, "const. value", Null, "analogue")
iMax = DataXChannel.Size
FOR i = 1 TO iMax
  n = PNo(TrnewXChannel, DataXChannel(i))
  DataYChannel(i) = TrnewYChannel(n)
NEXT ' i
Call WndShow("VIEW")

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[RussellSenior]

As I suspected, it's much too slow.  This is part of a larger binary search algorithm, so I end up doing this transformation 336 times on channels with ~1800 values.  My previous way of doing this, with the 8th order polynomial, took about 0.04seconds per transformation for a total of about 13 seconds.  With this method, I'm getting about 0.75 seconds per transformation, for a total of ~252 seconds. Also, this is being run on the simplest case, a 3 cylinder engine.  If I run it on an 8 cylinder, we add ~600 values per cylinder, and I don't know how to calculate Olog(n) for this, so many more than 336 transformations...

In addition, I think I need the transfer function to have an accuracy of 0.0001 over a span from 0.5 to 1.5 to avoid interpolating.  I achieved this by modifying your proposal and using slope for the exrapolation method in the linear map function, and 10,000 data points for the interpolation channel.  To be honest, I don't know what level of accuracy I actually need in the interpolation channel.  I want my search algorithm to produce an accuracy of 0.001, so I'm running the algorithm to 0.000001.  This isn't optimized, (more of a SWAG) but isn't a problem with 13s run time.

So, I guess, bottom line, in the current implementation I need the transformation to be <0.1s per.

(An interesting side note, if I did what I'm doing with a linear search algorithm, it would take ~10 million years to run 🙂 ) 

[Brad_Turpin]

Hey Russell,

Just checking.  When you repeated the calculation 336 times, you didn't run the linear mapping function 336 times, did you?  That's probably a small part of the duration, but it would only need to happen once, then you could re-use it as often as you need.

In related news, have you played with the "Script Profiler" in DIAdem 2015 much?  That might help you detect the bottlenecks in any approach you attempt.

By the way, your measured X values that you want to look up, do they ever repeat in one data set?  I assume so, but if by chance they don't, I do have another idea to try.  Or even if they rarely repeat.

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[RussellSenior]

I made sure not to rerun the linear map.  

No, I still have version 2014.  I'm a holdout!  🙂

I tried a peicewise approximation of my transfer function by splitting it into three sections.  Now I have 3 much simpler polynomial approximiations, which results in much less error.   The calculator string is quite a mess, but it works, and works quickly.