Mathscript / Formula node precision

[craigc]

Hi i was hoping someone could tell me more about a formula node / mathscript node.  I want to emulate a cordic Algorithm implemented in hardware within LabVIEW.  since the hardware uses 16bit integers i want to do all the mathematics in this precision.  If i pass (or if it is possible to pass) 16 bit integers to a formula node, does the node stick with this precision or does it do the maths in floating point or whatever and return the result as a 16 bit integer?

Any suggestions on how to approach this problem would be great.

Thanks in advance

Craig

[GrantM · Accepted solution]

Hello Craig,

It is possible to pass a 16-bit integer to a formula node.  Create a formula node and add an input variable.  Drop a numeric constant on the diagram and wire it to the input variable.  Right-click the constant and select Representation >> I16.  However, the formula node will perform the mathematical operations with double-precision internally and then return the results as a 16-bit integer.  The MathScript node also works with double-precision internally, but it will show you the coercion when you wire the 16-bit integer type to the input variable.

To remain with 16-bit integers, you could use LabVIEW primitives to perform the mathematical operations instead of using a formula or MathScript node.

Grant M.
Staff Software Engineer | LabVIEW Math & Signal Processing | National Instruments

[craigc]

Thanks Grant,

not exactly what i wanted to hear because as you say i may be back to primitives, however you answered my question exactly.

regards

Craig