Showing results for 
Search instead for 
Did you mean: 

what is better in arithmetic, SGL / FXP / DBL



  I'm using CompactRio and doing some arithmetic operation on RT,  I had seen in that  arithmetic operation result with SGL input is different from FXP or DBL one, 


  why it happen?




0 Kudos
Message 1 of 6

Different in what way?


They three representation us a different number of bits, so will operate with different precision. In general, the results should be quite similar except for rare cases.


For FXP, the precision is configurable and you can balance how many bits to use for the integer part and the fractional part.



0 Kudos
Message 2 of 6

exemple with the attached picture.


the result with cvi is    Ksin




there is difference!




0 Kudos
Message 3 of 6

Well, you only show a DBL computation and compare it with a computation in a datatype that you don't define. What datatype was used in CVI?


How do you know what is correct?

Do both versions of the algorithm do all computations in exactly the same order?

Do all inputs have exactly the same precision?


We cannot debug code pictures. Please attch the actual VI. Populate all control values with typical default data before saving. 

0 Kudos
Message 4 of 6

duplicate post, already asked here

Best regards,

using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 5 of 6

So the results agree withing 5 decimal digits, which is plenty for typical real-world signals.. Where is the data coming from? How many bits does the DAQ have? What are you going to do with the result of the computation?

0 Kudos
Message 6 of 6