09-25-2014 11:45 AM
@SASA1 wrote:
here is the vi there is a difference and pictures, there are differences
Yeah. I would expect some differences. The Double (DBL) has twice as many bits to represent an number than a Single (SGL).
09-25-2014 12:25 PM
SGL only gives you about 7 decimal digits of precision and you are doing quite a few operations, accumulating more rounding errors. The DBL result is significantly more accurate. SGL has the advantage of smaller data structures. You need to decide if it is precise enough. For example if the result drives a 16bit analog output, the difference is irrelevant.