Showing results for 
Search instead for 
Did you mean: 

did inverse tangent not support FXP

@SASA1 wrote:

here is the vi there is a difference and pictures, there are differences

Yeah.  I would expect some differences.  The Double (DBL) has twice as many bits to represent an number than a Single (SGL).

There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
0 Kudos
Message 11 of 12

SGL only gives you about 7 decimal digits of precision and you are doing quite a few operations, accumulating more rounding errors. The DBL result is significantly more accurate. SGL has the advantage of smaller data structures. You need to decide if it is precise enough. For example if the result drives a 16bit analog output, the difference is irrelevant.

LabVIEW Champion. It all comes together in GCentral GCentral
What does "Engineering Redefined" mean??
0 Kudos
Message 12 of 12