LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

different values beetween front page and saved data

Hello,

I have problems to saved data correctly. I acquire data with Labview 6.0 and a counter/timer card. These data appear on my front page. But when I save them in a table file, their format is modified, rounded.

For exemple: on my front page values are 20.675 , 21.012 and in my saved file these values are 20.750 and 21.000, even if I checked all the numbers format in the diagram, and declare them in extended precision.

Where is my mistake ?

Thank you for your help

0 Kudos
Message 1 of 9
(2,583 Views)
Hello shadok,

can you post some code? It would be much easier to find the misbehaviour...


Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 2 of 9
(2,568 Views)
0 Kudos
Message 3 of 9
(2,554 Views)
It doesn't matter what precision you specify for display or the numeric representation. When you use the Write to Spreadsheet function, you have a format specifier of %3f. This specifier writes a floating point number with a width of 3 and no decimals. The default is %.3f (note the decimal point) which says to write a floating point number with 3 significant digits.
0 Kudos
Message 4 of 9
(2,542 Views)
Hello shadok,

as Dennis wrote by now, you specify %3f as precision for writing. This means "floting point with 3 significant digits" (means NOT: "3 digits after decimal point"!!!).
Change this to %.3f. Or even better to %.4f, as all your data is U32 divided by 80 giving a resolution of 0.0125...

Hope this helps

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
Message 5 of 9
(2,536 Views)
The problem lies in the format that you are writing to the file, on the write from spreadsheet file .vi ther is a formatting that you set. right now you have it set to %.3f which means you will have three digits past the decimal in floating-point. you will need to change this. look at the help and follow the sytax to change it.



Joe.
"NOTHING IS EVER EASY"
0 Kudos
Message 6 of 9
(2,532 Views)

I'm sorry it doesn't work!

In my file I have now 4 numbers after the decimal point, but the real value is still rounded at the nearest multiple of 0.125

It gives me 21.875 for 21.862 on the front page

0 Kudos
Message 7 of 9
(2,527 Views)
Hello shadok,

that's strange 🙂

Try this simple example. If this works, then something with your data is wrong...

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 8 of 9
(2,517 Views)
Hello shadok,

one more guess:
Have you set any range properties of your indicators? I couldn't find any, when loading your vi, but perhaps they are vanished after loading into LV7.1...
If you set the increment to 0.125 in the data/range properties your behaviour could happen.

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 9 of 9
(2,515 Views)