8 significant digits

[Johan.svensson]

Im using two different numeric factors (in a VI) that I store in text fles. A few times an hour theese factors are collected (read from file into VI) for comparison.

The factors are entered manualy via an numeric controler.

Here's the thing... The numeric factors should have good precision, like "20,03581973". How do I configure the VI to save all 8 decimals?

By default t only saves 4 decímals

[Thomas L.]

How do you store the values?

You could use the Format Into File VI, it features a "format string".

E.g.: Put "%.8f" into a constant wired to this input (please don't beat me if it's not exactely the right syntax, I cannot look at it right now).

[smercurio_fc]

<posted at same time as previous post -  I hate it when that happens>

First, off which version of LabVIEW are you using? Also, how are you saving the file?. Are you using one of "Write to Spreadsheet File", or your own code? If you're using the "Write to Spreadsheet File" VI, it has an input for format. The default is "%.3f". In your case you would want "%.8f". If you've whipped up your own, chances are that you're using the basic number to string conversion functions. Those also have also have a "precision" input.

Please provide more details as to what you're actually doing. We're not mind readers, unfortunately.

Message Edited by smercurio_fc on 08-01-2006 08:28 AM

[jasonhill]

You may also want to look at the number of decimal places does on your indicator.  Once, I spent a couple hours tearing my hair out (what little remains) before I noticed that the default precision for indicators is 6.

[Johan.svensson]

look at pic below...

I think you're on the right track "jasonhill"

When I run program and use probe, the problem can be found directly on the connection from numeric controler.

How did you change it?

EDIT: Yes I only get 6 digit prec...

Message Edited by Johan.svensson on 08-01-2006 09:05 AM

[jasonhill]

Yeah, you will never get the last two digits because the number to string conversion is dropping them.  You need to wire a constant to the conversion.

Message Edited by jasonhill on 08-01-2006 09:09 AM

[jasonhill]

Don't trust the probes, they seem to have their own rounding rules.  Okay, I have a 6 digit control, I enter a 8 digit decimal, convert to string, and I can only see the last two digits on the string probe.  Even weirder?  I change the control precision to 0, rerun and get the same results.



I don't know why they act like this, perhaps someone else can enlighten us.

Message Edited by jasonhill on 08-01-2006 09:17 AM

[JamesC]

Jason

What is happening in your screen shot is that the numeric probe is rounding to 6 SF (default precision for DBL), but the wire contains all 8 SFs.
So when you convert to a string, the string probe displays the full string, which is the non rounded number.

So 0.12345678 loses the last 3 digits, and rounds up the 5 to a 6 (i.e 0.123457), but wire contains the orignal value.




Regards
JamesC
NIUK and Ireland

Message Edited by JamesC on 08-01-2006 04:01 PM

[jasonhill]

Thanks, James.  Is there any way to change the probe's precision? 

[JamesC]

Hi Jason,

Use a custom probe. Right Click the wire Custom Probe >> New
Create a new probe, don't work from a template, place down a numeric control right-click the control >> Format and Precision >> 8 SF

The VI will save to My Documents >> LabVIEW Data >> Probes
You can then select this probe from Custom Probe >> (Probe Names)

I attach a custom probe, try adding it to your My Documents >> LabVIEW Data >> Probes and see if it is incuded in your probe list, you might need to restart LabVIEW 8

Regards
JamesC
NIUK and Ireland

Message Edited by JamesC on 08-01-2006 04:09 PM