Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

Highlighted

11-29-2017 11:28 PM

Options

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report to a Moderator

Labels:

Can anyone please explain what does 'Approximate number of Decimal digits' indicated in 4th column of the image.??

11-29-2017 11:48 PM

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report to a Moderator

How many digits you can see if you wrote that number out in decimal, i.e. base 10.

Think significant digits.

11-29-2017 11:59 PM

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report to a Moderator

just for clarification, taking an example 'I8' numeric data type, we have '2' as approximate number of decimal digit written in 4th column of the image. The range is -128 to 127 , which involves three significant number..!! Ex: 123.

Am i right ?

11-30-2017 12:11 AM - edited 11-30-2017 12:18 AM

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report to a Moderator

I think it has to do with the fact that all floating point numbers cannot perfectly represent decimal digits or fractions of a whole number. The chart is a guide to where you will start to see error in the number in terms of number of digits to the right of the decimal.

Example: Divide 1 by 10 and show the result in a double indicator, the chart says you get *about* 15 digits of precision, in this case we got 17 digits before error.

11-30-2017 12:22 AM

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report to a Moderator

@vishal777 wrote:

just for clarification, taking an example 'I8' numeric data type, we have '2' as approximate number of decimal digit written in 4th column of the image. The range is -128 to 127 , which involves three significant number..!! Ex: 123.

Am i right ?

Yes. But it also says "approximate". That third digit isn't really a full digit. It is only 0, 1, or 2 for an unsigned U8, and -1, 0, or 1 for a signed 8-bit. So it is really 2+ significant digits.