Numerical precision in subtracting two almost identical variables?

As far as I understand it, Matlab uses 16 digits of precision. However, I have a program that subtracts two variables and very often gives results much smaller than 1e-16. For example. When asked it displays a=0.000000006129562, b=0.000000006129562 but a-b=1.0e-24 * 0.827180612553028 How does it distinguish two variables that differ by a quantity much smaller than the 15th decimal place, if it doesn't have an internal representation of it no larger than 16 digits? Shouldn't be it a-b=0 always?

[Stephen23]

In your question you incorrectly conflate two related but different topics. Here, I highlighted them for you:

"How does it distinguish two variables that differ by a quantity much smaller than the 15th decimal place, if it doesn't have an internal representation of it no larger than 16 digits?"

Decimal places is not the same thing as significant digits. Mixing up those two different things (like you are doing) is not going to help you do any computational mathematics, let alone do any science. There are many many many tutorials online which explain the difference, so I will not waste my time duplicating tutorial content here.

"Shouldn't be it a-b=0 always?"

No.

Answer 1

[Aquatris]

Here is your answer link

Its all in the low level things. If you display your values with 'format hex', you will see they differ by 1 bit, and that bits gives you the 1e-16 difference.

Example:

x = 0.6;
y = 6*0.1;
delta = x-y

delta = -1.1102e-16

format long
[x y]

ans = 1×2

0.600000000000000 0.600000000000000

format hex
[x y]

ans = 1×2

3fe3333333333333 3fe3333333333334

[Marco]

Yes, that's a 1e-16 difference. But how can it tell a 1e-24 difference?

[Stephen23]

"But how can it tell a 1e-24 difference?"

Very easily, if the values have magnitudes no more than 15-16 digits more than that.

You are making the mistake of thinking that that precision is absolute, i.e. something like "15 decimal places from zero". However that is incorrect: the name binary floating point number tells us that the values are not stored as absolute values (with a fixed, absolute number of decimal digits (from zero) which are stored/represented) but as floating point values whose precision is always relative to the magnitude of the specific values. The 15-17 digits of relative precision is the same, regardless if your value is huge (e.g. 1e+234) or miniscule (e.g. 1e-234).

3e-24 - 2e-24 % same precision here

ans = 1.0000e-24

3e123 - 2e123 % and here: 15-17 significant digits.

ans = 1.0000e+123

[Aquatris]

Here is to demonstrate @Stephen23 's point

x = 0.6*1e-15;
y = 6*0.1*1e-15;
delta = x-y

delta = -1.9722e-31

format long
[x y]

ans = 1x2

1.0e-15 * 0.600000000000000 0.600000000000000

format hex
[x y]

ans = 1x2

1.0e+00 * 3cc59e05f1e2674d 3cc59e05f1e2674f

[John D'Errico]

@Marco

You are missing that MATLAB does not store this number 0.00000123456789 as those 15 decimal digits, in cluding the leading zeros. That would be a bad thing, because it would fail due to exactly what you are worried about.

Instead, you store numbers using a scientific notation.

format long g
0.00000123456789

ans = 
            1.23456789e-06

Of course, this shows the number using powers of 10. MATLAB stores the number internally in a binary form, but the two forms are very similar. Because there is always an exponent attached to each number, this form is called FLOATING POINT. Essentially, the decimal point is allowed to float.

Th nice thing is, now a number like 0.00000123456789

is stored with a relative error only down in the least significant bits of that number. That is given by eps.

eps(0.00000123456789)

ans = 
      2.11758236813575e-22

So if you subtract two numbers of that size, you can now start to see differences with an order of magnitude 1e-22.

[Marco]

Ok, so if I understand it, the FP internal representation does not include the leading zeros. This makes sense. Is there a way to visualize the value of the two variables which difference is delta to more than 15 decimal places? Format long g does not.

For example:

Displayed are a=0.000000006129562 and b=0.000000006129562, but delta=a-b=1.0e-24 * 0.827180612553028

Then I expect it to be internally a=6.129562xxxxxxxxxe-9 and b= a=6.129562yyyyyyyyye-9

Could those decimal places in x and y made be visible?

[Stephen23]

Use SPRINTF (results depend on the OS / MATLAB version)

Or use NUM2STREXACT: https://www.mathworks.com/matlabcentral/fileexchange/22239-num2strexact-exact-version-of-num2str

Note that digits after the 16-17th have no significance.