Machine epsilon

What does it mean for MATLAB to have a machine epsilon 2.2204e-16? Does that mean we can only be sure of a number up to the 15th decimal place, so something else?

Thanks.

Answer 1

[the cyclist]

It roughly means that numbers are stored with about 15-16 digits of precision. If a number is approximately 1, then that means it can be stored with an error of around 10^(-16) or so. If the number is approximately 1000, then it is stored with an error around 10^(-13) or so.

This is a VERY rough, non-technical explanation.

[Richard]

Thanks! That is very helpful. Would you mind explaining what double precision means -- saw it in Walter's link? (I have a background in mathematics, but I am not familiar with tech stuff...)

[the cyclist]

"Single precision" and "double precision" means that one has 32 computer bits or 64 computer bits, respectively, to store each number. A bit is a 0-or-1 unit of storage. Still trying not to be overly technical for you, so this is not a perfect definition.

[Walter Roberson]

A "double precision number" is one in which approximately twice as much storage is allocated compared to a "single precision number".

In modern computer systems, "double precision" _usually_ means 64 bits total storage for the number, and _usually_ means IEEE 754 Floating Point standards are being followed. However, Embedded Systems may use a different storage representation and no particular standard.

"single precision" these days _usually_ means 32 bits of storage for the number.

Double Precision was standardized before Single Precision: companies invented their own floating point representations Back Then that were good enough to get through on their own systems; IEEE then came along later and created a well-considered double precision floating point standard that did not tread on anyone's toes because no-one had a really usable double precision implemented.

If you are working with older computer systems, designed before 1985 or so, or are working with an embedded system, you might find that some weird and wonderful floating point or fixed-point system is implemented. Or if you are working with an IBM Z-series system, you might find that it also has a built-in _decimal_ floating point unit as well as a _binary_ floating point unit.

[Richard]

Thanks everyone! :)

Answer 2

[Walter Roberson]

The value is the same as eps(1) which is described at http://www.mathworks.com/help/techdoc/ref/eps.html

[Richard]

Hi, Walter, thanks for the link. However, it is a bit too technical -- I don't quite understand the jargon... Is it possible to explain it in layman language? Does it mean that the outputs can only be accurate up to 10^{-4}?

[Richard]

I meant to type 10^{-15}...

[Walter Roberson]

"d = eps(X) is the positive distance from abs(X) to the next larger in magnitude floating point number of the same precision as X"

That says that d = eps(1) is the smallest positive value such that (1+d) is exactly representable and is different than 1.

In hex representation of the binary floating point numbers,

>> num2hex(1)

3ff0000000000000

>> num2hex(1+eps(1))

3ff0000000000001

1+eps(1) is the smallest representable number greater than 1, a single bit difference in the least significant (smallest change) bit.

The difference between 1 and 1+eps(1), which is to say eps(1), is the 2.22E-16 that you noted above.

This eps() value scales in linear jumps, so eps(2) is 2*eps(1), eps(16) is 16*eps(1), eps(1/16) is eps(1)/16 .

It is not a number of decimal places; it is _relative_ tolerance. eps(1/1024) is eps(1)/1024 so if you were working with base values in the approximate range of 0.001 then you would not still be limited to 2E-16 accuracy; you would be limited to about (2E-16)/1000 accuracy. And likewise, if you are working with values in the range of 1000, you do not get 1000 + 2E-16 accuracy, you get about 1000 + 1000*2E-16 = 1000 + 2E-13 accuracy

Now, there are also all kinds of rounding reasons and limits on accuracy of special functions like sin(), that can result in worse accuracy; eps() gives best-case accuracy.

[Richard]

Thanks Walter! This is a GREAT explanation! :-)