Range and Precision

The range of a number gives the limits of the representation, while the precision gives the distance between successive numbers in the representation. The range and precision of a fixed-point number depend on the length of the word and the scaling.

Range

The following figure illustrates the range of representable numbers for an unsigned fixed-point number of size ws, scaling S, and bias B.

The following figure illustrates the range of representable numbers for a two's complement fixed-point number of size ws, scaling S, and bias B where the values of ws, scaling S, and bias B allow for both negative and positive numbers.

For both the signed and unsigned fixed-point numbers of any data type, the number of different bit patterns is 2ws.

For example, if the fixed-point data type is an integer with scaling defined as S=1 and B = 0, then the maximum unsigned value is 2ws1, because zero must be represented. In two's complement, negative numbers must be represented as well as zero, so the maximum value is 2ws11. Additionally, since there is only one representation for zero, there must be an unequal number of positive and negative numbers. This means there is a representation for 2ws1 but not for 2ws1.

Precision

The precision of a data type is given by the slope. In this usage, precision means the difference between neighboring representable values.

Fixed-Point Data Type Parameters

The low limit, high limit, and default binary-point-only scaling for the supported fixed-point data types discussed in Binary-Point-Only Scaling are given in the following table. SeeLimitations on Precision and Limitations on Range for more information.

Fixed-Point Data Type Range and Default Scaling

Name

Data Type

Low Limit

High Limit

Default Scaling (~Precision)

Unsigned Integer

fixdt(0,ws,0)

0

2ws1

1

Signed Integer

fixdt(1,ws,0)

2ws1

2ws11

1

Unsigned Binary Point

fixdt(0,ws,fl)

0

(2ws1)2fl

2fl

Signed Binary Point

fixdt(1,ws,fl)

2ws1fl

(2ws11)2fl

2fl

Unsigned Slope Bias

fixdt(0,ws,s,b)

b

s(2ws1)+b

s

Signed Slope Bias

fixdt(1,ws,s,b)

s(2ws1)+b

s(2ws11)+b

s

s = Slope, b = Bias, ws = WordLength, fl = FractionLength

Range of an 8-Bit Fixed-Point Data Type — Binary-Point-Only Scaling

The precisions, range of signed values, and range of unsigned values for an 8-bit generalized fixed-point data type with binary-point-only scaling are listed in the follow table. Note that the first scaling value (21) represents a binary point that is not contiguous with the word.

Scaling

Precision

Range of Signed Values (Low, High)

Range of Unsigned Values (Low, High)

21

2.0

-256, 254

0, 510

20

1.0

-128, 127

0, 255

2-1

0.5

-64, 63.5

0, 127.5

2-2

0.25

-32, 31.75

0, 63.75

2-3

0.125

-16, 15.875

0, 31.875

2-4

0.0625

-8, 7.9375

0, 15.9375

2-5

0.03125

-4, 3.96875

0, 7.96875

2-6

0.015625

-2, 1.984375

0, 3.984375

2-7

0.0078125

-1, 0.9921875

0, 1.9921875

2-8

0.00390625

-0.5, 0.49609375

0, 0.99609375

Range of an 8-Bit Fixed-Point Data Type — Slope and Bias Scaling

The precision and ranges of signed and unsigned values for an 8-bit fixed-point data type using slope and bias scaling are listed in the following table. The slope starts at a value of 1.25 with a bias of 1.0 for all slopes. Note that the slope is the same as the precision.

Bias

Slope/Precision

Range of Signed Values (low, high)

Range of Unsigned Values (low, high)

1

1.25

-159, 159.75

1, 319.75

1

0.625

-79, 80.375

1, 160.375

1

0.3125

-39, 40.6875

1, 80.6875

1

0.15625

-19, 20.84375

1, 40.84375

1

0.078125

-9, 10.921875

1, 20.921875

1

0.0390625

-4, 5.9609375

1, 10.9609375

1

0.01953125

-1.5, 3.48046875

1, 5.98046875

1

0.009765625

-0.25, 2.240234375

1, 3.490234375

1

0.0048828125

0.375, 1.6201171875

1, 2.2451171875