# RE: [R] comparison operator, decimals, and signif()

From: Ted Harding <Ted.Harding_at_nessie.mcc.ac.uk>
Date: Sun 22 May 2005 - 19:07:23 EST

On 22-May-05 Nick Drew wrote:
> Hi, I recently spent quite a bit of time trouble
> shooting a function that I had written only to
> discover that the problem I was having was with the
> comparison operator. I assumed that the following
> would return TRUE:
>

```>> testMean <- 82.8 + 0.1
>> testMean
```

> [1] 82.9
```>> testMean == 82.9
```

> [1] FALSE
>
>
> Apparently this has to do with deciml places. Look:
>
```>> newTest <- 82.0
>> newTest
```

> [1] 82
```>> newTest == 82
```

> [1] TRUE
```>> newTest == 82.0
```

> [1] TRUE
```>>
```

>
> What does signif() do to my object called "testMean"
> so that the comparison now evaluates to TRUE?
>
>> signif(testMean, 3) == 82.9
> [1] TRUE
While not an exact explanation, the following strongly hints at the underlying answer:

> print(82.9,digits=20)
[1] 82.9

> print(82.8,digits=20)
[1] 82.8

> print(0.1,digits=20)
[1] 0.1

> print(82.8+0.1,digits=20)
[1] 82.89999999999999

> (82.8+0.1)-82.9
[1] -1.421085e-14

So decimal arithmetic in a binary computer is not as exact as decimal arithmetic done with pencil and paper.

The underlying reason (in part) for the above apparent anomalies is that when 82.8 is added to 0.1, the smaller of these numbers (in its binary "floating point" representation) is shifted along until corresponding digits line up (the shift being accounted for in the "exponent"). In performing the shift, binary digits are dropped from the bottom end. Since "0.1" in binary in fact has an infinite number of significant decimal places, inaccuracy inevitably creeps in!

A close neighbour of the above calculations, such that there are no discrepancies in the binary representations, illustrates the above explanation:

> print((82.0 + 7/8),digits=20)
[1] 82.875

> print((82.0 + 7/8)+1/16,digits=20)
[1] 82.9375

> print(82.9375,digits=20)
[1] 82.9375

> ((82.0 + 7/8)+1/16)-82.9375
[1] 0

No discrepancies here! The reason is that the fractional parts of these numbers all have exact finite (and short) binary representations. Compare:

> ((82.0 + 7/8)+1/16)==82.9375
[1] TRUE   > ((82.0 + 8/10)+1/10)==82.9
[1] FALSE Hoping this helps,
Best wishes,
Ted.

E-Mail: (Ted Harding) <Ted.Harding@nessie.mcc.ac.uk> Fax-to-email: +44 (0)870 094 0861
```Date: 22-May-05                                       Time: 09:58:00
------------------------------ XFMail ------------------------------

______________________________________________
```
R-help@stat.math.ethz.ch mailing list