On 4/8/2004 at 2:10:03 PM, GUI-Majician wrote:

##### Quote

But, there is approximation and then there is gross approximation.

Being accurate to only 3 decimal places is worthless! I don't

know why I didn't noticed this inaccuracy before.

First, let me say that it is not *inaccurate*. In fact, it is 100%

accurate. It is just not very *precise*.

But why is that?

You are using the Java _float_ type, and that means 32-bit IEEE 754

floating point numbers, which have 23 bits of mantissa. Since 2 ^ 23

= 8388608, that means you can get 7 or 8 digits of precision in a Java

float. But remember that this is the best precision you can have. In

any operation, you can lose precision, and once precision is lost, you

can never get it back.

In your program, the final operation performed is to subtract two numbers:

the value of the variable "var0" and the temporary result of the

expression (var1 * var2). With infinite precision, this would be (1982.6

- 1975.6) = 7.0, but since those numbers are at best represented with 7 or

8 digits of precision, you get something like (1982.6000 - 1975.5999) =

0007.0001, about 5 digits of precision.

Note that this is the way floating point numbers work, regardless of the

language you are using. Precision is limited, and some operations, such

as subtracting numbers that are very close, result in lost precision.

There is no way to avoid this. You can use wider floating point numbers,

but that just pushes the problem out to larger numbers. The only way to

avoid it is to use a different data type for your numbers.

--

Regards,

John McGrath [TeamB]

---------------------------------------------------

Before sending me e-mail, please read:

www.JPMcGrath.net/newsgroups/e-mail.html