Board index » delphi » Need help comparing float types.

Douglas Tosi

Delphi Developer
Sat, 13 Sep 2003 03:16:38 GMT

Need help comparing float types.

Quote
Ded wrote:
> "Mark A. Grover" wrote:

> > When I insert a floating point number in an Interbase database table, it
> > doesn't save it exactly as the number was entered.  Instead, it often times
> > carries the decimal places out several positions.  This becomes troublesome
> > when comparing the the fields in Delphi because if a field was entered as
> > 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> > 1.5.  What is the best way to work around this?

>     Old good practice in comparing floats is
>  Abs(F1-F2)<precision.

I know someone would come up with a decent solution.

Thanks Ded.

dog

--
___________________________________________
IMPROART
Rua Emiliano Perneta, 297 - 6 And - Cj. 63
CEP 80050-010 - Curitiba-PR - Brasil
Tel/fax.: 55-41-323-7558

 
Mark A. Grove

Delphi Developer
Sat, 13 Sep 2003 02:47:15 GMT

Re:Need help comparing float types.


When I insert a floating point number in an Interbase database table, it
doesn't save it exactly as the number was entered.  Instead, it often times
carries the decimal places out several positions.  This becomes troublesome
when comparing the the fields in Delphi because if a field was entered as
1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
1.5.  What is the best way to work around this?
Ded

Delphi Developer
Sat, 13 Sep 2003 03:08:57 GMT

Re:Need help comparing float types.


Quote
"Mark A. Grover" wrote:
> When I insert a floating point number in an Interbase database table, it
> doesn't save it exactly as the number was entered.  Instead, it often times
> carries the decimal places out several positions.  This becomes troublesome
> when comparing the the fields in Delphi because if a field was entered as
> 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> 1.5.  What is the best way to work around this?

    Old good practice in comparing floats is
 Abs(F1-F2)<precision.
Douglas Tos

Delphi Developer
Sat, 13 Sep 2003 03:14:41 GMT

Re:Need help comparing float types.


Quote
"Mark A. Grover" wrote:
> When I insert a floating point number in an Interbase database table, it
> doesn't save it exactly as the number was entered.  Instead, it often times
> carries the decimal places out several positions.  This becomes troublesome
> when comparing the the fields in Delphi because if a field was entered as
> 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> 1.5.  What is the best way to work around this?

If you're using IB 6, use Numeric(x,y) instead of Float and Double Precision.
Numeric datatype stores precise numbers.
If you're still on 5.6 (as I do) or earlier, well use Double precision instead
of Float. That won't solve the problem, I know... One way is to compare the
strings, as in
FormatFloat('###,###,##0.0#', 1.5) = FormatFloat('###,###,##0.0#',
MyField.AsFloat);
Yes, I find that terrible too, but it's a solution and you can specify how many
decimal places you want to compare.
I hope someone can post a better one, as I also have this kind of problems.

hth,

dog

--
___________________________________________
IMPROART
Rua Emiliano Perneta, 297 - 6 And - Cj. 63
CEP 80050-010 - Curitiba-PR - Brasil
Tel/fax.: 55-41-323-7558

Jan Henrik Sylveste

Delphi Developer
Sat, 13 Sep 2003 04:02:35 GMT

Re:Need help comparing float types.


Interbase: Try to use NUMERIC instead of FLOAT -- it's stored as integer.
(fixed comma)
General: Never actually compare floating values. Compare the difference to
some small value.
e.g. FLOAT1 = FLOAT2 -> FLOAT1 - FLOAT2 < 1e-6
"Mark A. Grover" <mark.gro...@seaquistperfect.com> schrieb im Newsbeitrag
news:3abf8ef6$1_1@dnews...
Quote
> When I insert a floating point number in an Interbase database table, it
> doesn't save it exactly as the number was entered.  Instead, it often
times
> carries the decimal places out several positions.  This becomes
troublesome
> when comparing the the fields in Delphi because if a field was entered as
> 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> 1.5.  What is the best way to work around this?

Paul Sjoerdsm

Delphi Developer
Mon, 15 Sep 2003 08:17:45 GMT

Re:Need help comparing float types.


In article <3ABF9596.B9853...@improart.com.br>, doug...@improart.com.br
says...
Quote
> Ded wrote:

> > "Mark A. Grover" wrote:

> > > When I insert a floating point number in an Interbase database table, it
> > > doesn't save it exactly as the number was entered.  Instead, it often times
> > > carries the decimal places out several positions.  This becomes troublesome
> > > when comparing the the fields in Delphi because if a field was entered as
> > > 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> > > 1.5.  What is the best way to work around this?

> >     Old good practice in comparing floats is
> >  Abs(F1-F2)<precision.

> I know someone would come up with a decent solution.

> Thanks Ded.

This is the stored proc I use. It only uses 2 places for precision.

create procedure fp_equals (QTY_1 numeric(15, 2), QTY_2 numeric(15, 2))
returns (EQUALS smallint)
as
declare variable diff      numeric(15,2);
   declare variable absdiff   numeric(15,2);
begin
   diff     = qty_1 - qty_2;
   if (diff < 0.00) then
      absdiff  = -diff;
   else
      absdiff = diff;

   if (:absdiff < 0.01) then
      equals = 1;
   else
      equals = 0;
end

regards
Paul Sjoerdsma

Ded

Delphi Developer
Mon, 15 Sep 2003 18:29:56 GMT

Re:Need help comparing float types.


   There is standard UDF Abs in IB, have you encountered any troubles with it?

Other Threads

1. Help needed on comparing two float(currency) variables

2. numeric field type same as float type??

3. Float Type HELP ME !!!

4. I need help comparing 2 string var for order of presedence

5. help needed in comparing 2 interbase databases

6. !Help needed handling floating point fields

7. Need floating toolboxes help....

8. Need help with formating floats

9. Comparing 2 record type variables

10. Comparing type libraries