Board index » delphi » Need help comparing float types.

Need help comparing float types.

Quote
Ded wrote:
> "Mark A. Grover" wrote:

> > When I insert a floating point number in an Interbase database table, it
> > doesn't save it exactly as the number was entered.  Instead, it often times
> > carries the decimal places out several positions.  This becomes troublesome
> > when comparing the the fields in Delphi because if a field was entered as
> > 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> > 1.5.  What is the best way to work around this?

>     Old good practice in comparing floats is
>  Abs(F1-F2)<precision.

I know someone would come up with a decent solution.

Thanks Ded.

dog

--
___________________________________________
IMPROART
Rua Emiliano Perneta, 297 - 6 And - Cj. 63
CEP 80050-010 - Curitiba-PR - Brasil
Tel/fax.: 55-41-323-7558

 

Re:Need help comparing float types.


When I insert a floating point number in an Interbase database table, it
doesn't save it exactly as the number was entered.  Instead, it often times
carries the decimal places out several positions.  This becomes troublesome
when comparing the the fields in Delphi because if a field was entered as
1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
1.5.  What is the best way to work around this?

Re:Need help comparing float types.


Quote
"Mark A. Grover" wrote:
> When I insert a floating point number in an Interbase database table, it
> doesn't save it exactly as the number was entered.  Instead, it often times
> carries the decimal places out several positions.  This becomes troublesome
> when comparing the the fields in Delphi because if a field was entered as
> 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> 1.5.  What is the best way to work around this?

    Old good practice in comparing floats is
 Abs(F1-F2)<precision.

Re:Need help comparing float types.


Quote
"Mark A. Grover" wrote:
> When I insert a floating point number in an Interbase database table, it
> doesn't save it exactly as the number was entered.  Instead, it often times
> carries the decimal places out several positions.  This becomes troublesome
> when comparing the the fields in Delphi because if a field was entered as
> 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> 1.5.  What is the best way to work around this?

If you're using IB 6, use Numeric(x,y) instead of Float and Double Precision.
Numeric datatype stores precise numbers.
If you're still on 5.6 (as I do) or earlier, well use Double precision instead
of Float. That won't solve the problem, I know... One way is to compare the
strings, as in
FormatFloat('###,###,##0.0#', 1.5) = FormatFloat('###,###,##0.0#',
MyField.AsFloat);
Yes, I find that terrible too, but it's a solution and you can specify how many
decimal places you want to compare.
I hope someone can post a better one, as I also have this kind of problems.

hth,

dog

--
___________________________________________
IMPROART
Rua Emiliano Perneta, 297 - 6 And - Cj. 63
CEP 80050-010 - Curitiba-PR - Brasil
Tel/fax.: 55-41-323-7558

Re:Need help comparing float types.


Interbase: Try to use NUMERIC instead of FLOAT -- it's stored as integer.
(fixed comma)
General: Never actually compare floating values. Compare the difference to
some small value.
e.g. FLOAT1 = FLOAT2 -> FLOAT1 - FLOAT2 < 1e-6
"Mark A. Grover" <mark.gro...@seaquistperfect.com> schrieb im Newsbeitrag
news:3abf8ef6$1_1@dnews...
Quote
> When I insert a floating point number in an Interbase database table, it
> doesn't save it exactly as the number was entered.  Instead, it often
times
> carries the decimal places out several positions.  This becomes
troublesome
> when comparing the the fields in Delphi because if a field was entered as
> 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> 1.5.  What is the best way to work around this?

Re:Need help comparing float types.


In article <3ABF9596.B9853...@improart.com.br>, doug...@improart.com.br
says...
Quote
> Ded wrote:

> > "Mark A. Grover" wrote:

> > > When I insert a floating point number in an Interbase database table, it
> > > doesn't save it exactly as the number was entered.  Instead, it often times
> > > carries the decimal places out several positions.  This becomes troublesome
> > > when comparing the the fields in Delphi because if a field was entered as
> > > 1.5 but stored as 1.500000122 (or whatever) it no longer compares equal to
> > > 1.5.  What is the best way to work around this?

> >     Old good practice in comparing floats is
> >  Abs(F1-F2)<precision.

> I know someone would come up with a decent solution.

> Thanks Ded.

This is the stored proc I use. It only uses 2 places for precision.

create procedure fp_equals (QTY_1 numeric(15, 2), QTY_2 numeric(15, 2))
returns (EQUALS smallint)
as
declare variable diff      numeric(15,2);
   declare variable absdiff   numeric(15,2);
begin
   diff     = qty_1 - qty_2;
   if (diff < 0.00) then
      absdiff  = -diff;
   else
      absdiff = diff;

   if (:absdiff < 0.01) then
      equals = 1;
   else
      equals = 0;
end

regards
Paul Sjoerdsma

Re:Need help comparing float types.


   There is standard UDF Abs in IB, have you encountered any troubles with it?

Other Threads