Board index » delphi » FloatToStr gets crazy numbers

FloatToStr gets crazy numbers

We've been running across a problem in numerous apps weve created where numbers such as 0.95 when converted to string using FloatToStr come up as 0.95000001 (not exact example, but valid).

When I use Debug/Evaluate it sure as hell shows up as 0.95, so it's not an issue with the number itself.  And what's frustrating is that whatever function was used to generate the string in the Debug/Evaluate window seems to work fine.  Suggestions?

Since I can have a variable number of digits after the decimal I'm not sure if FloatToStrF would be appropriate.

Thanks in advance.

--
---------------------------------------------------------------
Tom O'Connor
Software Engineer
Boston Software, Corp.
t...@bostonsoftware.com
Ph: 781.449.8585   Fx: 781.449.8522

 

Re:FloatToStr gets crazy numbers


Quote
Richard Weeks <rich...@bostonsoftware.com> wrote in message

news:STmJ5.816$Mf4.130063@news.shore.net...
Quote
> We've been running across a problem in numerous apps weve created where

numbers such as 0.95 when converted to string using FloatToStr come up as
0.95000001 (not exact example, but valid).
Quote

> When I use Debug/Evaluate it sure as hell shows up as 0.95, so it's not an

issue with the number itself.  And what's frustrating is that whatever
function was used to generate the string in the Debug/Evaluate window seems
to work fine.  Suggestions?
Quote

> Since I can have a variable number of digits after the decimal I'm not

sure if FloatToStrF would be appropriate.

You might find Format is more useful since you can control the number of
decimal digits in the string. You are, I'm sure, aware that some real
numbers do not have an exact representation when using a floating point
type.

Re:FloatToStr gets crazy numbers


Dear Richard Weeks,

it sounds to me, that you use single or real48 (the old real type of delphi) precision numbers instead of higher precision or at least that your data transfer uses in some point a number of that low precision.

Better use double or extended precision (the last one can result in compartibilitiy problems, but double should always work) and than use FloatToStrF with at least 2 or three digits of PRECISION (ffGeneral) or DIGITS (ffFixed) less then the data type
garanties. This decimal precision of the data types can be found in the delphi help using double, real or extended as key words (you should get 15-16 for double, 19-20 for extended etc).

I'm not really sure what you mean with:

<<Since I can have a variable number of digits after the decimal I'm not sure if FloatToStrF would be appropriate.>>

since FloatToStr uses FloatToStrF with ffGeneral and Precision = 15 and Digits = 0. Or what else do you use?

And please note, as the former answerer Bruce Roberts suggested, that floating point numbers always are only approximations to real numbers!

Good luck,

Daniel Spangenberg

Re:FloatToStr gets crazy numbers


Look at FormatFloat in the Delphi help files

Mike
--
Mike Best Programming
Brisbane
Australia

Quote
Richard Weeks <rich...@bostonsoftware.com> wrote in message

news:STmJ5.816$Mf4.130063@news.shore.net...
Quote
> We've been running across a problem in numerous apps weve created where

numbers such as 0.95 when converted to string using FloatToStr come up as
0.95000001 (not exact example, but valid).
Quote

> When I use Debug/Evaluate it sure as hell shows up as 0.95, so it's not an

issue with the number itself.  And what's frustrating is that whatever
function was used to generate the string in the Debug/Evaluate window seems
to work fine.  Suggestions?
Quote

> Since I can have a variable number of digits after the decimal I'm not

sure if FloatToStrF would be appropriate.
Quote

> Thanks in advance.

> --
> ---------------------------------------------------------------
> Tom O'Connor
> Software Engineer
> Boston Software, Corp.
> t...@bostonsoftware.com
> Ph: 781.449.8585   Fx: 781.449.8522

Re:FloatToStr gets crazy numbers


Quote
In article <STmJ5.816$Mf4.130...@news.shore.net>, Richard Weeks wrote:
>V2UndmUgYmVlbiBydW5uaW5nIGFjcm9zcyBhIHByb2JsZW0gaW4gbnVtZXJvdXMgYXBwcyB3ZXZl

I fully agree.

Other Threads