##### Quote

>Subject: Weird maths (sqrt)

>From: ni...@trump.net.au (Nick Alcorso)

>Date: Sun, 28 Sep 1997 06:04:51 GMT

>Message-id: <342df1f4.16917...@newsroom.trump.net.au>

>I am getting stupid results in a procedure that i am trying to do.

>I am getting invalid floading point on a sqrt. Basically all i am

>doing is finding the distance froma point using pythagoras theorem but

>for some reason i am getting negative valuse for my sqr(a) + sqr(b)

>this is how it looks

>M :Array[1..4ish] of longint (i tried real, but it may be bigger)

><snip>

Your code is somewhat messed up but that's besides the point. The real

reason, which nobody who had posted follow-ups for this so far knew, has to

do with the way sqr() works. Specifically, _ the value the sqr function

returns is the ***same type as the parameter***_. This is important

because it leads to potential arithmetic overflows.

For example, let's say X is of type INTEGER, which stores numbers *within

the range -32768..32767*. Let's say X has the value 200. Then SQR (X)

should be equal to 40,000. However, this value will be stored to X's type,

the INTEGER type. This is a problem though, since it's not within the

range of -32768..32767 that the INTEGER type supports.

So what does TP do now? Basically it's going to do the following: 32768

become -32768, 32769 become -32767, 32770, becomes -32766, etc. So under

this system 40,000 is stored as the INTEGER -25536!!! And that's why

you're getting negative values! It's pretty much TP's fault. And it

doesn't matter if you store SQR (X) as a type with a bigger range, since

it's the type of *X* that counts.

The solution? There are basically two ways. One is to declare X to be a

type of larger range. For example, if in the above example we declare X as

the type LONGINT, which supports a range of about -2 billion to +2 billion,

then SQR (X) will give the correct value of 40000, since that value is

within the range of the LONGINT.

Sometimes, however, we might not want to increase the range of the number

just so that SQR (X) will be evaluted correctly. One might choose a type

with smaller range for speed or storage reasons. Fortunately, in TP [not

standard Pascal] there is the second solution: value typecasting. For

example, in the example above, all we want is for SQR (X) to be treated as

a LONGINT value, not an INTEGER value. To do so we can do the following:

SQR (LONGINT (X)). What that does is that we're telling TP to treat the X

as a type of LONGINT even though it's really an INTEGER. So now instead of

using the INTEGER type for the result, it'll treat the result as of type

LONGINT, and so you'll get the desired answer of 40,000!!!

OK, so how does all these apply to your program? Well, without a full

program it's hard for me to say, but a good place to start is to typecast

the entire expression in the SQR () function first to LONGINT, and if that

doesn't work, try REAL. If things still don't work out then typecast the

individual values and variables. Finally, make sure that you're assigning

the result to a type with an appropriate range. For example, if X is of

type INTEGER,

X := SQR (LONGINT (X)) will still *not* work, since although SQR (LONGINT

(X)) gives the correct value of 40,000, you can't store that number in the

variable X, since its type is INTEGER. It'll be like trying to stuff a

schoolbus load of people into a car. It simply won't work.