Number>>roundTo: seems wrong to me

[jtuchel]

Hi there,

not sure I understand the results of evaluating the following two expressions:

0.015 roundTo: 0.01 --> 0.02 -- I would consider this to be correct
1.015 roundTo: 0.01 --> 1.01 -- I need some explanation on this one: why is this 0.005 closer to zero than the 0.005 in the first expression where it seems to be closer to eternity??

The above results are answered in 7.5 as well as 8.0.2 (since the implementation is the same one from 1996).

BTW: converting the Numbers to ScaledDecimals before rounding makes no difference here.

[jtuchel]

Hi Again!

It must be me, since Pharo delivers the same result (it has the same implementation of roundTo:).

So is there any explanation for this? What's the idea of roundTo: ?

Joachim

[jtuchel]

... and again.

Since I thought this must be my fault and I am surely missing something, I decided to ask Wikipedia. What I've found is this page:
http://en.wikipedia.org/wiki/IEEE_754-2008#Rounding_algorithms.

So IEEE defines two rounding algorithms (it actually defines more, but only these two are relevant for my case):

* Round to nearest, ties to even – rounds to the nearest value; if the number falls midway it is rounded to the nearest value with an even (zero) least significant bit, which occurs 50% of the time; this is the default algorithm for binary floating-point and the recommended default for decimal
* Round to nearest, ties away from zero – rounds to the nearest value; if the number falls midway it is rounded to the nearest value above (for positive numbers) or below (for negative numbers)

The results of rounding in VA are somewhat surprising (needless to say that Pharo delivers the very same results)

(0.015 asScaledDecimal: 3) roundTo: 0.01 --> 0.02
(1.015 asScaledDecimal: 3) roundTo: 0.01 --> 1.01
(2.015 asScaledDecimal: 3) roundTo: 0.01 --> 2.02
(3.015 asScaledDecimal: 3) roundTo: 0.01 --> 3.02
(4.015 asScaledDecimal: 3) roundTo: 0.01 --> 4.01
(5.015 asScaledDecimal: 3) roundTo: 0.01 --> 5.01

So at first sight, definition 2 is not the one that's implemented. (But it's the one I'd like to see here)
Then, it seems like I am bitten by the least significant bit stuff. Both 0.015 and 1.015 have their LSB set, so they have to be rounded to the nearest value with an unset LSB.

I could possibly live with this effect for Float numbers, but surely not for Scaled Decimals...

Joachim

[jtuchel]

So all of this Float stuff is a mess, to make it short.

Look at this:

1.015s roundTo: 0.01 --> 1.01
1.015s roundTo: 0.01s --> 1.02s2
1.015 asScaledDecimal roundTo: 0.01s --> 1.01s2

This last one results from the fact that 1.015 is a Float, and asScaledDecimal turns it into a very precise representation of it.

Unfortunately, there is not a simple way to create a scaled decimal other than typing in the suffix or parsing a String .
The fun goes on:
(ScaledDecimal fromString: '1.015') roundTo: 0.01 --> 1.01
(ScaledDecimal fromString: '1.015') roundTo: 0.01s --> 1.02s

So in the end we've learned once again that Floats are evil for domains like money. And that roundTo: makes a difference between Floats and ScaledDecimals as their parameter (not really a surprise when you think about it).

Finally, I've found a solution that seems to work for my purpose:

(0.015 asScaledDecimal: 3) roundTo: 0.01s --> 0.02s2
(1.015 asScaledDecimal: 3) roundTo: 0.01s --> 1.02s2
(2.015 asScaledDecimal: 3) roundTo: 0.01s --> 2.02s2
(3.015 asScaledDecimal: 3) roundTo: 0.01s --> 3.02s2
(4.015 asScaledDecimal: 3) roundTo: 0.01s --> 4.02s2

Since this is not a Smalltalk-related problem, there is no good solution to this, other than avoiding Floats

[tstalzer]

Well - the whole "Float thing" is a classic as why we only use ScaledDecimal in our banking projects

Just try this

Code: Select all

99.9 - 101.1

You'll get

Code: Select all

-1.19999999999999

And banks / insurances don't like the result

--Thomas

[PhotonDemon]

Hi Joachim,

I'm not quite sure what you are trying to do If it is to enter ScaledDecimals in your code than stuff like 0.015s2 works fine, unless you don't like the way it looks. ScaledDecimal fromString: works if you read the string from a text file. If you get data from an SQL database decimal columns are ScaledDecimals. I agree floats are evil but I'm confused as to what the ScaledDecimal problem is?

Lou

[jtuchel]

Lou,

the main problem is a 15 year old code base with loads of little hacks around rounding problems and such which has been converting Decimal Columns from the Database to Floats from the very beginning. So it's not an afternoon hack to change everything to ScaledDecimals. They even made GUI converters for their monetary amount classes that produce floats. Up to a few weeks ago, we sometimes even had Fractions as monetary amounts... Getting rid of these made a few interesting places in the code base swim up to the surface...

I was probably a bit shocked about the fact that it even matters if you roundTo: a ScaledDecimal or a Float. Very logical once you think about it, but sometimes you sit in the middle of a mess and fail in looking at things while thinking about them. Especially when you are just hours away from the next delivery deadline ))

So what have we learned: nothing we didn't know before: beware of Floats if you handle business figures like money, percentages and the like. And the guys who introduced Floats back then should have to pay my bill for this bug

[tstalzer]

Joachim

Fractions are actually not so bad when it comes to precision. In the "Model" layer (where you do all the calculating, etc.) I prefer to have Fractions, Integers and ScaledDecimals. The "pain" comes, when you try to make it visible on the GUI. Especially if you have to recalculate values (e.g. 3 payments for 100 EUR are not three times(100/3) but 33, 33 and 34....)

--Thomas

[jtuchel]

Thomas,

Fractions are fine, so long as you know how to handle rounding and displaying. They can be very bad if you go and multiply them with large numbers and get a "Precision exceeds Maximum" Exception for no apparent reason based on the amounts the user entered.

But you are right, I'd rather handle Fractions than Floats...

[PhotonDemon]

Hi Joachim,

I know you're a smart guy, so I'm sure you have thought of this and have a good reason for not doing it but maybe it is time to re-consider dumping all the Float code and going to ScaledDecimal. You said the database columns are decimal and it seems this is a financial application so it should be the natural fit.

The original author may have chosen Floats for reasons of speed, float math in the hardware is faster than decimal math. But I doubt that ever payed off. The cost of converting to floats is high and I doubt the math was intense enough to make up for the conversion.

You mentioned that constants like 1.234 generate a Float and you need to enter 1.234s3 to get a decimal. This gave me an idea. Could the compiler be changed to check a flag (maybe set in a pragma) that meant real numbers would be compiled to decimals and not floats. The pragma could be defined in a class method of the application and be in effect for all methods housed in the application.

Lou

[marten]

... and again.

Since I thought this must be my fault and I am surely missing something, I decided to ask Wikipedia. What I've found is this page:
http://en.wikipedia.org/wiki/IEEE_754-2008#Rounding_algorithms.

Well, I think its also a problem with the implementation of the arithmetics. On the other hand - when working with floating points one should think about implementing an arithmetic subsystem like described in "Object-Oriented implementation of numerical methods" (also Smalltalk code is available from that book). It takes care of all kind of comparisions ...

Then it might be also a problem with the implementations. I did a wrapper around several numerical library function (base runtime libraries) of OpenOffice - e.g rounding calls and you will not see these problems there. I will upload these wrappers (MSKOOBaseDLLWrapper) to vastgoodies.

It will not help ypu - most of the people do not use OpenOffice - but it shows what good implementation can do. Actually these functions are used in the scalc (=excel like) module of that software application.

[Thomas Holzer]

Hi rounding error victims,

Another solution - long time ago we have written a precise rounding method for floats:

Code: Select all

Float>>preciseRoundTo: relevantDecimals
   "< relevantDecimals:  iOf Integer,  ^iOf Float>"
   "Given a float, return and round it to the given number of relevant decimals
   Since the IBM ST roundTo: returns *wrong* numbers like
   (0.35 * 1.5) roundTo: 0.01 -> 0.52
   (0.35 * 1.5) preciseRoundTo: 2 -> 0.53
   (0.35 * 1.5) Pocket Calculator ->  0.53"

   | precision |

   precision := (relevantDecimals + 1) powerOfTen.
   ^10.0 * ((self * precision + (self sign * 5.01)) * 0.1) truncated /
      precision asFloat


It answers the correct values:

0.015 preciseRoundTo: 2 -> 0.02
1.015 preciseRoundTo: 2 -> 1.02

Hope this helps

Regards

Thomas