by **Bob Nemec** » Fri Nov 30, 2007 10:59 am

I tripped across this number when checking the output of a calculation. It was included in an array of numbers (so the array also could not be printed). I'll see if I can reproduce the math.

As for our #displayString, we've written it deal with some issues on GemStone as well as VA (same code in both). We normally use an explicit number of decimal places, so in this case we'd use #displayStringTo8, which answers '0.06425880'.

Within our #displayString method, we send 'n := n - (n truncated)', which answers 0.0642588s24 ... then, to extract the digits, I change it to a large integer using 'n := ((n * (10 raisedTo: anInteger)) + 0.5) truncated', where anInteger is the requested number of digits. By default, it's the scale of the ScaledDecimal, 24 in this case and, in the case of #displayStringTo8, 8.

This gives 64258799999999998427136, which is where the extra digits come from. Since it rounds off ok, we can live with it. Buy I may look more at the #printOn: method for ScaledDecimal to clean this up in our VA version.

Bob Nemec

Northwater Capital