←back to thread

631 points wojtczyk | 1 comments | | HN request time: 0s | source
Show context
rgovostes ◴[] No.41410786[source]
A friend doing homework for a university assignment, circa Leopard or Snow Leopard, noticed that Calculator produced negative values when raising a negative number to an even power.

The bug turned out to be in CFNumber, in Core Foundation. CFNumber does a lot of fiddly stuff at the bit level for performance, and one of their optimizations for exponentiation was incorrect. Somehow it was never found by tests or due to buggy behaviors it created in other apps, but by someone clicking buttons and thinking critically about the output.

replies(5): >>41410865 #>>41411004 #>>41411294 #>>41411316 #>>41412437 #
LeoPanthera ◴[] No.41411294[source]
The Windows 3.1 calculator has a fun bug where 2.01 - 2.00 returns the result of 0.00, not 0.01.

That always amused me.

replies(1): >>41411669 #
canucker2016 ◴[] No.41411669[source]
Acording to https://en.wikipedia.org/wiki/Windows_Calculator#Windows_9x_..., the Win98 version of calc.exe got an arbitrary-precision math library for basic math functionality to fix this sort of problem.

also mentioned by Raymond Chen, https://devblogs.microsoft.com/oldnewthing/20040525-00/?p=39... :

"Today, Calc’s internal computations are done with infinite precision for basic operations (addition, subtraction, multiplication, division) and 32 digits of precision for advanced operations (square root, transcendental operators)."

And they fixed the square-root-of-a-perfect-square bug a few years ago, https://www.reddit.com/r/Windows10/comments/89s53g/microsoft...

replies(1): >>41415713 #
zelphirkalt ◴[] No.41415713[source]
Can this be true? Does the calculator have any irrational number? What if I enter PI * 2? Infinite precision, wouldn't that mean running until your RAM is full? Or would it simply work, because no irrational number is calculated to infinite (as many as the device could) digits and is therefore finite and can easily be multiplied?

Or another even simpler case: If division is infinite precision, and I enter (2/3) * X, does the calculator internally work with fractions? Otherwise it would have infinite digits to compute.

replies(2): >>41423206 #>>41437507 #
1. canucker2016 ◴[] No.41437507[source]
For PI, calc.exe, in scientific mode, returns 3.1415926535897932384626433832795 which is 32 digits of precision (33 chars if you include the decimal pt).