Project

Profile

Help

Bug #4424

closed

Arithmetic with large integers

Added by Michael Kay about 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
XPath Conformance
Sprint/Milestone:
-
Start date:
2020-01-14
Due date:
% Done:

0%

Estimated time:
Applies to JS Branch:
Fix Committed on JS Branch:
Fixed in JS Release:
SEF Generated with:
Platforms:
Company:
-
Contact person:
-
Additional contact persons:
-

Description

Test case op-numeric-subtractpint2args-3

The result of 52704602390610034 - 52704602390610033 is returned as 0 rather than 1.

The incorrect logic is in Calculator.js:

      "i-i": function (lhs, rhs) {
            const x = lhs.toNumber() - rhs.toNumber();
            return isSafeInteger(x) ? INT.fromNumber(x) : INT.fromBig(Big(lhs.toBig().minus(rhs.toBig())));

Here lhs.toNumber() and rhs.toNumber() both return the nearest float; the result of the subtraction is zero, which is a safe integer, so it is returned without further checking.

Although this is most likely to happen with a subtraction, the same problem applies to all the integer operations.

Please register to edit this issue

Also available in: Atom PDF Tracking page