Project

Profile

Help

Actions
Send by e-mail Copy link

Bug #4424

closed

Arithmetic with large integers

Added by Michael Kay over 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Low
Assignee:
Michael Kay
Category:
XPath Conformance
Sprint/Milestone:
-
Start date:
2020-01-14
Due date:
% Done:

0%

Estimated time:
Applies to JS Branch:
Fix Committed on JS Branch:
Fixed in JS Release:
Saxon-JS 2.0
SEF Generated with:
Platforms:
Company:
-
Contact person:
-
Additional contact persons:
-

Description

Test case op-numeric-subtractpint2args-3

The result of 52704602390610034 - 52704602390610033 is returned as 0 rather than 1.

The incorrect logic is in Calculator.js:

      "i-i": function (lhs, rhs) {
            const x = lhs.toNumber() - rhs.toNumber();
            return isSafeInteger(x) ? INT.fromNumber(x) : INT.fromBig(Big(lhs.toBig().minus(rhs.toBig())));

Here lhs.toNumber() and rhs.toNumber() both return the nearest float; the result of the subtraction is zero, which is a safe integer, so it is returned without further checking.

Although this is most likely to happen with a subtraction, the same problem applies to all the integer operations.

Actions
Copy link
 #1

Updated by Michael Kay over 4 years ago

Also noted, the logic for XdmDecimal.compareTo() seems to be wrong. I don't have a test case for this, but the path

           if (other.toDouble) {
                return this.value - other.toDouble().value;

makes no sense because (a) this.value is a Big, while (b) other.toDouble().value returns a JS Number.

Actions
Copy link
 #2

Updated by Michael Kay almost 4 years ago

  • Status changed from New to Closed
  • Fixed in JS Release set to Saxon-JS 2.0

This (both problems) appears to have been fixed in Saxon-JS 2.0

Actions
Copy link
 #3

Updated by Debbie Lockett almost 4 years ago

  • Category set to XPath Conformance

Please register to edit this issue

Actions
Send by e-mail Copy link

Also available in: Atom PDF Tracking page