Bug #4754
closedxs:decimal idiv operator (suspicious code)
100%
Description
I'm testing various changes in handling of Atomic.js, and hit a test failure in K-NumericIntegerDivide-1
. The test does
(xs:decimal(6) idiv xs:integer(2)) instance of xs:integer
which is supposed to return true, but it's coming out as false.
In the course of this I found code which I haven't changed and which looks wrong.
Calculate.js line 207 has
"c~c": function (lhs, rhs) {
// TODO explicit divByZero check? FOAR0002?
try {
const a = lhs.toBig(), b = rhs.toBig();
const x = a.div(b).round(0, 0);
return isSafeInteger(x) ? INT.fromNumber(x) : DEC.fromBig(x);
}
catch (e) {
divByZeroErr();
}
},
Here a
and b
are Big objects, so x
is also a Big
, but isSafeInteger
is designed to operate on a JS number.
I suspect isSafeInteger() applied to a Big is always returning false (because Math.abs() returns NaN), so we are always following the DEC.fromBig() path. With the current Atomic.js
, DEC.fromBig()
always returns an xs:decimal
, causing this test to fail. Perhaps in the released code, there was some compensating code that caused DEC.fromBig() to return an xs:integer
in this case?
Please register to edit this issue
Also available in: Atom PDF Tracking page