Bug #5690
closed
incorrect result of bin:unpack-integer when using 'little-endian' option
Applies to branch:
10, 11, trunk
Fix Committed on Branch:
10, 11, trunk
Fixed in Maintenance Release:
Description
the expath bin extension unpack-integer return an incorrect result when used with the 'little endian' option
- bin:unpack-unsigned-integer(bin:hex('8008'),0,2,'BE') returns 32776 as expected
- bin:unpack-unsigned-integer(bin:hex('8008'),0,2,'LE') returns 2176 as expected
- bin:unpack-integer(bin:hex('8008'),0,2,'BE') returns -32760 as expected
- bin:unpack-integer(bin:hex('8008'),0,2,'LE') returns -63360 when 2176 is expected.
For information, I've noticed that
Saxon result = Expected result - 65536
Seen in Saxon PE 11.4, Saxon EE 10.6 integrated in OxygenXML
Files
For what it's worth, I have tried bin:unpack-integer(bin:hex('8008'),0,2,'LE') in BaseX 10.1 and its result is -63360.
Thanks for reporting it. The bug seems to be the complement of bug #5447 - probably reflects the same oversight in devising a comprehensive set of test cases.
- Status changed from New to In Progress
I have added these four test cases to the expath-binary test set, and confirmed the bug.
I've fixed the problem: with LE order, the sign bit needs to be taken from the second octet, not the first,
The patch works for the 11.x and 12.x code.
For Saxon 10.x I had a little difficulty reproducing the problem. It turns out my Saxon 10 configuration is using a slightly out of date version of the EXPath tests. On updating this, I'm getting additional test failures in bin:pack-integer -- despite the fact that the record for bug #5447 indicates that the patch for that bug was applied to 10.x.
It seems that after closing 5447, more tests were added, and this led to further changes being made to the code, which were not retrofitted to the 10.x branch. This has now been corrected.
- Status changed from In Progress to Resolved
- Assignee set to Michael Kay
- Priority changed from Low to Normal
- Applies to branch 11, trunk added
- Fix Committed on Branch 10, 11, trunk added
Also seen on 4 bytes integers.
However, the result is correct on 8 bytes integer. I guess it is reaching the 64 bits limit!
I can confirm that the code takes a different path if the length exceeds 7 bytes (because the internal representation of the result is a BigInteger. in this case) The logic is a bit convoluted for this case and I haven't attempted to understand it fully.
- % Done changed from 0 to 100
- Fixed in Maintenance Release 12.0 added
Bug issue fix applied in the Saxon 12.0 Major Release.
Leaving this bug marked as Resolved until fix applied
- Fixed in Maintenance Release 11.5 added
Bug fix applied in the Saxon 11.5 maintenance release.
- Status changed from Resolved to Closed
- Fixed in Maintenance Release 10.9 added
Bug fix applied in the Saxon 10.9 maintenance release.
Please register to edit this issue
Also available in: Atom
PDF
.