Project

Profile

Help

Bug #5690

closed

incorrect result of bin:unpack-integer when using 'little-endian' option

Added by Olivier XILLO over 1 year ago. Updated about 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
2022-09-17
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
10, 11, trunk
Fix Committed on Branch:
10, 11, trunk
Fixed in Maintenance Release:
Platforms:
.NET, Java

Description

the expath bin extension unpack-integer return an incorrect result when used with the 'little endian' option

  • bin:unpack-unsigned-integer(bin:hex('8008'),0,2,'BE') returns 32776 as expected
  • bin:unpack-unsigned-integer(bin:hex('8008'),0,2,'LE') returns 2176 as expected
  • bin:unpack-integer(bin:hex('8008'),0,2,'BE') returns -32760 as expected
  • bin:unpack-integer(bin:hex('8008'),0,2,'LE') returns -63360 when 2176 is expected.

For information, I've noticed that

Saxon result = Expected result - 65536

Seen in Saxon PE 11.4, Saxon EE 10.6 integrated in OxygenXML


Files

binUnpack_bug.xsl (1001 Bytes) binUnpack_bug.xsl stylesheet to reproduce Olivier XILLO, 2022-09-17 16:03
Actions #1

Updated by Martin Honnen over 1 year ago

For what it's worth, I have tried bin:unpack-integer(bin:hex('8008'),0,2,'LE') in BaseX 10.1 and its result is -63360.

Actions #2

Updated by Michael Kay over 1 year ago

Thanks for reporting it. The bug seems to be the complement of bug #5447 - probably reflects the same oversight in devising a comprehensive set of test cases.

Actions #3

Updated by Michael Kay over 1 year ago

  • Status changed from New to In Progress

I have added these four test cases to the expath-binary test set, and confirmed the bug.

Actions #4

Updated by Michael Kay over 1 year ago

  • Platforms .NET added

I've fixed the problem: with LE order, the sign bit needs to be taken from the second octet, not the first,

Actions #5

Updated by Michael Kay over 1 year ago

The patch works for the 11.x and 12.x code.

For Saxon 10.x I had a little difficulty reproducing the problem. It turns out my Saxon 10 configuration is using a slightly out of date version of the EXPath tests. On updating this, I'm getting additional test failures in bin:pack-integer -- despite the fact that the record for bug #5447 indicates that the patch for that bug was applied to 10.x.

It seems that after closing 5447, more tests were added, and this led to further changes being made to the code, which were not retrofitted to the 10.x branch. This has now been corrected.

Actions #6

Updated by Michael Kay over 1 year ago

  • Status changed from In Progress to Resolved
  • Assignee set to Michael Kay
  • Priority changed from Low to Normal
  • Applies to branch 11, trunk added
  • Fix Committed on Branch 10, 11, trunk added
Actions #7

Updated by Olivier XILLO over 1 year ago

Also seen on 4 bytes integers. However, the result is correct on 8 bytes integer. I guess it is reaching the 64 bits limit!

Actions #8

Updated by Michael Kay over 1 year ago

I can confirm that the code takes a different path if the length exceeds 7 bytes (because the internal representation of the result is a BigInteger. in this case) The logic is a bit convoluted for this case and I haven't attempted to understand it fully.

Actions #9

Updated by Community Admin about 1 year ago

  • % Done changed from 0 to 100
  • Fixed in Maintenance Release 12.0 added

Bug issue fix applied in the Saxon 12.0 Major Release. Leaving this bug marked as Resolved until fix applied

Actions #10

Updated by O'Neil Delpratt about 1 year ago

  • Fixed in Maintenance Release 11.5 added

Bug fix applied in the Saxon 11.5 maintenance release.

Actions #11

Updated by O'Neil Delpratt about 1 year ago

  • Status changed from Resolved to Closed
  • Fixed in Maintenance Release 10.9 added

Bug fix applied in the Saxon 10.9 maintenance release.

Please register to edit this issue

Also available in: Atom PDF