Bug #5690: incorrect result of bin:unpack-integer when using 'little-endian' option - Saxon - Saxonica Developer Community

Actions

Send by e-mail Copy link

Bug #5690

closed

incorrect result of bin:unpack-integer when using 'little-endian' option

Added by Olivier XILLO about 2 years ago. Updated almost 2 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Michael Kay

Category:

Sprint/Milestone:

Start date:

2022-09-17

Due date:

% Done:

100%

Estimated time:

Legacy ID:

Applies to branch:

10, 11, trunk

Fix Committed on Branch:

10, 11, trunk

Fixed in Maintenance Release:

12.0, 11.5, 10.9

Platforms:

.NET, Java

Description

the expath bin extension unpack-integer return an incorrect result when used with the 'little endian' option

bin:unpack-unsigned-integer(bin:hex('8008'),0,2,'BE') returns 32776 as expected
bin:unpack-unsigned-integer(bin:hex('8008'),0,2,'LE') returns 2176 as expected
bin:unpack-integer(bin:hex('8008'),0,2,'BE') returns -32760 as expected
bin:unpack-integer(bin:hex('8008'),0,2,'LE') returns -63360 when 2176 is expected.

For information, I've noticed that

Saxon result = Expected result - 65536

Seen in Saxon PE 11.4, Saxon EE 10.6 integrated in OxygenXML

Files

binUnpack_bug.xsl (1001 Bytes) binUnpack_bug.xsl

stylesheet to reproduce

Olivier XILLO, 2022-09-17 16:03

Actions

Copy link

Updated by Martin Honnen about 2 years ago

For what it's worth, I have tried bin:unpack-integer(bin:hex('8008'),0,2,'LE') in BaseX 10.1 and its result is -63360.

Actions

Copy link

Updated by Michael Kay about 2 years ago

Thanks for reporting it. The bug seems to be the complement of bug #5447 - probably reflects the same oversight in devising a comprehensive set of test cases.

Actions

Copy link

Updated by Michael Kay about 2 years ago

Status changed from New to In Progress

I have added these four test cases to the expath-binary test set, and confirmed the bug.

Actions

Copy link

Updated by Michael Kay about 2 years ago

Platforms .NET added

I've fixed the problem: with LE order, the sign bit needs to be taken from the second octet, not the first,

Actions

Copy link

Updated by Michael Kay about 2 years ago

The patch works for the 11.x and 12.x code.

For Saxon 10.x I had a little difficulty reproducing the problem. It turns out my Saxon 10 configuration is using a slightly out of date version of the EXPath tests. On updating this, I'm getting additional test failures in bin:pack-integer -- despite the fact that the record for bug #5447 indicates that the patch for that bug was applied to 10.x.

It seems that after closing 5447, more tests were added, and this led to further changes being made to the code, which were not retrofitted to the 10.x branch. This has now been corrected.

Actions

Copy link

Updated by Michael Kay about 2 years ago

Status changed from In Progress to Resolved
Assignee set to Michael Kay
Priority changed from Low to Normal
Applies to branch 11, trunk added
Fix Committed on Branch 10, 11, trunk added

Actions

Copy link

Updated by Olivier XILLO about 2 years ago

Also seen on 4 bytes integers. However, the result is correct on 8 bytes integer. I guess it is reaching the 64 bits limit!

Actions

Copy link

Updated by Michael Kay about 2 years ago

I can confirm that the code takes a different path if the length exceeds 7 bytes (because the internal representation of the result is a BigInteger. in this case) The logic is a bit convoluted for this case and I haven't attempted to understand it fully.

Actions

Copy link