Project

Profile

Help

Bug #5292

closed

Wrong results from 11.1 XQuery (caused by ZenoSequence problem)

Added by Gunther Rademacher about 2 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
XQuery conformance
Sprint/Milestone:
-
Start date:
2022-02-07
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
11, trunk
Fix Committed on Branch:
11, trunk
Fixed in Maintenance Release:
Platforms:

Description

This is very similar to #5281, in that it omits some parts of a result. However I don't expect the fix for the latter to help here, because I understand that the fix is in ArrayIterator.Of.getReverseIterator, and this one does not pass by there. Also there is no invocation of last() here.

The attached repro2.xq produces different results in 10.6 and 11.1, the 11.1 ones being incorrect.


Files

repro2.xq (3.14 KB) repro2.xq Gunther Rademacher, 2022-02-07 16:37
Actions #1

Updated by Michael Kay about 2 years ago

  • Category set to XQuery conformance
  • Status changed from New to In Progress
  • Assignee set to Michael Kay
  • Priority changed from Low to Normal
  • Applies to branch trunk added

Thanks for reporting it! Very curious.

I added to line 3 so it reads

if (empty(trace($todo, '$todo'))) then

and the trace is the same for 10.x and 11.1 until we get to:

10.x:

$todo [1]: text(): root/text()[36]
$todo [1]: element(fragment, xs:untyped): root/fragment[23]
$todo [1]: element(fragment, xs:untyped): root/fragment[24]
$todo [1]: element(fragment, xs:untyped): root/fragment[25]
$todo [1]: element(tab, xs:untyped): root/tab[39]
$todo [1]: element(fragment, xs:untyped): root/fragment[26]
$todo [1]: element(tab, xs:untyped): root/tab[40]
$todo [1]: text(): root/text()[37]

at which point 11.1 does:

$todo [1]: text(): root/text()[36]
$todo [1]: element(fragment, xs:untyped): root/fragment[23]
$todo [1]: element(fragment, xs:untyped): root/fragment[24]
$todo [1]: element(fragment, xs:untyped): root/fragment[25]
$todo [1]: element(tab, xs:untyped): root/tab[39]
$todo [1]: element(fragment, xs:untyped): root/fragment[26]
$todo [1]: element(tab, xs:untyped): root/tab[40]
$todo: empty sequence

(note that empty() only reads, and therefore only traces, the first item in the sequence)

The failure occurs with or without bytecode generation.

Now I've added tracing on the value of count($todo), and in 10.x it drops incrementally down to zero, whereas in 11.1 it drops incrementally down to 12, and then ends with

$todo [1]: element(tab, xs:untyped): root/tab[40]
$count [1]: xs:integer: 12
$todo: empty sequence

I'm now breakpointing at the point where the trace(count) returns "12".

The next $item is <tab col="9"/>.

I see here that the value of the parameter $todo was a ZenoChain with segment lengths 1 and 12, and the result of evaluating tail() on this (written as $todo[position()>1]) is a ZenoChain with segment lengths 0 and 12. This is incorrect: the first segment should have been removed, rather than being reduced to zero length, and this is almost certainly why the test on empty() has gone wrong.

The ZenoChain structure is new in 11.x - it's an immutable list structure optimised for recursive addition or removal of items at the start or end of a sequence, designed to reduce the amount of copying that occurs when sequences are built incrementally.

Or to be a little more precise, ZenoSequence is an immutable list structure built on top of ZenoChain which is itself mutable. Both of them could do with better JavaDoc.

Actions #2

Updated by Michael Kay about 2 years ago

So, an off-by-one bug at ZenoChain.sublist(), line 219: if (offset + segment.size() >= start) should read if (offset + segment.size() > start).

Specifically, we should retain a segment if its size will be one-or-more, not if it will be zero-or-more.

It's worth adding some internal unit tests of ZenoChain, which is currently tested only via the XPath constructs it is used in, and that's probably not a very thorough collection - and certainly not oriented to probing the boundary conditions.

Actions #3

Updated by Michael Kay about 2 years ago

Documenting and testing the ZenoChain code reveals that there is confusion about whether it's supposed to be immutable or not. The add() and prepend() methods modify the structure in-situ, while concat() and sublist() don't. There's potential for some nasty bugs here. The ZenoSequence, which is built on top of the ZenoChain, is supposed to be immutable, but that's not the case unless the caller exercises great care.

Actions #4

Updated by Michael Kay about 2 years ago

  • Status changed from In Progress to Resolved
  • Fix Committed on Branch trunk added

I've done a substantial clean up of the ZenoChain and ZenoSequence classes. They are both now immutable, and they are much better documented and better tested. I've also address some performance issues (though those might have been introduced by my changes).

Actions #5

Updated by Michael Kay about 2 years ago

  • Subject changed from Wrong results from 11.1 XQuery to Wrong results from 11.1 XQuery (caused by ZenoSequence problem)
Actions #6

Updated by Debbie Lockett about 2 years ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fix Committed on Branch 11 added
  • Fixed in Maintenance Release 11.2 added

Bug fix applied in the Saxon 11.2 maintenance release. (But we failed to mark this bug as such at the time.)

Please register to edit this issue

Also available in: Atom PDF