Saxon

Reproduced. Although the problem goes away when you run without a license file, it doesn't go away when you run with -opt:0, and it does go away when you run with --allow-multithreading:off, so I think it's probably something to do with asynchronous handling of xsl:result-document rather than with any rewrite optimisation.

This looks like a very basic coding error: in SortedGroupIterator method buildArray, the assignment

values = new ObjectToBeSorted[allocated];

should read

values = new GroupToBeSorted[allocated];

It's one of those bugs that begs the question "How did this ever work?", and "How come the tests didn't catch this?"

A query against the test suite shows that there are no tests in which xsl:result-document appears as a descendant of xsl:for-each-group, which is the trigger condition for this bug.

The combinatorics of testing every function and every instruction in every (even just every valid) context is a little mind boggling. I wonder if a bunch of them could be generated. I mean, yes, a bunch of them could clearly be generated, but would the low hanging fruit be useful?

I refined the search and did find some tests, though most of them are streaming tests, which follow a different path:

xslt40-test/tests/fn/json-to-xml/json-to-xml-B.xsl
xslt40-test/tests/fn/json-to-xml/json-to-xml-B2.xsl 
xslt40-test/tests/strm/si-fork/si-fork-802.xsl 
xslt40-test/tests/strm/si-fork/si-fork-B.xsl 
xslt40-test/tests/strm/si-for-each-group/si-group-032.xsl 
xslt40-test/tests/strm/si-for-each-group/si-group-031.xsl 
xslt40-test/tests/strm/si-for-each-group/si-group-204.xsl 
xslt40-test/tests/strm/si-for-each-group/si-group-205.xsl

and the only two non-streaming stylesheets (json-to-xml-B and -B2) are not actually used in any tests!

I resuscitated test json-to-xml-013 which uses the stylesheet json-to-xml-B.xsl, and it didn't hit the bug. Turns out there's another precondition for the bug, which is that the xsl:for-each-group has an xsl:sort child - the problem is in SortedGroupIterator. Confirmed that adding the xsl:sort does trigger the bug, and that the proposed patch fixes it.

The combinatorics of testing every function and every instruction in every (even just every valid) context is a little mind boggling.

Indeed. We do make attempts to ensure coverage, for example there is an introspection test to ensure that every combination of XSLT element/attribute permitted by the schema is present in some non-error stylesheet. But there's certainly no attempt to ensure that every permitted parent/child element combination is present. And as the above comment demonstrates, we actually need a combination of three instructions here: xsl:for-each-group/xsl:sort/xsl:result-document.

Another way of approaching this is by looking at code coverage at the Java level. We've attempted that in the past, but the problem is simply that there are too many code fragments that don't get executed to examine them all in detail; and when you do examine them, most of them are there for a good reason, for example if a class has an equals() method then it's right and proper to also have a hashCode() method even though it is never used. I suppose in an ideal world we would also write a unit test to test the hashCode() method - but life is too short.

It appears this bug was introduced between Saxon 10 and 11 - the relevant line of code was correct in Saxon 10, and was changed for some reason.

It appears this bug was introduced between Saxon 10 and 11 - the relevant line of code was correct in Saxon 10, and was changed for some reason.

Looking at the commit history, the change was made on 9/3/2021 [I think that's 2021-03-09 - MHK] and was done to enable C# transpilation. I haven't yet checked that the fixed code transpiles successfully. It seems that if a variable is declared with type ObjectToBeSorted[] then in Java we can assign it to an array of type GroupToBeSorted given that GroupToBeSorted is a subtype of ObjectToBeSorted[]; this causes problems in C#. So we may need to do some rethinking.