Saxon

Weird bug.

I tried to check whether using a function instead of a template helps but the code

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet 
  version="3.0" 
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:mf="http://example.com/mf"
  exclude-result-prefixes="#all"
  expand-text="yes">

  <xsl:variable name="data">
    <root>
      <item name="bbb" />
      <item name="ddd" />
      <item name="bbb" />
      <item name="aaa" />
      <item name="ccc" />
    </root>
  </xsl:variable>
  
  <xsl:function name="mf:group-and-sort" as="element(t)*">
    <xsl:param name="items" as="element(item)*"/>
    <xsl:param name="ordering" as="xs:string"/>
    <xsl:for-each-group select="$items" group-by="@name">
      <xsl:sort select="current-grouping-key()" order="{$ordering}"/>
      <t>{current-grouping-key()}</t>
    </xsl:for-each-group>
  </xsl:function>
  
  <xsl:output indent="yes"/>

  <xsl:template name="xsl:initial-template">
    <root>
      <xsl:for-each select="'descending', 'ascending'">
        <order>{.}</order>
        <xsl:sequence select="mf:group-and-sort($data/root/item, .)"/>
      </xsl:for-each>
    </root>
  </xsl:template>
  
</xsl:stylesheet>

with 12.5 gives the same wrong result

<?xml version="1.0" encoding="UTF-8"?>
<root>
   <order>descending</order>
   <t>ddd</t>
   <t>ccc</t>
   <t>bbb</t>
   <t>aaa</t>
   <order>ascending</order>
   <t>ddd</t>
   <t>ccc</t>
   <t>bbb</t>
   <t>aaa</t>
</root>

Trying with Saxon 12.5 EE for SaxonJS reveals (perhaps?) the root of the problems (Internal Saxon error: local variable encountered whose binding has been deleted)

*** Internal Saxon error: local variable encountered whose binding has been deleted
Variable name: ordering
Line number of reference: 24 in file:/C:/Users/marti/OneDrive/Documents/xslt/blog-xslt-3-by-example/saxon12-group-order-bug/./sheet2.xsl
Line number of declaration: 35 in file:/C:/Users/marti/OneDrive/Documents/xslt/blog-xslt-3-by-example/saxon12-group-order-bug/./sheet2.xsl
DECLARATION:
<?xml version="1.0" encoding="utf-8"?>
<let baseUri='file:/C:/Users/marti/OneDrive/Documents/xslt/blog-xslt-3-by-example/saxon12-group-order-bug/./sheet2.xsl' ns='mf=http://example.com/mf xs=~ xsl=~' line='35' var='Q{}ordering' as='1AS' slot='-999'>
 <dot type='1AS'/>
 <forEachGroup line='23' algorithm='by'>
  <gVarRef role='select' name='Q{http://saxon.sf.net/generated-variable}gg1419332030' bSlot='-1'/>
  <attVal role='key' name='Q{}name'/>
  <sortKey role='sort' line='24'>
   <check role='select' card='?' diag='4|0|XTTE1020|xsl:sort/select'>
    <currentGroupingKey/>
   </check>
   <fn role='order' name='string-join'>
    <varRef name='Q{}ordering' slot='-999'/>
    <str val=' '/>
   </fn>
   <str role='lang' val=''/>
   <str role='caseOrder' val='#default'/>
   <str role='stable' val='yes'/>
   <str role='collation' val='http://www.w3.org/2005/xpath-functions/collation/codepoint'/>
  </sortKey>
  <str role='collation' val='http://www.w3.org/2005/xpath-functions/collation/codepoint'/>
  <elem role='content' line='25' name='t' nsuri='' flags='l'>
   <valueOf flags='l'>
    <fn name='string-join'>
     <convert from='A' to='AS'>
      <currentGroupingKey/>
     </convert>
     <str val=' '/>
    </fn>
   </valueOf>
  </elem>
 </forEachGroup>
</let>
java.lang.IllegalStateException: *** Internal Saxon error: local variable encountered whose binding has been deleted
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:636)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:646)
        at net.sf.saxon.style.StyleElement.allocateLocalSlots(StyleElement.java:1606)
        at net.sf.saxon.style.XSLTemplate.optimize(XSLTemplate.java:999)
        at net.sf.saxon.style.PrincipalStylesheetModule.optimizeTopLevel(PrincipalStylesheetModule.java:1502)
        at net.sf.saxon.style.PrincipalStylesheetModule.compile(PrincipalStylesheetModule.java:1321)
        at net.sf.saxon.style.Compilation.compilePackage(Compilation.java:341)
        at net.sf.saxon.s9api.XsltCompiler.compilePackage(XsltCompiler.java:672)
        at net.sf.saxon.Transform.doTransform(Transform.java:777)
        at net.sf.saxon.Transform.main(Transform.java:84)
Fatal error during transformation: java.lang.IllegalStateException: *** Internal Saxon error: local variable encountered whose binding has been deleted

"Trying with Saxon 12.5 EE for SaxonJS reveals (perhaps?) the root of the problems" was meant to say "Trying with Saxon 12.5 EE to compile/export for SaxonJS reveals (perhaps?) the root of the problems".

Thanks for reporting it.

I can see where it's going wrong: in ForEachGroup it is doing

expr.makeSortKeyEvaluators();

during the first-time-through code without checking that the sort options are constants.

Is there a way I can rewrite it to get it to work? I have some data I want to output multiple times with different sort keys and ordering, without having to duplicate the output code multiple times (the issue isn't affected by the sort key as a parameter which is why I didn't include it in the test case).

Adrian

I added the test case to the xslt40 suite and actually got a worse error: it crashed with

*** Internal Saxon error: local variable encountered whose binding has been deleted

That used to be a common problem but I haven't seen it for a long while; it implies a corruption of the expression tree during an optimisation rewrite. If I run with -opt:0 to disable optimization then the stylesheet runs, producing the incorrect output shown.

The only workaround I can suggest at the moment is to duplicate the code, with one template for ascending grouping and another for descending.

The problem is at line 848 of ForEachGroup.java where the lambda expression returned by getSortedGroupIteratorProvider() assigns to the array comps[]. which is part of the closure of the lambda expression. This has the effect of allocating a comparator on the first-time-through pass, which is then reused on subsequent invocations. The closure of the lambda expression should be immutable.

Rewrote getSortedGroupIteratorProvider() to return different lambda expressions for the fixed and variable cases. Test case is now working with -opt:0. Still getting the internal error if run with optimisation enabled.

A summary of the optimizer trace is:

OPT : At line 35 
OPT : Moved function mf:group-and-sort inline: 
OPT : Expression after rewrite: let $items := ($data/child::element(Q{}root))/child::element(Q{}item) return (let $ordering := . return (ForEachGroup(...
OPT : Inlined references to $items
OPT : Expression after rewrite: let $ordering := . return (ForEachGroup(....)
OPT : At line 23 of file:/Users/mike/GitHub/qt4cg/xslt40-test/tests/insn/for-each-group/for-each-group-093.xsl
OPT : Eliminated unused variable ordering

Inlining the function is reasonable, but it has been done incorrectly, failing to recognize that the binding is context-dependent. Everything else follows from that.