Saxon

Thanks for reporting it. With Saxon 9.8.0.6 this is failing

Type error at char 36 in xsl:variable/@select on line 125 column 175 of indexmodule.xsl:
  XPTY0004: A sequence of more than one item is not allowed as the first argument of
  fn:generate-id() (<xe/>, <xe/>) 
  at xsl:call-template name="process-index" (file:/Users/mike/bugs/2017/nanz/indexmodule.xsl#44)
  in built-in template rule for /manual/index[1] in the unnamed mode
  in built-in template rule for /manual in the unnamed mode

The code is doing Muenchian grouping and is assuming the XSLT 1.0 rule where generate-id() applied to a sequence ignores all items in the sequence other than the first. To retain that behaviour in later XSLT versions you need to enable backwards compatibility mode by setting version="1.0".

I will now see what happens under 9.7.

Under 9.7 it crashes as reported. Is there a good reason you can't move forward to 9.8?

It seems that the expression tree is being left in an inconsistent state after an optimization rewrite that rewrites

generate-id(X) = generate-id(Y)

as

X is Y

An unusual feature of this problem is that it's a little unpredictable. It's happening perhaps 3 times out of 4; other times it runs successfully.

I have simplified the code (to reduce the number of optimizer rewrites that make debugging difficult), and the problem still occurs with this template:

<xsl:template name="show_index_xml">
        <fo:block background-color="rgb(255,235,235)" page-break-before="always">
            <xsl:call-template name="show_xml">
                <xsl:with-param name="xml_source">
                    <sorted>
                        <xsl:for-each select="//xe[. is key('id-letter-key',substring(primary,1,1))[1]]">
                            <xsl:for-each select="key('id-letter-key',substring(primary,1,1))[. is key('id-level1-key',concat(substring(primary,1,1), ' ' ,primary))[1]]">
                                <xsl:for-each select="key('id-level1-key',concat(substring(primary,1,1), ' ' ,primary))[. is key('id-level2-key',concat(substring(primary,1,1), ' ' , primary, ' ' ,secondary))[1]]">
                                    <xsl:sort select="secondary" lang="{$language}"/>
                                    <xsl:copy-of select="secondary"/>
                                </xsl:for-each>
                            </xsl:for-each>
                        </xsl:for-each>
                    </sorted>
                </xsl:with-param>
            </xsl:call-template>
        </fo:block>
    </xsl:template>

I have now further reduced it to:

   <xsl:template name="show_index_xml">
        <xsl:for-each select="//xe">
            <xsl:for-each select="key('id-letter-key',primary)">
                <xsl:for-each select="key('id-level1-key',primary)">
                    <xsl:sort select="secondary" lang="{$language}"/>
                    <xsl:copy-of select="secondary"/>
                </xsl:for-each>
            </xsl:for-each>
        </xsl:for-each>   
    </xsl:template>

Thank you very much for looking into it so quickly!

Regarding your question on upgrading to 9.8: We wouldn't want to rush that, because we probably still have a pile of XSLT 1.0 Stylesheets still hanging around (this particular one, for example is originally XSLT 1.0) and as far as I understand the XSLT 1.0 compatibility has been removed from Saxon 9.8.

Also, I'm a bit concerned about our whole XSLT 2.0 codebase and how well that works with 9.8 as far as compatibility goes. We would probably need to thoroughly test that before making the move to 9.8.

I've just stepped through a run where (infuriatingly) the bug didn't happen, but what was going on was mighty peculiar. The $language variable reference was identified as suitable for promotion (loop-lifting) out of the intermediate for-each loop, which was done; but the next phase of optimization effectively undid this by recognizing the existence of a (constructed) expression of the form "let $x := $y return EXPR", which is rewritten by returning EXPR with all references to $x replaced by references to $y.

It's not clear why a global variable reference should be loop-lifted in the first place.

Note that the loop-lifting code was completely redesigned for 9.8.

Update: for this test I had changed the global variable declaration for $language to declare the type as xs:string. What was happening here was that the expression string-join($language, ' ') was first loop-lifted, was then optimized (using the type information) to $language, and was then inlined, leaving the expression essentially as it started. Now running again with the original xsl:param declaration for $language.

I changed it back to a param. Crashes when run normally, no crash when run under the debugger. A Heisenbug.

Note: it's likely to be a while before we do the next 9.7 maintenance release. A circumvention in the meantime is to add the variable declaration

<xsl:variable name="language" select="$language" as="xs:string"/>

to the start of the relevant template.

Thank you for the work-around. that will do for us.