Bug #3706
closedArray out of bounds exception results from function in template predicate
100%
Description
A template with a function in a predicate in combination with xsl:next-match is resulting in an array out of bounds exception.
Versions affected¶
I have reproduced this error in Saxon HE 9.8.0.8 and 9.7.0.19. It does not affect HE 9.6.0.8. I have tested with both Java and .NET releases of Saxon.
Steps to reproduce¶
- Convert the attached input.xml using the attached convert.xsl in Saxon HE 9.8.0.8.
Expected results¶
-
The transform produces an empty XML document.
-
Note that you can still reproduce the error even if you alter the XSLT to ensure that a root element should be in the output.
Actual results¶
- You get an exception stack trace such as the attached exception stacktrace.txt. Here is an initial summary:
java.lang.RuntimeException: Internal error evaluating template rule at line 14 in module file:/C:/Users/jgomez/Desktop/saxon-bug.xsl
at net.sf.saxon.expr.instruct.TemplateRule.applyLeavingTail(Unknown Source)
at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:447)
at net.sf.saxon.trans.TextOnlyCopyRuleSet.process(TextOnlyCopyRuleSet.java:65)
at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:433)
...
Files
Related issues
Updated by Michael Kay over 6 years ago
Thanks for reporting it. It's in the general area of component linking which changed with the introduction of XSLT 3.0 packages. Shouldn't be too hard to get to the bottom of it.
Updated by Michael Kay over 6 years ago
My first attempt to reproduce this is unsuccessful (the stylesheet runs successfully).
Could be related to https://saxonica.plan.io/issues/3508
Updated by Michael Kay over 6 years ago
Reproduced by running on Saxon-EE with all optimizations disabled. I suspect function inlining prevents the problem occurring.
Updated by Michael Kay over 6 years ago
Definitely related to bug 3508. I think the patch for that bug didn't consider all cases: it was concerned primarily with calls of the built-in template rules. In this situation it seems two related things are wrong:
(a) we're not creating a new context for evaluating the match patterns in xsl:next-match
(b) we're not setting the current component when evaluating these match patterns, so the binding to the user-defined function doesn't fix up correctly.
Updated by Michael Kay over 6 years ago
- Related to Bug #3508: Internal error evaluating template rule (involves xsl:next-match) added
Updated by Michael Kay over 6 years ago
- Related to Bug #2818: xsl:next-match as last instruction in a named template causes IndexOutOfBoundsException added
Updated by Michael Kay over 6 years ago
We need to consider three paths here: apply-templates, next-match, and apply-imports. The most performance-critical, of course, is apply-templates.
apply-templates starts by creating a new context c2 and a new focusIterator corresponding to the select expression. It calls Mode.getRule() with this new context c2, and Mode.getRule() uses c2 for evaluating match patterns except in the case where one or match patterns declares local variables, in which case it creates a new context for evaluating match patterns. The selected template rule is also called with context c2, which has the effect that within that template, position() and last() refer to the sequence of items selected by apply-templates/@select.
next-match calls a variant of Mode.getRule() supplying the context of the caller. The caller's context is therefore used for evaluating match patterns except in the case where one or match patterns declares local variables. A new context is then created for evaluating the selected template rule.
apply-imports shares some (but not all) code with next-match and the logic is basically equivalent.
There are other paths that are relevant including (a) streaming versions of these instructions, (b) implicit calls on apply-templates within built-in template rules, and (c) bytecode versions of the instructions. These follow similar logic, any any changes need to be kept in line.
The simplest change (A) is probably to create a new context for evaluating match patterns unconditionally. Unfortunately this would add instructions to the core apply-templates path, which is best avoided.
Another option (B) is to be smarter about when we create a new context for evaluating match patterns: specifically, as well as declaring range variables, calling user-defined functions should trigger this. But is this complexity really justified?
Option (C) is to change next-match and apply-imports to create the new context earlier, and use this for evaluating patterns. Possibly then the code in getRule() to create a new context for pattern evaluation becomes unnecessary. Unfortunately this involves changing bytecode and streaming paths, so a lot more testing and risk of introducing new problems.
Option (D) - a variant of (A) - is to create a new context for pattern evaluation unconditionally when doing next-match or apply-imports, but not when doing apply-templates.
Updated by Michael Kay over 6 years ago
On further investigation, it seems that the mechanism of allocating a new context for evaluating patterns is used only for accumulator rules. For template rules, it seems that the variable Mode.stackFrameSlotsNeeded is always zero, so no new context is allocated.
Note that the variable Mode.stackFrameSlotsNeeded is set from XSLTemplate.allocatePatternSlotNumbers, not from SimpleMode.allocatePatternSlots.
Updated by Michael Kay over 6 years ago
I've made some progress as follows:
(a) in Mode.makeNewContext(), set the current component to be the current mode
(b) in GeneralPositionalPattern.getDependencies(), retain dependencies on user-defined functions. (Will need to do the same for other kinds of pattern)
(c) in XSLTemplate.allocatePatternSlotNumbers(), ensure that at least one slot is allocated if the pattern calls external functions. The slot is not needed, but its presence forces a new context to be created for evaluating patterns.
I'm not convinced by (a). It works for xsl:next-match and xsl:apply-imports because we always want to use the current mode in these cases, but it's not right for apply-templates.
Updated by Michael Kay over 6 years ago
I think the change (a) above is safe, because on the applyTemplates path, the currentMode has already been set to the target mode.
Regarding (b), the only other affected Pattern is a GeneralNodePattern.
It's possible that the changes made for bug 3508 (creating a new context for a built-in apply-templates) are now unnecessary. I'll do some tests.
Updated by Michael Kay over 6 years ago
The changes cause 5 tests in test set source-document to fail, e.g. -t:non-stream-200. This appears to be a problem with patterns in accumulators; the test has accumulator patterns that refer to global variables.
The problem here is that the accumulator code is calling Mode.getRule() with a context in which the current component is the accumulator, but Mode.getNewContext() is changing this so that the current component is null (because the current mode is null).
We can fix this, rather clunkily, by making Mode.makeNewContext() recognise the case where the current component is an Accumulator.
With this, all XSLT 3.0 tests pass on 9.8, both with and without bytecode enabled.
Updated by Michael Kay over 6 years ago
- Category set to Internals
- Status changed from New to Resolved
- Assignee set to Michael Kay
- Priority changed from Low to Normal
- Applies to branch trunk added
- Applies to branch deleted (
9.7) - Fix Committed on Branch 9.8, trunk added
Fixed on the 9.8 and trunk branches.
Test case next-match-038 added.
Updated by O'Neil Delpratt over 6 years ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in Maintenance Release 9.8.0.10 added
Bug fix applied in the Saxon 9.8.0.10 maintenance release.
Please register to edit this issue