Bug #4108
closed
Saxon-EE crashes when optimizing stylesheet with "too many" user functions
100%
Description
When Saxon-EE tries to optimize an xslt with "too many" user functions, it gives the following crash:
Exception in thread "main" java.lang.AssertionError at net.sf.saxon.expr.Operand.setChildExpression(Operand.java:74) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:382) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.SingletonAtomizer.simplify(SingletonAtomizer.java:70) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.FilterExpression.simplify(FilterExpression.java:203) at net.sf.saxon.expr.ItemChecker.simplify(ItemChecker.java:86) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at net.sf.saxon.expr.Expression.simplifyChildren(Expression.java:381) at net.sf.saxon.expr.Expression.simplify(Expression.java:363) at com.saxonica.ee.optim.OptimizerEE.tryInlineFunctionCall(OptimizerEE.java:1142) at net.sf.saxon.expr.UserFunctionCall.optimize(UserFunctionCall.java:406) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.SlashExpression.optimize(SlashExpression.java:349) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.sort.DocumentSorter.optimize(DocumentSorter.java:102) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.UnaryExpression.optimize(UnaryExpression.java:102) at net.sf.saxon.expr.Atomizer.optimize(Atomizer.java:236) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.Expression.optimizeChildren(Expression.java:617) at net.sf.saxon.expr.FunctionCall.optimize(FunctionCall.java:236) at net.sf.saxon.expr.SystemFunctionCall.optimize(SystemFunctionCall.java:238) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.LetExpression.optimize(LetExpression.java:341) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.Expression.optimizeChildren(Expression.java:617) at net.sf.saxon.expr.instruct.Block.optimize(Block.java:658) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.LetExpression.optimize(LetExpression.java:353) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.instruct.ForEach.optimize(ForEach.java:297) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.Expression.optimizeChildren(Expression.java:617) at net.sf.saxon.expr.instruct.Block.optimize(Block.java:658) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.LetExpression.optimize(LetExpression.java:353) at net.sf.saxon.expr.Operand.optimize(Operand.java:206) at net.sf.saxon.expr.Expression.optimizeChildren(Expression.java:617) at net.sf.saxon.expr.instruct.ParentNodeConstructor.optimize(ParentNodeConstructor.java:221) at net.sf.saxon.expr.instruct.FixedElement.optimize(FixedElement.java:102) at net.sf.saxon.expr.parser.ExpressionTool.optimizeComponentBody(ExpressionTool.java:1201) at com.saxonica.ee.trans.TemplateRuleInitializer.init(TemplateRuleInitializer.java:63) at com.saxonica.ee.trans.TemplateRuleEE.initialize(TemplateRuleEE.java:85) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:496) at net.sf.saxon.trans.rules.TextOnlyCopyRuleSet.process(TextOnlyCopyRuleSet.java:71) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:482) at net.sf.saxon.trans.XsltController.applyTemplates(XsltController.java:733) at net.sf.saxon.s9api.AbstractXsltTransformer.applyTemplatesToSource(AbstractXsltTransformer.java:347) at net.sf.saxon.s9api.Xslt30Transformer.applyTemplates(Xslt30Transformer.java:311) at net.sf.saxon.Transform.processFile(Transform.java:1276) at net.sf.saxon.Transform.doTransform(Transform.java:804) at net.sf.saxon.Transform.main(Transform.java:80)
I'll upload a small test set still giving the problem. The xslt is part of an application checking the quality of export files. I removed the content of some functions not being used in this check, but needed by other checks.
There is no problem with saxon-he-9.9.1.1 or saxon-ee-9.8.0.15.
Files
Updated by Michael Kay almost 6 years ago
Thanks for reporting it; I have reproduced the problem.
It's a failure that occurs during function inlining, and as a temporary workaround, you can suppress function inlining using the command line option -opt:-f
Updated by Michael Kay almost 6 years ago
The optimizer is inlining the function shared:is-valid. The expression $entity/@validFromDate was originally compiled with a docOrder() operator which sorts the result into document order and eliminates duplicates; this is needed because no type information (as clause) was given for the $entity parameter, so the compiler has to assume the worst case. Once the function is inlined, (specifically the call shared:is-valid(., $validityDate), the optimizer knows that $entity is a singleton so the sort operation can be dispensed with, and the failure occurs while removing this operation from the expression tree.
So another workaround is to declare the type of the function parameter: <xsl:param name="entity" as="element()"/> - which is always good practice anyway.
Updated by Johan Gheys almost 6 years ago
OK, thanks for the advice. I will change the xslt accordingly.
Updated by Michael Kay almost 6 years ago
To fix this I have changed the simplify() method for ConditionalSorter to recognise the case where either the simplification of the condition, or the simplification of the documentSorter, indicates that sorting is no longer required. The same change is also made to the typeCheck() and optimize() methods, which although they aren't failing in this case, seem to exhibit the same underlying problem.
Because all three methods now need to be the same, I've taken the opportunity to do some generalization exploiting Java 8 higher order functions; simplify(), typeCheck(), and optimize() for this class now all invoke a generic rewrite() method taking a Rewriter as an argument, eliminating duplicate code. This feels like a mechanism that could be usefully extended to other classes in the future.
First attempt at fix caused regression (stack overflow with recursive optimization) in test axes-057. This is caused by the ConditionalSorter calling optimize() on the underlying DocumentSorter, which then rewrites itself to a ConditionalSorter. I have fixed this by adding a tweak to DocumentSorter.optimize() so it never rewrites itself as a ConditionalSorter if its parent expression is already a ConditionalSorter.
Updated by Michael Kay almost 6 years ago
- Category set to Internals
- Status changed from New to Resolved
- Assignee set to Michael Kay
- Fix Committed on Branch 9.9, trunk added
Fixed on 9.9 and trunk.
Updated by O'Neil Delpratt over 5 years ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in Maintenance Release 9.9.1.2 added
Bug issue fixed in the Saxon 9.9.1.2 maintenance release.
Please register to edit this issue
.