Bug #6339
closed
saxon:transform() throws ArrayIndexOutOfBoundsException
100%
Description
We have a transformation that splits one xml in several csv files, and this works fine when called with fn:transform(). In order to take advantage of the direct serialization of saxon:transform(), we switched to saxon:transform(), but this throws the following error:
java.lang.ArrayIndexOutOfBoundsException: -1
at net.sf.saxon.tree.tiny.TinyBuilder.makeTextNode(TinyBuilder.java:516)
at net.sf.saxon.tree.tiny.TinyBuilder.characters(TinyBuilder.java:483)
at net.sf.saxon.event.TreeReceiver.characters(TreeReceiver.java:176)
at net.sf.saxon.serialize.PrincipalOutputGatekeeper.characters(PrincipalOutputGatekeeper.java:83)
at net.sf.saxon.event.ComplexContentOutputter.characters(ComplexContentOutputter.java:275)
at net.sf.saxon.event.Outputter$1.close(Outputter.java:372)
at net.sf.saxon.functions.StringJoin.process(StringJoin.java:140)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPush$8(SystemFunctionCall.java:689)
at net.sf.saxon.expr.instruct.ValueOf$ValueOfElaborator.lambda$elaborateForPush$1(ValueOf.java:420)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$1(Block.java:851)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.instruct.TemplateRule.applyLeavingTail(TemplateRule.java:376)
at net.sf.saxon.trans.Mode.handleRuleNotNull(Mode.java:587)
at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:521)
at net.sf.saxon.trans.rules.TextOnlyCopyRuleSet.process(TextOnlyCopyRuleSet.java:72)
at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:518)
at net.sf.saxon.trans.XsltController.applyTemplates(XsltController.java:684)
at com.saxonica.functions.extfn.TransformFn.call(TransformFn.java:103)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$3(SystemFunctionCall.java:625)
at net.sf.saxon.expr.elab.LazyPullEvaluator.evaluate(LazyPullEvaluator.java:39)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$2(SystemFunctionCall.java:615)
at net.sf.saxon.expr.elab.LazyPullEvaluator.evaluate(LazyPullEvaluator.java:39)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$2(SystemFunctionCall.java:616)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPull$5(LetExpression.java:927)
at net.sf.saxon.expr.instruct.Block$BlockElaborator$BlockIterator.getNthChildIterator(Block.java:832)
at net.sf.saxon.expr.instruct.AbstractBlockIterator.next(AbstractBlockIterator.java:66)
at net.sf.saxon.value.SingletonClosure.asItem(SingletonClosure.java:105)
at net.sf.saxon.value.SingletonClosure.iterate(SingletonClosure.java:65)
at net.sf.saxon.expr.UserFunctionCall$UserFunctionCallElaborator.lambda$elaborateForPull$0(UserFunctionCall.java:746)
at net.sf.saxon.value.SingletonClosure.asItem(SingletonClosure.java:105)
at net.sf.saxon.value.SingletonClosure.materialize(SingletonClosure.java:156)
at net.sf.saxon.expr.elab.LocalVariableEvaluator.evaluate(LocalVariableEvaluator.java:31)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$1(SystemFunctionCall.java:605)
at net.sf.saxon.expr.sort.SortExpression$SortExprElaborator.lambda$elaborateForPull$0(SortExpression.java:511)
at net.sf.saxon.expr.instruct.ForEach$ForEachElaborator.lambda$elaborateForPush$5(ForEach.java:809)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$1(Block.java:853)
at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:247)
at net.sf.saxon.expr.instruct.CallTemplate$CallTemplateElaborator.lambda$elaborateForPush$1(CallTemplate.java:633)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$2(Block.java:864)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$1(Block.java:853)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:247)
at net.sf.saxon.expr.instruct.CallTemplate$CallTemplatePackage.processLeavingTail(CallTemplate.java:515)
at net.sf.saxon.expr.Expression.dispatchTailCall(Expression.java:976)
at net.sf.saxon.expr.instruct.CallTemplate$CallTemplateElaborator.lambda$elaborateForPush$1(CallTemplate.java:634)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$4(Block.java:895)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.TryCatch$TryCatchElaborator.lambda$elaborateForPush$1(TryCatch.java:335)
at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:247)
at net.sf.saxon.trans.XsltController.callTemplate(XsltController.java:872)
at net.sf.saxon.s9api.Xslt30Transformer.callTemplate(Xslt30Transformer.java:503)
at net.sf.saxon.Transform.processFile(Transform.java:1375)
at net.sf.saxon.Transform.doTransform(Transform.java:879)
at net.sf.saxon.Transform.main(Transform.java:83)
java.lang.RuntimeException: Internal error evaluating template rule in module file:/C:/Data/MyProjects/int-batch/src/test/resources/saxon/saxon-transform-xxxx/xslt/convert-xml-report-to-csv.xslt
at net.sf.saxon.expr.instruct.TemplateRule.applyLeavingTail(TemplateRule.java:386)
at net.sf.saxon.trans.Mode.handleRuleNotNull(Mode.java:587)
at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:521)
at net.sf.saxon.trans.rules.TextOnlyCopyRuleSet.process(TextOnlyCopyRuleSet.java:72)
at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:518)
at net.sf.saxon.trans.XsltController.applyTemplates(XsltController.java:684)
at com.saxonica.functions.extfn.TransformFn.call(TransformFn.java:103)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$3(SystemFunctionCall.java:625)
at net.sf.saxon.expr.elab.LazyPullEvaluator.evaluate(LazyPullEvaluator.java:39)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$2(SystemFunctionCall.java:615)
at net.sf.saxon.expr.elab.LazyPullEvaluator.evaluate(LazyPullEvaluator.java:39)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$2(SystemFunctionCall.java:616)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPull$5(LetExpression.java:927)
at net.sf.saxon.expr.instruct.Block$BlockElaborator$BlockIterator.getNthChildIterator(Block.java:832)
at net.sf.saxon.expr.instruct.AbstractBlockIterator.next(AbstractBlockIterator.java:66)
at net.sf.saxon.value.SingletonClosure.asItem(SingletonClosure.java:105)
at net.sf.saxon.value.SingletonClosure.iterate(SingletonClosure.java:65)
at net.sf.saxon.expr.UserFunctionCall$UserFunctionCallElaborator.lambda$elaborateForPull$0(UserFunctionCall.java:746)
at net.sf.saxon.value.SingletonClosure.asItem(SingletonClosure.java:105)
at net.sf.saxon.value.SingletonClosure.materialize(SingletonClosure.java:156)
at net.sf.saxon.expr.elab.LocalVariableEvaluator.evaluate(LocalVariableEvaluator.java:31)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPull$1(SystemFunctionCall.java:605)
at net.sf.saxon.expr.sort.SortExpression$SortExprElaborator.lambda$elaborateForPull$0(SortExpression.java:511)
at net.sf.saxon.expr.instruct.ForEach$ForEachElaborator.lambda$elaborateForPush$5(ForEach.java:809)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$1(Block.java:853)
at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:247)
at net.sf.saxon.expr.instruct.CallTemplate$CallTemplateElaborator.lambda$elaborateForPush$1(CallTemplate.java:633)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$2(Block.java:864)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$1(Block.java:853)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:247)
at net.sf.saxon.expr.instruct.CallTemplate$CallTemplatePackage.processLeavingTail(CallTemplate.java:515)
at net.sf.saxon.expr.Expression.dispatchTailCall(Expression.java:976)
at net.sf.saxon.expr.instruct.CallTemplate$CallTemplateElaborator.lambda$elaborateForPush$1(CallTemplate.java:634)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$4(Block.java:895)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.TryCatch$TryCatchElaborator.lambda$elaborateForPush$1(TryCatch.java:335)
at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:247)
at net.sf.saxon.trans.XsltController.callTemplate(XsltController.java:872)
at net.sf.saxon.s9api.Xslt30Transformer.callTemplate(Xslt30Transformer.java:503)
at net.sf.saxon.Transform.processFile(Transform.java:1375)
at net.sf.saxon.Transform.doTransform(Transform.java:879)
at net.sf.saxon.Transform.main(Transform.java:83)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at net.sf.saxon.tree.tiny.TinyBuilder.makeTextNode(TinyBuilder.java:516)
at net.sf.saxon.tree.tiny.TinyBuilder.characters(TinyBuilder.java:483)
at net.sf.saxon.event.TreeReceiver.characters(TreeReceiver.java:176)
at net.sf.saxon.serialize.PrincipalOutputGatekeeper.characters(PrincipalOutputGatekeeper.java:83)
at net.sf.saxon.event.ComplexContentOutputter.characters(ComplexContentOutputter.java:275)
at net.sf.saxon.event.Outputter$1.close(Outputter.java:372)
at net.sf.saxon.functions.StringJoin.process(StringJoin.java:140)
at net.sf.saxon.expr.SystemFunctionCall$SystemFunctionCallElaborator.lambda$elaborateForPush$8(SystemFunctionCall.java:689)
at net.sf.saxon.expr.instruct.ValueOf$ValueOfElaborator.lambda$elaborateForPush$1(ValueOf.java:420)
at net.sf.saxon.expr.instruct.Block$BlockElaborator.lambda$elaborateForPush$1(Block.java:851)
at net.sf.saxon.expr.LetExpression$LetExprElaborator.lambda$elaborateForPush$6(LetExpression.java:942)
at net.sf.saxon.expr.instruct.TemplateRule.applyLeavingTail(TemplateRule.java:376)
... 43 more
Fatal error during transformation: java.lang.RuntimeException: Internal error evaluating template rule in module file:/C:/Data/MyProjects/int-batch/src/test/resources/saxon/saxon-transform-xxxx/xslt/convert-xml-report-to-csv.xslt
Files
Updated by Johan Gheys 11 months ago
- File saxon-transform-6339.zip saxon-transform-6339.zip added
Updated by Michael Kay 11 months ago
Thanks for submitting it. I have reproduced the problem.
I can improve the diagnostics a little by running with -l (letter ell) on the command line, this builds the tree used for the secondary stylesheet with line numbers, which shows (in the debugger) that we are failing around line 13-16 of convert-xml-report-to-csv.xslt which reads:
<xsl:value-of separator="{$EOL}">
<xsl:sequence select="'serZone;type;tag;description;workingDate'"/>
<xsl:apply-templates select="$incoherency[@workingDate = $working-date]"/>
</xsl:value-of>
Now, that's beautiful code and I like to use that construct myself, but not many people know you can do that so it is probably under-exercised! We're on a path where we are constructing a text node in the result tree incrementally, in push mode; and it seems to be the first node in the result tree, and it seems to have failed because it's looking at whether it should be concatenating the text with a previous text node. That's the first impression, anyway.
Updated by Michael Kay 11 months ago
We somehow seem to be on a path where we are building a TinyTree to contain a single text node, and the TinyBuilder.makeTextNode() path isn't designed/tested to handle this. So there are two questions, (a) why is this stylesheet doing this, and (b) why is it so unusual?
Normally, if we're creating a tree consisting of a single leaf node (text node, comment, etc) we will create an Orphan object, not a TinyTree. But if the node is created using apply-templates, we don't know what kind of node will be created.
I think I need now to explore exactly what is happening on the call to saxon:transform(), which is something that's surely relevant to the problem.
Updated by Michael Kay 11 months ago
It seems that saxon:transform is implemented very differently to fn:transform. It's using internal APIs, whereas fn:transform invokes the transformation using s9api interfaces. It constructs a TinyBuilder, wraps a TreeReceiver around it, and then uses this as the destination for a call to XsltController.applyTemplates(). TreeReceiver.open() gets called (from XsltController.prepareOutputReceiver, from XsltController.applyTemplates() but there is no call on startDocument(). If I add calls to startDocument() and endDocument() (at the point where TransformFn calls XsltController.applyTemplates()) then the problem goes away. But is this the right thing to be doing?
Updated by Michael Kay 11 months ago
I've been doing some experiments to see how fn:transform() handle a stylesheet that outputs a single text node.
In QT4 test case fn-transform-89 I pass stylesheet-text and source-node. The delivery-format option defaults to document, and the returned value contains the generated text node wrapped in a document node. In test fn-transform-90 I change the delivery-format to raw, and the returned value now contains the generated text node without wrapping.
saxon:transform() does not have this delivery-format machinery. It is specified to always return a document node. So I think it should work like fn:transform with delivery-format="document". The way fn:transform does this is to create a DocumentDeliverer, which creates a s9api XdmDestination to use as the destination of the transform call. XdmDestination.getReceiver() constructs a TinyBuilder and wraps it in a SequenceNormalizer. So I think saxon:transform() should probably do the same. (The SequenceNormalizer essentially does the sequence normalization process described in the serialization spec.)
This appears to do the trick, existing tests for saxon:transform work. (Though there are very few tests, it has to be said.)
Updated by Michael Kay 11 months ago
Standing back, we describe saxon:transform as obsolescent, and we would obviously like to get rid of it at some stage in favour of fn:transform. If you're using it because it offers something that fn:transform doesn't, that's a cause for concern. You say you're using it "to take advantage of the direct serialization" - presumably of the output of xsl:result-document.
The theory here is that with fn:transform you should be able to supply a post-process function to perform direct serialization of the result. But although we implement the post-process option, it can't be used to achieve this effect.
I think we should probably provide an option on fn:transform, say "vendor-options": map{"saxon:write-result-documents":true()} which triggers this behaviour.
Updated by Michael Kay 11 months ago
- Status changed from New to In Progress
- Applies to branch 12, trunk added
- Fix Committed on Branch 12, trunk added
- Platforms .NET, Java added
I've committed the fix, and am now thinking about the enhancement to fn:transform. One difficulty with the idea is how to test it: generally we test xsl:result-document by capturing the output using a result document resolver, rather than allowing it to write to file store.
I guess the semantics of "saxon:write-result-documents":true() should probably be that the secondary transformation uses the result document resolver set up for the primary transformation, which means that when running in the test driver, the result documents of the secondary transformation will appear exactly as if they were result documents of the primary transformation.
Updated by Michael Kay 11 months ago
- Category set to Saxon extensions
- Status changed from In Progress to Resolved
- Assignee set to Michael Kay
I have implemented changes to fn:transform for Saxon 13 that will allow secondary result document output to be serialized direct to filestore, essentially by inheriting the result document handler from the invoking stylesheet. In addition, the base output URI of the target stylesheet defaults to that of the invoker.
Updated by Michael Kay 11 months ago
One more thing.
As a workaround pending the next maintenance release, wrapping the output of the target stylesheet explicitly in a document node using an xsl:document instruction should do the trick.
Updated by Johan Gheys 11 months ago
Thanks for the workaround and - as usual - you are right again:
<xsl:document>
<xsl:value-of separator="{$EOL}">
<xsl:sequence select="'serZone;type;tag;description;workingDate'"/>
<xsl:apply-templates select="$incoherency[@workingDate = $working-date]"/>
</xsl:value-of>
</xsl:document>
solves the problem. But the final solution where we can always use fn:transform() with an extra parameter is, of course, the ultimate. Thanks again. We are already looking forward to Saxon 13!
Updated by O'Neil Delpratt 6 months ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in Maintenance Release 12.5 added
Bug fix applied in the Saxon 12.5 Maintenance release.
Please register to edit this issue
.