Bug #2634
closed
XQuery using saxon:stream fails when falling back to non-streaming and byte code generation is in effect
Fixed in Maintenance Release:
Description
This had already been reported in Bug #2550, but I may have failed to make clear that for this one I am interested in saxon:stream's fallback to non-streaming mode, rather than getting it streaming. Later I got the impression that fallback might not be supported any more.
The query is
for $x in saxon:stream(doc('uriresolver:resolve')/*/*/element {node-name()} {node()}) return $x
and I am fine with no streaming being achievable for it. Rather I'd like to see it falling back to non-streaming.
This works with optimzation levels 1-9, and also with 10 when GENERATE_BYTE_CODE is turned off. However when turned on, it comes up with this exception:
net.sf.saxon.trans.XPathException: No watch found for xsl:element
at com.saxonica.ee.bytecode.util.Callback.makeXPathException(Callback.java:143)
at gen_SlashContextMappingFunction_1.map(Unknown Source)
at net.sf.saxon.expr.ContextMappingIterator.next(ContextMappingIterator.java:61)
at net.sf.saxon.query.XQueryExpression$ErrorReportingIterator.next(XQueryExpression.java:836)
at net.sf.saxon.s9api.XdmSequenceIterator.hasNext(XdmSequenceIterator.java:57)
A stack trace, and a complete test case is attached.
Files
- Category set to Streaming
- Status changed from New to In Progress
- Assignee set to Michael Kay
- Priority changed from Low to Normal
The reason fallback doesn't come into play here is that the streamability analysis is failing internally, rather than completing successfully with the conclusion that the expression is streamable or non-streamable. For me it's failing in the same way (No watch for xsl:element) whether or not bytecode is enabled, but this may be something to do with my test driver environment.
This one should really be streamable: now that we have streamable stylesheet functions in XSLT 3.0 (even though they only work in Saxon 9.7 when the function is inlineable), this is a construct that can be reproduced in XSLT 3.0 and should work.
The reason it's failing is something to do with the inserted CopyOf operation (see bug #2550). We're not adding the CopyOf as a node into the expression tree, only as a Feed into the push evaluation pipeline. For some reason the inversion of the element constructor is happening twice, and the second time around it fails to find a consuming operand of this expression. The investigation continues...
Hmmm, this is looking quite messy. It seems the initial streamability analysis of the saxon:stream construct is done before optimization is complete. At this stage we're building the inverted execution path and inserting the extra Copy feed into the push pipeline. But then we do further optimization which invalidates this. Worse, this subsequent optimization inserts a CopyOf node into the expression tree, which makes the Copy feed that we inserted into the pipeline redundant; and because the streamability analysis has already been done, the CopyOf node does not have posture and sweep properties; it's the absence of these properties that is the immediate cause of the failure.
So the solution should be to make sure we don't do the streamability analysis until optimization is complete. (And we can possibly then get rid of the CopyFeed insertion).
The sequence of events is roughly this:
-
During initial parsing of the expression, the function call saxon:stream is converted to a "streaming copy" instruction, corresponding to the old XSLT <xsl:copy read-once="yes">.
-
During the first optimization pass (which is run with optimizeForStreaming=false), this <xsl:copy read-once="yes"> expression is converted to a StreamInstr, more commonly used as the target of an xsl:stream instruction. The prepareForStreaming() method is called on this StreamInstr; this computes the inversion of the expression.
-
Once all the clauses of a FLWOR expression have been optimized, if it turns out to be a single-clause For..Return then it is converted to a ForExpression, and ForExpression.optimize() is called. One thing that happens here is that the verbose expression "for $x in S return $x" is turned into the simpler "S". At the same time, however, S.optimize() is called. This makes a second call on StreamInstr.optimize(). This sets the flag optimizeForStreaming=true while optimizing the body of the StreamInstr, which causes optimization of the element constructor to invoke makeCopyOperationsExplicit(), which inserts a CopyOf call around the content expression of the constructor. There is then a second call on StreamInstr.prepareForStreaming(), which recomputes the inversion of the expression, but this time with the extra CopyOf call. This second attempt to compute the inversion fails because the PostureAndSweep of the inserted CopyOf call is uninitialised.
- Status changed from In Progress to Resolved
I have fixed this test case so it actually uses streaming as follows:
When the <xsl:copy read-once="yes"> instruction is converted to a StreamInstr, then instead of calling StreamInstr.prepareForStreaming() as now, we instead call StreamInstr.optimize(). This calls prepareForStreaming(), but it also sets a flag to ensure that it only happens once.
All regression tests for streaming are passing so I will mark this resolved. I started a new QT3-extra test set for saxon:stream() and this is stream-001.
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in Maintenance Release 9.7.0.6 added
Bug fixed in maintenance release 9.7.0.6.
Please register to edit this issue
Also available in: Atom
PDF
.