Project

Profile

Help

Bug #2189

closed

Failure to stream output

Added by Michael Kay over 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Streaming
Sprint/Milestone:
Start date:
2014-10-18
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
9.6
Fix Committed on Branch:
9.6
Fixed in Maintenance Release:
Platforms:

Description

On 18 Oct 2014, at 10:23, Costello, Roger L. wrote [on the saxon-help list]:

Hi Michael,

The below program generates a huge amount of output, using streaming.

It worked fine in Saxon version 9.5.1.6

It fails in the latest version of Saxon - an Out of Memory error is generated.


<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"

           xmlns:math="http://www.w3.org/2005/xpath-functions/math"

           version="3.0">

<xsl:output method="xml" />

<xsl:mode streamable="yes" />

<xsl:template match="/">

   <sine-values>

       <xsl:for-each select="0 to 100000000">

           <sine>

               <number><xsl:value-of select="." /></number>

               <sine><xsl:value-of select="math:sin(.)" /></sine>

           </sine>

       </xsl:for-each>

   </sine-values>

</xsl:template>

</xsl:stylesheet>

My response:

Interesting!

Firstly, "streaming" in the XSLT 3.0 sense is all about streaming the input; the spec takes for granted that streaming the output isn't a problem. But here, the problem is that the output isn't being streamed.

What seems to be happening here is that when you execute a non-consuming template in streaming mode then it is evaluated in "pull" rather than "push" mode, which means any output from that template rule is built in memory rather than being streamed straight to the result stream. Your match="/" is non-consuming (it doesn't read anything from the input document), and it's executed in streaming mode (because the mode is declared streamable), so the element is being constructed in memory. I don't think there is any particularly good reason for this.

Michael Kay

Saxonica

The issue is at ApplyTemplatesAction line 233 where we do:

            SequenceIterator si = body.iterate(localContext);

            processItems(si, out);

Executing the body using iterate() rather than process() in this case causes the single element produced by the template to be materialized, and then written to the output in a single call of processItems.

Actions #1

Updated by Michael Kay over 9 years ago

For this test case, replacing these two lines by

body.process(localContext);

solves the problem. However, it will need careful regression testing for more complex streaming scenarios.

Actions #2

Updated by Michael Kay over 9 years ago

The change seems OK as far as the W3C streaming tests are concerned. Also need to test it with streaming extension tests involving overlapping matches.

Actions #3

Updated by Michael Kay over 9 years ago

  • Status changed from New to Resolved

The patch appears OK so I'm marking as resolved.

Actions #4

Updated by O'Neil Delpratt over 9 years ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in version set to 9.6.0.2

Bug fix applied to the maintenance release Saxon 9.6.0.2

Actions #5

Updated by O'Neil Delpratt over 8 years ago

  • Sprint/Milestone set to 9.6.0.2
  • Applies to branch 9.6 added
  • Fix Committed on Branch 9.6 added
  • Fixed in Maintenance Release 9.6.0.2 added

Please register to edit this issue

Also available in: Atom PDF