Saxon

Saxon will always attempt to stream the output when it can: you don't need to do anything special, other than ensuring that you choose a streamed output destination for the transformation (for example a JAXP StreamResult or a s9api Serializer). This is true (and has always been true) regardless of whether the input is streamed.

There are some things that will prevent output being streamed, the main one being try/catch (xsl:try has to buffer output in memory so it can be discarded in the event of a failure). The only other examples that comes to mind are xsl:on-empty / xsl:on-non-empty and xsl:fork

There were changes in the 10.x release to improve the output buffering of constructs that can generate large single text nodes, for example fn:concat(), fn:string-join(), fn:unparsed-text(), fn:xml-to-json(). These functions are now capable of executing in "push mode", writing their output incrementally to the serialized result tree, rather than accumulating it in memory.

As regards tutorials, I think Abel Braaksma has published some good conference papers.

Thank you very much, Michael. You wrote - "a streamed output destination for the transformation". Can I achieve this using the command-line interface, so that the tool is pure XSLT?

Sure, when running from the command line the output document is always serialized, so this always applies.

Ah - good to know, thank you! Remark: when running within the Oxygen IDE, the output is apparently not streamed. I suppose the Oxygen team can say something about configuration options required for streamed output.

As for an introduction to streaming in form of a primer, does the section https://www.saxonica.com/html/documentation/sourcedocs/streaming/ in the Saxon documentation help perhaps? Or did you already read it and it did not help at all?

Thank you for this tip, Martin - this is certainly a good start, which I had overlooked!

[A thorough tutorial might nevertheless make sense and fill what appears to me as a gap.]

Attempting a trivial transformation, I get this error:

Error on line 48837274 column 47 of foo.xml:
  SXXP0003   Error reported by XML parser: JAXP00010004: Die akkumulierte Größe von Entitys
  ist "50.000.001" und überschreitet den Grenzwert "50.000.000", der von
  "FEATURE_SECURE_PROCESSING" festgelegt wurde.: JAXP00010004: Die akkumulierte Größe von
  Entitys ist "50.000.001" und überschreitet den Grenzwert "50.000.000", der von
  "FEATURE_SECURE_PROCESSING" festgelegt wurde.
org.xml.sax.SAXParseException; systemId: file:/C:/products/saxon/result.xml; lineNumber: 48837274; columnNumber: 47; JAXP00010004: Die akkumulierte Größe von Entitys ist "50.000.001" und überschreitet den Grenzwert "50.000.000", der von "FEATURE_SECURE_PROCESSING" festgelegt wurde.

FEATURE_SECURE_PROCESSING? Does someone have a recipe what to do?

Using SaxonEE10-3J, command-line interface. Stylesheet:

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="3.0">
    <xsl:output indent="true"/>
    <xsl:mode streamable="yes" on-no-match="deep-skip"/>
    
    <xsl:template match="/">   
        <wiki>
            <xsl:iterate select="*/page">
                <page title="{substring(title, 1, 3)}"/>
            </xsl:iterate>
        </wiki>
    </xsl:template>
    
</xsl:stylesheet>

This page:

https://github.com/elastic/stream2es/issues/65

suggests setting the system property "jdk.xml.totalEntitySizeLimit (works for me using Java 8) or just totalEntitySizeLimit if that doesn't work".

I'm afraid XML parsers and their limits are not something we have any control over.

Thank you very much, Michael - this solved the problem.

To give a concrete example: I added to the command-line call this parameter ...

-Djdk.xml.totalEntitySizeLimit=921000000