Saxon » SaxonC

Here with the right formatting:

from saxonche import PySaxonProcessor
import psutil

xslt = """<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="3.0" expand-text="yes">

<xsl:template name="xsl:initial-template">
  <items>
    <xsl:for-each select="random-number-generator(current-dateTime())?permute(1 to 20)">
      <item>Item {.}</item>
    </xsl:for-each>
  </items>
</xsl:template>

</xsl:stylesheet>"""

count = 0
process = psutil.Process()
prev = process.memory_info().rss
print(f"{prev:,}")
print()

with PySaxonProcessor() as proc:

    xslt_processor = proc.new_xslt30_processor()

    xslt_executable = xslt_processor.compile_stylesheet(stylesheet_text=xslt)

    for _ in range(100):

        xslt_executable.call_template_returning_value()

        if (count := count + 1) % 10 == 0:
            m = process.memory_info().rss
            print(f"{m - prev:,}")
            prev = m

print()
print(f"{prev:,}")

Leak on Windows

26,128,384

8,466,432
782,336
782,336
786,432
778,240
778,240
782,336
782,336
782,336
778,240

41,627,648

and on Linux

32,747,520

15,384,576
811,008
811,008
811,008
811,008
811,008
811,008
811,008
811,008
540,672

55,160,832

I'm still investigating exactly what's going on, and I'm not yet sure how related all these cases are. There's a degree more complexity with memory usage here because, effectively, GraalVM means there's a JVM running inside the C / Python process, which is doing garbage collection of its own.

There is definitely a problem, but over long enough runs I am seeing some (but not enough) memory freed as a result of GraalVM's GC. What this really means is that it's harder to tease apart what's Graal wanting to avoid GC for as long as possible (because it's stop-the-world GC), and what's C-level memory leaking.

I'm going to keep these tickets open and separate, but if it turns out there's really only one common problem I will mark some of them as duplicates to keep noise to a minimum.