Saxon

My recommendation is to create a new XsltTransformer (by calling XsltExecutable.load()) for each transformation, unless you actually want to retain resources (such as keys) from one transformation to the next. The only reason to reuse an XsltTransformer is to reuse its resources.

Creating an XsltTransformer is a low-cost operation, provided that you re-use the XsltExecutable.

I'm a little surprised that clearDocumentPool() doesn't work for you; to understand why, it would be necessary to analyze a heap dump. There are other resources that could be the culprit, for example stylesheet parameters.

Thanks Michael for your quick response.

I believe a new XsltTransformer has already been created for each transformation:

XsltTransformer transformer = processor.newXsltCompiler().compile(xsltSource).load();

Even if I break down the code into this:

XsltCompiler xsltCompiler = processor.newXsltCompiler(); XsltExecutable xsltExecutable = xsltCompiler.compile(xsltSource); XsltTransformer transformer = xsltExecutable.load();

The high memory consumption issue still exist whenever transformer.transform(); is called.

As for the stylesheet parameters, it is just a simple stylesheet as shown:

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="text"/> <xsl:template match="/">NAME,ALIAS <xsl:for-each select="OUTPUT/DATA"> <xsl:value-of select="GIVEN_NAME"/>,<xsl:value-of select="GIVEN_ALIAS"/> </xsl:for-each> </xsl:template> </xsl:stylesheet>

OK sorry, I read your prose which says you are calling the transformer multiple times, and didn't look at your code which shows you are only calling it once.

In fact I see that you are even creating a new Processor for each transformation, which is an expensive thing to do, but means that there is no possibility for Saxon to hold on to any resources.

There have been some bugs in the past relating to caches (e.g. of type information) held in static data but there are no known problems in current releases. You haven't said which Saxon version you are running on.

Once you've checked you are using the current release (9.5.1.5), the next stage should probably be to get a heap dump (e.g. using HAT) to show which objects are accumulating in memory, and where they are linked from.

No worries for that.

I am using SaxonHE9.5.1.4. Btw, I am quite sure that the transformer is holding the memory as the high memory consumption issue will only occur if transformer.transform(); is called.

FYI, my xml data contains 2 millions of nodes thus I suspect it is too huge for the transformer to clear...

Obviously the memory use of the transformer will be high if the source document is large. (To avoid that, you need to think about using a streaming transformation, which needs Saxon-EE).

But once the transformation is complete, the memory should be cleared by the garbage collector.

I'm confused now as to what you are saying. Is your problem

(a) that a single transformation uses a large amount of memory, or

(b) that the memory acquired during a transformation is not being garbage collected when the transformation ends?

You say you are sure that the transformer is holding the memory, but that's inconsistent with (b): if the memory is still locked down when the Transformer is released this would imply that the memory is held by something other than the Transformer.