Saxon

My guess would be that it is the same issue, namely memory occupied by generated bytecode. How large is the stylesheet?

If it is the size of the bytecode that's causing the problem, then you may be able to solve the problem by increasing PermGen space, (note that -Xmx makes no difference to this), but the simplest fix is probably to disable bytecode generation for the stylesheet in question. It's unlikely the impact on execution time will be more than 25% or so.

I assume you are compiling the stylesheet once as an XsltExecutable (or JAXP Templates), and then using it repeatedly by creating a new Transformer for each transformation?

Michael Kay wrote:

My guess would be that it is the same issue, namely memory occupied by generated bytecode. How large is the stylesheet?

If it is the size of the bytecode that's causing the problem, then you may be able to solve the problem by increasing PermGen space, (note that -Xmx makes no difference to this), but the simplest fix is probably to disable bytecode generation for the stylesheet in question. It's unlikely the impact on execution time will be more than 25% or so.

I assume you are compiling the stylesheet once as an XsltExecutable (or JAXP Templates), and then using it repeatedly by creating a new Transformer for each transformation?

Thanks for the prompt reply. Yes, we do have to test the same with huge xmls. But right now the use case is with a permGen size configured as 128m, we are converting xmls to html almost 1200 times and it throws 'java.lang.OutOfMemoryError: PermGen space' and stops. If we increase it to 256m/512m it is able to do something like for eg 3K/6K transformations and stops. And the permGen is not getting cleaned up. In the thread mentioned (https://saxonica.plan.io/issues/1833) , I could see the workaround as assigning the transformer to null. Will this work in our case? Or is there any permanent solution added to this issue on the saxon library itself?

Thanks,

-Arjun.

Michael Kay wrote:

My guess would be that it is the same issue, namely memory occupied by generated bytecode. How large is the stylesheet?

If it is the size of the bytecode that's causing the problem, then you may be able to solve the problem by increasing PermGen space, (note that -Xmx makes no difference to this), but the simplest fix is probably to disable bytecode generation for the stylesheet in question. It's unlikely the impact on execution time will be more than 25% or so.

I assume you are compiling the stylesheet once as an XsltExecutable (or JAXP Templates), and then using it repeatedly by creating a new Transformer for each transformation?

Thanks for the prompt reply. Yes, we do have to test the same with huge xmls. But right now the use case is with a permGen size configured as 128m, we are converting xmls to html almost 1200 times and it throws 'java.lang.OutOfMemoryError: PermGen space' and stops. If we increase it to 256m/512m it is able to do something like for eg 3K/6K transformations and stops. And the permGen is not getting cleaned up. In the thread mentioned (https://saxonica.plan.io/issues/1833) , I could see the workaround as assigning the transformer to null. Will this work in our case? Or is there any permanent solution added to this issue on the saxon library itself?

Also we are not using the XsltExecutables. Our sample code is something like,

import com.saxonica.config.EnterpriseTransformerFactory;

import javax.xml.transform.Transformer;

import javax.xml.transform.TransformerFactory;

import javax.xml.transform.stream.StreamResult;

import javax.xml.transform.stream.StreamSource;

// Initialized once

TransformerFactory factory = new EnterpriseTransformerFactory();

// Called every time as one per transformation.

Transformer transformer = factory.newTransformer(new StreamSource(xsltFile));

transformer.transform(src, new StreamResult(fileOutputStream));

Let me know in case you need more details regarding this.

Thanks,

-Arjun.

Your code is recompiling the stylesheet on every transformation. (I assume it is the same stylesheet each time?). This is bad news from a performance perspective, because compiling the stylesheet is an expensive operation. But it's especially bad when bytecode is generated, because the bytecode is loaded into PermGen and is not easily garbage collected (also, bytecode generation takes time, so it is only really useful if the code is executed repeatedly.)

So if you possibly can, change your application to only compile the stylesheet once (by creating a Templates object). If that's not achievable, then switch off byte-code generation.

Thanks a lot Michael for the input. Let us make the changes and try it. And will revert in case we face any issue.

Thanks,

-Arjun

"change your application to only compile the stylesheet once (by creating a Templates object). If that's not achievable, then switch off byte-code generation."

I changed the application to compile the style sheet once per run. Which means the 1000+ transformations will do the compilation once. Would you still recommend to disable byte code generation? Switching off byte code will add an increased 25% time. But it will help us not increasing the permGen memory utilization? And as it is a one time compilation, we can afford to have 25% increase in the transformer creation time too.

Please let us know what you recommend.

Thank you very much,

-Arjun