Support #3488
closed
Java Heap Space in transformer.transform (Over time, not a single request)
0%
Description
We are currently having a problem of Java Heap Space and the log shows the problem is when the transofrm method is called.
Resuming our implemetation is something like this:
Transformer transformer = cachedXSLT.newTransformer();
String strXMLInput = this.toString();
StringReader xmlReader = new StringReader(strXMLInput);
transformer.transform(new StreamSource(xmlReader), result);
Where cachedXSLT is a Template and result a javax.xml.transform.Result
The log shows this:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:4138)
at java.util.Arrays.copyOf(Arrays.java:3870)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:119)
at java.io.PrintStream.write(PrintStream.java:454)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:355)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:425)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:138)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:187)
at java.io.PrintStream.write(PrintStream.java:501)
at java.io.PrintStream.print(PrintStream.java:643)
at java.io.PrintStream.println(PrintStream.java:780)
at net.sf.saxon.StandardErrorListener.error(StandardErrorListener.java:210)
at net.sf.saxon.Controller.recoverableError(Controller.java:865)
at net.sf.saxon.trans.Mode.reportAmbiguity(Mode.java:593)
at net.sf.saxon.trans.Mode.getRule(Mode.java:257)
at net.sf.saxon.trans.RuleManager.getTemplateRule(RuleManager.java:160)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:329)
at net.sf.saxon.instruct.ApplyTemplates$ApplyTemplatesPackage.processLeavingTail(ApplyTemplates.java:527)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:317)
at net.sf.saxon.instruct.ApplyTemplates$ApplyTemplatesPackage.processLeavingTail(ApplyTemplates.java:527)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:317)
at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210)
at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:93)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296)
at net.sf.saxon.instruct.Choose.processLeavingTail(Choose.java:686)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:93)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:93)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556)
at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345)
at net.sf.saxon.instruct.ApplyTemplates.defaultAction(ApplyTemplates.java:378)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:333)
at net.sf.saxon.Controller.transformDocument(Controller.java:1807)
at net.sf.saxon.Controller.transform(Controller.java:1621)
Analysing the heap its shown that the heap is occupied mostly by ByteArrayOutputStream
My question is: Does transform keep any cache or anything that the garbage colector is not able to grab?
Updated by Michael Kay over 6 years ago
Saxon doesn't use any ByteArrayOutputStream instances internally; I think the only instances are the ones you are (presumably) supplying to hold the transformation result. How big is the result? Why are you putting the result in a ByteArrayOutputStream?
Actually looking at the stack trace we have:
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:187)
at java.io.PrintStream.write(PrintStream.java:501)
at java.io.PrintStream.print(PrintStream.java:643)
at java.io.PrintStream.println(PrintStream.java:780)
at net.sf.saxon.StandardErrorListener.error(StandardErrorListener.java:210)
at net.sf.saxon.Controller.recoverableError(Controller.java:865)
at net.sf.saxon.trans.Mode.reportAmbiguity(Mode.java:593)
so the ByteArrayOutputStream here appears to be the destination for warning messages output by the StandardErrorListener. Have you redirected this output perhaps?
You haven't said which Saxon version you are using: it looks like an old one, judging by the line numbers.
It might be a good idea to get rid of the ambiguities in your template rules to avoid the warning messages: put a priority attribute on template rules where several rules can match the same node.
Updated by Caio Franco Barreiro over 6 years ago
Michael,
Thanks answering soo fast.
What you said actually makes sense.
Before the code I posted abode there is this:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream ps = new PrintStream(baos);
((StandardErrorListener)transformerFactory.getErrorListener()).setErrorOutput(ps);
I can't say the file size given it's dinamically generated and it changes at each request.
The version of this application is 9.1.0.8
So you are saying that the out of memory might be occurring when the transform tries to write an error message?
But... How this would explain every following request showing the Java Heap Space error also?
Updated by Michael Kay over 6 years ago
Well, if the ErrorListener is set on the TransformerFactory and the ByteArrayOutputStream is still connected to it then it can't be garbage collected...
Setting an ErrorListener on a TransformerFactory doesn't make much sense. If you run two transformations in parallel you don't want their messages to go to the same place...
Updated by Caio Franco Barreiro over 6 years ago
This aplication were not developed by me, so I can tell you for sure why this was develped this way.
What would be the best practice seting that?
So probably removing this ErrorListener would resolve that problem, right?
Thank you so much in advance.
Updated by Caio Franco Barreiro over 6 years ago
Another issue could be that the transformFactory is set in a static way.
this.transformerFactory = new TransformerFactoryImpl();
Does that make sense? Is that instantiation too heavy?
Updated by Michael Kay over 6 years ago
I strongly suspect someone was irritated by the warning messages and redirected them to a ByteArrayOutputStream to get rid of them: which of course has the disadvantage of suppressing all other error messages as well. I would (a) get rid of the custom ErrorListener, and (b) get rid of the warning messages by setting explicit priorities on template rules. (You can also suppress them with a configuration setting, but you'll have to look up what that is in 9.1).
I don't like having things like TransformerFactories in static variables myself, but if you want to fix it you'll have to take responsibility for such refactoring yourself. Whether it matters or not depends on the application architecture.
Updated by Caio Franco Barreiro over 6 years ago
We'll trying change to non-static and removing that custom listeners.
Thanks so much for the help!
Please register to edit this issue
.