Project

Profile

Help

Forums » Help »

XML Transformations running out of heap space

Added by Anonymous over 16 years ago

Legacy ID: #5071597 Legacy Poster: zxlllk (zxlllk)

Hello All, I am using saxon to perform thousands of XML transformations back to back to back. My program dynamically generates XML messages that range in size from 1 kB to around 750 kB. I run my program, and after a while I run out of heap space and get the OutOfMemoryError. The amount of messages generated before I run out of memory is usually around the same number. I have tried the -Xms1024m -Xmx1024m solution to my problem. I still run out of memory after about 1500 messages have been generated. The max number I have been able to generate is 2500 when i used -Xms1500 -Xmx1500. I think I have narrowed down my problem to the following lines: Source xsltSource = new StreamSource(xsltReader); Source xmlSource = new StreamSource(xmlReader); TransformerFactoryImpl factory = new TransformerFactoryImpl(); StringWriter writer = new StringWriter(); BufferedWriter bufferedWriter = new BufferedWriter(writer); Transformer transformer = factory.newTransformer(xsltSource); // This is the line that runs out of heap space transformer.transform(xmlSource, new StreamResult(bufferedWriter)); The above code is in a function, and a call to that function is inside of a loop. I can never get the full stack trace that leads back to my functions either. I can just see all saxon functions in it. Thanks for any help I can get!


Replies (1)

RE: XML Transformations running out of heap s - Added by Anonymous over 16 years ago

Legacy ID: #5071789 Legacy Poster: Michael Kay (mhkay)

There's not enough information here to see where your memory leak is. I'm not quite sure what you mean by "back to back to back". Presumably the code you have shown is doing one transformation, and you are executing it thousands of times. Because you are creating a new TransformerFactory each time, there seems to be no possibility that Saxon is holding on to resources, so I would suspect that the problem is at the application level. Stack traces are no use in diagnosing heap problems, they only tell you what the band was playing when the ship sank. Equally, it's not really much use knowing that you ran out of memory while running the transform() method, we could have guessed that. I would recommend using the Heap Analysis Tool, HAT, to see which objects are accumulating in memory (but only after a careful desk-checking of your application source code to look for possible causes). It's not clear to me whether all your transformations are using the same stylesheet. If they are, it's very inefficient to recompile the stylesheet for each one. But that's not the cause of your problem. Michael Kay Saxonica

    (1-1/1)

    Please register to reply