Saxon

Legacy ID: #3172313 Legacy Poster: Jon Vaughan (jon_vaughan)

Hi, We're running 8.1 with Xms/Xmx 1.2Gb and getting an Outofmemory running a complex xquery against a 20mb (on disk) doc. Stacktrace below. It amounts to the app requesting 85Mb of heap, and I think the thread as it goes fragments the heap so much that there is no chunk big enough left. I've tried profiling, have removed other memory hogging parts of app, also tried lazy tree pull method in 8.4, also requesting heap compaction on gc, no improvements. The obvious answers are refactor doc, refactor xquery, both of which are quite painful, so as a last ditch effort I thought i'd post - if anyone has any suggestions then they would be very welcome. Also, I've noticed that DOM representations of xml vary from 2x to up to 7x string memory required, any thoughts on what makes a low memory dom representation also helpful. Many thanks Jon >>> java.lang.OutOfMemoryError at java.io.StringWriter.write(StringWriter.java(Compiled Code))at net.sf.saxon.event.XMLEmitter.attribute(XMLEmitter.java(Compiled Code))at net.sf.saxon.event.XMLIndenter.attribute(XMLIndenter.java(Compiled Code))at net.sf.saxon.event.NamespaceReducer.attribute(NamespaceReducer.java(Compiled Code))at net.sf.saxon.event.ComplexContentOutputter.startContent(ComplexContentOutputter.java(Compiled Code))at net.sf.saxon.tinytree.TinyElementImpl.copy(TinyElementImpl.java(Compiled Code))at net.sf.saxon.event.ComplexContentOutputter.append(ComplexContentOutputter.java(Compiled Code))at net.sf.saxon.instruct.DocumentInstr.processLeavingTail(DocumentInstr.java:80) at et.sf.saxon.instruct.Instruction.process(Instruction.java(Compiled Code)) at net.sf.saxon.query.XQueryExpression.run(XQueryExpression.java:277) >> How we do it: private String executeQuery(XQueryExpression anXQueryExpression, Map someDocumentParameters, String aContext) throws XPathException { StringWriter myWriter = new StringWriter(); DynamicQueryContext myDynamicQueryContext = new DynamicQueryContext(theConfiguration); if (theURIResolver != null) { myDynamicQueryContext.setURIResolver(theURIResolver); } if (aContext != null) { StaticQueryContext myStaticQueryContext = new StaticQueryContext(theConfiguration); ByteArrayInputStream myInputStream = new ByteArrayInputStream(aContext.getBytes()); Source mySource = new StreamSource(myInputStream); Item myItem = myStaticQueryContext.buildDocument(mySource); myDynamicQueryContext.setContextItem(myItem); } if (someDocumentParameters != null) { for (Iterator i = someDocumentParameters.keySet().iterator(); i.hasNext();) { String myParameter = (String) i.next(); myDynamicQueryContext.setParameter(myParameter, someDocumentParameters.get(myParameter)); } } anXQueryExpression.run(myDynamicQueryContext, new StreamResult(myWriter), theProperties); return myWriter.toString(); }