Saxon

Legacy ID: #4990631 Legacy Poster: Michael Kay (mhkay)

It would be useful to give a complete query that demonstrates the problem; please also explain how you are running it (including how much memory you allocate), how you are measuring the memory usage, and how many documents, of what size, it handles before it fails.

Legacy ID: #4990666 Legacy Poster: ShankaraKrishnan Ramanathan (mahishanky)

HI Michael, Excellent to see a quick response.Here you go. <RootElement> let $fcDocName := local:getDocName("$xmlfileName", "$requestId", ",$machine") return if (doc-available($fcDocName)) then for $i in saxon:discard-document(doc($fcDocName))//rootelement/sibling1 return <saomeelement>{$i/siblingchild1</someelement> else() </RootElement> This code will get executed 800+ times wich each time the xmlFileFormat will be <<somename>>.<<requestid>>.<<machinename>> e.g test.100.1 to test.100.800.So every time the input xml document is loaded in java heap (with Size 1 GB).I wanted to dicard the document after processing.please guide me..I checked using Jconsole and seems the program at the end occupy 700 MB space out of 1 GB heap.Note: I need to proceed with other reports after I complete 873 ietrations.The document size is 132KB per document.Hope this helps

Legacy ID: #4990704 Legacy Poster: Michael Kay (mhkay)

Sorry, you're not being clear enough. Firstly, this clearly isn't your actual code - it contains typos (mismatched start/end tag, missing end brace, mismatched quotes), which means you've changed it for presentation, which means you may have unintentionally left out the critical piece of the jigsaw (if you made typos in transcribing the code, then who knows what other errors you made?). Secondly it's only loading a single document; presumably there is some outer code that causes this code to be executed repeatedly. I can't see why this code isn't working but that probably means that the problem is in the part of the application that you haven't shown me. Finally, you mention Saxon 8 in the title. Saxon 8 ranges from 8.0 issued in 2004 to 8.9 issued in 2007. The current release is 9.0.0.5. Could you be more precise about what you are using?

Legacy ID: #4990756 Legacy Poster: ShankaraKrishnan Ramanathan (mahishanky)

Hi Michael First of all Sorry that I did some typos in previous message.But never mind , typos are only in this message :) I am using Saxon 8.8 (through Apache XmlBeans 2.3.0) We are not suppose to show you the exact code since it is confidential.But Yes some typos are there.To be more clear on the functionality the xml documents are per host or you can say permachine.This means that for 800+ machine I have 800+ documents.In Java program I am trying to execute this Xquery per host meaning from 1 - 800 (800+ times).So during this process Java HeapSize is piling up to 800 MB out of 1 GB only for this Xquery execution (800+ iterations) Note: We need repeated execution but with different Xmlfiles :) Please go through below information and let me know if still does not help you :( so that I will try to articulate with Sample program Again Note: I am running only this Xquery transformation alone, not other part of the application. Will doc-available function just check for existing or it will pile up something in memory ?(I need to browse the source though tomorrow) The Java Code which I am using will follow the below Template //don't bother this below code is just for presentation to you :) For ( int i = 0; i < machineNames.length; i++) { HashMap XqueryParams ; XqueryTransform(XqlFile, XqueryParams); } The XqueryTransForm function looks like below //See here I have used Apache XmlObject public static InputStream xqueryTransform(String xqlFilePath, Map xqueryParameters) { XmlObject xmlObj = XmlObject.Factory.newInstance(getXmlOptions()); String xqueryExpr = applyParams(uriToString(filePath), xqueryParameters); //logger.log(Level.INFO, "xqueryExpr :" + xqueryExpr); XmlObject xmlObject[] = xmlObj.execQuery(xqueryExpr, getXmlOptions()); //logger.log(Level.INFO, "Transformed Text is " + xmlObject[0].xmlText()); InputStream inputStream = xmlObject[0].newInputStream(); return inputStream; } Xquery XmlFileName, RequestId, machine -- all are passed inside the HashMap XqueryParams <RootElement> let $fcDocName := local:getDocName("$xmlfileName", "$requestId", ",$machine") return if (doc-available($fcDocName)) then for $i in saxon:discard-document(doc($fcDocName))//rootelement/sibling1 return <someelement>{$i/siblingchild1}</someelement> else() </RootElement>

Legacy ID: #4990871 Legacy Poster: ShankaraKrishnan Ramanathan (mahishanky)

Hi Michael Please guide me to resolve this issue.Let me know whether my previous posting helped :( .I need to resolve this soon.So if you want any more information I can give

Legacy ID: #4991266 Legacy Poster: Michael Kay (mhkay)

As you know, Saxon-B is an open source product and the terms and conditions do not entitle you to free support. Despite this I'm normally very ready to help users. However, in your case there are three factors that count against this: (a) you are using an old release of the software (b) you are not prepared to reveal the code of your application (c) the problem occurs in an environment (XmlBeans) that I do not have installed and know very little about. I strongly suspect that the memory "leak" is within your own application. You haven't shown me enough of the application for me to prove this, but there are hints there. I'd suggest you try to analyze the heap usage using the hat tool. Alternatively, using the debugger, find the Saxon Controller object and examine its DocumentPool. If you would like me to help you solve this for you, please write to me privately and we can discuss (a) my rates for consultancy, and (b) signing a confidentiality agreement. Michael Kay

Legacy ID: #4992548 Legacy Poster: ShankaraKrishnan Ramanathan (mahishanky)

Hi Micahel Yes I agree and Appreciate your help so far. I have to discuss with my management in relation to cost for your consultancy.It is purely their decision :) Also I did one more tuning here.I did a small tweak in XQL meaning I have written user defined function to check for doc availability and used the same in XQL instead of doc-available in addition to saxon:discard-document.I have seen good performance improvement .I continuing with same approach now :) Thanks Shankar