Saxon

Legacy ID: #4938142 Legacy Poster: W. Eliot Kimber (drmacro)

I'm trying to process a very large collection of DITA topics (10's of 1000s) ultimately organized by a single root DITA map. I'm using the normal DITA Open Toolkit dita2html transform which operates on a single input topic and generates a single result topic. Just using the normal Open Toolkit Ant scripts to do this the process would blow up with any amount of memory I could allocate (max 2Gig on a 32-bit machine). I am using Saxon 9 (even though the current Open Toolkit style sheets are all XSLT 1). I finally realized that the XSLT itself is simple and shouldn't cause any memory problems. Digging into the details of the processing, I realized that the Toolkit first builds a list of topics to be processed (by processing the input DITA maps) and then uses that list in an Ant <xslt> task like so: <xslt processor="trax" basedir="${dita.temp.dir}" destdir="${output.dir}" includes="${fullditatopiclist}" style="${args.xsl}"> where "${fullditatopiclist} is, in my case, a very long list indeed. So obviously the memory problem is actually a side effect of using Ant in this way. Hmph. My question is: given that I have my own code that can get the list of topics and do whatever I want with those from my own Java code, which can include running a different form of Ant task or calling the transformer directly or running a command-line app, what is the best way process this large set of files so as to minimize the memory required and also maximize performance. Or maybe the better question is: why does Ant have a memory problem here? My naive guess would be that it's creating a new Transformer instance for each file. Does the Saxon-specific Ant task avoid that (I would assume it does but I haven't been able to try it yet). Or should I be managing the use of the Transformer in my own Java code (currently I'm just using Ant from my Java code to invoke the out-of-the-box Toolkit Ant scripts)? Thanks, Eliot