Project

Profile

Help

Out Of Memory

Added by Anonymous almost 16 years ago

Legacy ID: #5185179 Legacy Poster: rahi msft (rahi_msft)

Hi I am using XSLT transformation on a 4+ GB XML file. Using trial version - saxonsa9-1-0-1n on .net platform. It give me Out Of Memory exception after the process reaches 1.6 GB on "XdmNode input = builder.Build(sXml)" line. Can any one help me with any work around? On the website it says Saxon A can process files upto 20 GB. The code is below. String sourceUri = @"Input.XML"; String xsltUri = @"Input.xsl"; FileStream sXml = File.OpenRead(sourceUri); FileStream sXsl = File.OpenRead(xsltUri); Processor processor = new Processor(); DocumentBuilder builder = processor.NewDocumentBuilder(); Uri sUri = new Uri(sourceUri); builder.BaseUri = sUri; XdmNode input = builder.Build(sXml); <-- Line of error!! XsltTransformer transformer = processor.NewXsltCompiler().Compile(sXsl).Load(); transformer.InitialContextNode = input; Serializer serializer = new Serializer(); TextWriter w = new StreamWriter(@"C:\out.txt"); serializer.SetOutputWriter(w); transformer.Run(serializer);


Replies (1)

RE: Out Of Memory - Added by Anonymous almost 16 years ago

Legacy ID: #5186819 Legacy Poster: David Lee (daldei)

My guess is this. You are refering to this bullet: "Saxon-SA has a facility to process large documents in streaming mode. This enables documents to be handled that are too large to hold in memory (it has been tested up to 20Gb)." The key here is "in streaming mode" What your doing is not streaming mode. builder.Build(sXml) reads the entire document into memory. On a 32 bit machine at most you may be able to get 2 or 3GB of RAM in the VM depending on the kernal configuration. (more realistically 1.5G max ) I've never used streaming mode before but here's teh docs for it. http://www.saxonica.com/documentation/sourcedocs/serial.html I am guessing it is these functions that are quoted as being tested with up to 20G XML files.

    (1-1/1)

    Please register to reply