Project

Profile

Help

Actions
Send by e-mail Copy link

Support #2337

closed

Saxon 9-6.0 .net api, excessive memory usage

Added by ashona krithallal about 9 years ago. Updated over 8 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Michael Kay
Category:
.NET API
Sprint/Milestone:
-
Start date:
2015-03-24
Due date:
% Done:

0%

Estimated time:
Legacy ID:
Applies to branch:
Fix Committed on Branch:
Fixed in Maintenance Release:
Platforms:

Description

I have deployed the saxon-PE .net api in our Website. After 3 days we are now getting excessive memory usage to the point that the site is not usable. Is there a preferred Java JRE which we should be using with the .net api

we added the following code to the Processor

processor = new Processor(true);

processor.SetProperty("http://saxon.sf.net/feature/schema-validation", "4"); //SKIP

processor.SetProperty("http://saxon.sf.net/feature/compile-with-tracing", "0"); //FALSE

processor.SetProperty("http://saxon.sf.net/feature/generateByteCode", "false");

processor.SetProperty("http://saxon.sf.net/feature/xsltVersion", "1.0");//set to version 1.0

Files

Download all files
image003.jpg (33.9 KB) image003.jpg ashona krithallal, 2015-03-24 13:01
image004.jpg (26.2 KB) image004.jpg ashona krithallal, 2015-03-24 13:01
image005.jpg (43.1 KB) image005.jpg ashona krithallal, 2015-03-24 13:01
image006.jpg (20.7 KB) image006.jpg ashona krithallal, 2015-03-24 13:01
Actions
Copy link
 #1

Updated by Michael Kay about 9 years ago

Presumably you mean the memory usage slowly builds up over three days continuous running?

To help you with this we will need to understand details about your workload. If my interpretation of your question is correct, then presumably there is a build-up of objects that are not eligible for garbage collection. This could happen because you are using the API incorrectly. It would also help to know the sizes of source documents that you are processing.

Getting a snapshot of the objects present in the heap should help us to narrow the problem down. See for example https://msdn.microsoft.com/en-us/library/dn342825.aspx

Actions
Copy link
 #2

Updated by ashona krithallal about 9 years ago

Hi Michael,

I am using the API as follows:

In our ContetInfo.cs, we reference the Saxon.Api .dll

The following objects are created:

private Processor processor;

   private XsltCompiler compiler;


    //this stores xslt files in memory and redues calls from file system

    //private static Dictionary<string, XsltTransformer> _loadedXsltFiles;

   private static Dictionary<string, XsltExecuProcessor> _loadedXsltProcessor;

The Constructor for ContentInfo

The _loadedXsltProcessor is a Dictionary, key value is the name of the xslt as string and the Value, is the XsltExecuProcessor object, which is an object that contains the XsltExecutable object and Processor object as below:

I am storing the complied xslt as an XsltExecutable and the Processor used in the Dictionary so that it can be reused, using the xslt FilePath I check if there is an existing XsltExecutable and Processor and reuse them, If not the create the XsltExecutable and Processor, complie the xslt and sort in the Dictonary as below:

From the ConvertXml method

Please let me know if I have implemented the API incorrectly, and could the Dictionary be causing the memory usage problem?

I am investigating this.

Kind Regards

Ashona

From: Saxonica Developer Community []

Sent: Tuesday, March 24, 2015 12:12 PM

Subject: [Saxon - Support #2337] Saxon 9-6.0 .net api, excessive memory usage

Actions
Copy link
 #3

Updated by Michael Kay about 9 years ago

Clearly your cache of saved XsltExecutables could be causing the problem, but we have no way of knowing whether this actually is the problem as we have no metrics on the number of stylesheets you are saving in the cache, or their size. I would suggest you at least add some instrumentation on the number of entries in the cache: if it is constantly growing, then this would be a concern.

Normally in a cache of this kind I would use a single Processor instance, rather than creating a new one for each stylesheet. However, creating a new Processor is not necessarily wrong. There can be some benefits in reducing NamePool contention between concurrent activities, but there may be a cost in increased memory.

Actions
Copy link
 #4

Updated by Michael Kay about 9 years ago

  • Status changed from New to In Progress

Have you made any progress in collecting more information on this?

There's another thread on high memory usage in Saxon on .NET at https://saxonica.plan.io/issues/2339, does this yield any insights?

We can't really make any progress unless we have more information on what your application is doing. A memory dump that shows what objects are occupying the space would be very useful as a way forward.

Actions
Copy link
 #5

Updated by Michael Kay over 8 years ago

  • Status changed from In Progress to Resolved
  • Assignee set to Michael Kay
  • Priority changed from High to Normal

Our request for further information elicited no response, so I am closing the ticket.

Please register to edit this issue

Actions
Send by e-mail Copy link

Also available in: Atom PDF