Bug #2339
closed
Very high memory usage when compiling xslt
0%
Description
Hello:
We noticed RAM spikes over 1 GB when compiling xslt's using the Saxon .Net api. Also, the memory doesn't seem to be able to be reclaimed by our application. This will cause our application to crash after a period of time. It seems to be related to static XPath expression exceptions as shown in the screen shot.
How can we remove this memory footprint from the application?
What in the xslt may be causing this?
Files
Updated by O'Neil Delpratt over 9 years ago
- Assignee set to O'Neil Delpratt
Hi Bob,
Thanks for reporting back to us the problem you are having. It is an interesting problem that we would like to explore further.
Firstly, the StaticError could be genuinely gathering errors in your stylesheet which are not failure errors, but just warnings. It would be good to inspect your stylesheet. Is it possible that you could send us your code, privately or on this bug issue?
Secondly, it would be good to eliminate the bytecode generation feature from this, which might be the cause of the high memory usage, please could you switch it off and run your program again to see if the issue goes away.
If you are running Saxon from the commandline just add the following option:
--generateByteCode:off
If you are running Saxon from C# code the following code. (I assume you have a processor object):
Processor processor = new Processor(true);
processor.SetProperty("http://saxon.sf.net/feature/generateByteCode", "false");
In Saxon 9.6 we have moved to the latest IKVM, which has better garbage collection for the object created using the bytecode generation feature for .NET. We have also made some improvements. It might be worth evaluating the latest release if my suggestions above fail to solve your problem.
It is possible that the bottle neck is somewhere else. Could you please send us some more details of the memory dump?
Updated by Bob Williams over 9 years ago
- File nlm23dtd.zip nlm23dtd.zip added
- File nlm23xslt.zip nlm23xslt.zip added
- File XslTransformation.cs XslTransformation.cs added
Hi:
Attached, please fin the style sheets, DTD'S and a C# class we use to compile the xslt.
We are currently already turning off the byte code.
I will try to get more info on the memory dump.
Updated by O'Neil Delpratt over 9 years ago
- Status changed from New to In Progress
Hi,
sorry for the delay in getting back to you. I am trying to create a repo with the C# code and stylesheets you sent last week, but it seems like I need some other libraries to run your code. Specifically I am not sure where to find WKHMR.
Updated by Bob Williams over 9 years ago
We have seen that during compilation we saw the following issues causing some of the compiler exceptions.
The root cause of the compilation errors is the fact that we are running an XSLT function from an attribute, then expecting the attribute node to contain attributes. The specific code is listed below and is peppered throughout our mappings in various formats.
Bad
@xlink:href/my:externalLinkRef(@xlink:href,@xlink:format,@xlink:title, @xlink:type,@xlink:label, @xlink:role)
Good
@xlink:href/my:externalLinkRef(../@xlink:href,../@xlink:format,../@xlink:title, ../@xlink:type,../@xlink:label, ../@xlink:role)
We also noticed that the some of the xslt files are over 15 MB is size which takes 300 MB of RAM to compile. So in the short term we have cleaned up the xslt's a bit for the aforementioned issue, removed the ErrorsList property assignment in the compiler object and forced a .Net garbage collection on gen 2 and 0 after compilation. After the garbage collection the memory is getting cleared. I think you said 9.6 has more optimized garbage collection for the IKVM which is something we should investigate.
Updated by O'Neil Delpratt over 9 years ago
- Status changed from In Progress to Won't fix
Hi,
Glad to know that you have sorted out your memory issues in your XSLT. Therefore I am closing this bug issue as won't fix.
Any other problems please let us know.
Please register to edit this issue
.