Project

Profile

Help

Support #2347

closed

Need help in GC.

Added by Muthu Kaliyamurthy about 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
JAXP Java API
Sprint/Milestone:
-
Start date:
2015-04-01
Due date:
2015-04-08
% Done:

0%

Estimated time:
Legacy ID:
Applies to branch:
Fix Committed on Branch:
Fixed in Maintenance Release:
Platforms:

Description

Michael,

with regards to the email I sent is as below.

to Answer your Question

a) using 1 stylesheet with a few child ones getting called from the parent.

b) using the same template for transforming the files.

c) using JAXP API

Please let me know if you may require additional information.


Reply from Michael,

We'll be happy to try and help. But first:

(a) please use the support forum at http://saxonica.plan.io That way (i) we can track it properly, and (ii) any resolution we come up with is available to other users who might have problems in the same area.

(b) we can't see what's going wrong without seeing what you are doing. Are you running one stylesheet repeatedly, or many different stylesheets? Are you re-using the same Configuration or creating a new one? Are you using the JAXP API or the s9api API? You need to give us enough information to narrow down the possibilities.

Michael Kay

Saxonica

+44 (0) 118 946 5893

On 1 Apr 2015, at 03:35, Kaliyamurthy, Muthu wrote:

Hi Michael,

            Please help me in freeing the memory after the transformation is done. My application keeps waiting for the input XML to be processed anytime, at times the number of files to be processed would be approx. 900 in count and few would be huge. Once after these huge files are done my memory or the heap does not seems to be garbage collected. I need this to be free for rest of the application to run smoothly on that machine. Please help me in getting the top ones in the list to be garbage collected.               

<image001.png>

<image002.png>

Thanks & Regards,

K.Muthu


Files

saxon GC-1.png (18.8 KB) saxon GC-1.png Muthu Kaliyamurthy, 2015-04-01 23:17
saxon GC-2.png (47.7 KB) saxon GC-2.png Muthu Kaliyamurthy, 2015-04-01 23:17
Actions #1

Updated by Michael Kay about 9 years ago

I think the most likely explanation is that you are using the same JAXP Transformer repeatedly, in which case the documents it reads are held in memory in case they are needed again. You can avoid this by creating a new Transformer for each transformation.

If this is not the case, please show us your code. With performance problems, the devil is always in the detail.

Actions #2

Updated by Muthu Kaliyamurthy about 9 years ago

Hi Michael,

Thanks for your quick response, It would be nice to show you the code but due to policy reason I cannot copy/paste any piece of code here. If there is a screen share way that i could walk you over my code that would be nice as we talk over the phone.

By the way I am creating new Transformer every time. I have an EcecutorService with fixed thread pool and I keep creating Transformers and assigning it to the EcecutorService and calling the Futures of the callable object to get to know either success or failure. I create the template only once and the whole logic is driven by Quartz scheduler which keeps polling the DB for any new files to be transformed. When the batch is done and no files to be processed then I am calling GC to get the objects out, the screenshot is sent after 12 hours of idle time. I would hope to get the heap down to some 1G or even less.

Hope I explained in a nutshell. Please let me know if you may require additional information.

Actions #3

Updated by Michael Kay about 9 years ago

  • Status changed from New to In Progress

If you don't want to make your code public, you are welcome to send it by email - use support(at)saxonica(dot)com. I'm afraid that without seeing your code, there is very little we can do to help you.

Actions #4

Updated by Muthu Kaliyamurthy about 9 years ago

Hi Michael,

We are into an another issue right now, For performance improvement we planned to pass pre-generated jaxb nodes to the transformer and if the XSLT has saxon:threads the transformation fails with Null pointer exception pointing the line where saxon:threads is used. This is not consistent for the same XSLT and each time this points to a different XSLT(child having saxon:threads). Should we handle these saxon:threads differently in case of passing pregenerated nodes..?

Transformer trans=getTemplates().newTransformer();

trans.setParameter("xmlDetailsMap", new ObjectValue(xmlDetailsMap));

Actions #5

Updated by Muthu Kaliyamurthy about 9 years ago

Using Saxon Extensibility with Saxon:threads is was causing the above mentioned issue. Need help, as I end up no where in improving the performance. If there is any suggestion please pass on some tested code to accomplish this scenario.

Actions #6

Updated by Muthu Kaliyamurthy about 9 years ago

Any updates pls.

Actions #7

Updated by Michael Kay about 9 years ago

As I have tried to explain, we can't offer much help unless we can run the failing code and observe the symptoms for ourselves.

From what you are saying, it seems that you are running Saxon in multithreaded mode, but calling external Java code which isn't thread-safe. If that's the case, then you need either to switch multi-threading off, or make the external code thread-safe.

Actions #8

Updated by Michael Kay over 8 years ago

  • Status changed from In Progress to Closed
  • Priority changed from High to Normal

Although this hasn't been resolved, I think that things have moved on and the problem probably no longer arises in its original form. so I'm closing the ticket. Do feel free to open a new support request if you need further help in this area, preferably with a specific reproducible problem that we can investigate.

Please register to edit this issue

Also available in: Atom PDF