Saxon

A NullPointerException deep in Saxon code is (almost) always a Saxon bug rather than something you've done wrong.

The code here is essentially unchanged between 9.5 and 9.6.

It's looking at a rather tricky data structure involving weak references: there's a hash table that maps documents to accumulator data, and I think we're failing in dereferencing this weak references. The code should probably be a bit more defensive and rebuild the accumulator when the reference is null. I wouldn't be able to test a fix without first having a repro for the problem, however.

I made some progress (hopefully). I haven't tested it through an entire production run, but this seems to work so far..

Prior, I was calling this transform with:

/usr/local/bin/saxon9pe.jar _saxon options_

..and that would crash sometimes.

This call to the transform doesn't crash:

java -jar -Xmx83886080 -Xms6291456 /usr/local/bin/saxon9pe.jar _saxon options_

Also put an -opt:10 in the Saxon part of the command.

I found my hint to do this here http://docs.oracle.com/javase/6/docs/technotes/tools/windows/java.html

(side note: regarding URL above, I'm not using Windows or Java 6)

Maybe the command line default memory settings weren't enough for my larger transforms.

I hope this works for everything.

On reflection.. your comment "there's a hash table that maps documents to accumulator data" made me think of memory consumption.. that's why I explored memory command line parameters.

Accumulator code may actually be better than expected.. it's the environment holding it back? I only can suppose, since I am not a classicly trained software engineer.

Thanks for the extra info. This would seem to confirm the following hypothesis: because the accumulator data is only referenced by a weak reference, if memory is short then it gets garbage collected. In this case Saxon should detect that it has disappeared, and rebuild it (in the event that it's needed again). But it's failing to detect that it's gone, and is crashing with an NPE instead.

Hi,

Having been wrestling with accumulators for a few days and banging my head against some unpredictable NPEs, I came across this conversation and thought it might be useful if I corroborated the sighting. Unfortunately I'm in a similar boat to the original poster with regard to posting code, but I'm hopeful my superiors might let me share a set of example files privately, if it'd be of use - though it'll take me a day or two to obtain permission.

In the meantime, a few details that seem to me like they might be pertinent:

• Saxon EE 9.6.0.2, with the SQL extension and JDBC on the classpath
• input files are ISO-8859-1 and of varying length and complexity
• the accumulator in question is something like this: <xsl:accumulator name="words.count" initial-value="0" as="xs:integer" streamable="no" applies-to="/content[child::piece]"> <xsl:accumulator-rule match="text()[normalize-space()][ancestor::[local-name() = $extract.targets]]" new-value="$value" phase="start"/> <xsl:accumulator-rule match="text()[normalize-space()][ancestor::[local-name() = $extract.targets]]" phase="end"> <xsl:value-of select="if ($value ge 250) then $value else ($value + (count(tokenize(.,'\W+'))))"/> </xsl:accumulator-rule> </xsl:accumulator>
• values of accumulator-before() and -after() are checked in three template match patterns for text nodes
• the NPE seems to arrive sooner when there are more calls to the accumulator functions in the stylesheet (e.g. in xsl:message output etc)
• it goes away after giving java unseemly amounts (about 1.5gb) of initial heap space. Increasing max heap space seems not to make a difference.
• and here's a snippet from the stack trace: Caused by: java.lang.NullPointerException at com.saxonica.xslt3.instruct.AccumulatorManagerPE.getAccumulatorData(AccumulatorManagerPE.java:70) at com.saxonica.functions.xslt3.AccumulatorFn.getAccumulatorValue(AccumulatorFn.java:117) at com.saxonica.functions.xslt3.AccumulatorFn.iterate(AccumulatorFn.java:76) at net.sf.saxon.expr.SingletonAtomizer.evaluateItem(SingletonAtomizer.java:194) at net.sf.saxon.expr.SingletonAtomizer.evaluateItem(SingletonAtomizer.java:32) at net.sf.saxon.expr.ValueComparison.effectiveBooleanValue(ValueComparison.java:688) at net.sf.saxon.pattern.PatternWithPredicate.matchesPredicate(PatternWithPredicate.java:119) at net.sf.saxon.pattern.PatternWithPredicate.matches(PatternWithPredicate.java:111) at net.sf.saxon.pattern.PatternWithPredicate.matches(PatternWithPredicate.java:108) at net.sf.saxon.trans.Rule.matches(Rule.java:180) at net.sf.saxon.trans.Mode.searchRuleChain(Mode.java:639) at net.sf.saxon.trans.Mode.getRule(Mode.java:572) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:1124) at com.saxonica.xslt3.instruct.ShallowCopyRuleSet.process(ShallowCopyRuleSet.java:85) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:1128) at com.saxonica.xslt3.instruct.ShallowCopyRuleSet.process(ShallowCopyRuleSet.java:85) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:1128) at com.saxonica.xslt3.instruct.ShallowCopyRuleSet.process(ShallowCopyRuleSet.java:85) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:1128) at com.saxonica.xslt3.instruct.ShallowCopyRuleSet.process(ShallowCopyRuleSet.java:85) at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:1128)

Hope this is of some use!

I haven't been able to reproduce this: it's intrinsically difficult because it means finding a test case where the accumulator uses too much memory to succeed without garbage collection, but not so much that it still fails after garbage collection.

However, there's a path in the code that's clearly wrong, so I've fixed it without testing: see https://saxonica.plan.io/issues/2229

Just wanted to let it be known that 9.6.0.3 fixed the problem for me - thanks!