Project

Profile

Help

Bug #4555

closed

NullPointerException on Saxon 10.1

Added by Gunther Rademacher almost 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Internals
Sprint/Milestone:
-
Start date:
2020-05-15
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
10, 9.9
Fix Committed on Branch:
10, 9.9
Fixed in Maintenance Release:
Platforms:

Description

After upgrading to Saxon 10.1, I am facing a NullPointerException with this stacktrace:

java.lang.NullPointerException
        at net.sf.saxon.expr.parser.LoopLifter.markDependencies(LoopLifter.java:168)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:119)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:101)
        at net.sf.saxon.expr.parser.LoopLifter.process(LoopLifter.java:51)
        at net.sf.saxon.query.XQueryFunction.optimize(XQueryFunction.java:452)
        at net.sf.saxon.query.XQueryFunctionLibrary.optimizeGlobalFunctions(XQueryFunctionLibrary.java:320)
        at net.sf.saxon.query.QueryModule.optimizeGlobalFunctions(QueryModule.java:1199)
        at net.sf.saxon.expr.instruct.Executable.fixupQueryModules(Executable.java:458)
        at net.sf.saxon.query.XQueryParser.makeXQueryExpression(XQueryParser.java:177)
        at net.sf.saxon.query.StaticQueryContext.compileQuery(StaticQueryContext.java:568)
        at net.sf.saxon.s9api.XQueryCompiler.compile(XQueryCompiler.java:562)

Unfortunately I do not yet have a small reproduction for this. However it can be reproduced as follows (commands for Windows):

git clone https://github.com/GuntherRademacher/rr.git
cd rr
sed -i s/Saxon-HE:10.0/Saxon-HE:10.1/ build.gradle
call gradlew
java -jar build/libs/rr.war src/main/java/de/bottlecaps/railroad/core/Parser.ebnf
Actions #1

Updated by Gunther Rademacher almost 4 years ago

Some further analysis showed that this might occur while compiling the code in Q{de/bottlecaps/railroad/xq/ast-to-svg.xq}render-vertical-pass2.

I was able to build a minimal repro based on that. It does not make any sense, but it preserves the NPE. It can be run from command line.

declare variable $center-choice := false();

declare function local:render-vertical-pass2($max-width, $todo)
{
	if ($todo) then
		let $extension-length := $max-width
		let $ext1 := if ($center-choice) then $extension-length else 42
		let $ext2 := $extension-length - $ext1
		return <line x2="{$ext1}"/>[$ext2 != 0]
	else
	    ()
};

()

I guess that the problem is related to the condition on $center-choice, which is declared as false() statically.

Actions #2

Updated by Michael Kay almost 4 years ago

Thanks. I've reproduced this - it fails in HE only (or in EE with no license file).

Actions #3

Updated by Michael Kay almost 4 years ago

The NPE arises when the downwards (depth-first) search of the expression tree looking for candidates for loop-lifting finds an expression whose parent pointer in the tree is wrong - specifically, is an expression that hasn't been encountered during the depth-first search. The expression in question is a variable reference to $max-width, and this appears to arise because the redundant variable $extension-length has been eliminated.

The compiled expression is

if ($todo) then
   (let $ext1 := if (false()) 
                      then ($max-width) 
                      else if (true()) then (42) 
   return (let $ext2 := (convertUntyped(atomizeSingleton($max-width))) - (convertUntyped(atomizeSingleton($ext1))) 
               return ((<line {FixedAttribute(string-join(convert(data($ext1)), " "))}/>)[$ext2 != 0])))

and the "bad" variable reference is the second one, inside the atomizeSingleton() call, which started life as a reference to $extension-length.

Actions #4

Updated by Michael Kay almost 4 years ago

Both the variable references for $max-width are represented by the same LocalVariableReference object, which means that one of them has the wrong parent expression. This arises from a failure to copy the LocalVariableReference.

The offending code seems to be in FLWORExpression.optimize() line 520, where ExpressionTool.replaceVariableReferences() is called with mustCopy=false in the case where oneRef=true.

I'm now getting a failure "A license is needed to use custom serialization {http://saxon.sf.net/}indent-spaces". This is because I'm running Saxon-EE without a license, rather than running HE. EnterpriseConfiguration.makeXQueryExpression() should create an instance of XQueryExpression rather than XQueryExpressionEE under these circumstances. But the actual failure is that the Query command line, when preparing the "explain" output destination (at line 365) is testing whether edition="HE" rather than testing whether it is licensed.

Actions #5

Updated by Michael Kay almost 4 years ago

  • Category set to Internals
  • Status changed from New to Resolved
  • Assignee set to Michael Kay
  • Priority changed from Low to Normal
  • Applies to branch 9.9 added
  • Fix Committed on Branch 10, 9.9 added

In summary, the basic failure is a corruption of the expression tree when optimising FLWOR expressions, specifically those with a redundant variable declared in a let clause.

It's not failing in 9.9 because the variable oneRef is set to false; in effect 10.x has become smarter in analysing the variable references and this has triggered an optimisation that was not attempted in 9.9, but the optimisation turns out to be wrong. I will fix 9.9 so the failure to copy the variable reference is corrected, in cases there are other test cases that trigger this path.

Also, on the 10.x branch only, I am fixing a couple of places where Saxon-EE with no license file is failing when it should fall back to HE behaviour.

Actions #6

Updated by O'Neil Delpratt over 3 years ago

  • % Done changed from 0 to 100
  • Fixed in Maintenance Release 10.2 added

Bug fix applied in the Saxon 10.2 maintenance release.

Actions #7

Updated by Gunther Rademacher over 3 years ago

Thanks for the fix - this works fine on Saxon-HE 10.2.

Actions #8

Updated by O'Neil Delpratt over 3 years ago

  • Status changed from Resolved to Closed
  • Fixed in Maintenance Release 9.9.1.8 added

Bug fix applied on the Saxon 9.9.1.8 maintenance release.

Please register to edit this issue

Also available in: Atom PDF