Bug #2289
closed
NPE in ContextItemTypeExpression
Fix Committed on Branch:
9.6
Fixed in Maintenance Release:
Description
Bug issue reported by Hans-Juergen Rennau:
XQuery files were sent privately.
There seems to be a bug producing a NPE (version Saxon-HE 9.6.0.3J).
Problem: Trying to launch the query, I get a NPE (see below). You might reproduce the problem by unzipping the directory I send you and launching 'writeOtdsReport.xq':
xqueryx writeOtdsReport.xq doc=dummy.xml format=txt numOffers=1
The problem is triggered by the second parameter of the library function 'writeOtdsReport' called by the main module: after removing the (not used) second parameter and adapting the call from the main module, no NPE occurs.
The function 'writeOtdsReport' you find in module writeOtdsReport-logical.mod.xq
ava.lang.NullPointerException
at net.sf.saxon.expr.ContextItemExpression.typeCheck(ContextItemExpression.java:85)
at net.sf.saxon.expr.parser.ExpressionVisitor.typeCheck(ExpressionVisitor.java:188)
at net.sf.saxon.expr.FunctionCall.typeCheck(FunctionCall.java:125)
at net.sf.saxon.expr.parser.ExpressionVisitor.typeCheck(ExpressionVisitor.java:188)
at net.sf.saxon.expr.LetExpression.typeCheck(LetExpression.java:80)
at net.sf.saxon.expr.parser.ExpressionVisitor.typeCheck(ExpressionVisitor.java:188)
at net.sf.saxon.expr.SlashExpression.promoteFocusIndependentSubexpressions(SlashExpression.java:489)
at net.sf.saxon.expr.SlashExpression.optimize(SlashExpression.java:346)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.expr.sort.DocumentSorter.optimize(DocumentSorter.java:75)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.expr.CardinalityChecker.optimize(CardinalityChecker.java:123)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.expr.FunctionCall.optimize(FunctionCall.java:168)
at net.sf.saxon.expr.UserFunctionCall.optimize(UserFunctionCall.java:338)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.expr.SlashExpression.optimize(SlashExpression.java:300)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.expr.sort.DocumentSorter.optimize(DocumentSorter.java:75)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.expr.flwor.FLWORExpression$7.processExpression(FLWORExpression.java:488)
at net.sf.saxon.expr.flwor.LetClause.processSubExpressions(LetClause.java:106)
at net.sf.saxon.expr.flwor.FLWORExpression.optimize(FLWORExpression.java:486)
at net.sf.saxon.expr.parser.ExpressionVisitor.optimize(ExpressionVisitor.java:244)
at net.sf.saxon.query.XQueryFunction.optimize(XQueryFunction.java:471)
at net.sf.saxon.query.XQueryFunctionLibrary.optimizeGlobalFunctions(XQueryFunctionLibrary.java:289)
at net.sf.saxon.query.QueryModule.optimizeGlobalFunctions(QueryModule.java:1208)
at net.sf.saxon.expr.instruct.Executable.fixupQueryModules(Executable.java:597)
at net.sf.saxon.query.XQueryParser.makeXQueryExpression(XQueryParser.java:159)
at net.sf.saxon.query.StaticQueryContext.compileQuery(StaticQueryContext.java:570)
at net.sf.saxon.query.StaticQueryContext.compileQuery(StaticQueryContext.java:633)
at net.sf.saxon.s9api.XQueryCompiler.compile(XQueryCompiler.java:592)
at net.sf.saxon.Query.compileQuery(Query.java:787)
at net.sf.saxon.Query.doQuery(Query.java:340)
at net.sf.saxon.Query.main(Query.java:111)
Fatal error during query: java.lang.NullPointerException: (no message)
I managed to reproduce the problem on my virtual machines which have installed 32-bit and 64-bit windows 7, respectively. Both are running with Java 8.
I also tried a 64-bit windows machine which has install Java 7.25 which runs the query without problem. Mike had no issue on his Apple Mac running with Java 7 and 8.
More details added by Mike in an email to the cause of the problem:
Firstly, it's happening during the process of binding function calls to XQuery function declarations. This process iterates over a hash table of functions, so the order in which functions are processed is unpredictable, and a simple change in computed hashcode for some class in Java 8 could legitimately cause a different order to be chosen.
Secondly, what's actually going wrong is that (a) the dependencies of the function root() are being computed before the function is expanded to root(.), and (b) the dependencies of the expression (specifically, the fact that the expression depends on the context item) aren't being reset when "." is added as the argument. The fact that the dependencies are computed before the expansion is a consequence of the (unpredictable) order of events described in the previous paragraph. The main bug is (b) - the failure to reset properties when the expression changes. It's not worth analysing the chain of events that eventually leads to the NulllPointerException: once the expression tree holds incorrect data, anything can happen. But (a) is also worrying: if the dependencies of root() need to be known before adding "." as an argument, then the code that is asking for the dependencies is already getting the wrong answer.
Management of cached derived data on the expression tree has always been a somewhat thorny issue; bugs in this area can take a long time to be discovered. I've already done a fair bit of re-engineering of the expression tree for 9.7 to try and eliminate this source of unreliability. But we will now look at how to fix this specific issue on the 9.6 branch.
For the 9.7 branch, I'm making the following changes.
To the data defining the properties of system functions (StandardFunction.java), I'm adding properties where appropriate to say (a) the function takes the context item as an implicit argument, and/or (b) the function takes the context document as an implicit document. Then, if these properties are set, I'm extending the argument list in SystemFunctionCall.makeSystemFunction() and in SystemFunctionLibrary.bind(). The calls from the simplify() method of the relevant functions to useContextItemAsDefault() and/or addContextDocumentArgument() are then disabled. The effect of this change is to add the context-item/doc argument much earlier in the process, well before anyone can examine the function's dependencies.
This appears to be working OK though I haven't been able to do a full test because my working copy of the 9.7 code base has other problems. But I'll now move on to apply the same change to the 9.6 base; I think other changes may be needed there as well.
- Status changed from New to In Progress
OK, I've applied the same change to 9.6, and run the QT3 tests successfully. I think (subject to further regression testing) this is probably a sufficient fix. I was a bit concerned about the fact that resetStaticProperties() isn't working properly when called from simplify() (because there is no expression stack in the expression visitor); but it seems that the code that has now been inactivated was the only place where resetStaticProperties() was called during the simplify() phase.
- Category set to Internals
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
I confirm the patch has solved the original problem.
Bug fixed and will be available in the next maintenance release.
- Status changed from Resolved to In Progress
The patch as committed is causing regression in XSLT tests that use the id() or key() function in match patterns (e.g. id/id-025). The first problem is that the Pattern20Parser checks that id() has only one argument, and key() has exactly two, and this check is failing. An additional minor problem (which we can probably live with) is that by expanding id(x) to id(x, .) earlier, we can't tell at the time of this check whether the second argument was added by the parser or actually written by the user.
- Status changed from In Progress to Resolved
Exploring the history, it seems a further patch was committed on 18 Feb 2015, and the cited test case is now working, so I beleive the problem is now resolved.
- Status changed from Resolved to Closed
- Fixed in version set to 9.6.0.5
Bug fix applied in the Saxon 9.6.0.5 maintenance release.
- Sprint/Milestone set to 9.6.0.5
- Applies to branch 9.6 added
- Fix Committed on Branch 9.6 added
- Fixed in Maintenance Release 9.6.0.5 added
Please register to edit this issue
Also available in: Atom
PDF
.