Project

Profile

Help

Bug #2789

closed

File overwritten with empty file by XQuery Update

Added by Michael Kay over 8 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
XQuery Update
Sprint/Milestone:
-
Start date:
2016-06-10
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
9.6, 9.7, trunk
Fix Committed on Branch:
9.6, 9.7, trunk
Fixed in Maintenance Release:
Platforms:

Description

Joe Wicentowski reports on the xquery-talk list

The problem is that the XQuery Update statement corrupts my file. The resulting file has 0 bytes. When I comment out the XQuery Update statement and uncomment the $test variable in the return expression, I get expected results, so I think the logic is sound. I have a feeling that the problem may have to do with line 25, where I add attributes to an element; when I comment out this, the corruption doesn't occur. But I need the attributes in that line. So I'm stumped. I've never encountered data corruption with XQuery Update, so I really hope there's a solution.

I'm using Saxon-EE XQuery 9.6.0.7 inside of oXygen 17.1, with Saxon's options for XQuery 3.0 and XQuery Update enabled.

Actions #1

Updated by Michael Kay over 8 years ago

  • Status changed from New to In Progress

My first attempt to run this was under 9.7 from the command line using -update:discard.

This produced warnings that the predicate [@nr eq 3] would fail unless @nr was empty. The predicate is relying on a schema-aware comparison. So I ran again with the -sa option.

This ran to completion producing blank output.

I then ran with -update:on (having saved a copy of the source document). This crashed with an NPE:

Building tree for null using class net.sf.saxon.tree.linked.LinkedTreeBuilder

java.lang.NullPointerException

at net.sf.saxon.tree.linked.LinkedTreeBuilder.endElement(LinkedTreeBuilder.java:298)

at net.sf.saxon.functions.AnalyzeStringFn.call(AnalyzeStringFn.java:146)

at net.sf.saxon.functions.AnalyzeStringFn.call(AnalyzeStringFn.java:33)

at net.sf.saxon.expr.FunctionCall.iterate(FunctionCall.java:546)

Then I switched to Saxon 9.6 using the command line options

-q:test.xquery -update:discard -t

This gave a warning saying bytecode could not be generated, and then failed with a different NPE:

java.lang.NullPointerException

at net.sf.saxon.expr.instruct.ProcessRegexMatchInstruction.processLeavingTail(ProcessRegexMatchInstruction.java:46)

at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:144)

at net.sf.saxon.expr.parser.ExpressionTool.evaluate(ExpressionTool.java:325)

at net.sf.saxon.expr.LetExpression.eval(LetExpression.java:476)

at net.sf.saxon.expr.LetExpression.evaluatePendingUpdates(LetExpression.java:711)

at com.saxonica.ee.optim.XQueryExpressionEE.runUpdate(XQueryExpressionEE.java:212)

The error remains if bytecode generation is switched off, and the -sa option also has no effect.

I'm going to start by exploring the 9.7 problem, and then I'll come back to 9.6 later.

Actions #2

Updated by Michael Kay over 8 years ago

The NPE in LinkedTreeBuilder is caused by the fact that AnalyzeStringFn fails to issue a startContent() call after the first startElement(). This (apparently) causes no problems when a TinyTreeBuilder is used, but in XQuery Update the default is to use a LinkedTreeBuilder and this requires the client to follow the correct protocol.

Having made this change, and running with update:on, I get the message

Overwriting file /Users/mike/bugs/2016/wicentowski/test.xml

As far as I can see, the new file contains the required result.

It's possible that the LinkedTreeBuilder crash also occurred when running in oXygen, and oXygen masked the failure, but before the crash happened the input file had already been opened for writing and therefore the old contents were lost.

Actions #3

Updated by Michael Kay over 8 years ago

The code in 9.6 for the AnalyzeStringFn is completely different from the 9.7 code; we effectively compile the call on fn:analyze-string into a rather complex xsl:analyze-string instruction. This includes a call to a custom ProcessRegexMatchInstruction (which outputs the regex group elements), and the problem seems to be that this has been loop-lifted out of the AnalyzeString instruction that sets its context.

I've prevented the loop-lifting by giving the ProcessRegexMatchInstruction the static property HAS_SIDE_EFFECTS (justified by the fact that it creates new elements). With this change the query runs to completion. The updated file is different from the original in that in the original text

I'm not convinced that the query is producing the required output, however. Further checking needed.

Actions #4

Updated by Michael Kay over 8 years ago

It turned out the 9.6 version test was using -update:discard so it's not surprising the file wasn't updated. When I switch to -update:on, I get a new error:

java.lang.NullPointerException

at net.sf.saxon.tree.linked.NodeImpl.getNamePool(NodeImpl.java:362)

at net.sf.saxon.tree.linked.NodeImpl.getDisplayName(NodeImpl.java:411)

at net.sf.saxon.om.NameOfNode.getDisplayName(NameOfNode.java:54)

at net.sf.saxon.serialize.XMLEmitter.attribute(XMLEmitter.java:411)

at net.sf.saxon.event.ProxyReceiver.attribute(ProxyReceiver.java:166)

at net.sf.saxon.event.TreeReceiver.attribute(TreeReceiver.java:209)

at net.sf.saxon.tree.linked.ElementImpl.copy(ElementImpl.java:344)

This is caused when the updated tree is being serialized to the output file. The crash happens because an attribute node (which must be the rend='italic' node) has a parent element which is not on the tree and which itself has a null parent pointer.

Correction: it's not the attribute being serialized that has the incorrect parent element. The name of the attribute is a NameOfNode() object, and this references a Node in the original tree that has been disconnected from its root, so the link to the NamePool is broken and the name cannot be resolved.

I compared the execution with 9.7, and in 9.7 the name of the attribute node is represented by a FingerprintedQName rather than by a NameOfNode object.

This is because AttributeImpl.copy() in 9.6 calls out.attribute(new NameOfNode(this))@, while 9.7 calls @out.attribute(NameOfNode.makeName(this) - the latter method tests whether the node is mutable before creating a NameOfNode object that wraps it.

I've copied the 9.7 code to 9.6 and it now runs successfully.

Actions #5

Updated by Michael Kay over 8 years ago

  • Status changed from In Progress to Resolved
  • Fix Committed on Branch 9.6, 9.7, 9.8 added

Summary of changes: for 9.7 and 9.8, AnalyzeStringFn issues a startContent() call after calling startElement() on the outermost element of the returned data structure.

For 9.6: (a) we give the ProcessRegexMatchInstruction the static property HAS_SIDE_EFFECTS to prevent loop-lifting, and (b) when we copy an attribute we make sure that the name object we create is not simply a reference to the old (mutable) node.

Patches committed.

Actions #6

Updated by O'Neil Delpratt over 8 years ago

  • % Done changed from 0 to 100
  • Fixed in Maintenance Release 9.7.0.6 added

Bug fix applied in the Saxon 9.7.0.6 maintenance release. Leave open until applied in the next 9.6 maintenance release

Actions #7

Updated by O'Neil Delpratt almost 8 years ago

  • Status changed from Resolved to Closed
  • Fixed in Maintenance Release 9.6.0.10 added

Bug fix applied in the Saxon 9.6.0.10 maintenance release

Actions #8

Updated by O'Neil Delpratt over 7 years ago

  • Applies to branch trunk added
  • Applies to branch deleted (9.8)
Actions #9

Updated by O'Neil Delpratt over 7 years ago

  • Fix Committed on Branch trunk added
  • Fix Committed on Branch deleted (9.8)

Please register to edit this issue

Also available in: Atom PDF