Saxon

Test case regex-090 is producing the incorrect output 222222: it's failing to implement the rule that during evaluation of a dynamic function call (including a call on regex-group#1), the set of captured substrings is set to an empty sequence.

It seems that the test is working without tracing only because the body of the outer xsl:matching-substring is recognised as having no context dependency, and is therefore promoted to a global variable; the test seems to be working largely because the global variable is evaluated with no captured substrings in the context, not because of the dynamic function call.

I have demonstrated this by creating a new test case regex-091 which fails even without the -T option.

Fixed by changing SystemFunction.dynamicCall() to set currentRegexIterator in the newly created context to null.

Fixed the LRE streaming tests with a rather pragmatic change: in ComplexNodeEventFeed, change the startSelectedNode and endSelectedNode methods to do nothing rather than throwing an UnsupportedOperationException. I have to confess I'm not quite sure why this fixes the problem.

We now have the following test failures (with -T) still outstanding:

  si-assert : 1
  si-for-each-group : 7
  streamable : 2
  su-absorbing : 3
  use-package : 1

Now looking at the streamed grouping tests. The failing ones are:

Failing tests: 
  si-group-001
  si-group-016
  si-group-022
  si-group-023
  si-group-043
  si-group-045
  si-group-046

In group-001, the problem appears to be that current-group() effectively returns an empty sequence.

Without -T, test group-001 has a streaming pipeline comprising:

ForEachGroupPartitionAction ElementCreatorFeed DecomposingFeed ComplexContentOutputter

With -T this becomes

ForEachGroupPartitionAction
TraceFeed
ElementCreatorFeed (for <out>)
TraceFeed
DecomposingFeed
ComplexContentOutputter

Nothing obviously wrong there.

When we start a new group, the inner feed in the non-tracing case contains:

CopyOfFeed
BlockFeed
ElementCreatorFeed (for <batch>)
NoOpenOrCloseFeed
ElementCreatorFeed (for <out>)
DecomposingFeed
ComplexContentOutputter

With -T, apart from the addition of TraceFeeds into the pipeline, the significant difference appears to be that the CopyOfFeed is missing. This would account for the symptoms (the missing transaction elements in the output). The decision when and whether to insert a CopyOfFeed has always been rather convoluted.

In the non-streaming case, the copy-of operation is added to the expression tree by FixedElement.optimize() (for the batch LRE) calling optimizer.makeCopyOperationsExplicit().

In the streaming case, this isn't happening, because when tracing is enabled, optimization is suppressed.

Confirmed that if we remove the rule that enabling tracing disables optimization, the streamed grouping tests all run successfully with or without tracing.

I think the answer here is probably for tracing to disable optimization more selectively: in particular, it should only inhibit the major structural optimisations such as function inlining.

With this change, I get infinite recursion in LetExpression.optimize() for test case -s:import-schema -t:import-schema-192 (with -T option). This is nothing to do with tracing: it seems it can happen whenever optimization takes place, but inlining of variables is suppressed.

With this fixed, we now have only two tests failing with -T:

  streamable : 1
  use-package : 1

Test streamable-139 crashes:

java.lang.UnsupportedOperationException: net.sf.saxon.trans.XPathException: Navigation using preceding-sibling axis is not supported from a streamed input node
	at com.saxonica.ee.stream.om.FleetingNode.iterateAxis(FleetingNode.java:521)
	at com.saxonica.ee.stream.om.FleetingElementNode.iterateAxis(FleetingElementNode.java:127)
	at net.sf.saxon.tree.wrapper.VirtualCopy.iterateAxis(VirtualCopy.java:498)
	at net.sf.saxon.tree.wrapper.SnapshotNode.iterateAxis(SnapshotNode.java:260)
	at net.sf.saxon.tree.util.Navigator.getNumberSimple(Navigator.java:348)

The root cause of this appears to be that we have a ForEachAction with the property actionIsConsuming=false; but the action in question is snapshot(.) which surely is consuming.

Not so: the expression in question is text() ! snapshot(.), and snapshot() applied to a text node is indeed motionless.

We are constructing the result of snapshot() here as a virtual copy of a fleeting (streamed) text node, and the axis iteration on this structure can't cope. We either need to make it cope, or we need to use a different structure The former approach seems easier.

Fixed by adding logic to SnapshotNode.iterateAxis().