Saxon

Running in 9.9 branch project. Without -o option, get no console output with or without -t. But get console output for -T.

Using saxon:trace on accumulators. Without -T get output ending:

values BEFORE /root/fields/data: ()
remainder BEFORE /root/fields/data: ()
values BEFORE /root/fields/data/id1/text():  (3, 2, 1, ... [6])
B AFTER /root/fields/data/id1: 20
A AFTER /root/fields/data/id1: 6
values BEFORE /root/fields/data/id2/text():  (20, 20, 20, ... [6])
D AFTER /root/fields/data/id2: 6
C AFTER /root/fields/data/id2: 60
remainder-sum AFTER /root/fields/data: ()
remainder AFTER /root/fields/data: 1

With -T:

remainder BEFORE /root/fields/data: ()
values BEFORE /root/fields/data: ()
values BEFORE /root/fields/data/id1/text():  (3, 2, 1, ... [6])
A AFTER /root/fields/data/id1: 6
B AFTER /root/fields/data/id1: 20
values BEFORE /root/fields/data/id2/text():  (20, 20, 20, ... [6])
D AFTER /root/fields/data/id2: 6
C AFTER /root/fields/data/id2: 60
remainder AFTER /root/fields/data: 1
remainder-sum AFTER /root/fields/data: 7

remainder-sum depends on remainder (select="$value + accumulator-after('remainder')") so should be evaluated afterwards. But the trace suggests this isn't happening (without -T). Perhaps the order is actually random?

Thinking about how we might do this. In the worst case, dependencies among accumulators cannot be determined statically, because the argument to accumulator-before() or -after() is not required to be a string literal (moreover, the call to accumulator-before() or -after() can be dynamically invoked via a chain of function or template invocations). So one approach would be to evaiuate all the accumulators associated with a particular streaming event (logically) in parallel, with each thread waiting if it needs the value of an accumulator that is not yet available; and ideally some kind of deadlock detection (though the spec does say that "catastrophic failure" is allowed in the event of a cyclic dependency, which would permit an undetected deadlock in principle).

Wouldn't it be nice to have co-routines and a yield construct!

Short of a parallel implementation, another approach would be to choose an arbitrary order, and have an accumulator evaluation throw an exception if its dependencies are not satisfied; it would then go to the back of a queue for trying again. We could possibly optimize for the common case where the dependencies ARE known statically.

Further thoughts: rather than having each AccumulatorWatch called directly from the WatchManager, we could pass all the relevant events via a new AllAccumulatorsWatch class, which would activate each AccumulatorWatch in turn, allowing any of them to invoke the others using a recursive call (with a check to prevent reentrancy). The AllAccumulatorsWatch would start each node/phase event by marking all accumulators as unavailable (for that phase), and a request for an accumulator that is not available would trigger a recursive call to process that accumulator.

Added streaming and non-streaming versions of this test to XSLT3 test suite as accumulator-079[s].

I've implemented a MultiAccumulatorWatch to the main WatchManager now issues a single call for each event; the MultiAccumulatorWatch is responsible for distributing this to each individual AccumulatorWatch. This is working and seems a useful step forward.

In doing this I've noticed that AccumulatorWatch.getPreDescentValue() already has logic to allow for someone trying to get the BEFORE value before it has been computed; if the current node hasn't yet been notified then it gets processed now, before returning the value. There is no similar logic for getPostDescentValue(); perhaps that's all that's needed.

I'm now handling post-descent accumulator recursion the same way as pre-descent: if the accumulator value is requested, and we haven't yet seen the endElement event for the relevant node, then we fire the endElement action for the node, and note it as having been processed, so when the event is subsequently notified by the normal route, we ignore it.

The new design is working well, and seems sufficiently robust that I now intend to retrofit it as a patch to 9.9.

Also, the new test for cyclic accumulators (accumulator-080) crashes with stack overflow when bytecode is enabled. This is not technically a non-conformance (the spec explicitly allows "catastrophic failure" for this condition, just as with infinite function recursion) but it would be nice to do better.

The patch caused the saxon:capture feature on accumulators to stop working: this has now been fixed.

The problem was that saxon:capture adds a Watch which gathers a snapshot of a streamed node for use when the phase=end rule is evaluated; but the normal snapshot mechanism copies accumulator values, which must therefore already be available before the snapshot is completed. I don't actually know why this didn't cause a problem before. But I've fixed it by adding an option to streaming snapshot feeds to avoid copying accumulators, and setting this option for these internal snapshots.