Saxon

Looking at the first failure, the fifth test in the <parent> group is

<xsl:copy-of select="my:compare($n0/descendant-or-self::node(), concat($id0,':',$id1))"/>

This is evaluating generate-id on a temporary document node containing a single text node.

The output is <error required="d173a" actual="d173a:d173aA"/> .

It's difficult to make sense of this, because the value of @required in the output should be the value of the second argument passed to my:compare, which definitely contains a colon! It's almost as if the slot numbers for variables are being mixed up.

It looks as if in the SEF file, the call on concat($x, ':', $y) has become concat(($x, ':', $y)) - that is, a call with a single argument that is a sequence of strings. This probably reflects changes made in SaxonJ to support variadic functions in a more general way. The SaxonJS run-time is presumably just taking the first item in each argument, that is, the first of the three items.

I've fixed the concat() problem; SaxonJ now special-cases the export() code for a call on concat().

Still seeing test failures, however: the current list is

  accumulator: 1
  attribute: 1
  catalog: 1
  core-function: 4
  date: 2
  document: 2
  error: 4
  evaluate: 1
  global-context-item: 1
  merge: 1
  message: 4
  mode: 6
  next-match: 1
  output: 5
  result-document: 1
  snapshot: 1
  static: 3
  transform: 2
  type: 1
  xml-to-json: 60

Looking at the core-function failures, these are tests -019, -021, -037, -039 and these appear to be failures in concat(), so presumably the patch is not catching all cases.

core-function-019 contains the call concat($variable1,'From ','Variable'), which results in the SEF containing a call on concat() with two arguments, the second one being the sequence ('From', 'Variable') expressed as a Literal. The patch doesn't handle this case.

Fixed the patch, the number of failures is reduced slightly:

 accumulator: 1
  attribute: 1
  catalog: 1
  date: 2
  document: 2
  error: 4
  evaluate: 1
  global-context-item: 1
  merge: 1
  message: 4
  mode: 6
  next-match: 1
  output: 5
  result-document: 1
  snapshot: 1
  static: 3
  transform: 2
  type: 1
  xml-to-json: 60

Note for comparison, running the test suite with the XX compiler gives the following failures:

Failures by test-set: 
  attribute: 1
  catalog: 1
  document: 1
  error: 3
  key: 1
  match: 9
  message: 4
  mode: 2
  snapshot: 1
  transform: 2
  type-available: 1

Looking at test set mode, the failures are in -1106b, -1106e, -1107b, -1702a, -1705b, -1714err.

We're also seeing a number of messages of the form

s=mode t=mode-1440
** Expect no-matching-template warnings **
assert-xml: failed to parse/retrieve reference XML results: Non-whitespace before first tag. at line 0 column 1

The test has

            <assert-message>
               <assert-xml><![CDATA[** Expect no-matching-template warnings **]]></assert-xml>
            </assert-message>

and it's "discouraged" to use assert-xml when the result isn't a well-formed XML document. I'm going to change these to assert-string-value.

Looking at 1106b, the expected output is error XTDE3362, but this isn't detected. The spec says:

[ERR XTDE3362] It is a dynamic error to call the accumulator-before or accumulator-after function when the context item is a node in a tree to which the selected accumulator is not applicable (including the case where it is not applicable because the document is streamed and the accumulator is not declared with streamable="yes"). Implementations may raise this error but are not required to do so, if they are capable of streaming documents without imposing this restriction.

SaxonJ reports the error as:

Error on line 31 of mode-1106.xsl:
  XTDE3362  Accumulator counter is not applicable to the current document
  In template rule with match="/" on line 29 of mode-1106.xsl

The initial mode is Y, and the accumulator "counter" is not listed as an applicable accumulator for this mode.

In the SEF file entry for mode Y, the XX compiler generates the property useAcc: "", while with the XJ compiler, the property is absent.

I've changed mode.export() in SaxonJS to output useAcc="" if the set of applicable accumulators is empty, and the test now works. The code here hasn't changed since 10.x, so I suspect this bug was also present in 10.x. Ideally we should also change SaxonJS to accept the absence of the property as equivalent to an empty value.

The list of failing tests now looks like this:

Failures by test-set: 
  accumulator: 1
  attribute: 1
  catalog: 1
  copy: 1
  date: 2
  document: 2
  error: 4
  evaluate: 1
  global-context-item: 1
  message: 4
  mode: 3
  next-match: 1
  output: 5
  result-document: 1
  snapshot: 1
  static: 3
  transform: 2
  type: 1
  xml-to-json: 60

It's time we looked at xml-to-json.

Most of the xml-to-json failures were because the code uses streaming without having a test dependency on streaming; I've added the dependencies.

Test document-0401 is failing in SaxonJS with the XJ compiler but succeeds with the XX compiler. The failure is because it doesn't eliminate duplicate nodes. The XX compiler produces a SEF file containing a docOrder instruction (actually two, one of them redundant) while the XJ compiler output has no such instruction.

I think that SaxonJ is dropping the call on docOrder() because it believes that the result of document() will always be in document order with no duplicates. This is true of the SaxonJ implementation of document(), but it's not true of the SaxonJS implementation. I'll raise a separate bug on this.

Great, looks to me like you have addressed all of the XJ-compile regression from 10.7 previously raised: i.e. all tests which fail with 11.2 but pass with 10.7 now also pass when XJ-compiling the tests from the current Saxon 11 branch. You have also fixed some failures which have been lurking for a very long time!

But there is one new test failure following your updates: copy-3002. Please could you investigate that one?

Some of these tests (for example xml-to-json-A2-004) are failing

Internal error: Unknown expr: consume parent=undefined

This means the SaxonJ compiler is emitting a "consume" instruction which SaxonJS doesn't understand.

The "consume" represents a ConsumingOperand, which is supposedly associated with streaming. And indeed, these tests make use of streaming. There should be a dependency declared on streaming, and the tests should not be run in SaxonJS. (Note, this also illustrates a problem with the design of xsl:use-package. We are using a package that uses streaming or not, depending on the value of a static parameter; but xsl:use-package provides no way to control the setting of static parameters in the library package. We could potentially control it via the test driver, but it would add a fair bit of complexity.)

I've changed SaxonJS so it fails cleanly if a "consume" instruction is encountered.

copy-3002 is failing because of the change in comment #9. It fails saying the accumulator isn't applicable.

We're doing this:

    <xsl:variable name="v" as="element()">
      <xsl:copy-of select="//*[@x='a3-4']" copy-accumulators="yes"/>
    </xsl:variable>
    <xsl:value-of select="$v/accumulator-before('latest-pick'), 
                          $v/accumulator-after('latest-pick') "/>

so we're looking at an accumulator on a temporary tree that was copied from the accumulator values on the principal source document. The spec says (in §18.2.2 #5)

For a document containing nodes supplied in the initial match selection, the accumulators that are applicable are those determined by the xsl:mode declaration of the initial mode. This means that in the absence of an xsl:mode declaration, no accumulators are applicable.

and rule 6 says:

For a tree T created by copying a node in a tree S using the copy-of or snapshot functions, or the instruction xsl:copy-of with copy-accumulators="yes", an accumulator is applicable to T if and only if it is applicable to S.

So I think the accumulator isn't applicable, and the dynamic error is acceptable. On the other hand, I think if we started raising a dynamic error for test copy-3002 then a lot of user applications would break.

The spec for error 3362 says:

[ERR XTDE3362] It is a dynamic error to call the accumulator-before or accumulator-after function when the context item is a node in a tree to which the selected accumulator is not applicable (including the case where it is not applicable because the document is streamed and the accumulator is not declared with streamable="yes"). Implementations may raise this error but are not required to do so, if they are capable of streaming documents without imposing this restriction.

How should we interpret the final caveat "if they are capable of streaming documents without imposing this restriction"? How does this apply to non-streaming processors? Can we ignore the caveat and just treat all accumulators as if they were applicable to all documents?

For the moment, there is clearly a problem with the test: the spec seems to be clear that raising XTDE3362 is appropriate here, though it's not 100% clear that it's required. The question then becomes whether SaxonJ should be enforcing the rules more strictly, and whether we get a backwards compatibility problem if we start doing so.

I should move this to a different bug entry.

Returning to this after doing some significant restructing of the test drivers.

Running the test exports with the XJ compiler, I'm seeing the following failures:

  as : 4
  date : 1
  expose : 1
  format-date-en : 1
  next-match : 3
  package : 2

Then when we run the exported tests, we get the following failures:

RESULTS: 6440 passes, 13 failures, 23 wrong errors, 8007 not run
Failures by test-set: 
  accumulator: 1
  catalog: 1
  copy: 1
  error: 2
  key: 1
  message: 4
  mode: 1
  transform: 2

Looking at the first group (failure during SaxonJ export) in the first group the failures are:

  as-0106
  as-0110
  as-0111
  as-0112

All four report "Year out of range for SaxonJS" - which should be a "test not run" condition rather than a failure. But it seems the test driver isn't smart enough to handle this. Let's ignore this group of tests for now.

I suspect that the four message test failures are the result of a test driver bug when handling assertions against multiple messages: the actual message output looks correct.

We're handling messages, it seems, by passing deliverMessage as an option when we call fn:transform() to process the stylesheet-under-test. This option is documented as part of the SaxonJS.transform() API, but it's not documented at the level of fn:transform() (either as a standard W3C option, nor as a Saxon extension). That means it's not well-defined what it actually does. The test driver is assuming that it returns an result?messages entry containing a sequence of document nodes, but I think it might be an array of document nodes...

Indeed, further investigation shows that result?messages is an array of messages, not a sequence. Given that it's an undocumented extension, I think I'll accept that as correct behaviour and change the test driver to expect it.

Unfortunately this still leaves us with two failing message tests: message-0312 and message-0501

message-0312 uses assert-xml on content that is not well-formed XML, which is strongly discouraged. The assertion should be changed.

However, that's not the real problem. The xsl:message has both a select attribute and a sequence constructor, and the select attribute seems to be ignored. The SEF file appears to be correct. The test actually fails the same way whether compiled using XJ or XX.

Transferred to SaxonJS bug #5408.

The failure in key-099 is bug #5230, which I have re-opened.

The transform tests that fail are transform-008 and transform-009. They fail with both compilers.

Returning to this after doing a fair bit of work improving the SaxonJS test driver. New results (for the XJ compiler):

RESULTS: 6413 passes, 40 failures, 23 wrong errors, 8007 not run
Failures by test-set: 
  accumulator: 2
  catalog: 1
  copy: 1
  date: 2
  error: 4
  evaluate: 1
  global-context-item: 1
  merge: 1
  mode: 6
  next-match: 1
  output: 5
  result-document: 1
  static: 3
  transform: 2
  type: 1
  xml-to-json: 8

The above results were obtained running within IntelliJ. When running with gradle, I see 3 extra failures in the match test-set, and the number of failures in xml-to-json increases to 30.

Further updates to the test driver, exceptions and test suites (see bug #4926) have been made. Current results (with current saxon 11 branch build of jars):

RESULTS: 6442 passes, 31 failures, 22 wrong errors, 8065 not run
Failures by test-set: 
  accumulator: 1
  date: 2
  evaluate: 1
  global-context-item: 1
  merge: 1
  mode: 6
  next-match: 1
  output: 5
  result-document: 1
  static: 3
  type: 1
  xml-to-json: 8