Saxon » SaxonJS

In principle this should be tested by XSLT3 test collations-0118.

However, that test, with most of the others in the collations test-set, is on the exceptions list because it depends on DTD parsing - the collation in use is represented using an entity reference.

I have modified most of the XSLT3 tests in the collations test-set so they no longer depend on use of entity references, which means it is now possible to run them in Saxon-JS. Most of them are failing.

A test with xsl:perform-sort

   <out xsl:default-collation="http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive">
      <xsl:perform-sort select="$data">
        <xsl:sort select="."/>
      </xsl:perform-sort>
    </out>

shows that for this case, the default collation is being respected, but it is also output as a (non-namespaced) attribute in the result:

<out defaultCollation="http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive">....</out>

If I move the default-collation attribute to the containing xsl:template element it works correctly.

The conversion of @xsl:default-collation to a no-namespace @defaultCollation attribute is done by the compiler in normalize.xsl line 288; it's not clear what the rationale is here.

The code in pastFast.bind(), which binds function calls, handles the case of arguments which are to be treated as collations, but not cases where absent arguments default to (any) default collation. As this processing operates (only) deep in XPath parsing, this needs this code section to get smarter about adding default collations.

Adding:

 if(collationArg > args.length - 1) { // Need a collation - none supplied
      checkedArgs.push(new StringLiteral(env.defaultCollation));
 }

after the loop static checking arguments in parseFast.bind() seems to pass simple tests.

Patch committed. Further test failures in xslt30 collations test set to be reviewed.

Let's work through the tests in the collations test set in turn. Currently getting

RESULTS: 19 passes, 23 failures, 0 wrong errors, 0 not run

First failure is collations-0106. This declares a key; the xsl:stylesheet element has a default-collation attribute, which doesn't find its way onto the key declaration in the SEF.

Applying the same fix as for bug #4660 solves this one, Now getting:

RESULTS: 20 passes, 22 failures, 0 wrong errors, 0 not run

Next failure: collations-0111

This uses starts-with() with a default-collation on the containing xsl:value-of instruction.

The defaultCollation property is not present in the SEF file.

Fixing this by adding the defaultCollation attribute to the list of attributes copied at creating-new-nodes#289, gets us to

RESULTS: 31 passes, 11 failures, 0 wrong errors, 0 not run

However, this is rather unsatisfactory, because this fix was specific to a default-collation attribute appearing on the xsl:value-of instruction. If the code was wrong there, it's probably wrong elsewhere. But we'll push on for now.

Next failure is collations-0121.

Here an xsl:default-collation attribute appearing on a literal result document has surfaced in the final output as a defaultCollation attribute on the output element. This is the failure reported in comment #2.

The defaultCollation appears as an attribute of the element that's supplied to the template rule at creating-new-nodes#470. At this point it's impossible to distinguish it from a user-defined attribute of this name.

Changing attributes.xsl#65 to output leave the attribute named xsl:default-collation, rather than renaming it to defaultCollation; and changing creating-new-nodes#495 to pick up this attribute.

No; we also need to change normalize.xsl#291

I've now finally got rid of the defaultCollation attribute in the output, and the SEF file now looks correct, but it's still not using the specified collation. This is because we've generated a compareToString instruction that explicitly uses the codepoint collation: "comp": "GAC|http://www.w3.org/2005/xpath-functions/collation/codepoint".

This appears to derive from the XPath parser, Comparison.js#129 which calls env.getDefaultCollation().

The <xpath> element supplied to xpath.xsl#958, which drives the XPath parsing, doesn't have an @defaultCollation attribute.

Presumably this is because the template with mode=create.xpath in XSLT-SEF-compiler#1827 was called without the tunnel parameter $default-collation being set.

This is proving a battle. I'm now getting an error XTDE0420 with very little diagnostics.

Fixed that and its finally passing. Now getting:

RESULTS: 36 passes, 6 failures, 0 wrong errors, 0 not run

Can't help feeling there's a lot of quackery involved in this. We seem to have at least four mechanisms for passing the static context: individual tunnel parameters, an $attr tunnel parameter holding multiple attributes, a $staticContext tunnel parameter, plus individual attributes sometimes in the ex namespace and sometimes not. This could really do with rationalisation.

Next failure: collations-0127. This is giving

s=collations t=collations-0127
Actual error: XTSE0090
Expected success, got XTSE0090:

This is a new test; collations-0127 simply imports collations-0126.xsl, which has a default-collation on the xsl:stylesheet element.

The test driver gives us no useful diagnostics, but running the stylesheet from the command line gives:

Error XTSE0090 at xsl:import/@default-collation on line 6 in collations/collations-0127.xsl:
 xsl:import must not have attribute @default-collation

Adding a stack trace to f:wrongAttribute() in normalize.xsl gives:

WrongAttribute!!
    at function Q{MyFunctions}wrongAttribute in normalize.xsl#822

    Processing element <xsl:stylesheet>
    at template rule in static.xsl#131
    Processing element <xsl:stylesheet>
    at template rule in XSLT-SEF-compiler.xsl#388
    Processing document #1
    at template rule in XSLT-SEF-compiler.xsl#280

We're copying the default-collation attribute from the xsl:stylesheet element to all its children, which is fine: a default-collation attribute is allowed on any element, including xsl:import. The error is spurious. Fixed at static.xsl#544 by allowing all standard attributes on xsl:include/import.

Now getting

RESULTS: 37 passes, 5 failures, 0 wrong errors, 0 not run

Next failure: collations-0301

This one doesn't use a specific collation URI, rather it uses attributes such as case-order and lang to control sorting; and the problem with the result isn't the sort order, it's missing spaces (between first name and last name) in the output.

The spaces should be produced by a whitespace-only xsl:text element: <t:text> </t:text>. Perhaps my fix for bug #4681 affected this (the fix has been found to cause other test failures).

In the SEF file, the relevant xsl:text element becomes valueOf(str(val="")) which is incorrect (the space has been lost).

I think it's being lost in template name="staticText" (static.xsl#863) which after the fix for #4681 is now processing xsl:text elements, and needs special logic if they are whitespace-only.

Fixed this: modified the patch for #4681 so the xsl:next-match instruction is executed with xml-space-preserve=true.

Results now:

RESULTS: 41 passes, 1 failures, 0 wrong errors, 0 not run

The remaining test failure is collations-1006. This uses UCA collations with fallback=no. We are getting error FOCH0002 (unsupported collation). I think this is an acceptable outcome: the spec says:

If the fallback parameter is present with the value no, then the implementation must either use a collation that conforms with the rules in the Unicode specifications for the requested tailoring, or fail with a static or dynamic error indicating that it does not provide the collation

I will change the test to allow this outcome.

Results now:

RESULTS: 42 passes, 0 failures, 0 wrong errors, 0 not run

Confirmed that the status of collations-1006 is still as described: in the SEF file, the templateRule element has a correct defaultCollation attribute, but nested expressions such as compareToString and fn name="starts-with" have an explicit reference to the codepoint collation, which overrides the default.

The XX compiler code at xpath.xsl#984 is expecting the attribute @defaultCollation to be present on the xpath element, but in this test case it is not present. The attribute is present when compiling an expression appearing on xsl:if/@test, but it is not present in an expression appearing in an AVT or TVT.

The code at value-templates#65 is not generating the @defaultCollation attribute on the generated xpath element because the value of the tunnel parameter uniform-static-context is true. The code that generates xpath elements for @test or @select attributes at XSLT-SEF-compiler#1826 generates a defaultCollation attribute regardless of the value of uniform-static-context.

Changed value-templates to do the same - i.e. to generate @defaultCollation and @xpathDefaultNamespace attributes regardless of the value of $uniform-static-context. This has the desired effect of generating the correct collation URI in the SEF file.

Believed fixed; have committed the change (in value-templates.xsl line 84); not fully tested because of problems with the build environment.