Bug #4616
closeddefault-collation
Added by Michael Kay over 4 years ago. Updated almost 4 years ago.
100%
Description
If an [xsl:]default-collation attribute appears in the stylesheet, the XX compiler appears to ignore it.
Given the input
<xsl:copy-of select="saxon:highest($data, my:val#1)"
default-collation="http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive" />
<xsl:copy-of select="max($data)"
default-collation="http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive" />
The XJ compiler generates a defaultCollation
attribute on the sequence
node representing the sequence constructor (I think it detects that the static context for both instructions is the same). It also generates a collation
attribute on the fn:max function call - this is done by SystemFunction.export calling out.emitRetainedStaticContext().
The output of the XX compiler does not include the default collation anywhere, which means that neither the built-in function max()
nor the extension function saxon:highest()
can take it into account.
Updated by Michael Kay over 4 years ago
In principle this should be tested by XSLT3 test collations-0118.
However, that test, with most of the others in the collations test-set, is on the exceptions list because it depends on DTD parsing - the collation in use is represented using an entity reference.
I have modified most of the XSLT3 tests in the collations test-set so they no longer depend on use of entity references, which means it is now possible to run them in Saxon-JS. Most of them are failing.
Updated by Michael Kay over 4 years ago
A test with xsl:perform-sort
<out xsl:default-collation="http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive">
<xsl:perform-sort select="$data">
<xsl:sort select="."/>
</xsl:perform-sort>
</out>
shows that for this case, the default collation is being respected, but it is also output as a (non-namespaced) attribute in the result:
<out defaultCollation="http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive">....</out>
If I move the default-collation attribute to the containing xsl:template
element it works correctly.
The conversion of @xsl:default-collation
to a no-namespace @defaultCollation
attribute is done by the compiler in normalize.xsl line 288; it's not clear what the rationale is here.
Updated by Michael Kay over 4 years ago
- Status changed from New to In Progress
- Assignee changed from Michael Kay to John Lumley
- Priority changed from Low to Normal
- Applies to JS Branch 2.0 added
Updated by John Lumley about 4 years ago
The code in pastFast.bind()
, which binds function calls, handles the case of arguments which are to be treated as collations, but not cases where absent arguments default to (any) default collation. As this processing operates (only) deep in XPath parsing, this needs this code section to get smarter about adding default collations.
Updated by John Lumley about 4 years ago
Adding:
if(collationArg > args.length - 1) { // Need a collation - none supplied
checkedArgs.push(new StringLiteral(env.defaultCollation));
}
after the loop static checking arguments in parseFast.bind()
seems to pass simple tests.
Updated by Debbie Lockett about 4 years ago
Patch committed. Further test failures in xslt30 collations test set to be reviewed.
Updated by Michael Kay about 4 years ago
Let's work through the tests in the collations test set in turn. Currently getting
RESULTS: 19 passes, 23 failures, 0 wrong errors, 0 not run
First failure is collations-0106. This declares a key; the xsl:stylesheet
element has a default-collation
attribute, which doesn't find its way onto the key declaration in the SEF.
Applying the same fix as for bug #4660 solves this one, Now getting:
RESULTS: 20 passes, 22 failures, 0 wrong errors, 0 not run
Updated by Michael Kay about 4 years ago
- Assignee changed from John Lumley to Michael Kay
Next failure: collations-0109. This is a call on the fn:compare()
function with a default-collation defined at the xsl:template level.
In the SEF file, the templateRule
object has a correct defaultCollation
property.
The implementation of fn:compare calls Compare.getCollationUri() to get the collation URI. This makes no attempt to access the default collation; instead it appears to fall back to the codepoint collation URI. Changing it to look for a default collation fixes this test and gets us to
RESULTS: 21 passes, 21 failures, 0 wrong errors, 0 not run
Updated by Michael Kay about 4 years ago
Next failure: collations-0111
This uses starts-with()
with a default-collation
on the containing xsl:value-of
instruction.
The defaultCollation
property is not present in the SEF file.
Fixing this by adding the defaultCollation
attribute to the list of attributes copied at creating-new-nodes#289, gets us to
RESULTS: 31 passes, 11 failures, 0 wrong errors, 0 not run
However, this is rather unsatisfactory, because this fix was specific to a default-collation
attribute appearing on the xsl:value-of
instruction. If the code was wrong there, it's probably wrong elsewhere. But we'll push on for now.
Updated by Michael Kay about 4 years ago
Next failure is collations-0121.
Here an xsl:default-collation
attribute appearing on a literal result document has surfaced in the final output as a defaultCollation
attribute on the output element. This is the failure reported in comment #2.
The defaultCollation
appears as an attribute of the element that's supplied to the template rule at creating-new-nodes#470. At this point it's impossible to distinguish it from a user-defined attribute of this name.
Changing attributes.xsl#65 to output leave the attribute named xsl:default-collation
, rather than renaming it to defaultCollation
; and changing creating-new-nodes#495 to pick up this attribute.
No; we also need to change normalize.xsl#291
I've now finally got rid of the defaultCollation
attribute in the output, and the SEF file now looks correct, but it's still not using the specified collation. This is because we've generated a compareToString
instruction that explicitly uses the codepoint collation: "comp": "GAC|http://www.w3.org/2005/xpath-functions/collation/codepoint"
.
This appears to derive from the XPath parser, Comparison.js#129 which calls env.getDefaultCollation()
.
The <xpath>
element supplied to xpath.xsl#958, which drives the XPath parsing, doesn't have an @defaultCollation attribute.
Presumably this is because the template with mode=create.xpath in XSLT-SEF-compiler#1827 was called without the tunnel parameter $default-collation being set.
This is proving a battle. I'm now getting an error XTDE0420 with very little diagnostics.
Fixed that and its finally passing. Now getting:
RESULTS: 36 passes, 6 failures, 0 wrong errors, 0 not run
Can't help feeling there's a lot of quackery involved in this. We seem to have at least four mechanisms for passing the static context: individual tunnel parameters, an $attr tunnel parameter holding multiple attributes, a $staticContext tunnel parameter, plus individual attributes sometimes in the ex namespace and sometimes not. This could really do with rationalisation.
Updated by Michael Kay about 4 years ago
Next failure: collations-0127. This is giving
s=collations t=collations-0127
Actual error: XTSE0090
Expected success, got XTSE0090:
This is a new test; collations-0127 simply imports collations-0126.xsl, which has a default-collation on the xsl:stylesheet element.
The test driver gives us no useful diagnostics, but running the stylesheet from the command line gives:
Error XTSE0090 at xsl:import/@default-collation on line 6 in collations/collations-0127.xsl:
xsl:import must not have attribute @default-collation
Adding a stack trace to f:wrongAttribute() in normalize.xsl gives:
WrongAttribute!!
at function Q{MyFunctions}wrongAttribute in normalize.xsl#822
Processing element <xsl:stylesheet>
at template rule in static.xsl#131
Processing element <xsl:stylesheet>
at template rule in XSLT-SEF-compiler.xsl#388
Processing document #1
at template rule in XSLT-SEF-compiler.xsl#280
We're copying the default-collation attribute from the xsl:stylesheet element to all its children, which is fine: a default-collation attribute is allowed on any element, including xsl:import. The error is spurious. Fixed at static.xsl#544 by allowing all standard attributes on xsl:include/import.
Now getting
RESULTS: 37 passes, 5 failures, 0 wrong errors, 0 not run
Updated by Michael Kay about 4 years ago
Next failure: collations-0301
This one doesn't use a specific collation URI, rather it uses attributes such as case-order and lang to control sorting; and the problem with the result isn't the sort order, it's missing spaces (between first name and last name) in the output.
The spaces should be produced by a whitespace-only xsl:text
element: <t:text> </t:text>
. Perhaps my fix for bug #4681 affected this (the fix has been found to cause other test failures).
In the SEF file, the relevant xsl:text
element becomes valueOf(str(val=""))
which is incorrect (the space has been lost).
I think it's being lost in template name="staticText"
(static.xsl#863) which after the fix for #4681 is now processing xsl:text
elements, and needs special logic if they are whitespace-only.
Fixed this: modified the patch for #4681 so the xsl:next-match
instruction is executed with xml-space-preserve=true
.
Results now:
RESULTS: 41 passes, 1 failures, 0 wrong errors, 0 not run
Updated by Michael Kay about 4 years ago
The remaining test failure is collations-1006. This uses UCA collations with fallback=no. We are getting error FOCH0002 (unsupported collation). I think this is an acceptable outcome: the spec says:
If the fallback parameter is present with the value no, then the implementation must either use a collation that conforms with the rules in the Unicode specifications for the requested tailoring, or fail with a static or dynamic error indicating that it does not provide the collation
I will change the test to allow this outcome.
Results now:
RESULTS: 42 passes, 0 failures, 0 wrong errors, 0 not run
Updated by Michael Kay about 4 years ago
- Category set to XSLT Conformance
- Status changed from In Progress to Resolved
- Fix Committed on JS Branch Trunk added
I'm now committing these changes. There are still some XSLT3 test failures, but I'm hoping they are unrelated. Current results:
Duration: 7m 40.15s
RESULTS: 6304 passes, 34 failures, 46 wrong errors, 5349 not run
Failures by test-set:
accumulator: 4
attribute: 1
bug: 1
collection: 2
document: 1
error: 3
function-lookup: 1
global-context-item: 1
higher-order-functions: 2
match: 2
merge: 2
message: 4
namespace: 2
number: 1
output: 1
shadow: 1
snapshot: 1
system-property: 3
type: 1
Updated by Michael Kay about 4 years ago
- Status changed from Resolved to In Progress
Test collations-1006 is still failing. Although the test now allows FOCH0002, we aren't returning that; we are returning a success result that appears to ignore the requested collation.
The XX compiler is generating a SEF file in which the default-collation attribute is correctly present at the xsl:template level, but calls on functions such as starts-with() have an extra parameter injected, set to the codepoint collation URI.
Updated by Michael Kay almost 4 years ago
Confirmed that the status of collations-1006 is still as described: in the SEF file, the templateRule
element has a correct defaultCollation
attribute, but nested expressions such as compareToString
and fn name="starts-with"
have an explicit reference to the codepoint collation, which overrides the default.
Updated by Michael Kay almost 4 years ago
The XX compiler code at xpath.xsl#984
is expecting the attribute @defaultCollation
to be present on the xpath
element, but in this test case it is not present. The attribute is present when compiling an expression appearing on xsl:if/@test
, but it is not present in an expression appearing in an AVT or TVT.
The code at value-templates#65
is not generating the @defaultCollation
attribute on the generated xpath
element because the value of the tunnel parameter uniform-static-context
is true. The code that generates xpath
elements for @test
or @select
attributes at XSLT-SEF-compiler#1826
generates a defaultCollation
attribute regardless of the value of uniform-static-context
.
Changed value-templates to do the same - i.e. to generate @defaultCollation and @xpathDefaultNamespace attributes regardless of the value of $uniform-static-context
. This has the desired effect of generating the correct collation URI in the SEF file.
Updated by Michael Kay almost 4 years ago
Believed fixed; have committed the change (in value-templates.xsl line 84); not fully tested because of problems with the build environment.
Updated by Community Admin almost 4 years ago
- Applies to JS Branch 2 added
- Applies to JS Branch deleted (
2.0)
Updated by Debbie Lockett almost 4 years ago
- Status changed from In Progress to Resolved
- Fix Committed on JS Branch 2 added
- Fix Committed on JS Branch deleted (
Trunk)
Confirm the fix works and has been tested (collations-1006 now passes).
Updated by Debbie Lockett almost 4 years ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in JS Release set to Saxon-JS 2.1
Bug fix applied in the Saxon-JS 2.1 maintenance release.
Please register to edit this issue
Also available in: Atom PDF Tracking page