Bug #3711
closedSCM: inadequate representation of value constraints
100%
Description
In an SCM file representing the exported form of a compiled XSD schema, the information that is retained for value constraints (that is, "fixed" and "default" values of elements or attributes) does not match what the XSD 1.1 specification mandates for the {value constraint} property records.
Specifically, XSD 1.1 says that the schema component model should include both the "actual value" and the "lexical form".
Saxon's SCM file includes the lexical form (only) in the case of the default value, and the "actual value" (only) in the case of a fixed value. (The actual value, of course, has to be serialized in the XML representation of the SCM; but the point is that we are serializing it by converting the actual value back to a string, not by retaining the original lexical form.)
This applies to the ElementDecl, AttributeDecl, and AttributeUse components.
The practical consequence is that, in the case of the fixed value, the lexical form cannot be reconstituted, which means it is difficult for a schema validator to ensure that the value is consistent with facets that operate on the lexical form, such as the pattern facet.
Related issues
Updated by Michael Kay over 6 years ago
There was a change here between XSD 1.0 and XSD 1.1 In 1.0, the value constraint includes the typed value only; in 1.1 it contains both the lexical form and the typed value. Along with this, 1.1 drops the concept of "canonical lexical representation". This resolves issues as to how lexical facets such as "pattern" should be handled: it is now clear that they apply to "lexical form" as supplied (after whitespace normalization), not to the result of converting the typed value back to its lexical representation.
For an Element declaration (but not for an Attribute Declaration or Attribute Use) we are holding both the typed and lexical forms of the default value. But the lexical form we retain is obtained as the canonical lexical form, not the original supplied form.
Updated by Michael Kay over 6 years ago
- Category set to Schema-Aware processing
- Status changed from New to Resolved
- Priority changed from Low to Normal
- Applies to branch 9.8, trunk added
- Fix Committed on Branch 9.8, trunk added
I have changed the code to output the lexical form of the fixed value to the SCM, for element declarations, attribute uses, and attribute declarations.
Outputting the typed form of the default value is not really necessary since it can always be constructed from the lexical representation; the converse is not true.
Updated by Michael Kay over 6 years ago
- Status changed from Resolved to In Progress
Reopening. Two relevant XSD tests are failing when run with SCM enabled: Microsoft/additional addB149 and isDefault079. I'm not sure if this has always been the case or if it is caused by these changes, but it means further investigation is needed.
Updated by Michael Kay over 6 years ago
The change described in comment #2 is not quite right: for list types at least, it is outtputting the lexical-form of the fixed value in the form it was written in the source, not the form after applying the whitespace normalization rules. Test case MS/attribute/attO009 applies.
Applied a further fix: the method makeFixedValueTyped() in AttributeDecl, AttributeUse, and ElementDecl now sets the lexical form of the fixed value to the value after whitespace normalization.
Updated by Michael Kay about 6 years ago
- Description updated (diff)
For 9.9 I have introduced an interface ComponentWithValueConstraint
implemented by ElementDecl
, AttributeDecl
, and AttributeUse
, so the logic is now in one place rather than being replicated across the three classes. In addition these components now reference a ValueConstraint object holding the fixed or default value. The structure is now the same as the model in the XSD 1.1 specification.
This means that typed and lexical values are now held symmetrically for the "fixed" and "default" properties, on all three component types.
The XSD test suite is reporting 28 failures, and in addition there are several loose ends:
(a) SCM export/import - I don't understand in particular how fixed values are getting loaded from SCM files
(b) the saxon:schema() API - need to take advantage of the new model to expose value constraints properly (the original subject of this bug)
(c) the changes expose an asymmetry in the way ElementDecl handle fixed values versus the way they are handled for attributes. For elements we store a comparison key fo the fixed value, for attributes we don't use this.
(d) when an element with a fixed value has no content, we are injecting the canonical lexical representation of the typed value, rather than the original lexical form. Need to check this against the spec; there may be a difference between 1.0 and 1.1.
Updated by Michael Kay about 6 years ago
Here are the XSD test suite failures:
<out xmlns:saxon="http://saxon.sf.net/"
on="2018-11-14T16:09:28.551Z"
failed="28"
queried="54">
<test set="ElemDecl"
group="valueconstraint00101m2"
name="valueConstraint00101m2"/>
<test set="ElemDecl"
group="valueconstraint00401m2"
name="valueConstraint00401m2"/>
<test set="ElemDecl"
group="valueconstraint00401m6"
name="valueConstraint00401m6"/>
<test set="ElemDecl"
group="valueconstraint00401m8"
name="valueConstraint00401m8"/>
<test set="ElemDecl"
group="valueconstraint00402m2"
name="valueConstraint00402m2"/>
<test set="ElemDecl"
group="valueconstraint00402m6"
name="valueConstraint00402m6"/>
<test set="ElemDecl"
group="valueconstraint00402m8"
name="valueConstraint00402m8"/>
<test set="ElemDecl"
group="valueconstraint00601m2"
name="valueConstraint00601m2"/>
<test set="ElemDecl"
group="valueconstraint00601m4"
name="valueConstraint00601m4"/>
<test set="ElemDecl"
group="valueconstraint00601m6"
name="valueConstraint00601m6"/>
<test set="ElemDecl"
group="valueconstraint00601m8"
name="valueConstraint00601m8"/>
<test set="MS-Attribute2006-07-15" group="attO002" name="attO002"/>
<test set="MS-Attribute2006-07-15" group="attO003" name="attO003"/>
<test set="MS-Attribute2006-07-15" group="attO005" name="attO005"/>
<test set="MS-Attribute2006-07-15" group="attP006" name="attP006"/>
<test set="MS-Element2006-07-15" group="elemD005" name="elemD005"/>
<test set="MS-Element2006-07-15" group="elemZ012" name="elemZ012"/>
<test set="MS-SimpleType2006-07-15" group="stZ070" name="stZ070"/>
<test set="ID_IDREF" group="s3_3_4si01" name="s3_3_4si01s"/>
<test set="ID_IDREF" group="s3_3_4si02" name="s3_3_4si02s"/>
<test set="ID_IDREF" group="s3_3_4si03" name="s3_3_4si03s"/>
<test set="ID_IDREF" group="s3_3_4si04" name="s3_3_4si04s"/>
<test set="ID_IDREF" group="s3_3_4si05" name="s3_3_4si05s"/>
<test set="ID_IDREF" group="s3_3_4si06" name="s3_3_4si06s"/>
<test set="ID_IDREF" group="s3_3_4si07" name="s3_3_4si07s"/>
<test set="ID_IDREF" group="s3_3_4si08" name="s3_3_4si08s"/>
<test set="ID_IDREF" group="s3_3_4si09" name="s3_3_4si09s"/>
<test set="ID_IDREF" group="s3_3_4si10" name="s3_3_4si10s"/>
</out>
Updated by Michael Kay about 6 years ago
The XSD test suite failures were caused by a simple error on an error-reporting path. Now fixed.
Now looking at the SCM issues.
(1) For an (element declaration, attribute declaration, or attribute use) with a fixed value we get
<scm:fixed lexicalForm="23">
<scm:item type="#integer" value="23"/>
</scm:fixed>
(2) For an (element declaration, attribute declaration, or attribute use) with a default value we get
default="23"
The asymmetry is unfortunate. I think that where there is a default value we should output both the lexical form and the typed value in the same way as we do for fixed values (while retaining the representation as an attribute for backwards compatibility). Note that this involves a change to the schema for SCM.
I have made this change. All XSD tests again working.
Updated by Michael Kay about 6 years ago
I have now changed the code that tests attributes against a fixed value constraint to use the same logic (and indeed the same code) as that for elements. Specifically it compares the "schemaComparable()" comparison keys rather than doing a deepEquals() call.
All XSD tests passing.
Updated by Michael Kay about 6 years ago
- Status changed from In Progress to Resolved
- Applies to branch 9.9 added
- Applies to branch deleted (
trunk) - Fix Committed on Branch 9.9 added
- Fix Committed on Branch deleted (
trunk)
I have written more tests in qt3extra / saxon-schema to test the properties of value-constraint components associated with element declarations, attribute declarations, and attribute uses.
3 XSLT3 test failures, in test sets import-schema (-048 and -182) and validation.
import-schema-048 failing with: The actual value "0.01" of attribute @discount does not match the required fixed value "0.01"
Fixed - we weren't initializing the comparison key on this path.
QT3 tests are failing with an unrelated problem in bytecode for switch expressions.
Working OK without bytecode.
Committing changes and closing the bug.
Updated by O'Neil Delpratt almost 6 years ago
- % Done changed from 0 to 100
- Fixed in Maintenance Release 9.9.1.1 added
Bug fix applied to the Saxon 9.9.1.1 maintenance release.
Updated by Michael Kay over 5 years ago
- Related to Bug #4193: EE 9.9.1.2J: Generated SCM failed on XSD validation on "Invalid content was found starting with element '{"http://ns.saxonica.com/schema-component-model":default}'. One of '{"http://ns.saxonica.com/schema-component-model":fixed}' is expected." added
Updated by O'Neil Delpratt over 5 years ago
- Fixed in Maintenance Release 9.9.1.4 added
- Fixed in Maintenance Release deleted (
9.9.1.1)
Bug fix applied in the Saxon 9.9.1.4 maintenance release.
Updated by O'Neil Delpratt about 5 years ago
- Status changed from Resolved to Closed
- Fixed in Maintenance Release 9.9.1.5 added
- Fixed in Maintenance Release deleted (
9.9.1.4)
Bug fix applied in the Saxon 9.9.1.5 maintenance release.
Please register to edit this issue