Project

Profile

Help

How to connect?
Download (51.6 KB) Statistics
| Branch: | Revision:

he / src / userdoc / changes / changes-9.9.xml @ d9cb5c62

1
<?xml version="1.0" encoding="UTF-8"?>
2

    
3
<release number="9.9" date="2018-09-27" xmlns="http://www.saxonica.com/ns/doc/changes">
4
    
5
    <category name="documentation">
6
        <p>The documentation application now runs using Saxon-JS (XSLT 3.0 in the browser).</p>
7
        <p>The search mechanism has been updated to make it is easier to browse the pages with hits:
8
            as well as the original pop-up box (top left) showing the number of hits with up/down
9
            buttons used to move between these pages with hits, a second pop-up box (bottom left)
10
            provides a scrollable list of links to all page hits, referenced by section name.</p>
11
        <p>For ease of browsing, the option "ALL" has been added to the list of categories in the
12
            change history. This displays the changes for all categories (for the selected Saxon
13
            versions) on one page, which means you no longer need to view each category
14
            individually.</p>
15
    </category>
16
    
17
    <category name="xslt30">
18
        
19
        <p>Saxon 9.8 allowed <code>xsl:merge-source/@for-each-stream</code> as a synonym for 
20
            <code>xsl:merge-source/@for-each-source</code>, to ease transition from the XSLT 3.0 Candidate Recommendation.
21
        This synonym has now been dropped.</p>
22
        
23
        <p>The attribute <code>xsl:try/@rollback-output="no"</code> is now implemented. When evaluated in push
24
        mode, the output of the <code>xsl:try</code> is written to the result tree destination, and the
25
        <code>xsl:catch</code> takes effect only if no output has been written at the time the dynamic 
26
            error occurs.</p>
27
        
28
        <p>The XSLT 3.0 specification states:</p>
29
        
30
        <blockquote>In previous versions of this specification it was stated that when the raw result of the initial 
31
            template or function is an empty sequence, a result tree should be produced if and only if 
32
            the transformation generates no secondary results (that is, if it does not invoke xsl:result-document).... 
33
            Processor APIs offering backwards compatibility with earlier versions of XSLT must respect this behavior, 
34
            but there is no requirement for new processor APIs to do so.</blockquote>
35
        
36
        <p>Saxon 9.9 changes so that a principal result is always generated, even if it is empty, regardless whether
37
        any secondary results have been produced. This means, for example, that when running a transformation from
38
        the command line with <code>-o:dummy.xml</code>, where all output from the stylesheet is sent to specific
39
        files using <code>xsl:result-document</code>, an empty <code>dummy.xml</code> file will now be written.</p>
40
        
41
        <p>The XSLT 3.0 specification permits processors to support XPath 3.1, but also requires them to provide
42
            a mode of operation that only supports XPath 3.0. Saxon supports XPath 3.1 by default, but in the
43
            interests of conformance, a configuration option <code
44
                java="net.sf.saxon.lib.Feature#XPATH_VERSION_FOR_XSLT">Feature.XPATH_VERSION_FOR_XSLT</code> has been added;
45
            if set to the integer value "30", XPath expressions are restricted to use XPath 3.0 syntax, plus the
46
            extensions to XPath 3.0 syntax defined in the XSLT 3.0 recommendation.</p>
47
        
48
        <p>In 9.8, it was possible to export a SEF file using Saxon-EE with the options <code>-export:on
49
            -target:HE</code>, and provided the stylesheet used no Saxon-PE or Saxon-EE facilities, it could then be
50
            executed on a run-time version of Saxon with no license key - but only by downloading the Saxon-PE software.
51
            (However, see bug 3859.) This has now been rationalized: such a SEF file can now be executed using Saxon-HE alone.</p>
52
        
53
        <p>SEF files generated using Saxon 9.8 will in many cases continue to work using Saxon 9.9, but this has not
54
            been thoroughly tested, so recompiling is recommended.</p>
55
        
56
        <p>The data tables used to decide whether a character is alphanumeric, used primarily to support <code>xsl:number</code>,
57
        have been regenerated from Unicode 10.0.0 (previously Unicode 4.0.0 was used).</p>
58
        
59
        <p>In previous releases, Saxon passed all the W3C XSLT 3.0 test cases, but there were a few discrepancies on error tests where the
60
        wrong error code was generated. In 9.9, for the first time, the generated error codes in all cases match the test expectations.</p>
61
        
62
    </category>
63
    
64
    <category name="xpath31">
65
        
66
        <p>Whether in XSLT, XPath, or XQuery, previous releases of Saxon allowed sorting operations to mix <code>untypedAtomic</code>
67
        sort keys with keys of a more specific type, converting the <code>untypedAtomic</code> values to the other type if required.
68
        This has never been permitted by any of the specifications, and Saxon has now been changed to report a type
69
        error for such cases. This change is required for conformance with the W3C specifications, but it may cause
70
        existing applications to break.</p>
71
        
72
        <p>The Saxon XPath processor now supports XPath 2.0, 3.0, and 3.1, plus the version of XPath (3.0 with enhancements)
73
        defined in the XSLT 3.0 specification. Whichever version is chosen, XPath 1.0 backwards compatibility mode can be
74
        optionally enabled.</p>
75
        
76
        <p>Enhancements to the <a class="bodylink code" href="/functions/fn/transform">fn:transform()</a> function:</p>
77
        
78
        <ul>
79
           <!-- <li>The map entry <code>vendor-options?package-library</code> may be supplied to identify the locations of used
80
            packages. The value is a sequence of strings, each one being a file name, the location of a package which is available
81
            to be used in xsl:use-package declarations in the target stylesheet.</li>-->
82
            
83
            <li>The option <code>source-location</code> can be used (as an alternative to <code>source-node</code> or
84
            <code>initial-match-selection</code>) to supply the principal input to the stylesheet. This is necessary if 
85
            streamed processing is required (the W3C specification for <code>fn:transform()</code> provides no way of supplying
86
            streamed input to the transformation). The value is the URI location of the source document.</li>
87
            
88
            <!--<li>The option <code>source-text</code> can also be used (as an alternative to <code>source-node</code> or
89
                <code>initial-match-selection</code>) to supply the principal input to the stylesheet. The value is a string
90
            containing the textual content of the source document.</li>-->
91
        </ul>
92
        
93
    </category>
94
    
95
    <category name="functions31">
96
        <p>The data types <code>xs:dateTime</code>, <code>xs:time</code>, and <code>xs:duration</code>
97
        now handle values with a precision of nanoseconds (previously microsecond precision was supported).</p>
98
    </category>
99
    
100
    <category name="s9api">
101
        
102
        <p>A new method <code>setResultDocumentHandler()</code> has been added to <code java="net.sf.saxon.s9api.XsltTransformer">XsltTransformer</code>
103
            and <code java="net.sf.saxon.s9api.Xslt30Transformer">Xslt30Transformer</code>, to control the destination of the output of
104
        <code>xsl:result-document</code> instructions. The argument is a function that is supplied with
105
        a URI, and returns a <code>Destination</code> object. If the application needs to be notified when writing the <code>Destination</code> completes,
106
        it can register a callback using <code>Destination.onClose()</code>.</p>
107
        
108
        <p>At the <code>Controller</code> level, a new interface <code
109
            java="net.sf.saxon.lib.ResultDocumentResolver">ResultDocumentResolver</code> supersedes the old <code>OutputURIResolver</code>
110
        (which is still supported for the time being). This change was necessary to support the full flexibility of result document handling in XSLT 3.0,
111
        for example the ability to deliver raw output, to control sequence normalization, and the <code>item-separator</code> property.</p>
112
        
113
        <p>In previous releases an <a class="bodylink code" href="/xsl-elements/result-document">xsl:result-document</a> instruction with no <code>href</code> attribute, or with <code>href=""</code>,
114
        was treated as writing to the destination supplied (at API level) for the principal stylesheet output. From 9.9 it is treated
115
        in the same way as <code>&lt;xsl:result-document href="{$base-output-uri}"/></code> (where <code>$base-output-uri</code> is the
116
            initial value of <code>current-output-uri()</code>): this is much closer to the intent of the W3C language specification.
117
        The practical difference is that the destination for such a result document is now under the control of the <code>ResultDocumentResolver</code>
118
        (or <code>OutputURIResolver</code>) and need not be the same as the principal output destination supplied when invoking the transformation.</p>
119
        
120
        <p>A new subclass of <code>Destination</code> has been defined, called <code
121
            java="net.sf.saxon.s9api.RawDestination">RawDestination</code>: this
122
        forces <code>buildTree=no</code> so it captures the raw output of a query or stylesheet (or of
123
            <code>xsl:result-document</code>) as an <code>XdmValue</code>.</p>
124
        
125
        <p>The <code>Destination</code> interface has been extended with methods <code>onClose()</code> and <code>closeAndNotify()</code>;
126
        user-written implementations of <code>Destination</code> will need to implement these methods. This can be done easily by inheriting
127
        from the new <code>AbstractDestination</code> class, or if that is not possible, by delegating to <code>DestinationHelper</code>.</p>
128
        
129
        <p>Because destinations that do sequence normalization (including for example <code>XdmDestination</code>, <code>SAXDestination</code>,
130
        and <code>DOMDestination</code>) use the value of the <code>item-separator</code> property, Saxon now makes the serialization parameters
131
        available to the <code>Destination.getReceiver()</code> method in all cases, no longer treating the <code>Serializer</code>
132
        destination as a special case.</p>
133
        
134
        <p>In previous releases, the value of <code>xsl:result-document/@href</code> would be interpreted relative to the current working directory
135
        if no base output URI is available. This has been dropped as too dangerous (there is a tendency for result documents to be written where no-one
136
        can find them). Instead, if <code>@href</code> is a relative URI and no base output URI is available, an error is reported. However, when running from
137
        the command line, the value of the <code>-o</code> option is still interpreted as relative to the current working directory.</p>
138
        
139
        <p>When an <code>XdmDestination</code> is supplied for the result of a query, the raw results of the query
140
            are now wrapped into a document node according to the rules of "sequence normalization" in the <a class="bodylink"
141
                href="https://www.w3.org/TR/xslt-xquery-serialization-31/">Serialization specification</a>. 
142
        Previously an error was reported if the result was not a single node.</p>
143
        
144
        <p>The <code java="net.sf.saxon.s9api.SchemaValidator">SchemaValidator</code> class now has an <code>asSource()</code> method. This allows 
145
        a post-validation document to be supplied as input to any (s9api or JAXP) method expecting a <code>Source</code>
146
        as input. For example, it can be used to supply schema-validated streamed input to a transformation,
147
        or it can be used as the return value from a <code>URIResolver</code> to indicate that the result
148
            of a particular call to <code>doc()</code> or <code>document()</code> should be schema-validated.</p>
149
        
150
        <p>The <code java="net.sf.saxon.s9api.Xslt30Transformer">Xslt30Transformer</code> class now has an <code>asDocumentDestination()</code> method allowing
151
        the transformation to be used as a destination of other operations, for example a validation or
152
        another transformation. (The <code>XsltTransformer</code> class implements the <code>Destination</code>
153
        interface, providing a very convenient way of creating a pipeline of transformations. The ability
154
        to construct a <code>Destination</code> from a <code>Xslt30Transformer</code> provides the same
155
        capability for transformations that make full use of XSLT 3.0 capabilities.) Pipelines constructed
156
        in this way can be either streamed or unstreamed.</p>
157
        
158
        <aside>Note that the <code>build-tree</code> attribute on <code>xsl:output</code> and <code>xsl:result-document</code>
159
        no longer has any effect. Whether or not a tree is built (i.e. whether sequence normalization takes place) is
160
        now determined entirely by the kind of Destination supplied when the transformation is invoked.</aside>
161
        
162
        <p>The method <code
163
            java="net.sf.saxon.s9api.WhitespaceStrippingPolicy#makeCustomPolicy">WhitespaceStrippingPolicy.makeCustomPolicy()</code>
164
            has changed to use the standard Java 8 <code>Predicate</code> class in place of the Saxon version.</p>
165
        
166
        <p>The classes <code java="net.sf.saxon.s9api.XPathCompiler">XPathCompiler</code>, <code
167
            java="net.sf.saxon.s9api.XQueryCompiler">XQueryCompiler</code>, and <code
168
                java="net.sf.saxon.s9api.XsltCompiler">XsltCompiler</code> now have
169
        a method <code>setFastCompilation(true)</code> which causes the compiler to reduce the amount of time it spends
170
        optimizing the code for run-time execution speed; it may also reduce the effort devoted to ensuring good error diagnostics.
171
        This option is appropriate when compiling code that will only execute once, with a modest amount of input data,
172
        and where the code is known to be functionally correct.</p>
173
        
174
        <p>A new convenience method <code>XdmNode.getProcessor()</code> is provided to allow the <code>Processor</code>
175
            associated with a given <code java="net.sf.saxon.s9api.XdmNode">XdmNode</code> to be retrieved (for example,
176
            for the purpose of obtaining a <code>Serializer</code> or <code>XPathCompiler</code>).</p>
177
        
178
        <h2>Navigating XML Documents Using s9api</h2>
179
        
180
        <p>Saxon 9.9 introduces a major extension of the s9api interface to take advantage of the Streams functionality
181
        in Java 8. Whereas s9api has traditionally focused on mechanisms enabling Java applications to invoke XPath, XSLT,
182
        XQuery, and XSD processing, the new extensions are designed to make it easier to manipulate the inputs and outputs
183
        of these processors directly in the Java code. The starting point is that the results are generally delivered either 
184
        in the form of <code>XdmValue</code> objects or as <code>Iterator&lt;XdmItem></code> objects.</p>
185
        
186
        <p>The interface <code java="net.sf.saxon.s9api.streams.Step">net.sf.saxon.s9api.streams.Step</code> represents a function
187
            from an item to a stream of items. For example, <code>child()</code> (a static method defined in class <code
188
                java="net.sf.saxon.s9api.streams.Steps">net.sf.saxon.s9api.streams.Steps</code>) is a <code>Step</code> that delivers the children
189
            of a node. Given a node <code>N</code> and a step <code>S</code>, <code>N.select(S)</code> returns a stream of items; so
190
            <code>N.select(child())</code> returns a stream containing the children of the node.</p>
191
        
192
        <p>Combining a step with a predicate gives a new step: for example <code>child.where(isElement())</code> is a step that selects
193
            the element children of a node; so <code>N.select(child().where(isElement()))</code> returns the element children of <code>N</code>
194
            as a stream. The static method <code>isElement()</code> used here returns a predicate defined in the utility class
195
            <code java="net.sf.saxon.s9api.streams.Predicates">net.sf.saxon.s9api.streams.Predicates</code>,
196
            but of course any <code>Predicate</code> can be used in its place.</p>
197
        
198
        <p>Commonly-used filters can be expressed in abbreviated form. For example the first title element in a book can be obtained by
199
        writing <code>book.select(descendant("title").first())</code>.</p>
200
        
201
        <p>Steps can be composed using the <code>then()</code> method, so <code>N.select(child().then(child()))</code> selects all the grandchildren of a node.</p>
202
        
203
        <p>The resulting stream of nodes can be processed using standard Java 8 techniques, for example it can be collected into a list:
204
            <code>N.select(child().then(atomize())).collect(Collectors.toList())</code> atomizes the values of the child nodes and returns the result
205
            as a Java List whose members are <code>XdmAtomicValue</code> instances.</p>
206
        
207
        <p>The class <code>XdmValue</code> has a constructor accepting a <code>Stream</code>, so for example <code>new XdmValue(N.select(child()))</code>
208
        delivers an <code>XdmValue</code> containing the children of a supplied node <code>N</code>.</p>
209
        
210
        <p>The Java expressions that can be assembled using these primitives are very close to XPath in their expressive power. Although Saxon will
211
        not be able to optimize them in the same way as it can optimize XPath expressions (for example, a filter will always literally work by
212
        testing each item in a sequence to see if it matches), there is a considerable performance benefit over XPath in that the Java compiler
213
        does all the hard work of parsing; it will also detect many compile-time errors that would not be detected until execution time with XPath.</p>
214
        
215
        <p>In addition there are some new convenience methods such as <code>XdmNode.attribute("xxx")</code> which gets the value of an attribute given
216
            only its local name, <code>XdmNode.children()</code> which gets all the children of a node, and <code>XdmNode.children("xxx")</code> which
217
        gets all the child elements with local name <code>"xxx"</code>.</p>
218
        
219
    </category>
220
    
221
    <category name="spi">
222
        
223
        <p>The venerable and overgrown <code java="net.sf.saxon.Controller">Controller</code> class has been split into two: all
224
            functionality that applies to XSLT only has been moved into a subclass, <code
225
                java="net.sf.saxon.trans.XsltController">XsltController</code>. The <code>transform()</code>
226
        method (which at one time implemented the JAXP <code>Transformer.transform()</code> interface) has
227
        been dropped; instead the specific entry points such as <code>XsltController.applyTemplates()</code>
228
        and <code>XsltController.callTemplate()</code> must be used. These methods in turn have been changed
229
        so that they now deliver the raw result of the transformation (without the sequence normalization
230
        process that wraps the result into a document node). Any wrapping that is required is now performed
231
        in the s9api API layer (classes <code java="net.sf.saxon.s9api.XsltTransformer">XsltTransformer</code> and <code
232
            java="net.sf.saxon.s9api.Xslt30Transformer">Xslt30Transformer</code>), and
233
        is specific to the type of <code>Destination</code> supplied.</p>
234
        
235
        <p>The <code java="net.sf.saxon.Configuration">Configuration</code> class no longer has a <code>finalize()</code> method
236
            (because this is deprecated in Java 9). The only action of the <code>finalize()</code> method was to ensure that any
237
            log file was closed; an application that supplies a logger that needs closing is now responsible for ensuring
238
        that it is closed after use, which can be achieved if convenient by calling <code>Configuration.close()</code>.</p>
239
        
240
        <p>A number of key interfaces such as <code java="net.sf.saxon.om.Sequence">Sequence</code>, 
241
            <code  java="net.sf.saxon.om.SequenceIterator">SequenceIterator</code>, <code java="net.sf.saxon.event.Receiver">Receiver</code>, and
242
            <code java="net.sf.saxon.om.NodeInfo">NodeInfo</code> now have default implementations of some of their methods,
243
        making it easier to define new implementations of these interfaces. In some cases this has led to removal of the static
244
        helper methods that previously were used to avoid code duplication.</p>
245
        
246
    </category>
247
    
248
    <category name="extensibility">
249
        <p>An item representing an external Java object (for example, the result of calling <code>new()</code>
250
        in a reflexive extension function call) may now be treated as a map. There is one entry in the map for each
251
        public non-static method defined on the object (either directly, or inherited), provided that its name is
252
        unique. If there are several methods with the same name but different arity, then they are available
253
        under suffixed names such as "action_0" and "action_1". But if there are several methods with the same arity,
254
        this group of methods is not included in the map. The key of the entry is the method name, while the
255
        value is a function item that may be called using dynamic function call syntax to invoke the method.</p>
256
        
257
        <p>The conversion of the Java object to a map may be done explicitly or implicitly.</p>
258
        <ul>
259
            <li>It is done explicitly by calling the new extension function
260
                <a class="bodylink code" href="/functions/saxon/object-map">saxon:object-map()</a> which takes an external object as
261
                its argument, and returns the constructed map.</li>
262
            <li>It is done implicitly when an external object is used as the left-hand operand of the lookup operator "?".</li>
263
        </ul>
264

    
265
        <p>For example, the extension function <code>sql:connect</code> returns an external Java object of
266
        class <code>java.sql.Connection</code>. If <code>$connection</code> holds the result of this function,
267
        then <code>$connection?isClosed()</code> calls the <code>isClosed</code> method on the underlying object. The
268
        <code>rollback</code> method of this class has two variants; the arity-0 version may be invoked as
269
        <code>$connection?rollback_0()</code>.</p>
270
        
271
        <p>The conversion of XDM atomic values to Java values (when passing parameters to extension
272
        functions) now handles conversion of the XSD date/time types (<code>xs:dateTime</code>, etc)
273
        to the new Java 8 types <code>java.time.ZonedDateTime</code> and <code>java.time.Instant</code>.</p>
274

    
275
    </category>
276
    
277
    <category name="optimization">
278
        
279
        <p>Arrays have been re-implemented to use a "persistent" immutable data structure (based on
280
        the PCollections Java library). This means that operations such as <code>array:put()</code> no
281
        longer make a copy of the entire array.</p>
282
        
283
        <p>As an example of the performance savings, using <code>array:put()</code> to modify every item of an array takes the
284
            following times in microseconds:</p>
285
            
286
            <table>
287
                <thead>
288
                    <tr>
289
                        <th>No of members</th>
290
                        <th>Saxon 9.8 (microsecs)</th>
291
                        <th>Saxon 9.9 (microsecs)</th>
292
                    </tr>
293
                </thead>
294
                <tbody>
295
                    <tr>
296
                        <td>100</td>
297
                        <td>10</td>
298
                        <td>24</td>
299
                    </tr>
300
                    <tr>
301
                        <td>1000</td>
302
                        <td>1149</td>
303
                        <td>475</td>
304
                    </tr>
305
                    <tr>
306
                        <td>10000</td>
307
                        <td>113531</td>
308
                        <td>6680</td>
309
                    </tr>
310
                </tbody>
311
            </table>
312
            
313
 
314
        
315
        
316
       
317
        
318
        <p>Type checking has been improved: the main impact is to give better diagnostics, but there will also
319
        be cases where performance is improved. The main changes are:</p>
320
        
321
        <ul>
322
            <li>Static type checking of returned item types is now distributed to all operands of an expression that
323
            contribute directly to the returned result (operands with usage Transmission in terms of the XSLT 3.0
324
            streamability analysis). For example, if a template is required to return a sequence of integers, then
325
            every instruction in the template that contributes directly to the result is checked to ensure that it
326
            returns integers. This means that more errors are detected statically, and less time is spent at run-time checking
327
            the results of subexpressions that are statically known to be correct. It also enables more precise diagnostics as to
328
            which instruction or subexpression generated invalid results; this is particularly useful when functions have been
329
            inlined.</li>
330
            
331
            <li>Diagnostics for type errors now give more precise information, particularly for functions, maps, and arrays.
332
            When a supplied value V does not match the required type R, then instead of reporting that V is of type S, the message
333
            reports why V does not match the required type R. For example if R is <code>map(xs:string, item()*)</code> and
334
            V contains an entry whose key is of type <code>xs:untypedAtomic</code>, the error message in in 9.8 would
335
            report that V was of type <code>map(xs:anyAtomicType, item()*)</code>, whereas 9.9 will give a message in the form
336
                <em>"The map contains an entry with key ("abc") of type xs:untypedAtomic that is not an instance of the
337
                    required type xs:string"</em>.</li>
338
            
339
        </ul>
340
        
341
        <p>The lookup operator "?" has been re-implemented as a native expression (previously it was compiled into a complex
342
        expression involving loops and conditionals). As a result it should now be faster, but more particularly, it now
343
        benefits from improved type checking, and any run-time type errors are now much clearer because they relate to the
344
        expression as written, and not to some internally-generated code.</p>
345
        
346
        <p>The <code>map:merge()</code> function has been re-implemented to avoid quadratic performance when a large number of maps
347
        containing duplicate keys are combined using the <code>duplicates:combine</code> option. </p>
348
        
349
        <p>In Saxon-EE, regular expressions used in functions such as <code>matches()</code>, <code>replace()</code>,
350
        <code>tokenize()</code>, and <code>analyze-string()</code> are now cached so that when the same regular expression is used
351
        repeatedly, the compiled form of the regex is reused. (This is only relevant where the regular expression is not known statically:
352
        regular expressions that are known at query/stylesheet compile time are always compiled at compile time.) The cache is
353
        held at the level of a <code>Configuration</code>. The optimization can be suppressed using <code>opt:-e</code>.</p>
354
        
355
        <p>Saxon by default checks axis steps at compile time to ensure they are capable of returning a non-empty result.
356
        "Void" steps such as <code>comment()/*</code> result in a compile-time warning. This check appears to be quite
357
        expensive in some use cases, so an optimization flag (<code>opt:-d</code>) has been introduced to suppress it.</p>
358
        
359
        <p>A new optimization has been introduced to reduce the cost of executing simple path expressions like
360
        <code>books/book/price</code>. Specifically, if the right-hand operand of "/" is a simple axis step, Saxon
361
        now evaluates it without creating a new dynamic context object, and without tracking the context item and position,
362
        because they are never needed in this situation. This appears to give a speed-up of around 10% for such
363
        path expressions.</p>
364
        
365
        <h2>Changes to the TinyTree data structure</h2>
366
        
367
        <p>Two significant changes have been implemented for the TinyTree:</p>
368
        
369
        <ul>
370
        <li><p>Where an element has a single text node child (and no attributes or namespace declarations), the
371
            element and text nodes are now combined in a single entry in the main node array (called a TextualElement).
372
            For documents that have many such simple-content elements, this can reduce the space overhead (over and above
373
            the space used by character content) by up to 50%. (Note: further reductions can be achieved if whitespace
374
            is stripped.) As a side-effect, searching for elements using the child or descendant axes is fractionally faster
375
            because there is no need to skip the entries containing text nodes.</p></li>
376
            
377
        <li><p>It is now possible for a TinyTree to contain element nodes created as virtual copies of nodes in another external tree
378
        (typically but not necessarily another TinyTree). These are referred to as "grafted" nodes. They arise when a tree is constructed
379
        by copying parts of another tree. For example, given:</p>
380
        
381
            <samp><![CDATA[<xsl:variable name="sorted">
382
   <temp>
383
     <xsl:copy-of select="sort(//employee, function($e){number($e!salary)})"/>
384
   </temp>
385
</xsl:variable>]]></samp>
386
            
387
        <p>a new tree will be constructed that consists largely of grafted nodes containing references to the <code>employee</code>
388
        elements in the original unsorted input tree. This saves the cost in time and space of copying the employee subtrees from
389
        the old tree to the new. Saxon ensures that the nodes behave like copies even though they are not physically copied: for example,
390
        <code>generate-id()</code> will return a different value from the original, and navigating down into the subtree and then back up
391
        again will find the correct parent node.</p>    
392
        </li>
393
            
394
        </ul>    
395
        
396
    </category>
397

    
398
    <category name="installation">
399
        
400
        <p>For efficiency, Configuration options can now be written and read using a new set of constants: for example <code
401
            java="net.sf.saxon.lib.Feature#ALLOW_EXTERNAL_FUNCTIONS">Feature.ALLOW_EXTERNAL_FUNCTIONS</code> in place of
402
            <code>FeatureKeys.ALLOW_EXTERNAL_FUNCTIONS</code>. This avoids the expensive lookup using long string values. The old
403
            mechanism remains available but is deprecated.</p>
404
        
405
        <p>If the configuration option <code
406
            java="net.sf.saxon.lib.Feature#ALLOW_EXTERNAL_FUNCTIONS">Feature.ALLOW_EXTERNAL_FUNCTIONS</code> is set to false,
407
        environment variables are no longer visible to the functions <code>fn:available-environment-variables</code>
408
        and <code>fn:environment-variable</code> (the behaviour is as if the environment variable did not exist).
409
        This brings these functions into line with the <code>fn:system-property</code> function.</p>
410
        
411

    
412
        
413
    </category>
414

    
415
    <category name="serialization">
416
        
417
        <p>The serialization property <code>saxon:attribute-order</code> has been extended so that it may now include
418
        the token "*" to match unspecified attributes; this allows selected attributes to be forced to the end of the list.
419
        For example <code>saxon:attribute-order="a b c * xml:space"</code> will cause attributes to be output in the order
420
        <code>a</code>, then <code>b</code>, then <code>c</code>, then everything else (sorted by URI and local name), 
421
            then <code>xml:space</code>. Further details at <a class="bodylink"
422
                href="/extensions/output-extras/serialization-parameters">Serialization parameters</a>.</p>
423
        
424
        <p>The algorithm for indenting XML has been improved so that existing whitespace appearing
425
        in a position where indenting takes place (before a start tag or after an end tag) is replaced
426
        by the new whitespace, rather than being retained.</p>
427
        
428
        <p>The serialization property <code>suppress-indentation</code> is now implemented for the
429
        HTML and XHTML output methods.</p>
430
        
431
        <p>A new serialization property <code>saxon:property-order</code> controls the order of properties
432
            within a serialized map when using the JSON output method. It follows the same principles as
433
        <code>saxon:attribute-order</code>. In this case the value is a space-separated list of tokens, with "*"
434
            treated specially. Further details at <a class="bodylink"
435
                href="/extensions/output-extras/serialization-parameters">Serialization parameters</a>.</p>
436
        
437
        <p>With indented JSON output, the output is now more compact:</p>
438
            <ol>
439
                <li>The <code>saxon:indent-spaces</code> property now controls the amount of indentation (default is two spaces).</li>
440
                <li>The serializer avoids outputting a new line before or after a map or array containing less than two entries.</li>
441
                <li>The serializer output arrays and objects on a single line if none of the entries
442
                    is an array or map, and the total size of the entries is less than L, where L is the value of <code>saxon:line-length</code>
443
                    if present, or 80 if not specified.</li>
444
            </ol>
445

    
446
        <p>In JSON serialized output, integers (specifically, <code>xs:integer</code> values, and any other numeric values that are equal to a whole number
447
            and have absolute value less than 10^18) are now output as integers; exponential notation is no longer used in such cases.</p>
448

    
449
        <p>A new serialization property <code>saxon:canonical</code> is available (values <code>yes</code> or <code>no</code>). Used
450
        with the output method XML, the value <code>yes</code> requests output conforming to the W3C XML Canonicalization 1.1
451
        specification (C14N). This can be useful when comparing test results, or when documents need to be digitally signed.</p>
452
        
453
        <p>The restrictions on the use of <code>disable-output-escaping</code> in conjunction with <code>xsl:try</code> and
454
        <code>xsl:fork</code> have been lifted. These instructions need to buffer their output, rather than writing it straight
455
        to the serializer; the buffer is now held in the form of a list of push events, which allows the <code>disable-output-escaping</code>
456
        property to be retained along with the event.</p>
457
        
458

    
459
        
460
    </category>
461
    
462
    <category name="extensions">
463
        <h2>Extension Functions</h2>
464
        
465
        <p>For the extension functions <a class="bodylink code" href="/functions/saxon/highest">saxon:highest</a> and <a class="bodylink code"
466
            href="/functions/saxon/lowest">saxon:lowest</a>,
467
            the function that computes the key may now return an empty sequence, in which case the relevant item from the input is excluded
468
            from consideration (it will not feature in the result). The implementation has been made more robust in edge cases, for example
469
            where the key returns a value of an unordered type, or where the set of keys are not mutually comparable, or where they include NaN values.</p>
470
        
471
        <p>A new function <a class="bodylink code" href="/functions/saxon/map-search">saxon:map-search</a> is introduced. It is
472
            similar to <code>map:find</code>, in that it searches recursively through nested maps and arrays, but is more
473
            versatile. It differs from <code>map:find</code> in two main respects:</p>
474
        
475
        <ol>
476
            <li>Whereas <code>map:find</code> only takes as input the key value to look for, <code>saxon:map-search</code> also
477
                takes a function that can be used to test the associated value.
478
            </li>
479
            <li>Whereas <code>map:find</code> returns the value associated with the specified key, <code>saxon:map-search</code>
480
                returns much more information about where the match was found.</li>
481
        </ol>
482
        
483
        <p>The new function <a class="bodylink code" href="/functions/saxon/with-pedigree">saxon:with-pedigree()</a> takes a map or array as input (typically, the result of parsing JSON),
484
            and returns an equivalent map or array whose implementation has special behavior: any downward selections
485
            of nested maps/arrays using subscripting or key lookup return a map/array that has "pedigree" information;
486
            the pedigree information tells you how the map/array was selected within its containing tree, and enables
487
            you to retrace your steps. If a map or array has pedigree information, the <a  class="bodylink code" href="/functions/saxon/pedigree">saxon:pedigree()</a> function returns
488
            a map, with an entry "container" identifying the containing map or array, and an entry "key" or "index"
489
            identifying the key value in the containing map, or the index value in the containing array.</p>
490
        
491
        <p>The new (higher-order) extension function <a class="bodylink code"
492
            href="/functions/saxon/group-starting">saxon:group-starting</a> splits a sequence into groups, based on a starting
493
            condition for each group. For example, the function call <code>saxon:group-starting(following-sibling::*, fn{boolean(self::h1)})[1]?*</code> 
494
            selects the following-sibling elements of the context item up to (and excluding) the first <code>h1</code>
495
            element.</p>
496
        
497
        <p>The new extension function <a class="bodylink code" href="/functions/saxon/object-map">saxon:object-map</a>
498
            converts a Java object to an XDM map. For more information, see the "Extensibility mechanisms" changes
499
            category.</p>
500
        
501
        <h2>Extension Attributes</h2>
502
        
503
        <p>An extension attribute <a class="bodylink code" href="/extensions/attributes/options">saxon:options</a> has
504
            been added to <a class="bodylink code" href="/xsl-elements/evaluate">xsl:evaluate</a>: the value is an expression,
505
            which must evaluate to a map. The map allows additional Saxon-specific options for
506
            expression evaluation to be defined. Initially two options are defined:</p>
507
        
508
        <ul>
509
            <li>
510
                <p><b>allow-any-namespace</b></p>
511
                <p>If this option is set to the value <code>true()</code>, the XPath expression may reference schema components that
512
                    are present in the Saxon <code>Configuration</code> whether or not they have been imported into the stylesheet
513
                    using <code>xsl:import-schema</code>.</p>
514
            </li>
515
            <li>
516
                <p><b>default-collation</b></p>
517
                <p>The value of this option is a collation URI. This allows the default collation for evaluating the target
518
                    expression to be selected dynamically (standard XSLT 3.0 facilities only allow it to be selected statically).</p>
519
            </li>
520
            
521
        </ul>
522
        
523
        
524
        <p>An <a class="bodylink code" href="/xsl-elements/accumulator-rule">accumulator rule</a> for a 
525
            streamable accumulator that specifies <code>phase="end"</code> may now also specify
526
            <a class="bodylink code" href="/extensions/attributes/capture">saxon:capture="yes"</a>. This relaxes the rule
527
            that the accumulator evaluation must be motionless; instead it can now be consuming. When the "end-element" event for
528
            the accumulator rule is reached, the rule now has access to a snapshot of the entire element content (made using the
529
            <code>fn:snapshot</code> function). To avoid the snapshots become over-large, this is best used on elements that have
530
            minimal content; in such cases it can make programming an accumulator significantly easier.</p>
531
        
532
        
533
        <p>Elements that have an <code>as</code> attribute to define a required type (for example, <code>xsl:variable</code>,
534
            <code>xsl:param</code>, and <code>xsl:function</code>) now additionally recognize a <a
535
                class="bodylink code" href="/extensions/attributes/as">saxon:as</a> attribute
536
            that can be used to provide additional type information using Saxon extension syntax. For example, if the <code>as</code>
537
            attribute specifies <code>as="map(xs:string, xs:double)"</code> for a map used to represent a complex number, then
538
            the additional attribute <code>saxon:as="tuple(i: xs:double, r: xs:double)"</code> may be used to indicate the required
539
            keys that must be present in the map. Similarly, if a function returns either a date or dateTime and indicates this
540
            using <code>as="xs:anyAtomicType"</code>, the return type may be more precisely described using the syntax
541
            <code>saxon:as="union(xs:date, xs:dateTime)"</code>. This allows Saxon extended types to
542
            be used without sacrificing stylesheet portability.</p>
543
        
544
        <h2>Extension Instructions</h2>
545
        
546
        <p>A new XSLT extension instruction <a class="bodylink code" href="/extensions/instructions/do">saxon:do</a> is provided:
547
            it is designed for use when invoking 
548
            XPath expressions that have side-effects. For example, <code><![CDATA[<saxon:do action="$connection?close()"/>]]></code>
549
            calls the function <code>$connection?close()</code>, while <code><![CDATA[<saxon:do action="saxon:discard-document($doc)"/>]]></code> 
550
            calls <code>saxon:discard-document()</code>. Any result of the expression is discarded. The instruction is useful
551
            in preference to <code>xsl:sequence</code> firstly because the result of the expression is discarded, and secondly
552
            because it is recognized by the optimizer as signaling an expression that is being evaluated for its side-effects.
553
            The optimizer will attempt to avoid aggressive optimizations such as lifting the instruction
554
            out of an <code>xsl:for-each</code> loop. Note however that this does not propagate; just
555
            because a function <code>F</code> calls <code>saxon:do</code>, for example, does not prevent calls on <code>F</code> being moved
556
            out of a loop.</p>
557
        
558
        <p>The extension XSLT instruction <a class="bodylink code"
559
            href="/extensions/instructions/tabulate-maps">saxon:tabulate-maps</a> takes selected maps in a 
560
            structure containing maps and arrays, such as might be derived
561
            by parsing JSON input, and returns these maps, expanded with information taken from containing
562
            maps and arrays.</p>
563
        
564
        <p>The <a class="bodylink code" href="/extensions/instructions/deep-update">saxon:deep-update</a> instruction
565
            creates a new map or array that is the same as an existing
566
            map or array except for modified content that may be several layers deep within nested
567
            maps and arrays. Typically (but not necessarily) the map or array to which it is applied will have been constructed
568
            by parsing JSON content using the <a class="bodylink code" href="/functions/fn/parse-json">parse-json()</a> function.</p>
569
        
570
        <p>As a result of the continuing popularity of the DocBook 1.0 stylesheets, which rely on the <code>saxon:output</code>
571
            extension element present in Saxon 6.5.5 and earlier releases, the extension instruction
572
            <a class="bodylink code" href="/extensions/instructions/output">saxon6:output</a>
573
            (in namespace <code>xmlns:saxon6="http://icl.com/saxon"</code>) has been reinstated. It is a synonym for 
574
            <code>xsl:result-document</code>. This enables the DocBook 1.0 stylesheets to run without error using
575
            Saxon-PE/EE 9.9.</p>
576
        
577
        <p><b>XSLT Update Extension</b> (<em>Requires Saxon-EE</em>). New extension instructions
578
            have been introduced to provide an XSLT equivalent of the XQuery Update
579
            capability. For example, you can delete selected attributes using the construct:</p>
580
        
581
        <samp><![CDATA[<saxon:update select="//chapter[1]">
582
    <saxon:delete select=".//para[@deleted='yes']"/>
583
</saxon:update>]]></samp>
584
        
585
        <p>The outermost instruction is <a class="bodylink code"
586
            href="/extensions/instructions/update">saxon:update</a>. This has a <code>select</code> attribute to
587
            select the nodes whose subtrees will be updated. The default is <code>select="."</code>. The result of the
588
            instruction is a sequence of nodes that are modified copies of the selected nodes (unless any of these nodes
589
            are deleted, in which case they are omitted from the result). The original tree remains unchanged.</p>
590
        
591
        <p>Within the <code>saxon:update</code> instruction, the following subsidiary instructions may be evaluated: 
592
            <code>saxon:change</code>, <code>saxon:delete</code>, <code>saxon:insert</code>, <code>saxon:rename</code>, <code>saxon:replace</code>.</p>
593
        
594
        <p>The update extension for XSLT does not provide any equivalent to XQuery updating functions; all update instructions must be contained lexically
595
            within the <code>saxon:update</code> instruction.</p>
596
        
597
        
598
        <h2>XPath Syntax Extensions</h2>
599
        
600
        <p>Saxon 9.9 implements a number of extensions to the XPath 3.1 grammar, for more information see <a class="bodylink"
601
            href="/extensions/syntax-extensions">Syntax Extensions</a>. These are enabled only if the configuration option
602
            <code java="net.sf.saxon.lib.Feature#ALLOW_SYNTAX_EXTENSIONS">Feature.ALLOW_XPATH_SYNTAX_EXTENSIONS</code>
603
            is set, and they require Saxon-PE or higher.</p>
604
        
605
        <p>Some of these syntax extensions were issued experimentally in Saxon 9.8, and the detailed rules have been refined in the light of 
606
        experience.</p>
607
        
608
        <p>The tuple type syntax, for example <code>tuple(i: xs:double, r: xs:double)</code>, now allows a "?" after the field name to
609
        indicate that the field may be absent from the map, and allows a final ", *" to indicate that extra undeclared entries are permitted
610
        in the map.</p>
611
        
612
        <p><b>andAlso</b> and <b>orElse</b> operators. These operators (often called short-circuit operators) 
613
            have the same effect as <code>and</code> and <code>or</code>,
614
            but they are guaranteed to evaluate the left-hand operand before the right-hand operand, and to avoid evaluating the
615
            right-hand operand when it is not needed. For example <code>if ($x castable as xs:date andAlso xs:date($x) > current-date())...</code>
616
            ensures that no failure occurs when <code>$x</code> is not a valid date. (The XPath specification offers no such guarantee
617
            for the <code>and</code> and <code>or</code> operators. Saxon will occasionally rearrange the order of evaluation, as permitted
618
            by the specification, if it thinks this will improve performance.)</p>
619
        
620
        
621
        <h2>XSLT syntax extensions</h2>
622
        
623
        <p>The syntax of match patterns is extended to make it easier to match maps and arrays (see
624
            <a class="bodylink" href="/extensions/xslt-syntax-extensions">XSLT Syntax Extensions</a>).</p>
625
        
626
        <p>In particular the syntax <code>ma-type predicate*</code> is allowed, where <code>ma-type</code>
627
            is any of:</p>
628
        
629
        <ol>
630
            <li>~type-alias</li>
631
            <li>tuple(...)</li>
632
            <li>map(...)</li>
633
            <li>array(...)</li>
634
        </ol>
635
       
636
        <p>For example if a type alias has been declared:</p>
637
        
638
        <samp>&lt;saxon:type-alias name="cx:complex" type="tuple(r: xs:double, i: xs:double)"/></samp>
639
        
640
        <p>Then it can be used in a match pattern to match instances of the type, with or without predicates:</p>
641
        
642
        <samp>&lt;xsl:template match="~cx:complex[?i=0]">{?r}&lt;/xsl:template></samp>
643
        
644
        <samp>&lt;xsl:template match="~cx:complex">{?r}{if (?i ge 0) then '+' else ''}{?i}i&lt;/xsl:template></samp>
645
        
646
        <p>The construct <code>~T</code> at the start of a pattern can be regarded as an abbreviation for
647
        <code>.[. instance of ~T]</code>.</p>
648
        
649

    
650
    
651
    </category>
652
    
653
    <category name="sql">
654
        <p>A number of SQL extension functions are now available, designed to supersede the existing SQL extension
655
            instructions. See the <a class="bodylink" href="/sql-extension">Saxon SQL Extension</a> section and <a
656
                class="bodylink" href="/functions/sql">Function Library</a> for details.</p>
657
    </category>
658
    
659
    <category name="sourcedocs">
660
        
661
        <p>A number of interfaces now recognize the "data" URI scheme, defined in RFC 2397. This allows the actual content of a resource to
662
        be contained directly in the URI, using either %HH or base64 encoding.</p>
663
        
664
        <p>Specifically, "data" URIs are recognized:</p>
665
        
666
        <ul>
667
            <li>By the StandardURIResolver, allowing them to be used by the <code>doc()</code> and <code>document()</code> functions (as well as
668
            other less useful places, like <code>xsl:include</code> and <code>xsl:import</code>)</li>
669
            <li>By the <code>StandardUnparsedTextResolver</code>, allowing use in <code>unparsed-text()</code> and <code>unparsed-text-lines()</code></li>
670
            <li>In a collection catalog, allowing use in the <code>collection()</code> function.</li>
671
        </ul>
672
        
673
        <p>Applications using user-written URI resolvers or collection finders can take advantage of the same features by reference to the class
674
        <code>DataURIScheme</code>, which converts a Base64 "data" URI into a <code>BinaryResource</code>, and any other "data" URI into an
675
        <code>UnparsedTextResource</code>.</p>
676
    </category>
677
    
678
    <category name="xsd11">
679
        
680
        <p>The XSD 1.1 specification mandates support for XPath 2.0 in XPath expressions such as assertions.
681
            Saxon enforces conformance to XPath 2.0 by default, but also allows support for later versions
682
            to be configured. This can be achieved by setting the configuration option <code
683
                java="net.sf.saxon.lib.Feature#XPATH_VERSION_FOR_XSD">Feature.XPATH_VERSION_FOR_XSD</code>,
684
            for example setting the option to the integer value 31 enables support for XPath 3.1. (It is also possible,
685
        as in previous releases, to set the attribute <code>saxon:extensions="any-xpath-version"</code> on the
686
        <code>xs:schema</code> element.)</p>
687
        
688
        
689
    </category>
690
    
691
    <category name="xquery-update">
692
        
693
        <p>It is now possible to use the copy-modify expression without explicitly configuring Saxon to use a mutable tree model such as
694
        the linked tree. The linked tree will automatically be used for the temporary tree constructed by the copy-modify
695
        expression.</p>
696
        
697
        <p>The <a class="bodylink code" href="/functions/fn/put">fn:put()</a> function now takes account of serialization parameters set within the query body using declarations such
698
        as <code>declare option output:indent "yes".</code> (It does not, however, take account of serialization parameters set on the command
699
        line.)</p>
700
    </category>
701
    
702
    <category name="xquery31">
703
        
704
        <p>The pull-mode implementation of window clauses in FLWOR expressions has been rewritten, reducing memory requirements
705
        when performing windowing over a long sequence of items.</p>
706
        
707
        <p>In previous releases, Saxon passed all the W3C XQuery 3.1 test cases, but there were a few discrepancies on error tests where the
708
            wrong error code was generated. In 9.9, for the first time, the generated error codes in all cases match the test expectations.</p>
709
    </category>
710

    
711

    
712

    
713
</release>
(6-6/6)