https://saxonica.plan.io/https://saxonica.plan.io/favicon.ico2019-11-28T00:14:05ZSaxonica Developer CommunitySaxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=145612019-11-28T00:14:05ZMichael Kaymike@saxonica.com
<ul><li><strong>Category</strong> set to <i>XSLT conformance</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>Assignee</strong> set to <i>Michael Kay</i></li><li><strong>Priority</strong> changed from <i>Low</i> to <i>Normal</i></li></ul><p>Problem reproduced.</p>
<p>I'm afraid the comical error message is a bit of an old chestnut. When we're comparing A and B, and get a <code>ClassCastException</code>, we report "cannot compare A to B, because that's how we detect the error in the normal course of events. Of course if the ClassCastException is caused by some internal bug then the message is nonsense.</p> Saxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=145622019-11-28T00:26:39ZMichael Kaymike@saxonica.com
<ul></ul><p>Stack trace of the exception is:</p>
<pre><code>java.lang.ClassCastException: net.sf.saxon.value.Int64Value cannot be cast to net.sf.saxon.value.UntypedAtomicValue
at net.sf.saxon.expr.sort.UntypedNumericComparer.compareAtomicValues(UntypedNumericComparer.java:183)
at net.sf.saxon.expr.sort.UntypedNumericComparer.comparesEqual(UntypedNumericComparer.java:223)
at net.sf.saxon.expr.ValueComparison.compare(ValueComparison.java:742)
at net.sf.saxon.expr.ValueComparison.effectiveBooleanValue(ValueComparison.java:703)
</code></pre>
<p>The UntypedNumericComparer is supposed to handle comparisons of xs:untypedAtomic to numerics (it tries to do this "smartly", returning false for cases like "123" = 5 without actually doing a string-to-integer conversion).</p>
<p>The question here is, why has an UntypedNumericConverter been allocated? Something must have gone wrong with the type inferencing.</p> Saxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=145632019-11-28T00:37:58ZMichael Kaymike@saxonica.com
<ul></ul><p>As often happens with artificial test cases involving literals as input, the optimizer has gone to town on this one:</p>
<pre><code>OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Replaced general comparison by value comparison
OPT : Expression after rewrite: (count($Q{}seq[. = $val])) gt 1
OPT : At line 4 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Moved function functx:non-distinct-values inline:
OPT : Expression after rewrite: let $seq := (1, 2, 3, 3) return (for $val in distinct-values($seq) return ($val[(count($seq[. = $val])) gt 1]))
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Pre-evaluated function call fn:distinct-values(...)
OPT : Expression after rewrite: (1, 2, 3)
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Replaced general comparison by value comparison
OPT : Expression after rewrite: . eq $val
OPT : At line 4 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Inlined constant variable seq
OPT : Expression after rewrite: (1, 2, 3, 3)
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Rewrite count()>=N as:
OPT : Expression after rewrite: exists(tail((1, 2, 3, 3)[. eq $val]))
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Lifted (tail((1,...)[. eq $val])) above ($val[fn:exists(...)]) on line 12
OPT : Expression after rewrite: let $Q{http://saxon.sf.net/generated-variable}v0 := tail((1, 2, 3, 3)[. eq $val]) return ($val[exists($Q{http://saxon.sf.net/generated-variable}v0)])
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Lifted (bytecode(fn:exists(...))) above ($val[$vv:v0]) on line 12
OPT : Expression after rewrite: let $Q{http://saxon.sf.net/generated-variable}v0 := bytecode(exists($Q{http://saxon.sf.net/generated-variable}v0)) return ($val[$Q{http://saxon.sf.net/generated-variable}v0])
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Created indexed filter expression:
OPT : Expression after rewrite: IndexedFilterExpression($seq, ., $val)
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Rewrite count()>=N as:
OPT : Expression after rewrite: exists(tail(IndexedFilterExpression($seq, ., $val)))
OPT : At line 12 of file:/Users/mike/bugs/2019/4399-walmsley/test.xsl
OPT : Lifted (tail($seq[(:indexed:).=$val])) above ($val[fn:exists(...)]) on line 12
OPT : Expression after rewrite: let $Q{http://saxon.sf.net/generated-variable}v0 := tail(IndexedFilterExpression($seq, ., $val)) return ($val[exists($Q{http://saxon.sf.net/generated-variable}v0)])
</code></pre>
<p>I'm mildly surprised that it hasn't done the whole computation at compile time, since there is no dependency on anything in the source document. The reason an "innocent" query like this can exercise unusual paths in the Saxon code is because real life queries can rarely be optimized quite so heavily.</p> Saxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=145642019-11-28T08:21:04ZMichael Kaymike@saxonica.com
<ul></ul><p>A peculiarity, which I don't think I've ever seen before, is that it doesn't crash when run with the -explain option.</p>
<p>Without -explain, we seem to following a slightly different path in the optimizer. I suspect that it's simply an effect of the fact that the order in which functions/templates are processed by the optimizer is fairly unpredictable.</p>
<p>Looking more closely at the non-explain path in the debugger, it is actually oprimizing more ambitiously than the -explain case. In the <code>IndexedFilterExpression($seq, ., $val)</code>, the variable $seq has has been replaced by its value (1,2,3,3), and the optimizer now tries to undo the indexing because it can see that it has become pointless. But the undoing is its undoing, to coin a phrase, because it doesn't allow for the different focus for ".". In particular, it wrongly decides that the context item type for "." is document-node(), and that it therefore needs to be atomized, and from this point everything falls apart.</p> Saxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=145652019-11-28T09:26:15ZMichael Kaymike@saxonica.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>Applies to branch</strong> <i>9.9, trunk</i> added</li><li><strong>Applies to branch</strong> deleted (<del><i>9.8</i></del>)</li><li><strong>Fix Committed on Branch</strong> <i>9.9, trunk</i> added</li></ul><p>Resolved with a patch to<code> IndexedFilterExpression</code>: on the path in <code>optimize()</code> where it decides to revert to an ordinary filter expression, it now correctly sets the context item type for the predicate of the new filter expression.</p> Saxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=145662019-11-28T09:27:34ZMichael Kaymike@saxonica.com
<ul><li><strong>Applies to branch</strong> <i>9.8</i> added</li></ul> Saxon - Bug #4399: functx:non-distinct-values says cannot compare xs:integer to xs:integer, in 9.8 EE onlyhttps://saxonica.plan.io/issues/4399?journal_id=146202019-12-06T13:48:13ZO'Neil Delprattoneil@saxonica.com
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Closed</i></li><li><strong>Fixed in Maintenance Release</strong> <i>9.9.1.6</i> added</li></ul><p>Patch committed to the Saxon 9.9.1.6 maintenance release.</p>