https://saxonica.plan.io/https://saxonica.plan.io/favicon.ico2021-10-01T07:10:28ZSaxonica Developer CommunitySaxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185662021-10-01T07:10:28ZMartin Honnenmartin.honnen@gmx.de
<ul></ul><p>If I turn off any optimization with e.g. <code>-opt:0</code> (and add <code>declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:method 'adaptive';</code>) to code gives the wanted result.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185782021-10-06T08:29:14ZMichael Kaymike@saxonica.com
<ul></ul><p>Added as a test case to EXPath-binary-extra/EXPath-binary-powerset.</p>
<p>Problem reproduced.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185792021-10-06T09:13:52ZMichael Kaymike@saxonica.com
<ul></ul><p>In FilterExpression.java, evaluating <code>$input[....]</code>, it has wrongly decided that the value of the predicate is independent of the focus.</p>
<p>When first calculated in <code>FilterExpression.typeCheck()</code>, the value of <code>filterIsIndependent' is correctly set to </code>false<code>(and</code>filterIsPositional<code>is correctly set to</code>true`).</p>
<p>But in the optimize() phase, the re-calculation of <code>filterIsPositional</code> at line 389 wrongly sets it to <code>false</code>. Though on exit from <code>optimize()</code>, <code>filterIsIndependent' is still correctly set to </code>false`.</p>
<p>But then we attempt to rewrite the FLWOR expression as a "for" or "let" expression, which re-does the <code>typeCheck()</code> on the <code>FilterExpression</code>, and this time <code>filterIsIndependent</code> becomes true.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185802021-10-06T09:20:03ZMichael Kaymike@saxonica.com
<ul></ul><p>Note that the test runs correctly on the 11 branch.</p>
<p>I suspect (with no evidence) that the difference might be that the calls on <code>bin:xx()</code> functions are now implemented as integrated extension functions rather than reflexive extension functions, which may mean that they are handling context-dependent subexpressions better.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185812021-10-06T09:26:30ZMichael Kaymike@saxonica.com
<ul></ul><p>Correction: it doesn't work on the 11 branch. It no longer crashes, but it produces wrong output. The result on SaxonJ is</p>
<pre><code>AA BB AA BB CC AA CC BB CC AA BB CC DD AA DD BB DD AA BB DD CC DD AA CC DD BB CC DD AA BB CC DD
</code></pre>
<p>and on SaxonCS we get</p>
<pre><code>A B A B C A C B C A B C D A D B D A B D C D A C D B C D A B C D
</code></pre>
<p>(which is the same sequence without the doubling)</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185822021-10-06T10:09:14ZMichael Kaymike@saxonica.com
<ul></ul><p>If I run under SaxonCS with <code>!method=adaptive</code>, the output is</p>
<pre><code>[(),"A","B",("A","B"),"C",("A","C"),("B","C"),("A","B","C"),"D",("A","D"),("B","D"),("A","B","D"),("C","D"),("A","C","D"),("B","C","D"),("A","B","C","D")]
</code></pre>
<p>which is the same as BaseX. Phew!!!</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=185832021-10-06T13:30:53ZMichael Kaymike@saxonica.com
<ul></ul><p>The difference between the J and CS behaviour on the 11 branch is purely by chance. There is a bug in <code>UTF8Writer.write(UnicodeString)</code> whereby if the <code>UnicodeString</code> is a <code>UnicodeChar</code>, then the character is written twice. This bug doesn't affect the query as I was running it in SaxonCS because the output is being sent to a writer, not a stream, so the <code>UTF8Writer</code> is not used. But the UTF8Writer is present in SaxonCS and the bug could manifest itself under different circumstances.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=186752021-10-20T11:40:28ZMichael Kaymike@saxonica.com
<ul></ul><p>Picking this one up after a gap: current status is that it works correctly on 11.x, but fails on 10.x.</p>
<p>The -explain output for the local:combination() function is</p>
<pre><code><declareFunction name="local:combination" arity="2" tailRecursive="false">
<let baseUri="file:/Users/mike/bugs/2021/5110-Honnen/test.xq"
ns="array=~ bin=http://expath.org/ns/binary err=~ fn=~ local=http://www.w3.org/2005/xquery-local-functions math=~ saxon=~ xs=~ xsi=~ xml=~"
line="6"
var="Q{}counter-bin"
slot="2"
eval="8">
<javaCall name="Q{http://expath.org/ns/binary}from-octets" arg0type="1ADI">
<varRef name="Q{}counter" slot="0"/>
</javaCall>
<filter line="7" flags="ib">
<varRef name="Q{}input" slot="1"/>
<gc line="10" op="&gt;" card="N:1" comp="CAVC">
<javaCall name="Q{http://expath.org/ns/binary}to-octets" arg0type="1A2">
<check card="1" diag="0|0||bin:to-octets">
<javaCall line="9"
name="Q{http://expath.org/ns/binary}and"
arg0type="?A2"
arg1type="?A2">
<varRef name="Q{}counter-bin" slot="2"/>
<javaCall name="Q{http://expath.org/ns/binary}shift"
arg0type="?A2"
arg1type="1ADI">
<javaCall name="Q{http://expath.org/ns/binary}hex" arg0type="1AS">
<str val="1"/>
</javaCall>
<arith line="8" op="-" calc="i-i">
<fn name="position"/>
<int val="1"/>
</arith>
</javaCall>
</javaCall>
</check>
</javaCall>
<int val="0"/>
</gc>
</filter>
</let>
</declareFunction>
</code></pre>
<p>which looks perfectly OK. And yet the query works if I set -opt:0. The only significant optimisations seem to be (a) the conversion of a FLWOR expression to a let expression, and (b) the inlining of variables <code>$bin</code> and <code>$j</code> - which looks harmless.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=186762021-10-20T11:45:03ZMichael Kaymike@saxonica.com
<ul></ul><p>Actually, the thing that's critically wrong in the -explain output is the <code>flags="i"</code> on the filter expression. This means <code>filterIsIndependent</code> - the optimizer has concluded that the value of the predicate does not depend on the focus and can therefore be evaluated in the context of the outer expression - which fails because the outer context has no context item or position. So it's the calculation of dependencies that's wrong -- as explained in comment #3.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=186772021-10-20T16:43:06ZMichael Kaymike@saxonica.com
<ul></ul><p>Getting closer, but it's tough going.</p>
<p>In the FLWORExpression</p>
<pre><code> let $j:= position() - 1,
$bin := bin:and($counter-bin, bin:shift(bin:hex('1'), $j))
return bin:to-octets($bin) > 0]
</code></pre>
<p>both the variables get inlined in turn, producing</p>
<pre><code>bin:to-octets(bin:and($counter-bin, bin:shift(bin:hex('1'), position() - 1)) ) > 0]
</code></pre>
<p>and for some reason the cached properties of the GeneralComparison <code>bin:to-octets($bin) > 0]</code> (with no focus dependencies) are retained through the rewrite, so the final expression is considered to have no focus dependency either.</p>
<p>Note that because there are two variables, we are taking a slightly different route from normal - with one variable, we would probably turn the <code>FLWORExpression</code> into a <code>LetExpression</code> before eliminating the variable. This is no doubt why the problem has never been encountered before.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=186782021-10-20T16:47:44ZMichael Kaymike@saxonica.com
<ul></ul><p>Fixed with two changes:</p>
<p>(a) in ExpressionTool.replaceSelectedSubexpressions(), change the recursive call from</p>
<pre><code>replaced = replaceSelectedSubexpressions(....)
</code></pre>
<p>to</p>
<pre><code>replaced |= replaceSelectedSubexpressions(...)
</code></pre>
<p>so that the variable <code>replaced</code> is true if any operand has been replaced, rather than just the last one.</p>
<p>(b) In <code>ExpressionTool.replaceVariableReferences()</code>, if the call on <code>replaceSelectedSubexpressions()</code> returns true, reset cached properties of the expression. For safety we do this throughout the subtree.</p>
<p>Note that although the test case didn't fail in 11.x, the code here is exactly the same and the same changes should be applied.</p>
<p>There's a secondary issue which is that these optimization rewrites aren't being traced when -explain is used.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=186792021-10-20T17:07:55ZMichael Kaymike@saxonica.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li><li><strong>Applies to branch</strong> <i>11, trunk</i> added</li><li><strong>Fix Committed on Branch</strong> <i>10, trunk</i> added</li></ul><p>Fix committed on 10.x and 11.x branches.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=188182021-11-11T18:31:27ZMichael Kaymike@saxonica.com
<ul><li><strong>Assignee</strong> set to <i>Michael Kay</i></li></ul> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=192092022-02-01T17:58:09ZO'Neil Delprattoneil@saxonica.com
<ul><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li><li><strong>Fixed in Maintenance Release</strong> <i>11.1</i> added</li><li><strong>Platforms</strong> <i>.NET, Java</i> added</li></ul><p>Bug fix applied in the Saxon 11.1 release.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=192822022-02-01T21:08:31ZO'Neil Delprattoneil@saxonica.com
<ul></ul><p>Leaving bug as resolved until fix applied to the Saxon 10 maintenance release.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=197162022-03-01T14:53:57ZDebbie Lockettdebbie@saxonica.com
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Closed</i></li><li><strong>Fixed in Maintenance Release</strong> <i>10.7</i> added</li><li><strong>Fixed in Maintenance Release</strong> deleted (<del><i>11.1</i></del>)</li></ul><p>Bug fix applied in the Saxon 10.7 maintenance release.</p> Saxon - Bug #5110: XPDY0002 The context item is absent, so position() is undefined for use of position() inside predicatehttps://saxonica.plan.io/issues/5110?journal_id=197972022-03-01T15:18:56ZDebbie Lockettdebbie@saxonica.com
<ul><li><strong>Fixed in Maintenance Release</strong> <i>11.1</i> added</li></ul>