Bug #3188: Internal Saxon error (local variable encountered whose binding has been deleted) - Saxon - Saxonica Developer Community

Actions

Send by e-mail Copy link

Bug #3188

closed

Internal Saxon error (local variable encountered whose binding has been deleted)

Added by Chris de Vreeze over 7 years ago. Updated over 7 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Michael Kay

Category:

Internals

Sprint/Milestone:

Start date:

2017-03-31

Due date:

% Done:

100%

Estimated time:

Legacy ID:

Applies to branch:

9.7

Fix Committed on Branch:

9.7, trunk

Fixed in Maintenance Release:

9.7.0.18

Platforms:

Description

The following XPath throws an exception on Saxon-EE 9.7.0.15:

/xbrli:xbrl/xbrli:context[xbrli:scenario/xbrldi:typedMember[fn:resolve-QName(string(@dimension), .) =
  (xs:QName('ocw-dim:EquityIndividualSpecificationAxis'),
   xs:QName('ocw-dim:PropertyPlantEquipmentDepreciationSpecificationAxis'),
   xs:QName('ocw-dim:RealEstateEconomicValueSpecificationAxis'),
   xs:QName('ocw-dim:ContingentLiabilitiesSpecificationAxis'),
   xs:QName('ocw-dim:ContingentAssetsSpecificationAxis'),
   xs:QName('ocw-dim:ProvisionsIndividualSpecificationAxis'),
   xs:QName('ocw-dim:LiabilitiesNoncurrentIndividualSpecificationAxis'),
   xs:QName('ocw-dim:RelatedPartySpecificationAxis'),
   xs:QName('ocw-dim:GrantSpecificationAxis'),
   xs:QName('ocw-dim:ProjectSubsidyFinancialSpecificationAxis'))]]

For the input XML, see the attached file InstanceMinimal.xbrl. The XPath evaluator has been invoked as follows:

Processor processor = new Processor(true);

DocumentBuilder docBuilder = processor.newDocumentBuilder();
XdmItem contextItem = docBuilder.build(inputFile);

XPathCompiler xpathCompiler = processor.newXPathCompiler();
xpathCompiler.declareNamespace("fn", "http://www.w3.org/2005/xpath-functions");
xpathCompiler.declareNamespace("xbrli", "http://www.xbrl.org/2003/instance");
xpathCompiler.declareNamespace("xbrldi", "http://xbrl.org/2006/xbrldi");
xpathCompiler.declareNamespace("ocw-dim", "http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes");

XdmValue result = xpathCompiler.evaluate(xpathString, contextItem);

The error is as follows:

*** Internal Saxon error: local variable encountered whose binding has been deleted
Variable name: vv:loc2073707154
Line number of reference: 1 in 
Line number of declaration: 1 in 
DECLARATION:
<let xmlns:vv="http://saxon.sf.net/generated-variable" baseUri="" ns="xsl=http://www.w3.org/1999/XSL/Transform = xbrldi=http://xbrl.org/2006/xbrldi fn=http://www.w3.org/2005/xpath-functions ocw-dim=http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes saxon=http://saxon.sf.net/ xs=http://www.w3.org/2001/XMLSchema xbrli=http://www.xbrl.org/2003/instance" line="1" var="vv:loc2073707154" as="xs:QName+" indexable="true" slot="-999" eval="3">
 <literal count="10">
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="EquityIndividualSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="PropertyPlantEquipmentDepreciationSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="RealEstateEconomicValueSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="ContingentLiabilitiesSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="ContingentAssetsSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="ProvisionsIndividualSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="LiabilitiesNoncurrentIndividualSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="RelatedPartySpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="GrantSpecificationAxis"/>
  <qName pre="ocw-dim" uri="http://www.nltaxonomie.nl/nt11/ocw/20170301.b/dictionary/ocw-axes" loc="ProjectSubsidyFinancialSpecificationAxis"/>
 </literal>
 <filter flags="b">
  <axis name="child" nodeTest="element(Q{http://xbrl.org/2006/xbrldi}typedMember)"/>
  <gc op="=" card="M:N" comp="EQC">
   <fn name="resolve-QName">
    <fn name="string">
     <axis name="attribute" nodeTest="attribute(Q{}dimension)"/>
    </fn>
    <dot type="element(Q{http://xbrl.org/2006/xbrldi}typedMember)"/>
   </fn>
   <varRef name="vv:loc2073707154" slot="-999"/>
  </gc>
 </filter>
</let>
Exception in thread "main" java.lang.IllegalStateException: *** Internal Saxon error: local variable encountered whose binding has been deleted
	at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:777)
	at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:786)
	at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:786)
	at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:786)
	at net.sf.saxon.expr.parser.ExpressionTool.allocateSlots(ExpressionTool.java:786)
	at net.sf.saxon.sxpath.XPathEvaluator.createExpression(XPathEvaluator.java:156)
	at net.sf.saxon.s9api.XPathCompiler.internalCompile(XPathCompiler.java:506)
	at net.sf.saxon.s9api.XPathCompiler.compile(XPathCompiler.java:481)
	at net.sf.saxon.s9api.XPathCompiler.evaluate(XPathCompiler.java:529)

Strangely, if we remove a few QNames from the sequence of QNames, as in the XPath below, the error disappears. On Oracle Java 8 and IBM Java 6 we got the same results. The edited XPath:

/xbrli:xbrl/xbrli:context[xbrli:scenario/xbrldi:typedMember[fn:resolve-QName(string(@dimension), .) =
  (xs:QName('ocw-dim:EquityIndividualSpecificationAxis'),
   xs:QName('ocw-dim:PropertyPlantEquipmentDepreciationSpecificationAxis'),
   xs:QName('ocw-dim:RealEstateEconomicValueSpecificationAxis'),
   xs:QName('ocw-dim:ContingentLiabilitiesSpecificationAxis'),
   xs:QName('ocw-dim:ContingentAssetsSpecificationAxis'),
   xs:QName('ocw-dim:ProvisionsIndividualSpecificationAxis'),
   xs:QName('ocw-dim:LiabilitiesNoncurrentIndividualSpecificationAxis'),
   xs:QName('ocw-dim:ProjectSubsidyFinancialSpecificationAxis'))]]

When we rewrite the original XPath expression to the equivalent XPath below, as a workaround, the error goes away:

/xbrli:xbrl/xbrli:context[xbrli:scenario/xbrldi:typedMember[
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:EquityIndividualSpecificationAxis')) or 
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:PropertyPlantEquipmentDepreciationSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:RealEstateEconomicValueSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:ContingentLiabilitiesSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:ContingentAssetsSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:ProvisionsIndividualSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:LiabilitiesNoncurrentIndividualSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:RelatedPartySpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:GrantSpecificationAxis')) or
fn:resolve-QName(string(@dimension), .) = (xs:QName('ocw-dim:ProjectSubsidyFinancialSpecificationAxis'))]]

Files

InstanceMinimal.xbrl (1.14 KB) InstanceMinimal.xbrl

Chris de Vreeze, 2017-03-31 16:19

Actions

Copy link

Updated by Michael Kay over 7 years ago

Category set to Internals
Status changed from New to In Progress
Assignee set to Michael Kay

Thanks for reporting it. Problem reproduced as 9.7 jUnit test case s9apitest/TestXPathCompiler/testBug3188.

Actions

Copy link

Updated by Michael Kay over 7 years ago

The problem exists on the 9.7 branch but the test runs successfully on the 9.8 branch. It is probably related to the use of an optimization for the general comparison in the predicate, whereby a comparison against a sufficiently long list of constant values is turned into a hash map lookup.

It's following the (notorious!) path for optimizing filter expressions whereby it optimizes the predicate, then compares the optimized form to the original, and decides (quite wrongly in this case) that the original is better, and reinstates it. It's actually doing this on the 9.8 branch as well. (It's a particularly bad decision here, because having reverted to the original predicate thinking that it's indexable, it then discovers that it isn't after all, for reasons I haven't yet fathomed.)

A couple of asides about the optimization process:

(a) In OptimizerEE.makeMapTest, the static type of resolve-QName(string(x), .) comes out as xs:QName{0:1} where if we were a bit smarter we could infer the result will always be exactly-one QName, which would take us down a slightly different path

(b) in the same method, we're generating code that takes account of the need to match untypedAtomic values, when actually we know that the values are always QNames.

After reverting to the general comparison as the predicate, loop-lifting creates the variable, and the next stage creates an indexedLookupExpression for this variable. But remarkably, the FilterExpression optimization once again decides that the rewritten predicate is inferior to the original, and puts the original back.

Some time later (and I haven't tracked it down yet) the LetExpression that binds the generated variable is being optimized and is deciding incorrectly that the variable isn't used, and is therefore removing the binding.

Actions

Copy link

Updated by Chris de Vreeze over 7 years ago

Thanks for your quick reply (as always). For the moment we can fortunately work around the issue.

Actions

Copy link

Updated by Michael Kay over 7 years ago

Status changed from In Progress to Resolved
Priority changed from Low to Normal
Applies to branch 9.7, 9.8 added
Fix Committed on Branch 9.7, 9.8 added

I decided to drop the code in FilterExpression.optimize() that reverts to the original predicate if the new one is not indexable. The original raison d'etre for this code was to handle cases where (X=Y) was rewritten as (some $x in X satisfies $x=Y), but we no longer do that rewrite. Dropping this code makes the problem go away (though it is only an indirect cause of the problem).

On the 9.8 branch (only) I have improved the type inferencing for resolve-QName and many other functions so that if the first argument cannot be an empty sequence, then the result of the function cannot be an empty sequence. I have also improved the rewrite of a general comparison to a map contains test to take better account of untyped atomic values, NaN, etc.

Actions

Copy link