Bug #6394
closed
compileLibrary on modules which import a common module causes "Duplicate definition of global variable" exception
0%
Description
What am I doing? With a single XQueryCompiler object, I call compileLibrary on the following 3 xq library files (three calls to compileLibrary, once per library file).
xquery version "3.0" encoding "utf-8";
module namespace xmlch = "urn:activevos:avsf:xquery:utils:xmlcharutil";
declare variable $xmlch:xmlLetter := "foo";
xquery version "3.0" encoding "utf-8";
module namespace j2x = "urn:informatica:ae:xquery:json2xml";
import module namespace xmlch = "urn:activevos:avsf:xquery:utils:xmlcharutil";
xquery version "3.0" encoding "utf-8";
module namespace util = "urn:activevos:spi:functions:utilities";
import module namespace xmlch = "urn:activevos:avsf:xquery:utils:xmlcharutil";
import module namespace j2x = "urn:informatica:ae:xquery:json2xml";
What do I observe?
Exception in thread "main" net.sf.saxon.s9api.SaxonApiException: Duplicate definition of global variable xmlch:xmlLetter (see line 3 in module urn:activevos:avsf:xquery:utils:xmlcharutil)
at net.sf.saxon.s9api.XQueryCompiler.compileLibrary(XQueryCompiler.java:559)
at com.jmaurice.TestSaxonBug.main(TestSaxonBug.java:86)
Caused by: net.sf.saxon.trans.XPathException: Duplicate definition of global variable xmlch:xmlLetter (see line 3 in module urn:activevos:avsf:xquery:utils:xmlcharutil)
at net.sf.saxon.query.QueryModule.declareVariable(QueryModule.java:814)
at com.saxonica.expr.QueryLibraryImpl.link(QueryLibraryImpl.java:42)
at net.sf.saxon.query.XQueryParser.applyModuleImport(XQueryParser.java:1361)
at net.sf.saxon.query.XQueryParser.processPreamble(XQueryParser.java:969)
at net.sf.saxon.query.XQueryParser.parseLibraryModule(XQueryParser.java:460)
at com.saxonica.ee.optim.StaticQueryContextEE.compileLibrary(StaticQueryContextEE.java:160)
at com.saxonica.ee.optim.StaticQueryContextEE.compileLibrary(StaticQueryContextEE.java:204)
at net.sf.saxon.s9api.XQueryCompiler.compileLibrary(XQueryCompiler.java:557)
... 1 more
What do I expect? This should not throw an exception. A single XQueryCompiler object should be able to successfully compile these three xquery libraries with compileLibrary().
Other Details
A full repro (minus the Enterprise Edition license details) is attached as a zip, which includes the 3 xquery xq files and a Java source file for a standalone Java program used to reproduce the buggy behavior.
Found in Saxonica Enterprise Edition, release 11.5.
The bug category is either "XQuery Java API" or "Performance". It might be "Performance" because we can work around this bug by simply not precompiling the libraries with the compileLibrary call.
Note: This appears to be the same issue as: "https://saxonica.plan.io/issues/2429". I have tried with XQUERY_MULTIPLE_MODULE_IMPORTS true and false, and I have tried with various combinations and values of compiler.setBaseURI and StreamSource objects returned from the ModuleURIResolver, but I still get the same error.
PS: My company, Informatica, has developed a workaround, but it is quite complex and involves subclassing two Saxonica internal classes, com.saxonica.expr.QueryLibraryImpl and com.saxonica.ee.optim.StaticQueryContextEE.
Files
Updated by Joshua Maurice 14 days ago
PS: My company, Informatica, has developed a workaround, but it is quite complex and involves subclassing two Saxonica internal classes, com.saxonica.expr.QueryLibraryImpl and com.saxonica.ee.optim.StaticQueryContextEE.
Sorry. I meant that we have worked around the bug in such a way as to still call compileLibrary(). With our workaround, the memory usage per XQueryExecutable object is much smaller compared to simply not calling compileLibrary(). It is also possible to work around the issue by avoiding compileLibrary() altogether.
Updated by Michael Kay 14 days ago
The code here is working as designed, though (a) it's not especially well documented, and (b) we can understand why this design doesn't meet the needs of your use case.
In effect what's happening is that there's some early linking of libraries that are separately compiled within the same XQueryCompiler, on the assumption that the intent will be to use them together in the same query, and this means you get an error if the libraries are not compatible with each other. I'm going to explore (a) whether it is possible to do this differently, by not making this assumption, and (b) whether this can be done behind the existing API (i.e. is it likely to break other applications?) or whether API changes would be needed.
Updated by Michael Kay 13 days ago
Looking at it more closely I think I may have misunderstood what was happening here.
We've got some unit tests that do something remarkably close to what you are attempting, and it's not clear to me at present why our tests work but your code doesn't.
Updated by Michael Kay 13 days ago
I think the difference from our test case is that in our test, the conflict arose between different library modules imported by a main module, whereas here it arises between different library modules imported by a library module. It looks to me like there's a fix that's been applied for the first case but not for the very similar code path used in the second case. It's all about detecting whether a variable declaration reached via two different import routes is actually the same variable declaration.
Updated by Michael Kay 13 days ago
- Status changed from New to Resolved
- Assignee set to Michael Kay
- Priority changed from High to Normal
- Applies to branch 12, trunk added
- Fix Committed on Branch 11, 12, trunk added
- Platforms .NET added
Copying the relevant test (that the two variables are ultimately derived from the same declaration) appears to do the trick.
Updated by Joshua Maurice 13 days ago
Please make sure you have a test for "duplicate function definition" for the same code path. Thanks.
Please register to edit this issue
.