Bug #2429
closedcompileLibrary used several times ends with "Duplicate definition of global variable"
0%
Description
When I'm using compileLibrary several times on the same instance of XQueryCompiler compiler I'm getting following error:
net.sf.saxon.trans.XPathException: Duplicate definition of global variable moduleName:someVariableName (see line 12 in module file:/somePath/someModule.xq)
at net.sf.saxon.query.QueryParser.grumble(QueryParser.java:459)
at net.sf.saxon.query.QueryParser.parseVariableDeclaration(QueryParser.java:1621)
at net.sf.saxon.query.QueryParser.parseProlog(QueryParser.java:739)
at net.sf.saxon.query.QueryParser.parseLibraryModule(QueryParser.java:425)
at com.saxonica.expr.ee.StaticQueryContextEE.compileLibrary(StaticQueryContextEE.java:72)
at com.saxonica.expr.ee.StaticQueryContextEE.compileLibrary(StaticQueryContextEE.java:142)
at net.sf.saxon.s9api.XQueryCompiler.compileLibrary(XQueryCompiler.java:474)
It's connected to fact that different modules I'm compliling are using the same imports.
Issue affects not only variables but also functions.
Files
Updated by Michael Kay over 9 years ago
Any chance you could supply a repro, so we know exactly what you are trying to do? (I could try to create one myself, but I might well find quite different problems...)
There's a lot of fuzziness in the spec in this area, so the outcome might turn out to be clarification of the documentation to explain what the restrictions are. But I'll investigate first before coming to that conclusion.
Updated by Mariusz B over 9 years ago
Michael Kay wrote:
Any chance you could supply a repro, so we know exactly what you are trying to do? (I could try to create one myself, but I might well find quite different problems...)
There's a lot of fuzziness in the spec in this area, so the outcome might turn out to be clarification of the documentation to explain what the restrictions are. But I'll investigate first before coming to that conclusion.
Hi, I'll have few days of vacation now so it will take some time but I can prepare it afterwards.
In general there are (at least) two issues. One with global variable and second with function duplicates.
Regarding functions it's related to XQueryFunctionLibrary.declareFunction(XQueryFunction function) method.
When we're compiling a library that refers to a function that was already compiled there is a check to prevent registration of different function with the same name.
Check is done by "!=" instead of some better approach to check systemId + namespace + module name + maybe row number in file (just a suggestion, I'm sure there is better solution).
Updated by Mariusz B over 9 years ago
Hi,
I've reproduced one of the issues which occurs for compileLibrary in context of circular dependencies.
Please see XqueryIssueTest which contains two simple unit tests showing difference in behavior between compile and compileLibrary methods.
In order to execute it please add saxon-license.lic to src/main/resources and run XqueryIssueTest.
Regards,
Mariusz
Updated by Michael Kay over 9 years ago
Thanks. I've created a unit test (9.6/TestXQueryCompiler/testAbleToCompileWithCompileLibraryMethod()) which reproduces the problem. It seems that what is happening is that the "top level" library module loaded by compileLibrary() isn't included in the list of modules that the "import module" code checks for when looking for cycles, so this module is processed twice, leading to the "duplicate declaration" error.
More specifically, the top level module is only added to the list of known modules on completion, which means it isn't present in the list while the import module declarations are being processed.
I tried a patch and it didn't work, so I'm going to have to come back to it. I'm away for a week, but since this has been lingering for some while can I assume it's not especially urgent?
Updated by Michael Kay over 9 years ago
- Status changed from New to In Progress
- Assignee set to Michael Kay
Updated by Michael Kay over 9 years ago
I suspect the problem is related to module location URIs, and the unfortunate provision in the XQuery spec (XQ3.0, section 4.12.2) "Several different modules with the same target namespace can be used in the same query. The names of public variables and public functions must be unique within the module contexts of a query: that is, if two modules with the same module URI are used in the same query, the names of the public variables and functions in their module contexts must not overlap."
The problem is that when you do compileLibrary(), the module you compile has no known location URI, and so when you import that module again via a cyclic module import, the module you are importing has a different "location" from the top-level module, and is therefore assumed to be a different module with the same module URI.
How else should we decide whether two query files with the same module URI are the same module or different modules? Perhaps we should do it by comparing the textual content of the module, rather than its location URI. Perhaps we should have an option to disallow multiple modules with the same module URI, and set this by default, especially when compiling a library module. This area of the spec is murky territory. But our failure to detect the cycle here, with no obvious workaround available, is certainly unfortunate.
The test case supplied uses a single compileLibrary() of a library with cyclic imports. The original problem description talks of multiple compileLibrary() calls of modules that import a common module. Either way we have similar problems: how do we define module identity, given that the spec allows multiple modules with the same URI?
Updated by Michael Kay over 9 years ago
Another little issue here: the text I cited says "the names of the public variables and functions in their module contexts must not overlap". In this case the variable is private. But I suspect Saxon isn't allowing for two modules with the same target namespace both having private variables with the same name.
Updated by Mariusz B over 9 years ago
There's no big rush with this issue. As one of background tasks I'm working on reducing compilation time and memory overhead of saxon modules we're using.
For now I can work on reproducing rest of the issues I found.
As a workaround I may be able to get rid of circular dependencies as there shouldn't be any at first place.
Thanks for your support.
Updated by Mariusz B over 9 years ago
I've reproduced the same issue with duplicated functions, I'm not raising another ticket as it's connected to the same thing.
Please see attached zip file.
Updated by Mariusz B about 9 years ago
Solution provided in https://saxonica.plan.io/issues/2456 (setting baseUri) seems to help.
I'll analyze it further.
Updated by Michael Kay about 9 years ago
Essentially the resolution to this is that if two references to modules with the same module URI use different location URIs, then you can expect to get the error; and this applies also if there is no known location URI. The remedy is to ensure that the module has a known location URI, which can be achieved by calling setBaseURI() before doing compileLibrary(). For 9.6 I'm going to leave it at that.
For 9.7 I'm thinking of introducing a configuration option XQUERY_MULTIPLE_MODULE_IMPORTS based closely on the existing MULTIPLE_SCHEMA_IMPORTS. If set to false, any "import module" for a module URI is resolved to an existing known module with that module URI (without even calling the module URI resolver), ignoring an location URI that might be present.
Updated by Michael Kay about 9 years ago
- Status changed from In Progress to Resolved
Resolved as suggested. For 9.6, set the base URI. For 9.7, there's a new configuration setting, and by default any attempt to import a module with a given module URI will select an existing loaded module with that module URI regardless of the location hints.
Updated by Michael Kay almost 8 years ago
- Status changed from Resolved to Closed
- Fixed in version set to 9.7
- Applies to branch 9.6 added
Please register to edit this issue