Saxon

Here's a picture of a path from object to GC-root for one of the many objects that is responsible for the "excessive" memory usage. I clipped the picture at the Executable object. This object is one of many Executable objects that are stored in my memory cache, a java.util.Map instance.

It would be good to have some metrics. How many functions are present, how much memory is saved if you manually remove those that aren't used?

It's not possible to remove unused functions automatically because they are discoverable at run-time using fn:function-lookup(). But one could contemplate having something like a function annotation %discard-if-unused to enable such behaviour.

Michael Kay wrote in RE: Option to trim / remove unused functions from the fun...:

It would be good to have some metrics. How many functions are present, how much memory is saved if you manually remove those that aren't used?

It's not possible to remove unused functions automatically because they are discoverable at run-time using fn:function-lookup(). But one could contemplate having something like a function annotation %discard-if-unused to enable such behaviour.

Hi.

I set -Xmx2g and ran a simple test of using the same compiler to produce XQueryExecutable objects and put them into a single List until I got an OutOfMemory error. Then I repeated but with a call to loadLibrary with a xq file with a large number of functions and variables. There was many orders of magnitude difference in the number of executable objects I could create before exhausting memory.

However, I think I identified my problem along the way (which then leads to another error). The input to compileLibrary was a single constructed xq file that imported the other library xq files. Like this:

final String sharedXQuery = "" 
    + "xquery version "3.0" encoding "utf-8";\n" 
    + "module namespace hephaestus = "urn:com:jmaurice:root";\n" 
    + XQUERY_MODULES.values().stream().map(m -> "import module namespace " + m.prefix + " = \"" + m.moduleUri + "\";\n").collect(Collectors.joining()); 
compiler.compileLibrary(sharedXQuery);

I suspect the compileLibrary method needs to be called on each xq library individually. If I call compileLibrary on this single constructed thing that imports my real libraries, then I suspect that the compiled libraries won't be cached in the compiler and shared with the many Executable objects, and I will get separate copies of the compiled libraries in each Executable (because each query input to compile() has the import module statements for each separate library). Is this correct?

I assumed yes for now, and I rewrote it to call compileLibrary separately on each actual library which is imported in the query input to compile(). However, now I hit a new error. When I try to call compileLibrary separately, I get "Duplicate definition of global variable" on the third compileLibrary call. Should I open a new topic? I'll just post it here for now. Sorry. Thanks. I have created a minimal complete runnable repro (minus the Enterprise license key). The attached Zip file has the 3 xq input files and the Java test code in a main() method.

I've read some of the information available online, and I think I don't have any problems regarding different locations or hints. I have no "at" hints on the import module statements. I am java.io.File objects directly to compileLibrary instead of providing an in-memory String object. I think the saxon compiler should recognize that it's already been compiled and not compile it again, but it looks like the compiler does not recognize that it has a cached copy of the xmlCharUtil library and it tries to compile it again, which leads to the duplicate variable definition error.

There are several things going on here and it's a bit difficult to disentangle them.

Firstly, I can confirm that there is no caching in the XQueryCompiler. If you compile the same source file more than once, Saxon isn't going to notice.

Secondly, the XQueryCompiler maintains a single StaticContext object containing (among other things) all the function definitions, and compiling two library modules with conflicting function definitions is therefore going to give an error. I can't quite remember what the rationale was for this design (it was all done a long time ago).

Thirdly, when you build an Executable that import a number of library modules, then it should initially share the same function bodies with other executables that import the same library modules. However (a) it will build its own index of all the functions in the static context, and it wouldn't surprise me if the index entries occupy as much memory as the function bodies themselves (if they are small); and (b) there is an optimisation phase that may result in called functions being inlined within caller functions.

It may be worth experimenting with suppressing function inlining: xqueryCompiler.getUnderlyingStaticContext().setOptimizerOptions(new OptimizerOptions("-f")); to see what effect this has on memory usage.

Michael Kay wrote in RE: Option to trim / remove unused functions from the fun...:

There are several things going on here and it's a bit difficult to disentangle them.

Firstly, I can confirm that there is no caching in the XQueryCompiler. If you compile the same source file more than once, Saxon isn't going to notice.

Hi. Sorry. There's been a miscommunication. Could you take a look at my repro.zip file please? In short, consider these 3 xq files:

xmlCharUtil.xq

xquery version "3.0" encoding "utf-8";
module namespace xmlch = "urn:activevos:avsf:xquery:utils:xmlcharutil";
declare variable $xmlch:xmlLetter := "foo";

json2xml.xq

xquery version "3.0" encoding "utf-8";
module namespace j2x = "urn:informatica:ae:xquery:json2xml";
import module namespace xmlch = "urn:activevos:avsf:xquery:utils:xmlcharutil";

spi-util-functions.xq

xquery version "3.0" encoding "utf-8";
module namespace util = "urn:activevos:spi:functions:utilities";
import module namespace xmlch = "urn:activevos:avsf:xquery:utils:xmlcharutil";
import module namespace j2x = "urn:informatica:ae:xquery:json2xml";

For those 3 files, when I make the following three compileLibrary calls in succession on the same compiler object, what should happen? For me, it's throwing SaxonApiException "Duplicate definition of global variable xmlch:xmlLetter".

compiler.compileLibrary("xmlCharUtil.xq"); //passes
compiler.compileLibrary("json2xml.xq"); //passes
compiler.compileLibrary("spi-util-functions.xq"); //throws SaxonApiException "Duplicate definition of global variable xmlch:xmlLetter"

The repro.zip is a full complete runnable repro of this use case.

Based on some preliminary testing of some other use cases that don't hit this problem, I suspect this approach would solve my memory usage problems, but it throws unexpectedly. I expect that this should not throw.

Michael Kay wrote in RE: Option to trim / remove unused functions from the fun...:

...

Also, here are some more concrete numbers to show what I am chasing.

No compileLibrary calls. 
Main xquery imports only hash module.
149,276 Executables before out-of-memory.

Baseline.

compileLibrary only on hash module
Main xquery imports only hash module.
492,991 Executables before out-of-memory.

Observation: We see reduced memory usage by using compileLibrary.

compileLibrary on List of modules, call it list-M, including hash (but not including some modules that lead to the "duplicate variable definition" error)
Main xquery imports only hash module.
491,971 Executables before out-of-memory.

Observation: We see that adding more compileLibrary calls on the same compiler, where the additional modules are not used by the main xquery, does not increase memory usage by each Executable.

No compileLibrary calls. 
Main xquery imports every module from List-M
1,197 Executables before out-of-memory.

Observation: We see that importing a lot of big modules without compileLibrary calls is very expensive in terms of memory.

CompileLibrary on every module from List-M
Main xquery imports on every module from List-M
70,086 Executables before out-of-memory.

Observation: We see increased memory usage when importing modules in the main query that the main query does not use, but the memory usage is reduced with compileLibrary calls on all of the modules. It's still not as good as omitting the main query imports of unnecessary imports, but it's better with the compileLibrary calls than otherwise. This is probably because of some of the details you mentioned above, how the Executable keeps its own copy of an index.

Summary: If I could solve this "duplicate variable definition" problem, then I could get much better memory usage -- better by a factor of 70x. Further, with a little bit of work on my end to parse the main xquery to determine the list of imports that it actually needs, I could get closer to 500x better memory usage.

PS: It would be nice if the Saxon optimizer could detect that my main xquery and my module imports do not use xquery reflection, and then apply an optimization to remove unused functions and variables. As I just said, I could do a partial workaround myself by parsing the main xquery myself to find the minimum necessary list of imports, but that's inconvenient, and will still produce less than ideal results because I'll still keep all of the unused functions and variables from the modules that I do import.

PPS: Does Saxon treat the module prefix "util" specially? My repro.zip might be because Saxon has special rules for modules with the "util" prefix. I'm following up on this guess - let me rewrite my stuff to use another prefix and see if Saxon still complains.

Joshua Maurice wrote in RE: Option to trim / remove unused functions from the fun...:

PPS: Does Saxon treat the module prefix "util" specially? My repro.zip might be because Saxon has special rules for modules with the "util" prefix. I'm following up on this guess - let me rewrite my stuff to use another prefix and see if Saxon still complains.

And no, that's not it either.

Thanks for the statistics. Compiling half a million queries is clearly a rather extreme use case, but I'll have a think about how we can handle it better.

I'm wondering whether we could use something like the mechanism in:

XPathCompiler.addXsltFunctionLibrary()

which adds the public functions of an XSLT package to the static context of an XPath expression. This mechanism was devised to support the xsl:evaluate instruction, but was considered useful enough to expose it in the public API. Unlike separate compilation of XQuery modules, this seems to involve no copying of data, not even metadata about the relevant functions.

Or perhaps something based on fn:load-xquery-module() might work.

Michael Kay wrote in RE: Option to trim / remove unused functions from the fun...:

...

Hi. Please don't forget about the bug(?) that I identified in compileLibrary, with the repro in repro.zip. If possible, could I get some confirmation. Is this a bug? It looks like a bug to me. Or is there something that I could do to successfully call compileLibrary on those 3 tiny xq files in the repro.zip?

Thanks.

Michael Kay wrote in RE: Option to trim / remove unused functions from the fun...:

Thanks for the statistics. Compiling half a million queries is clearly a rather extreme use case, but I'll have a think about how we can handle it better.

This is not an extreme use case for me.

As I described above, my use case is an enterprise product with its own concept of a job specification, a job execution plan, and a job execution. We easily have thousands of jobs, and each of those jobs might contains dozen or hundreds of xqueries to be executed. If I could cache the execution plans of my jobs in memory, and if I could store the complied XQueryExecutable objects for each of those xqueries, then that could save me substantial (huge) amounts of wallclock time to run my jobs executions for my customers.

There is a wide variety of main xquery strings that are given to compile(), but all of the main xquery strings will import module only from a certain small list that is known up front. Hence why I am looking at using a single XQueryCompiler object, and calling compileLibrary() on this fixed list of modules, and then calling compile() on the wide variety of main xquery strings to produce XQueryExecutables which ideally are quite small.

Rather than being an extreme use case, it looks like the intention of compileLibrary() is support exactly this kind of use case. However, the bug identified in repro.zip is preventing me from using it to the fullest. This is in addition to the identified additional memory usage in each XQueryExecutable when adding module imports to the main xquery string that are not used in the main xquery string regardless of whether the compiler already did a compileLibrary call on that module.

Thanks.

Should I open a new issue? I appear to have permissions to do things at "https://saxonica.plan.io/issues".

Also, I have tried with every combination of:

• setting XQUERY_MULTIPLE_MODULE_IMPORTS true and false
• calling (and not calling) setBaseURI before each call to compileLibrary, trying with the moduleUri, a filename URN, and an absolute file URI
• calling compileLibrary on a File object and on an InputStream object.
• xq library files with and without "AT" filename hints on the "import module namespace" statements

Nothing works. I always get that exception on the third call to compileLibrary.

Update.

I am working on a new product, but my company already uses Saxon in another product, and I found out from a coworker today that Saxon can precompile these three libraries. However, we had to implement extensive workarounds in our code to do so. We have to implement a new subtypes of com.saxonica.expr.QueryLibraryImpl and com.saxonica.ee.optim.StaticQueryContextEE in order to work around the bug in the existing Saxon code.

Note: Even with these workarounds, I'm still getting roughly 70,000 XQueryExecutable objects before exhausting memory with -Xmx with "import module" statements of big modules in my main xquery string, compared to roughly 440,000 without the "import module" statements, which match the numbers above. So, the code workarounds that we have are a fix for the "duplicate variable definition" problem, but we are still left with the bug fix / feature request for improving memory usage of the XQueryExecutable object when the main query string has an import module that is not used by the main xquery string.

I think we'll track these issues with the following bug report and feature request:

https://saxonica.plan.io/issues/6394

https://saxonica.plan.io/issues/6395