Saxon

A quick google suggests this is a known (and recently solved) problem in Camel 3.x:

https://issues.apache.org/jira/browse/CAMEL-18492

The last comment in the thread suggests there's a fix in a very recent CAMEL release. You'll have to get the details from the CAMEL project.

(Summary of the problem: everyone thinks security is a good idea, and so they put their software through scanning utilities that look for "vulnerabilities". These scanners are actually very crude. In particular, they object to you running an XSLT transformation without setting the SECURE_PROCESSING option to true. But allowing a stylesheet to call out to Java is intrinsically insecure, so it's not allowed when SECURE_PROCESSING is switched on. If you trust the stylesheet to behave itself, then it's fine, but you don't get the chance to tell the security scanning tool that you trust it. So the CAMEL people disabled the feature by default.)

Hi Michael,

Thanks for your reply.

The Camel bug CAMEL-18492 was filed by me.

Prior to filing the camel bug, the received an Error during transformation as below:

Fatal error parsing XSLT file: Cannot find a 0-argument function named Q{java:java.util.Date}new(). External function calls have been disabled; SystemID: classpath:mapping/XSLTMapping2.xsl; Line\#: 4|

The error message suggested that either it was a Licence issue or some kind of flag. Looking into camel code, we could see that flag XMLConstants.FEATURE_SECURE_PROCESSING was being set, we wrote a small standalone program and sure enough we were able to reproduce this issue.

After the Camel Bug was raised, we down ported the camel changes and then we resumed our testing.

However we now see the issue:

Fatal error parsing XSLT file: Cannot find a 0-argument function named Q{java:java.util.Date}new(). For diagnostics on calls to Java methods, use the -TJ command line option or set the Configuration property FeatureKeys.TRACE_EXTERNAL_FUNCTIONS; SystemID: classpath:mapping/XSLTMapping1.xsl; Line\#: 4|

We do not get the External function calls have been disabled Error message.

This Error message seems to suggest that it might not be Camel related issue.

Further we queried the Saxon EnterpriseTransformerFactory class for the Feature http://javax.xml.XMLConstants/feature/secure-processing and it did return false and also we queried the Saxon EnterpriseConfiguration class for the Feature Feature.ALLOW_EXTERNAL_FUNCTIONS and this returned true.

We tried to enable the Feature Feature.TRACE_EXTERNAL_FUNCTIONS but we did not see any output (We are not sure where this outputs the data).

We are looking to information as to why Saxon is throwing this Error and Ways to debug this issue.

Whether it is a License issue or a Feature Flag related issue or any other flag issue or is it a class path/loader issue, we are not sure.

Please let us know how to further diagnose the issue.

Note: We have access to EnterpriseTransformerFactory class, please let us know if we can query/set any fields to check for more clues.

The TRACE_EXTERNAL_FUNCTIONS property should send diagnostic output to the standard error stream by default; it can be redirected by registering a Logger with the Saxon Configuration, or by setting the output stream used by the standard Logger. The messages are logged by calling StandardLogger.info(), so if you are able to debug into the application, then breakpointing that call would be a good way forward.

For a simple call like Q{java:java.util.Date}new() that works in other environments, it seems unlikely to be a failure loading the class unless there's some very peculiar ClassLoader in play. It's also unlikely to be an issue finding the method/constructor within the class -- unless perhaps reflection is disabled at JVM level, which might happen with an ahead-of-time environment such as GraalVM. You really need to see those trace messages.

Looking at the Camel 3 change log, I see that they use things like bytecode generation and reflection internally, so it's not at all impossible that they do something, for example with ClassLoaders, that interferes with Saxon's dynamic loading.

Hi Michael,

I redirected output to a file using the following:

    factory.setFeature("http://javax.xml.XMLConstants/feature/secure-processing", false);
    factory.setAttribute("http://saxon.sf.net/feature/standardErrorOutputFile", errFilePath);
    factory.setFeature("http://saxon.sf.net/feature/allow-external-functions", true);
    factory.setFeature("http://saxon.sf.net/feature/trace-external-functions", true);

I got the following output.

Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!

Still there is no indication as to why Saxon is unable to find the Java Language Function "new('')".

As an experiment, I tried using saxon:path() functio using the xslt as below, but got the same error:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:template match="/" xmlns:saxon="http://saxon.sf.net/">
        <xsl:value-of select="saxon:path()"/>
    </xsl:template>
</xsl:stylesheet>

Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{http://saxon.sf.net/}path not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{http://saxon.sf.net/}path not found!
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{http://saxon.sf.net/}path not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{http://saxon.sf.net/}path not found!

Additional Info: We are using Java 1.8, so there should not be any issues related to modularity.

OK, this tells us that the JavaExtensionLibrary isn't even on the list of function libraries that Saxon is looking in; that is, reflexive extension functions are completely off the agenda. Back in 9.9, saxon:path was implemented as a reflexive extension function, so that will be the same problem.

Since you've clearly got an EnterpriseConfiguration, I can only think of two explanations: either it's a license file issue, or the initialization of the EnterpriseConfiguration has been customised in some way.

Can you call isLicensedFeature(8) on the Configuration object to check the first possibility?

Then can you see what configuration.getExtensionBinder("java") returns? I would expect it to be either a JavaExtensionLibrary, or null. Knowing which it is will help us to know what we need to investigate next.

I don't know the Camel architecture. Presumably they create the Saxon configuration, probably using JAXP interfaces? Do you know whether they might alter its configuration (a) before you are looking at it, or (b) after you are looking at it?

Hi Michael,

The Camel XSLT architecture, creates a new Transformer Factory if there is not one already created or passed to the camel processor. This piece of Camel code might be of interest to you in this aspect: [[https://github.com/apache/camel/blob/797fd4c5fd493b814c73a49a8e3a6e5ea5d4b20a/components/camel-xslt-saxon/src/main/java/org/apache/camel/component/xslt/saxon/XsltSaxonEndpoint.java#L198]]

As far as the Configuration checks you had asked to investigate,

config.isLicensedFeature(8) => true
config.getExtensionBinder("java") => com.saxonica.config.JavaExtensionLibrary

The values are as above.

The extension binder returns JavaExtensionLibrary, but the Trace Log still reports as

Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!

Well, we've eliminated a lot of possibilities, so we're homing in, but we're not there yet.

The trace output corresponds exactly to the structure of the function library built by StylesheetPackage.createFunctionLibrary() except for the missing JavaExtensionLibrary. The code to build it reads:

        ...
        queryFunctions = new XQueryFunctionLibrary(config);
        functionLibrary.addFunctionLibrary(queryFunctions);
        functionLibrary.addFunctionLibrary(config.getIntegratedFunctionLibrary());
        **config.addExtensionBinders(functionLibrary);**
        functionLibrary.addFunctionLibrary(new StylesheetFunctionLibrary(this, false));
        ...

so we would expect the final entries on the list to be an XQueryFunctionLibrary, an IntegratedFunctionLibrary, all ExtensionBinders registered with the Configuration, and a StylesheetFunctionLibrary. The trace clearly indicates that at the time the function library was built, the set of ExtensionBinders in the Configuration was empty, which conflicts with the result you show for config.getExtensionBinder("java"). That suggests two possibilities (a) it's a different Configuration object, or (b) there's been a change to the Configuration between the time you examined it and the time it was used to compile the stylesheet. The latter seems more likely.

But looking at the Camel source code you pointed me to, with the way that it manipulates the TransformerFactoryImpl and the Configuration, I don't think that the first possibility can be immediately ruled out.

What's the sequence of events in your code, between creating the Saxon Configuration/TransformerFactory, examining its properties, and compiling the stylesheet (using factory.newTemplates(), presumably).

I'm running out of diagnostics to suggest, which means I'm starting to contemplate the very unwelcome step of downloading and building the Camel code so I can debug into it. In theory our support policy says we don't do that -- if a third party application that uses Saxon isn't working, then unless you can demonstrate a problem with Saxon running standalone, then it's not our problem. Do you have a contact on the Camel project who would be prepared to work with us on this?

Hi Michael,

We create a transformer factory with the Licence key and Configuration and pass the Factory object to camel (via the URI construct transformerFactory=#saxoneeTransformer)

The Configuration object was inspected just before the execution of the XSLT Transformation. This was done with the help of groovy script.

Camel does not create a new transformer factory, if it is provided with one.

Therefore I could not foresee that a new Transformer Factory object being created or the existing one being modified.

I am planning to connect the code to a debugger, Since we do not have knowledge on Saxon implementation, what should I be looking for, as a first step to diagnose the issue?

We need to look at the code in Configuration that maintains the function library list. I would want in the first instance to breakpoint calls on ProfessionalConfiguration.setExtensionBinder() and .addExtensionBinders(), and StylesheetPackage.createFunctionLibrary().

I think it may work best if we schedule a screen-sharing Zoom call where you can run this, and I can talk you through it.

Hi Michael,

Thanks for your inputs.

Our's a very large application and thus we are trying to reproduce the issue with a small Camel + XSLT spring boot application. If we are able to reproduce the same, it will be very easy to debug,

If we are unsuccessful in reproducing the issue in a small springboot application, we will prep the runtime for a shared debug session.

I will come back in a couple of days with proposals for Shared debug session, and then debug the application jointly.

Hi Michael,

We tried to debug the issue in a standalone Camel Spring Boot application.

When we looked at the output of the External Functions Trace, we could see that only "Built In Extension Library".

On a hunch, We add the below configuration and we were able to get the XSLT Mapping working.

        EnterpriseConfiguration config = (EnterpriseConfiguration) factory.getConfiguration();
        JavaExtensionLibrary library = new JavaExtensionLibrary(config);
        factory.getConfiguration().getBuiltInExtensionLibraryList().addFunctionLibrary(library);

The XSLT transformations worked, and we got the output of the Java function.

We have a couple of questions:

1. Is this a correct configuration?
2. If the configuration is correct, then why the XSLT transformations worked in Camel 2.X out of box, but not in 3.X.

Good work. This confirms my previous inference that the extension function calls were failing because the JavaExtensionLibrary is not present in the list of libraries to be searched: and the workaround of adding it back in manually seems perfectly sound. Unfortunately it gets us no closer to discovering why it is absent in the first place. I can only think this is something to do with the way in which Camel initialises the configuration.

Hi Michael,

Thanks for the response.

We create a TransformerFactory with the configuration and pass this TransformFactory object to camel. Camel is expected to use this.

The above works in Camel 2x, but fails for some reason in Camel 3x.

Our worry is; Our product allows customers to add external jars and call Java functions from these external libraries. If you could brief us about the process of library discovery, may be we would debug more in this area.

While we will try to test this, We would like to know a bit more on the configuration should be initialised, for Java calls to work.

Is there any recommended configuration that we can cross reference or test with ?

Please check the contents of the function library in the configuration at the time you create it, and compare with the configuration later at the point of the "external functions trace".

Is it the same configuration object?

Has the JavaExtensionLibrary disappeared in the meantime? If so, this can only be because Camel 3.x is doing something to remove it.

There are two ways to move forward: either find out why and how Camel 3.x is changing the configuration you supply, or live with this and work around it by reinstating the JavaExtensionLibrary as described in your previous comment.