Project

Profile

Help

Support #5722

open

Cannot use Java functions in XSLT transformation.

Added by Harish Annamalai about 1 month ago. Updated 13 days ago.

Status:
New
Priority:
High
Assignee:
Category:
JAXP Java API
Sprint/Milestone:
-
Start date:
2022-10-26
Due date:
% Done:

0%

Estimated time:
Legacy ID:
Applies to branch:
Fix Committed on Branch:
Fixed in Maintenance Release:
Platforms:
Java

Description

Hi,

We have ab enterprise licence for Saxon.

This is with regards to camel-xslt-saxon.

We are using saxon with camel and during our migration we are trying to invoke java functions from XSLT transformation.

We are hit with following exception: Fatal error parsing XSLT file: Cannot find a 1-argument function named Q{java:java.lang.String}new()

The Corresponding XSLT is simple:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
	<xsl:template match="/" xmlns:str="java:java.lang.String">
        <xsl:value-of select="str:new('Test')"/>
    </xsl:template>
</xsl:stylesheet>

We are using the camel Saxon URI as below

xslt-saxon:mapping/XSLTMapping1.xsl?transformerFactory=#saxoneeTransformer&amp;output=bytes&amp;saxonExtensionFunctions=#xsltExtensionsV11

While this works with in Camel 2.x this fails in Camel 3.x

We are passing an instance of EnterpriseTransformerFactory to camel.

We are not sure why the XSLT transformation fails.

Stack Trace:

2022 10 26 11:25:11#+00#ERROR#org.apache.camel.component.xslt.XsltErrorListener##anonymous#1666783509958-XSLTMapping_Worker-1####na#na#na#na#Fatal error parsing XSLT file: Cannot find a 1-argument function named Q{java:java.lang.String}new(). For diagnostics on calls to Java methods, use the -TJ command line option or set the Configuration property FeatureKeys.TRACE_EXTERNAL_FUNCTIONS; SystemID: classpath:mapping/XSLTMapping1.xsl; Line\#: 4|
2022 10 26 11:25:11#+00#ERROR#org.apache.camel.processor.errorhandler.DefaultErrorHandler##anonymous#1666783509958-XSLTMapping_Worker-1####na#na#na#na#Failed delivery for (MessageId: B86228E138F42E1-0000000000000001 on ExchangeId: B86228E138F42E1-0000000000000001). Exhausted after delivery attempt: 1 caught: net.sf.saxon.trans.XPathException: Errors were reported during stylesheet compilation

Message History (complete message history is disabled)
---------------------------------------------------------------------------------------------------------------------------------------
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[route17           ] [route17           ] [from[quartz://XSLTMappingTimerEventDefinition5693451?trigger.repeatCount=0&tri] [       148]
	...
[Process_1         ] [CallActivity_4_216] [xslt-saxon:mapping/XSLTMapping1.xsl?transformerFactory=\#saxoneeTransformer&out] [         0]

Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
net.sf.saxon.trans.XPathException: Errors were reported during stylesheet compilation
	at net.sf.saxon.style.XSLTemplate.jitCompile(XSLTemplate.java:819)
	at com.saxonica.ee.trans.TemplateRuleInitializer.init(TemplateRuleInitializer.java:52)
	at com.saxonica.ee.trans.TemplateRuleEE.initialize(TemplateRuleEE.java:85)
	at net.sf.saxon.trans.Mode.applyTemplates(Mode.java:509)
	at net.sf.saxon.trans.XsltController.applyTemplates(XsltController.java:746)
	at net.sf.saxon.s9api.AbstractXsltTransformer.applyTemplatesToSource(AbstractXsltTransformer.java:347)
	at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:349)
	at net.sf.saxon.jaxp.TransformerImpl.transform(TransformerImpl.java:71)
	at org.apache.camel.component.xslt.XsltBuilder.process(XsltBuilder.java:123)
	at org.apache.camel.support.ProcessorEndpoint.onExchange(ProcessorEndpoint.java:93)

We are using a simple String::new to test, even this fails.

We executed the method: getProductTitle() of Configuration class, and get a response as

Saxon-EE 9.9.1.6J from Saxonica

Therefore we believe the licence features are activated.

We created a small java program and executed the XSLT transformation, this works as a standalone application, but not in camel 3.x

We are looking for possible reasons as to why the XSLT transformation fails, possible reasons for failure and how to check whether the License features are activated.

Actions #1

Updated by Michael Kay about 1 month ago

A quick google suggests this is a known (and recently solved) problem in Camel 3.x:

https://issues.apache.org/jira/browse/CAMEL-18492

The last comment in the thread suggests there's a fix in a very recent CAMEL release. You'll have to get the details from the CAMEL project.

(Summary of the problem: everyone thinks security is a good idea, and so they put their software through scanning utilities that look for "vulnerabilities". These scanners are actually very crude. In particular, they object to you running an XSLT transformation without setting the SECURE_PROCESSING option to true. But allowing a stylesheet to call out to Java is intrinsically insecure, so it's not allowed when SECURE_PROCESSING is switched on. If you trust the stylesheet to behave itself, then it's fine, but you don't get the chance to tell the security scanning tool that you trust it. So the CAMEL people disabled the feature by default.)

Actions #2

Updated by Michael Kay about 1 month ago

  • Tracker changed from Bug to Support
  • Category set to JAXP Java API
  • Assignee set to Michael Kay
Actions #3

Updated by Harish Annamalai about 1 month ago

Hi Michael,

Thanks for your reply.

The Camel bug CAMEL-18492 was filed by me.

Prior to filing the camel bug, the received an Error during transformation as below:

Fatal error parsing XSLT file: Cannot find a 0-argument function named Q{java:java.util.Date}new(). External function calls have been disabled; SystemID: classpath:mapping/XSLTMapping2.xsl; Line\#: 4|

The error message suggested that either it was a Licence issue or some kind of flag. Looking into camel code, we could see that flag XMLConstants.FEATURE_SECURE_PROCESSING was being set, we wrote a small standalone program and sure enough we were able to reproduce this issue.

After the Camel Bug was raised, we down ported the camel changes and then we resumed our testing.

However we now see the issue:

Fatal error parsing XSLT file: Cannot find a 0-argument function named Q{java:java.util.Date}new(). For diagnostics on calls to Java methods, use the -TJ command line option or set the Configuration property FeatureKeys.TRACE_EXTERNAL_FUNCTIONS; SystemID: classpath:mapping/XSLTMapping1.xsl; Line\#: 4|

We do not get the External function calls have been disabled Error message.

This Error message seems to suggest that it might not be Camel related issue.

Further we queried the Saxon EnterpriseTransformerFactory class for the Feature http://javax.xml.XMLConstants/feature/secure-processing and it did return false and also we queried the Saxon EnterpriseConfiguration class for the Feature Feature.ALLOW_EXTERNAL_FUNCTIONS and this returned true.

We tried to enable the Feature Feature.TRACE_EXTERNAL_FUNCTIONS but we did not see any output (We are not sure where this outputs the data).

We are looking to information as to why Saxon is throwing this Error and Ways to debug this issue.

Whether it is a License issue or a Feature Flag related issue or any other flag issue or is it a class path/loader issue, we are not sure.

Please let us know how to further diagnose the issue.

Note: We have access to EnterpriseTransformerFactory class, please let us know if we can query/set any fields to check for more clues.

Actions #4

Updated by Michael Kay about 1 month ago

The TRACE_EXTERNAL_FUNCTIONS property should send diagnostic output to the standard error stream by default; it can be redirected by registering a Logger with the Saxon Configuration, or by setting the output stream used by the standard Logger. The messages are logged by calling StandardLogger.info(), so if you are able to debug into the application, then breakpointing that call would be a good way forward.

For a simple call like Q{java:java.util.Date}new() that works in other environments, it seems unlikely to be a failure loading the class unless there's some very peculiar ClassLoader in play. It's also unlikely to be an issue finding the method/constructor within the class -- unless perhaps reflection is disabled at JVM level, which might happen with an ahead-of-time environment such as GraalVM. You really need to see those trace messages.

Looking at the Camel 3 change log, I see that they use things like bytecode generation and reflection internally, so it's not at all impossible that they do something, for example with ClassLoaders, that interferes with Saxon's dynamic loading.

Actions #5

Updated by Harish Annamalai about 1 month ago

Hi Michael,

I redirected output to a file using the following:

    factory.setFeature("http://javax.xml.XMLConstants/feature/secure-processing", false);
    factory.setAttribute("http://saxon.sf.net/feature/standardErrorOutputFile", errFilePath);
    factory.setFeature("http://saxon.sf.net/feature/allow-external-functions", true);
    factory.setFeature("http://saxon.sf.net/feature/trace-external-functions", true);

I got the following output.

Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!

Still there is no indication as to why Saxon is unable to find the Java Language Function "new('')".

As an experiment, I tried using saxon:path() functio using the xslt as below, but got the same error:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:template match="/" xmlns:saxon="http://saxon.sf.net/">
        <xsl:value-of select="saxon:path()"/>
    </xsl:template>
</xsl:stylesheet>
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{http://saxon.sf.net/}path not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{http://saxon.sf.net/}path not found!
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{http://saxon.sf.net/}path#0
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{http://saxon.sf.net/}path not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{http://saxon.sf.net/}path not found!

Additional Info: We are using Java 1.8, so there should not be any issues related to modularity.

Actions #6

Updated by Michael Kay about 1 month ago

OK, this tells us that the JavaExtensionLibrary isn't even on the list of function libraries that Saxon is looking in; that is, reflexive extension functions are completely off the agenda. Back in 9.9, saxon:path was implemented as a reflexive extension function, so that will be the same problem.

Since you've clearly got an EnterpriseConfiguration, I can only think of two explanations: either it's a license file issue, or the initialization of the EnterpriseConfiguration has been customised in some way.

Can you call isLicensedFeature(8) on the Configuration object to check the first possibility?

Then can you see what configuration.getExtensionBinder("java") returns? I would expect it to be either a JavaExtensionLibrary, or null. Knowing which it is will help us to know what we need to investigate next.

I don't know the Camel architecture. Presumably they create the Saxon configuration, probably using JAXP interfaces? Do you know whether they might alter its configuration (a) before you are looking at it, or (b) after you are looking at it?

Actions #7

Updated by Harish Annamalai about 1 month ago

Hi Michael,

The Camel XSLT architecture, creates a new Transformer Factory if there is not one already created or passed to the camel processor. This piece of Camel code might be of interest to you in this aspect: [[https://github.com/apache/camel/blob/797fd4c5fd493b814c73a49a8e3a6e5ea5d4b20a/components/camel-xslt-saxon/src/main/java/org/apache/camel/component/xslt/saxon/XsltSaxonEndpoint.java#L198]]

As far as the Configuration checks you had asked to investigate,

config.isLicensedFeature(8) => true
config.getExtensionBinder("java") => com.saxonica.config.JavaExtensionLibrary

The values are as above.

The extension binder returns JavaExtensionLibrary, but the Trace Log still reports as

Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.functions.registry.XSLT30HOFunctionSet
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Trying net.sf.saxon.functions.FunctionLibraryList
Looking for function Q{java:java.lang.String}new#1
Trying com.saxonica.ee.extfn.VendorFunctionSetEE
Trying net.sf.saxon.functions.MathFunctionSet
Trying net.sf.saxon.ma.map.MapFunctionSet
Trying net.sf.saxon.ma.arrays.ArrayFunctionSet
Trying net.sf.saxon.functions.registry.ExsltCommonFunctionSet
Function Q{java:java.lang.String}new not found!
Trying net.sf.saxon.functions.registry.ConstructorFunctionLibrary
Trying net.sf.saxon.query.XQueryFunctionLibrary
Trying net.sf.saxon.functions.IntegratedFunctionLibrary
Trying net.sf.saxon.style.StylesheetFunctionLibrary
Function Q{java:java.lang.String}new not found!
Actions #8

Updated by Michael Kay about 1 month ago

Well, we've eliminated a lot of possibilities, so we're homing in, but we're not there yet.

The trace output corresponds exactly to the structure of the function library built by StylesheetPackage.createFunctionLibrary() except for the missing JavaExtensionLibrary. The code to build it reads:

        ...
        queryFunctions = new XQueryFunctionLibrary(config);
        functionLibrary.addFunctionLibrary(queryFunctions);
        functionLibrary.addFunctionLibrary(config.getIntegratedFunctionLibrary());
        **config.addExtensionBinders(functionLibrary);**
        functionLibrary.addFunctionLibrary(new StylesheetFunctionLibrary(this, false));
        ...

so we would expect the final entries on the list to be an XQueryFunctionLibrary, an IntegratedFunctionLibrary, all ExtensionBinders registered with the Configuration, and a StylesheetFunctionLibrary. The trace clearly indicates that at the time the function library was built, the set of ExtensionBinders in the Configuration was empty, which conflicts with the result you show for config.getExtensionBinder("java"). That suggests two possibilities (a) it's a different Configuration object, or (b) there's been a change to the Configuration between the time you examined it and the time it was used to compile the stylesheet. The latter seems more likely.

But looking at the Camel source code you pointed me to, with the way that it manipulates the TransformerFactoryImpl and the Configuration, I don't think that the first possibility can be immediately ruled out.

What's the sequence of events in your code, between creating the Saxon Configuration/TransformerFactory, examining its properties, and compiling the stylesheet (using factory.newTemplates(), presumably).

I'm running out of diagnostics to suggest, which means I'm starting to contemplate the very unwelcome step of downloading and building the Camel code so I can debug into it. In theory our support policy says we don't do that -- if a third party application that uses Saxon isn't working, then unless you can demonstrate a problem with Saxon running standalone, then it's not our problem. Do you have a contact on the Camel project who would be prepared to work with us on this?

Actions #9

Updated by Harish Annamalai 25 days ago

Hi Michael,

We create a transformer factory with the Licence key and Configuration and pass the Factory object to camel (via the URI construct transformerFactory=#saxoneeTransformer)

The Configuration object was inspected just before the execution of the XSLT Transformation. This was done with the help of groovy script.

Camel does not create a new transformer factory, if it is provided with one.

Therefore I could not foresee that a new Transformer Factory object being created or the existing one being modified.

I am planning to connect the code to a debugger, Since we do not have knowledge on Saxon implementation, what should I be looking for, as a first step to diagnose the issue?

Actions #10

Updated by Michael Kay 24 days ago

We need to look at the code in Configuration that maintains the function library list. I would want in the first instance to breakpoint calls on ProfessionalConfiguration.setExtensionBinder() and .addExtensionBinders(), and StylesheetPackage.createFunctionLibrary().

I think it may work best if we schedule a screen-sharing Zoom call where you can run this, and I can talk you through it.

Actions #11

Updated by Harish Annamalai 20 days ago

Hi Michael,

Thanks for your inputs.

Our's a very large application and thus we are trying to reproduce the issue with a small Camel + XSLT spring boot application. If we are able to reproduce the same, it will be very easy to debug,

If we are unsuccessful in reproducing the issue in a small springboot application, we will prep the runtime for a shared debug session.

I will come back in a couple of days with proposals for Shared debug session, and then debug the application jointly.

Actions #12

Updated by Harish Annamalai 17 days ago

Hi Michael,

We tried to debug the issue in a standalone Camel Spring Boot application.

When we looked at the output of the External Functions Trace, we could see that only "Built In Extension Library".

On a hunch, We add the below configuration and we were able to get the XSLT Mapping working.

        EnterpriseConfiguration config = (EnterpriseConfiguration) factory.getConfiguration();
        JavaExtensionLibrary library = new JavaExtensionLibrary(config);
        factory.getConfiguration().getBuiltInExtensionLibraryList().addFunctionLibrary(library);

The XSLT transformations worked, and we got the output of the Java function.

We have a couple of questions:

  1. Is this a correct configuration?
  2. If the configuration is correct, then why the XSLT transformations worked in Camel 2.X out of box, but not in 3.X.
Actions #13

Updated by Michael Kay 17 days ago

Good work. This confirms my previous inference that the extension function calls were failing because the JavaExtensionLibrary is not present in the list of libraries to be searched: and the workaround of adding it back in manually seems perfectly sound. Unfortunately it gets us no closer to discovering why it is absent in the first place. I can only think this is something to do with the way in which Camel initialises the configuration.

Actions #14

Updated by Harish Annamalai 13 days ago

Hi Michael,

Thanks for the response.

We create a TransformerFactory with the configuration and pass this TransformFactory object to camel. Camel is expected to use this.

The above works in Camel 2x, but fails for some reason in Camel 3x.

Our worry is; Our product allows customers to add external jars and call Java functions from these external libraries. If you could brief us about the process of library discovery, may be we would debug more in this area.

While we will try to test this, We would like to know a bit more on the configuration should be initialised, for Java calls to work.

Is there any recommended configuration that we can cross reference or test with ?

Actions #15

Updated by Michael Kay 13 days ago

Please check the contents of the function library in the configuration at the time you create it, and compare with the configuration later at the point of the "external functions trace".

Is it the same configuration object?

Has the JavaExtensionLibrary disappeared in the meantime? If so, this can only be because Camel 3.x is doing something to remove it.

There are two ways to move forward: either find out why and how Camel 3.x is changing the configuration you supply, or live with this and work around it by reinstating the JavaExtensionLibrary as described in your previous comment.

Please register to edit this issue

Also available in: Atom PDF