Saxon

The first part of the problem is expected: if the Connection object has more than one close() method, then it is necessary to select one of them explicitly (and even that isn't going to always work if there are multiple overloaded methods with the same arity).

The second part is trickier, and probably can't be solved without some detailed exploration of the Oracle JDBC driver. Why has the connect() function given us a connection object whose close() method isn't available? This may be due to some details of connection pooling or similar, where I'm afraid we're way out of our depth.

One way forward may be for for you to write a shim for the Oracle JDBC driver; at the very least this will give you a place to add diagnostics.

Can adding method.setAccessible(true) before invoking method on object be a solution to avoid java.lang.IllegalAccessException?

It's possible, but if they are using the Java module system to prevent this access then I think calling setAccessible() isn't going to bypass it. I'm afraid we don't have much experience with the Java module system and its impact on reflexive calls.

How do you obtain the connection object, as a matter of interest?

<xsl:variable name="connection" select="shared:new-connection($database, $user, $password, $autoCommit)"/>

with the function shared:new-connection() defined as:

<xsl:function name="shared:new-connection" as="java:java.sql.Connection">
  <xsl:param name="database" as="xs:string"/>
  <xsl:param name="user" as="xs:string?"/>
  <xsl:param name="password" as="xs:string?"/>
  <xsl:param name="autoCommit" as="xs:boolean"/>
  <xsl:variable name="options" as="map(*)">
	 <xsl:map>
		<xsl:map-entry key="'database'" select="$database"/>
		<xsl:map-entry key="'user'" select="$user"/>
		<xsl:map-entry key="'password'" select="$password"/>
		<xsl:map-entry key="'autoCommit'" select="$autoCommit"/>
	 </xsl:map>
  </xsl:variable>
  <xsl:sequence select="sql:connect($options)"/>
  <xsl:sequence select="shared:log('DEBUG', concat('Connected to ', $user, ' on ', $database))"/>
</xsl:function>

I'm puzzled by the two different class names mentioned: oracle.jdbc.driver.T4CConnection and oracle.jdbc.driver.PhysicalConnection.

According to https://stackoverflow.com/questions/45245621/what-is-the-relationship-between-the-oracle-jdbc-oracleconnection-interface-and?noredirect=1&lq=1,

T4CConnection implements OracleConnection via the following inheritance hierarchy: oracle.jdbc.driver.T4CConnection extends oracle.jdbc.driver.PhysicalConnection which extends oracle.jdbc.driver.OracleConnection which extends oracle.jdbc.OracleConnectionWrapper which implements oracle.jdbc.OracleConnection.

so presumably the call on sql:connect is actually returning an instance of oracle.jdbc.driver.T4CConnection whose close() method is presumably inherited from oracle.jdbc.driver.PhysicalConnection. Not that that gets us much further.

I think it would be useful if you could do some experiments at the Java level, taking Saxon out of the picture.

Saxon's logic for sql:connect (in the absence of a dataSource) is essentially:

        Connection connection = null;      // JDBC Database Connection

        MapItem options = (MapItem) arguments[0].head();

        String dataSource = str(options.get(new StringValue("dataSource")));
        String dbString = str(options.get(new StringValue("database")));
        String dbDriverString = str(options.get(new StringValue("driver")));
        String userString = str(options.get(new StringValue("user")));
        String pwdString = str(options.get(new StringValue("password")));
        String autoCommitString = str(options.get(new StringValue("autoCommit")));
        if (userString == null && pwdString == null) {
                    connection = DriverManager.getConnection(dbString);
                } else {
                    connection = DriverManager.getConnection(dbString, userString, pwdString);
                }

See if you can make that work in a freestanding program, and see what the class of the returned connection is. Then try both direct and reflexive calls on the close() method on that connection.

OK, I will check with my colleague if he can make such a small freestanding program.

Yes, that does look interesting. Unfortunately the use of $conn?close here is a generic mechanism that accesses the methods of the instance object $conn; perhaps we could try augmenting this mechanism somehow to retain information about the class whose methods should be used. Or we could try catching the error and looking for methods on superclasses/interfaces?

Not easy to develop and test any solution without first being able to reproduce it.

I'm going to have to try to reproduce this in an environment that doesn't depend on having Oracle installed.

I'm wondering if the problem still occurs if rather than calling $connection?close(), you use the traditional way of invoking Java methods by reflection: Q{java:java.sql.Connection}close($connection)

I can confirm that <saxon:do action="Q{java:java.sql.Connection}close($connection)"/> works fine.

I've been doing some experiments to try to simulate this. Since I'm new to using the Java module system within IntelliJ, I'll record the sordid detail here so that I can come back to it later when I need to.

I created a new IntelliJ project with two IntelliJ modules one and two.

Module one has a module-info.java that says requires two, and a main class that does:

public static void main(String[] args) throws Exception {
    System.err.println("Hello!");
    Connection c = new Creator().getConnection();
    System.err.println("Connection class " + c.getClass());
    Method closer = c.getClass().getMethod("close");
    closer.invoke(c);
    //c.close();
}

Module two contains two packages, bbb and ccc. Its module-info.java says:

module two {
    exports bbb;
}

In package bbb we have two classes, Creator.java which has a single method

     public Connection getConnection() {
        return new SpecificConnection();
    }

and an interface Connection.java which has a single method close().

Package ccc contains a single class SpecificConnection.java which implements Connection, with a close method that prints "Close!!!" to the console.

Calling the main() method fails on the reflective call to SpecificConnection.close() saying "java.lang.IllegalAccessException: class aaa.Hello (in module one) cannot access class ccc.SpecificConnection (in module two) because module two does not export ccc to module one".

If I change the main method to call the close() method obtained from the interface:

        Method closer = Connection.class.getMethod("close");
        closer.invoke(c);

it succeeds.

So how do we get $connection?close() to work at the XPath level? The variable $connection is an external Java object - represented by Saxon class net.sf.saxon.ObjectValue - and internally $connection?close() compiles to an ObjectLookupExpression which calls ObjectMap.makeMethodMap to get the list of available Java methods and present these as XPath function items.

I think the answer is for ObjectValue to hold an optional class (or interface) that is used to obtain the list of available methods, in place of using object.getClass(). The object is created by Saxon's SQLConnectFn which implements the sql:connect() extension function, and it is straightforward for this to supply java.sql.Connection as the value.

I have now reproduced the problem with a simple test case that has no additional dependencies. Test reflex-067 fails on JDK 17:

        let $factory := Q{java:javax.xml.parsers.SAXParserFactory}newDefaultInstance(),
             $parser := $factory?newSAXParser(),
             $isValidating := $parser?isValidating()
         return $isValidating

because the default SAX parser that is returned is in a private JDK package.

What we need to do to solve this is to bind the variable $parser not just to the returned Java object instance, but to the declared return type of the newSAXParser() method (namely javax.xml.parsers.SAXParser).

Now the problem with doing this unconditionally is that it will break code that uses methods that are in fact accessible. Perhaps the method map should include first, methods obtained from the interface type, supplemented with any methods from the instance object that don't clash with methods defined on the interface.

All done (on the 12.x branch), awaiting regression testing.

Good idea to have information about both the declared type and the actual type. We really appreciate your solution-oriented and hands-on approach. Thanks Michael.