Saxon

Excellent question.

The EXPath HTTP module defines an API for calling HTTP from XPath, and the third-party Saxon implementation at https://github.com/expath/expath-http-client-java appears to be actively maintained, though we've never attempted to test it or integrate it. And the API itself is very clumsy by modern standards as it fails to take advantage of maps: it could do with a thorough overhaul. We have redesigned it for SaxonJS, and that could potentially provide a model for a new SaxonJ implementation. I would be interested in feedback.

As for using XSLT to implement a REST endpoint, I don't know that anyone has attempted that. (It has probably been done more often using XQuery). One of the reasons for allowing a 3.0 stylesheet to be invoked with an xsl:function as an entry point was to enable this. It could potentially be done with the help of extension attributes on the xsl:function. But it's obviously a non-trivial amount of work.

Michael Kay wrote in RE: A way to call REST API from Saxon:

Excellent question.

The EXPath HTTP module defines an API for calling HTTP from XPath, and the third-party Saxon implementation at https://github.com/expath/expath-http-client-java appears to be actively maintained, though we've never attempted to test it or integrate it. And the API itself is very clumsy by modern standards as it fails to take advantage of maps: it could do with a thorough overhaul. We have redesigned it for SaxonJS, and that could potentially provide a model for a new SaxonJ implementation. I would be interested in feedback.

Thank you, I'll take a look at this.

As for using XSLT to implement a REST endpoint, I don't know that anyone has attempted that. (It has probably been done more often using XQuery). One of the reasons for allowing a 3.0 stylesheet to be invoked with an xsl:function as an entry point was to enable this. It could potentially be done with the help of extension attributes on the xsl:function. But it's obviously a non-trivial amount of work.

When I think about xslt as a REST service, something as simple as convention comes to mind, not requiring any extension of existing implementations. E.g.:

<xsl:template mode="http:get" match=".[. instanceof map(*)][?url ...]" as="...">
  ...
</xsl:template>

I decided to commit some experiment with ubiquitous nowadays AI. I'm going to verify and probably fix the code.

System
You're java and xslt expert, with deep knowledge of Saxon xslt engine.

User
Please write extension function to call REST. Use HttpClient to communicate with service. Use xpath maps to pass data to and from the function.

I want that function to be flexible and allow not only to make basic calls but to send headers, cookies, including authorization.

Please model function result after http response. It should contain status, headers and payload.

Assistant
Sure, I'll provide you with a Java extension function that can be used with the Saxon XSLT engine to call a REST service. This function will use HttpClient to communicate with the service and will handle headers, cookies, and authorization. The function will return an XPath map containing the HTTP response status, headers, and payload.

First, ensure you have the necessary dependencies in your project. You will need the following:

1. Saxon-HE (or Saxon-PE/EE if you have a license)
2. Java HTTP Client (available in Java 11 and later)

Here is the Java code for the extension function:

import net.sf.saxon.lib.ExtensionFunctionDefinition;
import net.sf.saxon.lib.ExtensionFunctionCall;
import net.sf.saxon.om.Sequence;
import net.sf.saxon.om.SequenceType;
import net.sf.saxon.om.StructuredQName;
import net.sf.saxon.s9api.MapItem;
import net.sf.saxon.s9api.QName;
import net.sf.saxon.s9api.XdmMap;
import net.sf.saxon.s9api.XdmValue;
import net.sf.saxon.trans.XPathException;
import net.sf.saxon.value.SequenceType;
import net.sf.saxon.value.StringValue;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class RestExtensionFunction extends ExtensionFunctionDefinition {

    private static final StructuredQName qName = new StructuredQName("", "http://example.com/saxon-extension", "call-rest");

    @Override
    public StructuredQName getFunctionQName() {
        return qName;
    }

    @Override
    public SequenceType[] getArgumentTypes() {
        return new SequenceType[]{
                SequenceType.SINGLE_STRING, // URL
                SequenceType.SINGLE_STRING, // HTTP Method
                SequenceType.OPTIONAL_MAP_ITEM, // Headers
                SequenceType.OPTIONAL_MAP_ITEM, // Cookies
                SequenceType.OPTIONAL_STRING // Payload
        };
    }

    @Override
    public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) {
        return SequenceType.SINGLE_MAP_ITEM;
    }

    @Override
    public ExtensionFunctionCall makeCallExpression() {
        return new ExtensionFunctionCall() {
            @Override
            public Sequence call(XPathContext context, Sequence[] arguments) throws XPathException {
                try {
                    String url = ((StringValue) arguments[0].head()).getStringValue();
                    String method = ((StringValue) arguments[1].head()).getStringValue();
                    MapItem headersMap = arguments.length > 2 && arguments[2].head() instanceof MapItem ? (MapItem) arguments[2].head() : null;
                    MapItem cookiesMap = arguments.length > 3 && arguments[3].head() instanceof MapItem ? (MapItem) arguments[3].head() : null;
                    String payload = arguments.length > 4 && arguments[4].head() instanceof StringValue ? ((StringValue) arguments[4].head()).getStringValue() : null;

                    HttpClient client = HttpClient.newHttpClient();
                    HttpRequest.Builder requestBuilder = HttpRequest.newBuilder()
                            .uri(new URI(url))
                            .method(method, payload != null ? HttpRequest.BodyPublishers.ofString(payload) : HttpRequest.BodyPublishers.noBody());

                    if (headersMap != null) {
                        for (QName key : headersMap.keySet()) {
                            requestBuilder.header(key.getLocalName(), headersMap.get(key).getStringValue());
                        }
                    }

                    if (cookiesMap != null) {
                        StringBuilder cookies = new StringBuilder();
                        for (QName key : cookiesMap.keySet()) {
                            if (cookies.length() > 0) {
                                cookies.append("; ");
                            }
                            cookies.append(key.getLocalName()).append("=").append(cookiesMap.get(key).getStringValue());
                        }
                        requestBuilder.header("Cookie", cookies.toString());
                    }

                    HttpRequest request = requestBuilder.build();
                    HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());

                    Map<QName, XdmValue> resultMap = new HashMap<>();
                    resultMap.put(new QName("status"), new StringValue(String.valueOf(response.statusCode())));
                    resultMap.put(new QName("payload"), new StringValue(response.body()));

                    XdmMap headersResultMap = new XdmMap();
                    response.headers().map().forEach((key, values) -> {
                        headersResultMap = headersResultMap.put(new QName(key), new StringValue(String.join(", ", values)));
                    });
                    resultMap.put(new QName("headers"), headersResultMap);

                    return new XdmMap(resultMap).getUnderlyingValue();
                } catch (Exception e) {
                    throw new XPathException("Error calling REST service", e);
                }
            }
        };
    }
}

Explanation:¶

1. Function Definition: The RestExtensionFunction class extends ExtensionFunctionDefinition and defines the function name, argument types, and result type.
2. Arguments:
   • URL (String)
   • HTTP Method (String)
   • Headers (Optional Map)
   • Cookies (Optional Map)
   • Payload (Optional String)
3. Result: The function returns an XPath map containing the HTTP response status, headers, and payload.
4. HttpClient: The function uses HttpClient to send the HTTP request and receive the response.
5. Headers and Cookies: The function processes headers and cookies if provided.
6. Response Handling: The function constructs an XPath map from the HTTP response, including status, headers, and payload.

Usage in XSLT:¶

To use this extension function in your XSLT, you need to register the extension function with Saxon and call it from your XSLT stylesheet.

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:ext="http://example.com/saxon-extension"
                exclude-result-prefixes="ext"
                version="3.0">

    <xsl:import-schema namespace="http://example.com/saxon-extension" schema-location="path/to/your/schema.xsd"/>

    <xsl:template match="/">
        <xsl:variable name="response" select="ext:call-rest('https://api.example.com/data', 'GET', map{'Authorization': 'Bearer token'}, map{'session': 'abc123'}, '')"/>
        <xsl:variable name="status" select="$response?status"/>
        <xsl:variable name="headers" select="$response?headers"/>
        <xsl:variable name="payload" select="$response?payload"/>

        <xsl:message>HTTP Status: <xsl:value-of select="$status"/></xsl:message>
        <xsl:message>Response Payload: <xsl:value-of select="$payload"/></xsl:message>
    </xsl:template>
</xsl:stylesheet>

Registering the Extension Function:¶

You need to register the extension function with Saxon. This can be done programmatically in your Java code where you set up the Saxon processor.

import net.sf.saxon.s9api.Processor;
import net.sf.saxon.s9api.XsltCompiler;
import net.sf.saxon.s9api.XsltExecutable;
import net.sf.saxon.s9api.XsltTransformer;

public class Main {
    public static void main(String[] args) throws Exception {
        Processor processor = new Processor(false);
        processor.registerExtensionFunction(new RestExtensionFunction());

        XsltCompiler compiler = processor.newXsltCompiler();
        XsltExecutable executable = compiler.compile(new StreamSource(new File("path/to/your/stylesheet.xsl")));
        XsltTransformer transformer = executable.load();

        transformer.setSource(new StreamSource(new File("path/to/your/input.xml")));
        transformer.setDestination(new Serializer(System.out));
        transformer.transform();
    }
}

This setup allows you to call REST services from within your XSLT stylesheets using the custom extension function.

Looks a pretty good start. Can you ask it to generate some test cases in QT3 format?

Well, have no idea how QT3 format looks like but that's what it answers:

User
Good! Can you generate some test cases in QT3 format?

Assistant
Certainly! QT3 (XQuery Test Suite) is a format used for testing XQuery and XPath functions. Below are some test cases in QT3 format for the call-rest extension function. These test cases assume that you have a local or mock REST service running that can respond to the requests.

Test Case 1: Basic GET Request¶

<test-case name="test-call-rest-get">
    <description>Test basic GET request</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            ext:call-rest("http://localhost:8080/api/test", "GET", map{}, map{}, "")
        </xquery>
    </test>
    <result>
        <assert>
            <and>
                <assert-eq>
                    <value-of select="map:get($result, 'status')"/>
                    <value>200</value>
                </assert-eq>
                <assert-eq>
                    <value-of select="map:get($result, 'payload')"/>
                    <value>{"message": "Hello, World!"}</value>
                </assert-eq>
            </and>
        </assert>
    </result>
</test-case>

Test Case 2: POST Request with Payload¶

<test-case name="test-call-rest-post">
    <description>Test POST request with payload</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            ext:call-rest("http://localhost:8080/api/test", "POST", map{}, map{}, '{"name": "John"}')
        </xquery>
    </test>
    <result>
        <assert>
            <and>
                <assert-eq>
                    <value-of select="map:get($result, 'status')"/>
                    <value>201</value>
                </assert-eq>
                <assert-eq>
                    <value-of select="map:get($result, 'payload')"/>
                    <value>{"message": "User created"}</value>
                </assert-eq>
            </and>
        </assert>
    </result>
</test-case>

Test Case 3: GET Request with Headers¶

<test-case name="test-call-rest-get-with-headers">
    <description>Test GET request with headers</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            ext:call-rest("http://localhost:8080/api/test", "GET", map{"Authorization": "Bearer token"}, map{}, "")
        </xquery>
    </test>
    <result>
        <assert>
            <and>
                <assert-eq>
                    <value-of select="map:get($result, 'status')"/>
                    <value>200</value>
                </assert-eq>
                <assert-eq>
                    <value-of select="map:get($result, 'payload')"/>
                    <value>{"message": "Authorized"}</value>
                </assert-eq>
            </and>
        </assert>
    </result>
</test-case>

Test Case 4: GET Request with Cookies¶

<test-case name="test-call-rest-get-with-cookies">
    <description>Test GET request with cookies</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            ext:call-rest("http://localhost:8080/api/test", "GET", map{}, map{"session": "abc123"}, "")
        </xquery>
    </test>
    <result>
        <assert>
            <and>
                <assert-eq>
                    <value-of select="map:get($result, 'status')"/>
                    <value>200</value>
                </assert-eq>
                <assert-eq>
                    <value-of select="map:get($result, 'payload')"/>
                    <value>{"message": "Session valid"}</value>
                </assert-eq>
            </and>
        </assert>
    </result>
</test-case>

Test Case 5: Invalid URL¶

<test-case name="test-call-rest-invalid-url">
    <description>Test request with invalid URL</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            ext:call-rest("http://invalid-url", "GET", map{}, map{}, "")
        </xquery>
    </test>
    <result>
        <assert>
            <assert-error code="FOER0000"/>
        </assert>
    </result>
</test-case>

Test Case 6: GET Request with Response Headers¶

<test-case name="test-call-rest-get-response-headers">
    <description>Test GET request and check response headers</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            let $result := ext:call-rest("http://localhost:8080/api/test", "GET", map{}, map{}, "")
            return map:get($result, 'headers')
        </xquery>
    </test>
    <result>
        <assert>
            <assert-eq>
                <value-of select="map:get($result, 'Content-Type')"/>
                <value>application/json</value>
            </assert-eq>
        </assert>
    </result>
</test-case>

Test Case 7: POST Request with Headers and Payload¶

<test-case name="test-call-rest-post-with-headers-and-payload">
    <description>Test POST request with headers and payload</description>
    <created by="Your Name" on="2023-10-01"/>
    <environment>
        <dependency type="feature" value="http://example.com/saxon-extension/call-rest"/>
    </environment>
    <test>
        <xquery>
            declare namespace ext = "http://example.com/saxon-extension";
            ext:call-rest("http://localhost:8080/api/test", "POST", map{"Content-Type": "application/json"}, map{}, '{"name": "John"}')
        </xquery>
    </test>
    <result>
        <assert>
            <and>
                <assert-eq>
                    <value-of select="map:get($result, 'status')"/>
                    <value>201</value>
                </assert-eq>
                <assert-eq>
                    <value-of select="map:get($result, 'payload')"/>
                    <value>{"message": "User created"}</value>
                </assert-eq>
            </and>
        </assert>
    </result>
</test-case>

These test cases cover various scenarios including basic GET and POST requests, handling headers and cookies, and dealing with invalid URLs. You can run these tests using a QT3 test runner that supports Saxon and your custom extension function. Make sure to adjust the URLs and expected results according to your actual REST service responses.

I asked it to redo the task and to model function after JavaScript fetch(). It looks even more promising to me.

Just in case, here I put the link to saved chat: https://platform.openai.com/playground/chat?models=gpt-4o&preset=preset-g530S91pSok5MJsZGbePv7LJ

You may open and try it yourself. You will need to create OpenAI account though.

Pretty impressive attempt. So impressive that one wonders how it came up with the bits that are wrong, like the <xquery> tag.