Saxon

Thanks for logging this!

As far as I can tell, the CollectionFinder does exist and we can set one using processor.SetProperty(Feature<CollectionFinder>.COLLECTION_FINDER, new CustomCollectionFinder()) (or using transformer.Implementation.setCollectionFinder(new CustomCollectionFinder()) to be more localized) and its findCollection method will be called.

I just don't see how we could affect how documents are loaded, as the documentation seems to suggest it only returns URIs and not loaded documents.

Sorry to confuse. What I meant to say was that the default in the .NET product is to use the Java CollectionFinder, which of course has no knowledge of .NET-specifics like the XmlResolver. (I'm not even sure if that statement is correct, it needs further investigation).

Just checking, this CollectionFinder will allow us to set a custom .NET XmlResolver to be used when loading individual entries of the collection, right?

Yes, the CollectionFinder should use your custom XsltTransformer.InputXmlResolver. If you have a sample application I will happy to test your setup with this new feature.

We don't have a ready-to-use sample application, but we can make one.

Since we don't have the change yet, we ran the following with transform.exe collection.xsl -it:main - but I'm confident you can turn this into a .NET testcase:

collection.xsl (which simply looks at all XML files in the same folder and prints their root element name)

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs" version="2.0">

    <xsl:template name="main">
        <xsl:apply-templates select="collection('./?select=*.xml')" mode="print-root" />
    </xsl:template>

    <xsl:template match="/" mode="print-root">
        <xsl:text>&#xA;</xsl:text>
        <xsl:value-of select="name(/*)" />
    </xsl:template>

</xsl:stylesheet>

a.xml in same folder as collection.xsl (the DTD does not really matter, but removing it allows the XSLT to run)

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE test SYSTEM "does-not-exist.dtd">
<a/>

You can add more XML files if you want, both with and without DOCTYPE, where the ones with a System ID trigger the exception from this issue.

In C#, we'd use a custom XmlResolver like this:

public class DtdIgnoringResolver : XmlResolver
{
    private readonly XmlResolver _innerResolver;

    public DtdIgnoringResolver(XmlResolver innerResolver)
    {
        _innerResolver = innerResolver ?? throw new ArgumentNullException(nameof(innerResolver));
    }

    public override ICredentials Credentials
    {
        set { _innerResolver.Credentials = value; }
    }

    public override object GetEntity(Uri absoluteUri, string role, Type ofObjectToReturn)
    {
        if (!string.IsNullOrEmpty(absoluteUri?.OriginalString) && IsDtdOrSchema(absoluteUri.OriginalString))
            return Stream.Null;
        return _innerResolver.GetEntity(absoluteUri, role, ofObjectToReturn);
    }

    private static bool IsDtdOrSchema(string filePath)
    {
        return
            filePath.EndsWith(".dtd", StringComparison.InvariantCultureIgnoreCase) ||
            filePath.EndsWith(".xsd", StringComparison.InvariantCultureIgnoreCase);
    }
}

The piece of code we use is a bit more involved and usually applies the XSLT to a file (rather than running a named template), but I'm sure this suffices. If not, let me know.

Thanks again.

Hi,

Thanks for sending this example, which I used to create a nunit test case. I can confirm it now works as expected. No longer seeing the exception.

Just checking, this should be available with Saxon-HE 10.5.1N, and it should just work if we set XsltTransformer.InputXmlResolver (without having to override the CollectionFinder) - right? Do note that we're not (yet) using Xslt30Transformer - mainly because we assumed they'd be equivalent anyways; but even when changing the code to use Xslt30Transformer we see the same exception as reported initially (I/O error because the DTD is not found; with our DtdIgnoringResolver not being called)

So far, we were unsuccessful in getting this to work; and the Repository only has 10.3 available so I can't look at your tests.

Is there anything we're missing?

Emanuel Wlaschitz wrote:

So far, we were unsuccessful in getting this to work; and the Repository only has 10.3 available so I can't look at your tests.

I think the repository should now be tat e.g. https://saxonica.plan.io/projects/saxonmirrorhe/repository/he/revisions/saxon10/entry/src/test/nunit/SaxonNUnit/SaxonNUnit/TestCollection.cs#L694 but I am not sure it is really in sync.

Martin Honnen wrote:

I think the repository should now be tat e.g. https://saxonica.plan.io/projects/saxonmirrorhe/repository/he/revisions/saxon10/entry/src/test/nunit/SaxonNUnit/SaxonNUnit/TestCollection.cs#L694 [...]

Thanks, I didn't realize this was a separate repository/project/whatever.

Martin Honnen wrote:

[...] but I am not sure it is really in sync.

It does look like it is missing the testdata, /src/test/testdata has up until collection3.xsl, but I cannot find collection4.xsl (which is used by that test).

The main difference between the test and our code is the fact that the test sets XsltCompiler.XmlResolver before calling XsltCompiler.Compile() while we attempted to set XsltTransformer.InputXmlResolver after XsltExecutable.Load() (or the equivalent for Xslt30Transformer). Using XsltCompiler.XmlResolver seems to work, but we'll have to make a few adjustments to make it pretty.

Fortunately, it doens't make a difference for us (because we don't reuse the XsltCompiler; we only reuse the Processor since we assumed it was a heavy class to create), but it really needs some documentation on how this works (especially since it differs from what is seen in this ticket).

Or was it supposed to also work when setting it on the XsltTransformer.InputXmlResolver? Naively I'd consider anything loaded by the Stylesheet (using document()/doc(), through collection() etc.) an Input, assuming it would go through that resolver, so the current behavior is a bit surprising.

The standard (default) CollectionFinder simply delegates to the underlying Java code. Unless this is customised (for example with a user-defined ResourceFactory for a particular media type) it will not use any URIResolver or XmlResolver to dereference the URIs of individual resources in the collection. The Java product offers a lot more exposed capability for customisation here than the .NET product, for example by defining additional types of ResourceCollection and associated ResourceFactory's - this is all available in theory to the .NET user, but it's a lot more deeply buried.

If you set your own CollectionFinder then of course it's entirely up to you what it does. You can either return an instance of one of Saxon's IResourceCollection implementations (for example DirectoryCollection) or you can implement IResourceCollection yourself. There are two methods to implement: getResources which underpins fn:collection(), and getResourceURIs which underpins fn:uri-collection. You then have another level of flexibility in that getResources returns a set of IResource objects which you can implement any way you choose, in particular it's entirely up to you how to dereference a URI to get a resource and turn it into an XDM Item. You can of course use an XmlResolver if it suits you to do so.

If the user calls fn:uri-collection() they can then dereference the returned URIs using (say) doc() or unparsed-text() or json-doc() and in the case of doc() this will of course invoke the run-time XmlResolver.

I hope this helps: it's a complex piece of machinery.

Yeah, I realize this is a fairly complex thing with many gears turning. Our primary use case here is (still) loading files from a folder using collection($uri?select=$pattern), which simply fails (or rather: failed) when files in there use a DOCTYPE declaration with SYSTEM id that isn't available. And the most convinient way for us (which we also use for the direct inputs that we fully control) is to plug a .NET XmlResolver in there that simply intercepts those and says "nope" for any DTD or XSD it encounters.

So in the end, the default CollectionFinder is suitable for our needs (in part because the code is already there and we'd just be re-implementing it in C#) under the assumption that its default getResources implementation uses the XmlResolver set anywhere up the tree.

👍 We've verified this is the case for XsltCompiler.XmlResolver when we set it before compiling the stylesheet.

👎 We couldn't get it to work when we only set XsltTransformer.InputXmlResolver (or Xslt30Transformer.InputXmlResolver), which was the initial assumption after reading O'Neils Note #10.

And thats why I commented on this issue again (since we were as stuck as before, not knowing how to support our use case properly). If this is now the intended way (using XsltCompiler.XmlResolver), I'm ok with it - I just wanted confirmation before making many cascading changes to support this on our side that would end up obsolete in another update.