Saxon

Information supplied by user:

Regarding the NPE: I use Saxon via its Java API. I have a custom CollectionFinder class. When Saxon run XQuery against collection(-s) I analyze expression tree and return back custom ResourceCollection which select documents dynamically from my external data store. So, the NPE happens even before I select any source document.

A piece of code where NPE happens is:

   if (ex instanceof GeneralComparison10 || ex instanceof GeneralComparison20 || ex instanceof ValueComparison) {
        BinaryExpression be = (BinaryExpression) ex;
        Object value = null;
        String pName = null;
        for (Operand o: be.operands()) {
            Expression e = o.getChildExpression();
            if (e instanceof VariableReference) {
                Binding bind = ((VariableReference) e).getBinding();
                if (bind instanceof LetExpression) {
                    Expression e2 = ((LetExpression) bind).getSequence();
                    if (e2 instanceof Atomizer) {
                        e2 = ((Atomizer) e2).getBaseExpression();
                        if (e2 instanceof VariableReference) {
                            // paired ref to the e
                            pName = ((VariableReference) e2).getBinding().getVariableQName().getLocalPart(); 
                        }
                    }
                }

                if (pName == null) {
                    pName = bind.getVariableQName().getClarkName();
                }
                value = getValues(bind.evaluateVariable(ctx)); // NPE happens here!
                break;
            } else if (e instanceof StringLiteral) {
                value = ((StringLiteral) e).getStringValue();
                break;
            } else if (e instanceof Literal) {
                value = getValues(((Literal) e).getValue()); 
                break;
            }
        }

It's not at all clear to me what you are trying to achieve here, but if you are writing Java code that accesses the expression tree at compile time then you're going to have to be prepared to debug the odd exception, because we make no claim that the methods at this level of the product are completely documented, stable across releases, or robust against arbitrary sequences of calls.

Your particular NPE occurs when you attempt to evaluate a variable on the expression tree. In general variables can't be evaluated at compile time. It's not surprising to me that this should fail. It could also happen that you get an NPE here because "bind" is null: I don't know when you are calling this code, it could be before the variable reference has been linked to the relevant variable binding.

You write: "The implementation of MappingIterator.next() method looks confusing: it returns null result even when it properly fetch nextSource from the underlying base iterator."

I admire anyone who can get as deep into the code as this and have any kind of understanding of what's going on; it's horrendously complex in places and I'm not at all surprised you are confused. Remember that when the base iterator of a mapping iterator returns a next item, it's possible that the mapping function maps that item to an empty sequence, in which case the mapping iterator loops round to select the next item from the base iterator, or returns null if there is no next item. The comments in the code suggest that the mapping function can indicate an empty sequence either by returning an empty iterator or by returning null. I suspect the option of returning null is no longer used.

I think it would be useful if you explained to us what you are trying to achieve. There may be a much more sanitary way of doing it.

Hi Michael,

Ok, I'll try to explain. The project I'm working on is: https://github.com/dsukhoroslov/bagri. All XML data is sliced and stored in distributed cache. Clients can access the system via XQJ or via proprietary interface. Saxon-HE is used as XQuery engine. I registered a number of custom resolvers (CollectionFinder, SourceResolver, URIResolver) in order to provide to Saxon XML documents dynamically from the cache.

The major point of intergation is my custom CollectionFinder implementation. Here (in the findCollection method) I analyze expression tree and build from it my own set of query expressions against distributed cache. The set of expressions is suplied to the resulting ResourceCollection, so it can dynamically fetch Resources from cache.

The most usual query processing scenario is:

        // _CollectionFinder clnFinder, already registered with Configuration_

        XQueryExpression xqExp = sqc.compileQuery(query);

        clnFinder.setExpression(xqExp); //_ clnFinder got already compiled expression_

        SequenceIterator itr = xqExp.iterator(dqc); // _clnFinder.findCollection called at this time, after expression compilation_

        return new XQIterator(itr); // _iteration on resources happens in this iterator_

when it works on query with only one collection then everything is simple and works fine. I have issues with join case between several collections (see the query above).

Well, the code to analyse expression tree is awkward, but is does work somehow.. most of the time :)..

..you get an NPE here because "bind" is null

no, bind is not null. The NPE happens in Assignation class:

public Sequence evaluateVariable(XPathContext context) throws XPathException {

    Sequence actual = context.evaluateLocalVariable(slotNumber); //_ returned actual is null_

    if (!(actual instanceof GroundedValue || actual instanceof NodeInfo)) {

        actual = SequenceTool.toGroundedValue(actual);

        context.setLocalVariable(slotNumber, actual);

    }

    return actual;

}

it happens for the join with second collection, so variable reference could be not linked with binding yet. Ok, I'll handle NPE myself.

I admire anyone who can get as deep into the code

It's not me, it's my debugger :).

.. it's possible that the mapping function maps that item to an empty sequence

yes, this is what exactly happens in my case. I found the reason why: I haven't specified collection uris, so there was no second call to collection finder and no way to find the Symbol value to join on.

Thank you so much for the explanations.

Denis.

Michael, could you suggest any better solution for expression tree analysis?

Regarding the issue itself (Bug #2695) - as it is not a bug, let's close it?

Thanks, Denis.

Ok, it makes sense. Thank you so much for the detailed answer. I'l think about Saxon XML view option..

Thanks, Denis.