Saxon

This is not a known problem, so we would need to reproduce it in order to create a fix.

However, there are known problems running Saxon in an OSGi environment. See for example your previous (unresolved) issue #4213 (and others found by searching for OSGi). However, OSGi could be a red herring here; it would be good to see if the problem can be reproduced without OSGi. Clearly, paths in the code involving exceptionally large XML trees are not going to be exercised as thoroughly in our test suites, so a bug in this area is by no means impossible.

Is there a stack trace?

From the information supplied, it looks as if it's happening during TinyBuilder.makeTextNode() and it's hard to see anything on that path that would be sensitive to the data size. But usually with bugs like this, the only way to solve them is to first reproduce them.

One additional observation is that with "huge" XML files, over a gigabyte, we start to hit built-in Java limits on the sizes of strings and arrays. We circumvent some of these limits, for example by holding the character content of a large document as an array of arrays, but we can't work around them all. It's very likely, for example, that either Saxon or the underlying XML parser will crash if a single text node exceeds the maximum size of a Java string. Apart from the problem that Saxon can't solve these problems on its own, because they involve dependencies, there is also a problem that testing documents of this size comprehensively becomes very impractical.

Having said that, the symptoms you have described don't suggest that this is the problem in this particular case.

Hi Michael,

As the issue is not consistently reproducible, we don't have detailed stack traces for this. Could you figure out from the provided error log, what could be the probable reason for this issue?

Also, wanted to know is the TinyTree mechanism is part of some performance optimization mechanism by Saxon? And if that's the case can we turn that feature Off by some mechanism? Because we have never seen this issue for any other customer before and also with an XML payload <1mb (approx), we are not getting this issue.

-Pranay

I would hate to walk away from this problem without solving it, but if it's not reproducible "in the lab" then our chances of solving it (& testing a solution!) are very low. The use of OSGi may be completely irrelevant, but it's always a cause for suspicion.

Certainly, you can try to run with a different tree model, such as Saxon's Linked Tree. It will probably use quite a bit more memory, but should work. You can choose the tree model used for building each document; the details depend on what API you are using to construct the tree.

I've looked again carefully at the code in this area, and I can't see anything wrong that would trigger these symptoms.

Do you happen to know whether the document is large because it contains many small text nodes, or if it contains a small number of very large text nodes? If we knew something of the profile, we could at least attempt some testing.

Hi Michael,

We are still facing this issue and below are the detailed traces:

java.lang.NullPointerException: while trying to invoke the method net.sf.saxon.tree.tiny.TinyTree.getCharacterBuffer() of a null object loaded from local variable 'tt' at net.sf.saxon.tree.tiny.TinyBuilder.makeTextNode(TinyBuilder.java:427) at net.sf.saxon.tree.tiny.TinyBuilder.characters(TinyBuilder.java:406) at net.sf.saxon.event.ProxyReceiver.characters(ProxyReceiver.java:193) at net.sf.saxon.event.ReceivingContentHandler.flush(ReceivingContentHandler.java:646) at net.sf.saxon.event.ReceivingContentHandler.startElement(ReceivingContentHandler.java:364) at org.apache.cxf.staxutils.StaxSource.parse(StaxSource.java:155) at org.apache.cxf.staxutils.StaxSource.parse(StaxSource.java:268) at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:434) at net.sf.saxon.event.Sender.send(Sender.java:141) at net.sf.saxon.Configuration.buildDocumentTree(Configuration.java:4237) at net.sf.saxon.expr.JPConverter$FromSource.convert(JPConverter.java:608) at net.sf.saxon.expr.JPConverter$FromSource.convert(JPConverter.java:598) at net.sf.saxon.xpath.XPathExpressionImpl.evaluate(XPathExpressionImpl.java:173)

Could you please give us some hints on what could be the possible root cause of this issue. I can send you the payload XML and the Xpaths the user is trying to evaluate, over the mail.

Regards Pranay

I note this comment in Saxon's StaxBridge class, which accepts input from a StAX parser:

        if (currentEvent == START_OF_INPUT) {
            // StAX isn't reporting START_DOCUMENT so we supply it ourselves
            currentEvent = START_DOCUMENT;
            return currentEvent;
        }

This code is clearly written to cope with a situation where a Stax parser doesn't report the START_DOCUMENT event, so we know this is something that can happen.

The Javadoc for XMLStreamReader doesn't actually say whether you are guaranteed to see a START_DOCUMENT event. By contrast the SAX ContentHandler javadoc makes it pretty clear that the ContentHandler can expect to see one.