Saxon

Legacy ID: #7098640 Legacy Poster: Michael Kay (mhkay)

I'll take a look at the detail, but the first point I would make is that I would strongly advise against using DOM in performance-critical applications. Not only does it make heavier access to the NamePool, but it's also much less efficient than Saxon's native tree format in every other way. A transformation using the DOM is typically 4-10 times slower than one using the TinyTree, even discounting NamePool contention effects. (Also, when multi-threading, you hit the fact that the DOM isn't thread-safe, even when used in read-only mode.)

Legacy ID: #7100670 Legacy Poster: Kevan Simpson (kevan1138)

Would that we have a choice re: DOM... it is too much a requirement in our system to change. That said, the DOM nodes are not accessed by multiple threads, only the XsltExecutable intances. Would converting to TinyTree help performance? What is the recommended way to convert, especially when allowing for large payloads (as much as or more than 1MB)? Thanks for investigating the issue. ~Kevan

Legacy ID: #7113589 Legacy Poster: Michael Kay (mhkay)

Given that your source documents are so tiny, I suspect that the same problem would arise even without the DOM - Saxon uses caching to reduce NamePool access when building from a SAXSource or StreamSource, but it isn't going to help much with a document as small as this, because the cache starts empty for each new document. Clearly running with 50 threads isn't optimum from a performance point of view, you will get much better throughput with a lower concurrency. I don't think there's all that much I can suggest. I do have ideas as to how this code could be redesigned to reduce the bottlenecks and I've tried to persuade other people who are hitting the limits to make an investment in this area, but so far without much success. If it's worth it to you, though, do consider funding some development work. You could consider writing your own filter that strips down multiple DOM documents and passes them to Saxon's SAX input mechanism - converting many DOM documents into one Saxon document to take advantage of the caching. But it feels very inelegant and there's no guarantee it would work.

Legacy ID: #7113631 Legacy Poster: Michael Kay (mhkay)

>Would converting to TinyTree help performance? What is the recommended way to convert, especially when allowing for large payloads (as much as or more than 1MB)? Perhaps I was wrong in extrapolating that all your documents were very small. If you have larger documents, then it may well be worth converting them to TinyTree rather than wrapping them as you are doing at the moment. The simplest way to do that in 9.1 is to wrap your DOMSource in an AugmentedSource with the property setWrapDocument(false).

Legacy ID: #7127365 Legacy Poster: Kevan Simpson (kevan1138)

This issue may be important enough for us to invest the time to address the bottlenecks. Our workaround involves pooling XsltExecutable instances to diminish the concurrency of our application. Would you please elaborate on your ideas on redesign? Thanks, Kevan