Project

Profile

Help

Performance of SaxonHE 10.6 vs 11.2

Added by Vladimir Nesterovsky about 2 years ago

I have run several times, side by side, a small batch of transformations using Saxon HE 10.6 and then 11.2.
My timing show that version 11.2 is somewhat slower, though there is nothing critical.
In fact some transformations run even slightly faster.

Whole batch runs around 3 minutes.
Both runs produce exactly the same output.
Each xslt transformation spins a new JVM, so, each time Saxon starts in "cold" state.

If it is possible, and you will guide me please, then I can add some a code to instruct Saxon to print some timing to help to understand why there is a difference.

I've attached a report that displays xslt compilation and transformation times.


Replies (8)

Please register to reply

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Michael Kay about 2 years ago

That matches our own experience - some things a bit slower, some things a bit faster. Essentially, some paths benefit from the new UnicodeString machinery, but other paths incur a conversion cost between UnicodeString and String. We've looked very hard at the really common paths, like getting data from the parser and passing it to the serialiser, but there will be many others we haven't examined in detail, and any feedback will be welcome - as we discovered with bug #5297, there will be many places where there is room for improvement, perhaps substantial improvement.

Getting repeatable timings with Java is difficult, as I'm sure you know. For the top-level timing for a run of Transform or Query, use the -t option with -repeat:20 or -repeat:50 (unless it takes 10 minutes or so anyway). For specific internal paths, use a micro-benchmarking framework (though I tend to use my own), and watch out very carefully for false readings due to your code being optimisable in the way that real code isn't (for example through loop-lifting or function inlining.)

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Vladimir Nesterovsky about 2 years ago

I'm running transformation from my code using JAXP API. I can reach Configuration or other objects before the run.

Can I pass or setup something to have the effect of -t command line parameter? Is this related to TimingTraceListener?

In my load I see decrease of performance on the level 10-15%.

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Martin Honnen about 2 years ago

If you set the configuration http://saxon.sf.net/feature/timing / TIMING to true it has the same effect as -t on the command line : https://www.saxonica.com/html/documentation11/configuration/config-features.html which means "basic timing and tracing information to be output to the standard error output stream."

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Michael Kay about 2 years ago

The timer output achieved with -t is specific to command-line invocation. The -t option also sets the Configuration.TIMING property, which does some progress logging, but not the specific timer output you get from the command line.

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Vladimir Nesterovsky about 2 years ago

I took a trace of some transformation that differs in 2 seconds between versions.

Attached is top 10 time consumers.

I don't see any major difference but rather some general slowdown.

Functions reported as top consumers mostly process data and don't read xmls or other files.

Another observation is that one function (F5) is called 16100 times in 10.6 and 16003 in 11.2. Yet results are the same, so I attribute this to some optimization in 11.2.

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Vladimir Nesterovsky about 2 years ago

I did multiple runs, and in each run results stays consistent.

I tried to re-normalize results by total time of F8 (sequence manipulator) or F9 (sub tree builder), or even by F7 (slightly more complex subtree builder). In each case 10.6 stays faster.

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Norm Tovey-Walsh about 2 years ago

Hi Vladimir,

Mike suggested that I try doing a little profiling to see if we can find any clear areas for improvement. Can you share any of the tests that you're running (I've looked back through this thread and I didn't see any attachments, but apologies if I overlooked something).

--norm

RE: Performance of SaxonHE 10.6 vs 11.2 - Added by Vladimir Nesterovsky about 2 years ago

I've looked back through this thread and I didn't see any attachments, but apologies if I overlooked something

You're right, I did not give a demo of the problem. It's for two reasons:

  • I was not able to isolate the problem;
  • I cannot share my original code base, at least those transformations that run several minutes slower.

Yet, I can share stylesheet that is converting text log into html, which exhibits the same performance difference though it's less noticeable (fractions of seconds) but consistent between runs.

Here I shall attach stylesheet. Log itself I need to provide privately. Please note that this stylesheet is rather different from others, so it does not represent our workload.

On input it receives any xml and parameter "log" of type xs:anyURI that refers to a log file. On output it produces html.

Please advice the best way to send log file. It's small, less than 1MB of text.

    (1-8/8)

    Please register to reply