Project

Profile

Help

Bug #2165

Commandline -repeat:N not working properly

Added by Aleksander Karro almost 5 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Command Line
Sprint/Milestone:
Start date:
2014-10-07
Due date:
% Done:

100%

Legacy ID:
Applies to branch:
9.6
Fix Committed on Branch:
9.6
Fixed in Maintenance Release:

Description

I'm seeing weird behaviour when trying to benchmark Saxon 9.6.0.1 ee using the following command line:

java -server -Xmx2g -jar saxon96ee.jar -repeat:30 -o:result.xml test.xml test.xsl

When I use -repeat:30 I get:

* Average execution time over last 15 runs: 801.00544msms

Then again when I use -repeat:25 I get:

* Average execution time over last 13 runs: 829.111915msms

Again with -repeat:15 I get:

* Average execution time over last 8 runs: 817.645476msms

Also notice that the output has ms twice. When I run this with latest 9.5.1.6 version with:

java -server -Xmx2g -jar saxon95ee.jar -repeat:30 -o:result.xml test.xml test.xsl

I get:

* Average execution time over 30 runs: 765ms

My Java version is:

java version "1.8.0_20"

Java(TM) SE Runtime Environment (build 1.8.0_20-b26)

Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode)

Running on Windows 7 x64 SP1

History

#1 Updated by Michael Kay almost 5 years ago

  • Assignee set to Michael Kay
  • Found in version changed from 9.6.0.1 to 9.6

The repeated "ms" in the message is a simple bug.

The fact that if you request 30 repetitions, we report the average over the last 15 is by design. The purpose of using repeated runs is to discount the effect of Java VM warmup time, and we found that averaging over all the runs didn't do this very effectively, while measuring over the last half of the runs gives a much more reliable figure.

#2 Updated by Michael Kay almost 5 years ago

  • Status changed from New to Resolved

#3 Updated by Michael Kay almost 5 years ago

A patch for the badly formatted message has been committed and tested on the 9.6 and 9.7 branches.

#4 Updated by Aleksander Karro almost 5 years ago

Thanks for clarifying that. It makes sense now, although it's a bit confusing that the previous version would show the average over 30 last runs. I couldn't find anything in the change log about it under the Command Line interface section. Maybe it's worthwhile adding it as a note to the docs?

#5 Updated by O'Neil Delpratt almost 5 years ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in version set to 9.6.0.2

Bug fix applied to the maintenance release Saxon 9.6.0.2

#6 Updated by O'Neil Delpratt almost 4 years ago

  • Sprint/Milestone set to 9.6.0.2
  • Applies to branch 9.6 added
  • Fix Committed on Branch 9.6 added
  • Fixed in Maintenance Release 9.6.0.2 added

Please register to edit this issue

Also available in: Atom PDF