Project

Profile

Help

Bug #5539

closed

Wasteful allocation of new ErrorReporter instances

Added by Michael Kay about 1 month ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Performance
Sprint/Milestone:
-
Start date:
2022-05-25
Due date:
% Done:

0%

Estimated time:
Legacy ID:
Applies to branch:
11, trunk
Fix Committed on Branch:
11, trunk
Fixed in Maintenance Release:
Platforms:

Description

Context information for a push pipeline with the Saxon internals is maintained in a PipelineConfiguration object. New instances of these class are allocated rather frequently (in one quite simple transformation, the method makePipelineConfiguration() was called 103639 times).

Each time a PipelineConfiguration is created, a new ErrorReporter is created, and the default ErrorReporter creates a new Logger, which allocates significant memory for I/O buffers. This shows up as the biggest source of memory allocation in the transformation.

Actions #1

Updated by Michael Kay about 1 month ago

  • Status changed from New to In Progress

The vast majority of calls on makePipelineConfiguration (103634 out of 103639) originate from Controller.allocateSequenceOutputter() and occur when switching from pull mode to push mode evaluation.

A simple fix would seem to be to create a PipelineConfiguration in which only the Controller is initialised, and allocate all other fields in the PipelineConfiguration on first reference. But there's probably scope for a more radical rethinking of the design.

Actions #2

Updated by Michael Kay about 1 month ago

I've tried this change on the main branch; I'm getting no new test failures in XSLT3, but 22 failures in QT3.

Note, in the XSLT3 test suite about 90% of the calls on getPipelineConfiguration() appear to come from allocateSequenceOutputter(); in QT3 it seems to be more like 50%, so there's another significant caller somewhere.

Actions #3

Updated by Michael Kay about 1 month ago

The 22 failures were for unrelated reasons, and have been fixed.

I think the change made here is probably sufficient to declare victory on this one.

Actions #4

Updated by Michael Kay about 1 month ago

I've retrofitted the changes to the 11.x branch and I'm getting a couple of unit test failures, associated with schema validation and error reporting.

TestXslt30Transformer.testValidationWarnings() - seems to ignore the property Feature.VALIDATION_WARNINGS and treats the validation error as fatal. This property is copied into the Configuration's defaultParseOptions as property setContinueAfterValidationErrors. I think we're not setting up the parseOptions in the PipelineConfiguration correctly.

Fixed.

Actions #5

Updated by Michael Kay about 1 month ago

Another failing unit test: JAXPTests / TransformWithValidation / testWithInputValidation. It expects an error count of 2 but only gets 1.

In Saxon 10 there are two calls on the ErrorListener one to report an actual error, and one to report "One error reported" at the end; this results in an errorCount of 2. In Saxon 11 we're only getting the second call. The first one is sent to a StandardErrorReporter, not to the user-supplied ErrorListener.

Comparing the Saxon 10 and 11 paths, looking at the revelant call Configuration.getDocumentValidator() (with validationMode == STRICT (1)) the 11 branch has validationOptions.errorReporter set incorrectly to a standard error reporter.

Stepping further back, on the revelant call to Sender.send(), the PipelineConfiguration in 10.x has getParseOptions().getErrorReporter() set correctly to an ErrorReporter that wraps the user-supplied ErrorListener; the equivalent in 11.x has errorReporter=null.

This comes from Controller.makeSourceTree(), which calls Controller.makeBuilder(), which calls Controller.makePipelineConfiguration(). In the new code this isn't setting the errorReporter.

Fixed by setting parser options (as previously) in Controller.makePipelineConfiguration().

Actions #6

Updated by Michael Kay about 1 month ago

  • Status changed from In Progress to Resolved
  • Applies to branch 11, trunk added
  • Fix Committed on Branch 11, trunk added

Fixed on the 11.x and 12.x branches

Please register to edit this issue

Also available in: Atom PDF