Bug #5539
closed
Wasteful allocation of new ErrorReporter instances
100%
Description
Context information for a push pipeline with the Saxon internals is maintained in a PipelineConfiguration object. New instances of these class are allocated rather frequently (in one quite simple transformation, the method makePipelineConfiguration() was called 103639 times).
Each time a PipelineConfiguration is created, a new ErrorReporter is created, and the default ErrorReporter creates a new Logger, which allocates significant memory for I/O buffers. This shows up as the biggest source of memory allocation in the transformation.
Updated by Michael Kay almost 2 years ago
- Status changed from New to In Progress
The vast majority of calls on makePipelineConfiguration (103634 out of 103639) originate from Controller.allocateSequenceOutputter() and occur when switching from pull mode to push mode evaluation.
A simple fix would seem to be to create a PipelineConfiguration in which only the Controller is initialised, and allocate all other fields in the PipelineConfiguration on first reference. But there's probably scope for a more radical rethinking of the design.
Updated by Michael Kay almost 2 years ago
I've tried this change on the main branch; I'm getting no new test failures in XSLT3, but 22 failures in QT3.
Note, in the XSLT3 test suite about 90% of the calls on getPipelineConfiguration() appear to come from allocateSequenceOutputter(); in QT3 it seems to be more like 50%, so there's another significant caller somewhere.
Updated by Michael Kay almost 2 years ago
The 22 failures were for unrelated reasons, and have been fixed.
I think the change made here is probably sufficient to declare victory on this one.
Updated by Michael Kay almost 2 years ago
I've retrofitted the changes to the 11.x branch and I'm getting a couple of unit test failures, associated with schema validation and error reporting.
TestXslt30Transformer.testValidationWarnings() - seems to ignore the property Feature.VALIDATION_WARNINGS and treats the validation error as fatal. This property is copied into the Configuration's defaultParseOptions as property setContinueAfterValidationErrors. I think we're not setting up the parseOptions in the PipelineConfiguration correctly.
Fixed.
Updated by Michael Kay almost 2 years ago
Another failing unit test: JAXPTests / TransformWithValidation / testWithInputValidation. It expects an error count of 2 but only gets 1.
In Saxon 10 there are two calls on the ErrorListener one to report an actual error, and one to report "One error reported" at the end; this results in an errorCount of 2. In Saxon 11 we're only getting the second call. The first one is sent to a StandardErrorReporter, not to the user-supplied ErrorListener.
Comparing the Saxon 10 and 11 paths, looking at the revelant call Configuration.getDocumentValidator() (with validationMode == STRICT (1)) the 11 branch has validationOptions.errorReporter set incorrectly to a standard error reporter.
Stepping further back, on the revelant call to Sender.send(), the PipelineConfiguration in 10.x has getParseOptions().getErrorReporter() set correctly to an ErrorReporter that wraps the user-supplied ErrorListener; the equivalent in 11.x has errorReporter=null.
This comes from Controller.makeSourceTree(), which calls Controller.makeBuilder(), which calls Controller.makePipelineConfiguration(). In the new code this isn't setting the errorReporter.
Fixed by setting parser options (as previously) in Controller.makePipelineConfiguration().
Updated by Michael Kay almost 2 years ago
- Status changed from In Progress to Resolved
- Applies to branch 11, trunk added
- Fix Committed on Branch 11, trunk added
Fixed on the 11.x and 12.x branches
Updated by Debbie Lockett over 1 year ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in Maintenance Release 11.4 added
Bug fix applied in the Saxon 11.4 maintenance release.
Please register to edit this issue
.