Project

Profile

Help

Bug #5978

closed

StackOverflow during multithreaded schema processing

Added by Michael Kay about 1 year ago. Updated 12 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Schema-Aware processing
Sprint/Milestone:
-
Start date:
2023-04-18
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
12, trunk
Fix Committed on Branch:
12, trunk
Fixed in Maintenance Release:
Platforms:
.NET, Java

Description

Reported via support email.

Actions #1

Updated by Michael Kay about 1 year ago

  • File deleted (ACSJournalXSD1-3-Dist.zip)
Actions #2

Updated by Michael Kay about 1 year ago

On first attempt, I can't reproduce the problem. It's an intermittent problem so that's not entirely surprising. Next step might be to try loading the schema concurrently in several threads into the same Configuration.

Actions #3

Updated by Michael Kay about 1 year ago

  • Category set to Schema-Aware processing
  • Priority changed from Low to Normal
Actions #4

Updated by Michael Kay about 1 year ago

I've now set up a test that is compiling the same schema in 100 threads in parallel (into the same Configuration) and I'm getting a variety of failures. I'm not getting a StackOverflow, but it's clearly not working properly and it wouldn't surprise me if a StackOverflow were a possible symptom.

Actions #5

Updated by Michael Kay about 1 year ago

  • Subject changed from StackOverflow during schema processing to StackOverflow during multithreaded schema processing
Actions #6

Updated by Michael Kay about 1 year ago

The bad news is that there appears to be no attempt to make schema compilation thread-safe. There doesn't seem to be any synchronization of the relevant methods on the Configuration for loading a schema, and the data structures used within the Configuration (which are updated incrementally as a schema is compiled) are regular HashMaps, rather than thread-safe varieties thereof.

I think the way this was supposed to work is that we process and validate a schema, to the maximum extent possible, in thread-local memory (specifically, in the SchemaCompiler) before copying the schema components over into shared configuration space. But it looks like pulling that design into shape might be significant effort. In the meantime, I guess we just have to do some fairly crude synchronization,

Actions #7

Updated by Michael Kay about 1 year ago

Synchronizing at the level of EnterpriseConfiguration.addSchemaSource() does the job (the test passes) but it would be nice to do better.

Also established that synchronizing at a finer-grained level isn't good enough. The problem is that we process each xs:import as we encounter it.

Actions #8

Updated by Michael Kay about 1 year ago

Raised internal issue #5982 to look at longer-term redesign options in this area.

Actions #9

Updated by Michael Kay 12 months ago

  • Status changed from New to Resolved
  • Applies to branch 12, trunk added
  • Fix Committed on Branch 12, trunk added
  • Platforms .NET, Java added

Marking this resolved because a patch has been raised that fixes the immediate problem. However there are remaining issues to tackle: schema compilation is not yet thread-safe.

Actions #10

Updated by O'Neil Delpratt 12 months ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in Maintenance Release 12.2 added

Bug fix applied in the Saxon 12.2 maintenance release

Please register to edit this issue

Also available in: Atom PDF