Project

Profile

Help

Bug #4237

Assertion error when bytecode generation fails

Added by Michael Kay 3 months ago. Updated 13 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
2019-06-18
Due date:
% Done:

100%

Legacy ID:
Applies to branch:
9.9, trunk
Fix Committed on Branch:
9.9, trunk
Fixed in Maintenance Release:

Description

When bytecode generation for validation against a simple type fails, the recovery path includes a Java "assert" statement which causes a crash on the recovery path if Java assertions are enabled.

Caused by: java.lang.AssertionError
    at com.saxonica.ee.schema.UserAtomicType.validateContent(UserAtomicType.java:404)
    at com.saxonica.ee.validate.SimpleContentValidator.endElement(SimpleContentValidator.java:237)
    at com.saxonica.ee.validate.ValidationStack.endElement(ValidationStack.java:401)

History

#1 Updated by Angelo Quaglia 3 months ago

I am trying to upgrade from 9-5-1-6J to 9-9-1-3J

and in the newer version my batch processing cannot complete successfully because of this issue.

I had to disable bytecode generation for performance reasons already in 9.5.1.6 using this:

        try {
            propertyName = "http://saxon.sf.net/feature/generateByteCode";
            localSchemaFactory.setProperty(propertyName, false);
        } catch (SAXNotSupportedException | SAXNotRecognizedException ex) {
            log.error(ex.getMessage());
        }

in my understanding this should also have prevented the assertion error which is supposed to be thrown exactly during bytecode generation. Unfortunately, it is not working. Perhaps the option name has changed or the method to set it? \

#2 Updated by Angelo Quaglia 3 months ago

I am not using assertions in my code and by default the JVM keeps them switched them off.

The test results seem to show that the issue is not linked to concurrency issues, not to the data. Simply, at some point during the processing, the issue starts appearing for each processed document, as if it was a matter of buffers being filled at some point but I could not find any precise indication in the raised exception (see an example below)

The only way to prevent the whole process from being blown up is to explicitly catch the AssertionError. It looks unnatural to me. In addition, real validation errors cannot be reported any more.

        try {
            schemaValidator.validate(ss);
        } catch (AssertionError ex) {
            log.debug(ex.getMessage(), ex);
            throw new XMLValidationException(ex.getMessage());
        }

18 Jun 2019 19:37:37,160 INFO [pool-11-thread-4] (PullProcessorHelper.java:1844) - Processing resource 6 of 50 18 Jun 2019 19:37:37,160 INFO [pool-11-thread-4] (PullProcessorHelper.java:1849) - Processing file: E:\test\INSPIRE-6ce292b6-91ef-11e9-bf3c-901b0ea6780d_20190618-193512\services\1\PullResults\151-200\services\6\inspire.geoportal.resource.service.xml 18 Jun 2019 19:37:37,196 DEBUG [pool-11-thread-5] (SolrHelper.java:602) - Posting to http://geoportal-dev.ies.jrc.it/solr/core5/update 18 Jun 2019 19:37:37,211 DEBUG [pool-11-thread-5] (SolrHelper.java:625) - Post successful 18 Jun 2019 19:37:37,380 DEBUG [pool-11-thread-3] (Transformer.java:4847) - java.lang.AssertionError at com.saxonica.ee.schema.UserAtomicType.validateContent(UserAtomicType.java:404) at com.saxonica.ee.validate.SimpleContentValidator.endElement(SimpleContentValidator.java:237) at com.saxonica.ee.validate.ValidationStack.endElement(ValidationStack.java:401) at net.sf.saxon.event.ProxyReceiver.endElement(ProxyReceiver.java:185) at net.sf.saxon.event.StartTagBuffer.endElement(StartTagBuffer.java:308) at com.saxonica.ee.validate.StartTagBufferEE.endElement(StartTagBufferEE.java:58) at net.sf.saxon.event.PathMaintainer.endElement(PathMaintainer.java:58) at net.sf.saxon.event.DocumentValidator.endElement(DocumentValidator.java:69) at net.sf.saxon.event.ReceivingContentHandler.endElement(ReceivingContentHandler.java:528) at org.xml.sax.helpers.XMLFilterImpl.endElement(XMLFilterImpl.java:570) at eu.europa.ec.inspire.jaxb.XMLValidationNamespaceFilter.endElement(XMLValidationNamespaceFilter.java:82) at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.xml.sax.helpers.XMLFilterImpl.parse(XMLFilterImpl.java:357) at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:434) at net.sf.saxon.event.Sender.send(Sender.java:141) at com.saxonica.ee.jaxp.ValidatorImpl.validate(ValidatorImpl.java:70) at javax.xml.validation.Validator.validate(Validator.java:124) at eu.europa.ec.inspire.model.transform.Transformer.validateDocument(Transformer.java:4845) at eu.europa.ec.inspire.model.transform.Transformer.validateDocument(Transformer.java:4574) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.validateInspireResourceAgainstGeoportalSchema(INSPIREResourceRepresentation.java:3246) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.validateInspireResource(INSPIREResourceRepresentation.java:3095) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.saveResource(INSPIREResourceRepresentation.java:2118) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.saveResource(INSPIREResourceRepresentation.java:1958) at eu.europa.ec.inspire.resource.discovery.PullProcessorHelper.analyzeBatchOfResources(PullProcessorHelper.java:2212) at eu.europa.ec.inspire.resource.discovery.PullProcessorHelper.analyzeBatchOfResources(PullProcessorHelper.java:1572) at eu.europa.ec.inspire.resource.discovery.Analyse.call(PullProcessor.java:3832) at eu.europa.ec.inspire.resource.discovery.Analyse.call(PullProcessor.java:3729) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

#3 Updated by Michael Kay 3 months ago

I have been doing some investigations into the use of bytecode during schema validation when invoked via the JAXP SchemaFactory interface. I've been hitting some issues which at the moment I;m not sure are related to this issue - further investigation needed.

(For my reference, JUnit TestValidator./testByteCodeJaxp)

#4 Updated by Angelo Quaglia 3 months ago

Corrections: 1) disabling byte code generation, prevents the assertion from being raised; 2) No processing exceptions are reported: neither with the schema nor with the validated documents

It would then seem to be indeed an issue with the bytecode generation.

This issue is no more blocking for us.

#5 Updated by Michael Kay 3 months ago

Would it be possible to supply a copy of the schema so we can establish why bytecode generation failed in the first place?

#6 Updated by Angelo Quaglia 3 months ago

Yes, of course. I will zip the files and upload them here.

#7 Updated by Angelo Quaglia 3 months ago

No, I have shared a OneDrive folder and sent you an email with the details.

#8 Updated by Michael Kay 3 months ago

  • Fix Committed on Branch 9.9, trunk added

We hit some logistical problems downloading the data and we need to come back to this.

We're preparing a 9.9 maintenance release, and in this I have include a fix to the most conspicuous problem, the throwing of an assertion error. It should now be the case that recoverable errors in bytecode generation for simple types are handled silently, unless the debugByteCode option is set.

#9 Updated by Michael Kay 3 months ago

  • Status changed from New to In Progress

#10 Updated by O'Neil Delpratt 3 months ago

The partial fix has been applied in the Saxon 9.9.1.4 maintenance release.

#12 Updated by Michael Kay about 1 month ago

@Angelo, we never succeeded in downloading the files you made available because we were all in different locations and didn't manage to coordinate before the password expired. Any chance we could try again? It would be nice to get to the bottom of understanding why the bytecode generation failed, even though we have now fixed the recovery path so the failure is no longer critical.

#13 Updated by Angelo Quaglia 26 days ago

@Michael, I have sent an email with a new link.

Best regards, Angelo

#14 Updated by Michael Kay 14 days ago

  • Status changed from In Progress to Resolved
  • Priority changed from Low to Normal
  • Applies to branch 9.9, trunk added

I've forced bytecode generation for every atomic type in the supplied schemas and the compilation is working without problems, so whatever the original problem was, it seems to have been fixed. Closing with no further action.

#15 Updated by O'Neil Delpratt 13 days ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in Maintenance Release 9.9.1.5 added

Bug fix applied in the Saxon 9.9.1.5 maintenance release.

Please register to edit this issue

Also available in: Atom PDF