Saxon

I am trying to upgrade from 9-5-1-6J to 9-9-1-3J

and in the newer version my batch processing cannot complete successfully because of this issue.

I had to disable bytecode generation for performance reasons already in 9.5.1.6 using this:

        try {
            propertyName = "http://saxon.sf.net/feature/generateByteCode";
            localSchemaFactory.setProperty(propertyName, false);
        } catch (SAXNotSupportedException | SAXNotRecognizedException ex) {
            log.error(ex.getMessage());
        }

in my understanding this should also have prevented the assertion error which is supposed to be thrown exactly during bytecode generation. Unfortunately, it is not working. Perhaps the option name has changed or the method to set it? \

I am not using assertions in my code and by default the JVM keeps them switched them off.

The test results seem to show that the issue is not linked to concurrency issues, not to the data. Simply, at some point during the processing, the issue starts appearing for each processed document, as if it was a matter of buffers being filled at some point but I could not find any precise indication in the raised exception (see an example below)

The only way to prevent the whole process from being blown up is to explicitly catch the AssertionError. It looks unnatural to me. In addition, real validation errors cannot be reported any more.

        try {
            schemaValidator.validate(ss);
        } catch (AssertionError ex) {
            log.debug(ex.getMessage(), ex);
            throw new XMLValidationException(ex.getMessage());
        }

18 Jun 2019 19:37:37,160 INFO [pool-11-thread-4] (PullProcessorHelper.java:1844) - Processing resource 6 of 50 18 Jun 2019 19:37:37,160 INFO [pool-11-thread-4] (PullProcessorHelper.java:1849) - Processing file: E:\test\INSPIRE-6ce292b6-91ef-11e9-bf3c-901b0ea6780d_20190618-193512\services\1\PullResults\151-200\services\6\inspire.geoportal.resource.service.xml 18 Jun 2019 19:37:37,196 DEBUG [pool-11-thread-5] (SolrHelper.java:602) - Posting to http://geoportal-dev.ies.jrc.it/solr/core5/update 18 Jun 2019 19:37:37,211 DEBUG [pool-11-thread-5] (SolrHelper.java:625) - Post successful 18 Jun 2019 19:37:37,380 DEBUG [pool-11-thread-3] (Transformer.java:4847) - java.lang.AssertionError at com.saxonica.ee.schema.UserAtomicType.validateContent(UserAtomicType.java:404) at com.saxonica.ee.validate.SimpleContentValidator.endElement(SimpleContentValidator.java:237) at com.saxonica.ee.validate.ValidationStack.endElement(ValidationStack.java:401) at net.sf.saxon.event.ProxyReceiver.endElement(ProxyReceiver.java:185) at net.sf.saxon.event.StartTagBuffer.endElement(StartTagBuffer.java:308) at com.saxonica.ee.validate.StartTagBufferEE.endElement(StartTagBufferEE.java:58) at net.sf.saxon.event.PathMaintainer.endElement(PathMaintainer.java:58) at net.sf.saxon.event.DocumentValidator.endElement(DocumentValidator.java:69) at net.sf.saxon.event.ReceivingContentHandler.endElement(ReceivingContentHandler.java:528) at org.xml.sax.helpers.XMLFilterImpl.endElement(XMLFilterImpl.java:570) at eu.europa.ec.inspire.jaxb.XMLValidationNamespaceFilter.endElement(XMLValidationNamespaceFilter.java:82) at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.xml.sax.helpers.XMLFilterImpl.parse(XMLFilterImpl.java:357) at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:434) at net.sf.saxon.event.Sender.send(Sender.java:141) at com.saxonica.ee.jaxp.ValidatorImpl.validate(ValidatorImpl.java:70) at javax.xml.validation.Validator.validate(Validator.java:124) at eu.europa.ec.inspire.model.transform.Transformer.validateDocument(Transformer.java:4845) at eu.europa.ec.inspire.model.transform.Transformer.validateDocument(Transformer.java:4574) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.validateInspireResourceAgainstGeoportalSchema(INSPIREResourceRepresentation.java:3246) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.validateInspireResource(INSPIREResourceRepresentation.java:3095) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.saveResource(INSPIREResourceRepresentation.java:2118) at eu.europa.ec.inspire.model.INSPIREResourceRepresentation.saveResource(INSPIREResourceRepresentation.java:1958) at eu.europa.ec.inspire.resource.discovery.PullProcessorHelper.analyzeBatchOfResources(PullProcessorHelper.java:2212) at eu.europa.ec.inspire.resource.discovery.PullProcessorHelper.analyzeBatchOfResources(PullProcessorHelper.java:1572) at eu.europa.ec.inspire.resource.discovery.Analyse.call(PullProcessor.java:3832) at eu.europa.ec.inspire.resource.discovery.Analyse.call(PullProcessor.java:3729) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

I have been doing some investigations into the use of bytecode during schema validation when invoked via the JAXP SchemaFactory interface. I've been hitting some issues which at the moment I;m not sure are related to this issue - further investigation needed.

(For my reference, JUnit TestValidator./testByteCodeJaxp)

Corrections:

1. disabling byte code generation, prevents the assertion from being raised;
2. No processing exceptions are reported: neither with the schema nor with the validated documents

It would then seem to be indeed an issue with the bytecode generation.

This issue is no more blocking for us.

Would it be possible to supply a copy of the schema so we can establish why bytecode generation failed in the first place?

Yes, of course. I will zip the files and upload them here.

No, I have shared a OneDrive folder and sent you an email with the details.

The partial fix has been applied in the Saxon 9.9.1.4 maintenance release.

@Angelo, we never succeeded in downloading the files you made available because we were all in different locations and didn't manage to coordinate before the password expired. Any chance we could try again? It would be nice to get to the bottom of understanding why the bytecode generation failed, even though we have now fixed the recovery path so the failure is no longer critical.

@Michael, I have sent an email with a new link.

Best regards, Angelo