Project

Profile

Help

Actions
Send by e-mail Copy link

Bug #3840

closed

Migrating from sourceforge saxon to Saxonica Saxon-EE, problem with sorting xslt

Added by Rick Vlaming almost 6 years ago. Updated almost 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Michael Kay
Category:
Internals
Sprint/Milestone:
-
Start date:
2018-07-14
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
9.8, trunk
Fix Committed on Branch:
9.8, trunk
Fixed in Maintenance Release:
9.8.0.14
Platforms:

Description

We are switching from sourceforge saxon 9.1.0.8 to saxonica saxon-ee 9.8.0.12. We are having trouble with a xslt which task is to translate to a sorted xml. The same xslt, which we used against the sourceforge saxon, is given an error using against the saxon-ee parser. There is no output.

We are developing using XML-Spy where we set saxon-ee as the external XSL transformation program to use via the following setting: java -jar C:\rick\xslt\saxon-ee-9.8.0.12.jar -s:%1 -o:%2 -xsl:%3

The error says: Error at char 9 in xsl:apply-templates/@select on line 17 column 44 of postXslt.xslt: Cannot evaluate net.sf.saxon.expr.sort.SortKeyDefinition in built-in template rule for /Products in the unnamed mode Cannot evaluate net.sf.saxon.expr.sort.SortKeyDefinition

The original XSLT, which works fine with sourceforge saxon, is:

<xsl:stylesheet version="2.0" 
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	xmlns:ns="http://xml.XXXX.nl/schemas/XXXX/2017/01"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

	<xsl:output method="xml" encoding="UTF-8" indent="no" omit-xml-declaration="no"/>
	<xsl:strip-space elements="*"/>

	<xsl:template match="@*|node()">
		<xsl:copy>
			<xsl:apply-templates select="@*|node()"/>
		</xsl:copy>
	</xsl:template>

	<xsl:template match="/*">
		<xsl:copy>
			<xsl:apply-templates select="ns:person">
				<xsl:sort select="ns:personTypeA/ns:idA"/>
				<xsl:sort select="ns:personTypeB/ns:idB"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:participation">
				<xsl:sort select="ns:notificationId"/>
				<xsl:sort select="ns:participant/ns:participantTypeA/ns:field1"/>
				<xsl:sort select="ns:participant/ns:participantTypeA/ns:field2"/>
				<xsl:sort select="ns:participant/ns:participantTypeA/ns:field3"/>
				<xsl:sort select="ns:participant/ns:participantTypeB/ns:field4"/>
				<xsl:sort select="ns:participant/ns:participantTypeB/ns:field5"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:notification">
				<xsl:sort select="ns:notificationId"/>
			</xsl:apply-templates>
		</xsl:copy>
	</xsl:template>

</xsl:stylesheet>

via deduction the problem is within de select of participation. Because when this select is omitted then the transformation works (offcourse without the participation part)

Translation of the following XML with the following XSLT gives the following output:

<Products xmlns="http://xml.XXXX.nl/schemas/XXXX/2017/01" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0" xsi:schemaLocation="http://xml.XXXX.nl/schemas/XXXX/2017/01 file://pc.XXXX.nl/Homes/Usr/XXXX/Mijn%20Documenten/Desktop/xml/XXXX/Products.xsd">
	<person>
		<personTypeB>
			<idB>4455667</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455669</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455671</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeA>
			<idA>10001109</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001111</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001110</idA>
		</personTypeA>
	</person>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeA>
				<field1>10001110</field1>
				<field2>Hetfield</field2>
				<field3>1960-10-01</field3>
			</participantTypeA>
		</participant>
	</participation>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeB>
				<field4>8050901</field4>
				<field5>The Blues Company</field5>
			</participantTypeB>
		</participant>
	</participation>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeA>
				<field1>10001110</field1>
				<field2>Hetfield</field2>
				<field3>1950-10-01</field3>
			</participantTypeA>
		</participant>
	</participation>
	<notification>
		<notificationId>1111222214</notificationId>
	</notification>
	<notification>
		<notificationId>1111222213</notificationId>
	</notification>
</Products>

<xsl:stylesheet version="2.0" 
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	xmlns:ns="http://xml.XXXX.nl/schemas/XXXX/2017/01"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

	<xsl:output method="xml" encoding="UTF-8" indent="no" omit-xml-declaration="no"/>
	<xsl:strip-space elements="*"/>

	<xsl:template match="@*|node()">
		<xsl:copy>
			<xsl:apply-templates select="@*|node()"/>
		</xsl:copy>
	</xsl:template>

	<xsl:template match="/*">
		<xsl:copy>
			<xsl:apply-templates select="ns:person">
				<xsl:sort select="ns:personTypeA/ns:idA"/>
				<xsl:sort select="ns:personTypeB/ns:idB"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:notification">
				<xsl:sort select="ns:notificationId"/>
			</xsl:apply-templates>
		</xsl:copy>
	</xsl:template>

</xsl:stylesheet>

<?xml version="1.0" encoding="UTF-8"?>
<Products xmlns="http://xml.XXXX.nl/schemas/XXXX/2017/01" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<person>
		<personTypeB>
			<idB>4455667</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455669</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455671</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeA>
			<idA>10001109</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001110</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001111</idA>
		</personTypeA>
	</person>
	<notification>
		<notificationId>1111222213</notificationId>
	</notification>
	<notification>
		<notificationId>1111222214</notificationId>
	</notification>
</Products>

When only the participation with only sort on notificationId is added then the same error arises again. That's the following XSLT:

<xsl:stylesheet version="2.0" 
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	xmlns:ns="http://xml.XXXX.nl/schemas/XXXX/2017/01"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

	<xsl:output method="xml" encoding="UTF-8" indent="no" omit-xml-declaration="no"/>
	<xsl:strip-space elements="*"/>

	<xsl:template match="@*|node()">
		<xsl:copy>
			<xsl:apply-templates select="@*|node()"/>
		</xsl:copy>
	</xsl:template>

	<xsl:template match="/*">
		<xsl:copy>
			<xsl:apply-templates select="ns:person">
				<xsl:sort select="ns:personTypeA/ns:idA"/>
				<xsl:sort select="ns:personTypeB/ns:idB"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:participation">
				<xsl:sort select="ns:notificationId"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:notification">
				<xsl:sort select="ns:notificationId"/>
			</xsl:apply-templates>
		</xsl:copy>
	</xsl:template>

</xsl:stylesheet>

A collegue found out that it looks like trouble arises because of a sort notificationId within two different selects. When on one of them a data-type is specified the xslt is working again. Ofcours not the sort we want in the final version.

<xsl:stylesheet version="2.0" 
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	xmlns:ns="http://xml.XXXX.nl/schemas/XXXX/2017/01"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

	<xsl:output method="xml" encoding="UTF-8" indent="no" omit-xml-declaration="no"/>
	<xsl:strip-space elements="*"/>

	<xsl:template match="@*|node()">
		<xsl:copy>
			<xsl:apply-templates select="@*|node()"/>
		</xsl:copy>
	</xsl:template>

	<xsl:template match="/*">
		<xsl:copy>
			<xsl:apply-templates select="ns:person">
				<xsl:sort select="ns:personTypeA/ns:idA"/>
				<xsl:sort select="ns:personTypeB/ns:idB"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:participation">
				<xsl:sort select="ns:notificationId" data-type="text"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:notification">
				<xsl:sort select="ns:notificationId"/>
			</xsl:apply-templates>
		</xsl:copy>
	</xsl:template>

</xsl:stylesheet>

<?xml version="1.0" encoding="UTF-8"?>
<Products xmlns="http://xml.XXXX.nl/schemas/XXXX/2017/01" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<person>
		<personTypeB>
			<idB>4455667</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455669</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455671</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeA>
			<idA>10001109</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001110</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001111</idA>
		</personTypeA>
	</person>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeA>
				<field1>10001110</field1>
				<field2>Hetfield</field2>
				<field3>1960-10-01</field3>
			</participantTypeA>
		</participant>
	</participation>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeB>
				<field4>8050901</field4>
				<field5>The Blues Company</field5>
			</participantTypeB>
		</participant>
	</participation>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeA>
				<field1>10001110</field1>
				<field2>Hetfield</field2>
				<field3>1950-10-01</field3>
			</participantTypeA>
		</participant>
	</participation>
	<notification>
		<notificationId>1111222213</notificationId>
	</notification>
	<notification>
		<notificationId>1111222214</notificationId>
	</notification>
</Products>

Then added the other sorts into the participation select and the translation works. BUT.... with this small xml it does. When I used a much bigger xml to translate the error arises again.... Very strange..

<xsl:stylesheet version="2.0" 
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	xmlns:ns="http://xml.XXXX.nl/schemas/XXXX/2017/01"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

	<xsl:output method="xml" encoding="UTF-8" indent="no" omit-xml-declaration="no"/>
	<xsl:strip-space elements="*"/>

	<xsl:template match="@*|node()">
		<xsl:copy>
			<xsl:apply-templates select="@*|node()"/>
		</xsl:copy>
	</xsl:template>

	<xsl:template match="/*">
		<xsl:copy>
			<xsl:apply-templates select="ns:person">
				<xsl:sort select="ns:personTypeA/ns:idA"/>
				<xsl:sort select="ns:personTypeB/ns:idB"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:participation">
				<xsl:sort select="ns:notificationId"/>
				<xsl:sort select="ns:participant/ns:participantTypeA/ns:field1"/>
				<xsl:sort select="ns:participant/ns:participantTypeA/ns:field2"/>
				<xsl:sort select="ns:participant/ns:participantTypeA/ns:field3"/>
				<xsl:sort select="ns:participant/ns:participantTypeB/ns:field4"/>
				<xsl:sort select="ns:participant/ns:participantTypeB/ns:field5"/>
			</xsl:apply-templates>
			<xsl:apply-templates select="ns:notification">
				<xsl:sort select="ns:notificationId" data-type="text"/>
			</xsl:apply-templates>
		</xsl:copy>
	</xsl:template>

</xsl:stylesheet>

<?xml version="1.0" encoding="UTF-8"?>
<Products xmlns="http://xml.XXXX.nl/schemas/XXXX/2017/01" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<person>
		<personTypeB>
			<idB>4455667</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455669</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeB>
			<idB>4455671</idB>
		</personTypeB>
	</person>
	<person>
		<personTypeA>
			<idA>10001109</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001110</idA>
		</personTypeA>
	</person>
	<person>
		<personTypeA>
			<idA>10001111</idA>
		</personTypeA>
	</person>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeB>
				<field4>8050901</field4>
				<field5>The Blues Company</field5>
			</participantTypeB>
		</participant>
	</participation>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeA>
				<field1>10001110</field1>
				<field2>Hetfield</field2>
				<field3>1950-10-01</field3>
			</participantTypeA>
		</participant>
	</participation>
	<participation>
		<notificationId>123</notificationId>
		<participant>
			<participantTypeA>
				<field1>10001110</field1>
				<field2>Hetfield</field2>
				<field3>1960-10-01</field3>
			</participantTypeA>
		</participant>
	</participation>
	<notification>
		<notificationId>1111222213</notificationId>
	</notification>
	<notification>
		<notificationId>1111222214</notificationId>
	</notification>
</Products>
Actions
Copy link
 #1

Updated by Michael Kay almost 6 years ago

  • Project changed from SaxonC to Saxon
  • Category deleted (Saxon Internal)
Actions
Copy link
 #2

Updated by Michael Kay almost 6 years ago

Thanks for reporting it. This is an optimization bug (Saxon-EE 9.8 attempts much more aggressive optimization than Saxon-B 9.1 ever did). The optimizer now attempts to identify common subexpressions to avoid evaluating the same subexpression repeatedly, and it has wrongly attempted this when it sees two identical xsl:sort elements in the same template rule (specifically <xsl:sort select="ns:notificationId"/>).

As a workaround, you can suppress this optimization using -opt:-s on the command line, or an equivalent setConfigurationProperty(FeatureKeys.OPTIMIZATION_LEVEL, ....) from the Java API. Alternatively, do something that makes the xsl:sort specifications different without affecting the result. (You already discovered that setting data-type achieved this effect). Alternatively, move the second xsl:apply-templates into a separate named template.

Actions
Copy link
 #3

Updated by Rick Vlaming almost 6 years ago

Thank you Michael!

I've tried the -opt:-s on the commandline and it works. Also the alternative of moving the second xsl:apply-templates into a separate named template works (also without the -opt:-s).

Does this separate named template have impact on performance? We have some big xml's to process. I was thinking of putting all the xsl:apply-templates into separate named template, because of better readability...

Actions
Copy link
 #4

Updated by Michael Kay almost 6 years ago

Both these workarounds could potentially affect performance, though I suspect the effect will be insignificant.

Actions
Copy link
 #5

Updated by Michael Kay almost 6 years ago

  • Tracker changed from Support to Bug
  • Category set to Internals
  • Status changed from New to Resolved
  • Assignee changed from O'Neil Delpratt to Michael Kay
  • Applies to branch 9.8, trunk added
  • Fix Committed on Branch 9.8, trunk added

Resolution: in CommonSubexpressionPromoter.gatherElligibles(), treat an expression E as ineligible for promotion if E.isLiftable() returns false.

Actions
Copy link
 #6

Updated by Debbie Lockett almost 6 years ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in Maintenance Release 9.8.0.14 added

Bug fix applied in the Saxon 9.8.0.14 maintenance release.

Please register to edit this issue

Actions
Send by e-mail Copy link

Also available in: Atom PDF