Bug #6460: Possible lazy evaluation side-effect - Saxon - Saxonica Developer Community

Actions

Send by e-mail Copy link

Bug #6460

open

Possible lazy evaluation side-effect

Added by Toshihiko Makita 5 months ago. Updated 5 months ago.

Status:

New

Priority:

Low

Assignee:

Category:

Sprint/Milestone:

Start date:

2024-06-26

Due date:

% Done:

Estimated time:

Legacy ID:

Applies to branch:

Fix Committed on Branch:

Fixed in Maintenance Release:

Platforms:

Description

1. Summary¶

I'm developing a stylesheet that inputs DITA merged middle file (generated by DITA Open Toolkit 4.2.3) and outputs text file using Saxon-HE-12.4.jar. I encountered the phenomenon that seems to be caused by variable lazy eveluation.

2. Code¶

  <xsl:template match="*[@class => contains-token('map/topicref')][*[@class => contains-token('map/topicmeta')]/*[@class => contains-token('pmap-d/bookref')]]" mode="MODE_EXTRACT_IMAGE" priority="40">
        <xsl:variable name="topicRef" as="element()" select="."/>
        <xsl:variable name="topicMeta" as="element()" select="$topicRef/*[@class => contains-token('map/topicmeta')]"/>
        <xsl:variable name="annotatedXmlPath" as="xs:string" select="$topicRef/*[@class => contains-token('map/topicmeta')]/*[@class => contains-token('pmap-d/bookref')]/@href"/>
        <xsl:variable name="annotatedXmlPathDecoded" as="xs:string" select="uriutil:decodeURI($annotatedXmlPath)"/>
        <xsl:variable name="annotatedXmlPathNative" as="xs:string" select="ahf:pathToNativeStyle($annotatedXmlPathDecoded, $pIsWindows) => ahf:substringBeforeLast($pFileSep)"/>
        <xsl:message select="'$annotatedXmlPath=' || $annotatedXmlPath"/>
        <xsl:message select="'$annotatedXmlPathDecoded=' || $annotatedXmlPathDecoded"/>
        <!--xsl:message select="'$annotatedXmlPathNative=' || $annotatedXmlPathNative"/-->
        <xsl:next-match>
            <xsl:with-param name="prmAnnotatedXmlPathNative" tunnel="yes" select="$annotatedXmlPathNative"/>
        </xsl:next-match>
    </xsl:template>

"uriutil:decodeURI($annotatedXmlPath)" is the simple Java extension function.

// URIUtil.java
package com.antennahouse.xslt.extension;

import java.io.UnsupportedEncodingException;
import java.nio.charset.StandardCharsets;

public class URIUtil {
    /**
     * Decode URI String
     */
    public static String decodeURI(String url) throws UnsupportedEncodingException{
        String result=java.net.URLDecoder.decode(url, StandardCharsets.UTF_8.name());
        return result;
    }
}

// DecodeURIFunctionDefinition.java
package com.antennahouse.xslt.extension;

import net.sf.saxon.expr.XPathContext;
import net.sf.saxon.lib.ExtensionFunctionCall;
import net.sf.saxon.lib.ExtensionFunctionDefinition;
import net.sf.saxon.om.Sequence;
import net.sf.saxon.om.StructuredQName;
import net.sf.saxon.trans.XPathException;
import net.sf.saxon.value.SequenceType;
import net.sf.saxon.value.StringValue;

/**
 * Function definition for URIUtil.decodeURI()
 * @see https://www.saxonica.com/html/documentation10/extensibility/integratedfunctions/ext-full-J.html
 * @see https://www.dita-ot.org/dev/topics/implement-saxon-extension-functions.html
 */
public class DecodeURIFunctionDefinition extends ExtensionFunctionDefinition{
	public StructuredQName getFunctionQName() {
		return new StructuredQName("", "java:com.antennahouse.xslt.extension.URIUtil", "decodeURI");
	}

	public SequenceType[] getArgumentTypes() {
	    return new SequenceType[] {SequenceType.SINGLE_STRING};
	  }
	  

	public SequenceType getResultType(SequenceType[] suppliedArgumentTypes) {
		return SequenceType.SINGLE_STRING;
	}
  
	public ExtensionFunctionCall makeCallExpression() {
		return new ExtensionFunctionCall() {
			@Override
			public Sequence call(XPathContext context, Sequence[] arguments) throws XPathException {
				try {
					String arg = arguments[0].toString();
					String result = URIUtil.decodeURI(arg);
					return StringValue.makeStringValue(result);
				} catch (Exception ex) {
					throw new XPathException(ex);
				}
			}
		};
	}
}

3. Result¶

When I run above code¶

Got the following text file

D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\..\basedesign\基本設計書①\topic\images_基本設計書①\下線入りダミー画像.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\..\basedesign\基本設計書①\topic\images_基本設計書①\ダミー.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\..\basedesign\基本設計書②\..\基本設計書①\topic\images_基本設計書①\下線入りダミー画像.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\基本設計書①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\..\basedesign\基本設計書②\..\基本設計書①\topic\images_基本設計書①\ダミー.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\基本設計書①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\..\basedesign\基本設計書③\..\基本設計書①\topic\images_基本設計書①\下線入りダミー画像.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\基本設計書①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\..\basedesign\基本設計書③\..\基本設計書①\topic\images_基本設計書①\ダミー.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\基本設計書①\topic\images_基本設計書①

The xsl:message result

$annotatedXmlPath=../../basedesign/%E5%9F%BA%E6%9C%AC%E8%A8%AD%E8%A8%88%E6%9B%B8%E2%91%A0/%E5%9F%BA%E6%9C%AC%E8%A8%AD%E8%A8%88%E6%9B%B8%E2%91%A0-annotated.xml
$annotatedXmlPathDecoded=../../basedesign/基本設計書①/基本設計書①-annotated.xml
$annotatedXmlPath=../../basedesign/%E5%9F%BA%E6%9C%AC%E8%A8%AD%E8%A8%88%E6%9B%B8%E2%91%A1/%E5%9F%BA%E6%9C%AC%E8%A8%AD%E8%A8%88%E6%9B%B8%E2%91%A1-annotated.xml
$annotatedXmlPathDecoded=../../basedesign/基本設計書②/基本設計書②-annotated.xml
$annotatedXmlPath=../../basedesign/%E5%9F%BA%E6%9C%AC%E8%A8%AD%E8%A8%88%E6%9B%B8%E2%91%A2/%E5%9F%BA%E6%9C%AC%E8%A8%AD%E8%A8%88%E6%9B%B8%E2%91%A2-annotated.xml
$annotatedXmlPathDecoded=../../basedesign/基本設計書③/基本設計書③-annotated.xml

When I remove two xsl:message and run above code¶

I got the following text file.

D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\\topic\images_基本設計書①\下線入りダミー画像.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\\topic\images_基本設計書①\ダミー.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\\..\基本設計書①\topic\images_基本設計書①\下線入りダミー画像.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\基本設計書①\topic\images_基本設計書①
D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\\..\基本設計書①\topic\images_基本設計書①\ダミー.svg;D:\My_Documents\XML2023\ah\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\..\基本設計書①\topic\images_基本設計書①

Difficulty¶

The result text file is composed of lines that describe image file path to copy and destination folder. In the first result, the copying ant step ended normaly. But in the latter case the step ended with exception that describes "Source file does not exists".

D:\DITA-OT\dita-ot-4.2.3\plugins\org.dita.base\build.xml:150: The following error occurred while executing this line:
D:\DITA-OT\dita-ot-4.2.3\plugins\com.antennahouse.map.expand\build.xml:108: The following error occurred while executing this line:
D:\DITA-OT\dita-ot-4.2.3\plugins\com.antennahouse.map.expand\build.xml:211: Error occured during copy. Message='Source file does not exists! Path='D:\My_Documents\XML2023\nbi\diffs\diff-2024-06-24\10_評価データ\parameter\●●書パラメータ表①\\topic\images_基本設計書①\下線入りダミー画像.svg''
	at org.apache.tools.ant.ProjectHelper.addLocationToBuildException(ProjectHelper.java:582)
	at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:440)
	at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:106)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:299)
	at jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
	at org.apache.tools.ant.Task.perform(Task.java:350)
	at org.apache.tools.ant.Target.execute(Target.java:449)
	at org.apache.tools.ant.Target.performTasks(Target.java:470)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1401)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1374)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1264)
	at org.dita.dost.invoker.Main.runBuild(Main.java:853)
	at org.dita.dost.invoker.Main.startAnt(Main.java:240)
	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284)
	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)

How do I solve this problem??

Files

Download all files

2024-07-06.png (319 KB) 2024-07-06.png	Running snapshot via VSCode	Toshihiko Makita, 2024-07-05 17:29
saxon-test-2024-07-06.zip (7.97 MB) saxon-test-2024-07-06.zip	Full test data	Toshihiko Makita, 2024-07-05 17:32

Actions

Copy link

Updated by Toshihiko Makita 5 months ago

Operating system: Windows 11.

Actions

Copy link

Updated by Michael Kay 5 months ago

Thanks for reporting it.

We'll need to start by reproducing the problem, so could I ask you please to supply a complete stylesheet / source document that demonstrates the problem?

Also, you appear to have done a fair bit of investigation to conclude that the problem is in the area of lazy evaluation, and I would be interested to know what led you to this conclusion.

Actions

Copy link

Updated by Toshihiko Makita 5 months ago

We'll need to start by reproducing the problem, so could I ask you please to supply a complete stylesheet / source document that demonstrates the problem?

Give me a time because the DITA-OT processing for this step is too complex. I will prepare them.

I would be interested to know what led you to this conclusion.

This is my guess. The two xsl:message will force Saxon to evaluate "uriutil:decodeURI($annotatedXmlPath)" before calling xsl:next-match.

Actions

Copy link

Updated by Toshihiko Makita 5 months ago

could I ask you please to supply a complete stylesheet / source document that demonstrates the problem?

The actual running environment with DITA-OT is so complex and hard to send all of them. I've extracted single XSLT stylesheet and all of the input files to the independent folder.

In the DITA-OT, I used "full interface extension function" because DITA-OT has the ability to wrap it in a simple manner. However, when I made the running environment without DITA-OT, I have changed "full interface extension function" to "reflexive interface" and use Saxon-PE.

As a result, I could not get the same result with DITA-OT. The error did not occur.

Are there any differences between "full interface extension function" and "reflexive interface" in variable binding?

Regards,

Actions

Copy link

Updated by Toshihiko Makita 5 months ago

File 2024-07-06.png 2024-07-06.png added
File saxon-test-2024-07-06.zip saxon-test-2024-07-06.zip added

I could reproduce the problem with Saxon-HE and "full interface extension function".

The full test data: saxon-test-2024-07-06.zip

Steps to reproduce the problem¶

NOTE: Tested only in Windows 11 and PowerShell 7.4.3.

Unzip saxon-test-2024-07-06.zip to any location that you have the rights for write-access.
Make command-line that invoke saxon-test/saxon-test.ps1 by referencing saxon-test/sample-script.txt (Needs Power Shell)
Invoke command-line and save saxon-test/out/image-copy-desc.txt to other name.
Comment-out the xsl:message written in saxon-test/xsl/com.ah.pmap.expand/xsl/pmap_image_map.xsl line 70 to 72.
Invoke same command-line once again.
Compare the saxon-test/out/image-copy-desc.txt with step 3. There should exist differences.

Test results¶

Here are my test results.

saxon-test/out/image-copy-desc-ok.txt (Generated in above step 3)

D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\..\..\basedesign\pg-a\topics\images\20160406_153525.jpg;D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\topics\images
D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\..\..\basedesign\pg-a\topics\images\20160406_161345.jpg;D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\topics\images
D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\..\..\basedesign\pg-a\topics\images\20160406_154551.jpg;D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\topics\images

saxon-test/out/image-copy-desc-ng.txt (Generated in above step 5)

D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\\topics\images\20160406_153525.jpg;D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\topics\images
D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\\topics\images\20160406_161345.jpg;D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\topics\images
D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\\topics\images\20160406_154551.jpg;D:\My_Documents\Temp\saxon-test\instance\parameter\para-01\topics\images

Running snapshot via VSCode¶

Running snapshot via VSCode

Hope this helps your debugging!

Regards,

Please register to edit this issue

Actions

Send by e-mail Copy link

Also available in: Atom PDF

Project

Profile

Help

Saxon

Bug #6460

Possible lazy evaluation side-effect

1. Summary¶

2. Code¶

3. Result¶

When I run above code¶

When I remove two xsl:message and run above code¶

Difficulty¶

Updated by Toshihiko Makita 5 months ago

Updated by Michael Kay 5 months ago

Updated by Toshihiko Makita 5 months ago

Updated by Toshihiko Makita 5 months ago

Updated by Toshihiko Makita 5 months ago

Steps to reproduce the problem¶

Test results¶

Running snapshot via VSCode¶