Project

Profile

Help

Bug #3958

closed

Scheduling algorithm for xsl:for-each/@saxon:threads=N

Added by Michael Kay almost 4 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
Multithreading
Sprint/Milestone:
-
Start date:
2018-10-09
Due date:
% Done:

100%

Estimated time:
Legacy ID:
Applies to branch:
9.9, trunk
Fix Committed on Branch:
9.9, trunk
Fixed in Maintenance Release:
Platforms:

Description

The scheduling algorithm used by xsl:for-each/@saxon:threads="N" is not especially smart. In particular, it gives poor throughput when the amount of work required to process different items in the input sequence is highly variable. The following test case illustrates the problem:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet exclude-result-prefixes="#all" version="3.0" xmlns:saxon="http://saxon.sf.net/"
	xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

	<xsl:output method="text" />

	<xsl:template name="main">
		<xsl:for-each saxon:threads="4" select="1 to 10">
			<xsl:choose>
				<xsl:when test=". eq 1">
					<!-- Will take 10 seconds -->
					<xsl:sequence select="json-doc('https://httpbin.org/delay/10')?url" />
				</xsl:when>
				<xsl:when test=". eq 2">
					<!-- Will take 9 seconds -->
					<xsl:sequence select="json-doc('https://httpbin.org/delay/9')?url" />
				</xsl:when>
				<xsl:when test=". eq 3">
					<!-- Will take 8 seconds -->
					<xsl:sequence select="json-doc('https://httpbin.org/delay/8')?url" />
				</xsl:when>
			</xsl:choose>
		</xsl:for-each>
		<xsl:text>&#x0A;</xsl:text>
	</xsl:template>
</xsl:stylesheet>

Ideally the elapsed time for this workload should be hardly longer than the longest processing time for one item, that is, 10 seconds. In practice it is about 30 seconds, because the threads are not well utilised. The problem can be solved by allocating more threads, but we should be able to do better.

Please register to edit this issue

Also available in: Atom PDF