Bug #4208

In XSLT value templates, closing brace is not recognized when followed by backtick

Added by Michael Kay over 2 years ago. Updated over 2 years ago.

XSLT conformance
Start date:
Due date:
% Done:


Estimated time:
Legacy ID:
Applies to branch:
9.9, trunk
Fix Committed on Branch:
9.9, trunk
Fixed in Maintenance Release:


In a construct such as

<xsl:template match="foo">`{.}`</xsl:template>

the backtick after the closing brace confuses the tokenizer into thinking that it's dealing with an XQuery "string constructor" (see XQuery ยง 3.10) and it reports that the expression in braces is not properly closed.


#1 Updated by Michael Kay over 2 years ago

Confirmed that this also affects XQuery - added test case StringConstructor-027 which does

<a>Today is `{adjust-date-to-timezone(current-date(), ())}`</a>

This means we can't solve it simply by disabling this tokenization logic when doing XPath.

#2 Updated by Michael Kay over 2 years ago

We also need to test perverse cases like

<a>This: {``[is `{``[text]``}`]``}`</a>

#3 Updated by Michael Kay over 2 years ago

In both string constructors, where we have embedded expression of the form


and value templates (XSLT/XQuery) where we have embedded expressions of the form


we are relying on the tokenizer to deliver a terminal token ( "}" or "}`......" . Now, the problem is that the "}" at the end of a value template can validly be followed by a backtick. So for the tokenizer to do this correctly, it needs to know which delimiter is expected; but that's not a simple state setting because either of these expressions can be nested in the other, recursively.

So it looks as if we are asking too much of the tokenizer.

I think we probably need a complete change of strategy for parsing StringConstructors, handling them more like we handle direct element constructors, where the parser switches into reading individual characters from the input, with tokenization disabled. Unfortunately this is a pretty complex solution.

#4 Updated by Michael Kay over 2 years ago

  • Status changed from New to Resolved
  • Applies to branch 9.9, trunk added
  • Fix Committed on Branch 9.9, trunk added

Resolved this by a rather ugly hack where the parsing of an XQuery StringConstructor is divided awkwardly between the parser and tokenizer. But it's no uglier than some of the parsing for element constructors...

Test cases added to both QT3 and XSLT3.

#5 Updated by Martin Honnen over 2 years ago

The bug also exists in 9.8.

Not sure what the current policy in terms of that branch is but it would be nice if it also gets the fix as without it using text value templates in XSLT 3 to create Markdown output where the backtick is an important character is not possible and one has to go back to use xsl:text/xsl:value-of.

#6 Updated by O'Neil Delpratt over 2 years ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in Maintenance Release added

Bug fix applied to the Saxon maintenance release

#7 Updated by Michael Kay over 2 years ago

@Martin, I think it's unlikely there will be any further maintenance releases for 9.8. We would do one only if such a patch is the only way to keep a customer on the road, which would imply (a) there's something stopping the customer moving to 9.9, and (b) there's no practical workaround. In general, we are no longer committing 9.8 versions of patches as a matter of course. If we were to make another 9.8 build for general release, we would review the set of bugs that are candidates for retro-fitting.

#8 Updated by Evan Lenz over 2 years ago

As a minor data point, I ran into this problem as well (also while creating Markdown-like output), and I'm glad to have found the explanation here so I can appropriately tailor the workaround. I'm also using Saxon-HE 9.8 due to problems with Saxon 9.9 that I haven't isolated yet (an uncaught java.lang.UnsupportedOperationException: "Cannot copy a variable reference whose binding is unknown") and based on the Saxon home page's indication that 9.8 is the most stable version. I'll try to create a report about the other issue.

#9 Updated by Debbie Lockett over 2 years ago

@Evan, can I ask where you got the information that 9.8 is most stable? You should find that says that 9.9 is most stable (this was updated some time ago, I think with the release). Thanks.

#10 Updated by Martin Honnen over 2 years ago

@Debbie, I don't know where Evan looked but for the HE edition people do consult and that continues to say "The most stable version is version 9.8"

#11 Updated by Evan Lenz over 2 years ago

@Debbie, the fifth paragraph on reads:

"The most stable version is version 9.8, first released on 8 June 2018 (the same day as the XSLT 3.0 Recommendation), and regularly updated with maintenance releases."

#12 Updated by Debbie Lockett over 2 years ago

Ah ha, thank you both - that information clearly needs updating...

Please register to edit this issue

Also available in: Atom PDF