Bug #4208
closedIn XSLT value templates, closing brace is not recognized when followed by backtick
100%
Description
In a construct such as
<xsl:template match="foo">`{.}`</xsl:template>
the backtick after the closing brace confuses the tokenizer into thinking that it's dealing with an XQuery "string constructor" (see XQuery ยง 3.10) and it reports that the expression in braces is not properly closed.
Updated by Michael Kay almost 5 years ago
Confirmed that this also affects XQuery - added test case StringConstructor-027 which does
<a>Today is `{adjust-date-to-timezone(current-date(), ())}`</a>
This means we can't solve it simply by disabling this tokenization logic when doing XPath.
Updated by Michael Kay almost 5 years ago
We also need to test perverse cases like
<a>This: {``[is `{``[text]``}`]``}`</a>
Updated by Michael Kay almost 5 years ago
In both string constructors, where we have embedded expression of the form
`{expr}`
and value templates (XSLT/XQuery) where we have embedded expressions of the form
{expr}
we are relying on the tokenizer to deliver a terminal token ( "}" or "}`......" . Now, the problem is that the "}" at the end of a value template can validly be followed by a backtick. So for the tokenizer to do this correctly, it needs to know which delimiter is expected; but that's not a simple state setting because either of these expressions can be nested in the other, recursively.
So it looks as if we are asking too much of the tokenizer.
I think we probably need a complete change of strategy for parsing StringConstructors, handling them more like we handle direct element constructors, where the parser switches into reading individual characters from the input, with tokenization disabled. Unfortunately this is a pretty complex solution.
Updated by Michael Kay almost 5 years ago
- Status changed from New to Resolved
- Applies to branch 9.9, trunk added
- Fix Committed on Branch 9.9, trunk added
Resolved this by a rather ugly hack where the parsing of an XQuery StringConstructor is divided awkwardly between the parser and tokenizer. But it's no uglier than some of the parsing for element constructors...
Test cases added to both QT3 and XSLT3.
Updated by Martin Honnen almost 5 years ago
The bug also exists in 9.8.
Not sure what the current policy in terms of that branch is but it would be nice if it also gets the fix as without it using text value templates in XSLT 3 to create Markdown output where the backtick is an important character is not possible and one has to go back to use xsl:text
/xsl:value-of
.
Updated by O'Neil Delpratt almost 5 years ago
- Status changed from Resolved to Closed
- % Done changed from 0 to 100
- Fixed in Maintenance Release 9.9.1.3 added
Bug fix applied to the Saxon 9.9.1.3 maintenance release
Updated by Michael Kay over 4 years ago
@Martin, I think it's unlikely there will be any further maintenance releases for 9.8. We would do one only if such a patch is the only way to keep a customer on the road, which would imply (a) there's something stopping the customer moving to 9.9, and (b) there's no practical workaround. In general, we are no longer committing 9.8 versions of patches as a matter of course. If we were to make another 9.8 build for general release, we would review the set of bugs that are candidates for retro-fitting.
Updated by Evan Lenz over 4 years ago
As a minor data point, I ran into this problem as well (also while creating Markdown-like output), and I'm glad to have found the explanation here so I can appropriately tailor the workaround. I'm also using Saxon-HE 9.8 due to problems with Saxon 9.9 that I haven't isolated yet (an uncaught java.lang.UnsupportedOperationException: "Cannot copy a variable reference whose binding is unknown") and based on the Saxon home page's indication that 9.8 is the most stable version. I'll try to create a report about the other issue.
Updated by Debbie Lockett over 4 years ago
@Evan, can I ask where you got the information that 9.8 is most stable? You should find that http://www.saxonica.com/download/download_page.xml says that 9.9 is most stable (this was updated some time ago, I think with the 9.9.1.2 release). Thanks.
Updated by Martin Honnen over 4 years ago
@Debbie, I don't know where Evan looked but for the HE edition people do consult http://saxon.sourceforge.net/ and that continues to say "The most stable version is version 9.8"
Updated by Evan Lenz over 4 years ago
@Debbie, the fifth paragraph on http://saxon.sourceforge.net/ reads:
"The most stable version is version 9.8, first released on 8 June 2018 (the same day as the XSLT 3.0 Recommendation), and regularly updated with maintenance releases."
Updated by Debbie Lockett over 4 years ago
Ah ha, thank you both - that information clearly needs updating...
Please register to edit this issue