Lazy evaluation of sequence constructors - instructions with side effects evaluated early
I have a problem with Saxon-JS. It seems to be related with either some optimization or, more likely, with concurrency.
I have a template that calls a named template "deactivate" (it modifies the current page sothat it looks disabled).
After that, it schedules an action for loading an external document (ixsl:schedule-action). The scheduled action is a named template that renders the document (<xsl:apply-templates select="doc(someurl)">). After rendering the document, it calls a named template "reactivate", which undoes the work of the earlier "deactivate".
I added a lot of <xsl:message>s, and in the console of Chrome, I see that template "reactivate" is sometimes called earlier than "deactivate". This seems to be especially the case when the doc() function has been passed a URL that it has seen before (so XSLT need not physically reload the document).
The "deactivate" template has the following structure:
- xsl:message "started"
-- message "inside for-each"
- xsl:message "completed"
Looking at the output in the Chrome console, the messages of the for-each show up at a strange moment, for example when the "completed" message of the entire template has already been printed.
I do recognize the fact that I may be doing things wrong, but I hope that you can shed some light on this. And if it does appear to be a bug, I think you want to know.
Now the application is relatively complex (and written in a mixture of Dutch, English and even a small amount of PHP), so if you are willing to investigate, perhaps I can give some instructions on how to reproduce the problem or even share my screen - I will be most happy to offer any help that is needed. Also, if I need to attempt to reduce the size of the application, let me know.
The behaviour is also seen in Firefox, but in my Firefox, xsl:messages are not shown in the console.
I attach the files plus a screen selection of the Chrome console. The numbers indicate, more or less, the expected order of messages.
#1 Updated by Pieter Masereeuw 8 months ago
I created a simplified version that shows more or less the same problem. Unfortunately, as I wanted to load a page that needs some time to be calculated, I still needed the PHP work-around for cross-server scripting. However, you can try it at [[http://masereeuw.xs4all.nl/doctest/index.html]] . I also attach the files (doctest.zip).
I order to test, press the button "Load doc1", wait for the result and then press it again. As you will hopefully see, the gray background is not turned back into white the second time. Unlike the real application, the order of xsl:message output is not counter-intuitive here, so the problem may just be related to the moment that Saxon-JS lets the browser apply the modifications.
The real application, with messages, can be seen here: [[http://masereeuw.xs4all.nl/gtbtest/index.html]]. Enter the string
koe* (including the wildcard) in the topmost box, then press the
Zoek button at the bottom. Following that, press the button
Laatste and finally
Eerste. After that, you'll hopefully see that the first output is restored, but the "Please wait" effect is not cleared.
#2 Updated by Debbie Lockett 8 months ago
- Assignee set to Debbie Lockett
Hi Pieter, thanks for reporting the issue and preparing the samples. I have not looked into it properly yet, but I can confirm that I can replicate the problem with the sample at http://masereeuw.xs4all.nl/doctest/index.html
We will investigate further as soon as possible.
#3 Updated by Pieter Masereeuw 8 months ago
Thank you - I have a work-around in place that seems to solve the problem (in an ugly way): I store all visited URLs in a property in xsl:page() and depending on the presence of a URL, I call the deactivate/reactivate templates.
The call to doc() is still being used, so it would seem that the problem is indeed related to known documents in combination with modifications elsewhere on the page. Good luck - let me know if I can help.
#4 Updated by Michael Kay 8 months ago
It's not actually a concurrency issue, it's a lazy evaluation issue. There are two instructions here that have side effects: xsl:message and ixsl:set-property, and because these have side effects, the order of evaluation becomes visible.
A sequence constructor is being evaluated using the logic:
(a) for each of the instructions in the sequence, get an iterator;
(b) read off the items delivered by these iterators in turn.
Now, both xsl:message and ixsl:set-property are doing their stuff (producing their side-effect) at the time you get the iterator (stage (a)) rather than at the time you read the iterator (stage (b)), which means that if a sequence constructor contains an xsl:for-each followed by an xsl:message, the xsl:message output will be displayed before the xsl;for-each is evaluated.
This is all legal according to the XSLT 3.0 specification, but very unhelpful. I think the policy we have generally adopted (and although it's not required by the spec, there are things in the XSLT 3.0 streamability analysis that assume it) is that the instructions in a sequence constructor should be evaluated in the order they are written. I think we should change the implementation of sequence constructors to follow that policy.
#5 Updated by Pieter Masereeuw 8 months ago
Thanks for your answer. Well yes, that explains a lot. However, I still do not fully understand how this influences the order of events when a new document is loaded vs. one that has already been loaded. After all, that is what seems to be causing the problem. If it is not concurrency, what is it?
Now that I have a better understanding, I came up with a work-around that seems to be more generally applicable than my earlier one: the call that resets the view to normal (template named "reactivate-tab") is now wrapped inside an <xsl:schedule-action> with a short wait-amount. This seems to work well.
Do you agree that this might be a feasible way to go?
The whole thing shows once more that side effects are, in the American president's words, "very bad". But now that they are here, I have the feeling that a solution like the one you are describing will certainly remove the counter-intuitive effects that I encountered. After all, my use case (informing the user about a lengthy operation) seems to be very normal.
#6 Updated by Debbie Lockett 8 months ago
- Status changed from New to In Progress
- Priority changed from Low to Normal
We have added a new Block iterator that evaluates a sequence of instructions in order. So now a sequence constructor is being evaluated using the logic:
(a) process the instructions in the sequence in turn;
(b) process an instruction by getting an iterator, and reading off the items delivered.
So now if we have a sequence constructor containing instructions A followed by B, where B produces side effects, the side effects may happen "early" within the evaluation of B (i.e. as the iterator is generated, rather than as its items are read); but certainly not before evaluating A.
The specific part of the doctest stylesheet where this is relevant is the showdoc template, whose sequence constructor contains
This sequence constructor was previously being evaluated as follows:
(1) Construct iterator for the call-template instruction. As this is constructed, we see its direct side-effects immediately (i.e. the xsl:messages); but NOT the result of evaluating the xsl:for-each (which contains the ixsl:set-style).
(2) Construct iterator for the ixsl:schedule-action. The ixsl:schedule-action is also an instruction with side effects - the side effect being the results of evaluating its sequence constructor. The instruction actually effectively says, "once you have this document, evaluate the sequence constructor". On the first click, there is some time delay as the document is retrieved. On the second click there is no delay. The result of evaluating the sequence constructor is:
(x) Get the results of evaluating the render-results template (page content, and reactivate ixsl:set-style action).
(3) Read off the results of the iterators. The result of the call-template iterator is the deactivate ixsl:set-style action (while the ixsl:schedule-action returned an empty iterator).
So, with the delay of retrieving a document, we see (x) happen after (3) as intended. But with no delay, (x) (unintuitively!) occurs before (3).
#7 Updated by Pieter Masereeuw 8 months ago
Wow - my compliments and thanks for offering a look into the internals of Saxon. This feels like an important discovery for the usefulness of Saxon-JS!
Will the fix involve the compiler or just the runtime? In the first case, as I am using the Oxygen-supplied version, I will have to invest in a Saxon Enterprise license. But considering the efforts you are taking, I think that will be money well spent!
#9 Updated by Debbie Lockett 8 months ago
No problem, indeed this has been very useful to work through. I had previously encountered similar "counter-intuitive effects" (good phrase) when working on our documentation app stylesheets. I knew the issue was something to do with side effects, but had not managed to pin down the problem. Like you I was using xsl:message to try to debug the behaviour, but getting unexpected results. Now it makes sense - previously the xsl:messages were unreliable, since they could in fact appear before a preceding instruction had been evaluated. So much better that we can actually trust the order of the xsl:messages now!!
Please register to edit this issue