This is the output I'm getting with 10.5 (it would be useful to see the output you are getting)
<test xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<select-on-original><!-- content as expected -->
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="2"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="1"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="3"/>
</select-on-original>
<select-on-variable><!-- empty but that is unexpected (namespace predicate problem)-->
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="1"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="2"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="3"/>
</select-on-variable>
<select-on-sorted-variable><!-- empty but that is unexpected (namespace predicate problem) -->
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="2"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="1"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="3"/>
</select-on-sorted-variable>
<select-on-sorted-variable><!-- output is sorted (without namespace predicate)-->
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="2"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="1"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="3"/>
</select-on-sorted-variable>
<select-on-sorted-variableWithSequence><!--output is not sorted unfortunately (namespace predicate works fine) -->
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="2"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="1"/>
<comp xmlns="urn:hl7-org:v3" xsi:type="IVL_TS" value="3"/>
</select-on-sorted-variableWithSequence>
</test>
It looks like the namespace issues have gone with 10.x, so let's focus on the sorting issue. At first sight, it certainly seems surprising that only the second run is sorted on value, when no sorting was requested for that case!
I think the explanation for this lies in the expressions $var//*:comp
and $varsorted//*:comp
. Because of the use of the '//' operator, these sort nodes into document order. But the nodes are in different trees ($var and $varsorted are sequences of parentless element nodes), and the result of sorting nodes in different trees into document order is undefined.
You can confirm this theory by replacing the operator "//" throughout by "!", in which case the output order is now much more intuitive.
I have a feeling that at some time in the past, Saxon allocated document numbers to parentless elements in order of creation, which had the accidental side-effect of making document order rather more predictable than it is now (and therefore, masking bugs in user code!).
I'll run this again in the debugger to take a more detailed look at what's going on but I'm pretty sure this is the explanation.