Saxon » SaxonC

Using the official preview 11.99 of SaxonC 12 HE I looked at multithreading again, using the code from https://saxonica.plan.io/issues/5373#note-2.

It runs through fine on Windows (using Python 3.9) but at the end gives an error "Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available.". I don't know how to interpret that, the only Saxon related clues are perhaps the stack trace mentioning

  SP 0x000000d8979ef2b0 IP 0x00007ffbcb7ed290  [image code] com.oracle.svm.core.jdk.VMErrorSubstitutions.shutdown(VMErrorSubstitutions.java:116)
  SP 0x000000d8979ef2b0 IP 0x00007ffbcb7ed290  [image code] com.oracle.svm.core.jdk.VMErrorSubstitutions.shouldNotReachHere(VMErrorSubstitutions.java:109)
  SP 0x000000d8979ef2e0 IP 0x00007ffbcb843d54  [image code] com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:65)
  SP 0x000000d8979ef2f0 IP 0x00007ffbcb7db98e  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.onYellowZoneMadeAvailable(StackOverflowCheckImpl.java:197)
  SP 0x000000d8979ef310 IP 0x00007ffbcb7dbcc2  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.makeYellowZoneAvailable(StackOverflowCheckImpl.java:172)
  SP 0x000000d8979ef310 IP 0x00007ffbcb7dbcc2  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.throwNewStackOverflowError(StackOverflowCheckImpl.java:308)
  SP 0x000000d8979ef490 IP 0x00007ffbcc62e27f  [image code] net.sf.saxon.option.cpp.SaxonCException.clearException(SaxonCException.java:169)
  SP 0x000000d8979ef4a0 IP 0x00007ffbcb78a827  [image code] com.oracle.svm.core.code.IsolateEnterStub.SaxonCException_clearException_e05c4e29a4d8a5a4ab3a88346928491e581f8a6b(IsolateEnterStub.java:0)

Norm told me to let you know of any problems so this is my attempt to do so; he also told me not to expect proper answers before the new year; so take your time as needed to answer and enjoy your holidays.

The complete code is e.g.

import threading
import time

from saxonc import *

exitFlag = 0


class myThread(threading.Thread):
    def __init__(self, threadID, name, counter, proc, xquery_processor):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
        self.xquery_processor = xquery_processor
        self.proc = proc

    def run(self):
        print("Starting " + self.name)
        example(self.name, self.counter, 5, self.proc, self.xquery_processor)
        print("Exiting " + self.name)


def example(threadName, delay, counter, proc, xquery_processor):
    while counter > 0:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)

        result = xquery_processor.run_query_to_string(query_text='"Hello from SaxonC: " || current-dateTime()')

        if result is None:
            threadName.exit()

        print("%s : %s" % (threadName, result))
        counter -= 1
        proc.detach_current_thread


def example2(threadName, delay, counter, xquery_processor):
    while counter > 0:
        if exitFlag:
            return
        time.sleep(delay)
        result = xquery_processor.run_query_to_string(query_text='"Hello from SaxonC: " || current-dateTime()')
        if result is None:
            return
        print("%s : %s" % (threadName, result))
        counter -= 1


saxon_proc = PySaxonProcessor(license=False)

# Create new threads
thread1 = myThread(1, "Thread-1", 1, saxon_proc, saxon_proc.new_xquery_processor())
thread2 = myThread(2, "Thread-2", 2, saxon_proc, saxon_proc.new_xquery_processor())

# Start new Threads
thread1.start()
thread2.start()
thread1.join()
thread2.join()
print("Exiting Main Thread")

and gives e.g. the following output:

Starting Thread-1
Starting Thread-2
Thread-1 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:53.88708+01:00
Thread-2 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:54.8810576+01:00
Thread-1 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:54.8971375+01:00
Thread-1 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:55.9113253+01:00
Thread-2 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:56.8816877+01:00
Thread-1 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:56.912936+01:00
Thread-1 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:57.9172847+01:00
Exiting Thread-1
Thread-2 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:18:58.887187+01:00
Thread-2 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:19:00.9021471+01:00
Thread-2 : <?xml version="1.0" encoding="UTF-8"?>Hello from SaxonC: 2022-12-22T21:19:02.904465+01:00
Exiting Thread-2
Exiting Main Thread
Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available. Possible reasons for that: 1) A call from native code to Java code provided the wrong JNI environment or the wrong IsolateThread; 2) Frames of native code filled the stack, and now there is not even enough stack space left to throw a regular StackOverflowError; 3) An internal VM error occurred.

Current timestamp: 1671740342906

Printing Instructions (ip=0x00007ffbcb7ed290):
  0x00007ffbcb7ed270: 0x00 0x41 0xc7 0x87 0xfc 0x00 0x00 0x00 0xfe 0xfe 0xfe 0x7e 0x48 0x8b 0x54 0x24
  0x00007ffbcb7ed280: 0x20 0x4c 0x8b 0x44 0x24 0x10 0x4c 0x8b 0x4c 0x24 0x18 0xe8 0x80 0xfc 0xff 0xff
  0x00007ffbcb7ed290: 0x90 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc
  0x00007ffbcb7ed2a0: 0x48 0x83 0xec 0x28 0x48 0x89 0x54 0x24 0x20 0x49 0x3b 0x67 0x08 0x0f 0x86 0x66

Top of stack (sp=0x000000d8979ef2b0):
  0x000000d8979ef2b0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef2c0: 0x000002b30c4c95d0 0x000002b30c000000
  0x000000d8979ef2d0: 0x00007ffbcb7db98e 0x00007ffbcb843d54
  0x000000d8979ef2e0: 0x000002b30a65f53f 0x00007ffbcb7db98e
  0x000000d8979ef2f0: 0x0000000000000400 0x000002b30a65c4d0
  0x000000d8979ef300: 0x000002b30a65ea50 0x00007ffbcb7dbcc2
  0x000000d8979ef310: 0x000002b309c2d7b0 0x0000000000000080
  0x000000d8979ef320: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef330: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef340: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef350: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef360: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef370: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef380: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef390: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef3a0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef3b0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef3c0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef3d0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef3e0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef3f0: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef400: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef410: 0x0000000000000000 0x0000000000000000
  0x000000d8979ef420: 0x0000000000000000 0x000002b30a80dfe8
  0x000000d8979ef430: 0x00007ffc0bc20000 0x000000d8979ef4e0
  0x000000d8979ef440: 0x0000000000000000 0x0000000000000011
  0x000000d8979ef450: 0x000002b309f35fc0 0x000002b30a662240
  0x000000d8979ef460: 0x000002b30a821710 0x000002b30a65f4a0
  0x000000d8979ef470: 0x000002b30a656640 0x000002b30a656640
  0x000000d8979ef480: 0x0000000000000003 0x00007ffbcc62e27f
  0x000000d8979ef490: 0x000002b30a80de30 0x00007ffbcb78a827
  0x000000d8979ef4a0: 0x000002b30a664d90 0x00007ffc0bc60d34

Top frame info:
  TotalFrameSize in CodeInfoTable 48
  
Threads:
  0x000002b30a656640 STATUS_IN_JAVA (PREVENT_VM_FROM_REACHING_SAFEPOINT) "System-1" - 0x000002b30d901028, daemon, stack(0x000000d8983a0000,0x000000d898590000)
  0x000002b30a659a80 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-0" - 0x000002b30d801028, daemon, stack(0x000000d8981b0000,0x000000d8983a0000)
  0x000002b309bddb80 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "Reference Handler" - 0x000002b30cb900a0, daemon, stack(0x000000d897fc0000,0x000000d8981b0000)
  0x000002b309bcc800 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "main" - 0x000002b30cb8fff0, stack(0x000000d897800000,0x000000d8979f0000)
  
VM thread locals for the failing thread 0x000002b30a656640:
  0 (8 bytes): JNIThreadLocalEnvironment.jniFunctions = (bytes) 
    0x000002b30a656640: 0x000002b30c720010
  8 (8 bytes): StackOverflowCheckImpl.stackBoundaryTL = (Word) 1 (0x0000000000000001)
  16 (4 bytes): Safepoint.safepointRequested = (int) 2147454477 (0x7fff8e0d)
  20 (4 bytes): StatusSupport.statusTL = (int) 1 (0x00000001)
  24 (32 bytes): ThreadLocalAllocation.regularTLAB = (bytes) 
    0x000002b30a656658: 0x000002b30d900000 0x000002b30da00000
    0x000002b30a656668: 0x000002b30d9abf20 0x0000000000000000
  56 (8 bytes): PlatformThreads.currentThread = (Object) java.lang.Thread (0x000002b30d901028)
  64 (8 bytes): JavaFrameAnchors.lastAnchor = (Word) 0 (0x0000000000000000)
  72 (8 bytes): AccessControlContextStack = (Object) null
  80 (8 bytes): ExceptionUnwind.currentException = (Object) null
  88 (8 bytes): IdentityHashCodeSupport.hashCodeGeneratorTL = (Object) java.util.SplittableRandom (0x000002b30d90ba20)
  96 (8 bytes): IsolatedCompileClient.currentClient = (Object) null
  104 (8 bytes): IsolatedCompileContext.currentContext = (Object) null
  112 (8 bytes): JNIObjectHandles.handles = (Object) com.oracle.svm.core.handles.ThreadLocalHandles (0x000002b30d9229b8)
  120 (8 bytes): JNIThreadLocalPendingException.pendingException = (Object) null
  128 (8 bytes): JNIThreadLocalPinnedObjects.pinnedObjectsListHead = (Object) null
  136 (8 bytes): JNIThreadOwnedMonitors.ownedMonitors = (Object) null
  144 (8 bytes): NoAllocationVerifier.openVerifiers = (Object) null
  152 (8 bytes): ThreadingSupportImpl.activeTimer = (Object) null
  160 (8 bytes): SubstrateDiagnostics.threadOnlyAttachedForCrashHandler = (bytes) 
    0x000002b30a6566e0: 0x0000000000000000
  168 (8 bytes): ThreadLocalAllocation.allocatedBytes = (Word) 0 (0x0000000000000000)
  176 (8 bytes): VMThreads.IsolateTL = (Word) 2968023728128 (0x000002b30c000000)
  184 (8 bytes): VMThreads.OSThreadHandleTL = (Word) 448 (0x00000000000001c0)
  192 (8 bytes): VMThreads.OSThreadIdTL = (Word) 34804 (0x00000000000087f4)
  200 (8 bytes): VMThreads.StackBase = (Word) 930268905472 (0x000000d898590000)
  208 (8 bytes): VMThreads.StackEnd = (Word) 930266873856 (0x000000d8983a0000)
  216 (8 bytes): VMThreads.StartedByCurrentIsolate = (bytes) 
    0x000002b30a656718: 0x0000000000000000
  224 (8 bytes): VMThreads.nextTL = (Word) 2967996832384 (0x000002b30a659a80)
  232 (8 bytes): VMThreads.unalignedIsolateThreadMemoryTL = (Word) 2967996819008 (0x000002b30a656640)
  240 (4 bytes): ActionOnExitSafepointSupport.actionTL = (int) 0 (0x00000000)
  244 (4 bytes): ActionOnTransitionToJavaSupport.actionTL = (int) 0 (0x00000000)
  248 (4 bytes): ImplicitExceptions.implicitExceptionsAreFatal = (int) 0 (0x00000000)
  252 (4 bytes): StackOverflowCheckImpl.yellowZoneStateTL = (int) 2130640638 (0x7efefefe)
  256 (4 bytes): StatusSupport.safepointBehaviorTL = (int) 1 (0x00000001)
  260 (4 bytes): ThreadingSupportImpl.currentPauseDepth = (int) 1 (0x00000001)
  
No VMOperation in progress

The 15 most recent VM operation status changes (oldest first):
  
Counters:
  
Java frame anchors for the failing thread 0x000002b30a656640:
  No anchors
  
Stacktrace for the failing thread 0x000002b30a656640:
  SP 0x000000d8979ef2b0 IP 0x00007ffbcb7ed290  [image code] com.oracle.svm.core.jdk.VMErrorSubstitutions.shutdown(VMErrorSubstitutions.java:116)
  SP 0x000000d8979ef2b0 IP 0x00007ffbcb7ed290  [image code] com.oracle.svm.core.jdk.VMErrorSubstitutions.shouldNotReachHere(VMErrorSubstitutions.java:109)
  SP 0x000000d8979ef2e0 IP 0x00007ffbcb843d54  [image code] com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:65)
  SP 0x000000d8979ef2f0 IP 0x00007ffbcb7db98e  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.onYellowZoneMadeAvailable(StackOverflowCheckImpl.java:197)
  SP 0x000000d8979ef310 IP 0x00007ffbcb7dbcc2  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.makeYellowZoneAvailable(StackOverflowCheckImpl.java:172)
  SP 0x000000d8979ef310 IP 0x00007ffbcb7dbcc2  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.throwNewStackOverflowError(StackOverflowCheckImpl.java:308)
  SP 0x000000d8979ef490 IP 0x00007ffbcc62e27f  [image code] net.sf.saxon.option.cpp.SaxonCException.clearException(SaxonCException.java:169)
  SP 0x000000d8979ef4a0 IP 0x00007ffbcb78a827  [image code] com.oracle.svm.core.code.IsolateEnterStub.SaxonCException_clearException_e05c4e29a4d8a5a4ab3a88346928491e581f8a6b(IsolateEnterStub.java:0)
  
VM mutexes:
  mutex "mainVMOperationControlWorkQueue" is unlocked.
  mutex "referencePendingList" is unlocked.
  mutex "thread" is unlocked.
  
AOT compiled code is mapped at 0x00007ffbcb731000 - 0x00007ffbccffa46f

Heap settings and statistics:
  Supports isolates: true
  Heap base: 0x000002b30c000000
  Object reference size: 8
  Aligned chunk size: 1048576
  Incremental collections: 0
  Complete collections: 0
  
Native image heap boundaries:
  ReadOnly Primitives: 0x000002b30c101028 - 0x000002b30c43ec20
  ReadOnly References: 0x000002b30c43ec20 - 0x000002b30c71f588
  ReadOnly Relocatables: 0x000002b30c720000 - 0x000002b30c96c980
  Writable Primitives: 0x000002b30c96d000 - 0x000002b30ca9b6f0
  Writable References: 0x000002b30ca9b6f0 - 0x000002b30cf563f0
  Writable Huge: 0x000002b30d000030 - 0x000002b30d0712e8
  ReadOnly Huge: 0x000002b30d072030 - 0x000002b30d663918

Heap:
  Young generation: 
    Eden: 
      edenSpace:
        aligned: 0/0 unaligned: 0/0
    Survivors: 
      Survivor-1 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-1 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-2 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-2 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-3 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-3 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-4 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-4 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-5 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-5 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-6 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-6 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-7 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-7 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-8 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-8 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-9 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-9 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-10 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-10 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-11 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-11 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-12 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-12 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-13 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-13 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-14 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-14 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-15 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-15 To:
        aligned: 0/0 unaligned: 0/0
  Old generation: 
    oldFromSpace:
      aligned: 0/0 unaligned: 0/0
    oldToSpace:
      aligned: 0/0 unaligned: 0/0
    
  Unused:
    aligned: 0/0

Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available. Possible reasons for that: 1) A call from native code to Java code provided the wrong JNI environment or the wrong IsolateThread; 2) Frames of native code filled the stack, and now there is not even enough stack space left to throw a regular StackOverflowError; 3) An internal VM error occurred.

Process finished with exit code 99