Project

Profile

Help

Actions
Send by e-mail Copy link

Bug #6304

closed

Multithreaded XSLT transformations runs through with SaxonC 12.4.1 HE but then on Windows gives lengthy error "Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available. "

Added by Martin Honnen 12 months ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
O'Neil Delpratt
Category:
Python API
Start date:
2023-12-22
Due date:
% Done:

100%

Estimated time:
Applies to branch:
Fix Committed on Branch:
Fixed in Maintenance Release:
Found in version:
12.4.1
Fixed in version:
12.4.2
SaxonC Languages:
SaxonC Platforms:
SaxonC Architecture:

Description

I have tried to write a Python 3 program using SaxonC HE 12.4.1, making use of a ThreadPoolExecutor.

The programs tries to run the same transformation on all files of an input directory and writes the result to some output directory.

On both Windows and Linux the XSLT transformations seems to succeed but while the Python 3 program under Linux seems to terminate fine under Windows it then gives a length error "Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available.", see below.

I need some advice whether the Python code needs to be fixed (throw in some detach_current_thread perhaps somewhere) or whether this is an issue with SaxonC and/or GraalVM under Windows.

I have put the source code (Python, XSLT, input XMLs) in a public Github repro: https://github.com/martin-honnen/SaxonC12ThreadPoolExecutorXSLTTransformation.

I will also show the Python code inline:

import concurrent
import os
from concurrent.futures import ThreadPoolExecutor

from saxonche import PySaxonProcessor
from saxonche import PySaxonApiError

def transform(saxon_processor, xslt30_executable, file_name):
    try:
        xdm_node = saxon_processor.parse_xml(xml_file_name=f'input/{file_name}')
        xslt30_executable.set_global_context_item(xdm_item=xdm_node)
        xslt30_executable.apply_templates_returning_file(xdm_value=xdm_node, output_file=f'output/{file_name}')
        return True
    except PySaxonApiError as e:
        return e.message
    finally:
        saxon_processor.detach_current_thread
def thread_pool_test():
    with PySaxonProcessor() as saxon_processor:
        xslt30_compiler = saxon_processor.new_xslt30_processor()

        try:
            xslt30_executable = xslt30_compiler.compile_stylesheet(stylesheet_file='identity.xsl')
            with ThreadPoolExecutor(max_workers=4) as executor:
                futures = {executor.submit(transform, saxon_processor, xslt30_executable, file): file for file in os.listdir('input')}
                for future in concurrent.futures.as_completed(futures):
                    file = futures[future]
                    result = future.result()
                    if (result == True):
                        print(f'File {file} transformed successfully!')
                    else:
                        print(f'File {file} transformation failed with {result}')

        except PySaxonApiError as e:
            print(f'XSLT compilation failed: {e}')



# Press the green button in the gutter to run the script.
if __name__ == '__main__':
    thread_pool_test()

What I have tried in Python code is

def transform(saxon_processor, xslt30_executable, file_name):
    try:
        xdm_node = saxon_processor.parse_xml(xml_file_name=f'input/{file_name}')
        xslt30_executable.set_global_context_item(xdm_item=xdm_node)
        xslt30_executable.apply_templates_returning_file(xdm_value=xdm_node, output_file=f'output/{file_name}')
        return True
    except PySaxonApiError as e:
        return e.message
    finally:
        saxon_processor.detach_current_thread

but this doesn't improve or change things, under Windows I still get the lengthy error.

Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available. Possible reasons for that: 1) A call from native code to Java code provid
ed the wrong JNI environment or the wrong IsolateThread; 2) Frames of native code filled the stack, and now there is not even enough stack space left to throw a regular StackOverflowError; 3) An internal VM error occurred.

Current timestamp: 1703249807822

Printing Instructions (ip=0x00007ff9a2fc23a0):
  0x00007ff9a2fc2380: 0x00 0x41 0xc7 0x87 0xfc 0x00 0x00 0x00 0xfe 0xfe 0xfe 0x7e 0x48 0x8b 0x54 0x24
  0x00007ff9a2fc2390: 0x20 0x4c 0x8b 0x44 0x24 0x10 0x4c 0x8b 0x4c 0x24 0x18 0xe8 0x80 0xfc 0xff 0xff
  0x00007ff9a2fc23a0: 0x90 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc 0xcc
  0x00007ff9a2fc23b0: 0x48 0x83 0xec 0x28 0x48 0x89 0x54 0x24 0x20 0x49 0x3b 0x67 0x08 0x0f 0x86 0x66

Top of stack (sp=0x000000ae421ef0c0):
  0x000000ae421ef0c0: 0x0000000000000000 0x00007ffb6852be70
  0x000000ae421ef0d0: 0x00000203b1509538 0x00000203b1000000
  0x000000ae421ef0e0: 0x00007ff9a2fb07ee 0x00007ff9a3018e64
  0x000000ae421ef0f0: 0x0000000000000000 0x00007ff9a2fb07ee
  0x000000ae421ef100: 0x00000203b05ac9cf 0x00000203b080e722
  0x000000ae421ef110: 0x00030000ffffffff 0x00007ff9a2fb0b22
  0x000000ae421ef120: 0x00000203b05aba80 0x00000203b0dc4b40
  0x000000ae421ef130: 0x00000203b0c63fa0 0x0000000000000000
  0x000000ae421ef140: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef150: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef160: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef170: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef180: 0x0000000000000000 0x41e0000000000000
  0x000000ae421ef190: 0x0000000000000000 0x8000000000000000
  0x000000ae421ef1a0: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef1b0: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef1c0: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef1d0: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef1e0: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef1f0: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef200: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef210: 0x0000000000000000 0x0000000000000000
  0x000000ae421ef220: 0x0000000000000000 0x00000203b055f3c0
  0x000000ae421ef230: 0x00000203b04f4680 0x00000203b04c0020
  0x000000ae421ef240: 0x00000203b0b768ca 0x0000000000000000
  0x000000ae421ef250: 0x0000000000000009 0x00000203b0b768ca
  0x000000ae421ef260: 0x0000000000000002 0x00000203b0be44d0
  0x000000ae421ef270: 0x0000000000000000 0x00000203b0dc2ca0
  0x000000ae421ef280: 0x00000203b0b7a3c0 0x00000203b0b7a3c0
  0x000000ae421ef290: 0x0000000000000002 0x00007ff9a3e29cad
  0x000000ae421ef2a0: 0x0000000000000000 0x0000fab650f82655
  0x000000ae421ef2b0: 0x00007ffa777bbfb0 0x00007ffb65b3401b

Top frame info:
  TotalFrameSize in CodeInfoTable 48

Threads:
  0x00000203b05c4a40 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "System-1" - 0x00000203b2c01028, daemon, stack(0x000000ae42ba0000,0x000000ae42d90000)
  0x00000203b0b7a3c0 STATUS_IN_JAVA (PREVENT_VM_FROM_REACHING_SAFEPOINT) "System-0" - 0x00000203b2b01028, daemon, stack(0x000000ae429b0000,0x000000ae42ba0000)
  0x00000203b0bb11c0 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "Reference Handler" - 0x00000203b1bbe6d0, daemon, stack(0x000000ae427c0000,0x000000ae429b0000)
  0x00000203b0bfa840 STATUS_IN_NATIVE (ALLOW_SAFEPOINT) "main" - 0x00000203b1bbe620, stack(0x000000ae42000000,0x000000ae421f0000)

VM thread locals for the failing thread 0x00000203b0b7a3c0:
  0 (8 bytes): JNIThreadLocalEnvironment.jniFunctions = (bytes)
    0x00000203b0b7a3c0: 0x00000203b1757010
  8 (8 bytes): StackOverflowCheckImpl.stackBoundaryTL = (Word) 1 (0x0000000000000001)
  16 (4 bytes): Safepoint.safepointRequested = (int) 2147325056 (0x7ffd9480)
  20 (4 bytes): StatusSupport.statusTL = (int) 1 (0x00000001)
  24 (32 bytes): ThreadLocalAllocation.regularTLAB = (bytes)
    0x00000203b0b7a3d8: 0x00000203b2d00000 0x00000203b2e00000
    0x00000203b0b7a3e8: 0x00000203b2dbe288 0x0000000000000000
  56 (8 bytes): PlatformThreads.currentThread = (Object) java.lang.Thread (0x00000203b2b01028)
  64 (8 bytes): JavaFrameAnchors.lastAnchor = (Word) 0 (0x0000000000000000)
  72 (8 bytes): AccessControlContextStack = (Object) java.util.ArrayDeque (0x00000203b2b03a70)
  80 (8 bytes): ExceptionUnwind.currentException = (Object) null
  88 (8 bytes): IdentityHashCodeSupport.hashCodeGeneratorTL = (Object) java.util.SplittableRandom (0x00000203b2b5ebf0)
  96 (8 bytes): IsolatedCompileClient.currentClient = (Object) null
  104 (8 bytes): IsolatedCompileContext.currentContext = (Object) null
  112 (8 bytes): JNIObjectHandles.handles = (Object) com.oracle.svm.core.handles.ThreadLocalHandles (0x00000203b2b214e8)
  120 (8 bytes): JNIThreadLocalPendingException.pendingException = (Object) null
  128 (8 bytes): JNIThreadLocalPinnedObjects.pinnedObjectsListHead = (Object) null
  136 (8 bytes): JNIThreadOwnedMonitors.ownedMonitors = (Object) null
  144 (8 bytes): NoAllocationVerifier.openVerifiers = (Object) null
  152 (8 bytes): ThreadingSupportImpl.activeTimer = (Object) null
  160 (8 bytes): SubstrateDiagnostics.threadOnlyAttachedForCrashHandler = (bytes)
    0x00000203b0b7a460: 0x0000000000000000
  168 (8 bytes): ThreadLocalAllocation.allocatedBytes = (Word) 1044408 (0x00000000000fefb8)
  176 (8 bytes): VMThreads.IsolateTL = (Word) 2214877724672 (0x00000203b1000000)
  184 (8 bytes): VMThreads.OSThreadHandleTL = (Word) 496 (0x00000000000001f0)
  192 (8 bytes): VMThreads.OSThreadIdTL = (Word) 194356 (0x000000000002f734)
  200 (8 bytes): VMThreads.StackBase = (Word) 748443795456 (0x000000ae42ba0000)
  208 (8 bytes): VMThreads.StackEnd = (Word) 748441763840 (0x000000ae429b0000)
  216 (8 bytes): VMThreads.StartedByCurrentIsolate = (bytes)
    0x00000203b0b7a498: 0x0000000000000000
  224 (8 bytes): VMThreads.nextTL = (Word) 2214873207232 (0x00000203b0bb11c0)
  232 (8 bytes): VMThreads.unalignedIsolateThreadMemoryTL = (Word) 2214872982448 (0x00000203b0b7a3b0)
  240 (4 bytes): ActionOnExitSafepointSupport.actionTL = (int) 0 (0x00000000)
  244 (4 bytes): ActionOnTransitionToJavaSupport.actionTL = (int) 0 (0x00000000)
  248 (4 bytes): ImplicitExceptions.implicitExceptionsAreFatal = (int) 0 (0x00000000)
  252 (4 bytes): StackOverflowCheckImpl.yellowZoneStateTL = (int) 2130640638 (0x7efefefe)
  256 (4 bytes): StatusSupport.safepointBehaviorTL = (int) 1 (0x00000001)
  260 (4 bytes): ThreadingSupportImpl.currentPauseDepth = (int) 1 (0x00000001)

No VMOperation in progress

The 15 most recent VM operation status changes (oldest first):

Counters:

Java frame anchors for the failing thread 0x00000203b0b7a3c0:
  No anchors

Stacktrace for the failing thread 0x00000203b0b7a3c0:
  SP 0x000000ae421ef0c0 IP 0x00007ff9a2fc23a0  [image code] com.oracle.svm.core.jdk.VMErrorSubstitutions.shutdown(VMErrorSubstitutions.java:116)
  SP 0x000000ae421ef0c0 IP 0x00007ff9a2fc23a0  [image code] com.oracle.svm.core.jdk.VMErrorSubstitutions.shouldNotReachHere(VMErrorSubstitutions.java:109)
  SP 0x000000ae421ef0f0 IP 0x00007ff9a3018e64  [image code] com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:65)
  SP 0x000000ae421ef100 IP 0x00007ff9a2fb07ee  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.onYellowZoneMadeAvailable(StackOverflowCheckImpl.java:197)        
  SP 0x000000ae421ef120 IP 0x00007ff9a2fb0b22  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.makeYellowZoneAvailable(StackOverflowCheckImpl.java:172)
  SP 0x000000ae421ef120 IP 0x00007ff9a2fb0b22  [image code] com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.throwNewStackOverflowError(StackOverflowCheckImpl.java:308)       
  SP 0x000000ae421ef2a0 IP 0x00007ff9a3e29cad  [image code] net.sf.saxon.option.cpp.SaxonCAPI.destroy(SaxonCAPI.java:354)
  SP 0x000000ae421ef2e0 IP 0x00007ff9a2f55509  [image code] com.oracle.svm.core.code.IsolateEnterStub.SaxonCAPI_destroy_af4003d90d189ca5b0d8f9b7f6f7fbc3c3e762de(IsolateEnterStub.java:0)

VM mutexes:
  mutex "mainVMOperationControlWorkQueue" is unlocked.
  mutex "referencePendingList" is unlocked.
  mutex "thread" is unlocked.

AOT compiled code is mapped at 0x00007ff9a2f01000 - 0x00007ff9a483885f

Heap settings and statistics:
  Supports isolates: true
  Heap base: 0x00000203b1000000
  Object reference size: 8
  Aligned chunk size: 1048576
  Incremental collections: 0
  Complete collections: 0

Native image heap boundaries:
  ReadOnly Primitives: 0x00000203b1101028 - 0x00000203b147a108
  ReadOnly References: 0x00000203b147a108 - 0x00000203b1756068
  ReadOnly Relocatables: 0x00000203b1757000 - 0x00000203b199aca8
  Writable Primitives: 0x00000203b199b000 - 0x00000203b1ac99d0
  Writable References: 0x00000203b1ac99d0 - 0x00000203b1f83310
  Writable Huge: 0x00000203b2000030 - 0x00000203b2072040
  ReadOnly Huge: 0x00000203b2073030 - 0x00000203b267c1d0

Heap:
  Young generation:
    Eden:
      edenSpace:
        aligned: 0/0 unaligned: 0/0
    Survivors:
      Survivor-1 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-1 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-2 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-2 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-3 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-3 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-4 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-4 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-5 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-5 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-6 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-6 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-7 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-7 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-8 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-8 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-9 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-9 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-10 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-10 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-11 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-11 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-12 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-12 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-13 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-13 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-14 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-14 To:
        aligned: 0/0 unaligned: 0/0
      Survivor-15 From:
        aligned: 0/0 unaligned: 0/0
      Survivor-15 To:
        aligned: 0/0 unaligned: 0/0
  Old generation:
    oldFromSpace:
      aligned: 0/0 unaligned: 0/0
    oldToSpace:
      aligned: 0/0 unaligned: 0/0

  Unused:
    aligned: 0/0

Fatal error: StackOverflowError: Enabling the yellow zone of the stack did not make any stack space available. Possible reasons for that: 1) A call from native code to Java code provid
ed the wrong JNI environment or the wrong IsolateThread; 2) Frames of native code filled the stack, and now there is not even enough stack space left to throw a regular StackOverflowError; 3) An internal VM error occurred.

Please register to edit this issue

Actions
Send by e-mail Copy link

Also available in: Atom PDF