Bug #5858
closed
Python Saxonche module doesn't work using Pyinstaller and UPX processing to create executable file
Added by Pierre Dostie almost 2 years ago. Updated about 1 year ago.
0%
Description
I'm using Pyinstaller with UPX processing to create executable files from Python scripts. I was using a Python module Saxonpy and decided to move away from this module to use instead Saxonche. The issue I have now by using this module is that my executable file no longer work when I create them using Pyinstaller with UPX processing. When I use Pyinsataller alone without UPX to create my executable files, it works but the size of my executable files are twice as big as if they were created using UPX processing.
Files
Updated by O'Neil Delpratt almost 2 years ago
Please can you detail the error message for description how it fails.
Updated by Pierre Dostie almost 2 years ago
The executable file simply doesn't launch as expected. For example, an executable from which a window should appear to compute information never pops up.
I have a simple Python script (the same one I sent you for the nodekind issue) that I tried to convert to an executable. This executable should generate a xml file. Using UPX processing with Pyinstaller would never generate the expected file whereas it does if the executable was created without UPX.
I'm using UPX 4.0.1 on Windows 64 and I attached the Pyhton script in question.
Updated by Norm Tovey-Walsh almost 2 years ago
I wasn't able to reproduce the problem, but I also wasn't able to see any improvement in the file size with UPX.
PS C:\tmp> del dist
Confirm
The item at C:\tmp\dist has children and the Recurse parameter was not specified. If you continue, all children will be
removed with the item. Are you sure you want to continue?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): A
PS C:\tmp> pyinstaller.exe -F .\test.pyw
156 INFO: PyInstaller: 5.7.0
156 INFO: Python: 3.8.10
171 INFO: Platform: Windows-10-10.0.19045-SP0
171 INFO: wrote C:\tmp\test.spec
187 INFO: UPX is not available.
187 INFO: Extending PYTHONPATH with paths
['C:\\tmp']
703 INFO: checking Analysis
734 INFO: checking PYZ
749 INFO: checking PKG
765 INFO: Bootloader C:\Users\Norman Walsh\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\PyInstaller\bootloader\Windows-64bit-intel\runw.exe
765 INFO: checking EXE
781 INFO: Rebuilding EXE-00.toc because test.exe missing
781 INFO: Building EXE from EXE-00.toc
781 INFO: Copying bootloader EXE to C:\tmp\dist\test.exe.notanexecutable
891 INFO: Copying icon to EXE
891 INFO: Copying icons from ['C:\\Users\\Norman Walsh\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python38\\site-packages\\PyInstaller\\bootloader\\images\\icon-windowed.ico']
984 INFO: Writing RT_GROUP_ICON 0 resource with 104 bytes
984 INFO: Writing RT_ICON 1 resource with 3752 bytes
1000 INFO: Writing RT_ICON 2 resource with 2216 bytes
1000 INFO: Writing RT_ICON 3 resource with 1384 bytes
1000 INFO: Writing RT_ICON 4 resource with 38188 bytes
1000 INFO: Writing RT_ICON 5 resource with 9640 bytes
1000 INFO: Writing RT_ICON 6 resource with 4264 bytes
1000 INFO: Writing RT_ICON 7 resource with 1128 bytes
1000 INFO: Copying 0 resources to EXE
1000 INFO: Embedding manifest in EXE
1000 INFO: Updating manifest in C:\tmp\dist\test.exe.notanexecutable
1110 INFO: Updating resource type 24 name 1 language 0
1110 INFO: Appending PKG archive to EXE
1125 INFO: Fixing EXE headers
3375 INFO: Building EXE from EXE-00.toc completed successfully.
PS C:\tmp> dir dist
Directory: C:\tmp\dist
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 1/27/2023 2:06 PM 22654913 test.exe
And with UPX:
PS C:\tmp> del dist
Confirm
The item at C:\tmp\dist has children and the Recurse parameter was not specified. If you continue, all children will be
removed with the item. Are you sure you want to continue?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"): A
PS C:\tmp> pyinstaller.exe -F --upx-dir upx .\test.pyw
140 INFO: PyInstaller: 5.7.0
140 INFO: Python: 3.8.10
156 INFO: Platform: Windows-10-10.0.19045-SP0
156 INFO: wrote C:\tmp\test.spec
203 INFO: UPX is available.
203 INFO: Extending PYTHONPATH with paths
['C:\\tmp']
719 INFO: checking Analysis
733 INFO: checking PYZ
750 INFO: checking PKG
765 INFO: Building because upx_binaries changed
765 INFO: Building PKG (CArchive) test.pkg
16562 INFO: Building PKG (CArchive) test.pkg completed successfully.
16578 INFO: Bootloader C:\Users\Norman Walsh\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\PyInstaller\bootloader\Windows-64bit-intel\runw.exe
16578 INFO: checking EXE
16594 INFO: Rebuilding EXE-00.toc because test.exe missing
16594 INFO: Building EXE from EXE-00.toc
16594 INFO: Copying bootloader EXE to C:\tmp\dist\test.exe.notanexecutable
16688 INFO: Copying icon to EXE
16688 INFO: Copying icons from ['C:\\Users\\Norman Walsh\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python38\\site-packages\\PyInstaller\\bootloader\\images\\icon-windowed.ico']
16797 INFO: Writing RT_GROUP_ICON 0 resource with 104 bytes
16797 INFO: Writing RT_ICON 1 resource with 3752 bytes
16797 INFO: Writing RT_ICON 2 resource with 2216 bytes
16797 INFO: Writing RT_ICON 3 resource with 1384 bytes
16797 INFO: Writing RT_ICON 4 resource with 38188 bytes
16797 INFO: Writing RT_ICON 5 resource with 9640 bytes
16797 INFO: Writing RT_ICON 6 resource with 4264 bytes
16797 INFO: Writing RT_ICON 7 resource with 1128 bytes
16812 INFO: Copying 0 resources to EXE
16812 INFO: Embedding manifest in EXE
16812 INFO: Updating manifest in C:\tmp\dist\test.exe.notanexecutable
16922 INFO: Updating resource type 24 name 1 language 0
16922 INFO: Appending PKG archive to EXE
16953 INFO: Fixing EXE headers
19234 INFO: Building EXE from EXE-00.toc completed successfully.
PS C:\tmp> dir dist
Directory: C:\tmp\dist
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 1/27/2023 2:08 PM 22654913 test.exe
I'd expect the executable to be quite large because it has to embed the SaxonC HE library.
Updated by Norm Tovey-Walsh almost 2 years ago
The UPX version does seem to work however. Sort of.
PS C:\tmp> cd dist
PS C:\tmp\dist> copy ..\test.xslt .
PS C:\tmp\dist> .\test.exe
PS C:\tmp\dist> dir
Directory: C:\tmp\dist
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 1/27/2023 2:08 PM 22654913 test.exe
-a---- 1/27/2023 1:52 PM 504 test.xslt
PS C:\tmp\dist> .\test.exe
PS C:\tmp\dist> dir
Directory: C:\tmp\dist
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 1/27/2023 2:10 PM 226 out.xml
-a---- 1/27/2023 2:08 PM 22654913 test.exe
-a---- 1/27/2023 1:52 PM 504 test.xslt
The first attempt to run test.exe takes several seconds and produces no output. The second run is very quick and produces the expected output. I have no explanation for that.
Updated by Pierre Dostie almost 2 years ago
Thank you for looking into that. Pyinstaller has a --upx-exclude option to exclude binary file such as dll because they might get corrupted using UPX.
I've tried using this option to exclude "libsaxon-hec-12.0.dll" but to no avail.
I don't know if I didn't use this option properly though.
Updated by Pierre Dostie almost 2 years ago
As for UPX not reducing the size, the file I sent is only an example to isolate the issue. The original script has many more modules and using UPX reduces greatly the size of the executable file.
I can live with UPX not reducing the size of Saxonche module at all but would be quite happy if others could be reduced as they used to when we were using Saxonpy instead of Saxonche. The reason we decided to move away from Saxonpy is that is was using a Java technology that is no longer supported. Saxonche fixes issues some users were having but the size of the executable file is not optimal.
Please register to edit this issue
.