Project

Profile

Help

Feature #5168

closed

Creation of PyPI Packages

Added by Al S over 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Python Build
Start date:
2021-11-18
Due date:
2021-11-26
% Done:

100%

Estimated time:
Found in version:
Fixed in version:
12.0
Platforms:

Description

I have had wanted to build Python tooling with the Saxon/C module and SWIG bindings and many colleagues find the library installation process tedious because it does not involve a simple pip install saxon command. In the past, assembling binary dependencies was harder, but now much simpler with the use of Python wheel deployments and improved tooling.

Would Saxon developers be willing to work towards installation in PyPI, like SaxonJS is in NPMJS? If not, given the MPL licensing, is it acceptable for community member like myself to do so (with your permission) without significant effort to modify branding if required?


Related issues

Is duplicate of SaxonC - Feature #4375: Create a PIP installable Python packageDuplicateO'Neil Delpratt2019-11-06

Actions
Actions #1

Updated by O'Neil Delpratt over 2 years ago

  • Is duplicate of Feature #4375: Create a PIP installable Python package added
Actions #2

Updated by O'Neil Delpratt over 2 years ago

Thanks for raising this issue. We'd prefer this to be done by Saxonica rather than a third party, but would appreciate any help you can offer.

Actions #3

Updated by Al S over 2 years ago

O'Neil Delpratt wrote:

Thanks for raising this issue. We'd prefer this to be done by Saxonica rather than a third party, but would appreciate any help you can offer.

Ok how can we proceed with that? Do you have some kind of CI/CD infrastructure such that, one a release is ready, and it will build the code and run additional steps to prepare the compressed archive on the website? I can work with you on examples with shell script, Makefile build steps, but how we approach this for the community largely depends on the tooling you use and what you are comfortable with.

Actions #4

Updated by O'Neil Delpratt over 2 years ago

Hi, Lets discuss this next week. Probably best to continue discussion via email () and on a Zoom/MS Team meeting.

Actions #5

Updated by Al S over 2 years ago

O'Neil Delpratt wrote:

Hi, Lets discuss this next week. Probably best to continue discussion via email () and on a Zoom/MS Team meeting.

Sounds good.

Actions #6

Updated by O'Neil Delpratt over 2 years ago

  • Status changed from New to AwaitingInfo

Hi, trying to reach out to you, but cannot see your email.

Actions #7

Updated by Al S over 2 years ago

O'Neil Delpratt wrote:

Hi, trying to reach out to you, but cannot see your email.

I am sorry about that, I dropped the ball here. Sending you an email.

Actions #8

Updated by O'Neil Delpratt over 1 year ago

  • Status changed from AwaitingInfo to In Progress

I thought I would update this bug issue on pypi since it has been a while. I have had some success in building a pypi package for SaxonC 12.1 (not yet released) for a local server. I have used the github project for saxonpy as a starting place.

For SaxonC 12.1 the library has been built using graalvm's native-image, which builds a shared library required for the C++ API and therefore the python extension linking at execution time.

Actions #9

Updated by O'Neil Delpratt over 1 year ago

For the python extension to work the shared library created by native-image has to be copied in the /usr/lib directory instead of the pip default location. This is not ideal. I am still investigating how to change the setup.py to have the library in the correct location.

Actions #10

Updated by O'Neil Delpratt over 1 year ago

See error:

ImportError: /home/ond1/.local/lib/python3.8/site-packages/saxonc.cpython-38-x86_64-linux-gnu.so: undefined symbol: j_callTemplateReturningValue

I found this forum post similar to my problem: https://stackoverflow.com/questions/72052512/importerror-so-cannot-open-shared-object-file-after-pip-installing-extensi

Actions #11

Updated by O'Neil Delpratt over 1 year ago

I have managed to solve my issue on loading the graalvm shared library. See solution below:

extra_link_args = []


if system() == 'Darwin':
    saxon_home = 'libs/darwin'
else:
    saxon_home = 'libs/nix'
    extra_link_args.append('-Wl,-rpath,' + saxon_home)

ext_modules = [Extension(
                        "saxonc", 
                        sources=[ ..., extra_link_args = extra_link_args,
                       libraries=['saxon-eec-12.1'], ... ), ... ]
Actions #12

Updated by Martin Honnen over 1 year ago

Will there also be a wheel for Windows? Just wondering and asking, admittedly I don't understand details of those scripts but

if system() == 'Darwin':
    saxon_home = 'libs/darwin'
else:
    saxon_home = 'libs/nix'

sounds as if Windows is not on the list.

Actions #13

Updated by O'Neil Delpratt over 1 year ago

Yes we plan to provide a wheel for windows too. Something like the following: https://github.com/tennom/saxonpy-win

Actions #14

Updated by O'Neil Delpratt over 1 year ago

  • Category set to Python Build
  • % Done changed from 0 to 100
  • Fixed in version set to 12.0

Happy to report that we have released SaxonC pypi package. See: https://pypi.org/user/saxonica/

This is available on a number of platforms and distributions including windows, Linux and MacOS. Therefore I am marking this bug issue as closed.

Actions #15

Updated by O'Neil Delpratt over 1 year ago

  • Status changed from In Progress to Closed

Please register to edit this issue

Also available in: Atom PDF