Bug #6392
closed
Memory leak in PySaxonProcessor (Python)
Estimated time:
(Total: 0:00 h)
Fix Committed on Branch:
12
Fixed in Maintenance Release:
Description
Consider the following code fragment:
from saxonche import PySaxonProcessor
import psutil
count = 0
prev = (mi := psutil.Process().memory_info)().rss
for _ in range(100):
with PySaxonProcessor(license=False):
pass
if (count := count + 1) % 10 == 0:
m = mi().rss
print(f"{m - prev:,}")
prev = m
It can be observed that memory usage increases over time. The exit functionality of the PySaxonProcessor class is not "cleaning up" as robustly as it should.
Python 3.12.2
saxonche 12.4.2
macOS 14.4.1
M2
Apologies for incorrect code formatting in initial submission:
from saxonche import PySaxonProcessor
import psutil
count = 0
prev = (mi := psutil.Process().memory_info)().rss
for _ in range(100):
with PySaxonProcessor(license=False):
pass
if (count := count + 1) % 10 == 0:
m = mi().rss
print(f"{m - prev:,}")
prev = m
- Project changed from Saxon-CE to SaxonC
Moving this project to the SaxonC project
- Status changed from New to In Progress
- Assignee set to Matt Patterson
- Status changed from In Progress to Resolved
- Fixed in version set to 12.5.0
This has been a bit tricky to track down, largely because it looks like we had fixed it as a side-effect of making more general changes after 12.4.2 released, and before this was reported.
After more extensive testing using memray and a debug-enabled build, I can't see any evidence of memory leaks in the current build, which I was able to see when build a debug-enabled version of 12.4.
It's worth noting that, depending on system memory constraints, and other environmental factors, running this example for a large number of iterations may not result in the RSS usage of the process stabilising. On my machine, I've been finding it needs thousands of iterations before the OS starts reclaiming some of the assigned-but-unused memory, but memray reveals that heap usage has settled, is constant and not increasing.
Because of the similarity of this to a number of memory-related issues that were opened, I'm going to mark the others as children of this issue...
- Status changed from Resolved to Closed
- Fixed in version changed from 12.5.0 to 12.5
Bug fix applied in the Saxon 12.5 Maintenance release.
- Fix Committed on Branch 12 added
Please register to edit this issue
Also available in: Atom
PDF
.