Open
Description
Affects: PythonCall / JuliaCall / Both [delete as appropriate]
Describe the bug
Using Python 3.12.8 and Julia 1.11, when I type <pyobject>.
into the REPL, I see a segfault.
Your system
julia> versioninfo()
Julia Version 1.11.2
Commit 5e9a32e7af2 (2024-12-01 20:02 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 64 × AMD Ryzen Threadripper 3970X 32-Core Processor
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver2)
Threads: 60 default, 0 interactive, 30 GC (on 64 virtual cores)
Environment:
JULIA_NUM_THREADS = 60
JULIA_PROJECT = @.
JULIA_EDITOR = emacs -nw
(...) pkg> st
Project HunFlair2 v0.1.0
Status `~/.../Project.toml`
[992eb4ea] CondaPkg v0.2.24
[6099a3de] PythonCall v0.9.23
(...) pkg> conda st
CondaPkg Status /.../CondaPkg.toml
Environment
/.../.CondaPkg/env
Pip packages
flair v0.14.0
Additional context
julia> s.
[57673] signal 11 (1): Segmentation fault
in expression starting at none:0
_PyInterpreterState_GET at /usr/local/src/conda/python-3.12.8/Include/internal/pycore_pystate.h:133 [inlined]
_PyType_Lookup at /usr/local/src/conda/python-3.12.8/Objects/typeobject.c:4729 [inlined]
_PyObject_LookupSpecial at /usr/local/src/conda/python-3.12.8/Objects/typeobject.c:2167
_dir_object at /usr/local/src/conda/python-3.12.8/Objects/object.c:1758 [inlined]
PyObject_Dir at /usr/local/src/conda/python-3.12.8/Objects/object.c:1790
PyObject_Dir at /home/tec/.julia/packages/PythonCall/Nr75f/src/C/pointers.jl:303 [inlined]
macro expansion at /home/tec/.julia/packages/PythonCall/Nr75f/src/Core/Py.jl:132 [inlined]
pydir at /home/tec/.julia/packages/PythonCall/Nr75f/src/Core/builtins.jl:216
propertynames at /home/tec/.julia/packages/PythonCall/Nr75f/src/Core/Py.jl:293
unknown function (ip: 0x7fab413505eb)
complete_symbol at /cache/build/tester-amdci5-12/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPLCompletions.jl:208
[...]
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
protogeezer commentedon Jan 23, 2025
I see that crash trying to use the Matlab PythonAPI installed in a conda venv:
(@v1.11) pkg> status
Status
~/.julia/environments/v1.11/Project.toml
[8f4d0f93] Conda v1.10.2
[992eb4ea] CondaPkg v0.2.24
[497a8b3b] DoubleFloats v1.4.2
[6099a3de] PythonCall v0.9.24
cjdoris commentedon Feb 22, 2025
Thanks both for reporting this. However I cannot reproduce this locally, e.g. I can do:
@tecosaur You didn't specify what
s
is in your example. Does the error occur with any Python object, or just certain ones? I notice you started Julia with multithreading, do you still get the error with one thread?@protogeezer I can't install
matlabengine
because I don't have matlab. Have you seen the problem elsewhere?Edit: I tried reproducing on Windows and on Linux, with 1 thread and with many.
protogeezer commentedon Feb 22, 2025
Fair enough. I submitted #591 (mentioned above) with more details. I tried the simple test you described - it didn't work here - it hung. However the regular expression example from the documentation still crashes as soon as I try to reference the imported regular expression module.
One additional detail is that I use python from Miniforge3. I do not use the base environment.
If you need further experiments, please let me know. Try more versions of Python? Do we need to use the "built in" version of python"?
I have contemplated getting more involved with this package because I depend on it so much. I would need some coaching to get off the ground. Actually even how to get on the runway... I did get to the point where I was fixing crashes in PyCall before PythonCall came along...
Stephen
cjdoris commentedon Feb 23, 2025
Can you give the steps to reproduce your environment please? Including versions of downloaded software and any settings you set.
tecosaur commentedon Feb 24, 2025
I had the impression it may be with any object, but it was originally with a module and I've just run into it again so I can give a more detailed reproducer:
Seems like this actually gives you two different errors for the price of one, fun! 😛
Since this was starting from an empty project using Julia 1.11.3 and the same system as the original issue, I think this should be all the relevant information?
tecosaur commentedon Feb 24, 2025
Huh, it seems alright if only one thread is used.
cjdoris commentedon Feb 24, 2025
Those examples are with Julia 1.12 which doesn't exist yet (??) do you get the same on 1.11?
tecosaur commentedon Feb 25, 2025
Ah oops, I meant to use 1.11 but mis-typed. Yep, it's the same experience.
protogeezer commentedon Feb 25, 2025
I have some news to report. Pursuant to Chris's request, a couple days ago, for the steps to reproduce my environment, what I was doing to create the recipe seems to have made things somewhat better.
Here is the example from the beginning of the PythonCall guide that didn't work (see the crash in a previous comment).
However, Chris's previous example still crashes:
At this point I've tried deleting the "official" macOS julia app (which stopped working for some reason - the error message had something to do with not being able to launch an app(?) in a terminal window); several old versions of julia that I built for analysis (which has now been obviated by juliaup); and finally I deleted and rebuilt the 1.11 env and reinstalled PythonCall. I suppose I could try deleting .julia to see if the problem really is a software conflict rather than a bug.
I'll try to finish rooting out possible conflicts in the next few days.
Stephen
protogeezer commentedon Mar 2, 2025
I have developed a simpler set of steps to cause the crash. This is what happens on the Dell XPS 17 3790 running Kubuntu 24.10:
The paths in the call stack appear to be gibberish - the majority certainly aren't paths that exist on the machine.
Here are the relevant settings in the .bashrc file:
These variable setting appear to work on macOS Sequoia (15.3) on a MBP with Apple Silicon. I'll continue to try to narrow down the example. Both Miniforge3 is writeable on both machines.
I hope this is useful.
Stephen
Beforerr commentedon Mar 5, 2025
This somehow could prevent the Segmentation fault
protogeezer commentedon Mar 5, 2025
@Beforerr: could you be a bit more specific about where your code should go?
With more testing I could soon conclude that the crash tecosaur and I reported depends only on the version of python that PythonCall is configured with. So far, I have been unable to reproduce the crash with python 3.11. Not so with Python 3.12. So far this pattern holds on a MacBook Pro M1 running sequoia, and a Dell with an i9 running Ubuntu 24.10.
Stephen
Beforerr commentedon Mar 6, 2025
It is just a workaround (putting attributes into Julia so
propertynames
would not call py related function).P.S:. It also happens in Mac Studio M2, M1 and Python 3.11
19 remaining items