Skip to content

When having on --with-system-expat, system still builds using bundled expat for tests #92875

New issue
New issue
Open
Open
When having on --with-system-expat, system still builds using bundled expat for tests#92875
Labels
buildThe build process and cross-buildtestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error
@mcepl

Description

@mcepl
mcepl
opened on May 17, 2022 · edited by mcepl
Contributor

Bug report

When building 3.11.0b1 for openSUSE (openSUSE/Tumbleweed as of 2022-05-17) we have --with-system-expat on (of course) and just to be sure we also rm -rf Modules/expat (to be sure, we really don't use bundled expat). Unfortunately, in such situation test_freeze_simple_script in test.test_tools.test_freeze.TestFreeze fails:

[ 1020s] test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... creating the script to be frozen at /tmp/tmpl92b_12m/app.py
[ 1020s] copying the source tree into /tmp/tmpl92b_12m/cpython...
[ 1020s] configuring python in /tmp/tmpl92b_12m/python-build...
[ 1020s] building python in /tmp/tmpl92b_12m/python-build...
[ 1020s] CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpl92b_12m/python-build', '-j8']' returned non-zero exit status 2.

[Plenty of lines of gcc spew skipped]

[ 1139s] make[1]: Entering directory '/tmp/tmpraer2tjx/python-build'
[ 1139s] gcc -c -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I/tmp/tmpraer2tjx/cpython/Include/internal -IObjects -IInclude -IPython -I. -I/tmp/tmpraer2tjx/cpython/Include    -fPIC -I/tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec -DCONFIG_64=1 -DANSI=1 -DHAVE_UINT128_T=1 -o Modules/_decimal/libmpdec/mpdecimal.o /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/mpdecimal.c
[ 1139s] make[1]: Leaving directory '/tmp/tmpraer2tjx/python-build'
[ 1139s] 
[ 1139s] --- STDERR ---
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c: In function ‘PySys_SetArgv’:
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c:3305:5: warning: ‘PySys_SetArgvEx’ is deprecated [-Wdeprecated-declarations]
[ 1139s]  3305 |     PySys_SetArgvEx(argc, argv, Py_IsolatedFlag == 0);
[ 1139s]       |     ^~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Python/sysmodule.c:3259:1: note: declared here
[ 1139s]  3259 | PySys_SetArgvEx(int argc, wchar_t **argv, int updatepath)
[ 1139s]       | ^~~~~~~~~~~~~~~
[ 1139s] make[1]: *** No rule to make target '/tmp/tmpraer2tjx/cpython/Modules/expat/xmlparse.c', needed by 'Modules/expat/xmlparse.o'.  Stop.
[ 1139s] make[1]: *** Waiting for unfinished jobs....
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:611:18:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:607:18:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] In function ‘word_to_string’,
[ 1139s]     inlined from ‘coeff_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:410:13,
[ 1139s]     inlined from ‘_mpd_to_string’ at /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:501:22:
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:355:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   355 |     case 19: EXTRACT_DIGIT(s, x, 1000000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:348:40: warning: array subscript 0 is outside array bounds of ‘char[0]’ [-Warray-bounds]
[ 1139s]   348 |         if (s == dot) *s++ = '.'; *s++ = '0' + (char)(x / d); x %= d
[ 1139s]       |                                   ~~~~~^~~~~~~~~~~~~~~~~~~~~
[ 1139s] /tmp/tmpraer2tjx/cpython/Modules/_decimal/libmpdec/io.c:356:14: note: in expansion of macro ‘EXTRACT_DIGIT’
[ 1139s]   356 |     case 18: EXTRACT_DIGIT(s, x, 100000000000000000ULL, dot);
[ 1139s]       |              ^~~~~~~~~~~~~
[ 1139s] 
[ 1139s] ---- END ----
[ 1139s] ERROR
[ 1139s] 
[ 1139s] ======================================================================
[ 1139s] ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
[ 1139s] ----------------------------------------------------------------------
[ 1139s] Traceback (most recent call last):
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
[ 1139s]     outdir, scriptfile, python = helper.prepare(script, outdir)
[ 1139s]                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Tools/freeze/test/freeze.py", line 159, in prepare
[ 1139s]     _run_quiet([MAKE, '-C', builddir, '-j8'])
[ 1139s]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Tools/freeze/test/freeze.py", line 25, in _run_quiet
[ 1139s]     return subprocess.run(
[ 1139s]            ^^^^^^^^^^^^^^^
[ 1139s]   File "/home/abuild/rpmbuild/BUILD/Python-3.11.0b1/Lib/subprocess.py", line 558, in run
[ 1139s]     raise CalledProcessError(retcode, process.args,
[ 1139s]     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ 1139s] subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpraer2tjx/python-build', '-j8']' returned non-zero exit status 2.
[ 1139s]

Complete build log with all details of building the package.

Activity

mcepl
added
type-bugAn unexpected behavior, bug, or error
on May 17, 2022
AlexWaygood
added
testsTests in the Lib/test dir
on May 17, 2022
mcepl

mcepl commented on May 17, 2022

@mcepl
mcepl
on May 17, 2022
ContributorAuthor

OK, so it is enough just to set CONFIG_ARGS variable to something like "--enable-ipv6 --enable-shared --with-ensurepip=no --with-system-ffi --with-system-expat --with-lto --enable-optimizations --with-system-libmpdec --enable-loadable-sqlite-extensions".

mcepl
closed this as completedon May 17, 2022
mcepl

mcepl commented on May 17, 2022

@mcepl
mcepl
on May 17, 2022
ContributorAuthor

Nope, it doesn't help. When I remove the bundled expat it fails just the same even with CONFIG_ARGS variable set.

mcepl
reopened this on May 17, 2022
erlend-aasland
added
buildThe build process and cross-build
on May 23, 2022
sharewax

sharewax commented on Aug 12, 2022

@sharewax
sharewax
on Aug 12, 2022 · edited by sharewax

One more report for it.
Trying to build SCL for Centos 7, issue is the same:

make: Leaving directory `/tmp/tmpzdbz0bvg/python-build'
--- STDERR ---
/tmp/tmpzdbz0bvg/cpython/Objects/obmalloc.c:1449:1: warning: always_inline function might not be inlinable [-Wattributes]
 1449 | arena_map_get(block *p, int create)
      | ^~~~~~~~~~~~~
make: *** No rule to make target `/tmp/tmpzdbz0bvg/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
make: *** Waiting for unfinished jobs....
---- END ----
ERROR
======================================================================
ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.11.0rc1/Lib/test/test_tools/test_freeze.py", line 27, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0rc1/Tools/freeze/test/freeze.py", line 159, in prepare
    _run_quiet([MAKE, '-C', builddir, '-j8'])
  File "/builddir/build/BUILD/Python-3.11.0rc1/Tools/freeze/test/freeze.py", line 25, in _run_quiet
    return subprocess.run(
           ^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/Python-3.11.0rc1/Lib/subprocess.py", line 569, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpzdbz0bvg/python-build', '-j8']' returned non-zero exit status 2.
----------------------------------------------------------------------
Ran 1 test in 78.506s
FAILED (errors=1)
test test_tools failed
1 test failed again:
    test_tools
== Tests result: FAILURE then FAILURE ==
409 tests OK.
10 slowest tests:
- test_concurrent_futures: 2 min 36 sec
- test_multiprocessing_spawn: 2 min 28 sec
- test_subprocess: 2 min 27 sec
- test_asyncio: 1 min 36 sec
- test_tools: 1 min 36 sec
- test_multiprocessing_forkserver: 1 min 28 sec
- test_multiprocessing_fork: 1 min 16 sec
- test_signal: 49.0 sec
- test_io: 35.8 sec
- test_compileall: 33.4 sec
1 test failed:
    test_tools
24 tests skipped:
    test_curses test_devpoll test_idle test_ioctl test_kqueue
    test_launcher test_msilib test_ossaudiodev test_smtpnet
    test_socketserver test_startfile test_tcl test_tix test_tk
    test_ttk_guionly test_ttk_textonly test_turtle test_urllib2net
    test_urllibnet test_winconsoleio test_winreg test_winsound
    test_xmlrpc_net test_zipfile64
1 re-run test:
    test_tools
Total duration: 4 min 27 sec
Tests result: FAILURE then FAILURE

I removed the internal Modules/expat, but have no idea then it takes make target for xmlparse in this case.

# grep expat build.log
+ rm -r Modules/expat
checking for --with-system-expat... yes
checking for stdlib extension module pyexpat... yes
building 'pyexpat' extension
...
0:00:33 load avg: 7.96 [264/434] test_pyexpat passed -- running: test_asyncio (32.8 sec)
make: *** No rule to make target `/tmp/tmp02i4l09d/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.
make: *** No rule to make target `/tmp/tmpzdbz0bvg/cpython/Modules/expat/xmlparse.c', needed by `Modules/expat/xmlparse.o'.  Stop.

Looks like #91752

Removed rm -r Modules/expat from spec file - all compiled ok. Centos7.

mcepl
mentioned this on Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildtestsTests in the Lib/test dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @mcepl@sharewax@erlend-aasland@AlexWaygood

        Issue actions
          When having on --with-system-expat, system still builds using bundled expat for tests · Issue #92875 · python/cpython