using emdawnwebgpu port shows file lock error

[jspanchu]

Just bumped the emsdk version to 4.0.10 and started to experiment with --use-port=emdawnwebgpu. For some reason this error does not consistently happen and occurs at random.

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.10 (b7dc6e5747465580df5984e723b9d1f10d8e804b)

Error:

FAILED: Rendering/WebGPU/CMakeFiles/RenderingWebGPU.dir/PointCloudMapperCopyDepthToWindow.cxx.o 
/VTK-src/.gitlab/sccache /emsdk/upstream/emscripten/em++ -Dkiss_fft_scalar=double -I/VTK-src/Rendering/WebGPU -I/VTK-src/build/Rendering/WebGPU -I/VTK-src/Common/Core -I/VTK-src/build/Common/Core -I/VTK-src/ThirdParty/token/vtktoken/token -I/VTK-src/build/ThirdParty/token/vtktoken/token -I/VTK-src/ThirdParty/token/vtktoken -I/VTK-src/build/ThirdParty/token/vtktoken -I/VTK-src/ThirdParty/nlohmannjson/vtknlohmannjson -I/VTK-src/build/ThirdParty/nlohmannjson/vtknlohmannjson -I/VTK-src/ThirdParty/nlohmannjson/vtknlohmannjson/include -I/VTK-src/Rendering/Core -I/VTK-src/build/Rendering/Core -I/VTK-src/Common/DataModel -I/VTK-src/build/Common/DataModel -I/VTK-src/Common/Math -I/VTK-src/build/Common/Math -I/VTK-src/ThirdParty/kissfft/vtkkissfft -I/VTK-src/build/ThirdParty/kissfft/vtkkissfft -I/VTK-src/Common/Transforms -I/VTK-src/build/Common/Transforms -I/VTK-src/Common/ExecutionModel -I/VTK-src/build/Common/ExecutionModel -I/VTK-src/Filters/Core -I/VTK-src/build/Filters/Core -I/VTK-src/Common/Misc -I/VTK-src/build/Common/Misc -I/VTK-src/IO/Image -I/VTK-src/build/IO/Image -I/VTK-src/Imaging/Core -I/VTK-src/build/Imaging/Core -I/VTK-src/Common/System -I/VTK-src/build/Common/System -I/VTK-src/ThirdParty/fmt/vtkfmt -I/VTK-src/build/ThirdParty/fmt/vtkfmt -isystem /VTK-src/build/Utilities/KWIML -isystem /VTK-src/Utilities/KWIML -isystem /VTK-src/build/Utilities/KWSys -isystem /VTK-src/Utilities/KWSys -isystem /VTK-src/build/ThirdParty/token -isystem /VTK-src/ThirdParty/token -isystem /VTK-src/build/ThirdParty/nlohmannjson -isystem /VTK-src/ThirdParty/nlohmannjson -isystem /VTK-src/build/ThirdParty/kissfft -isystem /VTK-src/ThirdParty/kissfft -isystem /VTK-src/build/ThirdParty/fmt -isystem /VTK-src/ThirdParty/fmt -Wno-limited-postlink-optimizations  -fwasm-exceptions -O3 -DNDEBUG -std=c++17 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden --use-port=emdawnwebgpu -MD -MT Rendering/WebGPU/CMakeFiles/RenderingWebGPU.dir/PointCloudMapperCopyDepthToWindow.cxx.o -MF Rendering/WebGPU/CMakeFiles/RenderingWebGPU.dir/PointCloudMapperCopyDepthToWindow.cxx.o.d -o Rendering/WebGPU/CMakeFiles/RenderingWebGPU.dir/PointCloudMapperCopyDepthToWindow.cxx.o -c /VTK-src/build/Rendering/WebGPU/PointCloudMapperCopyDepthToWindow.cxx
Traceback (most recent call last):
  File "/emsdk/upstream/emscripten/em++.py", line 15, in <module>
    sys.exit(emcc.main(sys.argv))
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/emsdk/upstream/emscripten/emcc.py", line 991, in main
    ret = run(args)
  File "/emsdk/upstream/emscripten/emcc.py", line 447, in run
    options, newargs = phase_parse_arguments(state)
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/emsdk/upstream/emscripten/emcc.py", line 602, in phase_parse_arguments
    options, settings_changes, user_js_defines, newargs = cmdline.parse_args(newargs)
  File "/emsdk/upstream/emscripten/tools/cmdline.py", line 587, in parse_args
    ports.handle_use_port_arg(settings, consume_arg())
  File "/emsdk/upstream/emscripten/tools/ports/__init__.py", line 599, in handle_use_port_arg
    handle_port_deps(name, error_handler)
  File "/emsdk/upstream/emscripten/tools/ports/__init__.py", line 565, in handle_port_deps
    port = get_port_by_name(name)
  File "/emsdk/upstream/emscripten/tools/ports/__init__.py", line 45, in get_port_by_name
    load_external_port(port)
  File "/emsdk/upstream/emscripten/tools/ports/__init__.py", line 100, in load_external_port
    up_to_date = Ports.fetch_port_artifact(name, external_port.EXTERNAL_PORT, external_port.SHA512)
  File "/emsdk/upstream/emscripten/tools/ports/__init__.py", line 394, in fetch_port_artifact
    with cache.lock('unpack port'):
  File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/emsdk/upstream/emscripten/tools/cache.py", line 72, in lock
    release_cache_lock()
  File "/emsdk/upstream/emscripten/tools/cache.py", line 61, in release_cache_lock
    cachelock.release()
  File "/emsdk/upstream/emscripten/tools/filelock.py", line 323, in release
    self._release()
  File "/emsdk/upstream/emscripten/tools/filelock.py", line 413, in _release
    os.unlink(self._lock_file)
FileNotFoundError: [Errno 2] No such file or directory: '/emsdk/upstream/emscripten/cache/cache.lock'

Complete build logs can be seen at https://gitlab.kitware.com/vtk/vtk-wasm-sdk/-/jobs/11300797/raw

[sbc100]

Are you able to reproduce this consistently?

[sbc100]

Sorry I just read that you its not consistent. It guess there must be some kind of race condition when removing the lock, likely between different calls to emcc. I imagine the building with -j1 would make the problem go away?

[jspanchu]

Would it be easier if emscripten suppressed the exception?

from contextlib import suppress

with suppress(FileNotFoundError):
  os.unlink(self._lock_file)