Skip to content

ci: one dev shell for developers, CI and the device - #123

Open
mrosseel wants to merge 3 commits into
nixosfrom
ci/tests-in-dev-shell
Open

mrosseel wants to merge 3 commits into
nixosfrom
ci/tests-in-dev-shell

Conversation

@mrosseel

Copy link
Copy Markdown
Owner

Developers, CI and the PiFinder now use the same Nix environment and one set of commands.

Faults this fixes

  • CI ran the checks with plain uv on x86 Ubuntu, with manylinux wheels. The docs said CI used nix develop.
  • There were three ruff versions: 0.4.8 in CI, 0.15.14 in the dev shell, and the ruff on PATH in pre-commit.
  • The dev shell used libcamera 0.7.0. The device uses 0.7.2 with the IMX290 patch.
  • Each workflow had its own copy of the Nix install and the cache login.
  • The web test workflow still used pip, requirements.txt and the removed tetra3 submodule.
  • PR builds pushed the mkShell output to the cache. That output references no packages, so the shell's packages did not go to the cache.

Changes

  • python/scripts/check.sh [lint|format|types|smoke|unit|integration] runs each check. It enters the dev shell when it is not already in it. CI, pre-commit and the docs use it.
  • ruff 0.4.8 is in the uv dev group. The nixpkgs ruff is removed from the shells.
  • The shell sets UV_NO_SYNC and UV_PROJECT_ENVIRONMENT, so uv run uses the Nix environment and makes no .venv.
  • nixos/libcamera-overlay.nix holds the libcamera overlay. The system and the dev shells both import it.
  • .github/actions/setup-nix installs Nix with the PiFinder cache. All jobs use it. Jobs that build another ref take it from a trusted .ci checkout.
  • .github/scripts/push_dev_shell.sh pushes the shell's inputDerivation. nox pushes the aarch64 shell after a green push, and a new x86 job pushes the x86 shells. PR builds and releases use the same script.
  • A new web dev shell adds chromium and chromedriver from nixpkgs without the overlay. The web tests run in it on aarch64.
  • nox runs on ubuntu-24.04-arm and also on pushes to nixos.

Tests

  • On x86: scripts/check.sh passes all stages (lint, types, smoke, unit, integration).
  • On x86: 52 web tests in nix develop .#web (test_web_interface.py and test_web_api_direct.py) pass against a fake-hardware PiFinder.
  • The device system derivation differs only in pifinder-src, the repo source. The libcamera derivation is the same in the system and in the aarch64 dev shell.
  • actionlint shows no new faults. The four faults it reports were there before.
  • The aarch64 checks and the cache pushes have not run yet. This PR runs the aarch64 nox job. The pushes run after the merge.

🤖 Generated with Claude Code

mrosseel and others added 3 commits November 10, 2026 15:33
CI ran the checks with plain uv on an x86 Ubuntu runner, with manylinux
wheels and ruff 0.4.8 from uvx. The dev shell had ruff 0.15 from nixpkgs.
The pre-commit hooks ran mypy on a different path. The docs said that CI
used nix develop, but it did not.

Now one environment and one script serve developers, pre-commit and CI:

- python/scripts/check.sh runs each stage. It enters the dev shell when
  it is not already in it.
- ruff 0.4.8 is in the uv dev group, so uv2nix puts it in the dev shell.
  The nixpkgs ruff is removed from both shells.
- The shell sets UV_NO_SYNC and UV_PROJECT_ENVIRONMENT, so uv uses the
  Nix environment and does not make a .venv of wheels.
- nox.yml runs check.sh on an aarch64 runner. The aarch64 dev shell is in
  the PiFinder cache. It also runs on pushes to nixos.
- The web test workflow uses the dev shell instead of pip and the removed
  tetra3 submodule.
- CLAUDE.md and the dev guide describe check.sh and drop the
  LD_LIBRARY_PATH workaround.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The workflows each had their own copy of the Nix install and the cache
login. The web tests ran on x86 with the browser of the runner image.
Only testable PR builds put the dev shell in the cache, and they pushed
the mkShell output, which references no packages. The dev shell also had
its own libcamera overlay with 0.7.0, but the device uses 0.7.2.

- .github/actions/setup-nix installs Nix with the PiFinder cache and logs
  in to Attic when a token exists. All jobs use it. Jobs that build code
  from another ref take it from a separate trusted checkout in .ci.
- .github/scripts/push_dev_shell.sh pushes the inputDerivation of a dev
  shell, which references every package in the shell.
- nox pushes the aarch64 dev shell after a green push. A new x86 job then
  pushes the x86 default and web shells. PR builds and releases push the
  aarch64 shell with the same script.
- The new web dev shell adds chromium and chromedriver from nixpkgs
  without the libcamera overlay, so both come from the NixOS cache. The
  web tests run in it on an aarch64 runner.
- nixos/libcamera-overlay.nix holds the device libcamera overlay. The
  system and the dev shells both import it, so both use the same
  libcamera derivation. The device packages do not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adafruit Blinka knows a Raspberry Pi and a generic x86 PC. On a generic
aarch64 host, such as the GitHub arm runner, it finds no chip, and every
module that does "import board" fails. The tests use no real hardware,
so conftest.py sets BLINKA_FORCECHIP and BLINKA_FORCEBOARD to the generic
PC values unless they are already set. The web test workflow sets the
same values for the fake-hardware PiFinder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant