Repository navigation
fix(build): the lift key is taken again only if the lift's other inputs did not change - #2686
Conversation
|
The run you asked for on #2674, of the fixture with its two commit ids set to current main and this head (Linux, Python 3.12, no game, no compiler):
So the window is closed and #2674's gain is kept. Your own two test files pass here on this head ( One case neither version nor the code before #2674 sees, for the record and not for this pull request: a file changed and changed back during one lift (a branch switched and switched back while it runs) compares equal before and after. Not tested: Windows, and a real lift (the fixture stands in for the lift and the split; everything around them is the repository's code). |
SimonKimDev
left a comment
There was a problem hiding this comment.
Small window left in the check: the full key is taken after the comparison, so a translator file edited between the second lift_key(..., planning=False) and lift_key(exefs, work, env) still stamps C from the old translator with the new key.
Taking the full key first closes it:
after = lift_key(exefs, work, env)
if lift_key(exefs, work, env, planning=False) == inputs:
key = afterAn edit before after is caught by the comparison; an edit after it doesn't matter, since after already hashed the old file.
|
@SimonKimDev right, thanks: the order left exactly that gap. 54e2652 takes the full key first and keeps it only if the @mooki000 thank you for running the fixture on both sides; that's the confirmation I couldn't get here. Noted on a file changed and changed back within one lift (a branch switched and back): no snapshot of hashes sees that. It's out of this PR, as you say. |
…nputs did not change NotRexed#2674 stores the key taken after the lift, so that the files the lift writes for the next one (image/hot_pieces.txt, image/chunks.txt) are in it. Taken whole, that key also covered a translator file edited while the lift ran, and certified C made from the old file under the new file's key: the next build skipped a lift it needed (mooki000's fixture on NotRexed#2674). translate() now takes lift_key(planning=False), everything but those two files, as the lift starts and again after it, and takes the key again only if that is unchanged. Otherwise the key from before stays, and the next build translates again, as before NotRexed#2674. tests/test_lift_cleanup_error.py adds the case of an input changed during the lift, and tests/test_lift_cache.py checks the order and what planning=False leaves out and keeps. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
The same fixture on |
The key was taken after lift_key(planning=False) was compared, so a translator file edited between the two would still stamp C from the old file with the new key (SimonKimDev on NotRexed#2686). It is now taken first and kept only when the comparison after it holds: an edit before it is caught by the comparison, and one after it is in no key that is kept. tests/test_lift_cache.py checks that order, and the cleanup test's changed-input case sees the key taken and not kept. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Upstream NotRexed/Hookshot fixed four builder problems the fork shares, on the path its Flutter builder and Rust engine use. Ported semantically, with the Rust engine's own layout and failure handling: - e7f5581: scripts/oot_saves.py finds Lichii's own data folder (~/.local/share/lichii, %APPDATA%\lichii) before Eden's, where a Lichii before build 150 still keeps its saves. `list` groups programs by the folder import and export use; the fork's `saves` list for the Flutter launcher follows the same rule, so Lichii is not offered from Eden's. - 6c529a2: a Btrfs folder has a device number no disk has, so the hard drive warning could not tell; nx2/disk.py and builder/rust/src/disk.rs now read /proc/self/mountinfo for the deepest mount and every disk of a multi-disk Btrfs (/sys/fs/btrfs/<id>/devices). - c544828 (+ NotRexed#2686, NotRexed#2675): the lift key is taken again after the lift and split_hot, from image/chunks.txt that the first lift writes, so a new work folder's second build no longer translates the game again; the new key is kept only when nothing else the lift reads changed while it ran. A compile that failed only in repository sources keeps the translation (a host typo no longer costs a lift); a lifted or generated file that failed still forgets it. The Rust driver never had a key but had no such distinction: compile failures now follow the same rule, a link failure forgets the lift, and a failed host source keeps it. - a740d67: host_mtune()/host::mtune() detect a Steam Deck APU (0405 or 0932) or RX_STEAMDECK=1 and add -mtune=znver2 beside -march, in nx2/build.py and the Rust driver's game and host compiles; RX_MTUNE names a tune. The fork's lift and host builds both get it. Tests: the upstream ones adapted (test_oot_saves, test_disk_warning, test_cpu_target, test_lift_cache, test_lift_cleanup_error), new tests/test_failed_compile_keeps_lift.py, and Rust tests for the Btrfs mount walk, the lift failure rule and the Zen 2 tune. No game files or generated game data. Assisted-by: opencode (DeepSeek V4.1 Flash 1017)
Joey-Magnus
left a comment
There was a problem hiding this comment.
SimonKimDev's ordering point is fixed in 54e2652. after = lift_key(...) is now taken before the planning=False comparison at nx2/make_game.py:2190, and test_lift_cache.py checks that order. mooki000's fixture on the new head confirms both cases: an edit made during the lift is not certified, and a build where only the planning files changed still skips the lift. I merged this into current main. tests/test_lift_cleanup_error.py passes (9 tests, including the new "input changed during the lift" case). tests/test_lift_cache.py has the same single failure here as on main (numpy isn't installed for the lift import check), and all the new checks pass. The fallback is the conservative one: if anything changed during the lift, the pre-lift key stays and the next build lifts again. That costs a few minutes at worst, never a stale build. An edit reverted during the lift (A to B to A) can't be seen by a content hash, but the C would be from A anyway, so that's fine. It merges cleanly with #2693, which also edits make_game.py.
#2674 can certify stale lifted C: if a translator file is edited while a lift runs, the key taken after the lift describes the new file, but the C came from the old one, and the next build skips a lift it needs. mooki000 found and reproduced this on #2674 just before it merged; this closes it.
Change
lift_key(..., planning=False): everything the key hashes except the two files the lift writes for the next one (image/chunks.txt,image/hot_pieces.txt).translate()takes that as the lift starts and again after it, and takes the key again only if it's unchanged. If something else changed while the lift ran, the key from before the lift stays, and the next build translates again, as before fix(build): a new work folder's second build no longer translates the game again #2674.This is mooki000's suggested shape: refresh the self-produced inputs only after confirming the rest are unchanged.
Testing
tests/test_lift_cleanup_error.py: a new case where an input changes during the lift writes the pre-lift key, and the key taken after doesn't validate. Againstmain'smake_game.pyit fails; with this change it passes. The existing cases still see lift → key taken again → key written.tests/test_lift_cache.py:nx2/lift.pyruns, and the check comes before the key is taken again.planning=Falseignores changes tohot_pieces.txtandchunks.txt, but follows the game's code and the hot list.main'smake_game.py, passes with this change.make_game.py(52), on Windows, passes apart fromtest_cpp_preflight,test_linux_updateandtest_prepare_stall, which fail the same way on a cleanmainhere.guardedvariant is the same check this PR adds; a run with--variantpointed at this branch's commit would confirm it.🤖 Generated with Claude Code