Repository navigation
perf(fpx): keep the hardware result for FMUL/FMLA S when the flags are already set - #917
Merged
Merged
Conversation
…e already set The fma32-shaped wrappers (a64_fpx_mul32_fma32, a64_fpx_fma32_fma32) fell back to the exact soft-float body whenever the fast path declined: a subnormal operand under FZ, a tiny or infinite result. FPSR's cumulative flags are sticky, so when every flag the Arm op could raise is already set, one hardware FMA (or multiply) result is the Arm result: a subnormal operand under FZ flushed to a zero of its sign with IDC set, a tiny result with UFC set, an infinite result with OFC set, an exact cancellation. a64_fpx_sticky32 keeps those, in place of a64_fpx_fz_mul32 for these two shapes, and falls back otherwise. tests/test_fpx_sticky.py checks both wrappers as the generator emits them against recomp_fpe_fma32, value and FPSR: 116M checks, 0 mismatches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…FIZ/AH/NEP, drop dead mul32 code Review of NotRexed#917: - The generated wrappers call A64_FPX_STICKY_PROBE() (empty in the real build) when the sticky rescue keeps a result; the test counts there instead of calling a64_fpx_sticky32 itself, so a broken route (fpx_header's branch, the mul argument) fails it (exit 4). - The test builds and runs twice: -march=x86-64-v3 and x86-64-v2 (RECOMP_FPX_FMA64 off: FMUL rescued by a plain multiply, FMLA must decline every case). v2 runs without the AVX2 check. - gen_state sets FPCR bits 0-2 (alone and in the random mask); the sweep adds one state per a: partial flags, IXC clear, FIZ/AH/NEP. A rescue under a closed gate fails the test (the exact body models none of FIZ/AH/NEP, so values alone cannot tell). - A64_FPX_FMA32 is gone: the helper uses RECOMP_FPX_FMA64. - Dead code removed: a64_fpx_fz_mul32, a64_fpx_fz_result's mul param, "mul32" in the fz elif, A64_FPX_FZ_OPEN; FPX_FZ_HEADER's comment updated. - The wrapper tests RECOMP_FPX_OPEN once; a64_fpx_sticky32 rejects a NaN operand, and FMUL with OFC and UFC clear and nothing flushed, before computing (same rescue counts with and without these rejects). - FPX1's contract comment in arm64_to_c.h points to the outlined wrappers' rescues and says the shadow build does not reach them. tests/test_fpx_sticky.py: 2 x (16M random + 150M sweep) checks, 0 mismatches. Each sabotage in its docstring fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s emit route tests/test_fpx_sticky.py gains a "lifted" case in both builds: four hand-encoded guest words (FMUL, FNMUL, FMADD, FMLA by element, S) go through a64emit.translate and a64_lift.op_text, the lifter's per-instruction pipeline (and hot_text, the hot functions' form), so the C calls the outlined wrapper exactly as lifted code does. Each runs against the same word translated with FPX off (the exact body inline) on 5 inputs that must be rescued and 7 that must fall back (IXC only, UFC/OFC/IDC clear, NaN, FZ|AH, a normal result): whole Ctx and the A64_FPX_STICKY_PROBE count must match. 96 runs a build, 0 mismatches; 40 rescues at x86-64-v3, 20 (FMUL/FNMUL only) at x86-64-v2. a64_lift.py: Lifter.write's per-instruction pipeline and the hot-function rewrite become op_text() and hot_text(), which the test calls (no change to what they emit). Sabotage: the emitter no longer outlining FMUL S (EmitFPMulAddValue's `FpxOutline()` made `FpxOutline() && !mul`) fails the lifted case in both builds (20 FMUL/FNMUL runs without a rescue) while the random and sweep runs, which call the wrappers by name, pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
klimovich008
force-pushed
the
perf-fp-sticky-rescue
branch
from
November 9, 2026 18:13
e5f07b2 to
0a2b38e
Compare
Contributor
Author
|
Thanks. All seven points are addressed in 1ff4abe and 0a2b38e:
Results: 2 × (16M random + 150M sweep + 96 lifted) checks, 0 mismatches. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Single-precision FMUL/FNMUL/FMLA results that the FPX1 fast path can't prove fell back to the soft-float exact body every time. These are tiny or infinite results, or a subnormal input under FZ. Code that runs with those cumulative FPSR flags already set paid that cost on each such op.
Why merge
The rescue keeps the hardware result only when it can raise no new FPSR bit. Everything else still goes to the exact body. The review found no correctness problem.
Change
a64_fpx_sticky32runs in the outlined wrappersa64_fpx_mul32_fma32anda64_fpx_fma32_fma32, inside the gate.RECOMP_FPX_FMA64; without a hardware FMA only FMUL is rescued.fz_mul32code is removed.Tested
tests/test_fpx_sticky.pybuilds at x86-64-v3 and x86-64-v2. Each build runs:There were 0 mismatches against the exact body. Rescues are counted inside the wrappers, and a rescue under a closed gate fails the test. Sabotages of the emitter route, the wrapper route, the FIZ/AH/NEP mask and the UFC/IDC checks each fail it.
Not covered
🤖 Generated with Claude Code