Skip to content

Half pairs: a constant's swizzle, .F32, and HSETP2 / HSET2 .FTZ - #1410

Closed
Copak731 wants to merge 2 commits into
seedds:mainfrom
Copak731:half-swizzle
Closed

Copak731 wants to merge 2 commits into
seedds:mainfrom
Copak731:half-swizzle

Conversation

@Copak731

@Copak731 Copak731 commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

What and why

Three half-pair defects, found by a GPU cross-check against an exact binary16 reference (one rounding, FMA included).

  • The swizzle of a constant operand was ignored. AHalfLanes took a constant as is, so HMUL2 R37, R37.H0_H0, c[0x0] [0x168].H0_H0 (a half scaled by one half of a uniform) gave the high lane the constant's other half. nvdisasm names the swizzle in all three slots (A 74..75, the 32 slot 60..61, the 64 slot 81..82); only an immediate has none. This also affected HSETP2 / HSET2 / HMNMX2 through AHalfSecond.
  • Swizzle value 1 is .F32, the operand's own 32-bit float (nvdisasm), not .H1_H1. Its exact rounding on NVIDIA is not documented; the float itself is the closer reading. MUFU.F16, whose bits 60..61 pick one half, keeps its reading (MufuF16 caught an early version of this change).
  • HSETP2 / HSET2 .FTZ (bit 80) was dropped. A subnormal half must compare as zero, as AFtz already does for FSETP / FSET; but the lanes are unpacked to 32 bits, where a half's subnormals are normal floats, so AHalfFtz flushes under the half's own bound (2^-14). .H_AND keeps the reading NAK's OpHSetP2.horizontal documents.

Reference entries: docs/reference/shaders.md, "Half-pair swizzle on a constant, and .F32" and "HSETP2 / HSET2 .FTZ".

Checked

  • Tests: DMSH_TEST_XLATE=<translator> python3 dm/gfx/tests/test_dmsh_ampere_alu.py, new HalfSwizzle (3) and HalfCompareFtz (3); the five checks of the fix fail on main. All test_dmsh_*.py pass.
  • GPU cross-check (The cross-checks translate real sm_86 encodings (an nvdisasm 13.5-validated corpus) with dmsh_xlate using the production options (debug and reduced on, as dmsh_build.py), convert with spirv-cross 1.4.363.0 at MSL 2.3, run them on the GPU and compare every bit with a reference written from the oracle, PTX's documentation and Mesa NAK where it applies.): HADD2 / HMUL2 / HFMA2 / HSETP2 / HSET2 / HMNMX2, 1,200 instructions: 89.7 % identical on main, 96.8 % with the change (86 constant-swizzle mismatches in HADD2 / HMUL2 / HFMA2 and 15 in HSETP2 to none; 4 FTZ compares in 400 subnormal inputs to none). Left: arithmetic .FTZ on subnormals (dropped by design), .RELU on NaN, and NaN payloads.
  • Devices and OS: MacBook, Apple M5, macOS 26.6.2, Xcode 27.2.
  • Games run: none (no game files were used).

Docs

  • The finding is in its docs/reference/shaders.md entry, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
  • No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

Copak731 and others added 2 commits November 5, 2026 22:37
AHalfLanes took a constant operand as is, ignoring its swizzle (bits 60..61
of the 32 slot), so HMUL2 R37, R37.H0_H0, c[0x0] [0x168].H0_H0 gave the high
lane the constant's other half. Value 1 was read as .H1_H1; nvdisasm names it
.F32 in all three slots, the operand's own 32-bit float. MUFU.F16, whose bits
60..61 pick one half, keeps its reading.

New HalfSwizzle tests in test_dmsh_ampere_alu.py (3 fail on main). A GPU
cross-check of 900 HADD2 / HMUL2 / HFMA2 against an exact binary16 reference
went from 86 constant-swizzle mismatches to none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bit 80 of HSETP2 and HSET2 was dropped. AFtz (FSETP / FSET) does not fit the
half lanes: unpacked to 32-bit floats, a half's subnormals are normal, so
FlushDenormal's bound never flushes them. AHalfFtz flushes each lane under the
half's own bound (2^-14) before the comparison.

New HalfCompareFtz tests in test_dmsh_ampere_alu.py (two fail on main). A GPU
cross-check with subnormal and zero inputs: 4 mismatches in 400, none with
the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation labels Nov 5, 2026
fpedrolucas95 pushed a commit to fpedrolucas95/nxnative that referenced this pull request Nov 6, 2026
fpedrolucas95 pushed a commit to fpedrolucas95/nxnative that referenced this pull request Nov 6, 2026
…, seedds#1410, seedds#1411, seedds#1412, seedds#1413, seedds#1414, seedds#1329 on top of main's translator fixes): chain c9cf9613657e

Publishes Ryujinx.Graphics.Shader, Ryujinx.Common, Spv.Generator and dmsh_xlate (osx-arm64, framework-dependent, the universal launcher kept) from the merged source; the
PRs' own binaries were built without this tree's translator changes (float atomic adds, F2IP, the growing raw pool table). The lock and dmsh_chain.h are written again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
javierdv7 pushed a commit to javierdv7/nxnative that referenced this pull request Nov 6, 2026
…#1462's CAS, seedds#1487, seedds#1329, seedds#1410, seedds#1411, seedds#1412, seedds#1357): lock and chain header; conflict leftovers removed from shaders.md and test_dmsh_ampere_alu.py
seedds added a commit that referenced this pull request Nov 7, 2026
…TZ as Ampere reads them (chain 9776347fc0a7, PR #1410)

Three half-pair defects found by a GPU cross-check against an exact binary16 reference: AHalfLanes took a constant operand as it is, so HMUL2 R37, R37.H0_H0, c[0x0][0x168].H0_H0 gave the high lane the constant's other half (nvdisasm names the swizzle in all three slots, A 74..75, the 32-bit slot 60..61, the 64-bit slot 81..82; only an immediate has none), which reached HSETP2 / HSET2 / HMNMX2 through AHalfSecond too; swizzle value 1 is .F32, the operand's own 32-bit float, not .H1_H1 (MUFU.F16's half select keeps its reading); and HSETP2 / HSET2 .FTZ (bit 80) was dropped: a subnormal half compares as zero, flushed under the half's own bound (2^-14) since the lanes are unpacked to 32 bits.

Checked: test_dmsh_ampere_alu HalfSwizzle and HalfCompareFtz (5 of their checks fail on the translator before), the 21 translator test files (test_dmsh_pairs' two NameErrors as before), test_nsp_chain, test_xlate_anycpu, test_docs. No game is named by the author; half math on uniforms is common in Switch 2 shaders. Regression: one Ocarina of Time run on the chain that ends this batch.

Co-authored-by: Copak731 <224629733+Copak731@users.noreply.github.com>
@seedds

seedds commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Landed as 9dc8195 with you as co-author (HalfSwizzle and HalfCompareFtz fail before, pass after). Thank you.

@seedds seedds closed this Nov 7, 2026
Aznracer pushed a commit to Aznracer/Mighty-Green-Kokiri-Kid that referenced this pull request Nov 8, 2026
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…TZ as Ampere reads them (chain 9776347fc0a7, PR seedds#1410)

Three half-pair defects found by a GPU cross-check against an exact binary16 reference: AHalfLanes took a constant operand as it is, so HMUL2 R37, R37.H0_H0, c[0x0][0x168].H0_H0 gave the high lane the constant's other half (nvdisasm names the swizzle in all three slots, A 74..75, the 32-bit slot 60..61, the 64-bit slot 81..82; only an immediate has none), which reached HSETP2 / HSET2 / HMNMX2 through AHalfSecond too; swizzle value 1 is .F32, the operand's own 32-bit float, not .H1_H1 (MUFU.F16's half select keeps its reading); and HSETP2 / HSET2 .FTZ (bit 80) was dropped: a subnormal half compares as zero, flushed under the half's own bound (2^-14) since the lanes are unpacked to 32 bits.

Checked: test_dmsh_ampere_alu HalfSwizzle and HalfCompareFtz (5 of their checks fail on the translator before), the 21 translator test files (test_dmsh_pairs' two NameErrors as before), test_nsp_chain, test_xlate_anycpu, test_docs. No game is named by the author; half math on uniforms is common in Switch 2 shaders. Regression: one Ocarina of Time run on the chain that ends this batch.

Co-authored-by: Copak731 <224629733+Copak731@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants