[RobertMrowiec]

What it does

With DM_FRAMEGEN=1, Zelda: Ocarina of Time (Switch 2, 0400486024DEC000) docked on an M1 Max shows 60 fps at a fixed 2048x1152 render size.

On the M1 Max the docked frame costs about 22-28 ms of GPU time. That barely changes with render size, so native 60 is out of reach. With frame generation, the game renders 30 real frames a second and the host shows 60 by generating a picture between each pair.

Everything is opt-in. Without DM_FRAMEGEN, and without DM_UP_SHARPEN on the MetalFX path, the host behaves exactly as main does.

I'd welcome review and testing from other M1/M2 owners, in particular on the motion estimation and on fast camera turns (see "Known").

Commits

1. nvn/upscaler: FSR1 RCAS sharpening after MetalFX's spatial scaler (DM_UP_SHARPEN), plus a log line with a histogram of the game's render size.
2. nvn/fg: frame generation (nvn_fg.inc, fg_core.h, display-link changes in nvn_mtl.m, main.c).
3. host/hid: DM_CTL burst <n> <path%d> captures n consecutive presented frames, used for testing.
4. game: a section in Ocarina of Time's note.

Results

M1 Max MacBook Pro (32-core GPU, 64 GB, on AC, macOS 27.2), docked, Kokiri Forest, the packed Mac app, warm pipeline cache.

• Played: the change on 34608c72 held 60 fps on screen in play, as reported by the player. Walking and character movement feel like 60. Fast camera turns are better than with MetalFX alone but are still not fully smooth.

• Game base rate (render side, 20 s windows): 29.7 28.8 29.8 27.7 30.0 30.0 30.0 30.0 29.2 30.0 30.0. The lower windows are loading at the start of the run. Frame generation needs a steady 30.

• Pictures shown per second, same changes on an older base (ed97f31d, feat(oot): M1 Max 60fps docked — frame generation, held-tick fix, RCAS sharpening RobertMrowiec/nxnative#1):

  Render size	Walk: mean / worst 20 s segment	Camera spin: mean / worst
  1707x960	59.9-60.0 / 59.5 (3 runs)
  1906x1072	59.9 / 58.8
  2048x1152	59.9 / 59.0 (25 segments)	60.0 / 59.8
  2276x1280	59.1 / 56.6
  2439x1372	58.1 / 52.9

• Without frame generation, the same Mac runs 25-36 fps docked.

• This branch was then rebased onto 8d44aaf8. The only conflict was in main.c, next to the new DM_PSO_PREBUILD check. The rebased host builds, but it hasn't been played, because main's new shader chain (a5e7fb6e4fa4) needs the game's shaders rebuilt.

How to use it (M1 Max, docked)

env.txt:

DM_FRAMEGEN=1
DM_FG_SCALE=0.5

The tests used a fixed render size of 2048x1152. The game's dynamic resolution doesn't settle on this Mac: its load threshold is measured against a 60 fps budget, so at a 30 fps target it sits at its floor (1067x600). The fixed size comes from a RomFS overlay that sets every docked DynamicResolutionParam level to 0.8, loaded with DM_ROMFS_OVERLAY=<folder holding Pack/>. The generator (scripts/oot_res_overlay.py <RomFS.img> <dir> 0.8 - flat) is a 48-line script on the branch of RobertMrowiec#1 and isn't part of this PR.

How it works

Frame generation (DM_FRAMEGEN=1)

• Frame rate and display link: main.c sets DM_FPS_TARGET=30 under DM_FRAMEGEN. The display link stays at 60 Hz, and the 30 Hz link and 120 Hz VRR pacing are off while frame generation is on.
• Order of pictures: on the tick a new frame arrives, the host shows a generated picture between the previous frame and the new one. On the next tick it shows the real frame. This adds one tick of latency (16.7 ms).
• Interpolation and motion: the picture comes from MetalFX MTL4FXFrameInterpolator. The game gives no motion vectors (its upscaler is spatial), so the host estimates motion itself: block matching on a quarter-resolution luma pyramid with 5 levels.
• Guard pass: where the motion residual is high, for example Link running against the background, the real frame's pixels are used instead. The thresholds are relative to local contrast (DM_FG_GUARD, DM_FG_GT0, DM_FG_GT1).
• Detail pass: at DM_FG_SCALE=0.5, interpolation costs about 2.3 ms of GPU time. The real frame's high frequencies, shifted by half the motion, are added back, which scores +4 dB against the true middle frame (DM_FG_DETAIL).
• Fast camera turns: with block-matched motion, MetalFX's picture breaks into blocks when the camera turns quickly (13-49 permille of the width per real frame in a recording), and the HUD hearts break up. Above DM_FG_MVBAIL permille of mean motion (default 12, 0 = off), the host builds the picture itself. For each pixel it picks the motion that best matches both frames (3x3 luma) out of none, the mean motion and the local estimate, then blends the two frames halfway along it. On those ticks this costs 0.85 ms more. DM_FG_WARP=0 shows the real frame instead. Blocks next to still blocks, such as the HUD, keep zero motion (DM_FG_STILL).

Held-tick fix (nvn_mtl.m, display-link callback)

• Bug: the callback returned on an empty present queue before checking for a pending held frame. When the game queued its frame just in time, every held tick was dropped and only generated pictures were shown, 30 a second. This showed up as random collapses to 9-17 real frames a second.
• Fix: the held real frame is now shown on an empty tick. DM_FG_EMPTYHOLD=0 restores the old order. The change only affects frame generation.
• Measured: before the fix, 17-segment walks at 1707x960 averaged 59.7, 56.3, 59.3 and 60.0, with collapses. After it, 3 of 3 runs averaged 59.9-60.0 with no segment under 59.5.

RCAS sharpening after MetalFX (copy.metal fsr, nvn_mtl.m up_sharpen)

• FSR1 RCAS with noise damping, run on reversibly tone-mapped HDR. It replaces the existing copy out of the MetalFX spatial scaler, so it adds no pass (DM_UP_SHARPEN 0 to 1, default 0.5).
• On main, Ocarina of Time docked keeps its own upscaler, so this has no effect on the game yet.

Not in this PR

• The fork's draw-barrier change is dropped, because e7750541 on main does the same thing better.
• The fork's CPU/GPU-overlap work (DM_NVN_METERFAKE and shadow copies) isn't ported. main had rewritten that code, and the 30 fps base holds without it.

Known / not tested

• Fast camera turns are not yet as smooth as walking. The player reports that character movement feels like 60 but the camera doesn't. I'm still working on this.
• Lighting on M1/M2 (separate from this PR): the forest haze and light shafts were missing on this M1 Max too, the same as Ocarina of Time (M1 Max): fog, light shafts and glow are no longer drawn between d8b2b64a and ced36121 #2276 / Ocarina of Time (0400486024DEC000) loses its forest haze: bounded uniform reads (26062f9e, 918e9f34 / #2127) miscompile in Apple's Metal compiler at -O1 #2425. Rewriting froxel shader 9ed48d1011f1b48b's select-form uniform reads as plain reads, compiled at -O1, brought them back at the same GPU time (26.7 ms p50). That fix was a loose metallib in the test app and isn't in this PR.
• Coverage: only Kokiri Forest and only the M1 Max were measured. Faster Macs probably don't need frame generation, and their default is unchanged.
• When the game drops below 30 real frames a second, frame generation can't hide it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DpDXMw11frENUzoe17nZN8