[MetaIX]

What and why

Symptom: on a Mac without hardware AV1 (before M3), dav1d decodes a heavy 60 fps movie too slowly for the game to show it at 60. Ocarina of Time's 1080p60 title movie (0400486024DEC000 v131072, profile in #1299) took 9–20 ms a packet on average in-game on an M1 Max, with single packets up to 134 ms, against the 16.7 ms a frame allows. For reference, ffmpeg's libdav1d needs 9.3 ms of CPU a frame for that movie on one thread.

Cause:

• dav1d ran one frame at a time (max_frame_delay 1, 2 threads). More threads on one frame barely help (6 measured the same as 2): dav1d scales by decoding frames in parallel.
• The decode ran under the decoder's lock, so the game's Release… and Acquire… calls waited behind it.

Change:

• dav1d gets 8 threads and 8 frames in flight.
• A picture then comes some packets after its own, sometimes two together. mav1_decode_ts passes the packet's timestamp through dav1d (Dav1dData.m) to its picture (a new ts field in mav1_picture). The wrapper queues pictures; mav1_next gives the others and mav1_drain gives those still decoding at the end of the stream.
• dav1d is asked for a picture once per input, because a second ask with no new data drains (waits for the frames in flight).
• cv_dec_submit decodes AV1 under a decode lock of its own (lock order: decode lock, then the decoder lock) and takes the decoder lock only to hold the pictures. Reset and Finalize take the decode lock first.
• A packet may add up to CV_BURST (8) pictures past CV_HOLD; packets are still refused at CV_HOLD.
• VideoToolbox, H.264, VP8 and VP9 are unchanged.

Checked

• Tests: make -C dm/host/tests GAME=0100000000010000 run passes (Ocarina's profile is not on main; this test does not depend on the game). t_codec_video changes in two ways:
  • a lone key frame's picture now comes at the end of the stream (mav1_drain);
  • new: 20 packets through dav1d give 20 pictures, in order, each with its packet's timestamp.
• Games run: Ocarina of Time v131072, the title movie and title screen, three movies in a row with their ends and restarts, on our stack (main 95f9a6a + [WIP] The Legend of Zelda: Ocarina of Time (Switch 2, 0400486024DEC000, v0, SDK 22.2.8) runs on the Mac through the title movie into Kokiri Forest #1299 + Ampere F2IP and a task shader's own payload read: Ocarina of Time's untranslated shaders translate, all but one that is data #1300 + Zelda: Ocarina of Time — Restore missing mesh and texture effects on Mac #1312 with this change).
  • Decode time: at most 26.6 ms a packet over 300-packet windows (134 ms before).
  • The game's frame times are unchanged (p95 16.8 ms at 60 fps).
• Not run: Fire Emblem: Fortune's Weave (AV1 2560×1440, also dav1d on this Mac). Its app here was built with another engine, and I did not rebuild it on this branch.
• Devices and OS: MacBook Pro, Apple M1 Max, 64 GB, macOS 26.6.2, Xcode 27.2.

Docs

• No game files, keys or data extracted from a game in the change or the attachments

A follow-up PR, stacked on this one, hands pictures out by the movie's clock when the game shows fewer a second than the movie has (Ocarina's 60 fps movies at 28 fps ran at 0.44×).

🤖 Generated with Claude Code

[seedds]

Thank you for this. You found the cause first: dav1d scales by frames in parallel, not by threads on one frame, and you carried each packet's time through Dav1dData.m. Main now has the same dav1d change in 09a91c5 (from #2240, which names your finding in its commit message), so I'm closing this one as superseded.

The difference is where the decode runs. #2240 gives each decoder a thread of its own, since the SDK's Submit…InputBuffer only queues the packet (on the console the decoder runs in another process). Here the decode still ran on the game's thread, under a lock of its own, so Ocarina's reader still waited up to ~27 ms a packet. #2240 also drains dav1d whenever no packet waits behind one (so a game that waits for each picture still gets it), runs every drain to its end (dav1d 1.5.4 deadlocks when new data interrupts one), and waits for room instead of growing the queue past CV_HOLD.

#1379 is built on this PR's interface and on decoding in the game's thread. If you'd like it considered, please rebase it onto main: the late-picture skip would sit where the decoder thread hands pictures back. It will get its own design review then.