Skip to contentSkip to conversation

libva-v4l2-request: RK3588 VDPU381 VP9 decode (fork)

ANNOUNCE Francesco Saverio Pavone 2 participants 2 messages May 20 Media (V4L/DVB)

lore
Share & tools
lore
#0 lore raw

Hi Nicolas,

Thanks for the candid feedback — and for the heads-up that the direction I picked is the dead end rather than the destination. That saves me a lot of investment in a path you and the V4L2 community already see as deprecated.

Two pointers would help me catch up:

1. The recent Chromium VP9 probabilities work for the V4L2 stateless path — could you share a commit, a Phabricator/Gerrit link, or a tracker ID? I would like to read it, test it on RK3588 RKVDEC2 (VDPU381 / VDPU346), and see if it covers the same compressed header / interp_filter corner that the libva-v4l2-request fork was fixing. If there is something useful I can contribute back — test coverage, bug reports, follow-up patches — I would rather put the effort there than on the libva side.

2. Vulkan Video status. Is there a public roadmap or a working branch for the V4L2-backed Vulkan Video decoder you are pushing? I am especially interested in how request_fd / per-frame controls map into the Vulkan Video API, and whether ARM SoCs with stateless decoders are part of the early target set. If there is a place to follow the work or contribute, please point me to it.

I will keep the libva-v4l2-request fork running locally for the ecosystem that still depends on it (mpv --hwdec=vaapi-copy, ffmpeg CLI), but I will stop pushing it toward upstream now that the direction is clear. The kernel-side fix that got VP9 working on RKVDEC2 (the PM runtime teardown cleanup, lore link in the original announce) is independent and stays useful for any of the userspace paths above.

Thanks again,
Francesco

#1 lore raw

Le mercredi 20 mai 2026 à 16:40 +0200, Francesco Saverio Pavone a écrit :

Quoted 9 lines

Hi Nicolas,

Thanks for the candid feedback — and for the heads-up that the direction I picked is the dead end rather than the destination. That saves me a lot of investment in a path you and the V4L2 community already see as deprecated.

Two pointers would help me catch up:

Quoted 8 lines

1. The recent Chromium VP9 probabilities work for the V4L2 stateless path — could you share a commit, a Phabricator/Gerrit link, or a tracker ID? I would like to read it, test it on RK3588 RKVDEC2 (VDPU381 / VDPU346), and see if it covers the same compressed header / interp_filter corner that the libva-v4l2-request fork was fixing. If there is something useful I can contribute back — test coverage, bug reports, follow-up patches — I would rather put the effort there than on the libva side.

Sure, the changes is (merged):

https://chromium-review.googlesource.com/c/chromium/src/+/7794420

The story behind it is that the code was owned by ChromeOS team (and assumed ChromeOS specific), and they removed compressed header parsing from the common code when the last RK Chromebook went end-of-life. That broke non-chromebook users of RK and VSI decoders. This remains quite niche set of users, so it took a bit of time, but its fixed by moving the relevant code into the V4L2 stateless decoder components. Its also only run when the HW actually needs it now.

As I'me starting to get a glimpse of the latest generation of CPU accessed VP9 decoders, I already know that some of the decoder needs to be passed the offsets to every tiles ahead of time. Information that is aliased inside the compressed video data. I don't know yet if its the case for the newer RKVDEC VP9 decoders, previous generation was decoding tiles in order, one-by-one. There is V4L2 controls spec for that coming soon.

This is of course all supported in GStreamer and LibreELEC fork of FFMPEG. GStreamer support being fully upstream, and FFMPEG upstreaming happening here:

https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20847

Quoted 7 lines

2. Vulkan Video status. Is there a public roadmap or a working branch for the V4L2-backed Vulkan Video decoder you are pushing? I am especially interested in how request_fd / per-frame controls map into the Vulkan Video API, and whether ARM SoCs with stateless decoders are part of the early target set. If there is a place to follow the work or contribute, please point me to it.

This is of course a much younger subject, whith less implementation. So far a quick vibe coded prototype has been made to verify if using V4L2 Stateless API under the hood of a Vulkan Video driver was even possible. I will have to ask the "author" if he's interested in sharing, though its of low value in term of code. The VP9 probabilities, along with few bit skip kind of parameters where found missing, the AI open coded parser to workaround it.

I really want to spec out these extensions this year, so we can have a non- parsing Vulkan Video layer as a demonstration instead. Parsing is achievable, and the PoC demonstrated it, but doing it correctly requires running a dependency manager (it is fence base execution) in userspace in case there would be a fence on the bitstream data you need to parse. Quite some work for something hypothetical, and in the Linux Vulkan echosystem, such as Mesa + DRM, the dependency management has been chosen to happen in the kernel only for performance reason, and parsing in kernel is a no-go for security reason.

As of where to discuss. Its not all very central and quite per project. On the Mesa side, public discussion have started here:

https://gitlab.freedesktop.org/mesa/mesa/-/work_items/14987

This is about a concurrent PoC, that make use of non-V4L2 drivers. I will discuss this subject with the Linux Media group in Nice next week. Understand that these are discussions, no one on either Linux Media or DRM subsystem have committed to this change of direction. The challenge being that on one end, we have a subsystem expert in low latency dynamic HW rendering pipeline running behind standard APIs, but not really having much clue on the handling of smooth playback for variable processing time decoders (about buffering), and on the other end, its us, Linux Media subsystem, entirely focus on smoothness really, and making their own standards and spec, but not particularly focus on latency, which happen to be good most of the time though.

Quoted 8 lines

I will keep the libva-v4l2-request fork running locally for the ecosystem that still depends on it (mpv --hwdec=vaapi-copy, ffmpeg CLI), but I will stop pushing it toward upstream now that the direction is clear. The kernel-side fix that got VP9 working on RKVDEC2 (the PM runtime teardown cleanup, lore link in the original announce) is independent and stays useful for any of the userspace paths above.

Thanks for the PM runtime fixes, its in my queue, you should get feedback soon. You should really give a look at LibreELEC FFMPEG work, and Jonas upstreaming effort. V4L2 Stateless (also called request) decoder is the active standard on Linux/ARM at the moment and is meant to be used directly without an abstraction. So if you can adopt support for that, you will thin your software layers quite a bit. For me, Vulkan Video will slowly take over, specially since they have got ahead of us on the encoder side. Its also a nice commodity when the exact same software can run on your PC and your embedded system. Though, there is still a lot of work to do, and there is a lot of firmware accessed decoder/encoder that are incompatible by firmware designs, which remain only covered by V4L2 Stateful driver layer until their vendor decide to offer new type of firmwares.

Nicolas

Quoted 3 lines

Thanks again,
Francesco