Skip to content

Introduce new video subsystem with multiple backends

David Milosevic qemu-devel 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest3 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3video: introduce video subsystem with inital v4l2 backendhw/core/qdev-properties-system.c hw/usb/Kconfig hw/usb/dev-video.c +14 more+3187 -10 replies
2/3video: add GStreamer backendmeson.build meson_options.txt qemu-options.hx +4 more+712 -00 replies
3/3video: add libcamera backend extensionvideo/libcamera.c video/meson.build+149 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/3] Introduce new video subsystem with multiple backends

As previously mentioned in our RFC, we at 9elements were working on a new video subsystem for QEMU.

This subsystem includes a USB Video Class (UVC) frontend that exposes a virtual camera device to the guest OS, backed by host-side video sources like v4l2, GStreamer and libcamera.

The patch series is split into three parts

- the first patch introduces the video subsystem with its first backend (v4l2) and implements the UVC device logic
- the second patch adds the GStreamer backend
- the third and final patch implements libcamera support

Currently we are able to:

- stream frames (currently YUYV only)
- dynamically enumerate supported camera modes (pixelformat, ...)
- discover and access camera controls (brightness, ...)

The patch series was tested on two different host devices and in combination with several guests

1. Fedora39 as host with ubuntu-24.04, Fedora41 and Fedora42 as guests
2. Fedora42 as host with Fedora42 as guest

Video streaming was successfully tested with, for example guvcview, v4l2-ctl, gst-launch-1.0 (v4l2src, pipewiresrc), cheese, snapshot, vlc, and Google Meet (from within Firefox) using the internal camera and several external USB cameras.

v4l2-ctl and qv4l2 were used to test camera controls.

Known issues:

We currently do not always achieve the target fps. Higher frame resolutions tend to reduce the fps observed in the guest. The following values were recorded using the v4l2 backend and guvcview

internal webcam (HP EliteBook):
640x360@30fps: 17.52fps
320x180@30fps: 30.00fps
320x240@30fps: 30.00fps
424x240@30fps: 30.00fps
640x480@30fps: 13.14fps

jiga webcam
1920x1080@5fps: 2.01fps
800x600@20fps: 8.44fps

I attempted to improve streaming performance by implementing a producer-consumer model. The idea was for the producer
to fetch frames from the backend and enqueue them, while the consumer (USBDeviceClass.handle_data) would dequeue and process them. However, I'm not very familiar with QEMU's threading model, and I likely didn’t implement it correctly - the producer and consumer are probably still running in the same event loop

The experimental async version can be found here:
https://github.com/9elements/qemu/tree/nlnet/qemu-video/dev-async

Another issue is that v4l2-ctl fails to stream certain video resolutions, even though other tools handle the same resolutions without problems. For some resolutions, v4l2-ctl only receives frames marked with the V4L2_BUF_FLAG_ERROR flag. This behavior requires further investigation.

Any help is very welcome!

RFC: https://lists.gnu.org/archive/html/qemu-devel/2025-03/msg02804.html
nlnet: https://nlnet.nl/project/Qemu-Camera/

Best regards,
David Milosevic

David Milosevic (3):
video: introduce video subsystem with inital v4l2 backend
video: add GStreamer backend
video: add libcamera backend extension

 hw/core/qdev-properties-system.c    |   52 ++
 hw/usb/Kconfig                      |    5 +
 hw/usb/dev-video.c                  | 1333 +++++++++++++++++++++++++++
 hw/usb/meson.build                  |    1 +
 hw/usb/trace-events                 |   10 +
 include/hw/qdev-properties-system.h |    4 +
 include/hw/usb.h                    |    2 +
 include/hw/usb/video.h              |  322 +++++++
 meson.build                         |   31 +
 meson_options.txt                   |    5 +
 qemu-options.hx                     |    9 +
 scripts/meson-buildoptions.sh       |    6 +
 system/vl.c                         |   39 +-
 video/gstreamer-common.h            |   49 +
 video/gstreamer.c                   |  642 +++++++++++++
 video/libcamera.c                   |  148 +++
 video/meson.build                   |   24 +
 video/v4l2.c                        |  619 +++++++++++++
 video/video.c                       |  450 +++++++++
 video/video.h                       |  298 ++++++
 20 files changed, 4048 insertions(+), 1 deletion(-)
 create mode 100644 hw/usb/dev-video.c
 create mode 100644 include/hw/usb/video.h
 create mode 100644 video/gstreamer-common.h
 create mode 100644 video/gstreamer.c
 create mode 100644 video/libcamera.c
 create mode 100644 video/meson.build
 create mode 100644 video/v4l2.c
 create mode 100644 video/video.c
 create mode 100644 video/video.h
Signature · 1 line
2.47.0