Skip to contentSkip to conversation

Initial Rust V4L2 support

PATCH In review Daniel Almeida 15 participants 53 messages Apr 6, 2023 – Jul 5, 2023 3 messages not in the archive

Subsystems (3)
lore
Share & tools
lore

Daniel Almeida

author reviewer
#0 lore raw

cover Initial Rust V4L2 support

Hi all, this is my first attempt at adding Rust support to the media subsystem.

It adds just enough support to write a clone of the virtio-camera prototype written by my colleague, Dmitry Osipenko, available at [0].

Basically, there's support for video_device_register, v4l2_device_register and for some ioctls in v4l2_ioctl_ops. There is also some initial vb2 support, alongside some wrappers for some types found in videodev2.h.

I wrote a sample Rust driver just to prove that this probes, and that you get a message on dmesg whenever an ioctl is called.

As there is no actual implementation for any of the ioctls, this module can misbehave with some programs. I can work around this in a future submission.

Note that this is based on the rust branch, as opposed to rust-next. The reasoning is simple: I expect this series to just kickstart some discussion around the subject. Actual upstreaming can come up much later, at which point I can rebase on the rust branch.

Lastly, the origins of this series trace back to a v4l2 virtIO driver I was writing in Rust. As the project was eventually shelved for other reasons, I picked both the virtIO and the v4l2 bindings into their own patches which I am now in the process of submitting. This is to say that I tested this code with said driver and CrosVM in the past, and it worked ok.

Please let me know your thoughts.

[0]: https://gitlab.collabora.com/dmitry.osipenko/linux-kernel-rd/-/commit/055a2c322e931a8b388f864f1db81bbdfd525602

Daniel Almeida (6):
rust: media: add the media module
rust: media: add initial videodev2.h abstractions
rust: sync: introduce FfiMutex
rust: media: videobuf2: add a videobuf2 abstraction
rust: media: add {video|v4l2}_device_register support
rust: media: add v4l2 rust sample

 rust/bindings/bindings_helper.h        |   8 +
 rust/kernel/lib.rs                     |   2 +
 rust/kernel/media/mod.rs               |   6 +
 rust/kernel/media/v4l2/capabilities.rs |  80 ++++
 rust/kernel/media/v4l2/dev.rs          | 369 +++++++++++++++
 rust/kernel/media/v4l2/device.rs       | 115 +++++
 rust/kernel/media/v4l2/enums.rs        | 135 ++++++
 rust/kernel/media/v4l2/format.rs       | 178 ++++++++
 rust/kernel/media/v4l2/framesize.rs    | 176 +++++++
 rust/kernel/media/v4l2/inputs.rs       | 104 +++++
 rust/kernel/media/v4l2/ioctls.rs       | 608 +++++++++++++++++++++++++
 rust/kernel/media/v4l2/mmap.rs         |  81 ++++
 rust/kernel/media/v4l2/mod.rs          |  13 +
 rust/kernel/media/videobuf2/core.rs    | 552 ++++++++++++++++++++++
 rust/kernel/media/videobuf2/mod.rs     |   5 +
 rust/kernel/sync.rs                    |   1 +
 rust/kernel/sync/ffi_mutex.rs          |  70 +++
 samples/rust/Kconfig                   |  11 +
 samples/rust/Makefile                  |   1 +
 samples/rust/rust_v4l2.rs              | 403 ++++++++++++++++
 20 files changed, 2918 insertions(+)
 create mode 100644 rust/kernel/media/mod.rs
 create mode 100644 rust/kernel/media/v4l2/capabilities.rs
 create mode 100644 rust/kernel/media/v4l2/dev.rs
 create mode 100644 rust/kernel/media/v4l2/device.rs
 create mode 100644 rust/kernel/media/v4l2/enums.rs
 create mode 100644 rust/kernel/media/v4l2/format.rs
 create mode 100644 rust/kernel/media/v4l2/framesize.rs
 create mode 100644 rust/kernel/media/v4l2/inputs.rs
 create mode 100644 rust/kernel/media/v4l2/ioctls.rs
 create mode 100644 rust/kernel/media/v4l2/mmap.rs
 create mode 100644 rust/kernel/media/v4l2/mod.rs
 create mode 100644 rust/kernel/media/videobuf2/core.rs
 create mode 100644 rust/kernel/media/videobuf2/mod.rs
 create mode 100644 rust/kernel/sync/ffi_mutex.rs
 create mode 100644 samples/rust/rust_v4l2.rs
Signature · 1 line
2.40.0
author reviewer
#1 lore raw

PATCH 1/6 rust: media: add the media module

As part of the initial rust v4l2 support, add the media module to the kernel crate.

---
 rust/kernel/lib.rs            | 2 ++
 rust/kernel/media/mod.rs      | 5 +++++
 rust/kernel/media/v4l2/mod.rs | 3 +++
 3 files changed, 10 insertions(+)
 create mode 100644 rust/kernel/media/mod.rs
 create mode 100644 rust/kernel/media/v4l2/mod.rs
rust/kernel/lib.rs +2 -0
index c20b37e88ab2..d7e8f3297405 100644
@@ -48,6 +48,8 @@ pub mod types;
 #[doc(hidden)]
 pub use bindings;
 pub use macros;
+#[cfg(CONFIG_MEDIA_SUPPORT)]
+pub mod media;
 
 #[cfg(CONFIG_ARM_AMBA)]
 pub mod amba;
rust/kernel/media/mod.rs new file +5 -0
new file mode 100644
index 000000000000..342e66382719
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Media subsystem
+
+pub mod v4l2;
rust/kernel/media/v4l2/mod.rs new file +3 -0
new file mode 100644
index 000000000000..068dd9b4863d
@@ -0,0 +1,3 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+
+//! Abstractions for include/media/v4l2-*.h
Signature · 1 line
2.40.0
author reviewer
#2 lore raw

PATCH 2/6 rust: media: add initial videodev2.h abstractions

In reply to Daniel Almeida

Add initial videodev2.h abstractions.

---
 rust/bindings/bindings_helper.h        |   1 +
 rust/kernel/media/v4l2/capabilities.rs |  80 +++++++++++
 rust/kernel/media/v4l2/enums.rs        | 135 +++++++++++++++++++
 rust/kernel/media/v4l2/format.rs       | 178 +++++++++++++++++++++++++
 rust/kernel/media/v4l2/framesize.rs    | 176 ++++++++++++++++++++++++
 rust/kernel/media/v4l2/inputs.rs       | 104 +++++++++++++++
 rust/kernel/media/v4l2/mmap.rs         |  81 +++++++++++
 rust/kernel/media/v4l2/mod.rs          |   7 +
 8 files changed, 762 insertions(+)
 create mode 100644 rust/kernel/media/v4l2/capabilities.rs
 create mode 100644 rust/kernel/media/v4l2/enums.rs
 create mode 100644 rust/kernel/media/v4l2/format.rs
 create mode 100644 rust/kernel/media/v4l2/framesize.rs
 create mode 100644 rust/kernel/media/v4l2/inputs.rs
 create mode 100644 rust/kernel/media/v4l2/mmap.rs
rust/bindings/bindings_helper.h +1 -0
index 048bae2197ac..3b3d6fcf110f 100644
@@ -35,6 +35,7 @@
 #include <linux/sysctl.h>
 #include <linux/uaccess.h>
 #include <linux/uio.h>
+#include <linux/videodev2.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
rust/kernel/media/v4l2/capabilities.rs new file +80 -0

80 lines not on this page · raw message

rust/kernel/media/v4l2/enums.rs new file +135 -0

135 lines not on this page · raw message

rust/kernel/media/v4l2/format.rs new file +178 -0

178 lines not on this page · raw message

rust/kernel/media/v4l2/framesize.rs new file +176 -0

176 lines not on this page · raw message

rust/kernel/media/v4l2/inputs.rs new file +104 -0

104 lines not on this page · raw message

rust/kernel/media/v4l2/mmap.rs new file +81 -0

81 lines not on this page · raw message

rust/kernel/media/v4l2/mod.rs +7 -0
index 068dd9b4863d..77864640f19e 100644
@@ -1,3 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0 OR MIT
 
 //! Abstractions for include/media/v4l2-*.h
+
+pub mod capabilities;
+pub mod enums;
+pub mod format;
+pub mod framesize;
+pub mod inputs;
+pub mod mmap;
Signature · 1 line
2.40.0
author reviewer
#3 lore raw

PATCH 3/6 rust: sync: introduce FfiMutex

In reply to Daniel Almeida

Introduce an owned mutex that can be passed as bindings::mutex when initializing C data structures. The kernel will be locking it without user intervention so there is no data to protect and no guard type.

---
 rust/kernel/sync.rs           |  1 +
 rust/kernel/sync/ffi_mutex.rs | 70 +++++++++++++++++++++++++++++++++++
 2 files changed, 71 insertions(+)
 create mode 100644 rust/kernel/sync/ffi_mutex.rs
rust/kernel/sync.rs +1 -0
index b07db83972ac..1415062be34d 100644
@@ -26,6 +26,7 @@ use core::{cell::UnsafeCell, mem::MaybeUninit, pin::Pin};
 
 mod arc;
 mod condvar;
+pub mod ffi_mutex;
 mod guard;
 mod locked_by;
 mod mutex;
rust/kernel/sync/ffi_mutex.rs new file +70 -0

70 lines not on this page · raw message

Signature · 1 line
2.40.0
author reviewer
#4 lore raw

PATCH 4/6 rust: media: videobuf2: add a videobuf2 abstraction

In reply to Daniel Almeida

Add a videobuf2 abstraction. Notably, only vb2_v4l2_buffer is supported as the subsystem-specific struct.

---
 rust/bindings/bindings_helper.h     |   3 +
 rust/kernel/media/mod.rs            |   1 +
 rust/kernel/media/videobuf2/core.rs | 552 ++++++++++++++++++++++++++++
 rust/kernel/media/videobuf2/mod.rs  |   5 +
 4 files changed, 561 insertions(+)
 create mode 100644 rust/kernel/media/videobuf2/core.rs
 create mode 100644 rust/kernel/media/videobuf2/mod.rs
rust/bindings/bindings_helper.h +3 -0
index 3b3d6fcf110f..3153894f426b 100644
@@ -36,6 +36,9 @@
 #include <linux/uaccess.h>
 #include <linux/uio.h>
 #include <linux/videodev2.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/videobuf2-dma-sg.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
rust/kernel/media/mod.rs +1 -0
index 342e66382719..95b11544af8d 100644
@@ -3,3 +3,4 @@
 //! Media subsystem
 
 pub mod v4l2;
+pub mod videobuf2;
rust/kernel/media/videobuf2/core.rs new file +552 -0

552 lines not on this page · raw message

rust/kernel/media/videobuf2/mod.rs new file +5 -0
new file mode 100644
index 000000000000..6b0be2c7a14d
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Videobuf2 Framework
+
+pub mod core;
Signature · 1 line
2.40.0
author reviewer
#5 lore raw

PATCH 5/6 rust: media: add {video|v4l2}_device_register support

In reply to Daniel Almeida

Add {video|v4l2}_device_register support, essentially introducing (initial) support for v4l2 drivers in Rust.

---
 rust/bindings/bindings_helper.h  |   4 +
 rust/kernel/media/v4l2/dev.rs    | 369 +++++++++++++++++++
 rust/kernel/media/v4l2/device.rs | 115 ++++++
 rust/kernel/media/v4l2/ioctls.rs | 608 +++++++++++++++++++++++++++++++
 rust/kernel/media/v4l2/mod.rs    |   3 +
 5 files changed, 1099 insertions(+)
 create mode 100644 rust/kernel/media/v4l2/dev.rs
 create mode 100644 rust/kernel/media/v4l2/device.rs
 create mode 100644 rust/kernel/media/v4l2/ioctls.rs
rust/bindings/bindings_helper.h +4 -0
index 3153894f426b..bf2e833e511c 100644
@@ -36,6 +36,10 @@
 #include <linux/uaccess.h>
 #include <linux/uio.h>
 #include <linux/videodev2.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-fh.h>
+#include <media/v4l2-ioctl.h>
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-v4l2.h>
 #include <media/videobuf2-dma-sg.h>
rust/kernel/media/v4l2/dev.rs new file +369 -0

369 lines not on this page · raw message

rust/kernel/media/v4l2/device.rs new file +115 -0

115 lines not on this page · raw message

rust/kernel/media/v4l2/ioctls.rs new file +608 -0

608 lines not on this page · raw message

rust/kernel/media/v4l2/mod.rs +3 -0
index 77864640f19e..c60083266202 100644
@@ -3,8 +3,11 @@
 //! Abstractions for include/media/v4l2-*.h
 
 pub mod capabilities;
+pub mod dev;
+pub mod device;
 pub mod enums;
 pub mod format;
 pub mod framesize;
 pub mod inputs;
+pub mod ioctls;
 pub mod mmap;
Signature · 1 line
2.40.0
author reviewer
#6 lore raw

PATCH 6/6 rust: media: add v4l2 rust sample

In reply to Daniel Almeida

Add a v4l2 rust sample. The sample driver showcases the current support available in the media module. It also proves that the device will indeed probe by printing a message to the terminal indicating that no error took place.

---
 samples/rust/Kconfig      |  11 ++
 samples/rust/Makefile     |   1 +
 samples/rust/rust_v4l2.rs | 403 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 415 insertions(+)
 create mode 100644 samples/rust/rust_v4l2.rs
samples/rust/Kconfig +11 -0
index 189c10ced6d4..4bdea8210ae0 100644
@@ -163,4 +163,15 @@ config SAMPLE_RUST_SELFTESTS
 
 	  If unsure, say N.
 
+config SAMPLE_RUST_V4L2
+	tristate "V4L2 driver"
+	depends on V4L_PLATFORM_DRIVERS
+	depends on VIDEO_DEV
+	select VIDEOBUF2_DMA_SG
+
+	help
+	  This option builds the V4L2 example.
+
+	  If unsure, say N.
+
 endif # SAMPLES_RUST
samples/rust/Makefile +1 -0
index 420bcefeb082..5ffa621f3968 100644
@@ -15,5 +15,6 @@ obj-$(CONFIG_SAMPLE_RUST_NETFILTER)		+= rust_netfilter.o
 obj-$(CONFIG_SAMPLE_RUST_ECHO_SERVER)		+= rust_echo_server.o
 obj-$(CONFIG_SAMPLE_RUST_FS)			+= rust_fs.o
 obj-$(CONFIG_SAMPLE_RUST_SELFTESTS)		+= rust_selftests.o
+obj-$(CONFIG_SAMPLE_RUST_V4L2)		+= rust_v4l2.o
 
 subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS)		+= hostprogs
samples/rust/rust_v4l2.rs new file +403 -0

403 lines not on this page · raw message

Signature · 1 line
2.40.0
author reviewer
#7 lore raw
In reply to Daniel Almeida

By the way, one of the things I dislike about this series is that there's a needless distinction between

struct Foo(bindgen::foo)

vs

struct FooRef(*mut bindgen::foo)

This gets in the way of having an owned Foo embedded into a larger struct. It also gets in the way of instantiating an owned Foo on the stack.

My first thought was to use enums:

enum Foo {
Owned(bindgen::foo),
NotOwned(*mut bindgen::foo),
}

But that would mean that users would invariably pay the price for the owned variant always, as enums use as much space as its largest variant.

My current understanding is that we can move all the implementations to traits, with a suitable bound on AsRef<bindings::foo> and AsMut<bindings::foo>.

Here is a code example for the wrapper of bindings::v4l2_format (see format.rs), which was extended to account for both owned and non-owned bindgen types:

```
use core::cell::UnsafeCell;

/// The shared implementation between Format and FormatRef.
pub trait FormatImpl: AsRef<bindings::v4l2_format> +
AsMut<bindings::v4l2_format> {
    /// Returns the `type_` field.
    fn type_(&self) -> u32 {
        self.as_ref().type_
    }

    /// Get the field `field` for the `pix` union member.
    fn pix_field(&self) -> Result<enums::Field> {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        enums::Field::try_from(pix.field)
    }

    /// Get the field `width` for the `pix` union member.
    fn pix_width(&self) -> u32 {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        pix.width
    }

    /// Get the field `height` for the `pix` union member.
    fn pix_height(&self) -> u32 {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        pix.height
    }

    /// Get the field `pixelformat` for the `pix` union member.
    fn pix_pixelformat(&self) -> u32 {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        pix.pixelformat
    }

    /// Get the field `bytesperline` for the `pix` union member.
    fn pix_bytesperline(&self) -> u32 {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        pix.bytesperline
    }

    /// Get the field `sizeimage` for the `pix` union member.
    fn pix_sizeimage(&self) -> u32 {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        pix.sizeimage
    }

    /// Get the field `colorspace` for the `pix` union member.
    fn pix_colorspace(&self) -> Result<enums::Colorspace> {
        let fmt = self.as_ref();
        let pix = &unsafe { fmt.fmt.pix };
        enums::Colorspace::try_from(pix.colorspace)
    }

    /// Set the field `field` for the `pix` union member.
    fn set_pix_field(&mut self, field: enums::Field) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.field = field as u32;
    }

    /// Set the field `width` for the `pix` union member.
    fn set_pix_width(&mut self, width: u32) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.width = width;
    }

    /// Set the field `height` for the `pix` union member.
    fn set_pix_height(&mut self, height: u32) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.height = height;
    }

    /// Set the field `pixelformat` for the `pix` union member.
    fn set_pix_pixel_format(&mut self, pixel_format: u32) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.pixelformat = pixel_format;
    }

    /// Set the field `bytesperline` for the `pix` union member.
    fn set_pix_bytesperline(&mut self, bytesperline: u32) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.bytesperline = bytesperline;
    }

    /// Set the field `sizeimage` for the `pix` union member.
    fn set_pix_sizeimage(&mut self, sizeimage: u32) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.sizeimage = sizeimage;
    }

    /// Set the field `sizeimage` for the `pix` union member.
    fn set_pix_colorspace(&mut self, colorspace: enums::Colorspace) {
        let fmt = self.as_mut();
        let pix = &mut unsafe { fmt.fmt.pix };
        pix.colorspace = colorspace as u32;
    }
}

/// A wrapper over a pointer to `struct v4l2_format`.
#[derive(Copy, Clone, Debug, PartialEq, PartialOrd)]
pub struct FormatRef(*mut bindings::v4l2_format);

impl FormatRef {
    /// # Safety
    /// The caller must ensure that `ptr` is valid and remains valid
for the lifetime of the
    /// returned [`FormatRef`] instance.
    pub unsafe fn from_ptr(ptr: *mut bindings::v4l2_format) -> Self {
        Self(ptr)
    }
}

impl AsRef<bindings::v4l2_format> for FormatRef {
    fn as_ref(&self) -> &bindings::v4l2_format {
        // SAFETY: ptr is safe during the lifetime of [`FormatRef`] as
per
        // the safety requirement in `from_ptr()`
        unsafe { self.0.as_ref().unwrap() }
    }
}

impl AsMut<bindings::v4l2_format> for FormatRef {
    fn as_mut(&mut self) -> &mut bindings::v4l2_format {
        // SAFETY: ptr is safe during the lifetime of [`FormatRef`] as
per
        // the safety requirement in `from_ptr()`
        unsafe { self.0.as_mut().unwrap() }
    }
}

impl FormatImpl for FormatRef {}

/// An owned version of `FormatRef`.
#[derive(Default)]
pub struct Format(UnsafeCell<bindings::v4l2_format>);

impl AsRef<bindings::v4l2_format> for Format {
    fn as_ref(&self) -> &bindings::v4l2_format {
        // SAFETY:
        // It is safe to dereference the pointer since it is valid
whenever this type is instantiated.
        // It is safe to cast this into a shared reference: because
this is the
        // only method that returns &bindings::v4l2_format and because
we take
        // &self, the compiler takes care of enforcing the Rust
reference rules for
        // us. Thus, enforcing the safety guarantees of
UnsafeCell::get() by
        // proxy.
        unsafe { &*self.0.get() }
    }
}

impl AsMut<bindings::v4l2_format> for Format {
    fn as_mut(&mut self) -> &mut bindings::v4l2_format {
        // SAFETY:
        // It is safe to dereference the pointer since it is valid
whenever this type is instantiated.
        // It is safe to cast this into an exclusive reference: because
this is the
        // only method that returns &mut bindings::v4l2_format and
because we take
        // &mut self, the compiler takes care of enforcing the Rust
reference rules for
        // us. Thus, enforcing the safety guarantees of
UnsafeCell::get() by
        // proxy.
        unsafe { &mut *self.0.get() }
    }
}

impl FormatImpl for Format {}

```

This makes it possible to:

- Share the implementations between Format and FormatRef,
- Have both Format (while paying the cost of storing the bindings::v4l2_format member) and FormatRef (while paying the cost of storing a pointer) separately.
- Be generic over Format and FormatRef when needed, e.g.:

```
fn some_fn(format: impl FormatImpl) {...}
``` 

Thoughts?

-- Daniel

#8 lore raw
In reply to Daniel Almeida
Quoted 6 lines · Daniel Almeida

Hi all, this is my first attempt at adding Rust support to the media subsystem.

Please let me know your thoughts.

Hi Daniel,

I think V4L2 should be written in primarily one language.

At first, I think Rust for V4L2 has no benefits for media drivers, webcams, DVB-S/T/T2, pointing tablets and so on. You assume that all code is running inside the kernel and needs to be perfect. But I think you could just aswell implement the next USB webcam V4L2 driver in Perl for that sake.

The reason for my point of view, is that I think most of the drivers in media/ should run in user-space, and not inside the kernel. The driver is killed when the device is detached, and all lost memory is reclaimed, automagically. Then there exist proper methods to lock-down all interfaces and file handles, so that device drivers will not do any harm, even if exploited. For example the Capsicum library, I'm using FreeBSD.

Debugging stuff using GDB in user-space, is so much more convenient than debugging stuff inside the kernel. And the development time is much faster.

The example of secure V4L2 programming is already here:
https://github.com/hselasky/webcamd

I would rather like more drive on that, than flowing down the Rust stream. Rust is cool, Java is cool, VM's are cool. The only bad about cool things, is that they are so slow. For many years I completely avoided C++ code for the sake it is very slow to compile, compared to bare C code. And when looking at how Firefox is building using Rust, I am a little worried, why we need so much code in there!

Engineering energy would be much more focused, if hardware vendors could agree more about what binary formats to use for their device protocols, than changing the coding language, so that now anyone can be let loose to program in the Linux kernel without risking any damage.

The goal for Linux driver development should be fewer drivers and not more. I'm glad if not everyone out there can do my job writing C-code for device drivers. We don't need more people to mess around there simply. I don't want Linux to become the next Microsoft, with gigabytes of drivers which are never used for anything.

The webcamd daemon already is close to 6 MBytes big on amd64 on FreeBSD. Inside there is support for 510 drivers (counting =y keywords), built straight off Linus Torvalds:

cat config | grep CONFIG | grep "=y" | wc -l

     510

ls -l which webcamd
-r-xr-xr-x 1 root wheel 5915016 Mar 30 19:09 /usr/local/sbin/webcamd

The USB video class is great, instead of tons of GSPCA devices, then yeah, we don't need to drag around so much legacy binaries, just to make everyone happy. What did Apple do? Custom PCI webcam devices? Why can't they just stick with virtual USB devices, and then have a dual configured device, one config for their own HD codec, and one config for people like me, just needing the framebuffer.

You asked for a comment and now you got one!

--HPS

author reviewer
#9 lore raw

Hi Hans! Thank you for chiming in!

There's a few things in your email that I disagree with and that I'd like to
address.

Quoted 1 line

I think V4L2 should be written in primarily one language.

It is, in C. This series is about adding *bindings* to write *drivers* in Rust
*for those interested*. The v4l2 core remains untouched, and I don't think there
are any plans to introduce Rust outside of drivers in the kernel at all, last I
heard.

Quoted 1 line

You assume that all code is running inside the kernel and needs to be

perfect.

No I do not assume that. In fact, Rust code is absolutely not guaranteed to be
bug free and definitely not "perfect".

On the other hand, I would take Rust over C any day. Thus I am contributing some
of the infrastructure to make this possible for me and for others.

IMHO I think you're approaching this from the wrong angle. It isn't that Linux
*needs* Rust. It's more about providing a new and safer choice with modern ergonomics for developers, is all.

Quoted 1 line

I would rather like more drive on that, than flowing down the Rust

stream.

These two things are not mutually exclusive :)

Quoted 1 line

Rust is cool, Java is cool, VM's are cool.

I don't see why Java and virtual machines are being brought into the discussion
for this patchset here. And compilation times are a part of life, sadly. Also,
can you substantiate your claim that Rust is slow?

Quoted 1 line

Engineering energy would be much more focused, if hardware vendors

could agree
more about what binary formats to use for their device protocols,

I understand, but my patchset is not to blame here. In fact, I have no say at
all over these things.

Quoted 1 line

than changing the coding language

This simply is not what is happening here. Again this is about giving kernel
developers another *option* of programming language, not about ditching C.

Quoted 1 line

that now anyone can be let loose to program in the Linux kernel

without
risking any damage

Who's "anyone"? Plus the review process stays in place, so hardly any changes to
code quality here.

Quoted 1 line

I'm glad if not everyone out there can do my job writing C-code for

device
drivers. We don't need more people to mess around there simply.

Ok we differ strongly here. In particular, I am totally neutral to your first
statement.

The reality is that it isn't up to anyone to say who should or shouldn't become
a kernel developer. The resources are out there for anyone to try, and if
maintainers take in their patches, then that's the end of the story.

-- Daniel

#10 lore raw

Hi Daniel,

Quoted 8 lines · Daniel Almeida

Hi Hans! Thank you for chiming in!

There's a few things in your email that I disagree with and that I'd like to
address.

Quoted 1 line

I think V4L2 should be written in primarily one language.

Quoted 7 lines

It is, in C. This series is about adding *bindings* to write *drivers* in Rust
*for those interested*. The v4l2 core remains untouched, and I don't think there
are any plans to introduce Rust outside of drivers in the kernel at all, last I
heard.

I see your point, but still I think it is better to have good examples, than to say, there is a room for everything, just come here :-)

Adding a dependency to build the Rust compiler even to build one or two V4L2 device drivers, would mean a lot to my small hselasky/webcamd project. It already has to fetch a copy of the Linux kernel, and now has to bootstrap Rust from stage0 to stageN. I personally say no. It's like XCode unfortunately. I download 100's of GBytes of upgrades to XCode, and barely upload one millionth worth of code back to Apple. It's not good. Software developers shouldn't have to download more stuff than they upload?

Quoted 7 lines
Quoted 2 lines

You assume that all code is running inside the kernel and needs to be

perfect.

No I do not assume that. In fact, Rust code is absolutely not guaranteed to be
bug free and definitely not "perfect".

The definition of "bugs" may vary of course. I was thinking more like stack exploits, missing validation of arrays and so on.

Quoted 3 lines

On the other hand, I would take Rust over C any day. Thus I am contributing some
of the infrastructure to make this possible for me and for others.

I must admit I'm not a Rust guy and don't see the advantages of Rust like you do.

Quoted 4 lines

IMHO I think you're approaching this from the wrong angle. It isn't that Linux
*needs* Rust. It's more about providing a new and safer choice with modern ergonomics for developers, is all.

Why not move Linux-V4L2 drivers to user-space? In my opinion Rust is much more easy to get going there than at the kernel level.

Quoted 6 lines
Quoted 2 lines

I would rather like more drive on that, than flowing down the Rust

stream.

These two things are not mutually exclusive :)

Quoted 7 lines
Quoted 1 line

Rust is cool, Java is cool, VM's are cool.

I don't see why Java and virtual machines are being brought into the discussion
for this patchset here. And compilation times are a part of life, sadly. Also,
can you substantiate your claim that Rust is slow?

Rust is slow based on my observations building Firefox from sources. The Rust compiler spends a significant amount of time per source file.

Quoted 8 lines
Quoted 1 line

Engineering energy would be much more focused, if hardware vendors

could agree
more about what binary formats to use for their device protocols,

I understand, but my patchset is not to blame here. In fact, I have no say at
all over these things.

Quoted 6 lines
Quoted 1 line

than changing the coding language

This simply is not what is happening here. Again this is about giving kernel
developers another *option* of programming language, not about ditching C.

I think this option belongs in user-space and not Linux (the kernel). More stuff should be moved there, that is my view.

Quoted 8 lines
Quoted 2 lines

that now anyone can be let loose to program in the Linux kernel

without
risking any damage

Who's "anyone"? Plus the review process stays in place, so hardly any changes to
code quality here.

Maybe the word "anyone" was a bit unclear in this regard. I take that back for now.

Quoted 8 lines
Quoted 1 line

I'm glad if not everyone out there can do my job writing C-code for

device
drivers. We don't need more people to mess around there simply.

Ok we differ strongly here. In particular, I am totally neutral to your first
statement.

Quoted 5 lines

The reality is that it isn't up to anyone to say who should or shouldn't become
a kernel developer. The resources are out there for anyone to try, and if
maintainers take in their patches, then that's the end of the story.

The GPLv2 license should not be the only reason behind Linux developers putting drivers in the kernel-space. I think moving more stuff to user-space would benefit a greater purpose.

Summed up:

My main objection is Rust compiler support for _kernel_ V4L2 drivers. My opinion it belongs to user-space for now and why not do something there instead?

--HPS

#11 lore raw
Quoted 5 lines · Hans Petter Selasky

Adding a dependency to build the Rust compiler even to build one or two V4L2 device drivers, would mean a lot to my small hselasky/webcamd project. It already has to fetch a copy of the Linux kernel, and now has to bootstrap Rust from stage0 to stageN. I personally say no. It's like

Do you mean you need to compile rustc? Could you please explain why? Could you use your distribution's, or fetch the standalone installers or cache your own toolchain?

Quoted 4 lines

XCode unfortunately. I download 100's of GBytes of upgrades to XCode, and barely upload one millionth worth of code back to Apple. It's not good. Software developers shouldn't have to download more stuff than they upload?

The Rust standalone installers are 2+ orders of magnitude lighter.

Quoted 2 lines

The definition of "bugs" may vary of course. I was thinking more like stack exploits, missing validation of arrays and so on.

The kernel definitely needs to avoid those. What do you mean?

Quoted 2 lines

I must admit I'm not a Rust guy and don't see the advantages of Rust like you do.

The advantages are fairly clear. The question has always been whether the cost is worth those benefits.

Quoted 2 lines

Why not move Linux-V4L2 drivers to user-space? In my opinion Rust is much more easy to get going there than at the kernel level.

That sounds like an orthogonal discussion.

In any case, please note that you would need to install the same Rust toolchain to compile them in userspace. So, if you are concerned about the size of the toolchain (as you mention above), it would not really make a difference.

Quoted 2 lines

Rust is slow based on my observations building Firefox from sources. The Rust compiler spends a significant amount of time per source file.

It is slower than compiling C, but it also provides more features, so it seems fair for what we are getting in exchange.

Cheers,
Miguel

#12 lore raw
Quoted 10 lines · Miguel Ojeda
Quoted 5 lines · Hans Petter Selasky

Adding a dependency to build the Rust compiler even to build one or two V4L2 device drivers, would mean a lot to my small hselasky/webcamd project. It already has to fetch a copy of the Linux kernel, and now has to bootstrap Rust from stage0 to stageN. I personally say no. It's like

Do you mean you need to compile rustc? Could you please explain why? Could you use your distribution's, or fetch the standalone installers or cache your own toolchain?

Hi Miguel,

Assume you need to update both the kernel and the rust compiler at the same time. How do you do that? In the binary download case you have two machines. One to build rust and one to build the kernel, so it is technically not possible?

The Rust compiler has a dependency on the kernel and the kernel has a dependency on the Rust compiler. That just means, some kind of changes can never happen. This is the ingredient for never ending problems. It's like you put some rock into the system: If this ever needs to change ...

I'll give you a real-life example to emphasis this. Apple and Microsoft has done something very bad in the file system area. They mistreat what happens to be the Norwegian character "å" (0xE5). Norway is where I live. Their solution is to split the "å" character into the "a" character (0x61) and the combining ring-over character (0x30A).

There are three problems:

1) Many Unicode implementations only expect one combining ring-over character. Either this leads directly to a stack exploit or a denial of service, depending on the actual code: CVE-2023-25193 (ongoing).

2) The proper solution would be to deny this kind of combining characters, also called umlauts in Germany. Only that requires both Apple and Microsoft to change and update both their filesystem and kernel at the same time! The "å" character (0xE5) is essential for quickly deleting files. Or disable this feature, and rewrite the directory table every time a file is deleted.

3) Apple and Microsoft managed to screw this up, so that you can create files under Microsoft (exFat-disk), that don't show up under MacOS. In iOS they show up however, but can't be copied or moved anywhere. And if you think your files are backed up in the iCloud, think again!

The consequences can be quite serious, that you could end up being unfairly judged by the Police in Norway, because court documents "just got lost" they say.

Do you think Microsoft and Apple will ever change this dependency, if a change means you need to re-format filesystems live or risk a serious performance degradation? I have my personal doubts.

I think the problem with the "å" character I've described above, is a forever problem created by Apple and Microsoft and IBM and who knows what more. It's not possible to solve, without a serious cost, and having this secret automagic trashbin for files that just a few people use, compared to the big picture, is not an issue for them. Even a few people going to jail for 21 years, is not an issue. Who cares, is the impression I get from customer support at both Microsoft and Apple. And not at least, who knows about this really!

Daniel and Miguel: By saying it is not a good thing to build systems completely from source, both kernel and toolchain and everything that goes with it, you basically say that permanent "dependencies" between the compilers and the kernel will never be a problem. You are building on a rock, and only the future knows if what you consider a rock today is really a problem tomorrow.

In my example the unicode alphabet is a problem. So tell me: How would you update a system, if the value of every single letter in the unicode alphabet would change?

Quoted 7 lines
Quoted 5 lines

XCode unfortunately. I download 100's of GBytes of upgrades to XCode, and barely upload one millionth worth of code back to Apple. It's not good. Software developers shouldn't have to download more stuff than they upload?

The Rust standalone installers are 2+ orders of magnitude lighter.

For people that build stuff on their laptops it still matters. If you have a beefy machine, it is a different case.

Quoted 5 lines
Quoted 3 lines

The definition of "bugs" may vary of course. I was thinking more like stack exploits, missing validation of arrays and so on.

The kernel definitely needs to avoid those. What do you mean?

I thought that Rust didn't allow you to write outside the bounds of arrays, similarly to the old Turbo Pascal language?

Quoted 6 lines
Quoted 3 lines

I must admit I'm not a Rust guy and don't see the advantages of Rust like you do.

The advantages are fairly clear. The question has always been whether the cost is worth those benefits.

If there could be one base compiler and toolchain, I would be happy.

Quoted 5 lines
Quoted 3 lines

Why not move Linux-V4L2 drivers to user-space? In my opinion Rust is much more easy to get going there than at the kernel level.

That sounds like an orthogonal discussion.

Sure.

Quoted 11 lines

In any case, please note that you would need to install the same Rust toolchain to compile them in userspace. So, if you are concerned about the size of the toolchain (as you mention above), it would not really make a difference.

Quoted 2 lines

Rust is slow based on my observations building Firefox from sources. The Rust compiler spends a significant amount of time per source file.

It is slower than compiling C, but it also provides more features, so it seems fair for what we are getting in exchange.

Right, so think about where that slowness may end up one day, if you suddenly need to re-build everything from sources so to say :-)

Thanks for your input!

--HPS

#13 lore raw
Quoted 5 lines · Hans Petter Selasky

Assume you need to update both the kernel and the rust compiler at the same time. How do you do that? In the binary download case you have two machines. One to build rust and one to build the kernel, so it is technically not possible?

I don't understand the problem -- you can build (or download) new toolchains without changing the kernel, and you can keep several kernels and several toolchains installed, too.

Quoted 5 lines

I'll give you a real-life example to emphasis this. Apple and Microsoft has done something very bad in the file system area. They mistreat what happens to be the Norwegian character "å" (0xE5). Norway is where I live. Their solution is to split the "å" character into the "a" character (0x61) and the combining ring-over character (0x30A).

Sorry, but I don't see how all that relates to the current discussion (kernel).

Quoted 2 lines

Daniel and Miguel: By saying it is not a good thing to build systems completely from source,

I haven't said that at all. I regularly build from source myself, in fact.

Quoted 2 lines

For people that build stuff on their laptops it still matters. If you have a beefy machine, it is a different case.

I don't follow. You said you are downloading 100s of GiB for XCode, but you are not OK with 100s of MiBs for Rust?

Anyway, both the Rust toolchain and the kernel can be built on laptops (I do so), and they don't need to be the highest end ones at all.

Quoted 2 lines

I thought that Rust didn't allow you to write outside the bounds of arrays, similarly to the old Turbo Pascal language?

It avoids all UB, including data races, not just out-of-bounds accesses, as long as the unsafe parts are sound (and there are no compiler bugs etc.). Which is one of the main reasons we want it in the kernel.

Quoted 1 line

If there could be one base compiler and toolchain, I would be happy.

If you mean a single vendor, then it may be interesting for you that GCC will include Rust support in future releases. It remains to be seen when their compiler is ready for building the kernel parts, but it is one of their goals as far as I understand.

Quoted 2 lines

Right, so think about where that slowness may end up one day, if you suddenly need to re-build everything from sources so to say :-)

If you want to build everything from source, then you will need some CPU time to do so. That is just how things work. Most people will just use the toolchain from their distribution.

Quoted 1 line

Thanks for your input!

Not at all, thanks for your input too :)

Cheers,
Miguel

#14 lore raw
Quoted 9 lines · Miguel Ojeda
Quoted 4 lines · Hans Petter Selasky

Assume you need to update both the kernel and the rust compiler at the same time. How do you do that? In the binary download case you have two machines. One to build rust and one to build the kernel, so it is technically not possible?

I don't understand the problem -- you can build (or download) new toolchains without changing the kernel, and you can keep several kernels and several toolchains installed, too.

Hi Miguel,

What I'm saying is a bit difficult to get about right off the bat.

Quoted 1 line

you can build (or download) new toolchains without changing the kernel

Yes, most of the time, but not always. Let me rephrase it:

If you cannot build a new toolchain without a new kernel.

And:

You cannot build a new kernel without a new toolchain.

Then you are stuck forever to build a new toolchain and kernel? Do you agree?

Or you can say, someone else needs to deal with it, but then you have a single point of failure.

It's like the next version of the Rust compiler depends on the previous version of the Rust compiler.

But now you have (Version of Linux and version of Rust) depends on (Version of Linux and version of Rust).

So in order to upgrade either Linux or Rust, you may be forced to go through multiple, upgrade kernel, reboot, upgrade rust, build kernel, reboot and so on.

And that is an annoying complication when upgrading a system.

--HPS

#15 lore raw
Quoted 2 lines · Hans Petter Selasky

If you cannot build a new toolchain without a new kernel.

Why not?

Quoted 2 lines

Then you are stuck forever to build a new toolchain and kernel? Do you agree?

No, I don't agree, because I don't understand why you cannot build the new toolchain in the old kernel, or use a pre-built toolchain for that matter (whether built by you or by somebody else).

Quoted 2 lines

Or you can say, someone else needs to deal with it, but then you have a single point of failure.

No, you could build your own toolchain and save it somewhere, if you don't want to rely on a build from somebody else.

Cheers,
Miguel

#16 lore raw
Quoted 4 lines · Miguel Ojeda
Quoted 2 lines · Hans Petter Selasky

If you cannot build a new toolchain without a new kernel.

Hi,

Quoted 1 line

Why not?

To me it is very simple:

Look at this:

-#define FE_GET_PROPERTY _IOR('o', 83, struct dtv_properties)
+#define FE_GET_PROPERTY _IOW('o', 83, struct dtv_properties)
+#define FE_GET_PROPERTY_OLD _IOR('o', 83, struct dtv_properties)

The FE_GET_PROPERTY IOCTL definition is incorrectly specified as reading data. While it is actually writing data. When will this be fixed in Linux - I think never. That's just the way both Linux and GIT works, unfortunately, though that's another discussion. You can put stuff in, but you can't easily get stuff out, without it having consequences.

Similarly rustc may depend on an incorrectly specified ioctl() definition, also via other libraries and static linking, that just have to stay incorrectly defined, because it was initially incorrectly defined.

Daniel, please explain why the few lines of chunk above (and there are some more) cannot be upstreamed into Linux?

Quoted 8 lines
Quoted 3 lines

Then you are stuck forever to build a new toolchain and kernel? Do you agree?

No, I don't agree, because I don't understand why you cannot build the new toolchain in the old kernel, or use a pre-built toolchain for that matter (whether built by you or by somebody else).

Quoted 5 lines
Quoted 2 lines

Or you can say, someone else needs to deal with it, but then you have a single point of failure.

No, you could build your own toolchain and save it somewhere, if you don't want to rely on a build from somebody else.

I'm trying to explain something difficult. And I'm OK that you neither understand nor agree about my viewpoint. See my replies above.

--HPS

#17 lore raw
Quoted 4 lines · Hans Petter Selasky

Similarly rustc may depend on an incorrectly specified ioctl() definition, also via other libraries and static linking, that just have to stay incorrectly defined, because it was initially incorrectly defined.

Why would a compiler depend on random ioctls? Even if it did, how is that related to the previous discussion? A compiler is just one more userspace application. Whether the kernel uses C or Rust internally has nothing to do with that.

Also, I don't follow your logic. You said you cannot upgrade your toolchain (for some reason), and your argument is that the kernel keeps interfaces stable? Well, yes, that is the point and what allows you to upgrade.

Moreover, what is special about rustc here? What about your C toolchain?

Quoted 2 lines

I'm trying to explain something difficult. And I'm OK that you neither understand nor agree about my viewpoint. See my replies above.

No, it is not a matter of being difficult. It is just that you have not shown how you would be prevented from upgrading a toolchain.

Cheers,
Miguel

#18 lore raw
Quoted 9 lines · Miguel Ojeda
Quoted 4 lines · Hans Petter Selasky

Similarly rustc may depend on an incorrectly specified ioctl() definition, also via other libraries and static linking, that just have to stay incorrectly defined, because it was initially incorrectly defined.

Why would a compiler depend on random ioctls? Even if it did, how is that related to the previous discussion? A compiler is just one more userspace application.

Hi,

Is the right hand knowing what the left hand is doing? Are the people behind Rust aware Rust is being used for kernel purposes or not?

That's why I brought up the file-system issue with Microsoft and Apple as an example. The Unicode guys probably knew nothing about what the letter valued 0xE5 was used for in various file systems, so they thought it was fine to assign a letter there, the Norwegian "å". I think neither anyone at the two big companies mentioned tried to stop Unicode from doing such a clear mistake either.

Microsoft and Apple is the left hand, and Unicode is the right hand.

That's why the toolchain should be included in the Linux kernel. So that the people using Linux know that the toolchain works as intended when compiling the Linux kernel.

It's a generic issue. If two organizations that make products for eachother, don't talk closely together, you risk exactly what I point at, that some stupid decision will be made by the one party, which doesn't really affect the other party, but innocent customers infact.

Quoted 1 line

Why would a compiler depend on random ioctls?

Can you say you can write even a test C-program to multiply two 32-bit numbers, bit by bit, without even deleting a single character once? People who say C-programmers never do mistakes, are naive. Even standard ioctls() may contain mistakes and there needs to be a plan to fix such issues. And when you think the code is right, the compiler is to blame, and when you think the compiler is right, the CPU is to blame and so it goes.

Quoted 2 lines

Whether the kernel uses C or Rust internally
has nothing to do with that.

The question is not OR, but AND related. If the kernel will need both at some point in the future, it's not good. The plan should be either OR: Rustc ^ GCC = true. Not Rustc | GCC = true :-)

Quoted 4 lines

Also, I don't follow your logic. You said you cannot upgrade your toolchain (for some reason), and your argument is that the kernel keeps interfaces stable? Well, yes, that is the point and what allows you to upgrade.

You need to see, stable interfaces may also need to be changed. That is where you invert my logic. If you fix that when reading my text, you will see what I'm saying is true and not false.

There may be bit-pattern things down at CPU level, triggering bit-flips, that CPU vendors will do nothing about, because the argument is typically about money and performance. If something costs both money and hurts performance, it will not be implemented. It's like the speculative instruction prediction and resulting cache pollution, allowing memory to leak from kernel level to user-space level. Isn't it enough to deal with this in GCC only? Does Rust handle such issues at all? I don't know simply.

And what about syscall numbers? What if someone from Intel says all syscall numbers must be divisible by four, because those two lower bit-lines are frequently subject to bit flips and we can do nothing about it.

Quoted 2 lines

Moreover, what is special about rustc here? What about your C toolchain?

I don't know Rustc that well, so I cannot answer what's special about it. But based on my existing experience with C toolchains, I don't expect it to be any easier, with regards to handling unforeseen issues.

Quoted 6 lines
Quoted 3 lines

I'm trying to explain something difficult. And I'm OK that you neither understand nor agree about my viewpoint. See my replies above.

No, it is not a matter of being difficult. It is just that you have not shown how you would be prevented from upgrading a toolchain.

The proof is in a principle. Principles are there to avoid unpredictable problems.

Apparently you don't accept the principle of talking closely together when you are in a supply chain.

I have a feeling you think like this: If I do my job great, and all others in the supply chain do their best, then the resulting product will be the great too!

Translated to your case: Linux is the most stable OS in the world, and Rust is the most secure compiler language in the world. Nothing can go wrong!

--HPS

Quoted 3 lines

Cheers,
Miguel

#19 lore raw
Quoted 3 lines · Hans Petter Selasky

That's why the toolchain should be included in the Linux kernel. So that the people using Linux know that the toolchain works as intended when compiling the Linux kernel.

That's not how Linux has ever worked, sorry. So this is not even a valid discussion anymore.

greg k-h

#20 lore raw
Quoted 8 lines · Greg KH
Quoted 3 lines · Hans Petter Selasky

That's why the toolchain should be included in the Linux kernel. So that the people using Linux know that the toolchain works as intended when compiling the Linux kernel.

That's not how Linux has ever worked, sorry. So this is not even a valid discussion anymore.

Well, maybe it's time to change your views on that and stop being a rock thrower on your friends, the compiler folks, whenever something goes wrong:

https://news.ycombinator.com/item?id=8089321

--HPS

#21 lore raw
In reply to Daniel Almeida
Quoted 6 lines · Daniel Almeida

and I don't
think there
are any plans to introduce Rust outside of drivers in the kernel at all, last I
heard.

This was indeed our original proposal (actually, for "leaf" modules in general, not only device drivers), but where the line is drawn is up to the kernel maintainers. For instance, some of them have expressed interest in potentially having Rust subsystems in the future.

Cheers,
Miguel

#22 lore raw
In reply to Hans Petter Selasky
Quoted 1 line · Hans Petter Selasky

You assume that all code is running inside the kernel and needs to be perfect.

Yes, of course. It's kernel code.

If you're borrowing kernel code for your userland stuff, than it's up to you to take care of it.

There is no such thing like stable/fixed in-kernel APIs - it always has been this way.

Quoted 1 line

But I think you could just aswell implement the next USB webcam V4L2 driver in Perl for that sake.

That would't be v4l anymore.

Quoted 1 line

The reason for my point of view, is that I think most of the drivers in media/ should run in user-space, and not inside the kernel.

Feel free to provide fully userspace drivers for those devices, but that's totally unrelated to kernel development (no matter whether you're copying over kernel code into your userland project).

Quoted 2 lines

The example of secure V4L2 programming is already here:
https://github.com/hselasky/webcamd

BSD code is not in scope of the LKML.

Quoted 1 line

I would rather like more drive on that, than flowing down the Rust stream.

Orthogonal topic.

Quoted 1 line

Rust is cool, Java is cool, VM's are cool. The only bad about cool things, is that they are so slow. For many years I completely avoided C++ code for the sake it is very slow to compile, compared to bare C code. And when looking at how Firefox is building using Rust, I am a little worried, why we need so much code in there!

Yes, compiling Rust is slow, compared to C. That's the price for sophisticated optimizations. How often do you have to do a full recompile ?

Quoted 1 line

Engineering energy would be much more focused, if hardware vendors could agree more about what binary formats to use for their device protocols,

Indeed. But we (kernel folks) have no influence on that. If we could, we'd already standardized HW interfaces for lots of things and so only a small percentage of the drivers we currently have, while still supporting the same number of HW (or even more). But unfortunately thats not under our control. Therefore offtopic.

Quoted 1 line

than changing the coding language, so that now anyone can be let loose to program in the Linux kernel without risking any damage.

AFAIK, this discussion isn't about changing the kernel's programming language, but just adding language bindings, so some new drivers could be written in that language. If this really happens and you really want a C implementation, feel free to send patches.

Quoted 1 line

The goal for Linux driver development should be fewer drivers and not more.

Depends on specific case. We already have lots of drivers that support wide range of devices. But it doesn't make sense having monster drivers for entirely different devices.

Quoted 1 line

I don't want Linux to become the next Microsoft, with gigabytes of drivers which are never used for anything.

Actually, we're doing a pretty good job of generalizing things that can be generalized (if you find room for more improvements, feel free to send patches). Nobody here seriously intents dropping the subsystem and lib architectures in favour of monolithic monster drivers like in Windows world.

Quoted 1 line

The webcamd daemon already is close to 6 MBytes big on amd64 on FreeBSD. Inside there is support for 510 drivers (counting =y keywords), built straight off Linus Torvalds:

You have ~500 drivers in one 6MB binary and blame us for writing too much code ? Maybe you should think about modularization (which we do have in the kernel).

And, btw, FreeBSD is completely off-topic here.

Quoted 1 line

The USB video class is great, instead of tons of GSPCA devices, then yeah, we don't need to drag around so much legacy binaries, just to make everyone happy. What did Apple do? Custom PCI webcam devices? Why can't they just stick with virtual USB devices, and then have a dual configured device, one config for their own HD codec, and one config for people like me, just needing the framebuffer.

Well, they just could have an USB device layered on-top of a tiny PCI- USB bridge. We're the wrong to blame - talk to Apple.

--mtx

Signature · 8 lines
---
Hinweis: unverschlüsselte E-Mails können leicht abgehört und manipuliert
werden ! Für eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schlüssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
#23 lore raw
In reply to Daniel Almeida
Quoted 14 lines · Daniel Almeida

Hi all, this is my first attempt at adding Rust support to the media subsystem.

It adds just enough support to write a clone of the virtio-camera prototype written by my colleague, Dmitry Osipenko, available at [0].

Basically, there's support for video_device_register, v4l2_device_register and for some ioctls in v4l2_ioctl_ops. There is also some initial vb2 support, alongside some wrappers for some types found in videodev2.h.

I wrote a sample Rust driver just to prove that this probes, and that you get a message on dmesg whenever an ioctl is called.

Quoted 8 lines

As there is no actual implementation for any of the ioctls, this module can misbehave with some programs. I can work around this in a future submission.

Note that this is based on the rust branch, as opposed to rust-next. The reasoning is simple: I expect this series to just kickstart some discussion around the subject. Actual upstreaming can come up much later, at which point I can rebase on the rust branch.

Hi Daniel - I don't know if the 'rust branch' is common knowledge but it helped me to know it was this:
https://github.com/Rust-for-Linux/linux

For what it's worth, I was able to get the V4L2 Rust sample probed pretty easily following the quick start instructions
https://docs.kernel.org/rust/quick-start.html

Cool to see this working!
Deb

Quoted 60 lines

Lastly, the origins of this series trace back to a v4l2 virtIO driver I was writing in Rust. As the project was eventually shelved for other reasons, I picked both the virtIO and the v4l2 bindings into their own patches which I am now in the process of submitting. This is to say that I tested this code with said driver and CrosVM in the past, and it worked ok.

Please let me know your thoughts.

[0]: https://gitlab.collabora.com/dmitry.osipenko/linux-kernel-rd/-/commit/055a2c322e931a8b388f864f1db81bbdfd525602

Daniel Almeida (6):
rust: media: add the media module
rust: media: add initial videodev2.h abstractions
rust: sync: introduce FfiMutex
rust: media: videobuf2: add a videobuf2 abstraction
rust: media: add {video|v4l2}_device_register support
rust: media: add v4l2 rust sample

 rust/bindings/bindings_helper.h        |   8 +
 rust/kernel/lib.rs                     |   2 +
 rust/kernel/media/mod.rs               |   6 +
 rust/kernel/media/v4l2/capabilities.rs |  80 ++++
 rust/kernel/media/v4l2/dev.rs          | 369 +++++++++++++++
 rust/kernel/media/v4l2/device.rs       | 115 +++++
 rust/kernel/media/v4l2/enums.rs        | 135 ++++++
 rust/kernel/media/v4l2/format.rs       | 178 ++++++++
 rust/kernel/media/v4l2/framesize.rs    | 176 +++++++
 rust/kernel/media/v4l2/inputs.rs       | 104 +++++
 rust/kernel/media/v4l2/ioctls.rs       | 608 +++++++++++++++++++++++++
 rust/kernel/media/v4l2/mmap.rs         |  81 ++++
 rust/kernel/media/v4l2/mod.rs          |  13 +
 rust/kernel/media/videobuf2/core.rs    | 552 ++++++++++++++++++++++
 rust/kernel/media/videobuf2/mod.rs     |   5 +
 rust/kernel/sync.rs                    |   1 +
 rust/kernel/sync/ffi_mutex.rs          |  70 +++
 samples/rust/Kconfig                   |  11 +
 samples/rust/Makefile                  |   1 +
 samples/rust/rust_v4l2.rs              | 403 ++++++++++++++++
 20 files changed, 2918 insertions(+)
 create mode 100644 rust/kernel/media/mod.rs
 create mode 100644 rust/kernel/media/v4l2/capabilities.rs
 create mode 100644 rust/kernel/media/v4l2/dev.rs
 create mode 100644 rust/kernel/media/v4l2/device.rs
 create mode 100644 rust/kernel/media/v4l2/enums.rs
 create mode 100644 rust/kernel/media/v4l2/format.rs
 create mode 100644 rust/kernel/media/v4l2/framesize.rs
 create mode 100644 rust/kernel/media/v4l2/inputs.rs
 create mode 100644 rust/kernel/media/v4l2/ioctls.rs
 create mode 100644 rust/kernel/media/v4l2/mmap.rs
 create mode 100644 rust/kernel/media/v4l2/mod.rs
 create mode 100644 rust/kernel/media/videobuf2/core.rs
 create mode 100644 rust/kernel/media/videobuf2/mod.rs
 create mode 100644 rust/kernel/sync/ffi_mutex.rs
 create mode 100644 samples/rust/rust_v4l2.rs
Signature · 1 line
2.40.0
#24 lore raw
Quoted 8 lines · Deborah Brouwer

Hi Daniel - I don't know if the 'rust branch' is common knowledge but it helped me to know it was this:
https://github.com/Rust-for-Linux/linux

For what it's worth, I was able to get the V4L2 Rust sample probed pretty easily following the quick start instructions
https://docs.kernel.org/rust/quick-start.html

Thanks, it is great to hear that the guide helped! :)

On resources: nowadays we have a webpage, too. Still to be completed, but you may find it useful already: https://rust-for-linux.com

Cheers,
Miguel

#25 lore raw
Quoted 5 lines · Miguel Ojeda

Thanks, it is great to hear that the guide helped! :)

On resources: nowadays we have a webpage, too. Still to be completed, but you may find it useful already: https://rust-for-linux.com

Something that would perhaps be useful is to document (a) what versions of Rust is available for various distributions, or pointers to how to get that information for various distributions. For example, you can get that information from Debian using [1]. It appears that Fedora isn't distributing rustc at *all*, at least according to [2], so apparently for Fedora people will need to install it from source.

[1] https://packages.debian.org/search?keywords=rustc&searchon=names&suite=all&section=all
[2] https://idroot.us/install-rust-fedora-37/

The other thing that would be worth documenting is (b) something about what versions of Rust people have actually tested. The comments at
[3] are quite scary, since per [4], the minimum version of Rustc supported is 1.62.0 --- and per [3], **only** Rust 1.62.0 is supported, since we use unstable Rust features.

[3] https://rust-for-linux.com/rust-version-policy
[4] https://docs.kernel.org/process/changes.html

But for example, with Debian, Debian stable is shipping Rust 1.48.0, and Debian testing (which is currently in "hard freeze" so it can be released as Debian stable this summer) is shipping Rustc 1.63.0.

Since I use Debian testing, the question which is foremost in my mind is whether I can expect to have things work if I use the distro-provided 1.63.0 rustc, or is this really a case of "it's not Rust 1.62.0, so good luck to you"?

If the goal is accelerate adoption of Rustc, and calm people's fears vis-a-vis using Rust, it's not enough to say, "why don't you use the distribution-provided version or Rust"? It would be helpful if those Rust pioneers can share what versions of Rust they have tested against, especially for those commonly used distributions, such as Debian, and give us a report whether we should expect things to work, so we can ignore the scary warning from the build system that we're using an unsupported version of Rust, and if it breaks, we get to keep both pieces.

And for those distributions that don't currently ship Rust, such as Fedora, if someone could build their own unofficial packages, until we can convince Red Hat to start shipping -their own supported Rust compilers, that might be a great way of bridging that gap.

Cheers,

- Ted

#26 lore raw
Quoted 8 lines · Theodore Ts'o

Something that would perhaps be useful is to document (a) what versions of Rust is available for various distributions, or pointers to how to get that information for various distributions. For example, you can get that information from Debian using [1]. It appears that Fedora isn't distributing rustc at *all*, at least according to [2], so apparently for Fedora people will need to install it from source.

As far as I understand, Fedora is actually one of the distributions that provide a very up-to-date Rust toolchain (the latest) and can be installed via dnf [1][2].

Cc'ing Josh Stone who maintains the toolchain in Fedora, just in case there is something I am missing that the webpage may be referring to.

[1] https://packages.fedoraproject.org/pkgs/rust/rust/
[2] https://developer.fedoraproject.org/tech/languages/rust/rust-installation.html

Quoted 5 lines

The other thing that would be worth documenting is (b) something about what versions of Rust people have actually tested. The comments at
[3] are quite scary, since per [4], the minimum version of Rustc supported is 1.62.0 --- and per [3], **only** Rust 1.62.0 is supported, since we use unstable Rust features.

The one that we test, for the moment, is the minimum one (since it is the "only" one) -- I will make it more clear in the webpage.

Quoted 8 lines

But for example, with Debian, Debian stable is shipping Rust 1.48.0, and Debian testing (which is currently in "hard freeze" so it can be released as Debian stable this summer) is shipping Rustc 1.63.0.

Since I use Debian testing, the question which is foremost in my mind is whether I can expect to have things work if I use the distro-provided 1.63.0 rustc, or is this really a case of "it's not Rust 1.62.0, so good luck to you"?

Distro versions should be fine (as in: it is not an issue of "official prebuilt images" vs. "distro binaries"). But short-term you likely need to match the numbered version (or perform small changes in the kernel side when needed). So, in practice, the easiest route is to use the binaries provided by Rust itself (via rustup or standalone installers). We could also provide them at kernel.org like for other toolchains if that helps.

So if distributions want to start using Rust code in their kernels right now (i.e. before we can declare a proper minimum) with their own rustc package, then one approach they can use is to provide an extra rustc-kernel package matching the version required by the kernel (or to change the kernel side a bit to make it compile).

We could, in principle, attempt to support several versions in the kernel side, but given the upstreaming of the first Rust modules is still going on (and the abstractions they depend on), we still have some time. Moreover, the first Rust modules may not be needed by most
distributions: what is being upstreamed is the Android Binder driver, the Asahi GPU driver and the NVMe driver so far.

So it is up to distributions to decide whether they need to use one of those modules early on, and if that is the case and they need to do so before we can declare a minimum, then I think it is reasonable to ask them to match the version. Some particular users, e.g. Android, as far as I understand, they are OK with matching the version for the time being.

In summary, the issue is that the minimum version we will eventually support is "in the future".

Quoted 9 lines

If the goal is accelerate adoption of Rustc, and calm people's fears vis-a-vis using Rust, it's not enough to say, "why don't you use the distribution-provided version or Rust"? It would be helpful if those Rust pioneers can share what versions of Rust they have tested against, especially for those commonly used distributions, such as Debian, and give us a report whether we should expect things to work, so we can ignore the scary warning from the build system that we're using an unsupported version of Rust, and if it breaks, we get to keep both pieces.

Definitely -- we should be testing distro-versions in CI as soon as it is (reasonably) possible. We could even do so already for those distributions that track the latest version, but then it means we will be upgrading more often (every 6 weeks) in the kernel side. There would still be small windows where it may not work depending on how the schedules match, though (but distros could keep a rustc-old for some days until it matches again, for instance).

Quoted 3 lines

Fedora, if someone could build their own unofficial packages, until we can convince Red Hat to start shipping -their own supported Rust compilers, that might be a great way of bridging that gap.

I think all major distributions already ship Rust. From a quick look:
Fedora (1.68.2), Gentoo (1.68.2), Arch (1.68.2), openSUSE (1.68.0 for the rolling one), Ubuntu (1.67.1 for 23.04), Debian (1.63.0 this summer), Red Hat (1.62.1 in 9.1)...

As mentioned above, if kernel maintainers are happy with more frequent upgrades (i.e. tracking the latest release), it would at least be easier for those in distributions that track the latest Rust release -- I would like to do that, in fact, if nobody opposes, since our idea is to anyhow upgrade the compiler required in the kernel until we hit the minimum.

What do you think?

Cheers,
Miguel

#27 lore raw
In reply to Theodore Ts'o
Quoted 7 lines · Theodore Ts'o
Quoted 5 lines · Miguel Ojeda

Thanks, it is great to hear that the guide helped! :)

On resources: nowadays we have a webpage, too. Still to be completed, but you may find it useful already: https://rust-for-linux.com

Quoted 7 lines

Something that would perhaps be useful is to document (a) what versions of Rust is available for various distributions, or pointers to how to get that information for various distributions. For example, you can get that information from Debian using [1]. It appears that Fedora isn't distributing rustc at *all*, at least according to [2], so apparently for Fedora people will need to install it from source.

You can get a list here:
https://repology.org/project/rust/versions

Another alternative is this webpage:
https://pkgs.org/download/rust

Signature · 2 lines
Morten Linderud
PGP: 9C02FF419FECBE16
#28 lore raw
In reply to Theodore Ts'o

Hi Theodore,

Le mardi 11 avril 2023 à 22:58 -0400, Theodore Ts'o a écrit :

Quoted 4 lines

And for those distributions that don't currently ship Rust, such as Fedora, if someone could build their own unofficial packages, until we can convince Red Hat to start shipping -their own supported Rust compilers, that might be a great way of bridging that gap.

Rust can be installed from package on Fedora. I sense a lot of unverified supposition to justify your argument. I don't believe this contribute much to the discussion. It takes about 30s to search on your preferred search engine and find the fact the Fedora ships rustc, and the version is very recent.

regards,
Nicolas

A message that is not in the archive (sent off-list or not yet received): [email protected]

#30 lore raw
Quoted 9 lines · metux IT consult

As already said in my other mail, one major problem IMHO is (recent enough) toolchain availability for the major distros and package build systems - including the embedded ones (ptxdist, buildroot, bitbake, ...).

IMHO, those who want Rust in the kernel, should take care of this first. (and no: asking to download some precompiled binary from somewhere is not any acceptable solution)

Some distributions already provide up-to-date Rust versions compiled by themselves. Those should work if we start tracking the latest version, which is what I discussed above.

You can, of course, do it yourself too and build the compiler yourself. Other that that, if you are not OK with third-party binaries, or binaries we could potentially upload to kernel.org, there is little we can do until the minimum version arrives to your favorite distribution.

Having said that, we still need to declare a minimum version, and for that, extra funding or engineer-hours would be helpful. If your organization/company is up for it, please contact me.

Quoted 3 lines

ACK. Maybe those folks could set up some CIs for at least building and deploying the Rust patches on as many distros as possible - hopefully before they're sent to lkml.

I am unsure what you are asking for. Testing patches for the Rust subsystem? We already do that, of course, and some independent CIs and companies have already started building with Rust some configs (e.g. KernelCI and 0-Day).

If you are concerned about distributions breaking their toolchains, well, sure, we could also test their packages. But it would be best that, instead, distributions looking to support kernel developers set up a test on their side, since they are the ones deciding what to do with their toolchain packages.

I talked with a few distributions' maintainers about this lately, and most of them were very helpful, so they may be interested in supporting kernel developers using their distribution for development. Would that help?

Cheers,
Miguel

#31 lore raw
In reply to Daniel Almeida

Hi Daniel,

Quoted 23 lines · Daniel Almeida

Hi all, this is my first attempt at adding Rust support to the media subsystem.

It adds just enough support to write a clone of the virtio-camera prototype written by my colleague, Dmitry Osipenko, available at [0].

Basically, there's support for video_device_register, v4l2_device_register and for some ioctls in v4l2_ioctl_ops. There is also some initial vb2 support, alongside some wrappers for some types found in videodev2.h.

I wrote a sample Rust driver just to prove that this probes, and that you get a message on dmesg whenever an ioctl is called.

As there is no actual implementation for any of the ioctls, this module can misbehave with some programs. I can work around this in a future submission.

Note that this is based on the rust branch, as opposed to rust-next. The reasoning is simple: I expect this series to just kickstart some discussion around the subject. Actual upstreaming can come up much later, at which point I can rebase on the rust branch.

Quoted 8 lines

Lastly, the origins of this series trace back to a v4l2 virtIO driver I was writing in Rust. As the project was eventually shelved for other reasons, I picked both the virtIO and the v4l2 bindings into their own patches which I am now in the process of submitting. This is to say that I tested this code with said driver and CrosVM in the past, and it worked ok.

Please let me know your thoughts.

I think this could be a good topic to discuss at the upcoming media summit:

https://lore.kernel.org/linux-media/[email protected]/

Please reply to that message with a topic description if you are interested.

One of my main concerns here is time: as subsystem maintainers we can barely keep up with all the incoming patches. Introducing support for a new language would add only more pressure. Even though these are mainly bindings (as I understand it), this would still require that every change to a C kAPI is duplicated in rust, requiring someone to do that work, and have maintainers with enough rust knowledge to verify it.

And just reading about a new programming language is not enough to be able to properly review code: that requires far more experience IMHO. The only way to acquire that experience is by writing non-trivial drivers in rust.

And that in turn takes a lot of time. Which we do not have.

Regards,

Hans

Quoted 48 lines

[0]: https://gitlab.collabora.com/dmitry.osipenko/linux-kernel-rd/-/commit/055a2c322e931a8b388f864f1db81bbdfd525602

Daniel Almeida (6):
rust: media: add the media module
rust: media: add initial videodev2.h abstractions
rust: sync: introduce FfiMutex
rust: media: videobuf2: add a videobuf2 abstraction
rust: media: add {video|v4l2}_device_register support
rust: media: add v4l2 rust sample

 rust/bindings/bindings_helper.h        |   8 +
 rust/kernel/lib.rs                     |   2 +
 rust/kernel/media/mod.rs               |   6 +
 rust/kernel/media/v4l2/capabilities.rs |  80 ++++
 rust/kernel/media/v4l2/dev.rs          | 369 +++++++++++++++
 rust/kernel/media/v4l2/device.rs       | 115 +++++
 rust/kernel/media/v4l2/enums.rs        | 135 ++++++
 rust/kernel/media/v4l2/format.rs       | 178 ++++++++
 rust/kernel/media/v4l2/framesize.rs    | 176 +++++++
 rust/kernel/media/v4l2/inputs.rs       | 104 +++++
 rust/kernel/media/v4l2/ioctls.rs       | 608 +++++++++++++++++++++++++
 rust/kernel/media/v4l2/mmap.rs         |  81 ++++
 rust/kernel/media/v4l2/mod.rs          |  13 +
 rust/kernel/media/videobuf2/core.rs    | 552 ++++++++++++++++++++++
 rust/kernel/media/videobuf2/mod.rs     |   5 +
 rust/kernel/sync.rs                    |   1 +
 rust/kernel/sync/ffi_mutex.rs          |  70 +++
 samples/rust/Kconfig                   |  11 +
 samples/rust/Makefile                  |   1 +
 samples/rust/rust_v4l2.rs              | 403 ++++++++++++++++
 20 files changed, 2918 insertions(+)
 create mode 100644 rust/kernel/media/mod.rs
 create mode 100644 rust/kernel/media/v4l2/capabilities.rs
 create mode 100644 rust/kernel/media/v4l2/dev.rs
 create mode 100644 rust/kernel/media/v4l2/device.rs
 create mode 100644 rust/kernel/media/v4l2/enums.rs
 create mode 100644 rust/kernel/media/v4l2/format.rs
 create mode 100644 rust/kernel/media/v4l2/framesize.rs
 create mode 100644 rust/kernel/media/v4l2/inputs.rs
 create mode 100644 rust/kernel/media/v4l2/ioctls.rs
 create mode 100644 rust/kernel/media/v4l2/mmap.rs
 create mode 100644 rust/kernel/media/v4l2/mod.rs
 create mode 100644 rust/kernel/media/videobuf2/core.rs
 create mode 100644 rust/kernel/media/videobuf2/mod.rs
 create mode 100644 rust/kernel/sync/ffi_mutex.rs
 create mode 100644 samples/rust/rust_v4l2.rs
#32 lore raw
Quoted 7 lines · Hans Verkuil

One of my main concerns here is time: as subsystem maintainers we can barely keep up with all the incoming patches. Introducing support for a new language would add only more pressure. Even though these are mainly bindings (as I understand it), this would still require that every change to a C kAPI is duplicated in rust, requiring someone to do that work, and have maintainers with enough rust knowledge to verify it.

Indeed, that is one of the main costs.

One potential solution is to have somebody step up as the maintainer of the Rust side (e.g. the author of the abstractions).

Of course, that will not make the work go to zero, since there still needs to be some degree of communication even if the new maintainer does all the Rust side work, but it may make it feasible, especially if the abstracted parts of the C API do not change too frequently.

It is also an opportunity for existing maintainers to see how the Rust side would work meanwhile the work gets done, and potentially a chance to get a new maintainer involved with the whole subsystem in the future.

Some subsystems may want to give that maintainer a different MAINTAINERS entry, e.g. as a child subsystem that sends PRs to the main one and may be marked as "experimental". This is also a way to see how the new abstractions work or not, giving maintainers more time to decide whether to commit to a Rust side or not.

I don't mean to say it would be doable for the media subsystem, but please consider it.

Cheers,
Miguel

#33 lore raw

Hi Miguel!

Quoted 24 lines · Miguel Ojeda
Quoted 7 lines · Hans Verkuil

One of my main concerns here is time: as subsystem maintainers we can barely keep up with all the incoming patches. Introducing support for a new language would add only more pressure. Even though these are mainly bindings (as I understand it), this would still require that every change to a C kAPI is duplicated in rust, requiring someone to do that work, and have maintainers with enough rust knowledge to verify it.

Indeed, that is one of the main costs.

One potential solution is to have somebody step up as the maintainer of the Rust side (e.g. the author of the abstractions).

Of course, that will not make the work go to zero, since there still needs to be some degree of communication even if the new maintainer does all the Rust side work, but it may make it feasible, especially if the abstracted parts of the C API do not change too frequently.

It is also an opportunity for existing maintainers to see how the Rust side would work meanwhile the work gets done, and potentially a chance to get a new maintainer involved with the whole subsystem in the future.

Quoted 8 lines

Some subsystems may want to give that maintainer a different MAINTAINERS entry, e.g. as a child subsystem that sends PRs to the main one and may be marked as "experimental". This is also a way to see how the new abstractions work or not, giving maintainers more time to decide whether to commit to a Rust side or not.

I don't mean to say it would be doable for the media subsystem, but please consider it.

This might sound strange, but I suspect that having a TAINT_RUST flag could possibly help maintainers that are already lacking time, because it may quickly allow some of them to ask "please try again without the Rust code to see if the problem is still there", just like happens with out-of-tree code for which the knowledge is limited to null. This could allow to route issue reports to one maintainer when an issue is confirmed in both cases or to another one when it only happens in a single case.

Of course it will not help with code reviews but we know that a great part of maintainers' time it spent trying to analyse problem reports that happen under vague conditions. All the time not spent debugging something not well understood is more time available for reviews.

Just my two cents,
Willy

author reviewer
#34 lore raw

Hans (Verkuil),

Quoted 2 lines

One potential solution is to have somebody step up as the maintainer of the Rust side (e.g. the author of the abstractions).

I'd be willing to step up as maintainer for the Rust V4L2 stuff. I will also be contributing more bindings if we decide that's worthwhile, because right now this is obviously very incomplete.

IIRC, Benjamin was originally looking into Rust for the AV1 driver he's just written in C, but back then there was just too much work to be done to even get started on the task. My point is that maybe these type of drivers - i.e.: m2m codec stuff - are good candidates for Rust, because the codec-specific bits are very self-contained.

-- Daniel

#35 lore raw
In reply to Willy Tarreau
Quoted 9 lines · Willy Tarreau

This might sound strange, but I suspect that having a TAINT_RUST flag could possibly help maintainers that are already lacking time, because it may quickly allow some of them to ask "please try again without the Rust code to see if the problem is still there", just like happens with out-of-tree code for which the knowledge is limited to null. This could allow to route issue reports to one maintainer when an issue is confirmed in both cases or to another one when it only happens in a single case.

Quoted 4 lines

Of course it will not help with code reviews but we know that a great part of maintainers' time it spent trying to analyse problem reports that happen under vague conditions. All the time not spent debugging something not well understood is more time available for reviews.

You can already ask to disable CONFIG_RUST.

In fact, we asked that a few times, when people reported a problem that looked unrelated to Rust, to confirm that was the case and thus redirect the report.

So it is definitely a good idea to ask for that when you get a report with RUST=y and you suspect it may be related to that, especially in the beginning where RUST=y should not be common.

However, I think Rust in-tree code is different to out-of-tree code, since you do have the code, and thus (in general) you should be able to reproduce the build, and you can ask for help to the given maintainers to understand it.

Cheers,
Miguel

#36 lore raw
Quoted 17 lines · Miguel Ojeda
Quoted 9 lines · Willy Tarreau

This might sound strange, but I suspect that having a TAINT_RUST flag could possibly help maintainers that are already lacking time, because it may quickly allow some of them to ask "please try again without the Rust code to see if the problem is still there", just like happens with out-of-tree code for which the knowledge is limited to null. This could allow to route issue reports to one maintainer when an issue is confirmed in both cases or to another one when it only happens in a single case.

Quoted 4 lines

Of course it will not help with code reviews but we know that a great part of maintainers' time it spent trying to analyse problem reports that happen under vague conditions. All the time not spent debugging something not well understood is more time available for reviews.

You can already ask to disable CONFIG_RUST.

Quoted 7 lines

In fact, we asked that a few times, when people reported a problem that looked unrelated to Rust, to confirm that was the case and thus redirect the report.

So it is definitely a good idea to ask for that when you get a report with RUST=y and you suspect it may be related to that, especially in the beginning where RUST=y should not be common.

But if that code is only under a module, there's no need to turn all that code off if it's sufficient to be certain the module was no loaded. Plus it's more friendly to the user who doesn't have to rebuild a kernel, just blacklist a module and check that the kernel doesn't get tainted again.

Quoted 4 lines

However, I think Rust in-tree code is different to out-of-tree code, since you do have the code, and thus (in general) you should be able to reproduce the build, and you can ask for help to the given maintainers to understand it.

It could depend on the layer where it plugs and the level of intimacy with the core. Sometimes you need a deep understanding of all interactions between elements to imagine possible scenarios.

Cheers,
Willy

#37 lore raw
Quoted 6 lines · Willy Tarreau

But if that code is only under a module, there's no need to turn all that code off if it's sufficient to be certain the module was no loaded. Plus it's more friendly to the user who doesn't have to rebuild a kernel, just blacklist a module and check that the kernel doesn't get tainted again.

That could apply to any foreign-to-us subsystems, including C code too. Should we taint per subsystem so that we can easily check for those that we may not trust?

I see one could argue for an experimental taint or making it depend on something like STAGING, i.e. based on grounds of being new code. But I don't see why that should be grounded on just being a different language or not being able to read the code.

Quoted 3 lines

It could depend on the layer where it plugs and the level of intimacy with the core. Sometimes you need a deep understanding of all interactions between elements to imagine possible scenarios.

Please note that the policy for submitting new Rust code is that the respective kernel maintainers and their lists are contacted. We also request that maintainers take the code through their tree if they can, rather than going through the Rust tree, precisely so that maintainers are aware of these potential interactions. See
https://rust-for-linux.com/contributing#the-rust-subsystem for details.

Cheers,
Miguel

#38 lore raw
Quoted 11 lines · Miguel Ojeda
Quoted 6 lines · Willy Tarreau

But if that code is only under a module, there's no need to turn all that code off if it's sufficient to be certain the module was no loaded. Plus it's more friendly to the user who doesn't have to rebuild a kernel, just blacklist a module and check that the kernel doesn't get tainted again.

That could apply to any foreign-to-us subsystems, including C code too. Should we taint per subsystem so that we can easily check for those that we may not trust?

I don't know, maybe that would be a bit too fine. But at least a tainted flag is much less intrusive than forcing a user to rebuild and disable possibly important features that they would only be willing to disable for just a test.

Quoted 2 lines

I see one could argue for an experimental taint or making it depend on something like STAGING, i.e. based on grounds of being new code.

It could also be an idea.

Quoted 3 lines

But
I don't see why that should be grounded on just being a different language or not being able to read the code.

Because being a different language means some maintainers will always have a hard time understanding that code that interacts with their subsystems, even if they try hard. It's exactly the same reason why 25 years ago Linus asked to stop abusing assembly code. If a language is only understood by a subset of developers, by nature it becomes more difficult to maintain in some areas.

Quoted 11 lines
Quoted 3 lines

It could depend on the layer where it plugs and the level of intimacy with the core. Sometimes you need a deep understanding of all interactions between elements to imagine possible scenarios.

Please note that the policy for submitting new Rust code is that the respective kernel maintainers and their lists are contacted. We also request that maintainers take the code through their tree if they can, rather than going through the Rust tree, precisely so that maintainers are aware of these potential interactions. See
https://rust-for-linux.com/contributing#the-rust-subsystem for details.

Sure, but as you said, "if they can". I thought that it could be both elegant, lightweight and convenient. But I'm not trying to sell this idea, just sharing it.

Cheers,
Willy

#39 lore raw
Quoted 5 lines · Willy Tarreau

I don't know, maybe that would be a bit too fine. But at least a tainted flag is much less intrusive than forcing a user to rebuild and disable possibly important features that they would only be willing to disable for just a test.

It may be useful early on to have an easy way to check if any Rust modules got loaded (though note that RUST is not tristate so far, so you would still have something loaded). It could be extra optional output in e.g. lsmod.

However, I don't know why that should imply tainting, especially medium- and long-term -- please see below.

Quoted 5 lines

have a hard time understanding that code that interacts with their subsystems, even if they try hard. It's exactly the same reason why 25 years ago Linus asked to stop abusing assembly code. If a language is only understood by a subset of developers, by nature it becomes more difficult to maintain in some areas.

Yeah, but that is why the idea was that Rust goes first into subsystems where maintainers are willing to put some time into it now and evaluate its merits. That way we also build more Rust expertise across the kernel over time, so that later it is easier for others (e.g. by having examples of API design and drivers, more people to refer to, better tooling...).

But, yes, if Rust grows to be really successful within the kernel, then at some point some basic understanding of Rust will be needed by most kernel developers. I think that is fine, as long as there is enough time to adjust.

Quoted 3 lines

Sure, but as you said, "if they can". I thought that it could be both elegant, lightweight and convenient. But I'm not trying to sell this idea, just sharing it.

To be clear, it is still up to each subsystem to decide whether to take Rust code. What I meant by "if they can" is that, if they are willing to, then ideally the code would go through their tree too. The exception are core APIs, where I asked for flexibility from all sides, so that those subsystems willing to try Rust do not get completely blocked.

Cheers,
Miguel

A message that is not in the archive (sent off-list or not yet received): [email protected]

#41 lore raw
Quoted 2 lines · metux IT consult

The tricky question is: how much time will be needed ?

That depends on how fast Rust grows in the kernel itself, so I would expect it will be a self-regulating feedback loop.

Quoted 1 line

Personally, I'm too overloaded for diving deeper into Rust anytime soon.

That is fine, one can always wait a bit more to see how things evolve.

Now, if somebody wants to use Rust in a subsystem you maintain, then you can always consider letting them maintain that part. As I mentioned above, that can be a quite nice approach to learn Rust on the side and recruit new future maintainers/reviewers.

Quoted 4 lines

Rust and golang share some common problems (when coming from traditional C + friends):
* entirely different toolchain concept (workflows are very different from what one's used from GCC + friends)

I am not sure what you mean, but rustc uses LLVM for codegen, and rustc_codegen_gcc and GCC Rust are coming.

If you mean on the developer UX side, for the kernel at least, you still call make as usual. Of course, some things differ here and there, and there are still things to improve, but it is fairly usable even at this stage.

We are also working on introducing some features that the C side does not have yet. So there can be upsides on this regard, too.

Quoted 2 lines

* fast-moving target (one has to be careful to expect/use the right toolchain version)

This currently applies to the kernel, yes, because we require some unstable features.

To be clear, this is something that we are working on solving, precisely because we know it is not ideal. In any case, the decision made was to go forward meanwhile that got solved, and it is not a blocker for some users/companies.

Quoted 1 line

* rarely understood by traditional kernel devs

Not sure what you mean by this, but a few traditional kernel devs have successfully picked up Rust already and are interested in increasing their usage of it.

Quoted 1 line

* distro/build engine integration/support still pretty infant,

Most distros package Rust as explained above, which may be enough for you depending on the distro you use.

Quoted 2 lines

IMHO, before we can practically use Rust at greater scale in the kernel, the problems above need to be resolved first. And that's something that

That depends on the user/company/entity. For instance, some companies and projects already want to use (or are using) Rust in the kernel, because they control their toolchains.

Quoted 7 lines

And beware: demanding newer toolchains (thus newer distros), just for building the kernel, can easily cause *huge* trouble many organisations, especially in embedded field. Linux is used in lots of highly safety critical environments that need special verification processes and so cannot easily upgrade toolchains. If Linux some day suddenly requires another language like Rust, those would be immediately cut-off from newer releases.

That is fine -- many companies and projects have different requirements, and nobody expects everybody to enable Rust in their kernel nor to make it a hard/unconditional requirement anytime soon (if it ever happens).

Quoted 2 lines

Ergo: the whole process of adding Rust to the Kernel needs to be done very, very carefully.

Indeed -- if you have particular concerns that you think have not been addressed yet in the last 2+ years, please contact us.

Quoted 3 lines

For the reasons above, the subsystems shouldn't take those decisions lightly, even if they happen to be Rust experts - this could have a dramatic effect on downstreams.

There is no effect on downstream, unless they drop support for what they already have. But that is just like any other proposed removal.

Quoted 4 lines

Maybe we should (for certain time) go a different path: move all new Rust stuff (except for bugfixes) to a separate downstream tree, that's rebased on mainline releases, but still let the patches fload through the corresponding subsystems.

That would not accomplish anything except making everything more opaque, not to mention harder for downstream users who are waiting for things to land.

Again, if you have particular concerns, please feel free to raise them, but please note that most of this has been discussed for a long time and decided upon.

Cheers,
Miguel

#42 lore raw
In reply to Miguel Ojeda
Quoted 8 lines · Miguel Ojeda
Quoted 7 lines · Hans Verkuil

One of my main concerns here is time: as subsystem maintainers we can barely keep up with all the incoming patches. Introducing support for a new language would add only more pressure. Even though these are mainly bindings (as I understand it), this would still require that every change to a C kAPI is duplicated in rust, requiring someone to do that work, and have maintainers with enough rust knowledge to verify it.

Another issue is that the V4L2 subsystem is plagued with lifetime management problems. I don't think rust bindings could be safely written for the MC and V4L2 subdev in-kernel APIs at the moment for instance. Sakari recently attempted to fix some of those issues (again), see [1]. Progress is slow on this front because V4L2 is generally understaffed.

[1] https://lore.kernel.org//[email protected]

Now, I hope that mentioning "lifetime management problems" will be enough to nerd-snipe a rust enthusiast or two to help fix the C code in order to implement proper rust bindings on top ;-)

Quoted 4 lines

Indeed, that is one of the main costs.

One potential solution is to have somebody step up as the maintainer of the Rust side (e.g. the author of the abstractions).

That would certainly be a required step, but I don't think it would be enough. On good days I see the media subsystem as barely able to cope with the current load, on bad days it feels it's completely collapsing.

We have homework to do when it comes to maintenance for the media subsystem, we're doing *really* badly at the moment regarding community management and attracting (and retaining) new core contributors. This is a topic I really want to discuss face to face during the media workshop in Prague (and I know that many people are looking forward to that discussion).

Quoted 18 lines

Of course, that will not make the work go to zero, since there still needs to be some degree of communication even if the new maintainer does all the Rust side work, but it may make it feasible, especially if the abstracted parts of the C API do not change too frequently.

It is also an opportunity for existing maintainers to see how the Rust side would work meanwhile the work gets done, and potentially a chance to get a new maintainer involved with the whole subsystem in the future.

Some subsystems may want to give that maintainer a different MAINTAINERS entry, e.g. as a child subsystem that sends PRs to the main one and may be marked as "experimental". This is also a way to see how the new abstractions work or not, giving maintainers more time to decide whether to commit to a Rust side or not.

I don't mean to say it would be doable for the media subsystem, but please consider it.

Signature · 3 lines
Regards,

Laurent Pinchart

A message that is not in the archive (sent off-list or not yet received): [email protected]

#44 lore raw
Quoted 7 lines · metux IT consult
Quoted 3 lines · Laurent Pinchart

We have homework to do when it comes to maintenance for the media subsystem, we're doing *really* badly at the moment regarding community management and attracting (and retaining) new core contributors.

Is this a problem of the subsys (core) itself or individual drivers ?

It's first and foremost a subsystem core issue. Drivers will also have to be fixed, but the core needs to be handled first.

Signature · 3 lines
Regards,

Laurent Pinchart
#45 lore raw
In reply to Laurent Pinchart
Quoted 4 lines · Laurent Pinchart

Now, I hope that mentioning "lifetime management problems" will be enough to nerd-snipe a rust enthusiast or two to help fix the C code in order to implement proper rust bindings on top ;-)

Good idea ;)

I think it is definitely a good opportunity to consider how Rust could fit the new design, and perhaps borrow some ideas from Rust for the new design, even. If you feel like a quick meeting could help on that, please let us know.

Quoted 10 lines

That would certainly be a required step, but I don't think it would be enough. On good days I see the media subsystem as barely able to cope with the current load, on bad days it feels it's completely collapsing.

We have homework to do when it comes to maintenance for the media subsystem, we're doing *really* badly at the moment regarding community management and attracting (and retaining) new core contributors. This is a topic I really want to discuss face to face during the media workshop in Prague (and I know that many people are looking forward to that discussion).

I am sorry to hear that. One idea would be offsetting the extra work by having the Rust person also take care of some of the C parts too. That way you can also potentially get them to be a full maintainer at some point, even if the Rust experiment does not pan out.

Of course, easier said than done, and managing more people always takes extra time, but getting more people seems to be part of the solution anyway, from what you say.

In any case, thanks a lot for at least considering it :)

Cheers,
Miguel

#46 lore raw

Hi Miguel,

(CC'ing Sakari Ailus)

Quoted 6 lines · Miguel Ojeda
Quoted 4 lines · Laurent Pinchart

Now, I hope that mentioning "lifetime management problems" will be enough to nerd-snipe a rust enthusiast or two to help fix the C code in order to implement proper rust bindings on top ;-)

Quoted 6 lines

Good idea ;)

I think it is definitely a good opportunity to consider how Rust could fit the new design, and perhaps borrow some ideas from Rust for the new design, even. If you feel like a quick meeting could help on that, please let us know.

I think we have a fairly good view of what needs to be done, the rules are the same regardless of the programming language and whether the compiler or reviewers enforce them (Hans, Sakari, please feel free to disagree). Thanks for your offer though, it's appreciated.

Quoted 11 lines
Quoted 10 lines

That would certainly be a required step, but I don't think it would be enough. On good days I see the media subsystem as barely able to cope with the current load, on bad days it feels it's completely collapsing.

We have homework to do when it comes to maintenance for the media subsystem, we're doing *really* badly at the moment regarding community management and attracting (and retaining) new core contributors. This is a topic I really want to discuss face to face during the media workshop in Prague (and I know that many people are looking forward to that discussion).

Quoted 4 lines

I am sorry to hear that. One idea would be offsetting the extra work by having the Rust person also take care of some of the C parts too. That way you can also potentially get them to be a full maintainer at some point, even if the Rust experiment does not pan out.

That's certainly something I would consider very positive. If anyone is interested in having a look at (part of) the problem and possible solutions, [1] is the most recent patch series posted to handle some of the lifetime issues, and [2] is a more generic version of part of [1].

[1] https://lore.kernel.org/linux-media/[email protected]/
[2] https://lore.kernel.org/linux-kernel/161117153248.2853729.2452425259045172318.stgit@dwillia2-desk3.amr.corp.intel.com/

Quoted 5 lines

Of course, easier said than done, and managing more people always takes extra time, but getting more people seems to be part of the solution anyway, from what you say.

In any case, thanks a lot for at least considering it :)

Signature · 3 lines
Regards,

Laurent Pinchart
author reviewer
#47 lore raw

Hi,

As I said higher up on this thread, I can maintain the Rust bits and help out with the issues around it.

IMHO, we should at least try this. Who knows, it might work out :)

Laurent, maybe we can take a piecemeal approach? Right now there are no bindings for MC, but I wouldn't complain about fixing some of the C code when the time comes.

Just FYI, I am writing some more bindings, just enough to write a stateless decoder driver. I hope to finish it in time for the media summit. It will give us a more in-depth idea of the pros and cons here.

-- Daniel

#48 lore raw

Hi Daniel,

Quoted 10 lines · Daniel Almeida

Hi,

As I said higher up on this thread, I can maintain the Rust bits and help out with the issues around it.

IMHO, we should at least try this. Who knows, it might work out :)

Laurent, maybe we can take a piecemeal approach? Right now there are no bindings for MC, but I wouldn't complain about fixing some of the C code when the time comes.

The lifetime issues affect plain V4L2 video nodes too I'm afraid :-)

Quoted 3 lines

Just FYI, I am writing some more bindings, just enough to write a stateless decoder driver. I hope to finish it in time for the media summit. It will give us a more in-depth idea of the pros and cons here.

Signature · 3 lines
Regards,

Laurent Pinchart
#49 lore raw

Le mercredi 26 avril 2023 à 20:25 +0300, Laurent Pinchart a écrit :

Quoted 8 lines

Hi Daniel,

Quoted 5 lines · Daniel Almeida

Hi,

As I said higher up on this thread, I can maintain the Rust bits and help out with the issues around it.

Quoted 7 lines
Quoted 5 lines

IMHO, we should at least try this. Who knows, it might work out :)

Laurent, maybe we can take a piecemeal approach? Right now there are no bindings for MC, but I wouldn't complain about fixing some of the C code when the time comes.

The lifetime issues affect plain V4L2 video nodes too I'm afraid :-)

Everything under the bindings is unsafe code, so it does not prevent doing upper implementation and have other things be memory safe. It just make Rust less helpful in some cases (I guess everything across ops).

There is low hanging fruit if some folks are interested. I see legitimate benefit in rewriting in rust the JPEG parser, the H.264 reference list generator, and maybe VP9 probability update lib. AV1 driver will need a lib to reduce duplicates, this could be done straight in Rust (offering a C interface of course, so it does not matter if the users are written in rust or C).

Nicolas

Quoted 10 lines
Quoted 4 lines

Just FYI, I am writing some more bindings, just enough to write a stateless decoder driver. I hope to finish it in time for the media summit. It will give us a more in-depth idea of the pros and cons here.

Signature · 3 lines
Regards,

Laurent Pinchart
#50 lore raw
Quoted 21 lines · Nicolas Dufresne

Le mercredi 26 avril 2023 à 20:25 +0300, Laurent Pinchart a écrit :

Quoted 8 lines

Hi Daniel,

Quoted 5 lines · Daniel Almeida

Hi,

As I said higher up on this thread, I can maintain the Rust bits and help out with the issues around it.

Quoted 7 lines
Quoted 5 lines

IMHO, we should at least try this. Who knows, it might work out :)

Laurent, maybe we can take a piecemeal approach? Right now there are no bindings for MC, but I wouldn't complain about fixing some of the C code when the time comes.

The lifetime issues affect plain V4L2 video nodes too I'm afraid :-)

Everything under the bindings is unsafe code, so it does not prevent doing upper implementation and have other things be memory safe. It just make Rust less helpful in some cases (I guess everything across ops).

Quoted 5 lines

There is low hanging fruit if some folks are interested. I see legitimate benefit in rewriting in rust the JPEG parser, the H.264 reference list generator, and maybe VP9 probability update lib. AV1 driver will need a lib to reduce duplicates, this could be done straight in Rust (offering a C interface of course, so it does not matter if the users are written in rust or C).

Unfortunately I don't think actually replacing the C implementations will be possible until Rust architecture support is on par with C, which probably means waiting until gccrs is ready...

We could have both implementations until then (and only use the C one where Rust doesn't work), but the code duplication has an extra maintenance cost so it's not free. That's why people are mostly focusing on drivers first instead of core code.

~~ Lina

#51 lore raw

Le mardi 02 mai 2023 à 05:17 +0900, Asahi Lina a écrit :

Quoted 28 lines · Nicolas Dufresne
Quoted 21 lines

Le mercredi 26 avril 2023 à 20:25 +0300, Laurent Pinchart a écrit :

Quoted 8 lines

Hi Daniel,

Quoted 5 lines · Daniel Almeida

Hi,

As I said higher up on this thread, I can maintain the Rust bits and help out with the issues around it.

Quoted 7 lines
Quoted 5 lines

IMHO, we should at least try this. Who knows, it might work out :)

Laurent, maybe we can take a piecemeal approach? Right now there are no bindings for MC, but I wouldn't complain about fixing some of the C code when the time comes.

The lifetime issues affect plain V4L2 video nodes too I'm afraid :-)

Everything under the bindings is unsafe code, so it does not prevent doing upper implementation and have other things be memory safe. It just make Rust less helpful in some cases (I guess everything across ops).

Quoted 5 lines

There is low hanging fruit if some folks are interested. I see legitimate benefit in rewriting in rust the JPEG parser, the H.264 reference list generator, and maybe VP9 probability update lib. AV1 driver will need a lib to reduce duplicates, this could be done straight in Rust (offering a C interface of course, so it does not matter if the users are written in rust or C).

Quoted 8 lines

Unfortunately I don't think actually replacing the C implementations will be possible until Rust architecture support is on par with C, which probably means waiting until gccrs is ready...

We could have both implementations until then (and only use the C one where Rust doesn't work), but the code duplication has an extra maintenance cost so it's not free. That's why people are mostly focusing on drivers first instead of core code.

Didn't know that, let's postpone this idea then.

thanks,
Nicolas

Quoted 4 lines

~~ Lina

#52 lore raw
In reply to Asahi Lina
Quoted 4 lines · Asahi Lina

Unfortunately I don't think actually replacing the C implementations will be possible until Rust architecture support is on par with C, which probably means waiting until gccrs is ready...

There is also a second approach via rustc_codegen_gcc: Antoni (Cc'd) showed in Kangrejos and LPC last year that it could compile the Rust kernel code (with a few tweaks).

Cheers,
Miguel

author reviewer
#53 lore raw

Hi Lina, all,

I disagree that we need to wait for anything as a precondition for writing the things Nicolas listed. The reason being that he listed out some very self-contained codebases. These would not depend on the kernel crate either for the most part (or at all, even, but going from memory here..).

Note that the codec library in particular would rarely be touched after it's written, as the algorithms in there are more or less "set in stone" by the codec specs.

Maintaining these until they can be merged would be essentially free, unless I am missing something?

-- Daniel

#54 lore raw
In reply to Laurent Pinchart

Hi Laurent, Miguel,

Quoted 3 lines · Laurent Pinchart

Hi Miguel,

(CC'ing Sakari Ailus)

Thanks for cc'ing me.

Quoted 15 lines · Miguel Ojeda
Quoted 6 lines · Laurent Pinchart
Quoted 4 lines

Now, I hope that mentioning "lifetime management problems" will be enough to nerd-snipe a rust enthusiast or two to help fix the C code in order to implement proper rust bindings on top ;-)

Quoted 6 lines

Good idea ;)

I think it is definitely a good opportunity to consider how Rust could fit the new design, and perhaps borrow some ideas from Rust for the new design, even. If you feel like a quick meeting could help on that, please let us know.

Quoted 4 lines

I think we have a fairly good view of what needs to be done, the rules are the same regardless of the programming language and whether the compiler or reviewers enforce them (Hans, Sakari, please feel free to disagree). Thanks for your offer though, it's appreciated.

I guess on many you don't need to care about lifetime management if you can assume that certain things never go away. Sometimes these assumptions prove incorrect, and that's what's happened here.

Quoted 17 lines
Quoted 12 lines
Quoted 11 lines

That would certainly be a required step, but I don't think it would be enough. On good days I see the media subsystem as barely able to cope with the current load, on bad days it feels it's completely collapsing.

We have homework to do when it comes to maintenance for the media subsystem, we're doing *really* badly at the moment regarding community management and attracting (and retaining) new core contributors. This is a topic I really want to discuss face to face during the media workshop in Prague (and I know that many people are looking forward to that discussion).

Quoted 4 lines

I am sorry to hear that. One idea would be offsetting the extra work by having the Rust person also take care of some of the C parts too. That way you can also potentially get them to be a full maintainer at some point, even if the Rust experiment does not pan out.

Quoted 7 lines

That's certainly something I would consider very positive. If anyone is interested in having a look at (part of) the problem and possible solutions, [1] is the most recent patch series posted to handle some of the lifetime issues, and [2] is a more generic version of part of [1].

[1] https://lore.kernel.org/linux-media/[email protected]/
[2] https://lore.kernel.org/linux-kernel/161117153248.2853729.2452425259045172318.stgit@dwillia2-desk3.amr.corp.intel.com/

Thanks for the pointer, this would be nice indeed.

I haven't had time to work on the media device referencing series, overall it should be good for merging but Hans found an issue I haven't had time to debug yet. I do intend to continue that in the near future though.

Signature · 3 lines
Regards,

Sakari Ailus
#55 lore raw
In reply to Daniel Almeida

Hi Daniel,

Quoted 23 lines · Daniel Almeida

Hi all, this is my first attempt at adding Rust support to the media subsystem.

It adds just enough support to write a clone of the virtio-camera prototype written by my colleague, Dmitry Osipenko, available at [0].

Basically, there's support for video_device_register, v4l2_device_register and for some ioctls in v4l2_ioctl_ops. There is also some initial vb2 support, alongside some wrappers for some types found in videodev2.h.

I wrote a sample Rust driver just to prove that this probes, and that you get a message on dmesg whenever an ioctl is called.

As there is no actual implementation for any of the ioctls, this module can misbehave with some programs. I can work around this in a future submission.

Note that this is based on the rust branch, as opposed to rust-next. The reasoning is simple: I expect this series to just kickstart some discussion around the subject. Actual upstreaming can come up much later, at which point I can rebase on the rust branch.

Quoted 8 lines

Lastly, the origins of this series trace back to a v4l2 virtIO driver I was writing in Rust. As the project was eventually shelved for other reasons, I picked both the virtIO and the v4l2 bindings into their own patches which I am now in the process of submitting. This is to say that I tested this code with said driver and CrosVM in the past, and it worked ok.

Please let me know your thoughts.

Based on our discussions during the Media Summit I am going to mark this series as RFC in patchwork.

Once we have a new maintenance process up and running and things have stabilized on that front, then this can be revisited, but by that time it is better to post a v2, rebased and updated.

Regards,

Hans

Quoted 48 lines

[0]: https://gitlab.collabora.com/dmitry.osipenko/linux-kernel-rd/-/commit/055a2c322e931a8b388f864f1db81bbdfd525602

Daniel Almeida (6):
rust: media: add the media module
rust: media: add initial videodev2.h abstractions
rust: sync: introduce FfiMutex
rust: media: videobuf2: add a videobuf2 abstraction
rust: media: add {video|v4l2}_device_register support
rust: media: add v4l2 rust sample

 rust/bindings/bindings_helper.h        |   8 +
 rust/kernel/lib.rs                     |   2 +
 rust/kernel/media/mod.rs               |   6 +
 rust/kernel/media/v4l2/capabilities.rs |  80 ++++
 rust/kernel/media/v4l2/dev.rs          | 369 +++++++++++++++
 rust/kernel/media/v4l2/device.rs       | 115 +++++
 rust/kernel/media/v4l2/enums.rs        | 135 ++++++
 rust/kernel/media/v4l2/format.rs       | 178 ++++++++
 rust/kernel/media/v4l2/framesize.rs    | 176 +++++++
 rust/kernel/media/v4l2/inputs.rs       | 104 +++++
 rust/kernel/media/v4l2/ioctls.rs       | 608 +++++++++++++++++++++++++
 rust/kernel/media/v4l2/mmap.rs         |  81 ++++
 rust/kernel/media/v4l2/mod.rs          |  13 +
 rust/kernel/media/videobuf2/core.rs    | 552 ++++++++++++++++++++++
 rust/kernel/media/videobuf2/mod.rs     |   5 +
 rust/kernel/sync.rs                    |   1 +
 rust/kernel/sync/ffi_mutex.rs          |  70 +++
 samples/rust/Kconfig                   |  11 +
 samples/rust/Makefile                  |   1 +
 samples/rust/rust_v4l2.rs              | 403 ++++++++++++++++
 20 files changed, 2918 insertions(+)
 create mode 100644 rust/kernel/media/mod.rs
 create mode 100644 rust/kernel/media/v4l2/capabilities.rs
 create mode 100644 rust/kernel/media/v4l2/dev.rs
 create mode 100644 rust/kernel/media/v4l2/device.rs
 create mode 100644 rust/kernel/media/v4l2/enums.rs
 create mode 100644 rust/kernel/media/v4l2/format.rs
 create mode 100644 rust/kernel/media/v4l2/framesize.rs
 create mode 100644 rust/kernel/media/v4l2/inputs.rs
 create mode 100644 rust/kernel/media/v4l2/ioctls.rs
 create mode 100644 rust/kernel/media/v4l2/mmap.rs
 create mode 100644 rust/kernel/media/v4l2/mod.rs
 create mode 100644 rust/kernel/media/videobuf2/core.rs
 create mode 100644 rust/kernel/media/videobuf2/mod.rs
 create mode 100644 rust/kernel/sync/ffi_mutex.rs
 create mode 100644 samples/rust/rust_v4l2.rs