Skip to contentSkip to conversation

Rewrite the VP9 codec library in Rust

RFC PATCH Superseded Daniel Almeida 4 participants 10 messages Feb 27, 2024 – Mar 7, 2024

Subsystems (4)
lore
Share & tools
lore

Daniel Almeida

author reviewer
#0 lore raw

cover Rewrite the VP9 codec library in Rust

Hi everyone,

This patch ports the VP9 library written by Andrzej into Rust as a proof-of-concept. This is so that we can evaluate the Rust in V4L2 initiative with source code in hand.

It converts rkvdec and hantro to use the Rust version of the library. These two were chosen merely because that's the hardware I currently own. Fluster scores remain the same when using either the Rust or C version. Please test this.

Much has been spoken at various occasions about drivers and I feel that the consensus is to wait for now. This is why I am proposing a different approach:
porting our codec libraries to Rust.

IMHO, these components can benefit greatly from Rust, as they implement complicated algorithms that also happen to process data received from userspace through V4L2 controls. These algorithms use the data received from userspace in order to index into a lot of arrays and thus benefit from Rust's memory safety.

The first thing about the code is that it does not include any layer of bindings. This was pointed out as a blocker several times due to the need of keeping them in sync with the C code.

The Rust code here also offers a C API for C drivers.This C API is automatically generated by cbindgen and I have provided instructions on how to do so. We can even use functions from both the C and Rust libraries at the same time since the ABI is the same.

The above can come in handy because it means that we can convert a given codebase piece by piece if need be. C drivers will work as usual through the C API and any new Rust driver will get to benefit from a native Rust interface.

Please take note that most of the code is *not* in the media tree, so I do not see how this can further stress our subsystem. I take responsibility for maintaining stuff in rust/kernel/media and honestly, this library will not need any further updates for the same reason we have never touched its C counterpart.

I hope to convince the community that this is a feasible way to slowly experiment with Rust code without tying us up too much to it.

Lastly, please note that this code is just a proof of concept, we can settle on a proper patch - with all that entails - if it is well received.

Those with hardware can follow the steps below to test this patch:

a) enable Rust (https://www.kernel.org/doc/html/latest/rust/quick-start.html).
Make sure CONFIG_RUST=y.

b) enable one of the converted drivers (CONFIG_VIDEO_HANTRO or CONFIG_VIDEO_ROCKCHIP_VDEC). This will select V4L2_VP9_RS.

c) download the Fluster tool (https://github.com/fluendo/fluster)

d) download the VP9 test suite from libvpx (fluster.py download VP9-TEST-VECTORS)

e) make sure you have a recent version of GStreamer (gst-inspect-1.0 v4l2codecs must not be empty)

f) run the test suite (fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 -ts VP9-TEST-VECTORS)

g) results should be the same both with and without this patch

-- Daniel

Applies on top of:

commit d9c1fae3e5b225f2e45e0bca519f9a2967cd1062

Author: Alice Ryhl <[email protected]>
Date:   Fri Feb 9 11:18:22 2024 +0000

rust: file: add abstraction for poll_table

For those looking for a branch instead:

https://gitlab.collabora.com/dwlsalmeida/for-upstream/-/tree/vp9-rs?ref_type=heads

Daniel Almeida (1):
v4l2-core: rewrite the VP9 library in Rust

 drivers/media/platform/verisilicon/Kconfig    |    2 +-
 .../platform/verisilicon/hantro_g2_vp9_dec.c  |   38 +-
 .../media/platform/verisilicon/hantro_hw.h    |    8 +-
 drivers/media/v4l2-core/Kconfig               |    4 +
 drivers/staging/media/rkvdec/Kconfig          |    2 +-
 drivers/staging/media/rkvdec/rkvdec-vp9.c     |   52 +-
 include/media/v4l2-vp9-rs.h                   |   97 +
 rust/bindings/bindings_helper.h               |    1 +
 rust/kernel/lib.rs                            |    2 +
 rust/kernel/media.rs                          |    5 +
 rust/kernel/media/v4l2_core.rs                |    6 +
 rust/kernel/media/v4l2_core/cbindgen.toml     |   26 +
 rust/kernel/media/v4l2_core/vp9.rs            | 1999 +++++++++++++++++
 13 files changed, 2192 insertions(+), 50 deletions(-)
 create mode 100644 include/media/v4l2-vp9-rs.h
 create mode 100644 rust/kernel/media.rs
 create mode 100644 rust/kernel/media/v4l2_core.rs
 create mode 100644 rust/kernel/media/v4l2_core/cbindgen.toml
 create mode 100644 rust/kernel/media/v4l2_core/vp9.rs
Signature · 1 line
2.43.0
author reviewer
#1 lore raw

RFC PATCH 1/1 v4l2-core: rewrite the VP9 library in Rust

---
 drivers/media/platform/verisilicon/Kconfig    |    2 +-
 .../platform/verisilicon/hantro_g2_vp9_dec.c  |   38 +-
 .../media/platform/verisilicon/hantro_hw.h    |    8 +-
 drivers/media/v4l2-core/Kconfig               |    4 +
 drivers/staging/media/rkvdec/Kconfig          |    2 +-
 drivers/staging/media/rkvdec/rkvdec-vp9.c     |   52 +-
 include/media/v4l2-vp9-rs.h                   |   97 +
 rust/bindings/bindings_helper.h               |    1 +
 rust/kernel/lib.rs                            |    2 +
 rust/kernel/media.rs                          |    5 +
 rust/kernel/media/v4l2_core.rs                |    6 +
 rust/kernel/media/v4l2_core/cbindgen.toml     |   26 +
 rust/kernel/media/v4l2_core/vp9.rs            | 1999 +++++++++++++++++
 13 files changed, 2192 insertions(+), 50 deletions(-)
 create mode 100644 include/media/v4l2-vp9-rs.h
 create mode 100644 rust/kernel/media.rs
 create mode 100644 rust/kernel/media/v4l2_core.rs
 create mode 100644 rust/kernel/media/v4l2_core/cbindgen.toml
 create mode 100644 rust/kernel/media/v4l2_core/vp9.rs
drivers/media/platform/verisilicon/Kconfig +1 -1
index 24b927d8f182..d8e4e9b5de2f 100644
@@ -12,7 +12,7 @@ config VIDEO_HANTRO
 	select VIDEOBUF2_VMALLOC
 	select V4L2_MEM2MEM_DEV
 	select V4L2_H264
-	select V4L2_VP9
+	select V4L2_VP9_RS
 	help
 	  Support for the Hantro IP based Video Processing Units present on
 	  Rockchip and NXP i.MX8M SoCs, which accelerate video and image
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c +20 -18
index 342e543dee4c..eabde1bb326a 100644
@@ -10,7 +10,7 @@
 #include <linux/kernel.h>
 #include <linux/vmalloc.h>
 #include <media/v4l2-mem2mem.h>
-#include <media/v4l2-vp9.h>
+#include <media/v4l2-vp9-rs.h>
 
 #include "hantro.h"
 #include "hantro_vp9.h"
@@ -63,7 +63,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
 	 *	frame_context_idx = 0
 	 * }
 	 */
-	fctx_idx = v4l2_vp9_reset_frame_ctx(*dec_params, vp9_ctx->frame_context);
+	fctx_idx = v4l2_vp9_reset_frame_ctx_rs(*dec_params, &vp9_ctx->frame_context);
 	vp9_ctx->cur.frame_context_idx = fctx_idx;
 
 	/* 6.1 frame(sz): load_probs() and load_probs2() */
@@ -83,7 +83,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
 	 * fw_update_probs() performs actual probs updates or leaves probs as-is
 	 * for values for which a zero was passed from userspace.
 	 */
-	v4l2_vp9_fw_update_probs(&vp9_ctx->probability_tables, prob_updates, *dec_params);
+	v4l2_vp9_fw_update_probs_rs(&vp9_ctx->probability_tables, prob_updates, *dec_params);
 
 	return 0;
 }
@@ -422,7 +422,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 		if (segment_enabled) {
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
-			if (v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+			if (v4l2_vp9_seg_feat_enabled_rs(vp9_ctx->feature_enabled, feat_id, segid))
 				feat_val = feat_val_clip3(feat_val,
 							  vp9_ctx->feature_data[segid][feat_id],
 							  absolute, 255);
@@ -435,7 +435,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 		if (segment_enabled) {
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
-			if (v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+			if (v4l2_vp9_seg_feat_enabled_rs(vp9_ctx->feature_enabled, feat_id, segid))
 				feat_val = feat_val_clip3(feat_val,
 							  vp9_ctx->feature_data[segid][feat_id],
 							  absolute, 63);
@@ -449,7 +449,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
 			if (!(dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME) &&
-			    v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+			    v4l2_vp9_seg_feat_enabled_rs(vp9_ctx->feature_enabled, feat_id, segid))
 				feat_val = vp9_ctx->feature_data[segid][feat_id] + 1;
 		}
 		hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
@@ -460,7 +460,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 		if (segment_enabled) {
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
-			feat_val = v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled,
+			feat_val = v4l2_vp9_seg_feat_enabled_rs(vp9_ctx->feature_enabled,
 							     feat_id, segid) ? 1 : 0;
 		}
 		hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
@@ -637,17 +637,17 @@ static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fram
 	struct hantro_g2_probs *adaptive;
 	struct hantro_g2_mv_probs *mv;
 	const struct v4l2_vp9_segmentation *seg = &dec_params->seg;
-	const struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+	const struct v4l2_vp9_frame_context_rs *probs = &vp9_ctx->probability_tables;
 	int i, j, k, l, m;
 
 	for (i = 0; i < ARRAY_SIZE(all_probs->kf_y_mode_prob); ++i)
 		for (j = 0; j < ARRAY_SIZE(all_probs->kf_y_mode_prob[0]); ++j) {
 			memcpy(all_probs->kf_y_mode_prob[i][j],
-			       v4l2_vp9_kf_y_mode_prob[i][j],
+			       v4l2_vp9_kf_y_mode_prob_rs[i][j],
 			       ARRAY_SIZE(all_probs->kf_y_mode_prob[i][j]));
 
 			all_probs->kf_y_mode_prob_tail[i][j][0] =
-				v4l2_vp9_kf_y_mode_prob[i][j][8];
+				v4l2_vp9_kf_y_mode_prob_rs[i][j][8];
 		}
 
 	memcpy(all_probs->mb_segment_tree_probs, seg->tree_probs,
@@ -657,10 +657,10 @@ static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fram
 	       sizeof(all_probs->segment_pred_probs));
 
 	for (i = 0; i < ARRAY_SIZE(all_probs->kf_uv_mode_prob); ++i) {
-		memcpy(all_probs->kf_uv_mode_prob[i], v4l2_vp9_kf_uv_mode_prob[i],
+		memcpy(all_probs->kf_uv_mode_prob[i], v4l2_vp9_kf_uv_mode_prob_rs[i],
 		       ARRAY_SIZE(all_probs->kf_uv_mode_prob[i]));
 
-		all_probs->kf_uv_mode_prob_tail[i][0] = v4l2_vp9_kf_uv_mode_prob[i][8];
+		all_probs->kf_uv_mode_prob_tail[i][0] = v4l2_vp9_kf_uv_mode_prob_rs[i][8];
 	}
 
 	adaptive = &all_probs->probs;
@@ -692,8 +692,8 @@ static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fram
 	}
 
 	for (i = 0; i < ARRAY_SIZE(adaptive->partition[0]); ++i) {
-		memcpy(adaptive->partition[0][i], v4l2_vp9_kf_partition_probs[i],
-		       sizeof(v4l2_vp9_kf_partition_probs[i]));
+		memcpy(adaptive->partition[0][i], v4l2_vp9_kf_partition_probs_rs[i],
+		       sizeof(v4l2_vp9_kf_partition_probs_rs[i]));
 
 		adaptive->partition[0][i][3] = 0;
 	}
@@ -933,7 +933,7 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 
 	if (!(vp9_ctx->cur.flags & V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE)) {
 		/* error_resilient_mode == 0 && frame_parallel_decoding_mode == 0 */
-		struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+		struct v4l2_vp9_frame_context_rs *probs = &vp9_ctx->probability_tables;
 		bool frame_is_intra = vp9_ctx->cur.flags &
 		    (V4L2_VP9_FRAME_FLAG_KEY_FRAME | V4L2_VP9_FRAME_FLAG_INTRA_ONLY);
 		struct tx_and_skip {
@@ -942,7 +942,7 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 			u8 tx32[2][3];
 			u8 skip[3];
 		} _tx_skip, *tx_skip = &_tx_skip;
-		struct v4l2_vp9_frame_symbol_counts *counts;
+		struct v4l2_vp9_frame_symbol_counts_rs *counts;
 		struct symbol_counts *hantro_cnts;
 		u32 tx16p[2][4];
 		int i;
@@ -968,7 +968,8 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 		}
 		counts->tx16p = &tx16p;
 
-		v4l2_vp9_adapt_coef_probs(probs, counts,
+		v4l2_vp9_adapt_coef_probs_rs(probs,
+								     counts,
 					  !vp9_ctx->last.valid ||
 					  vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME,
 					  frame_is_intra);
@@ -984,7 +985,8 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 				mv_mode[i][3] = hantro_cnts->inter_mode_counts[i][2][1];
 			}
 			counts->mv_mode = &mv_mode;
-			v4l2_vp9_adapt_noncoef_probs(&vp9_ctx->probability_tables, counts,
+			v4l2_vp9_adapt_noncoef_probs_rs(&vp9_ctx->probability_tables,
+										   counts,
 						     vp9_ctx->cur.reference_mode,
 						     vp9_ctx->cur.interpolation_filter,
 						     vp9_ctx->cur.tx_mode, vp9_ctx->cur.flags);
drivers/media/platform/verisilicon/hantro_hw.h +4 -4
index 9aec8a79acdc..f8baa898ef53 100644
@@ -12,7 +12,7 @@
 #include <linux/interrupt.h>
 #include <linux/v4l2-controls.h>
 #include <media/v4l2-ctrls.h>
-#include <media/v4l2-vp9.h>
+#include <media/v4l2-vp9-rs.h>
 #include <media/videobuf2-core.h>
 
 #include "rockchip_av1_entropymode.h"
@@ -231,9 +231,9 @@ struct hantro_vp9_dec_hw_ctx {
 	struct hantro_aux_buf tile_edge;
 	struct hantro_aux_buf segment_map;
 	struct hantro_aux_buf misc;
-	struct v4l2_vp9_frame_symbol_counts cnts;
-	struct v4l2_vp9_frame_context probability_tables;
-	struct v4l2_vp9_frame_context frame_context[4];
+	struct v4l2_vp9_frame_symbol_counts_rs cnts;
+	struct v4l2_vp9_frame_context_rs probability_tables;
+	struct v4l2_vp9_frame_context_rs frame_context[4];
 	struct hantro_vp9_frame_info cur;
 	struct hantro_vp9_frame_info last;
 
drivers/media/v4l2-core/Kconfig +4 -0
index 331b8e535e5b..f0e3a25db1f2 100644
@@ -48,6 +48,10 @@ config V4L2_H264
 config V4L2_VP9
 	tristate
 
+# Used by drivers that need the Rust equivalent to v4l2-vp9.ko
+config V4L2_VP9_RS
+	tristate
+
 # Used by drivers that need v4l2-mem2mem.ko
 config V4L2_MEM2MEM_DEV
 	tristate
drivers/staging/media/rkvdec/Kconfig +1 -1
index 5f3bdd848a2c..245dc47082d4 100644
@@ -8,7 +8,7 @@ config VIDEO_ROCKCHIP_VDEC
 	select VIDEOBUF2_VMALLOC
 	select V4L2_MEM2MEM_DEV
 	select V4L2_H264
-	select V4L2_VP9
+	select V4L2_VP9_RS
 	help
 	  Support for the Rockchip Video Decoder IP present on Rockchip SoCs,
 	  which accelerates video decoding.
drivers/staging/media/rkvdec/rkvdec-vp9.c +26 -26

171 lines not on this page · raw message

include/media/v4l2-vp9-rs.h new file +97 -0

97 lines not on this page · raw message

rust/bindings/bindings_helper.h +1 -0
index c651d38e5dd6..25e2c06bb967 100644
@@ -25,6 +25,7 @@
 #include <linux/sched.h>
 #include <linux/task_work.h>
 #include <linux/workqueue.h>
+#include <linux/v4l2-controls.h>
 
 /* `bindgen` gets confused at certain things. */
 const size_t RUST_CONST_HELPER_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN;
rust/kernel/lib.rs +2 -0
index cc39195b0157..724e1d3bf2af 100644
@@ -39,6 +39,8 @@
 pub mod ioctl;
 #[cfg(CONFIG_KUNIT)]
 pub mod kunit;
+#[cfg(CONFIG_MEDIA_SUPPORT)]
+pub mod media;
 #[cfg(CONFIG_NET)]
 pub mod net;
 pub mod prelude;
rust/kernel/media.rs new file +5 -0
new file mode 100644
index 000000000000..72f5cd716874
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Media support
+
+pub mod v4l2_core;
\ No newline at end of file
rust/kernel/media/v4l2_core.rs new file +6 -0
new file mode 100644
index 000000000000..7515fe748a65
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! V4L2 Core
+
+#[cfg(CONFIG_V4L2_VP9_RS)]
+pub mod vp9;
\ No newline at end of file
rust/kernel/media/v4l2_core/cbindgen.toml new file +26 -0
new file mode 100644
index 000000000000..ebbc8a5e2fff
@@ -0,0 +1,26 @@
+# Generate the C API with:
+# cbindgen  --lang c  -v --config rust/kernel/media/v4l2_core/cbindgen.toml --output include/media/v4l2-vp9-rs.h -- rust/kernel/media/v4l2_core/vp9.rs
+
+language = "C"
+
+# No <stdarg.h>, <stdbool.h>, <stdint.h>, <stdlib.h>, <uchar.h>
+no_includes = true
+style = "tag"
+
+sys_includes = ["media/v4l2-ctrls.h"]
+autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
+usize_is_size_t = true
+include_guard = "V4L2_VP9_RS_H"
+
+# Rename everything from bindgen
+# See https://github.com/mozilla/cbindgen/issues/621
+# Also rename our mangled structs into snake_case for the C API
+[export.rename]
+"v4l2_ctrl_vp9_compressed_hdr" = "struct v4l2_ctrl_vp9_compressed_hdr"
+"v4l2_ctrl_vp9_frame" = "struct v4l2_ctrl_vp9_frame"
+"FrameContext" = "v4l2_vp9_frame_context_rs"
+"FrameSymbolCounts" = "v4l2_vp9_frame_symbol_counts_rs"
+"FrameMvContext" = "v4l2_vp9_frame_mv_context_rs"
+
+[enum]
+prefix_with_name = true
rust/kernel/media/v4l2_core/vp9.rs new file +1999 -0

1,999 lines not on this page · raw message

Signature · 1 line
2.43.0
reviewer
#2 lore raw

Re: [RFC PATCH 1/1] v4l2-core: rewrite the VP9 library in Rust

rust/kernel/media/v4l2_core/vp9.rs line 1474

Quoted 11 lines · Daniel Almeida
rust/kernel/media/v4l2_core/vp9.rs new file +10 -0
+#[no_mangle]
+pub extern "C" fn v4l2_vp9_seg_feat_enabled_rs(
+    feature_enabled: &mut u8,
+    feature: u32,
+    segid: u32,
+) -> bool {
+    let feature_enabled = unsafe { core::slice::from_raw_parts_mut(feature_enabled, 8) };
+    let mask = 1 << feature;
+    feature_enabled[segid as usize] & mask != 0
+}

This seems to be the only unsafe block. Impressive!

I recommend taking an *mut u8 here instead of &mut u8 if you're going to use slice::from_raw_parts_mut. Or, if cbindgen allows it, take an &mut [u8; 8] instead and eliminate the unsafe block.

Alice

author reviewer
#3 lore raw

Re: [RFC PATCH 1/1] v4l2-core: rewrite the VP9 library in Rust

Hi Alice!

Quoted 3 lines · Alice Ryhl

I recommend taking an *mut u8 here instead of &mut u8 if you're going to use slice::from_raw_parts_mut. Or, if cbindgen allows it, take an &mut [u8; 8] instead and eliminate the unsafe block.

You're right, the signature here can change to &mut [u8; 8] instead. This means no unsafe blocks at all.

Thanks for the tip,

-- Daniel

author reviewer
#4 lore raw

v2 cover Rewrite parts of rkvdec driver and the VP9 codec library in Rust

In reply to Daniel Almeida

Hi Mauro, Hans,

While working on v1 for this patchset, I realized that we can go further by converting the error-prone sections of our codec drivers to Rust. This also does not need any bindings in order to work.

As yet another proof-of-concept, I have converted parts of the rkvdec driver. Refer to instructions in v1 to test this.

Notice how:

1) many problematic memcpy's go away, these become a simple assignment in Rust.

2) it can interop seamlessly with the code in rkvdec-vp9.c that was already converted in v1 of this series.

3) it can use the Rust version of seg_feat_enabled directly in vp9.rs, while also using the C APIs from the v4l2-vp9-rs library in rkvdec-vp9.c

4) more modern things become available for the programmer, like iterators and their methods without a performance penalty.

I want to propose the following:

Let's merge a non-RFC version of this series and gate it behind some kconfigs so that we can switch between the C and Rust implementations. Users get the C version by default, while we continuously test the Rust components on a CI for a few months. This will hopefully be enough time until the next Media Summit.

My aim is to eventually deprecate the C parts once we're confident that the Rust code is stable enough. I will keep my own tree, and send PRs to the media tree if a rebase or fix is needed.

I believe this will not be disruptive nor require any extra work from anyone but me.

-- Daniel

Again, applies on top of:

commit d9c1fae3e5b225f2e45e0bca519f9a2967cd1062

Author: Alice Ryhl <[email protected]>
Date:   Fri Feb 9 11:18:22 2024 +0000

rust: file: add abstraction for poll_table

For those looking for a branch instead: https://gitlab.collabora.com/dwlsalmeida/for-upstream/-/tree/vp9-rs-rkvdec?ref_type=heads

Daniel Almeida (2):
v4l2-core: rewrite the VP9 library in Rust
media: rkvdec: rewrite parts of the driver in Rust

 drivers/media/platform/verisilicon/Kconfig    |    2 +-
 .../platform/verisilicon/hantro_g2_vp9_dec.c  |   38 +-
 .../media/platform/verisilicon/hantro_hw.h    |    8 +-
 drivers/media/v4l2-core/Kconfig               |    5 +
 drivers/staging/media/rkvdec/Kconfig          |    3 +-
 drivers/staging/media/rkvdec/Makefile         |    2 +-
 drivers/staging/media/rkvdec/cbindgen.toml    |   36 +
 drivers/staging/media/rkvdec/common.rs        |   19 +
 drivers/staging/media/rkvdec/regs.rs          |  237 ++
 drivers/staging/media/rkvdec/rkvdec-vp9.c     |  607 +----
 drivers/staging/media/rkvdec/rkvdec_rs.h      |  125 +
 drivers/staging/media/rkvdec/rkvdec_rs.rs     |   14 +
 drivers/staging/media/rkvdec/vp9.rs           |  636 +++++
 include/media/v4l2-vp9-rs.h                   |   99 +
 rust/bindings/bindings_helper.h               |    1 +
 rust/helpers.c                                |    7 +
 rust/kernel/lib.rs                            |    2 +
 rust/kernel/media.rs                          |    5 +
 rust/kernel/media/v4l2_core.rs                |    6 +
 rust/kernel/media/v4l2_core/cbindgen.toml     |   26 +
 rust/kernel/media/v4l2_core/vp9.rs            | 2053 +++++++++++++++++
 21 files changed, 3415 insertions(+), 516 deletions(-)
 create mode 100644 drivers/staging/media/rkvdec/cbindgen.toml
 create mode 100644 drivers/staging/media/rkvdec/common.rs
 create mode 100644 drivers/staging/media/rkvdec/regs.rs
 create mode 100644 drivers/staging/media/rkvdec/rkvdec_rs.h
 create mode 100644 drivers/staging/media/rkvdec/rkvdec_rs.rs
 create mode 100644 drivers/staging/media/rkvdec/vp9.rs
 create mode 100644 include/media/v4l2-vp9-rs.h
 create mode 100644 rust/kernel/media.rs
 create mode 100644 rust/kernel/media/v4l2_core.rs
 create mode 100644 rust/kernel/media/v4l2_core/cbindgen.toml
 create mode 100644 rust/kernel/media/v4l2_core/vp9.rs
Signature · 1 line
2.43.0
author reviewer
#5 lore raw

RFC PATCH v2 1/2 v4l2-core: rewrite the VP9 library in Rust

---
 drivers/media/platform/verisilicon/Kconfig    |    2 +-
 .../platform/verisilicon/hantro_g2_vp9_dec.c  |   38 +-
 .../media/platform/verisilicon/hantro_hw.h    |    8 +-
 drivers/media/v4l2-core/Kconfig               |    5 +
 drivers/staging/media/rkvdec/Kconfig          |    2 +-
 drivers/staging/media/rkvdec/rkvdec-vp9.c     |   52 +-
 include/media/v4l2-vp9-rs.h                   |   99 +
 rust/bindings/bindings_helper.h               |    1 +
 rust/kernel/lib.rs                            |    2 +
 rust/kernel/media.rs                          |    5 +
 rust/kernel/media/v4l2_core.rs                |    6 +
 rust/kernel/media/v4l2_core/cbindgen.toml     |   26 +
 rust/kernel/media/v4l2_core/vp9.rs            | 2053 +++++++++++++++++
 13 files changed, 2249 insertions(+), 50 deletions(-)
 create mode 100644 include/media/v4l2-vp9-rs.h
 create mode 100644 rust/kernel/media.rs
 create mode 100644 rust/kernel/media/v4l2_core.rs
 create mode 100644 rust/kernel/media/v4l2_core/cbindgen.toml
 create mode 100644 rust/kernel/media/v4l2_core/vp9.rs
drivers/media/platform/verisilicon/Kconfig +1 -1
index 24b927d8f182..d8e4e9b5de2f 100644
@@ -12,7 +12,7 @@ config VIDEO_HANTRO
 	select VIDEOBUF2_VMALLOC
 	select V4L2_MEM2MEM_DEV
 	select V4L2_H264
-	select V4L2_VP9
+	select V4L2_VP9_RS
 	help
 	  Support for the Hantro IP based Video Processing Units present on
 	  Rockchip and NXP i.MX8M SoCs, which accelerate video and image
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c +20 -18
index 342e543dee4c..43990b550919 100644
@@ -10,7 +10,7 @@
 #include <linux/kernel.h>
 #include <linux/vmalloc.h>
 #include <media/v4l2-mem2mem.h>
-#include <media/v4l2-vp9.h>
+#include <media/v4l2-vp9-rs.h>
 
 #include "hantro.h"
 #include "hantro_vp9.h"
@@ -63,7 +63,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
 	 *	frame_context_idx = 0
 	 * }
 	 */
-	fctx_idx = v4l2_vp9_reset_frame_ctx(*dec_params, vp9_ctx->frame_context);
+	fctx_idx = v4l2_vp9_reset_frame_ctx_rs(*dec_params, &vp9_ctx->frame_context);
 	vp9_ctx->cur.frame_context_idx = fctx_idx;
 
 	/* 6.1 frame(sz): load_probs() and load_probs2() */
@@ -83,7 +83,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
 	 * fw_update_probs() performs actual probs updates or leaves probs as-is
 	 * for values for which a zero was passed from userspace.
 	 */
-	v4l2_vp9_fw_update_probs(&vp9_ctx->probability_tables, prob_updates, *dec_params);
+	v4l2_vp9_fw_update_probs_rs(&vp9_ctx->probability_tables, prob_updates, *dec_params);
 
 	return 0;
 }
@@ -422,7 +422,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 		if (segment_enabled) {
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
-			if (v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+			if (v4l2_vp9_seg_feat_enabled_rs(&vp9_ctx->feature_enabled, feat_id, segid))
 				feat_val = feat_val_clip3(feat_val,
 							  vp9_ctx->feature_data[segid][feat_id],
 							  absolute, 255);
@@ -435,7 +435,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 		if (segment_enabled) {
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
-			if (v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+			if (v4l2_vp9_seg_feat_enabled_rs(&vp9_ctx->feature_enabled, feat_id, segid))
 				feat_val = feat_val_clip3(feat_val,
 							  vp9_ctx->feature_data[segid][feat_id],
 							  absolute, 63);
@@ -449,7 +449,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
 			if (!(dec_params->flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME) &&
-			    v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled, feat_id, segid))
+			    v4l2_vp9_seg_feat_enabled_rs(&vp9_ctx->feature_enabled, feat_id, segid))
 				feat_val = vp9_ctx->feature_data[segid][feat_id] + 1;
 		}
 		hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
@@ -460,7 +460,7 @@ static void config_segment(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fr
 		if (segment_enabled) {
 			if (update_data)
 				update_feat_and_flag(vp9_ctx, seg, feat_id, segid);
-			feat_val = v4l2_vp9_seg_feat_enabled(vp9_ctx->feature_enabled,
+			feat_val = v4l2_vp9_seg_feat_enabled_rs(&vp9_ctx->feature_enabled,
 							     feat_id, segid) ? 1 : 0;
 		}
 		hantro_reg_write(ctx->dev, &seg_regs[segid][feat_id], feat_val);
@@ -637,17 +637,17 @@ static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fram
 	struct hantro_g2_probs *adaptive;
 	struct hantro_g2_mv_probs *mv;
 	const struct v4l2_vp9_segmentation *seg = &dec_params->seg;
-	const struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+	const struct v4l2_vp9_frame_context_rs *probs = &vp9_ctx->probability_tables;
 	int i, j, k, l, m;
 
 	for (i = 0; i < ARRAY_SIZE(all_probs->kf_y_mode_prob); ++i)
 		for (j = 0; j < ARRAY_SIZE(all_probs->kf_y_mode_prob[0]); ++j) {
 			memcpy(all_probs->kf_y_mode_prob[i][j],
-			       v4l2_vp9_kf_y_mode_prob[i][j],
+			       v4l2_vp9_kf_y_mode_prob_rs[i][j],
 			       ARRAY_SIZE(all_probs->kf_y_mode_prob[i][j]));
 
 			all_probs->kf_y_mode_prob_tail[i][j][0] =
-				v4l2_vp9_kf_y_mode_prob[i][j][8];
+				v4l2_vp9_kf_y_mode_prob_rs[i][j][8];
 		}
 
 	memcpy(all_probs->mb_segment_tree_probs, seg->tree_probs,
@@ -657,10 +657,10 @@ static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fram
 	       sizeof(all_probs->segment_pred_probs));
 
 	for (i = 0; i < ARRAY_SIZE(all_probs->kf_uv_mode_prob); ++i) {
-		memcpy(all_probs->kf_uv_mode_prob[i], v4l2_vp9_kf_uv_mode_prob[i],
+		memcpy(all_probs->kf_uv_mode_prob[i], v4l2_vp9_kf_uv_mode_prob_rs[i],
 		       ARRAY_SIZE(all_probs->kf_uv_mode_prob[i]));
 
-		all_probs->kf_uv_mode_prob_tail[i][0] = v4l2_vp9_kf_uv_mode_prob[i][8];
+		all_probs->kf_uv_mode_prob_tail[i][0] = v4l2_vp9_kf_uv_mode_prob_rs[i][8];
 	}
 
 	adaptive = &all_probs->probs;
@@ -692,8 +692,8 @@ static void config_probs(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_fram
 	}
 
 	for (i = 0; i < ARRAY_SIZE(adaptive->partition[0]); ++i) {
-		memcpy(adaptive->partition[0][i], v4l2_vp9_kf_partition_probs[i],
-		       sizeof(v4l2_vp9_kf_partition_probs[i]));
+		memcpy(adaptive->partition[0][i], v4l2_vp9_kf_partition_probs_rs[i],
+		       sizeof(v4l2_vp9_kf_partition_probs_rs[i]));
 
 		adaptive->partition[0][i][3] = 0;
 	}
@@ -933,7 +933,7 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 
 	if (!(vp9_ctx->cur.flags & V4L2_VP9_FRAME_FLAG_PARALLEL_DEC_MODE)) {
 		/* error_resilient_mode == 0 && frame_parallel_decoding_mode == 0 */
-		struct v4l2_vp9_frame_context *probs = &vp9_ctx->probability_tables;
+		struct v4l2_vp9_frame_context_rs *probs = &vp9_ctx->probability_tables;
 		bool frame_is_intra = vp9_ctx->cur.flags &
 		    (V4L2_VP9_FRAME_FLAG_KEY_FRAME | V4L2_VP9_FRAME_FLAG_INTRA_ONLY);
 		struct tx_and_skip {
@@ -942,7 +942,7 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 			u8 tx32[2][3];
 			u8 skip[3];
 		} _tx_skip, *tx_skip = &_tx_skip;
-		struct v4l2_vp9_frame_symbol_counts *counts;
+		struct v4l2_vp9_frame_symbol_counts_rs *counts;
 		struct symbol_counts *hantro_cnts;
 		u32 tx16p[2][4];
 		int i;
@@ -968,7 +968,8 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 		}
 		counts->tx16p = &tx16p;
 
-		v4l2_vp9_adapt_coef_probs(probs, counts,
+		v4l2_vp9_adapt_coef_probs_rs(probs,
+								     counts,
 					  !vp9_ctx->last.valid ||
 					  vp9_ctx->last.flags & V4L2_VP9_FRAME_FLAG_KEY_FRAME,
 					  frame_is_intra);
@@ -984,7 +985,8 @@ void hantro_g2_vp9_dec_done(struct hantro_ctx *ctx)
 				mv_mode[i][3] = hantro_cnts->inter_mode_counts[i][2][1];
 			}
 			counts->mv_mode = &mv_mode;
-			v4l2_vp9_adapt_noncoef_probs(&vp9_ctx->probability_tables, counts,
+			v4l2_vp9_adapt_noncoef_probs_rs(&vp9_ctx->probability_tables,
+										   counts,
 						     vp9_ctx->cur.reference_mode,
 						     vp9_ctx->cur.interpolation_filter,
 						     vp9_ctx->cur.tx_mode, vp9_ctx->cur.flags);
drivers/media/platform/verisilicon/hantro_hw.h +4 -4
index 9aec8a79acdc..f8baa898ef53 100644
@@ -12,7 +12,7 @@
 #include <linux/interrupt.h>
 #include <linux/v4l2-controls.h>
 #include <media/v4l2-ctrls.h>
-#include <media/v4l2-vp9.h>
+#include <media/v4l2-vp9-rs.h>
 #include <media/videobuf2-core.h>
 
 #include "rockchip_av1_entropymode.h"
@@ -231,9 +231,9 @@ struct hantro_vp9_dec_hw_ctx {
 	struct hantro_aux_buf tile_edge;
 	struct hantro_aux_buf segment_map;
 	struct hantro_aux_buf misc;
-	struct v4l2_vp9_frame_symbol_counts cnts;
-	struct v4l2_vp9_frame_context probability_tables;
-	struct v4l2_vp9_frame_context frame_context[4];
+	struct v4l2_vp9_frame_symbol_counts_rs cnts;
+	struct v4l2_vp9_frame_context_rs probability_tables;
+	struct v4l2_vp9_frame_context_rs frame_context[4];
 	struct hantro_vp9_frame_info cur;
 	struct hantro_vp9_frame_info last;
 
drivers/media/v4l2-core/Kconfig +5 -0
index 331b8e535e5b..878229c2490b 100644
@@ -48,6 +48,11 @@ config V4L2_H264
 config V4L2_VP9
 	tristate
 
+# Used by drivers that need the Rust equivalent to v4l2-vp9.ko
+config V4L2_VP9_RS
+	tristate
+	depends on RUST
+
 # Used by drivers that need v4l2-mem2mem.ko
 config V4L2_MEM2MEM_DEV
 	tristate
drivers/staging/media/rkvdec/Kconfig +1 -1
index 5f3bdd848a2c..245dc47082d4 100644
@@ -8,7 +8,7 @@ config VIDEO_ROCKCHIP_VDEC
 	select VIDEOBUF2_VMALLOC
 	select V4L2_MEM2MEM_DEV
 	select V4L2_H264
-	select V4L2_VP9
+	select V4L2_VP9_RS
 	help
 	  Support for the Rockchip Video Decoder IP present on Rockchip SoCs,
 	  which accelerates video decoding.
drivers/staging/media/rkvdec/rkvdec-vp9.c +26 -26

171 lines not on this page · raw message

include/media/v4l2-vp9-rs.h new file +99 -0

99 lines not on this page · raw message

rust/bindings/bindings_helper.h +1 -0
index c651d38e5dd6..25e2c06bb967 100644
@@ -25,6 +25,7 @@
 #include <linux/sched.h>
 #include <linux/task_work.h>
 #include <linux/workqueue.h>
+#include <linux/v4l2-controls.h>
 
 /* `bindgen` gets confused at certain things. */
 const size_t RUST_CONST_HELPER_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN;
rust/kernel/lib.rs +2 -0
index cc39195b0157..724e1d3bf2af 100644
@@ -39,6 +39,8 @@
 pub mod ioctl;
 #[cfg(CONFIG_KUNIT)]
 pub mod kunit;
+#[cfg(CONFIG_MEDIA_SUPPORT)]
+pub mod media;
 #[cfg(CONFIG_NET)]
 pub mod net;
 pub mod prelude;
rust/kernel/media.rs new file +5 -0
new file mode 100644
index 000000000000..72f5cd716874
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Media support
+
+pub mod v4l2_core;
\ No newline at end of file
rust/kernel/media/v4l2_core.rs new file +6 -0
new file mode 100644
index 000000000000..7515fe748a65
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! V4L2 Core
+
+#[cfg(CONFIG_V4L2_VP9_RS)]
+pub mod vp9;
\ No newline at end of file
rust/kernel/media/v4l2_core/cbindgen.toml new file +26 -0
new file mode 100644
index 000000000000..ebbc8a5e2fff
@@ -0,0 +1,26 @@
+# Generate the C API with:
+# cbindgen  --lang c  -v --config rust/kernel/media/v4l2_core/cbindgen.toml --output include/media/v4l2-vp9-rs.h -- rust/kernel/media/v4l2_core/vp9.rs
+
+language = "C"
+
+# No <stdarg.h>, <stdbool.h>, <stdint.h>, <stdlib.h>, <uchar.h>
+no_includes = true
+style = "tag"
+
+sys_includes = ["media/v4l2-ctrls.h"]
+autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
+usize_is_size_t = true
+include_guard = "V4L2_VP9_RS_H"
+
+# Rename everything from bindgen
+# See https://github.com/mozilla/cbindgen/issues/621
+# Also rename our mangled structs into snake_case for the C API
+[export.rename]
+"v4l2_ctrl_vp9_compressed_hdr" = "struct v4l2_ctrl_vp9_compressed_hdr"
+"v4l2_ctrl_vp9_frame" = "struct v4l2_ctrl_vp9_frame"
+"FrameContext" = "v4l2_vp9_frame_context_rs"
+"FrameSymbolCounts" = "v4l2_vp9_frame_symbol_counts_rs"
+"FrameMvContext" = "v4l2_vp9_frame_mv_context_rs"
+
+[enum]
+prefix_with_name = true
rust/kernel/media/v4l2_core/vp9.rs new file +2053 -0

2,053 lines not on this page · raw message

Signature · 1 line
2.43.0
author reviewer
#6 lore raw

RFC PATCH v2 2/2 media: rkvdec: rewrite parts of the driver in Rust

In reply to Daniel Almeida
---
 drivers/staging/media/rkvdec/Kconfig       |   1 +
 drivers/staging/media/rkvdec/Makefile      |   2 +-
 drivers/staging/media/rkvdec/cbindgen.toml |  36 ++
 drivers/staging/media/rkvdec/common.rs     |  19 +
 drivers/staging/media/rkvdec/regs.rs       | 237 ++++++++
 drivers/staging/media/rkvdec/rkvdec-vp9.c  | 585 ++++---------------
 drivers/staging/media/rkvdec/rkvdec_rs.h   | 125 ++++
 drivers/staging/media/rkvdec/rkvdec_rs.rs  |  14 +
 drivers/staging/media/rkvdec/vp9.rs        | 636 +++++++++++++++++++++
 rust/helpers.c                             |   7 +
 10 files changed, 1181 insertions(+), 481 deletions(-)
 create mode 100644 drivers/staging/media/rkvdec/cbindgen.toml
 create mode 100644 drivers/staging/media/rkvdec/common.rs
 create mode 100644 drivers/staging/media/rkvdec/regs.rs
 create mode 100644 drivers/staging/media/rkvdec/rkvdec_rs.h
 create mode 100644 drivers/staging/media/rkvdec/rkvdec_rs.rs
 create mode 100644 drivers/staging/media/rkvdec/vp9.rs
drivers/staging/media/rkvdec/Kconfig +1 -0
index 245dc47082d4..1877b6b0153b 100644
@@ -3,6 +3,7 @@ config VIDEO_ROCKCHIP_VDEC
 	tristate "Rockchip Video Decoder driver"
 	depends on ARCH_ROCKCHIP || COMPILE_TEST
 	depends on VIDEO_DEV
+	depends on RUST
 	select MEDIA_CONTROLLER
 	select VIDEOBUF2_DMA_CONTIG
 	select VIDEOBUF2_VMALLOC
drivers/staging/media/rkvdec/Makefile +1 -1
index cb86b429cfaa..8e04aa7dbf16 100644
@@ -1,3 +1,3 @@
 obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o
 
-rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.o
+rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.o rkvdec_rs.o
drivers/staging/media/rkvdec/cbindgen.toml new file +36 -0
new file mode 100644
index 000000000000..3ca785d26cc8
@@ -0,0 +1,36 @@
+# Generate the C API with:
+# cbindgen  --lang c  -v --config drivers/staging/media/rkvdec/cbindgen.toml --output drivers/staging/media/rkvdec/rkvdec_rs.h -- drivers/staging/media/rkvdec/rkvdec_rs.rs
+
+language = "C"
+
+# No <stdarg.h>, <stdbool.h>, <stdint.h>, <stdlib.h>, <uchar.h>
+no_includes = true
+style = "tag"
+
+sys_includes = ["media/v4l2-ctrls.h", "media/v4l2-vp9-rs.h", "linux/types.h"]
+autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
+usize_is_size_t = true
+include_guard = "RKVDEC_RS_H"
+
+# Rename everything from bindgen
+# See https://github.com/mozilla/cbindgen/issues/621
+# Also rename our mangled structs into snake_case for the C API
+[export.rename]
+"v4l2_ctrl_vp9_frame" = "struct v4l2_ctrl_vp9_frame"
+"v4l2_vp9_segmentation" = "struct v4l2_vp9_segmentation"
+"v4l2_vp9_loop_filter" = "struct v4l2_vp9_loop_filter"
+"Resolution" = "rkvdec_rs_resolution"
+"Mv" = " rkvdec_rs_vp9_mv"
+"InterFrameProbs" = " rkvdec_rs_vp9_inter_frame_probs"
+"IntraModeProbs" = " rkvdec_rs_vp9_intra_mode_probs"
+"IntraOnlyFrameProbs" = " rkvdec_rs_vp9_intra_only_frame_probs"
+"FrameProbs" = " rkvdec_rs_vp9_frame_probs"
+"HwProbs" = " rkvdec_rs_vp9_probs"
+"DecodedBufferInfo" = "rkvdec_rs_decoded_buffer_info"
+"FrameInfo" = "rkvdec_rs_frame_info"
+"BaseRun" = "rkvdec_rs_base_run"
+"Run" = "rkvdec_rs_run"
+"FrameContext" = "struct v4l2_vp9_frame_context_rs"
+
+[enum]
+prefix_with_name = true
drivers/staging/media/rkvdec/common.rs new file +19 -0
new file mode 100644
index 000000000000..0f9f35bc915f
@@ -0,0 +1,19 @@
+#![allow(missing_docs)]
+#![allow(non_upper_case_globals)]
+#![allow(dead_code)]
+#![allow(unused_variables)]
+
+use kernel::bindings;
+
+#[repr(C)]
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+pub(crate) struct Resolution {
+    pub(crate) width: u32,
+    pub(crate) height: u32,
+}
+
+#[repr(C)]
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+pub(crate) struct BaseRun {
+    pub(crate) decoded_resolution: Resolution,
+}
drivers/staging/media/rkvdec/regs.rs new file +237 -0

238 lines not on this page · raw message

drivers/staging/media/rkvdec/rkvdec-vp9.c +105 -480

657 lines not on this page · raw message

drivers/staging/media/rkvdec/rkvdec_rs.h new file +125 -0

125 lines not on this page · raw message

drivers/staging/media/rkvdec/rkvdec_rs.rs new file +14 -0
new file mode 100644
index 000000000000..a7a48b203225
@@ -0,0 +1,14 @@
+#![allow(missing_docs, unused_imports)]
+
+use kernel::prelude::*;
+
+const __LOG_PREFIX: &[u8] = b"rkvdec_rs\0";
+pub(crate) mod vp9;
+mod common;
+mod regs;
+
+macro_rules! round_up {
+    ($x:expr, $v:expr) => ((($x - 1) | ($v - 1)) + 1)
+}
+
+pub(crate) use round_up;
drivers/staging/media/rkvdec/vp9.rs new file +636 -0

636 lines not on this page · raw message

rust/helpers.c +7 -0
index d146bbf25aec..fde0520e1d57 100644
@@ -20,6 +20,7 @@
  * Sorted alphabetically.
  */
 
+#include "asm/io.h"
 #include <kunit/test-bug.h>
 #include <linux/bug.h>
 #include <linux/build_bug.h>
@@ -272,3 +273,9 @@ static_assert(
 	__alignof__(size_t) == __alignof__(uintptr_t),
 	"Rust code expects C `size_t` to match Rust `usize`"
 );
+
+void rust_helper_writel_relaxed(u32 val, void *addr)
+{
+	writel_relaxed(val, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writel_relaxed);
Signature · 1 line
2.43.0
#7 lore raw

Re: [RFC PATCH v2 2/2] media: rkvdec: rewrite parts of the driver in Rust

Hi Daniel,

as I'm already sold to the idea, I decided to discuss other things ;-P see below.

Le jeudi 07 mars 2024 à 16:08 -0300, Daniel Almeida a écrit :

drivers/staging/media/rkvdec/regs.rs line 7

Quoted 9 lines

+++ b/drivers/staging/media/rkvdec/regs.rs

drivers/staging/media/rkvdec/regs.rs new file +7 -0
@@ -0,0 +1,237 @@
+#![allow(dead_code)]
+#![allow(unused_macros)]
+
+pub(crate) const RKVDEC_REG_INTERRUPT: u32 = 0x004;
+pub(crate) const RKVDEC_INTERRUPT_DEC_E: u32 = 1 << 0;
+pub(crate) const RKVDEC_CONFIG_DEC_CLK_GATE_E: u32 = 1 << 1;
+pub(crate) const RKVDEC_E_STRMD_CLKGATE_DIS: u32 = 1 << 2;

Regs file are a bit our coded reference information on how the registers are layout in memory. So I believe the alignment, indent and readability of that file would at least need polishing.

But to the light of your comment, being able to use more modern utility, isn't there something in Rust we could use to better map the registers ? These variables are just mask offset to help someone write specific bits within a list of 32bit registers (Hantro and RKVDEC have that in common). In downstream mpp userspace driver, they maps all the register with a C struct.

struct reg123 {
val1 :3 // bit 31-29
val2 :20 // bit 28-9
val3 :9 // bit 8-0
};

I seriously think it looks nicer, and when the compiler does not screw it up (the main reason we don't use that), it is also a lot safer and simpler to use. Now, lets forget about C, my question is just if there is something in Rust that could give us the safety to edit the right portion of a register, but also allow expressing that map in a readable form.

Note that sometimes, we may want to read the register before editing it, something MPP giant C struct does not help with.

Nicolas

author reviewer
#8 lore raw

Re: [RFC PATCH v2 2/2] media: rkvdec: rewrite parts of the driver in Rust

Hi Nicolas,

Quoted 5 lines

struct reg123 {
val1 :3 // bit 31-29
val2 :20 // bit 28-9
val3 :9 // bit 8-0
};

What you're describing can be modeled as Ranges in Rust:

```
use core::ops::Range;

struct Foo {
    reg1: Range<u32>,
    reg2: Range<u32>,
    reg3: Range<u32>
}

const FOO_REGMAP: Foo = Foo {
        reg1: 0..3,
        reg2: 3..24,
        reg3: 24..32
};
```

It becomes more useful when you pair that with a bit writer. For an example of previous art, see Faith's work: [0]

This has asserts in the right places so that you do not shoot yourself in the foot. IMHO, such a data structure can be shared with the whole Rust code in the kernel.

You can then describe your writes using the ranges, e.g.: [1]

But as we've established, instead of writing the ranges down directly, you can simply refer to them as FOO_REGMAP.reg1, FOO_REGMAP.reg2 and so on.

I believe this is both more readable and safer.

[0]: https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/nouveau/compiler/bitview/lib.rs?ref_type=heads

[1]: https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/nouveau/compiler/nak/encode_sm70.rs?ref_type=heads#L228

#9 lore raw

Re: [RFC PATCH v2 0/2] Rewrite parts of rkvdec driver and the VP9 codec library in Rust

In reply to Daniel Almeida

Hi Daniel,

Quoted 8 lines · Daniel Almeida

Hi Mauro, Hans,

While working on v1 for this patchset, I realized that we can go further by converting the error-prone sections of our codec drivers to Rust. This also does not need any bindings in order to work.

As yet another proof-of-concept, I have converted parts of the rkvdec driver. Refer to instructions in v1 to test this.

I tested this on rk3399 rkvdec with:
./fluster.py run -d GStreamer-VP9-V4L2SL-Gst1.0 -ts VP9-TEST-VECTORS -j1

And I get the same result as on mainline (6.8-rc3)
Ran 220/305 tests successfully in 913.877 secs

So I can't say I understand all the Rust here but my testing didn't show any regressions in the VP9 decoding. :)

Deb

Quoted 83 lines

Notice how:

1) many problematic memcpy's go away, these become a simple assignment in Rust.

2) it can interop seamlessly with the code in rkvdec-vp9.c that was already converted in v1 of this series.

3) it can use the Rust version of seg_feat_enabled directly in vp9.rs, while also using the C APIs from the v4l2-vp9-rs library in rkvdec-vp9.c

4) more modern things become available for the programmer, like iterators and their methods without a performance penalty.

I want to propose the following:

Let's merge a non-RFC version of this series and gate it behind some kconfigs so that we can switch between the C and Rust implementations. Users get the C version by default, while we continuously test the Rust components on a CI for a few months. This will hopefully be enough time until the next Media Summit.

My aim is to eventually deprecate the C parts once we're confident that the Rust code is stable enough. I will keep my own tree, and send PRs to the media tree if a rebase or fix is needed.

I believe this will not be disruptive nor require any extra work from anyone but me.

-- Daniel

Again, applies on top of:

commit d9c1fae3e5b225f2e45e0bca519f9a2967cd1062

Author: Alice Ryhl <[email protected]>
Date:   Fri Feb 9 11:18:22 2024 +0000

rust: file: add abstraction for poll_table

For those looking for a branch instead: https://gitlab.collabora.com/dwlsalmeida/for-upstream/-/tree/vp9-rs-rkvdec?ref_type=heads

Daniel Almeida (2):
v4l2-core: rewrite the VP9 library in Rust
media: rkvdec: rewrite parts of the driver in Rust

 drivers/media/platform/verisilicon/Kconfig    |    2 +-
 .../platform/verisilicon/hantro_g2_vp9_dec.c  |   38 +-
 .../media/platform/verisilicon/hantro_hw.h    |    8 +-
 drivers/media/v4l2-core/Kconfig               |    5 +
 drivers/staging/media/rkvdec/Kconfig          |    3 +-
 drivers/staging/media/rkvdec/Makefile         |    2 +-
 drivers/staging/media/rkvdec/cbindgen.toml    |   36 +
 drivers/staging/media/rkvdec/common.rs        |   19 +
 drivers/staging/media/rkvdec/regs.rs          |  237 ++
 drivers/staging/media/rkvdec/rkvdec-vp9.c     |  607 +----
 drivers/staging/media/rkvdec/rkvdec_rs.h      |  125 +
 drivers/staging/media/rkvdec/rkvdec_rs.rs     |   14 +
 drivers/staging/media/rkvdec/vp9.rs           |  636 +++++
 include/media/v4l2-vp9-rs.h                   |   99 +
 rust/bindings/bindings_helper.h               |    1 +
 rust/helpers.c                                |    7 +
 rust/kernel/lib.rs                            |    2 +
 rust/kernel/media.rs                          |    5 +
 rust/kernel/media/v4l2_core.rs                |    6 +
 rust/kernel/media/v4l2_core/cbindgen.toml     |   26 +
 rust/kernel/media/v4l2_core/vp9.rs            | 2053 +++++++++++++++++
 21 files changed, 3415 insertions(+), 516 deletions(-)
 create mode 100644 drivers/staging/media/rkvdec/cbindgen.toml
 create mode 100644 drivers/staging/media/rkvdec/common.rs
 create mode 100644 drivers/staging/media/rkvdec/regs.rs
 create mode 100644 drivers/staging/media/rkvdec/rkvdec_rs.h
 create mode 100644 drivers/staging/media/rkvdec/rkvdec_rs.rs
 create mode 100644 drivers/staging/media/rkvdec/vp9.rs
 create mode 100644 include/media/v4l2-vp9-rs.h
 create mode 100644 rust/kernel/media.rs
 create mode 100644 rust/kernel/media/v4l2_core.rs
 create mode 100644 rust/kernel/media/v4l2_core/cbindgen.toml
 create mode 100644 rust/kernel/media/v4l2_core/vp9.rs
Signature · 1 line
2.43.0