Skip to contentSkip to conversation

Rust PuzzleFS filesystem driver

RFC PATCH In review Ariel Miculas 2 participants 24 messages May 16, 2024 – May 17, 2024

Subsystems (7)
x86 VFS Kbuild Rust +3 more subsystems
lore
Share & tools
lore
#0 lore raw

v3 cover Rust PuzzleFS filesystem driver

Hello everyone,

This series is the third version of the proof-of-concept PuzzleFS filesystem driver, an open-source next-generation container filesystem [1], designed to address the limitation of the existing OCI format. It supports direct mounting of container filesystems without an extraction step, and it uses content defined chunking (CDC) in order to often achieve substantial disk space savings. It is similar to casync [9] in this regard. For more details, I had a presentation last year at Open Source Summit Europe [2]. There's also a dedicated PuzzleFS page on the rust-for-linux website [3] and an LWN article about PuzzleFS [4].

This patch series is based on the latest filesystem abstraction patches
[5] sent to LKML a few days ago. I've been keeping PuzzleFS up-to-date with the multiple versions of the filesystem abstractions, even though I haven't sent patches after each change. However, Cisco is still interested in upstreaming PuzzleFS and I would like to gather another round of feedback.

I have described my kernel setup and how to get started with the PuzzleFS kernel driver in this blog post [7]. Below I've included a short demo of the PuzzleFS kernel driver.

To mount a PuzzleFS driver, we must pass the oci_root_dir and image_manifest as parameters.

```
~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_xattr" -o \
image_manifest="ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953c8ab" \
none /mnt
[ 6789.110219] PuzzleFS: opened puzzlefs [/home/puzzlefs_xattr]:[ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953c8ab]

~ # cd /mnt/
/mnt # ls -la
total 1
drwxr-xr-x    2 1000     1000             0 Jan  1  1970 test
drwxr-xr-x    2 1000     1000             0 Jan  1  1970 test-2
-rw-r--r--    1 1000     1000            20 Jan  1  1970 test.txt

/mnt # cat test.txt
The quick brown fox
```

In this example, /home/puzzlefs_xattr is the puzzlefs oci directory, obtained by building a PuzzleFS image via the puzzlefs build command. Here's the disk layout of the image (for more details about the image format, consult the README in the github repository [1]):

```
/mnt # ls -lR /home/puzzlefs_xattr/
/home/puzzlefs_xattr/:
total 8
drwxr-xr-x    3 1000     1000             0 May 15 20:24 blobs
-rw-r--r--    1 1000     1000           263 May 15 20:24 index.json
-rw-r--r--    1 1000     1000            37 May 15 20:24 oci-layout

/home/puzzlefs_xattr/blobs:
total 0
drwxr-xr-x    2 1000     1000             0 May 15 20:24 sha256

/home/puzzlefs_xattr/blobs/sha256:
total 12
-rw-------    1 1000     1000            20 May 15 20:24 35fb7cc2337d10d618a1bad35c7a9e957c213f00d0ed32f2454b2a99a9718
-rw-------    1 1000     1000           752 May 15 20:24 d1cbb5b11a9044ecb3889397e0df8c594484cc630294cf208e57ad9dcf6ca
-rw-------    1 1000     1000           272 May 15 20:24 ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953b
```

To get the PuzzleFS image manifest, we can look into the index.json file. The following digest, corresponding to the xattr tag, must be passed as a parameter when mounting the filesystem:
ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953c8ab

In userspace we can pass the xattr tag directly, but we don't want to parse the json format in the kernel.

```
$ cat index.json | jq
{
  "schemaVersion": -1,
  "manifests": [
    {
      "digest": "sha256:ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953c8ab",
      "size": 272,
      "media_type": "application/vnd.puzzlefs.image.rootfs.v1",
      "annotations": {
        "org.opencontainers.image.ref.name": "xattr"
      }
    }
  ],
  "annotations": {}
}
```

For testing extended attributes, I've written a small cli program called xattr-cli which is available here [8]. No reason to use this over getfattr, except it is not available in a busybox environment.

```
/mnt # /home/xattr-cli test
Extended attributes of test:
user.comment -> this is a dir
user.groceries -> want to buy a file
/mnt # /home/xattr-cli test-2/
Extended attributes of test-2/:
user.whatever -> want to buy a file
```

A git tree for this patch series is available here:
git://github.com/ariel-miculas/linux.git puzzlefs_rfc_v3

Web:
https://github.com/ariel-miculas/linux/tree/puzzlefs_rfc_v3

[1] https://github.com/project-machine/puzzlefs
[2] https://osseu2023.sched.com/event/1OGjk/puzzlefs-the-next-generation-container-filesystem-armand-ariel-miculas-cisco-systems
[3] https://rust-for-linux.com/puzzlefs-filesystem-driver
[4] https://lwn.net/Articles/945320/
[5] https://lore.kernel.org/rust-for-linux/[email protected]/
[6] https://lore.kernel.org/rust-for-linux/[email protected]/
[7] https://machinerun.io/puzzlefs/2023/10/30/Linux-kernel-setup.html
[8] https://github.com/ariel-miculas/xattr-cli
[9] https://0pointer.net/blog/casync-a-tool-for-distributing-file-system-images.html

---

Main changes since v2 [6]:
* PuzzleFS now uses the latest filesystem abstractions [5]
* added extended attributes support

---

Alice Ryhl (1):
rust: add improved version of ForeignOwnable::borrow_mut

Ariel Miculas (19):
kernel: configs: add qemu-busybox-min.config
rust: hex: import crate
rust: hex: add SPDX license identifiers
rust: Kbuild: enable hex
rust: hex: add encode_hex_iter and encode_hex_upper_iter methods
rust: capnp: import crate
rust: capnp: add SPDX License Identifiers
rust: capnp: return an error when trying to display floating point
values
rust: Kbuild: enable capnp
rust: kernel: add an abstraction over vfsmount to allow cloning a new
private mount
rust: file: Add support for reading files using their path
fs: puzzlefs: Implement the initial version of PuzzleFS
rust: kernel: add from_iter_fallible for Vec<T>
kernel: configs: add puzzlefs config fragment
scripts: add fs directory to rust-analyzer
fs: puzzlefs: add extended attributes support
Add borrow_mut implementation to a ForeignOwnable CString
fs: puzzlefs: add oci_root_dir and image_manifest mount parameters
fs: puzzlefs: implement statfs for puzzlefs

Wedson Almeida Filho (2):
rust: file: add bindings for struct file
rust: add support for file system parameters

 arch/x86/configs/qemu-busybox-min.config      |    11 +
 fs/Kconfig                                    |     1 +
 fs/Makefile                                   |     1 +
 fs/puzzlefs/Kconfig                           |    14 +
 fs/puzzlefs/Makefile                          |     8 +
 fs/puzzlefs/puzzle.rs                         |     5 +
 fs/puzzlefs/puzzle/error.rs                   |   105 +
 fs/puzzlefs/puzzle/inode.rs                   |   122 +
 fs/puzzlefs/puzzle/oci.rs                     |    70 +
 fs/puzzlefs/puzzle/types.rs                   |   377 +
 fs/puzzlefs/puzzle/types/manifest.capnp       |    15 +
 fs/puzzlefs/puzzle/types/manifest_capnp.rs    |   757 +
 fs/puzzlefs/puzzle/types/metadata.capnp       |    65 +
 fs/puzzlefs/puzzle/types/metadata_capnp.rs    |  4049 +++++
 fs/puzzlefs/puzzlefs.rs                       |   330 +
 kernel/configs/puzzlefs.config                |     4 +
 kernel/configs/qemu-busybox-min.config        |    56 +
 rust/Makefile                                 |    41 +-
 rust/bindings/bindings_helper.h               |     3 +
 rust/capnp/any_pointer.rs                     |   315 +
 rust/capnp/any_pointer_list.rs                |   210 +
 rust/capnp/capability.rs                      |   365 +
 rust/capnp/capability_list.rs                 |   299 +
 rust/capnp/constant.rs                        |    56 +
 rust/capnp/data.rs                            |    97 +
 rust/capnp/data_list.rs                       |   220 +
 rust/capnp/dynamic_list.rs                    |   410 +
 rust/capnp/dynamic_struct.rs                  |   784 +
 rust/capnp/dynamic_value.rs                   |   319 +
 rust/capnp/enum_list.rs                       |   239 +
 rust/capnp/introspect.rs                      |   284 +
 rust/capnp/io.rs                              |   204 +
 rust/capnp/lib.rs                             |   653 +
 rust/capnp/list_list.rs                       |   298 +
 rust/capnp/message.rs                         |   880 +
 rust/capnp/primitive_list.rs                  |   281 +
 rust/capnp/private/arena.rs                   |   419 +
 rust/capnp/private/capability.rs              |   157 +
 rust/capnp/private/layout.rs                  |  4212 +++++
 rust/capnp/private/layout_test.rs             |   194 +
 rust/capnp/private/mask.rs                    |    64 +
 rust/capnp/private/mod.rs                     |    38 +
 rust/capnp/private/primitive.rs               |   121 +
 rust/capnp/private/read_limiter.rs            |   115 +
 rust/capnp/private/units.rs                   |    70 +
 rust/capnp/private/zero.rs                    |    48 +
 rust/capnp/raw.rs                             |    71 +
 rust/capnp/schema.rs                          |   432 +
 rust/capnp/schema_capnp.rs                    | 14450 ++++++++++++++++
 rust/capnp/serialize.rs                       |   963 +
 .../serialize/no_alloc_buffer_segments.rs     |   629 +
 rust/capnp/serialize_packed.rs                |   620 +
 rust/capnp/stringify.rs                       |   187 +
 rust/capnp/struct_list.rs                     |   302 +
 rust/capnp/text.rs                            |   296 +
 rust/capnp/text_list.rs                       |   218 +
 rust/capnp/traits.rs                          |   235 +
 rust/helpers.c                                |    16 +
 rust/hex/error.rs                             |    61 +
 rust/hex/lib.rs                               |   495 +
 rust/hex/serde.rs                             |   104 +
 rust/kernel/alloc/vec_ext.rs                  |    12 +
 rust/kernel/error.rs                          |     4 +-
 rust/kernel/file.rs                           |   298 +
 rust/kernel/fs.rs                             |   212 +-
 rust/kernel/fs/inode.rs                       |    66 +-
 rust/kernel/fs/param.rs                       |   576 +
 rust/kernel/lib.rs                            |     9 +
 rust/kernel/mount.rs                          |    73 +
 rust/kernel/str.rs                            |    24 +
 rust/kernel/sync/arc.rs                       |    25 +-
 rust/kernel/types.rs                          |    93 +-
 samples/rust/rust_rofs.rs                     |    32 +-
 scripts/Makefile.build                        |     2 +-
 scripts/generate_rust_analyzer.py             |     2 +-
 75 files changed, 37843 insertions(+), 50 deletions(-)
 create mode 100644 arch/x86/configs/qemu-busybox-min.config
 create mode 100644 fs/puzzlefs/Kconfig
 create mode 100644 fs/puzzlefs/Makefile
 create mode 100644 fs/puzzlefs/puzzle.rs
 create mode 100644 fs/puzzlefs/puzzle/error.rs
 create mode 100644 fs/puzzlefs/puzzle/inode.rs
 create mode 100644 fs/puzzlefs/puzzle/oci.rs
 create mode 100644 fs/puzzlefs/puzzle/types.rs
 create mode 100644 fs/puzzlefs/puzzle/types/manifest.capnp
 create mode 100644 fs/puzzlefs/puzzle/types/manifest_capnp.rs
 create mode 100644 fs/puzzlefs/puzzle/types/metadata.capnp
 create mode 100644 fs/puzzlefs/puzzle/types/metadata_capnp.rs
 create mode 100644 fs/puzzlefs/puzzlefs.rs
 create mode 100644 kernel/configs/puzzlefs.config
 create mode 100644 kernel/configs/qemu-busybox-min.config
 create mode 100644 rust/capnp/any_pointer.rs
 create mode 100644 rust/capnp/any_pointer_list.rs
 create mode 100644 rust/capnp/capability.rs
 create mode 100644 rust/capnp/capability_list.rs
 create mode 100644 rust/capnp/constant.rs
 create mode 100644 rust/capnp/data.rs
 create mode 100644 rust/capnp/data_list.rs
 create mode 100644 rust/capnp/dynamic_list.rs
 create mode 100644 rust/capnp/dynamic_struct.rs
 create mode 100644 rust/capnp/dynamic_value.rs
 create mode 100644 rust/capnp/enum_list.rs
 create mode 100644 rust/capnp/introspect.rs
 create mode 100644 rust/capnp/io.rs
 create mode 100644 rust/capnp/lib.rs
 create mode 100644 rust/capnp/list_list.rs
 create mode 100644 rust/capnp/message.rs
 create mode 100644 rust/capnp/primitive_list.rs
 create mode 100644 rust/capnp/private/arena.rs
 create mode 100644 rust/capnp/private/capability.rs
 create mode 100644 rust/capnp/private/layout.rs
 create mode 100644 rust/capnp/private/layout_test.rs
 create mode 100644 rust/capnp/private/mask.rs
 create mode 100644 rust/capnp/private/mod.rs
 create mode 100644 rust/capnp/private/primitive.rs
 create mode 100644 rust/capnp/private/read_limiter.rs
 create mode 100644 rust/capnp/private/units.rs
 create mode 100644 rust/capnp/private/zero.rs
 create mode 100644 rust/capnp/raw.rs
 create mode 100644 rust/capnp/schema.rs
 create mode 100644 rust/capnp/schema_capnp.rs
 create mode 100644 rust/capnp/serialize.rs
 create mode 100644 rust/capnp/serialize/no_alloc_buffer_segments.rs
 create mode 100644 rust/capnp/serialize_packed.rs
 create mode 100644 rust/capnp/stringify.rs
 create mode 100644 rust/capnp/struct_list.rs
 create mode 100644 rust/capnp/text.rs
 create mode 100644 rust/capnp/text_list.rs
 create mode 100644 rust/capnp/traits.rs
 create mode 100644 rust/hex/error.rs
 create mode 100644 rust/hex/lib.rs
 create mode 100644 rust/hex/serde.rs
 create mode 100644 rust/kernel/file.rs
 create mode 100644 rust/kernel/fs/param.rs
 create mode 100644 rust/kernel/mount.rs
Signature · 1 line
2.34.1
#1 lore raw

RFC PATCH v3 1/22 kernel: configs: add qemu-busybox-min.config

This is a minimal configuration for running a busybox initramfs image with networking support. Added for convenience.

---
 arch/x86/configs/qemu-busybox-min.config | 11 +++++
 kernel/configs/qemu-busybox-min.config   | 56 ++++++++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100644 arch/x86/configs/qemu-busybox-min.config
 create mode 100644 kernel/configs/qemu-busybox-min.config
arch/x86/configs/qemu-busybox-min.config new file +11 -0
new file mode 100644
index 000000000000..9a2bf2549053
@@ -0,0 +1,11 @@
+CONFIG_64BIT=y
+CONFIG_ACPI=y
+
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PVH=y
+
+CONFIG_CMDLINE_BOOL=y
+CONFIG_CMDLINE="console=ttyS0 nokaslr rdinit=/sbin/init"
kernel/configs/qemu-busybox-min.config new file +56 -0

56 lines not on this page · raw message

Signature · 1 line
2.34.1
#2 lore raw

RFC PATCH v3 2/22 rust: hex: import crate

In reply to Ariel Miculas

This is a subset of the Rust hex crate, version v0.4.3, licensed under "Apache-2.0 OR MIT", from:

https://github.com/KokaKiwi/rust-hex/tree/v0.4.3/src

The files are copied as-is, with no modifications whatsoever (not even adding the SPDX identifiers).

For copyright details, please see:

https://github.com/KokaKiwi/rust-hex/blob/v0.4.3/README.md#license
https://github.com/KokaKiwi/rust-hex/blob/v0.4.3/LICENSE-APACHE
https://github.com/KokaKiwi/rust-hex/blob/v0.4.3/LICENSE-MIT

The next three patches modify these files as needed for use within the kernel. This patch split allows reviewers to double-check the import and to clearly see the differences introduced.

The following script may be used to verify the contents:

for path in $(cd rust/hex/ && find . -type f -name '*.rs'); do

    curl --silent --show-error --location \
        https://github.com/KokaKiwi/rust-hex/raw/v0.4.3/src/$path \
        | diff --unified rust/hex/$path - && echo $path: OK

done

---
 rust/hex/error.rs |  59 ++++++
 rust/hex/lib.rs   | 525 ++++++++++++++++++++++++++++++++++++++++++++++
 rust/hex/serde.rs | 102 +++++++++
 3 files changed, 686 insertions(+)
 create mode 100644 rust/hex/error.rs
 create mode 100644 rust/hex/lib.rs
 create mode 100644 rust/hex/serde.rs
rust/hex/error.rs new file +59 -0
new file mode 100644
index 000000000000..ff7a3b5c16bd
@@ -0,0 +1,59 @@
+use core::fmt;
+
+/// The error type for decoding a hex string into `Vec<u8>` or `[u8; N]`.
+#[derive(Debug, Clone, Copy, PartialEq)]
+pub enum FromHexError {
+    /// An invalid character was found. Valid ones are: `0...9`, `a...f`
+    /// or `A...F`.
+    InvalidHexCharacter { c: char, index: usize },
+
+    /// A hex string's length needs to be even, as two digits correspond to
+    /// one byte.
+    OddLength,
+
+    /// If the hex string is decoded into a fixed sized container, such as an
+    /// array, the hex string's length * 2 has to match the container's
+    /// length.
+    InvalidStringLength,
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for FromHexError {}
+
+impl fmt::Display for FromHexError {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        match *self {
+            FromHexError::InvalidHexCharacter { c, index } => {
+                write!(f, "Invalid character {:?} at position {}", c, index)
+            }
+            FromHexError::OddLength => write!(f, "Odd number of digits"),
+            FromHexError::InvalidStringLength => write!(f, "Invalid string length"),
+        }
+    }
+}
+
+#[cfg(test)]
+// this feature flag is here to suppress unused
+// warnings of `super::*` and `pretty_assertions::assert_eq`
+#[cfg(feature = "alloc")]
+mod tests {
+    use super::*;
+    #[cfg(feature = "alloc")]
+    use alloc::string::ToString;
+    use pretty_assertions::assert_eq;
+
+    #[test]
+    #[cfg(feature = "alloc")]
+    fn test_display() {
+        assert_eq!(
+            FromHexError::InvalidHexCharacter { c: '\n', index: 5 }.to_string(),
+            "Invalid character '\\n' at position 5"
+        );
+
+        assert_eq!(FromHexError::OddLength.to_string(), "Odd number of digits");
+        assert_eq!(
+            FromHexError::InvalidStringLength.to_string(),
+            "Invalid string length"
+        );
+    }
+}
rust/hex/lib.rs new file +525 -0

525 lines not on this page · raw message

rust/hex/serde.rs new file +102 -0

102 lines not on this page · raw message

Signature · 1 line
2.34.1
#3 lore raw

RFC PATCH v3 3/22 rust: hex: add SPDX license identifiers

In reply to Ariel Miculas

Add the SPDX license identifiers, since the initial patch added the upstream sources with no modifications whatsoever.

---
 rust/hex/error.rs | 2 ++
 rust/hex/lib.rs   | 2 ++
 rust/hex/serde.rs | 2 ++
 3 files changed, 6 insertions(+)
rust/hex/error.rs +2 -0
index ff7a3b5c16bd..e553a046c9ed 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use core::fmt;
 
 /// The error type for decoding a hex string into `Vec<u8>` or `[u8; N]`.
rust/hex/lib.rs +2 -0
index ec48961b9ffe..a23def3f80db 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // Copyright (c) 2013-2014 The Rust Project Developers.
 // Copyright (c) 2015-2020 The rust-hex Developers.
 //
rust/hex/serde.rs +2 -0
index 335a15132a27..40ab1edb2bc8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Hex encoding with `serde`.
 #[cfg_attr(
     all(feature = "alloc", feature = "serde"),
Signature · 1 line
2.34.1
#4 lore raw

RFC PATCH v3 4/22 rust: Kbuild: enable `hex`

In reply to Ariel Miculas

With all the new files in place from the hex crate, this patch adds support for them in the build system.

---
 rust/Makefile          | 24 ++++++++++++++++++++----
 scripts/Makefile.build |  2 +-
 2 files changed, 21 insertions(+), 5 deletions(-)
rust/Makefile +20 -4
index ca86abac4ed0..fef8e38fbebd 100644
@@ -15,7 +15,7 @@ always-$(CONFIG_RUST) += libmacros.so
 no-clean-files += libmacros.so
 
 always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
-obj-$(CONFIG_RUST) += alloc.o bindings.o kernel.o
+obj-$(CONFIG_RUST) += alloc.o bindings.o hex.o kernel.o
 always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
     exports_kernel_generated.h
 
@@ -65,6 +65,15 @@ alloc-cfgs = \
     --cfg no_rc \
     --cfg no_sync
 
+hex-skip_flags := \
+    --edition=2021 \
+    -Drust_2018_idioms \
+    -Dunreachable_pub
+
+hex-flags := \
+    --edition=2018 \
+    -Amissing_docs
+
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
       cmd_rustdoc = \
 	OBJTREE=$(abspath $(objtree)) \
@@ -455,10 +464,17 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \
     $(obj)/uapi/uapi_generated.rs FORCE
 	+$(call if_changed_dep,rustc_library)
 
+$(obj)/hex.o: private skip_clippy = 1
+$(obj)/hex.o: private skip_flags = $(hex-skip_flags)
+$(obj)/hex.o: private rustc_target_flags = $(hex-flags)
+$(obj)/hex.o: $(src)/hex/lib.rs $(obj)/compiler_builtins.o FORCE
+	$(call if_changed_dep,rustc_library)
+
 $(obj)/kernel.o: private rustc_target_flags = --extern alloc \
-    --extern build_error --extern macros --extern bindings --extern uapi
+    --extern build_error --extern macros --extern bindings --extern uapi \
+    --extern hex
 $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
-    $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE
-	+$(call if_changed_dep,rustc_library)
+    $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o $(obj)/hex.o FORCE
+	$(call if_changed_dep,rustc_library)
 
 endif # CONFIG_RUST
scripts/Makefile.build +1 -1
index 367cfeea74c5..2e9a64224c51 100644
@@ -273,7 +273,7 @@ rust_common_cmd = \
 	-Zallow-features=$(rust_allowed_features) \
 	-Zcrate-attr=no_std \
 	-Zcrate-attr='feature($(rust_allowed_features))' \
-	--extern alloc --extern kernel \
+	--extern alloc --extern kernel --extern hex \
 	--crate-type rlib -L $(objtree)/rust/ \
 	--crate-name $(basename $(notdir $@)) \
 	--sysroot=/dev/null \
Signature · 1 line
2.34.1
#5 lore raw

RFC PATCH v3 5/22 rust: hex: add encode_hex_iter and encode_hex_upper_iter methods

In reply to Ariel Miculas

Add encode_hex_iter and encode_hex_upper_iter which return an iterator, so it can be used in conjunction with the Vec::from_iter_fallible function to (try to) create a hex string.

Replace BytesToHexChars with BytesToHexByteSequence because String is not available. We need to collect the values in a Vec<u8>, append a NUL character and the create a CStr on top of it.

Also remove the ToHex trait because it uses the FromIterator trait which cannot be implemented in the kernel due to the prohibition of infallible allocations.

---
 rust/hex/lib.rs | 60 ++++++++++++-------------------------------------
 1 file changed, 14 insertions(+), 46 deletions(-)
rust/hex/lib.rs +14 -46
index a23def3f80db..3abf61ccbfc3 100644
@@ -57,43 +57,18 @@
 #[cfg(all(feature = "alloc", feature = "serde"))]
 pub use crate::serde::{serialize, serialize_upper};
 
-/// Encoding values as hex string.
-///
-/// This trait is implemented for all `T` which implement `AsRef<[u8]>`. This
-/// includes `String`, `str`, `Vec<u8>` and `[u8]`.
-///
-/// # Example
-///
-/// ```
-/// use hex::ToHex;
-///
-/// println!("{}", "Hello world!".encode_hex::<String>());
-/// # assert_eq!("Hello world!".encode_hex::<String>(), "48656c6c6f20776f726c6421".to_string());
-/// ```
-///
-/// *Note*: instead of using this trait, you might want to use [`encode()`].
-pub trait ToHex {
-    /// Encode the hex strict representing `self` into the result. Lower case
-    /// letters are used (e.g. `f9b4ca`)
-    fn encode_hex<T: iter::FromIterator<char>>(&self) -> T;
-
-    /// Encode the hex strict representing `self` into the result. Upper case
-    /// letters are used (e.g. `F9B4CA`)
-    fn encode_hex_upper<T: iter::FromIterator<char>>(&self) -> T;
-}
-
 const HEX_CHARS_LOWER: &[u8; 16] = b"0123456789abcdef";
 const HEX_CHARS_UPPER: &[u8; 16] = b"0123456789ABCDEF";
 
-struct BytesToHexChars<'a> {
+struct BytesToHexByteSequence<'a> {
     inner: ::core::slice::Iter<'a, u8>,
     table: &'static [u8; 16],
-    next: Option<char>,
+    next: Option<u8>,
 }
 
-impl<'a> BytesToHexChars<'a> {
-    fn new(inner: &'a [u8], table: &'static [u8; 16]) -> BytesToHexChars<'a> {
-        BytesToHexChars {
+impl<'a> BytesToHexByteSequence<'a> {
+    fn new(inner: &'a [u8], table: &'static [u8; 16]) -> BytesToHexByteSequence<'a> {
+        BytesToHexByteSequence {
             inner: inner.iter(),
             table,
             next: None,
@@ -101,15 +76,15 @@ impl<'a> BytesToHexChars<'a> {
     }
 }
 
-impl<'a> Iterator for BytesToHexChars<'a> {
-    type Item = char;
+impl<'a> Iterator for BytesToHexByteSequence<'a> {
+    type Item = u8;
 
     fn next(&mut self) -> Option<Self::Item> {
         match self.next.take() {
             Some(current) => Some(current),
             None => self.inner.next().map(|byte| {
-                let current = self.table[(byte >> 4) as usize] as char;
-                self.next = Some(self.table[(byte & 0x0F) as usize] as char);
+                let current = self.table[(byte >> 4) as usize];
+                self.next = Some(self.table[(byte & 0x0F) as usize]);
                 current
             }),
         }
@@ -121,7 +96,7 @@ fn size_hint(&self) -> (usize, Option<usize>) {
     }
 }
 
-impl<'a> iter::ExactSizeIterator for BytesToHexChars<'a> {
+impl<'a> iter::ExactSizeIterator for BytesToHexByteSequence<'a> {
     fn len(&self) -> usize {
         let mut length = self.inner.len() * 2;
         if self.next.is_some() {
@@ -131,19 +106,12 @@ fn len(&self) -> usize {
     }
 }
 
-#[inline]
-fn encode_to_iter<T: iter::FromIterator<char>>(table: &'static [u8; 16], source: &[u8]) -> T {
-    BytesToHexChars::new(source, table).collect()
+pub fn encode_hex_iter<'a>(source: &'a [u8]) -> impl iter::Iterator<Item = u8> + 'a {
+    BytesToHexByteSequence::new(source, HEX_CHARS_LOWER).into_iter()
 }
 
-impl<T: AsRef<[u8]>> ToHex for T {
-    fn encode_hex<U: iter::FromIterator<char>>(&self) -> U {
-        encode_to_iter(HEX_CHARS_LOWER, self.as_ref())
-    }
-
-    fn encode_hex_upper<U: iter::FromIterator<char>>(&self) -> U {
-        encode_to_iter(HEX_CHARS_UPPER, self.as_ref())
-    }
+pub fn encode_hex_upper_iter<'a>(source: &'a [u8]) -> impl iter::Iterator<Item = u8> + 'a {
+    BytesToHexByteSequence::new(source, HEX_CHARS_UPPER).into_iter()
 }
 
 /// Types that can be decoded from a hex string.
Signature · 1 line
2.34.1
#6 lore raw

RFC PATCH v3 6/22 rust: capnp: import crate

In reply to Ariel Miculas

This is a subset of the Rust capnp crate, version capnpc-v0.18.0, licensed under "MIT", from:

https://github.com/capnproto/capnproto-rust/tree/capnpc-v0.18.0/capnp/src

The files are copied as-is, with no modifications whatsoever (not even adding the SPDX identifiers).

For copyright details, please see:

https://github.com/capnproto/capnproto-rust/blob/capnpc-v0.18.0/LICENSE

The next patches modify these files as needed for use within the kernel. This patch split allows reviewers to double-check the import and to clearly see the differences introduced.

The following script may be used to verify the contents:

    for path in $(cd rust/capnp/ && find . -type f -name '*.rs'); do
        curl --silent --show-error --location \
            https://github.com/capnproto/capnproto-rust/raw/capnpc-v0.18.0/capnp/src/$path \
            | diff --unified rust/capnp/$path - && echo $path: OK
    done
---
 rust/capnp/any_pointer.rs                     |   313 +
 rust/capnp/any_pointer_list.rs                |   208 +
 rust/capnp/capability.rs                      |   363 +
 rust/capnp/capability_list.rs                 |   297 +
 rust/capnp/constant.rs                        |    54 +
 rust/capnp/data.rs                            |    95 +
 rust/capnp/data_list.rs                       |   218 +
 rust/capnp/dynamic_list.rs                    |   408 +
 rust/capnp/dynamic_struct.rs                  |   782 +
 rust/capnp/dynamic_value.rs                   |   317 +
 rust/capnp/enum_list.rs                       |   237 +
 rust/capnp/introspect.rs                      |   282 +
 rust/capnp/io.rs                              |   202 +
 rust/capnp/lib.rs                             |   651 +
 rust/capnp/list_list.rs                       |   296 +
 rust/capnp/message.rs                         |   878 +
 rust/capnp/primitive_list.rs                  |   279 +
 rust/capnp/private/arena.rs                   |   417 +
 rust/capnp/private/capability.rs              |   155 +
 rust/capnp/private/layout.rs                  |  4210 +++++
 rust/capnp/private/layout_test.rs             |   192 +
 rust/capnp/private/mask.rs                    |    62 +
 rust/capnp/private/mod.rs                     |    36 +
 rust/capnp/private/primitive.rs               |   119 +
 rust/capnp/private/read_limiter.rs            |   113 +
 rust/capnp/private/units.rs                   |    68 +
 rust/capnp/private/zero.rs                    |    46 +
 rust/capnp/raw.rs                             |    69 +
 rust/capnp/schema.rs                          |   430 +
 rust/capnp/schema_capnp.rs                    | 14448 ++++++++++++++++
 rust/capnp/serialize.rs                       |   961 +
 .../serialize/no_alloc_buffer_segments.rs     |   627 +
 rust/capnp/serialize_packed.rs                |   618 +
 rust/capnp/stringify.rs                       |   181 +
 rust/capnp/struct_list.rs                     |   300 +
 rust/capnp/text.rs                            |   294 +
 rust/capnp/text_list.rs                       |   216 +
 rust/capnp/traits.rs                          |   233 +
 38 files changed, 29675 insertions(+)
 create mode 100644 rust/capnp/any_pointer.rs
 create mode 100644 rust/capnp/any_pointer_list.rs
 create mode 100644 rust/capnp/capability.rs
 create mode 100644 rust/capnp/capability_list.rs
 create mode 100644 rust/capnp/constant.rs
 create mode 100644 rust/capnp/data.rs
 create mode 100644 rust/capnp/data_list.rs
 create mode 100644 rust/capnp/dynamic_list.rs
 create mode 100644 rust/capnp/dynamic_struct.rs
 create mode 100644 rust/capnp/dynamic_value.rs
 create mode 100644 rust/capnp/enum_list.rs
 create mode 100644 rust/capnp/introspect.rs
 create mode 100644 rust/capnp/io.rs
 create mode 100644 rust/capnp/lib.rs
 create mode 100644 rust/capnp/list_list.rs
 create mode 100644 rust/capnp/message.rs
 create mode 100644 rust/capnp/primitive_list.rs
 create mode 100644 rust/capnp/private/arena.rs
 create mode 100644 rust/capnp/private/capability.rs
 create mode 100644 rust/capnp/private/layout.rs
 create mode 100644 rust/capnp/private/layout_test.rs
 create mode 100644 rust/capnp/private/mask.rs
 create mode 100644 rust/capnp/private/mod.rs
 create mode 100644 rust/capnp/private/primitive.rs
 create mode 100644 rust/capnp/private/read_limiter.rs
 create mode 100644 rust/capnp/private/units.rs
 create mode 100644 rust/capnp/private/zero.rs
 create mode 100644 rust/capnp/raw.rs
 create mode 100644 rust/capnp/schema.rs
 create mode 100644 rust/capnp/schema_capnp.rs
 create mode 100644 rust/capnp/serialize.rs
 create mode 100644 rust/capnp/serialize/no_alloc_buffer_segments.rs
 create mode 100644 rust/capnp/serialize_packed.rs
 create mode 100644 rust/capnp/stringify.rs
 create mode 100644 rust/capnp/struct_list.rs
 create mode 100644 rust/capnp/text.rs
 create mode 100644 rust/capnp/text_list.rs
 create mode 100644 rust/capnp/traits.rs
rust/capnp/any_pointer.rs new file +313 -0
new file mode 100644
index 000000000000..c49216cd031e
@@ -0,0 +1,313 @@
+// Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
+// Licensed under the MIT License:
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+//
+// The above copyright notice and this permission notice shall be included in
+// all copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+// THE SOFTWARE.
+
+//! Untyped pointer that can be cast to any struct, list, or capability type.
+
+#[cfg(feature = "alloc")]
+use alloc::{boxed::Box, vec::Vec};
+
+#[cfg(feature = "alloc")]
+use crate::capability::FromClientHook;
+#[cfg(feature = "alloc")]
+use crate::private::capability::{ClientHook, PipelineHook, PipelineOp};
+use crate::private::layout::{PointerBuilder, PointerReader};
+use crate::traits::{FromPointerBuilder, FromPointerReader, SetPointerBuilder};
+use crate::Result;
+
+#[derive(Copy, Clone)]
+pub struct Owned(());
+
+impl crate::traits::Owned for Owned {
+    type Reader<'a> = Reader<'a>;
+    type Builder<'a> = Builder<'a>;
+}
+
+impl crate::introspect::Introspect for Owned {
+    fn introspect() -> crate::introspect::Type {
+        crate::introspect::TypeVariant::AnyPointer.into()
+    }
+}
+
+impl crate::traits::Pipelined for Owned {
+    type Pipeline = Pipeline;
+}
+
+#[derive(Copy, Clone)]
+pub struct Reader<'a> {
+    pub(crate) reader: PointerReader<'a>,
+}
+
+impl<'a> Reader<'a> {
+    pub fn new(reader: PointerReader<'_>) -> Reader<'_> {
+        Reader { reader }
+    }
+
+    #[inline]
+    pub fn is_null(&self) -> bool {
+        self.reader.is_null()
+    }
+
+    /// Gets the total size of the target and all of its children. Does not count far pointer overhead.
+    pub fn target_size(&self) -> Result<crate::MessageSize> {
+        self.reader.total_size()
+    }
+
+    #[inline]
+    pub fn get_as<T: FromPointerReader<'a>>(&self) -> Result<T> {
+        FromPointerReader::get_from_pointer(&self.reader, None)
+    }
+
+    #[cfg(feature = "alloc")]
+    pub fn get_as_capability<T: FromClientHook>(&self) -> Result<T> {
+        Ok(FromClientHook::new(self.reader.get_capability()?))
+    }
+
+    //# Used by RPC system to implement pipelining. Applications
+    //# generally shouldn't use this directly.
+    #[cfg(feature = "alloc")]
+    pub fn get_pipelined_cap(&self, ops: &[PipelineOp]) -> Result<Box<dyn ClientHook>> {
+        let mut pointer = self.reader;
+
+        for op in ops {
+            match *op {
+                PipelineOp::Noop => {}
+                PipelineOp::GetPointerField(idx) => {
+                    pointer = pointer.get_struct(None)?.get_pointer_field(idx as usize);
+                }
+            }
+        }
+
+        pointer.get_capability()
+    }
+}
+
+impl<'a> FromPointerReader<'a> for Reader<'a> {
+    fn get_from_pointer(
+        reader: &PointerReader<'a>,
+        default: Option<&'a [crate::Word]>,
+    ) -> Result<Reader<'a>> {
+        if default.is_some() {
+            panic!("Unsupported: any_pointer with a default value.");
+        }
+        Ok(Reader { reader: *reader })
+    }
+}
+
+impl<'a> crate::traits::SetPointerBuilder for Reader<'a> {
+    fn set_pointer_builder<'b>(
+        mut pointer: crate::private::layout::PointerBuilder<'b>,
+        value: Reader<'a>,
+        canonicalize: bool,
+    ) -> Result<()> {
+        pointer.copy_from(value.reader, canonicalize)
+    }
+}
+
+#[cfg(feature = "alloc")]
+impl<'a> crate::traits::Imbue<'a> for Reader<'a> {
+    fn imbue(&mut self, cap_table: &'a crate::private::layout::CapTable) {
+        self.reader
+            .imbue(crate::private::layout::CapTableReader::Plain(cap_table));
+    }
+}
+
+pub struct Builder<'a> {
+    builder: PointerBuilder<'a>,
+}
+
+impl<'a> Builder<'a> {
+    pub fn new(builder: PointerBuilder<'a>) -> Builder<'a> {
+        Builder { builder }
+    }
+
+    pub fn reborrow(&mut self) -> Builder<'_> {
+        Builder {
+            builder: self.builder.reborrow(),
+        }
+    }
+
+    pub fn is_null(&self) -> bool {
+        self.builder.is_null()
+    }
+
+    /// Gets the total size of the target and all of its children. Does not count far pointer overhead.
+    pub fn target_size(&self) -> Result<crate::MessageSize> {
+        self.builder.as_reader().total_size()
+    }
+
+    pub fn get_as<T: FromPointerBuilder<'a>>(self) -> Result<T> {
+        FromPointerBuilder::get_from_pointer(self.builder, None)
+    }
+
+    pub fn init_as<T: FromPointerBuilder<'a>>(self) -> T {
+        FromPointerBuilder::init_pointer(self.builder, 0)
+    }
+
+    pub fn initn_as<T: FromPointerBuilder<'a>>(self, size: u32) -> T {
+        FromPointerBuilder::init_pointer(self.builder, size)
+    }
+
+    pub fn set_as<From: SetPointerBuilder>(&mut self, value: From) -> Result<()> {
+        SetPointerBuilder::set_pointer_builder(self.builder.reborrow(), value, false)
+    }
+
+    // XXX value should be a user client.
+    #[cfg(feature = "alloc")]
+    pub fn set_as_capability(&mut self, value: Box<dyn ClientHook>) {
+        self.builder.set_capability(value);
+    }
+
+    #[inline]
+    pub fn clear(&mut self) {
+        self.builder.clear()
+    }
+
+    pub fn into_reader(self) -> Reader<'a> {
+        Reader {
+            reader: self.builder.into_reader(),
+        }
+    }
+}
+
+impl<'a> FromPointerBuilder<'a> for Builder<'a> {
+    fn init_pointer(mut builder: PointerBuilder<'a>, _len: u32) -> Builder<'a> {
+        if !builder.is_null() {
+            builder.clear();
+        }
+        Builder { builder }
+    }
+    fn get_from_pointer(
+        builder: PointerBuilder<'a>,
+        default: Option<&'a [crate::Word]>,
+    ) -> Result<Builder<'a>> {
+        if default.is_some() {
+            panic!("AnyPointer defaults are unsupported")
+        }
+        Ok(Builder { builder })
+    }
+}
+
+#[cfg(feature = "alloc")]
+impl<'a> crate::traits::ImbueMut<'a> for Builder<'a> {
+    fn imbue_mut(&mut self, cap_table: &'a mut crate::private::layout::CapTable) {
+        self.builder
+            .imbue(crate::private::layout::CapTableBuilder::Plain(cap_table));
+    }
+}
+
+pub struct Pipeline {
+    // XXX this should not be public
+    #[cfg(feature = "alloc")]
+    pub hook: Box<dyn PipelineHook>,
+
+    #[cfg(feature = "alloc")]
+    ops: Vec<PipelineOp>,
+}
+
+impl Pipeline {
+    #[cfg(feature = "alloc")]
+    pub fn new(hook: Box<dyn PipelineHook>) -> Self {
+        Self {
+            hook,
+            ops: Vec::new(),
+        }
+    }
+
+    #[cfg(feature = "alloc")]
+    pub fn noop(&self) -> Self {
+        Self {
+            hook: self.hook.add_ref(),
+            ops: self.ops.clone(),
+        }
+    }
+
+    #[cfg(not(feature = "alloc"))]
+    pub fn noop(&self) -> Self {
+        Self {}
+    }
+
+    #[cfg(feature = "alloc")]
+    pub fn get_pointer_field(&self, pointer_index: u16) -> Self {
+        let mut new_ops = Vec::with_capacity(self.ops.len() + 1);
+        for op in &self.ops {
+            new_ops.push(*op)
+        }
+        new_ops.push(PipelineOp::GetPointerField(pointer_index));
+        Self {
+            hook: self.hook.add_ref(),
+            ops: new_ops,
+        }
+    }
+
+    #[cfg(not(feature = "alloc"))]
+    pub fn get_pointer_field(&self, _pointer_index: u16) -> Self {
+        Self {}
+    }
+
+    #[cfg(feature = "alloc")]
+    pub fn as_cap(&self) -> Box<dyn ClientHook> {
+        self.hook.get_pipelined_cap(&self.ops)
+    }
+}
+
+impl crate::capability::FromTypelessPipeline for Pipeline {
+    fn new(typeless: Pipeline) -> Self {
+        typeless
+    }
+}
+
+impl<'a> From<Reader<'a>> for crate::dynamic_value::Reader<'a> {
+    fn from(a: Reader<'a>) -> crate::dynamic_value::Reader<'a> {
+        crate::dynamic_value::Reader::AnyPointer(a)
+    }
+}
+
+impl<'a> From<Builder<'a>> for crate::dynamic_value::Builder<'a> {
+    fn from(a: Builder<'a>) -> crate::dynamic_value::Builder<'a> {
+        crate::dynamic_value::Builder::AnyPointer(a)
+    }
+}
+
+#[cfg(feature = "alloc")]
+#[test]
+fn init_clears_value() {
+    let mut message = crate::message::Builder::new_default();
+    {
+        let root: crate::any_pointer::Builder = message.init_root();
+        let mut list: crate::primitive_list::Builder<u16> = root.initn_as(10);
+        for idx in 0..10 {
+            list.set(idx, idx as u16);
+        }
+    }
+
+    {
+        let root: crate::any_pointer::Builder = message.init_root();
+        assert!(root.is_null());
+    }
+
+    let mut output: Vec<u8> = Vec::new();
+    crate::serialize::write_message(&mut output, &message).unwrap();
+    assert_eq!(output.len(), 40);
+    for byte in &output[8..] {
+        // Everything not in the message header is zero.
+        assert_eq!(*byte, 0u8);
+    }
+}
rust/capnp/any_pointer_list.rs new file +208 -0

208 lines not on this page · raw message

rust/capnp/capability.rs new file +363 -0

363 lines not on this page · raw message

rust/capnp/capability_list.rs new file +297 -0

297 lines not on this page · raw message

rust/capnp/constant.rs new file +54 -0

54 lines not on this page · raw message

rust/capnp/data.rs new file +95 -0

95 lines not on this page · raw message

rust/capnp/data_list.rs new file +218 -0

218 lines not on this page · raw message

rust/capnp/dynamic_list.rs new file +408 -0

408 lines not on this page · raw message

rust/capnp/dynamic_struct.rs new file +782 -0

782 lines not on this page · raw message

rust/capnp/dynamic_value.rs new file +317 -0

262 lines not on this page · raw message

rust/capnp/enum_list.rs new file +237 -0

237 lines not on this page · raw message

rust/capnp/introspect.rs new file +282 -0

282 lines not on this page · raw message

rust/capnp/io.rs new file +202 -0

202 lines not on this page · raw message

rust/capnp/lib.rs new file +651 -0

651 lines not on this page · raw message

rust/capnp/list_list.rs new file +296 -0

296 lines not on this page · raw message

rust/capnp/message.rs new file +878 -0

878 lines not on this page · raw message

rust/capnp/primitive_list.rs new file +279 -0

279 lines not on this page · raw message

rust/capnp/private/arena.rs new file +417 -0

417 lines not on this page · raw message

rust/capnp/private/capability.rs new file +155 -0

155 lines not on this page · raw message

rust/capnp/private/layout.rs new file +4210 -0

4,210 lines not on this page · raw message

rust/capnp/private/layout_test.rs new file +192 -0

192 lines not on this page · raw message

rust/capnp/private/mask.rs new file +62 -0

62 lines not on this page · raw message

rust/capnp/private/mod.rs new file +36 -0
new file mode 100644
index 000000000000..1e6bcdab6d6e

Lines of this file are not shown. View the raw message.

rust/capnp/private/primitive.rs new file +119 -0

119 lines not on this page · raw message

rust/capnp/private/read_limiter.rs new file +113 -0

113 lines not on this page · raw message

rust/capnp/private/units.rs new file +68 -0

68 lines not on this page · raw message

rust/capnp/private/zero.rs new file +46 -0

46 lines not on this page · raw message

rust/capnp/raw.rs new file +69 -0

69 lines not on this page · raw message

rust/capnp/schema.rs new file +430 -0

430 lines not on this page · raw message

rust/capnp/schema_capnp.rs new file +14448 -0

14,448 lines not on this page · raw message

rust/capnp/serialize.rs new file +961 -0

961 lines not on this page · raw message

rust/capnp/serialize/no_alloc_buffer_segments.rs new file +627 -0

627 lines not on this page · raw message

rust/capnp/serialize_packed.rs new file +618 -0

618 lines not on this page · raw message

rust/capnp/stringify.rs new file +181 -0

181 lines not on this page · raw message

rust/capnp/struct_list.rs new file +300 -0

300 lines not on this page · raw message

rust/capnp/text.rs new file +294 -0

294 lines not on this page · raw message

rust/capnp/text_list.rs new file +216 -0

216 lines not on this page · raw message

rust/capnp/traits.rs new file +233 -0

233 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.34.1

Part of the diff is not shown here. View the raw message.

#7 lore raw

RFC PATCH v3 7/22 rust: capnp: add SPDX License Identifiers

In reply to Ariel Miculas

Add the SPDX license identifiers, since the initial patch added the upstream sources with no modifications whatsoever.

---
 rust/capnp/any_pointer.rs                        | 2 ++
 rust/capnp/any_pointer_list.rs                   | 2 ++
 rust/capnp/capability.rs                         | 2 ++
 rust/capnp/capability_list.rs                    | 2 ++
 rust/capnp/constant.rs                           | 2 ++
 rust/capnp/data.rs                               | 2 ++
 rust/capnp/data_list.rs                          | 2 ++
 rust/capnp/dynamic_list.rs                       | 2 ++
 rust/capnp/dynamic_struct.rs                     | 2 ++
 rust/capnp/dynamic_value.rs                      | 2 ++
 rust/capnp/enum_list.rs                          | 2 ++
 rust/capnp/introspect.rs                         | 2 ++
 rust/capnp/io.rs                                 | 2 ++
 rust/capnp/lib.rs                                | 2 ++
 rust/capnp/list_list.rs                          | 2 ++
 rust/capnp/message.rs                            | 2 ++
 rust/capnp/primitive_list.rs                     | 2 ++
 rust/capnp/private/arena.rs                      | 2 ++
 rust/capnp/private/capability.rs                 | 2 ++
 rust/capnp/private/layout.rs                     | 2 ++
 rust/capnp/private/layout_test.rs                | 2 ++
 rust/capnp/private/mask.rs                       | 2 ++
 rust/capnp/private/mod.rs                        | 2 ++
 rust/capnp/private/primitive.rs                  | 2 ++
 rust/capnp/private/read_limiter.rs               | 2 ++
 rust/capnp/private/units.rs                      | 2 ++
 rust/capnp/private/zero.rs                       | 2 ++
 rust/capnp/raw.rs                                | 2 ++
 rust/capnp/schema.rs                             | 2 ++
 rust/capnp/schema_capnp.rs                       | 2 ++
 rust/capnp/serialize.rs                          | 2 ++
 rust/capnp/serialize/no_alloc_buffer_segments.rs | 2 ++
 rust/capnp/serialize_packed.rs                   | 2 ++
 rust/capnp/stringify.rs                          | 2 ++
 rust/capnp/struct_list.rs                        | 2 ++
 rust/capnp/text.rs                               | 2 ++
 rust/capnp/text_list.rs                          | 2 ++
 rust/capnp/traits.rs                             | 2 ++
 38 files changed, 76 insertions(+)
rust/capnp/any_pointer.rs +2 -0
index c49216cd031e..83ae07a20a09 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/any_pointer_list.rs +2 -0
index 1c4e9e40c879..f7f4fae5678c 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2018 the capnproto-rust contributors
 //
 // Permission is hereby granted, free of charge, to any person obtaining a copy
rust/capnp/capability.rs +2 -0
index 7cff20bbbcb9..fbe778791c38 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/capability_list.rs +2 -0
index 083794e073d1..b139d566a58d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2017 David Renshaw and contributors
 //
 // Permission is hereby granted, free of charge, to any person obtaining a copy
rust/capnp/constant.rs +2 -0
index fdd1edf487ff..9f174366d89d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2017 Sandstorm Development Group, Inc. and contributors
 //
 // Permission is hereby granted, free of charge, to any person obtaining a copy
rust/capnp/data.rs +2 -0
index 89e0c6c19ea4..d777f1d626b0 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/data_list.rs +2 -0
index f5fee2575fd2..7771ea4bb04d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/dynamic_list.rs +2 -0
index f2c93e1cbe16..0b60674cdeab 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 //! Dynamically-typed lists.
 
 use crate::dynamic_value;
rust/capnp/dynamic_struct.rs +2 -0
index d0244fc29f40..9da5a1e5df33 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 //! Dynamically-typed structs.
 
 use crate::introspect::TypeVariant;
rust/capnp/dynamic_value.rs +2 -0
index 83646e26de31..aaca4d82453f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 //! Dynamically typed values.
 
 use crate::introspect::{self, TypeVariant};
rust/capnp/enum_list.rs +2 -0
index 8fa713029250..0192732fe16b 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/introspect.rs +2 -0
index d994b180c93a..a3494935b4d5 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 //! Traits and types to support run-time type introspection, i.e. reflection.
 
 use crate::private::layout::ElementSize;
rust/capnp/io.rs +2 -0
index 59d149b37c26..82c429f8c263 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 //! Custom I/O traits that roughly mirror `std::io::{Read, BufRead, Write}`.
 //! This extra layer of indirection enables support of no-std environments.
 
rust/capnp/lib.rs +2 -0
index daafc23c19d5..d3aa0f9f3451 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/list_list.rs +2 -0
index 7c31560958d3..ec7e830df7e1 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/message.rs +2 -0
index fe40e0a7c313..595f6975c489 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/primitive_list.rs +2 -0
index 4ea31a7b50eb..a761320788b5 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/arena.rs +2 -0
index 46626c1f0ce3..40c539d72142 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2017 Sandstorm Development Group, Inc. and contributors
 //
 // Permission is hereby granted, free of charge, to any person obtaining a copy
rust/capnp/private/capability.rs +2 -0
index ba725fb60535..522d0b1b0418 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/layout.rs +2 -0
index 02ecb6e7bfb5..dbfe97235cd9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/layout_test.rs +2 -0
index 8715e9edab43..fb5a8de9bb44 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/mask.rs +2 -0
index dfa211521ca7..70937b802bf6 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/mod.rs +2 -0
index 1e6bcdab6d6e..2126235199fe 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/primitive.rs +2 -0
index 00e41ce40f08..f70f3a7486f3 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 pub trait Primitive {
     type Raw;
 
rust/capnp/private/read_limiter.rs +2 -0
index 0bb6a7a8415c..f4f0cf348af1 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/units.rs +2 -0
index 90fbf663f777..37782edeef56 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/private/zero.rs +2 -0
index 00769cfebafc..12e4a723105a 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/raw.rs +2 -0
index 5883b1207be9..3127b6adc0fc 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2018 the capnproto-rust contributors
 //
 // Permission is hereby granted, free of charge, to any person obtaining a copy
rust/capnp/schema.rs +2 -0
index 4120868f4051..b7677056cc10 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 //! Convenience wrappers of the datatypes defined in schema.capnp.
 
 use crate::dynamic_value;
rust/capnp/schema_capnp.rs +2 -0
index 4f92ee288821..820a1187bf59 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // @generated by the capnpc-rust plugin to the Cap'n Proto schema compiler.
 // DO NOT EDIT.
 // source: schema.capnp
rust/capnp/serialize.rs +2 -0
index ad9e130e5ed7..025bcd45ee61 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/serialize/no_alloc_buffer_segments.rs +2 -0
index d3837738d702..de902a0ee4d4 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 use core::convert::TryInto;
 
 use crate::message::ReaderOptions;
rust/capnp/serialize_packed.rs +2 -0
index 286a36950981..63ed7264848e 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/stringify.rs +2 -0
index d11bfbeb7b14..426025fea0e8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 use crate::dynamic_value;
 use core::fmt::{self, Formatter};
 
rust/capnp/struct_list.rs +2 -0
index 5c2775ddc1fd..7133e69560bc 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/text.rs +2 -0
index fa2a72701444..406ddb5ba8ac 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/text_list.rs +2 -0
index 72cbd2647ba9..222521a6741e 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
rust/capnp/traits.rs +2 -0
index 39e050ab4abf..e12c38346d56 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: MIT
+
 // Copyright (c) 2013-2015 Sandstorm Development Group, Inc. and contributors
 // Licensed under the MIT License:
 //
Signature · 1 line
2.34.1
#8 lore raw

RFC PATCH v3 8/22 rust: capnp: return an error when trying to display floating point values

In reply to Ariel Miculas

The kernel doesn't support floating point values, so return an error when the kernel feature is enabled.

---
 rust/capnp/stringify.rs | 4 ++++
 1 file changed, 4 insertions(+)
rust/capnp/stringify.rs +4 -0
index 426025fea0e8..f93e3521ce0b 100644
@@ -71,8 +71,12 @@ pub(crate) fn print(
         dynamic_value::Reader::UInt16(x) => formatter.write_fmt(format_args!("{x}")),
         dynamic_value::Reader::UInt32(x) => formatter.write_fmt(format_args!("{x}")),
         dynamic_value::Reader::UInt64(x) => formatter.write_fmt(format_args!("{x}")),
+        #[cfg(not(feature = "kernel"))]
         dynamic_value::Reader::Float32(x) => formatter.write_fmt(format_args!("{x}")),
+        #[cfg(not(feature = "kernel"))]
         dynamic_value::Reader::Float64(x) => formatter.write_fmt(format_args!("{x}")),
+        #[cfg(feature = "kernel")]
+        dynamic_value::Reader::Float32(_) | dynamic_value::Reader::Float64(_) => Err(fmt::Error),
         dynamic_value::Reader::Enum(e) => match cvt(e.get_enumerant())? {
             Some(enumerant) => {
                 formatter.write_str(cvt(cvt(enumerant.get_proto().get_name())?.to_str())?)
Signature · 1 line
2.34.1
#9 lore raw

RFC PATCH v3 9/22 rust: Kbuild: enable `capnp`

In reply to Ariel Miculas

With all the new files in place for capnp, this patch adds support for it in the build system.

---
 rust/Makefile          | 23 ++++++++++++++++++++---
 scripts/Makefile.build |  2 +-
 2 files changed, 21 insertions(+), 4 deletions(-)
rust/Makefile +20 -3
index fef8e38fbebd..f088ff4bdd34 100644
@@ -15,7 +15,7 @@ always-$(CONFIG_RUST) += libmacros.so
 no-clean-files += libmacros.so
 
 always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
-obj-$(CONFIG_RUST) += alloc.o bindings.o hex.o kernel.o
+obj-$(CONFIG_RUST) += alloc.o bindings.o hex.o capnp.o kernel.o
 always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
     exports_kernel_generated.h
 
@@ -74,6 +74,16 @@ hex-flags := \
     --edition=2018 \
     -Amissing_docs
 
+capnp-skip_flags := \
+    -Drust_2018_idioms \
+    -Dunsafe_op_in_unsafe_fn \
+    -Dunreachable_pub
+
+capnp-flags := \
+    -Amissing_docs \
+    --cfg 'feature="kernel"' \
+    --cfg 'feature="sync_reader"'
+
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
       cmd_rustdoc = \
 	OBJTREE=$(abspath $(objtree)) \
@@ -470,11 +480,18 @@ $(obj)/hex.o: private rustc_target_flags = $(hex-flags)
 $(obj)/hex.o: $(src)/hex/lib.rs $(obj)/compiler_builtins.o FORCE
 	$(call if_changed_dep,rustc_library)
 
+$(obj)/canpproto.o: private skip_clippy = 1
+$(obj)/capnp.o: private skip_flags = $(capnp-skip_flags)
+$(obj)/capnp.o: private rustc_target_flags = $(capnp-flags)
+$(obj)/capnp.o: $(src)/capnp/lib.rs $(obj)/compiler_builtins.o FORCE
+	$(call if_changed_dep,rustc_library)
+
 $(obj)/kernel.o: private rustc_target_flags = --extern alloc \
     --extern build_error --extern macros --extern bindings --extern uapi \
-    --extern hex
+    --extern hex --extern capnp
 $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
-    $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o $(obj)/hex.o FORCE
+    $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o $(obj)/hex.o \
+    $(obj)/capnp.o FORCE
 	$(call if_changed_dep,rustc_library)
 
 endif # CONFIG_RUST
scripts/Makefile.build +1 -1
index 2e9a64224c51..8a500593ea05 100644
@@ -273,7 +273,7 @@ rust_common_cmd = \
 	-Zallow-features=$(rust_allowed_features) \
 	-Zcrate-attr=no_std \
 	-Zcrate-attr='feature($(rust_allowed_features))' \
-	--extern alloc --extern kernel --extern hex \
+	--extern alloc --extern kernel --extern hex --extern capnp \
 	--crate-type rlib -L $(objtree)/rust/ \
 	--crate-name $(basename $(notdir $@)) \
 	--sysroot=/dev/null \
Signature · 1 line
2.34.1
#10 lore raw

RFC PATCH v3 10/22 rust: kernel: add an abstraction over vfsmount to allow cloning a new private mount

In reply to Ariel Miculas

Provide an abstraction over vfsmount so that we can create a new private mount from Rust code.

PuzzleFS needs a private mount of the PuzzleFS image, so it can access the chunks from the data store independent of the future changes to the mount namespace.

---
 rust/bindings/bindings_helper.h |  1 +
 rust/kernel/lib.rs              |  1 +
 rust/kernel/mount.rs            | 73 +++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+)
 create mode 100644 rust/kernel/mount.rs
rust/bindings/bindings_helper.h +1 -0
index 629fce394dbe..2d87bb9f87c9 100644
@@ -16,6 +16,7 @@
 #include <linux/iomap.h>
 #include <linux/jiffies.h>
 #include <linux/mdio.h>
+#include <linux/namei.h>
 #include <linux/pagemap.h>
 #include <linux/phy.h>
 #include <linux/refcount.h>
rust/kernel/lib.rs +1 -0
index 445599d4bff6..7f0d89b902ce 100644
@@ -37,6 +37,7 @@
 #[cfg(CONFIG_KUNIT)]
 pub mod kunit;
 pub mod mem_cache;
+pub mod mount;
 #[cfg(CONFIG_NET)]
 pub mod net;
 pub mod prelude;
rust/kernel/mount.rs new file +73 -0

73 lines not on this page · raw message

Signature · 1 line
2.34.1
#11 lore raw

RFC PATCH v3 11/22 rust: file: add bindings for `struct file`

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

Using these bindings it becomes possible to access files from drivers written in Rust. This patch only adds support for accessing the flags, and for managing the refcount of the file.

---
 rust/bindings/bindings_helper.h |   1 +
 rust/helpers.c                  |  16 +++
 rust/kernel/file.rs             | 176 ++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs              |   1 +
 4 files changed, 194 insertions(+)
 create mode 100644 rust/kernel/file.rs
rust/bindings/bindings_helper.h +1 -0
index 2d87bb9f87c9..0491bb05270c 100644
@@ -13,6 +13,7 @@
 #include <linux/file.h>
 #include <linux/fs.h>
 #include <linux/fs_context.h>
+#include <linux/fs_parser.h>
 #include <linux/iomap.h>
 #include <linux/jiffies.h>
 #include <linux/mdio.h>
rust/helpers.c +16 -0
index c26aa07cb20f..90acf9677aa6 100644
@@ -28,6 +28,7 @@
 #include <linux/err.h>
 #include <linux/errname.h>
 #include <linux/fs.h>
+#include <linux/fs_parser.h>
 #include <linux/highmem.h>
 #include <linux/mm.h>
 #include <linux/mutex.h>
@@ -353,6 +354,21 @@ void rust_helper_set_delayed_call(struct delayed_call *call,
 }
 EXPORT_SYMBOL_GPL(rust_helper_set_delayed_call);
 
+struct inode *rust_helper_d_inode(const struct dentry *dentry)
+{
+	return d_inode(dentry);
+}
+EXPORT_SYMBOL_GPL(rust_helper_d_inode);
+
+int rust_helper_fs_parse(struct fs_context *fc,
+		const struct fs_parameter_spec *desc,
+		struct fs_parameter *param,
+		struct fs_parse_result *result)
+{
+	return fs_parse(fc, desc, param, result);
+}
+EXPORT_SYMBOL_GPL(rust_helper_fs_parse);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.
rust/kernel/file.rs new file +176 -0

176 lines not on this page · raw message

rust/kernel/lib.rs +1 -0
index 7f0d89b902ce..ea1411a25ee4 100644
@@ -30,6 +30,7 @@
 pub mod block;
 mod build_assert;
 pub mod error;
+pub mod file;
 pub mod folio;
 pub mod fs;
 pub mod init;
Signature · 1 line
2.34.1
#12 lore raw

RFC PATCH v3 12/22 rust: file: Add support for reading files using their path

In reply to Ariel Miculas

Implement from_path, from_path_in_root_mnt, read_with_offset, read_to_end and get_file_size methods for a RegularFile newtype. kernel_read_file is used under the hood for reading files.

These functions will be used for reading the PuzzleFS metadata files and the chunks in the data store.

---
 rust/kernel/error.rs |   2 +-
 rust/kernel/file.rs  | 124 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 124 insertions(+), 2 deletions(-)
rust/kernel/error.rs +1 -1
index edada157879a..3cf916bc884f 100644
@@ -263,7 +263,7 @@ pub fn to_result(err: core::ffi::c_int) -> Result {
 ///     from_err_ptr(unsafe { bindings::devm_platform_ioremap_resource(pdev.to_ptr(), index) })
 /// }
 /// ```
-pub(crate) fn from_err_ptr<T>(ptr: *mut T) -> Result<*mut T> {
+pub fn from_err_ptr<T>(ptr: *mut T) -> Result<*mut T> {
     // CAST: Casting a pointer to `*const core::ffi::c_void` is always valid.
     let const_ptr: *const core::ffi::c_void = ptr.cast();
     // SAFETY: The FFI function does not deref the pointer.
rust/kernel/file.rs +123 -1

140 lines not on this page · raw message

Signature · 1 line
2.34.1
#13 lore raw

RFC PATCH v3 13/22 fs: puzzlefs: Implement the initial version of PuzzleFS

In reply to Ariel Miculas

Implement the file system trait fs::FileSystem for PuzzleFS, directory iteration, inode lookup and the read_folio operations. The puzzlefs image path and the manifest digest are hard-coded for now due to the lack of module parameters support.

Puzzlefs uses capnproto for storing the filesystem metadata, defined in manifest.capnp and metadata.capnp. The files manifest_capnp.rs and metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4):

```
capnp compile -orust manifest.capnp
capnp compile -orust metadata.capnp
```
, formatted with rustfmt and I have also added the following line:
```
 #![allow(unreachable_pub)]
 #![allow(dead_code)]
```

to avoid warnings.

capnp version used:

```
$ capnp --version
Cap'n Proto version 0.10.4

$ cargo install --list | grep capnp
capnpc v0.18.0:
    capnpc-rust
    capnpc-rust-bootstrap
```

Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically generated at build time. However, this means the capnp binary becomes a dependency for the build and I didn't want this. Besides, the metadata schemas are not expected to change frequently, so it's acceptable to manually regenerate the two rust files when this happens.

The code is adapted from the puzzlefs FUSE driver [1]. The data structures required for the filesystem metadata are defined in types.rs. Each structure has a from_capnp method used to convert between the capnp memory layout and the native Rust structures. This leaves room for improvement [2].

inode.rs implements the PuzzleFS structure which contains a list of metadata layers.

---
 fs/Kconfig                                 |    1 +
 fs/Makefile                                |    1 +
 fs/puzzlefs/Kconfig                        |   14 +
 fs/puzzlefs/Makefile                       |    8 +
 fs/puzzlefs/puzzle.rs                      |    5 +
 fs/puzzlefs/puzzle/error.rs                |  105 +
 fs/puzzlefs/puzzle/inode.rs                |  100 +
 fs/puzzlefs/puzzle/oci.rs                  |   70 +
 fs/puzzlefs/puzzle/types.rs                |  377 ++
 fs/puzzlefs/puzzle/types/manifest.capnp    |   15 +
 fs/puzzlefs/puzzle/types/manifest_capnp.rs |  757 ++++
 fs/puzzlefs/puzzle/types/metadata.capnp    |   65 +
 fs/puzzlefs/puzzle/types/metadata_capnp.rs | 4049 ++++++++++++++++++++
 fs/puzzlefs/puzzlefs.rs                    |  227 ++
 rust/kernel/error.rs                       |    2 +-
 15 files changed, 5795 insertions(+), 1 deletion(-)
 create mode 100644 fs/puzzlefs/Kconfig
 create mode 100644 fs/puzzlefs/Makefile
 create mode 100644 fs/puzzlefs/puzzle.rs
 create mode 100644 fs/puzzlefs/puzzle/error.rs
 create mode 100644 fs/puzzlefs/puzzle/inode.rs
 create mode 100644 fs/puzzlefs/puzzle/oci.rs
 create mode 100644 fs/puzzlefs/puzzle/types.rs
 create mode 100644 fs/puzzlefs/puzzle/types/manifest.capnp
 create mode 100644 fs/puzzlefs/puzzle/types/manifest_capnp.rs
 create mode 100644 fs/puzzlefs/puzzle/types/metadata.capnp
 create mode 100644 fs/puzzlefs/puzzle/types/metadata_capnp.rs
 create mode 100644 fs/puzzlefs/puzzlefs.rs
fs/Kconfig +1 -0
index a46b0cbc4d8f..88ac055d6b49 100644
@@ -337,6 +337,7 @@ source "fs/sysv/Kconfig"
 source "fs/ufs/Kconfig"
 source "fs/erofs/Kconfig"
 source "fs/vboxsf/Kconfig"
+source "fs/puzzlefs/Kconfig"
 
 endif # MISC_FILESYSTEMS
 
fs/Makefile +1 -0
index 6ecc9b0a53f2..5cfc91653b4a 100644
@@ -129,3 +129,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
 obj-$(CONFIG_EROFS_FS)		+= erofs/
 obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
 obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
+obj-$(CONFIG_PUZZLEFS_FS)	+= puzzlefs/
fs/puzzlefs/Kconfig new file +14 -0
new file mode 100644
index 000000000000..256fb542dc92
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+config PUZZLEFS_FS
+	tristate "PuzzleFS file system support"
+	depends on RUST
+	help
+	  This option builds PuzzleFS, a next-generation container filesystem.
+
+	  To compile this as a module, choose M here:
+	  the module will be called puzzlefs.
+
+	  If unsure, say N.
+
fs/puzzlefs/Makefile new file +8 -0
new file mode 100644
index 000000000000..24eb4ff20224
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the linux puzzlefs filesystem.
+#
+
+obj-$(CONFIG_PUZZLEFS_FS) += puzzlefs.o
+
+puzzlefs-y := puzzlefs.o
fs/puzzlefs/puzzle.rs new file +5 -0
new file mode 100644
index 000000000000..e74a248c39ff
@@ -0,0 +1,5 @@
+pub(crate) mod error;
+pub(crate) mod types;
+pub(crate) use types::{manifest_capnp, metadata_capnp};
+pub(crate) mod inode;
+pub(crate) mod oci;
fs/puzzlefs/puzzle/error.rs new file +105 -0

105 lines not on this page · raw message

fs/puzzlefs/puzzle/inode.rs new file +100 -0

100 lines not on this page · raw message

fs/puzzlefs/puzzle/oci.rs new file +70 -0

70 lines not on this page · raw message

fs/puzzlefs/puzzle/types.rs new file +377 -0

377 lines not on this page · raw message

fs/puzzlefs/puzzle/types/manifest.capnp new file +15 -0
new file mode 100644
index 000000000000..24d8a6f1fe19
@@ -0,0 +1,15 @@
+@0xe0b6d460b1f22bb5;
+
+using Metadata = import "metadata.capnp";
+
+struct VerityData {
+        digest@0: Data;
+        verity@1: Data;
+}
+
+struct Rootfs {
+        metadatas@0: List(Metadata.BlobRef);
+        fsVerityData@1: List(VerityData);
+        manifestVersion@2: UInt64;
+}
+
fs/puzzlefs/puzzle/types/manifest_capnp.rs new file +757 -0

757 lines not on this page · raw message

fs/puzzlefs/puzzle/types/metadata.capnp new file +65 -0

65 lines not on this page · raw message

fs/puzzlefs/puzzle/types/metadata_capnp.rs new file +4049 -0

1,473 lines not on this page · raw message

fs/puzzlefs/puzzlefs.rs new file +227 -0

227 lines not on this page · raw message

rust/kernel/error.rs +1 -1
index 3cf916bc884f..f8c9aa485844 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.34.1

Part of the diff is not shown here. View the raw message.

#14 lore raw

RFC PATCH v3 14/22 rust: kernel: add from_iter_fallible for Vec<T>

In reply to Ariel Miculas

This function takes an iterator and creates a new vector with its elements. It is used in multiple places in the PuzzleFS code.

---
 rust/kernel/alloc/vec_ext.rs | 12 ++++++++++++
 1 file changed, 12 insertions(+)
rust/kernel/alloc/vec_ext.rs +12 -0
index 6ad0b6da45d3..cbc8b06183d7 100644
@@ -132,6 +132,18 @@ fn resize(&mut self, new_len: usize, value: T, flags: Flags) -> Result<(), Alloc
     {
         self.resize_with(new_len, flags, || value.clone())
     }
+
+    /// See https://doc.rust-lang.org/std/vec/struct.Vec.html#examples-31
+    fn from_iter_fallible(
+        iter: impl Iterator<Item = T>,
+        flags: Flags,
+    ) -> Result<Vec<T>, AllocError> {
+        let mut vec = Vec::new();
+        for value in iter {
+            vec.push(value, flags)?;
+        }
+        Ok(vec)
+    }
 }
 
 impl<T> VecExt<T> for Vec<T> {
Signature · 1 line
2.34.1
#15 lore raw

RFC PATCH v3 15/22 kernel: configs: add puzzlefs config fragment

In reply to Ariel Miculas

Add a config fragment for PuzzleFS for convenience. A minimal configuration which contains PuzzleFS can be generated by running:

make LLVM=1 allnoconfig qemu-busybox-min.config puzzlefs.config

---
 kernel/configs/puzzlefs.config | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 kernel/configs/puzzlefs.config
kernel/configs/puzzlefs.config new file +4 -0
new file mode 100644
index 000000000000..e4fc4b4a9eac
@@ -0,0 +1,4 @@
+CONFIG_BUFFER_HEAD=y
+CONFIG_RUST=y
+CONFIG_MISC_FILESYSTEMS=y
+CONFIG_PUZZLEFS_FS=y
Signature · 1 line
2.34.1
#16 lore raw

RFC PATCH v3 16/22 scripts: add fs directory to rust-analyzer

In reply to Ariel Miculas

This enables the Rust code navigation features inside fs/, where PuzzleFS resides.

---
 scripts/generate_rust_analyzer.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/generate_rust_analyzer.py +1 -1
index f270c7b0cf34..6985b9e37429 100755
@@ -116,7 +116,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
     # Then, the rest outside of `rust/`.
     #
     # We explicitly mention the top-level folders we want to cover.
-    extra_dirs = map(lambda dir: srctree / dir, ("samples", "drivers"))
+    extra_dirs = map(lambda dir: srctree / dir, ("samples", "drivers", "fs"))
     if external_src is not None:
         extra_dirs = [external_src]
     for folder in extra_dirs:
Signature · 1 line
2.34.1
#17 lore raw

RFC PATCH v3 17/22 fs: puzzlefs: add extended attributes support

In reply to Ariel Miculas

Implement the listxattr callback in the filesystem abstractions. Implement both read_xattr and listxattr for PuzzleFS.

---
 fs/puzzlefs/puzzlefs.rs | 50 +++++++++++++++++++++++++++++--
 rust/kernel/fs/inode.rs | 66 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 112 insertions(+), 4 deletions(-)
fs/puzzlefs/puzzlefs.rs +48 -2
index a062bf0249f6..9622ea71eda0 100644
@@ -107,8 +107,8 @@ fn fill_super(
         _: Option<inode::Mapper>,
     ) -> Result<Box<PuzzleFS>> {
         let puzzlefs = PuzzleFS::open(
-            c_str!("/home/puzzlefs_oci"),
-            c_str!("83aa96c40a20671edc4490cfefadbb487b2ab23dfc0570049b56f0cc49b56eaf"),
+            c_str!("/home/puzzlefs_xattr"),
+            c_str!("ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953c8ab"),
         );
 
         if let Err(ref e) = puzzlefs {
@@ -124,6 +124,36 @@ fn init_root(sb: &sb::SuperBlock<Self>) -> Result<dentry::Root<Self>> {
         let inode = Self::iget(sb, 1)?;
         dentry::Root::try_new(inode)
     }
+
+    fn read_xattr(
+        _dentry: &DEntry<Self>,
+        inode: &INode<Self>,
+        name: &CStr,
+        outbuf: &mut [u8],
+    ) -> Result<usize> {
+        let inode = inode.data();
+        let readonly = outbuf.len() == 0;
+        // pr_info!("outbuf len {}\n", outbuf.len());
+
+        if let Some(add) = &inode.additional {
+            let xattr = add
+                .xattrs
+                .iter()
+                .find(|elem| elem.key == name.as_bytes())
+                .ok_or(ENODATA)?;
+            if readonly {
+                return Ok(xattr.val.len());
+            }
+
+            if xattr.val.len() > outbuf.len() {
+                return Err(ERANGE);
+            }
+
+            outbuf[0..xattr.val.len()].copy_from_slice(xattr.val.as_slice());
+            return Ok(xattr.val.len());
+        }
+        Err(ENODATA)
+    }
 }
 
 #[vtable]
@@ -143,6 +173,22 @@ fn lookup(
         }
     }
 
+    fn listxattr(
+        inode: &INode<Self>,
+        mut add_entry: impl FnMut(&[i8]) -> Result<()>,
+    ) -> Result<()> {
+        let inode = inode.data();
+
+        if let Some(add) = &inode.additional {
+            for xattr in &add.xattrs {
+                // convert a u8 slice into an i8 slice
+                let i8slice = unsafe { &*(xattr.key.as_slice() as *const _ as *const [i8]) };
+                add_entry(i8slice)?;
+            }
+        }
+        Ok(())
+    }
+
     fn get_link<'a>(
         dentry: Option<&DEntry<PuzzleFsModule>>,
         inode: &'a INode<PuzzleFsModule>,
rust/kernel/fs/inode.rs +64 -2

90 lines not on this page · raw message

Signature · 1 line
2.34.1
#18 lore raw

RFC PATCH v3 18/22 rust: add improved version of `ForeignOwnable::borrow_mut`

In reply to Ariel Miculas

From: Alice Ryhl <[email protected]>

Previously, the ForeignOwnable trait had a method called borrow_mut that was intended to provide mutable access to the inner value. However, the method accidentally made it possible to change the address of the object being modified, which usually isn't what we want. (And when we want that, it can be done by calling from_foreign and into_foreign, like how the old borrow_mut was implemented.)

In this patch, we introduce an alternate definition of borrow_mut that solves the previous problem. Conceptually, given a pointer type P that implements ForeignOwnable, the borrow_mut method gives you the same kind of access as an &mut P would, except that it does not let you change the pointer P itself.

This is analogous to how the existing borrow method provides the same kind of access to the inner value as an &P.

Note that for types like Arc, having an &mut Arc<T> only gives you immutable access to the inner T. This is because mutable references assume exclusive access, but there might be other handles to the same reference counted value, so the access isn't exclusive. The Arc type implements this by making borrow_mut return the same type as borrow.

---
 rust/kernel/sync/arc.rs | 25 +++++++----
 rust/kernel/types.rs    | 93 ++++++++++++++++++++++++++++++-----------
 2 files changed, 86 insertions(+), 32 deletions(-)
rust/kernel/sync/arc.rs +17 -8
index 3673496c2363..9023479281f0 100644
@@ -356,26 +356,35 @@ pub fn into_unique_or_drop(self) -> Option<Pin<UniqueArc<T>>> {
 
 impl<T: 'static> ForeignOwnable for Arc<T> {
     type Borrowed<'a> = ArcBorrow<'a, T>;
+    // Mutable access to the `Arc` does not give any extra abilities over
+    // immutable access.
+    type BorrowedMut<'a> = ArcBorrow<'a, T>;
 
     fn into_foreign(self) -> *const core::ffi::c_void {
         ManuallyDrop::new(self).ptr.as_ptr() as _
     }
 
+    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {
+        // SAFETY: By the safety requirement of this function, we know that `ptr` came from
+        // a previous call to `Arc::into_foreign`, which guarantees that `ptr` is valid and
+        // holds a reference count increment that is transferrable to us.
+        unsafe { Self::from_inner(NonNull::new_unchecked(ptr as _)) }
+    }
+
     unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> ArcBorrow<'a, T> {
         // SAFETY: By the safety requirement of this function, we know that `ptr` came from
         // a previous call to `Arc::into_foreign`.
-        let inner = NonNull::new(ptr as *mut ArcInner<T>).unwrap();
+        let inner = unsafe { NonNull::new_unchecked(ptr as *mut ArcInner<T>) };
 
-        // SAFETY: The safety requirements of `from_foreign` ensure that the object remains alive
-        // for the lifetime of the returned value.
+        // SAFETY: The safety requirements ensure that we will not give up our
+        // foreign-owned refcount while the `ArcBorrow` is still live.
         unsafe { ArcBorrow::new(inner) }
     }
 
-    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {
-        // SAFETY: By the safety requirement of this function, we know that `ptr` came from
-        // a previous call to `Arc::into_foreign`, which guarantees that `ptr` is valid and
-        // holds a reference count increment that is transferrable to us.
-        unsafe { Self::from_inner(NonNull::new(ptr as _).unwrap()) }
+    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> ArcBorrow<'a, T> {
+        // SAFETY: The safety requirements for `borrow_mut` are a superset of the safety
+        // requirements for `borrow`.
+        unsafe { Self::borrow(ptr) }
     }
 }
 
rust/kernel/types.rs +69 -24

140 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#19 lore raw

Re: [RFC PATCH v3 18/22] rust: add improved version of `ForeignOwnable::borrow_mut`

Quoted 25 lines · Ariel Miculas

From: Alice Ryhl <[email protected]>

Previously, the ForeignOwnable trait had a method called borrow_mut that was intended to provide mutable access to the inner value. However, the method accidentally made it possible to change the address of the object being modified, which usually isn't what we want. (And when we want that, it can be done by calling from_foreign and into_foreign, like how the old borrow_mut was implemented.)

In this patch, we introduce an alternate definition of borrow_mut that solves the previous problem. Conceptually, given a pointer type P that implements ForeignOwnable, the borrow_mut method gives you the same kind of access as an &mut P would, except that it does not let you change the pointer P itself.

This is analogous to how the existing borrow method provides the same kind of access to the inner value as an &P.

Note that for types like Arc, having an &mut Arc<T> only gives you immutable access to the inner T. This is because mutable references assume exclusive access, but there might be other handles to the same reference counted value, so the access isn't exclusive. The Arc type implements this by making borrow_mut return the same type as borrow.

Quoted 6 lines

This particular patch something that I have abandoned because I did not end up having any user for it. If you have a user for it, then I recommend that you resend my patch and ask Miguel to take it.

Alice

#20 lore raw

RFC PATCH v3 19/22 Add borrow_mut implementation to a ForeignOwnable CString

In reply to Ariel Miculas

Since the borrow_mut function was added to the ForeignOwnable trait, we need to implement it for each type that implements it.

---
 rust/kernel/str.rs | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
rust/kernel/str.rs +24 -0
index c45612900fe2..b0a35d71bd49 100644
@@ -197,6 +197,25 @@ pub unsafe fn from_char_ptr<'a>(ptr: *const core::ffi::c_char) -> &'a Self {
         unsafe { Self::from_bytes_with_nul_unchecked(bytes) }
     }
 
+    /// Like from_char_ptr, but returns a mutable reference
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must be a valid pointer to a `NUL`-terminated C string, and it must
+    /// last at least `'a`. When `CStr` is alive, the memory pointed by `ptr`
+    /// must not be mutated.
+    #[inline]
+    pub unsafe fn from_char_ptr_mut<'a>(ptr: *const core::ffi::c_char) -> &'a mut Self {
+        // SAFETY: The safety precondition guarantees `ptr` is a valid pointer
+        // to a `NUL`-terminated C string.
+        let len = unsafe { bindings::strlen(ptr) } + 1;
+        // SAFETY: Lifetime guaranteed by the safety precondition.
+        let bytes = unsafe { core::slice::from_raw_parts_mut(ptr as _, len as _) };
+        // SAFETY: As `len` is returned by `strlen`, `bytes` does not contain interior `NUL`.
+        // As we have added 1 to `len`, the last byte is known to be `NUL`.
+        unsafe { Self::from_bytes_with_nul_unchecked_mut(bytes) }
+    }
+
     /// Creates a [`CStr`] from a `[u8]`.
     ///
     /// The provided slice must be `NUL`-terminated, does not contain any
@@ -901,6 +920,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
 
 impl ForeignOwnable for CString {
     type Borrowed<'a> = &'a CStr;
+    type BorrowedMut<'a> = &'a mut CStr;
 
     fn into_foreign(self) -> *const core::ffi::c_void {
         Box::into_raw(self.buf) as _
@@ -910,6 +930,10 @@ unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> Self::Borrowed<'a> {
         unsafe { CStr::from_char_ptr(ptr.cast::<core::ffi::c_char>()) }
     }
 
+    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> Self::BorrowedMut<'a> {
+        unsafe { CStr::from_char_ptr_mut(ptr.cast::<core::ffi::c_char>()) }
+    }
+
     unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {
         // SAFETY: The safety requirements of this function satisfy those of `Self::borrow`.
         let str = unsafe { Self::borrow(ptr) };
Signature · 1 line
2.34.1
#21 lore raw

RFC PATCH v3 20/22 rust: add support for file system parameters

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

This allows file system contexts to be further initialised with parameters from userspace before a fs is mounted or reconfigured.

[
Changes made:
* adapt for upstream
* replace UnsafeCell with Opaque,
* replace from_kernel_result with from_result
* replace Type with FileSystem
* add Context in FileSystem from [1]
* add the EmptyContext from [1]
* add the free_callback from [2]
* add count_paren_items from [3]
* add PAGE_SIZE constant from [4]
* initialize the Context in init_fs_context_callback
* get the data Context in fill_super_callback and pass it to super_params
* avoid freeing s_fs_info in free_callback, it's already freed in kill_sb_callback
[1] https://github.com/wedsonaf/linux/commit/6633c50b345f388a07b4c551f752efe34c15c089
[2] https://github.com/wedsonaf/linux/commit/3c3ccce85af9aae9dfc0e17e2331674da520177c
[3] https://github.com/wedsonaf/linux/commit/c3f395b1b0126b4db7eb833bdee769212c8c79d8
[4] https://github.com/wedsonaf/linux/commit/4021d947dea69506d11b148486e1565961a85e53
]

---
 fs/puzzlefs/puzzlefs.rs         |   1 +
 rust/bindings/bindings_helper.h |   1 +
 rust/kernel/fs.rs               | 212 +++++++++++-
 rust/kernel/fs/param.rs         | 576 ++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs              |   7 +
 samples/rust/rust_rofs.rs       |  32 +-
 6 files changed, 821 insertions(+), 8 deletions(-)
 create mode 100644 rust/kernel/fs/param.rs
fs/puzzlefs/puzzlefs.rs +1 -0
index 9622ea71eda0..f4e94568c9cc 100644
@@ -103,6 +103,7 @@ impl fs::FileSystem for PuzzleFsModule {
     const NAME: &'static CStr = c_str!("puzzlefs");
 
     fn fill_super(
+        _data: (),
         sb: &mut sb::SuperBlock<Self, sb::New>,
         _: Option<inode::Mapper>,
     ) -> Result<Box<PuzzleFS>> {
rust/bindings/bindings_helper.h +1 -0
index 0491bb05270c..3f18e5abcc5c 100644
@@ -27,6 +27,7 @@
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 #include <linux/xattr.h>
+#include <linux/fs_parser.h>
 
 /* `bindgen` gets confused at certain things. */
 const size_t RUST_CONST_HELPER_ARCH_SLAB_MINALIGN = ARCH_SLAB_MINALIGN;
rust/kernel/fs.rs +205 -7

296 lines not on this page · raw message

rust/kernel/fs/param.rs new file +576 -0

576 lines not on this page · raw message

rust/kernel/lib.rs +7 -0
index ea1411a25ee4..59253f5390cd 100644
@@ -12,11 +12,13 @@
 //! do so first instead of bypassing this crate.
 
 #![no_std]
+#![feature(associated_type_defaults)]
 #![feature(coerce_unsized)]
 #![feature(dispatch_from_dyn)]
 #![feature(new_uninit)]
 #![feature(receiver_trait)]
 #![feature(unsize)]
+#![feature(const_mut_refs)]
 
 // Ensure conditional compilation based on the kernel configuration works;
 // otherwise we may silently break things like initcall handling.
@@ -62,6 +64,11 @@
 #[doc(hidden)]
 pub use build_error::build_error;
 
+/// Page size defined in terms of the `PAGE_SHIFT` macro from C.
+///
+/// [`PAGE_SHIFT`]: ../../../include/asm-generic/page.h
+pub const PAGE_SIZE: usize = 1 << bindings::PAGE_SHIFT;
+
 /// Prefix to appear before log messages printed from within the `kernel` crate.
 const __LOG_PREFIX: &[u8] = b"rust_kernel\0";
 
samples/rust/rust_rofs.rs +31 -1
index 5d0d1936459d..d81ea8fdf174 100644
@@ -55,6 +55,31 @@ struct Entry {
 const DIR_IOPS: inode::Ops<RoFs> = inode::Ops::new::<RoFs>();
 const FILE_AOPS: address_space::Ops<RoFs> = address_space::Ops::new::<RoFs>();
 
+#[vtable]
+impl fs::Context<Self> for RoFs {
+    type Data = ();
+
+    kernel::define_fs_params! {(),
+        {flag, "flag", |_, v| { pr_info!("flag passed-in: {v}\n"); Ok(()) } },
+        {flag_no, "flagno", |_, v| { pr_info!("flagno passed-in: {v}\n"); Ok(()) } },
+        {bool, "bool", |_, v| { pr_info!("bool passed-in: {v}\n"); Ok(()) } },
+        {u32, "u32", |_, v| { pr_info!("u32 passed-in: {v}\n"); Ok(()) } },
+        {u32oct, "u32oct", |_, v| { pr_info!("u32oct passed-in: {v}\n"); Ok(()) } },
+        {u32hex, "u32hex", |_, v| { pr_info!("u32hex passed-in: {v}\n"); Ok(()) } },
+        {s32, "s32", |_, v| { pr_info!("s32 passed-in: {v}\n"); Ok(()) } },
+        {u64, "u64", |_, v| { pr_info!("u64 passed-in: {v}\n"); Ok(()) } },
+        {string, "string", |_, v| { pr_info!("string passed-in: {v}\n"); Ok(()) } },
+        {enum, "enum", [("first", 10), ("second", 20)], |_, v| {
+            pr_info!("enum passed-in: {v}\n"); Ok(()) }
+        },
+    }
+
+    fn try_new() -> Result {
+        pr_info!("context created!\n");
+        Ok(())
+    }
+}
+
 struct RoFs;
 
 impl RoFs {
@@ -103,11 +128,16 @@ fn iget(sb: &sb::SuperBlock<Self>, e: &'static Entry) -> Result<ARef<INode<Self>
 }
 
 impl fs::FileSystem for RoFs {
+    type Context = Self;
     type Data = ();
     type INodeData = &'static Entry;
     const NAME: &'static CStr = c_str!("rust_rofs");
 
-    fn fill_super(sb: &mut sb::SuperBlock<Self, sb::New>, _: Option<inode::Mapper>) -> Result {
+    fn fill_super(
+        _data: (),
+        sb: &mut sb::SuperBlock<Self, sb::New>,
+        _: Option<inode::Mapper>,
+    ) -> Result {
         sb.set_magic(0x52555354);
         Ok(())
     }
Signature · 1 line
2.34.1
#22 lore raw

RFC PATCH v3 21/22 fs: puzzlefs: add oci_root_dir and image_manifest mount parameters

In reply to Ariel Miculas

These parameters are passed when mounting puzzlefs using '-o' option of mount:
-o oci_root_dir="/path/to/oci/dir"
-o image_manifest="root_hash_of_image_manifest"

For a particular manifest in the manifests array in index.json (located in the oci_root_dir), the root hash of the image manifest is found in the digest field.

It would be nicer if we could pass the tag, but we don't support json deserialization.

Example of mount:
mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
none /mnt

---
 fs/puzzlefs/puzzlefs.rs | 49 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 44 insertions(+), 5 deletions(-)
fs/puzzlefs/puzzlefs.rs +44 -5
index f4e94568c9cc..932f31917992 100644
@@ -36,6 +36,32 @@ fn mode_to_fs_type(inode: &Inode) -> Result<DirEntryType> {
     })
 }
 
+#[derive(Default)]
+struct PuzzleFsParams {
+    oci_root_dir: Option<CString>,
+    image_manifest: Option<CString>,
+}
+
+#[vtable]
+impl fs::Context<Self> for PuzzleFsModule {
+    type Data = Box<PuzzleFsParams>;
+
+    kernel::define_fs_params! {Box<PuzzleFsParams>,
+        {string, "oci_root_dir", |s, v| {
+                                      s.oci_root_dir = Some(CString::try_from_fmt(format_args!("{v}"))?);
+                                      Ok(())
+                                  }},
+        {string, "image_manifest", |s, v| {
+                                      s.image_manifest = Some(CString::try_from_fmt(format_args!("{v}"))?);
+                                      Ok(())
+                                  }},
+    }
+
+    fn try_new() -> Result<Self::Data> {
+        Ok(Box::new(PuzzleFsParams::default(), GFP_KERNEL)?)
+    }
+}
+
 const DIR_FOPS: file::Ops<PuzzleFsModule> = file::Ops::new::<PuzzleFsModule>();
 const DIR_IOPS: inode::Ops<PuzzleFsModule> = inode::Ops::new::<PuzzleFsModule>();
 const FILE_AOPS: address_space::Ops<PuzzleFsModule> = address_space::Ops::new::<PuzzleFsModule>();
@@ -98,24 +124,37 @@ fn iget(sb: &sb::SuperBlock<Self>, ino: u64) -> Result<ARef<INode<Self>>> {
 }
 
 impl fs::FileSystem for PuzzleFsModule {
+    type Context = Self;
     type Data = Box<PuzzleFS>;
     type INodeData = Inode;
     const NAME: &'static CStr = c_str!("puzzlefs");
 
     fn fill_super(
-        _data: (),
+        data: Box<PuzzleFsParams>,
         sb: &mut sb::SuperBlock<Self, sb::New>,
         _: Option<inode::Mapper>,
     ) -> Result<Box<PuzzleFS>> {
-        let puzzlefs = PuzzleFS::open(
-            c_str!("/home/puzzlefs_xattr"),
-            c_str!("ed63ace21eccceabab08d89afb75e94dae47973f82a17a172396a19ea953c8ab"),
-        );
+        let Some(oci_root_dir) = data.oci_root_dir else {
+            pr_err!("missing oci_root_dir parameter!\n");
+            return Err(ENOTSUPP);
+        };
 
+        let Some(image_manifest) = data.image_manifest else {
+            pr_err!("missing image_manifest parameter!\n");
+            return Err(ENOTSUPP);
+        };
+
+        let puzzlefs = PuzzleFS::open(&oci_root_dir, &image_manifest);
         if let Err(ref e) = puzzlefs {
             pr_info!("error opening puzzlefs {e}\n");
         }
 
+        pr_info!(
+            "opened puzzlefs [{}]:[{}]\n",
+            &*oci_root_dir,
+            &*image_manifest
+        );
+
         let puzzlefs = puzzlefs?;
         sb.set_magic(0x7a7a7570);
         Ok(Box::new(puzzlefs, GFP_KERNEL)?)
Signature · 1 line
2.34.1
#23 lore raw

RFC PATCH v3 22/22 fs: puzzlefs: implement statfs for puzzlefs

In reply to Ariel Miculas

In order to use a filesystem as a lower filesystem in an overlay, it must implement statfs.

---
 fs/puzzlefs/puzzle/inode.rs | 26 ++++++++++++++++++++++++--
 fs/puzzlefs/puzzlefs.rs     | 21 +++++++++++++++++++--
 2 files changed, 43 insertions(+), 4 deletions(-)
fs/puzzlefs/puzzle/inode.rs +24 -2
index 318edbdc5163..a34f1064b632 100644
@@ -14,6 +14,8 @@
 pub(crate) struct PuzzleFS {
     pub(crate) oci: Image,
     layers: Vec<format::MetadataBlob>,
+    pub(crate) total_inodes: u64,
+    pub(crate) total_block_size: u64,
 }
 
 impl PuzzleFS {
@@ -22,13 +24,33 @@ pub(crate) fn open(oci_root_dir: &CStr, rootfs_path: &CStr) -> Result<PuzzleFS>
         let oci = Image::open(vfs_mount)?;
         let rootfs = oci.open_rootfs_blob(rootfs_path)?;
 
+        let mut total_block_size = 0;
+        let mut total_inodes: u64 = 0;
         let mut layers = Vec::new();
         for md in rootfs.metadatas.iter() {
             let digest = Digest::try_from(md)?;
-            layers.push(oci.open_metadata_blob(&digest)?, GFP_KERNEL)?;
+            let layer = oci.open_metadata_blob(&digest)?;
+
+            // This may take up too much time, but we need to implement statfs if we want to use
+            // puzzlefs as a lower filesystem in overlayfs
+            let inodes = layer.get_inode_vector()?;
+            total_inodes += u64::from(inodes.len());
+            for inode_number in 0..inodes.len() {
+                let inode = Inode::from_capnp(inodes.get(inode_number))?;
+                if let InodeMode::File { chunks } = inode.mode {
+                    total_block_size += chunks.iter().map(|chunk| chunk.len).sum::<u64>();
+                }
+            }
+
+            layers.push(layer, GFP_KERNEL)?;
         }
 
-        Ok(PuzzleFS { oci, layers })
+        Ok(PuzzleFS {
+            oci,
+            layers,
+            total_inodes,
+            total_block_size,
+        })
     }
 
     pub(crate) fn find_inode(&self, ino: u64) -> Result<Inode> {
fs/puzzlefs/puzzlefs.rs +19 -2
index 932f31917992..633f60983849 100644
@@ -4,7 +4,7 @@
 
 use kernel::fs::{
     address_space, dentry, dentry::DEntry, file, file::DirEntryType, file::File, inode,
-    inode::INode, sb, Offset,
+    inode::INode, sb, Offset, Stat,
 };
 use kernel::prelude::*;
 use kernel::types::{ARef, Either, Locked};
@@ -23,6 +23,10 @@
     license: "GPL",
 }
 
+const PUZZLEFS_BSIZE: u64 = 1 << PUZZLEFS_BSIZE_BITS;
+const PUZZLEFS_BSIZE_BITS: u8 = 12;
+const PUZZLEFS_MAGIC: usize = 0x7a7a7570;
+
 fn mode_to_fs_type(inode: &Inode) -> Result<DirEntryType> {
     Ok(match inode.mode {
         InodeMode::File { .. } => DirEntryType::Reg,
@@ -156,7 +160,7 @@ fn fill_super(
         );
 
         let puzzlefs = puzzlefs?;
-        sb.set_magic(0x7a7a7570);
+        sb.set_magic(PUZZLEFS_MAGIC);
         Ok(Box::new(puzzlefs, GFP_KERNEL)?)
     }
 
@@ -194,6 +198,19 @@ fn read_xattr(
         }
         Err(ENODATA)
     }
+
+    fn statfs(dentry: &DEntry<Self>) -> Result<Stat> {
+        let puzzlefs = dentry.super_block().data();
+
+        Ok(Stat {
+            magic: PUZZLEFS_MAGIC,
+            namelen: isize::MAX,
+            bsize: PUZZLEFS_BSIZE as _,
+            // Round total_block_size up
+            blocks: (puzzlefs.total_block_size + PUZZLEFS_BSIZE - 1) / PUZZLEFS_BSIZE,
+            files: puzzlefs.total_inodes,
+        })
+    }
 }
 
 #[vtable]
Signature · 1 line
2.34.1