Skip to contentSkip to conversation

Rust PuzzleFS filesystem driver

RFC PATCH Superseded Ariel Miculas 14 participants 135 messages Jun 9, 2023 – Jun 17, 2023

Subsystems (9)
Memory management x86 VFS Kbuild +5 more subsystems

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Rust PuzzleFS filesystem driver

Hi all!

This is a proof of concept driver written for the PuzzleFS next-generation container filesystem [1]. I've included a short abstract about puzzlefs further below. This driver is based on the rust-next branch, on top of which I've backported the filesystem abstractions from Wedson Almeida Filho [2][3] and Miguel Ojeda's third-party crates
support: proc-macro2, quote, syn, serde and serde_derive [4]. I've added the additional third-party crates serde_cbor[5] and hex [6]. Then I've adapted the user space puzzlefs code [1] so that the puzzlefs kernel module could present the directory hierarchy and implement the basic read functionality.
For some additional context, puzzlefs was started by Tycho Andersen and it's the successor of atomfs. This FOSDEM presentation from 2019 [12] covers the rationale for a new oci image format and presents a higher level overview of our goals with puzzlefs.
I've split the rest of the cover letter in following sections (using a markdown style):
* Example: it describes a practical example of what was achieved
* Limitations: it presents the existing limitations of this POC
* Upstreaming steps: it describes the steps needed for upstreaming this driver
* Setup: it shows how to setup the necessary environment for testing the puzzlefs driver
* Puzzlefs abstract: it provides a short overview of puzzlefs

# Example
An example is provided below:

```
~ # cat /proc/filesystems | grep puzzlefs
nodev   puzzlefs
~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o image_manifest="2d
6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" none /mnt
~ # ls -lR /mnt/
/mnt/:
total 0
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-1
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-2
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-3
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-4
-rw-r--r--    1 0        0                0 Jun  8 12:26 file1
-rw-r--r--    1 0        0                0 Jun  8 12:26 file2

/mnt/dir-1:
total 0

/mnt/dir-2:
total 0

/mnt/dir-3:
total 0

/mnt/dir-4:
total 0
~ # cat /mnt/file2
ana are mere bla bla bla
~ # wc /mnt/file1
      202       202      5454 /mnt/file1
```

In this example, /home/puzzlefs_oci is the puzzlefs oci directory:

```
~ # ls -lR /home/puzzlefs_oci/
/home/puzzlefs_oci/:
total 8
drwxr-xr-x    3 1000     1000             0 Jun  8 14:33 blobs
-rw-r--r--    1 1000     1000           266 Jun  8 14:33 index.json
-rw-r--r--    1 1000     1000            37 Jun  8 14:33 oci-layout

/home/puzzlefs_oci/blobs:
total 0
drwxr-xr-x    2 1000     1000             0 Jun  8 14:33 sha256

/home/puzzlefs_oci/blobs/sha256:
total 16
-rw-------    1 1000     1000            89 Jun  8 14:33 2d6602d678140540dc7e96de652a76a8b16eb
-rw-------    1 1000     1000           925 Jun  8 14:33 4df03518eea406343dbb55046720f6a478881
-rw-------    1 1000     1000          5479 Jun  8 14:33 d86a87b19bd9a2fec0d31687c1d669cdb59eb
```

2d6602d678140540dc7e96de652a76a8b16eb is the puzzlefs image manifest hash for the first_try tag:

```
$ cat /tmp/oci-simple/index.json | jq .
{
  "schemaVersion": -1,
  "manifests": [
    {
      "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
      "size": 89,
      "media_type": "application/vnd.puzzlefs.image.rootfs.v1",
      "annotations": {
        "org.opencontainers.image.ref.name": "first_try"
      }
    }
  ],
  "annotations": {}
}
```

I will describe how to build a puzzlefs image in the Setup section, at step 5.

# Limitations
One limitation is that the puzzlefs driver doesn't implement any lookup functionality and instead it inserts every directory entry into the dcache during init (see the DCACHE_BASED constant). This is similar to how the sample rust_fs driver works, but the goal is to implement proper lookup functions. However, more filesystem abstractions need to be implemented before this can be achieved.

# Upstreaming steps
Before the puzzlefs driver can be upstreamed, the following need to be merged:
* Wedson's filesystem abstractions [3]
* the necessary third-party crates [4] (with the preliminary discussion about whether this is desirable)

# Setup
My setup is based on Wedson's tutorial [8]. Next, I will describe the necessary steps to build an initrd and run a custom kernel under qemu.

1. Get the linux rust-next branch [9] and apply this patchset

2. Follow the rust quickstart guide [10]

3. configure and build the kernel

```
$ make LLVM=1 allnoconfig qemu-busybox-min.config rust.config
$ make LLVM=1 -j$(nproc)
```

4. setup busybox

```
git clone git://git.busybox.net/busybox
cd busybox
make menuconfig # enable 'Build static binary' config
make
make install
```

This will create the _install directory with the rootfs inside it.

5. create a home directory in the rootfs and copy a puzzlefs oci directory in home/puzzlefs_oci
To create a puzzlefs oci directory:
* download this custom puzzlefs repository [11] (it's custom because we want to build an image without verity data)
* run make release
* create a simple filesystem structure with a few directories and files (I've created one at ../test-puzzlefs/simple_rootfs)
* build a puzzlefs oci image at ~/work/busybox/_install/home/puzzlefs_oci (replace this path with your busybox path) with the tag first_try:

```
$ target/release/puzzlefs build --omit-verity \
../test-puzzlefs/simple_rootfs ~/work/busybox/_install/home/puzzlefs_oci \
first_try
```
* get first_try's image manifest from index.json (inside `puzzlefs_oci`)
```
$ cat index.json | jq . | grep digest
      "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
```

6. add the following 'init' script in the busybox rootfs (rootfs path defaults to `./_install'):

```
#!/bin/sh
mount -t devtmpfs none /dev
mkdir -p /proc
mount -t proc none /proc

ifconfig lo up
udhcpc -i eth0

mkdir /mnt
mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
none /mnt

setsid sh -c 'exec sh -l </dev/ttyS0 >/dev/ttyS0 2>&1'
```

Make sure to replace the image_manifest with your own digest. This init script will be passed to rdinit in the kernel command line.

7. generate the initramfs

Assuming busybox is in ~/work/busybox:

```
cd ~/work/busybox/_install && find . | cpio -H newc -o | gzip > ../ramdisk.img
```

This will generate a compressed ramdisk image in
~/work/busybox/ramdisk.img.

8. run with qemu (this assumes the linux tree is at '../linux' and busybox is at '../busybox'):

```
qemu-system-x86_64 \
    -accel kvm \
    -cpu host \
    -m 4G \
    -initrd ../busybox/ramdisk.img \
    -kernel ../linux/arch/x86/boot/bzImage \
    -nographic \
    -append 'console=ttyS0 nokaslr debug rdinit=/init' \
    -nic user,model=rtl8139 \
    -no-reboot
```

9. Check whether puzzlefs has been successfully mounted

```
~ # mount | grep puzzlefs
none on /mnt type puzzlefs (rw,relatime)
~ # ls /mnt/
dir-1  dir-2  dir-3  dir-4  file1  file2
```

# Puzzlefs abstract
Puzzlefs [1] is a container filesystem designed to address the limitations of the existing OCI format. The main goals of the project are reduced duplication, reproducible image builds, direct mounting support and memory safety guarantees, some inspired by the OCIv2 design document [7].

Reduced duplication is achieved using the content defined chunking algorithm FastCDC. This implementation allows chunks to be shared among layers. Building a new layer starting from an existing one allows reusing most of the chunks.

Another goal of the project is reproducible image builds, which is achieved by defining a canonical representation of the image format.

Direct mounting support is a key feature of puzzlefs and, together with fs-verity, it provides data integrity. Currently, puzzlefs is implemented as a userspace filesystem (FUSE). A read-only kernel filesystem driver is underway.

Lastly, memory safety is critical to puzzlefs, leading to the decision to implement it in Rust. Another goal is to share the same code between user space and kernel space in order to provide one secure implementation.

[1] https://github.com/anuvu/puzzlefs
[2] https://github.com/wedsonaf/linux/tree/fs
[3] https://github.com/Rust-for-Linux/linux/issues/1004
[4] https://github.com/Rust-for-Linux/linux/pull/1007
[5] https://docs.rs/serde_cbor/latest/serde_cbor/
[6] https://docs.rs/hex/0.4.3/hex/
[7] https://hackmd.io/@cyphar/ociv2-brainstorm
[8] https://www.youtube.com/watch?v=tPs1uRqOnlk
[9] https://github.com/Rust-for-Linux/linux/tree/rust-next
[10] https://docs.kernel.org/rust/quick-start.html
[11] https://github.com/ariel-miculas/puzzlefs/tree/support-no-verity-data
[12] https://archive.fosdem.org/2019/schedule/event/containers_atomfs/

Ariel Miculas (58):
rust: kernel: add libraries required by the filesystem abstractions
rust: kernel: backport the delay module from the rust branch
rust: kernel: add container_of macro
rust: kernel: add offset_of macro
drop: Add crate::pr_warn declaration
rust: kernel: rename from_kernel_errno to from_errno
rust: kernel: Rename from_pointer to from_foreing and into_pointer to
into_foreign
rust: kernel: add count_paren_items macro, needed by define_fs_params
macro
rust: helpers: add missing rust helper 'alloc_pages'
kernel: configs: add qemu-busybox-min.config
rust: kernel: format the rust code
samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs
kernel: configs: enable rust samples in rust.config
Add SAMPLE_RUST_SERDE in rust.config
rust: kernel: fix compile errors after rebase to rust-next
rust: serde_cbor: import crate
rust: serde_cbor: add SPDX License Identifiers
rust: serde_cbor: add no_fp_fmt_parse support
rust: Kbuild: enable serde_cbor
samples: rust: add cbor serialize/deserialize example
rust: serde_cbor: add support for serde_cbor's from_slice method by
using a custom alloc_kernel feature
rust: serde: add support for deserializing Vec with kernel_alloc
feature
rust: file: Replace UnsafeCell with Opaque for File
rust: kernel: implement fmt::Debug for CString
samples: puzzlefs: rename RustFs to PuzzleFs
samples: puzzlefs: add basic deserializing support for the puzzlefs
metadata
rust: file: present the filesystem context to the open function
rust: kernel: add an abstraction over vfsmount to allow cloning a new
private mount
rust: file: add from_path, from_path_in_root_mnt and read_with_offset
methods to File
samples: puzzlefs: pass the Vfsmount structure from open to read and
return the contents of the data file inside /home/puzzlefs_oci
rust: file: move from_path, from_path_in_root_mnt and read_with_offset
methods to a RegularFile newtype
rust: file: ensure RegularFile can only create regular files
rust: file: add get_pos method to RegularFile
rust: file: add methods read_to_end, get_file_size and update_pos to
RegularFile
rust: file: define a minimal Read trait and implement it for
RegularFile
samples: puzzlefs: add cbor_get_array_size method
samples: puzzlefs: add KernelError to WireFormatError and implement
From conversion
samples: puzzlefs: implement new for MetadataBlob
samples: puzzlefs: build puzzlefs into the kernel, thus avoiding the
need to export rust symbols
rust: alloc: add try_clone for Vec<T>
rust: alloc: add from_iter_fallible for Vec<T>
samples: puzzlefs: implement to_errno and from_errno for
WireFormatError
samples: puzzlefs: add TryReserveError (and from conversion) to
WireFormatError
samples: puzzlefs: add higher level inode related functionality
samples: puzzlefs: populate the directory entries with the inodes from
the puzzlefs metadata file
rust: hex: import crate
rust: hex: add SPDX license identifiers
rust: Kbuild: enable hex
rust: hex: implement FromHex trait and hex::decode using a custom
kernel_alloc feature
rust: hex: add encode_hex_iter and encode_hex_upper_iter methods
rust: puzzlefs: add HexError to WireFormatError and implement the From
conversion
rust: puzzlefs: display the error value for
WireFormatError::KernelError
samples: puzzlefs: add Rootfs and Digest structs to types.rs
samples: puzzlefs: implement the conversion from WireFormatError to
kernel::error::Error
rust: puzzlefs: read the puzzlefs image manifest instead of an
individual metadata layer
rust: puzzlefs: rename PuzzleFs to PuzzleFsModule to avoid confusion
with the PuzzleFS struct
rust: puzzlefs: add support for reading files
rust: puzzlefs: add oci_root_dir and image_manifest filesystem
parameters

Miguel Ojeda (15):
rust: proc-macro2: import crate
rust: proc-macro2: add SPDX License Identifiers
rust: proc-macro2: remove unicode_ident dependency
rust: quote: import crate
rust: quote: add SPDX License Identifiers
rust: syn: import crate
rust: syn: add SPDX License Identifiers
rust: syn: remove unicode-ident dependency
rust: serde: import crate
rust: serde: add no_fp_fmt_parse support
rust: serde: add SPDX License Identifiers
rust: serde_derive: import crate
rust: serde_derive: add SPDX License Identifiers
rust: Kbuild: enable proc-macro2, quote, syn, serde and
serde_derive
rust: test serde support

Wedson Almeida Filho (7):
rust: add definitions for ref-counted inodes and dentries
rust: add ability to register a file system
rust: define fs context
rust: add support for file system parameters
rust: allow fs driver to initialise new superblocks
rust: add module_fs macro
WIP: rust: allow fs to be populated

 Makefile                                  |   14 +-
 arch/x86/configs/qemu-busybox-min.config  |   11 +
 kernel/configs/qemu-busybox-min.config    |   56 +
 kernel/configs/rust.config                |   11 +
 rust/.gitignore                           |    1 +
 rust/Makefile                             |  232 +-
 rust/alloc/vec/mod.rs                     |   48 +
 rust/bindings/bindings_helper.h           |   14 +
 rust/bindings/lib.rs                      |    5 +
 rust/helpers.c                            |   76 +
 rust/hex/error.rs                         |   78 +
 rust/hex/lib.rs                           |  506 +++
 rust/hex/serde.rs                         |  104 +
 rust/kernel/cred.rs                       |   46 +
 rust/kernel/delay.rs                      |  104 +
 rust/kernel/driver.rs                     |   28 +
 rust/kernel/error.rs                      |   52 +-
 rust/kernel/file.rs                       | 1117 ++++++
 rust/kernel/fs.rs                         | 1478 ++++++++
 rust/kernel/fs/param.rs                   |  558 +++
 rust/kernel/io_buffer.rs                  |  153 +
 rust/kernel/iov_iter.rs                   |   81 +
 rust/kernel/lib.rs                        |   83 +
 rust/kernel/mm.rs                         |  149 +
 rust/kernel/mount.rs                      |   66 +
 rust/kernel/pages.rs                      |  144 +
 rust/kernel/str.rs                        |    6 +
 rust/kernel/test_serde.rs                 |   26 +
 rust/kernel/test_serde/de.rs              |  439 +++
 rust/kernel/test_serde/error.rs           |   73 +
 rust/kernel/test_serde/ser.rs             |  466 +++
 rust/kernel/user_ptr.rs                   |  175 +
 rust/proc-macro2/detection.rs             |   77 +
 rust/proc-macro2/fallback.rs              | 1004 ++++++
 rust/proc-macro2/lib.rs                   | 1341 ++++++++
 rust/proc-macro2/marker.rs                |   20 +
 rust/proc-macro2/parse.rs                 |  874 +++++
 rust/proc-macro2/rcvec.rs                 |  144 +
 rust/proc-macro2/wrapper.rs               |  996 ++++++
 rust/quote/ext.rs                         |  112 +
 rust/quote/format.rs                      |  170 +
 rust/quote/ident_fragment.rs              |   88 +
 rust/quote/lib.rs                         | 1436 ++++++++
 rust/quote/runtime.rs                     |  440 +++
 rust/quote/spanned.rs                     |   45 +
 rust/quote/to_tokens.rs                   |  211 ++
 rust/serde/de/format.rs                   |   32 +
 rust/serde/de/ignored_any.rs              |  246 ++
 rust/serde/de/impls.rs                    | 2755 +++++++++++++++
 rust/serde/de/mod.rs                      | 2313 +++++++++++++
 rust/serde/de/seed.rs                     |   21 +
 rust/serde/de/utf8.rs                     |   48 +
 rust/serde/de/value.rs                    | 1718 ++++++++++
 rust/serde/integer128.rs                  |   84 +
 rust/serde/lib.rs                         |  351 ++
 rust/serde/macros.rs                      |  238 ++
 rust/serde/private/de.rs                  | 2997 ++++++++++++++++
 rust/serde/private/doc.rs                 |  161 +
 rust/serde/private/mod.rs                 |   52 +
 rust/serde/private/ser.rs                 | 1316 +++++++
 rust/serde/private/size_hint.rs           |   23 +
 rust/serde/ser/fmt.rs                     |  180 +
 rust/serde/ser/impls.rs                   |  987 ++++++
 rust/serde/ser/impossible.rs              |  218 ++
 rust/serde/ser/mod.rs                     | 1992 +++++++++++
 rust/serde/std_error.rs                   |   50 +
 rust/serde_cbor/de.rs                     | 1370 ++++++++
 rust/serde_cbor/error.rs                  |  320 ++
 rust/serde_cbor/lib.rs                    |  371 ++
 rust/serde_cbor/read.rs                   |  647 ++++
 rust/serde_cbor/ser.rs                    |  748 ++++
 rust/serde_cbor/tags.rs                   |  224 ++
 rust/serde_cbor/value/de.rs               |  168 +
 rust/serde_cbor/value/mod.rs              |  158 +
 rust/serde_cbor/value/ser.rs              |  447 +++
 rust/serde_cbor/write.rs                  |  177 +
 rust/serde_derive/bound.rs                |  408 +++
 rust/serde_derive/de.rs                   | 3148 +++++++++++++++++
 rust/serde_derive/dummy.rs                |   46 +
 rust/serde_derive/fragment.rs             |   76 +
 rust/serde_derive/internals/ast.rs        |  204 ++
 rust/serde_derive/internals/attr.rs       | 1908 +++++++++++
 rust/serde_derive/internals/case.rs       |  199 ++
 rust/serde_derive/internals/check.rs      |  445 +++
 rust/serde_derive/internals/ctxt.rs       |   64 +
 rust/serde_derive/internals/mod.rs        |   30 +
 rust/serde_derive/internals/receiver.rs   |  287 ++
 rust/serde_derive/internals/respan.rs     |   18 +
 rust/serde_derive/internals/symbol.rs     |   71 +
 rust/serde_derive/lib.rs                  |  112 +
 rust/serde_derive/pretend.rs              |  203 ++
 rust/serde_derive/ser.rs                  | 1342 ++++++++
 rust/serde_derive/this.rs                 |   34 +
 rust/serde_derive/try.rs                  |   26 +
 rust/syn/attr.rs                          |  664 ++++
 rust/syn/await.rs                         |    4 +
 rust/syn/bigint.rs                        |   68 +
 rust/syn/buffer.rs                        |  400 +++
 rust/syn/custom_keyword.rs                |  255 ++
 rust/syn/custom_punctuation.rs            |  302 ++
 rust/syn/data.rs                          |  495 +++
 rust/syn/derive.rs                        |  276 ++
 rust/syn/discouraged.rs                   |  196 ++
 rust/syn/error.rs                         |  430 +++
 rust/syn/export.rs                        |   41 +
 rust/syn/expr.rs                          | 3560 +++++++++++++++++++
 rust/syn/ext.rs                           |  141 +
 rust/syn/file.rs                          |  127 +
 rust/syn/gen/clone.rs                     | 2243 ++++++++++++
 rust/syn/gen/debug.rs                     | 3044 +++++++++++++++++
 rust/syn/gen/eq.rs                        | 2197 ++++++++++++
 rust/syn/gen/fold.rs                      | 3343 ++++++++++++++++++
 rust/syn/gen/hash.rs                      | 2871 ++++++++++++++++
 rust/syn/gen/visit.rs                     | 3788 +++++++++++++++++++++
 rust/syn/gen/visit_mut.rs                 | 3788 +++++++++++++++++++++
 rust/syn/gen_helper.rs                    |  156 +
 rust/syn/generics.rs                      | 1339 ++++++++
 rust/syn/group.rs                         |  284 ++
 rust/syn/ident.rs                         |  103 +
 rust/syn/item.rs                          | 3315 ++++++++++++++++++
 rust/syn/lib.rs                           |  985 ++++++
 rust/syn/lifetime.rs                      |  156 +
 rust/syn/lit.rs                           | 1602 +++++++++
 rust/syn/lookahead.rs                     |  171 +
 rust/syn/mac.rs                           |  221 ++
 rust/syn/macros.rs                        |  179 +
 rust/syn/op.rs                            |  236 ++
 rust/syn/parse.rs                         | 1316 +++++++
 rust/syn/parse_macro_input.rs             |  181 +
 rust/syn/parse_quote.rs                   |  169 +
 rust/syn/pat.rs                           |  929 +++++
 rust/syn/path.rs                          |  856 +++++
 rust/syn/print.rs                         |   18 +
 rust/syn/punctuated.rs                    | 1070 ++++++
 rust/syn/reserved.rs                      |   46 +
 rust/syn/sealed.rs                        |    6 +
 rust/syn/span.rs                          |   69 +
 rust/syn/spanned.rs                       |  116 +
 rust/syn/stmt.rs                          |  351 ++
 rust/syn/thread.rs                        |   43 +
 rust/syn/token.rs                         | 1015 ++++++
 rust/syn/tt.rs                            |  109 +
 rust/syn/ty.rs                            | 1288 +++++++
 rust/syn/verbatim.rs                      |   35 +
 rust/syn/whitespace.rs                    |   67 +
 samples/rust/Kconfig                      |   28 +
 samples/rust/Makefile                     |    3 +
 samples/rust/local_data_format/de.rs      |  422 +++
 samples/rust/local_data_format/error.rs   |   73 +
 samples/rust/local_data_format/ser.rs     |  443 +++
 samples/rust/puzzle.rs                    |    4 +
 samples/rust/puzzle/error.rs              |   91 +
 samples/rust/puzzle/inode.rs              |  150 +
 samples/rust/puzzle/oci.rs                |   71 +
 samples/rust/puzzle/types.rs              |  389 +++
 samples/rust/puzzle/types/cbor_helpers.rs |   50 +
 samples/rust/puzzlefs.rs                  |  220 ++
 samples/rust/rust_fs.rs                   |  105 +
 samples/rust/rust_serde.rs                |  125 +
 scripts/Makefile.build                    |    4 +-
 160 files changed, 89204 insertions(+), 29 deletions(-)
 create mode 100644 arch/x86/configs/qemu-busybox-min.config
 create mode 100644 kernel/configs/qemu-busybox-min.config
 create mode 100644 rust/hex/error.rs
 create mode 100644 rust/hex/lib.rs
 create mode 100644 rust/hex/serde.rs
 create mode 100644 rust/kernel/cred.rs
 create mode 100644 rust/kernel/delay.rs
 create mode 100644 rust/kernel/driver.rs
 create mode 100644 rust/kernel/file.rs
 create mode 100644 rust/kernel/fs.rs
 create mode 100644 rust/kernel/fs/param.rs
 create mode 100644 rust/kernel/io_buffer.rs
 create mode 100644 rust/kernel/iov_iter.rs
 create mode 100644 rust/kernel/mm.rs
 create mode 100644 rust/kernel/mount.rs
 create mode 100644 rust/kernel/pages.rs
 create mode 100644 rust/kernel/test_serde.rs
 create mode 100644 rust/kernel/test_serde/de.rs
 create mode 100644 rust/kernel/test_serde/error.rs
 create mode 100644 rust/kernel/test_serde/ser.rs
 create mode 100644 rust/kernel/user_ptr.rs
 create mode 100644 rust/proc-macro2/detection.rs
 create mode 100644 rust/proc-macro2/fallback.rs
 create mode 100644 rust/proc-macro2/lib.rs
 create mode 100644 rust/proc-macro2/marker.rs
 create mode 100644 rust/proc-macro2/parse.rs
 create mode 100644 rust/proc-macro2/rcvec.rs
 create mode 100644 rust/proc-macro2/wrapper.rs
 create mode 100644 rust/quote/ext.rs
 create mode 100644 rust/quote/format.rs
 create mode 100644 rust/quote/ident_fragment.rs
 create mode 100644 rust/quote/lib.rs
 create mode 100644 rust/quote/runtime.rs
 create mode 100644 rust/quote/spanned.rs
 create mode 100644 rust/quote/to_tokens.rs
 create mode 100644 rust/serde/de/format.rs
 create mode 100644 rust/serde/de/ignored_any.rs
 create mode 100644 rust/serde/de/impls.rs
 create mode 100644 rust/serde/de/mod.rs
 create mode 100644 rust/serde/de/seed.rs
 create mode 100644 rust/serde/de/utf8.rs
 create mode 100644 rust/serde/de/value.rs
 create mode 100644 rust/serde/integer128.rs
 create mode 100644 rust/serde/lib.rs
 create mode 100644 rust/serde/macros.rs
 create mode 100644 rust/serde/private/de.rs
 create mode 100644 rust/serde/private/doc.rs
 create mode 100644 rust/serde/private/mod.rs
 create mode 100644 rust/serde/private/ser.rs
 create mode 100644 rust/serde/private/size_hint.rs
 create mode 100644 rust/serde/ser/fmt.rs
 create mode 100644 rust/serde/ser/impls.rs
 create mode 100644 rust/serde/ser/impossible.rs
 create mode 100644 rust/serde/ser/mod.rs
 create mode 100644 rust/serde/std_error.rs
 create mode 100644 rust/serde_cbor/de.rs
 create mode 100644 rust/serde_cbor/error.rs
 create mode 100644 rust/serde_cbor/lib.rs
 create mode 100644 rust/serde_cbor/read.rs
 create mode 100644 rust/serde_cbor/ser.rs
 create mode 100644 rust/serde_cbor/tags.rs
 create mode 100644 rust/serde_cbor/value/de.rs
 create mode 100644 rust/serde_cbor/value/mod.rs
 create mode 100644 rust/serde_cbor/value/ser.rs
 create mode 100644 rust/serde_cbor/write.rs
 create mode 100644 rust/serde_derive/bound.rs
 create mode 100644 rust/serde_derive/de.rs
 create mode 100644 rust/serde_derive/dummy.rs
 create mode 100644 rust/serde_derive/fragment.rs
 create mode 100644 rust/serde_derive/internals/ast.rs
 create mode 100644 rust/serde_derive/internals/attr.rs
 create mode 100644 rust/serde_derive/internals/case.rs
 create mode 100644 rust/serde_derive/internals/check.rs
 create mode 100644 rust/serde_derive/internals/ctxt.rs
 create mode 100644 rust/serde_derive/internals/mod.rs
 create mode 100644 rust/serde_derive/internals/receiver.rs
 create mode 100644 rust/serde_derive/internals/respan.rs
 create mode 100644 rust/serde_derive/internals/symbol.rs
 create mode 100644 rust/serde_derive/lib.rs
 create mode 100644 rust/serde_derive/pretend.rs
 create mode 100644 rust/serde_derive/ser.rs
 create mode 100644 rust/serde_derive/this.rs
 create mode 100644 rust/serde_derive/try.rs
 create mode 100644 rust/syn/attr.rs
 create mode 100644 rust/syn/await.rs
 create mode 100644 rust/syn/bigint.rs
 create mode 100644 rust/syn/buffer.rs
 create mode 100644 rust/syn/custom_keyword.rs
 create mode 100644 rust/syn/custom_punctuation.rs
 create mode 100644 rust/syn/data.rs
 create mode 100644 rust/syn/derive.rs
 create mode 100644 rust/syn/discouraged.rs
 create mode 100644 rust/syn/error.rs
 create mode 100644 rust/syn/export.rs
 create mode 100644 rust/syn/expr.rs
 create mode 100644 rust/syn/ext.rs
 create mode 100644 rust/syn/file.rs
 create mode 100644 rust/syn/gen/clone.rs
 create mode 100644 rust/syn/gen/debug.rs
 create mode 100644 rust/syn/gen/eq.rs
 create mode 100644 rust/syn/gen/fold.rs
 create mode 100644 rust/syn/gen/hash.rs
 create mode 100644 rust/syn/gen/visit.rs
 create mode 100644 rust/syn/gen/visit_mut.rs
 create mode 100644 rust/syn/gen_helper.rs
 create mode 100644 rust/syn/generics.rs
 create mode 100644 rust/syn/group.rs
 create mode 100644 rust/syn/ident.rs
 create mode 100644 rust/syn/item.rs
 create mode 100644 rust/syn/lib.rs
 create mode 100644 rust/syn/lifetime.rs
 create mode 100644 rust/syn/lit.rs
 create mode 100644 rust/syn/lookahead.rs
 create mode 100644 rust/syn/mac.rs
 create mode 100644 rust/syn/macros.rs
 create mode 100644 rust/syn/op.rs
 create mode 100644 rust/syn/parse.rs
 create mode 100644 rust/syn/parse_macro_input.rs
 create mode 100644 rust/syn/parse_quote.rs
 create mode 100644 rust/syn/pat.rs
 create mode 100644 rust/syn/path.rs
 create mode 100644 rust/syn/print.rs
 create mode 100644 rust/syn/punctuated.rs
 create mode 100644 rust/syn/reserved.rs
 create mode 100644 rust/syn/sealed.rs
 create mode 100644 rust/syn/span.rs
 create mode 100644 rust/syn/spanned.rs
 create mode 100644 rust/syn/stmt.rs
 create mode 100644 rust/syn/thread.rs
 create mode 100644 rust/syn/token.rs
 create mode 100644 rust/syn/tt.rs
 create mode 100644 rust/syn/ty.rs
 create mode 100644 rust/syn/verbatim.rs
 create mode 100644 rust/syn/whitespace.rs
 create mode 100644 samples/rust/local_data_format/de.rs
 create mode 100644 samples/rust/local_data_format/error.rs
 create mode 100644 samples/rust/local_data_format/ser.rs
 create mode 100644 samples/rust/puzzle.rs
 create mode 100644 samples/rust/puzzle/error.rs
 create mode 100644 samples/rust/puzzle/inode.rs
 create mode 100644 samples/rust/puzzle/oci.rs
 create mode 100644 samples/rust/puzzle/types.rs
 create mode 100644 samples/rust/puzzle/types/cbor_helpers.rs
 create mode 100644 samples/rust/puzzlefs.rs
 create mode 100644 samples/rust/rust_fs.rs
 create mode 100644 samples/rust/rust_serde.rs
Signature · 1 line
2.40.1
#1 lore raw

PATCH 1/80 rust: add definitions for ref-counted inodes and dentries

From: Wedson Almeida Filho <[email protected]>

This is in preparation for adding support for the implementation of file systems in Rust.

---
 rust/bindings/bindings_helper.h |  1 +
 rust/helpers.c                  |  7 +++++
 rust/kernel/fs.rs               | 53 +++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs              |  1 +
 4 files changed, 62 insertions(+)
 create mode 100644 rust/kernel/fs.rs
rust/bindings/bindings_helper.h +1 -0
index 50e7a76d5455..eeb5b7c0528a 100644
@@ -10,6 +10,7 @@
 #include <linux/refcount.h>
 #include <linux/wait.h>
 #include <linux/sched.h>
+#include <linux/fs.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
rust/helpers.c +7 -0
index 81e80261d597..e00c53c94c43 100644
@@ -26,6 +26,7 @@
 #include <linux/spinlock.h>
 #include <linux/sched/signal.h>
 #include <linux/wait.h>
+#include <linux/fs_parser.h>
 
 __noreturn void rust_helper_BUG(void)
 {
@@ -128,6 +129,12 @@ void rust_helper_put_task_struct(struct task_struct *t)
 }
 EXPORT_SYMBOL_GPL(rust_helper_put_task_struct);
 
+struct dentry *rust_helper_dget(struct dentry *dentry)
+{
+	return dget(dentry);
+}
+EXPORT_SYMBOL_GPL(rust_helper_dget);
+
 /*
  * We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
  * as the Rust `usize` type, so we can use it in contexts where Rust
rust/kernel/fs.rs new file +53 -0

53 lines not on this page · raw message

rust/kernel/lib.rs +1 -0
index 85b261209977..486f60912132 100644
@@ -30,6 +30,7 @@
 #[cfg(not(test))]
 #[cfg(not(testlib))]
 mod allocator;
+pub mod fs;
 mod build_assert;
 pub mod error;
 pub mod init;
Signature · 1 line
2.40.1
#2 lore raw

PATCH 2/80 rust: add ability to register a file system

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

Also add sample that uses the new public API. The registered file system cannot be mounted yet, but can be seen in /proc/filesystems.

More functionality will be added in subsequent commits.

---
 rust/kernel/fs.rs       | 116 +++++++++++++++++++++++++++++++++++++++-
 samples/rust/Kconfig    |  10 ++++
 samples/rust/Makefile   |   1 +
 samples/rust/rust_fs.rs |  31 +++++++++++
 scripts/Makefile.build  |   2 +-
 5 files changed, 157 insertions(+), 3 deletions(-)
 create mode 100644 samples/rust/rust_fs.rs
rust/kernel/fs.rs +114 -2
index 157fdb917ad1..b61e6f03e871 100644
@@ -4,9 +4,121 @@
 //!
 //! C headers: [`include/linux/fs.h`](../../../../include/linux/fs.h)
 
-use crate::{bindings};
+use crate::{bindings, error::code::*, str::CStr, ThisModule};
+use crate::error::{to_result, Result};
 use crate::types::AlwaysRefCounted;
-use core::{cell::UnsafeCell, ptr};
+use core::{cell::UnsafeCell, marker::PhantomPinned, pin::Pin, ptr};
+
+/// A file system type.
+pub trait Type {
+    /// The name of the file system type.
+    const NAME: &'static CStr;
+
+    /// The flags of this file system type.
+    ///
+    /// It is a combination of the flags in the [`flags`] module.
+    const FLAGS: i32;
+}
+
+/// File system flags.
+pub mod flags {
+    use crate::bindings;
+
+    /// The file system requires a device.
+    pub const REQUIRES_DEV: i32 = bindings::FS_REQUIRES_DEV as _;
+
+    /// The options provided when mounting are in binary form.
+    pub const BINARY_MOUNTDATA: i32 = bindings::FS_BINARY_MOUNTDATA as _;
+
+    /// The file system has a subtype. It is extracted from the name and passed in as a parameter.
+    pub const HAS_SUBTYPE: i32 = bindings::FS_HAS_SUBTYPE as _;
+
+    /// The file system can be mounted by userns root.
+    pub const USERNS_MOUNT: i32 = bindings::FS_USERNS_MOUNT as _;
+
+    /// Disables fanotify permission events.
+    pub const DISALLOW_NOTIFY_PERM: i32 = bindings::FS_DISALLOW_NOTIFY_PERM as _;
+
+    /// The file system has been updated to handle vfs idmappings.
+    pub const ALLOW_IDMAP: i32 = bindings::FS_ALLOW_IDMAP as _;
+
+    /// The file systen will handle `d_move` during `rename` internally.
+    pub const RENAME_DOES_D_MOVE: i32 = bindings::FS_RENAME_DOES_D_MOVE as _;
+}
+
+/// A file system registration.
+#[derive(Default)]
+pub struct Registration {
+    is_registered: bool,
+    fs: UnsafeCell<bindings::file_system_type>,
+    _pin: PhantomPinned,
+}
+
+// SAFETY: `Registration` doesn't really provide any `&self` methods, so it is safe to pass
+// references to it around.
+unsafe impl Sync for Registration {}
+
+// SAFETY: Both registration and unregistration are implemented in C and safe to be performed from
+// any thread, so `Registration` is `Send`.
+unsafe impl Send for Registration {}
+
+impl Registration {
+    /// Creates a new file system registration.
+    ///
+    /// It is not visible or accessible yet. A successful call to [`Registration::register`] needs
+    /// to be made before users can mount it.
+    pub fn new() -> Self {
+        Self {
+            is_registered: false,
+            fs: UnsafeCell::new(bindings::file_system_type::default()),
+            _pin: PhantomPinned,
+        }
+    }
+
+    /// Registers a file system so that it can be mounted by users.
+    ///
+    /// The file system is described by the [`Type`] argument.
+    ///
+    /// It is automatically unregistered when the registration is dropped.
+    pub fn register<T: Type>(self: Pin<&mut Self>, module: &'static ThisModule) -> Result {
+        // SAFETY: We never move out of `this`.
+        let this = unsafe { self.get_unchecked_mut() };
+
+        if this.is_registered {
+            return Err(EINVAL);
+        }
+
+        let mut fs = this.fs.get_mut();
+        fs.owner = module.0;
+        fs.name = T::NAME.as_char_ptr();
+        fs.fs_flags = T::FLAGS;
+        fs.init_fs_context = Some(Self::init_fs_context_callback::<T>);
+        fs.kill_sb = Some(Self::kill_sb_callback::<T>);
+        // SAFETY: Pointers stored in `fs` are either static so will live for as long as the
+        // registration is active (it is undone in `drop`).
+        to_result(unsafe { bindings::register_filesystem(this.fs.get()) })?;
+        this.is_registered = true;
+        Ok(())
+    }
+
+    unsafe extern "C" fn init_fs_context_callback<T: Type>(
+        _fc_ptr: *mut bindings::fs_context,
+    ) -> core::ffi::c_int {
+        EINVAL.to_errno()
+    }
+
+    unsafe extern "C" fn kill_sb_callback<T: Type>(_sb_ptr: *mut bindings::super_block) {}
+}
+
+impl Drop for Registration {
+    fn drop(&mut self) {
+        if self.is_registered {
+            // SAFETY: When `is_registered` is `true`, a previous call to `register_filesystem` has
+            // succeeded, so it is safe to unregister here.
+            unsafe { bindings::unregister_filesystem(self.fs.get()) };
+        }
+    }
+}
 
 /// Wraps the kernel's `struct inode`.
 ///
samples/rust/Kconfig +10 -0
index b0f74a81c8f9..2bd736f99189 100644
@@ -30,6 +30,16 @@ config SAMPLE_RUST_PRINT
 
 	  If unsure, say N.
 
+config SAMPLE_RUST_FS
+	tristate "File system"
+	help
+	  This option builds the Rust file system sample.
+
+	  To compile this as a module, choose M here:
+	  the module will be called rust_fs.
+
+	  If unsure, say N.
+
 config SAMPLE_RUST_HOSTPROGS
 	bool "Host programs"
 	help
samples/rust/Makefile +1 -0
index 03086dabbea4..e5941037e673 100644
@@ -2,5 +2,6 @@
 
 obj-$(CONFIG_SAMPLE_RUST_MINIMAL)		+= rust_minimal.o
 obj-$(CONFIG_SAMPLE_RUST_PRINT)			+= rust_print.o
+obj-$(CONFIG_SAMPLE_RUST_FS)			+= rust_fs.o
 
 subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS)		+= hostprogs
samples/rust/rust_fs.rs new file +31 -0
new file mode 100644
index 000000000000..2caacea50e73
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Rust file system sample.
+
+use kernel::prelude::*;
+use kernel::{c_str, fs};
+
+module! {
+    type: FsModule,
+    name: "rust_fs",
+    author: "Rust for Linux Contributors",
+    license: "GPL",
+}
+
+struct RustFs;
+impl fs::Type for RustFs {
+    const NAME: &'static CStr = c_str!("rustfs");
+    const FLAGS: i32 = fs::flags::USERNS_MOUNT;
+}
+
+struct FsModule {
+    _fs: Pin<Box<fs::Registration>>,
+}
+
+impl kernel::Module for FsModule {
+    fn init(module: &'static ThisModule) -> Result<Self> {
+        let mut reg = Pin::from(Box::try_new(fs::Registration::new())?);
+        reg.as_mut().register::<RustFs>(module)?;
+        Ok(Self { _fs: reg })
+    }
+}
scripts/Makefile.build +1 -1
index 78175231c969..1404967e908e 100644
@@ -277,7 +277,7 @@ $(obj)/%.lst: $(src)/%.c FORCE
 # Compile Rust sources (.rs)
 # ---------------------------------------------------------------------------
 
-rust_allowed_features := new_uninit
+rust_allowed_features := allocator_api,new_uninit
 
 rust_common_cmd = \
 	RUST_MODFILE=$(modfile) $(RUSTC_OR_CLIPPY) $(rust_flags) \
Signature · 1 line
2.40.1
#3 lore raw

Re: [PATCH 02/80] rust: add ability to register a file system

Quoted 9 lines · Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

Also add sample that uses the new public API. The registered file system cannot be mounted yet, but can be seen in /proc/filesystems.

More functionality will be added in subsequent commits.

Note that you need to add your SoB too in cases where you forward a patch, even if you are just handling the patch; see
https://docs.kernel.org/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin.

Cheers,
Miguel

#4 lore raw

PATCH 3/80 rust: define fs context

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

Also make fs mountable, but empty for now.

---
 rust/bindings/bindings_helper.h |   3 +
 rust/bindings/lib.rs            |   2 +
 rust/helpers.c                  |  12 ++
 rust/kernel/error.rs            |  46 +++++++
 rust/kernel/fs.rs               | 224 ++++++++++++++++++++++++++++++--
 rust/kernel/lib.rs              |   5 +
 samples/rust/rust_fs.rs         |  13 ++
 7 files changed, 297 insertions(+), 8 deletions(-)
rust/bindings/bindings_helper.h +3 -0
index eeb5b7c0528a..556f2e7c3ddb 100644
@@ -11,7 +11,10 @@
 #include <linux/wait.h>
 #include <linux/sched.h>
 #include <linux/fs.h>
+#include <linux/fs_parser.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
 const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO;
+
+const loff_t BINDINGS_MAX_LFS_FILESIZE = MAX_LFS_FILESIZE;
rust/bindings/lib.rs +2 -0
index 9bcbea04dac3..cd1fceb31390 100644
@@ -51,3 +51,5 @@ mod bindings_helper {
 
 pub const GFP_KERNEL: gfp_t = BINDINGS_GFP_KERNEL;
 pub const __GFP_ZERO: gfp_t = BINDINGS___GFP_ZERO;
+
+pub const MAX_LFS_FILESIZE: loff_t = BINDINGS_MAX_LFS_FILESIZE;
rust/helpers.c +12 -0
index e00c53c94c43..d6f277c3b7a7 100644
@@ -135,6 +135,18 @@ struct dentry *rust_helper_dget(struct dentry *dentry)
 }
 EXPORT_SYMBOL_GPL(rust_helper_dget);
 
+void rust_helper_lockdep_register_key(struct lock_class_key *key)
+{
+	lockdep_register_key(key);
+}
+EXPORT_SYMBOL_GPL(rust_helper_lockdep_register_key);
+
+void rust_helper_lockdep_unregister_key(struct lock_class_key *key)
+{
+	lockdep_unregister_key(key);
+}
+EXPORT_SYMBOL_GPL(rust_helper_lockdep_unregister_key);
+
 /*
  * We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
  * as the Rust `usize` type, so we can use it in contexts where Rust
rust/kernel/error.rs +46 -0

49 lines not on this page · raw message

rust/kernel/fs.rs +216 -8

258 lines not on this page · raw message

rust/kernel/lib.rs +5 -0
index 486f60912132..48da79b02422 100644
@@ -98,3 +98,8 @@ fn panic(info: &core::panic::PanicInfo<'_>) -> ! {
     // instead of `!`. See <https://github.com/rust-lang/rust-bindgen/issues/2094>.
     loop {}
 }
+
+/// Page size defined in terms of the `PAGE_SHIFT` macro from C.
+///
+/// [`PAGE_SHIFT`]: ../../../include/asm-generic/page.h
+pub const PAGE_SIZE: usize = 1 << bindings::PAGE_SHIFT;
samples/rust/rust_fs.rs +13 -0
index 2caacea50e73..2bd4e563c694 100644
@@ -13,9 +13,22 @@
 }
 
 struct RustFs;
+
+#[vtable]
+impl fs::Context<Self> for RustFs {
+    type Data = ();
+
+    fn try_new() -> Result {
+        pr_info!("context created!\n");
+        Ok(())
+    }
+}
+
 impl fs::Type for RustFs {
+    type Context = Self;
     const NAME: &'static CStr = c_str!("rustfs");
     const FLAGS: i32 = fs::flags::USERNS_MOUNT;
+    const MAGIC: u32 = 0x72757374;
 }
 
 struct FsModule {
Signature · 1 line
2.40.1
#5 lore raw

PATCH 4/80 rust: add support for file system parameters

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

This allows file system contexts to be further initialised with parameters from userspace before a fs is mounted or reconfigured.

---
 rust/helpers.c          |   9 +
 rust/kernel/fs.rs       | 142 ++++++++++-
 rust/kernel/fs/param.rs | 537 ++++++++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs      |   1 +
 samples/rust/rust_fs.rs |  15 ++
 5 files changed, 701 insertions(+), 3 deletions(-)
 create mode 100644 rust/kernel/fs/param.rs
rust/helpers.c +9 -0
index d6f277c3b7a7..b042d496649f 100644
@@ -147,6 +147,15 @@ void rust_helper_lockdep_unregister_key(struct lock_class_key *key)
 }
 EXPORT_SYMBOL_GPL(rust_helper_lockdep_unregister_key);
 
+int rust_helper_fs_parse(struct fs_context *fc,
+		const struct fs_parameter_spec *desc,
+		struct fs_parameter *param,
+		struct fs_parse_result *result)
+{
+	return fs_parse(fc, desc, param, result);
+}
+EXPORT_SYMBOL_GPL(rust_helper_fs_parse);
+
 /*
  * We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
  * as the Rust `usize` type, so we can use it in contexts where Rust
rust/kernel/fs.rs +139 -3

195 lines not on this page · raw message

rust/kernel/fs/param.rs new file +537 -0

537 lines not on this page · raw message

rust/kernel/lib.rs +1 -0
index 48da79b02422..6cf267119fda 100644
@@ -18,6 +18,7 @@
 #![feature(new_uninit)]
 #![feature(receiver_trait)]
 #![feature(unsize)]
+#![feature(const_mut_refs)]
 
 // Ensure conditional compilation based on the kernel configuration works;
 // otherwise we may silently break things like initcall handling.
samples/rust/rust_fs.rs +15 -0
index 2bd4e563c694..acaa603b0bb2 100644
@@ -18,6 +18,21 @@
 impl fs::Context<Self> for RustFs {
     type Data = ();
 
+    kernel::define_fs_params! {(),
+        {flag, "flag", |_, v| { pr_info!("flag passed-in: {v}\n"); Ok(()) } },
+        {flag_no, "flagno", |_, v| { pr_info!("flagno passed-in: {v}\n"); Ok(()) } },
+        {bool, "bool", |_, v| { pr_info!("bool passed-in: {v}\n"); Ok(()) } },
+        {u32, "u32", |_, v| { pr_info!("u32 passed-in: {v}\n"); Ok(()) } },
+        {u32oct, "u32oct", |_, v| { pr_info!("u32oct passed-in: {v}\n"); Ok(()) } },
+        {u32hex, "u32hex", |_, v| { pr_info!("u32hex passed-in: {v}\n"); Ok(()) } },
+        {s32, "s32", |_, v| { pr_info!("s32 passed-in: {v}\n"); Ok(()) } },
+        {u64, "u64", |_, v| { pr_info!("u64 passed-in: {v}\n"); Ok(()) } },
+        {string, "string", |_, v| { pr_info!("string passed-in: {v}\n"); Ok(()) } },
+        {enum, "enum", [("first", 10), ("second", 20)], |_, v| {
+            pr_info!("enum passed-in: {v}\n"); Ok(()) }
+        },
+    }
+
     fn try_new() -> Result {
         pr_info!("context created!\n");
         Ok(())
Signature · 1 line
2.40.1
#6 lore raw

PATCH 5/80 rust: kernel: add libraries required by the filesystem abstractions

In reply to Ariel Miculas

Add cred.rs, file.rs, io_buffer.rs, iov_iter.rs, mm.rs, pages.rs and user_ptr.rs from the rust branch.

---
 rust/bindings/bindings_helper.h |   4 +
 rust/helpers.c                  |  35 ++
 rust/kernel/cred.rs             |  46 ++
 rust/kernel/file.rs             | 913 ++++++++++++++++++++++++++++++++
 rust/kernel/io_buffer.rs        | 153 ++++++
 rust/kernel/iov_iter.rs         |  81 +++
 rust/kernel/lib.rs              |  11 +-
 rust/kernel/mm.rs               | 149 ++++++
 rust/kernel/pages.rs            | 144 +++++
 rust/kernel/user_ptr.rs         | 175 ++++++
 10 files changed, 1710 insertions(+), 1 deletion(-)
 create mode 100644 rust/kernel/cred.rs
 create mode 100644 rust/kernel/file.rs
 create mode 100644 rust/kernel/io_buffer.rs
 create mode 100644 rust/kernel/iov_iter.rs
 create mode 100644 rust/kernel/mm.rs
 create mode 100644 rust/kernel/pages.rs
 create mode 100644 rust/kernel/user_ptr.rs
rust/bindings/bindings_helper.h +4 -0
index 556f2e7c3ddb..b4297f6cb99f 100644
@@ -12,6 +12,10 @@
 #include <linux/sched.h>
 #include <linux/fs.h>
 #include <linux/fs_parser.h>
+#include <linux/cred.h>
+#include <linux/poll.h>
+#include <linux/uio.h>
+#include <linux/uaccess.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
rust/helpers.c +35 -0
index b042d496649f..ffe62af5ee20 100644
@@ -27,6 +27,7 @@
 #include <linux/sched/signal.h>
 #include <linux/wait.h>
 #include <linux/fs_parser.h>
+#include <linux/highmem.h>
 
 __noreturn void rust_helper_BUG(void)
 {
@@ -156,6 +157,40 @@ int rust_helper_fs_parse(struct fs_context *fc,
 }
 EXPORT_SYMBOL_GPL(rust_helper_fs_parse);
 
+const struct cred *rust_helper_get_cred(const struct cred *cred)
+{
+	return get_cred(cred);
+}
+EXPORT_SYMBOL_GPL(rust_helper_get_cred);
+
+void rust_helper_put_cred(const struct cred *cred) {
+	put_cred(cred);
+}
+EXPORT_SYMBOL_GPL(rust_helper_put_cred);
+
+struct file *rust_helper_get_file(struct file *f)
+{
+	return get_file(f);
+}
+EXPORT_SYMBOL_GPL(rust_helper_get_file);
+
+unsigned long rust_helper_clear_user(void __user *to, unsigned long n)
+{
+	return clear_user(to, n);
+}
+
+void *rust_helper_kmap(struct page *page)
+{
+	return kmap(page);
+}
+EXPORT_SYMBOL_GPL(rust_helper_kmap);
+
+void rust_helper_kunmap(struct page *page)
+{
+	return kunmap(page);
+}
+EXPORT_SYMBOL_GPL(rust_helper_kunmap);
+
 /*
  * We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
  * as the Rust `usize` type, so we can use it in contexts where Rust
rust/kernel/cred.rs new file +46 -0

46 lines not on this page · raw message

rust/kernel/file.rs new file +913 -0

913 lines not on this page · raw message

rust/kernel/io_buffer.rs new file +153 -0

153 lines not on this page · raw message

rust/kernel/iov_iter.rs new file +81 -0

81 lines not on this page · raw message

rust/kernel/lib.rs +10 -1
index 6cf267119fda..dd9c39071391 100644
@@ -13,7 +13,9 @@
 
 #![no_std]
 #![feature(allocator_api)]
+#![feature(associated_type_defaults)]
 #![feature(coerce_unsized)]
+#![feature(c_size_t)]
 #![feature(dispatch_from_dyn)]
 #![feature(new_uninit)]
 #![feature(receiver_trait)]
@@ -31,11 +33,17 @@
 #[cfg(not(test))]
 #[cfg(not(testlib))]
 mod allocator;
-pub mod fs;
 mod build_assert;
+pub mod cred;
 pub mod error;
+pub mod file;
+pub mod fs;
 pub mod init;
+pub mod io_buffer;
 pub mod ioctl;
+pub mod iov_iter;
+pub mod mm;
+pub mod pages;
 pub mod prelude;
 pub mod print;
 mod static_assert;
@@ -45,6 +53,7 @@
 pub mod sync;
 pub mod task;
 pub mod types;
+pub mod user_ptr;
 
 #[doc(hidden)]
 pub use bindings;
rust/kernel/mm.rs new file +149 -0

149 lines not on this page · raw message

rust/kernel/pages.rs new file +144 -0

144 lines not on this page · raw message

rust/kernel/user_ptr.rs new file +175 -0

175 lines not on this page · raw message

Signature · 1 line
2.40.1
#7 lore raw

Re: [PATCH 05/80] rust: kernel: add libraries required by the filesystem abstractions

Quoted 5 lines · Ariel Miculas

Add cred.rs, file.rs, io_buffer.rs, iov_iter.rs, mm.rs, pages.rs and user_ptr.rs from the rust branch.

I guess you added these in bulk for RFC purposes, so that is fine, but eventually these will need to be submitted individually etc.

An alternative (for RFCs), if you want people to focus on another part of the series, is to create a branch somewhere with the dependencies applied and then base the patches on top (being clear that the dependencies would be needed, of course). Some extra notes at [1] for others that may be interested in that approach.

[1] https://rust-for-linux.com/contributing#submitting-new-abstractions-and-modules

Cheers,
Miguel

#8 lore raw

PATCH 6/80 rust: allow fs driver to initialise new superblocks

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

They are also allowed to specify the type of superblock keying.

With this change, drivers get a new callback, fill_super, that they need to implement to initialise new superblocks.

In subsequent commits, they'll be able to populate the root with entries, but for now it's empty.

---
 rust/kernel/fs.rs       | 406 ++++++++++++++++++++++++++++++++--------
 rust/kernel/fs/param.rs |  17 +-
 samples/rust/rust_fs.rs |  14 +-
 3 files changed, 357 insertions(+), 80 deletions(-)
rust/kernel/fs.rs +328 -78
index e68b67b8adb0..fb4a814e1fc8 100644
@@ -7,11 +7,31 @@
 use crate::{bindings, error::code::*, str::CStr, ThisModule};
 use crate::error::{to_result, from_kernel_result, Error, Result};
 use crate::types::{AlwaysRefCounted, ForeignOwnable, ScopeGuard};
-use core::{cell::UnsafeCell, marker::PhantomPinned, pin::Pin, ptr};
+use core::{cell::UnsafeCell, marker::PhantomData, marker::PhantomPinned, pin::Pin, ptr};
 use macros::vtable;
 
 pub mod param;
 
+/// Type of superblock keying.
+///
+/// It determines how C's `fs_context_operations::get_tree` is implemented.
+pub enum Super {
+    /// Only one such superblock may exist.
+    Single,
+
+    /// As [`Super::Single`], but reconfigure if it exists.
+    SingleReconf,
+
+    /// Superblocks with different data pointers may exist.
+    Keyed,
+
+    /// Multiple independent superblocks may exist.
+    Independent,
+
+    /// Uses a block device.
+    BlockDev,
+}
+
 /// A file system context.
 ///
 /// It is used to gather configuration to then mount or reconfigure a file system.
@@ -48,6 +68,16 @@ fn parse_monolithic<'a>(
     ) -> Result<Option<&'a mut [u8]>> {
         Ok(None)
     }
+
+    /// Returns the superblock data to be used by this file system context.
+    ///
+    /// This is only needed when [`Type::SUPER_TYPE`] is [`Super::Keyed`], otherwise it is never
+    /// called. In the former case, when the fs is being mounted, an existing superblock is reused
+    /// if one can be found with the same data as the returned value; otherwise a new superblock is
+    /// created.
+    fn tree_key(_data: &mut Self::Data) -> Result<T::Data> {
+        Err(ENOTSUPP)
+    }
 }
 
 struct Tables<T: Type + ?Sized>(T);
@@ -67,12 +97,22 @@ impl<T: Type + ?Sized> Tables<T> {
 
     unsafe extern "C" fn free_callback(fc: *mut bindings::fs_context) {
         // SAFETY: The callback contract guarantees that `fc` is valid.
-        let ptr = unsafe { (*fc).fs_private };
+        let fc = unsafe { &*fc };
+
+        let ptr = fc.fs_private;
         if !ptr.is_null() {
             // SAFETY: `fs_private` was initialised with the result of a `to_pointer` call in
             // `init_fs_context_callback`, so it's ok to call `from_foreign` here.
             unsafe { <T::Context as Context<T>>::Data::from_foreign(ptr) };
         }
+
+        let ptr = fc.s_fs_info;
+        if !ptr.is_null() {
+            // SAFETY: `s_fs_info` may be initialised with the result of a `to_pointer` call in
+            // `get_tree_callback` when keyed superblocks are used (`get_tree_keyed` sets it), so
+            // it's ok to call `from_foreign` here.
+            unsafe { T::Data::from_foreign(ptr) };
+        }
     }
 
     unsafe extern "C" fn parse_param_callback(
@@ -84,8 +124,8 @@ impl<T: Type + ?Sized> Tables<T> {
             let ptr = unsafe { (*fc).fs_private };
 
             // SAFETY: The value of `ptr` (coming from `fs_private` was initialised in
-            // `init_fs_context_callback` to the result of an `into_pointer` call. Since the
-            // context is valid, `from_pointer` wasn't called yet, so `ptr` is valid. Additionally,
+            // `init_fs_context_callback` to the result of an `into_foreign` call. Since the
+            // context is valid, `from_foreign` wasn't called yet, so `ptr` is valid. Additionally,
             // the callback contract guarantees that callbacks are serialised, so it is ok to
             // mutably reference it.
             let mut data =
@@ -126,73 +166,91 @@ impl<T: Type + ?Sized> Tables<T> {
 
     unsafe extern "C" fn fill_super_callback(
         sb_ptr: *mut bindings::super_block,
-        _fc: *mut bindings::fs_context,
+        fc: *mut bindings::fs_context,
     ) -> core::ffi::c_int {
         from_kernel_result! {
-            // The following is temporary code to create the root inode and dentry. It will be
-            // replaced with calls to Rust code.
-
-            // SAFETY: The callback contract guarantees that `sb_ptr` is the only pointer to a
-            // newly-allocated superblock, so it is safe to mutably reference it.
-            let sb = unsafe { &mut *sb_ptr };
-
-            sb.s_maxbytes = bindings::MAX_LFS_FILESIZE;
-            sb.s_blocksize = crate::PAGE_SIZE as _;
-            sb.s_blocksize_bits = bindings::PAGE_SHIFT as _;
-            sb.s_magic = T::MAGIC as _;
-            sb.s_op = &Tables::<T>::SUPER_BLOCK;
-            sb.s_time_gran = 1;
-
-            // Create and initialise the root inode.
-
-            // SAFETY: `sb` was just created initialised, so it is safe pass it to `new_inode`.
-            let inode = unsafe { bindings::new_inode(sb) };
-            if inode.is_null() {
-                return Err(ENOMEM);
-            }
-
-            {
-                // SAFETY: This is a newly-created inode. No other references to it exist, so it is
-                // safe to mutably dereference it.
-                let inode = unsafe { &mut *inode };
-
-                // SAFETY: `current_time` requires that `inode.sb` be valid, which is the case here
-                // since we allocated the inode through the superblock.
-                let time = unsafe { bindings::current_time(inode) };
-                inode.i_ino = 1;
-                inode.i_mode = (bindings::S_IFDIR | 0o755) as _;
-                inode.i_mtime = time;
-                inode.i_atime = time;
-                inode.i_ctime = time;
-
-                // SAFETY: `simple_dir_operations` never changes, it's safe to reference it.
-                inode.__bindgen_anon_3.i_fop = unsafe { &bindings::simple_dir_operations };
-
-                // SAFETY: `simple_dir_inode_operations` never changes, it's safe to reference it.
-                inode.i_op = unsafe { &bindings::simple_dir_inode_operations };
-
-                // SAFETY: `inode` is valid for write.
-                unsafe { bindings::set_nlink(inode, 2) };
-            }
-
-            // SAFETY: `d_make_root` requires that `inode` be valid and referenced, which is the
-            // case for this call.
-            //
-            // It takes over the inode, even on failure, so we don't need to clean it up.
-            let dentry = unsafe { bindings::d_make_root(inode) };
-            if dentry.is_null() {
-                return Err(ENOMEM);
-            }
+            // SAFETY: The callback contract guarantees that `fc` is valid. It also guarantees that
+            // the callbacks are serialised for a given `fc`, so it is safe to mutably dereference
+            // it.
+            let fc = unsafe { &mut *fc };
+            let ptr = core::mem::replace(&mut fc.fs_private, ptr::null_mut());
 
-            sb.s_root = dentry;
+            // SAFETY: The value of `ptr` (coming from `fs_private` was initialised in
+            // `init_fs_context_callback` to the result of an `into_foreign` call. The context is
+            // being used to initialise a superblock, so we took over `ptr` (`fs_private` is set to
+            // null now) and call `from_foreign` below.
+            let data =
+                unsafe { <<T::Context as Context<T>>::Data as ForeignOwnable>::from_foreign(ptr) };
+
+            // SAFETY: The callback contract guarantees that `sb_ptr` is a unique pointer to a
+            // newly-created superblock.
+            let newsb = unsafe { NewSuperBlock::new(sb_ptr) };
+            T::fill_super(data, newsb)?;
             Ok(0)
         }
     }
 
     unsafe extern "C" fn get_tree_callback(fc: *mut bindings::fs_context) -> core::ffi::c_int {
-        // SAFETY: `fc` is valid per the callback contract. `fill_super_callback` also has the
-        // right type and is a valid callback.
-        unsafe { bindings::get_tree_nodev(fc, Some(Self::fill_super_callback)) }
+        // N.B. When new types are added below, we may need to update `kill_sb_callback` to ensure
+        // that we're cleaning up properly.
+        match T::SUPER_TYPE {
+            Super::Single => unsafe {
+                // SAFETY: `fc` is valid per the callback contract. `fill_super_callback` also has
+                // the right type and is a valid callback.
+                bindings::get_tree_single(fc, Some(Self::fill_super_callback))
+            },
+            Super::SingleReconf => unsafe {
+                // SAFETY: `fc` is valid per the callback contract. `fill_super_callback` also has
+                // the right type and is a valid callback.
+                bindings::get_tree_single_reconf(fc, Some(Self::fill_super_callback))
+            },
+            Super::Independent => unsafe {
+                // SAFETY: `fc` is valid per the callback contract. `fill_super_callback` also has
+                // the right type and is a valid callback.
+                bindings::get_tree_nodev(fc, Some(Self::fill_super_callback))
+            },
+            Super::BlockDev => unsafe {
+                // SAFETY: `fc` is valid per the callback contract. `fill_super_callback` also has
+                // the right type and is a valid callback.
+                bindings::get_tree_bdev(fc, Some(Self::fill_super_callback))
+            },
+            Super::Keyed => {
+                from_kernel_result! {
+                    // SAFETY: `fc` is valid per the callback contract.
+                    let ctx = unsafe { &*fc };
+                    let ptr = ctx.fs_private;
+
+                    // SAFETY: The value of `ptr` (coming from `fs_private` was initialised in
+                    // `init_fs_context_callback` to the result of an `into_foreign` call. Since
+                    // the context is valid, `from_foreign` wasn't called yet, so `ptr` is valid.
+                    // Additionally, the callback contract guarantees that callbacks are
+                    // serialised, so it is ok to mutably reference it.
+                    let mut data = unsafe {
+                        <<T::Context as Context<T>>::Data as ForeignOwnable>::borrow_mut(ptr)
+                    };
+                    let fs_data = T::Context::tree_key(&mut data)?;
+                    let fs_data_ptr = fs_data.into_foreign();
+
+                    // `get_tree_keyed` reassigns `ctx.s_fs_info`, which should be ok because
+                    // nowhere else is it assigned a non-null value. However, we add the assert
+                    // below to ensure that there are no unexpected paths on the C side that may do
+                    // this.
+                    assert_eq!(ctx.s_fs_info, core::ptr::null_mut());
+
+                    // SAFETY: `fc` is valid per the callback contract. `fill_super_callback` also
+                    // has the right type and is a valid callback. Lastly, we just called
+                    // `into_foreign` above, so `fs_data_ptr` is also valid.
+                    to_result(unsafe {
+                        bindings::get_tree_keyed(
+                            fc,
+                            Some(Self::fill_super_callback),
+                            fs_data_ptr as _,
+                        )
+                    })?;
+                    Ok(0)
+                }
+            }
+        }
     }
 
     unsafe extern "C" fn reconfigure_callback(_fc: *mut bindings::fs_context) -> core::ffi::c_int {
@@ -208,8 +266,8 @@ impl<T: Type + ?Sized> Tables<T> {
             let ptr = unsafe { (*fc).fs_private };
 
             // SAFETY: The value of `ptr` (coming from `fs_private` was initialised in
-            // `init_fs_context_callback` to the result of an `into_pointer` call. Since the
-            // context is valid, `from_pointer` wasn't called yet, so `ptr` is valid. Additionally,
+            // `init_fs_context_callback` to the result of an `into_foreign` call. Since the
+            // context is valid, `from_foreign` wasn't called yet, so `ptr` is valid. Additionally,
             // the callback contract guarantees that callbacks are serialised, so it is ok to
             // mutably reference it.
             let mut data =
@@ -270,18 +328,25 @@ pub trait Type {
     /// The context used to build fs configuration before it is mounted or reconfigured.
     type Context: Context<Self> + ?Sized;
 
+    /// Data associated with each file system instance.
+    type Data: ForeignOwnable + Send + Sync = ();
+
+    /// Determines how superblocks for this file system type are keyed.
+    const SUPER_TYPE: Super;
+
     /// The name of the file system type.
     const NAME: &'static CStr;
 
-    /// The magic number associated with the file system.
-    ///
-    /// This is normally one of the values in `include/uapi/linux/magic.h`.
-    const MAGIC: u32;
-
     /// The flags of this file system type.
     ///
     /// It is a combination of the flags in the [`flags`] module.
     const FLAGS: i32;
+
+    /// Initialises a super block for this file system type.
+    fn fill_super(
+        data: <Self::Context as Context<Self>>::Data,
+        sb: NewSuperBlock<'_, Self>,
+    ) -> Result<&SuperBlock<Self>>;
 }
 
 /// File system flags.
@@ -425,14 +490,33 @@ unsafe fn unregister_keys(fs: *mut bindings::file_system_type) {
     }
 
     unsafe extern "C" fn kill_sb_callback<T: Type + ?Sized>(sb_ptr: *mut bindings::super_block) {
-        // SAFETY: We always call `get_tree_nodev` from `get_tree_callback`, so we never have a
-        // device, so it is ok to call the function below. Additionally, the callback contract
-        // guarantees that `sb_ptr` is valid.
-        unsafe { bindings::kill_anon_super(sb_ptr) }
-
-        // SAFETY: The callback contract guarantees that `sb_ptr` is valid, and the `kill_sb`
-        // callback being called implies that the `s_type` is also valid.
-        unsafe { Self::unregister_keys((*sb_ptr).s_type) };
+        if let Super::BlockDev = T::SUPER_TYPE {
+            // SAFETY: When the superblock type is `BlockDev`, we have a block device so it's safe
+            // to call `kill_block_super`. Additionally, the callback contract guarantees that
+            // `sb_ptr` is valid.
+            unsafe { bindings::kill_block_super(sb_ptr) }
+        } else {
+            // SAFETY: We always call a `get_tree_nodev` variant from `get_tree_callback` without a
+            // device when `T::SUPER_TYPE` is not `BlockDev`, so we never have a device in such
+            // cases, therefore it is ok to call the function below. Additionally, the callback
+            // contract guarantees that `sb_ptr` is valid.
+            unsafe { bindings::kill_anon_super(sb_ptr) }
+        }
+
+        // SAFETY: The callback contract guarantees that `sb_ptr` is valid.
+        let sb = unsafe { &*sb_ptr };
+
+        // SAFETY: The `kill_sb` callback being called implies that the `s_type` field is valid.
+        unsafe { Self::unregister_keys(sb.s_type) };
+
+        let ptr = sb.s_fs_info;
+        if !ptr.is_null() {
+            // SAFETY: The only place where `s_fs_info` is assigned is `NewSuperBlock::init`, where
+            // it's initialised with the result of a `to_pointer` call. We checked above that ptr
+            // is non-null because it would be null if we never reached the point where we init the
+            // field.
+            unsafe { T::Data::from_foreign(ptr) };
+        }
     }
 }
 
@@ -446,6 +530,172 @@ fn drop(&mut self) {
     }
 }
 
+/// State of [`NewSuperBlock`] that indicates that [`NewSuperBlock::init`] needs to be called
+/// eventually.
+pub struct NeedsInit;
+
+/// State of [`NewSuperBlock`] that indicates that [`NewSuperBlock::init_root`] needs to be called
+/// eventually.
+pub struct NeedsRoot;
+
+/// Required superblock parameters.
+///
+/// This is used in [`NewSuperBlock::init`].
+pub struct SuperParams {
+    /// The magic number of the superblock.
+    pub magic: u32,
+
+    /// The size of a block in powers of 2 (i.e., for a value of `n`, the size is `2^n`.
+    pub blocksize_bits: u8,
+
+    /// Maximum size of a file.
+    pub maxbytes: i64,
+
+    /// Granularity of c/m/atime in ns (cannot be worse than a second).
+    pub time_gran: u32,
+}
+
+impl SuperParams {
+    /// Default value for instances of [`SuperParams`].
+    pub const DEFAULT: Self = Self {
+        magic: 0,
+        blocksize_bits: crate::PAGE_SIZE as _,
+        maxbytes: bindings::MAX_LFS_FILESIZE,
+        time_gran: 1,
+    };
+}
+
+/// A superblock that is still being initialised.
+///
+/// It uses type states to ensure that callers use the right sequence of calls.
+///
+/// # Invariants
+///
+/// The superblock is a newly-created one and this is the only active pointer to it.
+pub struct NewSuperBlock<'a, T: Type + ?Sized, S = NeedsInit> {
+    sb: *mut bindings::super_block,
+    _p: PhantomData<(&'a T, S)>,
+}
+
+impl<'a, T: Type + ?Sized> NewSuperBlock<'a, T, NeedsInit> {
+    /// Creates a new instance of [`NewSuperBlock`].
+    ///
+    /// # Safety
+    ///
+    /// `sb` must point to a newly-created superblock and it must be the only active pointer to it.
+    unsafe fn new(sb: *mut bindings::super_block) -> Self {
+        // INVARIANT: The invariants are satisfied by the safety requirements of this function.
+        Self {
+            sb,
+            _p: PhantomData,
+        }
+    }
+
+    /// Initialises the superblock so that it transitions to the [`NeedsRoot`] type state.
+    pub fn init(
+        self,
+        data: T::Data,
+        params: &SuperParams,
+    ) -> Result<NewSuperBlock<'a, T, NeedsRoot>> {
+        // SAFETY: The type invariant guarantees that `self.sb` is the only pointer to a
+        // newly-allocated superblock, so it is safe to mutably reference it.
+        let sb = unsafe { &mut *self.sb };
+
+        sb.s_magic = params.magic as _;
+        sb.s_op = &Tables::<T>::SUPER_BLOCK;
+        sb.s_maxbytes = params.maxbytes;
+        sb.s_time_gran = params.time_gran;
+        sb.s_blocksize_bits = params.blocksize_bits;
+        sb.s_blocksize = 1;
+        if sb.s_blocksize.leading_zeros() < params.blocksize_bits.into() {
+            return Err(EINVAL);
+        }
+        sb.s_blocksize = 1 << sb.s_blocksize_bits;
+
+        // Keyed file systems already have `s_fs_info` initialised.
+        let info = data.into_foreign() as *mut _;
+        if let Super::Keyed = T::SUPER_TYPE {
+            // SAFETY: We just called `into_foreign` above.
+            unsafe { T::Data::from_foreign(info) };
+
+            if sb.s_fs_info != info {
+                return Err(EINVAL);
+            }
+        } else {
+            sb.s_fs_info = info;
+        }
+
+        Ok(NewSuperBlock {
+            sb: self.sb,
+            _p: PhantomData,
+        })
+    }
+}
+
+impl<'a, T: Type + ?Sized> NewSuperBlock<'a, T, NeedsRoot> {
+    /// Initialises the root of the superblock.
+    pub fn init_root(self) -> Result<&'a SuperBlock<T>> {
+        // The following is temporary code to create the root inode and dentry. It will be replaced
+        // once we allow inodes and dentries to be created directly from Rust code.
+
+        // SAFETY: `sb` is initialised (`NeedsRoot` typestate implies it), so it is safe to pass it
+        // to `new_inode`.
+        let inode = unsafe { bindings::new_inode(self.sb) };
+        if inode.is_null() {
+            return Err(ENOMEM);
+        }
+
+        {
+            // SAFETY: This is a newly-created inode. No other references to it exist, so it is
+            // safe to mutably dereference it.
+            let inode = unsafe { &mut *inode };
+
+            // SAFETY: `current_time` requires that `inode.sb` be valid, which is the case here
+            // since we allocated the inode through the superblock.
+            let time = unsafe { bindings::current_time(inode) };
+            inode.i_ino = 1;
+            inode.i_mode = (bindings::S_IFDIR | 0o755) as _;
+            inode.i_mtime = time;
+            inode.i_atime = time;
+            inode.i_ctime = time;
+
+            // SAFETY: `simple_dir_operations` never changes, it's safe to reference it.
+            inode.__bindgen_anon_3.i_fop = unsafe { &bindings::simple_dir_operations };
+
+            // SAFETY: `simple_dir_inode_operations` never changes, it's safe to reference it.
+            inode.i_op = unsafe { &bindings::simple_dir_inode_operations };
+
+            // SAFETY: `inode` is valid for write.
+            unsafe { bindings::set_nlink(inode, 2) };
+        }
+
+        // SAFETY: `d_make_root` requires that `inode` be valid and referenced, which is the
+        // case for this call.
+        //
+        // It takes over the inode, even on failure, so we don't need to clean it up.
+        let dentry = unsafe { bindings::d_make_root(inode) };
+        if dentry.is_null() {
+            return Err(ENOMEM);
+        }
+
+        // SAFETY: The typestate guarantees that `self.sb` is valid.
+        unsafe { (*self.sb).s_root = dentry };
+
+        // SAFETY: The typestate guarantees that `self.sb` is initialised and we just finished
+        // setting its root, so it's a fully ready superblock.
+        Ok(unsafe { &mut *self.sb.cast() })
+    }
+}
+
+/// A file system super block.
+///
+/// Wraps the kernel's `struct super_block`.
+#[repr(transparent)]
+pub struct SuperBlock<T: Type + ?Sized>(
+    pub(crate) UnsafeCell<bindings::super_block>,
+    PhantomData<T>,
+);
+
 /// Wraps the kernel's `struct inode`.
 ///
 /// # Invariants
rust/kernel/fs/param.rs +16 -1
index f0b4393b6b67..44b4e895a1eb 100644
@@ -487,9 +487,24 @@ macro_rules! count_brace_items {
 ///
 /// # impl fs::Type for Example {
 /// #    type Context = Self;
+/// #    const SUPER_TYPE: fs::Super = fs::Super::Independent;
 /// #    const NAME: &'static CStr = c_str!("example");
 /// #    const FLAGS: i32 = 0;
-/// #    const MAGIC: u32 = 0x6578616d;
+/// #
+/// #    fn fill_super<'a>(
+/// #        _data: Box<State>,
+/// #        sb: fs::NewSuperBlock<'_, Self>,
+/// #    ) -> Result<&fs::SuperBlock<Self>> {
+/// #        let sb = sb.init(
+/// #            (),
+/// #            &fs::SuperParams {
+/// #                magic: 0x6578616d,
+/// #                ..fs::SuperParams::DEFAULT
+/// #            },
+/// #        )?;
+/// #        let sb = sb.init_root()?;
+/// #        Ok(sb)
+/// #    }
 /// # }
 /// ```
 #[macro_export]
samples/rust/rust_fs.rs +13 -1
index acaa603b0bb2..5d8880d5830b 100644
@@ -41,9 +41,21 @@ fn try_new() -> Result {
 
 impl fs::Type for RustFs {
     type Context = Self;
+    const SUPER_TYPE: fs::Super = fs::Super::Independent;
     const NAME: &'static CStr = c_str!("rustfs");
     const FLAGS: i32 = fs::flags::USERNS_MOUNT;
-    const MAGIC: u32 = 0x72757374;
+
+    fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBlock<Self>> {
+        let sb = sb.init(
+            (),
+            &fs::SuperParams {
+                magic: 0x72757374,
+                ..fs::SuperParams::DEFAULT
+            },
+        )?;
+        let sb = sb.init_root()?;
+        Ok(sb)
+    }
 }
 
 struct FsModule {
Signature · 1 line
2.40.1
#9 lore raw

PATCH 7/80 rust: add `module_fs` macro

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

This reduces the amount of code needed to implement a module that only implements a file system.

---
 rust/kernel/fs.rs       | 77 +++++++++++++++++++++++++++++++++++++++++
 rust/kernel/prelude.rs  |  1 +
 samples/rust/rust_fs.rs | 16 ++-------
 3 files changed, 80 insertions(+), 14 deletions(-)
rust/kernel/fs.rs +77 -0
index fb4a814e1fc8..86c306c19e0a 100644
@@ -4,6 +4,7 @@
 //!
 //! C headers: [`include/linux/fs.h`](../../../../include/linux/fs.h)
 
+use alloc::boxed::Box;
 use crate::{bindings, error::code::*, str::CStr, ThisModule};
 use crate::error::{to_result, from_kernel_result, Error, Result};
 use crate::types::{AlwaysRefCounted, ForeignOwnable, ScopeGuard};
@@ -757,3 +758,79 @@ pub(crate) unsafe fn from_ptr<'a>(ptr: *const bindings::filename) -> &'a Filenam
         unsafe { &*ptr.cast() }
     }
 }
+
+/// Kernel module that exposes a single file system implemented by `T`.
+pub struct Module<T: Type> {
+    _fs: Pin<Box<Registration>>,
+    _p: PhantomData<T>,
+}
+
+impl<T: Type + Sync> crate::Module for Module<T> {
+    fn init(_name: &'static CStr, module: &'static ThisModule) -> Result<Self> {
+        let mut reg = Pin::from(Box::try_new(Registration::new())?);
+        reg.as_mut().register::<T>(module)?;
+        Ok(Self {
+            _fs: reg,
+            _p: PhantomData,
+        })
+    }
+}
+
+/// Declares a kernel module that exposes a single file system.
+///
+/// The `type` argument must be a type which implements the [`Type`] trait. Also accepts various
+/// forms of kernel metadata.
+///
+/// # Examples
+///
+/// ```ignore
+/// use kernel::prelude::*;
+/// use kernel::{c_str, fs};
+///
+/// module_fs! {
+///     type: MyFs,
+///     name: b"my_fs_kernel_module",
+///     author: b"Rust for Linux Contributors",
+///     description: b"My very own file system kernel module!",
+///     license: b"GPL",
+/// }
+///
+/// struct MyFs;
+///
+/// #[vtable]
+/// impl fs::Context<Self> for MyFs {
+///     type Data = ();
+///     fn try_new() -> Result {
+///         Ok(())
+///     }
+/// }
+///
+/// impl fs::Type for MyFs {
+///     type Context = Self;
+///     const SUPER_TYPE: fs::Super = fs::Super::Independent;
+///     const NAME: &'static CStr = c_str!("example");
+///     const FLAGS: i32 = 0;
+///
+///     fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBlock<Self>> {
+///         let sb = sb.init(
+///             (),
+///             &fs::SuperParams {
+///                 magic: 0x6578616d,
+///                 ..fs::SuperParams::DEFAULT
+///             },
+///         )?;
+///         let sb = sb.init_root()?;
+///         Ok(sb)
+///     }
+/// }
+/// ```
+#[macro_export]
+macro_rules! module_fs {
+    (type: $type:ty, $($f:tt)*) => {
+        type ModuleType = $crate::fs::Module<$type>;
+        $crate::macros::module! {
+            type: ModuleType,
+            $($f)*
+        }
+    }
+}
rust/kernel/prelude.rs +1 -0
index c28587d68ebc..37789bc8a796 100644
@@ -28,6 +28,7 @@
 pub use super::{pr_alert, pr_crit, pr_debug, pr_emerg, pr_err, pr_info, pr_notice, pr_warn};
 
 pub use super::{init, pin_init, try_init, try_pin_init};
+pub use super::{module_fs, module_misc_device};
 
 pub use super::static_assert;
 
samples/rust/rust_fs.rs +2 -14
index 5d8880d5830b..064ead97dd98 100644
@@ -5,8 +5,8 @@
 use kernel::prelude::*;
 use kernel::{c_str, fs};
 
-module! {
-    type: FsModule,
+module_fs! {
+    type: RustFs,
     name: "rust_fs",
     author: "Rust for Linux Contributors",
     license: "GPL",
@@ -57,15 +57,3 @@ fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBl
         Ok(sb)
     }
 }
-
-struct FsModule {
-    _fs: Pin<Box<fs::Registration>>,
-}
-
-impl kernel::Module for FsModule {
-    fn init(module: &'static ThisModule) -> Result<Self> {
-        let mut reg = Pin::from(Box::try_new(fs::Registration::new())?);
-        reg.as_mut().register::<RustFs>(module)?;
-        Ok(Self { _fs: reg })
-    }
-}
Signature · 1 line
2.40.1
#10 lore raw

PATCH 8/80 WIP: rust: allow fs to be populated

In reply to Ariel Miculas

From: Wedson Almeida Filho <[email protected]>

---
 rust/bindings/bindings_helper.h |   4 +
 rust/bindings/lib.rs            |   3 +
 rust/helpers.c                  |   7 +
 rust/kernel/fs.rs               | 777 +++++++++++++++++++++++++++++---
 rust/kernel/fs/param.rs         |   8 +-
 rust/kernel/prelude.rs          |   1 -
 samples/rust/rust_fs.rs         |  51 ++-
 7 files changed, 778 insertions(+), 73 deletions(-)
rust/bindings/bindings_helper.h +4 -0
index b4297f6cb99f..d15a698439e1 100644
@@ -22,3 +22,7 @@ const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
 const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO;
 
 const loff_t BINDINGS_MAX_LFS_FILESIZE = MAX_LFS_FILESIZE;
+
+const slab_flags_t BINDINGS_SLAB_RECLAIM_ACCOUNT = SLAB_RECLAIM_ACCOUNT;
+const slab_flags_t BINDINGS_SLAB_MEM_SPREAD = SLAB_MEM_SPREAD;
+const slab_flags_t BINDINGS_SLAB_ACCOUNT = SLAB_ACCOUNT;
rust/bindings/lib.rs +3 -0
index cd1fceb31390..8655d73b6785 100644
@@ -53,3 +53,6 @@ mod bindings_helper {
 pub const __GFP_ZERO: gfp_t = BINDINGS___GFP_ZERO;
 
 pub const MAX_LFS_FILESIZE: loff_t = BINDINGS_MAX_LFS_FILESIZE;
+
+pub const SLAB_RECLAIM_ACCOUNT: slab_flags_t = BINDINGS_SLAB_RECLAIM_ACCOUNT;
+pub const SLAB_MEM_SPREAD: slab_flags_t = BINDINGS_SLAB_MEM_SPREAD;
rust/helpers.c +7 -0
index ffe62af5ee20..efbe9d917a57 100644
@@ -191,6 +191,13 @@ void rust_helper_kunmap(struct page *page)
 }
 EXPORT_SYMBOL_GPL(rust_helper_kunmap);
 
+void *rust_helper_alloc_inode_sb(struct super_block *sb,
+		struct kmem_cache *cache, gfp_t gfp)
+{
+	return alloc_inode_sb(sb, cache, gfp);
+}
+EXPORT_SYMBOL_GPL(rust_helper_alloc_inode_sb);
+
 /*
  * We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
  * as the Rust `usize` type, so we can use it in contexts where Rust
rust/kernel/fs.rs +709 -68

941 lines not on this page · raw message

rust/kernel/fs/param.rs +7 -1
index 44b4e895a1eb..1a31130c6d1e 100644
@@ -502,7 +502,13 @@ macro_rules! count_brace_items {
 /// #                ..fs::SuperParams::DEFAULT
 /// #            },
 /// #        )?;
-/// #        let sb = sb.init_root()?;
+/// #        let root_inode = sb.try_new_dcache_dir_inode(fs::INodeParams {
+/// #            mode: 0o755,
+/// #            ino: 1,
+/// #            value: (),
+/// #        })?;
+/// #        let root = sb.try_new_root_dentry(root_inode)?;
+/// #        let sb = sb.init_root(root)?;
 /// #        Ok(sb)
 /// #    }
 /// # }
rust/kernel/prelude.rs +0 -1
index 37789bc8a796..c28587d68ebc 100644
@@ -28,7 +28,6 @@
 pub use super::{pr_alert, pr_crit, pr_debug, pr_emerg, pr_err, pr_info, pr_notice, pr_warn};
 
 pub use super::{init, pin_init, try_init, try_pin_init};
-pub use super::{module_fs, module_misc_device};
 
 pub use super::static_assert;
 
samples/rust/rust_fs.rs +48 -3

78 lines not on this page · raw message

Signature · 1 line
2.40.1
#11 lore raw

PATCH 9/80 rust: kernel: backport the delay module from the rust branch

In reply to Ariel Miculas
---
 rust/kernel/delay.rs | 104 +++++++++++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs   |   1 +
 2 files changed, 105 insertions(+)
 create mode 100644 rust/kernel/delay.rs
rust/kernel/delay.rs new file +104 -0
new file mode 100644
index 000000000000..d28d45ec2e57
@@ -0,0 +1,104 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Delay functions for operations like sleeping.
+//!
+//! C header: [`include/linux/delay.h`](../../../../include/linux/delay.h)
+
+use crate::bindings;
+use core::{cmp::min, time::Duration};
+
+const MILLIS_PER_SEC: u64 = 1_000;
+
+fn coarse_sleep_conversion(duration: Duration) -> core::ffi::c_uint {
+    let milli_as_nanos = Duration::MILLISECOND.subsec_nanos();
+
+    // Rounds the nanosecond component of `duration` up to the nearest millisecond.
+    let nanos_as_millis = duration.subsec_nanos().wrapping_add(milli_as_nanos - 1) / milli_as_nanos;
+
+    // Saturates the second component of `duration` to `c_uint::MAX`.
+    let seconds_as_millis = min(
+        duration.as_secs().saturating_mul(MILLIS_PER_SEC),
+        u64::from(core::ffi::c_uint::MAX),
+    ) as core::ffi::c_uint;
+
+    seconds_as_millis.saturating_add(nanos_as_millis)
+}
+
+/// Sleeps safely even with waitqueue interruptions.
+///
+/// This function forwards the call to the C side `msleep` function. As a result,
+/// `duration` will be rounded up to the nearest millisecond if granularity less
+/// than a millisecond is provided. Any [`Duration`] that exceeds
+/// [`c_uint::MAX`][core::ffi::c_uint::MAX] in milliseconds is saturated.
+///
+/// # Examples
+///
+// Keep these in sync with `test_coarse_sleep_examples`.
+/// ```
+/// # use core::time::Duration;
+/// # use kernel::delay::coarse_sleep;
+/// coarse_sleep(Duration::ZERO);                   // Equivalent to `msleep(0)`.
+/// coarse_sleep(Duration::from_nanos(1));          // Equivalent to `msleep(1)`.
+///
+/// coarse_sleep(Duration::from_nanos(1_000_000));  // Equivalent to `msleep(1)`.
+/// coarse_sleep(Duration::from_nanos(1_000_001));  // Equivalent to `msleep(2)`.
+/// coarse_sleep(Duration::from_nanos(1_999_999));  // Equivalent to `msleep(2)`.
+///
+/// coarse_sleep(Duration::from_millis(1));         // Equivalent to `msleep(1)`.
+/// coarse_sleep(Duration::from_millis(2));         // Equivalent to `msleep(2)`.
+///
+/// coarse_sleep(Duration::from_secs(1));           // Equivalent to `msleep(1000)`.
+/// coarse_sleep(Duration::new(1, 1));              // Equivalent to `msleep(1001)`.
+/// coarse_sleep(Duration::new(1, 2));              // Equivalent to `msleep(1001)`.
+/// ```
+pub fn coarse_sleep(duration: Duration) {
+    // SAFETY: msleep is safe for all values of an `unsigned int`.
+    unsafe { bindings::msleep(coarse_sleep_conversion(duration)) }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{coarse_sleep_conversion, MILLIS_PER_SEC};
+    use core::time::Duration;
+
+    #[test]
+    fn test_coarse_sleep_examples() {
+        // Keep these in sync with `coarse_sleep`'s `# Examples` section.
+
+        assert_eq!(coarse_sleep_conversion(Duration::ZERO), 0);
+        assert_eq!(coarse_sleep_conversion(Duration::from_nanos(1)), 1);
+
+        assert_eq!(coarse_sleep_conversion(Duration::from_nanos(1_000_000)), 1);
+        assert_eq!(coarse_sleep_conversion(Duration::from_nanos(1_000_001)), 2);
+        assert_eq!(coarse_sleep_conversion(Duration::from_nanos(1_999_999)), 2);
+
+        assert_eq!(coarse_sleep_conversion(Duration::from_millis(1)), 1);
+        assert_eq!(coarse_sleep_conversion(Duration::from_millis(2)), 2);
+
+        assert_eq!(coarse_sleep_conversion(Duration::from_secs(1)), 1000);
+        assert_eq!(coarse_sleep_conversion(Duration::new(1, 1)), 1001);
+        assert_eq!(coarse_sleep_conversion(Duration::new(1, 2)), 1001);
+    }
+
+    #[test]
+    fn test_coarse_sleep_saturation() {
+        assert!(
+            coarse_sleep_conversion(Duration::new(
+                core::ffi::c_uint::MAX as u64 / MILLIS_PER_SEC,
+                0
+            )) < core::ffi::c_uint::MAX
+        );
+        assert_eq!(
+            coarse_sleep_conversion(Duration::new(
+                core::ffi::c_uint::MAX as u64 / MILLIS_PER_SEC,
+                999_999_999
+            )),
+            core::ffi::c_uint::MAX
+        );
+
+        assert_eq!(
+            coarse_sleep_conversion(Duration::MAX),
+            core::ffi::c_uint::MAX
+        );
+    }
+}
rust/kernel/lib.rs +1 -0
index dd9c39071391..9dc9124a5bd7 100644
@@ -35,6 +35,7 @@
 mod allocator;
 mod build_assert;
 pub mod cred;
+pub mod delay;
 pub mod error;
 pub mod file;
 pub mod fs;
Signature · 1 line
2.40.1
#12 lore raw

Re: [PATCH 09/80] rust: kernel: backport the delay module from the rust branch

Quoted 2 lines · Ariel Miculas

If we end up adding it as it was in the rust branch, then this would need proper attribution (Cc'ing John).

We are also trying to have the original authors from the rust branch to submit the code they wrote, if possible / they have the time, so that they are involved in the discussion.

Cheers,
Miguel

#13 lore raw

PATCH 10/80 rust: kernel: add container_of macro

In reply to Ariel Miculas
---
 rust/kernel/lib.rs | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
rust/kernel/lib.rs +31 -0
index 9dc9124a5bd7..9f68f804074d 100644
@@ -99,6 +99,37 @@ impl ThisModule {
     }
 }
 
+/// Produces a pointer to an object from a pointer to one of its fields.
+///
+/// # Safety
+///
+/// Callers must ensure that the pointer to the field is in fact a pointer to the specified field,
+/// as opposed to a pointer to another object of the same type. If this condition is not met,
+/// any dereference of the resulting pointer is UB.
+///
+/// # Example
+///
+/// ```
+/// # use kernel::container_of;
+/// struct Test {
+///     a: u64,
+///     b: u32,
+/// }
+///
+/// let test = Test { a: 10, b: 20 };
+/// let b_ptr = &test.b;
+/// let test_alias = container_of!(b_ptr, Test, b);
+/// assert!(core::ptr::eq(&test, test_alias));
+/// ```
+#[macro_export]
+macro_rules! container_of {
+    ($ptr:expr, $type:ty, $($f:tt)*) => {{
+        let ptr = $ptr as *const _ as *const u8;
+        let offset = $crate::offset_of!($type, $($f)*);
+        ptr.wrapping_offset(-offset) as *const $type
+    }}
+}
+
 #[cfg(not(any(testlib, test)))]
 #[panic_handler]
 fn panic(info: &core::panic::PanicInfo<'_>) -> ! {
Signature · 1 line
2.40.1
#14 lore raw

PATCH 11/80 rust: kernel: add offset_of macro

In reply to Ariel Miculas
---
 rust/kernel/lib.rs | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
rust/kernel/lib.rs +32 -0
index 9f68f804074d..7dfebbe39430 100644
@@ -97,6 +97,38 @@ impl ThisModule {
     pub const unsafe fn from_ptr(ptr: *mut bindings::module) -> ThisModule {
         ThisModule(ptr)
     }
+
+}
+/// Calculates the offset of a field from the beginning of the struct it belongs to.
+///
+/// # Example
+///
+/// ```
+/// # use kernel::prelude::*;
+/// # use kernel::offset_of;
+/// struct Test {
+///     a: u64,
+///     b: u32,
+/// }
+///
+/// assert_eq!(offset_of!(Test, b), 8);
+/// ```
+#[macro_export]
+macro_rules! offset_of {
+    ($type:ty, $($f:tt)*) => {{
+        let tmp = core::mem::MaybeUninit::<$type>::uninit();
+        let outer = tmp.as_ptr();
+        // To avoid warnings when nesting `unsafe` blocks.
+        #[allow(unused_unsafe)]
+        // SAFETY: The pointer is valid and aligned, just not initialised; `addr_of` ensures that
+        // we don't actually read from `outer` (which would be UB) nor create an intermediate
+        // reference.
+        let inner = unsafe { core::ptr::addr_of!((*outer).$($f)*) } as *const u8;
+        // To avoid warnings when nesting `unsafe` blocks.
+        #[allow(unused_unsafe)]
+        // SAFETY: The two pointers are within the same allocation block.
+        unsafe { inner.offset_from(outer as *const u8) }
+    }}
 }
 
 /// Produces a pointer to an object from a pointer to one of its fields.
Signature · 1 line
2.40.1
#15 lore raw

PATCH 12/80 drop: Add crate::pr_warn declaration

In reply to Ariel Miculas
---
 rust/kernel/fs.rs | 1 -
 1 file changed, 1 deletion(-)
rust/kernel/fs.rs +0 -1
index 2a0267b3c0b6..1d920fde4022 100644
@@ -17,7 +17,6 @@
 use core::{
     cell::UnsafeCell, marker::PhantomData, marker::PhantomPinned, ops::Deref, pin::Pin, ptr,
 };
-
 use macros::vtable;
 
 pub mod param;
Signature · 1 line
2.40.1
#16 lore raw

Re: [PATCH 12/80] drop: Add crate::pr_warn declaration

Quoted 4 lines · Ariel Miculas
---
 rust/kernel/fs.rs | 1 -
 1 file changed, 1 deletion(-)

Empty patch, probably due to rebasing fun? :)

Cheers,
Miguel

#17 lore raw

Re: [PATCH 12/80] drop: Add crate::pr_warn declaration

Yes, I should have dropped this commit, that's why it says "drop" in the message.

Regards,
Ariel

From: Miguel Ojeda <[email protected]>
Sent: Friday, June 9, 2023 12:29 PM
To: Ariel Miculas (amiculas) <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: [PATCH 12/80] drop: Add crate::pr_warn declaration

Quoted 4 lines · Ariel Miculas

---
  rust/kernel/fs.rs | 1 -

  1 file changed, 1 deletion(-)

Empty patch, probably due to rebasing fun? :)

Cheers,
Miguel

#18 lore raw

PATCH 13/80 rust: kernel: rename from_kernel_errno to from_errno

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/file.rs +1 -1
index 1b5934838833..44086d398731 100644
@@ -214,7 +214,7 @@ pub fn new(flags: u32) -> Result<Self> {
         // SAFETY: FFI call, there are no safety requirements on `flags`.
         let fd = unsafe { bindings::get_unused_fd_flags(flags) };
         if fd < 0 {
-            return Err(Error::from_kernel_errno(fd));
+            return Err(Error::from_errno(fd));
         }
         Ok(Self { fd: fd as _ })
     }
Signature · 1 line
2.40.1
#20 lore raw

PATCH 14/80 rust: kernel: Rename from_pointer to from_foreing and into_pointer to into_foreign

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
rust/kernel/file.rs +12 -12
index 44086d398731..586c0b386b5b 100644
@@ -330,7 +330,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
             let fileref = unsafe { File::from_ptr(file) };
             // SAFETY: `arg` was previously returned by `A::convert` and must
             // be a valid non-null pointer.
-            let ptr = T::open(unsafe { &*arg }, fileref)?.into_pointer();
+            let ptr = T::open(unsafe { &*arg }, fileref)?.into_foreign();
             // SAFETY: The C contract guarantees that `private_data` is available
             // for implementers of the file operations (no other C code accesses
             // it), so we know that there are no concurrent threads/CPUs accessing
@@ -349,7 +349,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
         from_kernel_result! {
             let mut data = unsafe { UserSlicePtr::new(buf as *mut core::ffi::c_void, len).writer() };
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -376,7 +376,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
             let file = unsafe { (*iocb).ki_filp };
             let offset = unsafe { (*iocb).ki_pos };
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -397,7 +397,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
         from_kernel_result! {
             let mut data = unsafe { UserSlicePtr::new(buf as *mut core::ffi::c_void, len).reader() };
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -424,7 +424,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
             let file = unsafe { (*iocb).ki_filp };
             let offset = unsafe { (*iocb).ki_pos };
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -441,7 +441,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
         file: *mut bindings::file,
     ) -> core::ffi::c_int {
         let ptr = mem::replace(unsafe { &mut (*file).private_data }, ptr::null_mut());
-        T::release(unsafe { T::Data::from_pointer(ptr as _) }, unsafe {
+        T::release(unsafe { T::Data::from_foreign(ptr as _) }, unsafe {
             File::from_ptr(file)
         });
         0
@@ -460,7 +460,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
                 _ => return Err(EINVAL),
             };
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -477,7 +477,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
     ) -> core::ffi::c_long {
         from_kernel_result! {
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -495,7 +495,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
     ) -> core::ffi::c_long {
         from_kernel_result! {
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -512,7 +512,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
     ) -> core::ffi::c_int {
         from_kernel_result! {
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -540,7 +540,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
             let end = end.try_into()?;
             let datasync = datasync != 0;
             // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-            // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the
+            // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the
             // `release` callback, which the C API guarantees that will be called only when all
             // references to `file` have been released, so we know it can't be called while this
             // function is running.
@@ -555,7 +555,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
         wait: *mut bindings::poll_table_struct,
     ) -> bindings::__poll_t {
         // SAFETY: `private_data` was initialised by `open_callback` with a value returned by
-        // `T::Data::into_pointer`. `T::Data::from_pointer` is only called by the `release`
+        // `T::Data::into_foreign`. `T::Data::from_foreign` is only called by the `release`
         // callback, which the C API guarantees that will be called only when all references to
         // `file` have been released, so we know it can't be called while this function is running.
         let f = unsafe { T::Data::borrow((*file).private_data) };
Signature · 1 line
2.40.1
#21 lore raw

Re: [PATCH 14/80] rust: kernel: Rename from_pointer to from_foreing and into_pointer to into_foreign

Quoted 2 lines · Ariel Miculas

This one should be squashed into the patch that adds the code too.

Cheers,
Miguel

#22 lore raw

PATCH 15/80 rust: kernel: add count_paren_items macro, needed by define_fs_params macro

In reply to Ariel Miculas

Changes:
* add count_paren_items in rust/kernel/driver.rs, used by define_fs_params macro in rust/kernel/fs/param.rs
* add missing uring_cmd_iopoll in file_operations VTABLE
* fix accesses to the vm_flags field in vm_area_struct
* fix init's signature in rust/kernel/fs.rs
* import module_fs in samples/rust/rust_fs.rs

---
 rust/bindings/bindings_helper.h |  1 +
 rust/kernel/driver.rs           | 28 ++++++++++++++++++++++++++++
 rust/kernel/file.rs             |  1 +
 rust/kernel/fs.rs               |  2 +-
 rust/kernel/lib.rs              |  3 ++-
 rust/kernel/mm.rs               |  4 ++--
 samples/rust/rust_fs.rs         |  1 +
 7 files changed, 36 insertions(+), 4 deletions(-)
 create mode 100644 rust/kernel/driver.rs
rust/bindings/bindings_helper.h +1 -0
index d15a698439e1..6182174663ba 100644
@@ -16,6 +16,7 @@
 #include <linux/poll.h>
 #include <linux/uio.h>
 #include <linux/uaccess.h>
+#include <linux/delay.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
rust/kernel/driver.rs new file +28 -0
new file mode 100644
index 000000000000..f743fcf5451b
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
+//!
+//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
+//! using the [`Registration`] class.
+
+/// Counts the number of parenthesis-delimited, comma-separated items.
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::count_paren_items;
+///
+/// assert_eq!(0, count_paren_items!());
+/// assert_eq!(1, count_paren_items!((A)));
+/// assert_eq!(1, count_paren_items!((A),));
+/// assert_eq!(2, count_paren_items!((A), (B)));
+/// assert_eq!(2, count_paren_items!((A), (B),));
+/// assert_eq!(3, count_paren_items!((A), (B), (C)));
+/// assert_eq!(3, count_paren_items!((A), (B), (C),));
+/// ```
+#[macro_export]
+macro_rules! count_paren_items {
+    (($($item:tt)*), $($remaining:tt)*) => { 1 + $crate::count_paren_items!($($remaining)*) };
+    (($($item:tt)*)) => { 1 };
+    () => { 0 };
+}
rust/kernel/file.rs +1 -0
index 586c0b386b5b..b3bbbf153925 100644
@@ -637,6 +637,7 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
             None
         },
         uring_cmd: None,
+        uring_cmd_iopoll: None,
         write_iter: if T::HAS_WRITE {
             Some(Self::write_iter_callback)
         } else {
rust/kernel/fs.rs +1 -1
index 1d920fde4022..ba98ae7caf00 100644
@@ -1263,7 +1263,7 @@ pub struct Module<T: Type> {
 }
 
 impl<T: Type + Sync> crate::Module for Module<T> {
-    fn init(_name: &'static CStr, module: &'static ThisModule) -> Result<Self> {
+    fn init(module: &'static ThisModule) -> Result<Self> {
         let mut reg = Pin::from(Box::try_new(Registration::new())?);
         reg.as_mut().register::<T>(module)?;
         Ok(Self {
rust/kernel/lib.rs +2 -1
index 7dfebbe39430..08f67833afcf 100644
@@ -21,6 +21,7 @@
 #![feature(receiver_trait)]
 #![feature(unsize)]
 #![feature(const_mut_refs)]
+#![feature(duration_constants)]
 
 // Ensure conditional compilation based on the kernel configuration works;
 // otherwise we may silently break things like initcall handling.
@@ -36,6 +37,7 @@
 mod build_assert;
 pub mod cred;
 pub mod delay;
+pub mod driver;
 pub mod error;
 pub mod file;
 pub mod fs;
@@ -97,7 +99,6 @@ impl ThisModule {
     pub const unsafe fn from_ptr(ptr: *mut bindings::module) -> ThisModule {
         ThisModule(ptr)
     }
-
 }
 /// Calculates the offset of a field from the beginning of the struct it belongs to.
 ///
rust/kernel/mm.rs +2 -2
index 779359d0c5cf..d159c45157a9 100644
@@ -38,7 +38,7 @@ pub(crate) unsafe fn from_ptr(vma: *mut bindings::vm_area_struct) -> Self {
         /// The possible flags are a combination of the constants in [`flags`].
         pub fn flags(&self) -> usize {
             // SAFETY: `self.vma` is valid by the type invariants.
-            unsafe { (*self.vma).vm_flags as _ }
+            unsafe { (*self.vma).__bindgen_anon_1.vm_flags as _ }
         }
 
         /// Sets the flags associated with the virtual memory area.
@@ -46,7 +46,7 @@ pub fn flags(&self) -> usize {
         /// The possible flags are a combination of the constants in [`flags`].
         pub fn set_flags(&mut self, flags: usize) {
             // SAFETY: `self.vma` is valid by the type invariants.
-            unsafe { (*self.vma).vm_flags = flags as _ };
+            unsafe { (*self.vma).__bindgen_anon_1.vm_flags = flags as _ };
         }
 
         /// Returns the start address of the virtual memory area.
samples/rust/rust_fs.rs +1 -0
index 18fd4542863b..7527681ee024 100644
@@ -2,6 +2,7 @@
 
 //! Rust file system sample.
 
+use kernel::module_fs;
 use kernel::prelude::*;
 use kernel::{c_str, file, fs, io_buffer::IoBufferWriter};
 
Signature · 1 line
2.40.1
#23 lore raw

PATCH 16/80 rust: helpers: add missing rust helper 'alloc_pages'

In reply to Ariel Miculas
---
 rust/helpers.c | 6 ++++++
 1 file changed, 6 insertions(+)
rust/helpers.c +6 -0
index efbe9d917a57..00a4c90c15e3 100644
@@ -198,6 +198,12 @@ void *rust_helper_alloc_inode_sb(struct super_block *sb,
 }
 EXPORT_SYMBOL_GPL(rust_helper_alloc_inode_sb);
 
+struct page *rust_helper_alloc_pages(gfp_t gfp_mask, unsigned int order)
+{
+	return alloc_pages(gfp_mask, order);
+}
+EXPORT_SYMBOL_GPL(rust_helper_alloc_pages);
+
 /*
  * We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
  * as the Rust `usize` type, so we can use it in contexts where Rust
Signature · 1 line
2.40.1
#24 lore raw

Re: [PATCH 16/80] rust: helpers: add missing rust helper 'alloc_pages'

Quoted 2 lines · Ariel Miculas

I think this could be added to the patch where the user/caller is. Otherwise, the commit message should explain the need for it.

Cheers,
Miguel

#25 lore raw

PATCH 17/80 kernel: configs: add qemu-busybox-min.config

In reply to Ariel Miculas
---
 arch/x86/configs/qemu-busybox-min.config | 11 +++++
 kernel/configs/qemu-busybox-min.config   | 56 ++++++++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100644 arch/x86/configs/qemu-busybox-min.config
 create mode 100644 kernel/configs/qemu-busybox-min.config
arch/x86/configs/qemu-busybox-min.config new file +11 -0
new file mode 100644
index 000000000000..9a2bf2549053
@@ -0,0 +1,11 @@
+CONFIG_64BIT=y
+CONFIG_ACPI=y
+
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PVH=y
+
+CONFIG_CMDLINE_BOOL=y
+CONFIG_CMDLINE="console=ttyS0 nokaslr rdinit=/sbin/init"
kernel/configs/qemu-busybox-min.config new file +56 -0

56 lines not on this page · raw message

Signature · 1 line
2.40.1
#27 lore raw

PATCH 18/80 rust: kernel: format the rust code

In reply to Ariel Miculas
---
 rust/kernel/file.rs     | 3 ++-
 rust/kernel/fs/param.rs | 2 +-
 rust/kernel/iov_iter.rs | 2 +-
 rust/kernel/mm.rs       | 2 +-
 rust/kernel/pages.rs    | 4 ++--
 rust/kernel/user_ptr.rs | 2 +-
 6 files changed, 8 insertions(+), 7 deletions(-)
rust/kernel/file.rs +2 -1
index b3bbbf153925..c090a19da9b2 100644
@@ -14,9 +14,10 @@
     iov_iter::IovIter,
     mm,
     sync::CondVar,
+    types::ARef,
+    types::AlwaysRefCounted,
     types::ForeignOwnable,
     user_ptr::{UserSlicePtr, UserSlicePtrReader, UserSlicePtrWriter},
-    types::ARef, types::AlwaysRefCounted,
 };
 use core::convert::{TryFrom, TryInto};
 use core::{cell::UnsafeCell, marker, mem, ptr};
rust/kernel/fs/param.rs +1 -1
index 1a31130c6d1e..947acf0d217c 100644
@@ -4,7 +4,7 @@
 //!
 //! C headers: [`include/linux/fs_parser.h`](../../../../../include/linux/fs_parser.h)
 
-use crate::{bindings, file, fs, str::CStr, error::Result};
+use crate::{bindings, error::Result, file, fs, str::CStr};
 use core::{marker::PhantomData, ptr};
 
 /// The value of a file system parameter.
rust/kernel/iov_iter.rs +1 -1
index 01c7fa065dba..4602cbd4b49c 100644
@@ -7,8 +7,8 @@
 use crate::{
     bindings,
     error::code::*,
-    io_buffer::{IoBufferReader, IoBufferWriter},
     error::Result,
+    io_buffer::{IoBufferReader, IoBufferWriter},
 };
 
 /// Wraps the kernel's `struct iov_iter`.
rust/kernel/mm.rs +1 -1
index d159c45157a9..72b504c949c1 100644
@@ -4,7 +4,7 @@
 //!
 //! C header: [`include/linux/mm.h`](../../../../include/linux/mm.h)
 
-use crate::{bindings, pages, error::to_result, error::Result};
+use crate::{bindings, error::to_result, error::Result, pages};
 
 /// Virtual memory.
 pub mod virt {
rust/kernel/pages.rs +2 -2
index c0c1f9fe03fc..c5630c29cec8 100644
@@ -5,8 +5,8 @@
 //! TODO: This module is a work in progress.
 
 use crate::{
-    bindings, error::code::*, io_buffer::IoBufferReader, user_ptr::UserSlicePtrReader, error::Result,
-    PAGE_SIZE,
+    bindings, error::code::*, error::Result, io_buffer::IoBufferReader,
+    user_ptr::UserSlicePtrReader, PAGE_SIZE,
 };
 use core::{marker::PhantomData, ptr};
 
rust/kernel/user_ptr.rs +1 -1
index 9fdacc2826ef..ba00147697ef 100644
@@ -7,8 +7,8 @@
 use crate::{
     bindings,
     error::code::*,
-    io_buffer::{IoBufferReader, IoBufferWriter},
     error::Result,
+    io_buffer::{IoBufferReader, IoBufferWriter},
 };
 use alloc::vec::Vec;
 
Signature · 1 line
2.40.1
#29 lore raw

PATCH 19/80 samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs

In reply to Ariel Miculas
---
 samples/rust/Kconfig     |  10 ++++
 samples/rust/Makefile    |   1 +
 samples/rust/puzzlefs.rs | 105 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 116 insertions(+)
 create mode 100644 samples/rust/puzzlefs.rs
samples/rust/Kconfig +10 -0
index 2bd736f99189..05ca21fbba06 100644
@@ -40,6 +40,16 @@ config SAMPLE_RUST_FS
 
 	  If unsure, say N.
 
+config SAMPLE_PUZZLEFS
+	tristate "Puzzlefs file system"
+	help
+	  This option builds the Rust puzzlefs file system sample.
+
+	  To compile this as a module, choose M here:
+	  the module will be called puzzlefs.
+
+	  If unsure, say N.
+
 config SAMPLE_RUST_HOSTPROGS
 	bool "Host programs"
 	help
samples/rust/Makefile +1 -0
index e5941037e673..364a38dbf90b 100644
@@ -3,5 +3,6 @@
 obj-$(CONFIG_SAMPLE_RUST_MINIMAL)		+= rust_minimal.o
 obj-$(CONFIG_SAMPLE_RUST_PRINT)			+= rust_print.o
 obj-$(CONFIG_SAMPLE_RUST_FS)			+= rust_fs.o
+obj-$(CONFIG_SAMPLE_PUZZLEFS)			+= puzzlefs.o
 
 subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS)		+= hostprogs
samples/rust/puzzlefs.rs new file +105 -0

105 lines not on this page · raw message

Signature · 1 line
2.40.1
#30 lore raw

PATCH 20/80 kernel: configs: enable rust samples in rust.config

In reply to Ariel Miculas
---
 kernel/configs/rust.config | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/configs/rust.config +10 -0
index 38a7c5362c9c..422b49798d92 100644
@@ -1 +1,11 @@
 CONFIG_RUST=y
+
+CONFIG_MODULES=y
+CONFIG_SAMPLES=y
+CONFIG_SAMPLES_RUST=y
+CONFIG_SAMPLE_RUST_MINIMAL=m
+CONFIG_SAMPLE_RUST_PRINT=m
+CONFIG_SAMPLE_RUST_FS=m
+CONFIG_SAMPLE_PUZZLEFS=m
+CONFIG_SAMPLE_RUST_HOSTPROGS=y
+
Signature · 1 line
2.40.1
#31 lore raw

Re: [PATCH 20/80] kernel: configs: enable rust samples in rust.config

Quoted 2 lines · Ariel Miculas

I think the intention of rust.config is to enable the Rust support, not everything related to Rust.

So it would be better to have another file to enable samples, or one to enable as much as possible (e.g. for testing/CI), etc.

Cheers,
Miguel

#32 lore raw

PATCH 22/80 rust: proc-macro2: add SPDX License Identifiers

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/proc-macro2/detection.rs | 2 ++
 rust/proc-macro2/fallback.rs  | 2 ++
 rust/proc-macro2/lib.rs       | 2 ++
 rust/proc-macro2/marker.rs    | 2 ++
 rust/proc-macro2/parse.rs     | 2 ++
 rust/proc-macro2/rcvec.rs     | 2 ++
 rust/proc-macro2/wrapper.rs   | 2 ++
 7 files changed, 14 insertions(+)
rust/proc-macro2/detection.rs +2 -0
index beba7b237395..3de448cb2dde 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use core::sync::atomic::{AtomicUsize, Ordering};
 use std::sync::Once;
 
rust/proc-macro2/fallback.rs +2 -0
index fe4f248d3d21..f795c7618118 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::parse::{self, Cursor};
 use crate::rcvec::{RcVec, RcVecBuilder, RcVecIntoIter, RcVecMut};
 use crate::{Delimiter, Spacing, TokenTree};
rust/proc-macro2/lib.rs +2 -0
index 47b48df22272..d3b0454a9c0f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! [![github]](https://github.com/dtolnay/proc-macro2)&ensp;[![crates-io]](https://crates.io/crates/proc-macro2)&ensp;[![docs-rs]](crate)
 //!
 //! [github]: https://img.shields.io/badge/github-8da0cb?style=for-the-badge&labelColor=555555&logo=github
rust/proc-macro2/marker.rs +2 -0
index 59fd0963056f..eea17649a9b9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use core::marker::PhantomData;
 use std::panic::{RefUnwindSafe, UnwindSafe};
 use std::rc::Rc;
rust/proc-macro2/parse.rs +2 -0
index 04c4833694dc..d1721a207da4 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::fallback::{
     is_ident_continue, is_ident_start, Group, LexError, Literal, Span, TokenStream,
     TokenStreamBuilder,
rust/proc-macro2/rcvec.rs +2 -0
index 86ca7d8089ac..da2398c789d9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use core::mem;
 use core::slice;
 use std::rc::Rc;
rust/proc-macro2/wrapper.rs +2 -0
index 47d149473244..dacb2e59bd04 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::detection::inside_proc_macro;
 use crate::{fallback, Delimiter, Punct, Spacing, TokenTree};
 use core::fmt::{self, Debug, Display};
Signature · 1 line
2.40.1
#33 lore raw

PATCH 23/80 rust: proc-macro2: remove `unicode_ident` dependency

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

The proc-macro2 crate depends on the unicode-ident crate to determine whether characters have the XID_Start or XID_Continue properties according to Unicode Standard Annex #31.

However, we only need ASCII identifiers in the kernel, thus we can simplify the check and remove completely that dependency.

---
 rust/proc-macro2/fallback.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
rust/proc-macro2/fallback.rs +2 -2
index f795c7618118..0d576c32162d 100644
@@ -724,11 +724,11 @@ pub fn set_span(&mut self, span: Span) {
 }
 
 pub(crate) fn is_ident_start(c: char) -> bool {
-    c == '_' || unicode_ident::is_xid_start(c)
+    c == '_' || c.is_ascii_alphabetic()
 }
 
 pub(crate) fn is_ident_continue(c: char) -> bool {
-    unicode_ident::is_xid_continue(c)
+    c == '_' || c.is_ascii_alphanumeric()
 }
 
 fn validate_ident(string: &str, raw: bool) {
Signature · 1 line
2.40.1
#34 lore raw

PATCH 24/80 rust: quote: import crate

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

This is a subset of the Rust quote crate, version 1.0.21, licensed under "Apache-2.0 OR MIT", from:

    https://github.com/dtolnay/quote/raw/1.0.21/src

The files are copied as-is, with no modifications whatsoever (not even adding the SPDX identifiers).

For copyright details, please see:

    https://github.com/dtolnay/quote/blob/1.0.21/README.md#license
    https://github.com/dtolnay/quote/blob/1.0.21/LICENSE-APACHE
    https://github.com/dtolnay/quote/blob/1.0.21/LICENSE-MIT

The next patch modifies these files as needed for use within the kernel. This patch split allows reviewers to double-check the import and to clearly see the differences introduced.

The following script may be used to verify the contents:

    for path in $(cd rust/quote/ && find . -type f -name '*.rs'); do
        curl --silent --show-error --location \
            https://github.com/dtolnay/quote/raw/1.0.21/src/$path \
            | diff --unified rust/quote/$path - && echo $path: OK
    done
---
 rust/quote/ext.rs            |  110 +++
 rust/quote/format.rs         |  168 ++++
 rust/quote/ident_fragment.rs |   86 ++
 rust/quote/lib.rs            | 1434 ++++++++++++++++++++++++++++++++++
 rust/quote/runtime.rs        |  438 +++++++++++
 rust/quote/spanned.rs        |   43 +
 rust/quote/to_tokens.rs      |  209 +++++
 7 files changed, 2488 insertions(+)
 create mode 100644 rust/quote/ext.rs
 create mode 100644 rust/quote/format.rs
 create mode 100644 rust/quote/ident_fragment.rs
 create mode 100644 rust/quote/lib.rs
 create mode 100644 rust/quote/runtime.rs
 create mode 100644 rust/quote/spanned.rs
 create mode 100644 rust/quote/to_tokens.rs
rust/quote/ext.rs new file +110 -0
new file mode 100644
index 000000000000..92c2315b182d
@@ -0,0 +1,110 @@
+use super::ToTokens;
+use core::iter;
+use proc_macro2::{TokenStream, TokenTree};
+
+/// TokenStream extension trait with methods for appending tokens.
+///
+/// This trait is sealed and cannot be implemented outside of the `quote` crate.
+pub trait TokenStreamExt: private::Sealed {
+    /// For use by `ToTokens` implementations.
+    ///
+    /// Appends the token specified to this list of tokens.
+    fn append<U>(&mut self, token: U)
+    where
+        U: Into<TokenTree>;
+
+    /// For use by `ToTokens` implementations.
+    ///
+    /// ```
+    /// # use quote::{quote, TokenStreamExt, ToTokens};
+    /// # use proc_macro2::TokenStream;
+    /// #
+    /// struct X;
+    ///
+    /// impl ToTokens for X {
+    ///     fn to_tokens(&self, tokens: &mut TokenStream) {
+    ///         tokens.append_all(&[true, false]);
+    ///     }
+    /// }
+    ///
+    /// let tokens = quote!(#X);
+    /// assert_eq!(tokens.to_string(), "true false");
+    /// ```
+    fn append_all<I>(&mut self, iter: I)
+    where
+        I: IntoIterator,
+        I::Item: ToTokens;
+
+    /// For use by `ToTokens` implementations.
+    ///
+    /// Appends all of the items in the iterator `I`, separated by the tokens
+    /// `U`.
+    fn append_separated<I, U>(&mut self, iter: I, op: U)
+    where
+        I: IntoIterator,
+        I::Item: ToTokens,
+        U: ToTokens;
+
+    /// For use by `ToTokens` implementations.
+    ///
+    /// Appends all tokens in the iterator `I`, appending `U` after each
+    /// element, including after the last element of the iterator.
+    fn append_terminated<I, U>(&mut self, iter: I, term: U)
+    where
+        I: IntoIterator,
+        I::Item: ToTokens,
+        U: ToTokens;
+}
+
+impl TokenStreamExt for TokenStream {
+    fn append<U>(&mut self, token: U)
+    where
+        U: Into<TokenTree>,
+    {
+        self.extend(iter::once(token.into()));
+    }
+
+    fn append_all<I>(&mut self, iter: I)
+    where
+        I: IntoIterator,
+        I::Item: ToTokens,
+    {
+        for token in iter {
+            token.to_tokens(self);
+        }
+    }
+
+    fn append_separated<I, U>(&mut self, iter: I, op: U)
+    where
+        I: IntoIterator,
+        I::Item: ToTokens,
+        U: ToTokens,
+    {
+        for (i, token) in iter.into_iter().enumerate() {
+            if i > 0 {
+                op.to_tokens(self);
+            }
+            token.to_tokens(self);
+        }
+    }
+
+    fn append_terminated<I, U>(&mut self, iter: I, term: U)
+    where
+        I: IntoIterator,
+        I::Item: ToTokens,
+        U: ToTokens,
+    {
+        for token in iter {
+            token.to_tokens(self);
+            term.to_tokens(self);
+        }
+    }
+}
+
+mod private {
+    use proc_macro2::TokenStream;
+
+    pub trait Sealed {}
+
+    impl Sealed for TokenStream {}
+}
rust/quote/format.rs new file +168 -0

168 lines not on this page · raw message

rust/quote/ident_fragment.rs new file +86 -0

86 lines not on this page · raw message

rust/quote/lib.rs new file +1434 -0

1,434 lines not on this page · raw message

rust/quote/runtime.rs new file +438 -0

438 lines not on this page · raw message

rust/quote/spanned.rs new file +43 -0

43 lines not on this page · raw message

rust/quote/to_tokens.rs new file +209 -0

209 lines not on this page · raw message

Signature · 1 line
2.40.1
#35 lore raw

PATCH 25/80 rust: quote: add SPDX License Identifiers

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/quote/ext.rs            | 2 ++
 rust/quote/format.rs         | 2 ++
 rust/quote/ident_fragment.rs | 2 ++
 rust/quote/lib.rs            | 2 ++
 rust/quote/runtime.rs        | 2 ++
 rust/quote/spanned.rs        | 2 ++
 rust/quote/to_tokens.rs      | 2 ++
 7 files changed, 14 insertions(+)
rust/quote/ext.rs +2 -0
index 92c2315b182d..977d2f0c5919 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::ToTokens;
 use core::iter;
 use proc_macro2::{TokenStream, TokenTree};
rust/quote/format.rs +2 -0
index 3cddbd2819d6..9fbf9c5949a9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 /// Formatting macro for constructing `Ident`s.
 ///
 /// <br>
rust/quote/ident_fragment.rs +2 -0
index cf74024b4825..e4a8d6cd30e9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use core::fmt;
 use proc_macro2::{Ident, Span};
 use std::borrow::Cow;
rust/quote/lib.rs +2 -0
index 35594827f869..ddab15fe4ebf 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! [![github]](https://github.com/dtolnay/quote)&ensp;[![crates-io]](https://crates.io/crates/quote)&ensp;[![docs-rs]](https://docs.rs/quote)
 //!
 //! [github]: https://img.shields.io/badge/github-8da0cb?style=for-the-badge&labelColor=555555&logo=github
rust/quote/runtime.rs +2 -0
index f3cdded3afb2..966025bc44ab 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::{IdentFragment, ToTokens, TokenStreamExt};
 use core::fmt;
 use core::iter;
rust/quote/spanned.rs +2 -0
index f64c8f5dfec7..aac63de0ba2f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::ToTokens;
 use proc_macro2::{Span, TokenStream};
 
rust/quote/to_tokens.rs +2 -0
index 57487217eeae..a91ba61573e0 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::TokenStreamExt;
 use core::iter;
 use proc_macro2::{Group, Ident, Literal, Punct, Span, TokenStream, TokenTree};
Signature · 1 line
2.40.1
#36 lore raw

PATCH 27/80 rust: syn: add SPDX License Identifiers

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/syn/attr.rs               | 2 ++
 rust/syn/await.rs              | 2 ++
 rust/syn/bigint.rs             | 2 ++
 rust/syn/buffer.rs             | 2 ++
 rust/syn/custom_keyword.rs     | 2 ++
 rust/syn/custom_punctuation.rs | 2 ++
 rust/syn/data.rs               | 2 ++
 rust/syn/derive.rs             | 2 ++
 rust/syn/discouraged.rs        | 2 ++
 rust/syn/error.rs              | 2 ++
 rust/syn/export.rs             | 2 ++
 rust/syn/expr.rs               | 2 ++
 rust/syn/ext.rs                | 2 ++
 rust/syn/file.rs               | 2 ++
 rust/syn/gen/clone.rs          | 2 ++
 rust/syn/gen/debug.rs          | 2 ++
 rust/syn/gen/eq.rs             | 2 ++
 rust/syn/gen/fold.rs           | 2 ++
 rust/syn/gen/hash.rs           | 2 ++
 rust/syn/gen/visit.rs          | 2 ++
 rust/syn/gen/visit_mut.rs      | 2 ++
 rust/syn/gen_helper.rs         | 2 ++
 rust/syn/generics.rs           | 2 ++
 rust/syn/group.rs              | 2 ++
 rust/syn/ident.rs              | 2 ++
 rust/syn/item.rs               | 2 ++
 rust/syn/lib.rs                | 2 ++
 rust/syn/lifetime.rs           | 2 ++
 rust/syn/lit.rs                | 2 ++
 rust/syn/lookahead.rs          | 2 ++
 rust/syn/mac.rs                | 2 ++
 rust/syn/macros.rs             | 2 ++
 rust/syn/op.rs                 | 2 ++
 rust/syn/parse.rs              | 2 ++
 rust/syn/parse_macro_input.rs  | 2 ++
 rust/syn/parse_quote.rs        | 2 ++
 rust/syn/pat.rs                | 2 ++
 rust/syn/path.rs               | 2 ++
 rust/syn/print.rs              | 2 ++
 rust/syn/punctuated.rs         | 2 ++
 rust/syn/reserved.rs           | 2 ++
 rust/syn/sealed.rs             | 2 ++
 rust/syn/span.rs               | 2 ++
 rust/syn/spanned.rs            | 2 ++
 rust/syn/stmt.rs               | 2 ++
 rust/syn/thread.rs             | 2 ++
 rust/syn/token.rs              | 2 ++
 rust/syn/tt.rs                 | 2 ++
 rust/syn/ty.rs                 | 2 ++
 rust/syn/verbatim.rs           | 2 ++
 rust/syn/whitespace.rs         | 2 ++
 51 files changed, 102 insertions(+)
rust/syn/attr.rs +2 -0
index bace94f43c85..cbcd158f45f6 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 use proc_macro2::TokenStream;
rust/syn/await.rs +2 -0
index 038c6a5d12cd..1aae244cafd3 100644
@@ -1,2 +1,4 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // See include!("await.rs") in token.rs.
 export_token_macro! {[await]}
rust/syn/bigint.rs +2 -0
index 5397d6beee1d..d8f7b8ca6863 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use std::ops::{AddAssign, MulAssign};
 
 // For implementing base10_digits() accessor on LitInt.
rust/syn/buffer.rs +2 -0
index 0d5cf30d5794..9a600f527f7a 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! A stably addressed token buffer supporting efficient traversal based on a
 //! cheaply copyable cursor.
 //!
rust/syn/custom_keyword.rs +2 -0
index a3ec9d4cb701..916f4c1bb820 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 /// Define a type that supports parsing and printing a given identifier as if it
 /// were a keyword.
 ///
rust/syn/custom_punctuation.rs +2 -0
index 118a8453daab..5cb8281623f8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 /// Define a type that supports parsing and printing a multi-character symbol
 /// as if it were a punctuation token.
 ///
rust/syn/data.rs +2 -0
index 3b466618f8ab..e4002648c635 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 
rust/syn/derive.rs +2 -0
index af9bb91b7a8d..455e72938067 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 
rust/syn/discouraged.rs +2 -0
index a46129b6a159..9843c82d54d2 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Extensions to the parsing API with niche applicability.
 
 use super::*;
rust/syn/error.rs +2 -0
index e301367d5e4e..262f3ab8cab7 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "parsing")]
 use crate::buffer::Cursor;
 use crate::thread::ThreadBound;
rust/syn/export.rs +2 -0
index f478d091ea12..5ea4dd75eb4a 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 pub use std::clone::Clone;
 pub use std::cmp::{Eq, PartialEq};
 pub use std::default::Default;
rust/syn/expr.rs +2 -0
index 93a59b0e20d7..54155892fe04 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 #[cfg(feature = "full")]
rust/syn/ext.rs +2 -0
index 98d5550f4836..c30a89902f7f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Extension traits to provide parsing methods on foreign types.
 //!
 //! *This module is available only if Syn is built with the `"parsing"` feature.*
rust/syn/file.rs +2 -0
index 280484f980d1..39796e1e3862 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 
 ast_struct! {
rust/syn/gen/clone.rs +2 -0
index a413e3ec700c..82a3f51d8c52 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen/debug.rs +2 -0
index a1f0afa790c0..488433cd8f01 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen/eq.rs +2 -0
index 20acb809d858..4af1c9c6a3d1 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen/fold.rs +2 -0
index 98bb5794aab9..d503f617d631 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen/hash.rs +2 -0
index d0400e19d6d8..8fbc237079ef 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen/visit.rs +2 -0
index 19ddd2e72602..0285e4dbee23 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen/visit_mut.rs +2 -0
index 239709d19410..4a46346c1807 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // This file is @generated by syn-internal-codegen.
 // It is not intended for manual editing.
 
rust/syn/gen_helper.rs +2 -0
index b2796126a339..3850100b247f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "fold")]
 pub mod fold {
     use crate::fold::Fold;
rust/syn/generics.rs +2 -0
index 9c2802f87b8e..de8a4b2ce1d5 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::{Iter, IterMut, Punctuated};
 #[cfg(all(feature = "printing", feature = "extra-traits"))]
rust/syn/group.rs +2 -0
index 7fd273c1de00..b02e79db2c20 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::error::Result;
 use crate::parse::ParseBuffer;
 use crate::token;
rust/syn/ident.rs +2 -0
index 8e3d8bda9746..6292a05f59f7 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "parsing")]
 use crate::buffer::Cursor;
 #[cfg(feature = "parsing")]
rust/syn/item.rs +2 -0
index a1ef7ab43eff..b256d94a25e2 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::derive::{Data, DataEnum, DataStruct, DataUnion, DeriveInput};
 use crate::punctuated::Punctuated;
rust/syn/lib.rs +2 -0
index 7d4df7110c6b..c32d6b3cb8cb 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! [![github]](https://github.com/dtolnay/syn)&ensp;[![crates-io]](https://crates.io/crates/syn)&ensp;[![docs-rs]](crate)
 //!
 //! [github]: https://img.shields.io/badge/github-8da0cb?style=for-the-badge&labelColor=555555&logo=github
rust/syn/lifetime.rs +2 -0
index 5dc1753a84fa..23b644e8ea72 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Ident, Span};
 use std::cmp::Ordering;
 use std::fmt::{self, Display};
rust/syn/lit.rs +2 -0
index 130b40ed1fb7..51229eb165f1 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "parsing")]
 use crate::lookahead;
 #[cfg(feature = "parsing")]
rust/syn/lookahead.rs +2 -0
index 0cf5cf5a83be..118fdbbe11ff 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::buffer::Cursor;
 use crate::error::{self, Error};
 use crate::sealed::lookahead::Sealed;
rust/syn/mac.rs +2 -0
index 3d84f8e48bf1..99dd6accd95d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::token::{Brace, Bracket, Paren};
 use proc_macro2::TokenStream;
rust/syn/macros.rs +2 -0
index da10a1a55057..86e64dc38992 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg_attr(
     not(any(feature = "full", feature = "derive")),
     allow(unknown_lints, unused_macro_rules)
rust/syn/op.rs +2 -0
index b8ef9a7fbd08..96ccd7d3e1f3 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 ast_enum! {
     /// A binary operator: `+`, `+=`, `&`.
     ///
rust/syn/parse.rs +2 -0
index bac4ca05abf9..aa07edc885dc 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Parsing interface for parsing a token stream into a syntax tree node.
 //!
 //! Parsing in Syn is built on parser functions that take in a [`ParseStream`]
rust/syn/parse_macro_input.rs +2 -0
index 6163cd70af7b..8c68fdda2f71 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 /// Parse the input TokenStream of a macro, triggering a compile error if the
 /// tokens fail to parse.
 ///
rust/syn/parse_quote.rs +2 -0
index 8deae4e52652..4eb592d099f6 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 /// Quasi-quotation macro that accepts input like the [`quote!`] macro but uses
 /// type inference to figure out a return type for those tokens.
 ///
rust/syn/pat.rs +2 -0
index b279186aa063..a2c2004e2b64 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 use proc_macro2::TokenStream;
rust/syn/path.rs +2 -0
index 6cdb43ac5416..b69c9ceb11d9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 
rust/syn/print.rs +2 -0
index da4e07e3b3cb..abc287ee9501 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::TokenStream;
 use quote::ToTokens;
 
rust/syn/punctuated.rs +2 -0
index 0fe1078cf404..b1ad84b301fa 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! A punctuated sequence of syntax tree nodes separated by punctuation.
 //!
 //! Lots of things in Rust are punctuated sequences.
rust/syn/reserved.rs +2 -0
index abfdf43a91e0..f6bb508a6907 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // Type for a syntax tree node that is reserved for future use.
 //
 // For example ExprReference contains a field `raw` of type Reserved. If `&raw
rust/syn/sealed.rs +2 -0
index 0b11bc99a1e1..02abcb1cd90d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "parsing")]
 pub mod lookahead {
     pub trait Sealed: Copy {}
rust/syn/span.rs +2 -0
index 27a7fe846d39..0574c49d7835 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::Span;
 
 pub trait IntoSpans<S> {
rust/syn/spanned.rs +2 -0
index d51ffb3fa5f5..ee65ea85841a 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! A trait that can provide the `Span` of the complete contents of a syntax
 //! tree node.
 //!
rust/syn/stmt.rs +2 -0
index 58bd013ecf4c..41b9378537ba 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 
 ast_struct! {
rust/syn/thread.rs +2 -0
index 9e5d8ad85efc..d535c913a9aa 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use std::fmt::{self, Debug};
 use std::thread::{self, ThreadId};
 
rust/syn/token.rs +2 -0
index 2208b07d0454..b9984c5782fb 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Tokens representing Rust punctuation, keywords, and delimiters.
 //!
 //! The type names in this module can be difficult to keep straight, so we
rust/syn/tt.rs +2 -0
index d87c0ed4dc1c..b4a9a3876f7d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Delimiter, TokenStream, TokenTree};
 use std::hash::{Hash, Hasher};
 
rust/syn/ty.rs +2 -0
index 4068be3c754b..ecdf93b952a5 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use super::*;
 use crate::punctuated::Punctuated;
 use proc_macro2::TokenStream;
rust/syn/verbatim.rs +2 -0
index 58cf68d17af8..15e651164a02 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use crate::parse::{ParseBuffer, ParseStream};
 use proc_macro2::{Delimiter, TokenStream};
 use std::cmp::Ordering;
rust/syn/whitespace.rs +2 -0
index 7be082e1a26d..99c2d3ab28f9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 pub fn skip(mut s: &str) -> &str {
     'skip: while !s.is_empty() {
         let byte = s.as_bytes()[0];
Signature · 1 line
2.40.1
#37 lore raw

PATCH 28/80 rust: syn: remove `unicode-ident` dependency

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

The syn crate depends on the unicode-ident crate
to determine whether characters have the XID_Start or XID_Continue properties according to Unicode Standard Annex #31.

However, we only need ASCII identifiers in the kernel, thus we can simplify the check and remove completely that dependency.

---
 rust/syn/ident.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
rust/syn/ident.rs +2 -2
index 6292a05f59f7..9ae2f0cbd83e 100644
@@ -91,11 +91,11 @@ fn from(token: Token![_]) -> Ident {
 pub fn xid_ok(symbol: &str) -> bool {
     let mut chars = symbol.chars();
     let first = chars.next().unwrap();
-    if !(first == '_' || unicode_ident::is_xid_start(first)) {
+    if !(first == '_' || first.is_ascii_alphabetic()) {
         return false;
     }
     for ch in chars {
-        if !unicode_ident::is_xid_continue(ch) {
+        if !(ch == '_' || ch.is_ascii_alphanumeric()) {
             return false;
         }
     }
Signature · 1 line
2.40.1
#38 lore raw

PATCH 30/80 rust: serde: add `no_fp_fmt_parse` support

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

We do not have formatting for floating point in the kernel, thus simple compile out all that.

Probably we should name it differently, more similar to integer128.

---
 rust/serde/de/ignored_any.rs |  1 +
 rust/serde/de/impls.rs       |  2 ++
 rust/serde/de/mod.rs         | 16 ++++++++++++++--
 rust/serde/de/value.rs       |  2 ++
 rust/serde/private/ser.rs    |  4 ++++
 rust/serde/ser/fmt.rs        |  8 ++++++--
 rust/serde/ser/impls.rs      |  2 ++
 rust/serde/ser/mod.rs        |  2 ++
 8 files changed, 33 insertions(+), 4 deletions(-)
rust/serde/de/ignored_any.rs +1 -0
index 9ed438e79795..7b65792e77c0 100644
@@ -152,6 +152,7 @@ fn visit_u128<E>(self, x: u128) -> Result<Self::Value, E> {
         }
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
     #[inline]
     fn visit_f64<E>(self, x: f64) -> Result<Self::Value, E> {
         let _ = x;
rust/serde/de/impls.rs +2 -0
index a2e2c4856df3..002bbf74d65b 100644
@@ -350,6 +350,7 @@ fn $visit<E>(self, v: $ty) -> Result<Self::Value, E>
     uint_to_self!(u32:visit_u32 u64:visit_u64);
 }
 
+#[cfg(not(no_fp_fmt_parse))]
 impl_deserialize_num! {
     f32, deserialize_f32
     num_self!(f32:visit_f32);
@@ -358,6 +359,7 @@ fn $visit<E>(self, v: $ty) -> Result<Self::Value, E>
     num_as_self!(u8:visit_u8 u16:visit_u16 u32:visit_u32 u64:visit_u64);
 }
 
+#[cfg(not(no_fp_fmt_parse))]
 impl_deserialize_num! {
     f64, deserialize_f64
     num_self!(f64:visit_f64);
rust/serde/de/mod.rs +14 -2
index ca29ec610bfb..017964f0d0d0 100644
@@ -344,6 +344,7 @@ pub enum Unexpected<'a> {
 
     /// The input contained a floating point `f32` or `f64` that was not
     /// expected.
+    #[cfg(not(no_fp_fmt_parse))]
     Float(f64),
 
     /// The input contained a `char` that was not expected.
@@ -400,6 +401,7 @@ fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
             Bool(b) => write!(formatter, "boolean `{}`", b),
             Unsigned(i) => write!(formatter, "integer `{}`", i),
             Signed(i) => write!(formatter, "integer `{}`", i),
+            #[cfg(not(no_fp_fmt_parse))]
             Float(f) => write!(formatter, "floating point `{}`", f),
             Char(c) => write!(formatter, "character `{}`", c),
             Str(s) => write!(formatter, "string {:?}", s),
@@ -996,12 +998,20 @@ fn deserialize_u128<V>(self, visitor: V) -> Result<V::Value, Self::Error>
     /// Hint that the `Deserialize` type is expecting a `f32` value.
     fn deserialize_f32<V>(self, visitor: V) -> Result<V::Value, Self::Error>
     where
-        V: Visitor<'de>;
+        V: Visitor<'de>
+    {
+        let _ = visitor;
+        Err(Error::custom("f32 is not supported"))
+    }
 
     /// Hint that the `Deserialize` type is expecting a `f64` value.
     fn deserialize_f64<V>(self, visitor: V) -> Result<V::Value, Self::Error>
     where
-        V: Visitor<'de>;
+        V: Visitor<'de>
+    {
+        let _ = visitor;
+        Err(Error::custom("f64 is not supported"))
+    }
 
     /// Hint that the `Deserialize` type is expecting a `char` value.
     fn deserialize_char<V>(self, visitor: V) -> Result<V::Value, Self::Error>
@@ -1446,6 +1456,7 @@ fn visit_u128<E>(self, v: u128) -> Result<Self::Value, E>
     /// The default implementation forwards to [`visit_f64`].
     ///
     /// [`visit_f64`]: #method.visit_f64
+    #[cfg(not(no_fp_fmt_parse))]
     fn visit_f32<E>(self, v: f32) -> Result<Self::Value, E>
     where
         E: Error,
@@ -1456,6 +1467,7 @@ fn visit_f32<E>(self, v: f32) -> Result<Self::Value, E>
     /// The input contains an `f64`.
     ///
     /// The default implementation fails with a type error.
+    #[cfg(not(no_fp_fmt_parse))]
     fn visit_f64<E>(self, v: f64) -> Result<Self::Value, E>
     where
         E: Error,
rust/serde/de/value.rs +2 -0
index 5d88862159b8..d847b8ec071c 100644
@@ -298,7 +298,9 @@ fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
 primitive_deserializer!(u16, "a `u16`.", U16Deserializer, visit_u16);
 primitive_deserializer!(u64, "a `u64`.", U64Deserializer, visit_u64);
 primitive_deserializer!(usize, "a `usize`.", UsizeDeserializer, visit_u64 as u64);
+#[cfg(not(no_fp_fmt_parse))]
 primitive_deserializer!(f32, "an `f32`.", F32Deserializer, visit_f32);
+#[cfg(not(no_fp_fmt_parse))]
 primitive_deserializer!(f64, "an `f64`.", F64Deserializer, visit_f64);
 primitive_deserializer!(char, "a `char`.", CharDeserializer, visit_char);
 
rust/serde/private/ser.rs +4 -0
index 528e8c125a39..fc9e7e6d7426 100644
@@ -46,6 +46,7 @@ struct TaggedSerializer<S> {
 enum Unsupported {
     Boolean,
     Integer,
+    #[cfg(not(no_fp_fmt_parse))]
     Float,
     Char,
     String,
@@ -64,6 +65,7 @@ fn fmt(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
         match *self {
             Unsupported::Boolean => formatter.write_str("a boolean"),
             Unsupported::Integer => formatter.write_str("an integer"),
+            #[cfg(not(no_fp_fmt_parse))]
             Unsupported::Float => formatter.write_str("a float"),
             Unsupported::Char => formatter.write_str("a char"),
             Unsupported::String => formatter.write_str("a string"),
@@ -150,10 +152,12 @@ fn serialize_u64(self, _: u64) -> Result<Self::Ok, Self::Error> {
         Err(self.bad_type(Unsupported::Integer))
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f32(self, _: f32) -> Result<Self::Ok, Self::Error> {
         Err(self.bad_type(Unsupported::Float))
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f64(self, _: f64) -> Result<Self::Ok, Self::Error> {
         Err(self.bad_type(Unsupported::Float))
     }
rust/serde/ser/fmt.rs +6 -2
index e7e09a1bfe94..94c3054fe36b 100644
@@ -55,13 +55,17 @@ impl<'a, 'b> Serializer for &'a mut fmt::Formatter<'b> {
         serialize_u16: u16,
         serialize_u32: u32,
         serialize_u64: u64,
-        serialize_f32: f32,
-        serialize_f64: f64,
         serialize_char: char,
         serialize_str: &str,
         serialize_unit_struct: &'static str,
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
+    fmt_primitives! {
+        serialize_f32: f32,
+        serialize_f64: f64,
+    }
+
     serde_if_integer128! {
         fmt_primitives! {
             serialize_i128: i128,
rust/serde/ser/impls.rs +2 -0
index a79326e5c558..639e480e252d 100644
@@ -29,7 +29,9 @@ fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
 primitive_impl!(u16, serialize_u16);
 primitive_impl!(u32, serialize_u32);
 primitive_impl!(u64, serialize_u64);
+#[cfg(not(no_fp_fmt_parse))]
 primitive_impl!(f32, serialize_f32);
+#[cfg(not(no_fp_fmt_parse))]
 primitive_impl!(f64, serialize_f64);
 primitive_impl!(char, serialize_char);
 
rust/serde/ser/mod.rs +2 -0
index 5c45426e802e..13ea792c58ef 100644
@@ -644,6 +644,7 @@ fn serialize_u128(self, v: u128) -> Result<Self::Ok, Self::Error> {
     ///     }
     /// }
     /// ```
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f32(self, v: f32) -> Result<Self::Ok, Self::Error>;
 
     /// Serialize an `f64` value.
@@ -662,6 +663,7 @@ fn serialize_u128(self, v: u128) -> Result<Self::Ok, Self::Error> {
     ///     }
     /// }
     /// ```
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f64(self, v: f64) -> Result<Self::Ok, Self::Error>;
 
     /// Serialize a character.
Signature · 1 line
2.40.1
#39 lore raw

PATCH 31/80 rust: serde: add SPDX License Identifiers

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)
rust/serde/de/format.rs +2 -0
index f14580b8d162..33233766fc27 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::fmt::{self, Write};
 use lib::str;
 
rust/serde/de/ignored_any.rs +2 -0
index 7b65792e77c0..d2cc83978b50 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 use de::{
rust/serde/de/impls.rs +2 -0
index 002bbf74d65b..9377f5eee650 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 use de::{
rust/serde/de/mod.rs +2 -0
index 017964f0d0d0..6da226da8ffc 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Generic data structure deserialization framework.
 //!
 //! The two most important traits in this module are [`Deserialize`] and
rust/serde/de/seed.rs +2 -0
index 13b7ea461b0c..e0ceeddb100a 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use de::{Deserialize, DeserializeSeed, Deserializer};
 
 /// A DeserializeSeed helper for implementing deserialize_in_place Visitors.
rust/serde/de/utf8.rs +2 -0
index 576fd03cfa79..1fb30488967c 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 const TAG_CONT: u8 = 0b1000_0000;
rust/serde/de/value.rs +2 -0
index d847b8ec071c..dc7be06e9257 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Building blocks for deserializing basic values using the `IntoDeserializer`
 //! trait.
 //!
rust/serde/integer128.rs +2 -0
index 904c2a233f8a..b47862335f28 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 /// Conditional compilation depending on whether Serde is built with support for
 /// 128-bit integers.
 ///
rust/serde/lib.rs +2 -0
index 699573d95e4b..c3694fef3321 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! # Serde
 //!
 //! Serde is a framework for ***ser***ializing and ***de***serializing Rust data
rust/serde/macros.rs +2 -0
index 6502a23a7af7..0b0a9ad823e2 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // Super explicit first paragraph because this shows up at the top level and
 // trips up people who are just looking for basic Serialize / Deserialize
 // documentation.
rust/serde/private/de.rs +2 -0
index e9c693d4dd31..707c053b7fce 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 use de::value::{BorrowedBytesDeserializer, BytesDeserializer};
rust/serde/private/doc.rs +2 -0
index f597af84451d..271e0c4f7b70 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // Used only by Serde doc tests. Not public API.
 
 use lib::*;
rust/serde/private/mod.rs +2 -0
index e896902360ba..c05a0ea5c8e3 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(not(no_serde_derive))]
 pub mod de;
 #[cfg(not(no_serde_derive))]
rust/serde/private/ser.rs +2 -0
index fc9e7e6d7426..6efd84e5fe57 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 use ser::{self, Impossible, Serialize, SerializeMap, SerializeStruct, Serializer};
rust/serde/private/size_hint.rs +2 -0
index ca71e616bb69..f050fca59deb 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 pub fn from_bounds<I>(iter: &I) -> Option<usize>
rust/serde/ser/fmt.rs +2 -0
index 94c3054fe36b..7906cf76d99f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 use ser::{Error, Impossible, Serialize, Serializer};
 
rust/serde/ser/impls.rs +2 -0
index 639e480e252d..d70533eb41c3 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::*;
 
 use ser::{Error, Serialize, SerializeTuple, Serializer};
rust/serde/ser/impossible.rs +2 -0
index e8df9ca7aad2..e25270ba70c1 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! This module contains `Impossible` serializer and its implementations.
 
 use lib::*;
rust/serde/ser/mod.rs +2 -0
index 13ea792c58ef..9a5e47d8c5c8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Generic data structure serialization framework.
 //!
 //! The two most important traits in this module are [`Serialize`] and
rust/serde/std_error.rs +2 -0
index 1055e0ffbcb2..b730d38a1e07 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::{Debug, Display};
 
 /// Either a re-export of std::error::Error or a new identical trait, depending
Signature · 1 line
2.40.1
#40 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 31 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)
rust/serde/de/format.rs +0 -0
index f14580b8d162..33233766fc27 100644

rust/serde/de/format.rs line 4

Quoted 4 lines
rust/serde/de/format.rs +2 -0
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use lib::fmt::{self, Write};
 use lib::str;

Hang on, are we applying our own local modifications to code vendored from an outside repository?

We gotta stop doing this - this turns our version into an unsupported fork. What do we then do when we need to merge in security updates from the upstream repository?

The only sane thing to do is keep the vendored code in lockstep with the upstream repository - i.e. always a direct copy of a specific commit. Any changes for the kernel need to be applied to the upstream repository (which could be an intermediate repository, not necessarly _the_ upstream repository; the point is, that way there's a proper revision history for _that_ repo, and changes can get merged upstream).

Otherwise, we're setting ourselves up for a maintainence nightmare...

maintainer
#41 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 2 lines · Kent Overstreet

Hang on, are we applying our own local modifications to code vendored from an outside repository?

For adding a single license line, I think it's going to be ok, those are trivial to handle :)

Quoted 3 lines

We gotta stop doing this - this turns our version into an unsupported fork. What do we then do when we need to merge in security updates from the upstream repository?

Same as we do for other parts of the kernel that are copies of other codebases, we merge them in as needed.

thanks,

greg k-h

#42 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 6 lines · Greg KH
Quoted 2 lines · Kent Overstreet

Hang on, are we applying our own local modifications to code vendored from an outside repository?

For adding a single license line, I think it's going to be ok, those are trivial to handle :)

They'll still be lost when the package is updated from upstream. Just do a proper upstream patch - I see no reason why SPDX identifiers wouldn't be accepted by the host project.

Quoted 7 lines
Quoted 4 lines

We gotta stop doing this - this turns our version into an unsupported fork. What do we then do when we need to merge in security updates from the upstream repository?

Same as we do for other parts of the kernel that are copies of other codebases, we merge them in as needed.

Yeah, and that workflow still leaves something to be desired...

maintainer
#43 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 11 lines · Kent Overstreet
Quoted 6 lines · Greg KH
Quoted 2 lines · Kent Overstreet

Hang on, are we applying our own local modifications to code vendored from an outside repository?

For adding a single license line, I think it's going to be ok, those are trivial to handle :)

They'll still be lost when the package is updated from upstream. Just do a proper upstream patch - I see no reason why SPDX identifiers wouldn't be accepted by the host project.

Great, then submit them upstream also, but for now, when added to the kernel tree, they need to be present.

thanks,

greg k-h

#44 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

In reply to Ariel Miculas
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)

Separately from that - are we sure we want to be pulling Serde into the kernel?

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it supports multiple formats), and object size is going to be a consideration. In particular, the way it works is by generating (via procedural macros) encode/decode functions for every type you use #[derive] an encoder/decoder for, so depending on how that was implemented that's going to add up.

I would need to see text size numbers - the bloaty tool might be useful here: https://github.com/google/bloaty

I'd give serious consideration to Cap'n proto instead, it's not as ergonomic as Serde (the get/set interface is not ideal, ask me for more details on how this could be fixed) but it's lower overhead than Serde.

https://github.com/capnproto/capnproto-rust

#45 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 27 lines · Kent Overstreet
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)
Quoted 8 lines

Separately from that - are we sure we want to be pulling Serde into the kernel?

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it

I think the json capability lives in serde_json, so this is actually not a full json parser.

BR Andreas

Quoted 14 lines

supports multiple formats), and object size is going to be a consideration. In particular, the way it works is by generating (via procedural macros) encode/decode functions for every type you use #[derive] an encoder/decoder for, so depending on how that was implemented that's going to add up.

I would need to see text size numbers - the bloaty tool might be useful here: https://github.com/google/bloaty

I'd give serious consideration to Cap'n proto instead, it's not as ergonomic as Serde (the get/set interface is not ideal, ask me for more details on how this could be fixed) but it's lower overhead than Serde.

https://github.com/capnproto/capnproto-rust

#46 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 33 lines · Andreas Hindborg (Samsung)
Quoted 30 lines · Kent Overstreet
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)

Separately from that - are we sure we want to be pulling Serde into the kernel?

Quoted 8 lines
Quoted 5 lines

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it

I think the json capability lives in serde_json, so this is actually not a full json parser.

Good to know

I don't think I'm the only one who's curious though - what's serde being used for here? On disk data structures of some kind?

I'm probably not the only filesystem author eagerly waiting for the day when I can stop using carefully packed C structs for this, with all the sharp edges that entails...

#47 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 44 lines · Kent Overstreet
Quoted 33 lines · Andreas Hindborg (Samsung)
Quoted 30 lines · Kent Overstreet
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)

Separately from that - are we sure we want to be pulling Serde into the kernel?

Quoted 8 lines
Quoted 5 lines

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it

I think the json capability lives in serde_json, so this is actually not a full json parser.

Quoted 8 lines

Good to know

I don't think I'm the only one who's curious though - what's serde being used for here? On disk data structures of some kind?

I'm probably not the only filesystem author eagerly waiting for the day when I can stop using carefully packed C structs for this, with all the sharp edges that entails...

serde, together with serde_cbor (added in another commit [3]), is used to decode the puzzlefs metadata (check the "format" section for the initial design [1]).
Sorry for the github links, but the patch series is incomplete because I didn't send them to [email protected], the entire "pull request" is available here [2].

https://github.com/anuvu/puzzlefs/pull/6#issue-801680988 [1]
https://github.com/Rust-for-Linux/linux/compare/rust-next...ariel-miculas:puzzlefs_rfc_v1
[2]
https://github.com/Rust-for-Linux/linux/commit/3fe9d58fbd89b0ed488d3150620eb07be17f1f16
[3]

#48 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

In reply to Kent Overstreet
Quoted 3 lines · Kent Overstreet

Separately from that - are we sure we want to be pulling Serde into the kernel?

No, we are not -- see https://rust-for-linux.com/third-party-crates
(these have my SoBs because I provide an experiment branch at
https://github.com/Rust-for-Linux/linux/pull/1007, since some people wanted to use them, but it is not decided which ones to have).

Cheers,
Miguel

#49 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

In reply to Kent Overstreet
Quoted 42 lines · Kent Overstreet
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)

Separately from that - are we sure we want to be pulling Serde into the kernel?

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it supports multiple formats), and object size is going to be a consideration. In particular, the way it works is by generating (via procedural macros) encode/decode functions for every type you use #[derive] an encoder/decoder for, so depending on how that was implemented that's going to add up.

Quoted 8 lines

I would need to see text size numbers - the bloaty tool might be useful here: https://github.com/google/bloaty

I'd give serious consideration to Cap'n proto instead, it's not as ergonomic as Serde (the get/set interface is not ideal, ask me for more details on how this could be fixed) but it's lower overhead than Serde.

https://github.com/capnproto/capnproto-rust

Unfortunately bloaty seems broken [1], but here's the output:

```
$ bloaty puzzlefs_vmlinux -- vmlinux
    FILE SIZE        VM SIZE
 --------------  --------------
  +102%  +921Ki  [ = ]       0    .debug_info
  +139%  +849Ki  [ = ]       0    .debug_str
  +251%  +574Ki  [ = ]       0    .debug_loc
  +116%  +246Ki  [ = ]       0    .strtab
  +292%  +226Ki  [ = ]       0    .debug_pubtypes
  +297%  +200Ki  [ = ]       0    .debug_pubnames
  +115%  +176Ki  +115%  +176Ki    .rodata
  +103%  +160Ki  [ = ]       0    .debug_line
  +287%  +152Ki  [ = ]       0    .debug_ranges
  +104% +44.9Ki  [ = ]       0    .debug_frame
  +103% +43.2Ki  [ = ]       0    .symtab
  +112% +24.0Ki  +112% +24.0Ki    __ksymtab_strings
  +100% +5.31Ki  [ = ]       0    .debug_abbrev
  +100% +4.20Ki  [ = ]       0    .debug_str_offsets
  +108% +2.89Ki  +108% +2.89Ki    __ksymtab_gpl
  +100% +2.13Ki  [ = ]       0    .debug_addr
  +100%    +452  [ = ]       0    .debug_loclists
  +129%    +416  [ = ]       0    .debug_aranges
  +0.0%     +60  +0.0%     +68    [8 Others]
   +99%     -64   +99%     -64    [LOAD #3 [RWX]]
   +96%  -202Ki  [ = ]       0    [Unmapped]
  +104% +90.2Mi  +101% +13.8Mi    TOTAL
```
where puzzlefs_vmlinux is the kernel built with my puzzlefs changes
and vmlinux is the kernel built on the rust-next branch.
```
$ ls -lah puzzlefs_vmlinux
-rwxr-xr-x 1 amiculas amiculas 91M Jun 12 14:38 puzzlefs_vmlinux
~/work/linux rust-next*
$ ls -lah vmlinux
-rwxr-xr-x 1 amiculas amiculas 87M Jun 12 14:38 vmlinux
```

It's a +4M difference between the two versions (3.35 M to be exact), but they are both debug builds (with CONFIG_DEBUG_KERNEL and CONFIG_KALLSYMS set).
If I should do additional comparisons, please let me know.

https://github.com/google/bloaty/issues/335 [1]

#50 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

In reply to Kent Overstreet
Quoted 42 lines · Kent Overstreet
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)

Separately from that - are we sure we want to be pulling Serde into the kernel?

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it supports multiple formats), and object size is going to be a consideration. In particular, the way it works is by generating (via procedural macros) encode/decode functions for every type you use #[derive] an encoder/decoder for, so depending on how that was implemented that's going to add up.

Quoted 6 lines

I would need to see text size numbers - the bloaty tool might be useful here: https://github.com/google/bloaty

I'd give serious consideration to Cap'n proto instead, it's not as ergonomic as Serde (the get/set interface is not ideal, ask me for more details on how this could be fixed) but it's lower overhead than Serde.

I've been checking out Cap'n proto and indeed that get/set interface seems cumbersome, so how do you fix it?

Quoted 2 lines

https://github.com/capnproto/capnproto-rust

Regards,
Ariel

#51 lore raw

Re: [PATCH 31/80] rust: serde: add SPDX License Identifiers

Quoted 44 lines · Ariel Miculas
Quoted 42 lines · Kent Overstreet
Quoted 17 lines · Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde/de/format.rs         | 2 ++
 rust/serde/de/ignored_any.rs    | 2 ++
 rust/serde/de/impls.rs          | 2 ++
 rust/serde/de/mod.rs            | 2 ++
 rust/serde/de/seed.rs           | 2 ++
 rust/serde/de/utf8.rs           | 2 ++
 rust/serde/de/value.rs          | 2 ++
 rust/serde/integer128.rs        | 2 ++
 rust/serde/lib.rs               | 2 ++
 rust/serde/macros.rs            | 2 ++
 rust/serde/private/de.rs        | 2 ++
 rust/serde/private/doc.rs       | 2 ++
 rust/serde/private/mod.rs       | 2 ++
Quoted 8 lines
 rust/serde/private/ser.rs       | 2 ++
 rust/serde/private/size_hint.rs | 2 ++
 rust/serde/ser/fmt.rs           | 2 ++
 rust/serde/ser/impls.rs         | 2 ++
 rust/serde/ser/impossible.rs    | 2 ++
 rust/serde/ser/mod.rs           | 2 ++
 rust/serde/std_error.rs         | 2 ++
 20 files changed, 40 insertions(+)

Separately from that - are we sure we want to be pulling Serde into the kernel?

Don't get me wrong, Serde is amazing - it's highly ergonomic, and makes a lot of compatibility issues just completely go away. It's impresssively well done.

But it's a full json parser (are you using it in json mode here? it supports multiple formats), and object size is going to be a consideration. In particular, the way it works is by generating (via procedural macros) encode/decode functions for every type you use #[derive] an encoder/decoder for, so depending on how that was implemented that's going to add up.

Quoted 8 lines
Quoted 6 lines

I would need to see text size numbers - the bloaty tool might be useful here: https://github.com/google/bloaty

I'd give serious consideration to Cap'n proto instead, it's not as ergonomic as Serde (the get/set interface is not ideal, ask me for more details on how this could be fixed) but it's lower overhead than Serde.

I've been checking out Cap'n proto and indeed that get/set interface seems cumbersome, so how do you fix it?

One thing that makes it cumbersome is that any get can return an error:
a straightforward change would be to add a method that does all validation/bounds checking up front, and returns a new type with methods that can't return errors.

Kenton did it that way to avoid any O(n) parsing overhead up front; with messages that can be arbitrarily sized it's definitely an option that you want to have, especially when you won't necessarily be reading the entire thing. But in my experience most messages are short, and there's a reading-off-the-disk step or reading-from-the-network step, and you probably want to do other validation, so doing all validation up front simplifies error handling.

For the next step, what we really what we'd want is for cap'n proto to generate normal Rust types but with a #[repr] that matches the cap'n proto type layout.

Having the cap'n proto compiler emit #[repr(c)] types with the appropriate padding isn't enough because:
- integers are stored little endian, and need byte swabbing to access
- booleans are stored as bitfields, and Rust doesn't have native bitfields

Doing this right would require importing computed properties from Swift:
a computed property is something that looks like a struct field but requires get/set calls to access it, the compiler just knows how to emit them.

Miguel might be able to point us towards rustc people to talk to about this :) Having native language-level types which are directly serializable is amazing, as we've already seen with Serde. If we could skip the encode/decode step and just mmap a file and start walking objects - this could really change how we write and design filesystems, databases - even compilers. Debug info (e.g. dwarf) is all about serializing compiler internals, and a lot of the problems compiler people are tackling right now basically require doing this in a more principled way.

#52 lore raw

PATCH 33/80 rust: serde_derive: add SPDX License Identifiers

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

---
 rust/serde_derive/bound.rs              | 2 ++
 rust/serde_derive/de.rs                 | 2 ++
 rust/serde_derive/dummy.rs              | 2 ++
 rust/serde_derive/fragment.rs           | 2 ++
 rust/serde_derive/internals/ast.rs      | 2 ++
 rust/serde_derive/internals/attr.rs     | 2 ++
 rust/serde_derive/internals/case.rs     | 2 ++
 rust/serde_derive/internals/check.rs    | 2 ++
 rust/serde_derive/internals/ctxt.rs     | 2 ++
 rust/serde_derive/internals/mod.rs      | 2 ++
 rust/serde_derive/internals/receiver.rs | 2 ++
 rust/serde_derive/internals/respan.rs   | 2 ++
 rust/serde_derive/internals/symbol.rs   | 2 ++
 rust/serde_derive/lib.rs                | 2 ++
 rust/serde_derive/pretend.rs            | 2 ++
 rust/serde_derive/ser.rs                | 2 ++
 rust/serde_derive/this.rs               | 2 ++
 rust/serde_derive/try.rs                | 2 ++
 18 files changed, 36 insertions(+)
rust/serde_derive/bound.rs +2 -0
index 74c95069c353..dc6b5907ff80 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use std::collections::HashSet;
 
 use syn;
rust/serde_derive/de.rs +2 -0
index 884c6335c7ae..06780ac192e8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Literal, Span, TokenStream};
 use quote::ToTokens;
 use syn::punctuated::Punctuated;
rust/serde_derive/dummy.rs +2 -0
index 2be502713293..3530303f23f2 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Ident, TokenStream};
 use quote::format_ident;
 
rust/serde_derive/fragment.rs +2 -0
index 324504aa4661..a62ed9114224 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::TokenStream;
 use quote::ToTokens;
 use syn::token;
rust/serde_derive/internals/ast.rs +2 -0
index 2a6950b2a710..139540571820 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! A Serde ast, parsed from the Syn ast and ready to generate Rust code.
 
 use internals::attr;
rust/serde_derive/internals/attr.rs +2 -0
index 5ff190f02e77..d013215a5e91 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use internals::symbol::*;
 use internals::{ungroup, Ctxt};
 use proc_macro2::{Spacing, Span, TokenStream, TokenTree};
rust/serde_derive/internals/case.rs +2 -0
index 554505160ea4..6a75c7460e67 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Code to convert the Rust-styled field/variant (e.g. `my_field`, `MyType`) to the
 //! case of the source (e.g. `my-field`, `MY_FIELD`).
 
rust/serde_derive/internals/check.rs +2 -0
index eb1297aa73bf..188499508684 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use internals::ast::{Container, Data, Field, Style};
 use internals::attr::{Identifier, TagType};
 use internals::{ungroup, Ctxt, Derive};
rust/serde_derive/internals/ctxt.rs +2 -0
index d692c2a449d3..7a0a9f92f992 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use quote::ToTokens;
 use std::cell::RefCell;
 use std::fmt::Display;
rust/serde_derive/internals/mod.rs +2 -0
index 5e9f416c46f2..9de56241adf3 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 pub mod ast;
 pub mod attr;
 
rust/serde_derive/internals/receiver.rs +2 -0
index b08c67096844..fc5bc1b89db2 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use internals::respan::respan;
 use proc_macro2::Span;
 use quote::ToTokens;
rust/serde_derive/internals/respan.rs +2 -0
index dcec7017b315..41aca1250c21 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Group, Span, TokenStream, TokenTree};
 
 pub(crate) fn respan(stream: TokenStream, span: Span) -> TokenStream {
rust/serde_derive/internals/symbol.rs +2 -0
index 9606edb5f236..fb7cda06056f 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use std::fmt::{self, Display};
 use syn::{Ident, Path};
 
rust/serde_derive/lib.rs +2 -0
index d3d136cb092c..de83885b9a4c 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! This crate provides Serde's two derive macros.
 //!
 //! ```edition2018
rust/serde_derive/pretend.rs +2 -0
index d7b953d6384f..522fc8c2f2d9 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::TokenStream;
 use quote::format_ident;
 
rust/serde_derive/ser.rs +2 -0
index 43695dd0c39a..00a667cea7fa 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Span, TokenStream};
 use syn::spanned::Spanned;
 use syn::{self, Ident, Index, Member};
rust/serde_derive/this.rs +2 -0
index 32731d08969d..7a27a7c7dbc2 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use internals::ast::Container;
 use syn::{Path, PathArguments, Token};
 
rust/serde_derive/try.rs +2 -0
index 48cceebaafe1..afed0fedb938 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use proc_macro2::{Punct, Spacing, TokenStream};
 
 // None of our generated code requires the `From::from` error conversion
Signature · 1 line
2.40.1
#53 lore raw

PATCH 34/80 rust: Kbuild: enable `proc-macro2`, `quote`, `syn`, `serde` and `serde_derive`

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

With all the new files in place from the new crates, this patch adds support for them in the build system.

Note that, if we decide to add third-party crates support to upstream, we would probably want to do it in a different way. But this is useful for playing around with these crates, experimenting, etc.

---
 Makefile               |  12 ++-
 rust/.gitignore        |   1 +
 rust/Makefile          | 194 ++++++++++++++++++++++++++++++++++++-----
 scripts/Makefile.build |   2 +-
 4 files changed, 185 insertions(+), 24 deletions(-)
Makefile +11 -1
index 836643eaefee..3b7f37977f04 100644
@@ -1606,7 +1606,7 @@ MRPROPER_FILES += include/config include/generated          \
 		  certs/x509.genkey \
 		  vmlinux-gdb.py \
 		  *.spec rpmbuild \
-		  rust/libmacros.so
+		  rust/libmacros.so rust/libserde_derive.so
 
 # clean - Delete most, but leave enough to build external modules
 #
@@ -1848,8 +1848,18 @@ PHONY += rustfmt rustfmtcheck
 rustfmt:
 	$(Q)find $(abs_srctree) -type f -name '*.rs' \
 		-o -path $(abs_srctree)/rust/alloc -prune \
+		-o -path $(abs_srctree)/rust/proc-macro2 -prune \
+		-o -path $(abs_srctree)/rust/quote -prune \
+		-o -path $(abs_srctree)/rust/syn -prune \
+		-o -path $(abs_srctree)/rust/serde -prune \
+		-o -path $(abs_srctree)/rust/serde_derive -prune \
 		-o -path $(abs_objtree)/rust/test -prune \
 		| grep -Fv $(abs_srctree)/rust/alloc \
+		| grep -Fv $(abs_srctree)/rust/proc-macro2 \
+		| grep -Fv $(abs_srctree)/rust/quote \
+		| grep -Fv $(abs_srctree)/rust/syn \
+		| grep -Fv $(abs_srctree)/rust/serde \
+		| grep -Fv $(abs_srctree)/rust/serde_derive \
 		| grep -Fv $(abs_objtree)/rust/test \
 		| grep -Fv generated \
 		| xargs $(RUSTFMT) $(rustfmt_flags)
rust/.gitignore +1 -0
index 21552992b401..f6c521160123 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
+*.rlib
 bindings_generated.rs
 bindings_helpers_generated.rs
 uapi_generated.rs
rust/Makefile +172 -22

277 lines not on this page · raw message

scripts/Makefile.build +1 -1
index 1404967e908e..3fa298bb7c31 100644
@@ -284,7 +284,7 @@ rust_common_cmd = \
 	-Zallow-features=$(rust_allowed_features) \
 	-Zcrate-attr=no_std \
 	-Zcrate-attr='feature($(rust_allowed_features))' \
-	--extern alloc --extern kernel \
+	--extern alloc --extern kernel --extern serde --extern serde_derive \
 	--crate-type rlib -L $(objtree)/rust/ \
 	--crate-name $(basename $(notdir $@)) \
 	--emit=dep-info=$(depfile)
Signature · 1 line
2.40.1
#54 lore raw

PATCH 35/80 rust: test `serde` support

In reply to Ariel Miculas

From: Miguel Ojeda <[email protected]>

A trivial example based on serde's `example-format' [1].

It contains both a in-kernel data format later used by the kernel module, as well as a local data format in the module.

The kernel module gives an output such as:

    [    0.801425] rust_serde: Rust serde sample (init)
    [    0.801634] rust_serde:             original = S { a: (), b: false, c: true, d: () }
    [    0.802079] rust_serde:           serialized = [2, 0, 1, 0, 1, 1, 0, 3]
    [    0.802506] rust_serde:         deserialized = S { a: (), b: false, c: true, d: () }
    [    0.802718] rust_serde:   serialized (local) = [2, 0, 1, 42, 1, 43, 0, 3]
    [    0.802895] rust_serde: deserialized (local) = S { a: (), b: false, c: true, d: () }
    [    0.808954] rust_serde: Rust serde sample (exit)

Note that this is just a quick draft/hack to check the previous commits work. It is not intended to be merged at all.

---
 rust/kernel/lib.rs                      |   1 +
 rust/kernel/test_serde.rs               |  26 ++
 rust/kernel/test_serde/de.rs            | 442 ++++++++++++++++++++++
 rust/kernel/test_serde/error.rs         |  73 ++++
 rust/kernel/test_serde/ser.rs           | 469 ++++++++++++++++++++++++
 samples/rust/Kconfig                    |   8 +
 samples/rust/Makefile                   |   1 +
 samples/rust/local_data_format/de.rs    | 422 +++++++++++++++++++++
 samples/rust/local_data_format/error.rs |  73 ++++
 samples/rust/local_data_format/ser.rs   | 443 ++++++++++++++++++++++
 samples/rust/rust_serde.rs              |  73 ++++
 11 files changed, 2031 insertions(+)
 create mode 100644 rust/kernel/test_serde.rs
 create mode 100644 rust/kernel/test_serde/de.rs
 create mode 100644 rust/kernel/test_serde/error.rs
 create mode 100644 rust/kernel/test_serde/ser.rs
 create mode 100644 samples/rust/local_data_format/de.rs
 create mode 100644 samples/rust/local_data_format/error.rs
 create mode 100644 samples/rust/local_data_format/ser.rs
 create mode 100644 samples/rust/rust_serde.rs
rust/kernel/lib.rs +1 -0
index 08f67833afcf..a94fb784d576 100644
@@ -178,3 +178,4 @@ fn panic(info: &core::panic::PanicInfo<'_>) -> ! {
 ///
 /// [`PAGE_SHIFT`]: ../../../include/asm-generic/page.h
 pub const PAGE_SIZE: usize = 1 << bindings::PAGE_SHIFT;
+pub mod test_serde;
rust/kernel/test_serde.rs new file +26 -0
new file mode 100644
index 000000000000..012bf58213a8
@@ -0,0 +1,26 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+//! Test `serde`.
+//!
+//! It contains a data format used by the `rust_serde` sample, as well
+//! as a quick check that `serde_derive` works in the `kernel` crate too.
+
+#![allow(missing_docs)]
+
+mod de;
+mod error;
+mod ser;
+
+pub use de::{from_bytes, Deserializer};
+pub use error::{Error, Result};
+pub use ser::{to_vec, Serializer};
+
+use serde_derive::{Deserialize, Serialize};
+
+#[derive(Serialize, Deserialize, Debug)]
+pub struct S {
+    a: (),
+    b: bool,
+    c: bool,
+    d: (),
+}
rust/kernel/test_serde/de.rs new file +442 -0

443 lines not on this page · raw message

rust/kernel/test_serde/error.rs new file +73 -0

73 lines not on this page · raw message

rust/kernel/test_serde/ser.rs new file +469 -0

469 lines not on this page · raw message

samples/rust/Kconfig +8 -0
index 05ca21fbba06..a408f098fbfd 100644
@@ -48,6 +48,14 @@ config SAMPLE_PUZZLEFS
 	  To compile this as a module, choose M here:
 	  the module will be called puzzlefs.
 
+config SAMPLE_RUST_SERDE
+	tristate "Serde"
+	help
+	  This option builds the Rust `serde` sample.
+
+	  To compile this as a module, choose M here:
+	  the module will be called rust_serde.
+
 	  If unsure, say N.
 
 config SAMPLE_RUST_HOSTPROGS
samples/rust/Makefile +1 -0
index 364a38dbf90b..bc27f97d71ad 100644
@@ -4,5 +4,6 @@ obj-$(CONFIG_SAMPLE_RUST_MINIMAL)		+= rust_minimal.o
 obj-$(CONFIG_SAMPLE_RUST_PRINT)			+= rust_print.o
 obj-$(CONFIG_SAMPLE_RUST_FS)			+= rust_fs.o
 obj-$(CONFIG_SAMPLE_PUZZLEFS)			+= puzzlefs.o
+obj-$(CONFIG_SAMPLE_RUST_SERDE)			+= rust_serde.o
 
 subdir-$(CONFIG_SAMPLE_RUST_HOSTPROGS)		+= hostprogs
samples/rust/local_data_format/de.rs new file +422 -0

422 lines not on this page · raw message

samples/rust/local_data_format/error.rs new file +73 -0

73 lines not on this page · raw message

samples/rust/local_data_format/ser.rs new file +443 -0

443 lines not on this page · raw message

samples/rust/rust_serde.rs new file +73 -0

73 lines not on this page · raw message

Signature · 1 line
2.40.1
#55 lore raw

PATCH 36/80 Add SAMPLE_RUST_SERDE in rust.config

In reply to Ariel Miculas
---
 kernel/configs/rust.config | 1 +
 1 file changed, 1 insertion(+)
kernel/configs/rust.config +1 -0
index 422b49798d92..5e29e66e8f7d 100644
@@ -8,4 +8,5 @@ CONFIG_SAMPLE_RUST_PRINT=m
 CONFIG_SAMPLE_RUST_FS=m
 CONFIG_SAMPLE_PUZZLEFS=m
 CONFIG_SAMPLE_RUST_HOSTPROGS=y
+CONFIG_SAMPLE_RUST_SERDE=y
 
Signature · 1 line
2.40.1
#56 lore raw

PATCH 37/80 rust: kernel: fix compile errors after rebase to rust-next

In reply to Ariel Miculas
---
 rust/kernel/iov_iter.rs | 2 +-
 rust/kernel/mm.rs       | 8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)
rust/kernel/iov_iter.rs +1 -1
index 4602cbd4b49c..78d573e03175 100644
@@ -23,7 +23,7 @@ pub struct IovIter {
 impl IovIter {
     fn common_len(&self) -> usize {
         // SAFETY: `IovIter::ptr` is guaranteed to be valid by the type invariants.
-        unsafe { (*self.ptr).count }
+        unsafe { (*self.ptr).__bindgen_anon_2.__bindgen_anon_1.count }
     }
 
     /// Constructs a new [`struct iov_iter`] wrapper.
rust/kernel/mm.rs +4 -4
index 72b504c949c1..edd022762aaa 100644
@@ -38,7 +38,7 @@ pub(crate) unsafe fn from_ptr(vma: *mut bindings::vm_area_struct) -> Self {
         /// The possible flags are a combination of the constants in [`flags`].
         pub fn flags(&self) -> usize {
             // SAFETY: `self.vma` is valid by the type invariants.
-            unsafe { (*self.vma).__bindgen_anon_1.vm_flags as _ }
+            unsafe { (*self.vma).__bindgen_anon_2.vm_flags as _ }
         }
 
         /// Sets the flags associated with the virtual memory area.
@@ -46,19 +46,19 @@ pub fn flags(&self) -> usize {
         /// The possible flags are a combination of the constants in [`flags`].
         pub fn set_flags(&mut self, flags: usize) {
             // SAFETY: `self.vma` is valid by the type invariants.
-            unsafe { (*self.vma).__bindgen_anon_1.vm_flags = flags as _ };
+            unsafe { (*self.vma).__bindgen_anon_2.vm_flags = flags as _ };
         }
 
         /// Returns the start address of the virtual memory area.
         pub fn start(&self) -> usize {
             // SAFETY: `self.vma` is valid by the type invariants.
-            unsafe { (*self.vma).vm_start as _ }
+            unsafe { (*self.vma).__bindgen_anon_1.__bindgen_anon_1.vm_start as _ }
         }
 
         /// Returns the end address of the virtual memory area.
         pub fn end(&self) -> usize {
             // SAFETY: `self.vma` is valid by the type invariants.
-            unsafe { (*self.vma).vm_end as _ }
+            unsafe { (*self.vma).__bindgen_anon_1.__bindgen_anon_1.vm_end as _ }
         }
 
         /// Maps a single page at the given address within the virtual memory area.
Signature · 1 line
2.40.1
#57 lore raw

Re: [PATCH 37/80] rust: kernel: fix compile errors after rebase to rust-next

Quoted 2 lines · Ariel Miculas

Not a big deal since it is an RFC, but please note that we want to keep all commits building (and passing docs, tests etc.).

If the patch is from somebody else (which may be the reason why you
split it), then ideally the original author would sign off on your
changes (or, even better, they can send the updated patch themselves),
potentially having you as Co-developed-by if they are bigger
changes, or you can otherwise use the [ ... ] notation in the
message to indicate what you changed (e.g. for small notes I use this
style: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f431c5c581fa).

Cheers,
Miguel

#58 lore raw

PATCH 39/80 rust: serde_cbor: add SPDX License Identifiers

In reply to Ariel Miculas
---
 rust/serde_cbor/de.rs        | 2 ++
 rust/serde_cbor/error.rs     | 2 ++
 rust/serde_cbor/lib.rs       | 2 ++
 rust/serde_cbor/read.rs      | 2 ++
 rust/serde_cbor/ser.rs       | 2 ++
 rust/serde_cbor/tags.rs      | 2 ++
 rust/serde_cbor/value/de.rs  | 2 ++
 rust/serde_cbor/value/mod.rs | 2 ++
 rust/serde_cbor/value/ser.rs | 2 ++
 rust/serde_cbor/write.rs     | 2 ++
 10 files changed, 20 insertions(+)
rust/serde_cbor/de.rs +2 -0
index 170e0593cf44..1c9a9acc8eb5 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Deserialization.
 
 use core::f32;
rust/serde_cbor/error.rs +2 -0
index b1a6a459e540..0b4b5be8edcc 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! When serializing or deserializing CBOR goes wrong.
 use core::fmt;
 use core::result;
rust/serde_cbor/lib.rs +2 -0
index 55668541539b..cf406a439340 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! CBOR and serialization.
 //!
 //! # Usage
rust/serde_cbor/read.rs +2 -0
index 1b53018df870..6c17282d2c5d 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "alloc")]
 use alloc::{vec, vec::Vec};
 #[cfg(feature = "std")]
rust/serde_cbor/ser.rs +2 -0
index 7016dc340a84..9f5bb67f0479 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Serialize a Rust data structure to CBOR data.
 
 #[cfg(feature = "alloc")]
rust/serde_cbor/tags.rs +2 -0
index 8adccb8ea8b5..057765b9d2f8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Support for cbor tags
 use core::fmt;
 use core::marker::PhantomData;
rust/serde_cbor/value/de.rs +2 -0
index f5bdbb74676c..366eda1e27ac 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use std::collections::BTreeMap;
 use std::fmt;
 
rust/serde_cbor/value/mod.rs +2 -0
index 7bd2255314ca..cc93c2b40a52 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! CBOR values, keys and serialization routines.
 
 mod de;
rust/serde_cbor/value/ser.rs +2 -0
index 347aae9601fa..e52987778288 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // Copyright 2017 Serde Developers
 //
 // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
rust/serde_cbor/write.rs +2 -0
index 94c326ef96b0..21baf7de9711 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 #[cfg(feature = "alloc")]
 use alloc::vec::Vec;
 #[cfg(not(feature = "std"))]
Signature · 1 line
2.40.1
#59 lore raw

PATCH 40/80 rust: serde_cbor: add no_fp_fmt_parse support

In reply to Ariel Miculas

We do not have formatting for floating point in the kernel, thus simple compile out all that.

---
 rust/serde_cbor/de.rs        | 8 ++++++++
 rust/serde_cbor/ser.rs       | 3 +++
 rust/serde_cbor/tags.rs      | 2 ++
 rust/serde_cbor/value/ser.rs | 2 ++
 4 files changed, 15 insertions(+)
rust/serde_cbor/de.rs +8 -0
index 1c9a9acc8eb5..534f9d53aa3b 100644
@@ -2,10 +2,12 @@
 
 //! Deserialization.
 
+#[cfg(not(no_fp_fmt_parse))]
 use core::f32;
 use core::marker::PhantomData;
 use core::result;
 use core::str;
+#[cfg(not(no_fp_fmt_parse))]
 use half::f16;
 use serde::de;
 #[cfg(feature = "std")]
@@ -566,14 +568,17 @@ fn parse_indefinite_enum<V>(&mut self, visitor: V) -> Result<V::Value>
         })
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
     fn parse_f16(&mut self) -> Result<f32> {
         Ok(f32::from(f16::from_bits(self.parse_u16()?)))
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
     fn parse_f32(&mut self) -> Result<f32> {
         self.parse_u32().map(|i| f32::from_bits(i))
     }
 
+    #[cfg(not(no_fp_fmt_parse))]
     fn parse_f64(&mut self) -> Result<f64> {
         self.parse_u64().map(|i| f64::from_bits(i))
     }
@@ -756,14 +761,17 @@ fn parse_value<V>(&mut self, visitor: V) -> Result<V::Value>
             0xf6 => visitor.visit_unit(),
             0xf7 => visitor.visit_unit(),
             0xf8 => Err(self.error(ErrorCode::UnassignedCode)),
+            #[cfg(not(no_fp_fmt_parse))]
             0xf9 => {
                 let value = self.parse_f16()?;
                 visitor.visit_f32(value)
             }
+            #[cfg(not(no_fp_fmt_parse))]
             0xfa => {
                 let value = self.parse_f32()?;
                 visitor.visit_f32(value)
             }
+            #[cfg(not(no_fp_fmt_parse))]
             0xfb => {
                 let value = self.parse_f64()?;
                 visitor.visit_f64(value)
rust/serde_cbor/ser.rs +3 -0
index 9f5bb67f0479..8c2273f9f544 100644
@@ -10,6 +10,7 @@
 pub use crate::write::{SliceWrite, Write};
 
 use crate::error::{Error, Result};
+#[cfg(not(no_fp_fmt_parse))]
 use half::f16;
 use serde::ser::{self, Serialize};
 #[cfg(feature = "std")]
@@ -311,6 +312,7 @@ fn serialize_u128(self, value: u128) -> Result<()> {
 
     #[inline]
     #[allow(clippy::float_cmp)]
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f32(self, value: f32) -> Result<()> {
         if value.is_infinite() {
             if value.is_sign_positive() {
@@ -334,6 +336,7 @@ fn serialize_f32(self, value: f32) -> Result<()> {
 
     #[inline]
     #[allow(clippy::float_cmp)]
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f64(self, value: f64) -> Result<()> {
         if !value.is_finite() || f64::from(value as f32) == value {
             self.serialize_f32(value as f32)
rust/serde_cbor/tags.rs +2 -0
index 057765b9d2f8..81f749fa68e8 100644
@@ -129,7 +129,9 @@ fn expecting(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
     delegate!(visit_u32, u32);
     delegate!(visit_u64, u64);
 
+    #[cfg(not(no_fp_fmt_parse))]
     delegate!(visit_f32, f32);
+    #[cfg(not(no_fp_fmt_parse))]
     delegate!(visit_f64, f64);
 
     delegate!(visit_char, char);
rust/serde_cbor/value/ser.rs +2 -0
index e52987778288..b4ff47c2e106 100644
@@ -101,11 +101,13 @@ fn serialize_u64(self, value: u64) -> Result<Value, Error> {
     }
 
     #[inline]
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f32(self, value: f32) -> Result<Value, Error> {
         self.serialize_f64(f64::from(value))
     }
 
     #[inline]
+    #[cfg(not(no_fp_fmt_parse))]
     fn serialize_f64(self, value: f64) -> Result<Value, Error> {
         Ok(Value::Float(value))
     }
Signature · 1 line
2.40.1
#60 lore raw

PATCH 41/80 rust: Kbuild: enable serde_cbor

In reply to Ariel Miculas
---
 Makefile               |  2 ++
 rust/Makefile          | 23 ++++++++++++++++++++---
 scripts/Makefile.build |  2 +-
 3 files changed, 23 insertions(+), 4 deletions(-)
Makefile +2 -0
index 3b7f37977f04..0abe8aaee974 100644
@@ -1853,6 +1853,7 @@ rustfmt:
 		-o -path $(abs_srctree)/rust/syn -prune \
 		-o -path $(abs_srctree)/rust/serde -prune \
 		-o -path $(abs_srctree)/rust/serde_derive -prune \
+		-o -path $(abs_srctree)/rust/serde_cbor -prune \
 		-o -path $(abs_objtree)/rust/test -prune \
 		| grep -Fv $(abs_srctree)/rust/alloc \
 		| grep -Fv $(abs_srctree)/rust/proc-macro2 \
@@ -1860,6 +1861,7 @@ rustfmt:
 		| grep -Fv $(abs_srctree)/rust/syn \
 		| grep -Fv $(abs_srctree)/rust/serde \
 		| grep -Fv $(abs_srctree)/rust/serde_derive \
+		| grep -Fv $(abs_srctree)/rust/serde_cbor \
 		| grep -Fv $(abs_objtree)/rust/test \
 		| grep -Fv generated \
 		| xargs $(RUSTFMT) $(rustfmt_flags)
rust/Makefile +20 -3
index 37952e93ff06..129d5a6dd07e 100644
@@ -13,7 +13,7 @@ always-$(CONFIG_RUST) += libserde_derive.so libmacros.so
 no-clean-files += libserde_derive.so libmacros.so
 
 always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
-obj-$(CONFIG_RUST) += alloc.o bindings.o serde.o kernel.o
+obj-$(CONFIG_RUST) += alloc.o bindings.o serde.o serde_cbor.o kernel.o
 always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
     exports_kernel_generated.h
 
@@ -122,6 +122,17 @@ serde-flags := \
     -Amissing_docs \
     --cfg no_fp_fmt_parse
 
+serde_cbor-skip_flags := \
+    --edition=2021 \
+    -Drust_2018_idioms \
+    -Dunreachable_pub
+
+serde_cbor-flags := \
+    --edition=2018 \
+    -Amissing_docs \
+    --cfg no_fp_fmt_parse \
+    --extern serde
+
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
       cmd_rustdoc = \
 	OBJTREE=$(abspath $(objtree)) \
@@ -568,12 +579,18 @@ $(obj)/serde.o: private rustc_target_flags = $(serde-flags)
 $(obj)/serde.o: $(src)/serde/lib.rs $(obj)/compiler_builtins.o FORCE
 	$(call if_changed_dep,rustc_library)
 
+$(obj)/serde_cbor.o: private skip_clippy = 1
+$(obj)/serde_cbor.o: private skip_flags = $(serde_cbor-skip_flags)
+$(obj)/serde_cbor.o: private rustc_target_flags = $(serde_cbor-flags)
+$(obj)/serde_cbor.o: $(src)/serde_cbor/lib.rs $(obj)/compiler_builtins.o FORCE
+	$(call if_changed_dep,rustc_library)
+
 $(obj)/kernel.o: private rustc_target_flags = --extern alloc \
     --extern build_error --extern macros --extern bindings --extern uapi \
-    --extern serde --extern serde_derive
+    --extern serde --extern serde_derive --extern serde_cbor
 $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
     $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o \
-    $(obj)/serde.o $(obj)/libserde_derive.so FORCE
+    $(obj)/serde.o $(obj)/libserde_derive.so $(obj)/serde_cbor.o FORCE
 	$(call if_changed_dep,rustc_library)
 
 endif # CONFIG_RUST
scripts/Makefile.build +1 -1
index 3fa298bb7c31..758b327fd54d 100644
@@ -284,7 +284,7 @@ rust_common_cmd = \
 	-Zallow-features=$(rust_allowed_features) \
 	-Zcrate-attr=no_std \
 	-Zcrate-attr='feature($(rust_allowed_features))' \
-	--extern alloc --extern kernel --extern serde --extern serde_derive \
+	--extern alloc --extern kernel --extern serde --extern serde_derive --extern serde_cbor \
 	--crate-type rlib -L $(objtree)/rust/ \
 	--crate-name $(basename $(notdir $@)) \
 	--emit=dep-info=$(depfile)
Signature · 1 line
2.40.1
#61 lore raw

Re: [PATCH 41/80] rust: Kbuild: enable serde_cbor

Quoted 2 lines · Ariel Miculas

Regarding serde_cbor: the repository upstream says it was archived almost two years ago.

While it says it is widely used, it also says it is unmaintained and recommends other two crates for new users.

Since this is an RFC, this is fine for the moment to have something that works, but if eventually the kernel decides to merge puzzlefs and its dependencies, we will need to consider who will maintain this, or whether another crate would be a better fit (or a kernel-made one), etc.

Cheers,
Miguel

#62 lore raw

PATCH 42/80 samples: rust: add cbor serialize/deserialize example

In reply to Ariel Miculas
---
 samples/rust/rust_serde.rs | 56 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
samples/rust/rust_serde.rs +56 -0
index 3d4957dd0822..0578f0fa137c 100644
@@ -7,6 +7,8 @@
 
 use kernel::prelude::*;
 use serde_derive::{Deserialize, Serialize};
+use serde_cbor::ser::SliceWrite;
+use serde::Serialize;
 
 module! {
     type: RustSerde,
@@ -38,6 +40,57 @@ struct S {
     d: (),
 }
 
+#[derive(Debug, PartialEq, Serialize, Deserialize)]
+struct User {
+    user_id: u32,
+    password_hash: [u8; 4],
+}
+
+fn cbor_serialize() -> Result<(), serde_cbor::Error> {
+    let mut buf = [0u8; 100];
+    let writer = SliceWrite::new(&mut buf[..]);
+    let mut ser = serde_cbor::Serializer::new(writer);
+    let user = User {
+        user_id: 42,
+        password_hash: [1, 2, 3, 4],
+    };
+    user.serialize(&mut ser)?;
+    let writer = ser.into_inner();
+    let size = writer.bytes_written();
+    let expected = [
+        0xa2, 0x67, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x2a, 0x6d,
+        0x70, 0x61, 0x73, 0x73, 0x77, 0x6f, 0x72, 0x64, 0x5f, 0x68, 0x61, 0x73,
+        0x68, 0x84, 0x1, 0x2, 0x3, 0x4
+    ];
+    assert_eq!(&buf[..size], expected);
+
+    crate::pr_info!("cbor serialized = {:?}", buf);
+
+    Ok(())
+}
+
+fn cbor_deserialize() -> Result<(), serde_cbor::Error> {
+    let value = [
+        0xa2, 0x67, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x2a, 0x6d,
+        0x70, 0x61, 0x73, 0x73, 0x77, 0x6f, 0x72, 0x64, 0x5f, 0x68, 0x61, 0x73,
+        0x68, 0x84, 0x1, 0x2, 0x3, 0x4
+    ];
+
+    // from_slice_with_scratch will not alter input data, use it whenever you
+    // borrow from somewhere else.
+    // You will have to size your scratch according to the input data you
+    // expect.
+    let mut scratch = [0u8; 32];
+    let user: User = serde_cbor::de::from_slice_with_scratch(&value[..], &mut scratch)?;
+    assert_eq!(user, User {
+        user_id: 42,
+        password_hash: [1, 2, 3, 4],
+    });
+
+    crate::pr_info!("cbor deserialized = {:?}", user);
+    Ok(())
+}
+
 impl kernel::Module for RustSerde {
     fn init(_module: &'static ThisModule) -> Result<Self> {
         pr_info!("Rust serde sample (init)\n");
@@ -62,6 +115,9 @@ fn init(_module: &'static ThisModule) -> Result<Self> {
         let deserialized: S = local_data_format::from_bytes(&serialized).unwrap();
         crate::pr_info!("deserialized (local) = {:?}", deserialized);
 
+        cbor_serialize().unwrap();
+        cbor_deserialize().unwrap();
+
         Ok(RustSerde)
     }
 }
Signature · 1 line
2.40.1
#63 lore raw

PATCH 43/80 rust: serde_cbor: add support for serde_cbor's from_slice method by using a custom alloc_kernel feature

In reply to Ariel Miculas
---
 rust/Makefile                 |  2 ++
 rust/kernel/test_serde/de.rs  |  7 ++-----
 rust/kernel/test_serde/ser.rs |  5 +----
 rust/serde_cbor/de.rs         |  6 +++---
 rust/serde_cbor/lib.rs        |  2 +-
 rust/serde_cbor/read.rs       | 18 +++++++++++++-----
 samples/rust/rust_serde.rs    | 32 ++++++++++++++------------------
 7 files changed, 36 insertions(+), 36 deletions(-)
rust/Makefile +2 -0
index 129d5a6dd07e..0b67547d8007 100644
@@ -131,6 +131,8 @@ serde_cbor-flags := \
     --edition=2018 \
     -Amissing_docs \
     --cfg no_fp_fmt_parse \
+    --cfg 'feature="kernel_alloc"' \
+    --extern alloc \
     --extern serde
 
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
rust/kernel/test_serde/de.rs +2 -5
index 84c98d1c1c66..68f4da2ace23 100644
@@ -434,9 +434,6 @@ struct Test {
     }
 
     let j = &[2, 0, 1, 0, 3];
-    let expected = Test {
-        a: (),
-        b: false,
-    };
+    let expected = Test { a: (), b: false };
     assert_eq!(expected, from_bytes(j).unwrap());
-}
\ No newline at end of file
+}
rust/kernel/test_serde/ser.rs +1 -4
index 56abe7095a5f..56439b81d4e3 100644
@@ -454,10 +454,7 @@ struct Test {
         b: bool,
     }
 
-    let test = Test {
-        a: (),
-        b: false,
-    };
+    let test = Test { a: (), b: false };
 
     let mut expected = Vec::new();
     expected.try_push(2).unwrap();
rust/serde_cbor/de.rs +3 -3
index 534f9d53aa3b..ab7572f24002 100644
@@ -21,7 +21,7 @@
 #[cfg(feature = "std")]
 pub use crate::read::IoRead;
 use crate::read::Offset;
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 pub use crate::read::SliceRead;
 pub use crate::read::{MutSliceRead, Read, SliceReadFixed};
 #[cfg(feature = "tags")]
@@ -47,7 +47,7 @@
 /// let value: &str = de::from_slice(&v[..]).unwrap();
 /// assert_eq!(value, "foobar");
 /// ```
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 pub fn from_slice<'a, T>(slice: &'a [u8]) -> Result<T>
 where
     T: de::Deserialize<'a>,
@@ -150,7 +150,7 @@ pub fn from_reader(reader: R) -> Deserializer<IoRead<R>> {
     }
 }
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 impl<'a> Deserializer<SliceRead<'a>> {
     /// Constructs a `Deserializer` which reads from a slice.
     ///
rust/serde_cbor/lib.rs +1 -1
index cf406a439340..c9751972dfc2 100644
@@ -324,7 +324,7 @@
 #[cfg(all(not(feature = "std"), test))]
 extern crate std;
 
-#[cfg(feature = "alloc")]
+#[cfg(any(feature = "alloc", feature = "kernel_alloc"))]
 extern crate alloc;
 
 pub mod de;
rust/serde_cbor/read.rs +13 -5
index 6c17282d2c5d..35ef08b84f31 100644
@@ -2,6 +2,8 @@
 
 #[cfg(feature = "alloc")]
 use alloc::{vec, vec::Vec};
+#[cfg(feature = "kernel_alloc")]
+use alloc::{vec::Vec};
 #[cfg(feature = "std")]
 use core::cmp;
 use core::mem;
@@ -298,7 +300,7 @@ fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
 }
 
 /// A CBOR input source that reads from a slice of bytes.
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 #[derive(Debug)]
 pub struct SliceRead<'a> {
     slice: &'a [u8],
@@ -306,13 +308,16 @@ pub struct SliceRead<'a> {
     index: usize,
 }
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 impl<'a> SliceRead<'a> {
     /// Creates a CBOR input source to read from a slice of bytes.
     pub fn new(slice: &'a [u8]) -> SliceRead<'a> {
         SliceRead {
             slice,
+            #[cfg(not(feature = "kernel_alloc"))]
             scratch: vec![],
+            #[cfg(feature = "kernel_alloc")]
+            scratch: Vec::new(),
             index: 0,
         }
     }
@@ -328,7 +333,7 @@ fn end(&self, n: usize) -> Result<usize> {
     }
 }
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 impl<'a> Offset for SliceRead<'a> {
     #[inline]
     fn byte_offset(&self) -> usize {
@@ -337,12 +342,12 @@ fn byte_offset(&self) -> usize {
 }
 
 #[cfg(all(
-    any(feature = "std", feature = "alloc"),
+    any(feature = "std", feature = "alloc", feature = "kernel_alloc"),
     not(feature = "unsealed_read_write")
 ))]
 impl<'a> private::Sealed for SliceRead<'a> {}
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 impl<'a> Read<'a> for SliceRead<'a> {
     #[inline]
     fn next(&mut self) -> Result<Option<u8>> {
@@ -371,7 +376,10 @@ fn clear_buffer(&mut self) {
     fn read_to_buffer(&mut self, n: usize) -> Result<()> {
         let end = self.end(n)?;
         let slice = &self.slice[self.index..end];
+        #[cfg(not(feature = "kernel_alloc"))]
         self.scratch.extend_from_slice(slice);
+        #[cfg(feature = "kernel_alloc")]
+        self.scratch.try_extend_from_slice(slice).unwrap();
         self.index = end;
 
         Ok(())
samples/rust/rust_serde.rs +14 -18
index 0578f0fa137c..29286e763ca3 100644
@@ -6,9 +6,9 @@
 //! one here ("local"). Then it uses both on a type that uses `serve_derive`.
 
 use kernel::prelude::*;
-use serde_derive::{Deserialize, Serialize};
-use serde_cbor::ser::SliceWrite;
 use serde::Serialize;
+use serde_cbor::ser::SliceWrite;
+use serde_derive::{Deserialize, Serialize};
 
 module! {
     type: RustSerde,
@@ -58,9 +58,8 @@ fn cbor_serialize() -> Result<(), serde_cbor::Error> {
     let writer = ser.into_inner();
     let size = writer.bytes_written();
     let expected = [
-        0xa2, 0x67, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x2a, 0x6d,
-        0x70, 0x61, 0x73, 0x73, 0x77, 0x6f, 0x72, 0x64, 0x5f, 0x68, 0x61, 0x73,
-        0x68, 0x84, 0x1, 0x2, 0x3, 0x4
+        0xa2, 0x67, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x2a, 0x6d, 0x70, 0x61, 0x73,
+        0x73, 0x77, 0x6f, 0x72, 0x64, 0x5f, 0x68, 0x61, 0x73, 0x68, 0x84, 0x1, 0x2, 0x3, 0x4,
     ];
     assert_eq!(&buf[..size], expected);
 
@@ -71,21 +70,18 @@ fn cbor_serialize() -> Result<(), serde_cbor::Error> {
 
 fn cbor_deserialize() -> Result<(), serde_cbor::Error> {
     let value = [
-        0xa2, 0x67, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x2a, 0x6d,
-        0x70, 0x61, 0x73, 0x73, 0x77, 0x6f, 0x72, 0x64, 0x5f, 0x68, 0x61, 0x73,
-        0x68, 0x84, 0x1, 0x2, 0x3, 0x4
+        0xa2, 0x67, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x69, 0x64, 0x18, 0x2a, 0x6d, 0x70, 0x61, 0x73,
+        0x73, 0x77, 0x6f, 0x72, 0x64, 0x5f, 0x68, 0x61, 0x73, 0x68, 0x84, 0x1, 0x2, 0x3, 0x4,
     ];
 
-    // from_slice_with_scratch will not alter input data, use it whenever you
-    // borrow from somewhere else.
-    // You will have to size your scratch according to the input data you
-    // expect.
-    let mut scratch = [0u8; 32];
-    let user: User = serde_cbor::de::from_slice_with_scratch(&value[..], &mut scratch)?;
-    assert_eq!(user, User {
-        user_id: 42,
-        password_hash: [1, 2, 3, 4],
-    });
+    let user: User = serde_cbor::de::from_slice(&value[..])?;
+    assert_eq!(
+        user,
+        User {
+            user_id: 42,
+            password_hash: [1, 2, 3, 4],
+        }
+    );
 
     crate::pr_info!("cbor deserialized = {:?}", user);
     Ok(())
Signature · 1 line
2.40.1
#64 lore raw

Re: [PATCH 43/80] rust: serde_cbor: add support for serde_cbor's from_slice method by using a custom alloc_kernel feature

rust/kernel/test_serde/de.rs struct Test { · line 437

Quoted 7 lines · Ariel Miculas
     let j = &[2, 0, 1, 0, 3];
rust/kernel/test_serde/de.rs +1 -4
-    let expected = Test {
-        a: (),
-        b: false,
-    };
+    let expected = Test { a: (), b: false };

It seems rustfmt was run within this commit, which makes it harder to spot the changes. Please apply it on the commits that add the code itself instead.

rust/serde_cbor/read.rs pub struct SliceRead<'a> { · line 320

Quoted 9 lines
 impl<'a> SliceRead<'a> {
     /// Creates a CBOR input source to read from a slice of bytes.
     pub fn new(slice: &'a [u8]) -> SliceRead<'a> {
         SliceRead {
             slice,
rust/serde_cbor/read.rs +3 -0
+            #[cfg(not(feature = "kernel_alloc"))]
             scratch: vec![],
+            #[cfg(feature = "kernel_alloc")]
+            scratch: Vec::new(),

Can this be simply replaced, i.e. without the gate, or is there a reason for it?

Either that, or we go ahead and make vec! work, but originally I thought it would be more confusing, see
https://github.com/rust-lang/rust/commit/8cec88ba76e1d6edc98f30101c40f9247c754898.

rust/serde_cbor/read.rs fn clear_buffer(&mut self) { · line 382

Quoted 8 lines
rust/serde_cbor/read.rs +3 -0
@@ -371,7 +376,10 @@ fn clear_buffer(&mut self) {
     fn read_to_buffer(&mut self, n: usize) -> Result<()> {
         let end = self.end(n)?;
         let slice = &self.slice[self.index..end];
+        #[cfg(not(feature = "kernel_alloc"))]
         self.scratch.extend_from_slice(slice);
+        #[cfg(feature = "kernel_alloc")]
+        self.scratch.try_extend_from_slice(slice).unwrap();

Do we know this allocation would not fail? i.e. we need to be extra careful on unwrap()s within the kernel. Especially for facilities that may have more than one caller/user like libraries.

Cheers,
Miguel

#65 lore raw

PATCH 44/80 rust: serde: add support for deserializing Vec with kernel_alloc feature

In reply to Ariel Miculas
---
 rust/Makefile                   |  4 +++-
 rust/serde/de/impls.rs          | 19 +++++++++++++++++--
 rust/serde/lib.rs               |  2 +-
 rust/serde/private/size_hint.rs |  2 +-
 4 files changed, 22 insertions(+), 5 deletions(-)
rust/Makefile +3 -1
index 0b67547d8007..c9feb8e1afe1 100644
@@ -120,7 +120,9 @@ serde-skip_flags := \
 
 serde-flags := \
     -Amissing_docs \
-    --cfg no_fp_fmt_parse
+    --cfg no_fp_fmt_parse \
+    --cfg 'feature="kernel_alloc"' \
+    --extern alloc
 
 serde_cbor-skip_flags := \
     --edition=2021 \
rust/serde/de/impls.rs +17 -2
index 9377f5eee650..6b860b2d9450 100644
@@ -11,9 +11,12 @@
 
 use seed::InPlaceSeed;
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 use __private::size_hint;
 
+#[cfg(feature = "kernel_alloc")]
+use alloc::vec::Vec;
+
 ////////////////////////////////////////////////////////////////////////////////
 
 struct UnitVisitor;
@@ -1012,7 +1015,7 @@ fn nop_reserve<T>(_seq: T, _n: usize) {}
 
 ////////////////////////////////////////////////////////////////////////////////
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 impl<'de, T> Deserialize<'de> for Vec<T>
 where
     T: Deserialize<'de>,
@@ -1039,10 +1042,16 @@ fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
             where
                 A: SeqAccess<'de>,
             {
+                #[cfg(not(feature = "kernel_alloc"))]
                 let mut values = Vec::with_capacity(size_hint::cautious(seq.size_hint()));
+                #[cfg(feature = "kernel_alloc")]
+                let mut values = Vec::try_with_capacity(size_hint::cautious(seq.size_hint())).unwrap();
 
                 while let Some(value) = try!(seq.next_element()) {
+                    #[cfg(not(feature = "kernel_alloc"))]
                     values.push(value);
+                    #[cfg(feature = "kernel_alloc")]
+                    values.try_push(value).unwrap();
                 }
 
                 Ok(values)
@@ -1077,7 +1086,10 @@ fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
             {
                 let hint = size_hint::cautious(seq.size_hint());
                 if let Some(additional) = hint.checked_sub(self.0.len()) {
+                    #[cfg(not(feature = "kernel_alloc"))]
                     self.0.reserve(additional);
+                    #[cfg(feature = "kernel_alloc")]
+                    self.0.try_reserve(additional).unwrap();
                 }
 
                 for i in 0..self.0.len() {
@@ -1092,7 +1104,10 @@ fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
                 }
 
                 while let Some(value) = try!(seq.next_element()) {
+                    #[cfg(not(feature = "kernel_alloc"))]
                     self.0.push(value);
+                    #[cfg(feature = "kernel_alloc")]
+                    self.0.try_push(value).unwrap();
                 }
 
                 Ok(())
rust/serde/lib.rs +1 -1
index c3694fef3321..bfc845e98075 100644
@@ -156,7 +156,7 @@
 
 ////////////////////////////////////////////////////////////////////////////////
 
-#[cfg(feature = "alloc")]
+#[cfg(any(feature = "alloc", feature = "kernel_alloc"))]
 extern crate alloc;
 
 /// A facade around all the types we need from the `std`, `core`, and `alloc`
rust/serde/private/size_hint.rs +1 -1
index f050fca59deb..ac3c9394e7d3 100644
@@ -9,7 +9,7 @@ pub fn from_bounds<I>(iter: &I) -> Option<usize>
     helper(iter.size_hint())
 }
 
-#[cfg(any(feature = "std", feature = "alloc"))]
+#[cfg(any(feature = "std", feature = "alloc", feature = "kernel_alloc"))]
 #[inline]
 pub fn cautious(hint: Option<usize>) -> usize {
     cmp::min(hint.unwrap_or(0), 4096)
Signature · 1 line
2.40.1
#66 lore raw

Re: [PATCH 44/80] rust: serde: add support for deserializing Vec with kernel_alloc feature

rust/serde/de/impls.rs fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error> · line 1048

Quoted 5 lines · Ariel Miculas
rust/serde/de/impls.rs +3 -0
+                #[cfg(not(feature = "kernel_alloc"))]
                 let mut values = Vec::with_capacity(size_hint::cautious(seq.size_hint()));
+                #[cfg(feature = "kernel_alloc")]
+                let mut values = Vec::try_with_capacity(size_hint::cautious(seq.size_hint())).unwrap();

Same note on these unwrap()s as in the other patch.

Cheers,
Miguel

#67 lore raw

PATCH 45/80 rust: file: Replace UnsafeCell with Opaque for File

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
rust/kernel/file.rs +3 -2
index c090a19da9b2..086c62000533 100644
@@ -17,10 +17,11 @@
     types::ARef,
     types::AlwaysRefCounted,
     types::ForeignOwnable,
+    types::Opaque,
     user_ptr::{UserSlicePtr, UserSlicePtrReader, UserSlicePtrWriter},
 };
 use core::convert::{TryFrom, TryInto};
-use core::{cell::UnsafeCell, marker, mem, ptr};
+use core::{marker, mem, ptr};
 use macros::vtable;
 
 /// Flags associated with a [`File`].
@@ -113,7 +114,7 @@ pub mod flags {
 /// Instances of this type are always ref-counted, that is, a call to `get_file` ensures that the
 /// allocation remains valid at least until the matching call to `fput`.
 #[repr(transparent)]
-pub struct File(pub(crate) UnsafeCell<bindings::file>);
+pub struct File(pub(crate) Opaque<bindings::file>);
 
 // TODO: Accessing fields of `struct file` through the pointer is UB because other threads may be
 // writing to them. However, this is how the C code currently operates: naked reads and writes to
Signature · 1 line
2.40.1
#68 lore raw

PATCH 46/80 rust: kernel: implement fmt::Debug for CString

In reply to Ariel Miculas
---
 rust/kernel/str.rs | 6 ++++++
 1 file changed, 6 insertions(+)
rust/kernel/str.rs +6 -0
index c9dd3bf59e34..bd535522323a 100644
@@ -236,6 +236,12 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
     }
 }
 
+impl fmt::Debug for CString {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        fmt::Debug::fmt(&**self, f)
+    }
+}
+
 impl fmt::Debug for CStr {
     /// Formats printable ASCII characters with a double quote on either end, escaping the rest.
     ///
Signature · 1 line
2.40.1
#69 lore raw

PATCH 47/80 samples: puzzlefs: rename RustFs to PuzzleFs

In reply to Ariel Miculas
---
 samples/rust/puzzlefs.rs | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
samples/rust/puzzlefs.rs +5 -5
index 51aa41533a35..e7ce5078bc99 100644
@@ -7,16 +7,16 @@
 use kernel::{c_str, file, fs, io_buffer::IoBufferWriter};
 
 module_fs! {
-    type: RustFs,
+    type: PuzzleFs,
     name: "puzzlefs",
     author: "Ariel Miculas",
     license: "GPL",
 }
 
-struct RustFs;
+struct PuzzleFs;
 
 #[vtable]
-impl fs::Context<Self> for RustFs {
+impl fs::Context<Self> for PuzzleFs {
     type Data = ();
 
     kernel::define_fs_params! {(),
@@ -39,7 +39,7 @@ fn try_new() -> Result {
     }
 }
 
-impl fs::Type for RustFs {
+impl fs::Type for PuzzleFs {
     type Context = Self;
     type INodeData = &'static [u8];
     const SUPER_TYPE: fs::Super = fs::Super::Independent;
@@ -97,7 +97,7 @@ fn read(
         offset: u64,
     ) -> Result<usize> {
         file::read_from_slice(
-            file.inode::<RustFs>().ok_or(EINVAL)?.fs_data(),
+            file.inode::<PuzzleFs>().ok_or(EINVAL)?.fs_data(),
             writer,
             offset,
         )
Signature · 1 line
2.40.1
#70 lore raw

PATCH 48/80 samples: puzzlefs: add basic deserializing support for the puzzlefs metadata

In reply to Ariel Miculas
---
 samples/rust/puzzle.rs                    |   2 +
 samples/rust/puzzle/error.rs              |  32 +++
 samples/rust/puzzle/types.rs              | 304 ++++++++++++++++++++++
 samples/rust/puzzle/types/cbor_helpers.rs |   9 +
 samples/rust/puzzlefs.rs                  |   2 +
 5 files changed, 349 insertions(+)
 create mode 100644 samples/rust/puzzle.rs
 create mode 100644 samples/rust/puzzle/error.rs
 create mode 100644 samples/rust/puzzle/types.rs
 create mode 100644 samples/rust/puzzle/types/cbor_helpers.rs
samples/rust/puzzle.rs new file +2 -0
new file mode 100644
index 000000000000..4d558561974d
@@ -0,0 +1,2 @@
+pub(crate) mod error;
+mod types;
samples/rust/puzzle/error.rs new file +32 -0
new file mode 100644
index 000000000000..3427c5d2f7e3
@@ -0,0 +1,32 @@
+use core::fmt::{self, Display};
+
+// TODO use String in error types (when it's available from the kernel)
+
+pub(crate) enum WireFormatError {
+    LocalRefError,
+    SeekOtherError,
+    ValueMissing,
+    CBORError(serde_cbor::Error),
+}
+
+impl Display for WireFormatError {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            WireFormatError::LocalRefError => f.write_str("cannot turn local ref into a digest"),
+            WireFormatError::SeekOtherError => f.write_str("cannot seek to other blob"),
+            WireFormatError::ValueMissing => f.write_str("no value present"),
+            WireFormatError::CBORError(_) => f.write_str("CBOR error"),
+        }
+    }
+}
+
+pub(crate) type Result<T> = kernel::error::Result<T, WireFormatError>;
+
+// TODO figure out how to use thiserror
+#[allow(unused_qualifications)]
+impl core::convert::From<serde_cbor::Error> for WireFormatError {
+    #[allow(deprecated)]
+    fn from(source: serde_cbor::Error) -> Self {
+        WireFormatError::CBORError(source)
+    }
+}
samples/rust/puzzle/types.rs new file +304 -0

304 lines not on this page · raw message

samples/rust/puzzle/types/cbor_helpers.rs new file +9 -0
new file mode 100644
index 000000000000..ae2aa4609428
@@ -0,0 +1,9 @@
+pub(crate) const fn cbor_size_of_list_header(size: usize) -> usize {
+    match size {
+        0..=23 => 1,
+        24..=255 => 2,
+        256..=65535 => 3,
+        65536..=4294967295 => 4,
+        _ => 8,
+    }
+}
samples/rust/puzzlefs.rs +2 -0
index e7ce5078bc99..b149af4e66ce 100644
@@ -6,6 +6,8 @@
 use kernel::prelude::*;
 use kernel::{c_str, file, fs, io_buffer::IoBufferWriter};
 
+mod puzzle;
+
 module_fs! {
     type: PuzzleFs,
     name: "puzzlefs",
Signature · 1 line
2.40.1
#71 lore raw

PATCH 49/80 rust: file: present the filesystem context to the open function

In reply to Ariel Miculas

This is not a good implementation because it does not ensure that the type Operations::FSData in file.rs is the same as Type::Data in fs.rs. This means that we could store a Box in the field s_fs_info and retrieve an Arc from there, which is obviously wrong and it leads to UB.

---
 rust/kernel/file.rs      | 18 ++++++++++++++++--
 samples/rust/puzzlefs.rs | 21 ++++++++++++++++++---
 samples/rust/rust_fs.rs  |  2 +-
 3 files changed, 35 insertions(+), 6 deletions(-)
rust/kernel/file.rs +16 -2
index 086c62000533..0062d8b17990 100644
@@ -330,9 +330,14 @@ impl<A: OpenAdapter<T::OpenData>, T: Operations> OperationsVtable<A, T> {
             // `fileref` never outlives this function, so it is guaranteed to be
             // valid.
             let fileref = unsafe { File::from_ptr(file) };
+
+            // SAFETY: into_foreign was called in fs::NewSuperBlock<..., NeedsInit>::init and
+            // it is valid until from_foreign will be called in fs::Tables::free_callback
+            let fs_info = unsafe { T::FSData::borrow((*(*inode).i_sb).s_fs_info) };
+
             // SAFETY: `arg` was previously returned by `A::convert` and must
             // be a valid non-null pointer.
-            let ptr = T::open(unsafe { &*arg }, fileref)?.into_foreign();
+            let ptr = T::open(fs_info, unsafe { &*arg }, fileref)?.into_foreign();
             // SAFETY: The C contract guarantees that `private_data` is available
             // for implementers of the file operations (no other C code accesses
             // it), so we know that there are no concurrent threads/CPUs accessing
@@ -793,10 +798,19 @@ pub trait Operations {
     /// The type of the context data passed to [`Operations::open`].
     type OpenData: Sync = ();
 
+    /// Data associated with each file system instance.
+    // SAFETY: this is not safe because we don't enforce the same type as fs::Type::Data, so it's
+    // possible to store a Box in s_fs_info and retrieve an Arc when open is called, leading to UB
+    type FSData: ForeignOwnable + Send + Sync = ();
+
     /// Creates a new instance of this file.
     ///
     /// Corresponds to the `open` function pointer in `struct file_operations`.
-    fn open(context: &Self::OpenData, file: &File) -> Result<Self::Data>;
+    fn open(
+        fs_info: <Self::FSData as ForeignOwnable>::Borrowed<'_>,
+        context: &Self::OpenData,
+        file: &File,
+    ) -> Result<Self::Data>;
 
     /// Cleans up after the last reference to the file goes away.
     ///
samples/rust/puzzlefs.rs +18 -3
index b149af4e66ce..e454bce7dbc6 100644
@@ -4,7 +4,7 @@
 
 use kernel::module_fs;
 use kernel::prelude::*;
-use kernel::{c_str, file, fs, io_buffer::IoBufferWriter};
+use kernel::{c_str, file, fs, io_buffer::IoBufferWriter, fmt, str::CString};
 
 mod puzzle;
 
@@ -17,6 +17,11 @@
 
 struct PuzzleFs;
 
+#[derive(Debug)]
+struct PuzzlefsInfo {
+    base_path: CString,
+}
+
 #[vtable]
 impl fs::Context<Self> for PuzzleFs {
     type Data = ();
@@ -44,14 +49,17 @@ fn try_new() -> Result {
 impl fs::Type for PuzzleFs {
     type Context = Self;
     type INodeData = &'static [u8];
+    type Data = Box<PuzzlefsInfo>;
     const SUPER_TYPE: fs::Super = fs::Super::Independent;
     const NAME: &'static CStr = c_str!("puzzlefs");
     const FLAGS: i32 = fs::flags::USERNS_MOUNT;
     const DCACHE_BASED: bool = true;
 
     fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBlock<Self>> {
+        let base_path = CString::try_from_fmt(fmt!("hello world"))?;
+        pr_info!("base_path {:?}\n", base_path);
         let sb = sb.init(
-            (),
+            Box::try_new(PuzzlefsInfo { base_path })?,
             &fs::SuperParams {
                 magic: 0x72757374,
                 ..fs::SuperParams::DEFAULT
@@ -87,8 +95,15 @@ fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBl
 #[vtable]
 impl file::Operations for FsFile {
     type OpenData = &'static [u8];
+    type FSData = Box<PuzzlefsInfo>;
+
+    fn open(
+        fs_info: &PuzzlefsInfo,
+        _context: &Self::OpenData,
+        _file: &file::File,
+    ) -> Result<Self::Data> {
+        pr_info!("got {:?}\n", fs_info);
 
-    fn open(_context: &Self::OpenData, _file: &file::File) -> Result<Self::Data> {
         Ok(())
     }
 
samples/rust/rust_fs.rs +1 -1
index 7527681ee024..36bca7da179f 100644
@@ -86,7 +86,7 @@ fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBl
 impl file::Operations for FsFile {
     type OpenData = &'static [u8];
 
-    fn open(_context: &Self::OpenData, _file: &file::File) -> Result<Self::Data> {
+    fn open(_fs_info: (), _context: &Self::OpenData, _file: &file::File) -> Result<Self::Data> {
         Ok(())
     }
 
Signature · 1 line
2.40.1
#72 lore raw

PATCH 50/80 rust: kernel: add an abstraction over vfsmount to allow cloning a new private mount

In reply to Ariel Miculas
---
 rust/bindings/bindings_helper.h |  1 +
 rust/kernel/lib.rs              |  1 +
 rust/kernel/mount.rs            | 66 +++++++++++++++++++++++++++++++++
 3 files changed, 68 insertions(+)
 create mode 100644 rust/kernel/mount.rs
rust/bindings/bindings_helper.h +1 -0
index 6182174663ba..3ea8e3f610ee 100644
@@ -17,6 +17,7 @@
 #include <linux/uio.h>
 #include <linux/uaccess.h>
 #include <linux/delay.h>
+#include <linux/namei.h>
 
 /* `bindgen` gets confused at certain things. */
 const gfp_t BINDINGS_GFP_KERNEL = GFP_KERNEL;
rust/kernel/lib.rs +1 -0
index a94fb784d576..5ad583ce6fb4 100644
@@ -46,6 +46,7 @@
 pub mod ioctl;
 pub mod iov_iter;
 pub mod mm;
+pub mod mount;
 pub mod pages;
 pub mod prelude;
 pub mod print;
rust/kernel/mount.rs new file +66 -0

66 lines not on this page · raw message

Signature · 1 line
2.40.1
#73 lore raw

PATCH 51/80 rust: file: add from_path, from_path_in_root_mnt and read_with_offset methods to File

In reply to Ariel Miculas
---
 rust/kernel/error.rs |  4 +--
 rust/kernel/file.rs  | 74 +++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 74 insertions(+), 4 deletions(-)
rust/kernel/error.rs +1 -3
index 231f7dfd8005..6fff9a4f0672 100644
@@ -234,9 +234,7 @@ pub fn to_result(err: core::ffi::c_int) -> Result {
 ///     }
 /// }
 /// ```
-// TODO: Remove `dead_code` marker once an in-kernel client is available.
-#[allow(dead_code)]
-pub(crate) fn from_err_ptr<T>(ptr: *mut T) -> Result<*mut T> {
+pub fn from_err_ptr<T>(ptr: *mut T) -> Result<*mut T> {
     // CAST: Casting a pointer to `*const core::ffi::c_void` is always valid.
     let const_ptr: *const core::ffi::c_void = ptr.cast();
     // SAFETY: The FFI function does not deref the pointer.
rust/kernel/file.rs +73 -1

90 lines not on this page · raw message

Signature · 1 line
2.40.1
#74 lore raw

PATCH 52/80 samples: puzzlefs: pass the Vfsmount structure from open to read and return the contents of the data file inside /home/puzzlefs_oci

In reply to Ariel Miculas
---
 samples/rust/puzzlefs.rs | 44 +++++++++++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 12 deletions(-)
samples/rust/puzzlefs.rs +32 -12
index e454bce7dbc6..db13a1d12466 100644
@@ -3,8 +3,14 @@
 //! Rust file system sample.
 
 use kernel::module_fs;
+use kernel::mount::Vfsmount;
 use kernel::prelude::*;
-use kernel::{c_str, file, fs, io_buffer::IoBufferWriter, fmt, str::CString};
+use kernel::{
+    c_str, file, fmt, fs,
+    io_buffer::IoBufferWriter,
+    str::CString,
+    sync::{Arc, ArcBorrow},
+};
 
 mod puzzle;
 
@@ -20,6 +26,7 @@
 #[derive(Debug)]
 struct PuzzlefsInfo {
     base_path: CString,
+    vfs_mount: Arc<Vfsmount>,
 }
 
 #[vtable]
@@ -58,8 +65,14 @@ impl fs::Type for PuzzleFs {
     fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBlock<Self>> {
         let base_path = CString::try_from_fmt(fmt!("hello world"))?;
         pr_info!("base_path {:?}\n", base_path);
+        let vfs_mount = Vfsmount::new_private_mount(c_str!("/home/puzzlefs_oci"))?;
+        pr_info!("vfs_mount {:?}\n", vfs_mount);
+
         let sb = sb.init(
-            Box::try_new(PuzzlefsInfo { base_path })?,
+            Box::try_new(PuzzlefsInfo {
+                base_path,
+                vfs_mount: Arc::try_new(vfs_mount)?,
+            })?,
             &fs::SuperParams {
                 magic: 0x72757374,
                 ..fs::SuperParams::DEFAULT
@@ -94,29 +107,36 @@ fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBl
 
 #[vtable]
 impl file::Operations for FsFile {
+    // must be the same as INodeData
     type OpenData = &'static [u8];
     type FSData = Box<PuzzlefsInfo>;
+    // this is an Arc because Data must be ForeignOwnable and the only implementors of it are Box,
+    // Arc and (); we cannot pass a reference to read, so we share Vfsmount using and Arc
+    type Data = Arc<Vfsmount>;
 
     fn open(
         fs_info: &PuzzlefsInfo,
         _context: &Self::OpenData,
         _file: &file::File,
     ) -> Result<Self::Data> {
-        pr_info!("got {:?}\n", fs_info);
-
-        Ok(())
+        Ok(fs_info.vfs_mount.clone())
     }
 
     fn read(
-        _data: (),
-        file: &file::File,
+        data: ArcBorrow<'_, Vfsmount>,
+        _file: &file::File,
         writer: &mut impl IoBufferWriter,
         offset: u64,
     ) -> Result<usize> {
-        file::read_from_slice(
-            file.inode::<PuzzleFs>().ok_or(EINVAL)?.fs_data(),
-            writer,
-            offset,
-        )
+        let mut buf = Vec::try_with_capacity(writer.len())?;
+        buf.try_resize(writer.len(), 0)?;
+        let file = file::File::from_path_in_root_mnt(
+            &data,
+            c_str!("data"),
+            file::flags::O_RDONLY.try_into().unwrap(),
+            0,
+        )?;
+        let nr_bytes_read = file.read_with_offset(&mut buf[..], offset)?;
+        file::read_from_slice(&buf[..nr_bytes_read], writer, 0)
     }
 }
Signature · 1 line
2.40.1
#75 lore raw

PATCH 53/80 rust: file: move from_path, from_path_in_root_mnt and read_with_offset methods to a RegularFile newtype

In reply to Ariel Miculas
---
 rust/kernel/file.rs      | 144 ++++++++++++++++++++-------------------
 samples/rust/puzzlefs.rs |   2 +-
 2 files changed, 75 insertions(+), 71 deletions(-)
rust/kernel/file.rs +74 -70
index a4926b316573..0d2771027150 100644
@@ -133,76 +133,6 @@ pub fn from_fd(fd: u32) -> Result<ARef<Self>> {
         Ok(unsafe { ARef::from_raw(ptr.cast()) })
     }
 
-    /// Constructs a new [`struct file`] wrapper from a path.
-    pub fn from_path(filename: &CStr, flags: i32, mode: u16) -> Result<ARef<Self>> {
-        let file_ptr = unsafe {
-            from_err_ptr(bindings::filp_open(
-                filename.as_ptr() as *const i8,
-                flags,
-                mode,
-            ))?
-        };
-        let file_ptr = ptr::NonNull::new(file_ptr).ok_or(ENOENT)?;
-
-        // SAFETY: `filp_open` initializes the refcount with 1
-        Ok(unsafe { ARef::from_raw(file_ptr.cast()) })
-    }
-
-    /// Constructs a new [`struct file`] wrapper from a path and a vfsmount.
-    pub fn from_path_in_root_mnt(
-        mount: &Vfsmount,
-        filename: &CStr,
-        flags: i32,
-        mode: u16,
-    ) -> Result<ARef<Self>> {
-        let file_ptr = unsafe {
-            let mnt = mount.get();
-            let raw_path = bindings::path {
-                mnt,
-                dentry: (*mnt).mnt_root,
-            };
-            from_err_ptr(bindings::file_open_root(
-                &raw_path,
-                filename.as_ptr() as *const i8,
-                flags,
-                mode,
-            ))?
-        };
-        let file_ptr = ptr::NonNull::new(file_ptr).ok_or(ENOENT)?;
-
-        // SAFETY: `file_open_root` increments the refcount before returning. (TODO does it?)
-        Ok(unsafe { ARef::from_raw(file_ptr.cast()) })
-    }
-
-    /// Read from the file into the specified buffer
-    pub fn read_with_offset(&self, buf: &mut [u8], offset: u64) -> Result<usize> {
-        Ok(unsafe {
-            // kernel_read_file expects a pointer to a "void *" buffer
-            let mut ptr_to_buf = buf.as_mut_ptr() as *mut core::ffi::c_void;
-            // Unless we give a non-null pointer to the file size:
-            // 1. we cannot give a non-zero value for the offset
-            // 2. we cannot have offset 0 and buffer_size > file_size
-            let mut file_size = 0;
-
-            // SAFETY: all the pointers to kernel_read_file are valid
-            let result = bindings::kernel_read_file(
-                self.0.get(),
-                offset.try_into()?,
-                &mut ptr_to_buf,
-                buf.len(),
-                &mut file_size,
-                bindings::kernel_read_file_id_READING_UNKNOWN,
-            );
-
-            // kernel_read_file returns the number of bytes read on success or negative on error.
-            if result < 0 {
-                return Err(Error::from_errno(result.try_into()?));
-            }
-
-            result.try_into()?
-        })
-    }
-
     /// Creates a reference to a [`File`] from a valid pointer.
     ///
     /// # Safety
@@ -273,6 +203,80 @@ unsafe fn dec_ref(obj: ptr::NonNull<Self>) {
     }
 }
 
+/// A newtype over file, specific to regular files
+pub struct RegularFile(ARef<File>);
+impl RegularFile {
+    /// Constructs a new [`struct file`] wrapper from a path.
+    pub fn from_path(filename: &CStr, flags: i32, mode: u16) -> Result<Self> {
+        let file_ptr = unsafe {
+            from_err_ptr(bindings::filp_open(
+                filename.as_ptr() as *const i8,
+                flags,
+                mode,
+            ))?
+        };
+        let file_ptr = ptr::NonNull::new(file_ptr).ok_or(ENOENT)?;
+
+        // SAFETY: `filp_open` initializes the refcount with 1
+        Ok(RegularFile(unsafe { ARef::from_raw(file_ptr.cast()) }))
+    }
+
+    /// Constructs a new [`struct file`] wrapper from a path and a vfsmount.
+    pub fn from_path_in_root_mnt(
+        mount: &Vfsmount,
+        filename: &CStr,
+        flags: i32,
+        mode: u16,
+    ) -> Result<Self> {
+        let file_ptr = unsafe {
+            let mnt = mount.get();
+            let raw_path = bindings::path {
+                mnt,
+                dentry: (*mnt).mnt_root,
+            };
+            from_err_ptr(bindings::file_open_root(
+                &raw_path,
+                filename.as_ptr() as *const i8,
+                flags,
+                mode,
+            ))?
+        };
+        let file_ptr = ptr::NonNull::new(file_ptr).ok_or(ENOENT)?;
+
+        // SAFETY: `file_open_root` increments the refcount before returning. (TODO does it?)
+        Ok(RegularFile(unsafe { ARef::from_raw(file_ptr.cast()) }))
+    }
+
+    /// Read from the file into the specified buffer
+    pub fn read_with_offset(&self, buf: &mut [u8], offset: u64) -> Result<usize> {
+        Ok(unsafe {
+            // kernel_read_file expects a pointer to a "void *" buffer
+            let mut ptr_to_buf = buf.as_mut_ptr() as *mut core::ffi::c_void;
+            // Unless we give a non-null pointer to the file size:
+            // 1. we cannot give a non-zero value for the offset
+            // 2. we cannot have offset 0 and buffer_size > file_size
+            let mut file_size = 0;
+
+            // SAFETY: all the pointers to kernel_read_file are valid
+            let result = bindings::kernel_read_file(
+                self.0 .0.get(),
+                offset.try_into()?,
+                &mut ptr_to_buf,
+                buf.len(),
+                &mut file_size,
+                bindings::kernel_read_file_id_READING_UNKNOWN,
+            );
+
+            // kernel_read_file returns the number of bytes read on success or negative on error.
+            if result < 0 {
+                return Err(Error::from_errno(result.try_into()?));
+            }
+
+            result.try_into()?
+        })
+    }
+}
+
 /// A file descriptor reservation.
 ///
 /// This allows the creation of a file descriptor in two steps: first, we reserve a slot for it,
samples/rust/puzzlefs.rs +1 -1
index db13a1d12466..50d62109a9c1 100644
@@ -130,7 +130,7 @@ fn read(
     ) -> Result<usize> {
         let mut buf = Vec::try_with_capacity(writer.len())?;
         buf.try_resize(writer.len(), 0)?;
-        let file = file::File::from_path_in_root_mnt(
+        let file = file::RegularFile::from_path_in_root_mnt(
             &data,
             c_str!("data"),
             file::flags::O_RDONLY.try_into().unwrap(),
Signature · 1 line
2.40.1
#76 lore raw

PATCH 54/80 rust: file: ensure RegularFile can only create regular files

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)
rust/kernel/file.rs +16 -4
index 0d2771027150..afb2084745de 100644
@@ -206,8 +206,21 @@ unsafe fn dec_ref(obj: ptr::NonNull<Self>) {
 /// A newtype over file, specific to regular files
 pub struct RegularFile(ARef<File>);
 impl RegularFile {
+    fn create_if_regular(file_ptr: ptr::NonNull<bindings::file>) -> Result<RegularFile> {
+        // make sure the file is a regular file
+        // TODO: create a helper function
+        // SAFETY: we have a NonNull pointer
+        unsafe {
+            let inode = core::ptr::addr_of!((*file_ptr.as_ptr()).f_inode).read();
+            if bindings::S_IFMT & ((*inode).i_mode) as u32 != bindings::S_IFREG {
+                return Err(EINVAL);
+            }
+        }
+        Ok(RegularFile(unsafe { ARef::from_raw(file_ptr.cast()) }))
+    }
     /// Constructs a new [`struct file`] wrapper from a path.
     pub fn from_path(filename: &CStr, flags: i32, mode: u16) -> Result<Self> {
+        // SAFETY: `filp_open` initializes the refcount with 1
         let file_ptr = unsafe {
             from_err_ptr(bindings::filp_open(
                 filename.as_ptr() as *const i8,
@@ -217,8 +230,7 @@ pub fn from_path(filename: &CStr, flags: i32, mode: u16) -> Result<Self> {
         };
         let file_ptr = ptr::NonNull::new(file_ptr).ok_or(ENOENT)?;
 
-        // SAFETY: `filp_open` initializes the refcount with 1
-        Ok(RegularFile(unsafe { ARef::from_raw(file_ptr.cast()) }))
+        Self::create_if_regular(file_ptr)
     }
 
     /// Constructs a new [`struct file`] wrapper from a path and a vfsmount.
@@ -228,6 +240,7 @@ pub fn from_path_in_root_mnt(
         flags: i32,
         mode: u16,
     ) -> Result<Self> {
+        // SAFETY: `file_open_root` increments the refcount before returning. (TODO does it?)
         let file_ptr = unsafe {
             let mnt = mount.get();
             let raw_path = bindings::path {
@@ -243,8 +256,7 @@ pub fn from_path_in_root_mnt(
         };
         let file_ptr = ptr::NonNull::new(file_ptr).ok_or(ENOENT)?;
 
-        // SAFETY: `file_open_root` increments the refcount before returning. (TODO does it?)
-        Ok(RegularFile(unsafe { ARef::from_raw(file_ptr.cast()) }))
+        Self::create_if_regular(file_ptr)
     }
 
     /// Read from the file into the specified buffer
Signature · 1 line
2.40.1
#77 lore raw

PATCH 55/80 rust: file: add get_pos method to RegularFile

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 5 +++++
 1 file changed, 5 insertions(+)
rust/kernel/file.rs +5 -0
index afb2084745de..5b119d6c4f72 100644
@@ -287,6 +287,11 @@ pub fn read_with_offset(&self, buf: &mut [u8], offset: u64) -> Result<usize> {
             result.try_into()?
         })
     }
+
+    /// Get the current file position
+    pub fn get_pos(&self) -> u64 {
+        self.0.pos()
+    }
 }
 
 /// A file descriptor reservation.
Signature · 1 line
2.40.1
#78 lore raw

PATCH 56/80 rust: file: add methods read_to_end, get_file_size and update_pos to RegularFile

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 52 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
rust/kernel/file.rs +52 -0
index 5b119d6c4f72..894ee573d147 100644
@@ -22,6 +22,7 @@
     types::Opaque,
     user_ptr::{UserSlicePtr, UserSlicePtrReader, UserSlicePtrWriter},
 };
+use alloc::vec::Vec;
 use core::convert::{TryFrom, TryInto};
 use core::{marker, mem, ptr};
 use macros::vtable;
@@ -288,10 +289,61 @@ pub fn read_with_offset(&self, buf: &mut [u8], offset: u64) -> Result<usize> {
         })
     }
 
+    /// Allocate and return a vector containing the contents of the entire file
+    pub fn read_to_end(&self) -> Result<Vec<u8>> {
+        let file_size = self.get_file_size()?;
+        let mut buffer = Vec::try_with_capacity(file_size)?;
+        buffer.try_resize(file_size, 0)?;
+        self.read_with_offset(&mut buffer, 0)?;
+        Ok(buffer)
+    }
+
     /// Get the current file position
     pub fn get_pos(&self) -> u64 {
         self.0.pos()
     }
+
+    fn get_file_size(&self) -> Result<usize> {
+        let mut buf = Vec::new();
+        unsafe {
+            // kernel_read_file expects a pointer to a "void *" buffer
+            let mut ptr_to_buf = buf.as_mut_ptr() as *mut core::ffi::c_void;
+            let mut file_size = 0;
+
+            // SAFETY: all the pointers to kernel_read_file are valid
+            let result = bindings::kernel_read_file(
+                self.0 .0.get(),
+                0,
+                &mut ptr_to_buf,
+                buf.len(),
+                &mut file_size,
+                bindings::kernel_read_file_id_READING_UNKNOWN,
+            );
+
+            // kernel_read_file returns the number of bytes read on success or negative on error.
+            if result < 0 {
+                return Err(Error::from_errno(result.try_into()?));
+            }
+            Ok(file_size)
+        }
+    }
+
+    fn update_pos(&self, where_to: SeekFrom) -> Result<u64> {
+        let (offset, whence): (i64, _) = match where_to {
+            SeekFrom::Start(off) => (off.try_into()?, bindings::SEEK_SET),
+            SeekFrom::Current(off) => (off, bindings::SEEK_CUR),
+            SeekFrom::End(off) => (off, bindings::SEEK_END),
+        };
+
+        let offset =
+            unsafe { bindings::generic_file_llseek(self.0 .0.get(), offset, whence.try_into()?) };
+        if offset < 0 {
+            return Err(Error::from_errno(offset.try_into()?));
+        }
+
+        // we just checked that offset is non negative
+        Ok(offset as u64)
+    }
 }
 
 /// A file descriptor reservation.
Signature · 1 line
2.40.1
#79 lore raw

PATCH 57/80 rust: file: define a minimal Read trait and implement it for RegularFile

In reply to Ariel Miculas
---
 rust/kernel/file.rs | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
rust/kernel/file.rs +42 -0
index 894ee573d147..e51675823724 100644
@@ -204,6 +204,40 @@ unsafe fn dec_ref(obj: ptr::NonNull<Self>) {
     }
 }
 
+/// adapted from https://doc.rust-lang.org/std/io/trait.Read.html
+pub trait Read {
+    // Required method
+    /// Pull some bytes from this source into the specified buffer, returning how many bytes were read.
+    /// See https://doc.rust-lang.org/std/io/trait.Read.html#tymethod.read for details
+    /// This is the equivalent of read from std::io::Read::read, but in kernel space instead of user space
+    /// The buffer is a kernel-allocated buffer
+    fn read(&mut self, buf: &mut [u8]) -> Result<usize>;
+
+    // Provided methods
+
+    // Implementation adapted from https://github.com/rust-lang/rust/blob/80917360d350fb55aebf383e7ff99efea41f63fd/library/std/src/io/mod.rs#L465
+    /// Read the exact number of bytes required to fill buf.
+    /// See https://doc.rust-lang.org/std/io/trait.Read.html#method.read_exact for details
+    fn read_exact(&mut self, mut buf: &mut [u8]) -> Result<()> {
+        while !buf.is_empty() {
+            match self.read(buf) {
+                Ok(0) => break,
+                Ok(n) => {
+                    let tmp = buf;
+                    buf = &mut tmp[n..];
+                }
+                Err(e) => return Err(e),
+            }
+        }
+
+        if !buf.is_empty() {
+            Err(EINVAL)
+        } else {
+            Ok(())
+        }
+    }
+}
+
 /// A newtype over file, specific to regular files
 pub struct RegularFile(ARef<File>);
 impl RegularFile {
@@ -346,6 +380,14 @@ fn update_pos(&self, where_to: SeekFrom) -> Result<u64> {
     }
 }
 
+impl Read for &mut RegularFile {
+    fn read(&mut self, buf: &mut [u8]) -> Result<usize> {
+        let nr_bytes_read = self.read_with_offset(buf, self.get_pos())?;
+        self.update_pos(SeekFrom::Current(nr_bytes_read.try_into()?))?;
+        Ok(nr_bytes_read)
+    }
+}
+
 /// A file descriptor reservation.
 ///
 /// This allows the creation of a file descriptor in two steps: first, we reserve a slot for it,
Signature · 1 line
2.40.1
#80 lore raw

PATCH 58/80 samples: puzzlefs: add cbor_get_array_size method

In reply to Ariel Miculas
---
 samples/rust/puzzle/types/cbor_helpers.rs | 41 +++++++++++++++++++++++
 1 file changed, 41 insertions(+)
samples/rust/puzzle/types/cbor_helpers.rs +41 -0
index ae2aa4609428..b09c104a9286 100644
@@ -1,3 +1,6 @@
+use crate::puzzle::types::{Result, WireFormatError};
+use kernel::file::Read;
+
 pub(crate) const fn cbor_size_of_list_header(size: usize) -> usize {
     match size {
         0..=23 => 1,
@@ -7,3 +10,41 @@ pub(crate) const fn cbor_size_of_list_header(size: usize) -> usize {
         _ => 8,
     }
 }
+
+fn parse_u8(mut reader: impl Read) -> Result<u8> {
+    let mut buf = [0; 1];
+    reader.read_exact(&mut buf)?;
+    Ok(u8::from_be_bytes(buf))
+}
+
+fn parse_u16(mut reader: impl Read) -> Result<u16> {
+    let mut buf = [0; 2];
+    reader.read_exact(&mut buf)?;
+    Ok(u16::from_be_bytes(buf))
+}
+
+fn parse_u32(mut reader: impl Read) -> Result<u32> {
+    let mut buf = [0; 4];
+    reader.read_exact(&mut buf)?;
+    Ok(u32::from_be_bytes(buf))
+}
+
+fn parse_u64(mut reader: impl Read) -> Result<u64> {
+    let mut buf = [0; 8];
+    reader.read_exact(&mut buf)?;
+    Ok(u64::from_be_bytes(buf))
+}
+
+pub(crate) fn cbor_get_array_size<R: Read>(mut reader: R) -> Result<u64> {
+    let mut buf = [0; 1];
+    reader.read_exact(&mut buf)?;
+
+    match buf[0] {
+        0x80..=0x97 => Ok((buf[0] - 0x80) as u64),
+        0x98 => parse_u8(reader).map(u64::from),
+        0x99 => parse_u16(reader).map(u64::from),
+        0x9a => parse_u32(reader).map(u64::from),
+        0x9b => parse_u64(reader).map(u64::from),
+        _ => Err(WireFormatError::ValueMissing),
+    }
+}
Signature · 1 line
2.40.1
#81 lore raw

PATCH 59/80 samples: puzzlefs: add KernelError to WireFormatError and implement From conversion

In reply to Ariel Miculas
---
 samples/rust/puzzle/error.rs | 10 ++++++++++
 1 file changed, 10 insertions(+)
samples/rust/puzzle/error.rs +10 -0
index 3427c5d2f7e3..5fa9c1404df4 100644
@@ -7,6 +7,7 @@ pub(crate) enum WireFormatError {
     SeekOtherError,
     ValueMissing,
     CBORError(serde_cbor::Error),
+    KernelError(kernel::error::Error),
 }
 
 impl Display for WireFormatError {
@@ -16,6 +17,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
             WireFormatError::SeekOtherError => f.write_str("cannot seek to other blob"),
             WireFormatError::ValueMissing => f.write_str("no value present"),
             WireFormatError::CBORError(_) => f.write_str("CBOR error"),
+            WireFormatError::KernelError(_) => f.write_str("Kernel error"),
         }
     }
 }
@@ -30,3 +32,11 @@ fn from(source: serde_cbor::Error) -> Self {
         WireFormatError::CBORError(source)
     }
 }
+
+#[allow(unused_qualifications)]
+impl core::convert::From<kernel::error::Error> for WireFormatError {
+    #[allow(deprecated)]
+    fn from(source: kernel::error::Error) -> Self {
+        WireFormatError::KernelError(source)
+    }
+}
Signature · 1 line
2.40.1
#82 lore raw

PATCH 60/80 samples: puzzlefs: implement new for MetadataBlob

In reply to Ariel Miculas
---
 samples/rust/puzzle/types.rs | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
samples/rust/puzzle/types.rs +14 -4
index 207e5c4f86fa..c6b69b94fa6d 100644
@@ -1,5 +1,4 @@
 use crate::puzzle::error::WireFormatError;
-use alloc::boxed::Box;
 use alloc::vec::Vec;
 use core::mem::size_of;
 use serde::de::Error as SerdeError;
@@ -8,7 +7,8 @@
 use serde_derive::Deserialize;
 mod cbor_helpers;
 use crate::puzzle::error::Result;
-pub(crate) use cbor_helpers::cbor_size_of_list_header;
+pub(crate) use cbor_helpers::{cbor_get_array_size, cbor_size_of_list_header};
+use kernel::file;
 
 #[derive(Deserialize, Debug)]
 pub(crate) struct InodeAdditional {
@@ -26,9 +26,10 @@ pub(crate) struct Xattr {
     pub(crate) val: Vec<u8>,
 }
 
+#[derive(Debug)]
 pub(crate) struct MetadataBlob {
-    mmapped_region: Box<[u8]>,
-    inode_count: usize,
+    pub(crate) mmapped_region: Vec<u8>,
+    pub(crate) inode_count: usize,
 }
 
 fn read_one_from_slice<'a, T: Deserialize<'a>>(bytes: &'a [u8]) -> Result<T> {
@@ -116,6 +117,15 @@ fn visit_bytes<E>(self, v: &[u8]) -> kernel::error::Result<BlobRef, E>
 }
 
 impl MetadataBlob {
+    pub(crate) fn new(mut f: file::RegularFile) -> Result<MetadataBlob> {
+        let inodes_count = cbor_get_array_size(&mut f)? as usize;
+        let mmapped_region = f.read_to_end()?;
+        Ok(MetadataBlob {
+            mmapped_region,
+            inode_count: inodes_count,
+        })
+    }
+
     pub(crate) fn seek_ref(&mut self, r: &BlobRef) -> Result<u64> {
         match r.kind {
             BlobRefKind::Other { .. } => Err(WireFormatError::SeekOtherError),
Signature · 1 line
2.40.1
#83 lore raw

PATCH 61/80 samples: puzzlefs: build puzzlefs into the kernel, thus avoiding the need to export rust symbols

In reply to Ariel Miculas
---
 kernel/configs/rust.config | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/configs/rust.config +1 -1
index 5e29e66e8f7d..4b109055f990 100644
@@ -6,7 +6,7 @@ CONFIG_SAMPLES_RUST=y
 CONFIG_SAMPLE_RUST_MINIMAL=m
 CONFIG_SAMPLE_RUST_PRINT=m
 CONFIG_SAMPLE_RUST_FS=m
-CONFIG_SAMPLE_PUZZLEFS=m
+CONFIG_SAMPLE_PUZZLEFS=y
 CONFIG_SAMPLE_RUST_HOSTPROGS=y
 CONFIG_SAMPLE_RUST_SERDE=y
 
Signature · 1 line
2.40.1
#84 lore raw

PATCH 62/80 rust: alloc: add try_clone for Vec<T>

In reply to Ariel Miculas
---
 rust/alloc/vec/mod.rs | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
rust/alloc/vec/mod.rs +17 -0
index 94995913566b..2ca945f7eb9b 100644
@@ -2489,6 +2489,15 @@ unsafe fn split_at_spare_mut_with_len(
     }
 }
 
+impl<T: Clone> Vec<T> {
+    /// Try to clone the vector using the global allocator
+    #[inline]
+    #[stable(feature = "kernel", since = "1.0.0")]
+    pub fn try_clone(&self) -> Result<Self, TryReserveError> {
+        self.try_clone_in(Global)
+    }
+}
+
 impl<T: Clone, A: Allocator> Vec<T, A> {
     /// Resizes the `Vec` in-place so that `len` is equal to `new_len`.
     ///
@@ -2613,6 +2622,14 @@ pub fn try_extend_from_slice(&mut self, other: &[T]) -> Result<(), TryReserveErr
         self.try_spec_extend(other.iter())
     }
 
+    /// Tries to clone the vector using the given allocator
+    #[stable(feature = "kernel", since = "1.0.0")]
+    pub fn try_clone_in(&self, allocator: A) -> Result<Self, TryReserveError> {
+        let mut new_vec = Vec::try_with_capacity_in(self.len(), allocator)?;
+        new_vec.try_extend_from_slice(&self)?;
+        Ok(new_vec)
+    }
+
     /// Copies elements from `src` range to the end of the vector.
     ///
     /// # Panics
Signature · 1 line
2.40.1
#85 lore raw

PATCH 63/80 rust: alloc: add from_iter_fallible for Vec<T>

In reply to Ariel Miculas
---
 rust/alloc/vec/mod.rs | 10 ++++++++++
 1 file changed, 10 insertions(+)
rust/alloc/vec/mod.rs +10 -0
index 2ca945f7eb9b..b755cf0b936c 100644
@@ -649,6 +649,16 @@ pub fn try_with_capacity(capacity: usize) -> Result<Self, TryReserveError> {
     pub unsafe fn from_raw_parts(ptr: *mut T, length: usize, capacity: usize) -> Self {
         unsafe { Self::from_raw_parts_in(ptr, length, capacity, Global) }
     }
+
+    /// See https://doc.rust-lang.org/std/vec/struct.Vec.html#examples-31
+    #[stable(feature = "rust1", since = "1.0.0")]
+    pub fn from_iter_fallible(iter: impl Iterator<Item=T>) -> Result<Vec<T>, TryReserveError> {
+        let mut vec = Vec::new();
+        for value in iter {
+            vec.try_push(value)?;
+        }
+        Ok(vec)
+    }
 }
 
 impl<T, A: Allocator> Vec<T, A> {
Signature · 1 line
2.40.1
#86 lore raw

Re: [PATCH 63/80] rust: alloc: add from_iter_fallible for Vec<T>

rust/alloc/vec/mod.rs pub fn try_with_capacity(capacity: usize) -> Result<Self, TryReserveError> { · line 654

Quoted 3 lines · Ariel Miculas
rust/alloc/vec/mod.rs +2 -0
+    /// See https://doc.rust-lang.org/std/vec/struct.Vec.html#examples-31
+    #[stable(feature = "rust1", since = "1.0.0")]

For additions to the alloc crate, please use kernel as the feature name. In addition, we write (or rather, copy, if possible, from the infallible ones :) docs.

Cheers,
Miguel

#87 lore raw

PATCH 64/80 samples: puzzlefs: implement to_errno and from_errno for WireFormatError

In reply to Ariel Miculas
---
 samples/rust/puzzle/error.rs | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
samples/rust/puzzle/error.rs +19 -0
index 5fa9c1404df4..254926e6b4e2 100644
@@ -1,4 +1,6 @@
+use core::ffi::c_int;
 use core::fmt::{self, Display};
+use kernel::prelude::{EINVAL, ENOENT, ESPIPE};
 
 // TODO use String in error types (when it's available from the kernel)
 
@@ -10,6 +12,23 @@ pub(crate) enum WireFormatError {
     KernelError(kernel::error::Error),
 }
 
+impl WireFormatError {
+    pub(crate) fn to_errno(&self) -> c_int {
+        match self {
+            WireFormatError::LocalRefError => kernel::error::Error::to_errno(EINVAL),
+            WireFormatError::SeekOtherError => kernel::error::Error::to_errno(ESPIPE),
+            WireFormatError::ValueMissing => kernel::error::Error::to_errno(ENOENT),
+            WireFormatError::CBORError(..) => kernel::error::Error::to_errno(EINVAL),
+            WireFormatError::KernelError(e) => kernel::error::Error::to_errno(*e),
+            WireFormatError::TryReserveError(e) => kernel::error::Error::to_errno(EINVAL),
+        }
+    }
+
+    pub(crate) fn from_errno(errno: kernel::error::Error) -> Self {
+        Self::KernelError(errno)
+    }
+}
+
 impl Display for WireFormatError {
     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
         match self {
Signature · 1 line
2.40.1
#88 lore raw

PATCH 65/80 samples: puzzlefs: add TryReserveError (and from conversion) to WireFormatError

In reply to Ariel Miculas
---
 samples/rust/puzzle/error.rs | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
samples/rust/puzzle/error.rs +12 -1
index 254926e6b4e2..e803c2177b72 100644
@@ -1,3 +1,4 @@
+use alloc::collections::TryReserveError;
 use core::ffi::c_int;
 use core::fmt::{self, Display};
 use kernel::prelude::{EINVAL, ENOENT, ESPIPE};
@@ -10,6 +11,7 @@ pub(crate) enum WireFormatError {
     ValueMissing,
     CBORError(serde_cbor::Error),
     KernelError(kernel::error::Error),
+    TryReserveError(TryReserveError),
 }
 
 impl WireFormatError {
@@ -20,7 +22,7 @@ pub(crate) fn to_errno(&self) -> c_int {
             WireFormatError::ValueMissing => kernel::error::Error::to_errno(ENOENT),
             WireFormatError::CBORError(..) => kernel::error::Error::to_errno(EINVAL),
             WireFormatError::KernelError(e) => kernel::error::Error::to_errno(*e),
-            WireFormatError::TryReserveError(e) => kernel::error::Error::to_errno(EINVAL),
+            WireFormatError::TryReserveError(_) => kernel::error::Error::to_errno(EINVAL),
         }
     }
 
@@ -37,6 +39,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
             WireFormatError::ValueMissing => f.write_str("no value present"),
             WireFormatError::CBORError(_) => f.write_str("CBOR error"),
             WireFormatError::KernelError(_) => f.write_str("Kernel error"),
+            WireFormatError::TryReserveError(_) => f.write_str("TryReserveError"),
         }
     }
 }
@@ -59,3 +62,11 @@ fn from(source: kernel::error::Error) -> Self {
         WireFormatError::KernelError(source)
     }
 }
+
+#[allow(unused_qualifications)]
+impl core::convert::From<TryReserveError> for WireFormatError {
+    #[allow(deprecated)]
+    fn from(source: TryReserveError) -> Self {
+        WireFormatError::TryReserveError(source)
+    }
+}
Signature · 1 line
2.40.1
#89 lore raw

PATCH 66/80 samples: puzzlefs: add higher level inode related functionality

In reply to Ariel Miculas
---
 samples/rust/puzzle.rs       |  3 +-
 samples/rust/puzzle/inode.rs | 83 ++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+), 1 deletion(-)
 create mode 100644 samples/rust/puzzle/inode.rs
samples/rust/puzzle.rs +2 -1
index 4d558561974d..1b989a2579c4 100644
@@ -1,2 +1,3 @@
 pub(crate) mod error;
-mod types;
+pub(crate) mod inode;
+pub(crate) mod types;
samples/rust/puzzle/inode.rs new file +83 -0

83 lines not on this page · raw message

Signature · 1 line
2.40.1
#90 lore raw

PATCH 67/80 samples: puzzlefs: populate the directory entries with the inodes from the puzzlefs metadata file

In reply to Ariel Miculas
---
 rust/kernel/fs.rs        |   4 +-
 samples/rust/puzzlefs.rs | 125 ++++++++++++++++++++++++++++++---------
 2 files changed, 100 insertions(+), 29 deletions(-)
rust/kernel/fs.rs +3 -1
index ba98ae7caf00..81c64faf717b 100644
@@ -775,7 +775,9 @@ impl SuperParams {
 ///
 /// The superblock is a newly-created one and this is the only active pointer to it.
 pub struct NewSuperBlock<'a, T: Type + ?Sized, S = NeedsInit> {
-    sb: &'a mut SuperBlock<T>,
+    /// Pointer to the superblock; this fields is public so puzzlefs can call
+    /// try_new_dcache_dir_inode when populating the directory hierarchy
+    pub sb: &'a mut SuperBlock<T>,
 
     // This also forces `'a` to be invariant.
     _p: PhantomData<&'a mut &'a S>,
samples/rust/puzzlefs.rs +97 -28

174 lines not on this page · raw message

Signature · 1 line
2.40.1
#91 lore raw

PATCH 68/80 rust: hex: import crate

In reply to Ariel Miculas

This is a subset of the Rust hex crate, version v0.4.3, licensed under "Apache-2.0 OR MIT", from:

https://github.com/KokaKiwi/rust-hex/tree/v0.4.3/src

The files are copied as-is, with no modifications whatsoever (not even adding the SPDX identifiers).

For copyright details, please see:

https://github.com/KokaKiwi/rust-hex/blob/v0.4.3/README.md#license
https://github.com/KokaKiwi/rust-hex/blob/v0.4.3/LICENSE-APACHE
https://github.com/KokaKiwi/rust-hex/blob/v0.4.3/LICENSE-MIT

The next three patches modify these files as needed for use within the kernel. This patch split allows reviewers to double-check the import and to clearly see the differences introduced.

The following script may be used to verify the contents:

for path in $(cd rust/hex/ && find . -type f -name '*.rs'); do

    curl --silent --show-error --location \
        https://github.com/KokaKiwi/rust-hex/raw/v0.4.3/src/$path \
        | diff --unified rust/hex/$path - && echo $path: OK

done

---
 rust/hex/error.rs |  59 ++++++
 rust/hex/lib.rs   | 525 ++++++++++++++++++++++++++++++++++++++++++++++
 rust/hex/serde.rs | 102 +++++++++
 3 files changed, 686 insertions(+)
 create mode 100644 rust/hex/error.rs
 create mode 100644 rust/hex/lib.rs
 create mode 100644 rust/hex/serde.rs
rust/hex/error.rs new file +59 -0
new file mode 100644
index 000000000000..ff7a3b5c16bd
@@ -0,0 +1,59 @@
+use core::fmt;
+
+/// The error type for decoding a hex string into `Vec<u8>` or `[u8; N]`.
+#[derive(Debug, Clone, Copy, PartialEq)]
+pub enum FromHexError {
+    /// An invalid character was found. Valid ones are: `0...9`, `a...f`
+    /// or `A...F`.
+    InvalidHexCharacter { c: char, index: usize },
+
+    /// A hex string's length needs to be even, as two digits correspond to
+    /// one byte.
+    OddLength,
+
+    /// If the hex string is decoded into a fixed sized container, such as an
+    /// array, the hex string's length * 2 has to match the container's
+    /// length.
+    InvalidStringLength,
+}
+
+#[cfg(feature = "std")]
+impl std::error::Error for FromHexError {}
+
+impl fmt::Display for FromHexError {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        match *self {
+            FromHexError::InvalidHexCharacter { c, index } => {
+                write!(f, "Invalid character {:?} at position {}", c, index)
+            }
+            FromHexError::OddLength => write!(f, "Odd number of digits"),
+            FromHexError::InvalidStringLength => write!(f, "Invalid string length"),
+        }
+    }
+}
+
+#[cfg(test)]
+// this feature flag is here to suppress unused
+// warnings of `super::*` and `pretty_assertions::assert_eq`
+#[cfg(feature = "alloc")]
+mod tests {
+    use super::*;
+    #[cfg(feature = "alloc")]
+    use alloc::string::ToString;
+    use pretty_assertions::assert_eq;
+
+    #[test]
+    #[cfg(feature = "alloc")]
+    fn test_display() {
+        assert_eq!(
+            FromHexError::InvalidHexCharacter { c: '\n', index: 5 }.to_string(),
+            "Invalid character '\\n' at position 5"
+        );
+
+        assert_eq!(FromHexError::OddLength.to_string(), "Odd number of digits");
+        assert_eq!(
+            FromHexError::InvalidStringLength.to_string(),
+            "Invalid string length"
+        );
+    }
+}
rust/hex/lib.rs new file +525 -0

525 lines not on this page · raw message

rust/hex/serde.rs new file +102 -0

102 lines not on this page · raw message

Signature · 1 line
2.40.1
#92 lore raw

PATCH 69/80 rust: hex: add SPDX license identifiers

In reply to Ariel Miculas
---
 rust/hex/error.rs | 2 ++
 rust/hex/lib.rs   | 2 ++
 rust/hex/serde.rs | 2 ++
 3 files changed, 6 insertions(+)
rust/hex/error.rs +2 -0
index ff7a3b5c16bd..e553a046c9ed 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 use core::fmt;
 
 /// The error type for decoding a hex string into `Vec<u8>` or `[u8; N]`.
rust/hex/lib.rs +2 -0
index ec48961b9ffe..a23def3f80db 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 // Copyright (c) 2013-2014 The Rust Project Developers.
 // Copyright (c) 2015-2020 The rust-hex Developers.
 //
rust/hex/serde.rs +2 -0
index 335a15132a27..40ab1edb2bc8 100644
@@ -1,3 +1,5 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
 //! Hex encoding with `serde`.
 #[cfg_attr(
     all(feature = "alloc", feature = "serde"),
Signature · 1 line
2.40.1
#93 lore raw

PATCH 70/80 rust: Kbuild: enable `hex`

In reply to Ariel Miculas

With all the new files in place from the hex crate, this patch adds support for them in the build system.

---
 rust/Makefile          | 21 ++++++++++++++++++---
 scripts/Makefile.build |  2 +-
 2 files changed, 19 insertions(+), 4 deletions(-)
rust/Makefile +18 -3
index c9feb8e1afe1..ba7f911d6919 100644
@@ -13,7 +13,7 @@ always-$(CONFIG_RUST) += libserde_derive.so libmacros.so
 no-clean-files += libserde_derive.so libmacros.so
 
 always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
-obj-$(CONFIG_RUST) += alloc.o bindings.o serde.o serde_cbor.o kernel.o
+obj-$(CONFIG_RUST) += alloc.o bindings.o serde.o serde_cbor.o hex.o kernel.o
 always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
     exports_kernel_generated.h
 
@@ -137,6 +137,15 @@ serde_cbor-flags := \
     --extern alloc \
     --extern serde
 
+hex-skip_flags := \
+    --edition=2021 \
+    -Drust_2018_idioms \
+    -Dunreachable_pub
+
+hex-flags := \
+    --edition=2018 \
+    -Amissing_docs
+
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
       cmd_rustdoc = \
 	OBJTREE=$(abspath $(objtree)) \
@@ -589,12 +598,18 @@ $(obj)/serde_cbor.o: private rustc_target_flags = $(serde_cbor-flags)
 $(obj)/serde_cbor.o: $(src)/serde_cbor/lib.rs $(obj)/compiler_builtins.o FORCE
 	$(call if_changed_dep,rustc_library)
 
+$(obj)/hex.o: private skip_clippy = 1
+$(obj)/hex.o: private skip_flags = $(hex-skip_flags)
+$(obj)/hex.o: private rustc_target_flags = $(hex-flags)
+$(obj)/hex.o: $(src)/hex/lib.rs $(obj)/compiler_builtins.o FORCE
+	$(call if_changed_dep,rustc_library)
+
 $(obj)/kernel.o: private rustc_target_flags = --extern alloc \
     --extern build_error --extern macros --extern bindings --extern uapi \
-    --extern serde --extern serde_derive --extern serde_cbor
+    --extern serde --extern serde_derive --extern serde_cbor --extern hex
 $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
     $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o \
-    $(obj)/serde.o $(obj)/libserde_derive.so $(obj)/serde_cbor.o FORCE
+    $(obj)/serde.o $(obj)/libserde_derive.so $(obj)/serde_cbor.o $(obj)/hex.o FORCE
 	$(call if_changed_dep,rustc_library)
 
 endif # CONFIG_RUST
scripts/Makefile.build +1 -1
index 758b327fd54d..e88cfd953833 100644
@@ -284,7 +284,7 @@ rust_common_cmd = \
 	-Zallow-features=$(rust_allowed_features) \
 	-Zcrate-attr=no_std \
 	-Zcrate-attr='feature($(rust_allowed_features))' \
-	--extern alloc --extern kernel --extern serde --extern serde_derive --extern serde_cbor \
+	--extern alloc --extern kernel --extern serde --extern serde_derive --extern serde_cbor --extern hex \
 	--crate-type rlib -L $(objtree)/rust/ \
 	--crate-name $(basename $(notdir $@)) \
 	--emit=dep-info=$(depfile)
Signature · 1 line
2.40.1
#94 lore raw

PATCH 71/80 rust: hex: implement FromHex trait and hex::decode using a custom kernel_alloc feature

In reply to Ariel Miculas

This commit also adds a process_results function for Vec<Result<T,E>> which transforms it into a Result<Vec<T,E>>, collecting all the values inside the Result elements into a new vector which directly stores the elements.
For this to work, the TryReserveError was added to the FromHexError enum.
This is probably not the ideal design, but since collect cannot be used directly, it's a good substitute.

---
 rust/Makefile         |  4 +++-
 rust/alloc/vec/mod.rs | 21 +++++++++++++++++++++
 rust/hex/error.rs     | 17 +++++++++++++++++
 rust/hex/lib.rs       | 19 +++++++++++++++----
 4 files changed, 56 insertions(+), 5 deletions(-)
rust/Makefile +3 -1
index ba7f911d6919..9b3488764d68 100644
@@ -144,7 +144,9 @@ hex-skip_flags := \
 
 hex-flags := \
     --edition=2018 \
-    -Amissing_docs
+    -Amissing_docs \
+    --cfg 'feature="kernel_alloc"' \
+    --extern alloc
 
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
       cmd_rustdoc = \
rust/alloc/vec/mod.rs +21 -0
index b755cf0b936c..2761f8bf5a9e 100644
@@ -661,6 +661,27 @@ pub fn from_iter_fallible(iter: impl Iterator<Item=T>) -> Result<Vec<T>, TryRese
     }
 }
 
+impl<T,E: core::convert::From<TryReserveError>> Vec<Result<T,E>> {
+    /// Iterates through the vector, consuming self, unwrapping the individual values in each
+    /// Result and creating a new vector with them. If there's any error value, it returns it
+    /// immediately
+    #[stable(feature = "rust1", since = "1.0.0")]
+    pub fn process_results(self) -> Result<Vec<T>,E> {
+        let mut vec = Vec::new();
+        for value in self {
+            match value {
+                Ok(v) => {
+                    vec.try_push(v)?;
+                },
+                Err(e) => {
+                    return Err(e);
+                }
+            }
+        }
+        Ok(vec)
+    }
+}
+
 impl<T, A: Allocator> Vec<T, A> {
     /// Constructs a new, empty `Vec<T, A>`.
     ///
rust/hex/error.rs +17 -0
index e553a046c9ed..c0dc2f558c4d 100644
@@ -1,5 +1,7 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
+#[cfg(feature = "kernel_alloc")]
+use alloc::collections::TryReserveError;
 use core::fmt;
 
 /// The error type for decoding a hex string into `Vec<u8>` or `[u8; N]`.
@@ -17,6 +19,10 @@ pub enum FromHexError {
     /// array, the hex string's length * 2 has to match the container's
     /// length.
     InvalidStringLength,
+
+    #[cfg(feature = "kernel_alloc")]
+    /// If an allocation fails, useful in the kernel context
+    TryReserveError,
 }
 
 #[cfg(feature = "std")]
@@ -30,10 +36,21 @@ fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
             }
             FromHexError::OddLength => write!(f, "Odd number of digits"),
             FromHexError::InvalidStringLength => write!(f, "Invalid string length"),
+            #[cfg(feature = "kernel_alloc")]
+            FromHexError::TryReserveError => write!(f, "TryReserveError"),
         }
     }
 }
 
+#[cfg(feature = "kernel_alloc")]
+#[allow(unused_qualifications)]
+impl core::convert::From<TryReserveError> for FromHexError {
+    #[allow(deprecated)]
+    fn from(_source: TryReserveError) -> Self {
+        FromHexError::TryReserveError
+    }
+}
+
 #[cfg(test)]
 // this feature flag is here to suppress unused
 // warnings of `super::*` and `pretty_assertions::assert_eq`
rust/hex/lib.rs +15 -4
index a23def3f80db..5cd90d622d16 100644
@@ -41,6 +41,8 @@
 
 #[cfg(feature = "alloc")]
 extern crate alloc;
+#[cfg(feature = "kernel_alloc")]
+use alloc::vec::Vec;
 #[cfg(feature = "alloc")]
 use alloc::{string::String, vec::Vec};
 
@@ -186,7 +188,7 @@ fn val(c: u8, idx: usize) -> Result<u8, FromHexError> {
     }
 }
 
-#[cfg(feature = "alloc")]
+#[cfg(any(feature = "alloc", feature = "kernel_alloc"))]
 impl FromHex for Vec<u8> {
     type Error = FromHexError;
 
@@ -196,10 +198,19 @@ fn from_hex<T: AsRef<[u8]>>(hex: T) -> Result<Self, Self::Error> {
             return Err(FromHexError::OddLength);
         }
 
-        hex.chunks(2)
+        #[cfg(not(feature = "kernel_alloc"))]
+        return hex
+            .chunks(2)
             .enumerate()
             .map(|(i, pair)| Ok(val(pair[0], 2 * i)? << 4 | val(pair[1], 2 * i + 1)?))
-            .collect()
+            .collect();
+        #[cfg(feature = "kernel_alloc")]
+        return Vec::from_iter_fallible(
+            hex.chunks(2)
+                .enumerate()
+                .map(|(i, pair)| Ok(val(pair[0], 2 * i)? << 4 | val(pair[1], 2 * i + 1)?)),
+        )?
+        .process_results();
     }
 }
 
@@ -294,7 +305,7 @@ pub fn encode_upper<T: AsRef<[u8]>>(data: T) -> String {
 /// assert_eq!(hex::decode("123"), Err(hex::FromHexError::OddLength));
 /// assert!(hex::decode("foo").is_err());
 /// ```
-#[cfg(feature = "alloc")]
+#[cfg(any(feature = "alloc", feature = "kernel_alloc"))]
 pub fn decode<T: AsRef<[u8]>>(data: T) -> Result<Vec<u8>, FromHexError> {
     FromHex::from_hex(data)
 }
Signature · 1 line
2.40.1
#95 lore raw

PATCH 72/80 rust: hex: add encode_hex_iter and encode_hex_upper_iter methods

In reply to Ariel Miculas

Add encode_hex_iter and encode_hex_upper_iter which return an iterator, so it can be used in conjunction with the Vec::from_iter_fallible function to (try to) create a hex string.

Replace BytesToHexChars with BytesToHexByteSequence because String is not available. We need to collect the values in a Vec<u8>, append a NUL character and the create a CStr on top of it.

Also remove the ToHex trait because it uses the FromIterator trait which cannot be implemented in the kernel due to the prohibition of infallible allocations.

---
 rust/hex/lib.rs | 60 ++++++++++++-------------------------------------
 1 file changed, 14 insertions(+), 46 deletions(-)
rust/hex/lib.rs +14 -46
index 5cd90d622d16..18f5c240c46b 100644
@@ -59,43 +59,18 @@
 #[cfg(all(feature = "alloc", feature = "serde"))]
 pub use crate::serde::{serialize, serialize_upper};
 
-/// Encoding values as hex string.
-///
-/// This trait is implemented for all `T` which implement `AsRef<[u8]>`. This
-/// includes `String`, `str`, `Vec<u8>` and `[u8]`.
-///
-/// # Example
-///
-/// ```
-/// use hex::ToHex;
-///
-/// println!("{}", "Hello world!".encode_hex::<String>());
-/// # assert_eq!("Hello world!".encode_hex::<String>(), "48656c6c6f20776f726c6421".to_string());
-/// ```
-///
-/// *Note*: instead of using this trait, you might want to use [`encode()`].
-pub trait ToHex {
-    /// Encode the hex strict representing `self` into the result. Lower case
-    /// letters are used (e.g. `f9b4ca`)
-    fn encode_hex<T: iter::FromIterator<char>>(&self) -> T;
-
-    /// Encode the hex strict representing `self` into the result. Upper case
-    /// letters are used (e.g. `F9B4CA`)
-    fn encode_hex_upper<T: iter::FromIterator<char>>(&self) -> T;
-}
-
 const HEX_CHARS_LOWER: &[u8; 16] = b"0123456789abcdef";
 const HEX_CHARS_UPPER: &[u8; 16] = b"0123456789ABCDEF";
 
-struct BytesToHexChars<'a> {
+struct BytesToHexByteSequence<'a> {
     inner: ::core::slice::Iter<'a, u8>,
     table: &'static [u8; 16],
-    next: Option<char>,
+    next: Option<u8>,
 }
 
-impl<'a> BytesToHexChars<'a> {
-    fn new(inner: &'a [u8], table: &'static [u8; 16]) -> BytesToHexChars<'a> {
-        BytesToHexChars {
+impl<'a> BytesToHexByteSequence<'a> {
+    fn new(inner: &'a [u8], table: &'static [u8; 16]) -> BytesToHexByteSequence<'a> {
+        BytesToHexByteSequence {
             inner: inner.iter(),
             table,
             next: None,
@@ -103,15 +78,15 @@ impl<'a> BytesToHexChars<'a> {
     }
 }
 
-impl<'a> Iterator for BytesToHexChars<'a> {
-    type Item = char;
+impl<'a> Iterator for BytesToHexByteSequence<'a> {
+    type Item = u8;
 
     fn next(&mut self) -> Option<Self::Item> {
         match self.next.take() {
             Some(current) => Some(current),
             None => self.inner.next().map(|byte| {
-                let current = self.table[(byte >> 4) as usize] as char;
-                self.next = Some(self.table[(byte & 0x0F) as usize] as char);
+                let current = self.table[(byte >> 4) as usize];
+                self.next = Some(self.table[(byte & 0x0F) as usize]);
                 current
             }),
         }
@@ -123,7 +98,7 @@ fn size_hint(&self) -> (usize, Option<usize>) {
     }
 }
 
-impl<'a> iter::ExactSizeIterator for BytesToHexChars<'a> {
+impl<'a> iter::ExactSizeIterator for BytesToHexByteSequence<'a> {
     fn len(&self) -> usize {
         let mut length = self.inner.len() * 2;
         if self.next.is_some() {
@@ -133,19 +108,12 @@ fn len(&self) -> usize {
     }
 }
 
-#[inline]
-fn encode_to_iter<T: iter::FromIterator<char>>(table: &'static [u8; 16], source: &[u8]) -> T {
-    BytesToHexChars::new(source, table).collect()
+pub fn encode_hex_iter<'a>(source: &'a [u8]) -> impl iter::Iterator<Item = u8> + 'a {
+    BytesToHexByteSequence::new(source, HEX_CHARS_LOWER).into_iter()
 }
 
-impl<T: AsRef<[u8]>> ToHex for T {
-    fn encode_hex<U: iter::FromIterator<char>>(&self) -> U {
-        encode_to_iter(HEX_CHARS_LOWER, self.as_ref())
-    }
-
-    fn encode_hex_upper<U: iter::FromIterator<char>>(&self) -> U {
-        encode_to_iter(HEX_CHARS_UPPER, self.as_ref())
-    }
+pub fn encode_hex_upper_iter<'a>(source: &'a [u8]) -> impl iter::Iterator<Item = u8> + 'a {
+    BytesToHexByteSequence::new(source, HEX_CHARS_UPPER).into_iter()
 }
 
 /// Types that can be decoded from a hex string.
Signature · 1 line
2.40.1
#96 lore raw

PATCH 73/80 rust: puzzlefs: add HexError to WireFormatError and implement the From conversion

In reply to Ariel Miculas
---
 samples/rust/puzzle/error.rs | 11 +++++++++++
 1 file changed, 11 insertions(+)
samples/rust/puzzle/error.rs +11 -0
index e803c2177b72..8554756be8d3 100644
@@ -12,6 +12,7 @@ pub(crate) enum WireFormatError {
     CBORError(serde_cbor::Error),
     KernelError(kernel::error::Error),
     TryReserveError(TryReserveError),
+    HexError(hex::FromHexError),
 }
 
 impl WireFormatError {
@@ -23,6 +24,7 @@ pub(crate) fn to_errno(&self) -> c_int {
             WireFormatError::CBORError(..) => kernel::error::Error::to_errno(EINVAL),
             WireFormatError::KernelError(e) => kernel::error::Error::to_errno(*e),
             WireFormatError::TryReserveError(_) => kernel::error::Error::to_errno(EINVAL),
+            WireFormatError::HexError(_) => kernel::error::Error::to_errno(EINVAL),
         }
     }
 
@@ -40,6 +42,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
             WireFormatError::CBORError(_) => f.write_str("CBOR error"),
             WireFormatError::KernelError(_) => f.write_str("Kernel error"),
             WireFormatError::TryReserveError(_) => f.write_str("TryReserveError"),
+            WireFormatError::HexError(_) => f.write_str("HexError"),
         }
     }
 }
@@ -70,3 +73,11 @@ fn from(source: TryReserveError) -> Self {
         WireFormatError::TryReserveError(source)
     }
 }
+
+#[allow(unused_qualifications)]
+impl core::convert::From<hex::FromHexError> for WireFormatError {
+    #[allow(deprecated)]
+    fn from(source: hex::FromHexError) -> Self {
+        WireFormatError::HexError(source)
+    }
+}
Signature · 1 line
2.40.1
#97 lore raw

PATCH 74/80 rust: puzzlefs: display the error value for WireFormatError::KernelError

In reply to Ariel Miculas
---
 samples/rust/puzzle/error.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
samples/rust/puzzle/error.rs +1 -1
index 8554756be8d3..8766d01ef932 100644
@@ -40,7 +40,7 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
             WireFormatError::SeekOtherError => f.write_str("cannot seek to other blob"),
             WireFormatError::ValueMissing => f.write_str("no value present"),
             WireFormatError::CBORError(_) => f.write_str("CBOR error"),
-            WireFormatError::KernelError(_) => f.write_str("Kernel error"),
+            WireFormatError::KernelError(e) => write!(f, "Kernel error {:?}", e.to_errno()),
             WireFormatError::TryReserveError(_) => f.write_str("TryReserveError"),
             WireFormatError::HexError(_) => f.write_str("HexError"),
         }
Signature · 1 line
2.40.1
#98 lore raw

PATCH 75/80 samples: puzzlefs: add Rootfs and Digest structs to types.rs

In reply to Ariel Miculas
---
 samples/rust/puzzle/types.rs | 75 ++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)
samples/rust/puzzle/types.rs +75 -0
index c6b69b94fa6d..69d4f5dfa12c 100644
@@ -8,7 +8,10 @@
 mod cbor_helpers;
 use crate::puzzle::error::Result;
 pub(crate) use cbor_helpers::{cbor_get_array_size, cbor_size_of_list_header};
+use core::fmt;
+use hex::{encode_hex_iter, FromHexError};
 use kernel::file;
+use kernel::str::CStr;
 
 #[derive(Deserialize, Debug)]
 pub(crate) struct InodeAdditional {
@@ -32,6 +35,8 @@ pub(crate) struct MetadataBlob {
     pub(crate) inode_count: usize,
 }
 
+pub(crate) const SHA256_BLOCK_SIZE: usize = 32;
+
 fn read_one_from_slice<'a, T: Deserialize<'a>>(bytes: &'a [u8]) -> Result<T> {
     // serde complains when we leave extra bytes on the wire, which we often want to do. as a
     // hack, we create a streaming deserializer for the type we're about to read, and then only
@@ -41,6 +46,23 @@ fn read_one_from_slice<'a, T: Deserialize<'a>>(bytes: &'a [u8]) -> Result<T> {
     v.ok_or(WireFormatError::ValueMissing)
 }
 
+#[derive(Deserialize, Debug)]
+pub(crate) struct Rootfs {
+    pub(crate) metadatas: Vec<BlobRef>,
+    // TODO: deserialize fs_verity_data, for the moment BTreeMap is not supported
+    #[allow(dead_code)]
+    pub(crate) fs_verity_data: (),
+    #[allow(dead_code)]
+    pub(crate) manifest_version: u64,
+}
+
+impl Rootfs {
+    pub(crate) fn open(file: file::RegularFile) -> Result<Rootfs> {
+        let buffer = file.read_to_end()?;
+        read_one_from_slice(&buffer)
+    }
+}
+
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub(crate) enum BlobRefKind {
     Local,
@@ -176,6 +198,59 @@ pub(crate) fn find_inode(&mut self, ino: Ino) -> Result<Option<Inode>> {
     }
 }
 
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub(crate) struct Digest([u8; SHA256_BLOCK_SIZE]);
+
+impl Digest {
+    pub(crate) fn underlying(&self) -> [u8; SHA256_BLOCK_SIZE] {
+        let mut dest = [0_u8; SHA256_BLOCK_SIZE];
+        dest.copy_from_slice(&self.0);
+        dest
+    }
+}
+
+impl fmt::Display for Digest {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        let mut hex_string =
+            Vec::from_iter_fallible(encode_hex_iter(&self.underlying())).map_err(|_| fmt::Error)?;
+        // append NUL character
+        hex_string.try_push(0).map_err(|_| fmt::Error)?;
+        let hex_string = CStr::from_bytes_with_nul(&hex_string).map_err(|_| fmt::Error)?;
+        write!(f, "{}", hex_string)
+    }
+}
+
+impl TryFrom<&CStr> for Digest {
+    type Error = FromHexError;
+    fn try_from(s: &CStr) -> kernel::error::Result<Self, Self::Error> {
+        let digest = hex::decode(s)?;
+        let digest: [u8; SHA256_BLOCK_SIZE] = digest
+            .try_into()
+            .map_err(|_| FromHexError::InvalidStringLength)?;
+        Ok(Digest(digest))
+    }
+}
+
+impl TryFrom<BlobRef> for Digest {
+    type Error = WireFormatError;
+    fn try_from(v: BlobRef) -> kernel::error::Result<Self, Self::Error> {
+        match v.kind {
+            BlobRefKind::Other { digest } => Ok(Digest(digest)),
+            BlobRefKind::Local => Err(WireFormatError::LocalRefError),
+        }
+    }
+}
+
+impl TryFrom<&BlobRef> for Digest {
+    type Error = WireFormatError;
+    fn try_from(v: &BlobRef) -> kernel::error::Result<Self, Self::Error> {
+        match v.kind {
+            BlobRefKind::Other { digest } => Ok(Digest(digest)),
+            BlobRefKind::Local => Err(WireFormatError::LocalRefError),
+        }
+    }
+}
+
 #[derive(Deserialize, Debug)]
 pub(crate) struct DirEnt {
     pub(crate) ino: Ino,
Signature · 1 line
2.40.1
#99 lore raw

PATCH 76/80 samples: puzzlefs: implement the conversion from WireFormatError to kernel::error::Error

In reply to Ariel Miculas
---
 rust/kernel/error.rs         | 2 +-
 samples/rust/puzzle/error.rs | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)
rust/kernel/error.rs +1 -1
index 6fff9a4f0672..8196a7462561 100644
@@ -96,7 +96,7 @@ impl Error {
     ///
     /// It is a bug to pass an out-of-range `errno`. `EINVAL` would
     /// be returned in such a case.
-    pub(crate) fn from_errno(errno: core::ffi::c_int) -> Error {
+    pub fn from_errno(errno: core::ffi::c_int) -> Error {
         if errno < -(bindings::MAX_ERRNO as i32) || errno >= 0 {
             // TODO: Make it a `WARN_ONCE` once available.
             crate::pr_warn!(
samples/rust/puzzle/error.rs +8 -0
index 8766d01ef932..c9635a96f9e2 100644
@@ -81,3 +81,11 @@ fn from(source: hex::FromHexError) -> Self {
         WireFormatError::HexError(source)
     }
 }
+
+#[allow(unused_qualifications)]
+impl core::convert::From<WireFormatError> for kernel::error::Error {
+    #[allow(deprecated)]
+    fn from(source: WireFormatError) -> Self {
+        kernel::error::Error::from_errno(source.to_errno())
+    }
+}
Signature · 1 line
2.40.1
#100 lore raw

PATCH 77/80 rust: puzzlefs: read the puzzlefs image manifest instead of an individual metadata layer

In reply to Ariel Miculas

The puzzlefs image manifest is the file referenced by index.json and it contains the array of metadata layers that describe the puzzlefs image. This file represents the root of the puzzlefs filesystem since we can't parse json files. When this filesystem is mounted, usermode will need to read the tag from index.json, find the corresponding puzzlefs image manifest and pass it to the kernel module.

Due to the lack of BTreeMap in the kernel, only image manifests without fs verity information are supported for now.

---
 samples/rust/puzzle.rs       |  1 +
 samples/rust/puzzle/inode.rs | 23 ++++++++++++----
 samples/rust/puzzle/oci.rs   | 51 ++++++++++++++++++++++++++++++++++++
 samples/rust/puzzlefs.rs     | 23 +++++++---------
 4 files changed, 80 insertions(+), 18 deletions(-)
 create mode 100644 samples/rust/puzzle/oci.rs
samples/rust/puzzle.rs +1 -0
index 1b989a2579c4..ae7c57efc92c 100644
@@ -1,3 +1,4 @@
 pub(crate) mod error;
 pub(crate) mod inode;
+pub(crate) mod oci;
 pub(crate) mod types;
samples/rust/puzzle/inode.rs +18 -5
index 980aac9c4fd3..eaaa6d964db2 100644
@@ -3,10 +3,15 @@
 
 use crate::puzzle::error::Result;
 use crate::puzzle::error::WireFormatError;
+use crate::puzzle::oci::Image;
 use crate::puzzle::types as format;
+use crate::puzzle::types::Digest;
 use crate::puzzle::types::{FileChunk, Ino, InodeAdditional, MetadataBlob};
 use alloc::vec::Vec;
+use kernel::mount::Vfsmount;
 use kernel::prelude::{ENOENT, ENOTDIR};
+use kernel::str::CStr;
+use kernel::sync::Arc;
 
 #[derive(Debug)]
 pub(crate) struct Inode {
@@ -17,17 +22,25 @@ pub(crate) struct Inode {
 }
 
 pub(crate) struct PuzzleFS {
+    pub(crate) oci: Image,
     layers: Vec<format::MetadataBlob>,
 }
 
 impl PuzzleFS {
-    pub(crate) fn new(md: MetadataBlob) -> Result<Self> {
-        let mut v = Vec::new();
-        v.try_push(md)?;
-        Ok(PuzzleFS { layers: v })
+    pub(crate) fn open(vfsmount: Arc<Vfsmount>, rootfs_path: &CStr) -> Result<PuzzleFS> {
+        let oci = Image::open(vfsmount)?;
+        let rootfs = oci.open_rootfs_blob(rootfs_path)?;
+
+        let layers =
+            Vec::from_iter_fallible(rootfs.metadatas.iter().map(|md| -> Result<MetadataBlob> {
+                let digest = Digest::try_from(md)?;
+                oci.open_metadata_blob(&digest)
+            }))?
+            .process_results()?;
+
+        Ok(PuzzleFS { oci, layers })
     }
 
-    // Temporary helper function used until PuzzleFs is integrated
     pub(crate) fn find_inode(&mut self, ino: u64) -> Result<Inode> {
         for layer in self.layers.iter_mut() {
             if let Some(inode) = layer.find_inode(ino)? {
samples/rust/puzzle/oci.rs new file +51 -0

51 lines not on this page · raw message

samples/rust/puzzlefs.rs +10 -13
index c160b5e5f911..d74e034538eb 100644
@@ -13,7 +13,6 @@
 
 mod puzzle;
 use puzzle::inode::{Inode, InodeMode, PuzzleFS};
-use puzzle::types::MetadataBlob;
 
 use kernel::fs::{DEntry, INodeParams, NeedsRoot, NewSuperBlock, RootDEntry};
 
@@ -67,7 +66,7 @@ fn puzzlefs_populate_dir(
         return Err(E2BIG);
     }
 
-    let inode = Arc::try_new(pfs.find_inode(ino).map_err(|_| EINVAL)?)?;
+    let inode = Arc::try_new(pfs.find_inode(ino)?)?;
     match &inode.mode {
         InodeMode::File { chunks: _ } => {
             let params = INodeParams {
@@ -152,19 +151,17 @@ fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBl
             },
         )?;
 
-        let file = file::RegularFile::from_path_in_root_mnt(
-            &arc_vfs_mount,
-            c_str!("997eed138af30d187e87d682dd2ae9f240fae78f668907a0519460b397c82467"),
-            file::flags::O_RDONLY.try_into().unwrap(),
-            0,
-        )?;
+        let puzzlefs = PuzzleFS::open(
+            arc_vfs_mount,
+            c_str!("2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b"),
+        );
 
-        // TODO: figure out how to go from WireFormatError to kernel::error::Error
-        let metadata = MetadataBlob::new(file).map_err(|_| EINVAL)?;
-        pr_info!("number of inodes {:?}\n", metadata.inode_count);
+        if let Err(ref e) = puzzlefs {
+            pr_info!("error opening puzzlefs {e}\n");
+        }
 
-        let mut puzzlefs = PuzzleFS::new(metadata).map_err(|_| EINVAL)?;
-        let root_inode = Arc::try_new(puzzlefs.find_inode(1).map_err(|_| EINVAL)?)?;
+        let mut puzzlefs = puzzlefs?;
+        let root_inode = Arc::try_new(puzzlefs.find_inode(1)?)?;
 
         let root = try_new_populated_root_puzzlefs_dentry(&sb, &mut puzzlefs, root_inode)?;
         let sb = sb.init_root(root)?;
Signature · 1 line
2.40.1
#101 lore raw

PATCH 78/80 rust: puzzlefs: rename PuzzleFs to PuzzleFsModule to avoid confusion with the PuzzleFS struct

In reply to Ariel Miculas
---
 samples/rust/puzzlefs.rs | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
samples/rust/puzzlefs.rs +9 -9
index d74e034538eb..f04a2e5bcb04 100644
@@ -17,13 +17,13 @@
 use kernel::fs::{DEntry, INodeParams, NeedsRoot, NewSuperBlock, RootDEntry};
 
 module_fs! {
-    type: PuzzleFs,
+    type: PuzzleFsModule,
     name: "puzzlefs",
     author: "Ariel Miculas",
     license: "GPL",
 }
 
-struct PuzzleFs;
+struct PuzzleFsModule;
 
 #[derive(Debug)]
 struct PuzzlefsInfo {
@@ -31,7 +31,7 @@ struct PuzzlefsInfo {
 }
 
 #[vtable]
-impl fs::Context<Self> for PuzzleFs {
+impl fs::Context<Self> for PuzzleFsModule {
     type Data = ();
 
     kernel::define_fs_params! {(),
@@ -55,9 +55,9 @@ fn try_new() -> Result {
 }
 
 fn puzzlefs_populate_dir(
-    sb: &NewSuperBlock<'_, PuzzleFs, NeedsRoot>,
+    sb: &NewSuperBlock<'_, PuzzleFsModule, NeedsRoot>,
     pfs: &mut PuzzleFS,
-    parent: &DEntry<PuzzleFs>,
+    parent: &DEntry<PuzzleFsModule>,
     ino: u64,
     name: &CStr,
     recursion: usize,
@@ -110,10 +110,10 @@ fn puzzlefs_populate_dir(
 
 /// Creates a new root dentry populated with the given entries.
 fn try_new_populated_root_puzzlefs_dentry(
-    sb: &NewSuperBlock<'_, PuzzleFs, NeedsRoot>,
+    sb: &NewSuperBlock<'_, PuzzleFsModule, NeedsRoot>,
     pfs: &mut PuzzleFS,
-    root_value: <PuzzleFs as fs::Type>::INodeData,
-) -> Result<RootDEntry<PuzzleFs>> {
+    root_value: <PuzzleFsModule as fs::Type>::INodeData,
+) -> Result<RootDEntry<PuzzleFsModule>> {
     let root_inode = sb.sb.try_new_dcache_dir_inode(INodeParams {
         mode: 0o755,
         ino: root_value.inode.ino,
@@ -125,7 +125,7 @@ fn try_new_populated_root_puzzlefs_dentry(
     Ok(root)
 }
 
-impl fs::Type for PuzzleFs {
+impl fs::Type for PuzzleFsModule {
     type Context = Self;
     // this is Arc so it can be cloned in puzzlefs_populate_dir
     type INodeData = Arc<Inode>;
Signature · 1 line
2.40.1
#102 lore raw

PATCH 79/80 rust: puzzlefs: add support for reading files

In reply to Ariel Miculas
---
 samples/rust/puzzle/inode.rs | 68 ++++++++++++++++++++++++++++++++----
 samples/rust/puzzle/oci.rs   | 32 +++++++++++++----
 samples/rust/puzzle/types.rs | 10 +++---
 samples/rust/puzzlefs.rs     | 59 ++++++++++++++-----------------
 4 files changed, 118 insertions(+), 51 deletions(-)
samples/rust/puzzle/inode.rs +61 -7
index eaaa6d964db2..678e9fe1af03 100644
@@ -8,10 +8,10 @@
 use crate::puzzle::types::Digest;
 use crate::puzzle::types::{FileChunk, Ino, InodeAdditional, MetadataBlob};
 use alloc::vec::Vec;
+use core::cmp::min;
 use kernel::mount::Vfsmount;
 use kernel::prelude::{ENOENT, ENOTDIR};
 use kernel::str::CStr;
-use kernel::sync::Arc;
 
 #[derive(Debug)]
 pub(crate) struct Inode {
@@ -21,14 +21,16 @@ pub(crate) struct Inode {
     pub(crate) additional: Option<InodeAdditional>,
 }
 
+#[derive(Debug)]
 pub(crate) struct PuzzleFS {
     pub(crate) oci: Image,
     layers: Vec<format::MetadataBlob>,
 }
 
 impl PuzzleFS {
-    pub(crate) fn open(vfsmount: Arc<Vfsmount>, rootfs_path: &CStr) -> Result<PuzzleFS> {
-        let oci = Image::open(vfsmount)?;
+    pub(crate) fn open(oci_root_dir: &CStr, rootfs_path: &CStr) -> Result<PuzzleFS> {
+        let vfs_mount = Vfsmount::new_private_mount(oci_root_dir)?;
+        let oci = Image::open(vfs_mount)?;
         let rootfs = oci.open_rootfs_blob(rootfs_path)?;
 
         let layers =
@@ -41,8 +43,8 @@ pub(crate) fn open(vfsmount: Arc<Vfsmount>, rootfs_path: &CStr) -> Result<Puzzle
         Ok(PuzzleFS { oci, layers })
     }
 
-    pub(crate) fn find_inode(&mut self, ino: u64) -> Result<Inode> {
-        for layer in self.layers.iter_mut() {
+    pub(crate) fn find_inode(&self, ino: u64) -> Result<Inode> {
+        for layer in self.layers.iter() {
             if let Some(inode) = layer.find_inode(ino)? {
                 return Inode::new(layer, inode);
             }
@@ -52,7 +54,7 @@ pub(crate) fn find_inode(&mut self, ino: u64) -> Result<Inode> {
 }
 
 impl Inode {
-    fn new(layer: &mut MetadataBlob, inode: format::Inode) -> Result<Inode> {
+    fn new(layer: &MetadataBlob, inode: format::Inode) -> Result<Inode> {
         let mode = match inode.mode {
             format::InodeMode::Reg { offset } => {
                 let chunks = layer.read_file_chunks(offset)?;
@@ -64,7 +66,7 @@ fn new(layer: &mut MetadataBlob, inode: format::Inode) -> Result<Inode> {
                     layer
                         .read_dir_list(offset)?
                         .entries
-                        .iter_mut()
+                        .iter()
                         .map(|de| (de.name.try_clone().unwrap(), de.ino)),
                 )?;
                 // Unstable sort is used because it avoids memory allocation
@@ -94,3 +96,55 @@ pub(crate) enum InodeMode {
     Dir { entries: Vec<(Vec<u8>, Ino)> },
     Other,
 }
+
+pub(crate) fn file_read(
+    oci: &Image,
+    inode: &Inode,
+    offset: usize,
+    data: &mut [u8],
+) -> Result<usize> {
+    let chunks = match &inode.mode {
+        InodeMode::File { chunks } => chunks,
+        _ => return Err(WireFormatError::from_errno(ENOTDIR)),
+    };
+
+    // TODO: fix all this casting...
+    let end = offset + data.len();
+
+    let mut file_offset = 0;
+    let mut buf_offset = 0;
+    for chunk in chunks {
+        // have we read enough?
+        if file_offset > end {
+            break;
+        }
+
+        // should we skip this chunk?
+        if file_offset + (chunk.len as usize) < offset {
+            file_offset += chunk.len as usize;
+            continue;
+        }
+
+        let addl_offset = if offset > file_offset {
+            offset - file_offset
+        } else {
+            0
+        };
+
+        // ok, need to read this chunk; how much?
+        let left_in_buf = data.len() - buf_offset;
+        let to_read = min(left_in_buf, chunk.len as usize - addl_offset);
+
+        let start = buf_offset;
+        let finish = start + to_read;
+        file_offset += addl_offset;
+
+        // how many did we actually read?
+        let n = oci.fill_from_chunk(chunk.blob, addl_offset as u64, &mut data[start..finish])?;
+        file_offset += n;
+        buf_offset += n;
+    }
+
+    // discard any extra if we hit EOF
+    Ok(buf_offset)
+}
samples/rust/puzzle/oci.rs +26 -6
index becb2b868450..935beb168079 100644
@@ -1,19 +1,21 @@
-use crate::puzzle::error::Result;
+use crate::puzzle::error::{Result, WireFormatError};
+use crate::puzzle::types as format;
 use crate::puzzle::types::{Digest, MetadataBlob, Rootfs};
 use kernel::c_str;
 use kernel::file;
 use kernel::file::RegularFile;
 use kernel::mount::Vfsmount;
-use kernel::pr_info;
+use kernel::pr_debug;
+use kernel::prelude::ENOTSUPP;
 use kernel::str::{CStr, CString};
-use kernel::sync::Arc;
 
+#[derive(Debug)]
 pub(crate) struct Image {
-    vfs_mount: Arc<Vfsmount>,
+    pub(crate) vfs_mount: Vfsmount,
 }
 
 impl Image {
-    pub(crate) fn open(vfsmount: Arc<Vfsmount>) -> Result<Self> {
+    pub(crate) fn open(vfsmount: Vfsmount) -> Result<Self> {
         Ok(Image {
             vfs_mount: vfsmount,
         })
@@ -26,7 +28,7 @@ pub(crate) fn blob_path_relative(&self) -> &CStr {
     fn open_raw_blob(&self, digest: &Digest) -> Result<RegularFile> {
         let filename =
             CString::try_from_fmt(format_args!("{}/{digest}", self.blob_path_relative()))?;
-        pr_info!("trying to open {:?}\n", &filename);
+        pr_debug!("trying to open {:?}\n", &filename);
 
         let file = RegularFile::from_path_in_root_mnt(
             &self.vfs_mount,
@@ -48,4 +50,22 @@ pub(crate) fn open_rootfs_blob(&self, path: &CStr) -> Result<Rootfs> {
         let rootfs = Rootfs::open(self.open_raw_blob(&digest)?)?;
         Ok(rootfs)
     }
+
+    pub(crate) fn fill_from_chunk(
+        &self,
+        chunk: format::BlobRef,
+        addl_offset: u64,
+        buf: &mut [u8],
+    ) -> Result<usize> {
+        let digest = &<Digest>::try_from(chunk)?;
+
+        let blob = if chunk.compressed {
+            return Err(WireFormatError::KernelError(ENOTSUPP));
+        } else {
+            self.open_raw_blob(digest)?
+        };
+
+        let n = blob.read_with_offset(buf, chunk.offset + addl_offset)?;
+        Ok(n)
+    }
 }
samples/rust/puzzle/types.rs +5 -5
index 69d4f5dfa12c..353323f73ab1 100644
@@ -148,28 +148,28 @@ pub(crate) fn new(mut f: file::RegularFile) -> Result<MetadataBlob> {
         })
     }
 
-    pub(crate) fn seek_ref(&mut self, r: &BlobRef) -> Result<u64> {
+    pub(crate) fn seek_ref(&self, r: &BlobRef) -> Result<u64> {
         match r.kind {
             BlobRefKind::Other { .. } => Err(WireFormatError::SeekOtherError),
             BlobRefKind::Local => Ok(r.offset),
         }
     }
 
-    pub(crate) fn read_file_chunks(&mut self, offset: u64) -> Result<Vec<FileChunk>> {
+    pub(crate) fn read_file_chunks(&self, offset: u64) -> Result<Vec<FileChunk>> {
         read_one_from_slice::<FileChunkList>(&self.mmapped_region[offset as usize..])
             .map(|cl| cl.chunks)
     }
 
-    pub(crate) fn read_dir_list(&mut self, offset: u64) -> Result<DirList> {
+    pub(crate) fn read_dir_list(&self, offset: u64) -> Result<DirList> {
         read_one_from_slice(&self.mmapped_region[offset as usize..])
     }
 
-    pub(crate) fn read_inode_additional(&mut self, r: &BlobRef) -> Result<InodeAdditional> {
+    pub(crate) fn read_inode_additional(&self, r: &BlobRef) -> Result<InodeAdditional> {
         let offset = self.seek_ref(r)? as usize;
         read_one_from_slice(&self.mmapped_region[offset..])
     }
 
-    pub(crate) fn find_inode(&mut self, ino: Ino) -> Result<Option<Inode>> {
+    pub(crate) fn find_inode(&self, ino: Ino) -> Result<Option<Inode>> {
         let mut left = 0;
         let mut right = self.inode_count;
 
samples/rust/puzzlefs.rs +26 -33

128 lines not on this page · raw message

Signature · 1 line
2.40.1
#103 lore raw

PATCH 80/80 rust: puzzlefs: add oci_root_dir and image_manifest filesystem parameters

In reply to Ariel Miculas

These parameters are passed when mounting puzzlefs using '-o' option of mount:
-o oci_root_dir="/path/to/oci/dir"
-o image_manifest="root_hash_of_image_manifest"

For a particular manifest in the manifests array in index.json (located in the oci_root_dir), the root hash of the image manifest is found in the digest field.

It would be nicer if we could pass the tag, but we don't support json deserialization.

Example of mount:
mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
none /mnt

---
 samples/rust/puzzlefs.rs | 63 ++++++++++++++++++++++++++--------------
 1 file changed, 41 insertions(+), 22 deletions(-)
samples/rust/puzzlefs.rs +41 -22
index 897a996a6ab3..fbd091d9d00d 100644
@@ -7,6 +7,7 @@
 use kernel::{
     c_str, file, fs,
     io_buffer::IoBufferWriter,
+    str::CString,
     sync::{Arc, ArcBorrow},
 };
 
@@ -29,27 +30,29 @@ struct PuzzlefsInfo {
     puzzlefs: Arc<PuzzleFS>,
 }
 
+#[derive(Default)]
+struct PuzzleFsParams {
+    oci_root_dir: Option<CString>,
+    image_manifest: Option<CString>,
+}
+
 #[vtable]
 impl fs::Context<Self> for PuzzleFsModule {
-    type Data = ();
-
-    kernel::define_fs_params! {(),
-        {flag, "flag", |_, v| { pr_info!("flag passed-in: {v}\n"); Ok(()) } },
-        {flag_no, "flagno", |_, v| { pr_info!("flagno passed-in: {v}\n"); Ok(()) } },
-        {bool, "bool", |_, v| { pr_info!("bool passed-in: {v}\n"); Ok(()) } },
-        {u32, "u32", |_, v| { pr_info!("u32 passed-in: {v}\n"); Ok(()) } },
-        {u32oct, "u32oct", |_, v| { pr_info!("u32oct passed-in: {v}\n"); Ok(()) } },
-        {u32hex, "u32hex", |_, v| { pr_info!("u32hex passed-in: {v}\n"); Ok(()) } },
-        {s32, "s32", |_, v| { pr_info!("s32 passed-in: {v}\n"); Ok(()) } },
-        {u64, "u64", |_, v| { pr_info!("u64 passed-in: {v}\n"); Ok(()) } },
-        {string, "string", |_, v| { pr_info!("string passed-in: {v}\n"); Ok(()) } },
-        {enum, "enum", [("first", 10), ("second", 20)], |_, v| {
-            pr_info!("enum passed-in: {v}\n"); Ok(()) }
-        },
+    type Data = Box<PuzzleFsParams>;
+
+    kernel::define_fs_params! {Box<PuzzleFsParams>,
+        {string, "oci_root_dir", |s, v| {
+                                      s.oci_root_dir = Some(CString::try_from_fmt(format_args!("{v}"))?);
+                                      Ok(())
+                                  }},
+        {string, "image_manifest", |s, v| {
+                                      s.image_manifest = Some(CString::try_from_fmt(format_args!("{v}"))?);
+                                      Ok(())
+                                  }},
     }
 
-    fn try_new() -> Result {
-        Ok(())
+    fn try_new() -> Result<Self::Data> {
+        Ok(Box::try_new(PuzzleFsParams::default())?)
     }
 }
 
@@ -134,11 +137,27 @@ impl fs::Type for PuzzleFsModule {
     const FLAGS: i32 = fs::flags::USERNS_MOUNT;
     const DCACHE_BASED: bool = true;
 
-    fn fill_super(_data: (), sb: fs::NewSuperBlock<'_, Self>) -> Result<&fs::SuperBlock<Self>> {
-        let puzzlefs = PuzzleFS::open(
-            c_str!("/home/puzzlefs_oci"),
-            c_str!("2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b"),
-        );
+    fn fill_super(
+        data: Box<PuzzleFsParams>,
+        sb: fs::NewSuperBlock<'_, Self>,
+    ) -> Result<&fs::SuperBlock<Self>> {
+        let oci_root_dir = match data.oci_root_dir {
+            Some(val) => val,
+            None => {
+                pr_err!("missing oci_root_dir parameter!\n");
+                return Err(ENOTSUPP);
+            }
+        };
+
+        let image_manifest = match data.image_manifest {
+            Some(val) => val,
+            None => {
+                pr_err!("missing image_manifest parameter!\n");
+                return Err(ENOTSUPP);
+            }
+        };
+
+        let puzzlefs = PuzzleFS::open(&oci_root_dir, &image_manifest);
 
         if let Err(ref e) = puzzlefs {
             pr_info!("error opening puzzlefs {e}\n");
Signature · 1 line
2.40.1
#104 lore raw
In reply to Ariel Miculas
Quoted 4 lines · Ariel Miculas

This is a proof of concept driver written for the PuzzleFS next-generation container filesystem [1]. I've included a short abstract about puzzlefs further below. This driver is based on the rust-next

Thanks for preparing this RFC Ariel!

I have left some procedural notes here and there.

Cheers,
Miguel

#105 lore raw
In reply to Ariel Miculas
Quoted 3 lines · Ariel Miculas

Hi all!

This is a proof of concept driver written for the PuzzleFS

Uhm, the PuzzleFS filesystem isn't actually sent to fsdevel? Yet I see tons of patches in there that add wrappers to our core fs data structures. I even see a ton of new files that all fall clearly into fsdevel territory:

create mode 100644 rust/kernel/cred.rs
create mode 100644 rust/kernel/delay.rs
create mode 100644 rust/kernel/driver.rs
create mode 100644 rust/kernel/file.rs
create mode 100644 rust/kernel/fs.rs
create mode 100644 rust/kernel/fs/param.rs
create mode 100644 rust/kernel/io_buffer.rs
create mode 100644 rust/kernel/iov_iter.rs
create mode 100644 rust/kernel/mm.rs
create mode 100644 rust/kernel/mount.rs
create mode 100644 rust/kernel/pages.rs

There's also quite a lot of new mm/ in there, no?

Any wrappers and code for core fs should be maintained as part of fs. Rust shouldn't become a way to avoid our reviews once you have a few wrappers added somewhere.

Quoted 677 lines

next-generation container filesystem [1]. I've included a short abstract about puzzlefs further below. This driver is based on the rust-next branch, on top of which I've backported the filesystem abstractions from Wedson Almeida Filho [2][3] and Miguel Ojeda's third-party crates
support: proc-macro2, quote, syn, serde and serde_derive [4]. I've added the additional third-party crates serde_cbor[5] and hex [6]. Then I've adapted the user space puzzlefs code [1] so that the puzzlefs kernel module could present the directory hierarchy and implement the basic read functionality.
For some additional context, puzzlefs was started by Tycho Andersen and it's the successor of atomfs. This FOSDEM presentation from 2019 [12] covers the rationale for a new oci image format and presents a higher level overview of our goals with puzzlefs.
I've split the rest of the cover letter in following sections (using a markdown style):
* Example: it describes a practical example of what was achieved
* Limitations: it presents the existing limitations of this POC
* Upstreaming steps: it describes the steps needed for upstreaming this driver
* Setup: it shows how to setup the necessary environment for testing the puzzlefs driver
* Puzzlefs abstract: it provides a short overview of puzzlefs

# Example
An example is provided below:

```
~ # cat /proc/filesystems | grep puzzlefs
nodev puzzlefs
~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o image_manifest="2d 6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" none /mnt ~ # ls -lR /mnt/
/mnt/:
total 0

drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-1
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-2
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-3
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-4
(quoted diff) +0 -2
-rw-r--r--    1 0        0                0 Jun  8 12:26 file1
-rw-r--r--    1 0        0                0 Jun  8 12:26 file2

/mnt/dir-1:
total 0

/mnt/dir-2:
total 0

/mnt/dir-3:
total 0

/mnt/dir-4:
total 0
~ # cat /mnt/file2
ana are mere bla bla bla
~ # wc /mnt/file1

      202       202      5454 /mnt/file1
```

In this example, /home/puzzlefs_oci is the puzzlefs oci directory:
```

~ # ls -lR /home/puzzlefs_oci/
/home/puzzlefs_oci/:
total 8
drwxr-xr-x 3 1000 1000 0 Jun 8 14:33 blobs

(quoted diff) +0 -2
-rw-r--r--    1 1000     1000           266 Jun  8 14:33 index.json
-rw-r--r--    1 1000     1000            37 Jun  8 14:33 oci-layout

/home/puzzlefs_oci/blobs:
total 0
drwxr-xr-x 2 1000 1000 0 Jun 8 14:33 sha256

/home/puzzlefs_oci/blobs/sha256:
total 16

(quoted diff) +0 -3
-rw-------    1 1000     1000            89 Jun  8 14:33 2d6602d678140540dc7e96de652a76a8b16eb
-rw-------    1 1000     1000           925 Jun  8 14:33 4df03518eea406343dbb55046720f6a478881
-rw-------    1 1000     1000          5479 Jun  8 14:33 d86a87b19bd9a2fec0d31687c1d669cdb59eb
```

`2d6602d678140540dc7e96de652a76a8b16eb` is the puzzlefs image manifest
hash for the first_try tag:
```
$ cat /tmp/oci-simple/index.json | jq .
{
  "schemaVersion": -1,
  "manifests": [
    {
      "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
      "size": 89,
      "media_type": "application/vnd.puzzlefs.image.rootfs.v1",
      "annotations": {
        "org.opencontainers.image.ref.name": "first_try"
      }
    }
  ],
  "annotations": {}
}
```

I will describe how to build a puzzlefs image in the Setup section, at step 5.

# Limitations
One limitation is that the puzzlefs driver doesn't implement any lookup functionality and instead it inserts every directory entry into the dcache during init (see the DCACHE_BASED constant). This is similar to how the sample rust_fs driver works, but the goal is to implement proper lookup functions. However, more filesystem abstractions need to be implemented before this can be achieved.

# Upstreaming steps
Before the puzzlefs driver can be upstreamed, the following need to be merged:
* Wedson's filesystem abstractions [3]
* the necessary third-party crates [4] (with the preliminary discussion about whether this is desirable)

# Setup
My setup is based on Wedson's tutorial [8]. Next, I will describe the necessary steps to build an initrd and run a custom kernel under qemu.

1. Get the linux rust-next branch [9] and apply this patchset

2. Follow the rust quickstart guide [10]

3. configure and build the kernel

```
$ make LLVM=1 allnoconfig qemu-busybox-min.config rust.config
$ make LLVM=1 -j$(nproc)
```

4. setup busybox

```
git clone git://git.busybox.net/busybox
cd busybox
make menuconfig # enable 'Build static binary' config
make
make install
```

This will create the _install directory with the rootfs inside it.

5. create a home directory in the rootfs and copy a puzzlefs oci directory in home/puzzlefs_oci
To create a puzzlefs oci directory:
* download this custom puzzlefs repository [11] (it's custom because we want to build an image without verity data)
* run make release
* create a simple filesystem structure with a few directories and files (I've created one at ../test-puzzlefs/simple_rootfs)
* build a puzzlefs oci image at ~/work/busybox/_install/home/puzzlefs_oci (replace this path with your busybox path) with the tag first_try:

```
$ target/release/puzzlefs build --omit-verity \
../test-puzzlefs/simple_rootfs ~/work/busybox/_install/home/puzzlefs_oci \
first_try
```
* get first_try's image manifest from index.json (inside `puzzlefs_oci`)
```
$ cat index.json | jq . | grep digest
      "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
```

6. add the following 'init' script in the busybox rootfs (rootfs path defaults to `./_install'):

```
#!/bin/sh
mount -t devtmpfs none /dev
mkdir -p /proc
mount -t proc none /proc

ifconfig lo up
udhcpc -i eth0

mkdir /mnt
mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
none /mnt

setsid sh -c 'exec sh -l </dev/ttyS0 >/dev/ttyS0 2>&1'
```

Make sure to replace the image_manifest with your own digest. This init script will be passed to rdinit in the kernel command line.

7. generate the initramfs

Assuming busybox is in ~/work/busybox:

```
cd ~/work/busybox/_install && find . | cpio -H newc -o | gzip > ../ramdisk.img
```

This will generate a compressed ramdisk image in
~/work/busybox/ramdisk.img.

8. run with qemu (this assumes the linux tree is at '../linux' and busybox is at '../busybox'):

```
qemu-system-x86_64 \
    -accel kvm \
    -cpu host \
    -m 4G \
    -initrd ../busybox/ramdisk.img \
    -kernel ../linux/arch/x86/boot/bzImage \
    -nographic \
    -append 'console=ttyS0 nokaslr debug rdinit=/init' \
    -nic user,model=rtl8139 \
    -no-reboot
```

9. Check whether puzzlefs has been successfully mounted

```
~ # mount | grep puzzlefs
none on /mnt type puzzlefs (rw,relatime)
~ # ls /mnt/
dir-1  dir-2  dir-3  dir-4  file1  file2
```

# Puzzlefs abstract
Puzzlefs [1] is a container filesystem designed to address the limitations of the existing OCI format. The main goals of the project are reduced duplication, reproducible image builds, direct mounting support and memory safety guarantees, some inspired by the OCIv2 design document [7].

Reduced duplication is achieved using the content defined chunking algorithm FastCDC. This implementation allows chunks to be shared among layers. Building a new layer starting from an existing one allows reusing most of the chunks.

Another goal of the project is reproducible image builds, which is achieved by defining a canonical representation of the image format.

Direct mounting support is a key feature of puzzlefs and, together with fs-verity, it provides data integrity. Currently, puzzlefs is implemented as a userspace filesystem (FUSE). A read-only kernel filesystem driver is underway.

Lastly, memory safety is critical to puzzlefs, leading to the decision to implement it in Rust. Another goal is to share the same code between user space and kernel space in order to provide one secure implementation.

[1] https://github.com/anuvu/puzzlefs
[2] https://github.com/wedsonaf/linux/tree/fs
[3] https://github.com/Rust-for-Linux/linux/issues/1004
[4] https://github.com/Rust-for-Linux/linux/pull/1007
[5] https://docs.rs/serde_cbor/latest/serde_cbor/
[6] https://docs.rs/hex/0.4.3/hex/
[7] https://hackmd.io/@cyphar/ociv2-brainstorm
[8] https://www.youtube.com/watch?v=tPs1uRqOnlk
[9] https://github.com/Rust-for-Linux/linux/tree/rust-next
[10] https://docs.kernel.org/rust/quick-start.html
[11] https://github.com/ariel-miculas/puzzlefs/tree/support-no-verity-data
[12] https://archive.fosdem.org/2019/schedule/event/containers_atomfs/

Ariel Miculas (58):
rust: kernel: add libraries required by the filesystem abstractions
rust: kernel: backport the delay module from the rust branch
rust: kernel: add container_of macro
rust: kernel: add offset_of macro
drop: Add crate::pr_warn declaration
rust: kernel: rename from_kernel_errno to from_errno
rust: kernel: Rename from_pointer to from_foreing and into_pointer to
into_foreign
rust: kernel: add count_paren_items macro, needed by define_fs_params
macro
rust: helpers: add missing rust helper 'alloc_pages'
kernel: configs: add qemu-busybox-min.config
rust: kernel: format the rust code
samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs
kernel: configs: enable rust samples in rust.config
Add SAMPLE_RUST_SERDE in rust.config
rust: kernel: fix compile errors after rebase to rust-next
rust: serde_cbor: import crate
rust: serde_cbor: add SPDX License Identifiers
rust: serde_cbor: add no_fp_fmt_parse support
rust: Kbuild: enable serde_cbor
samples: rust: add cbor serialize/deserialize example
rust: serde_cbor: add support for serde_cbor's from_slice method by
using a custom alloc_kernel feature
rust: serde: add support for deserializing Vec with kernel_alloc
feature
rust: file: Replace UnsafeCell with Opaque for File
rust: kernel: implement fmt::Debug for CString
samples: puzzlefs: rename RustFs to PuzzleFs
samples: puzzlefs: add basic deserializing support for the puzzlefs
metadata
rust: file: present the filesystem context to the open function
rust: kernel: add an abstraction over vfsmount to allow cloning a new
private mount
rust: file: add from_path, from_path_in_root_mnt and read_with_offset
methods to File
samples: puzzlefs: pass the Vfsmount structure from open to read and
return the contents of the data file inside /home/puzzlefs_oci
rust: file: move from_path, from_path_in_root_mnt and read_with_offset
methods to a RegularFile newtype
rust: file: ensure RegularFile can only create regular files
rust: file: add get_pos method to RegularFile
rust: file: add methods read_to_end, get_file_size and update_pos to
RegularFile
rust: file: define a minimal Read trait and implement it for
RegularFile
samples: puzzlefs: add cbor_get_array_size method
samples: puzzlefs: add KernelError to WireFormatError and implement
From conversion
samples: puzzlefs: implement new for MetadataBlob
samples: puzzlefs: build puzzlefs into the kernel, thus avoiding the
need to export rust symbols
rust: alloc: add try_clone for Vec<T>
rust: alloc: add from_iter_fallible for Vec<T>
samples: puzzlefs: implement to_errno and from_errno for
WireFormatError
samples: puzzlefs: add TryReserveError (and from conversion) to
WireFormatError
samples: puzzlefs: add higher level inode related functionality
samples: puzzlefs: populate the directory entries with the inodes from
the puzzlefs metadata file
rust: hex: import crate
rust: hex: add SPDX license identifiers
rust: Kbuild: enable hex
rust: hex: implement FromHex trait and hex::decode using a custom
kernel_alloc feature
rust: hex: add encode_hex_iter and encode_hex_upper_iter methods
rust: puzzlefs: add HexError to WireFormatError and implement the From
conversion
rust: puzzlefs: display the error value for
WireFormatError::KernelError
samples: puzzlefs: add Rootfs and Digest structs to types.rs
samples: puzzlefs: implement the conversion from WireFormatError to
kernel::error::Error
rust: puzzlefs: read the puzzlefs image manifest instead of an
individual metadata layer
rust: puzzlefs: rename PuzzleFs to PuzzleFsModule to avoid confusion
with the PuzzleFS struct
rust: puzzlefs: add support for reading files
rust: puzzlefs: add oci_root_dir and image_manifest filesystem
parameters

Miguel Ojeda (15):
rust: proc-macro2: import crate
rust: proc-macro2: add SPDX License Identifiers
rust: proc-macro2: remove unicode_ident dependency
rust: quote: import crate
rust: quote: add SPDX License Identifiers
rust: syn: import crate
rust: syn: add SPDX License Identifiers
rust: syn: remove unicode-ident dependency
rust: serde: import crate
rust: serde: add no_fp_fmt_parse support
rust: serde: add SPDX License Identifiers
rust: serde_derive: import crate
rust: serde_derive: add SPDX License Identifiers
rust: Kbuild: enable proc-macro2, quote, syn, serde and
serde_derive
rust: test serde support

Wedson Almeida Filho (7):
rust: add definitions for ref-counted inodes and dentries
rust: add ability to register a file system
rust: define fs context
rust: add support for file system parameters
rust: allow fs driver to initialise new superblocks
rust: add module_fs macro
WIP: rust: allow fs to be populated

 Makefile                                  |   14 +-
 arch/x86/configs/qemu-busybox-min.config  |   11 +
 kernel/configs/qemu-busybox-min.config    |   56 +
 kernel/configs/rust.config                |   11 +
 rust/.gitignore                           |    1 +
 rust/Makefile                             |  232 +-
 rust/alloc/vec/mod.rs                     |   48 +
 rust/bindings/bindings_helper.h           |   14 +
 rust/bindings/lib.rs                      |    5 +
 rust/helpers.c                            |   76 +
 rust/hex/error.rs                         |   78 +
 rust/hex/lib.rs                           |  506 +++
 rust/hex/serde.rs                         |  104 +
 rust/kernel/cred.rs                       |   46 +
 rust/kernel/delay.rs                      |  104 +
 rust/kernel/driver.rs                     |   28 +
 rust/kernel/error.rs                      |   52 +-
 rust/kernel/file.rs                       | 1117 ++++++
 rust/kernel/fs.rs                         | 1478 ++++++++
 rust/kernel/fs/param.rs                   |  558 +++
 rust/kernel/io_buffer.rs                  |  153 +
 rust/kernel/iov_iter.rs                   |   81 +
 rust/kernel/lib.rs                        |   83 +
 rust/kernel/mm.rs                         |  149 +
 rust/kernel/mount.rs                      |   66 +
 rust/kernel/pages.rs                      |  144 +
 rust/kernel/str.rs                        |    6 +
 rust/kernel/test_serde.rs                 |   26 +
 rust/kernel/test_serde/de.rs              |  439 +++
 rust/kernel/test_serde/error.rs           |   73 +
 rust/kernel/test_serde/ser.rs             |  466 +++
 rust/kernel/user_ptr.rs                   |  175 +
 rust/proc-macro2/detection.rs             |   77 +
 rust/proc-macro2/fallback.rs              | 1004 ++++++
 rust/proc-macro2/lib.rs                   | 1341 ++++++++
 rust/proc-macro2/marker.rs                |   20 +
 rust/proc-macro2/parse.rs                 |  874 +++++
 rust/proc-macro2/rcvec.rs                 |  144 +
 rust/proc-macro2/wrapper.rs               |  996 ++++++
 rust/quote/ext.rs                         |  112 +
 rust/quote/format.rs                      |  170 +
 rust/quote/ident_fragment.rs              |   88 +
 rust/quote/lib.rs                         | 1436 ++++++++
 rust/quote/runtime.rs                     |  440 +++
 rust/quote/spanned.rs                     |   45 +
 rust/quote/to_tokens.rs                   |  211 ++
 rust/serde/de/format.rs                   |   32 +
 rust/serde/de/ignored_any.rs              |  246 ++
 rust/serde/de/impls.rs                    | 2755 +++++++++++++++
 rust/serde/de/mod.rs                      | 2313 +++++++++++++
 rust/serde/de/seed.rs                     |   21 +
 rust/serde/de/utf8.rs                     |   48 +
 rust/serde/de/value.rs                    | 1718 ++++++++++
 rust/serde/integer128.rs                  |   84 +
 rust/serde/lib.rs                         |  351 ++
 rust/serde/macros.rs                      |  238 ++
 rust/serde/private/de.rs                  | 2997 ++++++++++++++++
 rust/serde/private/doc.rs                 |  161 +
 rust/serde/private/mod.rs                 |   52 +
 rust/serde/private/ser.rs                 | 1316 +++++++
 rust/serde/private/size_hint.rs           |   23 +
 rust/serde/ser/fmt.rs                     |  180 +
 rust/serde/ser/impls.rs                   |  987 ++++++
 rust/serde/ser/impossible.rs              |  218 ++
 rust/serde/ser/mod.rs                     | 1992 +++++++++++
 rust/serde/std_error.rs                   |   50 +
 rust/serde_cbor/de.rs                     | 1370 ++++++++
 rust/serde_cbor/error.rs                  |  320 ++
 rust/serde_cbor/lib.rs                    |  371 ++
 rust/serde_cbor/read.rs                   |  647 ++++
 rust/serde_cbor/ser.rs                    |  748 ++++
 rust/serde_cbor/tags.rs                   |  224 ++
 rust/serde_cbor/value/de.rs               |  168 +
 rust/serde_cbor/value/mod.rs              |  158 +
 rust/serde_cbor/value/ser.rs              |  447 +++
 rust/serde_cbor/write.rs                  |  177 +
 rust/serde_derive/bound.rs                |  408 +++
 rust/serde_derive/de.rs                   | 3148 +++++++++++++++++
 rust/serde_derive/dummy.rs                |   46 +
 rust/serde_derive/fragment.rs             |   76 +
 rust/serde_derive/internals/ast.rs        |  204 ++
 rust/serde_derive/internals/attr.rs       | 1908 +++++++++++
 rust/serde_derive/internals/case.rs       |  199 ++
 rust/serde_derive/internals/check.rs      |  445 +++
 rust/serde_derive/internals/ctxt.rs       |   64 +
 rust/serde_derive/internals/mod.rs        |   30 +
 rust/serde_derive/internals/receiver.rs   |  287 ++
 rust/serde_derive/internals/respan.rs     |   18 +
 rust/serde_derive/internals/symbol.rs     |   71 +
 rust/serde_derive/lib.rs                  |  112 +
 rust/serde_derive/pretend.rs              |  203 ++
 rust/serde_derive/ser.rs                  | 1342 ++++++++
 rust/serde_derive/this.rs                 |   34 +
 rust/serde_derive/try.rs                  |   26 +
 rust/syn/attr.rs                          |  664 ++++
 rust/syn/await.rs                         |    4 +
 rust/syn/bigint.rs                        |   68 +
 rust/syn/buffer.rs                        |  400 +++
 rust/syn/custom_keyword.rs                |  255 ++
 rust/syn/custom_punctuation.rs            |  302 ++
 rust/syn/data.rs                          |  495 +++
 rust/syn/derive.rs                        |  276 ++
 rust/syn/discouraged.rs                   |  196 ++
 rust/syn/error.rs                         |  430 +++
 rust/syn/export.rs                        |   41 +
 rust/syn/expr.rs                          | 3560 +++++++++++++++++++
 rust/syn/ext.rs                           |  141 +
 rust/syn/file.rs                          |  127 +
 rust/syn/gen/clone.rs                     | 2243 ++++++++++++
 rust/syn/gen/debug.rs                     | 3044 +++++++++++++++++
 rust/syn/gen/eq.rs                        | 2197 ++++++++++++
 rust/syn/gen/fold.rs                      | 3343 ++++++++++++++++++
 rust/syn/gen/hash.rs                      | 2871 ++++++++++++++++
 rust/syn/gen/visit.rs                     | 3788 +++++++++++++++++++++
 rust/syn/gen/visit_mut.rs                 | 3788 +++++++++++++++++++++
 rust/syn/gen_helper.rs                    |  156 +
 rust/syn/generics.rs                      | 1339 ++++++++
 rust/syn/group.rs                         |  284 ++
 rust/syn/ident.rs                         |  103 +
 rust/syn/item.rs                          | 3315 ++++++++++++++++++
 rust/syn/lib.rs                           |  985 ++++++
 rust/syn/lifetime.rs                      |  156 +
 rust/syn/lit.rs                           | 1602 +++++++++
 rust/syn/lookahead.rs                     |  171 +
 rust/syn/mac.rs                           |  221 ++
 rust/syn/macros.rs                        |  179 +
 rust/syn/op.rs                            |  236 ++
 rust/syn/parse.rs                         | 1316 +++++++
 rust/syn/parse_macro_input.rs             |  181 +
 rust/syn/parse_quote.rs                   |  169 +
 rust/syn/pat.rs                           |  929 +++++
 rust/syn/path.rs                          |  856 +++++
 rust/syn/print.rs                         |   18 +
 rust/syn/punctuated.rs                    | 1070 ++++++
 rust/syn/reserved.rs                      |   46 +
 rust/syn/sealed.rs                        |    6 +
 rust/syn/span.rs                          |   69 +
 rust/syn/spanned.rs                       |  116 +
 rust/syn/stmt.rs                          |  351 ++
 rust/syn/thread.rs                        |   43 +
 rust/syn/token.rs                         | 1015 ++++++
 rust/syn/tt.rs                            |  109 +
 rust/syn/ty.rs                            | 1288 +++++++
 rust/syn/verbatim.rs                      |   35 +
 rust/syn/whitespace.rs                    |   67 +
 samples/rust/Kconfig                      |   28 +
 samples/rust/Makefile                     |    3 +
 samples/rust/local_data_format/de.rs      |  422 +++
 samples/rust/local_data_format/error.rs   |   73 +
 samples/rust/local_data_format/ser.rs     |  443 +++
 samples/rust/puzzle.rs                    |    4 +
 samples/rust/puzzle/error.rs              |   91 +
 samples/rust/puzzle/inode.rs              |  150 +
 samples/rust/puzzle/oci.rs                |   71 +
 samples/rust/puzzle/types.rs              |  389 +++
 samples/rust/puzzle/types/cbor_helpers.rs |   50 +
 samples/rust/puzzlefs.rs                  |  220 ++
 samples/rust/rust_fs.rs                   |  105 +
 samples/rust/rust_serde.rs                |  125 +
 scripts/Makefile.build                    |    4 +-
 160 files changed, 89204 insertions(+), 29 deletions(-)
 create mode 100644 arch/x86/configs/qemu-busybox-min.config
 create mode 100644 kernel/configs/qemu-busybox-min.config
 create mode 100644 rust/hex/error.rs
 create mode 100644 rust/hex/lib.rs
 create mode 100644 rust/hex/serde.rs
 create mode 100644 rust/kernel/cred.rs
 create mode 100644 rust/kernel/delay.rs
 create mode 100644 rust/kernel/driver.rs
 create mode 100644 rust/kernel/file.rs
 create mode 100644 rust/kernel/fs.rs
 create mode 100644 rust/kernel/fs/param.rs
 create mode 100644 rust/kernel/io_buffer.rs
 create mode 100644 rust/kernel/iov_iter.rs
 create mode 100644 rust/kernel/mm.rs
 create mode 100644 rust/kernel/mount.rs
 create mode 100644 rust/kernel/pages.rs
 create mode 100644 rust/kernel/test_serde.rs
 create mode 100644 rust/kernel/test_serde/de.rs
 create mode 100644 rust/kernel/test_serde/error.rs
 create mode 100644 rust/kernel/test_serde/ser.rs
 create mode 100644 rust/kernel/user_ptr.rs
 create mode 100644 rust/proc-macro2/detection.rs
 create mode 100644 rust/proc-macro2/fallback.rs
 create mode 100644 rust/proc-macro2/lib.rs
 create mode 100644 rust/proc-macro2/marker.rs
 create mode 100644 rust/proc-macro2/parse.rs
 create mode 100644 rust/proc-macro2/rcvec.rs
 create mode 100644 rust/proc-macro2/wrapper.rs
 create mode 100644 rust/quote/ext.rs
 create mode 100644 rust/quote/format.rs
 create mode 100644 rust/quote/ident_fragment.rs
 create mode 100644 rust/quote/lib.rs
 create mode 100644 rust/quote/runtime.rs
 create mode 100644 rust/quote/spanned.rs
 create mode 100644 rust/quote/to_tokens.rs
 create mode 100644 rust/serde/de/format.rs
 create mode 100644 rust/serde/de/ignored_any.rs
 create mode 100644 rust/serde/de/impls.rs
 create mode 100644 rust/serde/de/mod.rs
 create mode 100644 rust/serde/de/seed.rs
 create mode 100644 rust/serde/de/utf8.rs
 create mode 100644 rust/serde/de/value.rs
 create mode 100644 rust/serde/integer128.rs
 create mode 100644 rust/serde/lib.rs
 create mode 100644 rust/serde/macros.rs
 create mode 100644 rust/serde/private/de.rs
 create mode 100644 rust/serde/private/doc.rs
 create mode 100644 rust/serde/private/mod.rs
 create mode 100644 rust/serde/private/ser.rs
 create mode 100644 rust/serde/private/size_hint.rs
 create mode 100644 rust/serde/ser/fmt.rs
 create mode 100644 rust/serde/ser/impls.rs
 create mode 100644 rust/serde/ser/impossible.rs
 create mode 100644 rust/serde/ser/mod.rs
 create mode 100644 rust/serde/std_error.rs
 create mode 100644 rust/serde_cbor/de.rs
 create mode 100644 rust/serde_cbor/error.rs
 create mode 100644 rust/serde_cbor/lib.rs
 create mode 100644 rust/serde_cbor/read.rs
 create mode 100644 rust/serde_cbor/ser.rs
 create mode 100644 rust/serde_cbor/tags.rs
 create mode 100644 rust/serde_cbor/value/de.rs
 create mode 100644 rust/serde_cbor/value/mod.rs
 create mode 100644 rust/serde_cbor/value/ser.rs
 create mode 100644 rust/serde_cbor/write.rs
 create mode 100644 rust/serde_derive/bound.rs
 create mode 100644 rust/serde_derive/de.rs
 create mode 100644 rust/serde_derive/dummy.rs
 create mode 100644 rust/serde_derive/fragment.rs
 create mode 100644 rust/serde_derive/internals/ast.rs
 create mode 100644 rust/serde_derive/internals/attr.rs
 create mode 100644 rust/serde_derive/internals/case.rs
 create mode 100644 rust/serde_derive/internals/check.rs
 create mode 100644 rust/serde_derive/internals/ctxt.rs
 create mode 100644 rust/serde_derive/internals/mod.rs
 create mode 100644 rust/serde_derive/internals/receiver.rs
 create mode 100644 rust/serde_derive/internals/respan.rs
 create mode 100644 rust/serde_derive/internals/symbol.rs
 create mode 100644 rust/serde_derive/lib.rs
 create mode 100644 rust/serde_derive/pretend.rs
 create mode 100644 rust/serde_derive/ser.rs
 create mode 100644 rust/serde_derive/this.rs
 create mode 100644 rust/serde_derive/try.rs
 create mode 100644 rust/syn/attr.rs
 create mode 100644 rust/syn/await.rs
 create mode 100644 rust/syn/bigint.rs
 create mode 100644 rust/syn/buffer.rs
 create mode 100644 rust/syn/custom_keyword.rs
 create mode 100644 rust/syn/custom_punctuation.rs
 create mode 100644 rust/syn/data.rs
 create mode 100644 rust/syn/derive.rs
 create mode 100644 rust/syn/discouraged.rs
 create mode 100644 rust/syn/error.rs
 create mode 100644 rust/syn/export.rs
 create mode 100644 rust/syn/expr.rs
 create mode 100644 rust/syn/ext.rs
 create mode 100644 rust/syn/file.rs
 create mode 100644 rust/syn/gen/clone.rs
 create mode 100644 rust/syn/gen/debug.rs
 create mode 100644 rust/syn/gen/eq.rs
 create mode 100644 rust/syn/gen/fold.rs
 create mode 100644 rust/syn/gen/hash.rs
 create mode 100644 rust/syn/gen/visit.rs
 create mode 100644 rust/syn/gen/visit_mut.rs
 create mode 100644 rust/syn/gen_helper.rs
 create mode 100644 rust/syn/generics.rs
 create mode 100644 rust/syn/group.rs
 create mode 100644 rust/syn/ident.rs
 create mode 100644 rust/syn/item.rs
 create mode 100644 rust/syn/lib.rs
 create mode 100644 rust/syn/lifetime.rs
 create mode 100644 rust/syn/lit.rs
 create mode 100644 rust/syn/lookahead.rs
 create mode 100644 rust/syn/mac.rs
 create mode 100644 rust/syn/macros.rs
 create mode 100644 rust/syn/op.rs
 create mode 100644 rust/syn/parse.rs
 create mode 100644 rust/syn/parse_macro_input.rs
 create mode 100644 rust/syn/parse_quote.rs
 create mode 100644 rust/syn/pat.rs
 create mode 100644 rust/syn/path.rs
 create mode 100644 rust/syn/print.rs
 create mode 100644 rust/syn/punctuated.rs
 create mode 100644 rust/syn/reserved.rs
 create mode 100644 rust/syn/sealed.rs
 create mode 100644 rust/syn/span.rs
 create mode 100644 rust/syn/spanned.rs
 create mode 100644 rust/syn/stmt.rs
 create mode 100644 rust/syn/thread.rs
 create mode 100644 rust/syn/token.rs
 create mode 100644 rust/syn/tt.rs
 create mode 100644 rust/syn/ty.rs
 create mode 100644 rust/syn/verbatim.rs
 create mode 100644 rust/syn/whitespace.rs
 create mode 100644 samples/rust/local_data_format/de.rs
 create mode 100644 samples/rust/local_data_format/error.rs
 create mode 100644 samples/rust/local_data_format/ser.rs
 create mode 100644 samples/rust/puzzle.rs
 create mode 100644 samples/rust/puzzle/error.rs
 create mode 100644 samples/rust/puzzle/inode.rs
 create mode 100644 samples/rust/puzzle/oci.rs
 create mode 100644 samples/rust/puzzle/types.rs
 create mode 100644 samples/rust/puzzle/types/cbor_helpers.rs
 create mode 100644 samples/rust/puzzlefs.rs
 create mode 100644 samples/rust/rust_fs.rs
 create mode 100644 samples/rust/rust_serde.rs
Signature · 1 line
2.40.1
#106 lore raw

Hello Christian,

I didn't send these patches to a wider audience because this is an initial prototype of the PuzzleFS driver, and it has a few prerequisites before it could be even considered for merging. First of all, the rust filesystem abstractions and their dependencies need to be upstreamed, then there needs to be a discussion regarding the inclusion of third-party crates in the linux kernel.

My plan was to send these patches to the rust-for-linux mailing list and then start a discussion with Miguel Ojeda regarding the upstreaming approach.
There are a lot of new files added in this patch series because I've included all the dependencies required so that my patches could be applied to the rust-next branch, but these dependencies will most likely need to be upstreamed separately.

It was never my intention to avoid your reviews, should I also send subsequent patches to linux-fsdevel, even if they're in the early stages of development?

Regards,
Ariel
________________________________
From: Christian Brauner <[email protected]>
Sent: Friday, June 9, 2023 1:36 PM
To: Ariel Miculas (amiculas) <[email protected]>; [email protected] <[email protected]>
Cc: [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 3 lines · Ariel Miculas

Hi all!

This is a proof of concept driver written for the PuzzleFS

Uhm, the PuzzleFS filesystem isn't actually sent to fsdevel? Yet I see tons of patches in there that add wrappers to our core fs data structures. I even see a ton of new files that all fall clearly into fsdevel territory:

create mode 100644 rust/kernel/cred.rs
create mode 100644 rust/kernel/delay.rs
create mode 100644 rust/kernel/driver.rs
create mode 100644 rust/kernel/file.rs
create mode 100644 rust/kernel/fs.rs
create mode 100644 rust/kernel/fs/param.rs
create mode 100644 rust/kernel/io_buffer.rs
create mode 100644 rust/kernel/iov_iter.rs
create mode 100644 rust/kernel/mm.rs
create mode 100644 rust/kernel/mount.rs
create mode 100644 rust/kernel/pages.rs

There's also quite a lot of new mm/ in there, no?

Any wrappers and code for core fs should be maintained as part of fs. Rust shouldn't become a way to avoid our reviews once you have a few wrappers added somewhere.

Quoted 677 lines

next-generation container filesystem [1]. I've included a short abstract about puzzlefs further below. This driver is based on the rust-next branch, on top of which I've backported the filesystem abstractions from Wedson Almeida Filho [2][3] and Miguel Ojeda's third-party crates
support: proc-macro2, quote, syn, serde and serde_derive [4]. I've added the additional third-party crates serde_cbor[5] and hex [6]. Then I've adapted the user space puzzlefs code [1] so that the puzzlefs kernel module could present the directory hierarchy and implement the basic read functionality.
For some additional context, puzzlefs was started by Tycho Andersen and it's the successor of atomfs. This FOSDEM presentation from 2019 [12] covers the rationale for a new oci image format and presents a higher level overview of our goals with puzzlefs.
I've split the rest of the cover letter in following sections (using a markdown style):
* Example: it describes a practical example of what was achieved
* Limitations: it presents the existing limitations of this POC
* Upstreaming steps: it describes the steps needed for upstreaming this driver
* Setup: it shows how to setup the necessary environment for testing the puzzlefs driver
* Puzzlefs abstract: it provides a short overview of puzzlefs

# Example
An example is provided below:

```
~ # cat /proc/filesystems | grep puzzlefs
nodev puzzlefs
~ # mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o image_manifest="2d 6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" none /mnt ~ # ls -lR /mnt/
/mnt/:
total 0

drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-1
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-2
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-3
drwxr-xr-x    2 0        0                0 Jun  8 12:26 dir-4
(quoted diff) +0 -2
-rw-r--r--    1 0        0                0 Jun  8 12:26 file1
-rw-r--r--    1 0        0                0 Jun  8 12:26 file2

/mnt/dir-1:
total 0

/mnt/dir-2:
total 0

/mnt/dir-3:
total 0

/mnt/dir-4:
total 0
~ # cat /mnt/file2
ana are mere bla bla bla
~ # wc /mnt/file1

      202       202      5454 /mnt/file1
```

In this example, /home/puzzlefs_oci is the puzzlefs oci directory:
```

~ # ls -lR /home/puzzlefs_oci/
/home/puzzlefs_oci/:
total 8
drwxr-xr-x 3 1000 1000 0 Jun 8 14:33 blobs

(quoted diff) +0 -2
-rw-r--r--    1 1000     1000           266 Jun  8 14:33 index.json
-rw-r--r--    1 1000     1000            37 Jun  8 14:33 oci-layout

/home/puzzlefs_oci/blobs:
total 0
drwxr-xr-x 2 1000 1000 0 Jun 8 14:33 sha256

/home/puzzlefs_oci/blobs/sha256:
total 16

(quoted diff) +0 -3
-rw-------    1 1000     1000            89 Jun  8 14:33 2d6602d678140540dc7e96de652a76a8b16eb
-rw-------    1 1000     1000           925 Jun  8 14:33 4df03518eea406343dbb55046720f6a478881
-rw-------    1 1000     1000          5479 Jun  8 14:33 d86a87b19bd9a2fec0d31687c1d669cdb59eb
```

`2d6602d678140540dc7e96de652a76a8b16eb` is the puzzlefs image manifest
hash for the first_try tag:
```
$ cat /tmp/oci-simple/index.json | jq .
{
  "schemaVersion": -1,
  "manifests": [
    {
      "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
      "size": 89,
      "media_type": "application/vnd.puzzlefs.image.rootfs.v1",
      "annotations": {
        "org.opencontainers.image.ref.name": "first_try"
      }
    }
  ],
  "annotations": {}
}
```

I will describe how to build a puzzlefs image in the Setup section, at step 5.

# Limitations
One limitation is that the puzzlefs driver doesn't implement any lookup functionality and instead it inserts every directory entry into the dcache during init (see the DCACHE_BASED constant). This is similar to how the sample rust_fs driver works, but the goal is to implement proper lookup functions. However, more filesystem abstractions need to be implemented before this can be achieved.

# Upstreaming steps
Before the puzzlefs driver can be upstreamed, the following need to be merged:
* Wedson's filesystem abstractions [3]
* the necessary third-party crates [4] (with the preliminary discussion about whether this is desirable)

# Setup
My setup is based on Wedson's tutorial [8]. Next, I will describe the necessary steps to build an initrd and run a custom kernel under qemu.

1. Get the linux rust-next branch [9] and apply this patchset

2. Follow the rust quickstart guide [10]

3. configure and build the kernel

```
$ make LLVM=1 allnoconfig qemu-busybox-min.config rust.config
$ make LLVM=1 -j$(nproc)
```

4. setup busybox

```
git clone git://git.busybox.net/busybox
cd busybox
make menuconfig # enable 'Build static binary' config
make
make install
```

This will create the _install directory with the rootfs inside it.

5. create a home directory in the rootfs and copy a puzzlefs oci directory in home/puzzlefs_oci
To create a puzzlefs oci directory:
* download this custom puzzlefs repository [11] (it's custom because we want to build an image without verity data)
* run make release
* create a simple filesystem structure with a few directories and files (I've created one at ../test-puzzlefs/simple_rootfs)
* build a puzzlefs oci image at ~/work/busybox/_install/home/puzzlefs_oci (replace this path with your busybox path) with the tag first_try:

```
$ target/release/puzzlefs build --omit-verity \
../test-puzzlefs/simple_rootfs ~/work/busybox/_install/home/puzzlefs_oci \
first_try
```
* get first_try's image manifest from index.json (inside `puzzlefs_oci`)
```
$ cat index.json | jq . | grep digest
      "digest": "sha256:2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b",
```

6. add the following 'init' script in the busybox rootfs (rootfs path defaults to `./_install'):

```
#!/bin/sh
mount -t devtmpfs none /dev
mkdir -p /proc
mount -t proc none /proc

ifconfig lo up
udhcpc -i eth0

mkdir /mnt
mount -t puzzlefs -o oci_root_dir="/home/puzzlefs_oci" -o \
image_manifest="2d6602d678140540dc7e96de652a76a8b16e8aca190bae141297bcffdcae901b" \
none /mnt

setsid sh -c 'exec sh -l </dev/ttyS0 >/dev/ttyS0 2>&1'
```

Make sure to replace the image_manifest with your own digest. This init script will be passed to rdinit in the kernel command line.

7. generate the initramfs

Assuming busybox is in ~/work/busybox:

```
cd ~/work/busybox/_install && find . | cpio -H newc -o | gzip > ../ramdisk.img
```

This will generate a compressed ramdisk image in
~/work/busybox/ramdisk.img.

8. run with qemu (this assumes the linux tree is at '../linux' and busybox is at '../busybox'):

```
qemu-system-x86_64 \
    -accel kvm \
    -cpu host \
    -m 4G \
    -initrd ../busybox/ramdisk.img \
    -kernel ../linux/arch/x86/boot/bzImage \
    -nographic \
    -append 'console=ttyS0 nokaslr debug rdinit=/init' \
    -nic user,model=rtl8139 \
    -no-reboot
```

9. Check whether puzzlefs has been successfully mounted

```
~ # mount | grep puzzlefs
none on /mnt type puzzlefs (rw,relatime)
~ # ls /mnt/
dir-1  dir-2  dir-3  dir-4  file1  file2
```

# Puzzlefs abstract
Puzzlefs [1] is a container filesystem designed to address the limitations of the existing OCI format. The main goals of the project are reduced duplication, reproducible image builds, direct mounting support and memory safety guarantees, some inspired by the OCIv2 design document [7].

Reduced duplication is achieved using the content defined chunking algorithm FastCDC. This implementation allows chunks to be shared among layers. Building a new layer starting from an existing one allows reusing most of the chunks.

Another goal of the project is reproducible image builds, which is achieved by defining a canonical representation of the image format.

Direct mounting support is a key feature of puzzlefs and, together with fs-verity, it provides data integrity. Currently, puzzlefs is implemented as a userspace filesystem (FUSE). A read-only kernel filesystem driver is underway.

Lastly, memory safety is critical to puzzlefs, leading to the decision to implement it in Rust. Another goal is to share the same code between user space and kernel space in order to provide one secure implementation.

[1] https://github.com/anuvu/puzzlefs
[2] https://github.com/wedsonaf/linux/tree/fs
[3] https://github.com/Rust-for-Linux/linux/issues/1004
[4] https://github.com/Rust-for-Linux/linux/pull/1007
[5] https://docs.rs/serde_cbor/latest/serde_cbor/
[6] https://docs.rs/hex/0.4.3/hex/
[7] https://hackmd.io/@cyphar/ociv2-brainstorm
[8] https://www.youtube.com/watch?v=tPs1uRqOnlk
[9] https://github.com/Rust-for-Linux/linux/tree/rust-next
[10] https://docs.kernel.org/rust/quick-start.html
[11] https://github.com/ariel-miculas/puzzlefs/tree/support-no-verity-data
[12] https://archive.fosdem.org/2019/schedule/event/containers_atomfs/

Ariel Miculas (58):
rust: kernel: add libraries required by the filesystem abstractions
rust: kernel: backport the delay module from the rust branch
rust: kernel: add container_of macro
rust: kernel: add offset_of macro
drop: Add crate::pr_warn declaration
rust: kernel: rename from_kernel_errno to from_errno
rust: kernel: Rename from_pointer to from_foreing and into_pointer to
into_foreign
rust: kernel: add count_paren_items macro, needed by define_fs_params
macro
rust: helpers: add missing rust helper 'alloc_pages'
kernel: configs: add qemu-busybox-min.config
rust: kernel: format the rust code
samples: puzzlefs: add initial puzzlefs sample, copied from rust_fs.rs
kernel: configs: enable rust samples in rust.config
Add SAMPLE_RUST_SERDE in rust.config
rust: kernel: fix compile errors after rebase to rust-next
rust: serde_cbor: import crate
rust: serde_cbor: add SPDX License Identifiers
rust: serde_cbor: add no_fp_fmt_parse support
rust: Kbuild: enable serde_cbor
samples: rust: add cbor serialize/deserialize example
rust: serde_cbor: add support for serde_cbor's from_slice method by
using a custom alloc_kernel feature
rust: serde: add support for deserializing Vec with kernel_alloc
feature
rust: file: Replace UnsafeCell with Opaque for File
rust: kernel: implement fmt::Debug for CString
samples: puzzlefs: rename RustFs to PuzzleFs
samples: puzzlefs: add basic deserializing support for the puzzlefs
metadata
rust: file: present the filesystem context to the open function
rust: kernel: add an abstraction over vfsmount to allow cloning a new
private mount
rust: file: add from_path, from_path_in_root_mnt and read_with_offset
methods to File
samples: puzzlefs: pass the Vfsmount structure from open to read and
return the contents of the data file inside /home/puzzlefs_oci
rust: file: move from_path, from_path_in_root_mnt and read_with_offset
methods to a RegularFile newtype
rust: file: ensure RegularFile can only create regular files
rust: file: add get_pos method to RegularFile
rust: file: add methods read_to_end, get_file_size and update_pos to
RegularFile
rust: file: define a minimal Read trait and implement it for
RegularFile
samples: puzzlefs: add cbor_get_array_size method
samples: puzzlefs: add KernelError to WireFormatError and implement
From conversion
samples: puzzlefs: implement new for MetadataBlob
samples: puzzlefs: build puzzlefs into the kernel, thus avoiding the
need to export rust symbols
rust: alloc: add try_clone for Vec<T>
rust: alloc: add from_iter_fallible for Vec<T>
samples: puzzlefs: implement to_errno and from_errno for
WireFormatError
samples: puzzlefs: add TryReserveError (and from conversion) to
WireFormatError
samples: puzzlefs: add higher level inode related functionality
samples: puzzlefs: populate the directory entries with the inodes from
the puzzlefs metadata file
rust: hex: import crate
rust: hex: add SPDX license identifiers
rust: Kbuild: enable hex
rust: hex: implement FromHex trait and hex::decode using a custom
kernel_alloc feature
rust: hex: add encode_hex_iter and encode_hex_upper_iter methods
rust: puzzlefs: add HexError to WireFormatError and implement the From
conversion
rust: puzzlefs: display the error value for
WireFormatError::KernelError
samples: puzzlefs: add Rootfs and Digest structs to types.rs
samples: puzzlefs: implement the conversion from WireFormatError to
kernel::error::Error
rust: puzzlefs: read the puzzlefs image manifest instead of an
individual metadata layer
rust: puzzlefs: rename PuzzleFs to PuzzleFsModule to avoid confusion
with the PuzzleFS struct
rust: puzzlefs: add support for reading files
rust: puzzlefs: add oci_root_dir and image_manifest filesystem
parameters

Miguel Ojeda (15):
rust: proc-macro2: import crate
rust: proc-macro2: add SPDX License Identifiers
rust: proc-macro2: remove unicode_ident dependency
rust: quote: import crate
rust: quote: add SPDX License Identifiers
rust: syn: import crate
rust: syn: add SPDX License Identifiers
rust: syn: remove unicode-ident dependency
rust: serde: import crate
rust: serde: add no_fp_fmt_parse support
rust: serde: add SPDX License Identifiers
rust: serde_derive: import crate
rust: serde_derive: add SPDX License Identifiers
rust: Kbuild: enable proc-macro2, quote, syn, serde and
serde_derive
rust: test serde support

Wedson Almeida Filho (7):
rust: add definitions for ref-counted inodes and dentries
rust: add ability to register a file system
rust: define fs context
rust: add support for file system parameters
rust: allow fs driver to initialise new superblocks
rust: add module_fs macro
WIP: rust: allow fs to be populated

 Makefile                                  |   14 +-
 arch/x86/configs/qemu-busybox-min.config  |   11 +
 kernel/configs/qemu-busybox-min.config    |   56 +
 kernel/configs/rust.config                |   11 +
 rust/.gitignore                           |    1 +
 rust/Makefile                             |  232 +-
 rust/alloc/vec/mod.rs                     |   48 +
 rust/bindings/bindings_helper.h           |   14 +
 rust/bindings/lib.rs                      |    5 +
 rust/helpers.c                            |   76 +
 rust/hex/error.rs                         |   78 +
 rust/hex/lib.rs                           |  506 +++
 rust/hex/serde.rs                         |  104 +
 rust/kernel/cred.rs                       |   46 +
 rust/kernel/delay.rs                      |  104 +
 rust/kernel/driver.rs                     |   28 +
 rust/kernel/error.rs                      |   52 +-
 rust/kernel/file.rs                       | 1117 ++++++
 rust/kernel/fs.rs                         | 1478 ++++++++
 rust/kernel/fs/param.rs                   |  558 +++
 rust/kernel/io_buffer.rs                  |  153 +
 rust/kernel/iov_iter.rs                   |   81 +
 rust/kernel/lib.rs                        |   83 +
 rust/kernel/mm.rs                         |  149 +
 rust/kernel/mount.rs                      |   66 +
 rust/kernel/pages.rs                      |  144 +
 rust/kernel/str.rs                        |    6 +
 rust/kernel/test_serde.rs                 |   26 +
 rust/kernel/test_serde/de.rs              |  439 +++
 rust/kernel/test_serde/error.rs           |   73 +
 rust/kernel/test_serde/ser.rs             |  466 +++
 rust/kernel/user_ptr.rs                   |  175 +
 rust/proc-macro2/detection.rs             |   77 +
 rust/proc-macro2/fallback.rs              | 1004 ++++++
 rust/proc-macro2/lib.rs                   | 1341 ++++++++
 rust/proc-macro2/marker.rs                |   20 +
 rust/proc-macro2/parse.rs                 |  874 +++++
 rust/proc-macro2/rcvec.rs                 |  144 +
 rust/proc-macro2/wrapper.rs               |  996 ++++++
 rust/quote/ext.rs                         |  112 +
 rust/quote/format.rs                      |  170 +
 rust/quote/ident_fragment.rs              |   88 +
 rust/quote/lib.rs                         | 1436 ++++++++
 rust/quote/runtime.rs                     |  440 +++
 rust/quote/spanned.rs                     |   45 +
 rust/quote/to_tokens.rs                   |  211 ++
 rust/serde/de/format.rs                   |   32 +
 rust/serde/de/ignored_any.rs              |  246 ++
 rust/serde/de/impls.rs                    | 2755 +++++++++++++++
 rust/serde/de/mod.rs                      | 2313 +++++++++++++
 rust/serde/de/seed.rs                     |   21 +
 rust/serde/de/utf8.rs                     |   48 +
 rust/serde/de/value.rs                    | 1718 ++++++++++
 rust/serde/integer128.rs                  |   84 +
 rust/serde/lib.rs                         |  351 ++
 rust/serde/macros.rs                      |  238 ++
 rust/serde/private/de.rs                  | 2997 ++++++++++++++++
 rust/serde/private/doc.rs                 |  161 +
 rust/serde/private/mod.rs                 |   52 +
 rust/serde/private/ser.rs                 | 1316 +++++++
 rust/serde/private/size_hint.rs           |   23 +
 rust/serde/ser/fmt.rs                     |  180 +
 rust/serde/ser/impls.rs                   |  987 ++++++
 rust/serde/ser/impossible.rs              |  218 ++
 rust/serde/ser/mod.rs                     | 1992 +++++++++++
 rust/serde/std_error.rs                   |   50 +
 rust/serde_cbor/de.rs                     | 1370 ++++++++
 rust/serde_cbor/error.rs                  |  320 ++
 rust/serde_cbor/lib.rs                    |  371 ++
 rust/serde_cbor/read.rs                   |  647 ++++
 rust/serde_cbor/ser.rs                    |  748 ++++
 rust/serde_cbor/tags.rs                   |  224 ++
 rust/serde_cbor/value/de.rs               |  168 +
 rust/serde_cbor/value/mod.rs              |  158 +
 rust/serde_cbor/value/ser.rs              |  447 +++
 rust/serde_cbor/write.rs                  |  177 +
 rust/serde_derive/bound.rs                |  408 +++
 rust/serde_derive/de.rs                   | 3148 +++++++++++++++++
 rust/serde_derive/dummy.rs                |   46 +
 rust/serde_derive/fragment.rs             |   76 +
 rust/serde_derive/internals/ast.rs        |  204 ++
 rust/serde_derive/internals/attr.rs       | 1908 +++++++++++
 rust/serde_derive/internals/case.rs       |  199 ++
 rust/serde_derive/internals/check.rs      |  445 +++
 rust/serde_derive/internals/ctxt.rs       |   64 +
 rust/serde_derive/internals/mod.rs        |   30 +
 rust/serde_derive/internals/receiver.rs   |  287 ++
 rust/serde_derive/internals/respan.rs     |   18 +
 rust/serde_derive/internals/symbol.rs     |   71 +
 rust/serde_derive/lib.rs                  |  112 +
 rust/serde_derive/pretend.rs              |  203 ++
 rust/serde_derive/ser.rs                  | 1342 ++++++++
 rust/serde_derive/this.rs                 |   34 +
 rust/serde_derive/try.rs                  |   26 +
 rust/syn/attr.rs                          |  664 ++++
 rust/syn/await.rs                         |    4 +
 rust/syn/bigint.rs                        |   68 +
 rust/syn/buffer.rs                        |  400 +++
 rust/syn/custom_keyword.rs                |  255 ++
 rust/syn/custom_punctuation.rs            |  302 ++
 rust/syn/data.rs                          |  495 +++
 rust/syn/derive.rs                        |  276 ++
 rust/syn/discouraged.rs                   |  196 ++
 rust/syn/error.rs                         |  430 +++
 rust/syn/export.rs                        |   41 +
 rust/syn/expr.rs                          | 3560 +++++++++++++++++++
 rust/syn/ext.rs                           |  141 +
 rust/syn/file.rs                          |  127 +
 rust/syn/gen/clone.rs                     | 2243 ++++++++++++
 rust/syn/gen/debug.rs                     | 3044 +++++++++++++++++
 rust/syn/gen/eq.rs                        | 2197 ++++++++++++
 rust/syn/gen/fold.rs                      | 3343 ++++++++++++++++++
 rust/syn/gen/hash.rs                      | 2871 ++++++++++++++++
 rust/syn/gen/visit.rs                     | 3788 +++++++++++++++++++++
 rust/syn/gen/visit_mut.rs                 | 3788 +++++++++++++++++++++
 rust/syn/gen_helper.rs                    |  156 +
 rust/syn/generics.rs                      | 1339 ++++++++
 rust/syn/group.rs                         |  284 ++
 rust/syn/ident.rs                         |  103 +
 rust/syn/item.rs                          | 3315 ++++++++++++++++++
 rust/syn/lib.rs                           |  985 ++++++
 rust/syn/lifetime.rs                      |  156 +
 rust/syn/lit.rs                           | 1602 +++++++++
 rust/syn/lookahead.rs                     |  171 +
 rust/syn/mac.rs                           |  221 ++
 rust/syn/macros.rs                        |  179 +
 rust/syn/op.rs                            |  236 ++
 rust/syn/parse.rs                         | 1316 +++++++
 rust/syn/parse_macro_input.rs             |  181 +
 rust/syn/parse_quote.rs                   |  169 +
 rust/syn/pat.rs                           |  929 +++++
 rust/syn/path.rs                          |  856 +++++
 rust/syn/print.rs                         |   18 +
 rust/syn/punctuated.rs                    | 1070 ++++++
 rust/syn/reserved.rs                      |   46 +
 rust/syn/sealed.rs                        |    6 +
 rust/syn/span.rs                          |   69 +
 rust/syn/spanned.rs                       |  116 +
 rust/syn/stmt.rs                          |  351 ++
 rust/syn/thread.rs                        |   43 +
 rust/syn/token.rs                         | 1015 ++++++
 rust/syn/tt.rs                            |  109 +
 rust/syn/ty.rs                            | 1288 +++++++
 rust/syn/verbatim.rs                      |   35 +
 rust/syn/whitespace.rs                    |   67 +
 samples/rust/Kconfig                      |   28 +
 samples/rust/Makefile                     |    3 +
 samples/rust/local_data_format/de.rs      |  422 +++
 samples/rust/local_data_format/error.rs   |   73 +
 samples/rust/local_data_format/ser.rs     |  443 +++
 samples/rust/puzzle.rs                    |    4 +
 samples/rust/puzzle/error.rs              |   91 +
 samples/rust/puzzle/inode.rs              |  150 +
 samples/rust/puzzle/oci.rs                |   71 +
 samples/rust/puzzle/types.rs              |  389 +++
 samples/rust/puzzle/types/cbor_helpers.rs |   50 +
 samples/rust/puzzlefs.rs                  |  220 ++
 samples/rust/rust_fs.rs                   |  105 +
 samples/rust/rust_serde.rs                |  125 +
 scripts/Makefile.build                    |    4 +-
 160 files changed, 89204 insertions(+), 29 deletions(-)
 create mode 100644 arch/x86/configs/qemu-busybox-min.config
 create mode 100644 kernel/configs/qemu-busybox-min.config
 create mode 100644 rust/hex/error.rs
 create mode 100644 rust/hex/lib.rs
 create mode 100644 rust/hex/serde.rs
 create mode 100644 rust/kernel/cred.rs
 create mode 100644 rust/kernel/delay.rs
 create mode 100644 rust/kernel/driver.rs
 create mode 100644 rust/kernel/file.rs
 create mode 100644 rust/kernel/fs.rs
 create mode 100644 rust/kernel/fs/param.rs
 create mode 100644 rust/kernel/io_buffer.rs
 create mode 100644 rust/kernel/iov_iter.rs
 create mode 100644 rust/kernel/mm.rs
 create mode 100644 rust/kernel/mount.rs
 create mode 100644 rust/kernel/pages.rs
 create mode 100644 rust/kernel/test_serde.rs
 create mode 100644 rust/kernel/test_serde/de.rs
 create mode 100644 rust/kernel/test_serde/error.rs
 create mode 100644 rust/kernel/test_serde/ser.rs
 create mode 100644 rust/kernel/user_ptr.rs
 create mode 100644 rust/proc-macro2/detection.rs
 create mode 100644 rust/proc-macro2/fallback.rs
 create mode 100644 rust/proc-macro2/lib.rs
 create mode 100644 rust/proc-macro2/marker.rs
 create mode 100644 rust/proc-macro2/parse.rs
 create mode 100644 rust/proc-macro2/rcvec.rs
 create mode 100644 rust/proc-macro2/wrapper.rs
 create mode 100644 rust/quote/ext.rs
 create mode 100644 rust/quote/format.rs
 create mode 100644 rust/quote/ident_fragment.rs
 create mode 100644 rust/quote/lib.rs
 create mode 100644 rust/quote/runtime.rs
 create mode 100644 rust/quote/spanned.rs
 create mode 100644 rust/quote/to_tokens.rs
 create mode 100644 rust/serde/de/format.rs
 create mode 100644 rust/serde/de/ignored_any.rs
 create mode 100644 rust/serde/de/impls.rs
 create mode 100644 rust/serde/de/mod.rs
 create mode 100644 rust/serde/de/seed.rs
 create mode 100644 rust/serde/de/utf8.rs
 create mode 100644 rust/serde/de/value.rs
 create mode 100644 rust/serde/integer128.rs
 create mode 100644 rust/serde/lib.rs
 create mode 100644 rust/serde/macros.rs
 create mode 100644 rust/serde/private/de.rs
 create mode 100644 rust/serde/private/doc.rs
 create mode 100644 rust/serde/private/mod.rs
 create mode 100644 rust/serde/private/ser.rs
 create mode 100644 rust/serde/private/size_hint.rs
 create mode 100644 rust/serde/ser/fmt.rs
 create mode 100644 rust/serde/ser/impls.rs
 create mode 100644 rust/serde/ser/impossible.rs
 create mode 100644 rust/serde/ser/mod.rs
 create mode 100644 rust/serde/std_error.rs
 create mode 100644 rust/serde_cbor/de.rs
 create mode 100644 rust/serde_cbor/error.rs
 create mode 100644 rust/serde_cbor/lib.rs
 create mode 100644 rust/serde_cbor/read.rs
 create mode 100644 rust/serde_cbor/ser.rs
 create mode 100644 rust/serde_cbor/tags.rs
 create mode 100644 rust/serde_cbor/value/de.rs
 create mode 100644 rust/serde_cbor/value/mod.rs
 create mode 100644 rust/serde_cbor/value/ser.rs
 create mode 100644 rust/serde_cbor/write.rs
 create mode 100644 rust/serde_derive/bound.rs
 create mode 100644 rust/serde_derive/de.rs
 create mode 100644 rust/serde_derive/dummy.rs
 create mode 100644 rust/serde_derive/fragment.rs
 create mode 100644 rust/serde_derive/internals/ast.rs
 create mode 100644 rust/serde_derive/internals/attr.rs
 create mode 100644 rust/serde_derive/internals/case.rs
 create mode 100644 rust/serde_derive/internals/check.rs
 create mode 100644 rust/serde_derive/internals/ctxt.rs
 create mode 100644 rust/serde_derive/internals/mod.rs
 create mode 100644 rust/serde_derive/internals/receiver.rs
 create mode 100644 rust/serde_derive/internals/respan.rs
 create mode 100644 rust/serde_derive/internals/symbol.rs
 create mode 100644 rust/serde_derive/lib.rs
 create mode 100644 rust/serde_derive/pretend.rs
 create mode 100644 rust/serde_derive/ser.rs
 create mode 100644 rust/serde_derive/this.rs
 create mode 100644 rust/serde_derive/try.rs
 create mode 100644 rust/syn/attr.rs
 create mode 100644 rust/syn/await.rs
 create mode 100644 rust/syn/bigint.rs
 create mode 100644 rust/syn/buffer.rs
 create mode 100644 rust/syn/custom_keyword.rs
 create mode 100644 rust/syn/custom_punctuation.rs
 create mode 100644 rust/syn/data.rs
 create mode 100644 rust/syn/derive.rs
 create mode 100644 rust/syn/discouraged.rs
 create mode 100644 rust/syn/error.rs
 create mode 100644 rust/syn/export.rs
 create mode 100644 rust/syn/expr.rs
 create mode 100644 rust/syn/ext.rs
 create mode 100644 rust/syn/file.rs
 create mode 100644 rust/syn/gen/clone.rs
 create mode 100644 rust/syn/gen/debug.rs
 create mode 100644 rust/syn/gen/eq.rs
 create mode 100644 rust/syn/gen/fold.rs
 create mode 100644 rust/syn/gen/hash.rs
 create mode 100644 rust/syn/gen/visit.rs
 create mode 100644 rust/syn/gen/visit_mut.rs
 create mode 100644 rust/syn/gen_helper.rs
 create mode 100644 rust/syn/generics.rs
 create mode 100644 rust/syn/group.rs
 create mode 100644 rust/syn/ident.rs
 create mode 100644 rust/syn/item.rs
 create mode 100644 rust/syn/lib.rs
 create mode 100644 rust/syn/lifetime.rs
 create mode 100644 rust/syn/lit.rs
 create mode 100644 rust/syn/lookahead.rs
 create mode 100644 rust/syn/mac.rs
 create mode 100644 rust/syn/macros.rs
 create mode 100644 rust/syn/op.rs
 create mode 100644 rust/syn/parse.rs
 create mode 100644 rust/syn/parse_macro_input.rs
 create mode 100644 rust/syn/parse_quote.rs
 create mode 100644 rust/syn/pat.rs
 create mode 100644 rust/syn/path.rs
 create mode 100644 rust/syn/print.rs
 create mode 100644 rust/syn/punctuated.rs
 create mode 100644 rust/syn/reserved.rs
 create mode 100644 rust/syn/sealed.rs
 create mode 100644 rust/syn/span.rs
 create mode 100644 rust/syn/spanned.rs
 create mode 100644 rust/syn/stmt.rs
 create mode 100644 rust/syn/thread.rs
 create mode 100644 rust/syn/token.rs
 create mode 100644 rust/syn/tt.rs
 create mode 100644 rust/syn/ty.rs
 create mode 100644 rust/syn/verbatim.rs
 create mode 100644 rust/syn/whitespace.rs
 create mode 100644 samples/rust/local_data_format/de.rs
 create mode 100644 samples/rust/local_data_format/error.rs
 create mode 100644 samples/rust/local_data_format/ser.rs
 create mode 100644 samples/rust/puzzle.rs
 create mode 100644 samples/rust/puzzle/error.rs
 create mode 100644 samples/rust/puzzle/inode.rs
 create mode 100644 samples/rust/puzzle/oci.rs
 create mode 100644 samples/rust/puzzle/types.rs
 create mode 100644 samples/rust/puzzle/types/cbor_helpers.rs
 create mode 100644 samples/rust/puzzlefs.rs
 create mode 100644 samples/rust/rust_fs.rs
 create mode 100644 samples/rust/rust_serde.rs

--
2.40.1

#107 lore raw
Quoted 7 lines · Ariel Miculas (amiculas)

Hello Christian,

I didn't send these patches to a wider audience because this is an initial prototype of the PuzzleFS driver, and it has a few prerequisites before it could be even considered for merging. First of all, the rust filesystem abstractions and their dependencies need to be upstreamed, then there needs to be a discussion regarding the

Yes.

Quoted 9 lines

inclusion of third-party crates in the linux kernel.

My plan was to send these patches to the rust-for-linux mailing list and then start a discussion with Miguel Ojeda regarding the upstreaming approach.
There are a lot of new files added in this patch series because I've included all the dependencies required so that my patches could be applied to the rust-next branch, but these dependencies will most likely need to be upstreamed separately.

It was never my intention to avoid your reviews, should I also send subsequent patches to linux-fsdevel, even if they're in the early stages of development?

Yeah, I think that would be great.

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

#108 lore raw

Adding Serge Hallyn for visibility.

Regards,
Ariel

From: Christian Brauner <[email protected]>
Sent: Friday, June 9, 2023 2:45 PM
To: Ariel Miculas (amiculas) <[email protected]>
Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 7 lines · Ariel Miculas (amiculas)

Hello Christian,

I didn't send these patches to a wider audience because this is an initial prototype of the PuzzleFS driver, and it has a few prerequisites before it could be even considered for merging. First of all, the rust filesystem abstractions and their dependencies need to be upstreamed, then there needs to be a discussion regarding the

Yes.

Quoted 9 lines

inclusion of third-party crates in the linux kernel.

My plan was to send these patches to the rust-for-linux mailing list and then start a discussion with Miguel Ojeda regarding the upstreaming approach.
There are a lot of new files added in this patch series because I've included all the dependencies required so that my patches could be applied to the rust-next branch, but these dependencies will most likely need to be upstreamed separately.

It was never my intention to avoid your reviews, should I also send subsequent patches to linux-fsdevel, even if they're in the early stages of development?

Yeah, I think that would be great.

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

#109 lore raw

On 2023/6/9 20:03, Ariel Miculas (amiculas) wrote:

...

Quoted 9 lines

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

Just a head up. Since Rust kernel infrastructure is too premature, it's impossible to handle page cache / iomap and many useful stuffs. In the long term, at least (someday) after Rust infrastructure is mature, I will implement EROFS ino Rust as a try as well.

As for chunk CDC, I don't see it's hard (since we already have some CDC approach since Linux v6.1) but as an effective disk filesystem for performance, EROFS on-disk data is all block-aligned to match storage and page cache alignment. If it's really needed, I could update a more complete (but ineffective and slow) index version to implement unaligned extents (both for decoded and encoded sides). Yet I really think the main purpose of a kernel filesystem is to make full use of kernel infrastructure for performance (like page cache handling) otherwise a FUSE approach is enough.

Finally, as for OCI container image stuffs, I'd like to avoid saying this topic anymore on the list (too tired about this). I've seen _three_ in-kernel approaches already before this one and I tend to avoid listing the complete names (including FUSE alternatives) here. I really suggest if you guys could sit down and plan at least a complete OCI standard for the next image format (even you don't want to reuse any exist filesystem for whatever reasons).

Thanks,
Gao Xiang

#110 lore raw
In reply to Christian Brauner
Quoted 8 lines · Christian Brauner

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

That would be great, thanks Christian! (Cc'ing Alice for binderfs -- I think Rust Binder is keeping binderfs in C for the moment, but if you are willing to try things, they are probably interested :)

Ariel: sorry, we crossed messages; I didn't receive your message at [1], the rust-for-linux list probably dropped it due to the included HTML.

[1] https://lore.kernel.org/linux-mm/CH0PR11MB529981313ED5A1F815350E41CD51A@CH0PR11MB5299.namprd11.prod.outlook.com/

Cheers,
Miguel

#111 lore raw

Sorry about that, it seems like I need to switch to plain text mode for every reply in outlook, which is annoying.

Regards,
Ariel

From: Miguel Ojeda <[email protected]>
Sent: Friday, June 9, 2023 3:07 PM
To: Christian Brauner <[email protected]>
Cc: Ariel Miculas (amiculas) <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; Alice Ryhl <[email protected]>
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 8 lines · Christian Brauner

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

That would be great, thanks Christian! (Cc'ing Alice for binderfs -- I think Rust Binder is keeping binderfs in C for the moment, but if you are willing to try things, they are probably interested :)

Ariel: sorry, we crossed messages; I didn't receive your message at [1], the rust-for-linux list probably dropped it due to the included HTML.

[1] https://lore.kernel.org/linux-mm/CH0PR11MB529981313ED5A1F815350E41CD51A@CH0PR11MB5299.namprd11.prod.outlook.com/

Cheers,
Miguel

maintainer
#112 lore raw
Quoted 1 line · Ariel Miculas (amiculas)

Sorry about that, it seems like I need to switch to plain text mode for every reply in outlook, which is annoying.

You should also turn off the top-posting mode, as that's not good etiquette on the kernel mailing lists for obvious reasons :)

thanks,

greg k-h

maintainer
#113 lore raw
In reply to Miguel Ojeda
Quoted 3 lines · Miguel Ojeda

That would be great, thanks Christian! (Cc'ing Alice for binderfs -- I think Rust Binder is keeping binderfs in C for the moment, but if you are willing to try things, they are probably interested :)

Yeah, Rust binder already needs bindings to a lot of other parts of the kernel, so I decided to not rewrite the binderfs part for now to cut down on the number of subsystems I would need to upstream bindings for. Upstreaming bindings has proved to be a lot of work.

If someone else wants to upstream the filesystem bindings that binderfs would need, then we can certainly look into using them in Rust binder.

Alice

#114 lore raw
In reply to Christian Brauner
Quoted 8 lines · Christian Brauner

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

(Not a kernel developer, but this argument makes sense to me)

Quoted 8 lines

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

I am sure you are aware there's not some "container world" monoculture, there are many organizations, people and companies here with some healthy co-opetition but also some duplication inherent from that.

That said at a practical level, Ariel in the https://github.com/containers GH organization we're kind of a "big tent" place. A subset of the organization is very heavily Rust oriented now (certainly the parts I touch) and briefly skimming the puzzlefs code, there are definitely some bits of code we could consider sharing in userspace. Actually though since this isn't releated to the in-kernel discussion I'll file an issue on Github and we can discuss there.

But there is definitely a subset of the discussion that Christian is referring to here that is about the intersections/overlap with the composefs approach that is relevant for this list. Maybe we could try to collaborate on an unbiased "puzzlefs vs composefs" document? (What's in https://github.com/anuvu/puzzlefs/tree/master/doc is a bit sparse right now)

#115 lore raw
Quoted 23 lines · Colin Walters
Quoted 8 lines · Christian Brauner

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

(Not a kernel developer, but this argument makes sense to me)

Quoted 8 lines

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

Quoted 2 lines

I am sure you are aware there's not some "container world" monoculture, there are many organizations, people and companies here

That submission here explicitly references OCI v2. Composefs explicitly advertises 100% compatibility with OCI. So, there's a set of OCI specs including runtime and image. As far as I'm concerned you're all one container world under the OCI umbrella.

We're not going to push multiple filesystems into the kernel that all do slightly different things but all serve the OCI container world and use some spec as an argument to move stuff into the kernel.

The OCI initiative is hailed as unifying the container ecosystem. Hence, we can expect coordination.

#116 lore raw
Quoted 24 lines · Christian Brauner
Quoted 23 lines · Colin Walters
Quoted 8 lines · Christian Brauner

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

(Not a kernel developer, but this argument makes sense to me)

Quoted 8 lines

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

Quoted 5 lines
Quoted 2 lines

I am sure you are aware there's not some "container world" monoculture, there are many organizations, people and companies here

That submission here explicitly references OCI v2. Composefs explicitly advertises 100% compatibility with OCI. So, there's a set of OCI specs

OCI v2 doesn't currently exist :) There were many design goals for it, and near as I can tell, after everyone discussed those together, everyone went off to work on implementing the bits the needed - which is a right and proper step before coming back and comparing notes about what went well, etc.

The two main things where puzzlefs is experimenting are the use of content defined chunking (CDC), and being written in rust (and, especially, to have the same rust code base be used for the in kernel driver, the fuse mounter, the builder, and the userspace extractor). (Well, and reproducible images through a canonical representation, but...)

It requires a POC in order to really determine whether the CDC will be worth it, or will have pitfalls. So far, it looks very promising. Adding that functionality to composefs one day could be cool.

Likewise, it requires a user in order to push on the infrastructure required to support a full filesystem in rust in the kernel. But that really isn't something we can "add to composefs". :)

The main goal of this posting, then, was to show the infrastructure pieces and work on that with the community. We're definitely not (currently :) asking for puzzlefs to be included. However, it is our (business and personal) justification for the rest of the work.

Quoted 2 lines

including runtime and image. As far as I'm concerned you're all one container world under the OCI umbrella.

One big happy family!

Quoted 6 lines

We're not going to push multiple filesystems into the kernel that all do slightly different things but all serve the OCI container world and use some spec as an argument to move stuff into the kernel.

The OCI initiative is hailed as unifying the container ecosystem. Hence, we can expect coordination.

#117 lore raw
In reply to Colin Walters

A "puzzlefs vs composefs" document sounds like a good idea. The documentation in puzzlefs is a little outdated and could be improved. Feel free to create a github issue and tag me in there.

PS: as soon as I figure out how to turn off the top-posting mode, I'll do it.

Regards,
Ariel

________________________________________
From: Colin Walters <[email protected]>
Sent: Friday, June 9, 2023 3:20 PM
To: Christian Brauner; Ariel Miculas (amiculas)
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 8 lines · Christian Brauner

Because the series you sent here touches on a lot of things in terms of infrastructure alone. That work could very well be rather interesting independent of PuzzleFS. We might just want to get enough infrastructure to start porting a tiny existing fs (binderfs or something similar small) to Rust to see how feasible this is and to wet our appetite for bigger changes such as accepting a new filesystem driver completely written in Rust.

(Not a kernel developer, but this argument makes sense to me)

Quoted 8 lines

But aside from the infrastructure discussion:

This is yet another filesystem for solving the container image problem in the kernel with the addition of yet another filesystem. We just went through this excercise with another filesystem. So I'd expect some reluctance here. Tbh, the container world keeps sending us filesystems at an alarming rate. That's two within a few months and that leaves a rather disorganized impression.

I am sure you are aware there's not some "container world" monoculture, there are many organizations, people and companies here with some healthy co-opetition but also some duplication inherent from that.

That said at a practical level, Ariel in the https://github.com/containers GH organization we're kind of a "big tent" place. A subset of the organization is very heavily Rust oriented now (certainly the parts I touch) and briefly skimming the puzzlefs code, there are definitely some bits of code we could consider sharing in userspace. Actually though since this isn't releated to the in-kernel discussion I'll file an issue on Github and we can discuss there.

But there is definitely a subset of the discussion that Christian is referring to here that is about the intersections/overlap with the composefs approach that is relevant for this list. Maybe we could try to collaborate on an unbiased "puzzlefs vs composefs" document? (What's in https://github.com/anuvu/puzzlefs/tree/master/doc is a bit sparse right now)

#118 lore raw
Quoted 5 lines · Ariel Miculas (amiculas)

A "puzzlefs vs composefs" document sounds like a good idea. The documentation in puzzlefs is a little outdated and could be improved. Feel free to create a github issue and tag me in there.

PS: as soon as I figure out how to turn off the top-posting mode, I'll do it.

Let me know as well if you could do w/ Outlook :). Switch to other email clients if possible.

---Trilok Soni

#119 lore raw

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected]. If this is not a hard requirement, then I could switch.

Regards,
Ariel

________________________________________
From: Trilok Soni <[email protected]>
Sent: Friday, June 9, 2023 8:10 PM
To: Ariel Miculas (amiculas); Colin Walters; Christian Brauner
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 5 lines · Ariel Miculas (amiculas)

A "puzzlefs vs composefs" document sounds like a good idea. The documentation in puzzlefs is a little outdated and could be improved. Feel free to create a github issue and tag me in there.

PS: as soon as I figure out how to turn off the top-posting mode, I'll do it.

Let me know as well if you could do w/ Outlook :). Switch to other email clients if possible.

---Trilok Soni

#120 lore raw
Quoted 3 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected]. If this is not a hard requirement, then I could switch.

For patches, yeah, that is ideal, so that it matches the Git author / From:.

But for the other emails, you could use your personal address, if that makes things easier.

Hope that helps!

Cheers,
Miguel

#121 lore raw
Quoted 8 lines · Miguel Ojeda
Quoted 5 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected]. If this is not a hard requirement, then I could switch.

Quoted 5 lines

For patches, yeah, that is ideal, so that it matches the Git author / From:.

But for the other emails, you could use your personal address, if that makes things easier.

It's still not a requirement, though. You can send from your gmail account and still have

From: Ariel Miculas <[email protected]>

As the first line (separated from the commit message by a blank line), which git am (or b4) will pick up as the author email. This behaviour is specifically for people who want the author to be their corporate email address, but have failed to persuade corporate IT to make it possible.

Regards,

James

#122 lore raw
Quoted 7 lines · James Bottomley
Quoted 2 lines · Miguel Ojeda

For patches, yeah, that is ideal, so that it matches the Git author / From:.

It's still not a requirement, though. You can send from your gmail account and still have

Yeah, that is what I said "ideal". When Ariel sent the first patch, he didn't use the From: tag within the email body, but attributed it to Cisco, and I had not seen any commit/email from his Cisco address yet, so I asked him if he could use his corporate address.

Cheers,
Miguel

#123 lore raw
In reply to James Bottomley

Yes, but then how can you be sure that [email protected] is the real author of the commit? I think that's why Miguel Ojeda asked me to send them from my business email, otherwise some random gmail account could claim that he is "Ariel Miculas", so he's entitled to sign-off as [email protected].

Regards,
Ariel

Quoted 31 lines · James Bottomley
Quoted 8 lines · Miguel Ojeda
Quoted 5 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected]. If this is not a hard requirement, then I could switch.

Quoted 5 lines

For patches, yeah, that is ideal, so that it matches the Git author / From:.

But for the other emails, you could use your personal address, if that makes things easier.

It's still not a requirement, though. You can send from your gmail account and still have

From: Ariel Miculas <[email protected]>

As the first line (separated from the commit message by a blank line), which git am (or b4) will pick up as the author email. This behaviour is specifically for people who want the author to be their corporate email address, but have failed to persuade corporate IT to make it possible.

Regards,

James

#124 lore raw
Quoted 5 lines · Ariel Miculas

Yes, but then how can you be sure that [email protected] is the real author of the commit? I think that's why Miguel Ojeda asked me to send them from my business email, otherwise some random gmail account could claim that he is "Ariel Miculas", so he's entitled to sign-off as [email protected].

Because it's a public list and the real one would observe and repudiate
...

The DCO is an attestation which we basically believe absent any proof to the contrary. It's also exhausting and an incredible amount of pointless admin not to believe anything until it's proved, which is why we don't.

James

#125 lore raw
In reply to Ariel Miculas
Quoted 6 lines · Ariel Miculas

Yes, but then how can you be sure that [email protected] is the real author of the commit? I think that's why Miguel Ojeda asked me to send them from my business email, otherwise some random gmail account could claim that he is "Ariel Miculas", so he's entitled to sign-off as [email protected].

Yeah, that was the main thing; and by asking that, if you managed to send that original patch from the corporate one, then there was no need for the From: to begin with :)

Cheers,
Miguel

#126 lore raw
In reply to Ariel Miculas (amiculas)
Quoted 4 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected].
If this is not a hard requirement, then I could switch.

For sending patches, you can simply use git-send-email. All you need to point it at is the outgoing email server (which should be a config setting in whatever tool you are using now). We have a (reasonably) up to date document with some recommendations:

https://www.kernel.org/doc/html/latest/process/email-clients.html

I've successfully used evolution with an exchange server for many years, but the interface isn't to everyone's taste and Mozilla Thunderbird is also known to connect to it. Basic outlook has proven impossible to configure correctly (which is why it doesn't have an entry).

Regards,

James

#127 lore raw

I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c

Regards,
Ariel

________________________________________
From: James Bottomley <[email protected]>
Sent: Friday, June 9, 2023 9:43 PM
To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 4 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected].
If this is not a hard requirement, then I could switch.

For sending patches, you can simply use git-send-email. All you need to point it at is the outgoing email server (which should be a config setting in whatever tool you are using now). We have a (reasonably) up to date document with some recommendations:

https://www.kernel.org/doc/html/latest/process/email-clients.html

I've successfully used evolution with an exchange server for many years, but the interface isn't to everyone's taste and Mozilla Thunderbird is also known to connect to it. Basic outlook has proven impossible to configure correctly (which is why it doesn't have an entry).

Regards,

James

#128 lore raw

Now if I could figure out how to disable "top posting" in gmail...

Regards,
Ariel

Quoted 37 lines · Ariel Miculas (amiculas)

I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c

Regards,
Ariel

________________________________________
From: James Bottomley <[email protected]>
Sent: Friday, June 9, 2023 9:43 PM
To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 4 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected].
If this is not a hard requirement, then I could switch.

For sending patches, you can simply use git-send-email. All you need to point it at is the outgoing email server (which should be a config setting in whatever tool you are using now). We have a (reasonably) up to date document with some recommendations:

https://www.kernel.org/doc/html/latest/process/email-clients.html

I've successfully used evolution with an exchange server for many years, but the interface isn't to everyone's taste and Mozilla Thunderbird is also known to connect to it. Basic outlook has proven impossible to configure correctly (which is why it doesn't have an entry).

Regards,

James

#129 lore raw
Quoted 39 lines · Ariel Miculas

Now if I could figure out how to disable "top posting" in gmail...

Regards,
Ariel

Quoted 32 lines · Ariel Miculas (amiculas)

I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c

Regards,
Ariel

________________________________________
From: James Bottomley <[email protected]>
Sent: Friday, June 9, 2023 9:43 PM
To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 4 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected].
If this is not a hard requirement, then I could switch.

For sending patches, you can simply use git-send-email. All you need to point it at is the outgoing email server (which should be a config setting in whatever tool you are using now). We have a (reasonably) up to date document with some recommendations:

https://www.kernel.org/doc/html/latest/process/email-clients.html

I've successfully used evolution with an exchange server for many years, but the interface isn't to everyone's taste and Mozilla Thunderbird is also known to connect to it. Basic outlook has proven impossible to configure correctly (which is why it doesn't have an entry).

One of the big reasons why I have quic_tsoni at quicinc dot com (Thunderbird/Mutt/whateveryoulike except Outlook) on top of tsoni at quicinc dot com (Outlook) to make sure that we comply w/ upstream guidelines. Two email IDs are hard to manage but it gives
the good separation and freedom.

---Trilok Soni

#130 lore raw
In reply to Ariel Miculas (amiculas)
Quoted 5 lines · Ariel Miculas (amiculas)

I did use git send-email for sending this patch series, but I cannot find any setting in the Outlook web client for disabling "top posting" when replying to emails:
https://answers.microsoft.com/en-us/outlook_com/forum/all/eliminate-top-posting/5e1e5729-30f8-41e9-84cb-fb5e81229c7c

Regards,
Ariel

You can also use git-send-email for sending replies. Just make a .txt file with this format:

Subject: <subject goes here>

<your reply after an empty line>

The lore.kernel.org site generates the appropriate git-send-email command for sending the reply, so you can copy-paste that to send it.

I send most of my replies like that.

Quoted 29 lines

From: James Bottomley <[email protected]>
Sent: Friday, June 9, 2023 9:43 PM
To: Ariel Miculas (amiculas); Trilok Soni; Colin Walters; Christian Brauner
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [RFC PATCH 00/80] Rust PuzzleFS filesystem driver

Quoted 4 lines · Ariel Miculas (amiculas)

I could switch to my personal gmail, but last time Miguel Ojeda asked me to use my cisco email when I send commits signed off by [email protected].
If this is not a hard requirement, then I could switch.

For sending patches, you can simply use git-send-email. All you need to point it at is the outgoing email server (which should be a config setting in whatever tool you are using now). We have a (reasonably) up to date document with some recommendations:

https://www.kernel.org/doc/html/latest/process/email-clients.html

I've successfully used evolution with an exchange server for many years, but the interface isn't to everyone's taste and Mozilla Thunderbird is also known to connect to it. Basic outlook has proven impossible to configure correctly (which is why it doesn't have an entry).

Regards,

James

#131 lore raw
In reply to Christian Brauner
Quoted 4 lines · Christian Brauner

Any wrappers and code for core fs should be maintained as part of fs. Rust shouldn't become a way to avoid our reviews once you have a few wrappers added somewhere.

Definitely and, to be clear, we are strict about it (e.g.
https://rust-for-linux.com/contributing#the-rust-subsystem).

In fact, we appreciate maintainers that are willing to take patches through their tree and take ownership of code too (e.g. KUnit and DRM are doing so already).

I imagine Ariel sent the RFC as a way to announce his work early on, especially given how some patches were split, the lack of commit messages and tags, etc.

In other cases, we have reviewed patches privately first to iron out this sort of thing, but I wasn't aware of this series coming (I knew Ariel was working on puzzlefs and that he wanted to submit it eventually, but not that an RFC was ready).

Also, some of the code here comes from the rust branch, but some of it may not be ready yet for upstreaming.

Cheers,
Miguel

#132 lore raw
In reply to Christian Brauner
Quoted 22 lines · Christian Brauner
Quoted 3 lines · Ariel Miculas

Hi all!

This is a proof of concept driver written for the PuzzleFS

Uhm, the PuzzleFS filesystem isn't actually sent to fsdevel? Yet I see tons of patches in there that add wrappers to our core fs data structures. I even see a ton of new files that all fall clearly into fsdevel territory:

create mode 100644 rust/kernel/cred.rs
create mode 100644 rust/kernel/delay.rs
create mode 100644 rust/kernel/driver.rs
create mode 100644 rust/kernel/file.rs
create mode 100644 rust/kernel/fs.rs
create mode 100644 rust/kernel/fs/param.rs
create mode 100644 rust/kernel/io_buffer.rs
create mode 100644 rust/kernel/iov_iter.rs
create mode 100644 rust/kernel/mm.rs
create mode 100644 rust/kernel/mount.rs
create mode 100644 rust/kernel/pages.rs

Quoted 5 lines

There's also quite a lot of new mm/ in there, no?

Any wrappers and code for core fs should be maintained as part of fs. Rust shouldn't become a way to avoid our reviews once you have a few wrappers added somewhere.

I'm of the opinion that Rust wrappers should generally live in the same directories as the code they're wrapping. The Rust wrappers are unsafe code that is intimately coupled with the C code they're creating safe interfaces for, and the organizational structure should reflect that.

This'll be particularly important when mixing Rust and C in the same module, as I intend to do in bcachefs in the not-too-distant future.

#133 lore raw
Quoted 5 lines · Kent Overstreet

I'm of the opinion that Rust wrappers should generally live in the same directories as the code they're wrapping. The Rust wrappers are unsafe code that is intimately coupled with the C code they're creating safe interfaces for, and the organizational structure should reflect that.

+1, that is the plan, which also means MAINTAINERS does not need extra F fields etc.

Cheers,
Miguel

#134 lore raw
In reply to Ariel Miculas
Quoted 4 lines · Ariel Miculas

Hi all!

This is a proof of concept driver written for the PuzzleFS next-generation container filesystem [1].

I can't find the link to the git tree that corresponds to the patch series you just posted. Could you make sure that is front and center when submitting large patch series? I find it a lot easier to get my head around a large body of code by fetching into my local repository.