Skip to contentSkip to conversation

Add Rust support, implement ARM PL011

RFC PATCH Superseded Manos Pitsidianakis 8 participants 38 messages Jul 4, 2024 – Jul 12, 2024

Subsystems (2)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v4 cover Add Rust support, implement ARM PL011

Changes from v3->v4:
- Add rust-specific files to .gitattributes
- Added help text to scripts/cargo_wrapper.py arguments (thanks Stephan)
- Split bindings separate crate
- Add declarative macros for symbols exported to QEMU to said crate
- Lowered MSRV to 1.77.2
- Removed auto-download and install of bindgen-cli
- Fixed re-compilation of Rust objects in case they are missing from filesystem
- Fixed optimized builds by adding #[used] (thanks Pierrick for the help debugging this)

Also, Pierrick helped confirming it works on Windows with some windows-specific changes. I confirmed it works on macos by allowing bindgen to detect system paths for clang, which is a workaround and not a solution. However this series doesn't have the windows changes integrated.

Changes from v2->v3:
- Addressed minor mistakes (thanks Stefan)
- Setup supported version checks for cargo, rustc and bindgen (thanks everyone who pointed it out / suggested it)
- Fixed problem with bindgen failing if certain system headers where needed by defining an allowlist for headers instead of a blocklist for what we don't want (thanks Alex Bennée for reporting it)
- Cleaned up bindgen target/dependendy definition in meson.build by removing unnecessary bits

Changes from v1->v2:
- Create bindgen target first, then add commit for device (thanks Pierrick)
- Create a special named generated.rs for each target as compilation would fail if more than one targets were defined. The generated.rs target names would clash.
- Add more descriptive commit messages
- Update MAINTAINERS
- Cleanup patch order for better review, hopefully

Manos Pitsidianakis (7):
build-sys: Add rust feature option
rust: add bindgen step as a meson dependency
rust: add crate to expose bindings and interfaces
rust: add PL011 device model
.gitattributes: add Rust diff and merge attributes
DO NOT MERGE: add rustdoc build for gitlab pages
DO NOT MERGE: replace TYPE_PL011 with x-pl011-rust in arm virt machine

 .gitattributes                    |   3 +
 .gitlab-ci.d/buildtest.yml        |  64 +++-
 MAINTAINERS                       |  20 ++
 configure                         |  11 +
 hw/arm/virt.c                     |   4 +
 meson.build                       |  72 ++++
 meson_options.txt                 |   5 +
 rust/.cargo/config.toml           |   2 +
 rust/.gitignore                   |   3 +
 rust/meson.build                  | 114 ++++++
 rust/pl011/.gitignore             |   2 +
 rust/pl011/Cargo.lock             | 125 +++++++
 rust/pl011/Cargo.toml             |  67 ++++
 rust/pl011/README.md              |  31 ++
 rust/pl011/deny.toml              |  57 +++
 rust/pl011/meson.build            |   7 +
 rust/pl011/rustfmt.toml           |   1 +
 rust/pl011/src/definitions.rs     |  39 +++
 rust/pl011/src/device.rs          | 509 +++++++++++++++++++++++++++
 rust/pl011/src/device_class.rs    |  48 +++
 rust/pl011/src/lib.rs             | 556 ++++++++++++++++++++++++++++++
 rust/pl011/src/memory_ops.rs      |  45 +++
 rust/qemu-api/.gitignore          |   2 +
 rust/qemu-api/Cargo.lock          |   7 +
 rust/qemu-api/Cargo.toml          |  59 ++++
 rust/qemu-api/README.md           |  17 +
 rust/qemu-api/build.rs            |  48 +++
 rust/qemu-api/deny.toml           |  57 +++
 rust/qemu-api/meson.build         |   0
 rust/qemu-api/rustfmt.toml        |   1 +
 rust/qemu-api/src/bindings.rs     |   8 +
 rust/qemu-api/src/definitions.rs  | 112 ++++++
 rust/qemu-api/src/device_class.rs | 131 +++++++
 rust/qemu-api/src/lib.rs          |  29 ++
 rust/qemu-api/src/tests.rs        |  48 +++
 rust/rustfmt.toml                 |   7 +
 rust/wrapper.h                    |  39 +++
 scripts/cargo_wrapper.py          | 288 ++++++++++++++++
 scripts/meson-buildoptions.sh     |   6 +
 39 files changed, 2625 insertions(+), 19 deletions(-)
 create mode 100644 rust/.cargo/config.toml
 create mode 100644 rust/.gitignore
 create mode 100644 rust/meson.build
 create mode 100644 rust/pl011/.gitignore
 create mode 100644 rust/pl011/Cargo.lock
 create mode 100644 rust/pl011/Cargo.toml
 create mode 100644 rust/pl011/README.md
 create mode 100644 rust/pl011/deny.toml
 create mode 100644 rust/pl011/meson.build
 create mode 120000 rust/pl011/rustfmt.toml
 create mode 100644 rust/pl011/src/definitions.rs
 create mode 100644 rust/pl011/src/device.rs
 create mode 100644 rust/pl011/src/device_class.rs
 create mode 100644 rust/pl011/src/lib.rs
 create mode 100644 rust/pl011/src/memory_ops.rs
 create mode 100644 rust/qemu-api/.gitignore
 create mode 100644 rust/qemu-api/Cargo.lock
 create mode 100644 rust/qemu-api/Cargo.toml
 create mode 100644 rust/qemu-api/README.md
 create mode 100644 rust/qemu-api/build.rs
 create mode 100644 rust/qemu-api/deny.toml
 create mode 100644 rust/qemu-api/meson.build
 create mode 120000 rust/qemu-api/rustfmt.toml
 create mode 100644 rust/qemu-api/src/bindings.rs
 create mode 100644 rust/qemu-api/src/definitions.rs
 create mode 100644 rust/qemu-api/src/device_class.rs
 create mode 100644 rust/qemu-api/src/lib.rs
 create mode 100644 rust/qemu-api/src/tests.rs
 create mode 100644 rust/rustfmt.toml
 create mode 100644 rust/wrapper.h
 create mode 100644 scripts/cargo_wrapper.py

base-commit: 7914bda497f07965f15a91905cd7ed9eaf1c1092

Signature · 1 line
γαῖα πυρί μιχθήτω
#1 lore raw

RFC PATCH v4 1/7 build-sys: Add rust feature option

Add options for Rust in meson_options.txt, meson.build, configure to prepare for adding Rust code in the followup commits.

rust is a reserved meson name, so we have to use an alternative. with_rust was chosen.

A cargo_wrapper.py script is added that is heavily based on the work of Marc-André Lureau from 2021.

https://patchew.org/QEMU/[email protected]/

---
 MAINTAINERS                   |   5 +
 configure                     |  11 ++
 meson.build                   |  11 ++
 meson_options.txt             |   5 +
 scripts/cargo_wrapper.py      | 294 ++++++++++++++++++++++++++++++++++
 scripts/meson-buildoptions.sh |   6 +
 6 files changed, 332 insertions(+)
 create mode 100644 scripts/cargo_wrapper.py
MAINTAINERS +5 -0
index 6725913c8b..d01bd06ab7 100644
@@ -4226,6 +4226,11 @@ F: docs/sphinx/
 F: docs/_templates/
 F: docs/devel/docs.rst
 
+Rust build system integration
+M: Manos Pitsidianakis <[email protected]>
+S: Maintained
+F: scripts/cargo_wrapper.py
+
 Miscellaneous
 -------------
 Performance Tools and Tests
configure +11 -0
index 8b6a2f16ce..180d87b9c9 100755
@@ -302,6 +302,9 @@ else
   objcc="${objcc-${cross_prefix}clang}"
 fi
 
+with_rust="auto"
+with_rust_target_triple=""
+
 ar="${AR-${cross_prefix}ar}"
 as="${AS-${cross_prefix}as}"
 ccas="${CCAS-$cc}"
@@ -757,6 +760,12 @@ for opt do
   ;;
   --gdb=*) gdb_bin="$optarg"
   ;;
+  --enable-with-rust) with_rust=enabled
+  ;;
+  --disable-with-rust) with_rust=disabled
+  ;;
+  --with-rust-target-triple=*) with_rust_target_triple="$optarg"
+  ;;
   # everything else has the same name in configure and meson
   --*) meson_option_parse "$opt" "$optarg"
   ;;
@@ -1789,6 +1798,8 @@ if test "$skip_meson" = no; then
   test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
   test "$plugins" = yes && meson_option_add "-Dplugins=true"
   test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg"
+  test "$with_rust" != enabled && meson_option_add "-Dwith_rust=$with_rust"
+  test "$with_rust_target_triple" != "" && meson_option_add "-Dwith_rust_target_triple=$with_rust_target_triple"
   run_meson() {
     NINJA=$ninja $meson setup "$@" "$PWD" "$source_path"
   }
meson.build +11 -0
index 2f981f936e..11b8b146da 100644
@@ -290,6 +290,12 @@ foreach lang : all_languages
   endif
 endforeach
 
+cargo = not_found
+if get_option('with_rust').allowed()
+  cargo = find_program('cargo', required: get_option('with_rust'))
+endif
+with_rust = cargo.found()
+
 # default flags for all hosts
 # We use -fwrapv to tell the compiler that we require a C dialect where
 # left shift of signed integers is well defined and has the expected
@@ -2118,6 +2124,7 @@ endif
 
 config_host_data = configuration_data()
 
+config_host_data.set('CONFIG_WITH_RUST', with_rust)
 audio_drivers_selected = []
 if have_system
   audio_drivers_available = {
@@ -4243,6 +4250,10 @@ if 'objc' in all_languages
 else
   summary_info += {'Objective-C compiler': false}
 endif
+summary_info += {'Rust support':      with_rust}
+if with_rust and get_option('with_rust_target_triple') != ''
+  summary_info += {'Rust target':     get_option('with_rust_target_triple')}
+endif
 option_cflags = (get_option('debug') ? ['-g'] : [])
 if get_option('optimization') != 'plain'
   option_cflags += ['-O' + get_option('optimization')]
meson_options.txt +5 -0
index 0269fa0f16..3443c48001 100644
@@ -371,3 +371,8 @@ option('hexagon_idef_parser', type : 'boolean', value : true,
 
 option('x86_version', type : 'combo', choices : ['0', '1', '2', '3', '4'], value: '1',
        description: 'tweak required x86_64 architecture version beyond compiler default')
+
+option('with_rust', type: 'feature', value: 'auto',
+       description: 'Enable Rust support')
+option('with_rust_target_triple', type : 'string', value: '',
+       description: 'Override Rust target triple')
scripts/cargo_wrapper.py new file +294 -0

294 lines not on this page · raw message

scripts/meson-buildoptions.sh +6 -0
index cfadb5ea86..23a24ccaa7 100644
@@ -79,6 +79,8 @@ meson_options_help() {
   printf "%s\n" '                           auto/sigaltstack/ucontext/windows)'
   printf "%s\n" '  --with-pkgversion=VALUE  use specified string as sub-version of the'
   printf "%s\n" '                           package'
+  printf "%s\n" '  --with-rust-target-triple=VALUE'
+  printf "%s\n" '                           Specify Rust host target triple'
   printf "%s\n" '  --with-suffix=VALUE      Suffix for QEMU data/modules/config directories'
   printf "%s\n" '                           (can be empty) [qemu]'
   printf "%s\n" '  --with-trace-file=VALUE  Trace file prefix for simple backend [trace]'
@@ -216,6 +218,7 @@ meson_options_help() {
   printf "%s\n" '  vvfat           vvfat image format support'
   printf "%s\n" '  werror          Treat warnings as errors'
   printf "%s\n" '  whpx            WHPX acceleration support'
+  printf "%s\n" '  with-rust       Enable Rust support'
   printf "%s\n" '  xen             Xen backend support'
   printf "%s\n" '  xen-pci-passthrough'
   printf "%s\n" '                  Xen PCI passthrough support'
@@ -552,6 +555,9 @@ _meson_option_parse() {
     --enable-whpx) printf "%s" -Dwhpx=enabled ;;
     --disable-whpx) printf "%s" -Dwhpx=disabled ;;
     --x86-version=*) quote_sh "-Dx86_version=$2" ;;
+    --enable-with-rust) printf "%s" -Dwith_rust=enabled ;;
+    --disable-with-rust) printf "%s" -Dwith_rust=disabled ;;
+    --with-rust-target-triple=*) quote_sh "-Dwith_rust_target_triple=$2" ;;
     --enable-xen) printf "%s" -Dxen=enabled ;;
     --disable-xen) printf "%s" -Dxen=disabled ;;
     --enable-xen-pci-passthrough) printf "%s" -Dxen_pci_passthrough=enabled ;;
Signature · 1 line
γαῖα πυρί μιχθήτω
#2 lore raw

Re: [RFC PATCH v4 1/7] build-sys: Add rust feature option

Quoted 6 lines · Manos Pitsidianakis

Add options for Rust in meson_options.txt, meson.build, configure to prepare for adding Rust code in the followup commits.

rust is a reserved meson name, so we have to use an alternative. with_rust was chosen.

Did you find any problem with the other approach that I sent, to support --cargo and the CARGO environment variable in configure?

Paolo

#3 lore raw

RFC PATCH v4 2/7 rust: add bindgen step as a meson dependency

In reply to Manos Pitsidianakis

Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.

This way bindings will be created before the rust crate is compiled.

The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name as a target:

ninja aarch64-softmmu-generated.rs

The way the bindings are generated is:

1. All required C headers are included in a single file, in our case rust/wrapper.h for convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.

2. Meson creates a generated_rs target that runs bindgen making sure the architecture etc header dependencies are present.

3. The generated_rs target takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.

4. Meson creates rust hardware target dependencies from the rust_targets dictionary defined in rust/meson.build.

Since we cannot declare a dependency on generated_rs before it is declared in meson.build, the rust crate targets must be defined after the generated_rs target for each target architecture is defined. This way meson sets up the dependency tree properly.

5. After compiling each rust crate with the cargo_wrapper.py script, its static library artifact is linked as a whole-archive with the final binary.

---
 MAINTAINERS              |   3 ++
 meson.build              |  57 ++++++++++++++++++++
 rust/.gitignore          |   3 ++
 rust/meson.build         | 112 +++++++++++++++++++++++++++++++++++++++
 rust/wrapper.h           |  39 ++++++++++++++
 scripts/cargo_wrapper.py |  18 +++----
 6 files changed, 220 insertions(+), 12 deletions(-)
 create mode 100644 rust/.gitignore
 create mode 100644 rust/meson.build
 create mode 100644 rust/wrapper.h
MAINTAINERS +3 -0
index d01bd06ab7..6e7b8207fb 100644
@@ -4230,6 +4230,9 @@ Rust build system integration
 M: Manos Pitsidianakis <[email protected]>
 S: Maintained
 F: scripts/cargo_wrapper.py
+F: rust/meson.build
+F: rust/wrapper.h
+F: rust/.gitignore
 
 Miscellaneous
 -------------
meson.build +57 -0

63 lines not on this page · raw message

rust/.gitignore new file +3 -0
new file mode 100644
index 0000000000..1bf71b1f68
@@ -0,0 +1,3 @@
+# Ignore any cargo development build artifacts; for qemu-wide builds, all build
+# artifacts will go to the meson build directory.
+target
rust/meson.build new file +112 -0

112 lines not on this page · raw message

rust/wrapper.h new file +39 -0
new file mode 100644
index 0000000000..51985f0ef1
@@ -0,0 +1,39 @@
+/*
+ * QEMU System Emulator
+ *
+ * Copyright 2024 Manos Pitsidianakis <[email protected]>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/module.h"
+#include "qemu-io.h"
+#include "sysemu/sysemu.h"
+#include "hw/sysbus.h"
+#include "exec/memory.h"
+#include "chardev/char-fe.h"
+#include "hw/clock.h"
+#include "hw/qdev-clock.h"
+#include "hw/qdev-properties.h"
+#include "hw/qdev-properties-system.h"
+#include "hw/irq.h"
+#include "qapi/error.h"
+#include "migration/vmstate.h"
+#include "chardev/char-serial.h"
scripts/cargo_wrapper.py +6 -12
index d2c7265461..e7d9238c16 100644
@@ -111,6 +111,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str]
 
     env = os.environ
     env["CARGO_ENCODED_RUSTFLAGS"] = cfg
+    env["MESON_BUILD_DIR"] = str(target_dir)
+    env["MESON_BUILD_ROOT"] = str(args.meson_build_root)
 
     return (env, cargo_cmd)
 
@@ -231,19 +233,11 @@ def main() -> None:
         default=[],
     )
     parser.add_argument(
-        "--meson-build-dir",
-        metavar="BUILD_DIR",
-        help="meson.current_build_dir()",
+        "--meson-build-root",
+        metavar="BUILD_ROOT",
+        help="meson.project_build_root(): the root build directory. Example: '/path/to/qemu/build'",
         type=Path,
-        dest="meson_build_dir",
-        required=True,
-    )
-    parser.add_argument(
-        "--meson-source-dir",
-        metavar="SOURCE_DIR",
-        help="meson.current_source_dir()",
-        type=Path,
-        dest="meson_build_dir",
+        dest="meson_build_root",
         required=True,
     )
     parser.add_argument(
Signature · 1 line
γαῖα πυρί μιχθήτω
#4 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 10 lines · Manos Pitsidianakis

Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.

This way bindings will be created before the rust crate is compiled.

The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name as a target:

ninja aarch64-softmmu-generated.rs

Is there anything target-dependent in these files? You can generate it just once I think.

meson.build foreach target : target_dirs · line 3937

Quoted 6 lines
meson.build +6 -0
+  if with_rust and target_type == 'system'
+       # FIXME: meson outputs the following warnings, which should be resolved
+       # before merging:
+       # > WARNING: Project specifies a minimum meson_version '>=0.63.0' but
+       # > uses features which were added in newer versions:
+       # > * 0.64.0: {'fs.copyfile'}

you can use configure_file() instead.

Quoted 1 line

+ # > * 1.0.0: {'dependencies arg in rust.bindgen', 'module rust as stable module'}

You can use

if meson.version().version_compare('>=1.0.0')
...
else
error('Rust support requires Meson version 1.0.0')
endif

meson.build foreach target : target_dirs · line 3974

Quoted 8 lines
meson.build +8 -0
+      if target in rust_targets
+        rust_hw = ss.source_set()
+        foreach t: rust_targets[target]
+          rust_device_cargo_build = custom_target(t['name'],
+                                       output: t['output'],
+                                       depends: [bindings_rs],
+                                       build_always_stale: true,
+                                       command: t['command'])

Also "console: true".

meson.build foreach target : target_dirs · line 3979

Quoted 5 lines
meson.build +5 -0
+          rust_dep = declare_dependency(link_args: [
+                                          '-Wl,--whole-archive',
+                                          t['output'],
+                                          '-Wl,--no-whole-archive'
+                                          ],

This is not portable, but I think you can use just "link_whole:
rust_device_cargo_build".

rust/meson.build line 33

Quoted 3 lines
rust/meson.build new file +3 -0
+msrv = {
+  'rustc': '1.77.2',
+  'cargo': '1.77.2',

I think rustc and cargo are always matching, so no need to check both of them?

rust/meson.build line 97

Quoted 2 lines
rust/meson.build new file +2 -0
+foreach rust_hw_target, rust_hws: rust_hw_target_list
+  foreach rust_hw_dev: rust_hws

Finding the available devices should be done using Kconfig symbols, probably by passing the path to the config-devices.mak file to build.rs via an environment variable.

Quoted 5 lines
rust/wrapper.h new file +5 -0
+#include "qemu/osdep.h"
+#include "qemu/module.h"
+#include "qemu-io.h"
+#include "sysemu/sysemu.h"
+#include "hw/sysbus.h"

rust/wrapper.h line 39

Quoted 10 lines
rust/wrapper.h new file +10 -0
+#include "exec/memory.h"
+#include "chardev/char-fe.h"
+#include "hw/clock.h"
+#include "hw/qdev-clock.h"
+#include "hw/qdev-properties.h"
+#include "hw/qdev-properties-system.h"
+#include "hw/irq.h"
+#include "qapi/error.h"
+#include "migration/vmstate.h"
+#include "chardev/char-serial.h"

I think all of these should be target-independent?

Quoted 27 lines
scripts/cargo_wrapper.py +5 -6
index d2c7265461..e7d9238c16 100644
@@ -111,6 +111,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str]
 
     env = os.environ
     env["CARGO_ENCODED_RUSTFLAGS"] = cfg
+    env["MESON_BUILD_DIR"] = str(target_dir)
+    env["MESON_BUILD_ROOT"] = str(args.meson_build_root)
 
     return (env, cargo_cmd)
 
@@ -231,19 +233,11 @@ def main() -> None:
         default=[],
     )
     parser.add_argument(
-        "--meson-build-dir",
-        metavar="BUILD_DIR",
-        help="meson.current_build_dir()",
+        "--meson-build-root",
+        metavar="BUILD_ROOT",
+        help="meson.project_build_root(): the root build directory. Example: '/path/to/qemu/build'",
         type=Path,
-        dest="meson_build_dir",
-        required=True,
-    )

scripts/cargo_wrapper.py def main() -> None: · line 243

Quoted 10 lines
scripts/cargo_wrapper.py +1 -6
-    parser.add_argument(
-        "--meson-source-dir",
-        metavar="SOURCE_DIR",
-        help="meson.current_source_dir()",
-        type=Path,
-        dest="meson_build_dir",
+        dest="meson_build_root",
         required=True,
     )
     parser.add_argument(

Please squash in the previous patch.

Paolo

#5 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

In reply to Manos Pitsidianakis
Quoted 36 lines · Manos Pitsidianakis

Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.

This way bindings will be created before the rust crate is compiled.

The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name as a target:

ninja aarch64-softmmu-generated.rs

The way the bindings are generated is:

1. All required C headers are included in a single file, in our case rust/wrapper.h for convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.

2. Meson creates a generated_rs target that runs bindgen making sure the architecture etc header dependencies are present.

3. The generated_rs target takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.

4. Meson creates rust hardware target dependencies from the rust_targets dictionary defined in rust/meson.build.

Since we cannot declare a dependency on generated_rs before it is declared in meson.build, the rust crate targets must be defined after the generated_rs target for each target architecture is defined. This way meson sets up the dependency tree properly.

5. After compiling each rust crate with the cargo_wrapper.py script, its static library artifact is linked as a whole-archive with the final binary.

<snip>

rust/meson.build line 35

Quoted 6 lines
rust/meson.build new file +6 -0
+
+msrv = {
+  'rustc': '1.77.2',
+  'cargo': '1.77.2',
+  'bindgen': '0.69.4',
+}

This is still pretty bleeding edge (it even tripped up on the .cargo/bin/cargo I have installed). This needs to be set to the baseline which from:

https://wiki.qemu.org/RustInQemu/2022

Looks to be 1.24.0 for rustc and I guess even lower for cargo (Debian says 0.66.0). While it might make sense to delay merging if we are waiting for one distro to produce a new LTS we shouldn't be needing rustup by default.

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#6 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 12 lines · Alex Bennée

rust/meson.build line 35

Quoted 5 lines · Manos Pitsidianakis
rust/meson.build new file +5 -0
+msrv = {
+  'rustc': '1.77.2',
+  'cargo': '1.77.2',
+  'bindgen': '0.69.4',
+}

This is still pretty bleeding edge (it even tripped up on the .cargo/bin/cargo I have installed). This needs to be set to the baseline which from:

Quoted 6 lines

https://wiki.qemu.org/RustInQemu/2022

Looks to be 1.24.0 for rustc and I guess even lower for cargo (Debian says 0.66.0). While it might make sense to delay merging if we are waiting for one distro to produce a new LTS we shouldn't be needing rustup by default.

I suspect that some of the older distro releases in that chart have already fallen off the end of our supported-platforms set, so the minimum is probably newer than 1.24.0 now.

My take on this one is that (at some point, not necessarily right now) we want to look at:

* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates. I think (but forget the details) that some of what we've done with Python where we accept that the older-but-still-supported distro will end up taking the "download at build time" path in the build system might apply also for Rust.
* what, on the Rust side, is the version requirement? Presumably there's some features that are pretty much "we really need this and can't do without it" and some which are "this would be pretty awkward not to have but if we have to we can implement some kind of fallback/alternative with a TODO note to come back and clean up when our baseline moves forwards".

At that point we have more information to figure out what if any tradeoff we want to make.

thanks
-- PMM

#7 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 7 lines · Peter Maydell

* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.

I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.

We can define a known-good set of versions:
* either via Cargo.lock (i.e. recording the versions and downloading them) or by including sources in the tree
* at any time in qemu.git, at branching time (three times a year), or on every release

(That's six possibilities overall).

Quoted 4 lines

I think (but forget the details) that some of what we've done with Python where we accept that the older-but-still-supported distro will end up taking the "download at build time" path in the build system might apply also for Rust.

Yes, and --without-download may be changed to the "--frozen" flag of cargo.

Quoted 6 lines

* what, on the Rust side, is the version requirement? Presumably there's some features that are pretty much "we really need this and can't do without it" and some which are "this would be pretty awkward not to have but if we have to we can implement some kind of fallback/alternative with a TODO note to come back and clean up when our baseline moves forwards".

Here are the stopping points that I found over the last couple weeks:

1.56.0: 2021 edition
1.59.0: const CStr::from_bytes_with_nul_unchecked (needed by cstr crate, see below)
1.64.0: std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals, offset_of!

I think 1.59.0 is pretty much the lower bound. Not having offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).

As far as I understand, Debian bullseye has 1.63.0 these days and it's the oldest platform we have.

Paolo

Quoted 6 lines

At that point we have more information to figure out what if any tradeoff we want to make.

thanks
-- PMM

#8 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell

* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.

Quoted 3 lines

I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.

Yep, Rust is new enough that it is highly likely that crates will be missing in multiple distros.

For ordinary users, cargo will happily download the missing pieces so its not an issue.

For distro packagers, they'll just have to either package up the crates, or bundle them in their QEMU build. Cargo makes the latter easy at least. If distros don't want bundling, they'll need to go the more involved route of packaging deps.

IOW, from a distro POV, IMHO, we should focus on the Rust toolchain versions we need as the minimum bar.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#9 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 25 lines · Daniel P. Berrangé
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell

* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.

Quoted 3 lines

I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.

Yep, Rust is new enough that it is highly likely that crates will be missing in multiple distros.

For ordinary users, cargo will happily download the missing pieces so its not an issue.

For distro packagers, they'll just have to either package up the crates, or bundle them in their QEMU build. Cargo makes the latter easy at least. If distros don't want bundling, they'll need to go the more involved route of packaging deps.

Quoted 5 lines

IOW, from a distro POV, IMHO, we should focus on the Rust toolchain versions we need as the minimum bar.

With regards,
Daniel

I would like to add that, contrary to pip packages for python, it can be *trusted* that Rust crates.io will keep packages and their different versions, as it was designed to be an immutable registry.

Indeed, users can't remove their package, except for strong security issues, to the opposite of pip.

Besides offline compilation scenario, are there any other reason for wanting to use distro packages instead of letting cargo do the work?

#10 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 29 lines · Pierrick Bouvier
Quoted 28 lines · Daniel P. Berrangé
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell

* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.

Quoted 3 lines

I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.

Yep, Rust is new enough that it is highly likely that crates will be missing in multiple distros.

For ordinary users, cargo will happily download the missing pieces so its not an issue.

For distro packagers, they'll just have to either package up the crates, or bundle them in their QEMU build. Cargo makes the latter easy at least. If distros don't want bundling, they'll need to go the more involved route of packaging deps.

IOW, from a distro POV, IMHO, we should focus on the Rust toolchain versions we need as the minimum bar.

Quoted 8 lines
Quoted 2 lines

With regards,
Daniel

I would like to add that, contrary to pip packages for python, it can be *trusted* that Rust crates.io will keep packages and their different versions, as it was designed to be an immutable registry.

Indeed, users can't remove their package, except for strong security issues, to the opposite of pip.

Besides offline compilation scenario, are there any other reason for wanting to use distro packages instead of letting cargo do the work?

Only distribution packaging is the concern here, where packaged build dependencies may be required.

#11 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

In reply to Paolo Bonzini
Quoted 45 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell

* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.

I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.

We can define a known-good set of versions:
* either via Cargo.lock (i.e. recording the versions and downloading them) or by including sources in the tree
* at any time in qemu.git, at branching time (three times a year), or on every release

(That's six possibilities overall).

Quoted 4 lines

I think (but forget the details) that some of what we've done with Python where we accept that the older-but-still-supported distro will end up taking the "download at build time" path in the build system might apply also for Rust.

Yes, and --without-download may be changed to the "--frozen" flag of cargo.

Quoted 6 lines

* what, on the Rust side, is the version requirement? Presumably there's some features that are pretty much "we really need this and can't do without it" and some which are "this would be pretty awkward not to have but if we have to we can implement some kind of fallback/alternative with a TODO note to come back and clean up when our baseline moves forwards".

Here are the stopping points that I found over the last couple weeks:

1.56.0: 2021 edition
1.59.0: const CStr::from_bytes_with_nul_unchecked (needed by cstr crate, see below)
1.64.0: std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals, offset_of!

Quoted 6 lines

I think 1.59.0 is pretty much the lower bound. Not having offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).

As far as I understand, Debian bullseye has 1.63.0 these days and it's the oldest platform we have.

I was going to say Bookworm has now superseded Bullseye which will reach its release + 3 year support point in August. However the version you mention in the Bookworm one!

Quoted 9 lines

Paolo

Quoted 6 lines

At that point we have more information to figure out what if any tradeoff we want to make.

thanks
-- PMM

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#12 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

In reply to Paolo Bonzini
Quoted 11 lines · Paolo Bonzini

Here are the stopping points that I found over the last couple weeks:

1.56.0: 2021 edition
1.59.0: const CStr::from_bytes_with_nul_unchecked (needed by cstr crate, see below)
1.64.0: std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals, offset_of!

Quoted 4 lines

I think 1.59.0 is pretty much the lower bound. Not having offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).

An additional question: does our minimum rust version requirement indicate that users with this rust version can compile other dependencies that satisfy QEMU requirements, such as bindgen?

Because I find 1.59.0 can only go to compile bindgen 0.63.0 [1].

[1]: bindgen 0.63.0 MSRV: https://github.com/rust-lang/rust-bindgen/tree/v0.63.0?tab=readme-ov-file#msrv

-Zhao

#13 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 13 lines · Zhao Liu
Quoted 11 lines · Paolo Bonzini

Here are the stopping points that I found over the last couple weeks:

1.56.0: 2021 edition
1.59.0: const CStr::from_bytes_with_nul_unchecked (needed by cstr crate, see below)
1.64.0: std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals, offset_of!

Quoted 8 lines
Quoted 4 lines

I think 1.59.0 is pretty much the lower bound. Not having offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).

An additional question: does our minimum rust version requirement indicate that users with this rust version can compile other dependencies that satisfy QEMU requirements, such as bindgen?

Yes (though in the case of bindgen, like cargo and rustc, we'll use it from the distro; Debian bookworm has 0.60.1 so that's what we'll have to stick with).

Paolo

#14 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 23 lines · Paolo Bonzini

Date: Wed, 10 Jul 2024 16:50:10 +0200
From: Paolo Bonzini <[email protected]>
Subject: Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

Quoted 18 lines · Zhao Liu
Quoted 15 lines · Paolo Bonzini

Here are the stopping points that I found over the last couple weeks:

1.56.0: 2021 edition
1.59.0: const CStr::from_bytes_with_nul_unchecked (needed by cstr crate, see below)
1.64.0: std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals, offset_of!

I think 1.59.0 is pretty much the lower bound. Not having offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).

Quoted 8 lines
Quoted 3 lines

An additional question: does our minimum rust version requirement indicate that users with this rust version can compile other dependencies that satisfy QEMU requirements, such as bindgen?

Yes (though in the case of bindgen, like cargo and rustc, we'll use it from the distro; Debian bookworm has 0.60.1 so that's what we'll have to stick with).

I just did some build tests to hopefully help Manos clarify the gap.

With 1.59.0 rust and 0.60.1 bindgen, I found the big problem is this version can't support current external crates ("bilge" and "arbitrary-int").

The lowest possible version of bilge is 0.1.1 (because 0.1.0 contains the removed unstable feature), but v0.1.1 bilge depends on arbitrary-int (lowest bound is 1.2.4), which is not supported by v1.59.0 rust. So for this case either we have to keep raising the minimum rust version, or we have to reconsider the external crates.

And with 1.63.0, the main problem is core_ffi_c is unstable (once_cell doesn't matter since it's unneeded). Roughly speaking, it seems that if we didn't have to use the c type as parameters, this problem could possibly be worked around.

So maybe 1.63.0 is the proper minimum version and this is aligned with Debian bullseye.

#15 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

In reply to Alex Bennée
Quoted 50 lines · Alex Bennée
Quoted 36 lines · Manos Pitsidianakis

Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.

This way bindings will be created before the rust crate is compiled.

The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name as a target:

ninja aarch64-softmmu-generated.rs

The way the bindings are generated is:

1. All required C headers are included in a single file, in our case rust/wrapper.h for convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.

2. Meson creates a generated_rs target that runs bindgen making sure the architecture etc header dependencies are present.

3. The generated_rs target takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.

4. Meson creates rust hardware target dependencies from the rust_targets dictionary defined in rust/meson.build.

Since we cannot declare a dependency on generated_rs before it is declared in meson.build, the rust crate targets must be defined after the generated_rs target for each target architecture is defined. This way meson sets up the dependency tree properly.

5. After compiling each rust crate with the cargo_wrapper.py script, its static library artifact is linked as a whole-archive with the final binary.

<snip>

rust/meson.build line 35

Quoted 6 lines
rust/meson.build new file +6 -0
+
+msrv = {
+  'rustc': '1.77.2',
+  'cargo': '1.77.2',
+  'bindgen': '0.69.4',
+}

This is still pretty bleeding edge (it even tripped up on the .cargo/bin/cargo I have installed). This needs to be set to the baseline which from:

Quoted 6 lines

https://wiki.qemu.org/RustInQemu/2022

Looks to be 1.24.0 for rustc and I guess even lower for cargo (Debian says 0.66.0). While it might make sense to delay merging if we are waiting for one distro to produce a new LTS we shouldn't be needing rustup by default.

Also bindgen, do we need such a new one? On Trixie:

Message:

../../rust/meson.build:41:4: ERROR: Problem encountered: bindgen version 0.66.1 is unsupported: Please upgrade to at least 0.69.4

A full log can be found at /home/alex/lsrc/qemu.git/builds/rust/meson-logs/meson-log.txt

ERROR: meson setup failed

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#16 lore raw

Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency

In reply to Manos Pitsidianakis
Quoted 3 lines · Manos Pitsidianakis

Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.

<snip>

meson.build foreach target : target_dirs · line 3952

Quoted 12 lines
meson.build +12 -0
+      # We need one bindings_rs build target per arch target, so give them
+      # arch-specific names.
+      copy = fs.copyfile('rust/wrapper.h',
+                         target + '_wrapper.h')
+      bindings_rs = rust_bindgen.bindgen(
+        input: copy,
+        dependencies: arch_deps + lib_deps,
+        output: 'bindings-' + target + '.rs',
+        include_directories: include_directories('.', 'include'),
+        args: [
+          '--ctypes-prefix', 'core::ffi',
+          '--formatter', 'rustfmt',

I guess we also need to detect rustfmt, although it seems to be non-fatal:

15:59:27 [alex@debian-arm64:~/l/q/b/rust] review/rust-pl011-v4 + ninja [831/3041] Generating bindings for Rust rustmod-bindgen-aarch64-softmmu_wrapper.h Failed to run rustfmt: cannot find binary path (non-fatal, continuing) [3041/3041] Linking target tests/qtest/qos-test

<snip>

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#17 lore raw

RFC PATCH v4 3/7 rust: add crate to expose bindings and interfaces

In reply to Manos Pitsidianakis

Add rust/qemu-api, which exposes rust-bindgen generated FFI bindings and provides some declaration macros for symbols visible to the rest of QEMU.

---
 MAINTAINERS                       |   7 ++
 rust/.cargo/config.toml           |   2 +
 rust/qemu-api/.gitignore          |   2 +
 rust/qemu-api/Cargo.lock          |   7 ++
 rust/qemu-api/Cargo.toml          |  59 ++++++++++++++
 rust/qemu-api/README.md           |  17 ++++
 rust/qemu-api/build.rs            |  48 +++++++++++
 rust/qemu-api/deny.toml           |  57 +++++++++++++
 rust/qemu-api/meson.build         |   0
 rust/qemu-api/rustfmt.toml        |   1 +
 rust/qemu-api/src/bindings.rs     |   8 ++
 rust/qemu-api/src/definitions.rs  | 112 +++++++++++++++++++++++++
 rust/qemu-api/src/device_class.rs | 131 ++++++++++++++++++++++++++++++
 rust/qemu-api/src/lib.rs          |  29 +++++++
 rust/qemu-api/src/tests.rs        |  48 +++++++++++
 rust/rustfmt.toml                 |   7 ++
 16 files changed, 535 insertions(+)
 create mode 100644 rust/.cargo/config.toml
 create mode 100644 rust/qemu-api/.gitignore
 create mode 100644 rust/qemu-api/Cargo.lock
 create mode 100644 rust/qemu-api/Cargo.toml
 create mode 100644 rust/qemu-api/README.md
 create mode 100644 rust/qemu-api/build.rs
 create mode 100644 rust/qemu-api/deny.toml
 create mode 100644 rust/qemu-api/meson.build
 create mode 120000 rust/qemu-api/rustfmt.toml
 create mode 100644 rust/qemu-api/src/bindings.rs
 create mode 100644 rust/qemu-api/src/definitions.rs
 create mode 100644 rust/qemu-api/src/device_class.rs
 create mode 100644 rust/qemu-api/src/lib.rs
 create mode 100644 rust/qemu-api/src/tests.rs
 create mode 100644 rust/rustfmt.toml
MAINTAINERS +7 -0
index 6e7b8207fb..8598d38eae 100644
@@ -3340,6 +3340,11 @@ F: hw/core/register.c
 F: include/hw/register.h
 F: include/hw/registerfields.h
 
+Rust
+M: Manos Pitsidianakis <[email protected]>
+S: Maintained
+F: rust/qemu-api
+
 SLIRP
 M: Samuel Thibault <[email protected]>
 S: Maintained
@@ -4233,6 +4238,8 @@ F: scripts/cargo_wrapper.py
 F: rust/meson.build
 F: rust/wrapper.h
 F: rust/.gitignore
+F: rust/rustfmt.toml
+F: rust/.cargo/config.toml
 
 Miscellaneous
 -------------
rust/.cargo/config.toml new file +2 -0
new file mode 100644
index 0000000000..241210ffa7
@@ -0,0 +1,2 @@
+[build]
+rustflags = ["-Crelocation-model=pic", "-Ctarget-feature=+crt-static"]
rust/qemu-api/.gitignore new file +2 -0
new file mode 100644
index 0000000000..71eaff2035
@@ -0,0 +1,2 @@
+# Ignore generated bindings file overrides.
+src/bindings.rs.inc
rust/qemu-api/Cargo.lock new file +7 -0
new file mode 100644
index 0000000000..e9c51a243a
@@ -0,0 +1,7 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "qemu_api"
+version = "0.1.0"
rust/qemu-api/Cargo.toml new file +59 -0

59 lines not on this page · raw message

rust/qemu-api/README.md new file +17 -0
new file mode 100644
index 0000000000..f16a1a929d
@@ -0,0 +1,17 @@
+# QEMU bindings and API wrappers
+
+This library exports helper Rust types, Rust macros and C FFI bindings for internal QEMU APIs.
+
+The C bindings can be generated with `bindgen`, using this build target:
+
+```console
+$ ninja bindings-aarch64-softmmu.rs
+```
+
+## Generate Rust documentation
+
+To generate docs for this crate, including private items:
+
+```sh
+cargo doc --no-deps --document-private-items
+```
rust/qemu-api/build.rs new file +48 -0

48 lines not on this page · raw message

rust/qemu-api/deny.toml new file +57 -0

57 lines not on this page · raw message

rust/qemu-api/meson.build new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
rust/qemu-api/rustfmt.toml new file +1 -0
new file mode 120000
index 0000000000..39f97b043b
@@ -0,0 +1 @@
+../rustfmt.toml
\ No newline at end of file
rust/qemu-api/src/bindings.rs new file +8 -0
new file mode 100644
index 0000000000..1220a3d8f0
@@ -0,0 +1,8 @@
+// Copyright 2024 Manos Pitsidianakis <[email protected]>
+// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later
+
+#[cfg(MESON_BINDINGS_RS)]
+include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
+
+#[cfg(not(MESON_BINDINGS_RS))]
+include!("bindings.rs.inc");
rust/qemu-api/src/definitions.rs new file +112 -0

112 lines not on this page · raw message

rust/qemu-api/src/device_class.rs new file +131 -0

131 lines not on this page · raw message

rust/qemu-api/src/lib.rs new file +29 -0
new file mode 100644
index 0000000000..74825c84e7
@@ -0,0 +1,29 @@
+// Copyright 2024 Manos Pitsidianakis <[email protected]>
+// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later
+
+#![doc = include_str!("../README.md")]
+
+// FIXME: remove improper_ctypes
+#[allow(
+    improper_ctypes_definitions,
+    improper_ctypes,
+    non_camel_case_types,
+    non_snake_case,
+    non_upper_case_globals
+)]
+#[allow(
+    clippy::missing_const_for_fn,
+    clippy::useless_transmute,
+    clippy::too_many_arguments,
+    clippy::approx_constant,
+    clippy::use_self,
+    clippy::cast_lossless,
+)]
+#[rustfmt::skip]
+pub mod bindings;
+
+pub mod definitions;
+pub mod device_class;
+
+#[cfg(test)]
+mod tests;
rust/qemu-api/src/tests.rs new file +48 -0

48 lines not on this page · raw message

rust/rustfmt.toml new file +7 -0
new file mode 100644
index 0000000000..ebecb99fe0
@@ -0,0 +1,7 @@
+edition = "2021"
+format_generated_files = false
+format_code_in_doc_comments = true
+format_strings = true
+imports_granularity = "Crate"
+group_imports = "StdExternalCrate"
+wrap_comments = true
Signature · 1 line
γαῖα πυρί μιχθήτω
#18 lore raw

Re: [RFC PATCH v4 3/7] rust: add crate to expose bindings and interfaces

Quoted 5 lines · Manos Pitsidianakis

Add rust/qemu-api, which exposes rust-bindgen generated FFI bindings and provides some declaration macros for symbols visible to the rest of QEMU.

Quoted 14 lines
---
  MAINTAINERS                       |   7 ++
  rust/.cargo/config.toml           |   2 +
  rust/qemu-api/.gitignore          |   2 +
  rust/qemu-api/Cargo.lock          |   7 ++
  rust/qemu-api/Cargo.toml          |  59 ++++++++++++++
  rust/qemu-api/README.md           |  17 ++++
  rust/qemu-api/build.rs            |  48 +++++++++++
  rust/qemu-api/deny.toml           |  57 +++++++++++++
  rust/qemu-api/meson.build         |   0
  rust/qemu-api/rustfmt.toml        |   1 +
  rust/qemu-api/src/bindings.rs     |   8 ++
  rust/qemu-api/src/definitions.rs  | 112 +++++++++++++++++++++++++
  rust/qemu-api/src/device_class.rs | 131 ++++++++++++++++++++++++++++++

I'd rather put the various definitions in directories that roughly correspond to the C files, so rust/qemu-api/src/{util,qom,hw/core/}.

rust/qemu-api/Cargo.toml line 30

Quoted 7 lines
rust/qemu-api/Cargo.toml new file +7 -0
+correctness = { level = "deny", priority = -1 }
+suspicious = { level = "deny", priority = -1 }
+complexity = { level = "deny", priority = -1 }
+perf = { level = "deny", priority = -1 }
+cargo = { level = "deny", priority = -1 }
+nursery = { level = "deny", priority = -1 }
+style = { level = "deny", priority = -1 }

Groups are problematic because they can (and will) cause breakage when new failures are added. I think we should have a non-fatal job to test with groups, but by default we can add a large number of lints and "allow" the groups.

Quoted 1 line

+cast_ptr_alignment = "allow"

Why?

Quoted 1 line

+missing_safety_doc = "allow"

Please add it to individual files at the top.

Quoted 83 lines
rust/qemu-api/README.md new file +17 -0
new file mode 100644
index 0000000000..f16a1a929d
@@ -0,0 +1,17 @@
+# QEMU bindings and API wrappers
+
+This library exports helper Rust types, Rust macros and C FFI bindings for internal QEMU APIs.
+
+The C bindings can be generated with `bindgen`, using this build target:
+
+```console
+$ ninja bindings-aarch64-softmmu.rs
+```
+
+## Generate Rust documentation
+
+To generate docs for this crate, including private items:
+
+```sh
+cargo doc --no-deps --document-private-items
+```
rust/qemu-api/build.rs new file +48 -0
new file mode 100644
index 0000000000..13164f8371
@@ -0,0 +1,48 @@
+// Copyright 2024 Manos Pitsidianakis <[email protected]>
+// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later
+
+use std::{env, path::Path};
+
+fn main() {
+    println!("cargo:rerun-if-env-changed=MESON_BUILD_ROOT");
+    println!("cargo:rerun-if-changed=src/bindings.rs.inc");
+
+    let out_dir = env::var_os("OUT_DIR").unwrap();
+
+    if let Some(build_dir) = std::env::var_os("MESON_BUILD_ROOT") {
+        let mut build_dir = Path::new(&build_dir).to_path_buf();
+        let mut out_dir = Path::new(&out_dir).to_path_buf();
+        assert!(
+            build_dir.exists(),
+            "MESON_BUILD_ROOT value does not exist on filesystem: {}",
+            build_dir.display()
+        );
+        assert!(
+            build_dir.is_dir(),
+            "MESON_BUILD_ROOT value is not actually a directory: {}",
+            build_dir.display()
+        );
+        // TODO: add logic for other guest target architectures.
+        build_dir.push("bindings-aarch64-softmmu.rs");
+        let bindings_rs = build_dir;
+        assert!(
+            bindings_rs.exists(),
+            "MESON_BUILD_ROOT/bindings-aarch64-softmmu.rs does not exist on filesystem: {}",
+            bindings_rs.display()
+        );
+        assert!(
+            bindings_rs.is_file(),
+            "MESON_BUILD_ROOT/bindings-aarch64-softmmu.rs is not a file: {}",
+            bindings_rs.display()
+        );
+        out_dir.push("bindings.rs");
+        std::fs::copy(bindings_rs, out_dir).unwrap();
+        println!("cargo:rustc-cfg=MESON_BINDINGS_RS");
+    } else if !Path::new("src/bindings.rs.inc").exists() {
+        panic!(
+            "No generated C bindings found! Either build them manually with bindgen or with meson \
+             (`ninja bindings-aarch64-softmmu.rs`) and copy them to src/bindings.rs.inc, or build \
+             through meson."
+        );
+    }
+}
rust/qemu-api/deny.toml new file +0 -0
new file mode 100644
index 0000000000..3992380509

rust/qemu-api/deny.toml line 10

Quoted 10 lines
rust/qemu-api/deny.toml new file +10 -0
@@ -0,0 +1,57 @@
+# cargo-deny configuration file
+
+[graph]
+targets = [
+    "aarch64-unknown-linux-gnu",
+    "x86_64-unknown-linux-gnu",
+    "x86_64-apple-darwin",
+    "aarch64-apple-darwin",
+    "x86_64-pc-windows-gnu",
+    "aarch64-pc-windows-gnullvm",

Do we actually support LLVM Windows targets?

rust/qemu-api/deny.toml line 20

Quoted 10 lines
rust/qemu-api/deny.toml new file +10 -0
+unsafe impl Sync for TypeInfo {}
+unsafe impl Sync for VMStateDescription {}
+
+#[macro_export]
+macro_rules! module_init {
+    ($func:expr, $type:expr) => {
+        #[used]
+        #[cfg_attr(target_os = "linux", link_section = ".ctors")]
+        #[cfg_attr(target_os = "macos", link_section = "__DATA,__mod_init_func")]
+        #[cfg_attr(target_os = "windows", link_section = ".CRT$XCU")]

I'd rather use the ctor crate. Also, this should be src/util/module.rs so that it's easy to find the matching C source.

Quoted 53 lines
rust/qemu-api/deny.toml new file +53 -0
+        pub static LOAD_MODULE: extern "C" fn() = {
+            assert!($type < $crate::bindings::module_init_type_MODULE_INIT_MAX);
+
+            extern "C" fn __load() {
+                // ::std::panic::set_hook(::std::boxed::Box::new(|_| {}));
+
+                unsafe {
+                    $crate::bindings::register_module_init(Some($func), $type);
+                }
+            }
+
+            __load
+        };
+    };
+    (qom: $func:ident => $body:block) => {
+        // NOTE: To have custom identifiers for the ctor func we need to either supply
+        // them directly as a macro argument or create them with a proc macro.
+        #[used]
+        #[cfg_attr(target_os = "linux", link_section = ".ctors")]
+        #[cfg_attr(target_os = "macos", link_section = "__DATA,__mod_init_func")]
+        #[cfg_attr(target_os = "windows", link_section = ".CRT$XCU")]
+        pub static LOAD_MODULE: extern "C" fn() = {
+            extern "C" fn __load() {
+                // ::std::panic::set_hook(::std::boxed::Box::new(|_| {}));
+                #[no_mangle]
+                unsafe extern "C" fn $func() {
+                    $body
+                }
+
+                unsafe {
+                    $crate::bindings::register_module_init(
+                        Some($func),
+                        $crate::bindings::module_init_type_MODULE_INIT_QOM,
+                    );
+                }
+            }
+
+            __load
+        };
+    };
+}
+
+#[macro_export]
+macro_rules! type_info {
+    ($(#[$outer:meta])*
+     $name:ident: $t:ty,
+     $(name: $tname:expr,)*
+     $(parent: $pname:expr,)*
+     $(instance_init: $ii_fn:expr,)*
+     $(instance_post_init: $ipi_fn:expr,)*
+     $(instance_finalize: $if_fn:expr,)*
+     $(abstract_: $a_val:expr,)*
+     $(class_init: $ci_fn:expr,)*

rust/qemu-api/deny.toml

Quoted 10 lines
rust/qemu-api/deny.toml new file +10 -0
+     $(class_base_init: $cbi_fn:expr,)*
+    ) => {
+        #[used]
+        $(#[$outer])*
+        pub static $name: $crate::bindings::TypeInfo = $crate::bindings::TypeInfo {
+                $(name: {
+                #[used]
+                static TYPE_NAME: &::core::ffi::CStr = $tname;
+                $tname.as_ptr()
+            },)*

Please use the cstr crate. Even better, add a trait

pub unsafe trait ObjectType: Sized {

    const TYPE_NAME: &'static CStr;
}

and then just <$t as ObjectType>::TYPE_NAME

Likewise, the parent could be mandatory like

+ $name:ident: $t:ty($p:ty)

and use <$p as ObjectType>::TYPE_NAME.

Quoted 1 line

+ ..unsafe { MaybeUninit::<$crate::bindings::TypeInfo>::zeroed().assume_init() }

I suggest something like the Zeroed trait I had in my experiment:

==========
#![allow(clippy::undocumented_unsafe_blocks)]

use std::mem::MaybeUninit;

/// Trait providing an easy way to obtain an all-zero
/// value for a struct
///
/// # Safety
///
/// Only add this to a type if MaybeUninit::zeroed().assume_init() /// is valid for that type.

pub unsafe trait Zeroed: Sized {

    fn zeroed() -> Self {
        // SAFETY: If this weren't safe, just do not add the
        // trait to a type.
        unsafe { MaybeUninit::zeroed().assume_init() }
    }
}

// Put here all the impls that you need for the bindgen-provided types. unsafe impl Zeroed for crate::bindings::TypeInfo {}
=========

So that you can simply write Zeroed::zeroed(); there are other occurrences of this for Property, VMStateDescription, MemoryRegionOps.

Quoted 1 line

+use std::sync::OnceLock;

OnceLock is not needed, initialization is single-threaded and executed once only. You can just make declare_properties return a fn() and

$crate::bindings::device_class_set_props(dc.as_mut(), $props());

rust/qemu-api/deny.toml

Quoted 2 lines
rust/qemu-api/deny.toml new file +2 -0
+            dc.as_mut().realize = $realize_fn;
+            dc.as_mut().reset = $reset_fn;

Nice. Might also add the same trick as type_info!, for example $(realize: $realize_fn:expr,)*, to remove the "Some()".

rust/qemu-api/deny.toml

Quoted 9 lines
rust/qemu-api/deny.toml new file +9 -0
+#[macro_export]
+macro_rules! define_property {
+    ($name:expr, $state:ty, $field:expr, $prop:expr, $type:expr, default = $defval:expr$(,)*) => {
+        $crate::bindings::Property {
+            name: {
+                #[used]
+                static _TEMP: &::core::ffi::CStr = $name;
+                _TEMP.as_ptr()
+            },

Also please use cstr!() here.

rust/qemu-api/deny.toml

Quoted 10 lines
rust/qemu-api/deny.toml new file +10 -0
+            bitnr: 0,
+            bitmask: 0,
+            set_default: true,
+            defval: $crate::bindings::Property__bindgen_ty_1 { u: $defval.into() },
+            arrayoffset: 0,
+            arrayinfo: ::core::ptr::null(),
+            arrayfieldsize: 0,
+            link_type: ::core::ptr::null(),
+        }
+    };

I think you can just use "..Zeroed::zeroed()" instead of listing all the fields one by one.

Quoted 45 lines
rust/qemu-api/deny.toml new file +33 -0
+#[repr(C)]
+pub struct Properties<const N: usize>(pub OnceLock<[Property; N]>, pub fn() -> [Property; N]);
+
+impl<const N: usize> Properties<N> {
+    pub unsafe fn as_mut_ptr(&mut self) -> *mut Property {
+        _ = self.0.get_or_init(self.1);
+        self.0.get_mut().unwrap().as_mut_ptr()
+    }
+ > +        #[no_mangle]
+        pub static mut $ident: $crate::device_class::Properties<PROP_LEN> = $crate::device_class::Properties(::std::sync::OnceLock::new(), _make_properties);
+    };
+}
+
+#[macro_export]
+macro_rules! vm_state_description {
+    ($(#[$outer:meta])*
+     $name:ident,
+     $(name: $vname:expr,)*
+     $(unmigratable: $um_val:expr,)*
+    ) => {
+        #[used]
+        $(#[$outer])*
+        pub static $name: $crate::bindings::VMStateDescription = $crate::bindings::VMStateDescription {
+            $(name: {
+                #[used]
+                static VMSTATE_NAME: &::core::ffi::CStr = $vname;
+                $vname.as_ptr()
+            },)*
+            unmigratable: true,
+            ..unsafe { ::core::mem::MaybeUninit::<$crate::bindings::VMStateDescription>::zeroed().assume_init() }
+        };
+    }
+}
rust/qemu-api/src/lib.rs new file +6 -0
new file mode 100644
index 0000000000..74825c84e7
@@ -0,0 +1,29 @@
+// Copyright 2024 Manos Pitsidianakis <[email protected]>
+// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later
+
+#![doc = include_str!("../README.md")]
+
+// FIXME: remove improper_ctypes

rust/qemu-api/src/lib.rs line 16

Quoted 10 lines
rust/qemu-api/src/lib.rs new file +10 -0
+#[allow(
+    improper_ctypes_definitions,
+    improper_ctypes,
+    non_camel_case_types,
+    non_snake_case,
+    non_upper_case_globals
+)]
+#[allow(
+    clippy::missing_const_for_fn,
+    clippy::useless_transmute,

This one definitely shouldn't be there.

Paolo

Quoted 67 lines
rust/qemu-api/src/lib.rs new file +13 -0
+    clippy::too_many_arguments,
+    clippy::approx_constant,
+    clippy::use_self,
+    clippy::cast_lossless,
+)]
+#[rustfmt::skip]
+pub mod bindings;
+
+pub mod definitions;
+pub mod device_class;
+
+#[cfg(test)]
+mod tests;
rust/qemu-api/src/tests.rs new file +48 -0
new file mode 100644
index 0000000000..88c26308ee
@@ -0,0 +1,48 @@
+// Copyright 2024 Manos Pitsidianakis <[email protected]>
+// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later
+
+use crate::{
+    bindings::*, declare_properties, define_property, device_class_init, vm_state_description,
+};
+
+#[test]
+fn test_device_decl_macros() {
+    // Test that macros can compile.
+    vm_state_description! {
+        VMSTATE,
+        name: c"name",
+        unmigratable: true,
+    }
+
+    #[repr(C)]
+    pub struct DummyState {
+        pub char_backend: CharBackend,
+        pub migrate_clock: bool,
+    }
+
+    declare_properties! {
+        DUMMY_PROPERTIES,
+            define_property!(
+                c"chardev",
+                DummyState,
+                char_backend,
+                unsafe { &qdev_prop_chr },
+                CharBackend
+            ),
+            define_property!(
+                c"migrate-clk",
+                DummyState,
+                migrate_clock,
+                unsafe { &qdev_prop_bool },
+                bool
+            ),
+    }
+
+    device_class_init! {
+        dummy_class_init,
+        props => DUMMY_PROPERTIES,
+        realize_fn => None,
+        reset_fn => None,
+        vmsd => VMSTATE,
+    }
+}
#19 lore raw

RFC PATCH v4 4/7 rust: add PL011 device model

In reply to Manos Pitsidianakis

This commit adds a re-implementation of hw/char/pl011.c in Rust.

How to build:

1. Make sure rust, cargo and bindgen (cargo install bindgen-cli) are installed
2. Configure a QEMU build with:
--enable-system --target-list=aarch64-softmmu --enable-with-rust
3. Launching a VM with qemu-system-aarch64 should use the Rust version of the pl011 device (unless it is not set up so in hw/arm/virt.c; the type of the UART device is hardcoded).

To confirm, inspect info qom-tree in the monitor and look for an x-pl011-rust device.

---
 MAINTAINERS                    |   5 +
 meson.build                    |   4 +
 rust/meson.build               |   2 +
 rust/pl011/.gitignore          |   2 +
 rust/pl011/Cargo.lock          | 125 ++++++++
 rust/pl011/Cargo.toml          |  67 ++++
 rust/pl011/README.md           |  31 ++
 rust/pl011/deny.toml           |  57 ++++
 rust/pl011/meson.build         |   7 +
 rust/pl011/rustfmt.toml        |   1 +
 rust/pl011/src/definitions.rs  |  39 +++
 rust/pl011/src/device.rs       | 509 ++++++++++++++++++++++++++++++
 rust/pl011/src/device_class.rs |  48 +++
 rust/pl011/src/lib.rs          | 556 +++++++++++++++++++++++++++++++++
 rust/pl011/src/memory_ops.rs   |  45 +++
 15 files changed, 1498 insertions(+)
 create mode 100644 rust/pl011/.gitignore
 create mode 100644 rust/pl011/Cargo.lock
 create mode 100644 rust/pl011/Cargo.toml
 create mode 100644 rust/pl011/README.md
 create mode 100644 rust/pl011/deny.toml
 create mode 100644 rust/pl011/meson.build
 create mode 120000 rust/pl011/rustfmt.toml
 create mode 100644 rust/pl011/src/definitions.rs
 create mode 100644 rust/pl011/src/device.rs
 create mode 100644 rust/pl011/src/device_class.rs
 create mode 100644 rust/pl011/src/lib.rs
 create mode 100644 rust/pl011/src/memory_ops.rs
MAINTAINERS +5 -0
index 8598d38eae..fedee0ddef 100644
@@ -1186,6 +1186,11 @@ F: include/hw/*/microbit*.h
 F: tests/qtest/microbit-test.c
 F: docs/system/arm/nrf.rst
 
+ARM PL011 Rust device
+M: Manos Pitsidianakis <[email protected]>
+S: Maintained
+F: rust/pl011/
+
 AVR Machines
 -------------
 
meson.build +4 -0
index 71011fd3b3..945ce6aaaf 100644
@@ -296,6 +296,10 @@ if get_option('with_rust').allowed()
 endif
 with_rust = cargo.found()
 
+if with_rust
+  subdir('rust')
+endif
+
 # default flags for all hosts
 # We use -fwrapv to tell the compiler that we require a C dialect where
 # left shift of signed integers is well defined and has the expected
rust/meson.build +2 -0
index 5fdc2621a3..21115ac56d 100644
@@ -93,6 +93,8 @@ subdir('qemu-api')
 # bindgen dependency is declared.
 rust_hw_target_list = {}
 
+subdir('pl011')
+
 foreach rust_hw_target, rust_hws: rust_hw_target_list
   foreach rust_hw_dev: rust_hws
     crate_metadata = {
rust/pl011/.gitignore new file +2 -0
new file mode 100644
index 0000000000..71eaff2035
@@ -0,0 +1,2 @@
+# Ignore generated bindings file overrides.
+src/bindings.rs.inc
rust/pl011/Cargo.lock new file +125 -0

125 lines not on this page · raw message

rust/pl011/Cargo.toml new file +67 -0

67 lines not on this page · raw message

rust/pl011/README.md new file +31 -0
new file mode 100644
index 0000000000..cd7dea3163
@@ -0,0 +1,31 @@
+# PL011 QEMU Device Model
+
+This library implements a device model for the PrimeCell® UART (PL011)
+device in QEMU.
+
+## Build static lib
+
+Host build target must be explicitly specified:
+
+```sh
+cargo build --target x86_64-unknown-linux-gnu
+```
+
+Replace host target triplet if necessary.
+
+## Generate Rust documentation
+
+To generate docs for this crate, including private items:
+
+```sh
+cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu
+```
+
+To include direct dependencies like `bilge` (bitmaps for register types):
+
+```sh
+cargo tree --depth 1 -e normal --prefix none \
+ | cut -d' ' -f1 \
+ | xargs printf -- '-p %s\n' \
+ | xargs cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu
+```
rust/pl011/deny.toml new file +57 -0

57 lines not on this page · raw message

rust/pl011/meson.build new file +7 -0
new file mode 100644
index 0000000000..cbac0fd94d
@@ -0,0 +1,7 @@
+rust_pl011 = {
+  'name': 'pl011',
+  'dirname': 'pl011',
+  'output': 'libpl011.a',
+  }
+
+rust_hw_target_list += {'aarch64-softmmu': [rust_pl011]}
rust/pl011/rustfmt.toml new file +1 -0
new file mode 120000
index 0000000000..39f97b043b
@@ -0,0 +1 @@
+../rustfmt.toml
\ No newline at end of file
rust/pl011/src/definitions.rs new file +39 -0
new file mode 100644
index 0000000000..5efe8ae7f0
@@ -0,0 +1,39 @@
+// Copyright 2024 Manos Pitsidianakis <[email protected]>
+// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later
+
+//! Definitions required by QEMU when registering the device.
+
+use core::{mem::MaybeUninit, ptr::NonNull};
+
+use qemu_api::bindings::*;
+
+use crate::{device::PL011State, device_class::pl011_class_init};
+
+qemu_api::type_info! {
+    PL011_ARM_INFO: PL011State,
+    name: c"x-pl011-rust",
+    parent: TYPE_SYS_BUS_DEVICE,
+    instance_init: Some(pl011_init),
+    abstract_: false,
+    class_init: Some(pl011_class_init),
+}
+
+#[used]
+pub static VMSTATE_PL011: VMStateDescription = VMStateDescription {
+    name: PL011_ARM_INFO.name,
+    unmigratable: true,
+    ..unsafe { MaybeUninit::<VMStateDescription>::zeroed().assume_init() }
+};
+
+#[no_mangle]
+pub unsafe extern "C" fn pl011_init(obj: *mut Object) {
+    assert!(!obj.is_null());
+    let mut state = NonNull::new_unchecked(obj.cast::<PL011State>());
+    state.as_mut().init();
+}
+
+qemu_api::module_init! {
+    qom: register_type => {
+        type_register_static(&PL011_ARM_INFO);
+    }
+}
rust/pl011/src/device.rs new file +509 -0

509 lines not on this page · raw message

rust/pl011/src/device_class.rs new file +48 -0

48 lines not on this page · raw message

rust/pl011/src/lib.rs new file +556 -0

556 lines not on this page · raw message

rust/pl011/src/memory_ops.rs new file +45 -0

45 lines not on this page · raw message

Signature · 1 line
γαῖα πυρί μιχθήτω
#20 lore raw

Re: [RFC PATCH v4 4/7] rust: add PL011 device model

MAINTAINERS F: include/hw/*/microbit*.h · line 1192

Quoted 4 lines · Manos Pitsidianakis
MAINTAINERS +4 -0
+ARM PL011 Rust device
+M: Manos Pitsidianakis <[email protected]>
+S: Maintained
+F: rust/pl011/

No need for this, since it's covered by rust/. If (when) it replaces the main one, the PL011-specific stanza will be assigned to ARM maintainers (while you keep covering it via rust/).

meson.build if get_option('with_rust').allowed() · line 301

Quoted 3 lines
meson.build +3 -0
+if with_rust
+  subdir('rust')
+endif

Should be in patch 3.

Quoted 1 line

+subdir('pl011')

As I said before it should be handled via Kconfig, but let's do that after the initial merge. However...

rust/pl011/Cargo.toml line 42

Quoted 11 lines
rust/pl011/Cargo.toml new file +11 -0
+correctness = { level = "deny", priority = -1 }
+suspicious = { level = "deny", priority = -1 }
+complexity = { level = "deny", priority = -1 }
+perf = { level = "deny", priority = -1 }
+cargo = { level = "deny", priority = -1 }
+nursery = { level = "deny", priority = -1 }
+style = { level = "deny", priority = -1 }
+# restriction group
+dbg_macro = "deny"
+rc_buffer = "deny"
+as_underscore = "deny"

... repeated lints really suggest that you should use a workspace and a single cargo invocation to build both the rust-qapi and pl011 crates, which I think is doable if you can run bindgen just once.

Quoted 1 line

+use core::{mem::MaybeUninit, ptr::NonNull};

Let's remove at least this unsafety.

rust/pl011/src/definitions.rs line 32

Quoted 12 lines
rust/pl011/src/definitions.rs new file +12 -0
+#[used]
+pub static VMSTATE_PL011: VMStateDescription = VMStateDescription {
+    name: PL011_ARM_INFO.name,
+    unmigratable: true,
+    ..unsafe { MaybeUninit::<VMStateDescription>::zeroed().assume_init() }
+};
+
+#[no_mangle]
+pub unsafe extern "C" fn pl011_init(obj: *mut Object) {
+    assert!(!obj.is_null());
+    let mut state = NonNull::new_unchecked(obj.cast::<PL011State>());
+    state.as_mut().init();

This is fine for now, but please add a

// TODO: this assumes that "all zeroes" is a valid state for all fields of // PL011State. This is not necessarily true of any #[repr(Rust)] structs, // including bilge-generated types. It should instead use MaybeUninit.

rust/pl011/src/definitions.rs line 38

Quoted 6 lines
rust/pl011/src/definitions.rs new file +6 -0
+}
+
+qemu_api::module_init! {
+    qom: register_type => {
+        type_register_static(&PL011_ARM_INFO);
+    }

Can you make the macro look like

   MODULE_INIT_QOM: fn register_type() {
     ...
   }

so that it's clear what "register_type" is, and so that it's easier to extend it to more values?

rust/pl011/src/device.rs line 67

Quoted 2 lines
rust/pl011/src/device.rs new file +2 -0
+    #[doc(alias = "clk")]
+    pub clock: NonNull<Clock>,

It's null when init() runs, so please use *mut Clock.

rust/pl011/src/device.rs line 69

Quoted 2 lines
rust/pl011/src/device.rs new file +2 -0
+    #[doc(alias = "migrate_clk")]
+    pub migrate_clock: bool,

Please put all properties together in the struct for readability.

Quoted 54 lines
rust/pl011/src/device.rs new file +54 -0
+}
+
+#[used]
+pub static CLK_NAME: &CStr = c"clk";
+
+impl PL011State {
+    pub fn init(&mut self) {
+        unsafe {
+            memory_region_init_io(
+                addr_of_mut!(self.iomem),
+                addr_of_mut!(*self).cast::<Object>(),
+                &PL011_OPS,
+                addr_of_mut!(*self).cast::<c_void>(),
+                PL011_ARM_INFO.name,
+                0x1000,
+            );
+            let sbd = addr_of_mut!(*self).cast::<SysBusDevice>();
+            let dev = addr_of_mut!(*self).cast::<DeviceState>();
+            sysbus_init_mmio(sbd, addr_of_mut!(self.iomem));
+            for irq in self.interrupts.iter_mut() {
+                sysbus_init_irq(sbd, irq);
+            }
+            self.clock = NonNull::new(qdev_init_clock_in(
+                dev,
+                CLK_NAME.as_ptr(),
+                None, /* pl011_clock_update */
+                addr_of_mut!(*self).cast::<c_void>(),
+                ClockEvent_ClockUpdate,
+            ))
+            .unwrap();
+        }
+    }
+
+    pub fn read(&mut self, offset: hwaddr, _size: core::ffi::c_uint) -> u64 {
+        use RegisterOffset::*;
+
+        match RegisterOffset::try_from(offset) {
+            Err(v) if (0x3f8..0x400).contains(&v) => {
+                u64::from(PL011_ID_ARM[((offset - 0xfe0) >> 2) as usize])
+            }
+            Err(_) => {
+                // qemu_log_mask(LOG_GUEST_ERROR, "pl011_read: Bad offset 0x%x\n", (int)offset);
+                0
+            }
+            Ok(DR) => {
+                // s->flags &= ~PL011_FLAG_RXFF;
+                self.flags.set_receive_fifo_full(false);
+                let c = self.read_fifo[self.read_pos];
+                if self.read_count > 0 {
+                    self.read_count -= 1;
+                    self.read_pos = (self.read_pos + 1) & (self.fifo_depth() - 1);
+                }
+                if self.read_count == 0 {
+                    // self.flags |= PL011_FLAG_RXFE;

rust/pl011/src/device.rs line 133

Quoted 10 lines
rust/pl011/src/device.rs new file +10 -0
+                    self.flags.set_receive_fifo_empty(true);
+                }
+                if self.read_count + 1 == self.read_trigger {
+                    //self.int_level &= ~ INT_RX;
+                    self.int_level &= !registers::INT_RX;
+                }
+                // Update error bits.
+                self.receive_status_error_clear = c.to_be_bytes()[3].into();
+                self.update();
+                unsafe { qemu_chr_fe_accept_input(&mut self.char_backend) };

Please add a comment here like

// TODO: this causes a callback that creates another "&mut self". // This is forbidden by Rust aliasing rules and has to be fixed // using interior mutability.

Paolo

#21 lore raw

RFC PATCH v4 5/7 .gitattributes: add Rust diff and merge attributes

In reply to Manos Pitsidianakis

Set rust source code to diff=rust (built-in with new git versions) and merge=binary for Cargo.lock files (they should not be merged but auto-generated by cargo)

---
 .gitattributes | 3 +++
 1 file changed, 3 insertions(+)
.gitattributes +3 -0
index a217cb7bfe..6dc6383d3d 100644
@@ -2,3 +2,6 @@
 *.h.inc         diff=c
 *.m             diff=objc
 *.py            diff=python
+*.rs            diff=rust
+*.rs.inc        diff=rust
+Cargo.lock      diff=toml merge=binary
Signature · 1 line
γαῖα πυρί μιχθήτω
Reviewed-by on v4 5/7
#22 lore raw

Re: [RFC PATCH v4 5/7] .gitattributes: add Rust diff and merge attributes

Quoted 5 lines · Manos Pitsidianakis

Set rust source code to diff=rust (built-in with new git versions) and merge=binary for Cargo.lock files (they should not be merged but auto-generated by cargo)

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#23 lore raw

RFC PATCH v4 6/7 DO NOT MERGE: add rustdoc build for gitlab pages

In reply to Manos Pitsidianakis

Deploy the generated rustdocs for my personal rust qemu fork on gitlab.

The URL is:

https://rust-for-qemu-epilys-aebb06ca9f9adfe6584811c14ae44156501d935ba4.gitlab.io/pl011/index.html

---
 .gitlab-ci.d/buildtest.yml | 64 +++++++++++++++++++++++++++-----------
 1 file changed, 45 insertions(+), 19 deletions(-)
.gitlab-ci.d/buildtest.yml +45 -19
index 0eec570310..a2fedc87bd 100644
@@ -716,31 +716,57 @@ build-tools-and-docs-debian:
 # For contributor forks we want to publish from any repo so
 # that users can see the results of their commits, regardless
 # of what topic branch they're currently using
+# pages:
+#   extends: .base_job_template
+#   image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG
+#   stage: test
+#   needs:
+#     - job: build-tools-and-docs-debian
+#   script:
+#     - mkdir -p public
+#     # HTML-ised source tree
+#     - make gtags
+#     # We unset variables to work around a bug in some htags versions
+#     # which causes it to fail when the environment is large
+#     - CI_COMMIT_MESSAGE= CI_COMMIT_TAG_MESSAGE= htags
+#         -anT --tree-view=filetree -m qemu_init
+#         -t "Welcome to the QEMU sourcecode"
+#     - mv HTML public/src
+#     # Project documentation
+#     - make -C build install DESTDIR=$(pwd)/temp-install
+#     - mv temp-install/usr/local/share/doc/qemu/* public/
+#   artifacts:
+#     when: on_success
+#     paths:
+#       - public
+#   variables:
+#     QEMU_JOB_PUBLISH: 1
+# The Docker image that will be used to build your app
 pages:
-  extends: .base_job_template
-  image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG
-  stage: test
-  needs:
-    - job: build-tools-and-docs-debian
+  image: rust:latest
   script:
-    - mkdir -p public
-    # HTML-ised source tree
-    - make gtags
-    # We unset variables to work around a bug in some htags versions
-    # which causes it to fail when the environment is large
-    - CI_COMMIT_MESSAGE= CI_COMMIT_TAG_MESSAGE= htags
-        -anT --tree-view=filetree -m qemu_init
-        -t "Welcome to the QEMU sourcecode"
-    - mv HTML public/src
-    # Project documentation
-    - make -C build install DESTDIR=$(pwd)/temp-install
-    - mv temp-install/usr/local/share/doc/qemu/* public/
+    - rustup component add rustfmt
+    - DEBIAN_FRONTEND=noninteractive apt-get update -y
+    - DEBIAN_FRONTEND=noninteractive apt-get install -y python3-venv meson libgcrypt20-dev zlib1g-dev autoconf automake libtool bison flex git libglib2.0-dev libfdt-dev libpixman-1-dev ninja-build make libclang-14-dev
+    - cargo install bindgen-cli
+    - mkdir ./build/
+    - cd ./build/
+    - ../configure --enable-system --disable-kvm --target-list=aarch64-softmmu --enable-with-rust
+    - ninja "bindings-aarch64-softmmu.rs"
+    - cp ./bindings-aarch64-softmmu.rs ../rust/qemu-api/src/bindings.rs.inc
+    - cd ../rust/pl011/
+    - cargo tree --depth 1 -e normal --prefix none | cut -d' ' -f1  | xargs
+      printf -- '-p %s\n'  | xargs cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu
+    - cd ./../..
+    - mv ./rust/pl011/target/x86_64-unknown-linux-gnu/doc ./public
   artifacts:
     when: on_success
     paths:
       - public
-  variables:
-    QEMU_JOB_PUBLISH: 1
+  rules:
+    # This ensures that only pushes to the default branch will trigger
+    # a pages deploy
+    - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
 
 coverity:
   image: $CI_REGISTRY_IMAGE/qemu/fedora:$QEMU_CI_CONTAINER_TAG
Signature · 1 line
γαῖα πυρί μιχθήτω
#24 lore raw

RFC PATCH v4 7/7 DO NOT MERGE: replace TYPE_PL011 with x-pl011-rust in arm virt machine

In reply to Manos Pitsidianakis

Convenience patch for testing the rust device.

---
 hw/arm/virt.c | 4 ++++
 1 file changed, 4 insertions(+)
hw/arm/virt.c +4 -0
index b0c68d66a3..49dd0b815e 100644
@@ -929,7 +929,11 @@ static void create_uart(const VirtMachineState *vms, int uart,
     int irq = vms->irqmap[uart];
     const char compat[] = "arm,pl011\0arm,primecell";
     const char clocknames[] = "uartclk\0apb_pclk";
+#ifdef CONFIG_WITH_RUST
+    DeviceState *dev = qdev_new("x-pl011-rust");
+#else
     DeviceState *dev = qdev_new(TYPE_PL011);
+#endif
     SysBusDevice *s = SYS_BUS_DEVICE(dev);
     MachineState *ms = MACHINE(vms);
 
Signature · 1 line
γαῖα πυρί μιχθήτω
#25 lore raw
In reply to Manos Pitsidianakis
Quoted 11 lines · Manos Pitsidianakis

Changes from v3->v4:
- Add rust-specific files to .gitattributes
- Added help text to scripts/cargo_wrapper.py arguments (thanks Stephan)
- Split bindings separate crate
- Add declarative macros for symbols exported to QEMU to said crate
- Lowered MSRV to 1.77.2
- Removed auto-download and install of bindgen-cli
- Fixed re-compilation of Rust objects in case they are missing from filesystem
- Fixed optimized builds by adding #[used] (thanks Pierrick for the help debugging this)

I think the largest issue is that I'd rather have a single cargo build using a virtual manifest, because my hunch is that it'd be the easiest path towards Kconfig integration. But it's better to do this after merge, as the changes are pretty large. It's also independent from any other changes targeted at removing unsafe code, so no need to hold back on merging.

Other comments I made that should however be addressed before merging, from most to least important:

- TODO comments when the code is doing potential undefined behavior

- module structure should IMO resemble the C part of the tree

- only generate bindings.rs.inc once

- a couple abstractions that I'd like to have now: a trait to store the CStr corresponding to the structs, and one to generate all-zero structs without having to type "unsafe { MaybeUninit::zeroed().assume_init() }"

- I pointed out a couple lints that are too broad and should be enabled per-file, even if right now it's basically all files that include them.

- add support for --cargo and CARGO environment variables (if my patch works without too much hassle)

- I'd like to use ctor instead of non-portable linker magic, and the cstr crate instead of CStr statics or c""

- please check if -Wl,--whole-archive can be replaced with link_whole:

- probably, until Rust is enabled by default we should treat dependencies as a moving target and not commit Cargo.lock files. In the meanwhile we can discuss how to handle them.

And a few aesthetic changes on top of this.

With respect to lints, marking entire groups as "deny" is problematic. Before merge, I'd rather have the groups as just "warn", and add a long list of denied lints[1]. After merge we can also add a non-fatal CI job that runs clippy with nightly rust and with groups marked as "deny". This matches the check-python-tox job in python/.

[1] https://github.com/bonzini/rust-qemu/commit/95b25f7c5f4e

Thanks,

Paolo

#26 lore raw
Quoted 41 lines · Paolo Bonzini
Quoted 11 lines · Manos Pitsidianakis

Changes from v3->v4:
- Add rust-specific files to .gitattributes
- Added help text to scripts/cargo_wrapper.py arguments (thanks Stephan)
- Split bindings separate crate
- Add declarative macros for symbols exported to QEMU to said crate
- Lowered MSRV to 1.77.2
- Removed auto-download and install of bindgen-cli
- Fixed re-compilation of Rust objects in case they are missing from filesystem
- Fixed optimized builds by adding #[used] (thanks Pierrick for the help debugging this)

I think the largest issue is that I'd rather have a single cargo build using a virtual manifest, because my hunch is that it'd be the easiest path towards Kconfig integration. But it's better to do this after merge, as the changes are pretty large. It's also independent from any other changes targeted at removing unsafe code, so no need to hold back on merging.

Other comments I made that should however be addressed before merging, from most to least important:

- TODO comments when the code is doing potential undefined behavior

- module structure should IMO resemble the C part of the tree

- only generate bindings.rs.inc once

- a couple abstractions that I'd like to have now: a trait to store the CStr corresponding to the structs, and one to generate all-zero structs without having to type "unsafe { MaybeUninit::zeroed().assume_init() }"

- I pointed out a couple lints that are too broad and should be enabled per-file, even if right now it's basically all files that include them.

- add support for --cargo and CARGO environment variables (if my patch works without too much hassle)

- I'd like to use ctor instead of non-portable linker magic, and the cstr crate instead of CStr statics or c""

Quoted 7 lines

- please check if -Wl,--whole-archive can be replaced with link_whole:

- probably, until Rust is enabled by default we should treat dependencies as a moving target and not commit Cargo.lock files. In the meanwhile we can discuss how to handle them.

And a few aesthetic changes on top of this.

This series is still missing changes to enable build on all targets during CI, including cross-compiles, to prove that we're doing the correct thing on all our targetted platforms. That's a must have before considering it suitable for merge.

I also believe we should default to enabling rust toolchain by default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.

This is to give users a clear message that Rust is likely to become a fundamental part of QEMU, so they need to give feedback if they hit any problems / have use cases we've not anticipated that are problematic wrt Rust.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#27 lore raw

Il lun 8 lug 2024, 18:33 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 5 lines

This series is still missing changes to enable build on all targets during CI, including cross-compiles, to prove that we're doing the correct thing on all our targetted platforms. That's a must have before considering it suitable for merge.

But we're not—in particular it's still using several features not in all supported distros.

I also believe we should default to enabling rust toolchain by

Quoted 4 lines

default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.

I guess the detection could be done, but actually enabling the build part needs to wait until the minimum supported version is low enough.

Paolo

Quoted 16 lines

This is to give users a clear message that Rust is likely to become a fundamental part of QEMU, so they need to give feedback if they hit any problems / have use cases we've not anticipated that are problematic wrt Rust.

With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|

#28 lore raw
Quoted 11 lines · Paolo Bonzini

Il lun 8 lug 2024, 18:33 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 5 lines

This series is still missing changes to enable build on all targets during CI, including cross-compiles, to prove that we're doing the correct thing on all our targetted platforms. That's a must have before considering it suitable for merge.

But we're not—in particular it's still using several features not in all supported distros.

That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.

Quoted 8 lines

I also believe we should default to enabling rust toolchain by

Quoted 4 lines

default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.

I guess the detection could be done, but actually enabling the build part needs to wait until the minimum supported version is low enough.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#29 lore raw

Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 6 lines

That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.

In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).

Paolo

Quoted 21 lines
Quoted 9 lines

I also believe we should default to enabling rust toolchain by

Quoted 4 lines

default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.

I guess the detection could be done, but actually enabling the build part needs to wait until the minimum supported version is low enough.

With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|

#30 lore raw
Quoted 12 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:

Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 6 lines

That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.

Quoted 5 lines

In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).

Paolo

If you do, I'd really appreciate it if you did not use any part of my patches.

Thanks,
Manos

#31 lore raw

Il lun 8 lug 2024, 20:39 Manos Pitsidianakis <[email protected]> ha scritto:

Quoted 16 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:
Quoted 13 lines

Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 6 lines

That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.

Quoted 7 lines
Quoted 3 lines

In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).

If you do, I'd really appreciate it if you did not use any part of my patches.

"We" means that you would accept patches, review them and apply them until any agreed-upon conditions for merging are satisfied, and then send either a final series or a pull request for inclusion in QEMU.

Paolo

#32 lore raw
Quoted 20 lines · Paolo Bonzini

Il lun 8 lug 2024, 20:39 Manos Pitsidianakis <[email protected]> ha scritto:

Quoted 16 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:
Quoted 12 lines

Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 6 lines

That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).

If you do, I'd really appreciate it if you did not use any part of my patches.

"We" means that you would accept patches, review them and apply them until any agreed-upon conditions for merging are satisfied, and then send either a final series or a pull request for inclusion in QEMU.

Ah, alright. That wasn't obvious because that e-mail was not directed to me nor did it mention my name :)

I do not want to do that, in any case. I do not think it's the right approach.

Signature · 2 lines
Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro Ltd
#33 lore raw
Quoted 2 lines · Manos Pitsidianakis

Ah, alright. That wasn't obvious because that e-mail was not directed to me nor did it mention my name :)

Oh, ok. Sorry about that. Generally when I say "we" I include as large a part of the community as applicable.

Quoted 1 line

I do not want to do that, in any case. I do not think it's the right approach.

No problem with that (and in fact I agree, as I'd prefer a speedy merge and doing the work on the QEMU master branch); however, we need to reach an agreement on that and everybody (including Daniel) needs to explain the reason for their position.

Daniel's proposed criteria for merging include:
- CI integration
- CI passing for all supported targets (thus lowering the MSRV to 1.63.0)
- plus any the code changes that were or will be requested during review

That seems to be a pretty high amount of work, and until it's done everyone else is unable to contribute, not even in directions orthogonal to the above (cross compilation support, less unsafe code, porting more devices). So something has to give: either we decide for an early merge, where the code is marked as experimental and disabled by default. Personally I think it's fine, the contingency plan is simply to "git rm -rf rust/". Or we can keep the above stringent requirements for merging, but then I don't see it as a one-person job.

If I can say so, developing on a branch would also be a useful warm-up for you in the maintainer role, if we expect that there will be significant community contributions to Rust.

Paolo

#34 lore raw
Quoted 20 lines · Paolo Bonzini
Quoted 2 lines · Manos Pitsidianakis

Ah, alright. That wasn't obvious because that e-mail was not directed to me nor did it mention my name :)

Oh, ok. Sorry about that. Generally when I say "we" I include as large a part of the community as applicable.

Quoted 1 line

I do not want to do that, in any case. I do not think it's the right approach.

No problem with that (and in fact I agree, as I'd prefer a speedy merge and doing the work on the QEMU master branch); however, we need to reach an agreement on that and everybody (including Daniel) needs to explain the reason for their position.

Daniel's proposed criteria for merging include:
- CI integration
- CI passing for all supported targets (thus lowering the MSRV to 1.63.0)
- plus any the code changes that were or will be requested during review

Quoted 4 lines

That seems to be a pretty high amount of work, and until it's done everyone else is unable to contribute, not even in directions orthogonal to the above (cross compilation support, less unsafe code, porting more devices).

My thought is that the initial merge focuses only on the build system integration. So that's basically patches 1 + 2 in this series.

IMHO that is small enough that we should be able to demonstrate that we detect Rust, run bindgen & compile its result, on all our supported platforms without an unreasonable amount of effort.

Quoted 5 lines

So something has to give: either we decide for
an early merge, where the code is marked as experimental and disabled by default. Personally I think it's fine, the contingency plan is simply to "git rm -rf rust/". Or we can keep the above stringent requirements for merging, but then I don't see it as a one-person job.

Patch 3, the high level APIs is where I see most of the work and collaboration being needed, but that doesn't need to be rushed into the first merge. We would have a "rust" subsystem + maintainer who would presumably have a staging tree, etc in the normal way we work and collaborate

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#35 lore raw
Quoted 8 lines · Daniel P. Berrangé

My thought is that the initial merge focuses only on the build system integration. So that's basically patches 1 + 2 in this series.

Patch 3, the high level APIs is where I see most of the work and collaboration being needed, but that doesn't need to be rushed into the first merge. We would have a "rust" subsystem + maintainer who would presumably have a staging tree, etc in the normal way we work and collaborate

It's complicated. A "perfect" build system integration would include integration with Kconfig, but it's not simple work and it may not be Manos's preference for what to work on (or maybe it is), and it's also not a blocker for further work on patches 3-4.

On the other hand, patches 3 and 4 are _almost_ ready except for requiring a very new Rust - we know how to tackle that, but again it may take some time and it's completely separate work from better build system integration.

In other words, improving build system integration is harder until merge, but merge is blocked by independent work on lowering the minimum supported Rust version. This is why I liked the idea of having either a development tree to allow a merge into early 9.2.

On the other hand, given the exceptional scope (completely new code that can be disabled at will) and circumstances, even a very early merge into 9.1 (disabled by default) might be better to provide developers with the easiest base for experimenting. The requirements for merging, here, would basically amount to a good roadmap and some established good habits.

An merge into early 9.2 would be a bit harder for experimenting, while merging it now would sacrifice CI integration in the initial stages of the work but make cooperation easier.

However, it's difficult to say what's best without knowing Manos's rationale for preferring not to have a development tree yet.

Paolo

#36 lore raw
Quoted 32 lines · Paolo Bonzini
Quoted 8 lines · Daniel P. Berrangé

My thought is that the initial merge focuses only on the build system integration. So that's basically patches 1 + 2 in this series.

Patch 3, the high level APIs is where I see most of the work and collaboration being needed, but that doesn't need to be rushed into the first merge. We would have a "rust" subsystem + maintainer who would presumably have a staging tree, etc in the normal way we work and collaborate

It's complicated. A "perfect" build system integration would include integration with Kconfig, but it's not simple work and it may not be Manos's preference for what to work on (or maybe it is), and it's also not a blocker for further work on patches 3-4.

On the other hand, patches 3 and 4 are _almost_ ready except for requiring a very new Rust - we know how to tackle that, but again it may take some time and it's completely separate work from better build system integration.

In other words, improving build system integration is harder until merge, but merge is blocked by independent work on lowering the minimum supported Rust version. This is why I liked the idea of having either a development tree to allow a merge into early 9.2.

On the other hand, given the exceptional scope (completely new code that can be disabled at will) and circumstances, even a very early merge into 9.1 (disabled by default) might be better to provide developers with the easiest base for experimenting. The requirements for merging, here, would basically amount to a good roadmap and some established good habits.

Quoted 3 lines

An merge into early 9.2 would be a bit harder for experimenting, while merging it now would sacrifice CI integration in the initial stages of the work but make cooperation easier.

I would be in favor of a 9.1 merge (disabled, not auto).
Noting that soft-freeze is two weeks from today, so no dilly dallying. :-)

The only reasonable alternative that I see is the kind of development branch that qemu is not used to having: a long-term shared branch on qemu-projects. With which I would be ok, up to and including a branch merge at the end, as I'm used to working that way with other projects.

I think the only reason to delay the current development work until 9.2 is if we were still questioning whether using Rust *at all* is a good idea. I think we're beyond that point.

r~

#37 lore raw
In reply to Manos Pitsidianakis
Quoted 16 lines · Manos Pitsidianakis
Quoted 12 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:

Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:

Quoted 6 lines

That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.

Quoted 8 lines
Quoted 3 lines

In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).

Paolo

If you do, I'd really appreciate it if you did not use any part of my patches.

Someone pointed out to me that my wording is really bad here and I agree, I apologize. I did not express myself with the right words. What I wanted to say is that hypothetically my work can be used as a base just not the patches/sign offs. The changes in the patches are all open source and part of the QEMU community now, of course.