Add Rust support, implement ARM PL011
Subsystems (2)
v4 cover Add Rust support, implement ARM PL011
Changes from v3->v4:
- Add rust-specific files to .gitattributes
- Added help text to scripts/cargo_wrapper.py arguments (thanks Stephan)
- Split bindings separate crate
- Add declarative macros for symbols exported to QEMU to said crate
- Lowered MSRV to 1.77.2
- Removed auto-download and install of bindgen-cli
- Fixed re-compilation of Rust objects in case they are missing from filesystem
- Fixed optimized builds by adding #[used] (thanks Pierrick for the help debugging this)
Also, Pierrick helped confirming it works on Windows with some windows-specific changes. I confirmed it works on macos by allowing bindgen to detect system paths for clang, which is a workaround and not a solution. However this series doesn't have the windows changes integrated.
Changes from v2->v3:
- Addressed minor mistakes (thanks Stefan)
- Setup supported version checks for cargo, rustc and bindgen (thanks everyone who pointed it out / suggested it)
- Fixed problem with bindgen failing if certain system headers where needed by defining an allowlist for headers instead of a blocklist for what we don't want (thanks Alex Bennée for reporting it)
- Cleaned up bindgen target/dependendy definition in meson.build by removing unnecessary bits
Changes from v1->v2:
- Create bindgen target first, then add commit for device (thanks Pierrick)
- Create a special named generated.rs for each target as compilation would fail if more than one targets were defined. The generated.rs target names would clash.
- Add more descriptive commit messages
- Update MAINTAINERS
- Cleanup patch order for better review, hopefully
Manos Pitsidianakis (7):
build-sys: Add rust feature option
rust: add bindgen step as a meson dependency
rust: add crate to expose bindings and interfaces
rust: add PL011 device model
.gitattributes: add Rust diff and merge attributes
DO NOT MERGE: add rustdoc build for gitlab pages
DO NOT MERGE: replace TYPE_PL011 with x-pl011-rust in arm virt machine
.gitattributes | 3 + .gitlab-ci.d/buildtest.yml | 64 +++- MAINTAINERS | 20 ++ configure | 11 + hw/arm/virt.c | 4 + meson.build | 72 ++++ meson_options.txt | 5 + rust/.cargo/config.toml | 2 + rust/.gitignore | 3 + rust/meson.build | 114 ++++++ rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 125 +++++++ rust/pl011/Cargo.toml | 67 ++++ rust/pl011/README.md | 31 ++ rust/pl011/deny.toml | 57 +++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 39 +++ rust/pl011/src/device.rs | 509 +++++++++++++++++++++++++++ rust/pl011/src/device_class.rs | 48 +++ rust/pl011/src/lib.rs | 556 ++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 45 +++ rust/qemu-api/.gitignore | 2 + rust/qemu-api/Cargo.lock | 7 + rust/qemu-api/Cargo.toml | 59 ++++ rust/qemu-api/README.md | 17 + rust/qemu-api/build.rs | 48 +++ rust/qemu-api/deny.toml | 57 +++ rust/qemu-api/meson.build | 0 rust/qemu-api/rustfmt.toml | 1 + rust/qemu-api/src/bindings.rs | 8 + rust/qemu-api/src/definitions.rs | 112 ++++++ rust/qemu-api/src/device_class.rs | 131 +++++++ rust/qemu-api/src/lib.rs | 29 ++ rust/qemu-api/src/tests.rs | 48 +++ rust/rustfmt.toml | 7 + rust/wrapper.h | 39 +++ scripts/cargo_wrapper.py | 288 ++++++++++++++++ scripts/meson-buildoptions.sh | 6 + 39 files changed, 2625 insertions(+), 19 deletions(-) create mode 100644 rust/.cargo/config.toml create mode 100644 rust/.gitignore create mode 100644 rust/meson.build create mode 100644 rust/pl011/.gitignore create mode 100644 rust/pl011/Cargo.lock create mode 100644 rust/pl011/Cargo.toml create mode 100644 rust/pl011/README.md create mode 100644 rust/pl011/deny.toml create mode 100644 rust/pl011/meson.build create mode 120000 rust/pl011/rustfmt.toml create mode 100644 rust/pl011/src/definitions.rs create mode 100644 rust/pl011/src/device.rs create mode 100644 rust/pl011/src/device_class.rs create mode 100644 rust/pl011/src/lib.rs create mode 100644 rust/pl011/src/memory_ops.rs create mode 100644 rust/qemu-api/.gitignore create mode 100644 rust/qemu-api/Cargo.lock create mode 100644 rust/qemu-api/Cargo.toml create mode 100644 rust/qemu-api/README.md create mode 100644 rust/qemu-api/build.rs create mode 100644 rust/qemu-api/deny.toml create mode 100644 rust/qemu-api/meson.build create mode 120000 rust/qemu-api/rustfmt.toml create mode 100644 rust/qemu-api/src/bindings.rs create mode 100644 rust/qemu-api/src/definitions.rs create mode 100644 rust/qemu-api/src/device_class.rs create mode 100644 rust/qemu-api/src/lib.rs create mode 100644 rust/qemu-api/src/tests.rs create mode 100644 rust/rustfmt.toml create mode 100644 rust/wrapper.h create mode 100644 scripts/cargo_wrapper.py
base-commit: 7914bda497f07965f15a91905cd7ed9eaf1c1092
Signature · 1 line
γαῖα πυρί μιχθήτω
RFC PATCH v4 1/7 build-sys: Add rust feature option
Add options for Rust in meson_options.txt, meson.build, configure to prepare for adding Rust code in the followup commits.
rust is a reserved meson name, so we have to use an alternative. with_rust was chosen.
A cargo_wrapper.py script is added that is heavily based on the work of Marc-André Lureau from 2021.
https://patchew.org/QEMU/[email protected]/
--- MAINTAINERS | 5 + configure | 11 ++ meson.build | 11 ++ meson_options.txt | 5 + scripts/cargo_wrapper.py | 294 ++++++++++++++++++++++++++++++++++ scripts/meson-buildoptions.sh | 6 + 6 files changed, 332 insertions(+) create mode 100644 scripts/cargo_wrapper.py
MAINTAINERS +5 -0
@@ -4226,6 +4226,11 @@ F: docs/sphinx/ F: docs/_templates/ F: docs/devel/docs.rst +Rust build system integration +M: Manos Pitsidianakis <[email protected]> +S: Maintained +F: scripts/cargo_wrapper.py + Miscellaneous ------------- Performance Tools and Tests
configure +11 -0
@@ -302,6 +302,9 @@ else objcc="${objcc-${cross_prefix}clang}" fi +with_rust="auto" +with_rust_target_triple="" + ar="${AR-${cross_prefix}ar}" as="${AS-${cross_prefix}as}" ccas="${CCAS-$cc}"
@@ -757,6 +760,12 @@ for opt do ;; --gdb=*) gdb_bin="$optarg" ;; + --enable-with-rust) with_rust=enabled + ;; + --disable-with-rust) with_rust=disabled + ;; + --with-rust-target-triple=*) with_rust_target_triple="$optarg" + ;; # everything else has the same name in configure and meson --*) meson_option_parse "$opt" "$optarg" ;;
@@ -1789,6 +1798,8 @@ if test "$skip_meson" = no; then test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE" test "$plugins" = yes && meson_option_add "-Dplugins=true" test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg" + test "$with_rust" != enabled && meson_option_add "-Dwith_rust=$with_rust" + test "$with_rust_target_triple" != "" && meson_option_add "-Dwith_rust_target_triple=$with_rust_target_triple" run_meson() { NINJA=$ninja $meson setup "$@" "$PWD" "$source_path" }
meson.build +11 -0
@@ -290,6 +290,12 @@ foreach lang : all_languages endif endforeach +cargo = not_found +if get_option('with_rust').allowed() + cargo = find_program('cargo', required: get_option('with_rust')) +endif +with_rust = cargo.found() + # default flags for all hosts # We use -fwrapv to tell the compiler that we require a C dialect where # left shift of signed integers is well defined and has the expected
@@ -2118,6 +2124,7 @@ endif config_host_data = configuration_data() +config_host_data.set('CONFIG_WITH_RUST', with_rust) audio_drivers_selected = [] if have_system audio_drivers_available = {
@@ -4243,6 +4250,10 @@ if 'objc' in all_languages else summary_info += {'Objective-C compiler': false} endif +summary_info += {'Rust support': with_rust} +if with_rust and get_option('with_rust_target_triple') != '' + summary_info += {'Rust target': get_option('with_rust_target_triple')} +endif option_cflags = (get_option('debug') ? ['-g'] : []) if get_option('optimization') != 'plain' option_cflags += ['-O' + get_option('optimization')]
meson_options.txt +5 -0
@@ -371,3 +371,8 @@ option('hexagon_idef_parser', type : 'boolean', value : true, option('x86_version', type : 'combo', choices : ['0', '1', '2', '3', '4'], value: '1', description: 'tweak required x86_64 architecture version beyond compiler default') + +option('with_rust', type: 'feature', value: 'auto', + description: 'Enable Rust support') +option('with_rust_target_triple', type : 'string', value: '', + description: 'Override Rust target triple')
scripts/cargo_wrapper.py new file +294 -0
294 lines not on this page · raw message
scripts/meson-buildoptions.sh +6 -0
@@ -79,6 +79,8 @@ meson_options_help() { printf "%s\n" ' auto/sigaltstack/ucontext/windows)' printf "%s\n" ' --with-pkgversion=VALUE use specified string as sub-version of the' printf "%s\n" ' package' + printf "%s\n" ' --with-rust-target-triple=VALUE' + printf "%s\n" ' Specify Rust host target triple' printf "%s\n" ' --with-suffix=VALUE Suffix for QEMU data/modules/config directories' printf "%s\n" ' (can be empty) [qemu]' printf "%s\n" ' --with-trace-file=VALUE Trace file prefix for simple backend [trace]'
@@ -216,6 +218,7 @@ meson_options_help() { printf "%s\n" ' vvfat vvfat image format support' printf "%s\n" ' werror Treat warnings as errors' printf "%s\n" ' whpx WHPX acceleration support' + printf "%s\n" ' with-rust Enable Rust support' printf "%s\n" ' xen Xen backend support' printf "%s\n" ' xen-pci-passthrough' printf "%s\n" ' Xen PCI passthrough support'
@@ -552,6 +555,9 @@ _meson_option_parse() { --enable-whpx) printf "%s" -Dwhpx=enabled ;; --disable-whpx) printf "%s" -Dwhpx=disabled ;; --x86-version=*) quote_sh "-Dx86_version=$2" ;; + --enable-with-rust) printf "%s" -Dwith_rust=enabled ;; + --disable-with-rust) printf "%s" -Dwith_rust=disabled ;; + --with-rust-target-triple=*) quote_sh "-Dwith_rust_target_triple=$2" ;; --enable-xen) printf "%s" -Dxen=enabled ;; --disable-xen) printf "%s" -Dxen=disabled ;; --enable-xen-pci-passthrough) printf "%s" -Dxen_pci_passthrough=enabled ;;
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v4 1/7] build-sys: Add rust feature option
Quoted 6 lines · Manos Pitsidianakis
Add options for Rust in meson_options.txt, meson.build, configure to prepare for adding Rust code in the followup commits.
rustis a reserved meson name, so we have to use an alternative.with_rustwas chosen.
Did you find any problem with the other approach that I sent, to support --cargo and the CARGO environment variable in configure?
Paolo
RFC PATCH v4 2/7 rust: add bindgen step as a meson dependency
In reply to Manos PitsidianakisAdd mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.
This way bindings will be created before the rust crate is compiled.
The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name as a target:
ninja aarch64-softmmu-generated.rs
The way the bindings are generated is:
1. All required C headers are included in a single file, in our case rust/wrapper.h for convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.
2. Meson creates a generated_rs target that runs bindgen making sure the architecture etc header dependencies are present.
3. The generated_rs target takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.
4. Meson creates rust hardware target dependencies from the rust_targets dictionary defined in rust/meson.build.
Since we cannot declare a dependency on generated_rs before it is declared in meson.build, the rust crate targets must be defined after the generated_rs target for each target architecture is defined. This way meson sets up the dependency tree properly.
5. After compiling each rust crate with the cargo_wrapper.py script, its static library artifact is linked as a whole-archive with the final binary.
--- MAINTAINERS | 3 ++ meson.build | 57 ++++++++++++++++++++ rust/.gitignore | 3 ++ rust/meson.build | 112 +++++++++++++++++++++++++++++++++++++++ rust/wrapper.h | 39 ++++++++++++++ scripts/cargo_wrapper.py | 18 +++---- 6 files changed, 220 insertions(+), 12 deletions(-) create mode 100644 rust/.gitignore create mode 100644 rust/meson.build create mode 100644 rust/wrapper.h
MAINTAINERS +3 -0
@@ -4230,6 +4230,9 @@ Rust build system integration M: Manos Pitsidianakis <[email protected]> S: Maintained F: scripts/cargo_wrapper.py +F: rust/meson.build +F: rust/wrapper.h +F: rust/.gitignore Miscellaneous -------------
meson.build +57 -0
63 lines not on this page · raw message
rust/.gitignore new file +3 -0
@@ -0,0 +1,3 @@ +# Ignore any cargo development build artifacts; for qemu-wide builds, all build +# artifacts will go to the meson build directory. +target
rust/meson.build new file +112 -0
112 lines not on this page · raw message
rust/wrapper.h new file +39 -0
@@ -0,0 +1,39 @@ +/* +* QEMU System Emulator +* +* Copyright 2024 Manos Pitsidianakis <[email protected]> +* +* Permission is hereby granted, free of charge, to any person obtaining a copy +* of this software and associated documentation files (the "Software"), to deal +* in the Software without restriction, including without limitation the rights +* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +* copies of the Software, and to permit persons to whom the Software is +* furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +* THE SOFTWARE. +*/ + +#include "qemu/osdep.h" +#include "qemu/module.h" +#include "qemu-io.h" +#include "sysemu/sysemu.h" +#include "hw/sysbus.h" +#include "exec/memory.h" +#include "chardev/char-fe.h" +#include "hw/clock.h" +#include "hw/qdev-clock.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/irq.h" +#include "qapi/error.h" +#include "migration/vmstate.h" +#include "chardev/char-serial.h"
scripts/cargo_wrapper.py +6 -12
@@ -111,6 +111,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str] env = os.environ env["CARGO_ENCODED_RUSTFLAGS"] = cfg + env["MESON_BUILD_DIR"] = str(target_dir) + env["MESON_BUILD_ROOT"] = str(args.meson_build_root) return (env, cargo_cmd)
@@ -231,19 +233,11 @@ def main() -> None: default=[], ) parser.add_argument( - "--meson-build-dir", - metavar="BUILD_DIR", - help="meson.current_build_dir()", + "--meson-build-root", + metavar="BUILD_ROOT", + help="meson.project_build_root(): the root build directory. Example: '/path/to/qemu/build'", type=Path, - dest="meson_build_dir", - required=True, - ) - parser.add_argument( - "--meson-source-dir", - metavar="SOURCE_DIR", - help="meson.current_source_dir()", - type=Path, - dest="meson_build_dir", + dest="meson_build_root", required=True, ) parser.add_argument(
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 10 lines · Manos Pitsidianakis
Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.
This way bindings will be created before the rust crate is compiled.
The bindings will end up in
BUILDDIR/{target}-generated.rsand have the same name as a target:ninja
aarch64-softmmu-generated.rs
Is there anything target-dependent in these files? You can generate it just once I think.
meson.build foreach target : target_dirs · line 3937
Quoted 6 lines
meson.build +6
-0+ if with_rust and target_type == 'system' + # FIXME: meson outputs the following warnings, which should be resolved + # before merging: + # > WARNING: Project specifies a minimum meson_version '>=0.63.0' but + # > uses features which were added in newer versions: + # > * 0.64.0: {'fs.copyfile'}
you can use configure_file() instead.
Quoted 1 line
+ # > * 1.0.0: {'dependencies arg in rust.bindgen', 'module rust as stable module'}
You can use
if meson.version().version_compare('>=1.0.0')
...
else
error('Rust support requires Meson version 1.0.0')
endif
meson.build foreach target : target_dirs · line 3974
Quoted 8 lines
meson.build +8
-0+ if target in rust_targets + rust_hw = ss.source_set() + foreach t: rust_targets[target] + rust_device_cargo_build = custom_target(t['name'], + output: t['output'], + depends: [bindings_rs], + build_always_stale: true, + command: t['command'])
Also "console: true".
meson.build foreach target : target_dirs · line 3979
Quoted 5 lines
meson.build +5
-0+ rust_dep = declare_dependency(link_args: [ + '-Wl,--whole-archive', + t['output'], + '-Wl,--no-whole-archive' + ],
This is not portable, but I think you can use just "link_whole:rust_device_cargo_build".
rust/meson.build line 33
Quoted 3 lines
rust/meson.build new file +3
-0+msrv = { + 'rustc': '1.77.2', + 'cargo': '1.77.2',
I think rustc and cargo are always matching, so no need to check both of them?
rust/meson.build line 97
Quoted 2 lines
rust/meson.build new file +2
-0+foreach rust_hw_target, rust_hws: rust_hw_target_list + foreach rust_hw_dev: rust_hws
Finding the available devices should be done using Kconfig symbols, probably by passing the path to the config-devices.mak file to build.rs via an environment variable.
Quoted 5 lines
rust/wrapper.h new file +5
-0+#include "qemu/osdep.h" +#include "qemu/module.h" +#include "qemu-io.h" +#include "sysemu/sysemu.h" +#include "hw/sysbus.h"
rust/wrapper.h line 39
Quoted 10 lines
rust/wrapper.h new file +10
-0+#include "exec/memory.h" +#include "chardev/char-fe.h" +#include "hw/clock.h" +#include "hw/qdev-clock.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/irq.h" +#include "qapi/error.h" +#include "migration/vmstate.h" +#include "chardev/char-serial.h"
I think all of these should be target-independent?
Quoted 27 lines
scripts/cargo_wrapper.py +5
-6@@ -111,6 +111,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str] env = os.environ env["CARGO_ENCODED_RUSTFLAGS"] = cfg + env["MESON_BUILD_DIR"] = str(target_dir) + env["MESON_BUILD_ROOT"] = str(args.meson_build_root) return (env, cargo_cmd)@@ -231,19 +233,11 @@ def main() -> None: default=[], ) parser.add_argument( - "--meson-build-dir", - metavar="BUILD_DIR", - help="meson.current_build_dir()", + "--meson-build-root", + metavar="BUILD_ROOT", + help="meson.project_build_root(): the root build directory. Example: '/path/to/qemu/build'", type=Path, - dest="meson_build_dir", - required=True, - )
scripts/cargo_wrapper.py def main() -> None: · line 243
Quoted 10 lines
scripts/cargo_wrapper.py +1
-6- parser.add_argument( - "--meson-source-dir", - metavar="SOURCE_DIR", - help="meson.current_source_dir()", - type=Path, - dest="meson_build_dir", + dest="meson_build_root", required=True, ) parser.add_argument(
Please squash in the previous patch.
Paolo
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
In reply to Manos PitsidianakisQuoted 36 lines · Manos Pitsidianakis
Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.
This way bindings will be created before the rust crate is compiled.
The bindings will end up in
BUILDDIR/{target}-generated.rsand have the same name as a target:ninja
aarch64-softmmu-generated.rsThe way the bindings are generated is:
1. All required C headers are included in a single file, in our case
rust/wrapper.hfor convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.2. Meson creates a
generated_rstarget that runs bindgen making sure the architecture etc header dependencies are present.3. The
generated_rstarget takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.4. Meson creates rust hardware target dependencies from the
rust_targetsdictionary defined inrust/meson.build.Since we cannot declare a dependency on
generated_rsbefore it is declared in meson.build, the rust crate targets must be defined after thegenerated_rstarget for each target architecture is defined. This way meson sets up the dependency tree properly.5. After compiling each rust crate with the
cargo_wrapper.pyscript, its static library artifact is linked as awhole-archivewith the final binary.Signed-off-by Manos Pitsidianakis <[email protected]>
<snip>
rust/meson.build line 35
Quoted 6 lines
rust/meson.build new file +6
-0+ +msrv = { + 'rustc': '1.77.2', + 'cargo': '1.77.2', + 'bindgen': '0.69.4', +}
This is still pretty bleeding edge (it even tripped up on the .cargo/bin/cargo I have installed). This needs to be set to the baseline which from:
https://wiki.qemu.org/RustInQemu/2022
Looks to be 1.24.0 for rustc and I guess even lower for cargo (Debian says 0.66.0). While it might make sense to delay merging if we are waiting for one distro to produce a new LTS we shouldn't be needing rustup by default.
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 12 lines · Alex Bennée
rust/meson.build line 35
Quoted 5 lines · Manos Pitsidianakis
rust/meson.build new file +5
-0+msrv = { + 'rustc': '1.77.2', + 'cargo': '1.77.2', + 'bindgen': '0.69.4', +}This is still pretty bleeding edge (it even tripped up on the .cargo/bin/cargo I have installed). This needs to be set to the baseline which from:
Quoted 6 lines
https://wiki.qemu.org/RustInQemu/2022
Looks to be 1.24.0 for rustc and I guess even lower for cargo (Debian says 0.66.0). While it might make sense to delay merging if we are waiting for one distro to produce a new LTS we shouldn't be needing rustup by default.
I suspect that some of the older distro releases in that chart have already fallen off the end of our supported-platforms set, so the minimum is probably newer than 1.24.0 now.
My take on this one is that (at some point, not necessarily right now) we want to look at:
* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates. I think (but forget the details) that some of what we've done with Python where we accept that the older-but-still-supported distro will end up taking the "download at build time" path in the build system might apply also for Rust.
* what, on the Rust side, is the version requirement? Presumably there's some features that are pretty much "we really need this and can't do without it" and some which are "this would be pretty awkward not to have but if we have to we can implement some kind of fallback/alternative with a TODO note to come back and clean up when our baseline moves forwards".
At that point we have more information to figure out what if any tradeoff we want to make.
thanks
-- PMM
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 7 lines · Peter Maydell
* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.
I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.
We can define a known-good set of versions:
* either via Cargo.lock (i.e. recording the versions and downloading them) or by including sources in the tree
* at any time in qemu.git, at branching time (three times a year), or on every release
(That's six possibilities overall).
Quoted 4 lines
I think (but forget the details) that some of what we've done with Python where we accept that the older-but-still-supported distro will end up taking the "download at build time" path in the build system might apply also for Rust.
Yes, and --without-download may be changed to the "--frozen" flag of cargo.
Quoted 6 lines
* what, on the Rust side, is the version requirement? Presumably there's some features that are pretty much "we really need this and can't do without it" and some which are "this would be pretty awkward not to have but if we have to we can implement some kind of fallback/alternative with a TODO note to come back and clean up when our baseline moves forwards".
Here are the stopping points that I found over the last couple weeks:
1.56.0: 2021 edition
1.59.0: const CStr::from_bytes_with_nul_unchecked (needed by cstr crate, see below)
1.64.0: std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals, offset_of!
I think 1.59.0 is pretty much the lower bound. Not having offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).
As far as I understand, Debian bullseye has 1.63.0 these days and it's the oldest platform we have.
Paolo
Quoted 6 lines
At that point we have more information to figure out what if any tradeoff we want to make.
thanks
-- PMM
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell
* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.
Quoted 3 lines
I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.
Yep, Rust is new enough that it is highly likely that crates will be missing in multiple distros.
For ordinary users, cargo will happily download the missing pieces so its not an issue.
For distro packagers, they'll just have to either package up the crates, or bundle them in their QEMU build. Cargo makes the latter easy at least. If distros don't want bundling, they'll need to go the more involved route of packaging deps.
IOW, from a distro POV, IMHO, we should focus on the Rust toolchain versions we need as the minimum bar.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 25 lines · Daniel P. Berrangé
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell
* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.
Quoted 3 lines
I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.
Yep, Rust is new enough that it is highly likely that crates will be missing in multiple distros.
For ordinary users, cargo will happily download the missing pieces so its not an issue.
For distro packagers, they'll just have to either package up the crates, or bundle them in their QEMU build. Cargo makes the latter easy at least. If distros don't want bundling, they'll need to go the more involved route of packaging deps.
Quoted 5 lines
IOW, from a distro POV, IMHO, we should focus on the Rust toolchain versions we need as the minimum bar.
With regards,
Daniel
I would like to add that, contrary to pip packages for python, it can be *trusted* that Rust crates.io will keep packages and their different versions, as it was designed to be an immutable registry.
Indeed, users can't remove their package, except for strong security issues, to the opposite of pip.
Besides offline compilation scenario, are there any other reason for wanting to use distro packages instead of letting cargo do the work?
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 29 lines · Pierrick Bouvier
Quoted 28 lines · Daniel P. Berrangé
Quoted 9 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell
* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.
Quoted 3 lines
I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.
Yep, Rust is new enough that it is highly likely that crates will be missing in multiple distros.
For ordinary users, cargo will happily download the missing pieces so its not an issue.
For distro packagers, they'll just have to either package up the crates, or bundle them in their QEMU build. Cargo makes the latter easy at least. If distros don't want bundling, they'll need to go the more involved route of packaging deps.
IOW, from a distro POV, IMHO, we should focus on the Rust toolchain versions we need as the minimum bar.
Quoted 8 lines
Quoted 2 lines
With regards,
DanielI would like to add that, contrary to pip packages for python, it can be *trusted* that Rust crates.io will keep packages and their different versions, as it was designed to be an immutable registry.
Indeed, users can't remove their package, except for strong security issues, to the opposite of pip.
Besides offline compilation scenario, are there any other reason for wanting to use distro packages instead of letting cargo do the work?
Only distribution packaging is the concern here, where packaged build dependencies may be required.
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
In reply to Paolo BonziniQuoted 45 lines · Paolo Bonzini
Quoted 7 lines · Peter Maydell
* what is the actual baseline requirement? We definitely want to support "using rustup on an older system" (should be no problem) and "current distro building QEMU using the distro's rust", I assume. It would certainly be nice to have "building QEMU on the older-but-still-in-our-support-list distro releases with that distro's rust", but this probably implies not just a minimum rust version but also a limited set of crates.
I don't think limiting ourselves to the set of crates in the distro is feasible. It's not the way the language works, for example I tried checking if the "cstr" crate exists and I didn't find it in Debian.
We can define a known-good set of versions:
* either via Cargo.lock (i.e. recording the versions and downloading them) or by including sources in the tree
* at any time in qemu.git, at branching time (three times a year), or on every release(That's six possibilities overall).
Quoted 4 lines
I think (but forget the details) that some of what we've done with Python where we accept that the older-but-still-supported distro will end up taking the "download at build time" path in the build system might apply also for Rust.
Yes, and --without-download may be changed to the "--frozen" flag of cargo.
Quoted 6 lines
* what, on the Rust side, is the version requirement? Presumably there's some features that are pretty much "we really need this and can't do without it" and some which are "this would be pretty awkward not to have but if we have to we can implement some kind of fallback/alternative with a TODO note to come back and clean up when our baseline moves forwards".
Here are the stopping points that I found over the last couple weeks:
1.56.0: 2021 edition
1.59.0: constCStr::from_bytes_with_nul_unchecked(needed by cstr crate, see below)
1.64.0:std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals,offset_of!
Quoted 6 lines
I think 1.59.0 is pretty much the lower bound. Not having
offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).As far as I understand, Debian bullseye has 1.63.0 these days and it's the oldest platform we have.
I was going to say Bookworm has now superseded Bullseye which will reach its release + 3 year support point in August. However the version you mention in the Bookworm one!
Quoted 9 lines
Paolo
Quoted 6 lines
At that point we have more information to figure out what if any tradeoff we want to make.
thanks
-- PMM
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
In reply to Paolo BonziniQuoted 11 lines · Paolo Bonzini
Here are the stopping points that I found over the last couple weeks:
1.56.0: 2021 edition
1.59.0: constCStr::from_bytes_with_nul_unchecked(needed by cstr crate, see below)
1.64.0:std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals,offset_of!
Quoted 4 lines
I think 1.59.0 is pretty much the lower bound. Not having
offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).
An additional question: does our minimum rust version requirement indicate that users with this rust version can compile other dependencies that satisfy QEMU requirements, such as bindgen?
Because I find 1.59.0 can only go to compile bindgen 0.63.0 [1].
[1]: bindgen 0.63.0 MSRV: https://github.com/rust-lang/rust-bindgen/tree/v0.63.0?tab=readme-ov-file#msrv
-Zhao
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 13 lines · Zhao Liu
Quoted 11 lines · Paolo Bonzini
Here are the stopping points that I found over the last couple weeks:
1.56.0: 2021 edition
1.59.0: constCStr::from_bytes_with_nul_unchecked(needed by cstr crate, see below)
1.64.0:std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals,offset_of!
Quoted 8 lines
Quoted 4 lines
I think 1.59.0 is pretty much the lower bound. Not having
offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).An additional question: does our minimum rust version requirement indicate that users with this rust version can compile other dependencies that satisfy QEMU requirements, such as bindgen?
Yes (though in the case of bindgen, like cargo and rustc, we'll use it from the distro; Debian bookworm has 0.60.1 so that's what we'll have to stick with).
Paolo
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
Quoted 23 lines · Paolo Bonzini
Date: Wed, 10 Jul 2024 16:50:10 +0200
From: Paolo Bonzini <[email protected]>
Subject: Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependencyQuoted 18 lines · Zhao Liu
Quoted 15 lines · Paolo Bonzini
Here are the stopping points that I found over the last couple weeks:
1.56.0: 2021 edition
1.59.0: constCStr::from_bytes_with_nul_unchecked(needed by cstr crate, see below)
1.64.0:std::ffi::c_char
1.65.0: Generic Associated Types
1.74.0: Clippy can be configured in Cargo.toml
1.77.0: C string literals,offset_of!I think 1.59.0 is pretty much the lower bound. Not having
offset_of! will be a bit painful, but it can be worked around (and pretty much has to be, because 1.77.0 is really new).
Quoted 8 lines
Quoted 3 lines
An additional question: does our minimum rust version requirement indicate that users with this rust version can compile other dependencies that satisfy QEMU requirements, such as bindgen?
Yes (though in the case of bindgen, like cargo and rustc, we'll use it from the distro; Debian bookworm has 0.60.1 so that's what we'll have to stick with).
I just did some build tests to hopefully help Manos clarify the gap.
With 1.59.0 rust and 0.60.1 bindgen, I found the big problem is this version can't support current external crates ("bilge" and "arbitrary-int").
The lowest possible version of bilge is 0.1.1 (because 0.1.0 contains the removed unstable feature), but v0.1.1 bilge depends on arbitrary-int (lowest bound is 1.2.4), which is not supported by v1.59.0 rust. So for this case either we have to keep raising the minimum rust version, or we have to reconsider the external crates.
And with 1.63.0, the main problem is core_ffi_c is unstable (once_cell doesn't matter since it's unneeded). Roughly speaking, it seems that if we didn't have to use the c type as parameters, this problem could possibly be worked around.
So maybe 1.63.0 is the proper minimum version and this is aligned with Debian bullseye.
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
In reply to Alex BennéeQuoted 50 lines · Alex Bennée
Quoted 36 lines · Manos Pitsidianakis
Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.
This way bindings will be created before the rust crate is compiled.
The bindings will end up in
BUILDDIR/{target}-generated.rsand have the same name as a target:ninja
aarch64-softmmu-generated.rsThe way the bindings are generated is:
1. All required C headers are included in a single file, in our case
rust/wrapper.hfor convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.2. Meson creates a
generated_rstarget that runs bindgen making sure the architecture etc header dependencies are present.3. The
generated_rstarget takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.4. Meson creates rust hardware target dependencies from the
rust_targetsdictionary defined inrust/meson.build.Since we cannot declare a dependency on
generated_rsbefore it is declared in meson.build, the rust crate targets must be defined after thegenerated_rstarget for each target architecture is defined. This way meson sets up the dependency tree properly.5. After compiling each rust crate with the
cargo_wrapper.pyscript, its static library artifact is linked as awhole-archivewith the final binary.Signed-off-by Manos Pitsidianakis <[email protected]><snip>
rust/meson.build line 35
Quoted 6 lines
rust/meson.build new file +6
-0+ +msrv = { + 'rustc': '1.77.2', + 'cargo': '1.77.2', + 'bindgen': '0.69.4', +}This is still pretty bleeding edge (it even tripped up on the .cargo/bin/cargo I have installed). This needs to be set to the baseline which from:
Quoted 6 lines
https://wiki.qemu.org/RustInQemu/2022
Looks to be 1.24.0 for rustc and I guess even lower for cargo (Debian says 0.66.0). While it might make sense to delay merging if we are waiting for one distro to produce a new LTS we shouldn't be needing rustup by default.
Also bindgen, do we need such a new one? On Trixie:
Message:
../../rust/meson.build:41:4: ERROR: Problem encountered: bindgen version 0.66.1 is unsupported: Please upgrade to at least 0.69.4
A full log can be found at /home/alex/lsrc/qemu.git/builds/rust/meson-logs/meson-log.txt
ERROR: meson setup failed
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [RFC PATCH v4 2/7] rust: add bindgen step as a meson dependency
In reply to Manos PitsidianakisQuoted 3 lines · Manos Pitsidianakis
Add mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.
<snip>
meson.build foreach target : target_dirs · line 3952
Quoted 12 lines
meson.build +12
-0+ # We need one bindings_rs build target per arch target, so give them + # arch-specific names. + copy = fs.copyfile('rust/wrapper.h', + target + '_wrapper.h') + bindings_rs = rust_bindgen.bindgen( + input: copy, + dependencies: arch_deps + lib_deps, + output: 'bindings-' + target + '.rs', + include_directories: include_directories('.', 'include'), + args: [ + '--ctypes-prefix', 'core::ffi', + '--formatter', 'rustfmt',
I guess we also need to detect rustfmt, although it seems to be non-fatal:
15:59:27 [alex@debian-arm64:~/l/q/b/rust] review/rust-pl011-v4 + ninja [831/3041] Generating bindings for Rust rustmod-bindgen-aarch64-softmmu_wrapper.h Failed to run rustfmt: cannot find binary path (non-fatal, continuing) [3041/3041] Linking target tests/qtest/qos-test
<snip>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
RFC PATCH v4 3/7 rust: add crate to expose bindings and interfaces
In reply to Manos PitsidianakisAdd rust/qemu-api, which exposes rust-bindgen generated FFI bindings and provides some declaration macros for symbols visible to the rest of QEMU.
--- MAINTAINERS | 7 ++ rust/.cargo/config.toml | 2 + rust/qemu-api/.gitignore | 2 + rust/qemu-api/Cargo.lock | 7 ++ rust/qemu-api/Cargo.toml | 59 ++++++++++++++ rust/qemu-api/README.md | 17 ++++ rust/qemu-api/build.rs | 48 +++++++++++ rust/qemu-api/deny.toml | 57 +++++++++++++ rust/qemu-api/meson.build | 0 rust/qemu-api/rustfmt.toml | 1 + rust/qemu-api/src/bindings.rs | 8 ++ rust/qemu-api/src/definitions.rs | 112 +++++++++++++++++++++++++ rust/qemu-api/src/device_class.rs | 131 ++++++++++++++++++++++++++++++ rust/qemu-api/src/lib.rs | 29 +++++++ rust/qemu-api/src/tests.rs | 48 +++++++++++ rust/rustfmt.toml | 7 ++ 16 files changed, 535 insertions(+) create mode 100644 rust/.cargo/config.toml create mode 100644 rust/qemu-api/.gitignore create mode 100644 rust/qemu-api/Cargo.lock create mode 100644 rust/qemu-api/Cargo.toml create mode 100644 rust/qemu-api/README.md create mode 100644 rust/qemu-api/build.rs create mode 100644 rust/qemu-api/deny.toml create mode 100644 rust/qemu-api/meson.build create mode 120000 rust/qemu-api/rustfmt.toml create mode 100644 rust/qemu-api/src/bindings.rs create mode 100644 rust/qemu-api/src/definitions.rs create mode 100644 rust/qemu-api/src/device_class.rs create mode 100644 rust/qemu-api/src/lib.rs create mode 100644 rust/qemu-api/src/tests.rs create mode 100644 rust/rustfmt.toml
MAINTAINERS +7 -0
@@ -3340,6 +3340,11 @@ F: hw/core/register.c F: include/hw/register.h F: include/hw/registerfields.h +Rust +M: Manos Pitsidianakis <[email protected]> +S: Maintained +F: rust/qemu-api + SLIRP M: Samuel Thibault <[email protected]> S: Maintained
@@ -4233,6 +4238,8 @@ F: scripts/cargo_wrapper.py F: rust/meson.build F: rust/wrapper.h F: rust/.gitignore +F: rust/rustfmt.toml +F: rust/.cargo/config.toml Miscellaneous -------------
rust/.cargo/config.toml new file +2 -0
@@ -0,0 +1,2 @@ +[build] +rustflags = ["-Crelocation-model=pic", "-Ctarget-feature=+crt-static"]
rust/qemu-api/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +# Ignore generated bindings file overrides. +src/bindings.rs.inc
rust/qemu-api/Cargo.lock new file +7 -0
@@ -0,0 +1,7 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "qemu_api" +version = "0.1.0"
rust/qemu-api/Cargo.toml new file +59 -0
59 lines not on this page · raw message
rust/qemu-api/README.md new file +17 -0
@@ -0,0 +1,17 @@ +# QEMU bindings and API wrappers + +This library exports helper Rust types, Rust macros and C FFI bindings for internal QEMU APIs. + +The C bindings can be generated with `bindgen`, using this build target: + +```console +$ ninja bindings-aarch64-softmmu.rs +``` + +## Generate Rust documentation + +To generate docs for this crate, including private items: + +```sh +cargo doc --no-deps --document-private-items +```
rust/qemu-api/build.rs new file +48 -0
48 lines not on this page · raw message
rust/qemu-api/deny.toml new file +57 -0
57 lines not on this page · raw message
rust/qemu-api/meson.build new file +0 -0
rust/qemu-api/rustfmt.toml new file +1 -0
@@ -0,0 +1 @@ +../rustfmt.toml
rust/qemu-api/src/bindings.rs new file +8 -0
@@ -0,0 +1,8 @@ +// Copyright 2024 Manos Pitsidianakis <[email protected]> +// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later + +#[cfg( MESON_BINDINGS_RS )] +include! (concat! (env! ("OUT_DIR" ),"/bindings.rs" )); + +#[cfg(not( MESON_BINDINGS_RS ))] +include! ("bindings.rs.inc" );
rust/qemu-api/src/definitions.rs new file +112 -0
112 lines not on this page · raw message
rust/qemu-api/src/device_class.rs new file +131 -0
131 lines not on this page · raw message
rust/qemu-api/src/lib.rs new file +29 -0
@@ -0,0 +1,29 @@ +// Copyright 2024 Manos Pitsidianakis <[email protected]> +// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later + +#![doc = include_str! ( "../README.md" )] + +// FIXME: remove improper_ctypes +#[allow( +improper_ctypes_definitions, +improper_ctypes, +non_camel_case_types, +non_snake_case, +non_upper_case_globals +)] +#[allow( +clippy::missing_const_for_fn, +clippy::useless_transmute, +clippy::too_many_arguments, +clippy::approx_constant, +clippy::use_self, +clippy::cast_lossless, +)] +#[rustfmt::skip] +pub mod bindings; + +pub mod definitions; +pub mod device_class; + +#[cfg(test)] +mod tests;
rust/qemu-api/src/tests.rs new file +48 -0
48 lines not on this page · raw message
rust/rustfmt.toml new file +7 -0
@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = true
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v4 3/7] rust: add crate to expose bindings and interfaces
Quoted 5 lines · Manos Pitsidianakis
Add
rust/qemu-api, which exposes rust-bindgen generated FFI bindings and provides some declaration macros for symbols visible to the rest of QEMU.Signed-off-by Manos Pitsidianakis <[email protected]>
Quoted 14 lines
--- MAINTAINERS | 7 ++ rust/.cargo/config.toml | 2 + rust/qemu-api/.gitignore | 2 + rust/qemu-api/Cargo.lock | 7 ++ rust/qemu-api/Cargo.toml | 59 ++++++++++++++ rust/qemu-api/README.md | 17 ++++ rust/qemu-api/build.rs | 48 +++++++++++ rust/qemu-api/deny.toml | 57 +++++++++++++ rust/qemu-api/meson.build | 0 rust/qemu-api/rustfmt.toml | 1 + rust/qemu-api/src/bindings.rs | 8 ++ rust/qemu-api/src/definitions.rs | 112 +++++++++++++++++++++++++ rust/qemu-api/src/device_class.rs | 131 ++++++++++++++++++++++++++++++
I'd rather put the various definitions in directories that roughly correspond to the C files, so rust/qemu-api/src/{util,qom,hw/core/}.
rust/qemu-api/Cargo.toml line 30
Quoted 7 lines
rust/qemu-api/Cargo.toml new file +7
-0+correctness = { level = "deny", priority = -1 } +suspicious = { level = "deny", priority = -1 } +complexity = { level = "deny", priority = -1 } +perf = { level = "deny", priority = -1 } +cargo = { level = "deny", priority = -1 } +nursery = { level = "deny", priority = -1 } +style = { level = "deny", priority = -1 }
Groups are problematic because they can (and will) cause breakage when new failures are added. I think we should have a non-fatal job to test with groups, but by default we can add a large number of lints and "allow" the groups.
Quoted 1 line
+cast_ptr_alignment = "allow"
Why?
Quoted 1 line
+missing_safety_doc = "allow"
Please add it to individual files at the top.
Quoted 83 lines
rust/qemu-api/README.md new file +17
-0@@ -0,0 +1,17 @@ +# QEMU bindings and API wrappers + +This library exports helper Rust types, Rust macros and C FFI bindings for internal QEMU APIs. + +The C bindings can be generated with `bindgen`, using this build target: + +```console +$ ninja bindings-aarch64-softmmu.rs +``` + +## Generate Rust documentation + +To generate docs for this crate, including private items: + +```sh +cargo doc --no-deps --document-private-items +```rust/qemu-api/build.rs new file +48
-0@@ -0,0 +1,48 @@ +// Copyright 2024 Manos Pitsidianakis <[email protected]> +// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later + +use std::{env, path::Path }; + +fn main () { +println! ("cargo:rerun-if-env-changed=MESON_BUILD_ROOT" ); +println! ("cargo:rerun-if-changed=src/bindings.rs.inc" ); + +let out_dir = env::var_os ("OUT_DIR" ).unwrap (); + +if let Some (build_dir) = std::env::var_os ("MESON_BUILD_ROOT" ) { +let mut build_dir =Path ::new (&build_dir).to_path_buf (); +let mut out_dir =Path ::new (&out_dir).to_path_buf (); +assert! ( + build_dir.exists (), +"MESON_BUILD_ROOT value does not exist on filesystem: {}" , + build_dir.display () + ); +assert! ( + build_dir.is_dir (), +"MESON_BUILD_ROOT value is not actually a directory: {}" , + build_dir.display () + ); +// TODO: add logic for other guest target architectures. + build_dir.push ("bindings-aarch64-softmmu.rs" ); +let bindings_rs = build_dir; +assert! ( + bindings_rs.exists (), +"MESON_BUILD_ROOT/bindings-aarch64-softmmu.rs does not exist on filesystem: {}" , + bindings_rs.display () + ); +assert! ( + bindings_rs.is_file (), +"MESON_BUILD_ROOT/bindings-aarch64-softmmu.rs is not a file: {}" , + bindings_rs.display () + ); + out_dir.push ("bindings.rs" ); + std::fs::copy (bindings_rs, out_dir).unwrap (); +println! ("cargo:rustc-cfg=MESON_BINDINGS_RS" ); + }else if !Path ::new ("src/bindings.rs.inc" ).exists () { +panic! ( +"No generated C bindings found! Either build them manually with bindgen or with meson \ +(`ninja bindings-aarch64-softmmu.rs`) and copy them to src/bindings.rs.inc, or build \ +through meson." + ); + } +}rust/qemu-api/deny.toml new file +0
-0
rust/qemu-api/deny.toml line 10
Quoted 10 lines
rust/qemu-api/deny.toml new file +10
-0@@ -0,0 +1,57 @@ +# cargo-deny configuration file + +[graph] +targets = [ + "aarch64-unknown-linux-gnu", + "x86_64-unknown-linux-gnu", + "x86_64-apple-darwin", + "aarch64-apple-darwin", + "x86_64-pc-windows-gnu", + "aarch64-pc-windows-gnullvm",
Do we actually support LLVM Windows targets?
rust/qemu-api/deny.toml line 20
Quoted 10 lines
rust/qemu-api/deny.toml new file +10
-0+unsafe impl Sync for TypeInfo {} +unsafe impl Sync for VMStateDescription {} + +#[macro_export] +macro_rules! module_init { + ($func:expr, $type:expr) => { + #[used] + #[cfg_attr(target_os = "linux", link_section = ".ctors")] + #[cfg_attr(target_os = "macos", link_section = "__DATA,__mod_init_func")] + #[cfg_attr(target_os = "windows", link_section = ".CRT$XCU")]
I'd rather use the ctor crate. Also, this should be src/util/module.rs so that it's easy to find the matching C source.
Quoted 53 lines
rust/qemu-api/deny.toml new file +53
-0+ pub static LOAD_MODULE: extern "C" fn() = { + assert!($type < $crate::bindings::module_init_type_MODULE_INIT_MAX); + + extern "C" fn __load() { + // ::std::panic::set_hook(::std::boxed::Box::new(|_| {})); + + unsafe { + $crate::bindings::register_module_init(Some($func), $type); + } + } + + __load + }; + }; + (qom: $func:ident => $body:block) => { + // NOTE: To have custom identifiers for the ctor func we need to either supply + // them directly as a macro argument or create them with a proc macro. + #[used] + #[cfg_attr(target_os = "linux", link_section = ".ctors")] + #[cfg_attr(target_os = "macos", link_section = "__DATA,__mod_init_func")] + #[cfg_attr(target_os = "windows", link_section = ".CRT$XCU")] + pub static LOAD_MODULE: extern "C" fn() = { + extern "C" fn __load() { + // ::std::panic::set_hook(::std::boxed::Box::new(|_| {})); + #[no_mangle] + unsafe extern "C" fn $func() { + $body + } + + unsafe { + $crate::bindings::register_module_init( + Some($func), + $crate::bindings::module_init_type_MODULE_INIT_QOM, + ); + } + } + + __load + }; + }; +} + +#[macro_export] +macro_rules! type_info { + ($(#[$outer:meta])* + $name:ident: $t:ty, + $(name: $tname:expr,)* + $(parent: $pname:expr,)* + $(instance_init: $ii_fn:expr,)* + $(instance_post_init: $ipi_fn:expr,)* + $(instance_finalize: $if_fn:expr,)* + $(abstract_: $a_val:expr,)* + $(class_init: $ci_fn:expr,)*
rust/qemu-api/deny.toml
Quoted 10 lines
rust/qemu-api/deny.toml new file +10
-0+ $(class_base_init: $cbi_fn:expr,)* + ) => { + #[used] + $(#[$outer])* + pub static $name: $crate::bindings::TypeInfo = $crate::bindings::TypeInfo { + $(name: { + #[used] + static TYPE_NAME: &::core::ffi::CStr = $tname; + $tname.as_ptr() + },)*
Please use the cstr crate. Even better, add a trait
pub unsafe trait ObjectType: Sized {
const TYPE_NAME: &'static CStr; }
and then just <$t as ObjectType>::TYPE_NAME
Likewise, the parent could be mandatory like
+ $name:ident: $t:ty($p:ty)
and use <$p as ObjectType>::TYPE_NAME.
Quoted 1 line
+ ..unsafe { MaybeUninit::<$crate::bindings::TypeInfo>::zeroed().assume_init() }
I suggest something like the Zeroed trait I had in my experiment:
==========
#![allow(clippy::undocumented_unsafe_blocks)]
use std::mem::MaybeUninit;
/// Trait providing an easy way to obtain an all-zero
/// value for a struct
///
/// # Safety
///
/// Only add this to a type if MaybeUninit::zeroed().assume_init() /// is valid for that type.
pub unsafe trait Zeroed: Sized {
fn zeroed() -> Self {
// SAFETY: If this weren't safe, just do not add the
// trait to a type.
unsafe { MaybeUninit::zeroed().assume_init() }
}
}// Put here all the impls that you need for the bindgen-provided types. unsafe impl Zeroed for crate::bindings::TypeInfo {}
=========
So that you can simply write Zeroed::zeroed(); there are other occurrences of this for Property, VMStateDescription, MemoryRegionOps.
Quoted 1 line
+use
std::sync::OnceLock;
OnceLock is not needed, initialization is single-threaded and executed once only. You can just make declare_properties return a fn() and
$crate::bindings::device_class_set_props(dc.as_mut(), $props());
rust/qemu-api/deny.toml
Quoted 2 lines
rust/qemu-api/deny.toml new file +2
-0+ dc.as_mut().realize = $realize_fn; + dc.as_mut().reset = $reset_fn;
Nice. Might also add the same trick as type_info!, for example $(realize: $realize_fn:expr,)*, to remove the "Some()".
rust/qemu-api/deny.toml
Quoted 9 lines
rust/qemu-api/deny.toml new file +9
-0+#[macro_export] +macro_rules! define_property { + ($name:expr, $state:ty, $field:expr, $prop:expr, $type:expr, default = $defval:expr$(,)*) => { + $crate::bindings::Property { + name: { + #[used] + static _TEMP: &::core::ffi::CStr = $name; + _TEMP.as_ptr() + },
Also please use cstr!() here.
rust/qemu-api/deny.toml
Quoted 10 lines
rust/qemu-api/deny.toml new file +10
-0+ bitnr: 0, + bitmask: 0, + set_default: true, + defval: $crate::bindings::Property__bindgen_ty_1 { u: $defval.into() }, + arrayoffset: 0, + arrayinfo: ::core::ptr::null(), + arrayfieldsize: 0, + link_type: ::core::ptr::null(), + } + };
I think you can just use "..Zeroed::zeroed()" instead of listing all the fields one by one.
Quoted 45 lines
rust/qemu-api/deny.toml new file +33
-0+#[repr(C)] +pub struct Properties<const N: usize>(pub OnceLock<[Property; N]>, pub fn() -> [Property; N]); + +impl<const N: usize> Properties<N> { + pub unsafe fn as_mut_ptr(&mut self) -> *mut Property { + _ = self.0.get_or_init(self.1); + self.0.get_mut().unwrap().as_mut_ptr() + } + > + #[no_mangle] + pub static mut $ident: $crate::device_class::Properties<PROP_LEN> = $crate::device_class::Properties(::std::sync::OnceLock::new(), _make_properties); + }; +} + +#[macro_export] +macro_rules! vm_state_description { + ($(#[$outer:meta])* + $name:ident, + $(name: $vname:expr,)* + $(unmigratable: $um_val:expr,)* + ) => { + #[used] + $(#[$outer])* + pub static $name: $crate::bindings::VMStateDescription = $crate::bindings::VMStateDescription { + $(name: { + #[used] + static VMSTATE_NAME: &::core::ffi::CStr = $vname; + $vname.as_ptr() + },)* + unmigratable: true, + ..unsafe { ::core::mem::MaybeUninit::<$crate::bindings::VMStateDescription>::zeroed().assume_init() } + }; + } +}rust/qemu-api/src/lib.rs new file +6
-0@@ -0,0 +1,29 @@ +// Copyright 2024 Manos Pitsidianakis <[email protected]> +// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later + +#![doc = include_str! ( "../README.md" )] + +// FIXME: remove improper_ctypes
rust/qemu-api/src/lib.rs line 16
Quoted 10 lines
rust/qemu-api/src/lib.rs new file +10
-0+#[allow( +improper_ctypes_definitions, +improper_ctypes, +non_camel_case_types, +non_snake_case, +non_upper_case_globals +)] +#[allow( + clippy::missing_const_for_fn, + clippy::useless_transmute,
This one definitely shouldn't be there.
Paolo
Quoted 67 lines
rust/qemu-api/src/lib.rs new file +13
-0+ clippy::too_many_arguments, + clippy::approx_constant, + clippy::use_self, + clippy::cast_lossless, +)] +#[rustfmt::skip] +pub mod bindings; + +pub mod definitions; +pub mod device_class; + +#[cfg(test)] +mod tests;rust/qemu-api/src/tests.rs new file +48
-0@@ -0,0 +1,48 @@ +// Copyright 2024 Manos Pitsidianakis <[email protected]> +// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later + +use crate ::{ + bindings::*, declare_properties, define_property, device_class_init, vm_state_description, +}; + +#[test] +fn test_device_decl_macros () { +// Test that macros can compile. +vm_state_description! { +VMSTATE , + name:c"name" , + unmigratable:true , + } + +#[repr( C )] +pub struct DummyState { +pub char_backend:CharBackend , +pub migrate_clock:bool , + } + +declare_properties! { +DUMMY_PROPERTIES , +define_property! ( +c"chardev" , +DummyState , + char_backend, +unsafe { &qdev_prop_chr }, +CharBackend + ), +define_property! ( +c"migrate-clk" , +DummyState , + migrate_clock, +unsafe { &qdev_prop_bool }, +bool + ), + } + +device_class_init! { + dummy_class_init, + props =>DUMMY_PROPERTIES , + realize_fn =>None , + reset_fn =>None , + vmsd =>VMSTATE , + } +}
RFC PATCH v4 4/7 rust: add PL011 device model
In reply to Manos PitsidianakisThis commit adds a re-implementation of hw/char/pl011.c in Rust.
How to build:
1. Make sure rust, cargo and bindgen (cargo install bindgen-cli) are installed
2. Configure a QEMU build with:
--enable-system --target-list=aarch64-softmmu --enable-with-rust
3. Launching a VM with qemu-system-aarch64 should use the Rust version of the pl011 device (unless it is not set up so in hw/arm/virt.c; the type of the UART device is hardcoded).
To confirm, inspect info qom-tree in the monitor and look for an x-pl011-rust device.
--- MAINTAINERS | 5 + meson.build | 4 + rust/meson.build | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 125 ++++++++ rust/pl011/Cargo.toml | 67 ++++ rust/pl011/README.md | 31 ++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 39 +++ rust/pl011/src/device.rs | 509 ++++++++++++++++++++++++++++++ rust/pl011/src/device_class.rs | 48 +++ rust/pl011/src/lib.rs | 556 +++++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 45 +++ 15 files changed, 1498 insertions(+) create mode 100644 rust/pl011/.gitignore create mode 100644 rust/pl011/Cargo.lock create mode 100644 rust/pl011/Cargo.toml create mode 100644 rust/pl011/README.md create mode 100644 rust/pl011/deny.toml create mode 100644 rust/pl011/meson.build create mode 120000 rust/pl011/rustfmt.toml create mode 100644 rust/pl011/src/definitions.rs create mode 100644 rust/pl011/src/device.rs create mode 100644 rust/pl011/src/device_class.rs create mode 100644 rust/pl011/src/lib.rs create mode 100644 rust/pl011/src/memory_ops.rs
MAINTAINERS +5 -0
@@ -1186,6 +1186,11 @@ F: include/hw/*/microbit*.h F: tests/qtest/microbit-test.c F: docs/system/arm/nrf.rst +ARM PL011 Rust device +M: Manos Pitsidianakis <[email protected]> +S: Maintained +F: rust/pl011/ + AVR Machines -------------
meson.build +4 -0
@@ -296,6 +296,10 @@ if get_option('with_rust').allowed() endif with_rust = cargo.found() +if with_rust + subdir('rust') +endif + # default flags for all hosts # We use -fwrapv to tell the compiler that we require a C dialect where # left shift of signed integers is well defined and has the expected
rust/meson.build +2 -0
@@ -93,6 +93,8 @@ subdir('qemu-api') # bindgen dependency is declared. rust_hw_target_list = {} +subdir('pl011') + foreach rust_hw_target, rust_hws: rust_hw_target_list foreach rust_hw_dev: rust_hws crate_metadata = {
rust/pl011/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +# Ignore generated bindings file overrides. +src/bindings.rs.inc
rust/pl011/Cargo.lock new file +125 -0
125 lines not on this page · raw message
rust/pl011/Cargo.toml new file +67 -0
67 lines not on this page · raw message
rust/pl011/README.md new file +31 -0
@@ -0,0 +1,31 @@ +# PL011 QEMU Device Model + +This library implements a device model for the PrimeCell® UART (PL011) +device in QEMU. + +## Build static lib + +Host build target must be explicitly specified: + +```sh +cargo build --target x86_64-unknown-linux-gnu +``` + +Replace host target triplet if necessary. + +## Generate Rust documentation + +To generate docs for this crate, including private items: + +```sh +cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu +``` + +To include direct dependencies like `bilge` (bitmaps for register types): + +```sh +cargo tree --depth 1 -e normal --prefix none \ + | cut -d' ' -f1 \ + | xargs printf -- '-p %s\n' \ + | xargs cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu +```
rust/pl011/deny.toml new file +57 -0
57 lines not on this page · raw message
rust/pl011/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +rust_pl011 = { + 'name': 'pl011', + 'dirname': 'pl011', + 'output': 'libpl011.a', + } + +rust_hw_target_list += {'aarch64-softmmu': [rust_pl011]}
rust/pl011/rustfmt.toml new file +1 -0
@@ -0,0 +1 @@ +../rustfmt.toml
rust/pl011/src/definitions.rs new file +39 -0
@@ -0,0 +1,39 @@ +// Copyright 2024 Manos Pitsidianakis <[email protected]> +// SPDX-License-Identifier: GPL-2.0 OR GPL-3.0-or-later + +//! Definitions required by QEMU when registering the device. + +use core::{mem::MaybeUninit , ptr::NonNull }; + +use qemu_api::bindings::*; + +use crate ::{device::PL011State , device_class::pl011_class_init}; + +qemu_api::type_info! { +PL011_ARM_INFO :PL011State , + name:c"x-pl011-rust" , + parent:TYPE_SYS_BUS_DEVICE , + instance_init:Some (pl011_init), + abstract_:false , + class_init:Some (pl011_class_init), +} + +#[used] +pub static VMSTATE_PL011 :VMStateDescription =VMStateDescription { + name:PL011_ARM_INFO .name, + unmigratable:true , + ..unsafe {MaybeUninit ::<VMStateDescription >::zeroed ().assume_init () } +}; + +#[no_mangle] +pub unsafe extern "C" fn pl011_init (obj: *mut Object ) { +assert! (!obj.is_null ()); +let mut state =NonNull ::new_unchecked (obj.cast ::<PL011State >()); + state.as_mut ().init (); +} + +qemu_api::module_init! { + qom: register_type => { +type_register_static (&PL011_ARM_INFO ); + } +}
rust/pl011/src/device.rs new file +509 -0
509 lines not on this page · raw message
rust/pl011/src/device_class.rs new file +48 -0
48 lines not on this page · raw message
rust/pl011/src/lib.rs new file +556 -0
556 lines not on this page · raw message
rust/pl011/src/memory_ops.rs new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v4 4/7] rust: add PL011 device model
MAINTAINERS F: include/hw/*/microbit*.h · line 1192
Quoted 4 lines · Manos Pitsidianakis
MAINTAINERS +4
-0+ARM PL011 Rust device +M: Manos Pitsidianakis <[email protected]> +S: Maintained +F: rust/pl011/
No need for this, since it's covered by rust/. If (when) it replaces the main one, the PL011-specific stanza will be assigned to ARM maintainers (while you keep covering it via rust/).
meson.build if get_option('with_rust').allowed() · line 301
Quoted 3 lines
meson.build +3
-0+if with_rust + subdir('rust') +endif
Should be in patch 3.
Quoted 1 line
+subdir('pl011')
As I said before it should be handled via Kconfig, but let's do that after the initial merge. However...
rust/pl011/Cargo.toml line 42
Quoted 11 lines
rust/pl011/Cargo.toml new file +11
-0+correctness = { level = "deny", priority = -1 } +suspicious = { level = "deny", priority = -1 } +complexity = { level = "deny", priority = -1 } +perf = { level = "deny", priority = -1 } +cargo = { level = "deny", priority = -1 } +nursery = { level = "deny", priority = -1 } +style = { level = "deny", priority = -1 } +# restriction group +dbg_macro = "deny" +rc_buffer = "deny" +as_underscore = "deny"
... repeated lints really suggest that you should use a workspace and a single cargo invocation to build both the rust-qapi and pl011 crates, which I think is doable if you can run bindgen just once.
Quoted 1 line
+use core::{mem::MaybeUninit,
ptr::NonNull};
Let's remove at least this unsafety.
rust/pl011/src/definitions.rs line 32
Quoted 12 lines
rust/pl011/src/definitions.rs new file +12
-0+#[used] +pub static VMSTATE_PL011 :VMStateDescription =VMStateDescription { + name:PL011_ARM_INFO .name, + unmigratable:true , + ..unsafe {MaybeUninit ::<VMStateDescription >::zeroed ().assume_init () } +}; + +#[no_mangle] +pub unsafe extern "C" fn pl011_init (obj: *mut Object ) { +assert! (!obj.is_null ()); +let mut state =NonNull ::new_unchecked (obj.cast ::<PL011State >()); + state.as_mut ().init ();
This is fine for now, but please add a
// TODO: this assumes that "all zeroes" is a valid state for all fields of // PL011State. This is not necessarily true of any #[repr(Rust)] structs, // including bilge-generated types. It should instead use MaybeUninit.
rust/pl011/src/definitions.rs line 38
Quoted 6 lines
rust/pl011/src/definitions.rs new file +6
-0+} + +qemu_api::module_init! { + qom: register_type => { +type_register_static (&PL011_ARM_INFO ); + }
Can you make the macro look like
MODULE_INIT_QOM: fn register_type() {
...
}so that it's clear what "register_type" is, and so that it's easier to extend it to more values?
rust/pl011/src/device.rs line 67
Quoted 2 lines
rust/pl011/src/device.rs new file +2
-0+#[doc(alias = "clk" )] + pub clock: NonNull<Clock>,
It's null when init() runs, so please use *mut Clock.
rust/pl011/src/device.rs line 69
Quoted 2 lines
rust/pl011/src/device.rs new file +2
-0+#[doc(alias = "migrate_clk" )] + pub migrate_clock: bool,
Please put all properties together in the struct for readability.
Quoted 54 lines
rust/pl011/src/device.rs new file +54
-0+} + +#[used] +pub static CLK_NAME : &CStr =c"clk" ; + +impl PL011State { +pub fn init (&mut self) { +unsafe { +memory_region_init_io ( +addr_of_mut! (self.iomem), +addr_of_mut! (*self).cast ::<Object >(), + &PL011_OPS , +addr_of_mut! (*self).cast ::<c_void >(), +PL011_ARM_INFO .name, +0x1000 , + ); +let sbd =addr_of_mut! (*self).cast ::<SysBusDevice >(); +let dev =addr_of_mut! (*self).cast ::<DeviceState >(); +sysbus_init_mmio (sbd,addr_of_mut! (self.iomem)); +for irqin self.interrupts.iter_mut () { +sysbus_init_irq (sbd, irq); + } + self.clock =NonNull ::new (qdev_init_clock_in ( + dev, +CLK_NAME .as_ptr (), +None ,/* pl011_clock_update */ +addr_of_mut! (*self).cast ::<c_void >(), +ClockEvent_ClockUpdate , + )) + .unwrap (); + } + } + +pub fn read (&mut self, offset:hwaddr , _size: core::ffi::c_uint ) ->u64 { +use RegisterOffset ::*; + +match RegisterOffset ::try_from (offset) { +Err (v)if (0x3f8 ..0x400 ).contains (&v) => { + u64::from (PL011_ID_ARM [((offset -0xfe0 ) >>2 )as usize ]) + } +Err (_) => { +// qemu_log_mask(LOG_GUEST_ERROR, "pl011_read: Bad offset 0x%x\n", (int)offset); +0 + } +Ok (DR ) => { +// s->flags &= ~PL011_FLAG_RXFF; + self.flags.set_receive_fifo_full (false ); +let c = self.read_fifo[self.read_pos]; +if self.read_count >0 { + self.read_count -=1 ; + self.read_pos = (self.read_pos +1 ) & (self.fifo_depth () -1 ); + } +if self.read_count ==0 { +// self.flags |= PL011_FLAG_RXFE;
rust/pl011/src/device.rs line 133
Quoted 10 lines
rust/pl011/src/device.rs new file +10
-0+ self.flags.set_receive_fifo_empty (true ); + } +if self.read_count +1 == self.read_trigger { +//self.int_level &= ~ INT_RX; + self.int_level &= !registers::INT_RX ; + } +// Update error bits. + self.receive_status_error_clear = c.to_be_bytes ()[3 ].into (); + self.update (); +unsafe {qemu_chr_fe_accept_input (&mut self.char_backend) };
Please add a comment here like
// TODO: this causes a callback that creates another "&mut self". // This is forbidden by Rust aliasing rules and has to be fixed // using interior mutability.
Paolo
RFC PATCH v4 5/7 .gitattributes: add Rust diff and merge attributes
In reply to Manos PitsidianakisSet rust source code to diff=rust (built-in with new git versions) and merge=binary for Cargo.lock files (they should not be merged but auto-generated by cargo)
--- .gitattributes | 3 +++ 1 file changed, 3 insertions(+)
.gitattributes +3 -0
@@ -2,3 +2,6 @@ *.h.inc diff=c *.m diff=objc *.py diff=python +*.rs diff=rust +*.rs.inc diff=rust +Cargo.lock diff=toml merge=binary
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v4 5/7] .gitattributes: add Rust diff and merge attributes
Quoted 5 lines · Manos Pitsidianakis
Set rust source code to diff=rust (built-in with new git versions) and merge=binary for Cargo.lock files (they should not be merged but auto-generated by cargo)
Signed-off-by Manos Pitsidianakis <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
RFC PATCH v4 6/7 DO NOT MERGE: add rustdoc build for gitlab pages
In reply to Manos PitsidianakisDeploy the generated rustdocs for my personal rust qemu fork on gitlab.
The URL is:
https://rust-for-qemu-epilys-aebb06ca9f9adfe6584811c14ae44156501d935ba4.gitlab.io/pl011/index.html
--- .gitlab-ci.d/buildtest.yml | 64 +++++++++++++++++++++++++++----------- 1 file changed, 45 insertions(+), 19 deletions(-)
.gitlab-ci.d/buildtest.yml +45 -19
@@ -716,31 +716,57 @@ build-tools-and-docs-debian: # For contributor forks we want to publish from any repo so # that users can see the results of their commits, regardless # of what topic branch they're currently using +# pages: +# extends: .base_job_template +# image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG +# stage: test +# needs: +# - job: build-tools-and-docs-debian +# script: +# - mkdir -p public +# # HTML-ised source tree +# - make gtags +# # We unset variables to work around a bug in some htags versions +# # which causes it to fail when the environment is large +# - CI_COMMIT_MESSAGE= CI_COMMIT_TAG_MESSAGE= htags +# -anT --tree-view=filetree -m qemu_init +# -t "Welcome to the QEMU sourcecode" +# - mv HTML public/src +# # Project documentation +# - make -C build install DESTDIR=$(pwd)/temp-install +# - mv temp-install/usr/local/share/doc/qemu/* public/ +# artifacts: +# when: on_success +# paths: +# - public +# variables: +# QEMU_JOB_PUBLISH: 1 +# The Docker image that will be used to build your app pages: - extends: .base_job_template - image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG - stage: test - needs: - - job: build-tools-and-docs-debian + image: rust:latest script: - - mkdir -p public - # HTML-ised source tree - - make gtags - # We unset variables to work around a bug in some htags versions - # which causes it to fail when the environment is large - - CI_COMMIT_MESSAGE= CI_COMMIT_TAG_MESSAGE= htags - -anT --tree-view=filetree -m qemu_init - -t "Welcome to the QEMU sourcecode" - - mv HTML public/src - # Project documentation - - make -C build install DESTDIR=$(pwd)/temp-install - - mv temp-install/usr/local/share/doc/qemu/* public/ + - rustup component add rustfmt + - DEBIAN_FRONTEND=noninteractive apt-get update -y + - DEBIAN_FRONTEND=noninteractive apt-get install -y python3-venv meson libgcrypt20-dev zlib1g-dev autoconf automake libtool bison flex git libglib2.0-dev libfdt-dev libpixman-1-dev ninja-build make libclang-14-dev + - cargo install bindgen-cli + - mkdir ./build/ + - cd ./build/ + - ../configure --enable-system --disable-kvm --target-list=aarch64-softmmu --enable-with-rust + - ninja "bindings-aarch64-softmmu.rs" + - cp ./bindings-aarch64-softmmu.rs ../rust/qemu-api/src/bindings.rs.inc + - cd ../rust/pl011/ + - cargo tree --depth 1 -e normal --prefix none | cut -d' ' -f1 | xargs + printf -- '-p %s\n' | xargs cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu + - cd ./../.. + - mv ./rust/pl011/target/x86_64-unknown-linux-gnu/doc ./public artifacts: when: on_success paths: - public - variables: - QEMU_JOB_PUBLISH: 1 + rules: + # This ensures that only pushes to the default branch will trigger + # a pages deploy + - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH coverity: image: $CI_REGISTRY_IMAGE/qemu/fedora:$QEMU_CI_CONTAINER_TAG
Signature · 1 line
γαῖα πυρί μιχθήτω
RFC PATCH v4 7/7 DO NOT MERGE: replace TYPE_PL011 with x-pl011-rust in arm virt machine
In reply to Manos PitsidianakisConvenience patch for testing the rust device.
--- hw/arm/virt.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/arm/virt.c +4 -0
@@ -929,7 +929,11 @@ static void create_uart(const VirtMachineState *vms, int uart,int irq = vms->irqmap[uart];const char compat[] ="arm,pl011\0arm,primecell" ;const char clocknames[] ="uartclk\0apb_pclk" ; +#ifdef CONFIG_WITH_RUST +DeviceState *dev =qdev_new ("x-pl011-rust" ); +#else DeviceState *dev =qdev_new (TYPE_PL011 ); +#endif SysBusDevice *s =SYS_BUS_DEVICE (dev);MachineState *ms =MACHINE (vms);
Signature · 1 line
γαῖα πυρί μιχθήτω
Quoted 11 lines · Manos Pitsidianakis
Changes from
v3->v4:
- Add rust-specific files to .gitattributes
- Added help text toscripts/cargo_wrapper.pyarguments (thanks Stephan)
- Split bindings separate crate
- Add declarative macros for symbols exported to QEMU to said crate
- Lowered MSRV to 1.77.2
- Removed auto-download and install of bindgen-cli
- Fixed re-compilation of Rust objects in case they are missing from filesystem
- Fixed optimized builds by adding #[used] (thanks Pierrick for the help debugging this)
I think the largest issue is that I'd rather have a single cargo build using a virtual manifest, because my hunch is that it'd be the easiest path towards Kconfig integration. But it's better to do this after merge, as the changes are pretty large. It's also independent from any other changes targeted at removing unsafe code, so no need to hold back on merging.
Other comments I made that should however be addressed before merging, from most to least important:
- TODO comments when the code is doing potential undefined behavior
- module structure should IMO resemble the C part of the tree
- only generate bindings.rs.inc once
- a couple abstractions that I'd like to have now: a trait to store the CStr corresponding to the structs, and one to generate all-zero structs without having to type "unsafe { MaybeUninit::zeroed().assume_init() }"
- I pointed out a couple lints that are too broad and should be enabled per-file, even if right now it's basically all files that include them.
- add support for --cargo and CARGO environment variables (if my patch works without too much hassle)
- I'd like to use ctor instead of non-portable linker magic, and the cstr crate instead of CStr statics or c""
- please check if -Wl,--whole-archive can be replaced with link_whole:
- probably, until Rust is enabled by default we should treat dependencies as a moving target and not commit Cargo.lock files. In the meanwhile we can discuss how to handle them.
And a few aesthetic changes on top of this.
With respect to lints, marking entire groups as "deny" is problematic. Before merge, I'd rather have the groups as just "warn", and add a long list of denied lints[1]. After merge we can also add a non-fatal CI job that runs clippy with nightly rust and with groups marked as "deny". This matches the check-python-tox job in python/.
[1] https://github.com/bonzini/rust-qemu/commit/95b25f7c5f4e
Thanks,
Paolo
Quoted 41 lines · Paolo Bonzini
Quoted 11 lines · Manos Pitsidianakis
Changes from
v3->v4:
- Add rust-specific files to .gitattributes
- Added help text toscripts/cargo_wrapper.pyarguments (thanks Stephan)
- Split bindings separate crate
- Add declarative macros for symbols exported to QEMU to said crate
- Lowered MSRV to 1.77.2
- Removed auto-download and install of bindgen-cli
- Fixed re-compilation of Rust objects in case they are missing from filesystem
- Fixed optimized builds by adding #[used] (thanks Pierrick for the help debugging this)I think the largest issue is that I'd rather have a single cargo build using a virtual manifest, because my hunch is that it'd be the easiest path towards
Kconfigintegration. But it's better to do this after merge, as the changes are pretty large. It's also independent from any other changes targeted at removing unsafe code, so no need to hold back on merging.Other comments I made that should however be addressed before merging, from most to least important:
- TODO comments when the code is doing potential undefined behavior
- module structure should IMO resemble the C part of the tree
- only generate bindings.rs.inc once
- a couple abstractions that I'd like to have now: a trait to store the CStr corresponding to the structs, and one to generate all-zero structs without having to type "unsafe { MaybeUninit::zeroed().assume_init() }"
- I pointed out a couple lints that are too broad and should be enabled per-file, even if right now it's basically all files that include them.
- add support for --cargo and CARGO environment variables (if my patch works without too much hassle)
- I'd like to use ctor instead of non-portable linker magic, and the cstr crate instead of CStr statics or c""
Quoted 7 lines
- please check if -Wl,--whole-archive can be replaced with
link_whole:- probably, until Rust is enabled by default we should treat dependencies as a moving target and not commit Cargo.lock files. In the meanwhile we can discuss how to handle them.
And a few aesthetic changes on top of this.
This series is still missing changes to enable build on all targets during CI, including cross-compiles, to prove that we're doing the correct thing on all our targetted platforms. That's a must have before considering it suitable for merge.
I also believe we should default to enabling rust toolchain by default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.
This is to give users a clear message that Rust is likely to become a fundamental part of QEMU, so they need to give feedback if they hit any problems / have use cases we've not anticipated that are problematic wrt Rust.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Il lun 8 lug 2024, 18:33 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 5 lines
This series is still missing changes to enable build on all targets during CI, including cross-compiles, to prove that we're doing the correct thing on all our targetted platforms. That's a must have before considering it suitable for merge.
But we're not—in particular it's still using several features not in all supported distros.
I also believe we should default to enabling rust toolchain by
Quoted 4 lines
default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.
I guess the detection could be done, but actually enabling the build part needs to wait until the minimum supported version is low enough.
Paolo
Quoted 16 lines
This is to give users a clear message that Rust is likely to become a fundamental part of QEMU, so they need to give feedback if they hit any problems / have use cases we've not anticipated that are problematic wrt Rust.
With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|
Quoted 11 lines · Paolo Bonzini
Il lun 8 lug 2024, 18:33 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 5 lines
This series is still missing changes to enable build on all targets during CI, including cross-compiles, to prove that we're doing the correct thing on all our targetted platforms. That's a must have before considering it suitable for merge.
But we're not—in particular it's still using several features not in all supported distros.
That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.
Quoted 8 lines
I also believe we should default to enabling rust toolchain by
Quoted 4 lines
default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.
I guess the detection could be done, but actually enabling the build part needs to wait until the minimum supported version is low enough.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 6 lines
That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.
In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).
Paolo
Quoted 21 lines
Quoted 9 lines
I also believe we should default to enabling rust toolchain by
Quoted 4 lines
default in configure, and require and explicit --without-rust to disable it, *despite* it not technically being a mandatory feature....yet.
I guess the detection could be done, but actually enabling the build part needs to wait until the minimum supported version is low enough.
With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|
Quoted 12 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:
Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 6 lines
That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.
Quoted 5 lines
In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).
Paolo
If you do, I'd really appreciate it if you did not use any part of my patches.
Thanks,
Manos
Il lun 8 lug 2024, 20:39 Manos Pitsidianakis <[email protected]> ha scritto:
Quoted 16 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:
Quoted 13 lines
Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 6 lines
That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.
Quoted 7 lines
Quoted 3 lines
In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).
If you do, I'd really appreciate it if you did not use any part of my patches.
"We" means that you would accept patches, review them and apply them until any agreed-upon conditions for merging are satisfied, and then send either a final series or a pull request for inclusion in QEMU.
Paolo
Quoted 20 lines · Paolo Bonzini
Il lun 8 lug 2024, 20:39 Manos Pitsidianakis <[email protected]> ha scritto:
Quoted 16 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:
Quoted 12 lines
Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 6 lines
That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).
If you do, I'd really appreciate it if you did not use any part of my patches.
"We" means that you would accept patches, review them and apply them until any agreed-upon conditions for merging are satisfied, and then send either a final series or a pull request for inclusion in QEMU.
Ah, alright. That wasn't obvious because that e-mail was not directed to me nor did it mention my name :)
I do not want to do that, in any case. I do not think it's the right approach.
Signature · 2 lines
Manos Pitsidianakis Emulation and Virtualization Engineer at Linaro Ltd
Quoted 2 lines · Manos Pitsidianakis
Ah, alright. That wasn't obvious because that e-mail was not directed to me nor did it mention my name :)
Oh, ok. Sorry about that. Generally when I say "we" I include as large a part of the community as applicable.
Quoted 1 line
I do not want to do that, in any case. I do not think it's the right approach.
No problem with that (and in fact I agree, as I'd prefer a speedy merge and doing the work on the QEMU master branch); however, we need to reach an agreement on that and everybody (including Daniel) needs to explain the reason for their position.
Daniel's proposed criteria for merging include:
- CI integration
- CI passing for all supported targets (thus lowering the MSRV to 1.63.0)
- plus any the code changes that were or will be requested during review
That seems to be a pretty high amount of work, and until it's done everyone else is unable to contribute, not even in directions orthogonal to the above (cross compilation support, less unsafe code, porting more devices). So something has to give: either we decide for an early merge, where the code is marked as experimental and disabled by default. Personally I think it's fine, the contingency plan is simply to "git rm -rf rust/". Or we can keep the above stringent requirements for merging, but then I don't see it as a one-person job.
If I can say so, developing on a branch would also be a useful warm-up for you in the maintainer role, if we expect that there will be significant community contributions to Rust.
Paolo
Quoted 20 lines · Paolo Bonzini
Quoted 2 lines · Manos Pitsidianakis
Ah, alright. That wasn't obvious because that e-mail was not directed to me nor did it mention my name :)
Oh, ok. Sorry about that. Generally when I say "we" I include as large a part of the community as applicable.
Quoted 1 line
I do not want to do that, in any case. I do not think it's the right approach.
No problem with that (and in fact I agree, as I'd prefer a speedy merge and doing the work on the QEMU master branch); however, we need to reach an agreement on that and everybody (including Daniel) needs to explain the reason for their position.
Daniel's proposed criteria for merging include:
- CI integration
- CI passing for all supported targets (thus lowering the MSRV to 1.63.0)
- plus any the code changes that were or will be requested during review
Quoted 4 lines
That seems to be a pretty high amount of work, and until it's done everyone else is unable to contribute, not even in directions orthogonal to the above (cross compilation support, less unsafe code, porting more devices).
My thought is that the initial merge focuses only on the build system integration. So that's basically patches 1 + 2 in this series.
IMHO that is small enough that we should be able to demonstrate that we detect Rust, run bindgen & compile its result, on all our supported platforms without an unreasonable amount of effort.
Quoted 5 lines
So something has to give: either we decide for
an early merge, where the code is marked as experimental and disabled by default. Personally I think it's fine, the contingency plan is simply to "git rm -rfrust/". Or we can keep the above stringent requirements for merging, but then I don't see it as a one-person job.
Patch 3, the high level APIs is where I see most of the work and collaboration being needed, but that doesn't need to be rushed into the first merge. We would have a "rust" subsystem + maintainer who would presumably have a staging tree, etc in the normal way we work and collaborate
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 8 lines · Daniel P. Berrangé
My thought is that the initial merge focuses only on the build system integration. So that's basically patches 1 + 2 in this series.
Patch 3, the high level APIs is where I see most of the work and collaboration being needed, but that doesn't need to be rushed into the first merge. We would have a "rust" subsystem + maintainer who would presumably have a staging tree, etc in the normal way we work and collaborate
It's complicated. A "perfect" build system integration would include integration with Kconfig, but it's not simple work and it may not be Manos's preference for what to work on (or maybe it is), and it's also not a blocker for further work on patches 3-4.
On the other hand, patches 3 and 4 are _almost_ ready except for requiring a very new Rust - we know how to tackle that, but again it may take some time and it's completely separate work from better build system integration.
In other words, improving build system integration is harder until merge, but merge is blocked by independent work on lowering the minimum supported Rust version. This is why I liked the idea of having either a development tree to allow a merge into early 9.2.
On the other hand, given the exceptional scope (completely new code that can be disabled at will) and circumstances, even a very early merge into 9.1 (disabled by default) might be better to provide developers with the easiest base for experimenting. The requirements for merging, here, would basically amount to a good roadmap and some established good habits.
An merge into early 9.2 would be a bit harder for experimenting, while merging it now would sacrifice CI integration in the initial stages of the work but make cooperation easier.
However, it's difficult to say what's best without knowing Manos's rationale for preferring not to have a development tree yet.
Paolo
Quoted 32 lines · Paolo Bonzini
Quoted 8 lines · Daniel P. Berrangé
My thought is that the initial merge focuses only on the build system integration. So that's basically patches 1 + 2 in this series.
Patch 3, the high level APIs is where I see most of the work and collaboration being needed, but that doesn't need to be rushed into the first merge. We would have a "rust" subsystem + maintainer who would presumably have a staging tree, etc in the normal way we work and collaborate
It's complicated. A "perfect" build system integration would include integration with
Kconfig, but it's not simple work and it may not be Manos's preference for what to work on (or maybe it is), and it's also not a blocker for further work on patches 3-4.On the other hand, patches 3 and 4 are _almost_ ready except for requiring a very new Rust - we know how to tackle that, but again it may take some time and it's completely separate work from better build system integration.
In other words, improving build system integration is harder until merge, but merge is blocked by independent work on lowering the minimum supported Rust version. This is why I liked the idea of having either a development tree to allow a merge into early 9.2.
On the other hand, given the exceptional scope (completely new code that can be disabled at will) and circumstances, even a very early merge into 9.1 (disabled by default) might be better to provide developers with the easiest base for experimenting. The requirements for merging, here, would basically amount to a good roadmap and some established good habits.
Quoted 3 lines
An merge into early 9.2 would be a bit harder for experimenting, while merging it now would sacrifice CI integration in the initial stages of the work but make cooperation easier.
I would be in favor of a 9.1 merge (disabled, not auto).
Noting that soft-freeze is two weeks from today, so no dilly dallying. :-)
The only reasonable alternative that I see is the kind of development branch that qemu is not used to having: a long-term shared branch on qemu-projects. With which I would be ok, up to and including a branch merge at the end, as I'm used to working that way with other projects.
I think the only reason to delay the current development work until 9.2 is if we were still questioning whether using Rust *at all* is a good idea. I think we're beyond that point.
r~
Quoted 16 lines · Manos Pitsidianakis
Quoted 12 lines · On Mon, 8 Jul 2024, 21:34 Paolo Bonzini, <[email protected]> wrote:
Il lun 8 lug 2024, 19:12 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 6 lines
That's exactly why I suggest its a pre-requisite for merging this. Unless we're able to demonstrate that we can enable Rust on all our CI platforms, the benefits of Rust will not be realized in QEMU, and we'll have never ending debates about whether each given feature needs to be in C or Rust.
Quoted 8 lines
Quoted 3 lines
In that case we should develop it on a branch, so that more than one person can contribute (unlike if we keep iterating on this RFC).
Paolo
If you do, I'd really appreciate it if you did not use any part of my patches.
Someone pointed out to me that my wording is really bad here and I agree, I apologize. I did not express myself with the right words. What I wanted to say is that hypothetically my work can be used as a base just not the patches/sign offs. The changes in the patches are all open source and part of the QEMU community now, of course.