Implement ARM PL011 in Rust
Subsystems (2)
v2 cover Implement ARM PL011 in Rust
Changes from v1->v2:
- Create bindgen target first, then add commit for device (thanks Pierrick)
- Create a special named generated.rs for each target as compilation would fail if more than one targets were defined. The generated.rs target names would clash.
- Add more descriptive commit messages
- Update MAINTAINERS
- Cleanup patch order for better review, hopefully
Previous series was <[email protected]>
--
Hello everyone,
This is an early draft of my work on implementing a very simple device, in this case the ARM PL011 (which in C code resides in hw/char/pl011.c and is used in hw/arm/virt.c).
The device is functional, with copied logic from the C code but with effort not to make a direct C to Rust translation. In other words, do not write Rust as a C developer would.
That goal is not complete but a best-effort case. To give a specific example, register values are typed but interrupt bit flags are not (but could be). I will leave such minutiae for later iterations.
By the way, the wiki page for Rust was revived to keep track of all current series on the mailing list https://wiki.qemu.org/RustInQemu
a #qemu-rust IRC channel was also created for rust-specific discussion that might flood #qemu
------------------------------------------------------------------------
A request: keep comments to Rust in relation to the QEMU project and no debates on the merits of the language itself. These are valid concerns, but it'd be better if they were on separate mailing list threads.
------------------------------------------------------------------------
Table of contents: [TOC]
- How can I try it? [howcanItryit]
- What are the most important points to focus on, at this point? [whatarethemostimportant]
- What are the issues with not using the compiler, rustc, directly? [whataretheissueswith]
1. Tooling
2. Rust dependencies
- Should QEMU use third-party dependencies? [shouldqemuusethirdparty]
- Should QEMU provide wrapping Rust APIs over QEMU internals? [qemuprovidewrappingrustapis]
- Will QEMU now depend on Rust and thus not build on my XYZ platform? [qemudependonrustnotbuildonxyz]
- How is the compilation structured? [howisthecompilationstructured]
- The generated.rs rust file includes a bunch of junk definitions? [generatedrsincludesjunk]
- The staticlib artifact contains a bunch of mangled .o objects? [staticlibmangledobjects]
How can I try it?
=================
[howcanItryit] Back to [TOC]
Hopefully applying this patches (or checking out master branch from
https://gitlab.com/epilys/rust-for-qemu/ )
Tag for this RFC is rust-pl011-rfc-v2
Rustdoc documentation is hosted on
https://rust-for-qemu-epilys-aebb06ca9f9adfe6584811c14ae44156501d935ba4.gitlab.io/pl011/index.html
If cargo and bindgen is installed in your system, you should be able to build qemu-system-aarch64 with configure flag --enable-rust and launch an arm virt VM. One of the patches hardcodes the default UART of the machine to the Rust one, so if something goes wrong you will see it upon launching qemu-system-aarch64.
To confirm it is there for sure, run e.g. info qom-tree on the monitor and look for x-pl011-rust.
What are the most important points to focus on, at this point?
==============================================================
[whatarethemostimportant] Back to [TOC]
In my opinion, integration of the go-to Rust build system (Cargo and crates.io) with the build system we use in QEMU. This is "easily" done in some definition of the word with a python wrapper script.
What are the issues with not using the compiler, rustc, directly?
-----------------------------------------------------------------
[whataretheissueswith] Back to [TOC]
1. Tooling
Mostly writing up the build-sys tooling to do so. Ideally we'd compile everything without cargo but rustc directly.
If we decide we need Rust's std library support, we could investigate whether building it from scratch is a good solution. This will only build the bits we need in our devices.
2. Rust dependencies
We could go without them completely. I chose deliberately to include one dependency in my UART implementation, bilge[0], because it has an elegant way of representing typed bitfields for the UART's registers.
[0]: Article: https://hecatia-elegua.github.io/blog/no-more-bit-fiddling/
Crates.io page: https://crates.io/crates/bilge
Repository: https://github.com/hecatia-elegua/bilge
Should QEMU use third-party dependencies?
-----------------------------------------
[shouldqemuusethirdparty] Back to [TOC]
In my personal opinion, if we need a dependency we need a strong argument for it. A dependency needs a trusted upstream source, a QEMU maintainer to make sure it us up-to-date in QEMU etc.
We already fetch some projects with meson subprojects, so this is not a new reality. Cargo allows you to define "locked" dependencies which is the same as only fetching specific commits by SHA. No suspicious tarballs, and no disappearing dependencies a la left-pad in npm.
However, I believe it's worth considering vendoring every dependency by default, if they prove to be few, for the sake of having a local QEMU git clone buildable without network access.
Should QEMU provide wrapping Rust APIs over QEMU internals?
-----------------------------------------------------------
[qemuprovidewrappingrustapis] Back to [TOC]
My personal opinion is no, with the reasoning being that QEMU internals are not documented or stable. However I do not see why creating stable opt-in interfaces is bad. It just needs someone to volunteer to maintain it and ensure there are no breakages through versions.
Will QEMU now depend on Rust and thus not build on my XYZ platform?
-------------------------------------------------------------------
[qemudependonrustnotbuildonxyz] Back to [TOC]
No, worry about this in some years if this experiment takes off. Rust has broad platform support and is present in most distro package managers. In the future we might have gcc support for it as well.
For now, Rust will have an experimental status, and will be aimed to those who wish to try it. I leave it to the project leaders to make proper decisions and statements on this if necessary.
How is the compilation structured?
==================================
[howisthecompilationstructured] Back to [TOC]
First, a meson target that runs bindgen on a bunch of header files (defined in rust/wrapper.h) is created as a target and as a dependency for any rust hardware device that needs it. You can see the generated bindings by running
ninja aarch64-softmmu-generated.rs
inside your build directory.
The devices are defined as dictionaries in rust/meson.build because they depend on the bindgen dependency, which is available much later in the meson process (when the static qemu lib and target emulator executables are defined).
A cargo wrapper python script under scripts/ exists to build the crate library, by providing the path to the generated.rs bindings via the environment. Then, the qemu-system-aarch64 binary links against the staticlib archive (i.e. libpl011.a)
The generated.rs rust file includes a bunch of junk definitions?
================================================================
[generatedrsincludesjunk] Back to [TOC]
Yes, bindgen allows you to block certain types and identifiers from being generated but they are simply too many. I have trimmed some of the fat but vast improvements can be made.
The staticlib artifact contains a bunch of mangled .o objects?
==============================================================
[staticlibmangledobjects] Back to [TOC]
Yes, until we compile without the std module library or we compile it manually instead of linking it, we will have some junk in it.
--
Manos Pitsidianakis (5):
build-sys: Add rust feature option
rust: add bindgen step as a meson dependency
rust: add PL011 device model
DO NOT MERGE: add rustdoc build for gitlab pages
DO NOT MERGE: replace TYPE_PL011 with x-pl011-rust in arm virt machine
.gitignore | 2 + .gitlab-ci.d/buildtest.yml | 64 ++-- MAINTAINERS | 13 + configure | 12 + hw/arm/virt.c | 4 + meson.build | 102 ++++++ meson_options.txt | 4 + rust/meson.build | 93 ++++++ rust/pl011/.cargo/config.toml | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 120 +++++++ rust/pl011/Cargo.toml | 66 ++++ rust/pl011/README.md | 42 +++ rust/pl011/build.rs | 44 +++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 95 ++++++ rust/pl011/src/device.rs | 531 ++++++++++++++++++++++++++++++ rust/pl011/src/device_class.rs | 95 ++++++ rust/pl011/src/generated.rs | 5 + rust/pl011/src/lib.rs | 581 +++++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 38 +++ rust/rustfmt.toml | 7 + rust/wrapper.h | 39 +++ scripts/cargo_wrapper.py | 221 +++++++++++++ scripts/meson-buildoptions.sh | 6 + 27 files changed, 2234 insertions(+), 19 deletions(-) create mode 100644 rust/meson.build create mode 100644 rust/pl011/.cargo/config.toml create mode 100644 rust/pl011/.gitignore create mode 100644 rust/pl011/Cargo.lock create mode 100644 rust/pl011/Cargo.toml create mode 100644 rust/pl011/README.md create mode 100644 rust/pl011/build.rs create mode 100644 rust/pl011/deny.toml create mode 100644 rust/pl011/meson.build create mode 120000 rust/pl011/rustfmt.toml create mode 100644 rust/pl011/src/definitions.rs create mode 100644 rust/pl011/src/device.rs create mode 100644 rust/pl011/src/device_class.rs create mode 100644 rust/pl011/src/generated.rs create mode 100644 rust/pl011/src/lib.rs create mode 100644 rust/pl011/src/memory_ops.rs create mode 100644 rust/rustfmt.toml create mode 100644 rust/wrapper.h create mode 100644 scripts/cargo_wrapper.py
base-commit: 01782d6b294f95bcde334386f0aaac593cd28c0d
Signature · 1 line
γαῖα πυρί μιχθήτω
RFC PATCH v2 1/5 build-sys: Add rust feature option
Add options for Rust in meson_options.txt, meson.build, configure to prepare for adding Rust code in the followup commits.
rust is a reserved meson name, so we have to use an alternative. with_rust was chosen.
A cargo_wrapper.py script is added that is heavily based on the work of Marc-André Lureau from 2021.
https://patchew.org/QEMU/[email protected]/
--- .gitignore | 2 + MAINTAINERS | 5 + configure | 12 ++ meson.build | 11 ++ meson_options.txt | 4 + scripts/cargo_wrapper.py | 211 ++++++++++++++++++++++++++++++++++ scripts/meson-buildoptions.sh | 6 + 7 files changed, 251 insertions(+) create mode 100644 scripts/cargo_wrapper.py
.gitignore +2 -0
@@ -2,6 +2,8 @@ /build/ /.cache/ /.vscode/ +/target/ +rust/**/target *.pyc .sdk .stgit-*
MAINTAINERS +5 -0
@@ -4223,6 +4223,11 @@ F: docs/sphinx/ F: docs/_templates/ F: docs/devel/docs.rst +Rust build system integration +M: Manos Pitsidianakis <[email protected]> +S: Maintained +F: scripts/cargo_wrapper.py + Miscellaneous ------------- Performance Tools and Tests
configure +12 -0
@@ -302,6 +302,9 @@ else objcc="${objcc-${cross_prefix}clang}" fi +with_rust="auto" +with_rust_target_triple="" + ar="${AR-${cross_prefix}ar}" as="${AS-${cross_prefix}as}" ccas="${CCAS-$cc}"
@@ -760,6 +763,12 @@ for opt do ;; --gdb=*) gdb_bin="$optarg" ;; + --enable-with-rust) with_rust=enabled + ;; + --disable-with-rust) with_rust=disabled + ;; + --with-rust-target-triple=*) with_rust_target_triple="$optarg" + ;; # everything else has the same name in configure and meson --*) meson_option_parse "$opt" "$optarg" ;;
@@ -1796,6 +1805,9 @@ if test "$skip_meson" = no; then test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE" test "$plugins" = yes && meson_option_add "-Dplugins=true" test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg" + test "$with_rust" != enabled && meson_option_add "-Dwith_rust=$with_rust" + test "$with_rust" != enabled && meson_option_add "-Dwith_rust=$with_rust" + test "$with_rust_target_triple" != "" && meson_option_add "-Dwith_rust_target_triple=$with_rust_target_triple" run_meson() { NINJA=$ninja $meson setup "$@" "$PWD" "$source_path" }
meson.build +11 -0
@@ -290,6 +290,12 @@ foreach lang : all_languages endif endforeach +cargo = not_found +if get_option('with_rust').allowed() + cargo = find_program('cargo', required: get_option('with_rust')) +endif +with_rust = cargo.found() + # default flags for all hosts # We use -fwrapv to tell the compiler that we require a C dialect where # left shift of signed integers is well defined and has the expected
@@ -2066,6 +2072,7 @@ endif config_host_data = configuration_data() +config_host_data.set('CONFIG_WITH_RUST', with_rust) audio_drivers_selected = [] if have_system audio_drivers_available = {
@@ -4190,6 +4197,10 @@ if 'objc' in all_languages else summary_info += {'Objective-C compiler': false} endif +summary_info += {'Rust support': with_rust} +if with_rust and get_option('with_rust_target_triple') != '' + summary_info += {'Rust target': get_option('with_rust_target_triple')} +endif option_cflags = (get_option('debug') ? ['-g'] : []) if get_option('optimization') != 'plain' option_cflags += ['-O' + get_option('optimization')]
meson_options.txt +4 -0
@@ -366,3 +366,7 @@ option('qemu_ga_version', type: 'string', value: '', option('hexagon_idef_parser', type : 'boolean', value : true, description: 'use idef-parser to automatically generate TCG code for the Hexagon frontend') +option('with_rust', type: 'feature', value: 'auto', + description: 'Enable Rust support') +option('with_rust_target_triple', type : 'string', value: '', + description: 'Rust target triple')
scripts/cargo_wrapper.py new file +211 -0
211 lines not on this page · raw message
scripts/meson-buildoptions.sh +6 -0
@@ -79,6 +79,8 @@ meson_options_help() { printf "%s\n" ' auto/sigaltstack/ucontext/windows)' printf "%s\n" ' --with-pkgversion=VALUE use specified string as sub-version of the' printf "%s\n" ' package' + printf "%s\n" ' --with-rust-target-triple=VALUE' + printf "%s\n" ' Specify Rust host target triple' printf "%s\n" ' --with-suffix=VALUE Suffix for QEMU data/modules/config directories' printf "%s\n" ' (can be empty) [qemu]' printf "%s\n" ' --with-trace-file=VALUE Trace file prefix for simple backend [trace]'
@@ -215,6 +217,7 @@ meson_options_help() { printf "%s\n" ' vvfat vvfat image format support' printf "%s\n" ' werror Treat warnings as errors' printf "%s\n" ' whpx WHPX acceleration support' + printf "%s\n" ' with-rust Enable Rust support' printf "%s\n" ' xen Xen backend support' printf "%s\n" ' xen-pci-passthrough' printf "%s\n" ' Xen PCI passthrough support'
@@ -550,6 +553,9 @@ _meson_option_parse() { --disable-werror) printf "%s" -Dwerror=false ;; --enable-whpx) printf "%s" -Dwhpx=enabled ;; --disable-whpx) printf "%s" -Dwhpx=disabled ;; + --enable-with-rust) printf "%s" -Dwith_rust=enabled ;; + --disable-with-rust) printf "%s" -Dwith_rust=disabled ;; + --with-rust-target-triple=*) quote_sh "-Dwith_rust_target_triple=$2" ;; --enable-xen) printf "%s" -Dxen=enabled ;; --disable-xen) printf "%s" -Dxen=disabled ;; --enable-xen-pci-passthrough) printf "%s" -Dxen_pci_passthrough=enabled ;;
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v2 1/5] build-sys: Add rust feature option
Quoted 22 lines · Manos Pitsidianakis
Add options for Rust in meson_options.txt, meson.build, configure to prepare for adding Rust code in the followup commits.
rustis a reserved meson name, so we have to use an alternative.with_rustwas chosen.A
cargo_wrapper.pyscript is added that is heavily based on the work of Marc-André Lureau from 2021.https://patchew.org/QEMU/[email protected]/
Signed-off-by Marc-André Lureau<[email protected]>Signed-off-by Manos Pitsidianakis<[email protected]>--- .gitignore | 2 + MAINTAINERS | 5 + configure | 12 ++ meson.build | 11 ++ meson_options.txt | 4 + scripts/cargo_wrapper.py | 211 ++++++++++++++++++++++++++++++++++ scripts/meson-buildoptions.sh | 6 + 7 files changed, 251 insertions(+)
Quoted 1 line
create mode 100644
scripts/cargo_wrapper.py
[13/5185] Generating rust_pl011_cargo with a custom command
FAILED: libpl011.a/home/rth/chroot-home/qemu/bld-x/pyvenv/bin/python3 /home/rth/chroot-home/qemu/src/scripts/cargo_wrapper.py --config-headers /home/rth/chroot-home/qemu/bld-x/config-host.h --meson-build-root /home/rth/chroot-home/qemu/bld-x --meson-build-dir /home/rth/chroot-home/qemu/bld-x/rust --meson-source-dir /home/rth/chroot-home/qemu/src/rust --color always --crate-dir /home/rth/chroot-home/qemu/src/rust/pl011 --profile release --target-triple x86_64-unknown-linux-gnu --outdir . build-lib /home/rth/chroot-home/qemu/src/scripts/cargo_wrapper.py:14: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
r~
Re: [RFC PATCH v2 1/5] build-sys: Add rust feature option
In reply to Manos PitsidianakisQuoted 16 lines · Manos Pitsidianakis
+++
b/scripts/cargo_wrapper.pyscripts/cargo_wrapper.py new file +14
-0@@ -0,0 +1,211 @@ +#!/usr/bin/env python3 +# Copyright (c) 2020 Red Hat, Inc. +# Copyright (c) 2023 Linaro Ltd. +# +# Authors: +# Manos Pitsidianakis<[email protected]> +# Marc-André Lureau<[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import configparser +import distutils.file_util
scripts/cargo_wrapper.py line 24
Quoted 10 lines
scripts/cargo_wrapper.py new file +10
-0+import json +import logging +import os +import os.path +import re +import subprocess +import sys +import pathlib +import shutil +import tomllib
Correct me if I'm wrong, but does this require python 3.11 for tomllib? AFAIK, we're limited to assuming 3.9 from Debian 11 until Feb 2026, or 3.10 from Ubuntu 22.04 until Apr 2026.
I presume this package can be downloaded from pip, and therefore should be added to the python venv that we create in configure?
r~
Re: [RFC PATCH v2 1/5] build-sys: Add rust feature option
Quoted 32 lines · Richard Henderson
Quoted 16 lines · Manos Pitsidianakis
+++
b/scripts/cargo_wrapper.pyscripts/cargo_wrapper.py new file +14
-0@@ -0,0 +1,211 @@ +#!/usr/bin/env python3 +# Copyright (c) 2020 Red Hat, Inc. +# Copyright (c) 2023 Linaro Ltd. +# +# Authors: +# Manos Pitsidianakis<[email protected]> +# Marc-André Lureau<[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import configparser +import distutils.file_utilscripts/cargo_wrapper.py line 24
Quoted 10 lines
scripts/cargo_wrapper.py new file +10
-0+import json +import logging +import os +import os.path +import re +import subprocess +import sys +import pathlib +import shutil +import tomllibCorrect me if I'm wrong, but does this require python 3.11 for tomllib? AFAIK, we're limited to assuming 3.9 from Debian 11 until Feb 2026, or 3.10 from Ubuntu 22.04 until Apr 2026.
Quoted 4 lines
I presume this package can be downloaded from pip, and therefore should be added to the python venv that we create in configure?
r~
That's absolutely correct. I will make it compatible with at least 3.9 in the next version, thanks!
Manos
RFC PATCH v2 2/5 rust: add bindgen step as a meson dependency
In reply to Manos PitsidianakisAdd mechanism to generate rust hw targets that depend on a custom bindgen target for rust bindings to C.
This way bindings will be created before the rust crate is compiled.
The bindings will end up in BUILDDIR/{target}-generated.rs and have the same name as a target:
ninja aarch64-softmmu-generated.rs
The way the bindings are generated is:
1. All required C headers are included in a single file, in our case rust/wrapper.h for convenience. Otherwise we'd have to provide a list of headers every time to the bindgen tool.
2. Meson creates a generated_rs target that runs bindgen making sure the architecture etc header dependencies are present.
3. The generated_rs target takes a list of files, type symbols, function symbols to block from being generated. This is not necessary for the bindings to work, but saves us time and space.
4. Meson creates rust hardware target dependencies from the rust_targets dictionary defined in rust/meson.build.
Since we cannot declare a dependency on generated_rs before it is declared in meson.build, the rust crate targets must be defined after the generated_rs target for each target architecture is defined. This way meson sets up the dependency tree properly.
5. After compiling each rust crate with the cargo_wrapper.py script, its static library artifact is linked as a whole-archive with the final binary.
--- MAINTAINERS | 2 + meson.build | 87 ++++++++++++++++++++++++++++++++++++++ rust/meson.build | 91 ++++++++++++++++++++++++++++++++++++++++ rust/wrapper.h | 39 +++++++++++++++++ scripts/cargo_wrapper.py | 10 +++++ 5 files changed, 229 insertions(+) create mode 100644 rust/meson.build create mode 100644 rust/wrapper.h
MAINTAINERS +2 -0
@@ -4227,6 +4227,8 @@ Rust build system integration M: Manos Pitsidianakis <[email protected]> S: Maintained F: scripts/cargo_wrapper.py +F: rust/meson.build +F: rust/wrapper.h Miscellaneous -------------
meson.build +87 -0
93 lines not on this page · raw message
rust/meson.build new file +91 -0
91 lines not on this page · raw message
rust/wrapper.h new file +39 -0
@@ -0,0 +1,39 @@ +/* +* QEMU System Emulator +* +* Copyright (c) 2003-2020 Fabrice Bellard +* +* Permission is hereby granted, free of charge, to any person obtaining a copy +* of this software and associated documentation files (the "Software"), to deal +* in the Software without restriction, including without limitation the rights +* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +* copies of the Software, and to permit persons to whom the Software is +* furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +* THE SOFTWARE. +*/ + +#include "qemu/osdep.h" +#include "qemu/module.h" +#include "qemu-io.h" +#include "sysemu/sysemu.h" +#include "hw/sysbus.h" +#include "exec/memory.h" +#include "chardev/char-fe.h" +#include "hw/clock.h" +#include "hw/qdev-clock.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/irq.h" +#include "qapi/error.h" +#include "migration/vmstate.h" +#include "chardev/char-serial.h"
scripts/cargo_wrapper.py +10 -0
@@ -94,6 +94,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str] env = os.environ env["CARGO_ENCODED_RUSTFLAGS"] = cfg + env["MESON_BUILD_DIR"] = str(target_dir) + env["MESON_BUILD_ROOT"] = str(args.meson_build_root) return (env, cargo_cmd)
@@ -164,6 +166,14 @@ def main() -> None: required=True, ) parser.add_argument( + "--meson-build-root", + metavar="BUILD_ROOT", + help="meson.project_build_root()", + type=pathlib.Path, + dest="meson_build_root", + required=True, + ) + parser.add_argument( "--meson-source-dir", metavar="SOURCE_DIR", help="meson.current_source_dir()",
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v2 2/5] rust: add bindgen step as a meson dependency
Just one somewhat larger request, otherwise just a collection of ideas.
Quoted 47 lines · Manos Pitsidianakis
rust/meson.build new file +40
-0@@ -0,0 +1,91 @@ +rust_targets = {} + +cargo_wrapper = [ + find_program(meson.global_source_root() / 'scripts/cargo_wrapper.py'), + '--config-headers', meson.project_build_root() / 'config-host.h', + '--meson-build-root', meson.project_build_root(), + '--meson-build-dir', meson.current_build_dir(), + '--meson-source-dir', meson.current_source_dir(), +] + +# TODO: verify rust_target_triple if given as an option +if rust_target_triple == '' + if not supported_oses.contains(host_os) + message() + error('QEMU does not support `' + host_os +'` as a Rust platform.') + elif not supported_cpus.contains(host_arch) + message() + error('QEMU does not support `' + host_arch +'` as a Rust architecture.') + endif + rust_target_triple = host_arch + rust_supported_oses[host_os] + if host_os == 'windows' and host_arch == 'aarch64' + rust_target_triple += 'llvm' + endif +endif + +if get_option('optimization') in ['0', '1', 'g'] + rs_build_type = 'debug' +else + rs_build_type = 'release' +endif + +rust_hw_target_list = {} + +foreach rust_hw_target, rust_hws: rust_hw_target_list + foreach rust_hw_dev: rust_hws + output = meson.current_build_dir() / rust_target_triple / rs_build_type / rust_hw_dev['output'] + crate_metadata = { + 'name': rust_hw_dev['name'], + 'output': [rust_hw_dev['output']], + 'output-path': output,
rust/meson.build line 86
Quoted 10 lines
rust/meson.build new file +10
-0+ 'command': [cargo_wrapper, + '--crate-dir', + meson.current_source_dir() / rust_hw_dev['dirname'], + '--profile', + rs_build_type, + '--target-triple', + rust_target_triple, + '--outdir', + '@OUTDIR@', + 'build-lib'
Probably needs to add config-devices.h as well to --config-headers? I think that we can have just one crate that is built per target, instead of many small crates like your rust_pl011_cargo. And then that crate is built many times with a rust/src/ hierarchy that resembles the meson.build files we use for C.
rust/src/mod.rs:
pub mod hw;
rust/src/hw/mod.rs:
pub mod char;
rust/src/hw/char/mod.rs:
#[cfg(feature = "CONFIG_PL011")] pub mod pl011;
(perhaps just src/? And we could even move all sources down one level to src/ for QEMU as well? But that can be done later, for now it can be rust/src/). This basically gives Kconfig integration for free if it works, so I'd ask you to try doing this for the next version?
Also, sooner or later we'll have to tackle building a set of common dependencies (similar to common_ss), and then bringing those as a dependency to the build of the per-target crates. Even if we don't get to the point of building devices once for all targets, I'd rather at least avoid having to build a dozen times the common deps of procedural macros.
This is not trivial, but should not be hard either, by using build.rs (cargo::rustc-link-search) to add the -Ldependency=/path/to/rlibs option to the per-target rustc invocations. This may also require grabbing the compilation log via "cargo build --message-format=json-render-diagnostics", but that's not hard to do in cargo_wrapper.py. And unlike the previous request about Kconfig, it can be done after the first merge.
Quoted 5 lines
rust/meson.build new file +5
-0+#include "qemu/osdep.h" +#include "qemu/module.h" +#include "qemu-io.h" +#include "sysemu/sysemu.h" +#include "hw/sysbus.h"
rust/meson.build
Quoted 10 lines
rust/meson.build new file +10
-0+#include "exec/memory.h" +#include "chardev/char-fe.h" +#include "hw/clock.h" +#include "hw/qdev-clock.h" +#include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" +#include "hw/irq.h" +#include "qapi/error.h" +#include "migration/vmstate.h" +#include "chardev/char-serial.h"
Please check for any headers included indirectly and whether it's worth including them here (e.g. it seems that qapi/error.h, qom/object.h, hw/qdev-core.h should be there).
Quoted 18 lines
scripts/cargo_wrapper.py +3
-0@@ -94,6 +94,8 @@ def get_cargo_rustc(args: argparse.Namespace) -> tuple[Dict[str, Any], List[str] env = os.environ env["CARGO_ENCODED_RUSTFLAGS"] = cfg + env["MESON_BUILD_DIR"] = str(target_dir) + env["MESON_BUILD_ROOT"] = str(args.meson_build_root) return (env, cargo_cmd)@@ -164,6 +166,14 @@ def main() -> None: required=True, ) parser.add_argument( + "--meson-build-root",
scripts/cargo_wrapper.py def main() -> None: · line 179
Quoted 10 lines
scripts/cargo_wrapper.py +7
-0+ metavar="BUILD_ROOT", + help="meson.project_build_root()", + type=pathlib.Path, + dest="meson_build_root", + required=True, + ) + parser.add_argument( "--meson-source-dir", metavar="SOURCE_DIR", help="meson.current_source_dir()",
Probably all of cargo_wrapper.py can be moved to this patch.
Paolo
RFC PATCH v2 3/5 rust: add PL011 device model
In reply to Manos PitsidianakisThis commit adds a re-implementation of hw/char/pl011.c in Rust.
It uses generated Rust bindings (produced by ninja aarch64-softmmu-generated.rs) to
register itself as a QOM type/class.
How to build:
1. Make sure rust, cargo and bindgen (cargo install bindgen-cli) are installed
2. Configure a QEMU build with:
--enable-system --target-list=aarch64-softmmu --enable-with-rust
3. Launching a VM with qemu-system-aarch64 should use the Rust version of the pl011 device (unless it is not set up so in hw/arm/virt.c; the type of the UART device is hardcoded).
To confirm, inspect info qom-tree in the monitor and look for an x-pl011-rust device.
--- MAINTAINERS | 6 + meson.build | 4 + rust/meson.build | 2 + rust/pl011/.cargo/config.toml | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 120 +++++++ rust/pl011/Cargo.toml | 66 ++++ rust/pl011/README.md | 42 +++ rust/pl011/build.rs | 44 +++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 95 ++++++ rust/pl011/src/device.rs | 531 ++++++++++++++++++++++++++++++ rust/pl011/src/device_class.rs | 95 ++++++ rust/pl011/src/generated.rs | 5 + rust/pl011/src/lib.rs | 581 +++++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 38 +++ rust/rustfmt.toml | 7 + 19 files changed, 1705 insertions(+) create mode 100644 rust/pl011/.cargo/config.toml create mode 100644 rust/pl011/.gitignore create mode 100644 rust/pl011/Cargo.lock create mode 100644 rust/pl011/Cargo.toml create mode 100644 rust/pl011/README.md create mode 100644 rust/pl011/build.rs create mode 100644 rust/pl011/deny.toml create mode 100644 rust/pl011/meson.build create mode 120000 rust/pl011/rustfmt.toml create mode 100644 rust/pl011/src/definitions.rs create mode 100644 rust/pl011/src/device.rs create mode 100644 rust/pl011/src/device_class.rs create mode 100644 rust/pl011/src/generated.rs create mode 100644 rust/pl011/src/lib.rs create mode 100644 rust/pl011/src/memory_ops.rs create mode 100644 rust/rustfmt.toml
MAINTAINERS +6 -0
@@ -1181,6 +1181,11 @@ F: include/hw/*/microbit*.h F: tests/qtest/microbit-test.c F: docs/system/arm/nrf.rst +ARM PL011 Rust device +M: Manos Pitsidianakis <[email protected]> +S: Maintained +F: rust/pl011/ + AVR Machines -------------
@@ -4229,6 +4234,7 @@ S: Maintained F: scripts/cargo_wrapper.py F: rust/meson.build F: rust/wrapper.h +F: rust/rustfmt.toml Miscellaneous -------------
meson.build +4 -0
@@ -296,6 +296,10 @@ if get_option('with_rust').allowed() endif with_rust = cargo.found() +if with_rust + subdir('rust') +endif + # default flags for all hosts # We use -fwrapv to tell the compiler that we require a C dialect where # left shift of signed integers is well defined and has the expected
rust/meson.build +2 -0
@@ -67,6 +67,8 @@ endif rust_hw_target_list = {} +subdir('pl011') + foreach rust_hw_target, rust_hws: rust_hw_target_list foreach rust_hw_dev: rust_hws output = meson.current_build_dir() / rust_target_triple / rs_build_type / rust_hw_dev['output']
rust/pl011/.cargo/config.toml new file +2 -0
@@ -0,0 +1,2 @@ +[build] +rustflags = ["-Crelocation-model=pic", "-Ctarget-feature=+crt-static"]
rust/pl011/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +target +src/generated.rs.inc
rust/pl011/Cargo.lock new file +120 -0
120 lines not on this page · raw message
rust/pl011/Cargo.toml new file +66 -0
66 lines not on this page · raw message
rust/pl011/README.md new file +42 -0
42 lines not on this page · raw message
rust/pl011/build.rs new file +44 -0
44 lines not on this page · raw message
rust/pl011/deny.toml new file +57 -0
57 lines not on this page · raw message
rust/pl011/meson.build new file +7 -0
@@ -0,0 +1,7 @@ +rust_pl011 = { + 'name': 'rust_pl011_cargo', + 'dirname': 'pl011', + 'output': 'libpl011.a', + } + +rust_hw_target_list += {'aarch64-softmmu': [rust_pl011]}
rust/pl011/rustfmt.toml new file +1 -0
@@ -0,0 +1 @@ +../rustfmt.toml
rust/pl011/src/definitions.rs new file +95 -0
95 lines not on this page · raw message
rust/pl011/src/device.rs new file +531 -0
531 lines not on this page · raw message
rust/pl011/src/device_class.rs new file +95 -0
95 lines not on this page · raw message
rust/pl011/src/generated.rs new file +5 -0
@@ -0,0 +1,5 @@ +#[cfg( MESON_GENERATED_RS )] +include! (concat! (env! ("OUT_DIR" ),"/generated.rs" )); + +#[cfg(not( MESON_GENERATED_RS ))] +include! ("generated.rs.inc" );
rust/pl011/src/lib.rs new file +581 -0
581 lines not on this page · raw message
rust/pl011/src/memory_ops.rs new file +38 -0
@@ -0,0 +1,38 @@ +use core::{mem::MaybeUninit , ptr::NonNull }; + +use crate ::{device::PL011State , generated::*}; + +pub const PL011_OPS :MemoryRegionOps =MemoryRegionOps { + read:Some (pl011_read), + write:Some (pl011_write), + read_with_attrs:None , + write_with_attrs:None , + endianness: device_endian_DEVICE_NATIVE_ENDIAN, + valid:unsafe {MaybeUninit ::<MemoryRegionOps__bindgen_ty_1 >::zeroed ().assume_init () }, + impl_:MemoryRegionOps__bindgen_ty_2 { + min_access_size:4 , + max_access_size:4 , + ..unsafe {MaybeUninit ::<MemoryRegionOps__bindgen_ty_2 >::zeroed ().assume_init () } + }, +}; + +unsafe extern "C" fn pl011_read ( + opaque: *mut core::ffi::c_void , + addr:hwaddr , + size: core::ffi::c_uint , +) ->u64 { +assert! (!opaque.is_null ()); +let mut state =NonNull ::new_unchecked (opaque.cast ::<PL011State >()); + state.as_mut ().read (addr, size) +} + +unsafe extern "C" fn pl011_write ( + opaque: *mut core::ffi::c_void , + addr:hwaddr , + data:u64 , + _size: core::ffi::c_uint , +) { +assert! (!opaque.is_null ()); +let mut state =NonNull ::new_unchecked (opaque.cast ::<PL011State >()); + state.as_mut ().write (addr, data) +}
rust/rustfmt.toml new file +7 -0
@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = true
Signature · 1 line
γαῖα πυρί μιχθήτω
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
I think this is extremely useful to show where we could go in the task of creating more idiomatic bindings.
rust/pl011/build.rs line 6
Quoted 4 lines · Manos Pitsidianakis
rust/pl011/build.rs new file +4
-0+fn main () { +println! ("cargo::rerun-if-env-changed=MESON_BUILD_DIR" ); +println! ("cargo::rerun-if-env-changed=MESON_BUILD_ROOT" ); +println! ("cargo::rerun-if-changed=src/generated.rs.inc" );
Why do you need this rerun-if-changed?
Quoted 5 lines
rust/pl011/src/definitions.rs new file +5
-0+pub const PL011_ARM_INFO :TypeInfo =TypeInfo { + name:TYPE_PL011 .as_ptr (), + parent:TYPE_SYS_BUS_DEVICE .as_ptr (), + instance_size: core::mem::size_of ::<PL011State >(), + instance_align: core::mem::align_of ::<PL011State >(),
rust/pl011/src/definitions.rs line 23
Quoted 10 lines
rust/pl011/src/definitions.rs new file +10
-0+ instance_init: Some(pl011_init), + instance_post_init: None, + instance_finalize: None, + abstract_: false, + class_size: 0, + class_init: Some(pl011_class_init), + class_base_init: None, + class_data: core::ptr::null_mut(), + interfaces: core::ptr::null_mut(), +};
QOM is certainly a major part of what we need to do for idiomatic bindings. This series includes both using QOM objects (chardev) and defining them.
For using QOM objects, there is only one strategy that we need to implement for both Chardev and DeviceState/SysBusDevice which is nice. We can probably take inspiration from glib-rs, see for example
- https://docs.rs/glib/latest/glib/object/trait.Cast.html
- https://docs.rs/glib/latest/glib/object/trait.ObjectType.html
- https://docs.rs/glib/latest/glib/object/struct.ObjectRef.html
For definining new classes I think it's okay if Rust does not support writing superclasses yet, only leaves.
I would make a QOM class written in Rust a struct that only contains the new fields. The struct must implement Default and possibly Drop (for finalize).
struct PL011_Inner {
iomem: MemoryRegion,
readbuff: u32.
...
}and then a macro defines a wrapper struct that includes just two fields, one for the superclass and one for the Rust struct. instance_init can initialize the latter with Default::default().
struct PL011 {
parent_obj: qemu::bindings::SysBusDevice,
private: PL011_Inner,
}"private" probably should be RefCell<PL011_Inner>, avoiding the unsafe
state.as_mut().read(addr, size)
There should also be macros to define the wrappers for MMIO MemoryRegions.
rust/pl011/src/device.rs line 356
Quoted 13 lines
rust/pl011/src/device.rs new file +13
-0+pub fn realize (&mut self) { +unsafe { +qemu_chr_fe_set_handlers ( +addr_of_mut! (self.char_backend), +Some (pl011_can_receive), +Some (pl011_receive), +Some (pl011_event), +None , +addr_of_mut! (*self).cast ::<c_void >(), + core::ptr::null_mut (), +true , + ); + }
Probably each set of callbacks (MemoryRegion, Chardev) needs to be a struct that implements a trait.
Quoted 7 lines
rust/pl011/src/device_class.rs new file +7
-0+#[macro_export] +macro_rules! define_property { + ($name:expr, $state:ty, $field:expr, $prop:expr, $type:expr,default = $defval:expr) => { + $crate::generated::Property { + name: $name, + info: $prop, + offset: ::core::mem::offset_of!($state, $field)as _ ,
rust/pl011/src/device_class.rs line 38
Quoted 10 lines
rust/pl011/src/device_class.rs new file +10
-0+ bitnr: 0, + bitmask: 0, + set_default: true, + defval: $crate::generated::Property__bindgen_ty_1 { u: $defval.into() }, + arrayoffset: 0, + arrayinfo: ::core::ptr::null(), + arrayfieldsize: 0, + link_type: ::core::ptr::null(), + } + };
Perhaps we can define macros similar to the C DEFINE_PROP_*, and const functions can be used to enforce some kind of type safety.
pub const fn check_type<T>(_x: &T, y: T) -> T { y }
static VAR: i16 = 42i16;
static TEST: i8 = check_type(&VAR, 43i8);
pub fn main() { println!("{}", TEST); }
error[E0308]: mismatched types
--> ff.rs:4:30
| 4 | static TEST: i8 = check_type(&VAR, 43i8); | ---------- ^^^^ expected `&i8`, found `&i16` | | | arguments to this function are incorrect |
= note: expected reference &i8
found reference &i16
Anyhow I think this is already very useful, because as the abstractions are developed, it is possible to see how the device code evolves.
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 11 lines · Paolo Bonzini
I think this is extremely useful to show where we could go in the task of creating more idiomatic bindings.
rust/pl011/build.rs line 6
Quoted 4 lines · Manos Pitsidianakis
rust/pl011/build.rs new file +4
-0+fn main () { +println! ("cargo::rerun-if-env-changed=MESON_BUILD_DIR" ); +println! ("cargo::rerun-if-env-changed=MESON_BUILD_ROOT" ); +println! ("cargo::rerun-if-changed=src/generated.rs.inc" );Why do you need this rerun-if-changed?
To show an error from build.rs in case the file is deleted and the build is not via meson
Quoted 21 lines
Quoted 6 lines
rust/pl011/src/definitions.rs new file +5
-0+pub const PL011_ARM_INFO :TypeInfo =TypeInfo { + name:TYPE_PL011 .as_ptr (), + parent:TYPE_SYS_BUS_DEVICE .as_ptr (), + instance_size: core::mem::size_of ::<PL011State >(), + instance_align: core::mem::align_of ::<PL011State >(),rust/pl011/src/definitions.rs line 23
Quoted 10 lines
rust/pl011/src/definitions.rs new file +10
-0+ instance_init: Some(pl011_init), + instance_post_init: None, + instance_finalize: None, + abstract_: false, + class_size: 0, + class_init: Some(pl011_class_init), + class_base_init: None, + class_data: core::ptr::null_mut(), + interfaces: core::ptr::null_mut(), +};QOM is certainly a major part of what we need to do for idiomatic bindings. This series includes both using QOM objects (chardev) and defining them.
Quoted 6 lines
For using QOM objects, there is only one strategy that we need to implement for both Chardev and DeviceState/SysBusDevice which is nice. We can probably take inspiration from glib-rs, see for example
- https://docs.rs/glib/latest/glib/object/trait.Cast.html
- https://docs.rs/glib/latest/glib/object/trait.ObjectType.html
- https://docs.rs/glib/latest/glib/object/struct.ObjectRef.html
There was consensus in the community call that we won't be writing Rust APIs for internal C QEMU interfaces; or at least, that's not the goal
It's not necessary to make bindings to write idiomatic Rust. If you notice, most callbacks QEMU calls cast the pointer into the Rust struct which then calls its idiomatic type methods. I like abstraction a lot, but it has diminishing returns. We'll see how future Rust devices look like and we can then decide if extra code for abstractions is a good trade-off.
Quoted 7 lines
For definining new classes I think it's okay if Rust does not support writing superclasses yet, only leaves.
I would make a QOM class written in Rust a struct that only contains the new fields. The struct must implement Default and possibly Drop (for finalize).
The object is allocated and freed from C, hence it is not Dropped. We're only ever accessing it from a reference retrieved from a QEMU provided raw pointer. If the struct gains heap object fields like Box or Vec or String, they'd have to be dropped manually on _unrealize.
Quoted 7 lines
struct PL011_Inner { iomem: MemoryRegion, readbuff: u32. ...}
Quoted 8 lines
and then a macro defines a wrapper struct that includes just two fields, one for the superclass and one for the Rust struct.
instance_initcan initialize the latter withDefault::default().struct PL011 { parent_obj: qemu::bindings::SysBusDevice, private: PL011_Inner,}
a nested struct is not necessary for using the Default trait. Consider a Default impl that sets parent_obj as a field memset to zero; on instance initialization we can do
*self = Self {
parent_obj: self.parent_obj,
..Self::default(),
};
Quoted 3 lines
"private" probably should be RefCell<PL011_Inner>, avoiding the unsafe
state.as_mut().read(addr, size)
RefCell etc are not FFI safe. Also, nested fields must be visible so that the offset_of! macro works, for QOM properties. Finally,
state.as_mut().read(addr, size)
Is safe since we receive a valid pointer from QEMU. This fact cannot be derived by the compiler, which is why it has an unsafe keyword. That does not mean that the use here is unsafe.
Quoted 2 lines
There should also be macros to define the wrappers for MMIO MemoryRegions.
Do you mean the MemoryRegionOps?
Quoted 29 lines
Quoted 4 lines
rust/pl011/src/device.rs new file +3
-0+pub fn realize (&mut self) { +unsafe { + qemu_chr_fe_set_handlers(rust/pl011/src/device.rs line 356
Quoted 10 lines
rust/pl011/src/device.rs new file +10
-0+ addr_of_mut!(self.char_backend), +Some (pl011_can_receive), +Some (pl011_receive), +Some (pl011_event), +None , +addr_of_mut! (*self).cast ::<c_void >(), + core::ptr::null_mut (), +true , + ); + }Probably each set of callbacks (MemoryRegion, Chardev) needs to be a struct that implements a trait.
Quoted 11 lines
rust/pl011/src/device_class.rs new file +11
-0+#[macro_export] +macro_rules! define_property { + ($name:expr, $state:ty, $field:expr, $prop:expr, $type:expr,default = $defval:expr) => { + $crate::generated::Property { + name: $name, + info: $prop, + offset: ::core::mem::offset_of!($state, $field)as _ , + bitnr:0 , + bitmask:0 , + set_default:true , + defval: $crate::generated::Property__bindgen_ty_1 { u: $defval.into () },
Quoted 8 lines
rust/pl011/src/device_class.rs line 38
Quoted 6 lines
rust/pl011/src/device_class.rs new file +6
-0+ arrayoffset: 0, + arrayinfo: ::core::ptr::null(), + arrayfieldsize: 0, + link_type: ::core::ptr::null(), + } + };Perhaps we can define macros similar to the C DEFINE_PROP_*,
Hopefully if I end up doing something like this, it'd be in a standalone crate for other devices to use
Quoted 15 lines
and const
functions can be used to enforce some kind of type safety.pub const fn check_type<T>(_x: &T, y: T) -> T { y }
static VAR: i16 = 42i16;
static TEST: i8 =check_type(&VAR, 43i8);pub fn
main(){ println!("{}", TEST); }error[E0308]: mismatched types
--> ff.rs:4:30
|
4 | static TEST: i8 =check_type(&VAR, 43i8);
Quoted 6 lines
| ---------- ^^^^ expected `&i8`, found `&i16` | | | arguments to this function are incorrect |= note: expected reference
&i8
found reference&i16
Yes this kind of type safety trick is easy to do (already done for a register macro in src/lib.rs).
I wanted to focus on the build system integration for the first RFC which is why there are some macros but not in every place it makes sense.
Thanks Paolo,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 2 lines · Manos Pitsidianakis
There was consensus in the community call that we won't be writing Rust APIs for internal C QEMU interfaces; or at least, that's not the goal
I disagree with that. We need _some_ kind of bindings, otherwise we have too much unsafe code, and the benefit of Rust becomes so much lower that I doubt the utility.
If something is used by only one device then fine, but when some kind of unsafe code repeats across most if not all devices, that is a problem. It can be macros, it can be smart pointers, that remains to be seen---but repetition should be a warning signal that _something_ is necessary.
Quoted 11 lines
Quoted 6 lines
For definining new classes I think it's okay if Rust does not support writing superclasses yet, only leaves.
I would make a QOM class written in Rust a struct that only contains the new fields. The struct must implement Default and possibly Drop (for finalize).
The object is allocated and freed from C, hence it is not Dropped. We're only ever accessing it from a reference retrieved from a QEMU provided raw pointer. If the struct gains heap object fields like Box or Vec or String, they'd have to be dropped manually on
_unrealize.
That's my point, if you have
struct MyDevice_Inner {
data: Vec<u8>,
}
struct MyDevice {
parent_obj: qemu::bindings::SysBusDevice,
private: ManuallyDrop<PL011_Inner>,
}then the instance_finalize method can simply do
pub instance_finalize(self: *c_void)
{
let dev = self as *mut MyDevice;
unsafe { ManuallyDrop::drop(dev.private) }
}Don't do it on _unrealize, create macros that do it for you.
Quoted 10 lines
Quoted 8 lines
and then a macro defines a wrapper struct that includes just two fields, one for the superclass and one for the Rust struct.
instance_initcan initialize the latter withDefault::default().struct PL011 { parent_obj: qemu::bindings::SysBusDevice, private: PL011_Inner,}
a nested struct is not necessary for using the Default trait
Agreed, but a nested struct is nice anyway in my opinion as a boundary between the C-ish and Rust idiomatic code.
Quoted 6 lines
Quoted 3 lines
"private" probably should be RefCell<PL011_Inner>, avoiding the unsafe
state.as_mut().read(addr, size)RefCell etc are not FFI safe.
Why does it matter? Everything after the SysBusDevice is private.
Quoted 1 line
Also, nested fields must be visible so that the
offset_of! macro works, for QOM properties.
Note that QOM properties do not use offset_of; qdev properties do. Using qdev properties is much easier because they hide visitors, but again - not necessary, sometimes going lower-level can be easier if the API you wrap is less C-ish.
Also, you can define constants (including properties) in contexts where non-public fields are visible:
use std::mem;
pub struct Foo {
_x: i32,
y: i32,
}
impl Foo {
pub const OFFSET_Y: usize = mem::offset_of!(Foo, y);
}
fn main() {
println!("{}", Foo::OFFSET_Y);
}Any offset needed to go past the SysBusDevice and any other fields before MyDevice_Inner can be added via macros. Also note that it doesn't _have_ to be RefCell; RefCell isn't particularly magic. We can implement our own interior mutability thingy that is more friendly to qdev properties, or that includes the ManuallyDrop<> thing from above, or both.
For example you could have
type PL011 = QOMImpl<qemu::bindings::SysBusDevice, PL011_Inner>;
and all the magic (for example Borrow<PL011_Inner>, the TypeInfo, the instance_init and instance_finalize function) would be in QOMImpl.
My point is: let's not focus on having a C-like API. It's the easiest thing to do but not the target.
Quoted 7 lines
Finally,
state.as_mut().read(addr, size)Is safe since we receive a valid pointer from QEMU. This fact cannot be derived by the compiler, which is why it has an
unsafekeyword. That does not mean that the use here is unsafe.
Yes, it is safe otherwise it would be undefined behavior, but there are no checks of the kind that you have in Rust whenever you have &mut.
state.as_mut() implies that no other references to state are in use; but there are (you pass it as the opaque value to both the MemoryRegionOps and the chardev frontend callbacks). This is why I think something like RefCell is needed to go from a shared reference to an exclusive one (interior mutability).
Quoted 3 lines
Quoted 1 line
There should also be macros to define the wrappers for MMIO MemoryRegions.
Do you mean the MemoryRegionOps?
Yes.
Quoted 3 lines
I wanted to focus on the build system integration for the first RFC which is why there are some macros but not in every place it makes sense.
Yes, absolutely. We need to start somewhere.
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Manos PitsidianakisQuoted 34 lines · Manos Pitsidianakis
Quoted 25 lines · Paolo Bonzini
I think this is extremely useful to show where we could go in the task of creating more idiomatic bindings.
Quoted 5 lines · Manos Pitsidianakis
rust/pl011/src/definitions.rs new file +5
-0+pub const PL011_ARM_INFO :TypeInfo =TypeInfo { + name:TYPE_PL011 .as_ptr (), + parent:TYPE_SYS_BUS_DEVICE .as_ptr (), + instance_size: core::mem::size_of ::<PL011State >(), + instance_align: core::mem::align_of ::<PL011State >(),rust/pl011/src/definitions.rs line 23
Quoted 10 lines
rust/pl011/src/definitions.rs new file +10
-0+ instance_init: Some(pl011_init), + instance_post_init: None, + instance_finalize: None, + abstract_: false, + class_size: 0, + class_init: Some(pl011_class_init), + class_base_init: None, + class_data: core::ptr::null_mut(), + interfaces: core::ptr::null_mut(), +};QOM is certainly a major part of what we need to do for idiomatic bindings. This series includes both using QOM objects (chardev) and defining them.
Quoted 6 lines
For using QOM objects, there is only one strategy that we need to implement for both Chardev and DeviceState/SysBusDevice which is nice. We can probably take inspiration from glib-rs, see for example
- https://docs.rs/glib/latest/glib/object/trait.Cast.html
- https://docs.rs/glib/latest/glib/object/trait.ObjectType.html
- https://docs.rs/glib/latest/glib/object/struct.ObjectRef.html
Quoted 2 lines
There was consensus in the community call that we won't be writing Rust APIs for internal C QEMU interfaces; or at least, that's not the goal
I think that is over-stating things. This was only mentioned in passing and my feeling was that we didn't have that detailed of a discussion because at this stage such a discussion is a bit like putting the cart before the horse.
While the initial focus might be to just consume a Rust API that is a 1:1 mapping of the C API, I expect that over time we'll end up writing various higher level Rust wrappers around the C API. If we didn't do that, then in effect we'd be making ourselves write psuedo-C code in Rust, undermining many of the benefits we could get.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 43 lines · Daniel P. Berrangé
Quoted 34 lines · Manos Pitsidianakis
Quoted 25 lines · Paolo Bonzini
I think this is extremely useful to show where we could go in the task of creating more idiomatic bindings.
Quoted 5 lines · Manos Pitsidianakis
rust/pl011/src/definitions.rs new file +5
-0+pub const PL011_ARM_INFO :TypeInfo =TypeInfo { + name:TYPE_PL011 .as_ptr (), + parent:TYPE_SYS_BUS_DEVICE .as_ptr (), + instance_size: core::mem::size_of ::<PL011State >(), + instance_align: core::mem::align_of ::<PL011State >(),rust/pl011/src/definitions.rs line 23
Quoted 10 lines
rust/pl011/src/definitions.rs new file +10
-0+ instance_init: Some(pl011_init), + instance_post_init: None, + instance_finalize: None, + abstract_: false, + class_size: 0, + class_init: Some(pl011_class_init), + class_base_init: None, + class_data: core::ptr::null_mut(), + interfaces: core::ptr::null_mut(), +};QOM is certainly a major part of what we need to do for idiomatic bindings. This series includes both using QOM objects (chardev) and defining them.
Quoted 6 lines
For using QOM objects, there is only one strategy that we need to implement for both Chardev and DeviceState/SysBusDevice which is nice. We can probably take inspiration from glib-rs, see for example
- https://docs.rs/glib/latest/glib/object/trait.Cast.html
- https://docs.rs/glib/latest/glib/object/trait.ObjectType.html
- https://docs.rs/glib/latest/glib/object/struct.ObjectRef.htmlQuoted 2 lines
There was consensus in the community call that we won't be writing Rust APIs for internal C QEMU interfaces; or at least, that's not the goal
I think that is over-stating things. This was only mentioned in passing and my feeling was that we didn't have that detailed of a discussion because at this stage such a discussion is a bit like putting the cart before the horse.
Quoted 6 lines
While the initial focus might be to just consume a Rust API that is a 1:1 mapping of the C API, I expect that over time we'll end up writing various higher level Rust wrappers around the C API. If we didn't do that, then in effect we'd be making ourselves write psuedo-C code in Rust, undermining many of the benefits we could get.
In any case, it is out of scope for this RFC. Introducing wrappers would be a gradual process.
Thanks,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis <
[email protected]> ha scritto:
Quoted 3 lines
In any case, it is out of scope for this RFC. Introducing wrappers would be a gradual process.
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
Paolo
Quoted 4 lines
Thanks,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Good morning Paolo,
Quoted 6 lines · Paolo Bonzini
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis < [email protected]> ha scritto:
Quoted 2 lines
In any case, it is out of scope for this RFC. Introducing wrappers would be a gradual process.
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
Hm the only thing that worries me is keeping it synced and postponing merge indefinitely. If we declare the rust parts as "experimental" we could evolve them quickly even on master. What do the other maintainers think?
Thanks,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Paolo BonziniQuoted 9 lines · Paolo Bonzini
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 3 lines
In any case, it is out of scope for this RFC. Introducing wrappers would be a gradual process.
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
IMHO any higher level binding APIs for Rust should be acceptable in the main QEMU tree as soon as we accept Rust functionality. They can evolve in-tree based on the needs of whomever is creating and/or consuming them.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Il gio 13 giu 2024, 09:13 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 9 lines · Paolo Bonzini
Quoted 4 lines
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 1 line
In any case, it is out of scope for this RFC. Introducing wrappers
would
Quoted 3 lines
Quoted 2 lines
be a gradual process.
Quoted 7 lines
Quoted 2 lines
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
IMHO any higher level binding APIs for Rust should be acceptable in the main QEMU tree as soon as we accept Rust functionality. They can evolve in-tree based on the needs of whomever is creating and/or consuming them.
My question is the opposite, should we accept Rust functionality without proper high level bindings? I am afraid that, if more Rust devices are contributed, it becomes technical debt to have a mix of idiomatic and C-ish code. If the answer is no, then this PL011 device has to be developed out of tree.
Paolo
Quoted 12 lines
With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 20 lines · Paolo Bonzini
Il gio 13 giu 2024, 09:13 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 9 lines · Paolo Bonzini
Quoted 4 lines
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 1 line
In any case, it is out of scope for this RFC. Introducing wrappers
would
Quoted 3 lines
Quoted 2 lines
be a gradual process.
Quoted 7 lines
Quoted 2 lines
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
IMHO any higher level binding APIs for Rust should be acceptable in the main QEMU tree as soon as we accept Rust functionality. They can evolve in-tree based on the needs of whomever is creating and/or consuming them.
Quoted 7 lines
My question is the opposite, should we accept Rust functionality without proper high level bindings? I am afraid that, if more Rust devices are contributed, it becomes technical debt to have a mix of idiomatic and C-ish code. If the answer is no, then this PL011 device has to be developed out of tree.
Paolo
Getting Rust into QEMU, at least for our team at Linaro, is a long term commitment, so we will be responsible for preventing and fixing technical debt. And it will be up to the hypothetical rust maintainers as well to "keep the garden tidy" so to speak.
To put it another way, I personally plan on making sure any bindings and any QEMU-ffi idioms that arise are all homogeneous and don't end up being a burden for the code base.
Your concern is valid, and thank you for raising it. I feel it is important to figure out how this will be managed, since it's also an argument for the final say in whether any of this code ends up in the upstream tree.
Thanks Paolo,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Paolo BonziniQuoted 20 lines · Paolo Bonzini
Il gio 13 giu 2024, 09:13 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 9 lines · Paolo Bonzini
Quoted 4 lines
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 1 line
In any case, it is out of scope for this RFC. Introducing wrappers
would
Quoted 3 lines
Quoted 2 lines
be a gradual process.
Quoted 7 lines
Quoted 2 lines
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
IMHO any higher level binding APIs for Rust should be acceptable in the main QEMU tree as soon as we accept Rust functionality. They can evolve in-tree based on the needs of whomever is creating and/or consuming them.
Quoted 5 lines
My question is the opposite, should we accept Rust functionality without proper high level bindings? I am afraid that, if more Rust devices are contributed, it becomes technical debt to have a mix of idiomatic and C-ish code. If the answer is no, then this PL011 device has to be developed out of tree.
I guess there's a balance to be had somewhere on the spectrum between doing everything against the raw C binding, vs everything against a perfectly idiomatic Rust API wrapping the C bniding. The latter might be the ideal, but from a pragmmatic POV I doubt we want the barrier to entry to be that high.
Is this not something we can figure out organically as part of the code design and review processes ?
e.g. if during review we see a device impl doing something where a higher level API would have unambiguous benefits, and creatino of such a higher level API is a practically achieveable task, then ask for it. If a higher level API is desirable, but possibly not practical, then raise it as an potential idea, but be willing to accept the technical debt.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 5 lines · Daniel P. Berrangé
I guess there's a balance to be had somewhere on the spectrum between doing everything against the raw C binding, vs everything against a perfectly idiomatic Rust API wrapping the C bniding. The latter might be the ideal, but from a pragmmatic POV I doubt we want the barrier to entry to be that high.
Yes, I agree. I guess we could make things work step by step, even committing something that only focuses on the build system like Manos's work (I'll review it).
I can try to look at the basic QOM interface.
Manos, can you create a page on the wiki? Something like
https://wiki.qemu.org/Features/Meson.
Paolo
Quoted 16 lines
Is this not something we can figure out organically as part of the code design and review processes ?
e.g. if during review we see a device impl doing something where a higher level API would have unambiguous benefits, and creatino of such a higher level API is a practically achieveable task, then ask for it. If a higher level API is desirable, but possibly not practical, then raise it as an potential idea, but be willing to accept the technical debt.
With regards,
Daniel
--|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 7 lines · Paolo Bonzini
Quoted 5 lines · Daniel P. Berrangé
I guess there's a balance to be had somewhere on the spectrum between doing everything against the raw C binding, vs everything against a perfectly idiomatic Rust API wrapping the C bniding. The latter might be the ideal, but from a pragmmatic POV I doubt we want the barrier to entry to be that high.
Quoted 8 lines
Yes, I agree. I guess we could make things work step by step, even committing something that only focuses on the build system like Manos's work (I'll review it).
I can try to look at the basic QOM interface.
Manos, can you create a page on the wiki? Something like
https://wiki.qemu.org/Features/Meson.
Certainly! Just to make sure I understood correctly, you mean a wiki page describing how things work and tracking the progress?
I added https://wiki.qemu.org/Features/Meson/Rust
And a Meson category https://wiki.qemu.org/Category:Meson
Thanks,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 15 lines · Manos Pitsidianakis
Quoted 13 lines · Paolo Bonzini
Quoted 5 lines · Daniel P. Berrangé
I guess there's a balance to be had somewhere on the spectrum between doing everything against the raw C binding, vs everything against a perfectly idiomatic Rust API wrapping the C bniding. The latter might be the ideal, but from a pragmmatic POV I doubt we want the barrier to entry to be that high.
Yes, I agree. I guess we could make things work step by step, even committing something that only focuses on the build system like Manos's work (I'll review it).
I can try to look at the basic QOM interface.
Quoted 8 lines
Quoted 2 lines
Manos, can you create a page on the wiki? Something like
https://wiki.qemu.org/Features/Meson.Certainly! Just to make sure I understood correctly, you mean a wiki page describing how things work and tracking the progress?
I moved it to https://wiki.qemu.org/Features/Rust/Meson :) and wrote
https://wiki.qemu.org/Features/Rust/QOM. I got to the point where at least this compiles:
qdev_define_type!(c"test-device", TestDevice);
impl ObjectImpl for TestDevice {}
impl DeviceImpl for TestDevice {}
fn main() {
let d = TestDevice::new();
d.cold_reset();
}Of course the code makes no sense but it's a start.
One thing that would be very useful is to have an Error implementation. Looking at what Marc-André did for Error*
(https://patchew.org/QEMU/[email protected]/[email protected]/),
his precise implementation relies on his code to go back and forth between Rust representation, borrowed C object with Rust bindings and owned C object with Rust bindings. But I think we can at least have something like this:
// qemu::Error
pub struct Error {
msg: String,
/// Appends the print string of the error to the msg if not None
cause: Option<Box<dyn std::error::Error>>,
location: Option<(String, u32)>
}impl std::error::Error for Error { ... }
impl Error {
...
fn into_c_error(self) -> *const bindings::Error { ... }
}
// qemu::Result<T>
type Result<T> = Result<T, Error>;
which can be implemented without too much code. This way any "bool f(..., Error *)" function (example: realize :)) could be implemented as returning qemu::Result<()>.
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 34 lines · Paolo Bonzini
Quoted 15 lines · Manos Pitsidianakis
Quoted 13 lines · Paolo Bonzini
Quoted 5 lines · Daniel P. Berrangé
I guess there's a balance to be had somewhere on the spectrum between doing everything against the raw C binding, vs everything against a perfectly idiomatic Rust API wrapping the C bniding. The latter might be the ideal, but from a pragmmatic POV I doubt we want the barrier to entry to be that high.
Yes, I agree. I guess we could make things work step by step, even committing something that only focuses on the build system like Manos's work (I'll review it).
I can try to look at the basic QOM interface.
Quoted 8 lines
Quoted 2 lines
Manos, can you create a page on the wiki? Something like
https://wiki.qemu.org/Features/Meson.Certainly! Just to make sure I understood correctly, you mean a wiki page describing how things work and tracking the progress?
I moved it to https://wiki.qemu.org/Features/Rust/Meson :) and wrote
https://wiki.qemu.org/Features/Rust/QOM. I got to the point where at least this compiles:qdev_define_type!(c"test-device", TestDevice);
impl ObjectImpl for TestDevice {}
impl DeviceImpl for TestDevice {}
Quoted 6 lines
fn main() { let d = TestDevice::new(); d.cold_reset(); }Of course the code makes no sense but it's a start.
Let's not rush into making interfaces without the need for them arising first. It's easy to wander off into bikeshedding territory; case in point, there has been little discussion on the code of this RFC and much more focus on hypotheticals.
For what it's worth, in my opinion looking at glib-rs for inspiration is a bad idea, because that project has to support an immutable public interface (glib) while we do not.
There's more to discuss about this topic which I am open to continuing on IRC instead!
-- Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro Ltd
Quoted 30 lines
One thing that would be very useful is to have an Error implementation. Looking at what Marc-André did for Error*
(https://patchew.org/QEMU/[email protected]/[email protected]/),
his precise implementation relies on his code to go back and forth between Rust representation, borrowed C object with Rust bindings and owned C object with Rust bindings. But I think we can at least have something like this://
qemu::Errorpub struct Error { msg: String, /// Appends the print string of the error to the msg if not None cause: Option<Box<dyn std::error::Error>>, location: Option<(String, u32)> }impl
std::error::Errorfor Error { ... }impl Error {
...
fninto_c_error(self)-> *constbindings::Error{ ... }
}// qemu::Result<T>
type Result<T> = Result<T, Error>;which can be implemented without too much code. This way any "bool
f(..., Error *)" function (example: realize :)) could be implemented as returning qemu::Result<()>.
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Il lun 17 giu 2024, 10:59 Manos Pitsidianakis <
[email protected]> ha scritto:
Quoted 9 lines
Quoted 9 lines
qdev_define_type!(c"test-device", TestDevice);
impl ObjectImpl for TestDevice {}
impl DeviceImpl for TestDevice {}fn main() { let d = TestDevice::new(); d.cold_reset(); }
Quoted 7 lines
Quoted 1 line
Of course the code makes no sense but it's a start.
Let's not rush into making interfaces without the need for them arising first. It's easy to wander off into bikeshedding territory; case in point, there has been little discussion on the code of this RFC and much more focus on hypotheticals.
I see your point but I think it's important to understand the road ahead of us.
Knowing that we can build and maintain a usable (does not have to be perfect) interface to QOM is important, and in fact it's already useful for the pl011 device's chardev. It's also important to play with more advanced usage of the language to ascertain what features of the language will be useful; for example, my current implementation uses generic associated types which are not available on Debian Bookworm—it should be easy to remove them but if I am wrong that's also a data point, and an important one.
Don't get me wrong: *for this first device* only, it makes a lot of sense to have a very C-ish implementation. It lets us sort out the build system integration, tackle idiomatic bindings one piece at a time, and is easier to review than Marc-André's approach of building the whole QAPI bindings. But at the same time, I don't consider a C-ish device better just because it's written in Rust: as things stand your code has all the disadvantages of C and all the disadvantages of Rust. What makes it (extremely) valuable is that your code can lead us along the path towards reaping the advantages of Rust.
I think we're actually in a great position. We have a PoC from Marc-André, plus the experience of glib-rs (see below), that shows us that our goal of idiomatic bindings is doable; and a complementary PoC from you that will guide us and let us reach it in little steps. The first 90% of the work is done, now we only have to do the second 90%... :) but then we can open the floodgates for Rust code in QEMU.
For what it's worth, in my opinion looking at glib-rs for inspiration is
Quoted 3 lines
a bad idea, because that project has to support an immutable public interface (glib) while we do not.
glib-rs includes support for GObject, which was itself an inspiration for QOM (with differences, granted).
There are a lot of libraries that we can take inspiration from: in addition to glib-rs, zbus has an interesting approach to serialization/deserialization for example that could inform future work on QAPI. It's not a coincidence that these libraries integrate with more traditional C code, because we are doing the same. Rust-vmm crates will also become an interesting topic sooner or later.
There's more to discuss about this topic which I am open to continuing
Quoted 2 lines
on IRC instead!
Absolutely, I will try to post code soonish and also review the build system integration.
Thanks,
Paolo
Quoted 37 lines
-- Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro LtdQuoted 4 lines
One thing that would be very useful is to have an Error implementation. Looking at what Marc-André did for Error* (
https://patchew.org/QEMU/[email protected]/[email protected]/
),Quoted 26 lines
his precise implementation relies on his code to go back and forth between Rust representation, borrowed C object with Rust bindings and owned C object with Rust bindings. But I think we can at least have something like this:
//
qemu::Errorpub struct Error { msg: String, /// Appends the print string of the error to the msg if not None cause: Option<Box<dyn std::error::Error>>, location: Option<(String, u32)> }impl
std::error::Errorfor Error { ... }impl Error {
...
fninto_c_error(self)-> *constbindings::Error{ ... }
}// qemu::Result<T>
type Result<T> = Result<T, Error>;which can be implemented without too much code. This way any "bool
f(..., Error *)" function (example: realize :)) could be implemented as returning qemu::Result<()>.
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 32 lines · Paolo Bonzini
Il lun 17 giu 2024, 10:59 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 9 lines
Quoted 9 lines
qdev_define_type!(c"test-device", TestDevice);
impl ObjectImpl for TestDevice {}
impl DeviceImpl for TestDevice {}fn main() { let d = TestDevice::new(); d.cold_reset(); }Quoted 7 lines
Quoted 1 line
Of course the code makes no sense but it's a start.
Let's not rush into making interfaces without the need for them arising first. It's easy to wander off into bikeshedding territory; case in point, there has been little discussion on the code of this RFC and much more focus on hypotheticals.
I see your point but I think it's important to understand the road ahead of us.
Knowing that we can build and maintain a usable (does not have to be perfect) interface to QOM is important, and in fact it's already useful for the pl011 device's chardev. It's also important to play with more advanced usage of the language to ascertain what features of the language will be useful; for example, my current implementation uses generic associated types which are not available on Debian Bookworm—it should be easy to remove them but if I am wrong that's also a data point, and an important one.
Quoted 9 lines
Don't get me wrong: *for this first device* only, it makes a lot of sense to have a very C-ish implementation. It lets us sort out the build system integration, tackle idiomatic bindings one piece at a time, and is easier to review than Marc-André's approach of building the whole QAPI bindings. But at the same time, I don't consider a C-ish device better just because it's written in Rust: as things stand your code has all the disadvantages of C and all the disadvantages of Rust. What makes it (extremely) valuable is that your code can lead us along the path towards reaping the advantages of Rust.
I respectfully disagree and recommend taking another look at the code.
The device actually performs all logic in non-unsafe methods and is typed instead of operating on raw integers as fields/state. The C stuff is the FFI boundary calls which you cannot avoid; they are the same things you'd wrap under these bindings we're talking about.
QEMU calls the device's FFI callbacks with its pointer and arguments, the pointer gets dereferenced to the actual Rust type which qemu guarantees is valid, then the Rust struct's methods are called to handle each functionality. There is nothing actually unsafe here, assuming QEMU's invariants and code are correct.
Quoted 72 lines
I think we're actually in a great position. We have a PoC from Marc-André, plus the experience of glib-rs (see below), that shows us that our goal of idiomatic bindings is doable; and a complementary PoC from you that will guide us and let us reach it in little steps. The first 90% of the work is done, now we only have to do the second 90%... :) but then we can open the floodgates for Rust code in QEMU.
For what it's worth, in my opinion looking at glib-rs for inspiration is
Quoted 3 lines
a bad idea, because that project has to support an immutable public interface (glib) while we do not.
glib-rs includes support for GObject, which was itself an inspiration for QOM (with differences, granted).
There are a lot of libraries that we can take inspiration from: in addition to glib-rs, zbus has an interesting approach to serialization/deserialization for example that could inform future work on QAPI. It's not a coincidence that these libraries integrate with more traditional C code, because we are doing the same. Rust-vmm crates will also become an interesting topic sooner or later.
There's more to discuss about this topic which I am open to continuing
Quoted 2 lines
on IRC instead!
Absolutely, I will try to post code soonish and also review the build system integration.
Thanks,
Paolo
Quoted 37 lines
-- Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro LtdQuoted 4 lines
One thing that would be very useful is to have an Error implementation. Looking at what Marc-André did for Error* (
https://patchew.org/QEMU/[email protected]/[email protected]/
),Quoted 26 lines
his precise implementation relies on his code to go back and forth between Rust representation, borrowed C object with Rust bindings and owned C object with Rust bindings. But I think we can at least have something like this:
//
qemu::Errorpub struct Error { msg: String, /// Appends the print string of the error to the msg if not None cause: Option<Box<dyn std::error::Error>>, location: Option<(String, u32)> }impl
std::error::Errorfor Error { ... }impl Error {
...
fninto_c_error(self)-> *constbindings::Error{ ... }
}// qemu::Result<T>
type Result<T> = Result<T, Error>;which can be implemented without too much code. This way any "bool
f(..., Error *)" function (example: realize :)) could be implemented as returning qemu::Result<()>.
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 6 lines · Manos Pitsidianakis
I respectfully disagree and recommend taking another look at the code.
The device actually performs all logic in non-unsafe methods and is typed instead of operating on raw integers as fields/state. The C stuff is the FFI boundary calls which you cannot avoid; they are the same things you'd wrap under these bindings we're talking about.
Indeed, but the whole point is that the bindings wrap unsafe code in such a way that the safety invariants hold. Not doing this, especially for a device that does not do DMA (so that there are very few ways that things can go wrong in the first place), runs counter to the whole philosophy of Rust.
For example, you have:
pub fn realize(&mut self) {
unsafe {
qemu_chr_fe_set_handlers(
addr_of_mut!(self.char_backend),
Some(pl011_can_receive),
Some(pl011_receive),
Some(pl011_event),
None,
addr_of_mut!(*self).cast::<c_void>(),
core::ptr::null_mut(),
true,
);
}
}where you are implicitly relying on the fact that pl011_can_receive(), pl011_receive(), pl011_event() are never called from the MemoryRegionOps read() and write() callbacks. Otherwise you'd have two mutable references at the same time, one as an argument to the callbacks:
pub fn read(&mut self, offset: hwaddr, ...
and one from e.g. "state.as_mut().put_fifo()" in pl011_receive().
This is not Rust code. It makes no attempt at enforcing the whole "shared XOR mutable" which is the basis of Rust's reference semantics. In other words, this is as safe as C code---sure, it can use nice abstractions for register access, it has "unsafe" added in front of pointer dereferences, but it is not safe.
Again, I'm not saying it's a bad first step. It's *awesome* if we treat it as what it is: a guide towards providing safe bindings between Rust and C (which often implies them being idiomatic). But if we don't accept this, if there is no plan to make the code safe, it is a potential huge source of technical debt.
Quoted 5 lines
QEMU calls the device's FFI callbacks with its pointer and arguments, the pointer gets dereferenced to the actual Rust type which qemu guarantees is valid, then the Rust struct's methods are called to handle each functionality. There is nothing actually unsafe here, assuming QEMU's invariants and code are correct.
The same can be said of C code, can't it? There is nothing unsafe in C code, assuming there are no bugs...
Paolo
Quoted 73 lines
Quoted 72 lines
I think we're actually in a great position. We have a PoC from Marc-André, plus the experience of glib-rs (see below), that shows us that our goal of idiomatic bindings is doable; and a complementary PoC from you that will guide us and let us reach it in little steps. The first 90% of the work is done, now we only have to do the second 90%... :) but then we can open the floodgates for Rust code in QEMU.
For what it's worth, in my opinion looking at glib-rs for inspiration is
Quoted 3 lines
a bad idea, because that project has to support an immutable public interface (glib) while we do not.
glib-rs includes support for GObject, which was itself an inspiration for QOM (with differences, granted).
There are a lot of libraries that we can take inspiration from: in addition to glib-rs, zbus has an interesting approach to serialization/deserialization for example that could inform future work on QAPI. It's not a coincidence that these libraries integrate with more traditional C code, because we are doing the same. Rust-vmm crates will also become an interesting topic sooner or later.
There's more to discuss about this topic which I am open to continuing
Quoted 2 lines
on IRC instead!
Absolutely, I will try to post code soonish and also review the build system integration.
Thanks,
Paolo
Quoted 37 lines
-- Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro LtdQuoted 4 lines
One thing that would be very useful is to have an Error implementation. Looking at what Marc-André did for Error* (
https://patchew.org/QEMU/[email protected]/[email protected]/
),Quoted 26 lines
his precise implementation relies on his code to go back and forth between Rust representation, borrowed C object with Rust bindings and owned C object with Rust bindings. But I think we can at least have something like this:
//
qemu::Errorpub struct Error { msg: String, /// Appends the print string of the error to the msg if not None cause: Option<Box<dyn std::error::Error>>, location: Option<(String, u32)> }impl
std::error::Errorfor Error { ... }impl Error {
...
fninto_c_error(self)-> *constbindings::Error{ ... }
}// qemu::Result<T>
type Result<T> = Result<T, Error>;which can be implemented without too much code. This way any "bool
f(..., Error *)" function (example: realize :)) could be implemented as returning qemu::Result<()>.
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 54 lines · Paolo Bonzini
Quoted 6 lines · Manos Pitsidianakis
I respectfully disagree and recommend taking another look at the code.
The device actually performs all logic in non-unsafe methods and is typed instead of operating on raw integers as fields/state. The C stuff is the FFI boundary calls which you cannot avoid; they are the same things you'd wrap under these bindings we're talking about.
Indeed, but the whole point is that the bindings wrap unsafe code in such a way that the safety invariants hold. Not doing this, especially for a device that does not do DMA (so that there are very few ways that things can go wrong in the first place), runs counter to the whole philosophy of Rust.
For example, you have:
pub fn realize(&mut self) { unsafe { qemu_chr_fe_set_handlers( addr_of_mut!(self.char_backend), Some(pl011_can_receive), Some(pl011_receive), Some(pl011_event), None, addr_of_mut!(*self).cast::<c_void>(), core::ptr::null_mut(), true, ); } }where you are implicitly relying on the fact that
pl011_can_receive(),pl011_receive(),pl011_event()are never called from the MemoryRegionOpsread()andwrite()callbacks. Otherwise you'd have two mutable references at the same time, one as an argument to the callbacks:pub fn read(&mut self, offset: hwaddr, ...
and one from e.g. "state.as_mut().put_fifo()" in
pl011_receive().This is not Rust code. It makes no attempt at enforcing the whole "shared XOR mutable" which is the basis of Rust's reference semantics. In other words, this is as safe as C code---sure, it can use nice abstractions for register access, it has "unsafe" added in front of pointer dereferences, but it is not safe.
Again, I'm not saying it's a bad first step. It's *awesome* if we treat it as what it is: a guide towards providing safe bindings between Rust and C (which often implies them being idiomatic). But if we don't accept this, if there is no plan to make the code safe, it is a potential huge source of technical debt.
Quoted 8 lines
Quoted 5 lines
QEMU calls the device's FFI callbacks with its pointer and arguments, the pointer gets dereferenced to the actual Rust type which qemu guarantees is valid, then the Rust struct's methods are called to handle each functionality. There is nothing actually unsafe here, assuming QEMU's invariants and code are correct.
The same can be said of C code, can't it? There is nothing unsafe in C code, assuming there are no bugs...
Paolo, first please tone down your condescending tone, it's incredibly offensive. I'm honestly certain this is not on purpose otherwise I'd not engage at all.
Secondly, are you implying that these callbacks are not operated under the BQL? I'm not seeing the C UART devices using mutexes. If they are not running under the BQL, then gladly we add mutexes, big deal. Just say this directly instead of writing all these amounts of text. If it's true I'd just accept it and move on with a new iteration. Isn't that the point of code review? It is really that simple. Why not do this right away? I'm frankly puzzled.
Finally, this is Rust code. You cannot turn off the type system, you cannot turn off the borrow checker, you can only go around creating new types and references out of raw memory addresses and tell the compiler 'trust me on this'. Ignoring that misses the entire point of Rust. The statement 'this is not Rust code but as good as C' is dishonest and misguided. Check for example the source code of the nix crate, which exposes libc and various POSIX/*nix APIs. Is it the same as C and not Rust code?
If you have specific scenarios in mind where such things exist in the code and end up doing invalid behavior please be kind and write them down explicitly and demonstrate them on code review. This approach of 'yes but no' is not constructive because it is not addressing any specific problems directly. Instead it comes out as vague dismissive FUD and I'm sure that is not what you or anyone else wants.
Please take some time to understand my POV here, it'd help both of us immensely.
Sincerely thank you in advance,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 61 lines · Manos Pitsidianakis
Quoted 60 lines · Paolo Bonzini
Quoted 6 lines · Manos Pitsidianakis
I respectfully disagree and recommend taking another look at the code.
The device actually performs all logic in non-unsafe methods and is typed instead of operating on raw integers as fields/state. The C stuff is the FFI boundary calls which you cannot avoid; they are the same things you'd wrap under these bindings we're talking about.
Indeed, but the whole point is that the bindings wrap unsafe code in such a way that the safety invariants hold. Not doing this, especially for a device that does not do DMA (so that there are very few ways that things can go wrong in the first place), runs counter to the whole philosophy of Rust.
For example, you have:
pub fn realize(&mut self) { unsafe { qemu_chr_fe_set_handlers( addr_of_mut!(self.char_backend), Some(pl011_can_receive), Some(pl011_receive), Some(pl011_event), None, addr_of_mut!(*self).cast::<c_void>(), core::ptr::null_mut(), true, ); } }where you are implicitly relying on the fact that
pl011_can_receive(),pl011_receive(),pl011_event()are never called from the MemoryRegionOpsread()andwrite()callbacks. Otherwise you'd have two mutable references at the same time, one as an argument to the callbacks:pub fn read(&mut self, offset: hwaddr, ...
and one from e.g. "state.as_mut().put_fifo()" in
pl011_receive().This is not Rust code. It makes no attempt at enforcing the whole "shared XOR mutable" which is the basis of Rust's reference semantics. In other words, this is as safe as C code---sure, it can use nice abstractions for register access, it has "unsafe" added in front of pointer dereferences, but it is not safe.
Again, I'm not saying it's a bad first step. It's *awesome* if we treat it as what it is: a guide towards providing safe bindings between Rust and C (which often implies them being idiomatic). But if we don't accept this, if there is no plan to make the code safe, it is a potential huge source of technical debt.
Quoted 5 lines
QEMU calls the device's FFI callbacks with its pointer and arguments, the pointer gets dereferenced to the actual Rust type which qemu guarantees is valid, then the Rust struct's methods are called to handle each functionality. There is nothing actually unsafe here, assuming QEMU's invariants and code are correct.
Quoted 6 lines
Quoted 2 lines
The same can be said of C code, can't it? There is nothing unsafe in C code, assuming there are no bugs...
Paolo, first please tone down your condescending tone, it's incredibly offensive. I'm honestly certain this is not on purpose otherwise I'd not engage at all.
The best compliment you had was "I'm not saying it's a bad first step", and I would say this differently: It's a great first step!
We should have a first version where we stick to the C API, with all the Rust code being as Rusty as possible: benefit from typed enums, error handling, bounds checking and other nice things.
It's useless to iterate/debate for two years on the list before landing something upstream. We can start with this, have one or two devices that use this build system, and then focus on designing a good interface for this.
Quoted 9 lines
Secondly, are you implying that these callbacks are not operated under the BQL? I'm not seeing the C UART devices using mutexes. If they are not running under the BQL, then gladly we add mutexes, big deal. Just say this directly instead of writing all these amounts of text. If it's true I'd just accept it and move on with a new iteration. Isn't that the point of code review? It is really that simple. Why not do this right away? I'm frankly puzzled.
As I mentioned in my previous answer, this device already makes a good progress: it eliminates a whole class of memory errors, and the only issue brought by unsafe code is concurrency issues. And this should be our focus once we get the build infrastructure done!
Quoted 21 lines
Finally, this is Rust code. You cannot turn off the type system, you cannot turn off the borrow checker, you can only go around creating new types and references out of raw memory addresses and tell the compiler 'trust me on this'. Ignoring that misses the entire point of Rust. The statement 'this is not Rust code but as good as C' is dishonest and misguided. Check for example the source code of the nix crate, which exposes libc and various POSIX/*nix APIs. Is it the same as C and not Rust code?
If you have specific scenarios in mind where such things exist in the code and end up doing invalid behavior please be kind and write them down explicitly and demonstrate them on code review. This approach of 'yes but no' is not constructive because it is not addressing any specific problems directly. Instead it comes out as vague dismissive FUD and I'm sure that is not what you or anyone else wants.
Please take some time to understand my POV here, it'd help both of us immensely.
Sincerely thank you in advance,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 66 lines · Pierrick Bouvier
Quoted 64 lines · Manos Pitsidianakis
Quoted 62 lines · Paolo Bonzini
Quoted 6 lines · Manos Pitsidianakis
I respectfully disagree and recommend taking another look at the code.
The device actually performs all logic in non-unsafe methods and is typed instead of operating on raw integers as fields/state. The C stuff is the FFI boundary calls which you cannot avoid; they are the same things you'd wrap under these bindings we're talking about.
Indeed, but the whole point is that the bindings wrap unsafe code in such a way that the safety invariants hold. Not doing this, especially for a device that does not do DMA (so that there are very few ways that things can go wrong in the first place), runs counter to the whole philosophy of Rust.
For example, you have:
pub fn realize(&mut self) { unsafe { qemu_chr_fe_set_handlers( addr_of_mut!(self.char_backend), Some(pl011_can_receive), Some(pl011_receive), Some(pl011_event), None, addr_of_mut!(*self).cast::<c_void>(), core::ptr::null_mut(), true, ); } }where you are implicitly relying on the fact that
pl011_can_receive(),pl011_receive(),pl011_event()are never called from the MemoryRegionOpsread()andwrite()callbacks. Otherwise you'd have two mutable references at the same time, one as an argument to the callbacks:pub fn read(&mut self, offset: hwaddr, ...
and one from e.g. "state.as_mut().put_fifo()" in
pl011_receive().This is not Rust code. It makes no attempt at enforcing the whole "shared XOR mutable" which is the basis of Rust's reference semantics. In other words, this is as safe as C code---sure, it can use nice abstractions for register access, it has "unsafe" added in front of pointer dereferences, but it is not safe.
Again, I'm not saying it's a bad first step. It's *awesome* if we treat it as what it is: a guide towards providing safe bindings between Rust and C (which often implies them being idiomatic). But if we don't accept this, if there is no plan to make the code safe, it is a potential huge source of technical debt.
Quoted 5 lines
QEMU calls the device's FFI callbacks with its pointer and arguments, the pointer gets dereferenced to the actual Rust type which qemu guarantees is valid, then the Rust struct's methods are called to handle each functionality. There is nothing actually unsafe here, assuming QEMU's invariants and code are correct.
The same can be said of C code, can't it? There is nothing unsafe in C code, assuming there are no bugs...
Quoted 6 lines
Quoted 3 lines
Paolo, first please tone down your condescending tone, it's incredibly offensive. I'm honestly certain this is not on purpose otherwise I'd not engage at all.
The best compliment you had was "I'm not saying it's a bad first step", and I would say this differently: It's a great first step!
Don't quote me out of context; I said It's an "awesome" first step, though I qualified that we should treat it as "a guide towards providing safe bindings between Rust and C". That is, as long as we agree that it is not production quality code. Which it doesn't have to be!
Quoted 8 lines
We should have a first version where we stick to the C API, with all the Rust code being as Rusty as possible: benefit from typed enums, error handling, bounds checking and other nice things.
It's useless to iterate/debate for two years on the list before landing something upstream. We can start with this, have one or two devices that use this build system, and then focus on designing a good interface for this.
I never said that I want perfection before landing upstream. I want _a path_.
When I read "there was consensus in the community call that we won't be writing Rust APIs for internal C QEMU interfaces; or at least, that's not the goal"[1], then sorry but I think that it's better to stick with C.
On the other hand, if there is a path towards proper, maintainable Rust, then I am even okay even with committing something that technically contains undefined behavior.
[1] https://lore.kernel.org/qemu-devel/[email protected]/
Quoted 4 lines
As I mentioned in my previous answer, this device already makes a good
progress: it eliminates a whole class of memory errors, and the only issue brought by unsafe code is concurrency issues. And this should be our focus once we get the build infrastructure done!
Let's not exaggerate the benefits: no pointers were converted to RAII (Box<> or Rc<>) in the course of writing the Rust code; so there are no memory errors that can be eliminated by the rewrite. In fact, new memory errors can also be introduced, because safe Rust has stricter aliasing rules than C.
This is not a problem! It's just that we need to be realistic to understand where to focus next and why. To build our path.
(Also, a small correction so that we're on the same page on the fix:
it's reentrancy, not concurrency).
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Manos PitsidianakisIl lun 17 giu 2024, 23:45 Manos Pitsidianakis
<[email protected]> ha scritto:
Quoted 2 lines
Secondly, are you implying that these callbacks are not operated under the BQL?
No, I'm implying that if you had the following nested calls:
unsafe read callback receives the opaque point
-> cast to &mut to call safe read callback
-> chardev function accessing the opaque value
-> unsafe chardev callback receives the opaque pointer
-> cast to & or &mut to call safe chardev callbackThen you would violate the Rust invariant that there can only be one active reference at a single time if it is &mut. The only exception is that you can create an &mut UnsafeCell<T> for an active &UnsafeCell<T>.
It doesn't matter if this cannot happen because of invariants in the QEMU code. If you are writing safe Rust, every cast to &mut requires very strong justification. The way you do it instead is to use RefCell, and borrow_mut() instead of casting to &mut. This way, at least, the invariant is checked at runtime.
And in fact this _can_ happen. I hadn't checked until now because I was mostly worried about the conceptual problem, not any specific way undefined behavior can happen. But here it is:
PL011State::read takes &mut
-> qemu_chr_fe_accept_input
-> mux_chr_accept_input
-> pl011_can_receive
-> creates & which is undefined behaviorYou probably can write a 20 lines test case in pure Rust that miri complains about.
Should we make it a requirement for v3, that all callback methods in PL011State (read, write, can_receive, receive, event) take a &self and achieve interior mutability through RefCell? Probably not, a comment is enough even though technically this is _not_ valid Rust. But it is a high priority task after the initial commit.
Quoted 1 line
Just say this directly instead of writing all these amounts of text
I said in the first review that the state should be behind a RefCell.
Quoted 4 lines
Finally, this is Rust code. You cannot turn off the type system, you cannot turn off the borrow checker, you can only go around creating new types and references out of raw memory addresses and tell the compiler 'trust me on this'
I am sorry if this sounds condescending. But this is _exactly_ what I'm complaining about: that there is too much trust placed in the programmer.
Converting from * to & does not turn off the borrow checker, but still you cannot trust anymore what the borrow checker says; and that's why you do it inside an abstraction, not in each and every callback of each and every device. This is what Marc-André did for QAPI; and he probably erred in the other direction for a PoC, but we _must_ agree that something as complete as his work is the direction that we _have_ to take.
Again: I am sorry that you feel this way about my remark, because I thought I had only praised your work. I didn't think I was being condescending or dismissing. But I am worried that without the proper abstractions this is going to be a technical debt magnet with only marginal benefits over C.
And frankly I am saying this from experience. Check out CVE-2019-18960 which was an out of bounds access in Firecracker caused by not using proper abstractions for DMA. And that's in a 100% Rust code base. Since we're starting from and interfacing with C it's only going to be worse; skimping on abstractions is simply something that we cannot afford.
It's also the way Linux is introducing Rust in the code base. Whenever something needs access to C functionality they introduce bindings to it. It's slower, but it's sustainable.
Quoted 5 lines
Ignoring that misses the entire point of Rust. The statement 'this is not Rust code but as good as C' is dishonest and misguided. Check for example the source code of the nix crate, which exposes libc and various POSIX/*nix APIs. Is it the same as C and not Rust code?
That's exactly my point. Libc provides mostly unsafe functions, which is on the level of what bindgen generates. Other crates on top provide *safe abstractions* such as nix's Dir
(https://docs.rs/nix/0.29.0/nix/dir/struct.Dir.html). If possible, leaf crates use safe Rust (nix), and avoid unsafe (libc) as much as possible.
Instead you're using the unsafe functions in the device itself. It's missing an equivalent of nix.
Quoted 3 lines
If you have specific scenarios in mind where such things exist in the code and end up doing invalid behavior please be kind and write them down explicitly and demonstrate them on code review.
It doesn't matter whether they exist or not. The point of Rust is that the type system ensures that these invalid behaviors are caught either at compile time or (with RefCell) at run time. As things stand, your code cannot catch them and the language provides a false sense of security.
On the other hand, I want to be clear agin that this is not a problem at all—but only as long as we agree that it's not the desired final state
Quoted 7 lines
This approach of
'yes but no' is not constructive because it is not addressing any specific problems directly. Instead it comes out as vague dismissive FUD and I'm sure that is not what you or anyone else wants.Please take some time to understand my POV here, it'd help both of us immensely.
I can ask the same though. Please take the time to understand that I am not being dismissive! My position is exactly "yes but no". Yes, this is a great way to introduce Rust in the code base. No, this is not a sustainable way to mix Rust and C—but I am willing to help.
Paolo
Quoted 4 lines
Sincerely thank you in advance,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Paolo BonziniQuoted 9 lines · Paolo Bonzini
Quoted 6 lines · Manos Pitsidianakis
I respectfully disagree and recommend taking another look at the code.
The device actually performs all logic in non-unsafe methods and is typed instead of operating on raw integers as fields/state. The C stuff is the FFI boundary calls which you cannot avoid; they are the same things you'd wrap under these bindings we're talking about.
Quoted 6 lines
Indeed, but the whole point is that the bindings wrap unsafe code in such a way that the safety invariants hold. Not doing this, especially for a device that does not do DMA (so that there are very few ways that things can go wrong in the first place), runs counter to the whole philosophy of Rust.
You are raising an interesting point, and should be a central discussion when designing the future Rust API for this subsystem.
It may not be intuitive to people that even a code without any unsafe section could still call code in a sequence that will violate some invariants, and break Rust rules. IMHO, this is one of the big challenge with the Rust/C interfacing, including for Linux.
But it's *not yet* the goal of this series. First, we should see how to build one device (I would even like to see a second, to factorize all the boilerplate), and then, focus on which interface we want to have to make it really better than the C version.
Quoted 25 lines
For example, you have:
pub fn realize(&mut self) { unsafe { qemu_chr_fe_set_handlers( addr_of_mut!(self.char_backend), Some(pl011_can_receive), Some(pl011_receive), Some(pl011_event), None, addr_of_mut!(*self).cast::<c_void>(), core::ptr::null_mut(), true, ); } }where you are implicitly relying on the fact that
pl011_can_receive(),pl011_receive(),pl011_event()are never called from the MemoryRegionOpsread()andwrite()callbacks. Otherwise you'd have two mutable references at the same time, one as an argument to the callbacks:pub fn read(&mut self, offset: hwaddr, ...
Quoted 8 lines
and one from e.g. "state.as_mut().put_fifo()" in
pl011_receive().This is not Rust code. It makes no attempt at enforcing the whole "shared XOR mutable" which is the basis of Rust's reference semantics. In other words, this is as safe as C code---sure, it can use nice abstractions for register access, it has "unsafe" added in front of pointer dereferences, but it is not safe.
I think that Manos did a great amount of work to reduce the use of unsafe code. Basically, he wrote the device as Rusty as possible, while still using the QEMU C API part that is inevitable today.
In more, given the current design, yes some race conditions are possible (depends on how QEMU calls callbacks installed), but a whole class of problems is still eliminated.
From the start of this series, it was precised that focus was on build system, and it seemed to me that we shifted on the hot debate of "How to interface with C code?".
Quoted 6 lines
Again, I'm not saying it's a bad first step. It's *awesome* if we treat it as what it is: a guide towards providing safe bindings between Rust and C (which often implies them being idiomatic). But if we don't accept this, if there is no plan to make the code safe, it is a potential huge source of technical debt.
I agree, it should be the next direction after a first device was written: How to remove almost all usage of unsafe code and maintain good invariants in the API?
While talking about idiomatic, Rust tends to favor message passing to memory share when it comes to concurrency (and IMHO, it's the right thing). And it's definitely now how QEMU is architected at this time.
With extra locking, we should be able to have a first correct interface that offers strong guarantees, and we can then work on making it faster with concurrency.
Quoted 9 lines
Quoted 5 lines
QEMU calls the device's FFI callbacks with its pointer and arguments, the pointer gets dereferenced to the actual Rust type which qemu guarantees is valid, then the Rust struct's methods are called to handle each functionality. There is nothing actually unsafe here, assuming QEMU's invariants and code are correct.
The same can be said of C code, can't it? There is nothing unsafe in C code, assuming there are no bugs...
Not the same, you still get other compile/runtime guarantees:
- strong typed enums, so no case is forgotten
- good error handling
- safe type conversions
- bound check of fifo access
The only open issue by calling unsafe code in this context is related to (potential) concurrency. If a first step to have a good Rust API is to run everything under a lock, there is good chance you already started to design the device in the right way to support real concurrency later, so it's still useful.
Pierrick
Quoted 76 lines
Paolo
Quoted 73 lines
Quoted 72 lines
I think we're actually in a great position. We have a PoC from Marc-André, plus the experience of glib-rs (see below), that shows us that our goal of idiomatic bindings is doable; and a complementary PoC from you that will guide us and let us reach it in little steps. The first 90% of the work is done, now we only have to do the second 90%... :) but then we can open the floodgates for Rust code in QEMU.
For what it's worth, in my opinion looking at glib-rs for inspiration is
Quoted 3 lines
a bad idea, because that project has to support an immutable public interface (glib) while we do not.
glib-rs includes support for GObject, which was itself an inspiration for QOM (with differences, granted).
There are a lot of libraries that we can take inspiration from: in addition to glib-rs, zbus has an interesting approach to serialization/deserialization for example that could inform future work on QAPI. It's not a coincidence that these libraries integrate with more traditional C code, because we are doing the same. Rust-vmm crates will also become an interesting topic sooner or later.
There's more to discuss about this topic which I am open to continuing
Quoted 2 lines
on IRC instead!
Absolutely, I will try to post code soonish and also review the build system integration.
Thanks,
Paolo
Quoted 37 lines
-- Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro LtdQuoted 4 lines
One thing that would be very useful is to have an Error implementation. Looking at what Marc-André did for Error* (
https://patchew.org/QEMU/[email protected]/[email protected]/
),Quoted 26 lines
his precise implementation relies on his code to go back and forth between Rust representation, borrowed C object with Rust bindings and owned C object with Rust bindings. But I think we can at least have something like this:
//
qemu::Errorpub struct Error { msg: String, /// Appends the print string of the error to the msg if not None cause: Option<Box<dyn std::error::Error>>, location: Option<(String, u32)> }impl
std::error::Errorfor Error { ... }impl Error {
...
fninto_c_error(self)-> *constbindings::Error{ ... }
}// qemu::Result<T>
type Result<T> = Result<T, Error>;which can be implemented without too much code. This way any "bool
f(..., Error *)" function (example: realize :)) could be implemented as returning qemu::Result<()>.
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Paolo BonziniQuoted 32 lines · Paolo Bonzini
Il lun 17 giu 2024, 10:59 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 9 lines
Quoted 9 lines
qdev_define_type!(c"test-device", TestDevice);
impl ObjectImpl for TestDevice {}
impl DeviceImpl for TestDevice {}fn main() { let d = TestDevice::new(); d.cold_reset(); }Quoted 7 lines
Quoted 1 line
Of course the code makes no sense but it's a start.
Let's not rush into making interfaces without the need for them arising first. It's easy to wander off into bikeshedding territory; case in point, there has been little discussion on the code of this RFC and much more focus on hypotheticals.
I see your point but I think it's important to understand the road ahead of us.
Knowing that we can build and maintain a usable (does not have to be perfect) interface to QOM is important, and in fact it's already useful for the pl011 device's chardev. It's also important to play with more advanced usage of the language to ascertain what features of the language will be useful; for example, my current implementation uses generic associated types which are not available on Debian Bookworm—it should be easy to remove them but if I am wrong that's also a data point, and an important one.
Quoted 9 lines
Don't get me wrong: *for this first device* only, it makes a lot of sense to have a very C-ish implementation. It lets us sort out the build system integration, tackle idiomatic bindings one piece at a time, and is easier to review than Marc-André's approach of building the whole QAPI bindings. But at the same time, I don't consider a C-ish device better just because it's written in Rust: as things stand your code has all the disadvantages of C and all the disadvantages of Rust. What makes it (extremely) valuable is that your code can lead us along the path towards reaping the advantages of Rust.
I wonder if starting with a device implementation is perhaps the wrong idea, in terms of a practical yet simple first step.
As devices go, the pl011 device is simple, but compared to other QOM impls in QEMU, devices are still relatively complex things, especially if we want to write against safe abstraction.
How about we go simpler still, and focus on one of the object backends. For example, the RNG backend interface is practically the most trivial QOM impl we can do in QEMU. It has one virtual method that needs to be implemented, which is passed a callback to receive entropy, and one native method to call to indicate completion.
Providing a safe Rust abstraction for implementing an RNG backend looks like a much quicker proposition that a safe abstraction for implementing a device. The various RNG impls have a few places where they touch other QEMU code (rng-builtin uses qemu_bh, rng-egd lightly touches chardev APIs, rng-random touches main loop FD handlers). Each of those things though, are small & useful API problems to look it solving.
If we did this I think we would not have to give a "free pass" for a hackish C-like first Rust impl. We would have something designed well from day 1, showing small, but tangible benefits, with a path to incrementally broadening the effort.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 6 lines · Daniel P. Berrangé
I wonder if starting with a device implementation is perhaps the wrong idea, in terms of a practical yet simple first step.
As devices go, the pl011 device is simple, but compared to other QOM impls in QEMU, devices are still relatively complex things, especially if we want to write against safe abstraction.
It's true, but I think _some_ complexity provides a better guide as to what are the next step.
I think it's clear that they are, not in this order:
* calling QOM methods (Chardev)
* implementing QOM objects
* implementing QOM devices
** qdev properties
** MemoryRegion callbacks
* implementing Chardev callbacks
* general technique for bindings for C structs (Error, QAPI)
Quoted 4 lines
If we did this I think we would not have to give a "free pass" for a hackish C-like first Rust impl. We would have something designed well from day 1, showing small, but tangible benefits, with a path to incrementally broadening the effort.
I don't think it's that easy to have something self contained for a single submission. Reviewing the build system is a completely different proposition than reviewing generic-heavy QOM bindings.
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 12 lines · Paolo Bonzini
Quoted 6 lines · Daniel P. Berrangé
I wonder if starting with a device implementation is perhaps the wrong idea, in terms of a practical yet simple first step.
As devices go, the pl011 device is simple, but compared to other QOM impls in QEMU, devices are still relatively complex things, especially if we want to write against safe abstraction.
It's true, but I think _some_ complexity provides a better guide as to what are the next step.
Quoted 8 lines
I think it's clear that they are, not in this order:
* calling QOM methods (Chardev)
* implementing QOM objects
* implementing QOM devices
** qdev properties
** MemoryRegion callbacks
* implementing Chardev callbacks
* general technique for bindings for C structs (Error, QAPI)
Right, this is why I suggested the pl011 as a device: I felt it provided enough complexity in terms of where it interconnects to the rest of QEMU to be a realistic way to find out where the points of difficulty are, without being super complicated simply as a device. We don't need to have fully worked out solutions to these things in the first pass, but I agree with Paolo that we do want to have a clear path forward that says "this is what we're expecting the solutions to these points of difficulty to end up looking like and how we plan to get there".
thanks
-- PMM
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Paolo BonziniHi Paolo,
Quoted 24 lines · Paolo Bonzini
Date: Thu, 13 Jun 2024 09:56:39 +0200
From: Paolo Bonzini <[email protected]>
Subject: Re: [RFC PATCH v2 3/5] rust: add PL011 device modelIl gio 13 giu 2024, 09:13 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 9 lines · Paolo Bonzini
Quoted 4 lines
Il mer 12 giu 2024, 22:58 Manos Pitsidianakis <
[email protected]> ha scritto:Quoted 1 line
In any case, it is out of scope for this RFC. Introducing wrappers
would
Quoted 3 lines
Quoted 2 lines
be a gradual process.
Quoted 7 lines
Quoted 2 lines
Sure, how would you feel about such bindings being developed on list, and maintained in a (somewhat) long-lived experimental branch?
IMHO any higher level binding APIs for Rust should be acceptable in the main QEMU tree as soon as we accept Rust functionality. They can evolve in-tree based on the needs of whomever is creating and/or consuming them.
Quoted 5 lines
My question is the opposite, should we accept Rust functionality without proper high level bindings? I am afraid that, if more Rust devices are contributed, it becomes technical debt to have a mix of idiomatic and C-ish code. If the answer is no, then this PL011 device has to be developed out of tree.
I think deeper and higher level bindings will have more opens and will likely require more discussion and exploration. So could we explore this direction on another reference Rust device?
I also think there won’t be too many Rust devices in the short term. Going back to tweak or enhance existing Rust devices may not require too much effort, once we have a definitive answer.
I wonder if x86 could also implement a rust device (like the x86 timer you mentioned before, hw/i386/kvm/i8254.c or hw/timer/i8254.c IIRC) to try this? Or would you recommend another x86 device? :-)
I'd be willing to help Manos try it if you think it's fine.
Regards,
Zhao
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 11 lines · Zhao Liu
I think deeper and higher level bindings will have more opens and will likely require more discussion and exploration. So could we explore this direction on another reference Rust device?
I also think there won’t be too many Rust devices in the short term. Going back to tweak or enhance existing Rust devices may not require too much effort, once we have a definitive answer.
I wonder if x86 could also implement a rust device (like the x86 timer you mentioned before,
hw/i386/kvm/i8254.corhw/timer/i8254.cIIRC) to try this? Or would you recommend another x86 device? :-)
A timer device is a good idea, just because it's another pretty stable low-level QEMU API.
The problem with hw/timer/i8254.c is that it has the KVM version, as you found. The HPET is an alternative though.
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Manos PitsidianakisQuoted 21 lines · Manos Pitsidianakis
Date: Tue, 11 Jun 2024 13:33:32 +0300
From: Manos Pitsidianakis <[email protected]>
Subject: [RFC PATCH v2 3/5] rust: add PL011 device model
X-Mailer: git-send-email 2.44.0This commit adds a re-implementation of
hw/char/pl011.cin Rust.It uses generated Rust bindings (produced by
ninja aarch64-softmmu-generated.rs) to
register itself as a QOM type/class.How to build:
1. Make sure rust, cargo and bindgen (cargo install bindgen-cli) are installed
2. Configure a QEMU build with:--enable-system --target-list=aarch64-softmmu --enable-with-rust3. Launching a VM with qemu-system-aarch64 should use the Rust version of the pl011 device (unless it is not set up so in
hw/arm/virt.c; the type of the UART device is hardcoded).
Quoted 5 lines
To confirm, inspect
info qom-treein the monitor and look for anx-pl011-rustdevice.Signed-off-by Manos Pitsidianakis <[email protected]>---
Hi Manos,
Thanks for your example!
Quoted 6 lines
rust/pl011/Cargo.toml new file +0
-0@@ -0,0 +1,66 @@
...
Quoted 39 lines
+[lints]
+[lints.rustdoc]rust/pl011/Cargo.toml new file +37
-0+broken_intra_doc_links = "deny" +redundant_explicit_links = "deny" +[lints.clippy] +# lint groups +correctness = { level = "deny", priority = -1 } +suspicious = { level = "deny", priority = -1 } +complexity = { level = "deny", priority = -1 } +perf = { level = "deny", priority = -1 } +cargo = { level = "deny", priority = -1 } +nursery = { level = "deny", priority = -1 } +style = { level = "deny", priority = -1 } +# restriction group +dbg_macro = "deny" +rc_buffer = "deny" +as_underscore = "deny" +assertions_on_result_states = "deny" +# pedantic group +doc_markdown = "deny" +expect_fun_call = "deny" +borrow_as_ptr = "deny" +case_sensitive_file_extension_comparisons = "deny" +cast_lossless = "deny" +cast_ptr_alignment = "allow" +large_futures = "deny" +waker_clone_wake = "deny" +unused_enumerate_index = "deny" +unnecessary_fallible_conversions = "deny" +struct_field_names = "deny" +manual_hash_one = "deny" +into_iter_without_iter = "deny" +option_if_let_else = "deny" +missing_const_for_fn = "deny" +significant_drop_tightening = "deny" +multiple_crate_versions = "deny" +significant_drop_in_scrutinee = "deny" +cognitive_complexity = "deny" +missing_safety_doc = "allow"
...
Quoted 7 lines
rust/pl011/rustfmt.toml new file +1
-0@@ -0,0 +1 @@ +../rustfmt.toml
...
Quoted 7 lines
rust/rustfmt.toml new file +0
-0
rust/rustfmt.toml line 7
Quoted 7 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = true
About the Rust style, inspired from the discussion on my previous simpletrace-rust [1], it looks like people prefer the default rust style and use the default check without custom configurations.
More style requirements are also an open, especially for unstable ones, and it would be better to split this part into a separate patch, so that the discussion about style doesn't overshadow the focus on your example.
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
Regards,
Zhao
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Hello Zhao :)
Quoted 112 lines · Zhao Liu
Quoted 21 lines · Manos Pitsidianakis
Date: Tue, 11 Jun 2024 13:33:32 +0300
From: Manos Pitsidianakis <[email protected]>
Subject: [RFC PATCH v2 3/5] rust: add PL011 device model
X-Mailer: git-send-email 2.44.0This commit adds a re-implementation of
hw/char/pl011.cin Rust.It uses generated Rust bindings (produced by
ninja aarch64-softmmu-generated.rs) to
register itself as a QOM type/class.How to build:
1. Make sure rust, cargo and bindgen (cargo install bindgen-cli) are installed
2. Configure a QEMU build with:--enable-system --target-list=aarch64-softmmu --enable-with-rust3. Launching a VM with qemu-system-aarch64 should use the Rust version of the pl011 device (unless it is not set up so in
hw/arm/virt.c; the type of the UART device is hardcoded).Quoted 5 lines
To confirm, inspect
info qom-treein the monitor and look for anx-pl011-rustdevice.Signed-off-by Manos Pitsidianakis <[email protected]>---
Hi Manos,
Thanks for your example!
Quoted 6 lines
rust/pl011/Cargo.toml new file +0
-0@@ -0,0 +1,66 @@...
Quoted 39 lines
+[lints]
+[lints.rustdoc]rust/pl011/Cargo.toml new file +37
-0+broken_intra_doc_links = "deny" +redundant_explicit_links = "deny" +[lints.clippy] +# lint groups +correctness = { level = "deny", priority = -1 } +suspicious = { level = "deny", priority = -1 } +complexity = { level = "deny", priority = -1 } +perf = { level = "deny", priority = -1 } +cargo = { level = "deny", priority = -1 } +nursery = { level = "deny", priority = -1 } +style = { level = "deny", priority = -1 } +# restriction group +dbg_macro = "deny" +rc_buffer = "deny" +as_underscore = "deny" +assertions_on_result_states = "deny" +# pedantic group +doc_markdown = "deny" +expect_fun_call = "deny" +borrow_as_ptr = "deny" +case_sensitive_file_extension_comparisons = "deny" +cast_lossless = "deny" +cast_ptr_alignment = "allow" +large_futures = "deny" +waker_clone_wake = "deny" +unused_enumerate_index = "deny" +unnecessary_fallible_conversions = "deny" +struct_field_names = "deny" +manual_hash_one = "deny" +into_iter_without_iter = "deny" +option_if_let_else = "deny" +missing_const_for_fn = "deny" +significant_drop_tightening = "deny" +multiple_crate_versions = "deny" +significant_drop_in_scrutinee = "deny" +cognitive_complexity = "deny" +missing_safety_doc = "allow"...
Quoted 7 lines
rust/pl011/rustfmt.toml new file +1
-0@@ -0,0 +1 @@ +../rustfmt.toml...
Quoted 7 lines
rust/rustfmt.toml new file +0
-0rust/rustfmt.toml line 7
Quoted 7 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = trueAbout the Rust style, inspired from the discussion on my previous simpletrace-rust [1], it looks like people prefer the default rust style and use the default check without custom configurations.
Quoted 6 lines
More style requirements are also an open, especially for unstable ones, and it would be better to split this part into a separate patch, so that the discussion about style doesn't overshadow the focus on your example.
I had read that discussion and had that in mind. There's no need to worry about format inconsistencies; these options are unstable -nightly only- format options and they don't affect the default rust style (they actually follow it). If you run a stable cargo fmt you will see the code won't change (but might complain that these settings are nightly only).
What they do is extra work on top of the default style. If anything ends up incompatible with stable I agree it must be removed, there's no sense in having a custom Rust style when the defaults are so reasonable.
To sum it up, the style is essentially the default one, so there's no problem here!
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 32 lines · Manos Pitsidianakis
Quoted 19 lines
Quoted 7 lines
rust/rustfmt.toml new file +0
-0rust/rustfmt.toml line 7
Quoted 7 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = trueAbout the Rust style, inspired from the discussion on my previous simpletrace-rust [1], it looks like people prefer the default rust style and use the default check without custom configurations.
Quoted 6 lines
More style requirements are also an open, especially for unstable ones, and it would be better to split this part into a separate patch, so that the discussion about style doesn't overshadow the focus on your example.
I had read that discussion and had that in mind. There's no need to worry about format inconsistencies; these options are unstable -nightly only- format options and they don't affect the default rust style (they actually follow it). If you run a stable cargo fmt you will see the code won't change (but might complain that these settings are nightly only).
Quoted 3 lines
What they do is extra work on top of the default style. If anything ends up incompatible with stable I agree it must be removed, there's no sense in having a custom Rust style when the defaults are so reasonable.
This doesn't make sense. One the one hand you're saying the rules don't have any effect on the code style vs the default, but on the otherhand saying they do "extra work" on top of the default style. Those can't both be true.
Quoted 2 lines
To sum it up, the style is essentially the default one, so there's no problem here!
If the style config is essentially the default one, then just remove this config and make it actually the default.
Having this file exist sets the (incorrect) expectation that we would be willing to accept changes that diverge from the default rust style, and that's not desirable IMHO.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 33 lines · Daniel P. Berrangé
Quoted 32 lines · Manos Pitsidianakis
Quoted 19 lines
Quoted 7 lines
rust/rustfmt.toml new file +0
-0rust/rustfmt.toml line 7
Quoted 7 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = trueAbout the Rust style, inspired from the discussion on my previous simpletrace-rust [1], it looks like people prefer the default rust style and use the default check without custom configurations.
Quoted 6 lines
More style requirements are also an open, especially for unstable ones, and it would be better to split this part into a separate patch, so that the discussion about style doesn't overshadow the focus on your example.
I had read that discussion and had that in mind. There's no need to worry about format inconsistencies; these options are unstable -nightly only- format options and they don't affect the default rust style (they actually follow it). If you run a stable cargo fmt you will see the code won't change (but might complain that these settings are nightly only).
Quoted 8 lines
Quoted 3 lines
What they do is extra work on top of the default style. If anything ends up incompatible with stable I agree it must be removed, there's no sense in having a custom Rust style when the defaults are so reasonable.
This doesn't make sense. One the one hand you're saying the rules don't have any effect on the code style vs the default, but on the otherhand saying they do "extra work" on top of the default style. Those can't both be true.
No, I fear there's a confusion here. It means that if you run the stable rustfmt with the default options the code doesn't change. I.e. it does not conflict with the default style.
What it does is group imports, format text in doc comments (which stable rustfmt doesn't touch at all) and also splits long strings into several lines, which are all helpful for e-mail patch reviews.
Thanks,
Manos
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 42 lines · Manos Pitsidianakis
Quoted 32 lines · Daniel P. Berrangé
Quoted 31 lines · Manos Pitsidianakis
Quoted 13 lines
Quoted 13 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = trueQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
About the Rust style, inspired from the discussion on my
previous
Quoted 3 lines
simpletrace-rust [1], it looks like people prefer the default rust style and use the default check without custom configurations.
Quoted 1 line
More style requirements are also an open, especially for
unstable ones,
Quoted 3 lines
and it would be better to split this part into a separate patch, so that the discussion about style doesn't overshadow the focus on your example.
Quoted 1 line
[1]:
https://lore.kernel.org/qemu-devel/[email protected]/
Quoted 1 line
I had read that discussion and had that in mind. There's no need to worry about format inconsistencies; these options are unstable -nightly only- format options and they don't affect the default rust style (they actually follow it). If you run a stable cargo fmt you will see the code won't change (but might complain that these settings are nightly only).
Quoted 8 lines
Quoted 3 lines
What they do is extra work on top of the default style. If anything ends up incompatible with stable I agree it must be removed, there's no sense in having a custom Rust style when the defaults are so reasonable.
This doesn't make sense. One the one hand you're saying the rules don't have any effect on the code style vs the default, but on the otherhand saying they do "extra work" on top of the default style. Those can't both be true.
Quoted 7 lines
No, I fear there's a confusion here. It means that if you run the stable rustfmt with the default options the code doesn't change. I.e. it does not conflict with the default style.
What it does is group imports, format text in doc comments (which stable rustfmt doesn't touch at all) and also splits long strings into several lines, which are all helpful for e-mail patch reviews.
Ah ok. Are we expecting these options to become part of stable rustfmt ?
I would expect our contributors to primarily be using the Rust toolchain that comes with their distro, and not unstable -nightly toolchain. So I still wonder if this rustfmt config will have much real world benefit ?
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Manos PitsidianakisQuoted 10 lines · Manos Pitsidianakis
This commit adds a re-implementation of
hw/char/pl011.cin Rust.It uses generated Rust bindings (produced by
ninja aarch64-softmmu-generated.rs) to
register itself as a QOM type/class.How to build:
1. Make sure rust, cargo and bindgen (cargo install bindgen-cli) are installed
Ah hah. Nevermind my previous question -- cargo install produces bindgen v0.69.4, quite a bit newer than the ubuntu 22.04 packaged version. I have progressed a bit.
Please bear with me as I attempt to learn Rust in the process of reviewing this. I promise no bikesheding and only to ask silly questions.
Quoted 9 lines
rust/pl011/.cargo/config.toml | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 120 +++++++ rust/pl011/Cargo.toml | 66 ++++ rust/pl011/README.md | 42 +++ rust/pl011/build.rs | 44 +++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 +
First silly question: how much of this is boiler plate that gets moved the moment that the second rust subdirectory is added?
rust/pl011/.cargo/config.toml line 2
Quoted 8 lines
rust/pl011/.cargo/config.toml new file +2
-0@@ -0,0 +1,2 @@ +[build] +rustflags = ["-Crelocation-model=pic", "-Ctarget-feature=+crt-static"]
It seems certain that this is not specific to pl011, and will be commot to other rust subdirectories. Or, given the .cargo directory name, is this generated by cargo and committed by mistake?
rust/pl011/.gitignore line 2
Quoted 8 lines
rust/pl011/.gitignore new file +2
-0@@ -0,0 +1,2 @@ +target +src/generated.rs.inc
Is this a symptom of generating files into the source directory and not build directory?
rust/pl011/Cargo.lock line 2
Quoted 8 lines
rust/pl011/Cargo.lock new file +2
-0@@ -0,0 +1,120 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing.
Second silly question: does this really need to be committed to the repository? It *appears* to be specific to the host+os-version of the build. It is certainly very specific about versions and checksums...
Quoted 19 lines
rust/pl011/Cargo.toml new file +13
-0@@ -0,0 +1,66 @@ +[package] +name = "pl011" +version = "0.1.0" +edition = "2021" +authors = ["Manos Pitsidianakis <[email protected]>"] +license = "GPL-2.0 OR GPL-3.0-or-later" +readme = "README.md" +homepage = "https://www.qemu.org" +description = "pl011 device model for QEMU" +repository = "https://gitlab.com/epilys/rust-for-qemu" +resolver = "2" +publish = false +keywords = []
rust/pl011/Cargo.toml line 23
Quoted 10 lines
rust/pl011/Cargo.toml new file +10
-0+categories = [] + +[lib] +crate-type = ["staticlib"] + +# bilge deps included here to include them with docs +[dependencies] +arbitrary-int = { version = "1.2.7" } +bilge = { version = "0.2.0" } +bilge-impl = { version = "0.2.0" }
Likewise.
Quoted 7 lines
rust/pl011/deny.toml new file +1
-0@@ -0,0 +1,57 @@ +# cargo-deny configuration file
rust/pl011/deny.toml line 11
Quoted 10 lines
rust/pl011/deny.toml new file +10
-0+ +[graph] +targets = [ + "aarch64-unknown-linux-gnu", + "x86_64-unknown-linux-gnu", + "x86_64-apple-darwin", + "aarch64-apple-darwin", + "x86_64-pc-windows-gnu", + "aarch64-pc-windows-gnullvm", +]
Very much likewise.
Since aarch64-pc-windows-gnullvm is not a host that qemu supports, if this is not auto-generated, I am confused.
rust/pl011/rustfmt.toml line 1
Quoted 8 lines
rust/pl011/rustfmt.toml new file +1
-0@@ -0,0 +1 @@ +../rustfmt.toml
Newline. Also, third silly question: is there a way we can avoid replicating this within every rust subdirectory? E.g. some search path option within one of the build tools?
Quoted 5 lines
+++
b/rust/pl011/src/definitions.rs
+++b/rust/pl011/src/device.rs
+++b/rust/pl011/src/device_class.rs
+++b/rust/pl011/src/lib.rs
+++b/rust/pl011/src/memory_ops.rs
Eek! Actual Rust! Skipping until I am better educated.
rust/pl011/src/generated.rs line 5
Quoted 11 lines
rust/pl011/src/generated.rs new file +5
-0@@ -0,0 +1,5 @@ +#[cfg( MESON_GENERATED_RS )] +include! (concat! (env! ("OUT_DIR" ),"/generated.rs" )); + +#[cfg(not( MESON_GENERATED_RS ))] +include! ("generated.rs.inc" );
Is this indicative of Rust not being prepared to separate source and build directories? I'm surprised there would have to be a file in the source to direct the compiler to look for a file in the build.
r~
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 1 line · Richard Henderson
First silly question: how much of this is boiler plate that gets moved the moment that the second rust subdirectory is added?
If my suggestion at https://lore.kernel.org/qemu-devel/CABgObfaP7DRD8dbSKNmUzhZNyxeHWO0MztaW3_EFYt=vf6SbzA@mail.gmail.com/ works, we'd have only two directories that have a Cargo.toml in it. For example it could be rust/qemu/ (common code) and rust/hw/ (code that depends on Kconfig).
I think we can also have a rust/Cargo.toml file as in https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-workspace, and then the various configuration files under rust/ will be valid for all subpackages.
Quoted 4 lines
Quoted 2 lines
+[build]
+rustflags = ["-Crelocation-model=pic", "-Ctarget-feature=+crt-static"]It seems certain that this is not specific to pl011, and will be commot to other rust subdirectories. Or, given the .cargo directory name, is this generated by cargo and committed by mistake?
-Crelocation-mode should be pie. But also, I am not sure it works because I think it's always going to be overridden by cargo_wrapper.py? See https://doc.rust-lang.org/cargo/reference/config.html#buildrustflags.
(I'm not sure what +crt-static is for).
I think the generate_cfg_flags() mechanism of cargo_wrapper.py has to be rewritten from Python to Rust and moved to build.rs (using cargo::rustc-cfg). By doing this, the cfg flags are added to whatever is in .cargo/config.toml, rather than replaced.
Quoted 10 lines
rust/pl011/.gitignore line 2
Quoted 8 lines
rust/pl011/.gitignore new file +2
-0@@ -0,0 +1,2 @@ +target +src/generated.rs.incIs this a symptom of generating files into the source directory and not build directory?
If I understand correctly, Manos considered two possible ways to invoke cargo on the Rust code:
- directly, in which case you need to copy the generated source file to rust/pl011/src/generated.rs.inc, because cargo does not know where the build tree
- do everything through meson, which does the right thing because cargo_wrapper.py knows about the build tree and passes the information.
To avoid this, the first workflow could be adjusted so that cargo can still be invoked directly, _but from the build tree_, not the source tree. For example configure could generate a .../build/.cargo/config.toml with
[env]
MESON_BUILD_ROOT = ".../build"
(extra advantage: -Crelocation-model=pie can be added based on --enable-pie/--disable-pie). configure can also create symlinks in the build tree for the source tree's rust/, Cargo.toml and Cargo.lock.
This allows rust/pl011/src/generated.rs (which probably will become something like rust/common/src/generated.rs) to be:
include!(concat!(env!("MESON_BUILD_ROOT"), "/generated.rs.inc"));when cargo is invoked from the build tree.
Putting everything together you'd have
build/
rust/
.cargo/
config.toml # generated by configure or meson.build
Cargo.toml # workspace generated by configure or meson.build
Cargo.lock # can be either linked to srctree or generated
qemu/ # symlink to srctree/rust/qemu
aarch64-softmmu-hw/
Cargo.toml # generated by meson.build (*)
src/ # symlink to srctree/rust/hw/
i386-softmmu-hw/
Cargo.toml # generated by meson.build
src/ # symlink to srctree/rust/hw/
generated/ # files generated by rust/generated/meson.build
(*) these have to be generated to change the package name, so configure_file() seems like a good match for it.
This is suspiciously similar to what tests/tcg/ looks like, except that tests/tcg/*/Makefile is just a symbolic link. I tried creating a similar directory structure in a toy project, and it seemed to work...
Quoted 1 line
Second silly question: does this really need to be committed to the repository? It *appears* to be specific to the host+os-version of the build. It is certainly very specific about versions and checksums...
Generally the idea is that libraries should not commit it, while applications should commit it. The idea is that the Cargo.lock file reproduces a working configuration, and dependencies are updated to known-good releases when CI passes. I don't think I like this, but it is what it is. I ascribe it to me being from the Jurassic.
But for now I would consider not committing Cargo.lock, because we don't have the infrastructure to do that automatic dependency update (assuming we want it). But we could generate it at release time so that it is included in tarballs, and create the symlink from srctree/rust/Cargo.lock into build/rust/ only if the file is present in the source tree.
Quoted 7 lines
rust/pl011/Cargo.toml line 23
Quoted 7 lines
rust/pl011/Cargo.toml +5
-0+# bilge deps included here to include them with docs +[dependencies] +arbitrary-int = { version = "1.2.7" } +bilge = { version = "0.2.0" } +bilge-impl = { version = "0.2.0" }
This one has to be included, but it is less restrictive than it seems.
It is equivalent to
arbitrary-int = { version = ">=1.2.7, <2.0.0" }
bilge = { version = ">=0.2.0, <0.3.0" }
bilge-impl = { version = ">=0.2.0, <0.3.0" }
That is, it assumes an API breakage when the first nonzero component changes in the version. It is also possible to put a caret in front of the version, so that it's clearer that this is a range; but the behavior is the same.
Paolo
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Quoted 74 lines · Paolo Bonzini
Quoted 2 lines · Richard Henderson
First silly question: how much of this is boiler plate that gets moved the moment that the second rust subdirectory is added?
If my suggestion at https://lore.kernel.org/qemu-devel/CABgObfaP7DRD8dbSKNmUzhZNyxeHWO0MztaW3_EFYt=vf6SbzA@mail.gmail.com/
works, we'd have only two directories that have a Cargo.toml in it. For
example it could berust/qemu/(common code) andrust/hw/(code that depends
onKconfig).I think we can also have a
rust/Cargo.tomlfile as in
https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-workspace,
and then the various configuration files underrust/will be valid for all subpackages.Quoted 6 lines
Quoted 2 lines
+[build]
+rustflags = ["-Crelocation-model=pic", "-Ctarget-feature=+crt-static"]It seems certain that this is not specific to pl011, and will be commot to other rust subdirectories. Or, given the .cargo directory name, is this generated by cargo and committed by mistake?
-Crelocation-mode should be pie. But also, I am not sure it works because I think it's always going to be overridden by
cargo_wrapper.py? See
https://doc.rust-lang.org/cargo/reference/config.html#buildrustflags.(I'm not sure what +crt-static is for).
I think the
generate_cfg_flags()mechanism ofcargo_wrapper.pyhas to be rewritten from Python to Rust and moved tobuild.rs(using cargo::rustc-cfg). By doing this, the cfg flags are added to whatever is in.cargo/config.toml, rather than replaced.Quoted 11 lines
rust/pl011/.gitignore line 2
Quoted 8 lines
rust/pl011/.gitignore new file +2
-0@@ -0,0 +1,2 @@ +target +src/generated.rs.incIs this a symptom of generating files into the source directory and not build directory?
If I understand correctly, Manos considered two possible ways to invoke cargo on the Rust code:
- directly, in which case you need to copy the generated source file to
rust/pl011/src/generated.rs.inc, because cargo does not know where the build tree- do everything through meson, which does the right thing because
cargo_wrapper.pyknows about the build tree and passes the information.To avoid this, the first workflow could be adjusted so that cargo can still be invoked directly,
_butfrom the buildtree_, not the source tree. For example configure could generate a.../build/.cargo/config.tomlwith[env] MESON_BUILD_ROOT = ".../build"(extra advantage: -Crelocation-model=pie can be added based on --enable-pie/--disable-pie). configure can also create symlinks in the build tree for the source tree's
rust/, Cargo.toml and Cargo.lock.This allows
rust/pl011/src/generated.rs(which probably will become something likerust/common/src/generated.rs) to be:include!(concat!(env!("MESON_BUILD_ROOT"), "/generated.rs.inc"));when cargo is invoked from the build tree.
Putting everything together you'd have
Quoted 14 lines
build/
rust/
.cargo/
config.toml # generated by configure or meson.build
Cargo.toml # workspace generated by configure or meson.build
Cargo.lock # can be either linked to srctree or generated
qemu/ # symlink to srctree/rust/qemu
aarch64-softmmu-hw/
Cargo.toml # generated by meson.build (*)
src/ # symlink to srctree/rust/hw/
i386-softmmu-hw/
Cargo.toml # generated by meson.build
src/ # symlink to srctree/rust/hw/
generated/ # files generated byrust/generated/meson.build
If we're generating a build tree to invoke cargo on, can we then avoid creating a completely separate dir hierarchy in the source tree rooted at /rust, and just have rust source within our existing hierarchy.
eg
aarch64-softmmu-hw/
Cargo.toml # generated by meson.build (*)
src/
pl011/ # symlink to srctree/hw/p1011/With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Il mer 19 giu 2024, 18:54 Daniel P. Berrangé <[email protected]> ha scritto:
Quoted 15 lines
Quoted 6 lines
build/
rust/
.cargo/
config.toml # generated by configure or meson.build
Cargo.toml # workspace generated by configure or meson.build
Cargo.lock # can be either linked to srctree or generatedQuoted 8 lines
qemu/ # symlink to srctree/rust/qemu
aarch64-softmmu-hw/
Cargo.toml # generated by meson.build (*)
src/ # symlink to srctree/rust/hw/
i386-softmmu-hw/
Cargo.toml # generated by meson.build
src/ # symlink to srctree/rust/hw/
generated/ # files generated byrust/generated/meson.build
Quoted 5 lines
If we're generating a build tree to invoke cargo on, can we then avoid creating a completely separate dir hierarchy in the source tree rooted at /rust, and just have rust source within our existing hierarchy.
Maybe... I hadn't even considered the possibility of having a single cargo invocation (and thus a cargo workspace in the build tree) until Richard pointed out the duplication in configuration files.
I suppose you could just link rust/aarch64-softmmu-hw to srctree/hw, and have a srctree/hw/lib.rs file in there to prime the search for submodules.
I think the resulting hierarchy would feel a little foreign though. Without seeing the code I can't judge but my impression is that, if we wanted to go that way, I would also move all C code under src/. Perhaps we can consider such a unification later, once we have more experience, but for now keep Rust and C code separate?
Paolo
Quoted 19 lines
eg
aarch64-softmmu-hw/ Cargo.toml # generated by meson.build (*) src/ pl011/ # symlink to srctree/hw/p1011/With regards,
Daniel
--
|: https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-
https://fstop138.berrange.com :|
|: https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
In reply to Manos PitsidianakisHi Manos and all,
Quoted 7 lines · Manos Pitsidianakis
rust/rustfmt.toml new file +0
-0
rust/rustfmt.toml line 7
Quoted 7 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = true
I find it's stiil necessary to strictly limit the width of the lines by "error_on_line_overflow = true" [1].
Currently rustfmt defaults the width limitation with "max_width = 100", but it has bugs and doesn't always work. For example, the line of rust/qemu-api/src/device_class.rs:108 comes to 157 characters and is ignored by rustfmt, which doesn't even fit in one line of my screen!
Of course I think it's feasible to manually review and fix similar cases, but it's definitely better to have readily available tool that can help us rigorously formatted...
[1]: https://github.com/rust-lang/rustfmt/blob/master/Configurations.md#error_on_line_overflow
-Zhao
Re: [RFC PATCH v2 3/5] rust: add PL011 device model
Hey Zhao,
Quoted 27 lines · Zhao Liu
Hi Manos and all,
Quoted 7 lines · Manos Pitsidianakis
rust/rustfmt.toml new file +0
-0rust/rustfmt.toml line 7
Quoted 7 lines
rust/rustfmt.toml new file +7
-0@@ -0,0 +1,7 @@ +edition = "2021" +format_generated_files = false +format_code_in_doc_comments = true +format_strings = true +imports_granularity = "Crate" +group_imports = "StdExternalCrate" +wrap_comments = trueI find it's stiil necessary to strictly limit the width of the lines by "
error_on_line_overflow= true" [1].Currently rustfmt defaults the width limitation with "
max_width= 100", but it has bugs and doesn't always work. For example, the line ofrust/qemu-api/src/device_class.rs:108comes to 157 characters and is ignored by rustfmt, which doesn't even fit in one line of my screen!
Quoted 7 lines
Of course I think it's feasible to manually review and fix similar cases, but it's definitely better to have readily available tool that can help us rigorously formatted...
[1]: https://github.com/rust-lang/rustfmt/blob/master/Configurations.md#error_on_line_overflow
-Zhao
Thank you for pointing that out, I hadn't noticed it! The problem is that macro definitions are also macros and macros aren't formatted because they have to be expanded, IIUC. I agree that a hard error on an overflow is necessary for readable code.
By the way, my usual go-to workaround for this bug is to format the macro body outside the macro_rules! { } scope, (rustfmt works even if the code does not compile) and then put it back in.
Manos
RFC PATCH v2 4/5 DO NOT MERGE: add rustdoc build for gitlab pages
In reply to Manos PitsidianakisDeploy the generated rustdocs for my personal rust qemu fork on gitlab.
The URL is:
https://rust-for-qemu-epilys-aebb06ca9f9adfe6584811c14ae44156501d935ba4.gitlab.io/pl011/index.html
--- .gitlab-ci.d/buildtest.yml | 64 +++++++++++++++++++++++++++----------- 1 file changed, 45 insertions(+), 19 deletions(-)
.gitlab-ci.d/buildtest.yml +45 -19
@@ -715,31 +715,57 @@ build-tools-and-docs-debian: # For contributor forks we want to publish from any repo so # that users can see the results of their commits, regardless # of what topic branch they're currently using +# pages: +# extends: .base_job_template +# image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG +# stage: test +# needs: +# - job: build-tools-and-docs-debian +# script: +# - mkdir -p public +# # HTML-ised source tree +# - make gtags +# # We unset variables to work around a bug in some htags versions +# # which causes it to fail when the environment is large +# - CI_COMMIT_MESSAGE= CI_COMMIT_TAG_MESSAGE= htags +# -anT --tree-view=filetree -m qemu_init +# -t "Welcome to the QEMU sourcecode" +# - mv HTML public/src +# # Project documentation +# - make -C build install DESTDIR=$(pwd)/temp-install +# - mv temp-install/usr/local/share/doc/qemu/* public/ +# artifacts: +# when: on_success +# paths: +# - public +# variables: +# QEMU_JOB_PUBLISH: 1 +# The Docker image that will be used to build your app pages: - extends: .base_job_template - image: $CI_REGISTRY_IMAGE/qemu/debian:$QEMU_CI_CONTAINER_TAG - stage: test - needs: - - job: build-tools-and-docs-debian + image: rust:latest script: - - mkdir -p public - # HTML-ised source tree - - make gtags - # We unset variables to work around a bug in some htags versions - # which causes it to fail when the environment is large - - CI_COMMIT_MESSAGE= CI_COMMIT_TAG_MESSAGE= htags - -anT --tree-view=filetree -m qemu_init - -t "Welcome to the QEMU sourcecode" - - mv HTML public/src - # Project documentation - - make -C build install DESTDIR=$(pwd)/temp-install - - mv temp-install/usr/local/share/doc/qemu/* public/ + - rustup component add rustfmt + - DEBIAN_FRONTEND=noninteractive apt-get update -y + - DEBIAN_FRONTEND=noninteractive apt-get install -y python3-venv meson libgcrypt20-dev zlib1g-dev autoconf automake libtool bison flex git libglib2.0-dev libfdt-dev libpixman-1-dev ninja-build make libclang-14-dev + - cargo install bindgen-cli + - mkdir ./build/ + - cd ./build/ + - ../configure --enable-system --disable-kvm --target-list=aarch64-softmmu --enable-with-rust + - ninja "aarch64-softmmu-generated.rs" + - cp ./aarch64-softmmu-generated.rs ../rust/pl011/src/generated.rs.inc + - cd ../rust/pl011/ + - cargo tree --depth 1 -e normal --prefix none | cut -d' ' -f1 | xargs + printf -- '-p %s\n' | xargs cargo doc --no-deps --document-private-items --target x86_64-unknown-linux-gnu + - cd ./../.. + - mv ./rust/pl011/target/x86_64-unknown-linux-gnu/doc ./public artifacts: when: on_success paths: - public - variables: - QEMU_JOB_PUBLISH: 1 + rules: + # This ensures that only pushes to the default branch will trigger + # a pages deploy + - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH coverity: image: $CI_REGISTRY_IMAGE/qemu/fedora:$QEMU_CI_CONTAINER_TAG
Signature · 1 line
γαῖα πυρί μιχθήτω
RFC PATCH v2 5/5 DO NOT MERGE: replace TYPE_PL011 with x-pl011-rust in arm virt machine
In reply to Manos PitsidianakisConvenience patch for testing the rust device.
--- hw/arm/virt.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/arm/virt.c +4 -0
@@ -912,7 +912,11 @@ static void create_uart(const VirtMachineState *vms, int uart,int irq = vms->irqmap[uart];const char compat[] ="arm,pl011\0arm,primecell" ;const char clocknames[] ="uartclk\0apb_pclk" ; +#ifdef CONFIG_WITH_RUST +DeviceState *dev =qdev_new ("x-pl011-rust" ); +#else DeviceState *dev =qdev_new (TYPE_PL011 ); +#endif SysBusDevice *s =SYS_BUS_DEVICE (dev);MachineState *ms =MACHINE (vms);
Signature · 1 line
γαῖα πυρί μιχθήτω
Quoted 20 lines · Manos Pitsidianakis
.gitignore | 2 + .gitlab-ci.d/buildtest.yml | 64 ++-- MAINTAINERS | 13 + configure | 12 + hw/arm/virt.c | 4 + meson.build | 102 ++++++ meson_options.txt | 4 + rust/meson.build | 93 ++++++ rust/pl011/.cargo/config.toml | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 120 +++++++ rust/pl011/Cargo.toml | 66 ++++ rust/pl011/README.md | 42 +++ rust/pl011/build.rs | 44 +++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 95 ++++++ rust/pl011/src/device.rs | 531 ++++++++++++++++++++++++++++++
Quoted 8 lines
rust/pl011/src/device_class.rs | 95 ++++++ rust/pl011/src/generated.rs | 5 + rust/pl011/src/lib.rs | 581 +++++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 38 +++ rust/rustfmt.toml | 7 + rust/wrapper.h | 39 +++ scripts/cargo_wrapper.py | 221 +++++++++++++ scripts/meson-buildoptions.sh | 6 +
Given the priority of getting the build system correct, what's missing here is updates/integration into our standard GitLab CI pipeline. If that can be shown to be working, that'll give alot more confidence in the overall solution.
Ideally this should not require anything more than updating the docker container definitions to add in the rust toolchain, plus the appropriate std library build for the given target - we cross compiler for every arch we officially care about.
Most of our dockerfiles these days are managed by lcitool, and it has nearly sufficient support for cross compiling with the rust std library. So to start with, this series should modify tests/lcitool/projects/qemu.yml to add
- rust
- rust-std
to the package list, and run 'make lcitool-refresh' to re-create the dockerfiles - see the docs/devel/testing.rst for more info about lcitool if needed.
Assuming these 2 rust packages are in the container, I would then expect QEMU to just "do the right thing" when building this rust code. If it does not, then that's a sign of gaps that need closing.
Getting rid of the need to use --rust-target-triple will be the immediate gap that needs fixing, as CI just passes --cross-prefix for cross-builds and expects everything to be set from that.
The main gap we have is that for Windows I need to update lcitool to pull in the mingw std lib target for rust, which I something I missed when adding rust cross compiler support.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Good morning Daniel,
Quoted 57 lines · Daniel P. Berrangé
Quoted 20 lines · Manos Pitsidianakis
.gitignore | 2 + .gitlab-ci.d/buildtest.yml | 64 ++-- MAINTAINERS | 13 + configure | 12 + hw/arm/virt.c | 4 + meson.build | 102 ++++++ meson_options.txt | 4 + rust/meson.build | 93 ++++++ rust/pl011/.cargo/config.toml | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 120 +++++++ rust/pl011/Cargo.toml | 66 ++++ rust/pl011/README.md | 42 +++ rust/pl011/build.rs | 44 +++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 95 ++++++ rust/pl011/src/device.rs | 531 ++++++++++++++++++++++++++++++Quoted 8 lines
rust/pl011/src/device_class.rs | 95 ++++++ rust/pl011/src/generated.rs | 5 + rust/pl011/src/lib.rs | 581 +++++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 38 +++ rust/rustfmt.toml | 7 + rust/wrapper.h | 39 +++ scripts/cargo_wrapper.py | 221 +++++++++++++ scripts/meson-buildoptions.sh | 6 +Given the priority of getting the build system correct, what's missing here is updates/integration into our standard GitLab CI pipeline. If that can be shown to be working, that'll give alot more confidence in the overall solution.
Ideally this should not require anything more than updating the docker container definitions to add in the rust toolchain, plus the appropriate std library build for the given target - we cross compiler for every arch we officially care about.
Most of our dockerfiles these days are managed by lcitool, and it has nearly sufficient support for cross compiling with the rust std library. So to start with, this series should modify
tests/lcitool/projects/qemu.ymlto add- rust
- rust-stdto the package list, and run 'make lcitool-refresh' to re-create the dockerfiles - see the
docs/devel/testing.rstfor more info about lcitool if needed.Assuming these 2 rust packages are in the container, I would then expect QEMU to just "do the right thing" when building this rust code. If it does not, then that's a sign of gaps that need closing.
Quoted 8 lines
Getting rid of the need to use --rust-target-triple will be the immediate gap that needs fixing, as CI just passes --cross-prefix for cross-builds and expects everything to be set from that.
The main gap we have is that for Windows I need to update lcitool to pull in the mingw std lib target for rust, which I something I missed when adding rust cross compiler support.
Thanks very much for the pointers! I will start dealing with this in the next RFC version.
Re: the target triple, I agree 100%. In fact it wasn't my addition, I kept it from the previous rust RFC patchset that was posted on the list some years ago. It should be possible to construct the triplets ourselves and let the user override if they want to as mentioned in another email.
The rust project has official Docker images, do you think it's something we could use or is it unnecessary?
https://github.com/rust-lang/docker-rust
Thanks,
Manos
Quoted 67 lines · Manos Pitsidianakis
Good morning Daniel,
Quoted 64 lines · Daniel P. Berrangé
Quoted 20 lines · Manos Pitsidianakis
.gitignore | 2 + .gitlab-ci.d/buildtest.yml | 64 ++-- MAINTAINERS | 13 + configure | 12 + hw/arm/virt.c | 4 + meson.build | 102 ++++++ meson_options.txt | 4 + rust/meson.build | 93 ++++++ rust/pl011/.cargo/config.toml | 2 + rust/pl011/.gitignore | 2 + rust/pl011/Cargo.lock | 120 +++++++ rust/pl011/Cargo.toml | 66 ++++ rust/pl011/README.md | 42 +++ rust/pl011/build.rs | 44 +++ rust/pl011/deny.toml | 57 ++++ rust/pl011/meson.build | 7 + rust/pl011/rustfmt.toml | 1 + rust/pl011/src/definitions.rs | 95 ++++++ rust/pl011/src/device.rs | 531 ++++++++++++++++++++++++++++++Quoted 8 lines
rust/pl011/src/device_class.rs | 95 ++++++ rust/pl011/src/generated.rs | 5 + rust/pl011/src/lib.rs | 581 +++++++++++++++++++++++++++++++++ rust/pl011/src/memory_ops.rs | 38 +++ rust/rustfmt.toml | 7 + rust/wrapper.h | 39 +++ scripts/cargo_wrapper.py | 221 +++++++++++++ scripts/meson-buildoptions.sh | 6 +Given the priority of getting the build system correct, what's missing here is updates/integration into our standard GitLab CI pipeline. If that can be shown to be working, that'll give alot more confidence in the overall solution.
Ideally this should not require anything more than updating the docker container definitions to add in the rust toolchain, plus the appropriate std library build for the given target - we cross compiler for every arch we officially care about.
Most of our dockerfiles these days are managed by lcitool, and it has nearly sufficient support for cross compiling with the rust std library. So to start with, this series should modify
tests/lcitool/projects/qemu.ymlto add- rust
- rust-stdto the package list, and run 'make lcitool-refresh' to re-create the dockerfiles - see the
docs/devel/testing.rstfor more info about lcitool if needed.Assuming these 2 rust packages are in the container, I would then expect QEMU to just "do the right thing" when building this rust code. If it does not, then that's a sign of gaps that need closing.
Getting rid of the need to use --rust-target-triple will be the immediate gap that needs fixing, as CI just passes --cross-prefix for cross-builds and expects everything to be set from that.
The main gap we have is that for Windows I need to update lcitool to pull in the mingw std lib target for rust, which I something I missed when adding rust cross compiler support.
I've updated lcitool for this gap:
https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/492
Quoted 4 lines
The rust project has official Docker images, do you think it's something we could use or is it unnecessary?
They're irrelevant - we need the containers with the full set of QEMU build dependencies present, and it is the Rust versions that are in the distros that matter to us.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 5 lines · Manos Pitsidianakis
If
cargoandbindgenis installed in your system, you should be able to build qemu-system-aarch64 with configure flag --enable-rust and launch an arm virt VM. One of the patches hardcodes the default UART of the machine to the Rust one, so if something goes wrong you will see it upon launching qemu-system-aarch64.
What version is required?
On my stock ubuntu 22.04 system, I get
/usr/bin/bindgen aarch64-softmmu_wrapper.h ...
error: Found argument '--formatter' which wasn't expected, or isn't valid in this context
USAGE:
bindgen [FLAGS] [OPTIONS] <header> -- <clang-args>...
$ bindgen --version bindgen 0.59.1
r~
Quoted 11 lines · Richard Henderson
Quoted 5 lines · Manos Pitsidianakis
If
cargoandbindgenis installed in your system, you should be able to build qemu-system-aarch64 with configure flag --enable-rust and launch an arm virt VM. One of the patches hardcodes the default UART of the machine to the Rust one, so if something goes wrong you will see it upon launching qemu-system-aarch64.What version is required?
On my stock ubuntu 22.04 system, I get
Quoted 8 lines
/usr/bin/bindgenaarch64-softmmu_wrapper.h...
error: Found argument '--formatter' which wasn't expected, or isn't valid in this contextUSAGE:
bindgen [FLAGS] [OPTIONS] <header> -- <clang-args>...$ bindgen --version bindgen 0.59.1
I added version checks in the (yet unpublished) next version:
(Which we will also need to match with distro ones whenever possible)
+# FIXME: Latest stable versions, refine to actual minimum ones. +msrv = {
+ 'rustc': '1.79.0', + 'cargo': '1.79.0', + 'bindgen': '0.69.4',
+}
+
+# rustup = find_program('rustup', required: false) +foreach bin_dep: msrv.keys()
+ bin = find_program(bin_dep, required: true)
+ if bin.version() < msrv[bin_dep]
+ # TODO verify behavior
+ if bin == 'bindgen' and get_option('wrap_mode') != 'nodownload'
+ run_command(cargo, 'install', 'bindgen', capture: true, check: true) + bin = find_program(bin_dep, required: true) + if bin.version() >= msrv[bin_dep] + continue + endif + endif + message() + error(bin_dep + ' version ' + bin.version() + ' is unsupported: Please upgrade to at least ' + msrv[bin_dep])
+ endif
+endforeach