virt: bao: add Bao hypervisor IPC and I/O dispatcher drivers
Subsystems (2)
v5 cover virt: bao: add Bao hypervisor IPC and I/O dispatcher drivers
This series adds guest-side drivers for the Bao static-partitioning
hypervisor: an IPC shared-memory driver, an I/O dispatcher that lets a backend guest service VirtIO I/O for frontend guests, their UAPI and a MAINTAINERS entry.
Bao is a lightweight static-partitioning hypervisor for embedded and safety-critical systems (https://github.com/bao-project).
- The IPC shared-memory driver lets Bao guests exchange data through a shared-memory region split into a read and a write channel, exposed as a misc character device (read(), write(), mmap()). The channels are declared on the kernel command line.
- The I/O dispatcher bridges Bao's Remote I/O mechanism to a userspace
VMM: the VMM creates each device model from /dev/bao, receives the frontend's MMIO accesses and completes them, with ioeventfd/irqfd support for the fast paths.
Changes since v4
----------------
All findings of the Sashiko review of v4 are addressed:
- creating a device model (BAO_IOCTL_CREATE_DM) requires CAP_SYS_RAWIO, is refused under lockdown, rejects shared-memory regions overlapping System RAM and claims the region with request_mem_region(), mirroring /dev/mem;
- the notification line is requested exclusively and is checked against the sources the interrupt controller declares before it is mapped, since irqchip drivers trust the specifiers they are given (a PLIC line above riscv,ndev wrote out of bounds). Its mapping is not disposed when request_irq() fails, though: irq_create_of_mapping() returns the existing mapping of a line that is already mapped without counting its users, so disposing it would free the interrupt of whoever owns the line. A leftover mapping is reused when the line is mapped again;
- a request that cannot be delivered is completed and fetching goes on, so pending requests are no longer stranded;
- the ioeventfds and irqfds of a device model are bounded (256 each) and their flags validated; requests still queued when a client is destroyed are completed back to the hypervisor;
- the IPC driver registers its devices only once every channel has been parsed and mapped, and never unwinds a device it has registered.
Other changes:
- the requests handed to the VMM are tracked: a completion must name one of them and only provides its value, so userspace can no longer pick the hypercall operation (an ASK lost a request), and the requests a VMM leaves behind when it exits are completed on release;
- a device model id can only be bound to one file descriptor at a time;
- RISC-V: the hypercall status is taken from the SBI value (a1), where Bao returns it, not from the SBI error (a0), which is always 0; a failed hypercall was taken as a success. Both drivers depend on RISCV_SBI, so M-mode kernels no longer fail to build them;
- mmap goes through .mmap_prepare and only shared mappings are allowed; .compat_ioctl is set, the structures having the same layout for 32-bit userspace; DM_GET_INFO and IO_CLIENT_ATTACH are _IOR; the ioctl table reserves 0xA7 00-0F;
- bao_ipcshmem.channels= is documented in kernel-parameters.txt;
- rebased on v7.4-rc1.
Changes since v3
----------------
The two design comments on v3 changed the shape of the series, which went from six patches to three.
- No device tree (Krzysztof Kozlowski): both bindings and the "bao" vendor prefix are dropped. The IPC channels are a software contract between the hypervisor and the guest and are declared on the kernel command line (bao_ipcshmem.channels=...). The set of device models a backend serves is a contract between the hypervisor and the VMM, so, following drivers/virt/acrn, the I/O dispatcher exposes a single /dev/bao control device and the VMM creates each device model with BAO_IOCTL_CREATE_DM (shared-memory region + notification line), which returns a per-DM file descriptor; the DM lives as long as the descriptor. The notification line is resolved against the device tree's root interrupt parent, like any device's interrupt.
- No architecture code (Will Deacon): the hypercall helpers live in drivers/virt/bao/bao_hypercall.h and use arm_smccc_hvc() for the IPC hypercall and, because the Remote I/O hypercall returns the request in x1-x6, the SMCCC v1.2 arm_smccc_1_2_hvc(). RISC-V uses sbi_ecall() for IPC and a local ecall for Remote I/O (see open items). 32-bit Arm has no SMCCC v1.2 helper, so the I/O dispatcher is limited to arm64 and RISC-V for now; the IPC driver still supports 32-bit Arm through SMCCC v1.1 (HAVE_ARM_SMCCC).
- The standalone "consolidate the IPC hypercall ID" patch is folded into the driver patches (Andrew Jones).
- All findings of the Sashiko review of v3 are addressed. Details are in the per-patch changelogs.
- The ioctl type is 0xA7: 7.3-rc1 registered 0xA6 for memory allocation profiling.
Testing
-------
Built on v7.4-rc1 with GCC 15 (W=1) for arm64, arm and riscv, at every commit, and with clang 21 (W=1) for arm64 and riscv. Run-time tested under Bao v2.0.0 on QEMU aarch64 virt and QEMU riscv64 virt with the bao-demos demos: the IPC driver exchanging messages in both directions between a Linux guest and a FreeRTOS guest, and the I/O dispatcher with a Linux backend serving console, network and block device models to a FreeRTOS guest and to Linux guests, including killing and restarting the VMMs, plus a userspace test of the /dev/bao error paths (privilege, region and line checks, duplicate ids, limits, completions) run next to the live VMMs. 32-bit Arm is compile-tested only.
Open items
----------
- RISC-V: the Bao SBI extension still uses the experimental extension space (0x08000ba0), and the Remote I/O hypercall returns the request in a2-a7, which does not follow the SBI calling convention (Andrew Jones, v2). Changing that means a hypervisor ABI change (status in a0/a1, request through shared memory); the RISC-V support is experimental until then, or it can be split out of this series if preferred.
- Bao does not implement the SMCCC vendor-hypervisor UID call, so the drivers cannot detect the hypervisor yet; adding it is planned on the hypervisor side.
- The notification line is a line of the interrupt controller the device tree's root node points at through interrupt-parent; a guest whose root node has none cannot use the I/O dispatcher yet.
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
João Peixoto (3):
virt: bao: add IPC shared-memory driver
virt: bao: add I/O dispatcher driver
MAINTAINERS: add Bao hypervisor entry
.../admin-guide/kernel-parameters.txt | 15 + .../userspace-api/ioctl/ioctl-number.rst | 2 + MAINTAINERS | 8 + drivers/virt/Kconfig | 2 + drivers/virt/Makefile | 1 + drivers/virt/bao/Kconfig | 5 + drivers/virt/bao/Makefile | 4 + drivers/virt/bao/bao_hypercall.h | 205 ++++++++ drivers/virt/bao/io-dispatcher/Kconfig | 20 + drivers/virt/bao/io-dispatcher/Makefile | 4 + drivers/virt/bao/io-dispatcher/bao_drv.h | 401 ++++++++++++++++ drivers/virt/bao/io-dispatcher/dm.c | 438 ++++++++++++++++++ drivers/virt/bao/io-dispatcher/driver.c | 71 +++ drivers/virt/bao/io-dispatcher/intc.c | 186 ++++++++ drivers/virt/bao/io-dispatcher/io_client.c | 430 +++++++++++++++++ .../virt/bao/io-dispatcher/io_dispatcher.c | 173 +++++++ drivers/virt/bao/io-dispatcher/ioeventfd.c | 350 ++++++++++++++ drivers/virt/bao/io-dispatcher/irqfd.c | 321 +++++++++++++ drivers/virt/bao/ipcshmem/Kconfig | 16 + drivers/virt/bao/ipcshmem/Makefile | 3 + drivers/virt/bao/ipcshmem/ipcshmem.c | 421 +++++++++++++++++ include/uapi/linux/bao.h | 126 +++++ 22 files changed, 3202 insertions(+) create mode 100644 drivers/virt/bao/Kconfig create mode 100644 drivers/virt/bao/Makefile create mode 100644 drivers/virt/bao/bao_hypercall.h create mode 100644 drivers/virt/bao/io-dispatcher/Kconfig create mode 100644 drivers/virt/bao/io-dispatcher/Makefile create mode 100644 drivers/virt/bao/io-dispatcher/bao_drv.h create mode 100644 drivers/virt/bao/io-dispatcher/dm.c create mode 100644 drivers/virt/bao/io-dispatcher/driver.c create mode 100644 drivers/virt/bao/io-dispatcher/intc.c create mode 100644 drivers/virt/bao/io-dispatcher/io_client.c create mode 100644 drivers/virt/bao/io-dispatcher/io_dispatcher.c create mode 100644 drivers/virt/bao/io-dispatcher/ioeventfd.c create mode 100644 drivers/virt/bao/io-dispatcher/irqfd.c create mode 100644 drivers/virt/bao/ipcshmem/Kconfig create mode 100644 drivers/virt/bao/ipcshmem/Makefile create mode 100644 drivers/virt/bao/ipcshmem/ipcshmem.c create mode 100644 include/uapi/linux/bao.h
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
Signature · 1 line
2.53.0
PATCH v5 1/3 virt: bao: add IPC shared-memory driver
Add a driver that lets guests running on the Bao static-partitioning hypervisor communicate through shared memory. Each guest is assigned a read and a write region within a shared-memory area.
The IPC channels are a pure software contract between the Bao hypervisor and its guests, so they are not described in the device tree. Each channel is instead declared on the kernel command line:
bao_ipcshmem.channels=<channel>[;<channel>...]
where each <channel> is
<id>,<read_base>,<read_size>,<write_base>,<write_size>
The parameter is documented in kernel-parameters.txt.
Userspace accesses the regions through a misc character device using read(), write() and mmap(). The read region is written by the peer and mapped read-only at stage 2, so a writable mapping of it is refused, and so are private mappings, whose writes would never reach the peer. A write() notifies the peer guest through a hypercall issued with the architecture's standard hypervisor call convention: an SMCCC fast call in the vendor-specific hypervisor service range (HVC) on arm/arm64 and an SBI extension call (ecall) on RISC-V. The helpers live in the driver directory, built on the generic SMCCC and SBI support, so no architecture code is needed.
The regions are validated when parsed (page-aligned, not wrapping, not overlapping System RAM, so a channel cannot be pointed at kernel memory). The misc devices are only registered once every channel is set up, and once one of them is registered a later failure only drops its own channel, so module init never fails with a device already open.
---
v5:
- Register the misc devices only once every channel has been parsed and mapped (Sashiko review), and never unregister one from module init:
once a device is registered, a later misc_register() failure only drops its own channel, since the device may already be open. Refuse duplicate channel ids.
- Refuse regions overlapping System RAM (region_intersects()), so a channel cannot be pointed at kernel memory.
- Fail write() with -EIO when the notification hypercall fails.
- Use .mmap_prepare (.mmap is deprecated) and refuse private mappings, whose writes would never reach the peer.
- RISC-V: take the hypercall status from the SBI value (a1), where Bao returns it; the SBI error (a0) only says whether the extension exists.
- Kconfig: depend on RISCV_SBI instead of RISCV; sbi_ecall() does not exist in M-mode kernels, which failed to build the driver.
- Document bao_ipcshmem.channels= in kernel-parameters.txt.
- Rebase on v7.4-rc1; use kzalloc_obj().
v4:
- Drop the device-tree binding and the platform driver; the channels are declared on the kernel command line (bao_ipcshmem.channels=<id>,<read_base>, <read_size>,<write_base>,<write_size>[;...]) and the misc devices are created from module init (Krzysztof Kozlowski).
- Move the hypercall helper out of arch/ into drivers/virt/bao/bao_hypercall.h, built on arm_smccc_hvc() (arm/arm64) and sbi_ecall() (RISC-V); the hypercall ID is defined there, folding the v3 "consolidate the IPC hypercall ID" patch (Will Deacon, Andrew Jones).
- Kconfig: depend on HAVE_ARM_SMCCC || RISCV; the module is now bao_ipcshmem, so the documented parameter name is real.
- Add wmb() before the notify hypercall; refuse writable mappings of the read region and clear VM_MAYWRITE; require page-aligned regions and reject ranges that do not fit phys_addr_t/size_t; add .llseek; enlarge the label buffer; do the mmap offset arithmetic in u64 (Sashiko review).
- Devices live and die with the module and open files pin it, closing the unbind use-after-free of v3.
.../admin-guide/kernel-parameters.txt | 15 + drivers/virt/Kconfig | 2 + drivers/virt/Makefile | 1 + drivers/virt/bao/Kconfig | 3 + drivers/virt/bao/Makefile | 3 + drivers/virt/bao/bao_hypercall.h | 97 ++++ drivers/virt/bao/ipcshmem/Kconfig | 16 + drivers/virt/bao/ipcshmem/Makefile | 3 + drivers/virt/bao/ipcshmem/ipcshmem.c | 421 ++++++++++++++++++ 9 files changed, 561 insertions(+) create mode 100644 drivers/virt/bao/Kconfig create mode 100644 drivers/virt/bao/Makefile create mode 100644 drivers/virt/bao/bao_hypercall.h create mode 100644 drivers/virt/bao/ipcshmem/Kconfig create mode 100644 drivers/virt/bao/ipcshmem/Makefile create mode 100644 drivers/virt/bao/ipcshmem/ipcshmem.c
Documentation/admin-guide/kernel-parameters.txt +15 -0
@@ -621,6 +621,21 @@ Kernel parameters Format: <int> (must be >=0) Default: 64 + bao_ipcshmem.channels= + [ARM,ARM64,RISCV] Shared-memory IPC channels of a + guest of the Bao hypervisor (CONFIG_BAO_SHMEM), each + exposed as /dev/baoipc<id>. + Format: <channel>[;<channel>...] + where <channel> is + <id>,<read_base>,<read_size>,<write_base>,<write_size> + <id> is the channel id assigned by the hypervisor + and passed to its notification hypercall. The read + region is written by the peer guest, the write + region by this one. Bases and sizes are physical + addresses and sizes, decimal or 0x-prefixed hex, + page-aligned and outside System RAM. + Example: 0,0xf0000000,0x2000,0xf0002000,0x2000 + bdev_allow_write_mounted= Format: <bool> Control the ability to open a mounted block device
drivers/virt/Kconfig +2 -0
@@ -64,6 +64,8 @@ source "drivers/virt/nitro_enclaves/Kconfig" source "drivers/virt/acrn/Kconfig" +source "drivers/virt/bao/Kconfig" + endif source "drivers/virt/coco/Kconfig"
drivers/virt/Makefile +1 -0
@@ -11,3 +11,4 @@ obj-y += vboxguest/ obj-$(CONFIG_NITRO_ENCLAVES) += nitro_enclaves/ obj-$(CONFIG_ACRN_HSM) += acrn/ obj-y += coco/ +obj-y += bao/
drivers/virt/bao/Kconfig new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +source "drivers/virt/bao/ipcshmem/Kconfig"
drivers/virt/bao/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_BAO_SHMEM) += ipcshmem/
drivers/virt/bao/bao_hypercall.h new file +97 -0
97 lines not on this page · raw message
drivers/virt/bao/ipcshmem/Kconfig new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0 +config BAO_SHMEM + tristate "Bao hypervisor shared memory support" + depends on HAVE_ARM_SMCCC || RISCV_SBI + help + This enables support for Bao shared memory communication. + It allows the kernel to interface with guests running under + the Bao hypervisor, providing a character device interface + for exchanging data through dedicated shared-memory regions. + Channels are declared on the kernel command line via + "bao_ipcshmem.channels=". + + To compile this driver as a module, choose M here: the module + will be called bao_ipcshmem. + + If unsure, say N.
drivers/virt/bao/ipcshmem/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_BAO_SHMEM) += bao_ipcshmem.o +bao_ipcshmem-y := ipcshmem.o
drivers/virt/bao/ipcshmem/ipcshmem.c new file +421 -0
421 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v5 2/3 virt: bao: add I/O dispatcher driver
In reply to João PeixotoAdd the Bao I/O dispatcher, used by backend VMs to service I/O on behalf of frontend guests. It bridges Bao's Remote I/O mechanism to userspace VirtIO backend device models.
The set of device models a backend serves is a software contract between the hypervisor and the userspace VMM, not hardware, so it is not described in the device tree. Following the model used by other hypervisor drivers (e.g. drivers/virt/acrn), the driver exposes a single /dev/bao control device; the VMM creates each device model from its own configuration with the BAO_IOCTL_CREATE_DM ioctl, which returns a per-DM file descriptor. Each device model has a contiguous shared-memory region for exchanging I/O buffers with its frontend and an interrupt the hypervisor uses to signal pending requests; both are provided by the VMM at creation time. The interrupt is given as a line of the system's root interrupt controller (the one the device tree's root node points at through interrupt-parent) and mapped by the driver. Userspace then drives the device model through a set of ioctls on that descriptor.
Creating a device model maps a physical region chosen by the VMM into userspace and claims an interrupt line, so BAO_IOCTL_CREATE_DM requires CAP_SYS_RAWIO, is refused under lockdown (LOCKDOWN_DEV_MEM), rejects regions overlapping System RAM and claims the region with request_mem_region(), mirroring /dev/mem. The line is checked against the sources the interrupt controller declares, since irqchip drivers trust the specifiers they are given, and is requested exclusively, so one already in use cannot be taken over. A DM id can only be bound to one descriptor at a time.
The queued requests, ioeventfds and irqfds of a device model are bounded, and a request that cannot be delivered (no client, full queue, teardown) is completed back to the hypervisor so the frontend never hangs. The requests handed to userspace are tracked until it completes
them: a completion only takes the value from userspace, and the requests a VMM leaves behind when it exits are completed by the kernel.
The Remote I/O hypercall returns the pending request in x1-x6 on top of the status in x0, so on arm64 it is issued as an SMCCC v1.2 call through arm_smccc_1_2_hvc(); no architecture code is needed. 32-bit Arm has no SMCCC v1.2 helper, so the I/O dispatcher is limited to arm64 and RISC-V for now. On RISC-V the request comes back in a2-a7 through the (experimental) Bao SBI extension, which sbi_ecall() cannot express, hence the local ecall helper. The extension always succeeds from the SBI point of view and reports the hypercall status as the SBI value, so that is where both helpers take it from.
---
v5:
- BAO_IOCTL_CREATE_DM requires CAP_SYS_RAWIO, is refused under lockdown (LOCKDOWN_DEV_MEM), validates the shared-memory region (non-empty, page-aligned, not wrapping, not overlapping System RAM) and claims it with request_mem_region() (Sashiko review).
- Check the notification line against the sources the root interrupt controller declares (riscv,ndev for a PLIC, riscv,num-sources for an APLIC, SPIs 0-987 for a GIC) before mapping it: irqchip drivers do not validate specifiers, and an out-of-range PLIC line wrote past the end of its enable-state array. Request the line exclusively (Sashiko review).
- Never dispose the IRQ mapping. irq_create_of_mapping() returns the existing mapping of a line that is already mapped without counting its users, so disposing it after a failed request_irq() freed the interrupt of the DM or driver owning the line. A leftover mapping is reused when the same line is mapped again.
- Refuse a second file descriptor for a DM id in use (-EEXIST).
- Keep fetching after a request cannot be delivered: complete it and return the pending count instead of an error (Sashiko review).
- Bound the ioeventfds and irqfds per DM (256 each, -ENOSPC); validate their flags and the reserved field (Sashiko review).
- Complete the requests still queued when a client is destroyed instead of dropping them (Sashiko review).
- Track the requests handed to userspace: IO_REQUEST_COMPLETE only completes a read or write handed out on that file descriptor (-ENOENT otherwise) and only takes the value from userspace, and the requests a VMM leaves behind when it exits are completed on release. Userspace could previously issue an ASK through it, which lost a request.
- RISC-V: take the Remote I/O hypercall status from a1, where Bao returns it, instead of the always-successful SBI error in a0. An ASK with nothing pending (every DM teardown, or a racing notification) was taken as a success and its stale registers dispatched as a request. Leave the context untouched on failure on both architectures.
- ioeventfd deassign also matches the data match and returns -ENOENT when nothing matches; irqfd passes the DM id as u32 and uses key_to_poll().
- Kconfig: depend on RISCV_SBI instead of RISCV.
- Use .mmap_prepare and refuse private mappings; set .compat_ioctl to compat_ptr_ioctl, the structures having the same layout for 32-bit userspace.
- DM_GET_INFO and IO_CLIENT_ATTACH are _IOR (they only write to userspace); define the request operations (BAO_IO_WRITE, BAO_IO_READ) in the UAPI header; reserve 0xA7 00-0F instead of the whole type; use fixed_size_llseek(); copy the ioctl arguments on the stack; pr_fmt prefix for the messages.
- Rebase on v7.4-rc1; use kzalloc_obj().
v4:
- Drop the binding and the platform device: a single /dev/bao control device; BAO_IOCTL_CREATE_DM takes {shmem_addr, shmem_size, id, irq} from the VMM and returns a per-DM anonymous-inode fd; the DM lives as long as the fd (Krzysztof Kozlowski; also removes the unbind use-after-free).
- Resolve the notification line against the device tree's root interrupt parent instead of binding to a node (1-cell PLIC, 2-cell APLIC and 3/4-cell GIC specifiers).
- Issue the Remote I/O hypercall with arm_smccc_1_2_hvc() on arm64 (the request is returned in x1-x6, an SMCCC v1.2 call); no arch/ code. RISC-V keeps a local ecall since sbi_ecall() only exposes a0/a1. 32-bit Arm is not supported by this driver for now (no SMCCC v1.2 helper) (Will Deacon).
- UAPI: ioctl type 0xA7 (0xA6 is taken by alloc_tag since 7.3-rc1); struct bao_dm_info reordered to remove implicit padding, ioeventfd and irqfd flag values exported, ioeventfd fd signed, linux/ioctl.h included.
- Sashiko review: take the DM id from the fd, not from userspace; hold the irqfd lock across vfs_poll(), single-shot cleanup work and flush before destroy_workqueue(); per-DM interrupt handler; persistent request_irq() name; return after ioeventfd deassign and match the full {fd, addr, len}; cap queued requests; propagate -ERESTARTSYS; free pending requests on destroy; drop the unused kernel mapping of the shared memory; complete requests that cannot be delivered so the frontend does not hang.
- Found while testing: the kernel thread only exits through kthread_stop(); per-DM workqueue/work instead of static arrays (no more 16-DM limit); WQ_PERCPU on the dispatcher workqueue; fix the range-list walk type in bao_io_client_destroy(); module renamed to bao_io_dispatcher; ERR_PTR() errors from bao_dm_create().
.../userspace-api/ioctl/ioctl-number.rst | 2 + drivers/virt/bao/Kconfig | 2 + drivers/virt/bao/Makefile | 1 + drivers/virt/bao/bao_hypercall.h | 108 +++++ drivers/virt/bao/io-dispatcher/Kconfig | 20 + drivers/virt/bao/io-dispatcher/Makefile | 4 + drivers/virt/bao/io-dispatcher/bao_drv.h | 401 ++++++++++++++++ drivers/virt/bao/io-dispatcher/dm.c | 438 ++++++++++++++++++ drivers/virt/bao/io-dispatcher/driver.c | 71 +++ drivers/virt/bao/io-dispatcher/intc.c | 186 ++++++++ drivers/virt/bao/io-dispatcher/io_client.c | 430 +++++++++++++++++ .../virt/bao/io-dispatcher/io_dispatcher.c | 173 +++++++ drivers/virt/bao/io-dispatcher/ioeventfd.c | 350 ++++++++++++++ drivers/virt/bao/io-dispatcher/irqfd.c | 321 +++++++++++++ include/uapi/linux/bao.h | 126 +++++ 15 files changed, 2633 insertions(+) create mode 100644 drivers/virt/bao/io-dispatcher/Kconfig create mode 100644 drivers/virt/bao/io-dispatcher/Makefile create mode 100644 drivers/virt/bao/io-dispatcher/bao_drv.h create mode 100644 drivers/virt/bao/io-dispatcher/dm.c create mode 100644 drivers/virt/bao/io-dispatcher/driver.c create mode 100644 drivers/virt/bao/io-dispatcher/intc.c create mode 100644 drivers/virt/bao/io-dispatcher/io_client.c create mode 100644 drivers/virt/bao/io-dispatcher/io_dispatcher.c create mode 100644 drivers/virt/bao/io-dispatcher/ioeventfd.c create mode 100644 drivers/virt/bao/io-dispatcher/irqfd.c create mode 100644 include/uapi/linux/bao.h
Documentation/userspace-api/ioctl/ioctl-number.rst +2 -0
@@ -349,6 +349,8 @@ Code Seq# Include File Comments <mailto:[email protected]> 0xA6 00-0F uapi/linux/alloc_tag.h Memory allocation profiling <mailto:[email protected]> +0xA7 00-0F uapi/linux/bao.h Bao hypervisor + <mailto:[email protected]> 0xAA 00-3F linux/uapi/linux/userfaultfd.h 0xAB 00-1F linux/nbd.h 0xAC 00-1F linux/raw.h
drivers/virt/bao/Kconfig +2 -0
@@ -1,3 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 source "drivers/virt/bao/ipcshmem/Kconfig" + +source "drivers/virt/bao/io-dispatcher/Kconfig"
drivers/virt/bao/Makefile +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_BAO_SHMEM) += ipcshmem/ +obj-$(CONFIG_BAO_IO_DISPATCHER) += io-dispatcher/
drivers/virt/bao/bao_hypercall.h +108 -0
126 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/Kconfig new file +20 -0
@@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0 +config BAO_IO_DISPATCHER + tristate "Bao Hypervisor I/O Dispatcher" + depends on ARM64 || RISCV_SBI + select EVENTFD + help + The Bao I/O Dispatcher is a kernel module for backend Linux VMs + running under the Bao hypervisor. It establishes the connection + between the Remote I/O system (Bao's mechanism for forwarding + I/O requests from frontend VMs to the backend VMs) and the + VirtIO backend device. + + This provides a unified API to support various VirtIO backends, + allowing Bao guests to perform I/O through the hypervisor + transparently. + + To compile this driver as a module, choose M here: the module + will be called bao_io_dispatcher. + + If unsure, say N.
drivers/virt/bao/io-dispatcher/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_BAO_IO_DISPATCHER) += bao_io_dispatcher.o +bao_io_dispatcher-y := driver.o dm.o intc.o io_client.o io_dispatcher.o \ + ioeventfd.o irqfd.o
drivers/virt/bao/io-dispatcher/bao_drv.h new file +401 -0
401 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/dm.c new file +438 -0
438 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/driver.c new file +71 -0
71 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/intc.c new file +186 -0
186 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/io_client.c new file +430 -0
430 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/io_dispatcher.c new file +173 -0
173 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/ioeventfd.c new file +350 -0
350 lines not on this page · raw message
drivers/virt/bao/io-dispatcher/irqfd.c new file +321 -0
321 lines not on this page · raw message
include/uapi/linux/bao.h new file +126 -0
126 lines not on this page · raw message
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH v5 2/3] virt: bao: add I/O dispatcher driver
Thank you for your contribution! Sashiko AI review found 7 potential issues to consider: - [Medium] virt: bao: race between attach error path and complete in bao_dm_ioctl() - [Medium] virt: bao…
Thank you for your contribution! Sashiko AI review found 7 potential issues to consider:
- [Medium] virt: bao: race between attach error path and complete in bao_dm_ioctl()
- [Medium] virt: bao: missing 32-bit size_t truncation check in bao_dm_info_valid()
- [Medium] virt: bao: 32-bit unsigned long truncation on RV32 in bao_remio_hypercall()
- [Medium] virt: bao: broken client teardown order in bao_io_client_destroy()
- [Medium] virt: bao: missing eventfd read and -ENOENT in bao_irqfd_wakeup()/deassign()
- [Medium] virt: bao: broken conflict and datamatch handling in bao_ioeventfd_handler()
- [Low] virt: bao: range overflow and width truncation in bao_ioeventfd_match()
--
commit c7f53fde4f821f921b87ccd26ae05833f79dbbe0
Author: João Peixoto <[email protected]>
virt: bao: add I/O dispatcher driver
This commit adds the Bao hypervisor I/O dispatcher driver, exposing a /dev/bao control device and per-device-model file descriptors to bridge Bao Remote I/O requests, ioeventfds, and irqfds to userspace VirtIO backends.
[Severity: Medium]
In drivers/virt/bao/io-dispatcher/Kconfig (line 4), BAO_IO_DISPATCHER depends on ARM64 || RISCV_SBI, which allows building the driver on 32-bit RISC-V (CONFIG_32BIT=y && CONFIG_RISCV_SBI=y). Can bao_remio_hypercall() in drivers/virt/bao/bao_hypercall.h (line 174) and the dm->inflight XArray operations in drivers/virt/bao/io-dispatcher/dm.c (lines 53, 144, and 166) silently truncate 64-bit values on RV32?
In bao_remio_hypercall(), the u64 fields of struct bao_remio_hypercall_ctx (dm_id, addr, op, value, request_id) are loaded into unsigned long registers a0-a4, truncating 64-bit MMIO addresses, 8-byte I/O values, and request IDs to 32 bits. Similarly, xa_insert() and xa_erase() on dm->inflight take an unsigned long index, truncating the upper 32 bits of req->request_id. Should BAO_IO_DISPATCHER depend on ARM64 || (RISCV_SBI && 64BIT)?
[Severity: Medium]
In drivers/virt/bao/io-dispatcher/dm.c, within bao_dm_ioctl() at lines 132-184, can the BAO_IOCTL_IO_CLIENT_ATTACH error path race with a concurrent BAO_IOCTL_IO_REQUEST_COMPLETE on the same file descriptor and complete the same request twice?
CPU0 CPU1bao_dm_ioctl(IO_CLIENT_ATTACH)
bao_dm_inflight_add(dm, &req)
bao_dm_ioctl(IO_REQUEST_COMPLETE)
xa_erase(&dm->inflight, id)
bao_remio_hypercall(&ctx) copy_to_user(...) -> -EFAULT kfree(xa_erase(&dm->inflight, id)) bao_io_request_complete_error(...) <-- duplicate completion
bao_dm_inflight_add() publishes req.request_id into dm->inflight before copy_to_user(). If another thread completes that request_id while copy_to_user() faults, xa_erase() on the error path returns NULL (or erases a newly attached request if the hypervisor reused request_id), and bao_io_request_complete_error() issues a second completion hypercall to the hypervisor. Copy req to userspace before inserting it into dm->inflight, or only call bao_io_request_complete_error() when xa_erase() returns the entry.
[Severity: Medium]
In drivers/virt/bao/io-dispatcher/dm.c, within bao_dm_info_valid() at line 348, can a 64-bit info->shmem_size with zero lower 32 bits bypass validation on 32-bit builds by truncating to 0 in size_t and unsigned long?
When sizeof(size_t) < sizeof(u64), passing info->shmem_addr = 0 and info->shmem_size = 0x100000000ULL passes !info->shmem_size, PAGE_ALIGNED(info->shmem_size), and upper_32_bits(end) == 0, but truncates to size 0 in region_intersects() and request_mem_region() while leaving dm->info.shmem_size set to 4 GiB. Adding a check for sizeof(size_t) < sizeof(u64) && upper_32_bits(info->shmem_size) (matching bao_ipcshmem_region_valid()) would prevent this.
[Severity: Medium]
In drivers/virt/bao/io-dispatcher/io_client.c, within bao_io_client_destroy() at lines 140-161, does the teardown sequence destroy dm->ioeventfds before stopping the kernel thread and then prematurely resume the I/O dispatcher during DM teardown?
First, at line 140, bao_io_client_destroy() calls bao_ioeventfd_client_destroy(dm) before kthread_stop(client->thread), emptying and freeing dm->ioeventfds while bao_io_client_kernel_thread() is still draining queued requests via bao_ioeventfd_handler(), which silently drops those eventfd notifications.
Second, at line 161, bao_io_client_destroy() calls bao_io_dispatcher_resume(dm), which re-installs dm->intc_handler and queues dm->io_work on dm->io_wq. Because bao_io_client_destroy() is only called from bao_io_clients_destroy() during full DM teardown (in bao_dm_destroy() or the error unwind of bao_dm_create()), resuming the dispatcher mid-teardown causes io_dispatcher() to run BAO_IO_ASK hypercalls when a BAO_IOCTL_CREATE_DM call fails after creating control_client (for example, when bao_intc_init() fails), and during bao_dm_release() it re-queues dm->io_work between destroying ioeventfd_client and control_client, misrouting or error-completing requests before bao_io_dispatcher_destroy() finally drains the workqueue. Pausing or destroying the dispatcher at the start of bao_dm_destroy() (before completing inflight requests and destroying clients) and removing bao_io_dispatcher_resume() from bao_io_client_destroy() would avoid this.
[Severity: Low]
In drivers/virt/bao/io-dispatcher/io_client.c, within bao_io_request_in_range() at line 398, and drivers/virt/bao/io-dispatcher/ioeventfd.c, within bao_ioeventfd_match() at line 124 and bao_ioeventfd_handler() at line 294, can a malformed or top-of-address-space request wrap around in the range check or truncate req->access_width?
In bao_io_request_in_range(), evaluating (req->addr + req->access_width -
1) <= range->end without checking that req->access_width > 0 and does not overflow u64 allows a zero access_width or an address near U64_MAX to wrap around and falsely match the range. In addition, passing u64 req->access_width into the signed int len parameter of bao_ioeventfd_match() and testing p->length >= len without requiring len > 0 allows a zero or negative-truncated length to match any registered ioeventfd.
[Severity: Medium]
In drivers/virt/bao/io-dispatcher/ioeventfd.c, within bao_ioeventfd_is_conflict() at line 103 and bao_ioeventfd_handler() at line 292, are conflicting eventfds at the same address allowed, and are non-matching writes to a datamatch range silently dropped?
First, bao_ioeventfd_is_conflict() checks p->eventfd == ioeventfd->eventfd, which allows registering two different eventfds for the exact same address and data match even though bao_ioeventfd_match() only ever signals the first matching entry on dm->ioeventfds.
Second, bao_ioeventfd_assign() adds [addr, addr + length - 1] to dm->ioeventfd_client, and bao_io_client_find() in drivers/virt/bao/io-dispatcher/io_client.c (line 415) routes every request in that address range to dm->ioeventfd_client without checking whether any ioeventfd matches the request's data value, exact address, or access width. When a guest writes a virtqueue index with no registered ioeventfd to a shared QueueNotify register that has a BAO_IOEVENTFD_FLAG_DATAMATCH entry, bao_ioeventfd_handler() finds no match in bao_ioeventfd_match() and returns 0, completing the hypercall and dropping the write instead of forwarding it to dm->control_client.
[Severity: Medium]
In drivers/virt/bao/io-dispatcher/irqfd.c, within bao_irqfd_wakeup() at line 86 and bao_irqfd_deassign() at line 237, does irqfd signaling leave the eventfd counter unconsumed, and does deassigning an unknown irqfd succeed?
In bao_irqfd_wakeup(), EPOLLIN triggers bao_irqfd_inject() without calling eventfd_ctx_do_read() to consume the eventfd counter, leaving the eventfd permanently readable (and eventually saturating its counter at ULLONG_MAX
- 1). Also, in bao_irqfd_deassign(), if no matching entry is found in dm->irqfds, the function still flushes dm->irqfd_server and returns 0 instead of -ENOENT.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
PATCH v5 3/3 MAINTAINERS: add Bao hypervisor entry
In reply to João PeixotoAdd a MAINTAINERS entry covering the Bao hypervisor drivers and their UAPI header.
---
v5:
- No change.
v4:
- Drop the arch/*/include/asm/bao.h and include/linux/bao.h entries (the files are gone); use the directory form for drivers/virt/bao/.
MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+)
MAINTAINERS +8 -0
@@ -4712,6 +4712,14 @@ F: drivers/video/backlight/ F: include/linux/backlight.h F: include/linux/pwm_backlight.h +BAO HYPERVISOR +M: José Martins <[email protected]> +M: David Cerdeira <[email protected]> +M: João Peixoto <[email protected]> +S: Maintained +F: drivers/virt/bao/ +F: include/uapi/linux/bao.h + BARCO P50 GPIO DRIVER M: Santosh Kumar Yadav <[email protected]> M: Peter Korsgaard <[email protected]>
Signature · 1 line
2.53.0