Minimal Linux/arm64 VM firmware (written in Rust)
[email protected]
maintainercover Minimal Linux/arm64 VM firmware (written in Rust)
From: Ard Biesheuvel <[email protected]>
One of the tedious bits of booting a virtual machine under KVM on ARM is dealing with guest memory coherency. This is due the fact that running with the MMU off is problematic, as manipulations of memory by the guest are incoherent with the host's cached view of memory. For this reason, KVM needs to keep track of the MMU state of the guest, and perform cache maintenance to the point of coherency (PoC) on all memory that is exposed to the guest (and populated at stage 2) at that point.
Existing VM firmware is often based on bare metal firmware, which sets up page tables with the MMU and caches off, and does the necessary (as well as unnecessary *) cache maintenance to ensure that all manipulations of memory performed with the MMU off are coherent, and not covered by stale cachelines (either clean or dirty) that either obstruct the view of the real memory contents, or are at risk of corrupting them if such dirty cachelines are evicted and written back inadvertently.
As firmware is usually intimately tied to the memory topology of the platform, we can do much better than this. Instead of setting up the initial page tables at runtime, we can bake the into the boot image, provided that it runs at an a priori known address. This means we can enable MMU and caches straight out of reset, and defer all memory accesses that go via the D side until after.
This is the approach taken by this series: it implements a minimal firmware/bootloader for booting a Linux arm64 kernel on QEMU's mach-virt, which does minimal code execution and no memory access (other than instruction fetching) with the MMU disabled. Combined with the series that I sent out recently [0] for Linux, which implements something similar for the kernel itself, virtually all cache maintenance to the PoC can be dropped from the boot flow (with the exception of the .idmap page in the kernel itself). Given that no stores to memory occur at all with the MMU off, KVM should be able to detect that the PoC maintenance is no longer necessary when the MMU is turned on.
This is not only a simplification in itself, it also means that minimal code execution occurs while restricted memory permissions are being
honoured: the firmware boots with WXN protections enabled, and the Rust code itself as well as the text section of the loaded kernel Image need to be mapped with read-only permissions in order to execute them.
This prototype is presented as v0, as it cuts some corners, while the intent is to make this an implementation of EFI that provides all that Linux needs to boot. Most notably,
- only ~900 MiB of DRAM is supported, due to the fact that the page table code I nicked greedily maps down to pages, and the heap is only around 2 MiB, so we run out of memory if we try to map more.
- it boots via the kernel's 'bare metal' entrypoint as EFI features are entirely missing for the moment.
- only uncompressed kernels are supported
How to build and run:
(first, build a kernel with [0] applied, so the image tolerates being booted with MMU and caches enabled)
$ cargo build # using a nightly Rust compiler
$ objcopy -O binary target/aarch64-unknown-linux-gnu/debug/efilite efilite.bin
$ qemu-system-aarch64 \
-M virt,gic-version=host -cpu host -enable-kvm -smp 4 \
-net none -nographic -m 900m -bios efilite.bin -kernel path/to/Image \
-drive if=virtio,file=path/to/hda.xxx,format=xxx -append root=/dev/vda2* U-Boot in particular carries a lot of set/way cache maintenance that was cargo culted from the v7 days, and should never be needed in VM
[0] https://lore.kernel.org/all/[email protected]/
Ard Biesheuvel (6):
Implement a bare metal Rust runtime on top of QEMU's mach-virt Add DTB processing
Add paging code to manage the full ID map
Discover QEMU fwcfg device and use it to load the kernel Remap code section of loaded kernel and boot it Temporarily pass the kaslr seed via register X1
.cargo/config | 5 + .gitignore | 2 + Cargo.lock | 87 ++++ Cargo.toml | 12 + efilite.lds | 62 +++ src/cmo.rs | 37 ++ src/console.rs | 57 +++ src/cstring.rs | 9 + src/fwcfg.rs | 85 ++++ src/head.S | 121 +++++ src/main.rs | 155 +++++- src/pagealloc.rs | 44 ++ src/paging.rs | 499 ++++++++++++++++++++ src/pecoff.rs | 23 + src/ttable.S | 37 ++ 15 files changed, 1233 insertions(+), 2 deletions(-) create mode 100644 .cargo/config create mode 100644 Cargo.lock create mode 100644 efilite.lds create mode 100644 src/cmo.rs create mode 100644 src/console.rs create mode 100644 src/cstring.rs create mode 100644 src/fwcfg.rs create mode 100644 src/head.S create mode 100644 src/pagealloc.rs create mode 100644 src/paging.rs create mode 100644 src/pecoff.rs create mode 100644 src/ttable.S
Signature · 1 line
2.30.2
[email protected]
maintainerRFC PATCH 1/6 Implement a bare metal Rust runtime on top of QEMU's mach-virt
From: Ard Biesheuvel <[email protected]>
Implement the startup sequence to set up a runtime for Rust code, and populate it with a logger wired to the QEMU emulated PL011 UART, and a heap allocator.
The .text and .rodata parts are in emulated NOR flash, and the executable pieces execute in place. The .data and .bss sections as well as the stack are disjoint from the flash image, and reside in DRAM. The assembler startup code sets up the stack pointer and initializes the writable sections.
The startup code programs the MMU with a set of translation tables in NOR flash, which describe a 2M R-X region in flash, a 2 M R-- region covering the base of DRAM (which is where QEMU's mach-virt puts the device tree), and 2M RW- region used for the stack, .data/.bss and the heap.
--- .cargo/config | 5 + .gitignore | 2 + Cargo.lock | 73 ++++++++++++ Cargo.toml | 10 ++ efilite.lds | 61 ++++++++++ src/console.rs | 57 +++++++++ src/cstring.rs | 9 ++ src/head.S | 121 ++++++++++++++++++++ src/main.rs | 49 +++++++- src/ttable.S | 37 ++++++ 10 files changed, 422 insertions(+), 2 deletions(-)
.cargo/config new file +5 -0
@@ -0,0 +1,5 @@ +[target.aarch64-unknown-linux-gnu] +rustflags = ["-C", "relocation-model=static", "-C", "link-arg=-Wl,-Tefilite.lds,--orphan-handling=warn", "-C", "link-arg=-nostartfiles"] + +[build] +target = "aarch64-unknown-linux-gnu"
.gitignore +2 -0
@@ -1 +1,3 @@ /target +*.bin +.*.swp
Cargo.lock new file +73 -0
73 lines not on this page · raw message
Cargo.toml +10 -0
@@ -6,3 +6,13 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] +rlibc = "1.0.0" +linked_list_allocator = "0.9.1" +log = "0.4.14" +mmio = "2.1.0" + +[profile.dev] +panic = "abort" + +[profile.release] +panic = "abort"
efilite.lds new file +61 -0
61 lines not on this page · raw message
src/console.rs new file +57 -0
57 lines not on this page · raw message
src/cstring.rs new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +extern crate rlibc; +use rlibc::memcmp; + +#[no_mangle] +pub unsafe extern "C" fn bcmp (s1: *const u8 , s2: *const u8 , len:usize ) ->i32 { +memcmp (s1, s2, len) +}
src/head.S new file +121 -0
121 lines not on this page · raw message
src/main.rs +47 -2
50 lines not on this page · raw message
src/ttable.S new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + + .set .L_TT_TYPE_BLOCK, 0x1 + .set .L_TT_TYPE_TABLE, 0x3 + + .set .L_TT_AF, 0x1 << 10 + .set .L_TT_NG, 0x3 << 11 + .set .L_TT_RO, 0x2 << 6 + .set .L_TT_XN, 0x3 << 53 + + .set .L_TT_MT_DEV, 0x0 << 2 // MAIR #0 + .set .L_TT_MT_MEM, (0x1 << 2) | (0x3 << 8) // MAIR #1 + + .set BLOCK_XIP, .L_TT_TYPE_BLOCK | .L_TT_MT_MEM | .L_TT_AF | .L_TT_RO + .set BLOCK_DEV, .L_TT_TYPE_BLOCK | .L_TT_MT_DEV | .L_TT_AF | .L_TT_XN + .set BLOCK_MEM, .L_TT_TYPE_BLOCK | .L_TT_MT_MEM | .L_TT_AF | .L_TT_XN | .L_TT_NG + + .section ".rodata", "a", %progbits + .align 12 + /* level 2 */ +0: .quad BLOCK_XIP // 2 MB of R-X flash + .fill 63, 8, 0x0 // 126 MB of unused flash + .set idx, 64 + .rept 448 + .quad BLOCK_DEV | (idx << 21) // 896 MB of RW- device mappings + .set idx, idx + 1 + .endr +1: .quad BLOCK_XIP | 0x40000000 // DT provided by VMM + .quad BLOCK_MEM | 0x40200000 // 2 MB of DRAM + .fill 510, 8, 0x0 + + .globl idmap +idmap: + /* level 1 */ + .quad 0b + .L_TT_TYPE_TABLE // flash and device mappings + .quad 1b + .L_TT_TYPE_TABLE // up to 1 GB of DRAM + .fill 62, 8, 0x0 // 62 GB of remaining VA space
Signature · 1 line
2.30.2
[email protected]
maintainerRFC PATCH 2/6 Add DTB processing
In reply to [email protected]From: Ard Biesheuvel <[email protected]>
Add handling of the QEMU provided DTB, which we will need to consult to find the DRAM layout and the fwcfg device. Initially, just dump the /chosen/bootargs only, if one is provided.
--- Cargo.lock | 7 +++++++ Cargo.toml | 1 + src/main.rs | 10 ++++++++++ 3 files changed, 18 insertions(+)
Cargo.lock +7 -0
@@ -12,12 +12,19 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" name = "efilite" version = "0.1.0" dependencies = [ + "fdt", "linked_list_allocator", "log", "mmio", "rlibc", ] +[[package]] +name = "fdt" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b643857cf70949306b81d7e92cb9d47add673868edac9863c4a49c42feaf3f1e" + [[package]] name = "linked_list_allocator" version = "0.9.1"
Cargo.toml +1 -0
@@ -10,6 +10,7 @@ rlibc = "1.0.0" linked_list_allocator = "0.9.1" log = "0.4.14" mmio = "2.1.0" +fdt = "0.1.3" [profile.dev] panic = "abort"
src/main.rs +10 -0
@@ -13,6 +13,10 @@ use log::{error, info};#[global_allocator] pub static ALLOCATOR :LockedHeap =LockedHeap ::empty (); +extern "C" { +static _dtb:u8 ; +} +#[no_mangle] extern "C" fn efilite_main (base:usize , mapped:usize , used:usize ) {#[cfg(debug_assertions)]
@@ -28,6 +32,12 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) {unsafe {ALLOCATOR .lock ().init (base + used, mapped - used); } + +let fdt =unsafe { fdt::Fdt ::from_ptr (&_dtb).expect ("Failed to parse device tree" ) }; + + fdt.chosen () + .bootargs () + .map (|args|info! ("/chosen/bootargs: {:?}\n" , args)); }#[no_mangle]
Signature · 1 line
2.30.2
[email protected]
maintainerRFC PATCH 3/6 Add paging code to manage the full ID map
In reply to [email protected]From: Ard Biesheuvel <[email protected]>
We enter with a minimal ID map carried in NOR flash, but in order to load anything into DRAM and boot it, we need to map the memory first.
So add a paging module loosely based on the libhermit project's code, but with a few tweaks and necessary fixes. (In particular, remove all the code that is irrelevant for 1:1 mappings)
This code needs some more work: it currently greedily maps all memory ranges down to pages, which is unnecessary, and costly in terms of heap footprint. It also fails to deal with the need to split block mappings into table mappings.
--- Cargo.lock | 7 + Cargo.toml | 1 + efilite.lds | 1 + src/main.rs | 44 ++ src/pagealloc.rs | 44 ++ src/paging.rs | 499 ++++++++++++++++++++ 6 files changed, 596 insertions(+)
Cargo.lock +7 -0
@@ -2,6 +2,12 @@ # It is not intended for manual editing. version = 3 +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + [[package]] name = "cfg-if" version = "1.0.0"
@@ -12,6 +18,7 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" name = "efilite" version = "0.1.0" dependencies = [ + "bitflags", "fdt", "linked_list_allocator", "log",
Cargo.toml +1 -0
@@ -11,6 +11,7 @@ linked_list_allocator = "0.9.1" log = "0.4.14" mmio = "2.1.0" fdt = "0.1.3" +bitflags = "1.3" [profile.dev] panic = "abort"
efilite.lds +1 -0
@@ -27,6 +27,7 @@ SECTIONS .dtb (NOLOAD) : { _dtb = .; . += 0x200000; + _dtb_end = .; } >ram /*
src/main.rs +44 -0
69 lines not on this page · raw message
src/pagealloc.rs new file +44 -0
44 lines not on this page · raw message
src/paging.rs new file +499 -0
499 lines not on this page · raw message
Signature · 1 line
2.30.2
[email protected]
maintainerRFC PATCH 4/6 Discover QEMU fwcfg device and use it to load the kernel
In reply to [email protected]From: Ard Biesheuvel <[email protected]>
QEMU exposes a paravirtualized interface to load various items exposed by the host into the guest. Implement a minimal driver for it, and use it to load the kernel image into DRAM.
--- src/fwcfg.rs | 85 ++++++++++++++++++++ src/main.rs | 18 +++++ 2 files changed, 103 insertions(+)
src/fwcfg.rs new file +85 -0
@@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +use mmio::{Allow ,Deny ,VolBox }; + +pub struct FwCfg { +// read-only data register + data:VolBox <u64 ,Allow ,Deny >, + +// write-only selector register + selector:VolBox <u16 ,Deny ,Allow >, + +// write-only DMA register + dmacontrol:VolBox <u64 ,Deny ,Allow >, +} + +const CFG_KERNEL_SIZE :u16 =0x08 ; +const CFG_KERNEL_DATA :u16 =0x11 ; + +const CFG_DMACTL_DONE :u32 =0 ; +const CFG_DMACTL_ERROR :u32 =1 ; +const CFG_DMACTL_READ :u32 =2 ; + +#[repr( C )] +struct DmaTransfer { + control:u32 , + length:u32 , + address:u64 , +} + +impl FwCfg { +pub fn from_fdt_node (node: fdt::node::FdtNode ) ->Option <FwCfg > { +if let Some (mut iter) = node.reg () { + iter.next ().map (|reg| { +let addr = reg.starting_address; +unsafe { +FwCfg { + data:VolBox ::<u64 ,Allow ,Deny >::new (addras *mut u64 ), + selector:VolBox ::<u16 ,Deny ,Allow >::new (addr.offset (8 )as *mut u16 ), + dmacontrol:VolBox ::<u64 ,Deny ,Allow >::new (addr.offset (16 )as *mut u64 ), + } + } + }) + }else { +None + } + } + +unsafe fn dma_transfer ( + &mut self, + load_address: *mut u8 , + size:usize , + config_item:u16 , + ) ->Result <(), &str > { +let xfer =DmaTransfer { + control: u32::to_be (CFG_DMACTL_READ ), + length: u32::to_be (sizeas u32 ), + address: u64::to_be (load_addressas u64 ), + }; + self.selector.write (u16::to_be (config_item)); + self.dmacontrol.write (u64::to_be (&xferas *const _ as u64 )); + +let control =VolBox ::<u32 ,Allow ,Deny >::new (&xfer.controlas *const _ as *mut u32 ); +loop { +match control.read () { +CFG_DMACTL_DONE =>return Ok (()), +CFG_DMACTL_ERROR =>return Err ("fwcfg DMA error" ), + _ => (),// keep polling + } + } + } + +pub fn get_kernel_size (&mut self) ->usize { + self.selector.write (u16::to_be (CFG_KERNEL_SIZE )); + self.data.read ()as usize + } + +pub fn load_kernel_image (&mut self, load_address: *mut u8 ) ->Result <(), &str > { +let size = self.get_kernel_size (); +if size >0 { +unsafe { self.dma_transfer (load_address, size,CFG_KERNEL_DATA ) } + }else { +Err ("No kernel image provided by fwcfg" ) + } + } +}
src/main.rs +18 -0
@@ -8,6 +8,7 @@mod console;mod cstring; +mod fwcfg;mod pagealloc;mod paging;
@@ -28,6 +29,8 @@ extern "C" {static _dtb_end:u8 ; } +const LOAD_ADDRESS : *mut u8 =0x43210000 as _ ; +#[no_mangle] extern "C" fn efilite_main (base:usize , mapped:usize , used:usize ) {#[cfg(debug_assertions)]
@@ -79,6 +82,21 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) {// Switch to the new ID map so we can use all of DRAM paging::activate (); +let compat = ["qemu,fw-cfg-mmio" ]; +let fwcfg_node = fdt + .find_compatible (&compat) + .expect ("QEMU fwcfg node not found" ); + +info! ("QEMU fwcfg node found: {}\n" , fwcfg_node.name); + +let mut fwcfg = fwcfg::FwCfg ::from_fdt_node (fwcfg_node).expect ("Failed to open fwcfg device" ); + +// TODO allocate fwcfg.get_kernel_size() bytes here instead of using a fixed address + + fwcfg + .load_kernel_image (LOAD_ADDRESS ) + .expect ("Failed to load kernel image" ); +// Switch back to the initial ID map so we can remap // the loaded kernel image with different permissions paging::deactivate ();
Signature · 1 line
2.30.2
[email protected]
maintainerRFC PATCH 5/6 Remap code section of loaded kernel and boot it
In reply to [email protected]From: Ard Biesheuvel <[email protected]>
Implement the bare minimum needed to discover the size of the text/rodata region of the loaded image, and use it to remap this region read-only so that we can execute it while WXN protections are enabled.
Then, boot the loaded image by jumping to the start of it.
--- src/cmo.rs | 37 ++++++++++++++++++++ src/main.rs | 22 ++++++++++++ src/pecoff.rs | 23 ++++++++++++ 3 files changed, 82 insertions(+)
src/cmo.rs new file +37 -0
@@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +use core::arch::asm; + +const CTR_IDC :u64 =1 <<28 ; + +const CTR_DMINLINE_SHIFT :u64 =16 ; +const CTR_DMINLINE_MASK :u64 =0xf ; + +pub fn dcache_clean_to_pou (base: *const u8 , size:isize ) { +let ctr =unsafe { +let mut l:u64 ; +asm! ("mrs {reg}, ctr_el0" ,// CTR: cache type register + reg =out (reg) l, +options (pure, nomem, nostack, preserves_flags), + ); + l + }; + +// Perform the clean only if needed for coherency with the I side +if (ctr &CTR_IDC ) ==0 { +let line_shift =2 + ((ctr >>CTR_DMINLINE_SHIFT ) &CTR_DMINLINE_MASK ); +let line_size:isize =1 << line_shift; +let num_lines = (size + line_size -1 ) >> line_shift; +let mut offset:isize =0 ; + +for _in 1 ..=num_lines { +unsafe { +asm! ("dc cvau, {reg}" , + reg = in(reg) base.offset(offset), +options (nomem, nostack, preserves_flags), + ); + } + offset += line_size; + } + } +}
src/main.rs +22 -0
@@ -6,11 +6,13 @@#![allow(incomplete_features)] #![feature(specialization)] +mod cmo;mod console;mod cstring;mod fwcfg;mod pagealloc;mod paging; +mod pecoff;use core::{arch::global_asm, panic::PanicInfo };use linked_list_allocator::LockedHeap ;
@@ -29,6 +31,8 @@ extern "C" {static _dtb_end:u8 ; } +type EntryFn =unsafe extern "C" fn (*const u8 ,u64 ,u64 ,u64 ) -> !; +const LOAD_ADDRESS : *mut u8 =0x43210000 as _ ;#[no_mangle]
@@ -97,9 +101,27 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { .load_kernel_image(LOAD_ADDRESS) .expect ("Failed to load kernel image" ); +let pe_image = pecoff::Parser ::from_ptr (LOAD_ADDRESS ); + +// Clean the code region of the loaded image to the PoU so we +// can safely fetch instructions from it once the PXN/UXN +// attributes are cleared +let code_size = pe_image.get_code_size (); + cmo::dcache_clean_to_pou (LOAD_ADDRESS , code_sizeas isize ); +// Switch back to the initial ID map so we can remap // the loaded kernel image with different permissions paging::deactivate (); + +// Remap the text/rodata part of the image read-only so we will +// be able to execute it with WXN protections enabled + paging::map_range (LOAD_ADDRESS as u64 , code_size, nor_flags); + paging::activate (); + +unsafe { +let entrypoint:EntryFn = core::mem::transmute (LOAD_ADDRESS ); +entrypoint (&_dtbas *const _ ,0 ,0 ,0 ); + } }#[no_mangle]
src/pecoff.rs new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +pub struct Parser { + base_of_code:u64 , + size_of_code:u64 , +} + +impl Parser { +pub fn from_ptr (ptr: *const u8 ) ->Parser { +// TODO check magic number, arch, etc +// TODO deal with variable PE header offset +let pehdr_offset =64 ; + +Parser { + base_of_code:unsafe { *(ptr.offset (pehdr_offset +28 )as *const u32 ) }as u64 , + size_of_code:unsafe { *(ptr.offset (pehdr_offset +44 )as *const u32 ) }as u64 , + } + } + +pub fn get_code_size (&self) ->u64 { +return self.base_of_code + self.size_of_code; + } +}
Signature · 1 line
2.30.2
[email protected]
maintainerRFC PATCH 6/6 Temporarily pass the kaslr seed via register X1
In reply to [email protected]From: Ard Biesheuvel <[email protected]>
Currently, we boot the kernel via its 'bare metal' entry point, rather than via the EFI entry point, as we haven't implemented EFI yet.
Booting with the MMU enabled requires that the KASLR seed is known before setting up the page tables, as we will do so only once, rather than twice when reading the seed from the DT. For this reason, the EFI stub passes the KASLR seed via register X1 as well as the kaslr-seed property in chosen, and those values need to be in sync.
So as long as we are not using the EFI entry point, pass the DT's kaslr-seed value via register X1 at boot.
--- src/main.rs | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
src/main.rs +13 -1
@@ -118,9 +118,21 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { paging::map_range (LOAD_ADDRESS as u64 , code_size, nor_flags); paging::activate (); +// TODO remove this once we boot via the EFI entry point +// passing the kaslr seed via x1 is part of the stub's internal boot protocol +let kaslr_seed:u64 = { +let mut seed:u64 =0 ; +let chosen = fdt.find_node ("/chosen" ).unwrap (); +if let Some (prop) = chosen.property ("kaslr-seed" ) { + seed = prop.as_usize ().unwrap ()as _ ; +info! ("/chosen/kaslr-seed: {:#x}\n" , seed); + }; + seed + }; +unsafe {let entrypoint:EntryFn = core::mem::transmute (LOAD_ADDRESS ); -entrypoint (&_dtbas *const _ ,0 ,0 ,0 ); +entrypoint (&_dtbas *const _ , kaslr_seed,0 ,0 ); } }
Signature · 1 line
2.30.2