memory hole close to 4GB on qemu risc-v
Hi All,
I am working on enabling PCIe passthrough on qemu risc-v, in order for the guest to access the host x86 pci resource directly, ram on guest won't cover this range, so if guest has 4GB ram, two ranges are created:
1. 2G-3G
2. 4G-7G
u-boot is not able to handle this but reports:
Reserving 869k for U-Boot at: ff725000
initcall: 0000000080216f94
Reserving 8320k for malloc() at: fef05000
initcall: 0000000080216ee8
Unhandled exception: Store/AMO access fault
EP: 0000000080201a38 RA: 0000000080216f10 TVAL: 00000000fef04f90
It looks like fdtdec_setup_mem_size_base() in u-boot sets ram_base/size without considering all memory nodes, and board_get_usable_ram_top() returns 4G-1 that causes gd->relocaddr set to the wrong address which has no ram backed. More details can be found here:
https://patchew.org/QEMU/[email protected]/
Can we add support for this multi-range ram setting? Any comments?
Thanks,
Fei.
Quoted 18 lines · Fei
Hi All,
I am working on enabling PCIe passthrough on qemu risc-v, in order for the guest to access the host x86 pci resource directly, ram on guest won't cover this range, so if guest has 4GB ram, two ranges are created:
1. 2G-3G
2. 4G-7Gu-boot is not able to handle this but reports:
Reserving 869k for U-Boot at:
ff725000
initcall:0000000080216f94
Reserving 8320k formalloc()at:fef05000
initcall:0000000080216ee8
Unhandled exception: Store/AMO access faultEP:
0000000080201a38RA:0000000080216f10TVAL:00000000fef04f90
Quoted 9 lines
It looks like
fdtdec_setup_mem_size_base()in u-boot sets ram_base/size without considering all memory nodes, andboard_get_usable_ram_top()returns 4G-1 that causesgd->relocaddrset to the wrong address which has no ram backed. More details can be found here:
https://patchew.org/QEMU/[email protected]/Can we add support for this multi-range ram setting? Any comments?
The test command is:
qemu-system-riscv64 -machine virt -nographic -m 4G -smp 4 \
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \
-kernel /home/wufei/nvme/u-boot/u-boot.bin \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
-drive
file=ubuntu-22.04.1-preinstalled-server-riscv64+unmatched.img,if=virtio,format=raw
Here is a quick fix, is it good?
arch/riscv/cpu/generic/dram.c +1 -1
@@ -13,7 +13,7 @@ DECLARE_GLOBAL_DATA_PTR;int dram_init (void ) { -return fdtdec_setup_mem_size_base (); +return fdtdec_setup_mem_size_base_lowest (); }int dram_init_banksize (void )
(quoted diff) +0 -0
b/configs/qemu-riscv64_smode_defconfig
index 1d0f021ade..de08a49dab 100644
configs/qemu-riscv64_smode_defconfig +1 -1
@@ -1,6 +1,6 @@ CONFIG_RISCV=y CONFIG_SYS_MALLOC_LEN=0x800000 -CONFIG_NR_DRAM_BANKS=1 +CONFIG_NR_DRAM_BANKS=2 CONFIG_HAS_CUSTOM_SYS_INIT_SP_ADDR=y CONFIG_CUSTOM_SYS_INIT_SP_ADDR=0x80200000 CONFIG_ENV_SIZE=0x20000
Thanks,
Fei.