Skip to contentSkip to conversation

memory hole close to 4GB on qemu risc-v

DISCUSSION Wu, Fei 1 participant 2 messages Aug 7, 2023 – Aug 8, 2023 RISC-V

lore
Share & tools
lore

Wu, Fei

author
#0 lore raw

Hi All,

I am working on enabling PCIe passthrough on qemu risc-v, in order for the guest to access the host x86 pci resource directly, ram on guest won't cover this range, so if guest has 4GB ram, two ranges are created:
1. 2G-3G
2. 4G-7G

u-boot is not able to handle this but reports:

Reserving 869k for U-Boot at: ff725000
initcall: 0000000080216f94
Reserving 8320k for malloc() at: fef05000
initcall: 0000000080216ee8
Unhandled exception: Store/AMO access fault

EP: 0000000080201a38 RA: 0000000080216f10 TVAL: 00000000fef04f90

It looks like fdtdec_setup_mem_size_base() in u-boot sets ram_base/size without considering all memory nodes, and board_get_usable_ram_top() returns 4G-1 that causes gd->relocaddr set to the wrong address which has no ram backed. More details can be found here:
https://patchew.org/QEMU/[email protected]/

Can we add support for this multi-range ram setting? Any comments?

Thanks,
Fei.

author
#1 lore raw
Quoted 18 lines · Fei

Hi All,

I am working on enabling PCIe passthrough on qemu risc-v, in order for the guest to access the host x86 pci resource directly, ram on guest won't cover this range, so if guest has 4GB ram, two ranges are created:
1. 2G-3G
2. 4G-7G

u-boot is not able to handle this but reports:

Reserving 869k for U-Boot at: ff725000
initcall: 0000000080216f94
Reserving 8320k for malloc() at: fef05000
initcall: 0000000080216ee8
Unhandled exception: Store/AMO access fault

EP: 0000000080201a38 RA: 0000000080216f10 TVAL: 00000000fef04f90

Quoted 9 lines

It looks like fdtdec_setup_mem_size_base() in u-boot sets ram_base/size without considering all memory nodes, and board_get_usable_ram_top() returns 4G-1 that causes gd->relocaddr set to the wrong address which has no ram backed. More details can be found here:
https://patchew.org/QEMU/[email protected]/

Can we add support for this multi-range ram setting? Any comments?

The test command is:
qemu-system-riscv64 -machine virt -nographic -m 4G -smp 4 \
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \
-kernel /home/wufei/nvme/u-boot/u-boot.bin \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
-drive
file=ubuntu-22.04.1-preinstalled-server-riscv64+unmatched.img,if=virtio,format=raw

Here is a quick fix, is it good?

arch/riscv/cpu/generic/dram.c +1 -1
index 44e11bd56c..fb53a57b4e 100644
@@ -13,7 +13,7 @@ DECLARE_GLOBAL_DATA_PTR;
 
 int dram_init(void)
 {
-       return fdtdec_setup_mem_size_base();
+       return fdtdec_setup_mem_size_base_lowest();
 }
 
 int dram_init_banksize(void)
(quoted diff) +0 -0

b/configs/qemu-riscv64_smode_defconfig
index 1d0f021ade..de08a49dab 100644

configs/qemu-riscv64_smode_defconfig +1 -1
@@ -1,6 +1,6 @@
 CONFIG_RISCV=y
 CONFIG_SYS_MALLOC_LEN=0x800000
-CONFIG_NR_DRAM_BANKS=1
+CONFIG_NR_DRAM_BANKS=2
 CONFIG_HAS_CUSTOM_SYS_INIT_SP_ADDR=y
 CONFIG_CUSTOM_SYS_INIT_SP_ADDR=0x80200000
 CONFIG_ENV_SIZE=0x20000

Thanks,
Fei.