hw/arm/raspi4b: fix guest RAM capped at ~1 GiB regardless of machine size
v2 cover hw/arm/raspi4b: fix guest RAM capped at ~1 GiB regardless of machine size
While working on unrelated raspi4b issues, I noticed the guest never seemed to have as much RAM as I'd given it and tracked it down to hw/arm/raspi4b.c's raspi4_modify_dtb(): it decides whether to add a second memory node above the 1 GiB peripheral hole by checking info->ram_size, but that field is the boot loader's RAM budget for where it's allowed to place the kernel/initrd/dtb image, itself always capped to at most UPPER_RAM_BASE - vcram_size by raspi_base_machine_init(). Since that capped value can never exceed UPPER_RAM_BASE by construction, the condition was never true for any raspi4b configuration -- the second node was never added, and the guest never saw more than ~1 GiB of RAM regardless of the machine's nominal size. board_ram_size(info->board_id), computed one line above in the same function, is the value that was actually needed.
Patch 1 fixes it (one line); patch 2 adds a regression test.
Confirmed via direct measurement inside the guest ("free -h" / /proc/meminfo) on raspi4b's default 2 GiB configuration:
before: MemTotal: 943524 kB (~921 MiB)
after: MemTotal: 1905824 kB (~1861 MiB)Also verified against two real, unmodified Raspberry Pi OS releases (Debian 11/Bullseye and Debian 13/Trixie): both now report ~1.8 GiB of usable RAM instead of ~900 MiB, with clean boots, working SSH, and no kernel errors on either.
I also looked into extending this to raspi4b's real 4 GiB and 8 GiB hardware variants (real boards exist in 1/2/4/8 GiB configurations), since board_ram_size()'s encoding already supports arbitrary sizes and this same bug meant the necessary split-memory-node path had never actually been exercised at any size, ever. 2 GiB now works cleanly with this fix (tested above); 4 GiB hits a real kernel panic (kernel BUG at arch/arm64/mm/mmu.c:192! in early page-table setup, addresses landing in the BCM2711 peripheral region around 0xfc000000) that looks like it needs the "high peripheral alias" real BCM2711 firmware sets up for boards with enough RAM to otherwise collide with that region -- out of scope for this series, so I'm leaving 4/8 GiB support for later and keeping this fix minimal.
checkpatch.pl is clean (0 errors, 0 warnings) on both patches, and the series builds standalone from a clean worktree (--target-list=aarch64-softmmu). All 6 tests in tests/functional/aarch64/test_raspi4.py pass, including the 5 pre-existing ones (no regression) and the new one added here.
v2:
- Dropped the explanatory comment added in raspi4_modify_dtb() in patch 1 -- the commit message already covers it, per Philippe's review.
- Picked up Philippe Mathieu-Daudé's Reviewed-by on patch 1.
Marcelo Manzo (2):
hw/arm/raspi4b: fix guest never seeing more than ~1 GiB of RAM tests/functional/aarch64: add raspi4b full-RAM regression test
hw/arm/raspi4b.c | 2 +- tests/functional/aarch64/test_raspi4.py | 51 +++++++++++++++++++++++++ 2 files changed, 52 insertions(+), 1 deletion(-)
Signature · 1 line
2.47.1
PATCH v2 2/2 tests/functional/aarch64: add raspi4b full-RAM regression test
Guards against the bug fixed in the previous commit: boots raspi4b's default 2 GiB configuration and checks that the guest actually sees close to that (>1.9M kB), not the ~921 MiB the bug left it capped at. Deliberately checks a threshold rather than the exact byte count of either figure, since the precise number depends on how this specific pinned kernel accounts for its own early reservations; the threshold is comfortably between the two (943524 kB broken, 1905824 kB fixed, confirmed by hand against this exact kernel/initrd).
--- tests/functional/aarch64/test_raspi4.py | 51 +++++++++++++++++++++++++ 1 file changed, 51 insertions(+)
tests/functional/aarch64/test_raspi4.py +51 -0
@@ -154,6 +154,57 @@ def test_arm_raspi4_rng_thermal(self): exec_command_and_wait_for_pattern(self, 'halt', 'reboot: System halted') + def test_arm_raspi4_full_ram(self): + kernel_path = self.archive_extract(self.ASSET_KERNEL_20190215, + member='boot/kernel8.img') + dtb_path = self.archive_extract(self.ASSET_KERNEL_20190215, + member='boot/bcm2711-rpi-4-b.dtb') + initrd_path = self.uncompress(self.ASSET_INITRD) + + self.set_machine('raspi4b') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'earlycon=pl011,mmio32,0xfe201000 ' + + 'console=ttyAMA0,115200 ' + + 'panic=-1 noreboot ' + + 'dwc_otg.fiq_fsm_enable=0') + self.vm.add_args('-kernel', kernel_path, + '-dtb', dtb_path, + '-initrd', initrd_path, + '-append', kernel_command_line, + '-no-reboot') + self.vm.launch() + self.wait_for_console_pattern('Boot successful.') + + # raspi4b's default machine RAM is 2 GiB, but raspi4_modify_dtb() + # used to compare the wrong field when deciding whether to add a + # second memory node above the 1 GiB peripheral hole (it checked + # the boot-loader's kernel/initrd-loading RAM budget, which is + # itself always capped to at most 1 GiB, rather than the board's + # actual total RAM) -- so that second node was never added for any + # raspi4b configuration, and the guest never saw more than ~1 GiB + # regardless of the machine's nominal RAM size. With the bug, + # MemTotal here is 943524 kB; fixed, it is 1905824 kB. Guard + # against a regression back to the ~1 GiB figure without hardcoding + # the exact byte count of either, which depends on how this + # specific pinned kernel accounts for its own reservations. + mem_total_kb = None + cmd_output = exec_command_and_wait_for_pattern( + self, 'cat /proc/meminfo', 'MemAvailable') + for line in cmd_output.decode('ascii', errors='replace').splitlines(): + if line.startswith('MemTotal:'): + mem_total_kb = int(line.split()[1]) + break + self.assertIsNotNone(mem_total_kb, 'MemTotal line not found') + self.assertGreater(mem_total_kb, 1900000, + 'guest RAM (%d kB) is far below the ~1.9 GiB ' + 'expected for a 2 GiB raspi4b -- the second ' + 'memory node above the 1 GiB peripheral hole ' + 'is probably not being added' % mem_total_kb) + + exec_command_and_wait_for_pattern(self, 'halt', 'reboot: System halted') + + def test_arm_raspi4_pcie(self): kernel_path = self.archive_extract(self.ASSET_KERNEL_20190215, member='boot/kernel8.img')
Signature · 1 line
2.47.1
PATCH v2 1/2 hw/arm/raspi4b: fix guest never seeing more than ~1 GiB of RAM
In reply to Marcelo Manzoraspi4_modify_dtb() decides whether to add a second memory node above the 1 GiB peripheral hole by checking info->ram_size -- but that field is the boot loader's RAM budget for loading the kernel/initrd/dtb image, itself always capped to at most UPPER_RAM_BASE - vcram_size by raspi_base_machine_init(). Since that capped value can never exceed UPPER_RAM_BASE by construction, the condition was never true for any raspi4b configuration, and the second node was never added: the guest never saw more than ~1 GiB of its nominal RAM, regardless of the machine's actual size.
board_ram_size(info->board_id), computed one line above in the same function, is the value that was actually needed -- the board's real total RAM, not the boot loader's own budget for where it's allowed to place the kernel image.
Confirmed via direct measurement inside the guest ("free -h" / /proc/meminfo) on raspi4b's default 2 GiB configuration, before and after:
before: MemTotal: 943524 kB (~921 MiB)
after: MemTotal: 1905824 kB (~1861 MiB)Also verified against two real, unmodified Raspberry Pi OS releases (Debian 11/Bullseye and Debian 13/Trixie): both now report ~1.8 GiB of usable RAM instead of ~900 MiB, with clean boots, working SSH, and no kernel errors on either.
--- hw/arm/raspi4b.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
hw/arm/raspi4b.c +1 -1
@@ -64,7 +64,7 @@ static void raspi4_modify_dtb(const struct arm_boot_info *info, void *fdt) ram_size =board_ram_size (info->board_id); -if (info->ram_size >UPPER_RAM_BASE ) { +if (ram_size >UPPER_RAM_BASE ) {raspi_add_memory_node (fdt,UPPER_RAM_BASE , ram_size -UPPER_RAM_BASE ); } }
Signature · 1 line
2.47.1
Re: [PATCH v2 2/2] tests/functional/aarch64: add raspi4b full-RAM regression test
In reply to Marcelo ManzoQuoted 34 lines · Marcelo Manzo
Guards against the bug fixed in the previous commit: boots raspi4b's default 2 GiB configuration and checks that the guest actually sees close to that (>1.9M kB), not the ~921 MiB the bug left it capped at. Deliberately checks a threshold rather than the exact byte count of either figure, since the precise number depends on how this specific pinned kernel accounts for its own early reservations; the threshold is comfortably between the two (943524 kB broken, 1905824 kB fixed, confirmed by hand against this exact kernel/initrd).
Signed-off-by Marcelo Manzo <[email protected]>--- tests/functional/aarch64/test_raspi4.py | 51 +++++++++++++++++++++++++ 1 file changed, 51 insertions(+)tests/functional/aarch64/test_raspi4.py +11
-0@@ -154,6 +154,57 @@ def test_arm_raspi4_rng_thermal(self): exec_command_and_wait_for_pattern(self, 'halt', 'reboot: System halted') + def test_arm_raspi4_full_ram(self): + kernel_path = self.archive_extract(self.ASSET_KERNEL_20190215, + member='boot/kernel8.img') + dtb_path = self.archive_extract(self.ASSET_KERNEL_20190215, + member='boot/bcm2711-rpi-4-b.dtb') + initrd_path = self.uncompress(self.ASSET_INITRD) + + self.set_machine('raspi4b') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'earlycon=pl011,mmio32,0xfe201000 ' +
tests/functional/aarch64/test_raspi4.py def test_arm_raspi4_rng_thermal(self): · line 177
Quoted 10 lines
tests/functional/aarch64/test_raspi4.py +10
-0+ 'console=ttyAMA0,115200 ' + + 'panic=-1 noreboot ' + + 'dwc_otg.fiq_fsm_enable=0') + self.vm.add_args('-kernel', kernel_path, + '-dtb', dtb_path, + '-initrd', initrd_path, + '-append', kernel_command_line, + '-no-reboot') + self.vm.launch() + self.wait_for_console_pattern('Boot successful.')
This is an identical kernel/initrd boot to the existing test_arm_raspi4_initrd() test, right? I think it would be preferable to add the "check the RAM figure" to the existing test, after e.g. the checks on cpuinfo and iomem. That way we save the time it takes to do the full kernel boot. Our functional test suite is already quite slow because of how many "boot a guest" tests it needs to run, so I'm keen on keeping the runtime down where we can do that.
tests/functional/aarch64/test_raspi4.py def test_arm_raspi4_rng_thermal(self): · line 190
Quoted 12 lines
tests/functional/aarch64/test_raspi4.py +12
-0+ # raspi4b's default machine RAM is 2 GiB, but raspi4_modify_dtb() + # used to compare the wrong field when deciding whether to add a + # second memory node above the 1 GiB peripheral hole (it checked + # the boot-loader's kernel/initrd-loading RAM budget, which is + # itself always capped to at most 1 GiB, rather than the board's + # actual total RAM) -- so that second node was never added for any + # raspi4b configuration, and the guest never saw more than ~1 GiB + # regardless of the machine's nominal RAM size. With the bug, + # MemTotal here is 943524 kB; fixed, it is 1905824 kB. Guard + # against a regression back to the ~1 GiB figure without hardcoding + # the exact byte count of either, which depends on how this + # specific pinned kernel accounts for its own reservations.
I think this comment is a bit more verbose than we need -- we can just say that we used to get the RAM size wrong and we're guarding against a regression; anybody interested in the exact details of the old bug can look in the git history for it.
Quoted 10 lines
tests/functional/aarch64/test_raspi4.py +10
-0+ mem_total_kb = None + cmd_output = exec_command_and_wait_for_pattern( + self, 'cat /proc/meminfo', 'MemAvailable') + for line in cmd_output.decode('ascii', errors='replace').splitlines(): + if line.startswith('MemTotal:'): + mem_total_kb = int(line.split()[1]) + break + self.assertIsNotNone(mem_total_kb, 'MemTotal line not found') + self.assertGreater(mem_total_kb, 1900000, + 'guest RAM (%d kB) is far below the ~1.9 GiB '
tests/functional/aarch64/test_raspi4.py def test_arm_raspi4_rng_thermal(self): · line 210
Quoted 10 lines
tests/functional/aarch64/test_raspi4.py +7
-0+ 'expected for a 2 GiB raspi4b -- the second ' + 'memory node above the 1 GiB peripheral hole ' + 'is probably not being added' % mem_total_kb) + + exec_command_and_wait_for_pattern(self, 'halt', 'reboot: System halted') + + def test_arm_raspi4_pcie(self): kernel_path = self.archive_extract(self.ASSET_KERNEL_20190215, member='boot/kernel8.img')
thanks
-- PMM