hw/core/loader: allow loading larger ROMs
PATCH 1/1 hw/core/loader: allow loading larger ROMs
The read() syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into account, instead failing if all bytes are not read at once. This change wraps the read() syscall in a do/while loop to ensure all bytes of the ROM are read.
--- hw/core/loader.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
hw/core/loader.c +10 -7
@@ -1075,7 +1075,7 @@ ssize_t rom_add_file(const char *file, const char *fw_dir, {MachineClass *mc =MACHINE_GET_CLASS (qdev_get_machine ());Rom *rom; -ssize_t rc; +ssize_t rc, sz =0 ;int fd =-1 ;char devpath[100 ];
@@ -1116,12 +1116,15 @@ ssize_t rom_add_file(const char *file, const char *fw_dir, rom->datasize = rom->romsize; rom->data =g_malloc0 (rom->datasize);lseek (fd,0 ,SEEK_SET ); - rc =read (fd, rom->data, rom->datasize); -if (rc != rom->datasize) { -fprintf (stderr,"rom: file %-20s: read error: rc=%zd (expected %zd)\n" , - rom->name, rc, rom->datasize); - goto err; - } +do { + rc =read (fd, &rom->data[sz], rom->datasize); +if (rc ==-1 ) { +fprintf (stderr,"rom: file %-20s: read error: %s\n" , + rom->name,strerror (errno)); + goto err; + } + sz += rc; + }while (sz != rom->datasize);close (fd);rom_insert (rom);if (rom->fw_file && fw_cfg) {
Signature · 1 line
2.45.2
Hi, Gregor
Quoted 5 lines
The
read()syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into account, instead failing if all bytes are not read at once. This change wraps theread()syscall in a do/while loop to ensure all bytes of the ROM are read.
Can you reproduce this issue?
Thanks
Xingtao
Hi Xingtao,
Quoted 1 line
Can you reproduce this issue?
Absolutely! I encountered this when trying to load an OpenSBI payload firmware using the bios option for the QEMU RISC-V virt board. These payload firmwares bundle the entire next boot stage, which in my case is a build of the Linux kernel (which is a standard configuration, supported by tools such as Buildroot [1]). My kernel (configured with the default 64-bit RISC-V configuration) comes in at 9.8M, which is copied into the OpenSBI firmware of final size 10M. Then, I run the following QEMU command:
qemu-system-riscv64 -machine virt -m 4G -nographic -bios fw_payload.bin
and get the following output:
rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'
This is from my development machine, running Arch Linux with kernel 6.9.6 and root filesystem ZFS 2.2.4. Please let me know if you'd like me to make a minimal reproducer for this, or if you need any more information.
Thanks,
Gregor
[1]
https://github.com/buildroot/buildroot/blob/master/boot/opensbi/Config.in#L95
Quoted 13 lines · Xingtao Yao (Fujitsu)
Hi, Gregor
Quoted 3 lines
The
read()syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this intoaccount,
Quoted 2 lines
instead failing if all bytes are not read at once. This change wraps the
read()syscall in a do/while loop to ensure all bytes of the ROM areread.
Can you reproduce this issue?Thanks
Xingtao
Hi, Gregor
Quoted 2 lines
rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'
Thanks, I was able to reproduce the problem when the images size is larger than 2147479552.
I found that in my test environment, the maximum value returned by a read operation is 2147479552, which was affected by the operating system.
We can find this limitation in the man page:
NOTES
The types size_t and ssize_t are, respectively, unsigned and signed integer data types specified by POSIX.1.
On Linux, read() (and similar system calls) will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both
32-bit and 64-bit systems.)hw/core/loader.c ssize_t rom_add_file(const char *file, const char *fw_dir, · line 1127
Quoted 9 lines
hw/core/loader.c +9
-0+do { + rc =read (fd, &rom->data[sz], rom->datasize); +if (rc ==-1 ) { +fprintf (stderr,"rom: file %-20s: read error: %s\n" , + rom->name,strerror (errno)); + goto err; + } + sz += rc; + }while (sz != rom->datasize);
I think we can use load_image_size() instead.
From: Gregor Haas <[email protected]>
Sent: Friday, June 28, 2024 1:35 AM
To: Yao, Xingtao/姚 幸涛 <[email protected]>
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] hw/core/loader: allow loading larger ROMs
Hi Xingtao,
Quoted 1 line
Can you reproduce this issue?
Absolutely! I encountered this when trying to load an OpenSBI payload firmware using the bios option for the QEMU RISC-V virt board. These payload firmwares bundle the entire next boot stage, which in my case is a build of the Linux kernel (which is a standard configuration, supported by tools such as Buildroot [1]). My kernel (configured with the default 64-bit RISC-V configuration) comes in at 9.8M, which is copied into the OpenSBI firmware of final size 10M. Then, I run the following QEMU command:
qemu-system-riscv64 -machine virt -m 4G -nographic -bios fw_payload.bin
and get the following output:
rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'
This is from my development machine, running Arch Linux with kernel 6.9.6 and root filesystem ZFS 2.2.4. Please let me know if you'd like me to make a minimal reproducer for this, or if you need any more information.
Thanks,
Gregor
[1] https://github.com/buildroot/buildroot/blob/master/boot/opensbi/Config.in#L95
On Wed, Jun 26, 2024 at 11:11 PM Xingtao Yao (Fujitsu) <[email protected]<mailto:[email protected]>> wrote:
Hi, Gregor
Quoted 5 lines
The
read()syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into account, instead failing if all bytes are not read at once. This change wraps theread()syscall in a do/while loop to ensure all bytes of the ROM are read.
Can you reproduce this issue?
Thanks
Xingtao
Hi Xingtao,
Thank you for reproducing this -- I agree with your conclusion and will send a v2 patchset momentarily.
Thank you,
Gregor
Quoted 122 lines · Xingtao Yao (Fujitsu)
Hi, Gregor
Quoted 2 lines
rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'Thanks, I was able to reproduce the problem when the images size is larger than 2147479552.
I found that in my test environment, the maximum value returned by a read operation is 2147479552,
which was affected by the operating system.
We can find this limitation in the man page:
NOTES
The types
size_tandssize_tare, respectively, unsigned and
signed integer data types specified by POSIX.1.On Linux,
read()(and similar system calls) will transfer at most0x7ffff000(2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both32-bit and 64-bit systems.)Quoted 7 lines
hw/core/loader.c +4
-0+ do { + rc = read(fd, &rom->data[sz], rom->datasize); + if (rc == -1) { + fprintf(stderr, "rom: file %-20s: read error: %s\n",hw/core/loader.c ssize_t rom_add_file(const char *file, const char *fw_dir, · line 1127
Quoted 10 lines
hw/core/loader.c +5
-0+ rom->name, strerror(errno)); + goto err; + } + sz += rc; + }while (sz != rom->datasize);I think we can use
load_image_size()instead.*From:* Gregor Haas <[email protected]>
*Sent:* Friday, June 28, 2024 1:35 AM
*To:* Yao, Xingtao/姚 幸涛 <[email protected]>
*Cc:* [email protected]; [email protected]; [email protected]
*Subject:* Re: [PATCH] hw/core/loader: allow loading larger ROMsHi Xingtao,
Quoted 1 line
Can you reproduce this issue?
Absolutely! I encountered this when trying to load an OpenSBI payload firmware using the bios option for the QEMU RISC-V virt board. These payload firmwares bundle the entire next boot stage, which in my case is a build of the Linux kernel (which is a standard configuration, supported by tools such as Buildroot [1]). My kernel (configured with the default 64-bit RISC-V configuration) comes in at 9.8M, which is copied into the OpenSBI firmware of final size 10M. Then, I run the following QEMU command:
qemu-system-riscv64 -machine virt -m 4G -nographic -bios fw_payload.bin
and get the following output:
rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'This is from my development machine, running Arch Linux with kernel 6.9.6 and root filesystem ZFS 2.2.4. Please let me know if you'd like me to make a minimal reproducer for this, or if you need any more information.
Thanks,
Gregor[1]
https://github.com/buildroot/buildroot/blob/master/boot/opensbi/Config.in#L95On Wed, Jun 26, 2024 at 11:11 PM Xingtao Yao (Fujitsu) < [email protected]> wrote:
Hi, Gregor
Quoted 3 lines
The
read()syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this intoaccount,
Quoted 2 lines
instead failing if all bytes are not read at once. This change wraps the
read()syscall in a do/while loop to ensure all bytes of the ROM areread.
Can you reproduce this issue?Thanks
Xingtao