Skip to contentSkip to conversation

hw/core/loader: allow loading larger ROMs

PATCH Superseded Gregor Haas 2 participants 5 messages Jun 27, 2024 – Jun 28, 2024

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 hw/core/loader: allow loading larger ROMs

The read() syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into account, instead failing if all bytes are not read at once. This change wraps the read() syscall in a do/while loop to ensure all bytes of the ROM are read.

---
 hw/core/loader.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
hw/core/loader.c +10 -7
index 2f8105d7de..afa26dccb1 100644
@@ -1075,7 +1075,7 @@ ssize_t rom_add_file(const char *file, const char *fw_dir,
 {
     MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
     Rom *rom;
-    ssize_t rc;
+    ssize_t rc, sz = 0;
     int fd = -1;
     char devpath[100];
 
@@ -1116,12 +1116,15 @@ ssize_t rom_add_file(const char *file, const char *fw_dir,
     rom->datasize = rom->romsize;
     rom->data     = g_malloc0(rom->datasize);
     lseek(fd, 0, SEEK_SET);
-    rc = read(fd, rom->data, rom->datasize);
-    if (rc != rom->datasize) {
-        fprintf(stderr, "rom: file %-20s: read error: rc=%zd (expected %zd)\n",
-                rom->name, rc, rom->datasize);
-        goto err;
-    }
+    do {
+        rc = read(fd, &rom->data[sz], rom->datasize);
+        if (rc == -1) {
+                fprintf(stderr, "rom: file %-20s: read error: %s\n",
+                        rom->name, strerror(errno));
+                goto err;
+        }
+        sz += rc;
+    } while (sz != rom->datasize);
     close(fd);
     rom_insert(rom);
     if (rom->fw_file && fw_cfg) {
Signature · 1 line
2.45.2
#1 lore raw

Hi, Gregor

Quoted 5 lines

The read() syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into account, instead failing if all bytes are not read at once. This change wraps the read() syscall in a do/while loop to ensure all bytes of the ROM are read.

Can you reproduce this issue?

Thanks
Xingtao

#2 lore raw

Hi Xingtao,

Quoted 1 line

Can you reproduce this issue?

Absolutely! I encountered this when trying to load an OpenSBI payload firmware using the bios option for the QEMU RISC-V virt board. These payload firmwares bundle the entire next boot stage, which in my case is a build of the Linux kernel (which is a standard configuration, supported by tools such as Buildroot [1]). My kernel (configured with the default 64-bit RISC-V configuration) comes in at 9.8M, which is copied into the OpenSBI firmware of final size 10M. Then, I run the following QEMU command:

qemu-system-riscv64 -machine virt -m 4G -nographic -bios fw_payload.bin

and get the following output:

rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'

This is from my development machine, running Arch Linux with kernel 6.9.6 and root filesystem ZFS 2.2.4. Please let me know if you'd like me to make a minimal reproducer for this, or if you need any more information.

Thanks,
Gregor

[1]
https://github.com/buildroot/buildroot/blob/master/boot/opensbi/Config.in#L95

Quoted 13 lines · Xingtao Yao (Fujitsu)

Hi, Gregor

Quoted 3 lines

The read() syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into

account,

Quoted 2 lines

instead failing if all bytes are not read at once. This change wraps the read() syscall in a do/while loop to ensure all bytes of the ROM are

read.
Can you reproduce this issue?

Thanks
Xingtao

#3 lore raw

Hi, Gregor

Quoted 2 lines

rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'

Thanks, I was able to reproduce the problem when the images size is larger than 2147479552.

I found that in my test environment, the maximum value returned by a read operation is 2147479552, which was affected by the operating system.

We can find this limitation in the man page:
NOTES

       The types size_t and ssize_t are, respectively, unsigned and signed integer data types specified by POSIX.1.

       On Linux, read() (and similar system calls) will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred.  (This is true on both
       32-bit and 64-bit systems.)

hw/core/loader.c ssize_t rom_add_file(const char *file, const char *fw_dir, · line 1127

Quoted 9 lines
hw/core/loader.c +9 -0
+    do {
+        rc = read(fd, &rom->data[sz], rom->datasize);
+        if (rc == -1) {
+                fprintf(stderr, "rom: file %-20s: read error: %s\n",
+                        rom->name, strerror(errno));
+                goto err;
+        }
+        sz += rc;
+    } while (sz != rom->datasize);

I think we can use load_image_size() instead.

From: Gregor Haas <[email protected]>
Sent: Friday, June 28, 2024 1:35 AM
To: Yao, Xingtao/姚 幸涛 <[email protected]>
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] hw/core/loader: allow loading larger ROMs

Hi Xingtao,

Quoted 1 line

Can you reproduce this issue?

Absolutely! I encountered this when trying to load an OpenSBI payload firmware using the bios option for the QEMU RISC-V virt board. These payload firmwares bundle the entire next boot stage, which in my case is a build of the Linux kernel (which is a standard configuration, supported by tools such as Buildroot [1]). My kernel (configured with the default 64-bit RISC-V configuration) comes in at 9.8M, which is copied into the OpenSBI firmware of final size 10M. Then, I run the following QEMU command:

qemu-system-riscv64 -machine virt -m 4G -nographic -bios fw_payload.bin

and get the following output:

rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'

This is from my development machine, running Arch Linux with kernel 6.9.6 and root filesystem ZFS 2.2.4. Please let me know if you'd like me to make a minimal reproducer for this, or if you need any more information.

Thanks,
Gregor

[1] https://github.com/buildroot/buildroot/blob/master/boot/opensbi/Config.in#L95

On Wed, Jun 26, 2024 at 11:11 PM Xingtao Yao (Fujitsu) <[email protected]<mailto:[email protected]>> wrote:
Hi, Gregor

Quoted 5 lines

The read() syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into account, instead failing if all bytes are not read at once. This change wraps the read() syscall in a do/while loop to ensure all bytes of the ROM are read.

Can you reproduce this issue?

Thanks
Xingtao

#4 lore raw

Hi Xingtao,

Thank you for reproducing this -- I agree with your conclusion and will send a v2 patchset momentarily.

Thank you,
Gregor

Quoted 122 lines · Xingtao Yao (Fujitsu)

Hi, Gregor

Quoted 2 lines

rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'

Thanks, I was able to reproduce the problem when the images size is larger than 2147479552.

I found that in my test environment, the maximum value returned by a read operation is 2147479552,

which was affected by the operating system.

We can find this limitation in the man page:

NOTES

The types size_t and ssize_t are, respectively, unsigned and
signed integer data types specified by POSIX.1.

On Linux, read() (and similar system calls) will transfer at most
0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both

       32-bit and 64-bit systems.)
Quoted 7 lines
hw/core/loader.c +4 -0
+    do {
 
+        rc = read(fd, &rom->data[sz], rom->datasize);
 
+        if (rc == -1) {
 
+                fprintf(stderr, "rom: file %-20s: read error: %s\n",

hw/core/loader.c ssize_t rom_add_file(const char *file, const char *fw_dir, · line 1127

Quoted 10 lines
hw/core/loader.c +5 -0
 
+                        rom->name, strerror(errno));
 
+                goto err;
 
+        }
 
+        sz += rc;
 
+    } while (sz != rom->datasize);

I think we can use load_image_size() instead.

*From:* Gregor Haas <[email protected]>
*Sent:* Friday, June 28, 2024 1:35 AM
*To:* Yao, Xingtao/姚 幸涛 <[email protected]>
*Cc:* [email protected]; [email protected]; [email protected]
*Subject:* Re: [PATCH] hw/core/loader: allow loading larger ROMs

Hi Xingtao,

Quoted 1 line

Can you reproduce this issue?

Absolutely! I encountered this when trying to load an OpenSBI payload firmware using the bios option for the QEMU RISC-V virt board. These payload firmwares bundle the entire next boot stage, which in my case is a build of the Linux kernel (which is a standard configuration, supported by tools such as Buildroot [1]). My kernel (configured with the default 64-bit RISC-V configuration) comes in at 9.8M, which is copied into the OpenSBI firmware of final size 10M. Then, I run the following QEMU command:

qemu-system-riscv64 -machine virt -m 4G -nographic -bios fw_payload.bin

and get the following output:

rom: file fw_payload.bin: read error: rc=2147479552 (expected 2303760392)
qemu-system-riscv64: could not load firmware 'fw_payload.bin'

This is from my development machine, running Arch Linux with kernel 6.9.6 and root filesystem ZFS 2.2.4. Please let me know if you'd like me to make a minimal reproducer for this, or if you need any more information.

Thanks,
Gregor

[1]
https://github.com/buildroot/buildroot/blob/master/boot/opensbi/Config.in#L95

On Wed, Jun 26, 2024 at 11:11 PM Xingtao Yao (Fujitsu) < [email protected]> wrote:

Hi, Gregor

Quoted 3 lines

The read() syscall is not guaranteed to return all data from a file. The default ROM loader implementation currently does not take this into

account,

Quoted 2 lines

instead failing if all bytes are not read at once. This change wraps the read() syscall in a do/while loop to ensure all bytes of the ROM are

read.
Can you reproduce this issue?

Thanks
Xingtao