Skip to contentSkip to conversation

set qemuppc64 default nfs rootfs mount r/w size to 524288

PATCH In review Xiangyu Chen 3 participants 6 messages Jan 17, 2023 – Jan 25, 2023

lore
Share & tools
lore
#0 lore raw

cover set qemuppc64 default nfs rootfs mount r/w size to 524288

From: Xiangyu Chen <[email protected]>

On master oe, build a qemuppc64 with systemd as default init, when we use nfs bootup, the kernel might panic due to missing symbol in dynamic libraries as below:

hid-generic 0003:0627:0001.0003: input: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.0-3/input0 /sbin/init: /lib64/libm.so.6: version `XZ_5.0' not found (required by /usr/lib64/libkmod.so.2)

  Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
  CPU: 0 PID: 1 Comm: init Not tainted 5.15.78-yocto-standard #1
  Call Trace:
  [c000000007443ba0] [c0000000009538d0] dump_stack_lvl+0x74/0xa8 (unreliable)
  [c000000007443be0] [c000000000103524] panic+0x170/0x3cc
  [c000000007443c80] [c00000000010cf64] do_exit+0xb44/0xb50
  [c000000007443d50] [c00000000010d040] do_group_exit+0x60/0xd0
  [c000000007443d90] [c00000000010d0d4] sys_exit_group+0x24/0x30
  [c000000007443db0] [c00000000002cfd4] system_call_exception+0x194/0x2f0
  [c000000007443e10] [c00000000000c2cc] system_call_common+0xec/0x250
  --- interrupt: c00 at 0x7fff9ed9e840
  NIP:  00007fff9ed9e840 LR: 00007fff9ed7da20 CTR: 0000000000000000
  REGS: c000000007443e80 TRAP: 0c00   Not tainted (5.15.78-yocto-standard)
  MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 24022442  XER: 00000000

One or more of the libraries systemd depends on failed to load due to unresolved symbols/functions. This was intermittent - with a failure rate estimated between 5% and 30%.

After checking the code, this issue happens on gcc 12, kirkstone is using gcc 11 works well, with both using the exact same v5.15.84 kernel commit.

There is a kernel fix from upstream [1], they changed the rsize / wsize to a multiple of PAGE_SIZE, when we applied this patch, the qemuppc64's default r/wsize went from 4096 to 524288.But the qemuppc64 doesn't have its own linux-yocto kernel branch, so apply this change might cause regression with other platforms which share branch with qemuppc64.

So, we added an extra option for nfs rootfs, and set the qemuppc64 default r/w size to 524288 to line up with the kernel fix[1].

Yocto did a similar thing in the distant past[2] - prior to boot-arg adjustments existing - by allowing a Kconfig to set the defaults on nfsboot, in order to work around hardware limitations.

There are 3 patches for this fix:
the patch 1 added QB_NFSROOTFS_EXTRA_OPT in qemuboot.bbclass,it contains a default QB_NFSROOTFS_EXTRA_OPT value and some notes in comment. the patch 2 contains processing this value in runqemu script the patch 3 set the r/wsize value in qemuppc64 machine configuration file.

Reference:
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=940261a195080cf
[2] https://git.yoctoproject.org/linux-yocto-4.1/commit/?h=standard/base&id=a96cfd98add95

Xiangyu Chen (3):
qemuboot.bbclass: add QB_NFSROOTFS_EXTRA_OPT for nfs rootfs extra
option
runqemu: add process of option QB_NFSROOTFS_EXTRA_OPT
qemuppc64: set the qemuppc64 nfs r/wsize mount options to 524288

 meta/classes-recipe/qemuboot.bbclass | 3 +++
 meta/conf/machine/qemuppc64.conf     | 1 +
 scripts/runqemu                      | 6 +++++-
 3 files changed, 9 insertions(+), 1 deletion(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/3 qemuboot.bbclass: add QB_NFSROOTFS_EXTRA_OPT for nfs rootfs extra option

From: Xiangyu Chen <[email protected]>

This extra options to be appended to the nfs rootfs options in kernel boot arg.

---
 meta/classes-recipe/qemuboot.bbclass | 3 +++
 1 file changed, 3 insertions(+)
meta/classes-recipe/qemuboot.bbclass +3 -0
index 5a0e50ccfc..444871438d 100644
@@ -86,6 +86,8 @@
 #                      without the need to specify a dedicated qemu configuration
 #
 # QB_GRAPHICS: QEMU video card type (e.g. "-vga std")
+# QB_NFSROOTFS_EXTRA_OPT: extra options to be appended to the nfs rootfs options in kernel boot arg, e.g.,
+#                         "wsize=4096,rsize=4096"
 #
 # Usage:
 # IMAGE_CLASSES += "qemuboot"
@@ -103,6 +105,7 @@ QB_CMDLINE_IP_SLIRP ?= "ip=dhcp"
 QB_CMDLINE_IP_TAP ?= "ip=192.168.7.@CLIENT@::192.168.7.@GATEWAY@:255.255.255.0::eth0:off:8.8.8.8"
 QB_ROOTFS_EXTRA_OPT ?= ""
 QB_GRAPHICS ?= ""
+QB_NFSROOTFS_EXTRA_OPT ?= ""
 
 # This should be kept align with ROOT_VM
 QB_DRIVE_TYPE ?= "/dev/sd"
Signature · 1 line
2.34.1
#2 lore raw

PATCH 2/3 runqemu: add process of option QB_NFSROOTFS_EXTRA_OPT

In reply to Xiangyu Chen

From: Xiangyu Chen <[email protected]>

This extra options to be appended to the nfs rootfs options in kernel boot arg.

Example config with qemuppc64 in machine config:

add r/w size in the nfs rootfs extra option:
QB_NFSROOTFS_EXTRA_OPT = "wsize=524288,rsize=524288"

re-build and runqemu with nfs again, we can observe the kernel command line added our defined value in QB_NFSROOTFS_EXTRA_OPT:

Kernel command line: root=/dev/nfs nfsroot=10.0.2.2:/home/xchen5/testing/build/tmp-glibc/deploy/images/qemuppc64/testnfs,nfsvers=3,port=3049,tcp,mountport=3048,wsize=524288,rsize=524288 rw mem=256M ip=dhcp console=hvc0 console=hvc0 nohugevmalloc

---
 scripts/runqemu | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
scripts/runqemu +5 -1
index ccc557f308..5907390573 100755
@@ -1022,7 +1022,11 @@ class BaseConfig(object):
             # Use '%s' since they are integers
             os.putenv(k, '%s' % v)
 
-        self.unfs_opts="nfsvers=3,port=%s,tcp,mountport=%s" % (nfsd_port, mountd_port)
+        qb_nfsrootfs_extra_opt = self.get("QB_NFSROOTFS_EXTRA_OPT")
+        if qb_nfsrootfs_extra_opt and not qb_nfsrootfs_extra_opt.startswith(","):
+            qb_nfsrootfs_extra_opt = "," + qb_nfsrootfs_extra_opt
+
+        self.unfs_opts="nfsvers=3,port=%s,tcp,mountport=%s%s" % (nfsd_port, mountd_port, qb_nfsrootfs_extra_opt)
 
         # Extract .tar.bz2 or .tar.bz if no nfs dir
         if not (self.rootfs and os.path.isdir(self.rootfs)):
Signature · 1 line
2.34.1
#3 lore raw

PATCH 3/3 qemuppc64: set the qemuppc64 nfs r/wsize mount options to 524288

In reply to Xiangyu Chen

From: Xiangyu Chen <[email protected]>

On master oe, build a qemuppc64 with systemd as default init, when we use nfs bootup, the kernel might panic due to missing symbol in dynamic libraries as below:

hid-generic 0003:0627:0001.0003: input: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.0-3/input0 /sbin/init: /lib64/libm.so.6: version `XZ_5.0' not found (required by /usr/lib64/libkmod.so.2)

  Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
  CPU: 0 PID: 1 Comm: init Not tainted 5.15.78-yocto-standard #1
  Call Trace:
  [c000000007443ba0] [c0000000009538d0] dump_stack_lvl+0x74/0xa8 (unreliable)
  [c000000007443be0] [c000000000103524] panic+0x170/0x3cc
  [c000000007443c80] [c00000000010cf64] do_exit+0xb44/0xb50
  [c000000007443d50] [c00000000010d040] do_group_exit+0x60/0xd0
  [c000000007443d90] [c00000000010d0d4] sys_exit_group+0x24/0x30
  [c000000007443db0] [c00000000002cfd4] system_call_exception+0x194/0x2f0
  [c000000007443e10] [c00000000000c2cc] system_call_common+0xec/0x250
  --- interrupt: c00 at 0x7fff9ed9e840
  NIP:  00007fff9ed9e840 LR: 00007fff9ed7da20 CTR: 0000000000000000
  REGS: c000000007443e80 TRAP: 0c00   Not tainted (5.15.78-yocto-standard)
  MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 24022442  XER: 00000000

One or more of the libraries systemd depends on failed to load due to unresolved symbols/functions. This was intermittent - with a failure rate estimated between 5% and 30%.

After checking the code, this issue happens on gcc 12, kirkstone is using gcc 11 works well, with both using the exact same v5.15.84 kernel commit.

There is a kernel fix from upstream [1], they changed the rsize / wsize to a multiple of PAGE_SIZE, when we applied this patch, the qemuppc64's default r/wsize went from 4096 to 524288.But the qemuppc64 doesn't have its own linux-yocto kernel branch, so apply this change might cause regression with other platforms which share branch with qemuppc64.

So, we added an extra option for nfs rootfs, and set the qemuppc64 default r/w size to 524288 to line up with the kernel fix[1].

Yocto did a similar thing in the distant past[2] - prior to boot-arg adjustments existing - by allowing a Kconfig to set the defaults on nfsboot, in order to work around hardware limitations.

Reference:
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=940261a195080cf
[2] https://git.yoctoproject.org/linux-yocto-4.1/commit/?h=standard/base&id=a96cfd98add95

---
 meta/conf/machine/qemuppc64.conf | 1 +
 1 file changed, 1 insertion(+)
meta/conf/machine/qemuppc64.conf +1 -0
index 304f06a30d..7709339313 100644
@@ -15,6 +15,7 @@ QB_MACHINE = "-machine pseries"
 QB_CPU = "-cpu POWER9"
 QB_SMP = "-smp 2"
 
+QB_NFSROOTFS_EXTRA_OPT = "wsize=524288,rsize=524288"
 QB_KERNEL_CMDLINE_APPEND = "console=hvc0 nohugevmalloc"
 #QB_OPT_APPEND += "-device qemu-xhci -device usb-tablet -device usb-kbd"
 QB_OPT_APPEND = "-usb -device usb-tablet"
Signature · 1 line
2.34.1
#4 lore raw

Re: [OE-Core][PATCH 3/3] qemuppc64: set the qemuppc64 nfs r/wsize mount options to 524288

Quoted 58 lines · Xiangyu Chen

From: Xiangyu Chen <[email protected]>

On master oe, build a qemuppc64 with systemd as default init, when we use nfs bootup, the kernel might panic due to missing symbol in dynamic libraries as below:

hid-generic 0003:0627:0001.0003: input: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.0-3/input0 /sbin/init: /lib64/libm.so.6: version `XZ_5.0' not found (required by /usr/lib64/libkmod.so.2)

  Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
  CPU: 0 PID: 1 Comm: init Not tainted 5.15.78-yocto-standard #1
  Call Trace:
  [c000000007443ba0] [c0000000009538d0] dump_stack_lvl+0x74/0xa8 (unreliable)
  [c000000007443be0] [c000000000103524] panic+0x170/0x3cc
  [c000000007443c80] [c00000000010cf64] do_exit+0xb44/0xb50
  [c000000007443d50] [c00000000010d040] do_group_exit+0x60/0xd0
  [c000000007443d90] [c00000000010d0d4] sys_exit_group+0x24/0x30
  [c000000007443db0] [c00000000002cfd4] system_call_exception+0x194/0x2f0
  [c000000007443e10] [c00000000000c2cc] system_call_common+0xec/0x250
  --- interrupt: c00 at 0x7fff9ed9e840
  NIP:  00007fff9ed9e840 LR: 00007fff9ed7da20 CTR: 0000000000000000
  REGS: c000000007443e80 TRAP: 0c00   Not tainted (5.15.78-yocto-standard)
  MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 24022442  XER: 00000000

One or more of the libraries systemd depends on failed to load due to unresolved symbols/functions. This was intermittent - with a failure rate estimated between 5% and 30%.

After checking the code, this issue happens on gcc 12, kirkstone is using gcc 11 works well, with both using the exact same v5.15.84 kernel commit.

There is a kernel fix from upstream [1], they changed the rsize / wsize to a multiple of PAGE_SIZE, when we applied this patch, the qemuppc64's default r/wsize went from 4096 to 524288.But the qemuppc64 doesn't have its own linux-yocto kernel branch, so apply this change might cause regression with other platforms which share branch with qemuppc64.

So, we added an extra option for nfs rootfs, and set the qemuppc64 default r/w size to 524288 to line up with the kernel fix[1].

Yocto did a similar thing in the distant past[2] - prior to boot-arg adjustments existing - by allowing a Kconfig to set the defaults on nfsboot, in order to work around hardware limitations.

Reference:
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=940261a195080cf
[2] https://git.yoctoproject.org/linux-yocto-4.1/commit/?h=standard/base&id=a96cfd98add95

---
 meta/conf/machine/qemuppc64.conf | 1 +
 1 file changed, 1 insertion(+)
meta/conf/machine/qemuppc64.conf +0 -0
index 304f06a30d..7709339313 100644

meta/conf/machine/qemuppc64.conf QB_MACHINE = "-machine pseries" · line 18

Quoted 4 lines
meta/conf/machine/qemuppc64.conf +1 -0
@@ -15,6 +15,7 @@ QB_MACHINE = "-machine pseries"
 QB_CPU = "-cpu POWER9"
 QB_SMP = "-smp 2"
 
+QB_NFSROOTFS_EXTRA_OPT = "wsize=524288,rsize=524288"

lgtm

Quoted 15 lines
meta/conf/machine/qemuppc64.conf +0 -1
 QB_KERNEL_CMDLINE_APPEND = "console=hvc0 nohugevmalloc"
 #QB_OPT_APPEND += "-device qemu-xhci -device usb-tablet -device usb-kbd"
 QB_OPT_APPEND = "-usb -device usb-tablet"
--

2.34.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#176023): https://lists.openembedded.org/g/openembedded-core/message/176023
Mute This Topic: https://lists.openembedded.org/mt/96322356/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw
In reply to Xiangyu Chen
Quoted 60 lines · Xiangyu Chen

From: Xiangyu Chen <[email protected]>

On master oe, build a qemuppc64 with systemd as default init, when we use nfs bootup, the kernel might panic due to missing symbol in dynamic libraries as below:

hid-generic 0003:0627:0001.0003: input: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.0-3/input0 /sbin/init: /lib64/libm.so.6: version `XZ_5.0' not found (required by /usr/lib64/libkmod.so.2)

  Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
  CPU: 0 PID: 1 Comm: init Not tainted 5.15.78-yocto-standard #1
  Call Trace:
  [c000000007443ba0] [c0000000009538d0] dump_stack_lvl+0x74/0xa8 (unreliable)
  [c000000007443be0] [c000000000103524] panic+0x170/0x3cc
  [c000000007443c80] [c00000000010cf64] do_exit+0xb44/0xb50
  [c000000007443d50] [c00000000010d040] do_group_exit+0x60/0xd0
  [c000000007443d90] [c00000000010d0d4] sys_exit_group+0x24/0x30
  [c000000007443db0] [c00000000002cfd4] system_call_exception+0x194/0x2f0
  [c000000007443e10] [c00000000000c2cc] system_call_common+0xec/0x250
  --- interrupt: c00 at 0x7fff9ed9e840
  NIP:  00007fff9ed9e840 LR: 00007fff9ed7da20 CTR: 0000000000000000
  REGS: c000000007443e80 TRAP: 0c00   Not tainted (5.15.78-yocto-standard)
  MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 24022442  XER: 00000000

One or more of the libraries systemd depends on failed to load due to unresolved symbols/functions. This was intermittent - with a failure rate estimated between 5% and 30%.

After checking the code, this issue happens on gcc 12, kirkstone is using gcc 11 works well, with both using the exact same v5.15.84 kernel commit.

There is a kernel fix from upstream [1], they changed the rsize / wsize to a multiple of PAGE_SIZE, when we applied this patch, the qemuppc64's default r/wsize went from 4096 to 524288.But the qemuppc64 doesn't have its own linux-yocto kernel branch, so apply this change might cause regression with other platforms which share branch with qemuppc64.

So, we added an extra option for nfs rootfs, and set the qemuppc64 default r/w size to 524288 to line up with the kernel fix[1].

Yocto did a similar thing in the distant past[2] - prior to boot-arg adjustments existing - by allowing a Kconfig to set the defaults on nfsboot, in order to work around hardware limitations.

There are 3 patches for this fix:
the patch 1 added QB_NFSROOTFS_EXTRA_OPT in qemuboot.bbclass,it contains a default QB_NFSROOTFS_EXTRA_OPT value and some notes in comment. the patch 2 contains processing this value in runqemu script the patch 3 set the r/wsize value in qemuppc64 machine configuration file.

Reference:
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=940261a195080cf
[2] https://git.yoctoproject.org/linux-yocto-4.1/commit/?h=standard/base&id=a96cfd98add95

Xiangyu Chen (3):
qemuboot.bbclass: add QB_NFSROOTFS_EXTRA_OPT for nfs rootfs extra
option
runqemu: add process of option QB_NFSROOTFS_EXTRA_OPT
qemuppc64: set the qemuppc64 nfs r/wsize mount options to 524288

Quoted 4 lines
 meta/classes-recipe/qemuboot.bbclass | 3 +++
 meta/conf/machine/qemuppc64.conf     | 1 +
 scripts/runqemu                      | 6 +++++-
 3 files changed, 9 insertions(+), 1 deletion(-)

I've tried to understand this a couple of times and failed, I think there is some missing information in the commits.

What is the default page size on qemuppc64? What is the default nfs r/wsize? Is the issue the default page size on ppc64?

I also need to highlight that qemuppc64 isn't officially supported by Yocto Project and isn't part of our test matrix on the autobuilder. If anyone is relying on this support they really need to commit to maintaining that architecture.

Cheers,

Richard