Skip to contentSkip to conversation

linux/export: fix reference to exported functions for parisc64

PATCH In review Masahiro Yamada 4 participants 24 messages Sep 5, 2023 – Sep 14, 2023 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 linux/export: fix reference to exported functions for parisc64

John David Anglin reported parisc has been broken since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost").

I checked the assembler output, and noticed function references are prefixed with P%, so the situation in parisc64 is similar to ia64.

Fixes ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost")

---

I just checked the assembler output, and I created this patch based on my best guess. Only compile-tested.
I hope somebody will run-test this patch.

 include/linux/export-internal.h | 2 ++
 1 file changed, 2 insertions(+)
include/linux/export-internal.h +2 -0
index 1c849db953a5..45fca09b2319 100644
@@ -52,6 +52,8 @@
 
 #ifdef CONFIG_IA64
 #define KSYM_FUNC(name)		@fptr(name)
+#elif defined(CONFIG_PARISC) && defined(CONFIG_64BIT)
+#define KSYM_FUNC(name)		P%name
 #else
 #define KSYM_FUNC(name)		name
 #endif
Signature · 1 line
2.39.2
#1 lore raw

The patch get us slightly further but boot still fails in a similar way:

[...]
Run /init as init process
process '/usr/bin/sh' started with executable stack Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
usbcore alternatives: applied 0 out of 17 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
scsi_mod alternatives: applied 0 out of 7 patches
usbcore: registered new device driver usb
SCSI subsystem initialized
ehci_hcd alternatives: applied 0 out of 114 patches mptbase alternatives: applied 0 out of 73 patches
libata alternatives: applied 0 out of 3 patches
ehci_pci alternatives: applied 0 out of 2 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
ohci_hcd alternatives: applied 0 out of 144 patches ehci-pci 0000:60:01.2: EHCI Host Controller
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1

Backtrace:
 [<000000001071d5d8>] usb_hcd_pci_probe+0x330/0x4a0 [usbcore]
 [<000000001025e620>] ehci_pci_probe+0x50/0x70 [ehci_pci]
 [<00000000407d4004>] pci_device_probe+0x144/0x2a8
 [<000000004091df6c>] really_probe+0x12c/0x5a8
 [<000000004091e46c>] __driver_probe_device+0x84/0x1a0
 [<000000004091e66c>] driver_probe_device+0xe4/0x2c8
 [<000000004091eddc>] __driver_attach_async_helper+0x8c/0x160
 [<0000000040284ecc>] async_run_entry_fn+0x64/0x210
 [<000000004026b5e0>] process_one_work+0x268/0x478
 [<000000004026ba88>] worker_thread+0x298/0x740
 [<000000004027c3f4>] kthread+0x274/0x280
 [<0000000040202020>] ret_from_kernel_thread+0x20/0x28

Page fault: no context: Code=6 (Instruction TLB miss fault) at addr 0b3a029a8348

CPU: 3 PID: 57 Comm: kworker/u64:4 Not tainted 6.5.0+ #1
Hardware name: 9000/785/C8000
Workqueue: events_unbound async_run_entry_fn

     YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00001000000001001111111100001111 Not tainted
r00-03  000000ff0804ff0f 0b3a029a83406038 000000001025e770 0000000052008c30 r04-07  000000001025e000 0000000053501000 000000005156a0a0 0000000000000000 r08-11  000000005156a000 0000000053501800 0000000053501120 000000005156a0a0 r12-15  0000000000000002 0000000040d63640 00000000516b7328 0000000000000001 r16-19  0000000050d54580 0000000000000008 0000000050d54540 0000000000010000 r20-23  0000000000001a46 000000000000000f 0002000000000002 0000000000045b38 r24-27  0000000000000000 0000000000000003 0000000000000002 000000001025e000 r28-31  0000000000002395 0000000052008d40 0000000052008ce0 0000000000001033 sr00-03  00000000000cbc00 0000000000000000 0000000000000000 00000000000cbc00 sr04-07  0000000000000000 0000000000000000 0000000000000000 0000000000000000

IASQ: 000000000b3a029a 000000000b3a029a IAOQ: 0b3a029a83406038 0b3a029a8340603c
 IIR: 43ffff80    ISR: 0000000000000002  IOR: 0000000052008ea0 CPU:        3   CR30: 0000000051794c20 CR31: ffffffffffffffff
 ORIG_R28: 0000000000000000
 IAOQ[0]: 0xb3a029a83406038
 IAOQ[1]: 0xb3a029a8340603c
 RP(r2): ehci_pci_setup+0x100/0x780 [ehci_pci]

Backtrace:
 [<000000001071d5d8>] usb_hcd_pci_probe+0x330/0x4a0 [usbcore]
 [<000000001025e620>] ehci_pci_probe+0x50/0x70 [ehci_pci]
 [<00000000407d4004>] pci_device_probe+0x144/0x2a8
 [<000000004091df6c>] really_probe+0x12c/0x5a8
 [<000000004091e46c>] __driver_probe_device+0x84/0x1a0
 [<000000004091e66c>] driver_probe_device+0xe4/0x2c8
 [<000000004091eddc>] __driver_attach_async_helper+0x8c/0x160
 [<0000000040284ecc>] async_run_entry_fn+0x64/0x210
 [<000000004026b5e0>] process_one_work+0x268/0x478
 [<000000004026ba88>] worker_thread+0x298/0x740
 [<000000004027c3f4>] kthread+0x274/0x280
 [<0000000040202020>] ret_from_kernel_thread+0x20/0x28

Kernel panic - not syncing: Page fault: no context

This was with master.  I'll check ddb5cdbafaaa.

Dave

Quoted 33 lines · Masahiro Yamada

John David Anglin reported parisc has been broken since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost").

I checked the assembler output, and noticed function references are prefixed with P%, so the situation in parisc64 is similar to ia64.

Fixes ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost")

---

I just checked the assembler output, and I created this patch based on my best guess. Only compile-tested.
I hope somebody will run-test this patch.

  include/linux/export-internal.h | 2 ++
  1 file changed, 2 insertions(+)
include/linux/export-internal.h +2 -0
index 1c849db953a5..45fca09b2319 100644
@@ -52,6 +52,8 @@
 
  #ifdef CONFIG_IA64
  #define KSYM_FUNC(name)		@fptr(name)
+#elif defined(CONFIG_PARISC) && defined(CONFIG_64BIT)
+#define KSYM_FUNC(name)		P%name
  #else
  #define KSYM_FUNC(name)		name
  #endif
Signature · 1 line
John David Anglin  [email protected]
#2 lore raw
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix

Backtrace:
scsi host2: sata_sil24
 [<0000000040bf2c00>] mutex_lock+0x48/0xc8
 [<000000004023d370>] cpu_hotplug_disable+0x80/0x98
scsi host3: sata_sil24
 [<0000000040792314>] pci_device_probe+0x144/0x2a8
 [<00000000408af87c>] really_probe+0x12c/0x5a8
scsi host4: sata_sil24
 [<00000000408afd7c>] __driver_probe_device+0x84/0x1a0
 [<00000000408aff44>] driver_probe_device+0xac/0x260
scsi host5: sata_sil24
 [<00000000408b0684>] __driver_attach_async_helper+0x8c/0x160
 [<000000004028043c>] async_run_entry_fn+0x64/0x1d0
ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6
 [<0000000040269c88>] process_one_work+0x238/0x520
 [<000000004026a184>] worker_thread+0x214/0x770

ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 [<00000000402788d4>] kthread+0x274/0x280
ata5: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 [<0000000040202020>] ret_from_kernel_thread+0x20/0x28
ata6: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6

Page fault: no context: Code=6 (Instruction TLB miss fault) at addr 0b3a029a8348

CPU: 0 PID: 10 Comm: kworker/u64:0 Not tainted 6.4.0-rc2+ #1
Hardware name: 9000/785/C8000
Workqueue: events_unbound async_run_entry_fn

     YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
PSW: 00001000000001001111111100001111 Not tainted
r00-03  000000ff0804ff0f 0b3a029a83406038 0000000000010770 0000000050d94c50 r04-07  0000000000010000 0000000053a97000 00000000515398b0 0000000000000000 r08-11  0000000051539800 0000000053a97800 0000000053a97120 00000000515398b0 r12-15  0000000050c10000 0000000000000002 0000000040d54d60 0000000000000001 r16-19  0000000040ca1d20 0000000050ce46c0 0000000050d56150 0000000000020000 r20-23  000000007f41c000 000000000000000f 0002000000000002 0000000000044b38 r24-27  0000000000000000 0000000000000003 0000000000000002 0000000000010000 r28-31  0000000000002395 0000000050d94d60 0000000050d94d00 0000000000001033 sr00-03  00000000000c7000 0000000000000000 0000000000000000 00000000000c5c00 sr04-07  0000000000000000 0000000000000000 0000000000000000 0000000000000000

IASQ: 000000000b3a029a 000000000b3a029a IAOQ: 0b3a029a83406038 0b3a029a8340603c
 IIR: 43ffff80    ISR: 0000000000000dc0  IOR: 00000000402849ac CPU:        0   CR30: 0000000050d56150 CR31: ffffffffffffffff
 ORIG_R28: 0000000000000080
 IAOQ[0]: 0xb3a029a83406038
 IAOQ[1]: 0xb3a029a8340603c
 RP(r2): ehci_pci_setup+0x100/0x780 [ehci_pci]

Backtrace:
 [<0000000040bf2c00>] mutex_lock+0x48/0xc8
 [<000000004023d370>] cpu_hotplug_disable+0x80/0x98
 [<0000000040792314>] pci_device_probe+0x144/0x2a8
 [<00000000408af87c>] really_probe+0x12c/0x5a8
 [<00000000408afd7c>] __driver_probe_device+0x84/0x1a0
 [<00000000408aff44>] driver_probe_device+0xac/0x260
 [<00000000408b0684>] __driver_attach_async_helper+0x8c/0x160
 [<000000004028043c>] async_run_entry_fn+0x64/0x1d0
 [<0000000040269c88>] process_one_work+0x238/0x520
 [<000000004026a184>] worker_thread+0x214/0x770
 [<00000000402788d4>] kthread+0x274/0x280
 [<0000000040202020>] ret_from_kernel_thread+0x20/0x28

Kernel panic - not syncing: Page fault: no context

Dave

Signature · 1 line
John David Anglin  [email protected]
#3 lore raw
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.  I'm fairly certain that function references prefixed with P% on hppa64 need 8 byte alignment.

81662: 0000000040ea4358     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_system[...] 81663: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_syst[...] 81664: 0000000040e8e830     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_system[...] 81665: 0000000040ea4365     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_static[...] 81666: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_stat[...] 81667: 0000000040ea1640     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_static[...] 81668: 0000000040ea437c     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_reset_[...] 81669: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_rese[...] 81670: 0000000040e8bbc0     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_reset_[...] 81671: 0000000040ea438a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_loops_[...] 81672: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_loop[...] 81673: 0000000040e86610     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_loops_[...] 81674: 0000000040ea439a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_uts_ns 81675: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_init[...] 81676: 0000000040e99180     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_init_uts_ns 81677: 0000000040ea43a6     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_name_t[...] 81678: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_name[...] 81679: 0000000040e9b340     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_name_t[...] 81680: 0000000040ea43b4     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_wait_f[...] 81681: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_wait[...] 81682: 0000000040ea3638     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_wait_f[...] 81683: 0000000040ea43c7     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_task [...]

I'm not sure how we get symbols that aren't 8 byte aligned.  The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

Signature · 1 line
John David Anglin  [email protected]
#4 lore raw
Quoted 6 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 33 lines

81662: 0000000040ea4358 0 NOTYPE LOCAL DEFAULT 16 __kstrtab_system[...]

  81663: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_syst[...]
  81664: 0000000040e8e830     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_system[...]
  81665: 0000000040ea4365     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_static[...]
  81666: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_stat[...]
  81667: 0000000040ea1640     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_static[...]
  81668: 0000000040ea437c     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_reset_[...]
  81669: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_rese[...]
  81670: 0000000040e8bbc0     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_reset_[...]
  81671: 0000000040ea438a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_loops_[...]
  81672: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_loop[...]
  81673: 0000000040e86610     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_loops_[...]
  81674: 0000000040ea439a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_uts_ns
  81675: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_init[...]
  81676: 0000000040e99180     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_init_uts_ns
  81677: 0000000040ea43a6     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_name_t[...]
  81678: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_name[...]
  81679: 0000000040e9b340     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_name_t[...]
  81680: 0000000040ea43b4     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_wait_f[...]
  81681: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_wait[...]
  81682: 0000000040ea3638     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_wait_f[...]
  81683: 0000000040ea43c7     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_task

[...]

I'm not sure how we get symbols that aren't 8 byte aligned. The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

--
John David Anglin [email protected]

Signature · 2 lines
Best Regards
Masahiro Yamada
#5 lore raw

With a little more investigation,
I found arch/parisc/kernel/parisc_ksyms.c
is causing the issue.

That file is a collection of EXPORT_SYMBOL
of assembly code.

I will take a closer look tomorrow.

Quoted 61 lines · Masahiro Yamada
Quoted 6 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 25 lines

81662: 0000000040ea4358 0 NOTYPE LOCAL DEFAULT 16 __kstrtab_system[...]

  81663: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_syst[...]
  81664: 0000000040e8e830     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_system[...]
  81665: 0000000040ea4365     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_static[...]
  81666: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_stat[...]
  81667: 0000000040ea1640     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_static[...]
  81668: 0000000040ea437c     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_reset_[...]
  81669: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_rese[...]
  81670: 0000000040e8bbc0     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_reset_[...]
  81671: 0000000040ea438a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_loops_[...]
  81672: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_loop[...]
  81673: 0000000040e86610     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_loops_[...]
  81674: 0000000040ea439a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_uts_ns
  81675: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_init[...]
  81676: 0000000040e99180     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_init_uts_ns
  81677: 0000000040ea43a6     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_name_t[...]
  81678: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_name[...]
  81679: 0000000040e9b340     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_name_t[...]
  81680: 0000000040ea43b4     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_wait_f[...]
  81681: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_wait[...]
  81682: 0000000040ea3638     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_wait_f[...]
  81683: 0000000040ea43c7     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_task

[...]

Quoted 8 lines

I'm not sure how we get symbols that aren't 8 byte aligned. The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

--
John David Anglin [email protected]

--
Best Regards
Masahiro Yamada

Signature · 2 lines
Best Regards
Masahiro Yamada
#6 lore raw

Hi John, Helge,

Could you test the attached patch please?

Again, I only tested compilation for this. I do not have parisc64 hardware.
In my understanding, QEMU does not support hppa64. I do not find a way to test parisc64.

Masahiro Yamada

Quoted 88 lines · Masahiro Yamada

With a little more investigation,
I found arch/parisc/kernel/parisc_ksyms.c
is causing the issue.

That file is a collection of EXPORT_SYMBOL
of assembly code.

I will take a closer look tomorrow.

Quoted 53 lines · Masahiro Yamada
Quoted 6 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 30 lines

81662: 0000000040ea4358 0 NOTYPE LOCAL DEFAULT 16 __kstrtab_system[...]

  81663: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_syst[...]
  81664: 0000000040e8e830     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_system[...]
  81665: 0000000040ea4365     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_static[...]
  81666: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_stat[...]
  81667: 0000000040ea1640     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_static[...]
  81668: 0000000040ea437c     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_reset_[...]
  81669: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_rese[...]
  81670: 0000000040e8bbc0     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_reset_[...]
  81671: 0000000040ea438a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_loops_[...]
  81672: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_loop[...]
  81673: 0000000040e86610     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_loops_[...]
  81674: 0000000040ea439a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_uts_ns
  81675: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_init[...]
  81676: 0000000040e99180     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_init_uts_ns
  81677: 0000000040ea43a6     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_name_t[...]
  81678: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_name[...]
  81679: 0000000040e9b340     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_name_t[...]
  81680: 0000000040ea43b4     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_wait_f[...]
  81681: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_wait[...]
  81682: 0000000040ea3638     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_wait_f[...]
  81683: 0000000040ea43c7     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_task

[...]

I'm not sure how we get symbols that aren't 8 byte aligned. The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

Quoted 8 lines
Quoted 3 lines

--
John David Anglin [email protected]

--
Best Regards
Masahiro Yamada

--
Best Regards
Masahiro Yamada

Signature · 2 lines
Best Regards
Masahiro Yamada

From e473c9fa49801aea7c37d4cf19710d89ff358ab6 Mon Sep 17 00:00:00 2001
From: Masahiro Yamada <[email protected]>
Date: Wed, 6 Sep 2023 03:46:57 +0900
Subject: [PATCH] linux/export: fix reference to exported functions for
parisc64

John David Anglin reported parisc has been broken since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost").

Like ia64, parisc64 uses a function descriptor. The function references must be prefixed with P%.

Also, symbols prefixed $$ from the library have the symbol type STT_LOPROC instead of STT_FUNC. They should be handled as functions too.

Fixes ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost")
---
 include/linux/export-internal.h | 2 ++
 scripts/mod/modpost.c           | 9 +++++++++
 2 files changed, 11 insertions(+)
include/linux/export-internal.h +2 -0
index 1c849db953a5..45fca09b2319 100644
@@ -52,6 +52,8 @@
 
 #ifdef CONFIG_IA64
 #define KSYM_FUNC(name)		@fptr(name)
+#elif defined(CONFIG_PARISC) && defined(CONFIG_64BIT)
+#define KSYM_FUNC(name)		P%name
 #else
 #define KSYM_FUNC(name)		name
 #endif
scripts/mod/modpost.c +9 -0
index 34a5386d444a..de499dce5265 100644
@@ -1228,6 +1228,15 @@ static void check_export_symbol(struct module *mod, struct elf_info *elf,
 	 */
 	s->is_func = (ELF_ST_TYPE(sym->st_info) == STT_FUNC);
 
+	/*
+	 * For parisc64, symbols prefixed $$ from the library have the symbol type
+	 * STT_LOPROC. They should be handled as functions too.
+	 */
+	if (elf->hdr->e_ident[EI_CLASS] == ELFCLASS64 &&
+	    elf->hdr->e_machine == EM_PARISC &&
+	    ELF_ST_TYPE(sym->st_info) == STT_LOPROC)
+		s->is_func = true;
+
 	if (match(secname, PATTERNS(INIT_SECTIONS)))
 		warn("%s: %s: EXPORT_SYMBOL used for init symbol. Remove __init or EXPORT_SYMBOL.\n",
 		     mod->name, name);
Signature · 1 line
2.39.2
#7 lore raw

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6
ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6
ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6
ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72
ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5
random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ....
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

I'll see if I can find the commit that breaks 6.5.

Thanks,
Dave

Quoted 100 lines · Masahiro Yamada

Hi John, Helge,

Could you test the attached patch please?

Again, I only tested compilation for this. I do not have parisc64 hardware.
In my understanding, QEMU does not support hppa64. I do not find a way to test parisc64.

Masahiro Yamada

Quoted 80 lines · Masahiro Yamada

With a little more investigation,
I found arch/parisc/kernel/parisc_ksyms.c
is causing the issue.

That file is a collection of EXPORT_SYMBOL
of assembly code.

I will take a closer look tomorrow.

Quoted 48 lines · Masahiro Yamada
Quoted 5 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 29 lines

81662: 0000000040ea4358 0 NOTYPE LOCAL DEFAULT 16 __kstrtab_system[...]

   81663: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_syst[...]
   81664: 0000000040e8e830     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_system[...]
   81665: 0000000040ea4365     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_static[...]
   81666: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_stat[...]
   81667: 0000000040ea1640     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_static[...]
   81668: 0000000040ea437c     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_reset_[...]
   81669: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_rese[...]
   81670: 0000000040e8bbc0     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_reset_[...]
   81671: 0000000040ea438a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_loops_[...]
   81672: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_loop[...]
   81673: 0000000040e86610     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_loops_[...]
   81674: 0000000040ea439a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_uts_ns
   81675: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_init[...]
   81676: 0000000040e99180     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_init_uts_ns
   81677: 0000000040ea43a6     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_name_t[...]
   81678: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_name[...]
   81679: 0000000040e9b340     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_name_t[...]
   81680: 0000000040ea43b4     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_wait_f[...]
   81681: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_wait[...]
   81682: 0000000040ea3638     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_wait_f[...]
   81683: 0000000040ea43c7     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_task

[...]

I'm not sure how we get symbols that aren't 8 byte aligned. The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

Quoted 7 lines
Quoted 3 lines

--
John David Anglin [email protected]

--
Best Regards
Masahiro Yamada

--
Best Regards
Masahiro Yamada

Signature · 1 line
John David Anglin  [email protected]
Tested-by
#8 lore raw

Hi Masahiro,

I can confirm as well, that your patch
linux/export: fix reference to exported functions for parisc64
does indeed fix the boot issue on parisc64.

I did tested it on a C3000 workstation on top of Linus' v6.6-rc1 git tree. You may add:

Dave, I don't see the issue you mention below...

Helge

Quoted 189 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6
ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6
ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6
ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72
ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5
random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ....
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

I'll see if I can find the commit that breaks 6.5.

Thanks,
Dave

Quoted 85 lines · Masahiro Yamada

Hi John, Helge,

Could you test the attached patch please?

Again, I only tested compilation for this. I do not have parisc64 hardware.
In my understanding, QEMU does not support hppa64. I do not find a way to test parisc64.

Masahiro Yamada

Quoted 67 lines · Masahiro Yamada

With a little more investigation,
I found arch/parisc/kernel/parisc_ksyms.c
is causing the issue.

That file is a collection of EXPORT_SYMBOL
of assembly code.

I will take a closer look tomorrow.

Quoted 47 lines · Masahiro Yamada
Quoted 5 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

  I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 28 lines

81662: 0000000040ea4358     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_system[...] 81663: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_syst[...] 81664: 0000000040e8e830     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_system[...] 81665: 0000000040ea4365     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_static[...] 81666: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_stat[...] 81667: 0000000040ea1640     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_static[...] 81668: 0000000040ea437c     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_reset_[...] 81669: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_rese[...] 81670: 0000000040e8bbc0     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_reset_[...] 81671: 0000000040ea438a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_loops_[...] 81672: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_loop[...] 81673: 0000000040e86610     0 NOTYPE  LOCAL  DEFAULT   14 __ksymtab_loops_[...] 81674: 0000000040ea439a     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_uts_ns 81675: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_init[...] 81676: 0000000040e99180     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_init_uts_ns 81677: 0000000040ea43a6     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_name_t[...] 81678: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_name[...] 81679: 0000000040e9b340     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_name_t[...] 81680: 0000000040ea43b4     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_wait_f[...] 81681: 0000000040ea4748     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtabns_wait[...] 81682: 0000000040ea3638     0 NOTYPE  LOCAL  DEFAULT   15 __ksymtab_wait_f[...] 81683: 0000000040ea43c7     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_init_task [...]

I'm not sure how we get symbols that aren't 8 byte aligned.  The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

#9 lore raw

Hi Helge,

It occurs consistently on my c8000 but I'm having difficulty bisecting it.  Trying a bisect with --first-parent.

Note I had to pull ATI graphics card from the machine as it started to malfunction causing crashes. However, v6.1.52 boots fine.

Dave

Quoted 205 lines · Helge Deller

Hi Masahiro,

I can confirm as well, that your patch
 linux/export: fix reference to exported functions for parisc64 does indeed fix the boot issue on parisc64.

I did tested it on a C3000 workstation on top of Linus' v6.6-rc1 git tree. You may add:

Dave, I don't see the issue you mention below...

Helge

Quoted 189 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6
ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6
ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6
ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72
ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5
random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ....
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
  - Boot args (cat /proc/cmdline)
    - Check rootdelay= (did the system wait long enough?)
  - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

I'll see if I can find the commit that breaks 6.5.

Thanks,
Dave

Quoted 85 lines · Masahiro Yamada

Hi John, Helge,

Could you test the attached patch please?

Again, I only tested compilation for this. I do not have parisc64 hardware.
In my understanding, QEMU does not support hppa64. I do not find a way to test parisc64.

Masahiro Yamada

Quoted 67 lines · Masahiro Yamada

With a little more investigation,
I found arch/parisc/kernel/parisc_ksyms.c
is causing the issue.

That file is a collection of EXPORT_SYMBOL
of assembly code.

I will take a closer look tomorrow.

Quoted 47 lines · Masahiro Yamada
Quoted 5 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

  I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 28 lines

81662: 0000000040ea4358     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_system[...] 81663: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_syst[...] 81664: 0000000040e8e830     0 NOTYPE  LOCAL DEFAULT   14 __ksymtab_system[...] 81665: 0000000040ea4365     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_static[...] 81666: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_stat[...] 81667: 0000000040ea1640     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_static[...] 81668: 0000000040ea437c     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_reset_[...] 81669: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_rese[...] 81670: 0000000040e8bbc0     0 NOTYPE  LOCAL DEFAULT   14 __ksymtab_reset_[...] 81671: 0000000040ea438a     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_loops_[...] 81672: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_loop[...] 81673: 0000000040e86610     0 NOTYPE  LOCAL DEFAULT   14 __ksymtab_loops_[...] 81674: 0000000040ea439a     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_init_uts_ns 81675: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_init[...] 81676: 0000000040e99180     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_init_uts_ns 81677: 0000000040ea43a6     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_name_t[...] 81678: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_name[...] 81679: 0000000040e9b340     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_name_t[...] 81680: 0000000040ea43b4     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_wait_f[...] 81681: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_wait[...] 81682: 0000000040ea3638     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_wait_f[...] 81683: 0000000040ea43c7     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_init_task [...]

I'm not sure how we get symbols that aren't 8 byte aligned.  The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

Signature · 1 line
John David Anglin  [email protected]
#10 lore raw

Hi Dave,

Quoted 2 lines · John David Anglin

It occurs consistently on my c8000 but I'm having difficulty bisecting it.  Trying a bisect with --first-parent.

I just tried to boot the v6.6-rc1 with Masahiro's patch on c8000, and it succeeds as well. I've copied my pre-built kernel here:
http://backup.parisc-linux.org/kernel/linux-image-6.6.0-rc1-dirty_6.6.0-rc1-250_hppa.deb

So, I think Masahiro's patch is basically ok and probably isn't the root cause for your udev issues below.
Did you checked if initramfs included all necessary filesystem modules? Maybe updating your machine to latest ramfstools and re-installing your kernel?

Helge

Quoted 212 lines

Note I had to pull ATI graphics card from the machine as it started to malfunction causing crashes. However, v6.1.52 boots fine.

Quoted 205 lines · Helge Deller

Hi Masahiro,

I can confirm as well, that your patch
 linux/export: fix reference to exported functions for parisc64 does indeed fix the boot issue on parisc64.

I did tested it on a C3000 workstation on top of Linus' v6.6-rc1 git tree. You may add:

Dave, I don't see the issue you mention below...

Helge

Quoted 189 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6
ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6
ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6
ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72
ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5
random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ....
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
  - Boot args (cat /proc/cmdline)
    - Check rootdelay= (did the system wait long enough?)
  - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

I'll see if I can find the commit that breaks 6.5.

Thanks,
Dave

Quoted 85 lines · Masahiro Yamada

Hi John, Helge,

Could you test the attached patch please?

Again, I only tested compilation for this. I do not have parisc64 hardware.
In my understanding, QEMU does not support hppa64. I do not find a way to test parisc64.

Masahiro Yamada

Quoted 67 lines · Masahiro Yamada

With a little more investigation,
I found arch/parisc/kernel/parisc_ksyms.c
is causing the issue.

That file is a collection of EXPORT_SYMBOL
of assembly code.

I will take a closer look tomorrow.

Quoted 47 lines · Masahiro Yamada
Quoted 5 lines · John David Anglin
Quoted 3 lines · John David Anglin
Quoted 1 line · John David Anglin

I'll check ddb5cdbafaaa.

Similar fault with ddb5cdbafaaa:

The alignment of the __kstrtab_ symbols in vmlinux seems wrong.

__kstrtab_ symbols do not need alignment.

They were not aligned at all
before ddb5cdbafaaa^.

Quoted 2 lines

  I'm fairly certain that function
references prefixed with P% on hppa64 need 8 byte alignment.

Yeah.
In the following dump, all of __ksymtab_* are correctly 8-byte aligned.

Quoted 28 lines

81662: 0000000040ea4358     0 NOTYPE  LOCAL  DEFAULT   16 __kstrtab_system[...] 81663: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_syst[...] 81664: 0000000040e8e830     0 NOTYPE  LOCAL DEFAULT   14 __ksymtab_system[...] 81665: 0000000040ea4365     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_static[...] 81666: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_stat[...] 81667: 0000000040ea1640     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_static[...] 81668: 0000000040ea437c     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_reset_[...] 81669: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_rese[...] 81670: 0000000040e8bbc0     0 NOTYPE  LOCAL DEFAULT   14 __ksymtab_reset_[...] 81671: 0000000040ea438a     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_loops_[...] 81672: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_loop[...] 81673: 0000000040e86610     0 NOTYPE  LOCAL DEFAULT   14 __ksymtab_loops_[...] 81674: 0000000040ea439a     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_init_uts_ns 81675: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_init[...] 81676: 0000000040e99180     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_init_uts_ns 81677: 0000000040ea43a6     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_name_t[...] 81678: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_name[...] 81679: 0000000040e9b340     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_name_t[...] 81680: 0000000040ea43b4     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_wait_f[...] 81681: 0000000040ea4748     0 NOTYPE  LOCAL DEFAULT   16 __kstrtabns_wait[...] 81682: 0000000040ea3638     0 NOTYPE  LOCAL DEFAULT   15 __ksymtab_wait_f[...] 81683: 0000000040ea43c7     0 NOTYPE  LOCAL DEFAULT   16 __kstrtab_init_task [...]

I'm not sure how we get symbols that aren't 8 byte aligned.  The ".balign 4" directive in __KSYMTAB doesn't seem correct but it's not the whole problem.

Dave

#11 lore raw
Quoted 10 lines · Helge Deller
Quoted 2 lines · John David Anglin

It occurs consistently on my c8000 but I'm having difficulty bisecting it.  Trying a bisect with --first-parent.

I just tried to boot the v6.6-rc1 with Masahiro's patch on c8000, and it succeeds as well. I've copied my pre-built kernel here:
http://backup.parisc-linux.org/kernel/linux-image-6.6.0-rc1-dirty_6.6.0-rc1-250_hppa.deb

So, I think Masahiro's patch is basically ok and probably isn't the root cause for your udev issues below.

I agree.  I see the udev issue with the above kernel.  Continuing to bisect mainline.

Dave

Signature · 1 line
John David Anglin  [email protected]
#12 lore raw
In reply to John David Anglin
Quoted 98 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6 ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72 ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5 random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block .... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

Quoted 1 line

I'll see if I can find the commit that breaks 6.5.

I've traced this to the following merge commit:

dave@atlas:~/linux/linux$ git bisect good
ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7 is the first bad commit
commit ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7
Merge: 1546cd4bfda4 af92c02fb209
Author: Linus Torvalds <[email protected]>
Date:   Fri Jun 30 11:57:07 2023 -0700

    Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

    Pull SCSI updates from James Bottomley:
     "Updates to the usual drivers (ufs, pm80xx, libata-scsi, smartpqi, lpfc, qla2xxx).

      We have a couple of major core changes impacting other systems:

       - Command Duration Limits, which spills into block and ATA

       - block level Persistent Reservation Operations, which touches block, nvme, target and dm

      Both of these are added with merge commits containing a cover letter explaining what's going on"

    * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (187 commits)
      scsi: core: Improve warning message in scsi_device_block()
      scsi: core: Replace scsi_target_block() with scsi_block_targets()
      scsi: core: Don't wait for quiesce in scsi_device_block()
      scsi: core: Don't wait for quiesce in scsi_stop_queue()
      scsi: core: Merge scsi_internal_device_block() and device_block()
      scsi: sg: Increase number of devices
      scsi: bsg: Increase number of devices
      scsi: qla2xxx: Remove unused nvme_ls_waitq wait queue
      scsi: ufs: ufs-pci: Add support for Intel Arrow Lake
      scsi: sd: sd_zbc: Use PAGE_SECTORS_SHIFT
      scsi: ufs: wb: Add explicit flush_threshold sysfs attribute
      scsi: ufs: ufs-qcom: Switch to the new ICE API
      scsi: ufs: dt-bindings: qcom: Add ICE phandle
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_RTC quirk
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_INTR quirk
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_RTC
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_INTR
      scsi: ufs: core: Remove dedicated hwq for dev command
      scsi: ufs: core: mcq: Fix the incorrect OCS value for the device command
      scsi: ufs: dt-bindings: samsung,exynos: Drop unneeded quotes
      ...

dave@atlas:~/linux/linux$ lspci
00:01.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 02)
40:01.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
40:01.1 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
60:01.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
60:02.0 IDE interface: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
60:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

Dave

Signature · 1 line
John David Anglin  [email protected]
Reviewed-by Hannes Reinecke
#13 lore raw
Quoted 148 lines · John David Anglin
Quoted 98 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6 ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72 ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5 random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block .... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

Quoted 1 line

I'll see if I can find the commit that breaks 6.5.

I've traced this to the following merge commit:

dave@atlas:~/linux/linux$ git bisect good
ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7 is the first bad commit
commit ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7
Merge: 1546cd4bfda4 af92c02fb209
Author: Linus Torvalds <[email protected]>
Date:   Fri Jun 30 11:57:07 2023 -0700

    Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

    Pull SCSI updates from James Bottomley:
     "Updates to the usual drivers (ufs, pm80xx, libata-scsi, smartpqi, lpfc, qla2xxx).

      We have a couple of major core changes impacting other systems:

       - Command Duration Limits, which spills into block and ATA

       - block level Persistent Reservation Operations, which touches block, nvme, target and dm

      Both of these are added with merge commits containing a cover letter explaining what's going on"

    * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (187 commits)
      scsi: core: Improve warning message in scsi_device_block()
      scsi: core: Replace scsi_target_block() with scsi_block_targets()
      scsi: core: Don't wait for quiesce in scsi_device_block()
      scsi: core: Don't wait for quiesce in scsi_stop_queue()
      scsi: core: Merge scsi_internal_device_block() and device_block()
      scsi: sg: Increase number of devices
      scsi: bsg: Increase number of devices
      scsi: qla2xxx: Remove unused nvme_ls_waitq wait queue
      scsi: ufs: ufs-pci: Add support for Intel Arrow Lake
      scsi: sd: sd_zbc: Use PAGE_SECTORS_SHIFT
      scsi: ufs: wb: Add explicit flush_threshold sysfs attribute
      scsi: ufs: ufs-qcom: Switch to the new ICE API
      scsi: ufs: dt-bindings: qcom: Add ICE phandle
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_RTC quirk
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_INTR quirk
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_RTC
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_INTR
      scsi: ufs: core: Remove dedicated hwq for dev command
      scsi: ufs: core: mcq: Fix the incorrect OCS value for the device command
      scsi: ufs: dt-bindings: samsung,exynos: Drop unneeded quotes
      ...

Quoted 9 lines
dave@atlas:~/linux/linux$ lspci
00:01.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 02)
40:01.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
40:01.1 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
60:01.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
60:02.0 IDE interface: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
60:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

This was introduced by the following commit:

dave@atlas:~/linux/linux$ git bisect good
624885209f31eb9985bf51abe204ecbffe2fdeea is the first bad commit
commit 624885209f31eb9985bf51abe204ecbffe2fdeea
Author: Damien Le Moal <[email protected]>
Date:   Thu May 11 03:13:41 2023 +0200

    scsi: core: Detect support for command duration limits

    Introduce the function scsi_cdl_check() to detect if a device supports command duration limits (CDL). Support for the READ 16, WRITE 16, READ 32 and WRITE 32 commands are checked using the function scsi_report_opcode() to probe the rwcdlp and cdlp bits as they indicate the mode page defining the command duration limits descriptors that apply to the command being tested.

    If any of these commands support CDL, the field cdl_supported of struct scsi_device is set to 1 to indicate that the device supports CDL.

    Support for CDL for a device is advertizes through sysfs using the new cdl_supported device attribute. This attribute value is 1 for a device supporting CDL and 0 otherwise.

 Documentation/ABI/testing/sysfs-block-device |  9 ++++ drivers/scsi/scsi.c                          | 81 ++++++++++++++++++++++++++++ drivers/scsi/scsi_scan.c                     |  3 ++ drivers/scsi/scsi_sysfs.c                    |  2 + include/scsi/scsi_device.h                   |  3 ++

 5 files changed, 98 insertions(+)

Sometimes I see when booting a bad commit:
[...]
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell! Rebooting automatically due to panic= boot argument
ata4: SATA link down (SStatus 0 SControl 0)
ata5: SATA link down (SStatus 0 SControl 0)
ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata6.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata6.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata6.00: configured for UDMA/100
scsi 5:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5

Dave

Signature · 1 line
John David Anglin  [email protected]
#14 lore raw
Quoted 207 lines · John David Anglin
Quoted 148 lines · John David Anglin
Quoted 98 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6 ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72 ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5 random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block .... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

Quoted 1 line

I'll see if I can find the commit that breaks 6.5.

I've traced this to the following merge commit:

dave@atlas:~/linux/linux$ git bisect good
ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7 is the first bad commit
commit ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7
Merge: 1546cd4bfda4 af92c02fb209
Author: Linus Torvalds <[email protected]>
Date:   Fri Jun 30 11:57:07 2023 -0700

    Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

    Pull SCSI updates from James Bottomley:
     "Updates to the usual drivers (ufs, pm80xx, libata-scsi, smartpqi, lpfc, qla2xxx).

      We have a couple of major core changes impacting other systems:

       - Command Duration Limits, which spills into block and ATA

       - block level Persistent Reservation Operations, which touches block, nvme, target and dm

      Both of these are added with merge commits containing a cover letter explaining what's going on"

    * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (187 commits)
      scsi: core: Improve warning message in scsi_device_block()
      scsi: core: Replace scsi_target_block() with scsi_block_targets()
      scsi: core: Don't wait for quiesce in scsi_device_block()
      scsi: core: Don't wait for quiesce in scsi_stop_queue()
      scsi: core: Merge scsi_internal_device_block() and device_block()
      scsi: sg: Increase number of devices
      scsi: bsg: Increase number of devices
      scsi: qla2xxx: Remove unused nvme_ls_waitq wait queue
      scsi: ufs: ufs-pci: Add support for Intel Arrow Lake
      scsi: sd: sd_zbc: Use PAGE_SECTORS_SHIFT
      scsi: ufs: wb: Add explicit flush_threshold sysfs attribute
      scsi: ufs: ufs-qcom: Switch to the new ICE API
      scsi: ufs: dt-bindings: qcom: Add ICE phandle
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_RTC quirk
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_INTR quirk
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_RTC
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_INTR
      scsi: ufs: core: Remove dedicated hwq for dev command
      scsi: ufs: core: mcq: Fix the incorrect OCS value for the device command
      scsi: ufs: dt-bindings: samsung,exynos: Drop unneeded quotes
      ...

Quoted 9 lines
dave@atlas:~/linux/linux$ lspci
00:01.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 02)
40:01.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
40:01.1 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
60:01.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
60:02.0 IDE interface: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
60:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

This was introduced by the following commit:

dave@atlas:~/linux/linux$ git bisect good
624885209f31eb9985bf51abe204ecbffe2fdeea is the first bad commit
commit 624885209f31eb9985bf51abe204ecbffe2fdeea
Author: Damien Le Moal <[email protected]>
Date:   Thu May 11 03:13:41 2023 +0200

    scsi: core: Detect support for command duration limits

    Introduce the function scsi_cdl_check() to detect if a device supports command duration limits (CDL). Support for the READ 16, WRITE 16, READ 32 and WRITE 32 commands are checked using the function scsi_report_opcode() to probe the rwcdlp and cdlp bits as they indicate the mode page defining the command duration limits descriptors that apply to the command being tested.

    If any of these commands support CDL, the field cdl_supported of struct scsi_device is set to 1 to indicate that the device supports CDL.

    Support for CDL for a device is advertizes through sysfs using the new cdl_supported device attribute. This attribute value is 1 for a device supporting CDL and 0 otherwise.

 Documentation/ABI/testing/sysfs-block-device |  9 ++++ drivers/scsi/scsi.c                          | 81 ++++++++++++++++++++++++++++ drivers/scsi/scsi_scan.c                     |  3 ++ drivers/scsi/scsi_sysfs.c                    |  2 + include/scsi/scsi_device.h                   |  3 ++

 5 files changed, 98 insertions(+)

Sometimes I see when booting a bad commit:
[...]
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev) ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!

Quoted 8 lines

Rebooting automatically due to panic= boot argument
ata4: SATA link down (SStatus 0 SControl 0)
ata5: SATA link down (SStatus 0 SControl 0)
ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata6.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata6.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata6.00: configured for UDMA/100
scsi 5:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5

System boots master at e56b2b605799 if I disable CDL:

dave@atlas:~/linux/linux$ git diff drivers/scsi/scsi.c
drivers/scsi/scsi.c +0 -0
index d0911bc28663..dc3a283ebd75 100644
@@ -578,6 +578,8 @@ static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa,

        int ret;
        u8 cdlp;

+       return false;
+
        /* Check operation code */
        ret = scsi_report_opcode(sdev, buf, SCSI_CDL_CHECK_BUF_LEN, opcode, sa); if (ret <= 0)

Dave

Signature · 1 line
John David Anglin  [email protected]
maintainer
#15 lore raw
Quoted 225 lines · John David Anglin
Quoted 207 lines · John David Anglin
Quoted 148 lines · John David Anglin
Quoted 98 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6 ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72 ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5 random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block .... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

Quoted 1 line

I'll see if I can find the commit that breaks 6.5.

I've traced this to the following merge commit:

dave@atlas:~/linux/linux$ git bisect good
ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7 is the first bad commit
commit ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7
Merge: 1546cd4bfda4 af92c02fb209
Author: Linus Torvalds <[email protected]>
Date:   Fri Jun 30 11:57:07 2023 -0700

    Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

    Pull SCSI updates from James Bottomley:
     "Updates to the usual drivers (ufs, pm80xx, libata-scsi, smartpqi, lpfc, qla2xxx).

      We have a couple of major core changes impacting other systems:

       - Command Duration Limits, which spills into block and ATA

       - block level Persistent Reservation Operations, which touches block, nvme, target and dm

      Both of these are added with merge commits containing a cover letter explaining what's going on"

    * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (187 commits)
      scsi: core: Improve warning message in scsi_device_block()
      scsi: core: Replace scsi_target_block() with scsi_block_targets()
      scsi: core: Don't wait for quiesce in scsi_device_block()
      scsi: core: Don't wait for quiesce in scsi_stop_queue()
      scsi: core: Merge scsi_internal_device_block() and device_block()
      scsi: sg: Increase number of devices
      scsi: bsg: Increase number of devices
      scsi: qla2xxx: Remove unused nvme_ls_waitq wait queue
      scsi: ufs: ufs-pci: Add support for Intel Arrow Lake
      scsi: sd: sd_zbc: Use PAGE_SECTORS_SHIFT
      scsi: ufs: wb: Add explicit flush_threshold sysfs attribute
      scsi: ufs: ufs-qcom: Switch to the new ICE API
      scsi: ufs: dt-bindings: qcom: Add ICE phandle
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_RTC quirk
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_INTR quirk
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_RTC
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_INTR
      scsi: ufs: core: Remove dedicated hwq for dev command
      scsi: ufs: core: mcq: Fix the incorrect OCS value for the device command
      scsi: ufs: dt-bindings: samsung,exynos: Drop unneeded quotes
      ...

Quoted 9 lines
dave@atlas:~/linux/linux$ lspci
00:01.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 02)
40:01.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
40:01.1 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
60:01.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
60:02.0 IDE interface: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
60:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

This was introduced by the following commit:

dave@atlas:~/linux/linux$ git bisect good
624885209f31eb9985bf51abe204ecbffe2fdeea is the first bad commit
commit 624885209f31eb9985bf51abe204ecbffe2fdeea
Author: Damien Le Moal <[email protected]>
Date:   Thu May 11 03:13:41 2023 +0200

    scsi: core: Detect support for command duration limits

    Introduce the function scsi_cdl_check() to detect if a device supports command duration limits (CDL). Support for the READ 16, WRITE 16, READ 32 and WRITE 32 commands are checked using the function scsi_report_opcode() to probe the rwcdlp and cdlp bits as they indicate the mode page defining the command duration limits descriptors that apply to the command being tested.

    If any of these commands support CDL, the field cdl_supported of struct scsi_device is set to 1 to indicate that the device supports CDL.

    Support for CDL for a device is advertizes through sysfs using the new cdl_supported device attribute. This attribute value is 1 for a device supporting CDL and 0 otherwise.

 Documentation/ABI/testing/sysfs-block-device |  9 ++++ drivers/scsi/scsi.c                          | 81 ++++++++++++++++++++++++++++ drivers/scsi/scsi_scan.c                     |  3 ++ drivers/scsi/scsi_sysfs.c                    |  2 + include/scsi/scsi_device.h                   |  3 ++

 5 files changed, 98 insertions(+)

Sometimes I see when booting a bad commit:
[...]
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev) ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!

Quoted 8 lines

Rebooting automatically due to panic= boot argument
ata4: SATA link down (SStatus 0 SControl 0)
ata5: SATA link down (SStatus 0 SControl 0)
ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata6.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata6.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata6.00: configured for UDMA/100
scsi 5:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5

System boots master at e56b2b605799 if I disable CDL:

dave@atlas:~/linux/linux$ git diff drivers/scsi/scsi.c
drivers/scsi/scsi.c +0 -0
index d0911bc28663..dc3a283ebd75 100644

drivers/scsi/scsi.c static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa, · line 585

Quoted 8 lines
drivers/scsi/scsi.c +2 -0
@@ -578,6 +578,8 @@ static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa,
         int ret;
         u8 cdlp;
 
+       return false;
+
         /* Check operation code */
         ret = scsi_report_opcode(sdev, buf, SCSI_CDL_CHECK_BUF_LEN, opcode, sa);
         if (ret <= 0)

It is weird that this solves anything... the MAINTENANCE_IN command issued by scsi_report_opcode() ends up being emulated in libata with ata_scsiop_maint_in(). There are no actual commands issued to the drive, so nothing that could actually fail/cause issues. By the time this is issued, the ATA drive is also fully probed...

Or is the drive connected to the Broadcom HBA you have ? In that case, libata is not used and the HBA FW SAT (scsi-ata-translation) is likely to blame.

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

Signature · 2 lines
Damien Le Moal
Western Digital Research
#16 lore raw
Quoted 239 lines · Damien Le Moal
Quoted 224 lines · John David Anglin
Quoted 207 lines · John David Anglin
Quoted 148 lines · John David Anglin
Quoted 98 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6 ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72 ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5 random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block .... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

Quoted 1 line

I'll see if I can find the commit that breaks 6.5.

I've traced this to the following merge commit:

dave@atlas:~/linux/linux$ git bisect good
ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7 is the first bad commit
commit ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7
Merge: 1546cd4bfda4 af92c02fb209
Author: Linus Torvalds <[email protected]>
Date:   Fri Jun 30 11:57:07 2023 -0700

    Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

    Pull SCSI updates from James Bottomley:
     "Updates to the usual drivers (ufs, pm80xx, libata-scsi, smartpqi, lpfc, qla2xxx).

      We have a couple of major core changes impacting other systems:

       - Command Duration Limits, which spills into block and ATA

       - block level Persistent Reservation Operations, which touches block, nvme, target and dm

      Both of these are added with merge commits containing a cover letter explaining what's going on"

    * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (187 commits)
      scsi: core: Improve warning message in scsi_device_block()
      scsi: core: Replace scsi_target_block() with scsi_block_targets()
      scsi: core: Don't wait for quiesce in scsi_device_block()
      scsi: core: Don't wait for quiesce in scsi_stop_queue()
      scsi: core: Merge scsi_internal_device_block() and device_block()
      scsi: sg: Increase number of devices
      scsi: bsg: Increase number of devices
      scsi: qla2xxx: Remove unused nvme_ls_waitq wait queue
      scsi: ufs: ufs-pci: Add support for Intel Arrow Lake
      scsi: sd: sd_zbc: Use PAGE_SECTORS_SHIFT
      scsi: ufs: wb: Add explicit flush_threshold sysfs attribute
      scsi: ufs: ufs-qcom: Switch to the new ICE API
      scsi: ufs: dt-bindings: qcom: Add ICE phandle
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_RTC quirk
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_INTR quirk
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_RTC
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_INTR
      scsi: ufs: core: Remove dedicated hwq for dev command
      scsi: ufs: core: mcq: Fix the incorrect OCS value for the device command
      scsi: ufs: dt-bindings: samsung,exynos: Drop unneeded quotes
      ...

Quoted 9 lines
dave@atlas:~/linux/linux$ lspci
00:01.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 02)
40:01.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
40:01.1 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
60:01.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
60:02.0 IDE interface: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
60:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

This was introduced by the following commit:

dave@atlas:~/linux/linux$ git bisect good
624885209f31eb9985bf51abe204ecbffe2fdeea is the first bad commit
commit 624885209f31eb9985bf51abe204ecbffe2fdeea
Author: Damien Le Moal <[email protected]>
Date:   Thu May 11 03:13:41 2023 +0200

    scsi: core: Detect support for command duration limits

    Introduce the function scsi_cdl_check() to detect if a device supports command duration limits (CDL). Support for the READ 16, WRITE 16, READ 32 and WRITE 32 commands are checked using the function scsi_report_opcode() to probe the rwcdlp and cdlp bits as they indicate the mode page defining the command duration limits descriptors that apply to the command being tested.

    If any of these commands support CDL, the field cdl_supported of struct scsi_device is set to 1 to indicate that the device supports CDL.

    Support for CDL for a device is advertizes through sysfs using the new cdl_supported device attribute. This attribute value is 1 for a device supporting CDL and 0 otherwise.

  Documentation/ABI/testing/sysfs-block-device |  9 ++++
  drivers/scsi/scsi.c                          | 81 ++++++++++++++++++++++++++++
  drivers/scsi/scsi_scan.c                     |  3 ++
  drivers/scsi/scsi_sysfs.c                    |  2 +
  include/scsi/scsi_device.h                   |  3 ++
  5 files changed, 98 insertions(+)

Sometimes I see when booting a bad commit:
[...]
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!

Quoted 8 lines

Rebooting automatically due to panic= boot argument
ata4: SATA link down (SStatus 0 SControl 0)
ata5: SATA link down (SStatus 0 SControl 0)
ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata6.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata6.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata6.00: configured for UDMA/100
scsi 5:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5

System boots master at e56b2b605799 if I disable CDL:

dave@atlas:~/linux/linux$ git diff drivers/scsi/scsi.c
drivers/scsi/scsi.c +0 -0
index d0911bc28663..dc3a283ebd75 100644

drivers/scsi/scsi.c static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa, · line 585

Quoted 8 lines
drivers/scsi/scsi.c +2 -0
@@ -578,6 +578,8 @@ static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa,
          int ret;
          u8 cdlp;
 
+       return false;
+
          /* Check operation code */
          ret = scsi_report_opcode(sdev, buf, SCSI_CDL_CHECK_BUF_LEN, opcode, sa);
          if (ret <= 0)

It is weird that this solves anything... the MAINTENANCE_IN command issued by scsi_report_opcode() ends up being emulated in libata with ata_scsiop_maint_in(). There are no actual commands issued to the drive, so nothing that could actually fail/cause issues. By the time this is issued, the ATA drive is also fully probed...

Quoted 2 lines

Or is the drive connected to the Broadcom HBA you have ? In that case, libata is not used and the HBA FW SAT (scsi-ata-translation) is likely to blame.

/boot, / and swap partitions reside on a ST373207LW drive connected to a Broadcom HBA.  A ST4000VN008-2DR1 drive is connected to the  Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller.  It mounts on /home.  There's also a cdrom connected to the Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller and another ST4000VN008-2DR1 drive connected to a Broadcom HBA.  There are two Broadcom HBAs.

I think the issue is with the root ST373207LW drive.  The console output indicates that the ROOT drive doesn't exist when the boot fails.

Your change only appeared to affect actual SCSI drives.  That's why I tried disabling CDL.

Quoted 3 lines

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

I'll try to get this together tomorrow.

Dave

Signature · 1 line
John David Anglin  [email protected]
maintainer
#17 lore raw
Quoted 28 lines · John David Anglin
Quoted 20 lines
Quoted 6 lines
dave@atlas:~/linux/linux$ git diff drivers/scsi/scsi.c
drivers/scsi/scsi.c +0 -0
index d0911bc28663..dc3a283ebd75 100644

drivers/scsi/scsi.c static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa, · line 585

Quoted 8 lines
drivers/scsi/scsi.c +2 -0
@@ -578,6 +578,8 @@ static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa,
          int ret;
          u8 cdlp;
 
+       return false;
+
          /* Check operation code */
          ret = scsi_report_opcode(sdev, buf, SCSI_CDL_CHECK_BUF_LEN, opcode, sa);
          if (ret <= 0)

It is weird that this solves anything... the MAINTENANCE_IN command issued by scsi_report_opcode() ends up being emulated in libata with ata_scsiop_maint_in(). There are no actual commands issued to the drive, so nothing that could actually fail/cause issues. By the time this is issued, the ATA drive is also fully probed...

Quoted 2 lines

Or is the drive connected to the Broadcom HBA you have ? In that case, libata is not used and the HBA FW SAT (scsi-ata-translation) is likely to blame.

/boot, / and swap partitions reside on a ST373207LW drive connected to a Broadcom HBA.  A ST4000VN008-2DR1 drive is connected to the  Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller.  It mounts on /home.  There's also a cdrom connected to the Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller and another ST4000VN008-2DR1 drive connected to a Broadcom HBA.  There are two Broadcom HBAs.

Quoted 4 lines

I think the issue is with the root ST373207LW drive.  The console output indicates that the ROOT drive doesn't exist when the boot fails.

Your change only appeared to affect actual SCSI drives.  That's why I tried disabling CDL.

OK. I can see from the dmesg snippets you sent that the drives on the ATA ports seem OK. A quick search tells me that the ST373207LW drive is a Ultra320 SCSI drive, not ATA. So that MAINTENANCE_IN command issued by scsi_report_opcode() will straight as-is.

This command has been issued to devices since a long time ago, and given that your system was working, the drive is probably fine with it in its simplest form (one command format). CDL changes however added probing command support with the service action field (One command format with service action). And what may be happening is that the drive does not like/does not support that format and chokes on it.

Let me check the specs to see what scsi level support this format. What is sure is that Ultra320 SCSI disks will definitely *not* support CDL, so we could exit early in scsi_cdl_check_cmd() returning false for drives with an old scsi level support.

Let me send something along these lines.

Quoted 7 lines
Quoted 3 lines

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

I'll try to get this together tomorrow.

Dave

Signature · 2 lines
Damien Le Moal
Western Digital Research
maintainer
#18 lore raw
In reply to John David Anglin
Quoted 248 lines · John David Anglin
Quoted 239 lines · Damien Le Moal
Quoted 224 lines · John David Anglin
Quoted 207 lines · John David Anglin
Quoted 148 lines · John David Anglin
Quoted 98 lines · John David Anglin

Hi Masahiro,

The attached change fixed boot at ddb5cdbafaaa 😁

However, v6.5.x boot is still broken:

Run /init as init process
process '/usr/bin/sh' started with executable stack
Loading, please wait...
Starting systemd-udevd version 254.1-3
e1000 alternatives: applied 0 out of 569 patches
e1000: Intel(R) PRO/1000 Network Driver
e1000: Copyright (c) 1999-2006 Intel Corporation.
scsi_mod alternatives: applied 0 out of 7 patches
SCSI subsystem initialized
usbcore alternatives: applied 0 out of 18 patches
usbcore: registered new interface driver usbfs
libata alternatives: applied 0 out of 3 patches
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mptbase alternatives: applied 0 out of 73 patches
ehci_hcd alternatives: applied 0 out of 114 patches
sata_sil24 alternatives: applied 0 out of 56 patches
Fusion MPT base driver 3.04.20
Copyright (c) 1999-2008 LSI Corporation
sata_sil24 0000:00:01.0: Applying completion IRQ loss on PCI-X errata fix scsi host0: sata_sil24
scsi host1: sata_sil24
pata_sil680 0000:60:02.0: sil680: 133MHz clock.
scsi host2: sata_sil24
ehci_pci alternatives: applied 0 out of 2 patches
ohci_hcd alternatives: applied 0 out of 144 patches
ehci-pci 0000:60:01.2: EHCI Host Controller
scsi host3: pata_sil680
ehci-pci 0000:60:01.2: new USB bus registered, assigned bus number 1 scsi host4: sata_sil24
ata1: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80080000 ir6 ata2: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80082000 ir6 ata3: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80084000 ir6 ata4: SATA max UDMA/100 host m128@0xffffffff80088000 port 0xffffffff80086000 ir6 e1000 0000:60:03.0 eth0: (PCI:33MHz:32-bit) 00:11:0a:31:8a:77 ehci-pci 0000:60:01.2: irq 71, io mem 0xffffffffb00a1000
scsi host5: pata_sil680
ata5: PATA max UDMA/133 cmd 0x26058 ctl 0x26064 bmdma 0x26040 irq 72 ata6: PATA max UDMA/133 cmd 0x26050 ctl 0x26060 bmdma 0x26048 irq 72 e1000 0000:60:03.0 eth0: Intel(R) PRO/1000 Network Connection ehci-pci 0000:60:01.2: USB 2.0 started, EHCI 0.95
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.05 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 6.5.2-dirty ehci_hcd
usb usb1: SerialNumber: 0000:60:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ata1: SATA link down (SStatus 0 SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 0)
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata4.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata4.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/100
scsi 4:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5 ata6.00: ATAPI: HL-DT-STDVD+-RW GSA-H21L, 1.04, max UDMA/44
scsi 5:0:0:0: CD-ROM            HL-DT-ST DVD+-RW GSA-H21L 1.04 PQ: 0 ANSI: 5 random: crng init done
Timed out for waiting the udev queue being empty.
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... done.
Timed out for waiting the udev queue being empty.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block .... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!
Rebooting automatically due to panic= boot argument

Quoted 1 line

I'll see if I can find the commit that breaks 6.5.

I've traced this to the following merge commit:

dave@atlas:~/linux/linux$ git bisect good
ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7 is the first bad commit
commit ca7ce08d6a063e0ccb91dc57f9bc213120d0d1a7
Merge: 1546cd4bfda4 af92c02fb209
Author: Linus Torvalds <[email protected]>
Date:   Fri Jun 30 11:57:07 2023 -0700

    Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi

    Pull SCSI updates from James Bottomley:
     "Updates to the usual drivers (ufs, pm80xx, libata-scsi, smartpqi, lpfc, qla2xxx).

      We have a couple of major core changes impacting other systems:

       - Command Duration Limits, which spills into block and ATA

       - block level Persistent Reservation Operations, which touches block, nvme, target and dm

      Both of these are added with merge commits containing a cover letter explaining what's going on"

    * tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi: (187 commits)
      scsi: core: Improve warning message in scsi_device_block()
      scsi: core: Replace scsi_target_block() with scsi_block_targets()
      scsi: core: Don't wait for quiesce in scsi_device_block()
      scsi: core: Don't wait for quiesce in scsi_stop_queue()
      scsi: core: Merge scsi_internal_device_block() and device_block()
      scsi: sg: Increase number of devices
      scsi: bsg: Increase number of devices
      scsi: qla2xxx: Remove unused nvme_ls_waitq wait queue
      scsi: ufs: ufs-pci: Add support for Intel Arrow Lake
      scsi: sd: sd_zbc: Use PAGE_SECTORS_SHIFT
      scsi: ufs: wb: Add explicit flush_threshold sysfs attribute
      scsi: ufs: ufs-qcom: Switch to the new ICE API
      scsi: ufs: dt-bindings: qcom: Add ICE phandle
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_RTC quirk
      scsi: ufs: ufs-mediatek: Set UFSHCD_QUIRK_MCQ_BROKEN_INTR quirk
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_RTC
      scsi: ufs: core: Add host quirk UFSHCD_QUIRK_MCQ_BROKEN_INTR
      scsi: ufs: core: Remove dedicated hwq for dev command
      scsi: ufs: core: mcq: Fix the incorrect OCS value for the device command
      scsi: ufs: dt-bindings: samsung,exynos: Drop unneeded quotes
      ...

Quoted 9 lines
dave@atlas:~/linux/linux$ lspci
00:01.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 02)
40:01.0 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
40:01.1 SCSI storage controller: Broadcom / LSI 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
60:01.0 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.1 USB controller: NEC Corporation OHCI USB Controller (rev 41)
60:01.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 02)
60:02.0 IDE interface: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02)
60:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

This was introduced by the following commit:

dave@atlas:~/linux/linux$ git bisect good
624885209f31eb9985bf51abe204ecbffe2fdeea is the first bad commit
commit 624885209f31eb9985bf51abe204ecbffe2fdeea
Author: Damien Le Moal <[email protected]>
Date:   Thu May 11 03:13:41 2023 +0200

    scsi: core: Detect support for command duration limits

    Introduce the function scsi_cdl_check() to detect if a device supports command duration limits (CDL). Support for the READ 16, WRITE 16, READ 32 and WRITE 32 commands are checked using the function scsi_report_opcode() to probe the rwcdlp and cdlp bits as they indicate the mode page defining the command duration limits descriptors that apply to the command being tested.

    If any of these commands support CDL, the field cdl_supported of struct scsi_device is set to 1 to indicate that the device supports CDL.

    Support for CDL for a device is advertizes through sysfs using the new cdl_supported device attribute. This attribute value is 1 for a device supporting CDL and 0 otherwise.

  Documentation/ABI/testing/sysfs-block-device |  9 ++++
  drivers/scsi/scsi.c                          | 81 ++++++++++++++++++++++++++++
  drivers/scsi/scsi_scan.c                     |  3 ++
  drivers/scsi/scsi_sysfs.c                    |  2 +
  include/scsi/scsi_device.h                   |  3 ++
  5 files changed, 98 insertions(+)

Sometimes I see when booting a bad commit:
[...]
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  LABEL=ROOT does not exist.  Dropping to a shell!

Quoted 8 lines

Rebooting automatically due to panic= boot argument
ata4: SATA link down (SStatus 0 SControl 0)
ata5: SATA link down (SStatus 0 SControl 0)
ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
ata6.00: ATA-10: ST4000VN008-2DR166, SC60, max UDMA/133
ata6.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 31/32) ata6.00: configured for UDMA/100
scsi 5:0:0:0: Direct-Access     ATA      ST4000VN008-2DR1 SC60 PQ: 0 ANSI: 5

System boots master at e56b2b605799 if I disable CDL:

dave@atlas:~/linux/linux$ git diff drivers/scsi/scsi.c
drivers/scsi/scsi.c +0 -0
index d0911bc28663..dc3a283ebd75 100644

drivers/scsi/scsi.c static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa, · line 585

Quoted 8 lines
drivers/scsi/scsi.c +2 -0
@@ -578,6 +578,8 @@ static bool scsi_cdl_check_cmd(struct scsi_device *sdev, u8 opcode, u16 sa,
          int ret;
          u8 cdlp;
 
+       return false;
+
          /* Check operation code */
          ret = scsi_report_opcode(sdev, buf, SCSI_CDL_CHECK_BUF_LEN, opcode, sa);
          if (ret <= 0)

It is weird that this solves anything... the MAINTENANCE_IN command issued by scsi_report_opcode() ends up being emulated in libata with ata_scsiop_maint_in(). There are no actual commands issued to the drive, so nothing that could actually fail/cause issues. By the time this is issued, the ATA drive is also fully probed...

Quoted 2 lines

Or is the drive connected to the Broadcom HBA you have ? In that case, libata is not used and the HBA FW SAT (scsi-ata-translation) is likely to blame.

/boot, / and swap partitions reside on a ST373207LW drive connected to a Broadcom HBA.  A ST4000VN008-2DR1 drive is connected to the  Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller.  It mounts on /home.  There's also a cdrom connected to the Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller and another ST4000VN008-2DR1 drive connected to a Broadcom HBA.  There are two Broadcom HBAs.

Quoted 8 lines

I think the issue is with the root ST373207LW drive.  The console output indicates that the ROOT drive doesn't exist when the boot fails.

Your change only appeared to affect actual SCSI drives.  That's why I tried disabling CDL.

Quoted 3 lines

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

I'll try to get this together tomorrow.

Please also tell me the scsi_level reported for that drive (cat /sys/block/sdX/device/scsi_level and output of sg_inq /dev/sdX).

Thanks !

Quoted 3 lines

Dave

Signature · 2 lines
Damien Le Moal
Western Digital Research
maintainer
#19 lore raw
In reply to John David Anglin
Quoted 8 lines · John David Anglin

I think the issue is with the root ST373207LW drive.  The console output indicates that the ROOT drive doesn't exist when the boot fails.

Your change only appeared to affect actual SCSI drives.  That's why I tried disabling CDL.

Quoted 3 lines

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

I'll try to get this together tomorrow.

Please try the attached patch. That should address the issue with your drive.

Signature · 2 lines
Damien Le Moal
Western Digital Research

From 94d05ce51d5c8a6f47383afd14134f3c779d89e2 Mon Sep 17 00:00:00 2001
From: Damien Le Moal <[email protected]>
Date: Thu, 14 Sep 2023 11:08:58 +0900
Subject: [PATCH] scsi: Do no try to probe for CDL on old drives

Some old drives (e.g. an Ultra320 SCSI disk as reported by John) do not seem to execute MAINTENANCE_IN / MI_REPORT_SUPPORTED_OPERATION_CODES commands correctly and hang when a non-zero service action is specified (one command format with service action case in scsi_report_opcode()).

Currently, CDL probing with scsi_cdl_check_cmd() is the only caller using a non zero service action for scsi_report_opcode(). To avoid issues with these old drives, do not attempt CDL probe if the device reports support for an SPC version lower than 5 (CDL was introduced in SPC-5). To keep things working with ATA devices which probe for the CDL T2A and T2B pages introduced with SPC-6, modify ata_scsiop_inq_std() to claim SPC-6 version compatibility for ATA drives supporting CDL.

include/scsi/scsi.h is also modified to add the missing definitions for the SCSI_SPC_4 and SCSI_SPC_5 versions. SCSI_SPC_6 is not added as, oddly, the latest SPC-6 specification defines the same 7h version code as SPC-5.

Fixes 624885209f31 ("scsi: core: Detect support for command duration limits")
---
 drivers/ata/libata-scsi.c |  3 +++
 drivers/scsi/scsi.c       | 11 +++++++++++
 include/scsi/scsi.h       |  2 ++
 3 files changed, 16 insertions(+)
drivers/ata/libata-scsi.c +3 -0
index 92ae4b4f30ac..654ee9a0c064 100644
@@ -1828,6 +1828,9 @@ static unsigned int ata_scsiop_inq_std(struct ata_scsi_args *args, u8 *rbuf)
 		hdr[2] = 0x7; /* claim SPC-5 version compatibility */
 	}
 
+	if (args->dev->flags & ATA_DFLAG_CDL)
+		hdr[2] = 0x7; /* claim SPC-6 version compatibility */
+
 	memcpy(rbuf, hdr, sizeof(hdr));
 	memcpy(&rbuf[8], "ATA     ", 8);
 	ata_id_string(args->id, &rbuf[16], ATA_ID_PROD, 16);
drivers/scsi/scsi.c +11 -0
index d0911bc28663..89367c4bf0ef 100644
@@ -613,6 +613,17 @@ void scsi_cdl_check(struct scsi_device *sdev)
 	bool cdl_supported;
 	unsigned char *buf;
 
+	/*
+	 * Support for CDL was defined in SPC-5. Ignore devices reporting an
+	 * lower SPC version. This also avoids problems with old drives choking
+	 * on MAINTENANCE_IN / MI_REPORT_SUPPORTED_OPERATION_CODES with a
+	 * service action specified, as done in scsi_cdl_check_cmd().
+	 */
+	if (sdev->scsi_level < SCSI_SPC_5) {
+		sdev->cdl_supported = 0;
+		return;
+	}
+
 	buf = kmalloc(SCSI_CDL_CHECK_BUF_LEN, GFP_KERNEL);
 	if (!buf) {
 		sdev->cdl_supported = 0;
include/scsi/scsi.h +2 -0
index ec093594ba53..39f6bc7bff0f 100644
@@ -157,6 +157,8 @@ enum scsi_disposition {
 #define SCSI_3          4        /* SPC */
 #define SCSI_SPC_2      5
 #define SCSI_SPC_3      6
+#define SCSI_SPC_4	7
+#define SCSI_SPC_5	8
 
 /*
  * INQ PERIPHERAL QUALIFIERS
Signature · 1 line
2.41.0
#20 lore raw
Quoted 9 lines · Damien Le Moal
Quoted 7 lines · John David Anglin

I think the issue is with the root ST373207LW drive.  The console output indicates that the ROOT drive doesn't exist when the boot fails.

Your change only appeared to affect actual SCSI drives.  That's why I tried disabling CDL.

Quoted 2 lines

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

I'll try to get this together tomorrow.

Please try the attached patch. That should address the issue with your drive.

Mainline and v6.5.3 both booted successfully with the attached patch.

Thanks,
Dave

Signature · 1 line
John David Anglin  [email protected]
maintainer
#21 lore raw
Quoted 11 lines · John David Anglin
Quoted 9 lines · Damien Le Moal
Quoted 7 lines · John David Anglin

I think the issue is with the root ST373207LW drive.  The console output indicates that the ROOT drive doesn't exist when the boot fails.

Your change only appeared to affect actual SCSI drives.  That's why I tried disabling CDL.

Quoted 2 lines

Could you send a full dmesg output for a clean boot and for a failed one so that I can compare ?

I'll try to get this together tomorrow.

Please try the attached patch. That should address the issue with your drive.

Mainline and v6.5.3 both booted successfully with the attached patch.

Great ! Thanks for testing. Posting the patch.

Quoted 4 lines

Thanks,
Dave

Signature · 2 lines
Damien Le Moal
Western Digital Research
#22 lore raw
In reply to Masahiro Yamada
Quoted 2 lines · Masahiro Yamada

I checked the assembler output, and noticed function references are prefixed with P%, so the situation in parisc64 is similar to ia64.

Function references are prefixed with P% when they occur in data like the following:

    .dword    P%func

This causes the generation of a function descriptor.   The assembler and linker can't handle the P% prefix in other situations.

Dave

Signature · 1 line
John David Anglin  [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#24 lore raw
Quoted 2 lines · Helge Deller

I think ppc64 is affected too.

I tested ppc64 ABI v1, but did not see a breakage.

Quoted 47 lines

Search for dereference_function_descriptor() in kernel sources, e.g.
https://www.mail-archive.com/[email protected]/msg1494564.html
Helge

-------- Ursprüngliche Nachricht --------
Von: Masahiro Yamada <[email protected]>
Datum: 05.09.23 21:08 (GMT+01:00)
An: [email protected], Helge Deller <[email protected]>, John David Anglin <[email protected]>
Cc: [email protected], [email protected], Masahiro Yamada <[email protected]>, Nick Desaulniers <[email protected]>
Betreff: [PATCH] linux/export: fix reference to exported functions for parisc64

John David Anglin reported parisc has been broken since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost").

I checked the assembler output, and noticed function references are prefixed with P%, so the situation in parisc64 is similar to ia64.

Fixes ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by modpost")

---

I just checked the assembler output, and I created this patch based on my best guess. Only compile-tested.
I hope somebody will run-test this patch.

include/linux/export-internal.h | 2 ++

1 file changed, 2 insertions(+)
include/linux/export-internal.h +2 -1
index 1c849db953a5..45fca09b2319 100644
@@ -52,6 +52,8 @@
 
 #ifdef CONFIG_IA64
 #define KSYM_FUNC(name) @fptr(name)
+#elif defined(CONFIG_PARISC) && defined(CONFIG_64BIT)
+#define KSYM_FUNC(name) P%name
 #else
 #define KSYM_FUNC(name) name
 #endif
--

2.39.2

Signature · 2 lines
Best Regards
Masahiro Yamada