PCI: Do not enable pci atomics on VFs
Subsystems (2)
PATCH 1/1 PCI: Do not enable pci atomics on VFs
Host crashes when pci_enable_atomic_ops_to_root is called for VFs with virtual buses. The virtual buses added to SR-IOV has bus->self set to NULL and host crashes due to this.
PID: 4481 TASK: ffff89c6941b0000 CPU: 53 COMMAND: "bash"
#0 [ffff9a94817136d8] machine_kexec at ffffffffb90601a4 #1 [ffff9a9481713728] __crash_kexec at ffffffffb9190d5d #2 [ffff9a94817137f0] crash_kexec at ffffffffb9191c4d #3 [ffff9a9481713808] oops_end at ffffffffb9025cd6 #4 [ffff9a9481713828] page_fault_oops at ffffffffb906e417 #5 [ffff9a9481713888] exc_page_fault at ffffffffb9a0ad14 #6 [ffff9a94817138b0] asm_exc_page_fault at ffffffffb9c00ace
[exception RIP: pcie_capability_read_dword+28]
RIP: ffffffffb952fd5c RSP: ffff9a9481713960 RFLAGS: 00010246
RAX: 0000000000000001 RBX: ffff89c6b1096000 RCX: 0000000000000000
RDX: ffff9a9481713990 RSI: 0000000000000024 RDI: 0000000000000000
RBP: 0000000000000080 R8: 0000000000000008 R9: ffff89c64341a2f8
R10: 0000000000000002 R11: 0000000000000000 R12: ffff89c648bab000
R13: 0000000000000000 R14: 0000000000000000 R15: ffff89c648bab0c8
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6
#8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at ffffffffc08c1a33 [bnxt_re]
#9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re]
RIP: 00007f450602f648 RSP: 00007ffe880869e8 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f450602f648
RDX: 0000000000000002 RSI: 0000555c566c4a60 RDI: 0000000000000001
RBP: 0000555c566c4a60 R8: 000000000000000a R9: 00007f45060c2580
R10: 000000000000000a R11: 0000000000000246 R12: 00007f45063026e0
R13: 0000000000000002 R14: 00007f45062fd880 R15: 0000000000000002
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002bAtomicOp Requester Enable bit in the Device Control 2 register is reserved for VFs and drivers shouldn't enable it for VFs. Adding a check to return EINVAL if pci_enable_atomic_ops_to_root is called with VF pci device.
--- drivers/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/pci/pci.c +8 -0
@@ -3702,6 +3702,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)struct pci_dev *bridge;u32 cap, ctl2; +/* +* As per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable +* bit in the Device Control 2 register is reserved in VFs and the PF +* value applies to all associated VFs. Return -EINVAL if called for VFs. +*/ +if (dev->is_virtfn) +return -EINVAL ; +if (!pci_is_pcie (dev))return -EINVAL ;
Signature · 1 line
2.5.5
Quoted 3 lines · Selvin Xavier
Host crashes when
pci_enable_atomic_ops_to_rootis called for VFs with virtual buses. The virtual buses added to SR-IOV hasbus->selfset to NULL and host crashes due to this.
^^ I _hate_ to say this, but the extra space isn't ideal. Not
sure if the maintainers will want to hold-up the submission for this (or if they can fix while committing), but something to look out for next time.
Quoted 28 lines
PID: 4481 TASK:
ffff89c6941b0000CPU: 53 COMMAND: "bash"
#0 [ffff9a94817136d8]machine_kexecatffffffffb90601a4#1 [ffff9a9481713728]__crash_kexecatffffffffb9190d5d#2 [ffff9a94817137f0]crash_kexecatffffffffb9191c4d#3 [ffff9a9481713808]oops_endatffffffffb9025cd6#4 [ffff9a9481713828]page_fault_oopsatffffffffb906e417#5 [ffff9a9481713888]exc_page_faultatffffffffb9a0ad14#6 [ffff9a94817138b0]asm_exc_page_faultatffffffffb9c00ace[exception RIP: pcie_capability_read_dword+28] RIP: ffffffffb952fd5c RSP: ffff9a9481713960 RFLAGS: 00010246 RAX: 0000000000000001 RBX: ffff89c6b1096000 RCX: 0000000000000000 RDX: ffff9a9481713990 RSI: 0000000000000024 RDI: 0000000000000000 RBP: 0000000000000080 R8: 0000000000000008 R9: ffff89c64341a2f8 R10: 0000000000000002 R11: 0000000000000000 R12: ffff89c648bab000 R13: 0000000000000000 R14: 0000000000000000 R15: ffff89c648bab0c8 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6 #8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at ffffffffc08c1a33 [bnxt_re] #9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re] RIP: 00007f450602f648 RSP: 00007ffe880869e8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f450602f648 RDX: 0000000000000002 RSI: 0000555c566c4a60 RDI: 0000000000000001 RBP: 0000555c566c4a60 R8: 000000000000000a R9: 00007f45060c2580 R10: 000000000000000a R11: 0000000000000246 R12: 00007f45063026e0 R13: 0000000000000002 R14: 00007f45062fd880 R15: 0000000000000002 ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
Quoted 8 lines
AtomicOp Requester Enable bit in the Device Control 2 register is reserved for VFs and drivers shouldn't enable it for VFs. Adding a check to return EINVAL if
pci_enable_atomic_ops_to_rootis called with VF pci device.Fixes 35f5ace5dea4 ("RDMA/bnxt_re: Enable global atomic ops if platform supports")Fixes 430a23689dea ("PCI: Add pci_enable_atomic_ops_to_root()")Signed-off-by Selvin Xavier <[email protected]>
Thanks for this, Selvin. Technically this looks like a good fix.
Quoted 26 lines
--- drivers/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/pci/pci.c +8
-0@@ -3702,6 +3702,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)struct pci_dev *bridge;u32 cap, ctl2; +/* +* As per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable +* bit in the Device Control 2 register is reserved in VFs and the PF +* value applies to all associated VFs. Return -EINVAL if called for VFs. +*/ +if (dev->is_virtfn) +return -EINVAL ; +if (!pci_is_pcie (dev))return -EINVAL ;Signature · 1 line
2.5.5
Bjorn Helgaas
AppliedQuoted 31 lines · Selvin Xavier
Host crashes when
pci_enable_atomic_ops_to_rootis called for VFs with virtual buses. The virtual buses added to SR-IOV hasbus->selfset to NULL and host crashes due to this.PID: 4481 TASK:
ffff89c6941b0000CPU: 53 COMMAND: "bash"
#0 [ffff9a94817136d8]machine_kexecatffffffffb90601a4#1 [ffff9a9481713728]__crash_kexecatffffffffb9190d5d#2 [ffff9a94817137f0]crash_kexecatffffffffb9191c4d#3 [ffff9a9481713808]oops_endatffffffffb9025cd6#4 [ffff9a9481713828]page_fault_oopsatffffffffb906e417#5 [ffff9a9481713888]exc_page_faultatffffffffb9a0ad14#6 [ffff9a94817138b0]asm_exc_page_faultatffffffffb9c00ace[exception RIP: pcie_capability_read_dword+28] RIP: ffffffffb952fd5c RSP: ffff9a9481713960 RFLAGS: 00010246 RAX: 0000000000000001 RBX: ffff89c6b1096000 RCX: 0000000000000000 RDX: ffff9a9481713990 RSI: 0000000000000024 RDI: 0000000000000000 RBP: 0000000000000080 R8: 0000000000000008 R9: ffff89c64341a2f8 R10: 0000000000000002 R11: 0000000000000000 R12: ffff89c648bab000 R13: 0000000000000000 R14: 0000000000000000 R15: ffff89c648bab0c8 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #7 [ffff9a9481713988] pci_enable_atomic_ops_to_root at ffffffffb95359a6 #8 [ffff9a94817139c0] bnxt_qplib_determine_atomics at ffffffffc08c1a33 [bnxt_re] #9 [ffff9a94817139d0] bnxt_re_dev_init at ffffffffc08ba2d1 [bnxt_re] RIP: 00007f450602f648 RSP: 00007ffe880869e8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f450602f648 RDX: 0000000000000002 RSI: 0000555c566c4a60 RDI: 0000000000000001 RBP: 0000555c566c4a60 R8: 000000000000000a R9: 00007f45060c2580 R10: 000000000000000a R11: 0000000000000246 R12: 00007f45063026e0 R13: 0000000000000002 R14: 00007f45062fd880 R15: 0000000000000002 ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
Quoted 8 lines
AtomicOp Requester Enable bit in the Device Control 2 register is reserved for VFs and drivers shouldn't enable it for VFs. Adding a check to return EINVAL if
pci_enable_atomic_ops_to_rootis called with VF pci device.Fixes 35f5ace5dea4 ("RDMA/bnxt_re: Enable global atomic ops if platform supports")Fixes 430a23689dea ("PCI: Add pci_enable_atomic_ops_to_root()")Signed-off-by Selvin Xavier <[email protected]>
Applied to pci/enumeration for v5.16, thanks!
I fixed the extra space and rewrapped the code comment so it fits in 80 columns.
Quoted 26 lines
--- drivers/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/pci/pci.c +8
-0@@ -3702,6 +3702,14 @@ int pci_enable_atomic_ops_to_root(struct pci_dev *dev, u32 cap_mask)struct pci_dev *bridge;u32 cap, ctl2; +/* +* As per PCIe r5.0, sec 9.3.5.10, the AtomicOp Requester Enable +* bit in the Device Control 2 register is reserved in VFs and the PF +* value applies to all associated VFs. Return -EINVAL if called for VFs. +*/ +if (dev->is_virtfn) +return -EINVAL ; +if (!pci_is_pcie (dev))return -EINVAL ;Signature · 1 line
2.5.5