New: Dell PowerEdge R730 resets at boot since v6.15: failed disabled ROM BARs trigger PCI realloc, I350 moved

[+cc Ilpo, Mika, Mario, reporter in bcc]

---------- Forwarded message ---------
From: <[email protected]>
Date: Mon, Nov 9, 2026 at 6:12 PM
Subject: [Bug 222253] New: Dell PowerEdge R730 resets at boot since v6.15: failed disabled ROM BARs trigger PCI realloc, I350 moved
To: <[email protected]>

https://bugzilla.kernel.org/show_bug.cgi?id=222253

            Bug ID: 222253
           Summary: Dell PowerEdge R730 resets at boot since v6.15: failed
                    disabled ROM BARs trigger PCI realloc, I350 moved
           Product: Drivers
           Version: 2.5
    Kernel Version: 6.15 - 7.4-rc2
          Hardware: Intel
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: PCI
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: Yes
           Bisected 96336ec70264
         commit-id:

Since v6.15, a Dell PowerEdge R730 with "SR-IOV Global Enable" disabled in the BIOS setup hard-resets during boot when igb probes the on-board Intel I350 (8086:1521, behind PCH root port 00:1c.4). v6.14 boots fine.

iDRAC Lifecycle log for a failed boot:

  PWR2262  The Intel Management Engine has reported an internal system error.
  CPU0000  Internal error has occurred check for additional logs.
  RAC0703  Requested system hardreset.
  CPU0704  CPU 1 machine check error detected.

Normally the platform then power-cycles itself and boots the next OS; in 2 of the ~17 crashes during testing the BIOS halted in POST instead (PST0208 "System BIOS has halted") and needed a power cycle.

The same signature (PWR2262 / IERR on kernels >= 6.17, on R630/R730/XC730xd with I350) is reported by Proxmox users:
https://forum.proxmox.com/threads/186063/
https://forum.proxmox.com/threads/177201/
There, enabling X2APIC Mode, I/OAT DMA and SR-IOV Global in the BIOS is the usual workaround. On this machine only SR-IOV Global Enable matters (see below).

Hardware / firmware:
Dell PowerEdge R730, 2 sockets, BIOS 2.19.0 (also reproduced on 2.11.0), iDRAC8 2.75. On-board: 82599 (8086:10fb) on CPU root port 00:03.0, PERC H730P Mini (1000:005d) on 00:01.0, I350 (8086:1521) on PCH root port 00:1c.4. BIOS: X2APIC, I/OAT and SR-IOV Global disabled unless stated.

Bisection (v6.14 good, v6.17 bad; test kernels booted from RAM via PXE, a BAD boot detected through the BMC's SEL):
first bad commit: 96336ec70264 ("PCI: Perform reset_resource() and build fail list in sync")
96336ec70264 does not build on its own (pci_resource_is_disabled_rom() is not defined anywhere); it was tested with a one-line definition matching the check it replaced. Its parent e89df6d2beae boots.

Mechanism (from serial console logs with PCI dynamic debug):
- The firmware leaves the 82599 VF BARs unassigned, so pci_realloc_detect() enables reallocation.
- On the first pass the disabled ROM BARs of 02:00.0, 01:00.1 and 07:00.1 fail ("can't assign; no space").
- Before 96336ec70264 a failed disabled ROM BAR was not added to fail_head. 96336ec70264 tests that with an idx that is not set for the resource being checked, and 2499f5348431 ("PCI: Rework optional resource handling") removed the test, so the ROM BARs land on fail_head.
- "PCI: No. 2 try to assign unassigned res" follows: the windows of

    00:1c.4, 00:03.0 and 00:01.0 are released and reassigned; the I350 moves
    from 0x91a80000 (firmware) to 0x92200000.
  - The last console line before the reset is igb saving the config space
    of 07:00.0 with BAR0 = 0x92200000.

The new address range is free in /proc/iomem; why the platform resets is not known.

BIOS switch matrix with stock v6.17:
all three disabled -> resets
SR-IOV Global only -> boots (no realloc, no 2nd round)
X2APIC + I/OAT, SR-IOV Global disabled -> resets
any switches, pci=realloc=off -> boots (tested with all disabled)

Current mainline v7.4-rc2 still resets. A patch restoring the old behavior (do not put a failed disabled ROM BAR on fail_head) makes both v6.17 and v7.4-rc2 boot; it follows on linux-pci.

Attachments:
sol_v6.17-realloc-off.log serial console, GOOD (pci=realloc=off), with
/proc/iomem
sol_96336-bad-dbg.log serial console, BAD, first bad commit, PCI
dyndbg

  sol_v7.4-rc2-dbg.log            serial console, BAD, mainline
  sol_v7.4-rc2+fix1.log           serial console, GOOD, mainline + patch
  sol_v6.17+fix1.log              serial console, GOOD, v6.17 + patch
  sol_v6.17-stock-sriov-only.log  serial console, GOOD, SR-IOV Global only
  git_bisect_log.txt              git bisect log
  lclog-excerpts.txt              iDRAC Lifecycle log entries for a failed boot
  harness.tar.gz                  test scripts (addresses replaced by

placeholders), kernel .config, initramfs
lspci-vvv.txt from the working v6.8 kernel (BIOS switches
enabled; with
SR-IOV Global enabled the BIOS renumbers
buses, so the

                                  PERC is 03:00.0 and the I350 08:00.x there)

Notes: The analysis and patch were prepared with help from an AI coding assistant (Claude, by Anthropic); all results above come from test boots on the machine described.

--

#regzbot introduced: 96336ec70264
#regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=222253

Thanks, Bjorn.

For the new Cc's: the mechanism in this report is out of date. The ROM BAR change was intentional, and I've dropped my patch. After Ilpo's review I captured the end state. The assignment itself is valid, but the second pass puts 00:1c.4's window, and the I350's BAR 0, at 0x92200000. That 4K is still decoded by the PCH thermal function (D31:F6), which the BIOS hides on this C610/Wellsburg platform. The first MMIO read of the I350 there resets the machine. Reserving the range with memmap=4K$0x92200000 makes stock v6.17 boot normally.

Details:
https://lore.kernel.org/all/CACpBZmUoSU8Emvego8sEce5AbB4+48yMLbs=E-5HKMh=65Zq=A@mail.gmail.com/

The new serial logs are attached to the bugzilla entry.

So this looks like quirk territory: reserve the hidden thermal BAR before resources are assigned. Its address follows the firmware layout (0x92600000 with SR-IOV Global enabled), so the quirk has to find it rather than use a fixed range. Ilpo, Mika: is there a supported way to read the BAR of the hidden D31:F6 on C610/X99, similar to what p2sb.c does for P2SB?

Kameron

The investigation and these tests were done with the help of an AI
coding assistant (Claude, by Anthropic); I ran them on the hardware and
checked the results.

Kameron Markham
Mechanical Engineer
(253)347-3004

"The most difficult subjects can be explained to the most slow-witted
man if he has not formed any idea of them already; but the simplest
thing cannot be made clear to the most intelligent man if he is firmly
persuaded that he knows already, without a shadow of doubt, what is
laid before him" -Leo Tolstoy