PCI fixes for v3.19

Hi Linus,

These are fixes for:

- A resource management problem that causes a Radeon "Fatal error during GPU init" on machines where the BIOS programmed an invalid Root Port window. This was a regression in v3.16.

- An Atheros AR93xx device that doesn't handle PCI bus resets correctly. This was a regression in v3.14.

- An out-of-date email address.

Bjorn

The following changes since commit 97bf6af1f928216fd6c5a66e8a57bfa95a659672:

Linux 3.19-rc1 (2014-12-20 17:08:50 -0800)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git tags/pci-v3.19-fixes-1

for you to fetch changes up to f175aa2c9f6cc08f043e85ea37f44ef3676cbac1:

MAINTAINERS: Update Richard Zhu's email address (2015-01-22 13:40:59 -0600)

----------------------------------------------------------------
PCI updates for v3.19:

Resource management
- Clip bridge windows to fit in upstream windows (Yinghai Lu)

Virtualization
- Mark Atheros AR93xx to avoid using bus reset (Alex Williamson)

Miscellaneous
- Update Richard Zhu's email address (Lucas Stach)

----------------------------------------------------------------
Alex Williamson (2):
PCI: Add flag for devices where we can't use bus reset
PCI: Mark Atheros AR93xx to avoid bus reset

Lucas Stach (1):
MAINTAINERS: Update Richard Zhu's email address

Yinghai Lu (12):
PCI: Pass bridge device, not bus, when updating bridge windows
PCI: Add pci_bus_clip_resource() to clip to fit upstream window
PCI: Add pci_claim_bridge_resource() to clip window if necessary x86/PCI: Clip bridge windows to fit in upstream windows alpha/PCI: Clip bridge windows to fit in upstream windows frv/PCI: Clip bridge windows to fit in upstream windows ia64/PCI: Clip bridge windows to fit in upstream windows microblaze/PCI: Clip bridge windows to fit in upstream windows mn10300/PCI: Clip bridge windows to fit in upstream windows parisc/PCI: Clip bridge windows to fit in upstream windows powerpc/PCI: Clip bridge windows to fit in upstream windows sparc/PCI: Clip bridge windows to fit in upstream windows

 MAINTAINERS                             |  2 +-
 arch/alpha/kernel/pci.c                 |  8 +++--
 arch/frv/mb93090-mb00/pci-frv.c         |  2 +-
 arch/ia64/pci/pci.c                     | 48 +++++++++++++---------------
 arch/microblaze/pci/pci-common.c        | 13 +++++++-
 arch/mn10300/unit-asb2305/pci-asb2305.c |  2 +-
 arch/mn10300/unit-asb2305/pci.c         | 47 +++++++++++++--------------
 arch/powerpc/kernel/pci-common.c        | 12 ++++++-
 arch/sparc/kernel/pci.c                 |  5 ++-
 arch/x86/pci/i386.c                     |  2 +-
 drivers/parisc/lba_pci.c                |  5 ++-
 drivers/pci/bus.c                       | 43 +++++++++++++++++++++++++
 drivers/pci/pci.c                       | 40 ++++++++++++++++++++---
 drivers/pci/pci.h                       |  1 +
 drivers/pci/quirks.c                    | 14 +++++++++
 drivers/pci/setup-bus.c                 | 56 ++++++++++++++++++++++++++-------
 include/linux/pci.h                     |  3 ++
 17 files changed, 222 insertions(+), 81 deletions(-)

I'm seeing these new messages in v3.19-rc6 on ia64:

pci 0000:01:00.0: can't claim BAR 6 [mem 0xfffe0000-0xffffffff pref]:
no compatible bridge window
pci 0000:01:00.1: can't claim BAR 6 [mem 0xfffe0000-0xffffffff pref]:
no compatible bridge window
pci 0000:03:00.0: can't claim BAR 6 [mem 0xffe00000-0xffffffff pref]:
no compatible bridge window
pci 0000:0b:04.0: can't claim BAR 6 [mem 0xfffe0000-0xffffffff pref]:
no compatible bridge window
pci 0000:80:13.0: can't claim BAR 0 [mem 0xa0220000-0xa0220fff]: no compatible bridge window
pci 0000:80:16.0: can't claim BAR 0 [mem 0xa0200000-0xa0203fff 64bit]:
no compatible bridge window
pci 0000:80:16.1: can't claim BAR 0 [mem 0xa0204000-0xa0207fff 64bit]:
no compatible bridge window
pci 0000:80:16.2: can't claim BAR 0 [mem 0xa0208000-0xa020bfff 64bit]:
no compatible bridge window
pci 0000:80:16.3: can't claim BAR 0 [mem 0xa020c000-0xa020ffff 64bit]:
no compatible bridge window
pci 0000:80:16.4: can't claim BAR 0 [mem 0xa0210000-0xa0213fff 64bit]:
no compatible bridge window
pci 0000:80:16.5: can't claim BAR 0 [mem 0xa0214000-0xa0217fff 64bit]:
no compatible bridge window
pci 0000:80:16.6: can't claim BAR 0 [mem 0xa0218000-0xa021bfff 64bit]:
no compatible bridge window
pci 0000:80:16.7: can't claim BAR 0 [mem 0xa021c000-0xa021ffff 64bit]:
no compatible bridge window
pci 0000:80:01.0: can't claim BAR 8 [mem 0xa0100000-0xa01fffff]: no compatible bridge window
pci 0000:81:00.0: can't claim BAR 6 [mem 0xfffe0000-0xffffffff pref]:
no compatible bridge window
pci 0000:81:00.1: can't claim BAR 6 [mem 0xfffe0000-0xffffffff pref]:
no compatible bridge window
pci 0000:80:05.0: can't claim BAR 8 [mem 0xa0000000-0xa00fffff]: no compatible bridge window
pci 0000:83:00.0: can't claim BAR 6 [mem 0xffe00000-0xffffffff pref]:
no compatible bridge window

Bisection pointed to:

and reverting commit ce821ef0333fc1 makes them go away again.

How do I tell whether I really have a bunch of badly configured busses that we are just
now able to detect, or there is something spurious in this patch? Contents of /proc/iomem (from the kernel with the reverted patch) attached.

-Tony

00000000-00000fff : System RAM 00001000-0005ffff : System RAM 00060000-00087fff : System RAM 00088000-00096fff : System RAM 00097000-00097fff : System RAM 00098000-0009ffff : System RAM 000c0000-000fffff : reserved 00100000-003fffff : System RAM 00400000-00471fff : System RAM 00472000-00bfffff : System RAM 00c00000-00ffffff : System RAM 01000000-03ffffff : System RAM 04000000-05e6afff : System RAM
04000000-04c3873f : Kernel code 04c38740-05676f87 : Kernel data 05676f88-05e6afcf : Kernel bss
05e6b000-3a247fff : System RAM 3a248000-3a249fff : reserved 3a24a000-3a26dfff : reserved 3a26e000-3a4e9fff : System RAM 3a4ea000-3a4ebfff : ACPI Non-volatile Storage 3a4ec000-3a4effff : System RAM 3a4f0000-3a537fff : reserved 3a538000-3a59dfff : System RAM 3a59e000-3a5a1fff : System RAM 3a5a2000-3a5ebfff : reserved 3a5ec000-3a5f3fff : reserved 3a5f4000-3a6bdfff : reserved 3a6be000-3a6befff : System RAM 3a6bf000-3a6ebfff : System RAM 3a6ec000-3a73bfff : reserved 3a73c000-3a743fff : System RAM 3a744000-3a745fff : reserved 3a746000-3a747fff : System RAM 3a748000-3a749fff : reserved 3a74a000-3a74bfff : System RAM 3a74c000-3a74ffff : reserved 3a750000-3a759fff : reserved 3a75a000-3a75bfff : reserved 3a75c000-3a773fff : reserved 3a774000-3a774fff : System RAM 3a775000-3a845fff : System RAM 3a846000-3a847fff : System RAM 3a848000-3a913fff : System RAM 3a914000-3a918fff : System RAM 3a919000-3ab41fff : System RAM 3ab42000-3ab4afff : System RAM 3ab4b000-3ac4bfff : System RAM 3ac4c000-3ac57fff : reserved 3ac58000-3ac79fff : reserved 3ac7a000-3aca9fff : reserved 3acaa000-3ad0dfff : System RAM 3ad0e000-3ad59fff : System RAM 3ad5a000-3ad6dfff : reserved 3ad6e000-3ad99fff : System RAM 3ad9a000-3ad9bfff : reserved 3ad9c000-3ada9fff : reserved 3adaa000-3adaafff : System RAM 3adab000-3add1fff : System RAM 3add2000-3ade5fff : reserved 3ade6000-3adfbfff : System RAM 3adfc000-3ae1bfff : reserved 3ae1c000-3ae1ffff : System RAM 3ae20000-3ae3dfff : reserved 3ae3e000-3ae40fff : System RAM 3ae41000-3ae5bfff : System RAM 3ae5c000-3ae5dfff : ACPI Non-volatile Storage 3ae5e000-3ae81fff : System RAM 3ae82000-3ae9bfff : reserved 3ae9c000-3aeb7fff : reserved 3aeb8000-3aeb8fff : System RAM 3aeb9000-3aebffff : System RAM 3aec0000-3aff5fff : reserved 3aff6000-3b001fff : reserved 3b002000-3b003fff : reserved 3b004000-3b005fff : System RAM 3b006000-3b017fff : reserved 3b018000-3b019fff : System RAM 3b01a000-3b21bfff : reserved 3b21c000-3b265fff : reserved 3b266000-3b271fff : System RAM 3b272000-3b27ffff : reserved 3b280000-3b2dffff : System RAM 3b2e0000-3b2e1fff : reserved 3b2e2000-3b3a3fff : System RAM 3b3a4000-3b3b5fff : reserved 3b3b6000-3b3c7fff : System RAM 3b3c8000-3b3d1fff : System RAM 3b3d2000-3b935fff : System RAM 3b936000-3b95ffff : reserved 3b960000-3b964fff : System RAM 3b965000-3b989fff : System RAM 3b98a000-3b991fff : reserved 3b992000-3b9a7fff : System RAM 3b9a8000-3bbaffff : reserved 3bbb0000-3bbfffff : reserved 3bc00000-3befffff : reserved 3bf00000-3c0e5fff : reserved 3c0e6000-3c0e6fff : System RAM 3c0e7000-3c16dfff : System RAM 3c16e000-3c175fff : reserved 3c176000-3c217fff : System RAM 3c218000-3c22dfff : reserved 3c22e000-3c62dfff : System RAM 3c62e000-3c6a9fff : System RAM 3c6aa000-3c7dffff : System RAM 3c7e0000-3c7e3fff : reserved 3c7e4000-3c859fff : System RAM 3c85a000-3c888fff : System RAM 3c889000-3c895fff : System RAM 3c896000-3c8a1fff : reserved 3c8a2000-3c8a2fff : System RAM 3c8a3000-3c8b5fff : System RAM 3c8b6000-3c8c7fff : System RAM 3c8c8000-3c8c9fff : System RAM 3c8ca000-3c8d9fff : reserved 3c8da000-3c8f0fff : System RAM 3c8f1000-3c8f7fff : System RAM 3c8f8000-3ca4ffff : System RAM 3ca50000-3ca53fff : reserved 3ca54000-3ca59fff : System RAM 3ca5a000-3cab1fff : System RAM 3cab2000-3cab3fff : reserved 3cab4000-3cc85fff : System RAM 3cc86000-3cc87fff : reserved 3cc88000-3cdd9fff : System RAM 3cdda000-3cddbfff : reserved 3cddc000-3ce3bfff : System RAM 3ce3c000-3ce3dfff : reserved 3ce3e000-3d86bfff : System RAM 3d86c000-3d873fff : System RAM 3d874000-3d8abfff : System RAM 3d8ac000-3d8b5fff : System RAM 3d8b6000-3d8d7fff : System RAM 3d8d8000-3d8ddfff : System RAM 3d8de000-3d9effff : System RAM 3d9f0000-3d9f1fff : reserved 3d9f2000-3da41fff : System RAM 3da42000-3da43fff : reserved 3da44000-3dbf6fff : System RAM 3dbf7000-3dbfdfff : System RAM 3dbfe000-3e80dfff : System RAM 3e80e000-3e80ffff : reserved 3e810000-3e81dfff : System RAM 3e81e000-3e81ffff : reserved 3e820000-3e845fff : System RAM 3e846000-3e8bbfff : System RAM 3e8bc000-3f8d6fff : System RAM 3f8d7000-3fcd6fff : reserved 3fcd7000-3fce0fff : System RAM 3fce1000-3fdcefff : reserved 3fdcf000-3fdf2fff : System RAM 3fdf3000-3fffffff : reserved 50000000-9fffffff : PCI Bus 0000:00
50000000-580fffff : PCI Bus 0000:0b

    50000000-57ffffff : 0000:0b:04.0
    58000000-5800ffff : 0000:0b:04.0
  58100000-581fffff : PCI Bus 0000:03
    58100000-5810ffff : 0000:03:00.0
      58100000-5810ffff : mpt
    58110000-58113fff : 0000:03:00.0
      58110000-58113fff : mpt
  58200000-582fffff : PCI Bus 0000:01
    58200000-5821ffff : 0000:01:00.1
      58200000-5821ffff : igb
    58220000-5823ffff : 0000:01:00.1
      58220000-5823ffff : igb
    58240000-5825ffff : 0000:01:00.0
      58240000-5825ffff : igb
    58260000-5827ffff : 0000:01:00.0
      58260000-5827ffff : igb
    58280000-58283fff : 0000:01:00.1
      58280000-58283fff : igb
    58284000-58287fff : 0000:01:00.0
      58284000-58287fff : igb
  58300000-58303fff : 0000:00:16.0
  58304000-58307fff : 0000:00:16.1
  58308000-5830bfff : 0000:00:16.2
  5830c000-5830ffff : 0000:00:16.3
  58310000-58313fff : 0000:00:16.4
  58314000-58317fff : 0000:00:16.5
  58318000-5831bfff : 0000:00:16.6
  5831c000-5831ffff : 0000:00:16.7
  58320000-58320fff : 0000:00:13.0
  58321000-583213ff : 0000:00:1d.7
    58321000-583213ff : ehci_hcd
  58321400-583217ff : 0000:00:1a.7
    58321400-583217ff : ehci_hcd

58321800-583218ff : 0000:00:1f.3
a0000000-a000ffff : mpt
a0010000-a0013fff : mpt
a0100000-a011ffff : igb
a0120000-a013ffff : igb
a0140000-a015ffff : igb
a0160000-a017ffff : igb
a0180000-a0183fff : igb
a0184000-a0187fff : igb
fea00000-fea0001f : pnp 00:01
fec00000-fecfffff : PNP0003:00
fec40000-fec7ffff : PNP0003:01
fed1b000-fed1bfff : pnp 00:01
fed1c000-fed8bffe : pnp 00:01
fee00000-feefffff : pnp 00:01
ff000000-ffffffff : pnp 00:01
100000000-3ffffffff : System RAM
440000000-4faad5fff : System RAM
4faad6000-4fafa5fff : System RAM
4fafa6000-4fb00bfff : reserved
4fb00c000-4fb05dfff : System RAM
4fb05e000-4fb0b9fff : System RAM
4fb0ba000-4fb0d1fff : System RAM
4fb0d2000-4fb0dbfff : System RAM
4fb0d2018-4fb0d4118 : EFI Memory Map
4fb0d6018-4fb0d6068 : Boot parameter
4fb0dc000-4fb0e7fff : System RAM
4fb0e8000-4fb0e9fff : reserved
4fb0ea000-4fb0effff : System RAM
4fb0f0000-4fb113fff : System RAM
4fb114000-4fb115fff : System RAM
4fb116000-4fb153fff : System RAM
4fb154000-4fb155fff : System RAM
4fb156000-4fbffffff : System RAM
4fc000000-4ffffffff : reserved
1fffffc000000-1fffffc33dcf7 : PCI Bus 0000:00 I/O Ports 00000000-00000cf7 1fffffc400000-1fffffe3fffff : PCI Bus 0000:00 I/O Ports 00001000-00008fff 1fffffe400000-1fffffffffffe : PCI Bus 0000:80 I/O Ports 00009000-0000fffe

Hi Tony,

Sorry for the inconvenience. Can you collect a complete dmesg log and "lspci -vv" output, too (from the kernel with the reverted commit)? That will have more useful information than just /proc/iomem.

These are option ROMs, probably just not assigned. We should be able to assign space for them, so this might not even be a problem, but we certainly need a better message if that's all it is.

Interesting. There's no mention of bus 80 at all in /proc/iomem; maybe this is found by blind probing? Do we even do that on ia64? Dmesg should have a clue.

Looks like a bridge window (assuming you have CONFIG_PCI_IOV unset). There's definitely something wrong -- all those mpt and igb devices at the top level *should* be under some PCI hierarchy, but the hierarchy is missing from iomem.

Bjorn

Full dmesg, lspci -vv, and bonus .config (CONFIG_PCI_IOV is indeed not set)

-Tony

The ROM part is something we should fix:

pci 0000:01:00.1: can't claim BAR 6 [mem 0xfffe0000-0xffffffff pref]: no compatible bridge window

The ROM BAR is probably disabled, and we shouldn't complain about this if it's disabled. Yinghai?

The stuff on bus 0000:80 looks like at least partly a firmware problem:

ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 80-ff]) acpi PNP0A08:01: host bridge window [io 0x9000-0xfffe] pci 0000:80:01.0: PCI bridge to [bus 81]
pci 0000:80:01.0: bridge window [io 0xa000-0xafff]
pci 0000:80:01.0: bridge window [mem 0xa0100000-0xa01fffff]
pci 0000:80:13.0: reg 0x10: [mem 0xa0220000-0xa0220fff]

ACPI told us about an I/O port aperture, but didn't mention any MMIO apertures through the host bridge. Therefore, we have no MMIO space to assign to the devices on [bus 80-ff].

But presumably these devices, e.g., the igb devices at 81:00.0 and 81:00.1 and the mpt device at 83:00.0, actually DO work, so the PCI1 host bridge must actually forward some MMIO space that ACPI didn't tell us about. I think on x86, we would fall back to whatever the firmware left in those BARs. We should probably do the same on x86.

Bjorn

original ia64 code, has extra checking to hide the warning

-static int is_valid_resource(struct pci_dev *dev, int idx)
{
- unsigned int i, type_mask = IORESOURCE_IO | IORESOURCE_MEM;
- struct resource *devr = &dev->resource[idx], *busr;

        if (!dev->bus)

- return 0;
-
- pci_bus_for_each_resource(dev->bus, busr, i) {
- if (!busr || ((busr->flags ^ devr->flags) & type_mask))
- continue;
- if ((devr->start) && (devr->start >= busr->start) &&
- (devr->end <= busr->end))
- return 1;
- }
- return 0;
-}

also that is used to skip pci_claim_resource calling for invalid ...

so the root resource is totally wrong from ACPI.

Thanks

Yinghai

Yes, ACPI is clearly missing some information.

Just to make sure we're planning the same thing, I expect that we will fix or workaround both issues so it works just as well as v3.18 by the time v3.19 is released. This is a system in the field, and we should be able to figure out how to deal with the broken firmware. Asking users to upgrade their firmware is not the solution I'm looking for.

There is nothing ia64-specific about this issue, so we should be able to do this in some generic way.

Bjorn

Good to hear. BIOS in this machine is dated May, 2010 ... I doubt I could find anyone who still remembers how to build a BIOS for it, let alone has any motivation to actually do it.

-Tony

Sorry for the problem.

Can you please get boot bog with "debug ignore_logleve"?

we should get print out from

+ dev_printk(KERN_DEBUG, &dev->dev, "%pR clipped to %pR\n",
+ &orig_res, res);

Thanks

Yinghai

Good. so the system should just work as before but have annoying warnings.

Do we need to put the paper back to hide the warning?

Thanks

Yinghai

Ping, I'm not sure this is resolved.

Tony, does the system work as it did before? Is the only problem that now we have more warnings than we did before?

Yinghai, I sort of feel like I'm being left to sweep up behind your changes here. I *could* analyze this and figure out what's going on and fix it, but I don't have time to do that for everybody, and I consider that more your job.

If we start with the same _CRS config and same device config, ideally PCI enumeration would produce the same messages, same warnings, and same resource assignments no matter what arch we feed them to, because enumeration is not really arch-specific. So if ia64 does something different here, I think something needs to be fixed.

Bjorn

Yup - things seem to be working. Just have a bunch of new console messages.

It seems that at least some of them are valid - the BIOS isn't describing some things that it should. I don't think we need to re-apply code that used to paper
over this problem. I can just add these to the list of "harmless" messages to ignore when running new kernels on this old machine.

Or did I misunderstand some part of this thread?

-Tony

Nope, I think the complaints about BARs 0 and 8 on bus 80 are legitimate errors, and adding them to the list of "harmless" messages is the right thing to do.

The ones complaining about BAR 6 (an option ROM) are the ones I'm more concerned about. There's no address space assigned to them, but I don't think that indicates a firmware defect, and the message is sort of cryptic, so I don't want to emit it more often than we have to.

But if ia64 is handling these ROMs the same way as x86, this is just an opportunity for future improvement, and we don't need to do anything right now.

Bjorn

Sorry to hear that.

I did respond the email, and gave the the explanation.

ia64 was doing different,
1. It check if the range is valid with bus resources. and only call pci_claim_resource for that path.
2. and it does not reset invalid resource to allocate new one.

This patch change to call pci_claim_resource directly, so we have warning now. but just warning as we did not reset the resource.

Thanks

Yinghai

OK, I guess I just didn't understand what you were saying.

So I'll consider this resolved unless I hear somebody say otherwise.

Bjorn