Skip to contentSkip to conversation

several cleanup patches for fsl pci

PATCH In review Kevin Hao 3 participants 22 messages May 21, 2013 – Jul 28, 2013

Subsystems (2)
lore
Share & tools
lore

Kevin Hao

author
#0 lore raw

cover several cleanup patches for fsl pci

These patches are against Linus's tree and passed the boot test on T4240qds board.

Kevin Hao (3):
powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds
board
powerpc/fsl-pci: fix the unreachable warning message powerpc/fsl-pci: enable SWIOTLB in function setup_pci_atmu

 arch/powerpc/platforms/85xx/corenet_ds.c |  6 ------
 arch/powerpc/sysdev/fsl_pci.c            | 24 ++++--------------------
 2 files changed, 4 insertions(+), 26 deletions(-)
Signature · 1 line
1.8.1.4
author
#1 lore raw

PATCH 1/3 powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

The function pci_devs_phb_init is invoked more earlier than we really probe the pci controller, so it does nothing at all. And we also don't need the pci_dn stuff for the fsl powerpc64 boards, just remove it.

It also seems that we don't support ISA on all the current corenet ds boards. So picking a primary bus seems useless, remove that function too.

---
 arch/powerpc/platforms/85xx/corenet_ds.c | 6 ------
 1 file changed, 6 deletions(-)
arch/powerpc/platforms/85xx/corenet_ds.c +0 -6
index c59c617..aa3690b 100644
@@ -53,12 +53,6 @@ void __init corenet_ds_setup_arch(void)
 {
 	mpc85xx_smp_init();
 
-#if defined(CONFIG_PCI) && defined(CONFIG_PPC64)
-	pci_devs_phb_init();
-#endif
-
-	fsl_pci_assign_primary();
-
 	swiotlb_detect_4g();
 
 	pr_info("%s board from Freescale Semiconductor\n", ppc_md.name);
Signature · 1 line
1.8.1.4
#2 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 3 lines · Kevin Hao

It also seems that we don't support ISA on all the current corenet ds boards. So picking a primary bus seems useless, remove that function too.

IIRC that was due to some bugs in the PPC PCI code in the absence of =20 any primary bus. fsl_pci_assign_primary() will arbitrarily pick one to =20 be primary if there's no ISA. Have the bugs been fixed?

-Scott=

author
#3 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 7 lines · Scott Wood
Quoted 3 lines · Kevin Hao

It also seems that we don't support ISA on all the current corenet ds boards. So picking a primary bus seems useless, remove that function too.

IIRC that was due to some bugs in the PPC PCI code in the absence of any primary bus.

Do you know more about these bugs?

Quoted 2 lines

fsl_pci_assign_primary() will arbitrarily pick one
to be primary if there's no ISA. Have the bugs been fixed?

I know there should be some reason that we put the fsl_pci_assign_primary() here. But frankly I am not sure what bugs this workaround try to fix. For these corenet boards picking one to be primary has no effect to the 64bit kernel. And for 32bit kernel, the only effect of this is that isa_io_base is set to the io virtual base of the primary bus. But the isa_io_base only make sense when we do have a isa bus, so that we can access some well-known io ports directly by using outx/inx. But if we don't have isa bus on the board, the value of isa_io_base should make no sense at all. So we really don't need to pick a fake primary bus. Of course I may miss something, correct me if I am wrong. :-)

Thanks,
Kevin

Quoted 2 lines

-Scott

#4 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 4 lines · Kevin Hao
Quoted 2 lines · Scott Wood
Quoted 1 line · Kevin Hao

It also seems that we don't support ISA on all the current corenet =20

ds

Quoted 8 lines
Quoted 1 line
Quoted 1 line

boards. So picking a primary bus seems useless, remove that =20

function

Quoted 4 lines
Quoted 1 line

too.

IIRC that was due to some bugs in the PPC PCI code in the absence of any primary bus.

=20
Do you know more about these bugs?

Not off the top of my head -- either search the archives or ask Ben.

Quoted 13 lines
Quoted 2 lines

fsl_pci_assign_primary() will arbitrarily pick one
to be primary if there's no ISA. Have the bugs been fixed?

=20
I know there should be some reason that we put the =20 fsl_pci_assign_primary()
here. But frankly I am not sure what bugs this workaround try to fix. =20 For these
corenet boards picking one to be primary has no effect to the 64bit =20 kernel.
And for 32bit kernel, the only effect of this is that isa_io_base is =20 set to the
io virtual base of the primary bus. But the isa_io_base only make =20 sense when

Quoted 8 lines

we do have a isa bus, so that we can access some well-known io ports =20 directly
by using outx/inx. But if we don't have isa bus on the board, the =20 value of
isa_io_base should make no sense at all. So we really don't need to =20 pick a
fake primary bus. Of course I may miss something, correct me if I am =20 wrong. :-)

outx/inx can also be used for PCI I/O BARs.

-Scott=

author
#5 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 7 lines · Scott Wood
Quoted 6 lines · Kevin Hao
Quoted 2 lines · Scott Wood
Quoted 1 line · Kevin Hao

It also seems that we don't support ISA on all the current

corenet ds

Quoted 1 line
Quoted 1 line

boards. So picking a primary bus seems useless, remove that

function

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

too.

IIRC that was due to some bugs in the PPC PCI code in the absence of any primary bus.

Do you know more about these bugs?

Not off the top of my head -- either search the archives or ask Ben.

Hi Ben,

Could you shed some light on this issue? Do we really has the restriction that we have to pick one bus controller as primary even there is no ISA bus on the board? I did check the current code and found no code has a requirement for this. I also searched the archives and but found nothing useful. :-(

Thanks,
Kevin

Quoted 26 lines
Quoted 14 lines
Quoted 3 lines

fsl_pci_assign_primary() will arbitrarily pick one
to be primary if there's no ISA. Have the bugs been fixed?

I know there should be some reason that we put the fsl_pci_assign_primary()
here. But frankly I am not sure what bugs this workaround try to fix. For these
corenet boards picking one to be primary has no effect to the 64bit kernel.
And for 32bit kernel, the only effect of this is that isa_io_base is set to the
io virtual base of the primary bus. But the isa_io_base only make sense when

Quoted 8 lines

we do have a isa bus, so that we can access some well-known io ports directly
by using outx/inx. But if we don't have isa bus on the board, the value of
isa_io_base should make no sense at all. So we really don't need to pick a
fake primary bus. Of course I may miss something, correct me if I am wrong. :-)

outx/inx can also be used for PCI I/O BARs.

-Scott

#6 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 7 lines · Kevin Hao

Hi Ben,

Could you shed some light on this issue? Do we really has the restriction that we have to pick one bus controller as primary even there is no ISA bus on the board? I did check the current code and found no code has a requirement for this. I also searched the archives and but found nothing useful. :-(

You can just pick the first one as primary... The reason we somewhat need a primary is related to how we handle IO space.

We ioremap the IO space of all busses and assign the base of the primary one to a global "_IO_BASE". Then any "port" access is an offset from that which means that non-primary can end up having negative offsets. We fix up all resources, which works fine ... unless drivers do stupid casts and the wrap-around fails.

The main reason we did that originally is because we still had a slew of x86 originated HW that would access hard wired IO ports, especially on things like CHRP machines, looking for things like 8259 PIC, legacy kbd controllers, UARTs, etc... at fixed IO port numbers.

We still support some of these boxes (though I do wonder how long since somebody last booted a Pegasos) so I'm not quite yet keen on getting rid of that stuff...

Cheers,
Ben.

author
#7 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 18 lines · Benjamin Herrenschmidt
Quoted 7 lines · Kevin Hao

Hi Ben,

Could you shed some light on this issue? Do we really has the restriction that we have to pick one bus controller as primary even there is no ISA bus on the board? I did check the current code and found no code has a requirement for this. I also searched the archives and but found nothing useful. :-(

You can just pick the first one as primary... The reason we somewhat need a primary is related to how we handle IO space.

We ioremap the IO space of all busses and assign the base of the primary one to a global "_IO_BASE". Then any "port" access is an offset from that which means that non-primary can end up having negative offsets. We fix up all resources, which works fine ... unless drivers do stupid casts and the wrap-around fails.

Quoted 8 lines

The main reason we did that originally is because we still had a slew of x86 originated HW that would access hard wired IO ports, especially on things like CHRP machines, looking for things like 8259 PIC, legacy kbd controllers, UARTs, etc... at fixed IO port numbers.

We still support some of these boxes (though I do wonder how long since somebody last booted a Pegasos) so I'm not quite yet keen on getting rid of that stuff...

Thanks for the detailed explanation. But I don't mean to drop the support of the primary bus. As you said, this is definitely needed now in order to make some legacy device drivers work. What I want to do is not to pick a primary bus if there is no such ISA devices at all. For example, on a fsl p4080ds board, we would do the following in the current kernel:

	/* pick up a random host bridge as primary bus */
	for_each_matching_node(np, pci_ids) {
		...
	}

	/* for host bridge 0 */
	fsl_add_bridge(pdev, primary = 1) { 
		...
		pci_process_bridge_OF_ranges(hose, dev, is_primary = 1);
	}

	/* for host bridge 1*/
	fsl_add_bridge(pdev, primary = 0) {
		...
		pci_process_bridge_OF_ranges(hose, dev, is_primary = 0);
	}

But there is no ISA bus on this board and we also don't need to support any fixed IO port numbers. So it seems redundant to iterate the device list to pick a host bridge as primary bus. So we can simply change the above to:

	/* for host bridge 0 */
	fsl_add_bridge(pdev, primary = 0) {
		...
		pci_process_bridge_OF_ranges(hose, dev, is_primary = 0);
	}

	/* for host bridge 1*/
	fsl_add_bridge(pdev, primary = 0) {
		...
		pci_process_bridge_OF_ranges(hose, dev, is_primary = 0);
	}

The effect of this change is that the isa_io_base will be 0 and the IO resource are equal to the virtual address of the IO space. But the IO functions such as outx/inx should work as well. This is why I ask the above question. What do you think about this? Are there any subtle bugs that will be triggered by this?

Thanks,
Kevin

Quoted 5 lines

Cheers,
Ben.

#8 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 5 lines · Kevin Hao

The effect of this change is that the isa_io_base will be 0 and the IO resource are equal to the virtual address of the IO space. But the IO functions such as outx/inx should work as well. This is why I ask the above question. What do you think about this? Are there any subtle bugs that will be triggered by this?

I don't see any obvious reason why that wouldn't work but like anything in that area, it needs a bit of testing & hammering to be sure ;-)

In fact it would work on pmac32 as well since those generally don't have legacy crap either.

So I have no fundamental objection, it just needs testing. My worry is that we need to make sure we don't break old chrp and I don't have any to test with. I'm happy to drop support for stuff nobody uses anymore (we did drop PReP after all and I'm *that* close to drop power3) but as long as somebody is still using a CHRP RS6K or a Pegasos I can't quite drop those... And they do have legacy ISA crap to deal with.

Cheers,
Ben.

author
#9 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 10 lines · Benjamin Herrenschmidt
Quoted 5 lines · Kevin Hao

The effect of this change is that the isa_io_base will be 0 and the IO resource are equal to the virtual address of the IO space. But the IO functions such as outx/inx should work as well. This is why I ask the above question. What do you think about this? Are there any subtle bugs that will be triggered by this?

I don't see any obvious reason why that wouldn't work but like anything in that area, it needs a bit of testing & hammering to be sure ;-)

Agreed.

Quoted 7 lines

In fact it would work on pmac32 as well since those generally don't have legacy crap either.

So I have no fundamental objection, it just needs testing. My worry is that we need to make sure we don't break old chrp and I don't have any to test with.

Don't worry, my patch just drop the picking of primary bus for several fsl boards. All these changes are in board specific file, so it should have no affect to other boards at all.

For the other boards which have the same issue, since this is not emergency, we don't want handle them in a rush. So I would like to merge this patch first and then wait one or two kernel release cycle before handling the others to make sure that we don't break anything.

Thanks,
Kevin

Quoted 9 lines

I'm happy to drop support for stuff nobody uses anymore
(we did drop PReP after all and I'm *that* close to drop power3) but as long as somebody is still using a CHRP RS6K or a Pegasos I can't quite drop those... And they do have legacy ISA crap to deal with.

Cheers,
Ben.

#10 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 11 lines · Kevin Hao
Quoted 3 lines · On Sat, Jun 01, 2013 at 09:47:16PM +1000, Benjamin Herrenschmidt =20 wrote:
Quoted 1 line · Kevin Hao

The effect of this change is that the isa_io_base will be 0 and =20

the IO

Quoted 1 line
Quoted 1 line

resource are equal to the virtual address of the IO space. But =20

the IO

Quoted 1 line
Quoted 1 line

functions such as outx/inx should work as well. This is why I ask =20

the

Quoted 1 line
Quoted 1 line

above question. What do you think about this? Are there any =20

Quoted 8 lines

subtle bugs

Quoted 3 lines
Quoted 1 line

that will be triggered by this?

I don't see any obvious reason why that wouldn't work but like =20

anything

Quoted 1 line

in that area, it needs a bit of testing & hammering to be sure ;-)

=20
Agreed.

Please include QEMU in your testing, as that was where breakage was =20 observed that caused us to add the default primary.

Quoted 5 lines
Quoted 1 line

In fact it would work on pmac32 as well since those generally don't =20

have

Quoted 3 lines

legacy crap either.

So I have no fundamental objection, it just needs testing. My worry =20

Quoted 8 lines

is

Quoted 1 line

that we need to make sure we don't break old chrp and I don't have =20

any

Quoted 1 line

to test with.

=20
Don't worry, my patch just drop the picking of primary bus for several fsl boards. All these changes are in board specific file, so it should have no affect to other boards at all.

Is anything actually fixed by this?

-Scott=

author
#11 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 15 lines · Scott Wood
Quoted 14 lines · Kevin Hao
Quoted 3 lines · Benjamin Herrenschmidt
Quoted 1 line · Kevin Hao

The effect of this change is that the isa_io_base will be 0

and the IO

Quoted 1 line
Quoted 1 line

resource are equal to the virtual address of the IO space. But

the IO

Quoted 1 line
Quoted 1 line

functions such as outx/inx should work as well. This is why I

ask the

Quoted 1 line
Quoted 1 line

above question. What do you think about this? Are there any

subtle bugs

Quoted 2 lines
Quoted 1 line

that will be triggered by this?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I don't see any obvious reason why that wouldn't work but like

anything

Quoted 1 line

in that area, it needs a bit of testing & hammering to be sure ;-)

Agreed.

Please include QEMU in your testing, as that was where breakage was observed that caused us to add the default primary.

I tested this on a qemu with the following command:
./qemu-system-ppc -m 1024 -nographic -M ppce500 -kernel ./uImage \
-initrd /root/initrd.gz \
-append "root=/dev/ram rw console=ttyS0,115200 ramdisk_size=0x60000" \
-cpu e500mc -machine dt_compatible=fsl,,P4080DS

Before applying my patch:
PCI: Probing PCI hardware
fsl-pci e0008000.pci: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff] pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff
pci 0000:00:00.0: [1957:0030] type 01 class 0x060400
pci 0000:00:00.0: reg 10: [mem 0xfff00000-0xffffffff]
pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000
pci 0000:00:01.0: reg 10: [io 0x0000-0x001f]
pci 0000:00:01.0: reg 14: [mem 0x00000000-0x00000fff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:00.0: PCI bridge to [bus 01-ff]

  pci 0000:00:00.0:   bridge window [io  0x0000-0x0fff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff pref]

pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 01
PCI: Cannot allocate resource region 0 of device 0000:00:00.0, will remap PCI 0000:00 Cannot reserve Legacy IO [io 0x0000-0x0fff]
pci 0000:00:00.0: BAR 0: assigned [mem 0xc0000000-0xc00fffff] pci 0000:00:00.0: BAR 8: assigned [mem 0xc0100000-0xc01fffff] pci 0000:00:01.0: BAR 1: assigned [mem 0xc0200000-0xc0200fff] pci 0000:00:01.0: BAR 0: assigned [io 0x1000-0x101f]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0: bridge window [io 0x0000-0x0fff]
pci 0000:00:00.0: bridge window [mem 0xc0100000-0xc01fffff]
pci_bus 0000:00: resource 4 [io 0x0000-0xffff]
pci_bus 0000:00: resource 5 [mem 0xc0000000-0xdfffffff] pci_bus 0000:01: resource 0 [io 0x0000-0x0fff]
pci_bus 0000:01: resource 1 [mem 0xc0100000-0xc01fffff]

  # lspci -vvv
  00:00.0 PCI bridge: Freescale Semiconductor Inc MPC8533E (prog-if 00 [Normal decode])
          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
          Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
          Latency: 0
          Region 0: Memory at c0000000 (32-bit, non-prefetchable) [size=1M]
          Bus: primary=00, secondary=00, subordinate=00, sec-latency=0
          I/O behind bridge: 00000000-00000fff
          Memory behind bridge: 00000000-000fffff
          Prefetchable memory behind bridge: 00000000-000fffff
          Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
          BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                  PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-

00:01.0 Ethernet controller: Red Hat, Inc Virtio network device
Subsystem: Red Hat, Inc Device 0001
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 17
Region 0: I/O ports at 1000 [disabled] [size=32]
Region 1: Memory at c0200000 (32-bit, non-prefetchable) [disabled] [size=4K]
Capabilities: [40] MSI-X: Enable- Count=3 Masked-
Vector table: BAR=1 offset=00000000
PBA: BAR=1 offset=00000800

After applying my patch:
PCI: Probing PCI hardware
fsl-pci e0008000.pci: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io 0xf1020000-0xf102ffff] (bus address [0x0000-0xffff]) pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff
pci 0000:00:00.0: [1957:0030] type 01 class 0x060400
pci 0000:00:00.0: reg 10: [mem 0xfff00000-0xffffffff]
pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000
pci 0000:00:01.0: reg 10: [io 0xf1020000-0xf102001f]
pci 0000:00:01.0: reg 14: [mem 0x00000000-0x00000fff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:00.0: PCI bridge to [bus 01-ff]

  pci 0000:00:00.0:   bridge window [io  0xf1020000-0xf1020fff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff pref]

pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 01
PCI: Cannot allocate resource region 0 of device 0000:00:00.0, will remap
PCI: Cannot allocate resource region 0 of device 0000:00:01.0, will remap PCI 0000:00 Cannot reserve Legacy IO [io 0xf1020000-0xf1020fff] pci 0000:00:00.0: BAR 0: assigned [mem 0xc0000000-0xc00fffff] pci 0000:00:00.0: BAR 8: assigned [mem 0xc0100000-0xc01fffff] pci 0000:00:01.0: BAR 1: assigned [mem 0xc0200000-0xc0200fff] pci 0000:00:01.0: BAR 0: assigned [io 0xf1021000-0xf102101f] pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0: bridge window [io 0xf1020000-0xf1020fff]
pci 0000:00:00.0: bridge window [mem 0xc0100000-0xc01fffff]
pci_bus 0000:00: resource 4 [io 0xf1020000-0xf102ffff] pci_bus 0000:00: resource 5 [mem 0xc0000000-0xdfffffff] pci_bus 0000:01: resource 0 [io 0xf1020000-0xf1020fff] pci_bus 0000:01: resource 1 [mem 0xc0100000-0xc01fffff]

  # lspci -vvv
  00:00.0 PCI bridge: Freescale Semiconductor Inc MPC8533E (prog-if 00 [Normal decode])
          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
          Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
          Latency: 0
          Region 0: Memory at c0000000 (32-bit, non-prefetchable) [size=1M]
          Bus: primary=00, secondary=00, subordinate=00, sec-latency=0
          I/O behind bridge: 00000000-00000fff
          Memory behind bridge: 00000000-000fffff
          Prefetchable memory behind bridge: 00000000-000fffff
          Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
          BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                  PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-

00:01.0 Ethernet controller: Red Hat, Inc Virtio network device
Subsystem: Red Hat, Inc Device 0001
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 17
Region 0: I/O ports at 1000 [disabled] [size=32]
Region 1: Memory at c0200000 (32-bit, non-prefetchable) [disabled] [size=4K]
Capabilities: [40] MSI-X: Enable- Count=3 Masked-
Vector table: BAR=1 offset=00000000
PBA: BAR=1 offset=00000800

As you can see, the only difference between these two logs is the io resource address and all the mem and bus address are still the same.

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

In fact it would work on pmac32 as well since those generally

don't have

Quoted 3 lines

legacy crap either.

So I have no fundamental objection, it just needs testing. My

worry is

Quoted 1 line

that we need to make sure we don't break old chrp and I don't

Quoted 8 lines
Quoted 6 lines

have any

Quoted 1 line

to test with.

Don't worry, my patch just drop the picking of primary bus for several fsl boards. All these changes are in board specific file, so it should have no affect to other boards at all.

Is anything actually fixed by this?

No. As you know the reason we need a primary bus is that we need to support some ISA legacy devices. So it seems a little weird that we have to pick one primary bus even on a board which doesn't has ISA at all. Even though we can't drop the support of the primary bus due to these legacy devices, we had better narrow the scope of using it instead of propagating it to the boards which definitely don't care the primary bus at all.

Thanks,
Kevin

Quoted 2 lines

-Scott

#12 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 3 lines · Kevin Hao

As you can see, the only difference between these two logs is the io resource address and all the mem and bus address are still the same.

Hmm... The thing that originally motivated the picking of an arbitrary =20 primary was virtio being assigned a BAR of zero, and QEMU not liking =20 that. I guess something changed so that's no longer happening, even =20 without a primary.

Quoted 7 lines
Quoted 1 line

Is anything actually fixed by this?

=20
No. As you know the reason we need a primary bus is that we need to =20 support
some ISA legacy devices. So it seems a little weird that we have to =20 pick
one primary bus even on a board which doesn't has ISA at all.

Yeah, it was a workaround for some other issues, as discussed. It was =20 meant to be temporary.

-Scott=

#13 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

In reply to Kevin Hao
Quoted 68 lines · Kevin Hao
Quoted 10 lines · Scott Wood
Quoted 9 lines · Kevin Hao
Quoted 3 lines · Benjamin Herrenschmidt
Quoted 1 line · Kevin Hao

The effect of this change is that the isa_io_base will be 0

and the IO

Quoted 1 line
Quoted 1 line

resource are equal to the virtual address of the IO space. But

the IO

Quoted 1 line
Quoted 1 line

functions such as outx/inx should work as well. This is why I

Quoted 8 lines
Quoted 8 lines

ask the

Quoted 1 line
Quoted 1 line

above question. What do you think about this? Are there any

subtle bugs

Quoted 3 lines
Quoted 1 line

that will be triggered by this?

I don't see any obvious reason why that wouldn't work but like

anything

Quoted 1 line

in that area, it needs a bit of testing & hammering to be sure =20

;-)

Quoted 5 lines
Quoted 2 lines

Agreed.

Please include QEMU in your testing, as that was where breakage was observed that caused us to add the default primary.

=20
I tested this on a qemu with the following command:
./qemu-system-ppc -m 1024 -nographic -M ppce500 -kernel ./uImage \
-initrd /root/initrd.gz \
-append "root=3D/dev/ram rw console=3DttyS0,115200 =20
ramdisk_size=3D0x60000" \
-cpu e500mc -machine dt_compatible=3Dfsl,,P4080DS
=20
Before applying my patch:
PCI: Probing PCI hardware
fsl-pci e0008000.pci: PCI host bridge to bus 0000:00 pci_bus 0000:00: root bus resource [io 0x0000-0xffff] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff] pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff pci 0000:00:00.0: [1957:0030] type 01 class 0x060400 pci 0000:00:00.0: reg 10: [mem 0xfff00000-0xffffffff] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 pci 0000:00:01.0: reg 10: [io 0x0000-0x001f]
pci 0000:00:01.0: reg 14: [mem 0x00000000-0x00000fff] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), =20
reconfiguring
pci 0000:00:00.0: PCI bridge to [bus 01-ff]

  pci 0000:00:00.0:   bridge window [io  0x0000-0x0fff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff pref]

pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 01
PCI: Cannot allocate resource region 0 of device 0000:00:00.0, will =20
remap
PCI 0000:00 Cannot reserve Legacy IO [io 0x0000-0x0fff] pci 0000:00:00.0: BAR 0: assigned [mem 0xc0000000-0xc00fffff] pci 0000:00:00.0: BAR 8: assigned [mem 0xc0100000-0xc01fffff] pci 0000:00:01.0: BAR 1: assigned [mem 0xc0200000-0xc0200fff] pci 0000:00:01.0: BAR 0: assigned [io 0x1000-0x101f]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0: bridge window [io 0x0000-0x0fff]
pci 0000:00:00.0: bridge window [mem 0xc0100000-0xc01fffff]
pci_bus 0000:00: resource 4 [io 0x0000-0xffff]
pci_bus 0000:00: resource 5 [mem 0xc0000000-0xdfffffff] pci_bus 0000:01: resource 0 [io 0x0000-0x0fff]
pci_bus 0000:01: resource 1 [mem 0xc0100000-0xc01fffff]
=20

Quoted 11 lines
  # lspci -vvv
  00:00.0 PCI bridge: Freescale Semiconductor Inc MPC8533E (prog-if =20
00 [Normal decode])
          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- =20
ParErr- Stepping- SERR+ FastB2B- DisINTx-
          Status: Cap- 66MHz- UDF- FastB2B- ParErr- =20
DEVSEL=3Dfast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
          Latency: 0
          Region 0: Memory at c0000000 (32-bit, non-prefetchable) =20
[size=3D1M]
          Bus: primary=3D00, secondary=3D00, subordinate=3D00, sec-latenc=

y=3D0

Quoted 61 lines

I/O behind bridge: 00000000-00000fff
Memory behind bridge: 00000000-000fffff Prefetchable memory behind bridge: 00000000-000fffff Secondary status: 66MHz- FastB2B- ParErr- =20
DEVSEL=3Dfast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- =20
FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
=20
00:01.0 Ethernet controller: Red Hat, Inc Virtio network device
Subsystem: Red Hat, Inc Device 0001
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- =20
ParErr- Stepping- SERR- FastB2B- DisINTx-

          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- =20
DEVSEL=3Dfast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
          Interrupt: pin A routed to IRQ 17
          Region 0: I/O ports at 1000 [disabled] [size=3D32]
          Region 1: Memory at c0200000 (32-bit, non-prefetchable) =20
[disabled] [size=3D4K]
          Capabilities: [40] MSI-X: Enable- Count=3D3 Masked-
                  Vector table: BAR=3D1 offset=3D00000000
                  PBA: BAR=3D1 offset=3D00000800

=20
After applying my patch:
PCI: Probing PCI hardware
fsl-pci e0008000.pci: PCI host bridge to bus 0000:00 pci_bus 0000:00: root bus resource [io 0xf1020000-0xf102ffff] (bus =20
address [0x0000-0xffff])
pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff] pci_bus 0000:00: root bus resource [bus 00-ff]
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff pci 0000:00:00.0: [1957:0030] type 01 class 0x060400
pci 0000:00:00.0: reg 10: [mem 0xfff00000-0xffffffff]
pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000
pci 0000:00:01.0: reg 10: [io 0xf1020000-0xf102001f]
pci 0000:00:01.0: reg 14: [mem 0x00000000-0x00000fff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), =20
reconfiguring
pci 0000:00:00.0: PCI bridge to [bus 01-ff]

  pci 0000:00:00.0:   bridge window [io  0xf1020000-0xf1020fff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
  pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff pref]

pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 01
PCI: Cannot allocate resource region 0 of device 0000:00:00.0, will =20
remap
PCI: Cannot allocate resource region 0 of device 0000:00:01.0, will =20
remap
PCI 0000:00 Cannot reserve Legacy IO [io 0xf1020000-0xf1020fff] pci 0000:00:00.0: BAR 0: assigned [mem 0xc0000000-0xc00fffff] pci 0000:00:00.0: BAR 8: assigned [mem 0xc0100000-0xc01fffff] pci 0000:00:01.0: BAR 1: assigned [mem 0xc0200000-0xc0200fff] pci 0000:00:01.0: BAR 0: assigned [io 0xf1021000-0xf102101f] pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0: bridge window [io 0xf1020000-0xf1020fff]
pci 0000:00:00.0: bridge window [mem 0xc0100000-0xc01fffff]
pci_bus 0000:00: resource 4 [io 0xf1020000-0xf102ffff] pci_bus 0000:00: resource 5 [mem 0xc0000000-0xdfffffff] pci_bus 0000:01: resource 0 [io 0xf1020000-0xf1020fff] pci_bus 0000:01: resource 1 [mem 0xc0100000-0xc01fffff]
=20

Quoted 11 lines
  # lspci -vvv
  00:00.0 PCI bridge: Freescale Semiconductor Inc MPC8533E (prog-if =20
00 [Normal decode])
          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- =20
ParErr- Stepping- SERR+ FastB2B- DisINTx-
          Status: Cap- 66MHz- UDF- FastB2B- ParErr- =20
DEVSEL=3Dfast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
          Latency: 0
          Region 0: Memory at c0000000 (32-bit, non-prefetchable) =20
[size=3D1M]
          Bus: primary=3D00, secondary=3D00, subordinate=3D00, sec-latenc=

y=3D0

Quoted 22 lines

I/O behind bridge: 00000000-00000fff
Memory behind bridge: 00000000-000fffff Prefetchable memory behind bridge: 00000000-000fffff Secondary status: 66MHz- FastB2B- ParErr- =20
DEVSEL=3Dfast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- =20
FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
=20
00:01.0 Ethernet controller: Red Hat, Inc Virtio network device
Subsystem: Red Hat, Inc Device 0001
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- =20
ParErr- Stepping- SERR- FastB2B- DisINTx-

          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- =20
DEVSEL=3Dfast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
          Interrupt: pin A routed to IRQ 17
          Region 0: I/O ports at 1000 [disabled] [size=3D32]
          Region 1: Memory at c0200000 (32-bit, non-prefetchable) =20
[disabled] [size=3D4K]
          Capabilities: [40] MSI-X: Enable- Count=3D3 Masked-
                  Vector table: BAR=3D1 offset=3D00000000
                  PBA: BAR=3D1 offset=3D00000800
Quoted 5 lines

=20
=20
As you can see, the only difference between these two logs is the io resource address and all the mem and bus address are still the same.

I dug a bit deeper into this, and it's making by head hurt. It seems =20 that we're now getting saved by the host bridge (that for some reason =20 has the class code of a PCI-to-PCI bridge rather than a host bridge) =20 having I/O space of 0x1000 bytes[1], which gets allocated at zero. =20 There have been some changes in the QEMU PCI code since I saw the =20 problem, including changing the class code of the bridge, so that's =20 probably why it sort-of works now.

What QEMU is doing does not match what real hardware does, though. At =20 least on mpc8536 which is similar to mpc8544 (I wasn't able to quickly =20 get access to a working mpc8544 to test on), the PCI bridge has class =20 code Processor, rather than bridge of any sort. Thus, on real hardware =20 we would not get the 0x1000 reservation. What hardware does seems =20 broken to me (when PCI is configured as a host rather than as an =20 endpoint), but so does calling a host bridge a PCI-to-PCI bridge, and =20 so does relying on this mess (not that the primary-bus hack is much =20 better...).

Note that in the case of PCIe -- but not PCI -- Linux has a quirk that =20 relabels the "Processor" on FSL chips as a PCI-to-PCI bridge. Despite =20 the "Processor" labelling, Freescale PCIe (but not PCI) devices appear =20
to have a type 1 config space layout in host mode. I guess this is =20
considered to be the virtual root port.

-Scott

[1] Really, it's the bridge not having any I/O, but this is how an I/O =20 base/limit of zero are interpreted.=

author
#14 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 2 lines · Scott Wood
Quoted 1 line · Kevin Hao
                 Vector table: BAR=1 offset=00000000

<snip>

Quoted 7 lines
Quoted 6 lines
                 PBA: BAR=1 offset=00000800

As you can see, the only difference between these two logs is the io resource address and all the mem and bus address are still the same.

Quoted 7 lines

I dug a bit deeper into this, and it's making by head hurt. It seems that we're now getting saved by the host bridge (that for some reason has the class code of a PCI-to-PCI bridge rather than a host bridge) having I/O space of 0x1000 bytes[1], which gets allocated at zero. There have been some changes in the QEMU PCI code since I saw the problem, including changing the class code of the bridge, so that's probably why it sort-of works now.

We are not just lucky here. Even without the change of the qemu pci code we still can get the correct IO address in the current kernel. The following is the log by running the latest kernel on the qemu which doesn't have the PCI-to-PCI bridge change yet. As you can see we don't pick a primary pci bus in this case and the Ethernet controller still get the I/O ports at 1000.

    PCI: Probing PCI hardware
    fsl-pci e0008000.pci: PCI host bridge to bus 0000:00
    pci_bus 0000:00: root bus resource [io  0xf1020000-0xf102ffff] (bus address [0x0000-0xffff])
    pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff]
    pci_bus 0000:00: root bus resource [bus 00-ff]
    pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff
    pci 0000:00:00.0: [1957:0030] type 00 class 0x0b2000
    pci 0000:00:11.0: [1af4:1000] type 00 class 0x020000
    pci 0000:00:11.0: reg 0x10: [io  0xf1020000-0xf102001f]
    pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
    pci 0000:00:11.0: BAR 0: assigned [io  0xf1021000-0xf102101f]
    pci_bus 0000:00: resource 4 [io  0xf1020000-0xf102ffff]
    pci_bus 0000:00: resource 5 [mem 0xc0000000-0xdfffffff]
    
    root@localhost:~# lspci -v
    00:00.0 Power PC: Freescale Semiconductor Inc MPC8533E
    	Subsystem: Red Hat, Inc Device 1100
    	Flags: bus master, fast devsel, latency 0
    
    00:11.0 Ethernet controller: Red Hat, Inc Virtio network device
    	Subsystem: Red Hat, Inc Device 0001
    	Flags: fast devsel, IRQ 16
    	I/O ports at 1000 [disabled] [size=32]

The reason is that the ppc kernel assume that the BARs starting at 0 is unset and will reassign it later. There is a bug in the previous kernel, so the kernel maybe not work well for qemu in this case. But I think this has been fixed by the commit c5df457f (powerpc/pci: Check the bus address instead of resource address in pcibios_fixup_resources).

Quoted 6 lines

What QEMU is doing does not match what real hardware does, though. At least on mpc8536 which is similar to mpc8544 (I wasn't able to quickly get access to a working mpc8544 to test on), the PCI bridge has class code Processor, rather than bridge of any sort. Thus, on real hardware we would not get the 0x1000 reservation.

This doesn't matter. We can always make sure to reassign these resources starting at 0. I also done a test on the mpc8536ds board. Both the pci and pcie devices work pretty well without picking a primary pci bus on this board. The following is the log.

    Found FSL PCI host bridge at 0x00000000ffe08000. Firmware bus number: 0->0
    PCI host bridge /pci@ffe08000  ranges:
     MEM 0x0000000080000000..0x000000008fffffff -> 0x0000000080000000 
      IO 0x00000000ffc00000..0x00000000ffc0ffff -> 0x0000000000000000
    /pci@ffe08000: PCICSRBAR @ 0xfff00000
    Found FSL PCI host bridge at 0x00000000ffe09000. Firmware bus number: 0->0
    PCI host bridge /pcie@ffe09000  ranges:
     MEM 0x0000000098000000..0x000000009fffffff -> 0x0000000098000000 
      IO 0x00000000ffc20000..0x00000000ffc2ffff -> 0x0000000000000000
    /pcie@ffe09000: PCICSRBAR @ 0xfff00000
    Found FSL PCI host bridge at 0x00000000ffe0a000. Firmware bus number: 0->1
    PCI host bridge /pcie@ffe0a000  ranges:
     MEM 0x0000000090000000..0x0000000097ffffff -> 0x0000000090000000 
      IO 0x00000000ffc10000..0x00000000ffc1ffff -> 0x0000000000000000
    /pcie@ffe0a000: PCICSRBAR @ 0xfff00000
    Found FSL PCI host bridge at 0x00000000ffe0b000. Firmware bus number: 0->0
    PCI host bridge /pcie@ffe0b000  ranges:
     MEM 0x00000000a0000000..0x00000000bfffffff -> 0x00000000a0000000 
      IO 0x00000000ffc30000..0x00000000ffc3ffff -> 0x0000000000000000
    /pcie@ffe0b000: PCICSRBAR @ 0xfff00000
    PCI: Probing PCI hardware
    fsl-pci ffe08000.pci: PCI host bridge to bus 0000:00
    pci_bus 0000:00: root bus resource [io  0xe1020000-0xe102ffff] (bus address [0x0000-0xffff])
    pci_bus 0000:00: root bus resource [mem 0x80000000-0x8fffffff]
    pci_bus 0000:00: root bus resource [bus 00-ff]
    pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff
    pci 0000:00:00.0: [1957:0050] type 00 class 0x0b2000
    pci 0000:00:00.0: reg 0x10: [mem 0xfff00000-0xffffffff]
    pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x1fffffff pref]
    pci 0000:00:11.0: [8086:107c] type 00 class 0x020000
    pci 0000:00:11.0: reg 0x10: [mem 0x80000000-0x8001ffff]
    pci 0000:00:11.0: reg 0x14: [mem 0x80020000-0x8003ffff]
    pci 0000:00:11.0: reg 0x18: [io  0xe1021000-0xe102103f]
    pci 0000:00:11.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
    pci 0000:00:11.0: PME# supported from D0 D3hot D3cold
    pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
    fsl-pci ffe09000.pcie: PCI host bridge to bus 0001:01
    pci_bus 0001:01: root bus resource [io  0xe1040000-0xe104ffff] (bus address [0x0000-0xffff])
    pci_bus 0001:01: root bus resource [mem 0x98000000-0x9fffffff]
    pci_bus 0001:01: root bus resource [bus 01-ff]
    pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to ff
    pci 0001:01:00.0: [1957:0050] type 01 class 0x0b2000
    pci 0001:01:00.0: ignoring class 0x0b2000 (doesn't match header type 01)
    pci 0001:01:00.0: supports D1 D2
    pci 0001:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
    pci 0001:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
    pci 0001:01:00.0: PCI bridge to [bus 02-ff]
    pci 0001:01:00.0:   bridge window [io  0xe1040000-0xe1040fff]
    pci 0001:01:00.0:   bridge window [mem 0x00000000-0x000fffff]
    pci 0001:01:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
    pci_bus 0001:02: busn_res: [bus 02-ff] end is updated to 02
    pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to 02
    fsl-pci ffe0a000.pcie: PCI host bridge to bus 0002:03
    pci_bus 0002:03: root bus resource [io  0xe1060000-0xe106ffff] (bus address [0x0000-0xffff])
    pci_bus 0002:03: root bus resource [mem 0x90000000-0x97ffffff]
    pci_bus 0002:03: root bus resource [bus 03-ff]
    pci_bus 0002:03: busn_res: [bus 03-ff] end is updated to ff
    pci 0002:03:00.0: [1957:0050] type 01 class 0x0b2000
    pci 0002:03:00.0: ignoring class 0x0b2000 (doesn't match header type 01)
    pci 0002:03:00.0: supports D1 D2
    pci 0002:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold
    pci 0002:03:00.0: Primary bus is hard wired to 0
    pci 0002:03:00.0: bridge configuration invalid ([bus 01-01]), reconfiguring
    pci 0002:04:00.0: [8086:107d] type 00 class 0x020000
    pci 0002:04:00.0: reg 0x10: [mem 0x90000000-0x9001ffff]
    pci 0002:04:00.0: reg 0x14: [mem 0x90020000-0x9003ffff]
    pci 0002:04:00.0: reg 0x18: [io  0xe1061000-0xe106101f]
    pci 0002:04:00.0: PME# supported from D0 D3hot D3cold
    pci 0002:03:00.0: PCI bridge to [bus 04-ff]
    pci 0002:03:00.0:   bridge window [io  0xe1060000-0xe1060fff]
    pci 0002:03:00.0:   bridge window [mem 0x90000000-0x900fffff]
    pci_bus 0002:04: busn_res: [bus 04-ff] end is updated to 04
    pci_bus 0002:03: busn_res: [bus 03-ff] end is updated to 04
    fsl-pci ffe0b000.pcie: PCI host bridge to bus 0003:05
    pci_bus 0003:05: root bus resource [io  0xe1080000-0xe108ffff] (bus address [0x0000-0xffff])
    pci_bus 0003:05: root bus resource [mem 0xa0000000-0xbfffffff]
    pci_bus 0003:05: root bus resource [bus 05-ff]
    pci_bus 0003:05: busn_res: [bus 05-ff] end is updated to ff
    pci 0003:05:00.0: [1957:0050] type 01 class 0x0b2000
    pci 0003:05:00.0: ignoring class 0x0b2000 (doesn't match header type 01)
    pci 0003:05:00.0: supports D1 D2
    pci 0003:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold
    pci 0003:05:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
    pci 0003:05:00.0: PCI bridge to [bus 06-ff]
    pci 0003:05:00.0:   bridge window [io  0xe1080000-0xe1080fff]
    pci 0003:05:00.0:   bridge window [mem 0x00000000-0x000fffff]
    pci 0003:05:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
    pci_bus 0003:06: busn_res: [bus 06-ff] end is updated to 06
    pci_bus 0003:05: busn_res: [bus 05-ff] end is updated to 06
    PCI 0001:01 Cannot reserve Legacy IO [io  0xe1040000-0xe1040fff]
    PCI 0002:03 Cannot reserve Legacy IO [io  0xe1060000-0xe1060fff]
    PCI 0003:05 Cannot reserve Legacy IO [io  0xe1080000-0xe1080fff]
    pci 0000:00:11.0: BAR 6: assigned [mem 0x80040000-0x8005ffff pref]
    pci 0001:01:00.0: PCI bridge to [bus 02]
    pci 0001:01:00.0:   bridge window [io  0xe1040000-0xe104ffff]
    pci 0001:01:00.0:   bridge window [mem 0x98000000-0x9fffffff]
    pci 0002:03:00.0: PCI bridge to [bus 04]
    pci 0002:03:00.0:   bridge window [io  0xe1060000-0xe106ffff]
    pci 0002:03:00.0:   bridge window [mem 0x90000000-0x97ffffff]
    pci 0003:05:00.0: PCI bridge to [bus 06]
    pci 0003:05:00.0:   bridge window [io  0xe1080000-0xe108ffff]
    pci 0003:05:00.0:   bridge window [mem 0xa0000000-0xbfffffff]
    pci 0001:01:00.0: enabling device (0106 -> 0107)
    pci 0002:03:00.0: enabling device (0106 -> 0107)
    pci 0003:05:00.0: enabling device (0106 -> 0107)
    pci_bus 0000:00: resource 4 [io  0xe1020000-0xe102ffff]
    pci_bus 0000:00: resource 5 [mem 0x80000000-0x8fffffff]
    pci_bus 0001:01: resource 4 [io  0xe1040000-0xe104ffff]
    pci_bus 0001:01: resource 5 [mem 0x98000000-0x9fffffff]
    pci_bus 0001:02: resource 0 [io  0xe1040000-0xe104ffff]
    pci_bus 0001:02: resource 1 [mem 0x98000000-0x9fffffff]
    pci_bus 0002:03: resource 4 [io  0xe1060000-0xe106ffff]
    pci_bus 0002:03: resource 5 [mem 0x90000000-0x97ffffff]
    pci_bus 0002:04: resource 0 [io  0xe1060000-0xe106ffff]
    pci_bus 0002:04: resource 1 [mem 0x90000000-0x97ffffff]
    pci_bus 0003:05: resource 4 [io  0xe1080000-0xe108ffff]
    pci_bus 0003:05: resource 5 [mem 0xa0000000-0xbfffffff]
    pci_bus 0003:06: resource 0 [io  0xe1080000-0xe108ffff]
    pci_bus 0003:06: resource 1 [mem 0xa0000000-0xbfffffff]
    
    
    root@intel_tunnel_creek-2:/root> lspci -v
    0000:00:00.0 Power PC: Freescale Semiconductor Inc MPC8536E (rev 91)
    	Flags: bus master, 66MHz, fast devsel, latency 128
    	Memory at <ignored> (32-bit, non-prefetchable)
    	Memory at <unassigned> (32-bit, prefetchable)
    	Memory at <unassigned> (64-bit, non-prefetchable)
    	Memory at <unassigned> (64-bit, non-prefetchable)
    
    0000:00:11.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05)
    	Subsystem: Intel Corporation PRO/1000 GT Desktop Adapter
    	Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 35
    	Memory at 80000000 (32-bit, non-prefetchable) [size=128K]
    	Memory at 80020000 (32-bit, non-prefetchable) [size=128K]
    	I/O ports at 1000 [size=64]
    	[virtual] Expansion ROM at 80040000 [disabled] [size=128K]
    	Capabilities: [dc] Power Management version 2
    	Capabilities: [e4] PCI-X non-bridge device
    	Kernel driver in use: e1000
    
    0001:01:00.0 PCI bridge: Freescale Semiconductor Inc MPC8536E (rev 91) (prog-if 00 [Normal decode])
    	Flags: bus master, fast devsel, latency 0
    	Memory at <ignored> (32-bit, non-prefetchable)
    	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
    	I/O behind bridge: 00000000-00000fff
    	Memory behind bridge: 98000000-9fffffff
    	Capabilities: [44] Power Management version 2
    	Capabilities: [4c] Express Root Port (Slot-), MSI 00
    	Capabilities: [100] Advanced Error Reporting
    
    0002:03:00.0 PCI bridge: Freescale Semiconductor Inc MPC8536E (rev 91) (prog-if 00 [Normal decode])
    	Flags: bus master, fast devsel, latency 0
    	Memory at <ignored> (32-bit, non-prefetchable)
    	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
    	I/O behind bridge: 00000000-00000fff
    	Memory behind bridge: 90000000-97ffffff
    	Capabilities: [44] Power Management version 2
    	Capabilities: [4c] Express Root Port (Slot-), MSI 00
    	Capabilities: [100] Advanced Error Reporting
    
    0002:04:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
    	Subsystem: Intel Corporation PRO/1000 PT Server Adapter
    	Flags: bus master, fast devsel, latency 0, IRQ 38
    	Memory at 90000000 (32-bit, non-prefetchable) [size=128K]
    	Memory at 90020000 (32-bit, non-prefetchable) [size=128K]
    	I/O ports at 1000 [disabled] [size=32]
    	Capabilities: [c8] Power Management version 2
    	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    	Capabilities: [e0] Express Endpoint, MSI 00
    	Capabilities: [100] Advanced Error Reporting
    	Capabilities: [140] Device Serial Number 00-15-17-ff-ff-f0-ad-1f
    	Kernel driver in use: e1000e
    
    0003:05:00.0 PCI bridge: Freescale Semiconductor Inc MPC8536E (rev 91) (prog-if 00 [Normal decode])
    	Flags: bus master, fast devsel, latency 0
    	Memory at <ignored> (32-bit, non-prefetchable)
    	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
    	I/O behind bridge: 00000000-00000fff
    	Memory behind bridge: a0000000-bfffffff
    	Capabilities: [44] Power Management version 2
    	Capabilities: [4c] Express Root Port (Slot-), MSI 00
    	Capabilities: [100] Advanced Error Reporting
Quoted 11 lines

What
hardware does seems broken to me (when PCI is configured as a host rather than as an endpoint), but so does calling a host bridge a PCI-to-PCI bridge, and so does relying on this mess (not that the primary-bus hack is much better...).

Note that in the case of PCIe -- but not PCI -- Linux has a quirk that relabels the "Processor" on FSL chips as a PCI-to-PCI bridge. Despite the "Processor" labelling, Freescale PCIe (but not PCI) devices appear to have a type 1 config space layout in host mode. I guess this is considered to be the virtual root port.

The resource reassign will occur no matter whether it is a pci or pcie. All in all the resource starting at 0 is considered a invalid resource by the ppc kernel and then the kernel will try to reassign it later. So we really don't have a reason to pick up a pci primary bus on a board which doesn't have ISA device at all in the current kernel.

Thanks,
Kevin

Quoted 5 lines

-Scott

[1] Really, it's the bridge not having any I/O, but this is how an I/O base/limit of zero are interpreted.

#15 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 7 lines · Kevin Hao

The reason is that the ppc kernel assume that the BARs starting at 0 is unset and will reassign it later. There is a bug in the =20 previous
kernel, so the kernel maybe not work well for qemu in this case. But I think this has been fixed by the commit c5df457f (powerpc/pci: Check =20 the
bus address instead of resource address in pcibios_fixup_resources).

What ensures that the reassignment will not assign zero?

I'm not doubting the result that it does, but I'm having a hard time =20 seeing it in the code.

Or are you saying that the resource assignment code will run twice, =20 rather than just saying that we won't trust the firmware when we find =20 zero?

-Scott=

author
#16 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 10 lines · Scott Wood
Quoted 7 lines · Kevin Hao

The reason is that the ppc kernel assume that the BARs starting at 0 is unset and will reassign it later. There is a bug in the previous
kernel, so the kernel maybe not work well for qemu in this case. But I think this has been fixed by the commit c5df457f (powerpc/pci:
Check the
bus address instead of resource address in pcibios_fixup_resources).

What ensures that the reassignment will not assign zero?

In function pcibios_resource_survey() we will reserve the low IO area before reassignment. Please see the following comments and codes in this function.

        /* Before we start assigning unassigned resource, we try to reserve
         * the low IO area and the VGA memory area if they intersect the
         * bus available resources to avoid allocating things on top of them
         */
        if (!pci_has_flag(PCI_PROBE_ONLY)) {
                list_for_each_entry(b, &pci_root_buses, node)
                        pcibios_reserve_legacy_regions(b);
        }    

Thanks,
Kevin

Quoted 9 lines

I'm not doubting the result that it does, but I'm having a hard time seeing it in the code.

Or are you saying that the resource assignment code will run twice, rather than just saying that we won't trust the firmware when we find zero?

-Scott

author
#17 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

In reply to Scott Wood
Quoted 38 lines · Scott Wood
Quoted 4 lines · Kevin Hao
Quoted 2 lines · Scott Wood
Quoted 1 line · Kevin Hao

It also seems that we don't support ISA on all the current

corenet ds

Quoted 8 lines
Quoted 1 line
Quoted 1 line

boards. So picking a primary bus seems useless, remove that

function

Quoted 4 lines
Quoted 1 line

too.

IIRC that was due to some bugs in the PPC PCI code in the absence of any primary bus.

Do you know more about these bugs?

Not off the top of my head -- either search the archives or ask Ben.

Quoted 13 lines
Quoted 2 lines

fsl_pci_assign_primary() will arbitrarily pick one
to be primary if there's no ISA. Have the bugs been fixed?

I know there should be some reason that we put the fsl_pci_assign_primary()
here. But frankly I am not sure what bugs this workaround try to fix. For these
corenet boards picking one to be primary has no effect to the 64bit kernel.
And for 32bit kernel, the only effect of this is that isa_io_base is set to the
io virtual base of the primary bus. But the isa_io_base only make sense when

Quoted 8 lines

we do have a isa bus, so that we can access some well-known io ports directly
by using outx/inx. But if we don't have isa bus on the board, the value of
isa_io_base should make no sense at all. So we really don't need to pick a
fake primary bus. Of course I may miss something, correct me if I am wrong. :-)

Quoted 1 line

outx/inx can also be used for PCI I/O BARs.

Yes, I know there is also PIO. But the value of isa_io_base doesn't have any effect for this.

Thanks,
Kevin

Quoted 2 lines

-Scott

#18 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 40 lines · Kevin Hao
Quoted 10 lines · Scott Wood
Quoted 9 lines · Kevin Hao
Quoted 2 lines · Scott Wood
Quoted 1 line · Kevin Hao

It also seems that we don't support ISA on all the current

corenet ds

Quoted 1 line
Quoted 1 line

boards. So picking a primary bus seems useless, remove that

function

Quoted 3 lines
Quoted 1 line

too.

IIRC that was due to some bugs in the PPC PCI code in the =20

absence of

Quoted 28 lines
Quoted 3 lines
Quoted 1 line

any primary bus.

Do you know more about these bugs?

Not off the top of my head -- either search the archives or ask Ben.

Quoted 21 lines
Quoted 2 lines

fsl_pci_assign_primary() will arbitrarily pick one
to be primary if there's no ISA. Have the bugs been fixed?

I know there should be some reason that we put the fsl_pci_assign_primary()
here. But frankly I am not sure what bugs this workaround try to fix. For these
corenet boards picking one to be primary has no effect to the 64bit kernel.
And for 32bit kernel, the only effect of this is that isa_io_base is set to the
io virtual base of the primary bus. But the isa_io_base only make sense when
we do have a isa bus, so that we can access some well-known io ports directly
by using outx/inx. But if we don't have isa bus on the board, the value of
isa_io_base should make no sense at all. So we really don't need to pick a
fake primary bus. Of course I may miss something, correct me if I am wrong. :-)

Quoted 4 lines
Quoted 1 line

outx/inx can also be used for PCI I/O BARs.

=20
Yes, I know there is also PIO. But the value of isa_io_base doesn't have any effect for this.

See this e-mail for some of the issues I was referring to with =20 isa_io_base being zero:
https://lists.ozlabs.org/pipermail/linuxppc-dev/2012-June/098586.html

Reading it again I'm not so sure that the problem is so much that we =20 need a primary, as that somewhat bad things happen on non-primary =20 buses, such as the possibility of assigning a zero BAR. Some hardware =20 (including QEMU's PCI emulation) cares about this, though most =20 doesn't. We only have one PCI bus under QEMU, so when we started =20 picking an arbitrary bus to be primary, the problem went away because =20 there was only one bus and therefore there was no non-primary bus.

-Scott=

author
#19 lore raw

Re: [PATCH 1/3] powerpc/mpc85xx: remove the unneeded pci init functions for corenet ds board

Quoted 41 lines · Scott Wood
Quoted 40 lines · Kevin Hao
Quoted 10 lines · Scott Wood
Quoted 9 lines · Kevin Hao
Quoted 2 lines · Scott Wood
Quoted 1 line · Kevin Hao

It also seems that we don't support ISA on all the current

corenet ds

Quoted 1 line
Quoted 1 line

boards. So picking a primary bus seems useless, remove that

function

Quoted 3 lines
Quoted 1 line

too.

IIRC that was due to some bugs in the PPC PCI code in the

absence of

Quoted 28 lines
Quoted 3 lines
Quoted 1 line

any primary bus.

Do you know more about these bugs?

Not off the top of my head -- either search the archives or ask Ben.

Quoted 21 lines
Quoted 2 lines

fsl_pci_assign_primary() will arbitrarily pick one
to be primary if there's no ISA. Have the bugs been fixed?

I know there should be some reason that we put the fsl_pci_assign_primary()
here. But frankly I am not sure what bugs this workaround try to fix. For these
corenet boards picking one to be primary has no effect to the 64bit kernel.
And for 32bit kernel, the only effect of this is that isa_io_base is set to the
io virtual base of the primary bus. But the isa_io_base only make sense when
we do have a isa bus, so that we can access some well-known io ports directly
by using outx/inx. But if we don't have isa bus on the board, the value of
isa_io_base should make no sense at all. So we really don't need to pick a
fake primary bus. Of course I may miss something, correct me if I am wrong. :-)

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

outx/inx can also be used for PCI I/O BARs.

Yes, I know there is also PIO. But the value of isa_io_base doesn't have any effect for this.

See this e-mail for some of the issues I was referring to with isa_io_base being zero:
https://lists.ozlabs.org/pipermail/linuxppc-dev/2012-June/098586.html

Thanks for the reference.

Quoted 9 lines

Reading it again I'm not so sure that the problem is so much that we need a primary, as that somewhat bad things happen on non-primary buses, such as the possibility of assigning a zero BAR. Some hardware (including QEMU's PCI emulation) cares about this, though most doesn't. We only have one PCI bus under QEMU, so when we started picking an arbitrary bus to be primary, the problem went away because there was only one bus and therefore there was no non-primary bus.

Sorry, I am not sure what you mean. Do you mean that it will affect the resources assigned to a bridge when it is marked as a primary bus? I searched the code and didn't found anything related to this? Could you give me some hint? :-)

Thanks,
Kevin

Quoted 2 lines

-Scott

author
#20 lore raw

PATCH 2/3 powerpc/fsl-pci: fix the unreachable warning message

In reply to Kevin Hao

The (1ull << mem_log) is never greater than mem unless mem_log++;

---
 arch/powerpc/sysdev/fsl_pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/sysdev/fsl_pci.c +1 -1
index 028ac1f..3833c8f 100644
@@ -305,10 +305,10 @@ static void setup_pci_atmu(struct pci_controller *hose)
 	if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) {
 		/* Size window to exact size if power-of-two or one size up */
 		if ((1ull << mem_log) != mem) {
+			mem_log++;
 			if ((1ull << mem_log) > mem)
 				pr_info("%s: Setting PCI inbound window "
 					"greater than memory size\n", name);
-			mem_log++;
 		}
 
 		piwar |= ((mem_log - 1) & PIWAR_SZ_MASK);
Signature · 1 line
1.8.1.4
author
#21 lore raw

PATCH 3/3 powerpc/fsl-pci: enable SWIOTLB in function setup_pci_atmu

In reply to Kevin Hao

This function contains all the stuff we need to check if SWIOTLB should be enabled or not. So it is more convenient to enable the SWIOTLB here than later.

---
 arch/powerpc/sysdev/fsl_pci.c | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)
arch/powerpc/sysdev/fsl_pci.c +3 -19
index 3833c8f..793e64d 100644
@@ -381,7 +381,9 @@ static void setup_pci_atmu(struct pci_controller *hose)
 	}
 
 	if (hose->dma_window_size < mem) {
-#ifndef CONFIG_SWIOTLB
+#ifdef CONFIG_SWIOTLB
+		ppc_swiotlb_enable = 1;
+#else
 		pr_err("%s: ERROR: Memory size exceeds PCI ATMU ability to "
 			"map - enable CONFIG_SWIOTLB to avoid dma errors.\n",
 			 name);
@@ -934,28 +936,10 @@ static int fsl_pci_probe(struct platform_device *pdev)
 {
 	int ret;
 	struct device_node *node;
-#ifdef CONFIG_SWIOTLB
-	struct pci_controller *hose;
-#endif
 
 	node = pdev->dev.of_node;
 	ret = fsl_add_bridge(pdev, fsl_pci_primary == node);
 
-#ifdef CONFIG_SWIOTLB
-	if (ret == 0) {
-		hose = pci_find_hose_for_OF_device(pdev->dev.of_node);
-
-		/*
-		 * if we couldn't map all of DRAM via the dma windows
-		 * we need SWIOTLB to handle buffers located outside of
-		 * dma capable memory region
-		 */
-		if (memblock_end_of_DRAM() - 1 > hose->dma_window_base_cur +
-				hose->dma_window_size)
-			ppc_swiotlb_enable = 1;
-	}
-#endif
-
 	mpc85xx_pci_err_probe(pdev);
 
 	return 0;
Signature · 1 line
1.8.1.4