PCI: Always set prefetchable base/limit upper32 registers
PATCH v2 1/1 PCI: Always set prefetchable base/limit upper32 registers
Prior to 1f82de10 we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.
It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like 0xffffffffabc00000 - 0x00000000abc00000
Revert the chunk of 1f82de10 that made this conditional so we always write the upper32 registers and remove now unused pref_mem64 variable.
---
v2:
Remove pref_mem64 as it's now unused.
drivers/pci/setup-bus.c | 11 +++-------- 1 files changed, 3 insertions(+), 8 deletions(-)
drivers/pci/setup-bus.c +3 -8
@@ -142,7 +142,6 @@ static void pci_setup_bridge(struct pci_bus *bus)struct pci_dev *bridge = bus->self;struct pci_bus_region region;u32 l, bu, lu, io_upper16; -int pref_mem64;if (pci_is_enabled (bridge))return ;
@@ -198,7 +197,6 @@ static void pci_setup_bridge(struct pci_bus *bus)pci_write_config_dword (bridge,PCI_PREF_LIMIT_UPPER32 ,0 );/* Set up PREF base/limit. */ - pref_mem64 =0 ; bu = lu =0 ;pcibios_resource_to_bus (bridge, ®ion, bus->resource[2 ]);if (bus->resource[2 ]->flags &IORESOURCE_PREFETCH ) {
@@ -206,7 +204,6 @@ static void pci_setup_bridge(struct pci_bus *bus) l = (region.start >>16 ) &0xfff0 ; l |= region.end &0xfff00000 ;if (bus->resource[2 ]->flags &IORESOURCE_MEM_64 ) { - pref_mem64 =1 ; bu =upper_32_bits (region.start); lu =upper_32_bits (region.end); width =16 ;
@@ -221,11 +218,9 @@ static void pci_setup_bridge(struct pci_bus *bus) }pci_write_config_dword (bridge,PCI_PREF_MEMORY_BASE , l); -if (pref_mem64) { -/* Set the upper 32 bits of PREF base & limit. */ -pci_write_config_dword (bridge,PCI_PREF_BASE_UPPER32 , bu); -pci_write_config_dword (bridge,PCI_PREF_LIMIT_UPPER32 , lu); - } +/* Set the upper 32 bits of PREF base & limit. */ +pci_write_config_dword (bridge,PCI_PREF_BASE_UPPER32 , bu); +pci_write_config_dword (bridge,PCI_PREF_LIMIT_UPPER32 , lu);pci_write_config_word (bridge,PCI_BRIDGE_CONTROL , bus->bridge_ctl); }
Quoted 5 lines · Alex Williamson
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.
Quoted 8 lines
It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.Signed-off-by Alex Williamson <[email protected]>
thanks,
grant
Quoted 56 lines
---
v2:
Removepref_mem64as it's now unused.drivers/pci/setup-bus.c | 11 +++-------- 1 files changed, 3 insertions(+), 8 deletions(-)drivers/pci/setup-bus.c +3
-9@@ -142,7 +142,6 @@ static void pci_setup_bridge(struct pci_bus *bus)struct pci_dev *bridge = bus->self;struct pci_bus_region region;u32 l, bu, lu, io_upper16; -int pref_mem64;if (pci_is_enabled (bridge))return ;@@ -198,7 +197,6 @@ static void pci_setup_bridge(struct pci_bus *bus)pci_write_config_dword (bridge,PCI_PREF_LIMIT_UPPER32 ,0 );/* Set up PREF base/limit. */ - pref_mem64 =0 ; bu = lu =0 ;pcibios_resource_to_bus (bridge, ®ion, bus->resource[2 ]);if (bus->resource[2 ]->flags &IORESOURCE_PREFETCH ) {@@ -206,7 +204,6 @@ static void pci_setup_bridge(struct pci_bus *bus) l = (region.start >>16 ) &0xfff0 ; l |= region.end &0xfff00000 ;if (bus->resource[2 ]->flags &IORESOURCE_MEM_64 ) { - pref_mem64 =1 ; bu =upper_32_bits (region.start); lu =upper_32_bits (region.end); width =16 ;@@ -221,11 +218,9 @@ static void pci_setup_bridge(struct pci_bus *bus) }pci_write_config_dword (bridge,PCI_PREF_MEMORY_BASE , l); -if (pref_mem64) { -/* Set the upper 32 bits of PREF base & limit. */ -pci_write_config_dword (bridge,PCI_PREF_BASE_UPPER32 , bu); -pci_write_config_dword (bridge,PCI_PREF_LIMIT_UPPER32 , lu); - } +/* Set the upper 32 bits of PREF base & limit. */ +pci_write_config_dword (bridge,PCI_PREF_BASE_UPPER32 , bu); +pci_write_config_dword (bridge,PCI_PREF_LIMIT_UPPER32 , lu);pci_write_config_word (bridge,PCI_BRIDGE_CONTROL , bus->bridge_ctl); } --To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 10 lines · Grant Grundler
Quoted 9 lines · Alex Williamson
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000
Quoted 6 lines
Quoted 4 lines
Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.Signed-off-by Alex Williamson <[email protected]>Reviewed-by Grant Grundler <[email protected]>
NAK this - I messed up. Yinghai is correct. Something else is going on.
It might be perfectly OK to read 0xffffffffabc00000 if the bridge isn't using the upper32 Prefetchable register. Maybe the problem is some code is reading the upper32 value without checking that it's valid?
grant
Quoted 14 lines · Grant Grundler
Quoted 13 lines · Grant Grundler
Quoted 12 lines · Alex Williamson
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
Signed-off-by Alex Williamson <[email protected]>Reviewed-by Grant Grundler <[email protected]>NAK this - I messed up. Yinghai is correct. Something else is going on.
It might be perfectly OK to read
0xffffffffabc00000if the bridge isn't using the upper32 Prefetchable register. Maybe the problem is some code is reading the upper32 value without checking that it's valid?
maybe Alex is using old lspci?
YH
Quoted 16 lines · Grant Grundler
Quoted 15 lines · Grant Grundler
Quoted 13 lines · Alex Williamson
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.Signed-off-by Alex Williamson <[email protected]>
Quoted 7 lines
Quoted 1 line
Reviewed-by Grant Grundler <[email protected]>NAK this - I messed up. Yinghai is correct. Something else is going on.
It might be perfectly OK to read
0xffffffffabc00000if the bridge isn't using the upper32 Prefetchable register. Maybe the problem is some code is reading the upper32 value without checking that it's valid?
Apologies for not threading the v2 patch into the original thread. The prefetchable base register does support the upper32 bits and it does work correctly. However per the pci-to-pci bridge spec, a little lower on page 47, devices only supporting 32bit prefetchable ranges are to implement the upper32 registers as read-only registers that return zero. In the example above, -1 in the upper32 base simply means that base > limit, which disables the range.
Further investigation shows that the MEM_64 resource flag is setup for this range based on hardware capabilities, but then it gets removed in pbus_size_mem() because we want to use the range to map a 32bit option ROM. This leaves us entering pci_setup_bridge() with -1 in the upper32 base and the MEM_64 flag clear, so we never touch the upper32 base register. I think this patch is still a simple, safe solution. Thanks,
Alex
Grant Grundler
Reviewed-byQuoted 33 lines · Alex Williamson
Quoted 16 lines · Grant Grundler
Quoted 15 lines · Grant Grundler
Quoted 13 lines · Alex Williamson
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.Signed-off-by Alex Williamson <[email protected]>Quoted 7 lines
Quoted 1 line
Reviewed-by Grant Grundler <[email protected]>NAK this - I messed up. Yinghai is correct. Something else is going on.
It might be perfectly OK to read
0xffffffffabc00000if the bridge isn't using the upper32 Prefetchable register. Maybe the problem is some code is reading the upper32 value without checking that it's valid?Apologies for not threading the v2 patch into the original thread. The prefetchable base register does support the upper32 bits and it does work correctly. However per the pci-to-pci bridge spec, a little lower on page 47, devices only supporting 32bit prefetchable ranges are to implement the upper32 registers as read-only registers that return zero. In the example above, -1 in the upper32 base simply means that base > limit, which disables the range.
Quoted 6 lines
Further investigation shows that the
MEM_64resource flag is setup for this range based on hardware capabilities, but then it gets removed inpbus_size_mem()because we want to use the range to map a 32bit option ROM. This leaves us enteringpci_setup_bridge()with -1 in the upper32 base and theMEM_64flag clear, so we never touch the upper32 base register. I think this patch is still a simple, safe solution. Thanks,
Yup - after reading the PCI-PCI spec a 3rd time. I have to agree. Alex, sorry for the flip flopping. Pre-2.6.30 code was clearly working. Please add:
I assumed Yinghai's objection was based on a specific problem he had seen with writing upper32 register. Bjorn asked the right question. If there isn't a specific problem, I'd prefer AW's simpler patch.
I'm also thinking the resource allocation design which uses resource flags to indicate resources assigned (e.g a resource is 32-bit) rather than HW attributes is broken. We should be able to allocate 32-bit Option ROM into a 64-bit prefetchable MMIO window that is programmed with upper32 as zeros without changing the resource type. The resource allocation code only be looking at Resource "Type" when (re)programming window registers. The rest of the time (programming BARs) should be able to just test "if it fits".
cheers,
grant
Quoted 38 lines · Grant Grundler
Quoted 30 lines · Alex Williamson
Quoted 14 lines · Grant Grundler
Quoted 13 lines · Grant Grundler
Quoted 12 lines · Alex Williamson
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.Quoted 7 lines
Quoted 2 lines
Quoted 1 line
Signed-off-by Alex Williamson <[email protected]>Reviewed-by Grant Grundler <[email protected]>NAK this - I messed up. Yinghai is correct. Something else is going on.
It might be perfectly OK to read
0xffffffffabc00000if the bridge isn't using the upper32 Prefetchable register. Maybe the problem is some code is reading the upper32 value without checking that it's valid?Apologies for not threading the v2 patch into the original thread. The prefetchable base register does support the upper32 bits and it does work correctly. However per the pci-to-pci bridge spec, a little lower on page 47, devices only supporting 32bit prefetchable ranges are to implement the upper32 registers as read-only registers that return zero. In the example above, -1 in the upper32 base simply means that base > limit, which disables the range.
Quoted 6 lines
Further investigation shows that the
MEM_64resource flag is setup for this range based on hardware capabilities, but then it gets removed inpbus_size_mem()because we want to use the range to map a 32bit option ROM. This leaves us enteringpci_setup_bridge()with -1 in the upper32 base and theMEM_64flag clear, so we never touch the upper32 base register. I think this patch is still a simple, safe solution. Thanks,
Quoted 8 lines
Yup - after reading the PCI-PCI spec a 3rd time. I have to agree. Alex, sorry for the flip flopping. Pre-2.6.30 code was clearly working. Please add:
Reviewed-by Grant Grundler <[email protected]>I assumed Yinghai's objection was based on a specific problem he had seen with writing upper32 register. Bjorn asked the right question. If there isn't a specific problem, I'd prefer AW's simpler patch.
we just should not touch that register if the HW only support 32bit pref mmio.
Quoted 9 lines
I'm also thinking the resource allocation design which uses resource flags to indicate resources assigned (e.g a resource is 32-bit) rather than HW attributes is broken. We should be able to allocate 32-bit Option ROM into a 64-bit prefetchable MMIO window that is programmed with upper32 as zeros without changing the resource type. The resource allocation code only be looking at Resource "Type" when (re)programming window registers. The rest of the time (programming BARs) should be able to just test "if it fits".
IORESOURCE_MEM_64 is the flags that the resource could be assigned to >4g range. so it is NOT assigned resource ...
YH
On Tue, Dec 01, 2009 at 12:40:03PM -0800, Yinghai Lu wrote:
...
Quoted 5 lines
Quoted 3 lines
I assumed Yinghai's objection was based on a specific problem he had seen with writing upper32 register. Bjorn asked the right question. If there isn't a specific problem, I'd prefer AW's simpler patch.
we just should not touch that register if the HW only support 32bit pref mmio.
Why not?
I agree the PCI-PCI spec defines how to determine if a PCI Bridge supports 64-bit Pref MMIO (using upper32 - or not). But spec also doesn't prohibit writing to a read-only register. Writing this Read-Only register so far hasn't caused any problems.
Quoted 11 lines
Quoted 8 lines
I'm also thinking the resource allocation design which uses resource flags to indicate resources assigned (e.g a resource is 32-bit) rather than HW attributes is broken. We should be able to allocate 32-bit Option ROM into a 64-bit prefetchable MMIO window that is programmed with upper32 as zeros without changing the resource type. The resource allocation code only be looking at Resource "Type" when (re)programming window registers. The rest of the time (programming BARs) should be able to just test "if it fits".
IORESOURCE_MEM_64is the flags that the resource could be assigned to >4g range. so it is NOT assigned resource ...
*shrug* I don't have time to work on the broader issue.
thanks,
grant
Quoted 5 lines · Grant Grundler
On Tue, Dec 01, 2009 at 12:40:03PM -0800, Yinghai Lu wrote:
...Quoted 3 lines
Quoted 3 lines
I assumed Yinghai's objection was based on a specific problem he had seen with writing upper32 register. Bjorn asked the right question. If there isn't a specific problem, I'd prefer AW's simpler patch.
Quoted 8 lines
Quoted 1 line
we just should not touch that register if the HW only support 32bit pref mmio.
Why not?
I agree the PCI-PCI spec defines how to determine if a PCI Bridge supports 64-bit Pref MMIO (using upper32 - or not). But spec also doesn't prohibit writing to a read-only register. Writing this Read-Only register so far hasn't caused any problems.
if we can find out that is 32bit mmio pref, why waste cycle to write value to them?
YH
Quoted 9 lines · On Mon, 30 Nov 2009 14:51:44 -0700 Alex Williamson <[email protected]>…
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000
Quoted 6 lines
Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.Signed-off-by Alex Williamson <[email protected]>---
Applied this one, though it looks like your diff had some context that wasn't in my tree. Please double check that I fixed it up correctly.
Thanks,
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
Quoted 15 lines · Jesse Barnes
Quoted 12 lines · On Mon, 30 Nov 2009 14:51:44 -0700 Alex Williamson <[email protected]>…
Prior to
1f82de10we always initialized the upper 32bits of the prefetchable memory window, regardless of the address range used. Now we only touch it for a >32bit address, which means the upper32 registers remain whatever the BIOS initialized them too.It's valid for the BIOS to set the upper32 base/limit to 0xffffffff/0x00000000, which makes us program prefetchable ranges like
0xffffffffabc00000-0x00000000abc00000Revert the chunk of
1f82de10that made this conditional so we always write the upper32 registers and remove now unusedpref_mem64variable.
Quoted 6 lines
Quoted 3 lines
Signed-off-by Alex Williamson <[email protected]>---
Applied this one, though it looks like your diff had some context that wasn't in my tree. Please double check that I fixed it up correctly.
Yep, looks fine. Thanks Jesse.
Alex