PCI: Double removing fix and allocate 64bit mmio pref
Subsystems (3)
v3 cover PCI: Double removing fix and allocate 64bit mmio pref
First 4 are for Gu Zheng <[email protected]> to help double pci device removing via sysfs.
For patch 4, rafael like to have use lock instead of atomic.
Patch 6 are for not enabling realloc auto when 64bit mmio is not there for root bus.
Others are about mmio 64 allocation that could help Guo Chao <[email protected]> on powerpc mmio allocation. It will try to assign 64 bit resource above 4g at first.
Patch 1-4 and Patch 5-12 could be appliey seperatedly.
Could be found:
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-3.14
And it is based on current pci/for-linus.
-v2: update after patch that move device_del down to pci_destroy_dev.
add "Try best to allocate pref mmio 64bit above 4G"
-v3: refresh and send out after pci_clip_resource() changes,
as Bjorn is not happy with attachments.
Yinghai Lu (12):
PCI: Use device_release_driver in pci_stop_root_bus
PCI: Move back pci_proc_attach_devices calling
PCI: Move resources and bus_list releasing to pci_release_dev
PCI: Destroy pci dev only once
PCI: pcibus address to resource converting take bus instead of dev
PCI: Only enable realloc auto when root bus has 64bit mmio
PCI: Don't use 4G bus address directly in resource allocation
PCI: Try to allocate mem64 above 4G at first
PCI: Kill PCIBIOS_MAX_MEM_32
PCI: Try best to allocate pref mmio 64bit above 4g
PCI: Sort pci root bus resources list
intel-gtt: Read 64bit for gmar_bus_addr
arch/x86/include/asm/pci.h | 1 - drivers/char/agp/intel-gtt.c | 14 ++-- drivers/pci/bus.c | 74 ++++++++++++++++--- drivers/pci/host-bridge.c | 34 +++++---- drivers/pci/probe.c | 23 +++++- drivers/pci/remove.c | 31 ++------ drivers/pci/setup-bus.c | 165 ++++++++++++++++++++++++++++++------------- drivers/pci/setup-res.c | 14 +++- include/linux/pci.h | 10 +-- 9 files changed, 257 insertions(+), 109 deletions(-)
Signature · 1 line
1.8.1.4
PATCH v3 1/12 PCI: Use device_release_driver in pci_stop_root_bus
To be consistent with change in
| PCI: Move device_del() from pci_stop_dev() to pci_destroy_dev()
Use device_release_driver for root bus/hostbridge.
Also use device_unregister() in pci_remove_root_bus() instead of device_del/put_device, that will be corresponding device_register() for pci_create_root_bus for hostbridge.
--- drivers/pci/remove.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/remove.c +2 -2
@@ -128,7 +128,7 @@ void pci_stop_root_bus(struct pci_bus *bus)pci_stop_bus_device (child);/* stop the host bridge */ -device_del (&host_bridge->dev); +device_release_driver (&host_bridge->dev); }void pci_remove_root_bus (struct pci_bus *bus)
@@ -147,5 +147,5 @@ void pci_remove_root_bus(struct pci_bus *bus) host_bridge->bus =NULL ;/* remove the host bridge */ -put_device (&host_bridge->dev); +device_unregister (&host_bridge->dev); }
Signature · 1 line
1.8.1.4
PATCH v3 2/12 PCI: Move back pci_proc_attach_devices calling
In reply to Yinghai LuWe stop detach proc when pci_stop_device.
So should attach that during pci_bus_add_device.
--- drivers/pci/bus.c | 1 + drivers/pci/probe.c | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-)
drivers/pci/bus.c +1 -0
@@ -175,6 +175,7 @@ int pci_bus_add_device(struct pci_dev *dev)* are not assigned yet for some devices. */ pci_fixup_device (pci_fixup_final, dev); +pci_proc_attach_device (dev);pci_create_sysfs_dev_files (dev); dev->match_driver = true;
drivers/pci/probe.c +0 -2
@@ -1381,8 +1381,6 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) dev->match_driver = false; ret =device_add (&dev->dev);WARN_ON (ret <0 ); - -pci_proc_attach_device (dev); }struct pci_dev *__ref pci_scan_single_device (struct pci_bus *bus,int devfn)
Signature · 1 line
1.8.1.4
PATCH v3 3/12 PCI: Move resources and bus_list releasing to pci_release_dev
In reply to Yinghai LuWe should not release resource in pci_destroy that is too early as there could be still other use hold reference.
release them or remove it from bus devices list at last in pci_release_dev instead.
--- drivers/pci/probe.c | 21 +++++++++++++++++++-- drivers/pci/remove.c | 19 ------------------- 2 files changed, 19 insertions(+), 21 deletions(-)
drivers/pci/probe.c +19 -2
@@ -1154,6 +1154,18 @@ static void pci_release_capabilities(struct pci_dev *dev)pci_free_cap_save_buffers (dev); } +static void pci_free_resources (struct pci_dev *dev) +{ +int i; + +pci_cleanup_rom (dev); +for (i =0 ; i <PCI_NUM_RESOURCES ; i++) { +struct resource *res = dev->resource + i; +if (res->parent) +release_resource (res); + } +} +/** * pci_release_dev - free a pci device structure when all users of it are finished. * @dev: device that's been disconnected
@@ -1163,9 +1175,14 @@ static void pci_release_capabilities(struct pci_dev *dev)*/ static void pci_release_dev (struct device *dev) { -struct pci_dev *pci_dev; +struct pci_dev *pci_dev =to_pci_dev (dev); + +down_write (&pci_bus_sem); +list_del (&pci_dev->bus_list); +up_write (&pci_bus_sem); + +pci_free_resources (pci_dev); - pci_dev =to_pci_dev (dev);pci_release_capabilities (pci_dev);pci_release_of_node (pci_dev);pcibios_release_device (pci_dev);
drivers/pci/remove.c +0 -19
@@ -3,20 +3,6 @@#include <linux/pci-aspm.h> #include "pci.h" -static void pci_free_resources (struct pci_dev *dev) -{ -int i; - -msi_remove_pci_irq_vectors (dev); - -pci_cleanup_rom (dev); -for (i =0 ; i <PCI_NUM_RESOURCES ; i++) { -struct resource *res = dev->resource + i; -if (res->parent) -release_resource (res); - } -} -static void pci_stop_dev (struct pci_dev *dev) {pci_pme_active (dev, false);
@@ -36,11 +22,6 @@ static void pci_destroy_dev(struct pci_dev *dev) {device_del (&dev->dev); -down_write (&pci_bus_sem); -list_del (&dev->bus_list); -up_write (&pci_bus_sem); - -pci_free_resources (dev);put_device (&dev->dev); }
Signature · 1 line
1.8.1.4
PATCH v3 4/12 PCI: Destroy pci dev only once
In reply to Yinghai LuMutliple removing via /sys will call pci_destroy_dev two times.
| When concurent removing pci devices which are in the same pci subtree | via sysfs, such as: | echo -n 1 > /sys/bus/pci/devices/0000\:10\:00.0/remove ; echo -n 1 > | /sys/bus/pci/devices/0000\:1a\:01.0/remove | (1a:01.0 device is downstream from the 10:00.0 bridge) | | the following warning will show: | [ 1799.280918] ------------[ cut here ]------------ | [ 1799.336199] WARNING: CPU: 7 PID: 126 at lib/list_debug.c:53 __list_del_entry+0x63/0xd0() | [ 1799.433093] list_del corruption, ffff8807b4a7c000->next is LIST_POISON1 (dead000000100100) | [ 1800.276623] CPU: 7 PID: 126 Comm: kworker/u512:1 Tainted: G W 3.12.0-rc5+ #196 | [ 1800.508918] Workqueue: sysfsd sysfs_schedule_callback_work | [ 1800.574703] 0000000000000009 ffff8807adbadbd8 ffffffff8168b26c ffff8807c27d08a8 | [ 1800.663860] ffff8807adbadc28 ffff8807adbadc18 ffffffff810711dc ffff8807adbadc68 | [ 1800.753130] ffff8807b4a7c000 ffff8807b4a7c000 ffff8807ad089c00 0000000000000000 | [ 1800.842282] Call Trace: | [ 1800.871651] [<ffffffff8168b26c>] dump_stack+0x55/0x76 | [ 1800.933301] [<ffffffff810711dc>] warn_slowpath_common+0x8c/0xc0 | [ 1801.005283] [<ffffffff810712c6>] warn_slowpath_fmt+0x46/0x50 | [ 1801.074081] [<ffffffff8135a343>] __list_del_entry+0x63/0xd0 | [ 1801.141839] [<ffffffff8135a3c1>] list_del+0x11/0x40 | [ 1801.201320] [<ffffffff813734da>] pci_remove_bus_device+0x6a/0xe0 | [ 1801.274279] [<ffffffff8137356e>] pci_stop_and_remove_bus_device+0x1e/0x30 | [ 1801.356606] [<ffffffff8137b20b>] remove_callback+0x2b/0x40 | [ 1801.423412] [<ffffffff81251848>] sysfs_schedule_callback_work+0x18/0x60 | [ 1801.503744] [<ffffffff8108eab5>] process_one_work+0x1f5/0x540 | [ 1801.573640] [<ffffffff8108ea53>] ? process_one_work+0x193/0x540 | [ 1801.645616] [<ffffffff8108f2ac>] worker_thread+0x11c/0x370 | [ 1801.712337] [<ffffffff8108f190>] ? rescuer_thread+0x350/0x350 | [ 1801.782178] [<ffffffff8109731d>] kthread+0xed/0x100 | [ 1801.841661] [<ffffffff81097230>] ? kthread_create_on_node+0x160/0x160 | [ 1801.919919] [<ffffffff8169cc3c>] ret_from_fork+0x7c/0xb0 | [ 1801.984608] [<ffffffff81097230>] ? kthread_create_on_node+0x160/0x160 | [ 1802.062825] ---[ end trace d77f2054de000fb7 ]--- | | This issue is related to the bug 54411: | https://bugzilla.kernel.org/show_bug.cgi?id=54411
Add is_removed to record if pci_destroy_dev is called already.
During second calling, still have extra dev ref hold via device_schedule_call, so we are safe to check dev->is_removed.
It fixes the problem In Gu's test.
-v2: add partial changelog from Gu Zheng <[email protected]>
refresh after patch of moving device_del from Rafael.
-v3: use atomic operations to prevent racing that Rafael and Bjorn concern.
--- drivers/pci/probe.c | 2 ++ drivers/pci/remove.c | 8 +++++--- include/linux/pci.h | 1 + 3 files changed, 8 insertions(+), 3 deletions(-)
drivers/pci/probe.c +2 -0
@@ -1398,6 +1398,8 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) dev->match_driver = false; ret =device_add (&dev->dev);WARN_ON (ret <0 ); + +atomic_set (&dev->removed_count,-1 ); }struct pci_dev *__ref pci_scan_single_device (struct pci_bus *bus,int devfn)
drivers/pci/remove.c +5 -3
@@ -20,9 +20,11 @@ static void pci_stop_dev(struct pci_dev *dev)static void pci_destroy_dev (struct pci_dev *dev) { -device_del (&dev->dev); - -put_device (&dev->dev); +if (atomic_inc_and_test (&dev->removed_count)) { +device_del (&dev->dev); +put_device (&dev->dev); + }else +atomic_dec (&dev->removed_count); }void pci_remove_bus (struct pci_bus *bus)
include/linux/pci.h +1 -0
@@ -316,6 +316,7 @@ struct pci_dev {struct resource resource[DEVICE_COUNT_RESOURCE ];/* I/O and memory regions + expansion ROMs */ bool match_driver;/* Skip attaching driver */ +atomic_t removed_count;/* pci_destroy_dev is called */ /* These fields are used by common fixups */ unsigned int transparent:1; /* Subtractive decode PCI bridge */ unsigned int multifunction:1;/* Part of multi-function device */
Signature · 1 line
1.8.1.4
Re: [PATCH v3 04/12] PCI: Destroy pci dev only once
Quoted 47 lines · Yinghai Lu
Mutliple removing via /sys will call
pci_destroy_devtwo times.| When concurent removing pci devices which are in the same pci subtree | via sysfs, such as: | echo -n 1 > /sys/bus/pci/devices/0000\:10\:00.0/remove ; echo -n 1 > | /sys/bus/pci/devices/0000\:1a\:01.0/remove | (1a:01.0 device is downstream from the 10:00.0 bridge) | | the following warning will show: | [ 1799.280918] ------------[ cut here ]------------ | [ 1799.336199] WARNING: CPU: 7 PID: 126 at lib/list_debug.c:53 __list_del_entry+0x63/0xd0() | [ 1799.433093] list_del corruption, ffff8807b4a7c000->next is LIST_POISON1 (dead000000100100) | [ 1800.276623] CPU: 7 PID: 126 Comm: kworker/u512:1 Tainted: G W 3.12.0-rc5+ #196 | [ 1800.508918] Workqueue: sysfsd sysfs_schedule_callback_work | [ 1800.574703] 0000000000000009 ffff8807adbadbd8 ffffffff8168b26c ffff8807c27d08a8 | [ 1800.663860] ffff8807adbadc28 ffff8807adbadc18 ffffffff810711dc ffff8807adbadc68 | [ 1800.753130] ffff8807b4a7c000 ffff8807b4a7c000 ffff8807ad089c00 0000000000000000 | [ 1800.842282] Call Trace: | [ 1800.871651] [<ffffffff8168b26c>] dump_stack+0x55/0x76 | [ 1800.933301] [<ffffffff810711dc>] warn_slowpath_common+0x8c/0xc0 | [ 1801.005283] [<ffffffff810712c6>] warn_slowpath_fmt+0x46/0x50 | [ 1801.074081] [<ffffffff8135a343>] __list_del_entry+0x63/0xd0 | [ 1801.141839] [<ffffffff8135a3c1>] list_del+0x11/0x40 | [ 1801.201320] [<ffffffff813734da>] pci_remove_bus_device+0x6a/0xe0 | [ 1801.274279] [<ffffffff8137356e>] pci_stop_and_remove_bus_device+0x1e/0x30 | [ 1801.356606] [<ffffffff8137b20b>] remove_callback+0x2b/0x40 | [ 1801.423412] [<ffffffff81251848>] sysfs_schedule_callback_work+0x18/0x60 | [ 1801.503744] [<ffffffff8108eab5>] process_one_work+0x1f5/0x540 | [ 1801.573640] [<ffffffff8108ea53>] ? process_one_work+0x193/0x540 | [ 1801.645616] [<ffffffff8108f2ac>] worker_thread+0x11c/0x370 | [ 1801.712337] [<ffffffff8108f190>] ? rescuer_thread+0x350/0x350 | [ 1801.782178] [<ffffffff8109731d>] kthread+0xed/0x100 | [ 1801.841661] [<ffffffff81097230>] ? kthread_create_on_node+0x160/0x160 | [ 1801.919919] [<ffffffff8169cc3c>] ret_from_fork+0x7c/0xb0 | [ 1801.984608] [<ffffffff81097230>] ? kthread_create_on_node+0x160/0x160 | [ 1802.062825] ---[ end trace d77f2054de000fb7 ]--- | | This issue is related to the bug 54411: | https://bugzilla.kernel.org/show_bug.cgi?id=54411Add
is_removedto record ifpci_destroy_devis called already.During second calling, still have extra dev ref hold via
device_schedule_call, so we are safe to checkdev->is_removed.It fixes the problem In Gu's test.
Review
Quoted 3 lines
(quoted diff) +0
-2-v2: add partial changelog from Gu Zheng <[email protected]> refresh after patch of moving device_del from Rafael. -v3: use atomic operations to prevent racing that Rafael and Bjorn concern.
And this is not enough as I said, can you please stop ignoring me openly?
You've already commented the last patch I posted, so why don't we finish that discussion first?
Rafael
Re: [PATCH v3 04/12] PCI: Destroy pci dev only once
Quoted 6 lines · Rafael J. Wysocki
On Saturday, November 30, 2013 02:40:30 PM Yinghai Lu wrote:
And this is not enough as I said, can you please stop ignoring me openly?
You've already commented the last patch I posted, so why don't we finish that discussion first?
No, i did not mean that.
I resent the whole patchset as I added pci_clip_resource() and that will need to revise several critial patches, and Bjorn does not like to have patches to be attached format.
Also in the patcheset head mail, i did state that you have another solution using
lock.
And your patch is quite big, please give me some time to digest it.
Thanks
Yinghai
Re: [PATCH v3 04/12] PCI: Destroy pci dev only once
Quoted 5 lines · Yinghai Lu
Quoted 4 lines · Rafael J. Wysocki
On Saturday, November 30, 2013 02:40:30 PM Yinghai Lu wrote:
And this is not enough as I said, can you please stop ignoring me openly?
Quoted 8 lines
Quoted 2 lines
You've already commented the last patch I posted, so why don't we finish that discussion first?
No, i did not mean that.
I resent the whole patchset as I added
pci_clip_resource()and that will need to revise several critial patches, and Bjorn does not like to have patches to be attached format.
I see.
Quoted 5 lines
Also in the patcheset head mail, i did state that you have another solution using
lock.And your patch is quite big, please give me some time to digest it.
Yes, it is, please take your time. :-)
Thanks,
Rafael
PATCH v3 5/12 PCI: pcibus address to resource converting take bus instead of dev
In reply to Yinghai LuFor allocating resource under bus path, we do not have dev to pass along, and we only have bus to use instead.
-v2: drop pcibios_bus_addr_to_resource().
--- drivers/pci/host-bridge.c | 34 +++++++++++++++++++++------------- include/linux/pci.h | 5 +++++ 2 files changed, 26 insertions(+), 13 deletions(-)
drivers/pci/host-bridge.c +21 -13
@@ -9,22 +9,19 @@#include "pci.h" -static struct pci_bus *find_pci_root_bus (struct pci_dev *dev) +static struct pci_bus *find_pci_root_bus (struct pci_bus *bus) { -struct pci_bus *bus; - - bus = dev->bus;while (bus->parent) bus = bus->parent;return bus; } -static struct pci_host_bridge *find_pci_host_bridge (struct pci_dev *dev) +static struct pci_host_bridge *find_pci_host_bridge (struct pci_bus *bus) { -struct pci_bus *bus =find_pci_root_bus (dev); +struct pci_bus *root_bus =find_pci_root_bus (bus); -return to_pci_host_bridge (bus->bridge); +return to_pci_host_bridge (root_bus->bridge); } void pci_set_host_bridge_release(struct pci_host_bridge *bridge,
@@ -40,10 +37,11 @@ static bool resource_contains(struct resource *res1, struct resource *res2)return res1->start <= res2->start && res1->end >= res2->end; } -void pcibios_resource_to_bus (struct pci_dev *dev,struct pci_bus_region *region, -struct resource *res) +void __pcibios_resource_to_bus (struct pci_bus *bus, +struct pci_bus_region *region, +struct resource *res) { -struct pci_host_bridge *bridge =find_pci_host_bridge (dev); +struct pci_host_bridge *bridge =find_pci_host_bridge (bus);struct pci_host_bridge_window *window;resource_size_t offset =0 ;
@@ -60,6 +58,11 @@ void pcibios_resource_to_bus(struct pci_dev *dev, struct pci_bus_region *region, region->start = res->start - offset; region->end = res->end - offset; } +void pcibios_resource_to_bus (struct pci_dev *dev,struct pci_bus_region *region, +struct resource *res) +{ +__pcibios_resource_to_bus (dev->bus, region, res); +}EXPORT_SYMBOL (pcibios_resource_to_bus); static bool region_contains(struct pci_bus_region *region1,
@@ -68,10 +71,10 @@ static bool region_contains(struct pci_bus_region *region1,return region1->start <= region2->start && region1->end >= region2->end; } -void pcibios_bus_to_resource (struct pci_dev *dev,struct resource *res, -struct pci_bus_region *region) +void __pcibios_bus_to_resource (struct pci_bus *bus,struct resource *res, +struct pci_bus_region *region) { -struct pci_host_bridge *bridge =find_pci_host_bridge (dev); +struct pci_host_bridge *bridge =find_pci_host_bridge (bus);struct pci_host_bridge_window *window;resource_size_t offset =0 ;
@@ -93,4 +96,9 @@ void pcibios_bus_to_resource(struct pci_dev *dev, struct resource *res, res->start = region->start + offset; res->end = region->end + offset; } +void pcibios_bus_to_resource (struct pci_dev *dev,struct resource *res, +struct pci_bus_region *region) +{ +__pcibios_bus_to_resource (dev->bus, res, region); +}EXPORT_SYMBOL (pcibios_bus_to_resource);
include/linux/pci.h +5 -0
@@ -738,8 +738,13 @@ void pci_fixup_cardbus(struct pci_bus *);/* Generic PCI functions used internally */ +void __pcibios_resource_to_bus (struct pci_bus *bus, +struct pci_bus_region *region, +struct resource *res);void pcibios_resource_to_bus (struct pci_dev *dev,struct pci_bus_region *region,struct resource *res); +void __pcibios_bus_to_resource (struct pci_bus *bus,struct resource *res, +struct pci_bus_region *region);void pcibios_bus_to_resource (struct pci_dev *dev,struct resource *res,struct pci_bus_region *region);void pcibios_scan_specific_bus (int busn);
Signature · 1 line
1.8.1.4
PATCH v3 6/12 PCI: Only enable realloc auto when root bus has 64bit mmio
In reply to Yinghai LuJoseph found
| commit b07f2ebc109b607789f648dedcff4b125f9afec6 | Date: Thu Feb 23 19:23:32 2012 -0800 | | PCI: add a PCI resource reallocation config option
cause one system can not load driver for Intel x520 NIC's.
The root resource:
[ 1.212470] PCI host bridge to bus 0000:20 [ 1.212475] pci_bus 0000:20: root bus resource [bus 20-3e] [ 1.212479] pci_bus 0000:20: root bus resource [io 0xc000-0xdfff] [ 1.212483] pci_bus 0000:20: root bus resource [mem 0xfecc0000-0xfecfffff] [ 1.212487] pci_bus 0000:20: root bus resource [mem 0xe9400000-0xe97fffff]
and bios does not assign sriov, also have two function ROM bar point to same position.
[ 1.213197] pci 0000:22:00.0: [8086:10fb] type 00 class 0x020000 ... [ 1.213240] pci 0000:22:00.0: reg 0x30: [mem 0xe9500000-0xe957ffff pref] [ 1.213303] pci 0000:22:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit] [ 1.213317] pci 0000:22:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit] [ 1.213366] pci 0000:22:00.1: [8086:10fb] type 00 class 0x020000 ... [ 1.213408] pci 0000:22:00.1: reg 0x30: [mem 0xe9500000-0xe957ffff pref] [ 1.213468] pci 0000:22:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit] [ 1.213481] pci 0000:22:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit] [ 1.218527] pci 0000:20:03.0: PCI bridge to [bus 22] [ 1.218534] pci 0000:20:03.0: bridge window [io 0xd000-0xdfff] [ 1.218537] pci 0000:20:03.0: bridge window [mem 0xe9400000-0xe95fffff] ... [ 1.254103] pci 0000:22:00.1: address space collision: [mem 0xe9500000-0xe957ffff pref] conflicts with 0000:22:00.0 [mem 0xe9500000-0xe957ffff pref] [ 1.254111] pci 0000:23:00.1: address space collision: [mem 0xe9700000-0xe977ffff pref] conflicts with 0000:23:00.0 [mem 0xe9700000-0xe977ffff pref]
We don't need to enable realloc for this case, as we can not alter root bus mmio range to get big one to hold two rom bar, and sriov under 4G.
Add checking if pci root bus have 4G above mmio res, and don't enable realloc auto accordingly.
bug report at: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1245938
--- drivers/pci/setup-bus.c | 32 +++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-)
drivers/pci/setup-bus.c +27 -5
@@ -1432,17 +1432,39 @@ static int iov_resources_unassigned(struct pci_dev *dev, void *data)return 0 ; } +static bool pci_bus_mem_above_4g (struct pci_bus *bus) +{ +int i; +struct resource *res; + +pci_bus_for_each_resource (bus, res, i) { +struct pci_bus_region region; + +if (!res || !(res->flags &IORESOURCE_MEM )) +continue ; + +__pcibios_resource_to_bus (bus, ®ion, res); +if (region.end >0xffffffff ) +return true; + } + +return false; +} +static enum enable_type pci_realloc_detect (struct pci_bus *bus,enum enable_type enable_local) { -bool unassigned = false; -if (enable_local != undefined)return enable_local; -pci_walk_bus (bus, iov_resources_unassigned, &unassigned); -if (unassigned) -return auto_enabled; +/* only enable auto when root bus does support 64bit mmio */ +if (pci_bus_mem_above_4g (bus)) { +bool unassigned = false; + +pci_walk_bus (bus, iov_resources_unassigned, &unassigned); +if (unassigned) +return auto_enabled; + }return enable_local; }
Signature · 1 line
1.8.1.4
PATCH v3 7/12 PCI: Don't use 4G bus address directly in resource allocation
In reply to Yinghai LuCurrent we are using PCIBIOS_MAX_MEM_32 (4G limit) directly in the pci_bus_alloc_resource to make sure that don't allocate too high pref 64bit above 4G in the system that does not support that.
That is not right, as allocate_resource() should take resource limit.
Add pci_clip_resource() and use it check the pci bus address limit.
--- drivers/pci/bus.c | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-)
drivers/pci/bus.c +34 -7
@@ -98,6 +98,25 @@ void pci_bus_remove_resources(struct pci_bus *bus) } } +static struct pci_bus_region pci_mem_32 = {0 ,0xffffffff }; + +static void pci_clip_resource (struct resource *res,struct pci_bus *bus, +struct pci_bus_region *region) +{ +struct pci_bus_region r; + +__pcibios_resource_to_bus (bus, &r, res); +if (r.start < region->start) + r.start = region->start; +if (r.end > region->end) + r.end = region->end; + +if (r.end < r.start) + res->end = res->start -1 ; +else +__pcibios_bus_to_resource (bus, res, &r); +} +/** * pci_bus_alloc_resource - allocate a resource from a parent bus * @bus: PCI bus
@@ -125,15 +144,12 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, {int i, ret = -ENOMEM ;struct resource *r; -resource_size_t max =-1 ; type_mask |=IORESOURCE_IO |IORESOURCE_MEM ; -/* don't allocate too high if the pref mem doesn't support 64bit*/ -if (!(res->flags &IORESOURCE_MEM_64 )) - max =PCIBIOS_MAX_MEM_32 ; -pci_bus_for_each_resource (bus, r, i) { +struct resource avail; +if (!r)continue ;
@@ -147,10 +163,21 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, !(res->flags & IORESOURCE_PREFETCH))continue ; +/* +* don't allocate too high if the pref mem doesn't +* support 64bit. +*/ + avail = *r; +if (!(res->flags &IORESOURCE_MEM_64 )) { +pci_clip_resource (&avail, bus, &pci_mem_32); +if (!resource_size (&avail)) +continue ; + } +/* Ok, try it out.. */ ret =allocate_resource (r, res, size, - r->start ? : min, - max, align, +max (avail.start, r->start ? : min), + avail.end, align, alignf, alignf_data);if (ret ==0 )break ;
Signature · 1 line
1.8.1.4
PATCH v3 8/12 PCI: Try to allocate mem64 above 4G at first
In reply to Yinghai LuOn system with more pcie cards, we do not have enough range under 4G to allocate those pci devices.
On 64bit system, we could try to allocate mem64 above 4G at first, and fall back to below 4g if it can not find any above 4g.
x86 32bit without X86_PAE support will have bottom set to 0, because resource_size_t is 32bit.
For 32bit kernel that resource_size_t is 64bit when pae is support. we are safe because iomem_resource is limited to 32bit according to x86_phys_bits.
-v2: update bottom assigning to make it clear for non-pae support machine. -v3: Bjorn's change:
use MAX_RESOURCE instead of -1
use start/end instead of bottom/max
for all arch instead of just x86_64
-v4: updated after PCI_MAX_RESOURCE_32 change.
-v5: restore io handling to use PCI_MAX_RESOURCE_32 as limit. -v6: checking pcibios_resource_to_bus return for every bus res, to decide it
if we need to try high at first.
It supports all arches instead of just x86_64.
-v7: split 4G limit change out to another patch according to Bjorn.
also use pci_clip_resource instead.
--- drivers/pci/bus.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)
drivers/pci/bus.c +19 -2
@@ -99,6 +99,8 @@ void pci_bus_remove_resources(struct pci_bus *bus) }static struct pci_bus_region pci_mem_32 = {0 ,0xffffffff }; +static struct pci_bus_region pci_mem_64 = {(resource_size_t)(1ULL <<32 ), + (resource_size_t)(-1ULL )};static void pci_clip_resource (struct resource *res,struct pci_bus *bus,struct pci_bus_region *region)
@@ -149,6 +151,7 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,pci_bus_for_each_resource (bus, r, i) {struct resource avail; +int try_again =0 ;if (!r)continue ;
@@ -165,15 +168,23 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,/* * don't allocate too high if the pref mem doesn't -* support 64bit. +* support 64bit, also if this is a 64-bit mem +* resource, try above 4GB first */ avail = *r; -if (!(res->flags &IORESOURCE_MEM_64 )) { +if (res->flags &IORESOURCE_MEM_64 ) { +pci_clip_resource (&avail, bus, &pci_mem_64); +if (!resource_size (&avail)) + avail = *r; +else + try_again =1 ; + }else {pci_clip_resource (&avail, bus, &pci_mem_32);if (!resource_size (&avail))continue ; } +again:/* Ok, try it out.. */ ret = allocate_resource(r, res, size, max(avail.start, r->start ? : min),
@@ -181,6 +192,12 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, alignf, alignf_data);if (ret ==0 )break ; + +if (try_again) { + avail = *r; + try_again =0 ; + goto again; + } }return ret; }
Signature · 1 line
1.8.1.4
PATCH v3 9/12 PCI: Kill PCIBIOS_MAX_MEM_32
In reply to Yinghai LuNo user anymore.
--- arch/x86/include/asm/pci.h | 1 - include/linux/pci.h | 4 ---- 2 files changed, 5 deletions(-)
arch/x86/include/asm/pci.h +0 -1
@@ -125,7 +125,6 @@ int setup_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc,/* generic pci stuff */ #include <asm-generic/pci.h> -#define PCIBIOS_MAX_MEM_32 0xffffffff#ifdef CONFIG_NUMA /* Returns the node based on pci bus */
include/linux/pci.h +0 -4
@@ -1491,10 +1491,6 @@ static inline struct pci_dev *pci_dev_get(struct pci_dev *dev)#include <asm/pci.h> -#ifndef PCIBIOS_MAX_MEM_32 -#define PCIBIOS_MAX_MEM_32 (-1) -#endif -/* these helpers provide future and backwards compatibility * for accessing popular PCI BAR info */ #define pci_resource_start (dev, bar) ((dev)->resource[(bar)].start)
Signature · 1 line
1.8.1.4
PATCH v3 10/12 PCI: Try best to allocate pref mmio 64bit above 4g
In reply to Yinghai LuWhen one of children resources does not support MEM_64, MEM_64 for bridge get reset, so pull down whole pref resource on the bridge under 4G.
If the bridge support pref mem 64, will only allocate that with pref mem64 to children that support it.
For children resources if they only support pref mem 32, will allocate them from non pref mem instead.
If the bridge only support 32bit pref mmio, will still have all children pref mmio under that.
-v2: Add release bridge res support with bridge mem res for pref_mem children res. -v3: refresh and make it can be applied early before for_each_dev_res patchset.
--- drivers/pci/setup-bus.c | 133 ++++++++++++++++++++++++++++++++---------------- drivers/pci/setup-res.c | 14 ++++- 2 files changed, 101 insertions(+), 46 deletions(-)
drivers/pci/setup-bus.c +88 -45
@@ -711,12 +711,11 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)bus resource of a given type. Note: we intentionally skip the bus resources which have already been assigned (that is, have non-NULL parent resource). */ -static struct resource *find_free_bus_resource (struct pci_bus *bus,unsigned long type) +static struct resource *find_free_bus_resource (struct pci_bus *bus, +unsigned long type_mask,unsigned long type) {int i;struct resource *r; -unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM | -IORESOURCE_PREFETCH ;pci_bus_for_each_resource (bus, r, i) {if (r == &ioport_resource || r == &iomem_resource)
@@ -813,7 +812,8 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size, resource_size_t add_size, struct list_head *realloc_head) { struct pci_dev *dev; -struct resource *b_res =find_free_bus_resource (bus,IORESOURCE_IO ); +struct resource *b_res =find_free_bus_resource (bus,IORESOURCE_IO , +IORESOURCE_IO );resource_size_t size =0 , size0 =0 , size1 =0 ;resource_size_t children_add_size =0 ;resource_size_t min_align, align;
@@ -913,15 +913,16 @@ static inline resource_size_t calculate_mem_align(resource_size_t *aligns,* guarantees that all child resources fit in this size. */ static int pbus_size_mem (struct pci_bus *bus,unsigned long mask, -unsigned long type,resource_size_t min_size, -resource_size_t add_size, -struct list_head *realloc_head) +unsigned long type,unsigned long type2, +resource_size_t min_size,resource_size_t add_size, +struct list_head *realloc_head) {struct pci_dev *dev;resource_size_t min_align, align, size, size0, size1;resource_size_t aligns[12 ];/* Alignments from 1Mb to 2Gb */ int order, max_order; -struct resource *b_res =find_free_bus_resource (bus, type); +struct resource *b_res =find_free_bus_resource (bus, + mask |IORESOURCE_PREFETCH , type);unsigned int mem64_mask =0 ;resource_size_t children_add_size =0 ;
@@ -942,7 +943,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,struct resource *r = &dev->resource[i];resource_size_t r_size; -if (r->parent || (r->flags & mask) != type) +if (r->parent || ((r->flags & mask) != type && + (r->flags & mask) != type2))continue ; r_size =resource_size (r);#ifdef CONFIG_PCI_IOV
@@ -1115,8 +1117,9 @@ void __ref __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) {struct pci_dev *dev; -unsigned long mask, prefmask; +unsigned long mask, prefmask, type2 =0 ;resource_size_t additional_mem_size =0 , additional_io_size =0 ; +struct resource *b_res;list_for_each_entry (dev, &bus->devices, bus_list) {struct pci_bus *b = dev->subordinate;
@@ -1161,15 +1164,31 @@ void __ref __pci_bus_size_bridges(struct pci_bus *bus,has already been allocated by arch code, try non-prefetchable range for both types of PCI memory resources. */ + b_res = &bus->self->resource[PCI_BRIDGE_RESOURCES ]; mask =IORESOURCE_MEM ; prefmask =IORESOURCE_MEM |IORESOURCE_PREFETCH ; -if (pbus_size_mem (bus, prefmask, prefmask, +if (b_res[2 ].flags &IORESOURCE_MEM_64 ) { + prefmask |=IORESOURCE_MEM_64 ; +if (pbus_size_mem (bus, prefmask, prefmask, prefmask, realloc_head ?0 : additional_mem_size, - additional_mem_size, realloc_head)) - mask = prefmask;/* Success, size non-prefetch only. */ -else - additional_mem_size += additional_mem_size; -pbus_size_mem (bus, mask,IORESOURCE_MEM , + additional_mem_size, realloc_head)) { +/* Success, size non-pref64 only. */ + mask = prefmask; + type2 = prefmask & ~IORESOURCE_MEM_64 ; + } + } +if (!type2) { + prefmask &= ~IORESOURCE_MEM_64 ; +if (pbus_size_mem (bus, prefmask, prefmask, prefmask, + realloc_head ?0 : additional_mem_size, + additional_mem_size, realloc_head)) { +/* Success, size non-prefetch only. */ + mask = prefmask; + }else + additional_mem_size += additional_mem_size; + type2 =IORESOURCE_MEM ; + } +pbus_size_mem (bus, mask,IORESOURCE_MEM , type2, realloc_head ?0 : additional_mem_size, additional_mem_size, realloc_head);break ;
@@ -1255,42 +1274,66 @@ static void __ref __pci_bridge_assign_resources(const struct pci_dev *bridge,static void pci_bridge_release_resources (struct pci_bus *bus,unsigned long type) { -int idx; -bool changed = false; -struct pci_dev *dev; +struct pci_dev *dev = bus->self;struct resource *r;unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM | -IORESOURCE_PREFETCH ; +IORESOURCE_PREFETCH |IORESOURCE_MEM_64 ; +unsigned old_flags =0 ; +struct resource *b_res; +int idx =1 ; - dev = bus->self; -for (idx =PCI_BRIDGE_RESOURCES ; idx <=PCI_BRIDGE_RESOURCE_END ; - idx++) { - r = &dev->resource[idx]; -if ((r->flags & type_mask) != type) -continue ; -if (!r->parent) -continue ; -/* -* if there are children under that, we should release them -* all -*/ -release_child_resources (r); -if (!release_resource (r)) { -dev_printk (KERN_DEBUG , &dev->dev, -"resource %d %pR released\n" , idx, r); -/* keep the old size */ - r->end =resource_size (r) -1 ; - r->start =0 ; - r->flags =0 ; - changed = true; - } - } + b_res = &dev->resource[PCI_BRIDGE_RESOURCES ]; + +/* +* 1. if there is io port assign fail, will release bridge +* io port. +* 2. if there is non pref mmio assign fail, release bridge +* nonpref mmio. +* 3. if there is 64bit pref mmio assign fail, and bridge pref +* is 64bit, release bridge pref mmio. +* 4. if there is pref mmio assign fail, and bridge pref is +* 32bit mmio, release bridge pref mmio +* 5. if there is pref mmio assign fail, and bridge pref is not +* assigned, release bridge nonpref mmio. +*/ +if (type &IORESOURCE_IO ) + idx =0 ; +else if (!(type &IORESOURCE_PREFETCH )) + idx =1 ; +else if ((type &IORESOURCE_MEM_64 ) && + (b_res[2 ].flags &IORESOURCE_MEM_64 )) + idx =2 ; +else if (!(b_res[2 ].flags &IORESOURCE_MEM_64 ) && + (b_res[2 ].flags &IORESOURCE_PREFETCH )) + idx =2 ; +else + idx =1 ; + + r = &b_res[idx]; + +if (!r->parent) +return ; + +/* +* if there are children under that, we should release them +* all +*/ +release_child_resources (r); +if (!release_resource (r)) { + type = old_flags = r->flags & type_mask; +dev_printk (KERN_DEBUG , &dev->dev,"resource %d %pR released\n" , +PCI_BRIDGE_RESOURCES + idx, r); +/* keep the old size */ + r->end =resource_size (r) -1 ; + r->start =0 ; + r->flags =0 ; -if (changed) {/* avoiding touch the one without PREF */ if (type &IORESOURCE_PREFETCH ) type =IORESOURCE_PREFETCH ;__pci_setup_bridge (bus, type); +/* for next child res under same bridge */ + r->flags = old_flags; } }
@@ -1491,7 +1534,7 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)LIST_HEAD (fail_head);struct pci_dev_resource *fail_res;unsigned long type_mask =IORESOURCE_IO |IORESOURCE_MEM | -IORESOURCE_PREFETCH ; +IORESOURCE_PREFETCH |IORESOURCE_MEM_64 ;int pci_try_num =1 ;enum enable_type enable_local;
drivers/pci/setup-res.c +13 -1
@@ -208,9 +208,21 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,/* First, try exact prefetching match.. */ ret =pci_bus_alloc_resource (bus, res, size, align, min, -IORESOURCE_PREFETCH , +IORESOURCE_PREFETCH |IORESOURCE_MEM_64 , pcibios_align_resource, dev); +if (ret <0 && + (res->flags & (IORESOURCE_PREFETCH |IORESOURCE_MEM_64 ))) { +/* +* That failed. +* +* Try below 4g pref +*/ + ret =pci_bus_alloc_resource (bus, res, size, align, min, +IORESOURCE_PREFETCH , + pcibios_align_resource, dev); + } +if (ret <0 && (res->flags &IORESOURCE_PREFETCH )) {/* * That failed.
Signature · 1 line
1.8.1.4
PATCH v3 11/12 PCI: Sort pci root bus resources list
In reply to Yinghai LuSome x86 systems expose above 4G 64bit mmio in _CRS as non-pref mmio range.
[ 49.415281] PCI host bridge to bus 0000:00 [ 49.419921] pci_bus 0000:00: root bus resource [bus 00-1e] [ 49.426107] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] [ 49.433041] pci_bus 0000:00: root bus resource [io 0x1000-0x5fff] [ 49.440010] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff] [ 49.447768] pci_bus 0000:00: root bus resource [mem 0xfed8c000-0xfedfffff] [ 49.455532] pci_bus 0000:00: root bus resource [mem 0x90000000-0x9fffbfff] [ 49.463259] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x381fffffffff]
During assign unassigned 64bit mmio resource, it will go through every non-pref mmio for root bus in pci_bus_alloc_resource(). As the loop is with pci_bus_for_each_resource(), and could have chance to use under 4G mmio range instead of above 4G mmio range if the requested range is not big enough, even it could handle above 4G 64bit pref mmio.
For root bus, we can order list from high to low in pci_add_resource_offset(), during creating root bus, it will still keep the same order in final bus resource list.
pci_acpi_scan_root ==> add_resources ==> pci_add_resource_offset: # Add to temp resources ==> pci_create_root_bus ==> pci_bus_add_resource # add to final bus resources.
After that, we can make sure 64bit pref mmio for pci bridges will be allocated higest of mmio non-pref, and in this case it is above 4G instead of under 4G.
--- drivers/pci/bus.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/pci/bus.c +13 -2
@@ -21,7 +21,8 @@void pci_add_resource_offset (struct list_head *resources,struct resource *res,resource_size_t offset) { -struct pci_host_bridge_window *window; +struct pci_host_bridge_window *window, *tmp; +struct list_head *n; window =kzalloc (sizeof (struct pci_host_bridge_window ),GFP_KERNEL );if (!window) {
@@ -31,7 +32,17 @@ void pci_add_resource_offset(struct list_head *resources, struct resource *res, window->res = res; window->offset = offset; -list_add_tail (&window->list, resources); + +/* sorted it according to res end */ + n = resources; +list_for_each_entry (tmp, resources, list) +if (window->res->end > tmp->res->end) { + n = &tmp->list; +break ; + } + +/* Insert it just before n */ +list_add_tail (&window->list, n); }EXPORT_SYMBOL (pci_add_resource_offset);
Signature · 1 line
1.8.1.4
Bjorn Helgaas
maintainerRe: [PATCH v3 11/12] PCI: Sort pci root bus resources list
Quoted 19 lines · Yinghai Lu
Some x86 systems expose above 4G 64bit mmio in
_CRSas non-pref mmio range.[ 49.415281] PCI host bridge to bus 0000:00 [ 49.419921] pci_bus 0000:00: root bus resource [bus 00-1e] [ 49.426107] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] [ 49.433041] pci_bus 0000:00: root bus resource [io 0x1000-0x5fff] [ 49.440010] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff] [ 49.447768] pci_bus 0000:00: root bus resource [mem 0xfed8c000-0xfedfffff] [ 49.455532] pci_bus 0000:00: root bus resource [mem 0x90000000-0x9fffbfff] [ 49.463259] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x381fffffffff]During assign unassigned 64bit mmio resource, it will go through every non-pref mmio for root bus in
pci_bus_alloc_resource(). As the loop is withpci_bus_for_each_resource(), and could have chance to use under 4G mmio range instead of above 4G mmio range if the requested range is not big enough, even it could handle above 4G 64bit pref mmio.For root bus, we can order list from high to low in
pci_add_resource_offset(), during creating root bus, it will still keep the same order in final bus resource list.
Quoted 8 lines
pci_acpi_scan_root ==> add_resources ==> pci_add_resource_offset: # Add to temp resources ==> pci_create_root_bus ==> pci_bus_add_resource # add to final bus resources.After that, we can make sure 64bit pref mmio for pci bridges will be allocated higest of mmio non-pref, and in this case it is above 4G instead of under 4G.
It really irritates me when I ask a question [1] and you just repost the patch without even trying to answer it.
[1] http://lkml.kernel.org/r/CAErSpo4r6eJhgmfpth7haKDiKzDB+ZnEq0p_qdfTPo+kqySGgg@mail.gmail.com
Quoted 41 lines
Signed-off-by Yinghai Lu <[email protected]>--- drivers/pci/bus.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)drivers/pci/bus.c +13
-3@@ -21,7 +21,8 @@void pci_add_resource_offset (struct list_head *resources,struct resource *res,resource_size_t offset) { -struct pci_host_bridge_window *window; +struct pci_host_bridge_window *window, *tmp; +struct list_head *n; window =kzalloc (sizeof (struct pci_host_bridge_window ),GFP_KERNEL );if (!window) {@@ -31,7 +32,17 @@ void pci_add_resource_offset(struct list_head *resources, struct resource *res, window->res = res; window->offset = offset; -list_add_tail (&window->list, resources); + +/* sorted it according to res end */ + n = resources; +list_for_each_entry (tmp, resources, list) +if (window->res->end > tmp->res->end) { + n = &tmp->list; +break ; + } + +/* Insert it just before n */ +list_add_tail (&window->list, n); }EXPORT_SYMBOL (pci_add_resource_offset); --1.8.1.4
Re: [PATCH v3 11/12] PCI: Sort pci root bus resources list
Quoted 26 lines · Bjorn Helgaas
Quoted 25 lines · Yinghai Lu
Some x86 systems expose above 4G 64bit mmio in
_CRSas non-pref mmio range.[ 49.415281] PCI host bridge to bus 0000:00 [ 49.419921] pci_bus 0000:00: root bus resource [bus 00-1e] [ 49.426107] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7] [ 49.433041] pci_bus 0000:00: root bus resource [io 0x1000-0x5fff] [ 49.440010] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff] [ 49.447768] pci_bus 0000:00: root bus resource [mem 0xfed8c000-0xfedfffff] [ 49.455532] pci_bus 0000:00: root bus resource [mem 0x90000000-0x9fffbfff] [ 49.463259] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x381fffffffff]During assign unassigned 64bit mmio resource, it will go through every non-pref mmio for root bus in
pci_bus_alloc_resource(). As the loop is withpci_bus_for_each_resource(), and could have chance to use under 4G mmio range instead of above 4G mmio range if the requested range is not big enough, even it could handle above 4G 64bit pref mmio.For root bus, we can order list from high to low in
pci_add_resource_offset(), during creating root bus, it will still keep the same order in final bus resource list.pci_acpi_scan_root ==> add_resources ==> pci_add_resource_offset: # Add to temp resources ==> pci_create_root_bus ==> pci_bus_add_resource # add to final bus resources.
Quoted 7 lines
Quoted 2 lines
After that, we can make sure 64bit pref mmio for pci bridges will be allocated higest of mmio non-pref, and in this case it is above 4G instead of under 4G.
It really irritates me when I ask a question [1] and you just repost the patch without even trying to answer it.
[1] http://lkml.kernel.org/r/CAErSpo4r6eJhgmfpth7haKDiKzDB+ZnEq0p_qdfTPo+kqySGgg@mail.gmail.com
I expanded the changelog quite a bit, and thought it should have answered your concern.
The old nehalem-ex and westmere-ex platform have specify above 4g mmio as pref, but new ivybridge-ex platform only keep above 4g mmio as mmio as non-pref.
so we need allocation code to try the above 4G range for pref allocation at first to
avoid mmio pref taking under 4G mmio range.
We may need to backport it to old kernel or stable kernel, but should wait a while after it get into linus's tree for a while.
Thanks
Yinghai
PATCH v3 12/12 intel-gtt: Read 64bit for gmar_bus_addr
In reply to Yinghai LuThat bar could be 64bit pref mem and above 4G.
-v2: refresh to 3.13-rc1
--- drivers/char/agp/intel-gtt.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/char/agp/intel-gtt.c +10 -4
@@ -609,8 +609,10 @@ static bool intel_gtt_can_wc(void)static int intel_gtt_init (void ) {u32 gma_addr; +u32 addr_hi =0 ;u32 gtt_map_size;int ret; +int pos; ret = intel_private.driver->setup ();if (ret !=0 )
@@ -660,13 +662,17 @@ static int intel_gtt_init(void) }if (INTEL_GTT_GEN <=2 ) -pci_read_config_dword (intel_private.pcidev,I810_GMADDR , - &gma_addr); + pos =I810_GMADDR ;else -pci_read_config_dword (intel_private.pcidev,I915_GMADDR , - &gma_addr); + pos =I915_GMADDR ; + +pci_read_config_dword (intel_private.pcidev, pos, &gma_addr); + +if (gma_addr &PCI_BASE_ADDRESS_MEM_TYPE_64 ) +pci_read_config_dword (intel_private.pcidev, pos +4 , &addr_hi); intel_private.gma_bus_addr = (gma_addr &PCI_BASE_ADDRESS_MEM_MASK ); + intel_private.gma_bus_addr |= (u64 )addr_hi <<32 ;return 0 ; }
Signature · 1 line
1.8.1.4
Yinghai:
Would you please consider *not* bundling unrelated, or at least non-dependent, patches together in one series. As a distrobution subsystem maintainer, this type of bundling really complicates trying to identify and isolate key material for bringing in. It also makes trying to identify the core content for key fixes, which can occur many months or years later, significantly more difficult when they were buried within a series with other unrelated content
Thanks,
Myron.
Quoted 55 lines · Yinghai Lu
First 4 are for Gu Zheng <[email protected]> to help double pci device removing via sysfs.
For patch 4, rafael like to have use lock instead of atomic.Patch 6 are for not enabling realloc auto when 64bit mmio is not there for root bus.
Others are about mmio 64 allocation that could help Guo Chao <[email protected]> on powerpc mmio allocation. It will try to assign 64 bit resource above 4g at first.
Patch 1-4 and Patch 5-12 could be appliey seperatedly.
Could be found:
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.gitfor-pci-3.14And it is based on current pci/for-linus.
(quoted diff) +0
-2-v2: update after patch that move device_del down to pci_destroy_dev. add "Try best to allocate pref mmio 64bit above 4G" -v3: refresh and send out after pci_clip_resource() changes, as Bjorn is not happy with attachments.Yinghai Lu (12):
PCI: Usedevice_release_driverinpci_stop_root_bus
PCI: Move backpci_proc_attach_devicescalling
PCI: Move resources andbus_listreleasing topci_release_dev
PCI: Destroy pci dev only once
PCI: pcibus address to resource converting take bus instead of dev
PCI: Only enable realloc auto when root bus has 64bit mmio
PCI: Don't use 4G bus address directly in resource allocation
PCI: Try to allocate mem64 above 4G at first
PCI: KillPCIBIOS_MAX_MEM_32
PCI: Try best to allocate pref mmio 64bit above 4g
PCI: Sort pci root bus resources list
intel-gtt: Read 64bit forgmar_bus_addrarch/x86/include/asm/pci.h | 1 - drivers/char/agp/intel-gtt.c | 14 ++-- drivers/pci/bus.c | 74 ++++++++++++++++--- drivers/pci/host-bridge.c | 34 +++++---- drivers/pci/probe.c | 23 +++++- drivers/pci/remove.c | 31 ++------ drivers/pci/setup-bus.c | 165 ++++++++++++++++++++++++++++++------------- drivers/pci/setup-res.c | 14 +++- include/linux/pci.h | 10 +-- 9 files changed, 257 insertions(+), 109 deletions(-)--
1.8.1.4--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · Myron Stowe
Would you please consider *not* bundling unrelated, or at least non-dependent, patches together in one series. As a distrobution subsystem maintainer, this type of bundling really complicates trying to identify and isolate key material for bringing in. It also makes trying to identify the core content for key fixes, which can occur many months or years later, significantly more difficult when they were buried within a series with other unrelated content
ok.
Will resend them in three patchset.
Thanks
Yinghai