PCI: generic: map config window in one go
PATCH 1/1 PCI: generic: map config window in one go
Instead of iterating over the PCI config window and performing individual ioremap() calls on all the adjacent slices, perform a single ioremap() to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.
On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
---
huge-vmap for arm64 proposed here:
http://article.gmane.org/gmane.linux.kernel.hardened.devel/1661
drivers/pci/host/pci-host-generic.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/pci/host/pci-host-generic.c +7 -5
@@ -161,6 +161,7 @@ static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)struct device *dev = pci->host.dev.parent;struct device_node *np = dev->of_node;u32 sz =1 << pci->cfg.ops->bus_shift; +void *window; err =of_address_to_resource (np,0 , &pci->cfg.res);if (err) {
@@ -186,14 +187,15 @@ static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)return -ENOMEM ; bus_range = pci->cfg.bus_range; + window =devm_ioremap (dev, pci->cfg.res.start, + (bus_range->end - bus_range->start +1 ) * sz); +if (!window) +return -ENOMEM ; +for (busn = bus_range->start; busn <= bus_range->end; ++busn) {u32 idx = busn - bus_range->start; - pci->cfg.win[idx] =devm_ioremap (dev, - pci->cfg.res.start + idx * sz, - sz); -if (!pci->cfg.win[idx]) -return -ENOMEM ; + pci->cfg.win[idx] = window + idx * sz; }return 0 ;
Signature · 1 line
2.5.0
PATCH 1/1 PCI: generic: map config window in one go
Quoted 11 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
OK, this math is slightly off: 4 KB for each 2 MB section == 64 * 4 == 128 KB
Quoted 45 lines
Signed-off-by Ard Biesheuvel <[email protected]>---
huge-vmap for arm64 proposed here:
http://article.gmane.org/gmane.linux.kernel.hardened.devel/1661drivers/pci/host/pci-host-generic.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)drivers/pci/host/pci-host-generic.c +7
-6@@ -161,6 +161,7 @@ static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)struct device *dev = pci->host.dev.parent;struct device_node *np = dev->of_node;u32 sz =1 << pci->cfg.ops->bus_shift; +void *window; err =of_address_to_resource (np,0 , &pci->cfg.res);if (err) {@@ -186,14 +187,15 @@ static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)return -ENOMEM ; bus_range = pci->cfg.bus_range; + window =devm_ioremap (dev, pci->cfg.res.start, + (bus_range->end - bus_range->start +1 ) * sz); +if (!window) +return -ENOMEM ; +for (busn = bus_range->start; busn <= bus_range->end; ++busn) {u32 idx = busn - bus_range->start; - pci->cfg.win[idx] =devm_ioremap (dev, - pci->cfg.res.start + idx * sz, - sz); -if (!pci->cfg.win[idx]) -return -ENOMEM ; + pci->cfg.win[idx] = window + idx * sz; }return 0 ; --2.5.0
PATCH 1/1 PCI: generic: map config window in one go
Quoted 8 lines · Ard Biesheuvel
Quoted 7 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.
Quoted 7 lines
Quoted 4 lines
On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
OK, this math is slightly off: 4 KB for each 2 MB section == 64 * 4 == 128 KB
Sigh. 64 * 4 == 256 KB
Quoted 45 lines
Quoted 45 lines
Signed-off-by Ard Biesheuvel <[email protected]>---
huge-vmap for arm64 proposed here:
http://article.gmane.org/gmane.linux.kernel.hardened.devel/1661drivers/pci/host/pci-host-generic.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)drivers/pci/host/pci-host-generic.c +7
-6@@ -161,6 +161,7 @@ static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)struct device *dev = pci->host.dev.parent;struct device_node *np = dev->of_node;u32 sz =1 << pci->cfg.ops->bus_shift; +void *window; err =of_address_to_resource (np,0 , &pci->cfg.res);if (err) {@@ -186,14 +187,15 @@ static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)return -ENOMEM ; bus_range = pci->cfg.bus_range; + window =devm_ioremap (dev, pci->cfg.res.start, + (bus_range->end - bus_range->start +1 ) * sz); +if (!window) +return -ENOMEM ; +for (busn = bus_range->start; busn <= bus_range->end; ++busn) {u32 idx = busn - bus_range->start; - pci->cfg.win[idx] =devm_ioremap (dev, - pci->cfg.res.start + idx * sz, - sz); -if (!pci->cfg.win[idx]) -return -ENOMEM ; + pci->cfg.win[idx] = window + idx * sz; }return 0 ; --2.5.0
PATCH 1/1 PCI: generic: map config window in one go
In reply to Ard BiesheuvelHi Ard,
Quoted 7 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.
Quoted 6 lines
On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
Signed-off-by Ard Biesheuvel <[email protected]>---
The code was written this way in response to feedback during driver review that we couldn't necessarily grab that much contiguous vmalloc space on 32-bit ARM. Unless that's changed, we probably want to to predicate this change on having a 64-bit arch.
Will
PATCH 1/1 PCI: generic: map config window in one go
Quoted 14 lines · Will Deacon
Hi Ard,
Quoted 11 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Ard Biesheuvel <[email protected]>---
The code was written this way in response to feedback during driver review that we couldn't necessarily grab that much contiguous vmalloc space on 32-bit ARM. Unless that's changed, we probably want to to predicate this change on having a 64-bit arch.
Ah right. How about testing for the ARCH_HAVE_HUGE_VMAP Kconfig symbol explicitly?
PATCH 1/1 PCI: generic: map config window in one go
Quoted 18 lines · On Friday 29 January 2016 15:32:01 Ard Biesheuvel wrote:
Quoted 17 lines · Will Deacon
Hi Ard,
Quoted 13 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
Signed-off-by Ard Biesheuvel <[email protected]>---
Quoted 9 lines
Quoted 5 lines
The code was written this way in response to feedback during driver review that we couldn't necessarily grab that much contiguous vmalloc space on 32-bit ARM. Unless that's changed, we probably want to to predicate this change on having a 64-bit arch.
Ah right. How about testing for the
ARCH_HAVE_HUGE_VMAPKconfigsymbol explicitly?
Testing for 64BIT should be sufficient.
Arnd
PATCH 1/1 PCI: generic: map config window in one go
Quoted 25 lines · Arnd Bergmann
Quoted 24 lines · On Friday 29 January 2016 15:32:01 Ard Biesheuvel wrote:
Quoted 22 lines · Will Deacon
Hi Ard,
Quoted 7 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.Quoted 6 lines
On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
Signed-off-by Ard Biesheuvel <[email protected]>---
The code was written this way in response to feedback during driver review that we couldn't necessarily grab that much contiguous vmalloc space on 32-bit ARM. Unless that's changed, we probably want to to predicate this change on having a 64-bit arch.
Quoted 6 lines
Quoted 3 lines
Ah right. How about testing for the
ARCH_HAVE_HUGE_VMAPKconfigsymbol explicitly?Testing for 64BIT should be sufficient.
Does it make sense to spin a v2 for this patch? Given the discussion we had regarding allocating only the config regions for busses that are populated, perhaps there is a better approach here?
PATCH 1/1 PCI: generic: map config window in one go
Quoted 30 lines · On Friday 05 February 2016 11:48:54 Ard Biesheuvel wrote:
Quoted 29 lines · Arnd Bergmann
Quoted 27 lines · On Friday 29 January 2016 15:32:01 Ard Biesheuvel wrote:
Quoted 14 lines · Will Deacon
Hi Ard,
Quoted 11 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Ard Biesheuvel <[email protected]>---
The code was written this way in response to feedback during driver review that we couldn't necessarily grab that much contiguous vmalloc space on 32-bit ARM. Unless that's changed, we probably want to to predicate this change on having a 64-bit arch.
Ah right. How about testing for the
ARCH_HAVE_HUGE_VMAPKconfigsymbol explicitly?
Quoted 6 lines
Quoted 2 lines
Testing for 64BIT should be sufficient.
Does it make sense to spin a v2 for this patch? Given the discussion we had regarding allocating only the config regions for busses that are populated, perhaps there is a better approach here?
Allocating only the config regions that are actually used would be ideal, the problem is that you need to access the config space in order to know which ones are, so this is certainly a bit tricky.
Are there any downsides to the x86 approach of using fixmap to map each patch separately during the access? It's probably a bit slower per access, but we don't do a lot of those accesses after the system is booted.
Arnd
PATCH 1/1 PCI: generic: map config window in one go
Quoted 38 lines · Arnd Bergmann
Quoted 30 lines · On Friday 05 February 2016 11:48:54 Ard Biesheuvel wrote:
Quoted 29 lines · Arnd Bergmann
Quoted 27 lines · On Friday 29 January 2016 15:32:01 Ard Biesheuvel wrote:
Quoted 14 lines · Will Deacon
Hi Ard,
Quoted 11 lines · Ard Biesheuvel
Instead of iterating over the PCI config window and performing individual
ioremap()calls on all the adjacent slices, perform a singleioremap()to map the entire region, and divvy it up later. This not only prevents leaving some of it mapped if we fail half way through, it also ensures that archs that support huge-vmap can use section mappings to perform the mapping.On my Seattle A0 box, this transforms 128 separate 1 MB mappings that are mapped down to 4 KB pages into a single 128 MB mapping using 2 MB sections, saving 512 KB worth of page tables.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Ard Biesheuvel <[email protected]>---
The code was written this way in response to feedback during driver review that we couldn't necessarily grab that much contiguous vmalloc space on 32-bit ARM. Unless that's changed, we probably want to to predicate this change on having a 64-bit arch.
Ah right. How about testing for the
ARCH_HAVE_HUGE_VMAPKconfigsymbol explicitly?Quoted 6 lines
Quoted 2 lines
Testing for 64BIT should be sufficient.
Does it make sense to spin a v2 for this patch? Given the discussion we had regarding allocating only the config regions for busses that are populated, perhaps there is a better approach here?
Quoted 9 lines
Allocating only the config regions that are actually used would be ideal, the problem is that you need to access the config space in order to know which ones are, so this is certainly a bit tricky.
Are there any downsides to the x86 approach of using fixmap to map each patch separately during the access? It's probably a bit slower per access, but we don't do a lot of those accesses after the system is booted.
Yes, I guess that makes a lot more sense than ioremapping 10s to 100s of MBs and hardly ever using them.