Skip to contentSkip to conversation

patch "x86: MOVE PCI IO ECS code to x86/pci" breaks CPU hotplug

DISCUSSION Jan Beulich 3 participants 5 messages Aug 21, 2008 – Aug 22, 2008

lore
Share & tools
lore
#0 lore raw

Converting __cpuinit functions called out of init_amd() (and similar others) to __init (and making them subject of xxx_initcall() handling isn't valid, as they would no longer be called for hot plugged CPUs.

Further, since it's likely that in virtualized environments the MSR write would at best be ignored, I'd also recommend using the fault-safe accessors here *and* check that the bit actually got set before setting PCI_HAS_IO_ECS (one would obviously have to BUG() when hot-plugged CPUs fail to set the bit when those available at boot successfully did so).

Jan

#1 lore raw
Quoted 10 lines · Jan Beulich

Converting __cpuinit functions called out of init_amd() (and similar others) to __init (and making them subject of xxx_initcall() handling isn't valid, as they would no longer be called for hot plugged CPUs.

Further, since it's likely that in virtualized environments the MSR write would at best be ignored, I'd also recommend using the fault-safe accessors here *and* check that the bit actually got set before setting PCI_HAS_IO_ECS (one would obviously have to BUG() when hot-plugged CPUs fail to set the bit when those available at boot successfully did so).

hm, which patch is this exactly, and in what tree? It's not upstream nor in -tip.

Ingo

#2 lore raw
Quoted 14 lines · Ingo Molnar
Quoted 10 lines · Jan Beulich

Converting __cpuinit functions called out of init_amd() (and similar others) to __init (and making them subject of xxx_initcall() handling isn't valid, as they would no longer be called for hot plugged CPUs.

Further, since it's likely that in virtualized environments the MSR write would at best be ignored, I'd also recommend using the fault-safe accessors here *and* check that the bit actually got set before setting PCI_HAS_IO_ECS (one would obviously have to BUG() when hot-plugged CPUs fail to set the bit when those available at boot successfully did so).

Quoted 2 lines

hm, which patch is this exactly, and in what tree? It's not upstream nor in -tip.

Probably this in the mainline kernel is meant:

commit 3a27dd1ce5de08e21e0266ddf00e6f1f843bfe8b

Author: Robert Richter <[email protected]>
Date:   Thu Jun 12 20:19:23 2008 +0200

x86: Move PCI IO ECS code to x86/pci

"Form follows function". Code is now where it belongs to.

The problem is code like this:

+static int __init enable_pci_io_ecs(void)
+{
+ /* assume all cpus from fam10h have IO ECS */
+ if (boot_cpu_data.x86 < 0x10)
+ return 0;

+       on_each_cpu(enable_pci_io_ecs_per_cpu, NULL, 1, 1);
+       pci_probe |= PCI_HAS_IO_ECS;
+       return 0;

+}

It is only called during the setup and if the cpu is enabled. I intend to rewrite the cpu setup code using cpu notifiers.

-Robert

Signature · 3 lines
Advanced Micro Devices, Inc.
Operating System Research Center
email: [email protected]
#3 lore raw
Quoted 43 lines · Robert Richter
Quoted 14 lines · Ingo Molnar
Quoted 10 lines · Jan Beulich

Converting __cpuinit functions called out of init_amd() (and similar others) to __init (and making them subject of xxx_initcall() handling isn't valid, as they would no longer be called for hot plugged CPUs.

Further, since it's likely that in virtualized environments the MSR write would at best be ignored, I'd also recommend using the fault-safe accessors here *and* check that the bit actually got set before setting PCI_HAS_IO_ECS (one would obviously have to BUG() when hot-plugged CPUs fail to set the bit when those available at boot successfully did so).

Quoted 2 lines

hm, which patch is this exactly, and in what tree? It's not upstream nor in -tip.

Probably this in the mainline kernel is meant:

commit 3a27dd1ce5de08e21e0266ddf00e6f1f843bfe8b

Author: Robert Richter <[email protected]>
Date:   Thu Jun 12 20:19:23 2008 +0200

x86: Move PCI IO ECS code to x86/pci

"Form follows function". Code is now where it belongs to.

The problem is code like this:

(quoted diff) +9 -0
+static int __init enable_pci_io_ecs(void)
+{
+       /* assume all cpus from fam10h have IO ECS */
+        if (boot_cpu_data.x86 < 0x10)
+               return 0;
+       on_each_cpu(enable_pci_io_ecs_per_cpu, NULL, 1, 1);
+       pci_probe |= PCI_HAS_IO_ECS;
+       return 0;
+}
Quoted 2 lines

It is only called during the setup and if the cpu is enabled. I intend to rewrite the cpu setup code using cpu notifiers.

ok. Unfortunately the revert looks quite non-trivial for v2.6.27. Could we have some _really_ minimal fix for v2.6.27?

Ingo

#4 lore raw
In reply to Ingo Molnar
Quoted 15 lines
Quoted 1 line
Quoted 1 line

Ingo Molnar <[email protected]> 21.08.08 15:02 >>>

Quoted 10 lines · Jan Beulich

Converting __cpuinit functions called out of init_amd() (and similar others) to __init (and making them subject of xxx_initcall() handling isn't valid, as they would no longer be called for hot plugged CPUs.

Further, since it's likely that in virtualized environments the MSR write would at best be ignored, I'd also recommend using the fault-safe accessors here *and* check that the bit actually got set before setting PCI_HAS_IO_ECS (one would obviously have to BUG() when hot-plugged CPUs fail to set the bit when those available at boot successfully did so).

Quoted 2 lines

hm, which patch is this exactly, and in what tree? It's not upstream nor in -tip.

It is upstream:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=3a27dd1ce5de08e21e0266ddf00e6f1f843bfe8b
(and I just verified it didn't change between -rc3 and -rc4).

Jan