Skip to contentSkip to conversation

cpuhp: fix some st->target issues

PATCH Superseded Phil Auld 2 participants 7 messages Jul 11, 2022 – Aug 10, 2022 CPU hotplug

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Phil Auld

author
#0 lore raw

v3 cover cpuhp: fix some st->target issues

Several small fixes that clean up some cpuhp inconsistencies. The first prevents target_store() from calling cpu_down() when target == state which prevents the cpu being incorrectly marked as dying. The second just makes the boot cpu have a valid cpuhp target rather than 0 (CPU_OFFLINE) while being in state CPU_ONLINE.

A further issue which these two patches don't address is that the cpuX/online file looks at the device->offline state and can thus get out of sync with the actual cpuhp state if the cpuhp target is used to change state.

v3: Added code to make sure st->target == target in the nop case.

Phil Auld (2):
cpuhp: make target_store() a nop when target == state
cpuhp: Set cpuhp target for boot cpu

 kernel/cpu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH v3 1/2 cpuhp: make target_store() a nop when target == state

writing the current state back in hotplug/target calls cpu_down() which will set cpu dying even when it isn't and then nothing will ever clear it. A stress test that reads values and writes them back for all cpu device files in sysfs will trigger the BUG() in select_fallback_rq once all cpus are marked as dying.

kernel/cpu.c::target_store()

	...
        if (st->state < target)
                ret = cpu_up(dev->id, target);
        else
                ret = cpu_down(dev->id, target);

cpu_down() -> cpu_set_state()

	 bool bringup = st->state < target;
	 ...
	 if (cpu_dying(cpu) != !bringup)
		set_cpu_dying(cpu, !bringup);

Fix this by letting state==target fall through in the target_store() conditional. Also make sure st->target == target in that case.

---
 kernel/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/cpu.c +3 -1
index bbad5e375d3b..305694a2ca26 100644
@@ -2326,8 +2326,10 @@ static ssize_t target_store(struct device *dev, struct device_attribute *attr,
 
 	if (st->state < target)
 		ret = cpu_up(dev->id, target);
-	else
+	else if (st->state > target)
 		ret = cpu_down(dev->id, target);
+	else if (st->target != target)
+		st->target = target;
 out:
 	unlock_device_hotplug();
 	return ret ? ret : count;
Signature · 1 line
2.31.1
Reviewed-by on v3 1/2
#2 lore raw

Re: [PATCH v3 1/2] cpuhp: make target_store() a nop when target == state

Quoted 19 lines · Phil Auld

writing the current state back in hotplug/target calls cpu_down() which will set cpu dying even when it isn't and then nothing will ever clear it. A stress test that reads values and writes them back for all cpu device files in sysfs will trigger the BUG() in select_fallback_rq once all cpus are marked as dying.

kernel/cpu.c::target_store()

      ...
        if (st->state < target)
                ret = cpu_up(dev->id, target);
        else
                ret = cpu_down(dev->id, target);

cpu_down() -> cpu_set_state()

       bool bringup = st->state < target;
       ...
       if (cpu_dying(cpu) != !bringup)
              set_cpu_dying(cpu, !bringup);
Quoted 4 lines

Fix this by letting state==target fall through in the target_store() conditional. Also make sure st->target == target in that case.

One nit below, otherwise:

Quoted 9 lines
---
 kernel/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/cpu.c +0 -0
index bbad5e375d3b..305694a2ca26 100644

kernel/cpu.c static ssize_t target_store(struct device *dev, struct device_attribute *attr, · line 2331

Quoted 7 lines
kernel/cpu.c +2 -1
@@ -2326,8 +2326,10 @@ static ssize_t target_store(struct device *dev, struct device_attribute *attr,
 
      if (st->state < target)
              ret = cpu_up(dev->id, target);
-	else
+	else if (st->state > target)
              ret = cpu_down(dev->id, target);
+	else if (st->target != target)

Should we make this:

        else if (WARN(st->target != target))
Quoted 6 lines
kernel/cpu.c +1 -1
+		st->target = target;
 out:
      unlock_device_hotplug();
      return ret ? ret : count;
--

2.31.1

author
#3 lore raw

Re: [PATCH v3 1/2] cpuhp: make target_store() a nop when target == state

Quoted 20 lines · On Tue, Jul 19, 2022 at 03:34:03PM +0100 Valentin Schneider wrote:
Quoted 19 lines · Phil Auld

writing the current state back in hotplug/target calls cpu_down() which will set cpu dying even when it isn't and then nothing will ever clear it. A stress test that reads values and writes them back for all cpu device files in sysfs will trigger the BUG() in select_fallback_rq once all cpus are marked as dying.

kernel/cpu.c::target_store()

      ...
        if (st->state < target)
                ret = cpu_up(dev->id, target);
        else
                ret = cpu_down(dev->id, target);

cpu_down() -> cpu_set_state()

       bool bringup = st->state < target;
       ...
       if (cpu_dying(cpu) != !bringup)
              set_cpu_dying(cpu, !bringup);
Quoted 8 lines
Quoted 4 lines

Fix this by letting state==target fall through in the target_store() conditional. Also make sure st->target == target in that case.

One nit below, otherwise:

Thanks!

Quoted 19 lines
Quoted 9 lines
---
 kernel/cpu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/cpu.c +0 -0
index bbad5e375d3b..305694a2ca26 100644

kernel/cpu.c static ssize_t target_store(struct device *dev, struct device_attribute *attr, · line 2331

Quoted 7 lines
kernel/cpu.c +2 -1
@@ -2326,8 +2326,10 @@ static ssize_t target_store(struct device *dev, struct device_attribute *attr,
 
      if (st->state < target)
              ret = cpu_up(dev->id, target);
-	else
+	else if (st->state > target)
              ret = cpu_down(dev->id, target);
+	else if (st->target != target)

Should we make this:

Quoted 2 lines
        else if (WARN(st->target != target))

If you think that's important I can make it a WARN, sure.

I'll try to remember to keep your Reviewed-bys this time if that's okay.

Cheers,
Phil

Quoted 7 lines
Quoted 6 lines
kernel/cpu.c +1 -1
+		st->target = target;
 out:
      unlock_device_hotplug();
      return ret ? ret : count;
--

2.31.1

author
#4 lore raw

PATCH v3 2/2 cpuhp: Set cpuhp target for boot cpu

In reply to Phil Auld

Since the boot cpu does not go through the hotplug process it ends up with state == CPUHP_ONLINE but target == CPUHP_OFFLINE. Set the target to match in boot_cpu_hotplug_init().

---
 kernel/cpu.c | 1 +
 1 file changed, 1 insertion(+)
kernel/cpu.c +1 -0
index 305694a2ca26..104f43a6f4ad 100644
@@ -2690,6 +2690,7 @@ void __init boot_cpu_hotplug_init(void)
 	cpumask_set_cpu(smp_processor_id(), &cpus_booted_once_mask);
 #endif
 	this_cpu_write(cpuhp_state.state, CPUHP_ONLINE);
+	this_cpu_write(cpuhp_state.target, CPUHP_ONLINE);
 }
 
 /*
Signature · 1 line
2.31.1
author
#6 lore raw
In reply to Phil Auld
Quoted 12 lines · On Mon, Jul 11, 2022 at 05:16:17PM -0400 Phil Auld wrote:

Several small fixes that clean up some cpuhp inconsistencies. The first prevents target_store() from calling cpu_down() when target == state which prevents the cpu being incorrectly marked as dying. The second just makes the boot cpu have a valid cpuhp target rather than 0 (CPU_OFFLINE) while being in state CPU_ONLINE.

A further issue which these two patches don't address is that the cpuX/online file looks at the device->offline state and can thus get out of sync with the actual cpuhp state if the cpuhp target is used to change state.

Quoted 12 lines

v3: Added code to make sure st->target == target in the nop case.

Phil Auld (2):
cpuhp: make target_store() a nop when target == state
cpuhp: Set cpuhp target for boot cpu

 kernel/cpu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Signature · 1 line
2.31.1

Ping?

Any thoughts on this? I'm not sure what tree it needs to go in but getting a cpu shutdown by writing the current target/state back in here seems wrong.

Thanks,
Phil