Skip to contentSkip to conversation

cpufreq: Make cpufreq_online() call driver->offline() on errors

PATCH Superseded Rafael J. Wysocki 3 participants 5 messages Jun 21, 2021 – Jun 23, 2021 CPU frequency scaling framework

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 cpufreq: Make cpufreq_online() call driver->offline() on errors

From: Rafael J. Wysocki <[email protected]>

In the CPU removal path the ->offline() callback provided by the driver is always invoked before ->exit(), but in the cpufreq_online() error path it is not, so ->exit() is somehow expected to know the context in which it has been called and act accordingly.

That is less than straightforward, so make cpufreq_online() invoke the driver's ->offline() callback before ->exit() too.

This only potentially affects intel_pstate at this point.

Fixes 91a12e91dc39 ("cpufreq: Allow light-weight tear down and bring up of CPUs")
---
 drivers/cpufreq/cpufreq.c |    3 +++
 1 file changed, 3 insertions(+)
drivers/cpufreq/cpufreq.c +3 -0
Index: linux-pm/drivers/cpufreq/cpufreq.c
===================================================================
@@ -1516,6 +1516,9 @@ out_destroy_policy:
 	up_write(&policy->rwsem);
 
 out_exit_policy:
+	if (cpufreq_driver->offline)
+		cpufreq_driver->offline(policy);
+
 	if (cpufreq_driver->exit)
 		cpufreq_driver->exit(policy);
 
maintainer
#1 lore raw
Quoted 4 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

In the CPU removal path the ->offline() callback provided by the driver is always invoked before ->exit(),

Note that exit() doesn't get called if offline() is present in the CPU removal path. We call exit() _only_ when the cpufreq driver gets unregistered.

Quoted 3 lines

but in the cpufreq_online()
error path it is not, so ->exit() is somehow expected to know the context in which it has been called and act accordingly.

I agree, it isn't very clear.

Quoted 16 lines

That is less than straightforward, so make cpufreq_online() invoke the driver's ->offline() callback before ->exit() too.

This only potentially affects intel_pstate at this point.

Fixes 91a12e91dc39 ("cpufreq: Allow light-weight tear down and bring up of CPUs")
---
 drivers/cpufreq/cpufreq.c |    3 +++
 1 file changed, 3 insertions(+)

Index: linux-pm/drivers/cpufreq/cpufreq.c
===================================================================

drivers/cpufreq/cpufreq.c out_destroy_policy: · line 1523

Quoted 8 lines
drivers/cpufreq/cpufreq.c +3 -0
@@ -1516,6 +1516,9 @@ out_destroy_policy:
 	up_write(&policy->rwsem);
 
 out_exit_policy:
+	if (cpufreq_driver->offline)
+		cpufreq_driver->offline(policy);
+
 	if (cpufreq_driver->exit)
 		cpufreq_driver->exit(policy);

If we want to go down this path, then we better do more and make it very explicit that ->offline() follows a previous invocation of ->online().

Otherwise, with above we will end up calling ->offline() for two separate states, ->online() failed (i.e. two calls to offline() one after the other here) and other generic failures after ->init() passed.

So, better make it clear that online/offline are paired like init/exit.

drivers/cpufreq/cpufreq.c +15 -8
index 1d1b563cea4b..0ce48dcb2e8a 100644
@@ -1347,14 +1347,11 @@ static int cpufreq_online(unsigned int cpu)
        }
 
        if (!new_policy && cpufreq_driver->online) {
-               ret = cpufreq_driver->online(policy);
-               if (ret) {
-                       pr_debug("%s: %d: initialization failed\n", __func__,
-                                __LINE__);
-                       goto out_exit_policy;
-               }
-
-               /* Recover policy->cpus using related_cpus */
+               /*
+                * We did light-weight tear down earlier, don't need to do heavy
+                * initialization here. Just recover policy->cpus using
+                * related_cpus.
+                */
                cpumask_copy(policy->cpus, policy->related_cpus);
        } else {
                cpumask_copy(policy->cpus, cpumask_of(cpu));
@@ -1378,6 +1375,13 @@ static int cpufreq_online(unsigned int cpu)
                cpumask_copy(policy->related_cpus, policy->cpus);
        }
 
+       ret = cpufreq_driver->online(policy);
+       if (ret) {
+               pr_debug("%s: %d: %d: ->online() failed\n", __func__, __LINE__,
+                        ret);
+               goto out_exit_policy;
+       }
+
        down_write(&policy->rwsem);
        /*
         * affected cpus must always be the one, which are online. We aren't
@@ -1518,6 +1522,9 @@ static int cpufreq_online(unsigned int cpu)
 
        up_write(&policy->rwsem);
 
+       if (cpufreq_driver->offline)
+               cpufreq_driver->offline(policy);
+
 out_exit_policy:
        if (cpufreq_driver->exit)
                cpufreq_driver->exit(policy);
Signature · 1 line
viresh
#2 lore raw
Quoted 10 lines · Viresh Kumar
Quoted 4 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>

In the CPU removal path the ->offline() callback provided by the driver is always invoked before ->exit(),

Note that exit() doesn't get called if offline() is present in the CPU removal path. We call exit() _only_ when the cpufreq driver gets unregistered.

True, but that doesn't contradict what I said.

Quoted 35 lines
Quoted 3 lines

but in the cpufreq_online()
error path it is not, so ->exit() is somehow expected to know the context in which it has been called and act accordingly.

I agree, it isn't very clear.

Quoted 16 lines

That is less than straightforward, so make cpufreq_online() invoke the driver's ->offline() callback before ->exit() too.

This only potentially affects intel_pstate at this point.

Fixes 91a12e91dc39 ("cpufreq: Allow light-weight tear down and bring up of CPUs")
---
 drivers/cpufreq/cpufreq.c |    3 +++
 1 file changed, 3 insertions(+)

Index: linux-pm/drivers/cpufreq/cpufreq.c
===================================================================

drivers/cpufreq/cpufreq.c out_destroy_policy: · line 1523

Quoted 8 lines
drivers/cpufreq/cpufreq.c +3 -0
@@ -1516,6 +1516,9 @@ out_destroy_policy:
      up_write(&policy->rwsem);
 
 out_exit_policy:
+     if (cpufreq_driver->offline)
+             cpufreq_driver->offline(policy);
+
      if (cpufreq_driver->exit)
              cpufreq_driver->exit(policy);

If we want to go down this path, then we better do more and make it very explicit that ->offline() follows a previous invocation of ->online().

Quoted 3 lines

Otherwise, with above we will end up calling ->offline() for two separate states, ->online() failed (i.e. two calls to offline() one after the other here) and other generic failures after ->init() passed.

Good point.

Quoted 28 lines

So, better make it clear that online/offline are paired like init/exit.

drivers/cpufreq/cpufreq.c +5 -8
index 1d1b563cea4b..0ce48dcb2e8a 100644
@@ -1347,14 +1347,11 @@ static int cpufreq_online(unsigned int cpu)
        }
 
        if (!new_policy && cpufreq_driver->online) {
-               ret = cpufreq_driver->online(policy);
-               if (ret) {
-                       pr_debug("%s: %d: initialization failed\n", __func__,
-                                __LINE__);
-                       goto out_exit_policy;
-               }
-
-               /* Recover policy->cpus using related_cpus */
+               /*
+                * We did light-weight tear down earlier, don't need to do heavy
+                * initialization here. Just recover policy->cpus using
+                * related_cpus.
+                */
                cpumask_copy(policy->cpus, policy->related_cpus);
        } else {
                cpumask_copy(policy->cpus, cpumask_of(cpu));

drivers/cpufreq/cpufreq.c static int cpufreq_online(unsigned int cpu) · line 1378

Quoted 4 lines
drivers/cpufreq/cpufreq.c +1 -0
@@ -1378,6 +1375,13 @@ static int cpufreq_online(unsigned int cpu)
                cpumask_copy(policy->related_cpus, policy->cpus);
        }
 
+       ret = cpufreq_driver->online(policy);

But I wouldn't make this change, because that would require reworking ->init() in the driver too.

Let's continue assuming that ->init() will do ->online() if successful and so ->offline() is needed after a successful ->init() as well as after a successful ->online().

drivers/cpufreq/cpufreq.c static int cpufreq_online(unsigned int cpu) · line 1524

Quoted 13 lines
drivers/cpufreq/cpufreq.c +6 -0
+       if (ret) {
+               pr_debug("%s: %d: %d: ->online() failed\n", __func__, __LINE__,
+                        ret);
+               goto out_exit_policy;
+       }
+
        down_write(&policy->rwsem);
        /*
         * affected cpus must always be the one, which are online. We aren't
@@ -1518,6 +1522,9 @@ static int cpufreq_online(unsigned int cpu)
 
        up_write(&policy->rwsem);
 

So I think that a new label is needed here to avoid running ->offline() after a failing ->online().

Let me update the patch accordingly.

Quoted 6 lines
drivers/cpufreq/cpufreq.c +3 -0
+       if (cpufreq_driver->offline)
+               cpufreq_driver->offline(policy);
+
 out_exit_policy:
        if (cpufreq_driver->exit)
                cpufreq_driver->exit(policy);
#3 lore raw

PATCH v2 1/1 cpufreq: Make cpufreq_online() call driver->offline() on errors

In reply to Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>
Subject: [PATCH] cpufreq: Make cpufreq_online() call driver->offline() on errors

In the CPU removal path the ->offline() callback provided by the driver is always invoked before ->exit(), but in the cpufreq_online() error path it is not, so ->exit() is expected to somehow know the context in which it has been called and act accordingly.

That is less than straightforward, so make cpufreq_online() invoke the driver's ->offline() callback, if present, on errors before ->exit() too.

This only potentially affects intel_pstate.

Fixes 91a12e91dc39 ("cpufreq: Allow light-weight tear down and bring up of CPUs")

---

-> v2:
* Avoid calling ->offline() after a failing ->online().
* Add a comment regarding the expected state after calling ->init().
* Edit the changelog a bit.

---
 drivers/cpufreq/cpufreq.c |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
drivers/cpufreq/cpufreq.c +10 -1
Index: linux-pm/drivers/cpufreq/cpufreq.c
===================================================================
@@ -1367,9 +1367,14 @@ static int cpufreq_online(unsigned int c
 			goto out_free_policy;
 		}
 
+		/*
+		 * The initialization has succeeded and the policy is online.
+		 * If there is a problem with its frequency table, take it
+		 * offline and drop it.
+		 */
 		ret = cpufreq_table_validate_and_sort(policy);
 		if (ret)
-			goto out_exit_policy;
+			goto out_offline_policy;
 
 		/* related_cpus should at least include policy->cpus. */
 		cpumask_copy(policy->related_cpus, policy->cpus);
@@ -1515,6 +1520,10 @@ out_destroy_policy:
 
 	up_write(&policy->rwsem);
 
+out_offline_policy:
+	if (cpufreq_driver->offline)
+		cpufreq_driver->offline(policy);
+
 out_exit_policy:
 	if (cpufreq_driver->exit)
 		cpufreq_driver->exit(policy);
maintainer Acked-by on v2 1/1
#4 lore raw
Quoted 14 lines · Rafael J. Wysocki

From: Rafael J. Wysocki <[email protected]>
Subject: [PATCH] cpufreq: Make cpufreq_online() call driver->offline() on errors

In the CPU removal path the ->offline() callback provided by the driver is always invoked before ->exit(), but in the cpufreq_online() error path it is not, so ->exit() is expected to somehow know the context in which it has been called and act accordingly.

That is less than straightforward, so make cpufreq_online() invoke the driver's ->offline() callback, if present, on errors before ->exit() too.

This only potentially affects intel_pstate.

Quoted 8 lines
Fixes 91a12e91dc39 ("cpufreq: Allow light-weight tear down and bring up of CPUs")

---

-> v2:
* Avoid calling ->offline() after a failing ->online().
* Add a comment regarding the expected state after calling ->init().
* Edit the changelog a bit.

Signature · 1 line
viresh