Skip to contentSkip to conversation

perf/core: Fixes hung issue on perf stat command during cpu hotplug

RFC In review Kajol Jain 3 participants 3 messages Aug 26, 2020 – Aug 27, 2020 perf events

lore
Share & tools
lore

Kajol Jain

author
#0 lore raw

RFC v2 1/1 perf/core: Fixes hung issue on perf stat command during cpu hotplug

Commit 2ed6edd33a21 ("perf: Add cond_resched() to task_function_call()") added assignment of ret value as -EAGAIN in case function call to 'smp_call_function_single' fails.
For non-zero ret value, it did
'ret = !ret ? data.ret : -EAGAIN;', which always
assign -EAGAIN to ret and make second if condition useless.

In scenarios like when executing a perf stat with --per-thread option, and if any of the monitoring cpu goes offline, the 'smp_call_function_single' function could return -ENXIO, and with the above check, task_function_call hung and increases CPU
usage (because of repeated 'smp_call_function_single()')

Recration scenario:

	# perf stat -a --per-thread && (offline a CPU )

Patch here removes the tertiary condition added as part of that commit and added a check for NULL and -EAGAIN.

Fixes 2ed6edd33a21("perf: Add cond_resched() to task_function_call()")
---
 kernel/events/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Changelog:
- Remove addition of else in the first patch for
if(ret != -EAGAIN) condition.

kernel/events/core.c +2 -2
index 856d98c36f56..fe104fee097a 100644
@@ -115,8 +115,8 @@ task_function_call(struct task_struct *p, remote_function_f func, void *info)
 	for (;;) {
 		ret = smp_call_function_single(task_cpu(p), remote_function,
 					       &data, 1);
-		ret = !ret ? data.ret : -EAGAIN;
-
+		if(!ret)
+			ret = data.ret;
 		if (ret != -EAGAIN)
 			break;
 
Signature · 1 line
2.26.2
Reviewed-by on v2 1/1
#1 lore raw

Hi -

Quoted 35 lines · Kajol Jain

Commit 2ed6edd33a21 ("perf: Add cond_resched() to task_function_call()") added assignment of ret value as -EAGAIN in case function call to 'smp_call_function_single' fails.
For non-zero ret value, it did
'ret = !ret ? data.ret : -EAGAIN;', which always
assign -EAGAIN to ret and make second if condition useless.

In scenarios like when executing a perf stat with --per-thread option, and if any of the monitoring cpu goes offline, the 'smp_call_function_single' function could return -ENXIO, and with the above check, task_function_call hung and increases CPU
usage (because of repeated 'smp_call_function_single()')

Recration scenario:

	# perf stat -a --per-thread && (offline a CPU )

Patch here removes the tertiary condition added as part of that commit and added a check for NULL and -EAGAIN.

Fixes 2ed6edd33a21("perf: Add cond_resched() to task_function_call()")
---
  kernel/events/core.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

Changelog:
- Remove addition of else in the first patch for
if(ret != -EAGAIN) condition.

kernel/events/core.c +0 -0
index 856d98c36f56..fe104fee097a 100644

kernel/events/core.c task_function_call(struct task_struct *p, remote_function_f func, void *info) · line 118

Quoted 6 lines
kernel/events/core.c +1 -2
@@ -115,8 +115,8 @@ task_function_call(struct task_struct *p, remote_function_f func, void *info)
  	for (;;) {
  		ret = smp_call_function_single(task_cpu(p), remote_function,
  					       &data, 1);
-		ret = !ret ? data.ret : -EAGAIN;
-
+		if(!ret)
                  ^

Minor nit, you need a space after the if.

Also, the comment for the function says it returns @func's return val or -ESRCH. You could also add -ENXIO to that.

Thanks for the fix.

Quoted 5 lines
kernel/events/core.c +1 -0
+			ret = data.ret;
  		if (ret != -EAGAIN)
  			break;
 
Tested-by on v2 1/1
#2 lore raw
In reply to Kajol Jain
Quoted 14 lines · Kajol Jain

Commit 2ed6edd33a21 ("perf: Add cond_resched() to task_function_call()") added assignment of ret value as -EAGAIN in case function call to 'smp_call_function_single' fails.
For non-zero ret value, it did
'ret = !ret ? data.ret : -EAGAIN;', which always
assign -EAGAIN to ret and make second if condition useless.

In scenarios like when executing a perf stat with --per-thread option, and if any of the monitoring cpu goes offline, the 'smp_call_function_single' function could return -ENXIO, and with the above check, task_function_call hung and increases CPU
usage (because of repeated 'smp_call_function_single()')

Recration scenario:

Quoted 8 lines
	# perf stat -a --per-thread && (offline a CPU )

Patch here removes the tertiary condition added as part of that commit and added a check for NULL and -EAGAIN.

Fixes 2ed6edd33a21("perf: Add cond_resched() to task_function_call()")
Signature · 2 lines
Thanks and Regards
Srikar Dronamraju