Skip to contentSkip to conversation

kvm: fix preemption warnings in kvm_vcpu_block

PATCH In review Dominik Dingel 4 participants 5 messages Sep 17, 2015 – Sep 18, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kvm: fix preemption warnings in kvm_vcpu_block

Commit f78195129963 ("kvm: add halt_poll_ns module parameter") calls, with enabled preemption, single_task_running. When CONFIG_DEBUG_PREEMPT is enabled that will result in a debug_smp_processor_id() call.

---
 virt/kvm/kvm_main.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +7 -1
index 54534de..ce67dd6 100644
@@ -1971,6 +1971,7 @@ void kvm_vcpu_block(struct kvm_vcpu *vcpu)
 
 	start = cur = ktime_get();
 	if (vcpu->halt_poll_ns) {
+		bool solo;
 		ktime_t stop = ktime_add_ns(ktime_get(), vcpu->halt_poll_ns);
 
 		do {
@@ -1982,8 +1983,13 @@ void kvm_vcpu_block(struct kvm_vcpu *vcpu)
 				++vcpu->stat.halt_successful_poll;
 				goto out;
 			}
+
+			preempt_disable();
+			solo = single_task_running();
+			preempt_enable();
+
 			cur = ktime_get();
-		} while (single_task_running() && ktime_before(cur, stop));
+		} while (solo && ktime_before(cur, stop));
 	}
 
 	for (;;) {
Signature · 1 line
2.3.8
maintainer
#1 lore raw

single_task_running() vs. preemption warnings (was Re: [PATCH] kvm: fix preemption warnings in kvm_vcpu_block)

virt/kvm/kvm_main.c void kvm_vcpu_block(struct kvm_vcpu *vcpu) · line 1986

Quoted 6 lines · Dominik Dingel
virt/kvm/kvm_main.c +4 -1
+			preempt_disable();
+			solo = single_task_running();
+			preempt_enable();
+
 			cur = ktime_get();
-		} while (single_task_running() && ktime_before(cur, stop));

That's the obvious way to fix it, but the TOCTTOU problem (which was in the buggy code too) is obvious too. :) And the only other user of single_task_running() in drivers/crypto/mcryptd.c has the same issue.

In fact, because of the way the function is used ("maybe I can do a little bit of work before going to sleep") it will likely be called many times in a loop. This in turn means that:

- any wrong result due to a concurrent process migration would be rectified very soon

- preempt_disable()/preempt_enable() can actually be just as expensive or more expensive than single_task_running() itself.

Therefore, I wonder if single_task_running() should just use raw_smp_processor_id(). At least the TOCTTOU issue can be clearly documented in the function comment, instead of being hidden behind each of the callers.

Thanks,

Paolo

#2 lore raw

Re: single_task_running() vs. preemption warnings (was Re: [PATCH] kvm: fix preemption warnings in kvm_vcpu_block)

Quoted 13 lines · On Thu, 17 Sep 2015 18:45:00 +0200 Paolo Bonzini <[email protected]> wrote:

virt/kvm/kvm_main.c void kvm_vcpu_block(struct kvm_vcpu *vcpu) · line 1986

Quoted 6 lines · Dominik Dingel
virt/kvm/kvm_main.c +4 -1
+			preempt_disable();
+			solo = single_task_running();
+			preempt_enable();
+
 			cur = ktime_get();
-		} while (single_task_running() && ktime_before(cur, stop));

That's the obvious way to fix it, but the TOCTTOU problem (which was in the buggy code too) is obvious too. :) And the only other user of single_task_running() in drivers/crypto/mcryptd.c has the same issue.

Right, worst thing we fly another round.

I am not sure about the case for mcryptd.c. I think it might be that the worker there is bounded to one cpu and will not be migrated.

I really need to look more in the details what is happening with that worker.

Quoted 7 lines

In fact, because of the way the function is used ("maybe I can do a little bit of work before going to sleep") it will likely be called many times in a loop. This in turn means that:

- any wrong result due to a concurrent process migration would be rectified very soon

Quoted 7 lines

- preempt_disable()/preempt_enable() can actually be just as expensive or more expensive than single_task_running() itself.

Therefore, I wonder if single_task_running() should just use raw_smp_processor_id(). At least the TOCTTOU issue can be clearly documented in the function comment, instead of being hidden behind each of the callers.

Yes to be useful it should probably call raw_smp_processor_id,
and as a lot of code actually already does just does that I do not really see much down sides.

@Tim, would it be okay if I change single_task_running and add a specific comment on top?

Quoted 4 lines

Thanks,

Paolo

#3 lore raw

Re: single_task_running() vs. preemption warnings (was Re: [PATCH] kvm: fix preemption warnings in kvm_vcpu_block)

Quoted 39 lines · Dominik Dingel
Quoted 13 lines · On Thu, 17 Sep 2015 18:45:00 +0200 Paolo Bonzini <[email protected]> wrote:

virt/kvm/kvm_main.c void kvm_vcpu_block(struct kvm_vcpu *vcpu) · line 1986

Quoted 6 lines · Dominik Dingel
virt/kvm/kvm_main.c +4 -1
+			preempt_disable();
+			solo = single_task_running();
+			preempt_enable();
+
 			cur = ktime_get();
-		} while (single_task_running() && ktime_before(cur, stop));

That's the obvious way to fix it, but the TOCTTOU problem (which was in the buggy code too) is obvious too. :) And the only other user of single_task_running() in drivers/crypto/mcryptd.c has the same issue.

Right, worst thing we fly another round.

I am not sure about the case for mcryptd.c. I think it might be that the worker there is bounded to one cpu and will not be migrated.

I really need to look more in the details what is happening with that worker.

Quoted 7 lines

In fact, because of the way the function is used ("maybe I can do a little bit of work before going to sleep") it will likely be called many times in a loop. This in turn means that:

- any wrong result due to a concurrent process migration would be rectified very soon

Quoted 7 lines

- preempt_disable()/preempt_enable() can actually be just as expensive or more expensive than single_task_running() itself.

Therefore, I wonder if single_task_running() should just use raw_smp_processor_id(). At least the TOCTTOU issue can be clearly documented in the function comment, instead of being hidden behind each of the callers.

Quoted 5 lines

Yes to be useful it should probably call raw_smp_processor_id,
and as a lot of code actually already does just does that I do not really see much down sides.

@Tim, would it be okay if I change single_task_running and add a specific comment on top?

I have no objection to change single_task_running to use raw_smp_processor_id. The worker in mcryptd is bound to
the cpu so it has no migration/preemption issue. So it shouldn't care which smp_processor_id version is being used. Yes, please add a comment to alert the user of this caveat should you change single_task_running.

Thanks.

Tim

maintainer
#4 lore raw

Re: single_task_running() vs. preemption warnings (was Re: [PATCH] kvm: fix preemption warnings in kvm_vcpu_block)

Quoted 5 lines · Tim Chen

I have no objection to change single_task_running to use raw_smp_processor_id. The worker in mcryptd is bound to
the cpu so it has no migration/preemption issue. So it shouldn't care which smp_processor_id version is being used. Yes, please add a comment to alert the user of this caveat should you change single_task_running.

We actually have raw_rq() for that, and the whole if thing looks rather superfluous. So something like the below, except with a suitable comment on and tested etc.. ;-)

---
 kernel/sched/core.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
kernel/sched/core.c +1 -4
index 6ab415aa15c4..f39c0498e284 100644
@@ -2666,10 +2666,7 @@ unsigned long nr_running(void)
  */
 bool single_task_running(void)
 {
-	if (cpu_rq(smp_processor_id())->nr_running == 1)
-		return true;
-	else
-		return false;
+	return raw_rq()->nr_running == 1;
 }
 EXPORT_SYMBOL(single_task_running);