Skip to contentSkip to conversation

thinktime: Avoid calculating a negative time left to wait

PATCH Applied Michael Kelley (LINUX) 2 participants 2 messages Jul 14, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 thinktime: Avoid calculating a negative time left to wait

When the thinktime_spin option specifies a value that is within a few milliseconds of the thinktime value, in handle_thinktime() it's possible in a VM environment for the duration of usec_spin() to exceed the thinktime value. While doing usec_spin(), the vCPU could get de-scheduled or the hypervisor could steal CPU time from the vCPU. When the guest vCPU runs after being scheduled again, it may read the clock and find that more time has elapsed than intended. In such a case, the time left to wait could be calculated as a negative value. Subsequent calculations then go awry because the time left is cast as unsigned.

Fix this by detecting when the time left would go negative and just set it to zero.

Fixes 1a9bf8146 ("Add option to ignore thinktime for rated IO")
---
 backend.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
backend.c +10 -1
index d67a4a07..b06a11a5 100644
@@ -897,7 +897,16 @@ static void handle_thinktime(struct thread_data *td, enum fio_ddir ddir,
 	if (left)
 		total = usec_spin(left);
 
-	left = td->o.thinktime - total;
+	/*
+	 * usec_spin() might run for slightly longer than intended in a VM
+	 * where the vCPU could get descheduled or the hypervisor could steal
+	 * CPU time. Ensure "left" doesn't become negative.
+	 */
+	if (total < td->o.thinktime)
+		left = td->o.thinktime - total;
+	else
+		left = 0;
+
 	if (td->o.timeout) {
 		runtime_left = td->o.timeout - utime_since_now(&td->epoch);
 		if (runtime_left < (unsigned long long)left)
Signature · 1 line
2.25.1
Applied
#1 lore raw
Quoted 33 lines · Michael Kelley (LINUX)

When the thinktime_spin option specifies a value that is within a few milliseconds of the thinktime value, in handle_thinktime() it's possible in a VM environment for the duration of usec_spin() to exceed the thinktime value. While doing usec_spin(), the vCPU could get de-scheduled or the hypervisor could steal CPU time from the vCPU. When the guest vCPU runs after being scheduled again, it may read the clock and find that more time has elapsed than intended. In such a case, the time left to wait could be calculated as a negative value. Subsequent calculations then go awry because the time left is cast as unsigned.

Fix this by detecting when the time left would go negative and just set it to zero.

Fixes 1a9bf8146 ("Add option to ignore thinktime for rated IO")
---
  backend.c | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
backend.c +3 -1
index d67a4a07..b06a11a5 100644
@@ -897,7 +897,16 @@ static void handle_thinktime(struct thread_data *td, enum fio_ddir ddir,
  	if (left)
  		total = usec_spin(left);
 
-	left = td->o.thinktime - total;
+	/*
+	 * usec_spin() might run for slightly longer than intended in a VM
+	 * where the vCPU could get descheduled or the hypervisor could steal

backend.c static void handle_thinktime(struct thread_data *td, enum fio_ddir ddir, · line 912

Quoted 10 lines
backend.c +7 -0
+	 * CPU time. Ensure "left" doesn't become negative.
+	 */
+	if (total < td->o.thinktime)
+		left = td->o.thinktime - total;
+	else
+		left = 0;
+
  	if (td->o.timeout) {
  		runtime_left = td->o.timeout - utime_since_now(&td->epoch);
  		if (runtime_left < (unsigned long long)left)

Applied, thanks!

Vincent