Skip to contentSkip to conversation

tools/power turbostat: Make interval calculation per thread to reduce jitter

PATCH In review Ghannam, Yazen 2 participants 4 messages Mar 25, 2019 – Aug 31, 2019 Turbostat utility

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tools/power turbostat: Make interval calculation per thread to reduce jitter

From: Yazen Ghannam <[email protected]>

Turbostat currently normalizes TSC and other values by dividing by an interval. This interval is the delta between the start of one global (all counters on all CPUs) sampling and the start of another. However, this introduces a lot of jitter into the data.

In order to reduce jitter, the interval calculation should be based on timestamps taken per thread and close to the start of the thread's sampling.

Define a per thread time value to hold the delta between samples taken on the thread.

Use the timestamp taken at the beginning of sampling to calculate the delta.

Move the thread's beginning timestamp to after the CPU migration to avoid jitter due to the migration.

Use the global time delta for the average time delta.

---
 tools/power/x86/turbostat/turbostat.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
tools/power/x86/turbostat/turbostat.c +10 -3
index c3fad065c89c..6ebcc420021f 100644
@@ -166,6 +166,7 @@ size_t cpu_present_setsize, cpu_affinity_setsize, cpu_subset_size;
 struct thread_data {
 	struct timeval tv_begin;
 	struct timeval tv_end;
+	struct timeval tv_delta;
 	unsigned long long tsc;
 	unsigned long long aperf;
 	unsigned long long mperf;
@@ -896,7 +897,7 @@ int format_counters(struct thread_data *t, struct core_data *c,
 	if (DO_BIC(BIC_TOD))
 		outp += sprintf(outp, "%10ld.%06ld\t", t->tv_end.tv_sec, t->tv_end.tv_usec);
 
-	interval_float = tv_delta.tv_sec + tv_delta.tv_usec/1000000.0;
+	interval_float = t->tv_delta.tv_sec + t->tv_delta.tv_usec/1000000.0;
 
 	tsc = t->tsc * tsc_tweak;
 
@@ -1279,6 +1280,7 @@ delta_thread(struct thread_data *new, struct thread_data *old,
 	 * over-write old w/ new so we can print end of interval values
 	 */
 
+	timersub(&new->tv_begin, &old->tv_begin, &old->tv_delta);
 	old->tv_begin = new->tv_begin;
 	old->tv_end = new->tv_end;
 
@@ -1374,6 +1376,8 @@ void clear_counters(struct thread_data *t, struct core_data *c, struct pkg_data
 	t->tv_begin.tv_usec = 0;
 	t->tv_end.tv_sec = 0;
 	t->tv_end.tv_usec = 0;
+	t->tv_delta.tv_sec = 0;
+	t->tv_delta.tv_usec = 0;
 
 	t->tsc = 0;
 	t->aperf = 0;
@@ -1540,6 +1544,9 @@ void compute_average(struct thread_data *t, struct core_data *c,
 
 	for_all_cpus(sum_counters, t, c, p);
 
+	/* Use the global time delta for the average. */
+	average.threads.tv_delta = tv_delta;
+
 	average.threads.tsc /= topo.num_cpus;
 	average.threads.aperf /= topo.num_cpus;
 	average.threads.mperf /= topo.num_cpus;
@@ -1729,13 +1736,13 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p)
 	struct msr_counter *mp;
 	int i;
 
-	gettimeofday(&t->tv_begin, (struct timezone *)NULL);
-
 	if (cpu_migrate(cpu)) {
 		fprintf(outf, "Could not migrate to CPU %d\n", cpu);
 		return -1;
 	}
 
+	gettimeofday(&t->tv_begin, (struct timezone *)NULL);
+
 	if (first_counter_read)
 		get_apic_id(t);
 retry:
Signature · 1 line
2.17.1
#1 lore raw
Quoted 24 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Ghannam, Yazen
Sent: Monday, March 25, 2019 12:33 PM
To: [email protected]
Cc: Ghannam, Yazen <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] tools/power turbostat: Make interval calculation per thread to reduce jitter

From: Yazen Ghannam <[email protected]>

Turbostat currently normalizes TSC and other values by dividing by an interval. This interval is the delta between the start of one global (all counters on all CPUs) sampling and the start of another. However, this introduces a lot of jitter into the data.

In order to reduce jitter, the interval calculation should be based on timestamps taken per thread and close to the start of the thread's sampling.

Define a per thread time value to hold the delta between samples taken on the thread.

Use the timestamp taken at the beginning of sampling to calculate the delta.

Quoted 7 lines

Move the thread's beginning timestamp to after the CPU migration to avoid jitter due to the migration.

Use the global time delta for the average time delta.

---

Hi Len,

Any comments on this patch?

Thanks,
Yazen

#2 lore raw
Quoted 34 lines

-----Original Message-----
From: Ghannam, Yazen <[email protected]>
Sent: Tuesday, April 23, 2019 12:53 PM
To: Ghannam, Yazen <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; Len Brown <[email protected]>
Subject: RE: [PATCH] tools/power turbostat: Make interval calculation per thread to reduce jitter

Quoted 27 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Ghannam, Yazen
Sent: Monday, March 25, 2019 12:33 PM
To: [email protected]
Cc: Ghannam, Yazen <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] tools/power turbostat: Make interval calculation per thread to reduce jitter

From: Yazen Ghannam <[email protected]>

Turbostat currently normalizes TSC and other values by dividing by an interval. This interval is the delta between the start of one global (all counters on all CPUs) sampling and the start of another. However, this introduces a lot of jitter into the data.

In order to reduce jitter, the interval calculation should be based on timestamps taken per thread and close to the start of the thread's sampling.

Define a per thread time value to hold the delta between samples taken on the thread.

Use the timestamp taken at the beginning of sampling to calculate the delta.

Move the thread's beginning timestamp to after the CPU migration to avoid jitter due to the migration.

Quoted 9 lines
Quoted 4 lines

Use the global time delta for the average time delta.

---

Hi Len,

Any comments on this patch?

Hi Len,

Just wanted to check in. Do you have any comments on this patch?

Thanks,
Yazen

maintainer
#3 lore raw

Yeah, I like this patch, and I have applied it.

thanks!
-Len

Quoted 51 lines · Yazen
Quoted 35 lines

-----Original Message-----
From: Ghannam, Yazen <[email protected]>
Sent: Tuesday, April 23, 2019 12:53 PM
To: Ghannam, Yazen <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; Len Brown <[email protected]>
Subject: RE: [PATCH] tools/power turbostat: Make interval calculation per thread to reduce jitter

Quoted 27 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Ghannam, Yazen
Sent: Monday, March 25, 2019 12:33 PM
To: [email protected]
Cc: Ghannam, Yazen <[email protected]>; [email protected]; [email protected]
Subject: [PATCH] tools/power turbostat: Make interval calculation per thread to reduce jitter

From: Yazen Ghannam <[email protected]>

Turbostat currently normalizes TSC and other values by dividing by an interval. This interval is the delta between the start of one global (all counters on all CPUs) sampling and the start of another. However, this introduces a lot of jitter into the data.

In order to reduce jitter, the interval calculation should be based on timestamps taken per thread and close to the start of the thread's sampling.

Define a per thread time value to hold the delta between samples taken on the thread.

Use the timestamp taken at the beginning of sampling to calculate the delta.

Move the thread's beginning timestamp to after the CPU migration to avoid jitter due to the migration.

Quoted 9 lines
Quoted 4 lines

Use the global time delta for the average time delta.

---

Hi Len,

Any comments on this patch?

Hi Len,

Just wanted to check in. Do you have any comments on this patch?

Thanks,
Yazen

Signature · 1 line
Len Brown, Intel Open Source Technology Center