rtla fixes for 6.5
cover rtla fixes for 6.5
Here are some fixes for rtla timerlat auto-analysis.
The first one zeroes the variable that accounts for the amount of thread interference when a new cycle begins.
The second fixes an imprecision on the IRQ delay account when the value is near 0.
The last one fixes the case when an IRQ interference for the next activation is considered for the previous one.
Daniel Bristot de Oliveira (3):
rtla/timerlat_aa: Zero thread sum after every sample analysis rtla/timerlat_aa: Fix negative irq delay rtla/timerlat_aa: Fix previous IRQ delay for IRQs that happens after
thread sample
tools/tracing/rtla/src/timerlat_aa.c | 32 ++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-)
Signature · 1 line
2.38.1
PATCH 1/3 rtla/timerlat_aa: Zero thread sum after every sample analysis
The thread thread_thread_sum accounts for thread interference during a single activation. It was not being zeroed, so it was accumulating thread interference over all activations.
It was not that visible when timerlat was the highest priority.
--- tools/tracing/rtla/src/timerlat_aa.c | 1 + 1 file changed, 1 insertion(+)
tools/tracing/rtla/src/timerlat_aa.c +1 -0
@@ -159,6 +159,7 @@ static int timerlat_aa_irq_latency(struct timerlat_aa_data *taa_data, taa_data->thread_nmi_sum =0 ; taa_data->thread_irq_sum =0 ; taa_data->thread_softirq_sum =0 ; + taa_data->thread_thread_sum =0 ; taa_data->thread_blocking_duration =0 ; taa_data->timer_irq_start_time =0 ; taa_data->timer_irq_duration =0 ;
Signature · 1 line
2.38.1
PATCH 2/3 rtla/timerlat_aa: Fix negative IRQ delay
In reply to Daniel Bristot de OliveiraWhen estimating the IRQ timer delay, we are dealing with two different clock sources: the external clock source that timerlat uses as a reference and the clock used by the tracer. There are also two moments: the time reading the clock and the timer in which the event is placed in the buffer (the trace event timestamp).
If the processor is slow or there is some hardware noise, the difference between the timestamp and the external clock, read can be longer than the IRQ handler delay, resulting in a negative time.
If so, set IRQ to start delay as 0. In the end, it is less near-zero and relevant then the noise.
--- tools/tracing/rtla/src/timerlat_aa.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
tools/tracing/rtla/src/timerlat_aa.c +17 -1
@@ -338,7 +338,23 @@ static int timerlat_aa_irq_handler(struct trace_seq *s, struct tep_record *recor taa_data->timer_irq_start_time = start; taa_data->timer_irq_duration = duration; - taa_data->timer_irq_start_delay = taa_data->timer_irq_start_time - expected_start; +/* +* We are dealing with two different clock sources: the +* external clock source that timerlat uses as a reference +* and the clock used by the tracer. There are also two +* moments: the time reading the clock and the timer in +* which the event is placed in the buffer (the trace +* event timestamp). If the processor is slow or there +* is some hardware noise, the difference between the +* timestamp and the external clock read can be longer +* than the IRQ handler delay, resulting in a negative +* time. If so, set IRQ start delay as 0. In the end, +* it is less relevant than the noise. +*/ +if (expected_start < taa_data->timer_irq_start_time) + taa_data->timer_irq_start_delay = taa_data->timer_irq_start_time - expected_start; +else + taa_data->timer_irq_start_delay =0 ;/* * not exit from idle.
Signature · 1 line
2.38.1
PATCH 3/3 rtla/timerlat_aa: Fix previous IRQ delay for IRQs that happens after thread sample
In reply to Daniel Bristot de Oliveiratimerlat auto-analysis takes note of all IRQs, before or after the execution of the timerlat thread.
Because we cannot go backward in the trace (we will fix it when moving to trace-cmd lib?), timerlat aa take note of the last IRQ execution in the waiting for the IRQ state, and then print it if it is executed after the expected timer IRQ starting time.
After the thread sample, the timerlat starts recording the next IRQs as "previous" irq for the next occurrence.
However, if an IRQ happens after the thread measurement but before the tracing stops, it is classified as a previous IRQ. That is not wrong, as it can be "previous" for the subsequent activation. What is wrong is considering it as a potential source for the last activation.
Ignore the IRQ interference that happens after the IRQ starting time for now. A future improvement for timerlat can be either keeping a list of previous IRQ execution or using the trace-cmd library. Still, it requires further investigation - it is a new feature.
--- tools/tracing/rtla/src/timerlat_aa.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tools/tracing/rtla/src/timerlat_aa.c +8 -5
@@ -545,7 +545,7 @@ static int timerlat_aa_kworker_start_handler(struct trace_seq *s, struct tep_recstatic void timerlat_thread_analysis (struct timerlat_aa_data *taa_data,int cpu,int irq_thresh,int thread_thresh) { -unsigned long long exp_irq_ts; +long long exp_irq_ts;int total;int irq;
@@ -562,12 +562,15 @@ static void timerlat_thread_analysis(struct timerlat_aa_data *taa_data, int cpu,/* * Expected IRQ arrival time using the trace clock as the base. +* +* TODO: Add a list of previous IRQ, and then run the list backwards. */ exp_irq_ts = taa_data->timer_irq_start_time - taa_data->timer_irq_start_delay; - -if (exp_irq_ts < taa_data->prev_irq_timstamp + taa_data->prev_irq_duration) -printf (" Previous IRQ interference: \t\t up to %9.2f us\n" , -ns_to_usf (taa_data->prev_irq_duration)); +if (exp_irq_ts < taa_data->prev_irq_timstamp + taa_data->prev_irq_duration) { +if (taa_data->prev_irq_timstamp < taa_data->timer_irq_start_time) +printf (" Previous IRQ interference: \t\t up to %9.2f us\n" , +ns_to_usf (taa_data->prev_irq_duration)); + }/* * The delay that the IRQ suffered before starting.
Signature · 1 line
2.38.1