tick/sched: Ensure quiet_vmstat() is called when the idle tick was stopped too

Hi Frederic,

If I understand correctly, in the context of the idle task and a nohz_full CPU, quiet_vmstat() can be called: before stopping the idle tick, entering an idle state and on exit. In particular, for the latter case, when the idle task is required to reschedule, the idle tick can remain stopped and the timer expiration time endless i.e., KTIME_MAX. Now, indeed before a nohz_full CPU enters an idle state, CPU-specific vmstat counters should be processed to ensure the respective values have been reset and folded into the zone specific vm_stat[]. That being said, it can only occur when:
the idle tick was previously stopped, and reprogramming of the timer is not required.

A customer provided some evidence which indicates that the idle tick was stopped; albeit, CPU-specific vmstat counters still remained populated. Thus one can only assume quiet_vmstat() was not invoked on return to the idle loop.

Unfortunately, I suspect this divergence might erroneously prevent a reclaim attempt by kswapd. If the number of zone specific free pages are below their per-cpu drift value then zone_page_state_snapshot() is used to compute a more accurate view of the aforementioned statistic. Thus any task blocked on the NUMA node specific pfmemalloc_wait queue will be unable to make significant progress via direct reclaim unless it is killed after being woken up by kswapd (see throttle_direct_reclaim()). That being said, eventually reclaim should give up if the conditions are correct, no?

Consider the following theoretical scenario:

1. CPU Y migrated running task A to CPU X that was
in an idle state i.e. waiting for an IRQ - not polling; marked the current task on CPU X to need/or require a reschedule i.e., set TIF_NEED_RESCHED and invoked a reschedule IPI to CPU X (see sched_move_task())

2. CPU X acknowledged the reschedule IPI from CPU Y;
generic idle loop code noticed the TIF_NEED_RESCHED flag against the idle task and attempts to exit of the loop and calls the main scheduler function i.e. __schedule().

Since the idle tick was previously stopped no scheduling-clock tick would occur. So, no deferred timers would be handled

3. Post transition to kernel execution Task A
running on CPU Y, indirectly released a few pages (e.g. see __free_one_page()); CPU Y's vm_stat_diff[NR_FREE_PAGES] was updated and zone specific vm_stat[] update was deferred as per the CPU-specific stat threshold

4. Task A does invoke exit(2) and the kernel does
remove the task from the run-queue; the idle task was selected to execute next since there are no other runnable tasks assigned to the given CPU (see pick_next_task() and pick_next_task_idle())

5. On return to the idle loop since the idle tick
was already stopped and can remain so (see [1] below) e.g. no pending soft IRQs, no attempt is made to zero and fold CPU Y's vmstat counters since reprogramming of the scheduling-clock tick is not required/or needed (see [2])

...
do_idle
{

__current_set_polling()
tick_nohz_idle_enter()

while (!need_resched()) {

local_irq_disable()

...

/* No polling or broadcast event */ cpuidle_idle_call()
{

if (cpuidle_not_available(drv, dev)) {
tick_nohz_idle_stop_tick()
__tick_nohz_idle_stop_tick(this_cpu_ptr(&tick_cpu_sched)) {
int cpu = smp_processor_id()

if (ts->timer_expires_base)
expires = ts->timer_expires
else if (can_stop_idle_tick(cpu, ts))
(1) -------> expires = tick_nohz_next_event(ts, cpu)
else
return

ts->idle_calls++

if (expires > 0LL) {

tick_nohz_stop_tick(ts, cpu) {

if (ts->tick_stopped && (expires == ts->next_tick)) {
(2) -------> if (tick == KTIME_MAX || ts->next_tick ==
hrtimer_get_expires(&ts->sched_timer)) return
}
...
}

The idea with this patch is to ensure refresh_cpu_vm_stats(false) is called on return to the idle loop when the idle tick was previously stopped.

Any feedback/or testing would be appreciated.

Signed-off-by Aaron Tomlin <[email protected]>

---
 kernel/time/tick-sched.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

kernel/time/tick-sched.c +3 -1

index 17a283ce2b20..61874df064b6 100644
@@ -876,6 +876,9 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu)
 		ts->do_timer_last = 0;
 	}
 
+	/* Attempt to fold when the idle tick is stopped or not */
+	quiet_vmstat();
+
 	/* Skip reprogram of event if its not changed */
 	if (ts->tick_stopped && (expires == ts->next_tick)) {
 		/* Sanity check: make sure clockevent is actually programmed */

@@ -897,7 +900,6 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu)
 	 */
 	if (!ts->tick_stopped) {
 		calc_load_nohz_start();
-		quiet_vmstat();
 
 		ts->last_tick = hrtimer_get_expires(&ts->sched_timer);
 		ts->tick_stopped = 1;

Signature · 1 line

2.34.1

As I said earlier, I don't think you want to call quiet_vmstat() unconditionally. And I don't think this will catch the cases you are trying to fix. Once the tick is stopped tick_nohz_stop_tick should not be getting called again until it's been restarted.

Quoted 4 lines · Phil Auld

As I said earlier, I don't think you want to call quiet_vmstat() unconditionally. And I don't think this will catch the cases you are trying to fix. Once the tick is stopped tick_nohz_stop_tick should not be getting called again until it's been restarted.

Phil,

Sorry about the delay. If I understand correctly, I see a scenario by which tick_nohz_stop_tick() can be called on transition/or exit from idle (e.g. default_idle_call()):

1. The idle/or scheduling-clock was previously
stopped

2. It is considered safe for the scheduling-clock
tick to remain "stopped"/or omitted; no need to reprogram and enable a periodic tick (e.g. no queued/or expired pending timer)

...
do_idle
cpuidle_idle_call
{

...

.-- default_idle_call
| arch_cpu_idle
| goto exit_idle
|
| exit_idle:
| __current_set_polling()
|
| }
| tick_nohz_idle_exit()
| {
|
| tick_stopped = ts->tick_stopped
|
| if (tick_stopped)
| tick_nohz_idle_update_tick(ts, now)
| if (tick_nohz_full_cpu(smp_processor_id()))
| __tick_nohz_full_update_tick(ts, now)
| {
| int cpu = smp_processor_id()
|
| if (can_stop_full_tick(cpu, ts))
| tick_nohz_stop_sched_tick(ts, cpu)
| if (tick_nohz_next_event(ts, cpu))
'-- tick_nohz_stop_tick(ts, cpu)
}
}

If I understand correctly, __tick_nohz_full_update_tick() can return with no changes to the current tick (e.g. expire time == KTIME_MAX), no?

Kind regards,

Signature · 1 line

Aaron Tomlin

Quoted 4 lines · Phil Auld

As I said earlier, I don't think you want to call quiet_vmstat() unconditionally. And I don't think this will catch the cases you are trying to fix. Once the tick is stopped tick_nohz_stop_tick should not be getting called again until it's been restarted.

Phil,

Sorry about the delay. If I understand correctly, I see a scenario by which tick_nohz_stop_tick() can be called on transition/or exit from idle (e.g. default_idle_call()):

1. The idle/or scheduling-clock was previously
stopped

2. It is considered safe for the scheduling-clock
tick to remain "stopped"/or omitted; no need to reprogram and enable a periodic tick (e.g. no queued/or expired pending timer)

...
do_idle
cpuidle_idle_call
{

...

.-- default_idle_call
| arch_cpu_idle
| goto exit_idle
|
| exit_idle:
| __current_set_polling()
|
| }
| tick_nohz_idle_exit()
| {
|
| tick_stopped = ts->tick_stopped
|
| if (tick_stopped)
| tick_nohz_idle_update_tick(ts, now)
| if (tick_nohz_full_cpu(smp_processor_id()))
| __tick_nohz_full_update_tick(ts, now)
| {
| int cpu = smp_processor_id()
|
| if (can_stop_full_tick(cpu, ts))
| tick_nohz_stop_sched_tick(ts, cpu)
| if (tick_nohz_next_event(ts, cpu))
'-- tick_nohz_stop_tick(ts, cpu)
}
}

If I understand correctly, __tick_nohz_full_update_tick() can return with no changes to the current tick (e.g. expire time == KTIME_MAX), no?

Kind regards,

Signature · 1 line

Aaron Tomlin

Hmm, but does it matter? The issue seem to be that we can enter in idle loop without flushing vmstat. Or am I missing something else?

Hi Frederic,

If I understand correctly, in the context of the idle task and a nohz_full CPU, quiet_vmstat() can be called: before stopping the idle tick, entering an idle state and on exit. In particular, for the latter case, when the idle task is required to reschedule, the idle tick can remain stopped and the timer expiration time endless i.e., KTIME_MAX. Now, indeed before a nohz_full CPU enters an idle state, CPU-specific vmstat counters should be processed to ensure the respective values have been reset and folded into the zone specific vm_stat[]. That being said, it can only occur when:
the idle tick was previously stopped, and reprogramming of the timer is not required.

A customer provided some evidence which indicates that the idle tick was stopped; albeit, CPU-specific vmstat counters still remained populated. Thus one can only assume quiet_vmstat() was not invoked on return to the idle loop.

Unfortunately, I suspect this divergence might erroneously prevent a reclaim attempt by kswapd. If the number of zone specific free pages are below their per-cpu drift value then zone_page_state_snapshot() is used to compute a more accurate view of the aforementioned statistic. Thus any task blocked on the NUMA node specific pfmemalloc_wait queue will be unable to make significant progress via direct reclaim unless it is killed after being woken up by kswapd (see throttle_direct_reclaim()). That being said, eventually reclaim should give up if the conditions are correct, no?

So, to make sure I understand, the issue is that with nohz_full, we may well enter into the idle loop with the tick already stopped. We may also exit from idle without restarting the tick (again only with nohz_full). And so this can cause the vmstat to not be flushed upon idle entry. Right?

Quoted 5 lines

A customer provided some evidence which indicates that the idle tick was stopped; albeit, CPU-specific vmstat counters still remained populated. Thus one can only assume quiet_vmstat() was not invoked on return to the idle loop.

Quoted 9 lines

Unfortunately, I suspect this divergence might erroneously prevent a reclaim attempt by kswapd. If the number of zone specific free pages are below their per-cpu drift value then zone_page_state_snapshot() is used to compute a more accurate view of the aforementioned statistic. Thus any task blocked on the NUMA node specific pfmemalloc_wait queue will be unable to make significant progress via direct reclaim unless it is killed after being woken up by kswapd (see throttle_direct_reclaim()). That being said, eventually reclaim should give up if the conditions are correct, no?

Now if quiet_vmstat() isn't called, the vmstat_work should fix this later, right? Or does that happen too late perhaps?

Quoted 4 lines · Frederic Weisbecker

So, to make sure I understand, the issue is that with nohz_full, we may well enter into the idle loop with the tick already stopped. We may also exit from idle without restarting the tick (again only with nohz_full). And so this can cause the vmstat to not be flushed upon idle entry. Right?

Hi Frederic,

Yes - this is exactly it.

Quoted 14 lines

Quoted 14 lines

A customer provided some evidence which indicates that the idle tick was stopped; albeit, CPU-specific vmstat counters still remained populated. Thus one can only assume quiet_vmstat() was not invoked on return to the idle loop.

Unfortunately, I suspect this divergence might erroneously prevent a reclaim attempt by kswapd. If the number of zone specific free pages are below their per-cpu drift value then zone_page_state_snapshot() is used to compute a more accurate view of the aforementioned statistic. Thus any task blocked on the NUMA node specific pfmemalloc_wait queue will be unable to make significant progress via direct reclaim unless it is killed after being woken up by kswapd (see throttle_direct_reclaim()). That being said, eventually reclaim should give up if the conditions are correct, no?

Quoted 2 lines

Now if quiet_vmstat() isn't called, the vmstat_work should fix this later, right? Or does that happen too late perhaps?

If I understand correctly, in the context of nohz_full, since such work is deferred, it will only be handled in a scenario when the periodic/or scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit from idle.

Quoted 4 lines

If I understand correctly, in the context of nohz_full, since such work is deferred, it will only be handled in a scenario when the periodic/or scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit from idle.

Oh I see, it's a deferrable delayed work...
Then I can see two other issues:

1) Can an interrupt in idle modify the vmstat and thus trigger the need to flush it? I believe it's the case and then the problem goes beyond nohz_full because if the idle interrupt fired while the tick is stopped and didn't set TIF_RESCHED, we go back to sleep without calling quiet_vmstat().

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.
Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...

Quoted 5 lines · Frederic Weisbecker

Quoted 4 lines

If I understand correctly, in the context of nohz_full, since such work is deferred, it will only be handled in a scenario when the periodic/or scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit from idle.

Quoted 7 lines

Oh I see, it's a deferrable delayed work...
Then I can see two other issues:

1) Can an interrupt in idle modify the vmstat and thus trigger the need to flush it? I believe it's the case and then the problem goes beyond nohz_full because if the idle interrupt fired while the tick is stopped and didn't set TIF_RESCHED, we go back to sleep without calling quiet_vmstat().

Yes: e.g. a nohz_full CPU, in idle code, could indeed receive a reschedule IPI; re-enable local IRQs and generic idle code sees the TIF_NEED_RESCHED flag against the idle task. Additionally, the selected task could indirectly released a few pages [to satisfy a low-memory condition] and modify CPU-specific vmstat data i.e. vm_stat_diff[NR_FREE_PAGES].

Quoted 7 lines

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.
Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...

If I understand correctly, I agree. For the latter case, nothing can be done unfortunately since the scheduling-clock tick is stopped.

Quoted 6 lines · Aaron Tomlin

Quoted 5 lines · Frederic Weisbecker

Quoted 4 lines

If I understand correctly, in the context of nohz_full, since such work is deferred, it will only be handled in a scenario when the periodic/or scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit from idle.

Quoted 5 lines

Quoted 5 lines

Oh I see, it's a deferrable delayed work...
Then I can see two other issues:

1) Can an interrupt in idle modify the vmstat and thus trigger the need to flush it?

Quoted 19 lines

Quoted 3 lines

I believe it's the case and then the problem goes beyond nohz_full because if the idle interrupt fired while the tick is stopped and didn't set TIF_RESCHED, we go back to sleep without calling quiet_vmstat().

Yes: e.g. a nohz_full CPU, in idle code, could indeed receive a reschedule IPI; re-enable local IRQs and generic idle code sees the TIF_NEED_RESCHED flag against the idle task. Additionally, the selected task could indirectly released a few pages [to satisfy a low-memory condition] and modify CPU-specific vmstat data i.e. vm_stat_diff[NR_FREE_PAGES].

Quoted 7 lines

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.
Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...

Quoted 2 lines

If I understand correctly, I agree. For the latter case, nothing can be done unfortunately since the scheduling-clock tick is stopped.

Hi Frederic,

As far vmstat_updateas I understand, in the context of nohz_full, options are indeed limited; albeit, if we can ensure CPU-specific vmstat data is folded on return to idle [when required] then this should be good enough.

Quoted 7 lines · Aaron Tomlin

Quoted 6 lines · Aaron Tomlin

Quoted 5 lines · Frederic Weisbecker

Quoted 4 lines

If I understand correctly, in the context of nohz_full, since such work is deferred, it will only be handled in a scenario when the periodic/or scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit from idle.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Oh I see, it's a deferrable delayed work...
Then I can see two other issues:

1) Can an interrupt in idle modify the vmstat and thus trigger the need to flush it?

Yes. Page allocation and page freeing for example.

   6   3730  ../mm/page_alloc.c <<rmqueue>>
             __mod_zone_freepage_state(zone, -(1 << order),
   4   1096  ../mm/page_alloc.c <<__free_one_page>>
             __mod_zone_freepage_state(zone, -(1 << order),

Quoted 18 lines

Quoted 18 lines

Quoted 3 lines

I believe it's the case and then the problem goes beyond nohz_full because if the idle interrupt fired while the tick is stopped and didn't set TIF_RESCHED, we go back to sleep without calling quiet_vmstat().

Yes: e.g. a nohz_full CPU, in idle code, could indeed receive a reschedule IPI; re-enable local IRQs and generic idle code sees the TIF_NEED_RESCHED flag against the idle task. Additionally, the selected task could indirectly released a few pages [to satisfy a low-memory condition] and modify CPU-specific vmstat data i.e. vm_stat_diff[NR_FREE_PAGES].

Quoted 7 lines

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.
Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.
Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...

Can't grasp the sentence above "The problem is that ...". What rule?

Quoted 7 lines · Aaron Tomlin

Quoted 6 lines · Aaron Tomlin

Quoted 5 lines · Frederic Weisbecker

Quoted 4 lines

If I understand correctly, in the context of nohz_full, since such work is deferred, it will only be handled in a scenario when the periodic/or scheduling-clock tick is enabled i.e. the timer was reprogrammed on exit from idle.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Oh I see, it's a deferrable delayed work...
Then I can see two other issues:

1) Can an interrupt in idle modify the vmstat and thus trigger the need to flush it?

Yes. Page allocation and page freeing for example.

   6   3730  ../mm/page_alloc.c <<rmqueue>>
             __mod_zone_freepage_state(zone, -(1 << order),
   4   1096  ../mm/page_alloc.c <<__free_one_page>
             __mod_zone_freepage_state(zone, -(1 << order),

Quoted 19 lines

Quoted 19 lines

Quoted 3 lines

I believe it's the case and then the problem goes beyond nohz_full because if the idle interrupt fired while the tick is stopped and didn't set TIF_RESCHED, we go back to sleep without calling quiet_vmstat().

Yes: e.g. a nohz_full CPU, in idle code, could indeed receive a reschedule IPI; re-enable local IRQs and generic idle code sees the TIF_NEED_RESCHED flag against the idle task. Additionally, the selected task could indirectly released a few pages [to satisfy a low-memory condition] and modify CPU-specific vmstat data i.e. vm_stat_diff[NR_FREE_PAGES].

Quoted 7 lines

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.
Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...

Quoted 8 lines

Quoted 2 lines

If I understand correctly, I agree. For the latter case, nothing can be done unfortunately since the scheduling-clock tick is stopped.

Hi Frederic,

As far vmstat_updateas I understand, in the context of nohz_full, options are indeed limited; albeit, if we can ensure CPU-specific vmstat data is folded on return to idle [when required] then this should be good enough.

I suppose the desired behaviour, with the deferred timer for vmstat_sync, is:

"Allow the per-CPU vmstats to be out of sync, but for a maximum of sysctl_stat_interval".

But Aaron, vmstat_shepherd should be ensuring that per-CPU vmstat_update work are queued, if the per-CPU vmstat are out of sync.

And:

static void
trigger_dyntick_cpu(struct timer_base *base, struct timer_list *timer)

{
        if (!is_timers_nohz_active())
                return;

        /*
         * TODO: This wants some optimizing similar to the code below, but we
         * will do that when we switch from push to pull for deferrable timers.
         */
        if (timer->flags & TIMER_DEFERRABLE) {
                if (tick_nohz_full_cpu(base->cpu))
                        wake_up_nohz_cpu(base->cpu);
                return;
        }

* @TIMER_DEFERRABLE: A deferrable timer will work normally when the
* system is busy, but will not cause a CPU to come out of idle just
* to service it; instead, the timer will be serviced when the CPU
* eventually wakes up with a subsequent non-deferrable timer.

You'd want that vmstat_update to execute regardless of whether there are armed non-deferrable timers.

Should fix both 1 and 2 AFAICS.

But Aaron, vmstat_shepherd should be ensuring that per-CPU vmstat_update work are queued, if the per-CPU vmstat are out of sync.

And:

static void
trigger_dyntick_cpu(struct timer_base *base, struct timer_list *timer)

{
        if (!is_timers_nohz_active())
                return;

        /*
         * TODO: This wants some optimizing similar to the code below, but we
         * will do that when we switch from push to pull for deferrable timers.
         */
        if (timer->flags & TIMER_DEFERRABLE) {
                if (tick_nohz_full_cpu(base->cpu))
                        wake_up_nohz_cpu(base->cpu);
                return;
        }

* @TIMER_DEFERRABLE: A deferrable timer will work normally when the
* system is busy, but will not cause a CPU to come out of idle just
* to service it; instead, the timer will be serviced when the CPU
* eventually wakes up with a subsequent non-deferrable timer.

You'd want that vmstat_update to execute regardless of whether there are armed non-deferrable timers.

Should fix both 1 and 2 AFAICS.

Quoted 2 lines · Marcelo Tosatti

But Aaron, vmstat_shepherd should be ensuring that per-CPU vmstat_update work are queued, if the per-CPU vmstat are out of sync.

Hi Marcelo,

Yes, I agree; albeit, as far as I understand, in the context of a nohz_full CPU that has its scheduling-clock tick stopped, we cannot rely on any deferred work.

The purpose of my patch was to prevent a nohz_full CPU from entering idle state when CPU-specific vmstat data is non-zero.

Quoted 26 lines

And:

static void
trigger_dyntick_cpu(struct timer_base *base, struct timer_list *timer)

{
        if (!is_timers_nohz_active())
                return;

        /*
         * TODO: This wants some optimizing similar to the code below, but we
         * will do that when we switch from push to pull for deferrable timers.
         */
        if (timer->flags & TIMER_DEFERRABLE) {
                if (tick_nohz_full_cpu(base->cpu))
                        wake_up_nohz_cpu(base->cpu);
                return;
        }

* @TIMER_DEFERRABLE: A deferrable timer will work normally when the
* system is busy, but will not cause a CPU to come out of idle just
* to service it; instead, the timer will be serviced when the CPU
* eventually wakes up with a subsequent non-deferrable timer.

You'd want that vmstat_update to execute regardless of whether there are armed non-deferrable timers.

Quoted 2 lines

Should fix both 1 and 2 AFAICS.

If I understand correctly, you are suggesting to switch to a non-deferred timer for such work when the scheduling-clock tick is stopped? Indeed, it would address both scenarios yet I'm not sure we'd want that due to the performance impact which might be more than negligible.

If the per-CPU vmstat_update is limited to happen once per second, that shouldnt be a significant performance impact?

Quoted 2 lines · Marcelo Tosatti

If the per-CPU vmstat_update is limited to happen once per second, that shouldnt be a significant performance impact?

Perhaps not. Albeit, is the interrupt worth it?

Then again it could indeed
be a long time before the idle task is selected and a return to idle code were we'd check for any remaining differentials with the aforementioned patch.

Quoted 5 lines · Aaron Tomlin

Quoted 2 lines · Marcelo Tosatti

If the per-CPU vmstat_update is limited to happen once per second, that shouldnt be a significant performance impact?

Perhaps not. Albeit, is the interrupt worth it?

Its a requirement for correctness right?

Quoted 4 lines

Then again it could indeed
be a long time before the idle task is selected and a return to idle code were we'd check for any remaining differentials with the aforementioned patch.

Or a long time before userspace returns to kernel.

Then I can see two other issues:

1) Can an interrupt in idle modify the vmstat and thus trigger the need to flush it? I believe it's the case and then the problem goes beyond nohz_full because if the idle interrupt fired while the tick is stopped and didn't set TIF_RESCHED, we go back to sleep without calling quiet_vmstat().

2) What if we are running task A in kernel mode while the tick is stopped (nohz_full). Task A modifies the vmstat and goes to userspace for a long while.

Your patch fixes case 1) but not case 2). The problem is that TIMER_DEFERRABLE should really be about dynticks-idle only and not dynticks-full. I've always been afraid about enforcing that rule though because that would break old noise-free setups. But perhaps I should...