mm: add cond_resched() to free_pud_range()
Subsystems (2)
PATCH 1/1 mm: add cond_resched() to free_pud_range()
Packet receive timeouts were seen in production. Tracing showed that an exiting process with a sparse 2.5 TiB mapping could remain in kernel context for over 20 ms without reaching a scheduling point while freeing PTE page-table pages. Hard IRQs could still be handled, but the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying NET_RX softirq work queued to ksoftirqd.
Like zap_pud_range(), add cond_resched() to free_pud_range() so ksoftirqd and other runnable tasks can run between PUD entries. Testing with PREEMPT_NONE showed that the maximum interval between scheduling points fell from over 20 ms to below 2 ms.
--- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory.c +1 -1
@@ -231,7 +231,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d,if (pud_none_or_clear_bad (pud))continue ;free_pmd_range (tlb, pud, addr, next, floor, ceiling); - }while (pud++, addr = next, addr != end); + }while (pud++,cond_resched (), addr = next, addr != end); start &=P4D_MASK ;if (start < floor)
Signature · 1 line
2.55.0
Michal Hocko
reviewerQuoted 11 lines · Leon Hwang
Packet receive timeouts were seen in production. Tracing showed that an exiting process with a sparse 2.5 TiB mapping could remain in kernel context for over 20 ms without reaching a scheduling point while freeing PTE page-table pages. Hard IRQs could still be handled, but the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying
NET_RXsoftirq work queued to ksoftirqd.Like
zap_pud_range(), addcond_resched()tofree_pud_range()so ksoftirqd and other runnable tasks can run between PUD entries. Testing withPREEMPT_NONEshowed that the maximum interval between scheduling points fell from over 20 ms to below 2 ms.
PREEMPT_NONE is effectivelly dead and most cond_resched will/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?
Quoted 22 lines
Reported-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by Leon Hwang <[email protected]>--- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/memory.c +1
-1@@ -231,7 +231,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d,if (pud_none_or_clear_bad (pud))continue ;free_pmd_range (tlb, pud, addr, next, floor, ceiling); - }while (pud++, addr = next, addr != end); + }while (pud++,cond_resched (), addr = next, addr != end); start &=P4D_MASK ;if (start < floor)Signature · 1 line
2.55.0
Signature · 2 lines
Michal Hocko SUSE Labs
Mike Rapoport
reviewerQuoted 24 lines · Leon Hwang
Packet receive timeouts were seen in production. Tracing showed that an exiting process with a sparse 2.5 TiB mapping could remain in kernel context for over 20 ms without reaching a scheduling point while freeing PTE page-table pages. Hard IRQs could still be handled, but the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying
NET_RXsoftirq work queued to ksoftirqd.Like
zap_pud_range(), addcond_resched()tofree_pud_range()so ksoftirqd and other runnable tasks can run between PUD entries. Testing withPREEMPT_NONEshowed that the maximum interval between scheduling points fell from over 20 ms to below 2 ms.Reported-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by Leon Hwang <[email protected]>--- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/memory.c +0
-0
mm/memory.c static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d, · line 234
Quoted 5 lines
mm/memory.c +1
-1@@ -231,7 +231,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d,if (pud_none_or_clear_bad (pud))continue ;free_pmd_range (tlb, pud, addr, next, floor, ceiling); - }while (pud++, addr = next, addr != end); + }while (pud++,cond_resched (), addr = next, addr != end);
This is really obscure, can't it be in the loop body?
Quoted 7 lines
mm/memory.c +0
-0start &=P4D_MASK ;if (start < floor)Signature · 1 line
2.55.0
Signature · 2 lines
Sincerely yours, Mike.
Quoted 8 lines · Michal Hocko
Quoted 7 lines · Leon Hwang
Packet receive timeouts were seen in production. Tracing showed that an exiting process with a sparse 2.5 TiB mapping could remain in kernel context for over 20 ms without reaching a scheduling point while freeing PTE page-table pages. Hard IRQs could still be handled, but the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying
NET_RXsoftirq work queued to ksoftirqd.
Quoted 8 lines
Quoted 4 lines
Like
zap_pud_range(), addcond_resched()tofree_pud_range()so ksoftirqd and other runnable tasks can run between PUD entries. Testing withPREEMPT_NONEshowed that the maximum interval between scheduling points fell from over 20 ms to below 2 ms.
PREEMPT_NONEis effectivelly dead and mostcond_reschedwill/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?
Seems that we have reached a consensus about adding cond_resched() [1].
[1]
https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
Leon
Quoted 24 lines
Quoted 23 lines
Reported-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by Leon Hwang <[email protected]>--- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/memory.c +1
-1@@ -231,7 +231,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d,if (pud_none_or_clear_bad (pud))continue ;free_pmd_range (tlb, pud, addr, next, floor, ceiling); - }while (pud++, addr = next, addr != end); + }while (pud++,cond_resched (), addr = next, addr != end); start &=P4D_MASK ;if (start < floor)Signature · 1 line
2.55.0
Quoted 20 lines · Mike Rapoport
Quoted 19 lines · Leon Hwang
Packet receive timeouts were seen in production. Tracing showed that an exiting process with a sparse 2.5 TiB mapping could remain in kernel context for over 20 ms without reaching a scheduling point while freeing PTE page-table pages. Hard IRQs could still be handled, but the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying
NET_RXsoftirq work queued to ksoftirqd.Like
zap_pud_range(), addcond_resched()tofree_pud_range()so ksoftirqd and other runnable tasks can run between PUD entries. Testing withPREEMPT_NONEshowed that the maximum interval between scheduling points fell from over 20 ms to below 2 ms.Reported-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by Leon Hwang <[email protected]>--- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 12 lines
mm/memory.c static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d, · line 234
Quoted 10 lines
mm/memory.c +1
-1@@ -231,7 +231,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d,if (pud_none_or_clear_bad (pud))continue ;free_pmd_range (tlb, pud, addr, next, floor, ceiling); - }while (pud++, addr = next, addr != end); + }while (pud++,cond_resched (), addr = next, addr != end);This is really obscure, can't it be in the loop body?
Agreed. It should be in the loop body, after free_pmd_range().
Thanks,
Leon
Quoted 8 lines
Quoted 7 lines
mm/memory.c +0
-0start &=P4D_MASK ;if (start < floor)Signature · 1 line
2.55.0
Quoted 14 lines · Leon Hwang
Quoted 13 lines · Michal Hocko
Quoted 11 lines · Leon Hwang
Packet receive timeouts were seen in production. Tracing showed that an exiting process with a sparse 2.5 TiB mapping could remain in kernel context for over 20 ms without reaching a scheduling point while freeing PTE page-table pages. Hard IRQs could still be handled, but the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying
NET_RXsoftirq work queued to ksoftirqd.Like
zap_pud_range(), addcond_resched()tofree_pud_range()so ksoftirqd and other runnable tasks can run between PUD entries. Testing withPREEMPT_NONEshowed that the maximum interval between scheduling points fell from over 20 ms to below 2 ms.
Quoted 5 lines
Quoted 3 lines
PREEMPT_NONEis effectivelly dead and mostcond_reschedwill/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?Seems that we have reached a consensus about adding
cond_resched()[1].
I disagree. It makes zero sense to add more cond_resched calls unless they are essentially necessary. In this case you are talking about latency requirements while using non-preemptible kernel which is in stark contradiction. So either you have other very good reasons or just base on the current changelog
Quoted 31 lines
[1]
https://lore.kernel.org/linux-mm/[email protected]/Thanks,
LeonQuoted 24 lines
Quoted 23 lines
Reported-by Lance Yang <[email protected]>Tested-by Lance Yang <[email protected]>Signed-off-by Leon Hwang <[email protected]>--- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/memory.c +1
-1@@ -231,7 +231,7 @@ static inline void free_pud_range(struct mmu_gather *tlb, p4d_t *p4d,if (pud_none_or_clear_bad (pud))continue ;free_pmd_range (tlb, pud, addr, next, floor, ceiling); - }while (pud++, addr = next, addr != end); + }while (pud++,cond_resched (), addr = next, addr != end); start &=P4D_MASK ;if (start < floor)Signature · 1 line
2.55.0
Signature · 2 lines
Michal Hocko SUSE Labs