hugetlb: add cond_resched() to __unmap_hugepage_range()

PATCH In review Leon Hwang 6 participants 14 messages Aug 18 – Aug 19

Subsystems (2)

Memory management HugeTLB

lore

Share & tools

lore

Leon Hwang

author

2026-08-18 13:50 UTC #0 lore raw

PATCH 1/1 hugetlb: add cond_resched() to __unmap_hugepage_range()

Packet receive timeouts were traced to sparse HugeTLB unmapping in production. A task unmapping a sparse 2.5 TiB HugeTLB mapping could remain in kernel context for over 40 ms without reaching a scheduling point while walking empty huge PTEs. Although hard IRQs could still be handled, the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying NET_RX softirq work queued to ksoftirqd.

Add cond_resched() at the beginning of the hugepage loop so ksoftirqd and other runnable tasks can run between iterations. Testing with PREEMPT_NONE showed that the maximum interval between scheduling points fell from over 40 ms to below 2.5 ms. Total time spent in __unmap_hugepage_range() remained about 36 ms.

Reported-by Lance Yang <[email protected]>

Tested-by Lance Yang <[email protected]>

Signed-off-by Leon Hwang <[email protected]>

---
 mm/hugetlb.c | 2 ++
 1 file changed, 2 insertions(+)

mm/hugetlb.c +2 -0

index dded1768193a..0a91aac2369f 100644
@@ -5233,6 +5233,8 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	last_addr_mask = hugetlb_mask_last_page(h);
 	address = start;
 	for (; address < end; address += sz) {
+		cond_resched();
+
 		ptep = hugetlb_walk(vma, address, sz);
 		if (!ptep) {
 			address |= last_addr_mask;

Signature · 1 line

2.55.0

David Hildenbrand (Arm)

reviewer

2026-08-18 14:23 UTC #1 lore raw

Quoted 26 lines · Leon Hwang

Packet receive timeouts were traced to sparse HugeTLB unmapping in production. A task unmapping a sparse 2.5 TiB HugeTLB mapping could remain in kernel context for over 40 ms without reaching a scheduling point while walking empty huge PTEs. Although hard IRQs could still be handled, the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying NET_RX softirq work queued to ksoftirqd.
Add cond_resched() at the beginning of the hugepage loop so ksoftirqd and other runnable tasks can run between iterations. Testing with PREEMPT_NONE showed that the maximum interval between scheduling points fell from over 40 ms to below 2.5 ms. Total time spent in __unmap_hugepage_range() remained about 36 ms.
Reported-by Lance Yang <[email protected]>
Tested-by Lance Yang <[email protected]>
Signed-off-by Leon Hwang <[email protected]>
---
 mm/hugetlb.c | 2 ++
 1 file changed, 2 insertions(+)
mm/hugetlb.c +0 -0
index dded1768193a..0a91aac2369f 100644

mm/hugetlb.c void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 5240

Quoted 8 lines

mm/hugetlb.c +2 -0

@@ -5233,6 +5233,8 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	last_addr_mask = hugetlb_mask_last_page(h);
 	address = start;
 	for (; address < end; address += sz) {
+		cond_resched();
+
 		ptep = hugetlb_walk(vma, address, sz);
 		if (!ptep) {
 			address |= last_addr_mask;

As Michal just put it:

"PREEMPT_NONE is effectivelly dead and most cond_resched will/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?"

https://lore.kernel.org/r/aoRnUxUlgf_kRlm8@tiehlicka

Signature · 3 lines

Cheers,

David

Andrew Morton

maintainer

2026-08-18 18:17 UTC #2 lore raw

Quoted 9 lines · On Tue, 18 Aug 2026 16:23:29 +0200 "David Hildenbrand (Arm)"…

Quoted 8 lines · Leon Hwang
Packet receive timeouts were traced to sparse HugeTLB unmapping in production. A task unmapping a sparse 2.5 TiB HugeTLB mapping could remain in kernel context for over 40 ms without reaching a scheduling point while walking empty huge PTEs. Although hard IRQs could still be handled, the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying NET_RX softirq work queued to ksoftirqd.

Quoted 8 lines

Quoted 8 lines
Add cond_resched() at the beginning of the hugepage loop so ksoftirqd and other runnable tasks can run between iterations. Testing with PREEMPT_NONE showed that the maximum interval between scheduling points fell from over 40 ms to below 2.5 ms. Total time spent in __unmap_hugepage_range() remained about 36 ms.
Reported-by Lance Yang <[email protected]>
Tested-by Lance Yang <[email protected]>

Is there a Link: to Lance's report?

Quoted 21 lines

Quoted 10 lines
Signed-off-by Leon Hwang <[email protected]>
---
 mm/hugetlb.c | 2 ++
 1 file changed, 2 insertions(+)
mm/hugetlb.c +0 -0
index dded1768193a..0a91aac2369f 100644
mm/hugetlb.c void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 5240
Quoted 8 lines
mm/hugetlb.c +2 -0
@@ -5233,6 +5233,8 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	last_addr_mask = hugetlb_mask_last_page(h);
 	address = start;
 	for (; address < end; address += sz) {
+		cond_resched();
+
 		ptep = hugetlb_walk(vma, address, sz);
 		if (!ptep) {
 			address |= last_addr_mask;
As Michal just put it:

Quoted 5 lines

"PREEMPT_NONE is effectivelly dead and most cond_resched will/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?"
https://lore.kernel.org/r/aoRnUxUlgf_kRlm8@tiehlicka

That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?

If yes, we do want to fix older kernels then we should merge this.

David Hildenbrand (Arm)

reviewer

2026-08-18 18:21 UTC #3 lore raw

Quoted 47 lines · Andrew Morton

Quoted 9 lines · On Tue, 18 Aug 2026 16:23:29 +0200 "David Hildenbrand (Arm)"…
Quoted 8 lines · Leon Hwang
Packet receive timeouts were traced to sparse HugeTLB unmapping in production. A task unmapping a sparse 2.5 TiB HugeTLB mapping could remain in kernel context for over 40 ms without reaching a scheduling point while walking empty huge PTEs. Although hard IRQs could still be handled, the per-CPU ksoftirqd thread and other runnable tasks could not run during that interval, delaying NET_RX softirq work queued to ksoftirqd.
Quoted 8 lines
Quoted 8 lines
Add cond_resched() at the beginning of the hugepage loop so ksoftirqd and other runnable tasks can run between iterations. Testing with PREEMPT_NONE showed that the maximum interval between scheduling points fell from over 40 ms to below 2.5 ms. Total time spent in __unmap_hugepage_range() remained about 36 ms.
Reported-by Lance Yang <[email protected]>
Tested-by Lance Yang <[email protected]>
Is there a Link: to Lance's report?
Quoted 25 lines
Quoted 10 lines
Signed-off-by Leon Hwang <[email protected]>
---
 mm/hugetlb.c | 2 ++
 1 file changed, 2 insertions(+)
mm/hugetlb.c +0 -0
index dded1768193a..0a91aac2369f 100644
mm/hugetlb.c void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 5240
Quoted 8 lines
mm/hugetlb.c +2 -0
@@ -5233,6 +5233,8 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	last_addr_mask = hugetlb_mask_last_page(h);
 	address = start;
 	for (; address < end; address += sz) {
+		cond_resched();
+
 		ptep = hugetlb_walk(vma, address, sz);
 		if (!ptep) {
 			address |= last_addr_mask;
As Michal just put it:
"PREEMPT_NONE is effectivelly dead and most cond_resched will/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?"

Quoted 8 lines

Quoted 1 line
https://lore.kernel.org/r/aoRnUxUlgf_kRlm8@tiehlicka
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.

Okay, but that would be stable-only fixes?

Signature · 3 lines

Cheers,

David

Andrew Morton

maintainer

2026-08-18 18:55 UTC #4 lore raw

Quoted 11 lines · On Tue, 18 Aug 2026 20:21:37 +0200 "David Hildenbrand (Arm)"…

Quoted 11 lines
Quoted 10 lines
mm/hugetlb.c void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma, · line 5240
Quoted 2 lines
mm/hugetlb.c +0 -0
 		if (!ptep) {
 			address |= last_addr_mask;
As Michal just put it:
"PREEMPT_NONE is effectivelly dead and most cond_resched will/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?"
https://lore.kernel.org/r/aoRnUxUlgf_kRlm8@tiehlicka

Quoted 8 lines

Quoted 6 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.
Okay, but that would be stable-only fixes?

Not understanding.

Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).

Lance Yang

2026-08-18 22:19 UTC #5 lore raw

+Cc Michal

Quoted 5 lines · Andrew Morton

Quoted 3 lines · On Tue, 18 Aug 2026 20:21:37 +0200 "David Hildenbrand (Arm)"…
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
mm/hugetlb.c +0 -0
 		if (!ptep) {
 			address |= last_addr_mask;

Quoted 7 lines

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
As Michal just put it:
"PREEMPT_NONE is effectivelly dead and most cond_resched will/should be removed. Is there any reason why you are not using full preemption when requiring low latencies?"
https://lore.kernel.org/r/aoRnUxUlgf_kRlm8@tiehlicka

Ah, I missed that PREEMPT_LAZY is now the default on major archs and PREEMPT_NONE is effectively gone there ...

Quoted 8 lines

Quoted 8 lines
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.

Quoted 7 lines

Quoted 1 line
Okay, but that would be stable-only fixes?
Not understanding.
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).

Since cond_resched() is a scheduling no-op under LAZY/FULL anyway (only the __might_resched() debug check remains), why not take this upstream with Cc: stable?

Mainline scheduling stays unchanged, and stable can pick it up for old PREEMPT_NONE kernels. wdyt?

Thanks, Lance

Andrew Morton

maintainer

2026-08-18 23:24 UTC #6 lore raw

Quoted 12 lines · On Wed, 19 Aug 2026 06:18:58 +0800 Lance Yang <[email protected]> wrote:

Quoted 12 lines
Quoted 9 lines
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.
Okay, but that would be stable-only fixes?
Not understanding.

Quoted 7 lines

Quoted 3 lines
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).
Since cond_resched() is a scheduling no-op under LAZY/FULL anyway (only the __might_resched() debug check remains), why not take this upstream with Cc: stable?

I think so - as long as PREEPMT_NONE exists we should support it as well as we can. If the day comes that PREEMPT_NONE is removed, then we get to remove lots of cond_resched()s. mm/ has 200 of the things.

Quoted 2 lines

Mainline scheduling stays unchanged, and stable can pick it up for old PREEMPT_NONE kernels. wdyt?

Addressing this issue in older kernels is another reason.

Matthew Wilcox

2026-08-18 23:46 UTC #7 lore raw

Quoted 22 lines · Andrew Morton

Quoted 12 lines · On Wed, 19 Aug 2026 06:18:58 +0800 Lance Yang <[email protected]> wrote:
Quoted 12 lines
Quoted 9 lines
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.
Okay, but that would be stable-only fixes?
Not understanding.
Quoted 7 lines
Quoted 3 lines
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).
Since cond_resched() is a scheduling no-op under LAZY/FULL anyway (only the __might_resched() debug check remains), why not take this upstream with Cc: stable?

Quoted 8 lines

I think so - as long as PREEPMT_NONE exists we should support it as well as we can. If the day comes that PREEMPT_NONE is removed, then we get to remove lots of cond_resched()s. mm/ has 200 of the things.
Quoted 2 lines
Mainline scheduling stays unchanged, and stable can pick it up for old PREEMPT_NONE kernels. wdyt?
Addressing this issue in older kernels is another reason.

That is not the direction that the scheduler developers wish us to take.

Andrew Morton

maintainer

2026-08-18 23:59 UTC #8 lore raw

Quoted 27 lines · On Wed, 19 Aug 2026 00:46:09 +0100 Matthew Wilcox <[email protected]> wrote:

Quoted 26 lines · Andrew Morton
Quoted 12 lines · On Wed, 19 Aug 2026 06:18:58 +0800 Lance Yang <[email protected]> wrote:
Quoted 12 lines
Quoted 9 lines
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.
Okay, but that would be stable-only fixes?
Not understanding.
Quoted 7 lines
Quoted 3 lines
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).
Since cond_resched() is a scheduling no-op under LAZY/FULL anyway (only the __might_resched() debug check remains), why not take this upstream with Cc: stable?
I think so - as long as PREEPMT_NONE exists we should support it as well as we can. If the day comes that PREEMPT_NONE is removed, then we get to remove lots of cond_resched()s. mm/ has 200 of the things.

Quoted 6 lines

Quoted 4 lines
Quoted 2 lines
Mainline scheduling stays unchanged, and stable can pick it up for old PREEMPT_NONE kernels. wdyt?
Addressing this issue in older kernels is another reason.
That is not the direction that the scheduler developers wish us to take.

Well back luck.

"Packet receive timeouts were traced to sparse HugeTLB unmapping in production". Do sched developers have a proposal to fix that in year-old kernels?

Michal Hocko

2026-08-19 06:55 UTC #9 lore raw

Quoted 34 lines · Andrew Morton

Quoted 32 lines · On Wed, 19 Aug 2026 00:46:09 +0100 Matthew Wilcox <[email protected]> wrote:
Quoted 30 lines · Andrew Morton
Quoted 12 lines · On Wed, 19 Aug 2026 06:18:58 +0800 Lance Yang <[email protected]> wrote:
Quoted 12 lines
Quoted 9 lines
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.
Okay, but that would be stable-only fixes?
Not understanding.
Quoted 7 lines
Quoted 3 lines
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).
Since cond_resched() is a scheduling no-op under LAZY/FULL anyway (only the __might_resched() debug check remains), why not take this upstream with Cc: stable?
I think so - as long as PREEPMT_NONE exists we should support it as well as we can. If the day comes that PREEMPT_NONE is removed, then we get to remove lots of cond_resched()s. mm/ has 200 of the things.
Quoted 2 lines
Mainline scheduling stays unchanged, and stable can pick it up for old PREEMPT_NONE kernels. wdyt?
Addressing this issue in older kernels is another reason.

Quoted 7 lines

Quoted 1 line
That is not the direction that the scheduler developers wish us to take.
Well back luck.
"Packet receive timeouts were traced to sparse HugeTLB unmapping in production". Do sched developers have a proposal to fix that in year-old kernels?

Yes, do not use PREEMPT_NONE in anything that is latency sensitive. It makes very little sense to add more cond_resched, just to be removed later.

Signature · 2 lines

Michal Hocko
SUSE Labs

David Hildenbrand (Arm)

reviewer

2026-08-19 07:53 UTC #10 lore raw

In reply to Andrew Morton

Quoted 10 lines · Andrew Morton

Quoted 8 lines · On Tue, 18 Aug 2026 20:21:37 +0200 "David Hildenbrand (Arm)"…
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.

Quoted 7 lines

Quoted 1 line
Okay, but that would be stable-only fixes?
Not understanding.
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).

It's tricky: if a problem only exists in stable (there is nothing to fix in Linus' tree), then a stable-only fix is acceptable.

Otherwise we'd have to route unnecessary churn through upstream just for it to reach stable, which doesn't make any sense.

When sending a stable patch, it should document how this is only a stable problem and why no corresponding upstream fix is required (or how the actual fix for upstream is actually deprecating the relevant config).

Signature · 3 lines

Cheers,

David

Michal Hocko

2026-08-19 08:50 UTC #11 lore raw

Quoted 13 lines · David Hildenbrand

Quoted 12 lines · Andrew Morton
Quoted 9 lines · On Tue, 18 Aug 2026 20:21:37 +0200 "David Hildenbrand (Arm)"…
Quoted 7 lines
That's pretty bad behavior and we might want to fix it in earlier kernels. Is PREEMPT_NONE effectively dead in 6.18.x and its existing users?
If yes, we do want to fix older kernels then we should merge this.
Okay, but that would be stable-only fixes?

Quoted 8 lines

Quoted 5 lines
Not understanding.
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).
It's tricky: if a problem only exists in stable (there is nothing to fix in Linus' tree), then a stable-only fix is acceptable.

The crucial quiestion is whether this is something that needs a code fix or a configuration fix. Really fighting for low latencies with PREEMPT_NONE is a kinda lost battle. You might want to play whack a mole...

Signature · 2 lines

Michal Hocko
SUSE Labs

David Hildenbrand (Arm)

reviewer

2026-08-19 08:54 UTC #12 lore raw

Quoted 10 lines · Michal Hocko

Quoted 9 lines · David Hildenbrand
Quoted 7 lines · Andrew Morton
Not understanding.
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).

Quoted 7 lines

Quoted 2 lines
It's tricky: if a problem only exists in stable (there is nothing to fix in Linus' tree), then a stable-only fix is acceptable.
The crucial quiestion is whether this is something that needs a code fix or a configuration fix. Really fighting for low latencies with PREEMPT_NONE is a kinda lost battle. You might want to play whack a mole...

Yes, I read your comment on the other thread afterwards and I agree.

The whole reason we added cond_resched() all over the place over the years was to avoid splats from false detected hung tasks (e.g., 30s ...).

Not to optimize latency in the ms range.

Signature · 3 lines

Cheers,

David

Michal Hocko

2026-08-19 08:59 UTC #13 lore raw

Quoted 19 lines · David Hildenbrand

Quoted 10 lines · Michal Hocko
Quoted 9 lines · David Hildenbrand
Quoted 7 lines · Andrew Morton
Not understanding.
Maybe you refer to adding a patch to -stable but not to -linus? That's against the -stable rules (Documentation/process/stable-kernel-rules.rst).
Quoted 7 lines
Quoted 2 lines
It's tricky: if a problem only exists in stable (there is nothing to fix in Linus' tree), then a stable-only fix is acceptable.
The crucial quiestion is whether this is something that needs a code fix or a configuration fix. Really fighting for low latencies with PREEMPT_NONE is a kinda lost battle. You might want to play whack a mole...

Quoted 6 lines

Yes, I read your comment on the other thread afterwards and I agree.
The whole reason we added cond_resched() all over the place over the years was to avoid splats from false detected hung tasks (e.g., 30s ...).
Not to optimize latency in the ms range.

Exactly, they aimed to provide reasonable upper boundary of no-preemption with non-preemptive scheduling. And those are on decline which is a reason to keep bar for adding new ones high and also optimizing low latencies fundamentally makes no sense for those models. So even more reason to not add them in these cases. This will just add more future work when non-preemptive models are gone which will eventually happen AFAIU.

Signature · 2 lines

Michal Hocko
SUSE Labs