Skip to contentSkip to conversation

lib/sort.c: Add _nonatomic() variants with cond_resched()

PATCH In review Kent Overstreet 3 participants 5 messages Mar 26, 2025 – Mar 27, 2025 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lib/sort.c: Add _nonatomic() variants with cond_resched()

Andrew - if you're ok with this patch I'd like to get it in soon as a bugfix, I've been getting quite a few reports on this one.

I don't much care for the naming though, thoughts there?

-- >8 --

bcachefs calls sort() during recovery to sort all keys it found in the journal, and this may be very large - gigabytes on large machines.

This has been causing "task blocked" warnings, so needs a cond_resched().

Cc Kuan-Wei Chiu <[email protected]>
Cc Andrew Morton <[email protected]>
---
 include/linux/sort.h | 11 +++++++++++
 lib/sort.c           | 46 +++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 52 insertions(+), 5 deletions(-)
include/linux/sort.h +11 -0
index e163287ac6c1..8e5603b10941 100644
@@ -13,4 +13,15 @@ void sort(void *base, size_t num, size_t size,
 	  cmp_func_t cmp_func,
 	  swap_func_t swap_func);
 
+/* Versions that periodically call cond_resched(): */
+
+void sort_r_nonatomic(void *base, size_t num, size_t size,
+		      cmp_r_func_t cmp_func,
+		      swap_r_func_t swap_func,
+		      const void *priv);
+
+void sort_nonatomic(void *base, size_t num, size_t size,
+		    cmp_func_t cmp_func,
+		    swap_func_t swap_func);
+
 #endif
lib/sort.c +41 -5

77 lines not on this page · raw message

Signature · 1 line
2.49.0
#1 lore raw
Quoted 13 lines · Kent Overstreet

Andrew - if you're ok with this patch I'd like to get it in soon as a bugfix, I've been getting quite a few reports on this one.

I don't much care for the naming though, thoughts there?

-- >8 --

bcachefs calls sort() during recovery to sort all keys it found in the journal, and this may be very large - gigabytes on large machines.

This has been causing "task blocked" warnings, so needs a cond_resched().

Quoted 8 lines
Cc Kuan-Wei Chiu <[email protected]>
Cc Andrew Morton <[email protected]>
---
 include/linux/sort.h | 11 +++++++++++
 lib/sort.c           | 46 +++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 52 insertions(+), 5 deletions(-)

I don't have strong opinions on this, but I recall that UBIFS had a similar issue with list_sort(), and they addressed it by calling cond_resched() within the compare function. Would that approach be simpler and more appropriate than introducing a new API in the library code?

Regards,
Kuan-Wei

#2 lore raw
Quoted 23 lines · Kuan-Wei Chiu
Quoted 13 lines · Kent Overstreet

Andrew - if you're ok with this patch I'd like to get it in soon as a bugfix, I've been getting quite a few reports on this one.

I don't much care for the naming though, thoughts there?

-- >8 --

bcachefs calls sort() during recovery to sort all keys it found in the journal, and this may be very large - gigabytes on large machines.

This has been causing "task blocked" warnings, so needs a cond_resched().

Quoted 8 lines
Cc Kuan-Wei Chiu <[email protected]>
Cc Andrew Morton <[email protected]>
---
 include/linux/sort.h | 11 +++++++++++
 lib/sort.c           | 46 +++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 52 insertions(+), 5 deletions(-)
Quoted 5 lines

I don't have strong opinions on this, but I recall that UBIFS had a similar issue with list_sort(), and they addressed it by calling cond_resched() within the compare function. Would that approach be simpler and more appropriate than introducing a new API in the library code?

That'd be an option, but it would be heavier; sort() has nested loops so it has a more natural place to put it.

And I'd say the nonatomic scheduling version should likely be the default, anyways; even if other users aren't hitting the 10 second warning, going 1 second without scheduling isn't good.

Not going to audit all the existing callers, but we should probably provide it.

maintainer
#3 lore raw
In reply to Kent Overstreet
Quoted 5 lines · On Wed, 26 Mar 2025 11:26:06 -0400 Kent Overstreet <[email protected]>…

bcachefs calls sort() during recovery to sort all keys it found in the journal, and this may be very large - gigabytes on large machines.

This has been causing "task blocked" warnings, so needs a cond_resched().

I assume this has been happening for a while, so a cc:stable is appropriate?

#4 lore raw
Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Wed, 26 Mar 2025 11:26:06 -0400 Kent Overstreet <[email protected]>…

bcachefs calls sort() during recovery to sort all keys it found in the journal, and this may be very large - gigabytes on large machines.

This has been causing "task blocked" warnings, so needs a cond_resched().

I assume this has been happening for a while, so a cc:stable is appropriate?

Well, right now I'm only doing backports for the most critical stuff, and this doesn't meet that bar. I'm planning on doing normal backports after I take the experimental label off, in at most another year.

And I've told the stable team I don't want them touching fs/bcachefs/, I want to be doing those backports and running them through my normal QA process. We could for this, but perhaps better not to let the process get confused :)