f2fs: fix lock priority inversion issue
Subsystems (2)
Chao Yu
maintainerPATCH 1/2 f2fs: fix lock priority inversion issue
If userspace thread has held f2fs rw semaphore, due to its low priority, it could be runnable or preempted state for long time, during the time, it will block high priority thread which is trying to grab the same rw semaphore, e.g. cp_rwsem, io_rwsem...
To fix such issue, let's detect thread's priority when it tries to grab f2fs_rwsem lock, if the priority is lower than a priority threshold, let's uplift the priority before it enters into critical region of lock, and restore the priority after it leaves from critical region.
Meanwhile, introducing two new sysfs nodes:
- /sys/fs/f2fs/<disk>/adjust_lock_priority, it is used to control whether the functionality is enable or not.
========== ================== Flag_Value Flag_Description ========== ================== 0x00000000 Disabled (default) 0x00000001 cp_rwsem 0x00000002 node_change 0x00000004 node_write 0x00000008 gc_lock 0x00000010 cp_global 0x00000020 io_rwsem ========== ==================
- /sys/fs/f2fs/<disk>/lock_duration_priority, it is used to control priority threshold.
--- Documentation/ABI/testing/sysfs-fs-f2fs | 24 +++++++++ fs/f2fs/checkpoint.c | 66 ++++++++++++++++++++++++- fs/f2fs/f2fs.h | 12 +++++ fs/f2fs/super.c | 2 + fs/f2fs/sysfs.c | 18 +++++++ 5 files changed, 120 insertions(+), 2 deletions(-)
Documentation/ABI/testing/sysfs-fs-f2fs +24 -0
@@ -963,3 +963,27 @@ Description: This sysfs entry can be used to change type of injected timeout: 0x00000003 Simulate Non-IO type sleep time 0x00000004 Simulate runnable time ========== =============================== + +What: /sys/fs/f2fs/<disk>/adjust_lock_priority +Date: January 2026 +Contact: "Chao Yu" <[email protected]> +Description: This sysfs entry can be used to enable/disable to adjust priority for task + which is in critical region covered by lock. + ========== ================== + Flag_Value Flag_Description + ========== ================== + 0x00000000 Disabled (default) + 0x00000001 cp_rwsem + 0x00000002 node_change + 0x00000004 node_write + 0x00000008 gc_lock + 0x00000010 cp_global + 0x00000020 io_rwsem + ========== ================== + +What: /sys/fs/f2fs/<disk>/lock_duration_priority +Date: January 2026 +Contact: "Chao Yu" <[email protected]> +Description: f2fs can tune priority of thread which has entered into critical region covered by + f2fs rwsemphore lock. This sysfs entry can be used to control priority value, the + range is [100,139], by default the value is 120.
fs/f2fs/checkpoint.c +64 -2
105 lines not on this page · raw message
fs/f2fs/f2fs.h +12 -0
@@ -185,6 +185,7 @@ enum f2fs_lock_name {LOCK_NAME_GC_LOCK ,LOCK_NAME_CP_GLOBAL ,LOCK_NAME_IO_RWSEM , + LOCK_NAME_MAX, }; enum f2fs_timeout_type {
@@ -1447,7 +1448,10 @@ struct f2fs_time_stat {struct f2fs_lock_context {struct f2fs_time_stat ts; +int orig_nice; +int new_nice;bool lock_trace; +bool need_restore; };struct f2fs_gc_control {
@@ -1588,6 +1592,8 @@ enum node_type {/* a threshold of maximum elapsed time in critical region to print tracepoint */ #define MAX_LOCK_ELAPSED_TIME 500 +#define F2FS_DEFAULT_TASK_PRIORITY (DEFAULT_PRIO) +static inline int f2fs_test_bit (unsigned int nr,char *addr);static inline void f2fs_set_bit (unsigned int nr,char *addr);static inline void f2fs_clear_bit (unsigned int nr,char *addr);
@@ -1998,6 +2004,12 @@ struct f2fs_sb_info {/* max elapsed time threshold in critical region that lock covered */ unsigned long long max_lock_elapsed_time; +/* enable/disable to adjust task priority in critical region covered by lock */ +unsigned int adjust_lock_priority; + +/* adjust priority for task which is in critical region covered by lock */ +unsigned int lock_duration_priority; +#ifdef CONFIG_F2FS_FS_COMPRESSION struct kmem_cache *page_array_slab;/* page array entry */ unsigned int page_array_slab_size;/* default page array slab size */
fs/f2fs/super.c +2 -0
@@ -4338,6 +4338,8 @@ static void init_sb_info(struct f2fs_sb_info *sbi)spin_lock_init (&sbi->gc_remaining_trials_lock);atomic64_set (&sbi->current_atomic_write,0 ); sbi->max_lock_elapsed_time =MAX_LOCK_ELAPSED_TIME ; + sbi->adjust_lock_priority =0 ; + sbi->lock_duration_priority =F2FS_DEFAULT_TASK_PRIORITY ; sbi->sum_blocksize =f2fs_sb_has_packed_ssa (sbi) ?4096 : sbi->blocksize;
fs/f2fs/sysfs.c +18 -0
@@ -955,6 +955,20 @@ static ssize_t __sbi_store(struct f2fs_attr *a,return count; } +if (!strcmp (a->attr.name,"adjust_lock_priority" )) { +if (t >=BIT (LOCK_NAME_MAX -1 )) +return -EINVAL ; + sbi->adjust_lock_priority = t; +return count; + } + +if (!strcmp (a->attr.name,"lock_duration_priority" )) { +if (t <NICE_TO_PRIO (MIN_NICE ) || t >NICE_TO_PRIO (MAX_NICE )) +return -EINVAL ; + sbi->lock_duration_priority = t; +return count; + } +__sbi_store_value (a, sbi, ptr + a->offset, t);return count;
@@ -1272,6 +1286,8 @@ F2FS_SBI_GENERAL_RW_ATTR(carve_out);F2FS_SBI_GENERAL_RW_ATTR (reserved_pin_section);F2FS_SBI_GENERAL_RW_ATTR (bggc_io_aware);F2FS_SBI_GENERAL_RW_ATTR (max_lock_elapsed_time); +F2FS_SBI_GENERAL_RW_ATTR (lock_duration_priority); +F2FS_SBI_GENERAL_RW_ATTR (adjust_lock_priority);/* STAT_INFO ATTR */ #ifdef CONFIG_F2FS_STAT_FS
@@ -1478,6 +1494,8 @@ static struct attribute *f2fs_attrs[] = {ATTR_LIST (allocate_section_hint),ATTR_LIST (allocate_section_policy),ATTR_LIST (max_lock_elapsed_time), +ATTR_LIST (lock_duration_priority), +ATTR_LIST (adjust_lock_priority),NULL , };ATTRIBUTE_GROUPS (f2fs);
Signature · 1 line
2.40.1
Chao Yu
maintainerPATCH 2/2 f2fs: introduce trace_f2fs_priority_update
This patch introduces two new tracepoints for debug purpose.
--- fs/f2fs/checkpoint.c | 17 +++++++---- include/trace/events/f2fs.h | 57 +++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 5 deletions(-)
fs/f2fs/checkpoint.c +12 -5
@@ -130,9 +130,13 @@ static void uplift_priority(struct f2fs_rwsem *sem, struct f2fs_lock_context *lcreturn ;set_user_nice (current, lc->new_nice); lc->need_restore = true; + +trace_f2fs_priority_uplift (sem->sbi, sem->name, is_write, current, +NICE_TO_PRIO (lc->orig_nice),NICE_TO_PRIO (lc->new_nice)); } -static void restore_priority (struct f2fs_lock_context *lc) +static void restore_priority (struct f2fs_rwsem *sem,struct f2fs_lock_context *lc, +bool is_write) {if (!lc->need_restore)return ;
@@ -140,6 +144,9 @@ static void restore_priority(struct f2fs_lock_context *lc)if (task_nice (current) != lc->new_nice)return ;set_user_nice (current, lc->orig_nice); + +trace_f2fs_priority_restore (sem->sbi, sem->name, is_write, current, +NICE_TO_PRIO (lc->orig_nice),NICE_TO_PRIO (lc->new_nice)); }void f2fs_down_read_trace (struct f2fs_rwsem *sem,struct f2fs_lock_context *lc)
@@ -153,7 +160,7 @@ int f2fs_down_read_trylock_trace(struct f2fs_rwsem *sem, struct f2fs_lock_contex {uplift_priority (sem, lc, false);if (!f2fs_down_read_trylock (sem)) { -restore_priority (lc); +restore_priority (sem, lc, false);return 0 ; }trace_lock_elapsed_time_start (sem, lc);
@@ -163,7 +170,7 @@ int f2fs_down_read_trylock_trace(struct f2fs_rwsem *sem, struct f2fs_lock_contexvoid f2fs_up_read_trace (struct f2fs_rwsem *sem,struct f2fs_lock_context *lc) {f2fs_up_read (sem); -restore_priority (lc); +restore_priority (sem, lc, false);trace_lock_elapsed_time_end (sem, lc, false); }
@@ -178,7 +185,7 @@ int f2fs_down_write_trylock_trace(struct f2fs_rwsem *sem, struct f2fs_lock_conte {uplift_priority (sem, lc, true);if (!f2fs_down_write_trylock (sem)) { -restore_priority (lc); +restore_priority (sem, lc, true);return 0 ; }trace_lock_elapsed_time_start (sem, lc);
@@ -188,7 +195,7 @@ int f2fs_down_write_trylock_trace(struct f2fs_rwsem *sem, struct f2fs_lock_contevoid f2fs_up_write_trace (struct f2fs_rwsem *sem,struct f2fs_lock_context *lc) {f2fs_up_write (sem); -restore_priority (lc); +restore_priority (sem, lc, true);trace_lock_elapsed_time_end (sem, lc, true); }
include/trace/events/f2fs.h +57 -0
63 lines not on this page · raw message
Signature · 1 line
2.40.1
Quoted 10 lines · Chao Yu
If userspace thread has held f2fs rw semaphore, due to its low priority, it could be runnable or preempted state for long time, during the time, it will block high priority thread which is trying to grab the same rw semaphore, e.g.
cp_rwsem,io_rwsem...To fix such issue, let's detect thread's priority when it tries to grab
f2fs_rwsemlock, if the priority is lower than a priority threshold, let's uplift the priority before it enters into critical region of lock, and restore the priority after it leaves from critical region.
Hi Chao,
Is this even possible if can_nice() returns false, for example due to missing CAP_SYS_NICE?
Proxy execution [1] is currently under development to address general priority inversion; hopefully, it will resolve this issue.
Thanks
Barry
Chao Yu
maintainerQuoted 12 lines · Barry Song
Quoted 10 lines · Chao Yu
If userspace thread has held f2fs rw semaphore, due to its low priority, it could be runnable or preempted state for long time, during the time, it will block high priority thread which is trying to grab the same rw semaphore, e.g.
cp_rwsem,io_rwsem...To fix such issue, let's detect thread's priority when it tries to grab
f2fs_rwsemlock, if the priority is lower than a priority threshold, let's uplift the priority before it enters into critical region of lock, and restore the priority after it leaves from critical region.
Quoted 4 lines
Hi Chao,
Is this even possible if
can_nice()returns false, for example due to missingCAP_SYS_NICE?
Hi Barry,
I think so, the priority will be temporarily uplifted and restored in lock duration.
Quoted 3 lines
Proxy execution [1] is currently under development to address general priority inversion; hopefully, it will resolve this issue.
Thanks for providing the material.
Yes, there should be generic solution in cpu scheduler and lock primitive to resolve priority inversion issue. Then it will be benefit for all lock users.
Not sure, it will be a little bit hard for rw semaphore, it needs to track and check all reader threads' context in priority inversion scenario.
Thanks,
Quoted 5 lines
Thanks
Barry
This series was applied to jaegeuk/f2fs.git (dev) by Jaegeuk Kim <[email protected]>:
Hello:
This series was applied to jaegeuk/f2fs.git (dev)
by Jaegeuk Kim <[email protected]>:
Quoted 11 lines · On Fri, 30 Jan 2026 21:28:08 +0800 you wrote:
If userspace thread has held f2fs rw semaphore, due to its low priority, it could be runnable or preempted state for long time, during the time, it will block high priority thread which is trying to grab the same rw semaphore, e.g.
cp_rwsem,io_rwsem...To fix such issue, let's detect thread's priority when it tries to grab
f2fs_rwsemlock, if the priority is lower than a priority threshold, let's uplift the priority before it enters into critical region of lock, and restore the priority after it leaves from critical region.[...]
Here is the summary with links:
- [f2fs-dev,1/2] f2fs: fix lock priority inversion issue
https://git.kernel.org/jaegeuk/f2fs/c/07de55cbf576 - [f2fs-dev,2/2] f2fs: introduce trace_f2fs_priority_update https://git.kernel.org/jaegeuk/f2fs/c/bc367775f602
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html