nilfs2: reject CLEAN_SEGMENTS ioctl with out-of-range segment numbers
Subsystems (2)
PATCH v3 1/1 nilfs2: reject CLEAN_SEGMENTS ioctl with out-of-range segment numbers
Syzbot reported a hung task in nilfs_transaction_begin() where multiple tasks performing chmod() on a nilfs2 mount blocked for over 143 seconds waiting to acquire ns_segctor_sem for read:
INFO: task syz.0.17:5918 blocked for more than 143 seconds. Call Trace: schedule+0x164/0x360 rwsem_down_read_slowpath+0x6d9/0x940 down_read+0x99/0x2e0 nilfs_transaction_begin+0x364/0x710 fs/nilfs2/segment.c:221 nilfs_setattr+0x124/0x2c0 fs/nilfs2/inode.c:921 notify_change+0xc1a/0xf40 chmod_common+0x273/0x4a0 do_fchmodat+0x12d/0x230
The writer holding ns_segctor_sem was a concurrent NILFS_IOCTL_CLEAN_SEGMENTS caller, stuck inside printk while emitting per-element warnings from nilfs_sufile_updatev():
__nilfs_msg+0x373/0x450 fs/nilfs2/super.c:78 nilfs_sufile_updatev+0x21c/0x6d0 fs/nilfs2/sufile.c:186 nilfs_sufile_freev fs/nilfs2/sufile.h:93 [inline] nilfs_free_segments fs/nilfs2/segment.c:1140 [inline] nilfs_segctor_collect_blocks fs/nilfs2/segment.c:1261 [inline] nilfs_segctor_do_construct+0x1f55/0x76c0 nilfs_clean_segments+0x3bd/0xa50 nilfs_ioctl_clean_segments fs/nilfs2/ioctl.c:922 [inline] nilfs_ioctl+0x261f/0x2780
The root cause is that user-supplied segment numbers are not validated before nilfs_clean_segments() begins doing work; the range check on each segnum is performed deep inside the call chain by nilfs_sufile_updatev(), which emits a nilfs_warn() per invalid entry while still holding the segctor lock and the sufile mi_sem. Under load (repeated invocations across multiple mounts saturating the global printk path), the cumulative printk latency keeps ns_segctor_sem held long enough to trip the hung_task watchdog, blocking concurrent operations such as chmod() that need ns_segctor_sem for read.
Fix by validating the contents of kbufs[4] in nilfs_clean_segments() immediately after acquiring ns_segctor_sem via nilfs_transaction_lock(). Holding ns_segctor_sem serializes the check against nilfs_ioctl_resize(), which can modify ns_nsegments, so the validation uses a consistent value. Out-of-range segment numbers are rejected with -EINVAL before any segment-cleaning work begins, so the bad entries never reach the per-element diagnostic path inside nilfs_sufile_updatev().
---
Changes in v3:
- Move validation from nilfs_ioctl_clean_segments() into nilfs_clean_segments(), under ns_segctor_sem held for write by nilfs_transaction_lock(), to serialize against nilfs_ioctl_resize() which can modify ns_nsegments (Ryusuke Konishi)
- Introduce local variables segnumv and nfreesegs for readability, rather than open-coding casts of kbufs[4] (Ryusuke Konishi)
- Emit nilfs_err() once on the first out-of-range segnum and bail out, instead of nilfs_warn() per element (Ryusuke Konishi)
- Add bail_unlock label for the early-failure path, parallel to the existing out_unlock structure (Ryusuke Konishi)
Changes in v2:
- Reuse existing 'n' loop variable instead of introducing a new one (Slava Dubeyko)
- Add dedicated out_free_segnums label so the validation-failure path falls through the existing cleanup ladder rather than duplicating kfree(kbufs[4]) inline (Slava Dubeyko)
--- fs/nilfs2/segment.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
fs/nilfs2/segment.c +22 -0
@@ -2512,12 +2512,33 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv,struct nilfs_sc_info *sci = nilfs->ns_writer;struct nilfs_transaction_info ti;int err; +size_t i, nfreesegs = argv[4 ].v_nmembs; +__u64 *segnumv = kbufs[4 ];if (unlikely (!sci))return -EROFS ;nilfs_transaction_lock (sb, &ti,1 ); +/* +* Validate segment numbers under ns_segctor_sem (held for write +* by nilfs_transaction_lock above) so the check is serialized +* against nilfs_ioctl_resize(), which can modify ns_nsegments. +* Rejecting bad input here, before any segment-cleaning work +* begins, avoids the per-element diagnostic path inside +* nilfs_sufile_updatev() that would otherwise run under this +* same lock and stall concurrent readers. +*/ +for (i =0 ; i < nfreesegs; i++) { +if (segnumv[i] >= nilfs->ns_nsegments) { +nilfs_err (sb, +"Segment number %llu to be freed is out of range" , + (unsigned long long )segnumv[i]); + err = -EINVAL ; + goto bail_unlock; + } + } + err =nilfs_mdt_save_to_shadow_map (nilfs->ns_dat);if (unlikely (err)) goto out_unlock;
@@ -2558,6 +2579,7 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv, sci->sc_freesegs =NULL ; sci->sc_nfreesegs =0 ;nilfs_mdt_clear_shadow_map (nilfs->ns_dat); + bail_unlock:nilfs_transaction_unlock (sb);return err; }
Signature · 1 line
2.43.0
Ryusuke Konishi
maintainerQuoted 49 lines · Deepanshu Kartikey
Syzbot reported a hung task in
nilfs_transaction_begin()where multiple tasks performingchmod()on a nilfs2 mount blocked for over 143 seconds waiting to acquirens_segctor_semfor read:INFO: task syz.0.17:5918 blocked for more than 143 seconds. Call Trace: schedule+0x164/0x360 rwsem_down_read_slowpath+0x6d9/0x940 down_read+0x99/0x2e0 nilfs_transaction_begin+0x364/0x710 fs/nilfs2/segment.c:221 nilfs_setattr+0x124/0x2c0 fs/nilfs2/inode.c:921 notify_change+0xc1a/0xf40 chmod_common+0x273/0x4a0 do_fchmodat+0x12d/0x230The writer holding
ns_segctor_semwas a concurrentNILFS_IOCTL_CLEAN_SEGMENTScaller, stuck inside printk while emitting per-element warnings fromnilfs_sufile_updatev():__nilfs_msg+0x373/0x450 fs/nilfs2/super.c:78 nilfs_sufile_updatev+0x21c/0x6d0 fs/nilfs2/sufile.c:186 nilfs_sufile_freev fs/nilfs2/sufile.h:93 [inline] nilfs_free_segments fs/nilfs2/segment.c:1140 [inline] nilfs_segctor_collect_blocks fs/nilfs2/segment.c:1261 [inline] nilfs_segctor_do_construct+0x1f55/0x76c0 nilfs_clean_segments+0x3bd/0xa50 nilfs_ioctl_clean_segments fs/nilfs2/ioctl.c:922 [inline] nilfs_ioctl+0x261f/0x2780The root cause is that user-supplied segment numbers are not validated before
nilfs_clean_segments()begins doing work; the range check on each segnum is performed deep inside the call chain bynilfs_sufile_updatev(), which emits anilfs_warn()per invalid entry while still holding the segctor lock and the sufilemi_sem. Under load (repeated invocations across multiple mounts saturating the global printk path), the cumulative printk latency keepsns_segctor_semheld long enough to trip thehung_taskwatchdog, blocking concurrent operations such aschmod()that needns_segctor_semfor read.Fix by validating the contents of kbufs[4] in
nilfs_clean_segments()immediately after acquiringns_segctor_semvianilfs_transaction_lock(). Holdingns_segctor_semserializes the check againstnilfs_ioctl_resize(), which can modifyns_nsegments, so the validation uses a consistent value. Out-of-range segment numbers are rejected with-EINVALbefore any segment-cleaning work begins, so the bad entries never reach the per-element diagnostic path insidenilfs_sufile_updatev().
Quoted 4 lines
Reported-by [email protected]Tested-by [email protected]Fixes 4f6b828837b4 ("nilfs2: fix lock order reversal in nilfs_clean_segments ioctl")
The cause appears to be commit 071cb4b81987 ("nilfs2: eliminate removal list of segments"), which removed the segment release logic that used a list of segment information structures. Prior to that, the validity check of segment numbers was performed within nilfs_ioctl_prepare_clean_segments().
Everything else seems OK, so I'll fix only that tag myself, perform a final check, and then send it upstream.
Thanks,
Ryusuke Konishi
Quoted 76 lines
Signed-off-by Deepanshu Kartikey <[email protected]>---
Changes in v3:
- Move validation fromnilfs_ioctl_clean_segments()intonilfs_clean_segments(), underns_segctor_semheld for write bynilfs_transaction_lock(), to serialize againstnilfs_ioctl_resize()which can modifyns_nsegments(Ryusuke Konishi)
- Introduce local variables segnumv and nfreesegs for readability, rather than open-coding casts of kbufs[4] (Ryusuke Konishi)
- Emitnilfs_err()once on the first out-of-range segnum and bail out, instead ofnilfs_warn()per element (Ryusuke Konishi)
- Addbail_unlocklabel for the early-failure path, parallel to the existingout_unlockstructure (Ryusuke Konishi)Changes in v2:
- Reuse existing 'n' loop variable instead of introducing a new one (Slava Dubeyko)
- Add dedicatedout_free_segnumslabel so the validation-failure path falls through the existing cleanup ladder rather than duplicatingkfree(kbufs[4])inline (Slava Dubeyko)--- fs/nilfs2/segment.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)fs/nilfs2/segment.c +22
-1@@ -2512,12 +2512,33 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv,struct nilfs_sc_info *sci = nilfs->ns_writer;struct nilfs_transaction_info ti;int err; +size_t i, nfreesegs = argv[4 ].v_nmembs; +__u64 *segnumv = kbufs[4 ];if (unlikely (!sci))return -EROFS ;nilfs_transaction_lock (sb, &ti,1 ); +/* +* Validate segment numbers under ns_segctor_sem (held for write +* by nilfs_transaction_lock above) so the check is serialized +* against nilfs_ioctl_resize(), which can modify ns_nsegments. +* Rejecting bad input here, before any segment-cleaning work +* begins, avoids the per-element diagnostic path inside +* nilfs_sufile_updatev() that would otherwise run under this +* same lock and stall concurrent readers. +*/ +for (i =0 ; i < nfreesegs; i++) { +if (segnumv[i] >= nilfs->ns_nsegments) { +nilfs_err (sb, +"Segment number %llu to be freed is out of range" , + (unsigned long long )segnumv[i]); + err = -EINVAL ; + goto bail_unlock; + } + } + err =nilfs_mdt_save_to_shadow_map (nilfs->ns_dat);if (unlikely (err)) goto out_unlock;@@ -2558,6 +2579,7 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv, sci->sc_freesegs =NULL ; sci->sc_nfreesegs =0 ;nilfs_mdt_clear_shadow_map (nilfs->ns_dat); + bail_unlock:nilfs_transaction_unlock (sb);return err; } --2.43.0
Quoted 83 lines · Deepanshu Kartikey
Syzbot reported a hung task in
nilfs_transaction_begin()where multiple tasks performingchmod()on a nilfs2 mount blocked for over 143 seconds waiting to acquirens_segctor_semfor read:INFO: task syz.0.17:5918 blocked for more than 143 seconds. Call Trace: schedule+0x164/0x360 rwsem_down_read_slowpath+0x6d9/0x940 down_read+0x99/0x2e0 nilfs_transaction_begin+0x364/0x710 fs/nilfs2/segment.c:221 nilfs_setattr+0x124/0x2c0 fs/nilfs2/inode.c:921 notify_change+0xc1a/0xf40 chmod_common+0x273/0x4a0 do_fchmodat+0x12d/0x230The writer holding
ns_segctor_semwas a concurrentNILFS_IOCTL_CLEAN_SEGMENTScaller, stuck inside printk while emitting per-element warnings fromnilfs_sufile_updatev():__nilfs_msg+0x373/0x450 fs/nilfs2/super.c:78 nilfs_sufile_updatev+0x21c/0x6d0 fs/nilfs2/sufile.c:186 nilfs_sufile_freev fs/nilfs2/sufile.h:93 [inline] nilfs_free_segments fs/nilfs2/segment.c:1140 [inline] nilfs_segctor_collect_blocks fs/nilfs2/segment.c:1261 [inline] nilfs_segctor_do_construct+0x1f55/0x76c0 nilfs_clean_segments+0x3bd/0xa50 nilfs_ioctl_clean_segments fs/nilfs2/ioctl.c:922 [inline] nilfs_ioctl+0x261f/0x2780The root cause is that user-supplied segment numbers are not validated before
nilfs_clean_segments()begins doing work; the range check on each segnum is performed deep inside the call chain bynilfs_sufile_updatev(), which emits anilfs_warn()per invalid entry while still holding the segctor lock and the sufilemi_sem. Under load (repeated invocations across multiple mounts saturating the global printk path), the cumulative printk latency keepsns_segctor_semheld long enough to trip thehung_taskwatchdog, blocking concurrent operations such aschmod()that needns_segctor_semfor read.Fix by validating the contents of kbufs[4] in
nilfs_clean_segments()immediately after acquiringns_segctor_semvianilfs_transaction_lock(). Holdingns_segctor_semserializes the check againstnilfs_ioctl_resize(), which can modifyns_nsegments, so the validation uses a consistent value. Out-of-range segment numbers are rejected with-EINVALbefore any segment-cleaning work begins, so the bad entries never reach the per-element diagnostic path insidenilfs_sufile_updatev().Reported-by [email protected]Tested-by [email protected]Fixes 4f6b828837b4 ("nilfs2: fix lock order reversal in nilfs_clean_segments ioctl")Signed-off-by Deepanshu Kartikey <[email protected]>---
Changes in v3:
- Move validation fromnilfs_ioctl_clean_segments()intonilfs_clean_segments(), underns_segctor_semheld for write bynilfs_transaction_lock(), to serialize againstnilfs_ioctl_resize()which can modifyns_nsegments(Ryusuke Konishi)
- Introduce local variables segnumv and nfreesegs for readability, rather than open-coding casts of kbufs[4] (Ryusuke Konishi)
- Emitnilfs_err()once on the first out-of-range segnum and bail out, instead ofnilfs_warn()per element (Ryusuke Konishi)
- Addbail_unlocklabel for the early-failure path, parallel to the existingout_unlockstructure (Ryusuke Konishi)Changes in v2:
- Reuse existing 'n' loop variable instead of introducing a new one (Slava Dubeyko)
- Add dedicatedout_free_segnumslabel so the validation-failure path falls through the existing cleanup ladder rather than duplicatingkfree(kbufs[4])inline (Slava Dubeyko)--- fs/nilfs2/segment.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)fs/nilfs2/segment.c +0
-0
fs/nilfs2/segment.c int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv, · line 2514
Quoted 3 lines
fs/nilfs2/segment.c +0
-0@@ -2512,12 +2512,33 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv,struct nilfs_sc_info *sci = nilfs->ns_writer;struct nilfs_transaction_info ti;int err;
Usually, I prefer to keep the err variable at the end of declarations. Because, it is the ending state of the function. And I am feeling that something is wrong every time when likewise variable is hidden inside of declaration list. :) There is nothing critical in my remark. But anyway... :)
The path looks good to me.
Thanks,
Slava.
Quoted 38 lines
fs/nilfs2/segment.c +22
-0+size_t i, nfreesegs = argv[4 ].v_nmembs; +__u64 *segnumv = kbufs[4 ];if (unlikely (!sci))return -EROFS ;nilfs_transaction_lock (sb, &ti,1 ); +/* +* Validate segment numbers under ns_segctor_sem (held for write +* by nilfs_transaction_lock above) so the check is serialized +* against nilfs_ioctl_resize(), which can modify ns_nsegments. +* Rejecting bad input here, before any segment-cleaning work +* begins, avoids the per-element diagnostic path inside +* nilfs_sufile_updatev() that would otherwise run under this +* same lock and stall concurrent readers. +*/ +for (i =0 ; i < nfreesegs; i++) { +if (segnumv[i] >= nilfs->ns_nsegments) { +nilfs_err (sb, +"Segment number %llu to be freed is out of range" , + (unsigned long long )segnumv[i]); + err = -EINVAL ; + goto bail_unlock; + } + } + err =nilfs_mdt_save_to_shadow_map (nilfs->ns_dat);if (unlikely (err)) goto out_unlock;@@ -2558,6 +2579,7 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv, sci->sc_freesegs =NULL ; sci->sc_nfreesegs =0 ;nilfs_mdt_clear_shadow_map (nilfs->ns_dat); + bail_unlock:nilfs_transaction_unlock (sb);return err; }
Quoted 11 lines · Viacheslav Dubeyko
Usually, I prefer to keep the err variable at the end of declarations. Because, it is the ending state of the function. And I am feeling that something is wrong every time when likewise variable is hidden inside of declaration list. :) There is nothing critical in my remark. But anyway... :)
The path looks good to me.
Thanks,
Slava.
Hi Viacheslav,
Gentle Reminder. I want to know the status of the patch. Let me know if anything is required from my side.
Thanks
Deepanshu
[email protected]
maintainerQuoted 13 lines · Deepanshu Kartikey
Quoted 11 lines · Viacheslav Dubeyko
Usually, I prefer to keep the err variable at the end of declarations. Because,
it is the ending state of the function. And I am feeling that something is wrong
every time when likewise variable is hidden inside of declaration list. :) There
is nothing critical in my remark. But anyway... :)The path looks good to me.
Quoted 9 lines
Quoted 3 lines
Thanks,
Slava.Hi Viacheslav,
Gentle Reminder. I want to know the status of the patch. Let me know if anything is required from my side.
This patch in the for-next branch several weeks already. I will send it to upstream for 7.2-rc1 merge cycle.
Thanks,
Slava.