ceph: fix leaked inode reference on writeback abort at umount
PATCH v2 1/1 ceph: fix leaked inode reference on writeback abort at umount
ceph_dirty_folio() takes a wrbuffer claim on each newly dirtied folio: it bumps i_wrbuffer_ref (taking an ihold() on the 0->1 transition) and attaches the snap_context to folio->private. That claim is released only by ceph_put_wrbuffer_cap_refs(), which for a submitted write runs from writepages_finish().
In ceph_submit_write(), if ceph_inc_osd_stopping_blocker() fails -- which happens during umount -- the request is aborted before submission: the already-collected folios are only redirtied and unlocked, so writepages_finish() never runs and the claim is leaked. redirty_page_for_writepage() -> folio_redirty_for_writepage() -> filemap_dirty_folio() sets PG_dirty directly and does not go through ->dirty_folio, so ceph_dirty_folio() is not re-entered to rebalance it. Because every subsequent writeback also fails the osd_stopping_blocker, i_wrbuffer_ref never returns to 0, the ihold() is never dropped, and the inode cannot be evicted:
VFS: Busy inodes after unmount of ceph
kernel BUG at fs/super.c:650!
Release the orphaned claim in the abort path before redirtying, via ceph_undo_wrbuffer_claim(): detach the snap_context, drop the wrbuffer reference (letting i_wrbuffer_ref reach 0 and iput() the inode), and drop the snap_context reference -- i.e. do what writepages_finish() would have done for these never-submitted folios.
Only the locked_pages entries are undone; folios still in the fbatch were never dirty-cleared by this call (folio_clear_dirty_for_io() is the ownership-transfer point, and a successful move NULLs the fbatch slot), so they hold no claim this call owns.
---
v2:
- Reword the commit message: the leak is an orphaned wrbuffer claim on an unsubmitted write (writepages_finish() never runs), not redirty_page_for_writepage() re-entering ceph_dirty_folio(). Thanks to Alex Markuze for the correction.
- Convert ceph_undo_wrbuffer_claim() to the folio API
(struct folio * / folio_detach_private()), per Alex Markuze.
- Spell out the locked_pages/fbatch ownership split in the message, per Xiubo Li's comment.
v1: https://lore.kernel.org/all/[email protected]/
fs/ceph/addr.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
fs/ceph/addr.c +11 -0
@@ -1426,6 +1426,16 @@ void ceph_shift_unused_folios_left(struct folio_batch *fbatch) fbatch->nr = n; } +static void ceph_undo_wrbuffer_claim (struct inode *inode,struct folio *folio) +{ +struct ceph_snap_context *snapc =folio_detach_private (folio); + +if (!snapc) +return ; +ceph_put_wrbuffer_cap_refs (ceph_inode (inode),1 , snapc); +ceph_put_snap_context (snapc); +} + static int ceph_submit_write(struct address_space *mapping, struct writeback_control *wbc,
@@ -1489,6 +1499,7 @@ int ceph_submit_write(struct address_space *mapping,if (!page)continue ; +ceph_undo_wrbuffer_claim (inode,page_folio (page));redirty_page_for_writepage (wbc, page);unlock_page (page); }
Signature · 1 line
2.44.0
Hi Matthew,
LGTM.
Thanks
- Xiubo
Quoted 82 lines · Matthew Brown
ceph_dirty_folio()takes a wrbuffer claim on each newly dirtied folio: it bumpsi_wrbuffer_ref(taking anihold()on the 0->1 transition) and attaches thesnap_contexttofolio->private. That claim is released only byceph_put_wrbuffer_cap_refs(), which for a submitted write runs fromwritepages_finish().In
ceph_submit_write(), ifceph_inc_osd_stopping_blocker()fails -- which happens during umount -- the request is aborted before submission: the already-collected folios are only redirtied and unlocked, sowritepages_finish()never runs and the claim is leaked.redirty_page_for_writepage()->folio_redirty_for_writepage()->filemap_dirty_folio()setsPG_dirtydirectly and does not go through ->dirty_folio, soceph_dirty_folio()is not re-entered to rebalance it. Because every subsequent writeback also fails theosd_stopping_blocker,i_wrbuffer_refnever returns to 0, theihold()is never dropped, and the inode cannot be evicted:VFS: Busy inodes after unmount of ceph
kernel BUG atfs/super.c:650!Release the orphaned claim in the abort path before redirtying, via
ceph_undo_wrbuffer_claim(): detach thesnap_context, drop the wrbuffer reference (lettingi_wrbuffer_refreach 0 andiput()the inode), and drop thesnap_contextreference -- i.e. do whatwritepages_finish()would have done for these never-submitted folios.Only the
locked_pagesentries are undone; folios still in the fbatch were never dirty-cleared by this call (folio_clear_dirty_for_io()is the ownership-transfer point, and a successful move NULLs the fbatch slot), so they hold no claim this call owns.Fixes fd7449d937e7 ("ceph: fix generic/421 test failure")Signed-off-by Matthew Brown <[email protected]>---
v2:
- Reword the commit message: the leak is an orphaned wrbuffer claim on an unsubmitted write (writepages_finish()never runs), notredirty_page_for_writepage()re-enteringceph_dirty_folio(). Thanks to Alex Markuze for the correction.
- Convertceph_undo_wrbuffer_claim()to the folio API
(struct folio * /folio_detach_private()), per Alex Markuze.
- Spell out the locked_pages/fbatch ownership split in the message, per Xiubo Li's comment.v1: https://lore.kernel.org/all/[email protected]/
fs/ceph/addr.c | 11 +++++++++++ 1 file changed, 11 insertions(+)fs/ceph/addr.c +11
-1@@ -1426,6 +1426,16 @@ void ceph_shift_unused_folios_left(struct folio_batch *fbatch) fbatch->nr = n; } +static void ceph_undo_wrbuffer_claim (struct inode *inode,struct folio *folio) +{ +struct ceph_snap_context *snapc =folio_detach_private (folio); + +if (!snapc) +return ; +ceph_put_wrbuffer_cap_refs (ceph_inode (inode),1 , snapc); +ceph_put_snap_context (snapc); +} + static int ceph_submit_write(struct address_space *mapping, struct writeback_control *wbc,@@ -1489,6 +1499,7 @@ int ceph_submit_write(struct address_space *mapping,if (!page)continue ; +ceph_undo_wrbuffer_claim (inode,page_folio (page));redirty_page_for_writepage (wbc, page);unlock_page (page); } --2.44.0