Skip to contentSkip to conversation

ceph: fix leaked inode reference on writeback abort at umount

PATCH In review Matthew Brown 2 participants 2 messages Aug 12 – Aug 13 Ceph client

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 ceph: fix leaked inode reference on writeback abort at umount

ceph_dirty_folio() takes a wrbuffer claim on each newly dirtied folio: it bumps i_wrbuffer_ref (taking an ihold() on the 0->1 transition) and attaches the snap_context to folio->private. That claim is released only by ceph_put_wrbuffer_cap_refs(), which for a submitted write runs from writepages_finish().

In ceph_submit_write(), if ceph_inc_osd_stopping_blocker() fails -- which happens during umount -- the request is aborted before submission: the already-collected folios are only redirtied and unlocked, so writepages_finish() never runs and the claim is leaked. redirty_page_for_writepage() -> folio_redirty_for_writepage() -> filemap_dirty_folio() sets PG_dirty directly and does not go through ->dirty_folio, so ceph_dirty_folio() is not re-entered to rebalance it. Because every subsequent writeback also fails the osd_stopping_blocker, i_wrbuffer_ref never returns to 0, the ihold() is never dropped, and the inode cannot be evicted:

VFS: Busy inodes after unmount of ceph
kernel BUG at fs/super.c:650!

Release the orphaned claim in the abort path before redirtying, via ceph_undo_wrbuffer_claim(): detach the snap_context, drop the wrbuffer reference (letting i_wrbuffer_ref reach 0 and iput() the inode), and drop the snap_context reference -- i.e. do what writepages_finish() would have done for these never-submitted folios.

Only the locked_pages entries are undone; folios still in the fbatch were never dirty-cleared by this call (folio_clear_dirty_for_io() is the ownership-transfer point, and a successful move NULLs the fbatch slot), so they hold no claim this call owns.

Fixes fd7449d937e7 ("ceph: fix generic/421 test failure")

---

v2:
- Reword the commit message: the leak is an orphaned wrbuffer claim on an unsubmitted write (writepages_finish() never runs), not redirty_page_for_writepage() re-entering ceph_dirty_folio(). Thanks to Alex Markuze for the correction.
- Convert ceph_undo_wrbuffer_claim() to the folio API
(struct folio * / folio_detach_private()), per Alex Markuze.
- Spell out the locked_pages/fbatch ownership split in the message, per Xiubo Li's comment.

v1: https://lore.kernel.org/all/[email protected]/

 fs/ceph/addr.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
fs/ceph/addr.c +11 -0
index ecf33b6..e9f561b 100644
@@ -1426,6 +1426,16 @@ void ceph_shift_unused_folios_left(struct folio_batch *fbatch)
 	fbatch->nr = n;
 }
 
+static void ceph_undo_wrbuffer_claim(struct inode *inode, struct folio *folio)
+{
+	struct ceph_snap_context *snapc = folio_detach_private(folio);
+
+	if (!snapc)
+		return;
+	ceph_put_wrbuffer_cap_refs(ceph_inode(inode), 1, snapc);
+	ceph_put_snap_context(snapc);
+}
+
 static
 int ceph_submit_write(struct address_space *mapping,
 			struct writeback_control *wbc,
@@ -1489,6 +1499,7 @@ int ceph_submit_write(struct address_space *mapping,
 			if (!page)
 				continue;
 
+			ceph_undo_wrbuffer_claim(inode, page_folio(page));
 			redirty_page_for_writepage(wbc, page);
 			unlock_page(page);
 		}
Signature · 1 line
2.44.0
Reviewed-by on v2 1/1
#1 lore raw

Hi Matthew,

LGTM.

Thanks
- Xiubo

Quoted 82 lines · Matthew Brown

ceph_dirty_folio() takes a wrbuffer claim on each newly dirtied folio: it bumps i_wrbuffer_ref (taking an ihold() on the 0->1 transition) and attaches the snap_context to folio->private. That claim is released only by ceph_put_wrbuffer_cap_refs(), which for a submitted write runs from writepages_finish().

In ceph_submit_write(), if ceph_inc_osd_stopping_blocker() fails -- which happens during umount -- the request is aborted before submission: the already-collected folios are only redirtied and unlocked, so writepages_finish() never runs and the claim is leaked. redirty_page_for_writepage() -> folio_redirty_for_writepage() -> filemap_dirty_folio() sets PG_dirty directly and does not go through ->dirty_folio, so ceph_dirty_folio() is not re-entered to rebalance it. Because every subsequent writeback also fails the osd_stopping_blocker, i_wrbuffer_ref never returns to 0, the ihold() is never dropped, and the inode cannot be evicted:

VFS: Busy inodes after unmount of ceph
kernel BUG at fs/super.c:650!

Release the orphaned claim in the abort path before redirtying, via ceph_undo_wrbuffer_claim(): detach the snap_context, drop the wrbuffer reference (letting i_wrbuffer_ref reach 0 and iput() the inode), and drop the snap_context reference -- i.e. do what writepages_finish() would have done for these never-submitted folios.

Only the locked_pages entries are undone; folios still in the fbatch were never dirty-cleared by this call (folio_clear_dirty_for_io() is the ownership-transfer point, and a successful move NULLs the fbatch slot), so they hold no claim this call owns.

Fixes fd7449d937e7 ("ceph: fix generic/421 test failure")

---

v2:
- Reword the commit message: the leak is an orphaned wrbuffer claim on an unsubmitted write (writepages_finish() never runs), not redirty_page_for_writepage() re-entering ceph_dirty_folio(). Thanks to Alex Markuze for the correction.
- Convert ceph_undo_wrbuffer_claim() to the folio API
(struct folio * / folio_detach_private()), per Alex Markuze.
- Spell out the locked_pages/fbatch ownership split in the message, per Xiubo Li's comment.

v1: https://lore.kernel.org/all/[email protected]/

 fs/ceph/addr.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
fs/ceph/addr.c +11 -1
index ecf33b6..e9f561b 100644
@@ -1426,6 +1426,16 @@ void ceph_shift_unused_folios_left(struct folio_batch *fbatch)
        fbatch->nr = n;
 }
 
+static void ceph_undo_wrbuffer_claim(struct inode *inode, struct folio *folio)
+{
+       struct ceph_snap_context *snapc = folio_detach_private(folio);
+
+       if (!snapc)
+               return;
+       ceph_put_wrbuffer_cap_refs(ceph_inode(inode), 1, snapc);
+       ceph_put_snap_context(snapc);
+}
+
 static
 int ceph_submit_write(struct address_space *mapping,
                        struct writeback_control *wbc,
@@ -1489,6 +1499,7 @@ int ceph_submit_write(struct address_space *mapping,
                        if (!page)
                                continue;
 
+                       ceph_undo_wrbuffer_claim(inode, page_folio(page));
                        redirty_page_for_writepage(wbc, page);
                        unlock_page(page);
                }
--

2.44.0