Skip to contentSkip to conversation

mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

PATCH In review David Carlier 7 participants 14 messages Mar 30 – Apr 2

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/page_io: fix PSWPIN undercount for large folios in sio_read_complete()

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages. For large folios this undercounts compared to the per-memcg path which correctly uses folio_nr_pages(), and compared to the bdev read paths which also use folio_nr_pages().

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Fixes a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_io.c +1 -1
index 63b262f4c5a9..1389cd57ca88 100644
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
 			folio_mark_uptodate(folio);
 			folio_unlock(folio);
 		}
-		count_vm_events(PSWPIN, sio->pages);
+		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
 	} else {
 		for (p = 0; p < sio->pages; p++) {
 			struct folio *folio = page_folio(sio->bvec[p].bv_page);
Signature · 1 line
2.53.0
maintainer
#1 lore raw
Quoted 8 lines · On Mon, 30 Mar 2026 08:12:29 +0100 David Carlier <[email protected]> wrote:

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages. For large folios this undercounts compared to the per-memcg path which correctly uses folio_nr_pages(), and compared to the bdev read paths which also use folio_nr_pages().

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Thanks. What are the userspace-visible runtime effects of the bug? Just /proc/vmstat wrongness?

Seems pretty minor, and it's been in there for several years. But I guess I'll slip a cc:stable in there, let it trickle back at a later time.

Quoted 14 lines
mm/page_io.c +1 -1
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
 			folio_mark_uptodate(folio);
 			folio_unlock(folio);
 		}
-		count_vm_events(PSWPIN, sio->pages);
+		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
 	} else {
 		for (p = 0; p < sio->pages; p++) {
 			struct folio *folio = page_folio(sio->bvec[p].bv_page);
Signature · 1 line
2.53.0
#2 lore raw
In reply to David Carlier
Quoted 7 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages. For large folios this undercounts compared to the per-memcg path which correctly uses folio_nr_pages(), and compared to the bdev read paths which also use folio_nr_pages().

Quoted 5 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Fixes a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")

The patch seems theoretically correct, but I’m wondering where we can swap in mTHP for filesystem-based swap?

In both do_swap_page() and shmem_swapin_folio(), we check data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating large folios. Am I missing something?

Quoted 18 lines
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_io.c +1 -2
index 63b262f4c5a9..1389cd57ca88 100644
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
                        folio_mark_uptodate(folio);
                        folio_unlock(folio);
                }
-               count_vm_events(PSWPIN, sio->pages);
+               count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
        } else {
                for (p = 0; p < sio->pages; p++) {
                        struct folio *folio = page_folio(sio->bvec[p].bv_page);
--
Quoted 2 lines

2.53.0

Thanks
Barry

#3 lore raw
Quoted 15 lines · Barry Song
Quoted 7 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages. For large folios this undercounts compared to the per-memcg path which correctly uses folio_nr_pages(), and compared to the bdev read paths which also use folio_nr_pages().

Quoted 5 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Fixes a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
Quoted 6 lines

The patch seems theoretically correct, but I’m wondering where we can swap in mTHP for filesystem-based swap?

In both do_swap_page() and shmem_swapin_folio(), we check data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating large folios. Am I missing something?

▎ The patch seems theoretically correct, but I'm wondering
▎ where we can swap in mTHP for filesystem-based swap?

▎ In both do_swap_page() and shmem_swapin_folio(), we check ▎ data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating ▎ large folios. Am I missing something?

You're right, I missed that. SWP_FS_OPS is only set by NFS and SMB which have no bdev, so SWP_SYNCHRONOUS_IO can never be set alongside it. Large folios can't currently reach this path since both do_swap_page() and shmem_swapin_folio() gate mTHP allocation on SWP_SYNCHRONOUS_IO.

That said, sio_read_complete() already calls count_mthp_stat() and the per-memcg accounting uses folio_nr_pages(), so the code seems written with large folios in mind even if the path is currently unreachable. Using sio->pages (bvec entry count) for a base-page count is still semantically wrong, but I understand the practical impact is nil today.

Happy to either drop this or keep it as a correctness cleanup, whatever you and Andrew prefer.

Cheers !

Quoted 24 lines
Quoted 19 lines
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_io.c +1 -2
index 63b262f4c5a9..1389cd57ca88 100644
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
                        folio_mark_uptodate(folio);
                        folio_unlock(folio);
                }
-               count_vm_events(PSWPIN, sio->pages);
+               count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
        } else {
                for (p = 0; p < sio->pages; p++) {
                        struct folio *folio = page_folio(sio->bvec[p].bv_page);
--
Quoted 2 lines

2.53.0

Thanks
Barry

#4 lore raw
Quoted 43 lines · David CARLIER
Quoted 15 lines · Barry Song
Quoted 7 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages. For large folios this undercounts compared to the per-memcg path which correctly uses folio_nr_pages(), and compared to the bdev read paths which also use folio_nr_pages().

Quoted 5 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Fixes a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
Quoted 6 lines

The patch seems theoretically correct, but I’m wondering where we can swap in mTHP for filesystem-based swap?

In both do_swap_page() and shmem_swapin_folio(), we check data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating large folios. Am I missing something?

▎ The patch seems theoretically correct, but I'm wondering
▎ where we can swap in mTHP for filesystem-based swap?

▎ In both do_swap_page() and shmem_swapin_folio(), we check ▎ data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating ▎ large folios. Am I missing something?

You're right, I missed that. SWP_FS_OPS is only set by NFS and SMB which have no bdev, so SWP_SYNCHRONOUS_IO can never be set alongside it. Large folios can't currently reach this path since both do_swap_page() and shmem_swapin_folio() gate mTHP allocation on SWP_SYNCHRONOUS_IO.

That said, sio_read_complete() already calls count_mthp_stat() and the per-memcg accounting uses folio_nr_pages(), so the code seems written with large folios in mind even if the path is currently unreachable. Using sio->pages (bvec entry count) for a base-page count is still semantically wrong, but I understand the practical impact is nil today.

Quoted 2 lines

Happy to either drop this or keep it as a correctness cleanup, whatever you and Andrew prefer.

Best to rework the patch description (clarify that it's a cleanup) and drop the Fixes: I guess.

Signature · 3 lines
Cheers,

David
reviewer
#5 lore raw
In reply to David CARLIER
Quoted 44 lines · David CARLIER
Quoted 15 lines · Barry Song
Quoted 7 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages. For large folios this undercounts compared to the per-memcg path which correctly uses folio_nr_pages(), and compared to the bdev read paths which also use folio_nr_pages().

Quoted 5 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Fixes a1a0dfd56f97 ("mm: handle THP in swap_*page_fs()")
Quoted 6 lines

The patch seems theoretically correct, but I’m wondering where we can swap in mTHP for filesystem-based swap?

In both do_swap_page() and shmem_swapin_folio(), we check data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating large folios. Am I missing something?

▎ The patch seems theoretically correct, but I'm wondering
▎ where we can swap in mTHP for filesystem-based swap?

▎ In both do_swap_page() and shmem_swapin_folio(), we check ▎ data_race(si->flags & SWP_SYNCHRONOUS_IO) before allocating ▎ large folios. Am I missing something?

You're right, I missed that. SWP_FS_OPS is only set by NFS and SMB which have no bdev, so SWP_SYNCHRONOUS_IO can never be set alongside it. Large folios can't currently reach this path since both do_swap_page() and shmem_swapin_folio() gate mTHP allocation on SWP_SYNCHRONOUS_IO.

That said, sio_read_complete() already calls count_mthp_stat() and the per-memcg accounting uses folio_nr_pages(), so the code seems written with large folios in mind even if the path is currently unreachable. Using sio->pages (bvec entry count) for a base-page count is still semantically wrong, but I understand the practical impact is nil today.

Quoted 3 lines

Happy to either drop this or keep it as a correctness cleanup, whatever you and Andrew prefer.

The patch looks correct, so I’d personally support keeping it after cleaning up the description, as David suggested.

Best Regards
Barry

#6 lore raw

PATCH v3 1/1 mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

In reply to David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

While large folios cannot currently reach this path (SWP_FS_OPS and SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically inconsistent with the per-memcg path which correctly uses folio_nr_pages().

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_io.c +1 -1
index 63b262f4c5a9..1389cd57ca88 100644
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
 			folio_mark_uptodate(folio);
 			folio_unlock(folio);
 		}
-		count_vm_events(PSWPIN, sio->pages);
+		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
 	} else {
 		for (p = 0; p < sio->pages; p++) {
 			struct folio *folio = page_folio(sio->bvec[p].bv_page);
Signature · 1 line
2.53.0
Acked-by on v3 1/1
#7 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

Quoted 10 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

While large folios cannot currently reach this path (SWP_FS_OPS and SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically inconsistent with the per-memcg path which correctly uses folio_nr_pages().

Quoted 4 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

For the next time: Please don't send new revisions as reply to other revisions :)

Quoted 9 lines
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_io.c +0 -0
index 63b262f4c5a9..1389cd57ca88 100644

mm/page_io.c static void sio_read_complete(struct kiocb *iocb, long ret) · line 503

Quoted 8 lines
mm/page_io.c +1 -1
@@ -497,7 +497,7 @@ static void sio_read_complete(struct kiocb *iocb, long ret)
 			folio_mark_uptodate(folio);
 			folio_unlock(folio);
 		}
-		count_vm_events(PSWPIN, sio->pages);
+		count_vm_events(PSWPIN, sio->len >> PAGE_SHIFT);
 	} else {
 		for (p = 0; p < sio->pages; p++) {
 			struct folio *folio = page_folio(sio->bvec[p].bv_page);

sio->len always covers full pages as processed in swap_read_folio_fs(), so there should not be any difference.

Signature · 3 lines
Cheers,

David
maintainer
#8 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

Quoted 11 lines · On Wed, 1 Apr 2026 10:32:30 +0200 "David Hildenbrand (Arm)" <[email protected]>…
Quoted 10 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

While large folios cannot currently reach this path (SWP_FS_OPS and SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically inconsistent with the per-memcg path which correctly uses folio_nr_pages().

Quoted 7 lines
Quoted 4 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

For the next time: Please don't send new revisions as reply to other revisions :)

Also, altering the Subject: causes akpm confusion (not hard). But having accurate titles is more important than avoiding akpm confusion, so there is that.

A little "what i changed since v2" note after the "---" separator is helpful, please.

But I figured it out! Thanks, I'll pop this into my after-rc1 pile.

reviewer Reviewed-by on v3 1/1
#9 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

In reply to David Carlier
Quoted 11 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

While large folios cannot currently reach this path (SWP_FS_OPS and SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically inconsistent with the per-memcg path which correctly uses folio_nr_pages().

Quoted 4 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

LGTM,

maintainer Acked-by on v3 1/1
#10 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

In reply to David Carlier
Quoted 11 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

While large folios cannot currently reach this path (SWP_FS_OPS and SWP_SYNCHRONOUS_IO are mutually exclusive, and mTHP swap-in allocation is gated on SWP_SYNCHRONOUS_IO), the accounting is semantically inconsistent with the per-memcg path which correctly uses folio_nr_pages().

Quoted 4 lines

Use sio->len >> PAGE_SHIFT instead, which gives the correct base page count since sio->len is accumulated via folio_size(folio).

Chris

#11 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

In reply to David Carlier
Quoted 3 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

Since you're messing with this file anyway, do you want to submit a third patch to rename swap_iocb->pages to ->bvecs?

reviewer
#12 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

Quoted 8 lines · Matthew Wilcox
Quoted 3 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

Since you're messing with this file anyway, do you want to submit a third patch to rename swap_iocb->pages to ->bvecs?

+1, that would be great.

#13 lore raw

Re: [PATCH v3] mm/page_io: use sio->len for PSWPIN accounting in sio_read_complete()

In reply to Matthew Wilcox

Sure, will send as a follow-up patch.

Cheers.

Quoted 8 lines · Matthew Wilcox
Quoted 3 lines · David Carlier

sio_read_complete() uses sio->pages to account global PSWPIN vm events, but sio->pages tracks the number of bvec entries (folios), not base pages.

Since you're messing with this file anyway, do you want to submit a third patch to rename swap_iocb->pages to ->bvecs?