migrate_pages(): batch TLB flushing
Status from this message: [merged mm-stable] migrate_pages-move-thp-hugetlb-migration-support-check-to-simplify-code.patch removed from -mm tree
Get series with b4
b4 am [email protected]Versions newest first
v5 from the cover letter
- Added Xin's test results on ARM 64. Thanks Xin!
- Fixed many comments. Thanks Zi and Xin!
- Collected reviewed-by and tested-by.
Patches in v1 All 9 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/9 | migrate_pages: organize stats with struct migrate_pages_statsmm/ | +34 | 0 replies | Applied |
| 2/9 | migrate_pages: separate hugetlb folios migrationmm/ | +119 | 0 replies | Applied |
| 3/9 | migrate_pages: restrict number of pages to migrate in batchmm/ | +106 | 0 replies | Applied |
| 4/9 | migrate_pages: split unmap_and_move() to _unmap() and _move()include/ mm/ | +129 | 0 replies | Applied |
| 5/9 | migrate_pages: batch _unmap and _movemm/ | +189 | 0 replies | Applied |
| 6/9 | migrate_pages: move migrate_folio_unmap()mm/ | +50 | 0 replies | Applied |
| 7/9 | migrate_pages: share more code between _unmap and _movemm/ | +85 | 0 replies | Applied |
| 8/9 | migrate_pages: batch flushing TLBmm/ mm/ | +21 | 0 replies | Applied |
| 9/9 | migrate_pages: move THP/hugetlb migration support check to simplify codemm/ | +36 | 0 replies | Applied |
Reviews trailers per patch
7 of 9 patches have a Reviewed-by or Acked-by.
| Patch | Alistair Popple | Zi Yan | Baolin Wang | Xin Hao | Hyeonggon Yoo |
|---|---|---|---|---|---|
| 1/9 | R-b | R-b | R-b | R-b | |
| 2/9 | R-b | R-b | |||
| 3/9 | R-b | ||||
| 4/9 | R-b | R-b | |||
| 5/9 | T-b | ||||
| 6/9 | R-b | ||||
| 7/9 | |||||
| 8/9 | R-b | T-b R-b | |||
| 9/9 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.
Cover letter [PATCH -v5 0/9] migrate_pages(): batch TLB flushing
From: "Huang, Ying" <[email protected]>
Now, migrate_pages() migrate folios one by one, like the fake code as follows,
for each folio
unmap
flush TLB
copy
restore map
If multiple folios are passed to migrate_pages(), there are opportunities to batch the TLB flushing and copying. That is, we can change the code to something as follows,
for each folio
unmap
for each folio
flush TLB
for each folio
copy
for each folio
restore map
The total number of TLB flushing IPI can be reduced considerably. And we may use some hardware accelerator such as DSA to accelerate the folio copying.
So in this patch, we refactor the migrate_pages() implementation and implement the TLB flushing batching. Base on this, hardware accelerated folio copying can be implemented.
If too many folios are passed to migrate_pages(), in the naive batched implementation, we may unmap too many folios at the same time. The possibility for a task to wait for the migrated folios to be mapped again increases. So the latency may be hurt. To deal with this issue, the max number of folios be unmapped in batch is restricted to no more than HPAGE_PMD_NR in the unit of page. That is, the influence is at the same level of THP migration.
We use the following test to measure the performance impact of the patchset,
On a 2-socket Intel server,
- Run pmbench memory accessing benchmark
- Run migratepages to migrate pages of pmbench between node 0 and node 1 back and forth.
With the patch, the TLB flushing IPI reduces 99.1% during the test and the number of pages migrated successfully per second increases 291.7%.
Xin Hao helped to test the patchset on an ARM64 server with 128 cores, 2 NUMA nodes. Test results show that the page migration performance increases up to 78%.
This patchset is based on mm-unstable 2023-02-10.
Changes:
v5:
- Added Xin's test results on ARM 64. Thanks Xin!
- Fixed many comments. Thanks Zi and Xin!
- Collected reviewed-by and tested-by.
v4:
- Fixed another bug about non-LRU folio migration. Thanks Hyeonggon!
v3:
- Rebased on v6.2-rc4
- Fixed a bug about non-LRU folio migration. Thanks Mike!
- Fixed some comments. Thanks Baolin!
- Collected reviewed-by.
v2:
- Rebased on v6.2-rc3
- Fixed type force cast warning. Thanks Kees!
- Added more comments and cleaned up the code. Thanks Andrew, Zi, Alistair, Dan!
- Collected reviewed-by.
from rfc to v1:
- Rebased on v6.2-rc1
- Fix the deadlock issue caused by locking multiple pages synchronously per Alistair's comments. Thanks!
- Fix the autonumabench panic per Rao's comments and fix. Thanks!
- Other minor fixes per comments. Thanks!
Best Regards,
Huang, Ying