per memcg lru lock
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v21 Latest19 patches 39 repliesIn review 15 of 19 applied
- v2020 patches 46 repliesSuperseded
- v1920 patches 10 repliesSuperseded
Earlier versions · 10
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v18 | 32 patches | 70 replies | Superseded | |
| v17 | 21 patches | 61 replies | Superseded | |
| v16 | 22 patches | 43 replies | Superseded | |
| v15 posting 2/2 | 20 of 21 patches | 2 replies | Superseded | |
| v15 posting 1/2 | 0 of 20 patches | 0 replies | Superseded | |
| v14 | 20 patches | 15 replies | Superseded | |
| v13 | 18 patches | 2 replies | Superseded | |
| v12 | 16 patches | 4 replies | Superseded | |
| v11 | 16 patches | 7 replies | Superseded | |
| v10 | 15 patches | 0 replies | Superseded |
Patches in v21 15 of 19 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/19 | mm/thp: move lru_add_page_tail func to huge_memory.cinclude/ mm/ mm/ | +30 | 0 replies | |
| 2/19 | mm/thp: use head for head page in lru_add_page_tailmm/ | +11 | 0 replies | Applied |
| 3/19 | mm/thp: Simplify lru_add_page_tail()mm/ | +6 | 0 replies | Applied |
| 4/19 | mm/thp: narrow lru lockingmm/ | +13 | 0 replies | Applied |
| 5/19 | mm/vmscan: remove unnecessary lruvec addingmm/ | +25 | 1 reply | Applied |
| 6/19 | mm/rmap: stop store reordering issue on page->mappingmm/ | +6 | 3 replies | |
| 7/19 | mm: page_idle_get_page() does not need lru_lockmm/ | +0 | 3 replies | Applied |
| 8/19 | mm/memcg: add debug checking in lock_page_memcgmm/ | +6 | 0 replies | Applied |
| 9/19 | mm/swap.c: fold vm event PGROTATED into pagevec_move_tail_fnmm/ | +23 | 0 replies | Applied |
| 10/19 | mm/lru: move lock into lru_note_costmm/ mm/ mm/ | +4 | 0 replies | Applied |
| 11/19 | mm/vmscan: remove lruvec reget in move_pages_to_lrumm/ | +6 | 0 replies | Applied |
| 12/19 | mm/mlock: remove lru_lock on TestClearPageMlockedmm/ | +5 | 1 reply | Applied |
| 13/19 | mm/mlock: remove __munlock_isolate_lru_pagemm/ | +9 | 1 reply | Applied |
| 14/19 | mm/lru: introduce TestClearPageLRUinclude/ mm/ mm/ | +21 | 3 replies | Applied |
| 15/19 | mm/compaction: do page isolation first in compactioninclude/ mm/ mm/ | +56 | 4 replies | Applied |
| 16/19 | mm/swap.c: serialize memcg changes in pagevec_lru_move_fnmm/ | +35 | 1 reply | Applied |
| 17/19 | mm/lru: replace pgdat lru_lock with lruvec lockinclude/ include/ mm/ +7 more | +270 | 7 replies | |
| 18/19 | mm/lru: introduce the relock_page_lruvec functioninclude/ mm/ mm/ +1 more | +62 | 3 replies | |
| 19/19 | mm/lru: revise the comments of lru_lockDocumentation/ Documentation/ Documentation/ +6 more | +50 | 1 reply | Applied |
Reviews trailers per patch
All 19 patches have a Reviewed-by or Acked-by.
| Patch | Kirill A. Shutemov | Hugh Dickins | Johannes Weiner | Matthew Wilcox (Oracle) | Huang, Ying | Vlastimil Babka |
|---|---|---|---|---|---|---|
| 1/19 | R-b | A-b | A-b | |||
| 2/19 | R-b | A-b | A-b | R-b | ||
| 3/19 | R-b | A-b | ||||
| 4/19 | R-b | A-b | ||||
| 5/19 | A-b | A-b | A-b | |||
| 6/19 | A-b | A-b | ||||
| 7/19 | A-b | A-b | A-b | |||
| 8/19 | A-b | A-b | ||||
| 9/19 | A-b | A-b | ||||
| 10/19 | A-b | A-b | ||||
| 11/19 | A-b | A-b | ||||
| 12/19 | A-b | A-b | A-b | |||
| 13/19 | A-b | A-b | A-b | |||
| 14/19 | A-b | A-b | A-b | |||
| 15/19 | A-b | A-b | A-b | |||
| 16/19 | A-b | A-b | A-b | |||
| 17/19 | A-b A-b A-b | A-b | A-b | |||
| 18/19 | A-b A-b | A-b A-b A-b | A-b | |||
| 19/19 | A-b | A-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.
Cover letter [PATCH v21 00/19] per memcg lru lock
This version rebase on next/master 20201104, with much of Johannes's Acks and some changes according to Johannes comments. And add a new patch v21-0006-mm-rmap-stop-store-reordering-issue-on-page-mapp.patch to support v21-0007.
This patchset followed 2 memcg VM_WARN_ON_ONCE_PAGE patches which were added to -mm tree yesterday.
Many thanks for line by line review by Hugh Dickins, Alexander Duyck and Johannes Weiner.
So now this patchset includes 3 parts:
1, some code cleanup and minimum optimization as a preparation. 2, use TestCleanPageLRU as page isolation's precondition. 3, replace per node lru_lock with per memcg per node lru_lock.
Current lru_lock is one for each of node, pgdat->lru_lock, that guard for lru lists, but now we had moved the lru lists into memcg for long time. Still using per node lru_lock is clearly unscalable, pages on each of memcgs have to compete each others for a whole lru_lock. This patchset try to use per lruvec/memcg lru_lock to repleace per node lru lock to guard lru lists, make it scalable for memcgs and get performance gain.
Currently lru_lock still guards both lru list and page's lru bit, that's ok. but if we want to use specific lruvec lock on the page, we need to pin down the page's lruvec/memcg during locking. Just taking lruvec lock first may be undermined by the page's memcg charge/migration. To fix this problem, we could take out the page's lru bit clear and use it as pin down action to block the memcg changes. That's the reason for new atomic func TestClearPageLRU. So now isolating a page need both actions: TestClearPageLRU and hold the lru_lock.
The typical usage of this is isolate_migratepages_block() in compaction.c we have to take lru bit before lru lock, that serialized the page isolation in memcg page charge/migration which will change page's lruvec and new lru_lock in it.
The above solution suggested by Johannes Weiner, and based on his new memcg charge path, then have this patchset. (Hugh Dickins tested and contributed much code from compaction fix to general code polish, thanks a lot!).
Daniel Jordan's testing show 62% improvement on modified readtwice case on his 2P * 10 core * 2 HT broadwell box on v18, which has no much different with this v20.
https://lore.kernel.org/lkml/20200915165807.kpp7uhiw7l3loofu-S51bK0XF4qpuJJETbFA3a0B3C2bhBk7L0E9HWUfgJXw@public.gmane.org/
Thanks Hugh Dickins and Konstantin Khlebnikov, they both brought this idea 8 years ago, and others who give comments as well: Daniel Jordan, Mel Gorman, Shakeel Butt, Matthew Wilcox, Alexander Duyck etc.
Thanks for Testing support from Intel 0day and Rong Chen, Fengguang Wu, and Yun Wang. Hugh Dickins also shared his kbuild-swap case. Thanks!
Alex Shi (16):
mm/thp: move lru_add_page_tail func to huge_memory.c mm/thp: use head for head page in lru_add_page_tail mm/thp: Simplify lru_add_page_tail()
mm/thp: narrow lru locking
mm/vmscan: remove unnecessary lruvec adding mm/rmap: stop store reordering issue on page->mapping mm/memcg: add debug checking in lock_page_memcg mm/swap.c: fold vm event PGROTATED into pagevec_move_tail_fn mm/lru: move lock into lru_note_cost
mm/vmscan: remove lruvec reget in move_pages_to_lru mm/mlock: remove lru_lock on TestClearPageMlocked mm/mlock: remove __munlock_isolate_lru_page mm/lru: introduce TestClearPageLRU
mm/compaction: do page isolation first in compaction mm/swap.c: serialize memcg changes in pagevec_lru_move_fn mm/lru: replace pgdat lru_lock with lruvec lock
Alexander Duyck (1):
mm/lru: introduce the relock_page_lruvec function
Hugh Dickins (2):
mm: page_idle_get_page() does not need lru_lock
mm/lru: revise the comments of lru_lock
Documentation/admin-guide/cgroup-v1/memcg_test.rst | 15 +- Documentation/admin-guide/cgroup-v1/memory.rst | 21 +-- Documentation/trace/events-kmem.rst | 2 +- Documentation/vm/unevictable-lru.rst | 22 +-- include/linux/memcontrol.h | 110 +++++++++++ include/linux/mm_types.h | 2 +- include/linux/mmzone.h | 6 +- include/linux/page-flags.h | 1 + include/linux/swap.h | 4 +- mm/compaction.c | 94 +++++++--- mm/filemap.c | 4 +- mm/huge_memory.c | 45 +++-- mm/memcontrol.c | 79 +++++++- mm/mlock.c | 63 ++----- mm/mmzone.c | 1 + mm/page_alloc.c | 1 - mm/page_idle.c | 4 - mm/rmap.c | 11 +- mm/swap.c | 208 ++++++++------------- mm/vmscan.c | 207 ++++++++++---------- mm/workingset.c | 2 - 21 files changed, 530 insertions(+), 372 deletions(-)
Signature · 1 line
1.8.3.1