Notes from March 12, 2026
Hi everybody,
Here are the notes from the last Linux Memory Hotness and Promotion call that happened on Thursday, March 12. Thanks to everybody who was involved!
These notes are intended to bring people up to speed who could not attend the call as well as keep the conversation going in between meetings.
----->o-----
Bharata updated that he is working on v6 of his patch series. He noted significant work being done on NUMAB since in the initial work he had bypassed the regular balancing mode; he made the NUMA hint faults work only with hot page promotion. In his next series of changes, he added back proper NUMAB=1 support.
This turned out to require that NUMA Balancing would start to depend on pghot. All of the functions are required by pghot and this will make it easier when future sources of page hotness come later.
A concern was earlier raised about memcg stats reporting which should now be fixed. There were other small fixes merged as well. It will take a week or so before patches will be ready to post. v6 should be posted by the next meeting. Nobody expressed direct concerns about this approach.
----->o-----
We chatted about the upcoming LSF/MM/BPF conference. Bharata's suggestion for his series was to continue the discussion on the mailing list since it was covered in last LSF/MM/BPF and then more recently at LPC.
----->o-----
Joshua had been doing some testing with tier-aware memcg limits. He found in the general case when everything is behaving as appropriate that there weren't major benefits. When there is a memcg performing poorly, however, this can punish those workloads as needed. He showed a graph that showed a very wide range of performance with NUMAB=2 and then a much smaller variance with tier-aware memcg limits. The important part is that since the variance is low that you then have direct control over where the average throughput could be. He is going to be including this data in the v2 posting of his patch series.
He wanted to get feedback from memcg stakeholders upstream and discussed providing some guidance to end users on how to put these pieces of the puzzle together. This includes translating tier-unaware memcg limits into tier-aware memcg limits.
Gregory noted that one of the next steps will be to test both tier-aware memcg limits with Bharata's series. Bharata would also take a look at Joshua's series.
Joshua sought feedback on the best way to compare with and without tier-aware memcg limits. Gregory said that we'll eventually move to workload testing but in the meantime it would be useful to understand if the floor of the throughput raises when you start to use pghot -- this would be an outright win. Joshua was planning on having pghot + tier-aware memcg limits data to share for LSF/MM/BPF.
----->o-----
Shivank updated on the progress for hardware assists for migrate_pages(). He said that he posted a v4 of the patch series a couple days ago that includes several changes. He pivoted to doing pre-copy of data from the source to the destination and, if successful, then the metadata is updated.
There were no major blockers for this work. David Hildenbrand suggested a few minor changes.
We also touched on non-temporal stores and enlightenment for migrate_pages(). Yiannis started work on this, there was some feedback upstream for a new migrate mode to be supported. Yiannis has been discussing this with his colleagues and will be able to update; he is targeting having patches on the list in the next two weeks.
Wei Xu had a similar approach to the new migrate mode and ended up splitting out a helper function that would check for both MIGRATE_ASYNC as well as the new mode.
----->o-----
Next meeting will be on Thursday, March 26 at 8:30am PDT (UTC-7), everybody is welcome: https://meet.google.com/jak-ytdx-hnm
Topics for the next meeting:
- v6 of Bharata's patch series, including NUMA Balancing dependency on pghot
- v2 of tier-aware memcg limits (reclaim fairness)
- v4 of enlightening migrate_pages() for hardware assists and how this work will be charged to userspace, including for memory compaction
- non-temporal stores enlightenment for memory tiering
- Gregory's testing of tier aware memcg limits with Bharata's changes
- discuss generalized subsystem for providing bandwidth information independent of the underlying platform, ideally through resctrl, otherwise utilizing bandwidth information will be challenging
+ preferably this bandwidth monitoring is not per NUMA node but rather slow and fast
Please let me know if you'd like to propose additional topics for discussion, thank you!