Skip to contentSkip to conversation

Weighted interleave auto-tuning

LSF/MM/BPF TOPIC Joshua Hahn 5 participants 10 messages Jan 9, 2025 – Mar 27, 2025 Memory management

lore
Share & tools
lore
#0 lore raw

Hello everyone, I hope everyone has had a great start to 2025!

Recently, I have been working on a patch series [1] with
Gregory Price <[email protected]> that provides new default interleave weights, along with dynamic re-weighting on hotplug events and a series of UAPIs that allow users to configure how they want the defaults to behave.

In introducing these new defaults, discussions have opened up in the community regarding how best to create a UAPI that can provide coherent and transparent interactions for the user. In particular, consider this scenario: when a hotplug event happens and a node comes online with new bandwidth information (and therefore changing the bandwidth distributions across the system), should user-set weights be overwritten to reflect the new distributions? If so, how can we justify overwriting user-set values in a sysfs interface? If not, how will users manually adjust the node weights to the optimal weight?

I would like to revisit some of the design choices made for this patch, including how the defaults were derived, and open the conversation to hear what the community believes is a reasonable way to allow users to tune weighted interleave weights. More broadly, I hope to get gather community insight on how they use weighted interleave, and do my best to reflect those workflows in the patch.

Of course, I would also love to hear your thoughts about this topic in this thread, or in the RFC thread (attached) as well. Have a great day! Joshua

[1] https://lore.kernel.org/all/[email protected]/

#1 lore raw
Quoted 17 lines · On Thu, 9 Jan 2025 13:50:48 -0500 Joshua Hahn <[email protected]> wrote:

Hello everyone, I hope everyone has had a great start to 2025!

Recently, I have been working on a patch series [1] with
Gregory Price <[email protected]> that provides new default interleave weights, along with dynamic re-weighting on hotplug events and a series of UAPIs that allow users to configure how they want the defaults to behave.

In introducing these new defaults, discussions have opened up in the community regarding how best to create a UAPI that can provide coherent and transparent interactions for the user. In particular, consider this scenario: when a hotplug event happens and a node comes online with new bandwidth information (and therefore changing the bandwidth distributions across the system), should user-set weights be overwritten to reflect the new distributions? If so, how can we justify overwriting user-set values in a sysfs interface? If not, how will users manually adjust the node weights to the optimal weight?

Quoted 6 lines

I would like to revisit some of the design choices made for this patch, including how the defaults were derived, and open the conversation to hear what the community believes is a reasonable way to allow users to tune weighted interleave weights. More broadly, I hope to get gather community insight on how they use weighted interleave, and do my best to reflect those workflows in the patch.

Weighted interleave has since moved onto v7 [1], and a v8 is currently being drafted. Through feedback from reviewers, we have landed on a coherent UAPI that gives users two options: auto mode, which leaves all weight calculation decisions to the system, and manual mode, which leaves weighted interleave the same as it is without the patch.

Given that the patch's functionality is mostly concrete and that the questions I hoped to raise during this slot were answered via patch feedback, I hope to ask another question during the talk:

Should the system dynamically change what metrics it uses to weight the nodes, based on what bottlenecks the system is currently facing?

In the patch, min(read_bandwidth, write_bandwidth) is used as the heuristic to determine what a node's weight should be. However, what if the system is not bottlenecked by bandwidth, but by latency? A system could also be bottlenecked by read bandwidth, but not by write bandwidth.

Consider a scenario where a system has many memory nodes with varying latencies and bandwidths. When the system is not bottlenecked by bandwidth, it might prefer to allocate memory from nodes with lower latency. Once the system starts feeling pressured by bandwidth, the weights for high bandwidth (but also high latency) nodes would slowly increase to alleviate pressure from the system. Once the system is back in a manageable state, weights for low latency nodes would start increasing again. Users would not have to be aware of any of this -- they would just see the system take control of the weight changes as the system's needs continue to change.

This proposal also has some concerns that need to be addressed:
- How reactive should the system be, and how aggressively should it tune the weights? We don't want the system to overreact to short spikes in pressure.
- Does dynamic weight adjusting lead to pages being "misplaced"? Should those "misplaced" pages be migrated? (probably not)
- Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces).

Thoughts & comments are appreciated! Thank you, and have a great day! Joshua

[1] https://lore.kernel.org/all/[email protected]/

Sent using hkml (https://github.com/sjp38/hackermail)

#2 lore raw
Quoted 44 lines · Joshua Hahn
Quoted 17 lines · On Thu, 9 Jan 2025 13:50:48 -0500 Joshua Hahn <[email protected]> wrote:

Hello everyone, I hope everyone has had a great start to 2025!

Recently, I have been working on a patch series [1] with
Gregory Price <[email protected]> that provides new default interleave weights, along with dynamic re-weighting on hotplug events and a series of UAPIs that allow users to configure how they want the defaults to behave.

In introducing these new defaults, discussions have opened up in the community regarding how best to create a UAPI that can provide coherent and transparent interactions for the user. In particular, consider this scenario: when a hotplug event happens and a node comes online with new bandwidth information (and therefore changing the bandwidth distributions across the system), should user-set weights be overwritten to reflect the new distributions? If so, how can we justify overwriting user-set values in a sysfs interface? If not, how will users manually adjust the node weights to the optimal weight?

Quoted 6 lines

I would like to revisit some of the design choices made for this patch, including how the defaults were derived, and open the conversation to hear what the community believes is a reasonable way to allow users to tune weighted interleave weights. More broadly, I hope to get gather community insight on how they use weighted interleave, and do my best to reflect those workflows in the patch.

Weighted interleave has since moved onto v7 [1], and a v8 is currently being drafted. Through feedback from reviewers, we have landed on a coherent UAPI that gives users two options: auto mode, which leaves all weight calculation decisions to the system, and manual mode, which leaves weighted interleave the same as it is without the patch.

Given that the patch's functionality is mostly concrete and that the questions I hoped to raise during this slot were answered via patch feedback, I hope to ask another question during the talk:

Should the system dynamically change what metrics it uses to weight the nodes, based on what bottlenecks the system is currently facing?

In the patch, min(read_bandwidth, write_bandwidth) is used as the heuristic to determine what a node's weight should be. However, what if the system is not bottlenecked by bandwidth, but by latency? A system could also be bottlenecked by read bandwidth, but not by write bandwidth.

Quoted 9 lines

Consider a scenario where a system has many memory nodes with varying latencies and bandwidths. When the system is not bottlenecked by bandwidth, it might prefer to allocate memory from nodes with lower latency. Once the system starts feeling pressured by bandwidth, the weights for high bandwidth (but also high latency) nodes would slowly increase to alleviate pressure from the system. Once the system is back in a manageable state, weights for low latency nodes would start increasing again. Users would not have to be aware of any of this -- they would just see the system take control of the weight changes as the system's needs continue to change.

IIUC, this assumes the capacity of all kinds of memory is large enough. However, this may be not true in some cases. So, another possibility is that, for a system with DRAM and CXL memory nodes.

- There is free space on DRAM node, the bandwidth of DRAM node isn't saturated, memory is allocated on DRAM node.

- There is no free space on DRAM node, the bandwidth of DRAM node isn't saturated, cold pages are migrated to CXL memory nodes, while hot pages are migrated to DRAM memory nodes.

- The bandwidth of DRAM node is saturated, hot pages are migrated to CXL memory nodes.

In general, I think that the real situation is complex and this makes it hard to implement a good policy in kernel. So, I suspect that it's better to start with the experiments in user space.

Quoted 8 lines

This proposal also has some concerns that need to be addressed:
- How reactive should the system be, and how aggressively should it tune the weights? We don't want the system to overreact to short spikes in pressure.
- Does dynamic weight adjusting lead to pages being "misplaced"? Should those "misplaced" pages be migrated? (probably not)
- Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces).

Quoted 6 lines

Thoughts & comments are appreciated! Thank you, and have a great day! Joshua

[1] https://lore.kernel.org/all/[email protected]/

Sent using hkml (https://github.com/sjp38/hackermail)

---
Best Regards,
Huang, Ying

#3 lore raw
Quoted 67 lines · Ying
Quoted 44 lines · Joshua Hahn
Quoted 17 lines · On Thu, 9 Jan 2025 13:50:48 -0500 Joshua Hahn <[email protected]> wrote:

Hello everyone, I hope everyone has had a great start to 2025!

Recently, I have been working on a patch series [1] with
Gregory Price <[email protected]> that provides new default interleave weights, along with dynamic re-weighting on hotplug events and a series of UAPIs that allow users to configure how they want the defaults to behave.

In introducing these new defaults, discussions have opened up in the community regarding how best to create a UAPI that can provide coherent and transparent interactions for the user. In particular, consider this scenario: when a hotplug event happens and a node comes online with new bandwidth information (and therefore changing the bandwidth distributions across the system), should user-set weights be overwritten to reflect the new distributions? If so, how can we justify overwriting user-set values in a sysfs interface? If not, how will users manually adjust the node weights to the optimal weight?

Quoted 6 lines

I would like to revisit some of the design choices made for this patch, including how the defaults were derived, and open the conversation to hear what the community believes is a reasonable way to allow users to tune weighted interleave weights. More broadly, I hope to get gather community insight on how they use weighted interleave, and do my best to reflect those workflows in the patch.

Weighted interleave has since moved onto v7 [1], and a v8 is currently being drafted. Through feedback from reviewers, we have landed on a coherent UAPI that gives users two options: auto mode, which leaves all weight calculation decisions to the system, and manual mode, which leaves weighted interleave the same as it is without the patch.

Given that the patch's functionality is mostly concrete and that the questions I hoped to raise during this slot were answered via patch feedback, I hope to ask another question during the talk:

Should the system dynamically change what metrics it uses to weight the nodes, based on what bottlenecks the system is currently facing?

In the patch, min(read_bandwidth, write_bandwidth) is used as the heuristic to determine what a node's weight should be. However, what if the system is not bottlenecked by bandwidth, but by latency? A system could also be bottlenecked by read bandwidth, but not by write bandwidth.

Quoted 9 lines

Consider a scenario where a system has many memory nodes with varying latencies and bandwidths. When the system is not bottlenecked by bandwidth, it might prefer to allocate memory from nodes with lower latency. Once the system starts feeling pressured by bandwidth, the weights for high bandwidth (but also high latency) nodes would slowly increase to alleviate pressure from the system. Once the system is back in a manageable state, weights for low latency nodes would start increasing again. Users would not have to be aware of any of this -- they would just see the system take control of the weight changes as the system's needs continue to change.

IIUC, this assumes the capacity of all kinds of memory is large enough. However, this may be not true in some cases. So, another possibility is that, for a system with DRAM and CXL memory nodes.

- There is free space on DRAM node, the bandwidth of DRAM node isn't saturated, memory is allocated on DRAM node.

- There is no free space on DRAM node, the bandwidth of DRAM node isn't saturated, cold pages are migrated to CXL memory nodes, while hot pages are migrated to DRAM memory nodes.

Quoted 14 lines

- The bandwidth of DRAM node is saturated, hot pages are migrated to CXL memory nodes.

In general, I think that the real situation is complex and this makes it hard to implement a good policy in kernel. So, I suspect that it's better to start with the experiments in user space.

Quoted 7 lines

This proposal also has some concerns that need to be addressed:
- How reactive should the system be, and how aggressively should it tune the weights? We don't want the system to overreact to short spikes in pressure.
- Does dynamic weight adjusting lead to pages being "misplaced"? Should those "misplaced" pages be migrated? (probably not)
- Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces).

If this was done in kernel, what metrics would make sense to drive this? Similar to hot page tracking we may run into contention with PMUs or similar and their other use cases.

Quoted 12 lines
Quoted 7 lines

Thoughts & comments are appreciated! Thank you, and have a great day! Joshua

[1] https://lore.kernel.org/all/[email protected]/

Sent using hkml (https://github.com/sjp38/hackermail)

---
Best Regards,
Huang, Ying

#4 lore raw
Quoted 7 lines · Jonathan Cameron
Quoted 2 lines
Quoted 2 lines

- Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces).

If this was done in kernel, what metrics would make sense to drive this? Similar to hot page tracking we may run into contention with PMUs or similar and their other use cases.

Rather than directly affect weighted interleave, I think this stemmed from the idea of a "smart policy" that adjusted allocations based on bandwidth pressure and VMA permissions (code should be local, stack should be local, heap could be interleaved - etc).

An example would be if DRAM bandwidth become pressured but CXL wasn't, then maybe tossing some extra allocations directly to CXL would actually decrease average latencies.

I'm not sure how we'd actually implement this in userland, and I think this is ultimately MPOL_PONIES, but it's an interesting exploration.

Some of this context was lost as we worked on weighted interleave auto-tuning.

~Gregory

#5 lore raw
In reply to Jonathan Cameron
Quoted 85 lines · On Fri, 14 Mar 2025 14:15:41 +0000 Jonathan Cameron…
Quoted 67 lines · Ying
Quoted 44 lines · Joshua Hahn
Quoted 17 lines · On Thu, 9 Jan 2025 13:50:48 -0500 Joshua Hahn <[email protected]> wrote:

Hello everyone, I hope everyone has had a great start to 2025!

Recently, I have been working on a patch series [1] with
Gregory Price <[email protected]> that provides new default interleave weights, along with dynamic re-weighting on hotplug events and a series of UAPIs that allow users to configure how they want the defaults to behave.

In introducing these new defaults, discussions have opened up in the community regarding how best to create a UAPI that can provide coherent and transparent interactions for the user. In particular, consider this scenario: when a hotplug event happens and a node comes online with new bandwidth information (and therefore changing the bandwidth distributions across the system), should user-set weights be overwritten to reflect the new distributions? If so, how can we justify overwriting user-set values in a sysfs interface? If not, how will users manually adjust the node weights to the optimal weight?

Quoted 6 lines

I would like to revisit some of the design choices made for this patch, including how the defaults were derived, and open the conversation to hear what the community believes is a reasonable way to allow users to tune weighted interleave weights. More broadly, I hope to get gather community insight on how they use weighted interleave, and do my best to reflect those workflows in the patch.

Weighted interleave has since moved onto v7 [1], and a v8 is currently being drafted. Through feedback from reviewers, we have landed on a coherent UAPI that gives users two options: auto mode, which leaves all weight calculation decisions to the system, and manual mode, which leaves weighted interleave the same as it is without the patch.

Given that the patch's functionality is mostly concrete and that the questions I hoped to raise during this slot were answered via patch feedback, I hope to ask another question during the talk:

Should the system dynamically change what metrics it uses to weight the nodes, based on what bottlenecks the system is currently facing?

In the patch, min(read_bandwidth, write_bandwidth) is used as the heuristic to determine what a node's weight should be. However, what if the system is not bottlenecked by bandwidth, but by latency? A system could also be bottlenecked by read bandwidth, but not by write bandwidth.

Quoted 9 lines

Consider a scenario where a system has many memory nodes with varying latencies and bandwidths. When the system is not bottlenecked by bandwidth, it might prefer to allocate memory from nodes with lower latency. Once the system starts feeling pressured by bandwidth, the weights for high bandwidth (but also high latency) nodes would slowly increase to alleviate pressure from the system. Once the system is back in a manageable state, weights for low latency nodes would start increasing again. Users would not have to be aware of any of this -- they would just see the system take control of the weight changes as the system's needs continue to change.

IIUC, this assumes the capacity of all kinds of memory is large enough. However, this may be not true in some cases. So, another possibility is that, for a system with DRAM and CXL memory nodes.

- There is free space on DRAM node, the bandwidth of DRAM node isn't saturated, memory is allocated on DRAM node.

- There is no free space on DRAM node, the bandwidth of DRAM node isn't saturated, cold pages are migrated to CXL memory nodes, while hot pages are migrated to DRAM memory nodes.

Quoted 14 lines

- The bandwidth of DRAM node is saturated, hot pages are migrated to CXL memory nodes.

In general, I think that the real situation is complex and this makes it hard to implement a good policy in kernel. So, I suspect that it's better to start with the experiments in user space.

Quoted 7 lines

This proposal also has some concerns that need to be addressed:
- How reactive should the system be, and how aggressively should it tune the weights? We don't want the system to overreact to short spikes in pressure.
- Does dynamic weight adjusting lead to pages being "misplaced"? Should those "misplaced" pages be migrated? (probably not)
- Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces).

Quoted 3 lines

If this was done in kernel, what metrics would make sense to drive this? Similar to hot page tracking we may run into contention with PMUs or similar and their other use cases.

Hello Jonathan, thank you for your interest in this proposal!

Yes, I think you and Ying both bring up great points about how this is probably something more suitable for a userspace program. Usespace probably has more information about the characteristics of the workload, and I agree with your point about contention.

If the kernel thread doesn't probe frequently, then it would be making poor allocation decisions based on stale data, but if it does probe frequently, it would incur lots of overhead from the contention (and make other contending threads slower as well). Not to mention, there is also the overhead of probing itself : -)

I will keep thinking about these questions, and see if I can come up with any interesting ideas to discuss during LSFMMBPF. Thank you again for your interest, I hope you have a great day!
Joshua

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Thoughts & comments are appreciated! Thank you, and have a great day! Joshua

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

[1] https://lore.kernel.org/all/[email protected]/

Sent using hkml (https://github.com/sjp38/hackermail)

---
Best Regards,
Huang, Ying

Sent using hkml (https://github.com/sjp38/hackermail)

#6 lore raw
In reply to Huang, Ying
Quoted 67 lines · Ying
Quoted 44 lines · Joshua Hahn
Quoted 17 lines · On Thu, 9 Jan 2025 13:50:48 -0500 Joshua Hahn <[email protected]> wrote:

Hello everyone, I hope everyone has had a great start to 2025!

Recently, I have been working on a patch series [1] with
Gregory Price <[email protected]> that provides new default interleave weights, along with dynamic re-weighting on hotplug events and a series of UAPIs that allow users to configure how they want the defaults to behave.

In introducing these new defaults, discussions have opened up in the community regarding how best to create a UAPI that can provide coherent and transparent interactions for the user. In particular, consider this scenario: when a hotplug event happens and a node comes online with new bandwidth information (and therefore changing the bandwidth distributions across the system), should user-set weights be overwritten to reflect the new distributions? If so, how can we justify overwriting user-set values in a sysfs interface? If not, how will users manually adjust the node weights to the optimal weight?

Quoted 6 lines

I would like to revisit some of the design choices made for this patch, including how the defaults were derived, and open the conversation to hear what the community believes is a reasonable way to allow users to tune weighted interleave weights. More broadly, I hope to get gather community insight on how they use weighted interleave, and do my best to reflect those workflows in the patch.

Weighted interleave has since moved onto v7 [1], and a v8 is currently being drafted. Through feedback from reviewers, we have landed on a coherent UAPI that gives users two options: auto mode, which leaves all weight calculation decisions to the system, and manual mode, which leaves weighted interleave the same as it is without the patch.

Given that the patch's functionality is mostly concrete and that the questions I hoped to raise during this slot were answered via patch feedback, I hope to ask another question during the talk:

Should the system dynamically change what metrics it uses to weight the nodes, based on what bottlenecks the system is currently facing?

In the patch, min(read_bandwidth, write_bandwidth) is used as the heuristic to determine what a node's weight should be. However, what if the system is not bottlenecked by bandwidth, but by latency? A system could also be bottlenecked by read bandwidth, but not by write bandwidth.

Quoted 9 lines

Consider a scenario where a system has many memory nodes with varying latencies and bandwidths. When the system is not bottlenecked by bandwidth, it might prefer to allocate memory from nodes with lower latency. Once the system starts feeling pressured by bandwidth, the weights for high bandwidth (but also high latency) nodes would slowly increase to alleviate pressure from the system. Once the system is back in a manageable state, weights for low latency nodes would start increasing again. Users would not have to be aware of any of this -- they would just see the system take control of the weight changes as the system's needs continue to change.

IIUC, this assumes the capacity of all kinds of memory is large enough. However, this may be not true in some cases. So, another possibility is that, for a system with DRAM and CXL memory nodes.

- There is free space on DRAM node, the bandwidth of DRAM node isn't saturated, memory is allocated on DRAM node.

- There is no free space on DRAM node, the bandwidth of DRAM node isn't saturated, cold pages are migrated to CXL memory nodes, while hot pages are migrated to DRAM memory nodes.

Quoted 6 lines

- The bandwidth of DRAM node is saturated, hot pages are migrated to CXL memory nodes.

In general, I think that the real situation is complex and this makes it hard to implement a good policy in kernel. So, I suspect that it's better to start with the experiments in user space.

Hi Ying, thank you so much for your feedback, as always!

Yes, I agree. I brought up this idea out of curiosity, since I thought that there might be room to experiment with different configurations for weighted interleave auto-tuning. As you know, we use min(read_bw, write_bw), which I think is a good heuristic that works for the intent of the weighted interleave auto-tuning patch-- I wanted to know what a system might look like, that might use different heuristics given the system's state. But I think you are right that it is difficult to implement in kernel.

Thanks again, Ying! Will you be attending LSFMMBPF this year? I would love to say hello in person : -)

Have a great day!
Joshua

Quoted 11 lines
Quoted 11 lines

This proposal also has some concerns that need to be addressed:
- How reactive should the system be, and how aggressively should it tune the weights? We don't want the system to overreact to short spikes in pressure.
- Does dynamic weight adjusting lead to pages being "misplaced"? Should those "misplaced" pages be migrated? (probably not)
- Does this need to be in the kernel? A userspace daemon that monitors kernel metrics has the ability to make the changes (via the nodeN interfaces).

Thoughts & comments are appreciated! Thank you, and have a great day! Joshua

Quoted 7 lines
Quoted 3 lines

[1] https://lore.kernel.org/all/[email protected]/

Sent using hkml (https://github.com/sjp38/hackermail)

---
Best Regards,
Huang, Ying

Sent using hkml (https://github.com/sjp38/hackermail)

#7 lore raw
In reply to Joshua Hahn
Quoted 1 line · Joshua Hahn

Hello everyone, I hope everyone has had a great start to 2025!

Hi Joshua,

as discussed in the LSFMM about how you can react to nodes becoming memory{aware,less}, you can register a hotplug memory notifier, as memory-tiering currently does.

The current use of the hotplug memory notifier by some consumers (e.g:
memory-tiering, slub, etc) is a bit suboptimal, as they only care about nodes changing its memory state, yet they get notified for every {online,offline}_pages operation.

I came up with [1]

I did not publish it yet upstream because I wanted to discuss it a bit with David, but you can give it a try to see if it works for you. But till it is upstream, you will have to use the hotplug memory notifier.

[1] https://github.com/leberus/linux.git numa-node-notifier

Signature · 2 lines
Oscar Salvador
SUSE Labs
#8 lore raw
Quoted 11 lines · Oscar Salvador

Hi Joshua,

as discussed in the LSFMM about how you can react to nodes becoming memory{aware,less}, you can register a hotplug memory notifier, as memory-tiering currently does.

The current use of the hotplug memory notifier by some consumers (e.g:
memory-tiering, slub, etc) is a bit suboptimal, as they only care about nodes changing its memory state, yet they get notified for every {online,offline}_pages operation.

Quoted 9 lines

I came up with [1]

I did not publish it yet upstream because I wanted to discuss it a bit with David, but you can give it a try to see if it works for you. But till it is upstream, you will have to use the hotplug memory notifier.

[1] https://github.com/leberus/linux.git numa-node-notifier

+CC: Yunjeong Mun and Rakie Kim

Something to consider as a follow up to your series.

Thanks Oscar, we were just discussing this. Seems there's multiple users doing the same thing, so it seems reasonable to discuss. This would probably deal with my race condition concerns here as well:

https://lore.kernel.org/linux-mm/[email protected]/

Quoted 3 lines
Signature · 2 lines
Oscar Salvador
SUSE Labs
#9 lore raw
In reply to Oscar Salvador
Quoted 14 lines · On Thu, 27 Mar 2025 12:11:55 +0100 Oscar Salvador <[email protected]> wrote:
Quoted 1 line · Joshua Hahn

Hello everyone, I hope everyone has had a great start to 2025!

Hi Joshua,

as discussed in the LSFMM about how you can react to nodes becoming memory{aware,less}, you can register a hotplug memory notifier, as memory-tiering currently does.

The current use of the hotplug memory notifier by some consumers (e.g:
memory-tiering, slub, etc) is a bit suboptimal, as they only care about nodes changing its memory state, yet they get notified for every {online,offline}_pages operation.

Quoted 6 lines

I came up with [1]

I did not publish it yet upstream because I wanted to discuss it a bit with David, but you can give it a try to see if it works for you. But till it is upstream, you will have to use the hotplug memory notifier.

Hi Oscar,

This is great, thank you for taking a look at this. I'll also do some experimenting on my end and let you know with any results that I see from my end, using weighted interleave.

In the meantime yes -- I think the hotplug memory notifier should do the trick. Thank you again, have a great day!
Joshua

Quoted 5 lines

[1] https://github.com/leberus/linux.git numa-node-notifier

Signature · 2 lines
Oscar Salvador
SUSE Labs

Sent using hkml (https://github.com/sjp38/hackermail)