Skip to contentSkip to conversation

virtio-mem: paravirtualized memory

RFC PATCH Superseded David Hildenbrand 2 participants 65 messages Sep 19, 2019 – Oct 25, 2019

Subsystems (7)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover virtio-mem: paravirtualized memory

Long time no RFC! I finally had time to get the next version of the Linux driver side of virtio-mem into shape, incorporating ideas and feedback from previous discussions.

This RFC is based on the series currently on the mm list:
- [PATCH 0/3] Remove __online_page_set_limits()
- [PATCH v1 0/3] mm/memory_hotplug: Export generic_online_page()
- [PATCH v4 0/8] mm/memory_hotplug: Shrink zones before removing memory The current state (kept updated) is available on
- https://github.com/davidhildenbrand/linux/tree/virtio-mem

The basic idea of virtio-mem is to provide a flexible, cross-architecture memory hot(un)plug solution that avoids many limitations imposed by existing technologies, architectures, and interfaces. More details can be found below and in linked material.

This RFC was only tested on x86-64, however, should theoretically work on any Linux architecture that implements memory hot(un)plug - like s390x. On x86-64, it is currently possible to add/remove memory to the system in >= 4MB granularity. Memory hotplug works very reliable. For memory unplug, there are no guarantees how much memory can actually get unplugged, it depends on the setup. I have plans to improve that in the future.

--------------------------------------------------------------------------
1. virtio-mem
--------------------------------------------------------------------------

The basic idea behind virtio-mem was presented at KVM Forum 2018. The slides can be found at [1]. The previous RFC can be found at [2]. The first RFC can be found at [3]. However, the concept evolved over time. The KVM Forum slides roughly match the current design.

Patch #2 ("virtio-mem: Paravirtualized memory hotplug") contains quite some information, especially in "include/uapi/linux/virtio_mem.h":

Each virtio-mem device manages a dedicated region in physical address space. Each device can belong to a single NUMA node, multiple devices for a single NUMA node are possible. A virtio-mem device is like a "resizable DIMM" consisting of small memory blocks that can be plugged or unplugged. The device driver is responsible for (un)plugging memory blocks on demand.

Virtio-mem devices can only operate on their assigned memory region in order to (un)plug memory. A device cannot (un)plug memory belonging to other devices.

The "region_size" corresponds to the maximum amount of memory that can be provided by a device. The "size" corresponds to the amount of memory that is currently plugged. "requested_size" corresponds to a request from the device to the device driver to (un)plug blocks. The device driver should try to (un)plug blocks in order to reach the "requested_size". It is impossible to plug more memory than requested.

The "usable_region_size" represents the memory region that can actually be used to (un)plug memory. It is always at least as big as the "requested_size" and will grow dynamically. It will only shrink when explicitly triggered (VIRTIO_MEM_REQ_UNPLUG).

Memory in the usable region can usually be read, however, there are no guarantees. It can happen that the device cannot process a request, because it is busy. The device driver has to retry later.

Usually, during system resets all memory will get unplugged, so the device driver can start with a clean state. However, in specific scenarios (if the device is busy) it can happen that the device still has memory plugged. The device driver can request to unplug all memory (VIRTIO_MEM_REQ_UNPLUG) - which might take a while to succeed if the device is busy.

--------------------------------------------------------------------------
2. Linux Implementation
--------------------------------------------------------------------------

This RFC reuses quite some existing MM infrastructure, however, has to expose some additional functionality.

Memory blocks (e.g., 128MB) are added/removed on demand. Within these memory blocks, subblocks (e.g., 4MB) are plugged/unplugged. The sizes depend on the target architecture, MAX_ORDER + pageblock_order, and the block size of a virtio-mem device.

add_memory()/try_remove_memory() is used to add/remove memory blocks. virtio-mem will not online memory blocks itself. This has to be done by user space, or configured into the kernel (CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE). virtio-mem will only unplug memory that was online to the ZONE_NORMAL. Memory is suggested to be onlined to the ZONE_NORMAL for now.

The memory hotplug notifier is used to properly synchronize against onlining/offlining of memory blocks and to track the states of memory blocks (including the zone memory blocks are onlined to). Locking is done similar to the PPC CMA implementation.

The set_online_page() callback is used to keep unplugged subblocks of a memory block fake-offline when onlining the memory block. generic_online_page() is used to fake-online plugged subblocks. This handling is similar to the Hyper-V balloon driver.

PG_offline is used to mark unplugged subblocks as offline, so e.g., dumping tools (makedumpfile) will skip these pages. This is similar to other balloon drivers like virtio-balloon and Hyper-V.

PG_offline + reference count of 0 [new] is now also used to mark pages as a "skip" when offlining memory blocks. This allows to offline memory blocks that have partially unplugged subblocks - or are completely unplugged.

alloc_contig_range()/free_contig_range() [now exposed] is used to unplug/plug subblocks of memory blocks the are already exposed to Linux.

offline_and_remove_memory() [new] is used to offline a fully unplugged memory block and remove it from Linux.

A lot of additional information can be found in the separate patches and as comments in the code itself.

--------------------------------------------------------------------------
3. Major changes since last RFC
--------------------------------------------------------------------------

A lot of things changed, especially also on the QEMU + virtio side. The biggest changes on the Linux driver side are:
- Onlining/offlining of subblocks is now emulated on top of memory blocks. set_online_page()+alloc_contig_range()+free_contig_range() is now used for that. Core MM does not have to be modified and will continue to online/offline full memory blocks.
- Onlining/offlining of memory blocks is no longer performed by virtio-mem.
- Pg_offline is upstream and can be used. It is also used to allow offlining of partially unplugged memory blocks.
- Memory block states + subblocks are now tracked more space-efficient.
- Proper kexec(), kdump(), driver unload, driver reload, ZONE_MOVABLE, ... handling.

--------------------------------------------------------------------------
4. Future work
--------------------------------------------------------------------------

The separate patches contain a lot of future work items. One of the next steps is to make memory unplug more likely to succeed - currently, there are no guarantees on how much memory can get unplugged again. I have various ideas on how to limit fragmentation of all memory blocks that virtio-mem added.

Memory hotplug:
- Reduce the amount of memory resources if that turnes out to be an issue. Or try to speed up relevant code paths to deal with many resources.
- Allocate the vmemmap from the added memory. Makes hotplug more likely to succeed, the vmemmap is stored on the same NUMA node and that unmovable memory will later not hinder unplug.

Memory hotunplug:
- Performance improvements:
-- Sense (lockless) if it make sense to try alloc_contig_range() at all
before directly trying to isolate and taking locks.
-- Try to unplug bigger chunks if possible first.
-- Identify free areas first, that don't have to be evacuated.
- Make unplug more likely to succeed:
-- The "issue" is that in the ZONE_NORMAL, the buddy will randomly
allocate memory. Only pageblocks somewhat limit fragmentation, however we would want to limit fragmentation on subblock granularity and even memory block granularity. One idea is to have a new ZONE_PREFER_MOVABLE. Memory blocks will then be onlined to ZONE_NORMAL / ZONE_PREFER_MOVABLE in a certain ratio per node (e.g., 1:4). This makes unplug of quite some memory likely to succeed in most setups. ZONE_PREFER_MOVABLE is then a mixture of ZONE_NORMAL and ZONE_MOVABlE. Especially, movable data can end up on that zone, but only if really required - avoiding running out of memory on ZONE imbalances. The zone fallback order would be MOVABLE=>PREFER_MOVABLE=>HIGHMEM=>NORMAL=>PREFER_MOVABLE=>DMA32=>DMA
-- Allocate memmap from added memory. This way, less unmovable data can
end up on the memory blocks.
-- Call drop_slab() before trying to unplug. Eventually shrink other
caches.
- Better retry handling in case memory is busy. We certainly don't want to try for ever in a short interval to try to get some memory back.
- OOM handling, e.g., via an OOM handler.

--------------------------------------------------------------------------
5. Example Usage
--------------------------------------------------------------------------

A very basic QEMU prototype (kept updated) is available at:
- https://github.com/davidhildenbrand/qemu/tree/virtio-mem

It lacks various features, however works to test the guest driver side:
- No support for resizable memory regions / memory backends
- No protection of unplugged memory (esp., userfaultfd-wp)
- No dump/migration/XXX optimizations to skip over unplugged memory

Start QEMU with two virtio-mem devices (one per NUMA node):

 $ qemu-system-x86_64 -m 4G,maxmem=20G \
  -smp sockets=2,cores=2 \
  -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \
  [...]
  -object memory-backend-ram,id=mem0,size=8G \
  -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=128M \
  -object memory-backend-ram,id=mem1,size=8G \
  -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=80M

Query the configuration:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) info memory-devices
Memory device [virtio-mem]: "vm0"
memaddr: 0x140000000
node: 0
requested-size: 134217728
size: 134217728
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem0
Memory device [virtio-mem]: "vm1"
memaddr: 0x340000000
node: 1
requested-size: 83886080
size: 83886080
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem1

Add some memory to node 1:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) qom-set vm1 requested-size 1G

Remove some memory from node 0:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) qom-set vm0 requested-size 64M

Query the configuration:
(qemu) info memory-devices
Memory device [virtio-mem]: "vm0"
memaddr: 0x140000000
node: 0
requested-size: 67108864
size: 67108864
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem0
Memory device [virtio-mem]: "vm1"
memaddr: 0x340000000
node: 1
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem1

--------------------------------------------------------------------------
6. Q/A
--------------------------------------------------------------------------

Q: Why add/remove parts ("subblocks") of memory blocks/sections?
A: Flexibility (section size depends on the architecture) - e.g., some
architectures have a section size of 2GB. Also, the memory block size is variable (e.g., on x86-64). I want to avoid any such restrictions. Some use cases want to add/remove memory in smaller granularities to a VM (e.g., the Hyper-V balloon also implements this) - especially smaller VMs like used for kata containers. Also, on memory unplug, it is more reliable to free-up and unplug multiple small chunks instead of one big chunk. E.g., if one page of a DIMM is either unmovable or pinned, the DIMM can't get unplugged. This approach is basically a compromise between DIMM-based memory hot(un)plug and balloon inflation/deflation, which works mostly on page granularity.

Q: Why care about memory blocks?
A: They are the way to tell user space about new memory. This way,
memory can get onlined/offlined by user space. Also, e.g., kdump relies on udev events to reload kexec when memory blocks are onlined/offlined. Memory blocks are the "real" memory hot(un)plug granularity. Everything that's smaller has to be emulated "on top".

Q: Won't memory unplug of subblocks fragment memory?
A: Yes and no. Unplugging e.g., >=4MB subblocks on x86-64 will not really
fragment memory like unplugging random pages like a balloon driver does. Buddy merging will not be limited. However, any allocation that requires bigger consecutive memory chunks (e.g., gigantic pages) might observe the fragmentation. Possible solutions: Allocate gigantic huge pages before unplugging memory, don't unplug memory, combine virtio-mem with DIMM based memory or bigger initial memory. Remember, a virtio-mem device will only unplug on the memory range it manages, not on other DIMMs. Unplug of single memory blocks will result in similar fragmentation in respect to gigantic huge pages.

Q: How reliable is memory unplug?
A: There are no guarantees on how much memory can get unplugged
again. However, it is more likely to find 4MB chunks to unplug than e.g., 128MB chunks. If memory is terribly fragmented, there is nothing we can do - for now. I consider memory hotplug the first primary use of virtio-mem. Memory unplug might usually work, but we want to improve the performance and the amount of memory we can actually unplug later.

Q: Why not unplug from the ZONE_MOVABLE?
A: Unplugged memory chunks are unmovable. Unmovable data must not end up
on the ZONE_MOVABLE - similar to gigantic pages - they will never be allocated from ZONE_MOVABLE. Teaching MM to move unplugged chunks within a device might be problematic and will require a new guest->hypervisor command to move unplugged chunks. virtio-mem added memory can be onlined to the ZONE_MOVABLE, but subblocks will not get unplugged from it.

Q: How big should the initial (!virtio-mem) memory of a VM be?
A: virtio-mem memory will not go to the DMA zones. So to avoid running out
of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case.

[1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red-Hat-1.pdf
[2] https://lwn.net/Articles/755423/
[3] https://lists.gnu.org/archive/html/qemu-devel/2017-06/msg03870.html

---

David Hildenbrand (9):
ACPI: NUMA: export pxm_to_node
virtio-mem: Paravirtualized memory hotplug
virtio-mem: Paravirtualized memory hotunplug part 1
mm: Export alloc_contig_range() / free_contig_range()
virtio-mem: Paravirtualized memory hotunplug part 2
mm: Allow to offline PageOffline() pages with a reference count of 0
virtio-mem: Allow to offline partially unplugged memory blocks mm/memory_hotplug: Introduce offline_and_remove_memory()
virtio-mem: Offline and remove completely unplugged memory blocks

 drivers/acpi/numa.c             |    1 +
 drivers/virtio/Kconfig          |   18 +
 drivers/virtio/Makefile         |    1 +
 drivers/virtio/virtio_mem.c     | 1900 +++++++++++++++++++++++++++++++
 include/linux/memory_hotplug.h  |    1 +
 include/linux/page-flags.h      |    4 +
 include/linux/page-isolation.h  |    4 +-
 include/uapi/linux/virtio_ids.h |    1 +
 include/uapi/linux/virtio_mem.h |  204 ++++
 mm/memory_hotplug.c             |   44 +-
 mm/page_alloc.c                 |   24 +-
 mm/page_isolation.c             |   18 +-
 mm/swap.c                       |    9 +
 13 files changed, 2219 insertions(+), 10 deletions(-)
 create mode 100644 drivers/virtio/virtio_mem.c
 create mode 100644 include/uapi/linux/virtio_mem.h
Signature · 1 line
2.21.0
#1 lore raw

RFC PATCH v3 1/9 ACPI: NUMA: export pxm_to_node

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
drivers/acpi/numa.c +1 -0
index eadbf90e65d1..d5847fa7ac69 100644
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);
 
 int node_to_pxm(int node)
 {
Signature · 1 line
2.21.0
#2 lore raw

RFC PATCH v3 1/9 ACPI: NUMA: export pxm_to_node

In reply to David Hildenbrand

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
drivers/acpi/numa.c +1 -0
index eadbf90e65d1..d5847fa7ac69 100644
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);
 
 int node_to_pxm(int node)
 {
Signature · 1 line
2.21.0
#3 lore raw

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

Quoted 15 lines · David Hildenbrand

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
drivers/acpi/numa.c +0 -0
index eadbf90e65d1..d5847fa7ac69 100644

drivers/acpi/numa.c int pxm_to_node(int pxm) · line 38

Quoted 4 lines
drivers/acpi/numa.c +1 -0
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);

FWIW, this is a fairly old patch I dragged along and I think I'll convert this to EXPORT_SYMBOL_GPL now that I know better :)

Quoted 4 lines
drivers/acpi/numa.c +0 -0
 int node_to_pxm(int node)
 {
 
Signature · 3 lines
Thanks,

David / dhildenb
#4 lore raw

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

In reply to David Hildenbrand
Quoted 15 lines · David Hildenbrand

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
drivers/acpi/numa.c +0 -0
index eadbf90e65d1..d5847fa7ac69 100644

drivers/acpi/numa.c int pxm_to_node(int pxm) · line 38

Quoted 4 lines
drivers/acpi/numa.c +1 -0
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);

FWIW, this is a fairly old patch I dragged along and I think I'll convert this to EXPORT_SYMBOL_GPL now that I know better :)

Quoted 4 lines
drivers/acpi/numa.c +0 -0
 int node_to_pxm(int node)
 {
 
Signature · 3 lines
Thanks,

David / dhildenb
#5 lore raw

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

Quoted 11 lines · David Hildenbrand
Quoted 10 lines · David Hildenbrand

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 12 lines

drivers/acpi/numa.c int pxm_to_node(int pxm) · line 38

Quoted 9 lines
drivers/acpi/numa.c +1 -0
index eadbf90e65d1..d5847fa7ac69 100644
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);

FWIW, this is a fairly old patch I dragged along and I think I'll convert this to EXPORT_SYMBOL_GPL now that I know better :)

All other exports in this file are EXPORT_SYMBOL. Why is this one considered special to restrict the access?

Signature · 2 lines
Michal Hocko
SUSE Labs
#6 lore raw

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

Quoted 25 lines · Michal Hocko
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · David Hildenbrand

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 12 lines

drivers/acpi/numa.c int pxm_to_node(int pxm) · line 38

Quoted 9 lines
drivers/acpi/numa.c +1 -0
index eadbf90e65d1..d5847fa7ac69 100644
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);

FWIW, this is a fairly old patch I dragged along and I think I'll convert this to EXPORT_SYMBOL_GPL now that I know better :)

Quoted 3 lines

All other exports in this file are EXPORT_SYMBOL. Why is this one considered special to restrict the access?

Uh, so I did actually use my brain two years ago back when I was crafting this patch :)

Signature · 3 lines
Thanks,

David / dhildenb
#7 lore raw

Re: [PATCH RFC v3 1/9] ACPI: NUMA: export pxm_to_node

In reply to Michal Hocko
Quoted 25 lines · Michal Hocko
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · David Hildenbrand

Will be needed by virtio-mem to identify the node from a pxm.

---
 drivers/acpi/numa.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 12 lines

drivers/acpi/numa.c int pxm_to_node(int pxm) · line 38

Quoted 9 lines
drivers/acpi/numa.c +1 -0
index eadbf90e65d1..d5847fa7ac69 100644
@@ -35,6 +35,7 @@ int pxm_to_node(int pxm)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
+EXPORT_SYMBOL(pxm_to_node);

FWIW, this is a fairly old patch I dragged along and I think I'll convert this to EXPORT_SYMBOL_GPL now that I know better :)

Quoted 3 lines

All other exports in this file are EXPORT_SYMBOL. Why is this one considered special to restrict the access?

Uh, so I did actually use my brain two years ago back when I was crafting this patch :)

Signature · 3 lines
Thanks,

David / dhildenb
#8 lore raw

RFC PATCH v3 2/9 virtio-mem: Paravirtualized memory hotplug

In reply to David Hildenbrand

Each virtio-mem device owns exactly one memory region. It is responsible for adding/removing memory from that memory region on request.

When the device driver starts up, the requested amount of memory is queried and then plugged to Linux. On request, further memory can be plugged or unplugged. This patch only implements the plugging part.

On x86-64, memory can currently be plugged in 4MB ("subblock") granularity. When required, a new memory block will be added (e.g., usually 128MB on x86-64) in order to plug more subblocks. Only x86-64 was tested for now.

The online_page callback is used to keep unplugged subblocks offline when onlining memory - similar to the Hyper-V balloon driver. Unplugged pages are marked PG_offline, to tell dump tools (e.g., makedumpfile) to skip them.

User space is usually responsible for onlining the added memory. The memory hotplug notifier is used to synchronize virtio-mem activity against memory onlining/offlining.

Each virtio-mem device can belong to a NUMA node, which allows us to easily add/remove small chunks of memory to/from a specific NUMA node by using multiple virtio-mem devices. Something that works even when the guest has no idea about the NUMA topology.

One way to view virtio-mem is as a "resizable DIMM" or a DIMM with many "sub-DIMMS".

This patch directly introduces the basic infrastructure to implement memory unplug. Especially the memory block states and subblock bitmaps will be heavily used there.

Notes:
- Memory blocks that are partally unplugged must not be onlined to the MOVABLE_ZONE. They contain unmovable parts and might e.g., result in warnings when trying to offline them.
- In the kdump kernel, we don't want to load the driver, to not mess with memory to be dumped.
- In case memory is to be onlined by user space, we limit the amount of offline memory blocks, to not run out of memory.
- Suspend/Hibernate is not supported due to the way virtio-mem devices behave. Limited support might be possible in the future.
- Reloading the device driver is not supported.
- When unloading the driver, we have to remove partially plugged offline memory blocks. Otherwise they could get fully onlined later on, when we no longer have control over via the online_page callback.
- As the hypervisor might suddenly be busy (during reboot, in-between requests, when adding of memory fails), we have to take care of some corner cases - especially virtio_mem_unplug_pending_mb() and virtio_mem_send_unplug_all_request(). The hypervisor could for example be busy if it is currently migrating the guest.

Future work:
- Reduce the amount of memory resources if that turnes out to be an issue. Or try to speed up relevant code paths to deal with many resources.
- Allocate the vmemmap from the added memory. Makes hotplug more likely to succeed, the vmemmap is stored on the same NUMA node and that unmovable memory will later not hinder unplug.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/Kconfig          |   17 +
 drivers/virtio/Makefile         |    1 +
 drivers/virtio/virtio_mem.c     | 1580 +++++++++++++++++++++++++++++++
 include/uapi/linux/virtio_ids.h |    1 +
 include/uapi/linux/virtio_mem.h |  204 ++++
 5 files changed, 1803 insertions(+)
 create mode 100644 drivers/virtio/virtio_mem.c
 create mode 100644 include/uapi/linux/virtio_mem.h
drivers/virtio/Kconfig +17 -0
index 078615cf2afc..294720d53057 100644
@@ -64,6 +64,23 @@ config VIRTIO_BALLOON
 
 	 If unsure, say M.
 
+config VIRTIO_MEM
+	tristate "Virtio mem driver"
+	default m
+	depends on X86_64
+	depends on VIRTIO
+	depends on MEMORY_HOTPLUG_SPARSE
+	depends on MEMORY_HOTREMOVE
+	help
+	 This driver provides access to virtio-mem paravirtualized memory
+	 devices, allowing to hotplug and hotunplug memory.
+
+	 This driver is was only tested under x86-64, but should
+	 theoretically work on all architectures that support memory
+	 hotplug and hotremove.
+
+	 If unsure, say M.
+
 config VIRTIO_INPUT
 	tristate "Virtio input driver"
 	depends on VIRTIO
drivers/virtio/Makefile +1 -0
index 3a2b5c5dcf46..906d5a00ac85 100644
@@ -6,3 +6,4 @@ virtio_pci-y := virtio_pci_modern.o virtio_pci_common.o
 virtio_pci-$(CONFIG_VIRTIO_PCI_LEGACY) += virtio_pci_legacy.o
 obj-$(CONFIG_VIRTIO_BALLOON) += virtio_balloon.o
 obj-$(CONFIG_VIRTIO_INPUT) += virtio_input.o
+obj-$(CONFIG_VIRTIO_MEM) += virtio_mem.o
drivers/virtio/virtio_mem.c new file +1580 -0

1,580 lines not on this page · raw message

include/uapi/linux/virtio_ids.h +1 -0
index 585e07b27333..934e0b444454 100644
@@ -44,6 +44,7 @@
 #define VIRTIO_ID_VSOCK        19 /* virtio vsock transport */
 #define VIRTIO_ID_CRYPTO       20 /* virtio crypto */
 #define VIRTIO_ID_IOMMU        23 /* virtio IOMMU */
+#define VIRTIO_ID_MEM          24 /* virtio mem */
 #define VIRTIO_ID_FS           26 /* virtio filesystem */
 #define VIRTIO_ID_PMEM         27 /* virtio pmem */
 
include/uapi/linux/virtio_mem.h new file +204 -0

204 lines not on this page · raw message

Signature · 1 line
2.21.0
#9 lore raw

RFC PATCH v3 2/9 virtio-mem: Paravirtualized memory hotplug

In reply to David Hildenbrand

Each virtio-mem device owns exactly one memory region. It is responsible for adding/removing memory from that memory region on request.

When the device driver starts up, the requested amount of memory is queried and then plugged to Linux. On request, further memory can be plugged or unplugged. This patch only implements the plugging part.

On x86-64, memory can currently be plugged in 4MB ("subblock") granularity. When required, a new memory block will be added (e.g., usually 128MB on x86-64) in order to plug more subblocks. Only x86-64 was tested for now.

The online_page callback is used to keep unplugged subblocks offline when onlining memory - similar to the Hyper-V balloon driver. Unplugged pages are marked PG_offline, to tell dump tools (e.g., makedumpfile) to skip them.

User space is usually responsible for onlining the added memory. The memory hotplug notifier is used to synchronize virtio-mem activity against memory onlining/offlining.

Each virtio-mem device can belong to a NUMA node, which allows us to easily add/remove small chunks of memory to/from a specific NUMA node by using multiple virtio-mem devices. Something that works even when the guest has no idea about the NUMA topology.

One way to view virtio-mem is as a "resizable DIMM" or a DIMM with many "sub-DIMMS".

This patch directly introduces the basic infrastructure to implement memory unplug. Especially the memory block states and subblock bitmaps will be heavily used there.

Notes:
- Memory blocks that are partally unplugged must not be onlined to the MOVABLE_ZONE. They contain unmovable parts and might e.g., result in warnings when trying to offline them.
- In the kdump kernel, we don't want to load the driver, to not mess with memory to be dumped.
- In case memory is to be onlined by user space, we limit the amount of offline memory blocks, to not run out of memory.
- Suspend/Hibernate is not supported due to the way virtio-mem devices behave. Limited support might be possible in the future.
- Reloading the device driver is not supported.
- When unloading the driver, we have to remove partially plugged offline memory blocks. Otherwise they could get fully onlined later on, when we no longer have control over via the online_page callback.
- As the hypervisor might suddenly be busy (during reboot, in-between requests, when adding of memory fails), we have to take care of some corner cases - especially virtio_mem_unplug_pending_mb() and virtio_mem_send_unplug_all_request(). The hypervisor could for example be busy if it is currently migrating the guest.

Future work:
- Reduce the amount of memory resources if that turnes out to be an issue. Or try to speed up relevant code paths to deal with many resources.
- Allocate the vmemmap from the added memory. Makes hotplug more likely to succeed, the vmemmap is stored on the same NUMA node and that unmovable memory will later not hinder unplug.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/Kconfig          |   17 +
 drivers/virtio/Makefile         |    1 +
 drivers/virtio/virtio_mem.c     | 1580 +++++++++++++++++++++++++++++++
 include/uapi/linux/virtio_ids.h |    1 +
 include/uapi/linux/virtio_mem.h |  204 ++++
 5 files changed, 1803 insertions(+)
 create mode 100644 drivers/virtio/virtio_mem.c
 create mode 100644 include/uapi/linux/virtio_mem.h
drivers/virtio/Kconfig +17 -0
index 078615cf2afc..294720d53057 100644
@@ -64,6 +64,23 @@ config VIRTIO_BALLOON
 
 	 If unsure, say M.
 
+config VIRTIO_MEM
+	tristate "Virtio mem driver"
+	default m
+	depends on X86_64
+	depends on VIRTIO
+	depends on MEMORY_HOTPLUG_SPARSE
+	depends on MEMORY_HOTREMOVE
+	help
+	 This driver provides access to virtio-mem paravirtualized memory
+	 devices, allowing to hotplug and hotunplug memory.
+
+	 This driver is was only tested under x86-64, but should
+	 theoretically work on all architectures that support memory
+	 hotplug and hotremove.
+
+	 If unsure, say M.
+
 config VIRTIO_INPUT
 	tristate "Virtio input driver"
 	depends on VIRTIO
drivers/virtio/Makefile +1 -0
index 3a2b5c5dcf46..906d5a00ac85 100644
@@ -6,3 +6,4 @@ virtio_pci-y := virtio_pci_modern.o virtio_pci_common.o
 virtio_pci-$(CONFIG_VIRTIO_PCI_LEGACY) += virtio_pci_legacy.o
 obj-$(CONFIG_VIRTIO_BALLOON) += virtio_balloon.o
 obj-$(CONFIG_VIRTIO_INPUT) += virtio_input.o
+obj-$(CONFIG_VIRTIO_MEM) += virtio_mem.o
drivers/virtio/virtio_mem.c new file +1580 -0

1,580 lines not on this page · raw message

include/uapi/linux/virtio_ids.h +1 -0
index 585e07b27333..934e0b444454 100644
@@ -44,6 +44,7 @@
 #define VIRTIO_ID_VSOCK        19 /* virtio vsock transport */
 #define VIRTIO_ID_CRYPTO       20 /* virtio crypto */
 #define VIRTIO_ID_IOMMU        23 /* virtio IOMMU */
+#define VIRTIO_ID_MEM          24 /* virtio mem */
 #define VIRTIO_ID_FS           26 /* virtio filesystem */
 #define VIRTIO_ID_PMEM         27 /* virtio pmem */
 
include/uapi/linux/virtio_mem.h new file +204 -0

204 lines not on this page · raw message

Signature · 1 line
2.21.0
#10 lore raw

RFC PATCH v3 3/9 virtio-mem: Paravirtualized memory hotunplug part 1

In reply to David Hildenbrand

Unplugging subblocks of memory blocks that are offline is easy. All we have to do is watch out for concurrent onlining acticity.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/virtio_mem.c | 109 +++++++++++++++++++++++++++++++++++-
 1 file changed, 107 insertions(+), 2 deletions(-)
drivers/virtio/virtio_mem.c +107 -2
index 597fab4933f6..6fb55d4b6f6c 100644
@@ -119,7 +119,7 @@ struct virtio_mem {
 	 *
 	 * When this lock is held the pointers can't change, ONLINE and
 	 * OFFLINE blocks can't change the state and no subblocks will get
-	 * plugged.
+	 * plugged/unplugged.
 	 */
 	struct mutex hotplug_mutex;
 	bool hotplug_active;
@@ -322,6 +322,19 @@ static bool virtio_mem_mb_test_sb_plugged(struct virtio_mem *vm,
 	       bit + count;
 }
 
+/*
+ * Test if all selected subblocks are unplugged.
+ */
+static bool virtio_mem_mb_test_sb_unplugged(struct virtio_mem *vm,
+					    unsigned long mb_id, int sb_id,
+					    int count)
+{
+	const int bit = (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb + sb_id;
+
+	/* TODO: Helper similar to bitmap_set() */
+	return find_next_bit(vm->sb_bitmap, bit + count, bit) >= bit + count;
+}
+
 /*
  * Find the first plugged subblock. Returns vm->nb_sb_per_mb in case there is
  * none.
@@ -512,6 +525,9 @@ static void virtio_mem_notify_offline(struct virtio_mem *vm,
 		BUG();
 		break;
 	}
+
+	/* trigger the workqueue, maybe we can now unplug memory. */
+	virtio_mem_retry(vm);
 }
 
 static void virtio_mem_notify_online(struct virtio_mem *vm, unsigned long mb_id,
@@ -1129,6 +1145,93 @@ static int virtio_mem_plug_request(struct virtio_mem *vm, uint64_t diff)
 	return rc;
 }
 
+/*
+ * Unplug the desired number of plugged subblocks of an offline memory block.
+ * Will fail if any subblock cannot get unplugged (instead of skipping it).
+ *
+ * Will modify the state of the memory block. Might temporarily drop the
+ * hotplug_mutex.
+ *
+ * Note: Can fail after some subblocks were successfully unplugged.
+ */
+static int virtio_mem_mb_unplug_any_sb_offline(struct virtio_mem *vm,
+					       unsigned long mb_id,
+					       uint64_t *nb_sb)
+{
+	int rc;
+
+	rc = virtio_mem_mb_unplug_any_sb(vm, mb_id, nb_sb);
+
+	/* some subblocks might have been unplugged even on failure */
+	if (!virtio_mem_mb_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb))
+		virtio_mem_mb_set_state(vm, mb_id,
+					VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL);
+	if (rc)
+		return rc;
+
+	if (virtio_mem_mb_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) {
+		/*
+		 * Remove the block from Linux - this should never fail.
+		 * Hinder the block from getting onlined by marking it
+		 * unplugged. Temporarily drop the mutex, so
+		 * any pending GOING_ONLINE requests can be serviced/rejected.
+		 */
+		virtio_mem_mb_set_state(vm, mb_id,
+					VIRTIO_MEM_MB_STATE_UNUSED);
+
+		mutex_unlock(&vm->hotplug_mutex);
+		rc = virtio_mem_mb_remove(vm, mb_id);
+		BUG_ON(rc);
+		mutex_lock(&vm->hotplug_mutex);
+	}
+	return 0;
+}
+
+/*
+ * Try to unplug the requested amount of memory.
+ */
+static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff)
+{
+	uint64_t nb_sb = diff / vm->subblock_size;
+	unsigned long mb_id;
+	int rc;
+
+	if (!nb_sb)
+		return 0;
+
+	/*
+	 * We'll drop the mutex a couple of times when it is safe to do so.
+	 * This might result in some blocks switching the state (online/offline)
+	 * and we could miss them in this run - we will retry again later.
+	 */
+	mutex_lock(&vm->hotplug_mutex);
+
+	/* Try to unplug subblocks of partially plugged offline blocks. */
+	virtio_mem_for_each_mb_state(vm, mb_id,
+				     VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL) {
+		rc = virtio_mem_mb_unplug_any_sb_offline(vm, mb_id,
+							 &nb_sb);
+		if (rc || !nb_sb)
+			goto out_unlock;
+		cond_resched();
+	}
+
+	/* Try to unplug subblocks of plugged offline blocks. */
+	virtio_mem_for_each_mb_state(vm, mb_id, VIRTIO_MEM_MB_STATE_OFFLINE) {
+		rc = virtio_mem_mb_unplug_any_sb_offline(vm, mb_id,
+							 &nb_sb);
+		if (rc || !nb_sb)
+			goto out_unlock;
+		cond_resched();
+	}
+
+	mutex_unlock(&vm->hotplug_mutex);
+	return 0;
+out_unlock:
+	mutex_unlock(&vm->hotplug_mutex);
+	return rc;
+}
+
 /*
  * Try to unplug all blocks that couldn't be unplugged before, for example,
  * because the hypervisor was busy.
@@ -1209,8 +1312,10 @@ static void virtio_mem_run_wq(struct work_struct *work)
 		if (vm->requested_size > vm->plugged_size) {
 			diff = vm->requested_size - vm->plugged_size;
 			rc = virtio_mem_plug_request(vm, diff);
+		} else {
+			diff = vm->plugged_size - vm->requested_size;
+			rc = virtio_mem_unplug_request(vm, diff);
 		}
-		/* TODO: try to unplug memory */
 	}
 
 	switch (rc) {
Signature · 1 line
2.21.0
#11 lore raw

RFC PATCH v3 3/9 virtio-mem: Paravirtualized memory hotunplug part 1

In reply to David Hildenbrand

Unplugging subblocks of memory blocks that are offline is easy. All we have to do is watch out for concurrent onlining acticity.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/virtio_mem.c | 109 +++++++++++++++++++++++++++++++++++-
 1 file changed, 107 insertions(+), 2 deletions(-)
drivers/virtio/virtio_mem.c +107 -2
index 597fab4933f6..6fb55d4b6f6c 100644
@@ -119,7 +119,7 @@ struct virtio_mem {
 	 *
 	 * When this lock is held the pointers can't change, ONLINE and
 	 * OFFLINE blocks can't change the state and no subblocks will get
-	 * plugged.
+	 * plugged/unplugged.
 	 */
 	struct mutex hotplug_mutex;
 	bool hotplug_active;
@@ -322,6 +322,19 @@ static bool virtio_mem_mb_test_sb_plugged(struct virtio_mem *vm,
 	       bit + count;
 }
 
+/*
+ * Test if all selected subblocks are unplugged.
+ */
+static bool virtio_mem_mb_test_sb_unplugged(struct virtio_mem *vm,
+					    unsigned long mb_id, int sb_id,
+					    int count)
+{
+	const int bit = (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb + sb_id;
+
+	/* TODO: Helper similar to bitmap_set() */
+	return find_next_bit(vm->sb_bitmap, bit + count, bit) >= bit + count;
+}
+
 /*
  * Find the first plugged subblock. Returns vm->nb_sb_per_mb in case there is
  * none.
@@ -512,6 +525,9 @@ static void virtio_mem_notify_offline(struct virtio_mem *vm,
 		BUG();
 		break;
 	}
+
+	/* trigger the workqueue, maybe we can now unplug memory. */
+	virtio_mem_retry(vm);
 }
 
 static void virtio_mem_notify_online(struct virtio_mem *vm, unsigned long mb_id,
@@ -1129,6 +1145,93 @@ static int virtio_mem_plug_request(struct virtio_mem *vm, uint64_t diff)
 	return rc;
 }
 
+/*
+ * Unplug the desired number of plugged subblocks of an offline memory block.
+ * Will fail if any subblock cannot get unplugged (instead of skipping it).
+ *
+ * Will modify the state of the memory block. Might temporarily drop the
+ * hotplug_mutex.
+ *
+ * Note: Can fail after some subblocks were successfully unplugged.
+ */
+static int virtio_mem_mb_unplug_any_sb_offline(struct virtio_mem *vm,
+					       unsigned long mb_id,
+					       uint64_t *nb_sb)
+{
+	int rc;
+
+	rc = virtio_mem_mb_unplug_any_sb(vm, mb_id, nb_sb);
+
+	/* some subblocks might have been unplugged even on failure */
+	if (!virtio_mem_mb_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb))
+		virtio_mem_mb_set_state(vm, mb_id,
+					VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL);
+	if (rc)
+		return rc;
+
+	if (virtio_mem_mb_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) {
+		/*
+		 * Remove the block from Linux - this should never fail.
+		 * Hinder the block from getting onlined by marking it
+		 * unplugged. Temporarily drop the mutex, so
+		 * any pending GOING_ONLINE requests can be serviced/rejected.
+		 */
+		virtio_mem_mb_set_state(vm, mb_id,
+					VIRTIO_MEM_MB_STATE_UNUSED);
+
+		mutex_unlock(&vm->hotplug_mutex);
+		rc = virtio_mem_mb_remove(vm, mb_id);
+		BUG_ON(rc);
+		mutex_lock(&vm->hotplug_mutex);
+	}
+	return 0;
+}
+
+/*
+ * Try to unplug the requested amount of memory.
+ */
+static int virtio_mem_unplug_request(struct virtio_mem *vm, uint64_t diff)
+{
+	uint64_t nb_sb = diff / vm->subblock_size;
+	unsigned long mb_id;
+	int rc;
+
+	if (!nb_sb)
+		return 0;
+
+	/*
+	 * We'll drop the mutex a couple of times when it is safe to do so.
+	 * This might result in some blocks switching the state (online/offline)
+	 * and we could miss them in this run - we will retry again later.
+	 */
+	mutex_lock(&vm->hotplug_mutex);
+
+	/* Try to unplug subblocks of partially plugged offline blocks. */
+	virtio_mem_for_each_mb_state(vm, mb_id,
+				     VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL) {
+		rc = virtio_mem_mb_unplug_any_sb_offline(vm, mb_id,
+							 &nb_sb);
+		if (rc || !nb_sb)
+			goto out_unlock;
+		cond_resched();
+	}
+
+	/* Try to unplug subblocks of plugged offline blocks. */
+	virtio_mem_for_each_mb_state(vm, mb_id, VIRTIO_MEM_MB_STATE_OFFLINE) {
+		rc = virtio_mem_mb_unplug_any_sb_offline(vm, mb_id,
+							 &nb_sb);
+		if (rc || !nb_sb)
+			goto out_unlock;
+		cond_resched();
+	}
+
+	mutex_unlock(&vm->hotplug_mutex);
+	return 0;
+out_unlock:
+	mutex_unlock(&vm->hotplug_mutex);
+	return rc;
+}
+
 /*
  * Try to unplug all blocks that couldn't be unplugged before, for example,
  * because the hypervisor was busy.
@@ -1209,8 +1312,10 @@ static void virtio_mem_run_wq(struct work_struct *work)
 		if (vm->requested_size > vm->plugged_size) {
 			diff = vm->requested_size - vm->plugged_size;
 			rc = virtio_mem_plug_request(vm, diff);
+		} else {
+			diff = vm->plugged_size - vm->requested_size;
+			rc = virtio_mem_unplug_request(vm, diff);
 		}
-		/* TODO: try to unplug memory */
 	}
 
 	switch (rc) {
Signature · 1 line
2.21.0
#12 lore raw

RFC PATCH v3 4/9 mm: Export alloc_contig_range() / free_contig_range()

In reply to David Hildenbrand

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task.

In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to alloc_contig_range()+unplug them on unplug request, or plug+free_contig_range() unplugged chunks on plug requests.

A virtio-mem device will use alloc_contig_range() / free_contig_range() only on ranges that belong to the same node/zone in at least MAX(MAX_ORDER - 1, pageblock_order) order granularity - e.g., 4MB on x86-64. The virtio-mem device added that memory, so the memory exists and does not contain any holes. virtio-mem will only try to allocate on ZONE_NORMAL, never on ZONE_MOVABLE, just like when allocating gigantic pages (we don't put unmovable data into the movable zone).

Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Potapenko <[email protected]>
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)
mm/page_alloc.c +2 -0
index 3334a769eb91..d5d7944954b3 100644
@@ -8469,6 +8469,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
 				pfn_max_align_up(end), migratetype);
 	return ret;
 }
+EXPORT_SYMBOL_GPL(alloc_contig_range);
 #endif /* CONFIG_CONTIG_ALLOC */
 
 void free_contig_range(unsigned long pfn, unsigned int nr_pages)
@@ -8483,6 +8484,7 @@ void free_contig_range(unsigned long pfn, unsigned int nr_pages)
 	}
 	WARN(count != 0, "%d pages are still in use!\n", count);
 }
+EXPORT_SYMBOL_GPL(free_contig_range);
 
 #ifdef CONFIG_MEMORY_HOTPLUG
 /*
Signature · 1 line
2.21.0
#13 lore raw

RFC PATCH v3 4/9 mm: Export alloc_contig_range() / free_contig_range()

In reply to David Hildenbrand

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task.

In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to alloc_contig_range()+unplug them on unplug request, or plug+free_contig_range() unplugged chunks on plug requests.

A virtio-mem device will use alloc_contig_range() / free_contig_range() only on ranges that belong to the same node/zone in at least MAX(MAX_ORDER - 1, pageblock_order) order granularity - e.g., 4MB on x86-64. The virtio-mem device added that memory, so the memory exists and does not contain any holes. virtio-mem will only try to allocate on ZONE_NORMAL, never on ZONE_MOVABLE, just like when allocating gigantic pages (we don't put unmovable data into the movable zone).

Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Potapenko <[email protected]>
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)
mm/page_alloc.c +2 -0
index 3334a769eb91..d5d7944954b3 100644
@@ -8469,6 +8469,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
 				pfn_max_align_up(end), migratetype);
 	return ret;
 }
+EXPORT_SYMBOL_GPL(alloc_contig_range);
 #endif /* CONFIG_CONTIG_ALLOC */
 
 void free_contig_range(unsigned long pfn, unsigned int nr_pages)
@@ -8483,6 +8484,7 @@ void free_contig_range(unsigned long pfn, unsigned int nr_pages)
 	}
 	WARN(count != 0, "%d pages are still in use!\n", count);
 }
+EXPORT_SYMBOL_GPL(free_contig_range);
 
 #ifdef CONFIG_MEMORY_HOTPLUG
 /*
Signature · 1 line
2.21.0
Acked-by on v3 4/9
#14 lore raw

Re: [PATCH RFC v3 4/9] mm: Export alloc_contig_range() / free_contig_range()

Quoted 12 lines · David Hildenbrand

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task.

In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to alloc_contig_range()+unplug them on unplug request, or plug+free_contig_range() unplugged chunks on plug requests.

Quoted 7 lines

A virtio-mem device will use alloc_contig_range() / free_contig_range() only on ranges that belong to the same node/zone in at least MAX(MAX_ORDER - 1, pageblock_order) order granularity - e.g., 4MB on x86-64. The virtio-mem device added that memory, so the memory exists and does not contain any holes. virtio-mem will only try to allocate on ZONE_NORMAL, never on ZONE_MOVABLE, just like when allocating gigantic pages (we don't put unmovable data into the movable zone).

Is there any real reason to export as GPL rather than generic EXPORT_SYMBOL? In other words do we need to restrict the usage this interface only to GPL modules and why if so. All other allocator APIs are EXPORT_SYMBOL so there should better be a good reason for this one to differ. I can understand that this one is slightly different by requesting a specific range of the memory but it is still under a full control of the core MM to say no.

Quoted 11 lines
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Potapenko <[email protected]>

Other than that, I do not think exporting this function is harmful. It would be worse to reinvent it and do it wrong.

I usually prefer to add a caller in the same patch, though, because it makes the usage explicit and clear.

Quoted 26 lines
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)
mm/page_alloc.c +2 -0
index 3334a769eb91..d5d7944954b3 100644
@@ -8469,6 +8469,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
 				pfn_max_align_up(end), migratetype);
 	return ret;
 }
+EXPORT_SYMBOL_GPL(alloc_contig_range);
 #endif /* CONFIG_CONTIG_ALLOC */
 
 void free_contig_range(unsigned long pfn, unsigned int nr_pages)
@@ -8483,6 +8484,7 @@ void free_contig_range(unsigned long pfn, unsigned int nr_pages)
 	}
 	WARN(count != 0, "%d pages are still in use!\n", count);
 }
+EXPORT_SYMBOL_GPL(free_contig_range);
 
 #ifdef CONFIG_MEMORY_HOTPLUG
 /*
Signature · 1 line
2.21.0
Signature · 2 lines
Michal Hocko
SUSE Labs
Acked-by on v3 4/9
#15 lore raw

Re: [PATCH RFC v3 4/9] mm: Export alloc_contig_range() / free_contig_range()

In reply to David Hildenbrand
Quoted 12 lines · David Hildenbrand

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task.

In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to alloc_contig_range()+unplug them on unplug request, or plug+free_contig_range() unplugged chunks on plug requests.

Quoted 7 lines

A virtio-mem device will use alloc_contig_range() / free_contig_range() only on ranges that belong to the same node/zone in at least MAX(MAX_ORDER - 1, pageblock_order) order granularity - e.g., 4MB on x86-64. The virtio-mem device added that memory, so the memory exists and does not contain any holes. virtio-mem will only try to allocate on ZONE_NORMAL, never on ZONE_MOVABLE, just like when allocating gigantic pages (we don't put unmovable data into the movable zone).

Is there any real reason to export as GPL rather than generic EXPORT_SYMBOL? In other words do we need to restrict the usage this interface only to GPL modules and why if so. All other allocator APIs are EXPORT_SYMBOL so there should better be a good reason for this one to differ. I can understand that this one is slightly different by requesting a specific range of the memory but it is still under a full control of the core MM to say no.

Quoted 11 lines
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Potapenko <[email protected]>

Other than that, I do not think exporting this function is harmful. It would be worse to reinvent it and do it wrong.

I usually prefer to add a caller in the same patch, though, because it makes the usage explicit and clear.

Quoted 26 lines
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)
mm/page_alloc.c +2 -0
index 3334a769eb91..d5d7944954b3 100644
@@ -8469,6 +8469,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
 				pfn_max_align_up(end), migratetype);
 	return ret;
 }
+EXPORT_SYMBOL_GPL(alloc_contig_range);
 #endif /* CONFIG_CONTIG_ALLOC */
 
 void free_contig_range(unsigned long pfn, unsigned int nr_pages)
@@ -8483,6 +8484,7 @@ void free_contig_range(unsigned long pfn, unsigned int nr_pages)
 	}
 	WARN(count != 0, "%d pages are still in use!\n", count);
 }
+EXPORT_SYMBOL_GPL(free_contig_range);
 
 #ifdef CONFIG_MEMORY_HOTPLUG
 /*
Signature · 1 line
2.21.0
Signature · 2 lines
Michal Hocko
SUSE Labs
#16 lore raw

Re: [PATCH RFC v3 4/9] mm: Export alloc_contig_range() / free_contig_range()

Quoted 21 lines · Michal Hocko
Quoted 12 lines · David Hildenbrand

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task.

In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to alloc_contig_range()+unplug them on unplug request, or plug+free_contig_range() unplugged chunks on plug requests.

Quoted 7 lines

A virtio-mem device will use alloc_contig_range() / free_contig_range() only on ranges that belong to the same node/zone in at least MAX(MAX_ORDER - 1, pageblock_order) order granularity - e.g., 4MB on x86-64. The virtio-mem device added that memory, so the memory exists and does not contain any holes. virtio-mem will only try to allocate on ZONE_NORMAL, never on ZONE_MOVABLE, just like when allocating gigantic pages (we don't put unmovable data into the movable zone).

Quoted 7 lines

Is there any real reason to export as GPL rather than generic EXPORT_SYMBOL? In other words do we need to restrict the usage this interface only to GPL modules and why if so. All other allocator APIs are EXPORT_SYMBOL so there should better be a good reason for this one to differ. I can understand that this one is slightly different by requesting a specific range of the memory but it is still under a full control of the core MM to say no.

I thought that we might - at least initially - might want to know all users. If you prefer, I can drop the GPL.

Quoted 13 lines
Quoted 12 lines
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Potapenko <[email protected]>
Quoted 6 lines

Other than that, I do not think exporting this function is harmful. It would be worse to reinvent it and do it wrong.

I usually prefer to add a caller in the same patch, though, because it makes the usage explicit and clear.

It's the next patch in this series (I prefer to split this from the actual driver):

https://lkml.org/lkml/2019/9/19/486

Quoted 1 line

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#17 lore raw

Re: [PATCH RFC v3 4/9] mm: Export alloc_contig_range() / free_contig_range()

In reply to Michal Hocko
Quoted 21 lines · Michal Hocko
Quoted 12 lines · David Hildenbrand

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task.

In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to alloc_contig_range()+unplug them on unplug request, or plug+free_contig_range() unplugged chunks on plug requests.

Quoted 7 lines

A virtio-mem device will use alloc_contig_range() / free_contig_range() only on ranges that belong to the same node/zone in at least MAX(MAX_ORDER - 1, pageblock_order) order granularity - e.g., 4MB on x86-64. The virtio-mem device added that memory, so the memory exists and does not contain any holes. virtio-mem will only try to allocate on ZONE_NORMAL, never on ZONE_MOVABLE, just like when allocating gigantic pages (we don't put unmovable data into the movable zone).

Quoted 7 lines

Is there any real reason to export as GPL rather than generic EXPORT_SYMBOL? In other words do we need to restrict the usage this interface only to GPL modules and why if so. All other allocator APIs are EXPORT_SYMBOL so there should better be a good reason for this one to differ. I can understand that this one is slightly different by requesting a specific range of the memory but it is still under a full control of the core MM to say no.

I thought that we might - at least initially - might want to know all users. If you prefer, I can drop the GPL.

Quoted 13 lines
Quoted 12 lines
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Potapenko <[email protected]>
Quoted 6 lines

Other than that, I do not think exporting this function is harmful. It would be worse to reinvent it and do it wrong.

I usually prefer to add a caller in the same patch, though, because it makes the usage explicit and clear.

It's the next patch in this series (I prefer to split this from the actual driver):

https://lkml.org/lkml/2019/9/19/486

Quoted 1 line

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#18 lore raw

RFC PATCH v3 5/9 virtio-mem: Paravirtualized memory hotunplug part 2

In reply to David Hildenbrand

We can use alloc_contig_range() to try to unplug subblocks. Unplugged blocks will be marked PG_offline, however, don't have the PG_reserved flag set. This way, we can differentiate these allocated subblocks from subblocks that were never onlined and handle them properly in virtio_mem_fake_online(). free_contig_range() is used to hand back subblocks to Linux.

It is worth noting that there are no guarantess on how much memory can actually get unplugged again. All device memory might completely be fragmented with unmovable data, such that no subblock can get unplugged. We might want to improve the unplugging capability in the future.

We are not touching the ZONE_MOVABLE. If memory is onlined to the ZONE_MOVABLE, it can only get unplugged after that memory was offlined manually by user space. In normal operation, virtio-mem memory is suggested to be onlined to ZONE_NORMAL. In the future, we will try to make unplug more likely to succeed.

Future work:
- Offline + remove memory blocks once all subblocks were unplugged. This might then free up unmovable data un other memory blocks.
- Performance improvements:
-- Sense (lockless) if it make sense to try alloc_contig_range() at all
before directly trying to isolate and taking locks.
-- Try to unplug bigger chunks if possible first.
-- Identify free areas first, that don't have to be evacuated.
- Make unplug more likely to succeed:
-- The "issue" is that in the ZONE_NORMAL, the buddy will randomly
allocate memory. Only pageblocks somewhat limit fragmentation, however we would want to limit fragmentation on subblock granularity and even memory block granularity. One idea is to have a new ZONE_PREFER_MOVABLE. Memory blocks will then be onlined to ZONE_NORMAL / ZONE_PREFER_MOVABLE in a certain ratio per node (e.g., 1:4). This makes unplug of quite some memory likely to succeed in most setups. ZONE_PREFER_MOVABLE is then a mixture of ZONE_NORMAL and ZONE_MOVABlE. Especially, movable data can end up on that zone, but only if really required - avoiding running out of memory on ZONE imbalances. The zone fallback order would be MOVABLE=>PREFER_MOVABLE=>HIGHMEM=>NORMAL=>PREFER_MOVABLE=>DMA32=>DMA
-- Allocate memmap from added memory. This way, less unmovable data can
end up on the memory blocks.
-- Call drop_slab() before trying to unplug. Eventually shrink other
caches.
- Better retry handling in case memory is busy. We certainly don't want to try for ever in a short interval to try to get some memory back.
- OOM handling, e.g., via an OOM handler.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/Kconfig      |   1 +
 drivers/virtio/virtio_mem.c | 106 +++++++++++++++++++++++++++++++++++-
 2 files changed, 104 insertions(+), 3 deletions(-)
drivers/virtio/Kconfig +1 -0
index 294720d53057..75a760f32ec7 100644
@@ -71,6 +71,7 @@ config VIRTIO_MEM
 	depends on VIRTIO
 	depends on MEMORY_HOTPLUG_SPARSE
 	depends on MEMORY_HOTREMOVE
+	select CONTIG_ALLOC
 	help
 	 This driver provides access to virtio-mem paravirtualized memory
 	 devices, allowing to hotplug and hotunplug memory.
drivers/virtio/virtio_mem.c +103 -3

131 lines not on this page · raw message

Signature · 1 line
2.21.0
#19 lore raw

RFC PATCH v3 5/9 virtio-mem: Paravirtualized memory hotunplug part 2

In reply to David Hildenbrand

We can use alloc_contig_range() to try to unplug subblocks. Unplugged blocks will be marked PG_offline, however, don't have the PG_reserved flag set. This way, we can differentiate these allocated subblocks from subblocks that were never onlined and handle them properly in virtio_mem_fake_online(). free_contig_range() is used to hand back subblocks to Linux.

It is worth noting that there are no guarantess on how much memory can actually get unplugged again. All device memory might completely be fragmented with unmovable data, such that no subblock can get unplugged. We might want to improve the unplugging capability in the future.

We are not touching the ZONE_MOVABLE. If memory is onlined to the ZONE_MOVABLE, it can only get unplugged after that memory was offlined manually by user space. In normal operation, virtio-mem memory is suggested to be onlined to ZONE_NORMAL. In the future, we will try to make unplug more likely to succeed.

Future work:
- Offline + remove memory blocks once all subblocks were unplugged. This might then free up unmovable data un other memory blocks.
- Performance improvements:
-- Sense (lockless) if it make sense to try alloc_contig_range() at all
before directly trying to isolate and taking locks.
-- Try to unplug bigger chunks if possible first.
-- Identify free areas first, that don't have to be evacuated.
- Make unplug more likely to succeed:
-- The "issue" is that in the ZONE_NORMAL, the buddy will randomly
allocate memory. Only pageblocks somewhat limit fragmentation, however we would want to limit fragmentation on subblock granularity and even memory block granularity. One idea is to have a new ZONE_PREFER_MOVABLE. Memory blocks will then be onlined to ZONE_NORMAL / ZONE_PREFER_MOVABLE in a certain ratio per node (e.g., 1:4). This makes unplug of quite some memory likely to succeed in most setups. ZONE_PREFER_MOVABLE is then a mixture of ZONE_NORMAL and ZONE_MOVABlE. Especially, movable data can end up on that zone, but only if really required - avoiding running out of memory on ZONE imbalances. The zone fallback order would be MOVABLE=>PREFER_MOVABLE=>HIGHMEM=>NORMAL=>PREFER_MOVABLE=>DMA32=>DMA
-- Allocate memmap from added memory. This way, less unmovable data can
end up on the memory blocks.
-- Call drop_slab() before trying to unplug. Eventually shrink other
caches.
- Better retry handling in case memory is busy. We certainly don't want to try for ever in a short interval to try to get some memory back.
- OOM handling, e.g., via an OOM handler.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/Kconfig      |   1 +
 drivers/virtio/virtio_mem.c | 106 +++++++++++++++++++++++++++++++++++-
 2 files changed, 104 insertions(+), 3 deletions(-)
drivers/virtio/Kconfig +1 -0
index 294720d53057..75a760f32ec7 100644
@@ -71,6 +71,7 @@ config VIRTIO_MEM
 	depends on VIRTIO
 	depends on MEMORY_HOTPLUG_SPARSE
 	depends on MEMORY_HOTREMOVE
+	select CONTIG_ALLOC
 	help
 	 This driver provides access to virtio-mem paravirtualized memory
 	 devices, allowing to hotplug and hotunplug memory.
drivers/virtio/virtio_mem.c +103 -3

131 lines not on this page · raw message

Signature · 1 line
2.21.0
#20 lore raw

RFC PATCH v3 6/9 mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Cc Andrew Morton <[email protected]>
Cc Juergen Gross <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Anthony Yznaga <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Pingfan Liu <[email protected]>
Cc Qian Cai <[email protected]>
Cc Dan Williams <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Wei Yang <[email protected]>
Cc Alexander Potapenko <[email protected]>
Cc Anshuman Khandual <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Cc Stephen Rothwell <[email protected]>
Cc Mauro Carvalho Chehab <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Yu Zhao <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Yang Shi <[email protected]>
Cc Ira Weiny <[email protected]>
Cc Andrey Ryabinin <[email protected]>
---
 include/linux/page-flags.h     |  4 ++++
 include/linux/page-isolation.h |  4 +++-
 mm/memory_hotplug.c            |  9 ++++++---
 mm/page_alloc.c                | 22 +++++++++++++++++++++-
 mm/page_isolation.c            | 18 +++++++++++++-----
 mm/swap.c                      |  9 +++++++++
 6 files changed, 56 insertions(+), 10 deletions(-)
include/linux/page-flags.h +4 -0
index f91cb8898ff0..7e563eab6b4b 100644
@@ -745,6 +745,10 @@ PAGE_TYPE_OPS(Buddy, buddy)
  * not onlined when onlining the section).
  * The content of these pages is effectively stale. Such pages should not
  * be touched (read/write/dump/save) except by their owner.
+ *
+ * PageOffline() pages that have a reference count of 0 will be treated
+ * like free pages when offlining pages, allowing the containing memory
+ * block to get offlined.
  */
 PAGE_TYPE_OPS(Offline, offline)
 
include/linux/page-isolation.h +3 -1
index 1099c2fee20f..024e02b60346 100644
@@ -32,6 +32,8 @@ static inline bool is_migrate_isolate(int migratetype)
 
 #define SKIP_HWPOISON	0x1
 #define REPORT_FAILURE	0x2
+/* Skip PageOffline() pages with a reference count of 0. */
+#define SKIP_OFFLINE	0x4
 
 bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
 			 int migratetype, int flags);
@@ -58,7 +60,7 @@ undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
  * Test all pages in [start_pfn, end_pfn) are isolated or not.
  */
 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn,
-			bool skip_hwpoisoned_pages);
+			int flags);
 
 struct page *alloc_migrate_target(struct page *page, unsigned long private);
 
mm/memory_hotplug.c +6 -3
index f08eb429b8f3..d23ff7c5c96b 100644
@@ -1127,7 +1127,8 @@ static bool is_pageblock_removable_nolock(unsigned long pfn)
 	if (!zone_spans_pfn(zone, pfn))
 		return false;
 
-	return !has_unmovable_pages(zone, page, 0, MIGRATE_MOVABLE, SKIP_HWPOISON);
+	return !has_unmovable_pages(zone, page, 0, MIGRATE_MOVABLE,
+				    SKIP_HWPOISON | SKIP_OFFLINE);
 }
 
 /* Checks if this range of memory is likely to be hot-removable. */
@@ -1344,7 +1345,8 @@ static int
 check_pages_isolated_cb(unsigned long start_pfn, unsigned long nr_pages,
 			void *data)
 {
-	return test_pages_isolated(start_pfn, start_pfn + nr_pages, true);
+	return test_pages_isolated(start_pfn, start_pfn + nr_pages,
+				   SKIP_HWPOISON | SKIP_OFFLINE);
 }
 
 static int __init cmdline_parse_movable_node(char *p)
@@ -1455,7 +1457,8 @@ static int __ref __offline_pages(unsigned long start_pfn,
 	/* set above range as isolated */
 	ret = start_isolate_page_range(start_pfn, end_pfn,
 				       MIGRATE_MOVABLE,
-				       SKIP_HWPOISON | REPORT_FAILURE);
+				       SKIP_HWPOISON | REPORT_FAILURE |
+				       SKIP_OFFLINE);
 	if (ret < 0) {
 		reason = "failure to isolate range";
 		goto failed_removal;
mm/page_alloc.c +21 -1
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
 		if (!page_ref_count(page)) {
 			if (PageBuddy(page))
 				iter += (1 << page_order(page)) - 1;
+			/*
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
+					goto unmovable;
 			continue;
 		}
 
@@ -8444,7 +8453,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
 	}
 
 	/* Make sure the range is really isolated. */
-	if (test_pages_isolated(outer_start, end, false)) {
+	if (test_pages_isolated(outer_start, end, 0)) {
 		pr_info_ratelimited("%s: [%lx, %lx) PFNs busy\n",
 			__func__, outer_start, end);
 		ret = -EBUSY;
@@ -8563,6 +8572,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)
 			offlined_pages++;
 			continue;
 		}
+		/*
+		 * Memory devices allow to offline a page if it is marked
+		 * PG_offline and has a reference count of 0.
+		 */
+		if (PageOffline(page) && !page_count(page)) {
+			BUG_ON(PageBuddy(page));
+			pfn++;
+			SetPageReserved(page);
+			offlined_pages++;
+			continue;
+		}
 
 		BUG_ON(page_count(page));
 		BUG_ON(!PageBuddy(page));
mm/page_isolation.c +13 -5
index 89c19c0feadb..0a75019d7e7c 100644
@@ -171,6 +171,8 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages)
  *			SKIP_HWPOISON - ignore hwpoison pages
  *			REPORT_FAILURE - report details about the failure to
  *			isolate the range
+ *			SKIP_OFFLINE - ignore PageOffline() pages with a
+ *			reference count of 0
  *
  * Making page-allocation-type to be MIGRATE_ISOLATE means free pages in
  * the range will never be allocated. Any free pages and pages freed in the
@@ -257,7 +259,7 @@ void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn,
  */
 static unsigned long
 __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn,
-				  bool skip_hwpoisoned_pages)
+				  int flags)
 {
 	struct page *page;
 
@@ -274,9 +276,16 @@ __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn,
 			 * simple way to verify that as VM_BUG_ON(), though.
 			 */
 			pfn += 1 << page_order(page);
-		else if (skip_hwpoisoned_pages && PageHWPoison(page))
+		else if ((flags & SKIP_HWPOISON) && PageHWPoison(page))
 			/* A HWPoisoned page cannot be also PageBuddy */
 			pfn++;
+		else if ((flags & SKIP_OFFLINE) && PageOffline(page) &&
+			 !page_count(page))
+			/*
+			 * Memory devices allow to offline a page if it is
+			 * marked PG_offline and has a reference count of 0.
+			 */
+			pfn++;
 		else
 			break;
 	}
@@ -286,7 +295,7 @@ __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn,
 
 /* Caller should ensure that requested range is in a single zone */
 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn,
-			bool skip_hwpoisoned_pages)
+			int isol_flags)
 {
 	unsigned long pfn, flags;
 	struct page *page;
@@ -308,8 +317,7 @@ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn,
 	/* Check all pages are free or marked as ISOLATED */
 	zone = page_zone(page);
 	spin_lock_irqsave(&zone->lock, flags);
-	pfn = __test_page_isolated_in_pageblock(start_pfn, end_pfn,
-						skip_hwpoisoned_pages);
+	pfn = __test_page_isolated_in_pageblock(start_pfn, end_pfn, isol_flags);
 	spin_unlock_irqrestore(&zone->lock, flags);
 
 	trace_test_pages_isolated(start_pfn, end_pfn, pfn);
mm/swap.c +9 -0
index 38c3fa4308e2..f98987656ecc 100644
@@ -107,6 +107,15 @@ void __put_page(struct page *page)
 		 * not return it to page allocator.
 		 */
 		return;
+	} else if (PageOffline(page)) {
+		/*
+		 * Memory devices allow to offline a page if it is
+		 * marked PG_offline and has a reference count of 0. So if
+		 * somebody puts a reference of such a page and the
+		 * reference count drops to 0, don't return the page to the
+		 * buddy.
+		 */
+		return;
 	}
 
 	if (unlikely(PageCompound(page)))
Signature · 1 line
2.21.0
#21 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 25 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

Quoted 6 lines

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

[...]

Quoted 9 lines
mm/page_alloc.c +1 -0
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
 		if (!page_ref_count(page)) {
 			if (PageBuddy(page))
 				iter += (1 << page_order(page)) - 1;
+			/*

mm/page_alloc.c bool has_unmovable_pages(struct zone *zone, struct page *page, int count, · line 8234

Quoted 10 lines
mm/page_alloc.c +8 -0
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
+					goto unmovable;
 			continue;
 		}

Do we really need to distinguish offline and hwpoison pages? They are both unmovable for allocator purposes and offlineable for the hotplug, right? Should we just hide them behind a helper and use it rather than an explicit SKIP_$FOO?

Signature · 2 lines
Michal Hocko
SUSE Labs
#22 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 25 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

Quoted 6 lines

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

[...]

Quoted 9 lines
mm/page_alloc.c +1 -0
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
 		if (!page_ref_count(page)) {
 			if (PageBuddy(page))
 				iter += (1 << page_order(page)) - 1;
+			/*

mm/page_alloc.c bool has_unmovable_pages(struct zone *zone, struct page *page, int count, · line 8234

Quoted 10 lines
mm/page_alloc.c +8 -0
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
+					goto unmovable;
 			continue;
 		}

Do we really need to distinguish offline and hwpoison pages? They are both unmovable for allocator purposes and offlineable for the hotplug, right? Should we just hide them behind a helper and use it rather than an explicit SKIP_$FOO?

Signature · 2 lines
Michal Hocko
SUSE Labs
#23 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Once offline, the memmap is garbage. When onlining again:

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

The driver that marked these pages to be skipped when offlining is responsible for registering the online_page_callback_t callback where these pages will get excluded.

This is exactly the same as when onling a memory block that is partially populated (e.g., HpyerV balloon right now).

So it's effectively "re-initializing the memmap using the driver knowledge" when onlining.

Quoted 1 line

Should we allow to try_remove_memory to succeed with these pages?

I think we should first properly offline them (mark sections offline and memory blocks, fixup numbers, shrink zones ...). The we can cleanly remove the memory. (see [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory())

Once offline, the memmap is irrelevant and try_remove_memory() can do its job.

Quoted 6 lines

Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

Returning them to the buddy is problematic for various reasons. Let's have a look at __offline_pages():

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that.

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch
these pages. I don't see a way to hack around that.

3) scan_movable_pages() ...

4a) memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);

-> Offlining failed for whatever reason.
-> Pages are in the buddy, but we already un-isolated them. BAD.

By not going via the buddy we avoid these issues and can leave PG_offline set until the section is fully offline. Something that is very desirable for virtio-mem (and as far as I can tell also HyperV in the future).

Quoted 18 lines

[...]

Quoted 16 lines
mm/page_alloc.c +8 -0
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
  		if (!page_ref_count(page)) {
  			if (PageBuddy(page))
  				iter += (1 << page_order(page)) - 1;
+			/*
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
Quoted 8 lines

mm/page_alloc.c bool has_unmovable_pages(struct zone *zone, struct page *page, int count, · line 8234

Quoted 3 lines
mm/page_alloc.c +1 -0
+					goto unmovable;
  			continue;
  		}

Do we really need to distinguish offline and hwpoison pages? They are both unmovable for allocator purposes and offlineable for the hotplug, right? Should we just hide them behind a helper and use it rather than an explicit SKIP_$FOO?

Makes sense. It really boils down to "offline" vs. "allocate" use cases.

So maybe instead of "SKIP_FOO" something like "MEMORY_OFFLINE". ?

Signature · 3 lines
Thanks,

David / dhildenb
#24 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Once offline, the memmap is garbage. When onlining again:

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

The driver that marked these pages to be skipped when offlining is responsible for registering the online_page_callback_t callback where these pages will get excluded.

This is exactly the same as when onling a memory block that is partially populated (e.g., HpyerV balloon right now).

So it's effectively "re-initializing the memmap using the driver knowledge" when onlining.

Quoted 1 line

Should we allow to try_remove_memory to succeed with these pages?

I think we should first properly offline them (mark sections offline and memory blocks, fixup numbers, shrink zones ...). The we can cleanly remove the memory. (see [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory())

Once offline, the memmap is irrelevant and try_remove_memory() can do its job.

Quoted 6 lines

Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

Returning them to the buddy is problematic for various reasons. Let's have a look at __offline_pages():

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that.

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch
these pages. I don't see a way to hack around that.

3) scan_movable_pages() ...

4a) memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);

-> Offlining failed for whatever reason.
-> Pages are in the buddy, but we already un-isolated them. BAD.

By not going via the buddy we avoid these issues and can leave PG_offline set until the section is fully offline. Something that is very desirable for virtio-mem (and as far as I can tell also HyperV in the future).

Quoted 18 lines

[...]

Quoted 16 lines
mm/page_alloc.c +8 -0
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
  		if (!page_ref_count(page)) {
  			if (PageBuddy(page))
  				iter += (1 << page_order(page)) - 1;
+			/*
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
Quoted 8 lines

mm/page_alloc.c bool has_unmovable_pages(struct zone *zone, struct page *page, int count, · line 8234

Quoted 3 lines
mm/page_alloc.c +1 -0
+					goto unmovable;
  			continue;
  		}

Do we really need to distinguish offline and hwpoison pages? They are both unmovable for allocator purposes and offlineable for the hotplug, right? Should we just hide them behind a helper and use it rather than an explicit SKIP_$FOO?

Makes sense. It really boils down to "offline" vs. "allocate" use cases.

So maybe instead of "SKIP_FOO" something like "MEMORY_OFFLINE". ?

Signature · 3 lines
Thanks,

David / dhildenb
#25 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 37 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Quoted 6 lines

Once offline, the memmap is garbage. When onlining again:

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

But we can skip those pages when onlining and keep them in the offline state right? We do not poison offlined pages.

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Quoted 9 lines

The driver that marked these pages to be skipped when offlining is responsible for registering the online_page_callback_t callback where these pages will get excluded.

This is exactly the same as when onling a memory block that is partially populated (e.g., HpyerV balloon right now).

So it's effectively "re-initializing the memmap using the driver knowledge" when onlining.

I am not sure I follow. So you exclude those pages when onlining?

Quoted 6 lines
Quoted 1 line

Should we allow to try_remove_memory to succeed with these pages?

I think we should first properly offline them (mark sections offline and memory blocks, fixup numbers, shrink zones ...). The we can cleanly remove the memory. (see [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory())

I will have a look, but just to quick question. try_remove_memory would fail if the range is offline (via user interface) but there are still some pages in the driver Offline state?

Quoted 31 lines

Once offline, the memmap is irrelevant and try_remove_memory() can do its job.

Quoted 6 lines

Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

Returning them to the buddy is problematic for various reasons. Let's have a look at __offline_pages():

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that.

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch
these pages. I don't see a way to hack around that.

3) scan_movable_pages() ...

4a) memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);
Quoted 6 lines

-> Offlining failed for whatever reason.
-> Pages are in the buddy, but we already un-isolated them. BAD.

By not going via the buddy we avoid these issues and can leave PG_offline set until the section is fully offline. Something that is very desirable for virtio-mem (and as far as I can tell also HyperV in the future).

I am not sure I follow. Maybe my original question was confusing. Let me ask again. Why do we need to hook into __put_page?

Quoted 26 lines
Quoted 17 lines

[...]

Quoted 16 lines
mm/page_alloc.c +8 -0
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
  		if (!page_ref_count(page)) {
  			if (PageBuddy(page))
  				iter += (1 << page_order(page)) - 1;
+			/*
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
Quoted 8 lines

mm/page_alloc.c bool has_unmovable_pages(struct zone *zone, struct page *page, int count, · line 8234

Quoted 3 lines
mm/page_alloc.c +1 -0
+					goto unmovable;
  			continue;
  		}

Do we really need to distinguish offline and hwpoison pages? They are both unmovable for allocator purposes and offlineable for the hotplug, right? Should we just hide them behind a helper and use it rather than an explicit SKIP_$FOO?

Quoted 3 lines

Makes sense. It really boils down to "offline" vs. "allocate" use cases.

So maybe instead of "SKIP_FOO" something like "MEMORY_OFFLINE". ?

Yes, that would be a better fit.

Signature · 2 lines
Michal Hocko
SUSE Labs
#26 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 37 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Quoted 6 lines

Once offline, the memmap is garbage. When onlining again:

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

But we can skip those pages when onlining and keep them in the offline state right? We do not poison offlined pages.

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Quoted 9 lines

The driver that marked these pages to be skipped when offlining is responsible for registering the online_page_callback_t callback where these pages will get excluded.

This is exactly the same as when onling a memory block that is partially populated (e.g., HpyerV balloon right now).

So it's effectively "re-initializing the memmap using the driver knowledge" when onlining.

I am not sure I follow. So you exclude those pages when onlining?

Quoted 6 lines
Quoted 1 line

Should we allow to try_remove_memory to succeed with these pages?

I think we should first properly offline them (mark sections offline and memory blocks, fixup numbers, shrink zones ...). The we can cleanly remove the memory. (see [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory())

I will have a look, but just to quick question. try_remove_memory would fail if the range is offline (via user interface) but there are still some pages in the driver Offline state?

Quoted 31 lines

Once offline, the memmap is irrelevant and try_remove_memory() can do its job.

Quoted 6 lines

Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

Returning them to the buddy is problematic for various reasons. Let's have a look at __offline_pages():

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that.

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch
these pages. I don't see a way to hack around that.

3) scan_movable_pages() ...

4a) memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);
Quoted 6 lines

-> Offlining failed for whatever reason.
-> Pages are in the buddy, but we already un-isolated them. BAD.

By not going via the buddy we avoid these issues and can leave PG_offline set until the section is fully offline. Something that is very desirable for virtio-mem (and as far as I can tell also HyperV in the future).

I am not sure I follow. Maybe my original question was confusing. Let me ask again. Why do we need to hook into __put_page?

Quoted 26 lines
Quoted 17 lines

[...]

Quoted 16 lines
mm/page_alloc.c +8 -0
index d5d7944954b3..fef74720d8b4 100644
@@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
  		if (!page_ref_count(page)) {
  			if (PageBuddy(page))
  				iter += (1 << page_order(page)) - 1;
+			/*
+			* Memory devices allow to offline a page if it is
+			* marked PG_offline and has a reference count of 0.
+			* However, the pages are not movable as it would be
+			* required e.g., for alloc_contig_range().
+			*/
+			if (PageOffline(page) && !(flags & SKIP_OFFLINE))
+				if (++found > count)
Quoted 8 lines

mm/page_alloc.c bool has_unmovable_pages(struct zone *zone, struct page *page, int count, · line 8234

Quoted 3 lines
mm/page_alloc.c +1 -0
+					goto unmovable;
  			continue;
  		}

Do we really need to distinguish offline and hwpoison pages? They are both unmovable for allocator purposes and offlineable for the hotplug, right? Should we just hide them behind a helper and use it rather than an explicit SKIP_$FOO?

Quoted 3 lines

Makes sense. It really boils down to "offline" vs. "allocate" use cases.

So maybe instead of "SKIP_FOO" something like "MEMORY_OFFLINE". ?

Yes, that would be a better fit.

Signature · 2 lines
Michal Hocko
SUSE Labs
#27 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 40 lines · Michal Hocko
Quoted 39 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Once offline, the memmap is garbage. When onlining again:

Quoted 7 lines
Quoted 4 lines

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

But we can skip those pages when onlining and keep them in the offline state right? We do not poison offlined pages.

Right now, I do that via the online_page_callback_t call (similar to HyperV), as the memmap is basically garbage and not trustworthy.

Quoted 6 lines

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Sorry to say, but whenever we online memory the memmap is overwritten, because there is no way you could tell it contains garbage or not. You have to assume it is garbage. (my recent patch even poisons the memmap when offlining, which helped to find a lot of these "garbage memmap" BUGs)

online_pages()

	...
	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL);
	...
		memmap_init_zone()
			-> memmap initialized

So yes, offlining memory with HWPoison and re-onlining it effectively drops HWPoison markers. On the next access, you will trigger a new HWPoison.

Quoted 5 lines
Quoted 5 lines

The driver that marked these pages to be skipped when offlining is responsible for registering the online_page_callback_t callback where these pages will get excluded.

Quoted 7 lines
Quoted 5 lines

This is exactly the same as when onling a memory block that is partially populated (e.g., HpyerV balloon right now).

So it's effectively "re-initializing the memmap using the driver knowledge" when onlining.

I am not sure I follow. So you exclude those pages when onlining?

Exactly, using the online_page callback. The pages will - again - be marked PG_offline with a refcount of 0. They will not be put to the buddy.

Quoted 11 lines
Quoted 7 lines
Quoted 2 lines

Should we allow to try_remove_memory to succeed with these pages?

I think we should first properly offline them (mark sections offline and memory blocks, fixup numbers, shrink zones ...). The we can cleanly remove the memory. (see [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory())

I will have a look, but just to quick question. try_remove_memory would fail if the range is offline (via user interface) but there are still some pages in the driver Offline state?

try_remove_memory() does not check any memmap (because it is garbage), it only makes sure that the memory blocks are properly marked as offline. (IOW, device_offline() was called on the memory block).

Quoted 35 lines
Quoted 35 lines

Once offline, the memmap is irrelevant and try_remove_memory() can do its job.

Quoted 6 lines

Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

Returning them to the buddy is problematic for various reasons. Let's have a look at __offline_pages():

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that.

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch
these pages. I don't see a way to hack around that.

3) scan_movable_pages() ...

4a) memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
     memory_notify(MEM_CANCEL_OFFLINE, &arg);

-> Offlining failed for whatever reason.
-> Pages are in the buddy, but we already un-isolated them. BAD.

Quoted 6 lines
Quoted 3 lines

By not going via the buddy we avoid these issues and can leave PG_offline set until the section is fully offline. Something that is very desirable for virtio-mem (and as far as I can tell also HyperV in the future).

I am not sure I follow. Maybe my original question was confusing. Let me ask again. Why do we need to hook into __put_page?

Just replied again answering this question, before I read this mail :)

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#28 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 40 lines · Michal Hocko
Quoted 39 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Once offline, the memmap is garbage. When onlining again:

Quoted 7 lines
Quoted 4 lines

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

But we can skip those pages when onlining and keep them in the offline state right? We do not poison offlined pages.

Right now, I do that via the online_page_callback_t call (similar to HyperV), as the memmap is basically garbage and not trustworthy.

Quoted 6 lines

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Sorry to say, but whenever we online memory the memmap is overwritten, because there is no way you could tell it contains garbage or not. You have to assume it is garbage. (my recent patch even poisons the memmap when offlining, which helped to find a lot of these "garbage memmap" BUGs)

online_pages()

	...
	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL);
	...
		memmap_init_zone()
			-> memmap initialized

So yes, offlining memory with HWPoison and re-onlining it effectively drops HWPoison markers. On the next access, you will trigger a new HWPoison.

Quoted 5 lines
Quoted 5 lines

The driver that marked these pages to be skipped when offlining is responsible for registering the online_page_callback_t callback where these pages will get excluded.

Quoted 7 lines
Quoted 5 lines

This is exactly the same as when onling a memory block that is partially populated (e.g., HpyerV balloon right now).

So it's effectively "re-initializing the memmap using the driver knowledge" when onlining.

I am not sure I follow. So you exclude those pages when onlining?

Exactly, using the online_page callback. The pages will - again - be marked PG_offline with a refcount of 0. They will not be put to the buddy.

Quoted 11 lines
Quoted 7 lines
Quoted 2 lines

Should we allow to try_remove_memory to succeed with these pages?

I think we should first properly offline them (mark sections offline and memory blocks, fixup numbers, shrink zones ...). The we can cleanly remove the memory. (see [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory())

I will have a look, but just to quick question. try_remove_memory would fail if the range is offline (via user interface) but there are still some pages in the driver Offline state?

try_remove_memory() does not check any memmap (because it is garbage), it only makes sure that the memory blocks are properly marked as offline. (IOW, device_offline() was called on the memory block).

Quoted 35 lines
Quoted 35 lines

Once offline, the memmap is irrelevant and try_remove_memory() can do its job.

Quoted 6 lines

Do we really have hook into __put_page? Why do we even care about the reference count of those pages? Wouldn't it be just more consistent to elevate the reference count (I guess this is what you suggest in the last paragraph) and the virtio driver would return that page to the buddy by regular put_page. This is also related to the above question about the physical memory remove.

Returning them to the buddy is problematic for various reasons. Let's have a look at __offline_pages():

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that.

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch
these pages. I don't see a way to hack around that.

3) scan_movable_pages() ...

4a) memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
     memory_notify(MEM_CANCEL_OFFLINE, &arg);

-> Offlining failed for whatever reason.
-> Pages are in the buddy, but we already un-isolated them. BAD.

Quoted 6 lines
Quoted 3 lines

By not going via the buddy we avoid these issues and can leave PG_offline set until the section is fully offline. Something that is very desirable for virtio-mem (and as far as I can tell also HyperV in the future).

I am not sure I follow. Maybe my original question was confusing. Let me ask again. Why do we need to hook into __put_page?

Just replied again answering this question, before I read this mail :)

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#29 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 1 line · David Hildenbrand

On 16.10.19 15:45, Michal Hocko wrote:

[...]

Quoted 12 lines
Quoted 5 lines

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Sorry to say, but whenever we online memory the memmap is overwritten, because there is no way you could tell it contains garbage or not. You have to assume it is garbage. (my recent patch even poisons the memmap when offlining, which helped to find a lot of these "garbage memmap" BUGs)

online_pages()

Quoted 8 lines
	...
	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL);
	...
		memmap_init_zone()
			-> memmap initialized

So yes, offlining memory with HWPoison and re-onlining it effectively drops HWPoison markers. On the next access, you will trigger a new HWPoison.

Right you are! I need to sit on this much more and think about it with a clean head.

Signature · 2 lines
Michal Hocko
SUSE Labs
#30 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 1 line · David Hildenbrand

On 16.10.19 15:45, Michal Hocko wrote:

[...]

Quoted 12 lines
Quoted 5 lines

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Sorry to say, but whenever we online memory the memmap is overwritten, because there is no way you could tell it contains garbage or not. You have to assume it is garbage. (my recent patch even poisons the memmap when offlining, which helped to find a lot of these "garbage memmap" BUGs)

online_pages()

Quoted 8 lines
	...
	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL);
	...
		memmap_init_zone()
			-> memmap initialized

So yes, offlining memory with HWPoison and re-onlining it effectively drops HWPoison markers. On the next access, you will trigger a new HWPoison.

Right you are! I need to sit on this much more and think about it with a clean head.

Signature · 2 lines
Michal Hocko
SUSE Labs
#31 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 21 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 15:45, Michal Hocko wrote:

[...]

Quoted 18 lines
Quoted 5 lines

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Sorry to say, but whenever we online memory the memmap is overwritten, because there is no way you could tell it contains garbage or not. You have to assume it is garbage. (my recent patch even poisons the memmap when offlining, which helped to find a lot of these "garbage memmap" BUGs)

online_pages()

	...
	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL);
	...
		memmap_init_zone()
			-> memmap initialized
Quoted 6 lines
Quoted 2 lines

So yes, offlining memory with HWPoison and re-onlining it effectively drops HWPoison markers. On the next access, you will trigger a new HWPoison.

Right you are! I need to sit on this much more and think about it with a clean head.

Sure, take your time and let me know if you need any details on how virtio-mem works, how it uses these interfaces, and how the mm internals it uses play along.

Thanks Michal!

Signature · 3 lines
Thanks,

David / dhildenb
#32 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 21 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 15:45, Michal Hocko wrote:

[...]

Quoted 18 lines
Quoted 5 lines

There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing that and maybe we don't handle that. But we cannot simply online hwpoisoned page. Offlining the range will not make a broken memory OK all of the sudden. And your usecase sounds similar to me.

Sorry to say, but whenever we online memory the memmap is overwritten, because there is no way you could tell it contains garbage or not. You have to assume it is garbage. (my recent patch even poisons the memmap when offlining, which helped to find a lot of these "garbage memmap" BUGs)

online_pages()

	...
	move_pfn_range_to_zone(zone, pfn, nr_pages, NULL);
	...
		memmap_init_zone()
			-> memmap initialized
Quoted 6 lines
Quoted 2 lines

So yes, offlining memory with HWPoison and re-onlining it effectively drops HWPoison markers. On the next access, you will trigger a new HWPoison.

Right you are! I need to sit on this much more and think about it with a clean head.

Sure, take your time and let me know if you need any details on how virtio-mem works, how it uses these interfaces, and how the mm internals it uses play along.

Thanks Michal!

Signature · 3 lines
Thanks,

David / dhildenb
#33 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 40 lines · Michal Hocko
Quoted 39 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 5 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?

Once offline, the memmap is garbage. When onlining again:

Quoted 7 lines
Quoted 4 lines

a) memmap will be re-initialized
b) online_page_callback_t will be called for every page in the section. The driver can mark them offline again and not give them to the buddy.
c) section will be marked online.

But we can skip those pages when onlining and keep them in the offline state right? We do not poison offlined pages.

https://lkml.org/lkml/2019/10/6/60

But again, onlining will overwrite the whole memmap right now and there is no way to identify if a memmap contains garbage or not.

We would have to identify/remember if re-onlining, but I am not yet sure if re-using memmaps when onlining is such a good idea ...

Signature · 3 lines
Thanks,

David / dhildenb
#34 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 8 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages?

Oh, I forgot to answer this questions. The __put_page() change is necessary for the following race I identified:

Page has a refcount of 1 (e.g., allocated by virtio-mem using alloc_contig_range()).

a) kernel: get_page_unless_zero(page): refcount = 2
b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1
c) kernel: put_page(page): refcount = 0

The page would suddenly be given to the buddy. which is bad.

Signature · 3 lines
Thanks,

David / dhildenb
#35 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 8 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages?

Oh, I forgot to answer this questions. The __put_page() change is necessary for the following race I identified:

Page has a refcount of 1 (e.g., allocated by virtio-mem using alloc_contig_range()).

a) kernel: get_page_unless_zero(page): refcount = 2
b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1
c) kernel: put_page(page): refcount = 0

The page would suddenly be given to the buddy. which is bad.

Signature · 3 lines
Thanks,

David / dhildenb
#36 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 43 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 8 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages?

Oh, I forgot to answer this questions. The __put_page() change is necessary for the following race I identified:

Quoted 8 lines

Page has a refcount of 1 (e.g., allocated by virtio-mem using alloc_contig_range()).

a) kernel: get_page_unless_zero(page): refcount = 2
b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1
c) kernel: put_page(page): refcount = 0

The page would suddenly be given to the buddy. which is bad.

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

An elevated ref count could help to detect that the memory hotremove is not safe until the driver removes all potential metadata it might still hold. You also know that memory online should skip such a page.

All in all your page is still in use by the driver and the life cycle is controlled by that driver.

Or am I am missing something?

Signature · 2 lines
Michal Hocko
SUSE Labs
#37 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 43 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 8 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages?

Oh, I forgot to answer this questions. The __put_page() change is necessary for the following race I identified:

Quoted 8 lines

Page has a refcount of 1 (e.g., allocated by virtio-mem using alloc_contig_range()).

a) kernel: get_page_unless_zero(page): refcount = 2
b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1
c) kernel: put_page(page): refcount = 0

The page would suddenly be given to the buddy. which is bad.

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

An elevated ref count could help to detect that the memory hotremove is not safe until the driver removes all potential metadata it might still hold. You also know that memory online should skip such a page.

All in all your page is still in use by the driver and the life cycle is controlled by that driver.

Or am I am missing something?

Signature · 2 lines
Michal Hocko
SUSE Labs
#38 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 51 lines · Michal Hocko
Quoted 50 lines · David Hildenbrand
Quoted 30 lines · Michal Hocko
Quoted 29 lines · David Hildenbrand

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's use "PageOffline() + reference count = 0" as a sign to memory offlining code that these pages can simply be skipped when offlining, similar to free or HWPoison pages.

Pass flags to test_pages_isolated(), similar as already done for has_unmovable_pages(). Use a new flag to indicate the requirement of memory offlining to skip over these special pages.

In has_unmovable_pages(), make sure the pages won't be detected as movable. This is not strictly necessary, however makes e.g., alloc_contig_range() stop early, trying to isolate such page blocks - compared to failing later when testing if all pages were isolated.

Also, make sure that when a reference to a PageOffline() page is dropped, that the page will not be returned to the buddy.

memory devices (like virtio-mem) that want to make use of this functionality have to make sure to synchronize against memory offlining, using the memory hotplug notifier.

Quoted 8 lines
Quoted 2 lines

Alternative: Allow to offline with a reference count of 1 and use some other sign in the struct page that offlining is permitted.

Few questions. I do not see onlining code to take care of this special case. What should happen when offline && online?
Should we allow to try_remove_memory to succeed with these pages? Do we really have hook into __put_page? Why do we even care about the reference count of those pages?

Oh, I forgot to answer this questions. The __put_page() change is necessary for the following race I identified:

Page has a refcount of 1 (e.g., allocated by virtio-mem using alloc_contig_range()).

a) kernel: get_page_unless_zero(page): refcount = 2
b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1
c) kernel: put_page(page): refcount = 0

Quoted 7 lines
Quoted 1 line

The page would suddenly be given to the buddy. which is bad.

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

See below on details on how to revive such unplugged memory.

Quoted 4 lines

An elevated ref count could help to detect that the memory hotremove is not safe until the driver removes all potential metadata it might still hold. You also know that memory online should skip such a page.

Again, when onlining memory you have to assume the memmap is complete garbage. No trusting *at all* on that as of now. This represents a BIG change to what we have right now. Not totally against that, but it sounds like a bigger rework that mainly helps to fix HWPoison.

Quoted 3 lines

All in all your page is still in use by the driver and the life cycle is controlled by that driver.

The driver let go of all direct references. The page (or rather chunks) are in no list. The "knowledge" that these pages are offline are stored in some external metadata. This metadata is updated when notified via the memory notifier.

What happens in case virtio-mem wants to revive a chunk (IOW, plug unplugged memory)?

a) it makes sure no concurrent memory onlining/offlining can happen (locking via memory notifiers)
b) it grabs a reference to the page (increasing the refcount)
c) it clears PG_offline and issues free_contig_range().

Quoted 3 lines

Or am I am missing something?

It's just complex stuff :) I guess the part you are missing is that the driver officially signals "I have no direct reference, you can offline this memory, I know how to deal with that". It's not like "this is a balloon inflated page I keep in a list".

Signature · 3 lines
Thanks,

David / dhildenb
#39 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Let's say that you still keep a reference to your offlined pages and mark them offlined. That should make sure that no consumer of the pfn_to_online_page will touch the page's content nor the state. Now admin might want to offline/hotremove the whole memory block via sysfs. An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not. If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

Or do I still see it too simplistically and the notifier is not a good place to handle the reference count?

Signature · 2 lines
Michal Hocko
SUSE Labs
#40 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Let's say that you still keep a reference to your offlined pages and mark them offlined. That should make sure that no consumer of the pfn_to_online_page will touch the page's content nor the state. Now admin might want to offline/hotremove the whole memory block via sysfs. An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not. If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

Or do I still see it too simplistically and the notifier is not a good place to handle the reference count?

Signature · 2 lines
Michal Hocko
SUSE Labs
#41 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 14 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Quoted 6 lines

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Thanks for thinking about this. I still believe it is problematic. And I don't like releasing "pages that should not be used" to the buddy. It comes with some problems if offlining fails (see below).

Quoted 4 lines

Let's say that you still keep a reference to your offlined pages and mark them offlined. That should make sure that no consumer of the pfn_to_online_page will touch the page's content nor the state. Now

pfn_to_online_page() does not check against PG_offline. (which is correct IMHO. As documented, PG_offline means "logically offline".

The memmap is valid, however the page content should not be accessed. (PG_logically_offline would have been ugly ;) ).

But that shouldn't be an issue right now.

Quoted 1 line

admin might want to offline/hotremove the whole memory block via sysfs.

The admin can only trigger offlining via sysfs. Hotremove is under control of the driver. (removing the whole memory block)

And with the current virtio-mem prototype, this works like a charm.

Quoted 3 lines

An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface

And here is the thing: The owner of the page does not have to do anything to untrack the page. I mean that's what a reference count of zero actually means - no direct reference.

I could emphasize how this is to be used in the documentation:

PageOffline() pages that have a reference count of 0 will be treated
like free pages when offlining pages, allowing the containing memory
block to get offlined. In case a driver wants to revive such a page, it has to synchronize against memory onlining/offlining (e.g., using memory notifiers) while incrementing the reference count. Also, a driver that relies in this feature is aware that re-onlining the memory will require to re-set the pages PageOffline() - e.g., via the online_page_callback_t.

Quoted 2 lines

for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not.

As I explained, this is too late and fragile. I post again what I posted before with some further explanations

__offline_pages() works like this:

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> offlining aborted. (see below on the memory isolation notifier)

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

3) scan_movable_pages() ...

4a) Memory offlining succeeded: memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) Memory offlining failed

    undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);

-> Offlining failed for whatever reason. Once we reach the notifier, the
pages are already back in use by the buddy.
-> Pages are in the buddy, they are no longer under control of the
driver.

To summarize the important parts

1. PG_offline has to remain set until the section is offline.

2. When we reach MEM_GOING_OFFLINE, it is too late. The pages were already detected as unmovable.

3. When we release pages that are supposed to be PG_offline to the buddy and offlining fails, we are in big trouble.

Quoted 3 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Quoted 4 lines

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

The driver still has to synchronize against onlining/offlining requests and track the state of the memory blocks.

Simple example: virtio-mem wants to try yo unplug a 4MB chunk. If the memory block is online, it has to use alloc contig_range(). If the memory block is offline (e.g., user space has not onlined it yet), it is sufficient to update metadata. It has to be aware of the state of the memory blocks and synchronize against onlining/offlining.

Quoted 3 lines

Or do I still see it too simplistically and the notifier is not a good place to handle the reference count?

Yes :) I could hack something via the memory isolation notifier (which is ugly) and MEM_GOING_OFFLINE. But if offlining fails, the pages are in the buddy and the driver lost control (which is one issue with PPC CMA implementation right now).

Again, I think routing pages via the buddy is the problematic part. (if offlining fails, but also the period where the pages are neither PageBuddy() nor PageOffline()))

Signature · 3 lines
Thanks,

David / dhildenb
#42 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 14 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Quoted 6 lines

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Thanks for thinking about this. I still believe it is problematic. And I don't like releasing "pages that should not be used" to the buddy. It comes with some problems if offlining fails (see below).

Quoted 4 lines

Let's say that you still keep a reference to your offlined pages and mark them offlined. That should make sure that no consumer of the pfn_to_online_page will touch the page's content nor the state. Now

pfn_to_online_page() does not check against PG_offline. (which is correct IMHO. As documented, PG_offline means "logically offline".

The memmap is valid, however the page content should not be accessed. (PG_logically_offline would have been ugly ;) ).

But that shouldn't be an issue right now.

Quoted 1 line

admin might want to offline/hotremove the whole memory block via sysfs.

The admin can only trigger offlining via sysfs. Hotremove is under control of the driver. (removing the whole memory block)

And with the current virtio-mem prototype, this works like a charm.

Quoted 3 lines

An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface

And here is the thing: The owner of the page does not have to do anything to untrack the page. I mean that's what a reference count of zero actually means - no direct reference.

I could emphasize how this is to be used in the documentation:

PageOffline() pages that have a reference count of 0 will be treated
like free pages when offlining pages, allowing the containing memory
block to get offlined. In case a driver wants to revive such a page, it has to synchronize against memory onlining/offlining (e.g., using memory notifiers) while incrementing the reference count. Also, a driver that relies in this feature is aware that re-onlining the memory will require to re-set the pages PageOffline() - e.g., via the online_page_callback_t.

Quoted 2 lines

for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not.

As I explained, this is too late and fragile. I post again what I posted before with some further explanations

__offline_pages() works like this:

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> offlining aborted. (see below on the memory isolation notifier)

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

3) scan_movable_pages() ...

4a) Memory offlining succeeded: memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) Memory offlining failed

    undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);

-> Offlining failed for whatever reason. Once we reach the notifier, the
pages are already back in use by the buddy.
-> Pages are in the buddy, they are no longer under control of the
driver.

To summarize the important parts

1. PG_offline has to remain set until the section is offline.

2. When we reach MEM_GOING_OFFLINE, it is too late. The pages were already detected as unmovable.

3. When we release pages that are supposed to be PG_offline to the buddy and offlining fails, we are in big trouble.

Quoted 3 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Quoted 4 lines

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

The driver still has to synchronize against onlining/offlining requests and track the state of the memory blocks.

Simple example: virtio-mem wants to try yo unplug a 4MB chunk. If the memory block is online, it has to use alloc contig_range(). If the memory block is offline (e.g., user space has not onlined it yet), it is sufficient to update metadata. It has to be aware of the state of the memory blocks and synchronize against onlining/offlining.

Quoted 3 lines

Or do I still see it too simplistically and the notifier is not a good place to handle the reference count?

Yes :) I could hack something via the memory isolation notifier (which is ugly) and MEM_GOING_OFFLINE. But if offlining fails, the pages are in the buddy and the driver lost control (which is one issue with PPC CMA implementation right now).

Again, I think routing pages via the buddy is the problematic part. (if offlining fails, but also the period where the pages are neither PageBuddy() nor PageOffline()))

Signature · 3 lines
Thanks,

David / dhildenb
#43 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 22 lines · David Hildenbrand
Quoted 14 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Quoted 6 lines

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Quoted 3 lines

Thanks for thinking about this. I still believe it is problematic. And I don't like releasing "pages that should not be used" to the buddy. It comes with some problems if offlining fails (see below).

Sure I will not be pushing if that turns out to be unfeasible. But I would like to explore that path before giving up on it.

[...]

Quoted 7 lines
Quoted 3 lines

An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface

And here is the thing: The owner of the page does not have to do anything to untrack the page. I mean that's what a reference count of zero actually means - no direct reference.

Will this be the case for other potential users of the similar/same mechanism? I thought that this would become a more spread mechanism.

[...]

Quoted 11 lines
Quoted 2 lines

for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not.

As I explained, this is too late and fragile. I post again what I posted before with some further explanations

__offline_pages() works like this:

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> offlining aborted. (see below on the memory isolation notifier)

I am assuming that has_unmovable_pages would skip over those pages. Your patch already does that, no?

Quoted 5 lines

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

Well, this is fully under control of the driver, no? Reference count shouldn't play any role here AFAIU.

Quoted 10 lines

3) scan_movable_pages() ...

4a) Memory offlining succeeded: memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) Memory offlining failed

    undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);

Doesn't this return pages back to buddy only when they were marked Buddy already?

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

One think that I would like to clarify because my previous email could be misleading a bit. You do not really have to drop the reference by releasing the page to the allocator (via put_page). You can also set the reference count to 0 explicitly. The driver is in control of the page, right? And that is the whole point I wanted to make. There is an explicit control via the reference count which is the standard way to control the struct page life cycle.

Anyway hooking into __put_page (which tends to be a hot path with something that is barely used on most systems) doesn't sound nice to me. This is the whole point which made me think about the whole reference count approach in the first place.

I do realize that the reference count doesn't solve the problem with onlining. Drivers still have to special case the onlining and that is something I really dislike but I do not have a good answer for.

Quoted 10 lines
Quoted 3 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Could you be more specific please?

Quoted 4 lines
Quoted 3 lines

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

Quoted 8 lines

The driver still has to synchronize against onlining/offlining requests and track the state of the memory blocks.

Simple example: virtio-mem wants to try yo unplug a 4MB chunk. If the memory block is online, it has to use alloc contig_range(). If the memory block is offline (e.g., user space has not onlined it yet), it is sufficient to update metadata. It has to be aware of the state of the memory blocks and synchronize against onlining/offlining.

Hmm, so the driver might offline a part of the memory which never got onlined? Is there really a sensible usecases for that?

Signature · 2 lines
Michal Hocko
SUSE Labs
#44 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 22 lines · David Hildenbrand
Quoted 14 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Quoted 6 lines

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Quoted 3 lines

Thanks for thinking about this. I still believe it is problematic. And I don't like releasing "pages that should not be used" to the buddy. It comes with some problems if offlining fails (see below).

Sure I will not be pushing if that turns out to be unfeasible. But I would like to explore that path before giving up on it.

[...]

Quoted 7 lines
Quoted 3 lines

An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface

And here is the thing: The owner of the page does not have to do anything to untrack the page. I mean that's what a reference count of zero actually means - no direct reference.

Will this be the case for other potential users of the similar/same mechanism? I thought that this would become a more spread mechanism.

[...]

Quoted 11 lines
Quoted 2 lines

for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not.

As I explained, this is too late and fragile. I post again what I posted before with some further explanations

__offline_pages() works like this:

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> offlining aborted. (see below on the memory isolation notifier)

I am assuming that has_unmovable_pages would skip over those pages. Your patch already does that, no?

Quoted 5 lines

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

Well, this is fully under control of the driver, no? Reference count shouldn't play any role here AFAIU.

Quoted 10 lines

3) scan_movable_pages() ...

4a) Memory offlining succeeded: memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

4b) Memory offlining failed

    undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
    memory_notify(MEM_CANCEL_OFFLINE, &arg);

Doesn't this return pages back to buddy only when they were marked Buddy already?

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

One think that I would like to clarify because my previous email could be misleading a bit. You do not really have to drop the reference by releasing the page to the allocator (via put_page). You can also set the reference count to 0 explicitly. The driver is in control of the page, right? And that is the whole point I wanted to make. There is an explicit control via the reference count which is the standard way to control the struct page life cycle.

Anyway hooking into __put_page (which tends to be a hot path with something that is barely used on most systems) doesn't sound nice to me. This is the whole point which made me think about the whole reference count approach in the first place.

I do realize that the reference count doesn't solve the problem with onlining. Drivers still have to special case the onlining and that is something I really dislike but I do not have a good answer for.

Quoted 10 lines
Quoted 3 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Could you be more specific please?

Quoted 4 lines
Quoted 3 lines

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

Quoted 8 lines

The driver still has to synchronize against onlining/offlining requests and track the state of the memory blocks.

Simple example: virtio-mem wants to try yo unplug a 4MB chunk. If the memory block is online, it has to use alloc contig_range(). If the memory block is offline (e.g., user space has not onlined it yet), it is sufficient to update metadata. It has to be aware of the state of the memory blocks and synchronize against onlining/offlining.

Hmm, so the driver might offline a part of the memory which never got onlined? Is there really a sensible usecases for that?

Signature · 2 lines
Michal Hocko
SUSE Labs
#45 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 23 lines · Michal Hocko
Quoted 22 lines · David Hildenbrand
Quoted 20 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 16.10.19 16:03, Michal Hocko wrote:

[...]

Quoted 10 lines
Quoted 5 lines

But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned to the host then it has ref count at 1. Once the page is returned to the guest for whatever reason it can free it to the system by clearing the offline state and put_page.

I think I explained how the reference count of 1 is problematic when wanting to offline the memory. After all that's the problem I try to solve: Keep PG_offline set until the memory is offline and make sure nobody will touch the page.

Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear after onlining. But I believe that you can achieve the same with an elevated reference count and have a more robust design as well.

Quoted 6 lines
Quoted 3 lines

Thanks for thinking about this. I still believe it is problematic. And I don't like releasing "pages that should not be used" to the buddy. It comes with some problems if offlining fails (see below).

Sure I will not be pushing if that turns out to be unfeasible. But I would like to explore that path before giving up on it.

Sure, I appreciate that. I wrapped my head around a lot of these things for a long time and came to the conclusion that it is complicated :)

[read below, I think it makes sense if you can summarize the idea you have that avoids releasing pages to the buddy - similar to this approach]

Quoted 6 lines

[...]

Quoted 4 lines
Quoted 3 lines

An elevated reference count would prevent offlining to finish. And I believe this is a good thing because the owner of the offline page might still need to do something to "untrack" that page. We have an interface

Quoted 6 lines
Quoted 3 lines

And here is the thing: The owner of the page does not have to do anything to untrack the page. I mean that's what a reference count of zero actually means - no direct reference.

Will this be the case for other potential users of the similar/same mechanism? I thought that this would become a more spread mechanism.

Anybody who wants to use that mechanism has to respect these rules. I am going to document that. It does not make sense for all balloon drivers that can simply implement page compaction. E.g., PPC CMM, it makes much more sense to switch to balloon compaction instead (because that's what it really is, a balloon driver).

I could imagine that HyperV might want to use that in the future. And it should be possible to make them play with the rules. They already use memory notifiers and online_page_callback_t.

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not.

As I explained, this is too late and fragile. I post again what I posted before with some further explanations

Quoted 8 lines
Quoted 5 lines

__offline_pages() works like this:

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> offlining aborted. (see below on the memory isolation notifier)

I am assuming that has_unmovable_pages would skip over those pages. Your patch already does that, no?

Yes, this works IFF the reference count is 0 (IOW, this patch). Not with a reference count of 1 (unless the pages are movable, like with balloon compaction).

Please note that we have other users that use PG_offline + refcount >= 1 (HyperV balloon, XEN). We should not affect these users (IOW, has_unmovable_pages() has to stop right there if we see one of these pages).

Quoted 9 lines
Quoted 6 lines

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

Well, this is fully under control of the driver, no? Reference count shouldn't play any role here AFAIU.

Yes, this is more a PG_offline issue. The reference count is an issue of reaching this call :) If we want to go via the buddy:

1. Clear PG_offline
2. Free page (gets set PG_buddy)

Between 1 and 2, a dumping tool could not exclude these pages and therefore try to read from these pages. That is an issue IFF we want to return the pages back to the buddy instead of doing what I propose here.

Quoted 2 lines
Quoted 2 lines

3) scan_movable_pages() ...

Please note that when we don't put the pages back to the buddy and don't implement something like I have in this patch, we'll loop/fail here. Especially if we have pages with PG_offline + refcount >= 1 .

Quoted 5 lines
Quoted 5 lines

4a) Memory offlining succeeded: memory_notify(MEM_OFFLINE, &arg);

Perfect, it worked. Sections are offline.

Quoted 7 lines
Quoted 4 lines

4b) Memory offlining failed

     undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
     memory_notify(MEM_CANCEL_OFFLINE, &arg);

Doesn't this return pages back to buddy only when they were marked Buddy already?

Yes, but I think you asked for evaluating what it would take to make the reference count stay at 1 and return the pages to the buddy. I tried to explain the pitfalls of that approach here :)

Can you clarify what exactly you have in mind right now? (how to make isolation work, how to make offlining work without putting pages back to the buddy, etc .). I have the feeling I am missing the one part that does not put the pages back to the buddy.

Quoted 3 lines

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that tries to avoid putting pages to the buddy.

Quoted 6 lines

One think that I would like to clarify because my previous email could be misleading a bit. You do not really have to drop the reference by releasing the page to the allocator (via put_page). You can also set the reference count to 0 explicitly. The driver is in control of the page, right? And that is the whole point I wanted to make. There is an

This is what virtio-mem does whenever it allocates a range using alloc_contig_range(). It uses page_ref_dec() instead of put_page()

/*
* Set a range of pages PG_offline and drop the reference. The dropped
* reference (0) and the flag allows isolation code to isolate thisrange
* and offline code to offline it.
*/
static void virtio_mem_set_fake_offline(unsigned long pfn,
unsigned int nr_pages)

{
       for (; nr_pages--; pfn++) {
               __SetPageOffline(pfn_to_page(pfn));
               page_ref_dec(pfn_to_page(pfn));
       }
}

The put_page() change is really only needed to avoid the mentioned race if somebody succeeds with a get_page_unless_zero(page) followed by a put_page() (bewlow).

Quoted 7 lines

explicit control via the reference count which is the standard way to control the struct page life cycle.

Anyway hooking into __put_page (which tends to be a hot path with something that is barely used on most systems) doesn't sound nice to me. This is the whole point which made me think about the whole reference count approach in the first place.

Again, the race I think that is possible

somebody: get_page_unless_zero(page)
virtio_mem: page_ref_dec(pfn_to_page(pfn)
somebody: put_page() -> straight to the buddy

I am not yet sure if this will really be a performance issue in put_page(). The branch predictor should do an excellent job here. (especially on systems without users)

Quoted 4 lines

I do realize that the reference count doesn't solve the problem with onlining. Drivers still have to special case the onlining and that is something I really dislike but I do not have a good answer for.

Yeah, especially due to HWPoison, this is to be solved differently. The PG_offline part might not be nice, but certainly easier to handle (online_page_callback_t).

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

Quoted 8 lines
Quoted 6 lines

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Could you be more specific please?

Let's take a look at how arch/powerpc/platforms/pseries/cmm.c handles it:

cmm_memory_isolate_cb() -> cmm_count_pages(arg):
- Memory Isolation notifier callback
- Count how many pages in the range to be isolated are in the ballooon
- This makes has_unmovable_pages() succeed. Pages can be isolated.

cmm_memory_cb -> cmm_mem_going_offline(arg):
- Memory notifier (online/offline)
- Release all pages in the range to the buddy

If offlining fails, the pages are now in the buddy, no longer in the balloon. MEM_CANCEL_ONLINE is too late, because the range is already unisolated again and the pages might be in use.

For CMM it might not be that bad, because it can actually "reloan" any pages. In contrast, virtio-mem cannot simply go ahead and reuse random memory in unplugged. Any access to these pages would be evil. Giving them back to the buddy is dangerous.

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

An advantage is that the driver has the full control over offlining and also you do not really have to track a new online request to do the right thing.

The driver still has to synchronize against onlining/offlining requests and track the state of the memory blocks.

Quoted 8 lines
Quoted 5 lines

Simple example: virtio-mem wants to try yo unplug a 4MB chunk. If the memory block is online, it has to use alloc contig_range(). If the memory block is offline (e.g., user space has not onlined it yet), it is sufficient to update metadata. It has to be aware of the state of the memory blocks and synchronize against onlining/offlining.

Hmm, so the driver might offline a part of the memory which never got onlined? Is there really a sensible usecases for that?

Yes there is in general a demand for unplugging offline memory. E.g., some configurable mode in the future could be that virtio-mem only unplugs offline memory, because this way user space can control which memory will actually get unplugged (compared to virtio-mem going ahead and trying to find online chunks to unplug).

Also, virtio-mem will be very careful with ZONE_MOVABLE memory. ZONE_MOVABLE memory first has to be offlined by user space before virtio-mem will go ahead and unplug parts (of the now offline memory). The reason is that I don't want unmovable pages (IOW via alloc_contig_range()) to end up in ZONE_MOVABLE - similar to gigantic pages. But that's yet another discussion :)

Thanks Michal!

Signature · 3 lines
Thanks,

David / dhildenb
#46 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 3 lines · David Hildenbrand
Quoted 2 lines · Michal Hocko
Quoted 1 line · David Hildenbrand

On 18.10.19 10:15, Michal Hocko wrote:

[...]

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 2 lines

for that - MEM_GOING_OFFLINE notification. This sounds like a good place for the driver to decide whether it is safe to let the page go or not.

As I explained, this is too late and fragile. I post again what I posted before with some further explanations

__offline_pages() works like this:

1) start_isolate_page_range()
-> offline pages with a reference count of one will be detected as unmovable -> offlining aborted. (see below on the memory isolation notifier)

Quoted 6 lines
Quoted 2 lines

I am assuming that has_unmovable_pages would skip over those pages. Your patch already does that, no?

Yes, this works IFF the reference count is 0 (IOW, this patch). Not with a reference count of 1 (unless the pages are movable, like with balloon compaction).

I am pretty sure that has_unmovable_pages can special these pages regardless of the reference count for the memory hotplug. We already do that for HWPoison pages.

Quoted 3 lines

Please note that we have other users that use PG_offline + refcount >= 1 (HyperV balloon, XEN). We should not affect these users (IOW, has_unmovable_pages() has to stop right there if we see one of these pages).

OK, this is exactly what I was worried about. I can see why you might want to go an easier way and rule those users out but wouldn't be it actually more reasonable to explicitly request PageOffline users to implement MEM_GOING_OFFLINE and prepare their offlined pages for the offlining operation or fail right there if that is not possible. If you fail right there during the isolation phase then there is no way to allow the offlining to proceed from that context.

Quoted 12 lines
Quoted 8 lines
Quoted 5 lines

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

Well, this is fully under control of the driver, no? Reference count shouldn't play any role here AFAIU.

Yes, this is more a PG_offline issue. The reference count is an issue of reaching this call :) If we want to go via the buddy:

Quoted 6 lines

1. Clear PG_offline
2. Free page (gets set PG_buddy)

Between 1 and 2, a dumping tool could not exclude these pages and therefore try to read from these pages. That is an issue IFF we want to return the pages back to the buddy instead of doing what I propose here.

If the driver is going to free page to the allocator then it would have to claim the page back and so it is usable again. If it cannot free it then it would simply set the reference count to 0. It can even keep the PG_offline if necessary although I have to admit I am not really sure it is necessary.

Quoted 5 lines
Quoted 1 line
Quoted 1 line

3) scan_movable_pages() ...

Please note that when we don't put the pages back to the buddy and don't implement something like I have in this patch, we'll loop/fail here. Especially if we have pages with PG_offline + refcount >= 1 .

You should have your reference count 0 at this stage as it is after MEM_GOING_OFFLINE.

Quoted 5 lines
Quoted 2 lines

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that tries to avoid putting pages to the buddy.

Yeah, set_page_count(page, 0) if you do not want to release that page from the notifier context to reflect that the page is ok to be offlined with the rest.

[...]

Quoted 8 lines
Quoted 7 lines

explicit control via the reference count which is the standard way to control the struct page life cycle.

Anyway hooking into __put_page (which tends to be a hot path with something that is barely used on most systems) doesn't sound nice to me. This is the whole point which made me think about the whole reference count approach in the first place.

Quoted 5 lines

Again, the race I think that is possible

somebody: get_page_unless_zero(page)
virtio_mem: page_ref_dec(pfn_to_page(pfn)
somebody: put_page() -> straight to the buddy

Who is that somebody? I thought that it is only the owner/driver to have a control over the page. Also the above is not possible as long as the owner/driver keeps a reference to the PageOffline page throughout the time it is marked that way.

[...]

Quoted 24 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

Quoted 8 lines
Quoted 6 lines

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Could you be more specific please?

Let's take a look at how arch/powerpc/platforms/pseries/cmm.c handles it:

cmm_memory_isolate_cb() -> cmm_count_pages(arg):
- Memory Isolation notifier callback
- Count how many pages in the range to be isolated are in the ballooon
- This makes has_unmovable_pages() succeed. Pages can be isolated.

cmm_memory_cb -> cmm_mem_going_offline(arg):
- Memory notifier (online/offline)
- Release all pages in the range to the buddy

Quoted 8 lines

If offlining fails, the pages are now in the buddy, no longer in the balloon. MEM_CANCEL_ONLINE is too late, because the range is already unisolated again and the pages might be in use.

For CMM it might not be that bad, because it can actually "reloan" any pages. In contrast, virtio-mem cannot simply go ahead and reuse random memory in unplugged. Any access to these pages would be evil. Giving them back to the buddy is dangerous.

Thanks, I was not aware of that code. But from what I understood this is an outright bug in this code because cmm_mem_going_offline releases pages to the buddy allocator which is something that is not recoverable on a later failure.

Signature · 2 lines
Michal Hocko
SUSE Labs
#47 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 11 lines
Quoted 3 lines

Please note that we have other users that use PG_offline + refcount >= 1 (HyperV balloon, XEN). We should not affect these users (IOW, has_unmovable_pages() has to stop right there if we see one of these pages).

OK, this is exactly what I was worried about. I can see why you might want to go an easier way and rule those users out but wouldn't be it actually more reasonable to explicitly request PageOffline users to implement MEM_GOING_OFFLINE and prepare their offlined pages for the offlining operation or fail right there if that is not possible. If you fail right there during the isolation phase then there is no way to allow the offlining to proceed from that context.

I am not sure I agree. But let's discuss the details. See below.

Quoted 20 lines
Quoted 13 lines
Quoted 9 lines
Quoted 6 lines

2) memory_notify(MEM_GOING_OFFLINE, &arg);
-> Here, we could release all pages to the buddy, clearing PG_offline -> PF_offline must not be cleared so dumping tools will not touch
these pages. There is a time where pages are !PageBuddy() and !PageOffline().

Well, this is fully under control of the driver, no? Reference count shouldn't play any role here AFAIU.

Yes, this is more a PG_offline issue. The reference count is an issue of reaching this call :) If we want to go via the buddy:

Quoted 6 lines

1. Clear PG_offline
2. Free page (gets set PG_buddy)

Between 1 and 2, a dumping tool could not exclude these pages and therefore try to read from these pages. That is an issue IFF we want to return the pages back to the buddy instead of doing what I propose here.

Quoted 5 lines

If the driver is going to free page to the allocator then it would have to claim the page back and so it is usable again. If it cannot free it then it would simply set the reference count to 0. It can even keep the PG_offline if necessary although I have to admit I am not really sure it is necessary.

Yes it is necessary to keep PG_offline set to avoid anybody touching the page until the section is offline. (especially, dumping tools) But that's another discussion. The important part is to not go via the buddy.

Quoted 13 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

3) scan_movable_pages() ...

Please note that when we don't put the pages back to the buddy and don't implement something like I have in this patch, we'll loop/fail here. Especially if we have pages with PG_offline + refcount >= 1 .

You should have your reference count 0 at this stage as it is after MEM_GOING_OFFLINE.

Quoted 3 lines
Quoted 2 lines

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

Quoted 7 lines
Quoted 2 lines

The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that tries to avoid putting pages to the buddy.

Yeah, set_page_count(page, 0) if you do not want to release that page from the notifier context to reflect that the page is ok to be offlined with the rest.

I neither see how you deal with __test_page_isolated_in_pageblock() nor with __offline_isolated_pages(). Sorry, but what I read is incomplete and you probably have a full proposal in your head. Please read below how I think you want to solve it.

Quoted 11 lines
Quoted 11 lines
Quoted 8 lines

explicit control via the reference count which is the standard way to control the struct page life cycle.

Anyway hooking into __put_page (which tends to be a hot path with something that is barely used on most systems) doesn't sound nice to me. This is the whole point which made me think about the whole reference count approach in the first place.

Again, the race I think that is possible

Quoted 9 lines
Quoted 3 lines

somebody: get_page_unless_zero(page)
virtio_mem: page_ref_dec(pfn_to_page(pfn)
somebody: put_page() -> straight to the buddy

Who is that somebody? I thought that it is only the owner/driver to have a control over the page. Also the above is not possible as long as the owner/driver keeps a reference to the PageOffline page throughout the time it is marked that way.

I was reading

include/linux/mm_types.h:

"If you want to use the refcount field, it must be used in such a way
that other CPUs temporarily incrementing and then decrementing the refcount does not cause problems"

And that made me think "anybody can go ahead and try get_page_unless_zero()".

If I am missing something here and this can indeed not happen (e.g., because PageOffline() pages are never mapped to user space), then I'll happily remove this code.

Quoted 34 lines
Quoted 25 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

If you can let the page go then just drop the reference count. The page is isolated already by that time. If you cannot let it go for whatever reason you can fail the offlining.

Quoted 8 lines
Quoted 6 lines

We do have one hack in current MM code, which is the memory isolation notifier only used by CMM on PPC. It allows to "cheat" has_unmovable_pages() to skip over unmovable pages. But quite frankly, I rather want to get rid of that crap (something I am working on right now) than introduce new users. This stuff is racy as hell and for CMM, if memory offlining fails, the ballooned pages are suddenly part of the buddy. Fragile.

Could you be more specific please?

Let's take a look at how arch/powerpc/platforms/pseries/cmm.c handles it:

cmm_memory_isolate_cb() -> cmm_count_pages(arg):
- Memory Isolation notifier callback
- Count how many pages in the range to be isolated are in the ballooon
- This makes has_unmovable_pages() succeed. Pages can be isolated.

cmm_memory_cb -> cmm_mem_going_offline(arg):
- Memory notifier (online/offline)
- Release all pages in the range to the buddy

Quoted 8 lines

If offlining fails, the pages are now in the buddy, no longer in the balloon. MEM_CANCEL_ONLINE is too late, because the range is already unisolated again and the pages might be in use.

For CMM it might not be that bad, because it can actually "reloan" any pages. In contrast, virtio-mem cannot simply go ahead and reuse random memory in unplugged. Any access to these pages would be evil. Giving them back to the buddy is dangerous.

Quoted 5 lines

Thanks, I was not aware of that code. But from what I understood this is an outright bug in this code because cmm_mem_going_offline releases pages to the buddy allocator which is something that is not recoverable on a later failure.

Yes, and that should be gone if we switch to balloon compaction.

Let's recap what I suggest:

"PageOffline() pages that have a reference count of 0 will be treated
like free pages when offlining pages, allowing the containing memory block to get offlined. In case a driver wants to revive such a page, it has to synchronize against memory onlining/offlining (e.g., using memory notifiers) while incrementing the reference count. Also, a driver that relies in this feature is aware that re-onlining the memory will require to re-set the pages PageOffline() - e.g., via the online_page_callback_t."

a) has_unmovable_pages() already skips over pages with a refcount of zero. The code I add to not skip over these pages when !MEMORY_OFFLINE is a pure optimization to fail early when trying to allocate from that range.

b) __test_page_isolated_in_pageblock() is modified to skip over PageOffline() pages with a refcount of zero

c) __offline_isolated_pages() is modified to skip over PageOffline() pages with a refcount of zero

d) __put_page() is modified to not return pages to the buddy in any case as a safety net. We might be able to get rid of that.

What I think you suggest:

a) has_unmovable_pages() skips over all PageOffline() pages.
This results in a lot of false negatives when trying to offline. Might be ok.

b) The driver decrements the reference count of the PageOffline pages in MEM_GOING_OFFLINE.

c) The driver increments the reference count of the PageOffline pages in MEM_CANCEL_OFFLINE. One issue might be that the pages are no longer isolated once we get that call. Might be ok.

d) How to make __test_page_isolated_in_pageblock() succeed? Like I propose in this patch (PageOffline() + refcount == 0)?

e) How to make __offline_isolated_pages() succeed? Like I propose in this patch (PageOffline() + refcount == 0)?

In summary, is what you suggest simply delaying setting the reference count to 0 in MEM_GOING_OFFLINE instead of right away when the driver unpluggs the pages? What's the big benefit you see and I fail to see?

Signature · 3 lines
Thanks,

David / dhildenb
#48 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On Tue 22-10-19 16:02:09, David Hildenbrand wrote:
[...]

Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that tries to avoid putting pages to the buddy.

Yeah, set_page_count(page, 0) if you do not want to release that page from the notifier context to reflect that the page is ok to be offlined with the rest.

Quoted 4 lines

I neither see how you deal with __test_page_isolated_in_pageblock() nor with __offline_isolated_pages(). Sorry, but what I read is incomplete and you probably have a full proposal in your head. Please read below how I think you want to solve it.

Yeah, sorry that I am throwing incomplete ideas at you. I am just trying to really nail down how to deal with reference counting here because it is an important aspect.

Quoted 28 lines
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines

explicit control via the reference count which is the standard way to control the struct page life cycle.

Anyway hooking into __put_page (which tends to be a hot path with something that is barely used on most systems) doesn't sound nice to me. This is the whole point which made me think about the whole reference count approach in the first place.

Again, the race I think that is possible

Quoted 9 lines
Quoted 3 lines

somebody: get_page_unless_zero(page)
virtio_mem: page_ref_dec(pfn_to_page(pfn)
somebody: put_page() -> straight to the buddy

Who is that somebody? I thought that it is only the owner/driver to have a control over the page. Also the above is not possible as long as the owner/driver keeps a reference to the PageOffline page throughout the time it is marked that way.

I was reading

include/linux/mm_types.h:

"If you want to use the refcount field, it must be used in such a way
that other CPUs temporarily incrementing and then decrementing the refcount does not cause problems"

Quoted 5 lines

And that made me think "anybody can go ahead and try get_page_unless_zero()".

If I am missing something here and this can indeed not happen (e.g., because PageOffline() pages are never mapped to user space), then I'll happily remove this code.

The point is that if the owner of the page is holding the only reference to the page then it is clear that nothing like that's happened.

[...]

Quoted 9 lines

Let's recap what I suggest:

"PageOffline() pages that have a reference count of 0 will be treated
like free pages when offlining pages, allowing the containing memory block to get offlined. In case a driver wants to revive such a page, it has to synchronize against memory onlining/offlining (e.g., using memory notifiers) while incrementing the reference count. Also, a driver that relies in this feature is aware that re-onlining the memory will require to re-set the pages PageOffline() - e.g., via the online_page_callback_t."

OK

[...]

Quoted 2 lines

d) __put_page() is modified to not return pages to the buddy in any case as a safety net. We might be able to get rid of that.

I do not like exactly this part

Quoted 7 lines

What I think you suggest:

a) has_unmovable_pages() skips over all PageOffline() pages.
This results in a lot of false negatives when trying to offline. Might be ok.

b) The driver decrements the reference count of the PageOffline pages in MEM_GOING_OFFLINE.

Well, driver should make the page unreferenced or fail. What is done really depends on the specific driver

Quoted 3 lines

c) The driver increments the reference count of the PageOffline pages in MEM_CANCEL_OFFLINE. One issue might be that the pages are no longer isolated once we get that call. Might be ok.

Only previous PageBuddy pages are returned to the allocator IIRC. Mostly because of MovablePage()

Quoted 2 lines

d) How to make __test_page_isolated_in_pageblock() succeed? Like I propose in this patch (PageOffline() + refcount == 0)?

Yep

Quoted 2 lines

e) How to make __offline_isolated_pages() succeed? Like I propose in this patch (PageOffline() + refcount == 0)?

Simply skip over PageOffline pages. Reference count should never be != 0 at this stage.

Quoted 2 lines

In summary, is what you suggest simply delaying setting the reference count to 0 in MEM_GOING_OFFLINE instead of right away when the driver unpluggs the pages?

Yes

Quoted 1 line

What's the big benefit you see and I fail to see?

Aparat from no hooks into __put_page it is also an explicit control over the page via reference counting. Do you see any downsides?

Signature · 2 lines
Michal Hocko
SUSE Labs
#49 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 13 lines · Michal Hocko

On Tue 22-10-19 16:02:09, David Hildenbrand wrote:
[...]

Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that tries to avoid putting pages to the buddy.

Yeah, set_page_count(page, 0) if you do not want to release that page from the notifier context to reflect that the page is ok to be offlined with the rest.

Quoted 8 lines
Quoted 4 lines

I neither see how you deal with __test_page_isolated_in_pageblock() nor with __offline_isolated_pages(). Sorry, but what I read is incomplete and you probably have a full proposal in your head. Please read below how I think you want to solve it.

Yeah, sorry that I am throwing incomplete ideas at you. I am just trying to really nail down how to deal with reference counting here because it is an important aspect.

I think we collected all the missing pieces now :) Thanks!

[...]

Quoted 9 lines
Quoted 9 lines

I was reading

include/linux/mm_types.h:

"If you want to use the refcount field, it must be used in such a way
that other CPUs temporarily incrementing and then decrementing the refcount does not cause problems"

Quoted 8 lines
Quoted 5 lines

And that made me think "anybody can go ahead and try get_page_unless_zero()".

If I am missing something here and this can indeed not happen (e.g., because PageOffline() pages are never mapped to user space), then I'll happily remove this code.

The point is that if the owner of the page is holding the only reference to the page then it is clear that nothing like that's happened.

Right, and I think the race I described won't happen in practice. Nobody should be trying to do a get_page_unless_zero() on random pages that are not even mapped to user space. I was (as so often) very careful :)

Quoted 10 lines
Quoted 9 lines

Let's recap what I suggest:

"PageOffline() pages that have a reference count of 0 will be treated
like free pages when offlining pages, allowing the containing memory block to get offlined. In case a driver wants to revive such a page, it has to synchronize against memory onlining/offlining (e.g., using memory notifiers) while incrementing the reference count. Also, a driver that relies in this feature is aware that re-onlining the memory will require to re-set the pages PageOffline() - e.g., via the online_page_callback_t."

Quoted 7 lines

OK

[...]

Quoted 2 lines

d) __put_page() is modified to not return pages to the buddy in any case as a safety net. We might be able to get rid of that.

I do not like exactly this part

Yeah, and I think I can drop it from this patch.

Quoted 22 lines
Quoted 8 lines

What I think you suggest:

a) has_unmovable_pages() skips over all PageOffline() pages.
This results in a lot of false negatives when trying to offline. Might be ok.

b) The driver decrements the reference count of the PageOffline pages in MEM_GOING_OFFLINE.

Well, driver should make the page unreferenced or fail. What is done really depends on the specific driver

Quoted 3 lines

c) The driver increments the reference count of the PageOffline pages in MEM_CANCEL_OFFLINE. One issue might be that the pages are no longer isolated once we get that call. Might be ok.

Only previous PageBuddy pages are returned to the allocator IIRC. Mostly because of MovablePage()

Quoted 2 lines

d) How to make __test_page_isolated_in_pageblock() succeed?

    Like I propose in this patch (PageOffline() + refcount == 0)?
Quoted 7 lines

Yep

Quoted 2 lines

e) How to make __offline_isolated_pages() succeed?

    Like I propose in this patch (PageOffline() + refcount == 0)?

Simply skip over PageOffline pages. Reference count should never be != 0 at this stage.

Right, that should be guaranteed by d). (as long as people play by the rules) Same applies to my current patch.

Quoted 10 lines
Quoted 3 lines

In summary, is what you suggest simply delaying setting the reference count to 0 in MEM_GOING_OFFLINE instead of right away when the driver unpluggs the pages?

Yes

Quoted 1 line

What's the big benefit you see and I fail to see?

Aparat from no hooks into __put_page it is also an explicit control over the page via reference counting. Do you see any downsides?

The only downside I see is that we get more false negatives on has_unmovable_pages(), eventually resulting in the offlining stage after isolation to loop forever (as some PageOffline() pages are not movable (especially, XEN balloon, HyperV balloon), there won't be progress).

I somewhat don't like forcing everybody that uses PageOffline() (especially all users of balloon compaction) to implement memory notifiers just to avoid that. Maybe, we even want to use PageOffline() in the future in the core (e.g., for memory holes instead of PG_reserved or similar).

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#50 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to Michal Hocko
Quoted 13 lines · Michal Hocko

On Tue 22-10-19 16:02:09, David Hildenbrand wrote:
[...]

Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step.

The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that tries to avoid putting pages to the buddy.

Yeah, set_page_count(page, 0) if you do not want to release that page from the notifier context to reflect that the page is ok to be offlined with the rest.

Quoted 8 lines
Quoted 4 lines

I neither see how you deal with __test_page_isolated_in_pageblock() nor with __offline_isolated_pages(). Sorry, but what I read is incomplete and you probably have a full proposal in your head. Please read below how I think you want to solve it.

Yeah, sorry that I am throwing incomplete ideas at you. I am just trying to really nail down how to deal with reference counting here because it is an important aspect.

I think we collected all the missing pieces now :) Thanks!

[...]

Quoted 9 lines
Quoted 9 lines

I was reading

include/linux/mm_types.h:

"If you want to use the refcount field, it must be used in such a way
that other CPUs temporarily incrementing and then decrementing the refcount does not cause problems"

Quoted 8 lines
Quoted 5 lines

And that made me think "anybody can go ahead and try get_page_unless_zero()".

If I am missing something here and this can indeed not happen (e.g., because PageOffline() pages are never mapped to user space), then I'll happily remove this code.

The point is that if the owner of the page is holding the only reference to the page then it is clear that nothing like that's happened.

Right, and I think the race I described won't happen in practice. Nobody should be trying to do a get_page_unless_zero() on random pages that are not even mapped to user space. I was (as so often) very careful :)

Quoted 10 lines
Quoted 9 lines

Let's recap what I suggest:

"PageOffline() pages that have a reference count of 0 will be treated
like free pages when offlining pages, allowing the containing memory block to get offlined. In case a driver wants to revive such a page, it has to synchronize against memory onlining/offlining (e.g., using memory notifiers) while incrementing the reference count. Also, a driver that relies in this feature is aware that re-onlining the memory will require to re-set the pages PageOffline() - e.g., via the online_page_callback_t."

Quoted 7 lines

OK

[...]

Quoted 2 lines

d) __put_page() is modified to not return pages to the buddy in any case as a safety net. We might be able to get rid of that.

I do not like exactly this part

Yeah, and I think I can drop it from this patch.

Quoted 22 lines
Quoted 8 lines

What I think you suggest:

a) has_unmovable_pages() skips over all PageOffline() pages.
This results in a lot of false negatives when trying to offline. Might be ok.

b) The driver decrements the reference count of the PageOffline pages in MEM_GOING_OFFLINE.

Well, driver should make the page unreferenced or fail. What is done really depends on the specific driver

Quoted 3 lines

c) The driver increments the reference count of the PageOffline pages in MEM_CANCEL_OFFLINE. One issue might be that the pages are no longer isolated once we get that call. Might be ok.

Only previous PageBuddy pages are returned to the allocator IIRC. Mostly because of MovablePage()

Quoted 2 lines

d) How to make __test_page_isolated_in_pageblock() succeed?

    Like I propose in this patch (PageOffline() + refcount == 0)?
Quoted 7 lines

Yep

Quoted 2 lines

e) How to make __offline_isolated_pages() succeed?

    Like I propose in this patch (PageOffline() + refcount == 0)?

Simply skip over PageOffline pages. Reference count should never be != 0 at this stage.

Right, that should be guaranteed by d). (as long as people play by the rules) Same applies to my current patch.

Quoted 10 lines
Quoted 3 lines

In summary, is what you suggest simply delaying setting the reference count to 0 in MEM_GOING_OFFLINE instead of right away when the driver unpluggs the pages?

Yes

Quoted 1 line

What's the big benefit you see and I fail to see?

Aparat from no hooks into __put_page it is also an explicit control over the page via reference counting. Do you see any downsides?

The only downside I see is that we get more false negatives on has_unmovable_pages(), eventually resulting in the offlining stage after isolation to loop forever (as some PageOffline() pages are not movable (especially, XEN balloon, HyperV balloon), there won't be progress).

I somewhat don't like forcing everybody that uses PageOffline() (especially all users of balloon compaction) to implement memory notifiers just to avoid that. Maybe, we even want to use PageOffline() in the future in the core (e.g., for memory holes instead of PG_reserved or similar).

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#51 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 11 lines · David Hildenbrand
Quoted 1 line

Do you see any downsides?

The only downside I see is that we get more false negatives on has_unmovable_pages(), eventually resulting in the offlining stage after isolation to loop forever (as some PageOffline() pages are not movable (especially, XEN balloon, HyperV balloon), there won't be progress).

I somewhat don't like forcing everybody that uses PageOffline() (especially all users of balloon compaction) to implement memory notifiers just to avoid that. Maybe, we even want to use PageOffline() in the future in the core (e.g., for memory holes instead of PG_reserved or similar).

There is only a handful of those and we need to deal with them anyway. If you do not want to enforce them to create their own notifiers then we can accomodate the hotplug code. __test_page_isolated_in_pageblock resp. the call chain up can distinguish temporary and permanent failures (EAGAIN vs. EBUSY). The current state when we always return EBUSY and keep retrying for ever is not optimal at all, right? A referenced PageOffline could be an example of EBUSY all other failures where we are effectively waiting for pages to get freed finaly would be EAGAIN.

It is a bit late in the process because a large portion of the work has been done already but this doesn't sound like something to lose sleep over.

Signature · 2 lines
Michal Hocko
SUSE Labs
#52 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

In reply to David Hildenbrand
Quoted 11 lines · David Hildenbrand
Quoted 1 line

Do you see any downsides?

The only downside I see is that we get more false negatives on has_unmovable_pages(), eventually resulting in the offlining stage after isolation to loop forever (as some PageOffline() pages are not movable (especially, XEN balloon, HyperV balloon), there won't be progress).

I somewhat don't like forcing everybody that uses PageOffline() (especially all users of balloon compaction) to implement memory notifiers just to avoid that. Maybe, we even want to use PageOffline() in the future in the core (e.g., for memory holes instead of PG_reserved or similar).

There is only a handful of those and we need to deal with them anyway. If you do not want to enforce them to create their own notifiers then we can accomodate the hotplug code. __test_page_isolated_in_pageblock resp. the call chain up can distinguish temporary and permanent failures (EAGAIN vs. EBUSY). The current state when we always return EBUSY and keep retrying for ever is not optimal at all, right? A referenced PageOffline could be an example of EBUSY all other failures where we are effectively waiting for pages to get freed finaly would be EAGAIN.

It is a bit late in the process because a large portion of the work has been done already but this doesn't sound like something to lose sleep over.

Signature · 2 lines
Michal Hocko
SUSE Labs
#53 lore raw

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

Quoted 8 lines · Michal Hocko
Quoted 7 lines · David Hildenbrand
Quoted 1 line

Do you see any downsides?

The only downside I see is that we get more false negatives on has_unmovable_pages(), eventually resulting in the offlining stage after isolation to loop forever (as some PageOffline() pages are not movable (especially, XEN balloon, HyperV balloon), there won't be progress).

Quoted 8 lines
Quoted 4 lines

I somewhat don't like forcing everybody that uses PageOffline() (especially all users of balloon compaction) to implement memory notifiers just to avoid that. Maybe, we even want to use PageOffline() in the future in the core (e.g., for memory holes instead of PG_reserved or similar).

There is only a handful of those and we need to deal with them anyway. If you do not want to enforce them to create their own notifiers then we can accomodate the hotplug code. __test_page_isolated_in_pageblock resp.

Yeah, I would prefer offlining code to be able to deal with that without notifier changes for all users.

Quoted 3 lines

the call chain up can distinguish temporary and permanent failures (EAGAIN vs. EBUSY). The current state when we always return EBUSY and keep retrying for ever is not optimal at all, right? A referenced PageOffline

Very right!

Quoted 2 lines

could be an example of EBUSY all other failures where we are effectively waiting for pages to get freed finaly would be EAGAIN.

We have to watch out for PageOffline() pages that are actually movable (balloon compaction). But that doesn't sound too hard.

Quoted 5 lines

It is a bit late in the process because a large portion of the work has been done already but this doesn't sound like something to lose sleep over.

Right. I'll look into that to find out if this would work. And see if I can reproduce what I described at all (theoretical thoughts) :)

Again, thanks for looking into this Michal!

Signature · 3 lines
Thanks,

David / dhildenb
#54 lore raw

RFC PATCH 1/1 mm: Allow to offline unmovable PageOffline() pages if the driver agrees

virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these pages will never get accessed (e.g., when dumping). The pages should not be handed back to the buddy (which would require clearing PageOffline() and result in issues if offlining fails and the pages are suddenly in the buddy).

Let's allow to do that by allowing to isolate any PageOffline() page when offlining. This way, we can reach the memory hotplug notifier MEM_GOING_OFFLINE, where the driver can signal that he is fine with offlining this page by dropping its reference count. PageOffline() pages with a reference count of 0 can then be skipped when offlining the pages (like if they were free, however they are not in the buddy).

Anybody who uses PageOffline() pages and does not agree to offline them (e.g., Hyper-V balloon, XEN balloon, VMWare balloon for 2MB pages) will not decrement the reference count and make offlining fail when trying to migrate such an unmovable page. So there should be no observerable change. Same applies to balloon compaction users (movable PageOffline() pages), the pages will simply be migrated.

Note 1: If offlining fails, a driver has to increment the reference
count again in MEM_CANCEL_OFFLINE.

Note 2: A driver that makes use of this has to be aware that re-onlining
the memory block has to be handled by hooking into onlining code (online_page_callback_t), resetting the page PageOffline() and not giving them to the buddy.

Cc Andrew Morton <[email protected]>
Cc Juergen Gross <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Alexander Duyck <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Anthony Yznaga <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Dan Williams <[email protected]>
Cc Anshuman Khandual <[email protected]>
Cc Qian Cai <[email protected]>
Cc Pingfan Liu <[email protected]>

---

Michal,

this is the approach where we allow has_unmovable_pages() to succeed to reach MEM_GOING_OFFLINE and fail later in case the driver did not agree.

Thoughts?

---
 include/linux/page-flags.h | 10 ++++++++++
 mm/memory_hotplug.c        | 41 ++++++++++++++++++++++++++++----------
 mm/page_alloc.c            | 24 ++++++++++++++++++++++
 mm/page_isolation.c        |  9 +++++++++
 4 files changed, 74 insertions(+), 10 deletions(-)
include/linux/page-flags.h +10 -0
index 3b8e5c5f7e1f..4897cc585af6 100644
@@ -757,6 +757,16 @@ PAGE_TYPE_OPS(Buddy, buddy)
  * not onlined when onlining the section).
  * The content of these pages is effectively stale. Such pages should not
  * be touched (read/write/dump/save) except by their owner.
+ *
+ * If a driver wants to allow to offline unmovable PageOffline() pages without
+ * putting them back to the buddy, it can do so via the memory notifier by
+ * decrementing the reference count in MEM_GOING_OFFLINE and incrementing the
+ * reference count in MEM_CANCEL_OFFLINE. When offlining, the PageOffline()
+ * pages (now with a reference count of zero) are treated like free pages,
+ * allowing the containing memory block to get offlined. A driver that
+ * relies on this feature is aware that re-onlining the memory block will
+ * require to re-set the pages PageOffline() and not giving them to the
+ * buddy via online_page_callback_t.
  */
 PAGE_TYPE_OPS(Offline, offline)
 
mm/memory_hotplug.c +31 -10

81 lines not on this page · raw message

mm/page_alloc.c +24 -0
index e2b0bdfdd586..1594f480532a 100644
@@ -8273,6 +8273,19 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
 		if ((flags & MEMORY_OFFLINE) && PageHWPoison(page))
 			continue;
 
+		/*
+		 * We treat all PageOffline() pages as movable when offlining
+		 * to give drivers a chance to decrement their reference count
+		 * in MEM_GOING_OFFLINE in order to signalize that these pages
+		 * can be offlined as there are no direct references anymore.
+		 * For actually unmovable PageOffline() where the driver does
+		 * not support this, we will fail later when trying to actually
+		 * move these pages that still have a reference count > 0.
+		 * (false negatives in this function only)
+		 */
+		if ((flags & MEMORY_OFFLINE) && PageOffline(page))
+			continue;
+
 		if (__PageMovable(page))
 			continue;
 
@@ -8702,6 +8715,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)
 			offlined_pages++;
 			continue;
 		}
+		/*
+		 * At this point all remaining PageOffline() pages have a
+		 * reference count of 0 and can simply be skipped.
+		 */
+		if (PageOffline(page)) {
+			BUG_ON(page_count(page));
+			BUG_ON(PageBuddy(page));
+			pfn++;
+			offlined_pages++;
+			continue;
+		}
 
 		BUG_ON(page_count(page));
 		BUG_ON(!PageBuddy(page));
mm/page_isolation.c +9 -0
index 04ee1663cdbe..43b4dabfedc8 100644
@@ -170,6 +170,7 @@ __first_valid_page(unsigned long pfn, unsigned long nr_pages)
  *			a bit mask)
  *			MEMORY_OFFLINE - isolate to offline (!allocate) memory
  *					 e.g., skip over PageHWPoison() pages
+ *					 and PageOffline() pages.
  *			REPORT_FAILURE - report details about the failure to
  *			isolate the range
  *
@@ -278,6 +279,14 @@ __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn,
 		else if ((flags & MEMORY_OFFLINE) && PageHWPoison(page))
 			/* A HWPoisoned page cannot be also PageBuddy */
 			pfn++;
+		else if ((flags & MEMORY_OFFLINE) && PageOffline(page) &&
+			 !page_count(page))
+			/*
+			 * The responsible driver agreed to offline
+			 * PageOffline() pages by dropping its reference in
+			 * MEM_GOING_OFFLINE.
+			 */
+			pfn++;
 		else
 			break;
 	}
Signature · 1 line
2.21.0
#55 lore raw

RFC PATCH v3 7/9 virtio-mem: Allow to offline partially unplugged memory blocks

In reply to David Hildenbrand

Dropping the reference count of PageOffline() pages allows offlining code to skip them. However, we also have to convert PG_reserved to another flag - let's use PG_dirty - so has_unmovable_pages() will properly handle them. PG_reserved pages get detected as unmovable right away.

We need the flag to see if we are onlining pages the first time, or if we allocated them via alloc_contig_range().

Properly take care of offlining code also modifying the stats and special handling in case the driver gets unloaded.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/virtio_mem.c | 102 ++++++++++++++++++++++++++++++++----
 1 file changed, 92 insertions(+), 10 deletions(-)
drivers/virtio/virtio_mem.c +92 -10
index 91052a37d10d..9cb31459b211 100644
@@ -561,6 +561,30 @@ static void virtio_mem_notify_online(struct virtio_mem *vm, unsigned long mb_id,
 		virtio_mem_retry(vm);
 }
 
+/*
+ * When we unplug subblocks, we already modify stats (e.g., subtract them
+ * from totalram_pages). Offlining code will modify the stats, too. So
+ * properly fixup the stats when GOING_OFFLINE and revert that when
+ * CANCEL_OFFLINE.
+ */
+static void virtio_mem_mb_going_offline_fixup_stats(struct virtio_mem *vm,
+						    unsigned long mb_id,
+						    bool cancel)
+{
+	const unsigned long nr_pages = PFN_DOWN(vm->subblock_size);
+	int sb_id;
+
+	for (sb_id = 0; sb_id < vm->nb_sb_per_mb; sb_id++) {
+		if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1))
+			continue;
+
+		if (cancel)
+			totalram_pages_add(-nr_pages);
+		else
+			totalram_pages_add(nr_pages);
+	}
+}
+
 /*
  * This callback will either be called synchonously from add_memory() or
  * asynchronously (e.g., triggered via user space). We have to be careful
@@ -608,6 +632,7 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb,
 			mutex_lock(&vm->hotplug_mutex);
 			vm->hotplug_active = true;
 		}
+		virtio_mem_mb_going_offline_fixup_stats(vm, mb_id, false);
 		break;
 	case MEM_GOING_ONLINE:
 		spin_lock_irq(&vm->removal_lock);
@@ -633,6 +658,8 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb,
 		mutex_unlock(&vm->hotplug_mutex);
 		break;
 	case MEM_CANCEL_OFFLINE:
+		virtio_mem_mb_going_offline_fixup_stats(vm, mb_id, true);
+		/* fall through */
 	case MEM_CANCEL_ONLINE:
 		/* We might not get a MEM_GOING* if somebody else canceled */
 		if (vm->hotplug_active) {
@@ -648,23 +675,55 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb,
 }
 
 /*
- * Set a range of pages PG_offline.
+ * Convert PG_reserved to PG_dirty. Needed to allow isolation code to
+ * not immediately consider them as unmovable.
+ */
+static void virtio_mem_reserved_to_dirty(unsigned long pfn,
+					 unsigned int nr_pages)
+{
+	for (; nr_pages--; pfn++) {
+		SetPageDirty(pfn_to_page(pfn));
+		ClearPageReserved(pfn_to_page(pfn));
+	}
+}
+
+/*
+ * Convert PG_dirty to PG_reserved. Needed so generic_online_page()
+ * works correctly.
+ */
+static void virtio_mem_dirty_to_reserved(unsigned long pfn,
+					 unsigned int nr_pages)
+{
+	for (; nr_pages--; pfn++) {
+		SetPageReserved(pfn_to_page(pfn));
+		ClearPageDirty(pfn_to_page(pfn));
+	}
+}
+
+/*
+ * Set a range of pages PG_offline and drop the reference. The dropped
+ * reference (0) and the flag allows isolation code to isolate this range
+ * and offline code to offline it.
  */
 static void virtio_mem_set_fake_offline(unsigned long pfn,
 					unsigned int nr_pages)
 {
-	for (; nr_pages--; pfn++)
+	for (; nr_pages--; pfn++) {
 		__SetPageOffline(pfn_to_page(pfn));
+		page_ref_dec(pfn_to_page(pfn));
+	}
 }
 
 /*
- * Clear PG_offline from a range of pages.
+ * Get a reference and clear PG_offline from a range of pages.
  */
 static void virtio_mem_clear_fake_offline(unsigned long pfn,
 					  unsigned int nr_pages)
 {
-	for (; nr_pages--; pfn++)
+	for (; nr_pages--; pfn++) {
+		page_ref_inc(pfn_to_page(pfn));
 		__ClearPageOffline(pfn_to_page(pfn));
+	}
 }
 
 /*
@@ -679,7 +738,7 @@ static void virtio_mem_fake_online(unsigned long pfn, unsigned int nr_pages)
 	/*
 	 * We are always called with subblock granularity, which is at least
 	 * aligned to MAX_ORDER - 1. All pages in a subblock are either
-	 * reserved or not.
+	 * PG_dirty (converted PG_reserved) or not.
 	 */
 	BUG_ON(!IS_ALIGNED(pfn, 1 << order));
 	BUG_ON(!IS_ALIGNED(nr_pages, 1 << order));
@@ -690,13 +749,14 @@ static void virtio_mem_fake_online(unsigned long pfn, unsigned int nr_pages)
 		struct page *page = pfn_to_page(pfn + i);
 
 		/*
-		 * If the page is reserved, it was kept fake-offline when
+		 * If the page is PG_dirty, it was kept fake-offline when
 		 * onlining the memory block. Otherwise, it was allocated
 		 * using alloc_contig_range().
 		 */
-		if (PageReserved(page))
+		if (PageDirty(page)) {
+			virtio_mem_dirty_to_reserved(pfn + i, 1 << order);
 			generic_online_page(page, order);
-		else {
+		} else {
 			free_contig_range(pfn + i, 1 << order);
 			totalram_pages_add(1 << order);
 		}
@@ -728,8 +788,10 @@ static void virtio_mem_online_page_cb(struct page *page, unsigned int order)
 		 */
 		if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1))
 			generic_online_page(page, order);
-		else
+		else {
 			virtio_mem_set_fake_offline(PFN_DOWN(addr), 1 << order);
+			virtio_mem_reserved_to_dirty(PFN_DOWN(addr), 1 << order);
+		}
 		rcu_read_unlock();
 		return;
 	}
@@ -1674,7 +1736,8 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 static void virtio_mem_remove(struct virtio_device *vdev)
 {
 	struct virtio_mem *vm = vdev->priv;
-	unsigned long mb_id;
+	unsigned long nr_pages = PFN_DOWN(vm->subblock_size);
+	unsigned long pfn, mb_id, sb_id;
 	int rc;
 
 	/*
@@ -1701,6 +1764,25 @@ static void virtio_mem_remove(struct virtio_device *vdev)
 		BUG_ON(rc);
 		mutex_lock(&vm->hotplug_mutex);
 	}
+	/*
+	 * After we unregistered our callbacks, user space can offline +
+	 * re-online partially plugged online blocks. Make sure they can't
+	 * get offlined by getting a reference. Also, restore PG_reserved.
+	 */
+	virtio_mem_for_each_mb_state(vm, mb_id,
+				     VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL) {
+		for (sb_id = 0; sb_id < vm->nb_sb_per_mb; sb_id++) {
+			if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1))
+				continue;
+			pfn = PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) +
+			      sb_id * vm->subblock_size);
+
+			if (PageDirty(pfn_to_page(pfn)))
+				virtio_mem_dirty_to_reserved(pfn, nr_pages);
+			for (; nr_pages--; pfn++)
+				page_ref_inc(pfn_to_page(pfn));
+		}
+	}
 	mutex_unlock(&vm->hotplug_mutex);
 
 	/* unregister callbacks */
Signature · 1 line
2.21.0
#56 lore raw

RFC PATCH v3 7/9 virtio-mem: Allow to offline partially unplugged memory blocks

In reply to David Hildenbrand

Dropping the reference count of PageOffline() pages allows offlining code to skip them. However, we also have to convert PG_reserved to another flag - let's use PG_dirty - so has_unmovable_pages() will properly handle them. PG_reserved pages get detected as unmovable right away.

We need the flag to see if we are onlining pages the first time, or if we allocated them via alloc_contig_range().

Properly take care of offlining code also modifying the stats and special handling in case the driver gets unloaded.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/virtio_mem.c | 102 ++++++++++++++++++++++++++++++++----
 1 file changed, 92 insertions(+), 10 deletions(-)
drivers/virtio/virtio_mem.c +92 -10
index 91052a37d10d..9cb31459b211 100644
@@ -561,6 +561,30 @@ static void virtio_mem_notify_online(struct virtio_mem *vm, unsigned long mb_id,
 		virtio_mem_retry(vm);
 }
 
+/*
+ * When we unplug subblocks, we already modify stats (e.g., subtract them
+ * from totalram_pages). Offlining code will modify the stats, too. So
+ * properly fixup the stats when GOING_OFFLINE and revert that when
+ * CANCEL_OFFLINE.
+ */
+static void virtio_mem_mb_going_offline_fixup_stats(struct virtio_mem *vm,
+						    unsigned long mb_id,
+						    bool cancel)
+{
+	const unsigned long nr_pages = PFN_DOWN(vm->subblock_size);
+	int sb_id;
+
+	for (sb_id = 0; sb_id < vm->nb_sb_per_mb; sb_id++) {
+		if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1))
+			continue;
+
+		if (cancel)
+			totalram_pages_add(-nr_pages);
+		else
+			totalram_pages_add(nr_pages);
+	}
+}
+
 /*
  * This callback will either be called synchonously from add_memory() or
  * asynchronously (e.g., triggered via user space). We have to be careful
@@ -608,6 +632,7 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb,
 			mutex_lock(&vm->hotplug_mutex);
 			vm->hotplug_active = true;
 		}
+		virtio_mem_mb_going_offline_fixup_stats(vm, mb_id, false);
 		break;
 	case MEM_GOING_ONLINE:
 		spin_lock_irq(&vm->removal_lock);
@@ -633,6 +658,8 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb,
 		mutex_unlock(&vm->hotplug_mutex);
 		break;
 	case MEM_CANCEL_OFFLINE:
+		virtio_mem_mb_going_offline_fixup_stats(vm, mb_id, true);
+		/* fall through */
 	case MEM_CANCEL_ONLINE:
 		/* We might not get a MEM_GOING* if somebody else canceled */
 		if (vm->hotplug_active) {
@@ -648,23 +675,55 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb,
 }
 
 /*
- * Set a range of pages PG_offline.
+ * Convert PG_reserved to PG_dirty. Needed to allow isolation code to
+ * not immediately consider them as unmovable.
+ */
+static void virtio_mem_reserved_to_dirty(unsigned long pfn,
+					 unsigned int nr_pages)
+{
+	for (; nr_pages--; pfn++) {
+		SetPageDirty(pfn_to_page(pfn));
+		ClearPageReserved(pfn_to_page(pfn));
+	}
+}
+
+/*
+ * Convert PG_dirty to PG_reserved. Needed so generic_online_page()
+ * works correctly.
+ */
+static void virtio_mem_dirty_to_reserved(unsigned long pfn,
+					 unsigned int nr_pages)
+{
+	for (; nr_pages--; pfn++) {
+		SetPageReserved(pfn_to_page(pfn));
+		ClearPageDirty(pfn_to_page(pfn));
+	}
+}
+
+/*
+ * Set a range of pages PG_offline and drop the reference. The dropped
+ * reference (0) and the flag allows isolation code to isolate this range
+ * and offline code to offline it.
  */
 static void virtio_mem_set_fake_offline(unsigned long pfn,
 					unsigned int nr_pages)
 {
-	for (; nr_pages--; pfn++)
+	for (; nr_pages--; pfn++) {
 		__SetPageOffline(pfn_to_page(pfn));
+		page_ref_dec(pfn_to_page(pfn));
+	}
 }
 
 /*
- * Clear PG_offline from a range of pages.
+ * Get a reference and clear PG_offline from a range of pages.
  */
 static void virtio_mem_clear_fake_offline(unsigned long pfn,
 					  unsigned int nr_pages)
 {
-	for (; nr_pages--; pfn++)
+	for (; nr_pages--; pfn++) {
+		page_ref_inc(pfn_to_page(pfn));
 		__ClearPageOffline(pfn_to_page(pfn));
+	}
 }
 
 /*
@@ -679,7 +738,7 @@ static void virtio_mem_fake_online(unsigned long pfn, unsigned int nr_pages)
 	/*
 	 * We are always called with subblock granularity, which is at least
 	 * aligned to MAX_ORDER - 1. All pages in a subblock are either
-	 * reserved or not.
+	 * PG_dirty (converted PG_reserved) or not.
 	 */
 	BUG_ON(!IS_ALIGNED(pfn, 1 << order));
 	BUG_ON(!IS_ALIGNED(nr_pages, 1 << order));
@@ -690,13 +749,14 @@ static void virtio_mem_fake_online(unsigned long pfn, unsigned int nr_pages)
 		struct page *page = pfn_to_page(pfn + i);
 
 		/*
-		 * If the page is reserved, it was kept fake-offline when
+		 * If the page is PG_dirty, it was kept fake-offline when
 		 * onlining the memory block. Otherwise, it was allocated
 		 * using alloc_contig_range().
 		 */
-		if (PageReserved(page))
+		if (PageDirty(page)) {
+			virtio_mem_dirty_to_reserved(pfn + i, 1 << order);
 			generic_online_page(page, order);
-		else {
+		} else {
 			free_contig_range(pfn + i, 1 << order);
 			totalram_pages_add(1 << order);
 		}
@@ -728,8 +788,10 @@ static void virtio_mem_online_page_cb(struct page *page, unsigned int order)
 		 */
 		if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1))
 			generic_online_page(page, order);
-		else
+		else {
 			virtio_mem_set_fake_offline(PFN_DOWN(addr), 1 << order);
+			virtio_mem_reserved_to_dirty(PFN_DOWN(addr), 1 << order);
+		}
 		rcu_read_unlock();
 		return;
 	}
@@ -1674,7 +1736,8 @@ static int virtio_mem_probe(struct virtio_device *vdev)
 static void virtio_mem_remove(struct virtio_device *vdev)
 {
 	struct virtio_mem *vm = vdev->priv;
-	unsigned long mb_id;
+	unsigned long nr_pages = PFN_DOWN(vm->subblock_size);
+	unsigned long pfn, mb_id, sb_id;
 	int rc;
 
 	/*
@@ -1701,6 +1764,25 @@ static void virtio_mem_remove(struct virtio_device *vdev)
 		BUG_ON(rc);
 		mutex_lock(&vm->hotplug_mutex);
 	}
+	/*
+	 * After we unregistered our callbacks, user space can offline +
+	 * re-online partially plugged online blocks. Make sure they can't
+	 * get offlined by getting a reference. Also, restore PG_reserved.
+	 */
+	virtio_mem_for_each_mb_state(vm, mb_id,
+				     VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL) {
+		for (sb_id = 0; sb_id < vm->nb_sb_per_mb; sb_id++) {
+			if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1))
+				continue;
+			pfn = PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) +
+			      sb_id * vm->subblock_size);
+
+			if (PageDirty(pfn_to_page(pfn)))
+				virtio_mem_dirty_to_reserved(pfn, nr_pages);
+			for (; nr_pages--; pfn++)
+				page_ref_inc(pfn_to_page(pfn));
+		}
+	}
 	mutex_unlock(&vm->hotplug_mutex);
 
 	/* unregister callbacks */
Signature · 1 line
2.21.0
#57 lore raw

RFC PATCH v3 8/9 mm/memory_hotplug: Introduce offline_and_remove_memory()

In reply to David Hildenbrand

virtio-mem wants to offline and remove a memory block once it unplugged all subblocks (e.g., using alloc_contig_range()). Let's provide an interface to do that from a driver. virtio-mem already supports to offline partially unplugged memory blocks. Offlining a fully unplugged memory block will not require to migrate any pages. All unplugged subblocks are PageOffline() and have a reference count of 0 - so offlining code will simply skip them.

All we need an interface to trigger the "offlining" and the removing in a single operation - to make sure the memory block cannot get onlined by user space again before it gets removed.

To keep things simple, allow to only work on a single memory block.

Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Wei Yang <[email protected]>
Cc Dan Williams <[email protected]>
Cc Qian Cai <[email protected]>
---
 include/linux/memory_hotplug.h |  1 +
 mm/memory_hotplug.c            | 35 ++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
include/linux/memory_hotplug.h +1 -0
index 384ffb3d69ab..a1bf868aaeba 100644
@@ -313,6 +313,7 @@ extern void try_offline_node(int nid);
 extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
 extern int remove_memory(int nid, u64 start, u64 size);
 extern void __remove_memory(int nid, u64 start, u64 size);
+extern int offline_and_remove_memory(int nid, u64 start, u64 size);
 
 #else
 static inline bool is_mem_section_removable(unsigned long pfn,
mm/memory_hotplug.c +35 -0
index d23ff7c5c96b..caf3c93f5f7c 100644
@@ -1742,4 +1742,39 @@ int remove_memory(int nid, u64 start, u64 size)
 	return rc;
 }
 EXPORT_SYMBOL_GPL(remove_memory);
+
+/*
+ * Try to offline and remove a memory block. Might take a long time to
+ * finish in case memory is still in use. Primarily useful for memory devices
+ * that logically unplugged all memory (so it's no longer in use) and want to
+ * offline + remove the memory block.
+ */
+int offline_and_remove_memory(int nid, u64 start, u64 size)
+{
+	struct memory_block *mem;
+	int rc = -EINVAL;
+
+	if (!IS_ALIGNED(start, memory_block_size_bytes()) ||
+	    size != memory_block_size_bytes())
+		return rc;
+
+	lock_device_hotplug();
+	mem = find_memory_block(__pfn_to_section(PFN_DOWN(start)));
+	if (mem)
+		rc = device_offline(&mem->dev);
+	/* Ignore if the device is already offline. */
+	if (rc > 0)
+		rc = 0;
+
+	/*
+	 * In case we succeeded to offline the memory block, remove it.
+	 * This cannot fail as it cannot get onlined in the meantime.
+	 */
+	if (!rc && try_remove_memory(nid, start, size))
+		BUG();
+	unlock_device_hotplug();
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(offline_and_remove_memory);
 #endif /* CONFIG_MEMORY_HOTREMOVE */
Signature · 1 line
2.21.0
#58 lore raw

RFC PATCH v3 8/9 mm/memory_hotplug: Introduce offline_and_remove_memory()

In reply to David Hildenbrand

virtio-mem wants to offline and remove a memory block once it unplugged all subblocks (e.g., using alloc_contig_range()). Let's provide an interface to do that from a driver. virtio-mem already supports to offline partially unplugged memory blocks. Offlining a fully unplugged memory block will not require to migrate any pages. All unplugged subblocks are PageOffline() and have a reference count of 0 - so offlining code will simply skip them.

All we need an interface to trigger the "offlining" and the removing in a single operation - to make sure the memory block cannot get onlined by user space again before it gets removed.

To keep things simple, allow to only work on a single memory block.

Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Wei Yang <[email protected]>
Cc Dan Williams <[email protected]>
Cc Qian Cai <[email protected]>
---
 include/linux/memory_hotplug.h |  1 +
 mm/memory_hotplug.c            | 35 ++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
include/linux/memory_hotplug.h +1 -0
index 384ffb3d69ab..a1bf868aaeba 100644
@@ -313,6 +313,7 @@ extern void try_offline_node(int nid);
 extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
 extern int remove_memory(int nid, u64 start, u64 size);
 extern void __remove_memory(int nid, u64 start, u64 size);
+extern int offline_and_remove_memory(int nid, u64 start, u64 size);
 
 #else
 static inline bool is_mem_section_removable(unsigned long pfn,
mm/memory_hotplug.c +35 -0
index d23ff7c5c96b..caf3c93f5f7c 100644
@@ -1742,4 +1742,39 @@ int remove_memory(int nid, u64 start, u64 size)
 	return rc;
 }
 EXPORT_SYMBOL_GPL(remove_memory);
+
+/*
+ * Try to offline and remove a memory block. Might take a long time to
+ * finish in case memory is still in use. Primarily useful for memory devices
+ * that logically unplugged all memory (so it's no longer in use) and want to
+ * offline + remove the memory block.
+ */
+int offline_and_remove_memory(int nid, u64 start, u64 size)
+{
+	struct memory_block *mem;
+	int rc = -EINVAL;
+
+	if (!IS_ALIGNED(start, memory_block_size_bytes()) ||
+	    size != memory_block_size_bytes())
+		return rc;
+
+	lock_device_hotplug();
+	mem = find_memory_block(__pfn_to_section(PFN_DOWN(start)));
+	if (mem)
+		rc = device_offline(&mem->dev);
+	/* Ignore if the device is already offline. */
+	if (rc > 0)
+		rc = 0;
+
+	/*
+	 * In case we succeeded to offline the memory block, remove it.
+	 * This cannot fail as it cannot get onlined in the meantime.
+	 */
+	if (!rc && try_remove_memory(nid, start, size))
+		BUG();
+	unlock_device_hotplug();
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(offline_and_remove_memory);
 #endif /* CONFIG_MEMORY_HOTREMOVE */
Signature · 1 line
2.21.0
#59 lore raw

Re: [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory()

Quoted 8 lines · David Hildenbrand

virtio-mem wants to offline and remove a memory block once it unplugged all subblocks (e.g., using alloc_contig_range()). Let's provide an interface to do that from a driver. virtio-mem already supports to offline partially unplugged memory blocks. Offlining a fully unplugged memory block will not require to migrate any pages. All unplugged subblocks are PageOffline() and have a reference count of 0 - so offlining code will simply skip them.

Quoted 5 lines

All we need an interface to trigger the "offlining" and the removing in a single operation - to make sure the memory block cannot get onlined by user space again before it gets removed.

To keep things simple, allow to only work on a single memory block.

Without a user it is not really clear why do we need this interface. I am also not really sure why do you want/need to control beyond the offlining stage. Care to explain some more?

Signature · 2 lines
Michal Hocko
SUSE Labs
#60 lore raw

Re: [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory()

Quoted 13 lines · Michal Hocko
Quoted 12 lines · David Hildenbrand

virtio-mem wants to offline and remove a memory block once it unplugged all subblocks (e.g., using alloc_contig_range()). Let's provide an interface to do that from a driver. virtio-mem already supports to offline partially unplugged memory blocks. Offlining a fully unplugged memory block will not require to migrate any pages. All unplugged subblocks are PageOffline() and have a reference count of 0 - so offlining code will simply skip them.

All we need an interface to trigger the "offlining" and the removing in a single operation - to make sure the memory block cannot get onlined by user space again before it gets removed.

Quoted 6 lines
Quoted 1 line

To keep things simple, allow to only work on a single memory block.

Without a user it is not really clear why do we need this interface. I am also not really sure why do you want/need to control beyond the offlining stage. Care to explain some more?

The user is the next (small) patch in this series:

https://lkml.org/lkml/2019/9/19/475

Let's assume virtio-mem added a memory block and that block was onlined (e.g. by user space). E.g. 128MB.

On request, virtio-mem used alloc_contig_range() to logically unplug all chunks (e.g., 4MB) of that memory block. virtio-mem marked all pages PG_offline and dropped the reference count to 0 (to allow the memory block to get offlined). Basically no memory of the memory block is still in use by the system. So it is very desirable to remove that memory block along with the vmemmap and the page tables. This frees up memory.

In order to remove the memory block, it first has to be officially offlined (e.g., make the memory block as offline). Then, the memory block can get cleanly removed. Otherwise, try_remove_memory() will fail.

To do this, virtio-mem needs an interface to perform both steps (offline + remove).

There is no interface for a driver to offline a memory block. What I propose here performs both steps (offline+remove) in a single step, as that is really what the driver wants.

Signature · 3 lines
Thanks,

David / dhildenb
#61 lore raw

RFC PATCH v3 9/9 virtio-mem: Offline and remove completely unplugged memory blocks

In reply to David Hildenbrand

Let's offline+remove memory blocks once all subblocks are unplugged. We can use the new Linux MM interface for that. As no memory is in use anymore, this shouldn't take a long time and shouldn't fail. There might be corner cases where the offlining could still fail (especially, if another notifier NACKs the offlining request).

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/virtio_mem.c | 39 ++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
drivers/virtio/virtio_mem.c +36 -3
index 9cb31459b211..01d5fc784d5d 100644
@@ -433,6 +433,28 @@ static int virtio_mem_mb_remove(struct virtio_mem *vm, unsigned long mb_id)
 	return remove_memory(nid, addr, memory_block_size_bytes());
 }
 
+/*
+ * Try to offline and remove a memory block from Linux.
+ *
+ * Must not be called with the vm->hotplug_mutex held (possible deadlock with
+ * onlining code).
+ *
+ * Will not modify the state of the memory block.
+ */
+static int virtio_mem_mb_offline_and_remove(struct virtio_mem *vm,
+					    unsigned long mb_id)
+{
+	const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
+	int nid = vm->nid;
+
+	if (nid == NUMA_NO_NODE)
+		nid = memory_add_physaddr_to_nid(addr);
+
+	dev_dbg(&vm->vdev->dev, "offlining and removing memory block: %lu\n",
+		mb_id);
+	return offline_and_remove_memory(nid, addr, memory_block_size_bytes());
+}
+
 /*
  * Trigger the workqueue so the device can perform its magic.
  */
@@ -1263,7 +1285,8 @@ static int virtio_mem_mb_unplug_any_sb_offline(struct virtio_mem *vm,
  * Unplug the desired number of plugged subblocks of an online memory block.
  * Will skip subblock that are busy.
  *
- * Will modify the state of the memory block.
+ * Will modify the state of the memory block. Might temporarily drop the
+ * hotplug_mutex.
  *
  * Note: Can fail after some subblocks were successfully unplugged. Can
  *       return 0 even if subblocks were busy and could not get unplugged.
@@ -1319,9 +1342,19 @@ static int virtio_mem_mb_unplug_any_sb_online(struct virtio_mem *vm,
 	}
 
 	/*
-	 * TODO: Once all subblocks of a memory block were unplugged, we want
-	 * to offline the memory block and remove it.
+	 * Once all subblocks of a memory block were unplugged, offline and
+	 * remove it. This will usually not fail, as no memory is in use
+	 * anymore - however some other notifiers might NACK the request.
 	 */
+	if (virtio_mem_mb_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) {
+		mutex_unlock(&vm->hotplug_mutex);
+		rc = virtio_mem_mb_offline_and_remove(vm, mb_id);
+		mutex_lock(&vm->hotplug_mutex);
+		if (!rc)
+			virtio_mem_mb_set_state(vm, mb_id,
+						VIRTIO_MEM_MB_STATE_UNUSED);
+	}
+
 	return 0;
 }
 
Signature · 1 line
2.21.0
#62 lore raw

RFC PATCH v3 9/9 virtio-mem: Offline and remove completely unplugged memory blocks

In reply to David Hildenbrand

Let's offline+remove memory blocks once all subblocks are unplugged. We can use the new Linux MM interface for that. As no memory is in use anymore, this shouldn't take a long time and shouldn't fail. There might be corner cases where the offlining could still fail (especially, if another notifier NACKs the offlining request).

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Dave Young <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Dan Williams <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc Vlastimil Babka <[email protected]>
---
 drivers/virtio/virtio_mem.c | 39 ++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
drivers/virtio/virtio_mem.c +36 -3
index 9cb31459b211..01d5fc784d5d 100644
@@ -433,6 +433,28 @@ static int virtio_mem_mb_remove(struct virtio_mem *vm, unsigned long mb_id)
 	return remove_memory(nid, addr, memory_block_size_bytes());
 }
 
+/*
+ * Try to offline and remove a memory block from Linux.
+ *
+ * Must not be called with the vm->hotplug_mutex held (possible deadlock with
+ * onlining code).
+ *
+ * Will not modify the state of the memory block.
+ */
+static int virtio_mem_mb_offline_and_remove(struct virtio_mem *vm,
+					    unsigned long mb_id)
+{
+	const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id);
+	int nid = vm->nid;
+
+	if (nid == NUMA_NO_NODE)
+		nid = memory_add_physaddr_to_nid(addr);
+
+	dev_dbg(&vm->vdev->dev, "offlining and removing memory block: %lu\n",
+		mb_id);
+	return offline_and_remove_memory(nid, addr, memory_block_size_bytes());
+}
+
 /*
  * Trigger the workqueue so the device can perform its magic.
  */
@@ -1263,7 +1285,8 @@ static int virtio_mem_mb_unplug_any_sb_offline(struct virtio_mem *vm,
  * Unplug the desired number of plugged subblocks of an online memory block.
  * Will skip subblock that are busy.
  *
- * Will modify the state of the memory block.
+ * Will modify the state of the memory block. Might temporarily drop the
+ * hotplug_mutex.
  *
  * Note: Can fail after some subblocks were successfully unplugged. Can
  *       return 0 even if subblocks were busy and could not get unplugged.
@@ -1319,9 +1342,19 @@ static int virtio_mem_mb_unplug_any_sb_online(struct virtio_mem *vm,
 	}
 
 	/*
-	 * TODO: Once all subblocks of a memory block were unplugged, we want
-	 * to offline the memory block and remove it.
+	 * Once all subblocks of a memory block were unplugged, offline and
+	 * remove it. This will usually not fail, as no memory is in use
+	 * anymore - however some other notifiers might NACK the request.
 	 */
+	if (virtio_mem_mb_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) {
+		mutex_unlock(&vm->hotplug_mutex);
+		rc = virtio_mem_mb_offline_and_remove(vm, mb_id);
+		mutex_lock(&vm->hotplug_mutex);
+		if (!rc)
+			virtio_mem_mb_set_state(vm, mb_id,
+						VIRTIO_MEM_MB_STATE_UNUSED);
+	}
+
 	return 0;
 }
 
Signature · 1 line
2.21.0
#63 lore raw
In reply to David Hildenbrand
Quoted 306 lines · David Hildenbrand

Long time no RFC! I finally had time to get the next version of the Linux driver side of virtio-mem into shape, incorporating ideas and feedback from previous discussions.

This RFC is based on the series currently on the mm list:
- [PATCH 0/3] Remove __online_page_set_limits()
- [PATCH v1 0/3] mm/memory_hotplug: Export generic_online_page()
- [PATCH v4 0/8] mm/memory_hotplug: Shrink zones before removing memory The current state (kept updated) is available on
- https://github.com/davidhildenbrand/linux/tree/virtio-mem

The basic idea of virtio-mem is to provide a flexible, cross-architecture memory hot(un)plug solution that avoids many limitations imposed by existing technologies, architectures, and interfaces. More details can be found below and in linked material.

This RFC was only tested on x86-64, however, should theoretically work on any Linux architecture that implements memory hot(un)plug - like s390x. On x86-64, it is currently possible to add/remove memory to the system in >= 4MB granularity. Memory hotplug works very reliable. For memory unplug, there are no guarantees how much memory can actually get unplugged, it depends on the setup. I have plans to improve that in the future.

--------------------------------------------------------------------------
1. virtio-mem
--------------------------------------------------------------------------

The basic idea behind virtio-mem was presented at KVM Forum 2018. The slides can be found at [1]. The previous RFC can be found at [2]. The first RFC can be found at [3]. However, the concept evolved over time. The KVM Forum slides roughly match the current design.

Patch #2 ("virtio-mem: Paravirtualized memory hotplug") contains quite some information, especially in "include/uapi/linux/virtio_mem.h":

Each virtio-mem device manages a dedicated region in physical address space. Each device can belong to a single NUMA node, multiple devices for a single NUMA node are possible. A virtio-mem device is like a "resizable DIMM" consisting of small memory blocks that can be plugged or unplugged. The device driver is responsible for (un)plugging memory blocks on demand.

Virtio-mem devices can only operate on their assigned memory region in order to (un)plug memory. A device cannot (un)plug memory belonging to other devices.

The "region_size" corresponds to the maximum amount of memory that can be provided by a device. The "size" corresponds to the amount of memory that is currently plugged. "requested_size" corresponds to a request from the device to the device driver to (un)plug blocks. The device driver should try to (un)plug blocks in order to reach the "requested_size". It is impossible to plug more memory than requested.

The "usable_region_size" represents the memory region that can actually be used to (un)plug memory. It is always at least as big as the "requested_size" and will grow dynamically. It will only shrink when explicitly triggered (VIRTIO_MEM_REQ_UNPLUG).

Memory in the usable region can usually be read, however, there are no guarantees. It can happen that the device cannot process a request, because it is busy. The device driver has to retry later.

Usually, during system resets all memory will get unplugged, so the device driver can start with a clean state. However, in specific scenarios (if the device is busy) it can happen that the device still has memory plugged. The device driver can request to unplug all memory (VIRTIO_MEM_REQ_UNPLUG) - which might take a while to succeed if the device is busy.

--------------------------------------------------------------------------
2. Linux Implementation
--------------------------------------------------------------------------

This RFC reuses quite some existing MM infrastructure, however, has to expose some additional functionality.

Memory blocks (e.g., 128MB) are added/removed on demand. Within these memory blocks, subblocks (e.g., 4MB) are plugged/unplugged. The sizes depend on the target architecture, MAX_ORDER + pageblock_order, and the block size of a virtio-mem device.

add_memory()/try_remove_memory() is used to add/remove memory blocks. virtio-mem will not online memory blocks itself. This has to be done by user space, or configured into the kernel (CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE). virtio-mem will only unplug memory that was online to the ZONE_NORMAL. Memory is suggested to be onlined to the ZONE_NORMAL for now.

The memory hotplug notifier is used to properly synchronize against onlining/offlining of memory blocks and to track the states of memory blocks (including the zone memory blocks are onlined to). Locking is done similar to the PPC CMA implementation.

The set_online_page() callback is used to keep unplugged subblocks of a memory block fake-offline when onlining the memory block. generic_online_page() is used to fake-online plugged subblocks. This handling is similar to the Hyper-V balloon driver.

PG_offline is used to mark unplugged subblocks as offline, so e.g., dumping tools (makedumpfile) will skip these pages. This is similar to other balloon drivers like virtio-balloon and Hyper-V.

PG_offline + reference count of 0 [new] is now also used to mark pages as a "skip" when offlining memory blocks. This allows to offline memory blocks that have partially unplugged subblocks - or are completely unplugged.

alloc_contig_range()/free_contig_range() [now exposed] is used to unplug/plug subblocks of memory blocks the are already exposed to Linux.

offline_and_remove_memory() [new] is used to offline a fully unplugged memory block and remove it from Linux.

A lot of additional information can be found in the separate patches and as comments in the code itself.

--------------------------------------------------------------------------
3. Major changes since last RFC
--------------------------------------------------------------------------

A lot of things changed, especially also on the QEMU + virtio side. The biggest changes on the Linux driver side are:
- Onlining/offlining of subblocks is now emulated on top of memory blocks. set_online_page()+alloc_contig_range()+free_contig_range() is now used for that. Core MM does not have to be modified and will continue to online/offline full memory blocks.
- Onlining/offlining of memory blocks is no longer performed by virtio-mem.
- Pg_offline is upstream and can be used. It is also used to allow offlining of partially unplugged memory blocks.
- Memory block states + subblocks are now tracked more space-efficient.
- Proper kexec(), kdump(), driver unload, driver reload, ZONE_MOVABLE, ... handling.

--------------------------------------------------------------------------
4. Future work
--------------------------------------------------------------------------

The separate patches contain a lot of future work items. One of the next steps is to make memory unplug more likely to succeed - currently, there are no guarantees on how much memory can get unplugged again. I have various ideas on how to limit fragmentation of all memory blocks that virtio-mem added.

Memory hotplug:
- Reduce the amount of memory resources if that turnes out to be an issue. Or try to speed up relevant code paths to deal with many resources.
- Allocate the vmemmap from the added memory. Makes hotplug more likely to succeed, the vmemmap is stored on the same NUMA node and that unmovable memory will later not hinder unplug.

Memory hotunplug:
- Performance improvements:
-- Sense (lockless) if it make sense to try alloc_contig_range() at all
before directly trying to isolate and taking locks.
-- Try to unplug bigger chunks if possible first.
-- Identify free areas first, that don't have to be evacuated.
- Make unplug more likely to succeed:
-- The "issue" is that in the ZONE_NORMAL, the buddy will randomly
allocate memory. Only pageblocks somewhat limit fragmentation, however we would want to limit fragmentation on subblock granularity and even memory block granularity. One idea is to have a new ZONE_PREFER_MOVABLE. Memory blocks will then be onlined to ZONE_NORMAL / ZONE_PREFER_MOVABLE in a certain ratio per node (e.g., 1:4). This makes unplug of quite some memory likely to succeed in most setups. ZONE_PREFER_MOVABLE is then a mixture of ZONE_NORMAL and ZONE_MOVABlE. Especially, movable data can end up on that zone, but only if really required - avoiding running out of memory on ZONE imbalances. The zone fallback order would be MOVABLE=>PREFER_MOVABLE=>HIGHMEM=>NORMAL=>PREFER_MOVABLE=>DMA32=>DMA
-- Allocate memmap from added memory. This way, less unmovable data can
end up on the memory blocks.
-- Call drop_slab() before trying to unplug. Eventually shrink other
caches.
- Better retry handling in case memory is busy. We certainly don't want to try for ever in a short interval to try to get some memory back.
- OOM handling, e.g., via an OOM handler.

--------------------------------------------------------------------------
5. Example Usage
--------------------------------------------------------------------------

A very basic QEMU prototype (kept updated) is available at:
- https://github.com/davidhildenbrand/qemu/tree/virtio-mem

It lacks various features, however works to test the guest driver side:
- No support for resizable memory regions / memory backends
- No protection of unplugged memory (esp., userfaultfd-wp)
- No dump/migration/XXX optimizations to skip over unplugged memory

Start QEMU with two virtio-mem devices (one per NUMA node):

  $ qemu-system-x86_64 -m 4G,maxmem=20G \
   -smp sockets=2,cores=2 \
   -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \
   [...]
   -object memory-backend-ram,id=mem0,size=8G \
   -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=128M \
   -object memory-backend-ram,id=mem1,size=8G \
   -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=80M

Query the configuration:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) info memory-devices
Memory device [virtio-mem]: "vm0"

    memaddr: 0x140000000
    node: 0
    requested-size: 134217728
    size: 134217728
    max-size: 8589934592
    block-size: 2097152
    memdev: /objects/mem0
  Memory device [virtio-mem]: "vm1"
    memaddr: 0x340000000
    node: 1
    requested-size: 83886080
    size: 83886080
    max-size: 8589934592
    block-size: 2097152
    memdev: /objects/mem1

Add some memory to node 1:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) qom-set vm1 requested-size 1G

Remove some memory from node 0:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) qom-set vm0 requested-size 64M

Query the configuration:
(qemu) info memory-devices
Memory device [virtio-mem]: "vm0"
memaddr: 0x140000000
node: 0
requested-size: 67108864
size: 67108864
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem0
Memory device [virtio-mem]: "vm1"
memaddr: 0x340000000
node: 1
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem1

--------------------------------------------------------------------------
6. Q/A
--------------------------------------------------------------------------

Q: Why add/remove parts ("subblocks") of memory blocks/sections?
A: Flexibility (section size depends on the architecture) - e.g., some
architectures have a section size of 2GB. Also, the memory block size is variable (e.g., on x86-64). I want to avoid any such restrictions. Some use cases want to add/remove memory in smaller granularities to a VM (e.g., the Hyper-V balloon also implements this) - especially smaller VMs like used for kata containers. Also, on memory unplug, it is more reliable to free-up and unplug multiple small chunks instead of one big chunk. E.g., if one page of a DIMM is either unmovable or pinned, the DIMM can't get unplugged. This approach is basically a compromise between DIMM-based memory hot(un)plug and balloon inflation/deflation, which works mostly on page granularity.

Q: Why care about memory blocks?
A: They are the way to tell user space about new memory. This way,
memory can get onlined/offlined by user space. Also, e.g., kdump relies on udev events to reload kexec when memory blocks are onlined/offlined. Memory blocks are the "real" memory hot(un)plug granularity. Everything that's smaller has to be emulated "on top".

Q: Won't memory unplug of subblocks fragment memory?
A: Yes and no. Unplugging e.g., >=4MB subblocks on x86-64 will not really
fragment memory like unplugging random pages like a balloon driver does. Buddy merging will not be limited. However, any allocation that requires bigger consecutive memory chunks (e.g., gigantic pages) might observe the fragmentation. Possible solutions: Allocate gigantic huge pages before unplugging memory, don't unplug memory, combine virtio-mem with DIMM based memory or bigger initial memory. Remember, a virtio-mem device will only unplug on the memory range it manages, not on other DIMMs. Unplug of single memory blocks will result in similar fragmentation in respect to gigantic huge pages.

Q: How reliable is memory unplug?
A: There are no guarantees on how much memory can get unplugged
again. However, it is more likely to find 4MB chunks to unplug than e.g., 128MB chunks. If memory is terribly fragmented, there is nothing we can do - for now. I consider memory hotplug the first primary use of virtio-mem. Memory unplug might usually work, but we want to improve the performance and the amount of memory we can actually unplug later.

Q: Why not unplug from the ZONE_MOVABLE?
A: Unplugged memory chunks are unmovable. Unmovable data must not end up
on the ZONE_MOVABLE - similar to gigantic pages - they will never be allocated from ZONE_MOVABLE. Teaching MM to move unplugged chunks within a device might be problematic and will require a new guest->hypervisor command to move unplugged chunks. virtio-mem added memory can be onlined to the ZONE_MOVABLE, but subblocks will not get unplugged from it.

Q: How big should the initial (!virtio-mem) memory of a VM be?
A: virtio-mem memory will not go to the DMA zones. So to avoid running out
of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case.

Quoted 6 lines

[1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red-Hat-1.pdf
[2] https://lwn.net/Articles/755423/
[3] https://lists.gnu.org/archive/html/qemu-devel/2017-06/msg03870.html

---

Gentle ping. I want to continue working on this. I need feedback from MM people on ...

Quoted 5 lines

David Hildenbrand (9):
ACPI: NUMA: export pxm_to_node
virtio-mem: Paravirtualized memory hotplug
virtio-mem: Paravirtualized memory hotunplug part 1
mm: Export alloc_contig_range() / free_contig_range()

... this patch, if it is okay to export alloc_contig_range() / free_contig_range() (see next patch how it is used)

Quoted 2 lines

virtio-mem: Paravirtualized memory hotunplug part 2
mm: Allow to offline PageOffline() pages with a reference count of 0

... this patch, to see if we can use "PageOffline() + refcount == 0" as a way to allow offlining memory with unplugged pieces. (see next patch how this is used)

Quoted 2 lines

virtio-mem: Allow to offline partially unplugged memory blocks mm/memory_hotplug: Introduce offline_and_remove_memory()

... I assume this patch is not that debatable.

Quoted 1 line

virtio-mem: Offline and remove completely unplugged memory blocks

So yeah, please feedback on

"mm: Export alloc_contig_range() / free_contig_range()" and "mm: Allow to offline PageOffline() pages with a reference count of 0", because they are the essential pieces apart from what we already have (generic_online_page(), memory notifiers ...)

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#64 lore raw
In reply to David Hildenbrand
Quoted 306 lines · David Hildenbrand

Long time no RFC! I finally had time to get the next version of the Linux driver side of virtio-mem into shape, incorporating ideas and feedback from previous discussions.

This RFC is based on the series currently on the mm list:
- [PATCH 0/3] Remove __online_page_set_limits()
- [PATCH v1 0/3] mm/memory_hotplug: Export generic_online_page()
- [PATCH v4 0/8] mm/memory_hotplug: Shrink zones before removing memory The current state (kept updated) is available on
- https://github.com/davidhildenbrand/linux/tree/virtio-mem

The basic idea of virtio-mem is to provide a flexible, cross-architecture memory hot(un)plug solution that avoids many limitations imposed by existing technologies, architectures, and interfaces. More details can be found below and in linked material.

This RFC was only tested on x86-64, however, should theoretically work on any Linux architecture that implements memory hot(un)plug - like s390x. On x86-64, it is currently possible to add/remove memory to the system in >= 4MB granularity. Memory hotplug works very reliable. For memory unplug, there are no guarantees how much memory can actually get unplugged, it depends on the setup. I have plans to improve that in the future.

--------------------------------------------------------------------------
1. virtio-mem
--------------------------------------------------------------------------

The basic idea behind virtio-mem was presented at KVM Forum 2018. The slides can be found at [1]. The previous RFC can be found at [2]. The first RFC can be found at [3]. However, the concept evolved over time. The KVM Forum slides roughly match the current design.

Patch #2 ("virtio-mem: Paravirtualized memory hotplug") contains quite some information, especially in "include/uapi/linux/virtio_mem.h":

Each virtio-mem device manages a dedicated region in physical address space. Each device can belong to a single NUMA node, multiple devices for a single NUMA node are possible. A virtio-mem device is like a "resizable DIMM" consisting of small memory blocks that can be plugged or unplugged. The device driver is responsible for (un)plugging memory blocks on demand.

Virtio-mem devices can only operate on their assigned memory region in order to (un)plug memory. A device cannot (un)plug memory belonging to other devices.

The "region_size" corresponds to the maximum amount of memory that can be provided by a device. The "size" corresponds to the amount of memory that is currently plugged. "requested_size" corresponds to a request from the device to the device driver to (un)plug blocks. The device driver should try to (un)plug blocks in order to reach the "requested_size". It is impossible to plug more memory than requested.

The "usable_region_size" represents the memory region that can actually be used to (un)plug memory. It is always at least as big as the "requested_size" and will grow dynamically. It will only shrink when explicitly triggered (VIRTIO_MEM_REQ_UNPLUG).

Memory in the usable region can usually be read, however, there are no guarantees. It can happen that the device cannot process a request, because it is busy. The device driver has to retry later.

Usually, during system resets all memory will get unplugged, so the device driver can start with a clean state. However, in specific scenarios (if the device is busy) it can happen that the device still has memory plugged. The device driver can request to unplug all memory (VIRTIO_MEM_REQ_UNPLUG) - which might take a while to succeed if the device is busy.

--------------------------------------------------------------------------
2. Linux Implementation
--------------------------------------------------------------------------

This RFC reuses quite some existing MM infrastructure, however, has to expose some additional functionality.

Memory blocks (e.g., 128MB) are added/removed on demand. Within these memory blocks, subblocks (e.g., 4MB) are plugged/unplugged. The sizes depend on the target architecture, MAX_ORDER + pageblock_order, and the block size of a virtio-mem device.

add_memory()/try_remove_memory() is used to add/remove memory blocks. virtio-mem will not online memory blocks itself. This has to be done by user space, or configured into the kernel (CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE). virtio-mem will only unplug memory that was online to the ZONE_NORMAL. Memory is suggested to be onlined to the ZONE_NORMAL for now.

The memory hotplug notifier is used to properly synchronize against onlining/offlining of memory blocks and to track the states of memory blocks (including the zone memory blocks are onlined to). Locking is done similar to the PPC CMA implementation.

The set_online_page() callback is used to keep unplugged subblocks of a memory block fake-offline when onlining the memory block. generic_online_page() is used to fake-online plugged subblocks. This handling is similar to the Hyper-V balloon driver.

PG_offline is used to mark unplugged subblocks as offline, so e.g., dumping tools (makedumpfile) will skip these pages. This is similar to other balloon drivers like virtio-balloon and Hyper-V.

PG_offline + reference count of 0 [new] is now also used to mark pages as a "skip" when offlining memory blocks. This allows to offline memory blocks that have partially unplugged subblocks - or are completely unplugged.

alloc_contig_range()/free_contig_range() [now exposed] is used to unplug/plug subblocks of memory blocks the are already exposed to Linux.

offline_and_remove_memory() [new] is used to offline a fully unplugged memory block and remove it from Linux.

A lot of additional information can be found in the separate patches and as comments in the code itself.

--------------------------------------------------------------------------
3. Major changes since last RFC
--------------------------------------------------------------------------

A lot of things changed, especially also on the QEMU + virtio side. The biggest changes on the Linux driver side are:
- Onlining/offlining of subblocks is now emulated on top of memory blocks. set_online_page()+alloc_contig_range()+free_contig_range() is now used for that. Core MM does not have to be modified and will continue to online/offline full memory blocks.
- Onlining/offlining of memory blocks is no longer performed by virtio-mem.
- Pg_offline is upstream and can be used. It is also used to allow offlining of partially unplugged memory blocks.
- Memory block states + subblocks are now tracked more space-efficient.
- Proper kexec(), kdump(), driver unload, driver reload, ZONE_MOVABLE, ... handling.

--------------------------------------------------------------------------
4. Future work
--------------------------------------------------------------------------

The separate patches contain a lot of future work items. One of the next steps is to make memory unplug more likely to succeed - currently, there are no guarantees on how much memory can get unplugged again. I have various ideas on how to limit fragmentation of all memory blocks that virtio-mem added.

Memory hotplug:
- Reduce the amount of memory resources if that turnes out to be an issue. Or try to speed up relevant code paths to deal with many resources.
- Allocate the vmemmap from the added memory. Makes hotplug more likely to succeed, the vmemmap is stored on the same NUMA node and that unmovable memory will later not hinder unplug.

Memory hotunplug:
- Performance improvements:
-- Sense (lockless) if it make sense to try alloc_contig_range() at all
before directly trying to isolate and taking locks.
-- Try to unplug bigger chunks if possible first.
-- Identify free areas first, that don't have to be evacuated.
- Make unplug more likely to succeed:
-- The "issue" is that in the ZONE_NORMAL, the buddy will randomly
allocate memory. Only pageblocks somewhat limit fragmentation, however we would want to limit fragmentation on subblock granularity and even memory block granularity. One idea is to have a new ZONE_PREFER_MOVABLE. Memory blocks will then be onlined to ZONE_NORMAL / ZONE_PREFER_MOVABLE in a certain ratio per node (e.g., 1:4). This makes unplug of quite some memory likely to succeed in most setups. ZONE_PREFER_MOVABLE is then a mixture of ZONE_NORMAL and ZONE_MOVABlE. Especially, movable data can end up on that zone, but only if really required - avoiding running out of memory on ZONE imbalances. The zone fallback order would be MOVABLE=>PREFER_MOVABLE=>HIGHMEM=>NORMAL=>PREFER_MOVABLE=>DMA32=>DMA
-- Allocate memmap from added memory. This way, less unmovable data can
end up on the memory blocks.
-- Call drop_slab() before trying to unplug. Eventually shrink other
caches.
- Better retry handling in case memory is busy. We certainly don't want to try for ever in a short interval to try to get some memory back.
- OOM handling, e.g., via an OOM handler.

--------------------------------------------------------------------------
5. Example Usage
--------------------------------------------------------------------------

A very basic QEMU prototype (kept updated) is available at:
- https://github.com/davidhildenbrand/qemu/tree/virtio-mem

It lacks various features, however works to test the guest driver side:
- No support for resizable memory regions / memory backends
- No protection of unplugged memory (esp., userfaultfd-wp)
- No dump/migration/XXX optimizations to skip over unplugged memory

Start QEMU with two virtio-mem devices (one per NUMA node):

  $ qemu-system-x86_64 -m 4G,maxmem=20G \
   -smp sockets=2,cores=2 \
   -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \
   [...]
   -object memory-backend-ram,id=mem0,size=8G \
   -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=128M \
   -object memory-backend-ram,id=mem1,size=8G \
   -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=80M

Query the configuration:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) info memory-devices
Memory device [virtio-mem]: "vm0"

    memaddr: 0x140000000
    node: 0
    requested-size: 134217728
    size: 134217728
    max-size: 8589934592
    block-size: 2097152
    memdev: /objects/mem0
  Memory device [virtio-mem]: "vm1"
    memaddr: 0x340000000
    node: 1
    requested-size: 83886080
    size: 83886080
    max-size: 8589934592
    block-size: 2097152
    memdev: /objects/mem1

Add some memory to node 1:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) qom-set vm1 requested-size 1G

Remove some memory from node 0:
QEMU 4.1.50 monitor - type 'help' for more information (qemu) qom-set vm0 requested-size 64M

Query the configuration:
(qemu) info memory-devices
Memory device [virtio-mem]: "vm0"
memaddr: 0x140000000
node: 0
requested-size: 67108864
size: 67108864
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem0
Memory device [virtio-mem]: "vm1"
memaddr: 0x340000000
node: 1
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem1

--------------------------------------------------------------------------
6. Q/A
--------------------------------------------------------------------------

Q: Why add/remove parts ("subblocks") of memory blocks/sections?
A: Flexibility (section size depends on the architecture) - e.g., some
architectures have a section size of 2GB. Also, the memory block size is variable (e.g., on x86-64). I want to avoid any such restrictions. Some use cases want to add/remove memory in smaller granularities to a VM (e.g., the Hyper-V balloon also implements this) - especially smaller VMs like used for kata containers. Also, on memory unplug, it is more reliable to free-up and unplug multiple small chunks instead of one big chunk. E.g., if one page of a DIMM is either unmovable or pinned, the DIMM can't get unplugged. This approach is basically a compromise between DIMM-based memory hot(un)plug and balloon inflation/deflation, which works mostly on page granularity.

Q: Why care about memory blocks?
A: They are the way to tell user space about new memory. This way,
memory can get onlined/offlined by user space. Also, e.g., kdump relies on udev events to reload kexec when memory blocks are onlined/offlined. Memory blocks are the "real" memory hot(un)plug granularity. Everything that's smaller has to be emulated "on top".

Q: Won't memory unplug of subblocks fragment memory?
A: Yes and no. Unplugging e.g., >=4MB subblocks on x86-64 will not really
fragment memory like unplugging random pages like a balloon driver does. Buddy merging will not be limited. However, any allocation that requires bigger consecutive memory chunks (e.g., gigantic pages) might observe the fragmentation. Possible solutions: Allocate gigantic huge pages before unplugging memory, don't unplug memory, combine virtio-mem with DIMM based memory or bigger initial memory. Remember, a virtio-mem device will only unplug on the memory range it manages, not on other DIMMs. Unplug of single memory blocks will result in similar fragmentation in respect to gigantic huge pages.

Q: How reliable is memory unplug?
A: There are no guarantees on how much memory can get unplugged
again. However, it is more likely to find 4MB chunks to unplug than e.g., 128MB chunks. If memory is terribly fragmented, there is nothing we can do - for now. I consider memory hotplug the first primary use of virtio-mem. Memory unplug might usually work, but we want to improve the performance and the amount of memory we can actually unplug later.

Q: Why not unplug from the ZONE_MOVABLE?
A: Unplugged memory chunks are unmovable. Unmovable data must not end up
on the ZONE_MOVABLE - similar to gigantic pages - they will never be allocated from ZONE_MOVABLE. Teaching MM to move unplugged chunks within a device might be problematic and will require a new guest->hypervisor command to move unplugged chunks. virtio-mem added memory can be onlined to the ZONE_MOVABLE, but subblocks will not get unplugged from it.

Q: How big should the initial (!virtio-mem) memory of a VM be?
A: virtio-mem memory will not go to the DMA zones. So to avoid running out
of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case.

Quoted 6 lines

[1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red-Hat-1.pdf
[2] https://lwn.net/Articles/755423/
[3] https://lists.gnu.org/archive/html/qemu-devel/2017-06/msg03870.html

---

Gentle ping. I want to continue working on this. I need feedback from MM people on ...

Quoted 5 lines

David Hildenbrand (9):
ACPI: NUMA: export pxm_to_node
virtio-mem: Paravirtualized memory hotplug
virtio-mem: Paravirtualized memory hotunplug part 1
mm: Export alloc_contig_range() / free_contig_range()

... this patch, if it is okay to export alloc_contig_range() / free_contig_range() (see next patch how it is used)

Quoted 2 lines

virtio-mem: Paravirtualized memory hotunplug part 2
mm: Allow to offline PageOffline() pages with a reference count of 0

... this patch, to see if we can use "PageOffline() + refcount == 0" as a way to allow offlining memory with unplugged pieces. (see next patch how this is used)

Quoted 2 lines

virtio-mem: Allow to offline partially unplugged memory blocks mm/memory_hotplug: Introduce offline_and_remove_memory()

... I assume this patch is not that debatable.

Quoted 1 line

virtio-mem: Offline and remove completely unplugged memory blocks

So yeah, please feedback on

"mm: Export alloc_contig_range() / free_contig_range()" and "mm: Allow to offline PageOffline() pages with a reference count of 0", because they are the essential pieces apart from what we already have (generic_online_page(), memory notifiers ...)

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb