Skip to contentSkip to conversation

Enhance memory utilization with DMEMFS

RFC In review [email protected] 5 participants 48 messages Dec 7, 2020 – Dec 24, 2020

Subsystems (15)
lore
Share & tools
lore
#0 lore raw

v2 cover Enhance memory utilization with DMEMFS

From: Yulei Zhang <[email protected]>

In current system each physical memory page is assocaited with a page structure which is used to track the usage of this page. But due to the memory usage rapidly growing in cloud environment, we find the resource consuming for page structure storage becomes more and more remarkable. So is it possible that we could reclaim such memory and make it reusable?

This patchset introduces an idea about how to save the extra memory through a new virtual filesystem -- dmemfs.

Dmemfs (Direct Memory filesystem) is device memory or reserved memory based filesystem. This kind of memory is special as it is not managed by kernel and most important it is without 'struct page'. Therefore we can leverage the extra memory from the host system to support more tenants in our cloud service.

As the belowing figure shows, we uses a kernel boot parameter 'dmem=' to reserve the system memory when the host system boots up, the remaining system memory is still managed by system memory management which is associated with "struct page", the reserved memory will be managed by dmem and assigned to guest system, the details can be checked in /Documentation/admin-guide/kernel-parameters.txt.

   +------------------+--------------------------------------+
   |  system memory   |     memory for guest system          | 
   +------------------+--------------------------------------+
    |                                   |
    v                                   |
struct page                             |
    |                                   |
    v                                   v
    system mem management             dmem  

And during the usage, the dmemfs will handle the memory request to allocate and free the reserved memory on each NUMA node, the user space application could leverage the mmap interface to access the memory, and kernel module such as kvm and vfio would be able to pin the memory thongh follow_pfn() and get_user_page() in different given page size granularities.

          +-----------+  +-----------+
          |   QEMU    |  |  dpdk etc.|      user
          +-----+-----+  +-----------+
  +-----------|------\------------------------------+
  |           |       v                    kernel   |
  |           |     +-------+  +-------+            |
  |           |     |  KVM  |  | vfio  |            |
  |           |     +-------+  +-------+            |
  |           |         |          |                |
  |      +----v---------v----------v------+         |
  |      |                                |         |
  |      |             Dmemfs             |         |
  |      |                                |         |
  |      +--------------------------------+         |
  +-----------/-----------------------\-------------+
             /                         \
     +------v-----+                +----v-------+
     |   node 0   |                |   node 1   |
     +------------+                +------------+

Theoretically for each 4k physical page it can save 64 bytes if we drop the 'struct page', so for guest memory with 320G it can save about 5G physical memory totally.

Detailed usage of dmemfs is included in
/Documentation/filesystem/dmemfs.rst.

V1->V2:
* Rebase the code the kernel version 5.10.0-rc3.
* Introudce dregion->memmap for dmem to add _refcount for each dmem page.
* Enable record_steal_time for dmem before entering guest system.
* Adjust page walking for dmem.

Yulei Zhang (37):
fs: introduce dmemfs module
mm: support direct memory reservation
dmem: implement dmem memory management
dmem: let pat recognize dmem
dmemfs: support mmap for dmemfs
dmemfs: support truncating inode down
dmem: trace core functions
dmem: show some statistic in debugfs
dmemfs: support remote access
dmemfs: introduce max_alloc_try_dpages parameter
mm: export mempolicy interfaces to serve dmem allocator
dmem: introduce mempolicy support
mm, dmem: introduce PFN_DMEM and pfn_t_dmem
mm, dmem: differentiate dmem-pmd and thp-pmd
mm: add pmd_special() check for pmd_trans_huge_lock()
dmemfs: introduce ->split() to dmemfs_vm_ops
mm, dmemfs: support unmap_page_range() for dmemfs pmd
mm: follow_pmd_mask() for dmem huge pmd
mm: gup_huge_pmd() for dmem huge pmd
mm: support dmem huge pmd for vmf_insert_pfn_pmd()
mm: support dmem huge pmd for follow_pfn()
kvm, x86: Distinguish dmemfs page from mmio page
kvm, x86: introduce VM_DMEM for syscall support usage
dmemfs: support hugepage for dmemfs
mm, x86, dmem: fix estimation of reserved page for vaddr_get_pfn() mm, dmem: introduce pud_special() for dmem huge pud support
mm: add pud_special() check to support dmem huge pud mm, dmemfs: support huge_fault() for dmemfs
mm: add follow_pte_pud() to support huge pud look up
dmem: introduce dmem_bitmap_alloc() and dmem_bitmap_free()
dmem: introduce mce handler
mm, dmemfs: register and handle the dmem mce
kvm, x86: enable record_steal_time for dmem
dmem: add dmem unit tests
mm, dmem: introduce dregion->memmap for dmem
vfio: support dmempage refcount for vfio
Add documentation for dmemfs

 Documentation/admin-guide/kernel-parameters.txt |   38 +
 Documentation/filesystems/dmemfs.rst            |   58 ++
 Documentation/filesystems/index.rst             |    1 +
 arch/x86/Kconfig                                |    1 +
 arch/x86/include/asm/pgtable.h                  |   32 +-
 arch/x86/include/asm/pgtable_types.h            |   13 +-
 arch/x86/kernel/setup.c                         |    3 +
 arch/x86/kvm/mmu/mmu.c                          |    1 +
 arch/x86/mm/pat/memtype.c                       |   21 +
 drivers/vfio/vfio_iommu_type1.c                 |   13 +-
 fs/Kconfig                                      |    1 +
 fs/Makefile                                     |    1 +
 fs/dmemfs/Kconfig                               |   16 +
 fs/dmemfs/Makefile                              |    8 +
 fs/dmemfs/inode.c                               | 1060 ++++++++++++++++++++
 fs/dmemfs/trace.h                               |   54 +
 fs/inode.c                                      |    6 +
 include/linux/dmem.h                            |   54 +
 include/linux/fs.h                              |    1 +
 include/linux/huge_mm.h                         |    5 +-
 include/linux/mempolicy.h                       |    3 +
 include/linux/mm.h                              |    9 +
 include/linux/pfn_t.h                           |   17 +-
 include/linux/pgtable.h                         |   22 +
 include/trace/events/dmem.h                     |   85 ++
 include/uapi/linux/magic.h                      |    1 +
 mm/Kconfig                                      |   19 +
 mm/Makefile                                     |    1 +
 mm/dmem.c                                       | 1196 +++++++++++++++++++++++
 mm/dmem_reserve.c                               |  303 ++++++
 mm/gup.c                                        |  101 +-
 mm/huge_memory.c                                |   19 +-
 mm/memory-failure.c                             |   70 +-
 mm/memory.c                                     |   74 +-
 mm/mempolicy.c                                  |    4 +-
 mm/mincore.c                                    |    8 +-
 mm/mprotect.c                                   |    7 +-
 mm/mremap.c                                     |    3 +
 mm/pagewalk.c                                   |    4 +-
 tools/testing/dmem/Kbuild                       |    1 +
 tools/testing/dmem/Makefile                     |   10 +
 tools/testing/dmem/dmem-test.c                  |  184 ++++
 virt/kvm/kvm_main.c                             |   13 +-
 43 files changed, 3483 insertions(+), 58 deletions(-)
 create mode 100644 Documentation/filesystems/dmemfs.rst
 create mode 100644 fs/dmemfs/Kconfig
 create mode 100644 fs/dmemfs/Makefile
 create mode 100644 fs/dmemfs/inode.c
 create mode 100644 fs/dmemfs/trace.h
 create mode 100644 include/linux/dmem.h
 create mode 100644 include/trace/events/dmem.h
 create mode 100644 mm/dmem.c
 create mode 100644 mm/dmem_reserve.c
 create mode 100644 tools/testing/dmem/Kbuild
 create mode 100644 tools/testing/dmem/Makefile
 create mode 100644 tools/testing/dmem/dmem-test.c
Signature · 1 line
1.8.3.1
#1 lore raw

RFC v2 1/37 fs: introduce dmemfs module

From: Yulei Zhang <[email protected]>

dmemfs (Direct Memory filesystem) is device memory or reserved memory based filesystem. This kind of memory is special as it is not managed by kernel and it is not associated with 'struct page'.

The original purpose for dmemfs is to drop the usage of 'struct page' to save extra system memory in public cloud enviornment.

This patch introduces the basic framework of dmemfs and only mkdir and create regular file are supported.

---
 fs/Kconfig                 |   1 +
 fs/Makefile                |   1 +
 fs/dmemfs/Kconfig          |  13 +++
 fs/dmemfs/Makefile         |   7 ++
 fs/dmemfs/inode.c          | 266 +++++++++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/magic.h |   1 +
 6 files changed, 289 insertions(+)
 create mode 100644 fs/dmemfs/Kconfig
 create mode 100644 fs/dmemfs/Makefile
 create mode 100644 fs/dmemfs/inode.c
fs/Kconfig +1 -0
index aa4c122..18e7208 100644
@@ -41,6 +41,7 @@ source "fs/btrfs/Kconfig"
 source "fs/nilfs2/Kconfig"
 source "fs/f2fs/Kconfig"
 source "fs/zonefs/Kconfig"
+source "fs/dmemfs/Kconfig"
 
 config FS_DAX
 	bool "Direct Access (DAX) support"
fs/Makefile +1 -0
index 999d1a2..34747ec 100644
@@ -136,3 +136,4 @@ obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
 obj-$(CONFIG_EROFS_FS)		+= erofs/
 obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
 obj-$(CONFIG_ZONEFS_FS)		+= zonefs/
+obj-$(CONFIG_DMEM_FS)		+= dmemfs/
fs/dmemfs/Kconfig new file +13 -0
new file mode 100644
index 00000000..d2894a5
@@ -0,0 +1,13 @@
+config DMEM_FS
+	tristate "Direct Memory filesystem support"
+	help
+	  dmemfs (Direct Memory filesystem) is device memory or reserved
+	  memory based filesystem. This kind of memory is special as it
+	  is not managed by kernel and it is without 'struct page'.
+
+	  The original purpose of dmemfs is saving extra memory of
+	  'struct page' that reduces the total cost of ownership (TCO)
+	  for cloud providers.
+
+	  To compile this file system support as a module, choose M here: the
+	  module will be called dmemfs.
fs/dmemfs/Makefile new file +7 -0
new file mode 100644
index 00000000..73bdc9c
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the linux dmem-filesystem routines.
+#
+obj-$(CONFIG_DMEM_FS) += dmemfs.o
+
+dmemfs-y += inode.o
fs/dmemfs/inode.c new file +266 -0

266 lines not on this page · raw message

include/uapi/linux/magic.h +1 -0
index f3956fc..3fbd066 100644
@@ -97,5 +97,6 @@
 #define DEVMEM_MAGIC		0x454d444d	/* "DMEM" */
 #define Z3FOLD_MAGIC		0x33
 #define PPC_CMM_MAGIC		0xc7571590
+#define DMEMFS_MAGIC		0x2ace90c6
 
 #endif /* __LINUX_MAGIC_H__ */
Signature · 1 line
1.8.3.1
#3 lore raw

RFC v2 2/37 mm: support direct memory reservation

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Introduce 'dmem=' to reserve system memory for DMEM (direct memory), comparing with 'mem=' and 'memmap', it reserves memory based on the topology of NUMA, for the detailed info, please refer to kernel-parameters.txt

---
 Documentation/admin-guide/kernel-parameters.txt |  38 +++
 arch/x86/kernel/setup.c                         |   3 +
 include/linux/dmem.h                            |  16 ++
 mm/Kconfig                                      |   8 +
 mm/Makefile                                     |   1 +
 mm/dmem.c                                       | 137 +++++++++++
 mm/dmem_reserve.c                               | 303 ++++++++++++++++++++++++
 7 files changed, 506 insertions(+)
 create mode 100644 include/linux/dmem.h
 create mode 100644 mm/dmem.c
 create mode 100644 mm/dmem_reserve.c
Documentation/admin-guide/kernel-parameters.txt +38 -0
index 526d65d..78caf11 100644
@@ -991,6 +991,44 @@
 			The filter can be disabled or changed to another
 			driver later using sysfs.
 
+	dmem=[!]size[KMG]
+			[KNL, NUMA] When CONFIG_DMEM is set, this means
+			the size of memory reserved for dmemfs on each NUMA
+			memory node and 'size' must be aligned to the default
+			alignment that is the size of memory section which is
+			128M by default on x86_64. If set '!', such amount of
+			memory on each node will be owned by kernel and dmemfs
+			owns the rest of memory on each node.
+			Example: Reserve 4G memory on each node for dmemfs
+				dmem = 4G
+
+	dmem=[!]size[KMG]:align[KMG]
+			[KNL, NUMA] Ditto. 'align' should be power of two and
+			not smaller than the default alignment. Also 'size'
+			must be aligned to 'align'.
+			Example: Bad dmem parameter because 'size' misaligned
+				dmem=0x40200000:1G
+
+	dmem=size[KMG]@addr[KMG]
+			[KNL] When CONFIG_DMEM is set, this marks specific
+			memory as reserved for dmemfs. Region of memory will be
+			used by dmemfs, from addr to addr + size. Reserving a
+			certain memory region for kernel is illegal so '!' is
+			forbidden. Should not assign 'addr' to 0 because kernel
+			will occupy fixed memory region beginning at 0 address.
+			Ditto, 'size' and 'addr' must be aligned to default
+			alignment.
+			Example: Exclude memory from 5G-6G for dmemfs.
+				dmem=1G@5G
+
+	dmem=size[KMG]@addr[KMG]:align[KMG]
+			[KNL] Ditto. 'align' should be power of two and
+			not smaller than the default alignment. Also 'size'
+			and 'addr' must be aligned to 'align'. Specially,
+			'@addr' and ':align' could occur in any order.
+			Example: Exclude memory from 5G-6G for dmemfs.
+				dmem=1G:1G@5G
+
 	driver_async_probe=  [KNL]
 			List of driver names to be probed asynchronously.
 			Format: <driver_name1>,<driver_name2>...
arch/x86/kernel/setup.c +3 -0
index 84f581c..9d05e1b 100644
@@ -48,6 +48,7 @@
 #include <asm/unwind.h>
 #include <asm/vsyscall.h>
 #include <linux/vmalloc.h>
+#include <linux/dmem.h>
 
 /*
  * max_low_pfn_mapped: highest directly mapped pfn < 4 GB
@@ -1149,6 +1150,8 @@ void __init setup_arch(char **cmdline_p)
 	if (boot_cpu_has(X86_FEATURE_GBPAGES))
 		hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT);
 
+	dmem_reserve_init();
+
 	/*
 	 * Reserve memory for crash kernel after SRAT is parsed so that it
 	 * won't consume hotpluggable memory.
include/linux/dmem.h new file +16 -0
new file mode 100644
index 00000000..5049322
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LINUX_DMEM_H
+#define _LINUX_DMEM_H
+
+#ifdef CONFIG_DMEM
+int dmem_reserve_init(void);
+void dmem_init(void);
+int dmem_region_register(int node, phys_addr_t start, phys_addr_t end);
+
+#else
+static inline int dmem_reserve_init(void)
+{
+	return 0;
+}
+#endif
+#endif	/* _LINUX_DMEM_H */
mm/Kconfig +8 -0
index d42423f..3a6d408 100644
@@ -226,6 +226,14 @@ config BALLOON_COMPACTION
 	  scenario aforementioned and helps improving memory defragmentation.
 
 #
+# support for direct memory basics
+config DMEM
+	bool "Direct Memory Reservation"
+	depends on SPARSEMEM
+	help
+	  Allow reservation of memory which could be for the dedicated use of dmem.
+	  It's the basis of dmemfs.
+
 # support for memory compaction
 config COMPACTION
 	bool "Allow for memory compaction"
mm/Makefile +1 -0
index d73aed0..775c8518 100644
@@ -120,3 +120,4 @@ obj-$(CONFIG_MEMFD_CREATE) += memfd.o
 obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o
 obj-$(CONFIG_PTDUMP_CORE) += ptdump.o
 obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o
+obj-$(CONFIG_DMEM) += dmem.o dmem_reserve.o
mm/dmem.c new file +137 -0

137 lines not on this page · raw message

mm/dmem_reserve.c new file +303 -0

303 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#4 lore raw

RFC v2 3/37 dmem: implement dmem memory management

In reply to [email protected]

From: Yulei Zhang <[email protected]>

The belowing figure shows the topology of dmem memory management, it reserves a few memory regions from the numa nodes, and in each region it leverages the bitmap to track the actual memory usage.

         +------+-+-------+---------+
         | Node0| | ...   | NodeN   |
         +--/-\-+-+-------+---------+
           /   \
      +---v----+v-----+----------+
      |region 0| ...  | region n |
      +--/---\-+------+----------+
        /     \
    +-+v+------v-------+-+-+-+
    | | |   bitmap     | | | |
    +-+-+--------------+-+-+-+

It introduces the interfaces to manage dmem pages that include:
- dmem_region_register(), it registers the reserved memory to the dmem management system

- dmem_alloc_init(), initiate dmem allocator, note the page size the allocator used isn't the same thing with the alignment used to reserve dmem memory

- dmem_alloc_pages_vma() and dmem_free_pages() are the interfaces allocating and freeing dmem memory, multiple pages can be allocated at one time, but it should be power of two

---
 include/linux/dmem.h |   3 +
 mm/dmem.c            | 674 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 677 insertions(+)
include/linux/dmem.h +3 -0
index 5049322..476a82e 100644
@@ -7,6 +7,9 @@
 void dmem_init(void);
 int dmem_region_register(int node, phys_addr_t start, phys_addr_t end);
 
+int dmem_alloc_init(unsigned long dpage_shift);
+void dmem_alloc_uinit(void);
+
 #else
 static inline int dmem_reserve_init(void)
 {
mm/dmem.c +674 -0

688 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#5 lore raw

RFC v2 4/37 dmem: let pat recognize dmem

In reply to [email protected]

From: Yulei Zhang <[email protected]>

x86 pat uses 'struct page' by only checking if it's system ram, however it is not true if dmem is used, let's teach pat to recognize this case if it is ram but it is !pfn_valid()

We always use WB for dmem and any attempt to change this behavior will be rejected and WARN_ON is triggered

---
 arch/x86/mm/pat/memtype.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
arch/x86/mm/pat/memtype.c +21 -0
index 8f665c3..fd8a298 100644
@@ -511,6 +511,13 @@ static int reserve_ram_pages_type(u64 start, u64 end,
 	for (pfn = (start >> PAGE_SHIFT); pfn < (end >> PAGE_SHIFT); ++pfn) {
 		enum page_cache_mode type;
 
+		/*
+		 * it's dmem if it's ram but not 'struct page' backend,
+		 * we always use WB
+		 */
+		if (WARN_ON(!pfn_valid(pfn)))
+			return -EBUSY;
+
 		page = pfn_to_page(pfn);
 		type = get_page_memtype(page);
 		if (type != _PAGE_CACHE_MODE_WB) {
@@ -539,6 +546,13 @@ static int free_ram_pages_type(u64 start, u64 end)
 	u64 pfn;
 
 	for (pfn = (start >> PAGE_SHIFT); pfn < (end >> PAGE_SHIFT); ++pfn) {
+		/*
+		 * it's dmem, see the comments in
+		 * reserve_ram_pages_type()
+		 */
+		if (WARN_ON(!pfn_valid(pfn)))
+			continue;
+
 		page = pfn_to_page(pfn);
 		set_page_memtype(page, _PAGE_CACHE_MODE_WB);
 	}
@@ -714,6 +728,13 @@ static enum page_cache_mode lookup_memtype(u64 paddr)
 	if (pat_pagerange_is_ram(paddr, paddr + PAGE_SIZE)) {
 		struct page *page;
 
+		/*
+		 * dmem always uses WB, see the comments in
+		 * reserve_ram_pages_type()
+		 */
+		if (!pfn_valid(paddr >> PAGE_SHIFT))
+			return rettype;
+
 		page = pfn_to_page(paddr >> PAGE_SHIFT);
 		return get_page_memtype(page);
 	}
Signature · 1 line
1.8.3.1
#6 lore raw

RFC v2 5/37 dmemfs: support mmap for dmemfs

In reply to [email protected]

From: Yulei Zhang <[email protected]>

This patch adds mmap support. Note the file will be extended if it's beyond mmap's offset, that drops the requirement of write() operation, however, it has not supported cutting file down yet.

---
 fs/dmemfs/inode.c    | 343 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 include/linux/dmem.h |  10 ++
 2 files changed, 351 insertions(+), 2 deletions(-)
fs/dmemfs/inode.c +341 -2
index 0aa3d3b..7b6e51d 100644
@@ -26,6 +26,7 @@
 #include <linux/pagevec.h>
 #include <linux/fs_parser.h>
 #include <linux/seq_file.h>
+#include <linux/dmem.h>
 
 MODULE_AUTHOR("Tencent Corporation");
 MODULE_LICENSE("GPL v2");
@@ -102,7 +103,255 @@ static int dmemfs_mkdir(struct inode *dir, struct dentry *dentry,
 	.getattr = simple_getattr,
 };
 
+static unsigned long dmem_pgoff_to_index(struct inode *inode, pgoff_t pgoff)
+{
+	struct super_block *sb = inode->i_sb;
+
+	return pgoff >> (sb->s_blocksize_bits - PAGE_SHIFT);
+}
+
+static void *dmem_addr_to_entry(struct inode *inode, phys_addr_t addr)
+{
+	struct super_block *sb = inode->i_sb;
+
+	addr >>= sb->s_blocksize_bits;
+	return xa_mk_value(addr);
+}
+
+static phys_addr_t dmem_entry_to_addr(struct inode *inode, void *entry)
+{
+	struct super_block *sb = inode->i_sb;
+
+	WARN_ON(!xa_is_value(entry));
+	return xa_to_value(entry) << sb->s_blocksize_bits;
+}
+
+static unsigned long
+dmem_addr_to_pfn(struct inode *inode, phys_addr_t addr, pgoff_t pgoff,
+		 unsigned int fault_shift)
+{
+	struct super_block *sb = inode->i_sb;
+	unsigned long pfn = addr >> PAGE_SHIFT;
+	unsigned long mask;
+
+	mask = (1UL << ((unsigned int)sb->s_blocksize_bits - fault_shift)) - 1;
+	mask <<= fault_shift - PAGE_SHIFT;
+
+	return pfn + (pgoff & mask);
+}
+
+static inline unsigned long dmem_page_size(struct inode *inode)
+{
+	return inode->i_sb->s_blocksize;
+}
+
+static int check_inode_size(struct inode *inode, loff_t offset)
+{
+	WARN_ON_ONCE(!rcu_read_lock_held());
+
+	if (offset >= i_size_read(inode))
+		return -EINVAL;
+
+	return 0;
+}
+
+static unsigned
+dmemfs_find_get_entries(struct address_space *mapping, unsigned long start,
+			unsigned int nr_entries, void **entries,
+			unsigned long *indices)
+{
+	XA_STATE(xas, &mapping->i_pages, start);
+
+	void *entry;
+	unsigned int ret = 0;
+
+	if (!nr_entries)
+		return 0;
+
+	rcu_read_lock();
+
+	xas_for_each(&xas, entry, ULONG_MAX) {
+		if (xas_retry(&xas, entry))
+			continue;
+
+		if (xa_is_value(entry))
+			goto export;
+
+		if (unlikely(entry != xas_reload(&xas)))
+			goto retry;
+
+export:
+		indices[ret] = xas.xa_index;
+		entries[ret] = entry;
+		if (++ret == nr_entries)
+			break;
+		continue;
+retry:
+		xas_reset(&xas);
+	}
+	rcu_read_unlock();
+	return ret;
+}
+
+static void *find_radix_entry_or_next(struct address_space *mapping,
+				      unsigned long start,
+				      unsigned long *eindex)
+{
+	void *entry = NULL;
+
+	dmemfs_find_get_entries(mapping, start, 1, &entry, eindex);
+	return entry;
+}
+
+/*
+ * find the entry in radix tree based on @index, create it if
+ * it does not exist
+ *
+ * return the entry with rcu locked, otherwise ERR_PTR()
+ * is returned
+ */
+static void *
+radix_get_create_entry(struct vm_area_struct *vma, unsigned long fault_addr,
+		       struct inode *inode, pgoff_t pgoff)
+{
+	struct address_space *mapping = inode->i_mapping;
+	unsigned long eindex, index;
+	loff_t offset;
+	phys_addr_t addr;
+	gfp_t gfp_masks = mapping_gfp_mask(mapping) & ~__GFP_HIGHMEM;
+	void *entry;
+	unsigned int try_dpages, dpages;
+	int ret;
+
+retry:
+	offset = ((loff_t)pgoff << PAGE_SHIFT);
+	index = dmem_pgoff_to_index(inode, pgoff);
+	rcu_read_lock();
+	ret = check_inode_size(inode, offset);
+	if (ret) {
+		rcu_read_unlock();
+		return ERR_PTR(ret);
+	}
+
+	try_dpages = dmem_pgoff_to_index(inode, (i_size_read(inode) - offset)
+				     >> PAGE_SHIFT);
+	entry = find_radix_entry_or_next(mapping, index, &eindex);
+	if (entry) {
+		WARN_ON(!xa_is_value(entry));
+		if (eindex == index)
+			return entry;
+
+		WARN_ON(eindex <= index);
+		try_dpages = eindex - index;
+	}
+	rcu_read_unlock();
+
+	/* entry does not exist, create it */
+	addr = dmem_alloc_pages_vma(vma, fault_addr, try_dpages, &dpages);
+	if (!addr) {
+		/*
+		 * do not return -ENOMEM as that will trigger OOM,
+		 * it is useless for reclaiming dmem page
+		 */
+		ret = -EINVAL;
+		goto exit;
+	}
+
+	try_dpages = dpages;
+	while (dpages) {
+		rcu_read_lock();
+		ret = check_inode_size(inode, offset);
+		if (ret)
+			goto unlock_rcu;
+
+		entry = dmem_addr_to_entry(inode, addr);
+		entry = xa_store(&mapping->i_pages, index, entry, gfp_masks);
+		if (!xa_is_err(entry)) {
+			addr += inode->i_sb->s_blocksize;
+			offset += inode->i_sb->s_blocksize;
+			dpages--;
+			mapping->nrexceptional++;
+			index++;
+		}
+
+unlock_rcu:
+		rcu_read_unlock();
+		if (ret)
+			break;
+	}
+
+	if (dpages)
+		dmem_free_pages(addr, dpages);
+
+	/* we have created some entries, let's retry it */
+	if (ret == -EEXIST || try_dpages != dpages)
+		goto retry;
+exit:
+	return ERR_PTR(ret);
+}
+
+static void radix_put_entry(void)
+{
+	rcu_read_unlock();
+}
+
+static vm_fault_t dmemfs_fault(struct vm_fault *vmf)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	struct inode *inode = file_inode(vma->vm_file);
+	phys_addr_t addr;
+	void *entry;
+	int ret;
+
+	if (vmf->pgoff > (MAX_LFS_FILESIZE >> PAGE_SHIFT))
+		return VM_FAULT_SIGBUS;
+
+	entry = radix_get_create_entry(vma, (unsigned long)vmf->address,
+				       inode, vmf->pgoff);
+	if (IS_ERR(entry)) {
+		ret = PTR_ERR(entry);
+		goto exit;
+	}
+
+	addr = dmem_entry_to_addr(inode, entry);
+	ret = vmf_insert_pfn(vma, (unsigned long)vmf->address,
+			    dmem_addr_to_pfn(inode, addr, vmf->pgoff,
+					     PAGE_SHIFT));
+	radix_put_entry();
+
+exit:
+	return ret;
+}
+
+static unsigned long dmemfs_pagesize(struct vm_area_struct *vma)
+{
+	return dmem_page_size(file_inode(vma->vm_file));
+}
+
+static const struct vm_operations_struct dmemfs_vm_ops = {
+	.fault = dmemfs_fault,
+	.pagesize = dmemfs_pagesize,
+};
+
+int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct inode *inode = file_inode(file);
+
+	if (vma->vm_pgoff & ((dmem_page_size(inode) - 1) >> PAGE_SHIFT))
+		return -EINVAL;
+
+	if (!(vma->vm_flags & VM_SHARED))
+		return -EINVAL;
+
+	vma->vm_flags |= VM_PFNMAP;
+
+	file_accessed(file);
+	vma->vm_ops = &dmemfs_vm_ops;
+	return 0;
+}
+
 static const struct file_operations dmemfs_file_operations = {
+	.mmap = dmemfs_file_mmap,
 };
 
 static int dmemfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
@@ -180,9 +429,86 @@ static int dmemfs_statfs(struct dentry *dentry, struct kstatfs *buf)
 	return 0;
 }
 
+/*
+ * should make sure the dmem page in the dropped region is not
+ * being mapped by any process
+ */
+static void inode_drop_dpages(struct inode *inode, loff_t start, loff_t end)
+{
+	struct address_space *mapping = inode->i_mapping;
+	struct pagevec pvec;
+	unsigned long istart, iend, indices[PAGEVEC_SIZE];
+	int i;
+
+	/* we never use normap page */
+	WARN_ON(mapping->nrpages);
+
+	/* if no dpage is allocated for the inode */
+	if (!mapping->nrexceptional)
+		return;
+
+	istart = dmem_pgoff_to_index(inode, start >> PAGE_SHIFT);
+	iend = dmem_pgoff_to_index(inode, end >> PAGE_SHIFT);
+	pagevec_init(&pvec);
+	while (istart < iend) {
+		pvec.nr = dmemfs_find_get_entries(mapping, istart,
+				min(iend - istart,
+				(unsigned long)PAGEVEC_SIZE),
+				(void **)pvec.pages,
+				indices);
+		if (!pvec.nr)
+			break;
+
+		for (i = 0; i < pagevec_count(&pvec); i++) {
+			phys_addr_t addr;
+
+			istart = indices[i];
+			if (istart >= iend)
+				break;
+
+			xa_erase(&mapping->i_pages, istart);
+			mapping->nrexceptional--;
+
+			addr = dmem_entry_to_addr(inode, pvec.pages[i]);
+			dmem_free_page(addr);
+		}
+
+		/*
+		 * only exception entries in pagevec, it's safe to
+		 * reinit it
+		 */
+		pagevec_reinit(&pvec);
+		cond_resched();
+		istart++;
+	}
+}
+
+static void dmemfs_evict_inode(struct inode *inode)
+{
+	/* no VMA works on it */
+	WARN_ON(!RB_EMPTY_ROOT(&inode->i_data.i_mmap.rb_root));
+
+	inode_drop_dpages(inode, 0, LLONG_MAX);
+	clear_inode(inode);
+}
+
+/*
+ * Display the mount options in /proc/mounts.
+ */
+static int dmemfs_show_options(struct seq_file *m, struct dentry *root)
+{
+	struct dmemfs_fs_info *fsi = root->d_sb->s_fs_info;
+
+	if (check_dpage_size(fsi->mount_opts.dpage_size))
+		seq_printf(m, ",pagesize=%lx", fsi->mount_opts.dpage_size);
+	return 0;
+}
+
 static const struct super_operations dmemfs_ops = {
 	.statfs	= dmemfs_statfs,
+	.evict_inode = dmemfs_evict_inode,
 	.drop_inode = generic_delete_inode,
+	.show_options = dmemfs_show_options,
 };
 
 static int
@@ -190,6 +516,7 @@ static int dmemfs_statfs(struct dentry *dentry, struct kstatfs *buf)
 {
 	struct inode *inode;
 	struct dmemfs_fs_info *fsi = sb->s_fs_info;
+	int ret;
 
 	sb->s_maxbytes = MAX_LFS_FILESIZE;
 	sb->s_blocksize = fsi->mount_opts.dpage_size;
@@ -198,11 +525,17 @@ static int dmemfs_statfs(struct dentry *dentry, struct kstatfs *buf)
 	sb->s_op = &dmemfs_ops;
 	sb->s_time_gran = 1;
 
+	ret = dmem_alloc_init(sb->s_blocksize_bits);
+	if (ret)
+		return ret;
+
 	inode = dmemfs_get_inode(sb, NULL, S_IFDIR);
 	sb->s_root = d_make_root(inode);
-	if (!sb->s_root)
-		return -ENOMEM;
 
+	if (!sb->s_root) {
+		dmem_alloc_uinit();
+		return -ENOMEM;
+	}
 	return 0;
 }
 
@@ -238,7 +571,13 @@ int dmemfs_init_fs_context(struct fs_context *fc)
 
 static void dmemfs_kill_sb(struct super_block *sb)
 {
+	bool has_inode = !!sb->s_root;
+
 	kill_litter_super(sb);
+
+	/* do not uninit dmem allocator if mount failed */
+	if (has_inode)
+		dmem_alloc_uinit();
 }
 
 static struct file_system_type dmemfs_fs_type = {
include/linux/dmem.h +10 -0
index 476a82e..8682d63 100644
@@ -10,6 +10,16 @@
 int dmem_alloc_init(unsigned long dpage_shift);
 void dmem_alloc_uinit(void);
 
+phys_addr_t
+dmem_alloc_pages_nodemask(int nid, nodemask_t *nodemask, unsigned int try_max,
+			  unsigned int *result_nr);
+
+phys_addr_t
+dmem_alloc_pages_vma(struct vm_area_struct *vma, unsigned long addr,
+		     unsigned int try_max, unsigned int *result_nr);
+
+void dmem_free_pages(phys_addr_t addr, unsigned int dpages_nr);
+#define dmem_free_page(addr)	dmem_free_pages(addr, 1)
 #else
 static inline int dmem_reserve_init(void)
 {
Signature · 1 line
1.8.3.1
#8 lore raw

RFC v2 6/37 dmemfs: support truncating inode down

In reply to [email protected]

From: Yulei Zhang <[email protected]>

To support cut inode down, it will
introduce the race between page fault handler and truncating handler as the entry to be deleted is being mapped into process's VMA

in order to make page fault faster (as it's the hot path), we use RCU to sync these two handlers. When inode's size is updated, the handler makes sure the new size is visible to page fault handler who will not use truncated entry anymore and will not create new entry in that region

---
 fs/dmemfs/inode.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 66 insertions(+), 1 deletion(-)
fs/dmemfs/inode.c +66 -1
index 7b6e51d..9ec62dc 100644
@@ -98,8 +98,73 @@ static int dmemfs_mkdir(struct inode *dir, struct dentry *dentry,
 	.rename		= simple_rename,
 };
 
+static void inode_drop_dpages(struct inode *inode, loff_t start, loff_t end);
+
+static int dmemfs_truncate(struct inode *inode, loff_t newsize)
+{
+	struct super_block *sb = inode->i_sb;
+	loff_t current_size;
+
+	if (newsize & ((1 << sb->s_blocksize_bits) - 1))
+		return -EINVAL;
+
+	current_size = i_size_read(inode);
+	i_size_write(inode, newsize);
+
+	if (newsize >= current_size)
+		return 0;
+
+	/* it cuts the inode down */
+
+	/*
+	 * we should make sure inode->i_size has been updated before
+	 * unmapping and dropping radix entries, so that other sides
+	 * can not create new i_mapping entry beyond inode->i_size
+	 * and the radix entry in the truncated region is not being
+	 * used
+	 *
+	 * see the comments in dmemfs_fault()
+	 */
+	synchronize_rcu();
+
+	/*
+	 * should unmap all mapping first as dmem pages are freed in
+	 * inode_drop_dpages()
+	 *
+	 * after that, dmem page in the truncated region is not used
+	 * by any process
+	 */
+	unmap_mapping_range(inode->i_mapping, newsize, 0, 1);
+
+	inode_drop_dpages(inode, newsize, LLONG_MAX);
+	return 0;
+}
+
+/*
+ * same logic as simple_setattr but we need to handle ftruncate
+ * carefully as we inserted self-defined entry into radix tree
+ */
+static int dmemfs_setattr(struct dentry *dentry, struct iattr *iattr)
+{
+	struct inode *inode = dentry->d_inode;
+	int error;
+
+	error = setattr_prepare(dentry, iattr);
+	if (error)
+		return error;
+
+	if (iattr->ia_valid & ATTR_SIZE) {
+		error = dmemfs_truncate(inode, iattr->ia_size);
+		if (error)
+			return error;
+	}
+	setattr_copy(inode, iattr);
+	mark_inode_dirty(inode);
+	return 0;
+}
+
 static const struct inode_operations dmemfs_file_inode_operations = {
-	.setattr = simple_setattr,
+	.setattr = dmemfs_setattr,
 	.getattr = simple_getattr,
 };
 
Signature · 1 line
1.8.3.1
#9 lore raw

RFC v2 7/37 dmem: trace core functions

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Add tracepoints for dmem alloc_init, alloc and free functions, that helps us to figure out what is happening inside dmem allocator

---
 fs/dmemfs/Makefile          |  1 +
 fs/dmemfs/inode.c           |  5 ++++
 fs/dmemfs/trace.h           | 54 +++++++++++++++++++++++++++++++++++
 include/trace/events/dmem.h | 68 +++++++++++++++++++++++++++++++++++++++++++++
 mm/dmem.c                   |  6 ++++
 5 files changed, 134 insertions(+)
 create mode 100644 fs/dmemfs/trace.h
 create mode 100644 include/trace/events/dmem.h
fs/dmemfs/Makefile +1 -0
index 73bdc9c..0b36d03 100644
@@ -2,6 +2,7 @@
 #
 # Makefile for the linux dmem-filesystem routines.
 #
+ccflags-y += -I $(srctree)/$(src)		# needed for trace events
 obj-$(CONFIG_DMEM_FS) += dmemfs.o
 
 dmemfs-y += inode.o
fs/dmemfs/inode.c +5 -0
index 9ec62dc..7723b58 100644
@@ -31,6 +31,9 @@
 MODULE_AUTHOR("Tencent Corporation");
 MODULE_LICENSE("GPL v2");
 
+#define CREATE_TRACE_POINTS
+#include "trace.h"
+
 struct dmemfs_mount_opts {
 	unsigned long dpage_size;
 };
@@ -336,6 +339,7 @@ static void *find_radix_entry_or_next(struct address_space *mapping,
 			offset += inode->i_sb->s_blocksize;
 			dpages--;
 			mapping->nrexceptional++;
+			trace_dmemfs_radix_tree_insert(index, entry);
 			index++;
 		}
 
@@ -532,6 +536,7 @@ static void inode_drop_dpages(struct inode *inode, loff_t start, loff_t end)
 				break;
 
 			xa_erase(&mapping->i_pages, istart);
+			trace_dmemfs_radix_tree_delete(istart, pvec.pages[i]);
 			mapping->nrexceptional--;
 
 			addr = dmem_entry_to_addr(inode, pvec.pages[i]);
fs/dmemfs/trace.h new file +54 -0

54 lines not on this page · raw message

include/trace/events/dmem.h new file +68 -0

68 lines not on this page · raw message

mm/dmem.c +6 -0
index a77a064..aa34bf2 100644
@@ -18,6 +18,8 @@
 #include <linux/debugfs.h>
 #include <linux/notifier.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/dmem.h>
 /*
  * There are two kinds of page in dmem management:
  * - nature page, it's the CPU's page size, i.e, 4K on x86
@@ -559,6 +561,8 @@ int dmem_alloc_init(unsigned long dpage_shift)
 
 	mutex_lock(&dmem_pool.lock);
 
+	trace_dmem_alloc_init(dpage_shift);
+
 	if (dmem_pool.dpage_shift) {
 		/*
 		 * double init on the same page size is okay
@@ -686,6 +690,7 @@ int dmem_alloc_init(unsigned long dpage_shift)
 			}
 		}
 
+		trace_dmem_alloc_pages_node(addr, node, try_max, *result_nr);
 		mutex_unlock(&dmem_pool.lock);
 	}
 	return addr;
@@ -791,6 +796,7 @@ void dmem_free_pages(phys_addr_t addr, unsigned int dpages_nr)
 
 	mutex_lock(&dmem_pool.lock);
 
+	trace_dmem_free_pages(addr, dpages_nr);
 	WARN_ON(!dmem_pool.dpage_shift);
 
 	dregion = find_dmem_region(addr, &pdnode);
Signature · 1 line
1.8.3.1
#10 lore raw

RFC v2 8/37 dmem: show some statistic in debugfs

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Create 'dmem' directory under debugfs and show some statistic for dmem pool, track total and free dpages on dmem pool and each numa node.

---
 mm/Kconfig |   8 +++++
 mm/dmem.c  | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 107 insertions(+), 1 deletion(-)
mm/Kconfig +8 -0
index 3a6d408..4dd8896 100644
@@ -234,6 +234,14 @@ config DMEM
 	  Allow reservation of memory which could be for the dedicated use of dmem.
 	  It's the basis of dmemfs.
 
+config DMEM_DEBUG_FS
+	bool "Enable debug information for direct memory"
+	depends on DMEM && DEBUG_FS
+	help
+	  This option enables showing various statistics of direct memory
+	  in debugfs filesystem.
+
+#
 # support for memory compaction
 config COMPACTION
 	bool "Allow for memory compaction"
mm/dmem.c +99 -1

112 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#11 lore raw

RFC v2 9/37 dmemfs: support remote access

In reply to [email protected]

From: Yulei Zhang <[email protected]>

It is required by ptrace_writedata and ptrace_readdata to access dmem memory remotely. The typical user is gdb, after this patch, gdb is able to read & write memory owned by the attached process

---
 fs/dmemfs/inode.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)
fs/dmemfs/inode.c +46 -0
index 7723b58..3192f31 100644
@@ -364,6 +364,51 @@ static void radix_put_entry(void)
 	rcu_read_unlock();
 }
 
+static bool check_vma_access(struct vm_area_struct *vma, int write)
+{
+	vm_flags_t vm_flags = write ? VM_WRITE : VM_READ;
+
+	return !!(vm_flags & vma->vm_flags);
+}
+
+static int
+dmemfs_access_dmem(struct vm_area_struct *vma, unsigned long addr,
+		   void *buf, int len, int write)
+{
+	struct inode *inode = file_inode(vma->vm_file);
+	struct super_block *sb = inode->i_sb;
+	void *entry, *maddr;
+	int offset, pgoff;
+
+	if (!check_vma_access(vma, write))
+		return -EACCES;
+
+	pgoff = linear_page_index(vma, addr);
+	if (pgoff > (MAX_LFS_FILESIZE >> PAGE_SHIFT))
+		return -EFAULT;
+
+	entry = radix_get_create_entry(vma, addr, inode, pgoff);
+	if (IS_ERR(entry))
+		return PTR_ERR(entry);
+
+	offset = addr & (sb->s_blocksize - 1);
+	addr = dmem_entry_to_addr(inode, entry);
+
+	/*
+	 * it is not beyond vma's region as the vma should be aligned
+	 * to blocksize
+	 */
+	len = min(len, (int)(sb->s_blocksize - offset));
+	maddr = __va(addr);
+	if (write)
+		memcpy(maddr + offset, buf, len);
+	else
+		memcpy(buf, maddr + offset, len);
+	radix_put_entry();
+
+	return len;
+}
+
 static vm_fault_t dmemfs_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
@@ -400,6 +445,7 @@ static unsigned long dmemfs_pagesize(struct vm_area_struct *vma)
 static const struct vm_operations_struct dmemfs_vm_ops = {
 	.fault = dmemfs_fault,
 	.pagesize = dmemfs_pagesize,
+	.access = dmemfs_access_dmem,
 };
 
 int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
Signature · 1 line
1.8.3.1
#12 lore raw

RFC v2 10/37 dmemfs: introduce max_alloc_try_dpages parameter

In reply to [email protected]

From: Yulei Zhang <[email protected]>

It specifies the dmem page number allocated at one time, then multiple radix entries can be created. That will relief the allocation pressure and make page fault more fast.

However that could cause no dmem page mmapped to userspace even if there are some free dmem pages.

Set it to 1 to completely disable this behavior.

---
 fs/dmemfs/inode.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
fs/dmemfs/inode.c +41 -0
index 3192f31..443f2e1 100644
@@ -34,6 +34,8 @@
 #define CREATE_TRACE_POINTS
 #include "trace.h"
 
+static uint __read_mostly max_alloc_try_dpages = 1;
+
 struct dmemfs_mount_opts {
 	unsigned long dpage_size;
 };
@@ -46,6 +48,44 @@ enum dmemfs_param {
 	Opt_dpagesize,
 };
 
+static int
+max_alloc_try_dpages_set(const char *val, const struct kernel_param *kp)
+{
+	uint sval;
+	int ret;
+
+	ret = kstrtouint(val, 0, &sval);
+	if (ret)
+		return ret;
+
+	/* should be 1 at least */
+	if (!sval)
+		return -EINVAL;
+
+	max_alloc_try_dpages = sval;
+	return 0;
+}
+
+static struct kernel_param_ops alloc_max_try_dpages_ops = {
+	.set = max_alloc_try_dpages_set,
+	.get = param_get_uint,
+};
+
+/*
+ * it specifies the dmem page number allocated at one time, then
+ * multiple radix entries can be created. That will relief the
+ * allocation pressure and make page fault more fast.
+ *
+ * however that could cause no dmem page mmapped to userspace
+ * even if there are some free dmem pages
+ *
+ * set it to 1 to completely disable this behavior
+ */
+fs_param_cb(max_alloc_try_dpages, &alloc_max_try_dpages_ops,
+	    &max_alloc_try_dpages, 0644);
+__MODULE_PARM_TYPE(max_alloc_try_dpages, "uint");
+MODULE_PARM_DESC(max_alloc_try_dpages, "Set the dmem page number allocated at one time, should be 1 at least");
+
 const struct fs_parameter_spec dmemfs_fs_parameters[] = {
 	fsparam_string("pagesize", Opt_dpagesize),
 	{}
@@ -314,6 +354,7 @@ static void *find_radix_entry_or_next(struct address_space *mapping,
 	}
 	rcu_read_unlock();
 
+	try_dpages = min(try_dpages, max_alloc_try_dpages);
 	/* entry does not exist, create it */
 	addr = dmem_alloc_pages_vma(vma, fault_addr, try_dpages, &dpages);
 	if (!addr) {
Signature · 1 line
1.8.3.1
#13 lore raw

RFC v2 11/37 mm: export mempolicy interfaces to serve dmem allocator

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Export interface interleave_nid() to serve dmem allocator.

---
 include/linux/mempolicy.h | 3 +++
 mm/mempolicy.c            | 4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)
include/linux/mempolicy.h +3 -0
index 5f1c74d..4789661 100644
@@ -139,6 +139,9 @@ struct mempolicy *mpol_shared_policy_lookup(struct shared_policy *sp,
 struct mempolicy *get_task_policy(struct task_struct *p);
 struct mempolicy *__get_vma_policy(struct vm_area_struct *vma,
 		unsigned long addr);
+struct mempolicy *get_vma_policy(struct vm_area_struct *vma, unsigned long addr);
+unsigned interleave_nid(struct mempolicy *pol, struct vm_area_struct *vma,
+			unsigned long addr, int shift);
 bool vma_policy_mof(struct vm_area_struct *vma);
 
 extern void numa_default_policy(void);
mm/mempolicy.c +2 -2
index 3ca4898..efd80e5 100644
@@ -1813,7 +1813,7 @@ struct mempolicy *__get_vma_policy(struct vm_area_struct *vma,
  * freeing by another task.  It is the caller's responsibility to free the
  * extra reference for shared policies.
  */
-static struct mempolicy *get_vma_policy(struct vm_area_struct *vma,
+struct mempolicy *get_vma_policy(struct vm_area_struct *vma,
 						unsigned long addr)
 {
 	struct mempolicy *pol = __get_vma_policy(vma, addr);
@@ -1978,7 +1978,7 @@ static unsigned offset_il_node(struct mempolicy *pol, unsigned long n)
 }
 
 /* Determine a node number for interleave */
-static inline unsigned interleave_nid(struct mempolicy *pol,
+unsigned interleave_nid(struct mempolicy *pol,
 		 struct vm_area_struct *vma, unsigned long addr, int shift)
 {
 	if (vma) {
Signature · 1 line
1.8.3.1
#14 lore raw

RFC v2 12/37 dmem: introduce mempolicy support

In reply to [email protected]

From: Yulei Zhang <[email protected]>

It adds mempolicy support for dmem to allocates memory from mempolicy specified nodes.

---
 arch/x86/Kconfig                     |  1 +
 arch/x86/include/asm/pgtable.h       |  7 ++++
 arch/x86/include/asm/pgtable_types.h | 13 +++++++-
 fs/dmemfs/Kconfig                    |  3 ++
 include/linux/pgtable.h              |  7 ++++
 mm/Kconfig                           |  3 ++
 mm/dmem.c                            | 63 ++++++++++++++++++++++++++++++++++--
 7 files changed, 94 insertions(+), 3 deletions(-)
arch/x86/Kconfig +1 -0
index f6946b8..9ccee76 100644
@@ -73,6 +73,7 @@ config X86
 	select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
 	select ARCH_HAS_PMEM_API		if X86_64
 	select ARCH_HAS_PTE_DEVMAP		if X86_64
+	select ARCH_HAS_PTE_DMEM		if X86_64
 	select ARCH_HAS_PTE_SPECIAL
 	select ARCH_HAS_UACCESS_FLUSHCACHE	if X86_64
 	select ARCH_HAS_COPY_MC			if X86_64
arch/x86/include/asm/pgtable.h +7 -0
index a02c672..dd4aff6 100644
@@ -452,6 +452,13 @@ static inline pmd_t pmd_mkdevmap(pmd_t pmd)
 	return pmd_set_flags(pmd, _PAGE_DEVMAP);
 }
 
+#ifdef CONFIG_ARCH_HAS_PTE_DMEM
+static inline pmd_t pmd_mkdmem(pmd_t pmd)
+{
+	return pmd_set_flags(pmd, _PAGE_SPECIAL | _PAGE_DMEM);
+}
+#endif
+
 static inline pmd_t pmd_mkhuge(pmd_t pmd)
 {
 	return pmd_set_flags(pmd, _PAGE_PSE);
arch/x86/include/asm/pgtable_types.h +12 -1
index 816b31c..ee4cae1 100644
@@ -23,6 +23,15 @@
 #define _PAGE_BIT_SOFTW2	10	/* " */
 #define _PAGE_BIT_SOFTW3	11	/* " */
 #define _PAGE_BIT_PAT_LARGE	12	/* On 2MB or 1GB pages */
+#define _PAGE_BIT_DMEM		57	/* Flag used to indicate dmem pmd.
+					 * Since _PAGE_BIT_SPECIAL is defined
+					 * same as _PAGE_BIT_CPA_TEST, we can
+					 * not only use _PAGE_BIT_SPECIAL, so
+					 * add _PAGE_BIT_DMEM to help
+					 * indicate it. Since dmem pte will
+					 * never be splitting, setting
+					 * _PAGE_BIT_SPECIAL for pte is enough.
+					 */
 #define _PAGE_BIT_SOFTW4	58	/* available for programmer */
 #define _PAGE_BIT_PKEY_BIT0	59	/* Protection Keys, bit 1/4 */
 #define _PAGE_BIT_PKEY_BIT1	60	/* Protection Keys, bit 2/4 */
@@ -112,9 +121,11 @@
 #if defined(CONFIG_X86_64) || defined(CONFIG_X86_PAE)
 #define _PAGE_NX	(_AT(pteval_t, 1) << _PAGE_BIT_NX)
 #define _PAGE_DEVMAP	(_AT(u64, 1) << _PAGE_BIT_DEVMAP)
+#define _PAGE_DMEM	(_AT(u64, 1) << _PAGE_BIT_DMEM)
 #else
 #define _PAGE_NX	(_AT(pteval_t, 0))
 #define _PAGE_DEVMAP	(_AT(pteval_t, 0))
+#define _PAGE_DMEM	(_AT(pteval_t, 0))
 #endif
 
 #define _PAGE_PROTNONE	(_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE)
@@ -128,7 +139,7 @@
 #define _PAGE_CHG_MASK	(PTE_PFN_MASK | _PAGE_PCD | _PAGE_PWT |		\
 			 _PAGE_SPECIAL | _PAGE_ACCESSED | _PAGE_DIRTY |	\
 			 _PAGE_SOFT_DIRTY | _PAGE_DEVMAP | _PAGE_ENC |  \
-			 _PAGE_UFFD_WP)
+			 _PAGE_UFFD_WP | _PAGE_DMEM)
 #define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE)
 
 /*
fs/dmemfs/Kconfig +3 -0
index d2894a5..19ca391 100644
@@ -1,5 +1,8 @@
 config DMEM_FS
 	tristate "Direct Memory filesystem support"
+	depends on DMEM
+	depends on TRANSPARENT_HUGEPAGE
+	depends on ARCH_HAS_PTE_DMEM
 	help
 	  dmemfs (Direct Memory filesystem) is device memory or reserved
 	  memory based filesystem. This kind of memory is special as it
include/linux/pgtable.h +7 -0
index 71125a4..9e65694 100644
@@ -1157,6 +1157,13 @@ static inline int pud_trans_unstable(pud_t *pud)
 #endif
 }
 
+#ifndef CONFIG_ARCH_HAS_PTE_DMEM
+static inline pmd_t pmd_mkdmem(pmd_t pmd)
+{
+	return pmd;
+}
+#endif
+
 #ifndef pmd_read_atomic
 static inline pmd_t pmd_read_atomic(pmd_t *pmdp)
 {
mm/Kconfig +3 -0
index 4dd8896..10fd7ff 100644
@@ -794,6 +794,9 @@ config IDLE_PAGE_TRACKING
 config ARCH_HAS_PTE_DEVMAP
 	bool
 
+config ARCH_HAS_PTE_DMEM
+	bool
+
 config ZONE_DEVICE
 	bool "Device memory (pmem, HMM, etc...) hotplug support"
 	depends on MEMORY_HOTPLUG
mm/dmem.c +61 -2

86 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#15 lore raw

RFC v2 13/37 mm, dmem: introduce PFN_DMEM and pfn_t_dmem

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Introduce PFN_DMEM as a new pfn flag for dmem pfn, define it by setting (BITS_PER_LONG_LONG - 6) bit.

Introduce pfn_t_dmem() helper to recognize dmem pfn.

---
 include/linux/pfn_t.h | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
include/linux/pfn_t.h +16 -1
index 2d91482..c6c0f1f 100644
@@ -11,6 +11,7 @@
  * PFN_MAP - pfn has a dynamic page mapping established by a device driver
  * PFN_SPECIAL - for CONFIG_FS_DAX_LIMITED builds to allow XIP, but not
  *		 get_user_pages
+ * PFN_DMEM - pfn references a dmem page
  */
 #define PFN_FLAGS_MASK (((u64) (~PAGE_MASK)) << (BITS_PER_LONG_LONG - PAGE_SHIFT))
 #define PFN_SG_CHAIN (1ULL << (BITS_PER_LONG_LONG - 1))
@@ -18,13 +19,15 @@
 #define PFN_DEV (1ULL << (BITS_PER_LONG_LONG - 3))
 #define PFN_MAP (1ULL << (BITS_PER_LONG_LONG - 4))
 #define PFN_SPECIAL (1ULL << (BITS_PER_LONG_LONG - 5))
+#define PFN_DMEM (1ULL << (BITS_PER_LONG_LONG - 6))
 
 #define PFN_FLAGS_TRACE \
 	{ PFN_SPECIAL,	"SPECIAL" }, \
 	{ PFN_SG_CHAIN,	"SG_CHAIN" }, \
 	{ PFN_SG_LAST,	"SG_LAST" }, \
 	{ PFN_DEV,	"DEV" }, \
-	{ PFN_MAP,	"MAP" }
+	{ PFN_MAP,	"MAP" }, \
+	{ PFN_DMEM,	"DMEM" }
 
 static inline pfn_t __pfn_to_pfn_t(unsigned long pfn, u64 flags)
 {
@@ -128,4 +131,16 @@ static inline bool pfn_t_special(pfn_t pfn)
 	return false;
 }
 #endif /* CONFIG_ARCH_HAS_PTE_SPECIAL */
+
+#ifdef CONFIG_ARCH_HAS_PTE_DMEM
+static inline bool pfn_t_dmem(pfn_t pfn)
+{
+	return (pfn.val & PFN_DMEM) == PFN_DMEM;
+}
+#else
+static inline bool pfn_t_dmem(pfn_t pfn)
+{
+	return false;
+}
+#endif /* CONFIG_ARCH_HAS_PTE_DMEM */
 #endif /* _LINUX_PFN_T_H_ */
Signature · 1 line
1.8.3.1
#16 lore raw

RFC v2 14/37 mm, dmem: differentiate dmem-pmd and thp-pmd

In reply to [email protected]

From: Yulei Zhang <[email protected]>

A dmem huge page is ultimately not a transparent huge page. As we decided to use pmd_special() to distinguish dmem-pmd from thp-pmd, we should make some slightly different semantics between pmd_special() and pmd_trans_huge(), just as pmd_devmap() in upstream. This distinction is especially important in some mm-core paths such as zap_pmd_range().

Explicitly mark the pmd_trans_huge() helpers that dmem needs by adding pmd_special() checks. This method could be reused in many mm-core paths.

---
 arch/x86/include/asm/pgtable.h | 10 +++++++++-
 include/linux/pgtable.h        |  5 +++++
 2 files changed, 14 insertions(+), 1 deletion(-)
arch/x86/include/asm/pgtable.h +9 -1
index dd4aff6..6ce85d4 100644
@@ -259,7 +259,7 @@ static inline int pmd_large(pmd_t pte)
 /* NOTE: when predicate huge page, consider also pmd_devmap, or use pmd_large */
 static inline int pmd_trans_huge(pmd_t pmd)
 {
-	return (pmd_val(pmd) & (_PAGE_PSE|_PAGE_DEVMAP)) == _PAGE_PSE;
+	return (pmd_val(pmd) & (_PAGE_PSE|_PAGE_DEVMAP|_PAGE_DMEM)) == _PAGE_PSE;
 }
 
 #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
@@ -275,6 +275,14 @@ static inline int has_transparent_hugepage(void)
 	return boot_cpu_has(X86_FEATURE_PSE);
 }
 
+#ifdef CONFIG_ARCH_HAS_PTE_DMEM
+static inline int pmd_special(pmd_t pmd)
+{
+	return (pmd_val(pmd) & (_PAGE_SPECIAL | _PAGE_DMEM)) ==
+		(_PAGE_SPECIAL | _PAGE_DMEM);
+}
+#endif
+
 #ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
 static inline int pmd_devmap(pmd_t pmd)
 {
include/linux/pgtable.h +5 -0
index 9e65694..30342b8 100644
@@ -1162,6 +1162,11 @@ static inline pmd_t pmd_mkdmem(pmd_t pmd)
 {
 	return pmd;
 }
+
+static inline int pmd_special(pmd_t pmd)
+{
+	return 0;
+}
 #endif
 
 #ifndef pmd_read_atomic
Signature · 1 line
1.8.3.1
#17 lore raw

RFC v2 15/37 mm: add pmd_special() check for pmd_trans_huge_lock()

In reply to [email protected]

From: Yulei Zhang <[email protected]>

As dmem-pmd had been distinguished from thp-pmd, we need to add pmd_special() such that pmd_trans_huge_lock could fetch ptl for dmem huge pmd and treat it as stable pmd.

---
 include/linux/huge_mm.h | 3 ++-
 mm/huge_memory.c        | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
include/linux/huge_mm.h +2 -1
index 0365aa9..2514b90 100644
@@ -242,7 +242,8 @@ static inline int is_swap_pmd(pmd_t pmd)
 static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd,
 		struct vm_area_struct *vma)
 {
-	if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd) || pmd_devmap(*pmd))
+	if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)
+		|| pmd_devmap(*pmd) || pmd_special(*pmd))
 		return __pmd_trans_huge_lock(pmd, vma);
 	else
 		return NULL;
mm/huge_memory.c +1 -1
index 9474dbc..31f9e83 100644
@@ -1890,7 +1890,7 @@ spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma)
 	spinlock_t *ptl;
 	ptl = pmd_lock(vma->vm_mm, pmd);
 	if (likely(is_swap_pmd(*pmd) || pmd_trans_huge(*pmd) ||
-			pmd_devmap(*pmd)))
+			pmd_devmap(*pmd) || pmd_special(*pmd)))
 		return ptl;
 	spin_unlock(ptl);
 	return NULL;
Signature · 1 line
1.8.3.1
#18 lore raw

RFC v2 16/37 dmemfs: introduce ->split() to dmemfs_vm_ops

In reply to [email protected]

From: Yulei Zhang <[email protected]>

It is required by __split_vma() to adjust vma. munmap() which create hole unaligned to pagesize in dmemfs-mapping should be forbidden.

---
 fs/dmemfs/inode.c | 8 ++++++++
 1 file changed, 8 insertions(+)
fs/dmemfs/inode.c +8 -0
index 443f2e1..ab6a492 100644
@@ -450,6 +450,13 @@ static bool check_vma_access(struct vm_area_struct *vma, int write)
 	return len;
 }
 
+static int dmemfs_split(struct vm_area_struct *vma, unsigned long addr)
+{
+	if (addr & (dmem_page_size(file_inode(vma->vm_file)) - 1))
+		return -EINVAL;
+	return 0;
+}
+
 static vm_fault_t dmemfs_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
@@ -484,6 +491,7 @@ static unsigned long dmemfs_pagesize(struct vm_area_struct *vma)
 }
 
 static const struct vm_operations_struct dmemfs_vm_ops = {
+	.split = dmemfs_split,
 	.fault = dmemfs_fault,
 	.pagesize = dmemfs_pagesize,
 	.access = dmemfs_access_dmem,
Signature · 1 line
1.8.3.1
#19 lore raw

RFC v2 17/37 mm, dmemfs: support unmap_page_range() for dmemfs pmd

In reply to [email protected]

From: Yulei Zhang <[email protected]>

It is required by munmap() for dmemfs mapping.

---
 mm/huge_memory.c | 2 ++
 mm/memory.c      | 8 +++++---
 2 files changed, 7 insertions(+), 3 deletions(-)
mm/huge_memory.c +2 -0
index 31f9e83..2a818ec 100644
@@ -1664,6 +1664,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
 		spin_unlock(ptl);
 		if (is_huge_zero_pmd(orig_pmd))
 			tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE);
+	} else if (pmd_special(orig_pmd)) {
+		spin_unlock(ptl);
 	} else if (is_huge_zero_pmd(orig_pmd)) {
 		zap_deposited_table(tlb->mm, pmd);
 		spin_unlock(ptl);
mm/memory.c +5 -3
index c48f8df..6b60981 100644
@@ -1338,10 +1338,12 @@ static inline unsigned long zap_pmd_range(struct mmu_gather *tlb,
 	pmd = pmd_offset(pud, addr);
 	do {
 		next = pmd_addr_end(addr, end);
-		if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd) || pmd_devmap(*pmd)) {
-			if (next - addr != HPAGE_PMD_SIZE)
+		if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd) ||
+			pmd_devmap(*pmd) || pmd_special(*pmd)) {
+			if (next - addr != HPAGE_PMD_SIZE) {
+				VM_BUG_ON(pmd_special(*pmd));
 				__split_huge_pmd(vma, pmd, addr, false, NULL);
-			else if (zap_huge_pmd(tlb, vma, pmd, addr))
+			} else if (zap_huge_pmd(tlb, vma, pmd, addr))
 				goto next;
 			/* fall through */
 		}
Signature · 1 line
1.8.3.1
#20 lore raw

RFC v2 18/37 mm: follow_pmd_mask() for dmem huge pmd

In reply to [email protected]

From: Yulei Zhang <[email protected]>

While follow_pmd_mask(), dmem huge pmd should be recognized and return error pointer of '-EEXIST' to indicate that proper page table entry exists in pmd special but no corresponding struct page, because dmem page means non struct page backend. We update pmd if foll_flags takes FOLL_TOUCH.

---
 mm/gup.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
mm/gup.c +42 -0
index 98eb8e6..ad1aede 100644
@@ -387,6 +387,42 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address,
 	return -EEXIST;
 }
 
+static struct page *
+follow_special_pmd(struct vm_area_struct *vma, unsigned long address,
+		   pmd_t *pmd, unsigned int flags)
+{
+	spinlock_t *ptl;
+
+	if ((flags & FOLL_DUMP) && is_huge_zero_pmd(*pmd))
+		/* Avoid special (like zero) pages in core dumps */
+		return ERR_PTR(-EFAULT);
+
+	/* No page to get reference */
+	if (flags & FOLL_GET)
+		return ERR_PTR(-EFAULT);
+
+	if (flags & FOLL_TOUCH) {
+		pmd_t _pmd;
+
+		ptl = pmd_lock(vma->vm_mm, pmd);
+		if (!pmd_special(*pmd)) {
+			spin_unlock(ptl);
+			return NULL;
+		}
+		_pmd = pmd_mkyoung(*pmd);
+		if (flags & FOLL_WRITE)
+			_pmd = pmd_mkdirty(_pmd);
+		if (pmdp_set_access_flags(vma, address & HPAGE_PMD_MASK,
+					  pmd, _pmd,
+					  flags & FOLL_WRITE))
+			update_mmu_cache_pmd(vma, address, pmd);
+		spin_unlock(ptl);
+	}
+
+	/* Proper page table entry exists, but no corresponding struct page */
+	return ERR_PTR(-EEXIST);
+}
+
 /*
  * FOLL_FORCE can write to even unwritable pte's, but only
  * after we've gone through a COW cycle and they are dirty.
@@ -571,6 +607,12 @@ static struct page *follow_pmd_mask(struct vm_area_struct *vma,
 			return page;
 		return no_page_table(vma, flags);
 	}
+	if (pmd_special(*pmd)) {
+		page = follow_special_pmd(vma, address, pmd, flags);
+		if (page)
+			return page;
+		return no_page_table(vma, flags);
+	}
 	if (is_hugepd(__hugepd(pmd_val(pmdval)))) {
 		page = follow_huge_pd(vma, address,
 				      __hugepd(pmd_val(pmdval)), flags,
Signature · 1 line
1.8.3.1
#21 lore raw

RFC v2 19/37 mm: gup_huge_pmd() for dmem huge pmd

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Add pmd_special() check in gup_huge_pmd() to support dmem huge pmd. GUP will return zero if enconter dmem page, and we could handle it outside GUP routine.

---
 mm/gup.c      | 6 +++++-
 mm/pagewalk.c | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)
mm/gup.c +5 -1
index ad1aede..47c8197 100644
@@ -2470,6 +2470,10 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr,
 	if (!pmd_access_permitted(orig, flags & FOLL_WRITE))
 		return 0;
 
+	/* Bypass dmem huge pmd. It will be handled in outside routine. */
+	if (pmd_special(orig))
+		return 0;
+
 	if (pmd_devmap(orig)) {
 		if (unlikely(flags & FOLL_LONGTERM))
 			return 0;
@@ -2572,7 +2576,7 @@ static int gup_pmd_range(pud_t *pudp, pud_t pud, unsigned long addr, unsigned lo
 			return 0;
 
 		if (unlikely(pmd_trans_huge(pmd) || pmd_huge(pmd) ||
-			     pmd_devmap(pmd))) {
+			     pmd_devmap(pmd) || pmd_special(pmd))) {
 			/*
 			 * NUMA hinting faults need to be handled in the GUP
 			 * slowpath for accounting purposes and so that they
mm/pagewalk.c +1 -1
index e81640d..e7c4575 100644
@@ -71,7 +71,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
 	do {
 again:
 		next = pmd_addr_end(addr, end);
-		if (pmd_none(*pmd) || (!walk->vma && !walk->no_vma)) {
+		if (pmd_none(*pmd) || (!walk->vma && !walk->no_vma) || pmd_special(*pmd)) {
 			if (ops->pte_hole)
 				err = ops->pte_hole(addr, next, depth, walk);
 			if (err)
Signature · 1 line
1.8.3.1
#22 lore raw

RFC v2 20/37 mm: support dmem huge pmd for vmf_insert_pfn_pmd()

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Since vmf_insert_pfn_pmd will BUG_ON non-pmd-devmap, we make pfn dmem pass the check.

Dmem huge pmd will be marked with _PAGE_SPECIAL and _PAGE_DMEM, so that follow_pfn() could recognize it.

---
 mm/huge_memory.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
mm/huge_memory.c +3 -1
index 2a818ec..6e52d57 100644
@@ -781,6 +781,8 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr,
 	entry = pmd_mkhuge(pfn_t_pmd(pfn, prot));
 	if (pfn_t_devmap(pfn))
 		entry = pmd_mkdevmap(entry);
+	else if (pfn_t_dmem(pfn))
+		entry = pmd_mkdmem(entry);
 	if (write) {
 		entry = pmd_mkyoung(pmd_mkdirty(entry));
 		entry = maybe_pmd_mkwrite(entry, vma);
@@ -827,7 +829,7 @@ vm_fault_t vmf_insert_pfn_pmd_prot(struct vm_fault *vmf, pfn_t pfn,
 	 * can't support a 'special' bit.
 	 */
 	BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) &&
-			!pfn_t_devmap(pfn));
+			!pfn_t_devmap(pfn) && !pfn_t_dmem(pfn));
 	BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) ==
 						(VM_PFNMAP|VM_MIXEDMAP));
 	BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags));
Signature · 1 line
1.8.3.1
#23 lore raw

RFC v2 21/37 mm: support dmem huge pmd for follow_pfn()

In reply to [email protected]

From: Yulei Zhang <[email protected]>

follow_pfn() will get pfn of pmd if huge pmd is encountered.

---
 mm/memory.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
mm/memory.c +11 -3
index 6b60981..abb9148 100644
@@ -4807,15 +4807,23 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address,
 	int ret = -EINVAL;
 	spinlock_t *ptl;
 	pte_t *ptep;
+	pmd_t *pmdp = NULL;
 
 	if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)))
 		return ret;
 
-	ret = follow_pte(vma->vm_mm, address, &ptep, &ptl);
+	ret = follow_pte_pmd(vma->vm_mm, address, NULL, &ptep, &pmdp, &ptl);
 	if (ret)
 		return ret;
-	*pfn = pte_pfn(*ptep);
-	pte_unmap_unlock(ptep, ptl);
+
+	if (pmdp) {
+		*pfn = pmd_pfn(*pmdp) + ((address & ~PMD_MASK) >> PAGE_SHIFT);
+		spin_unlock(ptl);
+	} else {
+		*pfn = pte_pfn(*ptep);
+		pte_unmap_unlock(ptep, ptl);
+	}
+
 	return 0;
 }
 EXPORT_SYMBOL(follow_pfn);
Signature · 1 line
1.8.3.1
#24 lore raw

RFC v2 22/37 kvm, x86: Distinguish dmemfs page from mmio page

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Dmem page is pfn invalid but not mmio, introduce API is_dmem_pfn() to distinguish that.

---
 arch/x86/kvm/mmu/mmu.c | 1 +
 include/linux/dmem.h   | 7 +++++++
 mm/dmem.c              | 7 +++++++
 3 files changed, 15 insertions(+)
arch/x86/kvm/mmu/mmu.c +1 -0
index 5bb1939..394508f 100644
@@ -43,6 +43,7 @@
 #include <linux/hash.h>
 #include <linux/kern_levels.h>
 #include <linux/kthread.h>
+#include <linux/dmem.h>
 
 #include <asm/page.h>
 #include <asm/memtype.h>
include/linux/dmem.h +7 -0
index 8682d63..59d3ef14 100644
@@ -19,11 +19,18 @@
 		     unsigned int try_max, unsigned int *result_nr);
 
 void dmem_free_pages(phys_addr_t addr, unsigned int dpages_nr);
+bool is_dmem_pfn(unsigned long pfn);
 #define dmem_free_page(addr)	dmem_free_pages(addr, 1)
 #else
 static inline int dmem_reserve_init(void)
 {
 	return 0;
 }
+
+static inline bool is_dmem_pfn(unsigned long pfn)
+{
+	return 0;
+}
+
 #endif
 #endif	/* _LINUX_DMEM_H */
mm/dmem.c +7 -0
index 2e61dbd..eb6df70 100644
@@ -972,3 +972,10 @@ void dmem_free_pages(phys_addr_t addr, unsigned int dpages_nr)
 }
 EXPORT_SYMBOL(dmem_free_pages);
 
+bool is_dmem_pfn(unsigned long pfn)
+{
+	struct dmem_node *dnode;
+
+	return !!find_dmem_region(__pfn_to_phys(pfn), &dnode);
+}
+EXPORT_SYMBOL(is_dmem_pfn);
Signature · 1 line
1.8.3.1
#25 lore raw

RFC v2 23/37 kvm, x86: introduce VM_DMEM for syscall support usage

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Currently dmemfs do not support memory readonly, so change_protection() will be disabled for dmemfs vma. Since vma->vm_flags could be changed to new flag in mprotect_fixup(), so we introduce a new vma flag VM_DMEM and check this flag in mprotect_fixup() to avoid changing vma->vm_flags.

We also check it in vma_to_resize() to disable mremap() for dmemfs vma.

---
 fs/dmemfs/inode.c  | 2 +-
 include/linux/mm.h | 7 +++++++
 mm/gup.c           | 7 +++++--
 mm/mincore.c       | 8 ++++++--
 mm/mprotect.c      | 5 ++++-
 mm/mremap.c        | 3 +++
 6 files changed, 26 insertions(+), 6 deletions(-)
fs/dmemfs/inode.c +1 -1
index ab6a492..b165bd3 100644
@@ -507,7 +507,7 @@ int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
 	if (!(vma->vm_flags & VM_SHARED))
 		return -EINVAL;
 
-	vma->vm_flags |= VM_PFNMAP;
+	vma->vm_flags |= VM_PFNMAP | VM_DMEM | VM_IO;
 
 	file_accessed(file);
 	vma->vm_ops = &dmemfs_vm_ops;
include/linux/mm.h +7 -0
index db6ae4d..2f3135fe 100644
@@ -311,6 +311,8 @@ int overcommit_policy_handler(struct ctl_table *, int, void *, size_t *,
 #define VM_HIGH_ARCH_4	BIT(VM_HIGH_ARCH_BIT_4)
 #endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */
 
+#define VM_DMEM		BIT(38)		/* Dmem page VM */
+
 #ifdef CONFIG_ARCH_HAS_PKEYS
 # define VM_PKEY_SHIFT	VM_HIGH_ARCH_BIT_0
 # define VM_PKEY_BIT0	VM_HIGH_ARCH_0	/* A protection key is a 4-bit value */
@@ -666,6 +668,11 @@ static inline bool vma_is_accessible(struct vm_area_struct *vma)
 	return vma->vm_flags & VM_ACCESS_FLAGS;
 }
 
+static inline bool vma_is_dmem(struct vm_area_struct *vma)
+{
+	return !!(vma->vm_flags & VM_DMEM);
+}
+
 #ifdef CONFIG_SHMEM
 /*
  * The vma_is_shmem is not inline because it is used only by slow
mm/gup.c +5 -2
index 47c8197..0ea9071 100644
@@ -492,8 +492,11 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
 			goto no_page;
 	} else if (unlikely(!page)) {
 		if (flags & FOLL_DUMP) {
-			/* Avoid special (like zero) pages in core dumps */
-			page = ERR_PTR(-EFAULT);
+			if (vma_is_dmem(vma))
+				page = ERR_PTR(-EEXIST);
+			else
+				/* Avoid special (like zero) pages in core dumps */
+				page = ERR_PTR(-EFAULT);
 			goto out;
 		}
 
mm/mincore.c +6 -2
index 02db1a8..f8d10e4 100644
@@ -78,8 +78,12 @@ static int __mincore_unmapped_range(unsigned long addr, unsigned long end,
 		pgoff_t pgoff;
 
 		pgoff = linear_page_index(vma, addr);
-		for (i = 0; i < nr; i++, pgoff++)
-			vec[i] = mincore_page(vma->vm_file->f_mapping, pgoff);
+		for (i = 0; i < nr; i++, pgoff++) {
+			if (vma_is_dmem(vma))
+				vec[i] = 1;
+			else
+				vec[i] = mincore_page(vma->vm_file->f_mapping, pgoff);
+		}
 	} else {
 		for (i = 0; i < nr; i++)
 			vec[i] = 0;
mm/mprotect.c +4 -1
index 56c02be..b1650b5 100644
@@ -236,7 +236,7 @@ static inline unsigned long change_pmd_range(struct vm_area_struct *vma,
 		 * for all the checks.
 		 */
 		if (!is_swap_pmd(*pmd) && !pmd_devmap(*pmd) &&
-		     pmd_none_or_clear_bad_unless_trans_huge(pmd))
+		     pmd_none_or_clear_bad_unless_trans_huge(pmd) && !pmd_special(*pmd))
 			goto next;
 
 		/* invoke the mmu notifier if the pmd is populated */
@@ -412,6 +412,9 @@ static int prot_none_test(unsigned long addr, unsigned long next,
 		return 0;
 	}
 
+	if (vma_is_dmem(vma))
+		return -EINVAL;
+
 	/*
 	 * Do PROT_NONE PFN permission checks here when we can still
 	 * bail out without undoing a lot of state. This is a rather
mm/mremap.c +3 -0
index 138abba..598e681 100644
@@ -482,6 +482,9 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr,
 	if (!vma || vma->vm_start > addr)
 		return ERR_PTR(-EFAULT);
 
+	if (vma_is_dmem(vma))
+		return ERR_PTR(-EINVAL);
+
 	/*
 	 * !old_len is a special case where an attempt is made to 'duplicate'
 	 * a mapping.  This makes no sense for private mappings as it will
Signature · 1 line
1.8.3.1
#27 lore raw

RFC v2 24/37 dmemfs: support hugepage for dmemfs

In reply to [email protected]

From: Yulei Zhang <[email protected]>

It add hugepage support for dmemfs. We use PFN_DMEM to notify vmf_insert_pfn_pmd, and dmem huge pmd will be marked with _PAGE_SPECIAL and _PAGE_DMEM. So that GUP-fast can separate dmemfs page from other page type and handle it correctly.

---
 fs/dmemfs/inode.c | 113 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 111 insertions(+), 2 deletions(-)
fs/dmemfs/inode.c +111 -2
index b165bd3..17a518c 100644
@@ -457,7 +457,7 @@ static int dmemfs_split(struct vm_area_struct *vma, unsigned long addr)
 	return 0;
 }
 
-static vm_fault_t dmemfs_fault(struct vm_fault *vmf)
+static vm_fault_t __dmemfs_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct inode *inode = file_inode(vma->vm_file);
@@ -485,6 +485,63 @@ static vm_fault_t dmemfs_fault(struct vm_fault *vmf)
 	return ret;
 }
 
+static vm_fault_t  __dmemfs_pmd_fault(struct vm_fault *vmf)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	unsigned long pmd_addr = vmf->address & PMD_MASK;
+	unsigned long page_addr;
+	struct inode *inode = file_inode(vma->vm_file);
+	void *entry;
+	phys_addr_t phys;
+	pfn_t pfn;
+	int ret;
+
+	if (dmem_page_size(inode) < PMD_SIZE)
+		return VM_FAULT_FALLBACK;
+
+	WARN_ON(pmd_addr < vma->vm_start ||
+		vma->vm_end < pmd_addr + PMD_SIZE);
+
+	page_addr = vmf->address & ~(dmem_page_size(inode) - 1);
+	entry = radix_get_create_entry(vma, page_addr, inode,
+				       linear_page_index(vma, page_addr));
+	if (IS_ERR(entry))
+		return (PTR_ERR(entry) == -ENOMEM) ?
+			VM_FAULT_OOM : VM_FAULT_SIGBUS;
+
+	phys = dmem_addr_to_pfn(inode, dmem_entry_to_addr(inode, entry),
+				linear_page_index(vma, pmd_addr), PMD_SHIFT);
+	phys <<= PAGE_SHIFT;
+	pfn = phys_to_pfn_t(phys, PFN_DMEM);
+	ret = vmf_insert_pfn_pmd(vmf, pfn, !!(vma->vm_flags & VM_WRITE));
+
+	radix_put_entry();
+	return ret;
+}
+
+static vm_fault_t dmemfs_huge_fault(struct vm_fault *vmf, enum page_entry_size pe_size)
+{
+	int ret;
+
+	switch (pe_size) {
+	case PE_SIZE_PTE:
+		ret = __dmemfs_fault(vmf);
+		break;
+	case PE_SIZE_PMD:
+		ret = __dmemfs_pmd_fault(vmf);
+		break;
+	default:
+		ret = VM_FAULT_SIGBUS;
+	}
+
+	return ret;
+}
+
+static vm_fault_t dmemfs_fault(struct vm_fault *vmf)
+{
+	return dmemfs_huge_fault(vmf, PE_SIZE_PTE);
+}
+
 static unsigned long dmemfs_pagesize(struct vm_area_struct *vma)
 {
 	return dmem_page_size(file_inode(vma->vm_file));
@@ -495,6 +552,7 @@ static unsigned long dmemfs_pagesize(struct vm_area_struct *vma)
 	.fault = dmemfs_fault,
 	.pagesize = dmemfs_pagesize,
 	.access = dmemfs_access_dmem,
+	.huge_fault = dmemfs_huge_fault,
 };
 
 int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
@@ -507,15 +565,66 @@ int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
 	if (!(vma->vm_flags & VM_SHARED))
 		return -EINVAL;
 
-	vma->vm_flags |= VM_PFNMAP | VM_DMEM | VM_IO;
+	vma->vm_flags |= VM_PFNMAP | VM_DONTCOPY | VM_DMEM | VM_IO;
+
+	if (dmem_page_size(inode) != PAGE_SIZE)
+		vma->vm_flags |= VM_HUGEPAGE;
 
 	file_accessed(file);
 	vma->vm_ops = &dmemfs_vm_ops;
 	return 0;
 }
 
+/*
+ * If the size of area returned by mm->get_unmapped_area() is one
+ * dmem pagesize larger than 'len', the returned addr by
+ * mm->get_unmapped_area() could be aligned to dmem pagesize to
+ * meet alignment demand.
+ */
+static unsigned long
+dmemfs_get_unmapped_area(struct file *file, unsigned long addr,
+			 unsigned long len, unsigned long pgoff,
+			 unsigned long flags)
+{
+	unsigned long len_pad;
+	unsigned long off = pgoff << PAGE_SHIFT;
+	unsigned long align;
+
+	align = dmem_page_size(file_inode(file));
+
+	/* For pud or pmd pagesize, could not support fault fallback. */
+	if (len & (align - 1))
+		return -EINVAL;
+	if (len > TASK_SIZE)
+		return -ENOMEM;
+
+	if (flags & MAP_FIXED) {
+		if (addr & (align - 1))
+			return -EINVAL;
+		return addr;
+	}
+
+	/*
+	 * Pad a extra align space for 'len', as we want to find a unmapped
+	 * area which is larger enough to align with dmemfs pagesize, if
+	 * pagesize of dmem is larger than 4K.
+	 */
+	len_pad = (align == PAGE_SIZE) ? len : len + align;
+
+	/* 'len' or 'off' is too large for pad. */
+	if (len_pad < len || (off + len_pad) < off)
+		return -EINVAL;
+
+	addr = current->mm->get_unmapped_area(file, addr, len_pad,
+					      pgoff, flags);
+
+	/* Now 'addr' could be aligned to upper boundary. */
+	return IS_ERR_VALUE(addr) ? addr : round_up(addr, align);
+}
+
 static const struct file_operations dmemfs_file_operations = {
 	.mmap = dmemfs_file_mmap,
+	.get_unmapped_area = dmemfs_get_unmapped_area,
 };
 
 static int dmemfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
Signature · 1 line
1.8.3.1
#28 lore raw

RFC v2 25/37 mm, x86, dmem: fix estimation of reserved page for vaddr_get_pfn()

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Fix estimation of reserved page for vaddr_get_pfn() and check 'ret' before checking writable permission

---
 drivers/vfio/vfio_iommu_type1.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/vfio/vfio_iommu_type1.c +4 -0
index 67e8276..c465d1a 100644
@@ -471,6 +471,10 @@ static int vaddr_get_pfn(struct mm_struct *mm, unsigned long vaddr,
 		if (ret == -EAGAIN)
 			goto retry;
 
+		if (!ret && (prot & IOMMU_WRITE) &&
+		    !(vma->vm_flags & VM_WRITE))
+			ret = -EFAULT;
+
 		if (!ret && !is_invalid_reserved_pfn(*pfn))
 			ret = -EFAULT;
 	}
Signature · 1 line
1.8.3.1
#29 lore raw

RFC v2 26/37 mm, dmem: introduce pud_special() for dmem huge pud support

In reply to [email protected]

From: Yulei Zhang <[email protected]>

pud_special() will check both _PAGE_SPECIAL and _PAGE_DMEM bit as pmd_special() does.

---
 arch/x86/include/asm/pgtable.h | 13 +++++++++++++
 include/linux/pgtable.h        | 10 ++++++++++
 2 files changed, 23 insertions(+)
arch/x86/include/asm/pgtable.h +13 -0
index 6ce85d4..9e36d42 100644
@@ -281,6 +281,12 @@ static inline int pmd_special(pmd_t pmd)
 	return (pmd_val(pmd) & (_PAGE_SPECIAL | _PAGE_DMEM)) ==
 		(_PAGE_SPECIAL | _PAGE_DMEM);
 }
+
+static inline int pud_special(pud_t pud)
+{
+	return (pud_val(pud) & (_PAGE_SPECIAL | _PAGE_DMEM)) ==
+		(_PAGE_SPECIAL | _PAGE_DMEM);
+}
 #endif
 
 #ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
@@ -516,6 +522,13 @@ static inline pud_t pud_mkdirty(pud_t pud)
 	return pud_set_flags(pud, _PAGE_DIRTY | _PAGE_SOFT_DIRTY);
 }
 
+#ifdef CONFIG_ARCH_HAS_PTE_DMEM
+static inline pud_t pud_mkdmem(pud_t pud)
+{
+	return pud_set_flags(pud, _PAGE_SPECIAL | _PAGE_DMEM);
+}
+#endif
+
 static inline pud_t pud_mkdevmap(pud_t pud)
 {
 	return pud_set_flags(pud, _PAGE_DEVMAP);
include/linux/pgtable.h +10 -0
index 30342b8..0ef03ff 100644
@@ -1167,6 +1167,16 @@ static inline int pmd_special(pmd_t pmd)
 {
 	return 0;
 }
+
+static inline pud_t pud_mkdmem(pud_t pud)
+{
+	return pud;
+}
+
+static inline int pud_special(pud_t pud)
+{
+	return 0;
+}
 #endif
 
 #ifndef pmd_read_atomic
Signature · 1 line
1.8.3.1
#30 lore raw

RFC v2 27/37 mm: add pud_special() check to support dmem huge pud

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Add pud_special() and follow_special_pud() to support dmem huge pud as we do for dmem huge pmd.

---
 arch/x86/include/asm/pgtable.h |  2 +-
 include/linux/huge_mm.h        |  2 +-
 mm/gup.c                       | 46 ++++++++++++++++++++++++++++++++++++++++++
 mm/huge_memory.c               | 11 ++++++----
 mm/memory.c                    |  4 ++--
 mm/mprotect.c                  |  2 ++
 mm/pagewalk.c                  |  2 +-
 7 files changed, 60 insertions(+), 9 deletions(-)
arch/x86/include/asm/pgtable.h +1 -1
index 9e36d42..2284387 100644
@@ -265,7 +265,7 @@ static inline int pmd_trans_huge(pmd_t pmd)
 #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
 static inline int pud_trans_huge(pud_t pud)
 {
-	return (pud_val(pud) & (_PAGE_PSE|_PAGE_DEVMAP)) == _PAGE_PSE;
+	return (pud_val(pud) & (_PAGE_PSE|_PAGE_DEVMAP|_PAGE_DMEM)) == _PAGE_PSE;
 }
 #endif
 
include/linux/huge_mm.h +1 -1
index 2514b90..b69c940 100644
@@ -251,7 +251,7 @@ static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd,
 static inline spinlock_t *pud_trans_huge_lock(pud_t *pud,
 		struct vm_area_struct *vma)
 {
-	if (pud_trans_huge(*pud) || pud_devmap(*pud))
+	if (pud_trans_huge(*pud) || pud_devmap(*pud) || pud_special(*pud))
 		return __pud_trans_huge_lock(pud, vma);
 	else
 		return NULL;
mm/gup.c +46 -0

64 lines not on this page · raw message

mm/huge_memory.c +7 -4
index 6e52d57..7c5385a 100644
@@ -883,6 +883,8 @@ static void insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr,
 	entry = pud_mkhuge(pfn_t_pud(pfn, prot));
 	if (pfn_t_devmap(pfn))
 		entry = pud_mkdevmap(entry);
+	if (pfn_t_dmem(pfn))
+		entry = pud_mkdmem(entry);
 	if (write) {
 		entry = pud_mkyoung(pud_mkdirty(entry));
 		entry = maybe_pud_mkwrite(entry, vma);
@@ -919,7 +921,7 @@ vm_fault_t vmf_insert_pfn_pud_prot(struct vm_fault *vmf, pfn_t pfn,
 	 * can't support a 'special' bit.
 	 */
 	BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) &&
-			!pfn_t_devmap(pfn));
+			!pfn_t_devmap(pfn) && !pfn_t_dmem(pfn));
 	BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) ==
 						(VM_PFNMAP|VM_MIXEDMAP));
 	BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags));
@@ -1911,7 +1913,7 @@ spinlock_t *__pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma)
 	spinlock_t *ptl;
 
 	ptl = pud_lock(vma->vm_mm, pud);
-	if (likely(pud_trans_huge(*pud) || pud_devmap(*pud)))
+	if (likely(pud_trans_huge(*pud) || pud_devmap(*pud) || pud_special(*pud)))
 		return ptl;
 	spin_unlock(ptl);
 	return NULL;
@@ -1922,6 +1924,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
 		 pud_t *pud, unsigned long addr)
 {
 	spinlock_t *ptl;
+	pud_t orig_pud;
 
 	ptl = __pud_trans_huge_lock(pud, vma);
 	if (!ptl)
@@ -1932,9 +1935,9 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	 * pgtable_trans_huge_withdraw after finishing pudp related
 	 * operations.
 	 */
-	pudp_huge_get_and_clear_full(tlb->mm, addr, pud, tlb->fullmm);
+	orig_pud = pudp_huge_get_and_clear_full(tlb->mm, addr, pud, tlb->fullmm);
 	tlb_remove_pud_tlb_entry(tlb, pud, addr);
-	if (vma_is_special_huge(vma)) {
+	if (vma_is_special_huge(vma) || pud_special(orig_pud)) {
 		spin_unlock(ptl);
 		/* No zero page support yet */
 	} else {
mm/memory.c +2 -2
index abb9148..01f3b05 100644
@@ -1078,7 +1078,7 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr,
 	src_pud = pud_offset(src_p4d, addr);
 	do {
 		next = pud_addr_end(addr, end);
-		if (pud_trans_huge(*src_pud) || pud_devmap(*src_pud)) {
+		if (pud_trans_huge(*src_pud) || pud_devmap(*src_pud) || pud_special(*src_pud)) {
 			int err;
 
 			VM_BUG_ON_VMA(next-addr != HPAGE_PUD_SIZE, src_vma);
@@ -1375,7 +1375,7 @@ static inline unsigned long zap_pud_range(struct mmu_gather *tlb,
 	pud = pud_offset(p4d, addr);
 	do {
 		next = pud_addr_end(addr, end);
-		if (pud_trans_huge(*pud) || pud_devmap(*pud)) {
+		if (pud_trans_huge(*pud) || pud_devmap(*pud) || pud_special(*pud)) {
 			if (next - addr != HPAGE_PUD_SIZE) {
 				mmap_assert_locked(tlb->mm);
 				split_huge_pud(vma, pud, addr);
mm/mprotect.c +2 -0
index b1650b5..05fa453 100644
@@ -292,6 +292,8 @@ static inline unsigned long change_pud_range(struct vm_area_struct *vma,
 	pud = pud_offset(p4d, addr);
 	do {
 		next = pud_addr_end(addr, end);
+		if (pud_special(*pud))
+			continue;
 		if (pud_none_or_clear_bad(pud))
 			continue;
 		pages += change_pmd_range(vma, pud, addr, next, newprot,
mm/pagewalk.c +1 -1
index e7c4575..afd8bca 100644
@@ -129,7 +129,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
 	do {
  again:
 		next = pud_addr_end(addr, end);
-		if (pud_none(*pud) || (!walk->vma && !walk->no_vma)) {
+		if (pud_none(*pud) || (!walk->vma && !walk->no_vma) || pud_special(*pud)) {
 			if (ops->pte_hole)
 				err = ops->pte_hole(addr, next, depth, walk);
 			if (err)
Signature · 1 line
1.8.3.1
#31 lore raw

RFC v2 28/37 mm, dmemfs: support huge_fault() for dmemfs

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Introduce __dmemfs_huge_fault() to handle 1G huge pud for dmemfs.

---
 fs/dmemfs/inode.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
fs/dmemfs/inode.c +40 -0
index 17a518c..f698b9d 100644
@@ -519,6 +519,43 @@ static vm_fault_t  __dmemfs_pmd_fault(struct vm_fault *vmf)
 	return ret;
 }
 
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static vm_fault_t __dmemfs_huge_fault(struct vm_fault *vmf)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	unsigned long pud_addr = vmf->address & PUD_MASK;
+	struct inode *inode = file_inode(vma->vm_file);
+	void *entry;
+	phys_addr_t phys;
+	pfn_t pfn;
+	int ret;
+
+	if (dmem_page_size(inode) < PUD_SIZE)
+		return VM_FAULT_FALLBACK;
+
+	WARN_ON(pud_addr < vma->vm_start ||
+		vma->vm_end < pud_addr + PUD_SIZE);
+
+	entry = radix_get_create_entry(vma, pud_addr, inode,
+				       linear_page_index(vma, pud_addr));
+	if (IS_ERR(entry))
+		return (PTR_ERR(entry) == -ENOMEM) ?
+			VM_FAULT_OOM : VM_FAULT_SIGBUS;
+
+	phys = dmem_entry_to_addr(inode, entry);
+	pfn = phys_to_pfn_t(phys, PFN_DMEM);
+	ret = vmf_insert_pfn_pud(vmf, pfn, !!(vma->vm_flags & VM_WRITE));
+
+	radix_put_entry();
+	return ret;
+}
+#else
+static vm_fault_t __dmemfs_huge_fault(struct vm_fault *vmf)
+{
+	return VM_FAULT_FALLBACK;
+}
+#endif /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
+
 static vm_fault_t dmemfs_huge_fault(struct vm_fault *vmf, enum page_entry_size pe_size)
 {
 	int ret;
@@ -530,6 +567,9 @@ static vm_fault_t dmemfs_huge_fault(struct vm_fault *vmf, enum page_entry_size p
 	case PE_SIZE_PMD:
 		ret = __dmemfs_pmd_fault(vmf);
 		break;
+	case PE_SIZE_PUD:
+		ret = __dmemfs_huge_fault(vmf);
+		break;
 	default:
 		ret = VM_FAULT_SIGBUS;
 	}
Signature · 1 line
1.8.3.1
#32 lore raw

RFC v2 29/37 mm: add follow_pte_pud() to support huge pud look up

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Since we had supported dmem huge pud, here support dmem huge pud for hva_to_pfn().

Similar to follow_pte_pmd(), follow_pte_pud() allows a PTE lead or a huge page PMD or huge page PUD to be found and returned.

---
 mm/memory.c | 52 ++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 44 insertions(+), 8 deletions(-)
mm/memory.c +44 -8
index 01f3b05..dfc95be 100644
@@ -4698,9 +4698,9 @@ int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
 }
 #endif /* __PAGETABLE_PMD_FOLDED */
 
-static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address,
+static int __follow_pte_pud(struct mm_struct *mm, unsigned long address,
 			    struct mmu_notifier_range *range,
-			    pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp)
+			    pte_t **ptepp, pmd_t **pmdpp, pud_t **pudpp, spinlock_t **ptlp)
 {
 	pgd_t *pgd;
 	p4d_t *p4d;
@@ -4717,6 +4717,26 @@ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address,
 		goto out;
 
 	pud = pud_offset(p4d, address);
+	VM_BUG_ON(pud_trans_huge(*pud));
+	if (pud_huge(*pud)) {
+		if (!pudpp)
+			goto out;
+
+		if (range) {
+			mmu_notifier_range_init(range, MMU_NOTIFY_CLEAR, 0,
+						NULL, mm, address & PUD_MASK,
+						(address & PUD_MASK) + PUD_SIZE);
+			mmu_notifier_invalidate_range_start(range);
+		}
+		*ptlp = pud_lock(mm, pud);
+		if (pud_huge(*pud)) {
+			*pudpp = pud;
+			return 0;
+		}
+		spin_unlock(*ptlp);
+		if (range)
+			mmu_notifier_invalidate_range_end(range);
+	}
 	if (pud_none(*pud) || unlikely(pud_bad(*pud)))
 		goto out;
 
@@ -4772,8 +4792,8 @@ static inline int follow_pte(struct mm_struct *mm, unsigned long address,
 
 	/* (void) is needed to make gcc happy */
 	(void) __cond_lock(*ptlp,
-			   !(res = __follow_pte_pmd(mm, address, NULL,
-						    ptepp, NULL, ptlp)));
+			   !(res = __follow_pte_pud(mm, address, NULL,
+						    ptepp, NULL, NULL, ptlp)));
 	return res;
 }
 
@@ -4785,12 +4805,24 @@ int follow_pte_pmd(struct mm_struct *mm, unsigned long address,
 
 	/* (void) is needed to make gcc happy */
 	(void) __cond_lock(*ptlp,
-			   !(res = __follow_pte_pmd(mm, address, range,
-						    ptepp, pmdpp, ptlp)));
+			   !(res = __follow_pte_pud(mm, address, range,
+						    ptepp, pmdpp, NULL, ptlp)));
 	return res;
 }
 EXPORT_SYMBOL(follow_pte_pmd);
 
+int follow_pte_pud(struct mm_struct *mm, unsigned long address,
+		   struct mmu_notifier_range *range,
+		   pte_t **ptepp, pmd_t **pmdpp, pud_t **pudpp, spinlock_t **ptlp)
+{
+	int res;
+
+	/* (void) is needed to make gcc happy */
+	(void) __cond_lock(*ptlp,
+			   !(res = __follow_pte_pud(mm, address, range,
+						    ptepp, pmdpp, pudpp, ptlp)));
+	return res;
+}
 /**
  * follow_pfn - look up PFN at a user virtual address
  * @vma: memory mapping
@@ -4808,15 +4840,19 @@ int follow_pfn(struct vm_area_struct *vma, unsigned long address,
 	spinlock_t *ptl;
 	pte_t *ptep;
 	pmd_t *pmdp = NULL;
+	pud_t *pudp = NULL;
 
 	if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)))
 		return ret;
 
-	ret = follow_pte_pmd(vma->vm_mm, address, NULL, &ptep, &pmdp, &ptl);
+	ret = follow_pte_pud(vma->vm_mm, address, NULL, &ptep, &pmdp, &pudp, &ptl);
 	if (ret)
 		return ret;
 
-	if (pmdp) {
+	if (pudp) {
+		*pfn = pud_pfn(*pudp) + ((address & ~PUD_MASK) >> PAGE_SHIFT);
+		spin_unlock(ptl);
+	} else if (pmdp) {
 		*pfn = pmd_pfn(*pmdp) + ((address & ~PMD_MASK) >> PAGE_SHIFT);
 		spin_unlock(ptl);
 	} else {
Signature · 1 line
1.8.3.1
#33 lore raw

RFC v2 30/37 dmem: introduce dmem_bitmap_alloc() and dmem_bitmap_free()

In reply to [email protected]

From: Yulei Zhang <[email protected]>

If dmem contained in dmem region is too large and dmemfs is mounted as 4K pagesize, size of bitmap in this dmem region maybe exceed maximal available memory of kzalloc(). It would cause kzalloc() fail.

So introduce dmem_bitmap_alloc() and use vzalloc() if bitmap is larger than PAGE_SIZE as vzalloc() will get sparse page.

---
 fs/inode.c         |  6 +++++
 include/linux/fs.h |  1 +
 mm/dmem.c          | 69 ++++++++++++++++++++++++++++++++++--------------------
 3 files changed, 50 insertions(+), 26 deletions(-)
fs/inode.c +6 -0
index 9d78c37..9b6363d3 100644
@@ -210,6 +210,12 @@ int inode_init_always(struct super_block *sb, struct inode *inode)
 }
 EXPORT_SYMBOL(inode_init_always);
 
+struct inode *alloc_inode_nonrcu(void)
+{
+	return kmem_cache_alloc(inode_cachep, GFP_KERNEL);
+}
+EXPORT_SYMBOL(alloc_inode_nonrcu);
+
 void free_inode_nonrcu(struct inode *inode)
 {
 	kmem_cache_free(inode_cachep, inode);
include/linux/fs.h +1 -0
index 8667d0c..bc7a89c 100644
@@ -2937,6 +2937,7 @@ static inline bool is_zero_ino(ino_t ino)
 extern void __destroy_inode(struct inode *);
 extern struct inode *new_inode_pseudo(struct super_block *sb);
 extern struct inode *new_inode(struct super_block *sb);
+extern struct inode *alloc_inode_nonrcu(void);
 extern void free_inode_nonrcu(struct inode *inode);
 extern int should_remove_suid(struct dentry *);
 extern int file_remove_privs(struct file *);
mm/dmem.c +43 -26

123 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#34 lore raw

RFC v2 31/37 dmem: introduce mce handler

In reply to [email protected]

From: Yulei Zhang <[email protected]>

dmem handle the mce if the pfn belongs to dmem when mce occurs.
1. check whether the pfn is handled by dmem. return if true.
2. mark the pfn in a new error bitmap defined in page.
3. a series of mechanism to ensure that the mce pfn is not allocated.

---
 include/linux/dmem.h        |   6 +++
 include/trace/events/dmem.h |  17 ++++++++
 mm/dmem.c                   | 103 +++++++++++++++++++++++++++++++-------------
 mm/memory-failure.c         |   6 +++
 4 files changed, 102 insertions(+), 30 deletions(-)
include/linux/dmem.h +6 -0
index 59d3ef14..cd17a91 100644
@@ -21,6 +21,8 @@
 void dmem_free_pages(phys_addr_t addr, unsigned int dpages_nr);
 bool is_dmem_pfn(unsigned long pfn);
 #define dmem_free_page(addr)	dmem_free_pages(addr, 1)
+
+bool dmem_memory_failure(unsigned long pfn, int flags);
 #else
 static inline int dmem_reserve_init(void)
 {
@@ -32,5 +34,9 @@ static inline bool is_dmem_pfn(unsigned long pfn)
 	return 0;
 }
 
+static inline bool dmem_memory_failure(unsigned long pfn, int flags)
+{
+	return false;
+}
 #endif
 #endif	/* _LINUX_DMEM_H */
include/trace/events/dmem.h +17 -0
index 10d1b90..f8eeb3c 100644
@@ -62,6 +62,23 @@
 	TP_printk("addr %#lx dpages_nr %d", (unsigned long)__entry->addr,
 		  __entry->dpages_nr)
 );
+
+TRACE_EVENT(dmem_memory_failure,
+	TP_PROTO(unsigned long pfn, bool used),
+	TP_ARGS(pfn, used),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, pfn)
+		__field(bool, used)
+	),
+
+	TP_fast_assign(
+		__entry->pfn = pfn;
+		__entry->used = used;
+	),
+
+	TP_printk("pfn=%#lx used=%d", __entry->pfn, __entry->used)
+);
 #endif
 
 /* This part must be outside protection */
mm/dmem.c +73 -30

127 lines not on this page · raw message

mm/memory-failure.c +6 -0
index 5d880d4..dda45d2 100644
@@ -35,6 +35,7 @@
  */
 #include <linux/kernel.h>
 #include <linux/mm.h>
+#include <linux/dmem.h>
 #include <linux/page-flags.h>
 #include <linux/kernel-page-flags.h>
 #include <linux/sched/signal.h>
@@ -1323,6 +1324,11 @@ int memory_failure(unsigned long pfn, int flags)
 	if (!sysctl_memory_failure_recovery)
 		panic("Memory failure on page %lx", pfn);
 
+	if (dmem_memory_failure(pfn, flags)) {
+		pr_info("MCE %#lx: handled by dmem\n", pfn);
+		return 0;
+	}
+
 	p = pfn_to_online_page(pfn);
 	if (!p) {
 		if (pfn_valid(pfn)) {
Signature · 1 line
1.8.3.1
#35 lore raw

RFC v2 32/37 mm, dmemfs: register and handle the dmem mce

In reply to [email protected]

From: Yulei Zhang <[email protected]>

dmemfs register the mce handler, send signal to the procs whose vma is mapped in mce pfn.

---
 fs/dmemfs/inode.c    | 141 +++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/dmem.h |   7 +++
 include/linux/mm.h   |   2 +
 mm/dmem.c            |  34 +++++++++++++
 mm/memory-failure.c  |  64 ++++++++++++++++-------
 5 files changed, 231 insertions(+), 17 deletions(-)
fs/dmemfs/inode.c +141 -0
index f698b9d..4303bcdc 100644
@@ -36,6 +36,47 @@
 
 static uint __read_mostly max_alloc_try_dpages = 1;
 
+struct dmemfs_inode {
+	struct inode *inode;
+	struct list_head link;
+};
+
+static LIST_HEAD(dmemfs_inode_list);
+static DEFINE_SPINLOCK(dmemfs_inode_lock);
+
+static struct dmemfs_inode *
+dmemfs_create_dmemfs_inode(struct inode *inode)
+{
+	struct dmemfs_inode *dmemfs_inode;
+
+	spin_lock(&dmemfs_inode_lock);
+	dmemfs_inode = kmalloc(sizeof(struct dmemfs_inode), GFP_NOIO);
+	if (!dmemfs_inode) {
+		pr_err("DMEMFS: Out of memory while getting dmemfs inode\n");
+		goto out;
+	}
+	dmemfs_inode->inode = inode;
+	list_add_tail(&dmemfs_inode->link, &dmemfs_inode_list);
+out:
+	spin_unlock(&dmemfs_inode_lock);
+	return dmemfs_inode;
+}
+
+static void dmemfs_delete_dmemfs_inode(struct inode *inode)
+{
+	struct dmemfs_inode *i, *next;
+
+	spin_lock(&dmemfs_inode_lock);
+	list_for_each_entry_safe(i, next, &dmemfs_inode_list, link) {
+		if (i->inode == inode) {
+			list_del(&i->link);
+			kfree(i);
+			break;
+		}
+	}
+	spin_unlock(&dmemfs_inode_lock);
+}
+
 struct dmemfs_mount_opts {
 	unsigned long dpage_size;
 };
@@ -218,6 +259,13 @@ static unsigned long dmem_pgoff_to_index(struct inode *inode, pgoff_t pgoff)
 	return pgoff >> (sb->s_blocksize_bits - PAGE_SHIFT);
 }
 
+static pgoff_t dmem_index_to_pgoff(struct inode *inode, unsigned long index)
+{
+	struct super_block *sb = inode->i_sb;
+
+	return index << (sb->s_blocksize_bits - PAGE_SHIFT);
+}
+
 static void *dmem_addr_to_entry(struct inode *inode, phys_addr_t addr)
 {
 	struct super_block *sb = inode->i_sb;
@@ -806,6 +854,23 @@ static void dmemfs_evict_inode(struct inode *inode)
 	clear_inode(inode);
 }
 
+static struct inode *dmemfs_alloc_inode(struct super_block *sb)
+{
+	struct inode *inode;
+
+	inode = alloc_inode_nonrcu();
+	if (inode)
+		dmemfs_create_dmemfs_inode(inode);
+	return inode;
+}
+
+static void dmemfs_destroy_inode(struct inode *inode)
+{
+	if (inode)
+		dmemfs_delete_dmemfs_inode(inode);
+	free_inode_nonrcu(inode);
+}
+
 /*
  * Display the mount options in /proc/mounts.
  */
@@ -819,9 +884,11 @@ static int dmemfs_show_options(struct seq_file *m, struct dentry *root)
 }
 
 static const struct super_operations dmemfs_ops = {
+	.alloc_inode = dmemfs_alloc_inode,
 	.statfs	= dmemfs_statfs,
 	.evict_inode = dmemfs_evict_inode,
 	.drop_inode = generic_delete_inode,
+	.destroy_inode = dmemfs_destroy_inode,
 	.show_options = dmemfs_show_options,
 };
 
@@ -901,17 +968,91 @@ static void dmemfs_kill_sb(struct super_block *sb)
 	.kill_sb	= dmemfs_kill_sb,
 };
 
+static struct inode *
+dmemfs_find_inode_by_addr(phys_addr_t addr, pgoff_t *pgoff)
+{
+	struct dmemfs_inode *di;
+	struct inode *inode;
+	struct address_space *mapping;
+	void *entry, **slot;
+	void *mce_entry;
+
+	list_for_each_entry(di, &dmemfs_inode_list, link) {
+		inode = di->inode;
+		mapping = inode->i_mapping;
+		mce_entry = dmem_addr_to_entry(inode, addr);
+		XA_STATE(xas, &mapping->i_pages, 0);
+		rcu_read_lock();
+
+		xas_for_each(&xas, entry, ULONG_MAX) {
+			if (xas_retry(&xas, entry))
+				continue;
+
+			if (unlikely(entry != xas_reload(&xas)))
+				goto retry;
+
+			if (mce_entry != entry)
+				continue;
+			*pgoff = dmem_index_to_pgoff(inode, xas.xa_index);
+			rcu_read_unlock();
+			return inode;
+retry:
+			xas_reset(&xas);
+		}
+		rcu_read_unlock();
+	}
+	return NULL;
+}
+
+static int dmemfs_mce_handler(struct notifier_block *this, unsigned long pfn,
+			      void *v)
+{
+	struct dmem_mce_notifier_info *info =
+		(struct dmem_mce_notifier_info *)v;
+	int flags = info->flags;
+	struct inode *inode;
+	phys_addr_t mce_addr = __pfn_to_phys(pfn);
+	pgoff_t pgoff;
+
+	spin_lock(&dmemfs_inode_lock);
+	inode = dmemfs_find_inode_by_addr(mce_addr, &pgoff);
+	if (!inode || !atomic_read(&inode->i_count))
+		goto out;
+
+	collect_procs_and_signal_inode(inode, pgoff, pfn, flags);
+out:
+	spin_unlock(&dmemfs_inode_lock);
+	return 0;
+}
+
+static struct notifier_block dmemfs_mce_notifier = {
+	.notifier_call	= dmemfs_mce_handler,
+};
+
 static int __init dmemfs_init(void)
 {
 	int ret;
 
+	pr_info("dmemfs initialized\n");
 	ret = register_filesystem(&dmemfs_fs_type);
+	if (ret)
+		goto reg_fs_fail;
+
+	ret = dmem_register_mce_notifier(&dmemfs_mce_notifier);
+	if (ret)
+		goto reg_notifier_fail;
 
+	return 0;
+
+reg_notifier_fail:
+	unregister_filesystem(&dmemfs_fs_type);
+reg_fs_fail:
 	return ret;
 }
 
 static void __exit dmemfs_uninit(void)
 {
+	dmem_unregister_mce_notifier(&dmemfs_mce_notifier);
 	unregister_filesystem(&dmemfs_fs_type);
 }
 
include/linux/dmem.h +7 -0
index cd17a91..fe0b270 100644
@@ -23,6 +23,13 @@
 #define dmem_free_page(addr)	dmem_free_pages(addr, 1)
 
 bool dmem_memory_failure(unsigned long pfn, int flags);
+
+struct dmem_mce_notifier_info {
+	int flags;
+};
+
+int dmem_register_mce_notifier(struct notifier_block *nb);
+int dmem_unregister_mce_notifier(struct notifier_block *nb);
 #else
 static inline int dmem_reserve_init(void)
 {
include/linux/mm.h +2 -0
index 2f3135fe..fa20f9c 100644
@@ -3041,6 +3041,8 @@ enum mf_flags {
 extern void memory_failure_queue(unsigned long pfn, int flags);
 extern void memory_failure_queue_kick(int cpu);
 extern int unpoison_memory(unsigned long pfn);
+extern void collect_procs_and_signal_inode(struct inode *inode, pgoff_t pgoff,
+				    unsigned long pfn, int flags);
 extern int sysctl_memory_failure_early_kill;
 extern int sysctl_memory_failure_recovery;
 extern void shake_page(struct page *p, int access);
mm/dmem.c +34 -0
index 16438db..dd81b24 100644
@@ -70,6 +70,7 @@ struct dmem_node {
 
 struct dmem_pool {
 	struct mutex lock;
+	struct raw_notifier_head mce_notifier_chain;
 
 	unsigned long region_num;
 	unsigned long registered_pages;
@@ -92,6 +93,7 @@ struct dmem_pool {
 
 static struct dmem_pool dmem_pool = {
 	.lock = __MUTEX_INITIALIZER(dmem_pool.lock),
+	.mce_notifier_chain = RAW_NOTIFIER_INIT(dmem_pool.mce_notifier_chain),
 };
 
 #define DMEM_PAGE_SIZE		(1UL << dmem_pool.dpage_shift)
@@ -121,6 +123,35 @@ struct dmem_pool {
 #define for_each_dmem_region(_dnode, _dregion)				\
 	list_for_each_entry(_dregion, &(_dnode)->regions, node)
 
+int dmem_register_mce_notifier(struct notifier_block *nb)
+{
+	int ret;
+
+	mutex_lock(&dmem_pool.lock);
+	ret = raw_notifier_chain_register(&dmem_pool.mce_notifier_chain, nb);
+	mutex_unlock(&dmem_pool.lock);
+	return ret;
+}
+EXPORT_SYMBOL(dmem_register_mce_notifier);
+
+int dmem_unregister_mce_notifier(struct notifier_block *nb)
+{
+	int ret;
+
+	mutex_lock(&dmem_pool.lock);
+	ret = raw_notifier_chain_unregister(&dmem_pool.mce_notifier_chain, nb);
+	mutex_unlock(&dmem_pool.lock);
+	return ret;
+}
+EXPORT_SYMBOL(dmem_unregister_mce_notifier);
+
+static int dmem_mce_notify(unsigned long pfn,
+			   struct dmem_mce_notifier_info *info)
+{
+	return raw_notifier_call_chain(&dmem_pool.mce_notifier_chain,
+				       pfn, info);
+}
+
 static inline int *dmem_nodelist(int nid)
 {
 	return nid_to_dnode(nid)->nodelist;
@@ -1003,6 +1034,7 @@ bool dmem_memory_failure(unsigned long pfn, int flags)
 	u64 pos;
 	phys_addr_t addr = __pfn_to_phys(pfn);
 	bool used = false;
+	struct dmem_mce_notifier_info info;
 
 	dregion = find_dmem_region(addr, &pdnode);
 	if (!dregion)
@@ -1022,6 +1054,8 @@ bool dmem_memory_failure(unsigned long pfn, int flags)
 	pos = phys_to_dpage(addr) - dregion->dpage_start_pfn;
 	if (__test_and_set_bit(pos, dregion->bitmap)) {
 		used = true;
+		info.flags = flags;
+		dmem_mce_notify(pfn, &info);
 	} else {
 		pr_info("MCE: free dpage, mark %#lx disabled in dmem\n", pfn);
 		dnode_count_free_dpages(pdnode, -1);
mm/memory-failure.c +47 -17

113 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#37 lore raw

RFC v2 33/37 kvm, x86: enable record_steal_time for dmem

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Adjust the kvm_map_gfn while using dmemfs to enable record_steal_time when entering the guest.

---
 virt/kvm/kvm_main.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
virt/kvm/kvm_main.c +9 -4
index 2541a17..500b170 100644
@@ -51,6 +51,7 @@
 #include <linux/io.h>
 #include <linux/lockdep.h>
 #include <linux/kthread.h>
+#include <linux/dmem.h>
 
 #include <asm/processor.h>
 #include <asm/ioctl.h>
@@ -2164,7 +2165,10 @@ static int __kvm_map_gfn(struct kvm_memslots *slots, gfn_t gfn,
 			hva = kmap(page);
 #ifdef CONFIG_HAS_IOMEM
 	} else if (!atomic) {
-		hva = memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB);
+		if (is_dmem_pfn(pfn))
+			hva = __va(PFN_PHYS(pfn));
+		else
+			hva = memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB);
 	} else {
 		return -EINVAL;
 #endif
@@ -2214,9 +2218,10 @@ static void __kvm_unmap_gfn(struct kvm_memory_slot *memslot,
 			kunmap(map->page);
 	}
 #ifdef CONFIG_HAS_IOMEM
-	else if (!atomic)
-		memunmap(map->hva);
-	else
+	else if (!atomic) {
+		if (!is_dmem_pfn(map->pfn))
+			memunmap(map->hva);
+	} else
 		WARN_ONCE(1, "Unexpected unmapping in atomic context");
 #endif
 
Signature · 1 line
1.8.3.1
#38 lore raw

RFC v2 34/37 dmem: add dmem unit tests

In reply to [email protected]

From: Yulei Zhang <[email protected]>

This test case is used to test dmem management system.

---
 tools/testing/dmem/Kbuild      |   1 +
 tools/testing/dmem/Makefile    |  10 +++
 tools/testing/dmem/dmem-test.c | 184 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 195 insertions(+)
 create mode 100644 tools/testing/dmem/Kbuild
 create mode 100644 tools/testing/dmem/Makefile
 create mode 100644 tools/testing/dmem/dmem-test.c
tools/testing/dmem/Kbuild new file +1 -0
new file mode 100644
index 00000000..04988f7
@@ -0,0 +1 @@
+obj-m += dmem-test.o
tools/testing/dmem/Makefile new file +10 -0
new file mode 100644
index 00000000..21f141f
@@ -0,0 +1,10 @@
+KDIR ?= ../../../
+
+default:
+	$(MAKE) -C $(KDIR) M=$$PWD
+
+install: default
+	$(MAKE) -C $(KDIR) M=$$PWD modules_install
+
+clean:
+	rm -f *.o *.ko Module.* modules.* *.mod.c
tools/testing/dmem/dmem-test.c new file +184 -0

184 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#39 lore raw

RFC v2 35/37 mm, dmem: introduce dregion->memmap for dmem

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Append 'memmap' into struct dmem_region, mapping each page of dmem with struct dmempage.

Currently there is just one member '_refcount' in struct dmempage to reflect the number of all modules which occupied the dmem page.

Modules which allocates the dmem page from dmempool will make first reference and set _refcount to 1.

Modules which try to free the dmem page to dmempool will decrease 1 at _refcount and free it if _refcount is tested as zero after decrease.

At each time module A passes dmem page to module B, module B should call get_dmem_pfn() to increase _refcount for dmem page before making use of it to avoid referencing a dmem page which is occasionally freeed by any other module in parallel. Vice versa after finishing usage of that dmem page need call put_dmem_pfn() to decrease the _refcount.

---
 include/linux/dmem.h |   5 ++
 mm/dmem.c            | 147 ++++++++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 139 insertions(+), 13 deletions(-)
include/linux/dmem.h +5 -0
index fe0b270..8aaa80b 100644
@@ -22,6 +22,9 @@
 bool is_dmem_pfn(unsigned long pfn);
 #define dmem_free_page(addr)	dmem_free_pages(addr, 1)
 
+void get_dmem_pfn(unsigned long pfn);
+#define put_dmem_pfn(pfn)	dmem_free_page(PFN_PHYS(pfn))
+
 bool dmem_memory_failure(unsigned long pfn, int flags);
 
 struct dmem_mce_notifier_info {
@@ -45,5 +48,7 @@ static inline bool dmem_memory_failure(unsigned long pfn, int flags)
 {
 	return false;
 }
+void get_dmem_pfn(unsigned long pfn) {}
+void put_dmem_pfn(unsigned long pfn) {}
 #endif
 #endif	/* _LINUX_DMEM_H */
mm/dmem.c +134 -13

224 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#41 lore raw

RFC v2 36/37 vfio: support dmempage refcount for vfio

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Add get/put_dmem_pfn(), each time when vfio module reference/release dmempages.

---
 drivers/vfio/vfio_iommu_type1.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/vfio/vfio_iommu_type1.c +8 -1
index c465d1a..4856a89 100644
@@ -39,6 +39,7 @@
 #include <linux/notifier.h>
 #include <linux/dma-iommu.h>
 #include <linux/irqdomain.h>
+#include <linux/dmem.h>
 
 #define DRIVER_VERSION  "0.2"
 #define DRIVER_AUTHOR   "Alex Williamson <[email protected]>"
@@ -411,7 +412,10 @@ static int put_pfn(unsigned long pfn, int prot)
 
 		unpin_user_pages_dirty_lock(&page, 1, prot & IOMMU_WRITE);
 		return 1;
-	}
+	} else if (is_dmem_pfn(pfn))
+		put_dmem_pfn(pfn);
+
+	/* Dmem page is not counted against user. */
 	return 0;
 }
 
@@ -477,6 +481,9 @@ static int vaddr_get_pfn(struct mm_struct *mm, unsigned long vaddr,
 
 		if (!ret && !is_invalid_reserved_pfn(*pfn))
 			ret = -EFAULT;
+
+		if (!ret && is_dmem_pfn(*pfn))
+			get_dmem_pfn(*pfn);
 	}
 done:
 	mmap_read_unlock(mm);
Signature · 1 line
1.8.3.1
#44 lore raw

RFC v2 37/37 Add documentation for dmemfs

In reply to [email protected]

From: Yulei Zhang <[email protected]>

Introduce dmemfs.rst to document the basic usage of dmemfs.

---
 Documentation/filesystems/dmemfs.rst | 58 ++++++++++++++++++++++++++++++++++++
 Documentation/filesystems/index.rst  |  1 +
 2 files changed, 59 insertions(+)
 create mode 100644 Documentation/filesystems/dmemfs.rst
Documentation/filesystems/dmemfs.rst new file +58 -0
new file mode 100644
index 00000000..f13ed0c
@@ -0,0 +1,58 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====================================
+The Direct Memory Filesystem - DMEMFS
+=====================================
+
+
+.. Table of contents
+
+   - Overview
+   - Compilation
+   - Usage
+
+Overview
+========
+
+Dmemfs (Direct Memory filesystem) is device memory or reserved
+memory based filesystem. This kind of memory is special as it
+is not managed by kernel and it is without 'struct page'. Therefore
+it can save extra memory from the host system for various usage,
+especially for guest virtual machines.
+
+It uses a kernel boot parameter ``dmem=`` to reserve the system
+memory when the host system boots up, the details can be checked
+in /Documentation/admin-guide/kernel-parameters.txt.
+
+Compilation
+===========
+
+The filesystem should be enabled by turning on the kernel configuration
+options::
+
+        CONFIG_DMEM_FS          - Direct Memory filesystem support
+        CONFIG_DMEM             - Allow reservation of memory for dmem
+
+
+Additionally, the following can be turned on to aid debugging::
+
+        CONFIG_DMEM_DEBUG_FS    - Enable debug information for dmem
+
+Usage
+========
+
+Dmemfs supports mapping ``4K``, ``2M`` and ``1G`` size of pages to
+the userspace, for example ::
+
+    # mount -t dmemfs none -o pagesize=4K /mnt/
+
+The it can create the backing storage with 4G size ::
+
+    # truncate /mnt/dmemfs-uuid --size 4G
+
+To use as backing storage for virtual machine starts with qemu, just need
+to specify the memory-backed-file in the qemu command line like this ::
+
+    # -object memory-backend-file,id=ram-node0,mem-path=/mnt/dmemfs-uuid \
+        share=yes,size=4G,host-nodes=0,policy=preferred -numa node,nodeid=0,memdev=ram-node0
+
Documentation/filesystems/index.rst +1 -0
index 98f59a8..23e944b 100644
@@ -120,3 +120,4 @@ Documentation for filesystem implementations.
    xfs-delayed-logging-design
    xfs-self-describing-metadata
    zonefs
+   dmemfs
Signature · 1 line
1.8.3.1
#46 lore raw
In reply to [email protected]
Quoted 9 lines · On 07.12.20 12:30, [email protected] wrote:

From: Yulei Zhang <[email protected]>

In current system each physical memory page is assocaited with a page structure which is used to track the usage of this page. But due to the memory usage rapidly growing in cloud environment, we find the resource consuming for page structure storage becomes more and more remarkable. So is it possible that we could reclaim such memory and make it reusable?

Quoted 8 lines

This patchset introduces an idea about how to save the extra memory through a new virtual filesystem -- dmemfs.

Dmemfs (Direct Memory filesystem) is device memory or reserved memory based filesystem. This kind of memory is special as it is not managed by kernel and most important it is without 'struct page'. Therefore we can leverage the extra memory from the host system to support more tenants in our cloud service.

"is not managed by kernel" well, it's obviously is managed by the kernel. It's not managed by the buddy ;)

How is this different to using "mem=X" and mapping the relevant memory directly into applications? Is this "simply" a control instance on top that makes sure unprivileged process can access it and not step onto each others feet? Is that the reason why it's called a "file system"? (an example would have helped here, showing how it's used)

It's worth noting that memory hotunplug, memory poisoning and probably more is currently fundamentally incompatible with this approach - which should better be pointed out in the cover letter.

Also, I think something similar can be obtained by using dax/hmat infrastructure with "memmap=", at least I remember a talk where this was discussed (but not sure if they modified the firmware to expose selected memory as soft-reserved - we would only need a cmdline parameter to achieve the same - Dan might know more).

Quoted 18 lines

As the belowing figure shows, we uses a kernel boot parameter 'dmem=' to reserve the system memory when the host system boots up, the remaining system memory is still managed by system memory management which is associated with "struct page", the reserved memory will be managed by dmem and assigned to guest system, the details can be checked in /Documentation/admin-guide/kernel-parameters.txt.

   +------------------+--------------------------------------+
   |  system memory   |     memory for guest system          | 
   +------------------+--------------------------------------+
    |                                   |
    v                                   |
struct page                             |
    |                                   |
    v                                   v
    system mem management             dmem  
Quoted 6 lines

And during the usage, the dmemfs will handle the memory request to allocate and free the reserved memory on each NUMA node, the user space application could leverage the mmap interface to access the memory, and kernel module such as kvm and vfio would be able to pin the memory thongh follow_pfn() and get_user_page() in different given page size granularities.

I cannot say that I really like this approach. I really prefer the proposal to free-up most vmemmap pages for huge/gigantic pages instead if all this is about is reducing the memmap size.

Signature · 3 lines
Thanks,

David / dhildenb
#2 lore raw

Re: [RFC V2 01/37] fs: introduce dmemfs module

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on vfio/next]
[also build test WARNING on linus/master hnaz-linux-mm/master v5.10-rc7] [cannot apply to tip/x86/core next-20201207]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/b97b91f94ab6c85095605bfebadea716c8485027
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
        git checkout b97b91f94ab6c85095605bfebadea716c8485027
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> fs/dmemfs/inode.c:225:5: warning: no previous prototype for 'dmemfs_init_fs_context' [-Wmissing-prototypes]

225 | int dmemfs_init_fs_context(struct fs_context *fc)
| ^~~~~~~~~~~~~~~~~~~~~~

vim +/dmemfs_init_fs_context +225 fs/dmemfs/inode.c

224

 > 225	int dmemfs_init_fs_context(struct fs_context *fc)

226 {
227 struct dmemfs_fs_info *fsi;
228
229 fsi = kzalloc(sizeof(*fsi), GFP_KERNEL);
230 if (!fsi)
231 return -ENOMEM;
232
233 fsi->mount_opts.dpage_size = PAGE_SIZE;
234 fc->s_fs_info = fsi;
235 fc->ops = &dmemfs_context_ops;
236 return 0;
237 }
238

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#42 lore raw

Re: [RFC V2 36/37] vfio: support dmempage refcount for vfio

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on vfio/next]
[also build test WARNING on linus/master v5.10-rc7]
[cannot apply to tip/x86/core hnaz-linux-mm/master next-20201207] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: s390-randconfig-r002-20201207 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project a2f922140f5380571fb74179f2bf622b3b925697)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install s390 cross compiling tool for clang build
# apt-get install binutils-s390x-linux-gnu
# https://github.com/0day-ci/linux/commit/fa80e126c6159426d3f10e5a1a44c50cf76659e8
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717 git checkout fa80e126c6159426d3f10e5a1a44c50cf76659e8
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=s390

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

                                                             ^
   include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
           ___constant_swab32(x) :                 \
                              ^
   include/uapi/linux/swab.h:19:12: note: expanded from macro '___constant_swab32'
           (((__u32)(x) & (__u32)0x000000ffUL) << 24) |            \
                     ^
   In file included from drivers/vfio/vfio_iommu_type1.c:27:
   In file included from include/linux/iommu.h:10:
   In file included from include/linux/scatterlist.h:9:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
                                                           ~~~~~~~~~~ ^
   include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
   #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
                                                             ^
   include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
           ___constant_swab32(x) :                 \
                              ^
   include/uapi/linux/swab.h:20:12: note: expanded from macro '___constant_swab32'
           (((__u32)(x) & (__u32)0x0000ff00UL) <<  8) |            \
                     ^
   In file included from drivers/vfio/vfio_iommu_type1.c:27:
   In file included from include/linux/iommu.h:10:
   In file included from include/linux/scatterlist.h:9:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
                                                           ~~~~~~~~~~ ^
   include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
   #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
                                                             ^
   include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
           ___constant_swab32(x) :                 \
                              ^
   include/uapi/linux/swab.h:21:12: note: expanded from macro '___constant_swab32'
           (((__u32)(x) & (__u32)0x00ff0000UL) >>  8) |            \
                     ^
   In file included from drivers/vfio/vfio_iommu_type1.c:27:
   In file included from include/linux/iommu.h:10:
   In file included from include/linux/scatterlist.h:9:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
                                                           ~~~~~~~~~~ ^
   include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
   #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
                                                             ^
   include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
           ___constant_swab32(x) :                 \
                              ^
   include/uapi/linux/swab.h:22:12: note: expanded from macro '___constant_swab32'
           (((__u32)(x) & (__u32)0xff000000UL) >> 24)))
                     ^
   In file included from drivers/vfio/vfio_iommu_type1.c:27:
   In file included from include/linux/iommu.h:10:
   In file included from include/linux/scatterlist.h:9:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
                                                           ~~~~~~~~~~ ^
   include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
   #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
                                                             ^
   include/uapi/linux/swab.h:120:12: note: expanded from macro '__swab32'
           __fswab32(x))
                     ^
   In file included from drivers/vfio/vfio_iommu_type1.c:27:
   In file included from include/linux/iommu.h:10:
   In file included from include/linux/scatterlist.h:9:
   In file included from arch/s390/include/asm/io.h:80:
   include/asm-generic/io.h:501:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           __raw_writeb(value, PCI_IOBASE + addr);
                               ~~~~~~~~~~ ^
   include/asm-generic/io.h:511:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           __raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr);
                                                         ~~~~~~~~~~ ^
   include/asm-generic/io.h:521:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           __raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr);
                                                         ~~~~~~~~~~ ^
   include/asm-generic/io.h:609:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           readsb(PCI_IOBASE + addr, buffer, count);
                  ~~~~~~~~~~ ^
   include/asm-generic/io.h:617:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           readsw(PCI_IOBASE + addr, buffer, count);
                  ~~~~~~~~~~ ^
   include/asm-generic/io.h:625:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           readsl(PCI_IOBASE + addr, buffer, count);
                  ~~~~~~~~~~ ^
   include/asm-generic/io.h:634:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           writesb(PCI_IOBASE + addr, buffer, count);
                   ~~~~~~~~~~ ^
   include/asm-generic/io.h:643:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           writesw(PCI_IOBASE + addr, buffer, count);
                   ~~~~~~~~~~ ^
   include/asm-generic/io.h:652:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
           writesl(PCI_IOBASE + addr, buffer, count);
                   ~~~~~~~~~~ ^
   In file included from drivers/vfio/vfio_iommu_type1.c:42:
>> include/linux/dmem.h:51:6: warning: no previous prototype for function 'get_dmem_pfn' [-Wmissing-prototypes]
   void get_dmem_pfn(unsigned long pfn) {}
        ^
   include/linux/dmem.h:51:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
   void get_dmem_pfn(unsigned long pfn) {}
   ^
   static 
>> include/linux/dmem.h:52:6: warning: no previous prototype for function 'put_dmem_pfn' [-Wmissing-prototypes]
   void put_dmem_pfn(unsigned long pfn) {}
        ^

include/linux/dmem.h:52:1: note: declare 'static' if the function is not intended to be used outside of this translation unit void put_dmem_pfn(unsigned long pfn) {}
^
static
22 warnings generated.

vim +/get_dmem_pfn +51 include/linux/dmem.h

182e41693f1d8c Yulei Zhang 2020-12-07  46  
9c950a98bb121c Yulei Zhang 2020-12-07  47  static inline bool dmem_memory_failure(unsigned long pfn, int flags)
9c950a98bb121c Yulei Zhang 2020-12-07  48  {
9c950a98bb121c Yulei Zhang 2020-12-07  49  	return false;
9c950a98bb121c Yulei Zhang 2020-12-07  50  }

b1e9c8986fe1d6 Yulei Zhang 2020-12-07 @51 void get_dmem_pfn(unsigned long pfn) {} b1e9c8986fe1d6 Yulei Zhang 2020-12-07 @52 void put_dmem_pfn(unsigned long pfn) {}

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#7 lore raw

Re: [RFC V2 05/37] dmemfs: support mmap for dmemfs

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on vfio/next]
[also build test WARNING on linus/master v5.10-rc7]
[cannot apply to tip/x86/core hnaz-linux-mm/master next-20201207] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/8d5ab8946768060e8933f8c9960f893618db7745
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
        git checkout 8d5ab8946768060e8933f8c9960f893618db7745
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> fs/dmemfs/inode.c:336:5: warning: no previous prototype for 'dmemfs_file_mmap' [-Wmissing-prototypes]

336 | int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
| ^~~~~~~~~~~~~~~~
fs/dmemfs/inode.c:558:5: warning: no previous prototype for 'dmemfs_init_fs_context' [-Wmissing-prototypes]
558 | int dmemfs_init_fs_context(struct fs_context *fc)
| ^~~~~~~~~~~~~~~~~~~~~~

vim +/dmemfs_file_mmap +336 fs/dmemfs/inode.c

335

 > 336	int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)

337 {
338 struct inode *inode = file_inode(file);
339
340 if (vma->vm_pgoff & ((dmem_page_size(inode) - 1) >> PAGE_SHIFT))
341 return -EINVAL;
342
343 if (!(vma->vm_flags & VM_SHARED))
344 return -EINVAL;
345
346 vma->vm_flags |= VM_PFNMAP;
347
348 file_accessed(file);
349 vma->vm_ops = &dmemfs_vm_ops;
350 return 0;
351 }
352

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#26 lore raw

Re: [RFC V2 23/37] kvm, x86: introduce VM_DMEM for syscall support usage

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on vfio/next]
[also build test WARNING on linus/master v5.10-rc7]
[cannot apply to tip/x86/core hnaz-linux-mm/master next-20201207] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: powerpc64-randconfig-r004-20201207 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project a2f922140f5380571fb74179f2bf622b3b925697)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install powerpc64 cross compiling tool for clang build
# apt-get install binutils-powerpc64-linux-gnu
# https://github.com/0day-ci/linux/commit/d38abd9f54f4002b764829dc886360e28b8342f6
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717 git checkout d38abd9f54f4002b764829dc886360e28b8342f6
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=powerpc64

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

In file included from arch/powerpc/kernel/asm-offsets.c:21:
In file included from include/linux/mman.h:5:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~

1 warning generated.
--
In file included from mm/filemap.c:15:
In file included from include/linux/dax.h:6:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/filemap.c:830:14: warning: no previous prototype for function '__add_to_page_cache_locked' [-Wmissing-prototypes]
   noinline int __add_to_page_cache_locked(struct page *page,
                ^
   mm/filemap.c:830:10: note: declare 'static' if the function is not intended to be used outside of this translation unit
   noinline int __add_to_page_cache_locked(struct page *page,
            ^
            static 
   2 warnings generated.
--
   In file included from mm/gup.c:7:
>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/gup.c:396:29: error: implicit declaration of function 'is_huge_zero_pmd' [-Werror,-Wimplicit-function-declaration]
           if ((flags & FOLL_DUMP) && is_huge_zero_pmd(*pmd))
                                      ^
   mm/gup.c:396:29: note: did you mean 'is_huge_zero_pud'?
   include/linux/huge_mm.h:455:20: note: 'is_huge_zero_pud' declared here
   static inline bool is_huge_zero_pud(pud_t pud)
                      ^
   mm/gup.c:412:10: error: implicit declaration of function 'pmd_mkyoung' [-Werror,-Wimplicit-function-declaration]
                   _pmd = pmd_mkyoung(*pmd);
                          ^
   mm/gup.c:412:10: note: did you mean 'pte_mkyoung'?
   arch/powerpc/include/asm/nohash/32/pgtable.h:185:21: note: 'pte_mkyoung' declared here
   static inline pte_t pte_mkyoung(pte_t pte)
                       ^
   mm/gup.c:412:8: error: assigning to 'pmd_t' from incompatible type 'int'
                   _pmd = pmd_mkyoung(*pmd);
                        ^ ~~~~~~~~~~~~~~~~~
   mm/gup.c:414:11: error: implicit declaration of function 'pmd_mkdirty' [-Werror,-Wimplicit-function-declaration]
                           _pmd = pmd_mkdirty(_pmd);
                                  ^
   mm/gup.c:414:11: note: did you mean 'pte_mkdirty'?
   arch/powerpc/include/asm/nohash/32/pgtable.h:180:21: note: 'pte_mkdirty' declared here
   static inline pte_t pte_mkdirty(pte_t pte)
                       ^
   mm/gup.c:414:9: error: assigning to 'pmd_t' from incompatible type 'int'
                           _pmd = pmd_mkdirty(_pmd);
                                ^ ~~~~~~~~~~~~~~~~~
   mm/gup.c:418:4: error: implicit declaration of function 'update_mmu_cache_pmd' [-Werror,-Wimplicit-function-declaration]
                           update_mmu_cache_pmd(vma, address, pmd);
                           ^
   mm/gup.c:418:4: note: did you mean 'update_mmu_cache'?
   arch/powerpc/include/asm/nohash/pgtable.h:299:6: note: 'update_mmu_cache' declared here
   void update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep) {}
        ^

1 warning and 6 errors generated.
--
In file included from mm/memory.c:42:
In file included from include/linux/kernel_stat.h:9:
In file included from include/linux/interrupt.h:11:
In file included from include/linux/hardirq.h:10:
In file included from arch/powerpc/include/asm/hardirq.h:6:
In file included from include/linux/irq.h:20:
In file included from include/linux/io.h:13:
In file included from arch/powerpc/include/asm/io.h:29:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/memory.c:4820:10: error: implicit declaration of function 'pmd_pfn' [-Werror,-Wimplicit-function-declaration]
                   *pfn = pmd_pfn(*pmdp) + ((address & ~PMD_MASK) >> PAGE_SHIFT);
                          ^
   1 warning and 1 error generated.
--
   In file included from mm/vmalloc.c:12:
>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/vmalloc.c:1318:6: warning: no previous prototype for function 'set_iounmap_nonlazy' [-Wmissing-prototypes]
   void set_iounmap_nonlazy(void)
        ^

mm/vmalloc.c:1318:1: note: declare 'static' if the function is not intended to be used outside of this translation unit void set_iounmap_nonlazy(void)
^
static
mm/vmalloc.c:472:1: warning: unused function 'compute_subtree_max_size' [-Wunused-function] compute_subtree_max_size(struct vmap_area *va)
^
3 warnings generated.
--
In file included from mm/page_alloc.c:19:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/page_alloc.c:3558:15: warning: no previous prototype for function 'should_fail_alloc_page' [-Wmissing-prototypes]
   noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
                 ^
   mm/page_alloc.c:3558:10: note: declare 'static' if the function is not intended to be used outside of this translation unit
   noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)
            ^
            static 
   mm/page_alloc.c:6188:23: warning: no previous prototype for function 'memmap_init' [-Wmissing-prototypes]
   void __meminit __weak memmap_init(unsigned long size, int nid,
                         ^

mm/page_alloc.c:6188:1: note: declare 'static' if the function is not intended to be used outside of this translation unit void __meminit __weak memmap_init(unsigned long size, int nid,
^
static
3 warnings generated.
--
In file included from mm/early_ioremap.c:13:
In file included from include/linux/io.h:13:
In file included from arch/powerpc/include/asm/io.h:29:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/early_ioremap.c:34:24: warning: no previous prototype for function 'early_memremap_pgprot_adjust' [-Wmissing-prototypes]
   pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr,
                          ^

mm/early_ioremap.c:34:1: note: declare 'static' if the function is not intended to be used outside of this translation unit pgprot_t __init __weak early_memremap_pgprot_adjust(resource_size_t phys_addr,
^
static
2 warnings generated.
--
In file included from mm/hmm.c:11:
In file included from include/linux/pagewalk.h:5:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~
   mm/hmm.c:174:29: warning: unused function 'pmd_to_hmm_pfn_flags' [-Wunused-function]
   static inline unsigned long pmd_to_hmm_pfn_flags(struct hmm_range *range,
                               ^

2 warnings generated.
--
In file included from arch/powerpc/kernel/asm-offsets.c:21:
In file included from include/linux/mman.h:5:

>> include/linux/mm.h:673:28: warning: shift count >= width of type [-Wshift-count-overflow]
           return !!(vma->vm_flags & VM_DMEM);
                                     ^~~~~~~
   include/linux/mm.h:314:18: note: expanded from macro 'VM_DMEM'
   #define VM_DMEM         BIT(38)         /* Dmem page VM */
                           ^~~~~~~
   include/vdso/bits.h:7:26: note: expanded from macro 'BIT'
   #define BIT(nr)                 (UL(1) << (nr))
                                          ^  ~~~~

1 warning generated.

vim +673 include/linux/mm.h

670
671 static inline bool vma_is_dmem(struct vm_area_struct *vma)
672 {

 > 673		return !!(vma->vm_flags & VM_DMEM);

674 }
675

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#36 lore raw

Re: [RFC V2 32/37] mm, dmemfs: register and handle the dmem mce

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on vfio/next]
[also build test WARNING on linus/master v5.10-rc7]
[cannot apply to tip/x86/core hnaz-linux-mm/master next-20201207] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/dc3ab3d8e49e2439ea8b8c67164712cd70b91d7a
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
        git checkout dc3ab3d8e49e2439ea8b8c67164712cd70b91d7a
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

fs/dmemfs/inode.c:646:5: warning: no previous prototype for 'dmemfs_file_mmap' [-Wmissing-prototypes]
646 | int dmemfs_file_mmap(struct file *file, struct vm_area_struct *vma)
| ^~~~~~~~~~~~~~~~
fs/dmemfs/inode.c:939:5: warning: no previous prototype for 'dmemfs_init_fs_context' [-Wmissing-prototypes]
939 | int dmemfs_init_fs_context(struct fs_context *fc)
| ^~~~~~~~~~~~~~~~~~~~~~
In file included from include/linux/radix-tree.h:19,

                    from include/linux/idr.h:15,
                    from include/linux/kernfs.h:13,
                    from include/linux/sysfs.h:16,
                    from include/linux/kobject.h:20,
                    from include/linux/module.h:20,
                    from fs/dmemfs/inode.c:12:
   fs/dmemfs/inode.c: In function 'dmemfs_find_inode_by_addr':
>> include/linux/xarray.h:1349:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
    1349 |  struct xa_state name = __XA_STATE(array, index, 0, 0)
         |  ^~~~~~
   fs/dmemfs/inode.c:984:3: note: in expansion of macro 'XA_STATE'
     984 |   XA_STATE(xas, &mapping->i_pages, 0);
         |   ^~~~~~~~
   fs/dmemfs/inode.c:977:17: warning: unused variable 'slot' [-Wunused-variable]
     977 |  void *entry, **slot;
         |                 ^~~~

vim +1349 include/linux/xarray.h

ad3d6c7263e368 Matthew Wilcox 2017-11-07  1339  
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1340  /**
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1341   * XA_STATE() - Declare an XArray operation state.
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1342   * @name: Name of this operation state (usually xas).
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1343   * @array: Array to operate on.
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1344   * @index: Initial index of interest.
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1345   *
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1346   * Declare and initialise an xa_state on the stack.
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1347   */
ad3d6c7263e368 Matthew Wilcox 2017-11-07  1348  #define XA_STATE(name, array, index)				\

ad3d6c7263e368 Matthew Wilcox 2017-11-07 @1349 struct xa_state name = __XA_STATE(array, index, 0, 0)
ad3d6c7263e368 Matthew Wilcox 2017-11-07 1350

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#40 lore raw

Re: [RFC V2 35/37] mm, dmem: introduce dregion->memmap for dmem

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on vfio/next]
[also build test ERROR on linus/master v5.10-rc7]
[cannot apply to tip/x86/core hnaz-linux-mm/master next-20201207] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-randconfig-a016-20201207 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/b1e9c8986fe1d62e7fb893105940904fd570cba1
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
        git checkout b1e9c8986fe1d62e7fb893105940904fd570cba1
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

ld: arch/x86/kvm/mmu/mmu.o: in function `get_dmem_pfn':

>> include/linux/dmem.h:51: multiple definition of `get_dmem_pfn'; arch/x86/../../virt/kvm/kvm_main.o:include/linux/dmem.h:51: first defined here
   ld: arch/x86/kvm/mmu/mmu.o: in function `put_dmem_pfn':
>> include/linux/dmem.h:51: multiple definition of `put_dmem_pfn'; arch/x86/../../virt/kvm/kvm_main.o:include/linux/dmem.h:51: first defined here
   ld: arch/x86/kernel/setup.o: in function `get_dmem_pfn':
>> include/linux/dmem.h:51: multiple definition of `get_dmem_pfn'; arch/x86/../../virt/kvm/kvm_main.o:include/linux/dmem.h:51: first defined here
   ld: arch/x86/kernel/setup.o: in function `put_dmem_pfn':
>> include/linux/dmem.h:51: multiple definition of `put_dmem_pfn'; arch/x86/../../virt/kvm/kvm_main.o:include/linux/dmem.h:51: first defined here

vim +51 include/linux/dmem.h

    46	
    47	static inline bool dmem_memory_failure(unsigned long pfn, int flags)
    48	{
    49		return false;
    50	}
  > 51	void get_dmem_pfn(unsigned long pfn) {}

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#43 lore raw

Re: [RFC V2 36/37] vfio: support dmempage refcount for vfio

In reply to [email protected]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on vfio/next]
[also build test ERROR on linus/master v5.10-rc7]
[cannot apply to tip/x86/core hnaz-linux-mm/master next-20201207] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-randconfig-a002-20201207 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project a2f922140f5380571fb74179f2bf622b3b925697)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/fa80e126c6159426d3f10e5a1a44c50cf76659e8
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review yulei-kernel-gmail-com/Enhance-memory-utilization-with-DMEMFS/20201207-193717 git checkout fa80e126c6159426d3f10e5a1a44c50cf76659e8
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

>> ld.lld: error: duplicate symbol: get_dmem_pfn
   >>> defined at dmem.h:51 (include/linux/dmem.h:51)
   >>> kernel/setup.o:(get_dmem_pfn) in archive arch/x86/built-in.a
   >>> defined at dmem.h:51 (include/linux/dmem.h:51)
   >>> vfio/vfio_iommu_type1.o:(.text+0x0) in archive drivers/built-in.a
--
>> ld.lld: error: duplicate symbol: put_dmem_pfn
   >>> defined at dmem.h:52 (include/linux/dmem.h:52)
   >>> kernel/setup.o:(put_dmem_pfn) in archive arch/x86/built-in.a
   >>> defined at dmem.h:52 (include/linux/dmem.h:52)
   >>> vfio/vfio_iommu_type1.o:(.text+0x20) in archive drivers/built-in.a

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#47 lore raw
In reply to David Hildenbrand
Quoted 33 lines · David Hildenbrand
Quoted 9 lines · On 07.12.20 12:30, [email protected] wrote:

From: Yulei Zhang <[email protected]>

In current system each physical memory page is assocaited with a page structure which is used to track the usage of this page. But due to the memory usage rapidly growing in cloud environment, we find the resource consuming for page structure storage becomes more and more remarkable. So is it possible that we could reclaim such memory and make it reusable?

Quoted 8 lines

This patchset introduces an idea about how to save the extra memory through a new virtual filesystem -- dmemfs.

Dmemfs (Direct Memory filesystem) is device memory or reserved memory based filesystem. This kind of memory is special as it is not managed by kernel and most important it is without 'struct page'. Therefore we can leverage the extra memory from the host system to support more tenants in our cloud service.

"is not managed by kernel" well, it's obviously is managed by the kernel. It's not managed by the buddy ;)

How is this different to using "mem=X" and mapping the relevant memory directly into applications? Is this "simply" a control instance on top that makes sure unprivileged process can access it and not step onto each others feet? Is that the reason why it's called a "file system"? (an example would have helped here, showing how it's used)

It's worth noting that memory hotunplug, memory poisoning and probably more is currently fundamentally incompatible with this approach - which should better be pointed out in the cover letter.

Quoted 5 lines

Also, I think something similar can be obtained by using dax/hmat infrastructure with "memmap=", at least I remember a talk where this was discussed (but not sure if they modified the firmware to expose selected memory as soft-reserved - we would only need a cmdline parameter to achieve the same - Dan might know more).

There is currently the efi_fake_mem parameter that can add the "EFI_MEMORY_SP" attribute on EFI platforms:

    efi_fake_mem=4G@9G:0x40000

...this results in a /dev/dax instance that can be further partitioned via the device-dax sub-division facility merged for 5.10. That could be generalized to something else for non-EFI platforms, but there has not been a justification to go that route yet.

Joao pointed this out in a previous posting of DMEMFS, and I have yet to see an explanation of incremental benefit the kernel gains from having yet another parallel memory management interface.

reviewer
#45 lore raw

Re: [RFC V2 37/37] Add documentation for dmemfs

In reply to [email protected]

Hi,

Quoted 27 lines · On 12/7/20 3:31 AM, [email protected] wrote:

From: Yulei Zhang <[email protected]>

Introduce dmemfs.rst to document the basic usage of dmemfs.

---
 Documentation/filesystems/dmemfs.rst | 58 ++++++++++++++++++++++++++++++++++++
 Documentation/filesystems/index.rst  |  1 +
 2 files changed, 59 insertions(+)
 create mode 100644 Documentation/filesystems/dmemfs.rst
Documentation/filesystems/dmemfs.rst new file +10 -0
new file mode 100644
index 00000000..f13ed0c
@@ -0,0 +1,58 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====================================
+The Direct Memory Filesystem - DMEMFS
+=====================================
+
+
+.. Table of contents
+
+   - Overview

Documentation/filesystems/dmemfs.rst line 20

Quoted 10 lines
Documentation/filesystems/dmemfs.rst new file +10 -0
+   - Compilation
+   - Usage
+
+Overview
+========
+
+Dmemfs (Direct Memory filesystem) is device memory or reserved
+memory based filesystem. This kind of memory is special as it
+is not managed by kernel and it is without 'struct page'. Therefore
+it can save extra memory from the host system for various usage,

usages,
or uses,

Documentation/filesystems/dmemfs.rst line 24

Quoted 4 lines
Documentation/filesystems/dmemfs.rst new file +4 -0
+especially for guest virtual machines.
+
+It uses a kernel boot parameter ``dmem=`` to reserve the system
+memory when the host system boots up, the details can be checked

boots up. The detail

Documentation/filesystems/dmemfs.rst line 34

Quoted 10 lines
Documentation/filesystems/dmemfs.rst new file +10 -0
+in /Documentation/admin-guide/kernel-parameters.txt.
+
+Compilation
+===========
+
+The filesystem should be enabled by turning on the kernel configuration
+options::
+
+        CONFIG_DMEM_FS          - Direct Memory filesystem support
+        CONFIG_DMEM             - Allow reservation of memory for dmem

Would anyone want DMEM_FS without DMEM?

Documentation/filesystems/dmemfs.rst line 44

Quoted 10 lines
Documentation/filesystems/dmemfs.rst new file +10 -0
+
+
+Additionally, the following can be turned on to aid debugging::
+
+        CONFIG_DMEM_DEBUG_FS    - Enable debug information for dmem
+
+Usage
+========
+
+Dmemfs supports mapping ``4K``, ``2M`` and ``1G`` size of pages to

sizes

Documentation/filesystems/dmemfs.rst line 45

Quoted 1 line
Documentation/filesystems/dmemfs.rst new file +1 -0
+the userspace, for example ::

userspace. For example::

Documentation/filesystems/dmemfs.rst line 49

Quoted 4 lines
Documentation/filesystems/dmemfs.rst new file +4 -0
+
+    # mount -t dmemfs none -o pagesize=4K /mnt/
+
+The it can create the backing storage with 4G size ::

Then

Documentation/filesystems/dmemfs.rst line 53

Quoted 4 lines
Documentation/filesystems/dmemfs.rst new file +4 -0
+
+    # truncate /mnt/dmemfs-uuid --size 4G
+
+To use as backing storage for virtual machine starts with qemu, just need

started with qemu, just specify
the memory-backed-file

Documentation/filesystems/dmemfs.rst line 56

Quoted 3 lines
Documentation/filesystems/dmemfs.rst new file +3 -0
+to specify the memory-backed-file in the qemu command line like this ::
+
+    # -object memory-backend-file,id=ram-node0,mem-path=/mnt/dmemfs-uuid \

backed

Quoted 2 lines
Documentation/filesystems/dmemfs.rst new file +2 -0
+        share=yes,size=4G,host-nodes=0,policy=preferred -numa node,nodeid=0,memdev=ram-node0
+
Signature · 1 line
~Randy