Skip to contentSkip to conversation

numa: add unit testing plus fix regression

PATCH Superseded Roger Pau Monne 5 participants 21 messages Jun 1 – Jun 3

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover numa: add unit testing plus fix regression

Hello,

The first 4 patches add some basic unit testing for the NUMA setup logic. The last patch expands the test cases and fixes an issue the new test-cases would otherwise trigger.

Thanks, Roger.

Roger Pau Monne (5):
tools/bitops: adjust bitmap_or() interface to match hypervisor tools/macros: adjust ROUNDUP() interface to match hypervisor xen/numa: prepare NUMA setup code for unit testing tests/numa: add unit tests for NUMA setup logic xen/numa: fix setup of non-aligned memory affinity ranges

 tools/console/daemon/io.c                 |   2 +-
 tools/include/xen-tools/bitops.h          |   7 +-
 tools/include/xen-tools/common-macros.h   |   5 +-
 tools/libs/call/buffer.c                  |   3 +-
 tools/libs/foreignmemory/linux.c          |   2 +-
 tools/libs/gnttab/freebsd.c               |   2 +-
 tools/libs/gnttab/linux.c                 |   2 +-
 tools/libs/guest/xg_core.c                |   2 +-
 tools/libs/guest/xg_dom_arm.c             |   6 +-
 tools/libs/guest/xg_dom_x86.c             |   2 +-
 tools/libs/guest/xg_private.h             |   4 +-
 tools/libs/guest/xg_sr_common.c           |   6 +-
 tools/libs/guest/xg_sr_save.c             |   3 +-
 tools/libs/guest/xg_sr_stream_format.h    |   2 +-
 tools/libs/light/libxl_arm_acpi.c         |  24 +-
 tools/libs/light/libxl_create.c           |   2 +-
 tools/libs/light/libxl_sr_stream_format.h |   2 +-
 tools/libs/light/libxl_stream_read.c      |   2 +-
 tools/libs/light/libxl_stream_write.c     |   4 +-
 tools/misc/xen-mfndump.c                  |   2 +-
 tools/ocaml/libs/xc/xenctrl_stubs.c       |   2 +-
 tools/tests/Makefile                      |   1 +
 tools/tests/numa/.gitignore               |   2 +
 tools/tests/numa/Makefile                 |  47 ++++
 tools/tests/numa/harness.h                | 184 +++++++++++++++
 tools/tests/numa/test-numa.c              | 267 ++++++++++++++++++++++
 tools/xenstored/core.c                    |   4 +-
 tools/xenstored/domain.c                  |   9 +-
 tools/xenstored/watch.c                   |   2 +-
 xen/common/numa.c                         |  22 +-
 30 files changed, 569 insertions(+), 55 deletions(-)
 create mode 100644 tools/tests/numa/.gitignore
 create mode 100644 tools/tests/numa/Makefile
 create mode 100644 tools/tests/numa/harness.h
 create mode 100644 tools/tests/numa/test-numa.c
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/5 tools/bitops: adjust bitmap_or() interface to match hypervisor

Adjust the only toolstack caller to use the new interface. No functional change intended.

---
I'm tempted to rewrite bitmap_or() to use unsigned long (just like the hypervisor side), but that's outside the scope of this change.

---
 tools/include/xen-tools/bitops.h | 7 ++++---
 tools/libs/guest/xg_sr_save.c    | 3 ++-
 2 files changed, 6 insertions(+), 4 deletions(-)
tools/include/xen-tools/bitops.h +4 -3
index 3b98fba6d74c..29587e89fa6d 100644
@@ -81,14 +81,15 @@ static inline int test_and_set_bit(unsigned long nr, void *addr)
     return oldbit;
 }
 
-static inline void bitmap_or(void *_dst, const void *_other,
+static inline void bitmap_or(void *_dst, const void *_src1, const void *_src2,
                              unsigned long nr_bits)
 {
     char *dst = _dst;
-    const char *other = _other;
+    const char *src1 = _src1, *src2 = _src2;
     unsigned long i;
+
     for ( i = 0; i < bitmap_size(nr_bits); ++i )
-        dst[i] |= other[i];
+        dst[i] = src1[i] | src2[i];
 }
 
 #endif  /* __XEN_TOOLS_BITOPS_H__ */
tools/libs/guest/xg_sr_save.c +2 -1
index 3b2c5222e429..fdbceab52e46 100644
@@ -668,7 +668,8 @@ static int suspend_and_send_dirty(struct xc_sr_context *ctx)
     else
         xc_set_progress_prefix(xch, "Checkpointed save");
 
-    bitmap_or(dirty_bitmap, ctx->save.deferred_pages, ctx->save.p2m_size);
+    bitmap_or(dirty_bitmap, dirty_bitmap, ctx->save.deferred_pages,
+              ctx->save.p2m_size);
 
     if ( !ctx->save.live && ctx->stream_type == XC_STREAM_COLO )
     {
Signature · 1 line
2.53.0
Acked-by on v1 1/5
#2 lore raw

Re: [PATCH for-4.22 1/5] tools/bitops: adjust bitmap_or() interface to match hypervisor

Quoted 4 lines · Roger Pau Monne

Adjust the only toolstack caller to use the new interface. No functional change intended.

Quoted 3 lines

---
I'm tempted to rewrite bitmap_or() to use unsigned long (just like the hypervisor side), but that's outside the scope of this change.

You mean revert 11d0044a1689 ("tools/libxc: Modify bitmap operations to take void pointers") ? Or maybe the hypervisor code when trough the same path.

Thanks,

--
Anthony Perard | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech

#3 lore raw

Re: [PATCH for-4.22 1/5] tools/bitops: adjust bitmap_or() interface to match hypervisor

Quoted 7 lines · Anthony PERARD
Quoted 4 lines · Roger Pau Monne

Adjust the only toolstack caller to use the new interface. No functional change intended.

Thanks.

Quoted 7 lines
Quoted 3 lines

---
I'm tempted to rewrite bitmap_or() to use unsigned long (just like the hypervisor side), but that's outside the scope of this change.

You mean revert 11d0044a1689 ("tools/libxc: Modify bitmap operations to take void pointers") ? Or maybe the hypervisor code when trough the same path.

Kind of yes, so that it matches more closely the hypervisor interface. Anyway, not going to do that now either.

Regards, Roger.

#4 lore raw

PATCH 2/5 tools/macros: adjust ROUNDUP() interface to match hypervisor

In reply to Roger Pau Monne

Adjust user-space callers to use the new interface. No functional change intended.

---
 tools/console/daemon/io.c                 |  2 +-
 tools/include/xen-tools/common-macros.h   |  4 +---
 tools/libs/call/buffer.c                  |  3 ++-
 tools/libs/foreignmemory/linux.c          |  2 +-
 tools/libs/gnttab/freebsd.c               |  2 +-
 tools/libs/gnttab/linux.c                 |  2 +-
 tools/libs/guest/xg_core.c                |  2 +-
 tools/libs/guest/xg_dom_arm.c             |  6 +++---
 tools/libs/guest/xg_dom_x86.c             |  2 +-
 tools/libs/guest/xg_private.h             |  4 ++--
 tools/libs/guest/xg_sr_common.c           |  6 +++---
 tools/libs/guest/xg_sr_stream_format.h    |  2 +-
 tools/libs/light/libxl_arm_acpi.c         | 24 +++++++++++------------
 tools/libs/light/libxl_create.c           |  2 +-
 tools/libs/light/libxl_sr_stream_format.h |  2 +-
 tools/libs/light/libxl_stream_read.c      |  2 +-
 tools/libs/light/libxl_stream_write.c     |  4 ++--
 tools/misc/xen-mfndump.c                  |  2 +-
 tools/ocaml/libs/xc/xenctrl_stubs.c       |  2 +-
 tools/xenstored/core.c                    |  4 ++--
 tools/xenstored/domain.c                  |  9 +++++----
 tools/xenstored/watch.c                   |  2 +-
 22 files changed, 45 insertions(+), 45 deletions(-)
tools/console/daemon/io.c +1 -1
index 43d4973c247c..b6c46d11dea7 100644
@@ -1233,7 +1233,7 @@ static int set_fds(int fd, short events)
 		/* Round up to 2^8 boundary, in practice this just
 		 * make newsize larger than current_array_size.
 		 */
-		newsize = ROUNDUP(nr_fds + 1, 8);
+		newsize = ROUNDUP(nr_fds + 1, 1U << 8);
 
 		new_fds = realloc(fds, sizeof(struct pollfd)*newsize);
 		if (!new_fds)
tools/include/xen-tools/common-macros.h +1 -3
index 9838a108aaea..9e2799178235 100644
@@ -68,9 +68,7 @@
     })
 #endif
 
-#ifndef ROUNDUP
-#define ROUNDUP(_x,_w) (((unsigned long)(_x)+(1UL<<(_w))-1) & ~((1UL<<(_w))-1))
-#endif
+#define ROUNDUP(x, a) (((x) + (a) - 1) & ~((a) - 1))
 
 #define MASK_EXTR(v, m) (((v) & (m)) / ((m) & -(m)))
 #define MASK_INSR(v, m) (((v) * ((m) & -(m))) & (m))
tools/libs/call/buffer.c +2 -1
index 2579b8c71986..155e4f9d43eb 100644
@@ -155,7 +155,8 @@ struct allocation_header {
 
 void *xencall_alloc_buffer(xencall_handle *xcall, size_t size)
 {
-    size_t actual_size = ROUNDUP(size + sizeof(struct allocation_header), PAGE_SHIFT);
+    size_t actual_size = ROUNDUP(size + sizeof(struct allocation_header),
+                                 PAGE_SIZE);
     int nr_pages = actual_size >> PAGE_SHIFT;
     struct allocation_header *hdr;
 
tools/libs/foreignmemory/linux.c +1 -1
index 12f959765abd..6d2f30cdf1b6 100644
@@ -198,7 +198,7 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle *fmem,
          */
         privcmd_mmapbatch_t ioctlx;
         xen_pfn_t *pfn;
-        unsigned int pfn_arr_size = ROUNDUP((num * sizeof(*pfn)), XC_PAGE_SHIFT);
+        unsigned int pfn_arr_size = ROUNDUP(num * sizeof(*pfn), XC_PAGE_SIZE);
         int os_page_size = sysconf(_SC_PAGESIZE);
 
         if ( pfn_arr_size <= os_page_size )
tools/libs/gnttab/freebsd.c +1 -1
index d69d928a165e..801274478217 100644
@@ -74,7 +74,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
     int domids_stride;
     unsigned int refs_size = ROUNDUP(count *
                                      sizeof(struct ioctl_gntdev_grant_ref),
-                                     XC_PAGE_SHIFT);
+                                     XC_PAGE_SIZE);
     int os_page_size = getpagesize();
 
     domids_stride = (flags & XENGNTTAB_GRANT_MAP_SINGLE_DOMAIN) ? 0 : 1;
tools/libs/gnttab/linux.c +1 -1
index 7286c1d4fe83..829877e64baa 100644
@@ -101,7 +101,7 @@ void *osdep_gnttab_grant_map(xengnttab_handle *xgt,
         map = alloca(map_size);
     else
     {
-        map_size = ROUNDUP(map_size, XC_PAGE_SHIFT);
+        map_size = ROUNDUP(map_size, XC_PAGE_SIZE);
         map = mmap(NULL, map_size, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANON | MAP_POPULATE, -1, 0);
         if ( map == MAP_FAILED )
tools/libs/guest/xg_core.c +1 -1
index f83436d6cbf8..d3640f0ef891 100644
@@ -696,7 +696,7 @@ xc_domain_dumpcore_via_callback(xc_interface *xch,
     for ( i = 1; i < sheaders->num; i++ )
         sheaders->shdrs[i].sh_offset += fixup;
     offset += fixup;
-    dummy_len = ROUNDUP(offset, PAGE_SHIFT) - offset; /* padding length */
+    dummy_len = ROUNDUP(offset, PAGE_SIZE) - offset; /* padding length */
     offset += dummy_len;
 
     /* pages */
tools/libs/guest/xg_dom_arm.c +3 -3
index c8d091850622..739ec1c33837 100644
@@ -364,12 +364,12 @@ static int meminit(struct xc_dom_image *dom)
 
     /* Convenient */
     const uint64_t kernbase = dom->kernel_seg.vstart;
-    const uint64_t kernend = ROUNDUP(dom->kernel_seg.vend, 21/*2MB*/);
+    const uint64_t kernend = ROUNDUP(dom->kernel_seg.vend, MB(2));
     const uint64_t kernsize = kernend - kernbase;
     const uint64_t dtb_size = dom->devicetree_blob ?
-        ROUNDUP(dom->devicetree_size, XC_PAGE_SHIFT) : 0;
+        ROUNDUP(dom->devicetree_size, XC_PAGE_SIZE) : 0;
     const uint64_t ramdisk_size = dom->modules[0].blob ?
-        ROUNDUP(dom->modules[0].size, XC_PAGE_SHIFT) : 0;
+        ROUNDUP(dom->modules[0].size, XC_PAGE_SIZE) : 0;
     const uint64_t modsize = dtb_size + ramdisk_size;
     const uint64_t ram128mb = bankbase[0] + (128<<20);
 
tools/libs/guest/xg_dom_x86.c +1 -1
index 268936efe25b..9af83535944a 100644
@@ -678,7 +678,7 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
     {
         if ( dom->cmdline )
         {
-            dom->cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 3);
+            dom->cmdline_size = ROUNDUP(strlen(dom->cmdline) + 1, 1U << 3);
             start_info_size += dom->cmdline_size;
         }
     }
tools/libs/guest/xg_private.h +2 -2
index 285229cf82a3..31a79bccf75e 100644
@@ -135,7 +135,7 @@ typedef uint64_t x86_pgentry_t;
 #define PAGE_SIZE_X86           (1UL << PAGE_SHIFT_X86)
 #define PAGE_MASK_X86           (~(PAGE_SIZE_X86-1))
 
-#define NRPAGES(x) (ROUNDUP(x, PAGE_SHIFT) >> PAGE_SHIFT)
+#define NRPAGES(x) (ROUNDUP(x, PAGE_SIZE) >> PAGE_SHIFT)
 
 static inline xen_pfn_t xc_pfn_to_mfn(xen_pfn_t pfn, xen_pfn_t *p2m,
                                       unsigned gwidth)
@@ -167,7 +167,7 @@ int pin_table(xc_interface *xch, unsigned int type, unsigned long mfn,
  */
 #define M2P_SHIFT       L2_PAGETABLE_SHIFT_PAE
 #define M2P_CHUNK_SIZE  (1 << M2P_SHIFT)
-#define M2P_SIZE(_m)    ROUNDUP(((_m) * sizeof(xen_pfn_t)), M2P_SHIFT)
+#define M2P_SIZE(_m)    ROUNDUP(((_m) * sizeof(xen_pfn_t)), M2P_CHUNK_SIZE)
 #define M2P_CHUNKS(_m)  (M2P_SIZE((_m)) >> M2P_SHIFT)
 
 #if defined(__x86_64__) || defined(__i386__)
tools/libs/guest/xg_sr_common.c +3 -3
index 7ccdc3b1f6aa..54dde924a7c0 100644
@@ -56,11 +56,11 @@ const char *rec_type_to_str(uint32_t type)
 int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec,
                        void *buf, size_t sz)
 {
-    static const char zeroes[(1u << REC_ALIGN_ORDER) - 1] = { 0 };
+    static const char zeroes[REC_ALIGN - 1] = { 0 };
 
     xc_interface *xch = ctx->xch;
     typeof(rec->length) combined_length = rec->length + sz;
-    size_t record_length = ROUNDUP(combined_length, REC_ALIGN_ORDER);
+    size_t record_length = ROUNDUP(combined_length, REC_ALIGN);
     struct iovec parts[] = {
         { &rec->type,       sizeof(rec->type) },
         { &combined_length, sizeof(combined_length) },
@@ -110,7 +110,7 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec)
         return -1;
     }
 
-    datasz = ROUNDUP(rhdr.length, REC_ALIGN_ORDER);
+    datasz = ROUNDUP(rhdr.length, REC_ALIGN);
 
     if ( datasz )
     {
tools/libs/guest/xg_sr_stream_format.h +1 -1
index 8a0da26f7543..4310f4311e65 100644
@@ -53,7 +53,7 @@ struct xc_sr_rhdr
 };
 
 /* All records must be aligned up to an 8 octet boundary */
-#define REC_ALIGN_ORDER               (3U)
+#define REC_ALIGN                     (1U << 3)
 /* Somewhat arbitrary - 128MB */
 #define REC_LENGTH_MAX                (128U << 20)
 
tools/libs/light/libxl_arm_acpi.c +12 -12
index ba874c3d3224..ac8165de15b6 100644
@@ -107,12 +107,12 @@ int libxl__get_acpi_size(libxl__gc *gc,
     if (rc < 0)
         goto out;
 
-    *out = ROUNDUP(size, 3) +
-           ROUNDUP(sizeof(struct acpi_table_rsdp), 3) +
-           ROUNDUP(sizeof(struct acpi_table_xsdt), 3) +
-           ROUNDUP(sizeof(struct acpi_table_gtdt), 3) +
-           ROUNDUP(sizeof(struct acpi_table_fadt), 3) +
-           ROUNDUP(sizeof(dsdt_anycpu_arm_len), 3);
+    *out = ROUNDUP(size, 1U << 3) +
+           ROUNDUP(sizeof(struct acpi_table_rsdp), 1U << 3) +
+           ROUNDUP(sizeof(struct acpi_table_xsdt), 1U << 3) +
+           ROUNDUP(sizeof(struct acpi_table_gtdt), 1U << 3) +
+           ROUNDUP(sizeof(struct acpi_table_fadt), 1U << 3) +
+           ROUNDUP(sizeof(dsdt_anycpu_arm_len), 1U << 3);
 
 out:
     return rc;
@@ -128,7 +128,7 @@ static int libxl__allocate_acpi_tables(libxl__gc *gc,
 
     acpitables[RSDP].addr = GUEST_ACPI_BASE;
     acpitables[RSDP].size = sizeof(struct acpi_table_rsdp);
-    dom->acpi_modules[0].length += ROUNDUP(acpitables[RSDP].size, 3);
+    dom->acpi_modules[0].length += ROUNDUP(acpitables[RSDP].size, 1U << 3);
 
     acpitables[XSDT].addr = GUEST_ACPI_BASE + dom->acpi_modules[0].length;
     /*
@@ -137,11 +137,11 @@ static int libxl__allocate_acpi_tables(libxl__gc *gc,
      */
     acpitables[XSDT].size = sizeof(struct acpi_table_xsdt) +
                             sizeof(uint64_t) * 2;
-    dom->acpi_modules[0].length += ROUNDUP(acpitables[XSDT].size, 3);
+    dom->acpi_modules[0].length += ROUNDUP(acpitables[XSDT].size, 1U << 3);
 
     acpitables[GTDT].addr = GUEST_ACPI_BASE + dom->acpi_modules[0].length;
     acpitables[GTDT].size = sizeof(struct acpi_table_gtdt);
-    dom->acpi_modules[0].length += ROUNDUP(acpitables[GTDT].size, 3);
+    dom->acpi_modules[0].length += ROUNDUP(acpitables[GTDT].size, 1U << 3);
 
     acpitables[MADT].addr = GUEST_ACPI_BASE + dom->acpi_modules[0].length;
 
@@ -150,15 +150,15 @@ static int libxl__allocate_acpi_tables(libxl__gc *gc,
         goto out;
 
     acpitables[MADT].size = size;
-    dom->acpi_modules[0].length += ROUNDUP(acpitables[MADT].size, 3);
+    dom->acpi_modules[0].length += ROUNDUP(acpitables[MADT].size, 1U << 3);
 
     acpitables[FADT].addr = GUEST_ACPI_BASE + dom->acpi_modules[0].length;
     acpitables[FADT].size = sizeof(struct acpi_table_fadt);
-    dom->acpi_modules[0].length += ROUNDUP(acpitables[FADT].size, 3);
+    dom->acpi_modules[0].length += ROUNDUP(acpitables[FADT].size, 1U << 3);
 
     acpitables[DSDT].addr = GUEST_ACPI_BASE + dom->acpi_modules[0].length;
     acpitables[DSDT].size = dsdt_anycpu_arm_len;
-    dom->acpi_modules[0].length += ROUNDUP(acpitables[DSDT].size, 3);
+    dom->acpi_modules[0].length += ROUNDUP(acpitables[DSDT].size, 1U << 3);
 
     assert(dom->acpi_modules[0].length <= GUEST_ACPI_SIZE);
     dom->acpi_modules[0].data = libxl__zalloc(gc, dom->acpi_modules[0].length);
tools/libs/light/libxl_create.c +1 -1
index 6fd62d140389..a8b0c8c50070 100644
@@ -600,7 +600,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
                 .opts = 0, /* .opts will be set below */
                 .nr = b_info->altp2m_count,
             },
-            .vmtrace_size = ROUNDUP(b_info->vmtrace_buf_kb << 10, XC_PAGE_SHIFT),
+            .vmtrace_size = ROUNDUP(b_info->vmtrace_buf_kb << 10, XC_PAGE_SIZE),
             .cpupool_id = info->poolid,
         };
 
tools/libs/light/libxl_sr_stream_format.h +1 -1
index f8f4723c2e91..b02c954a388e 100644
@@ -29,7 +29,7 @@ typedef struct libxl__sr_rec_hdr
 } libxl__sr_rec_hdr;
 
 /* All records must be aligned up to an 8 octet boundary */
-#define REC_ALIGN_ORDER              3U
+#define REC_ALIGN                    (1U << 3)
 
 #define REC_TYPE_END                    0x00000000U
 #define REC_TYPE_LIBXC_CONTEXT          0x00000001U
tools/libs/light/libxl_stream_read.c +1 -1
index e64e8f0eadec..99c7607b6c54 100644
@@ -511,7 +511,7 @@ static void record_header_done(libxl__egc *egc,
         return;
     }
 
-    size_t bytes_to_read = ROUNDUP(rec->hdr.length, REC_ALIGN_ORDER);
+    size_t bytes_to_read = ROUNDUP(rec->hdr.length, REC_ALIGN);
     rec->body = libxl__malloc(NOGC, bytes_to_read);
 
     rc = setup_read(stream, "record body",
tools/libs/light/libxl_stream_write.c +2 -2
index 98d44597a732..9ea64369352f 100644
@@ -119,7 +119,7 @@ static void setup_generic_write(libxl__egc *egc,
                                 void *body,
                                 sws_record_done_cb cb)
 {
-    static const uint8_t zero_padding[1U << REC_ALIGN_ORDER] = { 0 };
+    static const uint8_t zero_padding[REC_ALIGN] = { 0 };
 
     libxl__datacopier_state *dc = &stream->dc;
     int rc;
@@ -136,7 +136,7 @@ static void setup_generic_write(libxl__egc *egc,
         return;
     }
 
-    size_t padsz = ROUNDUP(hdr->length, REC_ALIGN_ORDER) - hdr->length;
+    size_t padsz = ROUNDUP(hdr->length, REC_ALIGN) - hdr->length;
     uint32_t length = hdr->length;
 
     /* Insert header */
tools/misc/xen-mfndump.c +1 -1
index 28687afbf01a..99a0b1d3b5c0 100644
@@ -10,7 +10,7 @@
 
 #include <xen-tools/common-macros.h>
 
-#define M2P_SIZE(_m)    ROUNDUP(((_m) * sizeof(xen_pfn_t)), 21)
+#define M2P_SIZE(_m)    ROUNDUP(((_m) * sizeof(xen_pfn_t)), MB(2))
 #define is_mapped(pfn_type) (!((pfn_type) & 0x80000000UL))
 
 #define ERROR(msg, args...) fprintf(stderr, msg, ## args)
tools/ocaml/libs/xc/xenctrl_stubs.c +1 -1
index c55f73b265b2..7f6381cdd2fe 100644
@@ -221,7 +221,7 @@ CAMLprim value stub_xc_domain_create(value xch_val, value wanted_domid, value co
 	if ( altp2m_nr != (uint16_t)altp2m_nr )
 		caml_invalid_argument("altp2m_count");
 
-	vmtrace_size = ROUNDUP(vmtrace_size << 10, XC_PAGE_SHIFT);
+	vmtrace_size = ROUNDUP(vmtrace_size << 10, XC_PAGE_SIZE);
 	if ( vmtrace_size != (uint32_t)vmtrace_size )
 		caml_invalid_argument("vmtrace_buf_kb");
 
tools/xenstored/core.c +2 -2
index d6d462b7bc82..86c86b3e9a77 100644
@@ -466,7 +466,7 @@ int set_fd(int fd, short events)
 		/* Round up to 2^8 boundary, in practice this just
 		 * make newsize larger than current_array_size.
 		 */
-		newsize = ROUNDUP(nr_fds + 1, 8);
+		newsize = ROUNDUP(nr_fds + 1, 1U << 8);
 
 		new_fds = realloc(poll_fds, sizeof(struct pollfd)*newsize);
 		if (!new_fds)
@@ -3067,7 +3067,7 @@ static int dump_state_node(const void *ctx, struct connection *conn,
 	head.length += node->hdr.num_perms * sizeof(*sn.perms);
 	head.length += pathlen;
 	head.length += node->hdr.datalen;
-	head.length = ROUNDUP(head.length, 3);
+	head.length = ROUNDUP(head.length, 1U << 3);
 
 	if (fwrite(&head, sizeof(head), 1, fp) != 1)
 		return dump_state_node_err(data, "Dump node head error");
tools/xenstored/domain.c +5 -4
index 2db452144dd4..a880ff678ef9 100644
@@ -2159,7 +2159,7 @@ const char *dump_state_connections(FILE *fp)
 		if (ret)
 			return ret;
 		head.length += sc.data_in_len + sc.data_out_len;
-		head.length = ROUNDUP(head.length, 3);
+		head.length = ROUNDUP(head.length, 1U << 3);
 		if (c->domain) {
 			sc.fields |= XS_STATE_CONN_FIELDS_UNIQ_ID;
 			head.length += sizeof(uint64_t);
@@ -2232,7 +2232,8 @@ void read_state_connection(const void *ctx, const void *state)
 		unsigned long off;
 
 		off = sizeof(*sc) + sc->data_in_len + sc->data_out_len;
-		domain->unique_id = *(uint64_t *)(state + ROUNDUP(off, 3));
+		domain->unique_id =
+		    *(uint64_t *)(state + ROUNDUP(off, 1U << 3));
 	}
 }
 
@@ -2308,7 +2309,7 @@ static int dump_state_domain(const void *k, void *v, void *arg)
 	n_quota = get_quota_size(domain->acc, &rec_len);
 	rec_len += n_quota * sizeof(sd->quota_val[0]);
 	rec_len += sizeof(*sd);
-	rec_len = ROUNDUP(rec_len, 3);
+	rec_len = ROUNDUP(rec_len, 1U << 3);
 
 	record = talloc_size(NULL, rec_len + sizeof(*head));
 	if (!record)
@@ -2372,7 +2373,7 @@ const char *dump_state_glb_quota(FILE *fp)
 	n_quota = get_quota_size(quotas, &rec_len);
 	rec_len += n_quota * sizeof(glb->quota_val[0]);
 	rec_len += sizeof(*glb);
-	rec_len = ROUNDUP(rec_len, 3);
+	rec_len = ROUNDUP(rec_len, 1U << 3);
 
 	record = talloc_size(NULL, rec_len + sizeof(*head));
 	if (!record)
tools/xenstored/watch.c +1 -1
index a9a06e9e4816..309c5bb66bef 100644
@@ -349,7 +349,7 @@ const char *dump_state_watches(FILE *fp, struct connection *conn,
 		}
 
 		head.length += path_len + token_len;
-		head.length = ROUNDUP(head.length, 3);
+		head.length = ROUNDUP(head.length, 1U << 3);
 		if (fwrite(&head, sizeof(head), 1, fp) != 1)
 			return "Dump watch state error";
 
Signature · 1 line
2.53.0
#6 lore raw

PATCH 3/5 xen/numa: prepare NUMA setup code for unit testing

In reply to Roger Pau Monne

Introduce __XEN__ guards to differentiate between hypervisor vs unit test builds. Also move numa_set_node() so it's outside the __XEN__ guards. No functional change intended.

---
 xen/common/numa.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
xen/common/numa.c +9 -5
index ad75955a1622..8544a1598218 100644
@@ -4,6 +4,7 @@
  * Adapted for Xen: Ryan Harper <[email protected]>
  */
 
+#ifdef __XEN__
 #include <xen/init.h>
 #include <xen/keyhandler.h>
 #include <xen/mm.h>
@@ -13,6 +14,7 @@
 #include <xen/pfn.h>
 #include <xen/sched.h>
 #include <xen/softirq.h>
+#endif /* __XEN__ */
 
 static nodemask_t __initdata processor_nodes_parsed;
 static nodemask_t __initdata memory_nodes_parsed;
@@ -561,6 +563,12 @@ void __init numa_init_array(void)
     }
 }
 
+void numa_set_node(unsigned int cpu, nodeid_t node)
+{
+    cpu_to_node[cpu] = node;
+}
+
+#ifdef __XEN__
 #ifdef CONFIG_NUMA_EMU
 static unsigned int __initdata numa_fake;
 
@@ -661,11 +669,6 @@ void numa_add_cpu(unsigned int cpu)
     cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]);
 }
 
-void numa_set_node(unsigned int cpu, nodeid_t node)
-{
-    cpu_to_node[cpu] = node;
-}
-
 /* [numa=off] */
 static int __init cf_check numa_setup(const char *opt)
 {
@@ -830,3 +833,4 @@ static int __init cf_check register_numa_trigger(void)
     return 0;
 }
 __initcall(register_numa_trigger);
+#endif /* __XEN__ */
Signature · 1 line
2.53.0
#8 lore raw

PATCH 4/5 tests/numa: add unit tests for NUMA setup logic

In reply to Roger Pau Monne

NUMA setup, like PDX, requires certain amount of logic to configure the internal structures and parameters for NUMA operation. Introduce some very basic testing that allows building and testing NUMA setup logic in as a user-space unit test. This allows feeding synthetic memory affinity and map to the logic, allowing to reproduce bugs that would otherwise need access to real systems with such a configuration.

For the time being introduce a single test case, based on a known working NUMA setup for an AMD Turin system. Also the testing after setup is currently limited to ensuring the start and end RAM region addresses fall into a correctly setup memory block.

---
 tools/tests/Makefile         |   1 +
 tools/tests/numa/.gitignore  |   2 +
 tools/tests/numa/Makefile    |  47 ++++++++
 tools/tests/numa/harness.h   | 184 +++++++++++++++++++++++++++++
 tools/tests/numa/test-numa.c | 222 +++++++++++++++++++++++++++++++++++
 5 files changed, 456 insertions(+)
 create mode 100644 tools/tests/numa/.gitignore
 create mode 100644 tools/tests/numa/Makefile
 create mode 100644 tools/tests/numa/harness.h
 create mode 100644 tools/tests/numa/test-numa.c
tools/tests/Makefile +1 -0
index 6477a4386dda..fc0ed8091510 100644
@@ -4,6 +4,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 SUBDIRS-y :=
 SUBDIRS-y += domid
 SUBDIRS-y += mem-claim
+SUBDIRS-y += numa
 SUBDIRS-y += paging-mempool
 SUBDIRS-y += pdx
 SUBDIRS-y += rangeset
tools/tests/numa/.gitignore new file +2 -0
new file mode 100644
index 000000000000..0710a767f400
@@ -0,0 +1,2 @@
+/numa.h
+/test-numa
tools/tests/numa/Makefile new file +47 -0

47 lines not on this page · raw message

tools/tests/numa/harness.h new file +184 -0

184 lines not on this page · raw message

tools/tests/numa/test-numa.c new file +222 -0

222 lines not on this page · raw message

Signature · 1 line
2.53.0
#9 lore raw

Re: [PATCH for-4.22 4/5] tests/numa: add unit tests for NUMA setup logic

tools/tests/numa/.gitignore line 2

Quoted 5 lines · Roger Pau Monne
tools/tests/numa/.gitignore new file +2 -0
@@ -0,0 +1,2 @@
+/numa.h
+/test-numa

Why the leading slashes?

Quoted 19 lines
tools/tests/numa/Makefile new file +16 -0
@@ -0,0 +1,47 @@
+XEN_ROOT=$(CURDIR)/../../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+TARGETS := test-numa
+
+.PHONY: all
+all: $(TARGETS)
+
+.PHONY: run
+run: $(TARGETS)
+ifeq ($(CC),$(HOSTCC))
+	set -e;             \
+	for test in $? ; do \
+		./$$test ;  \
+	done
+else

tools/tests/numa/Makefile line 26

Quoted 10 lines
tools/tests/numa/Makefile new file +10 -0
+	$(warning HOSTCC != CC, will not run test)
+endif
+
+.PHONY: clean
+clean:
+	$(RM) -- *.o $(TARGETS) $(DEPS_RM) numa.h
+
+.PHONY: distclean
+distclean: clean
+	$(RM) -- *~

I see we remove *~ elsewhere, but not everywhere. I don't, however, know why we have that, and hence I wonder whether it really wants replicating.

Quoted 74 lines
tools/tests/numa/harness.h new file +71 -0
@@ -0,0 +1,184 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Unit tests for NUMA setup.
+ *
+ * Copyright (C) 2026 Cloud Software Group
+ */
+
+#ifndef _TEST_HARNESS_
+#define _TEST_HARNESS_
+
+#include <assert.h>
+#include <errno.h>
+#include <inttypes.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <xen-tools/bitops.h>
+#include <xen-tools/common-macros.h>
+
+#define CONFIG_DEBUG
+#define CONFIG_NUMA
+#define CONFIG_NR_NUMA_NODES 64
+#define NR_CPUS 256
+#define MAX_RANGES 128
+#define PADDR_BITS 52
+
+#define __init
+#define __initdata
+#define __ro_after_init
+#define __read_mostly
+
+#define printk printf
+#define XENLOG_INFO ""
+#define XENLOG_DEBUG ""
+#define XENLOG_WARNING ""
+#define KERN_INFO ""
+#define KERN_ERR ""
+#define KERN_WARNING ""
+#define KERN_DEBUG ""
+
+#define PAGE_SHIFT    12
+/* Some libcs define PAGE_SIZE in limits.h. */
+#undef  PAGE_SIZE
+#define PAGE_SIZE     (1L << PAGE_SHIFT)
+#define MAX_ORDER     18 /* 2 * PAGETABLE_ORDER (9) */
+
+#define PFN_DOWN(x)   ((x) >> PAGE_SHIFT)
+#define PFN_UP(x)     (((x) + PAGE_SIZE-1) >> PAGE_SHIFT)
+
+#define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
+#define mfn_to_pdx(mfn)   (mfn)
+#define paddr_to_pdx(pa)  ((pa) >> PAGE_SHIFT)
+#define mfn_to_maddr(mfn) ((mfn) << PAGE_SHIFT)
+
+#define ASSERT assert
+#define ASSERT_UNREACHABLE() assert(0)
+
+/* For the purposes of the testing assume arch NID == Xen NID. */
+#define numa_node_to_arch_nid(n) (n)
+
+typedef uint64_t paddr_t;
+#define PRIpaddr "016" PRIx64
+
+typedef unsigned long mfn_t;
+typedef uint8_t nodeid_t;
+
+#define __set_bit set_bit
+#define __clear_bit clear_bit

tools/tests/numa/harness.h line 81

Quoted 10 lines
tools/tests/numa/harness.h new file +10 -0
+
+static inline unsigned int find_next_bit(
+    const unsigned long *addr, unsigned int size, unsigned int off)
+{
+    unsigned int i;
+
+    ASSERT(size <= BITS_PER_LONG);
+
+    for ( i = off; i < size; i++ )
+        if ( !!(*addr & (1UL << i)) )

Why the !! ?

Quoted 6 lines
tools/tests/numa/harness.h new file +6 -0
+            return i;
+
+    return size;
+}
+
+#define find_first_bit(b, s) find_next_bit(b, s, 0)

tools/tests/numa/harness.h line 97

Quoted 10 lines
tools/tests/numa/harness.h new file +10 -0
+
+/* Minimal cpumask support. */
+typedef struct cpumask{ DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t;
+
+#define cpumask_clear_cpu(c, m) clear_bit((c), (m)->bits)
+
+/* Define the nodemask helpers used. */
+typedef struct nodemask{ DECLARE_BITMAP(bits, CONFIG_NR_NUMA_NODES); } nodemask_t;
+
+#define node_set(node, dst) set_bit((node), (dst).bits)

To aid readability, omit the parentheses around "node"? (More similar cases further down.)

Jan

#10 lore raw

Re: [PATCH for-4.22 4/5] tests/numa: add unit tests for NUMA setup logic

Quoted 8 lines · Jan Beulich

tools/tests/numa/.gitignore line 2

Quoted 5 lines · Roger Pau Monne
tools/tests/numa/.gitignore new file +2 -0
@@ -0,0 +1,2 @@
+/numa.h
+/test-numa

Why the leading slashes?

This is the format of the .gitignore that we use in the pdx, numa and rengeset testing. The slashes denote that the pattern is relative to the particular .gitignore itself, but won't match any level below the .gitignore.

Quoted 24 lines
Quoted 24 lines
tools/tests/numa/Makefile new file +21 -0
@@ -0,0 +1,47 @@
+XEN_ROOT=$(CURDIR)/../../..
+include $(XEN_ROOT)/tools/Rules.mk
+
+TARGETS := test-numa
+
+.PHONY: all
+all: $(TARGETS)
+
+.PHONY: run
+run: $(TARGETS)
+ifeq ($(CC),$(HOSTCC))
+	set -e;             \
+	for test in $? ; do \
+		./$$test ;  \
+	done
+else
+	$(warning HOSTCC != CC, will not run test)
+endif
+
+.PHONY: clean
+clean:
Quoted 8 lines

tools/tests/numa/Makefile line 26

Quoted 5 lines
tools/tests/numa/Makefile new file +5 -0
+	$(RM) -- *.o $(TARGETS) $(DEPS_RM) numa.h
+
+.PHONY: distclean
+distclean: clean
+	$(RM) -- *~

I see we remove *~ elsewhere, but not everywhere. I don't, however, know why we have that, and hence I wonder whether it really wants replicating.

Seems like *~ is a backup file created by some editors (Emacs or Vim for example. Again this is a verbatim copy of the Makefile that we use for other unit testing (I think I copied this from the PDX testing).

Quoted 78 lines
Quoted 78 lines
tools/tests/numa/harness.h new file +75 -0
@@ -0,0 +1,184 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Unit tests for NUMA setup.
+ *
+ * Copyright (C) 2026 Cloud Software Group
+ */
+
+#ifndef _TEST_HARNESS_
+#define _TEST_HARNESS_
+
+#include <assert.h>
+#include <errno.h>
+#include <inttypes.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <xen-tools/bitops.h>
+#include <xen-tools/common-macros.h>
+
+#define CONFIG_DEBUG
+#define CONFIG_NUMA
+#define CONFIG_NR_NUMA_NODES 64
+#define NR_CPUS 256
+#define MAX_RANGES 128
+#define PADDR_BITS 52
+
+#define __init
+#define __initdata
+#define __ro_after_init
+#define __read_mostly
+
+#define printk printf
+#define XENLOG_INFO ""
+#define XENLOG_DEBUG ""
+#define XENLOG_WARNING ""
+#define KERN_INFO ""
+#define KERN_ERR ""
+#define KERN_WARNING ""
+#define KERN_DEBUG ""
+
+#define PAGE_SHIFT    12
+/* Some libcs define PAGE_SIZE in limits.h. */
+#undef  PAGE_SIZE
+#define PAGE_SIZE     (1L << PAGE_SHIFT)
+#define MAX_ORDER     18 /* 2 * PAGETABLE_ORDER (9) */
+
+#define PFN_DOWN(x)   ((x) >> PAGE_SHIFT)
+#define PFN_UP(x)     (((x) + PAGE_SIZE-1) >> PAGE_SHIFT)
+
+#define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
+#define mfn_to_pdx(mfn)   (mfn)
+#define paddr_to_pdx(pa)  ((pa) >> PAGE_SHIFT)
+#define mfn_to_maddr(mfn) ((mfn) << PAGE_SHIFT)
+
+#define ASSERT assert
+#define ASSERT_UNREACHABLE() assert(0)
+
+/* For the purposes of the testing assume arch NID == Xen NID. */
+#define numa_node_to_arch_nid(n) (n)
+
+typedef uint64_t paddr_t;
+#define PRIpaddr "016" PRIx64
+
+typedef unsigned long mfn_t;
+typedef uint8_t nodeid_t;
+
+#define __set_bit set_bit
+#define __clear_bit clear_bit
+
+static inline unsigned int find_next_bit(
+    const unsigned long *addr, unsigned int size, unsigned int off)
+{
Quoted 8 lines

tools/tests/numa/harness.h line 81

Quoted 6 lines
tools/tests/numa/harness.h new file +6 -0
+    unsigned int i;
+
+    ASSERT(size <= BITS_PER_LONG);
+
+    for ( i = off; i < size; i++ )
+        if ( !!(*addr & (1UL << i)) )

Why the !! ?

I copied this from the PDX header and simplified the function because now it only cares about set values, and forgot to drop it. I can indeed drop the !!.

Quoted 12 lines
Quoted 12 lines
tools/tests/numa/harness.h new file +11 -0
+            return i;
+
+    return size;
+}
+
+#define find_first_bit(b, s) find_next_bit(b, s, 0)
+
+/* Minimal cpumask support. */
+typedef struct cpumask{ DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t;
+
+#define cpumask_clear_cpu(c, m) clear_bit((c), (m)->bits)
Quoted 8 lines

tools/tests/numa/harness.h line 97

Quoted 5 lines
tools/tests/numa/harness.h new file +5 -0
+
+/* Define the nodemask helpers used. */
+typedef struct nodemask{ DECLARE_BITMAP(bits, CONFIG_NR_NUMA_NODES); } nodemask_t;
+
+#define node_set(node, dst) set_bit((node), (dst).bits)

To aid readability, omit the parentheses around "node"? (More similar cases further down.)

Sure, this is all copied from the nodemask.h header. I should have adjusted.

Thanks, Roger.

#11 lore raw

Re: [PATCH for-4.22 4/5] tests/numa: add unit tests for NUMA setup logic

Quoted 10 lines · Roger Pau Monné
Quoted 8 lines · Jan Beulich

tools/tests/numa/.gitignore line 2

Quoted 5 lines · Roger Pau Monne
tools/tests/numa/.gitignore new file +2 -0
@@ -0,0 +1,2 @@
+/numa.h
+/test-numa

Why the leading slashes?

Quoted 4 lines

This is the format of the .gitignore that we use in the pdx, numa and rengeset testing. The slashes denote that the pattern is relative to the particular .gitignore itself, but won't match any level below the .gitignore.

Hmm, before replying I checked a few other .gitignore files under tests/, and none had leading slashes. For the purposes here either way is likely fine, but really we'd better be consistent overall. (Which isn't a request for you to change or do anything, merely a remark.)

Jan

#12 lore raw

Re: [PATCH for-4.22 4/5] tests/numa: add unit tests for NUMA setup logic

Quoted 16 lines · Jan Beulich
Quoted 10 lines · Roger Pau Monné
Quoted 8 lines · Jan Beulich

tools/tests/numa/.gitignore line 2

Quoted 5 lines · Roger Pau Monne
tools/tests/numa/.gitignore new file +2 -0
@@ -0,0 +1,2 @@
+/numa.h
+/test-numa

Why the leading slashes?

Quoted 4 lines

This is the format of the .gitignore that we use in the pdx, numa and rengeset testing. The slashes denote that the pattern is relative to the particular .gitignore itself, but won't match any level below the .gitignore.

Quoted 4 lines

Hmm, before replying I checked a few other .gitignore files under tests/, and none had leading slashes. For the purposes here either way is likely fine, but really we'd better be consistent overall. (Which isn't a request for you to change or do anything, merely a remark.)

% cat tools/tests/*/.gitignore
test-cpu-policy
generated
test-domid
test-mem-claim
/numa.h
/test-numa
test-paging-mempool
/pdx.h
/test-pdx-mask
/test-pdx-offset
/list.h
/rangeset.c
/rangeset.h
/test-rangeset
test-resource
test-tsx
test-xenstore

Some have, some doesn't. Using the slash is IMO more accurate matching, but I'm not going to adjust the others either unless I have to modify them for other reasons possibly.

Thanks, Roger.

#13 lore raw

PATCH 5/5 xen/numa: fix setup of non-aligned memory affinity ranges

In reply to Roger Pau Monne

The logic to populate memnodemap in populate_memnodemap() assumes that all ranges are aligned to the hash shift, this however is only true for the first address in a memory affinity node. Any subsequent ranges belonging to the same node might not be aligned to the hash shift value.

Such lack of alignment causes issues to the logic in populate_memnodemap(), as then the tail of the range might not be properly accounted for and setup in memnodemap. Fix this by forcing the start address of all regions to be aligned to the hash shift; if such alignment causes a region overlap it would always be between regions on the same node, and hence will never cause setup issues of the memnodemap array.

Introduce two additional test cases to the user-space NUMA setup unit testing, first test case is the native memory affinity and memory map of the system where this issue was found, second test case is a simplification to demonstrate the original problem more clearly.

Fixes 1666086b0044 ("x86/NUMA: improve memnode_shift calculation for multi node system")
---
 tools/include/xen-tools/common-macros.h |  1 +
 tools/tests/numa/test-numa.c            | 45 +++++++++++++++++++++++++
 xen/common/numa.c                       |  8 ++++-
 3 files changed, 53 insertions(+), 1 deletion(-)
tools/include/xen-tools/common-macros.h +1 -0
index 9e2799178235..88b4a0e5a693 100644
@@ -69,6 +69,7 @@
 #endif
 
 #define ROUNDUP(x, a) (((x) + (a) - 1) & ~((a) - 1))
+#define ROUNDDOWN(x, a) ((x) & ~((a) - 1))
 
 #define MASK_EXTR(v, m) (((v) & (m)) / ((m) & -(m)))
 #define MASK_INSR(v, m) (((v) * ((m) & -(m))) & (m))
tools/tests/numa/test-numa.c +45 -0

51 lines not on this page · raw message

xen/common/numa.c +7 -1
index 8544a1598218..8b2f29597f30 100644
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
         if ( spdx > epdx )
Signature · 1 line
2.53.0
#14 lore raw

Re: [PATCH for-4.22 5/5] xen/numa: fix setup of non-aligned memory affinity ranges

Quoted 9 lines · Roger Pau Monne
xen/common/numa.c +0 -1
index 8544a1598218..8b2f29597f30 100644
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 408

Quoted 10 lines
xen/common/numa.c +7 -0
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
         if ( spdx > epdx )

I like that this comes with a unit test, but this hunk needs to be standalone at the start of the series, because it needs backporting.

~Andrew

#15 lore raw

Re: [PATCH for-4.22 5/5] xen/numa: fix setup of non-aligned memory affinity ranges

Quoted 19 lines · Andrew Cooper
Quoted 18 lines · Roger Pau Monne
xen/common/numa.c +7 -1
index 8544a1598218..8b2f29597f30 100644
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
Quoted 4 lines

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 408

Quoted 1 line
xen/common/numa.c +0 -0
         if ( spdx > epdx )

I like that this comes with a unit test, but this hunk needs to be standalone at the start of the series, because it needs backporting.

I was supposed to add a post-commit note about this, but finding the offending commit took me a lot more time than I was expecting and forgot to add the note.

Yes, this needs to be backported. I was wondering whether it would be fine to just backport the fix, leaving the test changes alone. But I can also split this to a pre-commit.

Thanks, Roger.

#16 lore raw

Re: [PATCH for-4.22 5/5] xen/numa: fix setup of non-aligned memory affinity ranges

Quoted 23 lines · Roger Pau Monné
Quoted 17 lines · Andrew Cooper
Quoted 16 lines · Roger Pau Monne
xen/common/numa.c +7 -1
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
Quoted 4 lines

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 408

Quoted 1 line
xen/common/numa.c +0 -0
         if ( spdx > epdx )

I like that this comes with a unit test, but this hunk needs to be standalone at the start of the series, because it needs backporting.

Quoted 6 lines

I was supposed to add a post-commit note about this, but finding the offending commit took me a lot more time than I was expecting and forgot to add the note.

Yes, this needs to be backported. I was wondering whether it would be fine to just backport the fix, leaving the test changes alone.

I'd be perfectly fine doing so.

Jan

Quoted 3 lines

But I can also split this to a pre-commit.

Thanks, Roger.

#17 lore raw

Re: [PATCH for-4.22 5/5] xen/numa: fix setup of non-aligned memory affinity ranges

In reply to Roger Pau Monne
Quoted 4 lines · Roger Pau Monne
xen/common/numa.c +0 -0
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 405

Quoted 10 lines
xen/common/numa.c +7 -1
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);

Imo this rounding would better be done ...

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 408

Quoted 3 lines
xen/common/numa.c +0 -0
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
         if ( spdx > epdx )

... only after this check (and then perhaps also after the subsequent if()).

Jan

Reviewed-by
#18 lore raw

Re: [PATCH for-4.22 5/5] xen/numa: fix setup of non-aligned memory affinity ranges

Quoted 18 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne
xen/common/numa.c +0 -0
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 405

Quoted 10 lines
xen/common/numa.c +7 -1
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);

Imo this rounding would better be done ...

Quoted 5 lines

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 408

Quoted 3 lines
xen/common/numa.c +0 -0
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
         if ( spdx > epdx )

... only after this check (and then perhaps also after the subsequent if()).

Oh, and: With that adjustment

Jan

#19 lore raw

Re: [PATCH for-4.22 5/5] xen/numa: fix setup of non-aligned memory affinity ranges

Quoted 19 lines · Jan Beulich
Quoted 18 lines · Jan Beulich
Quoted 4 lines · Roger Pau Monne
xen/common/numa.c +0 -0
@@ -396,7 +396,13 @@ static int __init populate_memnodemap(const struct node *nodes,
 

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 405

Quoted 10 lines
xen/common/numa.c +7 -1
     for ( i = 0; i < numnodes; i++ )
     {
-        unsigned long spdx = paddr_to_pdx(nodes[i].start);
+        /*
+         * Round down start address: if start is not aligned to the memnodemap
+         * chunk size the tail remainder might not be added.  Overlaps created
+         * by rounding will fall into the same NUMA region.
+         */
+        unsigned long spdx = ROUNDDOWN(paddr_to_pdx(nodes[i].start),
+                                       1UL << shift);

Imo this rounding would better be done ...

Quoted 8 lines
Quoted 5 lines

xen/common/numa.c static int __init populate_memnodemap(const struct node *nodes, · line 408

Quoted 3 lines
xen/common/numa.c +0 -0
         unsigned long epdx = paddr_to_pdx(nodes[i].end - 1);
 
         if ( spdx > epdx )

... only after this check (and then perhaps also after the subsequent if()).

Oh, and: With that adjustment

Fair enough, I will adjust the rounding placement.

Thanks, Roger.

#20 lore raw
In reply to Roger Pau Monne
Quoted 46 lines · Roger Pau Monne

Hello,

The first 4 patches add some basic unit testing for the NUMA setup logic. The last patch expands the test cases and fixes an issue the new test-cases would otherwise trigger.

Thanks, Roger.

Roger Pau Monne (5):
tools/bitops: adjust bitmap_or() interface to match hypervisor tools/macros: adjust ROUNDUP() interface to match hypervisor xen/numa: prepare NUMA setup code for unit testing tests/numa: add unit tests for NUMA setup logic xen/numa: fix setup of non-aligned memory affinity ranges

  tools/console/daemon/io.c                 |   2 +-
  tools/include/xen-tools/bitops.h          |   7 +-
  tools/include/xen-tools/common-macros.h   |   5 +-
  tools/libs/call/buffer.c                  |   3 +-
  tools/libs/foreignmemory/linux.c          |   2 +-
  tools/libs/gnttab/freebsd.c               |   2 +-
  tools/libs/gnttab/linux.c                 |   2 +-
  tools/libs/guest/xg_core.c                |   2 +-
  tools/libs/guest/xg_dom_arm.c             |   6 +-
  tools/libs/guest/xg_dom_x86.c             |   2 +-
  tools/libs/guest/xg_private.h             |   4 +-
  tools/libs/guest/xg_sr_common.c           |   6 +-
  tools/libs/guest/xg_sr_save.c             |   3 +-
  tools/libs/guest/xg_sr_stream_format.h    |   2 +-
  tools/libs/light/libxl_arm_acpi.c         |  24 +-
  tools/libs/light/libxl_create.c           |   2 +-
  tools/libs/light/libxl_sr_stream_format.h |   2 +-
  tools/libs/light/libxl_stream_read.c      |   2 +-
  tools/libs/light/libxl_stream_write.c     |   4 +-
  tools/misc/xen-mfndump.c                  |   2 +-
  tools/ocaml/libs/xc/xenctrl_stubs.c       |   2 +-
  tools/tests/Makefile                      |   1 +
  tools/tests/numa/.gitignore               |   2 +
  tools/tests/numa/Makefile                 |  47 ++++
  tools/tests/numa/harness.h                | 184 +++++++++++++++
  tools/tests/numa/test-numa.c              | 267 ++++++++++++++++++++++
  tools/xenstored/core.c                    |   4 +-
  tools/xenstored/domain.c                  |   9 +-
  tools/xenstored/watch.c                   |   2 +-
  xen/common/numa.c                         |  22 +-
  30 files changed, 569 insertions(+), 55 deletions(-)
Quoted 5 lines

create mode 100644 tools/tests/numa/.gitignore
create mode 100644 tools/tests/numa/Makefile
create mode 100644 tools/tests/numa/harness.h
create mode 100644 tools/tests/numa/test-numa.c

The changes do not appear to be too intrusive, so I am okay with including all of the patches in 4.22, provided they receive proper review.

Thanks.

~ Oleksii