Skip to contentSkip to conversation

Move alloc_domain_struct() to common code

PATCH In review Oleksii Kurochko 3 participants 6 messages Jun 13, 2025 – Jun 16, 2025 RISC-V

lore
Share & tools
lore
#0 lore raw

cover Move alloc_domain_struct() to common code

x86 has specific requirements about the allocation of the domain structure, but that's not the case for ARM or likely other architectures.

Introduce a generic weak function that can be overwritten with an architecture specific implementation if required.

RISC-V has a compilation issue [1] after an introduction of [2] becuase of different way inclusion of asm/page.h and asm/mm.h (for more details please check the patch [3]).
So rework asm/mm.h and asm/page.h includes to match other architectures.

[1] https://gitlab.com/xen-project/people/royger/xen/-/jobs/10328429719#L553
[2] xen/domain: introduce generic weak function for domain struct allocation
[3] xen/riscv: rework asm/mm.h and asm/page.h includes to match other architectures

CI tests:
https://gitlab.com/xen-project/people/olkur/xen/-/pipelines/1868877289

Oleksii Kurochko (1):
xen/riscv: rework asm/mm.h and asm/page.h includes to match other
architectures

Roger Pau Monne (1):
xen/domain: introduce generic weak function for domain struct
allocation

 xen/arch/arm/domain.c             | 12 -----------
 xen/arch/ppc/stubs.c              |  5 -----
 xen/arch/riscv/include/asm/mm.h   | 24 ++++++++++++++++++---
 xen/arch/riscv/include/asm/page.h | 35 +------------------------------
 xen/arch/riscv/mm.c               | 14 +++++++++++++
 xen/arch/riscv/stubs.c            |  5 -----
 xen/common/domain.c               | 14 +++++++++++++
 7 files changed, 50 insertions(+), 59 deletions(-)
Signature · 1 line
2.49.0
#1 lore raw

PATCH 1/2 xen/riscv: rework asm/mm.h and asm/page.h includes to match other architectures

To align with other architectures where <asm/page.h> is included from <asm/mm.h>
(and not the other way around), the following changes are made:
- Since <asm/mm.h> is no longer included in <asm/page.h>:
- Move the definitions of paddr_to_pte() and pte_to_paddr() to <asm/mm.h>,
as paddr_to_pfn() and pte_to_paddr() are already defined there.
- Move _vmap_to_mfn() to <asm/mm.h> because mfn_from_pte() is defined there and
open-code it inside macros vmap_to_mfn().
- Drop the inclusion of <xen/domain_page.h> from <asm/page.h> to resolve a compilation error:
./include/xen/domain_page.h:63:12: error: implicit declaration of function '__mfn_to_virt'; did you mean 'mfn_to_nid'? [-Werror=implicit-function-declaration]
63 | return __mfn_to_virt(mfn_x(mfn));
This happens because __mfn_to_virt() is defined in <asm/mm.h>, but due to
the current include chain:

    <xen/domain.h>
      <asm/domain.h>
        <xen/mm.h>
          <asm/mm.h>
            <asm/page.h>
              <xen/domain_page.h>
                static inline void *map_domain_page_global(mfn_t mfn)
		{
		    return __mfn_to_virt(mfn_x(mfn));
                }
            ...
          ...
          #define __mfn_to_virt() ...

This leads to a circular dependency and the build error above.

As a result, since <xen/domain_page.h> is no longer included in <asm/page.h>, the flush_page_to_ram() definition cannot remain there. It is now moved to riscv/mm.c.

Including <asm/page.h> from <asm/mm.h> does not cause issues with the
declaration/definition of clear_page() when <xen/mm.h> is included, and
also prevents build errors such as:
common/domain.c: In function 'alloc_domain_struct':
common/domain.c:797:5: error: implicit declaration of function 'clear_page';did you mean 'steal_page'? [-Werror=implicit-function-declaration]

  797 |     clear_page(d);
      |     ^~~~~~~~~~
      |     steal_page

caused by using clear_page() in common/domain.c.

---
 xen/arch/riscv/include/asm/mm.h   | 24 ++++++++++++++++++---
 xen/arch/riscv/include/asm/page.h | 35 +------------------------------
 xen/arch/riscv/mm.c               | 14 +++++++++++++
 3 files changed, 36 insertions(+), 37 deletions(-)
xen/arch/riscv/include/asm/mm.h +21 -3
index 01bbd92a06..9283616c02 100644
@@ -12,6 +12,7 @@
 #include <xen/sections.h>
 #include <xen/types.h>
 
+#include <asm/page.h>
 #include <asm/page-bits.h>
 
 extern vaddr_t directmap_virt_start;
@@ -19,11 +20,21 @@ extern vaddr_t directmap_virt_start;
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+static inline pte_t paddr_to_pte(paddr_t paddr,
+                                 unsigned int permissions)
+{
+    return (pte_t) { .pte = (paddr_to_pfn(paddr) << PTE_PPN_SHIFT) | permissions };
+}
+
+static inline paddr_t pte_to_paddr(pte_t pte)
+{
+    return pfn_to_paddr(pte.pte >> PTE_PPN_SHIFT);
+}
+
 #define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
 #define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
 #define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
 #define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
-#define vmap_to_mfn(va)     _vmap_to_mfn((vaddr_t)(va))
 #define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
 
 static inline void *maddr_to_virt(paddr_t ma)
@@ -35,6 +46,15 @@ static inline void *maddr_to_virt(paddr_t ma)
     return (void *)va;
 }
 
+#define mfn_from_pte(pte) maddr_to_mfn(pte_to_paddr(pte))
+
+#define vmap_to_mfn(va)                             \
+({                                                  \
+    pte_t __entry = pt_walk((vaddr_t)(va), NULL);   \
+    BUG_ON(!pte_is_mapping(__entry));               \
+    maddr_to_mfn(pte_to_paddr(__entry));            \
+})
+
 /*
  * virt_to_maddr() is expected to work with virtual addresses from either
  * the directmap region or Xen's linkage (XEN_VIRT_START) region.
@@ -76,8 +96,6 @@ static inline unsigned long virt_to_maddr(unsigned long va)
 #define virt_to_mfn(va)     __virt_to_mfn(va)
 #define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
 
-#define mfn_from_pte(pte) maddr_to_mfn(pte_to_paddr(pte))
-
 struct page_info
 {
     /* Each frame can be threaded onto a doubly-linked list. */
xen/arch/riscv/include/asm/page.h +1 -34
index 4cb0179648..0684229790 100644
@@ -7,12 +7,10 @@
 
 #include <xen/bug.h>
 #include <xen/const.h>
-#include <xen/domain_page.h>
 #include <xen/errno.h>
 #include <xen/types.h>
 
 #include <asm/atomic.h>
-#include <asm/mm.h>
 #include <asm/page-bits.h>
 
 #define VPN_MASK                    (PAGETABLE_ENTRIES - 1UL)
@@ -114,17 +112,6 @@ typedef struct {
 #endif
 } pte_t;
 
-static inline pte_t paddr_to_pte(paddr_t paddr,
-                                 unsigned int permissions)
-{
-    return (pte_t) { .pte = (paddr_to_pfn(paddr) << PTE_PPN_SHIFT) | permissions };
-}
-
-static inline paddr_t pte_to_paddr(pte_t pte)
-{
-    return pfn_to_paddr(pte.pte >> PTE_PPN_SHIFT);
-}
-
 static inline bool pte_is_valid(pte_t p)
 {
     return p.pte & PTE_VALID;
@@ -198,18 +185,7 @@ static inline void invalidate_icache(void)
 #define clear_page(page) memset((void *)(page), 0, PAGE_SIZE)
 #define copy_page(dp, sp) memcpy(dp, sp, PAGE_SIZE)
 
-static inline void flush_page_to_ram(unsigned long mfn, bool sync_icache)
-{
-    const void *v = map_domain_page(_mfn(mfn));
-
-    if ( clean_and_invalidate_dcache_va_range(v, PAGE_SIZE) )
-        BUG();
-
-    unmap_domain_page(v);
-
-    if ( sync_icache )
-        invalidate_icache();
-}
+void flush_page_to_ram(unsigned long mfn, bool sync_icache);
 
 /* Write a pagetable entry. */
 static inline void write_pte(pte_t *p, pte_t pte)
@@ -231,15 +207,6 @@ static inline pte_t pte_from_mfn(mfn_t mfn, pte_attr_t flags)
 
 pte_t pt_walk(vaddr_t va, unsigned int *pte_level);
 
-static inline mfn_t _vmap_to_mfn(vaddr_t va)
-{
-    pte_t entry = pt_walk(va, NULL);
-
-    BUG_ON(!pte_is_mapping(entry));
-
-    return mfn_from_pte(entry);
-}
-
 #endif /* __ASSEMBLY__ */
 
 #endif /* ASM__RISCV__PAGE_H */
xen/arch/riscv/mm.c +14 -0
index 4047d67c0e..774ea42f2d 100644
@@ -3,6 +3,7 @@
 #include <xen/bootfdt.h>
 #include <xen/bug.h>
 #include <xen/compiler.h>
+#include <xen/domain_page.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/libfdt/libfdt.h>
@@ -613,3 +614,16 @@ void __iomem *ioremap(paddr_t pa, size_t len)
 {
     return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE);
 }
+
+void flush_page_to_ram(unsigned long mfn, bool sync_icache)
+{
+    const void *v = map_domain_page(_mfn(mfn));
+
+    if ( clean_and_invalidate_dcache_va_range(v, PAGE_SIZE) )
+        BUG();
+
+    unmap_domain_page(v);
+
+    if ( sync_icache )
+        invalidate_icache();
+}
Signature · 1 line
2.49.0
Acked-by on v1 1/2
#2 lore raw

Re: [PATCH v1 1/2] xen/riscv: rework asm/mm.h and asm/page.h includes to match other architectures

Quoted 37 lines · Oleksii Kurochko

To align with other architectures where <asm/page.h> is included from <asm/mm.h>
(and not the other way around), the following changes are made:
- Since <asm/mm.h> is no longer included in <asm/page.h>:
- Move the definitions of paddr_to_pte() and pte_to_paddr() to <asm/mm.h>,
as paddr_to_pfn() and pte_to_paddr() are already defined there.
- Move _vmap_to_mfn() to <asm/mm.h> because mfn_from_pte() is defined there and
open-code it inside macros vmap_to_mfn().
- Drop the inclusion of <xen/domain_page.h> from <asm/page.h> to resolve a compilation error:
./include/xen/domain_page.h:63:12: error: implicit declaration of function '__mfn_to_virt'; did you mean 'mfn_to_nid'? [-Werror=implicit-function-declaration]
63 | return __mfn_to_virt(mfn_x(mfn));
This happens because __mfn_to_virt() is defined in <asm/mm.h>, but due to
the current include chain:

    <xen/domain.h>
      <asm/domain.h>
        <xen/mm.h>
          <asm/mm.h>
            <asm/page.h>
              <xen/domain_page.h>
                static inline void *map_domain_page_global(mfn_t mfn)
		{
		    return __mfn_to_virt(mfn_x(mfn));
                }
            ...
          ...
          #define __mfn_to_virt() ...

This leads to a circular dependency and the build error above.

As a result, since <xen/domain_page.h> is no longer included in <asm/page.h>, the flush_page_to_ram() definition cannot remain there. It is now moved to riscv/mm.c.

Including <asm/page.h> from <asm/mm.h> does not cause issues with the
declaration/definition of clear_page() when <xen/mm.h> is included, and
also prevents build errors such as:
common/domain.c: In function 'alloc_domain_struct':
common/domain.c:797:5: error: implicit declaration of function 'clear_page';did you mean 'steal_page'? [-Werror=implicit-function-declaration]

Quoted 6 lines
  797 |     clear_page(d);
      |     ^~~~~~~~~~
      |     steal_page

caused by using clear_page() in common/domain.c.

This does want further cleaning, but is a step in the right direction.

~Andrew

#3 lore raw

PATCH 2/2 xen/domain: introduce generic weak function for domain struct allocation

In reply to Oleksii Kurochko

From: Roger Pau Monne <[email protected]>

x86 has specific requirements about the allocation of the domain structure, but that's not the case for ARM or likely other architectures.

Introduce a generic weak function that can be overwritten with an architecture specific implementation if required.

No functional change intended.

---
Changes:
- Add Reviewed-by: Oleksii Kurochko <[email protected]>.
- Add empty line before last return in alloc_domain_struct().

---
 xen/arch/arm/domain.c  | 12 ------------
 xen/arch/ppc/stubs.c   |  5 -----
 xen/arch/riscv/stubs.c |  5 -----
 xen/common/domain.c    | 14 ++++++++++++++
 4 files changed, 14 insertions(+), 22 deletions(-)
xen/arch/arm/domain.c +0 -12
index 45aeb8bddc..29588f869c 100644
@@ -479,18 +479,6 @@ void startup_cpu_idle_loop(void)
     reset_stack_and_jump(idle_loop);
 }
 
-struct domain *alloc_domain_struct(void)
-{
-    struct domain *d;
-    BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
-    d = alloc_xenheap_pages(0, 0);
-    if ( d == NULL )
-        return NULL;
-
-    clear_page(d);
-    return d;
-}
-
 void free_domain_struct(struct domain *d)
 {
     free_xenheap_page(d);
xen/arch/ppc/stubs.c +0 -5
index 671e71aa0a..d999d22718 100644
@@ -294,11 +294,6 @@ void vcpu_kick(struct vcpu *v)
     BUG_ON("unimplemented");
 }
 
-struct domain *alloc_domain_struct(void)
-{
-    BUG_ON("unimplemented");
-}
-
 struct vcpu *alloc_vcpu_struct(const struct domain *d)
 {
     BUG_ON("unimplemented");
xen/arch/riscv/stubs.c +0 -5
index e396b67cd3..155e5a7f58 100644
@@ -268,11 +268,6 @@ void vcpu_kick(struct vcpu *v)
     BUG_ON("unimplemented");
 }
 
-struct domain *alloc_domain_struct(void)
-{
-    BUG_ON("unimplemented");
-}
-
 struct vcpu *alloc_vcpu_struct(const struct domain *d)
 {
     BUG_ON("unimplemented");
xen/common/domain.c +14 -0
index e566a18747..c134868e95 100644
@@ -785,6 +785,20 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
     return arch_sanitise_domain_config(config);
 }
 
+struct domain *__weak alloc_domain_struct(void)
+{
+    struct domain *d = alloc_xenheap_pages(0, 0);
+
+    BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
+
+    if ( !d )
+        return NULL;
+
+    clear_page(d);
+
+    return d;
+}
+
 struct domain *domain_create(domid_t domid,
                              struct xen_domctl_createdomain *config,
                              unsigned int flags)
Signature · 1 line
2.49.0
#4 lore raw

Re: [PATCH v1 2/2] xen/domain: introduce generic weak function for domain struct allocation

xen/common/domain.c static int sanitise_domain_config(struct xen_domctl_createdomain *config) · line 790

Quoted 11 lines · Oleksii Kurochko
xen/common/domain.c +3 -0
index e566a18747..c134868e95 100644
@@ -785,6 +785,20 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
     return arch_sanitise_domain_config(config);
 }
 
+struct domain *__weak alloc_domain_struct(void)
+{
+    struct domain *d = alloc_xenheap_pages(0, 0);

I know you're just moving what ARM has, but this is spelt alloc_xenheap_page();

xen/common/domain.c static int sanitise_domain_config(struct xen_domctl_createdomain *config) · line 801

Quoted 11 lines
xen/common/domain.c +11 -0
+
+    BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
+
+    if ( !d )
+        return NULL;
+
+    clear_page(d);
+
+    return d;
+}
+

This looks fine, but you must do the same with free_domain_struct() at the same time to keep the pair symmetric.  That in turn gets you an even bigger negative diffstat, and x86 doesn't even need to override the common version.

vCPU really wants doing at the same time, although you're going to run into problems on ARM there with the BUILD_BUG_ON().

However, I suspect we want to be building with -ffunction-sections generally active, or IIRC we'll still have the weak copy present in the final image.

~Andrew

#5 lore raw

Re: [PATCH v1 2/2] xen/domain: introduce generic weak function for domain struct allocation

Quoted 33 lines · Andrew Cooper

xen/common/domain.c static int sanitise_domain_config(struct xen_domctl_createdomain *config) · line 790

Quoted 11 lines · Oleksii Kurochko
xen/common/domain.c +3 -0
index e566a18747..c134868e95 100644
@@ -785,6 +785,20 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
     return arch_sanitise_domain_config(config);
 }
 
+struct domain *__weak alloc_domain_struct(void)
+{
+    struct domain *d = alloc_xenheap_pages(0, 0);

I know you're just moving what ARM has, but this is spelt alloc_xenheap_page();

xen/common/domain.c static int sanitise_domain_config(struct xen_domctl_createdomain *config) · line 801

Quoted 11 lines
xen/common/domain.c +11 -0
+
+    BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);
+
+    if ( !d )
+        return NULL;
+
+    clear_page(d);
+
+    return d;
+}
+

This looks fine, but you must do the same with free_domain_struct() at the same time to keep the pair symmetric.  That in turn gets you an even bigger negative diffstat, and x86 doesn't even need to override the common version.

Quoted 6 lines

vCPU really wants doing at the same time, although you're going to run into problems on ARM there with the BUILD_BUG_ON().

However, I suspect we want to be building with -ffunction-sections generally active, or IIRC we'll still have the weak copy present in the final image.

Instead of introducing a weak function in common code, why don't we move the function there altogether, adding merely an arch hook to control the flags to be passed into alloc_xenheap_pages()?

Jan