Skip to contentSkip to conversation

Enable build of full Xen for RISC-V

PATCH Superseded Oleksii Kurochko 4 participants 41 messages Jun 25, 2024 – Jun 27, 2024 RISC-V

v13 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#1 lore raw

PATCH v13 1/10 xen: introduce generic non-atomic test_*bit()

In reply to Oleksii Kurochko

The following generic functions were introduced:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit

These functions and macros can be useful for architectures that don't have corresponding arch-specific instructions.

Also, the patch introduces the following generics which are used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE

The following approach was chosen for generic*() and arch*() bit operation functions:
If the bit operation function that is going to be generic starts with the prefix "__", then the corresponding generic/arch function will also contain the "__" prefix. For example:
* test_bit() will be defined using arch_test_bit() and generic_test_bit().
* __test_and_set_bit() will be defined using arch__test_and_set_bit() and generic__test_and_set_bit().

---
Changes in V13:
- add Reviewed-by: Jan Beulich <[email protected]>
---
Changes in V12:
- revert change of moving the definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h.
( a separate patch will be provided to put BITS_PER_BYTE to proper place )
- drop comments on top of generic_*() functions and update the comments above __test_*() and test_bit().
- update how static inline __test_*() are defined ( drop pointless fallback #define ) and test_bit().
- drop the footer after Signed-off-by.
Changes in V11:
- fix identation in generic_test_bit() function.
- move definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h
- drop the changes in arm64/livepatch.c.
- update the the comments on top of functions: generic__test_and_set_bit(), generic__test_and_clear_bit(), generic__test_and_change_bit(), generic_test_bit().
- Update footer after Signed-off section.
- Rebase the patch on top of staging branch, so it can be merged when necessary approves will be given.
- add Reviewed-by: Jan Beulich <[email protected]>.
---
Changes in V10:
- update the commit message. ( re-order paragraphs and add explanation usage of prefix "__" in bit
operation function names )
- add parentheses around the whole expression of bitop_bad_size() macros.
- move macros bitop_bad_size() above asm/bitops.h as it is not arch-specific anymore and there is
no need for overriding it.
- drop macros check_bitop_size() and use "if ( bitop_bad_size(addr) ) __bitop_bad_size();" implictly
where it is needed.
- in <xen/bitops.h> use 'int' as a first parameter for __test_and_*(), generic__test_and_*() to be
consistent with how the mentioned functions were declared in the original per-arch functions.
- add 'const' to p variable in generic_test_bit().
- move definition of BITOP_BITS_PER_WORD and bitop_uint_t to xen/bitops.h as we don't allow for arch
overrides these definitions anymore.
---
Changes in V9:
- move up xen/bitops.h in ppc/asm/page.h.
- update defintion of arch_check_bitop_size.
And drop correspondent macros from x86/asm/bitops.h
- drop parentheses in generic__test_and_set_bit() for definition of
local variable p.
- fix indentation inside #ifndef BITOP_TYPE...#endif
- update the commit message.
---
Changes in V8:
- drop __pure for function which uses volatile.
- drop unnessary () in generic__test_and_change_bit() for addr casting.
- update prototype of generic_test_bit() and test_bit(): now it returns bool
instead of int.
- update generic_test_bit() to use BITOP_MASK().
- Deal with fls{l} changes: it should be in the patch with introduced generic fls{l}.
- add a footer with explanation of dependency on an uncommitted patch after Signed-off.
- abstract bitop_size().
- move BITOP_TYPE define to <xen/types.h>.
---
Changes in V7:
- move everything to xen/bitops.h to follow the same approach for all generic
bit ops.
- put together BITOP_BITS_PER_WORD and bitops_uint_t.
- make BITOP_MASK more generic.
- drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
enough.
- drop "_" for generic__{test_and_set_bit,...}().
- drop " != 0" for functions which return bool.
- add volatile during the cast for generic__{...}().
- update the commit message.
- update arch related code to follow the proposed generic approach.
---
Changes in V6:
- Nothing changed ( only rebase )
---
Changes in V5:
- new patch

---
 xen/arch/arm/include/asm/bitops.h |  69 -----------
 xen/arch/ppc/include/asm/bitops.h |  54 ---------
 xen/arch/ppc/include/asm/page.h   |   2 +-
 xen/arch/ppc/mm-radix.c           |   2 +-
 xen/arch/x86/include/asm/bitops.h |  31 ++---
 xen/include/xen/bitops.h          | 182 ++++++++++++++++++++++++++++++
 6 files changed, 193 insertions(+), 147 deletions(-)
xen/arch/arm/include/asm/bitops.h +0 -69
index 8f4bdc09d1..db23d7edc3 100644
@@ -22,11 +22,6 @@
 #define __set_bit(n,p)            set_bit(n,p)
 #define __clear_bit(n,p)          clear_bit(n,p)
 
-#define BITOP_BITS_PER_WORD     32
-#define BITOP_MASK(nr)          (1UL << ((nr) % BITOP_BITS_PER_WORD))
-#define BITOP_WORD(nr)          ((nr) / BITOP_BITS_PER_WORD)
-#define BITS_PER_BYTE           8
-
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
@@ -76,70 +71,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,
 bool clear_mask16_timeout(uint16_t mask, volatile void *p,
                           unsigned int max_try);
 
-/**
- * __test_and_set_bit - Set a bit and return its old value
- * @nr: Bit to set
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_set_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old | mask;
-        return (old & mask) != 0;
-}
-
-/**
- * __test_and_clear_bit - Clear a bit and return its old value
- * @nr: Bit to clear
- * @addr: Address to count from
- *
- * This operation is non-atomic and can be reordered.
- * If two examples of this operation race, one can appear to succeed
- * but actually fail.  You must protect multiple accesses with a lock.
- */
-static inline int __test_and_clear_bit(int nr, volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old & ~mask;
-        return (old & mask) != 0;
-}
-
-/* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr,
-                                            volatile void *addr)
-{
-        unsigned int mask = BITOP_MASK(nr);
-        volatile unsigned int *p =
-                ((volatile unsigned int *)addr) + BITOP_WORD(nr);
-        unsigned int old = *p;
-
-        *p = old ^ mask;
-        return (old & mask) != 0;
-}
-
-/**
- * test_bit - Determine whether a bit is set
- * @nr: bit number to test
- * @addr: Address to start counting from
- */
-static inline int test_bit(int nr, const volatile void *addr)
-{
-        const volatile unsigned int *p = (const volatile unsigned int *)addr;
-        return 1UL & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD-1)));
-}
-
 #define arch_ffs(x)  ((x) ? 1 + __builtin_ctz(x) : 0)
 #define arch_ffsl(x) ((x) ? 1 + __builtin_ctzl(x) : 0)
 #define arch_fls(x)  ((x) ? 32 - __builtin_clz(x) : 0)
xen/arch/ppc/include/asm/bitops.h +0 -54

72 lines not on this page · raw message

xen/arch/ppc/include/asm/page.h +1 -1
index 890e285051..6d4cd2611c 100644
@@ -2,9 +2,9 @@
 #ifndef _ASM_PPC_PAGE_H
 #define _ASM_PPC_PAGE_H
 
+#include <xen/bitops.h>
 #include <xen/types.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 
 #define PDE_VALID     PPC_BIT(0)
xen/arch/ppc/mm-radix.c +1 -1
index ab5a10695c..9055730997 100644
@@ -1,11 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <xen/bitops.h>
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/mm.h>
 #include <xen/types.h>
 #include <xen/lib.h>
 
-#include <asm/bitops.h>
 #include <asm/byteorder.h>
 #include <asm/early_printk.h>
 #include <asm/page.h>
xen/arch/x86/include/asm/bitops.h +9 -22
index aa71542e7b..f9aa60111f 100644
@@ -19,9 +19,6 @@
 #define ADDR (*(volatile int *) addr)
 #define CONST_ADDR (*(const volatile int *) addr)
 
-extern void __bitop_bad_size(void);
-#define bitop_bad_size(addr) (sizeof(*(addr)) < 4)
-
 /**
  * set_bit - Atomically set a bit in memory
  * @nr: the bit to set
@@ -175,7 +172,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_set_bit - Set a bit and return its old value
+ * arch__test_and_set_bit - Set a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_set_bit(int nr, void *addr)
+static inline int arch__test_and_set_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -194,10 +191,7 @@ static inline int __test_and_set_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_set_bit(nr, addr) ({                 \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_set_bit(nr, addr);                       \
-})
+#define arch__test_and_set_bit arch__test_and_set_bit
 
 /**
  * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +218,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
 })
 
 /**
- * __test_and_clear_bit - Clear a bit and return its old value
+ * arch__test_and_clear_bit - Clear a bit and return its old value
  * @nr: Bit to set
  * @addr: Address to count from
  *
@@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)
  * If two examples of this operation race, one can appear to succeed
  * but actually fail.  You must protect multiple accesses with a lock.
  */
-static inline int __test_and_clear_bit(int nr, void *addr)
+static inline int arch__test_and_clear_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_clear_bit(nr, addr) ({               \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_clear_bit(nr, addr);                     \
-})
+#define arch__test_and_clear_bit arch__test_and_clear_bit
 
 /* WARNING: non atomic and it can be reordered! */
-static inline int __test_and_change_bit(int nr, void *addr)
+static inline int arch__test_and_change_bit(int nr, volatile void *addr)
 {
     int oldbit;
 
@@ -260,10 +251,7 @@ static inline int __test_and_change_bit(int nr, void *addr)
 
     return oldbit;
 }
-#define __test_and_change_bit(nr, addr) ({              \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
-    __test_and_change_bit(nr, addr);                    \
-})
+#define arch__test_and_change_bit arch__test_and_change_bit
 
 /**
  * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
     return oldbit;
 }
 
-#define test_bit(nr, addr) ({                           \
-    if ( bitop_bad_size(addr) ) __bitop_bad_size();     \
+#define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
         constant_test_bit(nr, addr) :                   \
         variable_test_bit(nr, addr);                    \
xen/include/xen/bitops.h +182 -0

194 lines not on this page · raw message

Signature · 1 line
2.45.2
#2 lore raw

PATCH v13 2/10 xen/riscv: introduce bitops.h

In reply to Oleksii Kurochko

Taken from Linux-6.4.0-rc1

Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
* The following function were removed as they aren't used in Xen:
* test_and_set_bit_lock
* clear_bit_unlock
* __clear_bit_unlock
* The following functions were renamed in the way how they are used by common code:
* __test_and_set_bit
* __test_and_clear_bit
* The declaration and implementation of the following functios were updated to make Xen build happy:
* clear_bit
* set_bit
* __test_and_clear_bit
* __test_and_set_bit

---
Changes in V11-V13:
- Nothing changed. Only rebase was done.
---
Changes in V10:
- update the error message BITS_PER_LONG -> BITOP_BITS_PER_WORD
---
Changes in V9:
- add Acked-by: Jan Beulich <[email protected]>
- drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects to work with 32-bit quantities.
- s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.
---
Changes in V8:
- define bitop_uint_t in <asm/types.h> after the changes in patch related to introduction of "introduce generic non-atomic test_*bit()".
- drop duplicated __set_bit() and __clear_bit().
- drop duplicated comment: /* Based on linux/arch/include/asm/bitops.h */.
- update type of res and mask in test_and_op_bit_ord(): unsigned long -> bitop_uint_t.
- drop 1 padding blank in test_and_op_bit_ord().
- update definition of test_and_set_bit(),test_and_clear_bit(),test_and_change_bit:
change return type to bool.
- change addr argument type of test_and_change_bit(): unsigned long * -> void *.
- move test_and_change_bit() closer to other test_and-s function.
- Code style fixes: tabs -> space.
- s/#undef __op_bit/#undef op_bit.
- update the commit message: delete information about generic-non-atomic.h changes as now it is a separate patch.
---
Changes in V7:
- Update the commit message.
- Drop "__" for __op_bit and __op_bit_ord as they are atomic.
- add comment above __set_bit and __clear_bit about why they are defined as atomic.
- align bitops_uint_t with __AMO().
- make changes after generic non-atomic test_*bit() were changed.
- s/__asm__ __volatile__/asm volatile
---
Changes in V6:
- rebase clean ups were done: drop unused asm-generic includes
---
Changes in V5:
- new patch

---
 xen/arch/riscv/include/asm/bitops.h | 137 ++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
xen/arch/riscv/include/asm/bitops.h new file +137 -0
new file mode 100644
index 0000000000..7f7af3fda1
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))
+
+/**
+ * test_and_set_bit - Set a bit and return its old value
+ * @nr: Bit to set
+ * @addr: Address to count from
+ */
+static inline bool test_and_set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * test_and_clear_bit - Clear a bit and return its old value
+ * @nr: Bit to clear
+ * @addr: Address to count from
+ */
+static inline bool test_and_clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(and, NOT, nr, addr);
+}
+
+/**
+ * test_and_change_bit - Toggle (change) a bit and return its old value
+ * @nr: Bit to change
+ * @addr: Address to count from
+ *
+ * This operation is atomic and cannot be reordered.
+ * It also implies a memory barrier.
+ */
+static inline bool test_and_change_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    return test_and_op_bit(xor, NOP, nr, addr);
+}
+
+/**
+ * set_bit - Atomically set a bit in memory
+ * @nr: the bit to set
+ * @addr: the address to start counting from
+ *
+ * Note that @nr may be almost arbitrarily large; this function is not
+ * restricted to acting on a single-word quantity.
+ */
+static inline void set_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(or, NOP, nr, addr);
+}
+
+/**
+ * clear_bit - Clears a bit in memory
+ * @nr: Bit to clear
+ * @addr: Address to start counting from
+ */
+static inline void clear_bit(int nr, volatile void *p)
+{
+    volatile bitop_uint_t *addr = p;
+
+    op_bit(and, NOT, nr, addr);
+}
+
+#undef test_and_op_bit
+#undef op_bit
+#undef NOP
+#undef NOT
+#undef __AMO
+
+#endif /* _ASM_RISCV_BITOPS_H */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
Signature · 1 line
2.45.2
#10 lore raw

PATCH v13 3/10 xen/riscv: add minimal stuff to mm.h to build full Xen

In reply to Oleksii Kurochko

---
Changes in V13:
- redefine mfn_to_page() and mfn_to_page().
- add inclusion of <xen/section.h> after rebasing on top of staging.
---
Changes in V8-V12:
- Nothing changed only rebase.
---
Changes in V7:
- update argument type of maddr_to_virt() function: unsigned long -> paddr_t
- rename argument of PFN_ORDER(): pfn -> pg.
- add Acked-by: Jan Beulich <[email protected]>
---
Changes in V6:
- drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
- parenthesize va in definition of vmap_to_mfn().
- Code style fixes.
---
Changes in V5:
- update the comment around "struct domain *domain;" : zero -> NULL
- fix ident. for unsigned long val;
- put page_to_virt() and virt_to_page() close to each other.
- drop unnessary leading underscore
- drop a space before the comment: /* Count of uses of this frame as its current type. */
- drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
- update an argument name of PFN_ORDERN macros.
- drop pad at the end of 'struct page_info'.
- Change message -> subject in "Changes in V3"
- delete duplicated macros from riscv/mm.h
- fix identation in struct page_info
- align comment for PGC_ macros
- update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
- drop unnessary comments.
- s/BUG/BUG_ON("...")
- define __virt_to_maddr, __maddr_to_virt as stubs
- add inclusion of xen/mm-frame.h for mfn_x and others
- include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
In file included from arch/riscv/setup.c:7:
./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
60 | struct page_list_entry list;
| ^~~~
./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
81 | unsigned long first_dirty:MAX_ORDER + 1;
| ^~~~~~~~~
./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
81 | unsigned long first_dirty:MAX_ORDER + 1;
- Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
- update the commit title
- introduce DIRECTMAP_VIRT_START.
- drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
[PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
- code style fixes.
- drop get_page_nr and put_page_nr as they don't need for time being
- drop CONFIG_STATIC_MEMORY related things
- code style fixes
---
Changes in V2:
- define stub for arch_get_dma_bitsize(void)

---
 xen/arch/riscv/include/asm/mm.h | 235 ++++++++++++++++++++++++++++++++
 xen/arch/riscv/mm.c             |   2 +-
 xen/arch/riscv/setup.c          |   2 +-
 3 files changed, 237 insertions(+), 2 deletions(-)
xen/arch/riscv/include/asm/mm.h +235 -0
index 07c7a0abba..8f05937b0d 100644
@@ -3,11 +3,241 @@
 #ifndef _ASM_RISCV_MM_H
 #define _ASM_RISCV_MM_H
 
+#include <public/xen.h>
+#include <xen/bug.h>
+#include <xen/mm-frame.h>
+#include <xen/pdx.h>
+#include <xen/types.h>
+
 #include <asm/page-bits.h>
 
 #define pfn_to_paddr(pfn) ((paddr_t)(pfn) << PAGE_SHIFT)
 #define paddr_to_pfn(pa)  ((unsigned long)((pa) >> PAGE_SHIFT))
 
+#define paddr_to_pdx(pa)    mfn_to_pdx(maddr_to_mfn(pa))
+#define gfn_to_gaddr(gfn)   pfn_to_paddr(gfn_x(gfn))
+#define gaddr_to_gfn(ga)    _gfn(paddr_to_pfn(ga))
+#define mfn_to_maddr(mfn)   pfn_to_paddr(mfn_x(mfn))
+#define maddr_to_mfn(ma)    _mfn(paddr_to_pfn(ma))
+#define vmap_to_mfn(va)     maddr_to_mfn(virt_to_maddr((vaddr_t)(va)))
+#define vmap_to_page(va)    mfn_to_page(vmap_to_mfn(va))
+
+static inline void *maddr_to_virt(paddr_t ma)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+#define virt_to_maddr(va) ({ BUG_ON("unimplemented"); 0; })
+
+/* Convert between Xen-heap virtual addresses and machine frame numbers. */
+#define __virt_to_mfn(va)  mfn_x(maddr_to_mfn(virt_to_maddr(va)))
+#define __mfn_to_virt(mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn)))
+
+/*
+ * We define non-underscored wrappers for above conversion functions.
+ * These are overriden in various source files while underscored version
+ * remain intact.
+ */
+#define virt_to_mfn(va)     __virt_to_mfn(va)
+#define mfn_to_virt(mfn)    __mfn_to_virt(mfn)
+
+struct page_info
+{
+    /* Each frame can be threaded onto a doubly-linked list. */
+    struct page_list_entry list;
+
+    /* Reference count and various PGC_xxx flags and fields. */
+    unsigned long count_info;
+
+    /* Context-dependent fields follow... */
+    union {
+        /* Page is in use: ((count_info & PGC_count_mask) != 0). */
+        struct {
+            /* Type reference count and various PGT_xxx flags and fields. */
+            unsigned long type_info;
+        } inuse;
+
+        /* Page is on a free list: ((count_info & PGC_count_mask) == 0). */
+        union {
+            struct {
+                /*
+                 * Index of the first *possibly* unscrubbed page in the buddy.
+                 * One more bit than maximum possible order to accommodate
+                 * INVALID_DIRTY_IDX.
+                 */
+#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1)
+                unsigned long first_dirty:MAX_ORDER + 1;
+
+                /* Do TLBs need flushing for safety before next page use? */
+                bool need_tlbflush:1;
+
+#define BUDDY_NOT_SCRUBBING    0
+#define BUDDY_SCRUBBING        1
+#define BUDDY_SCRUB_ABORT      2
+                unsigned long scrub_state:2;
+            };
+
+            unsigned long val;
+        } free;
+    } u;
+
+    union {
+        /* Page is in use */
+        struct {
+            /* Owner of this page (NULL if page is anonymous). */
+            struct domain *domain;
+        } inuse;
+
+        /* Page is on a free list. */
+        struct {
+            /* Order-size of the free chunk this page is the head of. */
+            unsigned int order;
+        } free;
+    } v;
+
+    union {
+        /*
+         * Timestamp from 'TLB clock', used to avoid extra safety flushes.
+         * Only valid for: a) free pages, and b) pages with zero type count
+         */
+        uint32_t tlbflush_timestamp;
+    };
+};
+
+#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
+
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)    (frame_table + mfn_x(mfn))
+#define page_to_mfn(pg)     _mfn((unsigned long)((pg) - frame_table))
+
+static inline void *page_to_virt(const struct page_info *pg)
+{
+    return mfn_to_virt(mfn_x(page_to_mfn(pg)));
+}
+
+/* Convert between Xen-heap virtual addresses and page-info structures. */
+static inline struct page_info *virt_to_page(const void *v)
+{
+    BUG_ON("unimplemented");
+    return NULL;
+}
+
+/*
+ * Common code requires get_page_type and put_page_type.
+ * We don't care about typecounts so we just do the minimum to make it
+ * happy.
+ */
+static inline int get_page_type(struct page_info *page, unsigned long type)
+{
+    return 1;
+}
+
+static inline void put_page_type(struct page_info *page)
+{
+}
+
+static inline void put_page_and_type(struct page_info *page)
+{
+    put_page_type(page);
+    put_page(page);
+}
+
+/*
+ * RISC-V does not have an M2P, but common code expects a handful of
+ * M2P-related defines and functions. Provide dummy versions of these.
+ */
+#define INVALID_M2P_ENTRY        (~0UL)
+#define SHARED_M2P_ENTRY         (~0UL - 1UL)
+#define SHARED_M2P(_e)           ((_e) == SHARED_M2P_ENTRY)
+
+#define set_gpfn_from_mfn(mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0)
+#define mfn_to_gfn(d, mfn) ((void)(d), _gfn(mfn_x(mfn)))
+
+#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS)
+
+static inline unsigned long domain_get_maximum_gpfn(struct domain *d)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+static inline long arch_memory_op(int op, XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    BUG_ON("unimplemented");
+    return 0;
+}
+
+/*
+ * On RISCV, all the RAM is currently direct mapped in Xen.
+ * Hence return always true.
+ */
+static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)
+{
+    return true;
+}
+
+#define PG_shift(idx)   (BITS_PER_LONG - (idx))
+#define PG_mask(x, idx) (x ## UL << PG_shift(idx))
+
+#define PGT_none          PG_mask(0, 1)  /* no special uses of this page   */
+#define PGT_writable_page PG_mask(1, 1)  /* has writable mappings?         */
+#define PGT_type_mask     PG_mask(1, 1)  /* Bits 31 or 63.                 */
+
+/* Count of uses of this frame as its current type. */
+#define PGT_count_width   PG_shift(2)
+#define PGT_count_mask    ((1UL << PGT_count_width) - 1)
+
+/*
+ * Page needs to be scrubbed. Since this bit can only be set on a page that is
+ * free (i.e. in PGC_state_free) we can reuse PGC_allocated bit.
+ */
+#define _PGC_need_scrub   _PGC_allocated
+#define PGC_need_scrub    PGC_allocated
+
+/* Cleared when the owning guest 'frees' this page. */
+#define _PGC_allocated    PG_shift(1)
+#define PGC_allocated     PG_mask(1, 1)
+/* Page is Xen heap? */
+#define _PGC_xen_heap     PG_shift(2)
+#define PGC_xen_heap      PG_mask(1, 2)
+/* Page is broken? */
+#define _PGC_broken       PG_shift(7)
+#define PGC_broken        PG_mask(1, 7)
+/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */
+#define PGC_state         PG_mask(3, 9)
+#define PGC_state_inuse   PG_mask(0, 9)
+#define PGC_state_offlining PG_mask(1, 9)
+#define PGC_state_offlined PG_mask(2, 9)
+#define PGC_state_free    PG_mask(3, 9)
+#define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st)
+
+/* Count of references to this frame. */
+#define PGC_count_width   PG_shift(9)
+#define PGC_count_mask    ((1UL << PGC_count_width) - 1)
+
+#define _PGC_extra        PG_shift(10)
+#define PGC_extra         PG_mask(1, 10)
+
+#define is_xen_heap_page(page) ((page)->count_info & PGC_xen_heap)
+#define is_xen_heap_mfn(mfn) \
+    (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn)))
+
+#define is_xen_fixed_mfn(mfn)                                   \
+    ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) &&   \
+     (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1)))
+
+#define page_get_owner(p)    (p)->v.inuse.domain
+#define page_set_owner(p, d) ((p)->v.inuse.domain = (d))
+
+/* TODO: implement */
+#define mfn_valid(mfn) ({ (void)(mfn); 0; })
+
+#define domain_set_alloc_bitsize(d) ((void)(d))
+#define domain_clamp_alloc_bitsize(d, b) ((void)(d), (b))
+
+#define PFN_ORDER(pg) ((pg)->v.free.order)
+
 extern unsigned char cpu0_boot_stack[];
 
 void setup_initial_pagetables(void);
@@ -20,4 +250,9 @@ unsigned long calc_phys_offset(void);
 
 void turn_on_mmu(unsigned long ra);
 
+static inline unsigned int arch_get_dma_bitsize(void)
+{
+    return 32; /* TODO */
+}
+
 #endif /* _ASM_RISCV_MM_H */
xen/arch/riscv/mm.c +1 -1
index 3ebaf6da01..ae381e9581 100644
@@ -4,13 +4,13 @@
 #include <xen/init.h>
 #include <xen/kernel.h>
 #include <xen/macros.h>
+#include <xen/mm.h>
 #include <xen/pfn.h>
 #include <xen/sections.h>
 
 #include <asm/early_printk.h>
 #include <asm/csr.h>
 #include <asm/current.h>
-#include <asm/mm.h>
 #include <asm/page.h>
 #include <asm/processor.h>
 
xen/arch/riscv/setup.c +1 -1
index 6593f601c1..98a94c4c48 100644
@@ -2,9 +2,9 @@
 
 #include <xen/compile.h>
 #include <xen/init.h>
+#include <xen/mm.h>
 
 #include <asm/early_printk.h>
-#include <asm/mm.h>
 
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
Signature · 1 line
2.45.2
#14 lore raw

PATCH v13 5/10 xen/riscv: enable full Xen build

In reply to Oleksii Kurochko

---
Changes in V13:
- implement get_upper_mfn_bound() as BUG_ON("unimplemented")
- drop the footer after R-by as Andrew's patch series were merged to staging.
---
Changes in V5-V12:
- Nothing changed. Only rebase.
- Add the footer after Signed-off section.
---
Changes in V4:
- drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
- drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
- drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
- Reviewed-by: Jan Beulich <[email protected]>
- unrealted change dropped in tiny64_defconfig
---
Changes in V2:
- Nothing changed. Only rebase.

---
 xen/arch/riscv/Makefile       |  16 +++-
 xen/arch/riscv/arch.mk        |   4 -
 xen/arch/riscv/early_printk.c | 168 ----------------------------------
 3 files changed, 15 insertions(+), 173 deletions(-)
xen/arch/riscv/Makefile +15 -1
index 60afbc0ad9..81b77b13d6 100644
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms
 	$(OBJCOPY) -O binary -S $< $@
 
 $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
-	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(objtree)/common/symbols-dummy.o -o $(dot-target).0
+	$(NM) -pa --format=sysv $(dot-target).0 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).0.S
+	$(MAKE) $(build)=$(@D) $(dot-target).0.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \
+	    $(dot-target).0.o -o $(dot-target).1
+	$(NM) -pa --format=sysv $(dot-target).1 \
+		| $(objtree)/tools/symbols $(all_symbols) --sysv --sort \
+		> $(dot-target).1.S
+	$(MAKE) $(build)=$(@D) $(dot-target).1.o
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \
+	    $(dot-target).1.o -o $@
 	$(NM) -pa --format=sysv $@ \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
 		> [email protected]
+	rm -f $(@D)/.$(@F).[0-9]*
 
 $(obj)/xen.lds: $(src)/xen.lds.S FORCE
 	$(call if_changed_dep,cpp_lds_S)
xen/arch/riscv/arch.mk +0 -4
index 8c071aff65..17827c302c 100644
@@ -38,7 +38,3 @@ extensions := $(subst $(space),,$(extensions))
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany
-
-# TODO: Drop override when more of the build is working
-override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
-override ALL_LIBS-y =
xen/arch/riscv/early_printk.c +0 -168

171 lines not on this page · raw message

Signature · 1 line
2.45.2
#23 lore raw

PATCH v13 8/10 xen/riscv: change .insn to .byte in cpu_relax()

In reply to Oleksii Kurochko

The .insn directive appears to check that the byte pattern is a known extension, where .4byte doesn't.

The following compilation error occurs:
./arch/riscv/include/asm/processor.h: Assembler messages:
./arch/riscv/include/asm/processor.h:70: Error: unrecognized opcode `0x0100000F'
In case of the following Binutils:

  $ riscv64-linux-gnu-as --version
  GNU assembler (GNU Binutils for Debian) 2.35.2

---
Changes in V13:
- new patch

---
 xen/arch/riscv/include/asm/processor.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
xen/arch/riscv/include/asm/processor.h +1 -1
index 6846151717..0e75122efb 100644
@@ -67,7 +67,7 @@ static inline void cpu_relax(void)
     __asm__ __volatile__ ( "pause" );
 #else
     /* Encoding of the pause instruction */
-    __asm__ __volatile__ ( ".insn 0x0100000F" );
+    __asm__ __volatile__ ( ".byte 0x0100000F" );
 #endif
 
     barrier();
Signature · 1 line
2.45.2
#0 lore raw

v13 cover Enable build of full Xen for RISC-V

***
I think that I need to write a separate email requesting approval for this patch series to be included in Xen 4.19. Most of the patches are RISC-V specific, so there is a low risk of breaking anything else.
There is only one patch that touches common or arch-specific files, and it doesn't introduce any functional changes.
Since I can't approve it myself, I am asking for someone else to do so.
***

This patch series performs all of the additions necessary to drop the build overrides for RISCV and enable the full Xen build. Except in cases where compatibile implementations already exist (e.g. atomic.h and bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
- [PATCH 3/3] xen/ppc: Avoid using the legacy __read_mostly/__ro_after_init definitions [1]

The link to the branch with rebased patches on top of [1] could be found here:
https://gitlab.com/xen-project/people/olkur/xen/-/commits/riscv-full-xen-build-v13

[1] https://lore.kernel.org/xen-devel/[email protected]/

---
Changes in V13:
- Patch was merged to staging:
- [PATCH v12 ] xen/riscv: Update Kconfig in preparation for a full
- Rebase on top of the current staging.
- Update cover letter message
- It was added 2 new patches ( patches 8 and 9 ) which are not necessary for testing environment we have in CI but they improves compatability with older gcc and binutils.
- It was added patch 10 as a clean up of [PATCH v12 2/8] xen: introduce generic non-atomic test_*bit() for x86.
- Drop [PATCH v12 4/8] xen/riscv: add definition of __read_mostly as it was defined as generic now for all architectures.
- It was added the patch [PATCH v13 07/10] xen/common: fix build issue for common/trace.c to resolve compilation issue for RISC-V after reabsing on top of current staging.
- Other changes are specific to specific patches. Please look at changes for
specific patch.
---
Changes in V12:
- Rebase the patch series on top of [1] mentioned above.
- Update the cover letter message.
- "[PATCH v11 3/9] xen/bitops: implement fls{l}() in common logic" was droped
as it is a part of patch series [1] mentioned above.
- Other changes are specific to specific patches. Please look at changes for
specific patch.
---
Changes in V11:
- Patch was merged to staging:
- [PATCH v10 05/14] xen/riscv: introduce cmpxchg.h
- [PATCH v10 06/14] xen/riscv: introduce atomic.h
- [PATCH v10 07/14] xen/riscv: introduce monitor.h
- [PATCH v10 09/14] xen/riscv: add required things to current.h
- [PATCH v10 11/14] xen/riscv: introduce vm_event_*() functions
- Other changes are specific to specific patches. Please look at changes for
specific patch.
---
Changes in V10:
- Patch was merged to staging:
- [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
- Other changes are specific to specific patches. Please look at changes for
specific patch.
---
Changes in V9:
- Patch was merged to staging:
- [PATCH v8 07/17] xen/riscv: introduce io.h
- [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
- Other changes are specific to specific patches. Please look at changes for
specific patch.
---
Changes in V8:
- Patch was merged to staging:
- [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
- [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
- Other changes are specific to specific patches. Please look at changes for
specific patch.
- Update the commit message:
- drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
- Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
- Patch was merged to staging:
[PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
- Other changes are specific to specific patches. Please look at changes for
specific patch.
---
Changes in V6:
- Update the cover letter message: drop already merged dependecies and add
a new one.
- Patches were merged to staging:
- [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
- [PATCH v5 03/23] xen/riscv: introduce nospec.h
- [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
- Introduce new patches:
- xen/riscv: introduce extenstion support check by compiler
- xen/bitops: put __ffs() and ffz() into linux compatible header
- xen/bitops: implement fls{l}() in common logic
- The following patches were dropped:
- drop some patches related to bitops operations as they were introduced in another

     patch series [...]

- introduce new version for generic __ffs(), ffz() and fls{l}().
- Merge patch from patch series "[PATCH v9 0/7] Introduce generic headers" to this patch series as only one patch left in the generic headers patch series and it is more about RISC-V.
- Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
- Update the cover letter as one of the dependencies were merged to staging.
- Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
- Moved fence.h to separate patch to deal with some patches dependecies on fence.h
- Patches were dropped as they were merged to staging:
* [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
* [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
* [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
* [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
* [PATCH v4 08/30] xen/riscv: introduce setup.h
* [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
* [PATCH v4 11/30] xen/riscv: introduce smp.h
* [PATCH v4 15/30] xen/riscv: introduce irq.h
* [PATCH v4 16/30] xen/riscv: introduce p2m.h
* [PATCH v4 17/30] xen/riscv: introduce regs.h
* [PATCH v4 18/30] xen/riscv: introduce time.h
* [PATCH v4 19/30] xen/riscv: introduce event.h
* [PATCH v4 22/30] xen/riscv: define an address of frame table
- Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
- Update the cover letter message: new patch series dependencies.
- Some patches were merged to staging, so they were dropped in this patch series:
[PATCH v3 09/34] xen/riscv: introduce system.h
[PATCH v3 18/34] xen/riscv: introduce domain.h
[PATCH v3 19/34] xen/riscv: introduce guest_access.h
- Was sent out of this patch series:
[PATCH v3 16/34] xen/lib: introduce generic find next bit operations
- [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
- All other changes are specific to a specific patch.
---
Changes in V3:
- Update the cover letter message
- The following patches were dropped as they were merged to staging:
[PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
[PATCH v2 04/39] xen/riscv: add public arch-riscv.h
[PATCH v2 05/39] xen/riscv: introduce spinlock.h
[PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
[PATCH v2 34/39] xen: add RISCV support for pmu.h
[PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
- Instead of the following patches were introduced new:
[PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
[PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
- remove "asm/" for commit messages which start with "xen/riscv:"
- code style updates.
- add emulation of {cmp}xchg_* for 1 and 2 bytes types.
- code style fixes.
- add SPDX and footer for the newly added headers.
- introduce generic find-next-bit.c.
- some other mionor changes. ( details please find in a patch )
---
Changes in V2:
- Drop the following patches as they are the part of [2]:
[PATCH v1 06/57] xen/riscv: introduce paging.h
[PATCH v1 08/57] xen/riscv: introduce asm/device.h
[PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
[PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
[PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
[PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
[PATCH v1 18/57] xen/riscv: introduce asm/random.h
[PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
[PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
[PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
[PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
[PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
[PATCH v1 39/57] xen/riscv: introduce asm/numa.h
[PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
- xen/lib.h in most of the cases were changed to xen/bug.h as
mostly functionilty of bug.h is used.
- align arch-riscv.h with Arm's version of it.
- change the Author of commit with introduction of asm/atomic.h.
- update some definition from spinlock.h.
- code style changes.
---

Oleksii Kurochko (10):
xen: introduce generic non-atomic test_*bit() xen/riscv: introduce bitops.h
xen/riscv: add minimal stuff to mm.h to build full Xen xen/riscv: add minimal amount of stubs to build full Xen xen/riscv: enable full Xen build
xen/README: add compiler and binutils versions for RISC-V64 xen/common: fix build issue for common/trace.c xen/riscv: change .insn to .byte in cpu_relax() xen/riscv: introduce ANDN_INSN
xen/x86: drop constanst_test_bit() in asm/bitops.h

 README                                 |   3 +
 xen/arch/arm/include/asm/bitops.h      |  69 ----
 xen/arch/ppc/include/asm/bitops.h      |  54 ----
 xen/arch/ppc/include/asm/page.h        |   2 +-
 xen/arch/ppc/mm-radix.c                |   2 +-
 xen/arch/riscv/Makefile                |  17 +-
 xen/arch/riscv/arch.mk                 |   4 -
 xen/arch/riscv/early_printk.c          | 168 ----------
 xen/arch/riscv/include/asm/bitops.h    | 137 ++++++++
 xen/arch/riscv/include/asm/cmpxchg.h   |  16 +-
 xen/arch/riscv/include/asm/mm.h        | 235 ++++++++++++++
 xen/arch/riscv/include/asm/processor.h |   2 +-
 xen/arch/riscv/mm.c                    |  43 ++-
 xen/arch/riscv/setup.c                 |  10 +-
 xen/arch/riscv/stubs.c                 | 418 +++++++++++++++++++++++++
 xen/arch/riscv/traps.c                 |  25 ++
 xen/arch/x86/include/asm/bitops.h      |  39 +--
 xen/common/trace.c                     |   1 +
 xen/include/xen/bitops.h               | 182 +++++++++++
 19 files changed, 1096 insertions(+), 331 deletions(-)
 create mode 100644 xen/arch/riscv/include/asm/bitops.h
 create mode 100644 xen/arch/riscv/stubs.c
Signature · 1 line
2.45.2
#13 lore raw

PATCH v13 4/10 xen/riscv: add minimal amount of stubs to build full Xen

---
Changes in V13:
- drop no_irq_type because of the patch series ( [PATCH for-4.19 0/2] arch/irq: Untangle no_irq_type ) which was merged to staging.
- Drop unnessary stubs after rebasing on top of staging ( [PATCH for-4.19 0/2] arch/irq: Untangle no_irq_type )
- implement get_upper_mfn_bound() as BUG_ON("unimplemented") to not introduce max_page which will be dropped in the next patch ( xen/riscv: enable full Xen build )
- remove trainling space in stubs.c after the comment /* guest_access.h */
- drop frametable_base_pdx
- drop frametable_virt_end
---
Changes in V7-V12:
- Only rebase was done.
---
Changes in V6:
- update the commit in stubs.c around /* ... common/irq.c ... */
- add Acked-by: Jan Beulich <[email protected]>
---
Changes in V5:
- drop unrelated changes
- assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
- added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die() from smpboot.c
- back changes related to printk() in early_printk() as they should be removed in the next patch to avoid compilation error.
- update definition of cpu_khz: __read_mostly -> __ro_after_init.
- drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
- move vm_event_*() functions from stubs.c to riscv/vm_event.c.
- s/BUG/BUG_ON("unimplemented") in stubs.c
- back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment, so this function are needed to avoid compilation error.
- defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will be compiled.
---
Changes in V3:
- code style fixes.
- update attribute for frametable_base_pdx and frametable_virt_end to __ro_after_init. insteaf of read_mostly.
- use BUG() instead of assert_failed/WARN for newly introduced stubs.
- drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
- drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
- define udelay stub
- remove 'select HAS_PDX' from RISC-V Kconfig because of
https://lore.kernel.org/xen-devel/[email protected]/

---
 xen/arch/riscv/Makefile |   1 +
 xen/arch/riscv/mm.c     |  41 ++++
 xen/arch/riscv/setup.c  |   8 +
 xen/arch/riscv/stubs.c  | 418 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/riscv/traps.c  |  25 +++
 5 files changed, 493 insertions(+)
 create mode 100644 xen/arch/riscv/stubs.c
xen/arch/riscv/Makefile +1 -0
index 1ed1a8369b..60afbc0ad9 100644
@@ -4,6 +4,7 @@ obj-y += mm.o
 obj-$(CONFIG_RISCV_64) += riscv64/
 obj-y += sbi.o
 obj-y += setup.o
+obj-y += stubs.o
 obj-y += traps.o
 obj-y += vm_event.o
 
xen/arch/riscv/mm.c +41 -0

49 lines not on this page · raw message

xen/arch/riscv/setup.c +8 -0
index 98a94c4c48..8bb5bdb2ae 100644
@@ -1,11 +1,19 @@
 /* SPDX-License-Identifier: GPL-2.0-only */
 
+#include <xen/bug.h>
 #include <xen/compile.h>
 #include <xen/init.h>
 #include <xen/mm.h>
 
+#include <public/version.h>
+
 #include <asm/early_printk.h>
 
+void arch_get_xen_caps(xen_capabilities_info_t *info)
+{
+    BUG_ON("unimplemented");
+}
+
 /* Xen stack for bringing up the first CPU. */
 unsigned char __initdata cpu0_boot_stack[STACK_SIZE]
     __aligned(STACK_SIZE);
xen/arch/riscv/stubs.c new file +418 -0

418 lines not on this page · raw message

xen/arch/riscv/traps.c +25 -0
index ccd3593f5a..5415cf8d90 100644
@@ -4,6 +4,10 @@
  *
  * RISC-V Trap handlers
  */
+
+#include <xen/lib.h>
+#include <xen/sched.h>
+
 #include <asm/processor.h>
 #include <asm/traps.h>
 
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs)
 {
     die();
 }
+
+void vcpu_show_execution_state(struct vcpu *v)
+{
+    BUG_ON("unimplemented");
+}
+
+void show_execution_state(const struct cpu_user_regs *regs)
+{
+    printk("implement show_execution_state(regs)\n");
+}
+
+void arch_hypercall_tasklet_result(struct vcpu *v, long res)
+{
+    BUG_ON("unimplemented");
+}
+
+enum mc_disposition arch_do_multicall_call(struct mc_state *state)
+{
+    BUG_ON("unimplemented");
+    return mc_continue;
+}
Signature · 1 line
2.45.2
#17 lore raw

PATCH v13 6/10 xen/README: add compiler and binutils versions for RISC-V64

In reply to Oleksii Kurochko

This patch doesn't represent a strict lower bound for GCC and GNU Binutils; rather, these versions are specifically employed by the Xen RISC-V container and are anticipated to undergo continuous testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.

While it is feasible to utilize Clang, it's important to note that, currently, there is no Xen RISC-V CI job in place to verify the seamless functioning of the build with Clang.

--
Changes in V13:
- drop the line "Older GCC and GNU Binutils would work, but this is not a guarantee." in README
- add Reviewed-by: Andrew Cooper <[email protected]>
--
Changes in V5-V12:
- Nothing changed. Only rebase.
---
Changes in V6:
- update the message in README.
---
Changes in V5:
- update the commit message and README file with additional explanation about GCC and GNU Binutils version. Additionally, it was added information about Clang.
---
Changes in V4:
- Update version of GCC (12.2) and GNU Binutils (2.39) to the version which are in Xen's contrainter for RISC-V
---
Changes in V3:
- new patch

---
 README | 3 +++
 1 file changed, 3 insertions(+)
README +3 -0
index c8a108449e..9a898125e1 100644
@@ -48,6 +48,9 @@ provided by your OS distributor:
       - For ARM 64-bit:
         - GCC 5.1 or later
         - GNU Binutils 2.24 or later
+      - For RISC-V 64-bit:
+        - GCC 12.2 or later
+        - GNU Binutils 2.39 or later
     * POSIX compatible awk
     * Development install of zlib (e.g., zlib-dev)
     * Development install of Python 2.7 or later (e.g., python-dev)
Signature · 1 line
2.45.2
#18 lore raw

PATCH v13 7/10 xen/common: fix build issue for common/trace.c

In reply to Oleksii Kurochko

During Gitlab CI randconfig job for RISC-V failed witn an error:
common/trace.c:57:22: error: expected '=', ',', ';', 'asm' or

                              '__attribute__' before '__read_mostly'

57 | static u32 data_size __read_mostly;

---
Changes in V13:
- new patch

---
 xen/common/trace.c | 1 +
 1 file changed, 1 insertion(+)
xen/common/trace.c +1 -0
index c9c555094b..c33f115b6c 100644
@@ -29,6 +29,7 @@
 #include <xen/mm.h>
 #include <xen/percpu.h>
 #include <xen/pfn.h>
+#include <xen/sections.h>
 #include <xen/cpu.h>
 #include <asm/atomic.h>
 #include <public/sysctl.h>
Signature · 1 line
2.45.2
#27 lore raw

PATCH v13 9/10 xen/riscv: introduce ANDN_INSN

In reply to Oleksii Kurochko

RISC-V does a conditional toolchain for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().

Fixes 51dabd6312c ("xen/riscv: introduce cmpxchg.h")

---
Changes in V13:
- new patch

---
 xen/arch/riscv/include/asm/cmpxchg.h | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
xen/arch/riscv/include/asm/cmpxchg.h +15 -1
index d5e678c036..12278be577 100644
@@ -18,6 +18,20 @@
         : "r" (new) \
         : "memory" );
 
+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form
+ * it of a NOT+AND pair
+ */
+#ifdef __riscv_zbb
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "andn " rd ", " rs1 ", " rs2 "\n"
+#else
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "not " rd ", " rs2 "\n"                     \
+    "and " rd ", " rs1 ", " rd "\n"
+#endif
+
 /*
  * For LR and SC, the A extension requires that the address held in rs1 be
  * naturally aligned to the size of the operand (i.e., eight-byte aligned
@@ -48,7 +62,7 @@
     \
     asm volatile ( \
         "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
-        "   andn  %[scratch], %[old], %[mask]\n" \
+        ANDN_INSN("%[scratch]", "%[old]", "%[mask]") \
         "   or   %[scratch], %[scratch], %z[new_]\n" \
         "   sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
         "   bnez %[scratch], 0b\n" \
Signature · 1 line
2.45.2
#36 lore raw

PATCH v13 10/10 xen/x86: drop constanst_test_bit() in asm/bitops.h

In reply to Oleksii Kurochko

constant_test_bit() is functionally the same as generic_test_bit(), so constant_test_bit() can be dropped and replaced with generic_test_bit().

---
Changes in V13:
- new patch ( this patch is dependent on
xen: introduce generic non-atomic test_*bit() )

---
 xen/arch/x86/include/asm/bitops.h | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
xen/arch/x86/include/asm/bitops.h +1 -7
index f9aa60111f..8c0403405a 100644
@@ -277,12 +277,6 @@ static inline int test_and_change_bit(int nr, volatile void *addr)
     test_and_change_bit(nr, addr);                      \
 })
 
-static inline int constant_test_bit(int nr, const volatile void *addr)
-{
-    return ((1U << (nr & 31)) &
-            (((const volatile unsigned int *)addr)[nr >> 5])) != 0;
-}
-
 static inline int variable_test_bit(int nr, const volatile void *addr)
 {
     int oldbit;
@@ -297,7 +291,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)
 
 #define arch_test_bit(nr, addr) ({                      \
     __builtin_constant_p(nr) ?                          \
-        constant_test_bit(nr, addr) :                   \
+        generic_test_bit(nr, addr) :                    \
         variable_test_bit(nr, addr);                    \
 })
 
Signature · 1 line
2.45.2
#11 lore raw

Re: [PATCH v13 03/10] xen/riscv: add minimal stuff to mm.h to build full Xen

In reply to Oleksii Kurochko
Quoted 5 lines · Oleksii Kurochko

---
Changes in V13:
- redefine mfn_to_page() and mfn_to_page().

DYM page_to_mfn() here as one of the two?

xen/arch/riscv/include/asm/mm.h line 112

Quoted 3 lines
xen/arch/riscv/include/asm/mm.h +3 -0
+/* Convert between machine frame numbers and page-info structures. */
+#define mfn_to_page(mfn)    (frame_table + mfn_x(mfn))
+#define page_to_mfn(pg)     _mfn((unsigned long)((pg) - frame_table))

Is the cast really needed here?

Jan

Acked-by on v13 7/10
#19 lore raw

Re: [PATCH v13 07/10] xen/common: fix build issue for common/trace.c

In reply to Oleksii Kurochko
Quoted 6 lines · Oleksii Kurochko

During Gitlab CI randconfig job for RISC-V failed witn an error:
common/trace.c:57:22: error: expected '=', ',', ';', 'asm' or

                              '__attribute__' before '__read_mostly'

57 | static u32 data_size __read_mostly;

If you give a release-ack, this can go in right away, I think.

Jan

#24 lore raw

Re: [PATCH v13 08/10] xen/riscv: change .insn to .byte in cpu_relax()

In reply to Oleksii Kurochko
Quoted 2 lines · Oleksii Kurochko

The .insn directive appears to check that the byte pattern is a known extension, where .4byte doesn't.

Support for specifying "raw" insns was added only in 2.38. Moving away from .insn has other downsides (which may or may not matter here, but that would want discussing). But: Do we really need to move away? Can't you use .insn with operands that the older gas supports, e.g.

	.insn r MISC_MEM, 0, 0, x0, x0, x16

? I'm sorry, the oldest RISC-V gas I have to hand is 2.39, so I couldn't double check that 2.35 would grok this. From checking sources it should, though.

Quoted 6 lines

The following compilation error occurs:
./arch/riscv/include/asm/processor.h: Assembler messages:
./arch/riscv/include/asm/processor.h:70: Error: unrecognized opcode `0x0100000F'
In case of the following Binutils:

  $ riscv64-linux-gnu-as --version
  GNU assembler (GNU Binutils for Debian) 2.35.2

In patch 6 you say 2.39. Why is 2.35.2 suddenly becoming of interest?

xen/arch/riscv/include/asm/processor.h static inline void cpu_relax(void) · line 71

Quoted 9 lines
xen/arch/riscv/include/asm/processor.h +1 -1
@@ -67,7 +67,7 @@ static inline void cpu_relax(void)
     __asm__ __volatile__ ( "pause" );
 #else
     /* Encoding of the pause instruction */
-    __asm__ __volatile__ ( ".insn 0x0100000F" );
+    __asm__ __volatile__ ( ".byte 0x0100000F" );
 #endif

In the description you (correctly) say .4byte; why is it .byte here? Does this build at all?

Jan

#28 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Oleksii Kurochko

xen/arch/riscv/include/asm/cmpxchg.h line 23

Quoted 9 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +3 -0
@@ -18,6 +18,20 @@
         : "r" (new) \
         : "memory" );
 
+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest? Plus, why would age of the tool chain matter? What you care about is whether you're permitted to use the extension at runtime. Otherwise you could again ...

Also something went wrong with line wrapping here.

xen/arch/riscv/include/asm/cmpxchg.h line 32

Quoted 9 lines
xen/arch/riscv/include/asm/cmpxchg.h +9 -0
+ * it of a NOT+AND pair
+ */
+#ifdef __riscv_zbb
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "andn " rd ", " rs1 ", " rs2 "\n"
+#else
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "not " rd ", " rs2 "\n"                     \
+    "and " rd ", " rs1 ", " rd "\n"

... resort to .insn.

Jan

Acked-by on v13 10/10
#37 lore raw

Re: [PATCH v13 10/10] xen/x86: drop constanst_test_bit() in asm/bitops.h

In reply to Oleksii Kurochko
Quoted 5 lines · Oleksii Kurochko

constant_test_bit() is functionally the same as generic_test_bit(), so constant_test_bit() can be dropped and replaced with generic_test_bit().

(Didn't I ask for this to be done, so perhaps also Requested-by or alike?)

author
#12 lore raw

Re: [PATCH v13 03/10] xen/riscv: add minimal stuff to mm.h to build full Xen

In reply to Jan Beulich
Quoted 8 lines · Jan Beulich
Quoted 5 lines · Oleksii Kurochko

---
Changes in V13:
 - redefine mfn_to_page() and mfn_to_page().

DYM page_to_mfn() here as one of the two?

Yes, one of them is page_to_mfn(). I will update the comment in next version.

Quoted 8 lines
Quoted 6 lines

+/* Convert between machine frame numbers and page-info structures. */

(quoted diff) +2 -0
+#define mfn_to_page(mfn)    (frame_table + mfn_x(mfn))
+#define page_to_mfn(pg)     _mfn((unsigned long)((pg) -

frame_table))

Is the cast really needed here?

I checked and it can be really dropped.

~ Oleksii

author
#16 lore raw

Re: [PATCH v13 05/10] xen/riscv: enable full Xen build

In reply to Jan Beulich
Quoted 8 lines · Jan Beulich
Quoted 5 lines · Oleksii Kurochko

---
 Changes in V13:
  - implement get_upper_mfn_bound() as BUG_ON("unimplemented")

Odd, patch 4 also says this and also does so.

It should me mentioned only in patch 4. I will drop this line in the next patch version if this patch won't be merged.

~ Oleksii

author
#20 lore raw

Re: [PATCH v13 07/10] xen/common: fix build issue for common/trace.c

In reply to Jan Beulich
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko

During Gitlab CI randconfig job for RISC-V failed witn an error:
 common/trace.c:57:22: error: expected '=', ',', ';', 'asm' or '__attribute__' before '__read_mostly'
   57 | static u32 data_size __read_mostly;

Quoted 5 lines
Quoted 1 line

If you give a release-ack, this can go in right away, I think.

~ Oleksii

author
#38 lore raw

Re: [PATCH v13 10/10] xen/x86: drop constanst_test_bit() in asm/bitops.h

In reply to Jan Beulich
Quoted 10 lines · Jan Beulich
Quoted 5 lines · Oleksii Kurochko

constant_test_bit() is functionally the same as generic_test_bit(), so constant_test_bit() can be dropped and replaced with generic_test_bit().

(Didn't I ask for this to be done, so perhaps also Requested-by or alike?)

Yes, it was you, sorry.
For sure, we should added Requested-by. ( I had cherry-picking from other my branch where I didn't add Requested-by and just not double- check before send to xen-devel )

Could you please add Requested-by during the merge? Thank in advance!

~ Oleksii

author
#25 lore raw

Re: [PATCH v13 08/10] xen/riscv: change .insn to .byte in cpu_relax()

In reply to Jan Beulich
Quoted 12 lines · Jan Beulich
Quoted 3 lines · Oleksii Kurochko

The .insn directive appears to check that the byte pattern is a known
extension, where .4byte doesn't.

Support for specifying "raw" insns was added only in 2.38. Moving away
from .insn has other downsides (which may or may not matter here, but that would want discussing). But: Do we really need to move away? Can't
you use .insn with operands that the older gas supports, e.g.

Quoted 7 lines
	.insn r MISC_MEM, 0, 0, x0, x0, x16

? I'm sorry, the oldest RISC-V gas I have to hand is 2.39, so I couldn't
double check that 2.35 would grok this. From checking sources it should,
though.

We can do in this way too. I checked on https://godbolt.org/ even with RISC-V ( 64 bits ) gcc 8.2.0 the suggested option with .insn compiles well.

Quoted 10 lines
Quoted 8 lines

The following compilation error occurs:
  ./arch/riscv/include/asm/processor.h: Assembler messages:
  ./arch/riscv/include/asm/processor.h:70: Error: unrecognized opcode `0x0100000F'
In case of the following Binutils:

  $ riscv64-linux-gnu-as --version
  GNU assembler (GNU Binutils for Debian) 2.35.2

In patch 6 you say 2.39. Why is 2.35.2 suddenly becoming of interest?

Andrew has (or had) an older version of binutils and started to face the issues mentioned in this patch and the next one. As a result, some changes were suggested.

The version in the README wasn't changed because, in my opinion, this requires a separate CI job with a prebuilt or fixed toolchain version. At the moment, it is supported only the version mentioned in README and that one I have on my machine.

Quoted 13 lines

xen/arch/riscv/include/asm/processor.h static inline void cpu_relax(void) · line 71

Quoted 10 lines
xen/arch/riscv/include/asm/processor.h +1 -1
@@ -67,7 +67,7 @@ static inline void cpu_relax(void)
      __asm__ __volatile__ ( "pause" );
  #else
      /* Encoding of the pause instruction */
-    __asm__ __volatile__ ( ".insn 0x0100000F" );
+    __asm__ __volatile__ ( ".byte 0x0100000F" );
  #endif

In the description you (correctly) say .4byte; why is it .byte here? Does this build at all?

Yes, it is a typo. Should be .4byte and it is built, but with warning:

	value 0x100000f truncated to 0xf

~ Oleksii

author
#29 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Jan Beulich
Quoted 13 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
          : "r" (new) \
          : "memory" );


+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

Andrew has (or had) an older version of binutils and started to face the issues mentioned in this patch. As a result, some changes were suggested.

Quoted 2 lines

Plus, why would age of the
tool chain matter?

As it is mentioned in the comment binutils < 2.37 doesn't understand andn instruction.

Quoted 2 lines

What you care about is whether you're permitted to use the extension at runtime.

At the moment we can't check that at runtime, w/o having an exception, ( there is some option to check when dts parsing will be available in Xen ). I will add the check when dts parsing functionality will be available. Right now the best what we can do it is just mentioned that as requirement in docs.

Quoted 4 lines

Otherwise you could again ...

Also something went wrong with line wrapping here.

Quoted 11 lines

xen/arch/riscv/include/asm/cmpxchg.h line 32

Quoted 9 lines

+ * it of a NOT+AND pair
+ */

xen/arch/riscv/include/asm/cmpxchg.h +7 -0
+#ifdef __riscv_zbb
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "andn " rd ", " rs1 ", " rs2 "\n"
+#else
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "not " rd ", " rs2 "\n"                     \
+    "and " rd ", " rs1 ", " rd "\n"

... resort to .insn.

Hmm, good point, it could be an issue.

If this patch is still needed (Andrew, have you updated your toolchain?), then it should use .insn instead of andn. However, using .insn requires encoding rd, rs1, and rs2 through asm ("some_reg") (?), which seems overly complicated. And still, it is an open question if in reality someone will use binutils which doesn't support andn instruction...

~ Oleksii

#31 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Jan Beulich
Quoted 11 lines · Jan Beulich

xen/arch/riscv/include/asm/cmpxchg.h line 23

Quoted 9 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +3 -0
@@ -18,6 +18,20 @@
         : "r" (new) \
         : "memory" );
 
+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

You deleted the commit message which explains why:

Quoted 3 lines

RISC-V does a conditional toolchain test for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().

Either Zbb needs to be mandatory (both in the toolchain and the board running Xen), or emulate_xchg_1_2() needs to not use the ANDN instruction.

I opted for the latter.

~Andrew

author
#32 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

Quoted 16 lines · Andrew Cooper
Quoted 12 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
          : "r" (new) \
          : "memory" );


+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

You deleted the commit message which explains why:

Quoted 7 lines
Quoted 3 lines

RISC-V does a conditional toolchain test for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().

Either Zbb needs to be mandatory (both in the toolchain and the board running Xen), or emulate_xchg_1_2() needs to not use the ANDN instruction.

But we can't go without Zbb there are some things in <xen/bitops.h> which now requires this extension. At the current state of development it is mandatory.

~ Oleksii

Quoted 4 lines

I opted for the latter.

~Andrew

#34 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Andrew Cooper
Quoted 8 lines · Andrew Cooper
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
         : "r" (new) \
         : "memory" );
 
Quoted 6 lines
Quoted 4 lines

xen/arch/riscv/include/asm/cmpxchg.h line 23

Quoted 3 lines
xen/arch/riscv/include/asm/cmpxchg.h +3 -0
+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

You deleted the commit message which explains why:

Not really. My whole point is that while the intention of the change looks okay, description and comment describe things insufficiently, to say the least.

Quoted 8 lines
Quoted 3 lines

RISC-V does a conditional toolchain test for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().

Either Zbb needs to be mandatory (both in the toolchain and the board running Xen), or emulate_xchg_1_2() needs to not use the ANDN instruction.

I opted for the latter.

And I agree with that.

Jan

#33 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Oleksii
Quoted 21 lines · Oleksii
Quoted 20 lines · Andrew Cooper
Quoted 12 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
          : "r" (new) \
          : "memory" );


+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

You deleted the commit message which explains why:

Quoted 3 lines

RISC-V does a conditional toolchain test for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().

Quoted 6 lines
Quoted 3 lines

Either Zbb needs to be mandatory (both in the toolchain and the board running Xen), or emulate_xchg_1_2() needs to not use the ANDN instruction.

But we can't go without Zbb there are some things in <xen/bitops.h> which now requires this extension. At the current state of development it is mandatory.

Maybe that's the "another bug" that Andrew mentioned (without any details) on Matrix?

Jan

#30 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Oleksii
Quoted 14 lines · Oleksii
Quoted 13 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
          : "r" (new) \
          : "memory" );


+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

Quoted 8 lines

Andrew has (or had) an older version of binutils and started to face the issues mentioned in this patch. As a result, some changes were suggested.

Quoted 2 lines

Plus, why would age of the
tool chain matter?

As it is mentioned in the comment binutils < 2.37 doesn't understand andn instruction.

But that's not the point. If the tool chain is too old, our logic to detect that should arrange for __riscv_zbb to not be set. That's all that needed to cover gas not understanding the insn. The rest here isn't about the capabilities of the tool chain: Either we make Zbb a requirement (at which point .insn can be used to encode ANDN), or we don't (at which point the replacement code you have comes into play).

Quoted 23 lines
Quoted 2 lines

What you care about is whether you're permitted to use the extension at runtime.

At the moment we can't check that at runtime, w/o having an exception, ( there is some option to check when dts parsing will be available in Xen ). I will add the check when dts parsing functionality will be available. Right now the best what we can do it is just mentioned that as requirement in docs.

Quoted 4 lines

Otherwise you could again ...

Also something went wrong with line wrapping here.

Quoted 11 lines

xen/arch/riscv/include/asm/cmpxchg.h line 32

Quoted 9 lines

+ * it of a NOT+AND pair
+ */

xen/arch/riscv/include/asm/cmpxchg.h +7 -0
+#ifdef __riscv_zbb
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "andn " rd ", " rs1 ", " rs2 "\n"
+#else
+#define ANDN_INSN(rd, rs1, rs2)                 \
+    "not " rd ", " rs2 "\n"                     \
+    "and " rd ", " rs1 ", " rd "\n"

... resort to .insn.

Quoted 7 lines

Hmm, good point, it could be an issue.

If this patch is still needed (Andrew, have you updated your toolchain?), then it should use .insn instead of andn. However, using .insn requires encoding rd, rs1, and rs2 through asm ("some_reg") (?), which seems overly complicated.

Why? You don't want to use the raw form of .insn (which, as per the other sub-thread on this series, is available from gas 2.38 only anyway), but the one permitting operands to be spelled out (.insn r ...), along the lines of what I suggested for "pause".

Jan

#26 lore raw

Re: [PATCH v13 08/10] xen/riscv: change .insn to .byte in cpu_relax()

In reply to Oleksii
Quoted 11 lines · Oleksii
Quoted 10 lines · Jan Beulich
Quoted 7 lines · Oleksii Kurochko

The following compilation error occurs:
  ./arch/riscv/include/asm/processor.h: Assembler messages:
  ./arch/riscv/include/asm/processor.h:70: Error: unrecognized opcode `0x0100000F'
In case of the following Binutils:

  $ riscv64-linux-gnu-as --version
  GNU assembler (GNU Binutils for Debian) 2.35.2

In patch 6 you say 2.39. Why is 2.35.2 suddenly becoming of interest?

Quoted 8 lines

Andrew has (or had) an older version of binutils and started to face the issues mentioned in this patch and the next one. As a result, some changes were suggested.

The version in the README wasn't changed because, in my opinion, this requires a separate CI job with a prebuilt or fixed toolchain version. At the moment, it is supported only the version mentioned in README and that one I have on my machine.

So from my perspective, if you go to the lengths of making changes to support anything older than what you put into README, you will want to at least briefly mention why this is needed / wanted.

Plus, as to "separate CI job": That makes little sense to me, or else we'd need to have separate jobs for each and every compiler version out in the world (and within range of what README says). Not just for RISC-V, but also for other architectures. This imo simply wouldn't scale.

Jan

#3 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

In reply to Oleksii Kurochko
Quoted 47 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +44 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could alternatively
+ * implement with local interrupt masking.
+ */
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h line 54

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to come back on that: Up to here all we use is AMO.

And further down there's no asm() anymore. What were you referring to?

Jan

#21 lore raw

Re: [PATCH v13 07/10] xen/common: fix build issue for common/trace.c

In reply to Oleksii
Quoted 8 lines · Oleksii
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko

During Gitlab CI randconfig job for RISC-V failed witn an error:
 common/trace.c:57:22: error: expected '=', ',', ';', 'asm' or '__attribute__' before '__read_mostly'
   57 | static u32 data_size __read_mostly;

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

If you give a release-ack, this can go in right away, I think.

Thanks, but actually I was misled by the subject prefix. From a formal perspective this really wants an ack from George (and mine doesn't count anything at all).

Jan

#4 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

In reply to Jan Beulich
Quoted 64 lines · Jan Beulich
Quoted 48 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +22 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could

alternatively
+ * implement with local interrupt masking.
+ */

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h line 54

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to come back
on that: Up to here all we use is AMO.

Quoted 2 lines

And further down there's no asm() anymore. What were you referring to?

RISC-V doesn't have a CLZ instruction in the base
ISA. As a consequence, __builtin_ffs() emits a library call to ffs() on GCC, or a de Bruijn sequence on Clang.

The optional Zbb extension adds a CLZ instruction, after which __builtin_ffs() emits a very simple sequence.

~ Oleksii

#35 lore raw

Re: [PATCH v13 09/10] xen/riscv: introduce ANDN_INSN

In reply to Jan Beulich
Quoted 27 lines · Jan Beulich
Quoted 14 lines · Andrew Cooper
Quoted 12 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
          : "r" (new) \
          : "memory" );


+/*
+ * Binutils < 2.37 doesn't understand ANDN.  If the toolchain is too
+ld, form

Same question: Why's 2.37 suddenly of interest?

Quoted 1 line

You deleted the commit message which explains why:

Not really. My whole point is that while the intention of the change looks
okay, description and comment describe things insufficiently, to say the
least.

Quoted 4 lines
Quoted 3 lines

RISC-V does a conditional toolchain test for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().

Quoted 8 lines
Quoted 6 lines

Either Zbb needs to be mandatory (both in the toolchain and the board
running Xen), or emulate_xchg_1_2() needs to not use the ANDN instruction.

I opted for the latter.

And I agree with that.

Okay, then I'll update that in the next patch version.

~ Oleksii

#5 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

In reply to [email protected]
Quoted 65 lines · On 26.06.2024 19:27, [email protected] wrote:
Quoted 64 lines · Jan Beulich
Quoted 48 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +22 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California */
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could

alternatively
+ * implement with local interrupt masking.
+ */

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h line 54

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to come back
on that: Up to here all we use is AMO.

Quoted 5 lines
Quoted 2 lines

And further down there's no asm() anymore. What were you referring to?

RISC-V doesn't have a CLZ instruction in the base
ISA. As a consequence, __builtin_ffs() emits a library call to ffs() on GCC,

Oh, so we'd need to implement that libgcc function, along the lines of Arm32 implementing quite a few of them to support shifts on 64-bit quantities as well as division and modulo.

Jan

Quoted 6 lines

or a de Bruijn sequence on Clang.

The optional Zbb extension adds a CLZ instruction, after which __builtin_ffs() emits a very simple sequence.

~ Oleksii

#6 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

Quoted 76 lines · Jan Beulich
Quoted 67 lines · On 26.06.2024 19:27, [email protected] wrote:
Quoted 66 lines · Jan Beulich
Quoted 49 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +2 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California

*/

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could

alternatively
+ * implement with local interrupt masking.
+ */

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h line 54

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to come
back
on that: Up to here all we use is AMO.

Quoted 7 lines
Quoted 3 lines

And further down there's no asm() anymore. What were you referring
to?

RISC-V doesn't have a CLZ instruction in the base ISA.  As a consequence, __builtin_ffs() emits a library call to ffs()
on GCC,

Quoted 4 lines

Oh, so we'd need to implement that libgcc function, along the lines of
Arm32 implementing quite a few of them to support shifts on 64-bit quantities as well as division and modulo.

Why we can't just live with Zbb extension? Zbb extension is presented on every platform I have in access with hypervisor extension support.

~ Oleksii

Quoted 10 lines

Jan

Quoted 6 lines

or a de Bruijn sequence on Clang.

The optional Zbb extension adds a CLZ instruction, after which __builtin_ffs() emits a very simple sequence.

~ Oleksii

#7 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

Quoted 77 lines · On 27.06.2024 11:58, [email protected] wrote:
Quoted 76 lines · Jan Beulich
Quoted 74 lines · On 26.06.2024 19:27, [email protected] wrote:
Quoted 66 lines · Jan Beulich
Quoted 49 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +2 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of California

*/

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could

alternatively
+ * implement with local interrupt masking.
+ */

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h line 54

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to come
back
on that: Up to here all we use is AMO.

Quoted 3 lines

And further down there's no asm() anymore. What were you referring
to?

RISC-V doesn't have a CLZ instruction in the base ISA.  As a consequence, __builtin_ffs() emits a library call to ffs()
on GCC,

Quoted 6 lines
Quoted 4 lines

Oh, so we'd need to implement that libgcc function, along the lines of
Arm32 implementing quite a few of them to support shifts on 64-bit quantities as well as division and modulo.

Why we can't just live with Zbb extension? Zbb extension is presented on every platform I have in access with hypervisor extension support.

I'd be fine that way, but then you don't need to break up ANDN into NOT and AND. It is my understanding that Andrew has concerns here, even if
- iirc - it was him to originally suggest to build upon that extension being available. If these concerns are solely about being able to build with Zbb-unaware tool chains, then what to do about the build issues there has already been said.

Jan

#8 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

Quoted 91 lines · Jan Beulich
Quoted 85 lines · On 27.06.2024 11:58, [email protected] wrote:
Quoted 78 lines · Jan Beulich
Quoted 69 lines · On 26.06.2024 19:27, [email protected] wrote:
Quoted 68 lines · Jan Beulich
Quoted 50 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +2 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of

California
*/

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could

alternatively
+ * implement with local interrupt masking.
+ */

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h line 54

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to
come
back
on that: Up to here all we use is AMO.

Quoted 7 lines
Quoted 3 lines

And further down there's no asm() anymore. What were you referring
to?

RISC-V doesn't have a CLZ instruction in the base ISA.  As a consequence, __builtin_ffs() emits a library call to ffs()
on GCC,

Quoted 6 lines

Oh, so we'd need to implement that libgcc function, along the lines
of
Arm32 implementing quite a few of them to support shifts on 64- bit
quantities as well as division and modulo.

Quoted 4 lines

Why we can't just live with Zbb extension? Zbb extension is presented
on every platform I have in access with hypervisor extension support.

Quoted 10 lines

I'd be fine that way, but then you don't need to break up ANDN into NOT
and AND. It is my understanding that Andrew has concerns here, even if
- iirc - it was him to originally suggest to build upon that extension
being available. If these concerns are solely about being able to build
with Zbb-unaware tool chains, then what to do about the build issues there has already been said.

Not much we can do except probably using .insn, as you suggested for the "pause" instruction in cpu_relax(), for every instruction ( at the moment it is only ANDB but who knows which instruction will be used in the future ) from the Zbb extension.

But then we will need to do the same for each possible extension we are going to use, as there is still a small chance that we might encounter an extension-unaware toolchain.

I am a little bit confused about what we should do.

In my opinion, the best approach at the moment is to use .insn for the ANDN and PAUSE instructions and add an explanation to docs/misc/riscv/booting.txt or create a separate document where such issues are documented (I am not sure that README is the correct place for this).

I am also okay to go with ANDN break up int NOT and AND, but it is needed to come up with concept which instruction/extenstion should be used and how consistently to deal with such situations.

Furthermore, I don't think these changes should block the merging ( doesn't matter in 4.19 or in 4.20 Xen release ) of PATCHes v13 01-07 of this patch series.

~ Oleksii

#39 lore raw
In reply to Oleksii Kurochko

I saw a message in the xen-devel channel:```
erm... We've got a problem.
https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7185148004
very clearly failed with a panic(), but reported success out to Gitlab ```
However, I couldn't determine if this is related to the patches in this series or to patches that were merged earlier.

I would like to understand what needs to be done to be sure that this patch series could be merged.

Thanks in advance.

Best regards,
Oleksii

Quoted 269 lines · Oleksii Kurochko

***
I think that I need to write a separate email requesting approval for this patch
series to be included in Xen 4.19. Most of the patches are RISC-V specific,
so there is a low risk of breaking anything else.
There is only one patch that touches common or arch-specific files, and it
doesn't introduce any functional changes.
Since I can't approve it myself, I am asking for someone else to do so.
***

This patch series performs all of the additions necessary to drop the build overrides for RISCV and enable the full Xen build. Except in cases
where compatibile implementations already exist (e.g. atomic.h and bitops.h), the newly added definitions are simple.

The patch series is based on the following patch series:
 - [PATCH 3/3] xen/ppc: Avoid using the legacy __read_mostly/__ro_after_init definitions [1]

The link to the branch with rebased patches on top of [1] could be found here:

https://gitlab.com/xen-project/people/olkur/xen/-/commits/riscv-full-xen-build-v13

[1]
https://lore.kernel.org/xen-devel/[email protected]/

---
Changes in V13:
 - Patch was merged to staging:
   - [PATCH v12 ] xen/riscv: Update Kconfig in preparation for a full
 - Rebase on top of the current staging.
 - Update cover letter message
 - It was added 2 new patches ( patches 8 and 9 ) which are not necessary for testing environment we
   have in CI but they improves compatability with older gcc and binutils.
 - It was added patch 10 as a clean up of [PATCH v12 2/8] xen:
introduce generic non-atomic test_*bit()
   for x86.
 - Drop [PATCH v12 4/8] xen/riscv: add definition of __read_mostly as it was defined as generic now
   for all architectures.
 - It was added the patch [PATCH v13 07/10] xen/common: fix build issue for common/trace.c
   to resolve compilation issue for RISC-V after reabsing on top of current staging.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V12:
 - Rebase the patch series on top of [1] mentioned above.
 - Update the cover letter message.
 - "[PATCH v11 3/9] xen/bitops: implement fls{l}() in common logic" was droped
   as it is a part of patch series [1] mentioned above.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V11:
  - Patch was merged to staging:
    - [PATCH v10 05/14] xen/riscv: introduce cmpxchg.h
  - [PATCH v10 06/14] xen/riscv: introduce atomic.h
  - [PATCH v10 07/14] xen/riscv: introduce monitor.h
  - [PATCH v10 09/14] xen/riscv: add required things to
current.h
  - [PATCH v10 11/14] xen/riscv: introduce vm_event_*()
functions
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V10:
  - Patch was merged to staging:
    - [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V9:
 - Patch was merged to staging:
    - [PATCH v8 07/17] xen/riscv: introduce io.h
   - [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V8:
 - Patch was merged to staging:
    - [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
    - [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
 - Update the commit message:
     - drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
     - Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
 - Patch was merged to staging:
   [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
 - Other changes are specific to specific patches. Please look at changes for
   specific patch.
---
Changes in V6:
 - Update the cover letter message: drop already merged dependecies and add
   a new one.
 - Patches were merged to staging:
   - [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was
     merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
   - [PATCH v5 03/23] xen/riscv: introduce nospec.h
   - [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
 - Introduce new patches:
   - xen/riscv: introduce extenstion support check by compiler
   - xen/bitops: put __ffs() and ffz() into linux compatible header
   - xen/bitops: implement fls{l}() in common logic
 - The following patches were dropped:
   - drop some patches related to bitops operations as they were introduced in another
     patch series [...]
   - introduce new version for generic __ffs(), ffz() and fls{l}().
 - Merge patch from patch series "[PATCH v9 0/7]  Introduce generic headers" to this patch
   series as only one patch left in the generic headers patch series and it is more about
   RISC-V.
 - Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
 - Update the cover letter as one of the dependencies were merged to staging.
 - Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
 - Moved fence.h to separate patch to deal with some patches dependecies on fence.h
 - Patches were dropped as they were merged to staging:
   * [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
   * [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
   * [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
   * [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
   * [PATCH v4 08/30] xen/riscv: introduce setup.h
   * [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
   * [PATCH v4 11/30] xen/riscv: introduce smp.h
   * [PATCH v4 15/30] xen/riscv: introduce irq.h
   * [PATCH v4 16/30] xen/riscv: introduce p2m.h
   * [PATCH v4 17/30] xen/riscv: introduce regs.h
   * [PATCH v4 18/30] xen/riscv: introduce time.h
   * [PATCH v4 19/30] xen/riscv: introduce event.h
   * [PATCH v4 22/30] xen/riscv: define an address of frame table
 - Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
 - Update the cover letter message: new patch series dependencies.
 - Some patches were merged to staging, so they were dropped in this patch series:
     [PATCH v3 09/34] xen/riscv: introduce system.h
     [PATCH v3 18/34] xen/riscv: introduce domain.h
     [PATCH v3 19/34] xen/riscv: introduce guest_access.h
 - Was sent out of this patch series:
     [PATCH v3 16/34] xen/lib: introduce generic find next bit operations
 - [PATCH v3 17/34] xen/riscv: add compilation of generic find-next- bit.c was
   droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
 - All other changes are specific to a specific patch.
---
Changes in V3:
 - Update the cover letter message
 - The following patches were dropped as they were merged to staging:
    [PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h [PATCH v2 04/39] xen/riscv: add public arch-riscv.h [PATCH v2 05/39] xen/riscv: introduce spinlock.h [PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S [PATCH v2 34/39] xen: add RISCV support for pmu.h [PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
 - Instead of the following patches were introduced new:
    [PATCH v2 10/39] xen/riscv: introduce asm/iommu.h [PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
 - remove "asm/"  for commit messages which start with "xen/riscv:"
 - code style updates.
 - add emulation of {cmp}xchg_* for 1 and 2 bytes types.
 - code style fixes.
 - add SPDX and footer for the newly added headers.
 - introduce generic find-next-bit.c.
 - some other mionor changes. ( details please find in a patch )
---
Changes in V2:
  - Drop the following patches as they are the part of [2]:
      [PATCH v1 06/57] xen/riscv: introduce paging.h [PATCH v1 08/57] xen/riscv: introduce asm/device.h [PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h [PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h [PATCH v1 13/57] xen/riscv: introduce asm/iocap.h [PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h [PATCH v1 18/57] xen/riscv: introduce asm/random.h [PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h [PATCH v1 24/57] xen/riscv: introduce asm/percpu.h [PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h [PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h [PATCH v1 38/57] xen/riscv: introduce asm/monitor.h [PATCH v1 39/57] xen/riscv: introduce asm/numa.h [PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
  - xen/lib.h in most of the cases were changed to xen/bug.h as mostly functionilty of bug.h is used.
  - align arch-riscv.h with Arm's version of it.
  - change the Author of commit with introduction of asm/atomic.h.
  - update some definition from spinlock.h.
  - code style changes.
---

Oleksii Kurochko (10):
  xen: introduce generic non-atomic test_*bit() xen/riscv: introduce bitops.h
  xen/riscv: add minimal stuff to mm.h to build full Xen xen/riscv: add minimal amount of stubs to build full Xen xen/riscv: enable full Xen build
  xen/README: add compiler and binutils versions for RISC-V64 xen/common: fix build issue for common/trace.c xen/riscv: change .insn to .byte in cpu_relax() xen/riscv: introduce ANDN_INSN
  xen/x86: drop constanst_test_bit() in asm/bitops.h

 README                                 |   3 + xen/arch/arm/include/asm/bitops.h      |  69 ---- xen/arch/ppc/include/asm/bitops.h      |  54 ---- xen/arch/ppc/include/asm/page.h        |   2 +- xen/arch/ppc/mm-radix.c                |   2 +- xen/arch/riscv/Makefile                |  17 +- xen/arch/riscv/arch.mk                 |   4 - xen/arch/riscv/early_printk.c          | 168 ---------- xen/arch/riscv/include/asm/bitops.h    | 137 ++++++++ xen/arch/riscv/include/asm/cmpxchg.h   |  16 +- xen/arch/riscv/include/asm/mm.h        | 235 ++++++++++++++ xen/arch/riscv/include/asm/processor.h |   2 +- xen/arch/riscv/mm.c                    |  43 ++- xen/arch/riscv/setup.c                 |  10 +- xen/arch/riscv/stubs.c                 | 418
+++++++++++++++++++++++++
 xen/arch/riscv/traps.c                 |  25 ++ xen/arch/x86/include/asm/bitops.h      |  39 +-- xen/common/trace.c                     |   1 + xen/include/xen/bitops.h               | 182 +++++++++++

 19 files changed, 1096 insertions(+), 331 deletions(-)

 create mode 100644 xen/arch/riscv/include/asm/bitops.h create mode 100644 xen/arch/riscv/stubs.c

#9 lore raw

Re: [PATCH v13 02/10] xen/riscv: introduce bitops.h

In reply to [email protected]
Quoted 107 lines · On 27.06.2024 14:01, [email protected] wrote:
Quoted 91 lines · Jan Beulich
Quoted 85 lines · On 27.06.2024 11:58, [email protected] wrote:
Quoted 78 lines · Jan Beulich
Quoted 69 lines · On 26.06.2024 19:27, [email protected] wrote:
Quoted 68 lines · Jan Beulich
Quoted 50 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +2 -0
@@ -0,0 +1,137 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2012 Regents of the University of

California
*/

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+
+#ifndef _ASM_RISCV_BITOPS_H
+#define _ASM_RISCV_BITOPS_H
+
+#include <asm/system.h>
+
+#if BITOP_BITS_PER_WORD == 64
+#define __AMO(op)   "amo" #op ".d"
+#elif BITOP_BITS_PER_WORD == 32
+#define __AMO(op)   "amo" #op ".w"
+#else
+#error "Unexpected BITOP_BITS_PER_WORD"
+#endif
+
+/* Based on linux/arch/include/asm/bitops.h */
+
+/*
+ * Non-atomic bit manipulation.
+ *
+ * Implemented using atomics to be interrupt safe. Could

alternatively
+ * implement with local interrupt masking.
+ */

xen/arch/riscv/include/asm/bitops.h new file +20 -0
+#define __set_bit(n, p)      set_bit(n, p)
+#define __clear_bit(n, p)    clear_bit(n, p)
+
+#define test_and_op_bit_ord(op, mod, nr, addr, ord)     \
+({                                                      \
+    bitop_uint_t res, mask;                             \
+    mask = BITOP_MASK(nr);                              \
+    asm volatile (                                      \
+        __AMO(op) #ord " %0, %2, %1"                    \
+        : "=r" (res), "+A" (addr[BITOP_WORD(nr)])       \
+        : "r" (mod(mask))                               \
+        : "memory");                                    \
+    ((res & mask) != 0);                                \
+})
+
+#define op_bit_ord(op, mod, nr, addr, ord)      \
+    asm volatile (                              \
+        __AMO(op) #ord " zero, %1, %0"          \
+        : "+A" (addr[BITOP_WORD(nr)])           \
+        : "r" (mod(BITOP_MASK(nr)))             \

xen/arch/riscv/include/asm/bitops.h

Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10 -0
+        : "memory");
+
+#define test_and_op_bit(op, mod, nr, addr)    \
+    test_and_op_bit_ord(op, mod, nr, addr, .aqrl)
+#define op_bit(op, mod, nr, addr) \
+    op_bit_ord(op, mod, nr, addr, )
+
+/* Bitmask modifiers */
+#define NOP(x)    (x)
+#define NOT(x)    (~(x))

Since elsewhere you said we would use Zbb in bitops, I wanted to
come
back
on that: Up to here all we use is AMO.

Quoted 7 lines
Quoted 3 lines

And further down there's no asm() anymore. What were you referring
to?

RISC-V doesn't have a CLZ instruction in the base ISA.  As a consequence, __builtin_ffs() emits a library call to ffs()
on GCC,

Quoted 6 lines

Oh, so we'd need to implement that libgcc function, along the lines
of
Arm32 implementing quite a few of them to support shifts on 64- bit
quantities as well as division and modulo.

Quoted 4 lines

Why we can't just live with Zbb extension? Zbb extension is presented
on every platform I have in access with hypervisor extension support.

Quoted 10 lines

I'd be fine that way, but then you don't need to break up ANDN into NOT
and AND. It is my understanding that Andrew has concerns here, even if
- iirc - it was him to originally suggest to build upon that extension
being available. If these concerns are solely about being able to build
with Zbb-unaware tool chains, then what to do about the build issues there has already been said.

Not much we can do except probably using .insn, as you suggested for the "pause" instruction in cpu_relax(), for every instruction ( at the moment it is only ANDB but who knows which instruction will be used in the future ) from the Zbb extension.

Quoted 8 lines

But then we will need to do the same for each possible extension we are going to use, as there is still a small chance that we might encounter an extension-unaware toolchain.

I am a little bit confused about what we should do.

In my opinion, the best approach at the moment is to use .insn for the ANDN and PAUSE instructions

This would be my preference, but please also consult with Andrew.

Quoted 5 lines

and add an explanation to
docs/misc/riscv/booting.txt or create a separate document where such issues are documented (I am not sure that README is the correct place for this).

Quoted 7 lines

I am also okay to go with ANDN break up int NOT and AND, but it is needed to come up with concept which instruction/extenstion should be used and how consistently to deal with such situations.

Furthermore, I don't think these changes should block the merging ( doesn't matter in 4.19 or in 4.20 Xen release ) of PATCHes v13 01-07 of this patch series.

This can be looked at different ways. The splitting into NOT+AND was a rather late change.

Jan

#40 lore raw
In reply to [email protected]
Quoted 7 lines · On 27.06.2024 14:08, [email protected] wrote:

I saw a message in the xen-devel channel:```
erm... We've got a problem.
https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7185148004
very clearly failed with a panic(), but reported success out to Gitlab ```
However, I couldn't determine if this is related to the patches in this series or to patches that were merged earlier.

I don't think that's related to the progress of this series. It is technically related, nevertheless.

Quoted 2 lines

I would like to understand what needs to be done to be sure that this patch series could be merged.

First of there is - afaict - still no Arm ack for patch 1. Without that no progress is possible at all. And then there still is this vague "I found another bug" indication by Andrew. It wouldn't feel quite right to me to commit without this having been clarified one way or another, albeit formally - with acks in place and the issue not raised on the list - doing so would still be okay-ish.

Jan

Acked-by
#22 lore raw

Re: [PATCH v13 07/10] xen/common: fix build issue for common/trace.c

In reply to Jan Beulich
Quoted 12 lines · Jan Beulich
Quoted 10 lines · Oleksii
Quoted 9 lines · Jan Beulich
Quoted 7 lines · Oleksii Kurochko

During Gitlab CI randconfig job for RISC-V failed witn an error:
common/trace.c:57:22: error: expected '=', ',', ';', 'asm' or

                              '__attribute__' before

'__read_mostly'
57 | static u32 data_size __read_mostly;

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

If you give a release-ack, this can go in right away, I think.

Thanks, but actually I was misled by the subject prefix. From a formal perspective this really wants an ack from George (and mine doesn't count anything at all).