*Enable build of full Xen for RISC-V
v12 cover *Enable build of full Xen for RISC-V
This patch series performs all of the additions necessary to drop the build overrides for RISCV and enable the full Xen build. Except in cases where compatibile implementations already exist (e.g. atomic.h and bitops.h), the newly added definitions are simple.
The patch series is based on the following patch series:
- [PATCH v2 for-4.19 00/13] xen/bitops: Untangle ffs()/fls() infrastructure [1]
- [PATCH] move __read_mostly to xen/cache.h [2]
Right now, the patch series doesn't have a direct dependency on [2] and it provides __read_mostly in the patch:
[PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the final version of the patch [2] is provided.
As an option, it can be considered to merge arch-specific patch and then just rebase [2] and drop arch-specific changes.
The link to the branch with rebased patches on top of [1] could be found here:
https://gitlab.com/xen-project/people/olkur/xen/-/commits/riscv-full-xen-build-v12
[1] https://lore.kernel.org/xen-devel/[email protected]/
[2] https://lore.kernel.org/xen-devel/[email protected]/
***
I think that I need to write a separate email requesting approval for this patch series to be included in Xen 4.19. Most of the patches are RISC-V specific, so there is a low risk of breaking anything else.
There is only one patch that touches common or arch-specific files, and it doesn't introduce any functional changes.
Since I can't approve it myself, I am asking for someone else to do so.
***
---
Changes in V12:
- Rebase the patch series on top of [1] mentioned above.
- Update the cover letter message.
- "[PATCH v11 3/9] xen/bitops: implement fls{l}() in common logic" was droped as it is a part of patch series [1] mentioned above.
- Other changes are specific to specific patches. Please look at changes for specific patch.
---
Changes in V11:
- Patch was merged to staging:
- [PATCH v10 05/14] xen/riscv: introduce cmpxchg.h
- [PATCH v10 06/14] xen/riscv: introduce atomic.h
- [PATCH v10 07/14] xen/riscv: introduce monitor.h
- [PATCH v10 09/14] xen/riscv: add required things to current.h
- [PATCH v10 11/14] xen/riscv: introduce vm_event_*() functions
- Other changes are specific to specific patches. Please look at changes for specific patch.
---
Changes in V10:
- Patch was merged to staging:
- [PATCH v9 04/15] xen/bitops: put __ffs() into linux compatible header
- Other changes are specific to specific patches. Please look at changes for specific patch.
---
Changes in V9:
- Patch was merged to staging:
- [PATCH v8 07/17] xen/riscv: introduce io.h
- [PATCH v7 14/19] xen/riscv: add minimal stuff to page.h to build full Xen
- Other changes are specific to specific patches. Please look at changes for specific patch.
---
Changes in V8:
- Patch was merged to staging:
- [PATCH v7 01/19] automation: introduce fixed randconfig for RISC-V
- [PATCH v7 03/19] xen/riscv: introduce extenstion support check by compiler
- Other changes are specific to specific patches. Please look at changes for specific patch.
- Update the commit message:
- drop the dependency from STATIC_ASSERT_UNREACHABLE() implementation.
- Add suggestion to merge arch-specific changes related to __read_mostly.
---
Changes in V7:
- Patch was merged to staging:
[PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen.
- Other changes are specific to specific patches. Please look at changes for specific patch.
---
Changes in V6:
- Update the cover letter message: drop already merged dependecies and add
a new one.
- Patches were merged to staging:
- [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
- [PATCH v5 03/23] xen/riscv: introduce nospec.h
- [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
- Introduce new patches:
- xen/riscv: introduce extenstion support check by compiler
- xen/bitops: put __ffs() and ffz() into linux compatible header
- xen/bitops: implement fls{l}() in common logic
- The following patches were dropped:
- drop some patches related to bitops operations as they were introduced in another
patch series [...]
- introduce new version for generic __ffs(), ffz() and fls{l}().
- Merge patch from patch series "[PATCH v9 0/7] Introduce generic headers" to this patch series as only one patch left in the generic headers patch series and it is more about RISC-V.
- Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
- Update the cover letter as one of the dependencies were merged to staging.
- Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
- Moved fence.h to separate patch to deal with some patches dependecies on fence.h
- Patches were dropped as they were merged to staging:
* [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
* [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
* [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
* [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
* [PATCH v4 08/30] xen/riscv: introduce setup.h
* [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
* [PATCH v4 11/30] xen/riscv: introduce smp.h
* [PATCH v4 15/30] xen/riscv: introduce irq.h
* [PATCH v4 16/30] xen/riscv: introduce p2m.h
* [PATCH v4 17/30] xen/riscv: introduce regs.h
* [PATCH v4 18/30] xen/riscv: introduce time.h
* [PATCH v4 19/30] xen/riscv: introduce event.h
* [PATCH v4 22/30] xen/riscv: define an address of frame table
- Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
- Update the cover letter message: new patch series dependencies.
- Some patches were merged to staging, so they were dropped in this patch series:
[PATCH v3 09/34] xen/riscv: introduce system.h
[PATCH v3 18/34] xen/riscv: introduce domain.h
[PATCH v3 19/34] xen/riscv: introduce guest_access.h
- Was sent out of this patch series:
[PATCH v3 16/34] xen/lib: introduce generic find next bit operations
- [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
- All other changes are specific to a specific patch.
---
Changes in V3:
- Update the cover letter message
- The following patches were dropped as they were merged to staging:
[PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
[PATCH v2 04/39] xen/riscv: add public arch-riscv.h
[PATCH v2 05/39] xen/riscv: introduce spinlock.h
[PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
[PATCH v2 34/39] xen: add RISCV support for pmu.h
[PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
- Instead of the following patches were introduced new:
[PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
[PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
- remove "asm/" for commit messages which start with "xen/riscv:"
- code style updates.
- add emulation of {cmp}xchg_* for 1 and 2 bytes types.
- code style fixes.
- add SPDX and footer for the newly added headers.
- introduce generic find-next-bit.c.
- some other mionor changes. ( details please find in a patch )
---
Changes in V2:
- Drop the following patches as they are the part of [2]:
[PATCH v1 06/57] xen/riscv: introduce paging.h
[PATCH v1 08/57] xen/riscv: introduce asm/device.h
[PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
[PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
[PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
[PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
[PATCH v1 18/57] xen/riscv: introduce asm/random.h
[PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
[PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
[PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
[PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
[PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
[PATCH v1 39/57] xen/riscv: introduce asm/numa.h
[PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
- xen/lib.h in most of the cases were changed to xen/bug.h as
mostly functionilty of bug.h is used.
- align arch-riscv.h with Arm's version of it.
- change the Author of commit with introduction of asm/atomic.h.
- update some definition from spinlock.h.
- code style changes.
---
Oleksii Kurochko (8):
xen/riscv: disable unnecessary configs
xen: introduce generic non-atomic test_*bit() xen/riscv: introduce bitops.h
xen/riscv: add definition of __read_mostly xen/riscv: add minimal stuff to mm.h to build full Xen xen/riscv: add minimal amount of stubs to build full Xen xen/riscv: enable full Xen build
xen/README: add compiler and binutils versions for RISC-V64
README | 4 + automation/gitlab-ci/build.yaml | 4 + xen/arch/arm/include/asm/bitops.h | 67 ---- xen/arch/ppc/include/asm/bitops.h | 52 --- xen/arch/ppc/include/asm/page.h | 2 +- xen/arch/ppc/mm-radix.c | 2 +- xen/arch/riscv/Makefile | 17 +- xen/arch/riscv/arch.mk | 4 - xen/arch/riscv/configs/tiny64_defconfig | 12 +- xen/arch/riscv/early_printk.c | 168 ---------- xen/arch/riscv/include/asm/bitops.h | 137 ++++++++ xen/arch/riscv/include/asm/cache.h | 2 + xen/arch/riscv/include/asm/domain.h | 2 + xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++ xen/arch/riscv/include/asm/p2m.h | 2 + xen/arch/riscv/mm.c | 52 ++- xen/arch/riscv/setup.c | 10 +- xen/arch/riscv/stubs.c | 415 ++++++++++++++++++++++++ xen/arch/riscv/traps.c | 25 ++ xen/arch/x86/include/asm/bitops.h | 31 +- xen/include/xen/bitops.h | 146 +++++++++ 21 files changed, 1069 insertions(+), 325 deletions(-) create mode 100644 xen/arch/riscv/include/asm/bitops.h create mode 100644 xen/arch/riscv/stubs.c
Signature · 1 line
2.45.0
PATCH v12 1/8 xen/riscv: disable unnecessary configs
Disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.
Only configs which lead to compilation issues were disabled.
Remove lines related to disablement of configs which aren't affected compilation:
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
-# CONFIG_HYPFS is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
To allow CONFIG_ARGO build happy it was included <asm/p2m.h> to <asm/domain.h> as ARGO requires p2m_type_t ( p2m_ram_rw ) and declaration of check_get_page_from_gfn() from xen/p2m-common.h.
Also, it was included <xen/errno.h> to asm/p2m.h as after the latter was included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP aren't declared started to occur.
CONFIG_XSM=n as it requires an introduction of:
* boot_module_find_by_kind()
* BOOTMOD_XSM
* struct bootmodule
* copy_from_paddr()
The mentioned things aren't introduced now.
CPU_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and acpi_disabled, so it is disabled for now.
PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is also disabled for now, as RISC-V hasn't introduced this headers yet.
LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig, so to avoid compilation errors for randconfig it is disabled for now.
---
Changes in V10-V12:
- Nothing changed. Only rebase.
---
Changes in V9:
- update the commit message: add info about LIVEPATCH and PERF_COUNTERS.
---
Changes in V8:
- disabled CPU_BOOT_TIME_CPUPOOLS as it requires an introduction of cpu_physical_id() and acpi_disabled.
- leave XSM disabled, add explanation in the commit message.
- drop HYPFS as the patch was provided to resolve compilation issue when this condif is enabled for RISC-V.
- include asm/p2m.h to asm/domain.h, and xen/errno.h to asm/p2m.h to drop ARGO config from tiny64_defconfing and build.yaml.
- update the commit message.
---
Changes in V7:
- Disable only configs which cause compilation issues.
- Update the commit message.
---
Changes in V6:
- Nothing changed. Only rebase.
---
Changes in V5:
- Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
- Update the commit message
---
Changes in V4:
- Nothing changed. Only rebase
---
Changes in V3:
- Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
- Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
- update the commit message.
- remove xen/arch/riscv/Kconfig changes.
--- automation/gitlab-ci/build.yaml | 4 ++++ xen/arch/riscv/configs/tiny64_defconfig | 12 +++++------- xen/arch/riscv/include/asm/domain.h | 2 ++ xen/arch/riscv/include/asm/p2m.h | 2 ++ 4 files changed, 13 insertions(+), 7 deletions(-)
automation/gitlab-ci/build.yaml +4 -0
@@ -379,10 +379,14 @@ alpine-3.18-gcc-debug-arm64: .riscv-fixed-randconfig: variables: &riscv-fixed-randconfig EXTRA_FIXED_RANDCONFIG: | + CONFIG_BOOT_TIME_CPUPOOLS=n CONFIG_COVERAGE=n CONFIG_EXPERT=y CONFIG_GRANT_TABLE=n CONFIG_MEM_ACCESS=n + CONFIG_PERF_COUNTERS=n + CONFIG_LIVEPATCH=n + CONFIG_XSM=n archlinux-current-gcc-riscv64-debug: extends: .gcc-riscv64-cross-build-debug
xen/arch/riscv/configs/tiny64_defconfig +5 -7
@@ -1,12 +1,10 @@ -# CONFIG_SCHED_CREDIT is not set -# CONFIG_SCHED_RTDS is not set -# CONFIG_SCHED_NULL is not set -# CONFIG_SCHED_ARINC653 is not set -# CONFIG_TRACEBUFFER is not set -# CONFIG_HYPFS is not set +# CONFIG_BOOT_TIME_CPUPOOLS is not set # CONFIG_GRANT_TABLE is not set -# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set # CONFIG_MEM_ACCESS is not set +# CONFIG_PERF_COUNTERS is not set +# CONFIG_COVERAGE is not set +# CONFIG_LIVEPATCH is not set +# CONFIG_XSM is not set CONFIG_RISCV_64=y CONFIG_DEBUG=y
xen/arch/riscv/include/asm/domain.h +2 -0
@@ -5,6 +5,8 @@#include <xen/xmalloc.h> #include <public/hvm/params.h> +#include <asm/p2m.h> +struct hvm_domain {uint64_t params[HVM_NR_PARAMS ];
xen/arch/riscv/include/asm/p2m.h +2 -0
@@ -2,6 +2,8 @@#ifndef __ASM_RISCV_P2M_H__#define __ASM_RISCV_P2M_H__ +#include <xen/errno.h> +#include <asm/page-bits.h> #define paddr_bits PADDR_BITS
Signature · 1 line
2.45.0
PATCH v12 2/8 xen: introduce generic non-atomic test_*bit()
In reply to Oleksii KurochkoThe following generic functions were introduced:
* test_bit
* generic__test_and_set_bit
* generic__test_and_clear_bit
* generic__test_and_change_bit
These functions and macros can be useful for architectures that don't have corresponding arch-specific instructions.
Also, the patch introduces the following generics which are used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE
The following approach was chosen for generic*() and arch*() bit operation functions:
If the bit operation function that is going to be generic starts with the prefix "__", then the corresponding generic/arch function will also contain the "__" prefix. For example:
* test_bit() will be defined using arch_test_bit() and generic_test_bit().
* __test_and_set_bit() will be defined using arch__test_and_set_bit() and generic__test_and_set_bit().
---
Changes in V12:
- revert change of moving the definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h.
( a separate patch will be provided to put BITS_PER_BYTE to proper place )
- drop comments on top of generic_*() functions and update the comments above __test_*() and test_bit().
- update how static inline __test_*() are defined ( drop pointless fallback #define ) and test_bit().
- drop the footer after Signed-off-by.
---
Changes in V11:
- fix identation in generic_test_bit() function.
- move definition of BITS_PER_BYTE from <arch>/bitops.h to xen/bitops.h
- drop the changes in arm64/livepatch.c.
- update the the comments on top of functions: generic__test_and_set_bit(), generic__test_and_clear_bit(), generic__test_and_change_bit(), generic_test_bit().
- Update footer after Signed-off section.
- Rebase the patch on top of staging branch, so it can be merged when necessary approves will be given.
- ? add Reviewed-by: Jan Beulich <[email protected]>.
---
Changes in V10:
- update the commit message. ( re-order paragraphs and add explanation usage of prefix "__" in bit
operation function names )
- add parentheses around the whole expression of bitop_bad_size() macros.
- move macros bitop_bad_size() above asm/bitops.h as it is not arch-specific anymore and there is
no need for overriding it.
- drop macros check_bitop_size() and use "if ( bitop_bad_size(addr) ) __bitop_bad_size();" implictly
where it is needed.
- in <xen/bitops.h> use 'int' as a first parameter for __test_and_*(), generic__test_and_*() to be
consistent with how the mentioned functions were declared in the original per-arch functions.
- add 'const' to p variable in generic_test_bit().
- move definition of BITOP_BITS_PER_WORD and bitop_uint_t to xen/bitops.h as we don't allow for arch
overrides these definitions anymore.
---
Changes in V9:
- move up xen/bitops.h in ppc/asm/page.h.
- update defintion of arch_check_bitop_size.
And drop correspondent macros from x86/asm/bitops.h
- drop parentheses in generic__test_and_set_bit() for definition of
local variable p.
- fix indentation inside #ifndef BITOP_TYPE...#endif
- update the commit message.
---
Changes in V8:
- drop __pure for function which uses volatile.
- drop unnessary () in generic__test_and_change_bit() for addr casting.
- update prototype of generic_test_bit() and test_bit(): now it returns bool
instead of int.
- update generic_test_bit() to use BITOP_MASK().
- Deal with fls{l} changes: it should be in the patch with introduced generic fls{l}.
- add a footer with explanation of dependency on an uncommitted patch after Signed-off.
- abstract bitop_size().
- move BITOP_TYPE define to <xen/types.h>.
---
Changes in V7:
- move everything to xen/bitops.h to follow the same approach for all generic
bit ops.
- put together BITOP_BITS_PER_WORD and bitops_uint_t.
- make BITOP_MASK more generic.
- drop #ifdef ... #endif around BITOP_MASK, BITOP_WORD as they are generic
enough.
- drop "_" for generic__{test_and_set_bit,...}().
- drop " != 0" for functions which return bool.
- add volatile during the cast for generic__{...}().
- update the commit message.
- update arch related code to follow the proposed generic approach.
---
Changes in V6:
- Nothing changed ( only rebase )
---
Changes in V5:
- new patch
--- xen/arch/arm/include/asm/bitops.h | 67 -------------- xen/arch/ppc/include/asm/bitops.h | 52 ----------- xen/arch/ppc/include/asm/page.h | 2 +- xen/arch/ppc/mm-radix.c | 2 +- xen/arch/x86/include/asm/bitops.h | 31 ++----- xen/include/xen/bitops.h | 146 ++++++++++++++++++++++++++++++ 6 files changed, 157 insertions(+), 143 deletions(-)
xen/arch/arm/include/asm/bitops.h +0 -67
@@ -22,9 +22,6 @@#define __set_bit (n,p) set_bit(n,p)#define __clear_bit (n,p) clear_bit(n,p) -#define BITOP_BITS_PER_WORD 32 -#define BITOP_MASK (nr) (1UL << ((nr) % BITOP_BITS_PER_WORD)) -#define BITOP_WORD (nr) ((nr) / BITOP_BITS_PER_WORD)#define BITS_PER_BYTE 8#define ADDR (*(volatile int *) addr)
@@ -76,70 +73,6 @@ bool test_and_change_bit_timeout(int nr, volatile void *p,bool clear_mask16_timeout (uint16_t mask,volatile void *p,unsigned int max_try); -/** -* __test_and_set_bit - Set a bit and return its old value -* @nr: Bit to set -* @addr: Address to count from -* -* This operation is non-atomic and can be reordered. -* If two examples of this operation race, one can appear to succeed -* but actually fail. You must protect multiple accesses with a lock. -*/ -static inline int __test_and_set_bit (int nr,volatile void *addr) -{ -unsigned int mask =BITOP_MASK (nr); -volatile unsigned int *p = - ((volatile unsigned int *)addr) +BITOP_WORD (nr); -unsigned int old = *p; - - *p = old | mask; -return (old & mask) !=0 ; -} - -/** -* __test_and_clear_bit - Clear a bit and return its old value -* @nr: Bit to clear -* @addr: Address to count from -* -* This operation is non-atomic and can be reordered. -* If two examples of this operation race, one can appear to succeed -* but actually fail. You must protect multiple accesses with a lock. -*/ -static inline int __test_and_clear_bit (int nr,volatile void *addr) -{ -unsigned int mask =BITOP_MASK (nr); -volatile unsigned int *p = - ((volatile unsigned int *)addr) +BITOP_WORD (nr); -unsigned int old = *p; - - *p = old & ~mask; -return (old & mask) !=0 ; -} - -/* WARNING: non atomic and it can be reordered! */ -static inline int __test_and_change_bit (int nr, -volatile void *addr) -{ -unsigned int mask =BITOP_MASK (nr); -volatile unsigned int *p = - ((volatile unsigned int *)addr) +BITOP_WORD (nr); -unsigned int old = *p; - - *p = old ^ mask; -return (old & mask) !=0 ; -} - -/** -* test_bit - Determine whether a bit is set -* @nr: bit number to test -* @addr: Address to start counting from -*/ -static inline int test_bit (int nr,const volatile void *addr) -{ -const volatile unsigned int *p = (const volatile unsigned int *)addr; -return 1UL & (p[BITOP_WORD (nr)] >> (nr & (BITOP_BITS_PER_WORD -1 ))); -} -#define arch_ffs (x) ((x) ? 1 + __builtin_ctz(x) : 0)#define arch_ffsl (x) ((x) ? 1 + __builtin_ctzl(x) : 0)#define arch_fls (x) ((x) ? 32 - __builtin_clz(x) : 0)
xen/arch/ppc/include/asm/bitops.h +0 -52
70 lines not on this page · raw message
xen/arch/ppc/include/asm/page.h +1 -1
@@ -2,9 +2,9 @@#ifndef _ASM_PPC_PAGE_H#define _ASM_PPC_PAGE_H +#include <xen/bitops.h> #include <xen/types.h> -#include <asm/bitops.h> #include <asm/byteorder.h> #define PDE_VALID PPC_BIT(0)
xen/arch/ppc/mm-radix.c +1 -1
@@ -1,11 +1,11 @@/* SPDX-License-Identifier: GPL-2.0-or-later */ +#include <xen/bitops.h> #include <xen/init.h> #include <xen/kernel.h> #include <xen/mm.h> #include <xen/types.h> #include <xen/lib.h> -#include <asm/bitops.h> #include <asm/byteorder.h> #include <asm/early_printk.h> #include <asm/page.h>
xen/arch/x86/include/asm/bitops.h +9 -22
@@ -19,9 +19,6 @@#define ADDR (*(volatile int *) addr)#define CONST_ADDR (*(const volatile int *) addr) -extern void __bitop_bad_size (void ); -#define bitop_bad_size (addr) (sizeof(*(addr)) < 4) -/** * set_bit - Atomically set a bit in memory * @nr: the bit to set
@@ -175,7 +172,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr) })/** -* __test_and_set_bit - Set a bit and return its old value +* arch__test_and_set_bit - Set a bit and return its old value * @nr: Bit to set * @addr: Address to count from *
@@ -183,7 +180,7 @@ static inline int test_and_set_bit(int nr, volatile void *addr)* If two examples of this operation race, one can appear to succeed * but actually fail. You must protect multiple accesses with a lock. */ -static inline int __test_and_set_bit (int nr,void *addr) +static inline int arch__test_and_set_bit (int nr,volatile void *addr) {int oldbit;
@@ -194,10 +191,7 @@ static inline int __test_and_set_bit(int nr, void *addr)return oldbit; } -#define __test_and_set_bit (nr, addr) ({ \ - if ( bitop_bad_size(addr) ) __bitop_bad_size(); \ - __test_and_set_bit(nr, addr); \ -}) +#define arch__test_and_set_bit arch__test_and_set_bit/** * test_and_clear_bit - Clear a bit and return its old value
@@ -224,7 +218,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr) })/** -* __test_and_clear_bit - Clear a bit and return its old value +* arch__test_and_clear_bit - Clear a bit and return its old value * @nr: Bit to set * @addr: Address to count from *
@@ -232,7 +226,7 @@ static inline int test_and_clear_bit(int nr, volatile void *addr)* If two examples of this operation race, one can appear to succeed * but actually fail. You must protect multiple accesses with a lock. */ -static inline int __test_and_clear_bit (int nr,void *addr) +static inline int arch__test_and_clear_bit (int nr,volatile void *addr) {int oldbit;
@@ -243,13 +237,10 @@ static inline int __test_and_clear_bit(int nr, void *addr)return oldbit; } -#define __test_and_clear_bit (nr, addr) ({ \ - if ( bitop_bad_size(addr) ) __bitop_bad_size(); \ - __test_and_clear_bit(nr, addr); \ -}) +#define arch__test_and_clear_bit arch__test_and_clear_bit/* WARNING: non atomic and it can be reordered! */ -static inline int __test_and_change_bit (int nr,void *addr) +static inline int arch__test_and_change_bit (int nr,volatile void *addr) {int oldbit;
@@ -260,10 +251,7 @@ static inline int __test_and_change_bit(int nr, void *addr)return oldbit; } -#define __test_and_change_bit (nr, addr) ({ \ - if ( bitop_bad_size(addr) ) __bitop_bad_size(); \ - __test_and_change_bit(nr, addr); \ -}) +#define arch__test_and_change_bit arch__test_and_change_bit/** * test_and_change_bit - Change a bit and return its new value
@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)return oldbit; } -#define test_bit (nr, addr) ({ \ - if ( bitop_bad_size(addr) ) __bitop_bad_size(); \ +#define arch_test_bit (nr, addr) ({ \ __builtin_constant_p(nr) ? \ constant_test_bit(nr, addr) : \ variable_test_bit(nr, addr); \
xen/include/xen/bitops.h +146 -0
158 lines not on this page · raw message
Signature · 1 line
2.45.0
PATCH v12 6/8 xen/riscv: add minimal amount of stubs to build full Xen
In reply to Oleksii Kurochko---
Changes in V7-V12:
- Only rebase was done.
---
Changes in V6:
- update the commit in stubs.c around /* ... common/irq.c ... */
- add Acked-by: Jan Beulich <[email protected]>
---
Changes in V5:
- drop unrelated changes
- assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
- added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die() from smpboot.c
- back changes related to printk() in early_printk() as they should be removed in the next patch to avoid compilation error.
- update definition of cpu_khz: __read_mostly -> __ro_after_init.
- drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
- move vm_event_*() functions from stubs.c to riscv/vm_event.c.
- s/BUG/BUG_ON("unimplemented") in stubs.c
- back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment, so this function are needed to avoid compilation error.
- defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will be compiled.
---
Changes in V3:
- code style fixes.
- update attribute for frametable_base_pdx and frametable_virt_end to __ro_after_init. insteaf of read_mostly.
- use BUG() instead of assert_failed/WARN for newly introduced stubs.
- drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
- drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
- define udelay stub
- remove 'select HAS_PDX' from RISC-V Kconfig because of
https://lore.kernel.org/xen-devel/[email protected]/
--- xen/arch/riscv/Makefile | 1 + xen/arch/riscv/mm.c | 50 +++++ xen/arch/riscv/setup.c | 8 + xen/arch/riscv/stubs.c | 439 ++++++++++++++++++++++++++++++++++++++++ xen/arch/riscv/traps.c | 25 +++ 5 files changed, 523 insertions(+) create mode 100644 xen/arch/riscv/stubs.c
xen/arch/riscv/Makefile +1 -0
@@ -4,6 +4,7 @@ obj-y += mm.o obj-$(CONFIG_RISCV_64) += riscv64/ obj-y += sbi.o obj-y += setup.o +obj-y += stubs.o obj-y += traps.o obj-y += vm_event.o
xen/arch/riscv/mm.c +50 -0
64 lines not on this page · raw message
xen/arch/riscv/setup.c +8 -0
@@ -1,11 +1,19 @@/* SPDX-License-Identifier: GPL-2.0-only */ +#include <xen/bug.h> #include <xen/compile.h> #include <xen/init.h> #include <xen/mm.h> +#include <public/version.h> +#include <asm/early_printk.h> +void arch_get_xen_caps (xen_capabilities_info_t *info) +{ +BUG_ON ("unimplemented" ); +} +/* Xen stack for bringing up the first CPU. */ unsigned char __initdata cpu0_boot_stack[STACK_SIZE ]__aligned(STACK_SIZE) ;
xen/arch/riscv/stubs.c new file +439 -0
439 lines not on this page · raw message
xen/arch/riscv/traps.c +25 -0
@@ -4,6 +4,10 @@* * RISC-V Trap handlers */ + +#include <xen/lib.h> +#include <xen/sched.h> +#include <asm/processor.h> #include <asm/traps.h>
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs) {die (); } + +void vcpu_show_execution_state (struct vcpu *v) +{ +BUG_ON ("unimplemented" ); +} + +void show_execution_state (const struct cpu_user_regs *regs) +{ +printk ("implement show_execution_state(regs)\n" ); +} + +void arch_hypercall_tasklet_result (struct vcpu *v,long res) +{ +BUG_ON ("unimplemented" ); +} + +enum mc_disposition arch_do_multicall_call (struct mc_state *state) +{ +BUG_ON ("unimplemented" ); +return mc_continue; +}
Signature · 1 line
2.45.0
PATCH v12 3/8 xen/riscv: introduce bitops.h
In reply to Oleksii KurochkoTaken from Linux-6.4.0-rc1
Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
* The following function were removed as they aren't used in Xen:
* test_and_set_bit_lock
* clear_bit_unlock
* __clear_bit_unlock
* The following functions were renamed in the way how they are used by common code:
* __test_and_set_bit
* __test_and_clear_bit
* The declaration and implementation of the following functios were updated to make Xen build happy:
* clear_bit
* set_bit
* __test_and_clear_bit
* __test_and_set_bit
---
Changes in V11-V12:
- Nothing changed. Only rebase was done.
---
Changes in V10:
- update the error message BITS_PER_LONG -> BITOP_BITS_PER_WORD
---
Changes in V9:
- add Acked-by: Jan Beulich <[email protected]>
- drop redefinition of bitop_uint_t in asm/types.h as some operation in Xen common code expects to work with 32-bit quantities.
- s/BITS_PER_LONG/BITOP_BITS_PER_WORD in asm/bitops.h around __AMO() macros.
---
Changes in V8:
- define bitop_uint_t in <asm/types.h> after the changes in patch related to introduction of "introduce generic non-atomic test_*bit()".
- drop duplicated __set_bit() and __clear_bit().
- drop duplicated comment: /* Based on linux/arch/include/asm/bitops.h */.
- update type of res and mask in test_and_op_bit_ord(): unsigned long -> bitop_uint_t.
- drop 1 padding blank in test_and_op_bit_ord().
- update definition of test_and_set_bit(),test_and_clear_bit(),test_and_change_bit:
change return type to bool.
- change addr argument type of test_and_change_bit(): unsigned long * -> void *.
- move test_and_change_bit() closer to other test_and-s function.
- Code style fixes: tabs -> space.
- s/#undef __op_bit/#undef op_bit.
- update the commit message: delete information about generic-non-atomic.h changes as now it is a separate patch.
---
Changes in V7:
- Update the commit message.
- Drop "__" for __op_bit and __op_bit_ord as they are atomic.
- add comment above __set_bit and __clear_bit about why they are defined as atomic.
- align bitops_uint_t with __AMO().
- make changes after generic non-atomic test_*bit() were changed.
- s/__asm__ __volatile__/asm volatile
---
Changes in V6:
- rebase clean ups were done: drop unused asm-generic includes
---
Changes in V5:
- new patch
--- xen/arch/riscv/include/asm/bitops.h | 137 ++++++++++++++++++++++++++++ 1 file changed, 137 insertions(+) create mode 100644 xen/arch/riscv/include/asm/bitops.h
xen/arch/riscv/include/asm/bitops.h new file +137 -0
@@ -0,0 +1,137 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2012 Regents of the University of California */ + +#ifndef _ASM_RISCV_BITOPS_H +#define _ASM_RISCV_BITOPS_H + +#include <asm/system.h> + +#if BITOP_BITS_PER_WORD ==64 +#define __AMO (op) "amo" #op ".d" +#elif BITOP_BITS_PER_WORD ==32 +#define __AMO (op) "amo" #op ".w" +#else +#error "Unexpected BITOP_BITS_PER_WORD" +#endif + +/* Based on linux/arch/include/asm/bitops.h */ + +/* +* Non-atomic bit manipulation. +* +* Implemented using atomics to be interrupt safe. Could alternatively +* implement with local interrupt masking. +*/ +#define __set_bit (n, p) set_bit(n, p) +#define __clear_bit (n, p) clear_bit(n, p) + +#define test_and_op_bit_ord (op, mod, nr, addr, ord) \ +({ \ + bitop_uint_t res, mask; \ + mask = BITOP_MASK(nr); \ + asm volatile ( \ + __AMO(op) #ord " %0, %2, %1" \ + : "=r" (res), "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(mask)) \ + : "memory"); \ + ((res & mask) != 0); \ +}) + +#define op_bit_ord (op, mod, nr, addr, ord) \ + asm volatile ( \ + __AMO(op) #ord " zero, %1, %0" \ + : "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(BITOP_MASK(nr))) \ + : "memory"); + +#define test_and_op_bit (op, mod, nr, addr) \ + test_and_op_bit_ord(op, mod, nr, addr, .aqrl) +#define op_bit (op, mod, nr, addr) \ + op_bit_ord(op, mod, nr, addr, ) + +/* Bitmask modifiers */ +#define NOP (x) (x) +#define NOT (x) (~(x)) + +/** +* test_and_set_bit - Set a bit and return its old value +* @nr: Bit to set +* @addr: Address to count from +*/ +static inline bool test_and_set_bit (int nr,volatile void *p) +{ +volatile bitop_uint_t *addr = p; + +return test_and_op_bit (or,NOP , nr, addr); +} + +/** +* test_and_clear_bit - Clear a bit and return its old value +* @nr: Bit to clear +* @addr: Address to count from +*/ +static inline bool test_and_clear_bit (int nr,volatile void *p) +{ +volatile bitop_uint_t *addr = p; + +return test_and_op_bit (and,NOT , nr, addr); +} + +/** +* test_and_change_bit - Toggle (change) a bit and return its old value +* @nr: Bit to change +* @addr: Address to count from +* +* This operation is atomic and cannot be reordered. +* It also implies a memory barrier. +*/ +static inline bool test_and_change_bit (int nr,volatile void *p) +{ +volatile bitop_uint_t *addr = p; + +return test_and_op_bit (xor,NOP , nr, addr); +} + +/** +* set_bit - Atomically set a bit in memory +* @nr: the bit to set +* @addr: the address to start counting from +* +* Note that @nr may be almost arbitrarily large; this function is not +* restricted to acting on a single-word quantity. +*/ +static inline void set_bit (int nr,volatile void *p) +{ +volatile bitop_uint_t *addr = p; + +op_bit (or,NOP , nr, addr); +} + +/** +* clear_bit - Clears a bit in memory +* @nr: Bit to clear +* @addr: Address to start counting from +*/ +static inline void clear_bit (int nr,volatile void *p) +{ +volatile bitop_uint_t *addr = p; + +op_bit (and,NOT , nr, addr); +} + +#undef test_and_op_bit +#undef op_bit +#undef NOP +#undef NOT +#undef __AMO + +#endif /* _ASM_RISCV_BITOPS_H */ + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* indent-tabs-mode: nil +* End: +*/
Signature · 1 line
2.45.0
PATCH v12 5/8 xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Oleksii Kurochko---
Changes in V8-V12:
- Nothing changed only rebase.
---
Changes in V7:
- update argument type of maddr_to_virt() function: unsigned long -> paddr_t
- rename argument of PFN_ORDER(): pfn -> pg.
- add Acked-by: Jan Beulich <[email protected]>
---
Changes in V6:
- drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
- parenthesize va in definition of vmap_to_mfn().
- Code style fixes.
---
Changes in V5:
- update the comment around "struct domain *domain;" : zero -> NULL
- fix ident. for unsigned long val;
- put page_to_virt() and virt_to_page() close to each other.
- drop unnessary leading underscore
- drop a space before the comment: /* Count of uses of this frame as its current type. */
- drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
- update an argument name of PFN_ORDERN macros.
- drop pad at the end of 'struct page_info'.
- Change message -> subject in "Changes in V3"
- delete duplicated macros from riscv/mm.h
- fix identation in struct page_info
- align comment for PGC_ macros
- update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
- drop unnessary comments.
- s/BUG/BUG_ON("...")
- define __virt_to_maddr, __maddr_to_virt as stubs
- add inclusion of xen/mm-frame.h for mfn_x and others
- include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
In file included from arch/riscv/setup.c:7:
./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
60 | struct page_list_entry list;
| ^~~~
./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
81 | unsigned long first_dirty:MAX_ORDER + 1;
| ^~~~~~~~~
./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
81 | unsigned long first_dirty:MAX_ORDER + 1;
- Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
- update the commit title
- introduce DIRECTMAP_VIRT_START.
- drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
[PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
- code style fixes.
- drop get_page_nr and put_page_nr as they don't need for time being
- drop CONFIG_STATIC_MEMORY related things
- code style fixes
---
Changes in V2:
- define stub for arch_get_dma_bitsize(void)
--- xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++ xen/arch/riscv/mm.c | 2 +- xen/arch/riscv/setup.c | 2 +- 3 files changed, 242 insertions(+), 2 deletions(-)
xen/arch/riscv/include/asm/mm.h +240 -0
@@ -3,11 +3,246 @@#ifndef _ASM_RISCV_MM_H#define _ASM_RISCV_MM_H +#include <public/xen.h> +#include <xen/bug.h> +#include <xen/mm-frame.h> +#include <xen/pdx.h> +#include <xen/types.h> +#include <asm/page-bits.h> #define pfn_to_paddr (pfn) ((paddr_t)(pfn) << PAGE_SHIFT)#define paddr_to_pfn (pa) ((unsigned long)((pa) >> PAGE_SHIFT)) +#define paddr_to_pdx (pa) mfn_to_pdx(maddr_to_mfn(pa)) +#define gfn_to_gaddr (gfn) pfn_to_paddr(gfn_x(gfn)) +#define gaddr_to_gfn (ga) _gfn(paddr_to_pfn(ga)) +#define mfn_to_maddr (mfn) pfn_to_paddr(mfn_x(mfn)) +#define maddr_to_mfn (ma) _mfn(paddr_to_pfn(ma)) +#define vmap_to_mfn (va) maddr_to_mfn(virt_to_maddr((vaddr_t)(va))) +#define vmap_to_page (va) mfn_to_page(vmap_to_mfn(va)) + +static inline void *maddr_to_virt (paddr_t ma) +{ +BUG_ON ("unimplemented" ); +return NULL ; +} + +#define virt_to_maddr (va) ({ BUG_ON("unimplemented"); 0; }) + +/* Convert between Xen-heap virtual addresses and machine frame numbers. */ +#define __virt_to_mfn (va) mfn_x(maddr_to_mfn(virt_to_maddr(va))) +#define __mfn_to_virt (mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn))) + +/* +* We define non-underscored wrappers for above conversion functions. +* These are overriden in various source files while underscored version +* remain intact. +*/ +#define virt_to_mfn (va) __virt_to_mfn(va) +#define mfn_to_virt (mfn) __mfn_to_virt(mfn) + +struct page_info +{ +/* Each frame can be threaded onto a doubly-linked list. */ +struct page_list_entry list; + +/* Reference count and various PGC_xxx flags and fields. */ +unsigned long count_info; + +/* Context-dependent fields follow... */ +union { +/* Page is in use: ((count_info & PGC_count_mask) != 0). */ +struct { +/* Type reference count and various PGT_xxx flags and fields. */ +unsigned long type_info; + } inuse; + +/* Page is on a free list: ((count_info & PGC_count_mask) == 0). */ +union { +struct { +/* +* Index of the first *possibly* unscrubbed page in the buddy. +* One more bit than maximum possible order to accommodate +* INVALID_DIRTY_IDX. +*/ +#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1) +unsigned long first_dirty:MAX_ORDER +1 ; + +/* Do TLBs need flushing for safety before next page use? */ +bool need_tlbflush:1 ; + +#define BUDDY_NOT_SCRUBBING 0 +#define BUDDY_SCRUBBING 1 +#define BUDDY_SCRUB_ABORT 2 +unsigned long scrub_state:2 ; + }; + +unsigned long val; + } free; + } u; + +union { +/* Page is in use */ +struct { +/* Owner of this page (NULL if page is anonymous). */ +struct domain *domain; + } inuse; + +/* Page is on a free list. */ +struct { +/* Order-size of the free chunk this page is the head of. */ +unsigned int order; + } free; + } v; + +union { +/* +* Timestamp from 'TLB clock', used to avoid extra safety flushes. +* Only valid for: a) free pages, and b) pages with zero type count +*/ +uint32_t tlbflush_timestamp; + }; +}; + +#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START) + +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures. */ +#define mfn_to_page (mfn) \ + (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx)) +#define page_to_mfn (pg) \ + pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx) + +static inline void *page_to_virt (const struct page_info *pg) +{ +return mfn_to_virt (mfn_x (page_to_mfn (pg))); +} + +/* Convert between Xen-heap virtual addresses and page-info structures. */ +static inline struct page_info *virt_to_page (const void *v) +{ +BUG_ON ("unimplemented" ); +return NULL ; +} + +/* +* Common code requires get_page_type and put_page_type. +* We don't care about typecounts so we just do the minimum to make it +* happy. +*/ +static inline int get_page_type (struct page_info *page,unsigned long type) +{ +return 1 ; +} + +static inline void put_page_type (struct page_info *page) +{ +} + +static inline void put_page_and_type (struct page_info *page) +{ +put_page_type (page); +put_page (page); +} + +/* +* RISC-V does not have an M2P, but common code expects a handful of +* M2P-related defines and functions. Provide dummy versions of these. +*/ +#define INVALID_M2P_ENTRY (~0UL) +#define SHARED_M2P_ENTRY (~0UL - 1UL) +#define SHARED_M2P (_e) ((_e) == SHARED_M2P_ENTRY) + +#define set_gpfn_from_mfn (mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0) +#define mfn_to_gfn (d, mfn) ((void)(d), _gfn(mfn_x(mfn))) + +#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS) + +static inline unsigned long domain_get_maximum_gpfn (struct domain *d) +{ +BUG_ON ("unimplemented" ); +return 0 ; +} + +static inline long arch_memory_op (int op,XEN_GUEST_HANDLE_PARAM (void ) arg) +{ +BUG_ON ("unimplemented" ); +return 0 ; +} + +/* +* On RISCV, all the RAM is currently direct mapped in Xen. +* Hence return always true. +*/ +static inline bool arch_mfns_in_directmap (unsigned long mfn,unsigned long nr) +{ +return true; +} + +#define PG_shift (idx) (BITS_PER_LONG - (idx)) +#define PG_mask (x, idx) (x ## UL << PG_shift(idx)) + +#define PGT_none PG_mask(0, 1)/* no special uses of this page */ +#define PGT_writable_page PG_mask(1, 1)/* has writable mappings? */ +#define PGT_type_mask PG_mask(1, 1)/* Bits 31 or 63. */ + +/* Count of uses of this frame as its current type. */ +#define PGT_count_width PG_shift(2) +#define PGT_count_mask ((1UL << PGT_count_width) - 1) + +/* +* Page needs to be scrubbed. Since this bit can only be set on a page that is +* free (i.e. in PGC_state_free) we can reuse PGC_allocated bit. +*/ +#define _PGC_need_scrub _PGC_allocated +#define PGC_need_scrub PGC_allocated + +/* Cleared when the owning guest 'frees' this page. */ +#define _PGC_allocated PG_shift(1) +#define PGC_allocated PG_mask(1, 1) +/* Page is Xen heap? */ +#define _PGC_xen_heap PG_shift(2) +#define PGC_xen_heap PG_mask(1, 2) +/* Page is broken? */ +#define _PGC_broken PG_shift(7) +#define PGC_broken PG_mask(1, 7) +/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */ +#define PGC_state PG_mask(3, 9) +#define PGC_state_inuse PG_mask(0, 9) +#define PGC_state_offlining PG_mask(1, 9) +#define PGC_state_offlined PG_mask(2, 9) +#define PGC_state_free PG_mask(3, 9) +#define page_state_is (pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st) + +/* Count of references to this frame. */ +#define PGC_count_width PG_shift(9) +#define PGC_count_mask ((1UL << PGC_count_width) - 1) + +#define _PGC_extra PG_shift(10) +#define PGC_extra PG_mask(1, 10) + +#define is_xen_heap_page (page) ((page)->count_info & PGC_xen_heap) +#define is_xen_heap_mfn (mfn) \ + (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn))) + +#define is_xen_fixed_mfn (mfn) \ + ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) && \ + (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1))) + +#define page_get_owner (p) (p)->v.inuse.domain +#define page_set_owner (p, d) ((p)->v.inuse.domain = (d)) + +/* TODO: implement */ +#define mfn_valid (mfn) ({ (void)(mfn); 0; }) + +#define domain_set_alloc_bitsize (d) ((void)(d)) +#define domain_clamp_alloc_bitsize (d, b) ((void)(d), (b)) + +#define PFN_ORDER (pg) ((pg)->v.free.order) +extern unsigned char cpu0_boot_stack[];void setup_initial_pagetables (void );
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);void turn_on_mmu (unsigned long ra); +static inline unsigned int arch_get_dma_bitsize (void ) +{ +return 32 ;/* TODO */ +} +#endif /* _ASM_RISCV_MM_H */
xen/arch/riscv/mm.c +1 -1
@@ -5,12 +5,12 @@#include <xen/init.h> #include <xen/kernel.h> #include <xen/macros.h> +#include <xen/mm.h> #include <xen/pfn.h> #include <asm/early_printk.h> #include <asm/csr.h> #include <asm/current.h> -#include <asm/mm.h> #include <asm/page.h> #include <asm/processor.h>
xen/arch/riscv/setup.c +1 -1
@@ -2,9 +2,9 @@#include <xen/compile.h> #include <xen/init.h> +#include <xen/mm.h> #include <asm/early_printk.h> -#include <asm/mm.h> /* Xen stack for bringing up the first CPU. */ unsigned char __initdata cpu0_boot_stack[STACK_SIZE ]
Signature · 1 line
2.45.0
PATCH v12 4/8 xen/riscv: add definition of __read_mostly
In reply to Oleksii KurochkoThe definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/[email protected]/
The patch introduces it in arch-specific header to not block enabling of full Xen build for RISC-V.
---
- [PATCH] move __read_mostly to xen/cache.h [2]
Right now, the patch series doesn't have a direct dependency on [2] and it provides __read_mostly in the patch:
[PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the final version of the patch [2] is provided.
Considering that there is still no still final decision regarding patch [2] my suggestion is to merge RISC-V specific patch and just drop the changes in patch [2].
[2] https://lore.kernel.org/xen-devel/[email protected]/
---
Changes in V9-V12:
- Only rebase was done.
---
Change in V8:
- update the footer after Signed-off.
---
Changes in V4-V7:
- Nothing changed. Only rebase.
--- xen/arch/riscv/include/asm/cache.h | 2 ++ 1 file changed, 2 insertions(+)
xen/arch/riscv/include/asm/cache.h +2 -0
@@ -3,4 +3,6 @@#ifndef _ASM_RISCV_CACHE_H#define _ASM_RISCV_CACHE_H +#define __read_mostly __section(".data.read_mostly") +#endif /* _ASM_RISCV_CACHE_H */
Signature · 1 line
2.45.0
PATCH v12 8/8 xen/README: add compiler and binutils versions for RISC-V64
In reply to Oleksii KurochkoThis patch doesn't represent a strict lower bound for GCC and GNU Binutils; rather, these versions are specifically employed by the Xen RISC-V container and are anticipated to undergo continuous testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.
While it is feasible to utilize Clang, it's important to note that, currently, there is no Xen RISC-V CI job in place to verify the seamless functioning of the build with Clang.
--
Changes in V5-V12:
- Nothing changed. Only rebase.
---
Changes in V6:
- update the message in README.
---
Changes in V5:
- update the commit message and README file with additional explanation about GCC and GNU Binutils version. Additionally, it was added information about Clang.
---
Changes in V4:
- Update version of GCC (12.2) and GNU Binutils (2.39) to the version which are in Xen's contrainter for RISC-V
---
Changes in V3:
- new patch
--- README | 4 ++++ 1 file changed, 4 insertions(+)
README +4 -0
@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or later + Older GCC and GNU Binutils would work, but this is not a guarantee. * POSIX compatible awk * Development install of zlib (e.g., zlib-dev) * Development install of Python 2.7 or later (e.g., python-dev)
Signature · 1 line
2.45.0
PATCH v12 7/8 xen/riscv: enable full Xen build
In reply to Oleksii Kurochko---
At least this patch cann't be merged w/o Andrew's patch series is merged as ffs related functions are used from that patch series:
https://lore.kernel.org/xen-devel/[email protected]/T/#t
---
Changes in V5-V12:
- Nothing changed. Only rebase.
- Add the footer after Signed-off section.
---
Changes in V4:
- drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
- drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
- drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
- Reviewed-by: Jan Beulich <[email protected]>
- unrealted change dropped in tiny64_defconfig
---
Changes in V2:
- Nothing changed. Only rebase.
--- xen/arch/riscv/Makefile | 16 +++- xen/arch/riscv/arch.mk | 4 - xen/arch/riscv/early_printk.c | 168 ---------------------------------- xen/arch/riscv/stubs.c | 24 ----- 4 files changed, 15 insertions(+), 197 deletions(-)
xen/arch/riscv/Makefile +15 -1
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms $(OBJCOPY) -O binary -S $< $@ $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds - $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@ + $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \ + $(objtree)/common/symbols-dummy.o -o $(dot-target).0 + $(NM) -pa --format=sysv $(dot-target).0 \ + | $(objtree)/tools/symbols $(all_symbols) --sysv --sort \ + > $(dot-target).0.S + $(MAKE) $(build)=$(@D) $(dot-target).0.o + $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \ + $(dot-target).0.o -o $(dot-target).1 + $(NM) -pa --format=sysv $(dot-target).1 \ + | $(objtree)/tools/symbols $(all_symbols) --sysv --sort \ + > $(dot-target).1.S + $(MAKE) $(build)=$(@D) $(dot-target).1.o + $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \ + $(dot-target).1.o -o $@ $(NM) -pa --format=sysv $@ \ | $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \ > [email protected] + rm -f $(@D)/.$(@F).[0-9]* $(obj)/xen.lds: $(src)/xen.lds.S FORCE $(call if_changed_dep,cpp_lds_S)
xen/arch/riscv/arch.mk +0 -4
@@ -38,7 +38,3 @@ extensions := $(subst $(space),,$(extensions)) # -mcmodel=medlow would force Xen into the lower half. CFLAGS += $(riscv-generic-flags)$(extensions) -mstrict-align -mcmodel=medany - -# TODO: Drop override when more of the build is working -override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o -override ALL_LIBS-y =
xen/arch/riscv/early_printk.c +0 -168
171 lines not on this page · raw message
xen/arch/riscv/stubs.c +0 -24
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);nodemask_t __read_mostly node_online_map = { { [0 ] =1UL } }; -/* -* max_page is defined in page_alloc.c which isn't complied for now. -* definition of max_page will be remove as soon as page_alloc is built. -*/ -unsigned long __read_mostly max_page; -/* time.c */ unsigned long __ro_after_init cpu_khz;/* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) {BUG_ON ("unimplemented" ); } - -/* -* The following functions are defined in common/irq.c, but common/irq.c isn't -* built for now. These changes will be removed there when common/irq.c is -* ready. -*/ - -void cf_check irq_actor_none(struct irq_desc *desc) -{ -BUG_ON ("unimplemented" ); -} - -unsigned int cf_check irq_startup_none(struct irq_desc *desc) -{ -BUG_ON ("unimplemented" ); - -return 0 ; -}
Signature · 1 line
2.45.0
Re: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
In reply to Oleksii KurochkoThe subject should say "update Kconfig", because you're not (only) disabling.
I'd suggest "xen/riscv: Update Kconfig in preparation for a full Xen build".
Quoted 31 lines · Oleksii Kurochko
Disables unnecessary configs for two cases:
1. By utilizingEXTRA_FIXED_RANDCONFIGfor randconfig builds (GitLab CI jobs).
2. By usingtiny64_defconfigfor non-randconfig builds.Only configs which lead to compilation issues were disabled.
Remove lines related to disablement of configs which aren't affected compilation:
-#CONFIG_SCHED_CREDITis not set
-#CONFIG_SCHED_RTDSis not set
-#CONFIG_SCHED_NULLis not set
-#CONFIG_SCHED_ARINC653is not set
-#CONFIG_TRACEBUFFERis not set
-#CONFIG_HYPFSis not set
-#CONFIG_SPECULATIVE_HARDEN_ARRAYis not setTo allow
CONFIG_ARGObuild happy it was included <asm/p2m.h> to <asm/domain.h> as ARGO requiresp2m_type_t(p2m_ram_rw) and declaration ofcheck_get_page_from_gfn()fromxen/p2m-common.h.Also, it was included <
xen/errno.h> toasm/p2m.has after the latter was included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP aren't declared started to occur.
CONFIG_XSM=nas it requires an introduction of:
*boot_module_find_by_kind()
*BOOTMOD_XSM
* struct bootmodule
*copy_from_paddr()
The mentioned things aren't introduced now.
Quoted 2 lines
CPU_BOOT_TIME_CPUPOOLSrequires an introduction ofcpu_physical_id()andacpi_disabled, so it is disabled for now.
CONFIG_BOOT_TIME_CPUPOOLS
Also the "depends on DT" isn't good enough as a restriction IMO. It's very ARM-dom0less specific.
Quoted 4 lines
PERF_COUNTERSrequiresasm/perf.handasm/perfc-defn.h, so it is also disabled for now, as RISC-V hasn't introduced this headers yet. LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig, so to avoid compilation errors for randconfig it is disabled for now.
PERF_COUNTERS is x86-only, and both LIVEPATCH really should be guarded by have HAVE_$FOO selected by ARCH.
However, that's not work to get stuck into now.
It's quite unreasonable how much stuff doesn't work in simple builds...
Quoted 1 line
Signed-off-by Oleksii Kurochko <[email protected]>
I'm happy to fix up the two minor issues on commit.
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Oleksii KurochkoREADME provided by your OS distributor: · line 54
Quoted 12 lines · Oleksii Kurochko
README +4
-0@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or later + Older GCC and GNU Binutils would work, but this is not a guarantee.
This sentence isn't appropriate to live here.
The commit message saying "this is what we run in CI" is perfectly good enough.
With this dropped, Reviewed-by: Andrew Cooper
<[email protected]>. Can fix on commit.
Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
In reply to Oleksii KurochkoQuoted 29 lines · Oleksii Kurochko
xen/arch/riscv/stubs.c +0
-14@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);nodemask_t __read_mostly node_online_map = { { [0 ] =1UL } }; -/* -* max_page is defined in page_alloc.c which isn't complied for now. -* definition of max_page will be remove as soon as page_alloc is built. -*/ -unsigned long __read_mostly max_page; -/* time.c */ unsigned long __ro_after_init cpu_khz;/* CPU clock frequency in kHz. */ @@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) {BUG_ON ("unimplemented" ); } - -/* -* The following functions are defined in common/irq.c, but common/irq.c isn't -* built for now. These changes will be removed there when common/irq.c is -* ready. -*/ - -void cf_check irq_actor_none(struct irq_desc *desc)
xen/arch/riscv/stubs.c void __cpu_die(unsigned int cpu) · line 439
Quoted 10 lines
xen/arch/riscv/stubs.c +0
-10-{ -BUG_ON ("unimplemented" ); -} - -unsigned int cf_check irq_startup_none(struct irq_desc *desc) -{ -BUG_ON ("unimplemented" ); - -return 0 ; -}
All 3 of these are introduced in the previous patch and deleted again here. Looks like a rebasing accident.
(This patch has to be the final one touching build related things, so I can't simply fix it on commit)
~Andrew
Re: [PATCH v12 6/8] xen/riscv: add minimal amount of stubs to build full Xen
In reply to Oleksii Kurochkoxen/arch/riscv/stubs.c line 399
Quoted 13 lines · Oleksii Kurochko
xen/arch/riscv/stubs.c new file +6
-0@@ -0,0 +1,439 @@ +void udelay (unsigned long usecs) +{ +BUG_ON ("unimplemented" ); +} + +/* guest_access.h */
You've got trailing whitespace here.
Also, this patch is (still) a shocking indictment on how bad our common vs arch split really is.
~Andrew
Re: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
In reply to Andrew CooperQuoted 68 lines · Andrew Cooper
The subject should say "update
Kconfig", because you're not (only) disabling.I'd suggest "xen/riscv: Update
Kconfigin preparation for a full Xen build".Quoted 36 lines · Oleksii Kurochko
Disables unnecessary configs for two cases:
1. By utilizingEXTRA_FIXED_RANDCONFIGfor randconfig builds (GitLab CI jobs).
2. By usingtiny64_defconfigfor non-randconfig builds.Only configs which lead to compilation issues were disabled.
Remove lines related to disablement of configs which aren't affected
compilation:
-#CONFIG_SCHED_CREDITis not set
-#CONFIG_SCHED_RTDSis not set
-#CONFIG_SCHED_NULLis not set
-#CONFIG_SCHED_ARINC653is not set
-#CONFIG_TRACEBUFFERis not set
-#CONFIG_HYPFSis not set
-#CONFIG_SPECULATIVE_HARDEN_ARRAYis not setTo allow
CONFIG_ARGObuild happy it was included <asm/p2m.h> to <asm/domain.h>
as ARGO requiresp2m_type_t(p2m_ram_rw) and declaration ofcheck_get_page_from_gfn()fromxen/p2m-common.h.Also, it was included <
xen/errno.h> toasm/p2m.has after the latter was
included to <asm/domain.h> the compilation error that EINVAL, EOPNOTSUPP
aren't declared started to occur.
CONFIG_XSM=nas it requires an introduction of:
*boot_module_find_by_kind()
*BOOTMOD_XSM
* struct bootmodule
*copy_from_paddr()
The mentioned things aren't introduced now.Quoted 3 lines
CPU_BOOT_TIME_CPUPOOLSrequires an introduction ofcpu_physical_id()andacpi_disabled, so it is disabled for now.
CONFIG_BOOT_TIME_CPUPOOLSAlso the "depends on DT" isn't good enough as a restriction IMO. It's
very ARM-dom0less specific.Quoted 7 lines
PERF_COUNTERSrequiresasm/perf.handasm/perfc-defn.h, so it is also disabled for now, as RISC-V hasn't introduced this headers yet.
LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
so to avoid compilation errors for randconfig it is disabled for now.
PERF_COUNTERSis x86-only, and both LIVEPATCH really should be guarded
by have HAVE_$FOO selected by ARCH.However, that's not work to get stuck into now.
Quoted 8 lines
It's quite unreasonable how much stuff doesn't work in simple builds...
Quoted 1 line
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Andrew Cooper <[email protected]>I'm happy to fix up the two minor issues on commit.
Thanks. I Would appreciate that.
~ Oleksii
Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
In reply to Andrew CooperQuoted 41 lines · Andrew Cooper
Quoted 40 lines · Oleksii Kurochko
xen/arch/riscv/stubs.c +0
-0@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
nodemask_t__read_mostlynode_online_map= { { [0] = 1UL } };
-/*
- *max_pageis defined inpage_alloc.cwhich isn't complied for now.
- * definition ofmax_pagewill be remove as soon aspage_allocis built.
- */xen/arch/riscv/stubs.c +0
-2-unsigned long __read_mostly max_page; -/*
time.c*/
unsigned long__ro_after_initcpu_khz; /* CPU clock frequency in kHz. */xen/arch/riscv/stubs.c +0
-3@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) { BUG_ON("unimplemented"); } - -/* -* The following functions are defined in common/irq.c, but
common/irq.cisn't
- * built for now. These changes will be removed there whencommon/irq.cis
- * ready.
- */xen/arch/riscv/stubs.c +0
-7- -void cf_check irq_actor_none(struct irq_desc *desc) -{ - BUG_ON("unimplemented"); -} - -unsigned int cf_check irq_startup_none(struct irq_desc *desc)
Quoted 8 lines
xen/arch/riscv/stubs.c void __cpu_die(unsigned int cpu) · line 439
Quoted 5 lines
xen/arch/riscv/stubs.c +0
-5-{ - BUG_ON("unimplemented"); - - return 0; -}All 3 of these are introduced in the previous patch and deleted again here. Looks like a rebasing accident.
Not really.
This was done to avoid build failures for RISC-V. If the HEAD is on the previous patch where these changes are introduced and then we just drop them, it will lead to a linkage error because these functions are defined in common/irq.c, which isn't built for RISC-V if the HEAD is on the previous patch:
/build/xen/arch/riscv/entry.S:86: undefined reference to `max_page'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined reference to `irq_startup_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined reference to `irq_actor_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined reference to `irq_actor_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined reference to `irq_actor_none'
riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none' isn't defined
That is why these stubs were introduced in the previous patch (because common/irq.c isn't built at that moment) and are removed in this patch (since at the moment of this patch, common/irq.c is now being built).
~ Oleksii
Quoted 6 lines
(This patch has to be the final one touching build related things, so I
can't simply fix it on commit)~Andrew
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Andrew CooperQuoted 17 lines · Andrew Cooper
Quoted 5 lines · Oleksii Kurochko
README +0
-0@@ -48,6 +48,10 @@ provided by your OS distributor:Quoted 8 lines
- For ARM 64-bit:
- GCC 5.1 or later
- GNU Binutils 2.24 or later
+ - For RISC-V 64-bit:
+ - GCC 12.2 or later
+ - GNU Binutils 2.39 or later
+ Older GCC and GNU Binutils would work, but this is not a guarantee.This sentence isn't appropriate to live here.
Quoted 6 lines
The commit message saying "this is what we run in CI" is perfectly good
enough.With this dropped, Reviewed-by: Andrew Cooper
<[email protected]>. Can fix on commt.
I am okay with dropping this sentence, but someone ( unfortunately I don't remember who Jan? Julien? ) requested it, and I think it would be nice to hear their opinion before doing so.
~ Oleksii
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
Quoted 21 lines · Oleksii K.
Quoted 20 lines · Andrew Cooper
Quoted 5 lines · Oleksii Kurochko
README +0
-0@@ -48,6 +48,10 @@ provided by your OS distributor:Quoted 8 lines
- For ARM 64-bit:
- GCC 5.1 or later
- GNU Binutils 2.24 or later
+ - For RISC-V 64-bit:
+ - GCC 12.2 or later
+ - GNU Binutils 2.39 or later
+ Older GCC and GNU Binutils would work, but this is not a guarantee.This sentence isn't appropriate to live here.
The commit message saying "this is what we run in CI" is perfectly good
enough.
Quoted 5 lines
Quoted 2 lines
With this dropped, Reviewed-by: Andrew Cooper
<[email protected]>. Can fix on commt.I am okay with dropping this sentence, but someone ( unfortunately I don't remember who Jan? Julien? ) requested it, and I think it would be nice to hear their opinion before doing so.
It's line noise, and literally a redundant statement. The same is true of every other line in the file, and we don't write it because that would be incredibly stupid.
Anyone who wants to see why this was chosen can read the commit message.
~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Oleksii KurochkoQuoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>
This patch looks like it can go in independently? Or does it depend on having bitops.h working in practice?
However, one very strong suggestion...
Quoted 5 lines
xen/arch/riscv/include/asm/mm.h +0
-0
xen/arch/riscv/include/asm/mm.h line 117
Quoted 9 lines
xen/arch/riscv/include/asm/mm.h +8
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures. */ +#define mfn_to_page (mfn) \ + (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx)) +#define page_to_mfn (pg) \ + pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)
Do yourself a favour and not introduce frametable_base_pdx to begin with.
Every RISC-V board I can find has things starting from 0 in physical address space, with RAM starting immediately after.
Taking the microchip board as an example, RAM actually starts at 0x8000000, which means that having frametable_base_pdx and assuming it does get set to 0x8000 (which isn't even a certainty, given that I think you'll need struct pages covering the PLICs), then what you are trading off is:
* Saving 32k of virtual address space only (no need to even allocate memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn conversion, which is a screaming hotpath all over Xen.
It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally wouldn't - especially as there's no need to share Xen's VA space with guests, given no PV guests on ARM/RISC-V), but it's absolutely not at all in an a 64bit build with TB of VA space available.
Even if we do find a board with the first interesting thing in the frametable starting sufficiently away from 0 that it might be worth considering this slide, then it should still be Kconfig-able in a similar way to PDX_COMPRESSION.
You don't want to be penalising everyone because of a theoretical/weird corner case.
~Andrew
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Andrew CooperHi,
Quoted 22 lines · Andrew Cooper
Quoted 21 lines · Oleksii K.
Quoted 20 lines · Andrew Cooper
Quoted 12 lines · Oleksii Kurochko
README +4
-0@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or later + Older GCC and GNU Binutils would work, but this is not aQuoted 1 line
guarantee.
This sentence isn't appropriate to live here.
The commit message saying "this is what we run in CI" is perfectly good
enough.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
With this dropped, Reviewed-by: Andrew Cooper
<[email protected]>. Can fix on commt.I am okay with dropping this sentence, but someone ( unfortunately I don't remember who Jan? Julien? ) requested it, and I think it would be nice to hear their opinion before doing so.
I don't think it was me :). I have no issue with dropping the line.
Quoted 3 lines
It's line noise, and literally a redundant statement. The same is true of every other line in the file,
+1. If anyone wanted extra clarify, we could spell out at the beginning of the file that this is the versions are confident with.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
In reply to Oleksii K.Quoted 49 lines · Oleksii K.
Quoted 40 lines · Andrew Cooper
Quoted 39 lines · Oleksii Kurochko
xen/arch/riscv/stubs.c +0
-0@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
nodemask_t__read_mostlynode_online_map= { { [0] = 1UL } };
-/*
- *max_pageis defined inpage_alloc.cwhich isn't complied for now.
- * definition ofmax_pagewill be remove as soon aspage_allocis built.
- */xen/arch/riscv/stubs.c +0
-2-unsigned long __read_mostly max_page; -/*
time.c*/
unsigned long__ro_after_initcpu_khz; /* CPU clock frequency in kHz. */xen/arch/riscv/stubs.c +0
-3@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) { BUG_ON("unimplemented"); } - -/* -* The following functions are defined in common/irq.c, but
common/irq.cisn't
- * built for now. These changes will be removed there whencommon/irq.cis
- * ready.
- */xen/arch/riscv/stubs.c +0
-6- -void cf_check irq_actor_none(struct irq_desc *desc) -{ - BUG_ON("unimplemented"); -} -Quoted 8 lines
xen/arch/riscv/stubs.c void __cpu_die(unsigned int cpu) · line 439
Quoted 6 lines
xen/arch/riscv/stubs.c +0
-6-unsigned int cf_check irq_startup_none(struct irq_desc *desc) -{ - BUG_ON("unimplemented"); - - return 0; -}All 3 of these are introduced in the previous patch and deleted again here. Looks like a rebasing accident.
Quoted 8 lines
Not really.
This was done to avoid build failures for RISC-V. If the HEAD is on the previous patch where these changes are introduced and then we just drop them, it will lead to a linkage error because these functions are defined in
common/irq.c, which isn't built for RISC-V if the HEAD is on the previous patch:
/build/xen/arch/riscv/entry.S:86: undefined reference to `max_page'
Nothing in the series touches entry.S, so I'm not sure what this is complaining about.
The stub for get_upper_mfn_bound() references max_page, but it's only used in a SYSCTL so you can avoid the problem with a BUG_ON().
BTW, you also don't need a return after a BUG_ON(). __builtin_unreachable() takes care of everything properly for you.
Quoted 11 lines
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined reference to `irq_startup_none' riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined reference to `irq_actor_none' riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined reference to `irq_actor_none' riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined reference to `irq_actor_none' riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none' isn't defined
Quoted 3 lines
That is why these stubs were introduced in the previous patch (because
common/irq.cisn't built at that moment) and are removed in this patch (since at the moment of this patch,common/irq.cis now being built).
These OTOH are a side effect of how no_irq_type works, which is horrifying, and not something I can unsee.
I'll see about fixing it, because I really can't bare to leave it like this...
~Andrew
Re: [PATCH v12 6/8] xen/riscv: add minimal amount of stubs to build full Xen
In reply to Oleksii KurochkoQuoted 12 lines · Oleksii Kurochko
xen/arch/riscv/mm.c +1
-0@@ -1,5 +1,6 @@/* SPDX-License-Identifier: GPL-2.0-only */ +#include <xen/bug.h> #include <xen/cache.h> #include <xen/compiler.h> #include <xen/init.h>
xen/arch/riscv/mm.c line 19
Quoted 5 lines
xen/arch/riscv/mm.c +2
-0@@ -14,6 +15,9 @@#include <asm/page.h> #include <asm/processor.h> +unsigned long __ro_after_init frametable_base_pdx; +unsigned long __ro_after_init frametable_virt_end;
Along with removing base_pdx, virt_end appears to be unique to ARM before and after the series. I'd recommend dropping it too.
~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Andrew CooperQuoted 34 lines · Andrew Cooper
Quoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend on
havingbitops.hworking in practice?However, one very strong suggestion...
Quoted 11 lines
xen/arch/riscv/include/asm/mm.h +4
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures. Quoted 8 lines
*/
+#define
mfn_to_page(mfn) \
+ (frame_table+ (mfn_to_pdx(mfn)-frametable_base_pdx)) +#define
page_to_mfn(pg) \
+pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)Do yourself a favour and not introduce
frametable_base_pdxto begin with.
Quoted 2 lines
Every RISC-V board I can find has things starting from 0 in physical address space, with RAM starting immediately after.
I checked Linux kernel and grep there:
[ok@fedora linux-aia]$ grep -Rni "memory@" arch/riscv/boot/dts/ -- exclude "*.tmp" -I arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi:33:
memory@40000000 {
arch/riscv/boot/dts/starfive/jh7100-common.dtsi:28: memory@80000000
{
arch/riscv/boot/dts/microchip/mpfs-sev-kit.dts:49: ddrc_cache:
memory@1000000000 {
arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:33: ddrc_cache_lo:
memory@80000000 {
arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:37: ddrc_cache_hi:
memory@1040000000 {
arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:34: ddrc_cache_lo:
memory@80000000 {
arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:40: ddrc_cache_hi:
memory@1000000000 {
arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:22: ddrc_cache_lo:
memory@80000000 {
arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:27: ddrc_cache_hi:
memory@1000000000 {
arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:57: ddrc_cache_lo:
memory@80000000 {
arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:63: ddrc_cache_hi:
memory@1040000000 { arch/riscv/boot/dts/thead/th1520-beaglev-ahead.dts:32: memory@0 { arch/riscv/boot/dts/thead/th1520-lichee-module-4a.dtsi:14:
memory@0 {
arch/riscv/boot/dts/sophgo/cv1800b-milkv-duo.dts:26: memory@80000000
{
arch/riscv/boot/dts/sophgo/cv1812h.dtsi:12: memory@80000000 {
arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts:26: memory@80000000 {
arch/riscv/boot/dts/sifive/hifive-unleashed-a00.dts:25: memory@80000000 {
arch/riscv/boot/dts/canaan/k210.dtsi:82: sram: memory@80000000 {
And based on that majority of supported by Linux kernel boards has RAM starting not from 0 in physical address space. Am I confusing something?
Quoted 3 lines
Taking the microchip board as an example, RAM actually starts at
0x8000000,
Today we had conversation with the guy from SiFive in xen-devel channel and he mentioned that they are using "starfive visionfive2 and sifive unleashed platforms." which based on the grep above has RAM not at 0 address.
Also, QEMU uses 0x8000000.
Quoted 19 lines
which means that having
frametable_base_pdxand assuming it
does get set to0x8000(which isn't even a certainty, given that I think
you'll need struct pages covering the PLICs), then what you are trading
off is:* Saving 32k of virtual address space only (no need to even allocate memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn conversion, which is a screaming hotpath all over Xen.It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally wouldn't - especially as there's no need to share Xen's VA space with guests, given no PV guests on ARM/RISC-V), but it's absolutely not at all in an a 64bit build with TB of VA space available.
Quoted 4 lines
Even if we do find a board with the first interesting thing in the frametable starting sufficiently away from 0 that it might be worth considering this slide, then it should still be Kconfig-able in a similar way to
PDX_COMPRESSION.
I found your tradeoffs reasonable, but I don't understand how it will work if RAM does not start from 0, as the frametable address and RAM address are linked.
I tried to look at the PDX_COMPRESSION config and couldn't find any "slide" there. Could you please clarify this for me?
If to use this "slide" would it help to avoid the mentioned above tradeoffs?
One more question: if we decide to go without frametable_base_pdx, would it be sufficient to simply remove mentions of it from the code ( at least, for now )?
~ Oleksii
Quoted 8 lines
You don't want to be penalising everyone because of a theoretical/weird
corner case.~Andrew
Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
In reply to Andrew CooperQuoted 67 lines · Andrew Cooper
Quoted 55 lines · Oleksii K.
Quoted 44 lines · Andrew Cooper
Quoted 43 lines · Oleksii Kurochko
xen/arch/riscv/stubs.c +0
-0@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
nodemask_t__read_mostlynode_online_map= { { [0] = 1UL } };
-/*
- *max_pageis defined inpage_alloc.cwhich isn't complied for
now.
- * definition ofmax_pagewill be remove as soon aspage_allocis
built.
- */xen/arch/riscv/stubs.c +0
-2-unsigned long __read_mostly max_page; -/*
time.c*/
unsigned long__ro_after_initcpu_khz; /* CPU clock frequency in
kHz. */xen/arch/riscv/stubs.c +0
-3@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) { BUG_ON("unimplemented"); } - -/* -* The following functions are defined in common/irq.c, but
common/irq.cisn't
- * built for now. These changes will be removed there whencommon/irq.cis
- * ready.
- */xen/arch/riscv/stubs.c +0
-7- -void cf_check irq_actor_none(struct irq_desc *desc) -{ - BUG_ON("unimplemented"); -} - -unsigned int cf_check irq_startup_none(struct irq_desc *desc)Quoted 8 lines
xen/arch/riscv/stubs.c void __cpu_die(unsigned int cpu) · line 439
Quoted 5 lines
xen/arch/riscv/stubs.c +0
-5-{ - BUG_ON("unimplemented"); - - return 0; -}All 3 of these are introduced in the previous patch and deleted again
here. Looks like a rebasing accident.Not really.
Quoted 10 lines
This was done to avoid build failures for RISC-V. If the HEAD is on the
previous patch where these changes are introduced and then we just drop
them, it will lead to a linkage error because these functions are defined incommon/irq.c, which isn't built for RISC-V if the HEAD is on
the previous patch:
/build/xen/arch/riscv/entry.S:86: undefined reference to `max_page'
Quoted 5 lines
Nothing in the series touches
entry.S, so I'm not sure what this is complaining about.The stub for
get_upper_mfn_bound()referencesmax_page, but it's only used in a SYSCTL so you can avoid the problem with aBUG_ON().
I didn't get how I can use BUG_ON() with declaration of variable to avoid the compilation issue the undefined reference to max_page?
Quoted 28 lines
BTW, you also don't need a return after a
BUG_ON().__builtin_unreachable()takes care of everything properly for you.Quoted 16 lines
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined reference to
`irq_startup_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined reference to
`irq_actor_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined reference to
`irq_actor_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined reference to
`irq_actor_none'
riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none' isn't
definedQuoted 6 lines
That is why these stubs were introduced in the previous patch (because
common/irq.cisn't built at that moment) and are removed in this patch
(since at the moment of this patch,common/irq.cis now being built).
Quoted 6 lines
These OTOH are a side effect of how
no_irq_typeworks, which is horrifying, and not something I can unsee.I'll see about fixing it, because I really can't bare to leave it like
this...
I am really sorry, but I don't understand should I deal with this somehow now or the provided changes are okay?
~ Oleksii
Re: [PATCH v12 7/8] xen/riscv: enable full Xen build
Quoted 67 lines · Oleksii K.
Quoted 66 lines · Andrew Cooper
Quoted 65 lines · Oleksii K.
Quoted 44 lines · Andrew Cooper
Quoted 43 lines · Oleksii Kurochko
xen/arch/riscv/stubs.c +0
-0@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
nodemask_t__read_mostlynode_online_map= { { [0] = 1UL } };
-/*
- *max_pageis defined inpage_alloc.cwhich isn't complied for
now.
- * definition ofmax_pagewill be remove as soon aspage_allocis
built.
- */xen/arch/riscv/stubs.c +0
-2-unsigned long __read_mostly max_page; -/*
time.c*/
unsigned long__ro_after_initcpu_khz; /* CPU clock frequency in
kHz. */xen/arch/riscv/stubs.c +0
-3@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) { BUG_ON("unimplemented"); } - -/* -* The following functions are defined in common/irq.c, but
common/irq.cisn't
- * built for now. These changes will be removed there whencommon/irq.cis
- * ready.
- */xen/arch/riscv/stubs.c +0
-7- -void cf_check irq_actor_none(struct irq_desc *desc) -{ - BUG_ON("unimplemented"); -} - -unsigned int cf_check irq_startup_none(struct irq_desc *desc)Quoted 8 lines
xen/arch/riscv/stubs.c void __cpu_die(unsigned int cpu) · line 439
Quoted 5 lines
xen/arch/riscv/stubs.c +0
-5-{ - BUG_ON("unimplemented"); - - return 0; -}All 3 of these are introduced in the previous patch and deleted again
here. Looks like a rebasing accident.Not really.
This was done to avoid build failures for RISC-V. If the HEAD is on the
previous patch where these changes are introduced and then we just drop
them, it will lead to a linkage error because these functions are defined incommon/irq.c, which isn't built for RISC-V if the HEAD is on
the previous patch:
/build/xen/arch/riscv/entry.S:86: undefined reference to `max_page'
Quoted 7 lines
Quoted 5 lines
Nothing in the series touches
entry.S, so I'm not sure what this is complaining about.The stub for
get_upper_mfn_bound()referencesmax_page, but it's only used in a SYSCTL so you can avoid the problem with aBUG_ON().I didn't get how I can use
BUG_ON()with declaration of variable to avoid the compilation issue the undefined reference tomax_page?
You don't need to implement get_upper_mfn_bound() yet.
If you implement it as:
unsigned long get_upper_mfn_bound(void)
{
BUG_ON("unimplemented");
}
in the previous patch, then you also don't have any problems with max_page between these two patches either.
Quoted 27 lines
Quoted 27 lines
BTW, you also don't need a return after a
BUG_ON().__builtin_unreachable()takes care of everything properly for you.Quoted 22 lines
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x8): undefined reference to
`irq_startup_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x10): undefined reference to
`irq_actor_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x18): undefined reference to
`irq_actor_none'
riscv64-linux-gnu-ld: prelink.o:(.rodata+0x20): undefined reference to
`irq_actor_none'
riscv64-linux-gnu-ld: xen-syms: hidden symbol `irq_actor_none' isn't
definedThat is why these stubs were introduced in the previous patch (because
common/irq.cisn't built at that moment) and are removed in this patch
(since at the moment of this patch,common/irq.cis now being built).
Quoted 8 lines
Quoted 6 lines
These OTOH are a side effect of how
no_irq_typeworks, which is horrifying, and not something I can unsee.I'll see about fixing it, because I really can't bare to leave it like
this...I am really sorry, but I don't understand should I deal with this somehow now or the provided changes are okay?
I've emailed out a 2-patch series to unbreak this.
~Andrew
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Oleksii KurochkoREADME provided by your OS distributor: · line 53
Quoted 11 lines · Oleksii Kurochko
README +3
-0@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or later
I would like to petition for GCC 10 and Binutils 2.35.
These are the versions provided as cross-compilers by Debian, and therefore are the versions I would prefer to do smoke testing with.
One issue is in cpu_relax(), needing this diff to fix:
(quoted diff) +0 -0
b/xen/arch/riscv/include/asm/processor.h
index 6846151717f7..830a05dd8e3a 100644
xen/arch/riscv/include/asm/processor.h +0 -0
@@ -67,7 +67,7 @@ static inline void cpu_relax(void)
__asm__ __volatile__ ( "pause" ); #else
/* Encoding of the pause instruction */ - __asm__ __volatile__ ( ".insn 0x0100000F" ); + __asm__ __volatile__ ( ".4byte 0x0100000F" ); #endif
barrier();
The .insn directive appears to check that the byte pattern is a known extension, where .4byte doesn't. AFAICT, this makes .insn pretty useless for what I'd expect is it's main purpose...
The other problem is a real issue in cmpxchg.h, already committed to staging (51dabd6312c).
RISC-V does a conditional toolchain for the Zbb extension (xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction in emulate_xchg_1_2().
Nevertheless, this is also quite easy to fix:
(quoted diff) +0 -0
b/xen/arch/riscv/include/asm/cmpxchg.h
index d5e678c03678..12ecb0950701 100644
xen/arch/riscv/include/asm/cmpxchg.h +0 -0
@@ -18,6 +18,20 @@
: "r" (new) \
: "memory" );
+/*
+ * Binutils < 2.37 doesn't understand ANDN. If the toolchain is too old, form
+ * it of a NOT+AND pair
+ */
+#ifdef __riscv_zbb
+#define ANDN_INSN(rd, rs1, rs2) \
+ "andn " rd ", " rs1 ", " rs2 "\n"
+#else
+#define ANDN_INSN(rd, rs1, rs2) \
+ "not " rd ", " rs2 "\n" \
+ "and " rd ", " rs1 ", " rd "\n"
+#endif
+
+
/*
* For LR and SC, the A extension requires that the address held in rs1 be
* naturally aligned to the size of the operand (i.e., eight-byte aligned @@ -48,7 +62,7 @@
\
asm volatile ( \
"0: lr.w" lr_sfx " %[old], %[ptr_]\n" \
- " andn %[scratch], %[old], %[mask]\n" \
+ ANDN_INSN("%[scratch]", "%[old]", "%[mask]") \
" or %[scratch], %[scratch], %z[new_]\n" \
" sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \
" bnez %[scratch], 0b\n" \
And with that, everything builds... but doesn't link. I've got this:
LDS arch/riscv/xen.lds
riscv64-linux-gnu-ld -T arch/riscv/xen.lds -N prelink.o \
./common/symbols-dummy.o -o ./.xen-syms.0
riscv64-linux-gnu-ld: prelink.o: in function `keyhandler_crash_action':
/local/xen.git/xen/common/keyhandler.c:552: undefined reference to `guest_physmap_remove_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `guest_physmap_remove_page' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
which is completely bizarre.
keyhandler_crash_action() has no actual reference to guest_physmap_remove_page(), and keyhandler.o has no such relocation.
I'm still investigating this one.
~Andrew
Re: [PATCH v12 1/8] xen/riscv: disable unnecessary configs
In reply to Andrew CooperQuoted 4 lines · Andrew Cooper
Quoted 3 lines · Oleksii Kurochko
To allow
CONFIG_ARGObuild happy it was included <asm/p2m.h> to <asm/domain.h> as ARGO requiresp2m_type_t(p2m_ram_rw) and declaration ofcheck_get_page_from_gfn()fromxen/p2m-common.h.
Actually, this is an ARGO bug not a RISC-V bug.
It only builds in other architectures because of transitive dependencies through asm/domain.h
I've fixed up argo.c to include the right header, which also prevents (for now at least) adding to general tangle which is asm/domain.h
~Andrew
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Andrew CooperQuoted 15 lines · Andrew Cooper
README provided by your OS distributor: · line 53
Quoted 11 lines · Oleksii Kurochko
README +3
-0@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or laterI would like to petition for GCC 10 and Binutils 2.35.
Quoted 2 lines
These are the versions provided as cross-compilers by Debian, and therefore are the versions I would prefer to do smoke testing with.
See why I asked to amend the specified versions by a softening sentence that you (only now) said you dislike? The "this is what we use in CI" makes it a very random choice, entirely unrelated to the compiler's abilities.
Jan
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Andrew CooperQuoted 6 lines · Andrew Cooper
Quoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend on having
bitops.hworking in practice?
The latter, iirc. In fact I had already tried at least twice to pull this ahead.
Jan
Re: [PATCH v12 2/8] xen: introduce generic non-atomic test_*bit()
In reply to Oleksii KurochkoQuoted 26 lines · Oleksii Kurochko
The following generic functions were introduced:
*test_bit
*generic__test_and_set_bit
*generic__test_and_clear_bit
*generic__test_and_change_bitThese functions and macros can be useful for architectures that don't have corresponding arch-specific instructions.
Also, the patch introduces the following generics which are used by the functions mentioned above:
*BITOP_BITS_PER_WORD
*BITOP_MASK
*BITOP_WORD
*BITOP_TYPEThe following approach was chosen for generic*() and arch*() bit operation functions:
If the bit operation function that is going to be generic starts with the prefix "__", then the corresponding generic/arch function will also contain the "__" prefix. For example:
*test_bit()will be defined usingarch_test_bit()andgeneric_test_bit().
*__test_and_set_bit()will be defined usingarch__test_and_set_bit()andgeneric__test_and_set_bit().
Quoted 1 line
Signed-off-by Oleksii Kurochko <[email protected]>
Albeit I think you cound have gone further ...
xen/arch/x86/include/asm/bitops.h static inline int variable_test_bit(int nr, const volatile void *addr) · line 301
Quoted 10 lines
xen/arch/x86/include/asm/bitops.h +1
-2@@ -307,8 +295,7 @@ static inline int variable_test_bit(int nr, const volatile void *addr)return oldbit; } -#define test_bit (nr, addr) ({ \ - if ( bitop_bad_size(addr) ) __bitop_bad_size(); \ +#define arch_test_bit (nr, addr) ({ \ __builtin_constant_p(nr) ? \ constant_test_bit(nr, addr) : \ variable_test_bit(nr, addr); \
... here, as constant_test_bit() is functionally the same as generic_test_bit(), afaict. But that can well be cleaned up subsequently, in order to no further delay this work of yours.
Jan
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Andrew CooperQuoted 15 lines · Andrew Cooper
Quoted 11 lines · Oleksii Kurochko
README +0
-0@@ -48,6 +48,10 @@ provided by your OS distributor:- For ARM 64-bit:
- GCC 5.1 or later
- GNU Binutils 2.24 or later
+ - For RISC-V 64-bit:
+ - GCC 12.2 or later
+ - GNU Binutils 2.39 or laterI would like to petition for GCC 10 and Binutils 2.35.
Quoted 2 lines
These are the versions provided as cross-compilers by Debian, and therefore are the versions I would prefer to do smoke testing with.
We can consider that, but I prefer to make a separate patch series for that.
~ Oleksii
Quoted 95 lines
One issue is in
cpu_relax(), needing this diff to fix:xen/arch/riscv/include/asm/processor.h +1
-1@@ -67,7 +67,7 @@ static inline void cpu_relax(void) __asm__ __volatile__ ( "pause" ); #else /* Encoding of the pause instruction */ - __asm__ __volatile__ ( ".insn 0x0100000F" ); + __asm__ __volatile__ ( ".4byte 0x0100000F" ); #endif
barrier();The .insn directive appears to check that the byte pattern is a known extension, where .4byte doesn't. AFAICT, this makes .insn pretty useless for what I'd expect is it's main purpose...
The other problem is a real issue in
cmpxchg.h, already committed to staging (51dabd6312c).RISC-V does a conditional toolchain for the Zbb extension (
xen/arch/riscv/rules.mk), but unconditionally uses the ANDN instruction
inemulate_xchg_1_2().Nevertheless, this is also quite easy to fix:
xen/arch/riscv/include/asm/cmpxchg.h +0
-0@@ -18,6 +18,20 @@ : "r" (new) \ : "memory" );
+/*
+ * Binutils < 2.37 doesn't understand ANDN. If the toolchain is too old, form
+ * it of a NOT+AND pair
+ */xen/arch/riscv/include/asm/cmpxchg.h +10
-0+#ifdef __riscv_zbb +#define ANDN_INSN (rd, rs1, rs2) \ + "andn " rd ", " rs1 ", " rs2 "\n" +#else +#define ANDN_INSN (rd, rs1, rs2) \ + "not " rd ", " rs2 "\n" \ + "and " rd ", " rs1 ", " rd "\n" +#endif + +/*
* For LR and SC, the A extension requires that the address held in rs1 be
* naturally aligned to the size of the operand (i.e., eight-byte alignedxen/arch/riscv/include/asm/cmpxchg.h +1
-1@@ -48,7 +62,7 @@ \ asm volatile ( \ "0: lr.w" lr_sfx " %[old], %[ptr_]\n" \ - " andn %[scratch], %[old], %[mask]\n" \ + ANDN_INSN("%[scratch]", "%[old]", "%[mask]") \ " or %[scratch], %[scratch], %z[new_]\n" \ " sc.w" sc_sfx " %[scratch], %[scratch], %[ptr_]\n" \ " bnez %[scratch], 0b\n" \And with that, everything builds... but doesn't link. I've got this:
LDS
arch/riscv/xen.lds
riscv64-linux-gnu-ld -Tarch/riscv/xen.lds-N prelink.o \
./common/symbols-dummy.o-o ./.xen-syms.0
riscv64-linux-gnu-ld: prelink.o: in function `keyhandler_crash_action':
/local/xen.git/xen/common/keyhandler.c:552: undefined reference to `guest_physmap_remove_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `guest_physmap_remove_page' isn't defined
riscv64-linux-gnu-ld: final link failed: bad valuewhich is completely bizarre.
keyhandler_crash_action()has no actual reference toguest_physmap_remove_page(), and keyhandler.o has no such relocation.I'm still investigating this one.
~Andrew
Re: [PATCH v12 8/8] xen/README: add compiler and binutils versions for RISC-V64
In reply to Jan BeulichQuoted 15 lines · Jan Beulich
Quoted 14 lines · Andrew Cooper
README provided by your OS distributor: · line 53
Quoted 11 lines · Oleksii Kurochko
README +3
-0@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or laterI would like to petition for GCC 10 and Binutils 2.35.
Quoted 5 lines
Quoted 2 lines
These are the versions provided as cross-compilers by Debian, and therefore are the versions I would prefer to do smoke testing with.
See why I asked to amend the specified versions by a softening sentence that you (only now) said you dislike? The "this is what we use in CI" makes it a very random choice, entirely unrelated to the compiler's abilities.
"what's in CI" is an arbitrary choice, and that's *explicitly* fine and the right choice for Oleksii to have made.
He's got the hard job of making the damn thing work in the first place. Requiring him to also go and get some old compilers to backdate the support statement is unreasonable for you to demand.
In this case, I'm saying that it would be convenient for *me* if the numbers were older, because that's what *I* have and what *I'm* wanting to testing with. This means that I'm the one taking on the responsibility of playing backwards-compatibility-roulette.
Now, for other reasons I no longer have those versions, but one of the 3 bugs I raises is still a real bug needing fixing.
~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Andrew CooperQuoted 25 lines · Andrew Cooper
Quoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend on
havingbitops.hworking in practice?However, one very strong suggestion...
Quoted 14 lines
xen/arch/riscv/include/asm/mm.h +4
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures. */
+#define
mfn_to_page(mfn) \
Quoted 8 lines
Quoted 5 lines
+ (
frame_table+ (mfn_to_pdx(mfn)-frametable_base_pdx)) +#define
page_to_mfn(pg) \
+pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)Do yourself a favour and not introduce
frametable_base_pdxto begin with.
To drop frametable_base_pdx if the following changes will be enough:
diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h
index cc4a07a71c..fdac7e0646 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -107,14 +107,11 @@ struct page_info
#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
-/* PDX of the first page in the frame table. */ -extern unsigned long frametable_base_pdx; -
/* Convert between machine frame numbers and page-info structures. */
#define mfn_to_page(mfn) \
- (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+ (frame_table + mfn - FRAMETABLE_BASE_OFFSET))
#define page_to_mfn(pg) \ - pdx_to_mfn((unsigned long)((pg) - frame_table) +
frametable_base_pdx)
+ ((unsigned long)((pg) - frame_table) + FRAMETABLE_BASE_OFFSET)
static inline void *page_to_virt(const struct page_info *pg) {
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 9c0fd80588..8f6dbdc699 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -15,7 +15,7 @@
#include <asm/page.h>
#include <asm/processor.h> -unsigned long __ro_after_init frametable_base_pdx;
unsigned long __ro_after_init frametable_virt_end;
struct mmu_desc {Quoted 13 lines
Every RISC-V board I can find has things starting from 0 in physical address space, with RAM starting immediately after.
Taking the microchip board as an example, RAM actually starts at
0x8000000, which means that havingframetable_base_pdxand assuming it
does get set to0x8000(which isn't even a certainty, given that I think
you'll need struct pages covering the PLICs), then what you are trading
off is:
Quoted 4 lines
* Saving 32k of virtual address space only (no need to even allocate memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn conversion, which is a screaming hotpath all over Xen.
Are you referring here to mfn_to_pdx() used in mfn_to_page() and pdx_to_mfn() in page_to_mfn()?
My expectation was that when CONFIG_PDX_COMPRESSION is disabled then this macros doesn't do anything:
/* pdx<->pfn == identity */
#define pdx_to_pfn(x) (x)
#define pfn_to_pdx(x) (x)
Quoted 7 lines
It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally wouldn't - especially as there's no need to share Xen's VA space with guests, given no PV guests on ARM/RISC-V), but it's absolutely not at all in an a 64bit build with TB of VA space available.
Why 32k? If RAM is started at 0x8000_0000 then we have to cover 0x80000 entries, and the size of it is 0x8_0000 = 524288 * 64 = 33554432, so it is 32 Mb.
Am I confusing something?
P.S.: Should I map this start 32 mb? Or it is enough to have a slide ( FRAMETABLE_BASE_OFFSET ).
~ Oleksii
Quoted 11 lines
Even if we do find a board with the first interesting thing in the frametable starting sufficiently away from 0 that it might be worth considering this slide, then it should still be Kconfig-able in a similar way to
PDX_COMPRESSION.You don't want to be penalising everyone because of a theoretical/weird
corner case.~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Oleksii K.Quoted 116 lines · Oleksii K.
Quoted 32 lines · Andrew Cooper
Quoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend on
havingbitops.hworking in practice?However, one very strong suggestion...
Quoted 11 lines
xen/arch/riscv/include/asm/mm.h +4
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures. Quoted 8 lines
*/
+#define
mfn_to_page(mfn) \
+ (frame_table+ (mfn_to_pdx(mfn)-frametable_base_pdx)) +#define
page_to_mfn(pg) \
+pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)Do yourself a favour and not introduce
frametable_base_pdxto begin with.Quoted 2 lines
Every RISC-V board I can find has things starting from 0 in physical address space, with RAM starting immediately after.
I checked Linux kernel and grep there:
[ok@fedora linux-aia]$ grep -Rni "memory@" arch/riscv/boot/dts/ -- exclude "*.tmp" -I arch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi:33: memory@40000000 { arch/riscv/boot/dts/starfive/jh7100-common.dtsi:28: memory@80000000 { arch/riscv/boot/dts/microchip/mpfs-sev-kit.dts:49: ddrc_cache: memory@1000000000 { arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:33: ddrc_cache_lo: memory@80000000 { arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:37: ddrc_cache_hi: memory@1040000000 { arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:34: ddrc_cache_lo: memory@80000000 { arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:40: ddrc_cache_hi: memory@1000000000 { arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:22: ddrc_cache_lo: memory@80000000 { arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:27: ddrc_cache_hi: memory@1000000000 { arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:57: ddrc_cache_lo: memory@80000000 { arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:63: ddrc_cache_hi: memory@1040000000 { arch/riscv/boot/dts/thead/th1520-beaglev-ahead.dts:32: memory@0 { arch/riscv/boot/dts/thead/th1520-lichee-module-4a.dtsi:14: memory@0 { arch/riscv/boot/dts/sophgo/cv1800b-milkv-duo.dts:26: memory@80000000 { arch/riscv/boot/dts/sophgo/cv1812h.dtsi:12: memory@80000000 { arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts:26: memory@80000000 { arch/riscv/boot/dts/sifive/hifive-unleashed-a00.dts:25: memory@80000000 { arch/riscv/boot/dts/canaan/k210.dtsi:82: sram: memory@80000000 {And based on that majority of supported by Linux kernel boards has RAM starting not from 0 in physical address space. Am I confusing something?
Quoted 2 lines
Taking the microchip board as an example, RAM actually starts at
0x8000000,Today we had conversation with the guy from SiFive in xen-devel channel and he mentioned that they are using "starfive visionfive2 and sifive unleashed platforms." which based on the grep above has RAM not at 0 address.
Also, QEMU uses
0x8000000.Quoted 19 lines
which means that having
frametable_base_pdxand assuming it
does get set to0x8000(which isn't even a certainty, given that I think
you'll need struct pages covering the PLICs), then what you are trading
off is:* Saving 32k of virtual address space only (no need to even allocate memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn conversion, which is a screaming hotpath all over Xen.It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally wouldn't - especially as there's no need to share Xen's VA space with guests, given no PV guests on ARM/RISC-V), but it's absolutely not at all in an a 64bit build with TB of VA space available.
Quoted 4 lines
Even if we do find a board with the first interesting thing in the frametable starting sufficiently away from 0 that it might be worth considering this slide, then it should still be Kconfig-able in a similar way to
PDX_COMPRESSION.I found your tradeoffs reasonable, but I don't understand how it will work if RAM does not start from 0, as the frametable address and RAM address are linked.
I tried to look at thePDX_COMPRESSIONconfig and couldn't find any "slide" there. Could you please clarify this for me?
If to use this "slide" would it help to avoid the mentioned above tradeoffs?
Quoted 3 lines
One more question: if we decide to go without
frametable_base_pdx, would it be sufficient to simply remove mentions of it from the code ( at least, for now )?
There is a relationship between system/host physical addresses (what Xen called maddr/mfn), and the frametable. The frametable has one entry per mfn.
In the most simple case, there's a 1:1 relationship. i.e. frametable[0] = maddr(0), frametable[1] = maddr(4k) etc. This is very simple, and very easy to calculate (page_to_mfn()/mfn_to_page()).
The frametable is one big array. It starts at a compile-time fixed address, and needs to be long enough to cover everything interesting in memory. Therefore it potentially takes a large amount of virtual address space.
However, only interesting maddrs need to have data in the frametable, so it's fine for the backing RAM to be sparsely allocated/mapped in the frametable virtual addresses.
For 64bit, that's really all you need, because there's always far more virtual address space than physical RAM in the system, even when you're looking at TB-scale giant servers.
For 32bit, virtual address space is a limited resource. (Also to an extent, 64bit x86 with PV guests because we give 98% of the virtual address space to the guest kernel to use.)
There are two tricks to reduce the virtual address space used, but they both cost performance in fastpaths.
1) PDX Compression.
PDX compression makes a non-linear mfn <-> maddr mapping. This is for a usecase where you've got multiple RAM banks which are separated by a large distance (and evenly spaced), then you can "compress" a single range of 0's out of the middle of the system/host physical address.
The cost is that all page <-> mfn conversions need to read two masks and a shift-count from variables in memory, to split/shift/recombine the address bits.
2) A slide, which is frametable_base_pdx in this context.
When there's a big gap between 0 and the start of something interesting, you could chop out that range by just subtracting base_pdx. What qualifies as "big" is subjective, but Qemu starting at 128M certainly does not qualify as big enough to warrant frametable_base_pdx.
This is less expensive that PDX compression. It only adds one memory read to the fastpath, but it also doesn't save as much virtual address space as PDX compression.
When virtual address space is a major constraint (32 bit builds), both of these techniques are worth doing. But when there's no constraint on virtual address space (64 bit builds), there's no reason to use either; and the performance will definitely improve as a result.
~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
Quoted 221 lines · Andrew Cooper
Quoted 151 lines · Oleksii K.
Quoted 35 lines · Andrew Cooper
Quoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend
on
havingbitops.hworking in practice?However, one very strong suggestion...
Quoted 12 lines
xen/arch/riscv/include/asm/mm.h +4
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures.
Quoted 8 lines
*/
+#define
mfn_to_page(mfn) \
+ (frame_table+ (mfn_to_pdx(mfn)-frametable_base_pdx)) +#define
page_to_mfn(pg) \
+pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)Do yourself a favour and not introduce
frametable_base_pdxto begin
with.Quoted 3 lines
Every RISC-V board I can find has things starting from 0 in physical
address space, with RAM starting immediately after.I checked Linux kernel and grep there:
[ok@fedora linux-aia]$ grep -Rni "memory@"arch/riscv/boot/dts/--
exclude "*.tmp" -Iarch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi:33:
memory@40000000 {arch/riscv/boot/dts/starfive/jh7100-common.dtsi:28:
memory@80000000
{
arch/riscv/boot/dts/microchip/mpfs-sev-kit.dts:49:ddrc_cache:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:33:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:37:ddrc_cache_hi:
memory@1040000000 {arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:34:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:40:ddrc_cache_hi:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:22:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:27:ddrc_cache_hi:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:57:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:63:ddrc_cache_hi:
memory@1040000000 { arch/riscv/boot/dts/thead/th1520-beaglev-ahead.dts:32: memory@0 {arch/riscv/boot/dts/thead/th1520-lichee-module-4a.dtsi:14:
memory@0 {arch/riscv/boot/dts/sophgo/cv1800b-milkv-duo.dts:26:
memory@80000000
{
arch/riscv/boot/dts/sophgo/cv1812h.dtsi:12: memory@80000000 {arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts:26:
memory@80000000
{arch/riscv/boot/dts/sifive/hifive-unleashed-a00.dts:25:
memory@80000000
{
arch/riscv/boot/dts/canaan/k210.dtsi:82: sram:
memory@80000000 {
And based on that majority of supported by Linux kernel boards has RAM
starting not from 0 in physical address space. Am I confusing something?Quoted 2 lines
Taking the microchip board as an example, RAM actually starts at
0x8000000,Today we had conversation with the guy from SiFive in xen-devel channel
and he mentioned that they are using "starfive visionfive2 and sifive
unleashed platforms." which based on the grep above has RAM not at 0
address.Also, QEMU uses
0x8000000.Quoted 24 lines
which means that having
frametable_base_pdxand assuming it does get set to0x8000(which isn't even a certainty, given that I
think
you'll need struct pages covering the PLICs), then what you are trading
off is:* Saving 32k of virtual address space only (no need to even allocate
memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn conversion, which is a screaming hotpath all over Xen.It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally
wouldn't - especially as there's no need to share Xen's VA space with
guests, given no PV guests on ARM/RISC-V), but it's absolutely not at
all in an a 64bit build with TB of VA space available.Quoted 6 lines
Even if we do find a board with the first interesting thing in the
frametable starting sufficiently away from 0 that it might be worth
considering this slide, then it should still be Kconfig-able in a similar way toPDX_COMPRESSION.I found your tradeoffs reasonable, but I don't understand how it will
work if RAM does not start from 0, as the frametable address and RAM
address are linked.
I tried to look at thePDX_COMPRESSIONconfig and couldn't find any "slide" there. Could you please clarify this for me?
If to use this "slide" would it help to avoid the mentioned above tradeoffs?Quoted 4 lines
One more question: if we decide to go without
frametable_base_pdx, would it be sufficient to simply remove mentions of it from the code (
at least, for now )?There is a relationship between system/host physical addresses (what Xen
called maddr/mfn), and the frametable. The frametable has one entry per
mfn.In the most simple case, there's a 1:1 relationship. i.e. frametable[0]
= maddr(0), frametable[1] = maddr(4k) etc. This is very simple, and very easy to calculate (page_to_mfn()/mfn_to_page()).The frametable is one big array. It starts at a compile-time fixed address, and needs to be long enough to cover everything interesting in
memory. Therefore it potentially takes a large amount of virtual address space.However, only interesting maddrs need to have data in the frametable, so
it's fine for the backing RAM to be sparsely allocated/mapped in the frametable virtual addresses.For 64bit, that's really all you need, because there's always far more
virtual address space than physical RAM in the system, even when you're
looking at TB-scale giant servers.For 32bit, virtual address space is a limited resource. (Also to an extent, 64bit x86 with PV guests because we give 98% of the virtual address space to the guest kernel to use.)
There are two tricks to reduce the virtual address space used, but they
both cost performance in fastpaths.1) PDX Compression.
PDX compression makes a non-linear mfn <-> maddr mapping. This is for a
usecase where you've got multiple RAM banks which are separated by a large distance (and evenly spaced), then you can "compress" a single range of 0's out of the middle of the system/host physical address.The cost is that all page <-> mfn conversions need to read two masks and
a shift-count from variables in memory, to split/shift/recombine the address bits.2) A slide, which is
frametable_base_pdxin this context.When there's a big gap between 0 and the start of something interesting,
you could chop out that range by just subtracting base_pdx. What qualifies as "big" is subjective, but Qemu starting at 128M certainly does not qualify as big enough to warrantframetable_base_pdx.This is less expensive that PDX compression. It only adds one memory read to the fastpath, but it also doesn't save as much virtual address
space as PDX compression.
Quoted 7 lines
When virtual address space is a major constraint (32 bit builds), both
of these techniques are worth doing. But when there's no constraint on
virtual address space (64 bit builds), there's no reason to use either;
and the performance will definitely improve as a result.
Thanks for such good explanation.
For RISC-V we have PDX config disabled as I haven't seen multiple RAM banks at boards which has hypervisor extension. Thereby mfn_to_pdx() and pdx_to_mfn() do nothing. The same for frametable_base_pdx, in case of PDX disabled, it just an offset ( or a slide ).
IIUUC, you meant that it make sense to map frametable not to the address of where RAM is started, but to 0, so then we don't need this +-frametable_base_pdx. The price for that is loosing of VA space for the range from 0 to RAM start address.
Right now, we are trying to support 3 boards with the following RAM address:
1. 0x8000_0000 - QEMU and SiFive board
2. 0x40_0000_0000 - Microchip board
So if we mapping frametable to 0 ( not RAM start ) we will loose:
1. 0x8000_0 ( amount of page entries to cover range [0, 0x8000_0000] )
* 64 ( size of struct page_info ) = 32 MB
2. 0x40_0000_0 ( amount of page entries to cover range [0, 0x40_0000_0000] ) * 64 ( size of struct page_info ) = 4 Gb
In terms of available virtual address space for RV-64 we can consider both options as acceptable.
[OPTION 1] If we accepting of loosing 4 Gb of VA then we could implement mfn_to_page() and page_to_mfn() in the following way:
```
diff --git a/xen/arch/riscv/include/asm/mm.h
b/xen/arch/riscv/include/asm/mm.h
index cc4a07a71c..fdac7e0646 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -107,14 +107,11 @@ struct page_info
#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
-/* PDX of the first page in the frame table. */
-extern unsigned long frametable_base_pdx;
-
/* Convert between machine frame numbers and page-info structures.
*/
#define mfn_to_page(mfn)
\
- (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+ (frame_table + mfn))
#define page_to_mfn(pg)
\
- pdx_to_mfn((unsigned long)((pg) - frame_table) +
frametable_base_pdx)
+ ((unsigned long)((pg) - frame_table))
static inline void *page_to_virt(const struct page_info *pg)
{
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 9c0fd80588..8f6dbdc699 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -15,7 +15,7 @@
#include <asm/page.h>
#include <asm/processor.h>
-unsigned long __ro_after_init frametable_base_pdx;
unsigned long __ro_after_init frametable_virt_end;
struct mmu_desc {
```[OPTION 2] If we are not accepting of loosing 4 Gb of VA I think that there is no sense to rework that in the following way:
```
diff --git a/xen/arch/riscv/include/asm/mm.h
b/xen/arch/riscv/include/asm/mm.h
index cc4a07a71c..fdac7e0646 100644
--- a/xen/arch/riscv/include/asm/mm.h
+++ b/xen/arch/riscv/include/asm/mm.h
@@ -107,14 +107,11 @@ struct page_info
#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START)
-/* PDX of the first page in the frame table. */
-extern unsigned long frametable_base_pdx;
-
/* Convert between machine frame numbers and page-info structures.
*/
#define mfn_to_page(mfn)
\
- (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
+ (frame_table + mfn - FRAMETABLE_BASE_OFFSET))
#define page_to_mfn(pg)
\
- pdx_to_mfn((unsigned long)((pg) - frame_table) +
frametable_base_pdx)
+ ((unsigned long)((pg) - frame_table) + FRAMETABLE_BASE_OFFSET)
static inline void *page_to_virt(const struct page_info *pg)
{
diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c
index 9c0fd80588..8f6dbdc699 100644
--- a/xen/arch/riscv/mm.c
+++ b/xen/arch/riscv/mm.c
@@ -15,7 +15,7 @@
#include <asm/page.h>
#include <asm/processor.h>
-unsigned long __ro_after_init frametable_base_pdx;
unsigned long __ro_after_init frametable_virt_end;
struct mmu_desc {
```And I am not sure that there is any sense in option 2 as basically it the same to have the following macros definition with PDX disabled:
```
/* Convert between machine frame numbers and page-info structures.
*/
#define mfn_to_page(mfn)
\
(frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx))
#define page_to_mfn(pg)
\
pdx_to_mfn((unsigned long)((pg) - frame_table) +
frametable_base_pdx)
```The only sense of option 2 is the case when FRAMETABLE_BASE_OFFSET is equal to 0, so the compiler will generate the code without additional sub/add of FRAMETABLE_BASE_OFFSET.
Could you please clarify if my understanding is correct?
Should we still change the definition of mfn_to_page() and page_to_mfn() with the fact that PDX is disabled? If yes, OPTION_1 is okay or I am missing something?
Thanks in advance.
~ Oleksii
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
Quoted 245 lines · Oleksii K.
Quoted 220 lines · Andrew Cooper
Quoted 151 lines · Oleksii K.
Quoted 35 lines · Andrew Cooper
Quoted 2 lines · Oleksii Kurochko
Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend
on
havingbitops.hworking in practice?However, one very strong suggestion...
Quoted 12 lines
xen/arch/riscv/include/asm/mm.h +4
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures.
Quoted 8 lines
*/
+#define
mfn_to_page(mfn) \
+ (frame_table+ (mfn_to_pdx(mfn)-frametable_base_pdx)) +#define
page_to_mfn(pg) \
+pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)Do yourself a favour and not introduce
frametable_base_pdxto begin
with.Quoted 3 lines
Every RISC-V board I can find has things starting from 0 in physical
address space, with RAM starting immediately after.I checked Linux kernel and grep there:
[ok@fedora linux-aia]$ grep -Rni "memory@"arch/riscv/boot/dts/--
exclude "*.tmp" -Iarch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi:33:
memory@40000000 {arch/riscv/boot/dts/starfive/jh7100-common.dtsi:28:
memory@80000000
{
arch/riscv/boot/dts/microchip/mpfs-sev-kit.dts:49:ddrc_cache:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:33:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:37:ddrc_cache_hi:
memory@1040000000 {arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:34:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:40:ddrc_cache_hi:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:22:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:27:ddrc_cache_hi:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:57:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:63:ddrc_cache_hi:
memory@1040000000 { arch/riscv/boot/dts/thead/th1520-beaglev-ahead.dts:32: memory@0 {arch/riscv/boot/dts/thead/th1520-lichee-module-4a.dtsi:14:
memory@0 {arch/riscv/boot/dts/sophgo/cv1800b-milkv-duo.dts:26:
memory@80000000
{
arch/riscv/boot/dts/sophgo/cv1812h.dtsi:12: memory@80000000 {arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts:26:
memory@80000000
{arch/riscv/boot/dts/sifive/hifive-unleashed-a00.dts:25:
memory@80000000
{
arch/riscv/boot/dts/canaan/k210.dtsi:82: sram:
memory@80000000 {
And based on that majority of supported by Linux kernel boards has RAM
starting not from 0 in physical address space. Am I confusing something?Quoted 2 lines
Taking the microchip board as an example, RAM actually starts at
0x8000000,Today we had conversation with the guy from SiFive in xen-devel channel
and he mentioned that they are using "starfive visionfive2 and sifive
unleashed platforms." which based on the grep above has RAM not at 0
address.Also, QEMU uses
0x8000000.Quoted 24 lines
which means that having
frametable_base_pdxand assuming it does get set to0x8000(which isn't even a certainty, given that I
think
you'll need struct pages covering the PLICs), then what you are trading
off is:* Saving 32k of virtual address space only (no need to even allocate
memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn conversion, which is a screaming hotpath all over Xen.It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally
wouldn't - especially as there's no need to share Xen's VA space with
guests, given no PV guests on ARM/RISC-V), but it's absolutely not at
all in an a 64bit build with TB of VA space available.Quoted 6 lines
Even if we do find a board with the first interesting thing in the
frametable starting sufficiently away from 0 that it might be worth
considering this slide, then it should still be Kconfig-able in a similar way toPDX_COMPRESSION.I found your tradeoffs reasonable, but I don't understand how it will
work if RAM does not start from 0, as the frametable address and RAM
address are linked.
I tried to look at thePDX_COMPRESSIONconfig and couldn't find any "slide" there. Could you please clarify this for me?
If to use this "slide" would it help to avoid the mentioned above tradeoffs?Quoted 4 lines
One more question: if we decide to go without
frametable_base_pdx, would it be sufficient to simply remove mentions of it from the code (
at least, for now )?There is a relationship between system/host physical addresses (what Xen
called maddr/mfn), and the frametable. The frametable has one entry per
mfn.In the most simple case, there's a 1:1 relationship. i.e. frametable[0]
= maddr(0), frametable[1] = maddr(4k) etc. This is very simple, and very easy to calculate (page_to_mfn()/mfn_to_page()).The frametable is one big array. It starts at a compile-time fixed address, and needs to be long enough to cover everything interesting in
memory. Therefore it potentially takes a large amount of virtual address space.However, only interesting maddrs need to have data in the frametable, so
it's fine for the backing RAM to be sparsely allocated/mapped in the frametable virtual addresses.For 64bit, that's really all you need, because there's always far more
virtual address space than physical RAM in the system, even when you're
looking at TB-scale giant servers.For 32bit, virtual address space is a limited resource. (Also to an extent, 64bit x86 with PV guests because we give 98% of the virtual address space to the guest kernel to use.)
There are two tricks to reduce the virtual address space used, but they
both cost performance in fastpaths.1) PDX Compression.
PDX compression makes a non-linear mfn <-> maddr mapping. This is for a
usecase where you've got multiple RAM banks which are separated by a large distance (and evenly spaced), then you can "compress" a single range of 0's out of the middle of the system/host physical address.The cost is that all page <-> mfn conversions need to read two masks and
a shift-count from variables in memory, to split/shift/recombine the address bits.2) A slide, which is
frametable_base_pdxin this context.When there's a big gap between 0 and the start of something interesting,
you could chop out that range by just subtracting base_pdx. What qualifies as "big" is subjective, but Qemu starting at 128M certainly does not qualify as big enough to warrantframetable_base_pdx.This is less expensive that PDX compression. It only adds one memory read to the fastpath, but it also doesn't save as much virtual address
space as PDX compression.Quoted 7 lines
When virtual address space is a major constraint (32 bit builds), both
of these techniques are worth doing. But when there's no constraint on
virtual address space (64 bit builds), there's no reason to use either;
and the performance will definitely improve as a result.Thanks for such good explanation.
For RISC-V we have PDX config disabled as I haven't seen multiple RAM banks at boards which has hypervisor extension. Thereby
mfn_to_pdx()andpdx_to_mfn()do nothing. The same forframetable_base_pdx, in case of PDX disabled, it just an offset ( or a slide ).IIUUC, you meant that it make sense to map frametable not to the address of where RAM is started, but to 0, so then we don't need this +-frametable_base_pdx. The price for that is loosing of VA space for the range from 0 to RAM start address.
Right now, we are trying to support 3 boards with the following RAM address:
1. 0x8000_0000 - QEMU and SiFive board
2. 0x40_0000_0000 - Microchip board
Quoted 8 lines
So if we mapping frametable to 0 ( not RAM start ) we will loose:
1. 0x8000_0 ( amount of page entries to cover range [0, 0x8000_0000] )
* 64 ( size of structpage_info) = 32 MB
2. 0x40_0000_0 ( amount of page entries to cover range [0, 0x40_0000_0000] ) * 64 ( size of structpage_info) = 4 GbIn terms of available virtual address space for RV-64 we can consider both options as acceptable.
For Qemu and SiFive, 32M is definitely not worth worrying about.
I personally wouldn't worry about Microchip either. That's 4G of 1T VA space (assuming Sv39).
Quoted 34 lines
[OPTION 1] If we accepting of loosing 4 Gb of VA then we could implement
mfn_to_page()andpage_to_mfn()in the following way:``` diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h index cc4a07a71c..fdac7e0646 100644 --- a/xen/arch/riscv/include/asm/mm.h +++ b/xen/arch/riscv/include/asm/mm.h @@ -107,14 +107,11 @@ struct page_info #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START) -/* PDX of the first page in the frame table. */ -extern unsigned long frametable_base_pdx; - /* Convert between machine frame numbers and page-info structures. */ #define mfn_to_page(mfn) \ - (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx)) + (frame_table + mfn)) #define page_to_mfn(pg) \ - pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx) + ((unsigned long)((pg) - frame_table)) static inline void *page_to_virt(const struct page_info *pg) { diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 9c0fd80588..8f6dbdc699 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -15,7 +15,7 @@
Quoted 8 lines
#include <asm/page.h>
#include <asm/processor.h>
-unsigned long __ro_after_init frametable_base_pdx;
unsigned long __ro_after_init frametable_virt_end;
struct mmu_desc {
```I firmly recommend option 1, especially at this point.
If specific boards really have a problem with losing 4G of VA space, then option 2 can be added easily at a later point.
That said, I'd think carefully about doing option 2. Even subtracting a constant - which is far better than subtracting a variable - is still extra overhead in fastpaths. Option 2 needs careful consideration on a board-by-board case.
~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
Quoted 356 lines · Andrew Cooper
Quoted 299 lines · Oleksii K.
Quoted 261 lines · Andrew Cooper
Quoted 173 lines · Oleksii K.
Quoted 39 lines · Andrew Cooper
Quoted 3 lines · Oleksii Kurochko
Signed-off-by: Oleksii Kurochko
<[email protected]>Acked-by Jan Beulich <[email protected]>This patch looks like it can go in independently? Or does it depend
on
havingbitops.hworking in practice?However, one very strong suggestion...
Quoted 15 lines
xen/arch/riscv/include/asm/mm.h +4
-0@@ -3,11 +3,246 @@ +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures.
*/
+#definemfn_to_page(mfn)Quoted 8 lines
\
+ (frame_table+ (mfn_to_pdx(mfn)-frametable_base_pdx))
+#definepage_to_mfn(pg)
\
+pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx)Do yourself a favour and not introduce
frametable_base_pdxto begin
with.Quoted 3 lines
Every RISC-V board I can find has things starting from 0 in physical
address space, with RAM starting immediately after.I checked Linux kernel and grep there:
[ok@fedora linux-aia]$ grep -Rni "memory@"arch/riscv/boot/dts/
--
exclude "*.tmp" -Iarch/riscv/boot/dts/starfive/jh7110-starfive-visionfive-2.dtsi:33:
memory@40000000 {arch/riscv/boot/dts/starfive/jh7100-common.dtsi:28:
memory@80000000
{arch/riscv/boot/dts/microchip/mpfs-sev-kit.dts:49:ddrc_cache:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:33:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-m100pfsevp.dts:37:ddrc_cache_hi:
memory@1040000000 {arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:34:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-tysom-m.dts:40:ddrc_cache_hi:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:22:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-polarberry.dts:27:ddrc_cache_hi:
memory@1000000000 {arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:57:ddrc_cache_lo:
memory@80000000 {arch/riscv/boot/dts/microchip/mpfs-icicle-kit.dts:63:ddrc_cache_hi:
memory@1040000000 {arch/riscv/boot/dts/thead/th1520-beaglev-ahead.dts:32:
memory@0
{
arch/riscv/boot/dts/thead/th1520-lichee-module-4a.dtsi:14:
memory@0 {arch/riscv/boot/dts/sophgo/cv1800b-milkv-duo.dts:26:
memory@80000000
{
arch/riscv/boot/dts/sophgo/cv1812h.dtsi:12:
memory@80000000
{arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts:26:
memory@80000000
{arch/riscv/boot/dts/sifive/hifive-unleashed-a00.dts:25:
memory@80000000
{
arch/riscv/boot/dts/canaan/k210.dtsi:82: sram:
memory@80000000 {
And based on that majority of supported by Linux kernel boards has
RAM
starting not from 0 in physical address space. Am I confusing something?Quoted 3 lines
Taking the microchip board as an example, RAM actually starts at
0x8000000,Today we had conversation with the guy from SiFive in xen-devel channel
and he mentioned that they are using "starfive visionfive2 and sifive
unleashed platforms." which based on the grep above has RAM not at
0
address.Also, QEMU uses
0x8000000.Quoted 29 lines
which means that having
frametable_base_pdxand assuming it does get set to0x8000(which isn't even a certainty, given that
I
think
you'll need struct pages covering the PLICs), then what you are
trading
off is:* Saving 32k of virtual address space only (no need to even allocate
memory for this range of the framtable), by
* Having an extra memory load and add/sub in every page <-> mfn
conversion, which is a screaming hotpath all over Xen.It's a terribly short-sighted tradeoff.
32k of VA space might be worth saving in a 32bit build (I personally
wouldn't - especially as there's no need to share Xen's VA space
with
guests, given no PV guests on ARM/RISC-V), but it's absolutely
not at
all in an a 64bit build with TB of VA space available.Quoted 8 lines
Even if we do find a board with the first interesting thing in
the
frametable starting sufficiently away from 0 that it might be worth
considering this slide, then it should still be Kconfig-able in a
similar way toPDX_COMPRESSION.I found your tradeoffs reasonable, but I don't understand how it
will
work if RAM does not start from 0, as the frametable address and
RAM
address are linked.
I tried to look at thePDX_COMPRESSIONconfig and couldn't find any
"slide" there. Could you please clarify this for me? If to use this "slide" would it help to avoid the mentioned above
tradeoffs?Quoted 5 lines
One more question: if we decide to go without
frametable_base_pdx,
would it be sufficient to simply remove mentions of it from the code (
at least, for now )?There is a relationship between system/host physical addresses (what
Xen
called maddr/mfn), and the frametable. The frametable has one entry
per
mfn.In the most simple case, there's a 1:1 relationship. i.e. frametable[0]
= maddr(0), frametable[1] = maddr(4k) etc. This is very simple, and
very easy to calculate (page_to_mfn()/mfn_to_page()).The frametable is one big array. It starts at a compile-time fixed
address, and needs to be long enough to cover everything interesting
in
memory. Therefore it potentially takes a large amount of virtual address space.However, only interesting maddrs need to have data in the frametable,
so
it's fine for the backing RAM to be sparsely allocated/mapped in the
frametable virtual addresses.For 64bit, that's really all you need, because there's always far more
virtual address space than physical RAM in the system, even when you're
looking at TB-scale giant servers.For 32bit, virtual address space is a limited resource. (Also to an
extent, 64bit x86 with PV guests because we give 98% of the virtual
address space to the guest kernel to use.)There are two tricks to reduce the virtual address space used, but
they
both cost performance in fastpaths.1) PDX Compression.
PDX compression makes a non-linear mfn <-> maddr mapping. This is
for a
usecase where you've got multiple RAM banks which are separated by a
large distance (and evenly spaced), then you can "compress" a single
range of 0's out of the middle of the system/host physical address.The cost is that all page <-> mfn conversions need to read two masks
and
a shift-count from variables in memory, to split/shift/recombine the
address bits.2) A slide, which is
frametable_base_pdxin this context.When there's a big gap between 0 and the start of something interesting,
you could chop out that range by just subtracting base_pdx. What qualifies as "big" is subjective, but Qemu starting at 128M certainly
does not qualify as big enough to warrantframetable_base_pdx.This is less expensive that PDX compression. It only adds one memory
read to the fastpath, but it also doesn't save as much virtual address
space as PDX compression.Quoted 8 lines
When virtual address space is a major constraint (32 bit builds), both
of these techniques are worth doing. But when there's no constraint
on
virtual address space (64 bit builds), there's no reason to use either;
and the performance will definitely improve as a result.Thanks for such good explanation.
For RISC-V we have PDX config disabled as I haven't seen multiple RAM
banks at boards which has hypervisor extension. Therebymfn_to_pdx()
andpdx_to_mfn()do nothing. The same forframetable_base_pdx, in case
of PDX disabled, it just an offset ( or a slide ).IIUUC, you meant that it make sense to map frametable not to the address of where RAM is started, but to 0, so then we don't need this
+-frametable_base_pdx. The price for that is loosing of VA space for
the range from 0 to RAM start address.Right now, we are trying to support 3 boards with the following RAM address:
1. 0x8000_0000 - QEMU and SiFive board
2. 0x40_0000_0000 - Microchip boardSo if we mapping frametable to 0 ( not RAM start ) we will loose:
1. 0x8000_0 ( amount of page entries to cover range [0, 0x8000_0000] )
* 64 ( size of structpage_info) = 32 MB
2. 0x40_0000_0 ( amount of page entries to cover range [0, 0x40_0000_0000] ) * 64 ( size of structpage_info) = 4 GbQuoted 3 lines
In terms of available virtual address space for RV-64 we can consider
both options as acceptable.For Qemu and SiFive, 32M is definitely not worth worrying about.
I personally wouldn't worry about Microchip either. That's 4G of 1T VA
space (assuming Sv39).Quoted 37 lines
[OPTION 1] If we accepting of loosing 4 Gb of VA then we could implement
mfn_to_page()andpage_to_mfn()in the following way:``` diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h index cc4a07a71c..fdac7e0646 100644 --- a/xen/arch/riscv/include/asm/mm.h +++ b/xen/arch/riscv/include/asm/mm.h @@ -107,14 +107,11 @@ struct page_info #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START) -/* PDX of the first page in the frame table. */ -extern unsigned long frametable_base_pdx; - /* Convert between machine frame numbers and page-info structures. */ #define mfn_to_page(mfn) \ - (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx)) + (frame_table + mfn)) #define page_to_mfn(pg) \ - pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx) + ((unsigned long)((pg) - frame_table)) static inline void *page_to_virt(const struct page_info *pg) { diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 9c0fd80588..8f6dbdc699 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -15,7 +15,7 @@Quoted 8 lines
#include <asm/page.h> #include <asm/processor.h> -unsigned long __ro_after_init frametable_base_pdx; unsigned long __ro_after_init frametable_virt_end; struct mmu_desc { ```
Quoted 1 line
I firmly recommend option 1, especially at this point.
Jan, as you gave your Acked before, don't you mind to define mfn_to_page() and page_to_mfn as mentioned above ( Option 1 )?
~ Oleksi
Quoted 12 lines
If specific boards really have a problem with losing 4G of VA space, then option 2 can be added easily at a later point.
That said, I'd think carefully about doing option 2. Even subtracting a
constant - which is far better than subtracting a variable - is still extra overhead in fastpaths. Option 2 needs careful consideration on a
board-by-board case.~Andrew
Re: [PATCH v12 5/8] xen/riscv: add minimal stuff to mm.h to build full Xen
Quoted 48 lines · Oleksii K.
Quoted 47 lines · Andrew Cooper
Quoted 45 lines · Oleksii K.
[OPTION 1] If we accepting of loosing 4 Gb of VA then we could implement
mfn_to_page()andpage_to_mfn()in the following way:``` diff --git a/xen/arch/riscv/include/asm/mm.h b/xen/arch/riscv/include/asm/mm.h index cc4a07a71c..fdac7e0646 100644 --- a/xen/arch/riscv/include/asm/mm.h +++ b/xen/arch/riscv/include/asm/mm.h @@ -107,14 +107,11 @@ struct page_info #define frame_table ((struct page_info *)FRAMETABLE_VIRT_START) -/* PDX of the first page in the frame table. */ -extern unsigned long frametable_base_pdx; - /* Convert between machine frame numbers and page-info structures. */ #define mfn_to_page(mfn) \ - (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx)) + (frame_table + mfn)) #define page_to_mfn(pg) \ - pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx) + ((unsigned long)((pg) - frame_table)) static inline void *page_to_virt(const struct page_info *pg) { diff --git a/xen/arch/riscv/mm.c b/xen/arch/riscv/mm.c index 9c0fd80588..8f6dbdc699 100644 --- a/xen/arch/riscv/mm.c +++ b/xen/arch/riscv/mm.c @@ -15,7 +15,7 @@ #include <asm/page.h> #include <asm/processor.h> -unsigned long __ro_after_init frametable_base_pdx; unsigned long __ro_after_init frametable_virt_end; struct mmu_desc { ```
Quoted 3 lines
Quoted 1 line
I firmly recommend option 1, especially at this point.
Jan, as you gave your Acked before, don't you mind to define
mfn_to_page()andpage_to_mfnas mentioned above ( Option 1 )?
No, I don't mind. And please feel free to keep my ack if no other significant changes are made.
Jan