Enable build of full Xen for RISC-V
v6 cover Enable build of full Xen for RISC-V
This patch series performs all of the additions necessary to drop the build overrides for RISCV and enable the full Xen build. Except in cases where compatibile implementations already exist (e.g. atomic.h and bitops.h), the newly added definitions are simple.
The patch series is based on the following patch series:
- [PATCH 0/7] xen/bitops: Reduce the mess, starting with ffs() [1]
- [PATCH] move __read_mostly to xen/cache.h [2]
- [XEN PATCH v2 1/3] xen: introduce STATIC_ASSERT_UNREACHABLE() [3]
Right now, the patch series doesn't have a direct dependency on [2] and it provides __read_mostly in the patch:
[PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the final version of the patch [2] is provided.
[1] https://lore.kernel.org/xen-devel/[email protected]/T/#t
[2] https://lore.kernel.org/xen-devel/[email protected]/
[3] https://lore.kernel.org/xen-devel/42fc6ae8d3eb802429d29c774502ff232340dc84.1706259490.git.federico.serafini@bugseng.com/
---
Changes in V6:
- Update the cover letter message: drop already merged dependecies and add
a new one.
- Patches were merged to staging:
- [PATCH v5 02/23] xen/riscv: use some asm-generic headers ( even v4 was merged to staging branch, I just wasn't apply changes on top of the latest staging branch )
- [PATCH v5 03/23] xen/riscv: introduce nospec.h
- [PATCH v5 10/23] xen/riscv: introduces acrquire, release and full barriers
- Introduce new patches:
- xen/riscv: introduce extenstion support check by compiler
- xen/bitops: put __ffs() and ffz() into linux compatible header
- xen/bitops: implement fls{l}() in common logic
- The following patches were dropped:
- drop some patches related to bitops operations as they were introduced in another
patch series [...]
- introduce new version for generic __ffs(), ffz() and fls{l}().
- Merge patch from patch series "[PATCH v9 0/7] Introduce generic headers" to this patch series as only one patch left in the generic headers patch series and it is more about RISC-V.
- Other changes are specific to specific patches. please look at specific patch.
---
Changes in V5:
- Update the cover letter as one of the dependencies were merged to staging.
- Was introduced asm-generic for atomic ops and separate patches for asm-generic bit ops
- Moved fence.h to separate patch to deal with some patches dependecies on fence.h
- Patches were dropped as they were merged to staging:
* [PATCH v4 03/30] xen: add support in public/hvm/save.h for PPC and RISC-V
* [PATCH v4 04/30] xen/riscv: introduce cpufeature.h
* [PATCH v4 05/30] xen/riscv: introduce guest_atomics.h
* [PATCH v4 06/30] xen: avoid generation of empty asm/iommu.h
* [PATCH v4 08/30] xen/riscv: introduce setup.h
* [PATCH v4 10/30] xen/riscv: introduce flushtlb.h
* [PATCH v4 11/30] xen/riscv: introduce smp.h
* [PATCH v4 15/30] xen/riscv: introduce irq.h
* [PATCH v4 16/30] xen/riscv: introduce p2m.h
* [PATCH v4 17/30] xen/riscv: introduce regs.h
* [PATCH v4 18/30] xen/riscv: introduce time.h
* [PATCH v4 19/30] xen/riscv: introduce event.h
* [PATCH v4 22/30] xen/riscv: define an address of frame table
- Other changes are specific to specific patches. please look at specific patch
---
Changes in V4:
- Update the cover letter message: new patch series dependencies.
- Some patches were merged to staging, so they were dropped in this patch series:
[PATCH v3 09/34] xen/riscv: introduce system.h
[PATCH v3 18/34] xen/riscv: introduce domain.h
[PATCH v3 19/34] xen/riscv: introduce guest_access.h
- Was sent out of this patch series:
[PATCH v3 16/34] xen/lib: introduce generic find next bit operations
- [PATCH v3 17/34] xen/riscv: add compilation of generic find-next-bit.c was droped as CONFIG_GENERIC_FIND_NEXT_BIT was dropped.
- All other changes are specific to a specific patch.
---
Changes in V3:
- Update the cover letter message
- The following patches were dropped as they were merged to staging:
[PATCH v2 03/39] xen/riscv:introduce asm/byteorder.h
[PATCH v2 04/39] xen/riscv: add public arch-riscv.h
[PATCH v2 05/39] xen/riscv: introduce spinlock.h
[PATCH v2 20/39] xen/riscv: define bug frame tables in xen.lds.S
[PATCH v2 34/39] xen: add RISCV support for pmu.h
[PATCH v2 35/39] xen: add necessary headers to common to build full Xen for RISC-V
- Instead of the following patches were introduced new:
[PATCH v2 10/39] xen/riscv: introduce asm/iommu.h
[PATCH v2 11/39] xen/riscv: introduce asm/nospec.h
- remove "asm/" for commit messages which start with "xen/riscv:"
- code style updates.
- add emulation of {cmp}xchg_* for 1 and 2 bytes types.
- code style fixes.
- add SPDX and footer for the newly added headers.
- introduce generic find-next-bit.c.
- some other mionor changes. ( details please find in a patch )
---
Changes in V2:
- Drop the following patches as they are the part of [2]:
[PATCH v1 06/57] xen/riscv: introduce paging.h
[PATCH v1 08/57] xen/riscv: introduce asm/device.h
[PATCH v1 10/57] xen/riscv: introduce asm/grant_table.h
[PATCH v1 12/57] xen/riscv: introduce asm/hypercall.h
[PATCH v1 13/57] xen/riscv: introduce asm/iocap.h
[PATCH v1 15/57] xen/riscv: introduce asm/mem_access.h
[PATCH v1 18/57] xen/riscv: introduce asm/random.h
[PATCH v1 21/57] xen/riscv: introduce asm/xenoprof.h
[PATCH v1 24/57] xen/riscv: introduce asm/percpu.h
[PATCH v1 29/57] xen/riscv: introduce asm/hardirq.h
[PATCH v1 33/57] xen/riscv: introduce asm/altp2m.h
[PATCH v1 38/57] xen/riscv: introduce asm/monitor.h
[PATCH v1 39/57] xen/riscv: introduce asm/numa.h
[PATCH v1 42/57] xen/riscv: introduce asm/softirq.h
- xen/lib.h in most of the cases were changed to xen/bug.h as
mostly functionilty of bug.h is used.
- align arch-riscv.h with Arm's version of it.
- change the Author of commit with introduction of asm/atomic.h.
- update some definition from spinlock.h.
- code style changes.
---
Oleksii Kurochko (20):
automation: introduce fixed randconfig for RISC-V xen/riscv: disable unnecessary configs
xen/riscv: introduce extenstion support check by compiler xen/asm-generic: introduce generic non-atomic test_*bit() xen/bitops: implement fls{l}() in common logic xen/bitops: put __ffs() and ffz() into linux compatible header xen/riscv: introduce bitops.h
xen/riscv: introduce cmpxchg.h
xen/riscv: introduce io.h
xen/riscv: introduce atomic.h
xen/riscv: introduce monitor.h
xen/riscv: add definition of __read_mostly xen/riscv: add required things to current.h xen/riscv: add minimal stuff to page.h to build full Xen xen/riscv: add minimal stuff to processor.h to build full Xen xen/riscv: add minimal stuff to mm.h to build full Xen xen/riscv: introduce vm_event_*() functions xen/riscv: add minimal amount of stubs to build full Xen xen/riscv: enable full Xen build
xen/README: add compiler and binutils versions for RISC-V64
README | 4 + automation/gitlab-ci/build.yaml | 38 +- docs/misc/riscv/booting.txt | 16 + xen/arch/arm/include/asm/arm32/bitops.h | 2 +- xen/arch/arm/include/asm/arm64/bitops.h | 27 +- xen/arch/arm/include/asm/bitops.h | 7 +- xen/arch/ppc/include/asm/bitops.h | 24 - xen/arch/riscv/Makefile | 18 +- xen/arch/riscv/arch.mk | 14 +- xen/arch/riscv/configs/tiny64_defconfig | 17 + xen/arch/riscv/early_printk.c | 168 ------- xen/arch/riscv/include/asm/atomic.h | 263 +++++++++++ xen/arch/riscv/include/asm/bitops.h | 144 ++++++ xen/arch/riscv/include/asm/cache.h | 2 + xen/arch/riscv/include/asm/cmpxchg.h | 209 +++++++++ xen/arch/riscv/include/asm/config.h | 2 + xen/arch/riscv/include/asm/current.h | 19 + xen/arch/riscv/include/asm/io.h | 167 +++++++ xen/arch/riscv/include/asm/mm.h | 240 ++++++++++ xen/arch/riscv/include/asm/monitor.h | 26 ++ xen/arch/riscv/include/asm/page.h | 19 + xen/arch/riscv/include/asm/processor.h | 20 + xen/arch/riscv/mm.c | 52 ++- xen/arch/riscv/setup.c | 10 +- xen/arch/riscv/stubs.c | 415 ++++++++++++++++++ xen/arch/riscv/traps.c | 25 ++ xen/arch/riscv/vm_event.c | 19 + xen/arch/x86/include/asm/bitops.h | 6 +- xen/common/bitops.c | 22 + xen/drivers/passthrough/arm/smmu-v3.c | 2 + xen/include/asm-generic/atomic-ops.h | 97 ++++ xen/include/asm-generic/bitops/bitops-bits.h | 21 + .../asm-generic/bitops/generic-non-atomic.h | 89 ++++ xen/include/asm-generic/bitops/test-bit.h | 18 + xen/include/xen/bitops.h | 24 + xen/include/xen/linux-compat.h | 3 + xen/lib/find-next-bit.c | 1 + 37 files changed, 2012 insertions(+), 238 deletions(-) create mode 100644 docs/misc/riscv/booting.txt create mode 100644 xen/arch/riscv/include/asm/atomic.h create mode 100644 xen/arch/riscv/include/asm/bitops.h create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h create mode 100644 xen/arch/riscv/include/asm/io.h create mode 100644 xen/arch/riscv/include/asm/monitor.h create mode 100644 xen/arch/riscv/stubs.c create mode 100644 xen/arch/riscv/vm_event.c create mode 100644 xen/include/asm-generic/atomic-ops.h create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h create mode 100644 xen/include/asm-generic/bitops/generic-non-atomic.h create mode 100644 xen/include/asm-generic/bitops/test-bit.h
Signature · 1 line
2.43.0
PATCH v6 1/20 automation: introduce fixed randconfig for RISC-V
This patch introduces the anchor riscv-fixed-randconfig, which includes all configurations that should be disabled for randconfig builds.
---
Changes in V6:
- new patch for this patch series
- Reviewed-by and Acked-by was added. ( in another patch series they were recieved. )
--- automation/gitlab-ci/build.yaml | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
automation/gitlab-ci/build.yaml +10 -4
@@ -512,6 +512,14 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools: CONFIG_BOOT_TIME_CPUPOOLS=y # RISC-V 64 cross-build +.riscv-fixed-randconfig: + variables: &riscv-fixed-randconfig + EXTRA_FIXED_RANDCONFIG: | + CONFIG_COVERAGE=n + CONFIG_EXPERT=y + CONFIG_GRANT_TABLE=n + CONFIG_MEM_ACCESS=n + archlinux-current-gcc-riscv64: extends: .gcc-riscv64-cross-build variables:
@@ -532,8 +540,7 @@ archlinux-current-gcc-riscv64-randconfig: CONTAINER: archlinux:current-riscv64 KBUILD_DEFCONFIG: tiny64_defconfig RANDCONFIG: y - EXTRA_FIXED_RANDCONFIG: - CONFIG_COVERAGE=n + <<: *riscv-fixed-randconfig archlinux-current-gcc-riscv64-debug-randconfig: extends: .gcc-riscv64-cross-build-debug
@@ -541,8 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig: CONTAINER: archlinux:current-riscv64 KBUILD_DEFCONFIG: tiny64_defconfig RANDCONFIG: y - EXTRA_FIXED_RANDCONFIG: - CONFIG_COVERAGE=n + <<: *riscv-fixed-randconfig # Power cross-build debian-bullseye-gcc-ppc64le:
Signature · 1 line
2.43.0
PATCH v6 2/20 xen/riscv: disable unnecessary configs
In reply to Oleksii KurochkoThis patch disables unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.
---
Changes in V6:
- Nothing changed. Only rebase.
---
Changes in V5:
- Rebase and drop duplicated configs in EXTRA_FIXED_RANDCONFIG list
- Update the commit message
---
Changes in V4:
- Nothing changed. Only rebase
---
Changes in V3:
- Remove EXTRA_FIXED_RANDCONFIG for non-randconfig jobs.
For non-randconfig jobs, it is sufficient to disable configs by using the defconfig.
- Remove double blank lines in build.yaml file before archlinux-current-gcc-riscv64-debug
---
Changes in V2:
- update the commit message.
- remove xen/arch/riscv/Kconfig changes.
--- automation/gitlab-ci/build.yaml | 24 ++++++++++++++++++++++++ xen/arch/riscv/configs/tiny64_defconfig | 17 +++++++++++++++++ 2 files changed, 41 insertions(+)
automation/gitlab-ci/build.yaml +24 -0
@@ -519,6 +519,30 @@ alpine-3.18-gcc-debug-arm64-boot-cpupools: CONFIG_EXPERT=y CONFIG_GRANT_TABLE=n CONFIG_MEM_ACCESS=n + CONFIG_SCHED_CREDIT=n + CONFIG_SCHED_CREDIT2=n + CONFIG_SCHED_RTDS=n + CONFIG_SCHED_NULL=n + CONFIG_SCHED_ARINC653=n + CONFIG_TRACEBUFFER=n + CONFIG_HYPFS=n + CONFIG_SPECULATIVE_HARDEN_ARRAY=n + CONFIG_ARGO=n + CONFIG_HYPFS_CONFIG=n + CONFIG_CORE_PARKING=n + CONFIG_DEBUG_TRACE=n + CONFIG_IOREQ_SERVER=n + CONFIG_CRASH_DEBUG=n + CONFIG_KEXEC=n + CONFIG_LIVEPATCH=n + CONFIG_NUMA=n + CONFIG_PERF_COUNTERS=n + CONFIG_HAS_PMAP=n + CONFIG_XENOPROF=n + CONFIG_COMPAT=n + CONFIG_UBSAN=n + CONFIG_NEEDS_LIBELF=n + CONFIG_XSM=n archlinux-current-gcc-riscv64: extends: .gcc-riscv64-cross-build
xen/arch/riscv/configs/tiny64_defconfig +17 -0
@@ -7,6 +7,23 @@ # CONFIG_GRANT_TABLE is not set # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set # CONFIG_MEM_ACCESS is not set +# CONFIG_ARGO is not set +# CONFIG_HYPFS_CONFIG is not set +# CONFIG_CORE_PARKING is not set +# CONFIG_DEBUG_TRACE is not set +# CONFIG_IOREQ_SERVER is not set +# CONFIG_CRASH_DEBUG is not setz +# CONFIG_KEXEC is not set +# CONFIG_LIVEPATCH is not set +# CONFIG_NUMA is not set +# CONFIG_PERF_COUNTERS is not set +# CONFIG_HAS_PMAP is not set +# CONFIG_TRACEBUFFER is not set +# CONFIG_XENOPROF is not set +# CONFIG_COMPAT is not set +# CONFIG_COVERAGE is not set +# CONFIG_UBSAN is not set +# CONFIG_NEEDS_LIBELF is not set CONFIG_RISCV_64=y CONFIG_DEBUG=y
Signature · 1 line
2.43.0
Re: [PATCH v6 02/20] xen/riscv: disable unnecessary configs
xen/arch/riscv/configs/tiny64_defconfig line 11
Quoted 8 lines · Oleksii Kurochko
xen/arch/riscv/configs/tiny64_defconfig +2
-0@@ -7,6 +7,23 @@ # CONFIG_GRANT_TABLE is not set # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set # CONFIG_MEM_ACCESS is not set +# CONFIG_ARGO is not set +# CONFIG_HYPFS_CONFIG is not set
What exactly is wrong with permitting this?
xen/arch/riscv/configs/tiny64_defconfig line 12
Quoted 1 line
xen/arch/riscv/configs/tiny64_defconfig +1
-0+# CONFIG_CORE_PARKING is not set
At the example of this: It cannot be enabled by a user (or randconfig), it needs to be selected by an arch. Which RISC-V doesn't. Hence why specify (and not really override) the value here? This may apply to others as well, at the very least ...
xen/arch/riscv/configs/tiny64_defconfig line 20
Quoted 8 lines
xen/arch/riscv/configs/tiny64_defconfig +8
-0+# CONFIG_DEBUG_TRACE is not set +# CONFIG_IOREQ_SERVER is not set +# CONFIG_CRASH_DEBUG is not setz +# CONFIG_KEXEC is not set +# CONFIG_LIVEPATCH is not set +# CONFIG_NUMA is not set +# CONFIG_PERF_COUNTERS is not set +# CONFIG_HAS_PMAP is not set
... anything CONFIG_HAS_*.
In summary - please limit overrides to what is minimally necessary.
Jan
Quoted 9 lines
xen/arch/riscv/configs/tiny64_defconfig +6
-0+# CONFIG_TRACEBUFFER is not set +# CONFIG_XENOPROF is not set +# CONFIG_COMPAT is not set +# CONFIG_COVERAGE is not set +# CONFIG_UBSAN is not set +# CONFIG_NEEDS_LIBELF is not set CONFIG_RISCV_64=y CONFIG_DEBUG=y
Re: [PATCH v6 02/20] xen/riscv: disable unnecessary configs
Quoted 11 lines · Jan Beulich
xen/arch/riscv/configs/tiny64_defconfig line 11
Quoted 8 lines · Oleksii Kurochko
xen/arch/riscv/configs/tiny64_defconfig +2
-0@@ -7,6 +7,23 @@ # CONFIG_GRANT_TABLE is not set # CONFIG_SPECULATIVE_HARDEN_ARRAY is not set # CONFIG_MEM_ACCESS is not set +# CONFIG_ARGO is not set +# CONFIG_HYPFS_CONFIG is not setWhat exactly is wrong with permitting this?
Nothing wrong, I just chose the wrong approach about which configs should be disabled.
Considering your comment below, I'll limit overrides to what is minimally necessary.
~ Oleksii
Quoted 34 lines
xen/arch/riscv/configs/tiny64_defconfig line 12
Quoted 2 lines
xen/arch/riscv/configs/tiny64_defconfig +1
-0+# CONFIG_CORE_PARKING is not setAt the example of this: It cannot be enabled by a user (or randconfig),
it needs to be selected by an arch. Which RISC-V doesn't. Hence why specify (and not really override) the value here? This may apply to others as well, at the very least ...xen/arch/riscv/configs/tiny64_defconfig line 20
Quoted 8 lines
xen/arch/riscv/configs/tiny64_defconfig +8
-0+# CONFIG_DEBUG_TRACE is not set +# CONFIG_IOREQ_SERVER is not set +# CONFIG_CRASH_DEBUG is not setz +# CONFIG_KEXEC is not set +# CONFIG_LIVEPATCH is not set +# CONFIG_NUMA is not set +# CONFIG_PERF_COUNTERS is not set +# CONFIG_HAS_PMAP is not set... anything CONFIG_HAS_*.
In summary - please limit overrides to what is minimally necessary.
Jan
Quoted 9 lines
xen/arch/riscv/configs/tiny64_defconfig +6
-0+# CONFIG_TRACEBUFFER is not set +# CONFIG_XENOPROF is not set +# CONFIG_COMPAT is not set +# CONFIG_COVERAGE is not set +# CONFIG_UBSAN is not set +# CONFIG_NEEDS_LIBELF is not set
CONFIG_RISCV_64=y
CONFIG_DEBUG=y
PATCH v6 3/20 xen/riscv: introduce extenstion support check by compiler
In reply to Oleksii KurochkoCurrently, RISC-V requires two extensions: _zbb and _zihintpause.
This patch introduces a compiler check to check if these extensions are supported.
Additionally, it introduces the riscv/booting.txt file, which contains information about the extensions that should be supported by the platform.
In the future, a feature will be introduced to check whether an extension is supported at runtime.
However, this feature requires functionality for parsing device tree source (DTS), which is not yet available.
---
Changes in V6:
- new patch for this patch series
--- docs/misc/riscv/booting.txt | 16 ++++++++++++++++ xen/arch/riscv/arch.mk | 10 ++++++++-- 2 files changed, 24 insertions(+), 2 deletions(-) create mode 100644 docs/misc/riscv/booting.txt
docs/misc/riscv/booting.txt new file +16 -0
@@ -0,0 +1,16 @@ +System requirements +=================== + +The following extensions are expected to be supported by a system on which +Xen is run: +- Zbb: + RISC-V doesn't have a CLZ instruction in the base ISA. + As a consequence, __builtin_ffs() emits a library call to ffs() on GCC, + or a de Bruijn sequence on Clang. + Zbb extension adds a CLZ instruction, after which __builtin_ffs() emits + a very simple sequence. + The similar issue occurs with other __builtin_<bitop>, so it is needed to + provide a generic version of bitops in RISC-V bitops.h +- Zihintpause: + On a system that doesn't have this extension, cpu_relax() should be + implemented properly.
xen/arch/riscv/arch.mk +8 -2
@@ -3,16 +3,22 @@ $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS)) -CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64 +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32 +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64 riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c +extensions := $(call as-insn,$(CC) $(riscv-abi-y) -march=$(riscv-march-y)_zbb,"",_zbb) \ + $(call as-insn,$(CC) $(riscv-abi-y) -march=$(riscv-march-y)_zihintpause,"pause",_zihintpause) + +extensions := $(subst $(space),,$(extensions)) + # Note that -mcmodel=medany is used so that Xen can be mapped # into the upper half _or_ the lower half of the address space. # -mcmodel=medlow would force Xen into the lower half. -CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany +CFLAGS += $(riscv-abi-y) -march=$(riscv-march-y)$(extensions) -mstrict-align -mcmodel=medany # TODO: Drop override when more of the build is working override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o
Signature · 1 line
2.43.0
Re: [PATCH v6 03/20] xen/riscv: introduce extenstion support check by compiler
Quoted 1 line · Oleksii Kurochko
Currently, RISC-V requires two extensions:
_zbband_zihintpause.
Do we really require Zbb already?
Quoted 9 lines
This patch introduces a compiler check to check if these extensions are supported.
Additionally, it introduces theriscv/booting.txtfile, which contains information about the extensions that should be supported by the platform.In the future, a feature will be introduced to check whether an extension is supported at runtime.
However, this feature requires functionality for parsing device tree source (DTS), which is not yet available.
Can't you query the CPU for its features?
Quoted 5 lines
xen/arch/riscv/arch.mk +0
-0@@ -3,16 +3,22 @@ $(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))
xen/arch/riscv/arch.mk line 13
Quoted 10 lines
xen/arch/riscv/arch.mk +4
-1-CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64 +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32 +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64 riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c +extensions := $(call as-insn,$(CC) $(riscv-abi-y) -march=$(riscv-march-y)_zbb,"",_zbb) \ + $(call as-insn,$(CC) $(riscv-abi-y) -march=$(riscv-march-y)_zihintpause,"pause",_zihintpause)
Imo you want another helper macro here, where all one needs to pass in is the extension name (i.e. zbb and zihintpause as per above). That'll also help with line length, I hope.
Jan
Re: [PATCH v6 03/20] xen/riscv: introduce extenstion support check by compiler
Quoted 4 lines · Jan Beulich
Quoted 1 line · Oleksii Kurochko
Currently, RISC-V requires two extensions:
_zbband_zihintpause.Do we really require Zbb already?
After an introduction of Andrew C. patches [1] it is requited for __builtin_ffs{l}
[1]
https://lore.kernel.org/xen-devel/[email protected]/T/#t
Quoted 8 lines
Quoted 8 lines
This patch introduces a compiler check to check if these extensions are supported.
Additionally, it introduces theriscv/booting.txtfile, which contains
information about the extensions that should be supported by the platform.
Quoted 8 lines
Quoted 6 lines
In the future, a feature will be introduced to check whether an extension
is supported at runtime.
However, this feature requires functionality for parsing device tree
source (DTS), which is not yet available.Can't you query the CPU for its features?
I couldn't find such reg ( or SBI call ) in the spec.
SBI call sbi_probe_extension() exists, but it doesn't check for every possible extension. As far as I understand it checks only for that one which are present in SBI spec.
The most closest thing I see how to check that without dts is how it is done in OpenSBI:
#define csr_read_allowed(csr_num, trap) \
({ \
register ulong tinfo asm("a3") = (ulong)trap; \
register ulong ttmp asm("a4"); \
register ulong mtvec = sbi_hart_expected_trap_addr(); \
register ulong ret =
0; \
((struct sbi_trap_info *)(trap))->cause = 0; \
asm volatile( \
"add %[ttmp], %[tinfo],
zero\n" \
"csrrw %[mtvec], " STR(CSR_MTVEC) ", %[mtvec]\n"\
"csrr %[ret],
%[csr]\n" \
"csrw " STR(CSR_MTVEC) ", %[mtvec]" \
: [mtvec] "+&r"(mtvec), [tinfo] "+&r"(tinfo), \
[ttmp] "+&r"(ttmp), [ret] "=&r" (ret) \
: [csr] "i" (csr_num) \
: "memory"); \
ret; \
}) \
...
/* Detect if hart supports stimecmp CSR(Sstc extension) */
if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
csr_read_allowed(CSR_STIMECMP, (unsigned long)&trap);
if (!trap.cause)
__sbi_hart_update_extension(hfeatures,
SBI_HART_EXT_SSTC, true);
}~ Oleksii
Quoted 26 lines
Quoted 10 lines
xen/arch/riscv/arch.mk +0
-0@@ -3,16 +3,22 @@
$(call cc-options-add,CFLAGS,CC,$(EMBEDDED_EXTRA_CFLAGS))xen/arch/riscv/arch.mk +2
-1-CFLAGS-$(CONFIG_RISCV_64) += -mabi=lp64 +riscv-abi-$(CONFIG_RISCV_32) := -mabi=ilp32 +riscv-abi-$(CONFIG_RISCV_64) := -mabi=lp64Quoted 8 lines
riscv-march-$(CONFIG_RISCV_ISA_RV64G) := rv64g riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c
+extensions := $(call as-insn,$(CC) $(riscv-abi-y) -march=$(riscv- march-y)_zbb,"",_zbb) \
+ $(call as-insn,$(CC) $(riscv-abi-y) -march=$(riscv- march-y)_zihintpause,"pause",_zihintpause)Imo you want another helper macro here, where all one needs to pass in is
the extension name (i.e. zbb and zihintpause as per above). That'll also
help with line length, I hope.Jan
Re: [PATCH v6 03/20] xen/riscv: introduce extenstion support check by compiler
Quoted 25 lines · Oleksii
Quoted 4 lines · Jan Beulich
Quoted 1 line · Oleksii Kurochko
Currently, RISC-V requires two extensions:
_zbband_zihintpause.Do we really require Zbb already?
After an introduction of Andrew C. patches [1] it is requited for __builtin_ffs{l}
[1]
https://lore.kernel.org/xen-devel/[email protected]/T/#tQuoted 15 lines
Quoted 14 lines
This patch introduces a compiler check to check if these extensions are supported.
Additionally, it introduces theriscv/booting.txtfile, which contains
information about the extensions that should be supported by the platform.In the future, a feature will be introduced to check whether an extension
is supported at runtime.
However, this feature requires functionality for parsing device tree
source (DTS), which is not yet available.
Quoted 2 lines
Quoted 1 line
Can't you query the CPU for its features?
I couldn't find such reg ( or SBI call ) in the spec.
There isn't.
Quoted 3 lines
SBI call
sbi_probe_extension()exists, but it doesn't check for every possible extension. As far as I understand it checks only for that one which are present in SBI spec.
Yeah, it only checks for SBI extensions, not ISA extensions.
Quoted 2 lines
The most closest thing I see how to check that without dts is how it is done in OpenSBI:
IIRC this only "works" because the OpenSBI devs assume that there are no non-normative behaviours and all CSRs have their ~God~ RVI defined meanings. Reading a CSR to see if it traps is not behaviour you can really rely on unless the platform claims to support Sstrict - but Sstrict you'd have to detect from the DT so chicken and egg for you! It's one of these new "extensions" from the profiles spec, so it doesn't even have support in Linux's dt-bindings yet. Up to Xen devs if you guys want to make the same assumptions as OpenSBI. Linux doesn't and when we discussed this not too long ago on the U-Boot ML in the context of the rng CSR it was also decided not to do make the assumption there either.
Personally I wonder if you can just apply the same policy here as you did with Zbb in the other thread and assume that something with H will have Zihintpause and leave implementing a "fake" pause as an exercise for someone that introduces such a system?
If Jess is correct, and I do remember testing this, it's actually "always" safe to call the pause instruction on CPUs where the extension is not supported as it uses an encoding of fence that effectively makes it a into a nop:
https://lore.kernel.org/linux-riscv/[email protected]/
At worst, that'd make cpu_relax() a nop if someone didn't meet that requirement.
FWIW, Linux's cpu_relax() on RISC-V looks like:
static inline void cpu_relax(void)
{
#ifdef __riscv_muldiv
int dummy;
/* In lieu of a halt instruction, induce a long-latency stall. */
__asm__ __volatile__ ("div %0, %0, zero" : "=r" (dummy));
#endif
#ifdef CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE
/*
* Reduce instruction retirement.
* This assumes the PC changes.
*/
__asm__ __volatile__ ("pause");
#else
/* Encoding of the pause instruction */
__asm__ __volatile__ (".4byte 0x100000F");
#endif
barrier();
}I figure having div is part of your base requirements, so maybe you can just do something similar in the !zihintpause case if making that extension a requirement is problematic?
Doing that invalid div used to be conditional, but cpu_relax() is in the vdso so the static branch it used to be gated with got removed and its now unconditional. Probably that's not a constraint on Xen's cpu_relax()?
Oh ye, and we do the .4byte crap so that toolchain support wasn't needed for Zihintpause given we are using it in exactly one place.
Cheers,
Conor.
Quoted 29 lines
#define csr_read_allowed(csr_num, trap) \
({ \
register ulong tinfo asm("a3") = (ulong)trap; \
register ulong ttmp asm("a4"); \
register ulong mtvec = sbi_hart_expected_trap_addr(); \
register ulong ret =
0; \
((struct sbi_trap_info *)(trap))->cause = 0; \
asm volatile( \
"add %[ttmp], %[tinfo],
zero\n" \
"csrrw %[mtvec], " STR(CSR_MTVEC) ", %[mtvec]\n"\
"csrr %[ret],
%[csr]\n" \
"csrw " STR(CSR_MTVEC) ", %[mtvec]" \
: [mtvec] "+&r"(mtvec), [tinfo] "+&r"(tinfo), \
[ttmp] "+&r"(ttmp), [ret] "=&r" (ret) \
: [csr] "i" (csr_num) \
: "memory"); \
ret; \
}) \
...
/* Detect if hart supports stimecmp CSR(Sstc extension) */
if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
csr_read_allowed(CSR_STIMECMP, (unsigned long)&trap);
if (!trap.cause)
__sbi_hart_update_extension(hfeatures,
SBI_HART_EXT_SSTC, true);
}Re: [PATCH v6 03/20] xen/riscv: introduce extenstion support check by compiler
Quoted 29 lines · Conor Dooley
Quoted 25 lines · Oleksii
Quoted 4 lines · Jan Beulich
Quoted 1 line · Oleksii Kurochko
Currently, RISC-V requires two extensions:
_zbband_zihintpause.Do we really require Zbb already?
After an introduction of Andrew C. patches [1] it is requited for __builtin_ffs{l}
[1]
https://lore.kernel.org/xen-devel/[email protected]/T/#tQuoted 15 lines
Quoted 14 lines
This patch introduces a compiler check to check if these extensions are supported.
Additionally, it introduces theriscv/booting.txtfile, which contains
information about the extensions that should be supported by the platform.In the future, a feature will be introduced to check whether an extension
is supported at runtime.
However, this feature requires functionality for parsing device tree
source (DTS), which is not yet available.Quoted 2 lines
Quoted 1 line
Can't you query the CPU for its features?
I couldn't find such reg ( or SBI call ) in the spec.
Quoted 7 lines
There isn't.
Quoted 3 lines
SBI call
sbi_probe_extension()exists, but it doesn't check for every possible extension. As far as I understand it checks only for that one which are present in SBI spec.Yeah, it only checks for SBI extensions, not ISA extensions.
And there was never a consideration to add, at the architecture level, some straightforward way for all layers of software to be able to easily detect availability of extensions? I find the lack thereof pretty surprising ...
Jan
Re: [PATCH v6 03/20] xen/riscv: introduce extenstion support check by compiler
Quoted 36 lines · Jan Beulich
Quoted 35 lines · Conor Dooley
Quoted 25 lines · Oleksii
Quoted 4 lines · Jan Beulich
Quoted 1 line · Oleksii Kurochko
Currently, RISC-V requires two extensions:
_zbband_zihintpause.Do we really require Zbb already?
After an introduction of Andrew C. patches [1] it is requited for __builtin_ffs{l}
[1]
https://lore.kernel.org/xen-devel/[email protected]/T/#tQuoted 15 lines
Quoted 14 lines
This patch introduces a compiler check to check if these extensions are supported.
Additionally, it introduces theriscv/booting.txtfile, which contains
information about the extensions that should be supported by the platform.In the future, a feature will be introduced to check whether an extension
is supported at runtime.
However, this feature requires functionality for parsing device tree
source (DTS), which is not yet available.Quoted 2 lines
Quoted 1 line
Can't you query the CPU for its features?
I couldn't find such reg ( or SBI call ) in the spec.
There isn't.
Quoted 3 lines
SBI call
sbi_probe_extension()exists, but it doesn't check for every possible extension. As far as I understand it checks only for that one which are present in SBI spec.
Quoted 6 lines
Quoted 1 line
Yeah, it only checks for SBI extensions, not ISA extensions.
And there was never a consideration to add, at the architecture level, some straightforward way for all layers of software to be able to easily detect availability of extensions? I find the lack thereof pretty surprising ...
No, there sorta is. There's a "configuration structure" spec in the works, but the (public?) documentation for it is very nascent:
https://github.com/riscv/configuration-structure/
It uses (what I seem to recall is an m-mode) CSR to give the address of an ASN.1 structure in memory which is intended to encode a whole raft of information. The CSR itself might be M-Mode, but that doesn't prevent accessing the data itself at any layer in the stack. The last time I read it I got the impression it was supposed to be usable for describing an entire system (including things like i2c or spi controllers). I don't think it ever explicitly says that in the spec, but there's a note (IIRC) that mentions being unsuitable for describing devices on dynamic buses like PCI, so that kinda implies anything else is fair game.
Hope that helps?
Re: [PATCH v6 03/20] xen/riscv: introduce extenstion support check by compiler
In reply to Conor DooleyQuoted 18 lines · Conor Dooley
IIRC this only "works" because the OpenSBI devs assume that there are no
non-normative behaviours and all CSRs have their ~God~ RVI defined meanings. Reading a CSR to see if it traps is not behaviour you can really
rely on unless the platform claims to support Sstrict - but Sstrict you'd
have to detect from the DT so chicken and egg for you! It's one of these
new "extensions" from the profiles spec, so it doesn't even have support
in Linux's dt-bindings yet. Up to Xen devs if you guys want to make the
same assumptions as OpenSBI. Linux doesn't and when we discussed this not too long ago on the U-Boot ML in the context of the rng CSR it was
also decided not to do make the assumption there either.
Quoted 5 lines
Personally I wonder if you can just apply the same policy here as you did with Zbb in the other thread and assume that something with H will
have Zihintpause and leave implementing a "fake" pause as an exercise for someone that introduces such a system?
If i understand you correctly, then it is done in this way now. Only build time check is done, but it would be nice also have some runtime check, and for now, for runtime the "check" is only exists in booting.txt where it mentioned that Xen expects from CPU to support some extenstions; otherwise "please implement the following functions".
Anyway, at the moment, the best one runtime check which we can provide is detect availability of extensions from DT and what I mentioned in commit message.
~ Oleksii
PATCH v6 4/20 xen/asm-generic: introduce generic non-atomic test_*bit()
In reply to Oleksii KurochkoThe patch introduces the following generic functions:
* test_bit
* generic___test_and_set_bit
* generic___test_and_clear_bit
* generic___test_and_change_bit
Also, the patch introduces the following generics which are used by the functions mentioned above:
* BITOP_BITS_PER_WORD
* BITOP_MASK
* BITOP_WORD
* BITOP_TYPE
These functions and macros can be useful for architectures that don't have corresponding arch-specific instructions.
---
Changes in V6:
- Nothing changed ( only rebase )
---
Changes in V5:
- new patch
--- xen/include/asm-generic/bitops/bitops-bits.h | 21 +++++ .../asm-generic/bitops/generic-non-atomic.h | 89 +++++++++++++++++++ xen/include/asm-generic/bitops/test-bit.h | 18 ++++ 3 files changed, 128 insertions(+) create mode 100644 xen/include/asm-generic/bitops/bitops-bits.h create mode 100644 xen/include/asm-generic/bitops/generic-non-atomic.h create mode 100644 xen/include/asm-generic/bitops/test-bit.h
xen/include/asm-generic/bitops/bitops-bits.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_BITOPS_BITS_H_ +#define _ASM_GENERIC_BITOPS_BITS_H_ + +#ifndef BITOP_BITS_PER_WORD +#define BITOP_BITS_PER_WORD 32 +#endif + +#ifndef BITOP_MASK +#define BITOP_MASK (nr) (1U << ((nr) % BITOP_BITS_PER_WORD)) +#endif + +#ifndef BITOP_WORD +#define BITOP_WORD (nr) ((nr) / BITOP_BITS_PER_WORD) +#endif + +#ifndef BITOP_TYPE +typedef uint32_t bitops_uint_t ; +#endif + +#endif /* _ASM_GENERIC_BITOPS_BITS_H_ */
xen/include/asm-generic/bitops/generic-non-atomic.h new file +89 -0
89 lines not on this page · raw message
xen/include/asm-generic/bitops/test-bit.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_BITOPS_TESTBIT_H_ +#define _ASM_GENERIC_BITOPS_TESTBIT_H_ + +#include <asm-generic/bitops/bitops-bits.h> + +/** +* test_bit - Determine whether a bit is set +* @nr: bit number to test +* @addr: Address to start counting from +*/ +static inline int test_bit (int nr,const volatile void *addr) +{ +const volatile bitops_uint_t *p = addr; +return 1 & (p[BITOP_WORD (nr)] >> (nr & (BITOP_BITS_PER_WORD -1 ))); +} + +#endif /* _ASM_GENERIC_BITOPS_TESTBIT_H_ */
Signature · 1 line
2.43.0
Re: [PATCH v6 04/20] xen/asm-generic: introduce generic non-atomic test_*bit()
Quoted 13 lines · Oleksii Kurochko
The patch introduces the following generic functions:
*test_bit
*generic___test_and_set_bit
*generic___test_and_clear_bit
*generic___test_and_change_bitAlso, the patch introduces the following generics which are used by the functions mentioned above:
*BITOP_BITS_PER_WORD
*BITOP_MASK
*BITOP_WORD
*BITOP_TYPE
Quoted 4 lines
These functions and macros can be useful for architectures that don't have corresponding arch-specific instructions.
Signed-off-by Oleksii Kurochko <[email protected]>
First of all: How much syncing has there been versus Andrew's plans? I don't want to end up in the middle of two entirely different approaches to the same generalization goal.
Quoted 12 lines
xen/include/asm-generic/bitops/bitops-bits.h new file +9
-0@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_BITOPS_BITS_H_ +#define _ASM_GENERIC_BITOPS_BITS_H_ + +#ifndef BITOP_BITS_PER_WORD +#define BITOP_BITS_PER_WORD 32 +#endif + +#ifndef BITOP_MASK
xen/include/asm-generic/bitops/bitops-bits.h line 19
Quoted 10 lines
xen/include/asm-generic/bitops/bitops-bits.h new file +10
-0+#define BITOP_MASK (nr) (1U << ((nr) % BITOP_BITS_PER_WORD)) +#endif + +#ifndef BITOP_WORD +#define BITOP_WORD (nr) ((nr) / BITOP_BITS_PER_WORD) +#endif + +#ifndef BITOP_TYPE +typedef uint32_t bitops_uint_t ; +#endif
Is it of practical use to permit overriding of one of BITOP_TYPE and BITOP_BITS_PER_WORD? I think both want tying together.
Is it further of any use to allow overriding of BITOP_{MASK,WORD}? There's minimal generalization needed to allow an override of BITOP_TYPE, though:
#define BITOP_MASK(nr) ((bitop_uint_t)1 << ((nr) % BITOP_BITS_PER_WORD))
Note that I've omitted the 's' from the typedef name - the macros all having no 'S', imo the type also shouldn't (or the other way around).
Quoted 7 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +4
-0@@ -0,0 +1,89 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The file is based on Linux ( 6.4.0 ) header: +* include/asm-generic/bitops/generic-non-atomic.h
xen/include/asm-generic/bitops/generic-non-atomic.h line 14
Quoted 10 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +10
-0+* +* Only functions that can be reused in Xen were left; others were removed. +* +* Also, the following changes were done: +* - it was updated the message inside #ifndef ... #endif. +* - __always_inline -> always_inline to be align with definition in +* xen/compiler.h. +* - update function prototypes from +* generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr) to +* generic___test_and_*(unsigned long nr, volatile void *addr) to be
What's the point of having a whopping 3 successive inner underscores? Which btw ...
xen/include/asm-generic/bitops/generic-non-atomic.h line 18
Quoted 4 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +4
-0+* consistent with other related macros/defines. +* - convert identations from tabs to spaces. +* - inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long' +* to be generic.
... shrink to just 2 here.
Oh, and there's no generic__test_bit(), but just test_bit() in a separate header.
Quoted 22 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +22
-0+*/ + +#ifndef __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H +#define __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H + +#include <xen/compiler.h> + +#include <asm-generic/bitops/bitops-bits.h> + +#ifndef XEN_BITOPS_H +#error only <xen/bitops.h> can be included directly +#endif + +/* +* Generic definitions for bit operations, should not be used in regular code +* directly. +*/ + +/** +* generic___test_and_set_bit - Set a bit and return its old value +* @nr: Bit to set +* @addr: Address to count from
xen/include/asm-generic/bitops/generic-non-atomic.h line 50
Quoted 10 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +10
-0+* +* This operation is non-atomic and can be reordered. +* If two examples of this operation race, one can appear to succeed +* but actually fail. You must protect multiple accesses with a lock. +*/ +static always_inline bool +generic___test_and_set_bit (unsigned long nr,volatile void *addr) +{ +bitops_uint_t mask =BITOP_MASK (nr); +bitops_uint_t *p = ((bitops_uint_t *)addr) +BITOP_WORD (nr);
If you cast away the volatile here, what's the point of having it in the parameter declaration? Also such a cast doesn't need an outer pair of parentheses.
xen/include/asm-generic/bitops/generic-non-atomic.h line 54
Quoted 4 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +4
-0+bitops_uint_t old = *p; + + *p = old | mask; +return (old & mask) !=0 ;
Minor: The function returning bool, the "!= 0" (and then also the parentheses) could be omitted.
Quoted 9 lines
xen/include/asm-generic/bitops/test-bit.h new file +6
-0@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_BITOPS_TESTBIT_H_ +#define _ASM_GENERIC_BITOPS_TESTBIT_H_ + +#include <asm-generic/bitops/bitops-bits.h> +
xen/include/asm-generic/bitops/test-bit.h line 16
Quoted 10 lines
xen/include/asm-generic/bitops/test-bit.h new file +10
-0+/** +* test_bit - Determine whether a bit is set +* @nr: bit number to test +* @addr: Address to start counting from +*/ +static inline int test_bit (int nr,const volatile void *addr) +{ +const volatile bitops_uint_t *p = addr; +return 1 & (p[BITOP_WORD (nr)] >> (nr & (BITOP_BITS_PER_WORD -1 ))); +}
Interestingly you don't lose the volatile here. Overall the way this is written would likely benefit the other functions too. There's no cast needed here and thus also not there.
Jan
Re: [PATCH v6 04/20] xen/asm-generic: introduce generic non-atomic test_*bit()
Quoted 14 lines · Jan Beulich
Quoted 13 lines · Oleksii Kurochko
The patch introduces the following generic functions:
*test_bit
*generic___test_and_set_bit
*generic___test_and_clear_bit
*generic___test_and_change_bitAlso, the patch introduces the following generics which are used by the functions mentioned above:
*BITOP_BITS_PER_WORD
*BITOP_MASK
*BITOP_WORD
*BITOP_TYPE
Quoted 8 lines
Quoted 4 lines
These functions and macros can be useful for architectures that don't have corresponding arch-specific instructions.
Signed-off-by Oleksii Kurochko <[email protected]>First of all: How much syncing has there been versus Andrew's plans? I don't want to end up in the middle of two entirely different approaches to the same generalization goal.
We don't have synced about bit operations mentioned in this patch, so I decided to follow an approach I have for them. But if Andrew's approach is better, or it would be better to follow one approach instead of introducing two approaches, I will be happy to rework this patch.
Andrew, what do you think about this bit ops?
Quoted 32 lines
Quoted 15 lines
xen/include/asm-generic/bitops/bitops-bits.h new file +10
-0@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_BITOPS_BITS_H_ +#define _ASM_GENERIC_BITOPS_BITS_H_ + +#ifndef BITOP_BITS_PER_WORD +#define BITOP_BITS_PER_WORD 32 +#endif + +#ifndef BITOP_MASK +#define BITOP_MASK (nr) (1U << ((nr) %
BITOP_BITS_PER_WORD))xen/include/asm-generic/bitops/bitops-bits.h line 19
Quoted 9 lines
xen/include/asm-generic/bitops/bitops-bits.h new file +9
-0+#endif + +#ifndef BITOP_WORD +#define BITOP_WORD (nr) ((nr) / BITOP_BITS_PER_WORD) +#endif + +#ifndef BITOP_TYPE +typedef uint32_t bitops_uint_t ; +#endif Is it of practical use to permit overriding of one of
BITOP_TYPEandBITOP_BITS_PER_WORD? I think both want tying together.Is it further of any use to allow overriding of BITOP_{MASK,WORD}? There's minimal generalization needed to allow an override of
BITOP_TYPE, though:
Quoted 6 lines
#define BITOP_MASK(nr) ((bitop_uint_t)1 << ((nr) %
BITOP_BITS_PER_WORD))Note that I've omitted the 's' from the typedef name - the macros all having no 'S', imo the type also shouldn't (or the other way around).
If to generalize BITOP_MASK in way you suggest, then there is no need to allow overriding of BITOP_{MASK,WORD}. I'll update this part.
Quoted 32 lines
Quoted 14 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +6
-0@@ -0,0 +1,89 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The file is based on Linux ( 6.4.0 ) header: +* include/asm-generic/bitops/generic-non-atomic.h +* +* Only functions that can be reused in Xen were left; others were removed.
+ *
+ * Also, the following changes were done:
+ * - it was updated the message inside #ifndef ... #endif.Quoted 8 lines
+ * -
__always_inline->always_inlineto be align with definition in
+ *xen/compiler.h.
+ * - update function prototypes from
+ * generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr) to
+ * generic___test_and_*(unsigned long nr, volatile void *addr) to beWhat's the point of having a whopping 3 successive inner underscores? Which btw ...
Quoted 5 lines
+ * consistent with other related macros/defines.
+ * - convert identations from tabs to spaces.
+ * - inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
+ * to be generic.
Quoted 5 lines
... shrink to just 2 here.
Oh, and there's no
generic__test_bit(), but justtest_bit()in a separate
header.
I just wanted to show that it is a generic version of __test_and_* operations. If there is no sense in that, I am okay to update function names.
~ Oleksii
Quoted 75 lines
Quoted 31 lines
+ */
xen/include/asm-generic/bitops/generic-non-atomic.h new file +14
-0+ +#ifndef __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H +#define __ASM_GENERIC_BITOPS_GENERIC_NON_ATOMIC_H + +#include <xen/compiler.h> + +#include <asm-generic/bitops/bitops-bits.h> + +#ifndef XEN_BITOPS_H +#error only <xen/bitops.h> can be included directly +#endif + +/* +* Generic definitions for bit operations, should not be used in regular code
+ * directly.
+ */xen/include/asm-generic/bitops/generic-non-atomic.h new file +8
-0+ +/** +* generic___test_and_set_bit - Set a bit and return its old value +* @nr: Bit to set +* @addr: Address to count from +* +* This operation is non-atomic and can be reordered. +* If two examples of this operation race, one can appear to succeed
+ * but actually fail. You must protect multiple accesses with a lock.
+ */xen/include/asm-generic/bitops/generic-non-atomic.h line 50
Quoted 5 lines
xen/include/asm-generic/bitops/generic-non-atomic.h new file +5
-0+static always_inline bool +generic___test_and_set_bit (unsigned long nr,volatile void *addr) +{ + bitops_uint_t mask = BITOP_MASK(nr); + bitops_uint_t *p = ((bitops_uint_t *)addr) + BITOP_WORD(nr);If you cast away the volatile here, what's the point of having it in the parameter declaration? Also such a cast doesn't need an outer pair of parentheses.
Quoted 4 lines
+
bitops_uint_told = *p;
+
+ *p = old | mask;
+ return (old & mask) != 0;Minor: The function returning bool, the "!= 0" (and then also the parentheses) could be omitted.
Quoted 18 lines
xen/include/asm-generic/bitops/test-bit.h new file +15
-0@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_GENERIC_BITOPS_TESTBIT_H_ +#define _ASM_GENERIC_BITOPS_TESTBIT_H_ + +#include <asm-generic/bitops/bitops-bits.h> + +/** +* test_bit - Determine whether a bit is set +* @nr: bit number to test +* @addr: Address to start counting from +*/ +static inline int test_bit (int nr,const volatile void *addr) +{ + const volatile bitops_uint_t *p = addr; + return 1 & (p[BITOP_WORD(nr)] >> (nr & (BITOP_BITS_PER_WORD -Quoted 2 lines
1)));
+}Interestingly you don't lose the volatile here. Overall the way this is written would likely benefit the other functions too. There's no cast needed here and thus also not there.
Jan
PATCH v6 5/20 xen/bitops: implement fls{l}() in common logic
In reply to Oleksii KurochkoReturn type was left 'int' because of the following compilation error:
./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]
18 | (void) (&_x == &_y); \
| ^~
common/page_alloc.c:1843:34: note: in expansion of macro 'min'
1843 | unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0, the result in undefined.
---
Changes in V6:
- new patch for the patch series.
--- xen/arch/arm/include/asm/arm32/bitops.h | 2 +- xen/arch/arm/include/asm/arm64/bitops.h | 6 ++---- xen/arch/arm/include/asm/bitops.h | 7 ++----- xen/arch/ppc/include/asm/bitops.h | 3 --- xen/arch/x86/include/asm/bitops.h | 6 ++++-- xen/common/bitops.c | 22 ++++++++++++++++++++++ xen/include/xen/bitops.h | 24 ++++++++++++++++++++++++ 7 files changed, 55 insertions(+), 15 deletions(-)
xen/arch/arm/include/asm/arm32/bitops.h +1 -1
@@ -1,7 +1,7 @@#ifndef _ARM_ARM32_BITOPS_H#define _ARM_ARM32_BITOPS_H -#define flsl fls +#define arch_flsl fls/* * Little endian assembly bitops. nr = 0 -> byte 0 bit 0.
xen/arch/arm/include/asm/arm64/bitops.h +2 -4
@@ -22,17 +22,15 @@ static /*__*/always_inline unsigned long __ffs(unsigned long word)*/ #define ffz (x) __ffs(~(x)) -static inline int flsl (unsigned long x) +static inline int arch_flsl (unsigned long x) {uint64_t ret; -if (__builtin_constant_p (x)) -return generic_flsl (x); - asm("clz\t%0, %1" :"=r" (ret) :"r" (x));return BITS_PER_LONG - ret; } +#define arch_flsl arch_flsl/* Based on linux/include/asm-generic/bitops/find.h */
xen/arch/arm/include/asm/bitops.h +2 -5
@@ -145,17 +145,14 @@ static inline int test_bit(int nr, const volatile void *addr)* the clz instruction for much better code efficiency. */ -static inline int fls (unsigned int x) +static inline int arch_fls (unsigned int x) {int ret; -if (__builtin_constant_p (x)) -return generic_fls (x); - asm("clz\t%" __OP32"0, %" __OP32"1" :"=r" (ret) :"r" (x));return 32 - ret; } - +#define arch_fls arch_fls#define arch_ffs (x) ({ unsigned int __t = (x); fls(ISOLATE_LSB(__t)); })#define arch_ffsl (x) ({ unsigned long __t = (x); flsl(ISOLATE_LSB(__t)); })
xen/arch/ppc/include/asm/bitops.h +0 -3
@@ -171,9 +171,6 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)return (old & mask) !=0 ; } -#define flsl (x) generic_flsl(x) -#define fls (x) generic_fls(x) -/* Based on linux/include/asm-generic/bitops/ffz.h */ /* * ffz - find first zero in word.
xen/arch/x86/include/asm/bitops.h +4 -2
@@ -438,7 +438,7 @@ static always_inline unsigned int arch_ffsl(unsigned long x)* * This is defined the same way as ffs. */ -static inline int flsl (unsigned long x) +static always_inline int arch_flsl(unsigned long x) {long r;
@@ -448,8 +448,9 @@ static inline int flsl(unsigned long x) "1:" : "=r" (r) : "rm" (x)); return (int)r+1; } +#define arch_flsl arch_flsl -static inline int fls (unsigned int x) +static always_inline int arch_fls(unsigned int x) {int r;
@@ -459,6 +460,7 @@ static inline int fls(unsigned int x) "1:" : "=r" (r) : "rm" (x)); return r +1 ; } +#define arch_fls arch_fls/** * hweightN - returns the hamming weight of a N-bit word
xen/common/bitops.c +22 -0
@@ -62,9 +62,31 @@ static void test_ffs(void)CHECK (ffs64, (uint64_t )0x8000000000000000 ,64 ); } +static void test_fls (void ) +{ +/* unsigned int ffs(unsigned int) */ +CHECK (fls,1 ,1 ); +CHECK (fls,3 ,2 ); +CHECK (fls,3U <<30 ,32 ); + +/* unsigned int flsl(unsigned long) */ +CHECK (flsl,1 ,1 ); +CHECK (flsl,1UL << (BITS_PER_LONG -1 ),BITS_PER_LONG ); +#if BITS_PER_LONG >32 +CHECK (flsl,3UL <<32 ,34 ); +#endif + +/* unsigned int fls64(uint64_t) */ +CHECK (fls64,1 ,1 ); +CHECK (fls64,0x00000000C0000000ULL ,32 ); +CHECK (fls64,0x0000000180000000ULL ,33 ); +CHECK (fls64,0xC000000000000000ULL ,64 ); +} + static int __init cf_check test_bitops(void) {test_ffs (); +test_fls ();return 0 ; }
xen/include/xen/bitops.h +24 -0
@@ -69,6 +69,30 @@ static inline int generic_flsl(unsigned long x)#include <asm/bitops.h> +static always_inline __pure int fls(unsigned int x) +{ +if (__builtin_constant_p (x)) +return generic_fls (x); + +#ifndef arch_fls +#define arch_fls generic_fls +#endif + +return arch_fls (x); +} + +static always_inline __pure int flsl(unsigned long x) +{ +if (__builtin_constant_p (x)) +return generic_flsl (x); + +#ifndef arch_flsl +#define arch_flsl generic_flsl +#endif + +return arch_flsl (x); +} +/* * Find First Set bit. Bits are labelled from 1. */
Signature · 1 line
2.43.0
Re: [PATCH v6 05/20] xen/bitops: implement fls{l}() in common logic
Quoted 8 lines · Oleksii Kurochko
Return type was left 'int' because of the following compilation error:
./include/xen/kernel.h:18:21: error: comparison of distinct pointer types lacks a cast [-Werror]18 | (void) (&_x == &_y); \ | ^~ common/page_alloc.c:1843:34: note: in expansion of macro 'min' 1843 | unsigned int inc_order = min(MAX_ORDER, flsl(e - s) - 1);
Quoted 4 lines
generic_fls{l} was used instead of __builtin_clz{l}(x) as if x is 0, the result in undefined.
Signed-off-by Oleksii Kurochko <[email protected]>
Assuming Andrew's pending change goes in as posted, I'm okay with this to (as simply following suit). Without that having happened, I can't very well offer an ack here. Just one nit:
xen/include/xen/bitops.h static inline int generic_flsl(unsigned long x) · line 74
Quoted 9 lines
xen/include/xen/bitops.h +3
-0@@ -69,6 +69,30 @@ static inline int generic_flsl(unsigned long x)#include <asm/bitops.h> +static always_inline __pure int fls(unsigned int x) +{ +if (__builtin_constant_p (x))
Style (missing blanks immediately inside the parentheses), also again ...
xen/include/xen/bitops.h static inline int generic_flsl(unsigned long x) · line 86
Quoted 12 lines
xen/include/xen/bitops.h +12
-0+return generic_fls (x); + +#ifndef arch_fls +#define arch_fls generic_fls +#endif + +return arch_fls (x); +} + +static always_inline __pure int flsl(unsigned long x) +{ +if (__builtin_constant_p (x))
... here.
Jan
PATCH v6 6/20 xen/bitops: put __ffs() and ffz() into linux compatible header
In reply to Oleksii KurochkoThe mentioned macros exist only because of Linux compatible purpose.
The patch defines __ffs() in terms of Xen bitops and it is safe to define in this way ( as __ffs() - 1 ) as considering that __ffs() was defined as __builtin_ctzl(x), which has undefined behavior when x=0, so it is assumed that such cases are not encountered in the current code.
---
Changes in V6:
- new patch for the patch series.
--- xen/arch/arm/include/asm/arm64/bitops.h | 21 --------------------- xen/arch/ppc/include/asm/bitops.h | 21 --------------------- xen/drivers/passthrough/arm/smmu-v3.c | 2 ++ xen/include/xen/linux-compat.h | 3 +++ xen/lib/find-next-bit.c | 1 + 5 files changed, 6 insertions(+), 42 deletions(-)
xen/arch/arm/include/asm/arm64/bitops.h +0 -21
@@ -1,27 +1,6 @@#ifndef _ARM_ARM64_BITOPS_H#define _ARM_ARM64_BITOPS_H -/* Based on linux/include/asm-generic/bitops/builtin-__ffs.h */ -/** -* __ffs - find first bit in word. -* @word: The word to search -* -* Undefined if no bit exists, so code should check against 0 first. -*/ -static /*__*/ always_inline unsigned long __ffs (unsigned long word) -{ -return __builtin_ctzl (word); -} - -/* Based on linux/include/asm-generic/bitops/ffz.h */ -/* -* ffz - find first zero in word. -* @word: The word to search -* -* Undefined if no zero exists, so code should check against ~0UL first. -*/ -#define ffz (x) __ffs(~(x)) -static inline int arch_flsl (unsigned long x) {uint64_t ret;
xen/arch/ppc/include/asm/bitops.h +0 -21
@@ -171,15 +171,6 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)return (old & mask) !=0 ; } -/* Based on linux/include/asm-generic/bitops/ffz.h */ -/* -* ffz - find first zero in word. -* @word: The word to search -* -* Undefined if no zero exists, so code should check against ~0UL first. -*/ -#define ffz (x) __ffs(~(x)) -/** * hweightN - returns the hamming weight of a N-bit word * @x: the word to weigh
@@ -191,16 +182,4 @@ static inline int __test_and_clear_bit(int nr, volatile void *addr)#define hweight16 (x) __builtin_popcount((uint16_t)(x))#define hweight8 (x) __builtin_popcount((uint8_t)(x)) -/* Based on linux/include/asm-generic/bitops/builtin-__ffs.h */ -/** -* __ffs - find first bit in word. -* @word: The word to search -* -* Undefined if no bit exists, so code should check against 0 first. -*/ -static always_inline unsigned long __ffs (unsigned long word) -{ -return __builtin_ctzl (word); -} -#endif /* _ASM_PPC_BITOPS_H */
xen/drivers/passthrough/arm/smmu-v3.c +2 -0
@@ -72,12 +72,14 @@*/ #include <xen/acpi.h> +#include <xen/bitops.h> #include <xen/config.h> #include <xen/delay.h> #include <xen/errno.h> #include <xen/err.h> #include <xen/irq.h> #include <xen/lib.h> +#include <xen/linux-compat.h> #include <xen/list.h> #include <xen/mm.h> #include <xen/rbtree.h>
xen/include/xen/linux-compat.h +3 -0
@@ -19,4 +19,7 @@ typedef int64_t __s64;typedef paddr_t phys_addr_t ; +#define __ffs (x) (ffsl(x) - 1) +#define ffz (x) __ffs(~(x)) +#endif /* __XEN_LINUX_COMPAT_H__ */
xen/lib/find-next-bit.c +1 -0
@@ -9,6 +9,7 @@* 2 of the License, or (at your option) any later version. */ #include <xen/bitops.h> +#include <xen/linux-compat.h> #include <asm/byteorder.h>
Signature · 1 line
2.43.0
Re: [PATCH v6 06/20] xen/bitops: put __ffs() and ffz() into linux compatible header
xen/lib/find-next-bit.c line 14
Quoted 9 lines · Oleksii Kurochko
xen/lib/find-next-bit.c +1
-0@@ -9,6 +9,7 @@* 2 of the License, or (at your option) any later version. */ #include <xen/bitops.h> +#include <xen/linux-compat.h> #include <asm/byteorder.h>
Hmm, no, a library source would better not include this header. Surely the ffz() can be taken care of locally here?
Jan
Re: [PATCH v6 06/20] xen/bitops: put __ffs() and ffz() into linux compatible header
Quoted 14 lines · Jan Beulich
Quoted 9 lines · Oleksii Kurochko
xen/lib/find-next-bit.c +1
-0@@ -9,6 +9,7 @@* 2 of the License, or (at your option) any later version. */ #include <xen/bitops.h> +#include <xen/linux-compat.h>
#include <asm/byteorder.h>Hmm, no, a library source would better not include this header. Surely
theffz()can be taken care of locally here?
Considering that ffz() is only used here, then it makes sense to drop ffz() from xen/linux-compat.h> and defines it here as:
#define ffz(x) ffsl(~(x)) - 1
~ Oleksii
Re: [PATCH v6 06/20] xen/bitops: put __ffs() and ffz() into linux compatible header
In reply to Jan BeulichQuoted 14 lines · Jan Beulich
Quoted 9 lines · Oleksii Kurochko
xen/lib/find-next-bit.c +1
-0@@ -9,6 +9,7 @@* 2 of the License, or (at your option) any later version. */ #include <xen/bitops.h> +#include <xen/linux-compat.h>
#include <asm/byteorder.h>Hmm, no, a library source would better not include this header. Surely
theffz()can be taken care of locally here?
Except that __ffs() from xen/linux-compat.h is used in find-next-bit.c, so it seems that it is still need to include the header.
~ Oleksii
Re: [PATCH v6 06/20] xen/bitops: put __ffs() and ffz() into linux compatible header
Quoted 15 lines · Oleksii
Quoted 14 lines · Jan Beulich
Quoted 9 lines · Oleksii Kurochko
xen/lib/find-next-bit.c +1
-0@@ -9,6 +9,7 @@* 2 of the License, or (at your option) any later version. */ #include <xen/bitops.h> +#include <xen/linux-compat.h>
#include <asm/byteorder.h>Hmm, no, a library source would better not include this header. Surely
theffz()can be taken care of locally here?
Quoted 2 lines
Except that
__ffs()fromxen/linux-compat.his used infind-next-bit.c, so it seems that it is still need to include the header.
Hmm, no - that __ffs() use, if we mean that to become a Linux compat thing only, should then be replaced (or covered locally), too, imo.
Jan
PATCH v6 7/20 xen/riscv: introduce bitops.h
In reply to Oleksii KurochkoTaken from Linux-6.4.0-rc1
Xen's bitops.h consists of several Linux's headers:
* linux/arch/include/asm/bitops.h:
* The following function were removed as they aren't used in Xen:
* test_and_set_bit_lock
* clear_bit_unlock
* __clear_bit_unlock
* The following functions were renamed in the way how they are used by common code:
* __test_and_set_bit
* __test_and_clear_bit
* The declaration and implementation of the following functios were updated to make Xen build happy:
* clear_bit
* set_bit
* __test_and_clear_bit
* __test_and_set_bit
* linux/include/asm-generic/bitops/generic-non-atomic.h with the following changes:
* Only functions that can be reused in Xen were left;
others were removed.
* it was updated the message inside #ifndef ... #endif.
* __always_inline -> always_inline to be align with definition in xen/compiler.h.
* update function prototypes from generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr) to
generic___test_and_*(unsigned long nr, volatile void *addr) to be consistent with other related macros/defines.
* convert identations from tabs to spaces.
* inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long' to be generic.
---
Changes in V6:
- rebase clean ups were done: drop unused asm-generic includes
---
Changes in V5:
- new patch
--- xen/arch/riscv/include/asm/bitops.h | 144 ++++++++++++++++++++++++++++ xen/arch/riscv/include/asm/config.h | 2 + 2 files changed, 146 insertions(+) create mode 100644 xen/arch/riscv/include/asm/bitops.h
xen/arch/riscv/include/asm/bitops.h new file +144 -0
@@ -0,0 +1,144 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2012 Regents of the University of California */ + +#ifndef _ASM_RISCV_BITOPS_H +#define _ASM_RISCV_BITOPS_H + +#include <asm/system.h> + +#define BITOP_BITS_PER_WORD BITS_PER_LONG + +#define BITOP_TYPE +typedef uint64_t bitops_uint_t ; + +#include <asm-generic/bitops/bitops-bits.h> + +#define __set_bit (n, p) set_bit(n, p) +#define __clear_bit (n, p) clear_bit(n, p) + +/* Based on linux/arch/include/asm/bitops.h */ + +#if BITS_PER_LONG ==64 +#define __AMO (op) "amo" #op ".d" +#elif BITS_PER_LONG ==32 +#define __AMO (op) "amo" #op ".w" +#else +#error "Unexpected BITS_PER_LONG" +#endif + +#define test_and_op_bit_ord (op, mod, nr, addr, ord) \ +({ \ + unsigned long res, mask; \ + mask = BITOP_MASK(nr); \ + __asm__ __volatile__ ( \ + __AMO(op) #ord " %0, %2, %1" \ + : "=r" (res), "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(mask)) \ + : "memory"); \ + ((res & mask) != 0); \ +}) + +#define __op_bit_ord (op, mod, nr, addr, ord) \ + __asm__ __volatile__ ( \ + __AMO(op) #ord " zero, %1, %0" \ + : "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(BITOP_MASK(nr))) \ + : "memory"); + +#define test_and_op_bit (op, mod, nr, addr) \ + test_and_op_bit_ord(op, mod, nr, addr, .aqrl) +#define __op_bit (op, mod, nr, addr) \ + __op_bit_ord(op, mod, nr, addr, ) + +/* Bitmask modifiers */ +#define NOP (x) (x) +#define NOT (x) (~(x)) + +/** +* test_and_set_bit - Set a bit and return its old value +* @nr: Bit to set +* @addr: Address to count from +*/ +static inline int test_and_set_bit (int nr,volatile void *p) +{ +volatile bitops_uint_t *addr = p; + +return test_and_op_bit (or,NOP , nr, addr); +} + +/** +* test_and_clear_bit - Clear a bit and return its old value +* @nr: Bit to clear +* @addr: Address to count from +*/ +static inline int test_and_clear_bit (int nr,volatile void *p) +{ +volatile bitops_uint_t *addr = p; + +return test_and_op_bit (and,NOT , nr, addr); +} + +/** +* set_bit - Atomically set a bit in memory +* @nr: the bit to set +* @addr: the address to start counting from +* +* Note that @nr may be almost arbitrarily large; this function is not +* restricted to acting on a single-word quantity. +*/ +static inline void set_bit (int nr,volatile void *p) +{ +volatile bitops_uint_t *addr = p; + +__op_bit (or,NOP , nr, addr); +} + +/** +* clear_bit - Clears a bit in memory +* @nr: Bit to clear +* @addr: Address to start counting from +*/ +static inline void clear_bit (int nr,volatile void *p) +{ +volatile bitops_uint_t *addr = p; + +__op_bit (and,NOT , nr, addr); +} + +/** +* test_and_change_bit - Toggle (change) a bit and return its old value +* @nr: Bit to change +* @addr: Address to count from +* +* This operation is atomic and cannot be reordered. +* It also implies a memory barrier. +*/ +static inline int test_and_change_bit (int nr,volatile unsigned long *addr) +{ +return test_and_op_bit (xor,NOP , nr, addr); +} + +#undef test_and_op_bit +#undef __op_bit +#undef NOP +#undef NOT +#undef __AMO + +#include <asm-generic/bitops/generic-non-atomic.h> + +#define __test_and_set_bit generic___test_and_set_bit +#define __test_and_clear_bit generic___test_and_clear_bit +#define __test_and_change_bit generic___test_and_change_bit + +#include <asm-generic/bitops/test-bit.h> + +#endif /* _ASM_RISCV_BITOPS_H */ + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* indent-tabs-mode: nil +* End: +*/
xen/arch/riscv/include/asm/config.h +2 -0
@@ -113,6 +113,8 @@# error "Unsupported RISCV variant"#endif +#define BITS_PER_BYTE 8 +#define BYTES_PER_LONG (1 << LONG_BYTEORDER)#define BITS_PER_LONG (BYTES_PER_LONG << 3)#define POINTER_ALIGN BYTES_PER_LONG
Signature · 1 line
2.43.0
Re: [PATCH v6 07/20] xen/riscv: introduce bitops.h
Quoted 25 lines · Oleksii Kurochko
Taken from Linux-6.4.0-rc1
Xen's
bitops.hconsists of several Linux's headers:
*linux/arch/include/asm/bitops.h:
* The following function were removed as they aren't used in Xen:
*test_and_set_bit_lock
*clear_bit_unlock
*__clear_bit_unlock
* The following functions were renamed in the way how they are used by common code:
*__test_and_set_bit
*__test_and_clear_bit
* The declaration and implementation of the following functios were updated to make Xen build happy:
*clear_bit
*set_bit
*__test_and_clear_bit
*__test_and_set_bit
*linux/include/asm-generic/bitops/generic-non-atomic.hwith the following changes:
* Only functions that can be reused in Xen were left; others were removed.
* it was updated the message inside #ifndef ... #endif.
*__always_inline->always_inlineto be align with definition inxen/compiler.h.
Quoted 8 lines
* update function prototypes from generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr) to
generic___test_and_*(unsigned long nr, volatile void *addr) to be consistent with other related macros/defines.
* convert identations from tabs to spaces.
* inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long' to be generic.
This last middle level bullet point looks stale here here, with that header now being introduced in a separate patch.
Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +7
-0@@ -0,0 +1,144 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2012 Regents of the University of California */ + +#ifndef _ASM_RISCV_BITOPS_H +#define _ASM_RISCV_BITOPS_H + +#include <asm/system.h>
xen/arch/riscv/include/asm/bitops.h line 17
Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10
-0+ +#define BITOP_BITS_PER_WORD BITS_PER_LONG + +#define BITOP_TYPE +typedef uint64_t bitops_uint_t ; + +#include <asm-generic/bitops/bitops-bits.h> + +#define __set_bit (n, p) set_bit(n, p) +#define __clear_bit (n, p) clear_bit(n, p)
If these cam with a TODO, I wouldn't say anything. But without I take it they're meant to remain that way, at which point I'd like to ask about the performance aspect: Surely the AMO insns are more expensive than whatever more basic insns could be used instead? I'd even go as far as wondering whether
#define __set_bit(n, p) ((void)__test_and_set_bit(n, p))
#define __clear_bit(n, p) ((void)__test_and_clear_bit(n, p))
wouldn't be cheaper (the compiler would recognize the unused result and eliminate its calculation, I'm pretty sure).
xen/arch/riscv/include/asm/bitops.h line 24
Quoted 6 lines
xen/arch/riscv/include/asm/bitops.h new file +6
-0+/* Based on linux/arch/include/asm/bitops.h */ + +#if BITS_PER_LONG ==64 +#define __AMO (op) "amo" #op ".d" +#elif BITS_PER_LONG ==32 +#define __AMO (op) "amo" #op ".w"
This isn't in line with bitops_uint_t above. With BITOP_BITS_PER_WORD also expanding to BITS_PER_LONG, I think you mean to use unsigned long there. Or else you could move stuff into the conditional here (or really move the conditional here further up).
However, if you look at Arm64 and x86 code, you will notice that they avoid 64-bit operations here. I'm afraid I can't easily tell whether the reason(s) for this have meanwhile disappeared; I fear some of that is still around.
Quoted 17 lines
xen/arch/riscv/include/asm/bitops.h new file +17
-0+#else +#error "Unexpected BITS_PER_LONG" +#endif + +#define test_and_op_bit_ord (op, mod, nr, addr, ord) \ +({ \ + unsigned long res, mask; \ + mask = BITOP_MASK(nr); \ + __asm__ __volatile__ ( \ + __AMO(op) #ord " %0, %2, %1" \ + : "=r" (res), "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(mask)) \ + : "memory"); \ + ((res & mask) != 0); \ +}) + +#define __op_bit_ord (op, mod, nr, addr, ord) \
xen/arch/riscv/include/asm/bitops.h line 51
Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10
-0+ __asm__ __volatile__ ( \ + __AMO(op) #ord " zero, %1, %0" \ + :"+A" (addr[BITOP_WORD (nr)]) \ + :"r" (mod (BITOP_MASK (nr))) \ + :"memory" ); + +#define test_and_op_bit (op, mod, nr, addr) \ + test_and_op_bit_ord(op, mod, nr, addr, .aqrl) +#define __op_bit (op, mod, nr, addr) \ + __op_bit_ord(op, mod, nr, addr, )
Why are double underscores left on the non-test_and_ macros? Even without name space considerations that looks odd here, when talk is of the atomic forms of the operations, which by convention have no (double) underscore at their front.
xen/arch/riscv/include/asm/config.h line 120
Quoted 11 lines
xen/arch/riscv/include/asm/config.h +2
-0@@ -113,6 +113,8 @@# error "Unsupported RISCV variant"#endif +#define BITS_PER_BYTE 8 +#define BYTES_PER_LONG (1 << LONG_BYTEORDER)#define BITS_PER_LONG (BYTES_PER_LONG << 3)#define POINTER_ALIGN BYTES_PER_LONG
How's this addition related to the change at hand?
Jan
Re: [PATCH v6 07/20] xen/riscv: introduce bitops.h
Quoted 73 lines · Jan Beulich
Quoted 29 lines · Oleksii Kurochko
Taken from Linux-6.4.0-rc1
Xen's
bitops.hconsists of several Linux's headers:
*linux/arch/include/asm/bitops.h:
* The following function were removed as they aren't used in Xen:
*test_and_set_bit_lock
*clear_bit_unlock
*__clear_bit_unlock
* The following functions were renamed in the way how they are used by common code:
*__test_and_set_bit
*__test_and_clear_bit
* The declaration and implementation of the following functios were updated to make Xen build happy:
*clear_bit
*set_bit
*__test_and_clear_bit
*__test_and_set_bit
*linux/include/asm-generic/bitops/generic-non-atomic.hwith the following changes:
* Only functions that can be reused in Xen were left; others were removed.
* it was updated the message inside #ifndef ... #endif.
*__always_inline->always_inlineto be align with definition in
xen/compiler.h.
* update function prototypes from generic___test_and_*(unsigned long nr nr, volatile unsigned long *addr)Quoted 8 lines
to
generic___test_and_*(unsigned long nr, volatile void *addr) to be
consistent with other related macros/defines.
* convert identations from tabs to spaces.
* inside generic__test_and_* use 'bitops_uint_t' instead of 'unsigned long'
to be generic.This last middle level bullet point looks stale here here, with that header
now being introduced in a separate patch.Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +7
-0@@ -0,0 +1,144 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2012 Regents of the University of California */ + +#ifndef _ASM_RISCV_BITOPS_H +#define _ASM_RISCV_BITOPS_H + +#include <asm/system.h> xen/arch/riscv/include/asm/bitops.h line 17
Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10
-0+ +#define BITOP_BITS_PER_WORD BITS_PER_LONG + +#define BITOP_TYPE +typedef uint64_t bitops_uint_t ; + +#include <asm-generic/bitops/bitops-bits.h> + +#define __set_bit (n, p) set_bit(n, p) +#define __clear_bit (n, p) clear_bit(n, p)If these cam with a TODO, I wouldn't say anything. But without I take it
they're meant to remain that way, at which point I'd like to ask about
the performance aspect: Surely the AMO insns are more expensive than whatever more basic insns could be used instead? I'd even go as far as
wondering whether
Quoted 5 lines
#define
__set_bit(n, p)((void)__test_and_set_bit(n, p)) #define__clear_bit(n, p)((void)__test_and_clear_bit(n, p))wouldn't be cheaper (the compiler would recognize the unused result and eliminate its calculation, I'm pretty sure).
It was implemented using atomic ops because of Arm:
/*
* Non-atomic bit manipulation.
*
* Implemented using atomics to be interrupt safe. Could alternatively
* implement with local interrupt masking.
*/
#define __set_bit(n,p) set_bit(n,p)
#define __clear_bit(n,p) clear_bit(n,p)
I though that the same comment is true for x86, but after your comment I checked x86 implementation, I realized that x86 uses non-atomic operations.
In this case, it seems to me there is a sense to use non-atomic for RISC-V too.
Quoted 43 lines
xen/arch/riscv/include/asm/bitops.h line 24
Quoted 6 lines
xen/arch/riscv/include/asm/bitops.h new file +6
-0+/* Based on linux/arch/include/asm/bitops.h */ + +#if BITS_PER_LONG ==64 +#define __AMO (op) "amo" #op ".d" +#elif BITS_PER_LONG ==32 +#define __AMO (op) "amo" #op ".w"This isn't in line with
bitops_uint_tabove. WithBITOP_BITS_PER_WORDalso expanding toBITS_PER_LONG, I think you mean to use unsigned long
there. Or else you could move stuff into the conditional here (or really move the conditional here further up).However, if you look at Arm64 and x86 code, you will notice that they avoid 64-bit operations here. I'm afraid I can't easily tell whether the reason(s) for this have meanwhile disappeared; I fear some of that
is still around.Quoted 24 lines
xen/arch/riscv/include/asm/bitops.h new file +24
-0+#else +#error "Unexpected BITS_PER_LONG" +#endif + +#define test_and_op_bit_ord (op, mod, nr, addr, ord) \ +({ \ + unsigned long res, mask; \ + mask = BITOP_MASK(nr); \ + __asm__ __volatile__ ( \ + __AMO(op) #ord " %0, %2, %1" \ + : "=r" (res), "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(mask)) \ + : "memory"); \ + ((res & mask) != 0); \ +}) + +#define __op_bit_ord (op, mod, nr, addr, ord) \ + __asm__ __volatile__ ( \ + __AMO(op) #ord " zero, %1, %0" \ + : "+A" (addr[BITOP_WORD(nr)]) \ + : "r" (mod(BITOP_MASK(nr))) \ + : "memory"); + +#define test_and_op_bit (op, mod, nr, addr) \
Quoted 8 lines
xen/arch/riscv/include/asm/bitops.h line 51
Quoted 3 lines
xen/arch/riscv/include/asm/bitops.h new file +3
-0+ test_and_op_bit_ord(op, mod, nr, addr, .aqrl) +#define __op_bit (op, mod, nr, addr) \ + __op_bit_ord(op, mod, nr, addr, )Why are double underscores left on the non-test_and_ macros? Even without name space considerations that looks odd here, when talk is of the atomic forms of the operations, which by convention have no (double) underscore at their front.
Missed to update after Linux kernel. I'll take that into account.
Quoted 14 lines
Quoted 9 lines
xen/arch/riscv/include/asm/config.h +0
-0@@ -113,6 +113,8 @@ # error "Unsupported RISCV variant" #endifxen/arch/riscv/include/asm/config.h +2
-0+#define BITS_PER_BYTE 8 +Quoted 3 lines
#define
BYTES_PER_LONG(1 <<LONG_BYTEORDER)
#define BITS_PER_LONG (BYTES_PER_LONG<< 3)
#define POINTER_ALIGNBYTES_PER_LONGHow's this addition related to the change at hand?
It should be a part of cmpxchg.h patch. Thanks.
~ Oleksii
Quoted 2 lines
Jan
Re: [PATCH v6 07/20] xen/riscv: introduce bitops.h
Quoted 47 lines · Oleksii
Quoted 31 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/bitops.h new file +7
-0@@ -0,0 +1,144 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2012 Regents of the University of California */ + +#ifndef _ASM_RISCV_BITOPS_H +#define _ASM_RISCV_BITOPS_H + +#include <asm/system.h> xen/arch/riscv/include/asm/bitops.h line 17
Quoted 10 lines
xen/arch/riscv/include/asm/bitops.h new file +10
-0+ +#define BITOP_BITS_PER_WORD BITS_PER_LONG + +#define BITOP_TYPE +typedef uint64_t bitops_uint_t ; + +#include <asm-generic/bitops/bitops-bits.h> + +#define __set_bit (n, p) set_bit(n, p) +#define __clear_bit (n, p) clear_bit(n, p)If these cam with a TODO, I wouldn't say anything. But without I take it
they're meant to remain that way, at which point I'd like to ask about
the performance aspect: Surely the AMO insns are more expensive than whatever more basic insns could be used instead? I'd even go as far as
wondering whetherQuoted 5 lines
#define
__set_bit(n, p)((void)__test_and_set_bit(n, p)) #define__clear_bit(n, p)((void)__test_and_clear_bit(n, p))wouldn't be cheaper (the compiler would recognize the unused result and eliminate its calculation, I'm pretty sure).
It was implemented using atomic ops because of Arm:
/*
* Non-atomic bit manipulation.
*
* Implemented using atomics to be interrupt safe. Could alternatively
* implement with local interrupt masking.
*/
#define__set_bit(n,p)set_bit(n,p)
#define__clear_bit(n,p)clear_bit(n,p)
Quoted 6 lines
I though that the same comment is true for x86, but after your comment I checked x86 implementation, I realized that x86 uses non-atomic operations.
In this case, it seems to me there is a sense to use non-atomic for RISC-V too.
Hmm, wait: There's an important difference between x86 on one side and Arm/RISC-V/PPC and various other more or less RISC-like ones on the other. x86 has read-modify-write (memory) insns. Therefore even without using their atomic (LOCKed) forms, they do the update atomically as far as the local CPU is concerned. That's not the case when you need to use a three (or more) step load-op-store sequence.
Had you retained Arm's comment, I probably wouldn't even have asked. Please add such a comment while sticking to this aliasing you have.
Jan
PATCH v6 8/20 xen/riscv: introduce cmpxchg.h
In reply to Oleksii KurochkoThe header was taken from Linux kernl 6.4.0-rc1.
Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use them
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )
Implementation of 4- and 8-byte cases were updated according to the spec:
```
....
Linux Construct RVWMO AMO Mapping
atomic <op> relaxed amo<op>.{w|d}
atomic <op> acquire amo<op>.{w|d}.aq
atomic <op> release amo<op>.{w|d}.rl
atomic <op> amo<op>.{w|d}.aqrl
Linux Construct RVWMO LR/SC Mapping
atomic <op> relaxed loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop
atomic <op> acquire loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez loop OR
fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗ ; bnez loop
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop
Table A.5: Mappings from Linux memory primitives to RISC-V primitives
```---
Changes in V6:
- update the commit message? ( As before I don't understand this point. Can you give an example of what sort of opcode / instruction is missing?)
- Code style fixes
- change sizeof(*ptr) -> sizeof(*(ptr))
- update operands names and some local variables for macros emulate_xchg_1_2() and emulate_cmpxchg_1_2()
- drop {cmp}xchg_{relaxed,acquire,release) versions as they aren't needed for Xen
- update __amoswap_generic() prototype and defintion: drop pre and post barries.
- update emulate_xchg_1_2() prototype and definion: add lr_sfx, drop pre and post barries.
- rename __xchg_generic to __xchg(), make __xchg as static inline function to be able to "#ifndef CONFIG_32BIT case 8:... "
---
Changes in V5:
- update the commit message.
- drop ALIGN_DOWN().
- update the definition of emulate_xchg_1_2():
- lr.d -> lr.w, sc.d -> sc.w.
- drop ret argument.
- code style fixes around asm volatile.
- update prototype.
- use asm named operands.
- rename local variables.
- add comment above the macros
- update the definition of __xchg_generic:
- rename to __xchg()
- transform it to static inline
- code style fixes around switch()
- update prototype.
- redefine cmpxchg()
- update emulate_cmpxchg_1_2():
- update prototype
- update local variables names and usage of them
- use name asm operands.
- add comment above the macros
- drop pre and post, and use .aq,.rl, .aqrl suffixes.
- drop {cmp}xchg_{relaxed, aquire, release} as they are not used by Xen.
- drop unnessary details in comment above emulate_cmpxchg_1_2()
---
Changes in V4:
- Code style fixes.
- enforce in __xchg_*() has the same type for new and *ptr, also "\n"
was removed at the end of asm instruction.
- dependency from https://lore.kernel.org/xen-devel/[email protected]/
- switch from ASSERT_UNREACHABLE to STATIC_ASSERT_UNREACHABLE().
- drop xchg32(ptr, x) and xchg64(ptr, x) as they aren't used.
- drop cmpxcg{32,64}_{local} as they aren't used.
- introduce generic version of xchg_* and cmpxchg_*.
- update the commit message.
---
Changes in V3:
- update the commit message
- add emulation of {cmp}xchg_... for 1 and 2 bytes types
---
Changes in V2:
- update the comment at the top of the header.
- change xen/lib.h to xen/bug.h.
- sort inclusion of headers properly.
--- xen/arch/riscv/include/asm/cmpxchg.h | 209 +++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 xen/arch/riscv/include/asm/cmpxchg.h
xen/arch/riscv/include/asm/cmpxchg.h new file +209 -0
@@ -0,0 +1,209 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2014 Regents of the University of California */ + +#ifndef _ASM_RISCV_CMPXCHG_H +#define _ASM_RISCV_CMPXCHG_H + +#include <xen/compiler.h> +#include <xen/lib.h> + +#include <asm/fence.h> +#include <asm/io.h> +#include <asm/system.h> + +#define __amoswap_generic (ptr, new, ret, sfx) \ +({ \ + asm volatile ( \ + " amoswap" sfx " %0, %2, %1" \ + : "=r" (ret), "+A" (*ptr) \ + : "r" (new) \ + : "memory" ); \ +}) + +/* +* For LR and SC, the A extension requires that the address held in rs1 be +* naturally aligned to the size of the operand (i.e., eight-byte aligned +* for 64-bit words and four-byte aligned for 32-bit words). +* If the address is not naturally aligned, an address-misaligned exception +* or an access-fault exception will be generated. +* +* Thereby: +* - for 1-byte xchg access the containing word by clearing low two bits +* - for 2-byte xchg ccess the containing word by clearing bit 1. +* +* If resulting 4-byte access is still misalgined, it will fault just as +* non-emulated 4-byte access would. +*/ +#define emulate_xchg_1_2 (ptr, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr & ~(0x4 - sizeof(*(ptr)))); \ + unsigned int new_val_pos = ((unsigned long)(ptr) & (0x4 - sizeof(*(ptr)))) * BITS_PER_BYTE; \ + unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_pos; \ + unsigned int new_ = new << new_val_pos; \ + unsigned int old; \ + unsigned int scratch; \ + \ + asm volatile ( \ + "0: lr.w" lr_sfx " %[old], %[aligned_ptr]\n" \ + " and %[scratch], %[old], %z[nmask]\n" \ + " or %[scratch], %[scratch], %z[new_]\n" \ + " sc.w" sc_sfx " %[scratch], %[scratch], %[aligned_ptr]\n" \ + " bnez %[scratch], 0b\n" \ + : [old] "=&r" (old), [scratch] "=&r" (scratch), [aligned_ptr] "+A" (*aligned_ptr) \ + : [new_] "rJ" (new_), [nmask] "rJ" (~mask) \ + : "memory" ); \ + \ + (__typeof__(*(ptr)))((old & mask) >> new_val_pos); \ +}) + +static always_inline unsigned long __xchg (volatile void *ptr,unsigned long new,int size) +{ +unsigned long ret; + +switch ( size ) + { +case 1 : + ret =emulate_xchg_1_2 ((volatile uint8_t *)ptr, new,".aq" ,".aqrl" ); +break ; +case 2 : + ret =emulate_xchg_1_2 ((volatile uint16_t *)ptr, new,".aq" ,".aqrl" ); +break ; +case 4 : +__amoswap_generic ((volatile uint32_t *)ptr, new, ret,".w.aqrl" ); +break ; +#ifndef CONFIG_32BIT +case 8 : +__amoswap_generic ((volatile uint64_t *)ptr, new, ret,".d.aqrl" ); +break ; +#endif +default : +STATIC_ASSERT_UNREACHABLE (); + } + +return ret; +} + +#define xchg (ptr, x) \ +({ \ + __typeof__(*(ptr)) n_ = (x); \ + (__typeof__(*(ptr))) \ + __xchg((ptr), (unsigned long)(n_), sizeof(*(ptr))); \ +}) + +#define __generic_cmpxchg (ptr, old, new, ret, lr_sfx, sc_sfx) \ + ({ \ + register unsigned int rc; \ + __typeof__(*(ptr)) old__ = (__typeof__(*(ptr)))(old); \ + __typeof__(*(ptr)) new__ = (__typeof__(*(ptr)))(new); \ + asm volatile( \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + "1:\n" \ + : "=&r" (ret), "=&r" (rc), "+A" (*ptr) \ + : "rJ" (old__), "rJ" (new__) \ + : "memory"); \ + }) + +/* +* For LR and SC, the A extension requires that the address held in rs1 be +* naturally aligned to the size of the operand (i.e., eight-byte aligned +* for 64-bit words and four-byte aligned for 32-bit words). +* If the address is not naturally aligned, an address-misaligned exception +* or an access-fault exception will be generated. +* +* Thereby: +* - for 1-byte xchg access the containing word by clearing low two bits +* - for 2-byte xchg ccess the containing word by clearing first bit. +* +* If resulting 4-byte access is still misalgined, it will fault just as +* non-emulated 4-byte access would. +* +* old_val was casted to unsigned long for cmpxchgptr() +*/ +#define emulate_cmpxchg_1_2 (ptr, old, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr & ~(0x4 - sizeof(*(ptr)))); \ + uint8_t new_val_pos = ((unsigned long)(ptr) & (0x4 - sizeof(*(ptr)))) * BITS_PER_BYTE; \ + unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_pos; \ + unsigned int old_ = old << new_val_pos; \ + unsigned int new_ = new << new_val_pos; \ + unsigned int old_val; \ + unsigned int scratch; \ + \ + __asm__ __volatile__ ( \ + "0: lr.w" lr_sfx " %[scratch], %[aligned_ptr]\n" \ + " and %[old_val], %[scratch], %z[mask]\n" \ + " bne %[old_val], %z[old_], 1f\n" \ + " xor %[scratch], %[old_val], %[scratch]\n" \ + " or %[scratch], %[scratch], %z[new_]\n" \ + " sc.w" sc_sfx " %[scratch], %[scratch], %[aligned_ptr]\n" \ + " bnez %[scratch], 0b\n" \ + "1:\n" \ + : [old_val] "=&r" (old_val), [scratch] "=&r" (scratch), [aligned_ptr] "+A" (*aligned_ptr) \ + : [old_] "rJ" (old_), [new_] "rJ" (new_), \ + [mask] "rJ" (mask) \ + : "memory" ); \ + \ + (__typeof__(*(ptr)))((unsigned long)old_val >> new_val_pos); \ +}) + +/* +* Atomic compare and exchange. Compare OLD with MEM, if identical, +* store NEW in MEM. Return the initial value in MEM. Success is +* indicated by comparing RETURN with OLD. +*/ +static always_inline unsigned long __cmpxchg (volatile void *ptr, +unsigned long old, +unsigned long new, +int size) +{ +unsigned long ret; + +switch ( size ) + { +case 1 : + ret =emulate_cmpxchg_1_2 ((volatile uint8_t *)ptr, old, new, +".aq" ,".aqrl" ); +break ; +case 2 : + ret =emulate_cmpxchg_1_2 ((volatile uint16_t *)ptr, old, new, +".aq" ,".aqrl" ); +break ; +case 4 : +__generic_cmpxchg ((volatile uint32_t *)ptr, old, new, ret, +".w.aq" ,".w.aqrl" ); +break ; +#ifndef CONFIG_32BIT +case 8 : +__generic_cmpxchg ((volatile uint64_t *)ptr, old, new, + ret,".d.aq" ,".d.aqrl" ); +break ; +#endif +default : +STATIC_ASSERT_UNREACHABLE (); + } + +return ret; +} + +#define cmpxchg (ptr, o, n) \ +({ \ + __typeof__(*(ptr)) o_ = (o); \ + __typeof__(*(ptr)) n_ = (n); \ + (__typeof__(*(ptr))) \ + __cmpxchg((ptr), (unsigned long)(o_), (unsigned long)(n_), \ + sizeof(*(ptr))); \ +}) + +#endif /* _ASM_RISCV_CMPXCHG_H */ + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* indent-tabs-mode: nil +* End: +*/
Signature · 1 line
2.43.0
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
Quoted 9 lines · Oleksii Kurochko
The header was taken from Linux kernl 6.4.0-rc1.
Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use them
With this, ...
Quoted 12 lines
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )
Implementation of 4- and 8-byte cases were updated according to the spec:
``` .... Linux Construct RVWMO AMO Mapping atomic <op> relaxed amo<op>.{w|d} atomic <op> acquire amo<op>.{w|d}.aq atomic <op> release amo<op>.{w|d}.rl atomic <op> amo<op>.{w|d}.aqrl Linux Construct RVWMO LR/SC Mapping atomic <op> relaxed loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop
Quoted 8 lines
atomic <op> acquire loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez loop OR
fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗ ; bnez loop
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop
Table A.5: Mappings from Linux memory primitives to RISC-V primitives
```... I consider quoting this table in full, without any further remarks, as
confusing: Three of the lines each are inapplicable now, aiui.
Further what are the two * telling us? Quite likely they aren't there just accidentally.
Finally, why sc.{w|d}.aqrl when in principle one would expect just sc.{w|d}.rl?
Quoted 7 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +4
-0@@ -0,0 +1,209 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright (C) 2014 Regents of the University of California */ + +#ifndef _ASM_RISCV_CMPXCHG_H
xen/arch/riscv/include/asm/cmpxchg.h line 14
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+#define _ASM_RISCV_CMPXCHG_H + +#include <xen/compiler.h> +#include <xen/lib.h> + +#include <asm/fence.h> +#include <asm/io.h> +#include <asm/system.h> + +#define __amoswap_generic (ptr, new, ret, sfx) \
As before / elsewhere: Is there a strong need for two leading underscores here? Using just one would already be standard compliant afaict.
xen/arch/riscv/include/asm/cmpxchg.h line 21
Quoted 7 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +7
-0+({ \ + asmvolatile ( \ +" amoswap" sfx" %0, %2, %1" \ + :"=r" (ret),"+A" (*ptr) \ + :"r" (new) \ + :"memory" ); \ +})
This doesn't need the ({ }) (anymore?):
#define __amoswap_generic(ptr, new, ret, sfx) \
asm volatile ( \
" amoswap" sfx " %0, %2, %1" \
: "=r" (ret), "+A" (*(ptr)) \
: "r" (new) \
: "memory" )(note also the added parentheses).
xen/arch/riscv/include/asm/cmpxchg.h line 32
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+/* +* For LR and SC, the A extension requires that the address held in rs1 be +* naturally aligned to the size of the operand (i.e., eight-byte aligned +* for 64-bit words and four-byte aligned for 32-bit words). +* If the address is not naturally aligned, an address-misaligned exception +* or an access-fault exception will be generated. +* +* Thereby: +* - for 1-byte xchg access the containing word by clearing low two bits +* - for 2-byte xchg ccess the containing word by clearing bit 1.
Nit: "access"
xen/arch/riscv/include/asm/cmpxchg.h line 40
Quoted 7 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +7
-0+* If resulting 4-byte access is still misalgined, it will fault just as +* non-emulated 4-byte access would. +*/ +#define emulate_xchg_1_2 (ptr, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr & ~(0x4 - sizeof(*(ptr)))); \ + unsigned int new_val_pos = ((unsigned long)(ptr) & (0x4 - sizeof(*(ptr)))) * BITS_PER_BYTE; \
You parenthesize ptr here correctly, but not in the line above.
Instead of "_pos" in the name, maybe better "_bit"?
Finally, here and elsewhere, please limit line length to 80 chars. (Omitting the 0x here would help a little, but not quite enough. Question is whether these wouldn't better be sizeof(*aligned_ptr) anyway.)
xen/arch/riscv/include/asm/cmpxchg.h line 40
Quoted 2 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +2
-0+unsigned long mask =GENMASK (((sizeof (*(ptr))) *BITS_PER_BYTE ) -1 ,0 ) << new_val_pos; \ +unsigned int new_ = new << new_val_pos; \
Similarly new wants parenthesizing here.
xen/arch/riscv/include/asm/cmpxchg.h line 52
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+unsigned int old; \ +unsigned int scratch; \ + \ + asmvolatile ( \ +"0: lr.w" lr_sfx" %[old], %[aligned_ptr]\n" \ +" and %[scratch], %[old], %z[nmask]\n" \ +" or %[scratch], %[scratch], %z[new_]\n" \ +" sc.w" sc_sfx" %[scratch], %[scratch], %[aligned_ptr]\n" \ +" bnez %[scratch], 0b\n" \ + : [old] "=&r" (old), [scratch] "=&r" (scratch), [aligned_ptr] "+A" (*aligned_ptr) \
While for the variable name aligned_ptr is likely helpful, for the operand name just ptr would certainly do?
xen/arch/riscv/include/asm/cmpxchg.h line 53
Quoted 1 line
xen/arch/riscv/include/asm/cmpxchg.h new file +1
-0+ : [new_] "rJ" (new_), [nmask] "rJ" (~mask) \
Neither mask nor ~mask can be 0. Hence J here and the z modifier above look pointless. (new_, otoh, can be 0, so allowing x0 to be used in that case is certainly desirable.)
As to using ~mask here: Now that we look to have settled on requiring Zbb, you could use andn instead of and, thus allowing the same register to be used in the asm() and ...
xen/arch/riscv/include/asm/cmpxchg.h line 56
Quoted 3 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +3
-0+ : "memory" ); \ + \ + (__typeof__ (*(ptr )))((old & mask) >> new_val_pos); \
... for this calculation.
Quoted 8 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +8
-0+}) + +static always_inline unsigned long __xchg (volatile void *ptr,unsigned long new,int size) +{ +unsigned long ret; + +switch ( size ) + {
xen/arch/riscv/include/asm/cmpxchg.h line 74
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+case 1 : + ret =emulate_xchg_1_2 ((volatile uint8_t *)ptr, new,".aq" ,".aqrl" ); +break ; +case 2 : + ret =emulate_xchg_1_2 ((volatile uint16_t *)ptr, new,".aq" ,".aqrl" ); +break ; +case 4 : +__amoswap_generic ((volatile uint32_t *)ptr, new, ret,".w.aqrl" ); +break ; +#ifndef CONFIG_32BIT
There's no 32BIT Kconfig symbol; all we have is a 64BIT one.
Quoted 6 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +6
-0+case 8 : +__amoswap_generic ((volatile uint64_t *)ptr, new, ret,".d.aqrl" ); +break ; +#endif +default : +STATIC_ASSERT_UNREACHABLE ();
xen/arch/riscv/include/asm/cmpxchg.h line 90
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+ } + +return ret; +} + +#define xchg (ptr, x) \ +({ \ + __typeof__(*(ptr)) n_ = (x); \ + (__typeof__(*(ptr))) \ + __xchg((ptr), (unsigned long)(n_), sizeof(*(ptr))); \
Nit: While excess parentheses "only" harm readability, they would nevertheless better be omitted (here: the first argument passed).
xen/arch/riscv/include/asm/cmpxchg.h line 95
Quoted 5 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +5
-0+}) + +#define __generic_cmpxchg (ptr, old, new, ret, lr_sfx, sc_sfx) \ + ({ \ + register unsigned int rc; \
Nit: We don't normally use "register", unless accompanied by asm() tying a variable to a specific one.
xen/arch/riscv/include/asm/cmpxchg.h line 97
Quoted 2 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +2
-0+ __typeof__(*(ptr)) old__ = (__typeof__(*(ptr)))(old); \ + __typeof__(*(ptr)) new__ = (__typeof__(*(ptr)))(new); \
The casts aren't very nice to have here; I take they're needed for cmpxchg_ptr() to compile?
xen/arch/riscv/include/asm/cmpxchg.h line 98
Quoted 1 line
xen/arch/riscv/include/asm/cmpxchg.h new file +1
-0+ asm volatile( \
Nit: Missing blank once again. Would be really nice if you could go through and sort this uniformly for the series.
xen/arch/riscv/include/asm/cmpxchg.h line 105
Quoted 7 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +7
-0+"0: lr" lr_sfx" %0, %2\n" \ +" bne %0, %z3, 1f\n" \ +" sc" sc_sfx" %1, %z4, %2\n" \ +" bnez %1, 0b\n" \ +"1:\n" \ + : "=&r" (ret), "=&r" (rc), "+A" (*ptr) \ +: " rJ" (old__), " rJ" (new__) \
Please could I talk you into using named operands here, too?
Also ptr here is lacking parentheses again.
xen/arch/riscv/include/asm/cmpxchg.h line 106
Quoted 1 line
xen/arch/riscv/include/asm/cmpxchg.h new file +1
-0+ : "memory"); \
And yet another missing blank.
xen/arch/riscv/include/asm/cmpxchg.h line 105
Quoted 1 line
xen/arch/riscv/include/asm/cmpxchg.h new file +1
-0+ })
At the use site this construct having a normal return value (rather than ret being passed in) would overall look more natural.
Quoted 21 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +21
-0+/* +* For LR and SC, the A extension requires that the address held in rs1 be +* naturally aligned to the size of the operand (i.e., eight-byte aligned +* for 64-bit words and four-byte aligned for 32-bit words). +* If the address is not naturally aligned, an address-misaligned exception +* or an access-fault exception will be generated. +* +* Thereby: +* - for 1-byte xchg access the containing word by clearing low two bits +* - for 2-byte xchg ccess the containing word by clearing first bit. +* +* If resulting 4-byte access is still misalgined, it will fault just as +* non-emulated 4-byte access would. +* +* old_val was casted to unsigned long for cmpxchgptr() +*/ +#define emulate_cmpxchg_1_2 (ptr, old, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr & ~(0x4 - sizeof(*(ptr)))); \ + uint8_t new_val_pos = ((unsigned long)(ptr) & (0x4 - sizeof(*(ptr)))) * BITS_PER_BYTE; \ + unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE) - 1, 0) << new_val_pos; \
xen/arch/riscv/include/asm/cmpxchg.h line 139
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+unsigned int old_ = old << new_val_pos; \ +unsigned int new_ = new << new_val_pos; \ +unsigned int old_val; \ +unsigned int scratch; \ + \ + __asm__ __volatile__ ( \ +"0: lr.w" lr_sfx" %[scratch], %[aligned_ptr]\n" \ +" and %[old_val], %[scratch], %z[mask]\n" \ +" bne %[old_val], %z[old_], 1f\n" \ + " xor %[scratch], %[old_val], %[scratch]\n" \
To be honest I was hoping this line would come with a brief comment.
xen/arch/riscv/include/asm/cmpxchg.h line 150
Quoted 11 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +11
-0+" or %[scratch], %[scratch], %z[new_]\n" \ +" sc.w" sc_sfx" %[scratch], %[scratch], %[aligned_ptr]\n" \ +" bnez %[scratch], 0b\n" \ +"1:\n" \ + : [old_val] "=&r" (old_val), [scratch] "=&r" (scratch), [aligned_ptr] "+A" (*aligned_ptr) \ +: [old_] " rJ" (old_), [new_] " rJ" (new_), \ +[mask] " rJ" (mask) \ +: " memory" ); \ +\ +(__typeof__(*(ptr)))((unsigned long)old_val >> new_val_pos); \ +})
A few of the comments for emulate_xchg_1_2() apply here as well.
xen/arch/riscv/include/asm/cmpxchg.h line 160
Quoted 9 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +9
-0+/* +* Atomic compare and exchange. Compare OLD with MEM, if identical, +* store NEW in MEM. Return the initial value in MEM. Success is +* indicated by comparing RETURN with OLD. +*/ +static always_inline unsigned long __cmpxchg (volatile void *ptr, +unsigned long old, +unsigned long new, +int size)
Nit: Inappropriate indentation.
Jan
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
Quoted 36 lines · Jan Beulich
Quoted 9 lines · Oleksii Kurochko
The header was taken from Linux kernl 6.4.0-rc1.
Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use themWith this, ...
Quoted 23 lines
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )
Implementation of 4- and 8-byte cases were updated according to the spec:
```
....
Linux Construct RVWMO AMO Mappingatomic <op> relaxed amo<op>.{w|d} atomic <op> acquire amo<op>.{w|d}.aq atomic <op> release amo<op>.{w|d}.rlatomic <op> amo<op>.{w|d}.aqrl
Linux Construct RVWMO LR/SC Mapping
atomic <op> relaxed loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop atomic <op> acquire loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez loop OR
fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗ ; bnez loop
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loop
Quoted 8 lines
Quoted 4 lines
Table A.5: Mappings from Linux memory primitives to RISC-V primitives
```
... I consider quoting this table in full, without any further remarks, as
confusing: Three of the lines each are inapplicable now, aiui.
I'll shorten the table then.
Quoted 7 lines
Further what are the two * telling us? Quite likely they aren't there just
accidentally.Finally, why sc.{w|d}.aqrl when in principle one would expect just sc.{w|d}.rl?
Because according to the last line of table A.5:
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl;Here it is used sc.{w|d}.aqrl form, so I decided to stick to the what is mentioned in the table.
Quoted 17 lines
Quoted 17 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +3
-0+}) + +static always_inline unsigned long __xchg(volatile void *ptr,unsigned long new, int size)
xen/arch/riscv/include/asm/cmpxchg.h new file +7
-0+{ + unsigned long ret; + + switch ( size ) + { + case 1: + ret = emulate_xchg_1_2((volatile uint8_t *)ptr, new,".aq", ".aqrl");
+ break;
+ case 2:
+ ret =emulate_xchg_1_2((volatile uint16_t *)ptr, new, ".aq", ".aqrl");
Quoted 8 lines
Quoted 6 lines
+ break;
+ case 4:
+__amoswap_generic((volatile uint32_t *)ptr, new, ret, ".w.aqrl");
+ break;
+#ifndefCONFIG_32BITThere's no 32BIT
Kconfigsymbol; all we have is a 64BIT one.
I meant here CONFIG_RISCV_32.
Quoted 32 lines
Quoted 8 lines
+ case 8:
+__amoswap_generic((volatile uint64_t *)ptr, new, ret, ".d.aqrl");
+ break;xen/arch/riscv/include/asm/cmpxchg.h new file +3
-0+#endif + default: + STATIC_ASSERT_UNREACHABLE();xen/arch/riscv/include/asm/cmpxchg.h line 90
Quoted 10 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +10
-0+ } + + return ret; +} + +#define xchg (ptr, x) \ +({ \ + __typeof__(*(ptr)) n_ = (x); \ + (__typeof__(*(ptr))) \ + __xchg((ptr), (unsigned long)(n_), sizeof(*(ptr))); \Nit: While excess parentheses "only" harm readability, they would nevertheless better be omitted (here: the first argument passed).
xen/arch/riscv/include/asm/cmpxchg.h line 95
Quoted 5 lines
xen/arch/riscv/include/asm/cmpxchg.h new file +5
-0+}) + +#define __generic_cmpxchg (ptr, old, new, ret, lr_sfx, sc_sfx) \ + ({ \ + register unsigned int rc; \Nit: We don't normally use "register", unless accompanied by
asm()tying
a variable to a specific one.
Quoted 5 lines
Quoted 2 lines
+ __typeof__(*(ptr))
old__= (__typeof__(*(ptr)))(old); \
+ __typeof__(*(ptr))new__= (__typeof__(*(ptr)))(new); \The casts aren't very nice to have here; I take they're needed for
cmpxchg_ptr()to compile?
Not really, I have not faced an compilation issue.
The reason why it was added is that lr instruction places the sign- extended value in destination register, but if not to do cast value for old and new were generated without sign extension, so, for example:
u32= 0xbbbbbbbb;
cmpxchg(&u32, 0xbbbbbbbb, 0xCCCCCCCC), u32);
Will fail because after:
"0: lr" lr_sfx " %0, %2\n"
in %0 we will have 0xFFFFFFFFBBBBBBBB, but in %3 we will have
0xBBBBBBBB, so
bne %0, %z3, 1f\n"%0 and %3 are always inequal in case when the most significat bit of value read from %2 has 1.
But now I realized that it would be better just to use a mask and not be dependent from compiler code generation, so it would be better to in the following way ( of course, the macros should be updated accordingly to remarks you give me ):
#define __generic_cmpxchg(ptr, old, new, ret, lr_sfx, sc_sfx) \
({ \
register unsigned int rc; \
unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE)
- 1, 0); \
asm volatile( \
"0: lr" lr_sfx " %0, %2\n" \
" and %0, %0, %z[mask]\n" \
" bne %0, %z3, 1f\n" \
" sc" sc_sfx " %1, %z4, %2\n" \
" bnez %1, 0b\n" \
"1:\n" \
: "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
: "rJ" (old), "rJ" (new), [mask] "rJ" (mask) \
: "memory"); \
})Quoted 14 lines
Quoted 2 lines
+ asm volatile( \
Nit: Missing blank once again. Would be really nice if you could go through and sort this uniformly for the series.
Quoted 7 lines
+ "0: lr"
lr_sfx" %0, %2\n" \
+ " bne %0, %z3, 1f\n" \
+ " sc"sc_sfx" %1, %z4, %2\n" \
+ " bnez %1, 0b\n" \
+ "1:\n" \
+ : "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
+ : "rJ" (old__), "rJ" (new__) \
Quoted 1 line
Please could I talk you into using named operands here, too?
Sure, I will add them.
~ Oleksii
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
Quoted 52 lines · Oleksii
Quoted 36 lines · Jan Beulich
Quoted 9 lines · Oleksii Kurochko
The header was taken from Linux kernl 6.4.0-rc1.
Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use themWith this, ...
Quoted 23 lines
* drop barries and use instruction suffixices instead ( .aq, .rl, .aqrl )
Implementation of 4- and 8-byte cases were updated according to the spec:
```
....
Linux Construct RVWMO AMO Mappingatomic <op> relaxed amo<op>.{w|d} atomic <op> acquire amo<op>.{w|d}.aq atomic <op> release amo<op>.{w|d}.rlatomic <op> amo<op>.{w|d}.aqrl
Linux Construct RVWMO LR/SC Mapping
atomic <op> relaxed loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop atomic <op> acquire loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez loop OR
fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗ ; bnez loop
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loopQuoted 8 lines
Quoted 4 lines
Table A.5: Mappings from Linux memory primitives to RISC-V primitives
```
... I consider quoting this table in full, without any further remarks, as
confusing: Three of the lines each are inapplicable now, aiui.I'll shorten the table then.
Quoted 5 lines
Further what are the two * telling us? Quite likely they aren't there just
accidentally.
Quoted 6 lines
Quoted 2 lines
Finally, why sc.{w|d}.aqrl when in principle one would expect just sc.{w|d}.rl?
Because according to the last line of table A.5:
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl;Here it is used sc.{w|d}.aqrl form, so I decided to stick to the what is mentioned in the table.
I understand it's mentioned that way in the table. But it being that way is not explained anywhere. Hence my "Why?"
Quoted 16 lines
Quoted 5 lines
Quoted 2 lines
+ __typeof__(*(ptr))
old__= (__typeof__(*(ptr)))(old); \
+ __typeof__(*(ptr))new__= (__typeof__(*(ptr)))(new); \The casts aren't very nice to have here; I take they're needed for
cmpxchg_ptr()to compile?Not really, I have not faced an compilation issue.
The reason why it was added is that lr instruction places the sign- extended value in destination register, but if not to do cast value for old and new were generated without sign extension, so, for example:u32= 0xbbbbbbbb; cmpxchg(&u32, 0xbbbbbbbb, 0xCCCCCCCC), u32); Will fail because after: "0: lr" lr_sfx " %0, %2\n" in %0 we will have 0xFFFFFFFFBBBBBBBB, but in %3 we will have 0xBBBBBBBB, so bne %0, %z3, 1f\n"
Quoted 2 lines
%0 and %3 are always inequal in case when the most significat bit of value read from %2 has 1.
I'm afraid I don't follow: It's the type conversion you're after, but that would happen also with the casts omitted.
Quoted 4 lines
But now I realized that it would be better just to use a mask and not be dependent from compiler code generation, so it would be better to in the following way ( of course, the macros should be updated accordingly to remarks you give me ):
Quoted 16 lines
#define __generic_cmpxchg(ptr, old, new, ret, lr_sfx, sc_sfx) \
({ \
register unsigned int rc; \
unsigned long mask = GENMASK(((sizeof(*(ptr))) * BITS_PER_BYTE)
- 1, 0); \
asm volatile( \
"0: lr" lr_sfx " %0, %2\n" \
" and %0, %0, %z[mask]\n" \
" bne %0, %z3, 1f\n" \
" sc" sc_sfx " %1, %z4, %2\n" \
" bnez %1, 0b\n" \
"1:\n" \
: "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
: "rJ" (old), "rJ" (new), [mask] "rJ" (mask) \
: "memory"); \
})It'll be up to you whether to switch to such an alternative.
Jan
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
Quoted 96 lines · Jan Beulich
Quoted 60 lines · Oleksii
Quoted 43 lines · Jan Beulich
Quoted 11 lines · Oleksii Kurochko
The header was taken from Linux kernl 6.4.0-rc1.
Addionally, were updated:
* add emulation of {cmp}xchg for 1/2 byte types using 32-bit atomic
access.
* replace tabs with spaces
* replace __* variale with *__
* introduce generic version of xchg_* and cmpxchg_*.
* drop {cmp}xchg{release,relaxed,acquire} as Xen doesn't use themWith this, ...
Quoted 28 lines
* drop barries and use instruction suffixices instead ( .aq, .rl,
.aqrl )Implementation of 4- and 8-byte cases were updated according to the
spec:
```
....
Linux Construct RVWMO AMO Mappingatomic <op> relaxed amo<op>.{w|d} atomic <op> acquire amo<op>.{w|d}.aq atomic <op> release amo<op>.{w|d}.rlatomic <op> amo<op>.{w|d}.aqrl
Linux Construct RVWMO LR/SC Mapping
atomic <op> relaxed loop: lr.{w|d}; <op>; sc.{w|d}; bnez loop
atomic <op> acquire loop: lr.{w|d}.aq; <op>; sc.{w|d}; bnez loop
atomic <op> release loop: lr.{w|d}; <op>; sc.{w|d}.aqrl∗ ; bnez
loop OR
fence.tso; loop: lr.{w|d}; <op>; sc.{w|d}∗
; bnez loop
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; bnez loopQuoted 8 lines
Quoted 4 lines
Table A.5: Mappings from Linux memory primitives to RISC-V primitives
```
... I consider quoting this table in full, without any further remarks, as
confusing: Three of the lines each are inapplicable now, aiui.I'll shorten the table then.
Quoted 6 lines
Further what are the two * telling us? Quite likely they aren't there
just
accidentally.Quoted 8 lines
Quoted 3 lines
Finally, why sc.{w|d}.aqrl when in principle one would expect just
sc.{w|d}.rl?Because according to the last line of table A.5:
atomic <op> loop: lr.{w|d}.aq; <op>; sc.{w|d}.aqrl; Here it is used sc.{w|d}.aqrl form, so I decided to stick to the what
is mentioned in the table.I understand it's mentioned that way in the table. But it being that way
is not explained anywhere. Hence my "Why?"Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
+ __typeof__(*(ptr))
old__= (__typeof__(*(ptr)))(old); \
+ __typeof__(*(ptr))new__= (__typeof__(*(ptr)))(new); \The casts aren't very nice to have here; I take they're needed for
cmpxchg_ptr()to compile?Quoted 15 lines
Not really, I have not faced an compilation issue.
The reason why it was added is that lr instruction places the sign- extended value in destination register, but if not to do cast value for
old and new were generated without sign extension, so, for example:
u32=0xbbbbbbbb;
cmpxchg(&u32, 0xbbbbbbbb, 0xCCCCCCCC), u32);
Will fail because after:
"0: lr"lr_sfx" %0, %2\n"
in %0 we will have0xFFFFFFFFBBBBBBBB, but in %3 we will have0xBBBBBBBB, so
bne %0, %z3, 1f\n"
%0 and %3 are always inequal in case when the most significat bit of
value read from %2 has 1.
Quoted 2 lines
I'm afraid I don't follow: It's the type conversion you're after, but that would happen also with the casts omitted.
Yes, agree it would happen also with the casts omitted, and it was pure luck that the compiler that with casts the compiler used an immediate with the most significant bit = 1:
ffffffffc00397f0: bbbbc7b7 lui a5,0xbbbbc ffffffffc00397f4: bbb78793 add a5,a5,-1093 # ffffffffbbbbbbbb <start-0x4444445> ffffffffc00397f8: fef42623 sw a5,-20(s0) ffffffffc00397fc: ccccd737 lui a4,0xccccd ffffffffc0039800: ccc7071b addw a4,a4,-820 # ffffffffcccccccc <__bss_end+0xcc7ff44> ffffffffc0039804: 56fd li a3,-1 ffffffffc0039806: 9281 srl a3,a3,0x20 ffffffffc0039808: fec40513 add a0,s0,-20 ffffffffc003980c: 140525af lr.w.aq a1,(a0) ffffffffc0039810: 00f59563 bne
a1,a5,ffffffffc003981a <start_xen+0x4e>ffffffffc0039814: 1ee5262f sc.w.aqrl
a2,a4,(a0)
I will update the code with the mask mentioned below to be sure that a5 has always correct value.
~ Oleksii
Quoted 30 lines
Quoted 18 lines
But now I realized that it would be better just to use a mask and not
be dependent from compiler code generation, so it would be better to in
the following way ( of course, the macros should be updated accordingly
to remarks you give me ):
#define __generic_cmpxchg(ptr, old, new, ret,lr_sfx,sc_sfx) \
({ \
register unsigned int rc; \
unsigned long mask = GENMASK(((sizeof(*(ptr))) *BITS_PER_BYTE)
- 1, 0); \
asm volatile( \
"0: lr"lr_sfx" %0, %2\n" \
" and %0, %0, %z[mask]\n" \Quoted 8 lines
" bne %0, %z3, 1f\n" \
" sc"sc_sfx" %1, %z4, %2\n" \
" bnez %1, 0b\n" \
"1:\n" \
: "=&r" (ret), "=&r" (rc), "+A" (*ptr) \
: "rJ" (old), "rJ" (new), [mask] "rJ" (mask) \
: "memory"); \
})It'll be up to you whether to switch to such an alternative.
Jan
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
In reply to Jan BeulichQuoted 13 lines · Jan Beulich
Quoted 10 lines
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ */xen/arch/riscv/include/asm/cmpxchg.h new file +3
-0+#define emulate_xchg_1_2 (ptr, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr &~(0x4 - sizeof(*(ptr)))); \
+ unsigned intnew_val_pos= ((unsigned long)(ptr) & (0x4- sizeof(*(ptr)))) *BITS_PER_BYTE; \You parenthesize ptr here correctly, but not in the line above.
Quoted 7 lines
Instead of "
_pos" in the name, maybe better "_bit"?Finally, here and elsewhere, please limit line length to 80 chars. (Omitting
the 0x here would help a little, but not quite enough. Question is whether
these wouldn't better besizeof(*aligned_ptr)anyway.)
I'm unsure if using sizeof(*aligned_ptr) is correct in this context. The intention was to determine the position for the value we're attempting to exchange.
Let's consider a scenario where we have 0xAABBCCDD at address 0x6. Even though this would result in misaligned access, I believe new_val_pos should still be calculated accurately. Let's say we want to exchange two bytes (AABB) with FFFF.
With the current implementation, we would calculate:0x6 & 2 = 2 * 8 = 16, which is the value on which the new value should be shifted.
However, if this value is then ANDed with sizeof(*aligned_ptr):0x6 & 4 = 6 * 8 = 48, which is not the expected value.
Am I overlooking something?
~ Oleksii
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
Quoted 31 lines · Oleksii
Quoted 13 lines · Jan Beulich
Quoted 10 lines
+ * If resulting 4-byte access is still misalgined, it will fault just as
+ * non-emulated 4-byte access would.
+ */xen/arch/riscv/include/asm/cmpxchg.h new file +3
-0+#define emulate_xchg_1_2 (ptr, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr &~(0x4 - sizeof(*(ptr)))); \
+ unsigned intnew_val_pos= ((unsigned long)(ptr) & (0x4- sizeof(*(ptr)))) *BITS_PER_BYTE; \You parenthesize ptr here correctly, but not in the line above.
Quoted 7 lines
Instead of "
_pos" in the name, maybe better "_bit"?Finally, here and elsewhere, please limit line length to 80 chars. (Omitting
the 0x here would help a little, but not quite enough. Question is whether
these wouldn't better besizeof(*aligned_ptr)anyway.)I'm unsure if using
sizeof(*aligned_ptr)is correct in this context. The intention was to determine the position for the value we're attempting to exchange.Let's consider a scenario where we have
0xAABBCCDDat address0x6. Even though this would result in misaligned access, I believenew_val_posshould still be calculated accurately. Let's say we want to exchange two bytes (AABB) with FFFF.
Quoted 8 lines
With the current implementation, we would calculate:
0x6& 2 = 2 * 8 = 16, which is the value on which the new value should be shifted.However, if this value is then ANDed with
sizeof(*aligned_ptr):0x6& 4 = 6 * 8 = 48, which is not the expected value.Am I overlooking something?
I'm afraid I can only reply with a counter question (feels like there is some misunderstanding): Do you agree that 0x4 == 4 == sizeof(*aligned_ptr)? It's the 0x4 that the latter part of my earlier reply was about. I'm pretty sure you have, over the various reviews I've done, noticed my dislike for the use of literal numbers, when those aren't truly "magic".
Jan
Re: [PATCH v6 08/20] xen/riscv: introduce cmpxchg.h
Quoted 49 lines · Jan Beulich
Quoted 43 lines · Oleksii
Quoted 16 lines · Jan Beulich
Quoted 11 lines
+ * If resulting 4-byte access is still misalgined, it will fault
just as
+ * non-emulated 4-byte access would.
+ */xen/arch/riscv/include/asm/cmpxchg.h new file +3
-0+#define emulate_xchg_1_2 (ptr, new, lr_sfx, sc_sfx) \ +({ \ + uint32_t *aligned_ptr = (uint32_t *)((unsigned long)ptr &~(0x4 - sizeof(*(ptr)))); \
+ unsigned intnew_val_pos= ((unsigned long)(ptr) & (0x4- sizeof(*(ptr)))) *BITS_PER_BYTE; \You parenthesize ptr here correctly, but not in the line above.
Instead of "
_pos" in the name, maybe better "_bit"?Quoted 7 lines
Finally, here and elsewhere, please limit line length to 80 chars.
(Omitting
the 0x here would help a little, but not quite enough. Question is
whether
these wouldn't better besizeof(*aligned_ptr)anyway.)I'm unsure if using
sizeof(*aligned_ptr)is correct in this context.
The intention was to determine the position for the value we're attempting to exchange.Let's consider a scenario where we have
0xAABBCCDDat address0x6. Even
though this would result in misaligned access, I believenew_val_pos
should still be calculated accurately. Let's say we want to exchange
two bytes (AABB) with FFFF.With the current implementation, we would calculate:
0x6& 2 = 2 * 8 = 16, which is the value on which the new value should
be shifted.Quoted 4 lines
However, if this value is then ANDed with
sizeof(*aligned_ptr):0x6& 4 = 6 * 8 = 48, which is not the expected value.Am I overlooking something?
Quoted 9 lines
I'm afraid I can only reply with a counter question (feels like there is
some misunderstanding): Do you agree that0x4== 4 ==sizeof(*aligned_ptr)?
It's the0x4that the latter part of my earlier reply was about. I'm pretty
sure you have, over the various reviews I've done, noticed my dislike for
the use of literal numbers, when those aren't truly "magic".
Yes, it was misunderstading. Thanks for clarifying.
~ Oleksii
PATCH v6 9/20 xen/riscv: introduce io.h
In reply to Oleksii KurochkoThe header taken form Linux 6.4.0-rc1 and is based on arch/riscv/include/asm/mmio.h with the following changes:
- drop forcing of endianess for read*(), write*() functions as
no matter what CPU endianness, what endianness a particular device (and hence its MMIO region(s)) is using is entirely independent. Hence conversion, where necessary, needs to occur at a layer up. Another one reason to drop endianess conversion here is:
https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
One of the answers of the author of the commit:
And we don't know if Linux will be around if that ever changes. The point is:
a) the current RISC-V spec is LE only
b) the current linux port is LE only except for this little bit
There is no point in leaving just this bitrotting code around. It just confuses developers, (very very slightly) slows down compiles and will bitrot. It also won't be any significant help to a future developer down the road doing a hypothetical BE RISC-V Linux port.
- drop unused argument of __io_ar() macros.
- drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}" as they are unnecessary.
- Adopt the Xen code style for this header, considering that significant changes are not anticipated in the future.
In the event of any issues, adapting them to Xen style should be easily manageable.
- drop unnecessary __r variables in macros read*_cpu()
- update inline assembler constraints for addr argument for __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that *addr will be accessed.
- add stubs for __raw_readq() and __raw_writeq() for RISCV_32
Addionally, to the header was added definions of ioremap_*().
---
Changes in V6:
- drop unnecessary spaces and fix typos in the file comment.
- s/CONFIG_64BIT/CONFIG_RISCV_32 as .d suffix for instruction doesn't exist for RV32.
- add stubs for __raw_readq() and __raw_writeq() for RISCV_32
- update inline assembler constraints for addr argument for __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell compiler that *addr will be accessed.
- s/u8/uint8_t
- update the commit message
---
Changes in V5:
- Xen code style related fixes
- drop #define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}
- drop cpu_to_le16()
- remove unuused argument in _io_ar()
- update the commit message
- drop unnessary __r variables in macros read*_cpu()
- update the comments at the top of the header.
---
Changes in V4:
- delete inner parentheses in macros.
- s/u<N>/uint<N>.
---
Changes in V3:
- re-sync with linux kernel
- update the commit message
---
Changes in V2:
- Nothing changed. Only rebase.
--- xen/arch/riscv/include/asm/io.h | 167 ++++++++++++++++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 xen/arch/riscv/include/asm/io.h
xen/arch/riscv/include/asm/io.h new file +167 -0
@@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The header taken form Linux 6.4.0-rc1 and is based on +* arch/riscv/include/asm/mmio.h with the following changes: +* - drop forcing of endianess for read*(), write*() functions as +* no matter what CPU endianness, what endianness a particular device +* (and hence its MMIO region(s)) is using is entirely independent. +* Hence conversion, where necessary, needs to occur at a layer up. +* Another one reason to drop endianess conversion is: +* https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/ +* One of the answers of the author of the commit: +* And we don't know if Linux will be around if that ever changes. +* The point is: +* a) the current RISC-V spec is LE only +* b) the current linux port is LE only except for this little bit +* There is no point in leaving just this bitrotting code around. It +* just confuses developers, (very very slightly) slows down compiles +* and will bitrot. It also won't be any significant help to a future +* developer down the road doing a hypothetical BE RISC-V Linux port. +* - drop unused argument of __io_ar() macros. +* - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}" +* as they are unnecessary. +* - Adopt the Xen code style for this header, considering that significant +* changes are not anticipated in the future. +* In the event of any issues, adapting them to Xen style should be easily +* manageable. +* - drop unnecessary __r variables in macros read*_cpu() +* - update inline assembler constraints for addr argument for +* __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that +* *addr will be accessed. +* +* Copyright (C) 1996-2000 Russell King +* Copyright (C) 2012 ARM Ltd. +* Copyright (C) 2014 Regents of the University of California +* Copyright (C) 2024 Vates +*/ + +#ifndef _ASM_RISCV_IO_H +#define _ASM_RISCV_IO_H + +#include <asm/byteorder.h> + +/* +* The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't +* change the properties of memory regions. This should be fixed by the +* upcoming platform spec. +*/ +#define ioremap_nocache (addr, size) ioremap(addr, size) +#define ioremap_wc (addr, size) ioremap(addr, size) +#define ioremap_wt (addr, size) ioremap(addr, size) + +/* Generic IO read/write. These perform native-endian accesses. */ +static inline void __raw_writeb (uint8_t val,volatile void __iomem *addr) +{ + asmvolatile ("sb %1, %0" :"=m" (*(volatile uint8_t __force *)addr) :"r" (val) ); +} + +static inline void __raw_writew (uint16_t val,volatile void __iomem *addr) +{ + asmvolatile ("sh %1, %0" :"=m" (*(volatile uint16_t __force *)addr) :"r" (val) ); +} + +static inline void __raw_writel (uint32_t val,volatile void __iomem *addr) +{ + asmvolatile ("sw %1, %0" :"=m" (*(volatile uint32_t __force *)addr) :"r" (val) ); +} + +#ifdef CONFIG_RISCV_32 +static inline void __raw_writeq (uint64_t val,volatile void __iomem *addr) +{ +BUILD_BUG_ON ("unimplemented\n" ); +} +#else +static inline void __raw_writeq (uint64_t val,volatile void __iomem *addr) +{ + asmvolatile ("sd %1, %0" :"=m" (*(volatile uint64_t __force *)addr) :"r" (val) ); +} +#endif + +static inline uint8_t __raw_readb (const volatile void __iomem *addr) +{ +uint8_t val; + + asmvolatile ("lb %0, %1" :"=r" (val) :"m" (*(volatile uint8_t __force *)addr) ); +return val; +} + +static inline uint16_t __raw_readw (const volatile void __iomem *addr) +{ +uint16_t val; + + asmvolatile ("lh %0, %1" :"=r" (val) :"m" (*(volatile uint16_t __force *)addr) ); +return val; +} + +static inline uint32_t __raw_readl (const volatile void __iomem *addr) +{ +uint32_t val; + + asmvolatile ("lw %0, %1" :"=r" (val) :"m" (*(volatile uint32_t __force *)addr) ); +return val; +} + +#ifdef CONFIG_RISCV_32 +static inline uint64_t __raw_readq (const volatile void __iomem *addr) +{ +BUILD_BUG_ON ("unimplemented\n" ); +return 0 ; +} +#else +static inline uint64_t __raw_readq (const volatile void __iomem *addr) +{ +uint64_t val; + + asmvolatile ("ld %0, %1" :"=r" (val) :"m" (*(volatile uint64_t __force *)addr) ); +return val; +} +#endif + +/* +* Unordered I/O memory access primitives. These are even more relaxed than +* the relaxed versions, as they don't even order accesses between successive +* operations to the I/O regions. +*/ +#define readb_cpu (c) __raw_readb(c) +#define readw_cpu (c) __raw_readw(c) +#define readl_cpu (c) __raw_readl(c) + +#define writeb_cpu (v, c) __raw_writeb(v, c) +#define writew_cpu (v, c) __raw_writew(v, c) +#define writel_cpu (v, c) __raw_writel(v, c) + +#define readq_cpu (c) __raw_readq(c) +#define writeq_cpu (v, c) __raw_writeq(v, c) + +/* +* I/O memory access primitives. Reads are ordered relative to any +* following Normal memory access. Writes are ordered relative to any prior +* Normal memory access. The memory barriers here are necessary as RISC-V +* doesn't define any ordering between the memory space and the I/O space. +*/ +#define __io_br () do { } while (0) +#define __io_ar () asm volatile ( "fence i,r" : : : "memory" ); +#define __io_bw () asm volatile ( "fence w,o" : : : "memory" ); +#define __io_aw () do { } while (0) + +#define readb (c) ({ uint8_t v; __io_br(); v = readb_cpu(c); __io_ar(); v; }) +#define readw (c) ({ uint16_t v; __io_br(); v = readw_cpu(c); __io_ar(); v; }) +#define readl (c) ({ uint32_t v; __io_br(); v = readl_cpu(c); __io_ar(); v; }) + +#define writeb (v, c) ({ __io_bw(); writeb_cpu(v, c); __io_aw(); }) +#define writew (v, c) ({ __io_bw(); writew_cpu(v, c); __io_aw(); }) +#define writel (v, c) ({ __io_bw(); writel_cpu(v, c); __io_aw(); }) + +#define readq (c) ({ uint64_t v; __io_br(); v = readq_cpu(c); __io_ar(); v; }) +#define writeq (v, c) ({ __io_bw(); writeq_cpu(v, c); __io_aw(); }) + +#endif /* _ASM_RISCV_IO_H */ + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* indent-tabs-mode: nil +* End: +*/
Signature · 1 line
2.43.0
Re: [PATCH v6 09/20] xen/riscv: introduce io.h
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/io.h new file +11
-0@@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The header taken form Linux 6.4.0-rc1 and is based on +* arch/riscv/include/asm/mmio.h with the following changes: +* - drop forcing of endianess for read*(), write*() functions as +* no matter what CPU endianness, what endianness a particular device +* (and hence its MMIO region(s)) is using is entirely independent. +* Hence conversion, where necessary, needs to occur at a layer up. +* Another one reason to drop endianess conversion is: +* https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/ +* One of the answers of the author of the commit:
xen/arch/riscv/include/asm/io.h line 21
Quoted 10 lines
xen/arch/riscv/include/asm/io.h new file +10
-0+* And we don't know if Linux will be around if that ever changes. +* The point is: +* a) the current RISC-V spec is LE only +* b) the current linux port is LE only except for this little bit +* There is no point in leaving just this bitrotting code around. It +* just confuses developers, (very very slightly) slows down compiles +* and will bitrot. It also won't be any significant help to a future +* developer down the road doing a hypothetical BE RISC-V Linux port. +* - drop unused argument of __io_ar() macros. +* - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"
In the commit message I'm not worried as much, but at least here the odd mention of d as suffixes would better be purged.
Quoted 24 lines
xen/arch/riscv/include/asm/io.h new file +24
-0+* as they are unnecessary. +* - Adopt the Xen code style for this header, considering that significant +* changes are not anticipated in the future. +* In the event of any issues, adapting them to Xen style should be easily +* manageable. +* - drop unnecessary __r variables in macros read*_cpu() +* - update inline assembler constraints for addr argument for +* __raw_read{b,w,l,q} and __raw_write{b,w,l,q} to tell a compiler that +* *addr will be accessed. +* +* Copyright (C) 1996-2000 Russell King +* Copyright (C) 2012 ARM Ltd. +* Copyright (C) 2014 Regents of the University of California +* Copyright (C) 2024 Vates +*/ + +#ifndef _ASM_RISCV_IO_H +#define _ASM_RISCV_IO_H + +#include <asm/byteorder.h> + +/* +* The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't +* change the properties of memory regions. This should be fixed by the
xen/arch/riscv/include/asm/io.h line 55
Quoted 10 lines
xen/arch/riscv/include/asm/io.h new file +10
-0+* upcoming platform spec. +*/ +#define ioremap_nocache (addr, size) ioremap(addr, size) +#define ioremap_wc (addr, size) ioremap(addr, size) +#define ioremap_wt (addr, size) ioremap(addr, size) + +/* Generic IO read/write. These perform native-endian accesses. */ +static inline void __raw_writeb (uint8_t val,volatile void __iomem *addr) +{ + asmvolatile ("sb %1, %0" :"=m" (*(volatile uint8_t __force *)addr) :"r" (val) );
Please respect line length restrictions.
Quoted 13 lines
xen/arch/riscv/include/asm/io.h new file +13
-0+} + +static inline void __raw_writew (uint16_t val,volatile void __iomem *addr) +{ + asmvolatile ("sh %1, %0" :"=m" (*(volatile uint16_t __force *)addr) :"r" (val) ); +} + +static inline void __raw_writel (uint32_t val,volatile void __iomem *addr) +{ + asmvolatile ("sw %1, %0" :"=m" (*(volatile uint32_t __force *)addr) :"r" (val) ); +} + +#ifdef CONFIG_RISCV_32
xen/arch/riscv/include/asm/io.h line 78
Quoted 10 lines
xen/arch/riscv/include/asm/io.h new file +10
-0+static inline void __raw_writeq (uint64_t val,volatile void __iomem *addr) +{ +BUILD_BUG_ON ("unimplemented\n" ); +} +#else +static inline void __raw_writeq (uint64_t val,volatile void __iomem *addr) +{ + asmvolatile ("sd %1, %0" :"=m" (*(volatile uint64_t __force *)addr) :"r" (val) ); +} +#endif
Please avoid code duplication if it easily can be avoided:
static inline void __raw_writeq(uint64_t val, volatile void __iomem *addr) {
#ifdef CONFIG_RISCV_32
BUILD_BUG_ON("unimplemented");
#else
asm volatile ( "sd %1, %0" : "=m" (*(volatile uint64_t __force *)addr) : "r" (val) );#endif
}
(Note also the dropped \n.)
xen/arch/riscv/include/asm/io.h line 84
Quoted 5 lines
xen/arch/riscv/include/asm/io.h new file +5
-0+static inline uint8_t __raw_readb (const volatile void __iomem *addr) +{ +uint8_t val; + + asmvolatile ("lb %0, %1" :"=r" (val) :"m" (*(volatile uint8_t __force *)addr) );
Please don't cast away const.
Quoted 40 lines
xen/arch/riscv/include/asm/io.h new file +40
-0+return val; +} + +static inline uint16_t __raw_readw (const volatile void __iomem *addr) +{ +uint16_t val; + + asmvolatile ("lh %0, %1" :"=r" (val) :"m" (*(volatile uint16_t __force *)addr) ); +return val; +} + +static inline uint32_t __raw_readl (const volatile void __iomem *addr) +{ +uint32_t val; + + asmvolatile ("lw %0, %1" :"=r" (val) :"m" (*(volatile uint32_t __force *)addr) ); +return val; +} + +#ifdef CONFIG_RISCV_32 +static inline uint64_t __raw_readq (const volatile void __iomem *addr) +{ +BUILD_BUG_ON ("unimplemented\n" ); +return 0 ; +} +#else +static inline uint64_t __raw_readq (const volatile void __iomem *addr) +{ +uint64_t val; + + asmvolatile ("ld %0, %1" :"=r" (val) :"m" (*(volatile uint64_t __force *)addr) ); +return val; +} +#endif + +/* +* Unordered I/O memory access primitives. These are even more relaxed than +* the relaxed versions, as they don't even order accesses between successive +* operations to the I/O regions. +*/
xen/arch/riscv/include/asm/io.h line 134
Quoted 10 lines
xen/arch/riscv/include/asm/io.h new file +10
-0+#define readb_cpu (c) __raw_readb(c) +#define readw_cpu (c) __raw_readw(c) +#define readl_cpu (c) __raw_readl(c) + +#define writeb_cpu (v, c) __raw_writeb(v, c) +#define writew_cpu (v, c) __raw_writew(v, c) +#define writel_cpu (v, c) __raw_writel(v, c) + +#define readq_cpu (c) __raw_readq(c) +#define writeq_cpu (v, c) __raw_writeq(v, c)
Since (now?) outside of any #ifdef (which is okay with me), why not each grouped with their respective 3 siblings?
xen/arch/riscv/include/asm/io.h line 147
Quoted 12 lines
xen/arch/riscv/include/asm/io.h new file +12
-0+/* +* I/O memory access primitives. Reads are ordered relative to any +* following Normal memory access. Writes are ordered relative to any prior +* Normal memory access. The memory barriers here are necessary as RISC-V +* doesn't define any ordering between the memory space and the I/O space. +*/ +#define __io_br () do { } while (0) +#define __io_ar () asm volatile ( "fence i,r" : : : "memory" ); +#define __io_bw () asm volatile ( "fence w,o" : : : "memory" ); +#define __io_aw () do { } while (0) + +#define readb (c) ({ uint8_t v; __io_br(); v = readb_cpu(c); __io_ar(); v; })
I'm wary of "v" as a macro local: What if at a use site there's
x = readb(v);
? That'll fail to compile, provided at the use site v is actually a pointer, but the reason for the error won't necessarily be obvious. (Hence why in the past people did prefix underscores on such local variable names, while now we're advocating for suffixes.)
xen/arch/riscv/include/asm/io.h line 156
Quoted 9 lines
xen/arch/riscv/include/asm/io.h new file +9
-0+#define readw (c) ({ uint16_t v; __io_br(); v = readw_cpu(c); __io_ar(); v; }) +#define readl (c) ({ uint32_t v; __io_br(); v = readl_cpu(c); __io_ar(); v; }) + +#define writeb (v, c) ({ __io_bw(); writeb_cpu(v, c); __io_aw(); }) +#define writew (v, c) ({ __io_bw(); writew_cpu(v, c); __io_aw(); }) +#define writel (v, c) ({ __io_bw(); writel_cpu(v, c); __io_aw(); }) + +#define readq (c) ({ uint64_t v; __io_br(); v = readq_cpu(c); __io_ar(); v; }) +#define writeq (v, c) ({ __io_bw(); writeq_cpu(v, c); __io_aw(); })
Same here then.
Jan
Re: [PATCH v6 09/20] xen/riscv: introduce io.h
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/io.h new file +5
-0@@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The header taken form Linux 6.4.0-rc1 and is based on +* arch/riscv/include/asm/mmio.h with the following changes: +* - drop forcing of endianess for read*(), write*() functions as
+ * no matter what CPU endianness, what endianness a particular device
+ * (and hence its MMIO region(s)) is using is entirely independent.
+ * Hence conversion, where necessary, needs to occur at a layer up.
+ * Another one reason to drop endianess conversion is:
+ *
https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
+ * One of the answers of the author of the commit:
+ * And we don't know if Linux will be around if that ever changes.
+ * The point is:
+ * a) the current RISC-V spec is LE only
+ * b) the current linux port is LE only except for this little bit
+ * There is no point in leaving just this bitrotting code around. It
+ * just confuses developers, (very very slightly) slows down compiles
+ * and will bitrot. It also won't be any significant help to a future
+ * developer down the road doing a hypothetical BE RISC-V
Quoted 8 lines
Quoted 4 lines
Linux port.
+ * - drop unused argument of__io_ar()macros.
+ * - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"In the commit message I'm not worried as much, but at least here the odd mention
of d as suffixes would better be purged.
Probably, I use incorrect words, but what I meant that it was dropped:
#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}
before declaration/definition of inline functions ( __raw_{read,write}{b,w,l,d,q} ).
~ Oleksii
Re: [PATCH v6 09/20] xen/riscv: introduce io.h
Quoted 42 lines · Oleksii
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/io.h new file +5
-0@@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The header taken form Linux 6.4.0-rc1 and is based on +* arch/riscv/include/asm/mmio.h with the following changes: +* - drop forcing of endianess for read*(), write*() functions as
+ * no matter what CPU endianness, what endianness a particular device
+ * (and hence its MMIO region(s)) is using is entirely independent.
+ * Hence conversion, where necessary, needs to occur at a layer up.
+ * Another one reason to drop endianess conversion is:
+ *
https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
+ * One of the answers of the author of the commit:
+ * And we don't know if Linux will be around if that ever changes.
+ * The point is:
+ * a) the current RISC-V spec is LE only
+ * b) the current linux port is LE only except for this little bit
+ * There is no point in leaving just this bitrotting code around. It
+ * just confuses developers, (very very slightly) slows down compiles
+ * and will bitrot. It also won't be any significant help to a future
+ * developer down the road doing a hypothetical BE RISC-VQuoted 8 lines
Quoted 4 lines
Linux port.
+ * - drop unused argument of__io_ar()macros.
+ * - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"In the commit message I'm not worried as much, but at least here the odd mention
of d as suffixes would better be purged.
Quoted 4 lines
Probably, I use incorrect words, but what I meant that it was dropped:
#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}before declaration/definition of inline functions ( __raw_{read,write}{b,w,l,d,q} ).
But where did you find a raw_readd() or raw_writed() (with no matter how many leading underscores)?
Jan
Re: [PATCH v6 09/20] xen/riscv: introduce io.h
Quoted 58 lines · Jan Beulich
Quoted 50 lines · Oleksii
Quoted 41 lines · Jan Beulich
Quoted 40 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/io.h new file +5
-0@@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* The header taken form Linux 6.4.0-rc1 and is based on +* arch/riscv/include/asm/mmio.h with the following changes: +* - drop forcing of endianess for read*(), write*() functions
as
+ * no matter what CPU endianness, what endianness a particular
device
+ * (and hence its MMIO region(s)) is using is entirely independent.
+ * Hence conversion, where necessary, needs to occur at a layer up.
+ * Another one reason to drop endianess conversion is:
+ *
https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
+ * One of the answers of the author of the commit:
+ * And we don't know if Linux will be around if that ever
changes.
+ * The point is:
+ * a) the current RISC-V spec is LE only
+ * b) the current linux port is LE only except for this little bit
+ * There is no point in leaving just this bitrotting code
around. It
+ * just confuses developers, (very very slightly) slows down
compiles
+ * and will bitrot. It also won't be any significant help to
a future
+ * developer down the road doing a hypothetical BE RISC- V
Linux port.Quoted 8 lines
Quoted 3 lines
+ * - drop unused argument of
__io_ar()macros.
+ * - drop "#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}"In the commit message I'm not worried as much, but at least here the
odd mention
of d as suffixes would better be purged.Quoted 6 lines
Probably, I use incorrect words, but what I meant that it was dropped:
#define _raw_{read,write}{b,w,l,d,q} _raw_{read,write}{b,w,l,d,q}
before declaration/definition of inline functions ( __raw_{read,write}{b,w,l,d,q} ).
Quoted 3 lines
But where did you find a
raw_readd()orraw_writed()(with no matter how
many leading underscores)?
Oh, {d} options didn't exist. Missed that, wrote it automatically. Thanks I'll update the commit message and the header comment.
~ Oleksii
PATCH v6 10/20 xen/riscv: introduce atomic.h
In reply to Oleksii KurochkoInitially the patch was introduced by Bobby, who takes the header from Linux kernel.
The following changes were done on top of Linux kernel header:
- atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated
to use__*xchg_generic()
- drop casts in write_atomic() as they are unnecessary
- drop introduction of WRITE_ONCE() and READ_ONCE().
Xen provides ACCESS_ONCE()
- remove zero-length array access in read_atomic()
- drop defines similar to pattern
- #define atomic_add_return_relaxed atomic_add_return_relaxed
- move not RISC-V specific functions to asm-generic/atomics-ops.h
- drop atomic##prefix##_{cmp}xchg_{release, aquire, release}() as they are not used in Xen.
- update the defintion of atomic##prefix##_{cmp}xchg according to {cmp}xchg() implementation in Xen.
---
Changes in V6:
- drop atomic##prefix##_{cmp}xchg_{release, aquire, relaxed} as they aren't used by Xen
- code style fixes.
- %s/__asm__ __volatile__/asm volatile
- add explanational comments.
- move inclusion of "#include <asm-generic/atomic-ops.h>" further down in atomic.h header.
---
Changes in V5:
- fence.h changes were moved to separate patch as patches related to io.h and cmpxchg.h, which are dependecies for this patch, also needed changes in fence.h
- remove accessing of zero-length array
- drops cast in write_atomic()
- drop introduction of WRITE_ONCE() and READ_ONCE().
- drop defines similar to pattern #define atomic_add_return_relaxed atomic_add_return_relaxed
- Xen code style fixes
- move not RISC-V specific functions to asm-generic/atomics-ops.h
---
Changes in V4:
- do changes related to the updates of [PATCH v3 13/34] xen/riscv: introduce cmpxchg.h
- drop casts in read_atomic_size(), write_atomic(), add_sized()
- tabs -> spaces
- drop #ifdef CONFIG_SMP ... #endif in fence.ha as it is simpler to handle NR_CPUS=1 the same as NR_CPUS>1 with accepting less than ideal performance.
---
Changes in V3:
- update the commit message
- add SPDX for fence.h
- code style fixes
- Remove /* TODO: ... */ for add_sized macros. It looks correct to me.
- re-order the patch
- merge to this patch fence.h
---
Changes in V2:
- Change an author of commit. I got this header from Bobby's old repo.
--- xen/arch/riscv/include/asm/atomic.h | 263 +++++++++++++++++++++++++++ xen/include/asm-generic/atomic-ops.h | 97 ++++++++++ 2 files changed, 360 insertions(+) create mode 100644 xen/arch/riscv/include/asm/atomic.h create mode 100644 xen/include/asm-generic/atomic-ops.h
xen/arch/riscv/include/asm/atomic.h new file +263 -0
@@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Taken and modified from Linux. +* +* The following changes were done: +* - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated +* to use__*xchg_generic() +* - drop casts in write_atomic() as they are unnecessary +* - drop introduction of WRITE_ONCE() and READ_ONCE(). +* Xen provides ACCESS_ONCE() +* - remove zero-length array access in read_atomic() +* - drop defines similar to pattern +* #define atomic_add_return_relaxed atomic_add_return_relaxed +* - move not RISC-V specific functions to asm-generic/atomics-ops.h +* +* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved. +* Copyright (C) 2012 Regents of the University of California +* Copyright (C) 2017 SiFive +* Copyright (C) 2024 Vates SAS +*/ + +#ifndef _ASM_RISCV_ATOMIC_H +#define _ASM_RISCV_ATOMIC_H + +#include <xen/atomic.h> + +#include <asm/cmpxchg.h> +#include <asm/fence.h> +#include <asm/io.h> +#include <asm/system.h> + +void __bad_atomic_size (void ); + +/* +* Legacy from Linux kernel. For some reason they wanted to have ordered +* read/write access. Thereby read* is used instead of read<X>_cpu() +*/ +static always_inline void read_atomic_size(const volatile void *p, +void *res, +unsigned int size) +{ +switch ( size ) + { +case 1 : *(uint8_t *)res =readb (p);break ; +case 2 : *(uint16_t *)res =readw (p);break ; +case 4 : *(uint32_t *)res =readl (p);break ; +case 8 : *(uint32_t *)res =readq (p);break ; +default :__bad_atomic_size ();break ; + } +} + +#define read_atomic (p) ({ \ + union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_; \ + read_atomic_size(p, x_.c, sizeof(*(p))); \ + x_.val; \ +}) + +#define write_atomic (p, x) \ +({ \ + typeof(*(p)) x__ = (x); \ + switch ( sizeof(*(p)) ) \ + { \ + case 1: writeb(x__, p); break; \ + case 2: writew(x__, p); break; \ + case 4: writel(x__, p); break; \ + case 8: writeq(x__, p); break; \ + default: __bad_atomic_size(); break; \ + } \ + x__; \ +}) + +#define add_sized (p, x) \ +({ \ + typeof(*(p)) x__ = (x); \ + switch ( sizeof(*(p)) ) \ + { \ + case 1: writeb(read_atomic(p) + x__, p); break; \ + case 2: writew(read_atomic(p) + x__, p); break; \ + case 4: writel(read_atomic(p) + x__, p); break; \ + case 8: writeq(read_atomic(p) + x__, p); break; \ + default: __bad_atomic_size(); break; \ + } \ +}) + +#define __atomic_acquire_fence () \ + asm volatile ( RISCV_ACQUIRE_BARRIER "" ::: "memory" ) + +#define __atomic_release_fence () \ + asm volatile ( RISCV_RELEASE_BARRIER "" ::: "memory" ) + +/* +* First, the atomic ops that have no ordering constraints and therefor don't +* have the AQ or RL bits set. These don't return anything, so there's only +* one version to worry about. +*/ +#define ATOMIC_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " zero, %1, %0" \ + : "+A" (v->counter) \ + : "r" (I) \ + : "memory" ); \ +} \ + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why +* last argument for ATOMIC_OP isn't used. +*/ +#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_OP (op, asm_op, I, w, int, ) + +ATOMIC_OPS(add, add, i) +ATOMIC_OPS(sub, add, -i) +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i) + +#undef ATOMIC_OP +#undef ATOMIC_OPS + +#include <asm-generic/atomic-ops.h> + +/* +* Atomic ops that have ordered, relaxed, acquire, and release variants. +* There's two flavors of these: the arithmatic ops have both fetch and return +* versions, while the logical ops only have fetch versions. +*/ +#define ATOMIC_FETCH_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +c_type atomic##prefix##_fetch_##op##_relaxed(c_type i, \ + atomic##prefix##_t *v) \ +{ \ + register c_type ret; \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " %1, %2, %0" \ + : "+A" (v->counter), "=r" (ret) \ + : "r" (I) \ + : "memory" ); \ + return ret; \ +} \ +static inline \ +c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + register c_type ret; \ + asm volatile ( \ + " amo" #asm_op "." #asm_type ".aqrl %1, %2, %0" \ + : "+A" (v->counter), "=r" (ret) \ + : "r" (I) \ + : "memory" ); \ + return ret; \ +} + +#define ATOMIC_OP_RETURN (op, asm_op, c_op,I , asm_type, c_type, prefix) \ +static inline \ +c_type atomic##prefix##_##op##_return_relaxed(c_type i, \ + atomic##prefix##_t *v) \ +{ \ + return atomic##prefix##_fetch_##op##_relaxed(i, v) c_op I; \ +} \ +static inline \ +c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v) \ +{ \ + return atomic##prefix##_fetch_##op(i, v) c_op I; \ +} + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why +* last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used. +*/ +#define ATOMIC_OPS (op, asm_op, c_op,I ) \ + ATOMIC_FETCH_OP( op, asm_op, I, w, int, ) \ + ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int, ) + +ATOMIC_OPS(add, add, +, i) +ATOMIC_OPS(sub, add, +, -i) + +#undef ATOMIC_OPS + +#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_FETCH_OP(op, asm_op, I, w, int, ) + +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i) + +#undef ATOMIC_OPS + +#undef ATOMIC_FETCH_OP +#undef ATOMIC_OP_RETURN + +/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless (atomic_t *v,int a,int u) +{ +int prev, rc; + + asmvolatile ( +"0: lr.w %[p], %[c]\n" +" beq %[p], %[u], 1f\n" +" add %[rc], %[p], %[a]\n" +" sc.w.rl %[rc], %[rc], %[c]\n" +" bnez %[rc], 0b\n" +RISCV_FULL_BARRIER +"1:\n" + : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) + : [a]"r" (a), [u]"r" (u) + :"memory" ); +return prev; +} + +/* +* atomic_{cmp,}xchg is required to have exactly the same ordering semantics as +* {cmp,}xchg and the operations that return. +*/ +#define ATOMIC_OP (c_t, prefix, size) \ +static inline \ +c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ +{ \ + return __xchg(&(v->counter), n, size); \ +} \ +static inline \ +c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \ +{ \ + return __cmpxchg(&v->counter, o, n, size); \ +} + +#define ATOMIC_OPS () \ + ATOMIC_OP(int, , 4) + +ATOMIC_OPS() + +#undef ATOMIC_OPS +#undef ATOMIC_OP + +static inline int atomic_sub_if_positive (atomic_t *v,int offset) +{ +int prev, rc; + + asmvolatile ( +"0: lr.w %[p], %[c]\n" +" sub %[rc], %[p], %[o]\n" +" bltz %[rc], 1f\n" +" sc.w.rl %[rc], %[rc], %[c]\n" +" bnez %[rc], 0b\n" +" fence rw, rw\n" +"1:\n" + : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) + : [o]"r" (offset) + :"memory" ); +return prev - offset; +} + +#endif /* _ASM_RISCV_ATOMIC_H */ + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* indent-tabs-mode: nil +* End: +*/
xen/include/asm-generic/atomic-ops.h new file +97 -0
97 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
Quoted 9 lines · Oleksii Kurochko
Initially the patch was introduced by Bobby, who takes the header from Linux kernel.
The following changes were done on top of Linux kernel header:
- atomic##prefix##_*xchg_*(atomic##prefix##_t *v,c_tn) were updatedto use__*xchg_generic()- drop casts in
write_atomic()as they are unnecessary
- drop introduction ofWRITE_ONCE()andREAD_ONCE(). Xen providesACCESS_ONCE()
Here and in the code comment: While this may be describing what you did on top of what Bobby had, here you're describing differences to the Linux header.
Quoted 2 lines
- remove zero-length array access in
read_atomic()
- drop defines similar to pattern
pattern? Which one? Oh, wait, ...
Quoted 1 line
- #define
atomic_add_return_relaxedatomic_add_return_relaxed
... this line really isn't a separate bullet point.
Quoted 5 lines
Changes in V6:
- drop atomic##prefix##_{cmp}xchg_{release, aquire, relaxed} as they aren't used by Xen
- code style fixes.
- %s/__asm__ __volatile__/asm volatile
Btw, this is another activity that could do with being carried out consistently through the series.
Quoted 29 lines
xen/arch/riscv/include/asm/atomic.h new file +26
-0@@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Taken and modified from Linux. +* +* The following changes were done: +* - * atomic##prefix##_*xchg_*(atomic##prefix##_t *v, c_t n) were updated +* to use__*xchg_generic() +* - drop casts in write_atomic() as they are unnecessary +* - drop introduction of WRITE_ONCE() and READ_ONCE(). +* Xen provides ACCESS_ONCE() +* - remove zero-length array access in read_atomic() +* - drop defines similar to pattern +* #define atomic_add_return_relaxed atomic_add_return_relaxed +* - move not RISC-V specific functions to asm-generic/atomics-ops.h +* +* Copyright (C) 2007 Red Hat, Inc. All Rights Reserved. +* Copyright (C) 2012 Regents of the University of California +* Copyright (C) 2017 SiFive +* Copyright (C) 2024 Vates SAS +*/ + +#ifndef _ASM_RISCV_ATOMIC_H +#define _ASM_RISCV_ATOMIC_H + +#include <xen/atomic.h> +
xen/arch/riscv/include/asm/atomic.h line 36
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+#include <asm/cmpxchg.h> +#include <asm/fence.h> +#include <asm/io.h> +#include <asm/system.h> + +void __bad_atomic_size (void ); + +/* +* Legacy from Linux kernel. For some reason they wanted to have ordered +* read/write access. Thereby read* is used instead of read<X>_cpu()
Either read<X> and read<X>_cpu() or read* and read*_cpu(), please.
xen/arch/riscv/include/asm/atomic.h line 47
Quoted 11 lines
xen/arch/riscv/include/asm/atomic.h new file +11
-0+*/ +static always_inline void read_atomic_size(const volatile void *p, +void *res, +unsigned int size) +{ +switch ( size ) + { +case 1 : *(uint8_t *)res =readb (p);break ; +case 2 : *(uint16_t *)res =readw (p);break ; +case 4 : *(uint32_t *)res =readl (p);break ; +case 8 : *(uint32_t *)res =readq (p);break ;
Nit: Excess blank before =.
Also - no #ifdef here to be RV32-ready?
xen/arch/riscv/include/asm/atomic.h line 53
Quoted 6 lines
xen/arch/riscv/include/asm/atomic.h new file +6
-0+default :__bad_atomic_size ();break ; + } +} + +#define read_atomic (p) ({ \ + union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_; \
One trailing underscore here, but ...
xen/arch/riscv/include/asm/atomic.h line 60
Quoted 7 lines
xen/arch/riscv/include/asm/atomic.h new file +7
-0+read_atomic_size (p, x_.c,sizeof (*(p))); \ + x_.val; \ +}) + +#define write_atomic (p, x) \ +({ \ + typeof(*(p)) x__ = (x); \
... two here and ...
Quoted 4 lines
xen/arch/riscv/include/asm/atomic.h new file +4
-0+switch (sizeof (*(p)) ) \ + { \ +case 1 :writeb (x__, p);break ; \ +case 2 :writew (x__, p);break ; \
xen/arch/riscv/include/asm/atomic.h line 74
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+case 4 :writel (x__, p);break ; \ +case 8 :writeq (x__, p);break ; \ +default :__bad_atomic_size ();break ; \ + } \ + x__; \ +}) + +#define add_sized (p, x) \ +({ \ + typeof(*(p)) x__ = (x); \
... here?
Quoted 19 lines
xen/arch/riscv/include/asm/atomic.h new file +19
-0+switch (sizeof (*(p)) ) \ + { \ +case 1 :writeb (read_atomic (p) + x__, p);break ; \ +case 2 :writew (read_atomic (p) + x__, p);break ; \ +case 4 :writel (read_atomic (p) + x__, p);break ; \ +case 8 :writeq (read_atomic (p) + x__, p);break ; \ +default :__bad_atomic_size ();break ; \ + } \ +}) + +#define __atomic_acquire_fence () \ + asm volatile ( RISCV_ACQUIRE_BARRIER "" ::: "memory" ) + +#define __atomic_release_fence () \ + asm volatile ( RISCV_RELEASE_BARRIER "" ::: "memory" ) + +/* +* First, the atomic ops that have no ordering constraints and therefor don't +* have the AQ or RL bits set. These don't return anything, so there's only
xen/arch/riscv/include/asm/atomic.h line 103
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+* one version to worry about. +*/ +#define ATOMIC_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " zero, %1, %0" \ + : "+A" (v->counter) \ + : "r" (I) \
Btw, I consider this pretty confusing. At the 1st and 2nd glance this looks like a mistake, i.e. as if i was meant. Imo ...
Quoted 5 lines
xen/arch/riscv/include/asm/atomic.h new file +5
-0+ : "memory" ); \ +} \ + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
xen/arch/riscv/include/asm/atomic.h line 118
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+* last argument for ATOMIC_OP isn't used. +*/ +#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_OP (op, asm_op, I, w, int, ) + +ATOMIC_OPS(add, add, i) +ATOMIC_OPS(sub, add, -i) +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS (xor, xor, i)
... here you want to only pass the (unary) operator (and leaving that blank is as fine as using +).
xen/arch/riscv/include/asm/atomic.h line 126
Quoted 7 lines
xen/arch/riscv/include/asm/atomic.h new file +7
-0+#undef ATOMIC_OP +#undef ATOMIC_OPS + +#include <asm-generic/atomic-ops.h> + +/* +* Atomic ops that have ordered, relaxed, acquire, and release variants.
Only the first is implemented afaict; imo the comment would better reflect that one way or another.
Quoted 6 lines
xen/arch/riscv/include/asm/atomic.h new file +6
-0+* There's two flavors of these: the arithmatic ops have both fetch and return +* versions, while the logical ops only have fetch versions. +*/ +#define ATOMIC_FETCH_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +c_type atomic##prefix##_fetch_##op##_relaxed(c_type i, \
xen/arch/riscv/include/asm/atomic.h line 142
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+ atomic##prefix##_t *v) \ +{ \ + register c_type ret; \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " %1, %2, %0" \ + : "+A" (v->counter), "=r" (ret) \ + : "r" (I) \ + : "memory" ); \ + return ret; \ +} \
Actually a relaxed form is provided here, but does that have any user?
Quoted 13 lines
xen/arch/riscv/include/asm/atomic.h new file +13
-0+static inline \ +c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + register c_type ret; \ + asm volatile ( \ + " amo" #asm_op "." #asm_type ".aqrl %1, %2, %0" \ + : "+A" (v->counter), "=r" (ret) \ + : "r" (I) \ + : "memory" ); \ + return ret; \ +} + +#define ATOMIC_OP_RETURN (op, asm_op, c_op,I , asm_type, c_type, prefix) \
xen/arch/riscv/include/asm/atomic.h line 165
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+static inline \ +c_type atomic##prefix##_##op##_return_relaxed(c_type i, \ + atomic##prefix##_t *v) \ +{ \ + return atomic##prefix##_fetch_##op##_relaxed(i, v) c_op I; \ +} \ +static inline \ +c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v) \ +{ \ + return atomic##prefix##_fetch_##op(i, v) c_op I; \
I (or whatever the replacement expression is going to be following the earlier comment) wants parenthesizing here.
Quoted 29 lines
xen/arch/riscv/include/asm/atomic.h new file +29
-0+} + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why +* last argument of ATOMIC_FETCH_OP, ATOMIC_OP_RETURN isn't used. +*/ +#define ATOMIC_OPS (op, asm_op, c_op,I ) \ + ATOMIC_FETCH_OP( op, asm_op, I, w, int, ) \ + ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int, ) + +ATOMIC_OPS(add, add, +, i) +ATOMIC_OPS(sub, add, +, -i) + +#undef ATOMIC_OPS + +#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_FETCH_OP(op, asm_op, I, w, int, ) + +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i) + +#undef ATOMIC_OPS + +#undef ATOMIC_FETCH_OP +#undef ATOMIC_OP_RETURN + +/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless (atomic_t *v,int a,int u)
xen/arch/riscv/include/asm/atomic.h line 204
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+{ +int prev, rc; + + asmvolatile ( +"0: lr.w %[p], %[c]\n" +" beq %[p], %[u], 1f\n" +" add %[rc], %[p], %[a]\n" +" sc.w.rl %[rc], %[rc], %[c]\n" +" bnez %[rc], 0b\n" + RISCV_FULL_BARRIER
With this and no .aq on the load, why the .rl on the store?
Quoted 6 lines
xen/arch/riscv/include/asm/atomic.h new file +6
-0+ "1:\n" + : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter) + : [a] "r" (a), [u] "r" (u) + : "memory"); +return prev; +}
xen/arch/riscv/include/asm/atomic.h line 220
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+ +/* +* atomic_{cmp,}xchg is required to have exactly the same ordering semantics as +* {cmp,}xchg and the operations that return. +*/ +#define ATOMIC_OP (c_t, prefix, size) \ +static inline \ +c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ +{ \ + return __xchg(&(v->counter), n, size); \
No need for the inner parentheses, just like ...
xen/arch/riscv/include/asm/atomic.h line 225
Quoted 5 lines
xen/arch/riscv/include/asm/atomic.h new file +5
-0+} \ +static inline \ +c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \ +{ \ + return __cmpxchg(&v->counter, o, n, size); \
... you have it here.
Quoted 17 lines
xen/arch/riscv/include/asm/atomic.h new file +17
-0+} + +#define ATOMIC_OPS () \ + ATOMIC_OP(int, , 4) + +ATOMIC_OPS() + +#undef ATOMIC_OPS +#undef ATOMIC_OP + +static inline int atomic_sub_if_positive (atomic_t *v,int offset) +{ +int prev, rc; + + asmvolatile ( +"0: lr.w %[p], %[c]\n" + " sub %[rc], %[p], %[o]\n"
xen/arch/riscv/include/asm/atomic.h line 252
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+" bltz %[rc], 1f\n" +" sc.w.rl %[rc], %[rc], %[c]\n" +" bnez %[rc], 0b\n" +" fence rw, rw\n" +"1:\n" + : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter) + : [o] "r" (offset) + : "memory" ); +return prev - offset; +}
This probably would be nicer if sitting next to atomic_add_unless().
xen/include/asm-generic/atomic-ops.h line 5
Quoted 8 lines
xen/include/asm-generic/atomic-ops.h new file +5
-0@@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* The header provides default implementations for every xen/atomic.h-provided +* forward inline declaration that can be synthesized from other atomic +* functions.
Or from scratch, as e.g. ...
Quoted 24 lines
xen/include/asm-generic/atomic-ops.h new file +24
-0+*/ +#ifndef _ASM_GENERIC_ATOMIC_OPS_H_ +#define _ASM_GENERIC_ATOMIC_OPS_H_ + +#include <xen/atomic.h> +#include <xen/lib.h> + +#ifndef ATOMIC_READ +static inline int atomic_read(const atomic_t *v) +{ + return ACCESS_ONCE(v->counter); +} +#endif + +#ifndef _ATOMIC_READ +static inline int _atomic_read(atomic_t v) +{ + return v.counter; +} +#endif + +#ifndef ATOMIC_SET +static inline void atomic_set(atomic_t *v, int i) +{
xen/include/asm-generic/atomic-ops.h line 39
Quoted 10 lines
xen/include/asm-generic/atomic-ops.h new file +10
-0+ ACCESS_ONCE(v->counter) = i; +} +#endif + +#ifndef _ATOMIC_SET +static inline void _atomic_set (atomic_t *v,int i) +{ + v->counter = i; +} +#endif
... all of these.
Quoted 45 lines
xen/include/asm-generic/atomic-ops.h new file +45
-0+#ifndef ATOMIC_SUB_AND_TEST +static inline int atomic_sub_and_test (int i,atomic_t *v) +{ +return atomic_sub_return (i, v) ==0 ; +} +#endif + +#ifndef ATOMIC_INC +static inline void atomic_inc (atomic_t *v) +{ +atomic_add (1 , v); +} +#endif + +#ifndef ATOMIC_INC_RETURN +static inline int atomic_inc_return (atomic_t *v) +{ +return atomic_add_return (1 , v); +} +#endif + +#ifndef ATOMIC_DEC +static inline void atomic_dec (atomic_t *v) +{ +atomic_sub (1 , v); +} +#endif + +#ifndef ATOMIC_DEC_RETURN +static inline int atomic_dec_return (atomic_t *v) +{ +return atomic_sub_return (1 , v); +} +#endif + +#ifndef ATOMIC_DEC_AND_TEST +static inline int atomic_dec_and_test (atomic_t *v) +{ +return atomic_sub_return (1 , v) ==0 ; +} +#endif + +#ifndef ATOMIC_ADD_NEGATIVE +static inline int atomic_add_negative (int i,atomic_t *v) +{
xen/include/asm-generic/atomic-ops.h line 95
Quoted 10 lines
xen/include/asm-generic/atomic-ops.h new file +10
-0+ return atomic_add_return(i, v) < 0; +} +#endif + +#ifndef ATOMIC_INC_AND_TEST +static inline int atomic_inc_and_test (atomic_t *v) +{ +return atomic_add_return (1 , v) ==0 ; +} +#endif
Can this be moved up a little, perhaps next to the other inc-s (or else next to dec_and_test), please?
Jan
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
Quoted 19 lines · Jan Beulich
Quoted 11 lines · Oleksii Kurochko
Initially the patch was introduced by Bobby, who takes the header from
Linux kernel.The following changes were done on top of Linux kernel header:
- atomic##prefix##_*xchg_*(atomic##prefix##_t *v,c_tn) were updated
to use__*xchg_generic()
- drop casts inwrite_atomic()as they are unnecessary
- drop introduction ofWRITE_ONCE()andREAD_ONCE(). Xen providesACCESS_ONCE()Here and in the code comment: While this may be describing what you did
on top of what Bobby had, here you're describing differences to the Linux
header.
Quoted 8 lines
Quoted 2 lines
- remove zero-length array access in
read_atomic()
- drop defines similar to patternpattern? Which one? Oh, wait, ...
Quoted 1 line
- #define atomic_add_return_relaxed
atomic_add_return_relaxed... this line really isn't a separate bullet point.
Yes, '-' is not needed in this text.
Quoted 15 lines
xen/arch/riscv/include/asm/atomic.h line 47
Quoted 12 lines
+ */
xen/arch/riscv/include/asm/atomic.h new file +10
-0+static always_inline void read_atomic_size(const volatile void *p, + void *res, + unsigned int size) +{ + switch ( size ) + { + case 1: *(uint8_t *)res = readb(p); break; + case 2: *(uint16_t *)res = readw(p); break; + case 4: *(uint32_t *)res = readl(p); break; + case 8: *(uint32_t *)res = readq(p); break;Nit: Excess blank before =.
Quoted 1 line
Also - no #ifdef here to be RV32-ready?
Because there is #ifdef RV32 in io.h for readq().
Quoted 20 lines
xen/arch/riscv/include/asm/atomic.h line 53
Quoted 7 lines
+ default:
__bad_atomic_size(); break;
+ }xen/arch/riscv/include/asm/atomic.h new file +4
-0+} + +#define read_atomic (p) ({ \ + union { typeof(*(p)) val; char c[sizeof(*(p))]; } x_; \One trailing underscore here, but ...
xen/arch/riscv/include/asm/atomic.h line 60
Quoted 7 lines
+
read_atomic_size(p, x_.c, sizeof(*(p))); \
+ x_.val; \xen/arch/riscv/include/asm/atomic.h new file +5
-0+}) + +#define write_atomic (p, x) \ +({ \ + typeof(*(p)) x__ = (x); \... two here and ...
Quoted 16 lines
Quoted 9 lines
+ switch ( sizeof(*(p)) ) \
+ { \
+ case 1:writeb(x__, p); break; \
+ case 2:writew(x__, p); break; \
+ case 4:writel(x__, p); break; \
+ case 8:writeq(x__, p); break; \
+ default:__bad_atomic_size(); break; \
+ } \
+ x__; \xen/arch/riscv/include/asm/atomic.h line 74
Quoted 5 lines
xen/arch/riscv/include/asm/atomic.h new file +5
-0+}) + +#define add_sized (p, x) \ +({ \ + typeof(*(p)) x__ = (x); \... here?
I'll update in the same way.
Quoted 48 lines
Quoted 24 lines
+ switch ( sizeof(*(p)) ) \
+ { \
+ case 1:writeb(read_atomic(p) + x__, p); break; \
+ case 2:writew(read_atomic(p) + x__, p); break; \
+ case 4:writel(read_atomic(p) + x__, p); break; \
+ case 8:writeq(read_atomic(p) + x__, p); break; \
+ default:__bad_atomic_size(); break; \
+ } \xen/arch/riscv/include/asm/atomic.h new file +10
-0+}) + +#define __atomic_acquire_fence () \ + asm volatile ( RISCV_ACQUIRE_BARRIER "" ::: "memory" ) + +#define __atomic_release_fence () \ + asm volatile ( RISCV_RELEASE_BARRIER "" ::: "memory" ) + +/* +* First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set. These don't return anything, so there's only
+ * one version to worry about.
+ */xen/arch/riscv/include/asm/atomic.h line 103
Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h new file +8
-0+#define ATOMIC_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " zero, %1, %0" \ + : "+A" (v->counter) \ + : "r" (I) \Btw, I consider this pretty confusing. At the 1st and 2nd glance this looks like a mistake, i.e. as if i was meant. Imo ...
Quoted 12 lines
+ : "memory" ); \
xen/arch/riscv/include/asm/atomic.h new file +4
-0+} \ + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument forATOMIC_OPisn't used.
+ */xen/arch/riscv/include/asm/atomic.h new file +4
-0+#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_OP (op, asm_op, I, w, int, ) + +ATOMIC_OPS(add, add, i)
Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h line 118
Quoted 4 lines
xen/arch/riscv/include/asm/atomic.h new file +4
-0+ATOMIC_OPS(sub, add, -i) +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i)... here you want to only pass the (unary) operator (and leaving that blank
is as fine as using +).
I agree that a game with 'i' and 'I' looks confusing, but I am not really understand what is wrong with using ' i' here. It seems that preprocessed macros looks fine:
static inline void atomic_add(int i, atomic_t *v) { asm volatile ( " amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (i) :
"memory" ); }
static inline void atomic_sub(int i, atomic_t *v) { asm volatile ( " amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (-i) :
"memory" ); }
Quoted 39 lines
Quoted 9 lines
xen/arch/riscv/include/asm/atomic.h new file +7
-0+#undef ATOMIC_OP +#undef ATOMIC_OPS + +#include <asm-generic/atomic-ops.h> + +/* +* Atomic ops that have ordered, relaxed, acquire, and release variants.
Only the first is implemented afaict; imo the comment would better reflect
that one way or another.Quoted 25 lines
+ * There's two flavors of these: the arithmatic ops have both fetch and return
+ * versions, while the logical ops only have fetch versions.
+ */
+#defineATOMIC_FETCH_OP(op, asm_op, I, asm_type, c_type, prefix)\
+static
inline \
+c_type atomic##prefix##_fetch_##op##_relaxed(c_type i, \
+ atomic##prefix##_t
*v) \
+{
\
+ registerc_type
ret; \
+ asm volatile
( \
+ " amo" #asm_op "." #asm_type " %1, %2, %0" \
+ : "+A" (v->counter), "=r"
(ret) \
+ : "r"
(I) \
+ : "memory"
Quoted 8 lines
Quoted 5 lines
); \
+ return
ret; \
+}
\Actually a relaxed form is provided here, but does that have any user?
There is no user for a relaxed form, just overlooked that.
Quoted 86 lines
Quoted 37 lines
+static
inline \
+c_type atomic##prefix##_fetch_##op(c_type i, atomic##prefix##_t *v) \
+{
\
+ registerc_type
ret; \
+ asm volatile
( \
+ " amo" #asm_op "." #asm_type ".aqrl %1, %2, %0" \
+ : "+A" (v->counter), "=r"
(ret) \
+ : "r"
(I) \
+ : "memory"
); \
+ return
ret; \xen/arch/riscv/include/asm/atomic.h new file +3
-0+} + +#define ATOMIC_OP_RETURN(op, asm_op, c_op, I, asm_type, c_type,prefix) \
+static
inline \
+c_type atomic##prefix##_##op##_return_relaxed(c_type i, \
+ atomic##prefix##_t
*v) \
+{
\
+ return atomic##prefix##_fetch_##op##_relaxed(i, v)c_opI; \
+}
\Quoted 8 lines
+static
inline \
+c_type atomic##prefix##_##op##_return(c_type i, atomic##prefix##_t *v) \
+{
\
+ return atomic##prefix##_fetch_##op(i, v)c_opI; \I (or whatever the replacement expression is going to be following the
earlier comment) wants parenthesizing here.Quoted 36 lines
xen/arch/riscv/include/asm/atomic.h new file +4
-0+} + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument ofATOMIC_FETCH_OP,ATOMIC_OP_RETURNisn't used.
+ */
+#define ATOMIC_OPS(op,asm_op,c_op,
I) \
+ATOMIC_FETCH_OP( op, asm_op, I, w, int, )\
+ATOMIC_OP_RETURN(op, asm_op, c_op, I, w, int, )xen/arch/riscv/include/asm/atomic.h new file +24
-0+ +ATOMIC_OPS(add, add, +, i) +ATOMIC_OPS(sub, add, +, -i) + +#undef ATOMIC_OPS + +#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_FETCH_OP(op, asm_op, I, w, int, ) + +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i) + +#undef ATOMIC_OPS + +#undef ATOMIC_FETCH_OP +#undef ATOMIC_OP_RETURN + +/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless (atomic_t *v,int a,int u) +{ + int prev, rc; + + asm volatile (
Quoted 8 lines
Quoted 6 lines
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n"
+ " sc.w.rl %[rc], %[rc], %[c]\n"
+ " bnez %[rc], 0b\n"
+RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring to the spec:
Ordering Annotation Fence-based Equivalent
l{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw
l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw
s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c}
s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}
amo<op>.aq amo<op>; fence r,rw
amo<op>.rl fence rw,w; amo<op>
amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw
Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existing
s{b|h|w|d|c} mapping rather than at the front of the
l{b|h|w|d|r} mapping. It is also safe to translate any .aq, .rl, or .aqrl annotation into the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added to the ISA.
So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.
Considering [1] and how this code looks before, it seems to me that it is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.
But in general it ( a combination of fence, .aq, .rl ) can be considered as the same things in this context, so it is possible to leave this function as is to be synced here with Linux kernel.
[1]https://lore.kernel.org/lkml/[email protected]/
~ Oleksii
Quoted 171 lines
Quoted 13 lines
+ "1:\n"
+ : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter) + : [a] "r" (a), [u] "r" (u)
+ : "memory");
+ return prev;xen/arch/riscv/include/asm/atomic.h new file +4
-0+} + +/* +* atomic_{cmp,}xchg is required to have exactly the same ordering semantics as
+ * {cmp,}xchg and the operations that return.
+ */xen/arch/riscv/include/asm/atomic.h line 220
Quoted 5 lines
xen/arch/riscv/include/asm/atomic.h new file +5
-0+#define ATOMIC_OP (c_t, prefix, size) \ +static inline \ +c_t atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ +{ \ + return __xchg(&(v->counter), n, size); \No need for the inner parentheses, just like ...
Quoted 6 lines
xen/arch/riscv/include/asm/atomic.h new file +3
-0+} \ +static inline \ +c_t atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n)\
+{ \
+ return__cmpxchg(&v->counter, o, n, size); \... you have it here.
Quoted 17 lines
xen/arch/riscv/include/asm/atomic.h new file +17
-0+} + +#define ATOMIC_OPS () \ + ATOMIC_OP(int, , 4) + +ATOMIC_OPS() + +#undef ATOMIC_OPS +#undef ATOMIC_OP + +static inline int atomic_sub_if_positive (atomic_t *v,int offset) +{ + int prev, rc; + + asm volatile ( + "0: lr.w %[p], %[c]\n" + " sub %[rc], %[p], %[o]\n"xen/arch/riscv/include/asm/atomic.h line 252
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+ " bltz %[rc], 1f\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n" + " fence rw, rw\n" + "1:\n" + : [p] "=&r" (prev), [rc] "=&r" (rc), [c] "+A" (v->counter) + : [o] "r" (offset) + : "memory" ); + return prev - offset; +}This probably would be nicer if sitting next to
atomic_add_unless().Quoted 10 lines
xen/include/asm-generic/atomic-ops.h new file +3
-0@@ -0,0 +1,97 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* The header provides default implementations for every xen/atomic.h-provided
+ * forward inline declaration that can be synthesized from other atomic
+ * functions.Or from scratch, as e.g. ...
Quoted 24 lines
+ */
xen/include/asm-generic/atomic-ops.h new file +23
-0+#ifndef _ASM_GENERIC_ATOMIC_OPS_H_ +#define _ASM_GENERIC_ATOMIC_OPS_H_ + +#include <xen/atomic.h> +#include <xen/lib.h> + +#ifndef ATOMIC_READ +static inline int atomic_read(const atomic_t *v) +{ + return ACCESS_ONCE(v->counter); +} +#endif + +#ifndef _ATOMIC_READ +static inline int _atomic_read(atomic_t v) +{ + return v.counter; +} +#endif + +#ifndef ATOMIC_SET +static inline void atomic_set(atomic_t *v, int i) +{xen/include/asm-generic/atomic-ops.h line 39
Quoted 10 lines
xen/include/asm-generic/atomic-ops.h new file +10
-0+ ACCESS_ONCE(v->counter) = i; +} +#endif + +#ifndef _ATOMIC_SET +static inline void _atomic_set (atomic_t *v,int i) +{ + v->counter = i; +} +#endif ... all of these.
Quoted 45 lines
xen/include/asm-generic/atomic-ops.h new file +45
-0+#ifndef ATOMIC_SUB_AND_TEST +static inline int atomic_sub_and_test (int i,atomic_t *v) +{ + return atomic_sub_return(i, v) == 0; +} +#endif + +#ifndef ATOMIC_INC +static inline void atomic_inc (atomic_t *v) +{ + atomic_add(1, v); +} +#endif + +#ifndef ATOMIC_INC_RETURN +static inline int atomic_inc_return (atomic_t *v) +{ + return atomic_add_return(1, v); +} +#endif + +#ifndef ATOMIC_DEC +static inline void atomic_dec (atomic_t *v) +{ + atomic_sub(1, v); +} +#endif + +#ifndef ATOMIC_DEC_RETURN +static inline int atomic_dec_return (atomic_t *v) +{ + return atomic_sub_return(1, v); +} +#endif + +#ifndef ATOMIC_DEC_AND_TEST +static inline int atomic_dec_and_test (atomic_t *v) +{ + return atomic_sub_return(1, v) == 0; +} +#endif + +#ifndef ATOMIC_ADD_NEGATIVE +static inline int atomic_add_negative (int i,atomic_t *v) +{xen/include/asm-generic/atomic-ops.h line 95
Quoted 10 lines
xen/include/asm-generic/atomic-ops.h new file +10
-0+ return atomic_add_return(i, v) < 0; +} +#endif + +#ifndef ATOMIC_INC_AND_TEST +static inline int atomic_inc_and_test (atomic_t *v) +{ + return atomic_add_return(1, v) == 0; +} +#endif Can this be moved up a little, perhaps next to the other inc-s (or else
next todec_and_test), please?Jan
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
Quoted 16 lines · Oleksii
Quoted 15 lines · Jan Beulich
xen/arch/riscv/include/asm/atomic.h line 47
Quoted 11 lines · Oleksii Kurochko
+ */
xen/arch/riscv/include/asm/atomic.h new file +10
-0+static always_inline void read_atomic_size(const volatile void *p, + void *res, + unsigned int size) +{ + switch ( size ) + { + case 1: *(uint8_t *)res = readb(p); break; + case 2: *(uint16_t *)res = readw(p); break; + case 4: *(uint32_t *)res = readl(p); break; + case 8: *(uint32_t *)res = readq(p); break;Nit: Excess blank before =.
Quoted 2 lines
Quoted 1 line
Also - no #ifdef here to be RV32-ready?
Because there is #ifdef RV32 in
io.hforreadq().
There you'd run into __raw_readq()'s BUILD_BUG_ON(), afaict even for 1-, 2-, or 4-byte accesses. That's not quite what we want here.
Quoted 42 lines
Quoted 31 lines
Quoted 7 lines
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set. These don't return anything, so there's only
+ * one version to worry about.
+ */xen/arch/riscv/include/asm/atomic.h line 103
Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h new file +8
-0+#define ATOMIC_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " zero, %1, %0" \ + : "+A" (v->counter) \ + : "r" (I) \Btw, I consider this pretty confusing. At the 1st and 2nd glance this looks like a mistake, i.e. as if i was meant. Imo ...
Quoted 12 lines
+ : "memory" ); \
xen/arch/riscv/include/asm/atomic.h new file +4
-0+} \ + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the reason why
+ * last argument forATOMIC_OPisn't used.
+ */xen/arch/riscv/include/asm/atomic.h new file +4
-0+#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_OP (op, asm_op, I, w, int, ) + +ATOMIC_OPS(add, add, i)Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h line 118
Quoted 4 lines
xen/arch/riscv/include/asm/atomic.h new file +4
-0+ATOMIC_OPS(sub, add, -i) +ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i)... here you want to only pass the (unary) operator (and leaving that blank
is as fine as using +).I agree that a game with 'i' and 'I' looks confusing, but I am not really understand what is wrong with using ' i' here. It seems that preprocessed macros looks fine:
Quoted 7 lines
static inline void atomic_add(int i, atomic_t *v) { asm volatile ( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (i) :
"memory" ); }
static inline void atomic_sub(int i, atomic_t *v) { asm volatile ( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (-i) :
"memory" ); }I didn't question the pre-processed result being correct. Instead I said that I consider the construct confusing to the reader, for looking as if there was a mistake (in the case of the letter i used). Note also in particular how the macro invocations need to be in sync with the macro implementation, for lower case i being used both in the macro and in its invocations. Anything parameterized would better be fully so, at the very least to avoid, as said, confusion. (Having macros depend on context at their use sites _may_ be okay for local helper macros, but here we're talking about a not even private header file.)
Quoted 19 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/riscv/include/asm/atomic.h new file +6
-0+/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless (atomic_t *v,int a,int u) +{ + int prev, rc; + + asm volatile (Quoted 8 lines
Quoted 6 lines
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n"
+ " sc.w.rl %[rc], %[rc], %[c]\n"
+ " bnez %[rc], 0b\n"
+RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring to the spec:
Ordering Annotation Fence-based Equivalent
Quoted 11 lines
l{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw
l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw
s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c}
s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}
amo<op>.aq amo<op>; fence r,rw
amo<op>.rl fence rw,w; amo<op>
amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw
Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existing
s{b|h|w|d|c} mapping rather than at the front of the
l{b|h|w|d|r} mapping.I'm afraid I can't spot the specific case in this table. None of the stores in the right column have a .rl suffix.
Quoted 7 lines
It is also safe to translate any .aq, .rl, or .aqrl annotation into the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added to the ISA.
Quoted 6 lines
So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.Considering [1] and how this code looks before, it seems to me that it is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.
Here you say "or". Then why dos the code use sc.?.rl _and_ a fence?
Quoted 3 lines
But in general it ( a combination of fence, .aq, .rl ) can be considered as the same things in this context, so it is possible to leave this function as is to be synced here with Linux kernel.
In turn I also don't understand this. Yes, the excess .rl certainly doesn't render things unsafe. But what's the purpose of the .rl? That's what my original question boiled down to.
Jan
Quoted 3 lines
[1]https://lore.kernel.org/lkml/[email protected]/
~ Oleksii
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii
Quoted 16 lines · Jan Beulich
Quoted 4 lines · Oleksii Kurochko
+ */
+staticalways_inlinevoid read_atomic_size(const volatile void *p,
+ void *res,Quoted 8 lines
+ unsigned int size) +{
+ switch ( size )
+ {
+ case 1: *(uint8_t *)res = readb(p); break;
+ case 2: *(uint16_t *)res = readw(p); break;
+ case 4: *(uint32_t *)res = readl(p); break;
+ case 8: *(uint32_t *)res = readq(p); break;Nit: Excess blank before =.
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Also - no #ifdef here to be RV32-ready?
Because there is #ifdef RV32 in
io.hforreadq().There you'd run into __raw_readq()'s
BUILD_BUG_ON(), afaict even for 1-, 2-, or 4-byte accesses. That's not quite what we want here.
Do you mean that if someone will redefine readq() in another way and not wrap it by #ifdef RV32? Except this I am not sure that there is an issue as it will be still a compilation error, so anyway it will be needed to provide an implementation for __raw_readq().
One of the reason why I decided to wrap with #ifdef RV32 in io.h to not go over the source code and add wrapping. Also for some code it will be needed to rewrite it. For example, I am not sure that I can add #ifdef inside macros, f.e.:
#define write_atomic(p, x) \
({ \
typeof(*(p)) x__ = (x); \
switch ( sizeof(*(p)) ) \
{ \
case 1: writeb(x__, p); break; \
case 2: writew(x__, p); break; \
case 4: writel(x__, p); break; \
case 8: writeq(x__, p); break; \
default: __bad_atomic_size(); break; \
} \
x__; \})
Quoted 59 lines
Quoted 44 lines
Quoted 36 lines
Quoted 9 lines
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set. These don't return anything, so
there's only
+ * one version to worry about.
+ */xen/arch/riscv/include/asm/atomic.h line 103
Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h new file +8
-0+#define ATOMIC_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " zero, %1, %0" \ + : "+A" (v->counter) \ + : "r" (I) \Btw, I consider this pretty confusing. At the 1st and 2nd glance this
looks like a mistake, i.e. as if i was meant. Imo ...Quoted 14 lines
+ : "memory" ); \
xen/arch/riscv/include/asm/atomic.h new file +4
-0+} \ + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the
reason why
+ * last argument forATOMIC_OPisn't used.
+ */xen/arch/riscv/include/asm/atomic.h new file +5
-0+#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_OP (op, asm_op, I, w, int, ) + +ATOMIC_OPS(add, add, i) +ATOMIC_OPS (sub, add, -i)Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h line 118
Quoted 3 lines
xen/arch/riscv/include/asm/atomic.h new file +3
-0+ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i)... here you want to only pass the (unary) operator (and leaving that
blank
is as fine as using +).Quoted 14 lines
I agree that a game with 'i' and 'I' looks confusing, but I am not really understand what is wrong with using ' i' here. It seems that preprocessed macros looks fine:
static inline voidatomic_add(int i, atomic_t *v){ asm volatile ( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (i) :
"memory" ); }
static inline voidatomic_sub(int i, atomic_t *v){ asm volatile ( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (-
i) :
"memory" ); }
Quoted 13 lines
I didn't question the pre-processed result being correct. Instead I said
that I consider the construct confusing to the reader, for looking as if
there was a mistake (in the case of the letter i used). Note also in particular how the macro invocations need to be in sync with the macro
implementation, for lower case i being used both in the macro and in its
invocations. Anything parameterized would better be fully so, at the very least to avoid, as said, confusion. (Having macros depend on context at their use sites _may_ be okay for local helper macros, but here we're talking about a not even private header file.)
I am not sure then I understand how mentioning '+i' will help significantly remove confusion.
Quoted 34 lines
Quoted 25 lines
Quoted 15 lines
xen/arch/riscv/include/asm/atomic.h line 204
Quoted 13 lines
xen/arch/riscv/include/asm/atomic.h new file +12
-0+/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless (atomic_t *v,int a,int u) +{ + int prev, rc; + + asm volatile ( + "0: lr.w %[p], %[c]\n" + " beq %[p], %[u], 1f\n" + " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n" + RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring
to the spec:
Ordering Annotation Fence-based Equivalentl{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c} s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}Quoted 8 lines
amo<op>.aq amo<op>; fence r,rw
amo<op>.rl fence rw,w; amo<op>
amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existings{b|h|w|d|c} mapping rather than at the front of the l{b|h|w|d|r} mapping.
Quoted 2 lines
I'm afraid I can't spot the specific case in this table. None of the stores in the right column have a .rl suffix.
Yes, it is expected.
I am reading this table as (f.e.) amo<op>.rl is an equivalent of fence rw,w; amo<op>. (without .rl)
Quoted 14 lines
Quoted 14 lines
It is also safe to translate any .aq, .rl, or .aqrl annotation into
the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added to
the ISA.So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.
Quoted 5 lines
Quoted 3 lines
Considering [1] and how this code looks before, it seems to me that it
is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.Here you say "or". Then why dos the code use sc.?.rl _and_ a fence?
I confused this line with amo<op>.aqrl, so based on the table 2.2 above s{b|h|w|d|c}.aqrl is transformed to "fence rw,w; s{b|h|w|d|c}", but Linux kernel decided to strengthen it with full barrier:
- "0:\n\t" - "lr.w.aqrl %[p], %[c]\n\t" - "beq %[p], %[u], 1f\n\t" - "add %[rc], %[p], %[a]\n\t" - "sc.w.aqrl %[rc], %[rc], %[c]\n\t" - "bnez %[rc], 0b\n\t" - "1:" + "0: lr.w %[p], %[c]\n" + " beq %[p], %[u], 1f\n" + " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n" + " fence rw, rw\n" + "1:\n"
As they have the following issue:
implementations of atomics such as atomic_cmpxchg() and atomic_add_unless() rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test below to end up with the state indicated in the "exists" clause.
Quoted 9 lines
Quoted 4 lines
But in general it ( a combination of fence, .aq, .rl ) can be considered as the same things in this context, so it is possible to leave this function as is to be synced here with Linux kernel.
In turn I also don't understand this. Yes, the excess .rl certainly doesn't render things unsafe. But what's the purpose of the .rl? That's
what my original question boiled down to.
I don't know, either. As I mentioned before, it is enough ( in my opinion ) to have a FULL barrier or .aq,.rl or .aqrl/.aqrl ( if it needed to be strengthened) like it was done before in Linux. It seems to me it is LKMM specific that they need more to be more strengthened as it RISC-V Memory model requires because:
"sc.w ; fence rw, rw" does not guarantee that all previous reads and writes finish before the sc itself is globally visible, which might matter if the sc is unlocking a lock or something.
Despite of the fact, for compare-and-swap loops, RISC-V international recommends lr.w.aq/lr.d.aq followed by sc.w.rl/sc.d.rl ( as it was implemeted before in Linux kernel ) I am okay just for safety reasons and for the reason I mentioned at the last sentence of previous paragraph to strengthen implementations with fences.
~ Oleksii
Quoted 9 lines
Jan
Quoted 5 lines
[1]
https://lore.kernel.org/lkml/[email protected]
/~ Oleksii
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
Quoted 262 lines · Oleksii
Quoted 20 lines · Jan Beulich
Quoted 19 lines · Oleksii
Quoted 18 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
+ */
+staticalways_inlinevoid read_atomic_size(const volatile void
*p,
+ void *res, + unsigned intQuoted 8 lines
size)
+{
+ switch ( size )
+ {
+ case 1: *(uint8_t *)res = readb(p); break;
+ case 2: *(uint16_t *)res = readw(p); break;
+ case 4: *(uint32_t *)res = readl(p); break;
+ case 8: *(uint32_t *)res = readq(p); break;Nit: Excess blank before =.
Quoted 6 lines
Quoted 2 lines
Quoted 1 line
Also - no #ifdef here to be RV32-ready?
Because there is #ifdef RV32 in
io.hforreadq().There you'd run into __raw_readq()'s
BUILD_BUG_ON(), afaict even for
1-, 2-, or 4-byte accesses. That's not quite what we want here.Do you mean that if someone will redefine
readq()in another way and not wrap it by #ifdef RV32? Except this I am not sure that there is an
issue as it will be still a compilation error, so anyway it will be needed to provide an implementation for__raw_readq().One of the reason why I decided to wrap with #ifdef RV32 in
io.hto not
go over the source code and add wrapping. Also for some code it will be
needed to rewrite it. For example, I am not sure that I can add #ifdef
inside macros, f.e.:
#definewrite_atomic(p, x)\
({ \
typeof(*(p))x__= (x); \
switch ( sizeof(*(p)) ) \
{ \
case 1:writeb(x__, p); break; \
case 2:writew(x__, p); break; \
case 4:writel(x__, p); break; \
case 8:writeq(x__, p); break; \
default:__bad_atomic_size(); break; \
} \
x__; \
})Quoted 89 lines
Quoted 70 lines
Quoted 50 lines
Quoted 18 lines
+/*
+ * First, the atomic ops that have no ordering constraints and
therefor don't
+ * have the AQ or RL bits set. These don't return anything, so
there's only
+ * one version to worry about.
+ */
+#defineATOMIC_OP(op, asm_op, I, asm_type, c_type, prefix)\
+static inline
\
+void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \
+{
\Quoted 8 lines
+ asm volatile (
\
+ " amo" #asm_op "." #asm_type " zero, %1, %0" \
+ : "+A" (v->counter)
\
+ : "r" (I)
\Btw, I consider this pretty confusing. At the 1st and 2nd glance
this
looks like a mistake, i.e. as if i was meant. Imo ...Quoted 18 lines
+ : "memory" );
\
+}
\xen/arch/riscv/include/asm/atomic.h new file +3
-0+ +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the
reason why
+ * last argument forATOMIC_OPisn't used.
+ */
+#defineATOMIC_OPS(op, asm_op, I)
\
+ATOMIC_OP(op,asm_op, I, w, int, )xen/arch/riscv/include/asm/atomic.h new file +4
-0+ +ATOMIC_OPS(add, add, i) +ATOMIC_OPS(sub, add, -i) +ATOMIC_OPS(and, and, i)Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i)... here you want to only pass the (unary) operator (and leaving
that
blank
is as fine as using +).I agree that a game with 'i' and 'I' looks confusing, but I am not
really understand what is wrong with using ' i' here. It seems that
preprocessed macros looks fine:
static inline voidatomic_add(int i, atomic_t *v){ asm volatile
( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r"
(i)
:
"memory" ); }Quoted 8 lines
static inline voidatomic_sub(int i, atomic_t *v){ asm volatile
( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (-
i) :
"memory" ); }I didn't question the pre-processed result being correct. Instead I said
that I consider the construct confusing to the reader, for looking as
if
there was a mistake (in the case of the letter i used). Note also in
particular how the macro invocations need to be in sync with the macro
implementation, for lower case i being used both in the macro andQuoted 8 lines
in
its
invocations. Anything parameterized would better be fully so, at the
very least to avoid, as said, confusion. (Having macros depend on context at their use sites _may_ be okay for local helper macros, but
here we're talking about a not even private header file.)I am not sure then I understand how mentioning '+i' will help significantly remove confusion.
Quoted 35 lines
Quoted 26 lines
Quoted 16 lines
Quoted 4 lines
(quoted diff) +2
-0+/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless(atomic_t *v, int a, intu)
xen/arch/riscv/include/asm/atomic.h line 204
Quoted 10 lines
xen/arch/riscv/include/asm/atomic.h new file +10
-0+{ + int prev, rc; + + asm volatile ( + "0: lr.w %[p], %[c]\n" + " beq %[p], %[u], 1f\n" + " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n" + RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring
to the spec:
Ordering Annotation Fence-based Equivalentl{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c} s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}Quoted 8 lines
amo<op>.aq amo<op>; fence r,rw
amo<op>.rl fence rw,w; amo<op>
amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existings{b|h|w|d|c} mapping rather than at the front of the l{b|h|w|d|r} mapping.Quoted 3 lines
I'm afraid I can't spot the specific case in this table. None of the
stores in the right column have a .rl suffix.Yes, it is expected.
I am reading this table as (f.e.) amo<op>.rl is an equivalent of fence
rw,w; amo<op>. (without .rl)Quoted 14 lines
Quoted 14 lines
It is also safe to translate any .aq, .rl, or .aqrl annotation into
the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added to
the ISA.So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.Quoted 6 lines
Quoted 4 lines
Considering [1] and how this code looks before, it seems to me that
it
is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.Here you say "or". Then why dos the code use sc.?.rl _and_ a fence?
I confused this line with amo<op>.aqrl, so based on the table 2.2 above
s{b|h|w|d|c}.aqrl is transformed to "fence rw,w; s{b|h|w|d|c}", but Linux kernel decided to strengthen it with full barrier:
- "0:\n\t"
- "lr.w.aqrl %[p], %[c]\n\t"
- "beq %[p], %[u], 1f\n\t"
- "add %[rc], %[p], %[a]\n\t"
- "sc.w.aqrl %[rc], %[rc], %[c]\n\t"
- "bnez %[rc], 0b\n\t"
- "1:"
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n"
+ " fence rw, rw\n"
+ "1:\n"
As they have the following issue:
implementations of atomics such asatomic_cmpxchg()andatomic_add_unless()rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test below to end up with the state indicated in the "exists" clause.Quoted 10 lines
Quoted 5 lines
But in general it ( a combination of fence, .aq, .rl ) can be considered as the same things in this context, so it is possible to
leave this function as is to be synced here with Linux kernel.In turn I also don't understand this. Yes, the excess .rl certainly doesn't render things unsafe. But what's the purpose of the .rl? That's
what my original question boiled down to.I don't know, either. As I mentioned before, it is enough ( in my opinion ) to have a FULL barrier or .aq,.rl or .aqrl/.aqrl ( if it needed to be strengthened) like it was done before in Linux. It seems to me it is LKMM specific that they need more to be more strengthened as it RISC-V Memory model requires because:
"sc.w ; fence rw, rw" does not guarantee that all previous reads and writes finish before the sc itself is globally visible, which might matter if the sc is unlocking a lock or something.
Quoted 5 lines
Despite of the fact, for compare-and-swap loops, RISC-V international recommends lr.w.aq/lr.d.aq followed by sc.w.rl/sc.d.rl ( as it was implemeted before in Linux kernel ) I am okay just for safety reasons and for the reason I mentioned at the last sentence of previous paragraph to strengthen implementations with fences.
Regarding the necessity of fence rw,rw, there is a commit that is proposingto remove fences: [2]. Additionally, it seems there is another reason why the fences were
added.
At the time when the patch introducing the usage of fences was
implemented,
there were no rules in RVWMO which allowed ld.aq+sc.aqrl to be
considered
as a full barrier [3].
Quoted 1 line
Quoted 1 line
note that Model 2018 explicitly says that "ld.aq+sc.aqrl" is
ordered
Quoted 1 line
Quoted 1 line
against "earlier or later memory operations from the same hart",
and
Quoted 3 lines
Quoted 3 lines
this statement was not in Model 2017.
So my understanding of the story is that at some point between
March and
Quoted 1 line
Quoted 1 line
May 2018, RISV memory model folks decided to add this rule,
which
does
Quoted 1 line
Quoted 1 line
look more consistent with other parts of the model and is
useful.
Quoted 2 lines
Quoted 2 lines
And this is why (and when) "ld.aq+sc.aqrl" can be used as a
fully-
ordered
Quoted 3 lines
Quoted 3 lines
barrier ;-)
Now if my understanding is correct, to move forward, it's better
that 1)
Quoted 1 line
Quoted 1 line
this patch gets resend with the above information (better
rewording
a
Quoted 1 line
Quoted 1 line
bit), and 2) gets an Acked-by from Dan to confirm this is a
correct
Quoted 1 line
Quoted 1 line
history ;-)
Based on patch [2], it may be possible to remove the full barrier and switch from sc..rl to sc..aqrl. However, I need to finish reading the Linux kernel mailing thread to understand why a similar change wasn't made for the lr instruction ( lr -> lr.aq or lr.aqrl as it was before ).
Does that make sense to you?
[2]
https://lore.kernel.org/linux-riscv/[email protected]/
[3]https://lore.kernel.org/linux-riscv/YrSo%2F3iUuO0AL76T@boqun-archlinux/
~ Oleksii
Quoted 7 lines
~ Oleksii
Quoted 5 lines
Jan
Quoted 2 lines
[1]
https://lore.kernel.org/lkml/[email protected]
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
/
~ Oleksii
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
In reply to OleksiiQuoted 20 lines · Oleksii
Quoted 19 lines · Jan Beulich
Quoted 18 lines · Oleksii
Quoted 17 lines · Jan Beulich
Quoted 4 lines · Oleksii Kurochko
+ */
+staticalways_inlinevoid read_atomic_size(const volatile void *p,
+ void *res,Quoted 8 lines
+ unsigned int size) +{
+ switch ( size )
+ {
+ case 1: *(uint8_t *)res = readb(p); break;
+ case 2: *(uint16_t *)res = readw(p); break;
+ case 4: *(uint32_t *)res = readl(p); break;
+ case 8: *(uint32_t *)res = readq(p); break;Nit: Excess blank before =.
Also - no #ifdef here to be RV32-ready?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Because there is #ifdef RV32 in
io.hforreadq().There you'd run into __raw_readq()'s
BUILD_BUG_ON(), afaict even for 1-, 2-, or 4-byte accesses. That's not quite what we want here.Do you mean that if someone will redefine
readq()in another way and not wrap it by #ifdef RV32? Except this I am not sure that there is an issue as it will be still a compilation error, so anyway it will be needed to provide an implementation for__raw_readq().
No. BUILD_BUG_ON() is a compile-time thing. The compiler will encounter this construct. And hence it will necessarily fail. Which is why the other approach (causing a linker error) is used elsewhere. And we're still only in the course of considering to utilize DCE for something like STATIC_ASSERT_UNREACHABLE(); iirc there was something getting in the way there.
Quoted 16 lines
One of the reason why I decided to wrap with #ifdef RV32 in
io.hto not go over the source code and add wrapping. Also for some code it will be needed to rewrite it. For example, I am not sure that I can add #ifdef inside macros, f.e.:#define write_atomic(p, x) \ ({ \ typeof(*(p)) x__ = (x); \ switch ( sizeof(*(p)) ) \ { \ case 1: writeb(x__, p); break; \ case 2: writew(x__, p); break; \ case 4: writel(x__, p); break; \ case 8: writeq(x__, p); break; \ default: __bad_atomic_size(); break; \ } \ x__; \
Quoted 1 line
})
You can't add #ifdef there. Such needs abstracting differently.
But of course there's the option of simply not making any of these constructs RV32-ready. Yet if so, that then will want doing consistently.
Quoted 71 lines
Quoted 58 lines
Quoted 43 lines
Quoted 35 lines
Quoted 8 lines
+/*
+ * First, the atomic ops that have no ordering constraints and therefor don't
+ * have the AQ or RL bits set. These don't return anything, so
there's only
+ * one version to worry about.
+ */xen/arch/riscv/include/asm/atomic.h line 103
Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h new file +8
-0+#define ATOMIC_OP (op, asm_op,I , asm_type, c_type, prefix) \ +static inline \ +void atomic##prefix##_##op(c_type i, atomic##prefix##_t *v) \ +{ \ + asm volatile ( \ + " amo" #asm_op "." #asm_type " zero, %1, %0" \ + : "+A" (v->counter) \ + : "r" (I) \Btw, I consider this pretty confusing. At the 1st and 2nd glance this
looks like a mistake, i.e. as if i was meant. Imo ...Quoted 14 lines
+ : "memory" ); \
xen/arch/riscv/include/asm/atomic.h new file +4
-0+} \ + +/* +* Only CONFIG_GENERIC_ATOMIC64=y was ported to Xen that is the
reason why
+ * last argument forATOMIC_OPisn't used.
+ */xen/arch/riscv/include/asm/atomic.h new file +5
-0+#define ATOMIC_OPS (op, asm_op,I ) \ + ATOMIC_OP (op, asm_op, I, w, int, ) + +ATOMIC_OPS(add, add, i) +ATOMIC_OPS (sub, add, -i)Quoted 8 lines
xen/arch/riscv/include/asm/atomic.h line 118
Quoted 3 lines
xen/arch/riscv/include/asm/atomic.h new file +3
-0+ATOMIC_OPS(and, and, i) +ATOMIC_OPS( or, or, i) +ATOMIC_OPS(xor, xor, i)... here you want to only pass the (unary) operator (and leaving that
blank
is as fine as using +).Quoted 14 lines
I agree that a game with 'i' and 'I' looks confusing, but I am not really understand what is wrong with using ' i' here. It seems that preprocessed macros looks fine:
static inline voidatomic_add(int i, atomic_t *v){ asm volatile ( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (i) :
"memory" ); }
static inline voidatomic_sub(int i, atomic_t *v){ asm volatile ( "
amo" "add" "." "w" " zero, %1, %0" : "+A" (v->counter) : "r" (-
i) :
"memory" ); }Quoted 13 lines
I didn't question the pre-processed result being correct. Instead I said
that I consider the construct confusing to the reader, for looking as if
there was a mistake (in the case of the letter i used). Note also in particular how the macro invocations need to be in sync with the macro
implementation, for lower case i being used both in the macro and in its
invocations. Anything parameterized would better be fully so, at the very least to avoid, as said, confusion. (Having macros depend on context at their use sites _may_ be okay for local helper macros, but here we're talking about a not even private header file.)
Quoted 2 lines
I am not sure then I understand how mentioning '+i' will help significantly remove confusion.
I'm afraid I don't understand: What "mentioning '+i'" are you referring to? I'm pretty sure I didn't suggest that. What I suggested was to pass a bare operator (- or +) as macro argument.
Quoted 33 lines
Quoted 33 lines
Quoted 32 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/riscv/include/asm/atomic.h new file +6
-0+/* This is required to provide a full barrier on success. */ +static inline int atomic_add_unless (atomic_t *v,int a,int u) +{ + int prev, rc; + + asm volatile (Quoted 8 lines
Quoted 6 lines
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n"
+ " sc.w.rl %[rc], %[rc], %[c]\n"
+ " bnez %[rc], 0b\n"
+RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring
to the spec:
Ordering Annotation Fence-based Equivalentl{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c} s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}amo<op>.aq amo<op>; fence r,rw
amo<op>.rl fence rw,w; amo<op>
amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existings{b|h|w|d|c} mapping rather than at the front of the l{b|h|w|d|r} mapping.
Quoted 6 lines
Quoted 2 lines
I'm afraid I can't spot the specific case in this table. None of the stores in the right column have a .rl suffix.
Yes, it is expected.
I am reading this table as (f.e.) amo<op>.rl is an equivalent of fence rw,w; amo<op>. (without .rl)
In which case: How does quoting the table answer my original question?
Quoted 35 lines
Quoted 13 lines
Quoted 13 lines
It is also safe to translate any .aq, .rl, or .aqrl annotation into
the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added to
the ISA.So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.Quoted 5 lines
Quoted 3 lines
Considering [1] and how this code looks before, it seems to me that it
is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.Here you say "or". Then why dos the code use sc.?.rl _and_ a fence?
I confused this line with amo<op>.aqrl, so based on the table 2.2 above s{b|h|w|d|c}.aqrl is transformed to "fence rw,w; s{b|h|w|d|c}", but Linux kernel decided to strengthen it with full barrier:
- "0:\n\t" - "lr.w.aqrl %[p], %[c]\n\t" - "beq %[p], %[u], 1f\n\t" - "add %[rc], %[p], %[a]\n\t" - "sc.w.aqrl %[rc], %[rc], %[c]\n\t" - "bnez %[rc], 0b\n\t" - "1:" + "0: lr.w %[p], %[c]\n" + " beq %[p], %[u], 1f\n" + " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n" + " fence rw, rw\n" + "1:\n"
Quoted 6 lines
As they have the following issue:
implementations of atomics such asatomic_cmpxchg()andatomic_add_unless()rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test below to end up with the state indicated in the "exists" clause.
Is that really "current implementations", not "the abstract model"? If so, the use of an explicit fence would be more like a workaround (and would hence want commenting to that effect).
In neither case can I see my original question answered: Why the .rl on the store when there is a full fence later?
Jan
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
On Wed, 2024-03-27 at 08:40 +0100, Jan Beulich wrote:
...
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
Quoted 33 lines
Quoted 8 lines
Quoted 8 lines
+/* This is required to provide a full barrier on success. */
+static inline intatomic_add_unless(atomic_t *v, int a, int u)xen/arch/riscv/include/asm/atomic.h new file +4
-0+{ + int prev, rc; + + asm volatile (Quoted 8 lines
Quoted 6 lines
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n"
+ " sc.w.rl %[rc], %[rc], %[c]\n"
+ " bnez %[rc], 0b\n"
+RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring
to the spec:
Ordering Annotation Fence-based Equivalentl{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c} s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}amo<op>.aq amo<op>; fence r,rw amo<op>.rl fence rw,w; amo<op> amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existingQuoted 2 lines
s{b|h|w|d|c} mapping rather than at the front of the l{b|h|w|d|r} mapping.I'm afraid I can't spot the specific case in this table. None of the
stores in the right column have a .rl suffix.
Quoted 8 lines
Quoted 5 lines
Yes, it is expected.
I am reading this table as (f.e.) amo<op>.rl is an equivalent of fence
rw,w; amo<op>. (without .rl)In which case: How does quoting the table answer my original question?
Agree, it is starting to be confusing, so let me give an answer to your question below.
Quoted 49 lines
Quoted 40 lines
Quoted 17 lines
Quoted 17 lines
It is also safe to translate any .aq, .rl, or .aqrl annotation
into
the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of
l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added
to
the ISA.So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.Quoted 7 lines
Quoted 4 lines
Considering [1] and how this code looks before, it seems to me that
it
is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.Here you say "or". Then why dos the code use sc.?.rl _and_ a fence?
I confused this line with amo<op>.aqrl, so based on the table 2.2 above
s{b|h|w|d|c}.aqrl is transformed to "fence rw,w; s{b|h|w|d|c}", but Linux kernel decided to strengthen it with full barrier:
- "0:\n\t"
- "lr.w.aqrl %[p], %[c]\n\t"
- "beq %[p], %[u], 1f\n\t"
- "add %[rc], %[p], %[a]\n\t"
- "sc.w.aqrl %[rc], %[rc], %[c]\n\t"
- "bnez %[rc], 0b\n\t"
- "1:"
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n"Quoted 8 lines
+ " fence rw, rw\n"
+ "1:\n"
As they have the following issue:
implementations of atomics such asatomic_cmpxchg()andatomic_add_unless()rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test below to end up with the state indicated in the "exists" clause.
Quoted 6 lines
Is that really "current implementations", not "the abstract model"? If so, the use of an explicit fence would be more like a workaround (and would hence want commenting to that effect).
In neither case can I see my original question answered: Why the .rl on the store when there is a full fence later?
The good explanation for that was provided in the commit addressing a similar issue for ARM64 [
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/
].
The same holds true for RISC-V since ARM also employs WMO.
I would also like to mention another point, as I indicated in another email thread
[ https://lists.xen.org/archives/html/xen-devel/2024-03/msg01582.html ] , that now this fence can be omitted and .aqrl can be used instead.
This was confirmed by Dan (the author of the RVWMO spec)
[https://lore.kernel.org/linux-riscv/[email protected]/
]
I hope this addresses your original question. Does it?
~ Oleksii
Re: [PATCH v6 10/20] xen/riscv: introduce atomic.h
Quoted 119 lines · Oleksii
On Wed, 2024-03-27 at 08:40 +0100, Jan Beulich wrote:
...Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
Quoted 33 lines
Quoted 8 lines
Quoted 8 lines
+/* This is required to provide a full barrier on success. */
+static inline intatomic_add_unless(atomic_t *v, int a, int u)(quoted diff) +4
-0+{ + int prev, rc; + + asm volatile (Quoted 8 lines
Quoted 6 lines
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n"
+ " sc.w.rl %[rc], %[rc], %[c]\n"
+ " bnez %[rc], 0b\n"
+RISCV_FULL_BARRIERWith this and no .aq on the load, why the .rl on the store?
It is something that LKMM requires [1].
This is not fully clear to me what is so specific in LKMM, but accoring
to the spec:
Ordering Annotation Fence-based Equivalentl{b|h|w|d|r}.aq l{b|h|w|d|r}; fence r,rw l{b|h|w|d|r}.aqrl fence rw,rw; l{b|h|w|d|r}; fence r,rw s{b|h|w|d|c}.rl fence rw,w; s{b|h|w|d|c} s{b|h|w|d|c}.aqrl fence rw,w; s{b|h|w|d|c}amo<op>.aq amo<op>; fence r,rw amo<op>.rl fence rw,w; amo<op> amo<op>.aqrl fence rw,rw; amo<op>; fence rw,rw Table 2.2: Mappings from .aq and/or .rl to fence-based equivalents.
An alternative mapping places a fence rw,rw after the existingQuoted 2 lines
s{b|h|w|d|c} mapping rather than at the front of the l{b|h|w|d|r} mapping.I'm afraid I can't spot the specific case in this table. None of the
stores in the right column have a .rl suffix.Quoted 8 lines
Quoted 5 lines
Yes, it is expected.
I am reading this table as (f.e.) amo<op>.rl is an equivalent of fence
rw,w; amo<op>. (without .rl)In which case: How does quoting the table answer my original question?
Agree, it is starting to be confusing, so let me give an answer to your question below.
Quoted 49 lines
Quoted 40 lines
Quoted 17 lines
Quoted 17 lines
It is also safe to translate any .aq, .rl, or .aqrl annotation
into
the fence-based snippets of
Table 2.2. These can also be used as a legal implementation of
l{b|h|w|d} or s{b|h|w|d} pseu-
doinstructions for as long as those instructions are not added
to
the ISA.So according to the spec, it should be:
sc.w ...
RISCV_FULL_BARRIER.Quoted 7 lines
Quoted 4 lines
Considering [1] and how this code looks before, it seems to me that
it
is safe to use lr.w.aq/sc.w.rl here or an fence equivalent.Here you say "or". Then why dos the code use sc.?.rl _and_ a fence?
I confused this line with amo<op>.aqrl, so based on the table 2.2 above
s{b|h|w|d|c}.aqrl is transformed to "fence rw,w; s{b|h|w|d|c}", but Linux kernel decided to strengthen it with full barrier:
- "0:\n\t"
- "lr.w.aqrl %[p], %[c]\n\t"
- "beq %[p], %[u], 1f\n\t"
- "add %[rc], %[p], %[a]\n\t"
- "sc.w.aqrl %[rc], %[rc], %[c]\n\t"
- "bnez %[rc], 0b\n\t"
- "1:"
+ "0: lr.w %[p], %[c]\n"
+ " beq %[p], %[u], 1f\n"
+ " add %[rc], %[p], %[a]\n" + " sc.w.rl %[rc], %[rc], %[c]\n" + " bnez %[rc], 0b\n"Quoted 8 lines
+ " fence rw, rw\n"
+ "1:\n"
As they have the following issue:
implementations of atomics such asatomic_cmpxchg()andatomic_add_unless()rely on LR/SC pairs,
which do not give full-ordering with .aqrl; for example, current implementations allow the "lr-sc-aqrl-pair-vs-full-barrier" test below to end up with the state indicated in the "exists" clause.Quoted 6 lines
Is that really "current implementations", not "the abstract model"? If so, the use of an explicit fence would be more like a workaround (and would hence want commenting to that effect).
In neither case can I see my original question answered: Why the .rl on the store when there is a full fence later?
The good explanation for that was provided in the commit addressing a similar issue for ARM64 [
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/
].
The same holds true for RISC-V since ARM also employs WMO.I would also like to mention another point, as I indicated in another email thread
[ https://lists.xen.org/archives/html/xen-devel/2024-03/msg01582.html ] , that now this fence can be omitted and .aqrl can be used instead.
Quoted 5 lines
This was confirmed by Dan (the author of the RVWMO spec)
[https://lore.kernel.org/linux-riscv/[email protected]/
]I hope this addresses your original question. Does it?
I think it does, thanks. Some of this will need putting in at least the patch description, if not a code comment.
Jan
PATCH v6 11/20 xen/riscv: introduce monitor.h
In reply to Oleksii Kurochko---
Changes in V4/V5/V6:
- Nothing changed. Only rebase.
---
Changes in V3:
- new patch.
--- xen/arch/riscv/include/asm/monitor.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 xen/arch/riscv/include/asm/monitor.h
xen/arch/riscv/include/asm/monitor.h new file +26 -0
@@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_RISCV_MONITOR_H__ +#define __ASM_RISCV_MONITOR_H__ + +#include <xen/bug.h> + +#include <asm-generic/monitor.h> + +struct domain ; + +static inline uint32_t arch_monitor_get_capabilities (struct domain *d) +{ +BUG_ON ("unimplemented" ); +return 0 ; +} + +#endif /* __ASM_RISCV_MONITOR_H__ */ + +/* +* Local variables: +* mode: C +* c-file-style: "BSD" +* c-basic-offset: 4 +* indent-tabs-mode: nil +* End: +*/
Signature · 1 line
2.43.0
PATCH v6 12/20 xen/riscv: add definition of __read_mostly
In reply to Oleksii KurochkoThe definition of __read_mostly should be removed in:
https://lore.kernel.org/xen-devel/[email protected]/
The patch introduces it in arch-specific header to not block enabling of full Xen build for RISC-V.
---
- [PATCH] move __read_mostly to xen/cache.h [2]
Right now, the patch series doesn't have a direct dependency on [2] and it provides __read_mostly in the patch:
[PATCH v3 26/34] xen/riscv: add definition of __read_mostly
However, it will be dropped as soon as [2] is merged or at least when the final version of the patch [2] is provided.
[2] https://lore.kernel.org/xen-devel/[email protected]/
---
Changes in V4-V7:
- Nothing changed. Only rebase.
--- xen/arch/riscv/include/asm/cache.h | 2 ++ 1 file changed, 2 insertions(+)
xen/arch/riscv/include/asm/cache.h +2 -0
@@ -3,4 +3,6 @@#ifndef _ASM_RISCV_CACHE_H#define _ASM_RISCV_CACHE_H +#define __read_mostly __section(".data.read_mostly") +#endif /* _ASM_RISCV_CACHE_H */
Signature · 1 line
2.43.0
PATCH v6 13/20 xen/riscv: add required things to current.h
In reply to Oleksii KurochkoAdd minimal requied things to be able to build full Xen.
---
Changes in V5/V6:
- Nothing changed. Only rebase.
---
Changes in V4:
- BUG() was changed to BUG_ON("unimplemented");
- Change "xen/bug.h" to "xen/lib.h" as BUG_ON is defined in xen/lib.h.
- Add Acked-by: Jan Beulich <[email protected]>
---
Changes in V3:
- add SPDX
- drop a forward declaration of struct vcpu;
- update guest_cpu_user_regs() macros
- replace get_processor_id with smp_processor_id
- update the commit message
- code style fixes
---
Changes in V2:
- Nothing changed. Only rebase.
--- xen/arch/riscv/include/asm/current.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
xen/arch/riscv/include/asm/current.h +19 -0
@@ -3,6 +3,21 @@#ifndef __ASM_CURRENT_H#define __ASM_CURRENT_H +#include <xen/lib.h> +#include <xen/percpu.h> +#include <asm/processor.h> + +#ifndef __ASSEMBLY__ + +/* Which VCPU is "current" on this PCPU. */ +DECLARE_PER_CPU(struct vcpu *, curr_vcpu); + +#define current this_cpu(curr_vcpu) +#define set_current (vcpu) do { current = (vcpu); } while (0) +#define get_cpu_current (cpu) per_cpu(curr_vcpu, cpu) + +#define guest_cpu_user_regs () ({ BUG_ON("unimplemented"); NULL; }) +#define switch_stack_and_jump (stack, fn) do { \ asm volatile ( \ "mv sp, %0\n" \
@@ -10,4 +25,8 @@unreachable (); \ } while ( false ) +#define get_per_cpu_offset () __per_cpu_offset[smp_processor_id()] + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_CURRENT_H */
Signature · 1 line
2.43.0
PATCH v6 14/20 xen/riscv: add minimal stuff to page.h to build full Xen
In reply to Oleksii Kurochko---
Changes in V5-6:
- Nothing changed. Only rebase.
---
Changes in V4:
---
- Change message -> subject in "Changes in V3"
- s/BUG/BUG_ON("...")
- Do proper rebase ( pfn_to_paddr() and paddr_to_pfn() aren't removed ).
---
Changes in V3:
- update the commit subject
- add implemetation of PAGE_HYPERVISOR macros
- add Acked-by: Jan Beulich <[email protected]>
- drop definition of pfn_to_addr, and paddr_to_pfn in <asm/mm.h>
---
Changes in V2:
- Nothing changed. Only rebase.
--- xen/arch/riscv/include/asm/page.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
xen/arch/riscv/include/asm/page.h +19 -0
@@ -6,6 +6,7 @@#ifndef __ASSEMBLY__#include <xen/const.h> +#include <xen/bug.h> #include <xen/types.h> #include <asm/mm.h>
@@ -32,6 +33,10 @@#define PTE_LEAF_DEFAULT (PTE_VALID | PTE_READABLE | PTE_WRITABLE)#define PTE_TABLE (PTE_VALID) +#define PAGE_HYPERVISOR_RW (PTE_VALID | PTE_READABLE | PTE_WRITABLE) + +#define PAGE_HYPERVISOR PAGE_HYPERVISOR_RW +/* Calculate the offsets into the pagetables for a given VA */ #define pt_linear_offset (lvl, va) ((va) >> XEN_PT_LEVEL_SHIFT(lvl))
@@ -62,6 +67,20 @@ static inline bool pte_is_valid(pte_t p)return p.pte &PTE_VALID ; } +static inline void invalidate_icache (void ) +{ +BUG_ON ("unimplemented" ); +} + +#define clear_page (page) memset((void *)(page), 0, PAGE_SIZE) +#define copy_page (dp, sp) memcpy(dp, sp, PAGE_SIZE) + +/* TODO: Flush the dcache for an entire page. */ +static inline void flush_page_to_ram (unsigned long mfn,bool sync_icache) +{ +BUG_ON ("unimplemented" ); +} +#endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_PAGE_H */
Signature · 1 line
2.43.0
PATCH v6 15/20 xen/riscv: add minimal stuff to processor.h to build full Xen
In reply to Oleksii KurochkoThe cpu_relax() function, introduced in this commit, is anticipated to support _zihintpause by the CPU.
---
Changes in V6:
- drop incorrect part in riscv/booting.txt and move the introduction of it to
separate patch.
- compiler check that __riscv_zihintpause exist was droped to separate patch.
- minor fixes.
- drop unnecessary comment.
- update the commit message.
---
Changes in V5:
- Code style fixes.
- drop introduced TOOLCHAIN_HAS_ZIHINTPAUSE and use as-insn instead and use
as-insn istead.
---
Changes in V4:
- Change message -> subject in "Changes in V3"
- Documentation about system requirement was added. In the future, it can be checked if the extension is supported
by system __riscv_isa_extension_available() ( https://gitlab.com/xen-project/people/olkur/xen/-/commit/737998e89ed305eb92059300c374dfa53d2143fa )
- update cpu_relax() function to check if __riscv_zihintpause is supported by a toolchain
- add conditional _zihintpause to -march if it is supported by a toolchain
Changes in V3:
- update the commit subject
- rename get_processor_id to smp_processor_id
- code style fixes
- update the cpu_relax instruction: use pause instruction instead of div %0, %0, zero
---
Changes in V2:
- Nothing changed. Only rebase.
--- xen/arch/riscv/include/asm/processor.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
xen/arch/riscv/include/asm/processor.h +20 -0
@@ -12,6 +12,9 @@#ifndef __ASSEMBLY__ +/* TODO: need to be implemeted */ +#define smp_processor_id () 0 +/* On stack VCPU state */ struct cpu_user_regs {
@@ -53,6 +56,23 @@ struct cpu_user_regsunsigned long pregs; }; +/* TODO: need to implement */ +#define cpu_to_core (cpu) 0 +#define cpu_to_socket (cpu) 0 + +static inline void cpu_relax (void ) +{ +#ifdef __riscv_zihintpause +/* Reduce instruction retirement. */ + __asm__ __volatile__ ("pause" ); +#else +/* Encoding of the pause instruction */ + __asm__ __volatile__ (".insn 0x0100000F" ); +#endif + +barrier (); +} +static inline void wfi (void ) { __asm__ __volatile__ ("wfi" );
Signature · 1 line
2.43.0
Re: [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen
Quoted 4 lines · Oleksii Kurochko
The
cpu_relax()function, introduced in this commit, is anticipated to support_zihintpauseby the CPU.Signed-off-by Oleksii Kurochko <[email protected]>
Looks like this can go in ahead of the other 14 patches?
Jan
Re: [PATCH v6 15/20] xen/riscv: add minimal stuff to processor.h to build full Xen
Quoted 8 lines · Jan Beulich
Quoted 5 lines · Oleksii Kurochko
The
cpu_relax()function, introduced in this commit, is anticipated to
support_zihintpauseby the CPU.Signed-off-by Oleksii Kurochko <[email protected]>Acked-by Jan Beulich <[email protected]>
Thanks.
Quoted 2 lines
Looks like this can go in ahead of the other 14 patches?
Yes, it can. I'll rebase it ahead.
~ Oleksii
PATCH v6 16/20 xen/riscv: add minimal stuff to mm.h to build full Xen
In reply to Oleksii Kurochko---
Changes in V6:
- drop __virt_to_maddr() ( transform to macro ) and __maddr_to_virt ( rename to maddr_to_virt ).
- parenthesize va in definition of vmap_to_mfn().
- Code style fixes.
---
Changes in V5:
- update the comment around "struct domain *domain;" : zero -> NULL
- fix ident. for unsigned long val;
- put page_to_virt() and virt_to_page() close to each other.
- drop unnessary leading underscore
- drop a space before the comment: /* Count of uses of this frame as its current type. */
- drop comment about a page 'not as a shadow'. it is not necessary for RISC-V
---
Changes in V4:
- update an argument name of PFN_ORDERN macros.
- drop pad at the end of 'struct page_info'.
- Change message -> subject in "Changes in V3"
- delete duplicated macros from riscv/mm.h
- fix identation in struct page_info
- align comment for PGC_ macros
- update definitions of domain_set_alloc_bitsize() and domain_clamp_alloc_bitsize()
- drop unnessary comments.
- s/BUG/BUG_ON("...")
- define __virt_to_maddr, __maddr_to_virt as stubs
- add inclusion of xen/mm-frame.h for mfn_x and others
- include "xen/mm.h" instead of "asm/mm.h" to fix compilation issues:
In file included from arch/riscv/setup.c:7:
./arch/riscv/include/asm/mm.h:60:28: error: field 'list' has incomplete type
60 | struct page_list_entry list;
| ^~~~
./arch/riscv/include/asm/mm.h:81:43: error: 'MAX_ORDER' undeclared here (not in a function)
81 | unsigned long first_dirty:MAX_ORDER + 1;
| ^~~~~~~~~
./arch/riscv/include/asm/mm.h:81:31: error: bit-field 'first_dirty' width not an integer constant
81 | unsigned long first_dirty:MAX_ORDER + 1;
- Define __virt_to_mfn() and __mfn_to_virt() using maddr_to_mfn() and mfn_to_maddr().
---
Changes in V3:
- update the commit title
- introduce DIRECTMAP_VIRT_START.
- drop changes related pfn_to_paddr() and paddr_to_pfn as they were remvoe in
[PATCH v2 32/39] xen/riscv: add minimal stuff to asm/page.h to build full Xen
- code style fixes.
- drop get_page_nr and put_page_nr as they don't need for time being
- drop CONFIG_STATIC_MEMORY related things
- code style fixes
---
Changes in V2:
- define stub for arch_get_dma_bitsize(void)
--- xen/arch/riscv/include/asm/mm.h | 240 ++++++++++++++++++++++++++++++++ xen/arch/riscv/mm.c | 2 +- xen/arch/riscv/setup.c | 2 +- 3 files changed, 242 insertions(+), 2 deletions(-)
xen/arch/riscv/include/asm/mm.h +240 -0
@@ -3,11 +3,246 @@#ifndef _ASM_RISCV_MM_H#define _ASM_RISCV_MM_H +#include <public/xen.h> +#include <xen/bug.h> +#include <xen/mm-frame.h> +#include <xen/pdx.h> +#include <xen/types.h> +#include <asm/page-bits.h> #define pfn_to_paddr (pfn) ((paddr_t)(pfn) << PAGE_SHIFT)#define paddr_to_pfn (pa) ((unsigned long)((pa) >> PAGE_SHIFT)) +#define paddr_to_pdx (pa) mfn_to_pdx(maddr_to_mfn(pa)) +#define gfn_to_gaddr (gfn) pfn_to_paddr(gfn_x(gfn)) +#define gaddr_to_gfn (ga) _gfn(paddr_to_pfn(ga)) +#define mfn_to_maddr (mfn) pfn_to_paddr(mfn_x(mfn)) +#define maddr_to_mfn (ma) _mfn(paddr_to_pfn(ma)) +#define vmap_to_mfn (va) maddr_to_mfn(virt_to_maddr((vaddr_t)(va))) +#define vmap_to_page (va) mfn_to_page(vmap_to_mfn(va)) + +static inline void *maddr_to_virt (unsigned long ma) +{ +BUG_ON ("unimplemented" ); +return NULL ; +} + +#define virt_to_maddr (va) ({ BUG_ON("unimplemented"); 0; }) + +/* Convert between Xen-heap virtual addresses and machine frame numbers. */ +#define __virt_to_mfn (va) mfn_x(maddr_to_mfn(virt_to_maddr(va))) +#define __mfn_to_virt (mfn) maddr_to_virt(mfn_to_maddr(_mfn(mfn))) + +/* +* We define non-underscored wrappers for above conversion functions. +* These are overriden in various source files while underscored version +* remain intact. +*/ +#define virt_to_mfn (va) __virt_to_mfn(va) +#define mfn_to_virt (mfn) __mfn_to_virt(mfn) + +struct page_info +{ +/* Each frame can be threaded onto a doubly-linked list. */ +struct page_list_entry list; + +/* Reference count and various PGC_xxx flags and fields. */ +unsigned long count_info; + +/* Context-dependent fields follow... */ +union { +/* Page is in use: ((count_info & PGC_count_mask) != 0). */ +struct { +/* Type reference count and various PGT_xxx flags and fields. */ +unsigned long type_info; + } inuse; + +/* Page is on a free list: ((count_info & PGC_count_mask) == 0). */ +union { +struct { +/* +* Index of the first *possibly* unscrubbed page in the buddy. +* One more bit than maximum possible order to accommodate +* INVALID_DIRTY_IDX. +*/ +#define INVALID_DIRTY_IDX ((1UL << (MAX_ORDER + 1)) - 1) +unsigned long first_dirty:MAX_ORDER +1 ; + +/* Do TLBs need flushing for safety before next page use? */ +bool need_tlbflush:1 ; + +#define BUDDY_NOT_SCRUBBING 0 +#define BUDDY_SCRUBBING 1 +#define BUDDY_SCRUB_ABORT 2 +unsigned long scrub_state:2 ; + }; + +unsigned long val; + } free; + } u; + +union { +/* Page is in use */ +struct { +/* Owner of this page (NULL if page is anonymous). */ +struct domain *domain; + } inuse; + +/* Page is on a free list. */ +struct { +/* Order-size of the free chunk this page is the head of. */ +unsigned int order; + } free; + } v; + +union { +/* +* Timestamp from 'TLB clock', used to avoid extra safety flushes. +* Only valid for: a) free pages, and b) pages with zero type count +*/ +uint32_t tlbflush_timestamp; + }; +}; + +#define frame_table ((struct page_info *)FRAMETABLE_VIRT_START) + +/* PDX of the first page in the frame table. */ +extern unsigned long frametable_base_pdx; + +/* Convert between machine frame numbers and page-info structures. */ +#define mfn_to_page (mfn) \ + (frame_table + (mfn_to_pdx(mfn) - frametable_base_pdx)) +#define page_to_mfn (pg) \ + pdx_to_mfn((unsigned long)((pg) - frame_table) + frametable_base_pdx) + +static inline void *page_to_virt (const struct page_info *pg) +{ +return mfn_to_virt (mfn_x (page_to_mfn (pg))); +} + +/* Convert between Xen-heap virtual addresses and page-info structures. */ +static inline struct page_info *virt_to_page (const void *v) +{ +BUG_ON ("unimplemented" ); +return NULL ; +} + +/* +* Common code requires get_page_type and put_page_type. +* We don't care about typecounts so we just do the minimum to make it +* happy. +*/ +static inline int get_page_type (struct page_info *page,unsigned long type) +{ +return 1 ; +} + +static inline void put_page_type (struct page_info *page) +{ +} + +static inline void put_page_and_type (struct page_info *page) +{ +put_page_type (page); +put_page (page); +} + +/* +* RISC-V does not have an M2P, but common code expects a handful of +* M2P-related defines and functions. Provide dummy versions of these. +*/ +#define INVALID_M2P_ENTRY (~0UL) +#define SHARED_M2P_ENTRY (~0UL - 1UL) +#define SHARED_M2P (_e) ((_e) == SHARED_M2P_ENTRY) + +#define set_gpfn_from_mfn (mfn, pfn) do { (void)(mfn), (void)(pfn); } while (0) +#define mfn_to_gfn (d, mfn) ((void)(d), _gfn(mfn_x(mfn))) + +#define PDX_GROUP_SHIFT (PAGE_SHIFT + VPN_BITS) + +static inline unsigned long domain_get_maximum_gpfn (struct domain *d) +{ +BUG_ON ("unimplemented" ); +return 0 ; +} + +static inline long arch_memory_op (int op,XEN_GUEST_HANDLE_PARAM (void ) arg) +{ +BUG_ON ("unimplemented" ); +return 0 ; +} + +/* +* On RISCV, all the RAM is currently direct mapped in Xen. +* Hence return always true. +*/ +static inline bool arch_mfns_in_directmap (unsigned long mfn,unsigned long nr) +{ +return true; +} + +#define PG_shift (idx) (BITS_PER_LONG - (idx)) +#define PG_mask (x, idx) (x ## UL << PG_shift(idx)) + +#define PGT_none PG_mask(0, 1)/* no special uses of this page */ +#define PGT_writable_page PG_mask(1, 1)/* has writable mappings? */ +#define PGT_type_mask PG_mask(1, 1)/* Bits 31 or 63. */ + +/* Count of uses of this frame as its current type. */ +#define PGT_count_width PG_shift(2) +#define PGT_count_mask ((1UL << PGT_count_width) - 1) + +/* +* Page needs to be scrubbed. Since this bit can only be set on a page that is +* free (i.e. in PGC_state_free) we can reuse PGC_allocated bit. +*/ +#define _PGC_need_scrub _PGC_allocated +#define PGC_need_scrub PGC_allocated + +/* Cleared when the owning guest 'frees' this page. */ +#define _PGC_allocated PG_shift(1) +#define PGC_allocated PG_mask(1, 1) +/* Page is Xen heap? */ +#define _PGC_xen_heap PG_shift(2) +#define PGC_xen_heap PG_mask(1, 2) +/* Page is broken? */ +#define _PGC_broken PG_shift(7) +#define PGC_broken PG_mask(1, 7) +/* Mutually-exclusive page states: { inuse, offlining, offlined, free }. */ +#define PGC_state PG_mask(3, 9) +#define PGC_state_inuse PG_mask(0, 9) +#define PGC_state_offlining PG_mask(1, 9) +#define PGC_state_offlined PG_mask(2, 9) +#define PGC_state_free PG_mask(3, 9) +#define page_state_is (pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st) + +/* Count of references to this frame. */ +#define PGC_count_width PG_shift(9) +#define PGC_count_mask ((1UL << PGC_count_width) - 1) + +#define _PGC_extra PG_shift(10) +#define PGC_extra PG_mask(1, 10) + +#define is_xen_heap_page (page) ((page)->count_info & PGC_xen_heap) +#define is_xen_heap_mfn (mfn) \ + (mfn_valid(mfn) && is_xen_heap_page(mfn_to_page(mfn))) + +#define is_xen_fixed_mfn (mfn) \ + ((mfn_to_maddr(mfn) >= virt_to_maddr((vaddr_t)_start)) && \ + (mfn_to_maddr(mfn) <= virt_to_maddr((vaddr_t)_end - 1))) + +#define page_get_owner (p) (p)->v.inuse.domain +#define page_set_owner (p, d) ((p)->v.inuse.domain = (d)) + +/* TODO: implement */ +#define mfn_valid (mfn) ({ (void)(mfn); 0; }) + +#define domain_set_alloc_bitsize (d) ((void)(d)) +#define domain_clamp_alloc_bitsize (d, b) ((void)(d), (b)) + +#define PFN_ORDER (pfn) ((pfn)->v.free.order) +extern unsigned char cpu0_boot_stack[];void setup_initial_pagetables (void );
@@ -20,4 +255,9 @@ unsigned long calc_phys_offset(void);void turn_on_mmu (unsigned long ra); +static inline unsigned int arch_get_dma_bitsize (void ) +{ +return 32 ;/* TODO */ +} +#endif /* _ASM_RISCV_MM_H */
xen/arch/riscv/mm.c +1 -1
@@ -5,12 +5,12 @@#include <xen/init.h> #include <xen/kernel.h> #include <xen/macros.h> +#include <xen/mm.h> #include <xen/pfn.h> #include <asm/early_printk.h> #include <asm/csr.h> #include <asm/current.h> -#include <asm/mm.h> #include <asm/page.h> #include <asm/processor.h>
xen/arch/riscv/setup.c +1 -1
@@ -2,9 +2,9 @@#include <xen/compile.h> #include <xen/init.h> +#include <xen/mm.h> #include <asm/early_printk.h> -#include <asm/mm.h> /* Xen stack for bringing up the first CPU. */ unsigned char __initdata cpu0_boot_stack[STACK_SIZE ]
Signature · 1 line
2.43.0
Re: [PATCH v6 16/20] xen/riscv: add minimal stuff to mm.h to build full Xen
Quoted 16 lines · Oleksii Kurochko
xen/arch/riscv/include/asm/mm.h +6
-0@@ -3,11 +3,246 @@#ifndef _ASM_RISCV_MM_H#define _ASM_RISCV_MM_H +#include <public/xen.h> +#include <xen/bug.h> +#include <xen/mm-frame.h> +#include <xen/pdx.h> +#include <xen/types.h> +#include <asm/page-bits.h> #define pfn_to_paddr (pfn) ((paddr_t)(pfn) << PAGE_SHIFT)#define paddr_to_pfn (pa) ((unsigned long)((pa) >> PAGE_SHIFT))
xen/arch/riscv/include/asm/mm.h line 25
Quoted 10 lines
xen/arch/riscv/include/asm/mm.h +9
-0+#define paddr_to_pdx (pa) mfn_to_pdx(maddr_to_mfn(pa)) +#define gfn_to_gaddr (gfn) pfn_to_paddr(gfn_x(gfn)) +#define gaddr_to_gfn (ga) _gfn(paddr_to_pfn(ga)) +#define mfn_to_maddr (mfn) pfn_to_paddr(mfn_x(mfn)) +#define maddr_to_mfn (ma) _mfn(paddr_to_pfn(ma)) +#define vmap_to_mfn (va) maddr_to_mfn(virt_to_maddr((vaddr_t)(va))) +#define vmap_to_page (va) mfn_to_page(vmap_to_mfn(va)) + +static inline void *maddr_to_virt (unsigned long ma)
paddr_t as parameter type?
xen/arch/riscv/include/asm/mm.h line 244
Quoted 6 lines
xen/arch/riscv/include/asm/mm.h +5
-0+{ +BUG_ON ("unimplemented" ); +return NULL ; +} +#define PFN_ORDER (pfn) ((pfn)->v.free.order)
Would be nice if RISC-V wouldn't repeat the parameter mis-naming: Any of p, pg, or page would be more suitable here.
With the adjustments:
Jan
PATCH v6 17/20 xen/riscv: introduce vm_event_*() functions
In reply to Oleksii Kurochko---
Changes in V5-V6:
- Only rebase was done.
---
Changes in V4:
- New patch.
--- xen/arch/riscv/Makefile | 1 + xen/arch/riscv/vm_event.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) create mode 100644 xen/arch/riscv/vm_event.c
xen/arch/riscv/Makefile +1 -0
@@ -5,6 +5,7 @@ obj-$(CONFIG_RISCV_64) += riscv64/ obj-y += sbi.o obj-y += setup.o obj-y += traps.o +obj-y += vm_event.o $(TARGET): $(TARGET)-syms $(OBJCOPY) -O binary -S $< $@
xen/arch/riscv/vm_event.c new file +19 -0
@@ -0,0 +1,19 @@ +#include <xen/bug.h> + +struct vm_event_st ; +struct vcpu ; + +void vm_event_fill_regs (struct vm_event_st *req) +{ +BUG_ON ("unimplemented" ); +} + +void vm_event_set_registers (struct vcpu *v,struct vm_event_st *rsp) +{ +BUG_ON ("unimplemented" ); +} + +void vm_event_monitor_next_interrupt (struct vcpu *v) +{ +/* Not supported on RISCV. */ +}
Signature · 1 line
2.43.0
PATCH v6 18/20 xen/riscv: add minimal amount of stubs to build full Xen
In reply to Oleksii Kurochko---
Changes in V6:
- update the commit in stubs.c around /* ... common/irq.c ... */
- add Acked-by: Jan Beulich <[email protected]>
---
Changes in V5:
- drop unrelated changes
- assert_failed("unimplmented...") change to BUG_ON()
---
Changes in V4:
- added new stubs which are necessary for compilation after rebase: __cpu_up(), __cpu_disable(), __cpu_die() from smpboot.c
- back changes related to printk() in early_printk() as they should be removed in the next patch to avoid compilation error.
- update definition of cpu_khz: __read_mostly -> __ro_after_init.
- drop vm_event_reset_vmtrace(). It is defibed in asm-generic/vm_event.h.
- move vm_event_*() functions from stubs.c to riscv/vm_event.c.
- s/BUG/BUG_ON("unimplemented") in stubs.c
- back irq_actor_none() and irq_actor_none() as common/irq.c isn't compiled at this moment, so this function are needed to avoid compilation error.
- defined max_page to avoid compilation error, it will be removed as soon as common/page_alloc.c will be compiled.
---
Changes in V3:
- code style fixes.
- update attribute for frametable_base_pdx and frametable_virt_end to __ro_after_init. insteaf of read_mostly.
- use BUG() instead of assert_failed/WARN for newly introduced stubs.
- drop "#include <public/vm_event.h>" in stubs.c and use forward declaration instead.
- drop ack_node() and end_node() as they aren't used now.
---
Changes in V2:
- define udelay stub
- remove 'select HAS_PDX' from RISC-V Kconfig because of
https://lore.kernel.org/xen-devel/[email protected]/
--- xen/arch/riscv/Makefile | 1 + xen/arch/riscv/mm.c | 50 +++++ xen/arch/riscv/setup.c | 8 + xen/arch/riscv/stubs.c | 439 ++++++++++++++++++++++++++++++++++++++++ xen/arch/riscv/traps.c | 25 +++ 5 files changed, 523 insertions(+) create mode 100644 xen/arch/riscv/stubs.c
xen/arch/riscv/Makefile +1 -0
@@ -4,6 +4,7 @@ obj-y += mm.o obj-$(CONFIG_RISCV_64) += riscv64/ obj-y += sbi.o obj-y += setup.o +obj-y += stubs.o obj-y += traps.o obj-y += vm_event.o
xen/arch/riscv/mm.c +50 -0
64 lines not on this page · raw message
xen/arch/riscv/setup.c +8 -0
@@ -1,11 +1,19 @@/* SPDX-License-Identifier: GPL-2.0-only */ +#include <xen/bug.h> #include <xen/compile.h> #include <xen/init.h> #include <xen/mm.h> +#include <public/version.h> +#include <asm/early_printk.h> +void arch_get_xen_caps (xen_capabilities_info_t *info) +{ +BUG_ON ("unimplemented" ); +} +/* Xen stack for bringing up the first CPU. */ unsigned char __initdata cpu0_boot_stack[STACK_SIZE ]__aligned(STACK_SIZE) ;
xen/arch/riscv/stubs.c new file +439 -0
439 lines not on this page · raw message
xen/arch/riscv/traps.c +25 -0
@@ -4,6 +4,10 @@* * RISC-V Trap handlers */ + +#include <xen/lib.h> +#include <xen/sched.h> +#include <asm/processor.h> #include <asm/traps.h>
@@ -11,3 +15,24 @@ void do_trap(struct cpu_user_regs *cpu_regs) {die (); } + +void vcpu_show_execution_state (struct vcpu *v) +{ +BUG_ON ("unimplemented" ); +} + +void show_execution_state (const struct cpu_user_regs *regs) +{ +printk ("implement show_execution_state(regs)\n" ); +} + +void arch_hypercall_tasklet_result (struct vcpu *v,long res) +{ +BUG_ON ("unimplemented" ); +} + +enum mc_disposition arch_do_multicall_call (struct mc_state *state) +{ +BUG_ON ("unimplemented" ); +return mc_continue; +}
Signature · 1 line
2.43.0
PATCH v6 19/20 xen/riscv: enable full Xen build
In reply to Oleksii Kurochko---
Changes in V5-V6:
- Nothing changed. Only rebase.
---
Changes in V4:
- drop stubs for irq_actor_none() and irq_actor_none() as common/irq.c is compiled now.
- drop defintion of max_page in stubs.c as common/page_alloc.c is compiled now.
- drop printk() related changes in riscv/early_printk.c as common version will be used.
---
Changes in V3:
- Reviewed-by: Jan Beulich <[email protected]>
- unrealted change dropped in tiny64_defconfig
---
Changes in V2:
- Nothing changed. Only rebase.
--- xen/arch/riscv/Makefile | 16 +++- xen/arch/riscv/arch.mk | 4 - xen/arch/riscv/early_printk.c | 168 ---------------------------------- xen/arch/riscv/stubs.c | 24 ----- 4 files changed, 15 insertions(+), 197 deletions(-)
xen/arch/riscv/Makefile +15 -1
@@ -12,10 +12,24 @@ $(TARGET): $(TARGET)-syms $(OBJCOPY) -O binary -S $< $@ $(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds - $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@ + $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \ + $(objtree)/common/symbols-dummy.o -o $(dot-target).0 + $(NM) -pa --format=sysv $(dot-target).0 \ + | $(objtree)/tools/symbols $(all_symbols) --sysv --sort \ + > $(dot-target).0.S + $(MAKE) $(build)=$(@D) $(dot-target).0.o + $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< \ + $(dot-target).0.o -o $(dot-target).1 + $(NM) -pa --format=sysv $(dot-target).1 \ + | $(objtree)/tools/symbols $(all_symbols) --sysv --sort \ + > $(dot-target).1.S + $(MAKE) $(build)=$(@D) $(dot-target).1.o + $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) \ + $(dot-target).1.o -o $@ $(NM) -pa --format=sysv $@ \ | $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \ > [email protected] + rm -f $(@D)/.$(@F).[0-9]* $(obj)/xen.lds: $(src)/xen.lds.S FORCE $(call if_changed_dep,cpp_lds_S)
xen/arch/riscv/arch.mk +0 -4
@@ -19,7 +19,3 @@ extensions := $(subst $(space),,$(extensions)) # -mcmodel=medlow would force Xen into the lower half. CFLAGS += $(riscv-abi-y) -march=$(riscv-march-y)$(extensions) -mstrict-align -mcmodel=medany - -# TODO: Drop override when more of the build is working -override ALL_OBJS-y = arch/$(SRCARCH)/built_in.o -override ALL_LIBS-y =
xen/arch/riscv/early_printk.c +0 -168
171 lines not on this page · raw message
xen/arch/riscv/stubs.c +0 -24
@@ -24,12 +24,6 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);nodemask_t __read_mostly node_online_map = { { [0 ] =1UL } }; -/* -* max_page is defined in page_alloc.c which isn't complied for now. -* definition of max_page will be remove as soon as page_alloc is built. -*/ -unsigned long __read_mostly max_page; -/* time.c */ unsigned long __ro_after_init cpu_khz;/* CPU clock frequency in kHz. */
@@ -419,21 +413,3 @@ void __cpu_die(unsigned int cpu) {BUG_ON ("unimplemented" ); } - -/* -* The following functions are defined in common/irq.c, but common/irq.c isn't -* built for now. These changes will be removed there when common/irq.c is -* ready. -*/ - -void cf_check irq_actor_none(struct irq_desc *desc) -{ -BUG_ON ("unimplemented" ); -} - -unsigned int cf_check irq_startup_none(struct irq_desc *desc) -{ -BUG_ON ("unimplemented" ); - -return 0 ; -}
Signature · 1 line
2.43.0
PATCH v6 20/20 xen/README: add compiler and binutils versions for RISC-V64
In reply to Oleksii KurochkoThis patch doesn't represent a strict lower bound for GCC and GNU Binutils; rather, these versions are specifically employed by the Xen RISC-V container and are anticipated to undergo continuous testing. Older GCC and GNU Binutils would work,
but this is not a guarantee.
While it is feasible to utilize Clang, it's important to note that, currently, there is no Xen RISC-V CI job in place to verify the seamless functioning of the build with Clang.
---
Changes in V6:
- update the message in README.
---
Changes in V5:
- update the commit message and README file with additional explanation about GCC and GNU Binutils version. Additionally, it was added information about Clang.
---
Changes in V4:
- Update version of GCC (12.2) and GNU Binutils (2.39) to the version which are in Xen's contrainter for RISC-V
---
Changes in V3:
- new patch
--- README | 4 ++++ 1 file changed, 4 insertions(+)
README +4 -0
@@ -48,6 +48,10 @@ provided by your OS distributor: - For ARM 64-bit: - GCC 5.1 or later - GNU Binutils 2.24 or later + - For RISC-V 64-bit: + - GCC 12.2 or later + - GNU Binutils 2.39 or later + Older GCC and GNU Binutils would work, but this is not a guarantee. * POSIX compatible awk * Development install of zlib (e.g., zlib-dev) * Development install of Python 2.7 or later (e.g., python-dev)
Signature · 1 line
2.43.0