Skip to contentSkip to conversation

riscv: Add vector ISA support

PATCH In review Greentime Hu 5 participants 22 messages May 11, 2022 – Nov 8, 2022

Subsystems (4)
RISC-V exec and binfmt ptrace support KVM RISC-V
lore
Share & tools
lore

Greentime Hu

author
#0 lore raw

v10 cover riscv: Add vector ISA support

This patchset is implemented based on vector 1.0 spec to add vector support in riscv Linux kernel. There are some assumptions for this implementations.

1. We assume all harts has the same ISA in the system.
2. We use vector like FPU in some ways instead of using vector like a specific IP.
3. We disable vector in kernel space by default unless kernel uses kernel mode vector kernel_rvv_begin()/kernel_rvv_end().
4. We detect "riscv,isa" to determine whether vector is support or not.

We defined a new structure __riscv_v_state in struct thread_struct to save /restore the vector related registers. It is used for both kernel space and user space.

- In kernel space, the datap pointer in __riscv_v_state will be allocated to save vector registers.
- In user space,
- In signal handler of user space, datap will point to the address of the __riscv_v_state data structure to save vector registers in stack. We also create a __reserved[] array in sigcontext for future extensions.
- In ptrace, the data will be put in ubuf in which we use riscv_vr_get()/riscv_vr_set() to get or set the __riscv_v_state data structure from/to it, datap pointer would be zeroed and vector registers will be copied to the address right after the __riscv_v_state structure in ubuf.

This patchset also adds support for kernel mode vector, kernel XOR implementation with vector ISA and includes several bug fixes and code refinement.

This patchset is rebased to v5.18-rc6 and it is tested by running several vector programs simultaneously. It also can get the correct ucontext_t in signal handler and restore correct context after sigreturn. It is also tested with ptrace() syscall to use PTRACE_GETREGSET/PTRACE_SETREGSET to get/set the vector registers.

TODO
1. Refine the __riscv_v_state allocation in start_thread()
2. Refine the vector context switching functions by lazy save/restore
3. Add AMP support. To support harts with different ISA

[1] https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc

---
Chnagelog V10
- Rebase to v5.18-rc6
- Merge several patches
- Refine codes
- Fix bugs
- Add kvm vector support

Changelog V9
- Rebase to v5.15
- Merge several patches
- Refine codes
- Fix a kernel panic issue

Changelog V8
- Rebase to v5.14
- Refine struct __riscv_v_state with struct __riscv_ctx_hdr
- Refine has_vector into a static key
- Defined __reserved space in struct sigcontext for vector and future extensions

Changelog V7
- Add support for kernel mode vector
- Add vector extension XOR implementation
- Optimize task switch codes of vector
- Allocate space for vector registers in start_thread()
- Fix an illegal instruction exception when accessing vlenb
- Optimize vector registers initialization
- Initialize vector registers with proper vsetvli then it can work normally
- Refine ptrace porting due to generic API changed
- Code clean up

Changelog V6
- Replace vle.v/vse.v instructions with vle8.v/vse8.v based on 0.9 spec
- Add comments based on mailinglist feedback
- Fix rv32 build error

Changelog V5
- Using regset_size() correctly in generic ptrace
- Fix the ptrace porting
- Fix compile warning

Changelog V4
- Support dynamic vlen
- Fix bugs: lazy save/resotre, not saving vtype
- Update VS bit offset based on latest vector spec
- Add new vector csr based on latest vector spec
- Code refine and removed unused macros

Changelog V3
- Rebase linux-5.6-rc3 and tested with qemu
- Seperate patches with Anup's advice
- Give out a ABI puzzle with unlimited vlen

Changelog V2
- Fixup typo "vecotr, fstate_save->vstate_save".
- Fixup wrong saved registers' length in vector.S.
- Seperate unrelated patches from this one.

Greentime Hu (9):
riscv: Add new csr defines related to vector extension
riscv: Add has_vector/riscv_vsize to save vector features.
riscv: Add vector struct and assembler definitions
riscv: Add task switch support for vector
riscv: Add ptrace vector support
riscv: Add sigcontext save/restore for vector
riscv: Add support for kernel mode vector
riscv: Add vector extension XOR implementation
riscv: Fix a kernel panic issue if $s2 is set to a specific value
before entering Linux

Guo Ren (4):
riscv: Rename __switch_to_aux -> fpu
riscv: Extending cpufeature.c to detect V-extension
riscv: Add vector feature to compile
riscv: Reset vector register

Vincent Chen (3):
riscv: signal: Report signal frame size to userspace via auxv
riscv: Add V extension to KVM ISA allow list
riscv: KVM: Add vector lazy save/restore support

 arch/riscv/Kconfig                       |  10 ++
 arch/riscv/Makefile                      |   1 +
 arch/riscv/include/asm/csr.h             |  16 ++-
 arch/riscv/include/asm/elf.h             |  41 +++---
 arch/riscv/include/asm/hwcap.h           |   1 +
 arch/riscv/include/asm/kvm_host.h        |   2 +
 arch/riscv/include/asm/kvm_vcpu_vector.h |  65 +++++++++
 arch/riscv/include/asm/processor.h       |   3 +
 arch/riscv/include/asm/switch_to.h       |  72 +++++++++-
 arch/riscv/include/asm/vector.h          |  17 +++
 arch/riscv/include/asm/xor.h             |  82 +++++++++++
 arch/riscv/include/uapi/asm/auxvec.h     |   1 +
 arch/riscv/include/uapi/asm/hwcap.h      |   1 +
 arch/riscv/include/uapi/asm/kvm.h        |   6 +
 arch/riscv/include/uapi/asm/ptrace.h     |  23 +++
 arch/riscv/include/uapi/asm/sigcontext.h |  24 ++++
 arch/riscv/kernel/Makefile               |   2 +
 arch/riscv/kernel/asm-offsets.c          |  15 ++
 arch/riscv/kernel/cpufeature.c           |  18 +++
 arch/riscv/kernel/entry.S                |   6 +-
 arch/riscv/kernel/head.S                 |  37 ++++-
 arch/riscv/kernel/kernel_mode_vector.c   | 132 +++++++++++++++++
 arch/riscv/kernel/process.c              |  43 ++++++
 arch/riscv/kernel/ptrace.c               |  71 ++++++++++
 arch/riscv/kernel/riscv_ksyms.c          |   6 +
 arch/riscv/kernel/setup.c                |   4 +
 arch/riscv/kernel/signal.c               | 173 ++++++++++++++++++++++-
 arch/riscv/kernel/vector.S               | 102 +++++++++++++
 arch/riscv/kvm/Makefile                  |   1 +
 arch/riscv/kvm/vcpu.c                    |  33 ++++-
 arch/riscv/kvm/vcpu_switch.S             |  69 +++++++++
 arch/riscv/kvm/vcpu_vector.c             | 173 +++++++++++++++++++++++
 arch/riscv/lib/Makefile                  |   1 +
 arch/riscv/lib/xor.S                     |  81 +++++++++++
 include/uapi/linux/elf.h                 |   1 +
 35 files changed, 1298 insertions(+), 35 deletions(-)
 create mode 100644 arch/riscv/include/asm/kvm_vcpu_vector.h
 create mode 100644 arch/riscv/include/asm/vector.h
 create mode 100644 arch/riscv/include/asm/xor.h
 create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
 create mode 100644 arch/riscv/kernel/vector.S
 create mode 100644 arch/riscv/kvm/vcpu_vector.c
 create mode 100644 arch/riscv/lib/xor.S
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#1 lore raw

PATCH v10 1/16 riscv: Rename __switch_to_aux -> fpu

From: Guo Ren <[email protected]>

The name of __switch_to_aux is not clear and rename it with the determine function: __switch_to_fpu. Next we could add other regs' switch.

Signed-off-by Guo Ren <[email protected]>
Signed-off-by Guo Ren <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
Reviewed-by Anup Patel <[email protected]>
Reviewed-by Palmer Dabbelt <[email protected]>
---
 arch/riscv/include/asm/switch_to.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/riscv/include/asm/switch_to.h +3 -3
index 0a3f4f95c555..ec83770b3d98 100644
@@ -45,7 +45,7 @@ static inline void fstate_restore(struct task_struct *task,
 	}
 }
 
-static inline void __switch_to_aux(struct task_struct *prev,
+static inline void __switch_to_fpu(struct task_struct *prev,
 				   struct task_struct *next)
 {
 	struct pt_regs *regs;

@@ -65,7 +65,7 @@ static __always_inline bool has_fpu(void)
 static __always_inline bool has_fpu(void) { return false; }
 #define fstate_save(task, regs) do { } while (0)
 #define fstate_restore(task, regs) do { } while (0)
-#define __switch_to_aux(__prev, __next) do { } while (0)
+#define __switch_to_fpu(__prev, __next) do { } while (0)
 #endif
 
 extern struct task_struct *__switch_to(struct task_struct *,

@@ -76,7 +76,7 @@ do {							\
 	struct task_struct *__prev = (prev);		\
 	struct task_struct *__next = (next);		\
 	if (has_fpu())					\
-		__switch_to_aux(__prev, __next);	\
+		__switch_to_fpu(__prev, __next);	\
 	((last) = __switch_to(__prev, __next));		\
 } while (0)
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#2 lore raw

PATCH v10 2/16 riscv: Extending cpufeature.c to detect V-extension

In reply to Greentime Hu

From: Guo Ren <[email protected]>

Current cpufeature.c doesn't support detecting V-extension, because "rv64" also contain a 'v' letter and we need to skip it.

Signed-off-by Guo Ren <[email protected]>
Signed-off-by Guo Ren <[email protected]>
Reviewed-by Anup Patel <[email protected]>
Reviewed-by Greentime Hu <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
Reviewed-by Palmer Dabbelt <[email protected]>
---
 arch/riscv/include/uapi/asm/hwcap.h | 1 +
 arch/riscv/kernel/cpufeature.c      | 1 +
 2 files changed, 2 insertions(+)
arch/riscv/include/uapi/asm/hwcap.h +1 -0
index 46dc3f5ee99f..c52bb7bbbabe 100644
@@ -21,5 +21,6 @@
 #define COMPAT_HWCAP_ISA_F	(1 << ('F' - 'A'))
 #define COMPAT_HWCAP_ISA_D	(1 << ('D' - 'A'))
 #define COMPAT_HWCAP_ISA_C	(1 << ('C' - 'A'))
+#define COMPAT_HWCAP_ISA_V	(1 << ('V' - 'A'))
 
 #endif /* _UAPI_ASM_RISCV_HWCAP_H */
arch/riscv/kernel/cpufeature.c +1 -0
index 1b2d42d7f589..92a8fd6bd3ed 100644
@@ -76,6 +76,7 @@ void __init riscv_fill_hwcap(void)
 	isa2hwcap['f'] = isa2hwcap['F'] = COMPAT_HWCAP_ISA_F;
 	isa2hwcap['d'] = isa2hwcap['D'] = COMPAT_HWCAP_ISA_D;
 	isa2hwcap['c'] = isa2hwcap['C'] = COMPAT_HWCAP_ISA_C;
+	isa2hwcap['v'] = isa2hwcap['V'] = COMPAT_HWCAP_ISA_V;
 
 	elf_hwcap = 0;
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#3 lore raw

PATCH v10 3/16 riscv: Add new csr defines related to vector extension

In reply to Greentime Hu

Follow the riscv vector spec to add new csr numbers.

[[email protected]: first porting for new vector related csr]
Acked-by Guo Ren <[email protected]>
Co-developed-by Guo Ren <[email protected]>
Signed-off-by Guo Ren <[email protected]>
Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
Reviewed-by Palmer Dabbelt <[email protected]>
---
 arch/riscv/include/asm/csr.h | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
arch/riscv/include/asm/csr.h +14 -2
index e935f27b10fd..ac95578a2908 100644
@@ -24,6 +24,12 @@
 #define SR_FS_CLEAN	_AC(0x00004000, UL)
 #define SR_FS_DIRTY	_AC(0x00006000, UL)
 
+#define SR_VS           _AC(0x00000600, UL) /* Vector Status */
+#define SR_VS_OFF       _AC(0x00000000, UL)
+#define SR_VS_INITIAL   _AC(0x00000200, UL)
+#define SR_VS_CLEAN     _AC(0x00000400, UL)
+#define SR_VS_DIRTY     _AC(0x00000600, UL)
+
 #define SR_XS		_AC(0x00018000, UL) /* Extension Status */
 #define SR_XS_OFF	_AC(0x00000000, UL)
 #define SR_XS_INITIAL	_AC(0x00008000, UL)

@@ -31,9 +37,9 @@
 #define SR_XS_DIRTY	_AC(0x00018000, UL)
 
 #ifndef CONFIG_64BIT
-#define SR_SD		_AC(0x80000000, UL) /* FS/XS dirty */
+#define SR_SD		_AC(0x80000000, UL) /* FS/VS/XS dirty */
 #else
-#define SR_SD		_AC(0x8000000000000000, UL) /* FS/XS dirty */
+#define SR_SD		_AC(0x8000000000000000, UL) /* FS/VS/XS dirty */
 #endif
 
 /* SATP flags */

@@ -268,6 +274,12 @@
 #define CSR_MIMPID		0xf13
 #define CSR_MHARTID		0xf14
 
+#define CSR_VSTART		0x8
+#define CSR_VCSR		0xf
+#define CSR_VL			0xc20
+#define CSR_VTYPE		0xc21
+#define CSR_VLENB		0xc22
+
 #ifdef CONFIG_RISCV_M_MODE
 # define CSR_STATUS	CSR_MSTATUS
 # define CSR_IE		CSR_MIE
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#4 lore raw

PATCH v10 4/16 riscv: Add vector feature to compile

In reply to Greentime Hu

From: Guo Ren <[email protected]>

This patch adds a new config option which could enable assembler's vector feature.

Signed-off-by Guo Ren <[email protected]>
Co-developed-by Greentime Hu <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/Kconfig  | 10 ++++++++++
 arch/riscv/Makefile |  1 +
 2 files changed, 11 insertions(+)
arch/riscv/Kconfig +10 -0
index 00fd9c548f26..f54f37cc746b 100644
@@ -343,6 +343,16 @@ config FPU
 
 	  If you don't know what to do here, say Y.
 
+config VECTOR
+	bool "VECTOR support"
+	depends on GCC_VERSION >= 120000 || CLANG_VERSION >= 130000
+	default n
+	help
+	  Say N here if you want to disable all vector related procedure
+	  in the kernel.
+
+	  If you don't know what to do here, say Y.
+
 endmenu
 
 menu "Kernel features"
arch/riscv/Makefile +1 -0
index 7d81102cffd4..1f19bdac6767 100644
@@ -50,6 +50,7 @@ riscv-march-$(CONFIG_ARCH_RV32I)	:= rv32ima
 riscv-march-$(CONFIG_ARCH_RV64I)	:= rv64ima
 riscv-march-$(CONFIG_FPU)		:= $(riscv-march-y)fd
 riscv-march-$(CONFIG_RISCV_ISA_C)	:= $(riscv-march-y)c
+riscv-march-$(CONFIG_VECTOR)		:= $(riscv-march-y)v
 
 # Newer binutils versions default to ISA spec version 20191213 which moves some
 # instructions from the I extension to the Zicsr and Zifencei extensions.
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#5 lore raw

PATCH v10 5/16 riscv: Add has_vector/riscv_vsize to save vector features.

In reply to Greentime Hu

This patch is used to detect vector support status of CPU and use riscv_vsize to save the size of all the vector registers. It assumes all harts has the same capabilities in SMP system.

[[email protected]: add has_vector checking]
Co-developed-by Guo Ren <[email protected]>
Signed-off-by Guo Ren <[email protected]>
Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/vector.h | 14 +++++
 arch/riscv/kernel/cpufeature.c  | 17 ++++++
 arch/riscv/kernel/riscv_ksyms.c |  6 +++
 arch/riscv/kernel/vector.S      | 93 +++++++++++++++++++++++++++++++++
 4 files changed, 130 insertions(+)
 create mode 100644 arch/riscv/include/asm/vector.h
 create mode 100644 arch/riscv/kernel/vector.S
arch/riscv/include/asm/vector.h new file +14 -0
new file mode 100644
index 000000000000..16304b0c6a6f
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2020 SiFive
+ */
+
+#ifndef __ASM_RISCV_VECTOR_H
+#define __ASM_RISCV_VECTOR_H
+
+#include <linux/types.h>
+
+void rvv_enable(void);
+void rvv_disable(void);
+
+#endif /* ! __ASM_RISCV_VECTOR_H */
arch/riscv/kernel/cpufeature.c +17 -0
index 92a8fd6bd3ed..f2dec6cb60ed 100644
@@ -24,6 +24,11 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly;
 #ifdef CONFIG_FPU
 __ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu);
 #endif
+#ifdef CONFIG_VECTOR
+#include <asm/vector.h>
+__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_vector);
+unsigned long riscv_vsize __read_mostly;
+#endif
 
 /**
  * riscv_isa_extension_base() - Get base extension word

@@ -76,7 +81,9 @@ void __init riscv_fill_hwcap(void)
 	isa2hwcap['f'] = isa2hwcap['F'] = COMPAT_HWCAP_ISA_F;
 	isa2hwcap['d'] = isa2hwcap['D'] = COMPAT_HWCAP_ISA_D;
 	isa2hwcap['c'] = isa2hwcap['C'] = COMPAT_HWCAP_ISA_C;
+#ifdef CONFIG_VECTOR
 	isa2hwcap['v'] = isa2hwcap['V'] = COMPAT_HWCAP_ISA_V;
+#endif
 
 	elf_hwcap = 0;
 

@@ -237,4 +244,14 @@ void __init riscv_fill_hwcap(void)
 	if (elf_hwcap & (COMPAT_HWCAP_ISA_F | COMPAT_HWCAP_ISA_D))
 		static_branch_enable(&cpu_hwcap_fpu);
 #endif
+
+#ifdef CONFIG_VECTOR
+	if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
+		static_branch_enable(&cpu_hwcap_vector);
+		/* There are 32 vector registers with vlenb length. */
+		rvv_enable();
+		riscv_vsize = csr_read(CSR_VLENB) * 32;
+		rvv_disable();
+	}
+#endif
 }
arch/riscv/kernel/riscv_ksyms.c +6 -0
index 5ab1c7e1a6ed..3489d2a20ca3 100644
@@ -15,3 +15,9 @@ EXPORT_SYMBOL(memmove);
 EXPORT_SYMBOL(__memset);
 EXPORT_SYMBOL(__memcpy);
 EXPORT_SYMBOL(__memmove);
+
+#ifdef CONFIG_VECTOR
+#include <asm/vector.h>
+EXPORT_SYMBOL(rvv_enable);
+EXPORT_SYMBOL(rvv_disable);
+#endif
arch/riscv/kernel/vector.S new file +93 -0

93 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#6 lore raw

Re: [PATCH v10 05/16] riscv: Add has_vector/riscv_vsize to save vector features.

arch/riscv/kernel/riscv_ksyms.c EXPORT_SYMBOL(memmove); · line 23

Quoted 5 lines · Greentime Hu
arch/riscv/kernel/riscv_ksyms.c +5 -0
+#ifdef CONFIG_VECTOR
+#include <asm/vector.h>
+EXPORT_SYMBOL(rvv_enable);
+EXPORT_SYMBOL(rvv_disable);
+#endif

As far as I can tell these are not used by any modular code, just by the later added kernel mode vector enable helpers.

Even if they were used this would need to be EXPORT_SYMBOL_GPL.

Quoted 5 lines
arch/riscv/kernel/vector.S new file +5 -0
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2019 Alibaba Group Holding Limited

arch/riscv/kernel/vector.S line 15

Quoted 10 lines
arch/riscv/kernel/vector.S new file +10 -0
+ *
+ *   This program is free software; you can redistribute it and/or
+ *   modify it under the terms of the GNU General Public License
+ *   as published by the Free Software Foundation, version 2.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *   GNU General Public License for more details.
+ */

No need for all the GPL boilerplate text when you have an SPDX tag.

#7 lore raw

Re: [PATCH v10 05/16] riscv: Add has_vector/riscv_vsize to save vector features.

Quoted 11 lines · Christoph Hellwig

arch/riscv/kernel/riscv_ksyms.c EXPORT_SYMBOL(memmove); · line 23

Quoted 5 lines · Greentime Hu
arch/riscv/kernel/riscv_ksyms.c +5 -0
+#ifdef CONFIG_VECTOR
+#include <asm/vector.h>
+EXPORT_SYMBOL(rvv_enable);
+EXPORT_SYMBOL(rvv_disable);
+#endif

As far as I can tell these are not used by any modular code, just by the later added kernel mode vector enable helpers.

Even if they were used this would need to be EXPORT_SYMBOL_GPL.

In the new version, I've made them static inline given the size and frequency of them being called. So this becomes moot.

Quoted 10 lines
Quoted 10 lines
arch/riscv/kernel/vector.S new file +9 -0
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017 SiFive
+ * Copyright (C) 2019 Alibaba Group Holding Limited
+ *
+ *   This program is free software; you can redistribute it and/or
+ *   modify it under the terms of the GNU General Public License
+ *   as published by the Free Software Foundation, version 2.
Quoted 8 lines
Quoted 6 lines

+ *

+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *   GNU General Public License for more details.

+ */

No need for all the GPL boilerplate text when you have an SPDX tag.

OK.

author
#8 lore raw

PATCH v10 6/16 riscv: Reset vector register

In reply to Greentime Hu

From: Guo Ren <[email protected]>

Reset vector registers at boot-time and disable vector instructions execution for kernel mode.

Signed-off-by Guo Ren <[email protected]>
Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Co-developed-by Han-Kuan Chen <[email protected]>
Signed-off-by Han-Kuan Chen <[email protected]>
Co-developed-by Greentime Hu <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
Reviewed-by Palmer Dabbelt <[email protected]>
---
 arch/riscv/kernel/entry.S |  6 +++---
 arch/riscv/kernel/head.S  | 35 +++++++++++++++++++++++++++++------
 2 files changed, 32 insertions(+), 9 deletions(-)
arch/riscv/kernel/entry.S +3 -3
index c8b9ce274b9a..559f1418e980 100644
@@ -77,10 +77,10 @@ _save_context:
 	 * Disable user-mode memory access as it should only be set in the
 	 * actual user copy routines.
 	 *
-	 * Disable the FPU to detect illegal usage of floating point in kernel
-	 * space.
+	 * Disable the FPU/Vector to detect illegal usage of floating point
+	 * or vector in kernel space.
 	 */
-	li t0, SR_SUM | SR_FS
+	li t0, SR_SUM | SR_FS | SR_VS
 
 	REG_L s0, TASK_TI_USER_SP(tp)
 	csrrc s1, CSR_STATUS, t0
arch/riscv/kernel/head.S +29 -6
index 893b8bb69391..2877af90b025 100644
@@ -139,10 +139,10 @@ secondary_start_sbi:
 	.option pop
 
 	/*
-	 * Disable FPU to detect illegal usage of
-	 * floating point in kernel space
+	 * Disable FPU & VECTOR to detect illegal usage of
+	 * floating point or vector in kernel space
 	 */
-	li t0, SR_FS
+	li t0, SR_FS | SR_VS
 	csrc CSR_STATUS, t0
 
 	/* Set trap vector to spin forever to help debug */

@@ -233,10 +233,10 @@ pmp_done:
 .option pop
 
 	/*
-	 * Disable FPU to detect illegal usage of
-	 * floating point in kernel space
+	 * Disable FPU & VECTOR to detect illegal usage of
+	 * floating point or vector in kernel space
 	 */
-	li t0, SR_FS
+	li t0, SR_FS | SR_VS
 	csrc CSR_STATUS, t0
 
 #ifdef CONFIG_RISCV_BOOT_SPINWAIT

@@ -429,6 +429,29 @@ ENTRY(reset_regs)
 	csrw	fcsr, 0
 	/* note that the caller must clear SR_FS */
 #endif /* CONFIG_FPU */
+
+#ifdef CONFIG_VECTOR
+	csrr	t0, CSR_MISA
+	li	t1, COMPAT_HWCAP_ISA_V
+	and	t0, t0, t1
+	beqz	t0, .Lreset_regs_done
+
+	/*
+	 * Clear vector registers and reset vcsr
+	 * VLMAX has a defined value, VLEN is a constant,
+	 * and this form of vsetvli is defined to set vl to VLMAX.
+	 */
+	li	t1, SR_VS
+	csrs	CSR_STATUS, t1
+	csrs	CSR_VCSR, x0
+	vsetvli t1, x0, e8, m8, ta, ma
+	vmv.v.i v0, 0
+	vmv.v.i v8, 0
+	vmv.v.i v16, 0
+	vmv.v.i v24, 0
+	/* note that the caller must clear SR_VS */
+#endif /* CONFIG_VECTOR */
+
 .Lreset_regs_done:
 	ret
 END(reset_regs)
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#9 lore raw

PATCH v10 7/16 riscv: Add vector struct and assembler definitions

In reply to Greentime Hu

Add vector state context struct in struct thread and asm-offsets.c definitions.

The vector registers will be saved in datap pointer of __riscv_v_state. It will be dynamically allocated in kernel space. It will be put right after the __riscv_v_state data structure in user space.

Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/processor.h   |  1 +
 arch/riscv/include/uapi/asm/ptrace.h | 17 +++++++++++++++++
 arch/riscv/kernel/asm-offsets.c      |  6 ++++++
 3 files changed, 24 insertions(+)
arch/riscv/include/asm/processor.h +1 -0
index 0749924d9e55..91f1c318328e 100644
@@ -35,6 +35,7 @@ struct thread_struct {
 	unsigned long s[12];	/* s[0]: frame pointer */
 	struct __riscv_d_ext_state fstate;
 	unsigned long bad_cause;
+	struct __riscv_v_state vstate;
 };
 
 /* Whitelist the fstate from the task_struct for hardened usercopy */
arch/riscv/include/uapi/asm/ptrace.h +17 -0
index 882547f6bd5c..6ee1ca2edfa7 100644
@@ -77,6 +77,23 @@ union __riscv_fp_state {
 	struct __riscv_q_ext_state q;
 };
 
+struct __riscv_v_state {
+	unsigned long vstart;
+	unsigned long vl;
+	unsigned long vtype;
+	unsigned long vcsr;
+	void *datap;
+	/*
+	 * In signal handler, datap will be set a correct user stack offset
+	 * and vector registers will be copied to the address of datap
+	 * pointer.
+	 *
+	 * In ptrace syscall, datap will be set to zero and the vector
+	 * registers will be copied to the address right after this
+	 * structure.
+	 */
+};
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _UAPI_ASM_RISCV_PTRACE_H */
arch/riscv/kernel/asm-offsets.c +6 -0
index df9444397908..37e3e6a8d877 100644
@@ -75,6 +75,12 @@ void asm_offsets(void)
 	OFFSET(TSK_STACK_CANARY, task_struct, stack_canary);
 #endif
 
+	OFFSET(RISCV_V_STATE_VSTART, __riscv_v_state, vstart);
+	OFFSET(RISCV_V_STATE_VL, __riscv_v_state, vl);
+	OFFSET(RISCV_V_STATE_VTYPE, __riscv_v_state, vtype);
+	OFFSET(RISCV_V_STATE_VCSR, __riscv_v_state, vcsr);
+	OFFSET(RISCV_V_STATE_DATAP, __riscv_v_state, datap);
+
 	DEFINE(PT_SIZE, sizeof(struct pt_regs));
 	OFFSET(PT_EPC, pt_regs, epc);
 	OFFSET(PT_RA, pt_regs, ra);
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#10 lore raw

PATCH v10 8/16 riscv: Add task switch support for vector

In reply to Greentime Hu

This patch adds task switch support for vector. It supports partial lazy save and restore mechanism. It also supports all lengths of vlen.

[[email protected]: First available porting to support vector context switching]
[[email protected]: Rewrite vector.S to support dynamic vlen, xlen and code refine]
[[email protected]: Fix the might_sleep issue in vstate_save, vstate_restore]
[[email protected]: Optimize task switch codes of vector] [[email protected]: Fix the arch_release_task_struct free wrong datap issue]

Suggested-by Andrew Waterman <[email protected]>
Co-developed-by Nick Knight <[email protected]>
Signed-off-by Nick Knight <[email protected]>
Co-developed-by Guo Ren <[email protected]>
Signed-off-by Guo Ren <[email protected]>
Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Co-developed-by Ruinland Tsai <[email protected]>
Signed-off-by Ruinland Tsai <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/switch_to.h | 66 ++++++++++++++++++++++++++++++
 arch/riscv/kernel/Makefile         |  1 +
 arch/riscv/kernel/process.c        | 43 +++++++++++++++++++
 3 files changed, 110 insertions(+)
arch/riscv/include/asm/switch_to.h +66 -0
index ec83770b3d98..6836c51885ce 100644
@@ -7,10 +7,12 @@
 #define _ASM_RISCV_SWITCH_TO_H
 
 #include <linux/jump_label.h>
+#include <linux/slab.h>
 #include <linux/sched/task_stack.h>
 #include <asm/processor.h>
 #include <asm/ptrace.h>
 #include <asm/csr.h>
+#include <asm/asm-offsets.h>
 
 #ifdef CONFIG_FPU
 extern void __fstate_save(struct task_struct *save_to);

@@ -68,6 +70,68 @@ static __always_inline bool has_fpu(void) { return false; }
 #define __switch_to_fpu(__prev, __next) do { } while (0)
 #endif
 
+#ifdef CONFIG_VECTOR
+extern struct static_key_false cpu_hwcap_vector;
+static __always_inline bool has_vector(void)
+{
+	return static_branch_likely(&cpu_hwcap_vector);
+}
+extern unsigned long riscv_vsize;
+extern void __vstate_save(struct __riscv_v_state *save_to, void *datap);
+extern void __vstate_restore(struct __riscv_v_state *restore_from, void *datap);
+
+static inline void __vstate_clean(struct pt_regs *regs)
+{
+	regs->status = (regs->status & ~(SR_VS)) | SR_VS_CLEAN;
+}
+
+static inline void vstate_off(struct task_struct *task,
+			      struct pt_regs *regs)
+{
+	regs->status = (regs->status & ~SR_VS) | SR_VS_OFF;
+}
+
+static inline void vstate_save(struct task_struct *task,
+			       struct pt_regs *regs)
+{
+	if ((regs->status & SR_VS) == SR_VS_DIRTY) {
+		struct __riscv_v_state *vstate = &(task->thread.vstate);
+
+		__vstate_save(vstate, vstate->datap);
+		__vstate_clean(regs);
+	}
+}
+
+static inline void vstate_restore(struct task_struct *task,
+				  struct pt_regs *regs)
+{
+	if ((regs->status & SR_VS) != SR_VS_OFF) {
+		struct __riscv_v_state *vstate = &(task->thread.vstate);
+
+		__vstate_restore(vstate, vstate->datap);
+		__vstate_clean(regs);
+	}
+}
+
+static inline void __switch_to_vector(struct task_struct *prev,
+				   struct task_struct *next)
+{
+	struct pt_regs *regs;
+
+	regs = task_pt_regs(prev);
+	if (unlikely(regs->status & SR_SD))
+		vstate_save(prev, regs);
+	vstate_restore(next, task_pt_regs(next));
+}
+
+#else
+static __always_inline bool has_vector(void) { return false; }
+#define riscv_vsize (0)
+#define vstate_save(task, regs) do { } while (0)
+#define vstate_restore(task, regs) do { } while (0)
+#define __switch_to_vector(__prev, __next) do { } while (0)
+#endif
+
 extern struct task_struct *__switch_to(struct task_struct *,
 				       struct task_struct *);
 

@@ -77,6 +141,8 @@ do {							\
 	struct task_struct *__next = (next);		\
 	if (has_fpu())					\
 		__switch_to_fpu(__prev, __next);	\
+	if (has_vector())					\
+		__switch_to_vector(__prev, __next);	\
 	((last) = __switch_to(__prev, __next));		\
 } while (0)
arch/riscv/kernel/Makefile +1 -0
index 87adbe47bc15..5dc550a9fb45 100644
@@ -40,6 +40,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/
 
 obj-$(CONFIG_RISCV_M_MODE)	+= traps_misaligned.o
 obj-$(CONFIG_FPU)		+= fpu.o
+obj-$(CONFIG_VECTOR)		+= vector.o
 obj-$(CONFIG_SMP)		+= smpboot.o
 obj-$(CONFIG_SMP)		+= smp.o
 obj-$(CONFIG_SMP)		+= cpu_ops.o
arch/riscv/kernel/process.c +43 -0

71 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
build bot
#11 lore raw

Re: [PATCH v10 08/16] riscv: Add task switch support for vector

I love your patch! Perhaps something to improve:

Hi Greentime,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linux/master]
[also build test WARNING on linus/master v5.18-rc6]
[cannot apply to kees/for-next/execve next-20220511]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Greentime-Hu/riscv-Add-vector-ISA-support/20220511-163610
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git feb9c5e19e913b53cb536a7aa7c9f20107bb51ec
config: riscv-allnoconfig (https://download.01.org/0day-ci/archive/20220511/[email protected]/config)
compiler: riscv64-linux-gcc (GCC) 11.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/dcb5b4bc0fc33bba0eb3219b9a8d29ddff58bbef
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Greentime-Hu/riscv-Add-vector-ISA-support/20220511-163610
git checkout dcb5b4bc0fc33bba0eb3219b9a8d29ddff58bbef
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=riscv SHELL=/bin/bash arch/riscv/kernel/

If you fix the issue, kindly add following tag as appropriate

Reported-by kernel test robot <[email protected]>

All warnings (new ones prefixed by >>):

>> arch/riscv/kernel/process.c:149:6: warning: no previous prototype for 'arch_release_task_struct' [-Wmissing-prototypes]

149 | void arch_release_task_struct(struct task_struct *tsk)
| ^~~~~~~~~~~~~~~~~~~~~~~~

vim +/arch_release_task_struct +149 arch/riscv/kernel/process.c

148

 > 149	void arch_release_task_struct(struct task_struct *tsk)

150 {
151 /* Free the vector context of datap. */
152 if (has_vector() && tsk->thread.vstate.datap)
153 kfree(tsk->thread.vstate.datap);
154 }
155

Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp
build bot
#12 lore raw

Re: [PATCH v10 08/16] riscv: Add task switch support for vector

In reply to Greentime Hu
I love your patch! Perhaps something to improve:

Hi Greentime,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linux/master]
[also build test WARNING on linus/master v5.18-rc6]
[cannot apply to kees/for-next/execve next-20220511]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Greentime-Hu/riscv-Add-vector-ISA-support/20220511-163610
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git feb9c5e19e913b53cb536a7aa7c9f20107bb51ec
config: riscv-randconfig-r031-20220509 (https://download.01.org/0day-ci/archive/20220512/[email protected]/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 18dd123c56754edf62c7042dcf23185c3727610f)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install riscv cross compiling tool for clang build
# apt-get install binutils-riscv64-linux-gnu
# https://github.com/intel-lab-lkp/linux/commit/dcb5b4bc0fc33bba0eb3219b9a8d29ddff58bbef
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Greentime-Hu/riscv-Add-vector-ISA-support/20220511-163610
git checkout dcb5b4bc0fc33bba0eb3219b9a8d29ddff58bbef
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=riscv SHELL=/bin/bash arch/riscv/kernel/

If you fix the issue, kindly add following tag as appropriate

Reported-by kernel test robot <[email protected]>

All warnings (new ones prefixed by >>):

>> arch/riscv/kernel/process.c:149:6: warning: no previous prototype for function 'arch_release_task_struct' [-Wmissing-prototypes]
   void arch_release_task_struct(struct task_struct *tsk)
        ^

arch/riscv/kernel/process.c:149:1: note: declare 'static' if the function is not intended to be used outside of this translation unit void arch_release_task_struct(struct task_struct *tsk)
^
static
1 warning generated.

vim +/arch_release_task_struct +149 arch/riscv/kernel/process.c

148

 > 149	void arch_release_task_struct(struct task_struct *tsk)

150 {
151 /* Free the vector context of datap. */
152 if (has_vector() && tsk->thread.vstate.datap)
153 kfree(tsk->thread.vstate.datap);
154 }
155

Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp
author
#13 lore raw

PATCH v10 9/16 riscv: Add ptrace vector support

In reply to Greentime Hu

This patch adds ptrace support for riscv vector. The vector registers will be saved in datap pointer of __riscv_v_state. This pointer will be set right after the __riscv_v_state data structure then it will be put in ubuf for ptrace system call to get or set. It will check if the datap got from ubuf is set to the correct address or not when the ptrace system call is trying to set the vector registers.

Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/uapi/asm/ptrace.h |  6 +++
 arch/riscv/kernel/ptrace.c           | 71 ++++++++++++++++++++++++++++
 include/uapi/linux/elf.h             |  1 +
 3 files changed, 78 insertions(+)
arch/riscv/include/uapi/asm/ptrace.h +6 -0
index 6ee1ca2edfa7..2491875be80d 100644
@@ -94,6 +94,12 @@ struct __riscv_v_state {
 	 */
 };
 
+/*
+ * According to spec: The number of bits in a single vector register,
+ * VLEN >= ELEN, which must be a power of 2, and must be no greater than
+ * 2^16 = 65536bits = 8192bytes
+ */
+#define RISCV_MAX_VLENB (8192)
 #endif /* __ASSEMBLY__ */
 
 #endif /* _UAPI_ASM_RISCV_PTRACE_H */
arch/riscv/kernel/ptrace.c +71 -0

89 lines not on this page · raw message

include/uapi/linux/elf.h +1 -0
index 7ce993e6786c..b4e8d7dde15b 100644
@@ -436,6 +436,7 @@ typedef struct elf64_shdr {
 #define NT_MIPS_DSP	0x800		/* MIPS DSP ASE registers */
 #define NT_MIPS_FP_MODE	0x801		/* MIPS floating-point mode */
 #define NT_MIPS_MSA	0x802		/* MIPS SIMD registers */
+#define NT_RISCV_VECTOR	0x900		/* RISC-V vector registers */
 
 /* Note types with note name "GNU" */
 #define NT_GNU_PROPERTY_TYPE_0	5
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#14 lore raw

PATCH v10 10/16 riscv: Add sigcontext save/restore for vector

In reply to Greentime Hu

This patch adds sigcontext save/restore for vector. The vector registers will be saved in datap pointer. The datap pointer will be allocated dynamically when the task needs in kernel space. The datap pointer will be set right after the __riscv_v_state data structure to save all the vector registers in the signal handler stack.

Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/uapi/asm/sigcontext.h |  24 ++++
 arch/riscv/kernel/asm-offsets.c          |   2 +
 arch/riscv/kernel/setup.c                |   4 +
 arch/riscv/kernel/signal.c               | 165 ++++++++++++++++++++++-
 4 files changed, 191 insertions(+), 4 deletions(-)
arch/riscv/include/uapi/asm/sigcontext.h +24 -0
index 84f2dfcfdbce..b8a0fd7d7cfc 100644
@@ -8,6 +8,23 @@
 
 #include <asm/ptrace.h>
 
+/* The Magic number for signal context frame header. */
+#define RVV_MAGIC	0x53465457
+#define END_MAGIC	0x0
+
+/* The size of END signal context header. */
+#define END_HDR_SIZE	0x0
+
+struct __riscv_ctx_hdr {
+	__u32 magic;
+	__u32 size;
+};
+
+struct __sc_riscv_v_state {
+	struct __riscv_ctx_hdr head;
+	struct __riscv_v_state v_state;
+} __attribute__((aligned(16)));
+
 /*
  * Signal context structure
  *

@@ -17,6 +34,13 @@
 struct sigcontext {
 	struct user_regs_struct sc_regs;
 	union __riscv_fp_state sc_fpregs;
+	/*
+	 * 4K + 128 reserved for vector state and future expansion.
+	 * This space is enough to store the vector context whose VLENB
+	 * is less or equal to 128.
+	 * (The size of the vector context is 4144 byte as VLENB is 128)
+	 */
+	__u8 __reserved[4224] __attribute__((__aligned__(16)));
 };
 
 #endif /* _UAPI_ASM_RISCV_SIGCONTEXT_H */
arch/riscv/kernel/asm-offsets.c +2 -0
index 37e3e6a8d877..80316ef7bb78 100644
@@ -75,6 +75,8 @@ void asm_offsets(void)
 	OFFSET(TSK_STACK_CANARY, task_struct, stack_canary);
 #endif
 
+	OFFSET(RISCV_V_STATE_MAGIC, __riscv_ctx_hdr, magic);
+	OFFSET(RISCV_V_STATE_SIZE, __riscv_ctx_hdr, size);
 	OFFSET(RISCV_V_STATE_VSTART, __riscv_v_state, vstart);
 	OFFSET(RISCV_V_STATE_VL, __riscv_v_state, vl);
 	OFFSET(RISCV_V_STATE_VTYPE, __riscv_v_state, vtype);
arch/riscv/kernel/setup.c +4 -0
index 834eb652a7b9..36b55669b2da 100644
@@ -260,6 +260,8 @@ static void __init parse_dtb(void)
 #endif
 }
 
+extern void __init init_rt_signal_env(void);
+
 void __init setup_arch(char **cmdline_p)
 {
 	parse_dtb();

@@ -295,6 +297,8 @@ void __init setup_arch(char **cmdline_p)
 #endif
 
 	riscv_fill_hwcap();
+
+	init_rt_signal_env();
 }
 
 static int __init topology_init(void)
arch/riscv/kernel/signal.c +161 -4

228 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#15 lore raw

PATCH v10 11/16 riscv: signal: Report signal frame size to userspace via auxv

In reply to Greentime Hu

From: Vincent Chen <[email protected]>

The vector register belongs to the signal context. They need to be stored and restored as entering and leaving the signal handler. According to the V-extension specification, the maximum length of the vector registers can be 2^(XLEN-1). Hence, if userspace refers to the MINSIGSTKSZ to create a sigframe, it may not be enough. To resolve this problem, this patch refers to the commit 94b07c1f8c39c
("arm64: signal: Report signal frame size to userspace via auxv") to enable userspace to know the minimum required sigframe size through the auxiliary vector and use it to allocate enough memory for signal context.

Signed-off-by Greentime Hu <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
---
 arch/riscv/include/asm/elf.h         | 41 +++++++++++++++++-----------
 arch/riscv/include/asm/processor.h   |  2 ++
 arch/riscv/include/uapi/asm/auxvec.h |  1 +
 arch/riscv/kernel/signal.c           |  8 ++++++
 4 files changed, 36 insertions(+), 16 deletions(-)
arch/riscv/include/asm/elf.h +25 -16
index f53c40026c7a..72e02a31a279 100644
@@ -61,22 +61,31 @@ extern unsigned long elf_hwcap;
 #define ELF_PLATFORM	(NULL)
 
 #ifdef CONFIG_MMU
-#define ARCH_DLINFO						\
-do {								\
-	NEW_AUX_ENT(AT_SYSINFO_EHDR,				\
-		(elf_addr_t)current->mm->context.vdso);		\
-	NEW_AUX_ENT(AT_L1I_CACHESIZE,				\
-		get_cache_size(1, CACHE_TYPE_INST));		\
-	NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY,			\
-		get_cache_geometry(1, CACHE_TYPE_INST));	\
-	NEW_AUX_ENT(AT_L1D_CACHESIZE,				\
-		get_cache_size(1, CACHE_TYPE_DATA));		\
-	NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY,			\
-		get_cache_geometry(1, CACHE_TYPE_DATA));	\
-	NEW_AUX_ENT(AT_L2_CACHESIZE,				\
-		get_cache_size(2, CACHE_TYPE_UNIFIED));		\
-	NEW_AUX_ENT(AT_L2_CACHEGEOMETRY,			\
-		get_cache_geometry(2, CACHE_TYPE_UNIFIED));	\
+#define ARCH_DLINFO						 \
+do {								 \
+	NEW_AUX_ENT(AT_SYSINFO_EHDR,				 \
+		(elf_addr_t)current->mm->context.vdso);		 \
+	NEW_AUX_ENT(AT_L1I_CACHESIZE,				 \
+		get_cache_size(1, CACHE_TYPE_INST));		 \
+	NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY,			 \
+		get_cache_geometry(1, CACHE_TYPE_INST));	 \
+	NEW_AUX_ENT(AT_L1D_CACHESIZE,				 \
+		get_cache_size(1, CACHE_TYPE_DATA));		 \
+	NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY,			 \
+		get_cache_geometry(1, CACHE_TYPE_DATA));	 \
+	NEW_AUX_ENT(AT_L2_CACHESIZE,				 \
+		get_cache_size(2, CACHE_TYPE_UNIFIED));		 \
+	NEW_AUX_ENT(AT_L2_CACHEGEOMETRY,			 \
+		get_cache_geometry(2, CACHE_TYPE_UNIFIED));	 \
+	/*							 \
+	 * Should always be nonzero unless there's a kernel bug. \
+	 * If we haven't determined a sensible value to give to	 \
+	 * userspace, omit the entry:				 \
+	 */							 \
+	if (likely(signal_minsigstksz))				 \
+		NEW_AUX_ENT(AT_MINSIGSTKSZ, signal_minsigstksz); \
+	else							 \
+		NEW_AUX_ENT(AT_IGNORE, 0);			 \
 } while (0)
 #define ARCH_HAS_SETUP_ADDITIONAL_PAGES
 struct linux_binprm;
arch/riscv/include/asm/processor.h +2 -0
index 91f1c318328e..ea2518f092e1 100644
@@ -7,6 +7,7 @@
 #define _ASM_RISCV_PROCESSOR_H
 
 #include <linux/const.h>
+#include <linux/cache.h>
 
 #include <vdso/processor.h>
 

@@ -82,6 +83,7 @@ int riscv_of_parent_hartid(struct device_node *node);
 extern void riscv_fill_hwcap(void);
 extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
 
+extern unsigned long signal_minsigstksz __ro_after_init;
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_RISCV_PROCESSOR_H */
arch/riscv/include/uapi/asm/auxvec.h +1 -0
index 32c73ba1d531..6610d24e6662 100644
@@ -33,5 +33,6 @@
 
 /* entries in ARCH_DLINFO */
 #define AT_VECTOR_SIZE_ARCH	7
+#define AT_MINSIGSTKSZ 51
 
 #endif /* _UAPI_ASM_RISCV_AUXVEC_H */
arch/riscv/kernel/signal.c +8 -0
index 21b48bddeb8f..8e7f20cb0404 100644
@@ -473,8 +473,16 @@ asmlinkage __visible void do_notify_resume(struct pt_regs *regs,
 		resume_user_mode_work(regs);
 }
 
+unsigned long __ro_after_init signal_minsigstksz;
+
 void init_rt_signal_env(void);
 void __init init_rt_signal_env(void)
 {
 	rvv_sc_size = sizeof(struct __sc_riscv_v_state) + riscv_vsize;
+	/*
+	 * Determine the stack space required for guaranteed signal delivery.
+	 * The signal_minsigstksz will be populated into the AT_MINSIGSTKSZ entry
+	 * in the auxiliary array at process startup.
+	 */
+	signal_minsigstksz = cal_rt_frame_size();
 }
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#16 lore raw

PATCH v10 12/16 riscv: Add support for kernel mode vector

In reply to Greentime Hu

Add kernel_rvv_begin() and kernel_rvv_end() function declarations and corresponding definitions in kernel_mode_vector.c

These are needed to wrap uses of vector in kernel mode.

Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/vector.h        |   3 +
 arch/riscv/kernel/Makefile             |   1 +
 arch/riscv/kernel/kernel_mode_vector.c | 132 +++++++++++++++++++++++++
 arch/riscv/kernel/vector.S             |   9 ++
 4 files changed, 145 insertions(+)
 create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
arch/riscv/include/asm/vector.h +3 -0
index 16304b0c6a6f..a59841cc81fb 100644
@@ -10,5 +10,8 @@
 
 void rvv_enable(void);
 void rvv_disable(void);
+void kernel_rvv_begin(void);
+void kernel_rvv_end(void);
+void vector_flush_cpu_state(void);
 
 #endif /* ! __ASM_RISCV_VECTOR_H */
arch/riscv/kernel/Makefile +1 -0
index 5dc550a9fb45..68a6fb0fdac8 100644
@@ -41,6 +41,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/
 obj-$(CONFIG_RISCV_M_MODE)	+= traps_misaligned.o
 obj-$(CONFIG_FPU)		+= fpu.o
 obj-$(CONFIG_VECTOR)		+= vector.o
+obj-$(CONFIG_VECTOR)		+= kernel_mode_vector.o
 obj-$(CONFIG_SMP)		+= smpboot.o
 obj-$(CONFIG_SMP)		+= smp.o
 obj-$(CONFIG_SMP)		+= cpu_ops.o
arch/riscv/kernel/kernel_mode_vector.c new file +132 -0

132 lines not on this page · raw message

arch/riscv/kernel/vector.S +9 -0
index 9f7dc70c4443..9c2de823c0d9 100644
@@ -91,3 +91,12 @@ ENTRY(rvv_disable)
 	csrc	CSR_STATUS, status
 	ret
 ENDPROC(rvv_disable)
+
+ENTRY(vector_flush_cpu_state)
+	vsetvli t0, x0, e8, m8, ta, ma
+	vmv.v.i v0, 0
+	vmv.v.i v8, 0
+	vmv.v.i v16, 0
+	vmv.v.i v24, 0
+	ret
+ENDPROC(vector_flush_cpu_state)
Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#17 lore raw

PATCH v10 13/16 riscv: Add vector extension XOR implementation

In reply to Greentime Hu

This patch adds support for vector optimized XOR and it is tested in qemu.

Co-developed-by Han-Kuan Chen <[email protected]>
Signed-off-by Han-Kuan Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/xor.h | 82 ++++++++++++++++++++++++++++++++++++
 arch/riscv/lib/Makefile      |  1 +
 arch/riscv/lib/xor.S         | 81 +++++++++++++++++++++++++++++++++++
 3 files changed, 164 insertions(+)
 create mode 100644 arch/riscv/include/asm/xor.h
 create mode 100644 arch/riscv/lib/xor.S
arch/riscv/include/asm/xor.h new file +82 -0
new file mode 100644
index 000000000000..d1f2eeb14afb
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2021 SiFive
+ */
+
+#include <linux/hardirq.h>
+#include <asm-generic/xor.h>
+#ifdef CONFIG_VECTOR
+#include <asm/vector.h>
+#include <asm/switch_to.h>
+
+void xor_regs_2_(unsigned long bytes, unsigned long * __restrict p1,
+		 const unsigned long * __restrict p2);
+void xor_regs_3_(unsigned long bytes, unsigned long * __restrict p1,
+		 const unsigned long * __restrict p2,
+		 const unsigned long * __restrict p3);
+void xor_regs_4_(unsigned long bytes, unsigned long * __restrict p1,
+		 const unsigned long * __restrict p2,
+		 const unsigned long * __restrict p3,
+		 const unsigned long * __restrict p4);
+void xor_regs_5_(unsigned long bytes, unsigned long * __restrict p1,
+		 const unsigned long * __restrict p2,
+		 const unsigned long * __restrict p3,
+		 const unsigned long * __restrict p4,
+		 const unsigned long * __restrict p5);
+
+static void xor_rvv_2(unsigned long bytes, unsigned long * __restrict p1,
+		      const unsigned long * __restrict p2)
+{
+	kernel_rvv_begin();
+	xor_regs_2_(bytes, p1, p2);
+	kernel_rvv_end();
+}
+
+static void xor_rvv_3(unsigned long bytes, unsigned long * __restrict p1,
+		      const unsigned long * __restrict p2,
+		      const unsigned long * __restrict p3)
+{
+	kernel_rvv_begin();
+	xor_regs_3_(bytes, p1, p2, p3);
+	kernel_rvv_end();
+}
+
+static void xor_rvv_4(unsigned long bytes, unsigned long * __restrict p1,
+		      const unsigned long * __restrict p2,
+		      const unsigned long * __restrict p3,
+		      const unsigned long * __restrict p4)
+{
+	kernel_rvv_begin();
+	xor_regs_4_(bytes, p1, p2, p3, p4);
+	kernel_rvv_end();
+}
+
+static void xor_rvv_5(unsigned long bytes, unsigned long * __restrict p1,
+		      const unsigned long * __restrict p2,
+		      const unsigned long * __restrict p3,
+		      const unsigned long * __restrict p4,
+		      const unsigned long * __restrict p5)
+{
+	kernel_rvv_begin();
+	xor_regs_5_(bytes, p1, p2, p3, p4, p5);
+	kernel_rvv_end();
+}
+
+static struct xor_block_template xor_block_rvv = {
+	.name = "rvv",
+	.do_2 = xor_rvv_2,
+	.do_3 = xor_rvv_3,
+	.do_4 = xor_rvv_4,
+	.do_5 = xor_rvv_5
+};
+
+#undef XOR_TRY_TEMPLATES
+#define XOR_TRY_TEMPLATES           \
+	do {        \
+		xor_speed(&xor_block_8regs);    \
+		xor_speed(&xor_block_32regs);    \
+		if (has_vector()) { \
+			xor_speed(&xor_block_rvv);\
+		} \
+	} while (0)
+#endif
arch/riscv/lib/Makefile +1 -0
index 25d5c9664e57..acd87ac86d24 100644
@@ -7,3 +7,4 @@ lib-$(CONFIG_MMU)	+= uaccess.o
 lib-$(CONFIG_64BIT)	+= tishift.o
 
 obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
+lib-$(CONFIG_VECTOR)	+= xor.o
arch/riscv/lib/xor.S new file +81 -0

81 lines not on this page · raw message

Signature · 7 lines
2.17.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#18 lore raw

PATCH v10 14/16 riscv: Fix a kernel panic issue if $s2 is set to a specific value before entering Linux

In reply to Greentime Hu

Panic log:

[    0.018707] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[    0.023060] Oops [#1]
[    0.023214] Modules linked in:
[    0.023725] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.14.0 #33
[    0.023955] Hardware name: SiFive,FU800 (DT)
[    0.024150] epc : __vstate_save+0x1c/0x48
[    0.024654]  ra : arch_dup_task_struct+0x70/0x108
[    0.024815] epc : ffffffff80005ad8 ra : ffffffff800035a8 sp : ffffffff81203d50
[    0.025020]  gp : ffffffff812e8290 tp : ffffffff8120bdc0 t0 : 0000000000000000
[    0.025216]  t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffff81203d80
[    0.025424]  s1 : ffffffff8120bdc0 a0 : ffffffff8120c820 a1 : 0000000000000000
[    0.025659]  a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000600
[    0.025869]  a5 : ffffffff8120cdc0 a6 : ffffffe00160b400 a7 : ffffffff80a1fe60
[    0.026069]  s2 : ffffffe0016b8000 s3 : ffffffff81204000 s4 : 0000000000004000
[    0.026267]  s5 : 0000000000000000 s6 : ffffffe0016b8000 s7 : ffffffe0016b9000
[    0.026475]  s8 : ffffffff81203ee0 s9 : 0000000000800300 s10: ffffffff812e9088
[    0.026689]  s11: ffffffd004008000 t3 : 0000000000000000 t4 : 0000000000000100
[    0.026900]  t5 : 0000000000000600 t6 : ffffffe00167bcc4
[    0.027057] status: 8000000000000720 badaddr: 0000000000000000 cause: 000000000000000f
[    0.027344] [<ffffffff80005ad8>] __vstate_save+0x1c/0x48
[    0.027567] [<ffffffff8000abe8>] copy_process+0x266/0x11a0
[    0.027739] [<ffffffff8000bc98>] kernel_clone+0x90/0x2aa
[    0.027915] [<ffffffff8000c062>] kernel_thread+0x76/0x92
[    0.028075] [<ffffffff8072e34c>] rest_init+0x26/0xfc
[    0.028242] [<ffffffff80800638>] arch_call_rest_init+0x10/0x18
[    0.028423] [<ffffffff80800c4a>] start_kernel+0x5ce/0x5fe
[    0.029188] ---[ end trace 9a59af33f7ba3df4 ]---
[    0.029479] Kernel panic - not syncing: Attempted to kill the idle task!
[    0.029907] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

The NULL pointer accessing caused the kernel panic. There is a NULL pointer is because in vstate_save() function it will check (regs->status & SR_VS) == SR_VS_DIRTY and this is true, but it shouldn't be true because vector is not used here. Since vector is not used, datap won't be allocated so it is NULL. The reason why regs->status is set to a wrong value is because pt_regs->status is put in stack and it is polluted after setup_vm() called.

In prologue of setup_vm(), we can observe it will save s2 to stack however s2 is meaningless here because the caller is assembly code and s2 is just some value from previous stage. The compiler will base on calling convention to save the register to stack. Then 0x80008638 in s2 is saved to stack. It might be any value. In this failure case it is 0x80008638 and it will accidentally cause SR_VS_DIRTY to call the vstate_save() function.

(gdb) info addr setup_vm
Symbol "setup_vm" is a function at address 0xffffffff80802c8a. (gdb) va2pa 0xffffffff80802c8a
$64 = 0x80a02c8a
(gdb) x/10i 0x80a02c8a

   0x80a02c8a:  addi    sp,sp,-48
   0x80a02c8c:  li      a3,-1
   0x80a02c8e:  auipc   a5,0xff7fd
   0x80a02c92:  addi    a5,a5,882
   0x80a02c96:  sd      s0,32(sp)
   0x80a02c98:  sd      s2,16(sp) <-- store to stack

After returning from setup_vm()
(gdb) x/20i 0x0000000080201138

   0x80201138:  mv      a0,s1
   0x8020113a:  auipc   ra,0x802
   0x8020113e:  jalr    -1200(ra)    <-- jump to setup_vm()
   0x80201142:  auipc   a0,0xa03

(gdb) p/x $sp
$70 = 0x81404000
(gdb) p/x *(struct pt_regs*)($sp-0x120)
$71 = {
epc = 0x0,
ra = 0x0,
sp = 0x0,
gp = 0x0,
tp = 0x0,
t0 = 0x0,
t1 = 0x0,
t2 = 0x0,
s0 = 0x0,
s1 = 0x0,
a0 = 0x0,
a1 = 0x0,
a2 = 0x0,
a3 = 0x81403f90,
a4 = 0x80c04000,
a5 = 0x1,
a6 = 0xffffffff81337000,
a7 = 0x81096700,
s2 = 0x81400000,
s3 = 0xffffffff81200000,
s4 = 0x81403fd0,
s5 = 0x80a02c6c,
s6 = 0x8000000000006800,
s7 = 0x0,
s8 = 0xfffffffffffffff3,
s9 = 0x80c01000,
s10 = 0x81096700,
s11 = 0x82200000,
t3 = 0x81404000,
t4 = 0x80a02dea,
t5 = 0x0,
t6 = 0x82200000,
status = 0x80008638, <- Wrong value in stack!!!
badaddr = 0x82200000,
cause = 0x0,
orig_a0 = 0x80201142
}
(gdb) p/x $pc
$72 = 0x80201142
(gdb) p/x sizeof(struct pt_regs)
$73 = 0x120

Co-developed-by ShihPo Hung <[email protected]>
Signed-off-by ShihPo Hung <[email protected]>
Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/kernel/head.S | 2 ++
 1 file changed, 2 insertions(+)
arch/riscv/kernel/head.S +2 -0
index 2877af90b025..0c307c0bd3d6 100644
@@ -299,6 +299,7 @@ clear_bss_done:
 	/* Initialize page tables and relocate to virtual addresses */
 	la sp, init_thread_union + THREAD_SIZE
 	XIP_FIXUP_OFFSET sp
+	addi sp, sp, -PT_SIZE
 #ifdef CONFIG_BUILTIN_DTB
 	la a0, __dtb_start
 	XIP_FIXUP_OFFSET a0

@@ -316,6 +317,7 @@ clear_bss_done:
 	/* Restore C environment */
 	la tp, init_task
 	la sp, init_thread_union + THREAD_SIZE
+	addi sp, sp, -PT_SIZE
 
 #ifdef CONFIG_KASAN
 	call kasan_early_init
Signature · 1 line
2.17.1
#19 lore raw

Re: [PATCH v10 14/16] riscv: Fix a kernel panic issue if $s2 is set to a specific value before entering Linux

Quoted 132 lines

Panic log:

[    0.018707] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[    0.023060] Oops [#1]
[    0.023214] Modules linked in:
[    0.023725] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.14.0 #33
[    0.023955] Hardware name: SiFive,FU800 (DT)
[    0.024150] epc : __vstate_save+0x1c/0x48
[    0.024654]  ra : arch_dup_task_struct+0x70/0x108
[    0.024815] epc : ffffffff80005ad8 ra : ffffffff800035a8 sp : ffffffff81203d50
[    0.025020]  gp : ffffffff812e8290 tp : ffffffff8120bdc0 t0 : 0000000000000000
[    0.025216]  t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffff81203d80
[    0.025424]  s1 : ffffffff8120bdc0 a0 : ffffffff8120c820 a1 : 0000000000000000
[    0.025659]  a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000600
[    0.025869]  a5 : ffffffff8120cdc0 a6 : ffffffe00160b400 a7 : ffffffff80a1fe60
[    0.026069]  s2 : ffffffe0016b8000 s3 : ffffffff81204000 s4 : 0000000000004000
[    0.026267]  s5 : 0000000000000000 s6 : ffffffe0016b8000 s7 : ffffffe0016b9000
[    0.026475]  s8 : ffffffff81203ee0 s9 : 0000000000800300 s10: ffffffff812e9088
[    0.026689]  s11: ffffffd004008000 t3 : 0000000000000000 t4 : 0000000000000100
[    0.026900]  t5 : 0000000000000600 t6 : ffffffe00167bcc4
[    0.027057] status: 8000000000000720 badaddr: 0000000000000000 cause: 000000000000000f
[    0.027344] [<ffffffff80005ad8>] __vstate_save+0x1c/0x48
[    0.027567] [<ffffffff8000abe8>] copy_process+0x266/0x11a0
[    0.027739] [<ffffffff8000bc98>] kernel_clone+0x90/0x2aa
[    0.027915] [<ffffffff8000c062>] kernel_thread+0x76/0x92
[    0.028075] [<ffffffff8072e34c>] rest_init+0x26/0xfc
[    0.028242] [<ffffffff80800638>] arch_call_rest_init+0x10/0x18
[    0.028423] [<ffffffff80800c4a>] start_kernel+0x5ce/0x5fe
[    0.029188] ---[ end trace 9a59af33f7ba3df4 ]---
[    0.029479] Kernel panic - not syncing: Attempted to kill the idle task!
[    0.029907] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

The NULL pointer accessing caused the kernel panic. There is a NULL pointer is because in vstate_save() function it will check (regs->status & SR_VS) == SR_VS_DIRTY and this is true, but it shouldn't be true because vector is not used here. Since vector is not used, datap won't be allocated so it is NULL. The reason why regs->status is set to a wrong value is because pt_regs->status is put in stack and it is polluted after setup_vm() called.

In prologue of setup_vm(), we can observe it will save s2 to stack however s2 is meaningless here because the caller is assembly code and s2 is just some value from previous stage. The compiler will base on calling convention to save the register to stack. Then 0x80008638 in s2 is saved to stack. It might be any value. In this failure case it is 0x80008638 and it will accidentally cause SR_VS_DIRTY to call the vstate_save() function.

(gdb) info addr setup_vm
Symbol "setup_vm" is a function at address 0xffffffff80802c8a. (gdb) va2pa 0xffffffff80802c8a
$64 = 0x80a02c8a
(gdb) x/10i 0x80a02c8a

    0x80a02c8a:  addi    sp,sp,-48
    0x80a02c8c:  li      a3,-1
    0x80a02c8e:  auipc   a5,0xff7fd
    0x80a02c92:  addi    a5,a5,882
    0x80a02c96:  sd      s0,32(sp)
    0x80a02c98:  sd      s2,16(sp) <-- store to stack

After returning from setup_vm()
(gdb) x/20i 0x0000000080201138

    0x80201138:  mv      a0,s1
    0x8020113a:  auipc   ra,0x802
    0x8020113e:  jalr    -1200(ra)    <-- jump to setup_vm()
    0x80201142:  auipc   a0,0xa03
(gdb) p/x $sp
$70 = 0x81404000
(gdb) p/x *(struct pt_regs*)($sp-0x120)
$71 = {
   epc = 0x0,
   ra = 0x0,
   sp = 0x0,
   gp = 0x0,
   tp = 0x0,
   t0 = 0x0,
   t1 = 0x0,
   t2 = 0x0,
   s0 = 0x0,
   s1 = 0x0,
   a0 = 0x0,
   a1 = 0x0,
   a2 = 0x0,
   a3 = 0x81403f90,
   a4 = 0x80c04000,
   a5 = 0x1,
   a6 = 0xffffffff81337000,
   a7 = 0x81096700,
   s2 = 0x81400000,
   s3 = 0xffffffff81200000,
   s4 = 0x81403fd0,
   s5 = 0x80a02c6c,
   s6 = 0x8000000000006800,
   s7 = 0x0,
   s8 = 0xfffffffffffffff3,
   s9 = 0x80c01000,
   s10 = 0x81096700,
   s11 = 0x82200000,
   t3 = 0x81404000,
   t4 = 0x80a02dea,
   t5 = 0x0,
   t6 = 0x82200000,
   status = 0x80008638, <- Wrong value in stack!!!
   badaddr = 0x82200000,
   cause = 0x0,
   orig_a0 = 0x80201142
}

(gdb) p/x $pc
$72 = 0x80201142
(gdb) p/x sizeof(struct pt_regs)
$73 = 0x120

Co-developed-by ShihPo Hung <[email protected]>
Signed-off-by ShihPo Hung <[email protected]>
Co-developed-by Vincent Chen <[email protected]>
Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
  arch/riscv/kernel/head.S | 2 ++
  1 file changed, 2 insertions(+)
arch/riscv/kernel/head.S +1 -0
index 2877af90b025..0c307c0bd3d6 100644
@@ -299,6 +299,7 @@ clear_bss_done:
  	/* Initialize page tables and relocate to virtual addresses */
  	la sp, init_thread_union + THREAD_SIZE
  	XIP_FIXUP_OFFSET sp
+	addi sp, sp, -PT_SIZE
  #ifdef CONFIG_BUILTIN_DTB
  	la a0, __dtb_start
  	XIP_FIXUP_OFFSET a0

arch/riscv/kernel/head.S clear_bss_done: · line 320

Quoted 4 lines
arch/riscv/kernel/head.S +1 -0
@@ -316,6 +317,7 @@ clear_bss_done:
  	/* Restore C environment */
  	la tp, init_task
  	la sp, init_thread_union + THREAD_SIZE
+	addi sp, sp, -PT_SIZE

Good point, Yes we should skip PT_SIZE in stack. I suggest wrap

la sp, init_thread_union + THREAD_SIZE
addi sp, sp, -PT_SIZE

into a macro and give the comment to explain why we need skip PT_SIZE space.

Best Regards
 Guo Ren

author
#20 lore raw

PATCH v10 15/16 riscv: Add V extension to KVM ISA allow list

In reply to Greentime Hu

From: Vincent Chen <[email protected]>

Add V extension to KVM_RISCV_ISA_ALLOWED list to enable VCPU to support V extension.

Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/hwcap.h | 1 +
 arch/riscv/kvm/vcpu.c          | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
arch/riscv/include/asm/hwcap.h +1 -0
index 0734e42f74f2..02ad7cf64d1c 100644
@@ -33,6 +33,7 @@ extern unsigned long elf_hwcap;
 #define RISCV_ISA_EXT_m		('m' - 'a')
 #define RISCV_ISA_EXT_s		('s' - 'a')
 #define RISCV_ISA_EXT_u		('u' - 'a')
+#define RISCV_ISA_EXT_v		('v' - 'a')
 
 /*
  * Increse this to higher value as kernel support more ISA extensions.
arch/riscv/kvm/vcpu.c +2 -1
index 7461f964d20a..c0a3be86d613 100644
@@ -39,7 +39,8 @@ const struct kvm_stats_header kvm_vcpu_stats_header = {
 };
 
 #define KVM_RISCV_ISA_DISABLE_ALLOWED	(riscv_isa_extension_mask(d) | \
-					riscv_isa_extension_mask(f))
+					riscv_isa_extension_mask(f)  | \
+					riscv_isa_extension_mask(v))
 
 #define KVM_RISCV_ISA_DISABLE_NOT_ALLOWED	(riscv_isa_extension_mask(a) | \
 						riscv_isa_extension_mask(c) | \
Signature · 1 line
2.17.1
author
#21 lore raw

PATCH v10 16/16 riscv: KVM: Add vector lazy save/restore support

In reply to Greentime Hu

From: Vincent Chen <[email protected]>

This patch adds vector context save/restore for guest VCPUs. To reduce the impact on KVM performance, the implementation imitates the FP context switch mechanism to lazily store and restore the vector context only when the kernel enters/exits the in-kernel run loop and not during the KVM world switch.

Signed-off-by Vincent Chen <[email protected]>
Signed-off-by Greentime Hu <[email protected]>
---
 arch/riscv/include/asm/kvm_host.h        |   2 +
 arch/riscv/include/asm/kvm_vcpu_vector.h |  65 +++++++++
 arch/riscv/include/uapi/asm/kvm.h        |   6 +
 arch/riscv/kernel/asm-offsets.c          |   7 +
 arch/riscv/kvm/Makefile                  |   1 +
 arch/riscv/kvm/vcpu.c                    |  30 ++++
 arch/riscv/kvm/vcpu_switch.S             |  69 +++++++++
 arch/riscv/kvm/vcpu_vector.c             | 173 +++++++++++++++++++++++
 8 files changed, 353 insertions(+)
 create mode 100644 arch/riscv/include/asm/kvm_vcpu_vector.h
 create mode 100644 arch/riscv/kvm/vcpu_vector.c
arch/riscv/include/asm/kvm_host.h +2 -0
index cd4bbcecb0fb..60fb3de61f5c 100644
@@ -14,6 +14,7 @@
 #include <linux/kvm_types.h>
 #include <asm/csr.h>
 #include <asm/kvm_vcpu_fp.h>
+#include <asm/kvm_vcpu_vector.h>
 #include <asm/kvm_vcpu_timer.h>
 
 #define KVM_MAX_VCPUS			\

@@ -122,6 +123,7 @@ struct kvm_cpu_context {
 	unsigned long sstatus;
 	unsigned long hstatus;
 	union __riscv_fp_state fp;
+	struct __riscv_v_state vector;
 };
 
 struct kvm_vcpu_csr {
arch/riscv/include/asm/kvm_vcpu_vector.h new file +65 -0

65 lines not on this page · raw message

arch/riscv/include/uapi/asm/kvm.h +6 -0
index f808ad1ce500..b16e2f72eeee 100644
@@ -123,6 +123,12 @@ struct kvm_riscv_timer {
 #define KVM_REG_RISCV_FP_D_REG(name)	\
 		(offsetof(struct __riscv_d_ext_state, name) / sizeof(__u64))
 
+/* V extension registers are mapped as type 7 */
+#define KVM_REG_RISCV_VECTOR		(0x07 << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_VECTOR_CSR_REG(name)	\
+		(offsetof(struct __riscv_v_state, name) / sizeof(unsigned long))
+#define KVM_REG_RISCV_VECTOR_REG(n)	\
+		((n) + sizeof(struct __riscv_v_state) / sizeof(unsigned long))
 #endif
 
 #endif /* __LINUX_KVM_RISCV_H */
arch/riscv/kernel/asm-offsets.c +7 -0
index 80316ef7bb78..2540b9146072 100644
@@ -278,6 +278,13 @@ void asm_offsets(void)
 	OFFSET(KVM_ARCH_FP_D_F31, kvm_cpu_context, fp.d.f[31]);
 	OFFSET(KVM_ARCH_FP_D_FCSR, kvm_cpu_context, fp.d.fcsr);
 
+	/* V extension */
+
+	OFFSET(KVM_ARCH_VECTOR_VSTART, kvm_cpu_context, vector.vstart);
+	OFFSET(KVM_ARCH_VECTOR_VL, kvm_cpu_context, vector.vl);
+	OFFSET(KVM_ARCH_VECTOR_VTYPE, kvm_cpu_context, vector.vtype);
+	OFFSET(KVM_ARCH_VECTOR_VCSR, kvm_cpu_context, vector.vcsr);
+	OFFSET(KVM_ARCH_VECTOR_DATAP, kvm_cpu_context, vector.datap);
 	/*
 	 * THREAD_{F,X}* might be larger than a S-type offset can handle, but
 	 * these are used in performance-sensitive assembly so we can't resort
arch/riscv/kvm/Makefile +1 -0
index e5c56182f48f..acf39cd87fbb 100644
@@ -17,6 +17,7 @@ kvm-y += mmu.o
 kvm-y += vcpu.o
 kvm-y += vcpu_exit.o
 kvm-y += vcpu_fp.o
+kvm-y += vcpu_vector.o
 kvm-y += vcpu_switch.o
 kvm-y += vcpu_sbi.o
 kvm-$(CONFIG_RISCV_SBI_V01) += vcpu_sbi_v01.o
arch/riscv/kvm/vcpu.c +30 -0
index c0a3be86d613..c3e941dffcd4 100644
@@ -19,6 +19,7 @@
 #include <linux/kvm_host.h>
 #include <asm/csr.h>
 #include <asm/hwcap.h>
+#include <asm/switch_to.h>
 
 const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = {
 	KVM_GENERIC_VCPU_STATS(),

@@ -74,6 +75,8 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu)
 
 	kvm_riscv_vcpu_fp_reset(vcpu);
 
+	kvm_riscv_vcpu_vector_reset(vcpu);
+
 	kvm_riscv_vcpu_timer_reset(vcpu);
 
 	WRITE_ONCE(vcpu->arch.irqs_pending, 0);

@@ -110,6 +113,15 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	cntx->hstatus |= HSTATUS_SPVP;
 	cntx->hstatus |= HSTATUS_SPV;
 
+	if (has_vector()) {
+		cntx->vector.datap = kmalloc(riscv_vsize, GFP_KERNEL);
+		if (!cntx->vector.datap)
+			return -ENOMEM;
+		vcpu->arch.host_context.vector.datap = kzalloc(riscv_vsize, GFP_KERNEL);
+		if (!vcpu->arch.host_context.vector.datap)
+			return -ENOMEM;
+	}
+
 	/* By default, make CY, TM, and IR counters accessible in VU mode */
 	reset_csr->scounteren = 0x7;
 

@@ -140,6 +152,9 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 	/* Free unused pages pre-allocated for Stage2 page table mappings */
 	kvm_mmu_free_memory_cache(&vcpu->arch.mmu_page_cache);
+
+	/* Free vector context space for host and guest kernel */
+	kvm_riscv_vcpu_free_vector_context(vcpu);
 }
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)

@@ -383,6 +398,9 @@ static int kvm_riscv_vcpu_set_reg(struct kvm_vcpu *vcpu,
 	else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_FP_D)
 		return kvm_riscv_vcpu_set_reg_fp(vcpu, reg,
 						 KVM_REG_RISCV_FP_D);
+	else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_VECTOR)
+		return kvm_riscv_vcpu_set_reg_vector(vcpu, reg,
+						 KVM_REG_RISCV_VECTOR);
 
 	return -EINVAL;
 }

@@ -404,6 +422,9 @@ static int kvm_riscv_vcpu_get_reg(struct kvm_vcpu *vcpu,
 	else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_FP_D)
 		return kvm_riscv_vcpu_get_reg_fp(vcpu, reg,
 						 KVM_REG_RISCV_FP_D);
+	else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_VECTOR)
+		return kvm_riscv_vcpu_get_reg_vector(vcpu, reg,
+						 KVM_REG_RISCV_VECTOR);
 
 	return -EINVAL;
 }

@@ -643,6 +664,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	kvm_riscv_vcpu_host_fp_save(&vcpu->arch.host_context);
 	kvm_riscv_vcpu_guest_fp_restore(&vcpu->arch.guest_context,
 					vcpu->arch.isa);
+	kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context);
+	kvm_riscv_vcpu_guest_vector_restore(&vcpu->arch.guest_context,
+					    vcpu->arch.isa);
 
 	vcpu->cpu = cpu;
 }

@@ -657,6 +681,12 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
 				     vcpu->arch.isa);
 	kvm_riscv_vcpu_host_fp_restore(&vcpu->arch.host_context);
 
+	kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context,
+					 vcpu->arch.isa);
+	kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context);
+
+	csr_write(CSR_HGATP, 0);
+
 	csr->vsstatus = csr_read(CSR_VSSTATUS);
 	csr->vsie = csr_read(CSR_VSIE);
 	csr->vstvec = csr_read(CSR_VSTVEC);
arch/riscv/kvm/vcpu_switch.S +69 -0

72 lines not on this page · raw message

arch/riscv/kvm/vcpu_vector.c new file +173 -0

173 lines not on this page · raw message

Signature · 1 line
2.17.1