Skip to contentSkip to conversation

KVM: arm64: selftests: Introduce arch_timer selftest

PATCH Superseded Raghavendra Rao Ananta 3 participants 15 messages Aug 13, 2021 – Aug 16, 2021

Subsystems (3)

v1 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

cover KVM: arm64: selftests: Introduce arch_timer selftest

Hello,

The patch series adds a KVM selftest to validate the behavior of ARM's generic timer (patch-10). The test programs the timer IRQs periodically, and for each interrupt, it validates the behaviour against the architecture specifications. The test further provides a command-line interface to configure the number of vCPUs, the period of the timer, and the number of iterations that the test has to run for.

Since the test heavily depends on interrupts, the patch series also adds a basic support for ARM Generic Interrupt Controller v3 (GICv3) to the KVM's aarch64 selftest framework (patch-9).

Furthermore, additional processor utilities such as accessing the MMIO (via readl/writel), read/write to assembler unsupported registers, basic delay generation, enable/disable local IRQs, spinlock support, and so on, are also introduced that the test/GICv3 takes advantage of. These are presented in patches 1 through 8.

The patch series, specifically the library support, is derived from the kvm-unit-tests and the kernel itself.

Regards,
Raghavendra

Raghavendra Rao Ananta (10):
KVM: arm64: selftests: Add MMIO readl/writel support
KVM: arm64: selftests: Add write_sysreg_s and read_sysreg_s
KVM: arm64: selftests: Add support for cpu_relax
KVM: arm64: selftests: Add basic support for arch_timers
KVM: arm64: selftests: Add basic support to generate delays
KVM: arm64: selftests: Add support to disable and enable local IRQs
KVM: arm64: selftests: Add support to get the vcpuid from MPIDR_EL1
KVM: arm64: selftests: Add light-weight spinlock support
KVM: arm64: selftests: Add basic GICv3 support
KVM: arm64: selftests: Add arch_timer test

 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   3 +-
 .../selftests/kvm/aarch64/arch_timer.c        | 382 ++++++++++++++++++
 .../kvm/include/aarch64/arch_timer.h          | 138 +++++++
 .../selftests/kvm/include/aarch64/delay.h     |  25 ++
 .../selftests/kvm/include/aarch64/gic.h       |  21 +
 .../selftests/kvm/include/aarch64/processor.h | 140 ++++++-
 .../selftests/kvm/include/aarch64/spinlock.h  |  13 +
 tools/testing/selftests/kvm/lib/aarch64/gic.c |  93 +++++
 .../selftests/kvm/lib/aarch64/gic_private.h   |  21 +
 .../selftests/kvm/lib/aarch64/gic_v3.c        | 240 +++++++++++
 .../selftests/kvm/lib/aarch64/gic_v3.h        |  70 ++++
 .../selftests/kvm/lib/aarch64/spinlock.c      |  27 ++
 13 files changed, 1172 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/arch_timer.c
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/arch_timer.h
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/delay.h
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/gic.h
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/spinlock.h
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic.c
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic_private.h
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic_v3.c
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic_v3.h
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/spinlock.c
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#1 lore raw

PATCH 1/10 KVM: arm64: selftests: Add MMIO readl/writel support

Define the readl() and writel() functions for the guests to access (4-byte) the MMIO region.

The routines, and their dependents, are inspired from the kernel's arch/arm64/include/asm/io.h and arch/arm64/include/asm/barrier.h.

---
 .../selftests/kvm/include/aarch64/processor.h | 45 ++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/include/aarch64/processor.h +44 -1
index 27dc5c2e56b9..14f68bf55036 100644
@@ -127,6 +127,49 @@ void vm_install_sync_handler(struct kvm_vm *vm,
 	val;								  \
 })
 
-#define isb()	asm volatile("isb" : : : "memory")
+#define isb()		asm volatile("isb" : : : "memory")
+#define dsb(opt)	asm volatile("dsb " #opt : : : "memory")
+#define dmb(opt)	asm volatile("dmb " #opt : : : "memory")
+
+#define dma_wmb()	dmb(oshst)
+#define __iowmb()	dma_wmb()
+
+#define dma_rmb()	dmb(oshld)
+
+#define __iormb(v)							\
+({									\
+	unsigned long tmp;						\
+									\
+	dma_rmb();							\
+									\
+	/*								\
+	 * Courtesy of arch/arm64/include/asm/io.h:			\
+	 * Create a dummy control dependency from the IO read to any	\
+	 * later instructions. This ensures that a subsequent call	\
+	 * to udelay() will be ordered due to the ISB in __delay().	\
+	 */								\
+	asm volatile("eor	%0, %1, %1\n"				\
+		     "cbnz	%0, ."					\
+		     : "=r" (tmp) : "r" ((unsigned long)(v))		\
+		     : "memory");					\
+})
+
+static __always_inline void __raw_writel(u32 val, volatile void *addr)
+{
+	asm volatile("str %w0, [%1]" : : "rZ" (val), "r" (addr));
+}
+
+static __always_inline u32 __raw_readl(const volatile void *addr)
+{
+	u32 val;
+	asm volatile("ldr %w0, [%1]" : "=r" (val) : "r" (addr));
+	return val;
+}
+
+#define writel_relaxed(v,c)	((void)__raw_writel((__force u32)cpu_to_le32(v),(c)))
+#define readl_relaxed(c)	({ u32 __r = le32_to_cpu((__force __le32)__raw_readl(c)); __r; })
+
+#define writel(v,c)		({ __iowmb(); writel_relaxed((v),(c));})
+#define readl(c)		({ u32 __v = readl_relaxed(c); __iormb(__v); __v; })
 
 #endif /* SELFTEST_KVM_PROCESSOR_H */
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#2 lore raw

PATCH 2/10 KVM: arm64: selftests: Add write_sysreg_s and read_sysreg_s

In reply to Raghavendra Rao Ananta

For register names that are unsupported by the assembler or the ones without architectural names, add the macros write_sysreg_s and read_sysreg_s to support them.

The functionality is derived from kvm-unit-tests and kernel's arch/arm64/include/asm/sysreg.h.

---
 .../selftests/kvm/include/aarch64/processor.h | 61 +++++++++++++++++++
 1 file changed, 61 insertions(+)
tools/testing/selftests/kvm/include/aarch64/processor.h +61 -0
index 14f68bf55036..b4bbce837288 100644
@@ -115,6 +115,67 @@ void vm_install_exception_handler(struct kvm_vm *vm,
 void vm_install_sync_handler(struct kvm_vm *vm,
 		int vector, int ec, handler_fn handler);
 
+/*
+ * ARMv8 ARM reserves the following encoding for system registers:
+ * (Ref: ARMv8 ARM, Section: "System instruction class encoding overview",
+ *  C5.2, version:ARM DDI 0487A.f)
+ *	[20-19] : Op0
+ *	[18-16] : Op1
+ *	[15-12] : CRn
+ *	[11-8]  : CRm
+ *	[7-5]   : Op2
+ */
+#define Op0_shift	19
+#define Op0_mask	0x3
+#define Op1_shift	16
+#define Op1_mask	0x7
+#define CRn_shift	12
+#define CRn_mask	0xf
+#define CRm_shift	8
+#define CRm_mask	0xf
+#define Op2_shift	5
+#define Op2_mask	0x7
+
+/*
+ * When accessed from guests, the ARM64_SYS_REG() doesn't work since it
+ * generates a different encoding for additional KVM processing, and is
+ * only suitable for userspace to access the register via ioctls.
+ * Hence, define a 'pure' sys_reg() here to generate the encodings as per spec.
+ */
+#define sys_reg(op0, op1, crn, crm, op2) \
+	(((op0) << Op0_shift) | ((op1) << Op1_shift) | \
+	 ((crn) << CRn_shift) | ((crm) << CRm_shift) | \
+	 ((op2) << Op2_shift))
+
+asm(
+"	.irp	num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
+"	.equ	.L__reg_num_x\\num, \\num\n"
+"	.endr\n"
+"	.equ	.L__reg_num_xzr, 31\n"
+"\n"
+"	.macro	mrs_s, rt, sreg\n"
+"	.inst	0xd5200000|(\\sreg)|(.L__reg_num_\\rt)\n"
+"	.endm\n"
+"\n"
+"	.macro	msr_s, sreg, rt\n"
+"	.inst	0xd5000000|(\\sreg)|(.L__reg_num_\\rt)\n"
+"	.endm\n"
+);
+
+/*
+ * read_sysreg_s() and write_sysreg_s()'s 'reg' has to be encoded via sys_reg()
+ */
+#define read_sysreg_s(reg) ({						\
+	u64 __val;							\
+	asm volatile("mrs_s %0, "__stringify(reg) : "=r" (__val));	\
+	__val;								\
+})
+
+#define write_sysreg_s(reg, val) do {					\
+	u64 __val = (u64)val;						\
+	asm volatile("msr_s "__stringify(reg) ", %x0" : : "rZ" (__val));\
+} while (0)
+
 #define write_sysreg(reg, val)						  \
 ({									  \
 	u64 __val = (u64)(val);						  \
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#3 lore raw

PATCH 3/10 KVM: arm64: selftests: Add support for cpu_relax

In reply to Raghavendra Rao Ananta

Implement the guest helper routine, cpu_relax(), to yield the processor to other tasks.

The function was derived from
arch/arm64/include/asm/vdso/processor.h.

---
 tools/testing/selftests/kvm/include/aarch64/processor.h | 5 +++++
 1 file changed, 5 insertions(+)
tools/testing/selftests/kvm/include/aarch64/processor.h +5 -0
index b4bbce837288..c83ff99282ed 100644
@@ -188,6 +188,11 @@ asm(
 	val;								  \
 })
 
+static inline void cpu_relax(void)
+{
+	asm volatile("yield" ::: "memory");
+}
+
 #define isb()		asm volatile("isb" : : : "memory")
 #define dsb(opt)	asm volatile("dsb " #opt : : : "memory")
 #define dmb(opt)	asm volatile("dmb " #opt : : : "memory")
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#4 lore raw

PATCH 4/10 KVM: arm64: selftests: Add basic support for arch_timers

In reply to Raghavendra Rao Ananta

Add a minimalistic library support to access the virtual timers, that can be used for simple timing functionalities, such as introducing delays in the guest.

---
 .../kvm/include/aarch64/arch_timer.h          | 138 ++++++++++++++++++
 1 file changed, 138 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/arch_timer.h
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +138 -0
new file mode 100644
index 000000000000..e6144ab95348
@@ -0,0 +1,138 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Generic Interrupt Controller (GIC) specific defines
+ */
+
+#ifndef SELFTEST_KVM_ARCH_TIMER_H
+#define SELFTEST_KVM_ARCH_TIMER_H
+
+#include <linux/sizes.h>
+
+#include "processor.h"
+
+enum arch_timer {
+	VIRTUAL,
+	PHYSICAL,
+};
+
+#define CTL_ENABLE	(1 << 0)
+#define CTL_ISTATUS	(1 << 2)
+#define CTL_IMASK	(1 << 1)
+
+#define msec_to_cycles(msec)	\
+	(timer_get_cntfrq() * (uint64_t)(msec) / 1000)
+
+#define usec_to_cycles(usec)	\
+	(timer_get_cntfrq() * (uint64_t)(usec) / 1000000)
+
+#define cycles_to_usec(cycles) \
+	((uint64_t)(cycles) * 1000000 / timer_get_cntfrq())
+
+static inline uint32_t timer_get_cntfrq(void)
+{
+	return read_sysreg(cntfrq_el0);
+}
+
+static inline uint64_t timer_get_cntct(enum arch_timer timer)
+{
+	isb();
+
+	switch (timer) {
+	case VIRTUAL:
+		return read_sysreg(cntvct_el0);
+	case PHYSICAL:
+		return read_sysreg(cntpct_el0);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	/* We should not reach here */
+	return 0;
+}
+
+static inline void timer_set_cval(enum arch_timer timer, uint64_t cval)
+{
+	switch (timer) {
+	case VIRTUAL:
+		return write_sysreg(cntv_cval_el0, cval);
+	case PHYSICAL:
+		return write_sysreg(cntp_cval_el0, cval);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	isb();
+}
+
+static inline uint64_t timer_get_cval(enum arch_timer timer)
+{
+	switch (timer) {
+	case VIRTUAL:
+		return read_sysreg(cntv_cval_el0);
+	case PHYSICAL:
+		return read_sysreg(cntp_cval_el0);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	/* We should not reach here */
+	return 0;
+}
+
+static inline void timer_set_tval(enum arch_timer timer, uint32_t tval)
+{
+	switch (timer) {
+	case VIRTUAL:
+		return write_sysreg(cntv_tval_el0, tval);
+	case PHYSICAL:
+		return write_sysreg(cntp_tval_el0, tval);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	isb();
+}
+
+static inline void timer_set_ctl(enum arch_timer timer, uint32_t ctl)
+{
+	switch (timer) {
+	case VIRTUAL:
+		return write_sysreg(cntv_ctl_el0, ctl);
+	case PHYSICAL:
+		return write_sysreg(cntp_ctl_el0, ctl);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	isb();
+}
+
+static inline uint32_t timer_get_ctl(enum arch_timer timer)
+{
+	switch (timer) {
+	case VIRTUAL:
+		return read_sysreg(cntv_ctl_el0);
+	case PHYSICAL:
+		return read_sysreg(cntp_ctl_el0);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	/* We should not reach here */
+	return 0;
+}
+
+static inline void timer_set_next_cval_ms(enum arch_timer timer, uint32_t msec)
+{
+	uint64_t now_ct = timer_get_cntct(timer);
+	uint64_t next_ct = now_ct + msec_to_cycles(msec);
+
+	timer_set_cval(timer, next_ct);
+}
+
+static inline void timer_set_next_tval_ms(enum arch_timer timer, uint32_t msec)
+{
+	timer_set_tval(timer, msec_to_cycles(msec));
+}
+
+#endif /* SELFTEST_KVM_ARCH_TIMER_H */
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
reviewer
#5 lore raw

Re: [PATCH 04/10] KVM: arm64: selftests: Add basic support for arch_timers

Quoted 16 lines · Raghavendra Rao Ananta

Add a minimalistic library support to access the virtual timers, that can be used for simple timing functionalities, such as introducing delays in the guest.

---
 .../kvm/include/aarch64/arch_timer.h          | 138 ++++++++++++++++++
 1 file changed, 138 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/arch_timer.h
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +0 -0
new file mode 100644
index 000000000000..e6144ab95348

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 3

Quoted 3 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +3 -0
@@ -0,0 +1,138 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Generic Interrupt Controller (GIC) specific defines

This isn't GIC specific, but arch timer.

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 9

Quoted 6 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +6 -0
+ */
+
+#ifndef SELFTEST_KVM_ARCH_TIMER_H
+#define SELFTEST_KVM_ARCH_TIMER_H
+
+#include <linux/sizes.h>

Do we really need it?

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 20

Quoted 11 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +11 -0
+
+#include "processor.h"
+
+enum arch_timer {
+	VIRTUAL,
+	PHYSICAL,
+};
+
+#define CTL_ENABLE	(1 << 0)
+#define CTL_ISTATUS	(1 << 2)
+#define CTL_IMASK	(1 << 1)

nitpick: Move CTL_IMASK before CTL_ISTATUS ?

Quoted 34 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +34 -0
+
+#define msec_to_cycles(msec)	\
+	(timer_get_cntfrq() * (uint64_t)(msec) / 1000)
+
+#define usec_to_cycles(usec)	\
+	(timer_get_cntfrq() * (uint64_t)(usec) / 1000000)
+
+#define cycles_to_usec(cycles) \
+	((uint64_t)(cycles) * 1000000 / timer_get_cntfrq())
+
+static inline uint32_t timer_get_cntfrq(void)
+{
+	return read_sysreg(cntfrq_el0);
+}
+
+static inline uint64_t timer_get_cntct(enum arch_timer timer)
+{
+	isb();
+
+	switch (timer) {
+	case VIRTUAL:
+		return read_sysreg(cntvct_el0);
+	case PHYSICAL:
+		return read_sysreg(cntpct_el0);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	/* We should not reach here */
+	return 0;
+}
+
+static inline void timer_set_cval(enum arch_timer timer, uint64_t cval)
+{

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 64

Quoted 10 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +10 -0
+	switch (timer) {
+	case VIRTUAL:
+		return write_sysreg(cntv_cval_el0, cval);
+	case PHYSICAL:
+		return write_sysreg(cntp_cval_el0, cval);
+	default:
+		GUEST_ASSERT_1(0, timer);
+	}
+
+	isb();

ISB should be performed before 'return'. And the same for timer_set_{tval,ctl}.

Thanks,
Zenghui

#6 lore raw

Re: [PATCH 04/10] KVM: arm64: selftests: Add basic support for arch_timers

Hi Zenghui,

Quoted 12 lines · Zenghui Yu
Quoted 10 lines · Raghavendra Rao Ananta

Add a minimalistic library support to access the virtual timers, that can be used for simple timing functionalities, such as introducing delays in the guest.

---
 .../kvm/include/aarch64/arch_timer.h          | 138 ++++++++++++++++++
 1 file changed, 138 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/arch_timer.h
Quoted 11 lines

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 3

Quoted 9 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +3 -0
new file mode 100644
index 000000000000..e6144ab95348
@@ -0,0 +1,138 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Generic Interrupt Controller (GIC) specific defines

This isn't GIC specific, but arch timer.

You are right. That's my bad.

Quoted 10 lines

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 9

Quoted 8 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +6 -0
+ */
+
+#ifndef SELFTEST_KVM_ARCH_TIMER_H
+#define SELFTEST_KVM_ARCH_TIMER_H
+
+#include <linux/sizes.h>

Do we really need it?

I must have left it from some code re-org. We don't need it anymore. Will remove.

Quoted 15 lines

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 20

Quoted 13 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +11 -0
+
+#include "processor.h"
+
+enum arch_timer {
+     VIRTUAL,
+     PHYSICAL,
+};
+
+#define CTL_ENABLE   (1 << 0)
+#define CTL_ISTATUS  (1 << 2)
+#define CTL_IMASK    (1 << 1)

nitpick: Move CTL_IMASK before CTL_ISTATUS ?

Sure, that's cleaner!

Quoted 41 lines
Quoted 41 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +39 -0
+
+#define msec_to_cycles(msec) \
+     (timer_get_cntfrq() * (uint64_t)(msec) / 1000)
+
+#define usec_to_cycles(usec) \
+     (timer_get_cntfrq() * (uint64_t)(usec) / 1000000)
+
+#define cycles_to_usec(cycles) \
+     ((uint64_t)(cycles) * 1000000 / timer_get_cntfrq())
+
+static inline uint32_t timer_get_cntfrq(void)
+{
+     return read_sysreg(cntfrq_el0);
+}
+
+static inline uint64_t timer_get_cntct(enum arch_timer timer)
+{
+     isb();
+
+     switch (timer) {
+     case VIRTUAL:
+             return read_sysreg(cntvct_el0);
+     case PHYSICAL:
+             return read_sysreg(cntpct_el0);
+     default:
+             GUEST_ASSERT_1(0, timer);
+     }
+
+     /* We should not reach here */
+     return 0;
+}
+
+static inline void timer_set_cval(enum arch_timer timer, uint64_t cval)
+{
+     switch (timer) {
+     case VIRTUAL:
+             return write_sysreg(cntv_cval_el0, cval);
+     case PHYSICAL:
+             return write_sysreg(cntp_cval_el0, cval);
Quoted 8 lines

tools/testing/selftests/kvm/include/aarch64/arch_timer.h line 64

Quoted 5 lines
tools/testing/selftests/kvm/include/aarch64/arch_timer.h new file +5 -0
+     default:
+             GUEST_ASSERT_1(0, timer);
+     }
+
+     isb();

ISB should be performed before 'return'. And the same for timer_set_{tval,ctl}.

Seems like a copy-paste error on my side. The timer_set_*() functions shouldn't even have a 'return'. Thanks for catching this! Will fix it.

Quoted 4 lines

Thanks,
Zenghui

Regards,
Raghavendra

#7 lore raw

PATCH 5/10 KVM: arm64: selftests: Add basic support to generate delays

In reply to Raghavendra Rao Ananta

Add udelay() support to generate a delay in the guest.

The routines are derived and simplified from kernel's arch/arm64/lib/delay.c.

---
 .../selftests/kvm/include/aarch64/delay.h     | 25 +++++++++++++++++++
 1 file changed, 25 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/delay.h
tools/testing/selftests/kvm/include/aarch64/delay.h new file +25 -0
new file mode 100644
index 000000000000..329e4f5079ea
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM simple delay routines
+ */
+
+#ifndef SELFTEST_KVM_ARM_DELAY_H
+#define SELFTEST_KVM_ARM_DELAY_H
+
+#include "arch_timer.h"
+
+static inline void __delay(uint64_t cycles)
+{
+	enum arch_timer timer = VIRTUAL;
+	uint64_t start = timer_get_cntct(timer);
+
+	while ((timer_get_cntct(timer) - start) < cycles)
+		cpu_relax();
+}
+
+static inline void udelay(unsigned long usec)
+{
+	__delay(usec_to_cycles(usec));
+}
+
+#endif /* SELFTEST_KVM_ARM_DELAY_H */
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#8 lore raw

PATCH 6/10 KVM: arm64: selftests: Add support to disable and enable local IRQs

In reply to Raghavendra Rao Ananta

Add functions local_irq_enable() and local_irq_disable() to enable and disable the IRQs from the guest, respectively.

---
 .../testing/selftests/kvm/include/aarch64/processor.h  | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/testing/selftests/kvm/include/aarch64/processor.h +10 -0
index c83ff99282ed..ae7a079ae180 100644
@@ -238,4 +238,14 @@ static __always_inline u32 __raw_readl(const volatile void *addr)
 #define writel(v,c)		({ __iowmb(); writel_relaxed((v),(c));})
 #define readl(c)		({ u32 __v = readl_relaxed(c); __iormb(__v); __v; })
 
+static inline void local_irq_enable(void)
+{
+	asm volatile("msr daifclr, #3" : : : "memory");
+}
+
+static inline void local_irq_disable(void)
+{
+	asm volatile("msr daifset, #3" : : : "memory");
+}
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#9 lore raw

PATCH 7/10 KVM: arm64: selftests: Add support to get the vcpuid from MPIDR_EL1

In reply to Raghavendra Rao Ananta

At times, such as when in the interrupt handler, the guest wants to get the vCPU-id that it's running on. As a result, introduce get_vcpuid() that parses the MPIDR_EL1 and returns the vcpuid to the requested caller.

---
 .../selftests/kvm/include/aarch64/processor.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
tools/testing/selftests/kvm/include/aarch64/processor.h +19 -0
index ae7a079ae180..e9342e63d05d 100644
@@ -248,4 +248,23 @@ static inline void local_irq_disable(void)
 	asm volatile("msr daifset, #3" : : : "memory");
 }
 
+#define MPIDR_LEVEL_BITS 8
+#define MPIDR_LEVEL_SHIFT(level) (MPIDR_LEVEL_BITS * level)
+#define MPIDR_LEVEL_MASK ((1 << MPIDR_LEVEL_BITS) - 1)
+#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
+	((mpidr >> MPIDR_LEVEL_SHIFT(level)) & MPIDR_LEVEL_MASK)
+
+static inline uint32_t get_vcpuid(void)
+{
+	uint32_t vcpuid = 0;
+	uint64_t mpidr = read_sysreg(mpidr_el1);
+
+	/* KVM limits only 16 vCPUs at level 0 */
+	vcpuid = mpidr & 0x0f;
+	vcpuid |= MPIDR_AFFINITY_LEVEL(mpidr, 1) << 4;
+	vcpuid |= MPIDR_AFFINITY_LEVEL(mpidr, 2) << 12;
+
+	return vcpuid;
+}
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#10 lore raw

PATCH 8/10 KVM: arm64: selftests: Add light-weight spinlock support

In reply to Raghavendra Rao Ananta

Add a simpler version of spinlock support for ARM64 for the guests to use.

The implementation is loosely based on the spinlock implementation in kvm-unit-tests.

---
 tools/testing/selftests/kvm/Makefile          |  2 +-
 .../selftests/kvm/include/aarch64/spinlock.h  | 13 +++++++++
 .../selftests/kvm/lib/aarch64/spinlock.c      | 27 +++++++++++++++++++
 3 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/spinlock.h
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/spinlock.c
tools/testing/selftests/kvm/Makefile +1 -1
index 5832f510a16c..8f6d82b570bd 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/rbtree.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/apic.c lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S lib/aarch64/spinlock.c
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/spinlock.h new file +13 -0
new file mode 100644
index 000000000000..cf0984106d14
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef SELFTEST_KVM_ARM64_SPINLOCK_H
+#define SELFTEST_KVM_ARM64_SPINLOCK_H
+
+struct spinlock {
+	int v;
+};
+
+extern void spin_lock(struct spinlock *lock);
+extern void spin_unlock(struct spinlock *lock);
+
+#endif /* SELFTEST_KVM_ARM64_SPINLOCK_H */
tools/testing/selftests/kvm/lib/aarch64/spinlock.c new file +27 -0
new file mode 100644
index 000000000000..6d66a3dac237
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 Spinlock support
+ */
+#include <stdint.h>
+
+#include "spinlock.h"
+
+void spin_lock(struct spinlock *lock)
+{
+	uint32_t val, res;
+
+	asm volatile(
+	"1:	ldaxr	%w0, [%2]\n"
+	"	cbnz	%w0, 1b\n"
+	"	mov	%w0, #1\n"
+	"	stxr	%w1, %w0, [%2]\n"
+	"	cbnz	%w1, 1b\n"
+	: "=&r" (val), "=&r" (res)
+	: "r" (&lock->v)
+	: "memory");
+}
+
+void spin_unlock(struct spinlock *lock)
+{
+	asm volatile("stlr wzr, [%0]\n"	: : "r" (&lock->v) : "memory");
+}
Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#11 lore raw

PATCH 9/10 KVM: arm64: selftests: Add basic GICv3 support

In reply to Raghavendra Rao Ananta

Add basic support for ARM Generic Interrupt Controller v3. The support provides guests to setup interrupts.

The work is inspired from kvm-unit-tests and the kernel's GIC driver (drivers/irqchip/irq-gic-v3.c).

---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/gic.h       |  21 ++
 tools/testing/selftests/kvm/lib/aarch64/gic.c |  93 +++++++
 .../selftests/kvm/lib/aarch64/gic_private.h   |  21 ++
 .../selftests/kvm/lib/aarch64/gic_v3.c        | 240 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/gic_v3.h        |  70 +++++
 6 files changed, 446 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/include/aarch64/gic.h
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic.c
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic_private.h
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic_v3.c
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/gic_v3.h
tools/testing/selftests/kvm/Makefile +1 -1
index 8f6d82b570bd..a170166334a3 100644
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/rbtree.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/apic.c lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S lib/aarch64/spinlock.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S lib/aarch64/spinlock.c lib/aarch64/gic.c lib/aarch64/gic_v3.c
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
tools/testing/selftests/kvm/include/aarch64/gic.h new file +21 -0
new file mode 100644
index 000000000000..85dd1e53048e
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Generic Interrupt Controller (GIC) specific defines
+ */
+
+#ifndef SELFTEST_KVM_GIC_H
+#define SELFTEST_KVM_GIC_H
+
+enum gic_type {
+	GIC_V3,
+	GIC_TYPE_MAX,
+};
+
+void gic_init(enum gic_type type, unsigned int nr_cpus,
+		void *dist_base, void *redist_base);
+void gic_irq_enable(unsigned int intid);
+void gic_irq_disable(unsigned int intid);
+unsigned int gic_get_and_ack_irq(void);
+void gic_set_eoi(unsigned int intid);
+
+#endif /* SELFTEST_KVM_GIC_H */
tools/testing/selftests/kvm/lib/aarch64/gic.c new file +93 -0

93 lines not on this page · raw message

tools/testing/selftests/kvm/lib/aarch64/gic_private.h new file +21 -0
new file mode 100644
index 000000000000..d81d739433dc
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM Generic Interrupt Controller (GIC) private defines that's only
+ * shared among the GIC library code.
+ */
+
+#ifndef SELFTEST_KVM_GIC_PRIVATE_H
+#define SELFTEST_KVM_GIC_PRIVATE_H
+
+struct gic_common_ops {
+	void (*gic_init)(unsigned int nr_cpus, void *dist_base);
+	void (*gic_cpu_init)(unsigned int cpu, void *redist_base);
+	void (*gic_irq_enable)(unsigned int intid);
+	void (*gic_irq_disable)(unsigned int intid);
+	uint64_t (*gic_read_iar)(void);
+	void (*gic_write_eoir)(uint32_t irq);
+};
+
+extern const struct gic_common_ops gicv3_ops;
+
+#endif /* SELFTEST_KVM_GIC_PRIVATE_H */
tools/testing/selftests/kvm/lib/aarch64/gic_v3.c new file +240 -0

240 lines not on this page · raw message

tools/testing/selftests/kvm/lib/aarch64/gic_v3.h new file +70 -0

70 lines not on this page · raw message

Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#12 lore raw

PATCH 10/10 KVM: arm64: selftests: Add arch_timer test

In reply to Raghavendra Rao Ananta

Add a KVM selftest to validate the arch_timer functionality. Primarily, the test sets up periodic timer interrupts and validates the basic architectural expectations upon its receipt.

The test provides command-line options to configure the period of the timer, number of iterations, and number of vCPUs.

---
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/arch_timer.c        | 382 ++++++++++++++++++
 3 files changed, 384 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/aarch64/arch_timer.c
tools/testing/selftests/kvm/.gitignore +1 -0
index 0709af0144c8..5f6cf0c76cf8 100644
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+/aarch64/arch_timer
 /aarch64/debug-exceptions
 /aarch64/get-reg-list
 /aarch64/vgic_init
tools/testing/selftests/kvm/Makefile +1 -0
index a170166334a3..62a2c3e4b50c 100644
@@ -84,6 +84,7 @@ TEST_GEN_PROGS_x86_64 += set_memory_region_test
 TEST_GEN_PROGS_x86_64 += steal_time
 TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test
 
+TEST_GEN_PROGS_aarch64 += aarch64/arch_timer
 TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/arch_timer.c new file +382 -0

382 lines not on this page · raw message

Signature · 1 line
2.33.0.rc1.237.g0d66db33f3-goog
#13 lore raw
In reply to Raghavendra Rao Ananta
Quoted 14 lines · Raghavendra Rao Ananta

Hello,

The patch series adds a KVM selftest to validate the behavior of ARM's generic timer (patch-10). The test programs the timer IRQs periodically, and for each interrupt, it validates the behaviour against the architecture specifications. The test further provides a command-line interface to configure the number of vCPUs, the period of the timer, and the number of iterations that the test has to run for.

Since the test heavily depends on interrupts, the patch series also adds a basic support for ARM Generic Interrupt Controller v3 (GICv3) to the KVM's aarch64 selftest framework (patch-9).

Quoted 9 lines

Furthermore, additional processor utilities such as accessing the MMIO (via readl/writel), read/write to assembler unsupported registers, basic delay generation, enable/disable local IRQs, spinlock support, and so on, are also introduced that the test/GICv3 takes advantage of. These are presented in patches 1 through 8.

The patch series, specifically the library support, is derived from the kvm-unit-tests and the kernel itself.

Hi Raghavendra,

I appreciate the new support being added to aarch64 kselftests in order to support new tests. I'm curious as to why the kvm-unit-tests timer test wasn't extended instead, though. Also, I'm curious if you've seen any room for improvements to the kvm-unit-tests code and, if so, if you plan to submit patches for those improvements.

Thanks,
drew

#14 lore raw
Quoted 26 lines · Andrew Jones
Quoted 14 lines · Raghavendra Rao Ananta

Hello,

The patch series adds a KVM selftest to validate the behavior of ARM's generic timer (patch-10). The test programs the timer IRQs periodically, and for each interrupt, it validates the behaviour against the architecture specifications. The test further provides a command-line interface to configure the number of vCPUs, the period of the timer, and the number of iterations that the test has to run for.

Since the test heavily depends on interrupts, the patch series also adds a basic support for ARM Generic Interrupt Controller v3 (GICv3) to the KVM's aarch64 selftest framework (patch-9).

Quoted 9 lines

Furthermore, additional processor utilities such as accessing the MMIO (via readl/writel), read/write to assembler unsupported registers, basic delay generation, enable/disable local IRQs, spinlock support, and so on, are also introduced that the test/GICv3 takes advantage of. These are presented in patches 1 through 8.

The patch series, specifically the library support, is derived from the kvm-unit-tests and the kernel itself.

Quoted 7 lines

Hi Raghavendra,

I appreciate the new support being added to aarch64 kselftests in order to support new tests. I'm curious as to why the kvm-unit-tests timer test wasn't extended instead, though. Also, I'm curious if you've seen any room for improvements to the kvm-unit-tests code and, if so, if you plan to submit patches for those improvements.

Hi Andrew,

Interesting question! It's more about ease and flexibility in controlling the guest via the VMM-
Since arch_timer's interface is mostly per-CPU, we'd like to extend this test case to be
more stressful, such as migrating the vCPUs across pCPUs rapidly, or even affining
a large number of vCPUs to a single pCPU, and so on.

On the other hand, since the patch series brings-in a lot of aarch64 goodies with it,
such as interrupt support, it might encourage others to add more arch specific tests
easily :) For example, we also plan to add tests that verifies KVM interface for interrupts,
for which the GIC support in the series would come handy.

I'm still gaining understanding of kvm-unit-tests. However, I'm curious to know your thoughts as
well in-support of kvm-unit-tests.

Unfortunately, I don't have any immediate plans to submit patches on arch_timer for
kvm-unit-tests.

Thanks,
Raghavendra

Quoted 5 lines

Thanks,
drew