Skip to contentSkip to conversation

arm64: GICv3 support

PATCH Superseded Marc Zyngier 8 participants 49 messages Feb 5, 2014 – Mar 15, 2014

Subsystems (7)

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

cover arm64: GICv3 support

GICv3 is the base for a new generation of interrupt controllers designed to overcome some of the most glaring limitations of GICv2.

Some of the features are:
- Support for more than 8 CPUs (a lot more...)
- System registers for CPU interface access (GICC, GICV, GICH)
- Message based interrupts

This patch series currently support:
- Affinity Routing
- System Registers
- Non-Secure Group-1 interrupts only
- KVM support (GICv3 host, GICv2 guest)

What is *not yet* supported in this series:
- LPI/ITS/MSI (patches are available in the below branch, though still in development)
- KVM GICv3 guest support (requires GICv3 distributor emulation)
- Any form of power management
- 32bit systems

To be built, this code requires a fairly recent compiler/binutils combo. Linaro 13.06 seems to do the trick. This has been tested on the ARM FVP model, with non-regressions run on a VExpress TC-2.

The code is also available at the following location:
git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/gicv3

Marc Zyngier (18):
arm64: initial support for GICv3
arm64: GICv3 device tree binding documentation
arm64: boot protocol documentation update for GICv3
KVM: arm/arm64: vgic: move GICv2 registers to their own structure
KVM: ARM: vgic: introduce vgic_ops and LR manipulation primitives
KVM: ARM: vgic: abstract access to the ELRSR bitmap
KVM: ARM: vgic: abstract EISR bitmap access
KVM: ARM: vgic: abstract MISR decoding
KVM: ARM: vgic: move underflow handling to vgic_ops
KVM: ARM: vgic: abstract VMCR access
KVM: ARM: vgic: introduce vgic_enable
KVM: ARM: introduce vgic_params structure
KVM: ARM: vgic: split GICv2 backend from the main vgic code
arm64: KVM: remove __kvm_hyp_code_{start,end} from hyp.S
arm64: KVM: split GICv2 world switch from hyp code
arm64: KVM: move hcr_el2 setting into vgic-v2-switch.S
KVM: ARM: vgic: add the GICv3 backend
arm64: KVM: vgic: add GICv3 world switch

 Documentation/arm64/booting.txt                  |   7 +
 Documentation/devicetree/bindings/arm/gic-v3.txt |  81 +++
 arch/arm/include/asm/kvm_host.h                  |   5 +
 arch/arm/kernel/asm-offsets.c                    |  14 +-
 arch/arm/kvm/Makefile                            |   1 +
 arch/arm/kvm/interrupts_head.S                   |  26 +-
 arch/arm64/Kconfig                               |   1 +
 arch/arm64/include/asm/kvm_asm.h                 |  18 +-
 arch/arm64/include/asm/kvm_host.h                |  21 +
 arch/arm64/include/asm/virt.h                    |   4 +
 arch/arm64/kernel/asm-offsets.c                  |  22 +-
 arch/arm64/kernel/head.S                         |  14 +
 arch/arm64/kernel/hyp-stub.S                     |  41 +-
 arch/arm64/kvm/Makefile                          |   4 +
 arch/arm64/kvm/hyp.S                             | 132 +----
 arch/arm64/kvm/vgic-v2-switch.S                  | 149 +++++
 arch/arm64/kvm/vgic-v3-switch.S                  | 275 ++++++++++
 drivers/irqchip/Kconfig                          |   5 +
 drivers/irqchip/Makefile                         |   1 +
 drivers/irqchip/irq-gic-v3.c                     | 665 +++++++++++++++++++++++
 include/kvm/arm_vgic.h                           | 103 +++-
 include/linux/irqchip/arm-gic-v3.h               | 190 +++++++
 virt/kvm/arm/vgic-v2.c                           | 228 ++++++++
 virt/kvm/arm/vgic-v3.c                           | 220 ++++++++
 virt/kvm/arm/vgic.c                              | 330 ++++++-----
 25 files changed, 2238 insertions(+), 319 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/gic-v3.txt
 create mode 100644 arch/arm64/kvm/vgic-v2-switch.S
 create mode 100644 arch/arm64/kvm/vgic-v3-switch.S
 create mode 100644 drivers/irqchip/irq-gic-v3.c
 create mode 100644 include/linux/irqchip/arm-gic-v3.h
 create mode 100644 virt/kvm/arm/vgic-v2.c
 create mode 100644 virt/kvm/arm/vgic-v3.c
Signature · 1 line
1.8.3.4
#1 lore raw

PATCH 1/18 arm64: initial support for GICv3

The Generic Interrupt Controller (version 3) offers services that are similar to GICv2, with a number of additional features:
- Affinity routing based on the CPU MPIDR (ARE)
- System register for the CPU interfaces (SRE)
- Support for more that 8 CPUs
- Locality-specific Peripheral Interrupts (LPIs)
- Interrupt Translation Services (ITS)

This patch adds preliminary support for GICv3 with ARE and SRE, non-secure mode only. It relies on higher exception levels to grant ARE and SRE access.

Support for LPI and ITS will be added at a later time.

---
 arch/arm64/Kconfig                 |   1 +
 arch/arm64/kernel/head.S           |  14 +
 arch/arm64/kernel/hyp-stub.S       |  41 ++-
 drivers/irqchip/Kconfig            |   5 +
 drivers/irqchip/Makefile           |   1 +
 drivers/irqchip/irq-gic-v3.c       | 665 +++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h | 190 +++++++++++
 7 files changed, 915 insertions(+), 2 deletions(-)
 create mode 100644 drivers/irqchip/irq-gic-v3.c
 create mode 100644 include/linux/irqchip/arm-gic-v3.h
arch/arm64/Kconfig +1 -0
index dd4327f..3a7e26d 100644
@@ -9,6 +9,7 @@ config ARM64
 	select ARM_AMBA
 	select ARM_ARCH_TIMER
 	select ARM_GIC
+	select ARM_GIC_V3
 	select BUILDTIME_EXTABLE_SORT
 	select CLONE_BACKWARDS
 	select COMMON_CLK
arch/arm64/kernel/head.S +14 -0
index 0b281ff..522649b 100644
@@ -22,6 +22,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -183,6 +184,19 @@ CPU_LE(	bic	x0, x0, #(3 << 24)	)	// Clear the EE and E0E bits for EL1
 	msr	cnthctl_el2, x0
 	msr	cntvoff_el2, xzr		// Clear virtual offset
 
+	/* GICv3 system register access */
+	mrs	x0, id_aa64pfr0_el1
+	ubfx	x0, x0, #24, #4
+	cmp	x0, #1
+	b.ne	3f
+
+	mrs	x0, ICC_SRE_EL2			// Set ICC_SRE_EL2.SRE==1
+	orr	x0, x0, #1
+	msr	ICC_SRE_EL2, x0
+	isb					// Make sure SRE is now 1
+	msr	ICH_HCR_EL2, xzr		// Reset ICC_HCR_EL2 to defaults
+
+3:
 	/* Populate ID registers. */
 	mrs	x0, midr_el1
 	mrs	x1, mpidr_el1
arch/arm64/kernel/hyp-stub.S +39 -2

61 lines not on this page · raw message

drivers/irqchip/Kconfig +5 -0
index 61ffdca..5b55c46 100644
@@ -10,6 +10,11 @@ config ARM_GIC
 config GIC_NON_BANKED
 	bool
 
+config ARM_GIC_V3
+	bool
+	select IRQ_DOMAIN
+	select MULTI_IRQ_HANDLER
+
 config ARM_NVIC
 	bool
 	select IRQ_DOMAIN
drivers/irqchip/Makefile +1 -0
index 86b484c..e6ac9fd 100644
@@ -14,6 +14,7 @@ obj-$(CONFIG_ORION_IRQCHIP)		+= irq-orion.o
 obj-$(CONFIG_ARCH_SUNXI)		+= irq-sun4i.o
 obj-$(CONFIG_ARCH_SPEAR3XX)		+= spear-shirq.o
 obj-$(CONFIG_ARM_GIC)			+= irq-gic.o
+obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o
 obj-$(CONFIG_ARM_NVIC)			+= irq-nvic.o
 obj-$(CONFIG_ARM_VIC)			+= irq-vic.o
 obj-$(CONFIG_IMGPDC_IRQ)		+= irq-imgpdc.o
drivers/irqchip/irq-gic-v3.c new file +665 -0

665 lines not on this page · raw message

include/linux/irqchip/arm-gic-v3.h new file +190 -0

190 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#2 lore raw

PATCH 1/18 arm64: initial support for GICv3

Hi Marc

Review

Quoted 2 lines · Marc Zyngier <marc.zyngier <at> arm.com> writes:
(quoted diff) +2 -0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)

I would suggest that this function be renamed (something like gic_base_for_irq?) since it returns dist or redist sgi base address. The name suggests it always returns the dist base address.

Quoted 14 lines
drivers/irqchip/irq-gic-v3.c new file +14 -0
+{
+	if (d->hwirq < 32)	/* SGI+PPI -> SGI_base for this CPU */
+		return gic_data_rdist_sgi_base();
+
+	if (d->hwirq <= 1023)	/* SPI -> dist_base */
+		return gic_data.dist_base;
+
+	if (d->hwirq >= 8192)
+		BUG();		/* LPI Detected!!! */
+
+	return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)

drivers/irqchip/irq-gic-v3.c line 77

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+{
+	return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
+{
+	u32 val;
+
+	do {
+		val = readl_relaxed(base + GICD_CTLR);

Maybe rename GICD_CTLR to GICx_CTLR since it is being used for both the distributor and the redistributor.

drivers/irqchip/irq-gic-v3.c line 79

Quoted 2 lines
drivers/irqchip/irq-gic-v3.c new file +2 -0
+		cpu_relax();
+	} while (val & GICD_CTLR_RWP);

Similar to above GICx_CTLR_RWP

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+
+static int gic_irq_domain_xlate(struct irq_domain *d,
+				struct device_node *controller,
+				const u32 *intspec, unsigned int intsize,
+				unsigned long *out_hwirq, unsigned int *out_type)
+{
+	if (d->of_node != controller)
+		return -EINVAL;

drivers/irqchip/irq-gic-v3.c line 559

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+	if (intsize < 3)
+		return -EINVAL;
+
+	switch(intspec[0]) {
+	case 0:			/* SPI */
+		*out_hwirq = intspec[1] + 32;
+		break;
+	case 1:			/* PPI */
+		*out_hwirq = intspec[1] + 16;
+		break;

I wonder if there is any value to syncing these with the defines in "include/dt-bindings/interrupt-controller/arm-gic.h" somehow.

Thanks
Arnab

#3 lore raw

PATCH 1/18 arm64: initial support for GICv3

Quoted 30 lines · Arnab Basu

Hi Marc

Review

Quoted 2 lines · Marc Zyngier <marc.zyngier <at> arm.com> writes:
(quoted diff) +2 -0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)

I would suggest that this function be renamed (something like gic_base_for_irq?) since it returns dist or redist sgi base address. The name suggests it always returns the dist base address.

Quoted 19 lines
drivers/irqchip/irq-gic-v3.c new file +19 -0
+{
+	if (d->hwirq < 32)	/* SGI+PPI -> SGI_base for this CPU */
+		return gic_data_rdist_sgi_base();
+
+	if (d->hwirq <= 1023)	/* SPI -> dist_base */
+		return gic_data.dist_base;
+
+	if (d->hwirq >= 8192)
+		BUG();		/* LPI Detected!!! */
+
+	return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+	return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 77

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+{
+	u32 val;
+
+	do {
+		val = readl_relaxed(base + GICD_CTLR);

Maybe rename GICD_CTLR to GICx_CTLR since it is being used for both the distributor and the redistributor.

GICx_ makes it sound relevant for GIC CPU interfaces (GICC) as well. Unless it is, maybe something more like GICDR_ for Generic Interrupt Controller Distributor/Redistributor would be best?

Christopher

Signature · 3 lines
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.
#4 lore raw

PATCH 1/18 arm64: initial support for GICv3

In reply to Arnab Basu

Hi Arnab,

Please do not trim the CC list. I missed this email (lost in the traffic), and contributors to the other MLs may also have comments.

Quoted 10 lines · Arnab Basu

Hi Marc

Review

Quoted 2 lines · Marc Zyngier <marc.zyngier <at> arm.com> writes:
(quoted diff) +2 -0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)

I would suggest that this function be renamed (something like gic_base_for_irq?) since it returns dist or redist sgi base address. The name suggests it always returns the dist base address.

I may rename it to gic_get_irq_base.

Quoted 19 lines
Quoted 19 lines
drivers/irqchip/irq-gic-v3.c new file +19 -0
+{
+	if (d->hwirq < 32)	/* SGI+PPI -> SGI_base for this CPU */
+		return gic_data_rdist_sgi_base();
+
+	if (d->hwirq <= 1023)	/* SPI -> dist_base */
+		return gic_data.dist_base;
+
+	if (d->hwirq >= 8192)
+		BUG();		/* LPI Detected!!! */
+
+	return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+	return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 77

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+{
+	u32 val;
+
+	do {
+		val = readl_relaxed(base + GICD_CTLR);

Maybe rename GICD_CTLR to GICx_CTLR since it is being used for both the distributor and the redistributor.

I really hate the "GICx" bit. I think instead, I'll add a comment so people don't get confuse about the name.

Quoted 19 lines

drivers/irqchip/irq-gic-v3.c line 79

Quoted 2 lines
drivers/irqchip/irq-gic-v3.c new file +2 -0
+		cpu_relax();
+	} while (val & GICD_CTLR_RWP);

Similar to above GICx_CTLR_RWP

Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+
+static int gic_irq_domain_xlate(struct irq_domain *d,
+				struct device_node *controller,
+				const u32 *intspec, unsigned int intsize,
+				unsigned long *out_hwirq, unsigned int *out_type)
+{
+	if (d->of_node != controller)
+		return -EINVAL;
+	if (intsize < 3)
+		return -EINVAL;
+
+	switch(intspec[0]) {
+	case 0:			/* SPI */
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 559

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+		*out_hwirq = intspec[1] + 32;
+		break;
+	case 1:			/* PPI */
+		*out_hwirq = intspec[1] + 16;
+		break;

I wonder if there is any value to syncing these with the defines in "include/dt-bindings/interrupt-controller/arm-gic.h" somehow.

I'd expect the whole DTS stuff to vanish from the kernel some day, so I'm keen on not depending on it.

Thanks for the review,

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#5 lore raw

PATCH 1/18 arm64: initial support for GICv3

In reply to Marc Zyngier
Quoted 57 lines · Marc Zyngier

The Generic Interrupt Controller (version 3) offers services that are similar to GICv2, with a number of additional features:
- Affinity routing based on the CPU MPIDR (ARE)
- System register for the CPU interfaces (SRE)
- Support for more that 8 CPUs
- Locality-specific Peripheral Interrupts (LPIs)
- Interrupt Translation Services (ITS)

This patch adds preliminary support for GICv3 with ARE and SRE, non-secure mode only. It relies on higher exception levels to grant ARE and SRE access.

Support for LPI and ITS will be added at a later time.

---
 arch/arm64/Kconfig                 |   1 +
 arch/arm64/kernel/head.S           |  14 +
 arch/arm64/kernel/hyp-stub.S       |  41 ++-
 drivers/irqchip/Kconfig            |   5 +
 drivers/irqchip/Makefile           |   1 +
 drivers/irqchip/irq-gic-v3.c       | 665 +++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h | 190 +++++++++++
 7 files changed, 915 insertions(+), 2 deletions(-)
 create mode 100644 drivers/irqchip/irq-gic-v3.c
 create mode 100644 include/linux/irqchip/arm-gic-v3.h
arch/arm64/Kconfig +1 -0
index dd4327f..3a7e26d 100644
@@ -9,6 +9,7 @@ config ARM64
 	select ARM_AMBA
 	select ARM_ARCH_TIMER
 	select ARM_GIC
+	select ARM_GIC_V3
 	select BUILDTIME_EXTABLE_SORT
 	select CLONE_BACKWARDS
 	select COMMON_CLK
arch/arm64/kernel/head.S +2 -0
index 0b281ff..522649b 100644
@@ -22,6 +22,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -183,6 +184,19 @@ CPU_LE(	bic	x0, x0, #(3 << 24)	)	// Clear the EE and E0E bits for EL1
 	msr	cnthctl_el2, x0
 	msr	cntvoff_el2, xzr		// Clear virtual offset
 
+	/* GICv3 system register access */

arch/arm64/kernel/head.S CPU_LE( bic x0, x0, #(3 << 24) ) // Clear the EE and E0E bits for EL1 · line 197

Quoted 10 lines
arch/arm64/kernel/head.S +10 -0
+	mrs	x0, id_aa64pfr0_el1
+	ubfx	x0, x0, #24, #4
+	cmp	x0, #1
+	b.ne	3f
+
+	mrs	x0, ICC_SRE_EL2			// Set ICC_SRE_EL2.SRE==1
+	orr	x0, x0, #1
+	msr	ICC_SRE_EL2, x0
+	isb					// Make sure SRE is now 1
+	msr	ICH_HCR_EL2, xzr		// Reset ICC_HCR_EL2 to defaults

Why do we not just set ICC_SRE_EL2.Enable here, but instead do the whole exception dance below?

Quoted 22 lines
arch/arm64/kernel/head.S +2 -0
+
+3:
 	/* Populate ID registers. */
 	mrs	x0, midr_el1
 	mrs	x1, mpidr_el1
arch/arm64/kernel/hyp-stub.S +2 -0
index 0959611..c850d1e 100644
@@ -19,6 +19,7 @@
 
 #include <linux/init.h>
 #include <linux/linkage.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -51,16 +52,52 @@ ENDPROC(__hyp_stub_vectors)
 
 	.align 11
 
+#define sysreg_to_iss(op0, op1, crn, crm, op2)	\

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 65

Quoted 10 lines
arch/arm64/kernel/hyp-stub.S +9 -0
+	(((op0 & 3) << 20) | ((op2 & 7 ) << 17) | \
+	 ((op1 & 7 ) << 14) | ((crn & 15) << 10) | \
+	 ((crm & 15) << 1))
+
 el1_sync:
+	// Hack alert: we don't have a stack, and we can't use the one
+	// from EL1 because our MMU is off. Thankfully, we have
+	// tpidr_el2, which is under our complete control, and
+	// tpidr_el0, which can be used as long as we haven't reached
+	// userspace. Oh hum...

Hmmm. I feel like either we should tone down the hack alert claim, because this is after all safe and we just explain what's going on, or we should just allocate a small stack space following el1_sync and set the SP upon entry to this handler?

Hopefully none of the code that will ever call into the sysregs trap ever makes the same assumption that user space is not called yet ;)

Quoted 18 lines
arch/arm64/kernel/hyp-stub.S +9 -2
+	msr	tpidr_el2, x1			// Preserve x1
+
 	mrs	x1, esr_el2
 	lsr	x1, x1, #26
 	cmp	x1, #0x16
-	b.ne	2f				// Not an HVC trap
+	b.ne	3f				// Not an HVC trap
+
+	// Handle getting/setting vbar_el2
 	cbz	x0, 1f
 	msr	vbar_el2, x0			// Set vbar_el2
 	b	2f
 1:	mrs	x0, vbar_el2			// Return vbar_el2
-2:	eret
+2:	mrs	x1, tpidr_el2			// Restore x1
+	eret
+
+3:	cmp	x1, #0x18			// MSR/MRS?

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 91

Quoted 10 lines
arch/arm64/kernel/hyp-stub.S +10 -0
+	b.ne	2b
+
+	// Handle trapped sys reg access
+	msr	tpidr_el0, x0			// Preserve x0
+	mrs	x1, esr_el2
+	ldr	x0, =sysreg_to_iss(3, 7, 15, 15, 7)
+	and	x1, x1, x0
+	ldr	x0, =sysreg_to_iss(3, 0, 12, 12, 5)
+	eor	x0, x0, x1			// Check for ICC_SRE_EL1
+	cbnz	x0, 4f				// Something bad happened

Shouldn't we try to do something here, like go to panic on the exception return here, or not worth it?

Quoted 335 lines
arch/arm64/kernel/hyp-stub.S +9 -0
+
+	// Set ICC_SRE_EL1 access enable
+	mrs	x1, ICC_SRE_EL2
+	orr	x1, x1, #(1 << 3)
+	msr	ICC_SRE_EL2, x1
+
+4:	mrs	x0, tpidr_el0			// Restore x0
+	mrs	x1, tpidr_el2			// Restore x1
+	eret
 ENDPROC(el1_sync)
 
 .macro invalid_vector	label
drivers/irqchip/Kconfig +5 -0
index 61ffdca..5b55c46 100644
@@ -10,6 +10,11 @@ config ARM_GIC
 config GIC_NON_BANKED
 	bool
 
+config ARM_GIC_V3
+	bool
+	select IRQ_DOMAIN
+	select MULTI_IRQ_HANDLER
+
 config ARM_NVIC
 	bool
 	select IRQ_DOMAIN
drivers/irqchip/Makefile +1 -0
index 86b484c..e6ac9fd 100644
@@ -14,6 +14,7 @@ obj-$(CONFIG_ORION_IRQCHIP)		+= irq-orion.o
 obj-$(CONFIG_ARCH_SUNXI)		+= irq-sun4i.o
 obj-$(CONFIG_ARCH_SPEAR3XX)		+= spear-shirq.o
 obj-$(CONFIG_ARM_GIC)			+= irq-gic.o
+obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o
 obj-$(CONFIG_ARM_NVIC)			+= irq-nvic.o
 obj-$(CONFIG_ARM_VIC)			+= irq-vic.o
 obj-$(CONFIG_IMGPDC_IRQ)		+= irq-imgpdc.o
drivers/irqchip/irq-gic-v3.c new file +289 -0
new file mode 100644
index 0000000..487e061
@@ -0,0 +1,665 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/cputype.h>
+#include <asm/exception.h>
+#include <asm/smp_plat.h>
+
+#include "irqchip.h"
+
+struct gic_chip_data {
+	void __iomem		*dist_base;
+	void __iomem		**redist_base;
+	void __percpu __iomem	**rdist;
+	struct irq_domain	*domain;
+	u64			redist_stride;
+	u32			redist_regions;
+	unsigned int		irq_nr;
+};
+
+static struct gic_chip_data gic_data __read_mostly;
+
+#define gic_data_rdist_rd_base()	(*__this_cpu_ptr(gic_data.rdist))
+#define gic_data_rdist_sgi_base()	(gic_data_rdist_rd_base() + SZ_64K)
+
+static DEFINE_RAW_SPINLOCK(dist_lock);
+
+#define DEFAULT_PMR_VALUE	0xf0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)
+{
+	if (d->hwirq < 32)	/* SGI+PPI -> SGI_base for this CPU */
+		return gic_data_rdist_sgi_base();
+
+	if (d->hwirq <= 1023)	/* SPI -> dist_base */
+		return gic_data.dist_base;
+
+	if (d->hwirq >= 8192)
+		BUG();		/* LPI Detected!!! */
+
+	return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+	return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
+{
+	u32 val;
+
+	do {
+		val = readl_relaxed(base + GICD_CTLR);
+		cpu_relax();
+	} while (val & GICD_CTLR_RWP);
+}
+
+/* Wait for completion of a distributor change */
+static void gic_dist_wait_for_rwp(void)
+{
+	gic_do_wait_for_rwp(gic_data.dist_base);
+}
+
+/* Wait for completion of a redistributor change */
+static void gic_redist_wait_for_rwp(void)
+{
+	gic_do_wait_for_rwp(gic_data_rdist_rd_base());
+}
+
+static void gic_wait_for_rwp(int irq)
+{
+	if (irq < 32)
+		gic_redist_wait_for_rwp();
+	else
+		gic_dist_wait_for_rwp();
+}
+
+/* Low level accessors */
+static u64 gic_read_iar(void)
+{
+	u64 irqstat;
+
+	asm volatile("mrs %0, " reg(ICC_IAR1_EL1) : "=r" (irqstat));
+	return irqstat;
+}
+
+static void gic_write_pmr(u64 val)
+{
+	asm volatile("msr " reg(ICC_PMR_EL1) ", %0" : : "r" (val));
+	isb();
+}
+
+static void gic_write_ctlr(u64 val)
+{
+	asm volatile("msr " reg(ICC_CTLR_EL1) ", %0" : : "r" (val));
+	isb();
+}
+
+static void gic_write_grpen1(u64 val)
+{
+	asm volatile("msr " reg(ICC_GRPEN1_EL1) ", %0" : : "r" (val));
+	isb();
+}
+
+static void gic_write_sgi1r(u64 val)
+{
+	asm volatile("msr " reg(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+	isb();
+}
+
+static void gic_enable_sre(void)
+{
+	u64 val;
+
+	/* This access is likely to trap into EL2 */
+	asm volatile("mrs %0, " reg(ICC_SRE_EL1) : "=r" (val));
+	val |= GICC_SRE_EL1_SRE;
+	asm volatile("msr " reg(ICC_SRE_EL1) ", %0" : : "r" (val));
+	isb();
+}
+
+static void gic_enable_redist(void)
+{
+	void __iomem *rbase;
+	u32 val;
+
+	rbase = gic_data_rdist_rd_base();
+
+	/* Wake up this CPU redistributor */
+	val = readl_relaxed(rbase + GICR_WAKER);
+	val &= ~GICR_WAKER_ProcessorSleep;
+	writel_relaxed(val, rbase + GICR_WAKER);
+
+	do {
+		val = readl_relaxed(rbase + GICR_WAKER);
+		cpu_relax();
+	} while (val & GICR_WAKER_ChildrenAsleep);
+}
+
+/*
+ * Routines to acknowledge, disable and enable interrupts
+ */
+static void gic_mask_irq(struct irq_data *d)
+{
+	u32 mask = 1 << (gic_irq(d) % 32);
+
+	raw_spin_lock(&dist_lock);
+	writel_relaxed(mask, gic_dist_base(d) + GICD_ICENABLER + (gic_irq(d) / 32) * 4);
+	gic_wait_for_rwp(gic_irq(d));
+	raw_spin_unlock(&dist_lock);
+}
+
+static void gic_unmask_irq(struct irq_data *d)
+{
+	u32 mask = 1 << (gic_irq(d) % 32);
+
+	raw_spin_lock(&dist_lock);
+	writel_relaxed(mask, gic_dist_base(d) + GICD_ISENABLER + (gic_irq(d) / 32) * 4);
+	gic_wait_for_rwp(gic_irq(d));
+	raw_spin_unlock(&dist_lock);
+}
+
+static void gic_eoi_irq(struct irq_data *d)
+{
+	gic_write_eoir(gic_irq(d));
+}
+
+static int gic_set_type(struct irq_data *d, unsigned int type)
+{
+	void __iomem *base = gic_dist_base(d);
+	unsigned int irq = gic_irq(d);
+	u32 enablemask = 1 << (irq % 32);
+	u32 enableoff = (irq / 32) * 4;
+	u32 confmask = 0x2 << ((irq % 16) * 2);
+	u32 confoff = (irq / 16) * 4;
+	bool enabled = false;
+	u32 val;
+
+	/* Interrupt configuration for SGIs can't be changed */
+	if (irq < 16)
+		return -EINVAL;
+
+	if (type != IRQ_TYPE_LEVEL_HIGH && type != IRQ_TYPE_EDGE_RISING)
+		return -EINVAL;
+
+	raw_spin_lock(&dist_lock);
+
+	val = readl_relaxed(base + GICD_ICFGR + confoff);
+	if (type == IRQ_TYPE_LEVEL_HIGH)
+		val &= ~confmask;
+	else if (type == IRQ_TYPE_EDGE_RISING)
+		val |= confmask;
+
+	/*
+	 * As recommended by the spec, disable the interrupt before changing
+	 * the configuration
+	 */
+	if (readl_relaxed(base + GICD_ISENABLER + enableoff) & enablemask) {
+		writel_relaxed(enablemask, base + GICD_ICENABLER + enableoff);
+		gic_wait_for_rwp(irq);
+		enabled = true;
+	}
+
+	writel_relaxed(val, base + GICD_ICFGR + confoff);
+
+	if (enabled) {
+		writel_relaxed(enablemask, base + GICD_ISENABLER + enableoff);
+		gic_wait_for_rwp(irq);
+	}
+
+	raw_spin_unlock(&dist_lock);
+
+	return 0;
+}
+
+static int gic_retrigger(struct irq_data *d)
+{
+	return -ENXIO;
+}
+
+static u64 gic_mpidr_to_affinity(u64 mpidr)
+{
+	/* Make sure we don't broadcast the interrupt */
+	return mpidr & ~GICD_IROUTER_SPI_MODE_ANY;
+}
+
+static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
+{
+	u64 irqnr;
+
+	do {
+		irqnr = gic_read_iar();
+
+		if (likely(irqnr > 15 && irqnr < 1021)) {
+			irqnr = irq_find_mapping(gic_data.domain, irqnr);
+			handle_IRQ(irqnr, regs);
+			continue;
+		}
+		if (irqnr < 16) {
+			gic_write_eoir(irqnr);
+#ifdef CONFIG_SMP
+			handle_IPI(irqnr, regs);
+#else
+			WARN_ONCE(true, "Unexpected SGI received!\n");
+#endif
+			continue;
+		}
+	} while (irqnr != 0x3ff);
+}
+
+static void __init gic_dist_init(void)
+{
+	unsigned int i;
+	u64 affinity;
+	int gic_irqs = gic_data.irq_nr;
+	void __iomem *base = gic_data.dist_base;
+
+	/* Disable the distributor */
+	writel_relaxed(0, base + GICD_CTLR);
+
+	/*
+	 * Set all global interrupts to be level triggered, active low.
+	 */
+	for (i = 32; i < gic_data.irq_nr; i += 16)
+		writel_relaxed(0, base + GICD_ICFGR + i / 4);

drivers/irqchip/irq-gic-v3.c line 299

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+	/*
+	 * Set priority on all global interrupts.
+	 */
+	for (i = 32; i < gic_irqs; i += 4)
+		writel_relaxed(0xa0a0a0a0, base + GICD_IPRIORITYR + i);
+
+	/*
+	 * Disable all interrupts.  Leave the PPI and SGIs alone
+	 * as these enables are banked registers.

as these are enabled through banked registers?

also, should banked registers be redistributor registers?

drivers/irqchip/irq-gic-v3.c line 307

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+	 */
+	for (i = 32; i < gic_irqs; i += 32)
+		writel_relaxed(0xffffffff, base + GICD_ICENABLER + i / 8);
+
+	gic_dist_wait_for_rwp();
+
+	writel_relaxed(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+		       base + GICD_CTLR);

why set GICD_CTLR_ENABLE_G1 ?

Quoted 9 lines
drivers/irqchip/irq-gic-v3.c new file +9 -0
+
+	/*
+	 * Set all global interrupts to the boot CPU only. ARE must be
+	 * enabled.
+	 */
+	affinity = gic_mpidr_to_affinity(read_cpuid_mpidr());
+	for (i = 32; i < gic_data.irq_nr; i++)
+		writeq_relaxed(affinity, base + GICD_IROUTER + i * 8);
+}

drivers/irqchip/irq-gic-v3.c line 326

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+static int __init gic_populate_rdist(void)
+{
+	u64 mpidr = cpu_logical_map(smp_processor_id());
+	u64 typer;
+	u64 aff;
+	int i;
+
+	aff  = mpidr & ((1 << 24) - 1);
+	aff |= (mpidr >> 8) & (0xffUL << 24);

Some defines would be nice for this if it makes sense in any way.

Quoted 45 lines
drivers/irqchip/irq-gic-v3.c new file +45 -0
+
+	for (i = 0; i < gic_data.redist_regions; i++) {
+		void __iomem *ptr = gic_data.redist_base[i];
+		u32 reg;
+
+		reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
+		if (reg != 0x30 && reg != 0x40) { /* We're in trouble... */
+			pr_warn("No redistributor present @%p\n", ptr);
+			break;
+		}
+
+		do {
+			typer = readq_relaxed(ptr + GICR_TYPER);
+			if ((typer >> 32) == aff) {
+				gic_data_rdist_rd_base() = ptr;
+				pr_info("CPU%d: found redistributor %llx @%p\n",
+					smp_processor_id(),
+					(unsigned long long) mpidr, ptr);
+				return 0;
+			}
+
+			if (gic_data.redist_stride) {
+				ptr += gic_data.redist_stride;
+			} else {
+				ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+				if (typer & GICR_TYPER_VLPIS)
+					ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+			}
+		} while (!(typer & GICR_TYPER_LAST));
+	}
+
+	/* We couldn't even deal with ourselves... */
+	WARN(true, "CPU%d: mpidr %lx has no re-distributor!\n",
+	     smp_processor_id(), (unsigned long)mpidr);
+	return -ENODEV;
+}
+
+static void __init gic_cpu_init(void)
+{
+	void __iomem *rbase;
+	int i;
+
+	/* Register ourselves with the rest of the world */
+	if (gic_populate_rdist())
+		return;

drivers/irqchip/irq-gic-v3.c line 381

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+	gic_enable_redist();
+
+	rbase = gic_data_rdist_sgi_base();
+
+	/*
+	 * Set priority on PPI and SGI interrupts
+	 */
+	for (i = 0; i < 32; i += 4)
+		writel_relaxed(0xa0a0a0a0, rbase + GICR_IPRIORITYR0 + i * 4 / 4);

(i * 4) / 4 == i, last I checked.

drivers/irqchip/irq-gic-v3.c line 385

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+
+	/*
+	 * Disable all PPI interrupts, ensure all SGI interrupts are
+	 * enabled.

Why do we disable all PPIs?

drivers/irqchip/irq-gic-v3.c line 398

Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+	 */
+	writel_relaxed(0xffff0000, rbase + GICR_ICENABLER0);
+	writel_relaxed(0x0000ffff, rbase + GICR_ISENABLER0);
+
+	gic_redist_wait_for_rwp();
+
+	/* Enable system registers */
+	gic_enable_sre();
+
+	/* Set priority mask register */
+	gic_write_pmr(DEFAULT_PMR_VALUE);
+
+	/* EOI drops priority too (mode 0) */

you mean deactives the interrupt too, right? It always performs a priority drop doesn't it?

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+	gic_write_ctlr(GICC_CTLR_EL1_EOImode_drop_dir);
+
+	/* ... and let's hit the road... */
+	gic_write_grpen1(1);
+}
+
+#ifdef CONFIG_SMP
+static int __init gic_secondary_init(struct notifier_block *nfb,

drivers/irqchip/irq-gic-v3.c line 416

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+				     unsigned long action, void *hcpu)
+{
+	if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+		gic_cpu_init();
+	return NOTIFY_OK;
+}
+
+/*
+ * Notifier for enabling the GIC CPU interface. Set an arbitrarily high
+ * priority because the GIC needs to be up before the ARM generic timers.

I actually don't understand this bit, but it also sits in the original code, so I assume it's correct.

Quoted 15 lines
drivers/irqchip/irq-gic-v3.c new file +15 -0
+ */
+static struct notifier_block __initdata gic_cpu_notifier = {
+	.notifier_call = gic_secondary_init,
+	.priority = 100,
+};
+
+static u16 gic_compute_target_list(int *base_cpu, const struct cpumask *mask,
+				   u64 cluster_id)
+{
+	int cpu = *base_cpu;
+	u64 mpidr = cpu_logical_map(cpu);
+	u16 tlist = 0;
+
+	while (cpu < nr_cpu_ids) {
+		/*

drivers/irqchip/irq-gic-v3.c line 441

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+		 * If we ever get a cluster of more than 16 CPUs, just
+		 * scream and skip that CPU.
+		 */
+		if (WARN_ON((mpidr & 0xff) >= 16))
+			goto out;
+
+		tlist |= 1 << (mpidr & 0xf);
+
+		cpu = cpumask_next(cpu, mask);
+		mpidr = cpu_logical_map(cpu);

Are you not accessing memory beyond the __cpu_logical_map array here when the cpu is the last one at the beginning of the loop?

Quoted 32 lines
drivers/irqchip/irq-gic-v3.c new file +32 -0
+
+		if (cluster_id != (mpidr & ~0xffUL)) {
+			cpu--;
+			goto out;
+		}
+	}
+out:
+	*base_cpu = cpu;
+	return tlist;
+}
+
+static void gic_send_sgi(u64 cluster_id, u16 tlist, unsigned int irq)
+{
+	u64 val;
+
+	val  = (cluster_id & 0xff00ff0000UL) << 16; /* Aff3 + Aff2 */
+	val |= (cluster_id & 0xff00) << 8;	    /* Aff1 */
+	val |= irq << 24;
+	val |= tlist;
+
+	pr_debug("CPU%d: ICC_SGI1R_EL1 %llx\n", smp_processor_id(), val);
+	gic_write_sgi1r(val);
+}
+
+static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
+{
+	int cpu;
+
+	if (WARN_ON(irq >= 16))
+		return;
+
+	/*

drivers/irqchip/irq-gic-v3.c line 483

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+	 * Ensure that stores to Normal memory are visible to the
+	 * other CPUs before issuing the IPI.
+	 */
+	dsb();
+
+	for_each_cpu_mask(cpu, *mask) {
+		u64 cluster_id = cpu_logical_map(cpu) & ~0xffUL;
+		u16 tlist;
+
+		tlist = gic_compute_target_list(&cpu, mask, cluster_id);

I think it would be potentially slightly prettier if this function returned the last cpu in the cluster so the loop iterator mangling was kept local to this function and took a pointer to the tlist it was to fill instead, but, this is a minor detail.

Quoted 44 lines
drivers/irqchip/irq-gic-v3.c new file +44 -0
+		gic_send_sgi(cluster_id, tlist, irq);
+	}
+}
+
+static void gic_smp_init(void)
+{
+	set_smp_cross_call(gic_raise_softirq);
+	register_cpu_notifier(&gic_cpu_notifier);
+}
+
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+			    bool force)
+{
+	unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
+	void __iomem *reg;
+	u64 val;
+
+	if (gic_irq(d) < 32)
+		return -EINVAL;
+
+	reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+	val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+	writeq_relaxed(val, reg);
+
+	return IRQ_SET_MASK_OK;
+}
+#else
+#define gic_set_affinity	NULL
+#define gic_smp_init()		do { } while(0)
+#endif
+
+static struct irq_chip gic_chip = {
+	.name			= "GICv3",
+	.irq_mask		= gic_mask_irq,
+	.irq_unmask		= gic_unmask_irq,
+	.irq_eoi		= gic_eoi_irq,
+	.irq_set_type		= gic_set_type,
+	.irq_retrigger		= gic_retrigger,
+	.irq_set_affinity	= gic_set_affinity,
+};
+
+static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
+				irq_hw_number_t hw)

drivers/irqchip/irq-gic-v3.c line 537

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+{
+	if (hw < 32) {
+		irq_set_percpu_devid(irq);
+		irq_set_chip_and_handler(irq, &gic_chip,
+					 handle_percpu_devid_irq);
+		set_irq_flags(irq, IRQF_VALID | IRQF_NOAUTOEN);
+	} else {
+		irq_set_chip_and_handler(irq, &gic_chip,
+					 handle_fasteoi_irq);
+		set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);

will this ever be compiled for arch/arm? If not, does it make sense to keep the set_irq_flags calls?

Quoted 40 lines
drivers/irqchip/irq-gic-v3.c new file +40 -0
+	}
+	irq_set_chip_data(irq, d->host_data);
+	return 0;
+}
+
+static int gic_irq_domain_xlate(struct irq_domain *d,
+				struct device_node *controller,
+				const u32 *intspec, unsigned int intsize,
+				unsigned long *out_hwirq, unsigned int *out_type)
+{
+	if (d->of_node != controller)
+		return -EINVAL;
+	if (intsize < 3)
+		return -EINVAL;
+
+	switch(intspec[0]) {
+	case 0:			/* SPI */
+		*out_hwirq = intspec[1] + 32;
+		break;
+	case 1:			/* PPI */
+		*out_hwirq = intspec[1] + 16;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	*out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
+	return 0;
+}
+
+static const struct irq_domain_ops gic_irq_domain_ops = {
+	.map = gic_irq_domain_map,
+	.xlate = gic_irq_domain_xlate,
+};
+
+static int __init gic_of_init(struct device_node *node, struct device_node *parent)
+{
+	void __iomem *dist_base;
+	void __iomem **redist_base;
+	u64 redist_stride;

drivers/irqchip/irq-gic-v3.c line 587

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+	u32 redist_regions;
+	u32 reg;
+	int gic_irqs;
+	int err;
+	int i;
+
+	dist_base = of_iomap(node, 0);
+	if (!dist_base) {
+		pr_warn("%s: unable to map gic dist registers\n",
+			node->full_name);

shouldn't these be pr_err?

drivers/irqchip/irq-gic-v3.c line 592

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+		return -ENXIO;
+	}
+
+	reg = readl_relaxed(dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
+	if (reg != 0x30 && reg != 0x40) {

Is everything we do going to work with a GICv4? Anything else we need to initialize if it's a GICv4?

Quoted 17 lines
drivers/irqchip/irq-gic-v3.c new file +17 -0
+		pr_warn("%s: no distributor detected, giving up\n",
+			node->full_name);
+		err = -ENODEV;
+		goto out_unmap_dist;
+	}
+
+	if (of_property_read_u32(node, "#redistributor-regions", &redist_regions))
+		redist_regions = 1;
+
+	redist_base = kzalloc(sizeof(*redist_base) * redist_regions, GFP_KERNEL);
+	if (!redist_base) {
+		err = -ENOMEM;
+		goto out_unmap_dist;
+	}
+
+	for (i = 0; i < redist_regions; i++) {
+		redist_base[i] = of_iomap(node, 1 + i);

drivers/irqchip/irq-gic-v3.c line 619

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+		if (!redist_base[i]) {
+			pr_warn("%s: couldn't map region %d\n",
+				node->full_name, i);
+			err = -ENODEV;
+			goto out_unmap_rdist;
+		}
+	}
+
+	if (of_property_read_u64(node, "redistributor-stride", &redist_stride))
+		redist_stride = 0;

The binding doesn't specify this as a two-cell property, so I assume this won't work if you follow the example in the bindings doc.

Quoted 133 lines
drivers/irqchip/irq-gic-v3.c new file +46 -0
+
+	gic_data.dist_base = dist_base;
+	gic_data.redist_base = redist_base;
+	gic_data.redist_regions = redist_regions;
+	gic_data.redist_stride = redist_stride;
+
+	/*
+	 * Find out how many interrupts are supported.
+	 * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)
+	 */
+	gic_irqs = readl_relaxed(gic_data.dist_base + GICD_TYPER) & 0x1f;
+	gic_irqs = (gic_irqs + 1) * 32;
+	if (gic_irqs > 1020)
+		gic_irqs = 1020;
+	gic_data.irq_nr = gic_irqs;
+
+	gic_data.domain = irq_domain_add_linear(node, gic_irqs - 16,
+						&gic_irq_domain_ops, &gic_data);
+	gic_data.rdist = alloc_percpu(typeof(*gic_data.rdist));
+
+	if (WARN_ON(!gic_data.domain) || WARN_ON(!gic_data.rdist)) {
+		err = -ENOMEM;
+		goto out_free;
+	}
+
+	set_handle_irq(gic_handle_irq);
+
+	gic_smp_init();
+	gic_dist_init();
+	gic_cpu_init();
+
+	return 0;
+
+out_free:
+	free_percpu(gic_data.rdist);
+out_unmap_rdist:
+	for (i = 0; i < redist_regions; i++)
+		if (redist_base[i])
+			iounmap(redist_base[i]);
+	kfree(redist_base);
+out_unmap_dist:
+	iounmap(dist_base);
+	return err;
+}
+
+IRQCHIP_DECLARE(gic_v3, "arm,gic-v3", gic_of_init);
include/linux/irqchip/arm-gic-v3.h new file +81 -0
new file mode 100644
index 0000000..e4d060a
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __LINUX_IRQCHIP_ARM_GIC_V3_H
+#define __LINUX_IRQCHIP_ARM_GIC_V3_H
+
+/*
+ * Distributor registers. We assume we're running non-secure, with ARE
+ * being set. Secure-only and non-ARE registers are not described.
+ */
+#define GICD_CTLR			0x0000
+#define GICD_TYPER			0x0004
+#define GICD_IIDR			0x0008
+#define GICD_STATUSR			0x0010
+#define GICD_SETSPI_NSR			0x0040
+#define GICD_CLRSPI_NSR			0x0048
+#define GICD_SETSPI_SR			0x0050
+#define GICD_CLRSPI_SR			0x0058
+#define GICD_SEIR			0x0068
+#define GICD_ISENABLER			0x0100
+#define GICD_ICENABLER			0x0180
+#define GICD_ISPENDR			0x0200
+#define GICD_ICPENDR			0x0280
+#define GICD_ISACTIVER			0x0300
+#define GICD_ICACTIVER			0x0380
+#define GICD_IPRIORITYR			0x0400
+#define GICD_ICFGR			0x0C00
+#define GICD_IROUTER			0x6000
+#define GICD_PIDR2			0xFFE8
+
+#define GICD_CTLR_RWP			(1U << 31)
+#define GICD_CTLR_ARE_NS		(1U << 4)
+#define GICD_CTLR_ENABLE_G1A		(1U << 1)
+#define GICD_CTLR_ENABLE_G1		(1U << 0)
+
+#define GICD_IROUTER_SPI_MODE_ONE	(0U << 31)
+#define GICD_IROUTER_SPI_MODE_ANY	(1U << 31)
+
+#define GIC_PIDR2_ARCH_MASK		0xf0
+
+/*
+ * Re-Distributor registers, offsets from RD_base
+ */
+#define GICR_CTLR			GICD_CTLR
+#define GICR_IIDR			0x0004
+#define GICR_TYPER			0x0008
+#define GICR_STATUSR			GICD_STATUSR
+#define GICR_WAKER			0x0014
+#define GICR_SETLPIR			0x0040
+#define GICR_CLRLPIR			0x0048
+#define GICR_SEIR			GICD_SEIR
+#define GICR_PROPBASER			0x0070
+#define GICR_PENDBASER			0x0078
+#define GICR_INVLPIR			0x00A0
+#define GICR_INVALLR			0x00B0
+#define GICR_SYNCR			0x00C0
+#define GICR_MOVLPIR			0x0100
+#define GICR_MOVALLR			0x0110
+#define GICR_PIDR2			GICD_PIDR2
+
+#define GICR_WAKER_ProcessorSleep	(1U << 1)
+#define GICR_WAKER_ChildrenAsleep	(1U << 2)
+
+/*
+ * Re-Distributor registers, offsets from SGI_base
+ */
+#define GICR_ISENABLER0			GICD_ISENABLER

include/linux/irqchip/arm-gic-v3.h line 91

Quoted 10 lines
include/linux/irqchip/arm-gic-v3.h new file +10 -0
+#define GICR_ICENABLER0			GICD_ICENABLER
+#define GICR_ISPENDR0			GICD_ISPENDR
+#define GICR_ICPENDR0			GICD_ICPENDR
+#define GICR_ISACTIVER0			GICD_ISACTIVER
+#define GICR_ICACTIVER0			GICD_ICACTIVER
+#define GICR_IPRIORITYR0		GICD_IPRIORITYR
+#define GICR_ICFGR0			GICD_ICFGR
+
+#define GICR_TYPER_VLPIS		(1U << 1)
+#define GICR_TYPER_LAST			(1U << 4)

nit: why are these TYPER definitions down here when the TYPER register is defined above and so the WAKER bit definitions?

Quoted 73 lines
include/linux/irqchip/arm-gic-v3.h new file +73 -0
+
+/*
+ * CPU interface registers
+ */
+#define GICC_CTLR_EL1_EOImode_drop_dir	(0U << 1)
+#define GICC_CTLR_EL1_EOImode_drop	(1U << 1)
+#define GICC_SRE_EL1_SRE		(1U << 0)
+
+/*
+ * Hypervisor interface registers (SRE only)
+ */
+#define GICH_LR_VIRTUAL_ID_MASK		((1UL << 32) - 1)
+
+#define GICH_LR_EOI			(1UL << 41)
+#define GICH_LR_GROUP			(1UL << 60)
+#define GICH_LR_STATE			(3UL << 62)
+#define GICH_LR_PENDING_BIT		(1UL << 62)
+#define GICH_LR_ACTIVE_BIT		(1UL << 63)
+
+#define GICH_MISR_EOI			(1 << 0)
+#define GICH_MISR_U			(1 << 1)
+
+#define GICH_HCR_EN			(1 << 0)
+#define GICH_HCR_UIE			(1 << 1)
+
+#define GICH_VMCR_CTLR_SHIFT		0
+#define GICH_VMCR_CTLR_MASK		(0x21f << GICH_VMCR_CTLR_SHIFT)
+#define GICH_VMCR_BPR1_SHIFT		18
+#define GICH_VMCR_BPR1_MASK		(7 << GICH_VMCR_BPR1_SHIFT)
+#define GICH_VMCR_BPR0_SHIFT		21
+#define GICH_VMCR_BPR0_MASK		(7 << GICH_VMCR_BPR0_SHIFT)
+#define GICH_VMCR_PMR_SHIFT		24
+#define GICH_VMCR_PMR_MASK		(0xffUL << GICH_VMCR_PMR_SHIFT)
+
+#define ICC_EOIR1_EL1			S3_0_C12_C12_1
+#define ICC_IAR1_EL1			S3_0_C12_C12_0
+#define ICC_SGI1R_EL1			S3_0_C12_C11_5
+#define ICC_PMR_EL1			S3_0_C4_C6_0
+#define ICC_CTLR_EL1			S3_0_C12_C12_4
+#define ICC_SRE_EL1			S3_0_C12_C12_5
+#define ICC_GRPEN1_EL1			S3_0_C12_C12_7
+
+#define ICC_SRE_EL2			S3_4_C12_C9_5
+
+#define ICH_VSEIR_EL2			S3_4_C12_C9_4
+#define ICH_HCR_EL2			S3_4_C12_C11_0
+#define ICH_VTR_EL2			S3_4_C12_C11_1
+#define ICH_MISR_EL2			S3_4_C12_C11_2
+#define ICH_EISR_EL2			S3_4_C12_C11_3
+#define ICH_ELSR_EL2			S3_4_C12_C11_5
+#define ICH_VMCR_EL2			S3_4_C12_C11_7
+
+#define __LR0_EL2(x)			S3_4_C12_C12_ ## x
+#define __LR8_EL2(x)			S3_4_C12_C13_ ## x
+
+#define ICH_LR0_EL2			__LR0_EL2(0)
+#define ICH_LR1_EL2			__LR0_EL2(1)
+#define ICH_LR2_EL2			__LR0_EL2(2)
+#define ICH_LR3_EL2			__LR0_EL2(3)
+#define ICH_LR4_EL2			__LR0_EL2(4)
+#define ICH_LR5_EL2			__LR0_EL2(5)
+#define ICH_LR6_EL2			__LR0_EL2(6)
+#define ICH_LR7_EL2			__LR0_EL2(7)
+#define ICH_LR8_EL2			__LR8_EL2(0)
+#define ICH_LR9_EL2			__LR8_EL2(1)
+#define ICH_LR10_EL2			__LR8_EL2(2)
+#define ICH_LR11_EL2			__LR8_EL2(3)
+#define ICH_LR12_EL2			__LR8_EL2(4)
+#define ICH_LR13_EL2			__LR8_EL2(5)
+#define ICH_LR14_EL2			__LR8_EL2(6)
+#define ICH_LR15_EL2			__LR8_EL2(7)
+
+#define __AP0Rx_EL2(x)			S3_4_C12_C8_ ## x

include/linux/irqchip/arm-gic-v3.h line 174

Quoted 10 lines
include/linux/irqchip/arm-gic-v3.h new file +10 -0
+#define ICH_AP0R0_EL2			__AP0Rx_EL2(0)
+#define ICH_AP0R1_EL2			__AP0Rx_EL2(1)
+#define ICH_AP0R2_EL2			__AP0Rx_EL2(2)
+#define ICH_AP0R3_EL2			__AP0Rx_EL2(3)
+
+#define __AP1Rx_EL2(x)			S3_4_C12_C9_ ## x
+#define ICH_AP1R0_EL2			__AP1Rx_EL2(0)
+#define ICH_AP1R1_EL2			__AP1Rx_EL2(1)
+#define ICH_AP1R2_EL2			__AP1Rx_EL2(2)
+#define ICH_AP1R3_EL2			__AP1Rx_EL2(3)

That was fun to review.

Quoted 19 lines
include/linux/irqchip/arm-gic-v3.h new file +16 -0
+
+#ifndef __ASSEMBLY__
+
+#include <linux/stringify.h>
+
+#define reg(x)		__stringify(x)
+
+static inline void gic_write_eoir(u64 irq)
+{
+	asm volatile("msr " reg(ICC_EOIR1_EL1) ", %0" : : "r" (irq));
+	isb();
+}
+
+#endif
+
+#endif
Signature · 1 line
1.8.3.4
#6 lore raw

PATCH 1/18 arm64: initial support for GICv3

Hi Christoffer,

Quoted 57 lines · Christoffer Dall
Quoted 56 lines · Marc Zyngier

The Generic Interrupt Controller (version 3) offers services that are similar to GICv2, with a number of additional features:
- Affinity routing based on the CPU MPIDR (ARE)
- System register for the CPU interfaces (SRE)
- Support for more that 8 CPUs
- Locality-specific Peripheral Interrupts (LPIs)
- Interrupt Translation Services (ITS)

This patch adds preliminary support for GICv3 with ARE and SRE, non-secure mode only. It relies on higher exception levels to grant ARE and SRE access.

Support for LPI and ITS will be added at a later time.

---
 arch/arm64/Kconfig                 |   1 +
 arch/arm64/kernel/head.S           |  14 +
 arch/arm64/kernel/hyp-stub.S       |  41 ++-
 drivers/irqchip/Kconfig            |   5 +
 drivers/irqchip/Makefile           |   1 +
 drivers/irqchip/irq-gic-v3.c       | 665 +++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h | 190 +++++++++++
 7 files changed, 915 insertions(+), 2 deletions(-)
 create mode 100644 drivers/irqchip/irq-gic-v3.c
 create mode 100644 include/linux/irqchip/arm-gic-v3.h
arch/arm64/Kconfig +1 -0
index dd4327f..3a7e26d 100644
@@ -9,6 +9,7 @@ config ARM64
      select ARM_AMBA
      select ARM_ARCH_TIMER
      select ARM_GIC
+     select ARM_GIC_V3
      select BUILDTIME_EXTABLE_SORT
      select CLONE_BACKWARDS
      select COMMON_CLK
arch/arm64/kernel/head.S +1 -0
index 0b281ff..522649b 100644
@@ -22,6 +22,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -183,6 +184,19 @@ CPU_LE(  bic     x0, x0, #(3 << 24)      )       // Clear the EE and E0E bits for EL1
      msr     cnthctl_el2, x0
      msr     cntvoff_el2, xzr                // Clear virtual offset
 
Quoted 14 lines

arch/arm64/kernel/head.S CPU_LE( bic x0, x0, #(3 << 24) ) // Clear the EE and E0E bits for EL1 · line 197

Quoted 11 lines
arch/arm64/kernel/head.S +11 -0
+     /* GICv3 system register access */
+     mrs     x0, id_aa64pfr0_el1
+     ubfx    x0, x0, #24, #4
+     cmp     x0, #1
+     b.ne    3f
+
+     mrs     x0, ICC_SRE_EL2                 // Set ICC_SRE_EL2.SRE==1
+     orr     x0, x0, #1
+     msr     ICC_SRE_EL2, x0
+     isb                                     // Make sure SRE is now 1
+     msr     ICH_HCR_EL2, xzr                // Reset ICC_HCR_EL2 to defaults

Why do we not just set ICC_SRE_EL2.Enable here, but instead do the whole exception dance below?

That's a very good question. The more I think of it, the more I believe it to be completely unnecessary.

I'm going to remove the trapping and see what breaks. Or who screams.

Quoted 21 lines
Quoted 21 lines
arch/arm64/kernel/head.S +2 -0
+
+3:
      /* Populate ID registers. */
      mrs     x0, midr_el1
      mrs     x1, mpidr_el1
arch/arm64/kernel/hyp-stub.S +1 -0
index 0959611..c850d1e 100644
@@ -19,6 +19,7 @@
 
 #include <linux/init.h>
 #include <linux/linkage.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -51,16 +52,52 @@ ENDPROC(__hyp_stub_vectors)
 
      .align 11
 
Quoted 16 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 65

Quoted 11 lines
arch/arm64/kernel/hyp-stub.S +10 -0
+#define sysreg_to_iss(op0, op1, crn, crm, op2)       \
+     (((op0 & 3) << 20) | ((op2 & 7 ) << 17) | \
+      ((op1 & 7 ) << 14) | ((crn & 15) << 10) | \
+      ((crm & 15) << 1))
+
 el1_sync:
+     // Hack alert: we don't have a stack, and we can't use the one
+     // from EL1 because our MMU is off. Thankfully, we have
+     // tpidr_el2, which is under our complete control, and
+     // tpidr_el0, which can be used as long as we haven't reached
+     // userspace. Oh hum...

Hmmm. I feel like either we should tone down the hack alert claim, because this is after all safe and we just explain what's going on, or we should just allocate a small stack space following el1_sync and set the SP upon entry to this handler?

Allocating a stack here is something I'd rather avoid (how big?). But yeah, I can probably tone the horror down a notch - I got used to see that code... ;-)

Quoted 2 lines

Hopefully none of the code that will ever call into the sysregs trap ever makes the same assumption that user space is not called yet ;)

Well, I'm going to nuke that code anyway. I'm almost convinced that we don't need it.

Quoted 23 lines
Quoted 23 lines
arch/arm64/kernel/hyp-stub.S +14 -2
+     msr     tpidr_el2, x1                   // Preserve x1
+
      mrs     x1, esr_el2
      lsr     x1, x1, #26
      cmp     x1, #0x16
-     b.ne    2f                              // Not an HVC trap
+     b.ne    3f                              // Not an HVC trap
+
+     // Handle getting/setting vbar_el2
      cbz     x0, 1f
      msr     vbar_el2, x0                    // Set vbar_el2
      b       2f
 1:   mrs     x0, vbar_el2                    // Return vbar_el2
-2:   eret
+2:   mrs     x1, tpidr_el2                   // Restore x1
+     eret
+
+3:   cmp     x1, #0x18                       // MSR/MRS?
+     b.ne    2b
+
+     // Handle trapped sys reg access
+     msr     tpidr_el0, x0                   // Preserve x0
+     mrs     x1, esr_el2
Quoted 8 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 91

Quoted 5 lines
arch/arm64/kernel/hyp-stub.S +5 -0
+     ldr     x0, =sysreg_to_iss(3, 7, 15, 15, 7)
+     and     x1, x1, x0
+     ldr     x0, =sysreg_to_iss(3, 0, 12, 12, 5)
+     eor     x0, x0, x1                      // Check for ICC_SRE_EL1
+     cbnz    x0, 4f                          // Something bad happened

Shouldn't we try to do something here, like go to panic on the exception return here, or not worth it?

Code is now gone... ;-)

Quoted 339 lines
Quoted 339 lines
arch/arm64/kernel/hyp-stub.S +9 -0
+
+     // Set ICC_SRE_EL1 access enable
+     mrs     x1, ICC_SRE_EL2
+     orr     x1, x1, #(1 << 3)
+     msr     ICC_SRE_EL2, x1
+
+4:   mrs     x0, tpidr_el0                   // Restore x0
+     mrs     x1, tpidr_el2                   // Restore x1
+     eret
 ENDPROC(el1_sync)
 
 .macro invalid_vector        label
drivers/irqchip/Kconfig +5 -0
index 61ffdca..5b55c46 100644
@@ -10,6 +10,11 @@ config ARM_GIC
 config GIC_NON_BANKED
      bool
 
+config ARM_GIC_V3
+     bool
+     select IRQ_DOMAIN
+     select MULTI_IRQ_HANDLER
+
 config ARM_NVIC
      bool
      select IRQ_DOMAIN
drivers/irqchip/Makefile +1 -0
index 86b484c..e6ac9fd 100644
@@ -14,6 +14,7 @@ obj-$(CONFIG_ORION_IRQCHIP)         += irq-orion.o
 obj-$(CONFIG_ARCH_SUNXI)             += irq-sun4i.o
 obj-$(CONFIG_ARCH_SPEAR3XX)          += spear-shirq.o
 obj-$(CONFIG_ARM_GIC)                        += irq-gic.o
+obj-$(CONFIG_ARM_GIC_V3)             += irq-gic-v3.o
 obj-$(CONFIG_ARM_NVIC)                       += irq-nvic.o
 obj-$(CONFIG_ARM_VIC)                        += irq-vic.o
 obj-$(CONFIG_IMGPDC_IRQ)             += irq-imgpdc.o
drivers/irqchip/irq-gic-v3.c new file +293 -0
new file mode 100644
index 0000000..487e061
@@ -0,0 +1,665 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/cputype.h>
+#include <asm/exception.h>
+#include <asm/smp_plat.h>
+
+#include "irqchip.h"
+
+struct gic_chip_data {
+     void __iomem            *dist_base;
+     void __iomem            **redist_base;
+     void __percpu __iomem   **rdist;
+     struct irq_domain       *domain;
+     u64                     redist_stride;
+     u32                     redist_regions;
+     unsigned int            irq_nr;
+};
+
+static struct gic_chip_data gic_data __read_mostly;
+
+#define gic_data_rdist_rd_base()     (*__this_cpu_ptr(gic_data.rdist))
+#define gic_data_rdist_sgi_base()    (gic_data_rdist_rd_base() + SZ_64K)
+
+static DEFINE_RAW_SPINLOCK(dist_lock);
+
+#define DEFAULT_PMR_VALUE    0xf0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)
+{
+     if (d->hwirq < 32)      /* SGI+PPI -> SGI_base for this CPU */
+             return gic_data_rdist_sgi_base();
+
+     if (d->hwirq <= 1023)   /* SPI -> dist_base */
+             return gic_data.dist_base;
+
+     if (d->hwirq >= 8192)
+             BUG();          /* LPI Detected!!! */
+
+     return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+     return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
+{
+     u32 val;
+
+     do {
+             val = readl_relaxed(base + GICD_CTLR);
+             cpu_relax();
+     } while (val & GICD_CTLR_RWP);
+}
+
+/* Wait for completion of a distributor change */
+static void gic_dist_wait_for_rwp(void)
+{
+     gic_do_wait_for_rwp(gic_data.dist_base);
+}
+
+/* Wait for completion of a redistributor change */
+static void gic_redist_wait_for_rwp(void)
+{
+     gic_do_wait_for_rwp(gic_data_rdist_rd_base());
+}
+
+static void gic_wait_for_rwp(int irq)
+{
+     if (irq < 32)
+             gic_redist_wait_for_rwp();
+     else
+             gic_dist_wait_for_rwp();
+}
+
+/* Low level accessors */
+static u64 gic_read_iar(void)
+{
+     u64 irqstat;
+
+     asm volatile("mrs %0, " reg(ICC_IAR1_EL1) : "=r" (irqstat));
+     return irqstat;
+}
+
+static void gic_write_pmr(u64 val)
+{
+     asm volatile("msr " reg(ICC_PMR_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_write_ctlr(u64 val)
+{
+     asm volatile("msr " reg(ICC_CTLR_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_write_grpen1(u64 val)
+{
+     asm volatile("msr " reg(ICC_GRPEN1_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_write_sgi1r(u64 val)
+{
+     asm volatile("msr " reg(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_enable_sre(void)
+{
+     u64 val;
+
+     /* This access is likely to trap into EL2 */
+     asm volatile("mrs %0, " reg(ICC_SRE_EL1) : "=r" (val));
+     val |= GICC_SRE_EL1_SRE;
+     asm volatile("msr " reg(ICC_SRE_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_enable_redist(void)
+{
+     void __iomem *rbase;
+     u32 val;
+
+     rbase = gic_data_rdist_rd_base();
+
+     /* Wake up this CPU redistributor */
+     val = readl_relaxed(rbase + GICR_WAKER);
+     val &= ~GICR_WAKER_ProcessorSleep;
+     writel_relaxed(val, rbase + GICR_WAKER);
+
+     do {
+             val = readl_relaxed(rbase + GICR_WAKER);
+             cpu_relax();
+     } while (val & GICR_WAKER_ChildrenAsleep);
+}
+
+/*
+ * Routines to acknowledge, disable and enable interrupts
+ */
+static void gic_mask_irq(struct irq_data *d)
+{
+     u32 mask = 1 << (gic_irq(d) % 32);
+
+     raw_spin_lock(&dist_lock);
+     writel_relaxed(mask, gic_dist_base(d) + GICD_ICENABLER + (gic_irq(d) / 32) * 4);
+     gic_wait_for_rwp(gic_irq(d));
+     raw_spin_unlock(&dist_lock);
+}
+
+static void gic_unmask_irq(struct irq_data *d)
+{
+     u32 mask = 1 << (gic_irq(d) % 32);
+
+     raw_spin_lock(&dist_lock);
+     writel_relaxed(mask, gic_dist_base(d) + GICD_ISENABLER + (gic_irq(d) / 32) * 4);
+     gic_wait_for_rwp(gic_irq(d));
+     raw_spin_unlock(&dist_lock);
+}
+
+static void gic_eoi_irq(struct irq_data *d)
+{
+     gic_write_eoir(gic_irq(d));
+}
+
+static int gic_set_type(struct irq_data *d, unsigned int type)
+{
+     void __iomem *base = gic_dist_base(d);
+     unsigned int irq = gic_irq(d);
+     u32 enablemask = 1 << (irq % 32);
+     u32 enableoff = (irq / 32) * 4;
+     u32 confmask = 0x2 << ((irq % 16) * 2);
+     u32 confoff = (irq / 16) * 4;
+     bool enabled = false;
+     u32 val;
+
+     /* Interrupt configuration for SGIs can't be changed */
+     if (irq < 16)
+             return -EINVAL;
+
+     if (type != IRQ_TYPE_LEVEL_HIGH && type != IRQ_TYPE_EDGE_RISING)
+             return -EINVAL;
+
+     raw_spin_lock(&dist_lock);
+
+     val = readl_relaxed(base + GICD_ICFGR + confoff);
+     if (type == IRQ_TYPE_LEVEL_HIGH)
+             val &= ~confmask;
+     else if (type == IRQ_TYPE_EDGE_RISING)
+             val |= confmask;
+
+     /*
+      * As recommended by the spec, disable the interrupt before changing
+      * the configuration
+      */
+     if (readl_relaxed(base + GICD_ISENABLER + enableoff) & enablemask) {
+             writel_relaxed(enablemask, base + GICD_ICENABLER + enableoff);
+             gic_wait_for_rwp(irq);
+             enabled = true;
+     }
+
+     writel_relaxed(val, base + GICD_ICFGR + confoff);
+
+     if (enabled) {
+             writel_relaxed(enablemask, base + GICD_ISENABLER + enableoff);
+             gic_wait_for_rwp(irq);
+     }
+
+     raw_spin_unlock(&dist_lock);
+
+     return 0;
+}
+
+static int gic_retrigger(struct irq_data *d)
+{
+     return -ENXIO;
+}
+
+static u64 gic_mpidr_to_affinity(u64 mpidr)
+{
+     /* Make sure we don't broadcast the interrupt */
+     return mpidr & ~GICD_IROUTER_SPI_MODE_ANY;
+}
+
+static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
+{
+     u64 irqnr;
+
+     do {
+             irqnr = gic_read_iar();
+
+             if (likely(irqnr > 15 && irqnr < 1021)) {
+                     irqnr = irq_find_mapping(gic_data.domain, irqnr);
+                     handle_IRQ(irqnr, regs);
+                     continue;
+             }
+             if (irqnr < 16) {
+                     gic_write_eoir(irqnr);
+#ifdef CONFIG_SMP
+                     handle_IPI(irqnr, regs);
+#else
+                     WARN_ONCE(true, "Unexpected SGI received!\n");
+#endif
+                     continue;
+             }
+     } while (irqnr != 0x3ff);
+}
+
+static void __init gic_dist_init(void)
+{
+     unsigned int i;
+     u64 affinity;
+     int gic_irqs = gic_data.irq_nr;
+     void __iomem *base = gic_data.dist_base;
+
+     /* Disable the distributor */
+     writel_relaxed(0, base + GICD_CTLR);
+
+     /*
+      * Set all global interrupts to be level triggered, active low.
+      */
+     for (i = 32; i < gic_data.irq_nr; i += 16)
+             writel_relaxed(0, base + GICD_ICFGR + i / 4);
+
+     /*
+      * Set priority on all global interrupts.
+      */
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 299

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+     for (i = 32; i < gic_irqs; i += 4)
+             writel_relaxed(0xa0a0a0a0, base + GICD_IPRIORITYR + i);
+
+     /*
+      * Disable all interrupts.  Leave the PPI and SGIs alone
+      * as these enables are banked registers.

as these are enabled through banked registers?

"The enable bits are banked registers", but I can see where the confision comes from. I'll rewrite this comment.

Quoted 1 line

also, should banked registers be redistributor registers?

Yup.

Quoted 10 lines

drivers/irqchip/irq-gic-v3.c line 307

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+      */
+     for (i = 32; i < gic_irqs; i += 32)
+             writel_relaxed(0xffffffff, base + GICD_ICENABLER + i / 8);
+
+     gic_dist_wait_for_rwp();
+
+     writel_relaxed(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+                    base + GICD_CTLR);

why set GICD_CTLR_ENABLE_G1 ?

Believe it or not, some people insist on running in secure mode.

Quoted 13 lines
Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+
+     /*
+      * Set all global interrupts to the boot CPU only. ARE must be
+      * enabled.
+      */
+     affinity = gic_mpidr_to_affinity(read_cpuid_mpidr());
+     for (i = 32; i < gic_data.irq_nr; i++)
+             writeq_relaxed(affinity, base + GICD_IROUTER + i * 8);
+}
+
+static int __init gic_populate_rdist(void)
+{
+     u64 mpidr = cpu_logical_map(smp_processor_id());
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 326

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+     u64 typer;
+     u64 aff;
+     int i;
+
+     aff  = mpidr & ((1 << 24) - 1);
+     aff |= (mpidr >> 8) & (0xffUL << 24);

Some defines would be nice for this if it makes sense in any way.

There is at least 3 methods GICv3 uses to express affinity. At some point, I gave up and decided that I needed mental support.

I'll update it with the appropriate macros (MPIDR_AFFINITY_LEVEL and co).

Quoted 49 lines
Quoted 49 lines
drivers/irqchip/irq-gic-v3.c new file +49 -0
+
+     for (i = 0; i < gic_data.redist_regions; i++) {
+             void __iomem *ptr = gic_data.redist_base[i];
+             u32 reg;
+
+             reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
+             if (reg != 0x30 && reg != 0x40) { /* We're in trouble... */
+                     pr_warn("No redistributor present @%p\n", ptr);
+                     break;
+             }
+
+             do {
+                     typer = readq_relaxed(ptr + GICR_TYPER);
+                     if ((typer >> 32) == aff) {
+                             gic_data_rdist_rd_base() = ptr;
+                             pr_info("CPU%d: found redistributor %llx @%p\n",
+                                     smp_processor_id(),
+                                     (unsigned long long) mpidr, ptr);
+                             return 0;
+                     }
+
+                     if (gic_data.redist_stride) {
+                             ptr += gic_data.redist_stride;
+                     } else {
+                             ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+                             if (typer & GICR_TYPER_VLPIS)
+                                     ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+                     }
+             } while (!(typer & GICR_TYPER_LAST));
+     }
+
+     /* We couldn't even deal with ourselves... */
+     WARN(true, "CPU%d: mpidr %lx has no re-distributor!\n",
+          smp_processor_id(), (unsigned long)mpidr);
+     return -ENODEV;
+}
+
+static void __init gic_cpu_init(void)
+{
+     void __iomem *rbase;
+     int i;
+
+     /* Register ourselves with the rest of the world */
+     if (gic_populate_rdist())
+             return;
+
+     gic_enable_redist();
+
+     rbase = gic_data_rdist_sgi_base();
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 381

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+
+     /*
+      * Set priority on PPI and SGI interrupts
+      */
+     for (i = 0; i < 32; i += 4)
+             writel_relaxed(0xa0a0a0a0, rbase + GICR_IPRIORITYR0 + i * 4 / 4);

(i * 4) / 4 == i, last I checked.

Recurring comment. I decided to keep it inline with the GICv2 practice. Read it as i * increment / interrupt-per-word. Maybe not worth it if it causes more problem than anything else...

Quoted 6 lines

drivers/irqchip/irq-gic-v3.c line 385

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+
+     /*
+      * Disable all PPI interrupts, ensure all SGI interrupts are
+      * enabled.

Why do we disable all PPIs?

Because interrupts should default to being disabled? They will get enabled when you do a request_irq.

Quoted 8 lines
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+      */
+     writel_relaxed(0xffff0000, rbase + GICR_ICENABLER0);
+     writel_relaxed(0x0000ffff, rbase + GICR_ISENABLER0);
+
+     gic_redist_wait_for_rwp();
+
+     /* Enable system registers */
+     gic_enable_sre();
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 398

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+
+     /* Set priority mask register */
+     gic_write_pmr(DEFAULT_PMR_VALUE);
+
+     /* EOI drops priority too (mode 0) */

you mean deactives the interrupt too, right? It always performs a priority drop doesn't it?

Yup. Just my usual brainfart.

Quoted 13 lines
Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+     gic_write_ctlr(GICC_CTLR_EL1_EOImode_drop_dir);
+
+     /* ... and let's hit the road... */
+     gic_write_grpen1(1);
+}
+
+#ifdef CONFIG_SMP
+static int __init gic_secondary_init(struct notifier_block *nfb,
+                                  unsigned long action, void *hcpu)
+{
+     if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+             gic_cpu_init();
+     return NOTIFY_OK;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 416

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+}
+
+/*
+ * Notifier for enabling the GIC CPU interface. Set an arbitrarily high
+ * priority because the GIC needs to be up before the ARM generic timers.

I actually don't understand this bit, but it also sits in the original code, so I assume it's correct.

GIC and timer on the secondary CPUs are intialized using the CPU notifiers. Since the need the GIC to be up and running before the timers can enable their interrupts, we use the relative priorities of the notifiers to order them.

Quoted 20 lines
Quoted 20 lines
drivers/irqchip/irq-gic-v3.c new file +20 -0
+ */
+static struct notifier_block __initdata gic_cpu_notifier = {
+     .notifier_call = gic_secondary_init,
+     .priority = 100,
+};
+
+static u16 gic_compute_target_list(int *base_cpu, const struct cpumask *mask,
+                                u64 cluster_id)
+{
+     int cpu = *base_cpu;
+     u64 mpidr = cpu_logical_map(cpu);
+     u16 tlist = 0;
+
+     while (cpu < nr_cpu_ids) {
+             /*
+              * If we ever get a cluster of more than 16 CPUs, just
+              * scream and skip that CPU.
+              */
+             if (WARN_ON((mpidr & 0xff) >= 16))
+                     goto out;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 441

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+
+             tlist |= 1 << (mpidr & 0xf);
+
+             cpu = cpumask_next(cpu, mask);
+             mpidr = cpu_logical_map(cpu);

Are you not accessing memory beyond the __cpu_logical_map array here when the cpu is the last one at the beginning of the loop?

Nice catch! I need to test cpu here before carrying on...

Quoted 39 lines
Quoted 39 lines
drivers/irqchip/irq-gic-v3.c new file +39 -0
+
+             if (cluster_id != (mpidr & ~0xffUL)) {
+                     cpu--;
+                     goto out;
+             }
+     }
+out:
+     *base_cpu = cpu;
+     return tlist;
+}
+
+static void gic_send_sgi(u64 cluster_id, u16 tlist, unsigned int irq)
+{
+     u64 val;
+
+     val  = (cluster_id & 0xff00ff0000UL) << 16; /* Aff3 + Aff2 */
+     val |= (cluster_id & 0xff00) << 8;          /* Aff1 */
+     val |= irq << 24;
+     val |= tlist;
+
+     pr_debug("CPU%d: ICC_SGI1R_EL1 %llx\n", smp_processor_id(), val);
+     gic_write_sgi1r(val);
+}
+
+static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
+{
+     int cpu;
+
+     if (WARN_ON(irq >= 16))
+             return;
+
+     /*
+      * Ensure that stores to Normal memory are visible to the
+      * other CPUs before issuing the IPI.
+      */
+     dsb();
+
+     for_each_cpu_mask(cpu, *mask) {
+             u64 cluster_id = cpu_logical_map(cpu) & ~0xffUL;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 483

Quoted 3 lines
drivers/irqchip/irq-gic-v3.c new file +3 -0
+             u16 tlist;
+
+             tlist = gic_compute_target_list(&cpu, mask, cluster_id);

I think it would be potentially slightly prettier if this function returned the last cpu in the cluster so the loop iterator mangling was kept local to this function and took a pointer to the tlist it was to fill instead, but, this is a minor detail.

Yeah, maybe. I'll try that and see how it looks. it's quite messy anyway...

Quoted 49 lines
Quoted 49 lines
drivers/irqchip/irq-gic-v3.c new file +49 -0
+             gic_send_sgi(cluster_id, tlist, irq);
+     }
+}
+
+static void gic_smp_init(void)
+{
+     set_smp_cross_call(gic_raise_softirq);
+     register_cpu_notifier(&gic_cpu_notifier);
+}
+
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+                         bool force)
+{
+     unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
+     void __iomem *reg;
+     u64 val;
+
+     if (gic_irq(d) < 32)
+             return -EINVAL;
+
+     reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+     val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+     writeq_relaxed(val, reg);
+
+     return IRQ_SET_MASK_OK;
+}
+#else
+#define gic_set_affinity     NULL
+#define gic_smp_init()               do { } while(0)
+#endif
+
+static struct irq_chip gic_chip = {
+     .name                   = "GICv3",
+     .irq_mask               = gic_mask_irq,
+     .irq_unmask             = gic_unmask_irq,
+     .irq_eoi                = gic_eoi_irq,
+     .irq_set_type           = gic_set_type,
+     .irq_retrigger          = gic_retrigger,
+     .irq_set_affinity       = gic_set_affinity,
+};
+
+static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
+                             irq_hw_number_t hw)
+{
+     if (hw < 32) {
+             irq_set_percpu_devid(irq);
+             irq_set_chip_and_handler(irq, &gic_chip,
+                                      handle_percpu_devid_irq);
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 537

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+             set_irq_flags(irq, IRQF_VALID | IRQF_NOAUTOEN);
+     } else {
+             irq_set_chip_and_handler(irq, &gic_chip,
+                                      handle_fasteoi_irq);
+             set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);

will this ever be compiled for arch/arm? If not, does it make sense to keep the set_irq_flags calls?

Not impossible at all, by the look of it. That will require some effort, but if someone feels like it, I don't see the point in actuvely removing that code.

Quoted 44 lines
Quoted 44 lines
drivers/irqchip/irq-gic-v3.c new file +44 -0
+     }
+     irq_set_chip_data(irq, d->host_data);
+     return 0;
+}
+
+static int gic_irq_domain_xlate(struct irq_domain *d,
+                             struct device_node *controller,
+                             const u32 *intspec, unsigned int intsize,
+                             unsigned long *out_hwirq, unsigned int *out_type)
+{
+     if (d->of_node != controller)
+             return -EINVAL;
+     if (intsize < 3)
+             return -EINVAL;
+
+     switch(intspec[0]) {
+     case 0:                 /* SPI */
+             *out_hwirq = intspec[1] + 32;
+             break;
+     case 1:                 /* PPI */
+             *out_hwirq = intspec[1] + 16;
+             break;
+     default:
+             return -EINVAL;
+     }
+
+     *out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
+     return 0;
+}
+
+static const struct irq_domain_ops gic_irq_domain_ops = {
+     .map = gic_irq_domain_map,
+     .xlate = gic_irq_domain_xlate,
+};
+
+static int __init gic_of_init(struct device_node *node, struct device_node *parent)
+{
+     void __iomem *dist_base;
+     void __iomem **redist_base;
+     u64 redist_stride;
+     u32 redist_regions;
+     u32 reg;
+     int gic_irqs;
+     int err;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 587

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+     int i;
+
+     dist_base = of_iomap(node, 0);
+     if (!dist_base) {
+             pr_warn("%s: unable to map gic dist registers\n",
+                     node->full_name);

shouldn't these be pr_err?

Indeed.

Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 592

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+             return -ENXIO;
+     }
+
+     reg = readl_relaxed(dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
+     if (reg != 0x30 && reg != 0x40) {

Is everything we do going to work with a GICv4? Anything else we need to initialize if it's a GICv4?

GICv4 is a strict superset of GICv3, and it should all just work as long as you don't try to use GICv4 specific features. *Should*.

Quoted 22 lines
Quoted 22 lines
drivers/irqchip/irq-gic-v3.c new file +22 -0
+             pr_warn("%s: no distributor detected, giving up\n",
+                     node->full_name);
+             err = -ENODEV;
+             goto out_unmap_dist;
+     }
+
+     if (of_property_read_u32(node, "#redistributor-regions", &redist_regions))
+             redist_regions = 1;
+
+     redist_base = kzalloc(sizeof(*redist_base) * redist_regions, GFP_KERNEL);
+     if (!redist_base) {
+             err = -ENOMEM;
+             goto out_unmap_dist;
+     }
+
+     for (i = 0; i < redist_regions; i++) {
+             redist_base[i] = of_iomap(node, 1 + i);
+             if (!redist_base[i]) {
+                     pr_warn("%s: couldn't map region %d\n",
+                             node->full_name, i);
+                     err = -ENODEV;
+                     goto out_unmap_rdist;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 619

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+             }
+     }
+
+     if (of_property_read_u64(node, "redistributor-stride", &redist_stride))
+             redist_stride = 0;

The binding doesn't specify this as a two-cell property, so I assume this won't work if you follow the example in the bindings doc.

Yup. I'll update the DT. Nice catch.

Quoted 138 lines
Quoted 138 lines
drivers/irqchip/irq-gic-v3.c new file +46 -0
+
+     gic_data.dist_base = dist_base;
+     gic_data.redist_base = redist_base;
+     gic_data.redist_regions = redist_regions;
+     gic_data.redist_stride = redist_stride;
+
+     /*
+      * Find out how many interrupts are supported.
+      * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)
+      */
+     gic_irqs = readl_relaxed(gic_data.dist_base + GICD_TYPER) & 0x1f;
+     gic_irqs = (gic_irqs + 1) * 32;
+     if (gic_irqs > 1020)
+             gic_irqs = 1020;
+     gic_data.irq_nr = gic_irqs;
+
+     gic_data.domain = irq_domain_add_linear(node, gic_irqs - 16,
+                                             &gic_irq_domain_ops, &gic_data);
+     gic_data.rdist = alloc_percpu(typeof(*gic_data.rdist));
+
+     if (WARN_ON(!gic_data.domain) || WARN_ON(!gic_data.rdist)) {
+             err = -ENOMEM;
+             goto out_free;
+     }
+
+     set_handle_irq(gic_handle_irq);
+
+     gic_smp_init();
+     gic_dist_init();
+     gic_cpu_init();
+
+     return 0;
+
+out_free:
+     free_percpu(gic_data.rdist);
+out_unmap_rdist:
+     for (i = 0; i < redist_regions; i++)
+             if (redist_base[i])
+                     iounmap(redist_base[i]);
+     kfree(redist_base);
+out_unmap_dist:
+     iounmap(dist_base);
+     return err;
+}
+
+IRQCHIP_DECLARE(gic_v3, "arm,gic-v3", gic_of_init);
include/linux/irqchip/arm-gic-v3.h new file +86 -0
new file mode 100644
index 0000000..e4d060a
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __LINUX_IRQCHIP_ARM_GIC_V3_H
+#define __LINUX_IRQCHIP_ARM_GIC_V3_H
+
+/*
+ * Distributor registers. We assume we're running non-secure, with ARE
+ * being set. Secure-only and non-ARE registers are not described.
+ */
+#define GICD_CTLR                    0x0000
+#define GICD_TYPER                   0x0004
+#define GICD_IIDR                    0x0008
+#define GICD_STATUSR                 0x0010
+#define GICD_SETSPI_NSR                      0x0040
+#define GICD_CLRSPI_NSR                      0x0048
+#define GICD_SETSPI_SR                       0x0050
+#define GICD_CLRSPI_SR                       0x0058
+#define GICD_SEIR                    0x0068
+#define GICD_ISENABLER                       0x0100
+#define GICD_ICENABLER                       0x0180
+#define GICD_ISPENDR                 0x0200
+#define GICD_ICPENDR                 0x0280
+#define GICD_ISACTIVER                       0x0300
+#define GICD_ICACTIVER                       0x0380
+#define GICD_IPRIORITYR                      0x0400
+#define GICD_ICFGR                   0x0C00
+#define GICD_IROUTER                 0x6000
+#define GICD_PIDR2                   0xFFE8
+
+#define GICD_CTLR_RWP                        (1U << 31)
+#define GICD_CTLR_ARE_NS             (1U << 4)
+#define GICD_CTLR_ENABLE_G1A         (1U << 1)
+#define GICD_CTLR_ENABLE_G1          (1U << 0)
+
+#define GICD_IROUTER_SPI_MODE_ONE    (0U << 31)
+#define GICD_IROUTER_SPI_MODE_ANY    (1U << 31)
+
+#define GIC_PIDR2_ARCH_MASK          0xf0
+
+/*
+ * Re-Distributor registers, offsets from RD_base
+ */
+#define GICR_CTLR                    GICD_CTLR
+#define GICR_IIDR                    0x0004
+#define GICR_TYPER                   0x0008
+#define GICR_STATUSR                 GICD_STATUSR
+#define GICR_WAKER                   0x0014
+#define GICR_SETLPIR                 0x0040
+#define GICR_CLRLPIR                 0x0048
+#define GICR_SEIR                    GICD_SEIR
+#define GICR_PROPBASER                       0x0070
+#define GICR_PENDBASER                       0x0078
+#define GICR_INVLPIR                 0x00A0
+#define GICR_INVALLR                 0x00B0
+#define GICR_SYNCR                   0x00C0
+#define GICR_MOVLPIR                 0x0100
+#define GICR_MOVALLR                 0x0110
+#define GICR_PIDR2                   GICD_PIDR2
+
+#define GICR_WAKER_ProcessorSleep    (1U << 1)
+#define GICR_WAKER_ChildrenAsleep    (1U << 2)
+
+/*
+ * Re-Distributor registers, offsets from SGI_base
+ */
+#define GICR_ISENABLER0                      GICD_ISENABLER
+#define GICR_ICENABLER0                      GICD_ICENABLER
+#define GICR_ISPENDR0                        GICD_ISPENDR
+#define GICR_ICPENDR0                        GICD_ICPENDR
+#define GICR_ISACTIVER0                      GICD_ISACTIVER
+#define GICR_ICACTIVER0                      GICD_ICACTIVER
Quoted 8 lines

include/linux/irqchip/arm-gic-v3.h line 91

Quoted 5 lines
include/linux/irqchip/arm-gic-v3.h new file +5 -0
+#define GICR_IPRIORITYR0             GICD_IPRIORITYR
+#define GICR_ICFGR0                  GICD_ICFGR
+
+#define GICR_TYPER_VLPIS             (1U << 1)
+#define GICR_TYPER_LAST                      (1U << 4)

nit: why are these TYPER definitions down here when the TYPER register is defined above and so the WAKER bit definitions?

I'll group them in the next version.

Quoted 77 lines
Quoted 77 lines
include/linux/irqchip/arm-gic-v3.h new file +77 -0
+
+/*
+ * CPU interface registers
+ */
+#define GICC_CTLR_EL1_EOImode_drop_dir       (0U << 1)
+#define GICC_CTLR_EL1_EOImode_drop   (1U << 1)
+#define GICC_SRE_EL1_SRE             (1U << 0)
+
+/*
+ * Hypervisor interface registers (SRE only)
+ */
+#define GICH_LR_VIRTUAL_ID_MASK              ((1UL << 32) - 1)
+
+#define GICH_LR_EOI                  (1UL << 41)
+#define GICH_LR_GROUP                        (1UL << 60)
+#define GICH_LR_STATE                        (3UL << 62)
+#define GICH_LR_PENDING_BIT          (1UL << 62)
+#define GICH_LR_ACTIVE_BIT           (1UL << 63)
+
+#define GICH_MISR_EOI                        (1 << 0)
+#define GICH_MISR_U                  (1 << 1)
+
+#define GICH_HCR_EN                  (1 << 0)
+#define GICH_HCR_UIE                 (1 << 1)
+
+#define GICH_VMCR_CTLR_SHIFT         0
+#define GICH_VMCR_CTLR_MASK          (0x21f << GICH_VMCR_CTLR_SHIFT)
+#define GICH_VMCR_BPR1_SHIFT         18
+#define GICH_VMCR_BPR1_MASK          (7 << GICH_VMCR_BPR1_SHIFT)
+#define GICH_VMCR_BPR0_SHIFT         21
+#define GICH_VMCR_BPR0_MASK          (7 << GICH_VMCR_BPR0_SHIFT)
+#define GICH_VMCR_PMR_SHIFT          24
+#define GICH_VMCR_PMR_MASK           (0xffUL << GICH_VMCR_PMR_SHIFT)
+
+#define ICC_EOIR1_EL1                        S3_0_C12_C12_1
+#define ICC_IAR1_EL1                 S3_0_C12_C12_0
+#define ICC_SGI1R_EL1                        S3_0_C12_C11_5
+#define ICC_PMR_EL1                  S3_0_C4_C6_0
+#define ICC_CTLR_EL1                 S3_0_C12_C12_4
+#define ICC_SRE_EL1                  S3_0_C12_C12_5
+#define ICC_GRPEN1_EL1                       S3_0_C12_C12_7
+
+#define ICC_SRE_EL2                  S3_4_C12_C9_5
+
+#define ICH_VSEIR_EL2                        S3_4_C12_C9_4
+#define ICH_HCR_EL2                  S3_4_C12_C11_0
+#define ICH_VTR_EL2                  S3_4_C12_C11_1
+#define ICH_MISR_EL2                 S3_4_C12_C11_2
+#define ICH_EISR_EL2                 S3_4_C12_C11_3
+#define ICH_ELSR_EL2                 S3_4_C12_C11_5
+#define ICH_VMCR_EL2                 S3_4_C12_C11_7
+
+#define __LR0_EL2(x)                 S3_4_C12_C12_ ## x
+#define __LR8_EL2(x)                 S3_4_C12_C13_ ## x
+
+#define ICH_LR0_EL2                  __LR0_EL2(0)
+#define ICH_LR1_EL2                  __LR0_EL2(1)
+#define ICH_LR2_EL2                  __LR0_EL2(2)
+#define ICH_LR3_EL2                  __LR0_EL2(3)
+#define ICH_LR4_EL2                  __LR0_EL2(4)
+#define ICH_LR5_EL2                  __LR0_EL2(5)
+#define ICH_LR6_EL2                  __LR0_EL2(6)
+#define ICH_LR7_EL2                  __LR0_EL2(7)
+#define ICH_LR8_EL2                  __LR8_EL2(0)
+#define ICH_LR9_EL2                  __LR8_EL2(1)
+#define ICH_LR10_EL2                 __LR8_EL2(2)
+#define ICH_LR11_EL2                 __LR8_EL2(3)
+#define ICH_LR12_EL2                 __LR8_EL2(4)
+#define ICH_LR13_EL2                 __LR8_EL2(5)
+#define ICH_LR14_EL2                 __LR8_EL2(6)
+#define ICH_LR15_EL2                 __LR8_EL2(7)
+
+#define __AP0Rx_EL2(x)                       S3_4_C12_C8_ ## x
+#define ICH_AP0R0_EL2                        __AP0Rx_EL2(0)
+#define ICH_AP0R1_EL2                        __AP0Rx_EL2(1)
+#define ICH_AP0R2_EL2                        __AP0Rx_EL2(2)
+#define ICH_AP0R3_EL2                        __AP0Rx_EL2(3)
Quoted 8 lines

include/linux/irqchip/arm-gic-v3.h line 174

Quoted 6 lines
include/linux/irqchip/arm-gic-v3.h new file +6 -0
+
+#define __AP1Rx_EL2(x)                       S3_4_C12_C9_ ## x
+#define ICH_AP1R0_EL2                        __AP1Rx_EL2(0)
+#define ICH_AP1R1_EL2                        __AP1Rx_EL2(1)
+#define ICH_AP1R2_EL2                        __AP1Rx_EL2(2)
+#define ICH_AP1R3_EL2                        __AP1Rx_EL2(3)

That was fun to review.

If you think so, then you're ready to join the lucky team of madmen reviewing DT patches... ;-)

Thanks anyway!

        M.

--
Jazz is not dead. It just smells funny...

#7 lore raw

PATCH 1/18 arm64: initial support for GICv3

Quoted 528 lines · Marc Zyngier

Hi Christoffer,

Quoted 57 lines · Christoffer Dall
Quoted 56 lines · Marc Zyngier

The Generic Interrupt Controller (version 3) offers services that are similar to GICv2, with a number of additional features:
- Affinity routing based on the CPU MPIDR (ARE)
- System register for the CPU interfaces (SRE)
- Support for more that 8 CPUs
- Locality-specific Peripheral Interrupts (LPIs)
- Interrupt Translation Services (ITS)

This patch adds preliminary support for GICv3 with ARE and SRE, non-secure mode only. It relies on higher exception levels to grant ARE and SRE access.

Support for LPI and ITS will be added at a later time.

---
 arch/arm64/Kconfig                 |   1 +
 arch/arm64/kernel/head.S           |  14 +
 arch/arm64/kernel/hyp-stub.S       |  41 ++-
 drivers/irqchip/Kconfig            |   5 +
 drivers/irqchip/Makefile           |   1 +
 drivers/irqchip/irq-gic-v3.c       | 665 +++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h | 190 +++++++++++
 7 files changed, 915 insertions(+), 2 deletions(-)
 create mode 100644 drivers/irqchip/irq-gic-v3.c
 create mode 100644 include/linux/irqchip/arm-gic-v3.h
arch/arm64/Kconfig +1 -0
index dd4327f..3a7e26d 100644
@@ -9,6 +9,7 @@ config ARM64
      select ARM_AMBA
      select ARM_ARCH_TIMER
      select ARM_GIC
+     select ARM_GIC_V3
      select BUILDTIME_EXTABLE_SORT
      select CLONE_BACKWARDS
      select COMMON_CLK
arch/arm64/kernel/head.S +1 -0
index 0b281ff..522649b 100644
@@ -22,6 +22,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -183,6 +184,19 @@ CPU_LE(  bic     x0, x0, #(3 << 24)      )       // Clear the EE and E0E bits for EL1
      msr     cnthctl_el2, x0
      msr     cntvoff_el2, xzr                // Clear virtual offset
 
Quoted 14 lines

arch/arm64/kernel/head.S CPU_LE( bic x0, x0, #(3 << 24) ) // Clear the EE and E0E bits for EL1 · line 197

Quoted 11 lines
arch/arm64/kernel/head.S +11 -0
+     /* GICv3 system register access */
+     mrs     x0, id_aa64pfr0_el1
+     ubfx    x0, x0, #24, #4
+     cmp     x0, #1
+     b.ne    3f
+
+     mrs     x0, ICC_SRE_EL2                 // Set ICC_SRE_EL2.SRE==1
+     orr     x0, x0, #1
+     msr     ICC_SRE_EL2, x0
+     isb                                     // Make sure SRE is now 1
+     msr     ICH_HCR_EL2, xzr                // Reset ICC_HCR_EL2 to defaults

Why do we not just set ICC_SRE_EL2.Enable here, but instead do the whole exception dance below?

That's a very good question. The more I think of it, the more I believe it to be completely unnecessary.

I'm going to remove the trapping and see what breaks. Or who screams.

Quoted 21 lines
Quoted 21 lines
arch/arm64/kernel/head.S +2 -0
+
+3:
      /* Populate ID registers. */
      mrs     x0, midr_el1
      mrs     x1, mpidr_el1
arch/arm64/kernel/hyp-stub.S +1 -0
index 0959611..c850d1e 100644
@@ -19,6 +19,7 @@
 
 #include <linux/init.h>
 #include <linux/linkage.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -51,16 +52,52 @@ ENDPROC(__hyp_stub_vectors)
 
      .align 11
 
Quoted 16 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 65

Quoted 11 lines
arch/arm64/kernel/hyp-stub.S +10 -0
+#define sysreg_to_iss(op0, op1, crn, crm, op2)       \
+     (((op0 & 3) << 20) | ((op2 & 7 ) << 17) | \
+      ((op1 & 7 ) << 14) | ((crn & 15) << 10) | \
+      ((crm & 15) << 1))
+
 el1_sync:
+     // Hack alert: we don't have a stack, and we can't use the one
+     // from EL1 because our MMU is off. Thankfully, we have
+     // tpidr_el2, which is under our complete control, and
+     // tpidr_el0, which can be used as long as we haven't reached
+     // userspace. Oh hum...

Hmmm. I feel like either we should tone down the hack alert claim, because this is after all safe and we just explain what's going on, or we should just allocate a small stack space following el1_sync and set the SP upon entry to this handler?

Allocating a stack here is something I'd rather avoid (how big?). But yeah, I can probably tone the horror down a notch - I got used to see that code... ;-)

Quoted 2 lines

Hopefully none of the code that will ever call into the sysregs trap ever makes the same assumption that user space is not called yet ;)

Well, I'm going to nuke that code anyway. I'm almost convinced that we don't need it.

Quoted 23 lines
Quoted 23 lines
arch/arm64/kernel/hyp-stub.S +14 -2
+     msr     tpidr_el2, x1                   // Preserve x1
+
      mrs     x1, esr_el2
      lsr     x1, x1, #26
      cmp     x1, #0x16
-     b.ne    2f                              // Not an HVC trap
+     b.ne    3f                              // Not an HVC trap
+
+     // Handle getting/setting vbar_el2
      cbz     x0, 1f
      msr     vbar_el2, x0                    // Set vbar_el2
      b       2f
 1:   mrs     x0, vbar_el2                    // Return vbar_el2
-2:   eret
+2:   mrs     x1, tpidr_el2                   // Restore x1
+     eret
+
+3:   cmp     x1, #0x18                       // MSR/MRS?
+     b.ne    2b
+
+     // Handle trapped sys reg access
+     msr     tpidr_el0, x0                   // Preserve x0
+     mrs     x1, esr_el2
Quoted 8 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 91

Quoted 5 lines
arch/arm64/kernel/hyp-stub.S +5 -0
+     ldr     x0, =sysreg_to_iss(3, 7, 15, 15, 7)
+     and     x1, x1, x0
+     ldr     x0, =sysreg_to_iss(3, 0, 12, 12, 5)
+     eor     x0, x0, x1                      // Check for ICC_SRE_EL1
+     cbnz    x0, 4f                          // Something bad happened

Shouldn't we try to do something here, like go to panic on the exception return here, or not worth it?

Code is now gone... ;-)

Quoted 339 lines
Quoted 339 lines
arch/arm64/kernel/hyp-stub.S +9 -0
+
+     // Set ICC_SRE_EL1 access enable
+     mrs     x1, ICC_SRE_EL2
+     orr     x1, x1, #(1 << 3)
+     msr     ICC_SRE_EL2, x1
+
+4:   mrs     x0, tpidr_el0                   // Restore x0
+     mrs     x1, tpidr_el2                   // Restore x1
+     eret
 ENDPROC(el1_sync)
 
 .macro invalid_vector        label
drivers/irqchip/Kconfig +5 -0
index 61ffdca..5b55c46 100644
@@ -10,6 +10,11 @@ config ARM_GIC
 config GIC_NON_BANKED
      bool
 
+config ARM_GIC_V3
+     bool
+     select IRQ_DOMAIN
+     select MULTI_IRQ_HANDLER
+
 config ARM_NVIC
      bool
      select IRQ_DOMAIN
drivers/irqchip/Makefile +1 -0
index 86b484c..e6ac9fd 100644
@@ -14,6 +14,7 @@ obj-$(CONFIG_ORION_IRQCHIP)         += irq-orion.o
 obj-$(CONFIG_ARCH_SUNXI)             += irq-sun4i.o
 obj-$(CONFIG_ARCH_SPEAR3XX)          += spear-shirq.o
 obj-$(CONFIG_ARM_GIC)                        += irq-gic.o
+obj-$(CONFIG_ARM_GIC_V3)             += irq-gic-v3.o
 obj-$(CONFIG_ARM_NVIC)                       += irq-nvic.o
 obj-$(CONFIG_ARM_VIC)                        += irq-vic.o
 obj-$(CONFIG_IMGPDC_IRQ)             += irq-imgpdc.o
drivers/irqchip/irq-gic-v3.c new file +293 -0
new file mode 100644
index 0000000..487e061
@@ -0,0 +1,665 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/cputype.h>
+#include <asm/exception.h>
+#include <asm/smp_plat.h>
+
+#include "irqchip.h"
+
+struct gic_chip_data {
+     void __iomem            *dist_base;
+     void __iomem            **redist_base;
+     void __percpu __iomem   **rdist;
+     struct irq_domain       *domain;
+     u64                     redist_stride;
+     u32                     redist_regions;
+     unsigned int            irq_nr;
+};
+
+static struct gic_chip_data gic_data __read_mostly;
+
+#define gic_data_rdist_rd_base()     (*__this_cpu_ptr(gic_data.rdist))
+#define gic_data_rdist_sgi_base()    (gic_data_rdist_rd_base() + SZ_64K)
+
+static DEFINE_RAW_SPINLOCK(dist_lock);
+
+#define DEFAULT_PMR_VALUE    0xf0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)
+{
+     if (d->hwirq < 32)      /* SGI+PPI -> SGI_base for this CPU */
+             return gic_data_rdist_sgi_base();
+
+     if (d->hwirq <= 1023)   /* SPI -> dist_base */
+             return gic_data.dist_base;
+
+     if (d->hwirq >= 8192)
+             BUG();          /* LPI Detected!!! */
+
+     return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+     return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
+{
+     u32 val;
+
+     do {
+             val = readl_relaxed(base + GICD_CTLR);
+             cpu_relax();
+     } while (val & GICD_CTLR_RWP);
+}
+
+/* Wait for completion of a distributor change */
+static void gic_dist_wait_for_rwp(void)
+{
+     gic_do_wait_for_rwp(gic_data.dist_base);
+}
+
+/* Wait for completion of a redistributor change */
+static void gic_redist_wait_for_rwp(void)
+{
+     gic_do_wait_for_rwp(gic_data_rdist_rd_base());
+}
+
+static void gic_wait_for_rwp(int irq)
+{
+     if (irq < 32)
+             gic_redist_wait_for_rwp();
+     else
+             gic_dist_wait_for_rwp();
+}
+
+/* Low level accessors */
+static u64 gic_read_iar(void)
+{
+     u64 irqstat;
+
+     asm volatile("mrs %0, " reg(ICC_IAR1_EL1) : "=r" (irqstat));
+     return irqstat;
+}
+
+static void gic_write_pmr(u64 val)
+{
+     asm volatile("msr " reg(ICC_PMR_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_write_ctlr(u64 val)
+{
+     asm volatile("msr " reg(ICC_CTLR_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_write_grpen1(u64 val)
+{
+     asm volatile("msr " reg(ICC_GRPEN1_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_write_sgi1r(u64 val)
+{
+     asm volatile("msr " reg(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_enable_sre(void)
+{
+     u64 val;
+
+     /* This access is likely to trap into EL2 */
+     asm volatile("mrs %0, " reg(ICC_SRE_EL1) : "=r" (val));
+     val |= GICC_SRE_EL1_SRE;
+     asm volatile("msr " reg(ICC_SRE_EL1) ", %0" : : "r" (val));
+     isb();
+}
+
+static void gic_enable_redist(void)
+{
+     void __iomem *rbase;
+     u32 val;
+
+     rbase = gic_data_rdist_rd_base();
+
+     /* Wake up this CPU redistributor */
+     val = readl_relaxed(rbase + GICR_WAKER);
+     val &= ~GICR_WAKER_ProcessorSleep;
+     writel_relaxed(val, rbase + GICR_WAKER);
+
+     do {
+             val = readl_relaxed(rbase + GICR_WAKER);
+             cpu_relax();
+     } while (val & GICR_WAKER_ChildrenAsleep);
+}
+
+/*
+ * Routines to acknowledge, disable and enable interrupts
+ */
+static void gic_mask_irq(struct irq_data *d)
+{
+     u32 mask = 1 << (gic_irq(d) % 32);
+
+     raw_spin_lock(&dist_lock);
+     writel_relaxed(mask, gic_dist_base(d) + GICD_ICENABLER + (gic_irq(d) / 32) * 4);
+     gic_wait_for_rwp(gic_irq(d));
+     raw_spin_unlock(&dist_lock);
+}
+
+static void gic_unmask_irq(struct irq_data *d)
+{
+     u32 mask = 1 << (gic_irq(d) % 32);
+
+     raw_spin_lock(&dist_lock);
+     writel_relaxed(mask, gic_dist_base(d) + GICD_ISENABLER + (gic_irq(d) / 32) * 4);
+     gic_wait_for_rwp(gic_irq(d));
+     raw_spin_unlock(&dist_lock);
+}
+
+static void gic_eoi_irq(struct irq_data *d)
+{
+     gic_write_eoir(gic_irq(d));
+}
+
+static int gic_set_type(struct irq_data *d, unsigned int type)
+{
+     void __iomem *base = gic_dist_base(d);
+     unsigned int irq = gic_irq(d);
+     u32 enablemask = 1 << (irq % 32);
+     u32 enableoff = (irq / 32) * 4;
+     u32 confmask = 0x2 << ((irq % 16) * 2);
+     u32 confoff = (irq / 16) * 4;
+     bool enabled = false;
+     u32 val;
+
+     /* Interrupt configuration for SGIs can't be changed */
+     if (irq < 16)
+             return -EINVAL;
+
+     if (type != IRQ_TYPE_LEVEL_HIGH && type != IRQ_TYPE_EDGE_RISING)
+             return -EINVAL;
+
+     raw_spin_lock(&dist_lock);
+
+     val = readl_relaxed(base + GICD_ICFGR + confoff);
+     if (type == IRQ_TYPE_LEVEL_HIGH)
+             val &= ~confmask;
+     else if (type == IRQ_TYPE_EDGE_RISING)
+             val |= confmask;
+
+     /*
+      * As recommended by the spec, disable the interrupt before changing
+      * the configuration
+      */
+     if (readl_relaxed(base + GICD_ISENABLER + enableoff) & enablemask) {
+             writel_relaxed(enablemask, base + GICD_ICENABLER + enableoff);
+             gic_wait_for_rwp(irq);
+             enabled = true;
+     }
+
+     writel_relaxed(val, base + GICD_ICFGR + confoff);
+
+     if (enabled) {
+             writel_relaxed(enablemask, base + GICD_ISENABLER + enableoff);
+             gic_wait_for_rwp(irq);
+     }
+
+     raw_spin_unlock(&dist_lock);
+
+     return 0;
+}
+
+static int gic_retrigger(struct irq_data *d)
+{
+     return -ENXIO;
+}
+
+static u64 gic_mpidr_to_affinity(u64 mpidr)
+{
+     /* Make sure we don't broadcast the interrupt */
+     return mpidr & ~GICD_IROUTER_SPI_MODE_ANY;
+}
+
+static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
+{
+     u64 irqnr;
+
+     do {
+             irqnr = gic_read_iar();
+
+             if (likely(irqnr > 15 && irqnr < 1021)) {
+                     irqnr = irq_find_mapping(gic_data.domain, irqnr);
+                     handle_IRQ(irqnr, regs);
+                     continue;
+             }
+             if (irqnr < 16) {
+                     gic_write_eoir(irqnr);
+#ifdef CONFIG_SMP
+                     handle_IPI(irqnr, regs);
+#else
+                     WARN_ONCE(true, "Unexpected SGI received!\n");
+#endif
+                     continue;
+             }
+     } while (irqnr != 0x3ff);
+}
+
+static void __init gic_dist_init(void)
+{
+     unsigned int i;
+     u64 affinity;
+     int gic_irqs = gic_data.irq_nr;
+     void __iomem *base = gic_data.dist_base;
+
+     /* Disable the distributor */
+     writel_relaxed(0, base + GICD_CTLR);
+
+     /*
+      * Set all global interrupts to be level triggered, active low.
+      */
+     for (i = 32; i < gic_data.irq_nr; i += 16)
+             writel_relaxed(0, base + GICD_ICFGR + i / 4);
+
+     /*
+      * Set priority on all global interrupts.
+      */
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 299

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+     for (i = 32; i < gic_irqs; i += 4)
+             writel_relaxed(0xa0a0a0a0, base + GICD_IPRIORITYR + i);
+
+     /*
+      * Disable all interrupts.  Leave the PPI and SGIs alone
+      * as these enables are banked registers.

as these are enabled through banked registers?

"The enable bits are banked registers", but I can see where the confision comes from. I'll rewrite this comment.

Quoted 1 line

also, should banked registers be redistributor registers?

Yup.

Quoted 10 lines

drivers/irqchip/irq-gic-v3.c line 307

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+      */
+     for (i = 32; i < gic_irqs; i += 32)
+             writel_relaxed(0xffffffff, base + GICD_ICENABLER + i / 8);
+
+     gic_dist_wait_for_rwp();
+
+     writel_relaxed(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+                    base + GICD_CTLR);

why set GICD_CTLR_ENABLE_G1 ?

Quoted 2 lines

Believe it or not, some people insist on running in secure mode.

Ah, didn't consider this. Maybe because the cover-letter ssays this patch only supports "Non-Secure Group-1 interrupts only".

Quoted 22 lines
Quoted 13 lines
Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+
+     /*
+      * Set all global interrupts to the boot CPU only. ARE must be
+      * enabled.
+      */
+     affinity = gic_mpidr_to_affinity(read_cpuid_mpidr());
+     for (i = 32; i < gic_data.irq_nr; i++)
+             writeq_relaxed(affinity, base + GICD_IROUTER + i * 8);
+}
+
+static int __init gic_populate_rdist(void)
+{
+     u64 mpidr = cpu_logical_map(smp_processor_id());
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 326

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+     u64 typer;
+     u64 aff;
+     int i;
+
+     aff  = mpidr & ((1 << 24) - 1);
+     aff |= (mpidr >> 8) & (0xffUL << 24);

Some defines would be nice for this if it makes sense in any way.

Quoted 5 lines

There is at least 3 methods GICv3 uses to express affinity. At some point, I gave up and decided that I needed mental support.

I'll update it with the appropriate macros (MPIDR_AFFINITY_LEVEL and co).

I don't feel strongly about it, maybe we need giant diagrams on A1 paper instead :)

Quoted 58 lines
Quoted 49 lines
Quoted 49 lines
drivers/irqchip/irq-gic-v3.c new file +49 -0
+
+     for (i = 0; i < gic_data.redist_regions; i++) {
+             void __iomem *ptr = gic_data.redist_base[i];
+             u32 reg;
+
+             reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
+             if (reg != 0x30 && reg != 0x40) { /* We're in trouble... */
+                     pr_warn("No redistributor present @%p\n", ptr);
+                     break;
+             }
+
+             do {
+                     typer = readq_relaxed(ptr + GICR_TYPER);
+                     if ((typer >> 32) == aff) {
+                             gic_data_rdist_rd_base() = ptr;
+                             pr_info("CPU%d: found redistributor %llx @%p\n",
+                                     smp_processor_id(),
+                                     (unsigned long long) mpidr, ptr);
+                             return 0;
+                     }
+
+                     if (gic_data.redist_stride) {
+                             ptr += gic_data.redist_stride;
+                     } else {
+                             ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+                             if (typer & GICR_TYPER_VLPIS)
+                                     ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+                     }
+             } while (!(typer & GICR_TYPER_LAST));
+     }
+
+     /* We couldn't even deal with ourselves... */
+     WARN(true, "CPU%d: mpidr %lx has no re-distributor!\n",
+          smp_processor_id(), (unsigned long)mpidr);
+     return -ENODEV;
+}
+
+static void __init gic_cpu_init(void)
+{
+     void __iomem *rbase;
+     int i;
+
+     /* Register ourselves with the rest of the world */
+     if (gic_populate_rdist())
+             return;
+
+     gic_enable_redist();
+
+     rbase = gic_data_rdist_sgi_base();
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 381

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+
+     /*
+      * Set priority on PPI and SGI interrupts
+      */
+     for (i = 0; i < 32; i += 4)
+             writel_relaxed(0xa0a0a0a0, rbase + GICR_IPRIORITYR0 + i * 4 / 4);

(i * 4) / 4 == i, last I checked.

Quoted 4 lines

Recurring comment. I decided to keep it inline with the GICv2 practice. Read it as i * increment / interrupt-per-word. Maybe not worth it if it causes more problem than anything else...

Yeah, I noticed it in the GICv2 code afterwards, but I don't think you're consistent with this approach throughout the file. I personaly find this notation mind-boggingly-difficult, but that is a personal observation.

Quoted 10 lines
Quoted 6 lines

drivers/irqchip/irq-gic-v3.c line 385

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+
+     /*
+      * Disable all PPI interrupts, ensure all SGI interrupts are
+      * enabled.

Why do we disable all PPIs?

Because interrupts should default to being disabled? They will get enabled when you do a request_irq.

Duh, yeah, and SGI don't need to be disabled because we're in control of generating them I guess. Thanks for the explanation.

Quoted 41 lines
Quoted 8 lines
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+      */
+     writel_relaxed(0xffff0000, rbase + GICR_ICENABLER0);
+     writel_relaxed(0x0000ffff, rbase + GICR_ISENABLER0);
+
+     gic_redist_wait_for_rwp();
+
+     /* Enable system registers */
+     gic_enable_sre();
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 398

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+
+     /* Set priority mask register */
+     gic_write_pmr(DEFAULT_PMR_VALUE);
+
+     /* EOI drops priority too (mode 0) */

you mean deactives the interrupt too, right? It always performs a priority drop doesn't it?

Yup. Just my usual brainfart.

Quoted 13 lines
Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+     gic_write_ctlr(GICC_CTLR_EL1_EOImode_drop_dir);
+
+     /* ... and let's hit the road... */
+     gic_write_grpen1(1);
+}
+
+#ifdef CONFIG_SMP
+static int __init gic_secondary_init(struct notifier_block *nfb,
+                                  unsigned long action, void *hcpu)
+{
+     if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+             gic_cpu_init();
+     return NOTIFY_OK;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 416

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+}
+
+/*
+ * Notifier for enabling the GIC CPU interface. Set an arbitrarily high
+ * priority because the GIC needs to be up before the ARM generic timers.

I actually don't understand this bit, but it also sits in the original code, so I assume it's correct.

Quoted 5 lines

GIC and timer on the secondary CPUs are intialized using the CPU notifiers. Since the need the GIC to be up and running before the timers can enable their interrupts, we use the relative priorities of the notifiers to order them.

thanks again for the explanation.

Quoted 29 lines
Quoted 20 lines
Quoted 20 lines
drivers/irqchip/irq-gic-v3.c new file +20 -0
+ */
+static struct notifier_block __initdata gic_cpu_notifier = {
+     .notifier_call = gic_secondary_init,
+     .priority = 100,
+};
+
+static u16 gic_compute_target_list(int *base_cpu, const struct cpumask *mask,
+                                u64 cluster_id)
+{
+     int cpu = *base_cpu;
+     u64 mpidr = cpu_logical_map(cpu);
+     u16 tlist = 0;
+
+     while (cpu < nr_cpu_ids) {
+             /*
+              * If we ever get a cluster of more than 16 CPUs, just
+              * scream and skip that CPU.
+              */
+             if (WARN_ON((mpidr & 0xff) >= 16))
+                     goto out;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 441

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+
+             tlist |= 1 << (mpidr & 0xf);
+
+             cpu = cpumask_next(cpu, mask);
+             mpidr = cpu_logical_map(cpu);

Are you not accessing memory beyond the __cpu_logical_map array here when the cpu is the last one at the beginning of the loop?

Quoted 2 lines

Nice catch! I need to test cpu here before carrying on...

You could also simply use for_each_cpu_mask here and in the beginning of the loop increment the first cpu to *base_cpu if it makes the code cleaner. Not sure.

Quoted 48 lines
Quoted 39 lines
Quoted 39 lines
drivers/irqchip/irq-gic-v3.c new file +39 -0
+
+             if (cluster_id != (mpidr & ~0xffUL)) {
+                     cpu--;
+                     goto out;
+             }
+     }
+out:
+     *base_cpu = cpu;
+     return tlist;
+}
+
+static void gic_send_sgi(u64 cluster_id, u16 tlist, unsigned int irq)
+{
+     u64 val;
+
+     val  = (cluster_id & 0xff00ff0000UL) << 16; /* Aff3 + Aff2 */
+     val |= (cluster_id & 0xff00) << 8;          /* Aff1 */
+     val |= irq << 24;
+     val |= tlist;
+
+     pr_debug("CPU%d: ICC_SGI1R_EL1 %llx\n", smp_processor_id(), val);
+     gic_write_sgi1r(val);
+}
+
+static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
+{
+     int cpu;
+
+     if (WARN_ON(irq >= 16))
+             return;
+
+     /*
+      * Ensure that stores to Normal memory are visible to the
+      * other CPUs before issuing the IPI.
+      */
+     dsb();
+
+     for_each_cpu_mask(cpu, *mask) {
+             u64 cluster_id = cpu_logical_map(cpu) & ~0xffUL;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 483

Quoted 3 lines
drivers/irqchip/irq-gic-v3.c new file +3 -0
+             u16 tlist;
+
+             tlist = gic_compute_target_list(&cpu, mask, cluster_id);

I think it would be potentially slightly prettier if this function returned the last cpu in the cluster so the loop iterator mangling was kept local to this function and took a pointer to the tlist it was to fill instead, but, this is a minor detail.

Quoted 2 lines

Yeah, maybe. I'll try that and see how it looks. it's quite messy anyway...

yeah, nothing I feel strongly about anyway.

Quoted 58 lines
Quoted 49 lines
Quoted 49 lines
drivers/irqchip/irq-gic-v3.c new file +49 -0
+             gic_send_sgi(cluster_id, tlist, irq);
+     }
+}
+
+static void gic_smp_init(void)
+{
+     set_smp_cross_call(gic_raise_softirq);
+     register_cpu_notifier(&gic_cpu_notifier);
+}
+
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+                         bool force)
+{
+     unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
+     void __iomem *reg;
+     u64 val;
+
+     if (gic_irq(d) < 32)
+             return -EINVAL;
+
+     reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+     val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+     writeq_relaxed(val, reg);
+
+     return IRQ_SET_MASK_OK;
+}
+#else
+#define gic_set_affinity     NULL
+#define gic_smp_init()               do { } while(0)
+#endif
+
+static struct irq_chip gic_chip = {
+     .name                   = "GICv3",
+     .irq_mask               = gic_mask_irq,
+     .irq_unmask             = gic_unmask_irq,
+     .irq_eoi                = gic_eoi_irq,
+     .irq_set_type           = gic_set_type,
+     .irq_retrigger          = gic_retrigger,
+     .irq_set_affinity       = gic_set_affinity,
+};
+
+static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
+                             irq_hw_number_t hw)
+{
+     if (hw < 32) {
+             irq_set_percpu_devid(irq);
+             irq_set_chip_and_handler(irq, &gic_chip,
+                                      handle_percpu_devid_irq);
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 537

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+             set_irq_flags(irq, IRQF_VALID | IRQF_NOAUTOEN);
+     } else {
+             irq_set_chip_and_handler(irq, &gic_chip,
+                                      handle_fasteoi_irq);
+             set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);

will this ever be compiled for arch/arm? If not, does it make sense to keep the set_irq_flags calls?

Quoted 4 lines

Not impossible at all, by the look of it. That will require some effort, but if someone feels like it, I don't see the point in actuvely removing that code.

ok

Quoted 61 lines
Quoted 44 lines
Quoted 44 lines
drivers/irqchip/irq-gic-v3.c new file +44 -0
+     }
+     irq_set_chip_data(irq, d->host_data);
+     return 0;
+}
+
+static int gic_irq_domain_xlate(struct irq_domain *d,
+                             struct device_node *controller,
+                             const u32 *intspec, unsigned int intsize,
+                             unsigned long *out_hwirq, unsigned int *out_type)
+{
+     if (d->of_node != controller)
+             return -EINVAL;
+     if (intsize < 3)
+             return -EINVAL;
+
+     switch(intspec[0]) {
+     case 0:                 /* SPI */
+             *out_hwirq = intspec[1] + 32;
+             break;
+     case 1:                 /* PPI */
+             *out_hwirq = intspec[1] + 16;
+             break;
+     default:
+             return -EINVAL;
+     }
+
+     *out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
+     return 0;
+}
+
+static const struct irq_domain_ops gic_irq_domain_ops = {
+     .map = gic_irq_domain_map,
+     .xlate = gic_irq_domain_xlate,
+};
+
+static int __init gic_of_init(struct device_node *node, struct device_node *parent)
+{
+     void __iomem *dist_base;
+     void __iomem **redist_base;
+     u64 redist_stride;
+     u32 redist_regions;
+     u32 reg;
+     int gic_irqs;
+     int err;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 587

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+     int i;
+
+     dist_base = of_iomap(node, 0);
+     if (!dist_base) {
+             pr_warn("%s: unable to map gic dist registers\n",
+                     node->full_name);

shouldn't these be pr_err?

Indeed.

Quoted 6 lines
Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+             return -ENXIO;
+     }
+
+     reg = readl_relaxed(dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
+     if (reg != 0x30 && reg != 0x40) {
Quoted 6 lines
Quoted 2 lines

Is everything we do going to work with a GICv4? Anything else we need to initialize if it's a GICv4?

GICv4 is a strict superset of GICv3, and it should all just work as long as you don't try to use GICv4 specific features. *Should*.

cool

Quoted 268 lines
Quoted 22 lines
Quoted 22 lines
drivers/irqchip/irq-gic-v3.c new file +22 -0
+             pr_warn("%s: no distributor detected, giving up\n",
+                     node->full_name);
+             err = -ENODEV;
+             goto out_unmap_dist;
+     }
+
+     if (of_property_read_u32(node, "#redistributor-regions", &redist_regions))
+             redist_regions = 1;
+
+     redist_base = kzalloc(sizeof(*redist_base) * redist_regions, GFP_KERNEL);
+     if (!redist_base) {
+             err = -ENOMEM;
+             goto out_unmap_dist;
+     }
+
+     for (i = 0; i < redist_regions; i++) {
+             redist_base[i] = of_iomap(node, 1 + i);
+             if (!redist_base[i]) {
+                     pr_warn("%s: couldn't map region %d\n",
+                             node->full_name, i);
+                     err = -ENODEV;
+                     goto out_unmap_rdist;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 619

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+             }
+     }
+
+     if (of_property_read_u64(node, "redistributor-stride", &redist_stride))
+             redist_stride = 0;

The binding doesn't specify this as a two-cell property, so I assume this won't work if you follow the example in the bindings doc.

Yup. I'll update the DT. Nice catch.

Quoted 138 lines
Quoted 138 lines
drivers/irqchip/irq-gic-v3.c new file +46 -0
+
+     gic_data.dist_base = dist_base;
+     gic_data.redist_base = redist_base;
+     gic_data.redist_regions = redist_regions;
+     gic_data.redist_stride = redist_stride;
+
+     /*
+      * Find out how many interrupts are supported.
+      * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)
+      */
+     gic_irqs = readl_relaxed(gic_data.dist_base + GICD_TYPER) & 0x1f;
+     gic_irqs = (gic_irqs + 1) * 32;
+     if (gic_irqs > 1020)
+             gic_irqs = 1020;
+     gic_data.irq_nr = gic_irqs;
+
+     gic_data.domain = irq_domain_add_linear(node, gic_irqs - 16,
+                                             &gic_irq_domain_ops, &gic_data);
+     gic_data.rdist = alloc_percpu(typeof(*gic_data.rdist));
+
+     if (WARN_ON(!gic_data.domain) || WARN_ON(!gic_data.rdist)) {
+             err = -ENOMEM;
+             goto out_free;
+     }
+
+     set_handle_irq(gic_handle_irq);
+
+     gic_smp_init();
+     gic_dist_init();
+     gic_cpu_init();
+
+     return 0;
+
+out_free:
+     free_percpu(gic_data.rdist);
+out_unmap_rdist:
+     for (i = 0; i < redist_regions; i++)
+             if (redist_base[i])
+                     iounmap(redist_base[i]);
+     kfree(redist_base);
+out_unmap_dist:
+     iounmap(dist_base);
+     return err;
+}
+
+IRQCHIP_DECLARE(gic_v3, "arm,gic-v3", gic_of_init);
include/linux/irqchip/arm-gic-v3.h new file +86 -0
new file mode 100644
index 0000000..e4d060a
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __LINUX_IRQCHIP_ARM_GIC_V3_H
+#define __LINUX_IRQCHIP_ARM_GIC_V3_H
+
+/*
+ * Distributor registers. We assume we're running non-secure, with ARE
+ * being set. Secure-only and non-ARE registers are not described.
+ */
+#define GICD_CTLR                    0x0000
+#define GICD_TYPER                   0x0004
+#define GICD_IIDR                    0x0008
+#define GICD_STATUSR                 0x0010
+#define GICD_SETSPI_NSR                      0x0040
+#define GICD_CLRSPI_NSR                      0x0048
+#define GICD_SETSPI_SR                       0x0050
+#define GICD_CLRSPI_SR                       0x0058
+#define GICD_SEIR                    0x0068
+#define GICD_ISENABLER                       0x0100
+#define GICD_ICENABLER                       0x0180
+#define GICD_ISPENDR                 0x0200
+#define GICD_ICPENDR                 0x0280
+#define GICD_ISACTIVER                       0x0300
+#define GICD_ICACTIVER                       0x0380
+#define GICD_IPRIORITYR                      0x0400
+#define GICD_ICFGR                   0x0C00
+#define GICD_IROUTER                 0x6000
+#define GICD_PIDR2                   0xFFE8
+
+#define GICD_CTLR_RWP                        (1U << 31)
+#define GICD_CTLR_ARE_NS             (1U << 4)
+#define GICD_CTLR_ENABLE_G1A         (1U << 1)
+#define GICD_CTLR_ENABLE_G1          (1U << 0)
+
+#define GICD_IROUTER_SPI_MODE_ONE    (0U << 31)
+#define GICD_IROUTER_SPI_MODE_ANY    (1U << 31)
+
+#define GIC_PIDR2_ARCH_MASK          0xf0
+
+/*
+ * Re-Distributor registers, offsets from RD_base
+ */
+#define GICR_CTLR                    GICD_CTLR
+#define GICR_IIDR                    0x0004
+#define GICR_TYPER                   0x0008
+#define GICR_STATUSR                 GICD_STATUSR
+#define GICR_WAKER                   0x0014
+#define GICR_SETLPIR                 0x0040
+#define GICR_CLRLPIR                 0x0048
+#define GICR_SEIR                    GICD_SEIR
+#define GICR_PROPBASER                       0x0070
+#define GICR_PENDBASER                       0x0078
+#define GICR_INVLPIR                 0x00A0
+#define GICR_INVALLR                 0x00B0
+#define GICR_SYNCR                   0x00C0
+#define GICR_MOVLPIR                 0x0100
+#define GICR_MOVALLR                 0x0110
+#define GICR_PIDR2                   GICD_PIDR2
+
+#define GICR_WAKER_ProcessorSleep    (1U << 1)
+#define GICR_WAKER_ChildrenAsleep    (1U << 2)
+
+/*
+ * Re-Distributor registers, offsets from SGI_base
+ */
+#define GICR_ISENABLER0                      GICD_ISENABLER
+#define GICR_ICENABLER0                      GICD_ICENABLER
+#define GICR_ISPENDR0                        GICD_ISPENDR
+#define GICR_ICPENDR0                        GICD_ICPENDR
+#define GICR_ISACTIVER0                      GICD_ISACTIVER
+#define GICR_ICACTIVER0                      GICD_ICACTIVER
Quoted 8 lines

include/linux/irqchip/arm-gic-v3.h line 91

Quoted 5 lines
include/linux/irqchip/arm-gic-v3.h new file +5 -0
+#define GICR_IPRIORITYR0             GICD_IPRIORITYR
+#define GICR_ICFGR0                  GICD_ICFGR
+
+#define GICR_TYPER_VLPIS             (1U << 1)
+#define GICR_TYPER_LAST                      (1U << 4)

nit: why are these TYPER definitions down here when the TYPER register is defined above and so the WAKER bit definitions?

I'll group them in the next version.

Quoted 77 lines
Quoted 77 lines
include/linux/irqchip/arm-gic-v3.h new file +77 -0
+
+/*
+ * CPU interface registers
+ */
+#define GICC_CTLR_EL1_EOImode_drop_dir       (0U << 1)
+#define GICC_CTLR_EL1_EOImode_drop   (1U << 1)
+#define GICC_SRE_EL1_SRE             (1U << 0)
+
+/*
+ * Hypervisor interface registers (SRE only)
+ */
+#define GICH_LR_VIRTUAL_ID_MASK              ((1UL << 32) - 1)
+
+#define GICH_LR_EOI                  (1UL << 41)
+#define GICH_LR_GROUP                        (1UL << 60)
+#define GICH_LR_STATE                        (3UL << 62)
+#define GICH_LR_PENDING_BIT          (1UL << 62)
+#define GICH_LR_ACTIVE_BIT           (1UL << 63)
+
+#define GICH_MISR_EOI                        (1 << 0)
+#define GICH_MISR_U                  (1 << 1)
+
+#define GICH_HCR_EN                  (1 << 0)
+#define GICH_HCR_UIE                 (1 << 1)
+
+#define GICH_VMCR_CTLR_SHIFT         0
+#define GICH_VMCR_CTLR_MASK          (0x21f << GICH_VMCR_CTLR_SHIFT)
+#define GICH_VMCR_BPR1_SHIFT         18
+#define GICH_VMCR_BPR1_MASK          (7 << GICH_VMCR_BPR1_SHIFT)
+#define GICH_VMCR_BPR0_SHIFT         21
+#define GICH_VMCR_BPR0_MASK          (7 << GICH_VMCR_BPR0_SHIFT)
+#define GICH_VMCR_PMR_SHIFT          24
+#define GICH_VMCR_PMR_MASK           (0xffUL << GICH_VMCR_PMR_SHIFT)
+
+#define ICC_EOIR1_EL1                        S3_0_C12_C12_1
+#define ICC_IAR1_EL1                 S3_0_C12_C12_0
+#define ICC_SGI1R_EL1                        S3_0_C12_C11_5
+#define ICC_PMR_EL1                  S3_0_C4_C6_0
+#define ICC_CTLR_EL1                 S3_0_C12_C12_4
+#define ICC_SRE_EL1                  S3_0_C12_C12_5
+#define ICC_GRPEN1_EL1                       S3_0_C12_C12_7
+
+#define ICC_SRE_EL2                  S3_4_C12_C9_5
+
+#define ICH_VSEIR_EL2                        S3_4_C12_C9_4
+#define ICH_HCR_EL2                  S3_4_C12_C11_0
+#define ICH_VTR_EL2                  S3_4_C12_C11_1
+#define ICH_MISR_EL2                 S3_4_C12_C11_2
+#define ICH_EISR_EL2                 S3_4_C12_C11_3
+#define ICH_ELSR_EL2                 S3_4_C12_C11_5
+#define ICH_VMCR_EL2                 S3_4_C12_C11_7
+
+#define __LR0_EL2(x)                 S3_4_C12_C12_ ## x
+#define __LR8_EL2(x)                 S3_4_C12_C13_ ## x
+
+#define ICH_LR0_EL2                  __LR0_EL2(0)
+#define ICH_LR1_EL2                  __LR0_EL2(1)
+#define ICH_LR2_EL2                  __LR0_EL2(2)
+#define ICH_LR3_EL2                  __LR0_EL2(3)
+#define ICH_LR4_EL2                  __LR0_EL2(4)
+#define ICH_LR5_EL2                  __LR0_EL2(5)
+#define ICH_LR6_EL2                  __LR0_EL2(6)
+#define ICH_LR7_EL2                  __LR0_EL2(7)
+#define ICH_LR8_EL2                  __LR8_EL2(0)
+#define ICH_LR9_EL2                  __LR8_EL2(1)
+#define ICH_LR10_EL2                 __LR8_EL2(2)
+#define ICH_LR11_EL2                 __LR8_EL2(3)
+#define ICH_LR12_EL2                 __LR8_EL2(4)
+#define ICH_LR13_EL2                 __LR8_EL2(5)
+#define ICH_LR14_EL2                 __LR8_EL2(6)
+#define ICH_LR15_EL2                 __LR8_EL2(7)
+
+#define __AP0Rx_EL2(x)                       S3_4_C12_C8_ ## x
+#define ICH_AP0R0_EL2                        __AP0Rx_EL2(0)
+#define ICH_AP0R1_EL2                        __AP0Rx_EL2(1)
+#define ICH_AP0R2_EL2                        __AP0Rx_EL2(2)
+#define ICH_AP0R3_EL2                        __AP0Rx_EL2(3)
Quoted 8 lines

include/linux/irqchip/arm-gic-v3.h line 174

Quoted 6 lines
include/linux/irqchip/arm-gic-v3.h new file +6 -0
+
+#define __AP1Rx_EL2(x)                       S3_4_C12_C9_ ## x
+#define ICH_AP1R0_EL2                        __AP1Rx_EL2(0)
+#define ICH_AP1R1_EL2                        __AP1Rx_EL2(1)
+#define ICH_AP1R2_EL2                        __AP1Rx_EL2(2)
+#define ICH_AP1R3_EL2                        __AP1Rx_EL2(3)

That was fun to review.

Quoted 3 lines

If you think so, then you're ready to join the lucky team of madmen reviewing DT patches... ;-)

I could not have been more sarcastic in that last comment. It required a full liter of Haagen-Dazs to get through.

-Christoffer

#8 lore raw

PATCH 1/18 arm64: initial support for GICv3

In reply to Marc Zyngier

Hi Marc,

Quoted 14 lines · Marc Zyngier
arch/arm64/kernel/head.S +1 -0
index 0b281ff..522649b 100644
@@ -22,6 +22,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -183,6 +184,19 @@ CPU_LE(    bic     x0, x0, #(3 << 24)      )       // Clear the EE and E0E bits for EL1
        msr     cnthctl_el2, x0

arch/arm64/kernel/head.S CPU_LE( bic x0, x0, #(3 << 24) ) // Clear the EE and E0E bits for EL1 · line 194

Quoted 10 lines
arch/arm64/kernel/head.S +8 -0
        msr     cntvoff_el2, xzr                // Clear virtual offset
 
+       /* GICv3 system register access */
+       mrs     x0, id_aa64pfr0_el1
+       ubfx    x0, x0, #24, #4
+       cmp     x0, #1
+       b.ne    3f
+
+       mrs     x0, ICC_SRE_EL2                 // Set ICC_SRE_EL2.SRE==1
+       orr     x0, x0, #1

Don't you need to set the Enable bit (bit 3) too? Also, does setting ICC_SRE_EL2.SRE force KVM to use the ICH_* registers? If so, that seems like a pretty big decision to take this early on (there's not even an #ifdef CONFIG_ARM_GICV3).

Quoted 25 lines
arch/arm64/kernel/head.S +5 -0
+       msr     ICC_SRE_EL2, x0
+       isb                                     // Make sure SRE is now 1
+       msr     ICH_HCR_EL2, xzr                // Reset ICC_HCR_EL2 to defaults
+
+3:
        /* Populate ID registers. */
        mrs     x0, midr_el1
        mrs     x1, mpidr_el1
arch/arm64/kernel/hyp-stub.S +2 -0
index 0959611..c850d1e 100644
@@ -19,6 +19,7 @@
 
 #include <linux/init.h>
 #include <linux/linkage.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -51,16 +52,52 @@ ENDPROC(__hyp_stub_vectors)
 
        .align 11
 
+#define sysreg_to_iss(op0, op1, crn, crm, op2) \

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 65

Quoted 10 lines
arch/arm64/kernel/hyp-stub.S +9 -0
+       (((op0 & 3) << 20) | ((op2 & 7 ) << 17) | \
+        ((op1 & 7 ) << 14) | ((crn & 15) << 10) | \
+        ((crm & 15) << 1))
+
 el1_sync:
+       // Hack alert: we don't have a stack, and we can't use the one
+       // from EL1 because our MMU is off. Thankfully, we have
+       // tpidr_el2, which is under our complete control, and
+       // tpidr_el0, which can be used as long as we haven't reached
+       // userspace. Oh hum...

Yuck. Why can't we create a minimal stack when we initialise EL2 when booting the host? Even simpler, just use some .quads in here (you're already using a literal pool).

Quoted 16 lines
arch/arm64/kernel/hyp-stub.S +7 -2
+       msr     tpidr_el2, x1                   // Preserve x1
+
        mrs     x1, esr_el2
        lsr     x1, x1, #26
        cmp     x1, #0x16
-       b.ne    2f                              // Not an HVC trap
+       b.ne    3f                              // Not an HVC trap
+
+       // Handle getting/setting vbar_el2
        cbz     x0, 1f
        msr     vbar_el2, x0                    // Set vbar_el2
        b       2f
 1:     mrs     x0, vbar_el2                    // Return vbar_el2
-2:     eret
+2:     mrs     x1, tpidr_el2                   // Restore x1
+       eret

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 89

Quoted 10 lines
arch/arm64/kernel/hyp-stub.S +10 -0
+
+3:     cmp     x1, #0x18                       // MSR/MRS?
+       b.ne    2b
+
+       // Handle trapped sys reg access
+       msr     tpidr_el0, x0                   // Preserve x0
+       mrs     x1, esr_el2
+       ldr     x0, =sysreg_to_iss(3, 7, 15, 15, 7)
+       and     x1, x1, x0
+       ldr     x0, =sysreg_to_iss(3, 0, 12, 12, 5)

I think sysreg_to_iss actually makes this *less* readable! Why not simply #define the mask to extract OPx, CRx then compare with another #define for ICC_SRE_EL1? If we can represent those as immediates, you can avoid the literal loads too.

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 96

Quoted 7 lines
arch/arm64/kernel/hyp-stub.S +7 -0
+       eor     x0, x0, x1                      // Check for ICC_SRE_EL1
+       cbnz    x0, 4f                          // Something bad happened
+
+       // Set ICC_SRE_EL1 access enable
+       mrs     x1, ICC_SRE_EL2
+       orr     x1, x1, #(1 << 3)
+       msr     ICC_SRE_EL2, x1

Ah, ok, you do this part lazily. Does that make sense for the host? (i.e. is it possible for KVM to use SRE at EL2 but not the host at EL1?).

Quoted 49 lines
arch/arm64/kernel/hyp-stub.S +4 -0
+
+4:     mrs     x0, tpidr_el0                   // Restore x0
+       mrs     x1, tpidr_el2                   // Restore x1
+       eret
 ENDPROC(el1_sync)
 
 .macro invalid_vector  label
drivers/irqchip/irq-gic-v3.c new file +36 -0
new file mode 100644
index 0000000..487e061
@@ -0,0 +1,665 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/cputype.h>
+#include <asm/exception.h>
+#include <asm/smp_plat.h>
+
+#include "irqchip.h"
+
+struct gic_chip_data {
+       void __iomem            *dist_base;
+       void __iomem            **redist_base;

drivers/irqchip/irq-gic-v3.c line 46

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       void __percpu __iomem   **rdist;
+       struct irq_domain       *domain;
+       u64                     redist_stride;
+       u32                     redist_regions;
+       unsigned int            irq_nr;
+};
+
+static struct gic_chip_data gic_data __read_mostly;
+
+#define gic_data_rdist_rd_base()       (*__this_cpu_ptr(gic_data.rdist))

You should avoid __this_cpu_ptr (it's being removed afaik). Take a look at the patches from Christoph Lameter, e.g. 170d800af83f ("block: Replace __get_cpu_var uses").

Quoted 23 lines
drivers/irqchip/irq-gic-v3.c new file +23 -0
+#define gic_data_rdist_sgi_base()      (gic_data_rdist_rd_base() + SZ_64K)
+
+static DEFINE_RAW_SPINLOCK(dist_lock);
+
+#define DEFAULT_PMR_VALUE      0xf0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)
+{
+       if (d->hwirq < 32)      /* SGI+PPI -> SGI_base for this CPU */
+               return gic_data_rdist_sgi_base();
+
+       if (d->hwirq <= 1023)   /* SPI -> dist_base */
+               return gic_data.dist_base;
+
+       if (d->hwirq >= 8192)
+               BUG();          /* LPI Detected!!! */
+
+       return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+       return d->hwirq;

drivers/irqchip/irq-gic-v3.c line 79

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
+{
+       u32 val;
+
+       do {
+               val = readl_relaxed(base + GICD_CTLR);
+               cpu_relax();
+       } while (val & GICD_CTLR_RWP);

You should add a timeout in here to save us in the face of broken hardware (especially as you hold the distributor lock). See, for example, arm_smmu_tlb_sync(...) in the arm-smmu driver.

Thinking about it, is this the only point of the dist_lock (to ensure you wait only for your request)? Do we actually need locking for that?

Quoted 7 lines
drivers/irqchip/irq-gic-v3.c new file +7 -0
+}
+
+/* Wait for completion of a distributor change */
+static void gic_dist_wait_for_rwp(void)
+{
+       gic_do_wait_for_rwp(gic_data.dist_base);
+}

drivers/irqchip/irq-gic-v3.c line 96

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+/* Wait for completion of a redistributor change */
+static void gic_redist_wait_for_rwp(void)
+{
+       gic_do_wait_for_rwp(gic_data_rdist_rd_base());
+}
+
+static void gic_wait_for_rwp(int irq)
+{
+       if (irq < 32)

Maybe we should just bite the bullet and have helpers for gic_irq_is_sgi(irq) etc. so we can avoid these magic numbers all over the place.

drivers/irqchip/irq-gic-v3.c line 100

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+               gic_redist_wait_for_rwp();
+       else
+               gic_dist_wait_for_rwp();
+}

Saying that, I question the value of mixing this into a single function. gic_redist_wait_for_rwp absolutely *must* be called with preemption disabled and on the appropriate CPU for the IRQ in question, whilst the latter can be called anywhere. So even though you can call this single function from a bunch of places, you're really relying on the irq parameter being correct for the context of the caller.

I reckon it would actually be clearer to call the relevant wait function explicitly in the caller. Same for gic_dist_base.

drivers/irqchip/irq-gic-v3.c line 107

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+/* Low level accessors */
+static u64 gic_read_iar(void)
+{
+       u64 irqstat;
+
+       asm volatile("mrs %0, " reg(ICC_IAR1_EL1) : "=r" (irqstat));

Hmm, reg() is a terrible name for a macro :) Can't you just use stringify directly?

Quoted 16 lines
drivers/irqchip/irq-gic-v3.c new file +16 -0
+       return irqstat;
+}
+
+static void gic_write_pmr(u64 val)
+{
+       asm volatile("msr " reg(ICC_PMR_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_write_ctlr(u64 val)
+{
+       asm volatile("msr " reg(ICC_CTLR_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_write_grpen1(u64 val)

drivers/irqchip/irq-gic-v3.c line 133

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+{
+       asm volatile("msr " reg(ICC_GRPEN1_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_write_sgi1r(u64 val)
+{
+       asm volatile("msr " reg(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+       isb();
+}

I'm not convinced you *always* need these isbs. For example, the gic_handle_irq loop probably just needs one at the end, before returning.

In fact, looking at the spec, ICC_PMR_EL1 is actually self-synchronising so you don't need anything there. For SGI1R at least, you need a dsb to ensure write observability by the GIC (it's not clear to me whether you also need to the isb).

Quoted 17 lines
drivers/irqchip/irq-gic-v3.c new file +17 -0
+static void gic_enable_sre(void)
+{
+       u64 val;
+
+       /* This access is likely to trap into EL2 */
+       asm volatile("mrs %0, " reg(ICC_SRE_EL1) : "=r" (val));
+       val |= GICC_SRE_EL1_SRE;
+       asm volatile("msr " reg(ICC_SRE_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_enable_redist(void)
+{
+       void __iomem *rbase;
+       u32 val;
+
+       rbase = gic_data_rdist_rd_base();

drivers/irqchip/irq-gic-v3.c line 161

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+       /* Wake up this CPU redistributor */
+       val = readl_relaxed(rbase + GICR_WAKER);
+       val &= ~GICR_WAKER_ProcessorSleep;
+       writel_relaxed(val, rbase + GICR_WAKER);
+
+       do {
+               val = readl_relaxed(rbase + GICR_WAKER);
+               cpu_relax();
+       } while (val & GICR_WAKER_ChildrenAsleep);

Again, you want a timeout here.

Quoted 19 lines
drivers/irqchip/irq-gic-v3.c new file +19 -0
+}
+
+/*
+ * Routines to acknowledge, disable and enable interrupts
+ */
+static void gic_mask_irq(struct irq_data *d)
+{
+       u32 mask = 1 << (gic_irq(d) % 32);
+
+       raw_spin_lock(&dist_lock);
+       writel_relaxed(mask, gic_dist_base(d) + GICD_ICENABLER + (gic_irq(d) / 32) * 4);
+       gic_wait_for_rwp(gic_irq(d));
+       raw_spin_unlock(&dist_lock);
+}
+
+static void gic_unmask_irq(struct irq_data *d)
+{
+       u32 mask = 1 << (gic_irq(d) % 32);
+

drivers/irqchip/irq-gic-v3.c line 190

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       raw_spin_lock(&dist_lock);
+       writel_relaxed(mask, gic_dist_base(d) + GICD_ISENABLER + (gic_irq(d) / 32) * 4);
+       gic_wait_for_rwp(gic_irq(d));
+       raw_spin_unlock(&dist_lock);
+}
+
+static void gic_eoi_irq(struct irq_data *d)
+{
+       gic_write_eoir(gic_irq(d));
+}

Can you inline the code here (this is the only user) and move the isb out?

Quoted 37 lines
drivers/irqchip/irq-gic-v3.c new file +37 -0
+static int gic_set_type(struct irq_data *d, unsigned int type)
+{
+       void __iomem *base = gic_dist_base(d);
+       unsigned int irq = gic_irq(d);
+       u32 enablemask = 1 << (irq % 32);
+       u32 enableoff = (irq / 32) * 4;
+       u32 confmask = 0x2 << ((irq % 16) * 2);
+       u32 confoff = (irq / 16) * 4;
+       bool enabled = false;
+       u32 val;
+
+       /* Interrupt configuration for SGIs can't be changed */
+       if (irq < 16)
+               return -EINVAL;
+
+       if (type != IRQ_TYPE_LEVEL_HIGH && type != IRQ_TYPE_EDGE_RISING)
+               return -EINVAL;
+
+       raw_spin_lock(&dist_lock);
+
+       val = readl_relaxed(base + GICD_ICFGR + confoff);
+       if (type == IRQ_TYPE_LEVEL_HIGH)
+               val &= ~confmask;
+       else if (type == IRQ_TYPE_EDGE_RISING)
+               val |= confmask;
+
+       /*
+        * As recommended by the spec, disable the interrupt before changing
+        * the configuration
+        */
+       if (readl_relaxed(base + GICD_ISENABLER + enableoff) & enablemask) {
+               writel_relaxed(enablemask, base + GICD_ICENABLER + enableoff);
+               gic_wait_for_rwp(irq);
+               enabled = true;
+       }
+
+       writel_relaxed(val, base + GICD_ICFGR + confoff);

drivers/irqchip/irq-gic-v3.c line 238

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+       if (enabled) {
+               writel_relaxed(enablemask, base + GICD_ISENABLER + enableoff);
+               gic_wait_for_rwp(irq);
+       }
+
+       raw_spin_unlock(&dist_lock);
+
+       return 0;
+}

I really think this should be merged with the existing GIC driver -- the code is basically identical.

drivers/irqchip/irq-gic-v3.c line 243

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+static int gic_retrigger(struct irq_data *d)
+{
+       return -ENXIO;
+}

Why bother? (just don't assign the pointer in the chip).

drivers/irqchip/irq-gic-v3.c line 248

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+static u64 gic_mpidr_to_affinity(u64 mpidr)
+{
+       /* Make sure we don't broadcast the interrupt */
+       return mpidr & ~GICD_IROUTER_SPI_MODE_ANY;

Yikes, mpidr meets the GIC! I'd be a lot more comfortable with this if we used our mpidr accessors to construct the value which GICD_IROUTER expects (e.g. using the MPIDR_AFFINITY_LEVEL macro).

drivers/irqchip/irq-gic-v3.c line 256

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+}
+
+static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
+{
+       u64 irqnr;
+
+       do {
+               irqnr = gic_read_iar();

What stops the CPU speculating this access around the loop?

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+               if (likely(irqnr > 15 && irqnr < 1021)) {
+                       irqnr = irq_find_mapping(gic_data.domain, irqnr);
+                       handle_IRQ(irqnr, regs);
+                       continue;
+               }

drivers/irqchip/irq-gic-v3.c line 272

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+               if (irqnr < 16) {
+                       gic_write_eoir(irqnr);
+#ifdef CONFIG_SMP
+                       handle_IPI(irqnr, regs);
+#else
+                       WARN_ONCE(true, "Unexpected SGI received!\n");
+#endif
+                       continue;
+               }
+       } while (irqnr != 0x3ff);

Any reason to adjust this control-flow from the existing GIC driver?

Quoted 31 lines
drivers/irqchip/irq-gic-v3.c new file +31 -0
+}
+
+static void __init gic_dist_init(void)
+{
+       unsigned int i;
+       u64 affinity;
+       int gic_irqs = gic_data.irq_nr;
+       void __iomem *base = gic_data.dist_base;
+
+       /* Disable the distributor */
+       writel_relaxed(0, base + GICD_CTLR);
+
+       /*
+        * Set all global interrupts to be level triggered, active low.
+        */
+       for (i = 32; i < gic_data.irq_nr; i += 16)
+               writel_relaxed(0, base + GICD_ICFGR + i / 4);
+
+       /*
+        * Set priority on all global interrupts.
+        */
+       for (i = 32; i < gic_irqs; i += 4)
+               writel_relaxed(0xa0a0a0a0, base + GICD_IPRIORITYR + i);
+
+       /*
+        * Disable all interrupts.  Leave the PPI and SGIs alone
+        * as these enables are banked registers.
+        */
+       for (i = 32; i < gic_irqs; i += 32)
+               writel_relaxed(0xffffffff, base + GICD_ICENABLER + i / 8);
+

drivers/irqchip/irq-gic-v3.c line 313

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       gic_dist_wait_for_rwp();
+
+       writel_relaxed(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+                      base + GICD_CTLR);
+
+       /*
+        * Set all global interrupts to the boot CPU only. ARE must be
+        * enabled.
+        */
+       affinity = gic_mpidr_to_affinity(read_cpuid_mpidr());

You can use the logical map here.

drivers/irqchip/irq-gic-v3.c line 316

Quoted 3 lines
drivers/irqchip/irq-gic-v3.c new file +3 -0
+       for (i = 32; i < gic_data.irq_nr; i++)
+               writeq_relaxed(affinity, base + GICD_IROUTER + i * 8);
+}

Again, this looks very similar to the existing GIC driver.

drivers/irqchip/irq-gic-v3.c line 326

Quoted 9 lines
drivers/irqchip/irq-gic-v3.c new file +9 -0
+static int __init gic_populate_rdist(void)
+{
+       u64 mpidr = cpu_logical_map(smp_processor_id());
+       u64 typer;
+       u64 aff;
+       int i;
+
+       aff  = mpidr & ((1 << 24) - 1);
+       aff |= (mpidr >> 8) & (0xffUL << 24);

We should have helpers for this in cputype.h.

Also, can we absolutely, 100%, "put-your-mother's-house-on-it" guarantee that the ITS, GICRn and the CPUs all agree on an identification scheme, as described in the mpidr? My reading of the spec doesn't fill me with confidence, and retrofitting a scheme in DT will end up with the same mess that we have for CPU topology.

drivers/irqchip/irq-gic-v3.c line 339

Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+
+       for (i = 0; i < gic_data.redist_regions; i++) {
+               void __iomem *ptr = gic_data.redist_base[i];
+               u32 reg;
+
+               reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
+               if (reg != 0x30 && reg != 0x40) { /* We're in trouble... */
+                       pr_warn("No redistributor present @%p\n", ptr);
+                       break;
+               }
+
+               do {
+                       typer = readq_relaxed(ptr + GICR_TYPER);

Not sure if you care, but readq is a show-stopper if you want to build this with arch/arm/.

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+                       if ((typer >> 32) == aff) {
+                               gic_data_rdist_rd_base() = ptr;
+                               pr_info("CPU%d: found redistributor %llx @%p\n",
+                                       smp_processor_id(),
+                                       (unsigned long long) mpidr, ptr);

drivers/irqchip/irq-gic-v3.c line 354

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+                               return 0;
+                       }
+
+                       if (gic_data.redist_stride) {
+                               ptr += gic_data.redist_stride;
+                       } else {
+                               ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+                               if (typer & GICR_TYPER_VLPIS)
+                                       ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+                       }

VLPIS is RES0 in GICv3.

drivers/irqchip/irq-gic-v3.c line 360

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+               } while (!(typer & GICR_TYPER_LAST));
+       }
+
+       /* We couldn't even deal with ourselves... */
+       WARN(true, "CPU%d: mpidr %lx has no re-distributor!\n",
+            smp_processor_id(), (unsigned long)mpidr);

Inconsistency with the type of mpidr (you use unsigned long long above).

Quoted 18 lines
drivers/irqchip/irq-gic-v3.c new file +18 -0
+       return -ENODEV;
+}
+
+static void __init gic_cpu_init(void)
+{
+       void __iomem *rbase;
+       int i;
+
+       /* Register ourselves with the rest of the world */
+       if (gic_populate_rdist())
+               return;
+
+       gic_enable_redist();
+
+       rbase = gic_data_rdist_sgi_base();
+
+       /*
+        * Set priority on PPI and SGI interrupts

drivers/irqchip/irq-gic-v3.c line 388

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+        */
+       for (i = 0; i < 32; i += 4)
+               writel_relaxed(0xa0a0a0a0, rbase + GICR_IPRIORITYR0 + i * 4 / 4);
+
+       /*
+        * Disable all PPI interrupts, ensure all SGI interrupts are
+        * enabled.
+        */
+       writel_relaxed(0xffff0000, rbase + GICR_ICENABLER0);
+       writel_relaxed(0x0000ffff, rbase + GICR_ISENABLER0);

Whilst there are some clear differences between this and the existing driver, you could still factor parts of this out into a common file.

Quoted 18 lines
drivers/irqchip/irq-gic-v3.c new file +18 -0
+       gic_redist_wait_for_rwp();
+
+       /* Enable system registers */
+       gic_enable_sre();
+
+       /* Set priority mask register */
+       gic_write_pmr(DEFAULT_PMR_VALUE);
+
+       /* EOI drops priority too (mode 0) */
+       gic_write_ctlr(GICC_CTLR_EL1_EOImode_drop_dir);
+
+       /* ... and let's hit the road... */
+       gic_write_grpen1(1);
+}
+
+#ifdef CONFIG_SMP
+static int __init gic_secondary_init(struct notifier_block *nfb,
+                                    unsigned long action, void *hcpu)

drivers/irqchip/irq-gic-v3.c line 417

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+{
+       if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+               gic_cpu_init();
+       return NOTIFY_OK;
+}
+
+/*
+ * Notifier for enabling the GIC CPU interface. Set an arbitrarily high
+ * priority because the GIC needs to be up before the ARM generic timers.
+ */

Why must the GIC be up before the timers? Obviously, the timers won't be able to generate interrupts without the GIC initialised, but the comment implies that something worse will happen.

Quoted 9 lines
drivers/irqchip/irq-gic-v3.c new file +9 -0
+static struct notifier_block __initdata gic_cpu_notifier = {
+       .notifier_call = gic_secondary_init,
+       .priority = 100,
+};
+
+static u16 gic_compute_target_list(int *base_cpu, const struct cpumask *mask,
+                                  u64 cluster_id)
+{
+       int cpu = *base_cpu;

drivers/irqchip/irq-gic-v3.c line 436

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       u64 mpidr = cpu_logical_map(cpu);
+       u16 tlist = 0;
+
+       while (cpu < nr_cpu_ids) {
+               /*
+                * If we ever get a cluster of more than 16 CPUs, just
+                * scream and skip that CPU.
+                */
+               if (WARN_ON((mpidr & 0xff) >= 16))
+                       goto out;

I'm not comfortable with this. The mpidr in ARMv8 is just a unique identifier, so relying on aff0 to provide some sort of upper bound on the number of CPUs in a cluster feels extremely fragile.

I think you'll need to deal with SGIs to target lists that you can't represent with a single sgi1r value (and instead batch them up and have lots of fun dealing with the result).

Quoted 12 lines
drivers/irqchip/irq-gic-v3.c new file +12 -0
+
+               tlist |= 1 << (mpidr & 0xf);
+
+               cpu = cpumask_next(cpu, mask);
+               mpidr = cpu_logical_map(cpu);
+
+               if (cluster_id != (mpidr & ~0xffUL)) {
+                       cpu--;
+                       goto out;
+               }
+       }
+out:

drivers/irqchip/irq-gic-v3.c line 458

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       *base_cpu = cpu;
+       return tlist;
+}
+
+static void gic_send_sgi(u64 cluster_id, u16 tlist, unsigned int irq)
+{
+       u64 val;
+
+       val  = (cluster_id & 0xff00ff0000UL) << 16; /* Aff3 + Aff2 */
+       val |= (cluster_id & 0xff00) << 8;          /* Aff1 */

Use the helpers.

Quoted 9 lines
drivers/irqchip/irq-gic-v3.c new file +9 -0
+       val |= irq << 24;
+       val |= tlist;
+
+       pr_debug("CPU%d: ICC_SGI1R_EL1 %llx\n", smp_processor_id(), val);
+       gic_write_sgi1r(val);
+}
+
+static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
+{

drivers/irqchip/irq-gic-v3.c line 477

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       int cpu;
+
+       if (WARN_ON(irq >= 16))
+               return;
+
+       /*
+        * Ensure that stores to Normal memory are visible to the
+        * other CPUs before issuing the IPI.
+        */
+       dsb();

dmb(ishst), obviously (!) ;)

Quoted 20 lines
drivers/irqchip/irq-gic-v3.c new file +20 -0
+
+       for_each_cpu_mask(cpu, *mask) {
+               u64 cluster_id = cpu_logical_map(cpu) & ~0xffUL;
+               u16 tlist;
+
+               tlist = gic_compute_target_list(&cpu, mask, cluster_id);
+               gic_send_sgi(cluster_id, tlist, irq);
+       }
+}
+
+static void gic_smp_init(void)
+{
+       set_smp_cross_call(gic_raise_softirq);
+       register_cpu_notifier(&gic_cpu_notifier);
+}
+
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+                           bool force)
+{
+       unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);

drivers/irqchip/irq-gic-v3.c line 507

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       void __iomem *reg;
+       u64 val;
+
+       if (gic_irq(d) < 32)
+               return -EINVAL;
+
+       reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+       val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+       writeq_relaxed(val, reg);

How do you ensure that the interrupt has actually moved?

Quoted 23 lines
drivers/irqchip/irq-gic-v3.c new file +23 -0
+       return IRQ_SET_MASK_OK;
+}
+#else
+#define gic_set_affinity       NULL
+#define gic_smp_init()         do { } while(0)
+#endif
+
+static struct irq_chip gic_chip = {
+       .name                   = "GICv3",
+       .irq_mask               = gic_mask_irq,
+       .irq_unmask             = gic_unmask_irq,
+       .irq_eoi                = gic_eoi_irq,
+       .irq_set_type           = gic_set_type,
+       .irq_retrigger          = gic_retrigger,
+       .irq_set_affinity       = gic_set_affinity,
+};
+
+static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
+                               irq_hw_number_t hw)
+{
+       if (hw < 32) {
+               irq_set_percpu_devid(irq);
+               irq_set_chip_and_handler(irq, &gic_chip,

drivers/irqchip/irq-gic-v3.c line 541

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+                                        handle_percpu_devid_irq);
+               set_irq_flags(irq, IRQF_VALID | IRQF_NOAUTOEN);
+       } else {
+               irq_set_chip_and_handler(irq, &gic_chip,
+                                        handle_fasteoi_irq);
+               set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
+       }
+       irq_set_chip_data(irq, d->host_data);
+       return 0;
+}

More copied GICv2 code...

Quoted 14 lines
drivers/irqchip/irq-gic-v3.c new file +14 -0
+static int gic_irq_domain_xlate(struct irq_domain *d,
+                               struct device_node *controller,
+                               const u32 *intspec, unsigned int intsize,
+                               unsigned long *out_hwirq, unsigned int *out_type)
+{
+       if (d->of_node != controller)
+               return -EINVAL;
+       if (intsize < 3)
+               return -EINVAL;
+
+       switch(intspec[0]) {
+       case 0:                 /* SPI */
+               *out_hwirq = intspec[1] + 32;
+               break;

drivers/irqchip/irq-gic-v3.c line 566

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       case 1:                 /* PPI */
+               *out_hwirq = intspec[1] + 16;
+               break;
+       default:
+               return -EINVAL;
+       }
+
+       *out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
+       return 0;
+}

... and some slightly mutated but functionally identical GICv2 code ;)

Quoted 29 lines
drivers/irqchip/irq-gic-v3.c new file +29 -0
+static const struct irq_domain_ops gic_irq_domain_ops = {
+       .map = gic_irq_domain_map,
+       .xlate = gic_irq_domain_xlate,
+};
+
+static int __init gic_of_init(struct device_node *node, struct device_node *parent)
+{
+       void __iomem *dist_base;
+       void __iomem **redist_base;
+       u64 redist_stride;
+       u32 redist_regions;
+       u32 reg;
+       int gic_irqs;
+       int err;
+       int i;
+
+       dist_base = of_iomap(node, 0);
+       if (!dist_base) {
+               pr_warn("%s: unable to map gic dist registers\n",
+                       node->full_name);
+               return -ENXIO;
+       }
+
+       reg = readl_relaxed(dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
+       if (reg != 0x30 && reg != 0x40) {
+               pr_warn("%s: no distributor detected, giving up\n",
+                       node->full_name);
+               err = -ENODEV;
+               goto out_unmap_dist;

drivers/irqchip/irq-gic-v3.c line 606

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+       }
+
+       if (of_property_read_u32(node, "#redistributor-regions", &redist_regions))
+               redist_regions = 1;
+
+       redist_base = kzalloc(sizeof(*redist_base) * redist_regions, GFP_KERNEL);
+       if (!redist_base) {
+               err = -ENOMEM;
+               goto out_unmap_dist;
+       }

Can you use the devm_* managed resource helpers in irqchip init functions? I see a couple of drivers doing it, but it's not clear to me whether things are actually freed on failure. Anyway, if you can, then it would clean this up a bit.

Quoted 36 lines
drivers/irqchip/irq-gic-v3.c new file +36 -0
+       for (i = 0; i < redist_regions; i++) {
+               redist_base[i] = of_iomap(node, 1 + i);
+               if (!redist_base[i]) {
+                       pr_warn("%s: couldn't map region %d\n",
+                               node->full_name, i);
+                       err = -ENODEV;
+                       goto out_unmap_rdist;
+               }
+       }
+
+       if (of_property_read_u64(node, "redistributor-stride", &redist_stride))
+               redist_stride = 0;
+
+       gic_data.dist_base = dist_base;
+       gic_data.redist_base = redist_base;
+       gic_data.redist_regions = redist_regions;
+       gic_data.redist_stride = redist_stride;
+
+       /*
+        * Find out how many interrupts are supported.
+        * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)
+        */
+       gic_irqs = readl_relaxed(gic_data.dist_base + GICD_TYPER) & 0x1f;
+       gic_irqs = (gic_irqs + 1) * 32;
+       if (gic_irqs > 1020)
+               gic_irqs = 1020;
+       gic_data.irq_nr = gic_irqs;
+
+       gic_data.domain = irq_domain_add_linear(node, gic_irqs - 16,
+                                               &gic_irq_domain_ops, &gic_data);
+       gic_data.rdist = alloc_percpu(typeof(*gic_data.rdist));
+
+       if (WARN_ON(!gic_data.domain) || WARN_ON(!gic_data.rdist)) {
+               err = -ENOMEM;
+               goto out_free;
+       }

drivers/irqchip/irq-gic-v3.c line 653

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+       set_handle_irq(gic_handle_irq);
+
+       gic_smp_init();
+       gic_dist_init();
+       gic_cpu_init();
+
+       return 0;
+
+out_free:

remove the irq_domain here?

Quoted 182 lines
drivers/irqchip/irq-gic-v3.c new file +12 -0
+       free_percpu(gic_data.rdist);
+out_unmap_rdist:
+       for (i = 0; i < redist_regions; i++)
+               if (redist_base[i])
+                       iounmap(redist_base[i]);
+       kfree(redist_base);
+out_unmap_dist:
+       iounmap(dist_base);
+       return err;
+}
+
+IRQCHIP_DECLARE(gic_v3, "arm,gic-v3", gic_of_init);
include/linux/irqchip/arm-gic-v3.h new file +164 -0
new file mode 100644
index 0000000..e4d060a
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __LINUX_IRQCHIP_ARM_GIC_V3_H
+#define __LINUX_IRQCHIP_ARM_GIC_V3_H
+
+/*
+ * Distributor registers. We assume we're running non-secure, with ARE
+ * being set. Secure-only and non-ARE registers are not described.
+ */
+#define GICD_CTLR                      0x0000
+#define GICD_TYPER                     0x0004
+#define GICD_IIDR                      0x0008
+#define GICD_STATUSR                   0x0010
+#define GICD_SETSPI_NSR                        0x0040
+#define GICD_CLRSPI_NSR                        0x0048
+#define GICD_SETSPI_SR                 0x0050
+#define GICD_CLRSPI_SR                 0x0058
+#define GICD_SEIR                      0x0068
+#define GICD_ISENABLER                 0x0100
+#define GICD_ICENABLER                 0x0180
+#define GICD_ISPENDR                   0x0200
+#define GICD_ICPENDR                   0x0280
+#define GICD_ISACTIVER                 0x0300
+#define GICD_ICACTIVER                 0x0380
+#define GICD_IPRIORITYR                        0x0400
+#define GICD_ICFGR                     0x0C00
+#define GICD_IROUTER                   0x6000
+#define GICD_PIDR2                     0xFFE8
+
+#define GICD_CTLR_RWP                  (1U << 31)
+#define GICD_CTLR_ARE_NS               (1U << 4)
+#define GICD_CTLR_ENABLE_G1A           (1U << 1)
+#define GICD_CTLR_ENABLE_G1            (1U << 0)
+
+#define GICD_IROUTER_SPI_MODE_ONE      (0U << 31)
+#define GICD_IROUTER_SPI_MODE_ANY      (1U << 31)
+
+#define GIC_PIDR2_ARCH_MASK            0xf0
+
+/*
+ * Re-Distributor registers, offsets from RD_base
+ */
+#define GICR_CTLR                      GICD_CTLR
+#define GICR_IIDR                      0x0004
+#define GICR_TYPER                     0x0008
+#define GICR_STATUSR                   GICD_STATUSR
+#define GICR_WAKER                     0x0014
+#define GICR_SETLPIR                   0x0040
+#define GICR_CLRLPIR                   0x0048
+#define GICR_SEIR                      GICD_SEIR
+#define GICR_PROPBASER                 0x0070
+#define GICR_PENDBASER                 0x0078
+#define GICR_INVLPIR                   0x00A0
+#define GICR_INVALLR                   0x00B0
+#define GICR_SYNCR                     0x00C0
+#define GICR_MOVLPIR                   0x0100
+#define GICR_MOVALLR                   0x0110
+#define GICR_PIDR2                     GICD_PIDR2
+
+#define GICR_WAKER_ProcessorSleep      (1U << 1)
+#define GICR_WAKER_ChildrenAsleep      (1U << 2)
+
+/*
+ * Re-Distributor registers, offsets from SGI_base
+ */
+#define GICR_ISENABLER0                        GICD_ISENABLER
+#define GICR_ICENABLER0                        GICD_ICENABLER
+#define GICR_ISPENDR0                  GICD_ISPENDR
+#define GICR_ICPENDR0                  GICD_ICPENDR
+#define GICR_ISACTIVER0                        GICD_ISACTIVER
+#define GICR_ICACTIVER0                        GICD_ICACTIVER
+#define GICR_IPRIORITYR0               GICD_IPRIORITYR
+#define GICR_ICFGR0                    GICD_ICFGR
+
+#define GICR_TYPER_VLPIS               (1U << 1)
+#define GICR_TYPER_LAST                        (1U << 4)
+
+/*
+ * CPU interface registers
+ */
+#define GICC_CTLR_EL1_EOImode_drop_dir (0U << 1)
+#define GICC_CTLR_EL1_EOImode_drop     (1U << 1)
+#define GICC_SRE_EL1_SRE               (1U << 0)
+
+/*
+ * Hypervisor interface registers (SRE only)
+ */
+#define GICH_LR_VIRTUAL_ID_MASK                ((1UL << 32) - 1)
+
+#define GICH_LR_EOI                    (1UL << 41)
+#define GICH_LR_GROUP                  (1UL << 60)
+#define GICH_LR_STATE                  (3UL << 62)
+#define GICH_LR_PENDING_BIT            (1UL << 62)
+#define GICH_LR_ACTIVE_BIT             (1UL << 63)
+
+#define GICH_MISR_EOI                  (1 << 0)
+#define GICH_MISR_U                    (1 << 1)
+
+#define GICH_HCR_EN                    (1 << 0)
+#define GICH_HCR_UIE                   (1 << 1)
+
+#define GICH_VMCR_CTLR_SHIFT           0
+#define GICH_VMCR_CTLR_MASK            (0x21f << GICH_VMCR_CTLR_SHIFT)
+#define GICH_VMCR_BPR1_SHIFT           18
+#define GICH_VMCR_BPR1_MASK            (7 << GICH_VMCR_BPR1_SHIFT)
+#define GICH_VMCR_BPR0_SHIFT           21
+#define GICH_VMCR_BPR0_MASK            (7 << GICH_VMCR_BPR0_SHIFT)
+#define GICH_VMCR_PMR_SHIFT            24
+#define GICH_VMCR_PMR_MASK             (0xffUL << GICH_VMCR_PMR_SHIFT)
+
+#define ICC_EOIR1_EL1                  S3_0_C12_C12_1
+#define ICC_IAR1_EL1                   S3_0_C12_C12_0
+#define ICC_SGI1R_EL1                  S3_0_C12_C11_5
+#define ICC_PMR_EL1                    S3_0_C4_C6_0
+#define ICC_CTLR_EL1                   S3_0_C12_C12_4
+#define ICC_SRE_EL1                    S3_0_C12_C12_5
+#define ICC_GRPEN1_EL1                 S3_0_C12_C12_7
+
+#define ICC_SRE_EL2                    S3_4_C12_C9_5
+
+#define ICH_VSEIR_EL2                  S3_4_C12_C9_4
+#define ICH_HCR_EL2                    S3_4_C12_C11_0
+#define ICH_VTR_EL2                    S3_4_C12_C11_1
+#define ICH_MISR_EL2                   S3_4_C12_C11_2
+#define ICH_EISR_EL2                   S3_4_C12_C11_3
+#define ICH_ELSR_EL2                   S3_4_C12_C11_5
+#define ICH_VMCR_EL2                   S3_4_C12_C11_7
+
+#define __LR0_EL2(x)                   S3_4_C12_C12_ ## x
+#define __LR8_EL2(x)                   S3_4_C12_C13_ ## x
+
+#define ICH_LR0_EL2                    __LR0_EL2(0)
+#define ICH_LR1_EL2                    __LR0_EL2(1)
+#define ICH_LR2_EL2                    __LR0_EL2(2)
+#define ICH_LR3_EL2                    __LR0_EL2(3)
+#define ICH_LR4_EL2                    __LR0_EL2(4)
+#define ICH_LR5_EL2                    __LR0_EL2(5)
+#define ICH_LR6_EL2                    __LR0_EL2(6)
+#define ICH_LR7_EL2                    __LR0_EL2(7)
+#define ICH_LR8_EL2                    __LR8_EL2(0)
+#define ICH_LR9_EL2                    __LR8_EL2(1)
+#define ICH_LR10_EL2                   __LR8_EL2(2)
+#define ICH_LR11_EL2                   __LR8_EL2(3)
+#define ICH_LR12_EL2                   __LR8_EL2(4)
+#define ICH_LR13_EL2                   __LR8_EL2(5)
+#define ICH_LR14_EL2                   __LR8_EL2(6)
+#define ICH_LR15_EL2                   __LR8_EL2(7)
+
+#define __AP0Rx_EL2(x)                 S3_4_C12_C8_ ## x

include/linux/irqchip/arm-gic-v3.h line 174

Quoted 10 lines
include/linux/irqchip/arm-gic-v3.h new file +10 -0
+#define ICH_AP0R0_EL2                  __AP0Rx_EL2(0)
+#define ICH_AP0R1_EL2                  __AP0Rx_EL2(1)
+#define ICH_AP0R2_EL2                  __AP0Rx_EL2(2)
+#define ICH_AP0R3_EL2                  __AP0Rx_EL2(3)
+
+#define __AP1Rx_EL2(x)                 S3_4_C12_C9_ ## x
+#define ICH_AP1R0_EL2                  __AP1Rx_EL2(0)
+#define ICH_AP1R1_EL2                  __AP1Rx_EL2(1)
+#define ICH_AP1R2_EL2                  __AP1Rx_EL2(2)
+#define ICH_AP1R3_EL2                  __AP1Rx_EL2(3)

I'm going to rely on Christoffer having looked at all these magic numbers (he at least had ice cream!).

include/linux/irqchip/arm-gic-v3.h line 186

Quoted 11 lines
include/linux/irqchip/arm-gic-v3.h new file +11 -0
+#ifndef __ASSEMBLY__
+
+#include <linux/stringify.h>
+
+#define reg(x)         __stringify(x)
+
+static inline void gic_write_eoir(u64 irq)
+{
+       asm volatile("msr " reg(ICC_EOIR1_EL1) ", %0" : : "r" (irq));
+       isb();
+}

Why is this in the header file?

Will

#9 lore raw

PATCH 1/18 arm64: initial support for GICv3

Quoted 19 lines · Will Deacon

Hi Marc,

Quoted 16 lines · Marc Zyngier
arch/arm64/kernel/head.S +1 -0
index 0b281ff..522649b 100644
@@ -22,6 +22,7 @@
 
 #include <linux/linkage.h>
 #include <linux/init.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -183,6 +184,19 @@ CPU_LE(    bic     x0, x0, #(3 << 24)      )       // Clear the EE and E0E bits for EL1
        msr     cnthctl_el2, x0
        msr     cntvoff_el2, xzr                // Clear virtual offset
 
Quoted 13 lines

arch/arm64/kernel/head.S CPU_LE( bic x0, x0, #(3 << 24) ) // Clear the EE and E0E bits for EL1 · line 194

Quoted 8 lines
arch/arm64/kernel/head.S +8 -0
+       /* GICv3 system register access */
+       mrs     x0, id_aa64pfr0_el1
+       ubfx    x0, x0, #24, #4
+       cmp     x0, #1
+       b.ne    3f
+
+       mrs     x0, ICC_SRE_EL2                 // Set ICC_SRE_EL2.SRE==1
+       orr     x0, x0, #1

Don't you need to set the Enable bit (bit 3) too? Also, does setting ICC_SRE_EL2.SRE force KVM to use the ICH_* registers? If so, that seems like a pretty big decision to take this early on (there's not even an #ifdef CONFIG_ARM_GICV3).

The test on id_aa64pfr0_el1 is an absolute guarantee that we have the GICv3 system registers.

As for the Enable bit, this is handle by the magic trapping we have below, but I decided to get rid of that code (see the previous discussion with Christoffer).

Quoted 25 lines
Quoted 25 lines
arch/arm64/kernel/head.S +5 -0
+       msr     ICC_SRE_EL2, x0
+       isb                                     // Make sure SRE is now 1
+       msr     ICH_HCR_EL2, xzr                // Reset ICC_HCR_EL2 to defaults
+
+3:
        /* Populate ID registers. */
        mrs     x0, midr_el1
        mrs     x1, mpidr_el1
arch/arm64/kernel/hyp-stub.S +1 -0
index 0959611..c850d1e 100644
@@ -19,6 +19,7 @@
 
 #include <linux/init.h>
 #include <linux/linkage.h>
+#include <linux/irqchip/arm-gic-v3.h>
 
 #include <asm/assembler.h>
 #include <asm/ptrace.h>
@@ -51,16 +52,52 @@ ENDPROC(__hyp_stub_vectors)
 
        .align 11
 
Quoted 15 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 65

Quoted 11 lines
arch/arm64/kernel/hyp-stub.S +10 -0
+#define sysreg_to_iss(op0, op1, crn, crm, op2) \
+       (((op0 & 3) << 20) | ((op2 & 7 ) << 17) | \
+        ((op1 & 7 ) << 14) | ((crn & 15) << 10) | \
+        ((crm & 15) << 1))
+
 el1_sync:
+       // Hack alert: we don't have a stack, and we can't use the one
+       // from EL1 because our MMU is off. Thankfully, we have
+       // tpidr_el2, which is under our complete control, and
+       // tpidr_el0, which can be used as long as we haven't reached
+       // userspace. Oh hum...

Yuck. Why can't we create a minimal stack when we initialise EL2 when booting the host? Even simpler, just use some .quads in here (you're already using a literal pool).

I really liked the "funkiness" here, but that code is now gone. RIP! ;-)

Quoted 23 lines
Quoted 23 lines
arch/arm64/kernel/hyp-stub.S +14 -2
+       msr     tpidr_el2, x1                   // Preserve x1
+
        mrs     x1, esr_el2
        lsr     x1, x1, #26
        cmp     x1, #0x16
-       b.ne    2f                              // Not an HVC trap
+       b.ne    3f                              // Not an HVC trap
+
+       // Handle getting/setting vbar_el2
        cbz     x0, 1f
        msr     vbar_el2, x0                    // Set vbar_el2
        b       2f
 1:     mrs     x0, vbar_el2                    // Return vbar_el2
-2:     eret
+2:     mrs     x1, tpidr_el2                   // Restore x1
+       eret
+
+3:     cmp     x1, #0x18                       // MSR/MRS?
+       b.ne    2b
+
+       // Handle trapped sys reg access
+       msr     tpidr_el0, x0                   // Preserve x0
+       mrs     x1, esr_el2
Quoted 8 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 89

Quoted 3 lines
arch/arm64/kernel/hyp-stub.S +3 -0
+       ldr     x0, =sysreg_to_iss(3, 7, 15, 15, 7)
+       and     x1, x1, x0
+       ldr     x0, =sysreg_to_iss(3, 0, 12, 12, 5)

I think sysreg_to_iss actually makes this *less* readable! Why not simply #define the mask to extract OPx, CRx then compare with another #define for ICC_SRE_EL1? If we can represent those as immediates, you can avoid the literal loads too.

Gone.

Quoted 10 lines

arch/arm64/kernel/hyp-stub.S ENDPROC(__hyp_stub_vectors) · line 96

Quoted 7 lines
arch/arm64/kernel/hyp-stub.S +7 -0
+       eor     x0, x0, x1                      // Check for ICC_SRE_EL1
+       cbnz    x0, 4f                          // Something bad happened
+
+       // Set ICC_SRE_EL1 access enable
+       mrs     x1, ICC_SRE_EL2
+       orr     x1, x1, #(1 << 3)
+       msr     ICC_SRE_EL2, x1

Ah, ok, you do this part lazily. Does that make sense for the host? (i.e. is it possible for KVM to use SRE at EL2 but not the host at EL1?).

You could, but I'm not sure it is worth supporting. In the interest of simplicity, I got rid of all that crap.

Quoted 55 lines
Quoted 55 lines
arch/arm64/kernel/hyp-stub.S +4 -0
+
+4:     mrs     x0, tpidr_el0                   // Restore x0
+       mrs     x1, tpidr_el2                   // Restore x1
+       eret
 ENDPROC(el1_sync)
 
 .macro invalid_vector  label
drivers/irqchip/irq-gic-v3.c new file +42 -0
new file mode 100644
index 0000000..487e061
@@ -0,0 +1,665 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+#include <linux/irqchip/arm-gic-v3.h>
+
+#include <asm/cputype.h>
+#include <asm/exception.h>
+#include <asm/smp_plat.h>
+
+#include "irqchip.h"
+
+struct gic_chip_data {
+       void __iomem            *dist_base;
+       void __iomem            **redist_base;
+       void __percpu __iomem   **rdist;
+       struct irq_domain       *domain;
+       u64                     redist_stride;
+       u32                     redist_regions;
+       unsigned int            irq_nr;
+};
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 46

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+
+static struct gic_chip_data gic_data __read_mostly;
+
+#define gic_data_rdist_rd_base()       (*__this_cpu_ptr(gic_data.rdist))

You should avoid __this_cpu_ptr (it's being removed afaik). Take a look at the patches from Christoph Lameter, e.g. 170d800af83f ("block: Replace __get_cpu_var uses").

Ah, interesting. I'll repaint this.

Quoted 29 lines
Quoted 29 lines
drivers/irqchip/irq-gic-v3.c new file +29 -0
+#define gic_data_rdist_sgi_base()      (gic_data_rdist_rd_base() + SZ_64K)
+
+static DEFINE_RAW_SPINLOCK(dist_lock);
+
+#define DEFAULT_PMR_VALUE      0xf0
+
+static inline void __iomem *gic_dist_base(struct irq_data *d)
+{
+       if (d->hwirq < 32)      /* SGI+PPI -> SGI_base for this CPU */
+               return gic_data_rdist_sgi_base();
+
+       if (d->hwirq <= 1023)   /* SPI -> dist_base */
+               return gic_data.dist_base;
+
+       if (d->hwirq >= 8192)
+               BUG();          /* LPI Detected!!! */
+
+       return NULL;
+}
+
+static inline unsigned int gic_irq(struct irq_data *d)
+{
+       return d->hwirq;
+}
+
+static void gic_do_wait_for_rwp(void __iomem *base)
+{
+       u32 val;
+
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 79

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+       do {
+               val = readl_relaxed(base + GICD_CTLR);
+               cpu_relax();
+       } while (val & GICD_CTLR_RWP);

You should add a timeout in here to save us in the face of broken hardware (especially as you hold the distributor lock). See, for example, arm_smmu_tlb_sync(...) in the arm-smmu driver.

Yup.

Quoted 2 lines

Thinking about it, is this the only point of the dist_lock (to ensure you wait only for your request)? Do we actually need locking for that?

You need to ensure that the distributor has propagated the request. I'm not sure what the effect would be if we had several CPUs writing to the distributor at once... I'll investigate.

Quoted 34 lines
Quoted 7 lines
drivers/irqchip/irq-gic-v3.c new file +7 -0
+}
+
+/* Wait for completion of a distributor change */
+static void gic_dist_wait_for_rwp(void)
+{
+       gic_do_wait_for_rwp(gic_data.dist_base);
+}

drivers/irqchip/irq-gic-v3.c line 96

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+
+/* Wait for completion of a redistributor change */
+static void gic_redist_wait_for_rwp(void)
+{
+       gic_do_wait_for_rwp(gic_data_rdist_rd_base());
+}
+
+static void gic_wait_for_rwp(int irq)
+{
+       if (irq < 32)

Maybe we should just bite the bullet and have helpers for gic_irq_is_sgi(irq) etc. so we can avoid these magic numbers all over the place.

drivers/irqchip/irq-gic-v3.c line 100

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+               gic_redist_wait_for_rwp();
+       else
+               gic_dist_wait_for_rwp();
+}

Saying that, I question the value of mixing this into a single function. gic_redist_wait_for_rwp absolutely *must* be called with preemption disabled and on the appropriate CPU for the IRQ in question, whilst the latter can be called anywhere. So even though you can call this single function from a bunch of places, you're really relying on the irq parameter being correct for the context of the caller.

Quoted 2 lines

I reckon it would actually be clearer to call the relevant wait function explicitly in the caller. Same for gic_dist_base.

Fair enough. I'll try to rework that part.

Quoted 9 lines

drivers/irqchip/irq-gic-v3.c line 107

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+/* Low level accessors */
+static u64 gic_read_iar(void)
+{
+       u64 irqstat;
+
+       asm volatile("mrs %0, " reg(ICC_IAR1_EL1) : "=r" (irqstat));

Hmm, reg() is a terrible name for a macro :) Can't you just use stringify directly?

Too long? ;-)

Quoted 30 lines
Quoted 16 lines
drivers/irqchip/irq-gic-v3.c new file +16 -0
+       return irqstat;
+}
+
+static void gic_write_pmr(u64 val)
+{
+       asm volatile("msr " reg(ICC_PMR_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_write_ctlr(u64 val)
+{
+       asm volatile("msr " reg(ICC_CTLR_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_write_grpen1(u64 val)

drivers/irqchip/irq-gic-v3.c line 133

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+{
+       asm volatile("msr " reg(ICC_GRPEN1_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_write_sgi1r(u64 val)
+{
+       asm volatile("msr " reg(ICC_SGI1R_EL1) ", %0" : : "r" (val));
+       isb();
+}

I'm not convinced you *always* need these isbs. For example, the gic_handle_irq loop probably just needs one at the end, before returning.

Quoted 4 lines

In fact, looking at the spec, ICC_PMR_EL1 is actually self-synchronising so you don't need anything there. For SGI1R at least, you need a dsb to ensure write observability by the GIC (it's not clear to me whether you also need to the isb).

The dsb is actually in gic_send_sgi. We may end-up doing several writes to ICC_SGI1R_EL1 if the targets are on different Aff{3,2,1} levels, so it made sense to put it there.

Quoted 21 lines
Quoted 21 lines
drivers/irqchip/irq-gic-v3.c new file +21 -0
+static void gic_enable_sre(void)
+{
+       u64 val;
+
+       /* This access is likely to trap into EL2 */
+       asm volatile("mrs %0, " reg(ICC_SRE_EL1) : "=r" (val));
+       val |= GICC_SRE_EL1_SRE;
+       asm volatile("msr " reg(ICC_SRE_EL1) ", %0" : : "r" (val));
+       isb();
+}
+
+static void gic_enable_redist(void)
+{
+       void __iomem *rbase;
+       u32 val;
+
+       rbase = gic_data_rdist_rd_base();
+
+       /* Wake up this CPU redistributor */
+       val = readl_relaxed(rbase + GICR_WAKER);
+       val &= ~GICR_WAKER_ProcessorSleep;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 161

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+       writel_relaxed(val, rbase + GICR_WAKER);
+
+       do {
+               val = readl_relaxed(rbase + GICR_WAKER);
+               cpu_relax();
+       } while (val & GICR_WAKER_ChildrenAsleep);

Again, you want a timeout here.

Agreed.

Quoted 23 lines
Quoted 23 lines
drivers/irqchip/irq-gic-v3.c new file +23 -0
+}
+
+/*
+ * Routines to acknowledge, disable and enable interrupts
+ */
+static void gic_mask_irq(struct irq_data *d)
+{
+       u32 mask = 1 << (gic_irq(d) % 32);
+
+       raw_spin_lock(&dist_lock);
+       writel_relaxed(mask, gic_dist_base(d) + GICD_ICENABLER + (gic_irq(d) / 32) * 4);
+       gic_wait_for_rwp(gic_irq(d));
+       raw_spin_unlock(&dist_lock);
+}
+
+static void gic_unmask_irq(struct irq_data *d)
+{
+       u32 mask = 1 << (gic_irq(d) % 32);
+
+       raw_spin_lock(&dist_lock);
+       writel_relaxed(mask, gic_dist_base(d) + GICD_ISENABLER + (gic_irq(d) / 32) * 4);
+       gic_wait_for_rwp(gic_irq(d));
+       raw_spin_unlock(&dist_lock);
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 190

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+}
+
+static void gic_eoi_irq(struct irq_data *d)
+{
+       gic_write_eoir(gic_irq(d));
+}

Can you inline the code here (this is the only user) and move the isb out?

Yup.

Quoted 42 lines
Quoted 42 lines
drivers/irqchip/irq-gic-v3.c new file +42 -0
+static int gic_set_type(struct irq_data *d, unsigned int type)
+{
+       void __iomem *base = gic_dist_base(d);
+       unsigned int irq = gic_irq(d);
+       u32 enablemask = 1 << (irq % 32);
+       u32 enableoff = (irq / 32) * 4;
+       u32 confmask = 0x2 << ((irq % 16) * 2);
+       u32 confoff = (irq / 16) * 4;
+       bool enabled = false;
+       u32 val;
+
+       /* Interrupt configuration for SGIs can't be changed */
+       if (irq < 16)
+               return -EINVAL;
+
+       if (type != IRQ_TYPE_LEVEL_HIGH && type != IRQ_TYPE_EDGE_RISING)
+               return -EINVAL;
+
+       raw_spin_lock(&dist_lock);
+
+       val = readl_relaxed(base + GICD_ICFGR + confoff);
+       if (type == IRQ_TYPE_LEVEL_HIGH)
+               val &= ~confmask;
+       else if (type == IRQ_TYPE_EDGE_RISING)
+               val |= confmask;
+
+       /*
+        * As recommended by the spec, disable the interrupt before changing
+        * the configuration
+        */
+       if (readl_relaxed(base + GICD_ISENABLER + enableoff) & enablemask) {
+               writel_relaxed(enablemask, base + GICD_ICENABLER + enableoff);
+               gic_wait_for_rwp(irq);
+               enabled = true;
+       }
+
+       writel_relaxed(val, base + GICD_ICFGR + confoff);
+
+       if (enabled) {
+               writel_relaxed(enablemask, base + GICD_ISENABLER + enableoff);
+               gic_wait_for_rwp(irq);
+       }
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 238

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+
+       raw_spin_unlock(&dist_lock);
+
+       return 0;
+}

I really think this should be merged with the existing GIC driver -- the code is basically identical.

Not sure. I agree that the code is extremely similar, but some things are slightly different:
- gic_dist_base can actually return a redistributor base address (which is per-cpu). That doesn't fit with with the GICv2 model very well.
- The synchronization function doesn't exist in GICv2.

The closest thing I can imagine is a primitive like:

void gic_configure_irq(unsigned int irq, unsigned int type,

		       void __iomem *base, void (*sync)(void));

that'd do the low level stuff.

Quoted 6 lines

drivers/irqchip/irq-gic-v3.c line 243

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+static int gic_retrigger(struct irq_data *d)
+{
+       return -ENXIO;
+}

Why bother? (just don't assign the pointer in the chip).

Fair enough.

Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 248

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+static u64 gic_mpidr_to_affinity(u64 mpidr)
+{
+       /* Make sure we don't broadcast the interrupt */
+       return mpidr & ~GICD_IROUTER_SPI_MODE_ANY;

Yikes, mpidr meets the GIC! I'd be a lot more comfortable with this if we used our mpidr accessors to construct the value which GICD_IROUTER expects (e.g. using the MPIDR_AFFINITY_LEVEL macro).

Sure.

Quoted 10 lines

drivers/irqchip/irq-gic-v3.c line 256

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+}
+
+static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
+{
+       u64 irqnr;
+
+       do {
+               irqnr = gic_read_iar();

What stops the CPU speculating this access around the loop?

Probably not much, if ICC_IAR_EL1 is not a bit "special". Speculating into this would be very dangerous though, but nothing says otherwise. I'll ask the question.

Quoted 9 lines
Quoted 9 lines
drivers/irqchip/irq-gic-v3.c new file +9 -0
+               if (likely(irqnr > 15 && irqnr < 1021)) {
+                       irqnr = irq_find_mapping(gic_data.domain, irqnr);
+                       handle_IRQ(irqnr, regs);
+                       continue;
+               }
+               if (irqnr < 16) {
+                       gic_write_eoir(irqnr);
+#ifdef CONFIG_SMP
+                       handle_IPI(irqnr, regs);
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 272

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+#else
+                       WARN_ONCE(true, "Unexpected SGI received!\n");
+#endif
+                       continue;
+               }
+       } while (irqnr != 0x3ff);

Any reason to adjust this control-flow from the existing GIC driver?

Mostly to accomodate the ITS code that goes right in the middle of this.

Quoted 35 lines
Quoted 35 lines
drivers/irqchip/irq-gic-v3.c new file +35 -0
+}
+
+static void __init gic_dist_init(void)
+{
+       unsigned int i;
+       u64 affinity;
+       int gic_irqs = gic_data.irq_nr;
+       void __iomem *base = gic_data.dist_base;
+
+       /* Disable the distributor */
+       writel_relaxed(0, base + GICD_CTLR);
+
+       /*
+        * Set all global interrupts to be level triggered, active low.
+        */
+       for (i = 32; i < gic_data.irq_nr; i += 16)
+               writel_relaxed(0, base + GICD_ICFGR + i / 4);
+
+       /*
+        * Set priority on all global interrupts.
+        */
+       for (i = 32; i < gic_irqs; i += 4)
+               writel_relaxed(0xa0a0a0a0, base + GICD_IPRIORITYR + i);
+
+       /*
+        * Disable all interrupts.  Leave the PPI and SGIs alone
+        * as these enables are banked registers.
+        */
+       for (i = 32; i < gic_irqs; i += 32)
+               writel_relaxed(0xffffffff, base + GICD_ICENABLER + i / 8);
+
+       gic_dist_wait_for_rwp();
+
+       writel_relaxed(GICD_CTLR_ARE_NS | GICD_CTLR_ENABLE_G1A | GICD_CTLR_ENABLE_G1,
+                      base + GICD_CTLR);
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 313

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+
+       /*
+        * Set all global interrupts to the boot CPU only. ARE must be
+        * enabled.
+        */
+       affinity = gic_mpidr_to_affinity(read_cpuid_mpidr());

You can use the logical map here.

OK.

Quoted 5 lines

drivers/irqchip/irq-gic-v3.c line 316

Quoted 3 lines
drivers/irqchip/irq-gic-v3.c new file +3 -0
+       for (i = 32; i < gic_data.irq_nr; i++)
+               writeq_relaxed(affinity, base + GICD_IROUTER + i * 8);
+}

Again, this looks very similar to the existing GIC driver.

I can probably do a similar thing as with gic_set_type, but I'm not sure it adds much in terms of being readable.

Quoted 11 lines

drivers/irqchip/irq-gic-v3.c line 326

Quoted 9 lines
drivers/irqchip/irq-gic-v3.c new file +9 -0
+static int __init gic_populate_rdist(void)
+{
+       u64 mpidr = cpu_logical_map(smp_processor_id());
+       u64 typer;
+       u64 aff;
+       int i;
+
+       aff  = mpidr & ((1 << 24) - 1);
+       aff |= (mpidr >> 8) & (0xffUL << 24);

We should have helpers for this in cputype.h.

Yes, I fixed that already in the latest version of the code.

Quoted 5 lines

Also, can we absolutely, 100%, "put-your-mother's-house-on-it" guarantee that the ITS, GICRn and the CPUs all agree on an identification scheme, as described in the mpidr? My reading of the spec doesn't fill me with confidence, and retrofitting a scheme in DT will end up with the same mess that we have for CPU topology.

My discussions with the architect of this fine piece of [insert expletive here] has led me to believe that the MPIDR is the only thing that holds the thing together. Of course, that doesn't preclude an implementer from wrecking the ship, but I don't plan to support something that doesn't follow the spec (or at least its intent).

Quoted 8 lines
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+
+       for (i = 0; i < gic_data.redist_regions; i++) {
+               void __iomem *ptr = gic_data.redist_base[i];
+               u32 reg;
+
+               reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
+               if (reg != 0x30 && reg != 0x40) { /* We're in trouble... */
+                       pr_warn("No redistributor present @%p\n", ptr);
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 339

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+                       break;
+               }
+
+               do {
+                       typer = readq_relaxed(ptr + GICR_TYPER);

Not sure if you care, but readq is a show-stopper if you want to build this with arch/arm/.

I realized that recently. Any reason why we don't have writeq/readq on LPAE-capable stuff (other than the obvious runtime detection madness)?

Quoted 16 lines
Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+                       if ((typer >> 32) == aff) {
+                               gic_data_rdist_rd_base() = ptr;
+                               pr_info("CPU%d: found redistributor %llx @%p\n",
+                                       smp_processor_id(),
+                                       (unsigned long long) mpidr, ptr);

drivers/irqchip/irq-gic-v3.c line 354

Quoted 10 lines
drivers/irqchip/irq-gic-v3.c new file +10 -0
+                               return 0;
+                       }
+
+                       if (gic_data.redist_stride) {
+                               ptr += gic_data.redist_stride;
+                       } else {
+                               ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+                               if (typer & GICR_TYPER_VLPIS)
+                                       ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+                       }
Quoted 1 line

VLPIS is RES0 in GICv3.

Indeed. But we still want this driver to run on GICv4 HW, even if we don't use VLPIs just yet.

Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 360

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+               } while (!(typer & GICR_TYPER_LAST));
+       }
+
+       /* We couldn't even deal with ourselves... */
+       WARN(true, "CPU%d: mpidr %lx has no re-distributor!\n",
+            smp_processor_id(), (unsigned long)mpidr);

Inconsistency with the type of mpidr (you use unsigned long long above).

Well spotted.

Quoted 23 lines
Quoted 23 lines
drivers/irqchip/irq-gic-v3.c new file +23 -0
+       return -ENODEV;
+}
+
+static void __init gic_cpu_init(void)
+{
+       void __iomem *rbase;
+       int i;
+
+       /* Register ourselves with the rest of the world */
+       if (gic_populate_rdist())
+               return;
+
+       gic_enable_redist();
+
+       rbase = gic_data_rdist_sgi_base();
+
+       /*
+        * Set priority on PPI and SGI interrupts
+        */
+       for (i = 0; i < 32; i += 4)
+               writel_relaxed(0xa0a0a0a0, rbase + GICR_IPRIORITYR0 + i * 4 / 4);
+
+       /*
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 388

Quoted 5 lines
drivers/irqchip/irq-gic-v3.c new file +5 -0
+        * Disable all PPI interrupts, ensure all SGI interrupts are
+        * enabled.
+        */
+       writel_relaxed(0xffff0000, rbase + GICR_ICENABLER0);
+       writel_relaxed(0x0000ffff, rbase + GICR_ISENABLER0);

Whilst there are some clear differences between this and the existing driver, you could still factor parts of this out into a common file.

OK.

Quoted 24 lines
Quoted 24 lines
drivers/irqchip/irq-gic-v3.c new file +24 -0
+       gic_redist_wait_for_rwp();
+
+       /* Enable system registers */
+       gic_enable_sre();
+
+       /* Set priority mask register */
+       gic_write_pmr(DEFAULT_PMR_VALUE);
+
+       /* EOI drops priority too (mode 0) */
+       gic_write_ctlr(GICC_CTLR_EL1_EOImode_drop_dir);
+
+       /* ... and let's hit the road... */
+       gic_write_grpen1(1);
+}
+
+#ifdef CONFIG_SMP
+static int __init gic_secondary_init(struct notifier_block *nfb,
+                                    unsigned long action, void *hcpu)
+{
+       if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+               gic_cpu_init();
+       return NOTIFY_OK;
+}
+
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 417

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+/*
+ * Notifier for enabling the GIC CPU interface. Set an arbitrarily high
+ * priority because the GIC needs to be up before the ARM generic timers.
+ */

Why must the GIC be up before the timers? Obviously, the timers won't be able to generate interrupts without the GIC initialised, but the comment implies that something worse will happen.

They will also fail to request the IRQ (no irq domain present yet), hence they will give up, and your kernel will grind to a halt...

Quoted 15 lines
Quoted 15 lines
drivers/irqchip/irq-gic-v3.c new file +15 -0
+static struct notifier_block __initdata gic_cpu_notifier = {
+       .notifier_call = gic_secondary_init,
+       .priority = 100,
+};
+
+static u16 gic_compute_target_list(int *base_cpu, const struct cpumask *mask,
+                                  u64 cluster_id)
+{
+       int cpu = *base_cpu;
+       u64 mpidr = cpu_logical_map(cpu);
+       u16 tlist = 0;
+
+       while (cpu < nr_cpu_ids) {
+               /*
+                * If we ever get a cluster of more than 16 CPUs, just
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 436

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c new file +4 -0
+                * scream and skip that CPU.
+                */
+               if (WARN_ON((mpidr & 0xff) >= 16))
+                       goto out;

I'm not comfortable with this. The mpidr in ARMv8 is just a unique identifier, so relying on aff0 to provide some sort of upper bound on the number of CPUs in a cluster feels extremely fragile.

And yet this is the only thing the GICv3 architecture supports. I've brought this up ages ago, and been told that "it is all fine". See the note at the bottom of 5.7.29: "this restricts distribution of SGIs to the first 16 processors of an affinity 1 cluster".

Quoted 3 lines

I think you'll need to deal with SGIs to target lists that you can't represent with a single sgi1r value (and instead batch them up and have lots of fun dealing with the result).

That's what the code already does. It is just that some processors may be out of the reach of GICv3 altogether (those with Aff0 >= 16).

Quoted 16 lines
Quoted 16 lines
drivers/irqchip/irq-gic-v3.c new file +16 -0
+
+               tlist |= 1 << (mpidr & 0xf);
+
+               cpu = cpumask_next(cpu, mask);
+               mpidr = cpu_logical_map(cpu);
+
+               if (cluster_id != (mpidr & ~0xffUL)) {
+                       cpu--;
+                       goto out;
+               }
+       }
+out:
+       *base_cpu = cpu;
+       return tlist;
+}
+
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 458

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+static void gic_send_sgi(u64 cluster_id, u16 tlist, unsigned int irq)
+{
+       u64 val;
+
+       val  = (cluster_id & 0xff00ff0000UL) << 16; /* Aff3 + Aff2 */
+       val |= (cluster_id & 0xff00) << 8;          /* Aff1 */

Use the helpers.

Yup.

Quoted 13 lines
Quoted 13 lines
drivers/irqchip/irq-gic-v3.c new file +13 -0
+       val |= irq << 24;
+       val |= tlist;
+
+       pr_debug("CPU%d: ICC_SGI1R_EL1 %llx\n", smp_processor_id(), val);
+       gic_write_sgi1r(val);
+}
+
+static void gic_raise_softirq(const struct cpumask *mask, unsigned int irq)
+{
+       int cpu;
+
+       if (WARN_ON(irq >= 16))
+               return;
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 477

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+
+       /*
+        * Ensure that stores to Normal memory are visible to the
+        * other CPUs before issuing the IPI.
+        */
+       dsb();

dmb(ishst), obviously (!) ;)

Indeed ;-).

Quoted 24 lines
Quoted 24 lines
drivers/irqchip/irq-gic-v3.c new file +24 -0
+
+       for_each_cpu_mask(cpu, *mask) {
+               u64 cluster_id = cpu_logical_map(cpu) & ~0xffUL;
+               u16 tlist;
+
+               tlist = gic_compute_target_list(&cpu, mask, cluster_id);
+               gic_send_sgi(cluster_id, tlist, irq);
+       }
+}
+
+static void gic_smp_init(void)
+{
+       set_smp_cross_call(gic_raise_softirq);
+       register_cpu_notifier(&gic_cpu_notifier);
+}
+
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+                           bool force)
+{
+       unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
+       void __iomem *reg;
+       u64 val;
+
+       if (gic_irq(d) < 32)
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 507

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+               return -EINVAL;
+
+       reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+       val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+       writeq_relaxed(val, reg);

How do you ensure that the interrupt has actually moved?

There is no way to ensure it actually did (other than reading back from the same register, if that's what you're aiming for?).

Quoted 27 lines
Quoted 27 lines
drivers/irqchip/irq-gic-v3.c new file +27 -0
+       return IRQ_SET_MASK_OK;
+}
+#else
+#define gic_set_affinity       NULL
+#define gic_smp_init()         do { } while(0)
+#endif
+
+static struct irq_chip gic_chip = {
+       .name                   = "GICv3",
+       .irq_mask               = gic_mask_irq,
+       .irq_unmask             = gic_unmask_irq,
+       .irq_eoi                = gic_eoi_irq,
+       .irq_set_type           = gic_set_type,
+       .irq_retrigger          = gic_retrigger,
+       .irq_set_affinity       = gic_set_affinity,
+};
+
+static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
+                               irq_hw_number_t hw)
+{
+       if (hw < 32) {
+               irq_set_percpu_devid(irq);
+               irq_set_chip_and_handler(irq, &gic_chip,
+                                        handle_percpu_devid_irq);
+               set_irq_flags(irq, IRQF_VALID | IRQF_NOAUTOEN);
+       } else {
+               irq_set_chip_and_handler(irq, &gic_chip,
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 541

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+                                        handle_fasteoi_irq);
+               set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
+       }
+       irq_set_chip_data(irq, d->host_data);
+       return 0;
+}

More copied GICv2 code...

I'll move it out, then.

Quoted 18 lines
Quoted 18 lines
drivers/irqchip/irq-gic-v3.c new file +18 -0
+static int gic_irq_domain_xlate(struct irq_domain *d,
+                               struct device_node *controller,
+                               const u32 *intspec, unsigned int intsize,
+                               unsigned long *out_hwirq, unsigned int *out_type)
+{
+       if (d->of_node != controller)
+               return -EINVAL;
+       if (intsize < 3)
+               return -EINVAL;
+
+       switch(intspec[0]) {
+       case 0:                 /* SPI */
+               *out_hwirq = intspec[1] + 32;
+               break;
+       case 1:                 /* PPI */
+               *out_hwirq = intspec[1] + 16;
+               break;
+       default:
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 566

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+               return -EINVAL;
+       }
+
+       *out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
+       return 0;
+}

... and some slightly mutated but functionally identical GICv2 code ;)

Same.

Quoted 36 lines
Quoted 36 lines
drivers/irqchip/irq-gic-v3.c new file +36 -0
+static const struct irq_domain_ops gic_irq_domain_ops = {
+       .map = gic_irq_domain_map,
+       .xlate = gic_irq_domain_xlate,
+};
+
+static int __init gic_of_init(struct device_node *node, struct device_node *parent)
+{
+       void __iomem *dist_base;
+       void __iomem **redist_base;
+       u64 redist_stride;
+       u32 redist_regions;
+       u32 reg;
+       int gic_irqs;
+       int err;
+       int i;
+
+       dist_base = of_iomap(node, 0);
+       if (!dist_base) {
+               pr_warn("%s: unable to map gic dist registers\n",
+                       node->full_name);
+               return -ENXIO;
+       }
+
+       reg = readl_relaxed(dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
+       if (reg != 0x30 && reg != 0x40) {
+               pr_warn("%s: no distributor detected, giving up\n",
+                       node->full_name);
+               err = -ENODEV;
+               goto out_unmap_dist;
+       }
+
+       if (of_property_read_u32(node, "#redistributor-regions", &redist_regions))
+               redist_regions = 1;
+
+       redist_base = kzalloc(sizeof(*redist_base) * redist_regions, GFP_KERNEL);
+       if (!redist_base) {
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 606

Quoted 3 lines
drivers/irqchip/irq-gic-v3.c new file +3 -0
+               err = -ENOMEM;
+               goto out_unmap_dist;
+       }

Can you use the devm_* managed resource helpers in irqchip init functions? I see a couple of drivers doing it, but it's not clear to me whether things are actually freed on failure. Anyway, if you can, then it would clean this up a bit.

The device model is not available at the stage where we probe the GIC. If it was, I'd have converted the GIC to be a proper driver a long time ago...

The only two users are secondary interrupt controllers, which are probed through DT, when the whole infrastructure is up and running.

Quoted 40 lines
Quoted 40 lines
drivers/irqchip/irq-gic-v3.c new file +40 -0
+       for (i = 0; i < redist_regions; i++) {
+               redist_base[i] = of_iomap(node, 1 + i);
+               if (!redist_base[i]) {
+                       pr_warn("%s: couldn't map region %d\n",
+                               node->full_name, i);
+                       err = -ENODEV;
+                       goto out_unmap_rdist;
+               }
+       }
+
+       if (of_property_read_u64(node, "redistributor-stride", &redist_stride))
+               redist_stride = 0;
+
+       gic_data.dist_base = dist_base;
+       gic_data.redist_base = redist_base;
+       gic_data.redist_regions = redist_regions;
+       gic_data.redist_stride = redist_stride;
+
+       /*
+        * Find out how many interrupts are supported.
+        * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)
+        */
+       gic_irqs = readl_relaxed(gic_data.dist_base + GICD_TYPER) & 0x1f;
+       gic_irqs = (gic_irqs + 1) * 32;
+       if (gic_irqs > 1020)
+               gic_irqs = 1020;
+       gic_data.irq_nr = gic_irqs;
+
+       gic_data.domain = irq_domain_add_linear(node, gic_irqs - 16,
+                                               &gic_irq_domain_ops, &gic_data);
+       gic_data.rdist = alloc_percpu(typeof(*gic_data.rdist));
+
+       if (WARN_ON(!gic_data.domain) || WARN_ON(!gic_data.rdist)) {
+               err = -ENOMEM;
+               goto out_free;
+       }
+
+       set_handle_irq(gic_handle_irq);
+
+       gic_smp_init();
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 653

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+       gic_dist_init();
+       gic_cpu_init();
+
+       return 0;
+
+out_free:

remove the irq_domain here?

Yup, good point.

Quoted 187 lines
Quoted 187 lines
drivers/irqchip/irq-gic-v3.c new file +12 -0
+       free_percpu(gic_data.rdist);
+out_unmap_rdist:
+       for (i = 0; i < redist_regions; i++)
+               if (redist_base[i])
+                       iounmap(redist_base[i]);
+       kfree(redist_base);
+out_unmap_dist:
+       iounmap(dist_base);
+       return err;
+}
+
+IRQCHIP_DECLARE(gic_v3, "arm,gic-v3", gic_of_init);
include/linux/irqchip/arm-gic-v3.h new file +169 -0
new file mode 100644
index 0000000..e4d060a
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2013 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <[email protected]>
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __LINUX_IRQCHIP_ARM_GIC_V3_H
+#define __LINUX_IRQCHIP_ARM_GIC_V3_H
+
+/*
+ * Distributor registers. We assume we're running non-secure, with ARE
+ * being set. Secure-only and non-ARE registers are not described.
+ */
+#define GICD_CTLR                      0x0000
+#define GICD_TYPER                     0x0004
+#define GICD_IIDR                      0x0008
+#define GICD_STATUSR                   0x0010
+#define GICD_SETSPI_NSR                        0x0040
+#define GICD_CLRSPI_NSR                        0x0048
+#define GICD_SETSPI_SR                 0x0050
+#define GICD_CLRSPI_SR                 0x0058
+#define GICD_SEIR                      0x0068
+#define GICD_ISENABLER                 0x0100
+#define GICD_ICENABLER                 0x0180
+#define GICD_ISPENDR                   0x0200
+#define GICD_ICPENDR                   0x0280
+#define GICD_ISACTIVER                 0x0300
+#define GICD_ICACTIVER                 0x0380
+#define GICD_IPRIORITYR                        0x0400
+#define GICD_ICFGR                     0x0C00
+#define GICD_IROUTER                   0x6000
+#define GICD_PIDR2                     0xFFE8
+
+#define GICD_CTLR_RWP                  (1U << 31)
+#define GICD_CTLR_ARE_NS               (1U << 4)
+#define GICD_CTLR_ENABLE_G1A           (1U << 1)
+#define GICD_CTLR_ENABLE_G1            (1U << 0)
+
+#define GICD_IROUTER_SPI_MODE_ONE      (0U << 31)
+#define GICD_IROUTER_SPI_MODE_ANY      (1U << 31)
+
+#define GIC_PIDR2_ARCH_MASK            0xf0
+
+/*
+ * Re-Distributor registers, offsets from RD_base
+ */
+#define GICR_CTLR                      GICD_CTLR
+#define GICR_IIDR                      0x0004
+#define GICR_TYPER                     0x0008
+#define GICR_STATUSR                   GICD_STATUSR
+#define GICR_WAKER                     0x0014
+#define GICR_SETLPIR                   0x0040
+#define GICR_CLRLPIR                   0x0048
+#define GICR_SEIR                      GICD_SEIR
+#define GICR_PROPBASER                 0x0070
+#define GICR_PENDBASER                 0x0078
+#define GICR_INVLPIR                   0x00A0
+#define GICR_INVALLR                   0x00B0
+#define GICR_SYNCR                     0x00C0
+#define GICR_MOVLPIR                   0x0100
+#define GICR_MOVALLR                   0x0110
+#define GICR_PIDR2                     GICD_PIDR2
+
+#define GICR_WAKER_ProcessorSleep      (1U << 1)
+#define GICR_WAKER_ChildrenAsleep      (1U << 2)
+
+/*
+ * Re-Distributor registers, offsets from SGI_base
+ */
+#define GICR_ISENABLER0                        GICD_ISENABLER
+#define GICR_ICENABLER0                        GICD_ICENABLER
+#define GICR_ISPENDR0                  GICD_ISPENDR
+#define GICR_ICPENDR0                  GICD_ICPENDR
+#define GICR_ISACTIVER0                        GICD_ISACTIVER
+#define GICR_ICACTIVER0                        GICD_ICACTIVER
+#define GICR_IPRIORITYR0               GICD_IPRIORITYR
+#define GICR_ICFGR0                    GICD_ICFGR
+
+#define GICR_TYPER_VLPIS               (1U << 1)
+#define GICR_TYPER_LAST                        (1U << 4)
+
+/*
+ * CPU interface registers
+ */
+#define GICC_CTLR_EL1_EOImode_drop_dir (0U << 1)
+#define GICC_CTLR_EL1_EOImode_drop     (1U << 1)
+#define GICC_SRE_EL1_SRE               (1U << 0)
+
+/*
+ * Hypervisor interface registers (SRE only)
+ */
+#define GICH_LR_VIRTUAL_ID_MASK                ((1UL << 32) - 1)
+
+#define GICH_LR_EOI                    (1UL << 41)
+#define GICH_LR_GROUP                  (1UL << 60)
+#define GICH_LR_STATE                  (3UL << 62)
+#define GICH_LR_PENDING_BIT            (1UL << 62)
+#define GICH_LR_ACTIVE_BIT             (1UL << 63)
+
+#define GICH_MISR_EOI                  (1 << 0)
+#define GICH_MISR_U                    (1 << 1)
+
+#define GICH_HCR_EN                    (1 << 0)
+#define GICH_HCR_UIE                   (1 << 1)
+
+#define GICH_VMCR_CTLR_SHIFT           0
+#define GICH_VMCR_CTLR_MASK            (0x21f << GICH_VMCR_CTLR_SHIFT)
+#define GICH_VMCR_BPR1_SHIFT           18
+#define GICH_VMCR_BPR1_MASK            (7 << GICH_VMCR_BPR1_SHIFT)
+#define GICH_VMCR_BPR0_SHIFT           21
+#define GICH_VMCR_BPR0_MASK            (7 << GICH_VMCR_BPR0_SHIFT)
+#define GICH_VMCR_PMR_SHIFT            24
+#define GICH_VMCR_PMR_MASK             (0xffUL << GICH_VMCR_PMR_SHIFT)
+
+#define ICC_EOIR1_EL1                  S3_0_C12_C12_1
+#define ICC_IAR1_EL1                   S3_0_C12_C12_0
+#define ICC_SGI1R_EL1                  S3_0_C12_C11_5
+#define ICC_PMR_EL1                    S3_0_C4_C6_0
+#define ICC_CTLR_EL1                   S3_0_C12_C12_4
+#define ICC_SRE_EL1                    S3_0_C12_C12_5
+#define ICC_GRPEN1_EL1                 S3_0_C12_C12_7
+
+#define ICC_SRE_EL2                    S3_4_C12_C9_5
+
+#define ICH_VSEIR_EL2                  S3_4_C12_C9_4
+#define ICH_HCR_EL2                    S3_4_C12_C11_0
+#define ICH_VTR_EL2                    S3_4_C12_C11_1
+#define ICH_MISR_EL2                   S3_4_C12_C11_2
+#define ICH_EISR_EL2                   S3_4_C12_C11_3
+#define ICH_ELSR_EL2                   S3_4_C12_C11_5
+#define ICH_VMCR_EL2                   S3_4_C12_C11_7
+
+#define __LR0_EL2(x)                   S3_4_C12_C12_ ## x
+#define __LR8_EL2(x)                   S3_4_C12_C13_ ## x
+
+#define ICH_LR0_EL2                    __LR0_EL2(0)
+#define ICH_LR1_EL2                    __LR0_EL2(1)
+#define ICH_LR2_EL2                    __LR0_EL2(2)
+#define ICH_LR3_EL2                    __LR0_EL2(3)
+#define ICH_LR4_EL2                    __LR0_EL2(4)
+#define ICH_LR5_EL2                    __LR0_EL2(5)
+#define ICH_LR6_EL2                    __LR0_EL2(6)
+#define ICH_LR7_EL2                    __LR0_EL2(7)
+#define ICH_LR8_EL2                    __LR8_EL2(0)
+#define ICH_LR9_EL2                    __LR8_EL2(1)
+#define ICH_LR10_EL2                   __LR8_EL2(2)
+#define ICH_LR11_EL2                   __LR8_EL2(3)
+#define ICH_LR12_EL2                   __LR8_EL2(4)
+#define ICH_LR13_EL2                   __LR8_EL2(5)
+#define ICH_LR14_EL2                   __LR8_EL2(6)
+#define ICH_LR15_EL2                   __LR8_EL2(7)
+
+#define __AP0Rx_EL2(x)                 S3_4_C12_C8_ ## x
+#define ICH_AP0R0_EL2                  __AP0Rx_EL2(0)
+#define ICH_AP0R1_EL2                  __AP0Rx_EL2(1)
+#define ICH_AP0R2_EL2                  __AP0Rx_EL2(2)
+#define ICH_AP0R3_EL2                  __AP0Rx_EL2(3)
+
Quoted 8 lines

include/linux/irqchip/arm-gic-v3.h line 174

Quoted 5 lines
include/linux/irqchip/arm-gic-v3.h new file +5 -0
+#define __AP1Rx_EL2(x)                 S3_4_C12_C9_ ## x
+#define ICH_AP1R0_EL2                  __AP1Rx_EL2(0)
+#define ICH_AP1R1_EL2                  __AP1Rx_EL2(1)
+#define ICH_AP1R2_EL2                  __AP1Rx_EL2(2)
+#define ICH_AP1R3_EL2                  __AP1Rx_EL2(3)

I'm going to rely on Christoffer having looked at all these magic numbers (he at least had ice cream!).

Quite! ;-)

Quoted 5 lines
Quoted 5 lines
include/linux/irqchip/arm-gic-v3.h new file +5 -0
+#ifndef __ASSEMBLY__
+
+#include <linux/stringify.h>
+
+#define reg(x)         __stringify(x)
Quoted 8 lines

include/linux/irqchip/arm-gic-v3.h line 186

Quoted 6 lines
include/linux/irqchip/arm-gic-v3.h new file +6 -0
+
+static inline void gic_write_eoir(u64 irq)
+{
+       asm volatile("msr " reg(ICC_EOIR1_EL1) ", %0" : : "r" (irq));
+       isb();
+}

Why is this in the header file?

For the benefit of the ITS part of the driver, which needs to EOI its own stuff.

Thanks for the excellent review!

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#10 lore raw

PATCH 1/18 arm64: initial support for GICv3

Quoted 6 lines · Marc Zyngier
Quoted 2 lines · Will Deacon

Not sure if you care, but readq is a show-stopper if you want to build this with arch/arm/.

I realized that recently. Any reason why we don't have writeq/readq on LPAE-capable stuff (other than the obvious runtime detection madness)?

I'm not sure that you'd be guaranteed single-copy atomicity for such an access, which is probably assumed by readq/writeq.

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
drivers/irqchip/irq-gic-v3.c new file +15 -0
+                       if ((typer >> 32) == aff) {
+                               gic_data_rdist_rd_base() = ptr;
+                               pr_info("CPU%d: found redistributor %llx @%p\n",
+                                       smp_processor_id(),
+                                       (unsigned long long) mpidr, ptr);
+                               return 0;
+                       }
+
+                       if (gic_data.redist_stride) {
+                               ptr += gic_data.redist_stride;
+                       } else {
+                               ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+                               if (typer & GICR_TYPER_VLPIS)
+                                       ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+                       }
Quoted 4 lines
Quoted 1 line

VLPIS is RES0 in GICv3.

Indeed. But we still want this driver to run on GICv4 HW, even if we don't use VLPIs just yet.

Ok, just mention that in the commit then.

Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+                           bool force)
+{
+       unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
+       void __iomem *reg;
+       u64 val;
+
+       if (gic_irq(d) < 32)
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 507

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+               return -EINVAL;
+
+       reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+       val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+       writeq_relaxed(val, reg);

How do you ensure that the interrupt has actually moved?

Quoted 2 lines

There is no way to ensure it actually did (other than reading back from the same register, if that's what you're aiming for?).

I guess it depends how angry Linux will get if the interrupt fires on the wrong CPU. If the interrupt is pending when the moving takes place, will it be asserted on the new target?

Will

#11 lore raw

PATCH 1/18 arm64: initial support for GICv3

Hi Marc,
I am from Cavium and we have tried your GICv3 driver on our internal emulation platform. We have booted systems with >8 cores and tested basic SGIs, PPIs and SPIs.
However we haven't exercised LPIs and ITS yet. We plan to do this soon.

If you have a new version of the patches coming out soon, we can exercise that as well on our platform.

regards,
Radha Mohan

Quoted 63 lines · Will Deacon
Quoted 6 lines · Marc Zyngier
Quoted 2 lines · Will Deacon

Not sure if you care, but readq is a show-stopper if you want to build this with arch/arm/.

I realized that recently. Any reason why we don't have writeq/readq on LPAE-capable stuff (other than the obvious runtime detection madness)?

I'm not sure that you'd be guaranteed single-copy atomicity for such an access, which is probably assumed by readq/writeq.

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
drivers/irqchip/irq-gic-v3.c new file +15 -0
+                       if ((typer >> 32) == aff) {
+                               gic_data_rdist_rd_base() = ptr;
+                               pr_info("CPU%d: found redistributor %llx @%p\n",
+                                       smp_processor_id(),
+                                       (unsigned long long) mpidr, ptr);
+                               return 0;
+                       }
+
+                       if (gic_data.redist_stride) {
+                               ptr += gic_data.redist_stride;
+                       } else {
+                               ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
+                               if (typer & GICR_TYPER_VLPIS)
+                                       ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+                       }
Quoted 4 lines
Quoted 1 line

VLPIS is RES0 in GICv3.

Indeed. But we still want this driver to run on GICv4 HW, even if we don't use VLPIs just yet.

Ok, just mention that in the commit then.

Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c new file +8 -0
+static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
+                           bool force)
+{
+       unsigned int cpu = cpumask_any_and(mask_val, cpu_online_mask);
+       void __iomem *reg;
+       u64 val;
+
+       if (gic_irq(d) < 32)
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c line 507

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c new file +6 -0
+               return -EINVAL;
+
+       reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8);
+       val = gic_mpidr_to_affinity(cpu_logical_map(cpu));
+
+       writeq_relaxed(val, reg);

How do you ensure that the interrupt has actually moved?

Quoted 2 lines

There is no way to ensure it actually did (other than reading back from the same register, if that's what you're aiming for?).

I guess it depends how angry Linux will get if the interrupt fires on the wrong CPU. If the interrupt is pending when the moving takes place, will it be asserted on the new target?

Will
_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm

#12 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

In reply to Marc Zyngier

Add the necessary documentation to support GICv3.

---
 Documentation/devicetree/bindings/arm/gic-v3.txt | 81 ++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/gic-v3.txt
Documentation/devicetree/bindings/arm/gic-v3.txt new file +81 -0
new file mode 100644
index 0000000..93852f6
@@ -0,0 +1,81 @@
+* ARM Generic Interrupt Controller, version 3
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.
+
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+
+  The 3rd cell is the flags, encoded as follows:
+	bits[3:0] trigger type and level flags.
+		1 = edge triggered
+		2 = edge triggered (deprecated, for compatibility with GICv2)
+		4 = level triggered
+		8 = level triggered (deprecated, for compatibility with GICv2)
+
+  Cells 4 and beyond are reserved for future use. Where the 1st cell
+  has a value of 0 or 1, cells 4 and beyond act as padding, and may be
+  ignored. It is recommended that padding cells have a value of 0.
+
+- reg : Specifies base physical address(s) and size of the GIC
+  registers, in the following order:
+  - GIC Distributor interface (GICD)
+  - GIC Redistributors (GICR), one range per redistributor region
+  - GIC CPU interface (GICC)
+  - GIC Hypervisor interface (GICH)
+  - GIC Virtual CPU interface (GICV)
+
+  GICC, GICH and GICV are optional.
+
+- interrupts : Interrupt source of the VGIC maintenance interrupt.
+
+Optional
+
+- redistributor-stride : If using padding pages, specifies the stride
+  of consecutive redistributors. Must be a multiple of 64kB.
+
+- #redistributor-regions: The number of independent contiguous regions
+  occupied by the redistributors. Required if more than one such
+  region is present.
+
+Examples:
+
+	gic: interrupt-controller at 2cf00000 {
+		compatible = "arm,gic-v3";
+		#interrupt-cells = <3>;
+		interrupt-controller;
+		reg = <0x0 0x2f000000 0 0x10000>,	// GICD
+		      <0x0 0x2f100000 0 0x200000>,	// GICR
+		      <0x0 0x2c000000 0 0x2000>,	// GICC
+		      <0x0 0x2c010000 0 0x2000>,	// GICH
+		      <0x0 0x2c020000 0 0x2000>;	// GICV
+		interrupts = <1 9 4>;
+	};
+
+	gic: interrupt-controller at 2c010000 {
+		compatible = "arm,gic-v3";
+		#interrupt-cells = <3>;
+		interrupt-controller;
+		redistributor-stride = 0x40000;		// 256kB stride
+		#redistributor-regions = <2>;
+		reg = <0x0 0x2c010000 0 0x10000>,	// GICD
+		      <0x0 0x2d000000 0 0x800000>,	// GICR 1: CPUs 0-31
+		      <0x0 0x2e000000 0 0x800000>;	// GICR 2: CPUs 32-63
+		      <0x0 0x2c040000 0 0x2000>,	// GICC
+		      <0x0 0x2c060000 0 0x2000>,	// GICH
+		      <0x0 0x2c080000 0 0x2000>;	// GICV
+		interrupts = <1 9 4>;
+	};
Signature · 1 line
1.8.3.4
#13 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

Hi Marc

Quoted 5 lines · Marc Zyngier <marc.zyngier <at> arm.com> writes:
Documentation/devicetree/bindings/arm/gic-v3.txt new file +5 -0
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).

Documentation/devicetree/bindings/arm/gic-v3.txt line 16

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.

These values are defined in "include/dt-bindings/interrupt-controller/arm-gic.h" maybe we should start mentioning that here and encourage future device treese to use those defines to improve readability.

Documentation/devicetree/bindings/arm/gic-v3.txt line 27

Quoted 11 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +11 -0
+
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+
+  The 3rd cell is the flags, encoded as follows:
+	bits[3:0] trigger type and level flags.
+		1 = edge triggered
+		2 = edge triggered (deprecated, for compatibility with GICv2)
+		4 = level triggered
+		8 = level triggered (deprecated, for compatibility with GICv2)

Similar to the above comment "include/dt-bindings/interrupt-controller/irq.h" defines the trigger type and level flags. Although this file currently contains the GICv2 bindings, we could update them.

Thanks
Arnab

Quoted 17 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +17 -0
+
+  Cells 4 and beyond are reserved for future use. Where the 1st cell
+  has a value of 0 or 1, cells 4 and beyond act as padding, and may be
+  ignored. It is recommended that padding cells have a value of 0.
+
+- reg : Specifies base physical address(s) and size of the GIC
+  registers, in the following order:
+  - GIC Distributor interface (GICD)
+  - GIC Redistributors (GICR), one range per redistributor region
+  - GIC CPU interface (GICC)
+  - GIC Hypervisor interface (GICH)
+  - GIC Virtual CPU interface (GICV)
+
+  GICC, GICH and GICV are optional.
+
+- interrupts : Interrupt source of the VGIC maintenance interrupt.
+
#14 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

Hi Arnab,

Quoted 16 lines · Arnab Basu

Hi Marc

Quoted 12 lines · Marc Zyngier <marc.zyngier <at> arm.com> writes:
Documentation/devicetree/bindings/arm/gic-v3.txt new file +12 -0
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
Quoted 8 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 16

Quoted 3 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +3 -0
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.

These values are defined in "include/dt-bindings/interrupt-controller/arm-gic.h" maybe we should start mentioning that here and encourage future device treese to use those defines to improve readability.

It may improve readability, but it makes the definition rely on something else. Definition and usage are two different things, and I want the definition to be completely self-contained and non ambiguous.

In DTS files, people can use whatever macro they decide, and it is their problem. They will even have out of tree DTS files, for which the include file is not available.

Quoted 17 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 27

Quoted 12 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +11 -0
+
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+
+  The 3rd cell is the flags, encoded as follows:
+	bits[3:0] trigger type and level flags.
+		1 = edge triggered
+		2 = edge triggered (deprecated, for compatibility with GICv2)
+		4 = level triggered
+		8 = level triggered (deprecated, for compatibility with GICv2)

Similar to the above comment "include/dt-bindings/interrupt-controller/irq.h" defines the trigger type and level flags. Although this file currently contains the GICv2 bindings, we could update them.

Same as above.

Thanks for the review!

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#15 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

Quoted 28 lines · Marc Zyngier

Hi Arnab,

Quoted 16 lines · Arnab Basu

Hi Marc

Quoted 12 lines · Marc Zyngier <marc.zyngier <at> arm.com> writes:
Documentation/devicetree/bindings/arm/gic-v3.txt new file +12 -0
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
Quoted 8 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 16

Quoted 3 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +3 -0
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.

These values are defined in "include/dt-bindings/interrupt-controller/arm-gic.h" maybe we should start mentioning that here and encourage future device treese to use those defines to improve readability.

Quoted 7 lines

It may improve readability, but it makes the definition rely on something else. Definition and usage are two different things, and I want the definition to be completely self-contained and non ambiguous.

In DTS files, people can use whatever macro they decide, and it is their problem. They will even have out of tree DTS files, for which the include file is not available.

Agreed.

Rob

#16 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

In reply to Marc Zyngier
Quoted 32 lines · Marc Zyngier

Add the necessary documentation to support GICv3.

---
 Documentation/devicetree/bindings/arm/gic-v3.txt | 81 ++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/gic-v3.txt
Documentation/devicetree/bindings/arm/gic-v3.txt new file +17 -0
new file mode 100644
index 0000000..93852f6
@@ -0,0 +1,81 @@
+* ARM Generic Interrupt Controller, version 3
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.
+

Documentation/devicetree/bindings/arm/gic-v3.txt line 27

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+
+  The 3rd cell is the flags, encoded as follows:
+       bits[3:0] trigger type and level flags.
+               1 = edge triggered
+               2 = edge triggered (deprecated, for compatibility with GICv2)
+               4 = level triggered
+               8 = level triggered (deprecated, for compatibility with GICv2)

I don't really think compatibility is needed here. I'd just list 1 and 4.

Documentation/devicetree/bindings/arm/gic-v3.txt line 31

Quoted 4 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +4 -0
+
+  Cells 4 and beyond are reserved for future use. Where the 1st cell
+  has a value of 0 or 1, cells 4 and beyond act as padding, and may be
+  ignored. It is recommended that padding cells have a value of 0.

What future use? LPIs I suppose.

Otherwise, looks fine to me:

Rob

#17 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

Quoted 37 lines · Rob Herring
Quoted 36 lines · Marc Zyngier

Add the necessary documentation to support GICv3.

---
 Documentation/devicetree/bindings/arm/gic-v3.txt | 81 ++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/gic-v3.txt
Documentation/devicetree/bindings/arm/gic-v3.txt new file +21 -0
new file mode 100644
index 0000000..93852f6
@@ -0,0 +1,81 @@
+* ARM Generic Interrupt Controller, version 3
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.
+
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+
Quoted 8 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 27

Quoted 6 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +6 -0
+  The 3rd cell is the flags, encoded as follows:
+       bits[3:0] trigger type and level flags.
+               1 = edge triggered
+               2 = edge triggered (deprecated, for compatibility with GICv2)
+               4 = level triggered
+               8 = level triggered (deprecated, for compatibility with GICv2)

I don't really think compatibility is needed here. I'd just list 1 and 4.

Fair enough.

Quoted 6 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 31

Quoted 4 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +4 -0
+
+  Cells 4 and beyond are reserved for future use. Where the 1st cell
+  has a value of 0 or 1, cells 4 and beyond act as padding, and may be
+  ignored. It is recommended that padding cells have a value of 0.

What future use? LPIs I suppose.

That's a possibility, for non-PCI devices. I'd rather not do that though, but I thought that leaving some room for (wacky) expansion.

Quoted 3 lines

Otherwise, looks fine to me:

Thanks!

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#18 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

In reply to Marc Zyngier
Quoted 15 lines · Marc Zyngier

Add the necessary documentation to support GICv3.

---
 Documentation/devicetree/bindings/arm/gic-v3.txt | 81 ++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/gic-v3.txt
Documentation/devicetree/bindings/arm/gic-v3.txt new file +0 -0
new file mode 100644
index 0000000..93852f6

Documentation/devicetree/bindings/arm/gic-v3.txt line 6

Quoted 6 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +6 -0
@@ -0,0 +1,81 @@
+* ARM Generic Interrupt Controller, version 3
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).

Super-over-ridiculous-nit: capitalize the definitions, e.g. Private Peripheral Interrupts (PPI)

Quoted 15 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +15 -0
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.
+
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+

Documentation/devicetree/bindings/arm/gic-v3.txt line 31

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+  The 3rd cell is the flags, encoded as follows:
+	bits[3:0] trigger type and level flags.
+		1 = edge triggered
+		2 = edge triggered (deprecated, for compatibility with GICv2)
+		4 = level triggered
+		8 = level triggered (deprecated, for compatibility with GICv2)
+
+  Cells 4 and beyond are reserved for future use. Where the 1st cell
+  has a value of 0 or 1, cells 4 and beyond act as padding, and may be
+  ignored. It is recommended that padding cells have a value of 0.

another-super-nit: Saying "If the 1st cell" or "When the 1st cell" sounds more clear to me, but I may be wrong.

Quoted 7 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +7 -0
+
+- reg : Specifies base physical address(s) and size of the GIC
+  registers, in the following order:
+  - GIC Distributor interface (GICD)
+  - GIC Redistributors (GICR), one range per redistributor region
+  - GIC CPU interface (GICC)
+  - GIC Hypervisor interface (GICH)

Documentation/devicetree/bindings/arm/gic-v3.txt line 48

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+  - GIC Virtual CPU interface (GICV)
+
+  GICC, GICH and GICV are optional.
+
+- interrupts : Interrupt source of the VGIC maintenance interrupt.
+
+Optional
+
+- redistributor-stride : If using padding pages, specifies the stride
+  of consecutive redistributors. Must be a multiple of 64kB.

How is this going to be used? What would the size of the padding then be? I couldn't find anything about this in the specs.

Or does this mean the total size of the set of contiguous pages for each of the redistributors? That seems to make the math in the example below work.

Quoted 14 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +14 -0
+
+- #redistributor-regions: The number of independent contiguous regions
+  occupied by the redistributors. Required if more than one such
+  region is present.
+
+Examples:
+
+	gic: interrupt-controller at 2cf00000 {
+		compatible = "arm,gic-v3";
+		#interrupt-cells = <3>;
+		interrupt-controller;
+		reg = <0x0 0x2f000000 0 0x10000>,	// GICD
+		      <0x0 0x2f100000 0 0x200000>,	// GICR
+		      <0x0 0x2c000000 0 0x2000>,	// GICC

Documentation/devicetree/bindings/arm/gic-v3.txt line 72

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+		      <0x0 0x2c010000 0 0x2000>,	// GICH
+		      <0x0 0x2c020000 0 0x2000>;	// GICV
+		interrupts = <1 9 4>;
+	};
+
+	gic: interrupt-controller at 2c010000 {
+		compatible = "arm,gic-v3";
+		#interrupt-cells = <3>;
+		interrupt-controller;
+		redistributor-stride = 0x40000;		// 256kB stride

I raised the point in patch 1 about the stide value being read as a 64-bit value, which would imply that the example should be the following, I think:

+ redistributor-stride = <0x0 0x40000>; // 256kB stride

Quoted 17 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +9 -0
+		#redistributor-regions = <2>;
+		reg = <0x0 0x2c010000 0 0x10000>,	// GICD
+		      <0x0 0x2d000000 0 0x800000>,	// GICR 1: CPUs 0-31
+		      <0x0 0x2e000000 0 0x800000>;	// GICR 2: CPUs 32-63
+		      <0x0 0x2c040000 0 0x2000>,	// GICC
+		      <0x0 0x2c060000 0 0x2000>,	// GICH
+		      <0x0 0x2c080000 0 0x2000>;	// GICV
+		interrupts = <1 9 4>;
+	};
Signature · 7 lines
1.8.3.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Otherwise it looks good.

#19 lore raw

PATCH 2/18 arm64: GICv3 device tree binding documentation

Quoted 10 lines · Christoffer Dall
Quoted 9 lines · Marc Zyngier

Add the necessary documentation to support GICv3.

---
 Documentation/devicetree/bindings/arm/gic-v3.txt | 81 ++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/gic-v3.txt
Quoted 15 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 6

Quoted 12 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +6 -0
new file mode 100644
index 0000000..93852f6
@@ -0,0 +1,81 @@
+* ARM Generic Interrupt Controller, version 3
+
+AArch64 SMP cores are often associated with a GICv3, providing private
+peripheral interrupts (PPI), shared peripheral interrupts (SPI),
+software generated interrupts (SGI), and locality-specific peripheral
+Interrupts (LPI).

Super-over-ridiculous-nit: capitalize the definitions, e.g. Private Peripheral Interrupts (PPI)

Sure.

Quoted 20 lines
Quoted 20 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +20 -0
+
+Main node required properties:
+
+- compatible : should at least contain  "arm,gic-v3".
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+  interrupt source. Must be a single cell with a value of at least 3.
+
+  The 1st cell is the interrupt type; 0 for SPI interrupts, 1 for PPI
+  interrupts. Other values are reserved for future use.
+
+  The 2nd cell contains the interrupt number for the interrupt type.
+  SPI interrupts are in the range [0-987]. PPI interrupts are in the
+  range [0-15].
+
+  The 3rd cell is the flags, encoded as follows:
+	bits[3:0] trigger type and level flags.
+		1 = edge triggered
+		2 = edge triggered (deprecated, for compatibility with GICv2)
+		4 = level triggered
Quoted 8 lines

Documentation/devicetree/bindings/arm/gic-v3.txt line 31

Quoted 5 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +5 -0
+		8 = level triggered (deprecated, for compatibility with GICv2)
+
+  Cells 4 and beyond are reserved for future use. Where the 1st cell
+  has a value of 0 or 1, cells 4 and beyond act as padding, and may be
+  ignored. It is recommended that padding cells have a value of 0.

another-super-nit: Saying "If the 1st cell" or "When the 1st cell" sounds more clear to me, but I may be wrong.

Works for me.

Quoted 21 lines
Quoted 7 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +7 -0
+
+- reg : Specifies base physical address(s) and size of the GIC
+  registers, in the following order:
+  - GIC Distributor interface (GICD)
+  - GIC Redistributors (GICR), one range per redistributor region
+  - GIC CPU interface (GICC)
+  - GIC Hypervisor interface (GICH)

Documentation/devicetree/bindings/arm/gic-v3.txt line 48

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+  - GIC Virtual CPU interface (GICV)
+
+  GICC, GICH and GICV are optional.
+
+- interrupts : Interrupt source of the VGIC maintenance interrupt.
+
+Optional
+
+- redistributor-stride : If using padding pages, specifies the stride
+  of consecutive redistributors. Must be a multiple of 64kB.

How is this going to be used? What would the size of the padding then be? I couldn't find anything about this in the specs.

Quoted 3 lines

Or does this mean the total size of the set of contiguous pages for each of the redistributors? That seems to make the math in the example below work.

Indeed. This is to allow overriding of the size of a single redistributor. The driver will follow the spec to the letter (two or four 64k pages, depending on GICR_TYPER.VLPI), except when this parameter is present.

Quoted 29 lines
Quoted 14 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +14 -0
+
+- #redistributor-regions: The number of independent contiguous regions
+  occupied by the redistributors. Required if more than one such
+  region is present.
+
+Examples:
+
+	gic: interrupt-controller at 2cf00000 {
+		compatible = "arm,gic-v3";
+		#interrupt-cells = <3>;
+		interrupt-controller;
+		reg = <0x0 0x2f000000 0 0x10000>,	// GICD
+		      <0x0 0x2f100000 0 0x200000>,	// GICR
+		      <0x0 0x2c000000 0 0x2000>,	// GICC

Documentation/devicetree/bindings/arm/gic-v3.txt line 72

Quoted 10 lines
Documentation/devicetree/bindings/arm/gic-v3.txt new file +10 -0
+		      <0x0 0x2c010000 0 0x2000>,	// GICH
+		      <0x0 0x2c020000 0 0x2000>;	// GICV
+		interrupts = <1 9 4>;
+	};
+
+	gic: interrupt-controller at 2c010000 {
+		compatible = "arm,gic-v3";
+		#interrupt-cells = <3>;
+		interrupt-controller;
+		redistributor-stride = 0x40000;		// 256kB stride

I raised the point in patch 1 about the stide value being read as a 64-bit value, which would imply that the example should be the following, I think:

Review

Quoted 1 line
(quoted diff) +1 -0
+		redistributor-stride = <0x0 0x40000>;		// 256kB stride

Indeed. I'll fix this.

Thanks for the review,

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#20 lore raw

PATCH 3/18 arm64: boot protocol documentation update for GICv3

In reply to Marc Zyngier

Linux has some requirements that must be satisfied in order to boot on a system built with a GICv3.

---
 Documentation/arm64/booting.txt | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.txt +7 -0
index a9691cc..4a02ebd 100644
@@ -131,6 +131,13 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
+  For systems with a GICv3 interrupt controller, it is expected that:
+  - ID_AA64PFR0_EL1.GIC (bits [27:24]) must have the value 0b0001
+  - If EL3 is present, it must program ICC_SRE_EL3.Enable (bit 3) to
+    0b1 and ICC_SRE_EL3.SRE (bit 0) to 0b1.
+  - If the kernel is entered at EL1, EL2 must set ICC_SRE_EL2.Enable
+    (bit 3) to 0b1 and ICC_SRE_EL2.SRE (bit 0) to 0b1.
+
 The requirements described above for CPU mode, caches, MMUs, architected
 timers, coherency and system registers apply to all CPUs.  All CPUs must
 enter the kernel in the same exception level.
Signature · 1 line
1.8.3.4
#22 lore raw

PATCH 3/18 arm64: boot protocol documentation update for GICv3

In reply to Marc Zyngier
Quoted 13 lines · Marc Zyngier

Linux has some requirements that must be satisfied in order to boot on a system built with a GICv3.

---
 Documentation/arm64/booting.txt | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.txt +0 -0
index a9691cc..4a02ebd 100644

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 135

Quoted 5 lines
Documentation/arm64/booting.txt +2 -0
@@ -131,6 +131,13 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
+  For systems with a GICv3 interrupt controller, it is expected that:
+  - ID_AA64PFR0_EL1.GIC (bits [27:24]) must have the value 0b0001

Since ID_AA64PFR0_EL1 is read-only at all exception levels, I don't see the value of this statement.

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 139

Quoted 4 lines
Documentation/arm64/booting.txt +4 -0
+  - If EL3 is present, it must program ICC_SRE_EL3.Enable (bit 3) to
+    0b1 and ICC_SRE_EL3.SRE (bit 0) to 0b1.
+  - If the kernel is entered at EL1, EL2 must set ICC_SRE_EL2.Enable
+    (bit 3) to 0b1 and ICC_SRE_EL2.SRE (bit 0) to 0b1.

Does this force the kernel to use SRE?

Will

#23 lore raw

PATCH 3/18 arm64: boot protocol documentation update for GICv3

Quoted 17 lines · Will Deacon
Quoted 16 lines · Marc Zyngier

Linux has some requirements that must be satisfied in order to boot on a system built with a GICv3.

---
 Documentation/arm64/booting.txt | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.txt +0 -0
index a9691cc..4a02ebd 100644
@@ -131,6 +131,13 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
Quoted 5 lines

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 135

Quoted 2 lines
Documentation/arm64/booting.txt +2 -0
+  For systems with a GICv3 interrupt controller, it is expected that:
+  - ID_AA64PFR0_EL1.GIC (bits [27:24]) must have the value 0b0001

Since ID_AA64PFR0_EL1 is read-only at all exception levels, I don't see the value of this statement.

Think virtualization. A hypervisor can control reads of ID_AA64PFR0_EL1 by setting HCR_EL2.TID3, and report whatever it wants.

Quoted 6 lines

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 139

Quoted 4 lines
Documentation/arm64/booting.txt +4 -0
+  - If EL3 is present, it must program ICC_SRE_EL3.Enable (bit 3) to
+    0b1 and ICC_SRE_EL3.SRE (bit 0) to 0b1.
+  - If the kernel is entered at EL1, EL2 must set ICC_SRE_EL2.Enable
+    (bit 3) to 0b1 and ICC_SRE_EL2.SRE (bit 0) to 0b1.

Does this force the kernel to use SRE?

No. The kernel still have to explicitly set ICC_SRE_EL1.SRE to 1.

Cheers,

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#24 lore raw

PATCH 3/18 arm64: boot protocol documentation update for GICv3

Quoted 18 lines · Marc Zyngier
Quoted 17 lines · Will Deacon
Quoted 16 lines · Marc Zyngier

Linux has some requirements that must be satisfied in order to boot on a system built with a GICv3.

---
 Documentation/arm64/booting.txt | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.txt +0 -0
index a9691cc..4a02ebd 100644
@@ -131,6 +131,13 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
Quoted 8 lines
Quoted 5 lines

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 135

Quoted 2 lines
Documentation/arm64/booting.txt +2 -0
+  For systems with a GICv3 interrupt controller, it is expected that:
+  - ID_AA64PFR0_EL1.GIC (bits [27:24]) must have the value 0b0001

Since ID_AA64PFR0_EL1 is read-only at all exception levels, I don't see the value of this statement.

Think virtualization. A hypervisor can control reads of ID_AA64PFR0_EL1 by setting HCR_EL2.TID3, and report whatever it wants.

Sure, but it seems unreasonable to me that we require a hypervisor to tell a guest about GICv3 if the system happens to have one. What if it wants to emulate a GICv2? In other words, requiring this in booting.txt seems superflous.

Will

#25 lore raw

PATCH 3/18 arm64: boot protocol documentation update for GICv3

Quoted 25 lines · Will Deacon
Quoted 24 lines · Marc Zyngier
Quoted 22 lines · Will Deacon
Quoted 13 lines · Marc Zyngier

Linux has some requirements that must be satisfied in order to boot on a system built with a GICv3.

---
 Documentation/arm64/booting.txt | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.txt +0 -0
index a9691cc..4a02ebd 100644

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 135

Quoted 5 lines
Documentation/arm64/booting.txt +2 -0
@@ -131,6 +131,13 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
+  For systems with a GICv3 interrupt controller, it is expected that:
+  - ID_AA64PFR0_EL1.GIC (bits [27:24]) must have the value 0b0001

Since ID_AA64PFR0_EL1 is read-only at all exception levels, I don't see the value of this statement.

Quoted 7 lines
Quoted 2 lines

Think virtualization. A hypervisor can control reads of ID_AA64PFR0_EL1 by setting HCR_EL2.TID3, and report whatever it wants.

Sure, but it seems unreasonable to me that we require a hypervisor to tell a guest about GICv3 if the system happens to have one. What if it wants to emulate a GICv2? In other words, requiring this in booting.txt seems superflous.

The hypervisor can perfectly mask out the GICv3 feature if it wants to. The point I'm trying to make here is that if a guest is expected to be able to use the GICv3 system registers, then ID_AA64PFR0_EL1 should report so.

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#26 lore raw

PATCH 3/18 arm64: boot protocol documentation update for GICv3

Quoted 34 lines · Marc Zyngier
Quoted 25 lines · Will Deacon
Quoted 24 lines · Marc Zyngier
Quoted 22 lines · Will Deacon
Quoted 13 lines · Marc Zyngier

Linux has some requirements that must be satisfied in order to boot on a system built with a GICv3.

---
 Documentation/arm64/booting.txt | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.txt +0 -0
index a9691cc..4a02ebd 100644

Documentation/arm64/booting.txt Before jumping into the kernel, the following conditions must be met: · line 135

Quoted 5 lines
Documentation/arm64/booting.txt +2 -0
@@ -131,6 +131,13 @@ Before jumping into the kernel, the following conditions must be met:
   the kernel image will be entered must be initialised by software at a
   higher exception level to prevent execution in an UNKNOWN state.
 
+  For systems with a GICv3 interrupt controller, it is expected that:
+  - ID_AA64PFR0_EL1.GIC (bits [27:24]) must have the value 0b0001

Since ID_AA64PFR0_EL1 is read-only at all exception levels, I don't see the value of this statement.

Quoted 7 lines
Quoted 2 lines

Think virtualization. A hypervisor can control reads of ID_AA64PFR0_EL1 by setting HCR_EL2.TID3, and report whatever it wants.

Sure, but it seems unreasonable to me that we require a hypervisor to tell a guest about GICv3 if the system happens to have one. What if it wants to emulate a GICv2? In other words, requiring this in booting.txt seems superflous.

Quoted 4 lines

The hypervisor can perfectly mask out the GICv3 feature if it wants to. The point I'm trying to make here is that if a guest is expected to be able to use the GICv3 system registers, then ID_AA64PFR0_EL1 should report so.

Fair enough, but that's not what the text says :)

Will

#27 lore raw

PATCH 4/18 KVM: arm/arm64: vgic: move GICv2 registers to their own structure

In reply to Marc Zyngier

In order to make way for the GICv3 registers, move the v2-specific registers to their own structure.

---
 arch/arm/kernel/asm-offsets.c   | 14 +++++------
 arch/arm/kvm/interrupts_head.S  | 26 ++++++++++-----------
 arch/arm64/kernel/asm-offsets.c | 14 +++++------
 arch/arm64/kvm/hyp.S            | 26 ++++++++++-----------
 include/kvm/arm_vgic.h          | 20 ++++++++++------
 virt/kvm/arm/vgic.c             | 52 ++++++++++++++++++++---------------------
 6 files changed, 79 insertions(+), 73 deletions(-)
arch/arm/kernel/asm-offsets.c +7 -7
index ded0417..dbe0476 100644
@@ -181,13 +181,13 @@ int main(void)
   DEFINE(VCPU_HYP_PC,		offsetof(struct kvm_vcpu, arch.fault.hyp_pc));
 #ifdef CONFIG_KVM_ARM_VGIC
   DEFINE(VCPU_VGIC_CPU,		offsetof(struct kvm_vcpu, arch.vgic_cpu));
-  DEFINE(VGIC_CPU_HCR,		offsetof(struct vgic_cpu, vgic_hcr));
-  DEFINE(VGIC_CPU_VMCR,		offsetof(struct vgic_cpu, vgic_vmcr));
-  DEFINE(VGIC_CPU_MISR,		offsetof(struct vgic_cpu, vgic_misr));
-  DEFINE(VGIC_CPU_EISR,		offsetof(struct vgic_cpu, vgic_eisr));
-  DEFINE(VGIC_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_elrsr));
-  DEFINE(VGIC_CPU_APR,		offsetof(struct vgic_cpu, vgic_apr));
-  DEFINE(VGIC_CPU_LR,		offsetof(struct vgic_cpu, vgic_lr));
+  DEFINE(VGIC_V2_CPU_HCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
+  DEFINE(VGIC_V2_CPU_VMCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
+  DEFINE(VGIC_V2_CPU_MISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_misr));
+  DEFINE(VGIC_V2_CPU_EISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_eisr));
+  DEFINE(VGIC_V2_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
+  DEFINE(VGIC_V2_CPU_APR,	offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
+  DEFINE(VGIC_V2_CPU_LR,	offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
   DEFINE(VGIC_CPU_NR_LR,	offsetof(struct vgic_cpu, nr_lr));
 #ifdef CONFIG_KVM_ARM_TIMER
   DEFINE(VCPU_TIMER_CNTV_CTL,	offsetof(struct kvm_vcpu, arch.timer_cpu.cntv_ctl));
arch/arm/kvm/interrupts_head.S +13 -13
index 6f18695..4a2a97a 100644
@@ -413,14 +413,14 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	ldr	r9, [r2, #GICH_ELRSR1]
 	ldr	r10, [r2, #GICH_APR]
 
-	str	r3, [r11, #VGIC_CPU_HCR]
-	str	r4, [r11, #VGIC_CPU_VMCR]
-	str	r5, [r11, #VGIC_CPU_MISR]
-	str	r6, [r11, #VGIC_CPU_EISR]
-	str	r7, [r11, #(VGIC_CPU_EISR + 4)]
-	str	r8, [r11, #VGIC_CPU_ELRSR]
-	str	r9, [r11, #(VGIC_CPU_ELRSR + 4)]
-	str	r10, [r11, #VGIC_CPU_APR]
+	str	r3, [r11, #VGIC_V2_CPU_HCR]
+	str	r4, [r11, #VGIC_V2_CPU_VMCR]
+	str	r5, [r11, #VGIC_V2_CPU_MISR]
+	str	r6, [r11, #VGIC_V2_CPU_EISR]
+	str	r7, [r11, #(VGIC_V2_CPU_EISR + 4)]
+	str	r8, [r11, #VGIC_V2_CPU_ELRSR]
+	str	r9, [r11, #(VGIC_V2_CPU_ELRSR + 4)]
+	str	r10, [r11, #VGIC_V2_CPU_APR]
 
 	/* Clear GICH_HCR */
 	mov	r5, #0
@@ -428,7 +428,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 
 	/* Save list registers */
 	add	r2, r2, #GICH_LR0
-	add	r3, r11, #VGIC_CPU_LR
+	add	r3, r11, #VGIC_V2_CPU_LR
 	ldr	r4, [r11, #VGIC_CPU_NR_LR]
 1:	ldr	r6, [r2], #4
 	str	r6, [r3], #4
@@ -455,9 +455,9 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	add	r11, vcpu, #VCPU_VGIC_CPU
 
 	/* We only restore a minimal set of registers */
-	ldr	r3, [r11, #VGIC_CPU_HCR]
-	ldr	r4, [r11, #VGIC_CPU_VMCR]
-	ldr	r8, [r11, #VGIC_CPU_APR]
+	ldr	r3, [r11, #VGIC_V2_CPU_HCR]
+	ldr	r4, [r11, #VGIC_V2_CPU_VMCR]
+	ldr	r8, [r11, #VGIC_V2_CPU_APR]
 
 	str	r3, [r2, #GICH_HCR]
 	str	r4, [r2, #GICH_VMCR]
@@ -465,7 +465,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 
 	/* Restore list registers */
 	add	r2, r2, #GICH_LR0
-	add	r3, r11, #VGIC_CPU_LR
+	add	r3, r11, #VGIC_V2_CPU_LR
 	ldr	r4, [r11, #VGIC_CPU_NR_LR]
 1:	ldr	r6, [r3], #4
 	str	r6, [r2], #4
arch/arm64/kernel/asm-offsets.c +7 -7
index 646f888..20fd488 100644
@@ -129,13 +129,13 @@ int main(void)
   DEFINE(KVM_TIMER_ENABLED,	offsetof(struct kvm, arch.timer.enabled));
   DEFINE(VCPU_KVM,		offsetof(struct kvm_vcpu, kvm));
   DEFINE(VCPU_VGIC_CPU,		offsetof(struct kvm_vcpu, arch.vgic_cpu));
-  DEFINE(VGIC_CPU_HCR,		offsetof(struct vgic_cpu, vgic_hcr));
-  DEFINE(VGIC_CPU_VMCR,		offsetof(struct vgic_cpu, vgic_vmcr));
-  DEFINE(VGIC_CPU_MISR,		offsetof(struct vgic_cpu, vgic_misr));
-  DEFINE(VGIC_CPU_EISR,		offsetof(struct vgic_cpu, vgic_eisr));
-  DEFINE(VGIC_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_elrsr));
-  DEFINE(VGIC_CPU_APR,		offsetof(struct vgic_cpu, vgic_apr));
-  DEFINE(VGIC_CPU_LR,		offsetof(struct vgic_cpu, vgic_lr));
+  DEFINE(VGIC_V2_CPU_HCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
+  DEFINE(VGIC_V2_CPU_VMCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
+  DEFINE(VGIC_V2_CPU_MISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_misr));
+  DEFINE(VGIC_V2_CPU_EISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_eisr));
+  DEFINE(VGIC_V2_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
+  DEFINE(VGIC_V2_CPU_APR,	offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
+  DEFINE(VGIC_V2_CPU_LR,	offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
   DEFINE(VGIC_CPU_NR_LR,	offsetof(struct vgic_cpu, nr_lr));
   DEFINE(KVM_VTTBR,		offsetof(struct kvm, arch.vttbr));
   DEFINE(KVM_VGIC_VCTRL,	offsetof(struct kvm, arch.vgic.vctrl_base));
arch/arm64/kvm/hyp.S +13 -13
index 3b47c36..c4c3a18 100644
@@ -412,14 +412,14 @@ CPU_BE(	rev	w9,  w9  )
 CPU_BE(	rev	w10, w10 )
 CPU_BE(	rev	w11, w11 )
 
-	str	w4, [x3, #VGIC_CPU_HCR]
-	str	w5, [x3, #VGIC_CPU_VMCR]
-	str	w6, [x3, #VGIC_CPU_MISR]
-	str	w7, [x3, #VGIC_CPU_EISR]
-	str	w8, [x3, #(VGIC_CPU_EISR + 4)]
-	str	w9, [x3, #VGIC_CPU_ELRSR]
-	str	w10, [x3, #(VGIC_CPU_ELRSR + 4)]
-	str	w11, [x3, #VGIC_CPU_APR]
+	str	w4, [x3, #VGIC_V2_CPU_HCR]
+	str	w5, [x3, #VGIC_V2_CPU_VMCR]
+	str	w6, [x3, #VGIC_V2_CPU_MISR]
+	str	w7, [x3, #VGIC_V2_CPU_EISR]
+	str	w8, [x3, #(VGIC_V2_CPU_EISR + 4)]
+	str	w9, [x3, #VGIC_V2_CPU_ELRSR]
+	str	w10, [x3, #(VGIC_V2_CPU_ELRSR + 4)]
+	str	w11, [x3, #VGIC_V2_CPU_APR]
 
 	/* Clear GICH_HCR */
 	str	wzr, [x2, #GICH_HCR]
@@ -427,7 +427,7 @@ CPU_BE(	rev	w11, w11 )
 	/* Save list registers */
 	add	x2, x2, #GICH_LR0
 	ldr	w4, [x3, #VGIC_CPU_NR_LR]
-	add	x3, x3, #VGIC_CPU_LR
+	add	x3, x3, #VGIC_V2_CPU_LR
 1:	ldr	w5, [x2], #4
 CPU_BE(	rev	w5, w5 )
 	str	w5, [x3], #4
@@ -452,9 +452,9 @@ CPU_BE(	rev	w5, w5 )
 	add	x3, x0, #VCPU_VGIC_CPU
 
 	/* We only restore a minimal set of registers */
-	ldr	w4, [x3, #VGIC_CPU_HCR]
-	ldr	w5, [x3, #VGIC_CPU_VMCR]
-	ldr	w6, [x3, #VGIC_CPU_APR]
+	ldr	w4, [x3, #VGIC_V2_CPU_HCR]
+	ldr	w5, [x3, #VGIC_V2_CPU_VMCR]
+	ldr	w6, [x3, #VGIC_V2_CPU_APR]
 CPU_BE(	rev	w4, w4 )
 CPU_BE(	rev	w5, w5 )
 CPU_BE(	rev	w6, w6 )
@@ -466,7 +466,7 @@ CPU_BE(	rev	w6, w6 )
 	/* Restore list registers */
 	add	x2, x2, #GICH_LR0
 	ldr	w4, [x3, #VGIC_CPU_NR_LR]
-	add	x3, x3, #VGIC_CPU_LR
+	add	x3, x3, #VGIC_V2_CPU_LR
 1:	ldr	w5, [x3], #4
 CPU_BE(	rev	w5, w5 )
 	str	w5, [x2], #4
include/kvm/arm_vgic.h +13 -7
index be85127..066f7d6 100644
@@ -110,6 +110,16 @@ struct vgic_dist {
 #endif
 };
 
+struct vgic_v2_cpu_if {
+	u32		vgic_hcr;
+	u32		vgic_vmcr;
+	u32		vgic_misr;	/* Saved only */
+	u32		vgic_eisr[2];	/* Saved only */
+	u32		vgic_elrsr[2];	/* Saved only */
+	u32		vgic_apr;
+	u32		vgic_lr[VGIC_MAX_LRS];
+};
+
 struct vgic_cpu {
 #ifdef CONFIG_KVM_ARM_VGIC
 	/* per IRQ to LR mapping */
@@ -126,13 +136,9 @@ struct vgic_cpu {
 	int		nr_lr;
 
 	/* CPU vif control registers for world switch */
-	u32		vgic_hcr;
-	u32		vgic_vmcr;
-	u32		vgic_misr;	/* Saved only */
-	u32		vgic_eisr[2];	/* Saved only */
-	u32		vgic_elrsr[2];	/* Saved only */
-	u32		vgic_apr;
-	u32		vgic_lr[VGIC_MAX_LRS];
+	union {
+		struct vgic_v2_cpu_if	vgic_v2;
+	};
 #endif
 };
 
virt/kvm/arm/vgic.c +26 -26

152 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#28 lore raw

PATCH 4/18 KVM: arm/arm64: vgic: move GICv2 registers to their own structure

Quoted 223 lines · Marc Zyngier

In order to make way for the GICv3 registers, move the v2-specific registers to their own structure.

---
 arch/arm/kernel/asm-offsets.c   | 14 +++++------
 arch/arm/kvm/interrupts_head.S  | 26 ++++++++++-----------
 arch/arm64/kernel/asm-offsets.c | 14 +++++------
 arch/arm64/kvm/hyp.S            | 26 ++++++++++-----------
 include/kvm/arm_vgic.h          | 20 ++++++++++------
 virt/kvm/arm/vgic.c             | 52 ++++++++++++++++++++---------------------
 6 files changed, 79 insertions(+), 73 deletions(-)
arch/arm/kernel/asm-offsets.c +7 -7
index ded0417..dbe0476 100644
@@ -181,13 +181,13 @@ int main(void)
   DEFINE(VCPU_HYP_PC,		offsetof(struct kvm_vcpu, arch.fault.hyp_pc));
 #ifdef CONFIG_KVM_ARM_VGIC
   DEFINE(VCPU_VGIC_CPU,		offsetof(struct kvm_vcpu, arch.vgic_cpu));
-  DEFINE(VGIC_CPU_HCR,		offsetof(struct vgic_cpu, vgic_hcr));
-  DEFINE(VGIC_CPU_VMCR,		offsetof(struct vgic_cpu, vgic_vmcr));
-  DEFINE(VGIC_CPU_MISR,		offsetof(struct vgic_cpu, vgic_misr));
-  DEFINE(VGIC_CPU_EISR,		offsetof(struct vgic_cpu, vgic_eisr));
-  DEFINE(VGIC_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_elrsr));
-  DEFINE(VGIC_CPU_APR,		offsetof(struct vgic_cpu, vgic_apr));
-  DEFINE(VGIC_CPU_LR,		offsetof(struct vgic_cpu, vgic_lr));
+  DEFINE(VGIC_V2_CPU_HCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
+  DEFINE(VGIC_V2_CPU_VMCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
+  DEFINE(VGIC_V2_CPU_MISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_misr));
+  DEFINE(VGIC_V2_CPU_EISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_eisr));
+  DEFINE(VGIC_V2_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
+  DEFINE(VGIC_V2_CPU_APR,	offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
+  DEFINE(VGIC_V2_CPU_LR,	offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
   DEFINE(VGIC_CPU_NR_LR,	offsetof(struct vgic_cpu, nr_lr));
 #ifdef CONFIG_KVM_ARM_TIMER
   DEFINE(VCPU_TIMER_CNTV_CTL,	offsetof(struct kvm_vcpu, arch.timer_cpu.cntv_ctl));
arch/arm/kvm/interrupts_head.S +13 -13
index 6f18695..4a2a97a 100644
@@ -413,14 +413,14 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	ldr	r9, [r2, #GICH_ELRSR1]
 	ldr	r10, [r2, #GICH_APR]
 
-	str	r3, [r11, #VGIC_CPU_HCR]
-	str	r4, [r11, #VGIC_CPU_VMCR]
-	str	r5, [r11, #VGIC_CPU_MISR]
-	str	r6, [r11, #VGIC_CPU_EISR]
-	str	r7, [r11, #(VGIC_CPU_EISR + 4)]
-	str	r8, [r11, #VGIC_CPU_ELRSR]
-	str	r9, [r11, #(VGIC_CPU_ELRSR + 4)]
-	str	r10, [r11, #VGIC_CPU_APR]
+	str	r3, [r11, #VGIC_V2_CPU_HCR]
+	str	r4, [r11, #VGIC_V2_CPU_VMCR]
+	str	r5, [r11, #VGIC_V2_CPU_MISR]
+	str	r6, [r11, #VGIC_V2_CPU_EISR]
+	str	r7, [r11, #(VGIC_V2_CPU_EISR + 4)]
+	str	r8, [r11, #VGIC_V2_CPU_ELRSR]
+	str	r9, [r11, #(VGIC_V2_CPU_ELRSR + 4)]
+	str	r10, [r11, #VGIC_V2_CPU_APR]
 
 	/* Clear GICH_HCR */
 	mov	r5, #0
@@ -428,7 +428,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 
 	/* Save list registers */
 	add	r2, r2, #GICH_LR0
-	add	r3, r11, #VGIC_CPU_LR
+	add	r3, r11, #VGIC_V2_CPU_LR
 	ldr	r4, [r11, #VGIC_CPU_NR_LR]
 1:	ldr	r6, [r2], #4
 	str	r6, [r3], #4
@@ -455,9 +455,9 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 	add	r11, vcpu, #VCPU_VGIC_CPU
 
 	/* We only restore a minimal set of registers */
-	ldr	r3, [r11, #VGIC_CPU_HCR]
-	ldr	r4, [r11, #VGIC_CPU_VMCR]
-	ldr	r8, [r11, #VGIC_CPU_APR]
+	ldr	r3, [r11, #VGIC_V2_CPU_HCR]
+	ldr	r4, [r11, #VGIC_V2_CPU_VMCR]
+	ldr	r8, [r11, #VGIC_V2_CPU_APR]
 
 	str	r3, [r2, #GICH_HCR]
 	str	r4, [r2, #GICH_VMCR]
@@ -465,7 +465,7 @@ vcpu	.req	r0		@ vcpu pointer always in r0
 
 	/* Restore list registers */
 	add	r2, r2, #GICH_LR0
-	add	r3, r11, #VGIC_CPU_LR
+	add	r3, r11, #VGIC_V2_CPU_LR
 	ldr	r4, [r11, #VGIC_CPU_NR_LR]
 1:	ldr	r6, [r3], #4
 	str	r6, [r2], #4
arch/arm64/kernel/asm-offsets.c +7 -7
index 646f888..20fd488 100644
@@ -129,13 +129,13 @@ int main(void)
   DEFINE(KVM_TIMER_ENABLED,	offsetof(struct kvm, arch.timer.enabled));
   DEFINE(VCPU_KVM,		offsetof(struct kvm_vcpu, kvm));
   DEFINE(VCPU_VGIC_CPU,		offsetof(struct kvm_vcpu, arch.vgic_cpu));
-  DEFINE(VGIC_CPU_HCR,		offsetof(struct vgic_cpu, vgic_hcr));
-  DEFINE(VGIC_CPU_VMCR,		offsetof(struct vgic_cpu, vgic_vmcr));
-  DEFINE(VGIC_CPU_MISR,		offsetof(struct vgic_cpu, vgic_misr));
-  DEFINE(VGIC_CPU_EISR,		offsetof(struct vgic_cpu, vgic_eisr));
-  DEFINE(VGIC_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_elrsr));
-  DEFINE(VGIC_CPU_APR,		offsetof(struct vgic_cpu, vgic_apr));
-  DEFINE(VGIC_CPU_LR,		offsetof(struct vgic_cpu, vgic_lr));
+  DEFINE(VGIC_V2_CPU_HCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
+  DEFINE(VGIC_V2_CPU_VMCR,	offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
+  DEFINE(VGIC_V2_CPU_MISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_misr));
+  DEFINE(VGIC_V2_CPU_EISR,	offsetof(struct vgic_cpu, vgic_v2.vgic_eisr));
+  DEFINE(VGIC_V2_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
+  DEFINE(VGIC_V2_CPU_APR,	offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
+  DEFINE(VGIC_V2_CPU_LR,	offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
   DEFINE(VGIC_CPU_NR_LR,	offsetof(struct vgic_cpu, nr_lr));
   DEFINE(KVM_VTTBR,		offsetof(struct kvm, arch.vttbr));
   DEFINE(KVM_VGIC_VCTRL,	offsetof(struct kvm, arch.vgic.vctrl_base));
arch/arm64/kvm/hyp.S +13 -13
index 3b47c36..c4c3a18 100644
@@ -412,14 +412,14 @@ CPU_BE(	rev	w9,  w9  )
 CPU_BE(	rev	w10, w10 )
 CPU_BE(	rev	w11, w11 )
 
-	str	w4, [x3, #VGIC_CPU_HCR]
-	str	w5, [x3, #VGIC_CPU_VMCR]
-	str	w6, [x3, #VGIC_CPU_MISR]
-	str	w7, [x3, #VGIC_CPU_EISR]
-	str	w8, [x3, #(VGIC_CPU_EISR + 4)]
-	str	w9, [x3, #VGIC_CPU_ELRSR]
-	str	w10, [x3, #(VGIC_CPU_ELRSR + 4)]
-	str	w11, [x3, #VGIC_CPU_APR]
+	str	w4, [x3, #VGIC_V2_CPU_HCR]
+	str	w5, [x3, #VGIC_V2_CPU_VMCR]
+	str	w6, [x3, #VGIC_V2_CPU_MISR]
+	str	w7, [x3, #VGIC_V2_CPU_EISR]
+	str	w8, [x3, #(VGIC_V2_CPU_EISR + 4)]
+	str	w9, [x3, #VGIC_V2_CPU_ELRSR]
+	str	w10, [x3, #(VGIC_V2_CPU_ELRSR + 4)]
+	str	w11, [x3, #VGIC_V2_CPU_APR]
 
 	/* Clear GICH_HCR */
 	str	wzr, [x2, #GICH_HCR]
@@ -427,7 +427,7 @@ CPU_BE(	rev	w11, w11 )
 	/* Save list registers */
 	add	x2, x2, #GICH_LR0
 	ldr	w4, [x3, #VGIC_CPU_NR_LR]
-	add	x3, x3, #VGIC_CPU_LR
+	add	x3, x3, #VGIC_V2_CPU_LR
 1:	ldr	w5, [x2], #4
 CPU_BE(	rev	w5, w5 )
 	str	w5, [x3], #4
@@ -452,9 +452,9 @@ CPU_BE(	rev	w5, w5 )
 	add	x3, x0, #VCPU_VGIC_CPU
 
 	/* We only restore a minimal set of registers */
-	ldr	w4, [x3, #VGIC_CPU_HCR]
-	ldr	w5, [x3, #VGIC_CPU_VMCR]
-	ldr	w6, [x3, #VGIC_CPU_APR]
+	ldr	w4, [x3, #VGIC_V2_CPU_HCR]
+	ldr	w5, [x3, #VGIC_V2_CPU_VMCR]
+	ldr	w6, [x3, #VGIC_V2_CPU_APR]
 CPU_BE(	rev	w4, w4 )
 CPU_BE(	rev	w5, w5 )
 CPU_BE(	rev	w6, w6 )
@@ -466,7 +466,7 @@ CPU_BE(	rev	w6, w6 )
 	/* Restore list registers */
 	add	x2, x2, #GICH_LR0
 	ldr	w4, [x3, #VGIC_CPU_NR_LR]
-	add	x3, x3, #VGIC_CPU_LR
+	add	x3, x3, #VGIC_V2_CPU_LR
 1:	ldr	w5, [x3], #4
 CPU_BE(	rev	w5, w5 )
 	str	w5, [x2], #4
include/kvm/arm_vgic.h +13 -7
index be85127..066f7d6 100644
@@ -110,6 +110,16 @@ struct vgic_dist {
 #endif
 };
 
+struct vgic_v2_cpu_if {
+	u32		vgic_hcr;
+	u32		vgic_vmcr;
+	u32		vgic_misr;	/* Saved only */
+	u32		vgic_eisr[2];	/* Saved only */
+	u32		vgic_elrsr[2];	/* Saved only */
+	u32		vgic_apr;
+	u32		vgic_lr[VGIC_MAX_LRS];
+};
+
 struct vgic_cpu {
 #ifdef CONFIG_KVM_ARM_VGIC
 	/* per IRQ to LR mapping */
@@ -126,13 +136,9 @@ struct vgic_cpu {
 	int		nr_lr;
 
 	/* CPU vif control registers for world switch */
-	u32		vgic_hcr;
-	u32		vgic_vmcr;
-	u32		vgic_misr;	/* Saved only */
-	u32		vgic_eisr[2];	/* Saved only */
-	u32		vgic_elrsr[2];	/* Saved only */
-	u32		vgic_apr;
-	u32		vgic_lr[VGIC_MAX_LRS];
+	union {
+		struct vgic_v2_cpu_if	vgic_v2;
+	};
 #endif
 };
 
virt/kvm/arm/vgic.c +0 -0
index 8ca405c..fdc2696 100644

virt/kvm/arm/vgic.c static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu, · line 605

Quoted 5 lines
virt/kvm/arm/vgic.c +1 -1
@@ -602,7 +602,7 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu,
 static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu *vgic_cpu)
 {
 	clear_bit(lr_nr, vgic_cpu->lr_used);
-	vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
+	vgic_cpu->vgic_v2.vgic_lr[lr] &= ~GICH_LR_STATE;

lr_nr, this breaks bisectability

Quoted 138 lines
virt/kvm/arm/vgic.c +22 -22
 	vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
 }
 
@@ -1007,7 +1007,7 @@ static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu)
 	int lr;
 
 	for_each_set_bit(lr, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
-		int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
+		int irq = vgic_cpu->vgic_v2.vgic_lr[lr] & GICH_LR_VIRTUALID;
 
 		if (!vgic_irq_is_enabled(vcpu, irq)) {
 			vgic_retire_lr(lr, irq, vgic_cpu);
@@ -1037,11 +1037,11 @@ static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
 
 	/* Do we have an active interrupt for the same CPUID? */
 	if (lr != LR_EMPTY &&
-	    (LR_CPUID(vgic_cpu->vgic_lr[lr]) == sgi_source_id)) {
+	    (LR_CPUID(vgic_cpu->vgic_v2.vgic_lr[lr]) == sgi_source_id)) {
 		kvm_debug("LR%d piggyback for IRQ%d %x\n",
-			  lr, irq, vgic_cpu->vgic_lr[lr]);
+			  lr, irq, vgic_cpu->vgic_v2.vgic_lr[lr]);
 		BUG_ON(!test_bit(lr, vgic_cpu->lr_used));
-		vgic_cpu->vgic_lr[lr] |= GICH_LR_PENDING_BIT;
+		vgic_cpu->vgic_v2.vgic_lr[lr] |= GICH_LR_PENDING_BIT;
 		return true;
 	}
 
@@ -1052,12 +1052,12 @@ static bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
 		return false;
 
 	kvm_debug("LR%d allocated for IRQ%d %x\n", lr, irq, sgi_source_id);
-	vgic_cpu->vgic_lr[lr] = MK_LR_PEND(sgi_source_id, irq);
+	vgic_cpu->vgic_v2.vgic_lr[lr] = MK_LR_PEND(sgi_source_id, irq);
 	vgic_cpu->vgic_irq_lr_map[irq] = lr;
 	set_bit(lr, vgic_cpu->lr_used);
 
 	if (!vgic_irq_is_edge(vcpu, irq))
-		vgic_cpu->vgic_lr[lr] |= GICH_LR_EOI;
+		vgic_cpu->vgic_v2.vgic_lr[lr] |= GICH_LR_EOI;
 
 	return true;
 }
@@ -1155,9 +1155,9 @@ static void __kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 
 epilog:
 	if (overflow) {
-		vgic_cpu->vgic_hcr |= GICH_HCR_UIE;
+		vgic_cpu->vgic_v2.vgic_hcr |= GICH_HCR_UIE;
 	} else {
-		vgic_cpu->vgic_hcr &= ~GICH_HCR_UIE;
+		vgic_cpu->vgic_v2.vgic_hcr &= ~GICH_HCR_UIE;
 		/*
 		 * We're about to run this VCPU, and we've consumed
 		 * everything the distributor had in store for
@@ -1173,21 +1173,21 @@ static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
 	bool level_pending = false;
 
-	kvm_debug("MISR = %08x\n", vgic_cpu->vgic_misr);
+	kvm_debug("MISR = %08x\n", vgic_cpu->vgic_v2.vgic_misr);
 
-	if (vgic_cpu->vgic_misr & GICH_MISR_EOI) {
+	if (vgic_cpu->vgic_v2.vgic_misr & GICH_MISR_EOI) {
 		/*
 		 * Some level interrupts have been EOIed. Clear their
 		 * active bit.
 		 */
 		int lr, irq;
 
-		for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_eisr,
+		for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_v2.vgic_eisr,
 				 vgic_cpu->nr_lr) {
-			irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
+			irq = vgic_cpu->vgic_v2.vgic_lr[lr] & GICH_LR_VIRTUALID;
 
 			vgic_irq_clear_active(vcpu, irq);
-			vgic_cpu->vgic_lr[lr] &= ~GICH_LR_EOI;
+			vgic_cpu->vgic_v2.vgic_lr[lr] &= ~GICH_LR_EOI;
 
 			/* Any additional pending interrupt? */
 			if (vgic_dist_irq_is_pending(vcpu, irq)) {
@@ -1201,13 +1201,13 @@ static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
 			 * Despite being EOIed, the LR may not have
 			 * been marked as empty.
 			 */
-			set_bit(lr, (unsigned long *)vgic_cpu->vgic_elrsr);
-			vgic_cpu->vgic_lr[lr] &= ~GICH_LR_ACTIVE_BIT;
+			set_bit(lr, (unsigned long *)vgic_cpu->vgic_v2.vgic_elrsr);
+			vgic_cpu->vgic_v2.vgic_lr[lr] &= ~GICH_LR_ACTIVE_BIT;
 		}
 	}
 
-	if (vgic_cpu->vgic_misr & GICH_MISR_U)
-		vgic_cpu->vgic_hcr &= ~GICH_HCR_UIE;
+	if (vgic_cpu->vgic_v2.vgic_misr & GICH_MISR_U)
+		vgic_cpu->vgic_v2.vgic_hcr &= ~GICH_HCR_UIE;
 
 	return level_pending;
 }
@@ -1226,21 +1226,21 @@ static void __kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 	level_pending = vgic_process_maintenance(vcpu);
 
 	/* Clear mappings for empty LRs */
-	for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_elrsr,
+	for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_v2.vgic_elrsr,
 			 vgic_cpu->nr_lr) {
 		int irq;
 
 		if (!test_and_clear_bit(lr, vgic_cpu->lr_used))
 			continue;
 
-		irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
+		irq = vgic_cpu->vgic_v2.vgic_lr[lr] & GICH_LR_VIRTUALID;
 
 		BUG_ON(irq >= VGIC_NR_IRQS);
 		vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
 	}
 
 	/* Check if we still have something up our sleeve... */
-	pending = find_first_zero_bit((unsigned long *)vgic_cpu->vgic_elrsr,
+	pending = find_first_zero_bit((unsigned long *)vgic_cpu->vgic_v2.vgic_elrsr,
 				      vgic_cpu->nr_lr);
 	if (level_pending || pending < vgic_cpu->nr_lr)
 		set_bit(vcpu->vcpu_id, &dist->irq_pending_on_cpu);
@@ -1436,10 +1436,10 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
 	 * points to their reset values. Anything else resets to zero
 	 * anyway.
 	 */
-	vgic_cpu->vgic_vmcr = 0;
+	vgic_cpu->vgic_v2.vgic_vmcr = 0;
 
 	vgic_cpu->nr_lr = vgic_nr_lr;
-	vgic_cpu->vgic_hcr = GICH_HCR_EN; /* Get the show on the road... */
+	vgic_cpu->vgic_v2.vgic_hcr = GICH_HCR_EN; /* Get the show on the road... */
 
 	return 0;
 }

virt/kvm/arm/vgic.c static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu, · line 1753

Quoted 10 lines
virt/kvm/arm/vgic.c +1 -1
@@ -1745,15 +1745,15 @@ static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
 	}
 
 	if (!mmio->is_write) {
-		reg = (vgic_cpu->vgic_vmcr & mask) >> shift;
+		reg = (vgic_cpu->vgic_v2.vgic_vmcr & mask) >> shift;
 		mmio_data_write(mmio, ~0, reg);
 	} else {
 		reg = mmio_data_read(mmio, ~0);
 		reg = (reg << shift) & mask;
 		if (reg != (vgic_cpu->vgic_vmcr & mask))

You forgot one here

Quoted 11 lines
virt/kvm/arm/vgic.c +2 -2
 			updated = true;
-		vgic_cpu->vgic_vmcr &= ~mask;
-		vgic_cpu->vgic_vmcr |= reg;
+		vgic_cpu->vgic_v2.vgic_vmcr &= ~mask;
+		vgic_cpu->vgic_v2.vgic_vmcr |= reg;
 	}
 	return updated;
 }
Signature · 1 line
1.8.3.4

I think you also left one out in vgic_unqueue_irqs.

#29 lore raw

PATCH 5/18 KVM: ARM: vgic: introduce vgic_ops and LR manipulation primitives

In reply to Marc Zyngier

In order to split the various register manipulation from the main vgic code, introduce a vgic_ops structure, and start by abstracting the LR manipulation code with a couple of accessors.

---
 include/kvm/arm_vgic.h |  18 ++++++
 virt/kvm/arm/vgic.c    | 170 +++++++++++++++++++++++++++++++++----------------
 2 files changed, 132 insertions(+), 56 deletions(-)
include/kvm/arm_vgic.h +18 -0
index 066f7d6..81ca525 100644
@@ -68,6 +68,24 @@ struct vgic_bytemap {
 	u32 shared[VGIC_NR_SHARED_IRQS  / 4];
 };
 
+struct kvm_vcpu;
+
+#define LR_STATE_PENDING	(1 << 0)
+#define LR_STATE_ACTIVE		(1 << 1)
+#define LR_STATE_MASK		(3 << 0)
+#define LR_EOI_INT		(1 << 2)
+
+struct vgic_lr {
+	u16	irq;
+	u8	source;
+	u8	state;
+};
+
+struct vgic_ops {
+	struct vgic_lr	(*get_lr)(const struct kvm_vcpu *, int);
+	void	(*set_lr)(struct kvm_vcpu *, int, struct vgic_lr);
+};
+
 struct vgic_dist {
 #ifdef CONFIG_KVM_ARM_VGIC
 	spinlock_t		lock;
virt/kvm/arm/vgic.c +114 -56

275 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#30 lore raw

PATCH 6/18 KVM: ARM: vgic: abstract access to the ELRSR bitmap

In reply to Marc Zyngier

Move the GICH_ELRSR access to its own function, and add it to the vgic_ops structure.

---
 include/kvm/arm_vgic.h |  1 +
 virt/kvm/arm/vgic.c    | 20 ++++++++++++++++----
 2 files changed, 17 insertions(+), 4 deletions(-)
include/kvm/arm_vgic.h +1 -0
index 81ca525..e82e3bb 100644
@@ -84,6 +84,7 @@ struct vgic_lr {
 struct vgic_ops {
 	struct vgic_lr	(*get_lr)(const struct kvm_vcpu *, int);
 	void	(*set_lr)(struct kvm_vcpu *, int, struct vgic_lr);
+	u64	(*get_elrsr)(const struct kvm_vcpu *vcpu);
 };
 
 struct vgic_dist {
virt/kvm/arm/vgic.c +16 -4
index e25309d..ca91c21 100644
@@ -1022,9 +1022,16 @@ static void vgic_v2_set_lr(struct kvm_vcpu *vcpu, int lr,
 		set_bit(lr, (unsigned long *)vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr);
 }
 
+static u64 vgic_v2_get_elrsr(const struct kvm_vcpu *vcpu)
+{
+	const u32 *elrsr = vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr;
+	return *(u64 *)elrsr;
+}
+
 static const struct vgic_ops vgic_ops = {
 	.get_lr			= vgic_v2_get_lr,
 	.set_lr			= vgic_v2_set_lr,
+	.get_elrsr		= vgic_v2_get_elrsr,
 };
 
 static inline struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr)
@@ -1038,6 +1045,11 @@ static inline void vgic_set_lr(struct kvm_vcpu *vcpu, int lr,
 	vgic_ops.set_lr(vcpu, lr, vlr);
 }
 
+static inline u64 vgic_get_elrsr(struct kvm_vcpu *vcpu)
+{
+	return vgic_ops.get_elrsr(vcpu);
+}
+
 static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
@@ -1278,14 +1290,15 @@ static void __kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
 	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+	u64 elrsr = vgic_get_elrsr(vcpu);
+	unsigned long *elrsr_ptr = (unsigned long *)&elrsr;
 	int lr, pending;
 	bool level_pending;
 
 	level_pending = vgic_process_maintenance(vcpu);
 
 	/* Clear mappings for empty LRs */
-	for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_v2.vgic_elrsr,
-			 vgic_cpu->nr_lr) {
+	for_each_set_bit(lr, elrsr_ptr, vgic_cpu->nr_lr) {
 		struct vgic_lr vlr;
 
 		if (!test_and_clear_bit(lr, vgic_cpu->lr_used))
@@ -1298,8 +1311,7 @@ static void __kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 	}
 
 	/* Check if we still have something up our sleeve... */
-	pending = find_first_zero_bit((unsigned long *)vgic_cpu->vgic_v2.vgic_elrsr,
-				      vgic_cpu->nr_lr);
+	pending = find_first_zero_bit(elrsr_ptr, vgic_cpu->nr_lr);
 	if (level_pending || pending < vgic_cpu->nr_lr)
 		set_bit(vcpu->vcpu_id, &dist->irq_pending_on_cpu);
 }
Signature · 1 line
1.8.3.4
#31 lore raw

PATCH 7/18 KVM: ARM: vgic: abstract EISR bitmap access

In reply to Marc Zyngier

Move the GICH_EISR access to its own function.

---
 include/kvm/arm_vgic.h |  1 +
 virt/kvm/arm/vgic.c    | 16 +++++++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)
include/kvm/arm_vgic.h +1 -0
index e82e3bb..7928909 100644
@@ -85,6 +85,7 @@ struct vgic_ops {
 	struct vgic_lr	(*get_lr)(const struct kvm_vcpu *, int);
 	void	(*set_lr)(struct kvm_vcpu *, int, struct vgic_lr);
 	u64	(*get_elrsr)(const struct kvm_vcpu *vcpu);
+	u64	(*get_eisr)(const struct kvm_vcpu *vcpu);
 };
 
 struct vgic_dist {
virt/kvm/arm/vgic.c +15 -1
index ca91c21..8cb5f22 100644
@@ -1028,10 +1028,17 @@ static u64 vgic_v2_get_elrsr(const struct kvm_vcpu *vcpu)
 	return *(u64 *)elrsr;
 }
 
+static u64 vgic_v2_get_eisr(const struct kvm_vcpu *vcpu)
+{
+	const u32 *eisr = vcpu->arch.vgic_cpu.vgic_v2.vgic_eisr;
+	return *(u64 *)eisr;
+}
+
 static const struct vgic_ops vgic_ops = {
 	.get_lr			= vgic_v2_get_lr,
 	.set_lr			= vgic_v2_set_lr,
 	.get_elrsr		= vgic_v2_get_elrsr,
+	.get_eisr		= vgic_v2_get_eisr,
 };
 
 static inline struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr)
@@ -1050,6 +1057,11 @@ static inline u64 vgic_get_elrsr(struct kvm_vcpu *vcpu)
 	return vgic_ops.get_elrsr(vcpu);
 }
 
+static inline u64 vgic_get_eisr(struct kvm_vcpu *vcpu)
+{
+	return vgic_ops.get_eisr(vcpu);
+}
+
 static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
@@ -1256,9 +1268,11 @@ static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
 		 * Some level interrupts have been EOIed. Clear their
 		 * active bit.
 		 */
+		u64 eisr = vgic_get_eisr(vcpu);
+		unsigned long *eisr_ptr = (unsigned long *)&eisr;
 		int lr;
 
-		for_each_set_bit(lr, (unsigned long *)vgic_cpu->vgic_v2.vgic_eisr,
+		for_each_set_bit(lr, eisr_ptr, vgic_cpu->nr_lr) {
 				 vgic_cpu->nr_lr) {
 			struct vgic_lr vlr = vgic_get_lr(vcpu, lr);
 
Signature · 1 line
1.8.3.4
#32 lore raw

PATCH 8/18 KVM: ARM: vgic: abstract MISR decoding

In reply to Marc Zyngier

Instead of directly dealing with the GICH_MISR bits, move the code to its own function and use a couple of public flags to represent the actual state.

---
 include/kvm/arm_vgic.h |  4 ++++
 virt/kvm/arm/vgic.c    | 26 +++++++++++++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)
include/kvm/arm_vgic.h +4 -0
index 7928909..95085a6 100644
@@ -86,6 +86,7 @@ struct vgic_ops {
 	void	(*set_lr)(struct kvm_vcpu *, int, struct vgic_lr);
 	u64	(*get_elrsr)(const struct kvm_vcpu *vcpu);
 	u64	(*get_eisr)(const struct kvm_vcpu *vcpu);
+	u32	(*get_interrupt_status)(const struct kvm_vcpu *vcpu);
 };
 
 struct vgic_dist {
@@ -164,6 +165,9 @@ struct vgic_cpu {
 
 #define LR_EMPTY	0xff
 
+#define INT_STATUS_EOI		(1 << 0)
+#define INT_STATUS_UNDERFLOW	(1 << 1)
+
 struct kvm;
 struct kvm_vcpu;
 struct kvm_run;
virt/kvm/arm/vgic.c +23 -3
index 8cb5f22..22423dd 100644
@@ -1034,11 +1034,25 @@ static u64 vgic_v2_get_eisr(const struct kvm_vcpu *vcpu)
 	return *(u64 *)eisr;
 }
 
+static u32 vgic_v2_get_interrupt_status(const struct kvm_vcpu *vcpu)
+{
+	u32 misr = vcpu->arch.vgic_cpu.vgic_v2.vgic_misr;
+	u32 ret = 0;
+
+	if (misr & GICH_MISR_EOI)
+		ret |= INT_STATUS_EOI;
+	if (misr & GICH_MISR_U)
+		ret |= INT_STATUS_UNDERFLOW;
+
+	return ret;
+}
+
 static const struct vgic_ops vgic_ops = {
 	.get_lr			= vgic_v2_get_lr,
 	.set_lr			= vgic_v2_set_lr,
 	.get_elrsr		= vgic_v2_get_elrsr,
 	.get_eisr		= vgic_v2_get_eisr,
+	.get_interrupt_status	= vgic_v2_get_interrupt_status,
 };
 
 static inline struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr)
@@ -1062,6 +1076,11 @@ static inline u64 vgic_get_eisr(struct kvm_vcpu *vcpu)
 	return vgic_ops.get_eisr(vcpu);
 }
 
+static inline u32 vgic_get_interrupt_status(struct kvm_vcpu *vcpu)
+{
+	return vgic_ops.get_interrupt_status(vcpu);
+}
+
 static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
@@ -1259,11 +1278,12 @@ epilog:
 static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	u32 status = vgic_get_interrupt_status(vcpu);
 	bool level_pending = false;
 
-	kvm_debug("MISR = %08x\n", vgic_cpu->vgic_v2.vgic_misr);
+	kvm_debug("STATUS = %08x\n", status);
 
-	if (vgic_cpu->vgic_v2.vgic_misr & GICH_MISR_EOI) {
+	if (status & INT_STATUS_EOI) {
 		/*
 		 * Some level interrupts have been EOIed. Clear their
 		 * active bit.
@@ -1290,7 +1310,7 @@ static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
 		}
 	}
 
-	if (vgic_cpu->vgic_v2.vgic_misr & GICH_MISR_U)
+	if (status & INT_STATUS_UNDERFLOW)
 		vgic_cpu->vgic_v2.vgic_hcr &= ~GICH_HCR_UIE;
 
 	return level_pending;
Signature · 1 line
1.8.3.4
#33 lore raw

PATCH 9/18 KVM: ARM: vgic: move underflow handling to vgic_ops

In reply to Marc Zyngier

Move the code dealing with LR underflow handling to its own functions, and make them accessible through vgic_ops.

---
 include/kvm/arm_vgic.h |  2 ++
 virt/kvm/arm/vgic.c    | 28 +++++++++++++++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)
include/kvm/arm_vgic.h +2 -0
index 95085a6..407e104 100644
@@ -87,6 +87,8 @@ struct vgic_ops {
 	u64	(*get_elrsr)(const struct kvm_vcpu *vcpu);
 	u64	(*get_eisr)(const struct kvm_vcpu *vcpu);
 	u32	(*get_interrupt_status)(const struct kvm_vcpu *vcpu);
+	void	(*set_underflow)(struct kvm_vcpu *vcpu);
+	void	(*clear_underflow)(struct kvm_vcpu *vcpu);
 };
 
 struct vgic_dist {
virt/kvm/arm/vgic.c +25 -3
index 22423dd..fe85740 100644
@@ -1047,12 +1047,24 @@ static u32 vgic_v2_get_interrupt_status(const struct kvm_vcpu *vcpu)
 	return ret;
 }
 
+static void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr |= GICH_HCR_UIE;
+}
+
+static void vgic_v2_clear_underflow(struct kvm_vcpu *vcpu)
+{
+	vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr &= ~GICH_HCR_UIE;
+}
+
 static const struct vgic_ops vgic_ops = {
 	.get_lr			= vgic_v2_get_lr,
 	.set_lr			= vgic_v2_set_lr,
 	.get_elrsr		= vgic_v2_get_elrsr,
 	.get_eisr		= vgic_v2_get_eisr,
 	.get_interrupt_status	= vgic_v2_get_interrupt_status,
+	.set_underflow		= vgic_v2_set_underflow,
+	.clear_underflow	= vgic_v2_clear_underflow,
 };
 
 static inline struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr)
@@ -1081,6 +1093,16 @@ static inline u32 vgic_get_interrupt_status(struct kvm_vcpu *vcpu)
 	return vgic_ops.get_interrupt_status(vcpu);
 }
 
+static inline void vgic_set_underflow(struct kvm_vcpu *vcpu)
+{
+	vgic_ops.set_underflow(vcpu);
+}
+
+static inline void vgic_clear_underflow(struct kvm_vcpu *vcpu)
+{
+	vgic_ops.clear_underflow(vcpu);
+}
+
 static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
@@ -1262,9 +1284,9 @@ static void __kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 
 epilog:
 	if (overflow) {
-		vgic_cpu->vgic_v2.vgic_hcr |= GICH_HCR_UIE;
+		vgic_set_underflow(vcpu);
 	} else {
-		vgic_cpu->vgic_v2.vgic_hcr &= ~GICH_HCR_UIE;
+		vgic_clear_underflow(vcpu);
 		/*
 		 * We're about to run this VCPU, and we've consumed
 		 * everything the distributor had in store for
@@ -1311,7 +1333,7 @@ static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
 	}
 
 	if (status & INT_STATUS_UNDERFLOW)
-		vgic_cpu->vgic_v2.vgic_hcr &= ~GICH_HCR_UIE;
+		vgic_clear_underflow(vcpu);
 
 	return level_pending;
 }
Signature · 1 line
1.8.3.4
#34 lore raw

PATCH 10/18 KVM: ARM: vgic: abstract VMCR access

In reply to Marc Zyngier

Instead of directly messing with with the GICH_VMCR bits for the CPU interface save/restore code, add accessors that encode/decode the entire set of registers exposed by VMCR.

Not the most efficient thing, but given that this code is only used by the save/restore code, performance is far from being critical.

---
 include/kvm/arm_vgic.h |  9 +++++++
 virt/kvm/arm/vgic.c    | 69 ++++++++++++++++++++++++++++++++++++++------------
 2 files changed, 62 insertions(+), 16 deletions(-)
include/kvm/arm_vgic.h +9 -0
index 407e104..ee6fa93 100644
@@ -81,6 +81,13 @@ struct vgic_lr {
 	u8	state;
 };
 
+struct vgic_vmcr {
+	u32	ctlr;
+	u32	abpr;
+	u32	bpr;
+	u32	pmr;
+};
+
 struct vgic_ops {
 	struct vgic_lr	(*get_lr)(const struct kvm_vcpu *, int);
 	void	(*set_lr)(struct kvm_vcpu *, int, struct vgic_lr);
@@ -89,6 +96,8 @@ struct vgic_ops {
 	u32	(*get_interrupt_status)(const struct kvm_vcpu *vcpu);
 	void	(*set_underflow)(struct kvm_vcpu *vcpu);
 	void	(*clear_underflow)(struct kvm_vcpu *vcpu);
+	void	(*get_vmcr)(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
+	void	(*set_vmcr)(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
 };
 
 struct vgic_dist {
virt/kvm/arm/vgic.c +53 -16

118 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#35 lore raw

PATCH 11/18 KVM: ARM: vgic: introduce vgic_enable

In reply to Marc Zyngier

Move the code dealing with kicking the VGIC on to vgic_ops.

---
 include/kvm/arm_vgic.h |  1 +
 virt/kvm/arm/vgic.c    | 29 +++++++++++++++++++++--------
 2 files changed, 22 insertions(+), 8 deletions(-)
include/kvm/arm_vgic.h +1 -0
index ee6fa93..46a42b1 100644
@@ -98,6 +98,7 @@ struct vgic_ops {
 	void	(*clear_underflow)(struct kvm_vcpu *vcpu);
 	void	(*get_vmcr)(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
 	void	(*set_vmcr)(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
+	void	(*enable)(struct kvm_vcpu *vcpu);
 };
 
 struct vgic_dist {
virt/kvm/arm/vgic.c +21 -8
index 574ca47..a6d70fc 100644
@@ -1081,6 +1081,19 @@ static void vgic_v2_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
 	vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = vmcr;
 }
 
+static void vgic_v2_enable(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * By forcing VMCR to zero, the GIC will restore the binary
+	 * points to their reset values. Anything else resets to zero
+	 * anyway.
+	 */
+	vcpu->arch.vgic_cpu.vgic_v2.vgic_vmcr = 0;
+
+	/* Get the show on the road... */
+	vcpu->arch.vgic_cpu.vgic_v2.vgic_hcr = GICH_HCR_EN;
+}
+
 static const struct vgic_ops vgic_ops = {
 	.get_lr			= vgic_v2_get_lr,
 	.set_lr			= vgic_v2_set_lr,
@@ -1091,6 +1104,7 @@ static const struct vgic_ops vgic_ops = {
 	.clear_underflow	= vgic_v2_clear_underflow,
 	.get_vmcr		= vgic_v2_get_vmcr,
 	.set_vmcr		= vgic_v2_set_vmcr,
+	.enable			= vgic_v2_enable,
 };
 
 static inline struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr)
@@ -1139,6 +1153,11 @@ static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
 	vgic_ops.set_vmcr(vcpu, vmcr);
 }
 
+static inline void vgic_enable(struct kvm_vcpu *vcpu)
+{
+	vgic_ops.enable(vcpu);
+}
+
 static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
@@ -1593,15 +1612,9 @@ int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
 		vgic_cpu->vgic_irq_lr_map[i] = LR_EMPTY;
 	}
 
-	/*
-	 * By forcing VMCR to zero, the GIC will restore the binary
-	 * points to their reset values. Anything else resets to zero
-	 * anyway.
-	 */
-	vgic_cpu->vgic_v2.vgic_vmcr = 0;
-
 	vgic_cpu->nr_lr = vgic_nr_lr;
-	vgic_cpu->vgic_v2.vgic_hcr = GICH_HCR_EN; /* Get the show on the road... */
+
+	vgic_enable(vcpu);
 
 	return 0;
 }
Signature · 1 line
1.8.3.4
#36 lore raw

PATCH 12/18 KVM: ARM: introduce vgic_params structure

In reply to Marc Zyngier

Move all the data specific to a given GIC implementation into its own little structure.

---
 include/kvm/arm_vgic.h | 11 +++++++++
 virt/kvm/arm/vgic.c    | 66 +++++++++++++++++++++-----------------------------
 2 files changed, 39 insertions(+), 38 deletions(-)
include/kvm/arm_vgic.h +11 -0
index 46a42b1..48dbca1 100644
@@ -101,6 +101,17 @@ struct vgic_ops {
 	void	(*enable)(struct kvm_vcpu *vcpu);
 };
 
+struct vgic_params {
+	/* Physical address of vgic virtual cpu interface */
+	phys_addr_t	vcpu_base;
+	/* Number of list registers */
+	u32		nr_lr;
+	/* Interrupt number */
+	unsigned int	maint_irq;
+	/* Virtual control interface base address */
+	void __iomem	*vctrl_base;
+};
+
 struct vgic_dist {
 #ifdef CONFIG_KVM_ARM_VGIC
 	spinlock_t		lock;
virt/kvm/arm/vgic.c +28 -38

192 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#37 lore raw

PATCH 13/18 KVM: ARM: vgic: split GICv2 backend from the main vgic code

In reply to Marc Zyngier

Brutally hack the innocent vgic code, and move the GICv2 specific code to its own file, using vgic_ops and vgic_params as a way to pass information between the two blocks.

---
 arch/arm/kvm/Makefile   |   1 +
 arch/arm64/kvm/Makefile |   2 +-
 include/kvm/arm_vgic.h  |   3 +
 virt/kvm/arm/vgic-v2.c  | 229 ++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.c     | 239 +++++++-----------------------------------------
 5 files changed, 266 insertions(+), 208 deletions(-)
 create mode 100644 virt/kvm/arm/vgic-v2.c
arch/arm/kvm/Makefile +1 -0
index 789bca9..f7057ed 100644
@@ -21,4 +21,5 @@ obj-y += kvm-arm.o init.o interrupts.o
 obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o
 obj-y += coproc.o coproc_a15.o coproc_a7.o mmio.o psci.o perf.o
 obj-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
+obj-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2.o
 obj-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
arch/arm64/kvm/Makefile +1 -1
index 72a9fd5..7e92952 100644
@@ -19,5 +19,5 @@ kvm-$(CONFIG_KVM_ARM_HOST) += emulate.o inject_fault.o regmap.o
 kvm-$(CONFIG_KVM_ARM_HOST) += hyp.o hyp-init.o handle_exit.o
 kvm-$(CONFIG_KVM_ARM_HOST) += guest.o reset.o sys_regs.o sys_regs_generic_v8.o
 
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
+kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o $(KVM)/arm/vgic-v2.o
 kvm-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
include/kvm/arm_vgic.h +3 -0
index 48dbca1..21eea24 100644
@@ -213,6 +213,9 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
 #define irqchip_in_kernel(k)	(!!((k)->arch.vgic.vctrl_base))
 #define vgic_initialized(k)	((k)->arch.vgic.ready)
 
+int vgic_v2_probe(const struct vgic_ops **ops,
+		  const struct vgic_params **params);
+
 #else
 static inline int kvm_vgic_hyp_init(void)
 {
virt/kvm/arm/vgic-v2.c new file +229 -0

229 lines not on this page · raw message

virt/kvm/arm/vgic.c +32 -207

370 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#38 lore raw

PATCH 14/18 arm64: KVM: remove __kvm_hyp_code_{start, end} from hyp.S

In reply to Marc Zyngier

We already have __hyp_text_{start,end} to express the boundaries of the HYP text section, and __kvm_hyp_code_{start,end} are getting in the way of a more modular world switch code.

Just turn __kvm_hyp_code_{start,end} into #defines mapping the linker-emited symbols.

---
 arch/arm64/include/asm/kvm_asm.h | 6 ++++--
 arch/arm64/include/asm/virt.h    | 4 ++++
 arch/arm64/kvm/hyp.S             | 6 ------
 3 files changed, 8 insertions(+), 8 deletions(-)
arch/arm64/include/asm/kvm_asm.h +4 -2
index b25763b..dddb345 100644
@@ -18,6 +18,8 @@
 #ifndef __ARM_KVM_ASM_H__
 #define __ARM_KVM_ASM_H__
 
+#include <asm/virt.h>
+
 /*
  * 0 is reserved as an invalid value.
  * Order *must* be kept in sync with the hyp switch code.
@@ -95,8 +97,8 @@ extern char __kvm_hyp_init_end[];
 
 extern char __kvm_hyp_vector[];
 
-extern char __kvm_hyp_code_start[];
-extern char __kvm_hyp_code_end[];
+#define	__kvm_hyp_code_start	__hyp_text_start
+#define	__kvm_hyp_code_end	__hyp_text_end
 
 extern void __kvm_flush_vm_context(void);
 extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
arch/arm64/include/asm/virt.h +4 -0
index 130e2be..290fb66 100644
@@ -63,6 +63,10 @@ static inline bool is_hyp_mode_mismatched(void)
 	return __boot_cpu_mode[0] != __boot_cpu_mode[1];
 }
 
+/* The section containing the hypervisor text */
+extern char __hyp_text_start[];
+extern char __hyp_text_end[];
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* ! __ASM__VIRT_H */
arch/arm64/kvm/hyp.S +0 -6
index c4c3a18..b57f286 100644
@@ -36,9 +36,6 @@
 	.pushsection	.hyp.text, "ax"
 	.align	PAGE_SHIFT
 
-__kvm_hyp_code_start:
-	.globl __kvm_hyp_code_start
-
 .macro save_common_regs
 	// x2: base address for cpu context
 	// x3: tmp register
@@ -851,7 +848,4 @@ ENTRY(__kvm_hyp_vector)
 	ventry	el1_error_invalid		// Error 32-bit EL1
 ENDPROC(__kvm_hyp_vector)
 
-__kvm_hyp_code_end:
-	.globl	__kvm_hyp_code_end
-
 	.popsection
Signature · 1 line
1.8.3.4
#39 lore raw

PATCH 15/18 arm64: KVM: split GICv2 world switch from hyp code

In reply to Marc Zyngier

Move the GICv2 world switch code into its own file, and add the necessary indirection to the arm64 switch code.

Also introduce a new type field to the vgic_params structure.

---
 arch/arm/include/asm/kvm_host.h   |   5 ++
 arch/arm64/include/asm/kvm_asm.h  |   8 +++
 arch/arm64/include/asm/kvm_host.h |  16 +++++
 arch/arm64/kvm/Makefile           |   4 +-
 arch/arm64/kvm/hyp.S              | 119 +++++---------------------------
 arch/arm64/kvm/vgic-v2-switch.S   | 141 ++++++++++++++++++++++++++++++++++++++
 include/kvm/arm_vgic.h            |   7 +-
 virt/kvm/arm/vgic-v2.c            |  15 ++--
 virt/kvm/arm/vgic.c               |   3 +
 9 files changed, 208 insertions(+), 110 deletions(-)
 create mode 100644 arch/arm64/kvm/vgic-v2-switch.S
arch/arm/include/asm/kvm_host.h +5 -0
index 098f7dd..228ae1c 100644
@@ -222,6 +222,11 @@ static inline int kvm_arch_dev_ioctl_check_extension(long ext)
 	return 0;
 }
 
+static inline void vgic_arch_setup(const struct vgic_params *vgic)
+{
+	BUG_ON(vgic->type != VGIC_V2);
+}
+
 int kvm_perf_init(void);
 int kvm_perf_teardown(void);
 
arch/arm64/include/asm/kvm_asm.h +8 -0
index dddb345..e5fbdce 100644
@@ -104,6 +104,14 @@ extern void __kvm_flush_vm_context(void);
 extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
 
 extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
+/*
+ * These are really code, but as they do not use the PCS we don't
+ * describe them as functions...
+ */
+extern char __save_vgic_v2_state[];
+extern char __restore_vgic_v2_state[];
+
 #endif
 
 #endif /* __ARM_KVM_ASM_H__ */
arch/arm64/include/asm/kvm_host.h +16 -0
index 0a1d697..b562d67 100644
@@ -200,4 +200,20 @@ static inline void __cpu_init_hyp_mode(phys_addr_t boot_pgd_ptr,
 		     hyp_stack_ptr, vector_ptr);
 }
 
+static inline void vgic_arch_setup(const struct vgic_params *vgic)
+{
+	extern char *__vgic_sr_vectors[2];
+
+	switch(vgic->type)
+	{
+	case VGIC_V2:
+		__vgic_sr_vectors[0] = __save_vgic_v2_state;
+		__vgic_sr_vectors[1] = __restore_vgic_v2_state;
+		break;
+
+	default:
+		BUG();
+	}
+}
+
 #endif /* __ARM64_KVM_HOST_H__ */
arch/arm64/kvm/Makefile +3 -1
index 7e92952..daf24dc 100644
@@ -19,5 +19,7 @@ kvm-$(CONFIG_KVM_ARM_HOST) += emulate.o inject_fault.o regmap.o
 kvm-$(CONFIG_KVM_ARM_HOST) += hyp.o hyp-init.o handle_exit.o
 kvm-$(CONFIG_KVM_ARM_HOST) += guest.o reset.o sys_regs.o sys_regs_generic_v8.o
 
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o $(KVM)/arm/vgic-v2.o
+kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
+kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2.o
+kvm-$(CONFIG_KVM_ARM_VGIC) += vgic-v2-switch.o
 kvm-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
arch/arm64/kvm/hyp.S +19 -100

155 lines not on this page · raw message

arch/arm64/kvm/vgic-v2-switch.S new file +141 -0

141 lines not on this page · raw message

include/kvm/arm_vgic.h +6 -1
index 21eea24..c95039a 100644
@@ -24,7 +24,6 @@
 #include <linux/irqreturn.h>
 #include <linux/spinlock.h>
 #include <linux/types.h>
-#include <linux/irqchip/arm-gic.h>
 
 #define VGIC_NR_IRQS		256
 #define VGIC_NR_SGIS		16
@@ -70,6 +69,10 @@ struct vgic_bytemap {
 
 struct kvm_vcpu;
 
+enum vgic_type {
+	VGIC_V2,		/* Good ol' GICv2 */
+};
+
 #define LR_STATE_PENDING	(1 << 0)
 #define LR_STATE_ACTIVE		(1 << 1)
 #define LR_STATE_MASK		(3 << 0)
@@ -102,6 +105,8 @@ struct vgic_ops {
 };
 
 struct vgic_params {
+	/* vgic type */
+	enum vgic_type	type;
 	/* Physical address of vgic virtual cpu interface */
 	phys_addr_t	vcpu_base;
 	/* Number of list registers */
virt/kvm/arm/vgic-v2.c +7 -8
index 52f438f..4c6606e 100644
@@ -170,7 +170,7 @@ int vgic_v2_probe(const struct vgic_ops **ops,
 
 	vgic_node = of_find_compatible_node(NULL, NULL, "arm,cortex-a15-gic");
 	if (!vgic_node) {
-		kvm_err("error: no compatible vgic node in DT\n");
+		kvm_err("error: no compatible GICv2 node in DT\n");
 		return -ENODEV;
 	}
 
@@ -183,15 +183,15 @@ int vgic_v2_probe(const struct vgic_ops **ops,
 
 	ret = of_address_to_resource(vgic_node, 2, &vctrl_res);
 	if (ret) {
-		kvm_err("Cannot obtain VCTRL resource\n");
-		goto out_free_irq;
+		kvm_err("Cannot obtain GICH resource\n");
+		goto out;
 	}
 
 	vgic->vctrl_base = of_iomap(vgic_node, 2);
 	if (!vgic->vctrl_base) {
-		kvm_err("Cannot ioremap VCTRL\n");
+		kvm_err("Cannot ioremap GICH\n");
 		ret = -ENOMEM;
-		goto out_free_irq;
+		goto out;
 	}
 
 	vgic->nr_lr = readl_relaxed(vgic->vctrl_base + GICH_VTR);
@@ -206,7 +206,7 @@ int vgic_v2_probe(const struct vgic_ops **ops,
 	}
 
 	if (of_address_to_resource(vgic_node, 3, &vcpu_res)) {
-		kvm_err("Cannot obtain VCPU resource\n");
+		kvm_err("Cannot obtain GICV resource\n");
 		ret = -ENXIO;
 		goto out_unmap;
 	}
@@ -215,14 +215,13 @@ int vgic_v2_probe(const struct vgic_ops **ops,
 	kvm_info("%s@%llx IRQ%d\n", vgic_node->name,
 		 vctrl_res.start, vgic->maint_irq);
 
+	vgic->type = VGIC_V2;
 	*ops = &vgic_v2_ops;
 	*params = vgic;
 	goto out;
 
 out_unmap:
 	iounmap(vgic->vctrl_base);
-out_free_irq:
-	free_percpu_irq(vgic->maint_irq, kvm_get_running_vcpus());
 out:
 	of_node_put(vgic_node);
 	return ret;
virt/kvm/arm/vgic.c +3 -0
index 613b492..8365189 100644
@@ -1531,6 +1531,9 @@ int kvm_vgic_hyp_init(void)
 
 	on_each_cpu(vgic_init_maintenance_interrupt, NULL, 1);
 
+	/* Callback into for arch code for setup */
+	vgic_arch_setup(vgic);
+
 	return 0;
 
 out_free_irq:
Signature · 1 line
1.8.3.4
#40 lore raw

PATCH 15/18 arm64: KVM: split GICv2 world switch from hyp code

Quoted 41 lines · Marc Zyngier

Move the GICv2 world switch code into its own file, and add the necessary indirection to the arm64 switch code.

Also introduce a new type field to the vgic_params structure.

---
 arch/arm/include/asm/kvm_host.h   |   5 ++
 arch/arm64/include/asm/kvm_asm.h  |   8 +++
 arch/arm64/include/asm/kvm_host.h |  16 +++++
 arch/arm64/kvm/Makefile           |   4 +-
 arch/arm64/kvm/hyp.S              | 119 +++++---------------------------
 arch/arm64/kvm/vgic-v2-switch.S   | 141 ++++++++++++++++++++++++++++++++++++++
 include/kvm/arm_vgic.h            |   7 +-
 virt/kvm/arm/vgic-v2.c            |  15 ++--
 virt/kvm/arm/vgic.c               |   3 +
 9 files changed, 208 insertions(+), 110 deletions(-)
 create mode 100644 arch/arm64/kvm/vgic-v2-switch.S
arch/arm/include/asm/kvm_host.h +5 -0
index 098f7dd..228ae1c 100644
@@ -222,6 +222,11 @@ static inline int kvm_arch_dev_ioctl_check_extension(long ext)
        return 0;
 }
 
+static inline void vgic_arch_setup(const struct vgic_params *vgic)
+{
+       BUG_ON(vgic->type != VGIC_V2);
+}
+
 int kvm_perf_init(void);
 int kvm_perf_teardown(void);
 
arch/arm64/include/asm/kvm_asm.h +0 -0
index dddb345..e5fbdce 100644

arch/arm64/include/asm/kvm_asm.h extern void __kvm_flush_vm_context(void); · line 111

Quoted 8 lines
arch/arm64/include/asm/kvm_asm.h +5 -0
@@ -104,6 +104,14 @@ extern void __kvm_flush_vm_context(void);
 extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
 
 extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
+/*
+ * These are really code, but as they do not use the PCS we don't
+ * describe them as functions...
+ */

I don't understand what the PCS has to do with anything.

Quoted 13 lines
arch/arm64/include/asm/kvm_asm.h +3 -0
+extern char __save_vgic_v2_state[];
+extern char __restore_vgic_v2_state[];
+
 #endif
 
 #endif /* __ARM_KVM_ASM_H__ */
arch/arm64/include/asm/kvm_host.h +0 -0
index 0a1d697..b562d67 100644
@@ -200,4 +200,20 @@ static inline void __cpu_init_hyp_mode(phys_addr_t boot_pgd_ptr,
                     hyp_stack_ptr, vector_ptr);
 }

arch/arm64/include/asm/kvm_host.h static inline void __cpu_init_hyp_mode(phys_addr_t boot_pgd_ptr, · line 211

Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +9 -0
 
+static inline void vgic_arch_setup(const struct vgic_params *vgic)
+{
+       extern char *__vgic_sr_vectors[2];
+
+       switch(vgic->type)
+       {
+       case VGIC_V2:
+               __vgic_sr_vectors[0] = __save_vgic_v2_state;
+               __vgic_sr_vectors[1] = __restore_vgic_v2_state;

If you make this array a struct then you can use asm_offsets to generate indices for your asm.

Will

#41 lore raw

PATCH 16/18 arm64: KVM: move hcr_el2 setting into vgic-v2-switch.S

In reply to Marc Zyngier

GICv3 requires the hcr_el2 switch to be tightly coupled with some of the interrupt controller's register switch.

In order to have similar code paths, start moving the hcr_el2 manipulation code to the GICv2 switch code.

---
 arch/arm64/kvm/hyp.S            | 7 -------
 arch/arm64/kvm/vgic-v2-switch.S | 8 ++++++++
 2 files changed, 8 insertions(+), 7 deletions(-)
arch/arm64/kvm/hyp.S +0 -7
index 5a868c0..20a58fe 100644
@@ -335,11 +335,6 @@
 .endm
 
 .macro activate_traps
-	ldr	x2, [x0, #VCPU_IRQ_LINES]
-	ldr	x1, [x0, #VCPU_HCR_EL2]
-	orr	x2, x2, x1
-	msr	hcr_el2, x2
-
 	ldr	x2, =(CPTR_EL2_TTA)
 	msr	cptr_el2, x2
 
@@ -353,8 +348,6 @@
 .endm
 
 .macro deactivate_traps
-	mov	x2, #HCR_RW
-	msr	hcr_el2, x2
 	msr	cptr_el2, xzr
 	msr	hstr_el2, xzr
 
arch/arm64/kvm/vgic-v2-switch.S +8 -0
index c5dc777..d36cd7a 100644
@@ -85,6 +85,9 @@ CPU_BE(	rev	w5, w5 )
 	sub	w4, w4, #1
 	cbnz	w4, 1b
 2:
+	mov	x2, #HCR_RW
+	msr	hcr_el2, x2
+	isb
 .endm
 
 /*
@@ -92,6 +95,11 @@ CPU_BE(	rev	w5, w5 )
  * x0: Register pointing to VCPU struct
  */
 .macro restore_vgic_v2_state
+	ldr	x2, [x0, #VCPU_IRQ_LINES]
+	ldr	x1, [x0, #VCPU_HCR_EL2]
+	orr	x2, x2, x1
+	msr	hcr_el2, x2
+
 	/* Get VGIC VCTRL base into x2 */
 	ldr	x2, [x0, #VCPU_KVM]
 	kern_hyp_va	x2
Signature · 1 line
1.8.3.4
#42 lore raw

PATCH 17/18 KVM: ARM: vgic: add the GICv3 backend

In reply to Marc Zyngier

Introduce the support code for emulating a GICv2 on top of GICv3 hardware.

---
 include/kvm/arm_vgic.h |  26 ++++++
 virt/kvm/arm/vgic-v3.c | 220 +++++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.c    |   2 +
 3 files changed, 248 insertions(+)
 create mode 100644 virt/kvm/arm/vgic-v3.c
include/kvm/arm_vgic.h +26 -0
index c95039a..caeb8f4 100644
@@ -32,6 +32,7 @@
 #define VGIC_NR_SHARED_IRQS	(VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
 #define VGIC_MAX_CPUS		KVM_MAX_VCPUS
 #define VGIC_MAX_LRS		(1 << 6)
+#define VGIC_V3_MAX_LRS		16
 
 /* Sanity checks... */
 #if (VGIC_MAX_CPUS > 8)
@@ -71,6 +72,7 @@ struct kvm_vcpu;
 
 enum vgic_type {
 	VGIC_V2,		/* Good ol' GICv2 */
+	VGIC_V3,		/* v2 on v3, really */
 };
 
 #define LR_STATE_PENDING	(1 << 0)
@@ -169,6 +171,19 @@ struct vgic_v2_cpu_if {
 	u32		vgic_lr[VGIC_MAX_LRS];
 };
 
+struct vgic_v3_cpu_if {
+#ifdef CONFIG_ARM_GIC_V3
+	u32		vgic_hcr;
+	u32		vgic_vmcr;
+	u32		vgic_misr;	/* Saved only */
+	u32		vgic_eisr;	/* Saved only */
+	u32		vgic_elrsr;	/* Saved only */
+	u32		vgic_ap0r[4];
+	u32		vgic_ap1r[4];
+	u64		vgic_lr[VGIC_V3_MAX_LRS];
+#endif
+};
+
 struct vgic_cpu {
 #ifdef CONFIG_KVM_ARM_VGIC
 	/* per IRQ to LR mapping */
@@ -187,6 +202,7 @@ struct vgic_cpu {
 	/* CPU vif control registers for world switch */
 	union {
 		struct vgic_v2_cpu_if	vgic_v2;
+		struct vgic_v3_cpu_if	vgic_v3;
 	};
 #endif
 };
@@ -220,6 +236,16 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
 
 int vgic_v2_probe(const struct vgic_ops **ops,
 		  const struct vgic_params **params);
+#ifdef CONFIG_ARM_GIC_V3
+int vgic_v3_probe(const struct vgic_ops **ops,
+		  const struct vgic_params **params);
+#else
+static inline int vgic_v3_probe(const struct vgic_ops **ops,
+				const struct vgic_params **params)
+{
+	return -ENODEV;
+}
+#endif
 
 #else
 static inline int kvm_vgic_hyp_init(void)
virt/kvm/arm/vgic-v3.c new file +220 -0

220 lines not on this page · raw message

virt/kvm/arm/vgic.c +2 -0
index 8365189..f29761b 100644
@@ -1514,6 +1514,8 @@ int kvm_vgic_hyp_init(void)
 
 	ret = vgic_v2_probe(&vgic_ops, &vgic);
 	if (ret)
+		ret = vgic_v3_probe(&vgic_ops, &vgic);
+	if (ret)
 		return ret;
 
 	ret = request_percpu_irq(vgic->maint_irq, vgic_maintenance_handler,
Signature · 1 line
1.8.3.4
#43 lore raw

PATCH 17/18 KVM: ARM: vgic: add the GICv3 backend

Quoted 17 lines · Marc Zyngier

Introduce the support code for emulating a GICv2 on top of GICv3 hardware.

---
 include/kvm/arm_vgic.h |  26 ++++++
 virt/kvm/arm/vgic-v3.c | 220 +++++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.c    |   2 +
 3 files changed, 248 insertions(+)
 create mode 100644 virt/kvm/arm/vgic-v3.c
include/kvm/arm_vgic.h +0 -0
index c95039a..caeb8f4 100644

include/kvm/arm_vgic.h line 35

Quoted 4 lines
include/kvm/arm_vgic.h +1 -0
@@ -32,6 +32,7 @@
 #define VGIC_NR_SHARED_IRQS	(VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
 #define VGIC_MAX_CPUS		KVM_MAX_VCPUS
 #define VGIC_MAX_LRS		(1 << 6)
+#define VGIC_V3_MAX_LRS		16

Since we have fewer list registers, doesn't the code in vgic.c need updating to honour the relevant bounds? (e.g. the use of find_first_zero_bit in vgic_queue_irq).

Will

#44 lore raw

PATCH 17/18 KVM: ARM: vgic: add the GICv3 backend

Quoted 13 lines · Will Deacon
Quoted 12 lines · Marc Zyngier

Introduce the support code for emulating a GICv2 on top of GICv3 hardware.

---
 include/kvm/arm_vgic.h |  26 ++++++
 virt/kvm/arm/vgic-v3.c | 220 +++++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.c    |   2 +
 3 files changed, 248 insertions(+)
 create mode 100644 virt/kvm/arm/vgic-v3.c
Quoted 13 lines

include/kvm/arm_vgic.h line 35

Quoted 9 lines
include/kvm/arm_vgic.h +1 -0
index c95039a..caeb8f4 100644
@@ -32,6 +32,7 @@
 #define VGIC_NR_SHARED_IRQS	(VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
 #define VGIC_MAX_CPUS		KVM_MAX_VCPUS
 #define VGIC_MAX_LRS		(1 << 6)
+#define VGIC_V3_MAX_LRS		16

Since we have fewer list registers, doesn't the code in vgic.c need updating to honour the relevant bounds? (e.g. the use of find_first_zero_bit in vgic_queue_irq).

I'm confused. We've always used a variable, as even with GICv2, we usually have far less list registers than the maximum (4 vs 64). Looking at the code you mention, I see this:

	/* Try to use another LR for this interrupt */
	lr = find_first_zero_bit((unsigned long *)vgic_cpu->lr_used,
			       vgic->nr_lr);

Am I looking at the wrong place?

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#45 lore raw

PATCH 17/18 KVM: ARM: vgic: add the GICv3 backend

Quoted 32 lines · Marc Zyngier
Quoted 13 lines · Will Deacon
Quoted 12 lines · Marc Zyngier

Introduce the support code for emulating a GICv2 on top of GICv3 hardware.

---
 include/kvm/arm_vgic.h |  26 ++++++
 virt/kvm/arm/vgic-v3.c | 220 +++++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.c    |   2 +
 3 files changed, 248 insertions(+)
 create mode 100644 virt/kvm/arm/vgic-v3.c
Quoted 13 lines

include/kvm/arm_vgic.h line 35

Quoted 9 lines
include/kvm/arm_vgic.h +1 -0
index c95039a..caeb8f4 100644
@@ -32,6 +32,7 @@
 #define VGIC_NR_SHARED_IRQS	(VGIC_NR_IRQS - VGIC_NR_PRIVATE_IRQS)
 #define VGIC_MAX_CPUS		KVM_MAX_VCPUS
 #define VGIC_MAX_LRS		(1 << 6)
+#define VGIC_V3_MAX_LRS		16

Since we have fewer list registers, doesn't the code in vgic.c need updating to honour the relevant bounds? (e.g. the use of find_first_zero_bit in vgic_queue_irq).

I'm confused. We've always used a variable, as even with GICv2, we usually have far less list registers than the maximum (4 vs 64). Looking at the code you mention, I see this:

Quoted 5 lines
	/* Try to use another LR for this interrupt */
	lr = find_first_zero_bit((unsigned long *)vgic_cpu->lr_used,
			       vgic->nr_lr);

Am I looking at the wrong place?

No, I'm being stupid. I was looking at mainline rather than your GICv3 branch and thought we were still using VGIC_MAX_LRS.

Will

#46 lore raw

PATCH 18/18 arm64: KVM: vgic: add GICv3 world switch

In reply to Marc Zyngier

Introduce the GICv3 world switch code and helper functions, enabling GICv2 emulation on GICv3 hardware.

---
 arch/arm64/include/asm/kvm_asm.h  |   4 +
 arch/arm64/include/asm/kvm_host.h |   5 +
 arch/arm64/kernel/asm-offsets.c   |   8 ++
 arch/arm64/kvm/Makefile           |   2 +
 arch/arm64/kvm/vgic-v3-switch.S   | 275 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 294 insertions(+)
 create mode 100644 arch/arm64/kvm/vgic-v3-switch.S
arch/arm64/include/asm/kvm_asm.h +4 -0
index e5fbdce..3d796b4 100644
@@ -105,12 +105,16 @@ extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
 
 extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 
+extern u64 __vgic_v3_get_ich_vtr_el2(void);
+
 /*
  * These are really code, but as they do not use the PCS we don't
  * describe them as functions...
  */
 extern char __save_vgic_v2_state[];
 extern char __restore_vgic_v2_state[];
+extern char __save_vgic_v3_state[];
+extern char __restore_vgic_v3_state[];
 
 #endif
 
arch/arm64/include/asm/kvm_host.h +5 -0
index b562d67..d52ab7e 100644
@@ -211,6 +211,11 @@ static inline void vgic_arch_setup(const struct vgic_params *vgic)
 		__vgic_sr_vectors[1] = __restore_vgic_v2_state;
 		break;
 
+	case VGIC_V3:
+		__vgic_sr_vectors[0] = __save_vgic_v3_state;
+		__vgic_sr_vectors[1] = __restore_vgic_v3_state;
+		break;
+
 	default:
 		BUG();
 	}
arch/arm64/kernel/asm-offsets.c +8 -0
index 20fd488..fe8b239 100644
@@ -136,6 +136,14 @@ int main(void)
   DEFINE(VGIC_V2_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
   DEFINE(VGIC_V2_CPU_APR,	offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
   DEFINE(VGIC_V2_CPU_LR,	offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
+  DEFINE(VGIC_V3_CPU_HCR,	offsetof(struct vgic_cpu, vgic_v3.vgic_hcr));
+  DEFINE(VGIC_V3_CPU_VMCR,	offsetof(struct vgic_cpu, vgic_v3.vgic_vmcr));
+  DEFINE(VGIC_V3_CPU_MISR,	offsetof(struct vgic_cpu, vgic_v3.vgic_misr));
+  DEFINE(VGIC_V3_CPU_EISR,	offsetof(struct vgic_cpu, vgic_v3.vgic_eisr));
+  DEFINE(VGIC_V3_CPU_ELRSR,	offsetof(struct vgic_cpu, vgic_v3.vgic_elrsr));
+  DEFINE(VGIC_V3_CPU_AP0R,	offsetof(struct vgic_cpu, vgic_v3.vgic_ap0r));
+  DEFINE(VGIC_V3_CPU_AP1R,	offsetof(struct vgic_cpu, vgic_v3.vgic_ap1r));
+  DEFINE(VGIC_V3_CPU_LR,	offsetof(struct vgic_cpu, vgic_v3.vgic_lr));
   DEFINE(VGIC_CPU_NR_LR,	offsetof(struct vgic_cpu, nr_lr));
   DEFINE(KVM_VTTBR,		offsetof(struct kvm, arch.vttbr));
   DEFINE(KVM_VGIC_VCTRL,	offsetof(struct kvm, arch.vgic.vctrl_base));
arch/arm64/kvm/Makefile +2 -0
index daf24dc..32a0961 100644
@@ -22,4 +22,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += guest.o reset.o sys_regs.o sys_regs_generic_v8.o
 kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
 kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2.o
 kvm-$(CONFIG_KVM_ARM_VGIC) += vgic-v2-switch.o
+kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v3.o
+kvm-$(CONFIG_KVM_ARM_VGIC) += vgic-v3-switch.o
 kvm-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
arch/arm64/kvm/vgic-v3-switch.S new file +275 -0

275 lines not on this page · raw message

Signature · 1 line
1.8.3.4
#47 lore raw

PATCH 18/18 arm64: KVM: vgic: add GICv3 world switch

Quoted 13 lines · Marc Zyngier

Introduce the GICv3 world switch code and helper functions, enabling GICv2 emulation on GICv3 hardware.

---
 arch/arm64/include/asm/kvm_asm.h  |   4 +
 arch/arm64/include/asm/kvm_host.h |   5 +
 arch/arm64/kernel/asm-offsets.c   |   8 ++
 arch/arm64/kvm/Makefile           |   2 +
 arch/arm64/kvm/vgic-v3-switch.S   | 275 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 294 insertions(+)
 create mode 100644 arch/arm64/kvm/vgic-v3-switch.S

[...]

arch/arm64/kvm/vgic-v3-switch.S line 45

Quoted 10 lines

+.macro save_vgic_v3_state

arch/arm64/kvm/vgic-v3-switch.S new file +9 -0
+	/* Compute the address of struct vgic_cpu */
+	add	x3, x0, #VCPU_VGIC_CPU
+
+	mrs	x5, ICC_SRE_EL2
+	orr	x5, x5, #(1 << 3)
+	msr	ICC_SRE_EL2, x5
+	isb
+
+	dsb	st

Aha, so you *do* use a dsb here (see my comments to patch 1/18). Does it really need to be full-system, and does -st apply to an msr?

Saying that, why would ICC_SRE_EL2 not already be initialised by the hyp setup code?

Quoted 6 lines
arch/arm64/kvm/vgic-v3-switch.S new file +6 -0
+
+	/* Save all interesting registers */
+	mrs	x4, ICH_HCR_EL2
+	mrs	x5, ICH_VMCR_EL2
+	mrs	x6, ICH_MISR_EL2
+	mrs	x7, ICH_EISR_EL2

arch/arm64/kvm/vgic-v3-switch.S line 61

Quoted 10 lines
arch/arm64/kvm/vgic-v3-switch.S new file +10 -0
+	mrs	x8, ICH_ELSR_EL2
+
+	str	w4, [x3, #VGIC_V3_CPU_HCR]
+	str	w5, [x3, #VGIC_V3_CPU_VMCR]
+	str	w6, [x3, #VGIC_V3_CPU_MISR]
+	str	w7, [x3, #VGIC_V3_CPU_EISR]
+	str	w8, [x3, #VGIC_V3_CPU_ELRSR]
+
+	msr	ICH_HCR_EL2, xzr
+	isb

Why do you need an isb here?

Quoted 72 lines
arch/arm64/kvm/vgic-v3-switch.S new file +72 -0
+
+	mrs	x21, ICH_VTR_EL2
+	and	w22, w21, #0xf
+	mov	w23, #0xf
+	sub	w23, w23, w22	// How many regs we have to skip
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2
+	br	x24
+
+1:
+	mrs	x20, ICH_LR15_EL2
+	mrs	x19, ICH_LR14_EL2
+	mrs	x18, ICH_LR13_EL2
+	mrs	x17, ICH_LR12_EL2
+	mrs	x16, ICH_LR11_EL2
+	mrs	x15, ICH_LR10_EL2
+	mrs	x14, ICH_LR9_EL2
+	mrs	x13, ICH_LR8_EL2
+	mrs	x12, ICH_LR7_EL2
+	mrs	x11, ICH_LR6_EL2
+	mrs	x10, ICH_LR5_EL2
+	mrs	x9, ICH_LR4_EL2
+	mrs	x8, ICH_LR3_EL2
+	mrs	x7, ICH_LR2_EL2
+	mrs	x6, ICH_LR1_EL2
+	mrs	x5, ICH_LR0_EL2
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2	// adr(1f) + unimp_nr*4
+	br	x24
+
+1:
+	str	x20, [x3, #(VGIC_V3_CPU_LR + 15*8)]
+	str	x19, [x3, #(VGIC_V3_CPU_LR + 14*8)]
+	str	x18, [x3, #(VGIC_V3_CPU_LR + 13*8)]
+	str	x17, [x3, #(VGIC_V3_CPU_LR + 12*8)]
+	str	x16, [x3, #(VGIC_V3_CPU_LR + 11*8)]
+	str	x15, [x3, #(VGIC_V3_CPU_LR + 10*8)]
+	str	x14, [x3, #(VGIC_V3_CPU_LR + 9*8)]
+	str	x13, [x3, #(VGIC_V3_CPU_LR + 8*8)]
+	str	x12, [x3, #(VGIC_V3_CPU_LR + 7*8)]
+	str	x11, [x3, #(VGIC_V3_CPU_LR + 6*8)]
+	str	x10, [x3, #(VGIC_V3_CPU_LR + 5*8)]
+	str	x9, [x3, #(VGIC_V3_CPU_LR + 4*8)]
+	str	x8, [x3, #(VGIC_V3_CPU_LR + 3*8)]
+	str	x7, [x3, #(VGIC_V3_CPU_LR + 2*8)]
+	str	x6, [x3, #(VGIC_V3_CPU_LR + 1*8)]
+	str	x5, [x3, #VGIC_V3_CPU_LR]
+
+	lsr	w22, w21, #29	// Get PRIbits
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	mrs	x20, ICH_AP0R3_EL2
+	str	w20, [x3, #(VGIC_V3_CPU_AP0R + 3*4)]
+	mrs	x19, ICH_AP0R2_EL2
+	str	w19, [x3, #(VGIC_V3_CPU_AP0R + 2*4)]
+6:	mrs	x18, ICH_AP0R1_EL2
+	str	w18, [x3, #(VGIC_V3_CPU_AP0R + 1*4)]
+5:	mrs	x17, ICH_AP0R0_EL2
+	str	w17, [x3, #VGIC_V3_CPU_AP0R]
+
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	mrs	x20, ICH_AP1R3_EL2
+	str	w20, [x3, #(VGIC_V3_CPU_AP1R + 3*4)]

arch/arm64/kvm/vgic-v3-switch.S line 143

Quoted 10 lines
arch/arm64/kvm/vgic-v3-switch.S new file +10 -0
+	mrs	x19, ICH_AP1R2_EL2
+	str	w19, [x3, #(VGIC_V3_CPU_AP1R + 2*4)]
+6:	mrs	x18, ICH_AP1R1_EL2
+	str	w18, [x3, #(VGIC_V3_CPU_AP1R + 1*4)]
+5:	mrs	x17, ICH_AP1R0_EL2
+	str	w17, [x3, #VGIC_V3_CPU_AP1R]
+
+	mov	x2, #HCR_RW
+	msr	hcr_el2, x2
+	isb

Same here -- HCR_EL2.RW is for lower exception levels.

Quoted 9 lines
arch/arm64/kvm/vgic-v3-switch.S new file +9 -0
+
+	// Restore SRE_EL1 access
+	mov	x5, #1
+	msr	ICC_SRE_EL1, x5
+.endm
+
+/*
+ * Restore the VGIC CPU state from memory
+ * x0: Register pointing to VCPU struct

arch/arm64/kvm/vgic-v3-switch.S line 162

Quoted 10 lines
arch/arm64/kvm/vgic-v3-switch.S new file +10 -0
+ */
+.macro	restore_vgic_v3_state
+	ldr	x2, [x0, #VCPU_IRQ_LINES]
+	ldr	x1, [x0, #VCPU_HCR_EL2]
+	orr	x2, x2, x1
+	msr	hcr_el2, x2
+
+	// Disable SRE_EL1 access
+	msr	ICC_SRE_EL1, xzr
+	isb

We're executing at EL2, why the isb?

Quoted 81 lines
arch/arm64/kvm/vgic-v3-switch.S new file +81 -0
+
+	/* Compute the address of struct vgic_cpu */
+	add	x3, x0, #VCPU_VGIC_CPU
+
+	/* Restore all interesting registers */
+	ldr	w4, [x3, #VGIC_V3_CPU_HCR]
+	ldr	w5, [x3, #VGIC_V3_CPU_VMCR]
+
+	msr	ICH_HCR_EL2, x4
+	msr	ICH_VMCR_EL2, x5
+
+	mrs	x21, ICH_VTR_EL2
+
+	lsr	w22, w21, #29	// Get PRIbits
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	ldr	w20, [x3, #(VGIC_V3_CPU_AP1R + 3*4)]
+	msr	ICH_AP1R3_EL2, x20
+	ldr	w19, [x3, #(VGIC_V3_CPU_AP1R + 2*4)]
+	msr	ICH_AP1R2_EL2, x19
+6:	ldr	w18, [x3, #(VGIC_V3_CPU_AP1R + 1*4)]
+	msr	ICH_AP1R1_EL2, x18
+5:	ldr	w17, [x3, #VGIC_V3_CPU_AP1R]
+	msr	ICH_AP1R0_EL2, x17
+
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	ldr	w20, [x3, #(VGIC_V3_CPU_AP0R + 3*4)]
+	msr	ICH_AP0R3_EL2, x20
+	ldr	w19, [x3, #(VGIC_V3_CPU_AP0R + 2*4)]
+	msr	ICH_AP0R2_EL2, x19
+6:	ldr	w18, [x3, #(VGIC_V3_CPU_AP0R + 1*4)]
+	msr	ICH_AP0R1_EL2, x18
+5:	ldr	w17, [x3, #VGIC_V3_CPU_AP0R]
+	msr	ICH_AP0R0_EL2, x17
+
+	and	w22, w21, #0xf
+	mov	w23, #0xf
+	sub	w23, w23, w22	// How many regs we have to skip
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2	// adr(1f) + unimp_nr*4
+	br	x24
+
+1:
+	ldr	x20, [x3, #(VGIC_V3_CPU_LR + 15*8)]
+	ldr	x19, [x3, #(VGIC_V3_CPU_LR + 14*8)]
+	ldr	x18, [x3, #(VGIC_V3_CPU_LR + 13*8)]
+	ldr	x17, [x3, #(VGIC_V3_CPU_LR + 12*8)]
+	ldr	x16, [x3, #(VGIC_V3_CPU_LR + 11*8)]
+	ldr	x15, [x3, #(VGIC_V3_CPU_LR + 10*8)]
+	ldr	x14, [x3, #(VGIC_V3_CPU_LR + 9*8)]
+	ldr	x13, [x3, #(VGIC_V3_CPU_LR + 8*8)]
+	ldr	x12, [x3, #(VGIC_V3_CPU_LR + 7*8)]
+	ldr	x11, [x3, #(VGIC_V3_CPU_LR + 6*8)]
+	ldr	x10, [x3, #(VGIC_V3_CPU_LR + 5*8)]
+	ldr	x9, [x3, #(VGIC_V3_CPU_LR + 4*8)]
+	ldr	x8, [x3, #(VGIC_V3_CPU_LR + 3*8)]
+	ldr	x7, [x3, #(VGIC_V3_CPU_LR + 2*8)]
+	ldr	x6, [x3, #(VGIC_V3_CPU_LR + 1*8)]
+	ldr	x5, [x3, #VGIC_V3_CPU_LR]
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2
+	br	x24
+
+1:
+	msr	ICH_LR15_EL2, x20
+	msr	ICH_LR14_EL2, x19
+	msr	ICH_LR13_EL2, x18
+	msr	ICH_LR12_EL2, x17
+	msr	ICH_LR11_EL2, x16
+	msr	ICH_LR10_EL2, x15
+	msr	ICH_LR9_EL2,  x14
+	msr	ICH_LR8_EL2,  x13

arch/arm64/kvm/vgic-v3-switch.S line 253

Quoted 10 lines
arch/arm64/kvm/vgic-v3-switch.S new file +10 -0
+	msr	ICH_LR7_EL2,  x12
+	msr	ICH_LR6_EL2,  x11
+	msr	ICH_LR5_EL2,  x10
+	msr	ICH_LR4_EL2,   x9
+	msr	ICH_LR3_EL2,   x8
+	msr	ICH_LR2_EL2,   x7
+	msr	ICH_LR1_EL2,   x6
+	msr	ICH_LR0_EL2,   x5
+
+	dsb	st

Now you have a dsb without an isb! There's no consistency here and the docs don't tell me much either.

arch/arm64/kvm/vgic-v3-switch.S line 257

Quoted 4 lines
arch/arm64/kvm/vgic-v3-switch.S new file +4 -0
+
+	mrs	x5, ICC_SRE_EL2
+	and	x5, x5, #~(1 << 3)
+	msr	ICC_SRE_EL2, x5

Why are you doing this? Should't we leave bits 0 and 3 of SRE_EL2 always set?

Will

#48 lore raw

PATCH 18/18 arm64: KVM: vgic: add GICv3 world switch

Quoted 17 lines · Will Deacon
Quoted 13 lines · Marc Zyngier

Introduce the GICv3 world switch code and helper functions, enabling GICv2 emulation on GICv3 hardware.

---
 arch/arm64/include/asm/kvm_asm.h  |   4 +
 arch/arm64/include/asm/kvm_host.h |   5 +
 arch/arm64/kernel/asm-offsets.c   |   8 ++
 arch/arm64/kvm/Makefile           |   2 +
 arch/arm64/kvm/vgic-v3-switch.S   | 275 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 294 insertions(+)
 create mode 100644 arch/arm64/kvm/vgic-v3-switch.S

[...]

Quoted 13 lines

arch/arm64/kvm/vgic-v3-switch.S line 45

Quoted 10 lines

+.macro save_vgic_v3_state

arch/arm64/kvm/vgic-v3-switch.S new file +9 -0
+	/* Compute the address of struct vgic_cpu */
+	add	x3, x0, #VCPU_VGIC_CPU
+
+	mrs	x5, ICC_SRE_EL2
+	orr	x5, x5, #(1 << 3)
+	msr	ICC_SRE_EL2, x5
+	isb
+
+	dsb	st

Aha, so you *do* use a dsb here (see my comments to patch 1/18). Does it really need to be full-system, and does -st apply to an msr?

It seems to apply indeed, according to the architects. As for the locality, this is unclear at best...

Quoted 2 lines

Saying that, why would ICC_SRE_EL2 not already be initialised by the hyp setup code?

We switch the SRE bit at EL2 when we enter the guest in order to prevent the guest from using SRE itself. Here, we come back to the host, hence restoring the SRE setting.

Quoted 10 lines
Quoted 10 lines
arch/arm64/kvm/vgic-v3-switch.S new file +10 -0
+
+	/* Save all interesting registers */
+	mrs	x4, ICH_HCR_EL2
+	mrs	x5, ICH_VMCR_EL2
+	mrs	x6, ICH_MISR_EL2
+	mrs	x7, ICH_EISR_EL2
+	mrs	x8, ICH_ELSR_EL2
+
+	str	w4, [x3, #VGIC_V3_CPU_HCR]
+	str	w5, [x3, #VGIC_V3_CPU_VMCR]
Quoted 8 lines

arch/arm64/kvm/vgic-v3-switch.S line 61

Quoted 6 lines
arch/arm64/kvm/vgic-v3-switch.S new file +6 -0
+	str	w6, [x3, #VGIC_V3_CPU_MISR]
+	str	w7, [x3, #VGIC_V3_CPU_EISR]
+	str	w8, [x3, #VGIC_V3_CPU_ELRSR]
+
+	msr	ICH_HCR_EL2, xzr
+	isb

Why do you need an isb here?

Hmmm. I can't really recall why I did put that one in, so I guess I should drop it.

Quoted 76 lines
Quoted 76 lines
arch/arm64/kvm/vgic-v3-switch.S new file +76 -0
+
+	mrs	x21, ICH_VTR_EL2
+	and	w22, w21, #0xf
+	mov	w23, #0xf
+	sub	w23, w23, w22	// How many regs we have to skip
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2
+	br	x24
+
+1:
+	mrs	x20, ICH_LR15_EL2
+	mrs	x19, ICH_LR14_EL2
+	mrs	x18, ICH_LR13_EL2
+	mrs	x17, ICH_LR12_EL2
+	mrs	x16, ICH_LR11_EL2
+	mrs	x15, ICH_LR10_EL2
+	mrs	x14, ICH_LR9_EL2
+	mrs	x13, ICH_LR8_EL2
+	mrs	x12, ICH_LR7_EL2
+	mrs	x11, ICH_LR6_EL2
+	mrs	x10, ICH_LR5_EL2
+	mrs	x9, ICH_LR4_EL2
+	mrs	x8, ICH_LR3_EL2
+	mrs	x7, ICH_LR2_EL2
+	mrs	x6, ICH_LR1_EL2
+	mrs	x5, ICH_LR0_EL2
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2	// adr(1f) + unimp_nr*4
+	br	x24
+
+1:
+	str	x20, [x3, #(VGIC_V3_CPU_LR + 15*8)]
+	str	x19, [x3, #(VGIC_V3_CPU_LR + 14*8)]
+	str	x18, [x3, #(VGIC_V3_CPU_LR + 13*8)]
+	str	x17, [x3, #(VGIC_V3_CPU_LR + 12*8)]
+	str	x16, [x3, #(VGIC_V3_CPU_LR + 11*8)]
+	str	x15, [x3, #(VGIC_V3_CPU_LR + 10*8)]
+	str	x14, [x3, #(VGIC_V3_CPU_LR + 9*8)]
+	str	x13, [x3, #(VGIC_V3_CPU_LR + 8*8)]
+	str	x12, [x3, #(VGIC_V3_CPU_LR + 7*8)]
+	str	x11, [x3, #(VGIC_V3_CPU_LR + 6*8)]
+	str	x10, [x3, #(VGIC_V3_CPU_LR + 5*8)]
+	str	x9, [x3, #(VGIC_V3_CPU_LR + 4*8)]
+	str	x8, [x3, #(VGIC_V3_CPU_LR + 3*8)]
+	str	x7, [x3, #(VGIC_V3_CPU_LR + 2*8)]
+	str	x6, [x3, #(VGIC_V3_CPU_LR + 1*8)]
+	str	x5, [x3, #VGIC_V3_CPU_LR]
+
+	lsr	w22, w21, #29	// Get PRIbits
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	mrs	x20, ICH_AP0R3_EL2
+	str	w20, [x3, #(VGIC_V3_CPU_AP0R + 3*4)]
+	mrs	x19, ICH_AP0R2_EL2
+	str	w19, [x3, #(VGIC_V3_CPU_AP0R + 2*4)]
+6:	mrs	x18, ICH_AP0R1_EL2
+	str	w18, [x3, #(VGIC_V3_CPU_AP0R + 1*4)]
+5:	mrs	x17, ICH_AP0R0_EL2
+	str	w17, [x3, #VGIC_V3_CPU_AP0R]
+
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	mrs	x20, ICH_AP1R3_EL2
+	str	w20, [x3, #(VGIC_V3_CPU_AP1R + 3*4)]
+	mrs	x19, ICH_AP1R2_EL2
+	str	w19, [x3, #(VGIC_V3_CPU_AP1R + 2*4)]
+6:	mrs	x18, ICH_AP1R1_EL2
+	str	w18, [x3, #(VGIC_V3_CPU_AP1R + 1*4)]
Quoted 8 lines

arch/arm64/kvm/vgic-v3-switch.S line 143

Quoted 6 lines

+5: mrs x17, ICH_AP1R0_EL2

arch/arm64/kvm/vgic-v3-switch.S new file +5 -0
+	str	w17, [x3, #VGIC_V3_CPU_AP1R]
+
+	mov	x2, #HCR_RW
+	msr	hcr_el2, x2
+	isb

Same here -- HCR_EL2.RW is for lower exception levels.

Yes, and it has an influence on the following setting of SRE just below. We have to absolutely sure that we've exited guest context before touching ICC_SRE_EL1.

Quoted 13 lines
Quoted 13 lines
arch/arm64/kvm/vgic-v3-switch.S new file +13 -0
+
+	// Restore SRE_EL1 access
+	mov	x5, #1
+	msr	ICC_SRE_EL1, x5
+.endm
+
+/*
+ * Restore the VGIC CPU state from memory
+ * x0: Register pointing to VCPU struct
+ */
+.macro	restore_vgic_v3_state
+	ldr	x2, [x0, #VCPU_IRQ_LINES]
+	ldr	x1, [x0, #VCPU_HCR_EL2]
Quoted 8 lines

arch/arm64/kvm/vgic-v3-switch.S line 162

Quoted 6 lines
arch/arm64/kvm/vgic-v3-switch.S new file +6 -0
+	orr	x2, x2, x1
+	msr	hcr_el2, x2
+
+	// Disable SRE_EL1 access
+	msr	ICC_SRE_EL1, xzr
+	isb

We're executing at EL2, why the isb?

If we don't make sure that SRE is effectively disabled at EL1, interrupts injected as group 0 will appear as FIQs on the guest side. Obviously, we don't want that.

Quoted 86 lines
Quoted 86 lines
arch/arm64/kvm/vgic-v3-switch.S new file +86 -0
+
+	/* Compute the address of struct vgic_cpu */
+	add	x3, x0, #VCPU_VGIC_CPU
+
+	/* Restore all interesting registers */
+	ldr	w4, [x3, #VGIC_V3_CPU_HCR]
+	ldr	w5, [x3, #VGIC_V3_CPU_VMCR]
+
+	msr	ICH_HCR_EL2, x4
+	msr	ICH_VMCR_EL2, x5
+
+	mrs	x21, ICH_VTR_EL2
+
+	lsr	w22, w21, #29	// Get PRIbits
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	ldr	w20, [x3, #(VGIC_V3_CPU_AP1R + 3*4)]
+	msr	ICH_AP1R3_EL2, x20
+	ldr	w19, [x3, #(VGIC_V3_CPU_AP1R + 2*4)]
+	msr	ICH_AP1R2_EL2, x19
+6:	ldr	w18, [x3, #(VGIC_V3_CPU_AP1R + 1*4)]
+	msr	ICH_AP1R1_EL2, x18
+5:	ldr	w17, [x3, #VGIC_V3_CPU_AP1R]
+	msr	ICH_AP1R0_EL2, x17
+
+	cmp	w22, #4		// 5 bits
+	b.eq	5f
+	cmp	w22, #5		// 6 bits
+	b.eq	6f
+				// 7 bits
+	ldr	w20, [x3, #(VGIC_V3_CPU_AP0R + 3*4)]
+	msr	ICH_AP0R3_EL2, x20
+	ldr	w19, [x3, #(VGIC_V3_CPU_AP0R + 2*4)]
+	msr	ICH_AP0R2_EL2, x19
+6:	ldr	w18, [x3, #(VGIC_V3_CPU_AP0R + 1*4)]
+	msr	ICH_AP0R1_EL2, x18
+5:	ldr	w17, [x3, #VGIC_V3_CPU_AP0R]
+	msr	ICH_AP0R0_EL2, x17
+
+	and	w22, w21, #0xf
+	mov	w23, #0xf
+	sub	w23, w23, w22	// How many regs we have to skip
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2	// adr(1f) + unimp_nr*4
+	br	x24
+
+1:
+	ldr	x20, [x3, #(VGIC_V3_CPU_LR + 15*8)]
+	ldr	x19, [x3, #(VGIC_V3_CPU_LR + 14*8)]
+	ldr	x18, [x3, #(VGIC_V3_CPU_LR + 13*8)]
+	ldr	x17, [x3, #(VGIC_V3_CPU_LR + 12*8)]
+	ldr	x16, [x3, #(VGIC_V3_CPU_LR + 11*8)]
+	ldr	x15, [x3, #(VGIC_V3_CPU_LR + 10*8)]
+	ldr	x14, [x3, #(VGIC_V3_CPU_LR + 9*8)]
+	ldr	x13, [x3, #(VGIC_V3_CPU_LR + 8*8)]
+	ldr	x12, [x3, #(VGIC_V3_CPU_LR + 7*8)]
+	ldr	x11, [x3, #(VGIC_V3_CPU_LR + 6*8)]
+	ldr	x10, [x3, #(VGIC_V3_CPU_LR + 5*8)]
+	ldr	x9, [x3, #(VGIC_V3_CPU_LR + 4*8)]
+	ldr	x8, [x3, #(VGIC_V3_CPU_LR + 3*8)]
+	ldr	x7, [x3, #(VGIC_V3_CPU_LR + 2*8)]
+	ldr	x6, [x3, #(VGIC_V3_CPU_LR + 1*8)]
+	ldr	x5, [x3, #VGIC_V3_CPU_LR]
+
+	adr	x24, 1f
+	add	x24, x24, x23, lsl #2
+	br	x24
+
+1:
+	msr	ICH_LR15_EL2, x20
+	msr	ICH_LR14_EL2, x19
+	msr	ICH_LR13_EL2, x18
+	msr	ICH_LR12_EL2, x17
+	msr	ICH_LR11_EL2, x16
+	msr	ICH_LR10_EL2, x15
+	msr	ICH_LR9_EL2,  x14
+	msr	ICH_LR8_EL2,  x13
+	msr	ICH_LR7_EL2,  x12
+	msr	ICH_LR6_EL2,  x11
+	msr	ICH_LR5_EL2,  x10
+	msr	ICH_LR4_EL2,   x9
+	msr	ICH_LR3_EL2,   x8
Quoted 8 lines

arch/arm64/kvm/vgic-v3-switch.S line 253

Quoted 5 lines
arch/arm64/kvm/vgic-v3-switch.S new file +5 -0
+	msr	ICH_LR2_EL2,   x7
+	msr	ICH_LR1_EL2,   x6
+	msr	ICH_LR0_EL2,   x5
+
+	dsb	st

Now you have a dsb without an isb! There's no consistency here and the docs don't tell me much either.

Hmmm. isb may be missing indeed. I'll grab "You Know Who" to have a chat... ;-)

Quoted 7 lines

arch/arm64/kvm/vgic-v3-switch.S line 257

Quoted 4 lines
arch/arm64/kvm/vgic-v3-switch.S new file +4 -0
+
+	mrs	x5, ICC_SRE_EL2
+	and	x5, x5, #~(1 << 3)
+	msr	ICC_SRE_EL2, x5

Why are you doing this? Should't we leave bits 0 and 3 of SRE_EL2 always set?

No. We want to prevent the guest from doing SRE, as we don't handle that at all.

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...