PCI: mvebu: Add support for PME and AER interrupts
Subsystems (6)
Pali Rohár
maintainercover PCI: mvebu: Add support for PME and AER interrupts
mvebu PCIe PME and AER interrupts are reported via PCIe summary interrupt. PCIe summary interrupt is reported via mvebu MPIC SoC error summary interrupt. And MPIC SoC error summary interrupt is reported via MPIC IRQ 4.
This patch series implements support for interrupts in MPIC SoC error hierarchy in irq-armada-370-xp.c driver and support for interrupts in mvebu PCIe hierarchy in pci-mvebu.c.
Finally PCIe PME and AER interrupts are routed to the correct PCIe Root Port, which allows kernel PME and AER drivers to take care of them.
Tested on A385 board and kernel PME and AER drivers works correctly:
[ 0.898482] pcieport 0000:00:01.0: PME: Signaling with IRQ 61 [ 0.904422] pcieport 0000:00:01.0: AER: enabled with IRQ 61 [ 0.910113] pcieport 0000:00:02.0: enabling device (0140 -> 0142) [ 0.916299] pcieport 0000:00:02.0: PME: Signaling with IRQ 62 [ 0.922216] pcieport 0000:00:02.0: AER: enabled with IRQ 62 [ 0.927917] pcieport 0000:00:03.0: enabling device (0140 -> 0142) [ 0.934090] pcieport 0000:00:03.0: PME: Signaling with IRQ 63 [ 0.940006] pcieport 0000:00:03.0: AER: enabled with IRQ 63
This change finally allows to debug PCIe issues on A385 boards.
Pali Rohár (6):
dt-bindings: irqchip: armada-370-xp: Update information about MPIC SoC
Error
irqchip/armada-370-xp: Implement SoC Error interrupts
ARM: dts: armada-38x.dtsi: Add node for MPIC SoC Error IRQ controller
dt-bindings: PCI: mvebu: Update information about summary interrupt
PCI: mvebu: Implement support for interrupts on emulated bridge
ARM: dts: armada-385.dtsi: Add definitions for PCIe summary interrupts
.../marvell,armada-370-xp-mpic.txt | 9 + .../devicetree/bindings/pci/mvebu-pci.txt | 1 + arch/arm/boot/dts/armada-385.dtsi | 20 +- arch/arm/boot/dts/armada-38x.dtsi | 5 + drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++- drivers/pci/controller/pci-mvebu.c | 208 +++++++++++++++-- 6 files changed, 426 insertions(+), 30 deletions(-)
Signature · 1 line
2.20.1
Pali Rohár
maintainerPATCH 1/6 dt-bindings: irqchip: armada-370-xp: Update information about MPIC SoC Error
--- .../interrupt-controller/marvell,armada-370-xp-mpic.txt | 9 +++++++++ 1 file changed, 9 insertions(+)
Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt +9 -0
@@ -24,6 +24,11 @@ Optional properties: connected as a slave to the Cortex-A9 GIC. The provided interrupt indicate to which GIC interrupt the MPIC output is connected. +Optional subnodes: + +- interrupt-controller@20 with interrupt-controller property for + MPIC SoC Error IRQ controller + Example: mpic: interrupt-controller@d0020000 {
@@ -35,4 +40,8 @@ Example: msi-controller; reg = <0xd0020a00 0x1d0>, <0xd0021070 0x58>; + soc_err: interrupt-controller@20 { + interrupt-controller; + #interrupt-cells = <1>; + }; };
Signature · 1 line
2.20.1
Rob Herring
maintainerRe: [PATCH 1/6] dt-bindings: irqchip: armada-370-xp: Update information about MPIC SoC Error
Quoted 1 line · Pali Rohár
Signed-off-by Pali Rohár <[email protected]>
Why do we need/want this change?
Quoted 21 lines
--- .../interrupt-controller/marvell,armada-370-xp-mpic.txt | 9 +++++++++ 1 file changed, 9 insertions(+)Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt +5
-0@@ -24,6 +24,11 @@ Optional properties: connected as a slave to the Cortex-A9 GIC. The provided interrupt indicate to which GIC interrupt the MPIC output is connected. +Optional subnodes: + +- interrupt-controller@20 with interrupt-controller property for + MPIC SoC Error IRQ controller + Example: mpic: interrupt-controller@d0020000 {
Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt Example: · line 43
Quoted 4 lines
Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt +1
-0@@ -35,4 +40,8 @@ Example: msi-controller; reg = <0xd0020a00 0x1d0>, <0xd0021070 0x58>; + soc_err: interrupt-controller@20 {
unit address without 'reg' is an error.
Quoted 8 lines
Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt +3
-0+ interrupt-controller; + #interrupt-cells = <1>; + }; };Signature · 1 line
2.20.1
Pali Rohár
maintainerPATCH 2/6 irqchip/armada-370-xp: Implement SoC Error interrupts
In reply to Pali RohárMPIC IRQ 4 is used as SoC Error Summary interrupt and provides access to another hierarchy of SoC Error interrupts. Implement a new IRQ chip and domain for accessing this IRQ hierarchy.
--- drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++++++++++++- 1 file changed, 210 insertions(+), 3 deletions(-)
drivers/irqchip/irq-armada-370-xp.c +210 -3
@@ -117,6 +117,8 @@/* Registers relative to main_int_base */ #define ARMADA_370_XP_INT_CONTROL (0x00)#define ARMADA_370_XP_SW_TRIG_INT_OFFS (0x04) +#define ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS (0x20) +#define ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS (0x24)#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)#define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS (0x34)#define ARMADA_370_XP_INT_SOURCE_CTL (irq) (0x100 + irq*4)
@@ -130,6 +132,8 @@#define ARMADA_370_XP_CPU_INTACK_OFFS (0x44)#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C) +#define ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF (0x50) +#define ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF (0x54)#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)#define ARMADA_370_XP_INT_CAUSE_PERF (cpu) (1 << cpu)
@@ -146,6 +150,8 @@static void __iomem *per_cpu_int_base;static void __iomem *main_int_base;static struct irq_domain *armada_370_xp_mpic_domain; +static struct irq_domain *armada_370_xp_soc_err_domain; +static unsigned int soc_err_irq_num_regs;static u32 doorbell_mask_reg;static int parent_irq;#ifdef CONFIG_PCI_MSI
@@ -156,6 +162,8 @@ static DEFINE_MUTEX(msi_used_lock);static phys_addr_t msi_doorbell_addr;#endif +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )
@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq); +if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + } + +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +ipi_resume (); }
@@ -546,8 +575,8 @@ static struct irq_chip armada_370_xp_irq_chip = {static int armada_370_xp_mpic_irq_map (struct irq_domain *h,unsigned int virq,irq_hw_number_t hw) { -/* IRQs 0 and 1 cannot be mapped, they are handled internally */ -if (hw <=1 ) +/* IRQs 0, 1 and 4 cannot be mapped, they are handled internally */ +if (hw <=1 || hw ==4 )return -EINVAL ;armada_370_xp_irq_mask (irq_get_irq_data (virq));
@@ -577,6 +606,99 @@ static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { .xlate = irq_domain_xlate_onecell, }; +static DEFINE_RAW_SPINLOCK (armada_370_xp_soc_err_lock); + +static void armada_370_xp_soc_err_irq_mask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask &= ~BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask |=BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static int armada_370_xp_soc_err_irq_mask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_mask (d); +return 0 ; +} + +static int armada_370_xp_soc_err_irq_unmask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_unmask (d); +return 0 ; +} + +static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); + +/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock (); + +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +} + +static struct irq_chip armada_370_xp_soc_err_irq_chip = { + .name ="MPIC SOC" , + .irq_mask = armada_370_xp_soc_err_irq_mask, + .irq_unmask = armada_370_xp_soc_err_irq_unmask, + .irq_set_affinity = armada_xp_soc_err_irq_set_affinity, +}; + +static int armada_370_xp_soc_err_irq_map (struct irq_domain *h, +unsigned int virq,irq_hw_number_t hw) +{ +armada_370_xp_soc_err_irq_mask (irq_get_irq_data (virq)); +irq_set_status_flags (virq,IRQ_LEVEL ); +irq_set_percpu_devid (virq); +irq_set_chip_and_handler (virq, &armada_370_xp_soc_err_irq_chip, + handle_percpu_devid_irq); +irq_set_probe (virq); +return 0 ; +} + +static const struct irq_domain_ops armada_370_xp_soc_err_irq_ops = { + .map = armada_370_xp_soc_err_irq_map, + .xlate = irq_domain_xlate_onecell, +}; +#ifdef CONFIG_PCI_MSI static void armada_370_xp_handle_msi_irq (struct pt_regs *regs,bool is_chained) {
@@ -605,6 +727,32 @@ static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained)static void armada_370_xp_handle_msi_irq (struct pt_regs *r,bool b) {}#endif +static void armada_370_xp_handle_soc_err_irq (void ) +{ +unsigned long status, bit; +u32 mask, cause; + +if (soc_err_irq_num_regs <1 ) +return ; + + mask =readl (per_cpu_int_base +ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ); + cause =readl (main_int_base +ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS ); + status = cause & mask; + +for_each_set_bit (bit, &status,32 ) +generic_handle_domain_irq (armada_370_xp_soc_err_domain, bit); + +if (soc_err_irq_num_regs <2 ) +return ; + + mask =readl (per_cpu_int_base +ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF ); + cause =readl (main_int_base +ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS ); + status = cause & mask; + +for_each_set_bit (bit, &status,32 ) +generic_handle_domain_irq (armada_370_xp_soc_err_domain, bit +32 ); +} +static void armada_370_xp_mpic_handle_cascade_irq (struct irq_desc *desc) {struct irq_chip *chip =irq_desc_get_chip (desc);
@@ -630,6 +778,11 @@ static void armada_370_xp_mpic_handle_cascade_irq(struct irq_desc *desc)continue ; } +if (irqn ==4 ) { +armada_370_xp_handle_soc_err_irq (); +continue ; + } +generic_handle_domain_irq (armada_370_xp_mpic_domain, irqn); }
@@ -649,7 +802,7 @@ armada_370_xp_handle_irq(struct pt_regs *regs)if (irqnr >1022 )break ; -if (irqnr >1 ) { +if (irqnr >1 && irqnr !=4 ) {generic_handle_domain_irq (armada_370_xp_mpic_domain, irqnr);continue ;
@@ -659,6 +812,10 @@ armada_370_xp_handle_irq(struct pt_regs *regs)if (irqnr ==1 )armada_370_xp_handle_msi_irq (regs, false); +/* SoC Error handling */ +if (irqnr ==4 ) +armada_370_xp_handle_soc_err_irq (); +#ifdef CONFIG_SMP /* IPI Handling */ if (irqnr ==0 ) {
@@ -722,6 +879,26 @@ static void armada_370_xp_mpic_resume(void) } } +/* Re-enable per-CPU SoC Error interrupts */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq); +if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +/* +* Re-enable on the current CPU, +* armada_xp_mpic_reenable_percpu() will take +* care of secondary CPUs when they come up. +*/ +if (irq_percpu_is_enabled (virq)) +armada_370_xp_soc_err_irq_unmask (data); + } +/* Reconfigure doorbells for IPIs and MSIs */ writel (doorbell_mask_reg, per_cpu_int_base +ARMADA_370_XP_IN_DRBEL_MSK_OFFS );
@@ -730,6 +907,10 @@ static void armada_370_xp_mpic_resume(void)if (doorbell_mask_reg &PCI_MSI_DOORBELL_MASK )writel (1 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +ipi_resume (); }
@@ -742,6 +923,7 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node, struct device_node *parent) {struct resource main_int_res, per_cpu_int_res; +struct device_node *soc_err_node;int nr_irqs, i;u32 control;
@@ -775,12 +957,37 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,BUG_ON (!armada_370_xp_mpic_domain);irq_domain_update_bus_token (armada_370_xp_mpic_domain,DOMAIN_BUS_WIRED ); + soc_err_node =of_get_next_child (node,NULL ); +if (!soc_err_node) { +pr_warn ("Missing SoC Error Interrupt Controller node\n" ); +pr_warn ("Extended interrupts are not supported\n" ); + }else { +pr_info ("Registering MPIC SoC Error Interrupt Controller\n" ); +/* +* Armada 370 and XP have only 32 SoC Error IRQs in one register +* and other Armada platforms have 64 IRQs in two registers. +*/ + soc_err_irq_num_regs = +of_machine_is_compatible ("marvell,armada-370-xp" ) ?1 :2 ; + armada_370_xp_soc_err_domain = +irq_domain_add_hierarchy (armada_370_xp_mpic_domain,0 , + soc_err_irq_num_regs *32 , + soc_err_node, + &armada_370_xp_soc_err_irq_ops, +NULL ); +BUG_ON (!armada_370_xp_soc_err_domain); + } +/* Setup for the boot CPU */ armada_xp_mpic_perf_init ();armada_xp_mpic_smp_cpu_init ();armada_370_xp_msi_init (node, main_int_res.start); +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); + parent_irq =irq_of_parse_and_map (node,0 );if (parent_irq <=0 ) {irq_set_default_host (armada_370_xp_mpic_domain);
Signature · 1 line
2.20.1
Re: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 60 lines · Pali Rohár
MPIC IRQ 4 is used as SoC Error Summary interrupt and provides access to another hierarchy of SoC Error interrupts. Implement a new IRQ chip and domain for accessing this IRQ hierarchy.
Signed-off-by Pali Rohár <[email protected]>--- drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++++++++++++- 1 file changed, 210 insertions(+), 3 deletions(-)drivers/irqchip/irq-armada-370-xp.c +14
-0@@ -117,6 +117,8 @@/* Registers relative to main_int_base */ #define ARMADA_370_XP_INT_CONTROL (0x00)#define ARMADA_370_XP_SW_TRIG_INT_OFFS (0x04) +#define ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS (0x20) +#define ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS (0x24)#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)#define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS (0x34)#define ARMADA_370_XP_INT_SOURCE_CTL (irq) (0x100 + irq*4)@@ -130,6 +132,8 @@#define ARMADA_370_XP_CPU_INTACK_OFFS (0x44)#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C) +#define ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF (0x50) +#define ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF (0x54)#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)#define ARMADA_370_XP_INT_CAUSE_PERF (cpu) (1 << cpu)@@ -146,6 +150,8 @@static void __iomem *per_cpu_int_base;static void __iomem *main_int_base;static struct irq_domain *armada_370_xp_mpic_domain; +static struct irq_domain *armada_370_xp_soc_err_domain; +static unsigned int soc_err_irq_num_regs;static u32 doorbell_mask_reg;static int parent_irq;#ifdef CONFIG_PCI_MSI @@ -156,6 +162,8 @@ static DEFINE_MUTEX(msi_used_lock);static phys_addr_t msi_doorbell_addr;#endif +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq);
drivers/irqchip/irq-armada-370-xp.c static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + }
So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Quoted 81 lines
drivers/irqchip/irq-armada-370-xp.c +65
-2+ +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +ipi_resume (); }@@ -546,8 +575,8 @@ static struct irq_chip armada_370_xp_irq_chip = {static int armada_370_xp_mpic_irq_map (struct irq_domain *h,unsigned int virq,irq_hw_number_t hw) { -/* IRQs 0 and 1 cannot be mapped, they are handled internally */ -if (hw <=1 ) +/* IRQs 0, 1 and 4 cannot be mapped, they are handled internally */ +if (hw <=1 || hw ==4 )return -EINVAL ;armada_370_xp_irq_mask (irq_get_irq_data (virq));@@ -577,6 +606,99 @@ static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { .xlate = irq_domain_xlate_onecell, }; +static DEFINE_RAW_SPINLOCK (armada_370_xp_soc_err_lock); + +static void armada_370_xp_soc_err_irq_mask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask &= ~BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask |=BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static int armada_370_xp_soc_err_irq_mask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_mask (d); +return 0 ; +} + +static int armada_370_xp_soc_err_irq_unmask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_unmask (d); +return 0 ; +} + +static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); +
drivers/irqchip/irq-armada-370-xp.c static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { · line 676
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock (); + +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +}
Aren't these per-CPU interrupts anyway? What does it mean to set their affinity? /me rolls eyes...
Quoted 106 lines
drivers/irqchip/irq-armada-370-xp.c +70
-1+ +static struct irq_chip armada_370_xp_soc_err_irq_chip = { + .name ="MPIC SOC" , + .irq_mask = armada_370_xp_soc_err_irq_mask, + .irq_unmask = armada_370_xp_soc_err_irq_unmask, + .irq_set_affinity = armada_xp_soc_err_irq_set_affinity, +}; + +static int armada_370_xp_soc_err_irq_map (struct irq_domain *h, +unsigned int virq,irq_hw_number_t hw) +{ +armada_370_xp_soc_err_irq_mask (irq_get_irq_data (virq)); +irq_set_status_flags (virq,IRQ_LEVEL ); +irq_set_percpu_devid (virq); +irq_set_chip_and_handler (virq, &armada_370_xp_soc_err_irq_chip, + handle_percpu_devid_irq); +irq_set_probe (virq); +return 0 ; +} + +static const struct irq_domain_ops armada_370_xp_soc_err_irq_ops = { + .map = armada_370_xp_soc_err_irq_map, + .xlate = irq_domain_xlate_onecell, +}; +#ifdef CONFIG_PCI_MSI static void armada_370_xp_handle_msi_irq (struct pt_regs *regs,bool is_chained) {@@ -605,6 +727,32 @@ static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained)static void armada_370_xp_handle_msi_irq (struct pt_regs *r,bool b) {}#endif +static void armada_370_xp_handle_soc_err_irq (void ) +{ +unsigned long status, bit; +u32 mask, cause; + +if (soc_err_irq_num_regs <1 ) +return ; + + mask =readl (per_cpu_int_base +ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ); + cause =readl (main_int_base +ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS ); + status = cause & mask; + +for_each_set_bit (bit, &status,32 ) +generic_handle_domain_irq (armada_370_xp_soc_err_domain, bit); + +if (soc_err_irq_num_regs <2 ) +return ; + + mask =readl (per_cpu_int_base +ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF ); + cause =readl (main_int_base +ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS ); + status = cause & mask; + +for_each_set_bit (bit, &status,32 ) +generic_handle_domain_irq (armada_370_xp_soc_err_domain, bit +32 ); +} +static void armada_370_xp_mpic_handle_cascade_irq (struct irq_desc *desc) {struct irq_chip *chip =irq_desc_get_chip (desc);@@ -630,6 +778,11 @@ static void armada_370_xp_mpic_handle_cascade_irq(struct irq_desc *desc)continue ; } +if (irqn ==4 ) { +armada_370_xp_handle_soc_err_irq (); +continue ; + } +generic_handle_domain_irq (armada_370_xp_mpic_domain, irqn); }@@ -649,7 +802,7 @@ armada_370_xp_handle_irq(struct pt_regs *regs)if (irqnr >1022 )break ; -if (irqnr >1 ) { +if (irqnr >1 && irqnr !=4 ) {generic_handle_domain_irq (armada_370_xp_mpic_domain, irqnr);continue ;@@ -659,6 +812,10 @@ armada_370_xp_handle_irq(struct pt_regs *regs)if (irqnr ==1 )armada_370_xp_handle_msi_irq (regs, false); +/* SoC Error handling */ +if (irqnr ==4 ) +armada_370_xp_handle_soc_err_irq (); +#ifdef CONFIG_SMP /* IPI Handling */ if (irqnr ==0 ) {@@ -722,6 +879,26 @@ static void armada_370_xp_mpic_resume(void) } } +/* Re-enable per-CPU SoC Error interrupts */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq); +if (virq ==0 ) +continue ; +
drivers/irqchip/irq-armada-370-xp.c static void armada_370_xp_mpic_resume(void) · line 900
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+ data =irq_get_irq_data (virq); + +/* +* Re-enable on the current CPU, +* armada_xp_mpic_reenable_percpu() will take +* care of secondary CPUs when they come up. +*/ +if (irq_percpu_is_enabled (virq)) +armada_370_xp_soc_err_irq_unmask (data); + }
As I said above, this is duplicated code that should be replaced with a simple write to the corresponding MMIO registers.
drivers/irqchip/irq-armada-370-xp.c static void armada_370_xp_mpic_resume(void) · line 912
Quoted 11 lines
drivers/irqchip/irq-armada-370-xp.c +4
-0+/* Reconfigure doorbells for IPIs and MSIs */ writel (doorbell_mask_reg, per_cpu_int_base +ARMADA_370_XP_IN_DRBEL_MSK_OFFS );@@ -730,6 +907,10 @@ static void armada_370_xp_mpic_resume(void)if (doorbell_mask_reg &PCI_MSI_DOORBELL_MASK )writel (1 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS );
Magic value?
Also, writing to this register tends to indicate that the whole thing should really be a chained irqchip... Maybe that's overkill in this instance, but the whole thing is rather oddly architected.
Quoted 18 lines
drivers/irqchip/irq-armada-370-xp.c +4
-0+ipi_resume (); }@@ -742,6 +923,7 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node, struct device_node *parent) {struct resource main_int_res, per_cpu_int_res; +struct device_node *soc_err_node;int nr_irqs, i;u32 control;@@ -775,12 +957,37 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,BUG_ON (!armada_370_xp_mpic_domain);irq_domain_update_bus_token (armada_370_xp_mpic_domain,DOMAIN_BUS_WIRED ); + soc_err_node =of_get_next_child (node,NULL ); +if (!soc_err_node) {
drivers/irqchip/irq-armada-370-xp.c static int __init armada_370_xp_mpic_of_init(struct device_node *node, · line 971
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+pr_warn ("Missing SoC Error Interrupt Controller node\n" ); +pr_warn ("Extended interrupts are not supported\n" ); + }else { +pr_info ("Registering MPIC SoC Error Interrupt Controller\n" ); +/* +* Armada 370 and XP have only 32 SoC Error IRQs in one register +* and other Armada platforms have 64 IRQs in two registers. +*/ + soc_err_irq_num_regs = +of_machine_is_compatible ("marvell,armada-370-xp" ) ?1 :2 ;
Don't you have an actual compatible string for the interrupt controller? It seems odd to rely on the SoC name.
Quoted 9 lines
drivers/irqchip/irq-armada-370-xp.c +9
-0+ armada_370_xp_soc_err_domain = +irq_domain_add_hierarchy (armada_370_xp_mpic_domain,0 , + soc_err_irq_num_regs *32 , + soc_err_node, + &armada_370_xp_soc_err_irq_ops, +NULL ); +BUG_ON (!armada_370_xp_soc_err_domain); + } +
drivers/irqchip/irq-armada-370-xp.c static int __init armada_370_xp_mpic_of_init(struct device_node *node, · line 990
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +4
-0/* Setup for the boot CPU */ armada_xp_mpic_perf_init ();armada_xp_mpic_smp_cpu_init ();armada_370_xp_msi_init (node, main_int_res.start); +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +
Magic value, duplicated this time?
Quoted 7 lines
drivers/irqchip/irq-armada-370-xp.c +0
-0parent_irq =irq_of_parse_and_map (node,0 );if (parent_irq <=0 ) {irq_set_default_host (armada_370_xp_mpic_domain);Signature · 1 line
2.20.1
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Pali Rohár
maintainerRe: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 78 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 60 lines · Pali Rohár
MPIC IRQ 4 is used as SoC Error Summary interrupt and provides access to another hierarchy of SoC Error interrupts. Implement a new IRQ chip and domain for accessing this IRQ hierarchy.
Signed-off-by Pali Rohár <[email protected]>--- drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++++++++++++- 1 file changed, 210 insertions(+), 3 deletions(-)drivers/irqchip/irq-armada-370-xp.c +14
-0@@ -117,6 +117,8 @@/* Registers relative to main_int_base */ #define ARMADA_370_XP_INT_CONTROL (0x00)#define ARMADA_370_XP_SW_TRIG_INT_OFFS (0x04) +#define ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS (0x20) +#define ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS (0x24)#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)#define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS (0x34)#define ARMADA_370_XP_INT_SOURCE_CTL (irq) (0x100 + irq*4)@@ -130,6 +132,8 @@#define ARMADA_370_XP_CPU_INTACK_OFFS (0x44)#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C) +#define ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF (0x50) +#define ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF (0x54)#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)#define ARMADA_370_XP_INT_CAUSE_PERF (cpu) (1 << cpu)@@ -146,6 +150,8 @@static void __iomem *per_cpu_int_base;static void __iomem *main_int_base;static struct irq_domain *armada_370_xp_mpic_domain; +static struct irq_domain *armada_370_xp_soc_err_domain; +static unsigned int soc_err_irq_num_regs;static u32 doorbell_mask_reg;static int parent_irq;#ifdef CONFIG_PCI_MSI @@ -156,6 +162,8 @@ static DEFINE_MUTEX(msi_used_lock);static phys_addr_t msi_doorbell_addr;#endif +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq);drivers/irqchip/irq-armada-370-xp.c static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Quoted 86 lines
Quoted 86 lines
drivers/irqchip/irq-armada-370-xp.c +70
-2+ +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +ipi_resume (); }@@ -546,8 +575,8 @@ static struct irq_chip armada_370_xp_irq_chip = {static int armada_370_xp_mpic_irq_map (struct irq_domain *h,unsigned int virq,irq_hw_number_t hw) { -/* IRQs 0 and 1 cannot be mapped, they are handled internally */ -if (hw <=1 ) +/* IRQs 0, 1 and 4 cannot be mapped, they are handled internally */ +if (hw <=1 || hw ==4 )return -EINVAL ;armada_370_xp_irq_mask (irq_get_irq_data (virq));@@ -577,6 +606,99 @@ static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { .xlate = irq_domain_xlate_onecell, }; +static DEFINE_RAW_SPINLOCK (armada_370_xp_soc_err_lock); + +static void armada_370_xp_soc_err_irq_mask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask &= ~BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask |=BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static int armada_370_xp_soc_err_irq_mask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_mask (d); +return 0 ; +} + +static int armada_370_xp_soc_err_irq_unmask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_unmask (d); +return 0 ; +} + +static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); + +/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock ();
Quoted 8 lines
drivers/irqchip/irq-armada-370-xp.c static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { · line 676
Quoted 5 lines
drivers/irqchip/irq-armada-370-xp.c +5
-0+ +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +}Aren't these per-CPU interrupts anyway? What does it mean to set their affinity? /me rolls eyes...
Yes, they are per-CPU interrupts. But to mask or unmask particular interrupt for specific CPU is possible only from that CPU. CPU 0 just cannot move interrupt from CPU 0 to CPU 1. CPU 0 can only mask that interrupt and CPU 1 has to unmask it.
Quoted 134 lines
Quoted 106 lines
drivers/irqchip/irq-armada-370-xp.c +70
-1+ +static struct irq_chip armada_370_xp_soc_err_irq_chip = { + .name ="MPIC SOC" , + .irq_mask = armada_370_xp_soc_err_irq_mask, + .irq_unmask = armada_370_xp_soc_err_irq_unmask, + .irq_set_affinity = armada_xp_soc_err_irq_set_affinity, +}; + +static int armada_370_xp_soc_err_irq_map (struct irq_domain *h, +unsigned int virq,irq_hw_number_t hw) +{ +armada_370_xp_soc_err_irq_mask (irq_get_irq_data (virq)); +irq_set_status_flags (virq,IRQ_LEVEL ); +irq_set_percpu_devid (virq); +irq_set_chip_and_handler (virq, &armada_370_xp_soc_err_irq_chip, + handle_percpu_devid_irq); +irq_set_probe (virq); +return 0 ; +} + +static const struct irq_domain_ops armada_370_xp_soc_err_irq_ops = { + .map = armada_370_xp_soc_err_irq_map, + .xlate = irq_domain_xlate_onecell, +}; +#ifdef CONFIG_PCI_MSI static void armada_370_xp_handle_msi_irq (struct pt_regs *regs,bool is_chained) {@@ -605,6 +727,32 @@ static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained)static void armada_370_xp_handle_msi_irq (struct pt_regs *r,bool b) {}#endif +static void armada_370_xp_handle_soc_err_irq (void ) +{ +unsigned long status, bit; +u32 mask, cause; + +if (soc_err_irq_num_regs <1 ) +return ; + + mask =readl (per_cpu_int_base +ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ); + cause =readl (main_int_base +ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS ); + status = cause & mask; + +for_each_set_bit (bit, &status,32 ) +generic_handle_domain_irq (armada_370_xp_soc_err_domain, bit); + +if (soc_err_irq_num_regs <2 ) +return ; + + mask =readl (per_cpu_int_base +ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF ); + cause =readl (main_int_base +ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS ); + status = cause & mask; + +for_each_set_bit (bit, &status,32 ) +generic_handle_domain_irq (armada_370_xp_soc_err_domain, bit +32 ); +} +static void armada_370_xp_mpic_handle_cascade_irq (struct irq_desc *desc) {struct irq_chip *chip =irq_desc_get_chip (desc);@@ -630,6 +778,11 @@ static void armada_370_xp_mpic_handle_cascade_irq(struct irq_desc *desc)continue ; } +if (irqn ==4 ) { +armada_370_xp_handle_soc_err_irq (); +continue ; + } +generic_handle_domain_irq (armada_370_xp_mpic_domain, irqn); }@@ -649,7 +802,7 @@ armada_370_xp_handle_irq(struct pt_regs *regs)if (irqnr >1022 )break ; -if (irqnr >1 ) { +if (irqnr >1 && irqnr !=4 ) {generic_handle_domain_irq (armada_370_xp_mpic_domain, irqnr);continue ;@@ -659,6 +812,10 @@ armada_370_xp_handle_irq(struct pt_regs *regs)if (irqnr ==1 )armada_370_xp_handle_msi_irq (regs, false); +/* SoC Error handling */ +if (irqnr ==4 ) +armada_370_xp_handle_soc_err_irq (); +#ifdef CONFIG_SMP /* IPI Handling */ if (irqnr ==0 ) {@@ -722,6 +879,26 @@ static void armada_370_xp_mpic_resume(void) } } +/* Re-enable per-CPU SoC Error interrupts */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq); +if (virq ==0 ) +continue ; +drivers/irqchip/irq-armada-370-xp.c static void armada_370_xp_mpic_resume(void) · line 900
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+ data =irq_get_irq_data (virq); + +/* +* Re-enable on the current CPU, +* armada_xp_mpic_reenable_percpu() will take +* care of secondary CPUs when they come up. +*/ +if (irq_percpu_is_enabled (virq)) +armada_370_xp_soc_err_irq_unmask (data); + }As I said above, this is duplicated code that should be replaced with a simple write to the corresponding MMIO registers.
drivers/irqchip/irq-armada-370-xp.c static void armada_370_xp_mpic_resume(void) · line 912
Quoted 11 lines
drivers/irqchip/irq-armada-370-xp.c +4
-0+/* Reconfigure doorbells for IPIs and MSIs */ writel (doorbell_mask_reg, per_cpu_int_base +ARMADA_370_XP_IN_DRBEL_MSK_OFFS );@@ -730,6 +907,10 @@ static void armada_370_xp_mpic_resume(void)if (doorbell_mask_reg &PCI_MSI_DOORBELL_MASK )writel (1 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS );Magic value?
Quoted 3 lines
Also, writing to this register tends to indicate that the whole thing should really be a chained irqchip... Maybe that's overkill in this instance, but the whole thing is rather oddly architected.
I used exactly same code style like it is used for MSI doorbell interrupts.
Yes, it looks like chained irqchip, but it is overkill.
Quoted 16 lines
Quoted 16 lines
drivers/irqchip/irq-armada-370-xp.c +2
-0+ipi_resume (); }@@ -742,6 +923,7 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node, struct device_node *parent) {struct resource main_int_res, per_cpu_int_res; +struct device_node *soc_err_node;int nr_irqs, i;u32 control;@@ -775,12 +957,37 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,BUG_ON (!armada_370_xp_mpic_domain);irq_domain_update_bus_token (armada_370_xp_mpic_domain,DOMAIN_BUS_WIRED );
Quoted 15 lines
drivers/irqchip/irq-armada-370-xp.c static int __init armada_370_xp_mpic_of_init(struct device_node *node, · line 971
Quoted 12 lines
drivers/irqchip/irq-armada-370-xp.c +12
-0+ soc_err_node =of_get_next_child (node,NULL ); +if (!soc_err_node) { +pr_warn ("Missing SoC Error Interrupt Controller node\n" ); +pr_warn ("Extended interrupts are not supported\n" ); + }else { +pr_info ("Registering MPIC SoC Error Interrupt Controller\n" ); +/* +* Armada 370 and XP have only 32 SoC Error IRQs in one register +* and other Armada platforms have 64 IRQs in two registers. +*/ + soc_err_irq_num_regs = +of_machine_is_compatible ("marvell,armada-370-xp" ) ?1 :2 ;Don't you have an actual compatible string for the interrupt controller? It seems odd to rely on the SoC name.
Compatible string is same for all those 32-bit Armada SoCs. So it cannot be used to distinguish between XP and 385. That is why I used of_machine_is_compatible.
Quoted 34 lines
Quoted 9 lines
drivers/irqchip/irq-armada-370-xp.c +9
-0+ armada_370_xp_soc_err_domain = +irq_domain_add_hierarchy (armada_370_xp_mpic_domain,0 , + soc_err_irq_num_regs *32 , + soc_err_node, + &armada_370_xp_soc_err_irq_ops, +NULL ); +BUG_ON (!armada_370_xp_soc_err_domain); + } +drivers/irqchip/irq-armada-370-xp.c static int __init armada_370_xp_mpic_of_init(struct device_node *node, · line 990
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +4
-0/* Setup for the boot CPU */ armada_xp_mpic_perf_init ();armada_xp_mpic_smp_cpu_init ();armada_370_xp_msi_init (node, main_int_res.start); +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +Magic value, duplicated this time?
Quoted 7 lines
drivers/irqchip/irq-armada-370-xp.c +0
-0parent_irq =irq_of_parse_and_map (node,0 );if (parent_irq <=0 ) {irq_set_default_host (armada_370_xp_mpic_domain);Signature · 1 line
2.20.1M.Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 80 lines · Pali Rohár
Quoted 78 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 60 lines · Pali Rohár
MPIC IRQ 4 is used as SoC Error Summary interrupt and provides access to another hierarchy of SoC Error interrupts. Implement a new IRQ chip and domain for accessing this IRQ hierarchy.
Signed-off-by Pali Rohár <[email protected]>--- drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++++++++++++- 1 file changed, 210 insertions(+), 3 deletions(-)drivers/irqchip/irq-armada-370-xp.c +14
-0@@ -117,6 +117,8 @@/* Registers relative to main_int_base */ #define ARMADA_370_XP_INT_CONTROL (0x00)#define ARMADA_370_XP_SW_TRIG_INT_OFFS (0x04) +#define ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS (0x20) +#define ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS (0x24)#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)#define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS (0x34)#define ARMADA_370_XP_INT_SOURCE_CTL (irq) (0x100 + irq*4)@@ -130,6 +132,8 @@#define ARMADA_370_XP_CPU_INTACK_OFFS (0x44)#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C) +#define ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF (0x50) +#define ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF (0x54)#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)#define ARMADA_370_XP_INT_CAUSE_PERF (cpu) (1 << cpu)@@ -146,6 +150,8 @@static void __iomem *per_cpu_int_base;static void __iomem *main_int_base;static struct irq_domain *armada_370_xp_mpic_domain; +static struct irq_domain *armada_370_xp_soc_err_domain; +static unsigned int soc_err_irq_num_regs;static u32 doorbell_mask_reg;static int parent_irq;#ifdef CONFIG_PCI_MSI @@ -156,6 +162,8 @@ static DEFINE_MUTEX(msi_used_lock);static phys_addr_t msi_doorbell_addr;#endif +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq);drivers/irqchip/irq-armada-370-xp.c static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 7 lines
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
Quoted 95 lines
Quoted 86 lines
Quoted 86 lines
drivers/irqchip/irq-armada-370-xp.c +70
-2+ +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +ipi_resume (); }@@ -546,8 +575,8 @@ static struct irq_chip armada_370_xp_irq_chip = {static int armada_370_xp_mpic_irq_map (struct irq_domain *h,unsigned int virq,irq_hw_number_t hw) { -/* IRQs 0 and 1 cannot be mapped, they are handled internally */ -if (hw <=1 ) +/* IRQs 0, 1 and 4 cannot be mapped, they are handled internally */ +if (hw <=1 || hw ==4 )return -EINVAL ;armada_370_xp_irq_mask (irq_get_irq_data (virq));@@ -577,6 +606,99 @@ static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { .xlate = irq_domain_xlate_onecell, }; +static DEFINE_RAW_SPINLOCK (armada_370_xp_soc_err_lock); + +static void armada_370_xp_soc_err_irq_mask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask &= ~BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask |=BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static int armada_370_xp_soc_err_irq_mask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_mask (d); +return 0 ; +} + +static int armada_370_xp_soc_err_irq_unmask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_unmask (d); +return 0 ; +} + +static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); + +/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock ();Quoted 8 lines
drivers/irqchip/irq-armada-370-xp.c static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { · line 676
Quoted 5 lines
drivers/irqchip/irq-armada-370-xp.c +5
-0+ +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +}Aren't these per-CPU interrupts anyway? What does it mean to set their affinity? /me rolls eyes...
Quoted 4 lines
Yes, they are per-CPU interrupts. But to mask or unmask particular interrupt for specific CPU is possible only from that CPU. CPU 0 just cannot move interrupt from CPU 0 to CPU 1. CPU 0 can only mask that interrupt and CPU 1 has to unmask it.
And that's no different form other per-CPU interrupts that have the exact same requirements. NAK to this sort of hacks.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Pali Rohár
maintainerRe: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 85 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 83 lines · Pali Rohár
Quoted 80 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 60 lines · Pali Rohár
MPIC IRQ 4 is used as SoC Error Summary interrupt and provides access to another hierarchy of SoC Error interrupts. Implement a new IRQ chip and domain for accessing this IRQ hierarchy.
Signed-off-by Pali Rohár <[email protected]>--- drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++++++++++++- 1 file changed, 210 insertions(+), 3 deletions(-)drivers/irqchip/irq-armada-370-xp.c +14
-0@@ -117,6 +117,8 @@/* Registers relative to main_int_base */ #define ARMADA_370_XP_INT_CONTROL (0x00)#define ARMADA_370_XP_SW_TRIG_INT_OFFS (0x04) +#define ARMADA_370_XP_INT_SOC_ERR_0_CAUSE_OFFS (0x20) +#define ARMADA_370_XP_INT_SOC_ERR_1_CAUSE_OFFS (0x24)#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)#define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS (0x34)#define ARMADA_370_XP_INT_SOURCE_CTL (irq) (0x100 + irq*4)@@ -130,6 +132,8 @@#define ARMADA_370_XP_CPU_INTACK_OFFS (0x44)#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C) +#define ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF (0x50) +#define ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF (0x54)#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)#define ARMADA_370_XP_INT_CAUSE_PERF (cpu) (1 << cpu)@@ -146,6 +150,8 @@static void __iomem *per_cpu_int_base;static void __iomem *main_int_base;static struct irq_domain *armada_370_xp_mpic_domain; +static struct irq_domain *armada_370_xp_soc_err_domain; +static unsigned int soc_err_irq_num_regs;static u32 doorbell_mask_reg;static int parent_irq;#ifdef CONFIG_PCI_MSI @@ -156,6 +162,8 @@ static DEFINE_MUTEX(msi_used_lock);static phys_addr_t msi_doorbell_addr;#endif +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq);drivers/irqchip/irq-armada-370-xp.c static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Quoted 7 lines
Quoted 4 lines
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into _better shape_ as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.
Quoted 95 lines
Quoted 95 lines
Quoted 94 lines
Quoted 81 lines
drivers/irqchip/irq-armada-370-xp.c +65
-2+ +/* Unmask summary SoC Error Interrupt */ +if (soc_err_irq_num_regs >0 ) +writel (4 , per_cpu_int_base +ARMADA_370_XP_INT_CLEAR_MASK_OFFS ); +ipi_resume (); }@@ -546,8 +575,8 @@ static struct irq_chip armada_370_xp_irq_chip = {static int armada_370_xp_mpic_irq_map (struct irq_domain *h,unsigned int virq,irq_hw_number_t hw) { -/* IRQs 0 and 1 cannot be mapped, they are handled internally */ -if (hw <=1 ) +/* IRQs 0, 1 and 4 cannot be mapped, they are handled internally */ +if (hw <=1 || hw ==4 )return -EINVAL ;armada_370_xp_irq_mask (irq_get_irq_data (virq));@@ -577,6 +606,99 @@ static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { .xlate = irq_domain_xlate_onecell, }; +static DEFINE_RAW_SPINLOCK (armada_370_xp_soc_err_lock); + +static void armada_370_xp_soc_err_irq_mask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask &= ~BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d) +{ +irq_hw_number_t hwirq =irqd_to_hwirq (d); +u32 reg, mask; + + reg = hwirq >=32 ?ARMADA_370_XP_INT_SOC_ERR_1_MASK_OFF + :ARMADA_370_XP_INT_SOC_ERR_0_MASK_OFF ; + +raw_spin_lock (&armada_370_xp_soc_err_lock); + mask =readl (per_cpu_int_base + reg); + mask |=BIT (hwirq %32 ); +writel (mask, per_cpu_int_base + reg); +raw_spin_unlock (&armada_370_xp_soc_err_lock); +} + +static int armada_370_xp_soc_err_irq_mask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_mask (d); +return 0 ; +} + +static int armada_370_xp_soc_err_irq_unmask_on_cpu (void *par) +{ +struct irq_data *d = par; +armada_370_xp_soc_err_irq_unmask (d); +return 0 ; +} + +static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); +drivers/irqchip/irq-armada-370-xp.c static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { · line 676
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock (); + +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +}Aren't these per-CPU interrupts anyway? What does it mean to set their affinity? /me rolls eyes...
Quoted 7 lines
Quoted 4 lines
Yes, they are per-CPU interrupts. But to mask or unmask particular interrupt for specific CPU is possible only from that CPU. CPU 0 just cannot move interrupt from CPU 0 to CPU 1. CPU 0 can only mask that interrupt and CPU 1 has to unmask it.
And that's no different form other per-CPU interrupts that have the exact same requirements. NAK to this sort of hacks.
You forgot to mention in your previous email how to do it, right? So we are waiting...
Re: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 48 lines · Pali Rohár
Quoted 46 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 43 lines · Pali Rohár
Quoted 34 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 16 lines · Pali Rohár
drivers/irqchip/irq-armada-370-xp.c +8
-0+static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq);drivers/irqchip/irq-armada-370-xp.c static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Quoted 7 lines
Quoted 2 lines
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into
_bettershape_as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.
Sorry, but that's not how this works. You are the one willing to change a sub-par piece of code, you get to make it better. You obviously have the means (the HW) and the incentive (these patches). But you don't get to make something even more unmaintainable because you're unwilling to do some extra work.
If you're unhappy with my position, that's fine. I suggest you take it with Thomas, and maybe even Linus. As I suggested before, you can also post a patch removing me as the irqchip maintainer. I'm sure that will spark an interesting discussion.
Quoted 31 lines
Quoted 31 lines
Quoted 30 lines
Quoted 17 lines
Quoted 17 lines
drivers/irqchip/irq-armada-370-xp.c +17
-0+static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); + +/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock ();Quoted 8 lines
drivers/irqchip/irq-armada-370-xp.c static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { · line 676
Quoted 5 lines
drivers/irqchip/irq-armada-370-xp.c +5
-0+ +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +}Aren't these per-CPU interrupts anyway? What does it mean to set their affinity? /me rolls eyes...
Yes, they are per-CPU interrupts. But to mask or unmask particular interrupt for specific CPU is possible only from that CPU. CPU 0 just cannot move interrupt from CPU 0 to CPU 1. CPU 0 can only mask that interrupt and CPU 1 has to unmask it.
Quoted 5 lines
Quoted 2 lines
And that's no different form other per-CPU interrupts that have the exact same requirements. NAK to this sort of hacks.
You forgot to mention in your previous email how to do it, right? So we are waiting...
I didn't forget. I explained that it should be handled just like any other per-CPU interrupt. There is plenty of example of how to do that in the tree (timers, for example), and if you had even looked at it, you'd have seen that your approach most probably results in an arbitrary pointer dereference on anything but CPU0 because the requesting driver knows nothing about per-CPU interrupts.
But you're obviously trying to make a very different point here. I'll let you play that game for as long as you want, no skin off my nose. Maybe in the future, you'll be more interested in actively collaborating on the kernel code instead of throwing your toys out of the pram.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Pali Rohár
maintainerRe: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 64 lines · On Saturday 07 May 2022 10:01:52 Marc Zyngier wrote:
Quoted 48 lines · Pali Rohár
Quoted 46 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 43 lines · Pali Rohár
Quoted 34 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 16 lines · Pali Rohár
drivers/irqchip/irq-armada-370-xp.c +8
-0+static void armada_370_xp_soc_err_irq_unmask (struct irq_data *d); +static inline bool is_percpu_irq (irq_hw_number_t irq) {if (irq <=ARMADA_370_XP_MAX_PER_CPU_IRQS )@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void)armada_370_xp_irq_unmask (data); } +/* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ +for (irq =0 ; irq < soc_err_irq_num_regs *32 ; irq++) { +struct irq_data *data; +int virq; + + virq =irq_linear_revmap (armada_370_xp_soc_err_domain, irq);drivers/irqchip/irq-armada-370-xp.c static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
drivers/irqchip/irq-armada-370-xp.c +10
-0+if (virq ==0 ) +continue ; + + data =irq_get_irq_data (virq); + +if (!irq_percpu_is_enabled (virq)) +continue ; + +armada_370_xp_soc_err_irq_unmask (data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Quoted 7 lines
Quoted 2 lines
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into
_bettershape_as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.Sorry, but that's not how this works. You are the one willing to change a sub-par piece of code, you get to make it better. You obviously have the means (the HW) and the incentive (these patches). But you don't get to make something even more unmaintainable because you're unwilling to do some extra work.
Quoted 4 lines
If you're unhappy with my position, that's fine. I suggest you take it with Thomas, and maybe even Linus. As I suggested before, you can also post a patch removing me as the irqchip maintainer. I'm sure that will spark an interesting discussion.
You have already suggested it in email [1] but apparently you are _not_ maintainer of mvebu pci controller. get_maintainer.pl for part about which you have talked in [1] says:
$ ./scripts/get_maintainer.pl -f drivers/pci/controller/pci-aardvark.c Thomas Petazzoni <[email protected]> (maintainer:PCI DRIVER FOR AARDVARK (Marvell Armada 3700)) "Pali Rohár" <[email protected]> (maintainer:PCI DRIVER FOR AARDVARK (Marvell Armada 3700)) Lorenzo Pieralisi <[email protected]> (supporter:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS) Rob Herring <[email protected]> (reviewer:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS) "Krzysztof Wilczyński" <[email protected]> (reviewer:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS) Bjorn Helgaas <[email protected]> (supporter:PCI SUBSYSTEM) [email protected] (open list:PCI DRIVER FOR AARDVARK (Marvell Armada 3700)) [email protected] (moderated list:PCI DRIVER FOR AARDVARK (Marvell Armada 3700)) [email protected] (open list)
So I do not have to remove anything, you are _not_ on that list. On the other hand, Thomas Petazzoni is on this list...
Quoted 44 lines
Quoted 31 lines
Quoted 31 lines
Quoted 30 lines
Quoted 17 lines
Quoted 17 lines
drivers/irqchip/irq-armada-370-xp.c +17
-0+static int armada_xp_soc_err_irq_set_affinity (struct irq_data *d, +const struct cpumask *mask, +bool force) +{ +unsigned int cpu; + +cpus_read_lock (); + +/* First disable IRQ on all cores */ +for_each_online_cpu (cpu) +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_mask_on_cpu, d, true); + +/* Select a single core from the affinity mask which is online */ + cpu =cpumask_any_and (mask, cpu_online_mask); +smp_call_on_cpu (cpu, armada_370_xp_soc_err_irq_unmask_on_cpu, d, true); + +cpus_read_unlock ();Quoted 8 lines
drivers/irqchip/irq-armada-370-xp.c static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { · line 676
Quoted 5 lines
drivers/irqchip/irq-armada-370-xp.c +5
-0+ +irq_data_update_effective_affinity (d,cpumask_of (cpu)); + +return IRQ_SET_MASK_OK ; +}Aren't these per-CPU interrupts anyway? What does it mean to set their affinity? /me rolls eyes...
Yes, they are per-CPU interrupts. But to mask or unmask particular interrupt for specific CPU is possible only from that CPU. CPU 0 just cannot move interrupt from CPU 0 to CPU 1. CPU 0 can only mask that interrupt and CPU 1 has to unmask it.
Quoted 5 lines
Quoted 2 lines
And that's no different form other per-CPU interrupts that have the exact same requirements. NAK to this sort of hacks.
You forgot to mention in your previous email how to do it, right? So we are waiting...
I didn't forget. I explained that it should be handled just like any other per-CPU interrupt. There is plenty of example of how to do that in the tree (timers, for example), and if you had even looked at it, you'd have seen that your approach most probably results in an arbitrary pointer dereference on anything but CPU0 because the requesting driver knows nothing about per-CPU interrupts.
Quoted 7 lines
But you're obviously trying to make a very different point here. I'll let you play that game for as long as you want, no skin off my nose. Maybe in the future, you'll be more interested in actively collaborating on the kernel code instead of throwing your toys out of the pram.
Thanks,
The only _toy_ here is your broken mvebu board which your ego was unable to fix, and you have put it into recycling pile [2] and since than for months you are trying to reject every change or improvement in mvebu drivers and trying to find out a way how to remove all mvebu code, like if you were not able to fix your toy, then broke it also to all other people. You have already expressed this, but I'm not going to search emails more and find these your statements.
Sorry, I'm stopping here. This is just a prove that you are not qualified in reviewing mvebu code.
[1] - https://lore.kernel.org/linux-pci/[email protected]/
[2] - https://lore.kernel.org/linux-pci/[email protected]/
Re: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 71 lines · Pali Rohár
Quoted 64 lines · On Saturday 07 May 2022 10:01:52 Marc Zyngier wrote:
Quoted 48 lines · Pali Rohár
Quoted 46 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 43 lines · Pali Rohár
Quoted 34 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 16 lines · Pali Rohár
(quoted diff) +8
-0+static void armada_370_xp_soc_err_irq_unmask(struct irq_data *d); + static inline bool is_percpu_irq(irq_hw_number_t irq) { if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void) armada_370_xp_irq_unmask(data); } + /* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ + for (irq = 0; irq < soc_err_irq_num_regs * 32; irq++) { + struct irq_data *data; + int virq; + + virq = irq_linear_revmap(armada_370_xp_soc_err_domain, irq);Review static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
(quoted diff) +10
-0+ if (virq == 0) + continue; + + data = irq_get_irq_data(virq); + + if (!irq_percpu_is_enabled(virq)) + continue; + + armada_370_xp_soc_err_irq_unmask(data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Quoted 7 lines
Quoted 2 lines
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into
_bettershape_as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.Sorry, but that's not how this works. You are the one willing to change a sub-par piece of code, you get to make it better. You obviously have the means (the HW) and the incentive (these patches). But you don't get to make something even more unmaintainable because you're unwilling to do some extra work.
Quoted 4 lines
If you're unhappy with my position, that's fine. I suggest you take it with Thomas, and maybe even Linus. As I suggested before, you can also post a patch removing me as the irqchip maintainer. I'm sure that will spark an interesting discussion.
Quoted 5 lines
You have already suggested it in email [1] but apparently you are _not_ maintainer of mvebu pci controller. get_maintainer.pl for part about which you have talked in [1] says:
$ ./scripts/get_maintainer.pl -f drivers/pci/controller/pci-aardvark.c
Remind me which file this patch is touching?
Quoted 7 lines
The only _toy_ here is your broken mvebu board which your ego was unable to fix, and you have put it into recycling pile [2] and since than for months you are trying to reject every change or improvement in mvebu drivers and trying to find out a way how to remove all mvebu code, like if you were not able to fix your toy, then broke it also to all other people. You have already expressed this, but I'm not going to search emails more and find these your statements.
At this stage, this is pure paranoia. Do you think I am so emotionally attached to HW purity that I would plot the annihilation of some ugly platform?
Quoted 2 lines
Sorry, I'm stopping here. This is just a prove that you are not qualified in reviewing mvebu code.
Happy not to have to review this code. Just stop Cc'ing me on your patches, and don't expect me to merge any IRQ related patches coming from you.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Pali Rohár
maintainerRe: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
Quoted 73 lines · On Saturday 07 May 2022 10:42:49 Marc Zyngier wrote:
Quoted 71 lines · Pali Rohár
Quoted 68 lines · On Saturday 07 May 2022 10:01:52 Marc Zyngier wrote:
Quoted 48 lines · Pali Rohár
Quoted 46 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 43 lines · Pali Rohár
Quoted 34 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 16 lines · Pali Rohár
(quoted diff) +8
-0+static void armada_370_xp_soc_err_irq_unmask(struct irq_data *d); + static inline bool is_percpu_irq(irq_hw_number_t irq) { if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void) armada_370_xp_irq_unmask(data); } + /* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ + for (irq = 0; irq < soc_err_irq_num_regs * 32; irq++) { + struct irq_data *data; + int virq; + + virq = irq_linear_revmap(armada_370_xp_soc_err_domain, irq);Review static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
(quoted diff) +10
-0+ if (virq == 0) + continue; + + data = irq_get_irq_data(virq); + + if (!irq_percpu_is_enabled(virq)) + continue; + + armada_370_xp_soc_err_irq_unmask(data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Quoted 7 lines
Quoted 2 lines
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into
_bettershape_as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.Sorry, but that's not how this works. You are the one willing to change a sub-par piece of code, you get to make it better. You obviously have the means (the HW) and the incentive (these patches). But you don't get to make something even more unmaintainable because you're unwilling to do some extra work.
If you're unhappy with my position, that's fine. I suggest you take it with Thomas, and maybe even Linus. As I suggested before, you can also post a patch removing me as the irqchip maintainer. I'm sure that will spark an interesting discussion.
Quoted 7 lines
Quoted 5 lines
You have already suggested it in email [1] but apparently you are _not_ maintainer of mvebu pci controller. get_maintainer.pl for part about which you have talked in [1] says:
$ ./scripts/get_maintainer.pl -f drivers/pci/controller/pci-aardvark.cRemind me which file this patch is touching?
So read again what you have presented in the past, in the email to which you have referenced. I sent link to that your email in previous email.
Or you absolutely incompetent and I should have remind also previous email to which you wrote your reaction?
Quoted 9 lines
Quoted 7 lines
The only _toy_ here is your broken mvebu board which your ego was unable to fix, and you have put it into recycling pile [2] and since than for months you are trying to reject every change or improvement in mvebu drivers and trying to find out a way how to remove all mvebu code, like if you were not able to fix your toy, then broke it also to all other people. You have already expressed this, but I'm not going to search emails more and find these your statements.
At this stage, this is pure paranoia.
No, just pure reality of your behavior of what you are doing and what you are saying.
Quoted 3 lines
Do you think I am so emotionally
attached to HW purity that I would plot the annihilation of some ugly platform?
I do not think. You personally, have presented this statement, and I'm just reminding it to you like you have asked for it.
Quoted 4 lines
Quoted 2 lines
Sorry, I'm stopping here. This is just a prove that you are not qualified in reviewing mvebu code.
Happy not to have to review this code.
You are doing it for more than one year. Are you happy with it? Seem absolutely.
Quoted 1 line
Just stop Cc'ing me on your patches
As there no progress from your side, nor change of your behavior from more than one year, I'm accepting this offer.
This is my last email to you and I'm stopping right now to read your emails.
I'm not obligated to remind you everything what you are asking just because you are lazy to find you what you have wrote in the past.
Quoted 7 lines
and don't expect me to merge any IRQ related patches coming from you.
M.Signature · 1 line
Without deviation from the norm, progress is not possible.
Rob Herring
maintainerRe: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
In reply to Pali RohárQuoted 74 lines · Pali Rohár
Quoted 64 lines · On Saturday 07 May 2022 10:01:52 Marc Zyngier wrote:
Quoted 48 lines · Pali Rohár
Quoted 46 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 43 lines · Pali Rohár
Quoted 34 lines · On Friday 06 May 2022 19:19:46 Marc Zyngier wrote:
Quoted 16 lines · Pali Rohár
(quoted diff) +8
-0+static void armada_370_xp_soc_err_irq_unmask(struct irq_data *d); + static inline bool is_percpu_irq(irq_hw_number_t irq) { if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)@@ -509,6 +517,27 @@ static void armada_xp_mpic_reenable_percpu(void) armada_370_xp_irq_unmask(data); } + /* Re-enable per-CPU SoC Error interrupts that were enabled before suspend */ + for (irq = 0; irq < soc_err_irq_num_regs * 32; irq++) { + struct irq_data *data; + int virq; + + virq = irq_linear_revmap(armada_370_xp_soc_err_domain, irq);Review static void armada_xp_mpic_reenable_percpu(void) · line 535
Quoted 10 lines
(quoted diff) +10
-0+ if (virq == 0) + continue; + + data = irq_get_irq_data(virq); + + if (!irq_percpu_is_enabled(virq)) + continue; + + armada_370_xp_soc_err_irq_unmask(data); + }So you do this loop and all these lookups, both here and in the resume function (duplicated code!) just to be able to call the unmask function? This would be better served by two straight writes of the mask register, which you'd conveniently save on suspend.
Quoted 2 lines
Yes, you have only duplicated the existing logic. But surely there is something better to do.
Yes, I just used existing logic.
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Quoted 7 lines
Quoted 2 lines
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into
_bettershape_as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.Sorry, but that's not how this works. You are the one willing to change a sub-par piece of code, you get to make it better. You obviously have the means (the HW) and the incentive (these patches). But you don't get to make something even more unmaintainable because you're unwilling to do some extra work.
Quoted 4 lines
If you're unhappy with my position, that's fine. I suggest you take it with Thomas, and maybe even Linus. As I suggested before, you can also post a patch removing me as the irqchip maintainer. I'm sure that will spark an interesting discussion.
You have already suggested it in email [1] but apparently you are _not_ maintainer of mvebu pci controller. get_maintainer.pl for part about which you have talked in [1] says:
Quoted 5 lines
$ ./scripts/get_maintainer.pl -f drivers/pci/controller/pci-aardvark.c Thomas Petazzoni <[email protected]> (maintainer:PCI DRIVER FOR AARDVARK (Marvell Armada 3700)) "Pali Rohár" <[email protected]> (maintainer:PCI DRIVER FOR AARDVARK (Marvell Armada 3700)) Lorenzo Pieralisi <[email protected]> (supporter:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS) Rob Herring <[email protected]> (reviewer:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS)
Please just refactor the code in question. You've wasted more time arguing about it than it would take to do. Having done a bit of PCI refactoring, I can tell you hardly anyone else does. I can barely even get comments/acks on refactoring until I break platforms (which happens a lot). Maintainers have no other leverage other than what Marc pointed out.
In any case, I think there's no way the PCI maintainers will take this as-is at this point.
Rob
Re: [PATCH 2/6] irqchip/armada-370-xp: Implement SoC Error interrupts
In reply to Pali RohárPali,
Quoted 11 lines · Pali Rohár
Quoted 5 lines · On Friday 06 May 2022 19:47:25 Marc Zyngier wrote:
Quoted 2 lines
I'm not rewriting driver or doing big refactor of it, as this is not in the scope of the PCIe AER interrupt support.
Fair enough. By the same logic, I'm not taking any change to the driver until it is put in a better shape. Your call.
If you are maintainer of this code then it is expected from _you_ to move the current code into
_bettershape_as you wrote and expect. And then show us exactly, how new changes in this driver should look like, in examples.
this is not how kernel development works.
Maintainers are not the servants who mop up the mess which random people dump into the tree. They are gatekeepers and one of their duties is to prevent that mess is created or existing mess is proliferated.
You are asking the maintainer to take your changes, deal with the fallout and maintain them for a long time free of charge. So it's a very reasonable request from a maintainer to ask for refactoring of existing code before adding new functionality to it.
With such a request the refactoring becomes scope of your work, whether you and your manager like it or not. If you don't want to do that extra work, then don't expect maintainers to care about your fancy new features.
Marc gave you very reasonable and consice directions how the code should be reworked. He spent a lot of time explaining it to you. Again, free of charge. Now you expect him to do your homework free of charge, so you can get your feature merged? Nice try.
Thanks,
Thomas
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Pali Rohár
maintainerPATCH 3/6 ARM: dts: armada-38x.dtsi: Add node for MPIC SoC Error IRQ controller
In reply to Pali RohárIt is child of the MPIC IRQ controller.
--- arch/arm/boot/dts/armada-38x.dtsi | 5 +++++ 1 file changed, 5 insertions(+)
arch/arm/boot/dts/armada-38x.dtsi +5 -0
@@ -398,6 +398,11 @@ interrupt-controller; msi-controller; interrupts = <GIC_PPI 15 IRQ_TYPE_LEVEL_HIGH>; + + soc_err: interrupt-controller@20 { + interrupt-controller; + #interrupt-cells = <1>; + }; }; timer: timer@20300 {
Signature · 1 line
2.20.1
Pali Rohár
maintainerPATCH 4/6 dt-bindings: PCI: mvebu: Update information about summary interrupt
In reply to Pali Rohár--- Documentation/devicetree/bindings/pci/mvebu-pci.txt | 1 + 1 file changed, 1 insertion(+)
Documentation/devicetree/bindings/pci/mvebu-pci.txt +1 -0
@@ -83,6 +83,7 @@ and the following optional properties: specified will default to 100ms, as required by the PCIe specification. - interrupt-names: list of interrupt names, supported are: - "intx" - interrupt line triggered by one of the legacy interrupt + - "summary" - interrupt line triggered by any event, including "intx" - interrupts or interrupts-extended: List of the interrupt sources which corresponding to the "interrupt-names". If non-empty then also additional 'interrupt-controller' subnode must be defined.
Signature · 1 line
2.20.1
Pali Rohár
maintainerPATCH 5/6 PCI: mvebu: Implement support for interrupts on emulated bridge
In reply to Pali RohárThis adds support for PME and ERR interrupts reported by emulated bridge (for PME and AER kernel drivers) via new Root Port irq chip as these interrupts from PCIe Root Ports are handled by mvebu hardware completely separately from INTx and MSI interrupts send by real PCIe devices.
With this change, kernel PME and AER drivers start working as they can acquire required interrupt lines (provided by mvebu rp virtual irq chip).
Note that for this support, device tree files has to be properly adjusted to provide "interrupts" or "interrupts-extended" property with summary interrupt source and "interrupt-names" property with "summary" string.
If device tree files do not provide these properties then driver would work as before and would not provide interrupts on emulated bridge, like before.
--- drivers/pci/controller/pci-mvebu.c | 208 ++++++++++++++++++++++++++--- 1 file changed, 189 insertions(+), 19 deletions(-)
drivers/pci/controller/pci-mvebu.c +189 -19
@@ -56,8 +56,16 @@#define PCIE_CONF_DATA_OFF 0x18fc#define PCIE_INT_CAUSE_OFF 0x1900#define PCIE_INT_UNMASK_OFF 0x1910 +#define PCIE_INT_DET_COR BIT(8) +#define PCIE_INT_DET_NONFATAL BIT(9) +#define PCIE_INT_DET_FATAL BIT(10) +#define PCIE_INT_ERR_FATAL BIT(16) +#define PCIE_INT_ERR_NONFATAL BIT(17) +#define PCIE_INT_ERR_COR BIT(18)#define PCIE_INT_INTX (i) BIT(24+i)#define PCIE_INT_PM_PME BIT(28) +#define PCIE_INT_DET_MASK (PCIE_INT_DET_COR | PCIE_INT_DET_NONFATAL | PCIE_INT_DET_FATAL) +#define PCIE_INT_ERR_MASK (PCIE_INT_ERR_FATAL | PCIE_INT_ERR_NONFATAL | PCIE_INT_ERR_COR)#define PCIE_INT_ALL_MASK GENMASK(31, 0)#define PCIE_CTRL_OFF 0x1a00#define PCIE_CTRL_X1_MODE 0x0001
@@ -120,9 +128,12 @@ struct mvebu_pcie_port {struct resource regs;u8 slot_power_limit_value;u8 slot_power_limit_scale; +struct irq_domain *rp_irq_domain;struct irq_domain *intx_irq_domain;raw_spinlock_t irq_lock; +int summary_irq;int intx_irq; +bool pme_pending; };static inline void mvebu_writel (struct mvebu_pcie_port *port,u32 val,u32 reg)
@@ -321,12 +332,27 @@ static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port)/* Clear all interrupt causes. */ mvebu_writel (port, ~PCIE_INT_ALL_MASK ,PCIE_INT_CAUSE_OFF ); -/* Check if "intx" interrupt was specified in DT. */ -if (port->intx_irq >0 ) +/* +* Unmask all error interrupts which are internally generated. +* They cannot be disabled by SERR# Enable bit in PCI Command register, +* see Figure 6-3: Pseudo Logic Diagram for Error Message Controls in +* PCIe base specification. +* Internally generated mvebu interrupts are reported via mvebu summary +* interrupt which requires "summary" interrupt to be specified in DT. +*/ +if (port->summary_irq >0 ) { + unmask =mvebu_readl (port,PCIE_INT_UNMASK_OFF ); + unmask |=PCIE_INT_DET_MASK ; +mvebu_writel (port, unmask,PCIE_INT_UNMASK_OFF ); + } + +/* Check if "summary" or "intx" interrupt was specified in DT. */ +if (port->summary_irq >0 || port->intx_irq >0 )return ;/* -* Fallback code when "intx" interrupt was not specified in DT: +* Fallback code when neither "summary" interrupt, nor "intx" +* interrupt was specified in DT: * Unmask all legacy INTx interrupts as driver does not provide a way * for masking and unmasking of individual legacy INTx interrupts. * Legacy INTx are reported via one shared GIC source and therefore
@@ -603,11 +629,16 @@ mvebu_pci_bridge_emul_base_conf_read(struct pci_bridge_emul *bridge,case PCI_INTERRUPT_LINE : {/* * From the whole 32bit register we support reading from HW only -* one bit: PCI_BRIDGE_CTL_BUS_RESET. +* two bits: PCI_BRIDGE_CTL_BUS_RESET and PCI_BRIDGE_CTL_SERR. * Other bits are retrieved only from emulated config buffer. */ __le32 *cfgspace = (__le32 *)&bridge->conf;u32 val =le32_to_cpu (cfgspace[PCI_INTERRUPT_LINE /4 ]); +if ((mvebu_readl (port,PCIE_INT_UNMASK_OFF ) & +PCIE_INT_ERR_MASK ) ==PCIE_INT_ERR_MASK ) + val |=PCI_BRIDGE_CTL_SERR <<16 ; +else + val &= ~(PCI_BRIDGE_CTL_SERR <<16 );if (mvebu_readl (port,PCIE_CTRL_OFF ) &PCIE_CTRL_MASTER_HOT_RESET ) val |=PCI_BRIDGE_CTL_BUS_RESET <<16 ;else
@@ -675,6 +706,11 @@ mvebu_pci_bridge_emul_pcie_conf_read(struct pci_bridge_emul *bridge,break ; } +case PCI_EXP_RTCTL : + *value = (mvebu_readl (port,PCIE_INT_UNMASK_OFF ) & +PCIE_INT_PM_PME ) ?PCI_EXP_RTCTL_PMEIE :0 ; +break ; +case PCI_EXP_RTSTA : *value =mvebu_readl (port,PCIE_RC_RTSTA );break ;
@@ -780,6 +816,14 @@ mvebu_pci_bridge_emul_base_conf_write(struct pci_bridge_emul *bridge,break ;case PCI_INTERRUPT_LINE : +if (mask & (PCI_BRIDGE_CTL_SERR <<16 )) { +u32 unmask =mvebu_readl (port,PCIE_INT_UNMASK_OFF ); +if (new & (PCI_BRIDGE_CTL_SERR <<16 )) + unmask |=PCIE_INT_ERR_MASK ; +else + unmask &= ~PCIE_INT_ERR_MASK ; +mvebu_writel (port, unmask,PCIE_INT_UNMASK_OFF ); + }if (mask & (PCI_BRIDGE_CTL_BUS_RESET <<16 )) {u32 ctrl =mvebu_readl (port,PCIE_CTRL_OFF );if (new & (PCI_BRIDGE_CTL_BUS_RESET <<16 ))
@@ -838,10 +882,25 @@ mvebu_pci_bridge_emul_pcie_conf_write(struct pci_bridge_emul *bridge,* PME Status bit in Root Status Register (PCIE_RC_RTSTA) * is read-only and can be cleared only by writing 0b to the * Interrupt Cause RW0C register (PCIE_INT_CAUSE_OFF). So -* clear PME via Interrupt Cause. +* clear PME via Interrupt Cause and also set port->pme_pending +* variable to false value to start processing PME interrupts +* in interrupt handler again. */ -if (new &PCI_EXP_RTSTA_PME ) +if (new &PCI_EXP_RTSTA_PME ) {mvebu_writel (port, ~PCIE_INT_PM_PME ,PCIE_INT_CAUSE_OFF ); + port->pme_pending = false; + } +break ; + +case PCI_EXP_RTCTL : +if (mask &PCI_EXP_RTCTL_PMEIE ) { +u32 unmask =mvebu_readl (port,PCIE_INT_UNMASK_OFF ); +if (new &PCI_EXP_RTCTL_PMEIE ) + unmask |=PCIE_INT_PM_PME ; +else + unmask &= ~PCIE_INT_PM_PME ; +mvebu_writel (port, unmask,PCIE_INT_UNMASK_OFF ); + }break ;case PCI_EXP_DEVCTL2 :
@@ -924,6 +983,14 @@ static int mvebu_pci_bridge_emul_init(struct mvebu_pcie_port *port) bridge_flags |=PCI_BRIDGE_EMUL_NO_IO_FORWARD ; } +/* +* Interrupts on emulated bridge are supported only when "summary" +* interrupt was specified in DT. Without it emulated bridge cannot +* emulate interrupts. +*/ +if (port->summary_irq >0 ) + bridge->conf.intpin =PCI_INTERRUPT_INTA ; +/* * Older mvebu hardware provides PCIe Capability structure only in * version 1. New hardware provides it in version 2.
@@ -1071,6 +1138,26 @@ static const struct irq_domain_ops mvebu_pcie_intx_irq_domain_ops = { .xlate = irq_domain_xlate_onecell, }; +static struct irq_chip rp_irq_chip = { + .name ="mvebu-rp" , +}; + +static int mvebu_pcie_rp_irq_map (struct irq_domain *h, +unsigned int virq,irq_hw_number_t hwirq) +{ +struct mvebu_pcie_port *port = h->host_data; + +irq_set_chip_and_handler (virq, &rp_irq_chip, handle_simple_irq); +irq_set_chip_data (virq, port); + +return 0 ; +} + +static const struct irq_domain_ops mvebu_pcie_rp_irq_domain_ops = { + .map = mvebu_pcie_rp_irq_map, + .xlate = irq_domain_xlate_onecell, +}; +static int mvebu_pcie_init_irq_domain (struct mvebu_pcie_port *port) {struct device *dev = &port->pcie->pdev->dev;
@@ -1093,6 +1180,22 @@ static int mvebu_pcie_init_irq_domain(struct mvebu_pcie_port *port)return -ENOMEM ; } +/* +* When "summary" interrupt was not specified in DT then there is no support +* for interrupts on emulated root bridge. So skip following initialization. +*/ +if (port->summary_irq <=0 ) +return 0 ; + + port->rp_irq_domain =irq_domain_add_linear (NULL ,1 , + &mvebu_pcie_rp_irq_domain_ops, + port); +if (!port->rp_irq_domain) { +irq_domain_remove (port->intx_irq_domain); +dev_err (dev,"Failed to add Root Port IRQ domain for %s\n" , port->name); +return -ENOMEM ; + } +return 0 ; }
@@ -1110,6 +1213,34 @@ static void mvebu_pcie_irq_handler(struct irq_desc *desc) unmask =mvebu_readl (port,PCIE_INT_UNMASK_OFF ); status = cause & unmask; +/* Process PME interrupt */ +if ((status &PCIE_INT_PM_PME ) && !port->pme_pending) { +/* +* Do not clear PME interrupt bit in Cause Register as it +* invalidates also content of Root Status Register. Instead +* set port->pme_pending variable to true to indicate that +* next time PME interrupt should be ignored until variable +* is back to the false value. +*/ + port->pme_pending = true; +if (generic_handle_domain_irq (port->rp_irq_domain,0 ) == -EINVAL ) +dev_err_ratelimited (dev,"unhandled PME IRQ\n" ); + } + +/* Process ERR interrupt */ +if (status &PCIE_INT_ERR_MASK ) { +mvebu_writel (port, ~PCIE_INT_ERR_MASK ,PCIE_INT_CAUSE_OFF ); +if (generic_handle_domain_irq (port->rp_irq_domain,0 ) == -EINVAL ) +dev_err_ratelimited (dev,"unhandled ERR IRQ\n" ); + } + +/* Process local ERR interrupt */ +if (status &PCIE_INT_DET_MASK ) { +mvebu_writel (port, ~PCIE_INT_DET_MASK ,PCIE_INT_CAUSE_OFF ); +if (generic_handle_domain_irq (port->rp_irq_domain,0 ) == -EINVAL ) +dev_err_ratelimited (dev,"unhandled ERR IRQ\n" ); + } +/* Process legacy INTx interrupts */ for (i =0 ; i <PCI_NUM_INTX ; i++) {if (!(status &PCIE_INT_INTX (i)))
@@ -1124,9 +1255,29 @@ static void mvebu_pcie_irq_handler(struct irq_desc *desc)static int mvebu_pcie_map_irq (const struct pci_dev *dev,u8 slot,u8 pin) { -/* Interrupt support on mvebu emulated bridges is not implemented yet */ -if (dev->bus->number ==0 ) -return 0 ;/* Proper return code 0 == NO_IRQ */ +struct mvebu_pcie_port *port; +struct mvebu_pcie *pcie; + +if (dev->bus->number ==0 ) { +/* +* Each emulated root bridge for every mvebu port has its own +* Root Port irq chip and irq domain. Argument pin is the INTx +* pin (1=INTA, 2=INTB, 3=INTC, 4=INTD) and hwirq for function +* irq_create_mapping() is indexed from zero. +*/ + pcie = dev->bus->sysdata; + port =mvebu_pcie_find_port (pcie, dev->bus,PCI_DEVFN (slot,0 )); +if (!port) +return 0 ;/* Proper return code 0 == NO_IRQ */ +/* +* port->rp_irq_domain is available only when "summary" +* interrupt was specified in DT. When is not available then +* interrupts for emulated root bridge are not provided. +*/ +if (port->summary_irq <=0 ) +return 0 ;/* Proper return code 0 == NO_IRQ */ +return irq_create_mapping (port->rp_irq_domain, pin -1 ); + }return of_irq_parse_and_map_pci (dev, slot, pin); }
@@ -1321,17 +1472,32 @@ static int mvebu_pcie_parse_port(struct mvebu_pcie *pcie, }/* -* Old DT bindings do not contain "intx" interrupt +* Old DT bindings do not contain "summary" interrupt * so do not fail probing driver when interrupt does not exist. */ - port->intx_irq =of_irq_get_byname (child,"intx" ); -if (port->intx_irq == -EPROBE_DEFER ) { - ret = port->intx_irq; + port->summary_irq =of_irq_get_byname (child,"summary" ); +if (port->summary_irq == -EPROBE_DEFER ) { + ret = port->summary_irq; goto err; - } -if (port->intx_irq <=0 ) { -dev_warn (dev,"%s: legacy INTx interrupts cannot be masked individually, " -"%pOF does not contain intx interrupt\n" , + }else if (port->summary_irq <=0 ) { +/* +* When "summary" interrupt (which includes also intx) is +* unsupported then fallback to dedicated "intx" interrupt. +* Old DT bindings do not contain "intx" interrupt so do not +* fail probing driver when interrupt does not exist. +*/ + port->intx_irq =of_irq_get_byname (child,"intx" ); +if (port->intx_irq == -EPROBE_DEFER ) { + ret = port->intx_irq; + goto err; + } +if (port->intx_irq <=0 ) { +dev_warn (dev,"%s: legacy INTx interrupts cannot be masked individually, " +"%pOF does not contain intx interrupt\n" , + port->name, child); + } +dev_warn (dev,"%s: interrupts on Root Port are unsupported, " +"%pOF does not contain summary interrupt\n" , port->name, child); }
@@ -1540,7 +1706,8 @@ static int mvebu_pcie_probe(struct platform_device *pdev)for (i =0 ; i < pcie->nports; i++) {struct mvebu_pcie_port *port = &pcie->ports[i]; -int irq = port->intx_irq; +int irq = (port->summary_irq >0 ) ? port->summary_irq : + (port->intx_irq >0 ) ? port->intx_irq :0 ; child = port->dn;if (!child)
@@ -1688,7 +1855,8 @@ static int mvebu_pcie_remove(struct platform_device *pdev)for (i =0 ; i < pcie->nports; i++) {struct mvebu_pcie_port *port = &pcie->ports[i]; -int irq = port->intx_irq; +int irq = (port->summary_irq >0 ) ? port->summary_irq : + (port->intx_irq >0 ) ? port->intx_irq :0 ;if (!port->base)continue ;
@@ -1710,6 +1878,8 @@ static int mvebu_pcie_remove(struct platform_device *pdev)/* Remove IRQ domains. */ if (port->intx_irq_domain)irq_domain_remove (port->intx_irq_domain); +if (port->rp_irq_domain) +irq_domain_remove (port->rp_irq_domain);/* Free config space for emulated root bridge. */ pci_bridge_emul_cleanup (&port->bridge);
Signature · 1 line
2.20.1
Pali Rohár
maintainerPATCH 6/6 ARM: dts: armada-385.dtsi: Add definitions for PCIe summary interrupts
In reply to Pali RohárPCIe summary interrupt is reported by MPIC SoC Error IRQ controller.
--- arch/arm/boot/dts/armada-385.dtsi | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
arch/arm/boot/dts/armada-385.dtsi +12 -8
@@ -69,8 +69,9 @@ reg = <0x0800 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; - interrupt-names = "intx"; - interrupts-extended = <&gic GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>; + interrupt-names = "intx", "summary"; + interrupts-extended = <&gic GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>, + <&soc_err 4>; #interrupt-cells = <1>; ranges = <0x82000000 0 0 0x82000000 0x1 0 1 0 0x81000000 0 0 0x81000000 0x1 0 1 0>;
@@ -97,8 +98,9 @@ reg = <0x1000 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; - interrupt-names = "intx"; - interrupts-extended = <&gic GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>; + interrupt-names = "intx", "summary"; + interrupts-extended = <&gic GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>, + <&soc_err 5>; #interrupt-cells = <1>; ranges = <0x82000000 0 0 0x82000000 0x2 0 1 0 0x81000000 0 0 0x81000000 0x2 0 1 0>;
@@ -125,8 +127,9 @@ reg = <0x1800 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; - interrupt-names = "intx"; - interrupts-extended = <&gic GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>; + interrupt-names = "intx", "summary"; + interrupts-extended = <&gic GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>, + <&soc_err 15>; #interrupt-cells = <1>; ranges = <0x82000000 0 0 0x82000000 0x3 0 1 0 0x81000000 0 0 0x81000000 0x3 0 1 0>;
@@ -156,8 +159,9 @@ reg = <0x2000 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; - interrupt-names = "intx"; - interrupts-extended = <&gic GIC_SPI 71 IRQ_TYPE_LEVEL_HIGH>; + interrupt-names = "intx", "summary"; + interrupts-extended = <&gic GIC_SPI 71 IRQ_TYPE_LEVEL_HIGH>, + <&soc_err 16>; #interrupt-cells = <1>; ranges = <0x82000000 0 0 0x82000000 0x4 0 1 0 0x81000000 0 0 0x81000000 0x4 0 1 0>;
Signature · 1 line
2.20.1
Pali Rohár
maintainerQuoted 23 lines · On Friday 06 May 2022 15:40:23 Pali Rohár wrote:
mvebu PCIe PME and AER interrupts are reported via PCIe summary interrupt. PCIe summary interrupt is reported via mvebu MPIC SoC error summary interrupt. And MPIC SoC error summary interrupt is reported via MPIC IRQ 4.
This patch series implements support for interrupts in MPIC SoC error hierarchy in
irq-armada-370-xp.cdriver and support for interrupts in mvebu PCIe hierarchy inpci-mvebu.c.Finally PCIe PME and AER interrupts are routed to the correct PCIe Root Port, which allows kernel PME and AER drivers to take care of them.
Tested on A385 board and kernel PME and AER drivers works correctly:
[ 0.898482] pcieport 0000:00:01.0: PME: Signaling with IRQ 61 [ 0.904422] pcieport 0000:00:01.0: AER: enabled with IRQ 61 [ 0.910113] pcieport 0000:00:02.0: enabling device (0140 -> 0142) [ 0.916299] pcieport 0000:00:02.0: PME: Signaling with IRQ 62 [ 0.922216] pcieport 0000:00:02.0: AER: enabled with IRQ 62 [ 0.927917] pcieport 0000:00:03.0: enabling device (0140 -> 0142) [ 0.934090] pcieport 0000:00:03.0: PME: Signaling with IRQ 63 [ 0.940006] pcieport 0000:00:03.0: AER: enabled with IRQ 63
Quoted 1 line
This change finally allows to debug PCIe issues on A385 boards.
FYI I tested that AER errors are now really handled by kernel AER driver:
[ 2733.258661] pcieport 0000:00:02.0: AER: Multiple Uncorrected (Non-Fatal) error received: 0000:02:00.0 [ 2733.258661] pcieport 0000:00:01.0: AER: Multiple Corrected error received: 0000:01:00.0 [ 2733.258682] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID) [ 2733.267932] ath10k_pci 0000:02:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID) [ 2733.275956] pcieport 0000:00:01.0: device [11ab:6820] error status/mask=00000001/00002000 [ 2733.285547] ath10k_pci 0000:02:00.0: device [168c:003c] error status/mask=00100000/00000000 [ 2733.296876] pcieport 0000:00:01.0: [ 0] RxErr (First) [ 2733.305245] ath10k_pci 0000:02:00.0: [20] UnsupReq (First) [ 2733.305251] ath10k_pci 0000:02:00.0: AER: TLP Header: 30000000 02080030 00000000 00000000 [ 2733.305282] ath10k_pci 0000:02:00.0: AER: can't recover (no error_detected callback) [ 2733.313816] nvme 0000:01:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Transmitter ID) [ 2733.320671] pcieport 0000:00:02.0: AER: device recovery failed [ 2733.327609] nvme 0000:01:00.0: device [1e0f:0001] error status/mask=00001041/00002000 [ 2733.367127] nvme 0000:01:00.0: [ 0] RxErr (First) [ 2733.373591] nvme 0000:01:00.0: [ 6] BadTLP [ 2733.379358] nvme 0000:01:00.0: [12] Timeout [ 2733.385120] nvme 0000:01:00.0: AER: Error of this Agent is reported first
Quoted 20 lines
Pali Rohár (6):
dt-bindings: irqchip: armada-370-xp: Update information about MPIC SoC
Error
irqchip/armada-370-xp: Implement SoC Error interrupts
ARM: dts:armada-38x.dtsi: Add node for MPIC SoC Error IRQ controller
dt-bindings: PCI: mvebu: Update information about summary interrupt
PCI: mvebu: Implement support for interrupts on emulated bridge
ARM: dts:armada-385.dtsi: Add definitions for PCIe summary interrupts.../marvell,armada-370-xp-mpic.txt | 9 + .../devicetree/bindings/pci/mvebu-pci.txt | 1 + arch/arm/boot/dts/armada-385.dtsi | 20 +- arch/arm/boot/dts/armada-38x.dtsi | 5 + drivers/irqchip/irq-armada-370-xp.c | 213 +++++++++++++++++- drivers/pci/controller/pci-mvebu.c | 208 +++++++++++++++-- 6 files changed, 426 insertions(+), 30 deletions(-)Signature · 1 line
2.20.1