GICv3 ITS fixes for 6.8
Marc Zyngier
maintainercover GICv3 ITS fixes for 6.8
Here's a small collection of patches that I posted separately in the past, but that are all more or less related, so it makes sense to repost them as a series.
It would be good to have them in 6.8, given that at least one of them fixes a pretty bad regression, and the others can lead to unpredictable behaviours.
Marc Zyngier (3):
irqchip/gic-v3-its: Handle non-coherent GICv4 redistributors irqchip/gic-v3-its: Restore quirk probing for ACPI-based systems irqchip/gic-v3-its: Fix GICv4.1 VPE affinity update
drivers/irqchip/irq-gic-v3-its.c | 62 ++++++++++++++++++++------------ 1 file changed, 40 insertions(+), 22 deletions(-)
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH 1/3 irqchip/gic-v3-its: Handle non-coherent GICv4 redistributors
Although the GICv3 code base has gained some handling of systems failing to handle the shareability attributes, the GICv4 side of things has been firmly ignored.
This is unfortunate, as the new recent addition of the "dma-noncoherent" is supposed to apply to all of the GICR tables, and not just the ones that are common to v3 and v4.
Add some checks to handle the VPROPBASE/VPENDBASE shareability and cacheability attributes in the same way we deal with the other GICR_BASE registers, wrapping the flag check in a helper for improved readability.
Note that this has been found by inspection only, as I don't have access to HW that suffers from this particular issue.
--- drivers/irqchip/irq-gic-v3-its.c | 37 +++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 12 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +25 -12
@@ -207,6 +207,11 @@ static bool require_its_list_vmovp(struct its_vm *vm, struct its_node *its)return (gic_rdists->has_rvpeid || vm->vlpi_count[its->list_nr]); } +static bool rdists_support_shareable (void ) +{ +return !(gic_rdists->flags &RDIST_FLAGS_FORCE_NON_SHAREABLE ); +} +static u16 get_its_list (struct its_vm *vm) {struct its_node *its;
@@ -2710,10 +2715,12 @@ static u64 inherit_vpe_l1_table_from_its(void)break ; } val |=FIELD_PREP (GICR_VPROPBASER_4_1_ADDR , addr >>12 ); - val |=FIELD_PREP (GICR_VPROPBASER_SHAREABILITY_MASK , -FIELD_GET (GITS_BASER_SHAREABILITY_MASK , baser)); - val |=FIELD_PREP (GICR_VPROPBASER_INNER_CACHEABILITY_MASK , -FIELD_GET (GITS_BASER_INNER_CACHEABILITY_MASK , baser)); +if (rdists_support_shareable ()) { + val |=FIELD_PREP (GICR_VPROPBASER_SHAREABILITY_MASK , +FIELD_GET (GITS_BASER_SHAREABILITY_MASK , baser)); + val |=FIELD_PREP (GICR_VPROPBASER_INNER_CACHEABILITY_MASK , +FIELD_GET (GITS_BASER_INNER_CACHEABILITY_MASK , baser)); + } val |=FIELD_PREP (GICR_VPROPBASER_4_1_SIZE ,GITS_BASER_NR_PAGES (baser) -1 );return val;
@@ -2936,8 +2943,10 @@ static int allocate_vpe_l1_table(void)WARN_ON (!IS_ALIGNED (pa, psz)); val |=FIELD_PREP (GICR_VPROPBASER_4_1_ADDR , pa >>12 ); - val |= GICR_VPROPBASER_RaWb; - val |= GICR_VPROPBASER_InnerShareable; +if (rdists_support_shareable ()) { + val |= GICR_VPROPBASER_RaWb; + val |= GICR_VPROPBASER_InnerShareable; + } val |=GICR_VPROPBASER_4_1_Z ; val |=GICR_VPROPBASER_4_1_VALID ;
@@ -3126,7 +3135,7 @@ static void its_cpu_init_lpis(void)gicr_write_propbaser (val, rbase +GICR_PROPBASER ); tmp =gicr_read_propbaser (rbase +GICR_PROPBASER ); -if (gic_rdists->flags &RDIST_FLAGS_FORCE_NON_SHAREABLE ) +if (!rdists_support_shareable ()) tmp &= ~GICR_PROPBASER_SHAREABILITY_MASK ;if ((tmp ^ val) &GICR_PROPBASER_SHAREABILITY_MASK ) {
@@ -3153,7 +3162,7 @@ static void its_cpu_init_lpis(void)gicr_write_pendbaser (val, rbase +GICR_PENDBASER ); tmp =gicr_read_pendbaser (rbase +GICR_PENDBASER ); -if (gic_rdists->flags &RDIST_FLAGS_FORCE_NON_SHAREABLE ) +if (!rdists_support_shareable ()) tmp &= ~GICR_PENDBASER_SHAREABILITY_MASK ;if (!(tmp &GICR_PENDBASER_SHAREABILITY_MASK )) {
@@ -3880,14 +3889,18 @@ static void its_vpe_schedule(struct its_vpe *vpe) val =virt_to_phys (page_address (vpe->its_vm->vprop_page)) &GENMASK_ULL (51 ,12 ); val |= (LPI_NRBITS -1 ) &GICR_VPROPBASER_IDBITS_MASK ; - val |= GICR_VPROPBASER_RaWb; - val |= GICR_VPROPBASER_InnerShareable; +if (rdists_support_shareable ()) { + val |= GICR_VPROPBASER_RaWb; + val |= GICR_VPROPBASER_InnerShareable; + }gicr_write_vpropbaser (val, vlpi_base +GICR_VPROPBASER ); val =virt_to_phys (page_address (vpe->vpt_page)) &GENMASK_ULL (51 ,16 ); - val |= GICR_VPENDBASER_RaWaWb; - val |= GICR_VPENDBASER_InnerShareable; +if (rdists_support_shareable ()) { + val |= GICR_VPENDBASER_RaWaWb; + val |= GICR_VPENDBASER_InnerShareable; + }/* * There is no good way of finding out if the pending table is * empty as we can race against the doorbell interrupt very
Signature · 1 line
2.39.2
[tip: irq/urgent] irqchip/gic-v3-its: Handle non-coherent GICv4 redistributors
The following commit has been merged into the irq/urgent branch of tip:
The following commit has been merged into the irq/urgent branch of tip:
Commit-ID: 846297e11e8ae428f8b00156a0cfe2db58100702 Gitweb: https://git.kernel.org/tip/846297e11e8ae428f8b00156a0cfe2db58100702 Author: Marc Zyngier <[email protected]> AuthorDate: Tue, 13 Feb 2024 10:12:04 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 13 Feb 2024 11:29:51 +01:00
irqchip/gic-v3-its: Handle non-coherent GICv4 redistributors
Although the GICv3 code base has gained some handling of systems failing to handle the shareability attributes, the GICv4 side of things has been firmly ignored.
This is unfortunate, as the new recent addition of the "dma-noncoherent" is supposed to apply to all of the GICR tables, and not just the ones that are common to v3 and v4.
Add some checks to handle the VPROPBASE/VPENDBASE shareability and cacheability attributes in the same way we deal with the other GICR_BASE registers, wrapping the flag check in a helper for improved readability.
Note that this has been found by inspection only, as I don't have access to HW that suffers from this particular issue.
--- drivers/irqchip/irq-gic-v3-its.c | 37 ++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 12 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +25 -12
@@ -207,6 +207,11 @@ static bool require_its_list_vmovp(struct its_vm *vm, struct its_node *its)return (gic_rdists->has_rvpeid || vm->vlpi_count[its->list_nr]); } +static bool rdists_support_shareable (void ) +{ +return !(gic_rdists->flags &RDIST_FLAGS_FORCE_NON_SHAREABLE ); +} +static u16 get_its_list (struct its_vm *vm) {struct its_node *its;
@@ -2710,10 +2715,12 @@ static u64 inherit_vpe_l1_table_from_its(void)break ; } val |=FIELD_PREP (GICR_VPROPBASER_4_1_ADDR , addr >>12 ); - val |=FIELD_PREP (GICR_VPROPBASER_SHAREABILITY_MASK , -FIELD_GET (GITS_BASER_SHAREABILITY_MASK , baser)); - val |=FIELD_PREP (GICR_VPROPBASER_INNER_CACHEABILITY_MASK , -FIELD_GET (GITS_BASER_INNER_CACHEABILITY_MASK , baser)); +if (rdists_support_shareable ()) { + val |=FIELD_PREP (GICR_VPROPBASER_SHAREABILITY_MASK , +FIELD_GET (GITS_BASER_SHAREABILITY_MASK , baser)); + val |=FIELD_PREP (GICR_VPROPBASER_INNER_CACHEABILITY_MASK , +FIELD_GET (GITS_BASER_INNER_CACHEABILITY_MASK , baser)); + } val |=FIELD_PREP (GICR_VPROPBASER_4_1_SIZE ,GITS_BASER_NR_PAGES (baser) -1 );return val;
@@ -2936,8 +2943,10 @@ static int allocate_vpe_l1_table(void)WARN_ON (!IS_ALIGNED (pa, psz)); val |=FIELD_PREP (GICR_VPROPBASER_4_1_ADDR , pa >>12 ); - val |= GICR_VPROPBASER_RaWb; - val |= GICR_VPROPBASER_InnerShareable; +if (rdists_support_shareable ()) { + val |= GICR_VPROPBASER_RaWb; + val |= GICR_VPROPBASER_InnerShareable; + } val |=GICR_VPROPBASER_4_1_Z ; val |=GICR_VPROPBASER_4_1_VALID ;
@@ -3126,7 +3135,7 @@ static void its_cpu_init_lpis(void)gicr_write_propbaser (val, rbase +GICR_PROPBASER ); tmp =gicr_read_propbaser (rbase +GICR_PROPBASER ); -if (gic_rdists->flags &RDIST_FLAGS_FORCE_NON_SHAREABLE ) +if (!rdists_support_shareable ()) tmp &= ~GICR_PROPBASER_SHAREABILITY_MASK ;if ((tmp ^ val) &GICR_PROPBASER_SHAREABILITY_MASK ) {
@@ -3153,7 +3162,7 @@ static void its_cpu_init_lpis(void)gicr_write_pendbaser (val, rbase +GICR_PENDBASER ); tmp =gicr_read_pendbaser (rbase +GICR_PENDBASER ); -if (gic_rdists->flags &RDIST_FLAGS_FORCE_NON_SHAREABLE ) +if (!rdists_support_shareable ()) tmp &= ~GICR_PENDBASER_SHAREABILITY_MASK ;if (!(tmp &GICR_PENDBASER_SHAREABILITY_MASK )) {
@@ -3880,14 +3889,18 @@ static void its_vpe_schedule(struct its_vpe *vpe) val =virt_to_phys (page_address (vpe->its_vm->vprop_page)) &GENMASK_ULL (51 ,12 ); val |= (LPI_NRBITS -1 ) &GICR_VPROPBASER_IDBITS_MASK ; - val |= GICR_VPROPBASER_RaWb; - val |= GICR_VPROPBASER_InnerShareable; +if (rdists_support_shareable ()) { + val |= GICR_VPROPBASER_RaWb; + val |= GICR_VPROPBASER_InnerShareable; + }gicr_write_vpropbaser (val, vlpi_base +GICR_VPROPBASER ); val =virt_to_phys (page_address (vpe->vpt_page)) &GENMASK_ULL (51 ,16 ); - val |= GICR_VPENDBASER_RaWaWb; - val |= GICR_VPENDBASER_InnerShareable; +if (rdists_support_shareable ()) { + val |= GICR_VPENDBASER_RaWaWb; + val |= GICR_VPENDBASER_InnerShareable; + }/* * There is no good way of finding out if the pending table is * empty as we can race against the doorbell interrupt very
Marc Zyngier
maintainerPATCH 2/3 irqchip/gic-v3-its: Restore quirk probing for ACPI-based systems
In reply to Marc ZyngierWhile refactoring the way the ITSs are probed, the handling of quirks applicable to ACPI-based platforms was lost. As a result, systems such as HIP07 lose their GICv4 functionnality, and some other may even fail to boot, unless they are configured to boot with DT.
Move the enabling of quirks into its_probe_one(), making it common to all firmware implementations.
--- drivers/irqchip/irq-gic-v3-its.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +2 -1
@@ -5091,6 +5091,8 @@ static int __init its_probe_one(struct its_node *its)u32 ctlr;int err; +its_enable_quirks (its); +if (is_v4 (its)) {if (!(its->typer &GITS_TYPER_VMOVP )) { err =its_compute_its_list_map (its);
@@ -5442,7 +5444,6 @@ static int __init its_of_probe(struct device_node *node)if (!its)return -ENOMEM ; -its_enable_quirks (its); err =its_probe_one (its);if (err) {its_node_destroy (its);
Signature · 1 line
2.39.2
[tip: irq/urgent] irqchip/gic-v3-its: Restore quirk probing for ACPI-based systems
The following commit has been merged into the irq/urgent branch of tip:
The following commit has been merged into the irq/urgent branch of tip:
Commit-ID: 8b02da04ad978827e5ccd675acf170198f747a7a Gitweb: https://git.kernel.org/tip/8b02da04ad978827e5ccd675acf170198f747a7a Author: Marc Zyngier <[email protected]> AuthorDate: Tue, 13 Feb 2024 10:12:05 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 13 Feb 2024 11:29:52 +01:00
irqchip/gic-v3-its: Restore quirk probing for ACPI-based systems
While refactoring the way the ITSs are probed, the handling of quirks applicable to ACPI-based platforms was lost. As a result, systems such as HIP07 lose their GICv4 functionnality, and some other may even fail to boot, unless they are configured to boot with DT.
Move the enabling of quirks into its_probe_one(), making it common to all firmware implementations.
--- drivers/irqchip/irq-gic-v3-its.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +2 -1
@@ -5091,6 +5091,8 @@ static int __init its_probe_one(struct its_node *its)u32 ctlr;int err; +its_enable_quirks (its); +if (is_v4 (its)) {if (!(its->typer &GITS_TYPER_VMOVP )) { err =its_compute_its_list_map (its);
@@ -5442,7 +5444,6 @@ static int __init its_of_probe(struct device_node *node)if (!its)return -ENOMEM ; -its_enable_quirks (its); err =its_probe_one (its);if (err) {its_node_destroy (its);
Marc Zyngier
maintainerPATCH 3/3 irqchip/gic-v3-its: Fix GICv4.1 VPE affinity update
In reply to Marc ZyngierWhen updating the affinity of a VPE, we currently skip the VMOVP command if the two CPUs are part of the same VPE affinity.
But this is wrong, as the doorbell corresponding to this VPE is still delivered on the 'old' CPU, which screws up the balancing. Furthermore, offlining that 'old' CPU results in doorbell interrupts generated for this VPE being discarded.
The harsh reality is that we cannot easily elide VMOVP when a set_affinity request occurs. It needs to be obeyed, and if an optimisation is to be made, it is at the point where the affinity change request is made (such as in KVM).
Drop the VMOVP elision altogether, and only use the vpe_table_mask to try and stay within the same ITS affinity group if at all possible.
--- drivers/irqchip/irq-gic-v3-its.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +13 -9
@@ -3826,8 +3826,9 @@ static int its_vpe_set_affinity(struct irq_data *d, bool force) {struct its_vpe *vpe =irq_data_get_irq_chip_data (d); -int from, cpu =cpumask_first (mask_val); +struct cpumask common, *table_mask;unsigned long flags; +int from, cpu;/* * Changing affinity is mega expensive, so let's be as lazy as
@@ -3843,19 +3844,22 @@ static int its_vpe_set_affinity(struct irq_data *d,* taken on any vLPI handling path that evaluates vpe->col_idx. */ from =vpe_to_cpuid_lock (vpe, &flags); -if (from == cpu) - goto out; - - vpe->col_idx = cpu; + table_mask =gic_data_rdist_cpu (from)->vpe_table_mask;/* -* GICv4.1 allows us to skip VMOVP if moving to a cpu whose RD -* is sharing its VPE table with the current one. +* If we are offered another CPU in the same GICv4.1 ITS +* affinity, pick this one. Otherwise, any CPU will do. */ -if (gic_data_rdist_cpu (cpu)->vpe_table_mask && -cpumask_test_cpu (from,gic_data_rdist_cpu (cpu)->vpe_table_mask)) +if (table_mask &&cpumask_and (&common, mask_val, table_mask)) + cpu =cpumask_test_cpu (from, &common) ? from :cpumask_first (&common); +else + cpu =cpumask_first (mask_val); + +if (from == cpu) goto out; + vpe->col_idx = cpu; +its_send_vmovp (vpe);its_vpe_db_proxy_move (vpe, from, cpu);
Signature · 1 line
2.39.2
[tip: irq/urgent] irqchip/gic-v3-its: Fix GICv4.1 VPE affinity update
The following commit has been merged into the irq/urgent branch of tip:
The following commit has been merged into the irq/urgent branch of tip:
Commit-ID: af9acbfc2c4b72c378d0b9a2ee023ed01055d3e2 Gitweb: https://git.kernel.org/tip/af9acbfc2c4b72c378d0b9a2ee023ed01055d3e2 Author: Marc Zyngier <[email protected]> AuthorDate: Tue, 13 Feb 2024 10:12:06 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 13 Feb 2024 11:29:52 +01:00
irqchip/gic-v3-its: Fix GICv4.1 VPE affinity update
When updating the affinity of a VPE, the VMOVP command is currently skipped if the two CPUs are part of the same VPE affinity.
But this is wrong, as the doorbell corresponding to this VPE is still delivered on the 'old' CPU, which screws up the balancing. Furthermore, offlining that 'old' CPU results in doorbell interrupts generated for this VPE being discarded.
The harsh reality is that VMOVP cannot be elided when a set_affinity() request occurs. It needs to be obeyed, and if an optimisation is to be made, it is at the point where the affinity change request is made (such as in KVM).
Drop the VMOVP elision altogether, and only use the vpe_table_mask to try and stay within the same ITS affinity group if at all possible.
--- drivers/irqchip/irq-gic-v3-its.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +13 -9
@@ -3826,8 +3826,9 @@ static int its_vpe_set_affinity(struct irq_data *d, bool force) {struct its_vpe *vpe =irq_data_get_irq_chip_data (d); -int from, cpu =cpumask_first (mask_val); +struct cpumask common, *table_mask;unsigned long flags; +int from, cpu;/* * Changing affinity is mega expensive, so let's be as lazy as
@@ -3843,19 +3844,22 @@ static int its_vpe_set_affinity(struct irq_data *d,* taken on any vLPI handling path that evaluates vpe->col_idx. */ from =vpe_to_cpuid_lock (vpe, &flags); -if (from == cpu) - goto out; - - vpe->col_idx = cpu; + table_mask =gic_data_rdist_cpu (from)->vpe_table_mask;/* -* GICv4.1 allows us to skip VMOVP if moving to a cpu whose RD -* is sharing its VPE table with the current one. +* If we are offered another CPU in the same GICv4.1 ITS +* affinity, pick this one. Otherwise, any CPU will do. */ -if (gic_data_rdist_cpu (cpu)->vpe_table_mask && -cpumask_test_cpu (from,gic_data_rdist_cpu (cpu)->vpe_table_mask)) +if (table_mask &&cpumask_and (&common, mask_val, table_mask)) + cpu =cpumask_test_cpu (from, &common) ? from :cpumask_first (&common); +else + cpu =cpumask_first (mask_val); + +if (from == cpu) goto out; + vpe->col_idx = cpu; +its_send_vmovp (vpe);its_vpe_db_proxy_move (vpe, from, cpu);