irqchip/gic-v3: Enable non-coherent GIC designs probing
Subsystems (5)
cover irqchip/gic-v3: Enable non-coherent GIC designs probing
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.
Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.
When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (2):
dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent
property
irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
.../interrupt-controller/arm,gic-v3.yaml | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 19 +++++++++++++++---- 2 files changed, 23 insertions(+), 4 deletions(-)
Signature · 7 lines
2.34.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 1/2 dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 8 ++++++++ 1 file changed, 8 insertions(+)
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +8 -0
@@ -106,6 +106,10 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: | + Present if the GIC redistributors are not cache coherent with the CPU. + msi-controller: description: Only present if the Message Based Interrupt functionality is
@@ -193,6 +197,10 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: | + Present if the GIC ITS is not cache coherent with the CPU. + msi-controller: true "#msi-cells":
Signature · 7 lines
2.34.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/2] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
Quoted 20 lines · Lorenzo Pieralisi
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 8 ++++++++ 1 file changed, 8 insertions(+)Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +0
-0
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml properties: · line 111
Quoted 6 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +3
-0@@ -106,6 +106,10 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: | + Present if the GIC redistributors are not cache coherent with the CPU.
I wonder if it's worth being a bit more specific here, e.g. "if the GIC {redistributors,ITS} permit programming cacheable inner-shareable memory attributes, but are connected to a non-coherent downstream interconnect." That might help clarify why the negative property, which could seem a bit backwards at first glance, and that it's not so important in the cases where the GIC itself is fundamentally non-coherent anyway (which *is* software-discoverable).
Otherwise, this is the same approach that I like and have previously lobbied for, so obviously I approve :)
(plus I do think it's the right shape to be able to slot an equivalent field into ACPI MADT entries without *too* much bother)
Thanks,
Robin.
Quoted 5 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +1
-0+ msi-controller: description: Only present if the Message Based Interrupt functionality is
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml patternProperties: · line 206
Quoted 10 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +4
-0@@ -193,6 +197,10 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: | + Present if the GIC ITS is not cache coherent with the CPU. + msi-controller: true "#msi-cells":
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/2] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
Quoted 16 lines · Robin Murphy
Quoted 15 lines · Lorenzo Pieralisi
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 8 ++++++++ 1 file changed, 8 insertions(+)
Quoted 14 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml properties: · line 110
Quoted 10 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +3
-0@@ -106,6 +106,10 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: | + Present if the GIC redistributors are not cache coherent with the CPU.I wonder if it's worth being a bit more specific here, e.g. "if the GIC {redistributors,ITS} permit programming cacheable inner-shareable memory attributes, but are connected to a non-coherent downstream interconnect."
In my opinion it is and I wanted to elaborate on what I wrote but then I thought that this is a standard DT property, I wasn't sure whether we really need to explain what it is there for.
We are using the property to plug a hole so I agree with you, we should be as clear as possible in the property definition but I will rely on Rob/Marc's opinion, I don't know what's the DT policy for this.
Quoted 4 lines
That might help clarify why the negative property, which could seem a bit backwards at first glance, and that it's not so important in the cases where the GIC itself is fundamentally non-coherent anyway (which *is* software-discoverable).
Is it ? Again, see above, are we defining "dma-noncoherent" to fix a bug or to fix the specs ? The shareability bits are writeable and even a fundamentally non-coherent GIC design could allow writing them, AFAIU.
I would avoid putting ourselves into a corner where we can't use this property because the binding itself is too strict on what it is solving.
Quoted 5 lines
Otherwise, this is the same approach that I like and have previously lobbied for, so obviously I approve :)
(plus I do think it's the right shape to be able to slot an equivalent field into ACPI MADT entries without *too* much bother)
We are in agreement, let's see what others think.
Thanks,
Lorenzo
Quoted 14 lines
Thanks,
Robin.Quoted 10 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +4
-0+ msi-controller: description: Only present if the Message Based Interrupt functionality is@@ -193,6 +197,10 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: | + Present if the GIC ITS is not cache coherent with the CPU.
Quoted 8 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml patternProperties: · line 206
Quoted 3 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +1
-0+ msi-controller: true "#msi-cells":_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/2] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
Quoted 40 lines · Lorenzo Pieralisi
Quoted 16 lines · Robin Murphy
Quoted 15 lines · Lorenzo Pieralisi
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 8 ++++++++ 1 file changed, 8 insertions(+)Quoted 14 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml properties: · line 110
Quoted 10 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +3
-0@@ -106,6 +106,10 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: | + Present if the GIC redistributors are not cache coherent with the CPU.I wonder if it's worth being a bit more specific here, e.g. "if the GIC {redistributors,ITS} permit programming cacheable inner-shareable memory attributes, but are connected to a non-coherent downstream interconnect."
In my opinion it is and I wanted to elaborate on what I wrote but then I thought that this is a standard DT property, I wasn't sure whether we really need to explain what it is there for.
We are using the property to plug a hole so I agree with you, we should be as clear as possible in the property definition but I will rely on Rob/Marc's opinion, I don't know what's the DT policy for this.
Quoted 8 lines
Quoted 4 lines
That might help clarify why the negative property, which could seem a bit backwards at first glance, and that it's not so important in the cases where the GIC itself is fundamentally non-coherent anyway (which *is* software-discoverable).
Is it ? Again, see above, are we defining "dma-noncoherent" to fix a bug or to fix the specs ? The shareability bits are writeable and even a fundamentally non-coherent GIC design could allow writing them, AFAIU.
I mean the case on GIC-500 and earlier where the register bits could be hard-wired. I'm not sure a GIC implementation which didn't even *try* to honour the programmed attributes in what it emits would be considered valid; it certainly couldn't be considered sensible :/
Quoted 3 lines
I would avoid putting ourselves into a corner where we can't use this property because the binding itself is too strict on what it is solving.
Really I'm just getting at the fact that if you do have a legacy GIC with hard-wired attributes then whatever DT says is most likely irrelevant anyway (unless the integrator has done something utterly bonkers and tied off the interconnect input to *different* attributes, but I would consider that beyond the bounds of fair reasoning...)
Cheers,
Robin.
Quoted 33 lines
Quoted 5 lines
Otherwise, this is the same approach that I like and have previously lobbied for, so obviously I approve :)
(plus I do think it's the right shape to be able to slot an equivalent field into ACPI MADT entries without *too* much bother)
We are in agreement, let's see what others think.
Thanks,
LorenzoQuoted 22 lines
Thanks,
Robin.Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml patternProperties: · line 204
Quoted 13 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +5
-0+ msi-controller: description: Only present if the Message Based Interrupt functionality is@@ -193,6 +197,10 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: | + Present if the GIC ITS is not cache coherent with the CPU. + msi-controller: true "#msi-cells":_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Rob Herring
maintainerRe: [PATCH 1/2] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
In reply to Lorenzo PieralisiQuoted 20 lines · Lorenzo Pieralisi
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 8 ++++++++ 1 file changed, 8 insertions(+)Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +0
-0
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml properties: · line 110
Quoted 5 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +2
-0@@ -106,6 +106,10 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: |
Don't need the '|' if no formatting to preserve.
Quoted 19 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +6
-0+ Present if the GIC redistributors are not cache coherent with the CPU. + msi-controller: description: Only present if the Message Based Interrupt functionality is@@ -193,6 +197,10 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: | + Present if the GIC ITS is not cache coherent with the CPU. + msi-controller: true "#msi-cells":Signature · 1 line
2.34.1
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 2/2 irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Lorenzo PieralisiThe GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers.
--- drivers/irqchip/irq-gic-v3-its.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +15 -4
@@ -5056,7 +5056,8 @@ static int __init its_compute_its_list_map(struct resource *res, }static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +struct fwnode_handle *handle,int numa_node, +bool non_coherent) {struct its_node *its;void __iomem *its_base;
@@ -5148,7 +5149,7 @@ static int __init its_probe_one(struct resource *res,gits_write_cbaser (baser, its->base +GITS_CBASER ); tmp =gits_read_cbaser (its->base +GITS_CBASER ); -if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE ) +if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE || non_coherent) tmp &= ~GITS_CBASER_SHAREABILITY_MASK ;if ((tmp ^ baser) &GITS_CBASER_SHAREABILITY_MASK ) {
@@ -5356,11 +5357,19 @@ static const struct of_device_id its_device_id[] = { {}, }; +static void of_check_rdists_coherent (struct device_node *node) +{ +if (of_property_read_bool (node,"dma-noncoherent" )) + gic_rdists->flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +of_check_rdists_coherent (node); +/* * Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to
@@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }
@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, } err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); +acpi_get_its_numa_node (its_entry->translation_id), + false);if (!err)return 0 ;
Signature · 7 lines
2.34.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Marc Zyngier
maintainerRe: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 57 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc Marc Zyngier <[email protected]>--- drivers/irqchip/irq-gic-v3-its.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/irqchip/irq-gic-v3-its.c +2
-1@@ -5056,7 +5056,8 @@ static int __init its_compute_its_list_map(struct resource *res, }static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +struct fwnode_handle *handle,int numa_node, +bool non_coherent) {struct its_node *its;void __iomem *its_base;
drivers/irqchip/irq-gic-v3-its.c static int __init its_probe_one(struct resource *res, · line 5153
Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +1
-1@@ -5148,7 +5149,7 @@ static int __init its_probe_one(struct resource *res,gits_write_cbaser (baser, its->base +GITS_CBASER ); tmp =gits_read_cbaser (its->base +GITS_CBASER ); -if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE ) +if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE || non_coherent) tmp &= ~GITS_CBASER_SHAREABILITY_MASK ;
Please use the non_coherent attribute to set the flag, instead of using it as some sideband signalling. Not having this information stored in the its_node structure makes it harder to debug.
We have an over-engineered quirk framework, and it should be put to a good use.
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +2
-0if ((tmp ^ baser) &GITS_CBASER_SHAREABILITY_MASK ) {@@ -5356,11 +5357,19 @@ static const struct of_device_id its_device_id[] = { {}, }; +static void of_check_rdists_coherent (struct device_node *node) +{
drivers/irqchip/irq-gic-v3-its.c static const struct of_device_id its_device_id[] = { · line 5371
Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +5
-0+if (of_property_read_bool (node,"dma-noncoherent" )) + gic_rdists->flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +of_check_rdists_coherent (node);
It really feels that the flag should instead be communicated by the base GIC driver, as it readily communicates the whole rdists structure already.
Quoted 15 lines
drivers/irqchip/irq-gic-v3-its.c +3
-1+/* * Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to @@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }
drivers/irqchip/irq-gic-v3-its.c static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +2
-1@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, } err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); +acpi_get_its_numa_node (its_entry->translation_id), + false);
I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Thanks,
M.
drivers/irqchip/irq-gic-common.h +4 -0
@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */
drivers/irqchip/irq-gic-v3-its.c +13 -4
@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)
@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {
@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } };
drivers/irqchip/irq-gic-v3.c +13 -0
@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,
@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 113 lines · Marc Zyngier
Quoted 57 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc Marc Zyngier <[email protected]>--- drivers/irqchip/irq-gic-v3-its.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/irqchip/irq-gic-v3-its.c +2
-1@@ -5056,7 +5056,8 @@ static int __init its_compute_its_list_map(struct resource *res, }static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +struct fwnode_handle *handle,int numa_node, +bool non_coherent) {struct its_node *its;void __iomem *its_base;drivers/irqchip/irq-gic-v3-its.c static int __init its_probe_one(struct resource *res, · line 5153
Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +1
-1@@ -5148,7 +5149,7 @@ static int __init its_probe_one(struct resource *res,gits_write_cbaser (baser, its->base +GITS_CBASER ); tmp =gits_read_cbaser (its->base +GITS_CBASER ); -if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE ) +if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE || non_coherent) tmp &= ~GITS_CBASER_SHAREABILITY_MASK ;Please use the
non_coherentattribute to set the flag, instead of using it as some sideband signalling. Not having this information stored in theits_nodestructure makes it harder to debug.We have an over-engineered quirk framework, and it should be put to a good use.
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +2
-0if ((tmp ^ baser) &GITS_CBASER_SHAREABILITY_MASK ) {@@ -5356,11 +5357,19 @@ static const struct of_device_id its_device_id[] = { {}, }; +static void of_check_rdists_coherent (struct device_node *node) +{drivers/irqchip/irq-gic-v3-its.c static const struct of_device_id its_device_id[] = { · line 5371
Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +5
-0+if (of_property_read_bool (node,"dma-noncoherent" )) + gic_rdists->flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +of_check_rdists_coherent (node);It really feels that the flag should instead be communicated by the base GIC driver, as it readily communicates the whole rdists structure already.
Quoted 17 lines
drivers/irqchip/irq-gic-v3-its.c +3
-1+/* * Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to @@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +2
-1err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); +acpi_get_its_numa_node (its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
FWIW I think I agree that looks a bit easier to follow overall, and especially since we already have the SoC-based Rockchip variant of this using a quirk, having two different ways of carrying the same underlying information through certain paths does seem a bit icky.
Cheers,
Robin.
Quoted 82 lines
Thanks,
M.drivers/irqchip/irq-gic-common.h +4
-0@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */ drivers/irqchip/irq-gic-v3-its.c +13
-4@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } };drivers/irqchip/irq-gic-v3.c +8
-0@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff,
drivers/irqchip/irq-gic-v3.c static const struct gic_quirk gic_quirks[] = { · line 1941
Quoted 10 lines
drivers/irqchip/irq-gic-v3.c +5
-0.init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Marc ZyngierQuoted 70 lines · Marc Zyngier
Quoted 57 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc Marc Zyngier <[email protected]>--- drivers/irqchip/irq-gic-v3-its.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/irqchip/irq-gic-v3-its.c +2
-1@@ -5056,7 +5056,8 @@ static int __init its_compute_its_list_map(struct resource *res, }static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +struct fwnode_handle *handle,int numa_node, +bool non_coherent) {struct its_node *its;void __iomem *its_base;drivers/irqchip/irq-gic-v3-its.c static int __init its_probe_one(struct resource *res, · line 5153
Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +1
-1@@ -5148,7 +5149,7 @@ static int __init its_probe_one(struct resource *res,gits_write_cbaser (baser, its->base +GITS_CBASER ); tmp =gits_read_cbaser (its->base +GITS_CBASER ); -if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE ) +if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE || non_coherent) tmp &= ~GITS_CBASER_SHAREABILITY_MASK ;Please use the
non_coherentattribute to set the flag, instead of using it as some sideband signalling. Not having this information stored in theits_nodestructure makes it harder to debug.
Quoted 2 lines
We have an over-engineered quirk framework, and it should be put to a good use.
That makes sense, I will do - I was wondering though whether this is a quirk or a binding that describes the architecture because the architecture can't provide that information.
That's the reason why I have not implemented it as a quirk but obviously that's exactly the feedback I need.
Quoted 18 lines
Quoted 18 lines
drivers/irqchip/irq-gic-v3-its.c +6
-0if ((tmp ^ baser) &GITS_CBASER_SHAREABILITY_MASK ) {@@ -5356,11 +5357,19 @@ static const struct of_device_id its_device_id[] = { {}, }; +static void of_check_rdists_coherent (struct device_node *node) +{ +if (of_property_read_bool (node,"dma-noncoherent" )) + gic_rdists->flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res;
Quoted 5 lines
drivers/irqchip/irq-gic-v3-its.c static const struct of_device_id its_device_id[] = { · line 5371
Quoted 1 line
drivers/irqchip/irq-gic-v3-its.c +1
-0+of_check_rdists_coherent (node);It really feels that the flag should instead be communicated by the base GIC driver, as it readily communicates the whole rdists structure already.
Yes I had the same feeling while writing the code but the patch below (if we treat this as quirk) is already much cleaner.
Quoted 17 lines
Quoted 17 lines
drivers/irqchip/irq-gic-v3-its.c +3
-1+/* * Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to @@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +2
-1err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); +acpi_get_its_numa_node (its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
The patch below makes sense and it is much cleaner - provided we consider this a quirk, which I am not sure it is, that's the only question I have.
Thanks,
Lorenzo
Quoted 98 lines
Thanks,
M.drivers/irqchip/irq-gic-common.h +4
-0@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */ drivers/irqchip/irq-gic-v3-its.c +13
-4@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } };drivers/irqchip/irq-gic-v3.c +13
-0@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };Signature · 7 lines
-- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Marc Zyngier
maintainerRe: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 9 lines · Lorenzo Pieralisi
Quoted 3 lines · Marc Zyngier
I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
The patch below makes sense and it is much cleaner - provided we consider this a quirk, which I am not sure it is, that's the only question I have.
Come on, the whole architecture is a bag of sick hacks. And given that it makes the change slightly less ugly, I'd rather have that.
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Marc ZyngierQuoted 170 lines · Marc Zyngier
Quoted 57 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc Marc Zyngier <[email protected]>--- drivers/irqchip/irq-gic-v3-its.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/irqchip/irq-gic-v3-its.c +2
-1@@ -5056,7 +5056,8 @@ static int __init its_compute_its_list_map(struct resource *res, }static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +struct fwnode_handle *handle,int numa_node, +bool non_coherent) {struct its_node *its;void __iomem *its_base;drivers/irqchip/irq-gic-v3-its.c static int __init its_probe_one(struct resource *res, · line 5153
Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +1
-1@@ -5148,7 +5149,7 @@ static int __init its_probe_one(struct resource *res,gits_write_cbaser (baser, its->base +GITS_CBASER ); tmp =gits_read_cbaser (its->base +GITS_CBASER ); -if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE ) +if (its->flags &ITS_FLAGS_FORCE_NON_SHAREABLE || non_coherent) tmp &= ~GITS_CBASER_SHAREABILITY_MASK ;Please use the
non_coherentattribute to set the flag, instead of using it as some sideband signalling. Not having this information stored in theits_nodestructure makes it harder to debug.We have an over-engineered quirk framework, and it should be put to a good use.
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +2
-0if ((tmp ^ baser) &GITS_CBASER_SHAREABILITY_MASK ) {@@ -5356,11 +5357,19 @@ static const struct of_device_id its_device_id[] = { {}, }; +static void of_check_rdists_coherent (struct device_node *node) +{drivers/irqchip/irq-gic-v3-its.c static const struct of_device_id its_device_id[] = { · line 5371
Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +5
-0+if (of_property_read_bool (node,"dma-noncoherent" )) + gic_rdists->flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +of_check_rdists_coherent (node);It really feels that the flag should instead be communicated by the base GIC driver, as it readily communicates the whole rdists structure already.
Quoted 15 lines
drivers/irqchip/irq-gic-v3-its.c +3
-1+/* * Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to @@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }drivers/irqchip/irq-gic-v3-its.c static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +2
-1@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, } err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); +acpi_get_its_numa_node (its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Thanks,
M.drivers/irqchip/irq-gic-common.h +4
-0@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */ drivers/irqchip/irq-gic-v3-its.c +8
-4@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {
drivers/irqchip/irq-gic-v3-its.c static const struct gic_quirk its_quirks[] = { · line 4819
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +5
-0@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + },
For the records, that requires adding a gic_enable_of_quirks() call for the ITS DT node that at the moment is not needed, something like this:
-- >8 --
drivers/irqchip/irq-gic-v3-its.c +4 -0
@@ -4826,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its)u32 iidr =readl_relaxed (its->base +GITS_IIDR );gic_enable_quirks (iidr, its_quirks, its); + +if (is_of_node (its->fwnode_handle)) +gic_enable_of_quirks (to_of_node (its->fwnode_handle), + its_quirks, its); }static int its_save_disable (void )
Quoted 42 lines
drivers/irqchip/irq-gic-v3-its.c +0
-0{ } };drivers/irqchip/irq-gic-v3.c +13
-0@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };Signature · 7 lines
-- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Marc Zyngier
maintainerRe: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 22 lines · Lorenzo Pieralisi
Review static const struct gic_quirk its_quirks[] = { · line 4819
Quoted 9 lines · Marc Zyngier
(quoted diff) +5
-0@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, }, #endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + },For the records, that requires adding a
gic_enable_of_quirks()call for the ITS DT node that at the moment is not needed, something like this:-- >8 --
drivers/irqchip/irq-gic-v3-its.c +0
-0
drivers/irqchip/irq-gic-v3-its.c static void its_enable_quirks(struct its_node *its) · line 4834
Quoted 9 lines
drivers/irqchip/irq-gic-v3-its.c +4
-0@@ -4826,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its)u32 iidr =readl_relaxed (its->base +GITS_IIDR );gic_enable_quirks (iidr, its_quirks, its); + +if (is_of_node (its->fwnode_handle)) +gic_enable_of_quirks (to_of_node (its->fwnode_handle), + its_quirks, its); }
Ah, well spotted. Yes, please fold this in.
Thanks,
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Lorenzo PieralisiQuoted 109 lines · Lorenzo Pieralisi
Quoted 53 lines · Marc Zyngier
I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Thanks,
M.drivers/irqchip/irq-gic-common.h +4
-0@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */ drivers/irqchip/irq-gic-v3-its.c +8
-4@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {drivers/irqchip/irq-gic-v3-its.c static const struct gic_quirk its_quirks[] = { · line 4819
Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +5
-0@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + },For the records, that requires adding a
gic_enable_of_quirks()call for the ITS DT node that at the moment is not needed, something like this:-- >8 --
drivers/irqchip/irq-gic-v3-its.c +4
-0@@ -4826,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its)u32 iidr =readl_relaxed (its->base +GITS_IIDR );gic_enable_quirks (iidr, its_quirks, its); + +if (is_of_node (its->fwnode_handle)) +gic_enable_of_quirks (to_of_node (its->fwnode_handle), + its_quirks, its); }static int its_save_disable (void )Quoted 27 lines
drivers/irqchip/irq-gic-v3-its.c +0
-0{ } };drivers/irqchip/irq-gic-v3.c +8
-0@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, },
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c static const struct gic_quirk gic_quirks[] = { · line 1941
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c +5
-0+ { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };
I have tested this patch above on my deivce and it works well. Looking forward the official release.
Marc Zyngier
maintainerRe: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 3 lines · Fang Xiang
I have tested this patch above on my deivce and it works well. Looking forward the official release.
If that's a 'Tested-by:' tag, please respond to v2 with such a tag.
Thanks,
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Marc ZyngierOn Tue, Sep 05, 2023 at 12:34:58PM +0100, Marc Zyngier wrote:
[...]
Quoted 15 lines
Quoted 15 lines
(quoted diff) +2
-1* Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to@@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node) continue; } - its_probe_one(&res, &np->fwnode, of_node_to_nid(np)); + its_probe_one(&res, &np->fwnode, of_node_to_nid(np), + of_property_read_bool(np, "dma-noncoherent")); } return 0; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }
Quoted 8 lines
Review static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
(quoted diff) +2
-1err = its_probe_one(&res, dom_handle, - acpi_get_its_numa_node(its_entry->translation_id)); + acpi_get_its_numa_node(its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Writing the ACPI bits. We can't use the quirks framework for ACPI (we don't have "properties" and I don't think we want to attach any to the fwnode_handle) that's why I generalized its_probe_one() above with an extra param, that would have simplified ACPI parsing:
- we alloc struct its_node in its_probe_one() but at that stage ACPI parsing was already done. If we have to parse the MADT(ITS) again just to scan for non-coherent we then have to match the MADT entries to the *current* struct its_node* we are handling (MADT parsing callbacks don't even take a param - we have to resort to global variables - definitely doable but it is a bit ugly).
I will post the resulting code (or something else if I find anything nicer) but I thought I would mention the reasoning behind the additional bool flag above.
Lorenzo
Quoted 99 lines
Thanks,
M.drivers/irqchip/irq-gic-common.h +4
-0@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */ drivers/irqchip/irq-gic-v3-its.c +13
-4@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } };drivers/irqchip/irq-gic-v3.c +13
-0@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };Signature · 7 lines
-- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 33 lines · Lorenzo Pieralisi
On Tue, Sep 05, 2023 at 12:34:58PM +0100, Marc Zyngier wrote:
[...]
Quoted 15 lines
Quoted 15 lines
* Make sure *all* the ITS are reset before we probe any, as
* they may be sharing memory. If any of the ITS fails to(quoted diff) +2
-1@@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node) continue; } - its_probe_one(&res, &np->fwnode, of_node_to_nid(np)); + its_probe_one(&res, &np->fwnode, of_node_to_nid(np), + of_property_read_bool(np, "dma-noncoherent")); } return 0; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }Quoted 8 lines
Review static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
(quoted diff) +2
-1err = its_probe_one(&res, dom_handle, - acpi_get_its_numa_node(its_entry->translation_id)); + acpi_get_its_numa_node(its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Writing the ACPI bits. We can't use the quirks framework for ACPI (we don't have "properties" and I don't think we want to attach any to the
fwnode_handle) that's why I generalizedits_probe_one()above with an extra param, that would have simplified ACPI parsing:
Quoted 6 lines
- we alloc struct
its_nodeinits_probe_one()but at that stage ACPI parsing was already done. If we have to parse the MADT(ITS) again just to scan for non-coherent we then have to match the MADT entries to the *current* struct its_node* we are handling (MADT parsing callbacks don't even take a param - we have to resort to global variables - definitely doable but it is a bit ugly).
How about a compromise of passing a whole MADT flags field into its_probe_one() (where its_of_probe() can just pass 0), to pass through to its_enable_quirks() to then match against an madt_flags field in the gic_quirk? gic_acpi_init() could then do something similar for the redistributor quirk, although I guess it would then need to distinguish GICC and GICR-based quirks cases since the respective flags are in different formats.
Thanks,
Robin.
Marc Zyngier
maintainerRe: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Lorenzo PieralisiQuoted 34 lines · Lorenzo Pieralisi
On Tue, Sep 05, 2023 at 12:34:58PM +0100, Marc Zyngier wrote:
[...]
Quoted 15 lines
Quoted 15 lines
(quoted diff) +2
-1* Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to@@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node) continue; } - its_probe_one(&res, &np->fwnode, of_node_to_nid(np)); + its_probe_one(&res, &np->fwnode, of_node_to_nid(np), + of_property_read_bool(np, "dma-noncoherent")); } return 0; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }Quoted 8 lines
Review static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
(quoted diff) +2
-1err = its_probe_one(&res, dom_handle, - acpi_get_its_numa_node(its_entry->translation_id)); + acpi_get_its_numa_node(its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Writing the ACPI bits. We can't use the quirks framework for ACPI (we don't have "properties" and I don't think we want to attach any to the
fwnode_handle) that's why I generalizedits_probe_one()above with an extra param, that would have simplified ACPI parsing:
Quoted 6 lines
- we alloc struct
its_nodeinits_probe_one()but at that stage ACPI parsing was already done. If we have to parse the MADT(ITS) again just to scan for non-coherent we then have to match the MADT entries to the *current* struct its_node* we are handling (MADT parsing callbacks don't even take a param - we have to resort to global variables - definitely doable but it is a bit ugly).
Well, a more acceptable approach would be for its_probe_one() to take an allocated and possibly pre-populated its_node structure (crucially, with the quirk flags set), which itself results in a bunch of low hanging cleanups, see the patch below.
I have boot tested it in a DT guest, so it is obviously perfect.
M.
From 978f654d4459adf0b8f3f8e896ca37035b3b114c Mon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Tue, 3 Oct 2023 17:35:27 +0100
Subject: [PATCH] irqchip/gic-v3-its: Split allocation from initialisation of
its_node
In order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (its_node) from the actual probing.
This will allow firmware-specific hooks to be added between these two points.
--- drivers/irqchip/irq-gic-v3-its.c | 151 +++++++++++++++++++------------ 1 file changed, 91 insertions(+), 60 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +91 -60
@@ -4952,7 +4952,7 @@ static void __init __iomem *its_map_one(struct resource *res, int *err)return NULL ; } -static int its_init_domain (struct fwnode_handle *handle,struct its_node *its) +static int its_init_domain (struct its_node *its) {struct irq_domain *inner_domain;struct msi_domain_info *info;
@@ -4966,7 +4966,7 @@ static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) inner_domain =irq_domain_create_hierarchy (its_parent, its->msi_domain_flags,0 , - handle, &its_domain_ops, + its->fwnode_handle, &its_domain_ops, info);if (!inner_domain) {kfree (info);
@@ -5017,8 +5017,7 @@ static int its_init_vpe_domain(void)return 0 ; } -static int __init its_compute_its_list_map (struct resource *res, -void __iomem *its_base) +static int __init its_compute_its_list_map (struct its_node *its) {int its_number;u32 ctlr;
@@ -5032,15 +5031,15 @@ static int __init its_compute_its_list_map(struct resource *res, its_number =find_first_zero_bit (&its_list_map, GICv4_ITS_LIST_MAX);if (its_number >= GICv4_ITS_LIST_MAX) {pr_err ("ITS@%pa: No ITSList entry available!\n" , - &res->start); + &its->phys_base);return -EINVAL ; } - ctlr =readl_relaxed (its_base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR ); ctlr &= ~GITS_CTLR_ITS_NUMBER ; ctlr |= its_number <<GITS_CTLR_ITS_NUMBER_SHIFT ; -writel_relaxed (ctlr, its_base +GITS_CTLR ); - ctlr =readl_relaxed (its_base +GITS_CTLR ); +writel_relaxed (ctlr, its->base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR );if ((ctlr &GITS_CTLR_ITS_NUMBER ) != (its_number <<GITS_CTLR_ITS_NUMBER_SHIFT )) { its_number = ctlr &GITS_CTLR_ITS_NUMBER ; its_number >>=GITS_CTLR_ITS_NUMBER_SHIFT ;
@@ -5048,75 +5047,50 @@ static int __init its_compute_its_list_map(struct resource *res,if (test_and_set_bit (its_number, &its_list_map)) {pr_err ("ITS@%pa: Duplicate ITSList entry %d\n" , - &res->start, its_number); + &its->phys_base, its_number);return -EINVAL ; }return its_number; } -static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +static int __init its_probe_one (struct its_node *its) { -struct its_node *its; -void __iomem *its_base; -u64 baser, tmp, typer; +u64 baser, tmp;struct page *page;u32 ctlr;int err; - its_base =its_map_one (res, &err); -if (!its_base) -return err; - -pr_info ("ITS %pR\n" , res); - - its =kzalloc (sizeof (*its),GFP_KERNEL ); -if (!its) { - err = -ENOMEM ; - goto out_unmap; - } - -raw_spin_lock_init (&its->lock); -mutex_init (&its->dev_alloc_lock); -INIT_LIST_HEAD (&its->entry); -INIT_LIST_HEAD (&its->its_device_list); - typer =gic_read_typer (its_base +GITS_TYPER ); - its->typer = typer; - its->base = its_base; - its->phys_base = res->start;if (is_v4 (its)) { -if (!(typer &GITS_TYPER_VMOVP )) { - err =its_compute_its_list_map (res, its_base); +if (!(its->typer &GITS_TYPER_VMOVP )) { + err =its_compute_its_list_map (its);if (err <0 ) - goto out_free_its; + goto out; its->list_nr = err;pr_info ("ITS@%pa: Using ITS number %d\n" , - &res->start, err); + &its->phys_base, err); }else { -pr_info ("ITS@%pa: Single VMOVP capable\n" , &res->start); +pr_info ("ITS@%pa: Single VMOVP capable\n" , &its->phys_base); }if (is_v4_1 (its)) { -u32 svpet =FIELD_GET (GITS_TYPER_SVPET , typer); +u32 svpet =FIELD_GET (GITS_TYPER_SVPET , its->typer); - its->sgir_base =ioremap (res->start +SZ_128K ,SZ_64K ); + its->sgir_base =ioremap (its->phys_base +SZ_128K ,SZ_64K );if (!its->sgir_base) { err = -ENOMEM ; - goto out_free_its; + goto out; } - its->mpidr =readl_relaxed (its_base +GITS_MPIDR ); + its->mpidr =readl_relaxed (its->base +GITS_MPIDR );pr_info ("ITS@%pa: Using GICv4.1 mode %08x %08x\n" , - &res->start, its->mpidr, svpet); + &its->phys_base, its->mpidr, svpet); } } - its->numa_node = numa_node; - page =alloc_pages_node (its->numa_node,GFP_KERNEL | __GFP_ZERO,get_order (ITS_CMD_QUEUE_SZ ));if (!page) {
@@ -5125,12 +5099,9 @@ static int __init its_probe_one(struct resource *res, } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->fwnode_handle = handle; its->get_msi_base = its_irq_get_msi_base; its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; -its_enable_quirks (its); - err =its_alloc_tables (its);if (err) goto out_free_cmd;
@@ -5174,7 +5145,7 @@ static int __init its_probe_one(struct resource *res, ctlr |= GITS_CTLR_ImDe;writel_relaxed (ctlr, its->base +GITS_CTLR ); - err =its_init_domain (handle, its); + err =its_init_domain (its);if (err) goto out_free_tables;
@@ -5191,11 +5162,8 @@ static int __init its_probe_one(struct resource *res, out_unmap_sgir:if (its->sgir_base)iounmap (its->sgir_base); -out_free_its: -kfree (its); -out_unmap: -iounmap (its_base); -pr_err ("ITS@%pa: failed probing (%d)\n" , &res->start, err); +out: +pr_err ("ITS@%pa: failed probing (%d)\n" , &its->phys_base, err);return err; }
@@ -5356,10 +5324,53 @@ static const struct of_device_id its_device_id[] = { {}, }; +static struct its_node __init *its_node_init (struct resource *res, +struct fwnode_handle *handle,int numa_node) +{ +void __iomem *its_base; +struct its_node *its; +int err; + + its_base =its_map_one (res, &err); +if (!its_base) +return NULL ; + +pr_info ("ITS %pR\n" , res); + + its =kzalloc (sizeof (*its),GFP_KERNEL ); +if (!its) + goto out_unmap; + +raw_spin_lock_init (&its->lock); +mutex_init (&its->dev_alloc_lock); +INIT_LIST_HEAD (&its->entry); +INIT_LIST_HEAD (&its->its_device_list); + + its->typer =gic_read_typer (its_base +GITS_TYPER ); + its->base = its_base; + its->phys_base = res->start; + + its->numa_node = numa_node; + its->fwnode_handle = handle; + +return its; + +out_unmap: +iounmap (its_base); +return NULL ; +} + +static void its_node_destroy (struct its_node *its) +{ +iounmap (its->base); +kfree (its); +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +int err;/* * Make sure *all* the ITS are reset before we probe any, as
@@ -5369,8 +5380,6 @@ static int __init its_of_probe(struct device_node *node)*/ for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { -int err; -if (!of_device_is_available (np) || !of_property_read_bool (np,"msi-controller" ) ||of_address_to_resource (np,0 , &res))
@@ -5383,6 +5392,8 @@ static int __init its_of_probe(struct device_node *node)for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { +struct its_node *its; +if (!of_device_is_available (np))continue ;if (!of_property_read_bool (np,"msi-controller" )) {
@@ -5396,7 +5407,17 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); + + its =its_node_init (&res, &np->fwnode,of_node_to_nid (np)); +if (!its) +return -ENOMEM ; + +its_enable_quirks (its); + err =its_probe_one (its); +if (err) { +its_node_destroy (its); +return err; + } }return 0 ; }
@@ -5508,6 +5529,7 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, {struct acpi_madt_generic_translator *its_entry;struct fwnode_handle *dom_handle; +struct its_node *its;struct resource res;int err;
@@ -5532,11 +5554,20 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto dom_err; } - err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); + its =its_node_init (&res, dom_handle, +acpi_get_its_numa_node (its_entry->translation_id)); +if (!its) { + err = -ENOMEM ; + goto node_err; + } + +/* Stick ACPI quirk handling here */ + + err =its_probe_one (its);if (!err)return 0 ; +node_err:iort_deregister_domain_token (its_entry->translation_id); dom_err:irq_domain_free_fwnode (dom_handle);
Signature · 4 lines
2.34.1 -- Without deviation from the norm, progress is not possible.
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
Quoted 43 lines · Marc Zyngier
Quoted 34 lines · Lorenzo Pieralisi
On Tue, Sep 05, 2023 at 12:34:58PM +0100, Marc Zyngier wrote:
[...]
Quoted 15 lines
Quoted 15 lines
drivers/irqchip/irq-gic-v3-its.c +2
-1* Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to @@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }Quoted 8 lines
Review static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
(quoted diff) +2
-1err = its_probe_one(&res, dom_handle, - acpi_get_its_numa_node(its_entry->translation_id)); + acpi_get_its_numa_node(its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Writing the ACPI bits. We can't use the quirks framework for ACPI (we don't have "properties" and I don't think we want to attach any to the
fwnode_handle) that's why I generalizedits_probe_one()above with an extra param, that would have simplified ACPI parsing:Quoted 6 lines
- we alloc struct
its_nodeinits_probe_one()but at that stage ACPI parsing was already done. If we have to parse the MADT(ITS) again just to scan for non-coherent we then have to match the MADT entries to the *current* struct its_node* we are handling (MADT parsing callbacks don't even take a param - we have to resort to global variables - definitely doable but it is a bit ugly).
Quoted 6 lines
Well, a more acceptable approach would be for
its_probe_one()to take an allocated and possibly pre-populatedits_nodestructure (crucially, with the quirk flags set), which itself results in a bunch of low hanging cleanups, see the patch below.I have boot tested it in a DT guest, so it is obviously perfect.
Thanks Marc I will test it and add the ACPI parsing bits.
Lorenzo
Quoted 325 lines
M.From
978f654d4459adf0b8f3f8e896ca37035b3b114cMon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Tue, 3 Oct 2023 17:35:27 +0100
Subject: [PATCH] irqchip/gic-v3-its: Split allocation from initialisation of
its_nodeIn order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (
its_node) from the actual probing.This will allow firmware-specific hooks to be added between these two points.
Signed-off-by Marc Zyngier <[email protected]>--- drivers/irqchip/irq-gic-v3-its.c | 151 +++++++++++++++++++------------ 1 file changed, 91 insertions(+), 60 deletions(-)drivers/irqchip/irq-gic-v3-its.c +91
-60@@ -4952,7 +4952,7 @@ static void __init __iomem *its_map_one(struct resource *res, int *err)return NULL ; } -static int its_init_domain (struct fwnode_handle *handle,struct its_node *its) +static int its_init_domain (struct its_node *its) {struct irq_domain *inner_domain;struct msi_domain_info *info;@@ -4966,7 +4966,7 @@ static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) inner_domain =irq_domain_create_hierarchy (its_parent, its->msi_domain_flags,0 , - handle, &its_domain_ops, + its->fwnode_handle, &its_domain_ops, info);if (!inner_domain) {kfree (info);@@ -5017,8 +5017,7 @@ static int its_init_vpe_domain(void)return 0 ; } -static int __init its_compute_its_list_map (struct resource *res, -void __iomem *its_base) +static int __init its_compute_its_list_map (struct its_node *its) {int its_number;u32 ctlr;@@ -5032,15 +5031,15 @@ static int __init its_compute_its_list_map(struct resource *res, its_number =find_first_zero_bit (&its_list_map, GICv4_ITS_LIST_MAX);if (its_number >= GICv4_ITS_LIST_MAX) {pr_err ("ITS@%pa: No ITSList entry available!\n" , - &res->start); + &its->phys_base);return -EINVAL ; } - ctlr =readl_relaxed (its_base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR ); ctlr &= ~GITS_CTLR_ITS_NUMBER ; ctlr |= its_number <<GITS_CTLR_ITS_NUMBER_SHIFT ; -writel_relaxed (ctlr, its_base +GITS_CTLR ); - ctlr =readl_relaxed (its_base +GITS_CTLR ); +writel_relaxed (ctlr, its->base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR );if ((ctlr &GITS_CTLR_ITS_NUMBER ) != (its_number <<GITS_CTLR_ITS_NUMBER_SHIFT )) { its_number = ctlr &GITS_CTLR_ITS_NUMBER ; its_number >>=GITS_CTLR_ITS_NUMBER_SHIFT ;@@ -5048,75 +5047,50 @@ static int __init its_compute_its_list_map(struct resource *res,if (test_and_set_bit (its_number, &its_list_map)) {pr_err ("ITS@%pa: Duplicate ITSList entry %d\n" , - &res->start, its_number); + &its->phys_base, its_number);return -EINVAL ; }return its_number; } -static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +static int __init its_probe_one (struct its_node *its) { -struct its_node *its; -void __iomem *its_base; -u64 baser, tmp, typer; +u64 baser, tmp;struct page *page;u32 ctlr;int err; - its_base =its_map_one (res, &err); -if (!its_base) -return err; - -pr_info ("ITS %pR\n" , res); - - its =kzalloc (sizeof (*its),GFP_KERNEL ); -if (!its) { - err = -ENOMEM ; - goto out_unmap; - } - -raw_spin_lock_init (&its->lock); -mutex_init (&its->dev_alloc_lock); -INIT_LIST_HEAD (&its->entry); -INIT_LIST_HEAD (&its->its_device_list); - typer =gic_read_typer (its_base +GITS_TYPER ); - its->typer = typer; - its->base = its_base; - its->phys_base = res->start;if (is_v4 (its)) { -if (!(typer &GITS_TYPER_VMOVP )) { - err =its_compute_its_list_map (res, its_base); +if (!(its->typer &GITS_TYPER_VMOVP )) { + err =its_compute_its_list_map (its);if (err <0 ) - goto out_free_its; + goto out; its->list_nr = err;pr_info ("ITS@%pa: Using ITS number %d\n" , - &res->start, err); + &its->phys_base, err); }else { -pr_info ("ITS@%pa: Single VMOVP capable\n" , &res->start); +pr_info ("ITS@%pa: Single VMOVP capable\n" , &its->phys_base); }if (is_v4_1 (its)) { -u32 svpet =FIELD_GET (GITS_TYPER_SVPET , typer); +u32 svpet =FIELD_GET (GITS_TYPER_SVPET , its->typer); - its->sgir_base =ioremap (res->start +SZ_128K ,SZ_64K ); + its->sgir_base =ioremap (its->phys_base +SZ_128K ,SZ_64K );if (!its->sgir_base) { err = -ENOMEM ; - goto out_free_its; + goto out; } - its->mpidr =readl_relaxed (its_base +GITS_MPIDR ); + its->mpidr =readl_relaxed (its->base +GITS_MPIDR );pr_info ("ITS@%pa: Using GICv4.1 mode %08x %08x\n" , - &res->start, its->mpidr, svpet); + &its->phys_base, its->mpidr, svpet); } } - its->numa_node = numa_node; - page =alloc_pages_node (its->numa_node,GFP_KERNEL | __GFP_ZERO,get_order (ITS_CMD_QUEUE_SZ ));if (!page) {@@ -5125,12 +5099,9 @@ static int __init its_probe_one(struct resource *res, } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->fwnode_handle = handle; its->get_msi_base = its_irq_get_msi_base; its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; -its_enable_quirks (its); - err =its_alloc_tables (its);if (err) goto out_free_cmd;@@ -5174,7 +5145,7 @@ static int __init its_probe_one(struct resource *res, ctlr |= GITS_CTLR_ImDe;writel_relaxed (ctlr, its->base +GITS_CTLR ); - err =its_init_domain (handle, its); + err =its_init_domain (its);if (err) goto out_free_tables;@@ -5191,11 +5162,8 @@ static int __init its_probe_one(struct resource *res, out_unmap_sgir:if (its->sgir_base)iounmap (its->sgir_base); -out_free_its: -kfree (its); -out_unmap: -iounmap (its_base); -pr_err ("ITS@%pa: failed probing (%d)\n" , &res->start, err); +out: +pr_err ("ITS@%pa: failed probing (%d)\n" , &its->phys_base, err);return err; }@@ -5356,10 +5324,53 @@ static const struct of_device_id its_device_id[] = { {}, }; +static struct its_node __init *its_node_init (struct resource *res, +struct fwnode_handle *handle,int numa_node) +{ +void __iomem *its_base; +struct its_node *its; +int err; + + its_base =its_map_one (res, &err); +if (!its_base) +return NULL ; + +pr_info ("ITS %pR\n" , res); + + its =kzalloc (sizeof (*its),GFP_KERNEL ); +if (!its) + goto out_unmap; + +raw_spin_lock_init (&its->lock); +mutex_init (&its->dev_alloc_lock); +INIT_LIST_HEAD (&its->entry); +INIT_LIST_HEAD (&its->its_device_list); + + its->typer =gic_read_typer (its_base +GITS_TYPER ); + its->base = its_base; + its->phys_base = res->start; + + its->numa_node = numa_node; + its->fwnode_handle = handle; + +return its; + +out_unmap: +iounmap (its_base); +return NULL ; +} + +static void its_node_destroy (struct its_node *its) +{ +iounmap (its->base); +kfree (its); +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +int err;/* * Make sure *all* the ITS are reset before we probe any, as @@ -5369,8 +5380,6 @@ static int __init its_of_probe(struct device_node *node)*/ for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { -int err; -if (!of_device_is_available (np) || !of_property_read_bool (np,"msi-controller" ) ||of_address_to_resource (np,0 , &res))@@ -5383,6 +5392,8 @@ static int __init its_of_probe(struct device_node *node)for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { +struct its_node *its; +if (!of_device_is_available (np))continue ;if (!of_property_read_bool (np,"msi-controller" )) {@@ -5396,7 +5407,17 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); + + its =its_node_init (&res, &np->fwnode,of_node_to_nid (np)); +if (!its) +return -ENOMEM ; + +its_enable_quirks (its); + err =its_probe_one (its); +if (err) { +its_node_destroy (its); +return err; + } }return 0 ; }@@ -5508,6 +5529,7 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, {struct acpi_madt_generic_translator *its_entry;struct fwnode_handle *dom_handle; +struct its_node *its;struct resource res;int err;@@ -5532,11 +5554,20 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto dom_err; } - err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); + its =its_node_init (&res, dom_handle, +acpi_get_its_numa_node (its_entry->translation_id)); +if (!its) { + err = -ENOMEM ; + goto node_err; + } + +/* Stick ACPI quirk handling here */ + + err =its_probe_one (its);if (!err)return 0 ; +node_err:iort_deregister_domain_token (its_entry->translation_id); dom_err:irq_domain_free_fwnode (dom_handle);Signature · 4 lines
2.34.1 -- Without deviation from the norm, progress is not possible.
Re: [PATCH 2/2] irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Marc ZyngierQuoted 43 lines · Marc Zyngier
Quoted 34 lines · Lorenzo Pieralisi
On Tue, Sep 05, 2023 at 12:34:58PM +0100, Marc Zyngier wrote:
[...]
Quoted 15 lines
Quoted 15 lines
drivers/irqchip/irq-gic-v3-its.c +2
-1* Make sure *all* the ITS are reset before we probe any, as * they may be sharing memory. If any of the ITS fails to @@ -5396,7 +5405,8 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); +its_probe_one (&res, &np->fwnode,of_node_to_nid (np), +of_property_read_bool (np,"dma-noncoherent" )); }return 0 ; }@@ -5533,7 +5543,8 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, }Quoted 8 lines
Review static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, · line 5547
Quoted 4 lines
(quoted diff) +2
-1err = its_probe_one(&res, dom_handle, - acpi_get_its_numa_node(its_entry->translation_id)); + acpi_get_its_numa_node(its_entry->translation_id), + false);I came up with the following alternative approach, which is as usual completely untested. It is entirely based on the quirk infrastructure, and doesn't touch the ACPI path at all.
Writing the ACPI bits. We can't use the quirks framework for ACPI (we don't have "properties" and I don't think we want to attach any to the
fwnode_handle) that's why I generalizedits_probe_one()above with an extra param, that would have simplified ACPI parsing:Quoted 6 lines
- we alloc struct
its_nodeinits_probe_one()but at that stage ACPI parsing was already done. If we have to parse the MADT(ITS) again just to scan for non-coherent we then have to match the MADT entries to the *current* struct its_node* we are handling (MADT parsing callbacks don't even take a param - we have to resort to global variables - definitely doable but it is a bit ugly).
Quoted 6 lines
Well, a more acceptable approach would be for
its_probe_one()to take an allocated and possibly pre-populatedits_nodestructure (crucially, with the quirk flags set), which itself results in a bunch of low hanging cleanups, see the patch below.I have boot tested it in a DT guest, so it is obviously perfect.
If you don't mind I will post it together with the resulting series.
I have just removed:
/* Stick ACPI quirk handling here */
point taken :)
Thanks,
Lorenzo
Quoted 324 lines
M.From
978f654d4459adf0b8f3f8e896ca37035b3b114cMon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Tue, 3 Oct 2023 17:35:27 +0100
Subject: [PATCH] irqchip/gic-v3-its: Split allocation from initialisation of
its_nodeIn order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (
its_node) from the actual probing.This will allow firmware-specific hooks to be added between these two points.
Signed-off-by Marc Zyngier <[email protected]>--- drivers/irqchip/irq-gic-v3-its.c | 151 +++++++++++++++++++------------ 1 file changed, 91 insertions(+), 60 deletions(-)drivers/irqchip/irq-gic-v3-its.c +91
-60@@ -4952,7 +4952,7 @@ static void __init __iomem *its_map_one(struct resource *res, int *err)return NULL ; } -static int its_init_domain (struct fwnode_handle *handle,struct its_node *its) +static int its_init_domain (struct its_node *its) {struct irq_domain *inner_domain;struct msi_domain_info *info;@@ -4966,7 +4966,7 @@ static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) inner_domain =irq_domain_create_hierarchy (its_parent, its->msi_domain_flags,0 , - handle, &its_domain_ops, + its->fwnode_handle, &its_domain_ops, info);if (!inner_domain) {kfree (info);@@ -5017,8 +5017,7 @@ static int its_init_vpe_domain(void)return 0 ; } -static int __init its_compute_its_list_map (struct resource *res, -void __iomem *its_base) +static int __init its_compute_its_list_map (struct its_node *its) {int its_number;u32 ctlr;@@ -5032,15 +5031,15 @@ static int __init its_compute_its_list_map(struct resource *res, its_number =find_first_zero_bit (&its_list_map, GICv4_ITS_LIST_MAX);if (its_number >= GICv4_ITS_LIST_MAX) {pr_err ("ITS@%pa: No ITSList entry available!\n" , - &res->start); + &its->phys_base);return -EINVAL ; } - ctlr =readl_relaxed (its_base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR ); ctlr &= ~GITS_CTLR_ITS_NUMBER ; ctlr |= its_number <<GITS_CTLR_ITS_NUMBER_SHIFT ; -writel_relaxed (ctlr, its_base +GITS_CTLR ); - ctlr =readl_relaxed (its_base +GITS_CTLR ); +writel_relaxed (ctlr, its->base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR );if ((ctlr &GITS_CTLR_ITS_NUMBER ) != (its_number <<GITS_CTLR_ITS_NUMBER_SHIFT )) { its_number = ctlr &GITS_CTLR_ITS_NUMBER ; its_number >>=GITS_CTLR_ITS_NUMBER_SHIFT ;@@ -5048,75 +5047,50 @@ static int __init its_compute_its_list_map(struct resource *res,if (test_and_set_bit (its_number, &its_list_map)) {pr_err ("ITS@%pa: Duplicate ITSList entry %d\n" , - &res->start, its_number); + &its->phys_base, its_number);return -EINVAL ; }return its_number; } -static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +static int __init its_probe_one (struct its_node *its) { -struct its_node *its; -void __iomem *its_base; -u64 baser, tmp, typer; +u64 baser, tmp;struct page *page;u32 ctlr;int err; - its_base =its_map_one (res, &err); -if (!its_base) -return err; - -pr_info ("ITS %pR\n" , res); - - its =kzalloc (sizeof (*its),GFP_KERNEL ); -if (!its) { - err = -ENOMEM ; - goto out_unmap; - } - -raw_spin_lock_init (&its->lock); -mutex_init (&its->dev_alloc_lock); -INIT_LIST_HEAD (&its->entry); -INIT_LIST_HEAD (&its->its_device_list); - typer =gic_read_typer (its_base +GITS_TYPER ); - its->typer = typer; - its->base = its_base; - its->phys_base = res->start;if (is_v4 (its)) { -if (!(typer &GITS_TYPER_VMOVP )) { - err =its_compute_its_list_map (res, its_base); +if (!(its->typer &GITS_TYPER_VMOVP )) { + err =its_compute_its_list_map (its);if (err <0 ) - goto out_free_its; + goto out; its->list_nr = err;pr_info ("ITS@%pa: Using ITS number %d\n" , - &res->start, err); + &its->phys_base, err); }else { -pr_info ("ITS@%pa: Single VMOVP capable\n" , &res->start); +pr_info ("ITS@%pa: Single VMOVP capable\n" , &its->phys_base); }if (is_v4_1 (its)) { -u32 svpet =FIELD_GET (GITS_TYPER_SVPET , typer); +u32 svpet =FIELD_GET (GITS_TYPER_SVPET , its->typer); - its->sgir_base =ioremap (res->start +SZ_128K ,SZ_64K ); + its->sgir_base =ioremap (its->phys_base +SZ_128K ,SZ_64K );if (!its->sgir_base) { err = -ENOMEM ; - goto out_free_its; + goto out; } - its->mpidr =readl_relaxed (its_base +GITS_MPIDR ); + its->mpidr =readl_relaxed (its->base +GITS_MPIDR );pr_info ("ITS@%pa: Using GICv4.1 mode %08x %08x\n" , - &res->start, its->mpidr, svpet); + &its->phys_base, its->mpidr, svpet); } } - its->numa_node = numa_node; - page =alloc_pages_node (its->numa_node,GFP_KERNEL | __GFP_ZERO,get_order (ITS_CMD_QUEUE_SZ ));if (!page) {@@ -5125,12 +5099,9 @@ static int __init its_probe_one(struct resource *res, } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->fwnode_handle = handle; its->get_msi_base = its_irq_get_msi_base; its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; -its_enable_quirks (its); - err =its_alloc_tables (its);if (err) goto out_free_cmd;@@ -5174,7 +5145,7 @@ static int __init its_probe_one(struct resource *res, ctlr |= GITS_CTLR_ImDe;writel_relaxed (ctlr, its->base +GITS_CTLR ); - err =its_init_domain (handle, its); + err =its_init_domain (its);if (err) goto out_free_tables;@@ -5191,11 +5162,8 @@ static int __init its_probe_one(struct resource *res, out_unmap_sgir:if (its->sgir_base)iounmap (its->sgir_base); -out_free_its: -kfree (its); -out_unmap: -iounmap (its_base); -pr_err ("ITS@%pa: failed probing (%d)\n" , &res->start, err); +out: +pr_err ("ITS@%pa: failed probing (%d)\n" , &its->phys_base, err);return err; }@@ -5356,10 +5324,53 @@ static const struct of_device_id its_device_id[] = { {}, }; +static struct its_node __init *its_node_init (struct resource *res, +struct fwnode_handle *handle,int numa_node) +{ +void __iomem *its_base; +struct its_node *its; +int err; + + its_base =its_map_one (res, &err); +if (!its_base) +return NULL ; + +pr_info ("ITS %pR\n" , res); + + its =kzalloc (sizeof (*its),GFP_KERNEL ); +if (!its) + goto out_unmap; + +raw_spin_lock_init (&its->lock); +mutex_init (&its->dev_alloc_lock); +INIT_LIST_HEAD (&its->entry); +INIT_LIST_HEAD (&its->its_device_list); + + its->typer =gic_read_typer (its_base +GITS_TYPER ); + its->base = its_base; + its->phys_base = res->start; + + its->numa_node = numa_node; + its->fwnode_handle = handle; + +return its; + +out_unmap: +iounmap (its_base); +return NULL ; +} + +static void its_node_destroy (struct its_node *its) +{ +iounmap (its->base); +kfree (its); +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +int err;/* * Make sure *all* the ITS are reset before we probe any, as @@ -5369,8 +5380,6 @@ static int __init its_of_probe(struct device_node *node)*/ for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { -int err; -if (!of_device_is_available (np) || !of_property_read_bool (np,"msi-controller" ) ||of_address_to_resource (np,0 , &res))@@ -5383,6 +5392,8 @@ static int __init its_of_probe(struct device_node *node)for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { +struct its_node *its; +if (!of_device_is_available (np))continue ;if (!of_property_read_bool (np,"msi-controller" )) {@@ -5396,7 +5407,17 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); + + its =its_node_init (&res, &np->fwnode,of_node_to_nid (np)); +if (!its) +return -ENOMEM ; + +its_enable_quirks (its); + err =its_probe_one (its); +if (err) { +its_node_destroy (its); +return err; + } }return 0 ; }@@ -5508,6 +5529,7 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, {struct acpi_madt_generic_translator *its_entry;struct fwnode_handle *dom_handle; +struct its_node *its;struct resource res;int err;@@ -5532,11 +5554,20 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto dom_err; } - err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); + its =its_node_init (&res, dom_handle, +acpi_get_its_numa_node (its_entry->translation_id)); +if (!its) { + err = -ENOMEM ; + goto node_err; + } + +/* Stick ACPI quirk handling here */ + + err =its_probe_one (its);if (!err)return 0 ; +node_err:iort_deregister_domain_token (its_entry->translation_id); dom_err:irq_domain_free_fwnode (dom_handle);Signature · 4 lines
2.34.1 -- Without deviation from the norm, progress is not possible.
v2 cover irqchip/gic-v3: Enable non-coherent GIC designs probing
In reply to Lorenzo PieralisiThis series is v2 of a previous version[1].
v1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructure
[1] https://lore.kernel.org/all/[email protected]
Original cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.
Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.
When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (2):
dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent
property
irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
.../interrupt-controller/arm,gic-v3.yaml | 12 +++++++++++ drivers/irqchip/irq-gic-common.h | 4 ++++ drivers/irqchip/irq-gic-v3-its.c | 21 +++++++++++++++---- drivers/irqchip/irq-gic-v3.c | 13 ++++++++++++ 4 files changed, 46 insertions(+), 4 deletions(-)
Signature · 1 line
2.34.1
PATCH v2 1/2 dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +12 -0
@@ -106,6 +106,12 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: + Present if the GIC redistributors permit programming shareability + and cacheability attributes but are connected to a non-coherent + downstream interconnect. + msi-controller: description: Only present if the Message Based Interrupt functionality is
@@ -193,6 +199,12 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: + Present if the GIC ITS permits programming shareability and + cacheability attributes but are connected to a non-coherent + downstream interconnect. + msi-controller: true "#msi-cells":
Signature · 1 line
2.34.1
Re: [PATCH v2 1/2] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
Quoted 9 lines · Lorenzo Pieralisi
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
Quoted 6 lines
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Re: [PATCH v2 1/2] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
In reply to Lorenzo PieralisiQuoted 33 lines · Lorenzo Pieralisi
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Rob Herring <[email protected]>--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+)Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +6
-0@@ -106,6 +106,12 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: + Present if the GIC redistributors permit programming shareability + and cacheability attributes but are connected to a non-coherent + downstream interconnect. + msi-controller: description: Only present if the Message Based Interrupt functionality is
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml patternProperties: · line 205
Quoted 7 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +4
-0@@ -193,6 +199,12 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: + Present if the GIC ITS permits programming shareability and + cacheability attributes but are connected to a non-coherent
s/are/is
Sorry, I will update the patch accordingly.
Lorenzo
Quoted 8 lines
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +2
-0+ downstream interconnect. + msi-controller: true "#msi-cells":Signature · 1 line
2.34.1
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v2 2/2 irqchip/gic-v3: Enable non-coherent redistributors/ITSes probing
In reply to Lorenzo PieralisiThe GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers through the GIC quirks mechanism.
--- drivers/irqchip/irq-gic-common.h | 4 ++++ drivers/irqchip/irq-gic-v3-its.c | 21 +++++++++++++++++---- drivers/irqchip/irq-gic-v3.c | 13 +++++++++++++ 3 files changed, 34 insertions(+), 4 deletions(-)
drivers/irqchip/irq-gic-common.h +4 -0
@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */
drivers/irqchip/irq-gic-v3-its.c +17 -4
@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)
@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {
@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } };
@@ -4817,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its)u32 iidr =readl_relaxed (its->base +GITS_IIDR );gic_enable_quirks (iidr, its_quirks, its); + +if (is_of_node (its->fwnode_handle)) +gic_enable_of_quirks (to_of_node (its->fwnode_handle), + its_quirks, its); }static int its_save_disable (void )
drivers/irqchip/irq-gic-v3.c +13 -0
@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,
@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };
Signature · 1 line
2.34.1
Marc Zyngier
maintainerQuoted 29 lines · Lorenzo Pieralisi
This series is v2 of a previous version[1].
v1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructure[1] https://lore.kernel.org/all/[email protected]
Original cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
Quoted 7 lines
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
What is the plan for this last point? I'd like to see at least a proposal before taking this series in.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Quoted 37 lines · Marc Zyngier
Quoted 36 lines · Lorenzo Pieralisi
This series is v2 of a previous version[1].
v1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructure[1]
https://lore.kernel.org/all/[email protected]Original cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.
Quoted 6 lines
Quoted 3 lines
When a consistent approach is agreed upon for DT an equivalent binding will
be put forward for ACPI based systems.What is the plan for this last point? I'd like to see at least a proposal before taking this series in.
Absolutely, I am starting a thread on related MADT changes, should not take too long.
Lorenzo
Quoted 42 lines · Lorenzo Pieralisi
Quoted 41 lines · Marc Zyngier
Quoted 39 lines · Lorenzo Pieralisi
This series is v2 of a previous version[1].
v1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructure[1]
https://lore.kernel.org/all/[email protected]Original cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will
be put forward for ACPI based systems.
Quoted 5 lines
Quoted 2 lines
What is the plan for this last point? I'd like to see at least a proposal before taking this series in.
Absolutely, I am starting a thread on related MADT changes, should not take too long.
Quick update, bindings filed, I will code against it but we should not merge anything till it is approved (could be missing v6.7 timeline).
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
Lorenzo
v3 cover irqchip/gic-v3: Enable non-coherent GIC designs probing
In reply to Lorenzo PieralisiThis series is v3 of previous series:
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]
v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approval
v1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructure
Original cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.
Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.
When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (4):
dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent
property
irqchip/gic-v3: Enable non-coherent redistributors/ITSes DT probing
ACPICA: Add new MADT GICC/GICR/ITS flags handling [code first] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Marc Zyngier (1):
irqchip/gic-v3-its: Split allocation from initialisation of its_node
.../interrupt-controller/arm,gic-v3.yaml | 12 ++ drivers/acpi/processor_core.c | 21 +++ drivers/irqchip/irq-gic-common.h | 12 ++ drivers/irqchip/irq-gic-v3-its.c | 174 +++++++++++------- drivers/irqchip/irq-gic-v3.c | 22 +++ include/acpi/actbl2.h | 11 +- include/linux/acpi.h | 3 + 7 files changed, 189 insertions(+), 66 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/5 dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
--- .../bindings/interrupt-controller/arm,gic-v3.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +12 -0
@@ -106,6 +106,12 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: + Present if the GIC redistributors permit programming shareability + and cacheability attributes but are connected to a non-coherent + downstream interconnect. + msi-controller: description: Only present if the Message Based Interrupt functionality is
@@ -193,6 +199,12 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: + Present if the GIC ITS permits programming shareability and + cacheability attributes but is connected to a non-coherent + downstream interconnect. + msi-controller: true "#msi-cells":
Signature · 1 line
2.34.1
[irqchip: irq/irqchip-fixes] dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
Commit-ID: 5e5c636c69bdba04033161bbb111fbb6f1f6661e Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/5e5c636c69bdba04033161bbb111fbb6f1f6661e Author: Lorenzo Pieralisi <[email protected]> AuthorDate: Fri, 06 Oct 2023 14:59:25 +02:00 Committer: Marc Zyngier <[email protected]> CommitterDate: Sat, 07 Oct 2023 12:47:12 +01:00
dt-bindings: interrupt-controller: arm,gic-v3: Add dma-noncoherent property
The GIC v3 specifications allow redistributors and ITSes interconnect ports used to access memory to be wired up in a way that makes the respective initiators/memory observers non-coherent.
Add the standard dma-noncoherent property to the GICv3 bindings to allow firmware to describe the redistributors/ITSes components and interconnect ports behaviour in system designs where the redistributors and ITSes are not coherent with the CPU.
--- Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml +12 -0
@@ -106,6 +106,12 @@ properties: $ref: /schemas/types.yaml#/definitions/uint32 maximum: 4096 + dma-noncoherent: + description: + Present if the GIC redistributors permit programming shareability + and cacheability attributes but are connected to a non-coherent + downstream interconnect. + msi-controller: description: Only present if the Message Based Interrupt functionality is
@@ -193,6 +199,12 @@ patternProperties: compatible: const: arm,gic-v3-its + dma-noncoherent: + description: + Present if the GIC ITS permits programming shareability and + cacheability attributes but is connected to a non-coherent + downstream interconnect. + msi-controller: true "#msi-cells":
PATCH v3 2/5 irqchip/gic-v3: Enable non-coherent redistributors/ITSes DT probing
In reply to Lorenzo PieralisiThe GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers through the GIC quirks mechanism.
--- drivers/irqchip/irq-gic-common.h | 4 ++++ drivers/irqchip/irq-gic-v3-its.c | 21 +++++++++++++++++---- drivers/irqchip/irq-gic-v3.c | 13 +++++++++++++ 3 files changed, 34 insertions(+), 4 deletions(-)
drivers/irqchip/irq-gic-common.h +4 -0
@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */
drivers/irqchip/irq-gic-v3-its.c +17 -4
@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)
@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {
@@ -4808,6 +4812,11 @@ static const struct gic_quirk its_quirks[] = { .init = its_enable_rk3588001, },#endif + { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, { } };
@@ -4817,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its)u32 iidr =readl_relaxed (its->base +GITS_IIDR );gic_enable_quirks (iidr, its_quirks, its); + +if (is_of_node (its->fwnode_handle)) +gic_enable_of_quirks (to_of_node (its->fwnode_handle), + its_quirks, its); }static int its_save_disable (void )
drivers/irqchip/irq-gic-v3.c +13 -0
@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,
@@ -1923,6 +1931,11 @@ static const struct gic_quirk gic_quirks[] = { .mask = 0xff0f0fff, .init = gic_enable_quirk_arm64_2941627, }, + { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, { } };
Signature · 1 line
2.34.1
[irqchip: irq/irqchip-fixes] irqchip/gic-v3: Enable non-coherent redistributors/ITSes DT probing
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
Commit-ID: 3a0fff0fb6a3861fa05416f21858cf0c75cbf944 Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/3a0fff0fb6a3861fa05416f21858cf0c75cbf944 Author: Lorenzo Pieralisi <[email protected]> AuthorDate: Fri, 06 Oct 2023 14:59:26 +02:00 Committer: Marc Zyngier <[email protected]> CommitterDate: Sat, 07 Oct 2023 12:47:12 +01:00
irqchip/gic-v3: Enable non-coherent redistributors/ITSes DT probing
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs, introduce the "dma-noncoherent" device tree property to allow firmware to describe redistributors and ITSes as non-coherent observers on the memory interconnect and use the property to force the shareability attributes to be programmed into the redistributors and ITSes registers through the GIC quirks mechanism.
--- drivers/irqchip/irq-gic-common.h | 4 ++++ drivers/irqchip/irq-gic-v3-its.c | 21 +++++++++++++++++---- drivers/irqchip/irq-gic-v3.c | 13 +++++++++++++ 3 files changed, 34 insertions(+), 4 deletions(-)
drivers/irqchip/irq-gic-common.h +4 -0
@@ -29,4 +29,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) +#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) +#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) +#endif /* _IRQ_GIC_COMMON_H */
drivers/irqchip/irq-gic-v3-its.c +17 -4
@@ -44,10 +44,6 @@#define ITS_FLAGS_WORKAROUND_CAVIUM_23144 (1ULL << 2)#define ITS_FLAGS_FORCE_NON_SHAREABLE (1ULL << 3) -#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0) -#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1) -#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2) -#define RD_LOCAL_LPI_ENABLED BIT(0)#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)#define RD_LOCAL_MEMRESERVE_DONE BIT(2)
@@ -4754,6 +4750,14 @@ static bool __maybe_unused its_enable_rk3588001(void *data)return true; } +static bool its_set_non_coherent (void *data) +{ +struct its_node *its = data; + + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} + static const struct gic_quirk its_quirks[] = { #ifdef CONFIG_CAVIUM_ERRATUM_22375 {
@@ -4809,6 +4813,11 @@ static const struct gic_quirk its_quirks[] = { },#endif { + .desc = "ITS: non-coherent attribute", + .property = "dma-noncoherent", + .init = its_set_non_coherent, + }, + { } };
@@ -4817,6 +4826,10 @@ static void its_enable_quirks(struct its_node *its)u32 iidr =readl_relaxed (its->base +GITS_IIDR );gic_enable_quirks (iidr, its_quirks, its); + +if (is_of_node (its->fwnode_handle)) +gic_enable_of_quirks (to_of_node (its->fwnode_handle), + its_quirks, its); }static int its_save_disable (void )
drivers/irqchip/irq-gic-v3.c +13 -0
@@ -1857,6 +1857,14 @@ static bool gic_enable_quirk_arm64_2941627(void *data)return true; } +static bool rd_set_non_coherent (void *data) +{ +struct gic_chip_data *d = data; + + d->rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +return true; +} +static const struct gic_quirk gic_quirks[] = { { .desc ="GICv3: Qualcomm MSM8996 broken firmware" ,
@@ -1924,6 +1932,11 @@ static const struct gic_quirk gic_quirks[] = { .init = gic_enable_quirk_arm64_2941627, }, { + .desc = "GICv3: non-coherent attribute", + .property = "dma-noncoherent", + .init = rd_set_non_coherent, + }, + { } };
PATCH v3 3/5 irqchip/gic-v3-its: Split allocation from initialisation of its_node
In reply to Lorenzo PieralisiFrom: Marc Zyngier <[email protected]>
In order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (its_node) from the actual probing.
This will allow firmware-specific hooks to be added between these two points.
--- drivers/irqchip/irq-gic-v3-its.c | 149 ++++++++++++++++++------------- 1 file changed, 89 insertions(+), 60 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +89 -60
@@ -4965,7 +4965,7 @@ static void __init __iomem *its_map_one(struct resource *res, int *err)return NULL ; } -static int its_init_domain (struct fwnode_handle *handle,struct its_node *its) +static int its_init_domain (struct its_node *its) {struct irq_domain *inner_domain;struct msi_domain_info *info;
@@ -4979,7 +4979,7 @@ static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) inner_domain =irq_domain_create_hierarchy (its_parent, its->msi_domain_flags,0 , - handle, &its_domain_ops, + its->fwnode_handle, &its_domain_ops, info);if (!inner_domain) {kfree (info);
@@ -5030,8 +5030,7 @@ static int its_init_vpe_domain(void)return 0 ; } -static int __init its_compute_its_list_map (struct resource *res, -void __iomem *its_base) +static int __init its_compute_its_list_map (struct its_node *its) {int its_number;u32 ctlr;
@@ -5045,15 +5044,15 @@ static int __init its_compute_its_list_map(struct resource *res, its_number =find_first_zero_bit (&its_list_map, GICv4_ITS_LIST_MAX);if (its_number >= GICv4_ITS_LIST_MAX) {pr_err ("ITS@%pa: No ITSList entry available!\n" , - &res->start); + &its->phys_base);return -EINVAL ; } - ctlr =readl_relaxed (its_base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR ); ctlr &= ~GITS_CTLR_ITS_NUMBER ; ctlr |= its_number <<GITS_CTLR_ITS_NUMBER_SHIFT ; -writel_relaxed (ctlr, its_base +GITS_CTLR ); - ctlr =readl_relaxed (its_base +GITS_CTLR ); +writel_relaxed (ctlr, its->base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR );if ((ctlr &GITS_CTLR_ITS_NUMBER ) != (its_number <<GITS_CTLR_ITS_NUMBER_SHIFT )) { its_number = ctlr &GITS_CTLR_ITS_NUMBER ; its_number >>=GITS_CTLR_ITS_NUMBER_SHIFT ;
@@ -5061,75 +5060,50 @@ static int __init its_compute_its_list_map(struct resource *res,if (test_and_set_bit (its_number, &its_list_map)) {pr_err ("ITS@%pa: Duplicate ITSList entry %d\n" , - &res->start, its_number); + &its->phys_base, its_number);return -EINVAL ; }return its_number; } -static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +static int __init its_probe_one (struct its_node *its) { -struct its_node *its; -void __iomem *its_base; -u64 baser, tmp, typer; +u64 baser, tmp;struct page *page;u32 ctlr;int err; - its_base =its_map_one (res, &err); -if (!its_base) -return err; - -pr_info ("ITS %pR\n" , res); - - its =kzalloc (sizeof (*its),GFP_KERNEL ); -if (!its) { - err = -ENOMEM ; - goto out_unmap; - } - -raw_spin_lock_init (&its->lock); -mutex_init (&its->dev_alloc_lock); -INIT_LIST_HEAD (&its->entry); -INIT_LIST_HEAD (&its->its_device_list); - typer =gic_read_typer (its_base +GITS_TYPER ); - its->typer = typer; - its->base = its_base; - its->phys_base = res->start;if (is_v4 (its)) { -if (!(typer &GITS_TYPER_VMOVP )) { - err =its_compute_its_list_map (res, its_base); +if (!(its->typer &GITS_TYPER_VMOVP )) { + err =its_compute_its_list_map (its);if (err <0 ) - goto out_free_its; + goto out; its->list_nr = err;pr_info ("ITS@%pa: Using ITS number %d\n" , - &res->start, err); + &its->phys_base, err); }else { -pr_info ("ITS@%pa: Single VMOVP capable\n" , &res->start); +pr_info ("ITS@%pa: Single VMOVP capable\n" , &its->phys_base); }if (is_v4_1 (its)) { -u32 svpet =FIELD_GET (GITS_TYPER_SVPET , typer); +u32 svpet =FIELD_GET (GITS_TYPER_SVPET , its->typer); - its->sgir_base =ioremap (res->start +SZ_128K ,SZ_64K ); + its->sgir_base =ioremap (its->phys_base +SZ_128K ,SZ_64K );if (!its->sgir_base) { err = -ENOMEM ; - goto out_free_its; + goto out; } - its->mpidr =readl_relaxed (its_base +GITS_MPIDR ); + its->mpidr =readl_relaxed (its->base +GITS_MPIDR );pr_info ("ITS@%pa: Using GICv4.1 mode %08x %08x\n" , - &res->start, its->mpidr, svpet); + &its->phys_base, its->mpidr, svpet); } } - its->numa_node = numa_node; - page =alloc_pages_node (its->numa_node,GFP_KERNEL | __GFP_ZERO,get_order (ITS_CMD_QUEUE_SZ ));if (!page) {
@@ -5138,12 +5112,9 @@ static int __init its_probe_one(struct resource *res, } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->fwnode_handle = handle; its->get_msi_base = its_irq_get_msi_base; its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; -its_enable_quirks (its); - err =its_alloc_tables (its);if (err) goto out_free_cmd;
@@ -5187,7 +5158,7 @@ static int __init its_probe_one(struct resource *res, ctlr |= GITS_CTLR_ImDe;writel_relaxed (ctlr, its->base +GITS_CTLR ); - err =its_init_domain (handle, its); + err =its_init_domain (its);if (err) goto out_free_tables;
@@ -5204,11 +5175,8 @@ static int __init its_probe_one(struct resource *res, out_unmap_sgir:if (its->sgir_base)iounmap (its->sgir_base); -out_free_its: -kfree (its); -out_unmap: -iounmap (its_base); -pr_err ("ITS@%pa: failed probing (%d)\n" , &res->start, err); +out: +pr_err ("ITS@%pa: failed probing (%d)\n" , &its->phys_base, err);return err; }
@@ -5369,10 +5337,53 @@ static const struct of_device_id its_device_id[] = { {}, }; +static struct its_node __init *its_node_init (struct resource *res, +struct fwnode_handle *handle,int numa_node) +{ +void __iomem *its_base; +struct its_node *its; +int err; + + its_base =its_map_one (res, &err); +if (!its_base) +return NULL ; + +pr_info ("ITS %pR\n" , res); + + its =kzalloc (sizeof (*its),GFP_KERNEL ); +if (!its) + goto out_unmap; + +raw_spin_lock_init (&its->lock); +mutex_init (&its->dev_alloc_lock); +INIT_LIST_HEAD (&its->entry); +INIT_LIST_HEAD (&its->its_device_list); + + its->typer =gic_read_typer (its_base +GITS_TYPER ); + its->base = its_base; + its->phys_base = res->start; + + its->numa_node = numa_node; + its->fwnode_handle = handle; + +return its; + +out_unmap: +iounmap (its_base); +return NULL ; +} + +static void its_node_destroy (struct its_node *its) +{ +iounmap (its->base); +kfree (its); +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +int err;/* * Make sure *all* the ITS are reset before we probe any, as
@@ -5382,8 +5393,6 @@ static int __init its_of_probe(struct device_node *node)*/ for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { -int err; -if (!of_device_is_available (np) || !of_property_read_bool (np,"msi-controller" ) ||of_address_to_resource (np,0 , &res))
@@ -5396,6 +5405,8 @@ static int __init its_of_probe(struct device_node *node)for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { +struct its_node *its; +if (!of_device_is_available (np))continue ;if (!of_property_read_bool (np,"msi-controller" )) {
@@ -5409,7 +5420,17 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); + + its =its_node_init (&res, &np->fwnode,of_node_to_nid (np)); +if (!its) +return -ENOMEM ; + +its_enable_quirks (its); + err =its_probe_one (its); +if (err) { +its_node_destroy (its); +return err; + } }return 0 ; }
@@ -5521,6 +5542,7 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, {struct acpi_madt_generic_translator *its_entry;struct fwnode_handle *dom_handle; +struct its_node *its;struct resource res;int err;
@@ -5545,11 +5567,18 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto dom_err; } - err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); + its =its_node_init (&res, dom_handle, +acpi_get_its_numa_node (its_entry->translation_id)); +if (!its) { + err = -ENOMEM ; + goto node_err; + } + + err =its_probe_one (its);if (!err)return 0 ; +node_err:iort_deregister_domain_token (its_entry->translation_id); dom_err:irq_domain_free_fwnode (dom_handle);
Signature · 1 line
2.34.1
[irqchip: irq/irqchip-fixes] irqchip/gic-v3-its: Split allocation from initialisation of its_node
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
Commit-ID: 9585a495ac936049dba141e8f9d99159ca06d46a Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/9585a495ac936049dba141e8f9d99159ca06d46a Author: Marc Zyngier <[email protected]> AuthorDate: Fri, 06 Oct 2023 14:59:27 +02:00 Committer: Marc Zyngier <[email protected]> CommitterDate: Sat, 07 Oct 2023 12:47:12 +01:00
irqchip/gic-v3-its: Split allocation from initialisation of its_node
In order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (its_node) from the actual probing.
This will allow firmware-specific hooks to be added between these two points.
--- drivers/irqchip/irq-gic-v3-its.c | 149 +++++++++++++++++------------- 1 file changed, 89 insertions(+), 60 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +89 -60
@@ -4952,7 +4952,7 @@ out_unmap:return NULL ; } -static int its_init_domain (struct fwnode_handle *handle,struct its_node *its) +static int its_init_domain (struct its_node *its) {struct irq_domain *inner_domain;struct msi_domain_info *info;
@@ -4966,7 +4966,7 @@ static int its_init_domain(struct fwnode_handle *handle, struct its_node *its) inner_domain =irq_domain_create_hierarchy (its_parent, its->msi_domain_flags,0 , - handle, &its_domain_ops, + its->fwnode_handle, &its_domain_ops, info);if (!inner_domain) {kfree (info);
@@ -5017,8 +5017,7 @@ static int its_init_vpe_domain(void)return 0 ; } -static int __init its_compute_its_list_map (struct resource *res, -void __iomem *its_base) +static int __init its_compute_its_list_map (struct its_node *its) {int its_number;u32 ctlr;
@@ -5032,15 +5031,15 @@ static int __init its_compute_its_list_map(struct resource *res, its_number =find_first_zero_bit (&its_list_map, GICv4_ITS_LIST_MAX);if (its_number >= GICv4_ITS_LIST_MAX) {pr_err ("ITS@%pa: No ITSList entry available!\n" , - &res->start); + &its->phys_base);return -EINVAL ; } - ctlr =readl_relaxed (its_base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR ); ctlr &= ~GITS_CTLR_ITS_NUMBER ; ctlr |= its_number <<GITS_CTLR_ITS_NUMBER_SHIFT ; -writel_relaxed (ctlr, its_base +GITS_CTLR ); - ctlr =readl_relaxed (its_base +GITS_CTLR ); +writel_relaxed (ctlr, its->base +GITS_CTLR ); + ctlr =readl_relaxed (its->base +GITS_CTLR );if ((ctlr &GITS_CTLR_ITS_NUMBER ) != (its_number <<GITS_CTLR_ITS_NUMBER_SHIFT )) { its_number = ctlr &GITS_CTLR_ITS_NUMBER ; its_number >>=GITS_CTLR_ITS_NUMBER_SHIFT ;
@@ -5048,75 +5047,50 @@ static int __init its_compute_its_list_map(struct resource *res,if (test_and_set_bit (its_number, &its_list_map)) {pr_err ("ITS@%pa: Duplicate ITSList entry %d\n" , - &res->start, its_number); + &its->phys_base, its_number);return -EINVAL ; }return its_number; } -static int __init its_probe_one (struct resource *res, -struct fwnode_handle *handle,int numa_node) +static int __init its_probe_one (struct its_node *its) { -struct its_node *its; -void __iomem *its_base; -u64 baser, tmp, typer; +u64 baser, tmp;struct page *page;u32 ctlr;int err; - its_base =its_map_one (res, &err); -if (!its_base) -return err; - -pr_info ("ITS %pR\n" , res); - - its =kzalloc (sizeof (*its),GFP_KERNEL ); -if (!its) { - err = -ENOMEM ; - goto out_unmap; - } - -raw_spin_lock_init (&its->lock); -mutex_init (&its->dev_alloc_lock); -INIT_LIST_HEAD (&its->entry); -INIT_LIST_HEAD (&its->its_device_list); - typer =gic_read_typer (its_base +GITS_TYPER ); - its->typer = typer; - its->base = its_base; - its->phys_base = res->start;if (is_v4 (its)) { -if (!(typer &GITS_TYPER_VMOVP )) { - err =its_compute_its_list_map (res, its_base); +if (!(its->typer &GITS_TYPER_VMOVP )) { + err =its_compute_its_list_map (its);if (err <0 ) - goto out_free_its; + goto out; its->list_nr = err;pr_info ("ITS@%pa: Using ITS number %d\n" , - &res->start, err); + &its->phys_base, err); }else { -pr_info ("ITS@%pa: Single VMOVP capable\n" , &res->start); +pr_info ("ITS@%pa: Single VMOVP capable\n" , &its->phys_base); }if (is_v4_1 (its)) { -u32 svpet =FIELD_GET (GITS_TYPER_SVPET , typer); +u32 svpet =FIELD_GET (GITS_TYPER_SVPET , its->typer); - its->sgir_base =ioremap (res->start +SZ_128K ,SZ_64K ); + its->sgir_base =ioremap (its->phys_base +SZ_128K ,SZ_64K );if (!its->sgir_base) { err = -ENOMEM ; - goto out_free_its; + goto out; } - its->mpidr =readl_relaxed (its_base +GITS_MPIDR ); + its->mpidr =readl_relaxed (its->base +GITS_MPIDR );pr_info ("ITS@%pa: Using GICv4.1 mode %08x %08x\n" , - &res->start, its->mpidr, svpet); + &its->phys_base, its->mpidr, svpet); } } - its->numa_node = numa_node; - page =alloc_pages_node (its->numa_node,GFP_KERNEL | __GFP_ZERO,get_order (ITS_CMD_QUEUE_SZ ));if (!page) {
@@ -5125,12 +5099,9 @@ static int __init its_probe_one(struct resource *res, } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->fwnode_handle = handle; its->get_msi_base = its_irq_get_msi_base; its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; -its_enable_quirks (its); - err =its_alloc_tables (its);if (err) goto out_free_cmd;
@@ -5174,7 +5145,7 @@ static int __init its_probe_one(struct resource *res, ctlr |= GITS_CTLR_ImDe;writel_relaxed (ctlr, its->base +GITS_CTLR ); - err =its_init_domain (handle, its); + err =its_init_domain (its);if (err) goto out_free_tables;
@@ -5191,11 +5162,8 @@ out_free_cmd: out_unmap_sgir:if (its->sgir_base)iounmap (its->sgir_base); -out_free_its: -kfree (its); -out_unmap: -iounmap (its_base); -pr_err ("ITS@%pa: failed probing (%d)\n" , &res->start, err); +out: +pr_err ("ITS@%pa: failed probing (%d)\n" , &its->phys_base, err);return err; }
@@ -5356,10 +5324,53 @@ static const struct of_device_id its_device_id[] = { {}, }; +static struct its_node __init *its_node_init (struct resource *res, +struct fwnode_handle *handle,int numa_node) +{ +void __iomem *its_base; +struct its_node *its; +int err; + + its_base =its_map_one (res, &err); +if (!its_base) +return NULL ; + +pr_info ("ITS %pR\n" , res); + + its =kzalloc (sizeof (*its),GFP_KERNEL ); +if (!its) + goto out_unmap; + +raw_spin_lock_init (&its->lock); +mutex_init (&its->dev_alloc_lock); +INIT_LIST_HEAD (&its->entry); +INIT_LIST_HEAD (&its->its_device_list); + + its->typer =gic_read_typer (its_base +GITS_TYPER ); + its->base = its_base; + its->phys_base = res->start; + + its->numa_node = numa_node; + its->fwnode_handle = handle; + +return its; + +out_unmap: +iounmap (its_base); +return NULL ; +} + +static void its_node_destroy (struct its_node *its) +{ +iounmap (its->base); +kfree (its); +} +static int __init its_of_probe (struct device_node *node) {struct device_node *np;struct resource res; +int err;/* * Make sure *all* the ITS are reset before we probe any, as
@@ -5369,8 +5380,6 @@ static int __init its_of_probe(struct device_node *node)*/ for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { -int err; -if (!of_device_is_available (np) || !of_property_read_bool (np,"msi-controller" ) ||of_address_to_resource (np,0 , &res))
@@ -5383,6 +5392,8 @@ static int __init its_of_probe(struct device_node *node)for (np =of_find_matching_node (node, its_device_id); np; np =of_find_matching_node (np, its_device_id)) { +struct its_node *its; +if (!of_device_is_available (np))continue ;if (!of_property_read_bool (np,"msi-controller" )) {
@@ -5396,7 +5407,17 @@ static int __init its_of_probe(struct device_node *node)continue ; } -its_probe_one (&res, &np->fwnode,of_node_to_nid (np)); + + its =its_node_init (&res, &np->fwnode,of_node_to_nid (np)); +if (!its) +return -ENOMEM ; + +its_enable_quirks (its); + err =its_probe_one (its); +if (err) { +its_node_destroy (its); +return err; + } }return 0 ; }
@@ -5508,6 +5529,7 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, {struct acpi_madt_generic_translator *its_entry;struct fwnode_handle *dom_handle; +struct its_node *its;struct resource res;int err;
@@ -5532,11 +5554,18 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto dom_err; } - err =its_probe_one (&res, dom_handle, -acpi_get_its_numa_node (its_entry->translation_id)); + its =its_node_init (&res, dom_handle, +acpi_get_its_numa_node (its_entry->translation_id)); +if (!its) { + err = -ENOMEM ; + goto node_err; + } + + err =its_probe_one (its);if (!err)return 0 ; +node_err:iort_deregister_domain_token (its_entry->translation_id); dom_err:irq_domain_free_fwnode (dom_handle);
Re: [PATCH v3 3/5] irqchip/gic-v3-its: Split allocation from initialisation of its_node
In reply to Lorenzo PieralisiHi,
Quoted 5 lines · Lorenzo Pieralisi
From: Marc Zyngier <[email protected]>
In order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (
its_node) from the actual probing.
it seems that this change breaks MSI-X (MSI?) reception on at least the TI AM64x, probably most/all of TI's recent devices (K3).
These devices rely on a quirk CONFIG_SOCIONEXT_SYNQUACER_PREITS that uses an address from the dts specified e.g. as
socionext,synquacer-pre-its = <0x1000000 0x400000>;
to configure a MSI base address that differs from the ARM default.
With this change, the quirk still sets its->get_msi_base and clears IRQ_DOMAIN_FLAG_ISOLATED_MSI from its->msi_domain_flags during its_of_probe, but both get overwritten again during its_probe_one with the defaults.
Previously the defaults would be set first and then the quirks were applied.
I have no idea whether TI's use of this quirk was "correct", but it did work, and since 6.6-rc6 MSI-X has been broken for us.
Best Regards,
Dominic
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Marc Zyngier
maintainerRe: [PATCH v3 3/5] irqchip/gic-v3-its: Split allocation from initialisation of its_node
Quoted 13 lines · Dominic Rath
Hi,
Quoted 5 lines · Lorenzo Pieralisi
From: Marc Zyngier <[email protected]>
In order to pave the way for more fancy quirk handling without making more of a mess of this terrible driver, split the allocation of the ITS descriptor (
its_node) from the actual probing.it seems that this change breaks MSI-X (MSI?) reception on at least the TI AM64x, probably most/all of TI's recent devices (K3).
Quoted 6 lines
These devices rely on a quirk
CONFIG_SOCIONEXT_SYNQUACER_PREITSthat uses an address from the dts specified e.g. associonext,synquacer-pre-its = <0x1000000 0x400000>;to configure a MSI base address that differs from the ARM default.
<rant>
Why on Earth are people using a property that is specific to another implementation? Not only the HW is braindead, but this is now tied to whatever additional implementation quirks we find...
This is just dumb.
</rant>
Quoted 7 lines
With this change, the quirk still sets
its->get_msi_baseand clearsIRQ_DOMAIN_FLAG_ISOLATED_MSIfromits->msi_domain_flagsduringits_of_probe, but both get overwritten again duringits_probe_onewith the defaults.Previously the defaults would be set first and then the quirks were applied.
Yeah, that's clearly a regression, and I've confirmed it on my Synquacer (which means the TI folks have accurately copied a dumb idea). Can you please give the patch below a go on your system and confirm asap whether it works for you?
Quoted 2 lines
I have no idea whether TI's use of this quirk was "correct", but it did work, and since 6.6-rc6 MSI-X has been broken for us.
Just as for bad SW, the worse HW ideas get replicated. Then I write bad SW for it.
Thanks,
M.
From b5571a69f09733ecfa0c944cc48baced6590d024 Mon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Tue, 24 Oct 2023 11:07:34 +0100
Subject: [PATCH] irqchip/gic-v3-its: Don't override quirk settings with
default values
When splitting the allocation of the ITS node from its configuration, some of the default settings were kept in the latter instead of being moved to the former.
This has the side effect of negating some of the quirk detection that have happened in between, amongst which the dreaded Synquacer hack (that also affect Dominic's TI platform).
Move the initialisation of these fields early, so that they can again be overriden by the Synquacer quirk.
Reported by: Dominic Rath <[email protected]>
--- drivers/irqchip/irq-gic-v3-its.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +2 -2
@@ -5112,8 +5112,6 @@ static int __init its_probe_one(struct its_node *its) } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->get_msi_base = its_irq_get_msi_base; - its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; err =its_alloc_tables (its);if (err)
@@ -5362,6 +5360,8 @@ static struct its_node __init *its_node_init(struct resource *res, its->typer =gic_read_typer (its_base +GITS_TYPER ); its->base = its_base; its->phys_base = res->start; + its->get_msi_base = its_irq_get_msi_base; + its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; its->numa_node = numa_node; its->fwnode_handle = handle;
Signature · 5 lines
2.39.2 -- Without deviation from the norm, progress is not possible.
Dominic Rath
Tested-byRe: [PATCH v3 3/5] irqchip/gic-v3-its: Split allocation from initialisation of its_node
Quoted 5 lines · Marc Zyngier
Yeah, that's clearly a regression, and I've confirmed it on my Synquacer (which means the TI folks have accurately copied a dumb idea). Can you please give the patch below a go on your system and confirm asap whether it works for you?
Thanks a lot, with that patch applied on top of 6.6-rc6 MSI-X interrupts work again for the AM64x.
Best Regards,
Dominic
Quoted 24 lines
Quoted 2 lines
I have no idea whether TI's use of this quirk was "correct", but it did work, and since 6.6-rc6 MSI-X has been broken for us.
Just as for bad SW, the worse HW ideas get replicated. Then I write bad SW for it.
Thanks,
M.From
b5571a69f09733ecfa0c944cc48baced6590d024Mon Sep 17 00:00:00 2001
From: Marc Zyngier <[email protected]>
Date: Tue, 24 Oct 2023 11:07:34 +0100
Subject: [PATCH] irqchip/gic-v3-its: Don't override quirk settings with
default valuesWhen splitting the allocation of the ITS node from its configuration, some of the default settings were kept in the latter instead of being moved to the former.
This has the side effect of negating some of the quirk detection that have happened in between, amongst which the dreaded Synquacer hack (that also affect Dominic's TI platform).
Quoted 6 lines
Move the initialisation of these fields early, so that they can again be overriden by the Synquacer quirk.
Fixes 9585a495ac93 ("irqchip/gic-v3-its: Split allocation from initialisation of its_node")Reported by: Dominic Rath <[email protected]>
Signed-off-by Marc Zyngier <[email protected]>
Quoted 33 lines
--- drivers/irqchip/irq-gic-v3-its.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/irqchip/irq-gic-v3-its.c +2
-2@@ -5112,8 +5112,6 @@ static int __init its_probe_one(struct its_node *its) } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->get_msi_base = its_irq_get_msi_base; - its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; err =its_alloc_tables (its);if (err)@@ -5362,6 +5360,8 @@ static struct its_node __init *its_node_init(struct resource *res, its->typer =gic_read_typer (its_base +GITS_TYPER ); its->base = its_base; its->phys_base = res->start; + its->get_msi_base = its_irq_get_msi_base; + its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; its->numa_node = numa_node; its->fwnode_handle = handle;Signature · 5 lines
2.39.2 -- Without deviation from the norm, progress is not possible.
[tip: irq/urgent] irqchip/gic-v3-its: Don't override quirk settings with default values
In reply to Dominic RathThe following commit has been merged into the irq/urgent branch of tip:
The following commit has been merged into the irq/urgent branch of tip:
Commit-ID: f199bf5bf84c19a4f488a39d7d694ab10787de35 Gitweb: https://git.kernel.org/tip/f199bf5bf84c19a4f488a39d7d694ab10787de35 Author: Marc Zyngier <[email protected]> AuthorDate: Tue, 24 Oct 2023 15:34:31 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 25 Oct 2023 21:44:49 +02:00
irqchip/gic-v3-its: Don't override quirk settings with default values
When splitting the allocation of the ITS node from its configuration, some of the default settings were kept in the latter instead of being moved to the former.
This has the side effect of negating some of the quirk detections that have happened in between, amongst which the dreaded Synquacer hack (that also affect Dominic's TI platform).
Move the initialisation of these fields early, so that they can again be overriden by the Synquacer quirk.
Reported by: Dominic Rath <[email protected]>
--- drivers/irqchip/irq-gic-v3-its.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +2 -2
@@ -5112,8 +5112,6 @@ static int __init its_probe_one(struct its_node *its) } its->cmd_base = (void *)page_address (page); its->cmd_write = its->cmd_base; - its->get_msi_base = its_irq_get_msi_base; - its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; err =its_alloc_tables (its);if (err)
@@ -5362,6 +5360,8 @@ static struct its_node __init *its_node_init(struct resource *res, its->typer =gic_read_typer (its_base +GITS_TYPER ); its->base = its_base; its->phys_base = res->start; + its->get_msi_base = its_irq_get_msi_base; + its->msi_domain_flags =IRQ_DOMAIN_FLAG_ISOLATED_MSI ; its->numa_node = numa_node; its->fwnode_handle = handle;
PATCH v3 4/5 ACPICA: Add new MADT GICC/GICR/ITS flags handling [code first]
In reply to Lorenzo PieralisiAdd new flags and related fields to the MADT GICC/GICR/ITS structures according to the code first ECR:
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
Temporary code waiting for ECR approval, for testing purpose only - eventually ACPICA changes will trickle into the kernel from the ACPICA project repos.
--- include/acpi/actbl2.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
include/acpi/actbl2.h +9 -2
@@ -1046,6 +1046,7 @@ struct acpi_madt_generic_interrupt {/* ACPI_MADT_ENABLED (1) Processor is usable if set */ #define ACPI_MADT_PERFORMANCE_IRQ_MODE (1<<1)/* 01: Performance Interrupt Mode */ #define ACPI_MADT_VGIC_IRQ_MODE (1<<2)/* 02: VGIC Maintenance Interrupt mode */ +#define ACPI_MADT_GICC_NON_COHERENT (1<<4)/* 04: GIC redistributor is not coherent */ /* 12: Generic Distributor (ACPI 5.0 + ACPI 6.0 changes) */
@@ -1090,21 +1091,27 @@ struct acpi_madt_generic_msi_frame {struct acpi_madt_generic_redistributor {struct acpi_subtable_header header; -u16 reserved;/* reserved - must be zero */ +u8 flags; +u8 reserved;/* reserved - must be zero */ u64 base_address;u32 length; }; +#define ACPI_MADT_GICR_NON_COHERENT (1) +/* 15: Generic Translator (ACPI 6.0) */ struct acpi_madt_generic_translator {struct acpi_subtable_header header; -u16 reserved;/* reserved - must be zero */ +u8 flags; +u8 reserved;/* reserved - must be zero */ u32 translation_id;u64 base_address;u32 reserved2; }; +#define ACPI_MADT_ITS_NON_COHERENT (1) +/* 16: Multiprocessor wakeup (ACPI 6.4) */ struct acpi_madt_multiproc_wakeup {
Signature · 1 line
2.34.1
PATCH v3 5/5 irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
In reply to Lorenzo PieralisiThe GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).
--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)
drivers/acpi/processor_core.c +21 -0
@@ -215,6 +215,27 @@ phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id)return rv; } +u8 __init acpi_get_madt_revision (void ) +{ +static u8 madt_revision__initdata ; +static bool madt_read__initdata ; +struct acpi_table_header *madt =NULL ; + +if (!madt_read) { + madt_read = true; + +acpi_get_table (ACPI_SIG_MADT ,0 , &madt); +if (!madt) +return madt_revision; + + madt_revision = madt->revision; + +acpi_put_table (madt); + } + +return madt_revision; +} +static phys_cpuid_t map_mat_entry (acpi_handle handle,int type,u32 acpi_id) {struct acpi_buffer buffer = {ACPI_ALLOCATE_BUFFER ,NULL };
drivers/irqchip/irq-gic-common.h +8 -0
@@ -6,6 +6,7 @@#ifndef _IRQ_GIC_COMMON_H#define _IRQ_GIC_COMMON_H +#include <linux/acpi.h> #include <linux/of.h> #include <linux/irqdomain.h> #include <linux/irqchip/arm-gic-common.h>
@@ -29,6 +30,13 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#ifdef CONFIG_ACPI +static inline bool gic_acpi_non_coherent_flag (u32 flags,u32 mask) +{ +return (acpi_get_madt_revision () >=7 ) && (flags & mask); +} +#endif +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2)
drivers/irqchip/irq-gic-v3-its.c +4 -0
@@ -5574,6 +5574,10 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto node_err; } +if (gic_acpi_non_coherent_flag (its_entry->flags, +ACPI_MADT_ITS_NON_COHERENT )) + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; + err =its_probe_one (its);if (!err)return 0 ;
drivers/irqchip/irq-gic-v3.c +9 -0
@@ -2364,6 +2364,11 @@ gic_acpi_parse_madt_redist(union acpi_subtable_headers *header,pr_err ("Couldn't map GICR region @%llx\n" , redist->base_address);return -ENOMEM ; } + +if (gic_acpi_non_coherent_flag (redist->flags, +ACPI_MADT_GICR_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_request_region (redist->base_address, redist->length,"GICR" );gic_acpi_register_redist (redist->base_address, redist_base);
@@ -2389,6 +2394,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_acpi_register_redist (gicc->gicr_base_address, redist_base);return 0 ; }
include/linux/acpi.h +3 -0
@@ -298,6 +298,9 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id)return phys_id ==PHYS_CPUID_INVALID ; } + +u8 __init acpi_get_madt_revision (void ); +/* Validate the processor object's proc_id */ bool acpi_duplicate_processor_id (int proc_id);/* Processor _CTS control */
Signature · 1 line
2.34.1
Re: [PATCH v3 5/5] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 44 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (
acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Marc Zyngier <[email protected]>
Quoted 7 lines
--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)
Hi Marc,
just a quick note to ask if, from an ACPI binding POW this patch and related approach make sense to you.
If so, we can start the process to get the ACPI changes drafted in:
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
and deployed in this patch into the ACPI specs, I can log an "ACK" in the tianocoreBZ entry above (we will be able to rework the code as much as we want, this is just for the bindings).
Thanks,
Lorenzo
Quoted 118 lines
drivers/acpi/processor_core.c +21
-0@@ -215,6 +215,27 @@ phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id)return rv; } +u8 __init acpi_get_madt_revision (void ) +{ +static u8 madt_revision__initdata ; +static bool madt_read__initdata ; +struct acpi_table_header *madt =NULL ; + +if (!madt_read) { + madt_read = true; + +acpi_get_table (ACPI_SIG_MADT ,0 , &madt); +if (!madt) +return madt_revision; + + madt_revision = madt->revision; + +acpi_put_table (madt); + } + +return madt_revision; +} +static phys_cpuid_t map_mat_entry (acpi_handle handle,int type,u32 acpi_id) {struct acpi_buffer buffer = {ACPI_ALLOCATE_BUFFER ,NULL };drivers/irqchip/irq-gic-common.h +8
-0@@ -6,6 +6,7 @@#ifndef _IRQ_GIC_COMMON_H#define _IRQ_GIC_COMMON_H +#include <linux/acpi.h> #include <linux/of.h> #include <linux/irqdomain.h> #include <linux/irqchip/arm-gic-common.h> @@ -29,6 +30,13 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#ifdef CONFIG_ACPI +static inline bool gic_acpi_non_coherent_flag (u32 flags,u32 mask) +{ +return (acpi_get_madt_revision () >=7 ) && (flags & mask); +} +#endif +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2)drivers/irqchip/irq-gic-v3-its.c +4
-0@@ -5574,6 +5574,10 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto node_err; } +if (gic_acpi_non_coherent_flag (its_entry->flags, +ACPI_MADT_ITS_NON_COHERENT )) + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; + err =its_probe_one (its);if (!err)return 0 ;drivers/irqchip/irq-gic-v3.c +9
-0@@ -2364,6 +2364,11 @@ gic_acpi_parse_madt_redist(union acpi_subtable_headers *header,pr_err ("Couldn't map GICR region @%llx\n" , redist->base_address);return -ENOMEM ; } + +if (gic_acpi_non_coherent_flag (redist->flags, +ACPI_MADT_GICR_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_request_region (redist->base_address, redist->length,"GICR" );gic_acpi_register_redist (redist->base_address, redist_base);@@ -2389,6 +2394,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_acpi_register_redist (gicc->gicr_base_address, redist_base);return 0 ; }include/linux/acpi.h +3
-0@@ -298,6 +298,9 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id)return phys_id ==PHYS_CPUID_INVALID ; } + +u8 __init acpi_get_madt_revision (void ); +/* Validate the processor object's proc_id */ bool acpi_duplicate_processor_id (int proc_id);/* Processor _CTS control */ Signature · 1 line
2.34.1
Marc Zyngier
maintainerRe: [PATCH v3 5/5] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 54 lines · Lorenzo Pieralisi
Quoted 44 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (
acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Marc Zyngier <[email protected]>Quoted 7 lines
--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)
Quoted 3 lines
Hi Marc,
just a quick note to ask if, from an ACPI binding POW
I guess you mean POV. POW has an entirely different meaning... :-/
Quoted 11 lines
this patch and related approach make sense to you.
If so, we can start the process to get the ACPI changes drafted in:
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
and deployed in this patch into the ACPI specs, I can log an "ACK" in the tianocoreBZ entry above (we will be able to rework the code as much as we want, this is just for the bindings).
I'm OK with the overall shape of it. However, I wonder what the rationale is for spreading the redistributor property all over the map (in both GICC and GICR structures), while it could be set once and for all in the core MADT flags (32 bits, of which only one is in use).
It is bad enough that there are two ways of getting the GICR regions. Why can't the properties that apply to all of the be common?
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 5/5] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 74 lines · Marc Zyngier
Quoted 54 lines · Lorenzo Pieralisi
Quoted 44 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (
acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Marc Zyngier <[email protected]>Quoted 7 lines
--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)Quoted 3 lines
Hi Marc,
just a quick note to ask if, from an ACPI binding POW
I guess you mean POV. POW has an entirely different meaning... :-/
Quoted 11 lines
this patch and related approach make sense to you.
If so, we can start the process to get the ACPI changes drafted in:
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
and deployed in this patch into the ACPI specs, I can log an "ACK" in the tianocoreBZ entry above (we will be able to rework the code as much as we want, this is just for the bindings).
Quoted 7 lines
I'm OK with the overall shape of it. However, I wonder what the rationale is for spreading the redistributor property all over the map (in both GICC and GICR structures), while it could be set once and for all in the core MADT flags (32 bits, of which only one is in use).
It is bad enough that there are two ways of getting the GICR regions. Why can't the properties that apply to all of the be common?
I don't think we are allowed to add arch specific flags to the MADT since those, supposedly, are cross-architecture (and the only one defined is quite old, though x86 specific).
The reason behind spreading the property is the nature of GICC/GICR subtables themselves - we wanted to apply flags only in subtables relevant to the components in question.
We could try to add a global flag to the MADT but I would not be surprised if the ECR would be rejected then for the reason I explained above.
Thanks,
Lorenzo
Marc Zyngier
maintainerRe: [PATCH v3 5/5] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 81 lines · Lorenzo Pieralisi
Quoted 79 lines · Marc Zyngier
Quoted 54 lines · Lorenzo Pieralisi
Quoted 44 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (
acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Marc Zyngier <[email protected]>Quoted 7 lines
--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)Quoted 3 lines
Hi Marc,
just a quick note to ask if, from an ACPI binding POW
I guess you mean POV. POW has an entirely different meaning... :-/
Quoted 11 lines
this patch and related approach make sense to you.
If so, we can start the process to get the ACPI changes drafted in:
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
and deployed in this patch into the ACPI specs, I can log an "ACK" in the tianocoreBZ entry above (we will be able to rework the code as much as we want, this is just for the bindings).
I'm OK with the overall shape of it. However, I wonder what the rationale is for spreading the redistributor property all over the map (in both GICC and GICR structures), while it could be set once and for all in the core MADT flags (32 bits, of which only one is in use).
Quoted 6 lines
Quoted 2 lines
It is bad enough that there are two ways of getting the GICR regions. Why can't the properties that apply to all of the be common?
I don't think we are allowed to add arch specific flags to the MADT since those, supposedly, are cross-architecture (and the only one defined is quite old, though x86 specific).
There is nothing that is truly cross-arch in this table. *everything* in MADT is arch-specific.
Quoted 7 lines
The reason behind spreading the property is the nature of GICC/GICR subtables themselves - we wanted to apply flags only in subtables relevant to the components in question.
We could try to add a global flag to the MADT but I would not be surprised if the ECR would be rejected then for the reason I explained above.
I don't think that's much of a reason, but I really don't care enough about this to argue otherwise.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
v4 cover irqchip/gic-v3: Enable non-coherent GIC designs probing
In reply to Lorenzo PieralisiThis series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]
v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3
v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approval
v1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructure
Original cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.
Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.
When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)
Signature · 1 line
2.34.1
PATCH v4 1/3 ACPICA: MADT: Add GICC online capable bit handling
ACPICA commit 16f0befdeddf25756f317907798192bbaa417e5e
Implement code to handle the GICC online capable bit management added into ACPI v6.5.
--- include/acpi/actbl2.h | 1 + 1 file changed, 1 insertion(+)
include/acpi/actbl2.h +1 -0
@@ -1046,6 +1046,7 @@ struct acpi_madt_generic_interrupt {/* ACPI_MADT_ENABLED (1) Processor is usable if set */ #define ACPI_MADT_PERFORMANCE_IRQ_MODE (1<<1)/* 01: Performance Interrupt Mode */ #define ACPI_MADT_VGIC_IRQ_MODE (1<<2)/* 02: VGIC Maintenance Interrupt mode */ +#define ACPI_MADT_GICC_ONLINE_CAPABLE (1<<3)/* 03: Processor is online capable */ /* 12: Generic Distributor (ACPI 5.0 + ACPI 6.0 changes) */
Signature · 1 line
2.34.1
Re: [PATCH v4 1/3] ACPICA: MADT: Add GICC online capable bit handling
Quoted 19 lines · Lorenzo Pieralisi
ACPICA commit
16f0befdeddf25756f317907798192bbaa417e5eImplement code to handle the GICC online capable bit management added into ACPI v6.5.
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robert Moore <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>--- include/acpi/actbl2.h | 1 + 1 file changed, 1 insertion(+)include/acpi/actbl2.h +0
-0
include/acpi/actbl2.h struct acpi_madt_generic_interrupt {
Quoted 8 lines
include/acpi/actbl2.h +1
-1@@ -1046,6 +1046,7 @@ struct acpi_madt_generic_interrupt {/* ACPI_MADT_ENABLED (1) Processor is usable if set */ #define ACPI_MADT_PERFORMANCE_IRQ_MODE (1<<1)/* 01: Performance Interrupt Mode */ #define ACPI_MADT_VGIC_IRQ_MODE (1<<2)/* 02: VGIC Maintenance Interrupt mode */ +#define ACPI_MADT_GICC_ONLINE_CAPABLE (1<<3)/* 03: Processor is online capable */ /* 12: Generic Distributor (ACPI 5.0 + ACPI 6.0 changes) */ --
Applied as 6.8-rc1 material along with the [2/3].
Thanks!
PATCH v4 2/3 ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling
In reply to Lorenzo PieralisiACPICA commit c5d2010744b1bf7efba0bd04a8a9c200ef8fb610
Add new flags and related fields to the MADT GICC/GICR/ITS structures according to the code first ECR:
https://bugzilla.tianocore.org/show_bug.cgi?id=4557
Update the MADT template to the latest MADT revision.
--- include/acpi/actbl2.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
include/acpi/actbl2.h +9 -2
@@ -1047,6 +1047,7 @@ struct acpi_madt_generic_interrupt {#define ACPI_MADT_PERFORMANCE_IRQ_MODE (1<<1)/* 01: Performance Interrupt Mode */ #define ACPI_MADT_VGIC_IRQ_MODE (1<<2)/* 02: VGIC Maintenance Interrupt mode */ #define ACPI_MADT_GICC_ONLINE_CAPABLE (1<<3)/* 03: Processor is online capable */ +#define ACPI_MADT_GICC_NON_COHERENT (1<<4)/* 04: GIC redistributor is not coherent */ /* 12: Generic Distributor (ACPI 5.0 + ACPI 6.0 changes) */
@@ -1091,21 +1092,27 @@ struct acpi_madt_generic_msi_frame {struct acpi_madt_generic_redistributor {struct acpi_subtable_header header; -u16 reserved;/* reserved - must be zero */ +u8 flags; +u8 reserved;/* reserved - must be zero */ u64 base_address;u32 length; }; +#define ACPI_MADT_GICR_NON_COHERENT (1) +/* 15: Generic Translator (ACPI 6.0) */ struct acpi_madt_generic_translator {struct acpi_subtable_header header; -u16 reserved;/* reserved - must be zero */ +u8 flags; +u8 reserved;/* reserved - must be zero */ u32 translation_id;u64 base_address;u32 reserved2; }; +#define ACPI_MADT_ITS_NON_COHERENT (1) +/* 16: Multiprocessor wakeup (ACPI 6.4) */ struct acpi_madt_multiproc_wakeup {
Signature · 1 line
2.34.1
PATCH v4 3/3 irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
In reply to Lorenzo PieralisiThe GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).
--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)
drivers/acpi/processor_core.c +21 -0
@@ -215,6 +215,27 @@ phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id)return rv; } +u8 __init acpi_get_madt_revision (void ) +{ +static u8 madt_revision__initdata ; +static bool madt_read__initdata ; +struct acpi_table_header *madt =NULL ; + +if (!madt_read) { + madt_read = true; + +acpi_get_table (ACPI_SIG_MADT ,0 , &madt); +if (!madt) +return madt_revision; + + madt_revision = madt->revision; + +acpi_put_table (madt); + } + +return madt_revision; +} +static phys_cpuid_t map_mat_entry (acpi_handle handle,int type,u32 acpi_id) {struct acpi_buffer buffer = {ACPI_ALLOCATE_BUFFER ,NULL };
drivers/irqchip/irq-gic-common.h +8 -0
@@ -6,6 +6,7 @@#ifndef _IRQ_GIC_COMMON_H#define _IRQ_GIC_COMMON_H +#include <linux/acpi.h> #include <linux/of.h> #include <linux/irqdomain.h> #include <linux/irqchip/arm-gic-common.h>
@@ -29,6 +30,13 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#ifdef CONFIG_ACPI +static inline bool gic_acpi_non_coherent_flag (u32 flags,u32 mask) +{ +return (acpi_get_madt_revision () >=7 ) && (flags & mask); +} +#endif +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2)
drivers/irqchip/irq-gic-v3-its.c +4 -0
@@ -5578,6 +5578,10 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto node_err; } +if (gic_acpi_non_coherent_flag (its_entry->flags, +ACPI_MADT_ITS_NON_COHERENT )) + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; + err =its_probe_one (its);if (!err)return 0 ;
drivers/irqchip/irq-gic-v3.c +9 -0
@@ -2356,6 +2356,11 @@ gic_acpi_parse_madt_redist(union acpi_subtable_headers *header,pr_err ("Couldn't map GICR region @%llx\n" , redist->base_address);return -ENOMEM ; } + +if (gic_acpi_non_coherent_flag (redist->flags, +ACPI_MADT_GICR_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_request_region (redist->base_address, redist->length,"GICR" );gic_acpi_register_redist (redist->base_address, redist_base);
@@ -2380,6 +2385,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_acpi_register_redist (gicc->gicr_base_address, redist_base);return 0 ; }
include/linux/acpi.h +3 -0
@@ -283,6 +283,9 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id)return phys_id ==PHYS_CPUID_INVALID ; } + +u8 __init acpi_get_madt_revision (void ); +/* Validate the processor object's proc_id */ bool acpi_duplicate_processor_id (int proc_id);/* Processor _CTS control */
Signature · 1 line
2.34.1
Marc Zyngier
maintainerRe: [PATCH v4 3/3] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 9 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers
IWB/OWB *barriers*? Unless you're talking about something else, IWB/OWB refers to cacheability, and only that.
Quoted 50 lines
but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (
acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Marc Zyngier <[email protected]>--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)drivers/acpi/processor_core.c +0
-0@@ -215,6 +215,27 @@ phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id)return rv;
drivers/acpi/processor_core.c phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id) · line 225
Quoted 10 lines
drivers/acpi/processor_core.c +8
-0} +u8 __init acpi_get_madt_revision (void ) +{ +static u8 madt_revision__initdata ; +static bool madt_read__initdata ; +struct acpi_table_header *madt =NULL ; + +if (!madt_read) { + madt_read = true;
Huh. Why do we need this hack? What's the issue with accessing the MADT? Can it disappear from under our feet? While we're walking it?
drivers/acpi/processor_core.c phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id) · line 229
Quoted 4 lines
drivers/acpi/processor_core.c +4
-0+ +acpi_get_table (ACPI_SIG_MADT ,0 , &madt); +if (!madt) +return madt_revision;
What does this mean? Can we have a revision 0 of MADT?
Quoted 25 lines
drivers/acpi/processor_core.c +9
-0+ + madt_revision = madt->revision; + +acpi_put_table (madt); + } + +return madt_revision; +} +static phys_cpuid_t map_mat_entry (acpi_handle handle,int type,u32 acpi_id) {struct acpi_buffer buffer = {ACPI_ALLOCATE_BUFFER ,NULL };drivers/irqchip/irq-gic-common.h +1
-0@@ -6,6 +6,7 @@#ifndef _IRQ_GIC_COMMON_H#define _IRQ_GIC_COMMON_H +#include <linux/acpi.h> #include <linux/of.h> #include <linux/irqdomain.h> #include <linux/irqchip/arm-gic-common.h>
drivers/irqchip/irq-gic-common.h void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks, · line 37
Quoted 8 lines
drivers/irqchip/irq-gic-common.h +5
-0@@ -29,6 +30,13 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#ifdef CONFIG_ACPI +static inline bool gic_acpi_non_coherent_flag (u32 flags,u32 mask) +{ +return (acpi_get_madt_revision () >=7 ) && (flags & mask); +}
Given that this checks *any* flag (or a combination of flags), the name of the helper is extremely misleading. Also, GICC flags are not necessarily tied to revision 7 of MADT.
To be honest, I don't think this helper bring much, and I'd rather see an explicit check (or 3) for the revision in the driver code.
Quoted 52 lines
drivers/irqchip/irq-gic-common.h +2
-0+#endif +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2)drivers/irqchip/irq-gic-v3-its.c +4
-0@@ -5578,6 +5578,10 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto node_err; } +if (gic_acpi_non_coherent_flag (its_entry->flags, +ACPI_MADT_ITS_NON_COHERENT )) + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; + err =its_probe_one (its);if (!err)return 0 ;drivers/irqchip/irq-gic-v3.c +9
-0@@ -2356,6 +2356,11 @@ gic_acpi_parse_madt_redist(union acpi_subtable_headers *header,pr_err ("Couldn't map GICR region @%llx\n" , redist->base_address);return -ENOMEM ; } + +if (gic_acpi_non_coherent_flag (redist->flags, +ACPI_MADT_GICR_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_request_region (redist->base_address, redist->length,"GICR" );gic_acpi_register_redist (redist->base_address, redist_base);@@ -2380,6 +2385,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_acpi_register_redist (gicc->gicr_base_address, redist_base);return 0 ; }include/linux/acpi.h +0
-0
include/linux/acpi.h static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id) · line 291
Quoted 9 lines
include/linux/acpi.h +3
-0@@ -283,6 +283,9 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id)return phys_id ==PHYS_CPUID_INVALID ; } + +u8 __init acpi_get_madt_revision (void ); +/* Validate the processor object's proc_id */ bool acpi_duplicate_processor_id (int proc_id);/* Processor _CTS control */
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v4 3/3] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 9 lines · Marc Zyngier
Quoted 7 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Quoted 5 lines
Quoted 2 lines
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers
IWB/OWB *barriers*? Unless you're talking about something else, IWB/OWB refers to cacheability, and only that.
Yes, it should be expressed differently. Unfortunately this sentence made it into the kernel with the DT counterpart - commit 3a0fff0fb6a3 log, apologies.
Quoted 52 lines
Quoted 52 lines
but it is not mandatory for designs to wire up the corresponding interconnect signals that control the cacheability/shareability of transactions.
Redistributors and ITSes interconnect ports can be connected to non-coherent interconnects that are not able to manage the shareability/cacheability attributes; this implicitly makes the redistributors and ITSes non-coherent observers.
So far, the GIC driver on probe executes a write to "probe" for the redistributors and ITSes registers shareability bitfields by writing a value (ie InnerShareable - the shareability domain the CPUs are in) and check it back to detect whether the value sticks or not; this hinges on a GIC programming model behaviour that predates the current specifications, that just define shareability bits as writeable but do not guarantee that writing certain shareability values enable the expected behaviour for the redistributors/ITSes memory interconnect ports.
To enable non-coherent GIC designs on ACPI based systems, parse the MADT GICC/GICR/ITS subtables non-coherent flags to determine whether the respective components are non-coherent observers and force the shareability attributes to be programmed into the redistributors and ITSes registers.
An ACPI global function (
acpi_get_madt_revision()) is added to retrieve the MADT revision, in that it is essential to check the MADT revision before checking for flags that were added with MADT revision 7 so that if the kernel is booted with ACPI tables (MADT rev < 7) it skips parsing the newly added flags (that should be zeroed reserved values for MADT versions < 7 but they could turn out to be buggy and should be ignored).Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Robin Murphy <[email protected]>Cc Mark Rutland <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Marc Zyngier <[email protected]>--- drivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/linux/acpi.h | 3 +++ 5 files changed, 45 insertions(+)drivers/acpi/processor_core.c +0
-0@@ -215,6 +215,27 @@ phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id)return rv; }
Quoted 11 lines
drivers/acpi/processor_core.c phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id) · line 225
Quoted 8 lines
drivers/acpi/processor_core.c +8
-0+u8 __init acpi_get_madt_revision (void ) +{ +static u8 madt_revision__initdata ; +static bool madt_read__initdata ; +struct acpi_table_header *madt =NULL ; + +if (!madt_read) { + madt_read = true;Huh. Why do we need this hack? What's the issue with accessing the MADT? Can it disappear from under our feet? While we're walking it?
It is an awkward attempt at stashing the revision instead of calling acpi_get_table() repeatedly (and from multiple files for the same reason - ie get an MADT rev number).
Side note: get_madt_table() does the same thing and I followed it - I am not sure it is very helpful either (or maybe there is something I don't know behind that reasoning).
I will remove it (or leave it there without the madt_read hack).
Quoted 6 lines
drivers/acpi/processor_core.c phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id) · line 229
Quoted 4 lines
drivers/acpi/processor_core.c +4
-0+ +acpi_get_table (ACPI_SIG_MADT ,0 , &madt); +if (!madt) +return madt_revision;What does this mean? Can we have a revision 0 of MADT?
No obviously you are right, it should have just returned an error code and that's what it wanted to signal.
Quoted 38 lines
Quoted 25 lines
drivers/acpi/processor_core.c +9
-0+ + madt_revision = madt->revision; + +acpi_put_table (madt); + } + +return madt_revision; +} +static phys_cpuid_t map_mat_entry (acpi_handle handle,int type,u32 acpi_id) {struct acpi_buffer buffer = {ACPI_ALLOCATE_BUFFER ,NULL };drivers/irqchip/irq-gic-common.h +1
-0@@ -6,6 +6,7 @@#ifndef _IRQ_GIC_COMMON_H#define _IRQ_GIC_COMMON_H +#include <linux/acpi.h> #include <linux/of.h> #include <linux/irqdomain.h> #include <linux/irqchip/arm-gic-common.h> drivers/irqchip/irq-gic-common.h void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks, · line 37
Quoted 8 lines
drivers/irqchip/irq-gic-common.h +5
-0@@ -29,6 +30,13 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,void gic_enable_of_quirks (const struct device_node *np,const struct gic_quirk *quirks,void *data); +#ifdef CONFIG_ACPI +static inline bool gic_acpi_non_coherent_flag (u32 flags,u32 mask) +{ +return (acpi_get_madt_revision () >=7 ) && (flags & mask); +}Given that this checks *any* flag (or a combination of flags), the name of the helper is extremely misleading. Also, GICC flags are not necessarily tied to revision 7 of MADT.
Quoted 2 lines
To be honest, I don't think this helper bring much, and I'd rather see an explicit check (or 3) for the revision in the driver code.
I will do, you have a point.
Thanks,
Lorenzo
Quoted 68 lines
Quoted 52 lines
drivers/irqchip/irq-gic-common.h +2
-0+#endif +#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2)drivers/irqchip/irq-gic-v3-its.c +4
-0@@ -5578,6 +5578,10 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header, goto node_err; } +if (gic_acpi_non_coherent_flag (its_entry->flags, +ACPI_MADT_ITS_NON_COHERENT )) + its->flags |=ITS_FLAGS_FORCE_NON_SHAREABLE ; + err =its_probe_one (its);if (!err)return 0 ;drivers/irqchip/irq-gic-v3.c +9
-0@@ -2356,6 +2356,11 @@ gic_acpi_parse_madt_redist(union acpi_subtable_headers *header,pr_err ("Couldn't map GICR region @%llx\n" , redist->base_address);return -ENOMEM ; } + +if (gic_acpi_non_coherent_flag (redist->flags, +ACPI_MADT_GICR_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_request_region (redist->base_address, redist->length,"GICR" );gic_acpi_register_redist (redist->base_address, redist_base);@@ -2380,6 +2385,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +gic_acpi_register_redist (gicc->gicr_base_address, redist_base);return 0 ; }include/linux/acpi.h +0
-0include/linux/acpi.h static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id) · line 291
Quoted 9 lines
include/linux/acpi.h +3
-0@@ -283,6 +283,9 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id)return phys_id ==PHYS_CPUID_INVALID ; } + +u8 __init acpi_get_madt_revision (void ); +/* Validate the processor object's proc_id */ bool acpi_duplicate_processor_id (int proc_id);/* Processor _CTS control */ Thanks,
M.Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerRe: [PATCH v4 3/3] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 14 lines · Lorenzo Pieralisi
Quoted 12 lines · Marc Zyngier
Quoted 9 lines · Lorenzo Pieralisi
The GIC architecture specification defines a set of registers for redistributors and ITSes that control the sharebility and cacheability attributes of redistributors/ITSes initiator ports on the interconnect (GICR_[V]PROPBASER, GICR_[V]PENDBASER, GITS_BASER<n>).
Architecturally the GIC provides a means to drive shareability and cacheability attributes signals and related IWB/OWB/ISH barriers
Quoted 6 lines
Quoted 2 lines
IWB/OWB *barriers*? Unless you're talking about something else, IWB/OWB refers to cacheability, and only that.
Yes, it should be expressed differently. Unfortunately this sentence made it into the kernel with the DT counterpart - commit
3a0fff0fb6a3log, apologies.
Oh well. At least please clean this one up when you repost.
[...]
Quoted 6 lines
Quoted 5 lines
drivers/acpi/processor_core.c phys_cpuid_t __init acpi_map_madt_entry(u32 acpi_id) · line 225
Quoted 2 lines
drivers/acpi/processor_core.c +2
-0+if (!madt_read) { + madt_read = true;Huh. Why do we need this hack? What's the issue with accessing the MADT? Can it disappear from under our feet? While we're walking it?
Quoted 7 lines
It is an awkward attempt at stashing the revision instead of calling
acpi_get_table()repeatedly (and from multiple files for the same reason - ie get an MADT rev number).Side note:
get_madt_table()does the same thing and I followed it - I am not sure it is very helpful either (or maybe there is something I don't know behind that reasoning).
This was introduced as part of 149fe9c293f76, as a cleanup. Not a great move IMHO.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v4 3/3] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
In reply to Lorenzo PieralisiOn Wed, Dec 27, 2023 at 12:00:38PM +0100, Lorenzo Pieralisi wrote:
[...]
drivers/irqchip/irq-gic-v3.c gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header, · line 2391
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c +4
-0@@ -2380,6 +2385,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +
Quick question before reposting it. We run this function for every GICC entry, I didn't add a check to make sure all GICC entries have the same flag value, please let me know if that's OK.
I don't think there is a point in keeping a live variable across calls to set the flag once for all either.
Thanks,
Lorenzo
Quoted 20 lines
drivers/irqchip/irq-gic-v3.c +0
-0gic_acpi_register_redist (gicc->gicr_base_address, redist_base);return 0 ; }include/linux/acpi.h +3
-0@@ -283,6 +283,9 @@ static inline bool invalid_phys_cpuid(phys_cpuid_t phys_id)return phys_id ==PHYS_CPUID_INVALID ; } + +u8 __init acpi_get_madt_revision (void ); +/* Validate the processor object's proc_id */ bool acpi_duplicate_processor_id (int proc_id);/* Processor _CTS control */ Signature · 1 line
2.34.1
Marc Zyngier
maintainerRe: [PATCH v4 3/3] irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probing
Quoted 19 lines · Lorenzo Pieralisi
On Wed, Dec 27, 2023 at 12:00:38PM +0100, Lorenzo Pieralisi wrote:
[...]
drivers/irqchip/irq-gic-v3.c gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header, · line 2391
Quoted 8 lines
drivers/irqchip/irq-gic-v3.c +4
-0@@ -2380,6 +2385,10 @@ gic_acpi_parse_madt_gicc(union acpi_subtable_headers *header,return -ENOMEM ;gic_request_region (gicc->gicr_base_address, size,"GICR" ); +if (gic_acpi_non_coherent_flag (gicc->flags, +ACPI_MADT_GICC_NON_COHERENT )) + gic_data.rdists.flags |=RDIST_FLAGS_FORCE_NON_SHAREABLE ; +Quick question before reposting it. We run this function for every GICC entry, I didn't add a check to make sure all GICC entries have the same flag value, please let me know if that's OK.
Quoted 2 lines
I don't think there is a point in keeping a live variable across calls to set the flag once for all either.
I don't think that's useful. Once we see the flag being set, we'll enforce the non-coherency. If it wasn't set before, it's because it wasn't necessary.
If one day we find a firmware that only randomly exposes the flag, we'll treat it as a quirk.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 64 lines · Lorenzo Pieralisi
This series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approvalv1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructureOriginal cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probingdrivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)
Quoted 1 line
--
I can apply the first 2 patches, but I would need an ACK for the 3rd one.
Alternatively, feel free to add
to the first 2 patches and route them via ARM64.
Thanks!
Marc Zyngier
maintainerQuoted 69 lines · Rafael J. Wysocki
Quoted 64 lines · Lorenzo Pieralisi
This series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approvalv1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructureOriginal cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probingdrivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)Quoted 1 line
--
Quoted 7 lines
I can apply the first 2 patches, but I would need an ACK for the 3rd one.
Alternatively, feel free to add
to the first 2 patches and route them via ARM64.
Thanks for that. I have some comments on the third patch, which I'd like to see addressed beforehand. This is probably all 6.9 material anyway (nobody is affected by this so far).
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 75 lines · Marc Zyngier
Quoted 73 lines · Rafael J. Wysocki
Quoted 64 lines · Lorenzo Pieralisi
This series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approvalv1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructureOriginal cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probingdrivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)Quoted 1 line
--
I can apply the first 2 patches, but I would need an ACK for the 3rd one.
Alternatively, feel free to add
Quoted 7 lines
Quoted 3 lines
to the first 2 patches and route them via ARM64.
Thanks for that. I have some comments on the third patch, which I'd like to see addressed beforehand. This is probably all 6.9 material anyway (nobody is affected by this so far).
From a purely selfish point of view, it would be useful to have the first patch merged merely to reduce the burden of patches for vcpu hotplug.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
Rafael J. Wysocki
maintainerQuoted 81 lines · Russell King (Oracle)
Quoted 79 lines · Marc Zyngier
Quoted 76 lines · Rafael J. Wysocki
Quoted 64 lines · Lorenzo Pieralisi
This series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approvalv1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructureOriginal cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probingdrivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)Quoted 1 line
--
I can apply the first 2 patches, but I would need an ACK for the 3rd one.
Alternatively, feel free to add
to the first 2 patches and route them via ARM64.
Quoted 7 lines
Quoted 3 lines
Thanks for that. I have some comments on the third patch, which I'd like to see addressed beforehand. This is probably all 6.9 material anyway (nobody is affected by this so far).
From a purely selfish point of view, it would be useful to have the first patch merged merely to reduce the burden of patches for vcpu hotplug.
OK, since there will be at least one more iteration of patch [3/3] AFAICS, I'll queue up patches [1-2/3] for 6.8 (next week, though).
Quoted 87 lines · Rafael J. Wysocki
Quoted 85 lines · Russell King (Oracle)
Quoted 82 lines · Marc Zyngier
Quoted 69 lines · Rafael J. Wysocki
Quoted 64 lines · Lorenzo Pieralisi
This series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approvalv1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructureOriginal cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probingdrivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)Quoted 1 line
--
Quoted 7 lines
I can apply the first 2 patches, but I would need an ACK for the 3rd one.
Alternatively, feel free to add
to the first 2 patches and route them via ARM64.
Thanks for that. I have some comments on the third patch, which I'd like to see addressed beforehand. This is probably all 6.9 material anyway (nobody is affected by this so far).
Quoted 6 lines
Quoted 3 lines
From a purely selfish point of view, it would be useful to have the first patch merged merely to reduce the burden of patches for vcpu hotplug.
OK, since there will be at least one more iteration of patch [3/3] AFAICS, I'll queue up patches [1-2/3] for 6.8 (next week, though).
Thanks!
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
Quoted 87 lines · Rafael J. Wysocki
Quoted 85 lines · Russell King (Oracle)
Quoted 82 lines · Marc Zyngier
Quoted 69 lines · Rafael J. Wysocki
Quoted 64 lines · Lorenzo Pieralisi
This series is v4 of previous series:
v3: https://lore.kernel.org/all/[email protected]
v2: https://lore.kernel.org/all/[email protected]
v1: https://lore.kernel.org/all/[email protected]v3 -> v4:
- Dropped patches [1-3], already merged
- Added Linuxized ACPICA changes accepted upstream
- Rebased against v6.7-rc3v2 -> v3:
- Added ACPICA temporary changes and ACPI changes to implement ECR https://bugzilla.tianocore.org/show_bug.cgi?id=4557
- ACPI changes are for testing purposes - subject to ECR code first approvalv1 -> v2:
- Updated DT bindings as per feedback
- Updated patch[2] to use GIC quirks infrastructureOriginal cover letter
---
The GICv3 architecture specifications provide a means for the system programmer to set the shareability and cacheability attributes the GIC components (redistributors and ITSes) use to drive memory transactions.Albeit the architecture give control over shareability/cacheability memory transactions attributes (and barriers), it is allowed to connect the GIC interconnect ports to non-coherent memory ports on the interconnect, basically tying off shareability/cacheability "wires" and de-facto making the redistributors and ITSes non-coherent memory observers.
This series aims at starting a discussion over a possible solution to this problem, by adding to the GIC device tree bindings the standard dma-noncoherent property. The GIC driver uses the property to force the redistributors and ITSes shareability attributes to non-shareable, which consequently forces the driver to use CMOs on GIC memory tables.
On ARM DT DMA is default non-coherent, so the GIC driver can't rely on the generic DT dma-coherent/non-coherent property management layer (
of_dma_is_coherent()) which would default all GIC designs in the field as non-coherent; it has to rely on ad-hoc dma-noncoherent property handling.When a consistent approach is agreed upon for DT an equivalent binding will be put forward for ACPI based systems.
Lorenzo Pieralisi (3):
ACPICA: MADT: Add GICC online capable bit handling
ACPICA: MADT: Add new MADT GICC/GICR/ITS non-coherent flags handling irqchip/gic-v3: Enable non-coherent redistributors/ITSes ACPI probingdrivers/acpi/processor_core.c | 21 +++++++++++++++++++++ drivers/irqchip/irq-gic-common.h | 8 ++++++++ drivers/irqchip/irq-gic-v3-its.c | 4 ++++ drivers/irqchip/irq-gic-v3.c | 9 +++++++++ include/acpi/actbl2.h | 12 ++++++++++-- include/linux/acpi.h | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)Quoted 1 line
--
Quoted 7 lines
I can apply the first 2 patches, but I would need an ACK for the 3rd one.
Alternatively, feel free to add
to the first 2 patches and route them via ARM64.
Thanks for that. I have some comments on the third patch, which I'd like to see addressed beforehand. This is probably all 6.9 material anyway (nobody is affected by this so far).
Quoted 6 lines
Quoted 3 lines
From a purely selfish point of view, it would be useful to have the first patch merged merely to reduce the burden of patches for vcpu hotplug.
OK, since there will be at least one more iteration of patch [3/3] AFAICS, I'll queue up patches [1-2/3] for 6.8 (next week, though).
Thank you Rafael.
Lorenzo