irqchip/gic-v3: wait irq done to set affinity
Subsystems (2)
RFC PATCH 1/1 irqchip/gic-v3: wait irq done to set affinity
Recently we have some problem about gic set affinity in our test.
This patch just aim to make some discuss about this problem.
For now, the implementation of gic set affinity going to take effects immediately, and without check if any irq are being processed.
So, This leads to some problem, think about this scenario:
1. First, we have an irq was generated by an device.
2. In the processing of this irq(after handle event, before clear IRQD_IRQ_INPROGRESS flag), we modify the route and the gic takes effect immediately,at the same time the new one was generated again.
3. The new irq will be processing in other cpu which different form the old one.
4. The new irq going to be discarded because of the flag IRQD_IRQ_INPROGRESS has been set.
I notice that if we set IRQF_ONESHOT when register the irq, this problem will gone.
But I'm also thinking about change the gic_set_affinity function, to wait current irq done on all cpus before gic_write_irouter.
I'm not sure if that's appropriate.
Is the best workaround to use IRQF_ONESHOT to prevent reentrancy?
Please let me know, if have any other suggestions on this issue.
--- drivers/irqchip/irq-gic-v3.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/irqchip/irq-gic-v3.c +2 -0
@@ -1348,6 +1348,8 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, reg =gic_dist_base (d) + offset + (index *8 ); val =gic_mpidr_to_affinity (cpu_logical_map (cpu)); +// wait irq done on all cpus +gic_write_irouter (val, reg);/*
Signature · 1 line
2.17.1
Marc Zyngier
maintainerQuoted 8 lines · Yipeng Zou
Recently we have some problem about gic set affinity in our test.
This patch just aim to make some discuss about this problem.
For now, the implementation of gic set affinity going to take effects immediately, and without check if any irq are being processed.
Quoted 7 lines
So, This leads to some problem, think about this scenario:
1. First, we have an irq was generated by an device.
2. In the processing of this
irq(after handle event, before clear IRQD_IRQ_INPROGRESS flag), we modify the route and the gic takes effect immediately,at the same time the new one was generated again.
How is that possible?
If it is affected by GICD_IROUTERn (as your patch suggests), then it is a SPI. If it is a SPI, it has an active state. Which means it cannot fire again without a deactivation (EOI if EOImode=0, EOI+DIR if EOImode=1) having taken place.
So either something has deactivated the interrupt without masking it beforehand, or the active state is not honoured. Either way, this is wrong.
Quoted 7 lines
3. The new irq will be processing in other cpu which different form the old one.
4. The new irq going to be discarded because of the flag
IRQD_IRQ_INPROGRESShas been set.
Quoted 6 lines
I notice that if we set
IRQF_ONESHOTwhen register the irq, this problem will gone.But I'm also thinking about change the
gic_set_affinityfunction, to wait current irq done on all cpus beforegic_write_irouter.
I'm not sure if that's appropriate.
The base architecture should guarantee that this is not a problem, thanks to the active state. If that was a LPI (which do not have an active state), that'd be a different problem. But this doesn't seem to be the case here.
I'm afraid to say that what you describe seem like a bug of some sort, either HW or SW.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 18 lines · Marc Zyngier
Quoted 7 lines · Yipeng Zou
Recently we have some problem about gic set affinity in our test.
This patch just aim to make some discuss about this problem.
For now, the implementation of gic set affinity going to take effects immediately, and without check if any irq are being processed.
Quoted 7 lines
So, This leads to some problem, think about this scenario:
1. First, we have an irq was generated by an device.
2. In the processing of this
irq(after handle event, before clear IRQD_IRQ_INPROGRESS flag), we modify the route and the gic takes effect immediately,at the same time the new one was generated again.How is that possible?
Quoted 8 lines
If it is affected by
GICD_IROUTERn(as your patch suggests), then it is a SPI. If it is a SPI, it has an active state. Which means it cannot fire again without a deactivation (EOI if EOImode=0, EOI+DIR if EOImode=1) having taken place.So either something has deactivated the interrupt without masking it beforehand, or the active state is not honoured. Either way, this is wrong.
Yes, agree, There is no possible in SPI case.
Quoted 9 lines
Quoted 9 lines
3. The new irq will be processing in other cpu which different form the old one.
4. The new irq going to be discarded because of the flag
IRQD_IRQ_INPROGRESShas been set.I notice that if we set
IRQF_ONESHOTwhen register the irq, this problem will gone.
Quoted 7 lines
Quoted 3 lines
But I'm also thinking about change the
gic_set_affinityfunction, to wait current irq done on all cpus beforegic_write_irouter.
I'm not sure if that's appropriate.The base architecture should guarantee that this is not a problem, thanks to the active state. If that was a LPI (which do not have an active state), that'd be a different problem. But this doesn't seem to be the case here.
Hi , Thanks for reply very much.
I have rechecked our test. Actually, that was a LPI in out test case.
It cause the problem since its_send_movi command.
I made a mistake when i modified the code. It should be as follow. Sorry for misleading you.
drivers/irqchip/irq-gic-v3-its.c +0 -0
index 973ede0197e3..fad08ccb7fd9 100644
drivers/irqchip/irq-gic-v3-its.c +0 -0
@@ -1667,6 +1667,9 @@ static int its_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
/* don't set the affinity when the target cpu is same as current one */ if (cpu != prev_cpu) {
+
+ // wait irq done on all cpus
+
target_col = &its_dev->its->collections[cpu]; its_send_movi(its_dev, target_col, id); its_dev->event_map.col_map[id] = cpu
Quoted 6 lines
I'm afraid to say that what you describe seem like a bug of some sort, either HW or SW.
Thanks,
M.
Signature · 2 lines
Regards, Yipeng Zou
drivers/irqchip/irq-gic-v3-its.c
Quoted 13 lines · Yipeng Zou
在 2023/1/6 19:55, Marc Zyngier 写道:
index973ede0197e3..fad08ccb7fd9100644drivers/irqchip/irq-gic-v3-its.c +0
-0@@ -1667,6 +1667,9 @@ static int its_set_affinity(struct irq_data *d, const struct cpumask *mask_val, /* don't set the affinity when the target cpu is same ascurrent one */
if (cpu !=prev_cpu) {drivers/irqchip/irq-gic-v3-its.c +3
-0+ + // wait irq done on all cpus +
There is no way to wait here. The caller holds the interrupt descriptor lock.
If this is really an issue for LPI, then the only way to deal with that is CONFIG_GENERIC_PENDING_IRQ, which delays the affinity change to interrupt context
Why on earth must all the known hardware mistakes be repeated over and over?
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel