dpaa2-eth: use napi_schedule to be compatible with PREEMPT_RT
PATCH 1/2 dpaa2-eth: use napi_schedule to be compatible with PREEMPT_RT
From: Jiafei Pan <[email protected]>
The driver calls napi_schedule_irqoff() from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.
In the call path of this function, __raise_softirq_irqoff() is modifying its per-CPU mask of pending softirqs that must be processed, using or_softirq_pending(). The or_softirq_pending() function is not atomic, but since interrupts are supposed to be disabled, nobody should be preempting it, and the operation should be safe.
Nonetheless, when running with hardirqs on, as in the PREEMPT_RT case, it isn't safe, and the pending softirqs mask can get corrupted, resulting in softirqs being lost and never processed.
To have common code that works with PREEMPT_RT and with mainline Linux, we can use plain napi_schedule() instead. The difference is that napi_schedule() (via __napi_schedule) also calls local_irq_save, which disables hardirqs if they aren't already. But, since they already are disabled in non-RT, this means that in practice we don't see any measurable difference in throughput or latency with this patch.
--- drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +1 -1
@@ -2330,7 +2330,7 @@ static void cdan_cb(struct dpaa2_io_notification_ctx *ctx)/* Update NAPI statistics */ ch->stats.cdan++; -napi_schedule_irqoff (&ch->napi); +napi_schedule (&ch->napi); }/* Allocate and configure a DPCON object */
Signature · 1 line
2.25.1
PATCH 2/2 enetc: use napi_schedule to be compatible with PREEMPT_RT
From: Jiafei Pan <[email protected]>
The driver calls napi_schedule_irqoff() from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.
In the call path of this function, __raise_softirq_irqoff() is modifying its per-CPU mask of pending softirqs that must be processed, using or_softirq_pending(). The or_softirq_pending() function is not atomic, but since interrupts are supposed to be disabled, nobody should be preempting it, and the operation should be safe.
Nonetheless, when running with hardirqs on, as in the PREEMPT_RT case, it isn't safe, and the pending softirqs mask can get corrupted, resulting in softirqs being lost and never processed.
To have common code that works with PREEMPT_RT and with mainline Linux, we can use plain napi_schedule() instead. The difference is that napi_schedule() (via __napi_schedule) also calls local_irq_save, which disables hardirqs if they aren't already. But, since they already are disabled in non-RT, this means that in practice we don't see any measurable difference in throughput or latency with this patch.
--- drivers/net/ethernet/freescale/enetc/enetc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/freescale/enetc/enetc.c +1 -1
@@ -270,7 +270,7 @@ static irqreturn_t enetc_msix(int irq, void *data)for_each_set_bit (i, &v->tx_rings_map,ENETC_MAX_NUM_TXQS )enetc_wr_reg (v->tbier_base +ENETC_BDR_OFF (i),0 ); -napi_schedule_irqoff (&v->napi); +napi_schedule (&v->napi);return IRQ_HANDLED ; }
Signature · 1 line
2.25.1
Re: [PATCH net-next 2/2] enetc: use napi_schedule to be compatible with PREEMPT_RT
From: Vladimir Oltean <[email protected]>
Date: Mon, 3 Aug 2020 23:10:09 +0300
Quoted 22 lines
From: Jiafei Pan <[email protected]>
The driver calls
napi_schedule_irqoff()from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.In the call path of this function,
__raise_softirq_irqoff()is modifying its per-CPU mask of pending softirqs that must be processed, usingor_softirq_pending(). Theor_softirq_pending()function is not atomic, but since interrupts are supposed to be disabled, nobody should be preempting it, and the operation should be safe.Nonetheless, when running with hardirqs on, as in the
PREEMPT_RTcase, it isn't safe, and the pending softirqs mask can get corrupted, resulting in softirqs being lost and never processed.To have common code that works with
PREEMPT_RTand with mainline Linux, we can use plainnapi_schedule()instead. The difference is thatnapi_schedule()(via__napi_schedule) also callslocal_irq_save, which disables hardirqs if they aren't already. But, since they already are disabled in non-RT, this means that in practice we don't see any measurable difference in throughput or latency with this patch.
Quoted 2 lines
Signed-off-by Jiafei Pan <[email protected]>Signed-off-by Vladimir Oltean <[email protected]>
Applied.
Re: [PATCH net-next 2/2] enetc: use napi_schedule to be compatible with PREEMPT_RT
Quoted 3 lines · David Miller
From: Vladimir Oltean <[email protected]>
Quoted 2 lines
The driver calls
napi_schedule_irqoff()from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.
…
Quoted 5 lines
Quoted 3 lines
Signed-off-by Jiafei Pan <[email protected]>Signed-off-by Vladimir Oltean <[email protected]>Applied.
Could these two patches be forwarded -stable, please? The changelog describes this as a problem on PREEMPT_RT but this also happens on !RT with the `threadirqs' commandline switch.
Sebastian
Re: [PATCH net-next 2/2] enetc: use napi_schedule to be compatible with PREEMPT_RT
Quoted 9 lines · Sebastian Andrzej Siewior
Quoted 3 lines · David Miller
From: Vladimir Oltean <[email protected]>
Quoted 2 lines
The driver calls
napi_schedule_irqoff()from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.…
Quoted 4 lines
Quoted 3 lines
Signed-off-by Jiafei Pan <[email protected]>Signed-off-by Vladimir Oltean <[email protected]>
Quoted 7 lines
Quoted 1 line
Applied.
Could these two patches be forwarded -stable, please? The changelog describes this as a problem on
PREEMPT_RTbut this also happens on !RT with the `threadirqs' commandline switch.Sebastian
I expect the driver maintainers to have something to say about this. I didn't test on stable kernels, and at least for dpaa2-eth, the change would need to go pretty deep down the stable line.
Also, not really sure who is using the threadirqs option except for testing purposes.
Thanks,
-Vladimir
Re: [PATCH net-next 2/2] enetc: use napi_schedule to be compatible with PREEMPT_RT
Quoted 3 lines · Vladimir Oltean
I expect the driver maintainers to have something to say about this. I didn't test on stable kernels, and at least for dpaa2-eth, the change would need to go pretty deep down the stable line.
Yes, each affected and maintained stable kernel. This would also ensure that it is part stable-RT trees.
Quoted 2 lines
Also, not really sure who is using the threadirqs option except for testing purposes.
Oh.
Quoted 2 lines
Thanks,
-Vladimir
Sebastian
Re: [PATCH net-next 2/2] enetc: use napi_schedule to be compatible with PREEMPT_RT
In reply to Vladimir OlteanHi Sasha,
Quoted 22 lines · Vladimir Oltean
Quoted 9 lines · Sebastian Andrzej Siewior
Quoted 3 lines · David Miller
From: Vladimir Oltean <[email protected]>
Quoted 2 lines
The driver calls
napi_schedule_irqoff()from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.…
Quoted 4 lines
Quoted 3 lines
Signed-off-by Jiafei Pan <[email protected]>Signed-off-by Vladimir Oltean <[email protected]>Quoted 7 lines
Quoted 1 line
Applied.
Could these two patches be forwarded -stable, please? The changelog describes this as a problem on
PREEMPT_RTbut this also happens on !RT with the `threadirqs' commandline switch.Sebastian
I expect the driver maintainers to have something to say about this. I didn't test on stable kernels, and at least for dpaa2-eth, the change would need to go pretty deep down the stable line.
Quoted 5 lines
Also, not really sure who is using the threadirqs option except for testing purposes.
Thanks,
-Vladimir
Do you think that this type of request is something that AUTOSEL can handle?
Thanks,
-Vladimir
From: Vladimir Oltean <[email protected]>
Date: Mon, 3 Aug 2020 23:10:08 +0300
Quoted 22 lines
From: Jiafei Pan <[email protected]>
The driver calls
napi_schedule_irqoff()from a context where, in RT, hardirqs are not disabled, since the IRQ handler is force-threaded.In the call path of this function,
__raise_softirq_irqoff()is modifying its per-CPU mask of pending softirqs that must be processed, usingor_softirq_pending(). Theor_softirq_pending()function is not atomic, but since interrupts are supposed to be disabled, nobody should be preempting it, and the operation should be safe.Nonetheless, when running with hardirqs on, as in the
PREEMPT_RTcase, it isn't safe, and the pending softirqs mask can get corrupted, resulting in softirqs being lost and never processed.To have common code that works with
PREEMPT_RTand with mainline Linux, we can use plainnapi_schedule()instead. The difference is thatnapi_schedule()(via__napi_schedule) also callslocal_irq_save, which disables hardirqs if they aren't already. But, since they already are disabled in non-RT, this means that in practice we don't see any measurable difference in throughput or latency with this patch.
Quoted 2 lines
Signed-off-by Jiafei Pan <[email protected]>Signed-off-by Vladimir Oltean <[email protected]>
Applied.