Skip to contentSkip to conversation

netdevsim: call napi_schedule from a timer context

PATCH Superseded Breno Leitao 2 participants 3 messages Feb 17, 2025 – Feb 19, 2025

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 netdevsim: call napi_schedule from a timer context

The netdevsim driver was experiencing NOHZ tick-stop errors during packet transmission due to pending softirq work when calling napi_schedule(). This issue was observed when running the netconsole selftest, which triggered the following error message:

NOHZ tick-stop error: local softirq work is pending, handler #08!!!

To fix this issue, introduce a timer that schedules napi_schedule() from a timer context instead of calling it directly from the TX path.

Create an hrtimer for each queue and kick it from the TX path, which then schedules napi_schedule() from the timer context.

---
Changes in v2:
- The approach implemented in v1 will not work, given that ndo_start_xmit() can be called with interrupt disable, and calling local_bh_enable() inside that function has nasty side effected.
Jakub suggested creating a timer and calling napi_schedule() from that timer.
- Link to v1: https://lore.kernel.org/r/[email protected]

---
 drivers/net/netdevsim/netdev.c    | 21 ++++++++++++++++++++-
 drivers/net/netdevsim/netdevsim.h |  1 +
 2 files changed, 21 insertions(+), 1 deletion(-)
drivers/net/netdevsim/netdev.c +20 -1
index 9b394ddc5206a7a5ca5440341551aac50c43e20c..57e4ae386e44a9c3f72afc37a3a59d3a504ebad3 100644
@@ -87,7 +87,8 @@ static netdev_tx_t nsim_start_xmit(struct sk_buff *skb, struct net_device *dev)
 	if (unlikely(nsim_forward_skb(peer_dev, skb, rq) == NET_RX_DROP))
 		goto out_drop_cnt;
 
-	napi_schedule(&rq->napi);
+	if (!hrtimer_active(&rq->napi_timer))
+		hrtimer_start(&rq->napi_timer, us_to_ktime(5), HRTIMER_MODE_REL);
 
 	rcu_read_unlock();
 	u64_stats_update_begin(&ns->syncp);
@@ -426,6 +427,22 @@ static int nsim_init_napi(struct netdevsim *ns)
 	return err;
 }
 
+static enum hrtimer_restart nsim_napi_schedule(struct hrtimer *timer)
+{
+	struct nsim_rq *rq;
+
+	rq = container_of(timer, struct nsim_rq, napi_timer);
+	napi_schedule(&rq->napi);
+
+	return HRTIMER_NORESTART;
+}
+
+static void nsim_rq_timer_init(struct nsim_rq *rq)
+{
+	hrtimer_init(&rq->napi_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	rq->napi_timer.function = nsim_napi_schedule;
+}
+
 static void nsim_enable_napi(struct netdevsim *ns)
 {
 	struct net_device *dev = ns->netdev;
@@ -436,6 +453,7 @@ static void nsim_enable_napi(struct netdevsim *ns)
 
 		netif_queue_set_napi(dev, i, NETDEV_QUEUE_TYPE_RX, &rq->napi);
 		napi_enable(&rq->napi);
+		nsim_rq_timer_init(rq);
 	}
 }
 
@@ -461,6 +479,7 @@ static void nsim_del_napi(struct netdevsim *ns)
 	for (i = 0; i < dev->num_rx_queues; i++) {
 		struct nsim_rq *rq = ns->rq[i];
 
+		hrtimer_cancel(&rq->napi_timer);
 		napi_disable(&rq->napi);
 		__netif_napi_del(&rq->napi);
 	}
drivers/net/netdevsim/netdevsim.h +1 -0
index 96d54c08043d3a62b0731efd43bc6a313998bf01..e757f85ed8617bb13ed0bf0e367803e4ddbd8e95 100644
@@ -97,6 +97,7 @@ struct nsim_rq {
 	struct napi_struct napi;
 	struct sk_buff_head skb_queue;
 	struct page_pool *page_pool;
+	struct hrtimer napi_timer;
 };
 
 struct netdevsim {

---
base-commit: 0784d83df3bfc977c13252a0599be924f0afa68d
change-id: 20250212-netdevsim-258d2d628175

Best regards,

Signature · 1 line
Breno Leitao <[email protected]>
maintainer
#1 lore raw
Quoted 5 lines · On Mon, 17 Feb 2025 09:35:29 -0800 Breno Leitao wrote:

The netdevsim driver was experiencing NOHZ tick-stop errors during packet transmission due to pending softirq work when calling napi_schedule(). This issue was observed when running the netconsole selftest, which triggered the following error message:

Quoted 7 lines

NOHZ tick-stop error: local softirq work is pending, handler #08!!!

To fix this issue, introduce a timer that schedules napi_schedule() from a timer context instead of calling it directly from the TX path.

Create an hrtimer for each queue and kick it from the TX path, which then schedules napi_schedule() from the timer context.

This crashes in the nl_netdev test.

I think you should move the hrtimer init to nsim_queue_alloc() and removal to nsim_queue_free()

Signature · 1 line
pw-bot: cr
#2 lore raw

Hello Jakub,

Quoted 8 lines · Jakub Kicinski
Quoted 7 lines · On Mon, 17 Feb 2025 09:35:29 -0800 Breno Leitao wrote:

The netdevsim driver was experiencing NOHZ tick-stop errors during packet transmission due to pending softirq work when calling napi_schedule(). This issue was observed when running the netconsole selftest, which triggered the following error message:

NOHZ tick-stop error: local softirq work is pending, handler #08!!!

Quoted 7 lines
Quoted 5 lines

To fix this issue, introduce a timer that schedules napi_schedule() from a timer context instead of calling it directly from the TX path.

Create an hrtimer for each queue and kick it from the TX path, which then schedules napi_schedule() from the timer context.

This crashes in the nl_netdev test.

Yea, a nasty crash. Looking at the crash, it seems to be disabling the timer before initializing it, and timer->base was not properly assigned/set.

Quoted 2 lines

I think you should move the hrtimer init to nsim_queue_alloc() and removal to nsim_queue_free()

That seems to make nl_netdev happier. Let me do more tests, and then ask NIPA do finish the work.

Thanks!