Skip to contentSkip to conversation

net, rps: bypass enqueue_to_backlog()

RFC PATCH In review Zhi Yong Wu 2 participants 6 messages Dec 18, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 net, rps: bypass enqueue_to_backlog()

From: Zhi Yong Wu <[email protected]>

When local cpu is just target cpu which will handle network soft irq, the packet should be directly injected to network stack, by bypassing enqueue_to_backlog(), it can speed up the packet processing.

HI, guys

I checked the first several versions of RPS patch which seemed to have this condition determination, but why was it removed later? Do i miss anything? if yes, please correct me, thanks.

---
 net/core/dev.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
net/core/dev.c +1 -1
index c482fe8..d29d61f 100644
@@ -3693,7 +3693,7 @@ int netif_receive_skb(struct sk_buff *skb)
 
 		cpu = get_rps_cpu(skb->dev, skb, &rflow);
 
-		if (cpu >= 0) {
+		if ((cpu >= 0) && (cpu != raw_smp_processor_id())) {
 			ret = enqueue_to_backlog(skb, cpu, &rflow->last_qtail);
 			rcu_read_unlock();
 			return ret;
Signature · 1 line
1.7.6.5
#1 lore raw
Quoted 6 lines · Zhi Yong Wu

From: Zhi Yong Wu <[email protected]>

When local cpu is just target cpu which will handle network soft irq, the packet should be directly injected to network stack, by bypassing enqueue_to_backlog(), it can speed up the packet processing.

Quoted 8 lines

HI, guys

I checked the first several versions of RPS patch which seemed to have this condition determination, but why was it removed later? Do i miss anything? if yes, please correct me, thanks.

---

Hmm... Could you elaborate ?

At which point do you think this condition was tested or removed ?

I think the idea was to drain NIC RX queues as fast as possible, then :

- Send the IPI to remote cpus
- process our queue in parallel with other cpus processing their own queue.

If we process our packets through whole stack, packets for other cpus will have a fair amount of extra latency.

Thats a tradeoff I suppose.

#2 lore raw
Quoted 22 lines · Eric Dumazet
Quoted 6 lines · Zhi Yong Wu

From: Zhi Yong Wu <[email protected]>

When local cpu is just target cpu which will handle network soft irq, the packet should be directly injected to network stack, by bypassing enqueue_to_backlog(), it can speed up the packet processing.

Quoted 8 lines

HI, guys

I checked the first several versions of RPS patch which seemed to have this condition determination, but why was it removed later? Do i miss anything? if yes, please correct me, thanks.

---

Hmm... Could you elaborate ?

At which point do you think this condition was tested or removed ?

I think the idea was to drain NIC RX queues as fast as possible, then :

Quoted 9 lines

- Send the IPI to remote cpus
- process our queue in parallel with other cpus processing their own queue.

If we process our packets through whole stack, packets for other cpus will have a fair amount of extra latency.

Thats a tradeoff I suppose.

Also note that going through the backlog permits thinks like

99bbc70741903c0 ("rps: selective flow shedding during softnet overflow")

#3 lore raw
Quoted 30 lines · Eric Dumazet
Quoted 29 lines · Eric Dumazet
Quoted 6 lines · Zhi Yong Wu

From: Zhi Yong Wu <[email protected]>

When local cpu is just target cpu which will handle network soft irq, the packet should be directly injected to network stack, by bypassing enqueue_to_backlog(), it can speed up the packet processing.

Quoted 8 lines

HI, guys

I checked the first several versions of RPS patch which seemed to have this condition determination, but why was it removed later? Do i miss anything? if yes, please correct me, thanks.

---

Hmm... Could you elaborate ?

At which point do you think this condition was tested or removed ?

I think the idea was to drain NIC RX queues as fast as possible, then :

- Send the IPI to remote cpus
- process our queue in parallel with other cpus processing their own queue.

If we process our packets through whole stack, packets for other cpus will have a fair amount of extra latency.

Quoted 7 lines
Quoted 2 lines

Thats a tradeoff I suppose.

Also note that going through the backlog permits thinks like

99bbc70741903c0 ("rps: selective flow shedding during softnet overflow"

it makes sense, but if cpu < 0, some packets seems to bypass the flow shedding stuff, That is, the flow limit will be not so accurate, right?

Quoted 3 lines
Signature · 3 lines
Regards,

Zhi Yong Wu
#4 lore raw
Quoted 3 lines · Zhi Yong Wu

it makes sense, but if cpu < 0, some packets seems to bypass the flow shedding stuff, That is, the flow limit will be not so accurate, right?

Not sure what you mean.

cpu = -1 on very rare occasions, like cpu hot unplug.

In netif_receive_skb(), we just process the packet normally.

#5 lore raw
In reply to Eric Dumazet

thanks for your explanation.

by the way, pls ignore this patch, thanks.

Quoted 32 lines · Eric Dumazet
Quoted 6 lines · Zhi Yong Wu

From: Zhi Yong Wu <[email protected]>

When local cpu is just target cpu which will handle network soft irq, the packet should be directly injected to network stack, by bypassing enqueue_to_backlog(), it can speed up the packet processing.

Quoted 8 lines

HI, guys

I checked the first several versions of RPS patch which seemed to have this condition determination, but why was it removed later? Do i miss anything? if yes, please correct me, thanks.

---

Hmm... Could you elaborate ?

At which point do you think this condition was tested or removed ?

I think the idea was to drain NIC RX queues as fast as possible, then :

- Send the IPI to remote cpus
- process our queue in parallel with other cpus processing their own queue.

If we process our packets through whole stack, packets for other cpus will have a fair amount of extra latency.

Thats a tradeoff I suppose.

Signature · 3 lines
Regards,

Zhi Yong Wu