Skip to contentSkip to conversation

Weird NET_RX softirq behavior

DISCUSSION Jisheng Zhang 3 participants 5 messages Aug 7, 2014 – Aug 8, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi list,

I observed one weird NET_RX softirq behavior:

1. CONFIG_RPS=y in kernel config file.

2. only one netdev in the system: eth0. eth0 only has one interrupt which is all handled by CPU0. eth0 doesn't support RFS_ACCEL

"cat /proc/interrupts" shows

~ # cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
...

 49:         52          0          0          0       GIC  mmc1
 52:         53          0          0          0       GIC  mmc2
 56:     430365          0          0          0       GIC  eth0
 60:         61          0          0          0       GIC  mmc0

...

3. RPS and RFS remains disabled as default.

~ # ls /sys/class/net/eth0/queues/
rx-0 rx-1 rx-2 rx-3 tx-0
~ # cat /sys/class/net/eth0/queues/rx-*/rps_cpus 0
0
0
0
~ # cat /proc/sys/net/core/rps_sock_flow_entries 0
~ # cat /sys/class/net/eth0/queues/rx-*/rps_flow_cnt 0
0
0
0

4. But NET_RX seems abnormal
~ # cat /proc/softirqs
CPU0 CPU1 CPU2 CPU3
HI: 0 0 0 0
TIMER: 92386 767 367 263
NET_TX: 6338 0 0 1
NET_RX: 445587 322 983 0
BLOCK: 0 0 0 0

BLOCK_IOPOLL:          0          0          0          0
     TASKLET:        150          0          0          0
       SCHED:      31414        490        275        179
     HRTIMER:          1          0          0          0
         RCU:      49029        570        285        182

I'm expecting NET_RX under CPU1, 2, 3 should be zero. Any suggestions about this abnormal behavior?

Thanks in advance,
Jisheng

#1 lore raw
Quoted 1 line · On Thu, 7 Aug 2014 17:10:50 +0800 Jisheng Zhang <[email protected]> wrote:

2. only one netdev in the system: eth0.

There should also be lo (loopback) at least.

Quoted 7 lines

4. But NET_RX seems abnormal
~ # cat /proc/softirqs
CPU0 CPU1 CPU2 CPU3
NET_RX: 445587 322 983 0

I'm expecting NET_RX under CPU1, 2, 3 should be zero. Any suggestions about this abnormal behavior?

Do you have any loopback traffic? It could be handled by CPU1/2 explaining non-zero NET_RX counters.

#2 lore raw

Hi Dmitry,

Quoted 6 lines · On Thu, 7 Aug 2014 07:18:13 -0700 Dmitry Popov <[email protected]> wrote:
Quoted 1 line · On Thu, 7 Aug 2014 17:10:50 +0800 Jisheng Zhang <[email protected]> wrote:

2. only one netdev in the system: eth0.

There should also be lo (loopback) at least.

Yep, I forget that ;)

Quoted 11 lines
Quoted 8 lines

4. But NET_RX seems abnormal
~ # cat /proc/softirqs
CPU0 CPU1 CPU2 CPU3
NET_RX: 445587 322 983 0

I'm expecting NET_RX under CPU1, 2, 3 should be zero. Any suggestions about this abnormal behavior?

Do you have any loopback traffic? It could be handled by CPU1/2 explaining non-zero NET_RX counters.

Yes. lo only send and recive 7 packets so far, about 1400bytes. Seems small comparing with the CPU1 and CPU2 NET_RX softirq numbers, right?

Any other possible case?

Thanks for your reply,
Jisheng

#3 lore raw
Quoted 4 lines · Jisheng Zhang

nd and recive 7 packets so far, about 1400bytes. Seems small comparing with the CPU1 and CPU2 NET_RX softirq numbers, right?

Any other possible case?

Multicast loop. Check dev_loopback_xmit() and its callers.

#4 lore raw
Quoted 9 lines · On Thu, 7 Aug 2014 22:47:58 -0700 Eric Dumazet <[email protected]> wrote:
Quoted 4 lines · Jisheng Zhang

nd and recive 7 packets so far, about 1400bytes. Seems small comparing with the CPU1 and CPU2 NET_RX softirq numbers, right?

Any other possible case?

Multicast loop. Check dev_loopback_xmit() and its callers.

That's it! This make sense

I didn't see any NET_RX on CPU1,2,3 if I disabled mulitcast

Thanks very much.