Weird NET_RX softirq behavior
Subsystems (2)
Hi list,
I observed one weird NET_RX softirq behavior:
1. CONFIG_RPS=y in kernel config file.
2. only one netdev in the system: eth0. eth0 only has one interrupt which is all handled by CPU0. eth0 doesn't support RFS_ACCEL
"cat /proc/interrupts" shows
~ # cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
...
49: 52 0 0 0 GIC mmc1 52: 53 0 0 0 GIC mmc2 56: 430365 0 0 0 GIC eth0 60: 61 0 0 0 GIC mmc0
...
3. RPS and RFS remains disabled as default.
~ # ls /sys/class/net/eth0/queues/
rx-0 rx-1 rx-2 rx-3 tx-0
~ # cat /sys/class/net/eth0/queues/rx-*/rps_cpus 0
0
0
0
~ # cat /proc/sys/net/core/rps_sock_flow_entries 0
~ # cat /sys/class/net/eth0/queues/rx-*/rps_flow_cnt 0
0
0
0
4. But NET_RX seems abnormal
~ # cat /proc/softirqs
CPU0 CPU1 CPU2 CPU3
HI: 0 0 0 0
TIMER: 92386 767 367 263
NET_TX: 6338 0 0 1
NET_RX: 445587 322 983 0
BLOCK: 0 0 0 0
BLOCK_IOPOLL: 0 0 0 0
TASKLET: 150 0 0 0
SCHED: 31414 490 275 179
HRTIMER: 1 0 0 0
RCU: 49029 570 285 182I'm expecting NET_RX under CPU1, 2, 3 should be zero. Any suggestions about this abnormal behavior?
Thanks in advance,
Jisheng
Quoted 1 line · On Thu, 7 Aug 2014 17:10:50 +0800 Jisheng Zhang <[email protected]> wrote:
2. only one netdev in the system: eth0.
There should also be lo (loopback) at least.
Quoted 7 lines
4. But
NET_RXseems abnormal
~ # cat/proc/softirqs
CPU0 CPU1 CPU2 CPU3
NET_RX: 445587 322 983 0I'm expecting
NET_RXunder CPU1, 2, 3 should be zero. Any suggestions about this abnormal behavior?
Do you have any loopback traffic? It could be handled by CPU1/2 explaining non-zero NET_RX counters.
Hi Dmitry,
Quoted 6 lines · On Thu, 7 Aug 2014 07:18:13 -0700 Dmitry Popov <[email protected]> wrote:
Quoted 1 line · On Thu, 7 Aug 2014 17:10:50 +0800 Jisheng Zhang <[email protected]> wrote:
2. only one netdev in the system: eth0.
There should also be lo (loopback) at least.
Yep, I forget that ;)
Quoted 11 lines
Quoted 8 lines
4. But
NET_RXseems abnormal
~ # cat/proc/softirqs
CPU0 CPU1 CPU2 CPU3
NET_RX: 445587 322 983 0I'm expecting
NET_RXunder CPU1, 2, 3 should be zero. Any suggestions about this abnormal behavior?Do you have any loopback traffic? It could be handled by CPU1/2 explaining non-zero
NET_RXcounters.
Yes. lo only send and recive 7 packets so far, about 1400bytes. Seems small comparing with the CPU1 and CPU2 NET_RX softirq numbers, right?
Any other possible case?
Thanks for your reply,
Jisheng
Quoted 4 lines · Jisheng Zhang
nd and recive 7 packets so far, about 1400bytes. Seems small comparing with the CPU1 and CPU2
NET_RXsoftirq numbers, right?Any other possible case?
Multicast loop. Check dev_loopback_xmit() and its callers.
Quoted 9 lines · On Thu, 7 Aug 2014 22:47:58 -0700 Eric Dumazet <[email protected]> wrote:
Quoted 4 lines · Jisheng Zhang
nd and recive 7 packets so far, about 1400bytes. Seems small comparing with the CPU1 and CPU2
NET_RXsoftirq numbers, right?Any other possible case?
Multicast loop. Check
dev_loopback_xmit()and its callers.
That's it! This make sense
I didn't see any NET_RX on CPU1,2,3 if I disabled mulitcast
Thanks very much.