New: kernel hang occasionally while running the sample of xdpsock
Begin forwarded message:
Date: Mon, 22 Jun 2020 10:13:52 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 208275] New: kernel hang occasionally while running the sample of xdpsock
https://bugzilla.kernel.org/show_bug.cgi?id=208275
Bug ID: 208275
Summary: kernel hang occasionally while running the sample of
xdpsock
Product: Networking
Version: 2.5
Kernel Version: 5.7.0
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: high
Priority: P1
Component: Other
Assignee: [email protected]
Reporter: [email protected]
Regression: NoDistribution:
5.7.0-1.el7.centos.x86_64
Hardware Environment:
Dell Inc. PowerEdge R730/0WCJNT, BIOS 2.1.7 06/16/2016
Software Environment:
Problem Description:
kernel hang occasionally while running the sample of xdpsock
Steps to reproduce:
I want to test the rx performace of AF_XDP. I change the nic to 4 queues by cmd ethtool -L p6p1 combined 4, then I will create 1 socket for every queue.
for ((i=0; i<4; i++));
do
./xdpsock -r -z -i p6p1 -m -q $i &
done
I run the xdpsock in samples/bpf using the shell command above. And occasionally the kernel hang, so I have to power off and on.
Additonal information:
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
Quoted 10 lines · Stephen Hemminger
Begin forwarded message:
Date: Mon, 22 Jun 2020 10:13:52 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 208275] New: kernel hang occasionally while running the sample of xdpsock
Thanks for forwarding, Stephen.
I'll have a look!
Björn
Quoted 50 lines
https://bugzilla.kernel.org/show_bug.cgi?id=208275
Bug ID: 208275 Summary: kernel hang occasionally while running the sample of xdpsock Product: Networking Version: 2.5 Kernel Version: 5.7.0 Hardware: All OS: Linux Tree: Mainline Status: NEW Severity: high Priority: P1 Component: Other Assignee: [email protected] Reporter: [email protected] Regression: NoDistribution:
5.7.0-1.el7.centos.x86_64Hardware Environment:
Dell Inc. PowerEdge R730/0WCJNT, BIOS 2.1.7 06/16/2016Software Environment:
Problem Description:
kernel hang occasionally while running the sample of xdpsockSteps to reproduce:
I want to test the rx performace of
AF_XDP. I change the nic to 4 queues by cmdethtool -L p6p1 combined 4, then I will create 1 socket for every queue.for ((i=0; i<4; i++));
do
./xdpsock -r -z -i p6p1 -m -q $i &
doneI run the xdpsock in
samples/bpfusing the shell command above. And occasionally the kernel hang, so I have to power off and on.Additonal information:
--
You are receiving this mail because:
You are the assignee for the bug.
Quoted 14 lines · Bj?rn T?pel
Quoted 10 lines · Stephen Hemminger
Begin forwarded message:
Date: Mon, 22 Jun 2020 10:13:52 +0000
From: bugzilla-daemon at bugzilla.kernel.org
To: stephen at networkplumber.org
Subject: [Bug 208275] New: kernel hang occasionally while running the sample of xdpsock
Quoted 4 lines
Thanks for forwarding, Stephen.
I'll have a look!
Intel ixgbe splat. Adding intel-wired-lan to To:.
Hi, Bjorn, Thank your response.
Could you describe it more clearly? I can not get it exactly. Thx.
Bj?rn T?pel <[email protected]> ?2020?6?23??? ??1:50???
Quoted 16 lines · Bj?rn T?pel
Quoted 6 lines
Quoted 3 lines · Stephen Hemminger
Quoted 6 lines
Quoted 6 lines
Begin forwarded message:
Date: Mon, 22 Jun 2020 10:13:52 +0000
From: bugzilla-daemon at bugzilla.kernel.org
To: stephen at networkplumber.org
Subject: [Bug 208275] New: kernel hang occasionally while running thesample of xdpsock
Quoted 2 lines
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Thanks for forwarding, Stephen.
I'll have a look!
Intel ixgbe splat. Adding intel-wired-lan to To:.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20200623/75f6ccfb/attachment.html>
Hi, Bjorn, Thx for your response.
I can not get it exactly. Could you describe the details more clearly? Thx.
Quoted 60 lines · Stephen Hemminger <[email protected]> 于2020年6月23日周二 上午12:08写道:
Begin forwarded message:
Date: Mon, 22 Jun 2020 10:13:52 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 208275] New: kernel hang occasionally while running the sample of xdpsockhttps://bugzilla.kernel.org/show_bug.cgi?id=208275
Bug ID: 208275 Summary: kernel hang occasionally while running the sample of xdpsock Product: Networking Version: 2.5 Kernel Version: 5.7.0 Hardware: All OS: Linux Tree: Mainline Status: NEW Severity: high Priority: P1 Component: Other Assignee: [email protected] Reporter: [email protected] Regression: NoDistribution:
5.7.0-1.el7.centos.x86_64Hardware Environment:
Dell Inc. PowerEdge R730/0WCJNT, BIOS 2.1.7 06/16/2016Software Environment:
Problem Description:
kernel hang occasionally while running the sample of xdpsockSteps to reproduce:
I want to test the rx performace of
AF_XDP. I change the nic to 4 queues by cmdethtool -L p6p1 combined 4, then I will create 1 socket for every queue.for ((i=0; i<4; i++));
do
./xdpsock -r -z -i p6p1 -m -q $i &
doneI run the xdpsock in
samples/bpfusing the shell command above. And occasionally the kernel hang, so I have to power off and on.Additonal information:
--
You are receiving this mail because:
You are the assignee for the bug.
Quoted 5 lines · Yahui Chen
Hi, Bjorn, Thank your response.
Could you describe it more clearly? I can not get it exactly. Thx.
When XDP is enabled, the ixgbe NIC does a (somewhat heavy)
reconfiguration. During the reconfiguration, for some reason, the rx_buffer->page is NULL in the following call chain:
ixgbe_down()->ixgbe_clean_all_rx_rings()->ixgbe_clean_rx_ring()->__page_frag_cache_drain()
This results in that when __page_frag_cache_drain() want to touch the reference counter, you get a NULL pointer dereference.
[277994.329145] BUG: kernel NULL pointer dereference, address: 0000000000000034 ... [277994.329428] RIP: 0010:__page_frag_cache_drain+0x5/0x40 [277994.329463] Code: d2 ff ff 31 f6 84 c0 74 04 0f b6 73 51 48 89 df
e8 70 ff ff ff eb dc 48 83 eb 01 eb d0 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 <f0> 29 77 34 74 01 c3 48 8b 07 55 48 89 e5 a9 00 00 01 00 74 0f 0f
2a:* f0 29 77 34 lock sub %esi,0x34(%rdi) <--
trapping instruction
I tried to reproduce the issue, but without success so far. I'll keep looking for the bug. Hopefully someone from Intel with better insight into ixgbe can help!
Bj?rn