freeze with dpdk-2.0.0
Hi,
I am trying to use DPDK-2.0.0 on Fedora 20 running in a qemu virtual machine.
After loading the uio and igb_uio module and setting up hugepages, I try to run the helloworld demo application. As soon as the helloworld displays the "hello from core 1, hello from core 0" messages, I loose all network connectivity to the VM.
If I try to run the helloworld application remotely the output freezes after the following:
# ./build/helloworld -c 3 -n 2 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 4 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x7000000 bytes EAL: Virtual area found at 0x7f273be00000 (size = 0x7000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f273ba00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f273b600000 (size = 0x200000) EAL: Ask a virtual area of 0xc00000 bytes EAL: Virtual area found at 0x7f273a800000 (size = 0xc00000) EAL: Requesting 64 pages of size 2MB from socket 0 EAL: TSC frequency is ~1995193 KHz EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: Master lcore 0 is ready (tid=449e9900;cpuset=[0]) PMD: ENICPMD trace: rte_enic_pmd_init EAL: lcore 1 is ready (tid=3a7ff700;cpuset=[1])
If I try to restart the network services, I get the error from syslog:
BUG: soft lockup - CPU#2 stuck for 22s!
The network devices exposed to the VM are 2 BCM5719 interfaces + 2 82599ES interfaces:
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 0d:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 0d:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Within the VM, lspci shows:
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device 00:04.0 Ethernet controller: Red Hat, Inc Virtio network device 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device 00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
After loading the kernel modules, I bind the interfaces to DPDK with:/root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:05.0 /root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:06.0
Would you know what is causing this?
I haven't been able to try DPDK 1.8.0 because it doesn't compile on the latest Fedora Server 20.
Thanks for your help,
Olivier.
Signature · 9 lines
*Olivier Demé* *Druid Software Ltd.* *Tel: +353 1 202 1831* *Email: odeme-UsMDwKmwmRBx67MzidHQgQC/[email protected] <mailto:odeme-UsMDwKmwmRBx67MzidHQgQC/[email protected]>* *URL: http://www.druidsoftware.com* Druid Software: Monetising enterprise small cells solutions. Druid_Footer_Logo
A message that is not in the archive (sent off-list or not yet received): 552D4B36.4030209-UsMDwKmwmRBx67MzidHQgQC/[email protected]
Hi Oliver
Quoted 65 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Olivier Deme
Sent: Wednesday, April 15, 2015 1:16 AM
To: [email protected]
Subject: [dpdk-dev] freeze with dpdk-2.0.0Hi,
I am trying to use DPDK-2.0.0 on Fedora 20 running in a qemu virtual machine. After loading the uio and
igb_uiomodule and setting up hugepages, I try to run the helloworld demo application.
As soon as the helloworld displays the "hello from core 1, hello from core 0" messages, I loose all network connectivity to the VM.If I try to run the helloworld application remotely the output freezes after the following:
# ./build/helloworld -c 3 -n 2 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 4 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x7000000 bytes EAL: Virtual area found at 0x7f273be00000 (size = 0x7000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f273ba00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f273b600000 (size = 0x200000) EAL: Ask a virtual area of 0xc00000 bytes EAL: Virtual area found at 0x7f273a800000 (size = 0xc00000) EAL: Requesting 64 pages of size 2MB from socket 0 EAL: TSC frequency is ~1995193 KHz EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: Master lcore 0 is ready (tid=449e9900;cpuset=[0]) PMD: ENICPMD trace: rte_enic_pmd_init EAL: lcore 1 is ready (tid=3a7ff700;cpuset=[1])If I try to restart the network services, I get the error from syslog:
BUG: soft lockup - CPU#2 stuck for 22s!The network devices exposed to the VM are 2 BCM5719 interfaces + 2 82599ES interfaces:
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0d:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
0d:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)Within the VM, lspci shows:
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device 00:04.0 Ethernet controller: Red Hat, Inc Virtio network device 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device 00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
Quoted 6 lines
After loading the kernel modules, I bind the interfaces to DPDK with:
/root/dpdk-2.0.0/tools/dpdk_nic_bind.py--bind=igb_uio 00:05.0/root/dpdk-2.0.0/tools/dpdk_nic_bind.py--bind=igb_uio 00:06.0Would you know what is causing this?
Maybe you can try with "--b 00:03.0 00:04.0" to force dpdk not to grab these 2 virtio devices To see if this issue is resolved.
For virtio dev, dpdk doesn't depend on uio/igb_uio any more after the single virtio,
It will also grab the virtio device to use it even without uio module except for you specify it into black-list.
Thanks
Changchun
A message that is not in the archive (sent off-list or not yet received): F52918179C57134FAEC9EA62FA2F962511ABED7C-E2R4CRU6q/[email protected]
Hi Changchun,
I confirm that blacklisting the first two network interfaces solve the problem.
Just to be clear, is it the case that it is not possible to have a DPDK application acquiring selectively virtio network devices without using the blacklisting (-b) option?
It seems a bit clumsy to have the application being configured with all network interfaces to be bound to DPDK.
Thanks,
Olivier.
Quoted 82 lines · Changchun
Hi Oliver
Quoted 65 lines
-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Olivier Deme
Sent: Wednesday, April 15, 2015 1:16 AM
To: [email protected]
Subject: [dpdk-dev] freeze with dpdk-2.0.0Hi,
I am trying to use DPDK-2.0.0 on Fedora 20 running in a qemu virtual machine. After loading the uio and
igb_uiomodule and setting up hugepages, I try to run the helloworld demo application.
As soon as the helloworld displays the "hello from core 1, hello from core 0" messages, I loose all network connectivity to the VM.If I try to run the helloworld application remotely the output freezes after the following:
# ./build/helloworld -c 3 -n 2 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 4 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x7000000 bytes EAL: Virtual area found at 0x7f273be00000 (size = 0x7000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f273ba00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f273b600000 (size = 0x200000) EAL: Ask a virtual area of 0xc00000 bytes EAL: Virtual area found at 0x7f273a800000 (size = 0xc00000) EAL: Requesting 64 pages of size 2MB from socket 0 EAL: TSC frequency is ~1995193 KHz EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: Master lcore 0 is ready (tid=449e9900;cpuset=[0]) PMD: ENICPMD trace: rte_enic_pmd_init EAL: lcore 1 is ready (tid=3a7ff700;cpuset=[1])If I try to restart the network services, I get the error from syslog:
BUG: soft lockup - CPU#2 stuck for 22s!The network devices exposed to the VM are 2 BCM5719 interfaces + 2 82599ES interfaces:
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01)
0d:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
0d:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)Within the VM, lspci shows:
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device 00:04.0 Ethernet controller: Red Hat, Inc Virtio network device 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device 00:06.0 Ethernet controller: Red Hat, Inc Virtio network deviceQuoted 6 lines
After loading the kernel modules, I bind the interfaces to DPDK with:
/root/dpdk-2.0.0/tools/dpdk_nic_bind.py--bind=igb_uio 00:05.0/root/dpdk-2.0.0/tools/dpdk_nic_bind.py--bind=igb_uio 00:06.0Would you know what is causing this?
Maybe you can try with "--b 00:03.0 00:04.0" to force dpdk not to grab these 2 virtio devices To see if this issue is resolved.
For virtio dev, dpdk doesn't depend on uio/igb_uio any more after the single virtio,
It will also grab the virtio device to use it even without uio module except for you specify it into black-list.Thanks
Changchun
Signature · 9 lines
*Olivier Demé* *Druid Software Ltd.* *Tel: +353 1 202 1831* *Email: odeme-UsMDwKmwmRBx67MzidHQgQC/[email protected] <mailto:odeme-UsMDwKmwmRBx67MzidHQgQC/[email protected]>* *URL: http://www.druidsoftware.com* Druid Software: Monetising enterprise small cells solutions. Druid_Footer_Logo
A message that is not in the archive (sent off-list or not yet received): 552E3413.7050708-UsMDwKmwmRBx67MzidHQgQC/[email protected]
Hi Olivier
From: Olivier Deme [mailto:[email protected]]
Sent: Wednesday, April 15, 2015 5:49 PM
To: Ouyang, Changchun; [email protected]
Subject: Re: [dpdk-dev] freeze with dpdk-2.0.0
Hi Changchun,
I confirm that blacklisting the first two network interfaces solve the problem.
Just to be clear, is it the case that it is not possible to have a DPDK application acquiring selectively virtio network devices without using the blacklisting (-b) option?
It seems a bit clumsy to have the application being configured with all network interfaces to be bound to DPDK.
just virtio device, other types device doesn’t has issue. as the virtio pmd don’t rely on uio/igb_uio module any more, so it needs use –b to put whatever you need to blacklist, while on default all device are on white list. Thanks
Changchun