net/ena: Add Windows support.
Subsystems (2)
RFC PATCH 1/1 net/ena: Add Windows support.
I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver. I want to know if this is the right direction, or whether I also need to change other places in ENA driver?
I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/
https://stackoverflow.com/questions/10905892/equivalent-of-gettimeday-for-windows
Thanks.
--- drivers/net/ena/base/ena_com.c | 4 +- drivers/net/ena/base/ena_plat.h | 2 +- drivers/net/ena/base/ena_plat_dpdk.h | 14 +++- drivers/net/ena/meson.build | 5 -- lib/eal/version.map | 2 +- lib/eal/windows/include/pthread.h | 105 ++++++++++++++++++++++++++ lib/eal/windows/include/rte_windows.h | 1 + 7 files changed, 120 insertions(+), 13 deletions(-)
drivers/net/ena/base/ena_com.c +2 -2
@@ -1375,11 +1375,11 @@ int ena_com_execute_admin_command(struct ena_com_admin_queue *admin_queue,if (IS_ERR (comp_ctx)) {if (comp_ctx ==ERR_PTR (ENA_COM_NO_DEVICE ))ena_trc_dbg (admin_queue->ena_dev, -"Failed to submit command [%ld]\n" , +"Failed to submit command [%" PRIu64"]\n" ,PTR_ERR (comp_ctx));else ena_trc_err (admin_queue->ena_dev, -"Failed to submit command [%ld]\n" , +"Failed to submit command [%" PRIu64"]\n" ,PTR_ERR (comp_ctx));return (int )PTR_ERR (comp_ctx);
drivers/net/ena/base/ena_plat.h +1 -1
@@ -21,7 +21,7 @@#include <ena_plat_dpdk.h> #endif #elif defined(_WIN32) -#include <ena_plat_windows.h> +#include <ena_plat_dpdk.h> #else #error "Invalid platform"#endif
drivers/net/ena/base/ena_plat_dpdk.h +10 -4
@@ -13,6 +13,7 @@#include <inttypes.h> #include <string.h> #include <errno.h> +#include <pthread.h> #include <rte_atomic.h> #include <rte_branch_prediction.h>
@@ -24,7 +25,12 @@#include <rte_prefetch.h> #include <rte_spinlock.h> +#if defined(_WIN32) +#include <rte_time.h> +#include <rte_thread.h> +#else #include <sys/time.h> +#endif #include <rte_memcpy.h> typedef uint64_t u64 ;
@@ -293,9 +299,9 @@ extern rte_atomic64_t ena_alloc_cnt;#define dma_rmb () rmb()#define MAX_ERRNO 4095 -#define IS_ERR (x) (((unsigned long)x) >= (unsigned long)-MAX_ERRNO) -#define ERR_PTR (error) ((void *)(long)error) -#define PTR_ERR (error) ((long)(void *)error) +#define IS_ERR (x) (((uintptr_t)x) >= (uintptr_t)-MAX_ERRNO) +#define ERR_PTR (error) ((void *)(uintptr_t)error) +#define PTR_ERR (error) ((uintptr_t)(void *)error)#define might_sleep ()#define prefetch (x) rte_prefetch0(x)
@@ -322,7 +328,7 @@ extern rte_atomic64_t ena_alloc_cnt;#define DIV_ROUND_UP (n, d) (((n) + (d) - 1) / (d)) -#define ENA_FFS (x) ffs(x) +#define ENA_FFS (x) __builtin_ffs(x)void ena_rss_key_fill (void *key,size_t size);
drivers/net/ena/meson.build +0 -5
@@ -1,11 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -if is_windows - build = false - reason = 'not supported on Windows' - subdir_done() -endif sources = files( 'ena_ethdev.c',
lib/eal/version.map +1 -1
@@ -198,7 +198,7 @@ DPDK_21 { rte_uuid_is_null; # WINDOWS_NO_EXPORT rte_uuid_parse; # WINDOWS_NO_EXPORT rte_uuid_unparse; # WINDOWS_NO_EXPORT - rte_version; # WINDOWS_NO_EXPORT + rte_version; rte_vfio_clear_group; # WINDOWS_NO_EXPORT rte_vfio_container_create; # WINDOWS_NO_EXPORT rte_vfio_container_destroy; # WINDOWS_NO_EXPORT
lib/eal/windows/include/pthread.h +105 -0
123 lines not on this page · raw message
lib/eal/windows/include/rte_windows.h +1 -0
@@ -31,6 +31,7 @@#include <psapi.h> #include <setupapi.h> #include <winioctl.h> +#include <winsock2.h> /* Have GUIDs defined. */ #ifndef INITGUID
Signature · 1 line
2.30.2
Hi William,
2021-08-14 03:36 (UTC+0000), William Tu:
Quoted 6 lines
I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver.
I want to know if this is the right direction,
or whether I also need to change other places in ENA driver?
Cc'ing maintainers.
Quoted 2 lines
I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/
We don't want to add more pthread shims, pthread.h in DPDK will go away. Condition variable support should be added to the new threading API:
http://inbox.dpdk.org/dev/[email protected]
I suppose it can be done independently of the new threading API series. When copying code to DPDK from elsewhere, please adapt its style to DPDK conventions (e.g. RTE_UNUSED(foo) instead if (void)foo) and mind the license if big pieces are copied verbatim (this is not the case here).
POSIX gettimeofday() should be replaced with standard C timespec_get().
Quoted 12 lines
Thanks.
Signed-off-by William Tu <[email protected]>--- drivers/net/ena/base/ena_com.c | 4 +- drivers/net/ena/base/ena_plat.h | 2 +- drivers/net/ena/base/ena_plat_dpdk.h | 14 +++- drivers/net/ena/meson.build | 5 -- lib/eal/version.map | 2 +- lib/eal/windows/include/pthread.h | 105 ++++++++++++++++++++++++++ lib/eal/windows/include/rte_windows.h | 1 + 7 files changed, 120 insertions(+), 13 deletions(-)
This should be a series of two patches:
1) adding condition variable wrappers to EAL;
2) supporting net/ena for Windows.
Quoted 16 lines · Dmitry Kozlyuk
Hi William,
2021-08-14 03:36 (UTC+0000), William Tu:
Quoted 6 lines
I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver.
I want to know if this is the right direction,
or whether I also need to change other places in ENA driver?Cc'ing maintainers.
Quoted 2 lines
I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/
Quoted 6 lines
We don't want to add more pthread shims,
pthread.hin DPDK will go away. Condition variable support should be added to the new threading API:http://inbox.dpdk.org/dev/[email protected]I suppose it can be done independently of the new threading API series.
yes, I think so.
Quoted 21 lines
When copying code to DPDK from elsewhere, please adapt its style to DPDK conventions (e.g.
RTE_UNUSED(foo)instead if(void)foo) and mind the license if big pieces are copied verbatim (this is not the case here).POSIX
gettimeofday()should be replaced with standard Ctimespec_get().Quoted 12 lines
Thanks.
Signed-off-by William Tu <[email protected]>--- drivers/net/ena/base/ena_com.c | 4 +- drivers/net/ena/base/ena_plat.h | 2 +- drivers/net/ena/base/ena_plat_dpdk.h | 14 +++- drivers/net/ena/meson.build | 5 -- lib/eal/version.map | 2 +- lib/eal/windows/include/pthread.h | 105 ++++++++++++++++++++++++++ lib/eal/windows/include/rte_windows.h | 1 + 7 files changed, 120 insertions(+), 13 deletions(-)
Quoted 4 lines
This should be a series of two patches:
1) adding condition variable wrappers to EAL;
2) supporting net/ena for Windows.
Thanks Dmitry! Will work on v2.
William
William
Quoted 43 lines · Dmitry Kozlyuk
Hi William,
2021-08-14 03:36 (UTC+0000), William Tu:
Quoted 6 lines
I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver.
I want to know if this is the right direction,
or whether I also need to change other places in ENA driver?Cc'ing maintainers.
Quoted 2 lines
I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/We don't want to add more pthread shims,
pthread.hin DPDK will go away. Condition variable support should be added to the new threading API:http://inbox.dpdk.org/dev/[email protected]I suppose it can be done independently of the new threading API series. When copying code to DPDK from elsewhere, please adapt its style to DPDK conventions (e.g.
RTE_UNUSED(foo)instead if(void)foo) and mind the license if big pieces are copied verbatim (this is not the case here).POSIX
gettimeofday()should be replaced with standard Ctimespec_get().Quoted 12 lines
Thanks.
Signed-off-by William Tu <[email protected]>--- drivers/net/ena/base/ena_com.c | 4 +- drivers/net/ena/base/ena_plat.h | 2 +- drivers/net/ena/base/ena_plat_dpdk.h | 14 +++- drivers/net/ena/meson.build | 5 -- lib/eal/version.map | 2 +- lib/eal/windows/include/pthread.h | 105 ++++++++++++++++++++++++++ lib/eal/windows/include/rte_windows.h | 1 + 7 files changed, 120 insertions(+), 13 deletions(-)
Quoted 4 lines
This should be a series of two patches:
1) adding condition variable wrappers to EAL;
2) supporting net/ena for Windows.
So I finally set up everything on AWS and tested this patch. Virt2phys and netuio (I added ena conf) load OK.
Finally, starting dpdk-testpmd, I got some errors below:
---
PS C:\dpdk-kmods> cd c:\dpdk
PS C:\dpdk> .\build\app\dpdk-testpmd.exe
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process support is requested, but not available.
EAL: WARNING: TSC frequency estimated roughly - clock timings may be less accurate.
EAL: Requested device 0000:00:05.0 cannot be used
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket 0)
EAL: eth_ena_pci_probe[ENA_COM:
ena_com_wait_and_process_admin_cq_interrupts]Invalid wait event. pthread ret: 1
[ENA_COM: ena_com_wait_and_process_admin_cq_interrupts]The ena device sent a completion but the driver didn
't receive a MSI-X interrupt (cmd 9), autopolling mode is OFF [ENA_COM: ena_com_set_dev_mtu]Failed to set mtu 1500. error: -137 ena_mtu_set(): Could not set MTU: 1500
Failed to set MTU to 1500 for port 0
testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.
Configuring Port 0 (socket 0)
[ENA_COM: ena_com_create_io_cq]Failed to create IO CQ. error: -19 ena_create_io_queue(): Failed to create IO queue[0] (qid:1), rc: -19 ena_queue_start(): Failed to create IO queue ena_queue_start_all(): Failed to start queue[0] of type(1) Fail to start port 0: No such device
Please stop the ports first
Done
Error during enabling promiscuous mode for port 0: Unknown error - ignore No commandline core given, start packet forwarding
Not all ports were started
---
I will see if I can fix these issues and I will submit v2 patch. Thanks
William
czw., 19 sie 2021 o 04:19 William Tu <[email protected]> napisał(a):
Quoted 29 lines
So I finally set up everything on AWS and tested this patch. Virt2phys and netuio (I added ena conf) load OK.
Finally, starting dpdk-testpmd, I got some errors below:
---
PS C:\dpdk-kmods> cd c:\dpdk
PS C:\dpdk> .\build\app\dpdk-testpmd.exe
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process support is requested, but not available.
EAL: WARNING: TSC frequency estimated roughly - clock timings may be less accurate.
EAL: Requested device 0000:00:05.0 cannot be used
EAL: Probe PCI driver:net_ena(1d0f:ec20) device: 0000:00:06.0 (socket 0)
EAL: eth_ena_pci_probe[ENA_COM:
ena_com_wait_and_process_admin_cq_interrupts]Invalid wait event. pthread ret: 1
[ENA_COM: ena_com_wait_and_process_admin_cq_interrupts]The ena device sent a completion but the driver didn
't receive a MSI-X interrupt (cmd 9), autopolling mode is OFF [ENA_COM: ena_com_set_dev_mtu]Failed to set mtu 1500. error: -137ena_mtu_set(): Could not set MTU: 1500
Failed to set MTU to 1500 for port 0
testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected:ring_mp_mcWarning! port-topology=paired and odd forward ports number, the last port will pair with itself.
Quoted 14 lines
Configuring Port 0 (socket 0)
[ENA_COM: ena_com_create_io_cq]Failed to create IO CQ. error: -19ena_create_io_queue(): Failed to create IO queue[0] (qid:1), rc: -19ena_queue_start(): Failed to create IO queueena_queue_start_all(): Failed to start queue[0] of type(1) Fail to start port 0: No such device
Please stop the ports first
Done
Error during enabling promiscuous mode for port 0: Unknown error - ignore No commandline core given, start packet forwarding
Not all ports were started
---
I will see if I can fix these issues and I will submit v2 patch. Thanks
Hi William,
It's great to hear that you're working on ENA support for Windows!
ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.
It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.
Thanks,
Michal
Quoted 1 line
William
On Mon, Aug 30, 2021 at 12:12 AM Michał Krawczyk <[email protected]> wrote:
[...]
Quoted 9 lines
Hi William,
It's great to hear that you're working on ENA support for Windows!
ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.
Quoted 4 lines
It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.
Hi Michal,
Thank you! Then I will wait for netuio support.
William
Quoted 11 lines · William Tu
On Mon, Aug 30, 2021 at 12:12 AM Michał Krawczyk <[email protected]> wrote:
[...]Quoted 9 lines
Hi William,
It's great to hear that you're working on ENA support for Windows!
ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.
Quoted 8 lines
Quoted 4 lines
It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.
Hi Michal,
Thank you! Then I will wait for netuio support.
William
Hi William, Michał,
This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?
sob., 21 maj 2022 o 00:08 Ferruh Yigit <[email protected]> napisał(a):
Quoted 23 lines · William Tu
Quoted 11 lines · Michał Krawczyk
[...]
Quoted 9 lines
Hi William,
It's great to hear that you're working on ENA support for Windows!
ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.
Quoted 8 lines
Quoted 4 lines
It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.
Hi Michal,
Thank you! Then I will wait for netuio support.
William
Quoted 4 lines
Hi William, Michał,
This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?
Hi Ferruh,
sorry for the late reply - nothing new from my side.
I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.
Thanks,
Michal
Quoted 29 lines · Michał Krawczyk
sob., 21 maj 2022 o 00:08 Ferruh Yigit <[email protected]> napisał(a):
Quoted 23 lines · William Tu
Quoted 11 lines · Michał Krawczyk
[...]
Quoted 9 lines
Hi William,
It's great to hear that you're working on ENA support for Windows!
ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.
Quoted 8 lines
Quoted 4 lines
It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.
Hi Michal,
Thank you! Then I will wait for netuio support.
WilliamQuoted 4 lines
Hi William, Michał,
This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?
Quoted 8 lines
Hi Ferruh,
sorry for the late reply - nothing new from my side.
I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.
Hi William, Dmitry,
Is there any update on the netuio interrupt support? What is the plan for this patch?
2023-03-23 13:13 (UTC+0000), Ferruh Yigit:
Quoted 39 lines · Michał Krawczyk
Quoted 29 lines
sob., 21 maj 2022 o 00:08 Ferruh Yigit <[email protected]> napisał(a):
Quoted 23 lines · William Tu
Quoted 11 lines · Michał Krawczyk
[...]
Quoted 9 lines
Hi William,
It's great to hear that you're working on ENA support for Windows!
ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.
Quoted 8 lines
Quoted 4 lines
It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.
Hi Michal,
Thank you! Then I will wait for netuio support.
WilliamQuoted 4 lines
Hi William, Michał,
This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?
Quoted 8 lines
Hi Ferruh,
sorry for the late reply - nothing new from my side.
I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.
Quoted 4 lines
Hi William, Dmitry,
Is there any update on the netuio interrupt support? What is the plan for this patch?
Hi Ferruh,
The work on the interrupt support has been abandoned unfortunately. I'd like to complete it one day, but can't make a commitment right now.
Ref:
http://patchwork.dpdk.org/project/dpdk/patch/[email protected]/
Quoted 17 lines · On Thu, 23 Mar 2023 17:19:55 +0300 Dmitry Kozlyuk <[email protected]>…
Quoted 12 lines
Quoted 11 lines
Quoted 2 lines
This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?
Hi Ferruh,
sorry for the late reply - nothing new from my side.
I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.
Quoted 4 lines
Hi William, Dmitry,
Is there any update on the netuio interrupt support? What is the plan for this patch?
Quoted 7 lines
Hi Ferruh,
The work on the interrupt support has been abandoned unfortunately. I'd like to complete it one day, but can't make a commitment right now.
Ref:
http://patchwork.dpdk.org/project/dpdk/patch/[email protected]/
Marking this patch as "Awaiting upstream" since it depends on interrupt support in Windows which is not available.