Skip to contentSkip to conversation

net/ena: Add Windows support.

RFC PATCH In review William Tu 5 participants 11 messages Aug 14, 2021 – Aug 10, 2023

Subsystems (2)
lore
Share & tools
lore

William Tu

author
#0 lore raw

RFC PATCH 1/1 net/ena: Add Windows support.

I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver. I want to know if this is the right direction, or whether I also need to change other places in ENA driver?

I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/
https://stackoverflow.com/questions/10905892/equivalent-of-gettimeday-for-windows
Thanks.

---
 drivers/net/ena/base/ena_com.c        |   4 +-
 drivers/net/ena/base/ena_plat.h       |   2 +-
 drivers/net/ena/base/ena_plat_dpdk.h  |  14 +++-
 drivers/net/ena/meson.build           |   5 --
 lib/eal/version.map                   |   2 +-
 lib/eal/windows/include/pthread.h     | 105 ++++++++++++++++++++++++++
 lib/eal/windows/include/rte_windows.h |   1 +
 7 files changed, 120 insertions(+), 13 deletions(-)
drivers/net/ena/base/ena_com.c +2 -2
index 5ca36ab6d9..587e7b6ed1 100644
@@ -1375,11 +1375,11 @@ int ena_com_execute_admin_command(struct ena_com_admin_queue *admin_queue,
 	if (IS_ERR(comp_ctx)) {
 		if (comp_ctx == ERR_PTR(ENA_COM_NO_DEVICE))
 			ena_trc_dbg(admin_queue->ena_dev,
-				    "Failed to submit command [%ld]\n",
+				    "Failed to submit command [%" PRIu64 "]\n",
 				    PTR_ERR(comp_ctx));
 		else
 			ena_trc_err(admin_queue->ena_dev,
-				    "Failed to submit command [%ld]\n",
+				    "Failed to submit command [%" PRIu64 "]\n",
 				    PTR_ERR(comp_ctx));
 
 		return (int)PTR_ERR(comp_ctx);
drivers/net/ena/base/ena_plat.h +1 -1
index 2583823080..555cf23e5e 100644
@@ -21,7 +21,7 @@
 #include <ena_plat_dpdk.h>
 #endif
 #elif defined(_WIN32)
-#include <ena_plat_windows.h>
+#include <ena_plat_dpdk.h>
 #else
 #error "Invalid platform"
 #endif
drivers/net/ena/base/ena_plat_dpdk.h +10 -4
index 4e7f52881a..bba61d7180 100644
@@ -13,6 +13,7 @@
 #include <inttypes.h>
 #include <string.h>
 #include <errno.h>
+#include <pthread.h>
 
 #include <rte_atomic.h>
 #include <rte_branch_prediction.h>
@@ -24,7 +25,12 @@
 #include <rte_prefetch.h>
 #include <rte_spinlock.h>
 
+#if defined(_WIN32)
+#include <rte_time.h>
+#include <rte_thread.h>
+#else
 #include <sys/time.h>
+#endif
 #include <rte_memcpy.h>
 
 typedef uint64_t u64;
@@ -293,9 +299,9 @@ extern rte_atomic64_t ena_alloc_cnt;
 #define dma_rmb() rmb()
 
 #define MAX_ERRNO       4095
-#define IS_ERR(x) (((unsigned long)x) >= (unsigned long)-MAX_ERRNO)
-#define ERR_PTR(error) ((void *)(long)error)
-#define PTR_ERR(error) ((long)(void *)error)
+#define IS_ERR(x) (((uintptr_t)x) >= (uintptr_t)-MAX_ERRNO)
+#define ERR_PTR(error) ((void *)(uintptr_t)error)
+#define PTR_ERR(error) ((uintptr_t)(void *)error)
 #define might_sleep()
 
 #define prefetch(x) rte_prefetch0(x)
@@ -322,7 +328,7 @@ extern rte_atomic64_t ena_alloc_cnt;
 
 #define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
 
-#define ENA_FFS(x) ffs(x)
+#define ENA_FFS(x) __builtin_ffs(x)
 
 void ena_rss_key_fill(void *key, size_t size);
 
drivers/net/ena/meson.build +0 -5
index d02ed3f64f..57f8050ccb 100644
@@ -1,11 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-if is_windows
-    build = false
-    reason = 'not supported on Windows'
-    subdir_done()
-endif
 
 sources = files(
         'ena_ethdev.c',
lib/eal/version.map +1 -1
index 887012d02a..3de59910cd 100644
@@ -198,7 +198,7 @@ DPDK_21 {
 	rte_uuid_is_null; # WINDOWS_NO_EXPORT
 	rte_uuid_parse; # WINDOWS_NO_EXPORT
 	rte_uuid_unparse; # WINDOWS_NO_EXPORT
-	rte_version; # WINDOWS_NO_EXPORT
+	rte_version;
 	rte_vfio_clear_group; # WINDOWS_NO_EXPORT
 	rte_vfio_container_create; # WINDOWS_NO_EXPORT
 	rte_vfio_container_destroy; # WINDOWS_NO_EXPORT
lib/eal/windows/include/pthread.h +105 -0

123 lines not on this page · raw message

lib/eal/windows/include/rte_windows.h +1 -0
index 0063b5d78c..3240970ab7 100644
@@ -31,6 +31,7 @@
 #include <psapi.h>
 #include <setupapi.h>
 #include <winioctl.h>
+#include <winsock2.h>
 
 /* Have GUIDs defined. */
 #ifndef INITGUID
Signature · 1 line
2.30.2
#1 lore raw

Hi William,

2021-08-14 03:36 (UTC+0000), William Tu:

Quoted 6 lines

I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver.
I want to know if this is the right direction,
or whether I also need to change other places in ENA driver?

Cc'ing maintainers.

Quoted 2 lines

I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/

We don't want to add more pthread shims, pthread.h in DPDK will go away. Condition variable support should be added to the new threading API:

	http://inbox.dpdk.org/dev/[email protected]

I suppose it can be done independently of the new threading API series. When copying code to DPDK from elsewhere, please adapt its style to DPDK conventions (e.g. RTE_UNUSED(foo) instead if (void)foo) and mind the license if big pieces are copied verbatim (this is not the case here).

Quoted 1 line

https://stackoverflow.com/questions/10905892/equivalent-of-gettimeday-for-windows

POSIX gettimeofday() should be replaced with standard C timespec_get().

Quoted 12 lines

Thanks.

---
 drivers/net/ena/base/ena_com.c        |   4 +-
 drivers/net/ena/base/ena_plat.h       |   2 +-
 drivers/net/ena/base/ena_plat_dpdk.h  |  14 +++-
 drivers/net/ena/meson.build           |   5 --
 lib/eal/version.map                   |   2 +-
 lib/eal/windows/include/pthread.h     | 105 ++++++++++++++++++++++++++
 lib/eal/windows/include/rte_windows.h |   1 +
 7 files changed, 120 insertions(+), 13 deletions(-)

This should be a series of two patches:
1) adding condition variable wrappers to EAL;
2) supporting net/ena for Windows.

author
#2 lore raw
Quoted 16 lines · Dmitry Kozlyuk

Hi William,

2021-08-14 03:36 (UTC+0000), William Tu:

Quoted 6 lines

I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver.
I want to know if this is the right direction,
or whether I also need to change other places in ENA driver?

Cc'ing maintainers.

Quoted 2 lines

I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/

Quoted 6 lines

We don't want to add more pthread shims, pthread.h in DPDK will go away. Condition variable support should be added to the new threading API:

        http://inbox.dpdk.org/dev/[email protected]

I suppose it can be done independently of the new threading API series.

yes, I think so.

Quoted 21 lines

When copying code to DPDK from elsewhere, please adapt its style to DPDK conventions (e.g. RTE_UNUSED(foo) instead if (void)foo) and mind the license if big pieces are copied verbatim (this is not the case here).

Quoted 1 line

https://stackoverflow.com/questions/10905892/equivalent-of-gettimeday-for-windows

POSIX gettimeofday() should be replaced with standard C timespec_get().

Quoted 12 lines

Thanks.

---
 drivers/net/ena/base/ena_com.c        |   4 +-
 drivers/net/ena/base/ena_plat.h       |   2 +-
 drivers/net/ena/base/ena_plat_dpdk.h  |  14 +++-
 drivers/net/ena/meson.build           |   5 --
 lib/eal/version.map                   |   2 +-
 lib/eal/windows/include/pthread.h     | 105 ++++++++++++++++++++++++++
 lib/eal/windows/include/rte_windows.h |   1 +
 7 files changed, 120 insertions(+), 13 deletions(-)
Quoted 4 lines

This should be a series of two patches:
1) adding condition variable wrappers to EAL;
2) supporting net/ena for Windows.

Thanks Dmitry! Will work on v2.
William

William

author
#3 lore raw
In reply to Dmitry Kozlyuk
Quoted 43 lines · Dmitry Kozlyuk

Hi William,

2021-08-14 03:36 (UTC+0000), William Tu:

Quoted 6 lines

I don't have a physical Windows testbed so I want to see if I can get virtual nic working, in this case ENA driver on AWS. The patch passes build on Windows, but I haven't tested loading the ena driver.
I want to know if this is the right direction,
or whether I also need to change other places in ENA driver?

Cc'ing maintainers.

Quoted 2 lines

I copy some of the pthread code from
https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/

We don't want to add more pthread shims, pthread.h in DPDK will go away. Condition variable support should be added to the new threading API:

        http://inbox.dpdk.org/dev/[email protected]

I suppose it can be done independently of the new threading API series. When copying code to DPDK from elsewhere, please adapt its style to DPDK conventions (e.g. RTE_UNUSED(foo) instead if (void)foo) and mind the license if big pieces are copied verbatim (this is not the case here).

Quoted 1 line

https://stackoverflow.com/questions/10905892/equivalent-of-gettimeday-for-windows

POSIX gettimeofday() should be replaced with standard C timespec_get().

Quoted 12 lines

Thanks.

---
 drivers/net/ena/base/ena_com.c        |   4 +-
 drivers/net/ena/base/ena_plat.h       |   2 +-
 drivers/net/ena/base/ena_plat_dpdk.h  |  14 +++-
 drivers/net/ena/meson.build           |   5 --
 lib/eal/version.map                   |   2 +-
 lib/eal/windows/include/pthread.h     | 105 ++++++++++++++++++++++++++
 lib/eal/windows/include/rte_windows.h |   1 +
 7 files changed, 120 insertions(+), 13 deletions(-)
Quoted 4 lines

This should be a series of two patches:
1) adding condition variable wrappers to EAL;
2) supporting net/ena for Windows.

So I finally set up everything on AWS and tested this patch. Virt2phys and netuio (I added ena conf) load OK.
Finally, starting dpdk-testpmd, I got some errors below:
---
PS C:\dpdk-kmods> cd c:\dpdk
PS C:\dpdk> .\build\app\dpdk-testpmd.exe
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process support is requested, but not available.
EAL: WARNING: TSC frequency estimated roughly - clock timings may be less accurate.
EAL: Requested device 0000:00:05.0 cannot be used
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket 0)
EAL: eth_ena_pci_probe[ENA_COM:
ena_com_wait_and_process_admin_cq_interrupts]Invalid wait event. pthread ret: 1
[ENA_COM: ena_com_wait_and_process_admin_cq_interrupts]The ena device sent a completion but the driver didn
't receive a MSI-X interrupt (cmd 9), autopolling mode is OFF [ENA_COM: ena_com_set_dev_mtu]Failed to set mtu 1500. error: -137 ena_mtu_set(): Could not set MTU: 1500
Failed to set MTU to 1500 for port 0
testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc

Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.

Configuring Port 0 (socket 0)
[ENA_COM: ena_com_create_io_cq]Failed to create IO CQ. error: -19 ena_create_io_queue(): Failed to create IO queue[0] (qid:1), rc: -19 ena_queue_start(): Failed to create IO queue ena_queue_start_all(): Failed to start queue[0] of type(1) Fail to start port 0: No such device
Please stop the ports first
Done
Error during enabling promiscuous mode for port 0: Unknown error - ignore No commandline core given, start packet forwarding
Not all ports were started
---
I will see if I can fix these issues and I will submit v2 patch. Thanks
William

#4 lore raw

czw., 19 sie 2021 o 04:19 William Tu <[email protected]> napisał(a):

Quoted 29 lines

So I finally set up everything on AWS and tested this patch. Virt2phys and netuio (I added ena conf) load OK.
Finally, starting dpdk-testpmd, I got some errors below:
---
PS C:\dpdk-kmods> cd c:\dpdk
PS C:\dpdk> .\build\app\dpdk-testpmd.exe
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process support is requested, but not available.
EAL: WARNING: TSC frequency estimated roughly - clock timings may be less accurate.
EAL: Requested device 0000:00:05.0 cannot be used
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket 0)
EAL: eth_ena_pci_probe[ENA_COM:
ena_com_wait_and_process_admin_cq_interrupts]Invalid wait event. pthread ret: 1
[ENA_COM: ena_com_wait_and_process_admin_cq_interrupts]The ena device sent a completion but the driver didn
't receive a MSI-X interrupt (cmd 9), autopolling mode is OFF [ENA_COM: ena_com_set_dev_mtu]Failed to set mtu 1500. error: -137 ena_mtu_set(): Could not set MTU: 1500
Failed to set MTU to 1500 for port 0
testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc

Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.

Quoted 14 lines

Configuring Port 0 (socket 0)
[ENA_COM: ena_com_create_io_cq]Failed to create IO CQ. error: -19 ena_create_io_queue(): Failed to create IO queue[0] (qid:1), rc: -19 ena_queue_start(): Failed to create IO queue ena_queue_start_all(): Failed to start queue[0] of type(1) Fail to start port 0: No such device
Please stop the ports first
Done
Error during enabling promiscuous mode for port 0: Unknown error - ignore No commandline core given, start packet forwarding
Not all ports were started
---
I will see if I can fix these issues and I will submit v2 patch. Thanks

Hi William,

It's great to hear that you're working on ENA support for Windows!

ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.

It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.

Thanks,
Michal

Quoted 1 line

William

author
#5 lore raw

On Mon, Aug 30, 2021 at 12:12 AM Michał Krawczyk <[email protected]> wrote:
[...]

Quoted 9 lines

Hi William,

It's great to hear that you're working on ENA support for Windows!

ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.

Quoted 4 lines

It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.

Hi Michal,
Thank you! Then I will wait for netuio support.
William

#6 lore raw
Quoted 11 lines · William Tu

On Mon, Aug 30, 2021 at 12:12 AM Michał Krawczyk <[email protected]> wrote:
[...]

Quoted 9 lines

Hi William,

It's great to hear that you're working on ENA support for Windows!

ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.

Quoted 8 lines
Quoted 4 lines

It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.

Hi Michal,
Thank you! Then I will wait for netuio support.
William

Hi William, Michał,

This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?

#7 lore raw

sob., 21 maj 2022 o 00:08 Ferruh Yigit <[email protected]> napisał(a):

Quoted 23 lines · William Tu
Quoted 11 lines · Michał Krawczyk

[...]

Quoted 9 lines

Hi William,

It's great to hear that you're working on ENA support for Windows!

ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.

Quoted 8 lines
Quoted 4 lines

It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.

Hi Michal,
Thank you! Then I will wait for netuio support.
William

Quoted 4 lines

Hi William, Michał,

This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?

Hi Ferruh,

sorry for the late reply - nothing new from my side.

I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.

Thanks,
Michal

#8 lore raw
Quoted 29 lines · Michał Krawczyk

sob., 21 maj 2022 o 00:08 Ferruh Yigit <[email protected]> napisał(a):

Quoted 23 lines · William Tu
Quoted 11 lines · Michał Krawczyk

[...]

Quoted 9 lines

Hi William,

It's great to hear that you're working on ENA support for Windows!

ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.

Quoted 8 lines
Quoted 4 lines

It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.

Hi Michal,
Thank you! Then I will wait for netuio support.
William

Quoted 4 lines

Hi William, Michał,

This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?

Quoted 8 lines

Hi Ferruh,

sorry for the late reply - nothing new from my side.

I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.

Hi William, Dmitry,

Is there any update on the netuio interrupt support? What is the plan for this patch?

#9 lore raw

2023-03-23 13:13 (UTC+0000), Ferruh Yigit:

Quoted 39 lines · Michał Krawczyk
Quoted 29 lines

sob., 21 maj 2022 o 00:08 Ferruh Yigit <[email protected]> napisał(a):

Quoted 23 lines · William Tu
Quoted 11 lines · Michał Krawczyk

[...]

Quoted 9 lines

Hi William,

It's great to hear that you're working on ENA support for Windows!

ENA PMD uses admin interrupt for processing all the commands like creating the IO queues, setting up the MTU, etc., and also for the AENQ events. With the current driver design it's critical to have this admin interrupt working.

Quoted 8 lines
Quoted 4 lines

It looks like the admin interrupt is not functional and from what I've seen in the email regarding the v21.11 roadmap for the Windows support, the netuio interrupt support is going to be added in the future. That might be the reason for you seeing those errors.

Hi Michal,
Thank you! Then I will wait for netuio support.
William

Quoted 4 lines

Hi William, Michał,

This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?

Quoted 8 lines

Hi Ferruh,

sorry for the late reply - nothing new from my side.

I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.

Quoted 4 lines

Hi William, Dmitry,

Is there any update on the netuio interrupt support? What is the plan for this patch?

Hi Ferruh,

The work on the interrupt support has been abandoned unfortunately. I'd like to complete it one day, but can't make a commitment right now.

Ref:
http://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

#10 lore raw
Quoted 17 lines · On Thu, 23 Mar 2023 17:19:55 +0300 Dmitry Kozlyuk <[email protected]>…
Quoted 12 lines
Quoted 11 lines
Quoted 2 lines

This is a very old thread, but still in the patchwork, I wonder if is there any update on the issue?

Hi Ferruh,

sorry for the late reply - nothing new from my side.

I'm not sure what's the current state of the netuio interrupt support for the Windows, but if it still didn't land, then the original issue still persists.

Quoted 4 lines

Hi William, Dmitry,

Is there any update on the netuio interrupt support? What is the plan for this patch?

Quoted 7 lines

Hi Ferruh,

The work on the interrupt support has been abandoned unfortunately. I'd like to complete it one day, but can't make a commitment right now.

Ref:
http://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

Marking this patch as "Awaiting upstream" since it depends on interrupt support in Windows which is not available.