Windows community call: MoM 2021-08-04

# About

The meeting takes place in MS Teams every two weeks on Wednesday 15:00 UTC. Ask Harini Ramakrishnan <[email protected]> for invitation.

# Attendees

* Microsoft:
- Khoa To
- Narcisa Ana Maria Vasile (Naty)
- Omar Cordona
- Tyler Retzlaff
* NVIDIA:
- Dmitry Kozlyuk (DmitryK)
- Tal Shnaiderman
* VMWare:
- Cheng-Chung William Tu (VMWare)
- Sergey Madaminov (VMWare)
* Mark Cheatham (Boulder Imaging)
* Nick Connolly (Datapath)
* Pallavi Kadam (Intel)
* Yan Vugenfirer (Daynix)

# Agenda

* Patch status review
* Porting OvS build system to meson status report
* Windows 21.11 roadmap planning
* Misc

# Patch review

1. eal: Add EAL API for threading v13

DmitryK to finish the review and ack.
Naty to follow up with Thomas if the series can be merged without the next one once acked because the code is used in unit tests at least.

2. Enable the internal EAL thread API v2

Naty to send v3.

3. [v2] eal/windows: ensure all enabled CPUs are counted v2

DmitryK acked, but suggested a shorted working. Naty to track the patch (maybe send v3 following suggestion).

4. windows/virt2phys: fix paging issue v2 (DmitryK)

Tyler to do a security-centered review and/or ping DmitryM.

# Porting OvS build system to meson (William Tu)

Status: OvS compiles with some features disabled, with a lot of warnings. Issues:

* vhost-user is Linux-specific.
[Omar] Microsoft is working on functional equivalent.
* rte_version* not exported.
AI William to send patches.
* rte_open_logstream() implementation relies on Linux-specific fopencookie(). We need a more generic facility to redirect logs.
AI William and DmitryK to discuss.
* meson not finding DPDK with pkg-config, maybe meson bug.
AI William and DmitryK to investigate.

[Omar] What are the use cases for OvS on Windows? [William]:
1. VMWare NSX on Windows;
2. AF_XDP replacement (fast data path); [Omar] Windows now has its own experimental AF_XDP
3. Kubernetes containers scenario.

# Windows 21.11 roadmap

AI DmitryK to send a roadmap patch, below are brief notes.

1. Harini, Omar, and Tyler will work on establishing the process of signing and publishing netuio and virt2phys. Likely some form of external signing will be used, i.e. not by Microsoft name. Audit before signing must be aligned with DPDK releases. CI for signing is also currently missing. Microsoft will discuss the topic externally and reach more people of needed.

2. DmitryK will work on interrupt support in DPDK and netuio. Intel will help with testing on their HW. Microsoft will help with code review.

3. Naty will finish the work on threading API.
DmitryK to track them and review on time.

4. Tyler will work on enabling shared build of DPDK, mostly solving the issues with thread-local storage. At least patches will be sent by 21.11, not sure if merged. DmitryK will help with review.

6. DmitryK will replace SetupAPI with cfgmgr32 API in lib/eal and bus/pci.

7. Harini will expedite investigation by Microsoft, why netuio doesn't work with vmxnet3 HW in VMWare hypervisor.

8. External issues to track:

8.1) wpcap lacks pkg-config file:

     https://github.com/nmap/npcap/issues/299
8.2) meson 0.58 unable to build DPDK:
     https://github.com/mesonbuild/meson/issues/8981

# Misc

* We need to better automation scripts for things in Windows GSG:
- to setup development environment;
- to enable "Lock pages in memory privilege";
- to install drivers.

* netuio needs tracing (logging), maybe it will be WPP tracing, maybe Microsoft will advise a better technology.

snip

To give more details about this for people who are interested. At OVS side, we tried to link the DPDK library, by doing below at meson.build file
libdpdk = dependency('libdpdk', method: 'pkg-config')" , or give
it a specific path

    libdpdk = cc.find_library('dpdk', dirs: ['C:\\temp\\dpdk\\lib'])

However, it doesn't work, with error below
Run-time dependency libdpdk found: NO (tried pkgconfig) meson.build:45:4: ERROR: Dependency "libdpdk" not found, tried pkgconfig

On the other hand, pkg-config seems OK for locating the dpdk library. PS C:\k8s-antrea-dpdk-win\ovs> pkg-config --list-all
libdpdk DPDK - The Data Plane Development Kit (DPDK).
libdpdk-libs dpdk-libs - Internal-only DPDK pkgconfig file. Not for direct use.

We are using meson 0.59.0 and pkg-config-lite 0.28
Currently we can work around it by linking the rte_xxx.lib one-by-one.
rte_eal_lib = cc.find_library('rte_eal', dirs: ['C:\\temp\\dpdk\\lib']) rte_ethdev_lib = cc.find_library('rte_ethdev', dirs:
['C:\\temp\\dpdk\\lib'])

Regards,
William

Can you share the meson log file snippet where it tries the pkg-config call? Does PKG_CONFIG_PATH have to be set to a special value to get the .pc files found?

For the "find_library" calls, that would not be expected to work since there is no single "libdpdk.so" file.

By any chance are you looking to investigate meson subproject support in future? [1] It should provide an easy way to provide DPDK with OVS for systems without it already installed, but may require some small changes to DPDK meson files to work neatly. I've always meant to investigate how to use it for applications using DPDK but never actually got around to it.

/Bruce

[1] https://mesonbuild.com/Subprojects.html

Hi Bruce,
Thanks!
The log below:
---
Target machine cpu: x86_64
Pkg-config binary for MachineChoice.HOST is not cached.
Pkg-config binary missing from cross or native file, or env var undefined. Trying a default Pkg-config fallback at pkg-config
Found pkg-config:
C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0 .28-1\bin\pkg-config.EXE (0.28)
Determining dependency 'libdpdk' with pkg-config executable 'C:\\ProgramData\\chocola tey\\lib\\pkgconfiglite\\tools\\pkg-config-lite-0.28-1\\bin\\pkg-config.EXE' env[PKG_CONFIG_PATH]:
Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\
pkg-config.EXE --modversion libdpdk` -> 1

Run-time dependency libdpdk found: NO (tried pkgconfig)
---
However, I copy the command and run it on powershell PS C:\k8s-antrea-dpdk-win\ovs> C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg- config-lite-0.28-1\bin\pkg-config.EXE --modversion libdpdk 21.08.0-rc1

Yes, for example:
$env:PKG_CONFIG_PATH = 'C:\temp\DPDK\lib\pkgconfig'

Sure, I'm new to Meson, will take a look at subproject support. Thanks
William

Looking at the meson log, it appears that the PKG_CONFIG_PATH is empty for the meson call to pkg-config "env[PKG_CONFIG_PATH]:", which may explain why DPDK is not found. I suspect meson only uses the PKG_CONFIG_PATH from the environment on first run, but beyond that you can configure a PKG_CONFIG_PATH using a meson configuration option "-Dpkg_config_path". Can you try explicitly setting that in your build and see if it fixes it for you?

/Bruce

[...]

Hi Bruce,

It works, thanks a lot! I did
PS C:\k8s-antrea-dpdk-win\ovs> meson --reconfigure build -Dpkg_config_path=C:\temp\dpdk\lib\pkgconfig\
Log below, if you're interested
----
The Meson build system
Version: 0.59.1
...
Pkg-config binary for MachineChoice.HOST is not cached.
Pkg-config binary missing from cross or native file, or env var undefined.
Trying a default Pkg-config fallback at pkg-config
Found pkg-config:
C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\pkg-config.EXE (0.28)
Determining dependency 'libdpdk' with pkg-config executable 'C:\\ProgramData\\chocolatey\\lib\\pkgconfiglit e\\tools\\pkg-config-lite-0.28-1\\bin\\pkg-config.EXE'
env[PKG_CONFIG_PATH]: C:/temp/dpdk/lib/pkgconfig
Called C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\pkg-config.EXE --modversion libdpdk -> 0
21.08.0-rc1
env[PKG_CONFIG_PATH]: C:/temp/dpdk/lib/pkgconfig
Called `C:\ProgramData\chocolatey\lib\pkgconfiglite\tools\pkg-config-lite-0.28-1\bin\pkg-config.EXE --cflag
s libdpdk` -> 0
-include rte_config.h -march=native -IC:/temp/dpdk/include env[PKG_CONFIG_ALLOW_SYSTEM_LIBS]: 1
env[PKG_CONFIG_PATH]: C:/temp/dpdk/lib/pkgconfig
---
William