Skip to contentSkip to conversation

Split logging out of EAL

RFC PATCH Superseded Bruce Richardson 5 participants 58 messages Aug 29, 2022 – Oct 23, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Split logging out of EAL

Following recent discussion on-list about EAL needing to be broken down a bit, here is an RFC where logging functionality is split out of EAL into a separate library. Most parts of this work is fairly straight forward - there are only two complications:

1. The logging functions use "fnmatch", which is not available on windows, and so has a compatibility fallback in EAL.

2. There are individual logging files for each supported OS.

For #1, there were really two options to avoid the circular dependency - either move fnmatch into the log library, or to create a new lower-level library for such back function fallbacks. For this RFC I've taken the second option as a better solution. Ideally, more of EAL compat functions should be moved to such a library if we create one, but for now, it's only the one function that was needed to be moved.

For #2, this was fixed using standard naming and the build system to only build the appropriately named file for each OS. The alternative of creating a subdir per-OS seems overkill for the single-file situation.

NOTE: this is an early RFC based on work I did some time back, and is intended just to inspire further discussion and work about splitting EAL, more than necessarily being a patchset for future merging.

Bruce Richardson (3):
os: begin separating some OS compatibility from EAL
log: separate logging functions out of EAL
telemetry: use standard logging

 lib/eal/common/eal_private.h                  |  7 ----
 lib/eal/common/meson.build                    |  1 -
 lib/eal/freebsd/eal.c                         |  6 +--
 lib/eal/include/meson.build                   |  1 -
 lib/eal/linux/eal.c                           |  6 +--
 lib/eal/linux/meson.build                     |  1 -
 lib/eal/meson.build                           |  2 +-
 lib/eal/version.map                           | 17 --------
 lib/eal/windows/meson.build                   |  2 -
 lib/kvargs/meson.build                        |  3 +-
 lib/{eal/common => log}/eal_common_log.c      |  1 -
 lib/{eal/common => log}/eal_log.h             | 11 ++++++
 .../linux/eal_log.c => log/eal_log_linux.c}   |  0
 .../eal_log.c => log/eal_log_windows.c}       |  0
 lib/log/meson.build                           |  8 ++++
 lib/{eal/include => log}/rte_log.h            |  0
 lib/log/version.map                           | 39 +++++++++++++++++++
 lib/meson.build                               | 12 +++---
 lib/os/freebsd/fnmatch.c                      |  3 ++
 lib/os/linux/fnmatch.c                        |  3 ++
 lib/os/meson.build                            |  8 ++++
 lib/os/os.c                                   |  3 ++
 lib/os/version.map                            |  7 ++++
 lib/{eal => os}/windows/fnmatch.c             |  0
 .../windows/include => os/windows}/fnmatch.h  |  0
 lib/telemetry/meson.build                     |  3 +-
 lib/telemetry/telemetry.c                     | 12 +++---
 lib/telemetry/telemetry_internal.h            |  3 +-
 28 files changed, 100 insertions(+), 59 deletions(-)
 rename lib/{eal/common => log}/eal_common_log.c (99%)
 rename lib/{eal/common => log}/eal_log.h (80%)
 rename lib/{eal/linux/eal_log.c => log/eal_log_linux.c} (100%)
 rename lib/{eal/windows/eal_log.c => log/eal_log_windows.c} (100%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
 create mode 100644 lib/os/freebsd/fnmatch.c
 create mode 100644 lib/os/linux/fnmatch.c
 create mode 100644 lib/os/meson.build
 create mode 100644 lib/os/os.c
 create mode 100644 lib/os/version.map
 rename lib/{eal => os}/windows/fnmatch.c (100%)
 rename lib/{eal/windows/include => os/windows}/fnmatch.h (100%)

--
2.34.1

#1 lore raw

RFC PATCH 1/3 os: begin separating some OS compatibility from EAL

Some library functionality we may want ahead of EAL build depends upon some OS-specific functionality, so we create a new lib for that to be built separately. For now, just includes fnmatch function for windows.

---
 lib/eal/windows/meson.build                       |  1 -
 lib/meson.build                                   | 11 ++++++-----
 lib/os/freebsd/fnmatch.c                          |  3 +++
 lib/os/linux/fnmatch.c                            |  3 +++
 lib/os/meson.build                                |  8 ++++++++
 lib/os/os.c                                       |  3 +++
 lib/os/version.map                                |  7 +++++++
 lib/{eal => os}/windows/fnmatch.c                 |  0
 lib/{eal/windows/include => os/windows}/fnmatch.h |  0
 9 files changed, 30 insertions(+), 6 deletions(-)
 create mode 100644 lib/os/freebsd/fnmatch.c
 create mode 100644 lib/os/linux/fnmatch.c
 create mode 100644 lib/os/meson.build
 create mode 100644 lib/os/os.c
 create mode 100644 lib/os/version.map
 rename lib/{eal => os}/windows/fnmatch.c (100%)
 rename lib/{eal/windows/include => os/windows}/fnmatch.h (100%)
lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
lib/meson.build +6 -5
index c648f7d800..7b61b2a5d7 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'os',   # load os compatibility material
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
@@ -106,6 +107,7 @@ if cc.has_argument('-Wno-format-truncation')
 endif
 
 enabled_libs = [] # used to print summary at the end
+default_deps = []
 
 foreach l:libraries
     build = true
@@ -124,11 +126,7 @@ foreach l:libraries
     # use "deps" for internal DPDK dependencies, and "ext_deps" for
     # external package/library requirements
     ext_deps = []
-    deps = []
-    # eal is standard dependency once built
-    if dpdk_conf.has('RTE_LIB_EAL')
-        deps += ['eal']
-    endif
+    deps = default_deps
 
     if disabled_libs.contains(l)
         build = false
@@ -271,4 +269,7 @@ foreach l:libraries
     if developer_mode
         message('lib/@0@: Defining dependency "@1@"'.format(l, name))
     endif
+    if name == 'os' or name == 'eal'
+        default_deps = [name]
+    endif
 endforeach
lib/os/freebsd/fnmatch.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/linux/fnmatch.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/meson.build new file +8 -0
new file mode 100644
index 0000000000..53949ca17e
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+includes += include_directories(exec_env)
+sources += files(
+        exec_env / 'fnmatch.c',
+)
lib/os/os.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/version.map new file +7 -0
new file mode 100644
index 0000000000..e1dbd6051e
@@ -0,0 +1,7 @@
+DPDK_22 {
+	global:
+
+	fnmatch;
+
+	local: *;
+};
lib/os/windows/fnmatch.c renamed from lib/eal/windows/fnmatch.c +0 -0
similarity index 100%
rename from lib/eal/windows/fnmatch.c
rename to lib/os/windows/fnmatch.c
lib/os/windows/fnmatch.h renamed from lib/eal/windows/include/fnmatch.h +0 -0
similarity index 100%
rename from lib/eal/windows/include/fnmatch.h
rename to lib/os/windows/fnmatch.h
Signature · 1 line
2.34.1
#2 lore raw

RE: [RFC PATCH 1/3] os: begin separating some OS compatibility from EAL

Quoted 10 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Monday, 29 August 2022 17.19
To: [email protected]
Cc: Bruce Richardson
Subject: [RFC PATCH 1/3] os: begin separating some OS compatibility from EAL

Some library functionality we may want ahead of EAL build depends upon some OS-specific functionality, so we create a new lib for that to be built separately. For now, just includes fnmatch function for windows.

The description given in patch 0/3 mentions that this causes a circular dependency between the EAL and Log libraries. You should mention that here too. Until I re-read that, I didn't understand the need to move fnmatch() out of the EAL library - I was even sidetracking wildly, considering if it had to do with needing it on the host computer (for some host compiler checks).

FYI, and not important: fnmatch() is a C library function (man 3), not a System call (man 2). But obviously still O/S specific, since it is not included with the C library for Windows.

#3 lore raw

Re: [RFC PATCH 1/3] os: begin separating some OS compatibility from EAL

Quoted 7 lines · Morten Brørup
Quoted 7 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Monday, 29 August 2022 17.19
To: [email protected]
Cc: Bruce Richardson
Subject: [RFC PATCH 1/3] os: begin separating some OS compatibility from EAL

Quoted 6 lines
Quoted 3 lines

Some library functionality we may want ahead of EAL build depends upon some OS-specific functionality, so we create a new lib for that to be built separately. For now, just includes fnmatch function for windows.

The description given in patch 0/3 mentions that this causes a circular dependency between the EAL and Log libraries. You should mention that here too. Until I re-read that, I didn't understand the need to move fnmatch() out of the EAL library - I was even sidetracking wildly, considering if it had to do with needing it on the host computer (for some host compiler checks).

Sorry about that! :-)

Quoted 2 lines

FYI, and not important: fnmatch() is a C library function (man 3), not a System call (man 2). But obviously still O/S specific, since it is not included with the C library for Windows.

Interesting, but as you say, it doesn't actually affect this patch.

/Bruce

#4 lore raw

Re: [RFC PATCH 1/3] os: begin separating some OS compatibility from EAL

In reply to Bruce Richardson
Quoted 102 lines · Bruce Richardson

Some library functionality we may want ahead of EAL build depends upon some OS-specific functionality, so we create a new lib for that to be built separately. For now, just includes fnmatch function for windows.

---
 lib/eal/windows/meson.build                       |  1 -
 lib/meson.build                                   | 11 ++++++-----
 lib/os/freebsd/fnmatch.c                          |  3 +++
 lib/os/linux/fnmatch.c                            |  3 +++
 lib/os/meson.build                                |  8 ++++++++
 lib/os/os.c                                       |  3 +++
 lib/os/version.map                                |  7 +++++++
 lib/{eal => os}/windows/fnmatch.c                 |  0
 lib/{eal/windows/include => os/windows}/fnmatch.h |  0
 9 files changed, 30 insertions(+), 6 deletions(-)
 create mode 100644 lib/os/freebsd/fnmatch.c
 create mode 100644 lib/os/linux/fnmatch.c
 create mode 100644 lib/os/meson.build
 create mode 100644 lib/os/os.c
 create mode 100644 lib/os/version.map
 rename lib/{eal => os}/windows/fnmatch.c (100%)
 rename lib/{eal/windows/include => os/windows}/fnmatch.h (100%)
lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
lib/meson.build +6 -5
index c648f7d800..7b61b2a5d7 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'os',   # load os compatibility material
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
@@ -106,6 +107,7 @@ if cc.has_argument('-Wno-format-truncation')
 endif
 
 enabled_libs = [] # used to print summary at the end
+default_deps = []
 
 foreach l:libraries
     build = true
@@ -124,11 +126,7 @@ foreach l:libraries
     # use "deps" for internal DPDK dependencies, and "ext_deps" for
     # external package/library requirements
     ext_deps = []
-    deps = []
-    # eal is standard dependency once built
-    if dpdk_conf.has('RTE_LIB_EAL')
-        deps += ['eal']
-    endif
+    deps = default_deps
 
     if disabled_libs.contains(l)
         build = false
@@ -271,4 +269,7 @@ foreach l:libraries
     if developer_mode
         message('lib/@0@: Defining dependency "@1@"'.format(l, name))
     endif
+    if name == 'os' or name == 'eal'
+        default_deps = [name]
+    endif
 endforeach
lib/os/freebsd/fnmatch.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/linux/fnmatch.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/meson.build new file +0 -0
new file mode 100644
index 0000000000..53949ca17e

lib/os/meson.build line 8

Quoted 8 lines
lib/os/meson.build new file +8 -0
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+includes += include_directories(exec_env)
+sources += files(
+        exec_env / 'fnmatch.c',
+)

Not really important (that's only a RFC), but os.c is not compiled anywhere.

Quoted 15 lines
lib/os/os.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/version.map new file +0 -0
new file mode 100644
index 0000000000..e1dbd6051e

lib/os/version.map line 7

Quoted 7 lines
lib/os/version.map new file +7 -0
@@ -0,0 +1,7 @@
+DPDK_22 {
+       global:
+
+       fnmatch;
+
+       local: *;
+};

Could we perhaps consider a per-os version.map file or some kind of inclusion of os specific symbols?
That would avoid odd exporting of a symbol that is provided by the C library in other OS.

Quoted 11 lines
lib/os/windows/fnmatch.c renamed from lib/eal/windows/fnmatch.c +0 -0
similarity index 100%
rename from lib/eal/windows/fnmatch.c
rename to lib/os/windows/fnmatch.c
lib/os/windows/fnmatch.h renamed from lib/eal/windows/include/fnmatch.h +0 -1
similarity index 100%
rename from lib/eal/windows/include/fnmatch.h
rename to lib/os/windows/fnmatch.h
--

2.34.1

Signature · 1 line
David Marchand
#5 lore raw

Re: [RFC PATCH 1/3] os: begin separating some OS compatibility from EAL

Quoted 135 lines · David Marchand
Quoted 102 lines · Bruce Richardson

Some library functionality we may want ahead of EAL build depends upon some OS-specific functionality, so we create a new lib for that to be built separately. For now, just includes fnmatch function for windows.

---
 lib/eal/windows/meson.build                       |  1 -
 lib/meson.build                                   | 11 ++++++-----
 lib/os/freebsd/fnmatch.c                          |  3 +++
 lib/os/linux/fnmatch.c                            |  3 +++
 lib/os/meson.build                                |  8 ++++++++
 lib/os/os.c                                       |  3 +++
 lib/os/version.map                                |  7 +++++++
 lib/{eal => os}/windows/fnmatch.c                 |  0
 lib/{eal/windows/include => os/windows}/fnmatch.h |  0
 9 files changed, 30 insertions(+), 6 deletions(-)
 create mode 100644 lib/os/freebsd/fnmatch.c
 create mode 100644 lib/os/linux/fnmatch.c
 create mode 100644 lib/os/meson.build
 create mode 100644 lib/os/os.c
 create mode 100644 lib/os/version.map
 rename lib/{eal => os}/windows/fnmatch.c (100%)
 rename lib/{eal/windows/include => os/windows}/fnmatch.h (100%)
lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
lib/meson.build +6 -5
index c648f7d800..7b61b2a5d7 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'os',   # load os compatibility material
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
@@ -106,6 +107,7 @@ if cc.has_argument('-Wno-format-truncation')
 endif
 
 enabled_libs = [] # used to print summary at the end
+default_deps = []
 
 foreach l:libraries
     build = true
@@ -124,11 +126,7 @@ foreach l:libraries
     # use "deps" for internal DPDK dependencies, and "ext_deps" for
     # external package/library requirements
     ext_deps = []
-    deps = []
-    # eal is standard dependency once built
-    if dpdk_conf.has('RTE_LIB_EAL')
-        deps += ['eal']
-    endif
+    deps = default_deps
 
     if disabled_libs.contains(l)
         build = false
@@ -271,4 +269,7 @@ foreach l:libraries
     if developer_mode
         message('lib/@0@: Defining dependency "@1@"'.format(l, name))
     endif
+    if name == 'os' or name == 'eal'
+        default_deps = [name]
+    endif
 endforeach
lib/os/freebsd/fnmatch.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/linux/fnmatch.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/meson.build new file +0 -0
new file mode 100644
index 0000000000..53949ca17e

lib/os/meson.build line 8

Quoted 8 lines
lib/os/meson.build new file +8 -0
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+includes += include_directories(exec_env)
+sources += files(
+        exec_env / 'fnmatch.c',
+)

Not really important (that's only a RFC), but os.c is not compiled anywhere.

Quoted 19 lines
lib/os/os.c new file +3 -0
new file mode 100644
index 0000000000..ca8a050fda
@@ -0,0 +1,3 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Intel Corporation
+ */
lib/os/version.map new file +4 -0
new file mode 100644
index 0000000000..e1dbd6051e
@@ -0,0 +1,7 @@
+DPDK_22 {
+       global:
+
+       fnmatch;
Quoted 9 lines

lib/os/version.map line 7

Quoted 3 lines
lib/os/version.map new file +3 -0
+
+       local: *;
+};

Could we perhaps consider a per-os version.map file or some kind of inclusion of os specific symbols?
That would avoid odd exporting of a symbol that is provided by the C library in other OS.

Yep, should be possible, especially if we update our minimum meson to 0.53, allowing us to use the fs module to check if particular files exist. That would allow us to check for a version.map.<exec_env> file and use that if it exists, falling back to regular version.map file if not.

Alternatively, we could use a script to create the version.map files, by appending an exec_env-specific version, if one exists, to the main version.map file for each lib.

/Bruce

#6 lore raw

RFC PATCH 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL.

---
 lib/eal/common/eal_private.h                  |  7 ----
 lib/eal/common/meson.build                    |  1 -
 lib/eal/include/meson.build                   |  1 -
 lib/eal/linux/meson.build                     |  1 -
 lib/eal/meson.build                           |  2 +-
 lib/eal/version.map                           | 17 --------
 lib/eal/windows/meson.build                   |  1 -
 lib/kvargs/meson.build                        |  3 +-
 lib/{eal/common => log}/eal_common_log.c      |  1 -
 lib/{eal/common => log}/eal_log.h             | 11 ++++++
 .../linux/eal_log.c => log/eal_log_linux.c}   |  0
 .../eal_log.c => log/eal_log_windows.c}       |  0
 lib/log/meson.build                           |  8 ++++
 lib/{eal/include => log}/rte_log.h            |  0
 lib/log/version.map                           | 39 +++++++++++++++++++
 lib/meson.build                               |  1 +
 lib/telemetry/meson.build                     |  3 +-
 17 files changed, 62 insertions(+), 34 deletions(-)
 rename lib/{eal/common => log}/eal_common_log.c (99%)
 rename lib/{eal/common => log}/eal_log.h (80%)
 rename lib/{eal/linux/eal_log.c => log/eal_log_linux.c} (100%)
 rename lib/{eal/windows/eal_log.c => log/eal_log_windows.c} (100%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
lib/eal/common/eal_private.h +0 -7
index 44d14241f0..734f1f334b 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index fd6e844224..7de7244065 100644
@@ -27,7 +27,6 @@ headers += files(
         'rte_keepalive.h',
         'rte_launch.h',
         'rte_lcore.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/meson.build +0 -1
index 3cccfa36c0..1b913acc06 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 056beb9461..af8c4eae4a 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index 1f293e768b..44e1712155 100644
@@ -133,20 +133,6 @@ DPDK_23 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -218,7 +204,6 @@ DPDK_23 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -291,7 +276,6 @@ DPDK_23 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
@@ -394,7 +378,6 @@ EXPERIMENTAL {
 	# added in 21.05
 	rte_devargs_reset;
 	rte_intr_callback_unregister_sync;
-	rte_log_list_types;
 	rte_thread_key_create;
 	rte_thread_key_delete;
 	rte_thread_value_get;
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/eal_common_log.c renamed from lib/eal/common/eal_common_log.c +0 -1
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/eal_common_log.c
index bd7b188ceb..895c4f2040 100644
@@ -16,7 +16,6 @@
 #include <rte_per_lcore.h>
 
 #include "eal_log.h"
-#include "eal_private.h"
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/eal_log.h renamed from lib/eal/common/eal_log.h +11 -0
similarity index 80%
rename from lib/eal/common/eal_log.h
rename to lib/log/eal_log.h
index c784fa6043..5baf19df59 100644
@@ -11,22 +11,33 @@
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
 #endif /* EAL_LOG_H */
lib/log/eal_log_linux.c renamed from lib/eal/linux/eal_log.c +0 -0
similarity index 100%
rename from lib/eal/linux/eal_log.c
rename to lib/log/eal_log_linux.c
lib/log/eal_log_windows.c renamed from lib/eal/windows/eal_log.c +0 -0
similarity index 100%
rename from lib/eal/windows/eal_log.c
rename to lib/log/eal_log_windows.c
lib/log/meson.build new file +8 -0
new file mode 100644
index 0000000000..02899c47b1
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+sources = files(
+        'eal_common_log.c',
+        'eal_log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +39 -0
new file mode 100644
index 0000000000..0ae3b7e55f
@@ -0,0 +1,39 @@
+DPDK_22 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+EXPERIMENTAL {
+	global:
+
+	rte_log_list_types;  # added 21.05
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index 7b61b2a5d7..1b8419d337 100644
@@ -10,6 +10,7 @@
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
         'os',   # load os compatibility material
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.34.1
#7 lore raw

RFC PATCH 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 12 +++++-------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 8 insertions(+), 19 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index 26fbc91b26..154ed65da6 100644
@@ -871,13 +871,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 37d29643a5..8d968f6e5c 100644
@@ -1312,13 +1312,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +5 -7
index c6fd03a5ab..83d73ac517 100644
@@ -51,11 +51,10 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
+extern int logtype;
 
 #define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+        rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -606,14 +605,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
@@ -624,3 +620,5 @@ rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_
 
 	return 0;
 }
+
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
lib/telemetry/telemetry_internal.h +1 -2
index d085c492dc..5c75d73183 100644
@@ -109,7 +109,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.34.1
#8 lore raw

v2 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There seems to be a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the previous RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, this v2 RFC takes a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   6 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/fnmatch.c                     | 172 -----------------
 lib/eal/windows/include/fnmatch.h             | 175 ++++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 lib/{eal/common => log}/eal_common_log.c      |   1 -
 lib/{eal/common => log}/eal_log.h             |  12 ++
 .../linux/eal_log.c => log/eal_log_linux.c}   |   0
 .../eal_log.c => log/eal_log_windows.c}       |   0
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  12 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 23 files changed, 229 insertions(+), 239 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common => log}/eal_common_log.c (99%)
 rename lib/{eal/common => log}/eal_log.h (78%)
 rename lib/{eal/linux/eal_log.c => log/eal_log_linux.c} (100%)
 rename lib/{eal/windows/eal_log.c => log/eal_log_windows.c} (100%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.34.1

#9 lore raw

RFC PATCH v2 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 -----------------------------
 lib/eal/windows/include/fnmatch.h | 175 +++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 162 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +162 -13

198 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.34.1
#10 lore raw

RE: [RFC PATCH v2 1/3] eal/windows: move fnmatch function to header file

Quoted 12 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Friday, 13 January 2023 17.20

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---

[...]

Quoted 4 lines

#define FNM_CASEFOLD 0x10
#define FNM_PREFIX_DIRS 0x20

+#define EOS '\0'

Careful about names in header files. Perhaps EOS should also have the FNM_ name space prefix to reduce the risk of collision. Or even better: just use '\0' in the code instead of defining a special name for it.

lib/eal/windows/include/fnmatch.h extern "C" { · line 35

Quoted 2 lines
lib/eal/windows/include/fnmatch.h +2 -0
+
+static const char *rangematch(const char *, char, int);

I don't think rangematch() is a POSIX function, so similar comment here. Prefix with fnm_ to reduce risk of collision.

With those fixes...

#11 lore raw

Re: [RFC PATCH v2 1/3] eal/windows: move fnmatch function to header file

Quoted 22 lines · Morten Brørup
Quoted 12 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Friday, 13 January 2023 17.20

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---

[...]

Quoted 4 lines

#define FNM_CASEFOLD 0x10
#define FNM_PREFIX_DIRS 0x20

+#define EOS '\0'

Careful about names in header files. Perhaps EOS should also have the FNM_ name space prefix to reduce the risk of collision. Or even better: just use '\0' in the code instead of defining a special name for it.

Quoted 9 lines

lib/eal/windows/include/fnmatch.h extern "C" { · line 35

Quoted 2 lines
lib/eal/windows/include/fnmatch.h +2 -0
+
+static const char *rangematch(const char *, char, int);

I don't think rangematch() is a POSIX function, so similar comment here. Prefix with fnm_ to reduce risk of collision.

With those fixes...

Sure, I can prefix those.

However, since this is a non-published header, private to the DPDK build, the only chances of collision come from the DPDK files which include the header. The objective of making it a header was so that none of this ever leaks outside of the DPDK libs that use the functions internally. That said, there is no harm in prefixing either, so I'll do so in any future version.

#12 lore raw

Re: [RFC PATCH v2 1/3] eal/windows: move fnmatch function to header file

Quoted 33 lines · Bruce Richardson
Quoted 22 lines · Morten Brørup
Quoted 12 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Friday, 13 January 2023 17.20

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---

[...]

Quoted 4 lines

#define FNM_CASEFOLD 0x10
#define FNM_PREFIX_DIRS 0x20

+#define EOS '\0'

Careful about names in header files. Perhaps EOS should also have the FNM_ name space prefix to reduce the risk of collision. Or even better: just use '\0' in the code instead of defining a special name for it.

Quoted 9 lines

lib/eal/windows/include/fnmatch.h extern "C" { · line 35

Quoted 2 lines
lib/eal/windows/include/fnmatch.h +2 -0
+
+static const char *rangematch(const char *, char, int);

I don't think rangematch() is a POSIX function, so similar comment here. Prefix with fnm_ to reduce risk of collision.

With those fixes...

Quoted 8 lines

Sure, I can prefix those.

However, since this is a non-published header, private to the DPDK build, the only chances of collision come from the DPDK files which include the header. The objective of making it a header was so that none of this ever leaks outside of the DPDK libs that use the functions internally. That said, there is no harm in prefixing either, so I'll do so in any future version.

and if a future collision occurs we should be able to adapt without compat break. you know for platforms that step on the application namespace... *cough* legacy windows headers.

thank you for doing this Bruce.

#13 lore raw

Re: [RFC PATCH v2 1/3] eal/windows: move fnmatch function to header file

Quoted 43 lines · Tyler Retzlaff
Quoted 33 lines · Bruce Richardson
Quoted 22 lines · Morten Brørup
Quoted 12 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Friday, 13 January 2023 17.20

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---

[...]

Quoted 4 lines

#define FNM_CASEFOLD 0x10
#define FNM_PREFIX_DIRS 0x20

+#define EOS '\0'

Careful about names in header files. Perhaps EOS should also have the FNM_ name space prefix to reduce the risk of collision. Or even better: just use '\0' in the code instead of defining a special name for it.

Quoted 9 lines

lib/eal/windows/include/fnmatch.h extern "C" { · line 35

Quoted 2 lines
lib/eal/windows/include/fnmatch.h +2 -0
+
+static const char *rangematch(const char *, char, int);

I don't think rangematch() is a POSIX function, so similar comment here. Prefix with fnm_ to reduce risk of collision.

With those fixes...

Quoted 8 lines

Sure, I can prefix those.

However, since this is a non-published header, private to the DPDK build, the only chances of collision come from the DPDK files which include the header. The objective of making it a header was so that none of this ever leaks outside of the DPDK libs that use the functions internally. That said, there is no harm in prefixing either, so I'll do so in any future version.

Quoted 7 lines

and if a future collision occurs we should be able to adapt without compat break. you know for platforms that step on the application namespace... *cough* legacy windows headers.

thank you for doing this Bruce.

Thanks.
Seems there is some support for the changes in this set generally, so I'll clean it up a little more and send out v3 as a non-RFC version. Main gap I see right now (having fixed some checkpatch and build issues) is the docs. :-(

#14 lore raw

RFC PATCH v2 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL.

---
 lib/eal/common/eal_private.h                  |  7 ----
 lib/eal/common/meson.build                    |  1 -
 lib/eal/include/meson.build                   |  1 -
 lib/eal/linux/meson.build                     |  1 -
 lib/eal/meson.build                           |  2 +-
 lib/eal/version.map                           | 17 ----------
 lib/eal/windows/meson.build                   |  1 -
 lib/kvargs/meson.build                        |  3 +-
 lib/{eal/common => log}/eal_common_log.c      |  1 -
 lib/{eal/common => log}/eal_log.h             | 12 +++++++
 .../linux/eal_log.c => log/eal_log_linux.c}   |  0
 .../eal_log.c => log/eal_log_windows.c}       |  0
 lib/log/meson.build                           |  9 +++++
 lib/{eal/include => log}/rte_log.h            |  0
 lib/log/version.map                           | 34 +++++++++++++++++++
 lib/meson.build                               |  1 +
 lib/telemetry/meson.build                     |  3 +-
 17 files changed, 59 insertions(+), 34 deletions(-)
 rename lib/{eal/common => log}/eal_common_log.c (99%)
 rename lib/{eal/common => log}/eal_log.h (78%)
 rename lib/{eal/linux/eal_log.c => log/eal_log_linux.c} (100%)
 rename lib/{eal/windows/eal_log.c => log/eal_log_windows.c} (100%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
lib/eal/common/eal_private.h +0 -7
index 0f4d75bb89..dbf60190f4 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index cfcd40aaed..1eaa074d1b 100644
@@ -27,7 +27,6 @@ headers += files(
         'rte_keepalive.h',
         'rte_launch.h',
         'rte_lcore.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/meson.build +0 -1
index 3cccfa36c0..1b913acc06 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 056beb9461..af8c4eae4a 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index 7ad12a7dc9..3fb11a16b2 100644
@@ -140,21 +140,6 @@ DPDK_23 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_23 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_23 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/eal_common_log.c renamed from lib/eal/common/eal_common_log.c +0 -1
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/eal_common_log.c
index bd7b188ceb..895c4f2040 100644
@@ -16,7 +16,6 @@
 #include <rte_per_lcore.h>
 
 #include "eal_log.h"
-#include "eal_private.h"
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/eal_log.h renamed from lib/eal/common/eal_log.h +12 -0
similarity index 78%
rename from lib/eal/common/eal_log.h
rename to lib/log/eal_log.h
index c784fa6043..d04662729b 100644
@@ -7,26 +7,38 @@
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
 #endif /* EAL_LOG_H */
lib/log/eal_log_linux.c renamed from lib/eal/linux/eal_log.c +0 -0
similarity index 100%
rename from lib/eal/linux/eal_log.c
rename to lib/log/eal_log_linux.c
lib/log/eal_log_windows.c renamed from lib/eal/windows/eal_log.c +0 -0
similarity index 100%
rename from lib/eal/windows/eal_log.c
rename to lib/log/eal_log_windows.c
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..4aeb17b265
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'eal_common_log.c',
+        'eal_log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..726ff9fbcf
@@ -0,0 +1,34 @@
+DPDK_23 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index a90fee31b7..06e8bd7206 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.34.1
#15 lore raw

RFC PATCH v2 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 12 +++++-------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 8 insertions(+), 19 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index 607684c1a3..820c4524e5 100644
@@ -871,13 +871,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 8c118d0d9f..56ebca302e 100644
@@ -1319,13 +1319,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +5 -7
index 8fbb4f3060..46afc5c4ac 100644
@@ -54,11 +54,10 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
+extern int logtype;
 
 #define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+        rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -612,14 +611,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
@@ -630,3 +626,5 @@ rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_
 
 	return 0;
 }
+
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
lib/telemetry/telemetry_internal.h +1 -2
index d085c492dc..5c75d73183 100644
@@ -109,7 +109,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.34.1
#16 lore raw

v3 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the v1 RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, this v2 RFC takes a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

NOTE: documentation updates are still missing in v3, will come in v4. Submitting v3 to allow CI testing and check that v2 broken builds are now working

v3:
* Fixed missing log file for BSD
* Removed "eal" from the filenames of files in the log directory
* added prefixes to elements in the fnmatch header to avoid conflicts
* fixed space indentation in new lines in telemetry.c (checkpatch)
* removed "extern int logtype" definition in telemetry.c (checkpatch)
* added log directory to list for doxygen scanning

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 doc/api/doxy-api.conf.in                      |   1 +
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   8 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/fnmatch.c                     | 172 -----------------
 lib/eal/windows/include/fnmatch.h             | 175 ++++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   3 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 +-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  11 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 27 files changed, 249 insertions(+), 249 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.37.2

#17 lore raw

PATCH v3 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 -----------------------------
 lib/eal/windows/include/fnmatch.h | 175 +++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 162 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +162 -13

198 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.37.2
#18 lore raw

PATCH v3 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL. Rename files as appropriate, and use meson.build to select the correct file to be built for each operating system, rather than having a subdir per-os.

---
 doc/api/doxy-api.conf.in                      |  1 +
 lib/eal/common/eal_common_options.c           |  2 +-
 lib/eal/common/eal_private.h                  |  7 ----
 lib/eal/common/meson.build                    |  1 -
 lib/eal/include/meson.build                   |  1 -
 lib/eal/linux/eal.c                           |  2 +-
 lib/eal/linux/meson.build                     |  1 -
 lib/eal/meson.build                           |  2 +-
 lib/eal/version.map                           | 17 ----------
 lib/eal/windows/eal.c                         |  2 +-
 lib/eal/windows/meson.build                   |  1 -
 lib/kvargs/meson.build                        |  3 +-
 .../common/eal_common_log.c => log/log.c}     |  3 +-
 lib/log/log_freebsd.c                         | 12 +++++++
 .../common/eal_log.h => log/log_internal.h}   | 18 ++++++++--
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |  2 +-
 .../windows/eal_log.c => log/log_windows.c}   |  2 +-
 lib/log/meson.build                           |  9 +++++
 lib/{eal/include => log}/rte_log.h            |  0
 lib/log/version.map                           | 34 +++++++++++++++++++
 lib/meson.build                               |  1 +
 lib/telemetry/meson.build                     |  3 +-
 22 files changed, 81 insertions(+), 43 deletions(-)
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
doc/api/doxy-api.conf.in +1 -0
index f0886c3bd1..442703c01a 100644
@@ -51,6 +51,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
lib/eal/common/eal_common_options.c +1 -1
index 2d6535781b..89c312f441 100644
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
lib/eal/common/eal_private.h +0 -7
index 0f4d75bb89..dbf60190f4 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index cfcd40aaed..1eaa074d1b 100644
@@ -27,7 +27,6 @@ headers += files(
         'rte_keepalive.h',
         'rte_launch.h',
         'rte_lcore.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/eal.c +1 -1
index 8c118d0d9f..0df9f1f353 100644
@@ -49,10 +49,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/linux/meson.build +0 -1
index 3cccfa36c0..1b913acc06 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 056beb9461..af8c4eae4a 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index 7ad12a7dc9..3fb11a16b2 100644
@@ -140,21 +140,6 @@ DPDK_23 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_23 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_23 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/eal.c +1 -1
index 56fadc7afe..a41d177d6c 100644
@@ -26,8 +26,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/log.c renamed from lib/eal/common/eal_common_log.c +1 -2
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index bd7b188ceb..5021e49e33 100644
@@ -15,8 +15,7 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/log_freebsd.c new file +12 -0
new file mode 100644
index 0000000000..698d3c5423
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
lib/log/log_internal.h renamed from lib/eal/common/eal_log.h +15 -3
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
lib/log/log_linux.c renamed from lib/eal/linux/eal_log.c +1 -1
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
lib/log/log_windows.c renamed from lib/eal/windows/eal_log.c +1 -1
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..6baff83ee5
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..726ff9fbcf
@@ -0,0 +1,34 @@
+DPDK_23 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index a90fee31b7..06e8bd7206 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.37.2
#19 lore raw

PATCH v3 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 11 +++--------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 6 insertions(+), 20 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index 607684c1a3..820c4524e5 100644
@@ -871,13 +871,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 0df9f1f353..dec0041094 100644
@@ -1319,13 +1319,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +3 -8
index 8fbb4f3060..13a32f4279 100644
@@ -54,11 +54,9 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
 
-#define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
+#define TMTY_LOG(l, ...) rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -612,14 +610,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
lib/telemetry/telemetry_internal.h +1 -2
index d085c492dc..5c75d73183 100644
@@ -109,7 +109,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.37.2
#20 lore raw

v4 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

v4:
* Fixed windows build error, due to missing strdup (_strdup on windows)
* Added doc updates to programmers guide.

v3:
* Fixed missing log file for BSD
* Removed "eal" from the filenames of files in the log directory
* added prefixes to elements in the fnmatch header to avoid conflicts
* fixed space indentation in new lines in telemetry.c (checkpatch)
* removed "extern int logtype" definition in telemetry.c (checkpatch)
* added log directory to list for doxygen scanning

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   8 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/fnmatch.c                     | 172 -----------------
 lib/eal/windows/include/fnmatch.h             | 175 ++++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 +-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  11 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 30 files changed, 370 insertions(+), 252 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.37.2

#21 lore raw

PATCH v4 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 -----------------------------
 lib/eal/windows/include/fnmatch.h | 175 +++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 162 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +162 -13

198 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.37.2
#22 lore raw

PATCH v4 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL. Rename files as appropriate, and use meson.build to select the correct file to be built for each operating system, rather than having a subdir per-os. Add new documentation section in programmers guide to cover logging in more detail.

---
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 --
 lib/eal/common/meson.build                    |   1 -
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   2 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 ---
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/meson.build                   |   1 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 ++-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 ++
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 25 files changed, 202 insertions(+), 46 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
doc/api/doxy-api.conf.in +1 -0
index f0886c3bd1..442703c01a 100644
@@ -51,6 +51,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
doc/guides/prog_guide/env_abstraction_layer.rst +1 -3
index 35fbebe1be..5fede6bc70 100644
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :ref:`Log_Library`.
 
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
doc/guides/prog_guide/index.rst +1 -0
index 8564883018..6344c178c1 100644
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
doc/guides/prog_guide/log_lib.rst new file +115 -0

115 lines not on this page · raw message

lib/eal/common/eal_common_options.c +1 -1
index 2d6535781b..89c312f441 100644
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
lib/eal/common/eal_private.h +0 -7
index 0f4d75bb89..dbf60190f4 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index cfcd40aaed..1eaa074d1b 100644
@@ -27,7 +27,6 @@ headers += files(
         'rte_keepalive.h',
         'rte_launch.h',
         'rte_lcore.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/eal.c +1 -1
index 8c118d0d9f..0df9f1f353 100644
@@ -49,10 +49,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/linux/meson.build +0 -1
index 3cccfa36c0..1b913acc06 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 056beb9461..af8c4eae4a 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index 7ad12a7dc9..3fb11a16b2 100644
@@ -140,21 +140,6 @@ DPDK_23 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_23 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_23 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/eal.c +1 -1
index 56fadc7afe..a41d177d6c 100644
@@ -26,8 +26,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/log.c renamed from lib/eal/common/eal_common_log.c +5 -2
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index bd7b188ceb..c3ede383b2 100644
@@ -15,8 +15,11 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+#define strdup _strdup
+#endif
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/log_freebsd.c new file +12 -0
new file mode 100644
index 0000000000..698d3c5423
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
lib/log/log_internal.h renamed from lib/eal/common/eal_log.h +15 -3
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
lib/log/log_linux.c renamed from lib/eal/linux/eal_log.c +1 -1
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
lib/log/log_windows.c renamed from lib/eal/windows/eal_log.c +1 -1
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..6baff83ee5
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..726ff9fbcf
@@ -0,0 +1,34 @@
+DPDK_23 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index a90fee31b7..06e8bd7206 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.37.2
#23 lore raw

PATCH v4 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 11 +++--------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 6 insertions(+), 20 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index 607684c1a3..820c4524e5 100644
@@ -871,13 +871,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 0df9f1f353..dec0041094 100644
@@ -1319,13 +1319,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +3 -8
index 8fbb4f3060..13a32f4279 100644
@@ -54,11 +54,9 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
 
-#define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
+#define TMTY_LOG(l, ...) rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -612,14 +610,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
lib/telemetry/telemetry_internal.h +1 -2
index d085c492dc..5c75d73183 100644
@@ -109,7 +109,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.37.2
#24 lore raw

Re: [PATCH v4 0/3] Split logging functionality out of EAL

In reply to Bruce Richardson

Hi Bruce,

Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

There is another complication.

The ABI check is not handling properly the case where symbols are moved to the new log library (even though the dependency to librte_log is explicit in librte_eal elf).
For now, I don't have a good way to handle this.

A workaround to pass the check is to suppress those symbols wrt the eal dump:
[suppress_function]
symbol_name_regexp = rte_log
[suppress_function]
symbol_name = rte_openlog_stream
[suppress_function]
symbol_name = rte_vlog

But this is not a good solution because we would be losing checks on them for the rest of the v23 ABI life.

Signature · 1 line
David Marchand
#25 lore raw

Re: [PATCH v4 0/3] Split logging functionality out of EAL

Quoted 43 lines · David Marchand

Hi Bruce,

Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

There is another complication.

The ABI check is not handling properly the case where symbols are moved to the new log library (even though the dependency to librte_log is explicit in librte_eal elf).
For now, I don't have a good way to handle this.

A workaround to pass the check is to suppress those symbols wrt the eal dump:
[suppress_function]
symbol_name_regexp = rte_log
[suppress_function]
symbol_name = rte_openlog_stream
[suppress_function]
symbol_name = rte_vlog

Quoted 3 lines

But this is not a good solution because we would be losing checks on them for the rest of the v23 ABI life.

Right, I got error messages from the CI job for this too, but I also have no idea how to work around this. Perhaps we only get to move content between libraries when we do an ABI bump? Seems a bit restrictive, though.

/Bruce

#26 lore raw

Re: [PATCH v4 0/3] Split logging functionality out of EAL

Quoted 45 lines · Bruce Richardson
Quoted 43 lines · David Marchand

Hi Bruce,

Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

There is another complication.

The ABI check is not handling properly the case where symbols are moved to the new log library (even though the dependency to librte_log is explicit in librte_eal elf).
For now, I don't have a good way to handle this.

A workaround to pass the check is to suppress those symbols wrt the eal dump:
[suppress_function]
symbol_name_regexp = rte_log
[suppress_function]
symbol_name = rte_openlog_stream
[suppress_function]
symbol_name = rte_vlog

Quoted 6 lines
Quoted 3 lines

But this is not a good solution because we would be losing checks on them for the rest of the v23 ABI life.

Right, I got error messages from the CI job for this too, but I also have no idea how to work around this. Perhaps we only get to move content between libraries when we do an ABI bump? Seems a bit restrictive, though.

Moving symbols as you did does not seem an ABI breakage.
An application that links to eal would see the dt_needed entry for the new log library, load it accordingly and gets the right symbols.

Signature · 1 line
David Marchand
#27 lore raw

Re: [PATCH v4 0/3] Split logging functionality out of EAL

Quoted 50 lines · David Marchand
Quoted 48 lines · Bruce Richardson
Quoted 46 lines · David Marchand

Hi Bruce,

Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

There is another complication.

The ABI check is not handling properly the case where symbols are moved to the new log library (even though the dependency to librte_log is explicit in librte_eal elf).
For now, I don't have a good way to handle this.

A workaround to pass the check is to suppress those symbols wrt the eal dump:
[suppress_function]
symbol_name_regexp = rte_log
[suppress_function]
symbol_name = rte_openlog_stream
[suppress_function]
symbol_name = rte_vlog

But this is not a good solution because we would be losing checks on them for the rest of the v23 ABI life.

Quoted 8 lines
Quoted 3 lines

Right, I got error messages from the CI job for this too, but I also have no idea how to work around this. Perhaps we only get to move content between libraries when we do an ABI bump? Seems a bit restrictive, though.

Moving symbols as you did does not seem an ABI breakage.
An application that links to eal would see the dt_needed entry for the new log library, load it accordingly and gets the right symbols.

Yes, I agree. However, I also agree with you that it is risky to lose symbol checking for the moved symbols if we need to remove them from analysis. That said, maybe others have some ideas as to how to work around this, or perhaps we just take the risk of disabling checking.

/Bruce

#28 lore raw

Re: [PATCH v4 0/3] Split logging functionality out of EAL

Quoted 56 lines · Bruce Richardson
Quoted 54 lines · David Marchand
Quoted 51 lines · Bruce Richardson
Quoted 43 lines · David Marchand

Hi Bruce,

Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

There is another complication.

The ABI check is not handling properly the case where symbols are moved to the new log library (even though the dependency to librte_log is explicit in librte_eal elf).
For now, I don't have a good way to handle this.

A workaround to pass the check is to suppress those symbols wrt the eal dump:
[suppress_function]
symbol_name_regexp = rte_log
[suppress_function]
symbol_name = rte_openlog_stream
[suppress_function]
symbol_name = rte_vlog

Quoted 3 lines

But this is not a good solution because we would be losing checks on them for the rest of the v23 ABI life.

Right, I got error messages from the CI job for this too, but I also have no idea how to work around this. Perhaps we only get to move content between libraries when we do an ABI bump? Seems a bit restrictive, though.

Quoted 8 lines
Quoted 4 lines

Moving symbols as you did does not seem an ABI breakage.
An application that links to eal would see the dt_needed entry for the new log library, load it accordingly and gets the right symbols.

Yes, I agree. However, I also agree with you that it is risky to lose symbol checking for the moved symbols if we need to remove them from analysis. That said, maybe others have some ideas as to how to work around this, or perhaps we just take the risk of disabling checking.

I opened a bz for libabigail.
https://sourceware.org/bugzilla/show_bug.cgi?id=30034

If it is handled fast enough, we may have a solution by the time 23.03 is released and we will remove this workaround for 23.07 development (no pressure Dodji :-p).

Signature · 1 line
David Marchand
#29 lore raw

v5 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

v5:
* rebased to latest main branch
* fixed trailing whitespace issues in new doc section

v4:
* Fixed windows build error, due to missing strdup (_strdup on windows)
* Added doc updates to programmers guide.

v3:
* Fixed missing log file for BSD
* Removed "eal" from the filenames of files in the log directory
* added prefixes to elements in the fnmatch header to avoid conflicts
* fixed space indentation in new lines in telemetry.c (checkpatch)
* removed "extern int logtype" definition in telemetry.c (checkpatch)
* added log directory to list for doxygen scanning

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   8 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/fnmatch.c                     | 172 -----------------
 lib/eal/windows/include/fnmatch.h             | 175 ++++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 +-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  11 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 30 files changed, 370 insertions(+), 252 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.39.2

#30 lore raw

PATCH v5 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 -----------------------------
 lib/eal/windows/include/fnmatch.h | 175 +++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 162 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +162 -13

198 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.39.2
#31 lore raw

PATCH v5 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL. Rename files as appropriate, and use meson.build to select the correct file to be built for each operating system, rather than having a subdir per-os. Add new documentation section in programmers guide to cover logging in more detail.

---
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 --
 lib/eal/common/meson.build                    |   1 -
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   2 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 ---
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/meson.build                   |   1 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 ++-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 ++
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 25 files changed, 202 insertions(+), 46 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
doc/api/doxy-api.conf.in +1 -0
index d230a19e1f..967aa11701 100644
@@ -51,6 +51,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
doc/guides/prog_guide/env_abstraction_layer.rst +1 -3
index 93c8a031be..8033f6cebd 100644
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :ref:`Log_Library`.
 
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
doc/guides/prog_guide/index.rst +1 -0
index 87333ee84a..ac01f0eae9 100644
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
doc/guides/prog_guide/log_lib.rst new file +115 -0

115 lines not on this page · raw message

lib/eal/common/eal_common_options.c +1 -1
index 0305933698..9cbe42e587 100644
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
lib/eal/common/eal_private.h +0 -7
index d7f83776c1..5eadba4902 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index b0db9b3b3a..a0463efac7 100644
@@ -28,7 +28,6 @@ headers += files(
         'rte_launch.h',
         'rte_lcore.h',
         'rte_lock_annotations.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/eal.c +1 -1
index ae323cd492..765ad1d783 100644
@@ -50,10 +50,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/linux/meson.build +0 -1
index 3cccfa36c0..1b913acc06 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 9aa941a5ae..0fb974c78b 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index 51a820d829..8f25f31499 100644
@@ -140,21 +140,6 @@ DPDK_23 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_23 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_23 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/eal.c +1 -1
index 033825c14c..725c8618e3 100644
@@ -27,8 +27,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/log.c renamed from lib/eal/common/eal_common_log.c +5 -2
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index bd7b188ceb..c3ede383b2 100644
@@ -15,8 +15,11 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+#define strdup _strdup
+#endif
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/log_freebsd.c new file +12 -0
new file mode 100644
index 0000000000..698d3c5423
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
lib/log/log_internal.h renamed from lib/eal/common/eal_log.h +15 -3
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
lib/log/log_linux.c renamed from lib/eal/linux/eal_log.c +1 -1
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
lib/log/log_windows.c renamed from lib/eal/windows/eal_log.c +1 -1
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..6baff83ee5
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..726ff9fbcf
@@ -0,0 +1,34 @@
+DPDK_23 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index dc8aa4ac84..ebcd0a5ebd 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index 73750d9ef4..72f135d7d2 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.39.2
#32 lore raw

PATCH v5 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 11 +++--------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 6 insertions(+), 20 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index 7daf22e314..f125a0d3de 100644
@@ -884,13 +884,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 765ad1d783..c80647d6b2 100644
@@ -1330,13 +1330,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +3 -8
index deba7f34a3..3feeebd814 100644
@@ -54,11 +54,9 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
 
-#define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
+#define TMTY_LOG(l, ...) rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -612,14 +610,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
lib/telemetry/telemetry_internal.h +1 -2
index d085c492dc..5c75d73183 100644
@@ -109,7 +109,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.39.2
#33 lore raw

v6 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

v6:
* Updated ABI version to DPDK_24 for new log library for 23.11 release.

v5:
* rebased to latest main branch
* fixed trailing whitespace issues in new doc section

v4:
* Fixed windows build error, due to missing strdup (_strdup on windows)
* Added doc updates to programmers guide.

v3:
* Fixed missing log file for BSD
* Removed "eal" from the filenames of files in the log directory
* added prefixes to elements in the fnmatch header to avoid conflicts
* fixed space indentation in new lines in telemetry.c (checkpatch)
* removed "extern int logtype" definition in telemetry.c (checkpatch)
* added log directory to list for doxygen scanning

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   8 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/fnmatch.c                     | 172 -----------------
 lib/eal/windows/include/fnmatch.h             | 175 ++++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 +-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  11 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 30 files changed, 370 insertions(+), 252 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.39.2

#34 lore raw

PATCH v6 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 -----------------------------
 lib/eal/windows/include/fnmatch.h | 175 +++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 162 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +162 -13

198 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.39.2
#35 lore raw

PATCH v6 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL. Rename files as appropriate, and use meson.build to select the correct file to be built for each operating system, rather than having a subdir per-os. Add new documentation section in programmers guide to cover logging in more detail.

---
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 --
 lib/eal/common/meson.build                    |   1 -
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   2 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 ---
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/meson.build                   |   1 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 ++-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 ++
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 25 files changed, 202 insertions(+), 46 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
doc/api/doxy-api.conf.in +1 -0
index 1a4210b948..40f7dd2877 100644
@@ -52,6 +52,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
doc/guides/prog_guide/env_abstraction_layer.rst +1 -3
index 93c8a031be..8033f6cebd 100644
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :ref:`Log_Library`.
 
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
doc/guides/prog_guide/index.rst +1 -0
index d89cd3edb6..ac91060992 100644
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
doc/guides/prog_guide/log_lib.rst new file +115 -0

115 lines not on this page · raw message

lib/eal/common/eal_common_options.c +1 -1
index 062f1d8d9c..d74b581567 100644
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
lib/eal/common/eal_private.h +0 -7
index d7f83776c1..5eadba4902 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index b0db9b3b3a..a0463efac7 100644
@@ -28,7 +28,6 @@ headers += files(
         'rte_launch.h',
         'rte_lcore.h',
         'rte_lock_annotations.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/eal.c +1 -1
index c6efd92014..2f9203c87d 100644
@@ -50,10 +50,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/linux/meson.build +0 -1
index 5af456db9e..e99ebed256 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 9aa941a5ae..0fb974c78b 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index bdb98cf479..7940431e5a 100644
@@ -140,21 +140,6 @@ DPDK_24 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_24 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_24 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/eal.c +1 -1
index 096cb98d1c..22dc7f6ee1 100644
@@ -27,8 +27,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/log.c renamed from lib/eal/common/eal_common_log.c +5 -2
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index bd7b188ceb..c3ede383b2 100644
@@ -15,8 +15,11 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+#define strdup _strdup
+#endif
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/log_freebsd.c new file +12 -0
new file mode 100644
index 0000000000..698d3c5423
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
lib/log/log_internal.h renamed from lib/eal/common/eal_log.h +15 -3
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
lib/log/log_linux.c renamed from lib/eal/linux/eal_log.c +1 -1
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
lib/log/log_windows.c renamed from lib/eal/windows/eal_log.c +1 -1
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..6baff83ee5
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..0648f8831a
@@ -0,0 +1,34 @@
+DPDK_24 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index fac2f52cad..c5e4642337 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.39.2
#36 lore raw

PATCH v6 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 11 +++--------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 6 insertions(+), 20 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index dfe973e6fd..0175d89e4b 100644
@@ -898,13 +898,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 2f9203c87d..543cbe75fa 100644
@@ -1333,13 +1333,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +3 -8
index 590720bfa6..7d0488a6d7 100644
@@ -54,11 +54,9 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
 
-#define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
+#define TMTY_LOG(l, ...) rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -627,14 +625,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
lib/telemetry/telemetry_internal.h +1 -2
index 80e2bd3a49..b331e9458f 100644
@@ -108,7 +108,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.39.2
#37 lore raw

v7 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

V7:
* re-submit to re-run CI with ABI checks disabled

v6:
* Updated ABI version to DPDK_24 for new log library for 23.11 release.

v5:
* rebased to latest main branch
* fixed trailing whitespace issues in new doc section

v4:
* Fixed windows build error, due to missing strdup (_strdup on windows)
* Added doc updates to programmers guide.

v3:
* Fixed missing log file for BSD
* Removed "eal" from the filenames of files in the log directory
* added prefixes to elements in the fnmatch header to avoid conflicts
* fixed space indentation in new lines in telemetry.c (checkpatch)
* removed "extern int logtype" definition in telemetry.c (checkpatch)
* added log directory to list for doxygen scanning

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   8 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/fnmatch.c                     | 172 -----------------
 lib/eal/windows/include/fnmatch.h             | 175 ++++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 +-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  11 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 30 files changed, 370 insertions(+), 252 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.39.2

#38 lore raw

PATCH v7 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 -----------------------------
 lib/eal/windows/include/fnmatch.h | 175 +++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 162 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +162 -13

198 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.39.2
#39 lore raw

Re: [PATCH v7 1/3] eal/windows: move fnmatch function to header file

Quoted 7 lines · Bruce Richardson

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

Overall, it lgtm.

I am surprised those 3 static symbols (see below) do not require being marked "inline" (to avoid "unused symbols" warnings).
The CI looks ok, so probably I am just paranoid.

I have also a comment on sccsid, see below.

Quoted 5 lines

---

[snip]

Quoted 14 lines
lib/eal/windows/include/fnmatch.h +5 -2
index c6b226bd5d..fbf1eef21c 100644
@@ -1,20 +1,25 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2019 Intel Corporation
+ * Copyright (c) 1989, 1993, 1994
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This code is derived from software contributed to Berkeley by
+ * Guido van Rossum.
  */
-

lib/eal/windows/include/fnmatch.h line 13

Quoted 10 lines
lib/eal/windows/include/fnmatch.h +3 -4
 #ifndef _FNMATCH_H_
 #define _FNMATCH_H_
 
-/**
- * This file is required to support the common code in eal_common_log.c
- * as Microsoft libc does not contain fnmatch.h. This may be removed in
- * future releases.
+#if defined(LIBC_SCCS) && !defined(lint)
+static const char sccsid[] = "@(#)fnmatch.c    8.2 (Berkeley) 4/16/94";
+#endif /* LIBC_SCCS and not lint */

Strange to keep this (what looks to be a canary) symbol in a header file.

[snip]

Quoted 11 lines
lib/eal/windows/include/fnmatch.h +4 -0
@@ -25,6 +30,10 @@ extern "C" {
 #define FNM_CASEFOLD 0x10
 #define FNM_PREFIX_DIRS 0x20
 
+#define FNM_EOS        '\0'
+
+static const char *fnm_rangematch(const char *, char, int);
+
 /**
  * This function is used for searching a given string source
  * with the given regular expression pattern.

[snip]

Quoted 10 lines
lib/eal/windows/include/fnmatch.h +5 -1
@@ -41,10 +50,150 @@ extern "C" {
  * @return
  *     if the pattern is found then return 0 or else FNM_NOMATCH
  */
-int fnmatch(const char *pattern, const char *string, int flags);
+static int
+fnmatch(const char *pattern, const char *string, int flags)
+{
+       const char *stringstart;
+       char c, test;
Signature · 1 line
David Marchand
#40 lore raw

Re: [PATCH v7 1/3] eal/windows: move fnmatch function to header file

Quoted 10 lines · David Marchand
Quoted 7 lines · Bruce Richardson

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

Quoted 5 lines

Overall, it lgtm.

I am surprised those 3 static symbols (see below) do not require being marked "inline" (to avoid "unused symbols" warnings).
The CI looks ok, so probably I am just paranoid.

Only the functions should need the inline, and I suspect that we don't get any warnings, since the only files including the header always use the functions. Will mark them as inline just in case in next version.

Quoted 39 lines

I have also a comment on sccsid, see below.

Quoted 5 lines

---

[snip]

Quoted 14 lines
lib/eal/windows/include/fnmatch.h +5 -2
index c6b226bd5d..fbf1eef21c 100644
@@ -1,20 +1,25 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright(c) 2019 Intel Corporation
+ * Copyright (c) 1989, 1993, 1994
+ *     The Regents of the University of California.  All rights reserved.
+ *
+ * This code is derived from software contributed to Berkeley by
+ * Guido van Rossum.
  */
-

lib/eal/windows/include/fnmatch.h line 13

Quoted 10 lines
lib/eal/windows/include/fnmatch.h +3 -4
 #ifndef _FNMATCH_H_
 #define _FNMATCH_H_
 
-/**
- * This file is required to support the common code in eal_common_log.c
- * as Microsoft libc does not contain fnmatch.h. This may be removed in
- * future releases.
+#if defined(LIBC_SCCS) && !defined(lint)
+static const char sccsid[] = "@(#)fnmatch.c    8.2 (Berkeley) 4/16/94";
+#endif /* LIBC_SCCS and not lint */

Strange to keep this (what looks to be a canary) symbol in a header file.

Quoted 2 lines

[snip]

Yeah, missed that, can probably be removed.

Quoted 30 lines
Quoted 11 lines
lib/eal/windows/include/fnmatch.h +4 -0
@@ -25,6 +30,10 @@ extern "C" {
 #define FNM_CASEFOLD 0x10
 #define FNM_PREFIX_DIRS 0x20
 
+#define FNM_EOS        '\0'
+
+static const char *fnm_rangematch(const char *, char, int);
+
 /**
  * This function is used for searching a given string source
  * with the given regular expression pattern.

[snip]

Quoted 10 lines
lib/eal/windows/include/fnmatch.h +5 -1
@@ -41,10 +50,150 @@ extern "C" {
  * @return
  *     if the pattern is found then return 0 or else FNM_NOMATCH
  */
-int fnmatch(const char *pattern, const char *string, int flags);
+static int
+fnmatch(const char *pattern, const char *string, int flags)
+{
+       const char *stringstart;
+       char c, test;
Signature · 1 line
David Marchand
#41 lore raw

PATCH v7 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL. Rename files as appropriate, and use meson.build to select the correct file to be built for each operating system, rather than having a subdir per-os. Add new documentation section in programmers guide to cover logging in more detail.

---
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 115 ++++++++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 --
 lib/eal/common/meson.build                    |   1 -
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   2 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 ---
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/meson.build                   |   1 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 ++-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 ++
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 25 files changed, 202 insertions(+), 46 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
doc/api/doxy-api.conf.in +1 -0
index 1a4210b948..40f7dd2877 100644
@@ -52,6 +52,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
doc/guides/prog_guide/env_abstraction_layer.rst +1 -3
index 93c8a031be..8033f6cebd 100644
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :ref:`Log_Library`.
 
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
doc/guides/prog_guide/index.rst +1 -0
index d89cd3edb6..ac91060992 100644
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
doc/guides/prog_guide/log_lib.rst new file +115 -0

115 lines not on this page · raw message

lib/eal/common/eal_common_options.c +1 -1
index 062f1d8d9c..d74b581567 100644
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
lib/eal/common/eal_private.h +0 -7
index d7f83776c1..5eadba4902 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index b0db9b3b3a..a0463efac7 100644
@@ -28,7 +28,6 @@ headers += files(
         'rte_launch.h',
         'rte_lcore.h',
         'rte_lock_annotations.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/eal.c +1 -1
index c6efd92014..2f9203c87d 100644
@@ -50,10 +50,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/linux/meson.build +0 -1
index 5af456db9e..e99ebed256 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 9aa941a5ae..0fb974c78b 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index bdb98cf479..7940431e5a 100644
@@ -140,21 +140,6 @@ DPDK_24 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_24 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_24 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/eal.c +1 -1
index 096cb98d1c..22dc7f6ee1 100644
@@ -27,8 +27,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/log.c renamed from lib/eal/common/eal_common_log.c +5 -2
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index bd7b188ceb..c3ede383b2 100644
@@ -15,8 +15,11 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+#define strdup _strdup
+#endif
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/log_freebsd.c new file +12 -0
new file mode 100644
index 0000000000..698d3c5423
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
lib/log/log_internal.h renamed from lib/eal/common/eal_log.h +15 -3
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
lib/log/log_linux.c renamed from lib/eal/linux/eal_log.c +1 -1
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
lib/log/log_windows.c renamed from lib/eal/windows/eal_log.c +1 -1
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..6baff83ee5
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..0648f8831a
@@ -0,0 +1,34 @@
+DPDK_24 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index fac2f52cad..c5e4642337 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.39.2
#42 lore raw

Re: [PATCH v7 2/3] log: separate logging functions out of EAL

doc/api/doxy-api.conf.in INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ · line 58

Quoted 12 lines · Bruce Richardson
doc/api/doxy-api.conf.in +1 -0
index 1a4210b948..40f7dd2877 100644
@@ -52,6 +52,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \

In case you are wondering about the reason.. the Intel CI reports an error on building the documentation:

*Build Failed #1:
OS: UB2204-64
Target: x86_64-native-linuxapp-doc
FAILED: doc/api/html
/usr/bin/python3 ../doc/api/generate_doxygen.py doc/api/html
/usr/bin/doxygen doc/api/doxy-api.conf /root/UB2204-64_K5.15.0_GCC11.3.0/x86_64-native-linuxapp-doc/29045/dpdk/doc/api/doxy-api-index.md:230:
error: unable to resolve reference to 'rte_log.h' for \ref command
(warning treated as error, aborting now)
Traceback (most recent call last):
File "/root/UB2204-64_K5.15.0_GCC11.3.0/x86_64-native-linuxapp-doc/29045/dpdk/x86_64-native-linuxapp-doc/../doc/api/generate_doxygen.py",
line 13, in <module>
subprocess.run(doxygen_command, check=True, stdout=out)
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/doxygen',
'doc/api/doxy-api.conf']' returned non-zero exit status 1.
[2772/2774] Compiling C object app/test/dpdk-test.p/test_ring.c.o
[2773/2774] Generating doc/guides/html_guides with a custom command
ninja: build stopped

IIRC Intel CI excludes changes on the doc/ directory when applying patches.
http://inbox.dpdk.org/dev/CY5PR11MB618735108D3E9D90C67EEF96F5DA9@CY5PR11MB6187.namprd11.prod.outlook.com/

.. which seems to match the failure reported here, as doxygen would fail to find rte_log.h.

Signature · 1 line
David Marchand
#43 lore raw

Re: [PATCH v7 2/3] log: separate logging functions out of EAL

Quoted 39 lines · David Marchand

doc/api/doxy-api.conf.in INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ · line 58

Quoted 12 lines · Bruce Richardson
doc/api/doxy-api.conf.in +1 -0
index 1a4210b948..40f7dd2877 100644
@@ -52,6 +52,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \

In case you are wondering about the reason.. the Intel CI reports an error on building the documentation:

*Build Failed #1:
OS: UB2204-64
Target: x86_64-native-linuxapp-doc
FAILED: doc/api/html
/usr/bin/python3 ../doc/api/generate_doxygen.py doc/api/html
/usr/bin/doxygen doc/api/doxy-api.conf /root/UB2204-64_K5.15.0_GCC11.3.0/x86_64-native-linuxapp-doc/29045/dpdk/doc/api/doxy-api-index.md:230:
error: unable to resolve reference to 'rte_log.h' for \ref command
(warning treated as error, aborting now)
Traceback (most recent call last):
File "/root/UB2204-64_K5.15.0_GCC11.3.0/x86_64-native-linuxapp-doc/29045/dpdk/x86_64-native-linuxapp-doc/../doc/api/generate_doxygen.py",
line 13, in <module>
subprocess.run(doxygen_command, check=True, stdout=out)
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/doxygen',
'doc/api/doxy-api.conf']' returned non-zero exit status 1.
[2772/2774] Compiling C object app/test/dpdk-test.p/test_ring.c.o
[2773/2774] Generating doc/guides/html_guides with a custom command
ninja: build stopped

Quoted 6 lines

IIRC Intel CI excludes changes on the doc/ directory when applying patches.
http://inbox.dpdk.org/dev/CY5PR11MB618735108D3E9D90C67EEF96F5DA9@CY5PR11MB6187.namprd11.prod.outlook.com/

.. which seems to match the failure reported here, as doxygen would fail to find rte_log.h.

Yes, I was very much wondering about the reason for failure, since I could find no issues with building the docs on my own system!

Thanks for the info.
/Bruce

#44 lore raw

Re: [PATCH v7 2/3] log: separate logging functions out of EAL

Quoted 40 lines · Bruce Richardson
Quoted 39 lines · David Marchand

doc/api/doxy-api.conf.in INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ · line 58

Quoted 12 lines · Bruce Richardson
doc/api/doxy-api.conf.in +1 -0
index 1a4210b948..40f7dd2877 100644
@@ -52,6 +52,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/kni \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \

In case you are wondering about the reason.. the Intel CI reports an error on building the documentation:

*Build Failed #1:
OS: UB2204-64
Target: x86_64-native-linuxapp-doc
FAILED: doc/api/html
/usr/bin/python3 ../doc/api/generate_doxygen.py doc/api/html
/usr/bin/doxygen doc/api/doxy-api.conf /root/UB2204-64_K5.15.0_GCC11.3.0/x86_64-native-linuxapp-doc/29045/dpdk/doc/api/doxy-api-index.md:230:
error: unable to resolve reference to 'rte_log.h' for \ref command
(warning treated as error, aborting now)
Traceback (most recent call last):
File "/root/UB2204-64_K5.15.0_GCC11.3.0/x86_64-native-linuxapp-doc/29045/dpdk/x86_64-native-linuxapp-doc/../doc/api/generate_doxygen.py",
line 13, in <module>
subprocess.run(doxygen_command, check=True, stdout=out)
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/doxygen',
'doc/api/doxy-api.conf']' returned non-zero exit status 1.
[2772/2774] Compiling C object app/test/dpdk-test.p/test_ring.c.o
[2773/2774] Generating doc/guides/html_guides with a custom command
ninja: build stopped

Quoted 9 lines
Quoted 6 lines

IIRC Intel CI excludes changes on the doc/ directory when applying patches.
http://inbox.dpdk.org/dev/CY5PR11MB618735108D3E9D90C67EEF96F5DA9@CY5PR11MB6187.namprd11.prod.outlook.com/

.. which seems to match the failure reported here, as doxygen would fail to find rte_log.h.

Yes, I was very much wondering about the reason for failure, since I could find no issues with building the docs on my own system!

Is this patchset ok for merge then, or is there something else outstanding holding it back?

Thanks,
/Bruce

#45 lore raw

Re: [PATCH v7 2/3] log: separate logging functions out of EAL

In reply to Bruce Richardson

Some nits on the doc (that I can fix when applying if you are ok with them) :

doc/guides/prog_guide/env_abstraction_layer.rst Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid · line 446

Quoted 12 lines · Bruce Richardson
doc/guides/prog_guide/env_abstraction_layer.rst +1 -3
index 93c8a031be..8033f6cebd 100644
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :ref:`Log_Library`.

:doc:log_lib

Quoted 21 lines
doc/guides/prog_guide/env_abstraction_layer.rst +0 -0
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
doc/guides/prog_guide/index.rst +1 -0
index d89cd3edb6..ac91060992 100644
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
doc/guides/prog_guide/log_lib.rst new file +0 -0
new file mode 100644
index 0000000000..706ddcfef3

doc/guides/prog_guide/log_lib.rst line 4

Quoted 4 lines
doc/guides/prog_guide/log_lib.rst new file +4 -0
@@ -0,0 +1,115 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2023 Intel Corporation.
+
+.. _log_library:

Since we can directly point at this part of the documentation via :doc:/path/to/log_lib, no need for an anchor at the top of the file.

doc/guides/prog_guide/log_lib.rst line 7

Quoted 3 lines
doc/guides/prog_guide/log_lib.rst new file +3 -0
+
+Log Library
+============

nit: too long by one =.

doc/guides/prog_guide/log_lib.rst line 15

Quoted 8 lines
doc/guides/prog_guide/log_lib.rst new file +8 -0
+
+The DPDK Log library provides the logging functionality for other DPDK libraries and drivers.
+By default, in a Linux application, logs are sent to syslog and also to the console.
+On FreeBSD and Windows applications, logs are sent only to the console.
+However, the log function can be overridden by the user to use a different logging mechanism.
+
+Log Levels
+-----------

Idem.

Quoted 10 lines
doc/guides/prog_guide/log_lib.rst new file +10 -0
+
+Log messages from apps and libraries are reported with a given level of severity.
+These levels, specified in ``rte_log.h`` are (from most to least important):
+
+#. Emergency
+#. Alert
+#. Critical
+#. Error
+#. Warning
+#. Notice

doc/guides/prog_guide/log_lib.rst line 35

Quoted 10 lines
doc/guides/prog_guide/log_lib.rst new file +10 -0
+#. Information
+#. Debug
+
+At runtime, only messages of a configured level or above (i.e. of higher importance)
+will be emitted by the application to the log output.
+That level can be configured either by the application calling the relevant APIs from the logging library,
+or by the user passing the ``--log-level`` parameter to the EAL via the application.
+
+Setting Global Log Level
+~~~~~~~~~~~~~~~~~~~~~~~~~

Idem.

Quoted 5 lines
doc/guides/prog_guide/log_lib.rst new file +5 -0
+
+To adjust the global log level for an application,
+just pass a numeric level or a level name to the ``--log-level`` EAL parameter.
+For example::
+

doc/guides/prog_guide/log_lib.rst line 50

Quoted 10 lines
doc/guides/prog_guide/log_lib.rst new file +10 -0
+       /path/to/app --log-level=error
+
+       /path/to/app --log-level=debug
+
+       /path/to/app --log-level=5   # warning
+
+Within an application, the log level can be similarly set using the ``rte_log_set_global_level`` API.
+
+Setting Log Level for a Component
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Idem.

Quoted 5 lines
doc/guides/prog_guide/log_lib.rst new file +5 -0
+
+In some cases, for example, for debugging purposes,
+it may be desirable to increase or decrease the log level for only a specific component, or set of components.
+To facilitate this, the ``--log-level`` argument also accepts an, optionally wildcarded, component name,
+along with the desired level for that component.

doc/guides/prog_guide/log_lib.rst line 65

Quoted 10 lines
doc/guides/prog_guide/log_lib.rst new file +10 -0
+For example::
+
+       /path/to/app --log-level=lib.eal:crit
+
+       /path/to/app --log-level=lib.*:warning
+
+Within an application, the same result can be got using the ``rte_log_set_level_pattern()`` or ``rte_log_set_level_regex()`` APIs.
+
+Using Logging APIs to Generate Log Messages
+--------------------------------------------

Idem.

Quoted 7 lines
doc/guides/prog_guide/log_lib.rst new file +7 -0
+
+To output log messages, ``rte_log()`` API function should be used.
+As well as the log message, ``rte_log()`` takes two additional parameters:
+
+* The log level
+* The log component type
+
Signature · 1 line
David Marchand
#46 lore raw

Re: [PATCH v7 2/3] log: separate logging functions out of EAL

In reply to Bruce Richardson

Sorry, two additional comments after running more checks.

lib/log files are not referenced in MAINTAINERS, can you add a new block?

Quoted 6 lines
lib/log/meson.build new file +0 -0
new file mode 100644
index 0000000000..6baff83ee5

lib/log/meson.build line 8

Quoted 8 lines
lib/log/meson.build new file +8 -0
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)

Missing a , (reported by check-meson.py).

Quoted 1 line
lib/log/meson.build new file +1 -0
+headers = files('rte_log.h')
Signature · 1 line
David Marchand
#47 lore raw

Re: [PATCH v7 2/3] log: separate logging functions out of EAL

Quoted 4 lines · David Marchand

Sorry, two additional comments after running more checks.

lib/log files are not referenced in MAINTAINERS, can you add a new block?

Yep.

Quoted 8 lines
Quoted 8 lines
lib/log/meson.build new file +2 -0
new file mode 100644
index 0000000000..6baff83ee5
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
Quoted 11 lines

lib/log/meson.build line 8

Quoted 6 lines
lib/log/meson.build new file +6 -0
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c'
+)

Missing a , (reported by check-meson.py).

Quoted 1 line
lib/log/meson.build new file +1 -0
+headers = files('rte_log.h')

Will review all feedback and spin a new revision.
thanks,
/Bruce

#48 lore raw

PATCH v7 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 11 +++--------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 6 insertions(+), 20 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index dfe973e6fd..0175d89e4b 100644
@@ -898,13 +898,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 2f9203c87d..543cbe75fa 100644
@@ -1333,13 +1333,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +3 -8
index 590720bfa6..7d0488a6d7 100644
@@ -54,11 +54,9 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
 
-#define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
+#define TMTY_LOG(l, ...) rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -627,14 +625,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
lib/telemetry/telemetry_internal.h +1 -2
index 80e2bd3a49..b331e9458f 100644
@@ -108,7 +108,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.39.2
#49 lore raw

v8 cover Split logging functionality out of EAL

In reply to Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

V8:
* Added "inline" to static functions in fnmatch header
* Removed SCCS tag as unneeded carryover from .c file
* Corrected doc cross-references and headers
* Added maintainers entry

V7:
* re-submit to re-run CI with ABI checks disabled

v6:
* Updated ABI version to DPDK_24 for new log library for 23.11 release.

v5:
* rebased to latest main branch
* fixed trailing whitespace issues in new doc section

v4:
* Fixed windows build error, due to missing strdup (_strdup on windows)
* Added doc updates to programmers guide.

v3:
* Fixed missing log file for BSD
* Removed "eal" from the filenames of files in the log directory
* added prefixes to elements in the fnmatch header to avoid conflicts
* fixed space indentation in new lines in telemetry.c (checkpatch)
* removed "extern int logtype" definition in telemetry.c (checkpatch)
* added log directory to list for doxygen scanning

Bruce Richardson (3):
eal/windows: move fnmatch function to header file
log: separate logging functions out of EAL
telemetry: use standard logging

 MAINTAINERS                                   |   6 +-
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 113 ++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 -
 lib/eal/common/meson.build                    |   1 -
 lib/eal/freebsd/eal.c                         |   6 +-
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   8 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 --
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/fnmatch.c                     | 172 ------------------
 lib/eal/windows/include/fnmatch.h             | 169 +++++++++++++++--
 lib/eal/windows/meson.build                   |   2 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 +-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 +
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 lib/telemetry/telemetry.c                     |  11 +-
 lib/telemetry/telemetry_internal.h            |   3 +-
 31 files changed, 367 insertions(+), 253 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 delete mode 100644 lib/eal/windows/fnmatch.c
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map

--
2.39.2

#50 lore raw

PATCH v8 1/3 eal/windows: move fnmatch function to header file

To allow the fnmatch function to be shared between libraries, without having to export it into the public namespace (since it's not prefixed with "rte"), we can convert fnmatch.c to replace fnmatch.h. This allows fnmatch function to be static and limited in scope to the current file, preventing duplicate definitions if it is used by two libraries, while also not requiring export for sharing.

---
 lib/eal/windows/fnmatch.c         | 172 ------------------------------
 lib/eal/windows/include/fnmatch.h | 169 ++++++++++++++++++++++++++---
 lib/eal/windows/meson.build       |   1 -
 3 files changed, 156 insertions(+), 186 deletions(-)
 delete mode 100644 lib/eal/windows/fnmatch.c
lib/eal/windows/fnmatch.c deleted +0 -172
deleted file mode 100644
index f622bf54c5..0000000000
@@ -1,172 +0,0 @@
-/* SPDX-License-Identifier: BSD-3-Clause
- * Copyright (c) 1989, 1993, 1994
- *	The Regents of the University of California.  All rights reserved.
- *
- * This code is derived from software contributed to Berkeley by
- * Guido van Rossum.
- */
-
-#if defined(LIBC_SCCS) && !defined(lint)
-static const char sccsid[] = "@(#)fnmatch.c	8.2 (Berkeley) 4/16/94";
-#endif /* LIBC_SCCS and not lint */
-
-/*
- * Function fnmatch() as specified in POSIX 1003.2-1992, section B.6.
- * Compares a filename or pathname to a pattern.
- */
-
-#include <ctype.h>
-#include <string.h>
-#include <stdio.h>
-
-#include "fnmatch.h"
-
-#define EOS	'\0'
-
-static const char *rangematch(const char *, char, int);
-
-int
-fnmatch(const char *pattern, const char *string, int flags)
-{
-	const char *stringstart;
-	char c, test;
-
-	for (stringstart = string;;)
-		switch (c = *pattern++) {
-		case EOS:
-			if ((flags & FNM_LEADING_DIR) && *string == '/')
-				return (0);
-			return (*string == EOS ? 0 : FNM_NOMATCH);
-		case '?':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && (flags & FNM_PATHNAME))
-				return (FNM_NOMATCH);
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '*':
-			c = *pattern;
-			/* Collapse multiple stars. */
-			while (c == '*')
-				c = *++pattern;
-
-			if (*string == '.' && (flags & FNM_PERIOD) &&
-			    (string == stringstart ||
-			    ((flags & FNM_PATHNAME) && *(string - 1) == '/')))
-				return (FNM_NOMATCH);
-
-			/* Optimize for pattern with * at end or before /. */
-			if (c == EOS)
-				if (flags & FNM_PATHNAME)
-					return ((flags & FNM_LEADING_DIR) ||
-					    strchr(string, '/') == NULL ?
-					    0 : FNM_NOMATCH);
-				else
-					return (0);
-			else if (c == '/' && flags & FNM_PATHNAME) {
-				string = strchr(string, '/');
-				if (string == NULL)
-					return (FNM_NOMATCH);
-				break;
-			}
-
-			/* General case, use recursion. */
-			while ((test = *string) != EOS) {
-				if (!fnmatch(pattern, string,
-					flags & ~FNM_PERIOD))
-					return (0);
-				if (test == '/' && flags & FNM_PATHNAME)
-					break;
-				++string;
-			}
-			return (FNM_NOMATCH);
-		case '[':
-			if (*string == EOS)
-				return (FNM_NOMATCH);
-			if (*string == '/' && flags & FNM_PATHNAME)
-				return (FNM_NOMATCH);
-			pattern = rangematch(pattern, *string, flags);
-			if (pattern == NULL)
-				return (FNM_NOMATCH);
-			++string;
-			break;
-		case '\\':
-			if (!(flags & FNM_NOESCAPE)) {
-				c = *pattern++;
-				if (c == EOS) {
-					c = '\\';
-					--pattern;
-				}
-			}
-			/* FALLTHROUGH */
-		default:
-			if (c == *string)
-				;
-			else if ((flags & FNM_CASEFOLD) &&
-				 (tolower((unsigned char)c) ==
-				  tolower((unsigned char)*string)))
-				;
-			else if ((flags & FNM_PREFIX_DIRS) && *string == EOS &&
-			     ((c == '/' && string != stringstart) ||
-			     (string == stringstart+1 && *stringstart == '/')))
-				return (0);
-			else
-				return (FNM_NOMATCH);
-			string++;
-			break;
-		}
-	/* NOTREACHED */
-}
-
-static const char *
-rangematch(const char *pattern, char test, int flags)
-{
-	int negate, ok;
-	char c, c2;
-
-	/*
-	 * A bracket expression starting with an unquoted circumflex
-	 * character produces unspecified results (IEEE 1003.2-1992,
-	 * 3.13.2).  This implementation treats it like '!', for
-	 * consistency with the regular expression syntax.
-	 * J.T. Conklin ([email protected])
-	 */
-	negate = (*pattern == '!' || *pattern == '^');
-	if (negate)
-		++pattern;
-
-	if (flags & FNM_CASEFOLD)
-		test = tolower((unsigned char)test);
-
-	for (ok = 0; (c = *pattern++) != ']';) {
-		if (c == '\\' && !(flags & FNM_NOESCAPE))
-			c = *pattern++;
-		if (c == EOS)
-			return (NULL);
-
-		if (flags & FNM_CASEFOLD)
-			c = tolower((unsigned char)c);
-
-		c2 = *(pattern + 1);
-		if (*pattern == '-' && c2 != EOS && c2 != ']') {
-			pattern += 2;
-			if (c2 == '\\' && !(flags & FNM_NOESCAPE))
-				c2 = *pattern++;
-			if (c2 == EOS)
-				return (NULL);
-
-			if (flags & FNM_CASEFOLD)
-				c2 = tolower((unsigned char)c2);
-
-			if ((unsigned char)c <= (unsigned char)test &&
-			    (unsigned char)test <= (unsigned char)c2)
-				ok = 1;
-		} else if (c == test)
-			ok = 1;
-	}
-	return (ok == negate ? NULL : pattern);
-}
lib/eal/windows/include/fnmatch.h +156 -13

192 lines not on this page · raw message

lib/eal/windows/meson.build +0 -1
index 845e406ca1..e4b2427610 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_mp.c',
         'eal_thread.c',
         'eal_timer.c',
-        'fnmatch.c',
         'getopt.c',
         'rte_thread.c',
 )
Signature · 1 line
2.39.2
#51 lore raw

PATCH v8 2/3 log: separate logging functions out of EAL

In reply to Bruce Richardson

Move the logging capability to a separate library, free from EAL. Rename files as appropriate, and use meson.build to select the correct file to be built for each operating system, rather than having a subdir per-os. Add new documentation section in programmers guide to cover logging in more detail.

---
 MAINTAINERS                                   |   6 +-
 doc/api/doxy-api.conf.in                      |   1 +
 .../prog_guide/env_abstraction_layer.rst      |   4 +-
 doc/guides/prog_guide/index.rst               |   1 +
 doc/guides/prog_guide/log_lib.rst             | 113 ++++++++++++++++++
 lib/eal/common/eal_common_options.c           |   2 +-
 lib/eal/common/eal_private.h                  |   7 --
 lib/eal/common/meson.build                    |   1 -
 lib/eal/include/meson.build                   |   1 -
 lib/eal/linux/eal.c                           |   2 +-
 lib/eal/linux/meson.build                     |   1 -
 lib/eal/meson.build                           |   2 +-
 lib/eal/version.map                           |  17 ---
 lib/eal/windows/eal.c                         |   2 +-
 lib/eal/windows/meson.build                   |   1 -
 lib/kvargs/meson.build                        |   3 +-
 .../common/eal_common_log.c => log/log.c}     |   7 +-
 lib/log/log_freebsd.c                         |  12 ++
 .../common/eal_log.h => log/log_internal.h}   |  18 ++-
 lib/{eal/linux/eal_log.c => log/log_linux.c}  |   2 +-
 .../windows/eal_log.c => log/log_windows.c}   |   2 +-
 lib/log/meson.build                           |   9 ++
 lib/{eal/include => log}/rte_log.h            |   0
 lib/log/version.map                           |  34 ++++++
 lib/meson.build                               |   1 +
 lib/telemetry/meson.build                     |   3 +-
 26 files changed, 205 insertions(+), 47 deletions(-)
 create mode 100644 doc/guides/prog_guide/log_lib.rst
 rename lib/{eal/common/eal_common_log.c => log/log.c} (99%)
 create mode 100644 lib/log/log_freebsd.c
 rename lib/{eal/common/eal_log.h => log/log_internal.h} (69%)
 rename lib/{eal/linux/eal_log.c => log/log_linux.c} (97%)
 rename lib/{eal/windows/eal_log.c => log/log_windows.c} (93%)
 create mode 100644 lib/log/meson.build
 rename lib/{eal/include => log}/rte_log.h (100%)
 create mode 100644 lib/log/version.map
MAINTAINERS +5 -1
index 6345e7f8a6..8c3f2c993f 100644
@@ -164,7 +164,6 @@ F: app/test/test_devargs.c
 F: app/test/test_eal*
 F: app/test/test_errno.c
 F: app/test/test_lcores.c
-F: app/test/test_logs.c
 F: app/test/test_memcpy*
 F: app/test/test_per_lcore.c
 F: app/test/test_pflock.c
@@ -177,6 +176,11 @@ F: app/test/test_tailq.c
 F: app/test/test_threads.c
 F: app/test/test_version.c
 
+Logging
+F: lib/log/
+F: doc/guides/prog_guide/log_lib.rst
+F: app/test/test_logs.c
+
 Trace - EXPERIMENTAL
 M: Jerin Jacob <[email protected]>
 M: Sunil Kumar Kori <[email protected]>
doc/api/doxy-api.conf.in +1 -0
index 31885039c7..a88accd907 100644
@@ -50,6 +50,7 @@ INPUT                   = @TOPDIR@/doc/api/doxy-api-index.md \
                           @TOPDIR@/lib/jobstats \
                           @TOPDIR@/lib/kvargs \
                           @TOPDIR@/lib/latencystats \
+                          @TOPDIR@/lib/log \
                           @TOPDIR@/lib/lpm \
                           @TOPDIR@/lib/mbuf \
                           @TOPDIR@/lib/member \
doc/guides/prog_guide/env_abstraction_layer.rst +1 -3
index 5d382fdd90..89014789de 100644
@@ -443,9 +443,7 @@ Per-lcore variables are implemented using *Thread Local Storage* (TLS) to provid
 Logs
 ~~~~
 
-A logging API is provided by EAL.
-By default, in a Linux application, logs are sent to syslog and also to the console.
-However, the log function can be overridden by the user to use a different logging mechanism.
+While originally part of EAL, DPDK logging functionality is now provided by the :doc:`log_lib`.
 
 Trace and Debug Functions
 ^^^^^^^^^^^^^^^^^^^^^^^^^
doc/guides/prog_guide/index.rst +1 -0
index 2c47d9d010..52a6d9e7aa 100644
@@ -12,6 +12,7 @@ Programmer's Guide
     overview
     source_org
     env_abstraction_layer
+    log_lib
     service_cores
     trace_lib
     rcu_lib
doc/guides/prog_guide/log_lib.rst new file +113 -0

113 lines not on this page · raw message

lib/eal/common/eal_common_options.c +1 -1
index 062f1d8d9c..d74b581567 100644
@@ -39,7 +39,7 @@
 #include "eal_options.h"
 #include "eal_filesystem.h"
 #include "eal_private.h"
-#include "eal_log.h"
+#include "log_internal.h"
 #ifndef RTE_EXEC_ENV_WINDOWS
 #include "eal_trace.h"
 #endif
lib/eal/common/eal_private.h +0 -7
index d7f83776c1..5eadba4902 100644
@@ -152,13 +152,6 @@ int rte_eal_tailqs_init(void);
  */
 int rte_eal_intr_init(void);
 
-/**
- * Close the default log stream
- *
- * This function is private to EAL.
- */
-void rte_eal_log_cleanup(void);
-
 /**
  * Init alarm mechanism. This is to allow a callback be called after
  * specific time.
lib/eal/common/meson.build +0 -1
index 917758cc65..22a626ba6f 100644
@@ -18,7 +18,6 @@ sources += files(
         'eal_common_interrupts.c',
         'eal_common_launch.c',
         'eal_common_lcore.c',
-        'eal_common_log.c',
         'eal_common_mcfg.c',
         'eal_common_memalloc.c',
         'eal_common_memory.c',
lib/eal/include/meson.build +0 -1
index b0db9b3b3a..a0463efac7 100644
@@ -28,7 +28,6 @@ headers += files(
         'rte_launch.h',
         'rte_lcore.h',
         'rte_lock_annotations.h',
-        'rte_log.h',
         'rte_malloc.h',
         'rte_mcslock.h',
         'rte_memory.h',
lib/eal/linux/eal.c +1 -1
index a1fefcd9d8..477ad03d38 100644
@@ -50,10 +50,10 @@
 #include "eal_hugepages.h"
 #include "eal_memcfg.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_options.h"
 #include "eal_vfio.h"
 #include "hotplug_mp.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/linux/meson.build +0 -1
index 5af456db9e..e99ebed256 100644
@@ -11,7 +11,6 @@ sources += files(
         'eal_hugepage_info.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_thread.c',
lib/eal/meson.build +1 -1
index 9aa941a5ae..0fb974c78b 100644
@@ -22,7 +22,7 @@ subdir(exec_env)
 
 subdir(arch_subdir)
 
-deps += ['kvargs']
+deps += ['log', 'kvargs']
 if not is_windows
     deps += ['telemetry']
 endif
lib/eal/version.map +0 -17
index bdb98cf479..7940431e5a 100644
@@ -140,21 +140,6 @@ DPDK_24 {
 	rte_lcore_iterate;
 	rte_lcore_to_cpu_id;
 	rte_lcore_to_socket_id;
-	rte_log;
-	rte_log_can_log;
-	rte_log_cur_msg_loglevel;
-	rte_log_cur_msg_logtype;
-	rte_log_dump;
-	rte_log_get_global_level;
-	rte_log_get_level;
-	rte_log_get_stream;
-	rte_log_list_types;
-	rte_log_register;
-	rte_log_register_type_and_pick_level;
-	rte_log_set_global_level;
-	rte_log_set_level;
-	rte_log_set_level_pattern;
-	rte_log_set_level_regexp;
 	rte_malloc;
 	rte_malloc_dump_heaps;
 	rte_malloc_dump_stats;
@@ -225,7 +210,6 @@ DPDK_24 {
 	rte_mp_request_async;
 	rte_mp_request_sync;
 	rte_mp_sendmsg;
-	rte_openlog_stream;
 	rte_rand;
 	rte_rand_max;
 	rte_realloc;
@@ -299,7 +283,6 @@ DPDK_24 {
 	rte_vfio_noiommu_is_enabled; # WINDOWS_NO_EXPORT
 	rte_vfio_release_device; # WINDOWS_NO_EXPORT
 	rte_vfio_setup_device; # WINDOWS_NO_EXPORT
-	rte_vlog;
 	rte_zmalloc;
 	rte_zmalloc_socket;
 
lib/eal/windows/eal.c +1 -1
index 096cb98d1c..22dc7f6ee1 100644
@@ -27,8 +27,8 @@
 #include "eal_firmware.h"
 #include "eal_hugepages.h"
 #include "eal_trace.h"
-#include "eal_log.h"
 #include "eal_windows.h"
+#include "log_internal.h"
 
 #define MEMSIZE_IF_NO_HUGE_PAGE (64ULL * 1024ULL * 1024ULL)
 
lib/eal/windows/meson.build +0 -1
index e4b2427610..7756d417be 100644
@@ -12,7 +12,6 @@ sources += files(
         'eal_hugepages.c',
         'eal_interrupts.c',
         'eal_lcore.c',
-        'eal_log.c',
         'eal_memalloc.c',
         'eal_memory.c',
         'eal_mp.c',
lib/kvargs/meson.build +1 -2
index b746516965..7eae744a8f 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('rte_kvargs.c')
 headers = files('rte_kvargs.h')
lib/log/log.c renamed from lib/eal/common/eal_common_log.c +5 -2
similarity index 99%
rename from lib/eal/common/eal_common_log.c
rename to lib/log/log.c
index 0a1d219d69..52c771fb6b 100644
@@ -15,8 +15,11 @@
 #include <rte_log.h>
 #include <rte_per_lcore.h>
 
-#include "eal_log.h"
-#include "eal_private.h"
+#include "log_internal.h"
+
+#ifdef RTE_EXEC_ENV_WINDOWS
+#define strdup _strdup
+#endif
 
 struct rte_log_dynamic_type {
 	const char *name;
lib/log/log_freebsd.c new file +12 -0
new file mode 100644
index 0000000000..698d3c5423
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Intel Corporation
+ */
+
+#include <rte_common.h>
+#include "log_internal.h"
+
+int
+eal_log_init(__rte_unused const char *id, __rte_unused int facility)
+{
+	return 0;
+}
lib/log/log_internal.h renamed from lib/eal/common/eal_log.h +15 -3
similarity index 69%
rename from lib/eal/common/eal_log.h
rename to lib/log/log_internal.h
index c784fa6043..20d6313898 100644
@@ -2,31 +2,43 @@
  * Copyright 2021 Mellanox Technologies, Ltd
  */
 
-#ifndef EAL_LOG_H
-#define EAL_LOG_H
+#ifndef LOG_INTERNAL_H
+#define LOG_INTERNAL_H
 
 #include <stdio.h>
 #include <stdint.h>
+#include <rte_compat.h>
 
 /*
  * Initialize the default log stream.
  */
+__rte_internal
 int eal_log_init(const char *id, int facility);
 
 /*
  * Determine where log data is written when no call to rte_openlog_stream.
  */
+__rte_internal
 void eal_log_set_default(FILE *default_log);
 
 /*
  * Save a log option for later.
  */
+__rte_internal
 int eal_log_save_regexp(const char *regexp, uint32_t level);
+__rte_internal
 int eal_log_save_pattern(const char *pattern, uint32_t level);
 
 /*
  * Convert log level to string.
  */
+__rte_internal
 const char *eal_log_level2str(uint32_t level);
 
-#endif /* EAL_LOG_H */
+/*
+ * Close the default log stream
+ */
+__rte_internal
+void rte_eal_log_cleanup(void);
+
+#endif /* LOG_INTERNAL_H */
lib/log/log_linux.c renamed from lib/eal/linux/eal_log.c +1 -1
similarity index 97%
rename from lib/eal/linux/eal_log.c
rename to lib/log/log_linux.c
index d44416fd65..2dfb0c974b 100644
@@ -8,7 +8,7 @@
 
 #include <rte_log.h>
 
-#include "eal_log.h"
+#include "log_internal.h"
 
 /*
  * default log function
lib/log/log_windows.c renamed from lib/eal/windows/eal_log.c +1 -1
similarity index 93%
rename from lib/eal/windows/eal_log.c
rename to lib/log/log_windows.c
index d4ea47f1c8..a6a0889550 100644
@@ -4,7 +4,7 @@
 
 #include <rte_common.h>
 #include <rte_log.h>
-#include "eal_log.h"
+#include "log_internal.h"
 
 /* set the log to default function, called during eal init process. */
 int
lib/log/meson.build new file +9 -0
new file mode 100644
index 0000000000..545cdb8084
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 Intel Corporation
+
+includes += global_inc
+sources = files(
+        'log.c',
+        'log_' + exec_env + '.c',
+)
+headers = files('rte_log.h')
lib/log/rte_log.h renamed from lib/eal/include/rte_log.h +0 -0
similarity index 100%
rename from lib/eal/include/rte_log.h
rename to lib/log/rte_log.h
lib/log/version.map new file +34 -0
new file mode 100644
index 0000000000..0648f8831a
@@ -0,0 +1,34 @@
+DPDK_24 {
+	global:
+
+	rte_log;
+	rte_log_cur_msg_loglevel;
+	rte_log_cur_msg_logtype;
+	rte_log_can_log;
+	rte_log_dump;
+	rte_log_get_global_level;
+	rte_log_get_level;
+	rte_log_get_stream;
+	rte_log_list_types;
+	rte_log_register;
+	rte_log_register_type_and_pick_level;
+	rte_log_set_global_level;
+	rte_log_set_level;
+	rte_log_set_level_pattern;
+	rte_log_set_level_regexp;
+	rte_openlog_stream;
+	rte_vlog;
+
+	local: *;
+};
+
+INTERNAL {
+	global:
+
+	eal_log_init;
+	eal_log_level2str;
+	eal_log_save_pattern;
+	eal_log_save_regexp;
+	eal_log_set_default;
+	rte_eal_log_cleanup;
+};
lib/meson.build +1 -0
index bbfa28ba86..92cbd6bb66 100644
@@ -9,6 +9,7 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [
+        'log',
         'kvargs', # eal depends on kvargs
         'telemetry', # basic info querying
         'eal', # everything depends on eal
lib/telemetry/meson.build +1 -2
index f84c9aa3be..489d000047 100644
@@ -1,8 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-includes = [global_inc]
-
+deps += 'log'
 sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c')
 headers = files('rte_telemetry.h')
 includes += include_directories('../metrics')
Signature · 1 line
2.39.2
#52 lore raw

PATCH v8 3/3 telemetry: use standard logging

In reply to Bruce Richardson

Now that logging is moved out of EAL, we don't need injection of the logtype and logging function from EAL to telemetry library, simplifying things.

---
 lib/eal/freebsd/eal.c              |  6 +-----
 lib/eal/linux/eal.c                |  6 +-----
 lib/telemetry/telemetry.c          | 11 +++--------
 lib/telemetry/telemetry_internal.h |  3 +--
 4 files changed, 6 insertions(+), 20 deletions(-)
lib/eal/freebsd/eal.c +1 -5
index dfe973e6fd..0175d89e4b 100644
@@ -898,13 +898,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/eal/linux/eal.c +1 -5
index 477ad03d38..ed50576d99 100644
@@ -1314,13 +1314,9 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY && !internal_conf->no_telemetry) {
-		int tlog = rte_log_register_type_and_pick_level(
-				"lib.telemetry", RTE_LOG_WARNING);
-		if (tlog < 0)
-			tlog = RTE_LOGTYPE_EAL;
 		if (rte_telemetry_init(rte_eal_get_runtime_dir(),
 				rte_version(),
-				&internal_conf->ctrl_cpuset, rte_log, tlog) != 0)
+				&internal_conf->ctrl_cpuset) != 0)
 			return -1;
 	}
 
lib/telemetry/telemetry.c +3 -8
index 590720bfa6..7d0488a6d7 100644
@@ -54,11 +54,9 @@ static struct socket v1_socket; /* socket for v1 telemetry */
 static const char *telemetry_version; /* save rte_version */
 static const char *socket_dir;        /* runtime directory */
 static rte_cpuset_t *thread_cpuset;
-static rte_log_fn rte_log_ptr;
-static uint32_t logtype;
 
-#define TMTY_LOG(l, ...) \
-        rte_log_ptr(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
+RTE_LOG_REGISTER_DEFAULT(logtype, WARNING);
+#define TMTY_LOG(l, ...) rte_log(RTE_LOG_ ## l, logtype, "TELEMETRY: " __VA_ARGS__)
 
 /* list of command callbacks, with one command registered by default */
 static struct cmd_callback *callbacks;
@@ -627,14 +625,11 @@ telemetry_v2_init(void)
 #endif /* !RTE_EXEC_ENV_WINDOWS */
 
 int32_t
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype)
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset)
 {
 	telemetry_version = rte_version;
 	socket_dir = runtime_dir;
 	thread_cpuset = cpuset;
-	rte_log_ptr = log_fn;
-	logtype = registered_logtype;
 
 #ifndef RTE_EXEC_ENV_WINDOWS
 	if (telemetry_v2_init() != 0)
lib/telemetry/telemetry_internal.h +1 -2
index 80e2bd3a49..b331e9458f 100644
@@ -108,7 +108,6 @@ typedef int (*rte_log_fn)(uint32_t level, uint32_t logtype, const char *format,
  */
 __rte_internal
 int
-rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset,
-		rte_log_fn log_fn, uint32_t registered_logtype);
+rte_telemetry_init(const char *runtime_dir, const char *rte_version, rte_cpuset_t *cpuset);
 
 #endif
Signature · 1 line
2.39.2
#53 lore raw

Re: [PATCH v8 0/3] Split logging functionality out of EAL

In reply to Bruce Richardson
Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

Series applied, thanks Bruce for this first step.

As mentionned during the maintainers weekly call yesterday, this is only a first "easy" step but, thinking of next steps, more splitting may not be that easy.

At least, on the libabigail topic, as we need the ABI check to handle libraries splits, a new feature has been cooked in (not yet released) 2.4 libabigail.
https://sourceware.org/git/?p=libabigail.git;a=commitdiff;h=0b338dfaf690993e123b6433201b3a8b8204d662
Hopefully, we will have a libabigail release available by the time we start the v24.03 release (and re-enable ABI checks).

Signature · 1 line
David Marchand
#54 lore raw

Re: [PATCH v8 0/3] Split logging functionality out of EAL

Quoted 26 lines · David Marchand
Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

Quoted 6 lines

Series applied, thanks Bruce for this first step.

As mentionned during the maintainers weekly call yesterday, this is only a first "easy" step but, thinking of next steps, more splitting may not be that easy.

I took a look after doing this patchset to try and find more easy to extract parts, but did not find many. The EAL is pretty intertwined now. As I look at it, there are really two ways to try and dis-entangle it - bottoms-up or top-down. This patchset is an example of the former, taking a logical set of related APIs and pulling them out into a separate library that EAL can depend upon. There may be some other API-sets like this we can pull out, but in my search I didn't find any.

The other tops-down approach may be worth looking at too. We can try and take the top-level EAL init function and separate it out into a separate initialization library (which may be called "EAL" with the rest being called something else). I have not tried this approach to see how it goes, but pulling out the init may allow further dis-entangling of other parts of EAL.

/Bruce

#55 lore raw

Re: [PATCH v8 0/3] Split logging functionality out of EAL

11/08/2023 14:59, Bruce Richardson:

Quoted 33 lines · David Marchand
Quoted 26 lines
Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

Quoted 5 lines

Series applied, thanks Bruce for this first step.

As mentionned during the maintainers weekly call yesterday, this is only a first "easy" step but, thinking of next steps, more splitting may not be that easy.

Quoted 7 lines

I took a look after doing this patchset to try and find more easy to extract parts, but did not find many. The EAL is pretty intertwined now. As I look at it, there are really two ways to try and dis-entangle it - bottoms-up or top-down. This patchset is an example of the former, taking a logical set of related APIs and pulling them out into a separate library that EAL can depend upon. There may be some other API-sets like this we can pull out, but in my search I didn't find any.

A small thing to easily separate is rte_bitmap.

Quoted 6 lines

The other tops-down approach may be worth looking at too. We can try and take the top-level EAL init function and separate it out into a separate initialization library (which may be called "EAL" with the rest being called something else). I have not tried this approach to see how it goes, but pulling out the init may allow further dis-entangling of other parts of EAL.

I agree we should start with separating the init logic.
I would call it rte_init. We may need rte_opts for command line parsing. I think the name EAL should be kept for the low-level functions, abstracting differences between OS, CPU and toolchains.

Next steps would be to try to separate memory and CPU management in 2 different libraries.
We'll need to decide whether rte_service is part of the CPU management. Same question for multiprocess communication rte_mp and rte_keepalive. I suppose we can keep IRQ and threading in EAL.
VFIO may be managed in a separate library perhaps.

Once we do that, the low-level libraries like log, kvargs and telemetry can depend on EAL, being the very low-level common denominator. The trace subsystem can probably become a separate library as well.

In a later step, we can think about bus and device management.

And the ideal would be to extract tailq, once all logic is out of EAL.

How does it sound?

#56 lore raw

RE: [PATCH v8 0/3] Split logging functionality out of EAL

Quoted 66 lines

From: Thomas Monjalon [mailto:[email protected]]
Sent: Friday, 1 September 2023 13.25

11/08/2023 14:59, Bruce Richardson:

Quoted 33 lines · David Marchand
Quoted 26 lines
Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

Quoted 5 lines

Series applied, thanks Bruce for this first step.

As mentionned during the maintainers weekly call yesterday, this is only a first "easy" step but, thinking of next steps, more splitting may not be that easy.

Quoted 7 lines

I took a look after doing this patchset to try and find more easy to extract parts, but did not find many. The EAL is pretty intertwined now. As I look at it, there are really two ways to try and dis-entangle it - bottoms-up or top-down. This patchset is an example of the former, taking a logical set of related APIs and pulling them out into a separate library that EAL can depend upon. There may be some other API-sets like this we can pull out, but in my search I didn't find any.

A small thing to easily separate is rte_bitmap.

Quoted 6 lines

The other tops-down approach may be worth looking at too. We can try and take the top-level EAL init function and separate it out into a separate initialization library (which may be called "EAL" with the rest being called something else). I have not tried this approach to see how it goes, but pulling out the init may allow further dis-entangling of other parts of EAL.

I agree we should start with separating the init logic.
I would call it rte_init. We may need rte_opts for command line parsing. I think the name EAL should be kept for the low-level functions, abstracting differences between OS, CPU and toolchains.

Next steps would be to try to separate memory and CPU management in 2 different libraries.
We'll need to decide whether rte_service is part of the CPU management. Same question for multiprocess communication rte_mp and rte_keepalive. I suppose we can keep IRQ and threading in EAL.
VFIO may be managed in a separate library perhaps.

Quoted 7 lines

Once we do that, the low-level libraries like log, kvargs and telemetry can depend on EAL, being the very low-level common denominator. The trace subsystem can probably become a separate library as well.

In a later step, we can think about bus and device management.

And the ideal would be to extract tailq, once all logic is out of EAL.

Yes, tailq is a good example of a utility library, which can stand on its own (i.e. it could be used for non-DPDK applications too).

Quoted 3 lines

How does it sound?

Not really a reply to Thomas, just thinking out loud here...

From a high level perspective, thinking about two dimensions, layers and silos, (the boxes a marketing person would draw on a PowerPoint slide to describe what DPDK comprises of) might also help defining the borders.

For vertical separation:

There are the architecture specific definitions, e.g. header files created when selecting an architecture. There should be no functions in this box, only descriptions/definitions.

Then there are the utility libraries/functions, e.g. bit manipulation and log2 functions, the command line parser, etc.. Some of these depend on the architecture definitions. But none of these should depend on the DPDK runtime itself - i.e. it should be possible to use this for writing a non-dataplane application (in theory).

Then there is the DPDK core itself. The core of this should be kept at an absolute minimum, making the various features less cross-dependent. (This is where the "mandatory" libraries live.)

And then there is the DPDK runtime, with application init/start, timers, service cores, etc.

And finally, there are all the DPDK libraries.

Maybe they can be grouped into "bus" libraries (e.g. pci), "driver" libraries (e.g. compressdev, cryptodev, ethdev), "feature" libraries (e.g. reorder, bpf, gro, frag) and "utility" libraries (e.g. net, ring, rcu, mempool).

Some can be classified as "core" libraries (e.g. mbuf), because many other libs can depend on them. And some might be classified as "mandatory" (e.g. ring and mempool), because the DPDK runtime depends on them.

#57 lore raw

Re: [PATCH v8 0/3] Split logging functionality out of EAL

In reply to David Marchand
Quoted 33 lines · David Marchand
Quoted 17 lines · Bruce Richardson

There is a general desire to reduce the size and scope of EAL. To this end, this patchset makes a (very) small step in that direction by taking the logging functionality out of EAL and putting it into its own library that can be built and maintained separately.

As with the first RFC for this, the main obstacle is the "fnmatch" function which is needed by both EAL and the new log function when building on windows. While the function cannot stay in EAL - or we would have a circular dependency, moving it to a new library or just putting it in the log library have the disadvantages that it then "leaks" into the public namespace without an rte_prefix, which could cause issues. Since only a single function is involved, subsequent versions take a different approach to v1, and just moves the offending function to be a static function in a header file. This allows use by multiple libs without conflicting names or making it public.

Quoted 6 lines

The other complication, as explained in v1 RFC was that of multiple implementations for different OS's. This is solved here in the same way as v1, by including the OS in the name and having meson pick the correct file for each build. Since only one file is involved, there seemed little need for replicating EAL's separate subdirectories per-OS.

Series applied, thanks Bruce for this first step.

As mentionned during the maintainers weekly call yesterday, this is only a first "easy" step but, thinking of next steps, more splitting may not be that easy.

Quoted 6 lines

At least, on the libabigail topic, as we need the ABI check to handle libraries splits, a new feature has been cooked in (not yet released) 2.4 libabigail.
https://sourceware.org/git/?p=libabigail.git;a=commitdiff;h=0b338dfaf690993e123b6433201b3a8b8204d662
Hopefully, we will have a libabigail release available by the time we start the v24.03 release (and re-enable ABI checks).

For the record, libabigail 2.4 got released last Friday.
I had already prepared a patch to use this new feature, I can send it when needed.

Signature · 1 line
David Marchand