log level control enhancements
v3 cover log level control enhancements
This patch set improves the log-level option in EAL. It adds symbolic names for log levels (error, info, debug, etc) and shell style matching of log levels.
--log-level='pmd.ixgbe.*:debug'
The original syntax (which is maintained for compatibility) used numeric values, comma as separator and regular expression to match log level.
It was often documented as:
--log-level='pmd.ixgbe.*,8' ...
but this only worked by accident. The period was really a regular expression match any character (not match a period); and the asterisk in regular expression is match repeated value.
v3
- rename internal function rte_eal_log_save_XXX to rte_log_save_XXX
- fixup typos in coding style doc
v2
- make facility table constant
- rebase to include support for dynamic log late binding
- add patch to make saving log level a private API
- update driver docs
Stephen Hemminger (5):
eal: make syslog facility table const
eal: allow symbolic log levels
eal: make eal_log_level save private
log: add ability to match dynamic log based on shell pattern
doc: update guides for current preferrred log level syntax
doc/guides/contributing/coding_style.rst | 4 +- doc/guides/cryptodevs/dpaa2_sec.rst | 2 +- doc/guides/cryptodevs/dpaa_sec.rst | 2 +- doc/guides/faq/faq.rst | 23 ++--- doc/guides/nics/dpaa2.rst | 4 +- doc/guides/nics/qede.rst | 2 +- doc/guides/nics/sfc_efx.rst | 12 +-- doc/guides/nics/tap.rst | 2 +- lib/librte_eal/common/eal_common_log.c | 103 ++++++++++++++++--- lib/librte_eal/common/eal_common_options.c | 113 +++++++++++++-------- lib/librte_eal/common/eal_private.h | 6 ++ lib/librte_eal/common/include/rte_log.h | 44 +++----- lib/librte_eal/rte_eal_version.map | 7 ++ 13 files changed, 211 insertions(+), 113 deletions(-)
Signature · 1 line
2.17.0
PATCH v3 1/5 eal: make syslog facility table const
The mapping fro facility name to value can be const.
--- lib/librte_eal/common/eal_common_options.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/common/eal_common_options.c +1 -1
@@ -881,7 +881,7 @@ static int eal_parse_syslog(const char *facility, struct internal_config *conf) { int i; - static struct { + static const struct { const char *name; int value; } map[] = {
Signature · 1 line
2.17.0
PATCH v3 2/5 eal: allow symbolic log levels
In reply to Stephen HemmingerMuch easeier to remember names than numbers. Allows
--log-level=pmd.net.ixgbe.*,debug
--- lib/librte_eal/common/eal_common_options.c | 67 ++++++++++++++++------ 1 file changed, 50 insertions(+), 17 deletions(-)
lib/librte_eal/common/eal_common_options.c +50 -17
@@ -917,10 +917,49 @@ eal_parse_syslog(const char *facility, struct internal_config *conf) }static int -eal_parse_log_level (const char *arg) +eal_parse_log_priority (const char *level) { -char *end, *str, *type, *level; +static const char *const levels[] = { + [RTE_LOG_EMERG ] ="emergency" , + [RTE_LOG_ALERT ] ="alert" , + [RTE_LOG_CRIT ] ="critical" , + [RTE_LOG_ERR ] ="error" , + [RTE_LOG_WARNING ] ="warning" , + [RTE_LOG_NOTICE ] ="notice" , + [RTE_LOG_INFO ] ="info" , + [RTE_LOG_DEBUG ] ="debug" , + }; +size_t len =strlen (level);unsigned long tmp; +char *end; +unsigned int i; + +if (len ==0 ) +return -1 ; + +/* look for named values, skip 0 which is not a valid level */ +for (i =1 ; i <RTE_DIM (levels); i++) { +if (strncmp (levels[i], level, len) ==0 ) +return i; + } + +/* not a string, maybe it is numeric */ + errno =0 ; + tmp =strtoul (level, &end,0 ); + +/* check for errors */ +if (errno !=0 || end ==NULL || *end !='\0' || + tmp >=UINT32_MAX ) +return -1 ; + +return tmp; +} + +static int +eal_parse_log_level (const char *arg) +{ +char *str, *type, *level; +int priority; str =strdup (arg);if (str ==NULL )
@@ -934,23 +973,17 @@ eal_parse_log_level(const char *arg) level =strsep (&str,"," ); } - errno =0 ; - tmp =strtoul (level, &end,0 ); - -/* check for errors */ -if ((errno !=0 ) || (level[0 ] =='\0' ) || - end ==NULL || (*end !='\0' )) - goto fail; - -/* log_level is a uint32_t */ -if (tmp >=UINT32_MAX ) + priority =eal_parse_log_priority (level); +if (priority <0 ) { +fprintf (stderr,"invalid log priority: %s\n" , level); goto fail; + }if (type ==NULL ) { -rte_log_set_global_level (tmp); - }else if (rte_log_set_level_regexp (type, tmp) <0 ) { -printf ("cannot set log level %s,%lu\n" , - type, tmp); +rte_log_set_global_level (priority); + }else if (rte_log_set_level_regexp (type, priority) <0 ) { +fprintf (stderr,"cannot set log level %s,%d\n" , + type, priority); goto fail; }else {struct rte_eal_opt_loglevel *opt_ll;
@@ -972,7 +1005,7 @@ eal_parse_log_level(const char *arg) goto fail; } - opt_ll->level = tmp; + opt_ll->level = priority;TAILQ_INSERT_HEAD (&opt_loglevel_list, opt_ll, next); }
Signature · 1 line
2.17.0
PATCH v3 3/5 eal: make eal_log_level save private
In reply to Stephen HemmingerWe don't want format of eal log level saved values to be visible in ABI. Move to private storage in eal_common_log.
Includes minor optimization. Compile the regular expression for each log match once, rather than each time it is used.
--- lib/librte_eal/common/eal_common_log.c | 51 +++++++++++++++++----- lib/librte_eal/common/eal_common_options.c | 26 ++--------- lib/librte_eal/common/eal_private.h | 5 +++ lib/librte_eal/common/include/rte_log.h | 26 ----------- 4 files changed, 49 insertions(+), 59 deletions(-)
lib/librte_eal/common/eal_common_log.c +41 -10
@@ -23,8 +23,19 @@ struct rte_logs rte_logs = { .file = NULL, }; -/** Global list of valid EAL log level options */ -struct rte_eal_opt_loglevel_list opt_loglevel_list = +struct rte_eal_opt_loglevel { +/** Next list entry */ +TAILQ_ENTRY (rte_eal_opt_loglevel) next; +/** Compiled regular expression obtained from the option */ +regex_t re_match; +/** Log level value obtained from the option */ +uint32_t level; +}; + +TAILQ_HEAD (rte_eal_opt_loglevel_list, rte_eal_opt_loglevel); + +/** List of valid EAL log level options */ +static struct rte_eal_opt_loglevel_list opt_loglevel_list =TAILQ_HEAD_INITIALIZER (opt_loglevel_list);/* Stream to use for logging if rte_logs.file is NULL */
@@ -119,6 +130,33 @@ rte_log_set_level_regexp(const char *pattern, uint32_t level)return 0 ; } +/* +* Save the type (regexp string) and the loglevel +* in the global storage so that it could be used +* to configure dynamic logtypes which are absent +* at the moment of EAL option processing but may +* be registered during runtime. +*/ +int rte_log_save_regexp (const char *regex,int tmp) +{ +struct rte_eal_opt_loglevel *opt_ll; + + opt_ll =malloc (sizeof (*opt_ll)); +if (opt_ll ==NULL ) +return -1 ; + +if (regcomp (&opt_ll->re_match, regex,0 ) !=0 ) + goto fail; + + opt_ll->level = tmp; + +TAILQ_INSERT_HEAD (&opt_loglevel_list, opt_ll, next); +return 0 ; +fail: +free (opt_ll); +return -1 ; +} +/* get the current loglevel for the message being processed */ int rte_log_cur_msg_loglevel (void ) {
@@ -203,18 +241,11 @@ rte_log_register_type_and_pick_level(const char *name, uint32_t level_def)return type; TAILQ_FOREACH(opt_ll, &opt_loglevel_list, next) { -regex_t r; -if (opt_ll->level >RTE_LOG_DEBUG )continue ; -if (regcomp (&r, opt_ll->re_type,0 ) !=0 ) -continue ; - -if (regexec (&r, name,0 ,NULL ,0 ) ==0 ) +if (regexec (&opt_ll->re_match, name,0 ,NULL ,0 ) ==0 ) level = opt_ll->level; - -regfree (&r); } rte_logs.dynamic_types[type].loglevel = level;
lib/librte_eal/common/eal_common_options.c +3 -23
@@ -27,6 +27,7 @@#include "eal_internal_cfg.h" #include "eal_options.h" #include "eal_filesystem.h" +#include "eal_private.h" #define BITS_PER_HEX 4#define LCORE_OPT_LST 1
@@ -985,29 +986,8 @@ eal_parse_log_level(const char *arg)fprintf (stderr,"cannot set log level %s,%d\n" , type, priority); goto fail; - }else { -struct rte_eal_opt_loglevel *opt_ll; - -/* -* Save the type (regexp string) and the loglevel -* in the global storage so that it could be used -* to configure dynamic logtypes which are absent -* at the moment of EAL option processing but may -* be registered during runtime. -*/ - opt_ll =malloc (sizeof (*opt_ll)); -if (opt_ll ==NULL ) - goto fail; - - opt_ll->re_type =strdup (type); -if (opt_ll->re_type ==NULL ) { -free (opt_ll); - goto fail; - } - - opt_ll->level = priority; - -TAILQ_INSERT_HEAD (&opt_loglevel_list, opt_ll, next); + }else if (rte_log_save_regexp (type, priority) <0 ) { + goto fail; }free (str);
lib/librte_eal/common/eal_private.h +5 -0
@@ -82,6 +82,11 @@ int rte_eal_timer_init(void);*/ int rte_eal_log_init (const char *id,int facility); +/** +* Save the log regexp for later +*/ +int rte_log_save_regexp (const char *type,int priority); +/** * Init tail queues for non-EAL library structures. This is to allow * the rings, mempools, etc. lists to be shared among multiple processes
lib/librte_eal/common/include/rte_log.h +0 -26
@@ -85,32 +85,6 @@ extern struct rte_logs rte_logs;#define RTE_LOG_INFO 7U/**< Informational. */ #define RTE_LOG_DEBUG 8U/**< Debug-level messages. */ -/** -* @warning -* @b EXPERIMENTAL: this API may change without prior notice -* -* Entry definition for the storage to keep EAL log level options -* which are found to have log type regular expressions specified. -*/ -struct rte_eal_opt_loglevel { -/** Next list entry */ -TAILQ_ENTRY (rte_eal_opt_loglevel) next; -/** Regular expression string obtained from the option */ -char *re_type; -/** Log level value obtained from the option */ -uint32_t level; -}; - -TAILQ_HEAD (rte_eal_opt_loglevel_list, rte_eal_opt_loglevel); - -/** -* @warning -* @b EXPERIMENTAL: this API may change without prior notice -* -* Global list of EAL log level options featuring log type expressions -*/ -extern struct rte_eal_opt_loglevel_list opt_loglevel_list; -/** * Change the stream that will be used by the logging system. *
Signature · 1 line
2.17.0
PATCH v3 4/5 log: add ability to match dynamic log based on shell pattern
In reply to Stephen HemmingerRegular expressions are not the best way to match a hierarchical pattern like dynamic log levels. And the separator for dynamic log levels is period which is the regex wildcard character.
A better solution is to use filename matching 'globbing' so that log levels match like file paths. For compatibility, use colon to separate pattern match style arguments. For example:
--log-level 'pmd.net.virtio.*:debug'
This also makes the documentation match what really happens internally.
--- doc/guides/contributing/coding_style.rst | 4 +- doc/guides/nics/qede.rst | 2 +- lib/librte_eal/common/eal_common_log.c | 76 +++++++++++++++++----- lib/librte_eal/common/eal_common_options.c | 42 ++++++++---- lib/librte_eal/common/eal_private.h | 1 + lib/librte_eal/common/include/rte_log.h | 18 ++++- lib/librte_eal/rte_eal_version.map | 7 ++ 7 files changed, 114 insertions(+), 36 deletions(-)
doc/guides/contributing/coding_style.rst +2 -2
@@ -614,8 +614,8 @@ In the DPDK environment, use the logging interface provided: * is DEBUG) */ rte_log_set_level(my_logtype2, RTE_LOG_NOTICE); - /* enable all PMD logs (whose identifier string starts with "pmd") */ - rte_log_set_level_regexp("pmd.*", RTE_LOG_DEBUG); + /* enable all PMD logs (whose identifier string starts with "pmd.") */ + rte_log_set_level_pattern("pmd.*", RTE_LOG_DEBUG); /* log in debug level */ rte_log_set_global_level(RTE_LOG_DEBUG);
doc/guides/nics/qede.rst +1 -1
@@ -193,7 +193,7 @@ This section provides instructions to configure SR-IOV with Linux OS. #. Running testpmd - (Supply ``--log-level="pmd.net.qede.driver",7`` to view informational messages): + (Supply ``--log-level="pmd.net.qede.driver:7`` to view informational messages): Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` to run
lib/librte_eal/common/eal_common_log.c +60 -16
126 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +28 -14
68 lines not on this page · raw message
lib/librte_eal/common/eal_private.h +1 -0
@@ -86,6 +86,7 @@ int rte_eal_log_init(const char *id, int facility);* Save the log regexp for later */ int rte_log_save_regexp (const char *type,int priority); +int rte_log_save_pattern (const char *pattern,int priority);/** * Init tail queues for non-EAL library structures. This is to allow
lib/librte_eal/common/include/rte_log.h +15 -3
@@ -130,16 +130,28 @@ uint32_t rte_log_get_global_level(void);int rte_log_get_level (uint32_t logtype);/** -* Set the log level for a given type. +* Set the log level for a given type based on shell pattern. * * @param pattern -* The regexp identifying the log type. +* The match pattern identifying the log type. +* @param level +* The level to be set. +* @return +* 0 on success, a negative value if level is invalid. +*/ +int rte_log_set_level_pattern (const char *pattern,uint32_t level); + +/** +* Set the log level for a given type based on regular expression. +* +* @param regex +* The regular expression identifying the log type. * @param level * The level to be set. * @return * 0 on success, a negative value if level is invalid. */ -int rte_log_set_level_regexp (const char *pattern,uint32_t level); +int rte_log_set_level_regexp (const char *regex,uint32_t level);/** * Set the log level for a given type.
lib/librte_eal/rte_eal_version.map +7 -0
@@ -209,6 +209,13 @@ DPDK_18.02 { } DPDK_17.11; +DPDK_18.05 { + global: + + rte_log_set_level_pattern; + +} DPDK_18_02; + EXPERIMENTAL { global:
Signature · 1 line
2.17.0
Re: [PATCH v3 4/5] log: add ability to match dynamic log based on shell pattern
25/04/2018 05:17, Stephen Hemminger:
lib/librte_eal/rte_eal_version.map DPDK_18.02 { · line 217
Quoted 12 lines
lib/librte_eal/rte_eal_version.map +6
-0@@ -209,6 +209,13 @@ DPDK_18.02 { } DPDK_17.11; +DPDK_18.05 { + global: + + rte_log_set_level_pattern; + +} DPDK_18_02;
Must be DPDK_18.02 with a dot. Yes it is a trap.
Re: [PATCH v3 4/5] log: add ability to match dynamic log based on shell pattern
Quoted 7 lines · On Wed, 25 Apr 2018 11:38:21 +0200 Thomas Monjalon <[email protected]> wrote:
25/04/2018 05:17, Stephen Hemminger:
Quoted 6 lines
lib/librte_eal/rte_eal_version.map +0
-0@@ -209,6 +209,13 @@ DPDK_18.02 { } DPDK_17.11;
Quoted 10 lines
lib/librte_eal/rte_eal_version.map DPDK_18.02 { · line 217
Quoted 6 lines
lib/librte_eal/rte_eal_version.map +6
-0+DPDK_18.05 { + global: + + rte_log_set_level_pattern; + +} DPDK_18_02;Must be DPDK_18.02 with a dot. Yes it is a trap.
Also the EXPERIMENTAL section needs to now depend
on 18.05
PATCH v3 5/5 doc: update guides for current preferrred log level syntax
In reply to Stephen HemmingerUse symbolic names and match rather than numbers and regex in the guides.
--- doc/guides/cryptodevs/dpaa2_sec.rst | 2 +- doc/guides/cryptodevs/dpaa_sec.rst | 2 +- doc/guides/faq/faq.rst | 23 ++++++++++------------- doc/guides/nics/dpaa2.rst | 4 ++-- doc/guides/nics/qede.rst | 2 +- doc/guides/nics/sfc_efx.rst | 12 ++++++------ doc/guides/nics/tap.rst | 2 +- 7 files changed, 22 insertions(+), 25 deletions(-)
doc/guides/cryptodevs/dpaa2_sec.rst +1 -1
@@ -211,7 +211,7 @@ For enabling logs, use the following EAL parameter: .. code-block:: console - ./your_crypto_application <EAL args> --log-level=pmd.crypto.dpaa2,<level> + ./your_crypto_application <EAL args> --log-level=pmd.crypto.dpaa2:<level> Using ``crypto.dpaa2`` as log matching criteria, all Crypto PMD logs can be enabled which are lower than logging ``level``.
doc/guides/cryptodevs/dpaa_sec.rst +1 -1
@@ -154,7 +154,7 @@ For enabling logs, use the following EAL parameter: .. code-block:: console - ./your_crypto_application <EAL args> --log-level=pmd.crypto.dpaa,<level> + ./your_crypto_application <EAL args> --log-level=pmd.crypto.dpaa:<level> Using ``pmd.crypto.dpaa`` as log matching criteria, all Crypto PMD logs can be enabled which are lower than logging ``level``.
doc/guides/faq/faq.rst +10 -13
@@ -62,19 +62,16 @@ the wrong socket, the application simply will not start. On application startup, there is a lot of EAL information printed. Is there any way to reduce this? --------------------------------------------------------------------------------------------------- -Yes, the option ``--log-level=`` accepts one of these numbers: - -.. code-block:: c - - #define RTE_LOG_EMERG 1U /* System is unusable. */ - #define RTE_LOG_ALERT 2U /* Action must be taken immediately. */ - #define RTE_LOG_CRIT 3U /* Critical conditions. */ - #define RTE_LOG_ERR 4U /* Error conditions. */ - #define RTE_LOG_WARNING 5U /* Warning conditions. */ - #define RTE_LOG_NOTICE 6U /* Normal but significant condition. */ - #define RTE_LOG_INFO 7U /* Informational. */ - #define RTE_LOG_DEBUG 8U /* Debug-level messages. */ - +Yes, the option ``--log-level=`` accepts either symbolic names (or numbers): + +1. emergency +2. alert +3. critical +4. error +5. warning +6. notice +7. info +8. debug How can I tune my network application to achieve lower latency? ---------------------------------------------------------------
doc/guides/nics/dpaa2.rst +2 -2
@@ -547,7 +547,7 @@ For enabling logging for DPAA2 PMD, following log-level prefix can be used: .. code-block:: console - <dpdk app> <EAL args> --log-level=bus.fslmc,<level> -- ... + <dpdk app> <EAL args> --log-level=bus.fslmc:<level> -- ... Using ``bus.fslmc`` as log matching criteria, all FSLMC bus logs can be enabled which are lower than logging ``level``.
@@ -556,7 +556,7 @@ which are lower than logging ``level``. .. code-block:: console - <dpdk app> <EAL args> --log-level=pmd.net.dpaa2,<level> -- ... + <dpdk app> <EAL args> --log-level=pmd.net.dpaa2:<level> -- ... Using ``pmd.dpaa2`` as log matching criteria, all PMD logs can be enabled which are lower than logging ``level``.
doc/guides/nics/qede.rst +1 -1
@@ -193,7 +193,7 @@ This section provides instructions to configure SR-IOV with Linux OS. #. Running testpmd - (Supply ``--log-level="pmd.net.qede.driver:7`` to view informational messages): + (Supply ``--log-level="pmd.net.qede.driver:info`` to view informational messages): Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` to run
doc/guides/nics/sfc_efx.rst +6 -6
@@ -356,26 +356,26 @@ Dynamic Logging Parameters One may leverage EAL option "--log-level" to change default levels for the log types supported by the driver. The option is used with -an argument typically consisting of two parts separated by a comma. +an argument typically consisting of two parts separated by a colon. -Level value is the last part which takes an integer greater than 0. -Log type is the former part which may contain a regular expression. +Level value is the last part which takes a symbolic name (or integer). +Log type is the former part which may shell match syntax. Depending on the choice of the expression, the given log level may be used either for some specific log type or for a subset of types. SFC EFX PMD provides the following log types available for control: -- ``pmd.net.sfc.driver`` (default level is **6** - ``RTE_LOG_NOTICE``) +- ``pmd.net.sfc.driver`` (default level is **notice**) Affects driver-wide messages unrelated to any particular devices. -- ``pmd.net.sfc.main`` (default level is **6** - ``RTE_LOG_NOTICE``) +- ``pmd.net.sfc.main`` (default level is **notice**) Matches a subset of per-port log types registered during runtime. A full name for a particular type may be obtained by appending a dot and a PCI device identifier (``XXXX:XX:XX.X``) to the prefix. -- ``pmd.net.sfc.mcdi`` (default level is **6** - ``RTE_LOG_NOTICE``) +- ``pmd.net.sfc.mcdi`` (default level is **notice**) Extra logging of the communication with the NIC's management CPU. The format of the log is consumed by the Solarflare netlogdecode
doc/guides/nics/tap.rst +1 -1
@@ -166,7 +166,7 @@ Run pktgen from the pktgen directory in a terminal with a commandline like the following:: sudo ./app/app/x86_64-native-linuxapp-gcc/app/pktgen -l 1-5 -n 4 \ - --proc-type auto --log-level 8 --socket-mem 512,512 --file-prefix pg \ + --proc-type auto --log-level debug --socket-mem 512,512 --file-prefix pg \ --vdev=net_tap0 --vdev=net_tap1 -b 05:00.0 -b 05:00.1 \ -b 04:00.0 -b 04:00.1 -b 04:00.2 -b 04:00.3 \ -b 81:00.0 -b 81:00.1 -b 81:00.2 -b 81:00.3 \
Signature · 1 line
2.17.0
Quoted 6 lines
Stephen Hemminger (5):
eal: make syslog facility table const
eal: allow symbolic log levels
eal: makeeal_log_levelsave private
log: add ability to match dynamic log based on shell pattern
doc: update guides for current preferrred log level syntax
Applied (with small fix in map file), thanks