add custom logging to DPDK meson runs
Subsystems (2)
cover add custom logging to DPDK meson runs
While the meson.log file in each build directory contains lots of information, much of it is not of interest to developers, and it can be hard to see the explicitly messages from the dpdk build files among all the rest of the content. Futhermore, while at the end of the a configuration run we print out the list of enable/disabled components, this is never recorded anywhere for anyone who wants to check that build configuration later.
This patchset attempts to solve these issues by supporting a dpdk-meson.log file in each DPDK build folder containing the output from the DPDK meson.build files - including the final summary information for later checking.
Bruce Richardson (3):
build: add a dpdk-specific meson log file
build: shorten top-level build file
build: write messages to dpdk build log file
app/test/meson.build | 2 +- buildtools/meson.build | 1 + buildtools/write-log-entry.py | 23 +++++++++++++++++++++++ config/arm/meson.build | 2 ++ config/meson.build | 18 ++++++++++++++++++ config/x86/meson.build | 2 ++ drivers/meson.build | 8 +++++--- examples/meson.build | 1 + lib/meson.build | 8 +++++--- meson.build | 31 ++++++++++++------------------- 10 files changed, 70 insertions(+), 26 deletions(-) create mode 100644 buildtools/write-log-entry.py
Signature · 1 line
2.25.1
RFC PATCH 1/3 build: add a dpdk-specific meson log file
While meson itself writes a log file of all the tests it runs and the output of those, to help debug any build problems and provide more developer information it can be helpful to have a logfile created and controlled by the dpdk meson.build files themselves.
Since meson doesn't directly support this we need to use run_command to implement it manually. Using a python script to do the writing of the log makes things cross-platform, and as initial log entries we can add in the messages about the defined internal dependencies, since these are of developer interest only.
--- buildtools/meson.build | 1 + buildtools/write-log-entry.py | 23 +++++++++++++++++++++++ drivers/meson.build | 2 +- lib/meson.build | 2 +- meson.build | 8 ++++++-- 5 files changed, 32 insertions(+), 4 deletions(-) create mode 100644 buildtools/write-log-entry.py
buildtools/meson.build +1 -0
@@ -17,3 +17,4 @@ else endif map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') +log = py3 + [files('write-log-entry.py'), 'dpdk-meson.log']
buildtools/write-log-entry.py new file +23 -0
@@ -0,0 +1,23 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +from sys import argv, exit +from time import strftime +from os import environ +from os.path import join as path_join + +if len(argv) < 2: # error if no filename + exit(1) +if len(argv) < 3: # do nothing if no contents to log + exit(0) + +build_dir = environ.get('MESON_BUILD_ROOT', '.') +filename = path_join(build_dir, argv[1]) + +# append dates to headings +if argv[2].startswith('===='): + argv.append(strftime('@@ %Y-%m-%d %X')) + +with open(filename, 'a') as log: + print(" ".join(argv[2:]), file=log)
drivers/meson.build +1 -1
@@ -228,7 +228,7 @@ foreach subpath:subdirs set_variable('shared_@0@'.format(lib_name), shared_dep) set_variable('static_@0@'.format(lib_name), static_dep) dependency_name = ''.join(lib_name.split('rte_')) - message('drivers/@0@: Defining dependency "@1@"'.format( + run_command(log, 'drivers/@0@: Defining dependency "@1@"'.format( drv_path, dependency_name)) endif # build endforeach
lib/meson.build +1 -1
@@ -200,7 +200,7 @@ foreach l:libraries set_variable('shared_rte_' + name, shared_dep) set_variable('static_rte_' + name, static_dep) - message('lib/@0@: Defining dependency "@1@"'.format( + run_command(log, 'lib/@0@: Defining dependency "@1@"'.format( dir_name, name)) endif # if build endforeach
meson.build +6 -2
@@ -11,6 +11,11 @@ project('DPDK', 'C', meson_version: '>= 0.47.1' ) +# get tool paths and scripts +subdir('buildtools') + +run_command(log, '======= Starting meson =======') + # set up some global vars for compiler, platform, configuration, etc. cc = meson.get_compiler('c') dpdk_conf = configuration_data()
@@ -40,8 +45,7 @@ global_inc = include_directories('.', 'config', 'lib/librte_eal/@0@/include'.format(arch_subdir), ) -# do configuration and get tool paths -subdir('buildtools') +# do main build config subdir('config') # build libs and drivers
Signature · 1 line
2.25.1
RFC PATCH 2/3 build: shorten top-level build file
In reply to Bruce RichardsonMove some basic configuration settings out of the top-level meson.build file to the "config/meson.build" file, so as to keep the top-level file as clear and as short as possible.
--- config/meson.build | 18 ++++++++++++++++++ meson.build | 17 ----------------- 2 files changed, 18 insertions(+), 17 deletions(-)
config/meson.build +18 -0
@@ -14,6 +14,24 @@ foreach env:supported_exec_envs set_variable('is_' + env, exec_env == env) endforeach +# set the basic ISA +if host_machine.cpu_family().startswith('x86') + arch_subdir = 'x86' +elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') + arch_subdir = 'arm' +elif host_machine.cpu_family().startswith('ppc') + arch_subdir = 'ppc' +endif + +# configure the build, and make sure configs here and in root folder are +# able to be included in any file. We also store a global array of include dirs +# for passing to pmdinfogen scripts +global_inc = include_directories('.', '..', + '../lib/librte_eal/include', + '../lib/librte_eal/@0@/include'.format(host_machine.system()), + '../lib/librte_eal/@0@/include'.format(arch_subdir), +) + # MS linker requires special treatment. # TODO: use cc.get_linker_id() with Meson >= 0.54 is_ms_linker = is_windows and (cc.get_id() == 'clang')
meson.build +0 -17
@@ -28,23 +28,6 @@ dpdk_libs_disabled = [] dpdk_drvs_disabled = [] abi_version_file = files('ABI_VERSION') -if host_machine.cpu_family().startswith('x86') - arch_subdir = 'x86' -elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') - arch_subdir = 'arm' -elif host_machine.cpu_family().startswith('ppc') - arch_subdir = 'ppc' -endif - -# configure the build, and make sure configs here and in config folder are -# able to be included in any file. We also store a global array of include dirs -# for passing to pmdinfogen scripts -global_inc = include_directories('.', 'config', - 'lib/librte_eal/include', - 'lib/librte_eal/@0@/include'.format(host_machine.system()), - 'lib/librte_eal/@0@/include'.format(arch_subdir), -) - # do main build config subdir('config')
Signature · 1 line
2.25.1
RFC PATCH 3/3 build: write messages to dpdk build log file
In reply to Bruce RichardsonAdd additional info into the DPDK-specific logs by copying or replacing all "message()" calls with writes to the log file, making the build configuration, including lists of enabled/disabled drivers etc. available after the meson run has completed.
--- app/test/meson.build | 2 +- config/arm/meson.build | 2 ++ config/x86/meson.build | 2 ++ drivers/meson.build | 6 ++++-- examples/meson.build | 1 + lib/meson.build | 5 +++-- meson.build | 6 ++++++ 7 files changed, 19 insertions(+), 5 deletions(-)
app/test/meson.build +1 -1
@@ -439,7 +439,7 @@ dpdk_test = executable('dpdk-test', install: true) has_hugepage = run_command('has-hugepage.sh').stdout().strip() != '0' -message('hugepage availability: @0@'.format(has_hugepage)) +run_command(log, 'hugepage availability: @0@'.format(has_hugepage)) # some perf tests (eg: memcpy perf autotest)take very long # to complete, so timeout to 10 minutes
config/arm/meson.build +2 -0
@@ -182,6 +182,7 @@ else endforeach message('Implementer : ' + machine[0]) + run_command(log, 'config/arm/meson.build: Implementer = ' + machine[0]) foreach flag: machine[1] if flag.length() > 0 dpdk_conf.set(flag[0], flag[1])
@@ -205,6 +206,7 @@ else endforeach endif message(machine_args) +run_command(log, 'config/arm/meson.build: machine_args = ', machine_args) if (cc.get_define('__ARM_NEON', args: machine_args) != '' or cc.get_define('__aarch64__', args: machine_args) != '')
config/x86/meson.build +2 -0
@@ -13,6 +13,8 @@ endif # we require SSE4.2 for DPDK if cc.get_define('__SSE4_2__', args: machine_args) == '' message('SSE 4.2 not enabled by default, explicitly enabling') + run_command(log, 'config/x86/meson.build:', + 'SSE 4.2 not enabled by default, explicitly enabling') machine_args += '-msse4' endif
drivers/meson.build +4 -2
@@ -90,8 +90,10 @@ foreach subpath:subdirs if not is_variable('shared_rte_' + d) build = false reason = 'Missing internal dependency, "@0@"'.format(d) - message('Disabling @1@ [@2@]: missing internal dependency "@0@"' - .format(d, name, 'drivers/' + drv_path)) + log_msg = 'Disabling @1@ [@2@]: missing internal dependency "@0@"'.format( + d, name, 'drivers/' + drv_path) + message(log_msg) + run_command(log, log_msg) else shared_deps += [get_variable('shared_rte_' + d)] static_deps += [get_variable('static_rte_' + d)]
examples/meson.build +1 -0
@@ -105,5 +105,6 @@ foreach example: examples error('Cannot build requested example "' + name + '"') else message('Skipping example "' + name + '"') + run_command(log, 'Skipping example "' + name + '"') endif endforeach
lib/meson.build +3 -2
@@ -112,8 +112,9 @@ foreach l:libraries static_dep = shared_dep else if is_windows and use_function_versioning - message('@0@: Function versioning is not supported by Windows.' - .format(name)) + log_msg = '@0@: Function versioning is not supported by Windows.'.format(name) + message(log_msg) + run_command(log, log_msg) endif if use_function_versioning
meson.build +6 -0
@@ -15,6 +15,8 @@ project('DPDK', 'C', subdir('buildtools') run_command(log, '======= Starting meson =======') +run_command(log, 'Meson version', meson.version()) +run_command(log, 'DPDK version', meson.project_version()) # set up some global vars for compiler, platform, configuration, etc. cc = meson.get_compiler('c')
@@ -76,6 +78,7 @@ foreach lib:enabled_libs endif endforeach message(output_message + '\n') +run_command(log, output_message) output_message = '\n===============\nDrivers Enabled\n===============\n' foreach class:dpdk_driver_classes
@@ -92,6 +95,7 @@ foreach class:dpdk_driver_classes endforeach endforeach message(output_message + '\n') +run_command(log, output_message) output_message = '\n=================\nContent Skipped\n=================\n' output_message += '\nlibs:\n\t'
@@ -105,3 +109,5 @@ foreach drv:dpdk_drvs_disabled output_message += drv + ':\t' + reason + '\n\t' endforeach message(output_message + '\n') +run_command(log, output_message) +run_command(log, 'Meson done\n')
Signature · 1 line
2.25.1
Quoted 13 lines · Bruce Richardson
While the meson.log file in each build directory contains lots of information, much of it is not of interest to developers, and it can be hard to see the explicitly messages from the dpdk build files among all the rest of the content. Futhermore, while at the end of the a configuration run we print out the list of enable/disabled components, this is never recorded anywhere for anyone who wants to check that build configuration later.
This patchset attempts to solve these issues by supporting a dpdk-meson.log file in each DPDK build folder containing the output from the DPDK meson.build files - including the final summary information for later checking.
Quoted 5 lines
Bruce Richardson (3):
build: add a dpdk-specific meson log file
build: shorten top-level build file
build: write messages to dpdk build log file
Ping on this old RFC set.
From lack of interest, I'd suggest we drop this set from patchwork as "Rejected", but if others do thing it's worth pursuing I can investigate doing a v2, to update to latest codebase.
/Bruce
Quoted 19 lines · Bruce Richardson
Quoted 13 lines · Bruce Richardson
While the meson.log file in each build directory contains lots of information, much of it is not of interest to developers, and it can be hard to see the explicitly messages from the dpdk build files among all the rest of the content. Futhermore, while at the end of the a configuration run we print out the list of enable/disabled components, this is never recorded anywhere for anyone who wants to check that build configuration later.
This patchset attempts to solve these issues by supporting a dpdk-meson.log file in each DPDK build folder containing the output from the DPDK meson.build files - including the final summary information for later checking.
Quoted 5 lines
Bruce Richardson (3):
build: add a dpdk-specific meson log file
build: shorten top-level build file
build: write messages to dpdk build log file
Quoted 6 lines
Ping on this old RFC set.
From lack of interest, I'd suggest we drop this set from patchwork as "Rejected", but if others do thing it's worth pursuing I can investigate doing a v2, to update to latest codebase.
No further feedback received. Marking as rejected to avoid filling up patchwork.