Skip to content

update and simplify telemetry library.

Ciara Power dpdk-dev 5 versions – Applied

Status from this message: Applied, thanks for the nice work.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest18 patches 4 repliesApplied
  2. v418 patches 6 repliesSuperseded
  3. v317 patches 0 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v216 patches26 repliesSuperseded
v112 patches8 repliesSuperseded

Changes in v5 from the cover letter

- General cleanup to address comments on v4 patchset.

Patches in v5 Applied

#Subject and filesSizeRepliesApplied
1/18build: add arch-specific header path to global includesconfig/meson.build lib/librte_eal/arm/include/meson.build lib/librte_eal/ppc/include/meson.build +2 more+9 -130 replies
2/18telemetry: move code to metrics for later reuselib/Makefile lib/librte_metrics/Makefile lib/librte_metrics/meson.build +9 more+857 -8170 replies
3/18metrics: reduce code taken from telemetrylib/librte_metrics/rte_metrics_telemetry.c lib/librte_metrics/rte_metrics_telemetry.h lib/librte_telemetry/rte_telemetry.c +1 more+101 -4240 replies
4/18telemetry: invert dependency on metricslib/Makefile lib/librte_metrics/Makefile lib/librte_metrics/meson.build +11 more+82 -220 replies
5/18telemetry: add utility functions for creating jsonapp/test/Makefile app/test/meson.build app/test/test_telemetry_json.c +1 more+300 -00 replies
6/18telemetry: introduce new telemetry functionalitylib/librte_telemetry/Makefile lib/librte_telemetry/meson.build lib/librte_telemetry/rte_telemetry.c +4 more+479 -20 replies
7/18telemetry: add functions for returning callback datalib/librte_telemetry/Makefile lib/librte_telemetry/meson.build lib/librte_telemetry/rte_telemetry.h +2 more+283 -10 replies
8/18telemetry: add default callback commandslib/librte_telemetry/telemetry.c+51 -00 replies
9/18usertools: add new telemetry python scriptusertools/dpdk-telemetry.py usertools/meson.build+84 -10 replies
10/18ethdev: add callback support for telemetrylib/librte_ethdev/Makefile lib/librte_ethdev/meson.build lib/librte_ethdev/rte_ethdev.c+115 -00 replies
11/18rawdev: add callback support for telemetrylib/librte_rawdev/Makefile lib/librte_rawdev/meson.build lib/librte_rawdev/rte_rawdev.c+87 -00 replies
12/18examples/l3fwd-power: enable use of new telemetryexamples/l3fwd-power/Makefile examples/l3fwd-power/main.c examples/l3fwd-power/meson.build+50 -140 replies
13/18telemetry: introduce telemetry backward compatibilitylib/librte_metrics/rte_metrics_telemetry.c lib/librte_telemetry/Makefile lib/librte_telemetry/meson.build +4 more+460 -10 replies
14/18telemetry: remove existing telemetry fileslib/librte_metrics/rte_metrics_telemetry.c lib/librte_telemetry/Makefile lib/librte_telemetry/meson.build +9 more+23 -26010 replies
15/18lib: add telemetry as eal dependencyconfig/common_base examples/l3fwd-power/Makefile examples/l3fwd-power/main.c +24 more+69 -822 replies
16/18eal: remove rte-option infrastructurelib/librte_eal/common/eal_common_options.c lib/librte_eal/common/eal_private.h lib/librte_eal/common/meson.build +8 more+2 -2280 replies
17/18eal: add eal telemetry callbackslib/librte_eal/common/eal_common_options.c lib/librte_eal/common/eal_options.h lib/librte_eal/freebsd/eal.c +1 more+84 -00 replies
18/18doc: update telemetry documentationdoc/guides/howto/telemetry.rst doc/guides/linux_gsg/eal_args.include.rst doc/guides/linux_gsg/sys_reqs.rst +3 more+136 -590 replies

Reviews trailers per patch

All 18 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchKeith Wiles
SeriesA-b
1/18R-b
2/18R-b
3/18R-b
4/18R-b
5/18R-b
6/18R-b
7/18R-b
8/18R-b
9/18R-b
10/18R-b
11/18R-b
12/18R-b
13/18R-b
14/18R-b
15/18R-b
16/18R-b
17/18R-b
18/18R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v5 thread

Cover letter [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library.

v5:
- General cleanup to address comments on v4 patchset.

v4:
- Internal representation now uses generic data format instead of JSON.
- Help command added to provide help text for a command.

v3:
- Removed rte_option as it is no longer used.
- General cleanup.

v2:
- Added JSON API, and unit tests, to simplify creation of valid json responses from libraries.
- Added printing of basic info, including max output buffer size, app PID and DPDK version on connection.
- Added /info command to report that basic info post-connect. This replaces the eal version command from v1.
- Renamed stats to xstats in commands to allow a future generic "stats" call.
- Added documentation, including updating existing howto and adding programmers guide section and API docs.
- Added link status command for ethdev ports.
- Fixed windows builds.

This patchset extensively reworks the telemetry library adding new functionality and simplifying much of the existing code, while maintaining backward compatibility.

This work is based on the previously sent RFC for a "process info"
library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741
However, rather than creating a new library, this patchset takes that work and merges it into the existing telemetry library, as mentioned above.

The telemetry library as shipped in 19.11 is based upon the metrics library and outputs all statistics based on that as a source. However, this limits the telemetry output to only port-level statistics information, rather than allowing it to be used as a general scheme for telemetry information across all DPDK libraries.

With this patchset applied, rather than the telemetry library being responsible for pulling ethdev stats and pushing them into the metrics library for retrieval later, each library e.g. ethdev, rawdev, and even the metrics library itself (for backwards compatiblity) now handle their own stats. Any library or app can register a callback function with telemetry, which will be called if requested by the client connected via the telemetry socket. The callback function in the library/app then formats its stats, or other data, into a generic data structure, and returns it to telemetry to be sent to the client in JSON format.

To maintain backward compatibility, e.g. to allow the dpdk telemetry collectd plugin to continue to work, some of the existing telemetry code is kept, but is moved into the metrics library, and callbacks are registered with telemetry for the legacy commands that were supported previously.

The new version of the library, apart from the legacy interface support for backward compatibility, does not have an external dependency on the Jansson library, allowing the library to be enabled by default.

Note: In this version of the patchset, telemetry output is provided by the ethdev, rawdev and eal libraries, but this may be expanded further in later versions which are planned ahead of the merge deadline for 20.05

Bruce Richardson (8):
build: add arch-specific header path to global includes
telemetry: invert dependency on metrics
telemetry: add utility functions for creating json
telemetry: introduce new telemetry functionality
telemetry: add functions for returning callback data
usertools: add new telemetry python script
ethdev: add callback support for telemetry
eal: add eal telemetry callbacks

Ciara Power (10):
telemetry: move code to metrics for later reuse
metrics: reduce code taken from telemetry
telemetry: add default callback commands
rawdev: add callback support for telemetry examples/l3fwd-power: enable use of new telemetry
telemetry: introduce telemetry backward compatibility
telemetry: remove existing telemetry files
lib: add telemetry as eal dependency
eal: remove rte-option infrastructure
doc: update telemetry documentation

 app/test/Makefile                             |    2 +
 app/test/meson.build                          |    4 +
 app/test/test_telemetry_json.c                |  136 ++
 config/common_base                            |    2 +-
 config/meson.build                            |    7 -
 doc/guides/howto/telemetry.rst                |  108 +-
 doc/guides/linux_gsg/eal_args.include.rst     |    8 +
 doc/guides/linux_gsg/sys_reqs.rst             |    2 -
 doc/guides/prog_guide/index.rst               |    1 +
 doc/guides/prog_guide/telemetry_lib.rst       |   61 +
 doc/guides/rel_notes/release_20_05.rst        |   15 +
 examples/l3fwd-power/Makefile                 |    2 +-
 examples/l3fwd-power/main.c                   |   52 +-
 examples/l3fwd-power/meson.build              |    2 +-
 lib/Makefile                                  |   10 +-
 lib/librte_eal/arm/include/meson.build        |    2 -
 lib/librte_eal/common/eal_common_options.c    |   83 +-
 lib/librte_eal/common/eal_internal_cfg.h      |    1 +
 lib/librte_eal/common/eal_options.h           |    9 +
 lib/librte_eal/common/eal_private.h           |   28 -
 lib/librte_eal/common/meson.build             |    2 -
 lib/librte_eal/common/rte_option.c            |   95 -
 lib/librte_eal/freebsd/Makefile               |    2 +-
 lib/librte_eal/freebsd/eal.c                  |   26 +-
 lib/librte_eal/freebsd/meson.build            |    2 +-
 lib/librte_eal/include/meson.build            |    1 -
 lib/librte_eal/include/rte_option.h           |   72 -
 lib/librte_eal/linux/Makefile                 |    2 +-
 lib/librte_eal/linux/eal.c                    |   26 +-
 lib/librte_eal/linux/meson.build              |    2 +-
 lib/librte_eal/meson.build                    |    5 +-
 lib/librte_eal/ppc/include/meson.build        |    2 -
 lib/librte_eal/rte_eal_version.map            |    1 -
 lib/librte_eal/x86/include/meson.build        |    2 -
 lib/librte_ethdev/Makefile                    |    2 +-
 lib/librte_ethdev/meson.build                 |    2 +-
 lib/librte_ethdev/rte_ethdev.c                |  101 +
 lib/librte_metrics/Makefile                   |   10 +
 lib/librte_metrics/meson.build                |    9 +
 lib/librte_metrics/rte_metrics.c              |    6 +-
 lib/librte_metrics/rte_metrics.h              |    5 +-
 lib/librte_metrics/rte_metrics_telemetry.c    |  541 +++++
 lib/librte_metrics/rte_metrics_telemetry.h    |   65 +
 lib/librte_metrics/rte_metrics_version.map    |    7 +
 lib/librte_rawdev/Makefile                    |    2 +-
 lib/librte_rawdev/meson.build                 |    2 +
 lib/librte_rawdev/rte_rawdev.c                |   73 +
 lib/librte_telemetry/Makefile                 |   15 +-
 lib/librte_telemetry/meson.build              |   15 +-
 lib/librte_telemetry/rte_telemetry.c          | 1895 -----------------
 lib/librte_telemetry/rte_telemetry.h          |  236 +-
 lib/librte_telemetry/rte_telemetry_internal.h |  112 -
 lib/librte_telemetry/rte_telemetry_legacy.h   |   87 +
 lib/librte_telemetry/rte_telemetry_parser.c   |  682 ------
 lib/librte_telemetry/rte_telemetry_parser.h   |   15 -
 .../rte_telemetry_parser_test.c               |  533 -----
 .../rte_telemetry_socket_tests.h              |   36 -
 .../rte_telemetry_version.map                 |   14 +-
 lib/librte_telemetry/telemetry.c              |  411 ++++
 lib/librte_telemetry/telemetry_data.c         |  130 ++
 lib/librte_telemetry/telemetry_data.h         |   46 +
 lib/librte_telemetry/telemetry_json.h         |  158 ++
 lib/librte_telemetry/telemetry_legacy.c       |  226 ++
 lib/meson.build                               |    3 +-
 meson.build                                   |    9 +
 mk/rte.app.mk                                 |   11 +-
 usertools/dpdk-telemetry.py                   |   83 +
 usertools/meson.build                         |    2 +-
 68 files changed, 2663 insertions(+), 3656 deletions(-)
 create mode 100644 app/test/test_telemetry_json.c
 create mode 100644 doc/guides/prog_guide/telemetry_lib.rst
 delete mode 100644 lib/librte_eal/common/rte_option.c
 delete mode 100644 lib/librte_eal/include/rte_option.h
 create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c
 create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h
 delete mode 100644 lib/librte_telemetry/rte_telemetry.c
 delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h
 create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h
 delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c
 delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h
 delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c
 delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h
 create mode 100644 lib/librte_telemetry/telemetry.c
 create mode 100644 lib/librte_telemetry/telemetry_data.c
 create mode 100644 lib/librte_telemetry/telemetry_data.h
 create mode 100644 lib/librte_telemetry/telemetry_json.h
 create mode 100644 lib/librte_telemetry/telemetry_legacy.c
 create mode 100755 usertools/dpdk-telemetry.py
Signature · 1 line
2.17.1