tools: ynl: pyynl: pull the --family resolution logic into the lib
Subsystems (3)
Jakub Kicinski
maintainerv2 cover tools: ynl: pyynl: pull the --family resolution logic into the lib
When packaging YNL as a system level utility we added a --family argument which auto-resolves the full spec path from a well known path in /usr/share. Spelling out full YAML spec files is at this point only done in-tree, for example in the selftests which need the very latest YAML. But the selftests have their own wrapping classes for each family so test authors aren't really bothered by having to spell the paths out.
Afford the same ease of use to the Python library users. Move the path resolution from the CLI code to the library. This simplifies the pyynl use by a lot:
from pyynl import YnlFamily
ynl = YnlFamily(family="netdev")
Unless I'm missing a trick, resolving the /usr/share path is hard enough for most users to lean towards shelling out to ynl CLI with --output-json, which is sad.
v2:
- fix the ethtool script (Donald)
- fix --family=X --validate (Clashiko)
v1: https://lore.kernel.org/[email protected]
Jakub Kicinski (2):
tools: ynl: pyynl: re-export the library API from the package root
tools: ynl: pyynl: pull the --family resolution logic into the lib
tools/net/ynl/pyynl/__init__.py | 9 +++++ tools/net/ynl/pyynl/cli.py | 56 +++++++---------------------- tools/net/ynl/pyynl/lib/__init__.py | 3 +- tools/net/ynl/pyynl/lib/nlspec.py | 22 ++++++++++-- tools/net/ynl/pyynl/lib/specdir.py | 51 ++++++++++++++++++++++++++ tools/net/ynl/pyynl/lib/ynl.py | 19 ++++++++-- tools/net/ynl/tests/ethtool.py | 7 +--- 7 files changed, 111 insertions(+), 56 deletions(-) create mode 100644 tools/net/ynl/pyynl/lib/specdir.py
Signature · 1 line
2.54.0
Jakub Kicinski
maintainerPATCH v2 1/2 tools: ynl: pyynl: re-export the library API from the package root
The public classes live in pyynl.lib, so users had to spell out
from pyynl.lib import YnlFamily
which I forget at least once a month. Re-export lib's API from the package __init__ so that
from pyynl import YnlFamily
works as well. I don't think there was a real reason not to do this?
--- tools/net/ynl/pyynl/__init__.py | 9 +++++++++ 1 file changed, 9 insertions(+)
tools/net/ynl/pyynl/__init__.py +9 -0
@@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause + +""" Python YNL (YAML Netlink) library. """ + +# Re-export the public library API so it can be imported straight from the +# package, e.g. `from pyynl import YnlFamily`. +# pylint: disable=wildcard-import,unused-wildcard-import +from .lib import * +from .lib import __all__
Signature · 1 line
2.54.0
Jakub Kicinski
maintainerPATCH v2 2/2 tools: ynl: pyynl: pull the --family resolution logic into the lib
In reply to Jakub KicinskiWhen packaging YNL as a system level utility we added a --family argument which auto-resolves the full spec path from a well known path in /usr/share. Spelling out full YAML spec files is at this point only done in-tree, for example in the selftests which need the very latest YAML. But the selftests have their own wrapping classes for each family so test authors aren't really bothered by having to spell the paths out.
Afford the same ease of use to the Python library users. Move the path resolution from the CLI code to the library. This simplifies the pyynl use by a lot:
from pyynl import YnlFamily
ynl = YnlFamily(family="netdev")
Unless I'm missing a trick, resolving the /usr/share path is hard enough for most users to lean towards shelling out to ynl CLI with --output-json, which is sad.
The ethtool script can now use family= instead of resolving the path (the helpers are removed from cli.py so this isn't just a cleanup).
---
v2:
- fix the ethtool script (Donald)
- fix --family=X --validate (Clashiko)
v1: https://lore.kernel.org/[email protected]
--- tools/net/ynl/pyynl/cli.py | 56 +++++++---------------------- tools/net/ynl/pyynl/lib/__init__.py | 3 +- tools/net/ynl/pyynl/lib/nlspec.py | 22 ++++++++++-- tools/net/ynl/pyynl/lib/specdir.py | 51 ++++++++++++++++++++++++++ tools/net/ynl/pyynl/lib/ynl.py | 19 ++++++++-- tools/net/ynl/tests/ethtool.py | 7 +--- 6 files changed, 102 insertions(+), 56 deletions(-) create mode 100644 tools/net/ynl/pyynl/lib/specdir.py
tools/net/ynl/pyynl/cli.py +12 -44
@@ -17,9 +17,7 @@ import textwrap # pylint: disable=no-name-in-module,wrong-import-position sys.path.append(pathlib.Path(__file__).resolve().parent.as_posix()) from lib import YnlFamily, Netlink, NlError, SpecFamily, SpecException, YnlException - -SYS_SCHEMA_DIR='/usr/share/ynl' -RELATIVE_SCHEMA_DIR='../../../../Documentation/netlink' +from lib import list_families # pylint: disable=too-few-public-methods,too-many-locals class Colors:
@@ -48,30 +46,6 @@ RELATIVE_SCHEMA_DIR='../../../../Documentation/netlink' """ Get terminal width in columns (80 if stdout is not a terminal) """ return shutil.get_terminal_size().columns -def schema_dir(): - """ - Return the effective schema directory, preferring in-tree before - system schema directory. - """ - script_dir = os.path.dirname(os.path.abspath(__file__)) - schema_dir_ = os.path.abspath(f"{script_dir}/{RELATIVE_SCHEMA_DIR}") - if not os.path.isdir(schema_dir_): - schema_dir_ = SYS_SCHEMA_DIR - if not os.path.isdir(schema_dir_): - raise YnlException(f"Schema directory {schema_dir_} does not exist") - return schema_dir_ - -def spec_dir(): - """ - Return the effective spec directory, relative to the effective - schema directory. - """ - spec_dir_ = schema_dir() + '/specs' - if not os.path.isdir(spec_dir_): - raise YnlException(f"Spec directory {spec_dir_} does not exist") - return spec_dir_ - - class YnlEncoder(json.JSONEncoder): """A custom encoder for emitting JSON with ynl-specific instance types""" def default(self, o):
@@ -272,9 +246,8 @@ RELATIVE_SCHEMA_DIR='../../../../Documentation/netlink' pprint.pprint(msg, width=term_width(), compact=True) if args.list_families: - for filename in sorted(os.listdir(spec_dir())): - if filename.endswith('.yaml'): - print(filename.removesuffix('.yaml')) + for family in list_families(): + print(family) return if args.no_schema:
@@ -284,28 +257,23 @@ RELATIVE_SCHEMA_DIR='../../../../Documentation/netlink' if args.json_text: attrs = json.loads(args.json_text) - if args.family: - spec = f"{spec_dir()}/{args.family}.yaml" - else: - spec = args.spec - if not os.path.isfile(spec): - raise YnlException(f"Spec file {spec} does not exist") + if args.spec and not os.path.isfile(args.spec): + raise YnlException(f"Spec file {args.spec} does not exist") + # Spec/YnlFamily will raise if both or neither spec and family are given if args.validate: + # Force validation even for installed specs (schema=True), unless the + # user explicitly picked a schema or opted out with --no-schema. + schema = True if args.schema is None else args.schema try: - SpecFamily(spec, args.schema) + SpecFamily(args.spec, schema_path=schema, family=args.family) except SpecException as error: print(error) sys.exit(1) return - if args.family: # set behaviour when using installed specs - if args.schema is None and spec.startswith(SYS_SCHEMA_DIR): - args.schema = '' # disable schema validation when installed - if args.process_unknown is None: - args.process_unknown = True - - ynl = YnlFamily(spec, args.schema, args.process_unknown, + ynl = YnlFamily(args.spec, schema=args.schema, family=args.family, + process_unknown=args.process_unknown, recv_size=args.dbg_small_recv) if args.dbg_small_recv: ynl.set_recv_dbg(True)
tools/net/ynl/pyynl/lib/__init__.py +2 -1
@@ -5,12 +5,13 @@ from .nlspec import SpecAttr, SpecAttrSet, SpecEnumEntry, SpecEnumSet, \ SpecFamily, SpecOperation, SpecSubMessage, SpecSubMessageFormat, \ SpecException +from .specdir import list_families from .ynl import YnlFamily, Netlink, NlError, NlPolicy, YnlException from .doc_generator import YnlDocGenerator __all__ = ["SpecAttr", "SpecAttrSet", "SpecEnumEntry", "SpecEnumSet", "SpecFamily", "SpecOperation", "SpecSubMessage", "SpecSubMessageFormat", - "SpecException", + "SpecException", "list_families", "YnlFamily", "Netlink", "NlError", "NlPolicy", "YnlException", "YnlDocGenerator"]
tools/net/ynl/pyynl/lib/nlspec.py +20 -2
@@ -12,6 +12,8 @@ import importlib import os import yaml as pyyaml +from .specdir import find_spec, SYS_SCHEMA_DIR + class SpecException(Exception): """Netlink spec exception.
@@ -444,7 +446,23 @@ import yaml as pyyaml except AttributeError: _yaml_loader = pyyaml.SafeLoader - def __init__(self, spec_path, schema_path=None, exclude_ops=None): + def __init__(self, spec_path=None, schema_path=None, exclude_ops=None, + family=None): + # schema_path selects how the spec is validated: + # None -- no preference: validate against the default schema, + # but trust (skip) installed specs selected by family= + # True -- always validate against the default schema + # path -- validate against this schema + # '' -- do not validate + if (spec_path is None) == (family is None): + raise ValueError("Specify exactly one of spec path or family name") + if family is not None: + spec_path = find_spec(family) + # Installed specs are assumed correct, so skip schema validation + # to save cycles unless the caller asked to validate. + if schema_path is None and spec_path.startswith(SYS_SCHEMA_DIR): + schema_path = '' + with open(spec_path, "r", encoding='utf-8') as stream: prefix = '# SPDX-License-Identifier: ' first = stream.readline().strip()
@@ -465,7 +483,7 @@ import yaml as pyyaml self.proto = self.yaml.get('protocol', 'genetlink') self.msg_id_model = self.yaml['operations'].get('enum-model', 'unified') - if schema_path is None: + if schema_path is None or schema_path is True: schema_path = os.path.dirname(os.path.dirname(spec_path)) + f'/{self.proto}.yaml' if schema_path: with open(schema_path, "r", encoding='utf-8') as stream:
tools/net/ynl/pyynl/lib/specdir.py new file +51 -0
51 lines not on this page · raw message
tools/net/ynl/pyynl/lib/ynl.py +16 -3
@@ -661,6 +661,14 @@ from .nlspec import SpecFamily """ YNL family -- a Netlink interface built from a YAML spec. + The spec can be selected either by file path (def_path=) or, when it + ships in a well-known location, by family name (family="xyz"); exactly + one of the two must be given. For example: + + from pyynl import YnlFamily + + ynl = YnlFamily(family="netdev") + Primary use of the class is to execute Netlink commands: ynl.<op_name>(attrs, ...)
@@ -691,11 +699,16 @@ from .nlspec import SpecFamily ynl.get_policy(op_name, mode) -- query kernel policy for an op """ - def __init__(self, def_path, schema=None, process_unknown=False, - recv_size=0): - super().__init__(def_path, schema) + def __init__(self, def_path=None, schema=None, process_unknown=None, + recv_size=0, family=None): + super().__init__(def_path, schema, family=family) self.include_raw = False + # Specs from /usr (selected by family=) have a higher chance of being + # stale, default to ignoring unknown attrs. In-tree users, and users + # who bundle the spec need to make a conscious decision. + if process_unknown is None: + process_unknown = family is not None self.process_unknown = process_unknown try:
tools/net/ynl/tests/ethtool.py +1 -6
@@ -11,12 +11,10 @@ import pathlib import pprint import sys import re -import os # pylint: disable=no-name-in-module,wrong-import-position sys.path.append(pathlib.Path(__file__).resolve().parent.parent.joinpath('pyynl').as_posix()) # pylint: disable=import-error -from cli import schema_dir, spec_dir from lib import YnlFamily
@@ -173,10 +171,7 @@ from lib import YnlFamily args = parser.parse_args() - spec = os.path.join(spec_dir(), 'ethtool.yaml') - schema = os.path.join(schema_dir(), 'genetlink-legacy.yaml') - - ynl = YnlFamily(spec, schema, recv_size=args.dbg_small_recv) + ynl = YnlFamily(family='ethtool', recv_size=args.dbg_small_recv) if args.dbg_small_recv: ynl.set_recv_dbg(True)
Signature · 1 line
2.54.0
Quoted 20 lines · Jakub Kicinski
When packaging YNL as a system level utility we added a --family argument which auto-resolves the full spec path from a well known path in
/usr/share. Spelling out full YAML spec files is at this point only done in-tree, for example in the selftests which need the very latest YAML. But the selftests have their own wrapping classes for each family so test authors aren't really bothered by having to spell the paths out.Afford the same ease of use to the Python library users. Move the path resolution from the CLI code to the library. This simplifies the pyynl use by a lot:
from pyynl import YnlFamily
ynl =
YnlFamily(family="netdev")Unless I'm missing a trick, resolving the
/usr/sharepath is hard enough for most users to lean towards shelling out to ynl CLI with --output-json, which is sad.
Quoted 5 lines
The ethtool script can now use family= instead of resolving the path (the helpers are removed from
cli.pyso this isn't just a cleanup).Signed-off-by Jakub Kicinski <[email protected]>
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Quoted 9 lines · On Tue, 30 Jun 2026 19:17:49 -0700 you wrote:
When packaging YNL as a system level utility we added a --family argument which auto-resolves the full spec path from a well known path in
/usr/share. Spelling out full YAML spec files is at this point only done in-tree, for example in the selftests which need the very latest YAML. But the selftests have their own wrapping classes for each family so test authors aren't really bothered by having to spell the paths out.[...]
Here is the summary with links:
- [net-next,v2,1/2] tools: ynl: pyynl: re-export the library API from the package root
https://git.kernel.org/netdev/net-next/c/df87e5c4e94e - [net-next,v2,2/2] tools: ynl: pyynl: pull the --family resolution logic into the lib https://git.kernel.org/netdev/net-next/c/10c90f1bba3a
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html