tools: ynl: add install target
Subsystems (4)
cover tools: ynl: add install target
This series adds an install target for ynl. The python code is moved to a subdirectory, so it can be used as a package with flat layout, as well as directly from the tree.
To try the install as a non-root user you can run:
$ mkdir /tmp/myroot $ make DESTDIR=/tmp/myroot install
$ PATH="/tmp/myroot/usr/bin:$PATH" PYTHONPATH="$(ls -1d /tmp/myroot/usr/lib/python*/site-packages)" ynl --help
Proposed install layout is described in last patch.
Jan Stancek (5):
tools: ynl: move python code to separate sub-directory
tools: ynl: rename ynl-gen-[c|rst] to ynl_gen_[c|rst]
tools: ynl: add initial pyproject.toml for packaging
tools: ynl: add install target for specs and docs
tools: ynl: add main install target
tools/net/ynl/Makefile | 24 ++++++++++++-
tools/net/ynl/generated/.gitignore | 1 +
tools/net/ynl/generated/Makefile | 36 ++++++++++++++++---
tools/net/ynl/lib/.gitignore | 1 -
tools/net/ynl/lib/Makefile | 1 -
tools/net/ynl/pyproject.toml | 26 ++++++++++++++
tools/net/ynl/pyynl/__init__.py | 0
tools/net/ynl/{ => pyynl}/cli.py | 0
tools/net/ynl/{ => pyynl}/ethtool.py | 0
tools/net/ynl/pyynl/lib/.gitignore | 1 +
tools/net/ynl/{ => pyynl}/lib/__init__.py | 0
tools/net/ynl/{ => pyynl}/lib/nlspec.py | 0
tools/net/ynl/{ => pyynl}/lib/ynl.py | 0
.../ynl/{ynl-gen-c.py => pyynl/ynl_gen_c.py} | 0
.../{ynl-gen-rst.py => pyynl/ynl_gen_rst.py} | 0
tools/net/ynl/ynl-regen.sh | 2 +-
16 files changed, 84 insertions(+), 8 deletions(-)
create mode 100644 tools/net/ynl/pyproject.toml
create mode 100644 tools/net/ynl/pyynl/__init__.py
rename tools/net/ynl/{ => pyynl}/cli.py (100%)
rename tools/net/ynl/{ => pyynl}/ethtool.py (100%)
create mode 100644 tools/net/ynl/pyynl/lib/.gitignore
rename tools/net/ynl/{ => pyynl}/lib/__init__.py (100%)
rename tools/net/ynl/{ => pyynl}/lib/nlspec.py (100%)
rename tools/net/ynl/{ => pyynl}/lib/ynl.py (100%)
rename tools/net/ynl/{ynl-gen-c.py => pyynl/ynl_gen_c.py} (100%)
rename tools/net/ynl/{ynl-gen-rst.py => pyynl/ynl_gen_rst.py} (100%)Signature · 1 line
2.43.0
PATCH 1/5 tools: ynl: move python code to separate sub-directory
Move python code to a separate directory so it can be packaged as a python module.
---
tools/net/ynl/Makefile | 1 +
tools/net/ynl/generated/Makefile | 2 +-
tools/net/ynl/lib/.gitignore | 1 -
tools/net/ynl/lib/Makefile | 1 -
tools/net/ynl/pyynl/__init__.py | 0
tools/net/ynl/{ => pyynl}/cli.py | 0
tools/net/ynl/{ => pyynl}/ethtool.py | 0
tools/net/ynl/pyynl/lib/.gitignore | 1 +
tools/net/ynl/{ => pyynl}/lib/__init__.py | 0
tools/net/ynl/{ => pyynl}/lib/nlspec.py | 0
tools/net/ynl/{ => pyynl}/lib/ynl.py | 0
tools/net/ynl/{ => pyynl}/ynl-gen-c.py | 0
tools/net/ynl/{ => pyynl}/ynl-gen-rst.py | 0
tools/net/ynl/ynl-regen.sh | 2 +-
14 files changed, 4 insertions(+), 4 deletions(-)
create mode 100644 tools/net/ynl/pyynl/__init__.py
rename tools/net/ynl/{ => pyynl}/cli.py (100%)
rename tools/net/ynl/{ => pyynl}/ethtool.py (100%)
create mode 100644 tools/net/ynl/pyynl/lib/.gitignore
rename tools/net/ynl/{ => pyynl}/lib/__init__.py (100%)
rename tools/net/ynl/{ => pyynl}/lib/nlspec.py (100%)
rename tools/net/ynl/{ => pyynl}/lib/ynl.py (100%)
rename tools/net/ynl/{ => pyynl}/ynl-gen-c.py (100%)
rename tools/net/ynl/{ => pyynl}/ynl-gen-rst.py (100%)tools/net/ynl/Makefile +1 -0
@@ -21,5 +21,6 @@ clean distclean: fi \ done rm -f libynl.a + rm -rf pyynl/lib/__pycache__ .PHONY: all clean distclean $(SUBDIRS)
tools/net/ynl/generated/Makefile +1 -1
@@ -12,7 +12,7 @@ include ../Makefile.deps YNL_GEN_ARG_ethtool:=--user-header linux/ethtool_netlink.h \ --exclude-op stats-get -TOOL:=../ynl-gen-c.py +TOOL:=../pyynl/ynl-gen-c.py GENS_PATHS=$(shell grep -nrI --files-without-match \ 'protocol: netlink' \
tools/net/ynl/lib/.gitignore +0 -1
@@ -1,2 +1 @@ -__pycache__/ *.d
tools/net/ynl/lib/Makefile +0 -1
@@ -19,7 +19,6 @@ ynl.a: $(OBJS) clean: rm -f *.o *.d *~ - rm -rf __pycache__ distclean: clean rm -f *.a
tools/net/ynl/pyynl/__init__.py new file +0 -0
tools/net/ynl/pyynl/cli.py renamed from tools/net/ynl/cli.py +0 -0
tools/net/ynl/pyynl/ethtool.py renamed from tools/net/ynl/ethtool.py +0 -0
tools/net/ynl/pyynl/lib/.gitignore new file +1 -0
@@ -0,0 +1 @@ +__pycache__/
tools/net/ynl/pyynl/lib/__init__.py renamed from tools/net/ynl/lib/__init__.py +0 -0
tools/net/ynl/pyynl/lib/nlspec.py renamed from tools/net/ynl/lib/nlspec.py +0 -0
tools/net/ynl/pyynl/lib/ynl.py renamed from tools/net/ynl/lib/ynl.py +0 -0
tools/net/ynl/pyynl/ynl-gen-c.py renamed from tools/net/ynl/ynl-gen-c.py +0 -0
tools/net/ynl/pyynl/ynl-gen-rst.py renamed from tools/net/ynl/ynl-gen-rst.py +0 -0
tools/net/ynl/ynl-regen.sh +1 -1
@@ -1,7 +1,7 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause -TOOL=$(dirname $(realpath $0))/ynl-gen-c.py +TOOL=$(dirname $(realpath $0))/pyynl/ynl-gen-c.py force= search=
Signature · 1 line
2.43.0
PATCH 2/5 tools: ynl: rename ynl-gen-[c|rst] to ynl_gen_[c|rst]
In reply to Jan StancekDon't use dashes as these prevent easy imports for entrypoints.
---
tools/net/ynl/generated/Makefile | 2 +-
tools/net/ynl/pyynl/{ynl-gen-c.py => ynl_gen_c.py} | 0
tools/net/ynl/pyynl/{ynl-gen-rst.py => ynl_gen_rst.py} | 0
tools/net/ynl/ynl-regen.sh | 2 +-
4 files changed, 2 insertions(+), 2 deletions(-)
rename tools/net/ynl/pyynl/{ynl-gen-c.py => ynl_gen_c.py} (100%)
rename tools/net/ynl/pyynl/{ynl-gen-rst.py => ynl_gen_rst.py} (100%)tools/net/ynl/generated/Makefile +1 -1
@@ -12,7 +12,7 @@ include ../Makefile.deps YNL_GEN_ARG_ethtool:=--user-header linux/ethtool_netlink.h \ --exclude-op stats-get -TOOL:=../pyynl/ynl-gen-c.py +TOOL:=../pyynl/ynl_gen_c.py GENS_PATHS=$(shell grep -nrI --files-without-match \ 'protocol: netlink' \
tools/net/ynl/pyynl/ynl_gen_c.py renamed from tools/net/ynl/pyynl/ynl-gen-c.py +0 -0
tools/net/ynl/pyynl/ynl_gen_rst.py renamed from tools/net/ynl/pyynl/ynl-gen-rst.py +0 -0
tools/net/ynl/ynl-regen.sh +1 -1
@@ -1,7 +1,7 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause -TOOL=$(dirname $(realpath $0))/pyynl/ynl-gen-c.py +TOOL=$(dirname $(realpath $0))/pyynl/ynl_gen_c.py force= search=
Signature · 1 line
2.43.0
PATCH 3/5 tools: ynl: add initial pyproject.toml for packaging
In reply to Jan StancekAnd define entrypoints for cli tools.
--- tools/net/ynl/pyproject.toml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 tools/net/ynl/pyproject.toml
tools/net/ynl/pyproject.toml new file +26 -0
@@ -0,0 +1,26 @@ +[build-system] +requires = ["setuptools>=61.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "pyynl" +authors = [ + {name = "Donald Hunter", email = "[email protected]"}, + {name = "Jakub Kicinski", email = "[email protected]"}, +] +description = "yaml netlink (ynl)" +version = "0.0.1" +requires-python = ">=3.9" +dependencies = [ + "pyyaml==6.*", + "jsonschema==4.*" +] + +[tool.setuptools.packages.find] +include = ["pyynl", "pyynl.lib"] + +[project.scripts] +ynl = "pyynl.cli:main" +ynl-ethtool = "pyynl.ethtool:main" +ynl-gen-c = "pyynl.ynl_gen_c:main" +ynl-gen-rst = "pyynl.ynl_gen_rst:main"
Signature · 1 line
2.43.0
PATCH 4/5 tools: ynl: add install target for specs and docs
In reply to Jan StancekGenerate docs using ynl_gen_rst and add install target for both specs and generates rst files.
--- tools/net/ynl/generated/.gitignore | 1 + tools/net/ynl/generated/Makefile | 34 +++++++++++++++++++++++++++--- 2 files changed, 32 insertions(+), 3 deletions(-)
tools/net/ynl/generated/.gitignore +1 -0
@@ -1,2 +1,3 @@ *-user.c *-user.h +*.rst
tools/net/ynl/generated/Makefile +31 -3
@@ -7,12 +7,18 @@ ifeq ("$(DEBUG)","1") CFLAGS += -g -fsanitize=address -fsanitize=leak -static-libasan endif +INSTALL ?= install +prefix ?= /usr +datarootdir ?= $(prefix)/share +docdir ?= $(datarootdir)/doc + include ../Makefile.deps YNL_GEN_ARG_ethtool:=--user-header linux/ethtool_netlink.h \ --exclude-op stats-get TOOL:=../pyynl/ynl_gen_c.py +TOOL_RST:=../pyynl/ynl_gen_rst.py GENS_PATHS=$(shell grep -nrI --files-without-match \ 'protocol: netlink' \
@@ -22,7 +28,11 @@ SRCS=$(patsubst %,%-user.c,${GENS}) HDRS=$(patsubst %,%-user.h,${GENS}) OBJS=$(patsubst %,%-user.o,${GENS}) -all: protos.a $(HDRS) $(SRCS) $(KHDRS) $(KSRCS) $(UAPI) +SPECS_PATHS=$(wildcard ../../../../Documentation/netlink/specs/*.yaml) +SPECS=$(patsubst ../../../../Documentation/netlink/specs/%.yaml,%,${SPECS_PATHS}) +RSTS=$(patsubst %,%.rst,${SPECS}) + +all: protos.a $(HDRS) $(SRCS) $(KHDRS) $(KSRCS) $(UAPI) $(RSTS) protos.a: $(OBJS) @echo -e "\tAR $@"
@@ -40,8 +50,12 @@ protos.a: $(OBJS) @echo -e "\tCC $@" @$(COMPILE.c) $(CFLAGS_$*) -o $@ $< +%.rst: ../../../../Documentation/netlink/specs/%.yaml $(TOOL2) + @echo -e "\tGEN_RST $@" + @$(TOOL_RST) -o $@ -i $< + clean: - rm -f *.o + rm -f *.o *.rst distclean: clean rm -f *.c *.h *.a
@@ -49,5 +63,19 @@ distclean: clean regen: @../ynl-regen.sh -.PHONY: all clean distclean regen +install-rsts: + @echo -e "\tINSTALL generated docs" + @$(INSTALL) -d $(DESTDIR)$(docdir)/ynl + @$(INSTALL) -m 0644 $(RSTS) $(DESTDIR)$(docdir)/ynl/ + +install-specs: + @echo -e "\tINSTALL specs" + @$(INSTALL) -d $(DESTDIR)$(datarootdir)/ynl + @$(INSTALL) -m 0644 ../../../../Documentation/netlink/*.yaml $(DESTDIR)$(datarootdir)/ynl/ + @$(INSTALL) -d $(DESTDIR)$(datarootdir)/ynl/specs + @$(INSTALL) -m 0644 ../../../../Documentation/netlink/specs/*.yaml $(DESTDIR)$(datarootdir)/ynl/specs/ + +install: install-rsts install-specs + +.PHONY: all clean distclean regen install .DEFAULT_GOAL: all
Signature · 1 line
2.43.0
PATCH 5/5 tools: ynl: add main install target
In reply to Jan StancekThis will install C library, specs, rsts and pyynl. The initial structure is:
$ mkdir /tmp/myroot $ make DESTDIR=/tmp/myroot install
/usr /usr/lib64 /usr/lib64/libynl.a /usr/lib/python3.XX/site-packages/pyynl/* /usr/lib/python3.XX/site-packages/pyynl-0.0.1.dist-info/* /usr/bin /usr/bin/ynl /usr/bin/ynl-ethtool /usr/bin/ynl-gen-c /usr/bin/ynl-gen-rst /usr/share /usr/share/doc /usr/share/doc/ynl /usr/share/doc/ynl/*.rst /usr/share/ynl /usr/share/ynl/genetlink-c.yaml /usr/share/ynl/genetlink-legacy.yaml /usr/share/ynl/genetlink.yaml /usr/share/ynl/netlink-raw.yaml /usr/share/ynl/specs /usr/share/ynl/specs/devlink.yaml /usr/share/ynl/specs/dpll.yaml /usr/share/ynl/specs/ethtool.yaml /usr/share/ynl/specs/fou.yaml /usr/share/ynl/specs/handshake.yaml /usr/share/ynl/specs/mptcp_pm.yaml /usr/share/ynl/specs/netdev.yaml /usr/share/ynl/specs/net_shaper.yaml /usr/share/ynl/specs/nfsd.yaml /usr/share/ynl/specs/nftables.yaml /usr/share/ynl/specs/nlctrl.yaml /usr/share/ynl/specs/ovs_datapath.yaml /usr/share/ynl/specs/ovs_flow.yaml /usr/share/ynl/specs/ovs_vport.yaml /usr/share/ynl/specs/rt_addr.yaml /usr/share/ynl/specs/rt_link.yaml /usr/share/ynl/specs/rt_neigh.yaml /usr/share/ynl/specs/rt_route.yaml /usr/share/ynl/specs/rt_rule.yaml /usr/share/ynl/specs/tcp_metrics.yaml /usr/share/ynl/specs/tc.yaml /usr/share/ynl/specs/team.yaml
--- tools/net/ynl/Makefile | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
tools/net/ynl/Makefile +22 -1
@@ -1,5 +1,16 @@ # SPDX-License-Identifier: GPL-2.0 +include ../../scripts/Makefile.arch + +INSTALL ?= install +prefix ?= /usr +ifeq ($(LP64), 1) + libdir_relative = lib64 +else + libdir_relative = lib +endif +libdir ?= $(prefix)/$(libdir_relative) + SUBDIRS = lib generated samples all: $(SUBDIRS) libynl.a
@@ -22,5 +33,15 @@ clean distclean: done rm -f libynl.a rm -rf pyynl/lib/__pycache__ + rm -rf pyynl.egg-info + rm -rf build + +install: libynl.a + @echo -e "\tINSTALL libynl.a" + @$(INSTALL) -d $(DESTDIR)$(libdir) + @$(INSTALL) -m 0644 libynl.a $(DESTDIR)$(libdir)/libynl.a + @echo -e "\tINSTALL pyynl" + @pip install --prefix=$(DESTDIR)$(prefix) . + @make -C generated install -.PHONY: all clean distclean $(SUBDIRS) +.PHONY: all clean distclean install $(SUBDIRS)
Signature · 1 line
2.43.0
Re: [PATCH 1/5] tools: ynl: move python code to separate sub-directory
In reply to Jan StancekQuoted 2 lines · Jan Stancek
Move python code to a separate directory so it can be packaged as a python module.
There is a bunch of selftests that depend on this location:
make -C tools/testing/selftests TARGETS="drivers/net" TEST_PROGS=ping.py TTEST_GEN_PROGS="" run_tests
make: Entering directory '/home/virtme/testing-18/tools/testing/selftests'
make[1]: Entering directory '/home/virtme/testing-18/tools/testing/selftests/drivers/net' make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/virtme/testing-18/tools/testing/selftests/drivers/net' make[1]: Entering directory '/home/virtme/testing-18/tools/testing/selftests/drivers/net' TAP version 13
1..1
# overriding timeout to 90 # selftests: drivers/net: ping.py # Traceback (most recent call last): # File "/home/virtme/testing-18/tools/testing/selftests/drivers/net/./ping.py", line 4, in <module> # from lib.py import ksft_run, ksft_exit # File "/home/virtme/testing-18/tools/testing/selftests/drivers/net/lib/py/__init__.py", line 10, in <module> # from net.lib.py import * # File "/home/virtme/testing-18/tools/testing/selftests/net/lib/py/__init__.py", line 8, in <module> # from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily # File "/home/virtme/testing-18/tools/testing/selftests/net/lib/py/ynl.py", line 23, in <module> # from net.ynl.lib import YnlFamily, NlError # ImportError: cannot import name 'YnlFamily' from 'net.ynl.lib' (unknown location) not ok 1 selftests: drivers/net: ping.py # exit=1 make[1]: Leaving directory '/home/virtme/testing-18/tools/testing/selftests/drivers/net' make: Leaving directory '/home/virtme/testing-18/tools/testing/selftests' xx__-> echo $? 0 xx__-> echo scan > /sys/kernel/debug/kmemleak && cat /sys/kernel/debug/kmemleak xx__->
---
pw-bot: cr
Donald Hunter
maintainerRe: [PATCH 1/5] tools: ynl: move python code to separate sub-directory
In reply to Jan StancekQuoted 4 lines · Jan Stancek
Move python code to a separate directory so it can be packaged as a python module.
Quoted 8 lines
Signed-off-by Jan Stancek <[email protected]>--- tools/net/ynl/Makefile | 1 + tools/net/ynl/generated/Makefile | 2 +- tools/net/ynl/lib/.gitignore | 1 - tools/net/ynl/lib/Makefile | 1 - tools/net/ynl/pyynl/__init__.py | 0 tools/net/ynl/{ => pyynl}/cli.py | 0
Perhaps we could have a symlink to cli.py from the original location for compatibility with existing in-place usage. Same for ethtool.py and other user-facing scripts.
Quoted 7 lines
tools/net/ynl/{ => pyynl}/ethtool.py | 0
tools/net/ynl/pyynl/lib/.gitignore | 1 +
tools/net/ynl/{ => pyynl}/lib/__init__.py | 0
tools/net/ynl/{ => pyynl}/lib/nlspec.py | 0
tools/net/ynl/{ => pyynl}/lib/ynl.py | 0
tools/net/ynl/{ => pyynl}/ynl-gen-c.py | 0
tools/net/ynl/{ => pyynl}/ynl-gen-rst.py | 0The documentation build depends on this location. This patch is required to fix it:
Documentation/Makefile +0 -0
@@ -104,7 +104,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath
$(BUILDDIR)/$3/$4)
YNL_INDEX:=$(srctree)/Documentation/networking/netlink_spec/index.rst YNL_RST_DIR:=$(srctree)/Documentation/networking/netlink_spec YNL_YAML_DIR:=$(srctree)/Documentation/netlink/specs
-YNL_TOOL:=$(srctree)/tools/net/ynl/ynl-gen-rst.py +YNL_TOOL:=$(srctree)/tools/net/ynl/pyynl/ynl_gen_rst.py
YNL_RST_FILES_TMP := $(patsubst %.yaml,%.rst,$(wildcard
$(YNL_YAML_DIR)/*.yaml))
YNL_RST_FILES := $(patsubst $(YNL_YAML_DIR)%,$(YNL_RST_DIR)%,
$(YNL_RST_FILES_TMP))
Re: [PATCH 1/5] tools: ynl: move python code to separate sub-directory
In reply to Stanislav FomichevQuoted 6 lines · Stanislav Fomichev
Quoted 2 lines · Jan Stancek
Move python code to a separate directory so it can be packaged as a python module.
There is a bunch of selftests that depend on this location:
Sorry about that, I haven't realized other places it's already used at.
Quoted 3 lines
Perhaps we could have a symlink to
cli.pyfrom the original location for compatibility with existing in-place usage. Same forethtool.pyand other user-facing scripts.
I can add those, but I'd still update docs references with new path.
Thanks,
Jan