Netlink protocol specs
Subsystems (4)
Jakub Kicinski
maintainerPATCH v4 1/8 docs: add more netlink docs (incl. spec docs)
In reply to Jakub KicinskiAdd documentation about the upcoming Netlink protocol specs.
--
v4:
- add one more sentence on consistency of dumps
v3:
- spelling from Guillaume
v2:
- plenty -> plenty of
- type: unspec -> type: unused
- ECHO edits from Guillaume
--- Documentation/core-api/index.rst | 1 + Documentation/core-api/netlink.rst | 101 +++++ .../userspace-api/netlink/c-code-gen.rst | 107 +++++ .../netlink/genetlink-legacy.rst | 96 ++++ Documentation/userspace-api/netlink/index.rst | 5 + Documentation/userspace-api/netlink/specs.rst | 422 ++++++++++++++++++ MAINTAINERS | 2 + 7 files changed, 734 insertions(+) create mode 100644 Documentation/core-api/netlink.rst create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst create mode 100644 Documentation/userspace-api/netlink/specs.rst
Documentation/core-api/index.rst +1 -0
@@ -127,6 +127,7 @@ Documents that don't fit elsewhere or which have yet to be categorized. :maxdepth: 1 librs + netlink .. only:: subproject and html
Documentation/core-api/netlink.rst new file +101 -0
101 lines not on this page · raw message
Documentation/userspace-api/netlink/c-code-gen.rst new file +107 -0
107 lines not on this page · raw message
Documentation/userspace-api/netlink/genetlink-legacy.rst new file +96 -0
96 lines not on this page · raw message
Documentation/userspace-api/netlink/index.rst +5 -0
@@ -10,3 +10,8 @@ Netlink documentation for users. :maxdepth: 2 intro + specs + c-code-gen + genetlink-legacy + +See also :ref:`Documentation/core-api/netlink.rst <kernel_netlink>`.
Documentation/userspace-api/netlink/specs.rst new file +422 -0
422 lines not on this page · raw message
MAINTAINERS +2 -0
@@ -14563,8 +14563,10 @@ Q: https://patchwork.kernel.org/project/netdevbpf/list/ B: mailto:[email protected] T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git T: git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git +F: Documentation/core-api/netlink.rst F: Documentation/networking/ F: Documentation/process/maintainer-netdev.rst +F: Documentation/userspace-api/netlink/ F: include/linux/in.h F: include/linux/net.h F: include/linux/netdevice.h
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerv4 cover Netlink protocol specs
I think the Netlink proto specs are far along enough to merge. Filling in all attribute types and quirks will be an ongoing effort but we have enough to cover FOU so it's somewhat complete.
I fully intend to continue polishing the code but at the same time I'd like to start helping others base their work on the specs (e.g. DPLL) and need to start working on some new families myself.
That's the progress / motivation for merging. The RFC [1] has more of a high level blurb, plus I created a lot of documentation, I'm not going to repeat it here. There was also the talk at LPC [2].
[1] https://lore.kernel.org/all/[email protected]/
[2] https://youtu.be/9QkXIQXkaQk?t=2562
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/1
v4:
- spec improvements (patch 2)
- Python cleanup (patch 3)
- rename auto-gen files and use the right comment style
Jakub Kicinski (8):
docs: add more netlink docs (incl. spec docs)
netlink: add schemas for YAML specs
net: add basic C code generators for Netlink
netlink: add a proto specification for FOU
net: fou: regenerate the uAPI from the spec
net: fou: rename the source for linking
net: fou: use policy and operation tables generated from the spec
tools: ynl: add a completely generic client
Documentation/core-api/index.rst | 1 +
Documentation/core-api/netlink.rst | 101 +
Documentation/netlink/genetlink-c.yaml | 333 +++
Documentation/netlink/genetlink-legacy.yaml | 356 +++
Documentation/netlink/genetlink.yaml | 298 +++
Documentation/netlink/specs/fou.yaml | 128 +
.../userspace-api/netlink/c-code-gen.rst | 107 +
.../netlink/genetlink-legacy.rst | 96 +
Documentation/userspace-api/netlink/index.rst | 5 +
Documentation/userspace-api/netlink/specs.rst | 422 +++
MAINTAINERS | 3 +
include/uapi/linux/fou.h | 54 +-
net/ipv4/Makefile | 1 +
net/ipv4/{fou.c => fou_core.c} | 47 +-
net/ipv4/fou_nl.c | 48 +
net/ipv4/fou_nl.h | 25 +
tools/net/ynl/samples/cli.py | 47 +
tools/net/ynl/samples/ynl.py | 534 ++++
tools/net/ynl/ynl-gen-c.py | 2373 +++++++++++++++++
tools/net/ynl/ynl-regen.sh | 30 +
20 files changed, 4941 insertions(+), 68 deletions(-)
create mode 100644 Documentation/core-api/netlink.rst
create mode 100644 Documentation/netlink/genetlink-c.yaml
create mode 100644 Documentation/netlink/genetlink-legacy.yaml
create mode 100644 Documentation/netlink/genetlink.yaml
create mode 100644 Documentation/netlink/specs/fou.yaml
create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
create mode 100644 Documentation/userspace-api/netlink/specs.rst
rename net/ipv4/{fou.c => fou_core.c} (94%)
create mode 100644 net/ipv4/fou_nl.c
create mode 100644 net/ipv4/fou_nl.h
create mode 100755 tools/net/ynl/samples/cli.py
create mode 100644 tools/net/ynl/samples/ynl.py
create mode 100755 tools/net/ynl/ynl-gen-c.py
create mode 100755 tools/net/ynl/ynl-regen.shSignature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 2/8 netlink: add schemas for YAML specs
Add schemas for Netlink spec files. As described in the docs we have 4 "protocols" or compatibility levels, and each one comes with its own schema, but the more general / legacy schemas are superset of more modern ones: genetlink is the smallest followed by genetlink-c and genetlink-legacy. There is no schema for raw netlink, yet, I haven't found the time..
I don't know enough jsonschema to do inheritance or something but the repetition is not too bad. I hope.
---
I did not manage to get the if to work. I works like a charm at the level where the property in the condition is defined, but we also want to allow additional properties deeper in the structure depending on a value of a global property.
v4:
- fix the $id and $schema
- use "dependencies" to select only one of subset-of or name-prefix
- adjust the format of enums everywhere (spaces after/before brackets)
- add min value of integers where it makes sense
- add some $defs for lengths and unsigned int
- improve the pattern for lengths
- fix spelling
--- Documentation/netlink/genetlink-c.yaml | 333 ++++++++++++++++++ Documentation/netlink/genetlink-legacy.yaml | 356 ++++++++++++++++++++ Documentation/netlink/genetlink.yaml | 298 ++++++++++++++++ 3 files changed, 987 insertions(+) create mode 100644 Documentation/netlink/genetlink-c.yaml create mode 100644 Documentation/netlink/genetlink-legacy.yaml create mode 100644 Documentation/netlink/genetlink.yaml
Documentation/netlink/genetlink-c.yaml new file +333 -0
@@ -0,0 +1,333 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://kernel.org/schemas/netlink/genetlink-c.yaml# +$schema: https://json-schema.org/draft-07/schema + +# Common defines +$defs: + uint: + type: integer + minimum: 0 + len-or-define: + type: [ string, integer ] + pattern: ^[0-9A-Za-z_]+( - 1)?$ + minimum: 0 + +# Schema for specs +title: Protocol +description: Specification of a genetlink protocol +type: object +required: [ name, doc, attribute-sets, operations ] +additionalProperties: False +properties: + name: + description: Name of the genetlink family. + type: string + doc: + type: string + version: + description: Generic Netlink family version. Default is 1. + type: integer + minimum: 1 + protocol: + description: Schema compatibility level. Default is "genetlink". + enum: [ genetlink, genetlink-c ] + # Start genetlink-c + uapi-header: + description: Path to the uAPI header, default is linux/${family-name}.h + type: string + c-family-name: + description: Name of the define for the family name. + type: string + c-version-name: + description: Name of the define for the verion of the family. + type: string + max-by-define: + description: Makes the number of attributes and commands be specified by a define, not an enum value. + type: boolean + # End genetlink-c + + definitions: + description: List of type and constant definitions (enums, flags, defines). + type: array + items: + type: object + required: [ type, name ] + additionalProperties: False + properties: + name: + type: string + header: + description: For C-compatible languages, header which already defines this value. + type: string + type: + enum: [ const, enum, flags ] + doc: + type: string + # For const + value: + description: For const - the value. + type: [ string, integer ] + # For enum and flags + value-start: + description: For enum or flags the literal initializer for the first value. + type: [ string, integer ] + entries: + description: For enum or flags array of values. + type: array + items: + oneOf: + - type: string + - type: object + required: [ name ] + additionalProperties: False + properties: + name: + type: string + value: + type: integer + doc: + type: string + render-max: + description: Render the max members for this enum. + type: boolean + # Start genetlink-c + enum-name: + description: Name for enum, if empty no name will be used. + type: [ string, "null" ] + name-prefix: + description: For enum the prefix of the values, optional. + type: string + # End genetlink-c + + attribute-sets: + description: Definition of attribute spaces for this family. + type: array + items: + description: Definition of a single attribute space. + type: object + required: [ name, attributes ] + additionalProperties: False + properties: + name: + description: | + Name used when referring to this space in other definitions, not used outside of the spec. + type: string + name-prefix: + description: | + Prefix for the C enum name of the attributes. Default family[name]-set[name]-a- + type: string + enum-name: + description: Name for the enum type of the attribute. + type: string + doc: + description: Documentation of the space. + type: string + subset-of: + description: | + Name of another space which this is a logical part of. Sub-spaces can be used to define + a limited group of attributes which are used in a nest. + type: string + # Start genetlink-c + attr-cnt-name: + description: The explicit name for constant holding the count of attributes (last attr + 1). + type: string + attr-max-name: + description: The explicit name for last member of attribute enum. + type: string + # End genetlink-c + attributes: + description: List of attributes in the space. + type: array + items: + type: object + required: [ name, type ] + additionalProperties: False + properties: + name: + type: string + type: &attr-type + enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64, + string, nest, array-nest, nest-type-value ] + doc: + description: Documentation of the attribute. + type: string + value: + description: Value for the enum item representing this attribute in the uAPI. + $ref: '#/$defs/uint' + type-value: + description: Name of the value extracted from the type of a nest-type-value attribute. + type: array + items: + type: string + byte-order: + enum: [ little-endian, big-endian ] + multi-attr: + type: boolean + nested-attributes: + description: Name of the space (sub-space) used inside the attribute. + type: string + enum: + description: Name of the enum type used for the attribute. + type: string + enum-as-flags: + description: | + Treat the enum as flags. In most cases enum is either used as flags or as values. + Sometimes, however, both forms are necessary, in which case header contains the enum + form while specific attributes may request to convert the values into a bitfield. + type: boolean + checks: + description: Kernel input validation. + type: object + additionalProperties: False + properties: + flags-mask: + description: Name of the flags constant on which to base mask (unsigned scalar types only). + type: string + min: + description: Min value for an integer attribute. + type: integer + min-len: + description: Min length for a binary attribute. + $ref: '#/$defs/len-or-define' + max-len: + description: Max length for a string or a binary attribute. + $ref: '#/$defs/len-or-define' + sub-type: *attr-type + + # Make sure name-prefix does not appear in subsets (subsets inherit naming) + dependencies: + name-prefix: + not: + required: [ subset-of ] + subset-of: + not: + required: [ name-prefix ] + + operations: + description: Operations supported by the protocol. + type: object + required: [ list ] + additionalProperties: False + properties: + enum-model: + description: | + The model of assigning values to the operations. + "unified" is the recommended model where all message types belong + to a single enum. + "directional" has the messages sent to the kernel and from the kernel + enumerated separately. + "notify-split" has the notifications and request-response types in + different enums. + enum: [ unified, directional, notify-split ] + name-prefix: + description: | + Prefix for the C enum name of the command. The name is formed by concatenating + the prefix with the upper case name of the command, with dashes replaced by underscores. + type: string + enum-name: + description: Name for the enum type with commands. + type: string + async-prefix: + description: Same as name-prefix but used to render notifications and events to separate enum. + type: string + async-enum: + description: Name for the enum type with notifications/events. + type: string + list: + description: List of commands + type: array + items: + type: object + additionalProperties: False + required: [ name, doc ] + properties: + name: + description: Name of the operation, also defining its C enum value in uAPI. + type: string + doc: + description: Documentation for the command. + type: string + value: + description: Value for the enum in the uAPI. + $ref: '#/$defs/uint' + attribute-set: + description: | + Attribute space from which attributes directly in the requests and replies + to this command are defined. + type: string + flags: &cmd_flags + description: Command flags. + type: array + items: + enum: [ admin-perm ] + dont-validate: + description: Kernel attribute validation flags. + type: array + items: + enum: [ strict, dump ] + do: &subop-type + description: Main command handler. + type: object + additionalProperties: False + properties: + request: &subop-attr-list + description: Definition of the request message for a given command. + type: object + additionalProperties: False + properties: + attributes: + description: | + Names of attributes from the attribute-set (not full attribute + definitions, just names). + type: array + items: + type: string + reply: *subop-attr-list + pre: + description: Hook for a function to run before the main callback (pre_doit or start). + type: string + post: + description: Hook for a function to run after the main callback (post_doit or done). + type: string + dump: *subop-type + notify: + description: Name of the command sharing the reply type with this notification. + type: string + event: + type: object + additionalProperties: False + properties: + attributes: + description: Explicit list of the attributes for the notification. + type: array + items: + type: string + mcgrp: + description: Name of the multicast group generating given notification. + type: string + mcast-groups: + description: List of multicast groups. + type: object + required: [ list ] + additionalProperties: False + properties: + list: + description: List of groups. + type: array + items: + type: object + required: [ name ] + additionalProperties: False + properties: + name: + description: | + The name for the group, used to form the define and the value of the define. + type: string + # Start genetlink-c + c-define-name: + description: Override for the name of the define in C uAPI. + type: string + # End genetlink-c + flags: *cmd_flags
Documentation/netlink/genetlink-legacy.yaml new file +356 -0
356 lines not on this page · raw message
Documentation/netlink/genetlink.yaml new file +298 -0
298 lines not on this page · raw message
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 4/8 netlink: add a proto specification for FOU
In reply to Jakub KicinskiFOU has a reasonably modern Genetlink family. Add a spec.
--- Documentation/netlink/specs/fou.yaml | 128 +++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) create mode 100644 Documentation/netlink/specs/fou.yaml
Documentation/netlink/specs/fou.yaml new file +128 -0
@@ -0,0 +1,128 @@ +name: fou + +protocol: genetlink-legacy + +doc: | + Foo-over-UDP. + +c-family-name: fou-genl-name +c-version-name: fou-genl-version +max-by-define: true +kernel-policy: global + +definitions: + - + type: enum + name: encap_type + name-prefix: fou-encap- + enum-name: + entries: [ unspec, direct, gue ] + +attribute-sets: + - + name: fou + name-prefix: fou-attr- + attributes: + - + name: unspec + type: unused + - + name: port + type: u16 + byte-order: big-endian + - + name: af + type: u8 + - + name: ipproto + type: u8 + - + name: type + type: u8 + - + name: remcsum_nopartial + type: flag + - + name: local_v4 + type: u32 + - + name: local_v6 + type: binary + checks: + min-len: 16 + - + name: peer_v4 + type: u32 + - + name: peer_v6 + type: binary + checks: + min-len: 16 + - + name: peer_port + type: u16 + byte-order: big-endian + - + name: ifindex + type: s32 + +operations: + list: + - + name: unspec + doc: unused + + - + name: add + doc: Add port. + attribute-set: fou + + dont-validate: [ strict, dump ] + flags: [ admin-perm ] + + do: + request: &all_attrs + attributes: + - port + - ipproto + - type + - remcsum_nopartial + - local_v4 + - peer_v4 + - local_v6 + - peer_v6 + - peer_port + - ifindex + + - + name: del + doc: Delete port. + attribute-set: fou + + dont-validate: [ strict, dump ] + flags: [ admin-perm ] + + do: + request: &select_attrs + attributes: + - af + - ifindex + - port + - peer_port + - local_v4 + - peer_v4 + - local_v6 + - peer_v6 + + - + name: get + doc: Get tunnel info. + attribute-set: fou + dont-validate: [ strict, dump ] + + do: + request: *select_attrs + reply: *all_attrs + + dump: + reply: *all_attrs
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 8/8 tools: ynl: add a completely generic client
In reply to Jakub KicinskiAdd a CLI sample which can take in arbitrary request in JSON format, convert it to Netlink and do the inverse for output.
It's meant as a development tool primarily and perhaps for selftests which need to tickle netlink in a special way.
--- tools/net/ynl/samples/cli.py | 47 +++ tools/net/ynl/samples/ynl.py | 534 +++++++++++++++++++++++++++++++++++ 2 files changed, 581 insertions(+) create mode 100755 tools/net/ynl/samples/cli.py create mode 100644 tools/net/ynl/samples/ynl.py
tools/net/ynl/samples/cli.py new file +47 -0
@@ -0,0 +1,47 @@ +#!/usr/bin/env python +# SPDX-License-Identifier: BSD-3-Clause + +import argparse +import json +import pprint +import time + +from ynl import YnlFamily + + +def main(): + parser = argparse.ArgumentParser(description='YNL CLI sample') + parser.add_argument('--spec', dest='spec', type=str, required=True) + parser.add_argument('--schema', dest='schema', type=str) + parser.add_argument('--json', dest='json_text', type=str) + parser.add_argument('--do', dest='do', type=str) + parser.add_argument('--dump', dest='dump', type=str) + parser.add_argument('--sleep', dest='sleep', type=int) + parser.add_argument('--subscribe', dest='ntf', type=str) + args = parser.parse_args() + + attrs = {} + if args.json_text: + attrs = json.loads(args.json_text) + + ynl = YnlFamily(args.spec, args.schema) + + if args.ntf: + ynl.ntf_subscribe(args.ntf) + + if args.sleep: + time.sleep(args.sleep) + + if args.do or args.dump: + method = getattr(ynl, args.do if args.do else args.dump) + + reply = method(attrs, dump=bool(args.dump)) + pprint.PrettyPrinter().pprint(reply) + + if args.ntf: + ynl.check_ntf() + pprint.PrettyPrinter().pprint(ynl.async_msg_queue) + + +if __name__ == "__main__": + main()
tools/net/ynl/samples/ynl.py new file +534 -0
534 lines not on this page · raw message
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 5/8 net: fou: regenerate the uAPI from the spec
In reply to Jakub KicinskiRegenerate the FOU uAPI header from the YAML spec.
The flags now come before attributes which use them, and the comments for type disappear (coders should look at the spec instead).
--- include/uapi/linux/fou.h | 54 +++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 28 deletions(-)
include/uapi/linux/fou.h +26 -28
@@ -1,32 +1,37 @@/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ -/* fou.h - FOU Interface */ +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/fou.yaml */ +/* YNL-GEN uapi header */ #ifndef _UAPI_LINUX_FOU_H#define _UAPI_LINUX_FOU_H -/* NETLINK_GENERIC related info -*/ #define FOU_GENL_NAME "fou" -#define FOU_GENL_VERSION 0x1 +#define FOU_GENL_VERSION 1enum { -FOU_ATTR_UNSPEC , -FOU_ATTR_PORT ,/* u16 */ -FOU_ATTR_AF ,/* u8 */ -FOU_ATTR_IPPROTO ,/* u8 */ -FOU_ATTR_TYPE ,/* u8 */ -FOU_ATTR_REMCSUM_NOPARTIAL ,/* flag */ -FOU_ATTR_LOCAL_V4 ,/* u32 */ -FOU_ATTR_LOCAL_V6 ,/* in6_addr */ -FOU_ATTR_PEER_V4 ,/* u32 */ -FOU_ATTR_PEER_V6 ,/* in6_addr */ -FOU_ATTR_PEER_PORT ,/* u16 */ -FOU_ATTR_IFINDEX ,/* s32 */ - - __FOU_ATTR_MAX, +FOU_ENCAP_UNSPEC , +FOU_ENCAP_DIRECT , +FOU_ENCAP_GUE , }; -#define FOU_ATTR_MAX (__FOU_ATTR_MAX - 1) +enum { +FOU_ATTR_UNSPEC , +FOU_ATTR_PORT , +FOU_ATTR_AF , +FOU_ATTR_IPPROTO , +FOU_ATTR_TYPE , +FOU_ATTR_REMCSUM_NOPARTIAL , +FOU_ATTR_LOCAL_V4 , +FOU_ATTR_LOCAL_V6 , +FOU_ATTR_PEER_V4 , +FOU_ATTR_PEER_V6 , +FOU_ATTR_PEER_PORT , +FOU_ATTR_IFINDEX , + + __FOU_ATTR_MAX +}; +#define FOU_ATTR_MAX (__FOU_ATTR_MAX - 1)enum { FOU_CMD_UNSPEC,
@@ -34,15 +39,8 @@ enum {FOU_CMD_DEL ,FOU_CMD_GET , - __FOU_CMD_MAX, + __FOU_CMD_MAX }; - -enum { -FOU_ENCAP_UNSPEC , -FOU_ENCAP_DIRECT , -FOU_ENCAP_GUE , -}; - -#define FOU_CMD_MAX (__FOU_CMD_MAX - 1) +#define FOU_CMD_MAX (__FOU_CMD_MAX - 1)#endif /* _UAPI_LINUX_FOU_H */
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 7/8 net: fou: use policy and operation tables generated from the spec
In reply to Jakub KicinskiGenerate and plug in the spec-based tables.
A little bit of renaming is needed in the FOU code.
---
v4:
- use _ in the names of files instead of -
- regen with /* */ comments
--- net/ipv4/Makefile | 2 +- net/ipv4/fou_core.c | 47 +++++++------------------------------------- net/ipv4/fou_nl.c | 48 +++++++++++++++++++++++++++++++++++++++++++++ net/ipv4/fou_nl.h | 25 +++++++++++++++++++++++ 4 files changed, 81 insertions(+), 41 deletions(-) create mode 100644 net/ipv4/fou_nl.c create mode 100644 net/ipv4/fou_nl.h
net/ipv4/Makefile +1 -1
@@ -26,7 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o obj-$(CONFIG_NET_IPIP) += ipip.o gre-y := gre_demux.o -fou-y := fou_core.o +fou-y := fou_core.o fou_nl.o obj-$(CONFIG_NET_FOU) += fou.o obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o obj-$(CONFIG_NET_IPGRE) += ip_gre.o
net/ipv4/fou_core.c +7 -40
94 lines not on this page · raw message
net/ipv4/fou_nl.c new file +48 -0
48 lines not on this page · raw message
net/ipv4/fou_nl.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause */ +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/fou.yaml */ +/* YNL-GEN kernel header */ + +#ifndef _LINUX_FOU_GEN_H +#define _LINUX_FOU_GEN_H + +#include <net/netlink.h> +#include <net/genetlink.h> + +#include <linux/fou.h> + +/* Global operation policy for fou */ +extern const struct nla_policy fou_nl_policy[FOU_ATTR_IFINDEX +1 ]; + +/* Ops table for fou */ +extern const struct genl_small_ops fou_nl_ops[3 ]; + +int fou_nl_add_doit (struct sk_buff *skb,struct genl_info *info); +int fou_nl_del_doit (struct sk_buff *skb,struct genl_info *info); +int fou_nl_get_doit (struct sk_buff *skb,struct genl_info *info); +int fou_nl_get_dumpit (struct sk_buff *skb,struct netlink_callback *cb); + +#endif /* _LINUX_FOU_GEN_H */
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 3/8 net: add basic C code generators for Netlink
In reply to Jakub KicinskiCode generators to turn Netlink specs into C code. I'm definitely not proud of it.
The main generator is in Python, there's a bash script to regen all code-gen'ed files in tree after making spec changes.
--
v2: - use /* */ comments instead of //
v4:
- remove unnecessary f-strings
- use get(, default)
- improve exception for unsupported checks
- remove empty __init__
- (more) consistently use ternary / inline ifs
- use c_upper() / c_lower() everywhere
- rename c_kw -> _C_KW
- actually use /* */ comments
--- MAINTAINERS | 1 + tools/net/ynl/ynl-gen-c.py | 2373 ++++++++++++++++++++++++++++++++++++ tools/net/ynl/ynl-regen.sh | 30 + 3 files changed, 2404 insertions(+) create mode 100755 tools/net/ynl/ynl-gen-c.py create mode 100755 tools/net/ynl/ynl-regen.sh
MAINTAINERS +1 -0
@@ -14578,6 +14578,7 @@ F: include/uapi/linux/netdevice.h F: lib/net_utils.c F: lib/random32.c F: net/ +F: tools/net/ F: tools/testing/selftests/net/ NETWORKING [IPSEC]
tools/net/ynl/ynl-gen-c.py new file +2373 -0
2,373 lines not on this page · raw message
tools/net/ynl/ynl-regen.sh new file +30 -0
@@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: BSD-3-Clause + +TOOL=$(dirname $(realpath $0))/ynl-gen-c.py + +force= + +while [ ! -z "$1" ]; do + case "$1" in + -f ) force=yes; shift ;; + * ) echo "Unrecognized option '$1'"; exit 1 ;; + esac +done + +KDIR=$(dirname $(dirname $(dirname $(dirname $(realpath $0))))) + +files=$(git grep --files-with-matches '^/\* YNL-GEN \(kernel\|uapi\)') +for f in $files; do + # params: 0 1 2 3 + # $YAML YNL-GEN kernel $mode + params=( $(git grep -B1 -h '/\* YNL-GEN' $f | sed 's@/\*\(.*\)\*/@\1@') ) + + if [ $f -nt ${params[0]} -a -z "$force" ]; then + echo -e "\tSKIP $f" + continue + fi + + echo -e "\tGEN ${params[2]}\t$f" + $TOOL --mode ${params[2]} --${params[3]} --spec $KDIR/${params[0]} -o $f +done
Signature · 1 line
2.39.0
Jakub Kicinski
maintainerPATCH v4 6/8 net: fou: rename the source for linking
In reply to Jakub KicinskiWe'll need to link two objects together to form the fou module. This means the source can't be called fou, the build system expects fou.o to be the combined object.
---
net/ipv4/Makefile | 1 +
net/ipv4/{fou.c => fou_core.c} | 0
2 files changed, 1 insertion(+)
rename net/ipv4/{fou.c => fou_core.c} (100%)net/ipv4/Makefile +1 -0
@@ -26,6 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o obj-$(CONFIG_NET_IPIP) += ipip.o gre-y := gre_demux.o +fou-y := fou_core.o obj-$(CONFIG_NET_FOU) += fou.o obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o obj-$(CONFIG_NET_IPGRE) += ip_gre.o
net/ipv4/fou_core.c renamed from net/ipv4/fou.c +0 -0
Signature · 1 line
2.39.0
Paolo Abeni
maintainerQuoted 9 lines · Jakub Kicinski
I think the Netlink proto specs are far along enough to merge. Filling in all attribute types and quirks will be an ongoing effort but we have enough to cover FOU so it's somewhat complete.
I fully intend to continue polishing the code but at the same time I'd like to start helping others base their work on the specs (e.g. DPLL) and need to start working on some new families myself.
Quoted 8 lines
That's the progress / motivation for merging. The RFC [1] has more of a high level blurb, plus I created a lot of documentation, I'm not going to repeat it here. There was also the talk at LPC [2].
[1] https://lore.kernel.org/all/[email protected]/
[2] https://youtu.be/9QkXIQXkaQk?t=2562
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/1
My understanding is that there is agreement on this, so merging as-is.
As possible follow-ups, I think it would be nice to drop the generated files from the source tree and instead trigger the re-generation on- demand. Additionally the yaml files could include an SPDX-License- Identifier in an initial comments, I think.
Thanks,
Paolo
p.s. I guess we have new todo item for the mptcp protocol ;)
This series was applied to netdev/net-next.git (master) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (master) by Paolo Abeni <[email protected]>:
Quoted 10 lines · On Fri, 20 Jan 2023 09:50:33 -0800 you wrote:
I think the Netlink proto specs are far along enough to merge. Filling in all attribute types and quirks will be an ongoing effort but we have enough to cover FOU so it's somewhat complete.
I fully intend to continue polishing the code but at the same time I'd like to start helping others base their work on the specs (e.g. DPLL) and need to start working on some new families myself.
[...]
Here is the summary with links:
- [net-next,v4,1/8] docs: add more netlink docs (incl. spec docs)
https://git.kernel.org/netdev/net-next/c/9d6a65079c98 - [net-next,v4,2/8] netlink: add schemas for YAML specs https://git.kernel.org/netdev/net-next/c/e616c07ca518 - [net-next,v4,3/8] net: add basic C code generators for Netlink https://git.kernel.org/netdev/net-next/c/be5bea1cc0bf - [net-next,v4,4/8] netlink: add a proto specification for FOU https://git.kernel.org/netdev/net-next/c/4eb77b4ecd3c - [net-next,v4,5/8] net: fou: regenerate the uAPI from the spec https://git.kernel.org/netdev/net-next/c/3a330496baa8 - [net-next,v4,6/8] net: fou: rename the source for linking https://git.kernel.org/netdev/net-next/c/08d323234d10 - [net-next,v4,7/8] net: fou: use policy and operation tables generated from the spec https://git.kernel.org/netdev/net-next/c/1d562c32e439 - [net-next,v4,8/8] tools: ynl: add a completely generic client https://git.kernel.org/netdev/net-next/c/e4b48ed460d3
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Re: [PATCH net-next v4 5/8] net: fou: regenerate the uAPI from the spec
In reply to Jakub KicinskiFrom: Jakub Kicinski <[email protected]>
Date: Fri, 20 Jan 2023 09:50:38 -0800
Quoted 5 lines
Regenerate the FOU uAPI header from the YAML spec.
The flags now come before attributes which use them, and the comments for type disappear (coders should look at the spec instead).
Sorry I missed the whole history of this topic. Wanted to ask: if we can generate these headers and even C files, why ship the generated with the source code and not generate them during building? Or it's slow and/or requires some software etc.?
Quoted 6 lines
Signed-off-by Jakub Kicinski <[email protected]>--- include/uapi/linux/fou.h | 54 +++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 28 deletions(-)
[...]
include/uapi/linux/fou.h enum { · line 46
Quoted 4 lines
include/uapi/linux/fou.h +1
-1-#define FOU_CMD_MAX (__FOU_CMD_MAX - 1) +#define FOU_CMD_MAX (__FOU_CMD_MAX - 1)#endif /* _UAPI_LINUX_FOU_H */
Thanks,
Olek
Re: [PATCH net-next v4 5/8] net: fou: regenerate the uAPI from the spec
Quoted 5 lines · Alexander Lobakin
From: Jakub Kicinski <[email protected]>
Date: Fri, 20 Jan 2023 09:50:38 -0800Quoted 2 lines
Regenerate the FOU uAPI header from the YAML spec.
Quoted 9 lines
Quoted 3 lines
The flags now come before attributes which use them, and the comments for type disappear (coders should look at the spec instead).
Sorry I missed the whole history of this topic. Wanted to ask: if we can generate these headers and even C files, why ship the generated with the source code and not generate them during building? Or it's slow and/or requires some software etc.?
Currently it requires python 3 (3.6+, I'd think?).
Python is currently not documented as a build requirement in Documentation/process/changes.rst afaict.
johannes
Jakub Kicinski
maintainerRe: [PATCH net-next v4 5/8] net: fou: regenerate the uAPI from the spec
Quoted 15 lines · On Tue, 24 Jan 2023 19:50:40 +0100 Johannes Berg wrote:
Quoted 5 lines · Alexander Lobakin
From: Jakub Kicinski <[email protected]>
Date: Fri, 20 Jan 2023 09:50:38 -0800Quoted 2 lines
Regenerate the FOU uAPI header from the YAML spec.
Quoted 8 lines
Quoted 3 lines
The flags now come before attributes which use them, and the comments for type disappear (coders should look at the spec instead).
Sorry I missed the whole history of this topic. Wanted to ask: if we can generate these headers and even C files, why ship the generated with the source code and not generate them during building? Or it's slow and/or requires some software etc.?
Quoted 4 lines
Currently it requires python 3 (3.6+, I'd think?).
Python is currently not documented as a build requirement in
Documentation/process/changes.rstafaict.
Yes, I wanted to avoid bundling in changes which could be controversial. Whether code is generated during build or committed is something we can revisit at any point.
Re: [PATCH net-next v4 5/8] net: fou: regenerate the uAPI from the spec
From: Jakub Kicinski <[email protected]>
Date: Tue, 24 Jan 2023 16:40:07 -0800
Quoted 16 lines · On Tue, 24 Jan 2023 19:50:40 +0100 Johannes Berg wrote:
Quoted 15 lines · Alexander Lobakin
Quoted 13 lines
From: Jakub Kicinski <[email protected]>
Date: Fri, 20 Jan 2023 09:50:38 -0800Quoted 5 lines
Regenerate the FOU uAPI header from the YAML spec.
The flags now come before attributes which use them, and the comments for type disappear (coders should look at the spec instead).
Sorry I missed the whole history of this topic. Wanted to ask: if we can generate these headers and even C files, why ship the generated with the source code and not generate them during building? Or it's slow and/or requires some software etc.?
Quoted 8 lines
Quoted 4 lines
Currently it requires python 3 (3.6+, I'd think?).
Python is currently not documented as a build requirement in
Documentation/process/changes.rstafaict.Yes, I wanted to avoid bundling in changes which could be controversial. Whether code is generated during build or committed is something we can revisit at any point.
+, got it, thanks!
The only reason I asked is that I was hoping it would allow us to remove a couple thousand locs because why not :D
At least the atomic functions are designed the same way (the tree has the generated code committed/included), so it's totally fine.
I like what this series and the idea in general does, thanks!
Olek