Skip to contentSkip to conversation

Netlink protocol specs

PATCH Superseded Jakub Kicinski 6 participants 31 messages Jan 19, 2023 – Jan 20, 2023

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

v3 cover Netlink protocol specs

I think the Netlink proto specs are far along enough to merge. Filling in all attribute types and quirks will be an ongoing effort but we have enough to cover FOU so it's somewhat complete.

I fully intend to continue polishing the code but at the same time I'd like to start helping others base their work on the specs (e.g. DPLL) and need to start working on some new families myself.

That's the progress / motivation for merging. The RFC [1] has more of a high level blurb, plus I created a lot of documentation, I'm not going to repeat it here. There was also the talk at LPC [2].

[1] https://lore.kernel.org/all/[email protected]/
[2] https://youtu.be/9QkXIQXkaQk?t=2562
v2: https://lore.kernel.org/all/[email protected]/

Jakub Kicinski (8):
docs: add more netlink docs (incl. spec docs)
netlink: add schemas for YAML specs
net: add basic C code generators for Netlink
netlink: add a proto specification for FOU
net: fou: regenerate the uAPI from the spec
net: fou: rename the source for linking
net: fou: use policy and operation tables generated from the spec
tools: ynl: add a completely generic client

 Documentation/core-api/index.rst              |    1 +
 Documentation/core-api/netlink.rst            |   99 +
 Documentation/netlink/genetlink-c.yaml        |  318 +++
 Documentation/netlink/genetlink-legacy.yaml   |  346 +++
 Documentation/netlink/genetlink.yaml          |  284 ++
 Documentation/netlink/specs/fou.yaml          |  128 +
 .../userspace-api/netlink/c-code-gen.rst      |  107 +
 .../netlink/genetlink-legacy.rst              |   96 +
 Documentation/userspace-api/netlink/index.rst |    5 +
 Documentation/userspace-api/netlink/specs.rst |  422 +++
 MAINTAINERS                                   |    3 +
 include/uapi/linux/fou.h                      |   54 +-
 net/ipv4/Makefile                             |    1 +
 net/ipv4/fou-nl.c                             |   48 +
 net/ipv4/fou-nl.h                             |   25 +
 net/ipv4/{fou.c => fou_core.c}                |   47 +-
 tools/net/ynl/samples/cli.py                  |   47 +
 tools/net/ynl/samples/ynl.py                  |  534 ++++
 tools/net/ynl/ynl-gen-c.py                    | 2376 +++++++++++++++++
 tools/net/ynl/ynl-regen.sh                    |   30 +
 20 files changed, 4903 insertions(+), 68 deletions(-)
 create mode 100644 Documentation/core-api/netlink.rst
 create mode 100644 Documentation/netlink/genetlink-c.yaml
 create mode 100644 Documentation/netlink/genetlink-legacy.yaml
 create mode 100644 Documentation/netlink/genetlink.yaml
 create mode 100644 Documentation/netlink/specs/fou.yaml
 create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
 create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
 create mode 100644 Documentation/userspace-api/netlink/specs.rst
 create mode 100644 net/ipv4/fou-nl.c
 create mode 100644 net/ipv4/fou-nl.h
 rename net/ipv4/{fou.c => fou_core.c} (94%)
 create mode 100755 tools/net/ynl/samples/cli.py
 create mode 100644 tools/net/ynl/samples/ynl.py
 create mode 100755 tools/net/ynl/ynl-gen-c.py
 create mode 100755 tools/net/ynl/ynl-regen.sh
Signature · 1 line
2.39.0
author maintainer
#1 lore raw

PATCH v3 1/8 docs: add more netlink docs (incl. spec docs)

Add documentation about the upcoming Netlink protocol specs.

--
v3:
- spelling from Guillaume
v2:
- plenty -> plenty of
- type: unspec -> type: unused
- ECHO edits from Guillaume

---
 Documentation/core-api/index.rst              |   1 +
 Documentation/core-api/netlink.rst            |  99 ++++
 .../userspace-api/netlink/c-code-gen.rst      | 107 +++++
 .../netlink/genetlink-legacy.rst              |  96 ++++
 Documentation/userspace-api/netlink/index.rst |   5 +
 Documentation/userspace-api/netlink/specs.rst | 422 ++++++++++++++++++
 MAINTAINERS                                   |   2 +
 7 files changed, 732 insertions(+)
 create mode 100644 Documentation/core-api/netlink.rst
 create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
 create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
 create mode 100644 Documentation/userspace-api/netlink/specs.rst
Documentation/core-api/index.rst +1 -0
index 77eb775b8b42..7a3a08d81f11 100644
@@ -127,6 +127,7 @@ Documents that don't fit elsewhere or which have yet to be categorized.
    :maxdepth: 1
 
    librs
+   netlink
 
 .. only:: subproject and html
 
Documentation/core-api/netlink.rst new file +99 -0

99 lines not on this page · raw message

Documentation/userspace-api/netlink/c-code-gen.rst new file +107 -0

107 lines not on this page · raw message

Documentation/userspace-api/netlink/genetlink-legacy.rst new file +96 -0

96 lines not on this page · raw message

Documentation/userspace-api/netlink/index.rst +5 -0
index b0c21538d97d..be250110c8f6 100644
@@ -10,3 +10,8 @@ Netlink documentation for users.
    :maxdepth: 2
 
    intro
+   specs
+   c-code-gen
+   genetlink-legacy
+
+See also :ref:`Documentation/core-api/netlink.rst <kernel_netlink>`.
Documentation/userspace-api/netlink/specs.rst new file +422 -0

422 lines not on this page · raw message

MAINTAINERS +2 -0
index f82dd8d43c2b..3842915d863d 100644
@@ -14563,8 +14563,10 @@ Q:	https://patchwork.kernel.org/project/netdevbpf/list/
 B:	mailto:[email protected]
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
+F:	Documentation/core-api/netlink.rst
 F:	Documentation/networking/
 F:	Documentation/process/maintainer-netdev.rst
+F:	Documentation/userspace-api/netlink/
 F:	include/linux/in.h
 F:	include/linux/net.h
 F:	include/linux/netdevice.h
Signature · 1 line
2.39.0
#3 lore raw

Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

In reply to Jakub Kicinski

Documentation/core-api/netlink.rst line 41

Quoted 13 lines · Jakub Kicinski
Documentation/core-api/netlink.rst new file +12 -0
+Answer requests
+---------------
+
+Older families do not reply to all of the commands, especially NEW / ADD
+commands. User only gets information whether the operation succeeded or
+not via the ACK. Try to find useful data to return. Once the command is
+added whether it replies with a full message or only an ACK is uAPI and
+cannot be changed. It's better to err on the side of replying.
+
+Specifically NEW and ADD commands should reply with information identifying
+the created object such as the allocated object's ID (without having to
+resort to using ``NLM_F_ECHO``).

I'm a bit on the fence on this recommendation (as written).

Yeah, it's nice to reply to things ... but!

In userspace, you often request and wait for the ACK to see if the operation succeeded. This is basically necessary. But then it's complicated to wait for *another* message to see the ID.

We've actually started using the "cookie" in the extack to report an ID of an object/... back, see uses of nl_set_extack_cookie_u64() in the tree.

So I'm not sure I wholeheartedly agree with the recommendation to send a separate answer. We've done that, but it's ugly on both sender side in the kernel (requiring an extra message allocation, ideally at the beginning of the operation so you can fail gracefully, etc.) and on the receiver (having to wait for another message if the operation was successful; possibly actually having to check for that message *before* the ACK arrives.)

Documentation/core-api/netlink.rst line 54

Quoted 5 lines
Documentation/core-api/netlink.rst new file +5 -0
+Support dump consistency
+------------------------
+
+If iterating over objects during dump may skip over objects or repeat
+them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.

That could be a bit more fleshed out on _how_ to do that, if it's not somewhere else?

Documentation/core-api/netlink.rst line 71

Quoted 7 lines
Documentation/core-api/netlink.rst new file +7 -0
+kernel-policy
+~~~~~~~~~~~~~
+
+Defines if the kernel validation policy is per operation (``per-op``)
+or for the entire family (``global``). New families should use ``per-op``
+(default) to be able to narrow down the attributes accepted by a specific
+command.

Again I'm not sure I agree with that recommendation, but I know it's your preference :-)

(IMHO some things become more complex, such as having a "ifindex" in each one of them)

Documentation/core-api/netlink.rst line 82

Quoted 10 lines
Documentation/core-api/netlink.rst new file +10 -0
+checks
+------
+
+Documentation for the ``checks`` sub-sections of attribute specs.
+
+unterminated-ok
+~~~~~~~~~~~~~~~
+
+Accept strings without the null-termination (for legacy families only).
+Switches from the ``NLA_NUL_STRING`` to ``NLA_STRING`` policy type.

Should we even document all the legacy bits in such a prominent place?

(or just move it after max-len/min-len?)

Documentation/userspace-api/netlink/genetlink-legacy.rst line 20

Quoted 4 lines
Documentation/userspace-api/netlink/genetlink-legacy.rst new file +4 -0
+Attribute type nests
+--------------------
+
+New Netlink families should use ``multi-attr`` to define arrays.

Unrelated to this particular document, but ...

I'm all for this, btw, but maybe we should have a way of representing in the policy that an attribute is used as multi-attr for an array, and a way of exposing that in the policy export? Hmm. Haven't thought about this for a while.

johannes

#4 lore raw

Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

Quoted 12 lines · Johannes Berg

Documentation/core-api/netlink.rst line 71

Quoted 7 lines · Jakub Kicinski
Documentation/core-api/netlink.rst new file +7 -0
+kernel-policy
+~~~~~~~~~~~~~
+
+Defines if the kernel validation policy is per operation (``per-op``)
+or for the entire family (``global``). New families should use ``per-op``
+(default) to be able to narrow down the attributes accepted by a specific
+command.

Again I'm not sure I agree with that recommendation, but I know it's your preference :-)

Quoted 3 lines

(IMHO some things become more complex, such as having a "ifindex" in each one of them)

Per op policy is important because otherwise it can become impossible to safely extend a new attribute to commands over multiple kernel releases.

If you add an attribute like DEVLINK_ATTR_DRY_RUN in one kernel, and add it to devlink_cmd_foo.. its no longer really possible to add it to another command if the policy is global.

#5 lore raw

Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

Quoted 4 lines · Jacob Keller

Per op policy is important because otherwise it can become impossible to safely extend a new attribute to commands over multiple kernel releases.

Yeah. I think I just realised that my issues is more with the fact that per-op policy implies per-op attribute (identifier/number/name)space, and if you don't have that you have attribute duplication etc.

Anyway, it just feels superfluous, not really dangerous I guess :)

johannes

#6 lore raw

RE: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

Quoted 20 lines

-----Original Message-----
From: Johannes Berg <[email protected]>
Sent: Friday, January 20, 2023 1:11 AM
To: Keller, Jacob E <[email protected]>; Jakub Kicinski <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; Bagas Sanjaya <[email protected]>
Subject: Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

Quoted 4 lines · Jacob Keller

Per op policy is important because otherwise it can become impossible to safely extend a new attribute to commands over multiple kernel releases.

Quoted 7 lines

Yeah. I think I just realised that my issues is more with the fact that per-op policy implies per-op attribute (identifier/number/name)space, and if you don't have that you have attribute duplication etc.

Anyway, it just feels superfluous, not really dangerous I guess :)

Johannes

I think we want per-family attribute and op IDs, but still per-op policy that indicates which attributes are supported for a command.

This way you can re-use attributes for multiple commands, and they should behave the same semantics for those commands which support them, but the kernel can properly express which attributes are valid for a given command using the per-op policy.

I know some commands do use separate attributes for each command, or separate attribute space for sub-attributes in nests, but I am not sure how widespread that is. Devlink uses a single space for all its attributes.

Thanks,
Jake

author maintainer
#7 lore raw

Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

In reply to Johannes Berg
Quoted 17 lines · On Thu, 19 Jan 2023 21:29:22 +0100 Johannes Berg wrote:

Documentation/core-api/netlink.rst line 41

Quoted 13 lines · Jakub Kicinski
Documentation/core-api/netlink.rst new file +12 -0
+Answer requests
+---------------
+
+Older families do not reply to all of the commands, especially NEW / ADD
+commands. User only gets information whether the operation succeeded or
+not via the ACK. Try to find useful data to return. Once the command is
+added whether it replies with a full message or only an ACK is uAPI and
+cannot be changed. It's better to err on the side of replying.
+
+Specifically NEW and ADD commands should reply with information identifying
+the created object such as the allocated object's ID (without having to
+resort to using ``NLM_F_ECHO``).  

I'm a bit on the fence on this recommendation (as written).

Quoted 5 lines

Yeah, it's nice to reply to things ... but!

In userspace, you often request and wait for the ACK to see if the operation succeeded. This is basically necessary. But then it's complicated to wait for *another* message to see the ID.

Maybe you're looking at this from the perspective of a person tired of manually writing the user space code?

If the netlink message handling code is all auto-generated it makes no difference to the user...

Quoted 3 lines

We've actually started using the "cookie" in the extack to report an ID of an object/... back, see uses of nl_set_extack_cookie_u64() in the tree.

... and to the middle-layer / RPC / auto-generated library pulling stuff out from protocol messages and interpreting them in a special way is a typical netlink vortex of magic :(

Quoted 7 lines

So I'm not sure I wholeheartedly agree with the recommendation to send a separate answer. We've done that, but it's ugly on both sender side in the kernel (requiring an extra message allocation, ideally at the beginning of the operation so you can fail gracefully, etc.) and on the receiver (having to wait for another message if the operation was successful; possibly actually having to check for that message *before* the ACK arrives.)

Right, response is before ACK. It's not different to a GET command, really.

Quoted 8 lines

Documentation/core-api/netlink.rst line 54

Quoted 5 lines
Documentation/core-api/netlink.rst new file +5 -0
+Support dump consistency
+------------------------
+
+If iterating over objects during dump may skip over objects or repeat
+them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.  

That could be a bit more fleshed out on _how_ to do that, if it's not somewhere else?

I was thinking about adding a sentence like "To avoid consistency issues store your objects in an Xarray and correctly use the ID during iteration".. but it seems to hand-wavy. Really the coder needs to understand dumps quite well to get what's going on, and then the consistency is kinda obvious. IDK. Almost nobody gets this right :(

Quoted 13 lines

Documentation/core-api/netlink.rst line 82

Quoted 10 lines
Documentation/core-api/netlink.rst new file +10 -0
+checks
+------
+
+Documentation for the ``checks`` sub-sections of attribute specs.
+
+unterminated-ok
+~~~~~~~~~~~~~~~
+
+Accept strings without the null-termination (for legacy families only).
+Switches from the ``NLA_NUL_STRING`` to ``NLA_STRING`` policy type.  

Should we even document all the legacy bits in such a prominent place?

Quoted 1 line

(or just move it after max-len/min-len?)

This is kernel-only info (checks) so I figured it's good enough until someone takes a more serious stab at supporting legacy families.

Quoted 11 lines

Documentation/userspace-api/netlink/genetlink-legacy.rst line 20

Quoted 4 lines
Documentation/userspace-api/netlink/genetlink-legacy.rst new file +4 -0
+Attribute type nests
+--------------------
+
+New Netlink families should use ``multi-attr`` to define arrays.  

Unrelated to this particular document, but ...

I'm all for this, btw, but maybe we should have a way of representing in the policy that an attribute is used as multi-attr for an array, and a way of exposing that in the policy export? Hmm. Haven't thought about this for a while.

Informational-only or enforced? Enforcing this now would be another backward-compat nightmare :(

FWIW I have a set parked on a branch to add "required" bit to policies, so for per-op policies one can reject requests with missing attrs during validation.

#8 lore raw

Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

Quoted 18 lines · Jakub Kicinski
Quoted 17 lines · On Thu, 19 Jan 2023 21:29:22 +0100 Johannes Berg wrote:

Documentation/core-api/netlink.rst line 41

Quoted 13 lines · Jakub Kicinski
Documentation/core-api/netlink.rst new file +12 -0
+Answer requests
+---------------
+
+Older families do not reply to all of the commands, especially NEW / ADD
+commands. User only gets information whether the operation succeeded or
+not via the ACK. Try to find useful data to return. Once the command is
+added whether it replies with a full message or only an ACK is uAPI and
+cannot be changed. It's better to err on the side of replying.
+
+Specifically NEW and ADD commands should reply with information identifying
+the created object such as the allocated object's ID (without having to
+resort to using ``NLM_F_ECHO``).  

I'm a bit on the fence on this recommendation (as written).

Quoted 8 lines
Quoted 5 lines

Yeah, it's nice to reply to things ... but!

In userspace, you often request and wait for the ACK to see if the operation succeeded. This is basically necessary. But then it's complicated to wait for *another* message to see the ID.

Maybe you're looking at this from the perspective of a person tired of manually writing the user space code?

Heh, I guess :)

Quoted 2 lines

If the netlink message handling code is all auto-generated it makes no difference to the user...

True. And I guess we can hope that'll be the case for most users, though I doubt it :)

Quoted 7 lines
Quoted 3 lines

We've actually started using the "cookie" in the extack to report an ID of an object/... back, see uses of nl_set_extack_cookie_u64() in the tree.

... and to the middle-layer / RPC / auto-generated library pulling stuff out from protocol messages and interpreting them in a special way is a typical netlink vortex of magic :(

:)

Quoted 10 lines
Quoted 7 lines

So I'm not sure I wholeheartedly agree with the recommendation to send a separate answer. We've done that, but it's ugly on both sender side in the kernel (requiring an extra message allocation, ideally at the beginning of the operation so you can fail gracefully, etc.) and on the receiver (having to wait for another message if the operation was successful; possibly actually having to check for that message *before* the ACK arrives.)

Right, response is before ACK. It's not different to a GET command, really.

True.

Quoted 9 lines
Quoted 8 lines

Documentation/core-api/netlink.rst line 54

Quoted 5 lines
Documentation/core-api/netlink.rst new file +5 -0
+Support dump consistency
+------------------------
+
+If iterating over objects during dump may skip over objects or repeat
+them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.  

That could be a bit more fleshed out on _how_ to do that, if it's not somewhere else?

Quoted 5 lines

I was thinking about adding a sentence like "To avoid consistency issues store your objects in an Xarray and correctly use the ID during iteration".. but it seems to hand-wavy. Really the coder needs to understand dumps quite well to get what's going on, and then the consistency is kinda obvious. IDK. Almost nobody gets this right :(

Yeah agree, it's tricky one way or the other. To be honest I was thinking less of documenting the mechanics of the underlying code to ensure that, but rather of the mechanics of using the APIs to ensure that, i.e. how to use cb->seq and friends.

Quoted 9 lines
Quoted 6 lines

Unrelated to this particular document, but ...

I'm all for this, btw, but maybe we should have a way of representing in the policy that an attribute is used as multi-attr for an array, and a way of exposing that in the policy export? Hmm. Haven't thought about this for a while.

Informational-only or enforced? Enforcing this now would be another backward-compat nightmare :(

More informational - for userspace to know from policy dump that certain attributes have that property. With nested it's easy to know (there's a special nested-array type), but multi-attr there's no way to distinguish "is this one" and "is this multiple".

Now ... you might say you don't really care now since you want everything to be auto-generated and then you have it in the docs (actually, do you?), and that's a fair point.

Quoted 3 lines

FWIW I have a set parked on a branch to add "required" bit to policies, so for per-op policies one can reject requests with missing attrs during validation.

Nice. That might yet convince me of per-op policies ;-)

Though IMHO the namespace issue remains - I'd still not like to have 100 definitions of NL80211_ATTR_IFINDEX or similar.

johannes

author maintainer
#9 lore raw

Re: [PATCH net-next v3 1/8] docs: add more netlink docs (incl. spec docs)

Quoted 15 lines · On Fri, 20 Jan 2023 10:15:39 +0100 Johannes Berg wrote:
Quoted 9 lines
Quoted 8 lines

Documentation/core-api/netlink.rst line 54

Quoted 5 lines
Documentation/core-api/netlink.rst new file +5 -0
+Support dump consistency
+------------------------
+
+If iterating over objects during dump may skip over objects or repeat
+them - make sure to report dump inconsistency with ``NLM_F_DUMP_INTR``.    

That could be a bit more fleshed out on _how_ to do that, if it's not somewhere else?

Quoted 5 lines

I was thinking about adding a sentence like "To avoid consistency issues store your objects in an Xarray and correctly use the ID during iteration".. but it seems to hand-wavy. Really the coder needs to understand dumps quite well to get what's going on, and then the consistency is kinda obvious. IDK. Almost nobody gets this right :(

Quoted 4 lines

Yeah agree, it's tricky one way or the other. To be honest I was thinking less of documenting the mechanics of the underlying code to ensure that, but rather of the mechanics of using the APIs to ensure that, i.e. how to use cb->seq and friends.

I see. Let me add that.
My hope was to steer people towards data structures with stable indexes, so the problem doesn't occur. But I'll add a mention of the helpers.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

Unrelated to this particular document, but ...

I'm all for this, btw, but maybe we should have a way of representing in the policy that an attribute is used as multi-attr for an array, and a way of exposing that in the policy export? Hmm. Haven't thought about this for a while.

Quoted 7 lines
Quoted 2 lines

Informational-only or enforced? Enforcing this now would be another backward-compat nightmare :(

More informational - for userspace to know from policy dump that certain attributes have that property. With nested it's easy to know (there's a special nested-array type), but multi-attr there's no way to distinguish "is this one" and "is this multiple".

Makes sense.

Quoted 3 lines

Now ... you might say you don't really care now since you want everything to be auto-generated and then you have it in the docs (actually, do you?), and that's a fair point.

Have in the docs that we want everything to be auto-generated?

Quoted 8 lines
Quoted 3 lines

FWIW I have a set parked on a branch to add "required" bit to policies, so for per-op policies one can reject requests with missing attrs during validation.

Nice. That might yet convince me of per-op policies ;-)

Though IMHO the namespace issue remains - I'd still not like to have 100 definitions of NL80211_ATTR_IFINDEX or similar.

Yeah, there's different ways of dealing with it. The ethtool way is pretty neat - have a nest in each command for "common attrs" with ifindex and stuff in it.

author maintainer
#10 lore raw

PATCH v3 2/8 netlink: add schemas for YAML specs

In reply to Jakub Kicinski

Add schemas for Netlink spec files. As described in the docs we have 4 "protocols" or compatibility levels, and each one comes with its own schema, but the more general / legacy schemas are superset of more modern ones: genetlink is the smallest followed by genetlink-c and genetlink-legacy. There is no schema for raw netlink, yet, I haven't found the time..

I don't know enough jsonschema to do inheritance or something but the repetition is not too bad. I hope.

---
 Documentation/netlink/genetlink-c.yaml      | 318 ++++++++++++++++++
 Documentation/netlink/genetlink-legacy.yaml | 346 ++++++++++++++++++++
 Documentation/netlink/genetlink.yaml        | 284 ++++++++++++++++
 3 files changed, 948 insertions(+)
 create mode 100644 Documentation/netlink/genetlink-c.yaml
 create mode 100644 Documentation/netlink/genetlink-legacy.yaml
 create mode 100644 Documentation/netlink/genetlink.yaml
Documentation/netlink/genetlink-c.yaml new file +318 -0
new file mode 100644
index 000000000000..53a7fe94a47e
@@ -0,0 +1,318 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]
+additionalProperties: False
+properties:
+  name:
+    description: Name of the genetlink family.
+    type: string
+  doc:
+    type: string
+  version:
+    description: Generic Netlink family version. Default is 1.
+    type: integer
+  protocol:
+    description: Schema compatibility level. Default is "genetlink".
+    enum: [ genetlink, genetlink-c]
+  # Start genetlink-c
+  uapi-header:
+    description: Path to the uAPI header, default is linux/${family-name}.h
+    type: string
+  c-family-name:
+    description: Name of the define for the family name.
+    type: string
+  c-version-name:
+    description: Name of the define for the verion of the family.
+    type: string
+  max-by-define:
+    description: Makes the number of attributes and commands be specified by a define, not an enum value.
+    type: boolean
+  # End genetlink-c
+
+  definitions:
+    description: Array of type and constant definitions.
+    type: array
+    items:
+      type: object
+      required: [ type, name ]
+      additionalProperties: False
+      properties:
+        name:
+          type: string
+        header:
+          description: For C-compatible languages, header which already defines this value.
+          type: string
+        type:
+          enum: [ const, enum, flags ]
+        doc:
+          type: string
+        # For const
+        value:
+          description: For const - the value.
+          type: [ string, integer ]
+        # For enum and flags
+        value-start:
+          description: For enum or flags the literal initializer for the first value.
+          type: [ string, integer ]
+        entries:
+          description: For enum or flags array of values
+          type: array
+          items:
+            oneOf:
+              - type: string
+              - type: object
+                required: [ name ]
+                additionalProperties: False
+                properties:
+                  name:
+                    type: string
+                  value:
+                    type: integer
+                  doc:
+                    type: string
+        render-max:
+          description: Render the max members for this enum.
+          type: boolean
+        # Start genetlink-c
+        enum-name:
+          description: Name for enum, if empty no name will be used.
+          type: [ string, "null" ]
+        name-prefix:
+          description: For enum the prefix of the values, optional.
+          type: string
+        # End genetlink-c
+
+  attribute-sets:
+    description: Definition of attribute spaces for this family.
+    type: array
+    items:
+      description: Definition of a single attribute space.
+      type: object
+      required: [ name, attributes ]
+      additionalProperties: False
+      properties:
+        name:
+          description: |
+            Name used when referring to this space in other definitions, not used outside of YAML.
+          type: string
+        # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
+        name-prefix:
+          description: |
+            Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+          type: string
+        enum-name:
+          description: Name for the enum type of the attribute.
+          type: string
+        doc:
+          description: Documentation of the space.
+          type: string
+        subset-of:
+          description: |
+            Name of another space which this is a logical part of. Sub-spaces can be used to define
+            a limited group of attributes which are used in a nest.
+          type: string
+        # Start genetlink-c
+        attr-cnt-name:
+          description: The explicit name for constant holding the count of attributes (last attr + 1).
+          type: string
+        attr-max-name:
+          description: The explicit name for last member of attribute enum.
+          type: string
+        # End genetlink-c
+        attributes:
+          description: List of attributes in the space.
+          type: array
+          items:
+            type: object
+            required: [ name, type ]
+            additionalProperties: False
+            properties:
+              name:
+                type: string
+              type: &attr-type
+                enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+                        string, nest, array-nest, nest-type-value ]
+              doc:
+                description: Documentation of the attribute.
+                type: string
+              value:
+                description: Value for the enum item representing this attribute in the uAPI.
+                type: integer
+              type-value:
+                description: Name of the value extracted from the type of a nest-type-value attribute.
+                type: array
+                items:
+                  type: string
+              byte-order:
+                enum: [little-endian, big-endian]
+              multi-attr:
+                type: boolean
+              nested-attributes:
+                description: Name of the space (sub-space) used inside the attribute.
+                type: string
+              enum:
+                description: Name of the enum type used for the atttribute.
+                type: string
+              enum-as-flags:
+                description: |
+                  Treat the enum as flags. In most cases enum is either used as flags or as values.
+                  Sometimes, however, both forms are necessary, in which case header contains the enum
+                  form while specific attributes may request to convert the values into a bitfield.
+                type: boolean
+              checks:
+                description: Kernel input validation.
+                type: object
+                additionalProperties: False
+                properties:
+                  flags-mask:
+                    description: Name of the flags constant on which to base mask (unsigned scalar types only).
+                    type: string
+                  min:
+                    description: Min value for an integer attribute.
+                    type: integer
+                  min-len:
+                    description: Min length for a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+                  max-len:
+                    description: Max length for a string or a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+              sub-type: *attr-type
+  operations:
+    description: Operations supported by the protocol.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      enum-model:
+        description: |
+          The model of assigning values to the operations.
+          "unified" is the recommended model where all message types belong
+          to a single enum.
+          "directional" has the messages sent to the kernel and from the kernel
+          enumerated separately.
+          "notify-split" has the notifications and request-response types in
+          different enums.
+        enum: [ unified, directional, notify-split ]
+      name-prefix:
+        description: |
+          Prefix for the C enum name of the command. The name is formed by concatenating
+          the prefix with the upper case name of the command, with dashes replaced by underscores.
+        type: string
+      enum-name:
+        description: Name for the enum type with commands.
+        type: string
+      async-prefix:
+        description: Same as name-prefix but used to render notifications and events to separate enum.
+        type: string
+      async-enum:
+        description: Name for the enum type with notifications/events.
+        type: string
+      list:
+        description: List of commands
+        type: array
+        items:
+          type: object
+          additionalProperties: False
+          required: [ name, doc ]
+          properties:
+            name:
+              description: Name of the operation, also defining its C enum value in uAPI.
+              type: string
+            doc:
+              description: Documentation for the command.
+              type: string
+            value:
+              description: Value for the enum in the uAPI.
+              type: integer
+            attribute-set:
+              description: |
+                Attribute space from which attributes directly in the requests and replies
+                to this command are defined.
+              type: string
+            flags: &cmd_flags
+              description: Command flags.
+              type: array
+              items:
+                enum: [ admin-perm ]
+            dont-validate:
+              description: Kernel attribute validation flags.
+              type: array
+              items:
+                enum: [ strict, dump ]
+            do: &subop-type
+              description: Main command handler.
+              type: object
+              additionalProperties: False
+              properties:
+                request: &subop-attr-list
+                  description: Definition of the request message for a given command.
+                  type: object
+                  additionalProperties: False
+                  properties:
+                    attributes:
+                      description: |
+                        Names of attributes from the attribute-set (not full attribute
+                        definitions, just names).
+                      type: array
+                      items:
+                        type: string
+                reply: *subop-attr-list
+                pre:
+                  description: Hook for a function to run before the main callback (pre_doit or start).
+                  type: string
+                post:
+                  description: Hook for a function to run after the main callback (post_doit or done).
+                  type: string
+            dump: *subop-type
+            notify:
+              description: Name of the command sharing the reply type with this notification.
+              type: string
+            event:
+              type: object
+              additionalProperties: False
+              properties:
+                attributes:
+                  description: Explicit list of the attributes for the notification.
+                  type: array
+                  items:
+                    type: string
+            mcgrp:
+              description: Name of the multicast group generating given notification.
+              type: string
+  mcast-groups:
+    description: List of multicast groups.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      list:
+        description: List of groups.
+        type: array
+        items:
+          type: object
+          required: [ name ]
+          additionalProperties: False
+          properties:
+            name:
+              description: |
+                The name for the group, used to form the define and the value of the define.
+              type: string
+            # Start genetlink-c
+            c-define-name:
+              description: Override for the name of the define in C uAPI.
+              type: string
+            # End genetlink-c
+            flags: *cmd_flags
Documentation/netlink/genetlink-legacy.yaml new file +346 -0

346 lines not on this page · raw message

Documentation/netlink/genetlink.yaml new file +284 -0

284 lines not on this page · raw message

Signature · 1 line
2.39.0
#11 lore raw

Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs

Quoted 10 lines · Jakub Kicinski

Add schemas for Netlink spec files. As described in the docs we have 4 "protocols" or compatibility levels, and each one comes with its own schema, but the more general / legacy schemas are superset of more modern ones: genetlink is the smallest followed by genetlink-c and genetlink-legacy. There is no schema for raw netlink, yet, I haven't found the time..

I don't know enough jsonschema to do inheritance or something but the repetition is not too bad. I hope.

Generally you put common schemas under '$defs' and the then reference them with '$ref'.

$defs:
some-prop-type:
type: integer
minimum: 0

properties:
foo:

    $ref: '#/$defs/some-prop-type'
  bar:
    $ref: '#/$defs/some-prop-type'

If you have objects with common sets of properties, you can do the same thing, but then you need 'unevaluatedProperties' if you want to define a base set of properties and add to them. We do that frequently in DT schemas. Unlike typical inheritance, you can't override the 'base' schema. It's an AND operation.

Quoted 16 lines
---
 Documentation/netlink/genetlink-c.yaml      | 318 ++++++++++++++++++
 Documentation/netlink/genetlink-legacy.yaml | 346 ++++++++++++++++++++
 Documentation/netlink/genetlink.yaml        | 284 ++++++++++++++++
 3 files changed, 948 insertions(+)
 create mode 100644 Documentation/netlink/genetlink-c.yaml
 create mode 100644 Documentation/netlink/genetlink-legacy.yaml
 create mode 100644 Documentation/netlink/genetlink.yaml
Documentation/netlink/genetlink-c.yaml new file +0 -0
new file mode 100644
index 000000000000..53a7fe94a47e

Documentation/netlink/genetlink-c.yaml line 5

Quoted 5 lines
Documentation/netlink/genetlink-c.yaml new file +5 -0
@@ -0,0 +1,318 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"

There's no core.yaml. If you don't have a custom meta-schema, then just set this to the schema for the json-schema version you are using. Then the tools can validate the schemas without your own validator class.

Also, you can drop quotes on these.

Quoted 5 lines
Documentation/netlink/genetlink-c.yaml new file +5 -0
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]

Documentation/netlink/genetlink-c.yaml line 20

Quoted 10 lines
Documentation/netlink/genetlink-c.yaml new file +10 -0
+additionalProperties: False
+properties:
+  name:
+    description: Name of the genetlink family.
+    type: string
+  doc:
+    type: string
+  version:
+    description: Generic Netlink family version. Default is 1.
+    type: integer

Constraints?

minimum: 0

Same on all other integers.

Documentation/netlink/genetlink-c.yaml line 32

Quoted 12 lines
Documentation/netlink/genetlink-c.yaml new file +12 -0
+  protocol:
+    description: Schema compatibility level. Default is "genetlink".
+    enum: [ genetlink, genetlink-c]
+  # Start genetlink-c
+  uapi-header:
+    description: Path to the uAPI header, default is linux/${family-name}.h
+    type: string
+  c-family-name:
+    description: Name of the define for the family name.
+    type: string
+  c-version-name:
+    description: Name of the define for the verion of the family.

typo

Quoted 63 lines
Documentation/netlink/genetlink-c.yaml new file +63 -0
+    type: string
+  max-by-define:
+    description: Makes the number of attributes and commands be specified by a define, not an enum value.
+    type: boolean
+  # End genetlink-c
+
+  definitions:
+    description: Array of type and constant definitions.
+    type: array
+    items:
+      type: object
+      required: [ type, name ]
+      additionalProperties: False
+      properties:
+        name:
+          type: string
+        header:
+          description: For C-compatible languages, header which already defines this value.
+          type: string
+        type:
+          enum: [ const, enum, flags ]
+        doc:
+          type: string
+        # For const
+        value:
+          description: For const - the value.
+          type: [ string, integer ]
+        # For enum and flags
+        value-start:
+          description: For enum or flags the literal initializer for the first value.
+          type: [ string, integer ]
+        entries:
+          description: For enum or flags array of values
+          type: array
+          items:
+            oneOf:
+              - type: string
+              - type: object
+                required: [ name ]
+                additionalProperties: False
+                properties:
+                  name:
+                    type: string
+                  value:
+                    type: integer
+                  doc:
+                    type: string
+        render-max:
+          description: Render the max members for this enum.
+          type: boolean
+        # Start genetlink-c
+        enum-name:
+          description: Name for enum, if empty no name will be used.
+          type: [ string, "null" ]
+        name-prefix:
+          description: For enum the prefix of the values, optional.
+          type: string
+        # End genetlink-c
+
+  attribute-sets:
+    description: Definition of attribute spaces for this family.
+    type: array
+    items:

Documentation/netlink/genetlink-c.yaml line 105

Quoted 10 lines
Documentation/netlink/genetlink-c.yaml new file +10 -0
+      description: Definition of a single attribute space.
+      type: object
+      required: [ name, attributes ]
+      additionalProperties: False
+      properties:
+        name:
+          description: |
+            Name used when referring to this space in other definitions, not used outside of YAML.
+          type: string
+        # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.

If one is required:

oneOf:
- required: [ name-prefix ]
- required: [ subset-of ]

Or if both are optional:

dependencies:
name-prefix:
not:
required: [ subset-of ]
subset-of:
not:
required: [ name-prefix ]

Quoted 46 lines
Documentation/netlink/genetlink-c.yaml new file +46 -0
+        name-prefix:
+          description: |
+            Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+          type: string
+        enum-name:
+          description: Name for the enum type of the attribute.
+          type: string
+        doc:
+          description: Documentation of the space.
+          type: string
+        subset-of:
+          description: |
+            Name of another space which this is a logical part of. Sub-spaces can be used to define
+            a limited group of attributes which are used in a nest.
+          type: string
+        # Start genetlink-c
+        attr-cnt-name:
+          description: The explicit name for constant holding the count of attributes (last attr + 1).
+          type: string
+        attr-max-name:
+          description: The explicit name for last member of attribute enum.
+          type: string
+        # End genetlink-c
+        attributes:
+          description: List of attributes in the space.
+          type: array
+          items:
+            type: object
+            required: [ name, type ]
+            additionalProperties: False
+            properties:
+              name:
+                type: string
+              type: &attr-type
+                enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+                        string, nest, array-nest, nest-type-value ]
+              doc:
+                description: Documentation of the attribute.
+                type: string
+              value:
+                description: Value for the enum item representing this attribute in the uAPI.
+                type: integer
+              type-value:
+                description: Name of the value extracted from the type of a nest-type-value attribute.
+                type: array
+                items:

Documentation/netlink/genetlink-c.yaml line 161

Quoted 10 lines
Documentation/netlink/genetlink-c.yaml new file +10 -0
+                  type: string
+              byte-order:
+                enum: [little-endian, big-endian]
+              multi-attr:
+                type: boolean
+              nested-attributes:
+                description: Name of the space (sub-space) used inside the attribute.
+                type: string
+              enum:
+                description: Name of the enum type used for the atttribute.

typo

Quoted 14 lines
Documentation/netlink/genetlink-c.yaml new file +14 -0
+                type: string
+              enum-as-flags:
+                description: |
+                  Treat the enum as flags. In most cases enum is either used as flags or as values.
+                  Sometimes, however, both forms are necessary, in which case header contains the enum
+                  form while specific attributes may request to convert the values into a bitfield.
+                type: boolean
+              checks:
+                description: Kernel input validation.
+                type: object
+                additionalProperties: False
+                properties:
+                  flags-mask:
+                    description: Name of the flags constant on which to base mask (unsigned scalar types only).

Documentation/netlink/genetlink-c.yaml line 185

Quoted 10 lines
Documentation/netlink/genetlink-c.yaml new file +10 -0
+                    type: string
+                  min:
+                    description: Min value for an integer attribute.
+                    type: integer
+                  min-len:
+                    description: Min length for a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer

How can a length be a string?

Anyways, this is something you could pull out into a $defs entry and reference. It will also work without the oneOf because 'pattern' will just be ignored for an integer. That's one gotcha with json-schema. If a keyword doesn't apply to the instance, it is silently ignored. (That includes unknown keywords such as ones with typos. Fun!). 'oneOf' will give you pretty crappy error messages, so it's good to avoid when possible.

Quoted 161 lines
Documentation/netlink/genetlink-c.yaml new file +133 -0
+                  max-len:
+                    description: Max length for a string or a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+              sub-type: *attr-type
+  operations:
+    description: Operations supported by the protocol.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      enum-model:
+        description: |
+          The model of assigning values to the operations.
+          "unified" is the recommended model where all message types belong
+          to a single enum.
+          "directional" has the messages sent to the kernel and from the kernel
+          enumerated separately.
+          "notify-split" has the notifications and request-response types in
+          different enums.
+        enum: [ unified, directional, notify-split ]
+      name-prefix:
+        description: |
+          Prefix for the C enum name of the command. The name is formed by concatenating
+          the prefix with the upper case name of the command, with dashes replaced by underscores.
+        type: string
+      enum-name:
+        description: Name for the enum type with commands.
+        type: string
+      async-prefix:
+        description: Same as name-prefix but used to render notifications and events to separate enum.
+        type: string
+      async-enum:
+        description: Name for the enum type with notifications/events.
+        type: string
+      list:
+        description: List of commands
+        type: array
+        items:
+          type: object
+          additionalProperties: False
+          required: [ name, doc ]
+          properties:
+            name:
+              description: Name of the operation, also defining its C enum value in uAPI.
+              type: string
+            doc:
+              description: Documentation for the command.
+              type: string
+            value:
+              description: Value for the enum in the uAPI.
+              type: integer
+            attribute-set:
+              description: |
+                Attribute space from which attributes directly in the requests and replies
+                to this command are defined.
+              type: string
+            flags: &cmd_flags
+              description: Command flags.
+              type: array
+              items:
+                enum: [ admin-perm ]
+            dont-validate:
+              description: Kernel attribute validation flags.
+              type: array
+              items:
+                enum: [ strict, dump ]
+            do: &subop-type
+              description: Main command handler.
+              type: object
+              additionalProperties: False
+              properties:
+                request: &subop-attr-list
+                  description: Definition of the request message for a given command.
+                  type: object
+                  additionalProperties: False
+                  properties:
+                    attributes:
+                      description: |
+                        Names of attributes from the attribute-set (not full attribute
+                        definitions, just names).
+                      type: array
+                      items:
+                        type: string
+                reply: *subop-attr-list
+                pre:
+                  description: Hook for a function to run before the main callback (pre_doit or start).
+                  type: string
+                post:
+                  description: Hook for a function to run after the main callback (post_doit or done).
+                  type: string
+            dump: *subop-type
+            notify:
+              description: Name of the command sharing the reply type with this notification.
+              type: string
+            event:
+              type: object
+              additionalProperties: False
+              properties:
+                attributes:
+                  description: Explicit list of the attributes for the notification.
+                  type: array
+                  items:
+                    type: string
+            mcgrp:
+              description: Name of the multicast group generating given notification.
+              type: string
+  mcast-groups:
+    description: List of multicast groups.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      list:
+        description: List of groups.
+        type: array
+        items:
+          type: object
+          required: [ name ]
+          additionalProperties: False
+          properties:
+            name:
+              description: |
+                The name for the group, used to form the define and the value of the define.
+              type: string
+            # Start genetlink-c
+            c-define-name:
+              description: Override for the name of the define in C uAPI.
+              type: string
+            # End genetlink-c
+            flags: *cmd_flags
Documentation/netlink/genetlink-legacy.yaml new file +22 -0
new file mode 100644
index 000000000000..5e57453eac1a
@@ -0,0 +1,346 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink-legacy.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]
+additionalProperties: False
+properties:
+  name:
+    description: Name of the genetlink family.
+    type: string
+  doc:
+    type: string
+  version:
+    description: Generic Netlink family version. Default is 1.
+    type: integer
+  protocol:
+    description: Schema compatibility level. Default is "genetlink".

Documentation/netlink/genetlink-legacy.yaml line 32

Quoted 10 lines
Documentation/netlink/genetlink-legacy.yaml new file +10 -0
+    enum: [ genetlink, genetlink-c, genetlink-legacy, netlink-raw ] # Trim
+  # Start genetlink-c
+  uapi-header:
+    description: Path to the uAPI header, default is linux/${family-name}.h
+    type: string
+  c-family-name:
+    description: Name of the define for the family name.
+    type: string
+  c-version-name:
+    description: Name of the define for the verion of the family.

typo

Quoted 75 lines
Documentation/netlink/genetlink-legacy.yaml new file +75 -0
+    type: string
+  max-by-define:
+    description: Makes the number of attributes and commands be specified by a define, not an enum value.
+    type: boolean
+  # End genetlink-c
+  # Start genetlink-legacy
+  kernel-policy:
+    description: |
+      Defines if the input policy in the kernel is global, per-operation, or split per operation type.
+      Default is split.
+    enum: [split, per-op, global]
+  # End genetlink-legacy
+
+  definitions:
+    description: Array of type and constant definitions.
+    type: array
+    items:
+      type: object
+      required: [ type, name ]
+      additionalProperties: False
+      properties:
+        name:
+          type: string
+        header:
+          description: For C-compatible languages, header which already defines this value.
+          type: string
+        type:
+          enum: [ const, enum, flags, struct ] # Trim
+        doc:
+          type: string
+        # For const
+        value:
+          description: For const - the value.
+          type: [ string, integer ]
+        # For enum and flags
+        value-start:
+          description: For enum or flags the literal initializer for the first value.
+          type: [ string, integer ]
+        entries:
+          description: For enum or flags array of values
+          type: array
+          items:
+            oneOf:
+              - type: string
+              - type: object
+                required: [ name ]
+                additionalProperties: False
+                properties:
+                  name:
+                    type: string
+                  value:
+                    type: integer
+                  doc:
+                    type: string
+        render-max:
+          description: Render the max members for this enum.
+          type: boolean
+        # Start genetlink-c
+        enum-name:
+          description: Name for enum, if empty no name will be used.
+          type: [ string, "null" ]
+        name-prefix:
+          description: For enum the prefix of the values, optional.
+          type: string
+        # End genetlink-c
+        # Start genetlink-legacy
+        members:
+          description: List of struct members. Only scalars and strings members allowed.
+          type: array
+          items:
+            type: object
+            required: [ name, type ]
+            additionalProperties: False
+            properties:
+              name:

Documentation/netlink/genetlink-legacy.yaml line 117

Quoted 10 lines
Documentation/netlink/genetlink-legacy.yaml new file +10 -0
+                type: string
+              type:
+                enum: [ u8, u16, u32, u64, s8, s16, s32, s64, string ]
+              len:
+                oneOf:
+                  -
+                    type: string
+                    pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                  -
+                    type: integer

Inconsistent style, but this is the subschema you can factor out.

Quoted 522 lines
Documentation/netlink/genetlink-legacy.yaml new file +229 -0
+        # End genetlink-legacy
+
+  attribute-sets:
+    description: Definition of attribute spaces for this family.
+    type: array
+    items:
+      description: Definition of a single attribute space.
+      type: object
+      required: [ name, attributes ]
+      additionalProperties: False
+      properties:
+        name:
+          description: |
+            Name used when referring to this space in other definitions, not used outside of YAML.
+          type: string
+        # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
+        name-prefix:
+          description: |
+            Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+          type: string
+        enum-name:
+          description: Name for the enum type of the attribute.
+          type: string
+        doc:
+          description: Documentation of the space.
+          type: string
+        subset-of:
+          description: |
+            Name of another space which this is a logical part of. Sub-spaces can be used to define
+            a limited group of attributes which are used in a nest.
+          type: string
+        # Start genetlink-c
+        attr-cnt-name:
+          description: The explicit name for constant holding the count of attributes (last attr + 1).
+          type: string
+        attr-max-name:
+          description: The explicit name for last member of attribute enum.
+          type: string
+        # End genetlink-c
+        attributes:
+          description: List of attributes in the space.
+          type: array
+          items:
+            type: object
+            required: [ name, type ]
+            additionalProperties: False
+            properties:
+              name:
+                type: string
+              type: &attr-type
+                enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+                        string, nest, array-nest, nest-type-value ]
+              doc:
+                description: Documentation of the attribute.
+                type: string
+              value:
+                description: Value for the enum item representing this attribute in the uAPI.
+                type: integer
+              type-value:
+                description: Name of the value extracted from the type of a nest-type-value attribute.
+                type: array
+                items:
+                  type: string
+              byte-order:
+                enum: [little-endian, big-endian]
+              multi-attr:
+                type: boolean
+              nested-attributes:
+                description: Name of the space (sub-space) used inside the attribute.
+                type: string
+              enum:
+                description: Name of the enum type used for the atttribute.
+                type: string
+              enum-as-flags:
+                description: |
+                  Treat the enum as flags. In most cases enum is either used as flags or as values.
+                  Sometimes, however, both forms are necessary, in which case header contains the enum
+                  form while specific attributes may request to convert the values into a bitfield.
+                type: boolean
+              checks:
+                description: Kernel input validation.
+                type: object
+                additionalProperties: False
+                properties:
+                  flags-mask:
+                    description: Name of the flags constant on which to base mask (unsigned scalar types only).
+                    type: string
+                  min:
+                    description: Min value for an integer attribute.
+                    type: integer
+                  min-len:
+                    description: Min length for a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+                  max-len:
+                    description: Max length for a string or a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+              sub-type: *attr-type
+  operations:
+    description: Operations supported by the protocol.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      enum-model:
+        description: |
+          The model of assigning values to the operations.
+          "unified" is the recommended model where all message types belong
+          to a single enum.
+          "directional" has the messages sent to the kernel and from the kernel
+          enumerated separately.
+          "notify-split" has the notifications and request-response types in
+          different enums.
+        enum: [ unified, directional, notify-split ]
+      name-prefix:
+        description: |
+          Prefix for the C enum name of the command. The name is formed by concatenating
+          the prefix with the upper case name of the command, with dashes replaced by underscores.
+        type: string
+      enum-name:
+        description: Name for the enum type with commands.
+        type: string
+      async-prefix:
+        description: Same as name-prefix but used to render notifications and events to separate enum.
+        type: string
+      async-enum:
+        description: Name for the enum type with notifications/events.
+        type: string
+      list:
+        description: List of commands
+        type: array
+        items:
+          type: object
+          additionalProperties: False
+          required: [ name, doc ]
+          properties:
+            name:
+              description: Name of the operation, also defining its C enum value in uAPI.
+              type: string
+            doc:
+              description: Documentation for the command.
+              type: string
+            value:
+              description: Value for the enum in the uAPI.
+              type: integer
+            attribute-set:
+              description: |
+                Attribute space from which attributes directly in the requests and replies
+                to this command are defined.
+              type: string
+            flags: &cmd_flags
+              description: Command flags.
+              type: array
+              items:
+                enum: [ admin-perm ]
+            dont-validate:
+              description: Kernel attribute validation flags.
+              type: array
+              items:
+                enum: [ strict, dump ]
+            do: &subop-type
+              description: Main command handler.
+              type: object
+              additionalProperties: False
+              properties:
+                request: &subop-attr-list
+                  description: Definition of the request message for a given command.
+                  type: object
+                  additionalProperties: False
+                  properties:
+                    attributes:
+                      description: |
+                        Names of attributes from the attribute-set (not full attribute
+                        definitions, just names).
+                      type: array
+                      items:
+                        type: string
+                reply: *subop-attr-list
+                pre:
+                  description: Hook for a function to run before the main callback (pre_doit or start).
+                  type: string
+                post:
+                  description: Hook for a function to run after the main callback (post_doit or done).
+                  type: string
+            dump: *subop-type
+            notify:
+              description: Name of the command sharing the reply type with this notification.
+              type: string
+            event:
+              type: object
+              additionalProperties: False
+              properties:
+                attributes:
+                  description: Explicit list of the attributes for the notification.
+                  type: array
+                  items:
+                    type: string
+            mcgrp:
+              description: Name of the multicast group generating given notification.
+              type: string
+  mcast-groups:
+    description: List of multicast groups.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      list:
+        description: List of groups.
+        type: array
+        items:
+          type: object
+          required: [ name ]
+          additionalProperties: False
+          properties:
+            name:
+              description: |
+                The name for the group, used to form the define and the value of the define.
+              type: string
+            # Start genetlink-c
+            c-define-name:
+              description: Override for the name of the define in C uAPI.
+              type: string
+            # End genetlink-c
+            flags: *cmd_flags
Documentation/netlink/genetlink.yaml new file +284 -1
new file mode 100644
index 000000000000..aafaaaabcfbe
@@ -0,0 +1,284 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: "http://kernel.org/schemas/netlink/genetlink.yaml#"
+$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"
+
+title: Protocol
+description: Specification of a genetlink protocol
+type: object
+required: [ name, doc, attribute-sets, operations ]
+additionalProperties: False
+properties:
+  name:
+    description: Name of the genetlink family.
+    type: string
+  doc:
+    type: string
+  version:
+    description: Generic Netlink family version. Default is 1.
+    type: integer
+  protocol:
+    description: Schema compatibility level. Default is "genetlink".
+    enum: [ genetlink ]
+
+  definitions:
+    description: Array of type and constant definitions.
+    type: array
+    items:
+      type: object
+      required: [ type, name ]
+      additionalProperties: False
+      properties:
+        name:
+          type: string
+        header:
+          description: For C-compatible languages, header which already defines this value.
+          type: string
+        type:
+          enum: [ const, enum, flags ]
+        doc:
+          type: string
+        # For const
+        value:
+          description: For const - the value.
+          type: [ string, integer ]
+        # For enum and flags
+        value-start:
+          description: For enum or flags the literal initializer for the first value.
+          type: [ string, integer ]
+        entries:
+          description: For enum or flags array of values
+          type: array
+          items:
+            oneOf:
+              - type: string
+              - type: object
+                required: [ name ]
+                additionalProperties: False
+                properties:
+                  name:
+                    type: string
+                  value:
+                    type: integer
+                  doc:
+                    type: string
+
+        render-max:
+          description: Render the max members for this enum.
+          type: boolean
+
+  attribute-sets:
+    description: Definition of attribute spaces for this family.
+    type: array
+    items:
+      description: Definition of a single attribute space.
+      type: object
+      required: [ name, attributes ]
+      additionalProperties: False
+      properties:
+        name:
+          description: |
+            Name used when referring to this space in other definitions, not used outside of YAML.
+          type: string
+        # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.
+        name-prefix:
+          description: |
+            Prefix for the C enum name of the attributes. Default family[name]-set[name]-a-
+          type: string
+        enum-name:
+          description: Name for the enum type of the attribute.
+          type: string
+        doc:
+          description: Documentation of the space.
+          type: string
+        subset-of:
+          description: |
+            Name of another space which this is a logical part of. Sub-spaces can be used to define
+            a limited group of attributes which are used in a nest.
+          type: string
+        attributes:
+          description: List of attributes in the space.
+          type: array
+          items:
+            type: object
+            required: [ name, type ]
+            additionalProperties: False
+            properties:
+              name:
+                type: string
+              type: &attr-type
+                enum: [ unused, pad, flag, binary, u8, u16, u32, u64, s32, s64,
+                        string, nest, array-nest, nest-type-value ]
+              doc:
+                description: Documentation of the attribute.
+                type: string
+              value:
+                description: Value for the enum item representing this attribute in the uAPI.
+                type: integer
+              type-value:
+                description: Name of the value extracted from the type of a nest-type-value attribute.
+                type: array
+                items:
+                  type: string
+              byte-order:
+                enum: [little-endian, big-endian]
+              multi-attr:
+                type: boolean
+              nested-attributes:
+                description: Name of the space (sub-space) used inside the attribute.
+                type: string
+              enum:
+                description: Name of the enum type used for the atttribute.
+                type: string
+              enum-as-flags:
+                description: |
+                  Treat the enum as flags. In most cases enum is either used as flags or as values.
+                  Sometimes, however, both forms are necessary, in which case header contains the enum
+                  form while specific attributes may request to convert the values into a bitfield.
+                type: boolean
+              checks:
+                description: Kernel input validation.
+                type: object
+                additionalProperties: False
+                properties:
+                  flags-mask:
+                    description: Name of the flags constant on which to base mask (unsigned scalar types only).
+                    type: string
+                  min:
+                    description: Min value for an integer attribute.
+                    type: integer
+                  min-len:
+                    description: Min length for a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+                  max-len:
+                    description: Max length for a string or a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
+              sub-type: *attr-type
+  operations:
+    description: Operations supported by the protocol.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      enum-model:
+        description: |
+          The model of assigning values to the operations.
+          "unified" is the recommended model where all message types belong
+          to a single enum.
+          "directional" has the messages sent to the kernel and from the kernel
+          enumerated separately.
+          "notify-split" has the notifications and request-response types in
+          different enums.
+        enum: [ unified, directional, notify-split ]
+      name-prefix:
+        description: |
+          Prefix for the C enum name of the command. The name is formed by concatenating
+          the prefix with the upper case name of the command, with dashes replaced by underscores.
+        type: string
+      enum-name:
+        description: Name for the enum type with commands.
+        type: string
+      async-prefix:
+        description: Same as name-prefix but used to render notifications and events to separate enum.
+        type: string
+      async-enum:
+        description: Name for the enum type with notifications/events.
+        type: string
+      list:
+        description: List of commands
+        type: array
+        items:
+          type: object
+          additionalProperties: False
+          required: [ name, doc ]
+          properties:
+            name:
+              description: Name of the operation, also defining its C enum value in uAPI.
+              type: string
+            doc:
+              description: Documentation for the command.
+              type: string
+            value:
+              description: Value for the enum in the uAPI.
+              type: integer
+            attribute-set:
+              description: |
+                Attribute space from which attributes directly in the requests and replies
+                to this command are defined.
+              type: string
+            flags: &cmd_flags
+              description: Command flags.
+              type: array
+              items:
+                enum: [ admin-perm ]
+            dont-validate:
+              description: Kernel attribute validation flags.
+              type: array
+              items:
+                enum: [ strict, dump ]
+            do: &subop-type
+              description: Main command handler.
+              type: object
+              additionalProperties: False
+              properties:
+                request: &subop-attr-list
+                  description: Definition of the request message for a given command.
+                  type: object
+                  additionalProperties: False
+                  properties:
+                    attributes:
+                      description: |
+                        Names of attributes from the attribute-set (not full attribute
+                        definitions, just names).
+                      type: array
+                      items:
+                        type: string
+                reply: *subop-attr-list
+                pre:
+                  description: Hook for a function to run before the main callback (pre_doit or start).
+                  type: string
+                post:
+                  description: Hook for a function to run after the main callback (post_doit or done).
+                  type: string
+            dump: *subop-type
+            notify:
+              description: Name of the command sharing the reply type with this notification.
+              type: string
+            event:
+              type: object
+              additionalProperties: False
+              properties:
+                attributes:
+                  description: Explicit list of the attributes for the notification.
+                  type: array
+                  items:
+                    type: string
+            mcgrp:
+              description: Name of the multicast group generating given notification.
+              type: string
+  mcast-groups:
+    description: List of multicast groups.
+    type: object
+    required: [ list ]
+    additionalProperties: False
+    properties:
+      list:
+        description: List of groups.
+        type: array
+        items:
+          type: object
+          required: [ name ]
+          additionalProperties: False
+          properties:
+            name:
+              description: |
+                The name for the group, used to form the define and the value of the define.
+              type: string
+            flags: *cmd_flags
--

2.39.0

author maintainer
#12 lore raw

Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs

Quoted 7 lines · On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
Quoted 2 lines

+$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#" +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"

There's no core.yaml. If you don't have a custom meta-schema, then just set this to the schema for the json-schema version you are using. Then the tools can validate the schemas without your own validator class.

$schema: https://json-schema.org/draft/2020-12/schema

or

$schema: https://json-schema.org/draft-09/schema

?

It seems like the documentation suggests the former but the latter appears widespread.

#13 lore raw

Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs

Quoted 10 lines · Jakub Kicinski
Quoted 7 lines · On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
Quoted 2 lines

+$id: "http://kernel.org/schemas/netlink/genetlink-c.yaml#" +$schema: "http://kernel.org/meta-schemas/netlink/core.yaml#"

There's no core.yaml. If you don't have a custom meta-schema, then just set this to the schema for the json-schema version you are using. Then the tools can validate the schemas without your own validator class.

Quoted 5 lines

$schema: https://json-schema.org/draft/2020-12/schema

or

$schema: https://json-schema.org/draft-09/schema

s/draft-09/draft\/2019-09/

Or did you mean draft-07?

Quoted 4 lines

?

It seems like the documentation suggests the former but the latter appears widespread.

Yes, 2019-09 is generally not recommended as it had some one off features that 2020-12 replaced (nothing you'd care about). DT uses 2019-09 because we needed unevaluatedProperties, but not 'prefixItems' replacing 'items' list variant in 2020-12. I think you only used the schema version of 'items' (not the list version), so you are probably compatible with any version. 'dependencies' changing to 'dependentSchema/dependentRequired' is the other difference you might hit.

TLDR: draft-07 is probably sufficient for your needs.

Rob

author maintainer
#14 lore raw

Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs

In reply to Rob Herring
Quoted 20 lines · On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
Quoted 10 lines · Jakub Kicinski

Add schemas for Netlink spec files. As described in the docs we have 4 "protocols" or compatibility levels, and each one comes with its own schema, but the more general / legacy schemas are superset of more modern ones: genetlink is the smallest followed by genetlink-c and genetlink-legacy. There is no schema for raw netlink, yet, I haven't found the time..

I don't know enough jsonschema to do inheritance or something but the repetition is not too bad. I hope.

Generally you put common schemas under '$defs' and the then reference them with '$ref'.

$defs:
some-prop-type:
type: integer
minimum: 0

Quoted 5 lines

properties:
foo:

    $ref: '#/$defs/some-prop-type'
  bar:
    $ref: '#/$defs/some-prop-type'

Thanks! Is it possible to move the common definitions to a separate file? I tried to create a file called defs.yaml and change the ref to:

$ref: "defs.yaml#/$defs/len-or-define"

But:

File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 257, in iter_errors
for error in errors:
File "/usr/lib/python3.11/site-packages/jsonschema/_validators.py", line 294, in ref
scope, resolved = validator.resolver.resolve(ref)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 856, in resolve
return url, self._remote_cache(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 870, in resolve_from_url
raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: Expecting value: line 1 column 1 (char 0)

Quoted 5 lines

If you have objects with common sets of properties, you can do the same thing, but then you need 'unevaluatedProperties' if you want to define a base set of properties and add to them. We do that frequently in DT schemas. Unlike typical inheritance, you can't override the 'base' schema. It's an AND operation.

This is hard to comprehend :o Most of the time I seem to need only the ability to add a custom "description" to the object, so for example:

$defs:
len-or-define:

    oneOf:
      -
        type: string
        pattern: ^[0-9A-Za-z_-]*( - 1)?$
      -
        type: integer
        minimum: 0

Then:

min-len:
description: Min length for a binary attribute. $ref: '#/$defs/len-or-define'

And that seems to work. Should I be using unevaluatedProperties somehow as well here?

Quoted 13 lines

Documentation/netlink/genetlink-c.yaml line 105

Quoted 4 lines
Documentation/netlink/genetlink-c.yaml new file +4 -0
+          description: |
+            Name used when referring to this space in other definitions, not used outside of YAML.
+          type: string
+        # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.  

If one is required:

oneOf:
- required: [ name-prefix ]
- required: [ subset-of ]

Or if both are optional:

Quoted 7 lines

dependencies:
name-prefix:
not:
required: [ subset-of ]
subset-of:
not:
required: [ name-prefix ]

Nice, let me try this.

Quoted 8 lines

Documentation/netlink/genetlink-c.yaml line 185

Quoted 6 lines
Documentation/netlink/genetlink-c.yaml new file +6 -0
+                  min-len:
+                    description: Min length for a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer  

How can a length be a string?

For readability in C I wanted to allow using a define for the length. Then the name of the define goes here, and the value can be fetched from the "definitions" section of the spec.

Quoted 7 lines

Anyways, this is something you could pull out into a $defs entry and reference. It will also work without the oneOf because 'pattern' will just be ignored for an integer. That's one gotcha with json-schema. If a keyword doesn't apply to the instance, it is silently ignored. (That includes unknown keywords such as ones with typos. Fun!). 'oneOf' will give you pretty crappy error messages, so it's good to avoid when possible.

Oh, interesting. Changed to:

$defs:
len-or-define:
type: [ string, integer ]
pattern: ^[0-9A-Za-z_-]*( - 1)?$
minimum: 0

#15 lore raw

Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs

Quoted 28 lines · Jakub Kicinski
Quoted 20 lines · On Thu, 19 Jan 2023 08:07:31 -0600 Rob Herring wrote:
Quoted 10 lines · Jakub Kicinski

Add schemas for Netlink spec files. As described in the docs we have 4 "protocols" or compatibility levels, and each one comes with its own schema, but the more general / legacy schemas are superset of more modern ones: genetlink is the smallest followed by genetlink-c and genetlink-legacy. There is no schema for raw netlink, yet, I haven't found the time..

I don't know enough jsonschema to do inheritance or something but the repetition is not too bad. I hope.

Generally you put common schemas under '$defs' and the then reference them with '$ref'.

$defs:
some-prop-type:
type: integer
minimum: 0

Quoted 5 lines

properties:
foo:

    $ref: '#/$defs/some-prop-type'
  bar:
    $ref: '#/$defs/some-prop-type'
Quoted 4 lines

Thanks! Is it possible to move the common definitions to a separate file? I tried to create a file called defs.yaml and change the ref to:

$ref: "defs.yaml#/$defs/len-or-define"

Yes, but...

Quoted 13 lines

But:

File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 257, in iter_errors
for error in errors:
File "/usr/lib/python3.11/site-packages/jsonschema/_validators.py", line 294, in ref
scope, resolved = validator.resolver.resolve(ref)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 856, in resolve
return url, self._remote_cache(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/jsonschema/validators.py", line 870, in resolve_from_url
raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: Expecting value: line 1 column 1 (char 0)

You either need the ref at the URL or a custom resolver to convert http path to a file path. Also, the default resolver doesn't handle YAML either. It's not much code to do and the DT tools do this. The hardest part was just getting the path right. For a ref with just a filename and no path, the URL will be the same path as the $id in the referring schema.

Quoted 8 lines
Quoted 5 lines

If you have objects with common sets of properties, you can do the same thing, but then you need 'unevaluatedProperties' if you want to define a base set of properties and add to them. We do that frequently in DT schemas. Unlike typical inheritance, you can't override the 'base' schema. It's an AND operation.

This is hard to comprehend :o Most of the time I seem to need only the ability to add a custom "description" to the object, so for example:

By object, schemas here which are 'type: object'. Those all have a list of properties. If several schemas for 'objects' have the same set of properties like 'name' and 'type' for example, then you could define a schema for that and then add more properties for each specific object. I didn't really look close enough whether that makes sense here.

Quoted 10 lines

$defs:
len-or-define:

    oneOf:
      -
        type: string
        pattern: ^[0-9A-Za-z_-]*( - 1)?$
      -
        type: integer
        minimum: 0
Quoted 8 lines

Then:

min-len:
description: Min length for a binary attribute. $ref: '#/$defs/len-or-define'

And that seems to work. Should I be using unevaluatedProperties somehow as well here?

No, it looks fine. 'unevaluatedProperties' wouldn't make sense here as 'type' is a string or integer.

Quoted 31 lines
Quoted 14 lines

Documentation/netlink/genetlink-c.yaml line 105

Quoted 5 lines
Documentation/netlink/genetlink-c.yaml new file +4 -0
+          description: |
+            Name used when referring to this space in other definitions, not used outside of YAML.
+          type: string
+        # Strictly speaking 'name-prefix' and 'subset-of' should be mutually exclusive.

If one is required:

oneOf:
- required: [ name-prefix ]
- required: [ subset-of ]

Or if both are optional:

Quoted 7 lines

dependencies:
name-prefix:
not:
required: [ subset-of ]
subset-of:
not:
required: [ name-prefix ]

Nice, let me try this.

Quoted 7 lines
Quoted 6 lines
Documentation/netlink/genetlink-c.yaml new file +6 -0
+                  min-len:
+                    description: Min length for a binary attribute.
+                    oneOf:
+                      - type: string
+                        pattern: ^[0-9A-Za-z_-]*( - 1)?$
+                      - type: integer
Quoted 5 lines
Quoted 1 line

How can a length be a string?

For readability in C I wanted to allow using a define for the length. Then the name of the define goes here, and the value can be fetched from the "definitions" section of the spec.

Ah, makes sense.

Don't you need to drop the '-' in the regex then? Also, the '*' should be a '+' for 1 or more instead of 0 or more.

Quoted 15 lines
Quoted 7 lines

Anyways, this is something you could pull out into a $defs entry and reference. It will also work without the oneOf because 'pattern' will just be ignored for an integer. That's one gotcha with json-schema. If a keyword doesn't apply to the instance, it is silently ignored. (That includes unknown keywords such as ones with typos. Fun!). 'oneOf' will give you pretty crappy error messages, so it's good to avoid when possible.

Oh, interesting. Changed to:

$defs:
len-or-define:
type: [ string, integer ]
pattern: ^[0-9A-Za-z_-]*( - 1)?$
minimum: 0

author maintainer
#16 lore raw

Re: [PATCH net-next v3 2/8] netlink: add schemas for YAML specs

In reply to Jakub Kicinski
Quoted 14 lines · On Thu, 19 Jan 2023 13:49:22 -0800 Jakub Kicinski wrote:
Quoted 8 lines

Generally you put common schemas under '$defs' and the then reference them with '$ref'.

$defs:
some-prop-type:
type: integer
minimum: 0

Quoted 5 lines

properties:
foo:

    $ref: '#/$defs/some-prop-type'
  bar:
    $ref: '#/$defs/some-prop-type'  
Quoted 4 lines

Thanks! Is it possible to move the common definitions to a separate file? I tried to create a file called defs.yaml and change the ref to:

$ref: "defs.yaml#/$defs/len-or-define"

Oh, oh. Instead of trying to create 3 different "levels" of spec, and having to pull out shared definitions maybe I can use the
if + unevaluatedProperties
to only allow certain properties depending on the value in the "protocol" attribute...

author maintainer
#17 lore raw

PATCH v3 3/8 net: add basic C code generators for Netlink

In reply to Jakub Kicinski

Code generators to turn Netlink specs into C code. I'm definitely not proud of it.

The main generator is in Python, there's a bash script to regen all code-gen'ed files in tree after making spec changes.

--
v2: - use /* */ comments instead of //

---
 MAINTAINERS                |    1 +
 tools/net/ynl/ynl-gen-c.py | 2376 ++++++++++++++++++++++++++++++++++++
 tools/net/ynl/ynl-regen.sh |   30 +
 3 files changed, 2407 insertions(+)
 create mode 100755 tools/net/ynl/ynl-gen-c.py
 create mode 100755 tools/net/ynl/ynl-regen.sh
MAINTAINERS +1 -0
index 3842915d863d..75a1395c8023 100644
@@ -14578,6 +14578,7 @@ F:	include/uapi/linux/netdevice.h
 F:	lib/net_utils.c
 F:	lib/random32.c
 F:	net/
+F:	tools/net/
 F:	tools/testing/selftests/net/
 
 NETWORKING [IPSEC]
tools/net/ynl/ynl-gen-c.py new file +2376 -0

2,376 lines not on this page · raw message

tools/net/ynl/ynl-regen.sh new file +30 -0
new file mode 100755
index 000000000000..43989ae48ed0
@@ -0,0 +1,30 @@
+#!/bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+
+TOOL=$(dirname $(realpath $0))/ynl-gen-c.py
+
+force=
+
+while [ ! -z "$1" ]; do
+  case "$1" in
+    -f ) force=yes; shift ;;
+    * )  echo "Unrecognized option '$1'"; exit 1 ;;
+  esac
+done
+
+KDIR=$(dirname $(dirname $(dirname $(dirname $(realpath $0)))))
+
+files=$(git grep --files-with-matches '^/\* YNL-GEN \(kernel\|uapi\)')
+for f in $files; do
+    # params:     0       1      2     3
+    #         $YAML YNL-GEN kernel $mode
+    params=( $(git grep -B1 -h '/\* YNL-GEN' $f | sed 's@/\*\(.*\)\*/@\1@') )
+
+    if [ $f -nt ${params[0]} -a -z "$force" ]; then
+	echo -e "\tSKIP $f"
+	continue
+    fi
+
+    echo -e "\tGEN ${params[2]}\t$f"
+    $TOOL --mode ${params[2]} --${params[3]} --spec $KDIR/${params[0]} -o $f
+done
Signature · 1 line
2.39.0
#18 lore raw

Re: [PATCH net-next v3 3/8] net: add basic C code generators for Netlink

tools/net/ynl/ynl-gen-c.py line 19

Quoted 4 lines · Jakub Kicinski
tools/net/ynl/ynl-gen-c.py new file +3 -0
+class BaseNlLib:
+    def __init__(self):
+        pass

That __init__ seems pointless?

Quoted 1 line

+ self.name = attr['name'].replace('-', '_')

You have a _lot_ of these ".replace('-', '_')" invocations - maybe make a function just like you have c_upper()/c_lower()?

tools/net/ynl/ynl-gen-c.py line 81

Quoted 2 lines
tools/net/ynl/ynl-gen-c.py new file +2 -0
+        if self.presence_type() == 'len':
+            pfx = '__' if space == 'user' else ''

this is how you define pfx

tools/net/ynl/ynl-gen-c.py line 107

Quoted 4 lines
tools/net/ynl/ynl-gen-c.py new file +4 -0
+        if member:
+            ptr = ''
+            if self. is_multi_val():
+                ptr = '*'

but this is how you define ptr - feels a bit inconsistent

Also note the extra space after "self." there

tools/net/ynl/ynl-gen-c.py line 164

Quoted 3 lines
tools/net/ynl/ynl-gen-c.py new file +3 -0
+        if not self.is_multi_val():
+            ri.cw.p(f"if (ynl_attr_validate(yarg, attr))")
+            ri.cw.p(f"return MNL_CB_ERROR;")

Those didn't need f-strings.

Does this "ri.cw.p()" do indentation for the code?

tools/net/ynl/ynl-gen-c.py line 318

Quoted 2 lines
tools/net/ynl/ynl-gen-c.py new file +2 -0
+    def attr_policy(self, cw):
+        if 'unterminated-ok' in self.checks and self.checks['unterminated-ok']:

maybe

if self.checks.get('unterminated-ok', 0):

Quoted 7 lines
tools/net/ynl/ynl-gen-c.py new file +7 -0
+class TypeBinary(Type):
+    def arg_member(self, ri):
+        return [f"const void *{self.c_name}", 'size_t len']
+
+    def presence_type(self):
+        return 'len'
+

tools/net/ynl/ynl-gen-c.py line 362

Quoted 10 lines
tools/net/ynl/ynl-gen-c.py new file +10 -0
+    def struct_member(self, ri):
+        ri.cw.p(f"void *{self.c_name};")
+
+    def _attr_typol(self):
+        return f'.type = YNL_PT_BINARY,'
+
+    def _attr_policy(self, policy):
+        mem = '{ '
+        if len(self.checks) == 1 and 'min-len' in self.checks:
+            mem += '.len = ' + str(self.checks['min-len'])

Why does the len(self.checks) matter?

tools/net/ynl/ynl-gen-c.py line 366

Quoted 4 lines
tools/net/ynl/ynl-gen-c.py new file +4 -0
+        elif len(self.checks) == 0:
+            mem += '.type = NLA_BINARY'
+        else:
+            raise Exception('Binary type not implemented, yet')

to get to that error messsage? The error messsage seems a bit misleading too though. It's that 'max-len' isn't implemented, or such, no?

Review

Quoted 2 lines
(quoted diff) +2 -0
+    def _complex_member_type(self, ri):
+        if 'type' not in self.attr or self.attr['type'] == 'nest':

could do

if self.attr.get('type', 'nest') == 'nest':

again like above

tools/net/ynl/ynl-gen-c.py line 429

Quoted 3 lines
tools/net/ynl/ynl-gen-c.py new file +3 -0
+            return f"struct {self.nested_render_name}"
+        elif self.attr['type'] in scalars:
+            scalar_pfx = '__' if ri.ku_space == 'user' else ''

You also have this

	... = '__' if ... == 'user' else ''

quite a few times, maybe add something like

def qualify_user(value, user):

    if user == 'user':
        return '__' + value
    return value
Quoted 1 line

+ return scalar_pfx + self.attr['type']

and then this is just

return qualify_user(self.attr['type'], ri.ku_space)

instead of the two lines

tools/net/ynl/ynl-gen-c.py line 438

Quoted 5 lines
tools/net/ynl/ynl-gen-c.py new file +5 -0
+    def free_needs_iter(self):
+        return 'type' not in self.attr or self.attr['type'] == 'nest'
+
+    def free(self, ri, var, ref):
+        if 'type' not in self.attr or self.attr['type'] == 'nest':

two more places that could use the .get trick

but it's really up to you. Just that the line like that seems rather long to me :-)

There are many more below but I'll stop commenting.

Review

Quoted 2 lines
(quoted diff) +2 -0
+        if self.c_name in c_kw:
+            self.c_name += '_'

You also have this pattern quite a lot. Maybe a "c_safe()" wrapper or something :)

FWIW, I'm still looking for "c_kw", maybe "pythonically" that should be "_C_KW" since it's a private global? I think? Haven't seen it yet :)

tools/net/ynl/ynl-gen-c.py line 1145

Quoted 3 lines
tools/net/ynl/ynl-gen-c.py new file +3 -0
+c_kw = {
+    'do'
+}

Aha, here it is :-)

Review

Quoted 2 lines
(quoted diff) +2 -0
+            if has_ntf:
+                cw.p('// --------------- Common notification parsing --------------- //')

You said you were using /* */ comments now but this is still there.

tools/net/ynl/ynl-gen-c.py line 2310

Quoted 4 lines
tools/net/ynl/ynl-gen-c.py new file +4 -0
+                print_ntf_parse_prototype(parsed, cw)
+            cw.nl()
+        else:
+            cw.p('// Policies')

and here too, etc.

Whew. I think I skipped some bits ;-)

Doesn't look that bad overall, IMHO. :)

johannes

author maintainer
#19 lore raw

Re: [PATCH net-next v3 3/8] net: add basic C code generators for Netlink

Quoted 6 lines · On Thu, 19 Jan 2023 21:53:12 +0100 Johannes Berg wrote:

tools/net/ynl/ynl-gen-c.py line 362

Quoted 4 lines
tools/net/ynl/ynl-gen-c.py new file +4 -0
+    def _attr_policy(self, policy):
+        mem = '{ '
+        if len(self.checks) == 1 and 'min-len' in self.checks:
+            mem += '.len = ' + str(self.checks['min-len'])  

Why does the len(self.checks) matter?

Trying to throw an exception if someone starts using checks I haven't gotten to implementing yet.

Quoted 10 lines

tools/net/ynl/ynl-gen-c.py line 438

Quoted 5 lines
tools/net/ynl/ynl-gen-c.py new file +5 -0
+    def free_needs_iter(self):
+        return 'type' not in self.attr or self.attr['type'] == 'nest'
+
+    def free(self, ri, var, ref):
+        if 'type' not in self.attr or self.attr['type'] == 'nest':  

two more places that could use the .get trick

but it's really up to you. Just that the line like that seems rather long to me :-)

Better question is why attr would not have a type :S Let me leave it be for now, I'll revisit when exercising this code in the future.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+            if has_ntf:
+                cw.p('// --------------- Common notification parsing --------------- //')  

You said you were using /* */ comments now but this is still there.

Ugh, now I'm worried I lost something in a rebase :S

Quoted 8 lines

tools/net/ynl/ynl-gen-c.py line 2310

Quoted 4 lines
tools/net/ynl/ynl-gen-c.py new file +4 -0
+                print_ntf_parse_prototype(parsed, cw)
+            cw.nl()
+        else:
+            cw.p('// Policies')  

and here too, etc.

Whew. I think I skipped some bits ;-)

Thanks for looking!! :)

Quoted 1 line

Doesn't look that bad overall, IMHO. :)

I hope we can avoid over-focusing on the python tools :P I'm no python expert and the code would use a lot of refactoring. But there's only so many hours in the day and the alternative seems that it will bit rot in my tree forever :(

#20 lore raw

Re: [PATCH net-next v3 3/8] net: add basic C code generators for Netlink

Quoted 4 lines · Jakub Kicinski
Quoted 2 lines

Doesn't look that bad overall, IMHO. :)

I hope we can avoid over-focusing on the python tools :P

Oh sure.

Quoted 4 lines

I'm no python expert and the code would use a lot of refactoring. But there's only so many hours in the day and the alternative seems that it will bit rot in my tree forever :(

:)

Yeah I don't mind. The quality of the implementation matters to a point, but honestly I was just reading through this to convince myself it wasn't just a totally stinking pile of hacks ;-) Which you/I have managed, so personally I'm happy with this.

johannes

author maintainer
#21 lore raw

PATCH v3 4/8 netlink: add a proto specification for FOU

In reply to Jakub Kicinski

FOU has a reasonably modern Genetlink family. Add a spec.

---
 Documentation/netlink/specs/fou.yaml | 128 +++++++++++++++++++++++++++
 1 file changed, 128 insertions(+)
 create mode 100644 Documentation/netlink/specs/fou.yaml
Documentation/netlink/specs/fou.yaml new file +128 -0
new file mode 100644
index 000000000000..266c386eedf3
@@ -0,0 +1,128 @@
+name: fou
+
+protocol: genetlink-legacy
+
+doc: |
+  Foo-over-UDP.
+
+c-family-name: fou-genl-name
+c-version-name: fou-genl-version
+max-by-define: true
+kernel-policy: global
+
+definitions:
+  -
+    type: enum
+    name: encap_type
+    name-prefix: fou-encap-
+    enum-name:
+    entries: [ unspec, direct, gue ]
+
+attribute-sets:
+  -
+    name: fou
+    name-prefix: fou-attr-
+    attributes:
+      -
+        name: unspec
+        type: unused
+      -
+        name: port
+        type: u16
+        byte-order: big-endian
+      -
+        name: af
+        type: u8
+      -
+        name: ipproto
+        type: u8
+      -
+        name: type
+        type: u8
+      -
+        name: remcsum_nopartial
+        type: flag
+      -
+        name: local_v4
+        type: u32
+      -
+        name: local_v6
+        type: binary
+        checks:
+          min-len: 16
+      -
+        name: peer_v4
+        type: u32
+      -
+        name: peer_v6
+        type: binary
+        checks:
+          min-len: 16
+      -
+        name: peer_port
+        type: u16
+        byte-order: big-endian
+      -
+        name: ifindex
+        type: s32
+
+operations:
+  list:
+    -
+      name: unspec
+      doc: unused
+
+    -
+      name: add
+      doc: Add port.
+      attribute-set: fou
+
+      dont-validate: [ strict, dump ]
+      flags: [ admin-perm ]
+
+      do:
+        request: &all_attrs
+          attributes:
+            - port
+            - ipproto
+            - type
+            - remcsum_nopartial
+            - local_v4
+            - peer_v4
+            - local_v6
+            - peer_v6
+            - peer_port
+            - ifindex
+
+    -
+      name: del
+      doc: Delete port.
+      attribute-set: fou
+
+      dont-validate: [ strict, dump ]
+      flags: [ admin-perm ]
+
+      do:
+        request:  &select_attrs
+          attributes:
+          - af
+          - ifindex
+          - port
+          - peer_port
+          - local_v4
+          - peer_v4
+          - local_v6
+          - peer_v6
+
+    -
+      name: get
+      doc: Get tunnel info.
+      attribute-set: fou
+      dont-validate: [ strict, dump ]
+
+      do:
+        request: *select_attrs
+        reply: *all_attrs
+
+      dump:
+        reply: *all_attrs
Signature · 1 line
2.39.0
author maintainer
#22 lore raw

PATCH v3 5/8 net: fou: regenerate the uAPI from the spec

In reply to Jakub Kicinski

Regenerate the FOU uAPI header from the YAML spec.

The flags now come before attributes which use them, and the comments for type disappear (coders should look at the spec instead).

---
 include/uapi/linux/fou.h | 54 +++++++++++++++++++---------------------
 1 file changed, 26 insertions(+), 28 deletions(-)
include/uapi/linux/fou.h +26 -28
index 87c2c9f08803..19ebbef41a63 100644
@@ -1,32 +1,37 @@
 /* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-/* fou.h - FOU Interface */
+/* Do not edit directly, auto-generated from: */
+/*	Documentation/netlink/specs/fou.yaml */
+/* YNL-GEN uapi header */
 
 #ifndef _UAPI_LINUX_FOU_H
 #define _UAPI_LINUX_FOU_H
 
-/* NETLINK_GENERIC related info
- */
 #define FOU_GENL_NAME		"fou"
-#define FOU_GENL_VERSION	0x1
+#define FOU_GENL_VERSION	1
 
 enum {
-	FOU_ATTR_UNSPEC,
-	FOU_ATTR_PORT,				/* u16 */
-	FOU_ATTR_AF,				/* u8 */
-	FOU_ATTR_IPPROTO,			/* u8 */
-	FOU_ATTR_TYPE,				/* u8 */
-	FOU_ATTR_REMCSUM_NOPARTIAL,		/* flag */
-	FOU_ATTR_LOCAL_V4,			/* u32 */
-	FOU_ATTR_LOCAL_V6,			/* in6_addr */
-	FOU_ATTR_PEER_V4,			/* u32 */
-	FOU_ATTR_PEER_V6,			/* in6_addr */
-	FOU_ATTR_PEER_PORT,			/* u16 */
-	FOU_ATTR_IFINDEX,			/* s32 */
-
-	__FOU_ATTR_MAX,
+	FOU_ENCAP_UNSPEC,
+	FOU_ENCAP_DIRECT,
+	FOU_ENCAP_GUE,
 };
 
-#define FOU_ATTR_MAX		(__FOU_ATTR_MAX - 1)
+enum {
+	FOU_ATTR_UNSPEC,
+	FOU_ATTR_PORT,
+	FOU_ATTR_AF,
+	FOU_ATTR_IPPROTO,
+	FOU_ATTR_TYPE,
+	FOU_ATTR_REMCSUM_NOPARTIAL,
+	FOU_ATTR_LOCAL_V4,
+	FOU_ATTR_LOCAL_V6,
+	FOU_ATTR_PEER_V4,
+	FOU_ATTR_PEER_V6,
+	FOU_ATTR_PEER_PORT,
+	FOU_ATTR_IFINDEX,
+
+	__FOU_ATTR_MAX
+};
+#define FOU_ATTR_MAX (__FOU_ATTR_MAX - 1)
 
 enum {
 	FOU_CMD_UNSPEC,
@@ -34,15 +39,8 @@ enum {
 	FOU_CMD_DEL,
 	FOU_CMD_GET,
 
-	__FOU_CMD_MAX,
+	__FOU_CMD_MAX
 };
-
-enum {
-	FOU_ENCAP_UNSPEC,
-	FOU_ENCAP_DIRECT,
-	FOU_ENCAP_GUE,
-};
-
-#define FOU_CMD_MAX	(__FOU_CMD_MAX - 1)
+#define FOU_CMD_MAX (__FOU_CMD_MAX - 1)
 
 #endif /* _UAPI_LINUX_FOU_H */
Signature · 1 line
2.39.0
author maintainer
#23 lore raw

PATCH v3 6/8 net: fou: rename the source for linking

In reply to Jakub Kicinski

We'll need to link two objects together to form the fou module. This means the source can't be called fou, the build system expects fou.o to be the combined object.

---
 net/ipv4/Makefile              | 1 +
 net/ipv4/{fou.c => fou_core.c} | 0
 2 files changed, 1 insertion(+)
 rename net/ipv4/{fou.c => fou_core.c} (100%)
net/ipv4/Makefile +1 -0
index af7d2cf490fb..fabbe46897ce 100644
@@ -26,6 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o
 obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o
 obj-$(CONFIG_NET_IPIP) += ipip.o
 gre-y := gre_demux.o
+fou-y := fou_core.o
 obj-$(CONFIG_NET_FOU) += fou.o
 obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o
 obj-$(CONFIG_NET_IPGRE) += ip_gre.o
net/ipv4/fou_core.c renamed from net/ipv4/fou.c +0 -0
similarity index 100%
rename from net/ipv4/fou.c
rename to net/ipv4/fou_core.c
Signature · 1 line
2.39.0
author maintainer
#24 lore raw

PATCH v3 7/8 net: fou: use policy and operation tables generated from the spec

In reply to Jakub Kicinski

Generate and plug in the spec-based tables.

A little bit of renaming is needed in the FOU code.

---
 net/ipv4/Makefile   |  2 +-
 net/ipv4/fou-nl.c   | 48 +++++++++++++++++++++++++++++++++++++++++++++
 net/ipv4/fou-nl.h   | 25 +++++++++++++++++++++++
 net/ipv4/fou_core.c | 47 +++++++-------------------------------------
 4 files changed, 81 insertions(+), 41 deletions(-)
 create mode 100644 net/ipv4/fou-nl.c
 create mode 100644 net/ipv4/fou-nl.h
net/ipv4/Makefile +1 -1
index fabbe46897ce..984da5159afb 100644
@@ -26,7 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o
 obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o
 obj-$(CONFIG_NET_IPIP) += ipip.o
 gre-y := gre_demux.o
-fou-y := fou_core.o
+fou-y := fou_core.o fou-nl.o
 obj-$(CONFIG_NET_FOU) += fou.o
 obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o
 obj-$(CONFIG_NET_IPGRE) += ip_gre.o
net/ipv4/fou-nl.c new file +48 -0

48 lines not on this page · raw message

net/ipv4/fou-nl.h new file +25 -0
new file mode 100644
index 000000000000..fc1d1c671c4d
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: BSD-3-Clause
+/* Do not edit directly, auto-generated from: */
+/*	Documentation/netlink/specs/fou.yaml */
+/* YNL-GEN kernel header */
+
+#ifndef _LINUX_FOU_GEN_H
+#define _LINUX_FOU_GEN_H
+
+#include <net/netlink.h>
+#include <net/genetlink.h>
+
+#include <linux/fou.h>
+
+// Global operation policy for fou
+extern const struct nla_policy fou_nl_policy[FOU_ATTR_IFINDEX + 1];
+
+// Ops table for fou
+extern const struct genl_small_ops fou_nl_ops[3];
+
+int fou_nl_add_doit(struct sk_buff *skb, struct genl_info *info);
+int fou_nl_del_doit(struct sk_buff *skb, struct genl_info *info);
+int fou_nl_get_doit(struct sk_buff *skb, struct genl_info *info);
+int fou_nl_get_dumpit(struct sk_buff *skb, struct netlink_callback *cb);
+
+#endif /* _LINUX_FOU_GEN_H */
net/ipv4/fou_core.c +7 -40

94 lines not on this page · raw message

Signature · 1 line
2.39.0
#25 lore raw

Re: [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec

Quoted 2 lines · Jakub Kicinski

+fou-y := fou_core.o fou-nl.o

I feel like you could've been consistent there and used "fou-core.o" with a dash, but hey :-)

johannes

#26 lore raw

Re: [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec

Quoted 8 lines · Johannes Berg
Quoted 2 lines · Jakub Kicinski

+fou-y := fou_core.o fou-nl.o

I feel like you could've been consistent there and used "fou-core.o" with a dash, but hey :-)

johannes

And here I lean towards using _ instead of - in file names but consistency is good, all else being equal.

author maintainer
#27 lore raw

Re: [PATCH net-next v3 7/8] net: fou: use policy and operation tables generated from the spec

Quoted 9 lines · On Thu, 19 Jan 2023 16:18:04 -0800 Jacob Keller wrote:
Quoted 5 lines · Johannes Berg
Quoted 1 line · Jakub Kicinski

+fou-y := fou_core.o fou-nl.o

I feel like you could've been consistent there and used "fou-core.o" with a dash, but hey :-)

And here I lean towards using _ instead of - in file names but consistency is good, all else being equal.

Done. IIRC the codegen used to assume the naming (it needs to know the name of the header to include). I fixed that but forgot to rename fou :S

author maintainer
#28 lore raw

PATCH v3 8/8 tools: ynl: add a completely generic client

In reply to Jakub Kicinski

Add a CLI sample which can take in arbitrary request in JSON format, convert it to Netlink and do the inverse for output.

It's meant as a development tool primarily and perhaps for selftests which need to tickle netlink in a special way.

---
 tools/net/ynl/samples/cli.py |  47 +++
 tools/net/ynl/samples/ynl.py | 534 +++++++++++++++++++++++++++++++++++
 2 files changed, 581 insertions(+)
 create mode 100755 tools/net/ynl/samples/cli.py
 create mode 100644 tools/net/ynl/samples/ynl.py
tools/net/ynl/samples/cli.py new file +47 -0
new file mode 100755
index 000000000000..b27159c70710
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+
+import argparse
+import json
+import pprint
+import time
+
+from ynl import YnlFamily
+
+
+def main():
+    parser = argparse.ArgumentParser(description='YNL CLI sample')
+    parser.add_argument('--spec', dest='spec', type=str, required=True)
+    parser.add_argument('--schema', dest='schema', type=str)
+    parser.add_argument('--json', dest='json_text', type=str)
+    parser.add_argument('--do', dest='do', type=str)
+    parser.add_argument('--dump', dest='dump', type=str)
+    parser.add_argument('--sleep', dest='sleep', type=int)
+    parser.add_argument('--subscribe', dest='ntf', type=str)
+    args = parser.parse_args()
+
+    attrs = {}
+    if args.json_text:
+        attrs = json.loads(args.json_text)
+
+    ynl = YnlFamily(args.spec, args.schema)
+
+    if args.ntf:
+        ynl.ntf_subscribe(args.ntf)
+
+    if args.sleep:
+        time.sleep(args.sleep)
+
+    if args.do or args.dump:
+        method = getattr(ynl, args.do if args.do else args.dump)
+
+        reply = method(attrs, dump=bool(args.dump))
+        pprint.PrettyPrinter().pprint(reply)
+
+    if args.ntf:
+        ynl.check_ntf()
+        pprint.PrettyPrinter().pprint(ynl.async_msg_queue)
+
+
+if __name__ == "__main__":
+    main()
tools/net/ynl/samples/ynl.py new file +534 -0

534 lines not on this page · raw message

Signature · 1 line
2.39.0
#29 lore raw

Re: [PATCH net-next v3 8/8] tools: ynl: add a completely generic client

Quoted 78 lines · Jakub Kicinski

Add a CLI sample which can take in arbitrary request in JSON format, convert it to Netlink and do the inverse for output.

It's meant as a development tool primarily and perhaps for selftests which need to tickle netlink in a special way.

---
 tools/net/ynl/samples/cli.py |  47 +++
 tools/net/ynl/samples/ynl.py | 534 +++++++++++++++++++++++++++++++++++
 2 files changed, 581 insertions(+)
 create mode 100755 tools/net/ynl/samples/cli.py
 create mode 100644 tools/net/ynl/samples/ynl.py
tools/net/ynl/samples/cli.py new file +47 -0
new file mode 100755
index 000000000000..b27159c70710
@@ -0,0 +1,47 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: BSD-3-Clause
+
+import argparse
+import json
+import pprint
+import time
+
+from ynl import YnlFamily
+
+
+def main():
+    parser = argparse.ArgumentParser(description='YNL CLI sample')
+    parser.add_argument('--spec', dest='spec', type=str, required=True)
+    parser.add_argument('--schema', dest='schema', type=str)
+    parser.add_argument('--json', dest='json_text', type=str)
+    parser.add_argument('--do', dest='do', type=str)
+    parser.add_argument('--dump', dest='dump', type=str)
+    parser.add_argument('--sleep', dest='sleep', type=int)
+    parser.add_argument('--subscribe', dest='ntf', type=str)
+    args = parser.parse_args()
+
+    attrs = {}
+    if args.json_text:
+        attrs = json.loads(args.json_text)
+
+    ynl = YnlFamily(args.spec, args.schema)
+
+    if args.ntf:
+        ynl.ntf_subscribe(args.ntf)
+
+    if args.sleep:
+        time.sleep(args.sleep)
+
+    if args.do or args.dump:
+        method = getattr(ynl, args.do if args.do else args.dump)
+
+        reply = method(attrs, dump=bool(args.dump))
+        pprint.PrettyPrinter().pprint(reply)
+
+    if args.ntf:
+        ynl.check_ntf()
+        pprint.PrettyPrinter().pprint(ynl.async_msg_queue)
+
+
+if __name__ == "__main__":
+    main()
tools/net/ynl/samples/ynl.py new file +4 -0
new file mode 100644
index 000000000000..b71523d71d46
@@ -0,0 +1,534 @@
+# SPDX-License-Identifier: BSD-3-Clause
+
+import functools
+import jsonschema

tools/net/ynl/samples/ynl.py line 14

Quoted 10 lines
tools/net/ynl/samples/ynl.py new file +10 -0
+import os
+import random
+import socket
+import struct
+import yaml
+
+#
+# Generic Netlink code which should really be in some library, but I can't quickly find one.
+#
+

There is pyroute2, but it might be overkill, and I recall it seeming to do some things a bit inconsistently.

Acked-by on the v3 series
#30 lore raw
In reply to Jakub Kicinski
Quoted 10 lines · Jakub Kicinski

I think the Netlink proto specs are far along enough to merge. Filling in all attribute types and quirks will be an ongoing effort but we have enough to cover FOU so it's somewhat complete.

I fully intend to continue polishing the code but at the same time I'd like to start helping others base their work on the specs (e.g. DPLL) and need to start working on some new families myself.

Quoted 7 lines

That's the progress / motivation for merging. The RFC [1] has more of a high level blurb, plus I created a lot of documentation, I'm not going to repeat it here. There was also the talk at LPC [2].

[1] https://lore.kernel.org/all/[email protected]/
[2] https://youtu.be/9QkXIQXkaQk?t=2562
v2: https://lore.kernel.org/all/[email protected]/

I think this is awesome and is definitely a step in the right direction. I did a light review, nothing pops up, thanks for the spec fields renamings.

If that helps feel free to attach:

Quoted 50 lines

Jakub Kicinski (8):
docs: add more netlink docs (incl. spec docs)
netlink: add schemas for YAML specs
net: add basic C code generators for Netlink
netlink: add a proto specification for FOU
net: fou: regenerate the uAPI from the spec
net: fou: rename the source for linking
net: fou: use policy and operation tables generated from the spec
tools: ynl: add a completely generic client

 Documentation/core-api/index.rst              |    1 +
 Documentation/core-api/netlink.rst            |   99 +
 Documentation/netlink/genetlink-c.yaml        |  318 +++
 Documentation/netlink/genetlink-legacy.yaml   |  346 +++
 Documentation/netlink/genetlink.yaml          |  284 ++
 Documentation/netlink/specs/fou.yaml          |  128 +
 .../userspace-api/netlink/c-code-gen.rst      |  107 +
 .../netlink/genetlink-legacy.rst              |   96 +
 Documentation/userspace-api/netlink/index.rst |    5 +
 Documentation/userspace-api/netlink/specs.rst |  422 +++
 MAINTAINERS                                   |    3 +
 include/uapi/linux/fou.h                      |   54 +-
 net/ipv4/Makefile                             |    1 +
 net/ipv4/fou-nl.c                             |   48 +
 net/ipv4/fou-nl.h                             |   25 +
 net/ipv4/{fou.c => fou_core.c}                |   47 +-
 tools/net/ynl/samples/cli.py                  |   47 +
 tools/net/ynl/samples/ynl.py                  |  534 ++++
 tools/net/ynl/ynl-gen-c.py                    | 2376 +++++++++++++++++
 tools/net/ynl/ynl-regen.sh                    |   30 +
 20 files changed, 4903 insertions(+), 68 deletions(-)
 create mode 100644 Documentation/core-api/netlink.rst
 create mode 100644 Documentation/netlink/genetlink-c.yaml
 create mode 100644 Documentation/netlink/genetlink-legacy.yaml
 create mode 100644 Documentation/netlink/genetlink.yaml
 create mode 100644 Documentation/netlink/specs/fou.yaml
 create mode 100644 Documentation/userspace-api/netlink/c-code-gen.rst
 create mode 100644 Documentation/userspace-api/netlink/genetlink-legacy.rst
 create mode 100644 Documentation/userspace-api/netlink/specs.rst
 create mode 100644 net/ipv4/fou-nl.c
 create mode 100644 net/ipv4/fou-nl.h
 rename net/ipv4/{fou.c => fou_core.c} (94%)
 create mode 100755 tools/net/ynl/samples/cli.py
 create mode 100644 tools/net/ynl/samples/ynl.py
 create mode 100755 tools/net/ynl/ynl-gen-c.py
 create mode 100755 tools/net/ynl/ynl-regen.sh

--
2.39.0