ynl: rename array-nest to indexed-array
Subsystems (4)
v3 cover ynl: rename array-nest to indexed-array
rename array-nest to indexed-array and add un-nest sub-type support
v3:
1. fix doc title underline too short issue (Jakub Kicinski)
v2:
1. raise exception for unsupported sub-type
2. merge all sub-type handler in _decode_array_attr
3. remove index shown in indexed-array as some implementations are non-contiguous.
Hangbin Liu (2):
ynl: rename array-nest to indexed-array
ynl: support binary/u32 sub-type for indexed-array
Documentation/netlink/genetlink-c.yaml | 2 +- Documentation/netlink/genetlink-legacy.yaml | 2 +- Documentation/netlink/genetlink.yaml | 2 +- Documentation/netlink/netlink-raw.yaml | 2 +- Documentation/netlink/specs/nlctrl.yaml | 6 +++-- Documentation/netlink/specs/rt_link.yaml | 3 ++- Documentation/netlink/specs/tc.yaml | 21 ++++++++++------ .../netlink/genetlink-legacy.rst | 24 ++++++++++++++----- tools/net/ynl/lib/ynl.py | 18 ++++++++++---- tools/net/ynl/ynl-gen-c.py | 18 +++++++++----- 10 files changed, 67 insertions(+), 31 deletions(-)
Signature · 1 line
2.43.0
PATCH v3 1/2 ynl: rename array-nest to indexed-array
Some implementations, like bonding, has nest array with same attr type. To support all kinds of entries under one nest array. As discussed[1], let's rename array-nest to indexed-array, and assuming the value is a nest by passing the type via sub-type.
[1] https://lore.kernel.org/netdev/[email protected]/
--- Documentation/netlink/genetlink-c.yaml | 2 +- Documentation/netlink/genetlink-legacy.yaml | 2 +- Documentation/netlink/genetlink.yaml | 2 +- Documentation/netlink/netlink-raw.yaml | 2 +- Documentation/netlink/specs/nlctrl.yaml | 6 ++++-- Documentation/netlink/specs/rt_link.yaml | 3 ++- Documentation/netlink/specs/tc.yaml | 21 ++++++++++++------- .../netlink/genetlink-legacy.rst | 12 ++++++++--- tools/net/ynl/lib/ynl.py | 13 +++++++----- tools/net/ynl/ynl-gen-c.py | 18 ++++++++++------ 10 files changed, 53 insertions(+), 28 deletions(-)
Documentation/netlink/genetlink-c.yaml +1 -1
@@ -158,7 +158,7 @@ properties: type: &attr-type enum: [ unused, pad, flag, binary, uint, sint, u8, u16, u32, u64, s32, s64, - string, nest, array-nest, nest-type-value ] + string, nest, indexed-array, nest-type-value ] doc: description: Documentation of the attribute. type: string
Documentation/netlink/genetlink-legacy.yaml +1 -1
@@ -201,7 +201,7 @@ properties: description: The netlink attribute type enum: [ unused, pad, flag, binary, bitfield32, uint, sint, u8, u16, u32, u64, s32, s64, - string, nest, array-nest, nest-type-value ] + string, nest, indexed-array, nest-type-value ] doc: description: Documentation of the attribute. type: string
Documentation/netlink/genetlink.yaml +1 -1
@@ -124,7 +124,7 @@ properties: type: &attr-type enum: [ unused, pad, flag, binary, uint, sint, u8, u16, u32, u64, s32, s64, - string, nest, array-nest, nest-type-value ] + string, nest, indexed-array, nest-type-value ] doc: description: Documentation of the attribute. type: string
Documentation/netlink/netlink-raw.yaml +1 -1
@@ -222,7 +222,7 @@ properties: description: The netlink attribute type enum: [ unused, pad, flag, binary, bitfield32, u8, u16, u32, u64, s8, s16, s32, s64, - string, nest, array-nest, nest-type-value, + string, nest, indexed-array, nest-type-value, sub-message ] doc: description: Documentation of the attribute.
Documentation/netlink/specs/nlctrl.yaml +4 -2
@@ -65,11 +65,13 @@ attribute-sets: type: u32 - name: ops - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: op-attrs - name: mcast-groups - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: mcast-group-attrs - name: policy
Documentation/netlink/specs/rt_link.yaml +2 -1
@@ -1690,7 +1690,8 @@ attribute-sets: type: binary - name: hw-s-info - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: hw-s-info-one - name: l3-stats
Documentation/netlink/specs/tc.yaml +14 -7
@@ -1937,7 +1937,8 @@ attribute-sets: nested-attributes: tc-ematch-attrs - name: act - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-act-attrs - name: police
@@ -2077,7 +2078,8 @@ attribute-sets: type: u32 - name: tin-stats - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-cake-tin-stats-attrs - name: deficit
@@ -2297,7 +2299,8 @@ attribute-sets: type: string - name: act - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-act-attrs - name: key-eth-dst
@@ -2798,7 +2801,8 @@ attribute-sets: type: string - name: act - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-act-attrs - name: mask
@@ -2951,7 +2955,8 @@ attribute-sets: type: u32 - name: act - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-act-attrs - name: flags
@@ -3324,7 +3329,8 @@ attribute-sets: nested-attributes: tc-police-attrs - name: act - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-act-attrs - name: tc-taprio-attrs
@@ -3542,7 +3548,8 @@ attribute-sets: nested-attributes: tc-police-attrs - name: act - type: array-nest + type: indexed-array + sub-type: nest nested-attributes: tc-act-attrs - name: indev
Documentation/userspace-api/netlink/genetlink-legacy.rst +9 -3
@@ -46,10 +46,16 @@ For reference the ``multi-attr`` array may look like this:: where ``ARRAY-ATTR`` is the array entry type. -array-nest -~~~~~~~~~~ +indexed-array +~~~~~~~~~~~~~ + +``indexed-array`` wraps the entire array in an extra attribute (hence +limiting its size to 64kB). The ``ENTRY`` nests are special and have the +index of the entry as their type instead of normal attribute type. -``array-nest`` creates the following structure:: +A ``sub-type`` is needed to describe what type in the ``ENTRY``. A ``nest`` +``sub-type`` means there are nest arrays in the ``ENTRY``, with the structure +looks like:: [SOME-OTHER-ATTR] [ARRAY-ATTR]
tools/net/ynl/lib/ynl.py +8 -5
@@ -630,15 +630,18 @@ class YnlFamily(SpecFamily): decoded = self._formatted_string(decoded, attr_spec.display_hint) return decoded - def _decode_array_nest(self, attr, attr_spec): + def _decode_array_attr(self, attr, attr_spec): decoded = [] offset = 0 while offset < len(attr.raw): item = NlAttr(attr.raw, offset) offset += item.full_len - subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) - decoded.append({ item.type: subattrs }) + if attr_spec["sub-type"] == 'nest': + subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) + decoded.append({ item.type: subattrs }) + else: + raise Exception(f'Unknown {attr_spec["sub-type"]} with name {attr_spec["name"]}') return decoded def _decode_nest_type_value(self, attr, attr_spec):
@@ -732,8 +735,8 @@ class YnlFamily(SpecFamily): decoded = attr.as_scalar(attr_spec['type'], attr_spec.byte_order) if 'enum' in attr_spec: decoded = self._decode_enum(decoded, attr_spec) - elif attr_spec["type"] == 'array-nest': - decoded = self._decode_array_nest(attr, attr_spec) + elif attr_spec["type"] == 'indexed-array': + decoded = self._decode_array_attr(attr, attr_spec) elif attr_spec["type"] == 'bitfield32': value, selector = struct.unpack("II", attr.raw) if 'enum' in attr_spec:
tools/net/ynl/ynl-gen-c.py +12 -6
@@ -841,8 +841,11 @@ class AttrSet(SpecAttrSet): t = TypeBitfield32(self.family, self, elem, value) elif elem['type'] == 'nest': t = TypeNest(self.family, self, elem, value) - elif elem['type'] == 'array-nest': - t = TypeArrayNest(self.family, self, elem, value) + elif elem['type'] == 'indexed-array' and 'sub-type' in elem: + if elem["sub-type"] == 'nest': + t = TypeArrayNest(self.family, self, elem, value) + else: + raise Exception(f'new_attr: unsupported sub-type {elem["sub-type"]}') elif elem['type'] == 'nest-type-value': t = TypeNestTypeValue(self.family, self, elem, value) else:
@@ -1055,7 +1058,7 @@ class Family(SpecFamily): if nested in self.root_sets: raise Exception("Inheriting members to a space used as root not supported") inherit.update(set(spec['type-value'])) - elif spec['type'] == 'array-nest': + elif spec['type'] == 'indexed-array': inherit.add('idx') self.pure_nested_structs[nested].set_inherited(inherit)
@@ -1619,9 +1622,12 @@ def _multi_parse(ri, struct, init_lines, local_vars): multi_attrs = set() needs_parg = False for arg, aspec in struct.member_list(): - if aspec['type'] == 'array-nest': - local_vars.append(f'const struct nlattr *attr_{aspec.c_name};') - array_nests.add(arg) + if aspec['type'] == 'indexed-array' and 'sub-type' in aspec: + if aspec["sub-type"] == 'nest': + local_vars.append(f'const struct nlattr *attr_{aspec.c_name};') + array_nests.add(arg) + else: + raise Exception(f'Not supported sub-type {aspec["sub-type"]}') if 'multi-attr' in aspec: multi_attrs.add(arg) needs_parg |= 'nested-attributes' in aspec
Signature · 1 line
2.43.0
PATCH v3 2/2 ynl: support binary/u32 sub-type for indexed-array
In reply to Hangbin LiuAdd binary/u32 sub-type support for indexed-array to display bond arp and ns targets. Here is what the result looks like:
# ip link add bond0 type bond mode 1 \
arp_ip_target 192.168.1.1,192.168.1.2 ns_ip6_target 2001::1,2001::2
# ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/rt_link.yaml \
--do getlink --json '{"ifname": "bond0"}' --output-json | jq '.linkinfo' "arp-ip-target": [
"192.168.1.1",
"192.168.1.2"
],
[...]
"ns-ip6-target": [
"2001::1",
"2001::2"
],--- .../userspace-api/netlink/genetlink-legacy.rst | 12 +++++++++--- tools/net/ynl/lib/ynl.py | 5 +++++ 2 files changed, 14 insertions(+), 3 deletions(-)
Documentation/userspace-api/netlink/genetlink-legacy.rst +9 -3
@@ -66,9 +66,15 @@ looks like:: [MEMBER1] [MEMBER2] -It wraps the entire array in an extra attribute (hence limiting its size -to 64kB). The ``ENTRY`` nests are special and have the index of the entry -as their type instead of normal attribute type. +Other ``sub-type`` like ``u32`` means there is only one member as described +in ``sub-type`` in the ``ENTRY``. The structure looks like:: + + [SOME-OTHER-ATTR] + [ARRAY-ATTR] + [ENTRY] + [MEMBER1] + [ENTRY] + [MEMBER1] type-value ~~~~~~~~~~
tools/net/ynl/lib/ynl.py +5 -0
@@ -640,6 +640,11 @@ class YnlFamily(SpecFamily): if attr_spec["sub-type"] == 'nest': subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) decoded.append({ item.type: subattrs }) + elif attr_spec["sub-type"] == 'binary' or attr_spec["sub-type"] == 'u32': + subattrs = item.as_bin() + if attr_spec.display_hint: + subattrs = self._formatted_string(subattrs, attr_spec.display_hint) + decoded.append(subattrs) else: raise Exception(f'Unknown {attr_spec["sub-type"]} with name {attr_spec["name"]}') return decoded
Signature · 1 line
2.43.0
Jakub Kicinski
maintainerRe: [PATCHv3 net-next 2/2] ynl: support binary/u32 sub-type for indexed-array
Documentation/userspace-api/netlink/genetlink-legacy.rst looks like:: · line 77
Quoted 9 lines · On Mon, 1 Apr 2024 11:56:51 +0800 Hangbin Liu wrote:
Documentation/userspace-api/netlink/genetlink-legacy.rst +9
-0+Other ``sub-type`` like ``u32`` means there is only one member as described +in ``sub-type`` in the ``ENTRY``. The structure looks like:: + + [SOME-OTHER-ATTR] + [ARRAY-ATTR] + [ENTRY] + [MEMBER1] + [ENTRY] + [MEMBER1]
I think that elsewhere in the doc we use [SOMETHING] to mean TLV of type SOMETHING, here MEMBER1/2 are presumably just payloads of each ENTRY? Maybe this is better:
[SOME-OTHER-ATTR]
[ARRAY-ATTR]
[ENTRY u32]
[ENTRY u32]?
tools/net/ynl/lib/ynl.py class YnlFamily(SpecFamily): · line 644
Quoted 12 lines
type-value
~~~~~~~~~~tools/net/ynl/lib/ynl.py +2
-0@@ -640,6 +640,11 @@ class YnlFamily(SpecFamily): if attr_spec["sub-type"] == 'nest': subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) decoded.append({ item.type: subattrs }) + elif attr_spec["sub-type"] == 'binary' or attr_spec["sub-type"] == 'u32': + subattrs = item.as_bin()
Are you sure that as_bin() will work for all u32s? Or just when there's a hint...
Quoted 3 lines
tools/net/ynl/lib/ynl.py +3
-0+ if attr_spec.display_hint: + subattrs = self._formatted_string(subattrs, attr_spec.display_hint) + decoded.append(subattrs)
Re: [PATCHv3 net-next 2/2] ynl: support binary/u32 sub-type for indexed-array
Quoted 9 lines · Jakub Kicinski
I think that elsewhere in the doc we use [SOMETHING] to mean TLV of type SOMETHING, here MEMBER1/2 are presumably just payloads of each ENTRY? Maybe this is better:
[SOME-OTHER-ATTR]
[ARRAY-ATTR][ENTRY u32] [ENTRY u32]
Thanks, I will update the doc.
Quoted 17 lines
?
tools/net/ynl/lib/ynl.py class YnlFamily(SpecFamily): · line 644
Quoted 12 lines
type-value
~~~~~~~~~~tools/net/ynl/lib/ynl.py +2
-0@@ -640,6 +640,11 @@ class YnlFamily(SpecFamily): if attr_spec["sub-type"] == 'nest': subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) decoded.append({ item.type: subattrs }) + elif attr_spec["sub-type"] == 'binary' or attr_spec["sub-type"] == 'u32': + subattrs = item.as_bin()Are you sure that
as_bin()will work for all u32s? Or just when there's a hint...
I didn't check other subsystem. For bonding only, if we don't have the hint. e.g.
-
name: arp-ip-target
type: indexed-array
sub-type: u32The result will looks like:
"arp-ip-target": [
"c0a80101",
"c0a80102"
],Which looks good to me. Do you have other suggestion?
Thanks
Hangbin
Jakub Kicinski
maintainerRe: [PATCHv3 net-next 2/2] ynl: support binary/u32 sub-type for indexed-array
Quoted 8 lines · On Wed, 3 Apr 2024 10:28:34 +0800 Hangbin Liu wrote:
I didn't check other subsystem. For bonding only, if we don't have the hint. e.g.
-
name: arp-ip-target type: indexed-array sub-type: u32
Quoted 8 lines
The result will looks like:
"arp-ip-target": [ "c0a80101", "c0a80102" ],Which looks good to me. Do you have other suggestion?
That doesn't look right, without the format hint if the type is u32 the members should be plain integers not hex strings.
Re: [PATCHv3 net-next 2/2] ynl: support binary/u32 sub-type for indexed-array
Quoted 16 lines · Jakub Kicinski
Quoted 15 lines · On Wed, 3 Apr 2024 10:28:34 +0800 Hangbin Liu wrote:
I didn't check other subsystem. For bonding only, if we don't have the hint. e.g.
-
name: arp-ip-target type: indexed-array sub-type: u32The result will looks like:
"arp-ip-target": [ "c0a80101", "c0a80102" ],
Quoted 4 lines
Quoted 1 line
Which looks good to me. Do you have other suggestion?
That doesn't look right, without the format hint if the type is u32 the members should be plain integers not hex strings.
OK, I can separate the binary and u32 dealing. How about like
tools/net/ynl/lib/ynl.py +10 -0
@@ -640,6 +640,16 @@ class YnlFamily(SpecFamily): if attr_spec["sub-type"] == 'nest': subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) decoded.append({ item.type: subattrs }) + elif attr_spec["sub-type"] == 'binary': + subattrs = item.as_bin() + if attr_spec.display_hint: + subattrs = self._formatted_string(subattrs, attr_spec.display_hint) + decoded.append(subattrs) + elif attr_spec["sub-type"] in NlAttr.type_formats: + subattrs = item.as_scalar(attr_spec['sub-type'], attr_spec.byte_order) + if attr_spec.display_hint: + subattrs = self._formatted_string(subattrs, attr_spec.display_hint) + decoded.append(subattrs) else: raise Exception(f'Unknown {attr_spec["sub-type"]} with name {attr_spec["name"]}') return decoded
With only sub-type: u32 it shows like
"arp-ip-target": [
3232235777,
3232235778
],Thanks
Hangbin
Jakub Kicinski
maintainerRe: [PATCHv3 net-next 2/2] ynl: support binary/u32 sub-type for indexed-array
Quoted 10 lines · On Wed, 3 Apr 2024 11:27:22 +0800 Hangbin Liu wrote:
OK, I can separate the binary and u32 dealing. How about like
tools/net/ynl/lib/ynl.py +0
-0@@ -640,6 +640,16 @@ class YnlFamily(SpecFamily): if attr_spec["sub-type"] == 'nest': subattrs = self._decode(NlAttrs(item.raw), attr_spec['nested-attributes']) decoded.append({ item.type: subattrs })
tools/net/ynl/lib/ynl.py class YnlFamily(SpecFamily): · line 652
Quoted 10 lines
tools/net/ynl/lib/ynl.py +10
-0+ elif attr_spec["sub-type"] == 'binary': + subattrs = item.as_bin() + if attr_spec.display_hint: + subattrs = self._formatted_string(subattrs, attr_spec.display_hint) + decoded.append(subattrs) + elif attr_spec["sub-type"] in NlAttr.type_formats: + subattrs = item.as_scalar(attr_spec['sub-type'], attr_spec.byte_order) + if attr_spec.display_hint: + subattrs = self._formatted_string(subattrs, attr_spec.display_hint) + decoded.append(subattrs)
Aha, this look right!