tools: ynl: handful of forward looking updates
Subsystems (3)
Jakub Kicinski
maintainercover tools: ynl: handful of forward looking updates
Small YNL improvements, mostly for work-in-progress families.
Jakub Kicinski (5):
tools: ynl: allow passing binary data
tools: ynl-gen: set length of binary fields
tools: ynl-gen: fix collecting global policy attrs
tools: ynl-gen: support empty attribute lists
netlink: specs: fix indent in fou
Documentation/netlink/specs/fou.yaml | 18 +++++++++--------- tools/net/ynl/generated/ethtool-user.h | 4 ++++ tools/net/ynl/generated/fou-user.h | 6 ++++++ tools/net/ynl/lib/ynl.py | 7 ++++++- tools/net/ynl/ynl-gen-c.py | 16 +++++++++++++--- 5 files changed, 38 insertions(+), 13 deletions(-)
Signature · 1 line
2.41.0
Jakub Kicinski
maintainerPATCH 1/5 tools: ynl: allow passing binary data
Recent changes made us assume that input for binary data is in hex. When using YNL as a Python library it's possible to pass in raw bytes. Bring the ability to do that back.
--- tools/net/ynl/lib/ynl.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
tools/net/ynl/lib/ynl.py +6 -1
@@ -410,7 +410,12 @@ genl_family_name_to_id = None elif attr["type"] == 'string': attr_payload = str(value).encode('ascii') + b'\x00' elif attr["type"] == 'binary': - attr_payload = bytes.fromhex(value) + if isinstance(value, bytes): + attr_payload = value + elif isinstance(value, str): + attr_payload = bytes.fromhex(value) + else: + raise Exception(f'Unknown type for binary attribute, value: {value}') elif attr['type'] in NlAttr.type_formats: format = NlAttr.get_format(attr['type'], attr.byte_order) attr_payload = format.pack(int(value))
Signature · 1 line
2.41.0
Re: [PATCH net-next 1/5] tools: ynl: allow passing binary data
Quoted 17 lines · Jakub Kicinski
Recent changes made us assume that input for binary data is in hex. When using YNL as a Python library it's possible to pass in raw bytes. Bring the ability to do that back.
Signed-off-by Jakub Kicinski <[email protected]>--- tools/net/ynl/lib/ynl.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)tools/net/ynl/lib/ynl.py +0
-0@@ -410,7 +410,12 @@ genl_family_name_to_id = None elif attr["type"] == 'string': attr_payload = str(value).encode('ascii') + b'\x00' elif attr["type"] == 'binary':
tools/net/ynl/lib/ynl.py genl_family_name_to_id = None · line 421
Quoted 10 lines
tools/net/ynl/lib/ynl.py +6
-1- attr_payload = bytes.fromhex(value) + if isinstance(value, bytes): + attr_payload = value + elif isinstance(value, str): + attr_payload = bytes.fromhex(value) + else: + raise Exception(f'Unknown type for binary attribute, value: {value}') elif attr['type'] in NlAttr.type_formats: format = NlAttr.get_format(attr['type'], attr.byte_order) attr_payload = format.pack(int(value))
Jakub Kicinski
maintainerPATCH 2/5 tools: ynl-gen: set length of binary fields
In reply to Jakub KicinskiRemember to set the length field in the request setters.
--- tools/net/ynl/generated/ethtool-user.h | 4 ++++ tools/net/ynl/generated/fou-user.h | 6 ++++++ tools/net/ynl/ynl-gen-c.py | 1 + 3 files changed, 11 insertions(+)
tools/net/ynl/generated/ethtool-user.h +4 -0
@@ -1422,6 +1422,7 @@ ethtool_wol_set_req_set_sopass(struct ethtool_wol_set_req *req, const void *sopass, size_t len) { free(req->sopass); + req->_present.sopass_len = len; req->sopass =malloc (req->_present.sopass_len);memcpy (req->sopass, sopass, req->_present.sopass_len); }
@@ -4071,6 +4072,7 @@ ethtool_fec_set_req_set_stats_corrected(struct ethtool_fec_set_req *req, const void *corrected, size_t len) { free(req->stats.corrected); + req->stats._present.corrected_len = len; req->stats.corrected =malloc (req->stats._present.corrected_len);memcpy (req->stats.corrected, corrected, req->stats._present.corrected_len); }
@@ -4079,6 +4081,7 @@ ethtool_fec_set_req_set_stats_uncorr(struct ethtool_fec_set_req *req, const void *uncorr, size_t len) { free(req->stats.uncorr); + req->stats._present.uncorr_len = len; req->stats.uncorr =malloc (req->stats._present.uncorr_len);memcpy (req->stats.uncorr, uncorr, req->stats._present.uncorr_len); }
@@ -4087,6 +4090,7 @@ ethtool_fec_set_req_set_stats_corr_bits(struct ethtool_fec_set_req *req, const void *corr_bits, size_t len) { free(req->stats.corr_bits); + req->stats._present.corr_bits_len = len; req->stats.corr_bits =malloc (req->stats._present.corr_bits_len);memcpy (req->stats.corr_bits, corr_bits, req->stats._present.corr_bits_len); }
tools/net/ynl/generated/fou-user.h +6 -0
@@ -91,6 +91,7 @@ fou_add_req_set_local_v6(struct fou_add_req *req, const void *local_v6, size_t len) {free (req->local_v6); + req->_present.local_v6_len = len; req->local_v6 =malloc (req->_present.local_v6_len);memcpy (req->local_v6, local_v6, req->_present.local_v6_len); }
@@ -99,6 +100,7 @@ fou_add_req_set_peer_v6(struct fou_add_req *req, const void *peer_v6, size_t len) {free (req->peer_v6); + req->_present.peer_v6_len = len; req->peer_v6 =malloc (req->_present.peer_v6_len);memcpy (req->peer_v6, peer_v6, req->_present.peer_v6_len); }
@@ -192,6 +194,7 @@ fou_del_req_set_local_v6(struct fou_del_req *req, const void *local_v6, size_t len) {free (req->local_v6); + req->_present.local_v6_len = len; req->local_v6 =malloc (req->_present.local_v6_len);memcpy (req->local_v6, local_v6, req->_present.local_v6_len); }
@@ -200,6 +203,7 @@ fou_del_req_set_peer_v6(struct fou_del_req *req, const void *peer_v6, size_t len) {free (req->peer_v6); + req->_present.peer_v6_len = len; req->peer_v6 =malloc (req->_present.peer_v6_len);memcpy (req->peer_v6, peer_v6, req->_present.peer_v6_len); }
@@ -280,6 +284,7 @@ fou_get_req_set_local_v6(struct fou_get_req *req, const void *local_v6, size_t len) {free (req->local_v6); + req->_present.local_v6_len = len; req->local_v6 =malloc (req->_present.local_v6_len);memcpy (req->local_v6, local_v6, req->_present.local_v6_len); }
@@ -288,6 +293,7 @@ fou_get_req_set_peer_v6(struct fou_get_req *req, const void *peer_v6, size_t len) {free (req->peer_v6); + req->_present.peer_v6_len = len; req->peer_v6 =malloc (req->_present.peer_v6_len);memcpy (req->peer_v6, peer_v6, req->_present.peer_v6_len); }
tools/net/ynl/ynl-gen-c.py +1 -0
@@ -428,6 +428,7 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S def _setter_lines(self, ri, member, presence): return [f"free({member});", + f"{presence}_len = len;", f"{member} = malloc({presence}_len);", f'memcpy({member}, {self.c_name}, {presence}_len);']
Signature · 1 line
2.41.0
Re: [PATCH net-next 2/5] tools: ynl-gen: set length of binary fields
Quoted 102 lines · Jakub Kicinski
Remember to set the length field in the request setters.
Signed-off-by Jakub Kicinski <[email protected]>--- tools/net/ynl/generated/ethtool-user.h | 4 ++++ tools/net/ynl/generated/fou-user.h | 6 ++++++ tools/net/ynl/ynl-gen-c.py | 1 + 3 files changed, 11 insertions(+)tools/net/ynl/generated/ethtool-user.h +4
-0@@ -1422,6 +1422,7 @@ ethtool_wol_set_req_set_sopass(struct ethtool_wol_set_req *req, const void *sopass, size_t len) { free(req->sopass); + req->_present.sopass_len = len; req->sopass =malloc (req->_present.sopass_len);memcpy (req->sopass, sopass, req->_present.sopass_len); }@@ -4071,6 +4072,7 @@ ethtool_fec_set_req_set_stats_corrected(struct ethtool_fec_set_req *req, const void *corrected, size_t len) { free(req->stats.corrected); + req->stats._present.corrected_len = len; req->stats.corrected =malloc (req->stats._present.corrected_len);memcpy (req->stats.corrected, corrected, req->stats._present.corrected_len); }@@ -4079,6 +4081,7 @@ ethtool_fec_set_req_set_stats_uncorr(struct ethtool_fec_set_req *req, const void *uncorr, size_t len) { free(req->stats.uncorr); + req->stats._present.uncorr_len = len; req->stats.uncorr =malloc (req->stats._present.uncorr_len);memcpy (req->stats.uncorr, uncorr, req->stats._present.uncorr_len); }@@ -4087,6 +4090,7 @@ ethtool_fec_set_req_set_stats_corr_bits(struct ethtool_fec_set_req *req, const void *corr_bits, size_t len) { free(req->stats.corr_bits); + req->stats._present.corr_bits_len = len; req->stats.corr_bits =malloc (req->stats._present.corr_bits_len);memcpy (req->stats.corr_bits, corr_bits, req->stats._present.corr_bits_len); }tools/net/ynl/generated/fou-user.h +6
-0@@ -91,6 +91,7 @@ fou_add_req_set_local_v6(struct fou_add_req *req, const void *local_v6, size_t len) {free (req->local_v6); + req->_present.local_v6_len = len; req->local_v6 =malloc (req->_present.local_v6_len);memcpy (req->local_v6, local_v6, req->_present.local_v6_len); }@@ -99,6 +100,7 @@ fou_add_req_set_peer_v6(struct fou_add_req *req, const void *peer_v6, size_t len) {free (req->peer_v6); + req->_present.peer_v6_len = len; req->peer_v6 =malloc (req->_present.peer_v6_len);memcpy (req->peer_v6, peer_v6, req->_present.peer_v6_len); }@@ -192,6 +194,7 @@ fou_del_req_set_local_v6(struct fou_del_req *req, const void *local_v6, size_t len) {free (req->local_v6); + req->_present.local_v6_len = len; req->local_v6 =malloc (req->_present.local_v6_len);memcpy (req->local_v6, local_v6, req->_present.local_v6_len); }@@ -200,6 +203,7 @@ fou_del_req_set_peer_v6(struct fou_del_req *req, const void *peer_v6, size_t len) {free (req->peer_v6); + req->_present.peer_v6_len = len; req->peer_v6 =malloc (req->_present.peer_v6_len);memcpy (req->peer_v6, peer_v6, req->_present.peer_v6_len); }@@ -280,6 +284,7 @@ fou_get_req_set_local_v6(struct fou_get_req *req, const void *local_v6, size_t len) {free (req->local_v6); + req->_present.local_v6_len = len; req->local_v6 =malloc (req->_present.local_v6_len);memcpy (req->local_v6, local_v6, req->_present.local_v6_len); }@@ -288,6 +293,7 @@ fou_get_req_set_peer_v6(struct fou_get_req *req, const void *peer_v6, size_t len) {free (req->peer_v6); + req->_present.peer_v6_len = len; req->peer_v6 =malloc (req->_present.peer_v6_len);memcpy (req->peer_v6, peer_v6, req->_present.peer_v6_len); }tools/net/ynl/ynl-gen-c.py +0
-0
tools/net/ynl/ynl-gen-c.py from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S · line 433
Quoted 6 lines
tools/net/ynl/ynl-gen-c.py +1
-0@@ -428,6 +428,7 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S def _setter_lines(self, ri, member, presence): return [f"free({member});", + f"{presence}_len = len;", f"{member} = malloc({presence}_len);", f'memcpy({member}, {self.c_name}, {presence}_len);']
Jakub Kicinski
maintainerPATCH 3/5 tools: ynl-gen: fix collecting global policy attrs
In reply to Jakub KicinskiWe look for attributes inside do.request, but there's another layer of nesting in the spec, look inside do.request.attributes.
This bug had no effect as all global policies we generate (fou) seem to be full, anyway, and we treat full and empty the same.
Next patch will change the treatment of empty policies.
--- tools/net/ynl/ynl-gen-c.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
tools/net/ynl/ynl-gen-c.py +3 -1
@@ -978,7 +978,9 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S for op_mode in ['do', 'dump']: if op_mode in op: - global_set.update(op[op_mode].get('request', [])) + req = op[op_mode].get('request') + if req: + global_set.update(req.get('attributes', [])) self.global_policy = [] self.global_policy_set = attr_set_name
Signature · 1 line
2.41.0
Re: [PATCH net-next 3/5] tools: ynl-gen: fix collecting global policy attrs
Quoted 18 lines · Jakub Kicinski
We look for attributes inside do.request, but there's another layer of nesting in the spec, look inside do.request.attributes.
This bug had no effect as all global policies we generate (fou) seem to be full, anyway, and we treat full and empty the same.
Next patch will change the treatment of empty policies.
Signed-off-by Jakub Kicinski <[email protected]>--- tools/net/ynl/ynl-gen-c.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)tools/net/ynl/ynl-gen-c.py +0
-0
tools/net/ynl/ynl-gen-c.py from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S · line 986
Quoted 10 lines
tools/net/ynl/ynl-gen-c.py +3
-1@@ -978,7 +978,9 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S for op_mode in ['do', 'dump']: if op_mode in op: - global_set.update(op[op_mode].get('request', [])) + req = op[op_mode].get('request') + if req: + global_set.update(req.get('attributes', [])) self.global_policy = [] self.global_policy_set = attr_set_name
Jakub Kicinski
maintainerPATCH 4/5 tools: ynl-gen: support empty attribute lists
In reply to Jakub KicinskiDifferentiate between empty list and None for member lists. New families may want to create request responses with no attribute. If we treat those the same as None we end up rendering
a full parsing policy in user space, instead of an empty one.
--- tools/net/ynl/ynl-gen-c.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
tools/net/ynl/ynl-gen-c.py +9 -2
@@ -615,7 +615,7 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S self.attr_list = [] self.attrs = dict() - if type_list: + if type_list is not None: for t in type_list: self.attr_list.append((t, self.attr_set[t]),) else:
@@ -1543,7 +1543,14 @@ _C_KW = { ri.cw.write_func_prot('int', f'{op_prefix(ri, "reply", deref=deref)}_parse', func_args) - _multi_parse(ri, ri.struct["reply"], init_lines, local_vars) + if ri.struct["reply"].member_list(): + _multi_parse(ri, ri.struct["reply"], init_lines, local_vars) + else: + # Empty reply + ri.cw.block_start() + ri.cw.p('return MNL_CB_OK;') + ri.cw.block_end() + ri.cw.nl() def print_req(ri):
Signature · 1 line
2.41.0
Re: [PATCH net-next 4/5] tools: ynl-gen: support empty attribute lists
Quoted 29 lines · Jakub Kicinski
Differentiate between empty list and None for member lists. New families may want to create request responses with no attribute. If we treat those the same as None we end up rendering
a full parsing policy in user space, instead of an empty one.Signed-off-by Jakub Kicinski <[email protected]>--- tools/net/ynl/ynl-gen-c.py | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)tools/net/ynl/ynl-gen-c.py +2
-2@@ -615,7 +615,7 @@ from lib import SpecFamily, SpecAttrSet, SpecAttr, SpecOperation, SpecEnumSet, S self.attr_list = [] self.attrs = dict() - if type_list: + if type_list is not None: for t in type_list: self.attr_list.append((t, self.attr_set[t]),) else:@@ -1543,7 +1543,14 @@ _C_KW = { ri.cw.write_func_prot('int', f'{op_prefix(ri, "reply", deref=deref)}_parse', func_args) - _multi_parse(ri, ri.struct["reply"], init_lines, local_vars) + if ri.struct["reply"].member_list():
tools/net/ynl/ynl-gen-c.py _C_KW = { · line 1556
Quoted 10 lines
tools/net/ynl/ynl-gen-c.py +7
-0+ _multi_parse(ri, ri.struct["reply"], init_lines, local_vars) + else: + # Empty reply + ri.cw.block_start() + ri.cw.p('return MNL_CB_OK;') + ri.cw.block_end() + ri.cw.nl() def print_req(ri):
Jakub Kicinski
maintainerPATCH 5/5 netlink: specs: fix indent in fou
In reply to Jakub KicinskiFix up the indentation. This has no functional effect, AFAICT.
--- Documentation/netlink/specs/fou.yaml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
Documentation/netlink/specs/fou.yaml +9 -9
@@ -107,16 +107,16 @@ kernel-policy: global flags: [ admin-perm ] do: - request: &select_attrs + request: &select_attrs attributes: - - af - - ifindex - - port - - peer_port - - local_v4 - - peer_v4 - - local_v6 - - peer_v6 + - af + - ifindex + - port + - peer_port + - local_v4 + - peer_v4 + - local_v6 + - peer_v6 - name: get
Signature · 1 line
2.41.0
Re: [PATCH net-next 5/5] netlink: specs: fix indent in fou
Quoted 27 lines · Jakub Kicinski
Fix up the indentation. This has no functional effect, AFAICT.
Signed-off-by Jakub Kicinski <[email protected]>--- Documentation/netlink/specs/fou.yaml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)Documentation/netlink/specs/fou.yaml +2
-9@@ -107,16 +107,16 @@ kernel-policy: global flags: [ admin-perm ] do: - request: &select_attrs + request: &select_attrs attributes: - - af - - ifindex - - port - - peer_port - - local_v4 - - peer_v4 - - local_v6 - - peer_v6 + - af
Documentation/netlink/specs/fou.yaml kernel-policy: global · line 122
Quoted 10 lines
Documentation/netlink/specs/fou.yaml +7
-0+ - ifindex + - port + - peer_port + - local_v4 + - peer_v4 + - local_v6 + - peer_v6 - name: get
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 10 lines · On Wed, 23 Aug 2023 17:30:51 -0700 you wrote:
Small YNL improvements, mostly for work-in-progress families.
Jakub Kicinski (5):
tools: ynl: allow passing binary data
tools: ynl-gen: set length of binary fields
tools: ynl-gen: fix collecting global policy attrs
tools: ynl-gen: support empty attribute lists
netlink: specs: fix indent in fou[...]
Here is the summary with links:
- [net-next,1/5] tools: ynl: allow passing binary data
https://git.kernel.org/netdev/net-next/c/649bde9004ac - [net-next,2/5] tools: ynl-gen: set length of binary fields https://git.kernel.org/netdev/net-next/c/a149a3a13bbc - [net-next,3/5] tools: ynl-gen: fix collecting global policy attrs https://git.kernel.org/netdev/net-next/c/dc2ef94d8926 - [net-next,4/5] tools: ynl-gen: support empty attribute lists https://git.kernel.org/netdev/net-next/c/4c8c24e801e6 - [net-next,5/5] netlink: specs: fix indent in fou https://git.kernel.org/netdev/net-next/c/e83d4e9b2d0f
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html