netlink: Add nftables spec w/ multi messages
Subsystems (4)
Donald Hunter
maintainerv2 cover netlink: Add nftables spec w/ multi messages
This series adds a ynl spec for nftables and extends ynl with a --multi command line option that makes it possible to send transactional batches for nftables.
This series includes a patch for nfnetlink which adds ACK processing for batch begin/end messages. If you'd prefer that to be sent separately to nf-next then I can do so, but I included it here so that it gets seen in context.
An example of usage is:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi batch-begin '{"res-id": 10}' \
--multi newtable '{"name": "test", "nfgen-family": 1}' \
--multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--multi batch-end '{"res-id": 10}'
It can also be used for bundling get requests:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi gettable '{"name": "test", "nfgen-family": 1}' \
--multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--output-json
[{"name": "test", "use": 1, "handle": 1, "flags": [],
"nfgen-family": 1, "version": 0, "res-id": 2},
{"table": "test", "name": "chain", "handle": 1, "use": 0, "nfgen-family": 1, "version": 0, "res-id": 2}]
There are 2 issues that may be worth resolving:
- ynl reports errors by raising an NlError exception so only the first error gets reported. This could be changed to add errors to the list of responses so that multiple errors can be reported.
- If any message does not get a response (e.g. batch-begin w/o patch 2) then ynl waits indefinitely. A recv timeout could be added which would allow ynl to terminate.
v1 -> v2:
- add a patch to nfnetlink to process ACKs for batch begin/end
- handle multiple responses correctly
Donald Hunter (3):
doc/netlink/specs: Add draft nftables spec
netfilter: nfnetlink: Handle ACK flags for batch messages tools/net/ynl: Add multi message support to ynl
Documentation/netlink/specs/nftables.yaml | 1264 +++++++++++++++++++++ net/netfilter/nfnetlink.c | 5 + tools/net/ynl/cli.py | 25 +- tools/net/ynl/lib/ynl.py | 64 +- 4 files changed, 1335 insertions(+), 23 deletions(-) create mode 100644 Documentation/netlink/specs/nftables.yaml
Signature · 1 line
2.43.0
Donald Hunter
maintainerPATCH v2 1/3 doc/netlink/specs: Add draft nftables spec
Add a spec for nftables that has nearly complete coverage of the ops, but limited coverage of rule types and subexpressions.
--- Documentation/netlink/specs/nftables.yaml | 1264 +++++++++++++++++++++ 1 file changed, 1264 insertions(+) create mode 100644 Documentation/netlink/specs/nftables.yaml
Documentation/netlink/specs/nftables.yaml new file +1264 -0
@@ -0,0 +1,1264 @@ +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) + +name: nftables +protocol: netlink-raw +protonum: 12 + +doc: + Netfilter nftables configuration over netlink. + +definitions: + - + name: nfgenmsg + type: struct + members: + - + name: nfgen-family + type: u8 + - + name: version + type: u8 + - + name: res-id + byte-order: big-endian + type: u16 + - + name: meta-keys + type: enum + entries: + - len + - protocol + - priority + - mark + - iif + - oif + - iifname + - oifname + - iftype + - oiftype + - skuid + - skgid + - nftrace + - rtclassid + - secmark + - nfproto + - l4-proto + - bri-iifname + - bri-oifname + - pkttype + - cpu + - iifgroup + - oifgroup + - cgroup + - prandom + - secpath + - iifkind + - oifkind + - bri-iifpvid + - bri-iifvproto + - time-ns + - time-day + - time-hour + - sdif + - sdifname + - bri-broute + - + name: cmp-ops + type: enum + entries: + - eq + - neq + - lt + - lte + - gt + - gte + - + name: object-type + type: enum + entries: + - unspec + - counter + - quota + - ct-helper + - limit + - connlimit + - tunnel + - ct-timeout + - secmark + - ct-expect + - synproxy + - + name: nat-range-flags + type: flags + entries: + - map-ips + - proto-specified + - proto-random + - persistent + - proto-random-fully + - proto-offset + - netmap + - + name: table-flags + type: flags + entries: + - dormant + - owner + - persist + - + name: chain-flags + type: flags + entries: + - base + - hw-offload + - binding + - + name: set-flags + type: flags + entries: + - anonymous + - constant + - interval + - map + - timeout + - eval + - object + - concat + - expr + +attribute-sets: + - + name: empty-attrs + attributes: + - + name: name + type: string + - + name: batch-attrs + attributes: + - + name: genid + type: u32 + byte-order: big-endian + - + name: table-attrs + attributes: + - + name: name + type: string + doc: name of the table + - + name: flags + type: u32 + byte-order: big-endian + doc: bitmask of flags + enum: table-flags + enum-as-flags: true + - + name: use + type: u32 + byte-order: big-endian + doc: number of chains in this table + - + name: handle + type: u64 + byte-order: big-endian + doc: numeric handle of the table + - + name: userdata + type: binary + doc: user data + - + name: chain-attrs + attributes: + - + name: table + type: string + doc: name of the table containing the chain + - + name: handle + type: u64 + byte-order: big-endian + doc: numeric handle of the chain + - + name: name + type: string + doc: name of the chain + - + name: hook + type: nest + nested-attributes: nft-hook-attrs + doc: hook specification for basechains + - + name: policy + type: u32 + byte-order: big-endian + doc: numeric policy of the chain + - + name: use + type: u32 + byte-order: big-endian + doc: number of references to this chain + - + name: type + type: string + doc: type name of the chain + - + name: counters + type: nest + nested-attributes: nft-counter-attrs + doc: counter specification of the chain + - + name: flags + type: u32 + byte-order: big-endian + doc: chain flags + enum: chain-flags + enum-as-flags: true + - + name: id + type: u32 + byte-order: big-endian + doc: uniquely identifies a chain in a transaction + - + name: userdata + type: binary + doc: user data + - + name: counter-attrs + attributes: + - + name: bytes + type: u64 + byte-order: big-endian + - + name: packets + type: u64 + byte-order: big-endian + - + name: pad + type: pad + - + name: nft-hook-attrs + attributes: + - + name: num + type: u32 + byte-order: big-endian + - + name: priority + type: s32 + byte-order: big-endian + - + name: dev + type: string + doc: net device name + - + name: devs + type: nest + nested-attributes: hook-dev-attrs + doc: list of net devices + - + name: hook-dev-attrs + attributes: + - + name: name + type: string + multi-attr: true + - + name: nft-counter-attrs + attributes: + - + name: bytes + type: u64 + - + name: packets + type: u64 + - + name: rule-attrs + attributes: + - + name: table + type: string + doc: name of the table containing the rule + - + name: chain + type: string + doc: name of the chain containing the rule + - + name: handle + type: u64 + byte-order: big-endian + doc: numeric handle of the rule + - + name: expressions + type: nest + nested-attributes: expr-list-attrs + doc: list of expressions + - + name: compat + type: nest + nested-attributes: rule-compat-attrs + doc: compatibility specifications of the rule + - + name: position + type: u64 + byte-order: big-endian + doc: numeric handle of the previous rule + - + name: userdata + type: binary + doc: user data + - + name: id + type: u32 + doc: uniquely identifies a rule in a transaction + - + name: position-id + type: u32 + doc: transaction unique identifier of the previous rule + - + name: chain-id + type: u32 + doc: add the rule to chain by ID, alternative to chain name + - + name: expr-list-attrs + attributes: + - + name: elem + type: nest + nested-attributes: expr-attrs + multi-attr: true + - + name: expr-attrs + attributes: + - + name: name + type: string + doc: name of the expression type + - + name: data + type: sub-message + sub-message: expr-ops + selector: name + doc: type specific data + - + name: rule-compat-attrs + attributes: + - + name: proto + type: binary + doc: numeric value of the handled protocol + - + name: flags + type: binary + doc: bitmask of flags + - + name: set-attrs + attributes: + - + name: table + type: string + doc: table name + - + name: name + type: string + doc: set name + - + name: flags + type: u32 + enum: set-flags + byte-order: big-endian + doc: bitmask of enum nft_set_flags + - + name: key-type + type: u32 + byte-order: big-endian + doc: key data type, informational purpose only + - + name: key-len + type: u32 + byte-order: big-endian + doc: key data length + - + name: data-type + type: u32 + byte-order: big-endian + doc: mapping data type + - + name: data-len + type: u32 + byte-order: big-endian + doc: mapping data length + - + name: policy + type: u32 + byte-order: big-endian + doc: selection policy + - + name: desc + type: nest + nested-attributes: set-desc-attrs + doc: set description + - + name: id + type: u32 + doc: uniquely identifies a set in a transaction + - + name: timeout + type: u64 + doc: default timeout value + - + name: gc-interval + type: u32 + doc: garbage collection interval + - + name: userdata + type: binary + doc: user data + - + name: pad + type: pad + - + name: obj-type + type: u32 + byte-order: big-endian + doc: stateful object type + - + name: handle + type: u64 + byte-order: big-endian + doc: set handle + - + name: expr + type: nest + nested-attributes: expr-attrs + doc: set expression + multi-attr: true + - + name: expressions + type: nest + nested-attributes: set-list-attrs + doc: list of expressions + - + name: set-desc-attrs + attributes: + - + name: size + type: u32 + byte-order: big-endian + doc: number of elements in set + - + name: concat + type: nest + nested-attributes: set-desc-concat-attrs + doc: description of field concatenation + multi-attr: true + - + name: set-desc-concat-attrs + attributes: + - + name: elem + type: nest + nested-attributes: set-field-attrs + - + name: set-field-attrs + attributes: + - + name: len + type: u32 + byte-order: big-endian + - + name: set-list-attrs + attributes: + - + name: elem + type: nest + nested-attributes: expr-attrs + multi-attr: true + - + name: setelem-attrs + attributes: + - + name: key + type: nest + nested-attributes: data-attrs + doc: key value + - + name: data + type: nest + nested-attributes: data-attrs + doc: data value of mapping + - + name: flags + type: binary + doc: bitmask of nft_set_elem_flags + - + name: timeout + type: u64 + doc: timeout value + - + name: expiration + type: u64 + doc: expiration time + - + name: userdata + type: binary + doc: user data + - + name: expr + type: nest + nested-attributes: expr-attrs + doc: expression + - + name: objref + type: string + doc: stateful object reference + - + name: key-end + type: nest + nested-attributes: data-attrs + doc: closing key value + - + name: expressions + type: nest + nested-attributes: expr-list-attrs + doc: list of expressions + - + name: setelem-list-elem-attrs + attributes: + - + name: elem + type: nest + nested-attributes: setelem-attrs + multi-attr: true + - + name: setelem-list-attrs + attributes: + - + name: table + type: string + - + name: set + type: string + - + name: elements + type: nest + nested-attributes: setelem-list-elem-attrs + - + name: set-id + type: u32 + - + name: gen-attrs + attributes: + - + name: id + type: u32 + byte-order: big-endian + doc: ruleset generation id + - + name: proc-pid + type: u32 + byte-order: big-endian + - + name: proc-name + type: string + - + name: obj-attrs + attributes: + - + name: table + type: string + doc: name of the table containing the expression + - + name: name + type: string + doc: name of this expression type + - + name: type + type: u32 + enum: object-type + byte-order: big-endian + doc: stateful object type + - + name: data + type: sub-message + sub-message: obj-data + selector: type + doc: stateful object data + - + name: use + type: u32 + byte-order: big-endian + doc: number of references to this expression + - + name: handle + type: u64 + byte-order: big-endian + doc: object handle + - + name: pad + type: pad + - + name: userdata + type: binary + doc: user data + - + name: quota-attrs + attributes: + - + name: bytes + type: u64 + byte-order: big-endian + - + name: flags # TODO + type: u32 + byte-order: big-endian + - + name: pad + type: pad + - + name: consumed + type: u64 + byte-order: big-endian + - + name: flowtable-attrs + attributes: + - + name: table + type: string + - + name: name + type: string + - + name: hook + type: nest + nested-attributes: flowtable-hook-attrs + - + name: use + type: u32 + byte-order: big-endian + - + name: handle + type: u64 + byte-order: big-endian + - + name: pad + type: pad + - + name: flags + type: u32 + byte-order: big-endian + - + name: flowtable-hook-attrs + attributes: + - + name: num + type: u32 + byte-order: big-endian + - + name: priority + type: u32 + byte-order: big-endian + - + name: devs + type: nest + nested-attributes: hook-dev-attrs + - + name: expr-cmp-attrs + attributes: + - + name: sreg + type: u32 + byte-order: big-endian + - + name: op + type: u32 + byte-order: big-endian + enum: cmp-ops + - + name: data + type: nest + nested-attributes: data-attrs + - + name: data-attrs + attributes: + - + name: value + type: binary + # sub-type: u8 + - + name: verdict + type: nest + nested-attributes: verdict-attrs + - + name: verdict-attrs + attributes: + - + name: code + type: u32 + byte-order: big-endian + - + name: chain + type: string + - + name: chain-id + type: u32 + - + name: expr-counter-attrs + attributes: + - + name: bytes + type: u64 + doc: Number of bytes + - + name: packets + type: u64 + doc: Number of packets + - + name: pad + type: pad + - + name: expr-flow-offload-attrs + attributes: + - + name: name + type: string + doc: Flow offload table name + - + name: expr-immediate-attrs + attributes: + - + name: dreg + type: u32 + byte-order: big-endian + - + name: data + type: nest + nested-attributes: data-attrs + - + name: expr-meta-attrs + attributes: + - + name: dreg + type: u32 + byte-order: big-endian + - + name: key + type: u32 + byte-order: big-endian + enum: meta-keys + - + name: sreg + type: u32 + byte-order: big-endian + - + name: expr-nat-attrs + attributes: + - + name: type + type: u32 + byte-order: big-endian + - + name: family + type: u32 + byte-order: big-endian + - + name: reg-addr-min + type: u32 + byte-order: big-endian + - + name: reg-addr-max + type: u32 + byte-order: big-endian + - + name: reg-proto-min + type: u32 + byte-order: big-endian + - + name: reg-proto-max + type: u32 + byte-order: big-endian + - + name: flags + type: u32 + byte-order: big-endian + enum: nat-range-flags + enum-as-flags: true + - + name: expr-payload-attrs + attributes: + - + name: dreg + type: u32 + byte-order: big-endian + - + name: base + type: u32 + byte-order: big-endian + - + name: offset + type: u32 + byte-order: big-endian + - + name: len + type: u32 + byte-order: big-endian + - + name: sreg + type: u32 + byte-order: big-endian + - + name: csum-type + type: u32 + byte-order: big-endian + - + name: csum-offset + type: u32 + byte-order: big-endian + - + name: csum-flags + type: u32 + byte-order: big-endian + - + name: expr-tproxy-attrs + attributes: + - + name: family + type: u32 + byte-order: big-endian + - + name: reg-addr + type: u32 + byte-order: big-endian + - + name: reg-port + type: u32 + byte-order: big-endian + +sub-messages: + - + name: expr-ops + formats: + - + value: bitwise # TODO + - + value: cmp + attribute-set: expr-cmp-attrs + - + value: counter + attribute-set: expr-counter-attrs + - + value: ct # TODO + - + value: flow_offload + attribute-set: expr-flow-offload-attrs + - + value: immediate + attribute-set: expr-immediate-attrs + - + value: lookup # TODO + - + value: meta + attribute-set: expr-meta-attrs + - + value: nat + attribute-set: expr-nat-attrs + - + value: payload + attribute-set: expr-payload-attrs + - + value: tproxy + attribute-set: expr-tproxy-attrs + - + name: obj-data + formats: + - + value: counter + attribute-set: counter-attrs + - + value: quota + attribute-set: quota-attrs + +operations: + enum-model: directional + list: + - + name: batch-begin + doc: Start a batch of operations + attribute-set: batch-attrs + fixed-header: nfgenmsg + do: + request: + value: 0x10 + attributes: + - genid + reply: + value: 0x10 + attributes: + - genid + - + name: batch-end + doc: Finish a batch of operations + attribute-set: batch-attrs + fixed-header: nfgenmsg + do: + request: + value: 0x11 + attributes: + - genid + - + name: newtable + doc: Create a new table. + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa00 + attributes: + - name + - + name: gettable + doc: Get / dump tables. + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa01 + attributes: + - name + reply: + value: 0xa00 + attributes: + - name + - + name: deltable + doc: Delete an existing table. + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa02 + attributes: + - name + - + name: destroytable + doc: Delete an existing table with destroy semantics (ignoring ENOENT errors). + attribute-set: table-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1a + attributes: + - name + - + name: newchain + doc: Create a new chain. + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa03 + attributes: + - name + - + name: getchain + doc: Get / dump chains. + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa04 + attributes: + - name + reply: + value: 0xa03 + attributes: + - name + - + name: delchain + doc: Delete an existing chain. + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa05 + attributes: + - name + - + name: destroychain + doc: Delete an existing chain with destroy semantics (ignoring ENOENT errors). + attribute-set: chain-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1b + attributes: + - name + - + name: newrule + doc: Create a new rule. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa06 + attributes: + - name + - + name: getrule + doc: Get / dump rules. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa07 + attributes: + - name + reply: + value: 0xa06 + attributes: + - name + - + name: getrule-reset + doc: Get / dump rules and reset stateful expressions. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa19 + attributes: + - name + reply: + value: 0xa06 + attributes: + - name + - + name: delrule + doc: Delete an existing rule. + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa08 + attributes: + - name + - + name: destroyrule + doc: Delete an existing rule with destroy semantics (ignoring ENOENT errors). + attribute-set: rule-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1c + attributes: + - name + - + name: newset + doc: Create a new set. + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa09 + attributes: + - name + - + name: getset + doc: Get / dump sets. + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0a + attributes: + - name + reply: + value: 0xa09 + attributes: + - name + - + name: delset + doc: Delete an existing set. + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0b + attributes: + - name + - + name: destroyset + doc: Delete an existing set with destroy semantics (ignoring ENOENT errors). + attribute-set: set-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1d + attributes: + - name + - + name: newsetelem + doc: Create a new set element. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0c + attributes: + - name + - + name: getsetelem + doc: Get / dump set elements. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0d + attributes: + - name + reply: + value: 0xa0c + attributes: + - name + - + name: getsetelem-reset + doc: Get / dump set elements and reset stateful expressions. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa21 + attributes: + - name + reply: + value: 0xa0c + attributes: + - name + - + name: delsetelem + doc: Delete an existing set element. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa0e + attributes: + - name + - + name: destroysetelem + doc: Delete an existing set element with destroy semantics. + attribute-set: setelem-list-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1e + attributes: + - name + - + name: getgen + doc: Get / dump rule-set generation. + attribute-set: gen-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa10 + attributes: + - name + reply: + value: 0xa0f + attributes: + - name + - + name: newobj + doc: Create a new stateful object. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa12 + attributes: + - name + - + name: getobj + doc: Get / dump stateful objects. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa13 + attributes: + - name + reply: + value: 0xa12 + attributes: + - name + - + name: delobj + doc: Delete an existing stateful object. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa14 + attributes: + - name + - + name: destroyobj + doc: Delete an existing stateful object with destroy semantics. + attribute-set: obj-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa1f + attributes: + - name + - + name: newflowtable + doc: Create a new flow table. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa16 + attributes: + - name + - + name: getflowtable + doc: Get / dump flow tables. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa17 + attributes: + - name + reply: + value: 0xa16 + attributes: + - name + - + name: delflowtable + doc: Delete an existing flow table. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa18 + attributes: + - name + - + name: destroyflowtable + doc: Delete an existing flow table with destroy semantics. + attribute-set: flowtable-attrs + fixed-header: nfgenmsg + do: + request: + value: 0xa20 + attributes: + - name + +mcast-groups: + list: + - + name: mgmt
Signature · 1 line
2.43.0
Donald Hunter
maintainerPATCH v2 2/3 netfilter: nfnetlink: Handle ACK flags for batch messages
In reply to Donald HunterThe NLM_F_ACK flag is not processed for nfnetlink batch messages. This is a problem for ynl which wants to receive an ack for every message it sends. Add processing for ACK and provide responses when requested.
I have checked that iproute2, pyroute2 and systemd are unaffected by this change since none of them use NLM_F_ACK for batch begin/end. I also ran a search on github and did not spot any usage that would break.
--- net/netfilter/nfnetlink.c | 5 +++++ 1 file changed, 5 insertions(+)
net/netfilter/nfnetlink.c +5 -0
@@ -427,6 +427,9 @@ static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh,nfnl_unlock (subsys_id); +if (nlh->nlmsg_flags &NLM_F_ACK ) +nfnl_err_add (&err_list, nlh,0 , &extack); +while (skb->len >=nlmsg_total_size (0 )) {int msglen, type;
@@ -463,6 +466,8 @@ static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh, goto done; }else if (type ==NFNL_MSG_BATCH_END ) { status |=NFNL_BATCH_DONE ; +if (nlh->nlmsg_flags &NLM_F_ACK ) +nfnl_err_add (&err_list, nlh,0 , &extack); goto done; }else if (type <NLMSG_MIN_TYPE ) { err = -EINVAL ;
Signature · 1 line
2.43.0
Pablo Neira Ayuso
maintainerRe: [PATCH net-next v2 2/3] netfilter: nfnetlink: Handle ACK flags for batch messages
Quoted 1 line · Donald Hunter
The
NLM_F_ACKflag is not processed for nfnetlink batch messages.
Let me clarify: It is not processed for the begin and end marker netlink message, but it is processed for command messages.
Quoted 3 lines
This is a problem for ynl which wants to receive an ack for every message it sends. Add processing for ACK and provide responses when requested.
NLM_F_ACK is regarded for the specific command messages that are contained in the batch, that is:
batch begin
command
command
...
command
batch end
Thus, NLM_F_ACK can be set on for the command messages and it is not ignore in that case.
May I ask why do you need this? Is it to make your userspace tool happy?
Quoted 25 lines
I have checked that iproute2, pyroute2 and systemd are unaffected by this change since none of them use
NLM_F_ACKfor batch begin/end. I also ran a search on github and did not spot any usage that would break.Signed-off-by Donald Hunter <[email protected]>--- net/netfilter/nfnetlink.c | 5 +++++ 1 file changed, 5 insertions(+)net/netfilter/nfnetlink.c +3
-0@@ -427,6 +427,9 @@ static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh,nfnl_unlock (subsys_id); +if (nlh->nlmsg_flags &NLM_F_ACK ) +nfnl_err_add (&err_list, nlh,0 , &extack); +while (skb->len >=nlmsg_total_size (0 )) {int msglen, type;
net/netfilter/nfnetlink.c static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh, · line 470
Quoted 5 lines
net/netfilter/nfnetlink.c +2
-0@@ -463,6 +466,8 @@ static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh, goto done; }else if (type ==NFNL_MSG_BATCH_END ) { status |=NFNL_BATCH_DONE ; +if (nlh->nlmsg_flags &NLM_F_ACK ) +nfnl_err_add (&err_list, nlh,0 , &extack);
if (status == NFNL_BATCH_DONE) should probably be a better place for this. I would like to have userspace that uses this, I don't have a usecase at this moment for this new code.
Thanks.
Quoted 6 lines
net/netfilter/nfnetlink.c +0
-0goto done; }else if (type <NLMSG_MIN_TYPE ) { err = -EINVAL ;Signature · 1 line
2.43.0
Donald Hunter
maintainerRe: [PATCH net-next v2 2/3] netfilter: nfnetlink: Handle ACK flags for batch messages
Quoted 5 lines · Pablo Neira Ayuso
Quoted 1 line · Donald Hunter
The
NLM_F_ACKflag is not processed for nfnetlink batch messages.Let me clarify: It is not processed for the begin and end marker netlink message, but it is processed for command messages.
That's a good point - my apologies for not making it clear in my description that it is only the batch begin and end messages where NLM_F_ACK is ignored. All the command messages between the begin and end messages do indeed give ack responses when requested. I will reword the commit message to make this clear.
Quoted 14 lines
Quoted 3 lines
This is a problem for ynl which wants to receive an ack for every message it sends. Add processing for ACK and provide responses when requested.
NLM_F_ACKis regarded for the specific command messages that are contained in the batch, that is:batch begin
command
command
...
command
batch end
Quoted 4 lines
Thus,
NLM_F_ACKcan be set on for the command messages and it is not ignore in that case.May I ask why do you need this? Is it to make your userspace tool happy?
Yes, as I mentioned this is a problem for ynl and it would also be a problem for any user space tool that is ynl spec driven, i.e. not hard-coded with special cases for a given netlink family.
Previous conversation:
https://lore.kernel.org/netdev/[email protected]/
Quoted 24 lines
Quoted 24 lines
I have checked that iproute2, pyroute2 and systemd are unaffected by this change since none of them use
NLM_F_ACKfor batch begin/end. I also ran a search on github and did not spot any usage that would break.Signed-off-by Donald Hunter <[email protected]>--- net/netfilter/nfnetlink.c | 5 +++++ 1 file changed, 5 insertions(+)net/netfilter/nfnetlink.c +3
-0@@ -427,6 +427,9 @@ static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh,nfnl_unlock (subsys_id); +if (nlh->nlmsg_flags &NLM_F_ACK ) +nfnl_err_add (&err_list, nlh,0 , &extack); +while (skb->len >=nlmsg_total_size (0 )) {int msglen, type;
Quoted 10 lines
net/netfilter/nfnetlink.c static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh, · line 470
Quoted 6 lines
net/netfilter/nfnetlink.c +2
-0@@ -463,6 +466,8 @@ static void nfnetlink_rcv_batch(struct sk_buff *skb, struct nlmsghdr *nlh, goto done; }else if (type ==NFNL_MSG_BATCH_END ) { status |=NFNL_BATCH_DONE ; +if (nlh->nlmsg_flags &NLM_F_ACK ) +nfnl_err_add (&err_list, nlh,0 , &extack);if (status ==
NFNL_BATCH_DONE) should probably be a better place for this. I would like to have userspace that uses this, I don't have a usecase at this moment for this new code.
I looked at putting it there but when the code reaches the 'done' processing, it is not obvious that nlh still refers to the correct message header in the skb. It seemed more natural to process all acks with nfnl_err_add() at the point where each message gets processed. I can take another look at moving it there if you prefer.
The userspace that uses this is the ynl tool with the nftables spec and --multi patch that is included in this patchset. I included an example of how to use it in the cover letter:
https://lore.kernel.org/netdev/[email protected]/T/
Here's the example:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi batch-begin '{"res-id": 10}' \
--multi newtable '{"name": "test", "nfgen-family": 1}' \
--multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--multi batch-end '{"res-id": 10}'
Thanks!
Donald Hunter
maintainerPATCH v2 3/3 tools/net/ynl: Add multi message support to ynl
In reply to Donald HunterAdd a "--multi <do-op> <json>" command line to ynl that makes it possible to add several operations to a single netlink request payload. The --multi command line option is repeated for each operation.
This is used by the nftables family for transaction batches. For example:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi batch-begin '{"res-id": 10}' \
--multi newtable '{"name": "test", "nfgen-family": 1}' \
--multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--multi batch-end '{"res-id": 10}'
It can also be used for bundling get requests:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi gettable '{"name": "test", "nfgen-family": 1}' \
--multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--output-json
[{"name": "test", "use": 1, "handle": 1, "flags": [],
"nfgen-family": 1, "version": 0, "res-id": 2},
{"table": "test", "name": "chain", "handle": 1, "use": 0, "nfgen-family": 1, "version": 0, "res-id": 2}]
--- tools/net/ynl/cli.py | 25 ++++++++++++++-- tools/net/ynl/lib/ynl.py | 64 +++++++++++++++++++++++++++------------- 2 files changed, 66 insertions(+), 23 deletions(-)
tools/net/ynl/cli.py +22 -3
@@ -19,13 +19,28 @@ class YnlEncoder(json.JSONEncoder): def main(): - parser = argparse.ArgumentParser(description='YNL CLI sample') + description = """ + YNL CLI utility - a general purpose netlink utility that uses YAML + specs to drive protocol encoding and decoding. + """ + epilog = """ + The --multi option can be repeated to include several do operations + in the same netlink payload. + """ + + parser = argparse.ArgumentParser(description=description, + epilog=epilog) parser.add_argument('--spec', dest='spec', type=str, required=True) parser.add_argument('--schema', dest='schema', type=str) parser.add_argument('--no-schema', action='store_true') parser.add_argument('--json', dest='json_text', type=str) - parser.add_argument('--do', dest='do', type=str) - parser.add_argument('--dump', dest='dump', type=str) + + group = parser.add_mutually_exclusive_group() + group.add_argument('--do', dest='do', metavar='DO-OPERATION', type=str) + group.add_argument('--multi', dest='multi', nargs=2, action='append', + metavar=('DO-OPERATION', 'JSON_TEXT'), type=str) + group.add_argument('--dump', dest='dump', metavar='DUMP-OPERATION', type=str) + parser.add_argument('--sleep', dest='sleep', type=int) parser.add_argument('--subscribe', dest='ntf', type=str) parser.add_argument('--replace', dest='flags', action='append_const',
@@ -73,6 +88,10 @@ def main(): if args.dump: reply = ynl.dump(args.dump, attrs) output(reply) + if args.multi: + ops = [ (item[0], json.loads(item[1]), args.flags or []) for item in args.multi ] + reply = ynl.do_multi(ops) + output(reply) except NlError as e: print(e) exit(1)
tools/net/ynl/lib/ynl.py +44 -20
110 lines not on this page · raw message
Signature · 1 line
2.43.0
Jakub Kicinski
maintainerRe: [PATCH net-next v2 3/3] tools/net/ynl: Add multi message support to ynl
tools/net/ynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1026
Quoted 7 lines · On Wed, 10 Apr 2024 23:11:08 +0100 Donald Hunter wrote:
tools/net/ynl/lib/ynl.py +5
-1- def do(self, method, vals, flags=None): + def _op(self, method, vals, flags): + ops = [(method, vals, flags)] + return self._ops(ops)[0] + + def do(self, method, vals, flags=[]): return self._op(method, vals, flags)
Commenting here instead of on my own series but there are already tests using dump=True in net-next:
tools/testing/selftests/drivers/net/stats.py: stats = netfam.qstats_get({}, dump=True)tools/testing/selftests/net/nl_netdev.py: devs = nf.dev_get({}, dump=True)
"flags=[Netlink.NLM_F_DUMP]" is going to be a lot less convenient to write :( Maybe we can keep support for both?
Donald Hunter
maintainerRe: [PATCH net-next v2 3/3] tools/net/ynl: Add multi message support to ynl
Quoted 9 lines · Jakub Kicinski
tools/net/ynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1026
Quoted 7 lines · On Wed, 10 Apr 2024 23:11:08 +0100 Donald Hunter wrote:
tools/net/ynl/lib/ynl.py +5
-1- def do(self, method, vals, flags=None): + def _op(self, method, vals, flags): + ops = [(method, vals, flags)] + return self._ops(ops)[0] + + def do(self, method, vals, flags=[]): return self._op(method, vals, flags)
Quoted 8 lines
Commenting here instead of on my own series but there are already tests using dump=True in net-next:
tools/testing/selftests/drivers/net/stats.py: stats =netfam.qstats_get({}, dump=True)tools/testing/selftests/net/nl_netdev.py: devs =nf.dev_get({}, dump=True)"flags=[Netlink.NLM_F_DUMP]" is going to be a lot less convenient to write :( Maybe we can keep support for both?
Good catch, I overlooked the generated methods. I'll make sure to add it back in.