Skip to content

netlink: add universal 'bigint' attribute type

Alexander Lobakin lkml netdev 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v2 Latest6 patches 10 repliesIn review

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6bitmap: try to optimize arr32 <-> bitmap on 64-bit LEsinclude/linux/bitmap.h lib/bitmap.c+43 -202 replies
2/6bitmap: add a couple more helpers to work with arrays of u32sinclude/linux/bitmap.h lib/bitmap.c+59 -13 replies
3/6lib/test_bitmap: verify intermediate arr32 when converting <-> bitmaplib/test_bitmap.c+4 -00 replies
4/6lib/test_bitmap: test the newly added arr32 functionslib/test_bitmap.c+43 -00 replies
5/6bitops: make BYTES_TO_BITS() treewide-availableinclude/linux/bitops.h kernel/trace/trace_probe.c tools/include/linux/bitops.h +1 more+2 -42 replies
6/6netlink: add universal 'bigint' attribute typeinclude/net/netlink.h include/uapi/linux/netlink.h lib/nlattr.c +1 more+294 -23 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 net-next 0/6] netlink: add universal 'bigint' attribute type

Add a new type of Netlink attribute -- big integer.

Basically bigints are just arrays of u32s, but can carry anything, with 1 bit precision. Using variable-length arrays of a fixed type gives the following:

* versatility: one type can carry scalars from u8 to u64, bitmaps, binary data etc.;
* scalability: the same Netlink attribute can be changed to a wider (or shorter) data type with no compatibility issues, same for growing bitmaps;
* optimization: 4-byte units don't require wasting slots for empty padding attributes (they always have natural alignment in Netlink messages).

The only downside is that get/put functions sometimes are not just direct assignment inlines due to the internal representation using bitmaps (longs) and the bitmap API. The first patch in the series partially addresses that.

Basic consumer functions/macros are:
* nla_put_bigint() and nla_get_bigint() -- to easily put a bigint to an skb or get it from a received message (only pointer to an unsigned long array and the number of bits in it are needed);
* nla_put_bigint_{u,be,le,net}{8,16,32,64}() -- alternatives to the already existing family to send/receive scalars using the new type (instead of distinct attr types);
* nla_total_size_bigint*() -- to provide estimate size in bytes to Netlink needed to store a bigint/type;
* NLA_POLICY_BIGINT*() -- to declare a Netlink policy for a bigint attribute.

There are also *_bitmap() aliases for the *_bigint() helpers which have no differences and designed to distinguish bigints from bitmaps in the call sites (for readability).

Netlink policy for a bigint can have an optional bitmap mask of bits supported by the code -- for example, to filter out obsolete bits removed some time ago or limit value to n bits (e.g. 53 instead of 64). Without it, Netlink will just make sure no bits past the passed number are set. Both variants can be requested from the userspace and the kernel will put a mask into a new policy attribute (%NL_POLICY_TYPE_ATTR_BIGINT_MASK).
Unlike BITFIELD32 or Ethtool bitsets, bigints don't implement "selectors" as a basic feature, but it's pretty easy to emulate it with just sending both selector and value in one data chunk (as these bigints are dynamically-sized) and masking unused gaps between them with the bitmap mask policy feature.

An example of using this API can be found in my IP tunnel tree[0] (to be submitted after that one hits the repo), the actual average number of locs to start both sending and receiving bitmaps in one subsys is ~10. And it looks like that some of the already existing APIs could be later converted to Netlink bigints or expanded as well.

And here's sample userspace output for failed in-kernel validation -- IP tunnel flags attribute was declared as 17-bit bitmap/bigint, but the modified userspace passed data with the 20th bit set:

$ ip/ip r add 14.0.1.8 encap ip id 30001 dst 10.0.18.210 dev vxlan1
Policy: type: BIGINT, mask: 0x0003ffff
Error: Attribute failed policy validation.

From v1[1]:
- use u32-array representation instead of u64-array to conserve attributes (no need to have "padding" dummy attrs) and some bytes (4-byte step instead of 8-byte) (Jakub);
- try to resolve arr32 <-> bitmap conversions on 64-bit LEs to inline code at compile-time to reduce get/set overhead (me);
- drop Endianness shenanigans: it makes no sense to encode bitmaps/ arrays to a specific Endian (Jakub);
- rename it from 'bitmap' to 'bigint' to increase usecase coverage (Jakub);
- introduce helpers to send scalars (u8-u64) via the API to make it universal / more useful (Jakub);
- change kfree() to bitmap_free() when freeing allocated bitmaps (Andy);
- make BYTES_TO_BITS() treewide-available and use it (Andy);
- make bitmap_validate_arr32() return bool: there were only two return values possible (Andy);
- drop redundant !!len check before memchr_inv() call: the function does that itself (Andy);
- expand the #if 0 presence explanation (Andy);
- add runtime tests for the new arr32 functions: 5 test cases that cover all condition branches (Andy);
- run make includecheck, ensure including <linux/bitmap.h> to <net/netlink.h> doesn't introduce compile time regressions, mention that in the commitmsg (Andy);
- drop more redundant if (len) condition checks before string operations (Andy);
- make bitmap_arr32_compat() and bitmap_{from,to}_arr32() a bit more readable (Andy);
- don't use min_t(typeof()), specify types explicitly (Andy);
- don't initialize the bitmap two times in a row in __netlink_policy_dump_write_attr_bigint() and use more simple bitmap_fill(): nla_put_bigint() will then clear the tail properly itself (Andy).

The series is also available on my open GitHub: [2]

[0] https://github.com/alobakin/linux/commits/ip_tunnel
[1] https://lore.kernel.org/all/[email protected]
[2] https://github.com/alobakin/linux/commits/netlink_bitmap

Alexander Lobakin (6):
bitmap: try to optimize arr32 <-> bitmap on 64-bit LEs
bitmap: add a couple more helpers to work with arrays of u32s lib/test_bitmap: verify intermediate arr32 when converting <-> bitmap lib/test_bitmap: test the newly added arr32 functions
bitops: make BYTES_TO_BITS() treewide-available
netlink: add universal 'bigint' attribute type

 include/linux/bitmap.h         |  71 ++++++++---
 include/linux/bitops.h         |   1 +
 include/net/netlink.h          | 208 ++++++++++++++++++++++++++++++++-
 include/uapi/linux/netlink.h   |   6 +
 kernel/trace/trace_probe.c     |   2 -
 lib/bitmap.c                   |  52 ++++++++-
 lib/nlattr.c                   |  42 ++++++-
 lib/test_bitmap.c              |  47 ++++++++
 net/netlink/policy.c           |  40 +++++++
 tools/include/linux/bitops.h   |   1 +
 tools/perf/util/probe-finder.c |   2 -
 11 files changed, 445 insertions(+), 27 deletions(-)

base-commit: f00909e2e6fe4ac6b2420e3863a0c533fe4f15e0

Signature · 1 line
2.37.3