Skip to content

Teach verifier to determine necessary log buffer size

Andrii Nakryiko bpf 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest8 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8bpf: ignore verifier log reset in BPF_LOG_KERNEL modekernel/bpf/log.c+1 -10 replies
2/8bpf: fix missing -EFAULT return on user log buf error in btf_parse()kernel/bpf/btf.c+4 -00 replies
3/8bpf: avoid incorrect -EFAULT error in BPF_LOG_KERNEL modekernel/bpf/verifier.c+1 -10 replies
4/8bpf: simplify logging related error conditions handlingkernel/bpf/btf.c kernel/bpf/log.c kernel/bpf/verifier.c+6 -60 replies
5/8bpf: keep track of total log content size in both fixed and rolling modesinclude/linux/bpf_verifier.h kernel/bpf/log.c+50 -300 replies
6/8bpf: add log_size_actual output field to return log contents sizeinclude/linux/bpf.h include/linux/btf.h include/uapi/linux/bpf.h +4 more+56 -260 replies
7/8libbpf: wire through log_size_actual value returned from kerneltools/lib/bpf/bpf.c tools/lib/bpf/bpf.h+14 -40 replies
8/8selftests/bpf: add tests to validate log_size_actual featuretools/testing/selftests/bpf/prog_tests/verifier_log.c+76 -160 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH bpf-next 0/8] Teach verifier to determine necessary log buffer size

My imagination is failing me on how to succinctly name this feature and patch set, but the point here is to perform internal accounting of what should be the necessary size of user-supplied log buffer such as to fit entire log contents without truncation, thus avoiding -ENOSPC.

This is a long-standing limitation, which causes users and BPF loader library writers to guess and pre-size log buffer, often allocating unnecessary extra memory for this or doing extra program verifications just to size logs better, ultimately wasting resources. This was requested most recently by Go BPF library maintainers ([0]).

Note, this patch set is based on top of not yet landed BPF verifier rotating mode patch set ([1]), as those changes make it easier to implement this for both fixed and rotating mode. This patch set is split from [1], as [1] is pretty much ready to go in, and this one is more centered around UAPI aspects and would probably require few iterations to finalize the UAPI. Regardless, getting this out early to get feedback and see if this is useful for users.

Patches #1-#4 are some preliminary clean ups, fixed, improvements. Patch #5 implements internal logic and changes how fixed log mode operates, see that patch for details. Patch #6 exposes log->len_max through UAPI. Patch #7 wires this feature in libbpf API. Patch #8 adds selftests for this feature, both for BPF_LOG_FIXED and default rotating log modes.

[0] https://lore.kernel.org/bpf/CAN+4W8iNoEbQzQVbB_o1W0MWBDV4xCJAq7K3f6psVE-kkCfMqg@mail.gmail.com/
[1] https://patchwork.kernel.org/project/netdevbpf/list/?series=734791&state=*

Andrii Nakryiko (8):
bpf: ignore verifier log reset in BPF_LOG_KERNEL mode
bpf: fix missing -EFAULT return on user log buf error in btf_parse()
bpf: avoid incorrect -EFAULT error in BPF_LOG_KERNEL mode
bpf: simplify logging related error conditions handling
bpf: keep track of total log content size in both fixed and rolling
modes
bpf: add log_size_actual output field to return log contents size
libbpf: wire through log_size_actual value returned from kernel selftests/bpf: add tests to validate log_size_actual feature

 include/linux/bpf.h                           |  2 +-
 include/linux/bpf_verifier.h                  | 12 +--
 include/linux/btf.h                           |  2 +-
 include/uapi/linux/bpf.h                      | 10 ++
 kernel/bpf/btf.c                              | 38 +++++---
 kernel/bpf/log.c                              | 68 ++++++++++----
 kernel/bpf/syscall.c                          | 16 ++--
 kernel/bpf/verifier.c                         | 12 ++-
 tools/include/uapi/linux/bpf.h                | 12 ++-
 tools/lib/bpf/bpf.c                           |  7 +-
 tools/lib/bpf/bpf.h                           | 11 ++-
 .../selftests/bpf/prog_tests/verifier_log.c   | 92 +++++++++++++++----
 12 files changed, 203 insertions(+), 79 deletions(-)
Signature · 1 line
2.34.1