Skip to contentSkip to conversation

bpf: Update kfunc __sz documentation

PATCH In review Joanne Koong 2 participants 3 messages Feb 14, 2023 – Feb 18, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bpf: Update kfunc __sz documentation

A bpf program calling a kfunc with a __sz-annotated arg must explicitly initialize the stack themselves if the pointer to the memory region is a pointer to the stack. This is because in the verifier, we do not explicitly initialize the stack space for reg type PTR_TO_STACK kfunc args. Thus, the verifier will reject the program with:

invalid indirect read from stack
arg#0 arg#1 memory, len pair leads to invalid memory access

Alternatively, the verifier could support initializing the stack space on behalf of the program for KF_ARG_PTR_TO_MEM_SIZE args, but this has some drawbacks. For example this would not allow the verifier to reject a program for passing in an uninitialized PTR_TO_STACK for an arg that should have valid data. Another example is that since there's no current way in a kfunc to differentiate between whether the arg should be treated as uninitialized or not, additional check_mem_access calls would need to be called even on PTR_TO_STACKs that have been initialized, which is inefficient. Please note that non-kfuncs don't have this problem because of the MEM_UNINIT tag; only if the arg is tagged as MEM_UNINIT, then do we call check_mem_access byte-by-byte for the size of the buffer.

---
 Documentation/bpf/kfuncs.rst | 35 +++++++++++++++++++++++++++++++----
 1 file changed, 31 insertions(+), 4 deletions(-)
Documentation/bpf/kfuncs.rst +31 -4
index ca96ef3f6896..97497a7879d6 100644
@@ -71,10 +71,37 @@ An example is given below::
         ...
         }
 
-Here, the verifier will treat first argument as a PTR_TO_MEM, and second
-argument as its size. By default, without __sz annotation, the size of the type
-of the pointer is used. Without __sz annotation, a kfunc cannot accept a void
-pointer.
+Here, the verifier will treat first argument (KF_ARG_PTR_TO_MEM_SIZE) as a
+pointer to the memory region and second argument as its size. By default,
+without __sz annotation, the size of the type of the pointer is used. Without
+__sz annotation, a kfunc cannot accept a void pointer.
+
+Please note that if the memory is on the stack, the stack space must be
+explicitly initialized by the program. For example:
+
+.. code-block:: c
+
+	SEC("tc")
+	int prog(struct __sk_buff *skb)
+	{
+		char buf[8];
+
+		bpf_memzero(buf, sizeof(buf));
+	...
+	}
+
+should be
+
+.. code-block:: c
+
+	SEC("tc")
+	int prog(struct __sk_buff *skb)
+	{
+		char buf[8] = {};
+
+		bpf_memzero(buf, sizeof(buf));
+	...
+	}
 
 2.2.2 __k Annotation
 --------------------
Signature · 1 line
2.30.2
#1 lore raw
Quoted 59 lines · Joanne Koong

A bpf program calling a kfunc with a __sz-annotated arg must explicitly initialize the stack themselves if the pointer to the memory region is a pointer to the stack. This is because in the verifier, we do not explicitly initialize the stack space for reg type PTR_TO_STACK kfunc args. Thus, the verifier will reject the program with:

invalid indirect read from stack
arg#0 arg#1 memory, len pair leads to invalid memory access

Alternatively, the verifier could support initializing the stack space on behalf of the program for KF_ARG_PTR_TO_MEM_SIZE args, but this has some drawbacks. For example this would not allow the verifier to reject a program for passing in an uninitialized PTR_TO_STACK for an arg that should have valid data. Another example is that since there's no current way in a kfunc to differentiate between whether the arg should be treated as uninitialized or not, additional check_mem_access calls would need to be called even on PTR_TO_STACKs that have been initialized, which is inefficient. Please note that non-kfuncs don't have this problem because of the MEM_UNINIT tag; only if the arg is tagged as MEM_UNINIT, then do we call check_mem_access byte-by-byte for the size of the buffer.

---
 Documentation/bpf/kfuncs.rst | 35 +++++++++++++++++++++++++++++++----
 1 file changed, 31 insertions(+), 4 deletions(-)
Documentation/bpf/kfuncs.rst +19 -4
index ca96ef3f6896..97497a7879d6 100644
@@ -71,10 +71,37 @@ An example is given below::
...
         }
 
-Here, the verifier will treat first argument as a PTR_TO_MEM, and second
-argument as its size. By default, without __sz annotation, the size of the type
-of the pointer is used. Without __sz annotation, a kfunc cannot accept a void
-pointer.
+Here, the verifier will treat first argument (KF_ARG_PTR_TO_MEM_SIZE) as a
+pointer to the memory region and second argument as its size. By default,
+without __sz annotation, the size of the type of the pointer is used. Without
+__sz annotation, a kfunc cannot accept a void pointer.
+
+Please note that if the memory is on the stack, the stack space must be
+explicitly initialized by the program. For example:
+
+.. code-block:: c
+
+       SEC("tc")
+       int prog(struct __sk_buff *skb)
+       {
+               char buf[8];
+
+               bpf_memzero(buf, sizeof(buf));
+       ...
+       }
+

Documentation/bpf/kfuncs.rst An example is given below:: · line 102

Quoted 10 lines
Documentation/bpf/kfuncs.rst +10 -0
+should be
+
+.. code-block:: c
+
+       SEC("tc")
+       int prog(struct __sk_buff *skb)
+       {
+               char buf[8] = {};
+
+               bpf_memzero(buf, sizeof(buf));

Actually we might go the other way.
Instead of asking users to explicitly init things
we will allow uninit memory.
See this discussion:
https://lore.kernel.org/bpf/[email protected]/

Eduard, is about to send those verifier patches.

In parallel we can relax __sz to accept uninit under allow_uninit_stack.

#2 lore raw
Quoted 78 lines · Alexei Starovoitov
Quoted 59 lines · Joanne Koong

A bpf program calling a kfunc with a __sz-annotated arg must explicitly initialize the stack themselves if the pointer to the memory region is a pointer to the stack. This is because in the verifier, we do not explicitly initialize the stack space for reg type PTR_TO_STACK kfunc args. Thus, the verifier will reject the program with:

invalid indirect read from stack
arg#0 arg#1 memory, len pair leads to invalid memory access

Alternatively, the verifier could support initializing the stack space on behalf of the program for KF_ARG_PTR_TO_MEM_SIZE args, but this has some drawbacks. For example this would not allow the verifier to reject a program for passing in an uninitialized PTR_TO_STACK for an arg that should have valid data. Another example is that since there's no current way in a kfunc to differentiate between whether the arg should be treated as uninitialized or not, additional check_mem_access calls would need to be called even on PTR_TO_STACKs that have been initialized, which is inefficient. Please note that non-kfuncs don't have this problem because of the MEM_UNINIT tag; only if the arg is tagged as MEM_UNINIT, then do we call check_mem_access byte-by-byte for the size of the buffer.

---
 Documentation/bpf/kfuncs.rst | 35 +++++++++++++++++++++++++++++++----
 1 file changed, 31 insertions(+), 4 deletions(-)
Documentation/bpf/kfuncs.rst +19 -4
index ca96ef3f6896..97497a7879d6 100644
@@ -71,10 +71,37 @@ An example is given below::
...
         }
 
-Here, the verifier will treat first argument as a PTR_TO_MEM, and second
-argument as its size. By default, without __sz annotation, the size of the type
-of the pointer is used. Without __sz annotation, a kfunc cannot accept a void
-pointer.
+Here, the verifier will treat first argument (KF_ARG_PTR_TO_MEM_SIZE) as a
+pointer to the memory region and second argument as its size. By default,
+without __sz annotation, the size of the type of the pointer is used. Without
+__sz annotation, a kfunc cannot accept a void pointer.
+
+Please note that if the memory is on the stack, the stack space must be
+explicitly initialized by the program. For example:
+
+.. code-block:: c
+
+       SEC("tc")
+       int prog(struct __sk_buff *skb)
+       {
+               char buf[8];
+
+               bpf_memzero(buf, sizeof(buf));
+       ...
+       }
+

Documentation/bpf/kfuncs.rst An example is given below:: · line 102

Quoted 10 lines
Documentation/bpf/kfuncs.rst +10 -0
+should be
+
+.. code-block:: c
+
+       SEC("tc")
+       int prog(struct __sk_buff *skb)
+       {
+               char buf[8] = {};
+
+               bpf_memzero(buf, sizeof(buf));

Actually we might go the other way.
Instead of asking users to explicitly init things
we will allow uninit memory.
See this discussion:
https://lore.kernel.org/bpf/[email protected]/

Quoted 3 lines

Eduard, is about to send those verifier patches.

In parallel we can relax __sz to accept uninit under allow_uninit_stack.

in that case, for kfuncs it needs to grow the stack state if the buffer + __sz is beyond the current allocated stack, or else check_stack_range_initialized() will automatically fail if the user tries to pass in an uninitialized buffer. i have a local patch for this in my tree, I'll tidy it up and submit it next week