dereference of modified ctx ptr
Hi -
Depending on how complicated my BPF programs are, the compiler may modify a context pointer before accessing the context.
The specific context is our struct bpf_ghost_msg:
https://github.com/google/ghost-userspace/blob/main/kernel/ghost_uapi.h#L385
But in general, it's something like this:
struct bar {
unsigned int x;
unsigned int y;
};
struct foo {
union {
struct bar bar;
struct baz baz;
};
};Given a struct foo *p, where I try to read p->bar.y, the compiler usually emits something like:
r1 = p r2 = *(u32 *)(r1 +4)
which is fine.
but it could also emit something like:
r1 = p r1 += 4 // uh oh! r2 = *(u32 *)(r1 +0)
I tried getting around it though various uses of "noinline" on functions that take context pointers (or the union structs), e.g. here:
https://github.com/google/ghost-userspace/blob/main/third_party/bpf/biff.bpf.c#L405
But that's extremely brittle, and the compiler can legally modify pointers to get to internal fields.
Recently, I've taken to just copying my context payload onto the stack and copying it back, which keeps the amount of times I run into this to a minimum, but is not ideal.
Can we change the verifier to allow a ctx pointer to be modified? Or is there some other trick I can do to prevent the compiler from modifying the pointer?
Thanks,
Barret
Quoted 27 lines · Barret Rhoden
Hi -
Depending on how complicated my BPF programs are, the compiler may modify a context pointer before accessing the context.
The specific context is our struct
bpf_ghost_msg:https://github.com/google/ghost-userspace/blob/main/kernel/ghost_uapi.h#L385
But in general, it's something like this:
struct bar {
unsigned int x;
unsigned int y;
};
struct foo {
union {
struct bar bar;
struct baz baz;
};
};Given a struct foo *p, where I try to read
p->bar.y, the compiler usually emits something like:r1 = p
r2 = *(u32 *)(r1 +4)
Quoted 7 lines
which is fine.
but it could also emit something like:
r1 = p
r1 += 4 // uh oh!
r2 = *(u32 *)(r1 +0)
Do you have an example that compiler generates code like the above? In bpf backend, we have an optimization to capture thing like this and optimize it to
r1 = p
r2 = *(u32 *)(r1 + 4)
Quoted 10 lines
I tried getting around it though various uses of "noinline" on functions that take context pointers (or the union structs), e.g. here:
https://github.com/google/ghost-userspace/blob/main/third_party/bpf/biff.bpf.c#L405
But that's extremely brittle, and the compiler can legally modify pointers to get to internal fields.
Recently, I've taken to just copying my context payload onto the stack and copying it back, which keeps the amount of times I run into this to a minimum, but is not ideal.
Can we change the verifier to allow a ctx pointer to be modified? Or is
We cannot do that for complicated case. Note that "ctx_ptr + offset" often subject to rewrite, so at verification, the verifer need to ensure the offset is a constant. This may not be the case. See an example at
In such cases, you could add memory barrier to prevent some llvm optimization so the result code will stay with "ctx_ptr + offset" format.
Also, Eduard has worked on this issue for a while and currently it is in progress.
Quoted 6 lines
there some other trick I can do to prevent the compiler from modifying the pointer?
Thanks,
Barret