bpf: Fix perf bpf event and audit prog id logging
Subsystems (3)
Jiri Olsa
reviewerRFC 1/1 bpf: Fix perf bpf event and audit prog id logging
hi,perf_event_bpf_event and bpf_audit_prog calls currently report zero program id for unload path.
It's because of the [1] change moved those audit calls into work queue and they are executed after the id is zeroed in bpf_prog_free_id.
I originally made a change that added 'id_audit' field to struct bpf_prog, which would be initialized as id, untouched and used in audit callbacks.
Then I realized we might actually not need to zero prog->aux->id in bpf_prog_free_id. It seems to be called just once on release paths. Tests seems ok with that.
thoughts?
thanks,
jirka
[1] d809e134be7a ("bpf: Prepare bpf_prog_put() to be called from irq context.")
--- kernel/bpf/syscall.c | 1 - 1 file changed, 1 deletion(-)
kernel/bpf/syscall.c +0 -1
@@ -1991,7 +1991,6 @@ void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock)__acquire (&prog_idr_lock);idr_remove (&prog_idr, prog->aux->id); - prog->aux->id =0 ;if (do_idr_lock)spin_unlock_irqrestore (&prog_idr_lock, flags);
Signature · 1 line
2.38.1
Daniel Borkmann
maintainerQuoted 31 lines · Jiri Olsa
hi,
perf_event_bpf_eventandbpf_audit_progcalls currently report zero program id for unload path.It's because of the [1] change moved those audit calls into work queue and they are executed after the id is zeroed in
bpf_prog_free_id.I originally made a change that added '
id_audit' field to structbpf_prog, which would be initialized as id, untouched and used in audit callbacks.Then I realized we might actually not need to zero
prog->aux->idinbpf_prog_free_id. It seems to be called just once on release paths. Tests seems ok with that.thoughts?
thanks,
jirka[1]
d809e134be7a("bpf: Preparebpf_prog_put()to be called from irq context.")--- kernel/bpf/syscall.c | 1 - 1 file changed, 1 deletion(-)kernel/bpf/syscall.c +0
-0
kernel/bpf/syscall.c void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock) · line 1994
Quoted 4 lines
kernel/bpf/syscall.c +0
-1@@ -1991,7 +1991,6 @@ void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock)__acquire (&prog_idr_lock);idr_remove (&prog_idr, prog->aux->id); - prog->aux->id =0 ;
This would trigger a race when offloaded progs are used, see also ad8ad79f4f60 ("bpf:
offload: free program id when device disappears"). __bpf_prog_offload_destroy() calls it, and my read is that later bpf_prog_free_id() then hits the early !prog->aux->id return path. Is there a reason for irq context to not defer the bpf_prog_free_id()?
Quoted 3 lines
kernel/bpf/syscall.c +0
-0if (do_idr_lock)spin_unlock_irqrestore (&prog_idr_lock, flags);
Quoted 27 lines · Daniel Borkmann
Quoted 26 lines · Jiri Olsa
hi,
perf_event_bpf_eventandbpf_audit_progcalls currently report zero program id for unload path.It's because of the [1] change moved those audit calls into work queue and they are executed after the id is zeroed in
bpf_prog_free_id.I originally made a change that added '
id_audit' field to structbpf_prog, which would be initialized as id, untouched and used in audit callbacks.Then I realized we might actually not need to zero
prog->aux->idinbpf_prog_free_id. It seems to be called just once on release paths. Tests seems ok with that.thoughts?
thanks,
jirka[1]
d809e134be7a("bpf: Preparebpf_prog_put()to be called from irq context.")--- kernel/bpf/syscall.c | 1 - 1 file changed, 1 deletion(-)
Quoted 13 lines
kernel/bpf/syscall.c void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock) · line 1993
Quoted 8 lines
kernel/bpf/syscall.c +0
-1@@ -1991,7 +1991,6 @@ void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock)__acquire (&prog_idr_lock);idr_remove (&prog_idr, prog->aux->id); - prog->aux->id =0 ;This would trigger a race when offloaded progs are used, see also
ad8ad79f4f60("bpf:
offload: free program id when device disappears").__bpf_prog_offload_destroy()calls it, and my read is that laterbpf_prog_free_id()then hits the early !prog->aux->id return path. Is there a reason for irq context to not defer thebpf_prog_free_id()?
there's comment saying:
/* bpf_prog_free_id() must be called first */
it was added together with the BPF_(PROG|MAP)_GET_NEXT_ID api:
34ad5580f8f9 bpf: Add BPF_(PROG|MAP)_GET_NEXT_ID command
Martin, any idea?
while looking on this I noticed we can remove the do_idr_lock argument (patch below) because it's always true and I think we need to upgrade all the prog_idr_lock locks to spin_lock_irqsave because bpf_prog_put could be called from irq context because of d809e134be7a
thanks,
jirka
---
include/linux/bpf.h +1 -1
@@ -1761,7 +1761,7 @@ void bpf_prog_inc(struct bpf_prog *prog);struct bpf_prog *__must_check bpf_prog_inc_not_zero (struct bpf_prog *prog);void bpf_prog_put (struct bpf_prog *prog); -void bpf_prog_free_id (struct bpf_prog *prog,bool do_idr_lock); +void bpf_prog_free_id (struct bpf_prog *prog);void bpf_map_free_id (struct bpf_map *map,bool do_idr_lock); struct btf_field *btf_record_find(const struct btf_record *rec,
kernel/bpf/offload.c +1 -1
@@ -217,7 +217,7 @@ static void __bpf_prog_offload_destroy(struct bpf_prog *prog) offload->offdev->ops->destroy (prog);/* Make sure BPF_PROG_GET_NEXT_ID can't find this dead program */ -bpf_prog_free_id (prog, true); +bpf_prog_free_id (prog);list_del_init (&offload->offloads);kfree (offload);
kernel/bpf/syscall.c +4 -10
@@ -1973,7 +1973,7 @@ static int bpf_prog_alloc_id(struct bpf_prog *prog)return id >0 ?0 : id; } -void bpf_prog_free_id (struct bpf_prog *prog,bool do_idr_lock) +void bpf_prog_free_id (struct bpf_prog *prog) {unsigned long flags;
@@ -1985,18 +1985,12 @@ void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock)if (!prog->aux->id)return ; -if (do_idr_lock) -spin_lock_irqsave (&prog_idr_lock, flags); -else -__acquire (&prog_idr_lock); +spin_lock_irqsave (&prog_idr_lock, flags);idr_remove (&prog_idr, prog->aux->id); prog->aux->id =0 ; -if (do_idr_lock) -spin_unlock_irqrestore (&prog_idr_lock, flags); -else -__release (&prog_idr_lock); +spin_unlock_irqrestore (&prog_idr_lock, flags); }static void __bpf_prog_put_rcu (struct rcu_head *rcu)
@@ -2048,7 +2042,7 @@ static void __bpf_prog_put(struct bpf_prog *prog, bool do_idr_lock)if (atomic64_dec_and_test (&aux->refcnt)) {/* bpf_prog_free_id() must be called first */ -bpf_prog_free_id (prog, do_idr_lock); +bpf_prog_free_id (prog);if (in_irq () ||irqs_disabled ()) {INIT_WORK (&aux->work, bpf_prog_put_deferred);
Quoted 49 lines · Jiri Olsa
Quoted 27 lines · Daniel Borkmann
Quoted 26 lines · Jiri Olsa
hi,
perf_event_bpf_eventandbpf_audit_progcalls currently report zero program id for unload path.It's because of the [1] change moved those audit calls into work queue and they are executed after the id is zeroed in
bpf_prog_free_id.I originally made a change that added '
id_audit' field to structbpf_prog, which would be initialized as id, untouched and used in audit callbacks.Then I realized we might actually not need to zero
prog->aux->idinbpf_prog_free_id. It seems to be called just once on release paths. Tests seems ok with that.thoughts?
thanks,
jirka[1]
d809e134be7a("bpf: Preparebpf_prog_put()to be called from irq context.")--- kernel/bpf/syscall.c | 1 - 1 file changed, 1 deletion(-)Quoted 13 lines
kernel/bpf/syscall.c void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock) · line 1993
Quoted 8 lines
kernel/bpf/syscall.c +0
-1@@ -1991,7 +1991,6 @@ void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock)__acquire (&prog_idr_lock);idr_remove (&prog_idr, prog->aux->id); - prog->aux->id =0 ;This would trigger a race when offloaded progs are used, see also
ad8ad79f4f60("bpf:
offload: free program id when device disappears").__bpf_prog_offload_destroy()calls it, and my read is that laterbpf_prog_free_id()then hits the early !prog->aux->id return path. Is there a reason for irq context to not defer thebpf_prog_free_id()?there's comment saying:
/*bpf_prog_free_id()must be called first */it was added together with the BPF_(PROG|MAP)_GET_NEXT_ID api:
34ad5580f8f9bpf: Add BPF_(PROG|MAP)_GET_NEXT_ID command
Quoted 6 lines
Martin, any idea?
while looking on this I noticed we can remove the
do_idr_lockargument (patch below) because it's always true and I think we need to upgrade all theprog_idr_locklocks tospin_lock_irqsavebecausebpf_prog_putcould be called from irq context because ofd809e134be7a
before we dive into rabbit hole let's understand
the severity of
"perf_event_bpf_event and bpf_audit_prog calls currently report zero
program id for unload path"
and why we cannot leave it as-is.
Quoted 57 lines · Alexei Starovoitov
Quoted 49 lines · Jiri Olsa
Quoted 27 lines · Daniel Borkmann
Quoted 26 lines · Jiri Olsa
hi,
perf_event_bpf_eventandbpf_audit_progcalls currently report zero program id for unload path.It's because of the [1] change moved those audit calls into work queue and they are executed after the id is zeroed in
bpf_prog_free_id.I originally made a change that added '
id_audit' field to structbpf_prog, which would be initialized as id, untouched and used in audit callbacks.Then I realized we might actually not need to zero
prog->aux->idinbpf_prog_free_id. It seems to be called just once on release paths. Tests seems ok with that.thoughts?
thanks,
jirka[1]
d809e134be7a("bpf: Preparebpf_prog_put()to be called from irq context.")--- kernel/bpf/syscall.c | 1 - 1 file changed, 1 deletion(-)Quoted 13 lines
kernel/bpf/syscall.c void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock) · line 1993
Quoted 8 lines
kernel/bpf/syscall.c +0
-1@@ -1991,7 +1991,6 @@ void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock)__acquire (&prog_idr_lock);idr_remove (&prog_idr, prog->aux->id); - prog->aux->id =0 ;This would trigger a race when offloaded progs are used, see also
ad8ad79f4f60("bpf:
offload: free program id when device disappears").__bpf_prog_offload_destroy()calls it, and my read is that laterbpf_prog_free_id()then hits the early !prog->aux->id return path. Is there a reason for irq context to not defer thebpf_prog_free_id()?there's comment saying:
/*bpf_prog_free_id()must be called first */it was added together with the BPF_(PROG|MAP)_GET_NEXT_ID api:
34ad5580f8f9bpf: Add BPF_(PROG|MAP)_GET_NEXT_ID commandQuoted 6 lines
Martin, any idea?
while looking on this I noticed we can remove the
do_idr_lockargument (patch below) because it's always true and I think we need to upgrade all theprog_idr_locklocks tospin_lock_irqsavebecausebpf_prog_putcould be called from irq context because ofd809e134be7a
Quoted 6 lines
before we dive into rabbit hole let's understand
the severity of
"perf_event_bpf_eventandbpf_audit_progcalls currently report zero
program id for unload path"and why we cannot leave it as-is.
I found this because I wanted use perf bpf-event to monitor bpf programs loads/unloads.. and I need 'id' on the unload event
perf_event_bpf_event is irq safe so quick fix for me would be to move it back:
if (atomic64_dec_and_test(&aux->refcnt)) {
+ perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
/* bpf_prog_free_id() must be called first */
bpf_prog_free_id(prog, do_idr_lock);FWIW I also used fix below for a while for testing
jirka
---
include/linux/bpf.h +1 -0
@@ -1163,6 +1163,7 @@ struct bpf_prog_aux {u32 max_tp_access;u32 stack_depth;u32 id; +u32 id_audit;u32 func_cnt;/* used by non-func prog as the number of func progs */ u32 func_idx;/* 0 for non-func prog, the index in func array for func prog */ u32 attach_btf_id;/* in-kernel BTF type id to attach to */
kernel/bpf/syscall.c +2 -2
@@ -1930,7 +1930,7 @@ static void bpf_audit_prog(const struct bpf_prog *prog, unsigned int op)if (unlikely (!ab))return ;audit_log_format (ab,"prog-id=%u op=%s" , - prog->aux->id, bpf_audit_str[op]); + prog->aux->id_audit, bpf_audit_str[op]);audit_log_end (ab); }
@@ -1942,7 +1942,7 @@ static int bpf_prog_alloc_id(struct bpf_prog *prog)spin_lock_bh (&prog_idr_lock); id =idr_alloc_cyclic (&prog_idr, prog,1 ,INT_MAX ,GFP_ATOMIC );if (id >0 ) - prog->aux->id = id; + prog->aux->id = prog->aux->id_audit = id;spin_unlock_bh (&prog_idr_lock);idr_preload_end ();
kernel/events/core.c +1 -1
@@ -9065,7 +9065,7 @@ void perf_event_bpf_event(struct bpf_prog *prog, }, .type = type, .flags = flags, - .id = prog->aux->id, + .id = prog->aux->id_audit, }, };