Skip to contentSkip to conversation

audit support for BPF notification

RFC Jiri Olsa 5 participants 12 messages Aug 9, 2019 – Nov 5, 2019

lore
Share & tools
lore

Jiri Olsa

author
#0 lore raw

hi,
I posted initial change that allows auditd to log BPF program load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

We tried to push pure AUDIT interface for BPF program notification, but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

thanks,
jirka

#1 lore raw

Hello,

Quoted 3 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 3 lines

We tried to push pure AUDIT interface for BPF program notification, but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

I'd like to understand what the basic problem is that needs to be solved.

-Steve

author
#2 lore raw
Quoted 14 lines · Steve Grubb

Hello,

Quoted 3 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 8 lines
Quoted 3 lines

We tried to push pure AUDIT interface for BPF program notification, but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

> I'm not sure exactly what the issue is. You can audit for specific syscall > and argument. So, if you want to see loads, then you can make a rule like:

> -a always,exit -F arch=b64 -S bpf -F a0=5

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status or license info

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Quoted 8 lines
Quoted 6 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

jirka

#3 lore raw
Quoted 33 lines · EDT Jiri Olsa
Quoted 16 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 2 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

The way this is normally done is to add a supplemental record. For example, when auditing the open syscall, we also get CWD & PATH supplemental records. When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what is selected for audit via audit rules. So, what one could do is set a rule for the bpf syscall and then when it triggers, we get these other records added to the bpf syscall event.

Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal termination being something we might want. Does the program go through something like an exit syscall internally?

Quoted 11 lines
Quoted 7 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends up.

Thanks,
-Steve

author
#4 lore raw
Quoted 40 lines · Steve Grubb
Quoted 33 lines · EDT Jiri Olsa
Quoted 16 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 2 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

Quoted 7 lines

The way this is normally done is to add a supplemental record. For example, when auditing the open syscall, we also get CWD & PATH supplemental records. When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what is selected for audit via audit rules. So, what one could do is set a rule for the bpf syscall and then when it triggers, we get these other records added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just single notification system without specific hooks for audit, so we ended up with perf specific interface

Quoted 11 lines
Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal termination being something we might want. Does the program go through something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count, which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial file descriptor is closed.. and then there's the networking world, which might have some other specific ways.. but it all ends up in bpf_prog_put and zero reference count.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka

author
#5 lore raw

hi,
Adding Vlad Dronov to the loop, because he asked
about this functionality some time ago.

Vlad, the full thread can be found in here:
https://www.redhat.com/archives/linux-audit/2019-August/msg00004.html

Any thoughts on this?

thanks,
jirka

Quoted 102 lines · Jiri Olsa
Quoted 40 lines · Steve Grubb
Quoted 33 lines · EDT Jiri Olsa
Quoted 16 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 2 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

Quoted 7 lines

The way this is normally done is to add a supplemental record. For example, when auditing the open syscall, we also get CWD & PATH supplemental records. When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what is selected for audit via audit rules. So, what one could do is set a rule for the bpf syscall and then when it triggers, we get these other records added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just single notification system without specific hooks for audit, so we ended up with perf specific interface

Quoted 11 lines
Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal termination being something we might want. Does the program go through something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count, which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial file descriptor is closed.. and then there's the networking world, which might have some other specific ways.. but it all ends up in bpf_prog_put and zero reference count.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka

author
#6 lore raw

cc-ing Petr Matousek

jirka

Quoted 115 lines · Jiri Olsa

hi,
Adding Vlad Dronov to the loop, because he asked
about this functionality some time ago.

Vlad, the full thread can be found in here:
https://www.redhat.com/archives/linux-audit/2019-August/msg00004.html

Any thoughts on this?

thanks,
jirka

Quoted 102 lines · Jiri Olsa
Quoted 40 lines · Steve Grubb
Quoted 33 lines · EDT Jiri Olsa
Quoted 16 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 2 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

Quoted 7 lines

The way this is normally done is to add a supplemental record. For example, when auditing the open syscall, we also get CWD & PATH supplemental records. When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what is selected for audit via audit rules. So, what one could do is set a rule for the bpf syscall and then when it triggers, we get these other records added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just single notification system without specific hooks for audit, so we ended up with perf specific interface

Quoted 11 lines
Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal termination being something we might want. Does the program go through something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count, which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial file descriptor is closed.. and then there's the networking world, which might have some other specific ways.. but it all ends up in bpf_prog_put and zero reference count.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka

#7 lore raw

Seems there have been no reply to this...

Jiri, what is the current status?

Vlad, what is the Product Security's view on this? Is the audit support for bpf programs loading/unloading a requirement for full support of eBPF (as opposed to tech preview)?

Thanks,

Jiri

Quoted 120 lines · Jiri Olsa

cc-ing Petr Matousek

jirka

Quoted 115 lines · Jiri Olsa

hi,
Adding Vlad Dronov to the loop, because he asked
about this functionality some time ago.

Vlad, the full thread can be found in here:
https://www.redhat.com/archives/linux-audit/2019-August/msg00004.html

Any thoughts on this?

thanks,
jirka

Quoted 102 lines · Jiri Olsa
Quoted 40 lines · Steve Grubb
Quoted 33 lines · EDT Jiri Olsa
Quoted 16 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 2 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5  
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

Quoted 7 lines

The way this is normally done is to add a supplemental record. For example, when auditing the open syscall, we also get CWD & PATH supplemental records. When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what is selected for audit via audit rules. So, what one could do is set a rule for the bpf syscall and then when it triggers, we get these other records added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just single notification system without specific hooks for audit, so we ended up with perf specific interface

Quoted 11 lines
Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal termination being something we might want. Does the program go through something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count, which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial file descriptor is closed.. and then there's the networking world, which might have some other specific ways.. but it all ends up in bpf_prog_put and zero reference count.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka

author
#8 lore raw
Quoted 3 lines · Jiri Benc

Seems there have been no reply to this...

Jiri, what is the current status?

same, there's still no answer from Product Security

jirka

Quoted 131 lines

Vlad, what is the Product Security's view on this? Is the audit support for bpf programs loading/unloading a requirement for full support of eBPF (as opposed to tech preview)?

Thanks,

Jiri

Quoted 120 lines · Jiri Olsa

cc-ing Petr Matousek

jirka

Quoted 115 lines · Jiri Olsa

hi,
Adding Vlad Dronov to the loop, because he asked
about this functionality some time ago.

Vlad, the full thread can be found in here:
https://www.redhat.com/archives/linux-audit/2019-August/msg00004.html

Any thoughts on this?

thanks,
jirka

Quoted 102 lines · Jiri Olsa
Quoted 40 lines · Steve Grubb
Quoted 33 lines · EDT Jiri Olsa
Quoted 16 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more before undertaking this effort. We normally do not audit from user space what happens in the kernel. Doing this can be racy and it keeps auditd from doing the one job it has - which is to grab events and record them to disk and send them out the realtime interface.

Quoted 2 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5  
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

Quoted 7 lines

The way this is normally done is to add a supplemental record. For example, when auditing the open syscall, we also get CWD & PATH supplemental records. When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what is selected for audit via audit rules. So, what one could do is set a rule for the bpf syscall and then when it triggers, we get these other records added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just single notification system without specific hooks for audit, so we ended up with perf specific interface

Quoted 11 lines
Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal termination being something we might want. Does the program go through something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count, which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial file descriptor is closed.. and then there's the networking world, which might have some other specific ways.. but it all ends up in bpf_prog_put and zero reference count.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I'd like to understand what the basic problem is that needs to be solved.

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka

#9 lore raw
In reply to Jiri Benc

Hello, Jiri,

Unfortunately, I'm not with the Product Security anymore and cannot speak in ProdSec name. My personal view is that we/customers must be able to log eBPF program load/unload events in some way (not necessary by audit).

Best regards,
Vladis Dronov | Red Hat, Inc. | The Core Kernel | Senior Software Engineer

----- Original Message -----

Quoted 17 lines

From: "Jiri Benc" <[email protected]>
To: "Jiri Olsa" <[email protected]>
Cc: "Steve Grubb" <[email protected]>, [email protected], "Stanislav Kozina" <[email protected]>, "Yauheni Kaliuta" <[email protected]>, "Toke Høiland-Jørgensen" <[email protected]>, "Arnaldo Carvalho de Melo" <[email protected]>, "Jesper Dangaard Brouer" <[email protected]>, "Vlad Dronov" <[email protected]>, "Petr Matousek" <[email protected]>, "Rashid Khan" <[email protected]>
Sent: Monday, November 4, 2019 2:05:18 PM
Subject: Re: [RFC] audit support for BPF notification

Seems there have been no reply to this...

Jiri, what is the current status?

Vlad, what is the Product Security's view on this? Is the audit support for bpf programs loading/unloading a requirement for full support of eBPF (as opposed to tech preview)?

Quoted 144 lines

Thanks,

Jiri

Quoted 137 lines · Jiri Olsa

cc-ing Petr Matousek

jirka

Quoted 132 lines · Jiri Olsa

hi,
Adding Vlad Dronov to the loop, because he asked
about this functionality some time ago.

Vlad, the full thread can be found in here:
https://www.redhat.com/archives/linux-audit/2019-August/msg00004.html

Any thoughts on this?

thanks,
jirka

Quoted 119 lines · Jiri Olsa
Quoted 47 lines · Steve Grubb
Quoted 40 lines · EDT Jiri Olsa
Quoted 21 lines · Steve Grubb

Hello,

Quoted 4 lines · EDT Jiri Olsa

I posted initial change that allows auditd to log BPF program

load/unload events, it's in here:
https://github.com/linux-audit/audit-userspace/pull/104

Thanks for the patch...but we probably should have talked a bit more
before undertaking this effort. We normally do not audit from user space
what happens in the kernel. Doing this can be racy and it keeps auditd
from doing the one job it has - which is to grab events and record them
to disk and send them out the realtime interface.

Quoted 3 lines

We tried to push pure AUDIT interface for BPF program notification,

Quoted 7 lines
Quoted 2 lines

but it was denied, the discussion is in here:
https://marc.info/?t=153866123200003&r=1&w=2

Hmm. The email I remember was here:
https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html

and was only 2 emails long with no answer to my question. :-)

oops, sry about that, your question was:

	> I'm not sure exactly what the issue is. You can audit for
	> specific
	> syscall
	> and argument. So, if you want to see loads, then you can make a
	> rule
	> like:
	> 
	> -a always,exit -F arch=b64 -S bpf -F a0=5
Quoted 5 lines

The problem with above for us is that we also:

- need to log also other properties of the BPF program, which are not visible from BPF syscall arguments, like its ID, JIT status

Quoted 12 lines

The way this is normally done is to add a supplemental record. For example,
when auditing the open syscall, we also get CWD & PATH supplemental records.
When auditing connect, we get a SOCKADDR supplemental record. We have requirements around selective audit whereby the admin is in control of what
is selected for audit via audit rules. So, what one could do is set a rule
for the bpf syscall and then when it triggers, we get these other records
added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just single notification system without specific hooks for audit, so we ended up with perf specific interface

Quoted 5 lines
Quoted 1 line

or license info

This ^^ is not a security issue.

Quoted 8 lines
Quoted 2 lines

- need to see BPF program UNLOAD, which is not done via syscall, so those would be unvisible at all

Is there a place in the kernel where this happens? I could see abnormal
termination being something we might want. Does the program go through
something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count, which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial file descriptor is closed.. and then there's the networking world, which might have some other specific ways.. but it all ends up in bpf_prog_put and zero reference count.

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines

The outcome of the discussion was to use perf event interface for BPF notification and use it in some deamon.. audit was our first choice.

thoughts?

I'd like to understand what the basic problem is that needs to be solved.

Quoted 7 lines
Quoted 3 lines

we need a way for administrators to see the history of loaded BPF programs, to help investigating issues related to BPF.. and the only BPF interface for this data is through perf ring buffer

That is really not the audit way. Let's keep talking to see where this ends
up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

#10 lore raw
In reply to Jiri Benc

[an addition]

I also believe that this log entry should include program source and/or bytecode checksum so customer/our support can verify that exactly this eBPF program was loaded/unloaded and not the program that someone states that it was loaded.

Best regards,
Vladis Dronov | Red Hat, Inc. | The Core Kernel | Senior Software Engineer

----- Original Message -----

Quoted 13 lines

From: "Jiri Benc" <[email protected]>
To: "Jiri Olsa" <[email protected]>
Cc: "Steve Grubb" <[email protected]>, [email protected], "Stanislav Kozina" <[email protected]>, "Yauheni Kaliuta" <[email protected]>, "Toke Høiland-Jørgensen" <[email protected]>, "Arnaldo Carvalho de Melo" <[email protected]>, "Jesper Dangaard Brouer" <[email protected]>, "Vlad Dronov" <[email protected]>, "Petr Matousek" <[email protected]>, "Rashid Khan" <[email protected]>
Sent: Monday, November 4, 2019 2:05:18 PM
Subject: Re: [RFC] audit support for BPF notification

Seems there have been no reply to this...

Jiri, what is the current status?

Quoted 7 lines

Vlad, what is the Product Security's view on this? Is the audit support for bpf programs loading/unloading a requirement for full support of eBPF (as opposed to tech preview)?

Thanks,

Jiri

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

#11 lore raw
In reply to Jiri Benc
Quoted 7 lines · Jiri Benc

Seems there have been no reply to this...

Jiri, what is the current status?

Vlad, what is the Product Security's view on this? Is the audit support for bpf programs loading/unloading a requirement for full support of eBPF (as opposed to tech preview)?

As a reminder, this is a public mailing list that focuses on the technical development of the upstream Linux audit project; Red Hat product decisions should not be discussed here.

Signature · 2 lines
paul moore
www.paul-moore.com