Upstream solution for auditing file system objects
Greetings,
I'm writing this e-mail to facilitate some discussion on an audit feature for inclusion to the mainline kernel's audit subsysystem. I've written out a "brief" description of the problem with some of the associated problems it introduces and some of the outlines of potential solutions below. For the most part, the idea stays the same, and the mechanism we use to implement it varies.
This message is fairly long. My apologies.
-----------
The Problem:
~~~~~~~~~~~~
Over the last two months, I've been given the daunting task of implementing a feature by which an administrator can specify from user space, a list of file system objects (namely regular files and directories) that he/she wishes to audit. The administrator would presumably like to,
1. Specify a file, "/etc/shadow"
2. Filter based on some criteria, "I only care if /etc/shadow is open()'d by uid=X" (Filtering could be more granular then this or less, that's arbitrary at this point)
3. Be reassured that one may not bypass the audit subsystem nor lose an audit record.
Assuming we can do the filtering aspect in user space via auditd (or whatever), we've introduced some kernel work. For instance, when we say we want to audit a file or directory, what we're really saying is,
"I'm interested in knowing when X syscall accesses/alters Y file/directory, so send me a record of this activity."
Some notable design problems to introduce now are with the "identity" of Y file/directory with respect to the kernel. In fact, we only really care about paths of which file/directory Y completes. I.e: If we want to audit /etc/shadow, we might not care if /etc/shadow is moved to /tmp/shadow. This means that any access/alteration of /tmp/shadow would go unnoticed (unless otherwise specified). However, if /tmp/shadow were a hardlink to /etc/shadow, we would then still care, because we are still effectively /etc/shadow... if that makes sense :-)
These are things that our design must consider in some way.
1. Identifying a file, such as /etc/shadow, solely by a pathname, enables someone to create a hardlink to /etc/shadow from a different directory and by-pass the audit subsystem. Also, we can't assume we have this user space definition of a file/directory in the kernel.
2. Identifying a file, such as /etc/shadow, solely by its inode, enables a by-pass of the audit subsystem, when say, /etc/shadow gets copied to /etc/shadow+ and moved back. Because the inode has changed, we're no longer the "same" file system object. Similarly, if we delete an audited file, /tmp/ultra_secret (in /tmp? sounds like my old university), and recreate it, we'd like /tmp/ultra_secret to be audited. This falls back on the "we don't want audit records to be lost" requirement.
3. How do we treat mounts and --bind mounts? I won't go into this in this e-mail. There's several design decisions to made with regards to this issue.
4. How can we obtain the full path of the file system object without reconstructing it in the kernel? Xattrs? Integer cookies/hashes that key a table in user space?
5. Whats the best approach at capturing information about the syscall accessing / altering the file/directory? We might lose records if we attempt to capture this information from VFS hooks as we could short-circuit before we ever reached that point in the function. We'd then need a way of related the audit of a system call that in place now with the file/directory its accessing. (This particiular issue is not talked about in this e-mail in any detail, really)
The Reason:
~~~~~~~~~~~
The current audit subsystem does not contain this capability. This capability is essential for meeting Controlled Access Protection Profile (CAPP) Evaluation Assurance Level (EAL) 4+ requirements.
The Solutions:
~~~~~~~~~~~~~~
(These are brief notes. More implementation detail can be provided it asked for. If someone is interested in discussing an actual solution, we can get into that - I'd like to get into that. I'm just trying to give you a sense of some of the things that have been thought about).
1. Inotify. Inotify implements a good portion of what we'd like to do and looks like its well on its way into the mainline kernel. What would need to be added to inotify:
A. A way in which it could, if desired, auto-watch the file/directory associated with an IN_CREATE_FILE/SUBDIR event. This would also require we have some way to communicate , "We want this <file/directory> to be auto-watched" -- There might be an issue here because we're now giving meaning to the file/directory name associated with the inode. We're now saying, "If the inode changes, but the name associated with it is the same, watch the new inode automatically."
B. Inotify would have to be able to notify the audit subsystem of events via a direct call internally, when the event is triggered. You'd have some device with an "audit" flag on it that says, "Hey I'm interested in receiving events on behalf of audit, send them to me, and I'll pass them directly on to the audit subsystem." This would be fairly easy to do, but would undermine the inotify event queuing process because we could not take any chances of losing event records in a queue that's overflowed.
2. Linux Security Module. This is perhaps the most daunting approach. This solution would allow us to use the i_security field of the inode to do our "book keeping." We'd also have at our disposal the the security hooks already in-place in the VFS. This would allow us to intercept the relevant syscalls via function pointers, to make decisions on what we want to do. Using the i_security field to hold our table of auditable files/directories on parent directory inodes, would enable us to ask the question, "Does this file/directory that's just been created beneath me need to be marked as "auditable?" from with in, say, security_inode_post_create().
The advantages to a module is that we can make this audit option more... optional. We're not introducing any new hooks and we're using the LSM's inode i_security field, too. This keeps the inode structure's size flat and minimizes our impact in the VFS. Disadvantages are that for it to play nicely with other LSMs like SELinux, we'd need the LSM stacker or else we'd need to disable SELinux. We'd also be forcing the question, "Is this an appropriate use of the LSM framework?" Although this feature is definitely security-relevant and related, is what it does an appropriate context for use of the LSM framework? Some would say that the LSM framework is for Access Control only, others would disagree.
3. SELinux. This feature is not currently in SELinux (as we've been told), but we've had interest by the NSA in implementing this feature for SELinux. We'd have a similar solution to the LSM, except we'd encapsulate ourselves within SELinux, and use its framework to drive the solution. This might prove less resilient, politically, because we'd be under its umbrella. I'm not clear on the exact implementation of this in SELinux -- It sounds like it'd be a patch to the security server and hooks, the creation of an audit label, and the invention of some additional structure for "book keeping" purposes. Also, there is a general problem with using SELinux specifically with figuring out how to certify with it enabled. There's a lot of complications (mostly unknowns) -- Complications we'd willingly work on, if need be. Obviously design decisions would have to be factor in as well.
4. Audit subsystem. To me, this is the best solution (maybe even worst solution? we'll see what you have to say), but probably the nastiest one. Let the audit subsystem hook VFS in all the necessary places and give audit an inode field to do the "book keeping." The advantages here is that we're encapsulating ourselves as much as possible in the audit subsystem and not depending on any other subsystem except for the one constant in all our solutions, VFS. If we were given the "go ahead" or some fuzzy feeling that this would be the best approach/attempt, we could implement this feature in the audit subsystem. The disadvantages are fairly clear. If this feature were configured Y, the inode structure would grow in size and if this were an inode field, we'd be keeping a list of "names," on it. We'd also be introducing yet another set of hooks to VFS which have a very specific use. And, on the kernel-side, we'd essentially be duplicating some of inotify's behavior.
However, the most obvious gain here, is that the subsystem is centralized and its intentions are clearer.
5. Kprobes. I know nothing about them. These have not been considered, but were mentioned to me just recently by two different people. I'm not sure if this approach could be useful or not. Any help here?
-------
Your feedback is essential to figuring out what the best way to approach this is. Thanks for reading :-)
Signature · 1 line
- Timothy R. Chavez
Quoted 19 lines · * Timothy Chavez ([email protected]) wrote:
Over the last two months, I've been given the daunting task of implementing a feature by which an administrator can specify from user space, a list of file system objects (namely regular files and directories) that he/she wishes to audit. The administrator would presumably like to,
1. Specify a file, "
/etc/shadow"2. Filter based on some criteria, "I only care if
/etc/shadowis open()'d by uid=X" (Filtering could be more granular then this or less, that's arbitrary at this point)3. Be reassured that one may not bypass the audit subsystem nor lose an audit record.
Assuming we can do the filtering aspect in user space via auditd (or whatever), we've introduced some kernel work. For instance, when we say we want to audit a file or directory, what we're really saying is,
Quoted 8 lines
"I'm interested in knowing when X syscall accesses/alters Y file/directory, so send me a record of this activity."
Some notable design problems to introduce now are with the "identity" of Y file/directory with respect to the kernel. In fact, we only really care about paths of which file/directory Y completes. I.e: If we want to audit
/etc/shadow, we might not care if/etc/shadowis moved to/tmp/shadow. This means that any access/alteration of
Of course you would. A mv to /tmp/shadow includes an unlink, which should be an auditable event.
Quoted 4 lines
/tmp/shadowwould go unnoticed (unless otherwise specified). However, if/tmp/shadowwere a hardlink to/etc/shadow, we would then still care, because we are still effectively/etc/shadow... if that makes sense :-)
Yes, at this point you care about the inode not the pathname (the latter is just a handle to the former).
Quoted 6 lines
These are things that our design must consider in some way.
1. Identifying a file, such as
/etc/shadow, solely by a pathname, enables someone to create a hardlink to/etc/shadowfrom a different directory and by-pass the audit subsystem. Also, we can't assume we have this user space definition of a file/directory in the kernel.
What does that mean, "user space definition of a file/dir in kernel"?
Quoted 4 lines
2. Identifying a file, such as
/etc/shadow, solely by its inode, enables a by-pass of the audit subsystem, when say,/etc/shadowgets copied to/etc/shadow+and moved back. Because the inode has changed, we're no longer the "same" file system object. Similarly, if we
OK, depends on the mechanism (O_TRUNC and write or unlink and rename).
Quoted 4 lines
delete an audited file,
/tmp/ultra_secret(in /tmp? sounds like my old university), and recreate it, we'd like/tmp/ultra_secretto be audited. This falls back on the "we don't want audit records to be lost" requirement.
So the pathname is relative to the parent dir, which needs to know the about these special names.
Quoted 3 lines
3. How do we treat mounts and --bind mounts? I won't go into this in this e-mail. There's several design decisions to made with regards to this issue.
Well, it's an important one, can't just punt on it.
Quoted 3 lines
4. How can we obtain the full path of the file system object without reconstructing it in the kernel? Xattrs? Integer cookies/hashes that key a table in user space?
There's no such thing. How do you store a single path in an xattr (which is on an inode) for an inode that has multiple hardlinks, or multiple vfsmounts (since you can't punt on --bind ;-). The _best_ you could do is go by the path that the user used (audit captures that already IIRC).
Quoted 7 lines
5. Whats the best approach at capturing information about the syscall accessing / altering the file/directory? We might lose records if we attempt to capture this information from VFS hooks as we could short-circuit before we ever reached that point in the function. We'd then need a way of related the audit of a system call that in place now with the file/directory its accessing. (This particiular issue is not talked about in this e-mail in any detail, really)
I think you'd need to do more filtering in userspace, and generate a lot of extra audit records. At each point (audit, and lsm hook) there's not enough context. I can't recall when the record is created, if it's at the syscall exit point, then the hook can tell you if you actually cared about the transaction.
Quoted 19 lines
The Reason:
~~~~~~~~~~~The current audit subsystem does not contain this capability. This capability is essential for meeting Controlled Access Protection Profile (CAPP) Evaluation Assurance Level (EAL) 4+ requirements.
The Solutions:
~~~~~~~~~~~~~~(These are brief notes. More implementation detail can be provided it asked for. If someone is interested in discussing an actual solution, we can get into that - I'd like to get into that. I'm just trying to give you a sense of some of the things that have been thought about).
1. Inotify. Inotify implements a good portion of what we'd like to do and looks like its well on its way into the mainline kernel. What would need to be added to inotify:
Quoted 7 lines
A. A way in which it could, if desired, auto-watch the file/directory associated with an IN_CREATE_FILE/SUBDIR event. This would also require we have some way to communicate , "We want this <file/directory> to be auto-watched" -- There might be an issue here because we're now giving meaning to the file/directory name associated with the inode. We're now saying, "If the inode changes, but the name associated with it is the same, watch the new inode automatically."
Not exactly, you're saying let me know if the name in this dir goes away or gets created. If it's created then you register for all it's access activity. </handwavy>
Quoted 8 lines
B. Inotify would have to be able to notify the audit subsystem of events via a direct call internally, when the event is triggered. You'd have some device with an "audit" flag on it that says, "Hey I'm interested in receiving events on behalf of audit, send them to me, and I'll pass them directly on to the audit subsystem." This would be fairly easy to do, but would undermine the inotify event queuing process because we could not take any chances of losing event records in a queue that's overflowed.
I wonder if you could register with inotify and it could callback to you for the events that you register for? Then you're responsible for dealing with audit subsystem, not inotify. Classic consumer deal, can you add yourself to being an event consumer?
Quoted 11 lines
2. Linux Security Module. This is perhaps the most daunting approach. This solution would allow us to use the
i_securityfield of the inode to do our "book keeping." We'd also have at our disposal the the security hooks already in-place in the VFS. This would allow us to intercept the relevant syscalls via function pointers, to make decisions on what we want to do. Using thei_securityfield to hold our table of auditable files/directories on parent directory inodes, would enable us to ask the question, "Does this file/directory that's just been created beneath me need to be marked as "auditable?" from with in, say,security_inode_post_create().
Quoted 11 lines
The advantages to a module is that we can make this audit option more... optional. We're not introducing any new hooks and we're using the LSM's inode
i_securityfield, too. This keeps the inode structure's size flat and minimizes our impact in the VFS. Disadvantages are that for it to play nicely with other LSMs like SELinux, we'd need the LSM stacker or else we'd need to disable SELinux. We'd also be forcing the question, "Is this an appropriate use of the LSM framework?" Although this feature is definitely security-relevant and related, is what it does an appropriate context for use of the LSM framework? Some would say that the LSM framework is for Access Control only, others would disagree.
It's on the edge. LSM is designed for access control, but does give the hooking needed to do this security relevant operation. We specifically did not include audit functionality into LSM because of the audit needs (syscall centric, early exit points are missed, etc.).
Quoted 14 lines
3. SELinux. This feature is not currently in SELinux (as we've been told), but we've had interest by the NSA in implementing this feature for SELinux. We'd have a similar solution to the LSM, except we'd encapsulate ourselves within SELinux, and use its framework to drive the solution. This might prove less resilient, politically, because we'd be under its umbrella. I'm not clear on the exact implementation of this in SELinux -- It sounds like it'd be a patch to the security server and hooks, the creation of an audit label, and the invention of some additional structure for "book keeping" purposes. Also, there is a general problem with using SELinux specifically with figuring out how to certify with it enabled. There's a lot of complications (mostly unknowns) -- Complications we'd willingly work on, if need be. Obviously design decisions would have to be factor in as well.
Quoted 15 lines
4. Audit subsystem. To me, this is the best solution (maybe even worst solution? we'll see what you have to say), but probably the nastiest one. Let the audit subsystem hook VFS in all the necessary places and give audit an inode field to do the "book keeping." The advantages here is that we're encapsulating ourselves as much as possible in the audit subsystem and not depending on any other subsystem except for the one constant in all our solutions, VFS. If we were given the "go ahead" or some fuzzy feeling that this would be the best approach/attempt, we could implement this feature in the audit subsystem. The disadvantages are fairly clear. If this feature were configured Y, the inode structure would grow in size and if this were an inode field, we'd be keeping a list of "names," on it. We'd also be introducing yet another set of hooks to VFS which have a very specific use. And, on the kernel-side, we'd essentially be duplicating some of inotify's behavior.
It's sounding like allowing the audit subsystem to subscribe to inotify events is the best alternative at first glance.
Quoted 7 lines
However, the most obvious gain here, is that the subsystem is centralized and its intentions are clearer.
5. Kprobes. I know nothing about them. These have not been considered, but were mentioned to me just recently by two different people. I'm not sure if this approach could be useful or not. Any help here?
Well, these just give you primarily ad-hoc hooking.
thanks,
-chris
Signature · 1 line
Linux Security Modules http://lsm.immunix.org http://lsm.bkbits.net
Chris,
Thanks for the response. Reply embedded...
(Also I removed serue from the CC, I guess I copied the wrong address for Serge)
Quoted 25 lines · Chris Wright
Quoted 22 lines · * Timothy Chavez ([email protected]) wrote:
Over the last two months, I've been given the daunting task of implementing a feature by which an administrator can specify from user space, a list of file system objects (namely regular files and directories) that he/she wishes to audit. The administrator would presumably like to,
1. Specify a file, "
/etc/shadow"2. Filter based on some criteria, "I only care if
/etc/shadowis open()'d by uid=X" (Filtering could be more granular then this or less, that's arbitrary at this point)3. Be reassured that one may not bypass the audit subsystem nor lose an audit record.
Assuming we can do the filtering aspect in user space via auditd (or whatever), we've introduced some kernel work. For instance, when we say we want to audit a file or directory, what we're really saying is,
"I'm interested in knowing when X syscall accesses/alters Y file/directory, so send me a record of this activity."
Quoted 8 lines
Quoted 5 lines
Some notable design problems to introduce now are with the "identity" of Y file/directory with respect to the kernel. In fact, we only really care about paths of which file/directory Y completes. I.e: If we want to audit
/etc/shadow, we might not care if/etc/shadowis moved to/tmp/shadow. This means that any access/alteration ofOf course you would. A mv to
/tmp/shadowincludes an unlink, which should be an auditable event.
Yes, you're absolutely right. Anyway, the point I was trying to drive home is that once moved to /tmp/shadow from an auditable path, /etc/shadow, audit records will no longer be generated. If we do mv /tmp/shadow to /etc/shadow then audit will resume on /etc/shadow... onward..
Quoted 8 lines
Quoted 4 lines
/tmp/shadowwould go unnoticed (unless otherwise specified). However, if/tmp/shadowwere a hardlink to/etc/shadow, we would then still care, because we are still effectively/etc/shadow... if that makes sense :-)Yes, at this point you care about the inode not the pathname (the latter is just a handle to the former).
Quoted 8 lines
Quoted 6 lines
These are things that our design must consider in some way.
1. Identifying a file, such as
/etc/shadow, solely by a pathname, enables someone to create a hardlink to/etc/shadowfrom a different directory and by-pass the audit subsystem. Also, we can't assume we have this user space definition of a file/directory in the kernel.What does that mean, "user space definition of a file/dir in kernel"?
Oh, I suppose description would have been a better word then definition. I was trying to say that we can't deal with path names. How we handle a file/directory in the kernel will be with dentry's and inodes. Sorry about that.
Quoted 6 lines
Quoted 4 lines
2. Identifying a file, such as
/etc/shadow, solely by its inode, enables a by-pass of the audit subsystem, when say,/etc/shadowgets copied to/etc/shadow+and moved back. Because the inode has changed, we're no longer the "same" file system object. Similarly, if weOK, depends on the mechanism (
O_TRUNCand write or unlink and rename).
Sure. But the fact remains that if we say, "audit /etc/shadow" that regardless of the underlying inode, relevant access / alterations to /etc/shadow are audited.
Quoted 7 lines
Quoted 4 lines
delete an audited file,
/tmp/ultra_secret(in /tmp? sounds like my old university), and recreate it, we'd like/tmp/ultra_secretto be audited. This falls back on the "we don't want audit records to be lost" requirement.So the pathname is relative to the parent dir, which needs to know the about these special names.
Correct. This is sufficient.
Quoted 5 lines
Quoted 3 lines
3. How do we treat mounts and --bind mounts? I won't go into this in this e-mail. There's several design decisions to made with regards to this issue.
Well, it's an important one, can't just punt on it.
I have carpel tunnel and ADD?? :-) Can we treat the filesystem as flat and just restart the auditd to recaliberate itself? Is that reasonable? I'm not sure. If designed right, I think we could slip by --bind mounts unscathed. If we mount --bind /etc/ /tmp/foo and we do touch /tmp/foo/shadow, shadow will consult the table associated with foo's inode, the same inode of /etc. So, I might be missing something, but this wouldn't give us any problem.
Quoted 8 lines
Quoted 3 lines
4. How can we obtain the full path of the file system object without reconstructing it in the kernel? Xattrs? Integer cookies/hashes that key a table in user space?
There's no such thing. How do you store a single path in an xattr (which is on an inode) for an inode that has multiple hardlinks, or multiple vfsmounts (since you can't punt on --bind ;-). The _best_ you could do is go by the path that the user used (audit captures that already IIRC).
You're right. But to be able to say something about the origin of the audit.... If you're generating audit records for /etc/shadow, which was specifically set "auditable," you at least will be able to get this in the log. The fact that /etc/shadow was accessed/altered is whats really important and not necessarily from where it was accessed/altered, should suffice. Right?
Quoted 8 lines
Quoted 7 lines
5. Whats the best approach at capturing information about the syscall accessing / altering the file/directory? We might lose records if we attempt to capture this information from VFS hooks as we could short-circuit before we ever reached that point in the function. We'd then need a way of related the audit of a system call that in place now with the file/directory its accessing. (This particiular issue is not talked about in this e-mail in any detail, really)
Quoted 5 lines
I think you'd need to do more filtering in userspace, and generate a lot of extra audit records. At each point (audit, and lsm hook) there's not enough context. I can't recall when the record is created, if it's at the syscall exit point, then the hook can tell you if you actually cared about the transaction.
It's probably not wise to collect information in lsm hooks because we could return out of the function before we ever hit the hook (I think you talk about this further down) and we'll miss potentially important audit records. So auditd would just have to piece together the complete message in userspace, based on a unique serial number or something, for all the message that says, "such and such was accessed / altered" with the their corresponding, "by this syscall under these conditions" part.
Quoted 29 lines
Quoted 21 lines
The Reason:
~~~~~~~~~~~The current audit subsystem does not contain this capability. This capability is essential for meeting Controlled Access Protection Profile (CAPP) Evaluation Assurance Level (EAL) 4+ requirements.
The Solutions:
~~~~~~~~~~~~~~(These are brief notes. More implementation detail can be provided it asked for. If someone is interested in discussing an actual solution, we can get into that - I'd like to get into that. I'm just trying to give you a sense of some of the things that have been thought about).
1. Inotify. Inotify implements a good portion of what we'd like to do and looks like its well on its way into the mainline kernel. What would need to be added to inotify:
Quoted 7 lines
A. A way in which it could, if desired, auto-watch the file/directory associated with an IN_CREATE_FILE/SUBDIR event. This would also require we have some way to communicate , "We want this <file/directory> to be auto-watched" -- There might be an issue here because we're now giving meaning to the file/directory name associated with the inode. We're now saying, "If the inode changes, but the name associated with it is the same, watch the new inode automatically."
Quoted 3 lines
Not exactly, you're saying let me know if the name in this dir goes away or gets created. If it's created then you register for all it's access activity. </handwavy>
Yes, but this decision and action would have to be taken in inotify.
Quoted 12 lines
Quoted 11 lines
B. Inotify would have to be able to notify the audit subsystem of events via a direct call internally, when the event is triggered. You'd have some device with an "audit" flag on it that says, "Hey I'm interested in receiving events on behalf of audit, send them to me, and I'll pass them directly on to the audit subsystem." This would be fairly easy to do, but would undermine the inotify event queuing process because we could not take any chances of losing event records in a queue that's overflowed.
Quoted 5 lines
I wonder if you could register with inotify and it could callback to you for the events that you register for? Then you're responsible for dealing with audit subsystem, not inotify. Classic consumer deal, can you add yourself to being an event consumer?
Some way for inotify to "notify" other parts of the kernel of file system activity would be good. This is the arguement I'd like to use.
If inotify can notify userspace apps of activity/events, why can't it
notify kernel subsystems? There might be a very good reason as to why this can't be so. "Just because" might be it :-) Whatever the reason, it'd be good to hear. I wouldn't want to destroy or degrade the intended use of inotify, but expand it. If that's not doable, then there's no way we can use inotify. This would have to be something John and Rob and whoever else contributes to Inotify would like in addition to the community as a whole.
Quoted 60 lines
Quoted 12 lines
2. Linux Security Module. This is perhaps the most daunting approach. This solution would allow us to use the
i_securityfield of the inode to do our "book keeping." We'd also have at our disposal the the security hooks already in-place in the VFS. This would allow us to intercept the relevant syscalls via function pointers, to make decisions on what we want to do. Using thei_securityfield to hold our table of auditable files/directories on parent directory inodes, would enable us to ask the question, "Does this file/directory that's just been created beneath me need to be marked as "auditable?" from with in, say,security_inode_post_create().Quoted 11 lines
The advantages to a module is that we can make this audit option more... optional. We're not introducing any new hooks and we're using the LSM's inode
i_securityfield, too. This keeps the inode structure's size flat and minimizes our impact in the VFS. Disadvantages are that for it to play nicely with other LSMs like SELinux, we'd need the LSM stacker or else we'd need to disable SELinux. We'd also be forcing the question, "Is this an appropriate use of the LSM framework?" Although this feature is definitely security-relevant and related, is what it does an appropriate context for use of the LSM framework? Some would say that the LSM framework is for Access Control only, others would disagree.It's on the edge. LSM is designed for access control, but does give the hooking needed to do this security relevant operation. We specifically did not include audit functionality into LSM because of the audit needs (syscall centric, early exit points are missed, etc.).
Quoted 14 lines
3. SELinux. This feature is not currently in SELinux (as we've been told), but we've had interest by the NSA in implementing this feature for SELinux. We'd have a similar solution to the LSM, except we'd encapsulate ourselves within SELinux, and use its framework to drive the solution. This might prove less resilient, politically, because we'd be under its umbrella. I'm not clear on the exact implementation of this in SELinux -- It sounds like it'd be a patch to the security server and hooks, the creation of an audit label, and the invention of some additional structure for "book keeping" purposes. Also, there is a general problem with using SELinux specifically with figuring out how to certify with it enabled. There's a lot of complications (mostly unknowns) -- Complications we'd willingly work on, if need be. Obviously design decisions would have to be factor in as well.
Quoted 15 lines
4. Audit subsystem. To me, this is the best solution (maybe even worst solution? we'll see what you have to say), but probably the nastiest one. Let the audit subsystem hook VFS in all the necessary places and give audit an inode field to do the "book keeping." The advantages here is that we're encapsulating ourselves as much as possible in the audit subsystem and not depending on any other subsystem except for the one constant in all our solutions, VFS. If we were given the "go ahead" or some fuzzy feeling that this would be the best approach/attempt, we could implement this feature in the audit subsystem. The disadvantages are fairly clear. If this feature were configured Y, the inode structure would grow in size and if this were an inode field, we'd be keeping a list of "names," on it. We'd also be introducing yet another set of hooks to VFS which have a very specific use. And, on the kernel-side, we'd essentially be duplicating some of inotify's behavior.
Quoted 3 lines
It's sounding like allowing the audit subsystem to subscribe to inotify events is the best alternative at first glance.
This would be good. Inotify would need some definate changes to make this work cleanly. Right now its very specific in that it queues events from the VFS to be written to a character device driver. With our method we assume everything being watched on a device with an audit_notify=1 will call the audit_log() function (which writes out to an audit buffer at which point the audit subsystem takes over).
Quoted 15 lines
Quoted 7 lines
However, the most obvious gain here, is that the subsystem is centralized and its intentions are clearer.
5. Kprobes. I know nothing about them. These have not been considered, but were mentioned to me just recently by two different people. I'm not sure if this approach could be useful or not. Any help here?
Well, these just give you primarily ad-hoc hooking.
thanks,
-chris
--
Linux Security Modules http://lsm.immunix.org http://lsm.bkbits.net
Signature · 1 line
- Timothy R. Chavez
On Fri, 2004-12-10 at 02:50 +0000, Timothy Chavez wrote:
Hi, Timothy. You work for IBM?
Quoted 10 lines
Some way for inotify to "notify" other parts of the kernel of file system activity would be good. This is the arguement I'd like to use.
If inotify can notify userspace apps of activity/events, why can't it
notify kernel subsystems? There might be a very good reason as to why this can't be so. "Just because" might be it :-) Whatever the reason, it'd be good to hear. I wouldn't want to destroy or degrade the intended use of inotify, but expand it. If that's not doable, then there's no way we can use inotify. This would have to be something John and Rob and whoever else contributes to Inotify would like in addition to the community as a whole.
I do not think it makes any sense for inotify to be the mechanism that implements auditing. What you want is a general file event mechanism at the level and time that we currently do the inotify hooks. I agree, that is good. What you also want is to do is hack into inotify your auditing code. I don't like that--I don't want inotify to grow into a generic file system tap.
What we both need, ultimately, is a generic file change notification system. This way inotify, dnotify, your audit thing, and whatever else can hook into the filesystem as desired.
Subverting the inotify project to add this functionality now will only hurt inotify. We are not yet in the kernel and we need to streamline and simplify ourselves, not bloat and featurize. Besides, indeed, we are not in the kernel yet--you can just as easily add the hooks yourself.
So my position would be that I am all for moving the inotify hooks to generic file change hooks, but that needs to be done either once inotify is in the kernel proper or as a separate project. Then inotify can be one consumer of the hooks and auditing another.
If you want to move forward with a project to hook file system events, go for it. Regardless, I think that you should post to lkml your intentions.
Best,
Robert Love
Quoted 4 lines · Robert Love
On Fri, 2004-12-10 at 02:50 +0000, Timothy Chavez wrote:
Hi, Timothy. You work for IBM?
Yep.
Quoted 13 lines
Quoted 12 lines
Some way for inotify to "notify" other parts of the kernel of file system activity would be good. This is the arguement I'd like to use.
If inotify can notify userspace apps of activity/events, why can't it
notify kernel subsystems? There might be a very good reason as to why this can't be so. "Just because" might be it :-) Whatever the reason, it'd be good to hear. I wouldn't want to destroy or degrade the intended use of inotify, but expand it. If that's not doable, then there's no way we can use inotify. This would have to be something John and Rob and whoever else contributes to Inotify would like in addition to the community as a whole.
Quoted 6 lines
I do not think it makes any sense for inotify to be the mechanism that implements auditing. What you want is a general file event mechanism at the level and time that we currently do the inotify hooks. I agree, that is good. What you also want is to do is hack into inotify your auditing code. I don't like that--I don't want inotify to grow into a generic file system tap.
Fair enough. I agree that having something more generic at the hook level would be ideal. I'm interested in what others might think about this.
Quoted 9 lines
What we both need, ultimately, is a generic file change notification system. This way inotify, dnotify, your audit thing, and whatever else can hook into the filesystem as desired.
Subverting the inotify project to add this functionality now will only hurt inotify. We are not yet in the kernel and we need to streamline and simplify ourselves, not bloat and featurize. Besides, indeed, we are not in the kernel yet--you can just as easily add the hooks yourself.
Right, but we like inotify and want to see it succeed :-)! We also want an upstream solution, so playing nicely is essential.
Quoted 4 lines
So my position would be that I am all for moving the inotify hooks to generic file change hooks, but that needs to be done either once inotify is in the kernel proper or as a separate project. Then inotify can be one consumer of the hooks and auditing another.
It's a reasonable compromise and it'll have to be considered and discussed.
Quoted 3 lines
If you want to move forward with a project to hook file system events, go for it. Regardless, I think that you should post to lkml your intentions.
Most definately. Thanks for the reply.
Quoted 5 lines
Best,
Robert Love
Signature · 1 line
- Timothy R. Chavez
Quoted 2 lines · Timothy Chavez
Right, but we like inotify and want to see it succeed :-)! We also want an upstream solution, so playing nicely is essential.
Awesome. ;)
I'm not adverse to doing the auditing in a generic hook mechanism, at all, assuming that LSM hooks and the other options are not the preferred and optimal solution.
Quoted 6 lines
Quoted 4 lines
So my position would be that I am all for moving the inotify hooks to generic file change hooks, but that needs to be done either once inotify is in the kernel proper or as a separate project. Then inotify can be one consumer of the hooks and auditing another.
It's a reasonable compromise and it'll have to be considered and discussed.
I've actually been thinking of doing this anyhow, because we currently have both dnotify and inotify hooks in the filesystem code. But one thing at a time.
Robert Love
Quoted 3 lines · * Robert Love ([email protected]) wrote:
What we both need, ultimately, is a generic file change notification system. This way inotify, dnotify, your audit thing, and whatever else can hook into the filesystem as desired.
Yup, makes sense. From my handwavy perspective inotify was the generic file event notifcation mechanism, but I agree with your point.
thanks,
-chris
Signature · 1 line
Linux Security Modules http://lsm.immunix.org http://lsm.bkbits.net
Quoted 9 lines · Robert Love
On Fri, 2004-12-10 at 02:50 +0000, Timothy Chavez wrote:
I do not think it makes any sense for inotify to be the mechanism that implements auditing. What you want is a general file event mechanism at the level and time that we currently do the inotify hooks. I agree, that is good. What you also want is to do is hack into inotify your auditing code. I don't like that--I don't want inotify to grow into a generic file system tap.
Quoted 3 lines
What we both need, ultimately, is a generic file change notification system. This way inotify, dnotify, your audit thing, and whatever else can hook into the filesystem as desired.
Yes, hopefully we can use some inotify bits when it comes time to do it.
Quoted 11 lines
Subverting the inotify project to add this functionality now will only hurt inotify. We are not yet in the kernel and we need to streamline and simplify ourselves, not bloat and featurize. Besides, indeed, we are not in the kernel yet--you can just as easily add the hooks yourself.
So my position would be that I am all for moving the inotify hooks to generic file change hooks, but that needs to be done either once inotify is in the kernel proper or as a separate project. Then inotify can be one consumer of the hooks and auditing another.
I am in complete agreement here. Inotify was not designed with any of this in mind. Inotify was designed with the desktop in mind. I think a more realistic approach would be to get inotify in, then refactor from there. Creating a generic layer that contains some of the inotify code plus whatever is needed to implement this audit stuff. Then the audit system and inotify could use this layer for the nitty gritty.
Signature · 1 line
John McCutchan <[email protected]>
Quoted 8 lines
Quoted 5 lines
Some notable design problems to introduce now are with the "identity" of Y file/directory with respect to the kernel. In fact, we only really care about paths of which file/directory Y completes. I.e: If we want to audit
/etc/shadow, we might not care if/etc/shadowis moved to/tmp/shadow. This means that any access/alteration ofOf course you would. A mv to
/tmp/shadowincludes an unlink, which should be an auditable event.
Does it really? If /etc and /tmp reside on the same filesystem, only a rename() is done as to my knowledge (and possibly, an fs-specific rebalance). And if they are on the same fs, they might even keep the inode number.
Quoted 4 lines
Quoted 4 lines
/tmp/shadowwould go unnoticed (unless otherwise specified). However, if/tmp/shadowwere a hardlink to/etc/shadow, we would then still care, because we are still effectively/etc/shadow... if that makes sense :-)
(Your mentioning of hardlinks proves my assumption of you having /etc and /tmp on the same fs.)
Jan Engelhardt
Signature · 1 line
ENOSPC
Quoted 4 lines · Timothy Chavez
Greetings,
I'm writing this e-mail to facilitate some discussion on an audit feature for inclusion to the mainline kernel's audit subsysystem.
Note, there is a mailing list intended for audit discussion at
https://www.redhat.com/archives/linux-audit/
It's been a bit quiet, but it may be useful for detailed discussions.
Quoted 1 line
2. Linux Security Module.
As Chris said, LSM is not suitable for CAPP auditing.
Quoted 1 line
3. SELinux.
I don't think it's a good idea to tie everyone in to using SELinux. Audit and SELinux do need to play well together though (and SELinux is already integrated with some of the existing audit subsystem).
Quoted 1 line
4. Audit subsystem.
IMHO we need a distinct Audit subsystem, perhaps even something which can accomodate different implementations.
Quoted 2 lines
However, the most obvious gain here, is that the subsystem is centralized and its intentions are clearer.
Indeed.
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 11 lines · Timothy Chavez
2. Filter based on some criteria, "I only care if
/etc/shadowis open()'d by uid=X" (Filtering could be more granular then this or less, that's arbitrary at this point)3. Be reassured that one may not bypass the audit subsystem nor lose an audit record.
Assuming we can do the filtering aspect in user space via auditd (or whatever), we've introduced some kernel work. For instance, when we say we want to audit a file or directory, what we're really saying is,
This could amount to a lot of data being passed from the kernel to user space. To do a complete audit from user space you would need to provide as much context from the kernel as possible.
Quoted 14 lines
"I'm interested in knowing when X syscall accesses/alters Y file/directory, so send me a record of this activity."
Some notable design problems to introduce now are with the "identity" of Y file/directory with respect to the kernel. In fact, we only really care about paths of which file/directory Y completes. I.e: If we want to audit
/etc/shadow, we might not care if/etc/shadowis moved to/tmp/shadow. This means that any access/alteration of/tmp/shadowwould go unnoticed (unless otherwise specified). However, if/tmp/shadowwere a hardlink to/etc/shadow, we would then still care, because we are still effectively/etc/shadow... if that makes sense :-)
Quoted 7 lines
These are things that our design must consider in some way.
1. Identifying a file, such as
/etc/shadow, solely by a pathname, enables someone to create a hardlink to/etc/shadowfrom a different directory and by-pass the audit subsystem. Also, we can't assume we have this user space definition of a file/directory in the kernel.
hardlinks share the same inode.
Quoted 9 lines
2. Identifying a file, such as
/etc/shadow, solely by its inode, enables a by-pass of the audit subsystem, when say,/etc/shadowgets copied to/etc/shadow+and moved back. Because the inode has changed, we're no longer the "same" file system object. Similarly, if we delete an audited file,/tmp/ultra_secret(in /tmp? sounds like my old university), and recreate it, we'd like/tmp/ultra_secretto be audited. This falls back on the "we don't want audit records to be lost" requirement.
Using inotify you can handle this. You watch /tmp for SUBFILE/SUBDIR creations, and then when you see ultra_secret again, start watching it. This is very racy though.
Quoted 3 lines
3. How do we treat mounts and --bind mounts? I won't go into this in this e-mail. There's several design decisions to made with regards to this issue.
More detail please..
Quoted 8 lines
A. A way in which it could, if desired, auto-watch the file/directory associated with an IN_CREATE_FILE/SUBDIR event. This would also require we have some way to communicate , "We want this <file/directory> to be auto-watched" -- There might be an issue here because we're now giving meaning to the file/directory name associated with the inode. We're now saying, "If the inode changes, but the name associated with it is the same, watch the new inode automatically."
Inotify can provide a racy solution to this problem [see above]. I don't know if it could be possible to provide an atomic solution to this problem. Al Viro has lots of nasty things to say about providing any kind of accurate view of the filesystem to user space.
Quoted 8 lines
B. Inotify would have to be able to notify the audit subsystem of events via a direct call internally, when the event is triggered. You'd have some device with an "audit" flag on it that says, "Hey I'm interested in receiving events on behalf of audit, send them to me, and I'll pass them directly on to the audit subsystem." This would be fairly easy to do, but would undermine the inotify event queuing process because we could not take any chances of losing event records in a queue that's overflowed.
Inotify could easily produce events for any part of the system.
This being said, I don't think that inotify should be the solution. Inotify was not designed with any of this in mind. A more generic solution is needed and inotify could sit on top of it.
Quoted 16 lines
4. Audit subsystem. To me, this is the best solution (maybe even worst solution? we'll see what you have to say), but probably the nastiest one. Let the audit subsystem hook VFS in all the necessary places and give audit an inode field to do the "book keeping." The advantages here is that we're encapsulating ourselves as much as possible in the audit subsystem and not depending on any other subsystem except for the one constant in all our solutions, VFS. If we were given the "go ahead" or some fuzzy feeling that this would be the best approach/attempt, we could implement this feature in the audit subsystem. The disadvantages are fairly clear. If this feature were configured Y, the inode structure would grow in size and if this were an inode field, we'd be keeping a list of "names," on it. We'd also be introducing yet another set of hooks to VFS which have a very specific use. And, on the kernel-side, we'd essentially be duplicating some of inotify's behavior.
Quoted 2 lines
However, the most obvious gain here, is that the subsystem is centralized and its intentions are clearer.
This is probably your best course of action. Later on we could merge generic bits from this and inotify.
Signature · 1 line
John McCutchan <[email protected]>
Hello,
You could also have a look at LIDS. Though the target is not the same, some useful idea may be gathered there...
Regards,
Paul
Paul Rolland, rol(at)as2917.net
ex-AS2917 Network administrator and Peering Coordinator
--
Please no HTML, I'm not a browser - Pas d'HTML, je ne suis pas un navigateur
"Some people dream of success... while others wake up and work hard at it"
Quoted 18 lines
-----Message d'origine-----
De : [email protected] [mailto:[email protected]] De la part de Timothy Chavez
Envoyé : vendredi 10 décembre 2004 01:03
À : [email protected]
Cc : [email protected]; [email protected]; [email protected]; [email protected]
Objet : [audit] Upstream solution for auditing file system objectsGreetings,
I'm writing this e-mail to facilitate some discussion on an audit feature for inclusion to the mainline kernel's audit subsysystem. I've written out a "brief" description of the problem with some of the associated problems it introduces and some of the outlines of potential solutions below. For the most part, the idea stays the same, and the mechanism we use to implement it varies.
Quoted 4 lines · Timothy Chavez
Over the last two months, I've been given the daunting task of implementing a feature by which an administrator can specify from user space, a list of file system objects (namely regular files and directories) that he/she wishes to audit.
One *big* question that you don't address at all in your mail:
Do you need *real time* tracking of changes/etc, in which case inotify or something based on it are probably an approach to follow (note that I don't think inotify currently deals with read-only access to a file).
Or do you not care about real-time tracking, but have a requirement to be able to go back and say "Oh, at 9:03:38.99342 last Tuesday, user XYZ tried to open this file" - if that's what you want, you probably want to look at the audit subsystem and its support for syscall auditing.
Quoted 9 lines · On Fri, 10 Dec 2004 09:29:18 -0500, [email protected]…
Quoted 4 lines · Timothy Chavez
Over the last two months, I've been given the daunting task of implementing a feature by which an administrator can specify from user space, a list of file system objects (namely regular files and directories) that he/she wishes to audit.
One *big* question that you don't address at all in your mail:
Quoted 9 lines
Do you need *real time* tracking of changes/etc, in which case inotify or something based on it are probably an approach to follow (note that I don't think inotify currently deals with read-only access to a file).
Or do you not care about real-time tracking, but have a requirement to be able to go back and say "Oh, at 9:03:38.99342 last Tuesday, user XYZ tried to open this file" - if that's what you want, you probably want to look at the audit subsystem and its support for syscall auditing.
Valdis,
Thank you for the reply. There is no need for real-time tracking to meet this requirement. You have the right idea with the timestamped message.
And yes, the audit subsystem that's in existence now is something we want to work with and improve. Like I had mentioned earlier, it could just be a matter of logging the VFS relevant portion ("this <file/directory> was accessed / altered by") with the same serial number as the syscall ("open() called under these <conditions>") and then piecing the entire audit record together in userspace and filter it accordingly.
The main goal here is to not be redundant and make use of our resources. I personally like Robert's idea with the general hooking framework. Making the audit subsystem, itself, modular also sounds like a pretty good idea.
Thanks again!
Signature · 1 line
- Timothy R. Chavez