Add End of Event record

Hi,

This patch adds an end of event record type. It will be sent by the kernel as the last record when a multi-record event is triggered. This will aid realtime analysis programs since they will now reliably know they have the last record to complete an event. The audit daemon filters this and will not write it to disk.

diff -urp linux-2.6.22.x86_64.orig/include/linux/audit.h linux-2.6.22.x86_64/include/linux/audit.h
@@ -96,6 +96,7 @@
 #define AUDIT_FD_PAIR		1317    /* audit record for pipe/socketpair */
 #define AUDIT_OBJ_PID		1318	/* ptrace target */
 #define AUDIT_TTY		1319	/* Input on an administrative TTY */
+#define AUDIT_EOE		1320	/* End of multi-record event */
 
 #define AUDIT_AVC		1400	/* SE Linux avc denial or grant */
 #define AUDIT_SELINUX_ERR	1401	/* Internal SE Linux Errors */

diff -urp linux-2.6.22.x86_64.orig/kernel/auditsc.c linux-2.6.22.x86_64/kernel/auditsc.c
@@ -1117,6 +1117,11 @@ static void audit_log_exit(struct audit_
 
 		audit_log_end(ab);
 	}
+
+	/* Send end of event record to help user space know we are finished */
+	ab = audit_log_start(context, GFP_KERNEL, AUDIT_EOE);
+	if (ab)
+		audit_log_end(ab);
 	if (call_panic)
 		audit_panic("error converting sid to string");
 }

Only other choice seems to be to determine up front how many records we are going to emit, but doing that is horrible compared to this just to save one message. We already will usually have more than one audit record.

First let me say AUDIT_EOE is a good idea and I'm glad it's been added, but I'd like to make a couple of observations to put it context.

How would a program determine AUDIT_EOE might be present in the audit "protocol" since there is no versioning of the "protocol" (using the term protocol loosely here, but in many respects streaming audit data is a protocol). Unless I'm mistaken the only way to determine if the protocol supports AUDIT_EOE is to watch for it and if it appears in the audit stream set a flag indicating event closure is determined by AUDIT_EOE as opposed to a duration interval.

AUDIT_EOE does not make event assembly easier, one must still cache records until the event is complete. The advantage of AUDIT_EOE is explicit knowledge the event is complete as opposed to implicit knowledge the event is complete because a duration interval elapsed. The duration interval must be time based, an interval based on the number of intervening records will fail to close the event if the audit stream becomes quiescent. But since there is no guarantee AUDIT_EOE's will be in the protocol stream one must still be prepared to close events implicitly after a duration interval. Thus the single advantage of AUDIT_EOE's is to expedite the flushing of a complete event earlier than would be possible in the absence of the AUDIT_EOE record, otherwise the complexity of event assembly is unmodified.

I believe the consequences are this:

1) A real time audit parsing library must still support both event closure mechanisms (note, parsing libraries are user space and independent of kernel versions and hosts).

2) The library when it opens an audit stream must start with it's closure mechanism set to "interval".

3) If AUDIT_EOE is seen the library sets it's closure mechanism to "EOE". Closed events will then be emitted earlier than previously.

John Dennis <[email protected]>

Yes.

If you design it so, yes. I'd rather just say its either timing out the connection or when the processed time in the file has elapsed beyond say 2 seconds...

Correct. This is all about speeding up the realtime analysis.

-Steve

John,
I have debated this in my head for a while, especially when I considered writing my own dispatcher. At a high level, it is starting to sound like these topics might be appropriate:
1. A (pseudo-)RFC describing the dispatcher "protocol"
2. A rigid, easily parsed record format - AUDIT_EOE might keep it easy on the reporting subsystem
3. Administrative records are passed, perhaps at dispatchers startup and at the start of a file when rotated, that documents which version of auditd, uname -r, output of gnu_get_libc_version(), and the local system date/time.

The administrative record, when mentioning auditd's version, may even indicate a "backward compatible to version..." so that 1.2.6 might still be able to parse 1.2.12, but as of 1.3 the format changed so backwards compatability is broken.

My goal is this: 3 years from now, an employee is being investigated. The investigator makes me pull up all the raw records from my network and analyze them. Now I've interpreted user names, group names, syscalls, and hostnames during capture (ausearch -i), but if the format changed through the years, I need to have analysis tools be aware of the format. This gets back to a previous posting I did on "Offline audit trail analysis."

Charlie Todd
Ball Aerospace & Technologies Corp.

This message and any enclosures are intended only for the addressee. Please notify the sender by email if you are not the intended recipient. If you are not the intended recipient, you may not use, copy, disclose, or distribute this message or its contents or enclosures to any other person and any such actions may be unlawful. Ball reserves the right to monitor and review all messages and enclosures sent to or from this email address.

What do you need to know? Plugins read from stdin, they must obey SIGHUP and SIGTERM, they must keep stdin empty. They can be sent either string data or a pseudo binary format straight from the audit daemon (I do not recommend that for anyone). The string format follows the guidelines laid out for the audit parser library.

That is defined in the audit parsing library specs (although it needs updating for the node field). As for EOE, I had planned to filter that out when writing to disk as it just wastes disk space. The real issue is that for realtime reactive systems, every second counts. This lets them start to take actions faster. For reporting, all events are on disk and the EOE can be deduced.

The auditd version is already recorded. uname -r output might be nice to add. I don't see why glibc version is needed. It has nothing to do with audit rules or events (the kernel does, though). Local system time is already recorded in every record.

This is unnecessary since the audit daemon takes the event and writes it straight to disk. The audit format should not be changing from name=value in the foreseeable future.

The nice thing is that if anyone wants to do something new, you can write a plugin to disassemble the native event and re-write it into any format you want.

names and groups are the one thing that needs to be thought about.

libaudit has a full table

hostnames are another thing to think about, but shouldn't be as troublesome as users and groups.

-Steve

I updated the DAEMON_START record to be like this:

type=DAEMON_START msg=audit(09/27/2007 13:18:04.858:8081) : auditd start, ver=1.6.3 format=raw kernel=2.6.23-0.202.rc8.fc8 auid=root pid=28173 res=success

So, 1.6.3 and later will have the kernel version & release.

-Steve

FYI: setroubleshoot has been modified to support AUDIT_EOE. If AUDIT_EOE is seen and if the event is one being watched it is immediately flushed.

Also, if AUDIT_EOE is ever seen in the audit stream setroubleshootd then disables it's 2 second timeout used to flush cached pending events. Disabling the timeout should make the powertop folks happy because setroubleshootd will no longer wake up as often.

John Dennis <[email protected]>