New ausearch output option & audit viewing in Spacewalk

Hello all.

As part of developing an audit viewing "plugin"[1] to Spacewalk[2], I wrote a small program to use libauparse to output (easily) machine-parsable audit logs. I think this functionality would be nice to have in ausearch, and as such, wrote a patch for it.

As well as reviewing this patch, I would like your feedback concerning the Spacewalk audit plugin. Any questions or constructive criticism is welcome.

Thanks,

Joshua Roys

[1]
https://www.redhat.com/archives/spacewalk-devel/2009-May/msg00121.html
http://www.stl.gtri.gatech.edu/jroys/
[2]
http://www.redhat.com/spacewalk/

index 179e127..2abd039 100644
@@ -44,7 +44,8 @@ extern int event_exit, event_exit_is_set;
 typedef enum { F_BOTH, F_FAILED, F_SUCCESS } failed_t;
 typedef enum { C_NEITHER, C_ADD, C_DEL } conf_act_t;
 typedef enum { S_UNSET=-1, S_FAILED, S_SUCCESS } success_t;
-typedef enum { RPT_RAW, RPT_DEFAULT, RPT_INTERP, RPT_PRETTY } report_t;
+typedef enum { RPT_RAW, RPT_DEFAULT, RPT_INTERP, RPT_PRETTY, RPT_MACHINEINTERP
+} report_t;
 
 extern failed_t event_failed;
 extern conf_act_t event_conf_act;

index 1f64b49..9edd59f 100644
@@ -74,7 +74,7 @@ S_HOSTNAME, S_INTERP, S_INFILE, S_MESSAGE_TYPE, S_PID, S_SYSCALL, S_OSUCCESS,
 S_TIME_END, S_TIME_START, S_TERMINAL, S_ALL_UID, S_EFF_UID, S_UID, S_LOGINID,
 S_VERSION, S_EXACT_MATCH, S_EXECUTABLE, S_CONTEXT, S_SUBJECT, S_OBJECT,
 S_PPID, S_KEY, S_RAW, S_NODE, S_IN_LOGS, S_JUST_ONE, S_SESSION, S_EXIT,
-S_LINEBUFFERED };
+S_LINEBUFFERED, S_MACHINEINTERP };
 
 static struct nv_pair optiontab[] = {
 	{ S_EVENT, "-a" },

@@ -97,6 +97,7 @@ static struct nv_pair optiontab[] = {
 	{ S_HOSTNAME, "--host" },
 	{ S_INTERP, "-i" },
 	{ S_INTERP, "--interpret" },
+	{ S_MACHINEINTERP, "-I" },
 	{ S_INFILE, "-if" },
 	{ S_INFILE, "--input" },
 	{ S_IN_LOGS, "--input-logs" },

@@ -173,6 +174,7 @@ static void usage(void)
 	"\t-h,--help\t\t\thelp\n"
 	"\t-hn,--host <Host Name>\t\tsearch based on remote host name\n"
 	"\t-i,--interpret\t\t\tInterpret results to be human readable\n"
+	"\t-I\t\t\t\tInterpret results to be machine readable\n"
 	"\t-if,--input <Input File name>\tuse this file instead of current logs\n"
 	"\t--input-logs\t\t\tUse the logs even if stdin is a pipe\n"
 	"\t--just-one\t\t\tEmit just one event\n"

@@ -474,6 +476,22 @@ int check_params(int count, char *vars[])
         	                retval = -1;
 			}
 			break;
+		case S_MACHINEINTERP:
+			if (report_format == RPT_DEFAULT)
+				report_format = RPT_MACHINEINTERP;
+			else {
+				fprintf(stderr, 
+					"Conflicting output format %s\n",
+					vars[c]);
+        	                retval = -1;
+			}
+			if (optarg) {
+				fprintf(stderr, 
+					"Argument is NOT required for %s\n",
+					vars[c]);
+        	                retval = -1;
+			}
+			break;
 		case S_INFILE:
 			if (!optarg) {
 				fprintf(stderr, 

Would you please post an example of the output and/or the parsing rules.

Thanks

John Dennis <[email protected]>

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

Basically the following:

type=USER_ACCT
serial=222
seconds=1244481001
milli=141
user pid=24777
uid=root
auid=unset
ses=4294967295
subj=system_u:system_r:crond_t:s0-s0:c0.c1023
op=PAM:accounting
acct=root
exe=/usr/sbin/crond
hostname=?
addr=?
terminal=cron
res=success
----
type=CRED_ACQ
serial=223
seconds=1244481001
milli=141
user pid=24777
uid=root
auid=unset
ses=4294967295
subj=system_u:system_r:crond_t:s0-s0:c0.c1023
op=PAM:setcred
acct=root
exe=/usr/sbin/crond
hostname=?
addr=?
terminal=cron
res=success
----
type=LOGIN
serial=224
seconds=1244481001
milli=141
login pid=24777
uid=root
old auid=unset
new auid=root
old ses=4294967295
new ses=34
----

Simply key=value with all nodes separated by "----\n". I should note that the patch has not been exhaustively tested, it's more of a RFC and also a request for those more knowledgeable of the audit logs to improve upon it.

Thanks,

Joshua Roys

Very interesting work. When you apply this patch and select its output format, what does the output look like?

I think this is a very interesting project. But, I have to admit that I don't use ausearch as the normal presentation program when I'm researching some audit events. For example, a typical investigation may go something like this:

1) you run aureport to see what is going on. hmm...no avcs...but lots of files, therefore you are getting hits on rules. wonder which ones?
2) you run the key report to see what the nature of hits is like. The access key seems to be getting a lot of hits, wonder which files it might be?
3) you run ausearch selecting the access key and pipe that into the file summary report. You notice one file is getting lots of hits. Wonder who is doing it?
4) you run ausearch selecting the access key and the file name and pipe that into the user summary report.
5) you notice its one acct and you wonder what all failures that person has had this session so you re-run the last ausearch command with --just-one so you can find the ses=value. Then you run ausearch --session value --success no and send that to aureport to get an overview of the session.
...

So, I'd recommend adding aureport's main summary and the aureport key summary reports to the output so that you can see if there is any reason to do a deeper investigation.

Interesting work!

-Steve

Hello,

If all you want is machine-readable name=value pairs, auparse provides a C/Python interface. It's not 100% the same as the ausearch output, though.
Mirek

[Steve may want to correct and/or comment about my statements on the parsing logic which is in ausearch.]

The idea is interesting but I think this is the wrong implementation approach, there should only be one library which knows how to read audit data, namely libauparse. The code you've added is duplicating some of the logic in libauparse. If the audit format ever changes (or you have a parsing bug) then this code will break. The fact ausearch has logic in it to parse audit data is historical, at the time ausearch was written libauparse did not exist yet. I believe Steve has said that ausearch needs to be rewritten to layer on top of libauparse.

I'm glad to see the use of "interpret" on the value, this is often valuable, but not always. It's critical for strings. But how about things like uid's? You probably want both the uid number and the name it maps to, perhaps it needs to output both the raw and interpreted values separated by deliminters, or make it an option. I'd rather see a blank line to delimit events rather than "----".

Also, it appears as though you're outputting records and not events (an event is the union of all records with the same ID (node,seconds,milli,serial). It think the output should be coallesced into events.

John Dennis <[email protected]>

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/

Actually, if you look at the test cases in the aupase library, you will see that it basically does the same thing. The core code from the test cases is this:

  do {
          if (auparse_first_record(au) <= 0)
                  exit(1);
          do {
                  const au_event_t *e = auparse_get_timestamp(au);
                  if (e == NULL)
                          exit(1);
                  printf("    event time: %u.%u:%lu, host=%s\n",
                          (unsigned)e->sec, 
                          e->milli, e->serial, e->host ? e->host : "?");
                  auparse_first_field(au);
                  do {
                          printf("        %s=%s (%s)\n",
                                          auparse_get_field_name(au),
                                          auparse_get_field_str(au),
                                          auparse_interpret_field(au));
                  } while (auparse_next_field(au) > 0);
                  printf("\n");
          } while(auparse_next_record(au) > 0);
  } while (auparse_next_event(au) > 0);

One could easily make a single purpose program in probably less that 30 lines of code that reproduces the same output as patching ausearch. The auparse library still can't reconnect interlaced records, but you could init the app with AUSOURCE_DESCRIPTOR as the data source (for stdin) and pipe the ouput of ausearch --raw into the single purpose reformatter.

This is very true. Some day it will be layered on top of auparse.

-Steve