New List Member: Intro & comments

Hi,
I'm primarily an applications developer, though I wear a
sys-admin/security-admin hat when necessary. I've done kernel hacking on V-7 and System-3 in the bad old days, but mostly stick in userland now.

My current goal is to get some OpenSuse 10.1 systems DSS certified.
I've been a RedHat user since about RH4 or 5, but this project decided to standardize on SuSE. My focus for using auditd isn't so much to make a truly secure and well audited system, but to answer the specific DSS/NISPOM Chapter 8 requirements.

SuSE supports AppArmor and the auditd security products, but has
very little SELinux support.

As I read things, AppArmor doesn't support the file audit
requirements, but Auditd can meet the DSS requirements.

I've updated the standard SuSE 10.1 kernel to 2.6.17.6 and brought in
the 3.0 glibc kernel headers. With these hacks, I can get the 1.2.5 auditd package to compile and install. It appears to be working.

Auditd 1.2.5 doesn't quite do what I need, but I'm getting close.
It has the framework, but it seems to take a little work to get the answers I really want, and to handle the requirements for record maintenance.

1) Auditing and reporting

I've copied the SYSCALL rules from the capp.rules sample, and I
think that covers what DSS will need audited. (Still checking and confirming that I haven't missed anything obvious.)

To get the answers I wanted a bit more easily, I've made a GUI based
search tool that lets me specify reports with a bit more precision and build customized reports. It's functional code, but is not pretty, and gets pretty ugly when the data is way outside what I expected.

2) Maintaining records

The traditional log-rotate with N logs makes it difficult to keep X
days of logs. When the system is busy, I can rotate the logs every 10 minutes.

I've put together a small cron job that looks for audit.log.1,
filters out some data I know I won't want, and zips it into a file with a name based on the timestamp.

My current report generator builds an SQLite database on the fly from
the flat ASCII logs. I'm thinking that the next rev of the file rotation code will move the data to an SQLite database instead of gzipped flat files and save that step.

I'm using SQLite instead of mySQL or Postgres because it it's fast,
mature and robust and doesn't require any database server (or dbadmin) to run it.

I put together a small audisp test application to read from stdin and
save data in a timestamped file. When I run this, I get nothing but empty reads, and finally an EOF from auditd. I'm expecting to see plain ASCII input.

Is this not what is sent to the audisp target?

I just tried the sample.c application, compiled it to a.out and put
that into the auditd.conf file. When I restart audispd, I see no output in /var/log/messages, and a.out does not show in the process stack. If I just run /tmp/a.out and type something, output appears in /var/log/messages.

If any of this is of interest or use, let me know, and I'll make it
available to the community.

Thanks,
Clif

.... Clif Flynt ... http://www.cflynt.com ... [email protected] ...
.. Tcl/Tk: A Developer's Guide (2nd edition) - Morgan Kauffman ..
..13th Annual Tcl/Tk Conference:  Oct 9-13, 2006,  Chicago, IL ..
.............  http://www.tcl.tk/community/tcl2006/  ............

Its not the responsibility of the access control mechanism to support audit. Audit has its own kernel patches for this. I don't know if Suse picked up those patches since yhey are in the upcoming 2.6.18 kernel.

What doesn't it do?

Long term, I want to natively support compressed logs by linking to gzip library and using it.

The 2.6.17 kernel lets you delete certain message types in the kermel. You would do something like:

-a always,exclude -F msgtype=login

This will filter all those message types in the kernel so they never make it to the logs.

Seems like this would be ideal to marry to the realtime audit event interface. You would set log_format = nolog, dispatcher = /sbin/your-dispatcher, and disp_qos = lossless to keep the audit system from writing to disk, send events to a program, and use blocking comminucation to do it.

I've been looking at using it too. I read some issues that made me wonder if it was really suitable:

http://www.sqlite.org/whentouse.html

At the bottom it mentions that if something has the database open for read, then writing is blocked. And the issue about the journal using 256 bytes for event MB of data made me wonder also.

I agree that this fits a database model and have been working to normalize the data so we can actually do this. I think all that's left to do is work on the avc messages since they seem to be overloaded.

Yes, you should.

No, real data is sent. The descriptor is likely to be non-blocking so that the audit deamon doesn't hang up when the buffers are full. So, you need to select on the descriptor.

skeleton.c? That should work fine.

Hmm strange. Works on FC6 machine.

The GUI based search tool might be nice for people to use. I plan to write one in the future, but its lower on the priority list right now. But I think people would like to try out your tool.

-Steve

We will likely do this later (with a SLES 10 Service Pack) to reach CAPP compliance.

Ciao, Marcus

My other reason for using SQLite is that I'm working with the
developer. I forwarded your concerns to Richard Hipp, and received this response:

Clif

.... Clif Flynt ... http://www.cflynt.com ... [email protected] ...
.. Tcl/Tk: A Developer's Guide (2nd edition) - Morgan Kauffman ..
..13th Annual Tcl/Tk Conference:  Oct 9-13, 2006,  Chicago, IL ..
.............  http://www.tcl.tk/community/tcl2006/  ............

The issue here is that you may need write to have priority since there is potentially a backlog building in the kernel. If the backlog gets too big you will get a panic. The write operation has to be fast.

But suppose there are readers. Does the write block or fail? More of a curiosity to me.

Database could be huge. A paranoid admin could easily get gigabytes of data in a short time. Or maybe someone that wrote a rule that captures too much data could run into a problem.

This is good to know. That would probably help.

-Steve

Should the following work???

auditctl -a exit,always -S all -F exit=-13

When I use a negative value for exit, I get no output into the logs when I should.
I am using audit-1.2.3 on SuSE Enterprise 10 with the 2.6.16.21 kernel.

Thanks,
Lane

Yes.

If this does not work, we will need a kernel patch for it.

-Steve

Hi Steve,

I tried it on Fedora with audit 1.2.4 and the 2.6.17-based lspp.41 kernel and it seems to work there.

It doesn't work on RHEL4 U2. I seem to recall that there was something funky about how to get failed syscalls back then but I don't recall the details.

-- ljk

Right, but that was not fixed due to audit system being under certification and a workaround was available and documented. At this point, the upstream kernel *should* work correctly or we forgot to fix it.

-Steve

What do the audit records look like that you expect to be matching, and what architecture are you running on? I recall a bug on ia64 where failed system calls were being audited with "success=yes" and the positive errno, and a patch to change that to negative errno to be consistent with other architectures.

Cf.:

	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=173500

which claims to be fixed by:

	http://rhn.redhat.com/errata/RHSA-2006-0132.html

-Klaus

The records that I care about are the permission denied records. If I do...

auditctl -a exit,always -S all -F success=0

I get ....

----
type=PATH msg=audit(08/03/06 08:49:37.229:78293) : item=0 name=/var/log/messages flags=follow,open inode=53150921 dev=08:03 mode=file,640 ouid=root ogid=root rdev=00:00
type=CWD msg=audit(08/03/06 08:49:37.229:78293) : cwd=/home/someuser
type=SYSCALL msg=audit(08/03/06 08:49:37.229:78293) : arch=x86_64 syscall=open success=no exit=-13(Permission denied) a0=7ffff362f541 a1=0 a2=1b6 a3=0 items=1 pid=6334 auid=unknown(4294967295) uid=someuser gid=users euid=someuser suid=someuser fsuid=someuser egid=users sgid=users fsgid=users comm=more exe=/bin/more
----

but, I also get a lot of other garbage that I do not want.....such as all of the "exit=-2(No such file or directory)".

I would like to....

auditctl -a exit,always -S all -F exit=-13

so I only get permission denied entries. Auditctl allows me to create the rule, and it list the rule. But nothing is logged, when I know it should be.

I am running the 2.6.16.21 kernel (SUSE Enterprise Desktop 10) on AMD64 dual core machines.

Lane

-----Original Message-----
From: Klaus Weidner [mailto:[email protected]]
Sent: Wed 8/2/2006 8:22 PM
To: Williams, P. Lane
Cc: [email protected]
Subject: Re: auditctl question

What do the audit records look like that you expect to be matching, and what architecture are you running on? I recall a bug on ia64 where failed system calls were being audited with "success=yes" and the positive errno, and a patch to change that to negative errno to be consistent with other architectures.

Cf.:

	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=173500

which claims to be fixed by:

	http://rhn.redhat.com/errata/RHSA-2006-0132.html

-Klaus

This is from "ausearch -i"? The raw audit log shouldn't have the "(Permission denied)" part in it, but apart from that it seems that the kernel is auditing things correctly and this is unrelated to the bug I had referred to.

This kernel has a snapshot of the audit code that was in development at the time. Can you please try with a newer upstream kernel and/or bug SUSE to incorporate the current audit fixes in an update?

-Klaus

OK.
I installed the 2.6.17.7 kernel and then tried to build audit-1.2.5 and received the following...

make[2]: Entering directory `/tmp/audit/audit-1.2.5/src'

gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib   -fPIE -DPIE -g -D_REENTRANT -D_GNU_SOURCE -g -O2 -c -o auditd-auditd.o `test -f 'auditd.c' || echo './'`auditd.c
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib   -fPIE -DPIE -g -D_REENTRANT -D_GNU_SOURCE -g -O2 -c -o auditd-auditd-event.o `test -f 'auditd-event.c' || echo './'`auditd-event.c
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib   -fPIE -DPIE -g -D_REENTRANT -D_GNU_SOURCE -g -O2 -c -o auditd-auditd-config.o `test -f 'auditd-config.c' || echo './'`auditd-config.c
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib   -fPIE -DPIE -g -D_REENTRANT -D_GNU_SOURCE -g -O2 -c -o auditd-auditd-reconfig.o `test -f 'auditd-reconfig.c' || echo './'`auditd-reconfig.c
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib   -fPIE -DPIE -g -D_REENTRANT -D_GNU_SOURCE -g -O2 -c -o auditd-auditd-sendmail.o `test -f 'auditd-sendmail.c' || echo './'`auditd-sendmail.c
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib   -fPIE -DPIE -g -D_REENTRANT -D_GNU_SOURCE -g -O2 -c -o auditd-auditd-dispatch.o `test -f 'auditd-dispatch.c' || echo './'`auditd-dispatch.c

/bin/sh ../libtool --tag=CC --mode=link gcc -D_REENTRANT -D_GNU_SOURCE -g -O2 -o auditd -pie -Wl,-z,relro auditd-auditd.o auditd-auditd-event.o auditd-auditd-config.o auditd-auditd-reconfig.o auditd-auditd-sendmail.o auditd-auditd-dispatch.o -lpthread -Lmt -lauditmt
mkdir .libs
gcc -D_REENTRANT -D_GNU_SOURCE -g -O2 -o auditd -pie -Wl,-z -Wl,relro auditd-auditd.o auditd-auditd-event.o auditd-auditd-config.o auditd-auditd-reconfig.o auditd-auditd-sendmail.o auditd-auditd-dispatch.o -lpthread -L/tmp/audit/audit-1.2.5/src/mt -lauditmt
gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -I../lib -D_REENTRANT -D_GNU_SOURCE -g -O2 -c auditctl.c
auditctl.c: In function ‘audit_print_reply’:
auditctl.c:1046: error: ‘AUDIT_SE_USER’ undeclared (first use in this function)
auditctl.c:1046: error: (Each undeclared identifier is reported only once
auditctl.c:1046: error: for each function it appears in.)
auditctl.c:1047: error: ‘AUDIT_SE_CLR’ undeclared (first use in this function)
make[2]: *** [auditctl.o] Error 1
make[2]: Leaving directory `/tmp/audit/audit-1.2.5/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/tmp/audit/audit-1.2.5'
make: *** [all] Error 2

I also received the same error with the other kernel. I did not build the SE-Linux stuff into the kernel, should I have?

thanks,

Lane

-----Original Message-----
From: Klaus Weidner [mailto:[email protected]]
Sent: Thu 8/3/2006 11:18 AM
To: Williams, P. Lane
Cc: [email protected]
Subject: Re: auditctl question

This is from "ausearch -i"? The raw audit log shouldn't have the "(Permission denied)" part in it, but apart from that it seems that the kernel is auditing things correctly and this is unrelated to the bug I had referred to.

This kernel has a snapshot of the audit code that was in development at the time. Can you please try with a newer upstream kernel and/or bug SUSE to incorporate the current audit fixes in an update?

-Klaus

Audit should not require SELinux, but I don't think it currently gets much testing with SELinux turned off which can cause such build issues. Steve, do you have a fix for this?

-Klaus

I think the kernel headers are your problem. I don't think I'd replace that. And you do not have to have SE Linux enabled since this is a define from the audit kernel headers. That said, I don't think you need to rebuild the user space package.

The problem you are seeing would be a kernel bug most likely. It is what evaluates the rules and decides if it needs to output an event. So, with the new kernel and auditctl from Suse, do you see the problem?

Thanks,
-Steve

I built the kernel from the Linux Kernel Archives, not SuSE. I got the latest audit source from the audit-1.2.x site. The audit software seems to compile fine upto audit-1.2.3, but it fails with the compile errors as described earlier, with versions audit-1.2.4 upwards. Is audit being developed strictly under the Red Hat build? Do I need some package that I am missing?

Thanks,
Lane

-----Original Message-----
From: Steve Grubb [mailto:[email protected]]
Sent: Thu 8/3/2006 5:06 PM
To: [email protected]
Cc: Williams, P. Lane; Klaus Weidner
Subject: Re: auditctl question

I think the kernel headers are your problem. I don't think I'd replace that. And you do not have to have SE Linux enabled since this is a define from the audit kernel headers. That said, I don't think you need to rebuild the user space package.

The problem you are seeing would be a kernel bug most likely. It is what evaluates the rules and decides if it needs to output an event. So, with the new kernel and auditctl from Suse, do you see the problem?

Thanks,
-Steve

Yes.

Well, /usr/include/linux/audit.h is the only other thing that would be needed. That is in the glibc-kernheaders package. I do not recommend a wholesale replacement of header files between distros.

-Steve

Maybe next time I will try everything, before I speak.....probably not. Well I recompiled the audit-1.2.3 with the new kernel and it works as it should. The newer audit releases still do not compile, but I am almost certain it is the fact that I did not compile in the SE-Linux support. audit.h does mention something about SE-Linux, when dealing with those includes.

Thanks for the help,
Lane

Hi,
I just talked the audit/sqlite stuff over with Richard Hipp.

One approach would be to have the audisp application put the data
into a TEMP table, then dump those tables to the main disk tables at some interval. After the data is copied to the permanent tables, the TEMP table is cleared.

The TEMP tables resides in memory, and are private to the application
- there can be no contention problems putting the data into those tables.

If writing the temp table to the main table fails because the db is
locked, the temp tables don't get cleared and there is just that much more to be dumped when the disk is available.

My experience with SQLite is that dumping multiple records is nearly
as fast as dumping one. It's a K+N*x type situation where N is the number of records, K is a "constant", and x is much smaller than K. I think the main contributor to K is the disk latency time, while the main contributor to x is memory access time.

When it comes to someone gathering so much audit data that the
logging routines get swamped, he agrees that dumping to a flat file is the fastest way to get the data out of memory and onto the disk.

He thinks that my current approach of scanning the flat files and
building a one-time database may be the best way to avoid dragging down the kernel.

The compromise might be to keep a permanent database, and just update
it with the new records when the reporter is run, or from a cron job, rather than needing to include all records since the beginning of time (or when you want to start generating a report from.)

The way write/read contention problems is handled is that the write
will block for up to N (configurable) milliseconds, and then will either succeed or fail and return a failure message.

Clif

.... Clif Flynt ... http://www.cflynt.com ... [email protected] ...
.. Tcl/Tk: A Developer's Guide (2nd edition) - Morgan Kauffman ..
..13th Annual Tcl/Tk Conference:  Oct 9-13, 2006,  Chicago, IL ..
.............  http://www.tcl.tk/community/tcl2006/  ............