need help interpreting ausearch results

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html"
type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar"
type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html
type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Thank you very much for the very valuable help.
Kind regards,
Stefano

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 36 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Quoted 8 lines

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

What's the actual rule? On my system, syscall 88 is either symlink (64 bit) or reboot (32 bit).

Quoted 50 lines · Stefano Schiavi

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 41 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 8 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

Stefano,

I assume your rules are file watches. Yes, on the surface, it looks like a service user, lanogbar, is running a php application which has make the chmod system call making the change to 777 (ie a1=1ff in the second event shown).

You could check the process id's (the one which does the chmod - pid = 8804, or the parent process id - ppid = 27717) although these might be temporary processes ... perhaps you could turn on execution audit

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

perhaps also specific chmods also ...

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

Also, as Peter suggests it does help if you present the output from ausearch -i rather than the raw data from /var/log/audit/audit.log.

Quoted 64 lines · Peter Moody

What's the actual rule? On my system, syscall 88 is either symlink (64 bit) or reboot (32 bit).

Quoted 50 lines · Stefano Schiavi

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 41 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 7 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

Hello and thank you very much for your responses!

You have to forgive me but it's a tad hard for me to follow.

I am not sure I can trace down the process by the pid since the incident happened about 8 days ago. Please correct me if wrong.

I have the following rules setup:

-a exit,always -F dir=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

-a exit,always -F path=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

I only watch directories not files.
Specifically the public_html whcih is the one causing the troubles.

Because I don't have much experience with this, would it be possible for you to suggest me how to change the above rules or what commands to run now?

Apologies for being unable to follow your guidance.

Please let me know anything else I can do.
It would be really great to finally nail down what is causing the website to break out of the blue every now and then.

Thank you again.
Kind regards,
Stefano

Quoted 98 lines · Burn Alting

Stefano,

I assume your rules are file watches. Yes, on the surface, it looks like a service user, lanogbar, is running a php application which has make the chmod system call making the change to 777 (ie a1=1ff in the second event shown).

You could check the process id's (the one which does the chmod - pid = 8804, or the parent process id - ppid = 27717) although these might be temporary processes ... perhaps you could turn on execution audit

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

perhaps also specific chmods also ...

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

Also, as Peter suggests it does help if you present the output from ausearch -i rather than the raw data from /var/log/audit/audit.log.

Quoted 64 lines · Peter Moody

What's the actual rule? On my system, syscall 88 is either symlink (64 bit) or reboot (32 bit).

Quoted 50 lines · Stefano Schiavi

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 41 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 7 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

Stefano,

Just add the lines to /etc/audit/audit.rules after your directory watches

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

Then reload the audit service with
service auditd reload
or
/etc/init.d/auditd reload

At this point you should start seeing more events in your /var/log/audit/audit.log files.

Then monitor your filesystem or audit till you see the 777 change again.

Can you review your web server code? In effect, auditd has already informed you that a php process executed the chmod system call ... the execve monitoring will just help with the arguments to the program and the parent processes (and their arguements).

The bottom line is that you will still need to be able to review your php code.

Rgds
Burn

Quoted 12 lines · Stefano Schiavi

Hello and thank you very much for your responses!

You have to forgive me but it's a tad hard for me to follow.

I am not sure I can trace down the process by the pid since the incident happened about 8 days ago. Please correct me if wrong.

I have the following rules setup:

-a exit,always -F dir=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

Quoted 8 lines

-a exit,always -F path=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

I only watch directories not files.
Specifically the public_html whcih is the one causing the troubles.

Because I don't have much experience with this, would it be possible for you to suggest me how to change the above rules or what commands to run

now?

Quoted 7 lines

Apologies for being unable to follow your guidance.

Please let me know anything else I can do.
It would be really great to finally nail down what is causing the website to break out of the blue every now and then.

Quoted 11 lines

Thank you again.
Kind regards,
Stefano

Quoted 3 lines · Burn Alting

Stefano,

I assume your rules are file watches. Yes, on the surface, it looks

like

Quoted 2 lines

Quoted 2 lines

a service user, lanogbar, is running a php application which has make the chmod system call making the change to 777 (ie a1=1ff in the

second

Quoted 21 lines

Quoted 21 lines

event shown).

You could check the process id's (the one which does the chmod - pid = 8804, or the parent process id - ppid = 27717) although these might be temporary processes ... perhaps you could turn on execution audit

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

perhaps also specific chmods also ...

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

Quoted 8 lines

Quoted 8 lines

Also, as Peter suggests it does help if you present the output from ausearch -i rather than the raw data from /var/log/audit/audit.log.

Quoted 1 line · Peter Moody

What's the actual rule? On my system, syscall 88 is either symlink

(64 bit) or reboot (32 bit).

Quoted 10 lines

Quoted 10 lines

Quoted 10 lines · Stefano Schiavi

Quoted 8 lines

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines · Stefano Schiavi

Quoted 5 lines

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change

the permissions of the

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

public_html directory since we found that once in a while it would

change to 777 out of the

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

blue.

It happened again yesterday and I believe these parts of the log

represent when the issue

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www"

inode=4980752 dev=08:08

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./"

inode=4980737 dev=08:08

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

mode=040711 ouid=501 ogid=501 rdev=00:00 type=PATH msg=audit(1386933561.795:7958476): item=0

name="public_html"

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e

syscall=88 success=yes exit=0

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731

auid=0 uid=501 gid=501

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none)

ses=117304 comm="gtar"

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no

experience parsing the

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08

mode=dir,750 ouid=lanogbar

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) :

cwd=/home/lanogbar/public_html

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) :

arch=x86_64 syscall=chmod

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1

ppid=27717 pid=8804 auid=root

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar

fsuid=lanogbar egid=lanogbar

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php

exe=/usr/bin/php

Quoted 20 lines

Quoted 19 lines

Quoted 18 lines

Quoted 7 lines

Quoted 7 lines

key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 7 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

Stefano,

Just add the lines to /etc/audit/audit.rules after your directory watches

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

Then reload the audit service with
service auditd reload
or
/etc/init.d/auditd reload

At this point you should start seeing more events in your /var/log/audit/audit.log files.

Then monitor your filesystem or audit till you see the 777 change again.

Can you review your web server code? In effect, auditd has already informed you that a php process executed the chmod system call ... the execve monitoring will just help with the arguments to the program and the parent processes (and their arguements).

The bottom line is that you will still need to be able to review your php code.

Rgds
Burn

Quoted 132 lines · Stefano Schiavi

Hello and thank you very much for your responses!

You have to forgive me but it's a tad hard for me to follow.

I am not sure I can trace down the process by the pid since the incident happened about 8 days ago. Please correct me if wrong.

I have the following rules setup:

-a exit,always -F dir=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

-a exit,always -F path=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

I only watch directories not files.
Specifically the public_html whcih is the one causing the troubles.

Because I don't have much experience with this, would it be possible for you to suggest me how to change the above rules or what commands to run now?

Apologies for being unable to follow your guidance.

Please let me know anything else I can do.
It would be really great to finally nail down what is causing the website to break out of the blue every now and then.

Thank you again.
Kind regards,
Stefano

Quoted 97 lines · Burn Alting

Stefano,

I assume your rules are file watches. Yes, on the surface, it looks like a service user, lanogbar, is running a php application which has make the chmod system call making the change to 777 (ie a1=1ff in the second event shown).

You could check the process id's (the one which does the chmod - pid = 8804, or the parent process id - ppid = 27717) although these might be temporary processes ... perhaps you could turn on execution audit

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

perhaps also specific chmods also ...

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

Also, as Peter suggests it does help if you present the output from ausearch -i rather than the raw data from /var/log/audit/audit.log.

Quoted 64 lines · Peter Moody

What's the actual rule? On my system, syscall 88 is either symlink (64 bit) or reboot (32 bit).

Quoted 50 lines · Stefano Schiavi

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 41 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 7 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

Hello Burn,

Thank you so much again.
I have implemented the steps you suggested below.

One question: is there anything you think I can do now to clarify what process triggered the chmod command 9 days ago?

Thank you!
kind regards,
Stefano

Quoted 174 lines · Burn Alting

Stefano,

Just add the lines to /etc/audit/audit.rules after your directory watches

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

Then reload the audit service with
service auditd reload
or
/etc/init.d/auditd reload

At this point you should start seeing more events in your /var/log/audit/audit.log files.

Then monitor your filesystem or audit till you see the 777 change again.

Can you review your web server code? In effect, auditd has already informed you that a php process executed the chmod system call ... the execve monitoring will just help with the arguments to the program and the parent processes (and their arguements).

The bottom line is that you will still need to be able to review your php code.

Rgds
Burn

Quoted 132 lines · Stefano Schiavi

Hello and thank you very much for your responses!

You have to forgive me but it's a tad hard for me to follow.

I am not sure I can trace down the process by the pid since the incident happened about 8 days ago. Please correct me if wrong.

I have the following rules setup:

-a exit,always -F dir=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

-a exit,always -F path=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

I only watch directories not files.
Specifically the public_html whcih is the one causing the troubles.

Because I don't have much experience with this, would it be possible for you to suggest me how to change the above rules or what commands to run now?

Apologies for being unable to follow your guidance.

Please let me know anything else I can do.
It would be really great to finally nail down what is causing the website to break out of the blue every now and then.

Thank you again.
Kind regards,
Stefano

Quoted 97 lines · Burn Alting

Stefano,

I assume your rules are file watches. Yes, on the surface, it looks like a service user, lanogbar, is running a php application which has make the chmod system call making the change to 777 (ie a1=1ff in the second event shown).

You could check the process id's (the one which does the chmod - pid = 8804, or the parent process id - ppid = 27717) although these might be temporary processes ... perhaps you could turn on execution audit

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

perhaps also specific chmods also ...

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

Also, as Peter suggests it does help if you present the output from ausearch -i rather than the raw data from /var/log/audit/audit.log.

Quoted 64 lines · Peter Moody

What's the actual rule? On my system, syscall 88 is either symlink (64 bit) or reboot (32 bit).

Quoted 50 lines · Stefano Schiavi

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 41 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 7 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

Stephano,

You could look at all processes owned by the user lanogbar and then review the code that they are running. I know you mentioned you had root access, but do you also have the actual php code for your web service? If so, you should do a code review on why the service would change directory permissions.

Perhaps look for any cron entries as well that run as lanogbar or spawn off processes for this user.

I'd also double and triple check all configurations for your web server code.

You may also want to research (google is your friend) for any known exploits against your web service. Chmod'ing the root to 777 is not very 'nice'.

Regards
Burn

Quoted 186 lines · Stefano Schiavi

Hello Burn,

Thank you so much again.
I have implemented the steps you suggested below.

One question: is there anything you think I can do now to clarify what process triggered the chmod command 9 days ago?

Thank you!
kind regards,
Stefano

Quoted 173 lines · Burn Alting

Stefano,

Just add the lines to /etc/audit/audit.rules after your directory watches

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

Then reload the audit service with
service auditd reload
or
/etc/init.d/auditd reload

At this point you should start seeing more events in your /var/log/audit/audit.log files.

Then monitor your filesystem or audit till you see the 777 change again.

Can you review your web server code? In effect, auditd has already informed you that a php process executed the chmod system call ... the execve monitoring will just help with the arguments to the program and the parent processes (and their arguements).

The bottom line is that you will still need to be able to review your php code.

Rgds
Burn

Quoted 132 lines · Stefano Schiavi

Hello and thank you very much for your responses!

You have to forgive me but it's a tad hard for me to follow.

I am not sure I can trace down the process by the pid since the incident happened about 8 days ago. Please correct me if wrong.

I have the following rules setup:

-a exit,always -F dir=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

-a exit,always -F path=/home/lanogbar/public_html -F perm=war -F key=lanogbar-pubhtmlwatch

I only watch directories not files.
Specifically the public_html whcih is the one causing the troubles.

Because I don't have much experience with this, would it be possible for you to suggest me how to change the above rules or what commands to run now?

Apologies for being unable to follow your guidance.

Please let me know anything else I can do.
It would be really great to finally nail down what is causing the website to break out of the blue every now and then.

Thank you again.
Kind regards,
Stefano

Quoted 97 lines · Burn Alting

Stefano,

I assume your rules are file watches. Yes, on the surface, it looks like a service user, lanogbar, is running a php application which has make the chmod system call making the change to 777 (ie a1=1ff in the second event shown).

You could check the process id's (the one which does the chmod - pid = 8804, or the parent process id - ppid = 27717) although these might be temporary processes ... perhaps you could turn on execution audit

(quoted diff) +0 -2

-a exit,always -F arch=b32 -S execve -k cmds
-a exit,always -F arch=b64 -S execve -k cmds

perhaps also specific chmods also ...

-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -S chown -S fchown -S fchownat -S lchown -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid!=0 -F auid! =4294967295 -k perm_mod

Also, as Peter suggests it does help if you present the output from ausearch -i rather than the raw data from /var/log/audit/audit.log.

Quoted 64 lines · Peter Moody

What's the actual rule? On my system, syscall 88 is either symlink (64 bit) or reboot (32 bit).

Quoted 50 lines · Stefano Schiavi

Hello,

Could anyone help with this? I really don't know where else to ask.

Thank you very much.
Stefano

Quoted 41 lines · Stefano Schiavi

Hello,

Thank you Steve and all for keeping up the great work here.

Some time ago I setup some audit rules to monitor what would change the permissions of the public_html directory since we found that once in a while it would change to 777 out of the blue.

It happened again yesterday and I believe these parts of the log represent when the issue happened:

type=PATH msg=audit(1386933561.795:7958476): item=2 name="./www" inode=4980752 dev=08:08 mode=0120777 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=1 name="./" inode=4980737 dev=08:08 mode=040711 ouid=501 ogid=501 rdev=00:00
type=PATH msg=audit(1386933561.795:7958476): item=0 name="public_html" type=CWD msg=audit(1386933561.795:7958476): cwd="/home/lanogbar" type=SYSCALL msg=audit(1386933561.795:7958476): arch=c000003e syscall=88 success=yes exit=0 a0=1306d160 a1=1306d200 a2=11 a3=0 items=3 ppid=18728 pid=18731 auid=0 uid=501 gid=501 euid=501 suid=501 fsuid=501 egid=501 sgid=501 fsgid=501 tty=(none) ses=117304 comm="gtar" exe="/bin/tar" key="lanogbar-www"

This is just a guess though and I can not be sure as I have no experience parsing the logs. Looking through with the I flag we can see the following::

type=PATH msg=audit(12/13/2013 15:00:03.759:7970202) : item=0 name=/home/lanogbar/public_html/ inode=4980744 dev=08:08 mode=dir,750 ouid=lanogbar ogid=nobody rdev=00:00
type=CWD msg=audit(12/13/2013 15:00:03.759:7970202) : cwd=/home/lanogbar/public_html type=SYSCALL msg=audit(12/13/2013 15:00:03.759:7970202) : arch=x86_64 syscall=chmod success=yes exit=0 a0=1585e520 a1=1ff a2=2f a3=146c1d40 items=1 ppid=27717 pid=8804 auid=root uid=lanogbar gid=lanogbar euid=lanogbar suid=lanogbar fsuid=lanogbar egid=lanogbar sgid=lanogbar fsgid=lanogbar tty=(none) ses=117304 comm=php exe=/usr/bin/php key=lanogbar-public_html

Do you think this is relevant?
If so it would seem a php script was responsible.

Would you have any suggestion on how to identify the script?

Quoted 7 lines

Quoted 3 lines

Thank you very much for the very valuable help.
Kind regards,
Stefano

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit

--
Linux-audit mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-audit