Using script(1) to log all user sessions
Hi list,
We're toying with the idea of using script(1) to log all user sessions towards a collection of systems, as one in a number of metheds to aid debugging, education and auditing.
In order to do so, I have a couple of questions regarding a few extensions we're considering to implement.
Firstly, script(1) is clearly and sanely designed to be invoked from the command line to record a limited portion of a user's session. In order for the user to have the logging started without manual invocation, it may come in handy to support config files, to supply configurable default values for some of the concepts normally passed in the environment or as command line arguments.
Secondly, we're considering to add functionality for script(1) to transmit the session transcript over the network to a collection daemon, to be able to store transcripts from multiple machines on one or more central systems.
This could involve:
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, optionally handling the timing functionality, and finally storing data in the same manner script(1) would.
2. Optionally linking against some crypto library to avoid putting users' console data on the wire in clear text.
Ideally we'd like to upstream any changes we make, and this post serves as an attempt to gauge the mood and ask for guidance as to which direction you guys think we should take our efforts.
Yours,
Signature · 3 lines
Jesper Dahl Nyerup Systems Engineer One.com, [email protected]
Quoted 6 lines
From: Jesper Dahl Nyerup <[email protected]>
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, optionally handling the timing functionality, and finally storing data in the same manner script(1) would.
You might want to consider using syslog(3) for this. Or conversely, if syslog won't work in your environment, that might indicate improvements that are needed in syslog.
Dale
Quoted 10 lines · Dale R. Worley
Quoted 6 lines
From: Jesper Dahl Nyerup <[email protected]>
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, optionally handling the timing functionality, and finally storing data in the same manner script(1) would.
You might want to consider using syslog(3) for this. Or conversely, if syslog won't work in your environment, that might indicate improvements that are needed in syslog.
We considered that, but kind of gave up on the idea.
Syslog is inherently newline delimited, where as the per-update payloads from script(1) are pretty much the opposite - one payload will often contain multiple newlines, others will consist of a single character.
Thus, (de)serializing to and from syslog will probably be more of a hassle than using it will be a benifit.
J.
Signature · 3 lines
Jesper Dahl Nyerup Systems Engineer One.com, [email protected]
Karel,
Quoted 7 lines · Jesper Dahl Nyerup
This could involve:
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, optionally handling the timing functionality, and finally storing data in the same manner script(1) would.
Quoted 6 lines
2. Optionally linking against some crypto library to avoid putting users' console data on the wire in clear text.
Ideally we'd like to upstream any changes we make, and this post serves as an attempt to gauge the mood and ask for guidance as to which direction you guys think we should take our efforts.
Do you think functionality like this belongs in script(1) et al., and do you think you'd accept the patches, if they look sane?
J.
Signature · 3 lines
Jesper Dahl Nyerup Systems Engineer One.com, [email protected]
Quoted 9 lines · Jesper Dahl Nyerup
Hi list,
We're toying with the idea of using script(1) to log all user sessions towards a collection of systems, as one in a number of metheds to aid debugging, education and auditing.
In order to do so, I have a couple of questions regarding a few extensions we're considering to implement.
Quoted 6 lines
Firstly, script(1) is clearly and sanely designed to be invoked from the command line to record a limited portion of a user's session. In order for the user to have the logging started without manual invocation, it may come in handy to support config files, to supply configurable default values for some of the concepts normally passed in the environment or as command line arguments.
You can start it from a script acting as the user shell, through sshd config or one of the shell init scripts. As you already need to start script somehow, those defaults could be passed there, too (although I don't see a problem with supporting config files either).
Quoted 11 lines
Secondly, we're considering to add functionality for script(1) to transmit the session transcript over the network to a collection daemon, to be able to store transcripts from multiple machines on one or more central systems.
This could involve:
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, optionally handling the timing functionality, and finally storing data in the same manner script(1) would.
I'm unsure about this bit. It may be needed. Perhaps a transfer after the session finishes also works.
Quoted 2 lines
2. Optionally linking against some crypto library to avoid putting users' console data on the wire in clear text.
Following unix philosophy, I would try to avoid reinventing crypto into the program, attempting instead to solve the issue by eg. using sftp to transfer the files and/or gpg to encrypt the data.
PS: I expect you are properly warning your users about the fascist-level logging done on your systems.
Quoted 10 lines · Ángel González
Quoted 3 lines · Jesper Dahl Nyerup
[...] support config files, to supply configurable default values for some of the concepts normally passed in the environment or as command line arguments.
You can start it from a script acting as the user shell, through sshd config or one of the shell init scripts. As you already need to start script somehow, those defaults could be passed there, too (although I don't see a problem with supporting config files either).
Yes, this is how we have it working in our current test deployments, but for simplicity we'd like to leave out these steps, and ideally have users' shell to be /usr/bin/script.
Quoted 5 lines
Quoted 2 lines
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, [...]
I'm unsure about this bit. It may be needed. Perhaps a transfer after the session finishes also works.
We also considered that, but we keep running in to theoretical corner cases where this could end up being a problem - logging in on systems with filled up mountpoints, securing the transcript even if the system crashes, and so on.
Quoted 6 lines
Quoted 2 lines
2. Optionally linking against some crypto library to avoid putting users' console data on the wire in clear text.
Following unix philosophy, I would try to avoid reinventing crypto into the program, attempting instead to solve the issue by eg. using sftp to transfer the files and/or gpg to encrypt the data.
I fully agree with this concern. This is just a necessary follow up, if we want to natively enable network support in script(1).
However as both you and others have suggested us to reconsider this, and as we also were pretty doubtful about this ourselves, we will probably find an alternative transport method, one way or the other.
Quoted 2 lines
PS: I expect you are properly warning your users about the fascist-level logging done on your systems.
I appreciate your concern for our users.
These users are myself as well as my colleagues, and we all have a shared interest in maintaining audit trails and tracebacks of who did what, when and where.
I can assure you that everyone are aware of these measures.
Yours,
Signature · 3 lines
Jesper Dahl Nyerup Systems Engineer One.com, [email protected]
Karel Zak
maintainerQuoted 10 lines · Jesper Dahl Nyerup
Quoted 5 lines
Quoted 2 lines
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, [...]
I'm unsure about this bit. It may be needed. Perhaps a transfer after the session finishes also works.
We also considered that, but we keep running in to theoretical corner cases where this could end up being a problem - logging in on systems with filled up mountpoints, securing the transcript even if the system crashes, and so on.
You don't have to transfer the logs after finished session, but in small segments (script(1) only add new entries to the file). The same idea uses large databases for on-line backups
(http://www.postgresql.org/docs/8.1/static/backup-online.html)
The another (IMHO the best) possibility is to log into systemd journal, it supports all necessary advantages. See
http://www.freedesktop.org/software/systemd/man/sd-journal.html
for example sd_journal_stream_fd(). I'm not sure, but it's probably already possible to somehow mirror the journal over network or log to remote machine.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 12 lines · Karel Zak
Quoted 10 lines · Jesper Dahl Nyerup
Quoted 5 lines
Quoted 2 lines
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, [...]
I'm unsure about this bit. It may be needed. Perhaps a transfer after the session finishes also works.
We also considered that, but we keep running in to theoretical corner cases where this could end up being a problem - logging in on systems with filled up mountpoints, securing the transcript even if the system crashes, and so on.
Quoted 4 lines
You don't have to transfer the logs after finished session, but in small segments (script(1) only add new entries to the file). The same idea uses large databases for on-line backups
(http://www.postgresql.org/docs/8.1/static/backup-online.html)
Good idea. But then we might as well pipe the output directly to some rsh, avoiding the data to touch storage in the first place.
Quoted 8 lines
The another (IMHO the best) possibility is to log into systemd journal, it supports all necessary advantages. See
http://www.freedesktop.org/software/systemd/man/sd-journal.html
for example
sd_journal_stream_fd(). I'm not sure, but it's probably already possible to somehow mirror the journal over network or log to remote machine.
That's a good idea – we'll look into that. Sadly some of the systems where we'd want this won't be running systemd for the next few years.
Yours,
Signature · 3 lines
Jesper Dahl Nyerup Systems Engineer One.com, [email protected]
Karel Zak
maintainerQuoted 1 line · Jesper Dahl Nyerup
Firstly, script(1) is clearly and sanely designed to be invoked from the
Well, from my point of view it current code is a bit fragile regards to signals. Maybe it would be nice to use signalfd() and check for signals together with changes on master/slave file descriptors by select/poll. But this is not so important for your plans.
Quoted 5 lines
command line to record a limited portion of a user's session. In order for the user to have the logging started without manual invocation, it may come in handy to support config files, to supply configurable default values for some of the concepts normally passed in the environment or as command line arguments.
OK.
Quoted 4 lines
Secondly, we're considering to add functionality for script(1) to transmit the session transcript over the network to a collection daemon, to be able to store transcripts from multiple machines on one or more central systems.
Hmm.. "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can."
I like git concept: here are files and it's your problem to transfer the staff over the network. You can use rsync, http, ssh, ...
Maybe all you need is to store timing and typescript data to the one place (e.g. /var/log/typescripts/user/<timestamp>.{session,timing}) and use ssh or so to transfer the data to another place.
Quoted 4 lines
1. Adding a daemon next to script(1) and scriptreplay(1), eg. scriptcollect(1), to be in the receiving end of the traffic, optionally handling the timing functionality, and finally storing data in the same manner script(1) would.
Seem like overkill.
Quoted 2 lines
2. Optionally linking against some crypto library to avoid putting users' console data on the wire in clear text.
Yep, but we have generic solutions like ssh, again see git.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 6 lines · Karel Zak
Quoted 3 lines · Jesper Dahl Nyerup
[...] support config files, to supply configurable default values for some of the concepts normally passed in the environment or as command line arguments.
OK.
We will produce a patch for this, and submit it fow review.
Quoted 8 lines
Quoted 4 lines
Secondly, we're considering to add functionality for script(1) to transmit the session transcript over the network to a collection daemon, to be able to store transcripts from multiple machines on one or more central systems.
Hmm.. "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can."
Quoted 6 lines
I like git concept: here are files and it's your problem to transfer the staff over the network. You can use rsync, http, ssh, ...
Maybe all you need is to store timing and typescript data to the one place (e.g. /var/log/typescripts/user/<timestamp>.{session,timing}) and use ssh or so to transfer the data to another place.
I see your point.
Storing the files locally before transferring them away has a number of drawbacks, for instance logging in on systems with filled up mountpoints, data loss during crashes on systems without persistent storage, tampering, and so on.
We'll probably end up piping the transcript data to ssh(1), and have something collecting the data remotely. One of the challenges is to ensure this transport mechanism doesn't end up blocking the user's terminal session, in case of network problems or other offload difficulties.
Thank you all for you input.
Yours,
Signature · 3 lines
Jesper Dahl Nyerup Systems Engineer One.com, [email protected]