proc connector: add event for process becoming session leader
Subsystems (3)
PATCH 1/1 proc connector: add event for process becoming session leader
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
The poster-child example is OpenSSH; the per-login children call setsid() so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.
This patch adds a new PROC_SID_EVENT and associated structure to the proc_event event_data union, it arranges for this to be emitted when the special PIDTYPE_SID pid is set.
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)
drivers/connector/cn_proc.c +25 -0
@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task) +{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data; +get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid; + ev->event_data.sid.process_tgid = task->tgid; + +memcpy (&msg->id, &cn_proc_event_id,sizeof (msg->id)); + msg->ack =0 ;/* not used */ + msg->len =sizeof (*ev); +cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); +} +void proc_exit_connector (struct task_struct *task) {struct cn_msg *msg;
include/linux/cn_proc.h +10 -0
@@ -52,6 +52,7 @@ struct proc_event {PROC_EVENT_EXEC =0x00000002 ,PROC_EVENT_UID =0x00000004 ,PROC_EVENT_GID =0x00000040 , +PROC_EVENT_SID =0x00000080 ,/* "next" should be 0x00000400 */ /* "last" is the last process event: exit */ PROC_EVENT_EXIT =0x80000000
@@ -89,6 +90,11 @@ struct proc_event { } e; } id; +struct sid_proc_event { +__kernel_pid_t process_pid; +__kernel_pid_t process_tgid; + } sid; +struct exit_proc_event {__kernel_pid_t process_pid;__kernel_pid_t process_tgid;
@@ -102,6 +108,7 @@ struct proc_event {void proc_fork_connector (struct task_struct *task);void proc_exec_connector (struct task_struct *task);void proc_id_connector (struct task_struct *task,int which_id); +void proc_sid_connector (struct task_struct *task);void proc_exit_connector (struct task_struct *task);#else static inline void proc_fork_connector (struct task_struct *task)
@@ -114,6 +121,9 @@ static inline void proc_id_connector(struct task_struct *task, int which_id) {} +static inline void proc_sid_connector (struct task_struct *task) +{} +static inline void proc_exit_connector (struct task_struct *task) {}#endif /* CONFIG_PROC_EVENTS */
kernel/exit.c +3 -1
@@ -360,8 +360,10 @@ void __set_special_pids(struct pid *pid) {struct task_struct *curr = current->group_leader; -if (task_session (curr) != pid) +if (task_session (curr) != pid) {change_pid (curr,PIDTYPE_SID , pid); +proc_sid_connector (curr); + }if (task_pgrp (curr) != pid)change_pid (curr,PIDTYPE_PGID , pid);
Signature · 1 line
1.6.0.5
Let's cc the Process Events developer..
Quoted 8 lines · On Mon, 15 Jun 2009 13:03:08 +0100 Scott James Remnant <[email protected]> wrote:
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
Quoted 9 lines
The poster-child example is OpenSSH; the per-login children call
setsid()so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.This patch adds a new
PROC_SID_EVENTand associated structure to theproc_eventevent_dataunion, it arranges for this to be emitted when the specialPIDTYPE_SIDpid is set.
hm, well, I don't have much useful to say about the overall idea, but it seems to slot into the existing code simply enough.
Quoted 5 lines
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)
We seem to have forgotten to document this entire interface, so I can't ding you for forgetting to update the forgotten documentation.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 142
Quoted 9 lines
drivers/connector/cn_proc.c +1
-0@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task)
It would be nice to have a nice comment explaining what this function does. Ditto all the others in there, really.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 153
Quoted 11 lines
drivers/connector/cn_proc.c +11
-0+{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data;
Please pass all patches through scripts/checkpatch.pl.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 158
Quoted 5 lines
drivers/connector/cn_proc.c +5
-0+get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid;
This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
Quoted 69 lines
drivers/connector/cn_proc.c +8
-0+ ev->event_data.sid.process_tgid = task->tgid; + +memcpy (&msg->id, &cn_proc_event_id,sizeof (msg->id)); + msg->ack =0 ;/* not used */ + msg->len =sizeof (*ev); +cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); +} +void proc_exit_connector (struct task_struct *task) {struct cn_msg *msg;include/linux/cn_proc.h +10
-0@@ -52,6 +52,7 @@ struct proc_event {PROC_EVENT_EXEC =0x00000002 ,PROC_EVENT_UID =0x00000004 ,PROC_EVENT_GID =0x00000040 , +PROC_EVENT_SID =0x00000080 ,/* "next" should be 0x00000400 */ /* "last" is the last process event: exit */ PROC_EVENT_EXIT =0x80000000 @@ -89,6 +90,11 @@ struct proc_event { } e; } id; +struct sid_proc_event { +__kernel_pid_t process_pid; +__kernel_pid_t process_tgid; + } sid; +struct exit_proc_event {__kernel_pid_t process_pid;__kernel_pid_t process_tgid;@@ -102,6 +108,7 @@ struct proc_event {void proc_fork_connector (struct task_struct *task);void proc_exec_connector (struct task_struct *task);void proc_id_connector (struct task_struct *task,int which_id); +void proc_sid_connector (struct task_struct *task);void proc_exit_connector (struct task_struct *task);#else static inline void proc_fork_connector (struct task_struct *task)@@ -114,6 +121,9 @@ static inline void proc_id_connector(struct task_struct *task, int which_id) {} +static inline void proc_sid_connector (struct task_struct *task) +{} +static inline void proc_exit_connector (struct task_struct *task) {}#endif /* CONFIG_PROC_EVENTS */ kernel/exit.c +3
-1@@ -360,8 +360,10 @@ void __set_special_pids(struct pid *pid) {struct task_struct *curr = current->group_leader; -if (task_session (curr) != pid) +if (task_session (curr) != pid) {change_pid (curr,PIDTYPE_SID , pid); +proc_sid_connector (curr); + }if (task_pgrp (curr) != pid)change_pid (curr,PIDTYPE_PGID , pid);
Quoted 2 lines · Andrew Morton
Let's cc the Process Events developer..
Ah great, thanks - I couldn't find an e-mail address.
Quoted 60 lines
Quoted 8 lines · On Mon, 15 Jun 2009 13:03:08 +0100 Scott James Remnant <[email protected]> wrote:
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
Quoted 9 lines
The poster-child example is OpenSSH; the per-login children call
setsid()so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.This patch adds a new
PROC_SID_EVENTand associated structure to theproc_eventevent_dataunion, it arranges for this to be emitted when the specialPIDTYPE_SIDpid is set.hm, well, I don't have much useful to say about the overall idea, but it seems to slot into the existing code simply enough.
Quoted 5 lines
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)We seem to have forgotten to document this entire interface, so I can't ding you for forgetting to update the forgotten documentation.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 142
Quoted 9 lines
drivers/connector/cn_proc.c +1
-0@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task)It would be nice to have a nice comment explaining what this function does. Ditto all the others in there, really.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 153
Quoted 11 lines
drivers/connector/cn_proc.c +11
-0+{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data;Please pass all patches through
scripts/checkpatch.pl.
Quoted 9 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 158
Quoted 5 lines
drivers/connector/cn_proc.c +5
-0+get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
I must admit, that I just copy and pasted the existing functions - bugs and all :-)
Quoted 61 lines
Quoted 61 lines
drivers/connector/cn_proc.c +8
-0+ ev->event_data.sid.process_tgid = task->tgid; + +memcpy (&msg->id, &cn_proc_event_id,sizeof (msg->id)); + msg->ack =0 ;/* not used */ + msg->len =sizeof (*ev); +cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); +} +void proc_exit_connector (struct task_struct *task) {struct cn_msg *msg;include/linux/cn_proc.h +10
-0@@ -52,6 +52,7 @@ struct proc_event {PROC_EVENT_EXEC =0x00000002 ,PROC_EVENT_UID =0x00000004 ,PROC_EVENT_GID =0x00000040 , +PROC_EVENT_SID =0x00000080 ,/* "next" should be 0x00000400 */ /* "last" is the last process event: exit */ PROC_EVENT_EXIT =0x80000000 @@ -89,6 +90,11 @@ struct proc_event { } e; } id; +struct sid_proc_event { +__kernel_pid_t process_pid; +__kernel_pid_t process_tgid; + } sid; +struct exit_proc_event {__kernel_pid_t process_pid;__kernel_pid_t process_tgid;@@ -102,6 +108,7 @@ struct proc_event {void proc_fork_connector (struct task_struct *task);void proc_exec_connector (struct task_struct *task);void proc_id_connector (struct task_struct *task,int which_id); +void proc_sid_connector (struct task_struct *task);void proc_exit_connector (struct task_struct *task);#else static inline void proc_fork_connector (struct task_struct *task)@@ -114,6 +121,9 @@ static inline void proc_id_connector(struct task_struct *task, int which_id) {} +static inline void proc_sid_connector (struct task_struct *task) +{} +static inline void proc_exit_connector (struct task_struct *task) {}#endif /* CONFIG_PROC_EVENTS */ kernel/exit.c +0
-0@@ -360,8 +360,10 @@ void __set_special_pids(struct pid *pid) {struct task_struct *curr = current->group_leader;
Quoted 9 lines
kernel/exit.c void __set_special_pids(struct pid *pid) · line 369
Quoted 8 lines
kernel/exit.c +3
-1-if (task_session (curr) != pid) +if (task_session (curr) != pid) {change_pid (curr,PIDTYPE_SID , pid); +proc_sid_connector (curr); + }if (task_pgrp (curr) != pid)change_pid (curr,PIDTYPE_PGID , pid);
Scott
Signature · 2 lines
Scott James Remnant [email protected]
Matt Helsley
Acked-byQuoted 18 lines · Andrew Morton
Let's cc the Process Events developer..
Quoted 13 lines · On Mon, 15 Jun 2009 13:03:08 +0100 Scott James Remnant…
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
The poster-child example is OpenSSH; the per-login children call
setsid()so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.
Quoted 7 lines
Quoted 4 lines
This patch adds a new
PROC_SID_EVENTand associated structure to theproc_eventevent_dataunion, it arranges for this to be emitted when the specialPIDTYPE_SIDpid is set.hm, well, I don't have much useful to say about the overall idea, but it seems to slot into the existing code simply enough.
No comment on the usefulness of the event here. The code is consistent with the process events interface though and I see no problems with the code.
Quoted 9 lines
Quoted 6 lines
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)We seem to have forgotten to document this entire interface, so I can't ding you for forgetting to update the forgotten documentation.
I've written down the essentials for Documentation/. I've included it as a patch including the SID info and following my reply to the container issue below.
Cc'ing manpages folks in case they'd like to review comment on missing information in the patch.
Quoted 27 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 142
Quoted 9 lines
drivers/connector/cn_proc.c +1
-0@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task)It would be nice to have a nice comment explaining what this function does. Ditto all the others in there, really.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 153
Quoted 11 lines
drivers/connector/cn_proc.c +11
-0+{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data;Please pass all patches through
scripts/checkpatch.pl.
Quoted 8 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 158
Quoted 5 lines
drivers/connector/cn_proc.c +5
-0+get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
An excellent point. It's broadcast via a netlink multicast address. That means we'd have pids and listeners from arbitrary combinations of pid namespaces.
One obvious but poor solution is to only send the pid of the initial pid namespace. Then it's not ambiguous what an event refers to. However it also means that the events would only be useful to tasks running in the initial pid namespace -- not a good solution given Scott's example and our desire to run things like sshd in separate pid namespaces.
Alternatively, we may be able to split up the connector such that the listeners only see events from their own pid namespace. I'm not sure that netlink and connectors can enable this change though.
Cc'ing pidns and containers folks
--
From: Matt Helsley <matthltc-r/[email protected]>
Subject: [PATCH] Add documentation for the process events connector
Document the process events connector user/kernel interface.
Documentation/connector/process-events.txt new file +173 -0
@@ -0,0 +1,173 @@ +OVERVIEW + +The process events connector is a kernel-userspace multicast socket + that reports process events like fork, exec, id change (real and effective +user and group ids), and exit events to userspace. Applications that may +find these events useful include auditing, system activity monitoring +(e.g. top), security, and likely many more. + + The low-level details describing the permissions needed and how to +read messages from the connector are best described in +Documentation/connector/connector.txt and netlink(3). This document describes +messages sent back and forth between the kernel and userspace listeners using +the connector message format. + +GETTING STARTED + + The necessary definitions of the connector address and userspace/kernel +data can be found by including: + + #include <linux/connector.h> + #include <linux/cn_proc.h> + + To listen for process events a task must open a netlink socket and +bind to a special address: + +struct sockaddr_nl cn_addr = { + .nl_family = AF_NETLINK, + .nl_groups = CN_IDX_PROC, + .nl_pid = getpid() +}; + +Once bound the listener must inform the kernel that at least one task is +interested in receiving multicast process events on this socket. This requires +sending a message with PROC_CN_MCAST_LISTEN in it's payload: + +union { + struct cn_msg listen_msg = { + .id = { + .idx = CN_IDX_PROC, + .val = CN_VAL_PROC, + }, + .seq = X, + .ack = Y, + .len = sizeof(enum proc_cn_mcast_op) + }; + char bytes[sizeof(struct cn_msg) + sizeof(enum proc_cn_mcast_op)]; +} buf; +... +*((enum proc_cn_mcast_op*)buf.listen_msg.data) = PROC_CN_MCAST_LISTEN; + +After sending this message to the kernel the application should expect +an acknowledgement message (more on ack messages below). Similar to the +PROC_CN_MCAST_LISTEN message, userspace may send a message with +PROC_CN_MCAST_IGNORE as its sole payload. Because it is bound to a multicast +address the IGNORE "control" operation does not necessarily prevent process +events from becoming available on the socket. Only the last application to +issue IGNORE ensures that no more process events will be delivered. + +Packets received from the kernel are either an acknowledgement of a control +message or a proper event. Both are described by struct proc_event: + +struct proc_event { + enum what { + PROC_EVENT_NONE = 0x00000000, + PROC_EVENT_FORK = 0x00000001, + PROC_EVENT_EXEC = 0x00000002, + PROC_EVENT_UID = 0x00000004, + PROC_EVENT_GID = 0x00000040, + /* "next" should be 0x00000400 */ + /* "last" is the last process event: exit */ + PROC_EVENT_EXIT = 0x80000000 + } what; + __u32 cpu; + __u64 __attribute__((aligned(8))) timestamp_ns; + /* Number of nano seconds since system boot */ + union { /* must be last field of proc_event struct */ + struct { + __u32 err; + } ack; + + struct fork_proc_event { + pid_t parent_pid; + pid_t parent_tgid; + pid_t child_pid; + pid_t child_tgid; + } fork; + + struct exec_proc_event { + pid_t process_pid; + pid_t process_tgid; + } exec; + + struct id_proc_event { + pid_t process_pid; + pid_t process_tgid; + union { + __u32 ruid; /* task uid */ + __u32 rgid; /* task gid */ + } r; + union { + __u32 euid; + __u32 egid; + } e; + } id; + struct sid_proc_event { + pid_t process_pid; + pid_t process_tgid; + } sid; + struct exit_proc_event { + pid_t process_pid; + pid_t process_tgid; + __u32 exit_code, exit_signal; + } exit; + } event_data; +}; + +All events have valid what, cpu, and timestamp_ns fields. These record what +the event is, which cpu reported it, and a monotonically-increasing timestamp +in nanosecond units. Note that while the timestamp is reported in nanoseconds +the actual time resolution may vary depending on the kernel's clock source(s). +The cpu, timestamp, and sequence number from the netlink message wrapper +provide a best-effort means of reassembling events from multiple CPUs +"in order". It is best-effort because strictly speaking the parallel nature of +execution between CPUs precludes offering the strongest ordering guarantees. + +Another wrinkle to consider: pids, tgids, session ids, exist in pid namespaces +while the netlink sockets are part of a network namespace. If these namespaces +do not coincide then the pids, tgids, and session ids may not exist or may +refer to the wrong tasks. + +TYPES OF PACKETS + +There are several types of messages, each with a corresponding .what value: + Acknowledgement PROC_EVENT_NONE + Fork PROC_EVENT_FORK + Exec PROC_EVENT_EXEC + UID Change PROC_EVENT_UID + GID Change PROC_EVENT_GID + Session ID Change PROC_EVENT_SID + Exit PROC_EVENT_EXIT + +(Note that while these values are encoded as bits (1 << n) the +kernel does not report multiple events in the same message.) Each event also +has a corresponding struct in the event_data union which describes the data +that is valid for the event. + +ACKNOWLEDGEMENT + +Acknowledgement messages contain an err field in their +event_data. When non-zero the err field indicates that the control operation +did not succeed. The value is suitable for, but not stored in, errno. For +example an illegal or unrecognized control operation returns an acknowledgement +message with event_data.ack.err == EINVAL. + +All other messages include at least one pid and a corresponding tgid describing +which task(s) the event relates to. As with normal Linux convention, pid == tid. +Each task described in the event has a pid and tgid pair. + +FORK + +Fork messages are emitted when a task calls the fork() or clone() system calls. +They contain the pid and tgid of the parent and child tasks. + +EXEC messages indicate via pid and tgid which task exec'd. + +UID messages indicate that the task's real and/or effective user id has changed. + +GID messages indicate that the task's real and/or effective group id has changed. + +SID messages indicate that the task has started a new session. + +EXIT messages indicate that the task has exitted and provide the exit code and +exit signal.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 10 lines · Matt Helsley
Quoted 8 lines · Andrew Morton
Review
Quoted 5 lines
(quoted diff) +5
-0+ get_seq(&msg->seq, &ev->cpu); + ktime_get_ts(&ts); /* get high res monotonic timestamp */ + put_unaligned(timespec_to_ns(&ts), (__u64 *)&ev->timestamp_ns); + ev->what = PROC_EVENT_SID; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
Quoted 4 lines
An excellent point. It's broadcast via a netlink multicast address. That means we'd have pids and listeners from arbitrary combinations of pid namespaces.
Yeah, right now that's a general problem with the netlink approach compared to the signal approach I was using before. Of course, it's also non-obvious how init in the initial pid namespace should deal with processes dying in a different pid namespace.
Quoted 10 lines
One obvious but poor solution is to only send the pid of the initial pid namespace. Then it's not ambiguous what an event refers to. However it also means that the events would only be useful to tasks running in the initial pid namespace -- not a good solution given Scott's example and our desire to run things like sshd in separate pid namespaces.
Alternatively, we may be able to split up the connector such that the listeners only see events from their own pid namespace. I'm not sure that netlink and connectors can enable this change though.
Or the netlink socket could include both the pid, and a descriptor of the pid namespace that it is in (isn't it just a pid itself?) That way listeners could check the namespace is the same before carrying on.
Though that obviously leaks information you may not actually want leaked?
Scott
Signature · 2 lines
Scott James Remnant scott-GeWIH/[email protected]
A message that is not in the archive (sent off-list or not yet received): 20090623210110.GB7931-52DBMbEzqgQ/wnmkkaCWp/[email protected]
Quoted 10 lines · Matt Helsley
Quoted 8 lines · Andrew Morton
Review
Quoted 5 lines
(quoted diff) +5
-0+ get_seq(&msg->seq, &ev->cpu); + ktime_get_ts(&ts); /* get high res monotonic timestamp */ + put_unaligned(timespec_to_ns(&ts), (__u64 *)&ev->timestamp_ns); + ev->what = PROC_EVENT_SID; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
Quoted 4 lines
An excellent point. It's broadcast via a netlink multicast address. That means we'd have pids and listeners from arbitrary combinations of pid namespaces.
Yeah, right now that's a general problem with the netlink approach compared to the signal approach I was using before. Of course, it's also non-obvious how init in the initial pid namespace should deal with processes dying in a different pid namespace.
Quoted 10 lines
One obvious but poor solution is to only send the pid of the initial pid namespace. Then it's not ambiguous what an event refers to. However it also means that the events would only be useful to tasks running in the initial pid namespace -- not a good solution given Scott's example and our desire to run things like sshd in separate pid namespaces.
Alternatively, we may be able to split up the connector such that the listeners only see events from their own pid namespace. I'm not sure that netlink and connectors can enable this change though.
Or the netlink socket could include both the pid, and a descriptor of the pid namespace that it is in (isn't it just a pid itself?) That way listeners could check the namespace is the same before carrying on.
Though that obviously leaks information you may not actually want leaked?
Scott
Signature · 2 lines
Scott James Remnant scott-GeWIH/[email protected]
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
A message that is not in the archive (sent off-list or not yet received): 20090622161909.e5706885.akpm-de/[email protected]
Matt Helsley
Acked-byQuoted 18 lines · Andrew Morton
Let's cc the Process Events developer..
Quoted 13 lines · On Mon, 15 Jun 2009 13:03:08 +0100 Scott James Remnant…
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
The poster-child example is OpenSSH; the per-login children call
setsid()so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.
Quoted 7 lines
Quoted 4 lines
This patch adds a new
PROC_SID_EVENTand associated structure to theproc_eventevent_dataunion, it arranges for this to be emitted when the specialPIDTYPE_SIDpid is set.hm, well, I don't have much useful to say about the overall idea, but it seems to slot into the existing code simply enough.
No comment on the usefulness of the event here. The code is consistent with the process events interface though and I see no problems with the code.
Quoted 9 lines
Quoted 6 lines
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)We seem to have forgotten to document this entire interface, so I can't ding you for forgetting to update the forgotten documentation.
I've written down the essentials for Documentation/. I've included it as a patch including the SID info and following my reply to the container issue below.
Cc'ing manpages folks in case they'd like to review comment on missing information in the patch.
Quoted 27 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 142
Quoted 9 lines
drivers/connector/cn_proc.c +1
-0@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task)It would be nice to have a nice comment explaining what this function does. Ditto all the others in there, really.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 153
Quoted 11 lines
drivers/connector/cn_proc.c +11
-0+{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data;Please pass all patches through
scripts/checkpatch.pl.
Quoted 8 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 158
Quoted 5 lines
drivers/connector/cn_proc.c +5
-0+get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
An excellent point. It's broadcast via a netlink multicast address. That means we'd have pids and listeners from arbitrary combinations of pid namespaces.
One obvious but poor solution is to only send the pid of the initial pid namespace. Then it's not ambiguous what an event refers to. However it also means that the events would only be useful to tasks running in the initial pid namespace -- not a good solution given Scott's example and our desire to run things like sshd in separate pid namespaces.
Alternatively, we may be able to split up the connector such that the listeners only see events from their own pid namespace. I'm not sure that netlink and connectors can enable this change though.
Cc'ing pidns and containers folks
--
From: Matt Helsley <matthltc-r/[email protected]>
Subject: [PATCH] Add documentation for the process events connector
Document the process events connector user/kernel interface.
Documentation/connector/process-events.txt new file +173 -0
@@ -0,0 +1,173 @@ +OVERVIEW + +The process events connector is a kernel-userspace multicast socket + that reports process events like fork, exec, id change (real and effective +user and group ids), and exit events to userspace. Applications that may +find these events useful include auditing, system activity monitoring +(e.g. top), security, and likely many more. + + The low-level details describing the permissions needed and how to +read messages from the connector are best described in +Documentation/connector/connector.txt and netlink(3). This document describes +messages sent back and forth between the kernel and userspace listeners using +the connector message format. + +GETTING STARTED + + The necessary definitions of the connector address and userspace/kernel +data can be found by including: + + #include <linux/connector.h> + #include <linux/cn_proc.h> + + To listen for process events a task must open a netlink socket and +bind to a special address: + +struct sockaddr_nl cn_addr = { + .nl_family = AF_NETLINK, + .nl_groups = CN_IDX_PROC, + .nl_pid = getpid() +}; + +Once bound the listener must inform the kernel that at least one task is +interested in receiving multicast process events on this socket. This requires +sending a message with PROC_CN_MCAST_LISTEN in it's payload: + +union { + struct cn_msg listen_msg = { + .id = { + .idx = CN_IDX_PROC, + .val = CN_VAL_PROC, + }, + .seq = X, + .ack = Y, + .len = sizeof(enum proc_cn_mcast_op) + }; + char bytes[sizeof(struct cn_msg) + sizeof(enum proc_cn_mcast_op)]; +} buf; +... +*((enum proc_cn_mcast_op*)buf.listen_msg.data) = PROC_CN_MCAST_LISTEN; + +After sending this message to the kernel the application should expect +an acknowledgement message (more on ack messages below). Similar to the +PROC_CN_MCAST_LISTEN message, userspace may send a message with +PROC_CN_MCAST_IGNORE as its sole payload. Because it is bound to a multicast +address the IGNORE "control" operation does not necessarily prevent process +events from becoming available on the socket. Only the last application to +issue IGNORE ensures that no more process events will be delivered. + +Packets received from the kernel are either an acknowledgement of a control +message or a proper event. Both are described by struct proc_event: + +struct proc_event { + enum what { + PROC_EVENT_NONE = 0x00000000, + PROC_EVENT_FORK = 0x00000001, + PROC_EVENT_EXEC = 0x00000002, + PROC_EVENT_UID = 0x00000004, + PROC_EVENT_GID = 0x00000040, + /* "next" should be 0x00000400 */ + /* "last" is the last process event: exit */ + PROC_EVENT_EXIT = 0x80000000 + } what; + __u32 cpu; + __u64 __attribute__((aligned(8))) timestamp_ns; + /* Number of nano seconds since system boot */ + union { /* must be last field of proc_event struct */ + struct { + __u32 err; + } ack; + + struct fork_proc_event { + pid_t parent_pid; + pid_t parent_tgid; + pid_t child_pid; + pid_t child_tgid; + } fork; + + struct exec_proc_event { + pid_t process_pid; + pid_t process_tgid; + } exec; + + struct id_proc_event { + pid_t process_pid; + pid_t process_tgid; + union { + __u32 ruid; /* task uid */ + __u32 rgid; /* task gid */ + } r; + union { + __u32 euid; + __u32 egid; + } e; + } id; + struct sid_proc_event { + pid_t process_pid; + pid_t process_tgid; + } sid; + struct exit_proc_event { + pid_t process_pid; + pid_t process_tgid; + __u32 exit_code, exit_signal; + } exit; + } event_data; +}; + +All events have valid what, cpu, and timestamp_ns fields. These record what +the event is, which cpu reported it, and a monotonically-increasing timestamp +in nanosecond units. Note that while the timestamp is reported in nanoseconds +the actual time resolution may vary depending on the kernel's clock source(s). +The cpu, timestamp, and sequence number from the netlink message wrapper +provide a best-effort means of reassembling events from multiple CPUs +"in order". It is best-effort because strictly speaking the parallel nature of +execution between CPUs precludes offering the strongest ordering guarantees. + +Another wrinkle to consider: pids, tgids, session ids, exist in pid namespaces +while the netlink sockets are part of a network namespace. If these namespaces +do not coincide then the pids, tgids, and session ids may not exist or may +refer to the wrong tasks. + +TYPES OF PACKETS + +There are several types of messages, each with a corresponding .what value: + Acknowledgement PROC_EVENT_NONE + Fork PROC_EVENT_FORK + Exec PROC_EVENT_EXEC + UID Change PROC_EVENT_UID + GID Change PROC_EVENT_GID + Session ID Change PROC_EVENT_SID + Exit PROC_EVENT_EXIT + +(Note that while these values are encoded as bits (1 << n) the +kernel does not report multiple events in the same message.) Each event also +has a corresponding struct in the event_data union which describes the data +that is valid for the event. + +ACKNOWLEDGEMENT + +Acknowledgement messages contain an err field in their +event_data. When non-zero the err field indicates that the control operation +did not succeed. The value is suitable for, but not stored in, errno. For +example an illegal or unrecognized control operation returns an acknowledgement +message with event_data.ack.err == EINVAL. + +All other messages include at least one pid and a corresponding tgid describing +which task(s) the event relates to. As with normal Linux convention, pid == tid. +Each task described in the event has a pid and tgid pair. + +FORK + +Fork messages are emitted when a task calls the fork() or clone() system calls. +They contain the pid and tgid of the parent and child tasks. + +EXEC messages indicate via pid and tgid which task exec'd. + +UID messages indicate that the task's real and/or effective user id has changed. + +GID messages indicate that the task's real and/or effective group id has changed. + +SID messages indicate that the task has started a new session. + +EXIT messages indicate that the task has exitted and provide the exit code and +exit signal.
Quoted 62 lines · Andrew Morton
Let's cc the Process Events developer..
Quoted 8 lines · On Mon, 15 Jun 2009 13:03:08 +0100 Scott James Remnant…
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
Quoted 9 lines
The poster-child example is OpenSSH; the per-login children call
setsid()so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.This patch adds a new
PROC_SID_EVENTand associated structure to theproc_eventevent_dataunion, it arranges for this to be emitted when the specialPIDTYPE_SIDpid is set.hm, well, I don't have much useful to say about the overall idea, but it seems to slot into the existing code simply enough.
Quoted 5 lines
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)We seem to have forgotten to document this entire interface, so I can't ding you for forgetting to update the forgotten documentation.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 142
Quoted 9 lines
drivers/connector/cn_proc.c +1
-0@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task)It would be nice to have a nice comment explaining what this function does. Ditto all the others in there, really.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 153
Quoted 11 lines
drivers/connector/cn_proc.c +11
-0+{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data;Please pass all patches through
scripts/checkpatch.pl.
Quoted 8 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 158
Quoted 5 lines
drivers/connector/cn_proc.c +5
-0+get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
I have been looking for ways to fix this. No good answers so far but there's even more to worry about :(. Because we report uids and gids via this interface any namespace-safe solution must work for combinations of pidns and userns.
Cheers,
-Matt Helsley
Quoted 62 lines · Andrew Morton
Let's cc the Process Events developer..
Quoted 8 lines · On Mon, 15 Jun 2009 13:03:08 +0100 Scott James Remnant <[email protected]> wrote:
The act of a process becoming a session leader is a useful signal to a supervising init daemon such as Upstart.
While a daemon will normally do this as part of the process of becoming a daemon, it is rare for its children to do so. When the children do, it is nearly always a sign that the child should be considered detached from the parent and not supervised along with it.
Quoted 9 lines
The poster-child example is OpenSSH; the per-login children call
setsid()so that they may control the pty connected to them. If the primary daemon dies or is restarted, we do not want to consider the per-login children and want to respawn the primary daemon without killing the children.This patch adds a new
PROC_SID_EVENTand associated structure to theproc_eventevent_dataunion, it arranges for this to be emitted when the specialPIDTYPE_SIDpid is set.hm, well, I don't have much useful to say about the overall idea, but it seems to slot into the existing code simply enough.
Quoted 5 lines
--- drivers/connector/cn_proc.c | 25 +++++++++++++++++++++++++ include/linux/cn_proc.h | 10 ++++++++++ kernel/exit.c | 4 +++- 3 files changed, 38 insertions(+), 1 deletions(-)We seem to have forgotten to document this entire interface, so I can't ding you for forgetting to update the forgotten documentation.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 142
Quoted 9 lines
drivers/connector/cn_proc.c +1
-0@@ -139,6 +139,31 @@ void proc_id_connector(struct task_struct *task, int which_id)cn_netlink_send (msg,CN_IDX_PROC ,GFP_KERNEL ); } +void proc_sid_connector (struct task_struct *task)It would be nice to have a nice comment explaining what this function does. Ditto all the others in there, really.
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 153
Quoted 11 lines
drivers/connector/cn_proc.c +11
-0+{ +struct cn_msg *msg; +struct proc_event *ev; +struct timespec ts; +__u8 buffer[CN_PROC_MSG_SIZE ]; + +if (atomic_read (&proc_event_num_listeners) <1 ) +return ; + + msg = (struct cn_msg *)buffer; + ev = (struct proc_event *)msg->data;Please pass all patches through
scripts/checkpatch.pl.
Quoted 8 lines
drivers/connector/cn_proc.c void proc_id_connector(struct task_struct *task, int which_id) · line 158
Quoted 5 lines
drivers/connector/cn_proc.c +5
-0+get_seq (&msg->seq, &ev->cpu); +ktime_get_ts (&ts);/* get high res monotonic timestamp */ +put_unaligned (timespec_to_ns (&ts), (__u64 *)&ev->timestamp_ns); + ev->what =PROC_EVENT_SID ; + ev->event_data.sid.process_pid = task->pid;This is a bit of a worry. In a containerised environment, pids are not unique. Now what do we do?
I have been looking for ways to fix this. No good answers so far but there's even more to worry about :(. Because we report uids and gids via this interface any namespace-safe solution must work for combinations of pidns and userns.
Cheers,
-Matt Helsley