kernel generic object IDs series

Hi,

when we do the checkpoint-restore we need to find out if various kernel objects (like mm_struct-s of file_struct-s) are shared between tasks (and restore them after).

While at restore time we can use CLONE_XXX flags and unshare syscall there is no way to find out sharing structures at checkpoint time. Thus, to chop the knit, we introduce generic-object-ids helpers which do basically encode kernel pointers into some form (at moment is's simple XOR operation over a random cookie value) and provide them back to userspace. So one can test if two resource are shared between different task.

Since such information is pretty valuable -- it's allowed for CAP_SYS_ADMIN only since xor encoded values has nothing to do with security but used only to break an impression that ID means something other than random "number" which should be used for "sameness" test only and nothing else.

The following objects are shown at the moment

- all namespaces living in /proc/pid/ns/
- open files (shown in /proc/pid/fdinfo/)
- objects, that can be shared with CLONE_XXX flags (except for namespaces)

Any kind of comments and especially complains (!) are very appreciated!

Cyrill

The routine XORs the given pointer with a random value producing an ID (32 or 64 bit, depending on the arch).

Since it's a valuable information -- only CAP_SYS_ADMIN is allowed to obtain it.

Based-on-patch-from: Pavel Emelyanov <[email protected]>

---
 include/linux/mm.h |   10 ++++++++++
 mm/Kconfig         |   16 ++++++++++++++++
 mm/util.c          |   49 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+)

Index: linux-2.6.git/include/linux/mm.h
===================================================================
@@ -1640,5 +1640,15 @@ extern void copy_user_huge_page(struct p
 				unsigned int pages_per_huge_page);
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */
 
+enum {
+	GEN_OBJ_ID_TYPES,
+};
+
+#ifdef CONFIG_GENERIC_OBJECT_IDS
+unsigned long gen_obj_id(void *ptr, int type);
+#else
+static inline unsigned long gen_obj_id(void *ptr, int type) { return 0; }
+#endif
+
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */

Index: linux-2.6.git/mm/Kconfig
===================================================================
@@ -373,3 +373,19 @@ config CLEANCACHE
 	  in a negligible performance hit.
 
 	  If unsure, say Y to enable cleancache
+
+config GENERIC_OBJECT_IDS
+	bool "Enable generic object ids infrastructure"
+	depends on CHECKPOINT_RESTORE
+	default n
+	help
+	  Turn on the functionality that can generate IDs for kernel
+	  objects, which are exported to userspace via /proc filesystem.
+
+	  It is useful if you need to examinate kernel objects and test
+	  if they are shared between several tasks. These IDs should never
+	  be used for anything but the "sameness" test. Besides, the IDs are
+	  dynamic and valid only while object is alive, once it get freed or
+	  kernel is rebooted -- the IDs will be changed.
+
+	  If unsure, say N here.

It can happen in case of object re-allocated from slab. But in case of two living pids it's impossible to get same pointers for different objects. Or I misunderstood the question, Alan? It's up to application to not compare objects from dead tasks.

Cyrill

How will it know the task has not died and been reallocated, or its resources not been freed and reallocated during the comparison ?

Your comparison appears to have a zero time validity - you can ask "is A the same as B" but by the time you get an answer your answer may no longer be true. It also externalises a current implementation detail in a very ugly way.

Would it also not be better to do the job right and simply have an interface to ask "who shares with A" or even something a bit more high level, it seems you are creating something nastier by trying to push all this in userspace than if you did the job or part of it kernel side where you had access to the right locking infrastructure and where the public API doesn't need to expose innards of the kernel ?

Alan

It's not differ from reading other data from /proc. How make you be sure the PPid read from a task status is the same once you've finished reading it? The same applies to say ps output, it's valid for almost zero time, then one need to restart ps again to get new process tree snapshot. The same here -- if I need a precise results I have to either stop tasks or froze them and compare IDs. That's what I've had in mind.

Need to think, thanks a lot for idea, Alan!

Cyrill

You know, Alan, I thought about different ways... but eventually I think having _this_ interface is better than anything else (of course along with addressing problems Tejun mentioned). "Who shares with A" doesn't make situation better, once I obtain the result it's not valid anymore and I have to either do the same requests again and again, or stop/freeze tasks before asking which resources they do share. The interface provided now gives the *minimun* information user-space needed to draw the shared resources affinity scheme, and I would like to escape putting more work on kernel side if possible. Sounds reasonable?

Cyrill

This patch adds proc_ns_read method which provides IDs for /proc/pid/ns/* files.

Based-on-patch-from: Pavel Emelyanov <[email protected]>

---
 Documentation/filesystems/proc.txt |   24 ++++++++++++++++++++++++
 fs/proc/namespaces.c               |   21 +++++++++++++++++++++
 include/linux/mm.h                 |    1 +
 3 files changed, 46 insertions(+)

Index: linux-2.6.git/Documentation/filesystems/proc.txt
===================================================================
@@ -40,6 +40,7 @@ Table of Contents
   3.4	/proc/<pid>/coredump_filter - Core dump filtering settings
   3.5	/proc/<pid>/mountinfo - Information about mounts
   3.6	/proc/<pid>/comm  & /proc/<pid>/task/<tid>/comm
+  3.7	/proc/<pid>/ns - Information about namespaces
 
 
 ------------------------------------------------------------------------------

@@ -1545,3 +1546,26 @@ a task to set its own or one of its thre
 is limited in size compared to the cmdline value, so writing anything longer
 then the kernel's TASK_COMM_LEN (currently 16 chars) will result in a truncated
 comm value.
+
+3.7	/proc/<pid>/ns - Information about namespaces
+-----------------------------------------------------
+
+This directory consists of the following files "net", "uts", "ipc",
+and depend if appropriate CONFIG_ entry is set, i.e. it's possible
+to have only one, two or all three files here.
+
+Currently file contents provides that named "object id" number, which
+is a number useful for the one purpose only -- to test if two differen
+<pid> share the namespace.
+
+A typical format is
+
+id:	445332486300860161
+
+i.e. "id" followed by a number. One should never assume the number
+means something, it is only useful for "sameness" test with another number
+obtained from another <pid>.
+
+Moreover, a safe approach is to remember it as a string, since format may
+change in future and id would be not a long integer value, but something
+else, say SHA1/2 or even uuid encoded stream.

Index: linux-2.6.git/fs/proc/namespaces.c
===================================================================
@@ -27,10 +27,31 @@ static const struct proc_ns_operations *
 #endif
 };
 
+#ifdef CONFIG_GENERIC_OBJECT_IDS
+static ssize_t proc_ns_read(struct file *file, char __user *buf,
+				      size_t len, loff_t *ppos)
+{
+	struct proc_inode *ei = PROC_I(file->f_dentry->d_inode);
+	char tmp[32];
+
+	snprintf(tmp, sizeof(tmp), "id:\t%lu\n",
+		gen_obj_id(ei->ns, GEN_OBJ_ID_NS));
+	return simple_read_from_buffer(buf, len, ppos, tmp, strlen(tmp));
+}
+
+static const struct file_operations ns_file_operations = {
+	.llseek		= no_llseek,
+	.read		= proc_ns_read,
+};
+
+#else
+
 static const struct file_operations ns_file_operations = {
 	.llseek		= no_llseek,
 };
 
+#endif /* CONFIG_GENERIC_OBJECT_IDS */
+
 static struct dentry *proc_ns_instantiate(struct inode *dir,
 	struct dentry *dentry, struct task_struct *task, const void *ptr)
 {

Index: linux-2.6.git/include/linux/mm.h
===================================================================
@@ -1641,6 +1641,7 @@ extern void copy_user_huge_page(struct p
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */
 
 enum {
+	GEN_OBJ_ID_NS,
 	GEN_OBJ_ID_TYPES,
 };
 

This patch adds "id" field in /proc/pid/fdinfo/<file> output which represent generic ID for 'struct file' being used by a task.

Based-on-patch-from: Pavel Emelyanov <[email protected]>

---
 fs/proc/base.c     |   34 +++++++++++++++++++++++++++++-----
 include/linux/mm.h |    1 +
 2 files changed, 30 insertions(+), 5 deletions(-)

Index: linux-2.6.git/fs/proc/base.c
===================================================================
@@ -1888,8 +1888,36 @@ out:
 	return ~0U;
 }
 
+#ifdef CONFIG_GENERIC_OBJECT_IDS
+
+#define PROC_FDINFO_MAX 128
+
+static void proc_fd_info_fill(char *info, struct file *file, unsigned int f_flags)
+{
+	snprintf(info, PROC_FDINFO_MAX,
+		"pos:\t%lli\n"
+		"flags:\t0%o\n"
+		"id:\t%lu\n",
+		(long long)file->f_pos,
+		f_flags,
+		gen_obj_id(file, GEN_OBJ_ID_FILE));
+}
+
+#else /* CONFIG_GENERIC_OBJECT_IDS */
+
 #define PROC_FDINFO_MAX 64
 
+static void proc_fd_info_fill(char *info, struct file *file, unsigned int f_flags)
+{
+	snprintf(info, PROC_FDINFO_MAX,
+		"pos:\t%lli\n"
+		"flags:\t0%o\n",
+		(long long)file->f_pos,
+		f_flags);
+}
+
+#endif
+
 static int proc_fd_info(struct inode *inode, struct path *path, char *info)
 {
 	struct task_struct *task = get_proc_task(inode);

@@ -1922,11 +1950,7 @@ static int proc_fd_info(struct inode *in
 				path_get(&file->f_path);
 			}
 			if (info)
-				snprintf(info, PROC_FDINFO_MAX,
-					 "pos:\t%lli\n"
-					 "flags:\t0%o\n",
-					 (long long) file->f_pos,
-					 f_flags);
+				proc_fd_info_fill(info, file, f_flags);
 			spin_unlock(&files->file_lock);
 			put_files_struct(files);
 			return 0;

Index: linux-2.6.git/include/linux/mm.h
===================================================================
@@ -1642,6 +1642,7 @@ extern void copy_user_huge_page(struct p
 
 enum {
 	GEN_OBJ_ID_NS,
+	GEN_OBJ_ID_FILE,
 	GEN_OBJ_ID_TYPES,
 };
 

On Thu, Dec 22, 2011 at 04:56:39PM +0400, Cyrill Gorcunov wrote:
...

Because of "CLONE_XXX" in subject, LKML dropped the fourth patch, so I renamed it and put it here. Sorry for inconvenince.

Cyrill
---
proc: Show IDs of objects cloned with CLONE_ in proc

An example of output is

	# cat /proc/2332/objects 
	VM        :445332486314238977
	FILES     :16574420397857109505
	FS        :7421276367695228033
	SIGHAND   :9517130248188834433
	IO        :9605099974489260945
	SYSVSEM   :0

each one represents an ID of appropriate resource used by a task -- memory, signals, fs and etc.

Based-on-patch-from: Pavel Emelyanov <[email protected]>

---
 Documentation/filesystems/proc.txt |   18 ++++++++++++++++++
 fs/proc/base.c                     |   29 +++++++++++++++++++++++++++++
 include/linux/mm.h                 |    6 ++++++
 3 files changed, 53 insertions(+)

Index: linux-2.6.git/Documentation/filesystems/proc.txt
===================================================================
@@ -41,6 +41,7 @@ Table of Contents
   3.5	/proc/<pid>/mountinfo - Information about mounts
   3.6	/proc/<pid>/comm  & /proc/<pid>/task/<tid>/comm
   3.7	/proc/<pid>/ns - Information about namespaces
+  3.8	/proc/<pid>/objects - Information about generic objects
 
 
 ------------------------------------------------------------------------------

@@ -1569,3 +1570,20 @@ obtained from another <pid>.
 Moreover, a safe approach is to remember it as a string, since format may
 change in future and id would be not a long integer value, but something
 else, say SHA1/2 or even uuid encoded stream.
+
+3.8	/proc/<pid>/objects - Information about generic objects
+---------------------------------------------------------------
+
+Similar to /proc/<pid>/ns this file provide generic object ids by
+the following format (better to illustrate with example)
+
+VM        :445332486300860161
+FILES     :16574420397866995457
+FS        :7421276367693613825
+SIGHAND   :9517130248196786369
+IO        :0
+SYSVSEM   :0
+
+The first column is an object name, the second -- object ID number.
+As being mentioned in 3.7 one should carry them as a string to be
+on a safe side.

Index: linux-2.6.git/fs/proc/base.c
===================================================================
@@ -312,6 +312,32 @@ out:
 	return res;
 }
 
+#ifdef CONFIG_GENERIC_OBJECT_IDS
+static int proc_pid_objs(struct task_struct *task, char * buffer)
+{
+	int ret = 0;
+
+#define SHOW_PID_OBJ(__type, __field)					\
+	do {								\
+		ret += sprintf(buffer + ret, "%-10s:%lu\n",		\
+				#__type,				\
+				gen_obj_id(task->__field,		\
+					GEN_OBJ_ID_##__type));		\
+	} while (0)
+
+	SHOW_PID_OBJ(VM, mm);
+	SHOW_PID_OBJ(FILES, files);
+	SHOW_PID_OBJ(FS, fs);
+	SHOW_PID_OBJ(SIGHAND, sighand);
+	SHOW_PID_OBJ(IO, io_context);
+	SHOW_PID_OBJ(SYSVSEM, sysvsem.undo_list);
+
+#undef SHOW_PID_OBJ
+
+	return ret;
+}
+#endif
+
 static int proc_pid_auxv(struct task_struct *task, char *buffer)
 {
 	struct mm_struct *mm = mm_for_maps(task);

@@ -3172,6 +3198,9 @@ static const struct pid_entry tgid_base_
 	INF("syscall",    S_IRUGO, proc_pid_syscall),
 #endif
 	INF("cmdline",    S_IRUGO, proc_pid_cmdline),
+#ifdef CONFIG_GENERIC_OBJECT_IDS
+	INF("objects",    S_IRUGO, proc_pid_objs),
+#endif
 	ONE("stat",       S_IRUGO, proc_tgid_stat),
 	ONE("statm",      S_IRUGO, proc_pid_statm),
 	REG("maps",       S_IRUGO, proc_maps_operations),

Index: linux-2.6.git/include/linux/mm.h
===================================================================
@@ -1643,6 +1643,12 @@ extern void copy_user_huge_page(struct p
 enum {
 	GEN_OBJ_ID_NS,
 	GEN_OBJ_ID_FILE,
+	GEN_OBJ_ID_VM,
+	GEN_OBJ_ID_FILES,
+	GEN_OBJ_ID_FS,
+	GEN_OBJ_ID_SIGHAND,
+	GEN_OBJ_ID_IO,
+	GEN_OBJ_ID_SYSVSEM,
 	GEN_OBJ_ID_TYPES,
 };
 

...

Ping, any opinions on the series?