IPC: checkpoint/restore in userspace enhancements
v8 cover IPC: checkpoint/restore in userspace enhancements
v8:
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new MSG_COPY flag for sys_msgrcv(). If MSG_COPY flag is set, then msgtyp is interpreted as message number.
The following series implements...
---
Stanislav Kinsbursky (5):
ipc: remove forced assignment of selected message
ipc: add sysctl to specify desired next object id
ipc: message queue receive cleanup
ipc: message queue copy feature introduced
test: IPC message queue copy feture test
include/linux/ipc_namespace.h | 1 include/linux/msg.h | 5 - include/uapi/linux/msg.h | 1 ipc/compat.c | 45 +++---- ipc/ipc_sysctl.c | 28 ++++ ipc/msg.c | 99 +++++++++++---- ipc/msgutil.c | 38 ++++++ ipc/util.c | 16 ++ ipc/util.h | 2 tools/testing/selftests/ipc/Makefile | 25 ++++ tools/testing/selftests/ipc/msgque.c | 231 ++++++++++++++++++++++++++++++++++ 11 files changed, 432 insertions(+), 59 deletions(-) create mode 100644 tools/testing/selftests/ipc/Makefile create mode 100644 tools/testing/selftests/ipc/msgque.c
PATCH v8 1/5 ipc: remove forced assignment of selected message
This is a cleanup patch. The assignment is redundant.
--- ipc/msg.c | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-)
ipc/msg.c +1 -4
@@ -793,12 +793,9 @@ long do_msgrcv(int msqid, long *pmtype, void __user *mtext, msg = walk_msg;if (mode ==SEARCH_LESSEQUAL && walk_msg->m_type !=1 ) { - msg = walk_msg; msgtyp = walk_msg->m_type -1 ; - }else { - msg = walk_msg; + }else break ; - } } tmp = tmp->next; }
PATCH v8 2/5 ipc: add sysctl to specify desired next object id
In reply to Stanislav KinsburskyThis patch adds 3 new variables and sysctls to tune them (by one "next_id" variable for messages, semaphores and shared memory respectively). This variable can be used to set desired id for next allocated IPC object. By default it's equal to -1 and old behaviour is preserved.
If this variable is non-negative, then desired idr will be extracted from it and used as a start value to search for free IDR slot.
Notes:
1) this patch doesn't garantee, that new object will have desired id. So it's up to user space how to handle new object with wrong id.
2) After sucessfull id allocation attempt, "next_id" will be set back to -1 (if it was non-negative).
--- include/linux/ipc_namespace.h | 1 + ipc/ipc_sysctl.c | 28 ++++++++++++++++++++++++++++ ipc/util.c | 16 ++++++++++++---- ipc/util.h | 1 + 4 files changed, 42 insertions(+), 4 deletions(-)
include/linux/ipc_namespace.h +1 -0
@@ -24,6 +24,7 @@ struct ipc_ids {unsigned short seq_max;struct rw_semaphore rw_mutex;struct idr ipcs_idr; +int next_id; };struct ipc_namespace {
ipc/ipc_sysctl.c +28 -0
@@ -158,6 +158,7 @@ static int proc_ipcauto_dointvec_minmax(ctl_table *table, int write,static int zero;static int one =1 ; +static int int_max =INT_MAX ;static struct ctl_table ipc_kern_table[] = { {
@@ -227,6 +228,33 @@ static struct ctl_table ipc_kern_table[] = { .extra1 = &zero, .extra2 = &one, }, + { + .procname = "sem_next_id", + .data = &init_ipc_ns.ids[IPC_SEM_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_SEM_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, + { + .procname = "msg_next_id", + .data = &init_ipc_ns.ids[IPC_MSG_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_MSG_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, + { + .procname = "shm_next_id", + .data = &init_ipc_ns.ids[IPC_SHM_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_SHM_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, {} };
ipc/util.c +12 -4
@@ -122,6 +122,7 @@ void ipc_init_ids(struct ipc_ids *ids) ids->in_use =0 ; ids->seq =0 ; + ids->next_id =-1 ; {int seq_limit =INT_MAX /SEQ_MULTIPLIER ;if (seq_limit >USHRT_MAX )
@@ -252,6 +253,7 @@ int ipc_addid(struct ipc_ids* ids, struct kern_ipc_perm* new, int size)kuid_t euid;kgid_t egid;int id, err; +int next_id = ids->next_id;if (size >IPCMNI ) size =IPCMNI ;
@@ -264,7 +266,8 @@ int ipc_addid(struct ipc_ids* ids, struct kern_ipc_perm* new, int size)rcu_read_lock ();spin_lock (&new->lock); - err =idr_get_new (&ids->ipcs_idr, new, &id); + err =idr_get_new_above (&ids->ipcs_idr, new, + (next_id <0 ) ?0 :ipcid_to_idx (next_id), &id);if (err) {spin_unlock (&new->lock);rcu_read_unlock ();
@@ -277,9 +280,14 @@ int ipc_addid(struct ipc_ids* ids, struct kern_ipc_perm* new, int size) new->cuid = new->uid = euid; new->gid = new->cgid = egid; - new->seq = ids->seq++; -if (ids->seq > ids->seq_max) - ids->seq =0 ; +if (next_id <0 ) { + new->seq = ids->seq++; +if (ids->seq > ids->seq_max) + ids->seq =0 ; + }else { + new->seq =ipcid_to_seqx (next_id); + ids->next_id =-1 ; + } new->id =ipc_buildid (id, new->seq);return id;
ipc/util.h +1 -0
@@ -92,6 +92,7 @@ void __init ipc_init_proc_interface(const char *path, const char *header,#define IPC_SHM_IDS 2#define ipcid_to_idx (id) ((id) % SEQ_MULTIPLIER) +#define ipcid_to_seqx (id) ((id) / SEQ_MULTIPLIER)/* must be called with ids->rw_mutex acquired for writing */ int ipc_addid (struct ipc_ids *,struct kern_ipc_perm *,int );
Re: [PATCH v8 2/5] ipc: add sysctl to specify desired next object id
Quoted 46 lines · On Wed, 24 Oct 2012 19:35:09 +0400 Stanislav Kinsbursky…
This patch adds 3 new variables and sysctls to tune them (by one "
next_id" variable for messages, semaphores and shared memory respectively). This variable can be used to set desired id for next allocated IPC object. By default it's equal to -1 and old behaviour is preserved.
If this variable is non-negative, then desired idr will be extracted from it and used as a start value to search for free IDR slot.Notes:
1) this patch doesn't garantee, that new object will have desired id. So it's up to user space how to handle new object with wrong id.
2) After sucessfull id allocation attempt, "next_id" will be set back to -1 (if it was non-negative).ipc/ipc_sysctl.c +20
-0@@ -158,6 +158,7 @@ static int proc_ipcauto_dointvec_minmax(ctl_table *table, int write,static int zero;static int one =1 ; +static int int_max =INT_MAX ;static struct ctl_table ipc_kern_table[] = { {@@ -227,6 +228,33 @@ static struct ctl_table ipc_kern_table[] = { .extra1 = &zero, .extra2 = &one, }, + { + .procname = "sem_next_id", + .data = &init_ipc_ns.ids[IPC_SEM_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_SEM_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, + { + .procname = "msg_next_id", + .data = &init_ipc_ns.ids[IPC_MSG_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_MSG_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, + {
ipc/ipc_sysctl.c static struct ctl_table ipc_kern_table[] = { · line 259
Quoted 10 lines
ipc/ipc_sysctl.c +8
-0+ .procname = "shm_next_id", + .data = &init_ipc_ns.ids[IPC_SHM_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_SHM_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, {} };
ipc_kern_table[] is (badly) documented in Documentation/sysctl/kernel.txt. Can we at least mention these controls in there? Better, create a new way of properly documenting each control and document these three in that manner? Better still, document all the other ones as well ;)
The patch adds these controls to CONFIG_CHECKPOINT_RESTORE=n kernels. Why is this?
Re: [PATCH v8 2/5] ipc: add sysctl to specify desired next object id
25.10.2012 01:41, Andrew Morton пишет:
Quoted 57 lines
Quoted 54 lines · On Wed, 24 Oct 2012 19:35:09 +0400 Stanislav Kinsbursky…
This patch adds 3 new variables and sysctls to tune them (by one "
next_id" variable for messages, semaphores and shared memory respectively). This variable can be used to set desired id for next allocated IPC object. By default it's equal to -1 and old behaviour is preserved.
If this variable is non-negative, then desired idr will be extracted from it and used as a start value to search for free IDR slot.Notes:
1) this patch doesn't garantee, that new object will have desired id. So it's up to user space how to handle new object with wrong id.
2) After sucessfull id allocation attempt, "next_id" will be set back to -1 (if it was non-negative).ipc/ipc_sysctl.c +28
-0@@ -158,6 +158,7 @@ static int proc_ipcauto_dointvec_minmax(ctl_table *table, int write,static int zero;static int one =1 ; +static int int_max =INT_MAX ;static struct ctl_table ipc_kern_table[] = { {@@ -227,6 +228,33 @@ static struct ctl_table ipc_kern_table[] = { .extra1 = &zero, .extra2 = &one, }, + { + .procname = "sem_next_id", + .data = &init_ipc_ns.ids[IPC_SEM_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_SEM_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, + { + .procname = "msg_next_id", + .data = &init_ipc_ns.ids[IPC_MSG_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_MSG_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + }, + { + .procname = "shm_next_id", + .data = &init_ipc_ns.ids[IPC_SHM_IDS].next_id, + .maxlen = sizeof(init_ipc_ns.ids[IPC_SHM_IDS].next_id), + .mode = 0644, + .proc_handler = proc_ipc_dointvec_minmax, + .extra1 = &zero, + .extra2 = &int_max, + },
Quoted 9 lines
ipc/ipc_sysctl.c static struct ctl_table ipc_kern_table[] = { · line 259
Quoted 2 lines
ipc/ipc_sysctl.c +0
-0{} };ipc_kern_table[] is (badly) documented in
Documentation/sysctl/kernel.txt. Can we at least mention these controls in there? Better, create a new way of properly documenting each control and document these three in that manner? Better still, document all the other ones as well ;)
Yes, sure. I'll do my best.
Quoted 3 lines
The patch adds these controls to
CONFIG_CHECKPOINT_RESTORE=nkernels. Why is this?
I'll fix this.
Signature · 2 lines
Best regards, Stanislav Kinsbursky
PATCH v8 3/5 ipc: message queue receive cleanup
In reply to Stanislav KinsburskyThis patch moves all message related manipulation into one function msg_fill(). Actually, two functions because of the compat one.
--- include/linux/msg.h | 5 +++-- ipc/compat.c | 45 +++++++++++++++++++-------------------------- ipc/msg.c | 44 +++++++++++++++++++++++--------------------- 3 files changed, 45 insertions(+), 49 deletions(-)
include/linux/msg.h +3 -2
@@ -34,7 +34,8 @@ struct msg_queue {/* Helper routines for sys_msgsnd and sys_msgrcv */ extern long do_msgsnd (int msqid,long mtype,void __user *mtext,size_t msgsz,int msgflg); -extern long do_msgrcv (int msqid,long *pmtype,void __user *mtext, -size_t msgsz,long msgtyp,int msgflg); +extern long do_msgrcv (int msqid,void __user *buf,size_t bufsz,long msgtyp, +int msgflg, +long (*msg_fill)(void __user *,struct msg_msg *,size_t ));#endif /* _LINUX_MSG_H */
ipc/compat.c +19 -26
74 lines not on this page · raw message
ipc/msg.c +23 -21
81 lines not on this page · raw message
PATCH v8 4/5 ipc: message queue copy feature introduced
In reply to Stanislav KinsburskyThis patch is required for checkpoint/restore in userspace.
IOW, c/r requires some way to get all pending IPC messages without deleting them from the queue (checkpoint can fail and in this case tasks will be resumed, so queue have to be valid).
To achive this, new operation flag MSG_COPY for sys_msgrcv() system call was introduced. If this flag was specified, then mtype is interpreted as number of the message to copy.
If MSG_COPY is set, then kernel will allocate dummy message with passed size, and then use new copy_msg() helper function to copy desired message (instead of unlinking it from the queue).
Notes:
1) Return -ENOSYS if MSG_COPY is specified, but CONFIG_CHECKPOINT_RESTORE is not set.
--- include/uapi/linux/msg.h | 1 + ipc/msg.c | 50 ++++++++++++++++++++++++++++++++++++++++++++-- ipc/msgutil.c | 38 +++++++++++++++++++++++++++++++++++ ipc/util.h | 1 + 4 files changed, 88 insertions(+), 2 deletions(-)
include/uapi/linux/msg.h +1 -0
@@ -10,6 +10,7 @@/* msgrcv options */ #define MSG_NOERROR 010000/* no error if message is too big */ #define MSG_EXCEPT 020000/* recv any msg except of specified type.*/ +#define MSG_COPY 040000/* copy (not remove) all queue messages */ /* Obsolete, used only for backwards compatibility and libc5 compiles */ struct msqid_ds {
ipc/msg.c +48 -2
89 lines not on this page · raw message
ipc/msgutil.c +38 -0
@@ -100,7 +100,45 @@ out_err:free_msg (msg);return ERR_PTR (err); } +#ifdef CONFIG_CHECKPOINT_RESTORE +struct msg_msg *copy_msg (struct msg_msg *src,struct msg_msg *dst) +{ +struct msg_msgseg *dst_pseg, *src_pseg; +int len = src->m_ts; +int alen; + +BUG_ON (dst ==NULL ); +if (src->m_ts > dst->m_ts) +return ERR_PTR (-EINVAL ); + + alen = len; +if (alen >DATALEN_MSG ) + alen =DATALEN_MSG ; + + dst->next =NULL ; + dst->security =NULL ; +memcpy (dst +1 , src +1 , alen); + + len -= alen; + dst_pseg = dst->next; + src_pseg = src->next; +while (len >0 ) { + alen = len; +if (alen >DATALEN_SEG ) + alen =DATALEN_SEG ; +memcpy (dst_pseg +1 , src_pseg +1 , alen); + dst_pseg = dst_pseg->next; + len -= alen; + src_pseg = src_pseg->next; + } + + dst->m_type = src->m_type; + dst->m_ts = src->m_ts; + +return dst; +} +#endif int store_msg (void __user *dest,struct msg_msg *msg,int len) {int alen;
ipc/util.h +1 -0
@@ -140,6 +140,7 @@ int ipc_parse_version (int *cmd);extern void free_msg (struct msg_msg *msg);extern struct msg_msg *load_msg (const void __user *src,int len); +extern struct msg_msg *copy_msg (struct msg_msg *src,struct msg_msg *dst);extern int store_msg (void __user *dest,struct msg_msg *msg,int len);extern void recompute_msgmni (struct ipc_namespace *);
Re: [PATCH v8 4/5] ipc: message queue copy feature introduced
Quoted 18 lines · On Wed, 24 Oct 2012 19:35:20 +0400 Stanislav Kinsbursky…
This patch is required for checkpoint/restore in userspace.
IOW, c/r requires some way to get all pending IPC messages without deleting them from the queue (checkpoint can fail and in this case tasks will be resumed, so queue have to be valid).
To achive this, new operation flagMSG_COPYforsys_msgrcv()system call was introduced. If this flag was specified, then mtype is interpreted as number of the message to copy.
IfMSG_COPYis set, then kernel will allocate dummy message with passed size, and then use newcopy_msg()helper function to copy desired message (instead of unlinking it from the queue)....
ipc/msg.c +1
-0@@ -777,19 +777,48 @@ long do_msgrcv(int msqid, void __user *buf, size_t bufsz, long msgtyp,struct msg_msg *msg;int mode;struct ipc_namespace *ns; +#ifdef CONFIG_CHECKPOINT_RESTORE
ipc/msg.c long do_msgrcv(int msqid, void __user *buf, size_t bufsz, long msgtyp, · line 790
Quoted 10 lines
ipc/msg.c +7
-0+struct msg_msg *copy =NULL ; +unsigned long copy_number =0 ; +#endif if (msqid <0 || (long ) bufsz <0 )return -EINVAL ; +if (msgflg &MSG_COPY ) { +#ifdef CONFIG_CHECKPOINT_RESTORE + +if (msgflg &MSG_COPY ) {
This test is't needed.
Quoted 68 lines
ipc/msg.c +34
-2+ copy_number = msgtyp; + msgtyp = 0; + } + +/* +* Create dummy message to copy real message to. +*/ + copy =load_msg (buf, bufsz); +if (IS_ERR (copy)) +return PTR_ERR (copy); + copy->m_ts = bufsz; +#else +return -ENOSYS ; +#endif + } mode =convert_mode (&msgtyp, msgflg); ns = current->nsproxy->ipc_ns; msq =msg_lock_check (ns, msqid); -if (IS_ERR (msq)) +if (IS_ERR (msq)) { +#ifdef CONFIG_CHECKPOINT_RESTORE +if (msgflg &MSG_COPY ) +free_msg (copy); +#endif return PTR_ERR (msq); + }for (;;) {struct msg_receiver msr_d;struct list_head *tmp; +long msg_counter =0 ; msg =ERR_PTR (-EACCES );if (ipcperms (ns, &msq->q_perm,S_IRUGO ))@@ -809,8 +838,16 @@ long do_msgrcv(int msqid, void __user *buf, size_t bufsz, long msgtyp,if (mode ==SEARCH_LESSEQUAL && walk_msg->m_type !=1 ) { msgtyp = walk_msg->m_type -1 ; +#ifdef CONFIG_CHECKPOINT_RESTORE + }else if (msgflg &MSG_COPY ) { +if (copy_number == msg_counter) { + msg =copy_msg (walk_msg, copy); +break ; + } +#endif }else break ; + msg_counter++; } tmp = tmp->next; }@@ -823,6 +860,10 @@ long do_msgrcv(int msqid, void __user *buf, size_t bufsz, long msgtyp, msg =ERR_PTR (-E2BIG ); goto out_unlock; } +#ifdef CONFIG_CHECKPOINT_RESTORE +if (msgflg &MSG_COPY ) + goto out_unlock; +#endif list_del (&msg->m_list); msq->q_qnum--; msq->q_rtime =get_seconds ();@@ -906,8 +947,13 @@ out_unlock:break ; } } -if (IS_ERR (msg))
ipc/msg.c out_unlock: · line 959
Quoted 10 lines
ipc/msg.c +6
-0+if (IS_ERR (msg)) { +#ifdef CONFIG_CHECKPOINT_RESTORE +if (msgflg &MSG_COPY ) +free_msg (copy); +#endif return PTR_ERR (msg); + } bufsz =msg_handler (buf, msg, bufsz);free_msg (msg);
It's all a bit ugly, but I don't really see much we can practically do about that.
You could add something like
#ifdef CONFIG_CHECKPOINT_RESTORE
static inline void free_copy(void *p, int msgflg, struct msg_msg *copy)
{
if (IS_ERR(p) && (msgflg & MSG_COPY))
free_msg(copy);
}#else
/* As a macro because `copy' will be undefined */ #define free_copy(p, msgflg, copy) do {} while (0) #endif
and use that in a couple of places. But that won't help much.
PATCH v8 5/5 test: IPC message queue copy feture test
In reply to Stanislav KinsburskyThis test can be used to check wheither kernel supports IPC message queue copy and restore features (required by CRIU project).
--- tools/testing/selftests/ipc/Makefile | 25 ++++ tools/testing/selftests/ipc/msgque.c | 231 ++++++++++++++++++++++++++++++++++ 2 files changed, 256 insertions(+), 0 deletions(-) create mode 100644 tools/testing/selftests/ipc/Makefile create mode 100644 tools/testing/selftests/ipc/msgque.c
tools/testing/selftests/ipc/Makefile new file +25 -0
@@ -0,0 +1,25 @@ +uname_M := $(shell uname -m 2>/dev/null || echo not) +ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/i386/) +ifeq ($(ARCH),i386) + ARCH := X86 + CFLAGS := -DCONFIG_X86_32 -D__i386__ +endif +ifeq ($(ARCH),x86_64) + ARCH := X86 + CFLAGS := -DCONFIG_X86_64 -D__x86_64__ +endif + +CFLAGS += -I../../../../usr/include/ + +all: +ifeq ($(ARCH),X86) + gcc $(CFLAGS) msgque.c -o msgque_test +else + echo "Not an x86 target, can't build msgque selftest" +endif + +run_tests: all + ./msgque_test + +clean: + rm -fr ./msgque_test
tools/testing/selftests/ipc/msgque.c new file +231 -0
231 lines not on this page · raw message
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav Kinsbursky…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.
OK, thanks, I grabbed these. They're not the worst c/r patches I've ever seen ;)
I'm not seeing any evidence that anyone else has reviewed or tested these?
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav Kinsbursky…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.
According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patch
It's not obvious (to me) that these things have been identified and fixed. What's the status, please?
19.12.2012 00:36, Andrew Morton пишет:
Quoted 18 lines
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav Kinsbursky…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.
Quoted 7 lines
According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patchIt's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you. But I can't find Sasha's issue. As I remember, there was some problem in early version of the patch set. But I believe its fixed now.
Quoted 24 lines · On Thu, 20 Dec 2012 08:06:32 +0400 Stanislav Kinsbursky…
19.12.2012 00:36, Andrew Morton __________:
Quoted 23 lines
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav Kinsbursky…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patch
Quoted 8 lines
Quoted 2 lines
It's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you. But I can't find Sasha's issue. As I remember, there was some problem in early
version of the patch set. But I believe its fixed now.
http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/01710.html
Subject: "ipc, msgqueue: NULL ptr deref in msgrcv"
21.12.2012 00:47, Andrew Morton пишет:
Quoted 29 lines
Quoted 26 lines · On Thu, 20 Dec 2012 08:06:32 +0400 Stanislav…
19.12.2012 00:36, Andrew Morton __________:
Quoted 25 lines
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patchIt's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Quoted 8 lines
Quoted 5 lines
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you. But I can't find Sasha's issue. As I remember, there was some problem in early
version of the patch set. But I believe its fixed now.http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/01710.html
Subject: "ipc, msgqueue: NULL ptr deref in msgrcv"
Ah, yes. Thanks.
Hi found it in initial version of code, which was significantly changed (or cleaned and simplified) by further patch series. And I cant find out, how this can happen, because this patch he bisect to do not modify the queue itself, while he found the problem in testmsg.
Quoted 35 lines · Stanislav Kinsbursky
21.12.2012 00:47, Andrew Morton пишет:
Quoted 34 lines
Quoted 31 lines · On Thu, 20 Dec 2012 08:06:32 +0400 Stanislav…
19.12.2012 00:36, Andrew Morton __________:
Quoted 18 lines
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.Quoted 7 lines
According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patchIt's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you. But I can't find Sasha's issue. As I remember, there was some problem in early
version of the patch set. But I believe its fixed now.
Quoted 8 lines
Quoted 3 lines
http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/01710.html
Subject: "ipc, msgqueue: NULL ptr deref in msgrcv"
Ah, yes. Thanks.
Hi found it in initial version of code, which was significantly changed (or cleaned and simplified) by further patch series. And I cant find out, how this can happen, because this patch he bisect to do not modify the queue itself, while he found the problem in testmsg.
I actually can't reproduce it on the latest -next.
I was reverting the IPC changes in the past couple of weeks so that I could test the rest of the IPC code with the fuzzer, and when I added them back in again I can't reproduce the issue I've reported earlier.
We can probably figure out where it got fixed by bisecting between -next trees if anyone is interested in that.
Thanks,
Sasha
Quoted 45 lines · Sasha Levin
Quoted 35 lines · Stanislav Kinsbursky
21.12.2012 00:47, Andrew Morton пишет:
Quoted 34 lines
Quoted 31 lines · On Thu, 20 Dec 2012 08:06:32 +0400 Stanislav…
19.12.2012 00:36, Andrew Morton __________:
Quoted 18 lines
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.Quoted 7 lines
According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patchIt's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you. But I can't find Sasha's issue. As I remember, there was some problem in early
version of the patch set. But I believe its fixed now.Quoted 8 lines
Quoted 3 lines
http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/01710.html
Subject: "ipc, msgqueue: NULL ptr deref in msgrcv"
Ah, yes. Thanks.
Hi found it in initial version of code, which was significantly changed (or cleaned and simplified) by further patch series. And I cant find out, how this can happen, because this patch he bisect to do not modify the queue itself, while he found the problem in testmsg.
Quoted 8 lines
I actually can't reproduce it on the latest -next.
I was reverting the IPC changes in the past couple of weeks so that I could test the rest of the IPC code with the fuzzer, and when I added them back in again I can't reproduce the issue I've reported earlier.
We can probably figure out where it got fixed by bisecting between -next trees if anyone is interested in that.
Ignore that. It just took more fuzzing to stumble on it again:
[ 103.164594] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010 [ 103.168159] IP: [<ffffffff81937155>] do_msgrcv+0x205/0x540 [ 103.170031] PGD c7cd067 PUD d274067 PMD 0 [ 103.170031] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC [ 103.170031] Dumping ftrace buffer: [ 103.170031] (ftrace buffer empty) [ 103.170031] CPU 4 [ 103.170031] Pid: 7056, comm: trinity Tainted: G W 3.7.0-next-20121221-sasha-00014-g339890c #229 [ 103.170031] RIP: 0010:[<ffffffff81937155>] [<ffffffff81937155>] do_msgrcv+0x205/0x540 [ 103.170031] RSP: 0018:ffff88000c7cfe88 EFLAGS: 00010246 [ 103.170031] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 103.170031] RDX: ffff880013681f00 RSI: 0000000000000124 RDI: ffff8800075a7810 [ 103.170031] RBP: ffff88000c7cff68 R08: 0000000000000000 R09: 0000000000000000 [ 103.170031] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000002 [ 103.170031] R13: ffff8800075a78c0 R14: 7fffffff00000000 R15: ffff8800075a7810 [ 103.170031] FS: 00007ffa529ae700(0000) GS:ffff880013c00000(0000) knlGS:0000000000000000 [ 103.170031] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 103.170031] CR2: 0000000000000010 CR3: 000000000c7cc000 CR4: 00000000000406e0 [ 103.170031] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 103.170031] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 103.170031] Process trinity (pid: 7056, threadinfo ffff88000c7ce000, task ffff88000c020000) [ 103.170031] Stack: [ 103.170031] ffff88000c7cfea8 ffff88000c020000 ffff88000c020000 ffff88000c020000 [ 103.170031] 0000000000000000 ffffffff81935e50 0000000000000008 0000000000000000 [ 103.170031] ffffffff858e91e0 0000000000000000 0000000000001001 ffff88000c020000 [ 103.170031] Call Trace: [ 103.170031] [<ffffffff81935e50>] ? load_msg+0x170/0x170 [ 103.170031] [<ffffffff8107e8c4>] ? syscall_trace_enter+0x24/0x2e0 [ 103.170031] [<ffffffff81184678>] ? trace_hardirqs_on_caller+0x118/0x140 [ 103.170031] [<ffffffff819374a0>] sys_msgrcv+0x10/0x20 [ 103.170031] [<ffffffff83cdf798>] tracesys+0xe1/0xe6 [ 103.170031] Code: 80 f5 ff ff ff 90 41 83 fc 03 74 32 41 83 fc 04 74 0c 41 83 fc 02 75 2c eb 11 0f 1f 40 00 4c 3b 73 10 7d 20 66 90 e9 94 00 00 00 <4c> 39 73 10 0f 85 8a 00 00 00 90 eb 0c 66 0f 1f 44 00 00 4c 39 [ 103.170031] RIP [<ffffffff81937155>] do_msgrcv+0x205/0x540 [ 103.170031] RSP <ffff88000c7cfe88> [ 103.170031] CR2: 0000000000000010 [ 103.228270] ---[ end trace ddc37199fdad82b0 ]---
Thanks,
Sasha
22.12.2012 19:43, Sasha Levin пишет:
Quoted 48 lines · Sasha Levin
Quoted 47 lines · Stanislav Kinsbursky
Quoted 35 lines
21.12.2012 00:47, Andrew Morton пишет:
Quoted 34 lines
Quoted 31 lines · On Thu, 20 Dec 2012 08:06:32 +0400 Stanislav…
19.12.2012 00:36, Andrew Morton __________:
Quoted 18 lines
Quoted 8 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav…
This respin of the patch set was significantly reworked. Most part of new API was replaced by sysctls (by one per messages, semaphores and shared memory), allowing to preset desired id for next new IPC object.
This patch set is aimed to provide additional functionality for all IPC objects, which is required for migration of these objects by user-space checkpoint/restore utils (CRIU).
Quoted 7 lines
The main problem here was impossibility to set up object id. This patch set solves the problem by adding new sysctls for preset of desired id for new IPC object.
Another problem was to peek messages from queues without deleting them. This was achived by introducing of new
MSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.Quoted 7 lines
According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a
bug in ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patchIt's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you. But I can't find Sasha's issue. As I remember, there was some problem in early
version of the patch set. But I believe its fixed now.Quoted 8 lines
Quoted 3 lines
http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/01710.html
Subject: "ipc, msgqueue: NULL ptr deref in msgrcv"
Ah, yes. Thanks.
Hi found it in initial version of code, which was significantly changed (or cleaned and simplified) by further patch series. And I cant find out, how this can happen, because this patch he bisect to do not modify the queue itself, while he found the problem in testmsg.I actually can't reproduce it on the latest -next.
Quoted 9 lines
Quoted 6 lines
I was reverting the IPC changes in the past couple of weeks so that I could test the rest of the IPC code with the fuzzer, and when I added them back in again I can't reproduce the issue I've reported earlier.
We can probably figure out where it got fixed by bisecting between -next trees if anyone is interested in that.
Ignore that. It just took more fuzzing to stumble on it again:
Hello, Sasha!
Thanks!
But I still can't understand, how this can happen... And I can't reproduce. Could you specify your load? I.e. how do you stumble on this panic? Looks like you don't use new "copy" feature.
Quoted 42 lines
[ 103.164594] BUG: unable to handle kernel NULL pointer dereference at 0000000000000010 [ 103.168159] IP: [<ffffffff81937155>] do_msgrcv+0x205/0x540 [ 103.170031] PGD c7cd067 PUD d274067 PMD 0 [ 103.170031] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC [ 103.170031] Dumping ftrace buffer: [ 103.170031] (ftrace buffer empty) [ 103.170031] CPU 4 [ 103.170031] Pid: 7056, comm: trinity Tainted: G W 3.7.0-next-20121221-sasha-00014-g339890c #229 [ 103.170031] RIP: 0010:[<ffffffff81937155>] [<ffffffff81937155>] do_msgrcv+0x205/0x540 [ 103.170031] RSP: 0018:ffff88000c7cfe88 EFLAGS: 00010246 [ 103.170031] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 103.170031] RDX: ffff880013681f00 RSI: 0000000000000124 RDI: ffff8800075a7810 [ 103.170031] RBP: ffff88000c7cff68 R08: 0000000000000000 R09: 0000000000000000 [ 103.170031] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000002 [ 103.170031] R13: ffff8800075a78c0 R14: 7fffffff00000000 R15: ffff8800075a7810 [ 103.170031] FS: 00007ffa529ae700(0000) GS:ffff880013c00000(0000) knlGS:0000000000000000 [ 103.170031] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 103.170031] CR2: 0000000000000010 CR3: 000000000c7cc000 CR4: 00000000000406e0 [ 103.170031] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 103.170031] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 103.170031] Process trinity (pid: 7056, threadinfo ffff88000c7ce000, task ffff88000c020000) [ 103.170031] Stack: [ 103.170031] ffff88000c7cfea8 ffff88000c020000 ffff88000c020000 ffff88000c020000 [ 103.170031] 0000000000000000 ffffffff81935e50 0000000000000008 0000000000000000 [ 103.170031] ffffffff858e91e0 0000000000000000 0000000000001001 ffff88000c020000 [ 103.170031] Call Trace: [ 103.170031] [<ffffffff81935e50>] ? load_msg+0x170/0x170 [ 103.170031] [<ffffffff8107e8c4>] ? syscall_trace_enter+0x24/0x2e0 [ 103.170031] [<ffffffff81184678>] ? trace_hardirqs_on_caller+0x118/0x140 [ 103.170031] [<ffffffff819374a0>] sys_msgrcv+0x10/0x20 [ 103.170031] [<ffffffff83cdf798>] tracesys+0xe1/0xe6 [ 103.170031] Code: 80 f5 ff ff ff 90 41 83 fc 03 74 32 41 83 fc 04 74 0c 41 83 fc 02 75 2c eb 11 0f 1f 40 00 4c 3b 73 10 7d 20 66 90 e9 94 00 00 00 <4c> 39 73 10 0f 85 8a 00 00 00 90 eb 0c 66 0f 1f 44 00 00 4c 39 [ 103.170031] RIP [<ffffffff81937155>] do_msgrcv+0x205/0x540 [ 103.170031] RSP <ffff88000c7cfe88> [ 103.170031] CR2: 0000000000000010 [ 103.228270] ---[ end trace ddc37199fdad82b0 ]---Thanks,
Sasha
Signature · 2 lines
Best regards, Stanislav Kinsbursky
Quoted 83 lines · Stanislav Kinsbursky
22.12.2012 19:43, Sasha Levin пишет:
Quoted 81 lines · Sasha Levin
Quoted 78 lines · Stanislav Kinsbursky
Quoted 57 lines
21.12.2012 00:47, Andrew Morton пишет:
Quoted 50 lines
Quoted 38 lines · On Thu, 20 Dec 2012 08:06:32 +0400 Stanislav…
19.12.2012 00:36, Andrew Morton __________:
Quoted 28 lines
Quoted 18 lines · On Wed, 24 Oct 2012 19:34:51 +0400 Stanislav…
This respin of the patch set was significantly reworked. Most part of new API
was replaced by sysctls (by one per messages, semaphores and shared memory),
allowing to preset desired id for next new IPC object.This patch set is aimed to provide additional functionality for all IPC
objects, which is required for migration of these objects by user-space
checkpoint/restore utils (CRIU).The main problem here was impossibility to set up object id. This patch set
solves the problem by adding new sysctls for preset of desired id for new IPC
object.Quoted 5 lines
Another problem was to peek messages from queues without deleting them.
This was achived by introducing of newMSG_COPYflag forsys_msgrcv(). IfMSG_COPYflag is set, then msgtyp is interpreted as message number.Quoted 8 lines
According to my extensive records, Sasha hit a bug in ipc-message-queue-copy-feature-introduced.patch and Fengguang found a bug in
ipc-message-queue-copy-feature-introduced-cleanup-do_msgrcv-aroung-msg_copy-feature.patch
It's not obvious (to me) that these things have been identified and fixed. What's the status, please?
Quoted 7 lines
Hello, Andrew.
Fengguang's issue was solved by "ipc: simplify message copying" I sent you.
But I can't find Sasha's issue. As I remember, there was some problem in
early
version of the patch set. But I believe its fixed now.Quoted 3 lines
http://lkml.indiana.edu/hypermail/linux/kernel/1210.3/01710.html
Subject: "ipc, msgqueue: NULL ptr deref in msgrcv"
Quoted 6 lines
Ah, yes. Thanks.
Hi found it in initial version of code, which was significantly changed (or cleaned and simplified) by further patch series.
And I cant find out, how this can happen, because this patch he bisect to do not modify the queue itself, while he found the problem in testmsg.I actually can't reproduce it on the latest -next.
I was reverting the IPC changes in the past couple of weeks so that I could test the
rest of the IPC code with the fuzzer, and when I added them back in again I can't
reproduce the issue I've reported earlier.We can probably figure out where it got fixed by bisecting between -next trees if anyone
is interested in that.
Quoted 8 lines
Quoted 2 lines
Ignore that. It just took more fuzzing to stumble on it again:
Hello, Sasha!
Thanks!
But I still can't understand, how this can happen... And I can't reproduce. Could you specify your load? I.e. how do you stumble on this panic? Looks like you don't use new "copy" feature.
I just fuzz with trinity
(http://git.codemonkey.org.uk/?p=trinity.git;a=tree) inside a pretty basic KVM guest.
Thanks,
Sasha