Skip to contentSkip to conversation

C/R netdev and netns support

PATCH In review Dan Smith 3 participants 14 messages Mar 18, 2010 – Mar 30, 2010 5 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Dan Smith

author
#0 lore raw

This is just the same patch set that has already been reviewed and acked, except for the following:

1. Rebased on v20
2. Storing the IP addresses in network byte order

A message that is not in the archive (sent off-list or not yet received): 1268945512-18814-1-git-send-email-danms-r/[email protected]

author
#2 lore raw

PATCH 1/7 Make restore_obj() tolerate a preexisting object in the hash (v2)

... as long as the pointer is the same as that returned from the restore function. Also move the compulsory ref_drop() so that it only gets done if we created the new object.

The existing object tolerance is important for netdev restore because it means that I can refer to a peer by its objref instead of needing the (previously-rejected) veth_peer() function. If this is not acceptable, then I'll need to keep a separate list of pairs.

Changes in v2:
- Check that the type of the object already in the hash matches that of the objref header we're reading.
- Add a comment about why and how we might get into this sort of situation.

---
 checkpoint/objhash.c |   31 ++++++++++++++++++++++---------
 1 files changed, 22 insertions(+), 9 deletions(-)
checkpoint/objhash.c +22 -9
index 7208382..65875e3 100644
@@ -1064,16 +1064,29 @@ int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h)
 	if (IS_ERR(ptr))
 		return PTR_ERR(ptr);
 
-	if (obj_find_by_objref(ctx, h->objref))
-		obj = ERR_PTR(-EINVAL);
-	else
+	obj = obj_find_by_objref(ctx, h->objref);
+	if (!obj) {
 		obj = obj_new(ctx, ptr, h->objref, h->objtype);
-	/*
-	 * Drop an extra reference to the object returned by ops->restore:
-	 * On success, this clears the extra reference taken by obj_new(),
-	 * and on failure, this cleans up the object itself.
-	 */
-	ops->ref_drop(ptr, 0);
+		/*
+		 * Drop an extra reference to the object returned by
+		 * ops->restore: On success, this clears the extra
+		 * reference taken by obj_new(), and on failure, this
+		 * cleans up the object itself.
+		 */
+		ops->ref_drop(ptr, 0);
+	} else if ((obj->ptr != ptr) || (obj->ops->obj_type != h->objtype)) {
+		/* Normally, we expect an object to not already exist
+		 * in the hash.  However, for some special scenarios
+		 * where we're restoring sets of objects that must be
+		 * co-allocated (such, as veth netdev pairs) we need
+		 * to tolerate this case if the second restore returns
+		 * the correct type and pointer, as specified in the
+		 * existing object.  If either of those don't match,
+		 * we fail.
+		 */
+		obj = ERR_PTR(-EINVAL);
+	}
+
 	if (IS_ERR(obj)) {
 		ops->ref_drop(ptr, 1);
 		return PTR_ERR(obj);
Signature · 1 line
1.6.2.5

A message that is not in the archive (sent off-list or not yet received): 1268945512-18814-2-git-send-email-danms-r/[email protected]

#4 lore raw

Re: [PATCH 1/7] Make restore_obj() tolerate a preexisting object in the hash (v2)

Quoted 35 lines · Dan Smith

... as long as the pointer is the same as that returned from the restore function. Also move the compulsory ref_drop() so that it only gets done if we created the new object.

The existing object tolerance is important for netdev restore because it means that I can refer to a peer by its objref instead of needing the (previously-rejected) veth_peer() function. If this is not acceptable, then I'll need to keep a separate list of pairs.

Changes in v2:
- Check that the type of the object already in the hash matches that of the objref header we're reading.
- Add a comment about why and how we might get into this sort of situation.

---
 checkpoint/objhash.c |   31 ++++++++++++++++++++++---------
 1 files changed, 22 insertions(+), 9 deletions(-)
checkpoint/objhash.c +2 -4
index 7208382..65875e3 100644
@@ -1064,16 +1064,29 @@ int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h)
 	if (IS_ERR(ptr))
 		return PTR_ERR(ptr);
 
-	if (obj_find_by_objref(ctx, h->objref))
-		obj = ERR_PTR(-EINVAL);
-	else
+	obj = obj_find_by_objref(ctx, h->objref);
+	if (!obj) {
 		obj = obj_new(ctx, ptr, h->objref, h->objtype);
-	/*

checkpoint/objhash.c int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h) · line 1074

Quoted 10 lines
checkpoint/objhash.c +5 -5
-	 * Drop an extra reference to the object returned by ops->restore:
-	 * On success, this clears the extra reference taken by obj_new(),
-	 * and on failure, this cleans up the object itself.
-	 */
-	ops->ref_drop(ptr, 0);
+		/*
+		 * Drop an extra reference to the object returned by
+		 * ops->restore: On success, this clears the extra
+		 * reference taken by obj_new(), and on failure, this
+		 * cleans up the object itself.

I don't think this part of the comment is quite right here. The "on failure this cleans up the object itself" really is for the ref_drop under IS_ERR() check below.

The ref_drop here is for the ref taken by obj_new(), which is only done in this path of course.

Quoted 6 lines
checkpoint/objhash.c +6 -0
+		 */
+		ops->ref_drop(ptr, 0);
+	} else if ((obj->ptr != ptr) || (obj->ops->obj_type != h->objtype)) {
+		/* Normally, we expect an object to not already exist
+		 * in the hash.  However, for some special scenarios
+		 * where we're restoring sets of objects that must be

checkpoint/objhash.c int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h) · line 1090

Quoted 10 lines
checkpoint/objhash.c +9 -0
+		 * co-allocated (such, as veth netdev pairs) we need
+		 * to tolerate this case if the second restore returns
+		 * the correct type and pointer, as specified in the
+		 * existing object.  If either of those don't match,
+		 * we fail.
+		 */
+		obj = ERR_PTR(-EINVAL);
+	}
+
 	if (IS_ERR(obj)) {

Here point out that we are putting the reference taken by ops->restore(). If obj is not an error, then we keep an extra ref to pin obj while it is on the hash.

checkpoint/objhash.c int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h) · line 1092

Quoted 2 lines
checkpoint/objhash.c +0 -0
 		ops->ref_drop(ptr, 1);
 		return PTR_ERR(obj);

-serge

A message that is not in the archive (sent off-list or not yet received): 20100319000434.GA5505-r/[email protected]

author
#6 lore raw

Re: [PATCH 1/7] Make restore_obj() tolerate a preexisting object in the hash (v2)

SH> I don't think this part of the comment is quite right here. The SH> "on failure this cleans up the object itself" really is for the SH> ref_drop under IS_ERR() check below.

SH> The ref_drop here is for the ref taken by obj_new(), which is only SH> done in this path of course.

As just discussed on IRC, I couldn't really correct the comment because the logic it was describing was incorrect. Below is a version with the correct comment *and* logic :)

Signature · 26 lines
Dan Smith
IBM Linux Technology Center
email: danms-r/[email protected]

Make restore_obj() tolerate a preexisting object in the hash (v3)

... as long as the pointer is the same as that returned from the restore
function.  Also move the compulsory ref_drop() so that it only gets
done if we created the new object.

The existing object tolerance is important for netdev restore because it
means that I can refer to a peer by its objref instead of needing the
(previously-rejected) veth_peer() function.  If this is not acceptable,
then I'll need to keep a separate list of pairs.

Changes in v3:
 - Fix the logic in the case where we need to do an obj_new() and fail,
   so that we don't do ref_drop() twice

Changes in v2:
 - Check that the type of the object already in the hash matches that
   of the objref header we're reading.
 - Add a comment about why and how we might get into this sort of
   situation.

Signed-off-by: Dan Smith <danms-r/[email protected]>
checkpoint/objhash.c +24 -9
index 7208382..3b360cb 100644
@@ -1064,17 +1064,32 @@ int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h)
 	if (IS_ERR(ptr))
 		return PTR_ERR(ptr);
 
-	if (obj_find_by_objref(ctx, h->objref))
-		obj = ERR_PTR(-EINVAL);
-	else
+	obj = obj_find_by_objref(ctx, h->objref);
+	if (!obj) {
 		obj = obj_new(ctx, ptr, h->objref, h->objtype);
-	/*
-	 * Drop an extra reference to the object returned by ops->restore:
-	 * On success, this clears the extra reference taken by obj_new(),
-	 * and on failure, this cleans up the object itself.
-	 */
-	ops->ref_drop(ptr, 0);
+		/*
+		 * Drop an extra reference to the object returned by
+		 * ops->restore to balance the one taken by obj_new()
+		 */
+		if (!IS_ERR(obj))
+			ops->ref_drop(ptr, 0);
+	} else if ((obj->ptr != ptr) || (obj->ops->obj_type != h->objtype)) {
+		/* Normally, we expect an object to not already exist
+		 * in the hash.  However, for some special scenarios
+		 * where we're restoring sets of objects that must be
+		 * co-allocated (such, as veth netdev pairs) we need
+		 * to tolerate this case if the second restore returns
+		 * the correct type and pointer, as specified in the
+		 * existing object.  If either of those don't match,
+		 * we fail.
+		 */
+		obj = ERR_PTR(-EINVAL);
+	}
+
 	if (IS_ERR(obj)) {
+		/* This releases our final reference on the object
+		 * returned by ops->restore()
+		 */
 		ops->ref_drop(ptr, 1);
 		return PTR_ERR(obj);
 	}

A message that is not in the archive (sent off-list or not yet received): 8739zwgxua.fsf-FLMGYpZoEPULwtHQx/[email protected]

#8 lore raw

Re: [PATCH 1/7] Make restore_obj() tolerate a preexisting object in the hash (v2)

Quoted 4 lines · Dan Smith

SH> I don't think this part of the comment is quite right here. The SH> "on failure this cleans up the object itself" really is for the SH> ref_drop under IS_ERR() check below.

Quoted 34 lines

SH> The ref_drop here is for the ref taken by obj_new(), which is only SH> done in this path of course.

As just discussed on IRC, I couldn't really correct the comment because the logic it was describing was incorrect. Below is a version with the correct comment *and* logic :)

Signature · 26 lines
Dan Smith
IBM Linux Technology Center
email: danms-r/[email protected]

Make restore_obj() tolerate a preexisting object in the hash (v3)

... as long as the pointer is the same as that returned from the restore
function.  Also move the compulsory ref_drop() so that it only gets
done if we created the new object.

The existing object tolerance is important for netdev restore because it
means that I can refer to a peer by its objref instead of needing the
(previously-rejected) veth_peer() function.  If this is not acceptable,
then I'll need to keep a separate list of pairs.

Changes in v3:
 - Fix the logic in the case where we need to do an obj_new() and fail,
   so that we don't do ref_drop() twice

Changes in v2:
 - Check that the type of the object already in the hash matches that
   of the objref header we're reading.
 - Add a comment about why and how we might get into this sort of
   situation.

Signed-off-by: Dan Smith <danms-r/[email protected]>

Thanks, Dan, for also fixing pre-existing bug :)

-serge

Quoted 47 lines
checkpoint/objhash.c +24 -9
index 7208382..3b360cb 100644
@@ -1064,17 +1064,32 @@ int restore_obj(struct ckpt_ctx *ctx, struct ckpt_hdr_objref *h)
 	if (IS_ERR(ptr))
 		return PTR_ERR(ptr);
 
-	if (obj_find_by_objref(ctx, h->objref))
-		obj = ERR_PTR(-EINVAL);
-	else
+	obj = obj_find_by_objref(ctx, h->objref);
+	if (!obj) {
 		obj = obj_new(ctx, ptr, h->objref, h->objtype);
-	/*
-	 * Drop an extra reference to the object returned by ops->restore:
-	 * On success, this clears the extra reference taken by obj_new(),
-	 * and on failure, this cleans up the object itself.
-	 */
-	ops->ref_drop(ptr, 0);
+		/*
+		 * Drop an extra reference to the object returned by
+		 * ops->restore to balance the one taken by obj_new()
+		 */
+		if (!IS_ERR(obj))
+			ops->ref_drop(ptr, 0);
+	} else if ((obj->ptr != ptr) || (obj->ops->obj_type != h->objtype)) {
+		/* Normally, we expect an object to not already exist
+		 * in the hash.  However, for some special scenarios
+		 * where we're restoring sets of objects that must be
+		 * co-allocated (such, as veth netdev pairs) we need
+		 * to tolerate this case if the second restore returns
+		 * the correct type and pointer, as specified in the
+		 * existing object.  If either of those don't match,
+		 * we fail.
+		 */
+		obj = ERR_PTR(-EINVAL);
+	}
+
 	if (IS_ERR(obj)) {
+		/* This releases our final reference on the object
+		 * returned by ops->restore()
+		 */
 		ops->ref_drop(ptr, 1);
 		return PTR_ERR(obj);
 	}
author
#9 lore raw

PATCH 2/7 Add checkpoint and collect hooks to net_device_ops

In reply to a message not in the archive

These will be implemented per-driver by those that support such operations.

---
 include/linux/netdevice.h |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
include/linux/netdevice.h +6 -0
index a3fccc8..415a791 100644
@@ -673,6 +673,12 @@ struct net_device_ops {
 	int			(*ndo_fcoe_get_wwn)(struct net_device *dev,
 						    u64 *wwn, int type);
 #endif
+#ifdef CONFIG_CHECKPOINT
+	int			(*ndo_collect)(struct ckpt_ctx *ctx,
+					       struct net_device *dev);
+	int			(*ndo_checkpoint)(struct ckpt_ctx *ctx,
+						  struct net_device *dev);
+#endif
 };
 
 /*
Signature · 1 line
1.6.2.5
author
#10 lore raw

PATCH 3/7 C/R: Basic support for network namespaces and devices (v6)

In reply to a message not in the archive

When checkpointing a task tree with network namespaces, we hook into do_checkpoint_ns() along with the others. Any devices in a given namespace are checkpointed (including their peer, in the case of veth) sequentially. Each network device stores a list of protocol addresses, as well as other information, such as hardware address.

This patch supports veth pairs, as well as the loopback adapter. The loopback support is there to make sure that any additional addresses and state (such as up/down) is copied to the loopback adapter that we are given in the new network namespace.

On restart, we instantiate new network namespaces and veth pairs as necessary. Any device we encounter that isn't in a network namespace that was checkpointed as part of a task is left in the namespace of the restarting process. This will be the case for a veth half that exists in the init netns to provide network access to a container.

Still to do are:

1. Routes
2. Netfilter rules
3. IPv6 addresses
4. Other virtual device types (e.g. bridges)
5. Multicast
6. Device config info (ipv4_devconf)
7. Additional ipv4 address attributes

Changes in v6:
- Store addresses in network byte order, per Dave's recommendation

Changes in v5:
- Rebase
- Remove checkpoint_container() noise
- Factor out some common bits of the RTNL newlink operations
- Add macvlan support

Changes in v4:
- Fix allocation under lock in ckpt_netdev_inet_addrs()
- Add comment for case where there is no netns info in checkpoint image
- Fix inner structure alignment in netdev_addr header
- Fix instances of kfree(skb)
- Remove init_netns_ref from container header and checkpoint context
- Add 'extern' to checkpoint.h prototypes
- Swizzle do_restore_netns() to handle netns more like the others
- Return E2BIG for failure case when collecting inet addrs
- Report case where device doesn't support checkpoint
- Remove nested netns check from may_checkpoint_task()
- Move veth-specific netdev attributes into unioned struct to set an example for specific attributes of additional device types
- Add 'sit' device restore path that doesn't really do anything
- Fail instead of skip when encountering a device with no checkpoint support

Changes in v3:
- Use dev->checkpoint() for per-device checkpoint operation
- Use RTNL for veth pair creation on restart
- Export some of the functions that will be needed by dev->ndo_checkpoint()

Changes in v2:
- Add CONFIG_CHECKPOINT_NETNS that is dependent on NET, NET_NS, and CHECKPOINT. Conditionally compile the checkpoint_dev code based on it.
- Updated comment on should_checkpoint_netdev()
- Updated checkpoint_netdev() to explicitly check for "veth" in name
- Changed checkpoint_netns() to use BUG() for impossible condition
- Fixed a bug on restart with all devices in the init netns
- Lock the dev_base_lock while traversing interface addresses
- Collect all addresses for an interface before writing out in one single pass

---
 checkpoint/checkpoint.c          |    6 +-
 checkpoint/objhash.c             |   48 +++
 include/linux/checkpoint.h       |   23 ++
 include/linux/checkpoint_hdr.h   |   58 +++
 include/linux/checkpoint_types.h |    1 +
 kernel/nsproxy.c                 |   20 +-
 net/Kconfig                      |    4 +
 net/Makefile                     |    1 +
 net/checkpoint_dev.c             |  815 ++++++++++++++++++++++++++++++++++++++
 9 files changed, 970 insertions(+), 6 deletions(-)
 create mode 100644 net/checkpoint_dev.c
checkpoint/checkpoint.c +1 -5
index f27af41..4359106 100644
@@ -188,6 +188,7 @@ static int checkpoint_container(struct ckpt_ctx *ctx)
 	h = ckpt_hdr_get_type(ctx, sizeof(*h), CKPT_HDR_CONTAINER);
 	if (!h)
 		return -ENOMEM;
+
 	ret = ckpt_write_obj(ctx, &h->h);
 	ckpt_hdr_put(ctx, h);
 
@@ -288,11 +289,6 @@ static int may_checkpoint_task(struct ckpt_ctx *ctx, struct task_struct *t)
 		_ckpt_err(ctx, -EPERM, "%(T)Nested mnt_ns unsupported\n");
 		ret = -EPERM;
 	}
-	/* no support for >1 private netns */
-	if (nsproxy->net_ns != ctx->root_nsproxy->net_ns) {
-		_ckpt_err(ctx, -EPERM, "%(T)Nested net_ns unsupported\n");
-		ret = -EPERM;
-	}
 	/* no support for >1 private pidns */
 	if (nsproxy->pid_ns != ctx->root_nsproxy->pid_ns) {
 		_ckpt_err(ctx, -EPERM, "%(T)Nested pid_ns unsupported\n");
checkpoint/objhash.c +48 -0

60 lines not on this page · raw message

include/linux/checkpoint.h +23 -0
index 792b523..0fb647b 100644
@@ -35,6 +35,7 @@
 #include <linux/checkpoint_types.h>
 #include <linux/checkpoint_hdr.h>
 #include <linux/err.h>
+#include <linux/inetdevice.h>
 #include <net/sock.h>
 
 /* sycall helpers */
@@ -119,6 +120,28 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx,
 extern struct sk_buff *sock_restore_skb(struct ckpt_ctx *ctx, struct sock *sk);
 extern void sock_listening_list_free(struct list_head *head);
 
+#ifdef CONFIG_CHECKPOINT_NETNS
+extern int checkpoint_netns(struct ckpt_ctx *ctx, void *ptr);
+extern void *restore_netns(struct ckpt_ctx *ctx);
+extern int checkpoint_netdev(struct ckpt_ctx *ctx, void *ptr);
+extern void *restore_netdev(struct ckpt_ctx *ctx);
+
+extern int ckpt_netdev_in_init_netns(struct ckpt_ctx *ctx,
+				     struct net_device *dev);
+extern int ckpt_netdev_inet_addrs(struct in_device *indev,
+				  struct ckpt_netdev_addr *list[]);
+extern int ckpt_netdev_hwaddr(struct net_device *dev,
+			      struct ckpt_hdr_netdev *h);
+extern struct ckpt_hdr_netdev *ckpt_netdev_base(struct ckpt_ctx *ctx,
+						struct net_device *dev,
+						struct ckpt_netdev_addr *addrs[]);
+#else
+# define checkpoint_netns NULL
+# define restore_netns NULL
+# define checkpoint_netdev NULL
+# define restore_netdev NULL
+#endif
+
 /* ckpt kflags */
 #define ckpt_set_ctx_kflag(__ctx, __kflag)  \
 	set_bit(__kflag##_BIT, &(__ctx)->kflags)
include/linux/checkpoint_hdr.h +58 -0

88 lines not on this page · raw message

include/linux/checkpoint_types.h +1 -0
index ecd3e91..e9cc1d8 100644
@@ -82,6 +82,7 @@ struct ckpt_ctx {
 	wait_queue_head_t ghostq;	/* waitqueue for ghost tasks */
 	struct cred *realcred, *ecred;	/* tmp storage for cred at restart */
 	struct list_head listen_sockets;/* listening parent sockets */
+	int init_netns_ref;             /* Objref of root net namespace */
 
 	struct ckpt_stats stats;	/* statistics */
 
kernel/nsproxy.c +19 -1
index 0da0d83..b0e67ff 100644
@@ -248,6 +248,11 @@ int ckpt_collect_ns(struct ckpt_ctx *ctx, struct task_struct *t)
 	ret = ckpt_obj_collect(ctx, nsproxy->uts_ns, CKPT_OBJ_UTS_NS);
 	if (ret < 0)
 		goto out;
+#ifdef CONFIG_CHECKPOINT_NETNS
+	ret = ckpt_obj_collect(ctx, nsproxy->net_ns, CKPT_OBJ_NET_NS);
+	if (ret < 0)
+		goto out;
+#endif
 	ret = ckpt_obj_collect(ctx, nsproxy->ipc_ns, CKPT_OBJ_IPC_NS);
 	if (ret < 0)
 		goto out;
@@ -288,6 +293,12 @@ static int do_checkpoint_ns(struct ckpt_ctx *ctx, struct nsproxy *nsproxy)
 	if (ret < 0)
 		goto out;
 	h->ipc_objref = ret;
+#ifdef CONFIG_CHECKPOINT_NETNS
+	ret = checkpoint_obj(ctx, nsproxy->net_ns, CKPT_OBJ_NET_NS);
+	if (ret < 0)
+		goto out;
+	h->net_objref = ret;
+#endif
 
 	/* FIXME: for now, only marked visited to pacify leaks */
 	ret = ckpt_obj_visit(ctx, nsproxy->mnt_ns, CKPT_OBJ_MNT_NS);
@@ -328,6 +339,14 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx)
 		ret = PTR_ERR(uts_ns);
 		goto out;
 	}
+	if (h->net_objref == 0)
+		net_ns = current->nsproxy->net_ns;
+	else
+		net_ns = ckpt_obj_fetch(ctx, h->net_objref, CKPT_OBJ_NET_NS);
+	if (IS_ERR(net_ns)) {
+		ret = PTR_ERR(net_ns);
+		goto out;
+	}
 
 	if (h->ipc_objref == 0)
 		ipc_ns = ctx->root_nsproxy->ipc_ns;
@@ -339,7 +358,6 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx)
 	}
 
 	mnt_ns = ctx->root_nsproxy->mnt_ns;
-	net_ns = ctx->root_nsproxy->net_ns;
 
 	if (uts_ns == current->nsproxy->uts_ns &&
 	    ipc_ns == current->nsproxy->ipc_ns &&
net/Kconfig +4 -0
index 041c35e..64dd3cd 100644
@@ -276,4 +276,8 @@ source "net/wimax/Kconfig"
 source "net/rfkill/Kconfig"
 source "net/9p/Kconfig"
 
+config CHECKPOINT_NETNS
+       bool
+       default y if NET && NET_NS && CHECKPOINT
+
 endif   # if NET
net/Makefile +1 -0
index 74b038f..570ee98 100644
@@ -67,3 +67,4 @@ endif
 obj-$(CONFIG_WIMAX)		+= wimax/
 
 obj-$(CONFIG_CHECKPOINT)	+= checkpoint.o
+obj-$(CONFIG_CHECKPOINT_NETNS)	+= checkpoint_dev.o
net/checkpoint_dev.c new file +815 -0

815 lines not on this page · raw message

Signature · 1 line
1.6.2.5

A message that is not in the archive (sent off-list or not yet received): 1268945512-18814-4-git-send-email-danms-r/[email protected]

#12 lore raw

Re: [PATCH 3/7] C/R: Basic support for network namespaces and devices (v6)

Quoted 1 line · Dan Smith

- Remove init_netns_ref from container header and checkpoint context

...

Quoted 8 lines
include/linux/checkpoint_hdr.h +1 -0
@@ -313,6 +323,7 @@ struct ckpt_hdr_tail {
 /* container configuration section header */
 struct ckpt_hdr_container {
 	struct ckpt_hdr h;
+	__s32 init_netns_ref;
 	/*
 	 * the header is followed by the string:
 	 *   char lsm_name[SECURITY_NAME_MAX + 1]

...

include/linux/checkpoint_types.h struct ckpt_ctx { · line 88

Quoted 9 lines

+++ b/include/linux/checkpoint_types.h

include/linux/checkpoint_types.h +1 -0
@@ -82,6 +82,7 @@ struct ckpt_ctx {
 	wait_queue_head_t ghostq;	/* waitqueue for ghost tasks */
 	struct cred *realcred, *ecred;	/* tmp storage for cred at restart */
 	struct list_head listen_sockets;/* listening parent sockets */
+	int init_netns_ref;             /* Objref of root net namespace */
 
 	struct ckpt_stats stats;	/* statistics */
 

Near as I can tell by grepping you don't seem to actually set or use init_netns_ref anywhere. Did it stay in these headers by accident?

-serge

author
#13 lore raw

Re: [PATCH 3/7] C/R: Basic support for network namespaces and devices (v6)

SH> Near as I can tell by grepping you don't seem to actually set or SH> use init_netns_ref anywhere. Did it stay in these headers by SH> accident?

Yep, thanks. The funny thing is, removing that bit is in the changelog, so it must have crept back in when I was rebasing things.

Fixed below.

Signature · 77 lines
Dan Smith
IBM Linux Technology Center
email: danms-r/[email protected]

C/R: Basic support for network namespaces and devices (v6)

When checkpointing a task tree with network namespaces, we hook into
do_checkpoint_ns() along with the others.  Any devices in a given namespace
are checkpointed (including their peer, in the case of veth) sequentially.
Each network device stores a list of protocol addresses, as well as other
information, such as hardware address.

This patch supports veth pairs, as well as the loopback adapter.  The
loopback support is there to make sure that any additional addresses and
state (such as up/down) is copied to the loopback adapter that we are
given in the new network namespace.

On restart, we instantiate new network namespaces and veth pairs as
necessary.  Any device we encounter that isn't in a network namespace
that was checkpointed as part of a task is left in the namespace of the
restarting process.  This will be the case for a veth half that exists
in the init netns to provide network access to a container.

Still to do are:

  1. Routes
  2. Netfilter rules
  3. IPv6 addresses
  4. Other virtual device types (e.g. bridges)
  5. Multicast
  6. Device config info (ipv4_devconf)
  7. Additional ipv4 address attributes

Changes in v6:
 - Store addresses in network byte order, per Dave's recommendation

Changes in v5:
 - Rebase
 - Remove checkpoint_container() noise
 - Factor out some common bits of the RTNL newlink operations
 - Add macvlan support

Changes in v4:
 - Fix allocation under lock in ckpt_netdev_inet_addrs()
 - Add comment for case where there is no netns info in checkpoint image
 - Fix inner structure alignment in netdev_addr header
 - Fix instances of kfree(skb)
 - Remove init_netns_ref from container header and checkpoint context
 - Add 'extern' to checkpoint.h prototypes
 - Swizzle do_restore_netns() to handle netns more like the others
 - Return E2BIG for failure case when collecting inet addrs
 - Report case where device doesn't support checkpoint
 - Remove nested netns check from may_checkpoint_task()
 - Move veth-specific netdev attributes into unioned struct to set an
   example for specific attributes of additional device types
 - Add 'sit' device restore path that doesn't really do anything
 - Fail instead of skip when encountering a device with no checkpoint
   support

Changes in v3:
 - Use dev->checkpoint() for per-device checkpoint operation
 - Use RTNL for veth pair creation on restart
 - Export some of the functions that will be needed by dev->ndo_checkpoint()

Changes in v2:
 - Add CONFIG_CHECKPOINT_NETNS that is dependent on NET, NET_NS, and
   CHECKPOINT.  Conditionally compile the checkpoint_dev code based on it.
 - Updated comment on should_checkpoint_netdev()
 - Updated checkpoint_netdev() to explicitly check for "veth" in name
 - Changed checkpoint_netns() to use BUG() for impossible condition
 - Fixed a bug on restart with all devices in the init netns
 - Lock the dev_base_lock while traversing interface addresses
 - Collect all addresses for an interface before writing out in one
   single pass

Signed-off-by: Dan Smith <danms-r/[email protected]>
Acked-by: David S. Miller <davem-fT/PcQaiUtIeIZ0/[email protected]>
checkpoint/checkpoint.c +1 -5
index f27af41..4359106 100644
@@ -188,6 +188,7 @@ static int checkpoint_container(struct ckpt_ctx *ctx)
 	h = ckpt_hdr_get_type(ctx, sizeof(*h), CKPT_HDR_CONTAINER);
 	if (!h)
 		return -ENOMEM;
+
 	ret = ckpt_write_obj(ctx, &h->h);
 	ckpt_hdr_put(ctx, h);
 
@@ -288,11 +289,6 @@ static int may_checkpoint_task(struct ckpt_ctx *ctx, struct task_struct *t)
 		_ckpt_err(ctx, -EPERM, "%(T)Nested mnt_ns unsupported\n");
 		ret = -EPERM;
 	}
-	/* no support for >1 private netns */
-	if (nsproxy->net_ns != ctx->root_nsproxy->net_ns) {
-		_ckpt_err(ctx, -EPERM, "%(T)Nested net_ns unsupported\n");
-		ret = -EPERM;
-	}
 	/* no support for >1 private pidns */
 	if (nsproxy->pid_ns != ctx->root_nsproxy->pid_ns) {
 		_ckpt_err(ctx, -EPERM, "%(T)Nested pid_ns unsupported\n");
checkpoint/objhash.c +48 -0

60 lines not on this page · raw message

include/linux/checkpoint.h +23 -0
index 792b523..0fb647b 100644
@@ -35,6 +35,7 @@
 #include <linux/checkpoint_types.h>
 #include <linux/checkpoint_hdr.h>
 #include <linux/err.h>
+#include <linux/inetdevice.h>
 #include <net/sock.h>
 
 /* sycall helpers */
@@ -119,6 +120,28 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx,
 extern struct sk_buff *sock_restore_skb(struct ckpt_ctx *ctx, struct sock *sk);
 extern void sock_listening_list_free(struct list_head *head);
 
+#ifdef CONFIG_CHECKPOINT_NETNS
+extern int checkpoint_netns(struct ckpt_ctx *ctx, void *ptr);
+extern void *restore_netns(struct ckpt_ctx *ctx);
+extern int checkpoint_netdev(struct ckpt_ctx *ctx, void *ptr);
+extern void *restore_netdev(struct ckpt_ctx *ctx);
+
+extern int ckpt_netdev_in_init_netns(struct ckpt_ctx *ctx,
+				     struct net_device *dev);
+extern int ckpt_netdev_inet_addrs(struct in_device *indev,
+				  struct ckpt_netdev_addr *list[]);
+extern int ckpt_netdev_hwaddr(struct net_device *dev,
+			      struct ckpt_hdr_netdev *h);
+extern struct ckpt_hdr_netdev *ckpt_netdev_base(struct ckpt_ctx *ctx,
+						struct net_device *dev,
+						struct ckpt_netdev_addr *addrs[]);
+#else
+# define checkpoint_netns NULL
+# define restore_netns NULL
+# define checkpoint_netdev NULL
+# define restore_netdev NULL
+#endif
+
 /* ckpt kflags */
 #define ckpt_set_ctx_kflag(__ctx, __kflag)  \
 	set_bit(__kflag##_BIT, &(__ctx)->kflags)
include/linux/checkpoint_hdr.h +57 -0

81 lines not on this page · raw message

kernel/nsproxy.c +19 -1
index 0da0d83..b0e67ff 100644
@@ -248,6 +248,11 @@ int ckpt_collect_ns(struct ckpt_ctx *ctx, struct task_struct *t)
 	ret = ckpt_obj_collect(ctx, nsproxy->uts_ns, CKPT_OBJ_UTS_NS);
 	if (ret < 0)
 		goto out;
+#ifdef CONFIG_CHECKPOINT_NETNS
+	ret = ckpt_obj_collect(ctx, nsproxy->net_ns, CKPT_OBJ_NET_NS);
+	if (ret < 0)
+		goto out;
+#endif
 	ret = ckpt_obj_collect(ctx, nsproxy->ipc_ns, CKPT_OBJ_IPC_NS);
 	if (ret < 0)
 		goto out;
@@ -288,6 +293,12 @@ static int do_checkpoint_ns(struct ckpt_ctx *ctx, struct nsproxy *nsproxy)
 	if (ret < 0)
 		goto out;
 	h->ipc_objref = ret;
+#ifdef CONFIG_CHECKPOINT_NETNS
+	ret = checkpoint_obj(ctx, nsproxy->net_ns, CKPT_OBJ_NET_NS);
+	if (ret < 0)
+		goto out;
+	h->net_objref = ret;
+#endif
 
 	/* FIXME: for now, only marked visited to pacify leaks */
 	ret = ckpt_obj_visit(ctx, nsproxy->mnt_ns, CKPT_OBJ_MNT_NS);
@@ -328,6 +339,14 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx)
 		ret = PTR_ERR(uts_ns);
 		goto out;
 	}
+	if (h->net_objref == 0)
+		net_ns = current->nsproxy->net_ns;
+	else
+		net_ns = ckpt_obj_fetch(ctx, h->net_objref, CKPT_OBJ_NET_NS);
+	if (IS_ERR(net_ns)) {
+		ret = PTR_ERR(net_ns);
+		goto out;
+	}
 
 	if (h->ipc_objref == 0)
 		ipc_ns = ctx->root_nsproxy->ipc_ns;
@@ -339,7 +358,6 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx)
 	}
 
 	mnt_ns = ctx->root_nsproxy->mnt_ns;
-	net_ns = ctx->root_nsproxy->net_ns;
 
 	if (uts_ns == current->nsproxy->uts_ns &&
 	    ipc_ns == current->nsproxy->ipc_ns &&
net/Kconfig +4 -0
index 041c35e..64dd3cd 100644
@@ -276,4 +276,8 @@ source "net/wimax/Kconfig"
 source "net/rfkill/Kconfig"
 source "net/9p/Kconfig"
 
+config CHECKPOINT_NETNS
+       bool
+       default y if NET && NET_NS && CHECKPOINT
+
 endif   # if NET
net/Makefile +1 -0
index 74b038f..570ee98 100644
@@ -67,3 +67,4 @@ endif
 obj-$(CONFIG_WIMAX)		+= wimax/
 
 obj-$(CONFIG_CHECKPOINT)	+= checkpoint.o
+obj-$(CONFIG_CHECKPOINT_NETNS)	+= checkpoint_dev.o
net/checkpoint_dev.c new file +815 -0

815 lines not on this page · raw message

author
#14 lore raw

PATCH 4/7 Add checkpoint support for veth devices (v2)

In reply to a message not in the archive

Adds an ndo_checkpoint() handler for veth devices to checkpoint themselves. Writes out the pairing information, addresses, and initiates a checkpoint on the peer if the peer won't be reached from another netns. Throws an error of our peer's netns isn't already in the hash (i.e., a tree leak).

Changes in v2:
- Fix check detecting if peer is in the init netns

---
 drivers/net/veth.c |   76 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 76 insertions(+), 0 deletions(-)
drivers/net/veth.c +76 -0
index 3a15de5..db92de8 100644
@@ -16,6 +16,9 @@
 #include <net/xfrm.h>
 #include <linux/veth.h>
 
+#include <linux/checkpoint.h>
+#include <linux/checkpoint_hdr.h>
+
 #define DRV_NAME	"veth"
 #define DRV_VERSION	"1.0"
 
@@ -284,6 +287,76 @@ static void veth_dev_free(struct net_device *dev)
 	free_netdev(dev);
 }
 
+#ifdef CONFIG_CHECKPOINT
+static int veth_checkpoint(struct ckpt_ctx *ctx, struct net_device *dev)
+{
+	struct ckpt_hdr_netdev *h;
+	struct veth_priv *priv = netdev_priv(dev);
+	struct net_device *peer = priv->peer;
+	struct ckpt_netdev_addr *addrs;
+	int ret;
+	int n;
+
+	if (!peer) {
+		ckpt_err(ctx, -EINVAL, "veth device has no peer!\n");
+		return -EINVAL;
+	}
+
+	h = ckpt_netdev_base(ctx, dev, &addrs);
+	if (IS_ERR(h))
+		return PTR_ERR(h);
+
+	h->type = CKPT_NETDEV_VETH;
+
+	ret = h->veth.this_ref = ckpt_obj_lookup_add(ctx, dev,
+						     CKPT_OBJ_NETDEV, &n);
+	if (ret < 0)
+		goto out;
+
+	ret = h->veth.peer_ref = ckpt_obj_lookup_add(ctx, peer,
+						     CKPT_OBJ_NETDEV, &n);
+	if (ret < 0)
+		goto out;
+
+	ret = ckpt_write_obj(ctx, (struct ckpt_hdr *)h);
+	if (ret < 0)
+		goto out;
+
+	ret = ckpt_write_buffer(ctx, dev->name, IFNAMSIZ);
+	if (ret < 0)
+		goto out;
+
+	ret = ckpt_write_buffer(ctx, peer->name, IFNAMSIZ);
+	if (ret < 0)
+		goto out;
+
+	if (h->inet_addrs > 0) {
+		int len = (sizeof(struct ckpt_netdev_addr) * h->inet_addrs);
+		ret = ckpt_write_buffer(ctx, addrs, len);
+		if (ret)
+			goto out;
+	}
+
+	/* Only checkpoint peer if we're not going to arrive at it
+	 * via another task's netns.  Fail if the pipe exits
+	 * our container to a netns not already in the hash
+	 */
+	if (ckpt_netdev_in_init_netns(ctx, peer))
+		ret = checkpoint_obj(ctx, peer, CKPT_OBJ_NETDEV);
+	else if (!ckpt_obj_lookup(ctx, peer->nd_net, CKPT_OBJ_NET_NS)) {
+		ret = -EINVAL;
+		ckpt_err(ctx, ret,
+			 "Peer %s of %s not in checkpointed namespaces\n",
+			 peer->name, dev->name);
+	}
+ out:
+	ckpt_hdr_put(ctx, h);
+	kfree(addrs);
+
+	return ret;
+}
+#endif
+
 static const struct net_device_ops veth_netdev_ops = {
 	.ndo_init            = veth_dev_init,
 	.ndo_open            = veth_open,
@@ -292,6 +365,9 @@ static const struct net_device_ops veth_netdev_ops = {
 	.ndo_change_mtu      = veth_change_mtu,
 	.ndo_get_stats       = veth_get_stats,
 	.ndo_set_mac_address = eth_mac_addr,
+#ifdef CONFIG_CHECKPOINT
+	.ndo_checkpoint      = veth_checkpoint,
+#endif
 };
 
 static void veth_setup(struct net_device *dev)
Signature · 1 line
1.6.2.5
author
#15 lore raw

PATCH 5/7 Add loopback checkpoint support (v2)

In reply to a message not in the archive

Adds a small ndo_checkpoint() handler for loopback devices to write the name and addresses like other interfaces.

Changes in v2:
- Add CONFIG_CHECKPOINT around the handler

---
 drivers/net/loopback.c |   44 +++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 41 insertions(+), 3 deletions(-)
drivers/net/loopback.c +41 -3
index b9fcc98..77023a7 100644
@@ -57,6 +57,8 @@
 #include <linux/ip.h>
 #include <linux/tcp.h>
 #include <linux/percpu.h>
+#include <linux/checkpoint.h>
+#include <linux/checkpoint_hdr.h>
 #include <net/net_namespace.h>
 
 struct pcpu_lstats {
@@ -153,10 +155,46 @@ static void loopback_dev_free(struct net_device *dev)
 	free_netdev(dev);
 }
 
+#ifdef CONFIG_CHECKPOINT
+static int loopback_checkpoint(struct ckpt_ctx *ctx, struct net_device *dev)
+{
+	struct ckpt_hdr_netdev *h;
+	struct ckpt_netdev_addr *addrs;
+	int ret;
+
+	h = ckpt_netdev_base(ctx, dev, &addrs);
+	if (IS_ERR(h))
+		return PTR_ERR(h);
+
+	h->type = CKPT_NETDEV_LO;
+
+	ret = ckpt_write_obj(ctx, (struct ckpt_hdr *)h);
+	if (ret < 0)
+		goto out;
+
+	ret = ckpt_write_buffer(ctx, dev->name, IFNAMSIZ);
+	if (ret < 0)
+		goto out;
+
+	if (h->inet_addrs > 0) {
+		int len = (sizeof(struct ckpt_netdev_addr) * h->inet_addrs);
+		ret = ckpt_write_buffer(ctx, addrs, len);
+	}
+
+ out:
+	ckpt_hdr_put(ctx, h);
+	kfree(addrs);
+
+	return ret;
+}
+#endif
 static const struct net_device_ops loopback_ops = {
-	.ndo_init      = loopback_dev_init,
-	.ndo_start_xmit= loopback_xmit,
-	.ndo_get_stats = loopback_get_stats,
+	.ndo_init       = loopback_dev_init,
+	.ndo_start_xmit = loopback_xmit,
+	.ndo_get_stats  = loopback_get_stats,
+#ifdef CONFIG_CHECKPOINT
+	.ndo_checkpoint = loopback_checkpoint,
+#endif
 };
 
 /*
Signature · 1 line
1.6.2.5
author
#16 lore raw

PATCH 6/7 Add a checkpoint handler to the 'sit' device

In reply to a message not in the archive

This handler doesn't really do much to checkpoint the device, other than the minimum required to support the restart process. When we add IPv6 support to this, then we can fill this out.

This allows us to avoid skipping unsupported interfaces on a normal system.

---
 net/ipv6/sit.c |   30 ++++++++++++++++++++++++++++++
 1 files changed, 30 insertions(+), 0 deletions(-)
net/ipv6/sit.c +30 -0
index 976e682..a9fc331 100644
@@ -32,6 +32,8 @@
 #include <linux/init.h>
 #include <linux/netfilter_ipv4.h>
 #include <linux/if_ether.h>
+#include <linux/checkpoint.h>
+#include <linux/checkpoint_hdr.h>
 
 #include <net/sock.h>
 #include <net/snmp.h>
@@ -1085,11 +1087,39 @@ static int ipip6_tunnel_change_mtu(struct net_device *dev, int new_mtu)
 	return 0;
 }
 
+static int ipip6_checkpoint(struct ckpt_ctx *ctx, struct net_device *dev)
+{
+	struct ckpt_hdr_netdev *h;
+	struct ckpt_netdev_addr *addrs;
+	int ret;
+
+	h = ckpt_netdev_base(ctx, dev, &addrs);
+	if (IS_ERR(h))
+		return PTR_ERR(h);
+
+	h->type = CKPT_NETDEV_SIT;
+
+	ret = ckpt_write_obj(ctx, (struct ckpt_hdr *) h);
+	if (ret < 0)
+		goto out;
+
+	if (h->inet_addrs > 0) {
+		int len = (sizeof(struct ckpt_netdev_addr) * h->inet_addrs);
+		ret = ckpt_write_buffer(ctx, addrs, len);
+	}
+ out:
+	ckpt_hdr_put(ctx, h);
+	kfree(addrs);
+
+	return ret;
+}
+
 static const struct net_device_ops ipip6_netdev_ops = {
 	.ndo_uninit	= ipip6_tunnel_uninit,
 	.ndo_start_xmit	= ipip6_tunnel_xmit,
 	.ndo_do_ioctl	= ipip6_tunnel_ioctl,
 	.ndo_change_mtu	= ipip6_tunnel_change_mtu,
+	.ndo_checkpoint	= ipip6_checkpoint,
 };
 
 static void ipip6_tunnel_setup(struct net_device *dev)
Signature · 1 line
1.6.2.5
author
#17 lore raw

PATCH 7/7 Add checkpoint support to macvlan driver

In reply to a message not in the archive
---
 drivers/net/macvlan.c |   44 ++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)
drivers/net/macvlan.c +44 -0
index 21a9c9a..4d99501 100644
@@ -28,6 +28,8 @@
 #include <linux/if_arp.h>
 #include <linux/if_link.h>
 #include <linux/if_macvlan.h>
+#include <linux/checkpoint.h>
+#include <linux/checkpoint_hdr.h>
 #include <net/rtnetlink.h>
 #include <net/xfrm.h>
 
@@ -493,6 +495,47 @@ static struct net_device_stats *macvlan_dev_get_stats(struct net_device *dev)
 	return stats;
 }
 
+static int macvlan_checkpoint(struct ckpt_ctx *ctx, struct net_device *dev)
+{
+	struct macvlan_dev *vlan = netdev_priv(dev);
+	struct ckpt_hdr_netdev *h;
+	struct ckpt_netdev_addr *addrs;
+	int ret;
+
+	ckpt_debug("Checkpointing macvlan %s:%s\n",
+		   dev->name, vlan->lowerdev->name);
+
+	h = ckpt_netdev_base(ctx, dev, &addrs);
+	if (IS_ERR(h))
+		return PTR_ERR(h);
+
+	h->type = CKPT_NETDEV_MACVLAN;
+
+	h->macvlan.mode = vlan->mode;
+
+	ret = ckpt_write_obj(ctx, (struct ckpt_hdr *)h);
+	if (ret < 0)
+		goto out;
+
+	ret = ckpt_write_buffer(ctx, dev->name, IFNAMSIZ);
+	if (ret < 0)
+		goto out;
+
+	ret = ckpt_write_buffer(ctx, vlan->lowerdev->name, IFNAMSIZ);
+	if (ret < 0)
+		goto out;
+
+	if (h->inet_addrs > 0) {
+		int len = (sizeof(struct ckpt_netdev_addr) * h->inet_addrs);
+		ret = ckpt_write_buffer(ctx, addrs, len);
+	}
+ out:
+	ckpt_hdr_put(ctx, h);
+	kfree(addrs);
+
+	return ret;
+}
+
 static void macvlan_ethtool_get_drvinfo(struct net_device *dev,
 					struct ethtool_drvinfo *drvinfo)
 {
@@ -539,6 +582,7 @@ static const struct net_device_ops macvlan_netdev_ops = {
 	.ndo_set_multicast_list	= macvlan_set_multicast_list,
 	.ndo_get_stats		= macvlan_dev_get_stats,
 	.ndo_validate_addr	= eth_validate_addr,
+	.ndo_checkpoint		= macvlan_checkpoint,
 };
 
 static void macvlan_setup(struct net_device *dev)
Signature · 1 line
1.6.2.5
#18 lore raw
In reply to a message not in the archive

Thanks -- series applied for ckpt-vc21.

Quoted 13 lines · Dan Smith

This is just the same patch set that has already been reviewed and acked, except for the following:

1. Rebased on v20
2. Storing the IP addresses in network byte order

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers