Network device and namespace checkpoint/restart (v3)
Subsystems (2)
This patch set adds checkpoint/restart support for network namespaces, as well as the network devices within. Currently supports veth and loopback device types.
Minor changes this time around, based on feedback from the previous posting. Changes are detailed per-patch, and a new one at the end adds a checkpoint handler for SIT devices (which does the bare minimum so that we can fail when a device has no support).
PATCH 1/5 Add checkpoint and collect hooks to net_device_ops
These will be implemented per-driver by those that support such operations.
--- include/linux/netdevice.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
include/linux/netdevice.h +6 -0
@@ -673,6 +673,12 @@ struct net_device_ops {int (*ndo_fcoe_get_wwn)(struct net_device *dev,u64 *wwn,int type);#endif +#ifdef CONFIG_CHECKPOINT +int (*ndo_collect)(struct ckpt_ctx *ctx, +struct net_device *dev); +int (*ndo_checkpoint)(struct ckpt_ctx *ctx, +struct net_device *dev); +#endif };/*
Signature · 1 line
1.6.2.5
PATCH 2/5 C/R: Basic support for network namespaces and devices (v4)
In reply to Dan SmithWhen checkpointing a task tree with network namespaces, we hook into do_checkpoint_ns() along with the others. Any devices in a given namespace are checkpointed (including their peer, in the case of veth) sequentially. Each network device stores a list of protocol addresses, as well as other information, such as hardware address.
This patch supports veth pairs, as well as the loopback adapter. The loopback support is there to make sure that any additional addresses and state (such as up/down) is copied to the loopback adapter that we are given in the new network namespace.
On restart, we instantiate new network namespaces and veth pairs as necessary. Any device we encounter that isn't in a network namespace that was checkpointed as part of a task is left in the namespace of the restarting process. This will be the case for a veth half that exists in the init netns to provide network access to a container.
Still to do are:
1. Routes
2. Netfilter rules
3. IPv6 addresses
4. Other virtual device types (e.g. bridges)
5. Multicast
6. Device config info (ipv4_devconf)
7. Additional ipv4 address attributes
Changes in v4:
- Fix allocation under lock in ckpt_netdev_inet_addrs()
- Add comment for case where there is no netns info in checkpoint image
- Fix inner structure alignment in netdev_addr header
- Fix instances of kfree(skb)
- Remove init_netns_ref from container header and checkpoint context
- Add 'extern' to checkpoint.h prototypes
- Swizzle do_restore_netns() to handle netns more like the others
- Return E2BIG for failure case when collecting inet addrs
- Report case where device doesn't support checkpoint
- Remove nested netns check from may_checkpoint_task()
- Move veth-specific netdev attributes into unioned struct to set an example for specific attributes of additional device types
- Add 'sit' device restore path that doesn't really do anything
- Fail instead of skip when encountering a device with no checkpoint support
Changes in v3:
- Use dev->checkpoint() for per-device checkpoint operation
- Use RTNL for veth pair creation on restart
- Export some of the functions that will be needed by dev->ndo_checkpoint()
Changes in v2:
- Add CONFIG_CHECKPOINT_NETNS that is dependent on NET, NET_NS, and CHECKPOINT. Conditionally compile the checkpoint_dev code based on it.
- Updated comment on should_checkpoint_netdev()
- Updated checkpoint_netdev() to explicitly check for "veth" in name
- Changed checkpoint_netns() to use BUG() for impossible condition
- Fixed a bug on restart with all devices in the init netns
- Lock the dev_base_lock while traversing interface addresses
- Collect all addresses for an interface before writing out in one single pass
--- checkpoint/checkpoint.c | 18 +- checkpoint/objhash.c | 48 +++ include/linux/checkpoint.h | 23 ++ include/linux/checkpoint_hdr.h | 54 +++ include/linux/checkpoint_types.h | 1 + kernel/nsproxy.c | 24 ++- net/Kconfig | 4 + net/Makefile | 1 + net/checkpoint_dev.c | 703 ++++++++++++++++++++++++++++++++++++++ 9 files changed, 864 insertions(+), 12 deletions(-) create mode 100644 net/checkpoint_dev.c
checkpoint/checkpoint.c +8 -10
@@ -185,11 +185,10 @@ static int checkpoint_container(struct ckpt_ctx *ctx) h =ckpt_hdr_get_type (ctx,sizeof (*h),CKPT_HDR_CONTAINER );if (!h)return -ENOMEM ; - ret =ckpt_write_obj (ctx, &h->h); -ckpt_hdr_put (ctx, h); + ret =ckpt_write_obj (ctx, &h->h);if (ret <0 ) -return ret; + goto out;memset (ctx->lsm_name,0 ,CHECKPOINT_LSM_NAME_MAX +1 ); strlcpy(ctx->lsm_name, security_get_lsm_name(),
@@ -197,9 +196,13 @@ static int checkpoint_container(struct ckpt_ctx *ctx) ret =ckpt_write_buffer (ctx, ctx->lsm_name,CHECKPOINT_LSM_NAME_MAX +1 );if (ret <0 ) -return ret; + goto out; -return security_checkpoint_header (ctx); + ret =security_checkpoint_header (ctx); + out: +ckpt_hdr_put (ctx, h); + +return ret; }/* write the checkpoint trailer */
@@ -285,11 +288,6 @@ static int may_checkpoint_task(struct ckpt_ctx *ctx, struct task_struct *t)_ckpt_err (ctx, -EPERM ,"%(T)Nested mnt_ns unsupported\n" ); ret = -EPERM ; } -/* no support for >1 private netns */ -if (nsproxy->net_ns != ctx->root_nsproxy->net_ns) { -_ckpt_err (ctx, -EPERM ,"%(T)Nested net_ns unsupported\n" ); - ret = -EPERM ; - }/* no support for >1 private pidns */ if (nsproxy->pid_ns != ctx->root_nsproxy->pid_ns) {_ckpt_err (ctx, -EPERM ,"%(T)Nested pid_ns unsupported\n" );
checkpoint/objhash.c +48 -0
60 lines not on this page · raw message
include/linux/checkpoint.h +23 -0
@@ -35,6 +35,7 @@#include <linux/checkpoint_types.h> #include <linux/checkpoint_hdr.h> #include <linux/err.h> +#include <linux/inetdevice.h> #include <net/sock.h> /* sycall helpers */
@@ -119,6 +120,28 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx,extern struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx,struct sock *sk);extern void sock_listening_list_free (struct list_head *head); +#ifdef CONFIG_CHECKPOINT_NETNS +extern int checkpoint_netns (struct ckpt_ctx *ctx,void *ptr); +extern void *restore_netns (struct ckpt_ctx *ctx); +extern int checkpoint_netdev (struct ckpt_ctx *ctx,void *ptr); +extern void *restore_netdev (struct ckpt_ctx *ctx); + +extern int ckpt_netdev_in_init_netns (struct ckpt_ctx *ctx, +struct net_device *dev); +extern int ckpt_netdev_inet_addrs (struct in_device *indev, +struct ckpt_netdev_addr *list[]); +extern int ckpt_netdev_hwaddr (struct net_device *dev, +struct ckpt_hdr_netdev *h); +extern struct ckpt_hdr_netdev *ckpt_netdev_base (struct ckpt_ctx *ctx, +struct net_device *dev, +struct ckpt_netdev_addr *addrs[]); +#else +# define checkpoint_netns NULL +# define restore_netns NULL +# define checkpoint_netdev NULL +# define restore_netdev NULL +#endif +/* ckpt kflags */ #define ckpt_set_ctx_kflag (__ctx, __kflag) \ set_bit(__kflag##_BIT, &(__ctx)->kflags)
include/linux/checkpoint_hdr.h +54 -0
84 lines not on this page · raw message
include/linux/checkpoint_types.h +1 -0
@@ -86,6 +86,7 @@ struct ckpt_ctx {wait_queue_head_t ghostq;/* waitqueue for ghost tasks */ struct cred *realcred, *ecred;/* tmp storage for cred at restart */ struct list_head listen_sockets;/* listening parent sockets */ +int init_netns_ref;/* Objref of root net namespace */ struct ckpt_stats stats;/* statistics */
kernel/nsproxy.c +22 -2
@@ -248,6 +248,11 @@ int ckpt_collect_ns(struct ckpt_ctx *ctx, struct task_struct *t) ret =ckpt_obj_collect (ctx, nsproxy->uts_ns,CKPT_OBJ_UTS_NS );if (ret <0 ) goto out; +#ifdef CONFIG_CHECKPOINT_NETNS + ret =ckpt_obj_collect (ctx, nsproxy->net_ns,CKPT_OBJ_NET_NS ); +if (ret <0 ) + goto out; +#endif ret =ckpt_obj_collect (ctx, nsproxy->ipc_ns,CKPT_OBJ_IPC_NS );if (ret <0 ) goto out;
@@ -288,6 +293,12 @@ static int do_checkpoint_ns(struct ckpt_ctx *ctx, struct nsproxy *nsproxy)if (ret <0 ) goto out; h->ipc_objref = ret; +#ifdef CONFIG_CHECKPOINT_NETNS + ret =checkpoint_obj (ctx, nsproxy->net_ns,CKPT_OBJ_NET_NS ); +if (ret <0 ) + goto out; + h->net_objref = ret; +#endif /* FIXME: for now, only marked visited to pacify leaks */ ret =ckpt_obj_visit (ctx, nsproxy->mnt_ns,CKPT_OBJ_MNT_NS );
@@ -312,6 +323,7 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx)struct nsproxy *nsproxy =NULL ;struct uts_namespace *uts_ns;struct ipc_namespace *ipc_ns; +struct net *net_ns;int ret; h =ckpt_read_obj_type (ctx,sizeof (*h),CKPT_HDR_NS );
@@ -333,6 +345,14 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx) ret =PTR_ERR (ipc_ns); goto out; } +if (h->net_objref ==0 ) + net_ns = current->nsproxy->net_ns; +else + net_ns =ckpt_obj_fetch (ctx, h->net_objref,CKPT_OBJ_NET_NS ); +if (IS_ERR (net_ns)) { + ret =PTR_ERR (net_ns); + goto out; + }#if defined(COFNIG_UTS_NS ) || defined(CONFIG_IPC_NS ) ret = -ENOMEM ;
@@ -344,13 +364,13 @@ static struct nsproxy *do_restore_ns(struct ckpt_ctx *ctx) nsproxy->uts_ns = uts_ns;get_ipc_ns (ipc_ns); nsproxy->ipc_ns = ipc_ns; +get_net (net_ns); + nsproxy->net_ns = net_ns;get_pid_ns (current->nsproxy->pid_ns); nsproxy->pid_ns = current->nsproxy->pid_ns;get_mnt_ns (current->nsproxy->mnt_ns); nsproxy->mnt_ns = current->nsproxy->mnt_ns; -get_net (current->nsproxy->net_ns); - nsproxy->net_ns = current->nsproxy->net_ns;#else nsproxy = current->nsproxy;get_nsproxy (nsproxy);
net/Kconfig +4 -0
@@ -276,4 +276,8 @@ source "net/wimax/Kconfig" source "net/rfkill/Kconfig" source "net/9p/Kconfig" +config CHECKPOINT_NETNS + bool + default y if NET && NET_NS && CHECKPOINT + endif # if NET
net/Makefile +1 -0
@@ -67,3 +67,4 @@ endif obj-$(CONFIG_WIMAX) += wimax/ obj-$(CONFIG_CHECKPOINT) += checkpoint.o +obj-$(CONFIG_CHECKPOINT_NETNS) += checkpoint_dev.o
net/checkpoint_dev.c new file +703 -0
703 lines not on this page · raw message
Signature · 1 line
1.6.2.5
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 2/5] C/R: Basic support for network namespaces and devices (v4)
SH> the above two hunks change the flow in checkpoint_container(), but SH> they don't seem to actually add anything. And I don't see (with a SH> quick browse) any later patch in this series changing this either. SH> Is this just noise?
Ah, yeah, I think that's left over from a previous version where I had to insert something there. Sorry about that :)
Quoted 4 lines
net/checkpoint_dev.c line 112
Quoted 4 lines
net/checkpoint_dev.c new file +4
-0+int ckpt_netdev_in_init_netns (struct ckpt_ctx *ctx,struct net_device *dev) +{ +return dev->nd_net == current->nsproxy->net_ns; +}
SH> You are comparing it to the net_ns of the checkpointing task. I'm SH> not sure that makes sense - but I'm also not sure what if anything SH> makes more sense.
SH> What exactly do you mean by the 'init' netns here? Do you mean SH> the init_net_ns for the container, or that it is the net_ns of SH> whatever task created the container?
In this case, 'current' is the task doing the checkpoint, right? So, we're treating the netns that it is in as the "top level" and will restore the tree, as visible from that task, relative to the netns of the restart process. We had an IRC conversation about this, I believe :)
SH> How about a
SH> ckpt_err(ctx, -ENOSYS,
SH> Device %s does not support checkpoint\n",
dev-> name);
SH> here to put a meaningful msg in the user's log?
Yep, definitely.
Thanks!
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: [email protected]
Re: [PATCH 2/5] C/R: Basic support for network namespaces and devices (v4)
Quoted 17 lines · Dan Smith
SH> the above two hunks change the flow in
checkpoint_container(), but SH> they don't seem to actually add anything. And I don't see (with a SH> quick browse) any later patch in this series changing this either. SH> Is this just noise?Ah, yeah, I think that's left over from a previous version where I had to insert something there. Sorry about that :)
Quoted 4 lines
net/checkpoint_dev.c line 112
Quoted 4 lines
net/checkpoint_dev.c new file +4
-0+int ckpt_netdev_in_init_netns (struct ckpt_ctx *ctx,struct net_device *dev) +{ +return dev->nd_net == current->nsproxy->net_ns; +}SH> You are comparing it to the
net_nsof the checkpointing task. I'm SH> not sure that makes sense - but I'm also not sure what if anything SH> makes more sense.
Quoted 8 lines
SH> What exactly do you mean by the 'init' netns here? Do you mean SH> the
init_net_nsfor the container, or that it is thenet_nsof SH> whatever task created the container?In this case, 'current' is the task doing the checkpoint, right? So, we're treating the netns that it is in as the "top level" and will restore the tree, as visible from that task, relative to the netns of the restart process. We had an IRC conversation about this, I believe :)
But there is no guarantee that the checkpointer is in the netns which we would call the 'top level' netns. Which means that, at restart, whether or not the devices which are in what we call the top level netns are in fact inherited or not, will depend on conditions of the checkpointer. Do we care? (I thought we did, but maybe we don't... it's unlikely to happen anyway)
Quoted 19 lines
SH> How about a
SH> ckpt_err(ctx,-ENOSYS,
SH> Device %s does not support checkpoint\n",
dev-> name);SH> here to put a meaningful msg in the user's log?
Yep, definitely.
Thanks!
Signature · 7 lines
Dan Smith IBM Linux Technology Center email: [email protected] _______________________________________________ Containers mailing list [email protected] https://lists.linux-foundation.org/mailman/listinfo/containers
Re: [PATCH 2/5] C/R: Basic support for network namespaces and devices (v4)
SH> But there is no guarantee that the checkpointer is in the netns SH> which we would call the 'top level' netns. Which means that, at SH> restart, whether or not the devices which are in what we call the SH> top level netns are in fact inherited or not, will depend on SH> conditions of the checkpointer. Do we care? (I thought we did, SH> but maybe we don't... it's unlikely to happen anyway)
Well, when we discussed this on IRC with Oren, I think we came to the conclusion that since network namespaces aren't hierarchical, that we would restore things from the "viewpoint" of the process that checkpointed them. It gives us a sane way to ensure that the peer devices residing in the init netns can be put back there, even though we don't checkpoint everything in the init netns (like eth0).
If you checkpoint a veth from within the container and you have a peer device that is outside the container (but not in a netns that is checkpointed as part of a task), it's going to fail and tell you that one of your peers leaked to the outside. I think that's sane and preferred behavior, no? If you're using macvlan and you checkpoint from within the container, I think you should be okay, as long as there is a appropriately named device to base the restored devices on in whatever netns your restore process is in.
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: [email protected]
Re: [PATCH 2/5] C/R: Basic support for network namespaces and devices (v4)
Quoted 14 lines · Dan Smith
SH> But there is no guarantee that the checkpointer is in the netns SH> which we would call the 'top level' netns. Which means that, at SH> restart, whether or not the devices which are in what we call the SH> top level netns are in fact inherited or not, will depend on SH> conditions of the checkpointer. Do we care? (I thought we did, SH> but maybe we don't... it's unlikely to happen anyway)
Well, when we discussed this on IRC with Oren, I think we came to the conclusion that since network namespaces aren't hierarchical, that we would restore things from the "viewpoint" of the process that checkpointed them. It gives us a sane way to ensure that the peer devices residing in the init netns can be put back there, even though we don't checkpoint everything in the init netns (like eth0).
Quoted 5 lines
If you checkpoint a veth from within the container and you have a peer device that is outside the container (but not in a netns that is checkpointed as part of a task), it's going to fail and tell you that one of your peers leaked to the outside. I think that's sane and preferred behavior, no?
Well I don't think it is, but it's a fine starting point, so let's worry about it later.
thanks,
-serge
Quoted 9 lines
If you're using macvlan and you checkpoint
from within the container, I think you should be okay, as long as there is a appropriately named device to base the restored devices on in whatever netns your restore process is in.Signature · 3 lines
Dan Smith IBM Linux Technology Center email: [email protected]
PATCH 3/5 Add checkpoint support for veth devices (v2)
In reply to Dan SmithAdds an ndo_checkpoint() handler for veth devices to checkpoint themselves. Writes out the pairing information, addresses, and initiates a checkpoint on the peer if the peer won't be reached from another netns. Throws an error of our peer's netns isn't already in the hash (i.e., a tree leak).
Changes in v2:
- Fix check detecting if peer is in the init netns
--- drivers/net/veth.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 76 insertions(+), 0 deletions(-)
drivers/net/veth.c +76 -0
@@ -16,6 +16,9 @@#include <net/xfrm.h> #include <linux/veth.h> +#include <linux/checkpoint.h> +#include <linux/checkpoint_hdr.h> +#define DRV_NAME "veth"#define DRV_VERSION "1.0"
@@ -284,6 +287,76 @@ static void veth_dev_free(struct net_device *dev)free_netdev (dev); } +#ifdef CONFIG_CHECKPOINT +static int veth_checkpoint (struct ckpt_ctx *ctx,struct net_device *dev) +{ +struct ckpt_hdr_netdev *h; +struct veth_priv *priv =netdev_priv (dev); +struct net_device *peer = priv->peer; +struct ckpt_netdev_addr *addrs; +int ret; +int n; + +if (!peer) { +ckpt_err (ctx, -EINVAL ,"veth device has no peer!\n" ); +return -EINVAL ; + } + + h =ckpt_netdev_base (ctx, dev, &addrs); +if (IS_ERR (h)) +return PTR_ERR (h); + + h->type =CKPT_NETDEV_VETH ; + + ret = h->veth.this_ref =ckpt_obj_lookup_add (ctx, dev, +CKPT_OBJ_NETDEV , &n); +if (ret <0 ) + goto out; + + ret = h->veth.peer_ref =ckpt_obj_lookup_add (ctx, peer, +CKPT_OBJ_NETDEV , &n); +if (ret <0 ) + goto out; + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *)h); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, dev->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, peer->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + +if (h->inet_addrs >0 ) { +int len = (sizeof (struct ckpt_netdev_addr ) * h->inet_addrs); + ret =ckpt_write_buffer (ctx, addrs, len); +if (ret) + goto out; + } + +/* Only checkpoint peer if we're not going to arrive at it +* via another task's netns. Fail if the pipe exits +* our container to a netns not already in the hash +*/ +if (ckpt_netdev_in_init_netns (ctx, peer)) + ret =checkpoint_obj (ctx, peer,CKPT_OBJ_NETDEV ); +else if (!ckpt_obj_lookup (ctx, peer->nd_net,CKPT_OBJ_NET_NS )) { + ret = -EINVAL ; +ckpt_err (ctx, ret, +"Peer %s of %s not in checkpointed namespaces\n" , + peer->name, dev->name); + } + out: +ckpt_hdr_put (ctx, h); +kfree (addrs); + +return ret; +} +#endif +static const struct net_device_ops veth_netdev_ops = { .ndo_init = veth_dev_init, .ndo_open = veth_open,
@@ -292,6 +365,9 @@ static const struct net_device_ops veth_netdev_ops = { .ndo_change_mtu = veth_change_mtu, .ndo_get_stats = veth_get_stats, .ndo_set_mac_address = eth_mac_addr, +#ifdef CONFIG_CHECKPOINT + .ndo_checkpoint = veth_checkpoint, +#endif };static void veth_setup (struct net_device *dev)
Signature · 1 line
1.6.2.5
A message that is not in the archive (sent off-list or not yet received): 1266336187-19105-4-git-send-email-danms-r/[email protected]
Re: [PATCH 3/5] Add checkpoint support for veth devices (v2)
Quoted 82 lines · Dan Smith
Adds an
ndo_checkpoint()handler for veth devices to checkpoint themselves. Writes out the pairing information, addresses, and initiates a checkpoint on the peer if the peer won't be reached from another netns. Throws an error of our peer's netns isn't already in the hash (i.e., a tree leak).Changes in v2:
- Fix check detecting if peer is in the init netnsSigned-off-by Dan Smith <danms-r/[email protected]>--- drivers/net/veth.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 76 insertions(+), 0 deletions(-)drivers/net/veth.c +54
-0@@ -16,6 +16,9 @@#include <net/xfrm.h> #include <linux/veth.h> +#include <linux/checkpoint.h> +#include <linux/checkpoint_hdr.h> +#define DRV_NAME "veth"#define DRV_VERSION "1.0"@@ -284,6 +287,76 @@ static void veth_dev_free(struct net_device *dev)free_netdev (dev); } +#ifdef CONFIG_CHECKPOINT +static int veth_checkpoint (struct ckpt_ctx *ctx,struct net_device *dev) +{ +struct ckpt_hdr_netdev *h; +struct veth_priv *priv =netdev_priv (dev); +struct net_device *peer = priv->peer; +struct ckpt_netdev_addr *addrs; +int ret; +int n; + +if (!peer) { +ckpt_err (ctx, -EINVAL ,"veth device has no peer!\n" ); +return -EINVAL ; + } + + h =ckpt_netdev_base (ctx, dev, &addrs); +if (IS_ERR (h)) +return PTR_ERR (h); + + h->type =CKPT_NETDEV_VETH ; + + ret = h->veth.this_ref =ckpt_obj_lookup_add (ctx, dev, +CKPT_OBJ_NETDEV , &n); +if (ret <0 ) + goto out; + + ret = h->veth.peer_ref =ckpt_obj_lookup_add (ctx, peer, +CKPT_OBJ_NETDEV , &n); +if (ret <0 ) + goto out; + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *)h); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, dev->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, peer->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + +if (h->inet_addrs >0 ) { +int len = (sizeof (struct ckpt_netdev_addr ) * h->inet_addrs); + ret =ckpt_write_buffer (ctx, addrs, len); +if (ret) + goto out; + } + +/* Only checkpoint peer if we're not going to arrive at it
drivers/net/veth.c static void veth_dev_free(struct net_device *dev) · line 350
Quoted 10 lines
drivers/net/veth.c +10
-0+* via another task's netns. Fail if the pipe exits +* our container to a netns not already in the hash +*/ +if (ckpt_netdev_in_init_netns (ctx, peer)) + ret =checkpoint_obj (ctx, peer,CKPT_OBJ_NETDEV ); +else if (!ckpt_obj_lookup (ctx, peer->nd_net,CKPT_OBJ_NET_NS )) { + ret = -EINVAL ; +ckpt_err (ctx, ret, +"Peer %s of %s not in checkpointed namespaces\n" , + peer->name, dev->name);
I'm not sure this check does what you think it does: note that ckpt_netdev_base(), defined in the previous patch, and called higher
up in this function, is going to checkpoint peer->nd_net. :)
(right?)
Quoted 29 lines
drivers/net/veth.c +12
-0+ } + out: +ckpt_hdr_put (ctx, h); +kfree (addrs); + +return ret; +} +#endif +static const struct net_device_ops veth_netdev_ops = { .ndo_init = veth_dev_init, .ndo_open = veth_open,@@ -292,6 +365,9 @@ static const struct net_device_ops veth_netdev_ops = { .ndo_change_mtu = veth_change_mtu, .ndo_get_stats = veth_get_stats, .ndo_set_mac_address = eth_mac_addr, +#ifdef CONFIG_CHECKPOINT + .ndo_checkpoint = veth_checkpoint, +#endif };static void veth_setup (struct net_device *dev)Signature · 6 lines
1.6.2.5 _______________________________________________ Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org https://lists.linux-foundation.org/mailman/listinfo/containers
Re: [PATCH 3/5] Add checkpoint support for veth devices (v2)
Quoted 5 lines
drivers/net/veth.c static void veth_dev_free(struct net_device *dev) · line 350
Quoted 5 lines
drivers/net/veth.c +5
-0+else if (!ckpt_obj_lookup (ctx, peer->nd_net,CKPT_OBJ_NET_NS )) { + ret = -EINVAL ; +ckpt_err (ctx, ret, +"Peer %s of %s not in checkpointed namespaces\n" , + peer->name, dev->name);
SH> I'm not sure this check does what you think it does: note that SH> ckpt_netdev_base(), defined in the previous patch, and called SH> higher up in this function, is going to checkpoint peer->nd_net. SH> :)
Actually, no, ckpt_netdev_base() can't checkpoint peer->nd_net because it's device-agnostic and has no knowledge of dev->peer.
The idea here was that we checkpoint a netns when we arrive at it via nsproxy. Doing that, we checkpoint the devices within. We encounter a veth device, which has a peer, so we decide if:
1. We won't arrive at the peer later because it is in the init namespace, so we checkpoint it now.
2. We will arrive at it later because the peer's netns is in the list we've already collected, so checkpoint the peer with its namespace
3. Neither are true and we won't arrive at it later and therefore we can't allow checkpoint to continue
#2 depends on the collect process having put all the task's netns' in
the hash ahead of time.
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: [email protected]
Re: [PATCH 3/5] Add checkpoint support for veth devices (v2)
Quoted 11 lines · Dan Smith
Quoted 5 lines
drivers/net/veth.c static void veth_dev_free(struct net_device *dev) · line 350
Quoted 5 lines
drivers/net/veth.c +5
-0+else if (!ckpt_obj_lookup (ctx, peer->nd_net,CKPT_OBJ_NET_NS )) { + ret = -EINVAL ; +ckpt_err (ctx, ret, +"Peer %s of %s not in checkpointed namespaces\n" , + peer->name, dev->name);SH> I'm not sure this check does what you think it does: note that SH>
ckpt_netdev_base(), defined in the previous patch, and called SH> higher up in this function, is going to checkpointpeer->nd_net. SH> :)
Quoted 2 lines
Actually, no,
ckpt_netdev_base()can't checkpointpeer->nd_netbecause it's device-agnostic and has no knowledge ofdev->peer.
Oh, ok.
Quoted 11 lines
The idea here was that we checkpoint a netns when we arrive at it via nsproxy. Doing that, we checkpoint the devices within. We encounter a veth device, which has a peer, so we decide if:
1. We won't arrive at the peer later because it is in the init namespace, so we checkpoint it now.
2. We will arrive at it later because the peer's netns is in the list we've already collected, so checkpoint the peer with its namespace
3. Neither are true and we won't arrive at it later and therefore we can't allow checkpoint to continue
Quoted 2 lines
#2 depends on the collect process having put all the task's netns' in
the hash ahead of time.
Right, that was what I was originally starting to hunt down when I thought I saw ckpt_netdev_base() checkpointing peer's netns.
So do you actually know that the peer's netns will have been checkpointed? I'm a little fuzzy about where netns and netdevs are checkpointed. If you have two private netns's in a container, with a veth connecting them, and you checkpoint a task in netns 1, will you fail bc netns 2 hasn't been checkpointed yet bc no task in it has been checkpointed yet?
-serge
Re: [PATCH 3/5] Add checkpoint support for veth devices (v2)
SH> So do you actually know that the peer's netns will have been SH> checkpointed? I'm a little fuzzy about where netns and netdevs SH> are checkpointed. If you have two private netns's in a container, SH> with a veth connecting them, and you checkpoint a task in netns 1, SH> will you fail bc netns 2 hasn't been checkpointed yet bc no task SH> in it has been checkpointed yet?
Nope, because they're collect()'ed before we checkpoint().
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: [email protected]
PATCH 4/5 Add loopback checkpoint support
In reply to Dan SmithAdds a small ndo_checkpoint() handler for loopback devices to write the name and addresses like other interfaces.
--- drivers/net/loopback.c | 41 ++++++++++++++++++++++++++++++++++++++--- 1 files changed, 38 insertions(+), 3 deletions(-)
drivers/net/loopback.c +38 -3
@@ -57,6 +57,8 @@#include <linux/ip.h> #include <linux/tcp.h> #include <linux/percpu.h> +#include <linux/checkpoint.h> +#include <linux/checkpoint_hdr.h> #include <net/net_namespace.h> struct pcpu_lstats {
@@ -153,10 +155,43 @@ static void loopback_dev_free(struct net_device *dev)free_netdev (dev); } +static int loopback_checkpoint (struct ckpt_ctx *ctx,struct net_device *dev) +{ +struct ckpt_hdr_netdev *h; +struct ckpt_netdev_addr *addrs; +int ret; + + h =ckpt_netdev_base (ctx, dev, &addrs); +if (IS_ERR (h)) +return PTR_ERR (h); + + h->type =CKPT_NETDEV_LO ; + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *)h); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, dev->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + +if (h->inet_addrs >0 ) { +int len = (sizeof (struct ckpt_netdev_addr ) * h->inet_addrs); + ret =ckpt_write_buffer (ctx, addrs, len); + } + + out: +ckpt_hdr_put (ctx, h); +kfree (addrs); + +return ret; +} +static const struct net_device_ops loopback_ops = { - .ndo_init = loopback_dev_init, - .ndo_start_xmit= loopback_xmit, - .ndo_get_stats = loopback_get_stats, + .ndo_init = loopback_dev_init, + .ndo_start_xmit = loopback_xmit, + .ndo_get_stats = loopback_get_stats, + .ndo_checkpoint = loopback_checkpoint, };/*
Signature · 1 line
1.6.2.5
Re: [PATCH 4/5] Add loopback checkpoint support
Quoted 22 lines · Dan Smith
Adds a small
ndo_checkpoint()handler for loopback devices to write the name and addresses like other interfaces.Signed-off-by Dan Smith <[email protected]>--- drivers/net/loopback.c | 41 ++++++++++++++++++++++++++++++++++++++--- 1 files changed, 38 insertions(+), 3 deletions(-)drivers/net/loopback.c +2
-0@@ -57,6 +57,8 @@#include <linux/ip.h> #include <linux/tcp.h> #include <linux/percpu.h> +#include <linux/checkpoint.h> +#include <linux/checkpoint_hdr.h> #include <net/net_namespace.h> struct pcpu_lstats {
drivers/net/loopback.c static void loopback_dev_free(struct net_device *dev) · line 157
Quoted 3 lines
drivers/net/loopback.c +0
-0@@ -153,10 +155,43 @@ static void loopback_dev_free(struct net_device *dev)free_netdev (dev); }
Dont you have a #ifdef CONFIG_CHECKPOINT or something to avoid this for small machines ?
Quoted 43 lines
drivers/net/loopback.c +36
-3+static int loopback_checkpoint (struct ckpt_ctx *ctx,struct net_device *dev) +{ +struct ckpt_hdr_netdev *h; +struct ckpt_netdev_addr *addrs; +int ret; + + h =ckpt_netdev_base (ctx, dev, &addrs); +if (IS_ERR (h)) +return PTR_ERR (h); + + h->type =CKPT_NETDEV_LO ; + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *)h); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, dev->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + +if (h->inet_addrs >0 ) { +int len = (sizeof (struct ckpt_netdev_addr ) * h->inet_addrs); + ret =ckpt_write_buffer (ctx, addrs, len); + } + + out: +ckpt_hdr_put (ctx, h); +kfree (addrs); + +return ret; +} +static const struct net_device_ops loopback_ops = { - .ndo_init = loopback_dev_init, - .ndo_start_xmit= loopback_xmit, - .ndo_get_stats = loopback_get_stats, + .ndo_init = loopback_dev_init, + .ndo_start_xmit = loopback_xmit, + .ndo_get_stats = loopback_get_stats, + .ndo_checkpoint = loopback_checkpoint, };/*
Re: [PATCH 4/5] Add loopback checkpoint support
ED> Dont you have a #ifdef CONFIG_CHECKPOINT or something to avoid ED> this for small machines ?
Yes, and the veth patch used it appropriately. It should look like this:
Add loopback checkpoint support (v2)
Adds a small ndo_checkpoint() handler for loopback devices to write the name and addresses like other interfaces.
Changes in v2:
- Add CONFIG_CHECKPOINT around the handler
drivers/net/loopback.c +41 -3
@@ -57,6 +57,8 @@#include <linux/ip.h> #include <linux/tcp.h> #include <linux/percpu.h> +#include <linux/checkpoint.h> +#include <linux/checkpoint_hdr.h> #include <net/net_namespace.h> struct pcpu_lstats {
@@ -153,10 +155,46 @@ static void loopback_dev_free(struct net_device *dev)free_netdev (dev); } +#ifdef CONFIG_CHECKPOINT +static int loopback_checkpoint (struct ckpt_ctx *ctx,struct net_device *dev) +{ +struct ckpt_hdr_netdev *h; +struct ckpt_netdev_addr *addrs; +int ret; + + h =ckpt_netdev_base (ctx, dev, &addrs); +if (IS_ERR (h)) +return PTR_ERR (h); + + h->type =CKPT_NETDEV_LO ; + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *)h); +if (ret <0 ) + goto out; + + ret =ckpt_write_buffer (ctx, dev->name,IFNAMSIZ ); +if (ret <0 ) + goto out; + +if (h->inet_addrs >0 ) { +int len = (sizeof (struct ckpt_netdev_addr ) * h->inet_addrs); + ret =ckpt_write_buffer (ctx, addrs, len); + } + + out: +ckpt_hdr_put (ctx, h); +kfree (addrs); + +return ret; +} +#endif static const struct net_device_ops loopback_ops = { - .ndo_init = loopback_dev_init, - .ndo_start_xmit= loopback_xmit, - .ndo_get_stats = loopback_get_stats, + .ndo_init = loopback_dev_init, + .ndo_start_xmit = loopback_xmit, + .ndo_get_stats = loopback_get_stats, +#ifdef CONFIG_CHECKPOINT + .ndo_checkpoint = loopback_checkpoint, +#endif };/*
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: [email protected]
PATCH 5/5 Add a checkpoint handler to the 'sit' device
In reply to Dan SmithThis handler doesn't really do much to checkpoint the device, other than the minimum required to support the restart process. When we add IPv6 support to this, then we can fill this out.
This allows us to avoid skipping unsupported interfaces on a normal system.
--- net/ipv6/sit.c | 30 ++++++++++++++++++++++++++++++ 1 files changed, 30 insertions(+), 0 deletions(-)
net/ipv6/sit.c +30 -0
@@ -32,6 +32,8 @@#include <linux/init.h> #include <linux/netfilter_ipv4.h> #include <linux/if_ether.h> +#include <linux/checkpoint.h> +#include <linux/checkpoint_hdr.h> #include <net/sock.h> #include <net/snmp.h>
@@ -1085,11 +1087,39 @@ static int ipip6_tunnel_change_mtu(struct net_device *dev, int new_mtu)return 0 ; } +static int ipip6_checkpoint (struct ckpt_ctx *ctx,struct net_device *dev) +{ +struct ckpt_hdr_netdev *h; +struct ckpt_netdev_addr *addrs; +int ret; + + h =ckpt_netdev_base (ctx, dev, &addrs); +if (IS_ERR (h)) +return PTR_ERR (h); + + h->type =CKPT_NETDEV_SIT ; + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *) h); +if (ret <0 ) + goto out; + +if (h->inet_addrs >0 ) { +int len = (sizeof (struct ckpt_netdev_addr ) * h->inet_addrs); + ret =ckpt_write_buffer (ctx, addrs, len); + } + out: +ckpt_hdr_put (ctx, h); +kfree (addrs); + +return ret; +} +static const struct net_device_ops ipip6_netdev_ops = { .ndo_uninit = ipip6_tunnel_uninit, .ndo_start_xmit = ipip6_tunnel_xmit, .ndo_do_ioctl = ipip6_tunnel_ioctl, .ndo_change_mtu = ipip6_tunnel_change_mtu, + .ndo_checkpoint = ipip6_checkpoint, };static void ipip6_tunnel_setup (struct net_device *dev)
Signature · 1 line
1.6.2.5