Add support for connected INET sockets
Subsystems (4)
This set includes the previous INET connected sockets patch with a few trivial changes suggested by Oren, as well as the readme update. The first patch fixes socket buffer checkpoint to include fragments, and includes the changes in the previous first patch that added header mark information. As a result, this set now also includes a fix to the UNIX code to cope with the change.
A message that is not in the archive (sent off-list or not yet received): 1257878856-25520-1-git-send-email-danms-r/[email protected]
PATCH 1/4 Unify skb read/write functions and fix for fragmented buffers
The INET code often creates socket buffers by attaching fragments instead of writing to the linear region. This extends the skb write functions to write out the linear and fragment regions of an skb, and adds a function to be used by others wishing to restore an skb in the same way. This also includes the header-mark-setting bits from a previous patch.
--- include/linux/checkpoint.h | 1 + include/linux/checkpoint_hdr.h | 11 ++ net/checkpoint.c | 253 ++++++++++++++++++++++++++++++++++++---- 3 files changed, 242 insertions(+), 23 deletions(-)
include/linux/checkpoint.h +1 -0
@@ -100,6 +100,7 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx,struct socket *socket, struct sockaddr *loc, unsigned *loc_len, struct sockaddr *rem, unsigned *rem_len); +struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx);/* ckpt kflags */ #define ckpt_set_ctx_kflag (__ctx, __kflag) \
include/linux/checkpoint_hdr.h +11 -0
@@ -561,8 +561,19 @@ struct ckpt_hdr_socket_queue {struct ckpt_hdr_socket_buffer {struct ckpt_hdr h; +__u64 transport_header; +__u64 network_header; +__u64 mac_header; +__u64 lin_len;/* Length of linear data */ +__u64 frg_len;/* Length of fragment data */ +__u64 skb_len;/* Length of skb (adjusted) */ +__u64 hdr_len;/* Length of skipped header */ +__u64 mac_len;__s32 sk_objref;__s32 pr_objref; +__u16 protocol; +__u16 nr_frags; +__u8 cb[48 ]; };#define CKPT_UNIX_LINKED 1
net/checkpoint.c +230 -23
283 lines not on this page · raw message
Signature · 1 line
1.6.3.3
A message that is not in the archive (sent off-list or not yet received): 1257878856-25520-2-git-send-email-danms-r/[email protected]
Serge E. Hallyn
Acked-byRe: [PATCH 1/4] Unify skb read/write functions and fix for fragmented buffers
Quoted 7 lines · Dan Smith
The INET code often creates socket buffers by attaching fragments instead of writing to the linear region. This extends the skb write functions to write out the linear and fragment regions of an skb, and adds a function to be used by others wishing to restore an skb in the same way. This also includes the header-mark-setting bits from a previous patch.
Signed-off-by Dan Smith <danms-r/[email protected]>
Quoted 340 lines
--- include/linux/checkpoint.h | 1 + include/linux/checkpoint_hdr.h | 11 ++ net/checkpoint.c | 253 ++++++++++++++++++++++++++++++++++++---- 3 files changed, 242 insertions(+), 23 deletions(-)include/linux/checkpoint.h +1
-0@@ -100,6 +100,7 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx,struct socket *socket, struct sockaddr *loc, unsigned *loc_len, struct sockaddr *rem, unsigned *rem_len); +struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx);/* ckpt kflags */ #define ckpt_set_ctx_kflag (__ctx, __kflag) \include/linux/checkpoint_hdr.h +11
-0@@ -561,8 +561,19 @@ struct ckpt_hdr_socket_queue {struct ckpt_hdr_socket_buffer {struct ckpt_hdr h; +__u64 transport_header; +__u64 network_header; +__u64 mac_header; +__u64 lin_len;/* Length of linear data */ +__u64 frg_len;/* Length of fragment data */ +__u64 skb_len;/* Length of skb (adjusted) */ +__u64 hdr_len;/* Length of skipped header */ +__u64 mac_len;__s32 sk_objref;__s32 pr_objref; +__u16 protocol; +__u16 nr_frags; +__u8 cb[48 ]; };#define CKPT_UNIX_LINKED 1net/checkpoint.c +230
-23@@ -17,9 +17,11 @@#include <linux/syscalls.h> #include <linux/sched.h> #include <linux/fs_struct.h> +#include <linux/highmem.h> #include <net/af_unix.h> #include <net/tcp_states.h> +#include <net/tcp.h> #include <linux/deferqueue.h> #include <linux/checkpoint.h> @@ -88,6 +90,233 @@ static int sock_copy_buffers(struct sk_buff_head *from,return -EAGAIN ; } +static void sock_record_header_info (struct sk_buff *skb, +struct ckpt_hdr_socket_buffer *h) +{ + + h->mac_len = skb->mac_len; + h->skb_len = skb->len; + h->hdr_len = skb->data - skb->head; + h->lin_len = (__u64)(skb->tail - skb->head); + h->frg_len = skb->data_len; + +#ifdef NET_SKBUFF_DATA_USES_OFFSET + h->transport_header = skb->transport_hdr; + h->network_header = skb->network_header; + h->mac_header = skb->mac_header; +#else + h->transport_header = skb->transport_header - skb->head; + h->network_header = skb->network_header - skb->head; + h->mac_header = skb->mac_header - skb->head; +#endif + +memcpy (h->cb, skb->cb,sizeof (skb->cb)); + h->nr_frags =skb_shinfo (skb)->nr_frags; +} + +int sock_restore_header_info (struct sk_buff *skb, +struct ckpt_hdr_socket_buffer *h) +{ +if (h->mac_header + h->mac_len != h->network_header) { +ckpt_debug ("skb mac_header %llu+%llu != network header %llu\n" , + h->mac_header, h->mac_len, h->network_header); +return -EINVAL ; + } + +if (h->network_header > h->lin_len) { +ckpt_debug ("skb network header %llu > linear length %llu\n" , + h->network_header, h->lin_len); +return -EINVAL ; + } + +if (h->transport_header > h->lin_len) { +ckpt_debug ("skb transport header %llu > linear length %llu\n" , + h->transport_header, h->lin_len); +return -EINVAL ; + } + +if (h->skb_len >SKB_MAX_ALLOC ) { +ckpt_debug ("skb total length %llu larger than max of %lu\n" , + h->skb_len,SKB_MAX_ALLOC ); +return -EINVAL ; + } + +skb_set_transport_header (skb, h->transport_header); +skb_set_network_header (skb, h->network_header); +skb_set_mac_header (skb, h->mac_header); + skb->mac_len = h->mac_len; + +/* FIXME: This should probably be sanitized per-protocol to +* make sure nothing bad happens if it is hijacked. For the +* current set of protocols that we restore this way, the data +* contained within is not very risky (flags and sequence +* numbers) but could still be evalutated from a +* could-the-user- have-set-these-flags point of view. +*/ +memcpy (skb->cb, h->cb,sizeof (skb->cb)); + + skb->data = skb->head + skb->hdr_len; + skb->len = h->skb_len; + +return 0 ; +} + +static int sock_restore_skb_frag (struct ckpt_ctx *ctx, +struct sk_buff *skb, +int frag_idx) +{ +int ret =0 ; +int fraglen; +struct page *page; +void *buf; + + fraglen =_ckpt_read_obj_type (ctx,NULL ,0 ,CKPT_HDR_BUFFER ); +if (fraglen <0 ) +return fraglen; + +if (fraglen >PAGE_SIZE ) { +ckpt_debug ("skb frag size %i > PAGE_SIZE\n" , fraglen); +return -EINVAL ; + } + + page =alloc_page (GFP_KERNEL ); +if (!page) +return -ENOMEM ; + + buf =kmap (page); + ret =ckpt_kread (ctx, buf, fraglen); +kunmap (page); + +if (ret) { +ckpt_debug ("failed to read fragment: %i\n" , ret); + ret = -EINVAL ; +__free_page (page); + }else { +ckpt_debug ("read %i for fragment %i\n" , fraglen, frag_idx); +skb_add_rx_frag (skb, frag_idx, page,0 , fraglen); + } + +return ret <0 ? ret : fraglen; +} + +struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx) +{ +struct ckpt_hdr_socket_buffer *h; +struct sk_buff *skb =NULL ; +int i; +int ret =0 ; + + h =ckpt_read_obj_type (ctx,sizeof (*h),CKPT_HDR_SOCKET_BUFFER ); +if (IS_ERR (h)) +return (struct sk_buff *)h; + +if (h->lin_len >SKB_MAX_ALLOC ) { +ckpt_debug ("socket linear buffer too big (%llu > %lu)\n" , + h->lin_len,SKB_MAX_ALLOC ); + ret = -ENOSPC ; + goto out; + }else if (h->frg_len >SKB_MAX_ALLOC ) { +ckpt_debug ("socket frag size too big (%llu > %lu\n" , + h->frg_len,SKB_MAX_ALLOC ); + ret = -ENOSPC ; + goto out; + } + + skb =alloc_skb (h->lin_len,GFP_KERNEL ); +if (!skb) { + ret = -ENOMEM ; + goto out; + } + + ret =_ckpt_read_obj_type (ctx,skb_put (skb, h->lin_len), + h->lin_len,CKPT_HDR_BUFFER ); +ckpt_debug ("read linear skb length %llu: %i\n" , h->lin_len, ret); +if (ret <0 ) { + goto out; + } + +for (i =0 ; i < h->nr_frags; i++) { + ret =sock_restore_skb_frag (ctx, skb, i); +ckpt_debug ("read skb frag %i/%i: %i\n" , + i +1 , h->nr_frags, ret); +if (ret <0 ) + goto out; + h->frg_len -= ret; + } + +if (h->frg_len !=0 ) { +ckpt_debug ("length %llu remaining after reading frags\n" , + h->frg_len); + ret = -EINVAL ; + goto out; + } + +sock_restore_header_info (skb, h); + + out: +ckpt_hdr_put (ctx, h); +if (ret <0 ) { +kfree_skb (skb); + skb =ERR_PTR (ret); + } + +return skb; +} + +static int __sock_write_skb (struct ckpt_ctx *ctx, +struct sk_buff *skb, +int dst_objref) +{ +struct ckpt_hdr_socket_buffer *h; +int ret =0 ; +int i; + + h =ckpt_hdr_get_type (ctx,sizeof (*h),CKPT_HDR_SOCKET_BUFFER ); +if (!h) +return -ENOMEM ; + +if (dst_objref >0 ) { +BUG_ON (!skb->sk); + ret =checkpoint_obj (ctx, skb->sk,CKPT_OBJ_SOCK ); +if (ret <0 ) + goto out; + h->sk_objref = ret; + h->pr_objref = dst_objref; + } + +sock_record_header_info (skb, h); + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *) h); +if (ret <0 ) + goto out; + + ret =ckpt_write_obj_type (ctx, skb->head, h->lin_len,CKPT_HDR_BUFFER ); +ckpt_debug ("writing skb linear region %llu: %i\n" , h->lin_len, ret); +if (ret <0 ) + goto out; + +for (i =0 ; i <skb_shinfo (skb)->nr_frags; i++) { +skb_frag_t *frag = &skb_shinfo (skb)->frags[i]; +u8 *vaddr =kmap (frag->page); + +ckpt_debug ("writing buffer fragment %i/%i (%i)\n" , + i +1 , h->nr_frags, frag->size); + ret =ckpt_write_obj_type (ctx, vaddr + frag->page_offset, + frag->size,CKPT_HDR_BUFFER ); +kunmap (frag->page); + h->frg_len -= frag->size; +if (ret <0 ) + goto out; + } + +WARN_ON (h->frg_len !=0 ); + + out: +ckpt_hdr_put (ctx, h); + +return ret; +} +static int __sock_write_buffers (struct ckpt_ctx *ctx,struct sk_buff_head *queue,int dst_objref)@@ -95,13 +324,8 @@ static int __sock_write_buffers(struct ckpt_ctx *ctx,struct sk_buff *skb; skb_queue_walk(queue, skb) { -struct ckpt_hdr_socket_buffer *h;int ret =0 ; -/* FIXME: This could be a false positive for non-unix -* buffers, so add a type check here in the -* future -*/ if (UNIXCB (skb).fp) {ckpt_write_err (ctx,"TE" ,"af_unix: pass fd" , -EBUSY );return -EBUSY ;@@ -113,25 +337,8 @@ static int __sock_write_buffers(struct ckpt_ctx *ctx,* because we don't save out (or restore) the control * information contained in the skb. */ - h =ckpt_hdr_get_type (ctx,sizeof (*h),CKPT_HDR_SOCKET_BUFFER ); -if (!h) -return -ENOMEM ; - -BUG_ON (!skb->sk); - ret =checkpoint_obj (ctx, skb->sk,CKPT_OBJ_SOCK ); -if (ret <0 ) - goto end; - h->sk_objref = ret; - h->pr_objref = dst_objref; - - ret =ckpt_write_obj (ctx, (struct ckpt_hdr *) h); -if (ret <0 ) - goto end; - ret =ckpt_write_obj_type (ctx, skb->data, skb->len, -CKPT_HDR_BUFFER ); - end: -ckpt_hdr_put (ctx, h); + ret =__sock_write_skb (ctx, skb, dst_objref);if (ret <0 )return ret; }Signature · 6 lines
1.6.3.3 _______________________________________________ Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org https://lists.linux-foundation.org/mailman/listinfo/containers
Re: [PATCH 1/4] Unify skb read/write functions and fix for fragmented buffers
In reply to a message not in the archiveQuoted 31 lines · Dan Smith
The INET code often creates socket buffers by attaching fragments instead of writing to the linear region. This extends the skb write functions to write out the linear and fragment regions of an skb, and adds a function to be used by others wishing to restore an skb in the same way. This also includes the header-mark-setting bits from a previous patch.
Signed-off-by Dan Smith <danms-r/[email protected]>--- include/linux/checkpoint.h | 1 + include/linux/checkpoint_hdr.h | 11 ++ net/checkpoint.c | 253 ++++++++++++++++++++++++++++++++++++---- 3 files changed, 242 insertions(+), 23 deletions(-)include/linux/checkpoint.h +1
-0@@ -100,6 +100,7 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx,struct socket *socket, struct sockaddr *loc, unsigned *loc_len, struct sockaddr *rem, unsigned *rem_len); +struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx);/* ckpt kflags */ #define ckpt_set_ctx_kflag (__ctx, __kflag) \include/linux/checkpoint_hdr.h +0
-0@@ -561,8 +561,19 @@ struct ckpt_hdr_socket_queue {
include/linux/checkpoint_hdr.h struct ckpt_hdr_socket_queue { · line 571
Quoted 10 lines
include/linux/checkpoint_hdr.h +8
-0struct ckpt_hdr_socket_buffer {struct ckpt_hdr h; +__u64 transport_header; +__u64 network_header; +__u64 mac_header; +__u64 lin_len;/* Length of linear data */ +__u64 frg_len;/* Length of fragment data */ +__u64 skb_len;/* Length of skb (adjusted) */ +__u64 hdr_len;/* Length of skipped header */ +__u64 mac_len;
Can you use u32 (or even less ?) for these ?
include/linux/checkpoint_hdr.h struct ckpt_hdr_socket_queue { · line 576
Quoted 5 lines
include/linux/checkpoint_hdr.h +3
-0__s32 sk_objref;__s32 pr_objref; +__u16 protocol; +__u16 nr_frags; +__u8 cb[48 ];
Do you it will ever be required that cb[] be aligned ?
Quoted 1 line
include/linux/checkpoint_hdr.h +0
-0};
[...]
Quoted 15 lines
include/linux/checkpoint_hdr.h +15
-0+static int sock_restore_skb_frag (struct ckpt_ctx *ctx, +struct sk_buff *skb, +int frag_idx) +{ +int ret =0 ; +int fraglen; +struct page *page; +void *buf; + + fraglen =_ckpt_read_obj_type (ctx,NULL ,0 ,CKPT_HDR_BUFFER ); +if (fraglen <0 ) +return fraglen; + +if (fraglen >PAGE_SIZE ) { +ckpt_debug ("skb frag size %i > PAGE_SIZE\n" , fraglen);
include/linux/checkpoint_hdr.h
Quoted 10 lines
include/linux/checkpoint_hdr.h +10
-0+return -EINVAL ; + } + + page =alloc_page (GFP_KERNEL ); +if (!page) +return -ENOMEM ; + + buf =kmap (page); + ret =ckpt_kread (ctx, buf, fraglen); +kunmap (page);
I'm unsure how much of a performance issue this is - I sort of expected a comment from Dave Hansen about this; Did you consider reusing the restore_read_page() from checkpoint/memory.c ? (and the matching checkpoint_dump_page() for the checkpoint).
Quoted 40 lines
include/linux/checkpoint_hdr.h +40
-0+ +if (ret) { +ckpt_debug ("failed to read fragment: %i\n" , ret); + ret = -EINVAL ; +__free_page (page); + }else { +ckpt_debug ("read %i for fragment %i\n" , fraglen, frag_idx); +skb_add_rx_frag (skb, frag_idx, page,0 , fraglen); + } + +return ret <0 ? ret : fraglen; +} + +struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx) +{ +struct ckpt_hdr_socket_buffer *h; +struct sk_buff *skb =NULL ; +int i; +int ret =0 ; + + h =ckpt_read_obj_type (ctx,sizeof (*h),CKPT_HDR_SOCKET_BUFFER ); +if (IS_ERR (h)) +return (struct sk_buff *)h; + +if (h->lin_len >SKB_MAX_ALLOC ) { +ckpt_debug ("socket linear buffer too big (%llu > %lu)\n" , + h->lin_len,SKB_MAX_ALLOC ); + ret = -ENOSPC ; + goto out; + }else if (h->frg_len >SKB_MAX_ALLOC ) { +ckpt_debug ("socket frag size too big (%llu > %lu\n" , + h->frg_len,SKB_MAX_ALLOC ); + ret = -ENOSPC ; + goto out; + } + + skb =alloc_skb (h->lin_len,GFP_KERNEL ); +if (!skb) { + ret = -ENOMEM ; + goto out;
include/linux/checkpoint_hdr.h
Quoted 10 lines
include/linux/checkpoint_hdr.h +10
-0+ } + + ret =_ckpt_read_obj_type (ctx,skb_put (skb, h->lin_len), + h->lin_len,CKPT_HDR_BUFFER ); +ckpt_debug ("read linear skb length %llu: %i\n" , h->lin_len, ret); +if (ret <0 ) { + goto out; + } + +for (i =0 ; i < h->nr_frags; i++) {
Sanitize h->nr_frags ?
Quoted 52 lines
include/linux/checkpoint_hdr.h +52
-0+ ret =sock_restore_skb_frag (ctx, skb, i); +ckpt_debug ("read skb frag %i/%i: %i\n" , + i +1 , h->nr_frags, ret); +if (ret <0 ) + goto out; + h->frg_len -= ret; + } + +if (h->frg_len !=0 ) { +ckpt_debug ("length %llu remaining after reading frags\n" , + h->frg_len); + ret = -EINVAL ; + goto out; + } + +sock_restore_header_info (skb, h); + + out: +ckpt_hdr_put (ctx, h); +if (ret <0 ) { +kfree_skb (skb); + skb =ERR_PTR (ret); + } + +return skb; +} + +static int __sock_write_skb (struct ckpt_ctx *ctx, +struct sk_buff *skb, +int dst_objref) +{ +struct ckpt_hdr_socket_buffer *h; +int ret =0 ; +int i; + + h =ckpt_hdr_get_type (ctx,sizeof (*h),CKPT_HDR_SOCKET_BUFFER ); +if (!h) +return -ENOMEM ; + +if (dst_objref >0 ) { +BUG_ON (!skb->sk); + ret =checkpoint_obj (ctx, skb->sk,CKPT_OBJ_SOCK ); +if (ret <0 ) + goto out; + h->sk_objref = ret; + h->pr_objref = dst_objref; + } + +sock_record_header_info (skb, h); + + ret =ckpt_write_obj (ctx, (struct ckpt_hdr *) h); +if (ret <0 )
include/linux/checkpoint_hdr.h
Quoted 10 lines
include/linux/checkpoint_hdr.h +10
-0+ goto out; + + ret =ckpt_write_obj_type (ctx, skb->head, h->lin_len,CKPT_HDR_BUFFER ); +ckpt_debug ("writing skb linear region %llu: %i\n" , h->lin_len, ret); +if (ret <0 ) + goto out; + +for (i =0 ; i <skb_shinfo (skb)->nr_frags; i++) { +skb_frag_t *frag = &skb_shinfo (skb)->frags[i]; +u8 *vaddr =kmap (frag->page);
Here, too, consider checkpoint_dump_page() to avoid kmap() ?
It makes sense to have a scratch page to be used (also) for this, on ckpt_ctx - to avoid alloc/dealloc repeatedly.
include/linux/checkpoint_hdr.h
Quoted 6 lines
include/linux/checkpoint_hdr.h +6
-0+ +ckpt_debug ("writing buffer fragment %i/%i (%i)\n" , + i +1 , h->nr_frags, frag->size); + ret =ckpt_write_obj_type (ctx, vaddr + frag->page_offset, + frag->size,CKPT_HDR_BUFFER ); +kunmap (frag->page);
[...]
Oren.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 1/4] Unify skb read/write functions and fix for fragmented buffers
Quoted 10 lines
include/linux/checkpoint_hdr.h struct ckpt_hdr_socket_queue { · line 571
Quoted 10 lines
struct
ckpt_hdr_socket_buffer{
structckpt_hdrh;include/linux/checkpoint_hdr.h +8
-0+__u64 transport_header; +__u64 network_header; +__u64 mac_header; +__u64 lin_len;/* Length of linear data */ +__u64 frg_len;/* Length of fragment data */ +__u64 skb_len;/* Length of skb (adjusted) */ +__u64 hdr_len;/* Length of skipped header */ +__u64 mac_len;
OL> Can you use u32 (or even less ?) for these ?
Well, in the structure the len is an unsigned int, so u64 seemed appropriate.
Quoted 5 lines
include/linux/checkpoint_hdr.h struct ckpt_hdr_socket_queue { · line 576
Quoted 5 lines
__s32sk_objref;__s32pr_objref;include/linux/checkpoint_hdr.h +3
-0+__u16 protocol; +__u16 nr_frags; +__u8 cb[48 ];
OL> Do you it will ever be required that cb[] be aligned ?
It's not aligned in the real structure and it's mostly opaque data, so no, I wouldn't think so.
OL> I'm unsure how much of a performance issue this is - I sort of OL> expected a comment from Dave Hansen about this; Did you consider OL> reusing the restore_read_page() from checkpoint/memory.c ? (and OL> the matching checkpoint_dump_page() for the checkpoint).
Nope, I'll take a look.
Quoted 1 line
net/checkpoint.c static int sock_copy_buffers(struct sk_buff_head *from, · line 238
Quoted 1 line
net/checkpoint.c +1
-0+for (i =0 ; i < h->nr_frags; i++) {
OL> Sanitize h->nr_frags ?
To what? I've already checked h->frg_len at this point, so I think maybe just making sure frag_len never crosses zero would be sufficient.
Quoted 3 lines
net/checkpoint.c static int sock_copy_buffers(struct sk_buff_head *from, · line 300
Quoted 3 lines
net/checkpoint.c +3
-0+for (i =0 ; i <skb_shinfo (skb)->nr_frags; i++) { +skb_frag_t *frag = &skb_shinfo (skb)->frags[i]; +u8 *vaddr =kmap (frag->page);
OL> Here, too, consider checkpoint_dump_page() to avoid kmap() ?
OL> It makes sense to have a scratch page to be used (also) for this, OL> on ckpt_ctx - to avoid alloc/dealloc repeatedly.
Hmm, is there an alloc/dealloc here?
I'll take a look at checkpoint_dump_page()...
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: danms-r/[email protected]
A message that is not in the archive (sent off-list or not yet received): 87einyuwum.fsf-FLMGYpZoEPULwtHQx/[email protected]
Re: [PATCH 1/4] Unify skb read/write functions and fix for fragmented buffers
Quoted 34 lines · Dan Smith
Quoted 10 lines
include/linux/checkpoint_hdr.h struct ckpt_hdr_socket_queue { · line 571
Quoted 10 lines
struct
ckpt_hdr_socket_buffer{
structckpt_hdrh;include/linux/checkpoint_hdr.h +8
-0+__u64 transport_header; +__u64 network_header; +__u64 mac_header; +__u64 lin_len;/* Length of linear data */ +__u64 frg_len;/* Length of fragment data */ +__u64 skb_len;/* Length of skb (adjusted) */ +__u64 hdr_len;/* Length of skipped header */ +__u64 mac_len;OL> Can you use u32 (or even less ?) for these ?
Well, in the structure the len is an unsigned int, so u64 seemed appropriate.
Quoted 5 lines
include/linux/checkpoint_hdr.h struct ckpt_hdr_socket_queue { · line 576
Quoted 5 lines
__s32sk_objref;__s32pr_objref;include/linux/checkpoint_hdr.h +3
-0+__u16 protocol; +__u16 nr_frags; +__u8 cb[48 ];OL> Do you it will ever be required that cb[] be aligned ?
It's not aligned in the real structure and it's mostly opaque data, so no, I wouldn't think so.
OL> I'm unsure how much of a performance issue this is - I sort of OL> expected a comment from Dave Hansen about this; Did you consider OL> reusing the
restore_read_page()fromcheckpoint/memory.c? (and OL> the matchingcheckpoint_dump_page()for the checkpoint).Nope, I'll take a look.
Quoted 7 lines
Quoted 1 line
net/checkpoint.c static int sock_copy_buffers(struct sk_buff_head *from, · line 238
Quoted 1 line
net/checkpoint.c +1
-0+for (i =0 ; i < h->nr_frags; i++) {OL> Sanitize
h->nr_frags?To what? I've already checked
h->frg_lenat this point, so I think maybe just making surefrag_lennever crosses zero would be sufficient.
MAX_SKB_FRAGS.
You use @i when calling skb_add_rx_frag() which in turn uses it to index in the skb's fragments array without checking.
Quoted 11 lines
Quoted 4 lines
net/checkpoint.c static int sock_copy_buffers(struct sk_buff_head *from, · line 300
Quoted 4 lines
net/checkpoint.c +3
-0+for (i =0 ; i <skb_shinfo (skb)->nr_frags; i++) { +skb_frag_t *frag = &skb_shinfo (skb)->frags[i]; +u8 *vaddr =kmap (frag->page);OL> Here, too, consider
checkpoint_dump_page()to avoidkmap()?OL> It makes sense to have a scratch page to be used (also) for this, OL> on
ckpt_ctx- to avoid alloc/dealloc repeatedly.Hmm, is there an alloc/dealloc here?
Sorry, should have been more explicit: these two functions use an temporary page to read the data to (write the data from) without holding kmap(); instead they use kmap_atomic(), copy the data to the scratch page, kunmap_atomic(), and then write the data.
When dumping memory, the scratch page is allocated once for every big group of pages and then freed Here, you will need to allocate for each skb, which makes less sense - hence my suggestion that we keep a scratch page for that purpose. (and it can be used by the memory dump/restore too).
Quoted 3 lines
I'll take a look at
checkpoint_dump_page()...
Oren.
PATCH 3/4 Update the UNIX buffer restore code to match the new format saved in the image file
In reply to a message not in the archive--- net/unix/checkpoint.c | 54 ++++++++++++++++++++++++++---------------------- 1 files changed, 29 insertions(+), 25 deletions(-)
net/unix/checkpoint.c +29 -25
@@ -200,7 +200,6 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx,uint8_t peer_shutdown =0 ;void *buf =NULL ;int sndbuf; -int len;int ret =0 ;memset (&msg,0 ,sizeof (msg));
@@ -209,14 +208,30 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx,if (IS_ERR (h))return PTR_ERR (h); - len =_ckpt_read_obj_type (ctx,NULL ,0 ,CKPT_HDR_BUFFER ); -if (len <0 ) { - ret = len; +if (h->lin_len >SKB_MAX_ALLOC ) { +ckpt_debug ("socket buffer too big (%llu > %lu)\n" , + h->lin_len,SKB_MAX_ALLOC ); + ret = -EINVAL ; + goto out; + }else if (h->nr_frags !=0 ) { +ckpt_debug ("unix socket claims to have fragments\n" ); + ret = -EINVAL ; goto out; - }else if (len >SKB_MAX_ALLOC ) { -ckpt_debug ("Socket buffer too big (%i > %lu)" , - len,SKB_MAX_ALLOC ); - ret = -ENOSPC ; + } + + buf =kmalloc (h->lin_len,GFP_KERNEL ); +if (!buf) { + ret = -ENOMEM ; + goto out; + } + + kvec.iov_len = h->lin_len; + kvec.iov_base = buf; + ret =_ckpt_read_obj_type (ctx, kvec.iov_base, + h->lin_len,CKPT_HDR_BUFFER ); +ckpt_debug ("read unix socket buffer %llu: %i\n" , h->lin_len, ret); +if (ret < h->lin_len) { + ret = -EINVAL ; goto out; }
@@ -245,18 +260,6 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx, } } - kvec.iov_len = len; - buf =kmalloc (len,GFP_KERNEL ); - kvec.iov_base = buf; -if (!buf) { - ret = -ENOMEM ; - goto out; - } - - ret =ckpt_kread (ctx, kvec.iov_base, len); -if (ret <0 ) - goto out; - msg.msg_name = addr; msg.msg_namelen = addrlen;
@@ -272,15 +275,16 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx,/* Make sure there's room in the send buffer */ sndbuf = sk->sk_sndbuf; -if (((sk->sk_sndbuf -atomic_read (&sk->sk_wmem_alloc)) < len) && +if (((sk->sk_sndbuf -atomic_read (&sk->sk_wmem_alloc)) < h->lin_len) &&capable (CAP_NET_ADMIN )) - sk->sk_sndbuf += len; + sk->sk_sndbuf += h->lin_len;else sk->sk_sndbuf = sysctl_wmem_max; - ret =kernel_sendmsg (sk->sk_socket, &msg, &kvec,1 , len); -ckpt_debug ("kernel_sendmsg(%i,%i): %i\n" , h->sk_objref, len, ret); -if ((ret >0 ) && (ret != len)) + ret =kernel_sendmsg (sk->sk_socket, &msg, &kvec,1 , h->lin_len); +ckpt_debug ("kernel_sendmsg(%i,%llu): %i\n" , + h->sk_objref, h->lin_len, ret); +if ((ret >0 ) && (ret != h->lin_len)) ret = -ENOMEM ; sk->sk_sndbuf = sndbuf;
Signature · 1 line
1.6.3.3
A message that is not in the archive (sent off-list or not yet received): 1257878856-25520-4-git-send-email-danms-r/[email protected]
Re: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
net/unix/checkpoint.c static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx, · line 282
Quoted 9 lines · Dan Smith
net/unix/checkpoint.c +2
-2/* Make sure there's room in the send buffer */ sndbuf = sk->sk_sndbuf; -if (((sk->sk_sndbuf -atomic_read (&sk->sk_wmem_alloc)) < len) && +if (((sk->sk_sndbuf -atomic_read (&sk->sk_wmem_alloc)) < h->lin_len) &&capable (CAP_NET_ADMIN )) - sk->sk_sndbuf += len; + sk->sk_sndbuf += h->lin_len;else sk->sk_sndbuf = sysctl_wmem_max;
Can you explain what's going on here? 'if the size of the send buffer minus tranmit queue bytes committed is less than linear length, then if you're privileged you add h->len to sk->sk_sndbuf, but if either you're not privileged or the length was greater than linear length, then you add sysctl_wmem_max.' ?
I realize that's a question on the original code, not on this patch...
-serge
A message that is not in the archive (sent off-list or not yet received): 20091111213851.GE8761-r/[email protected]
Re: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
Quoted 8 lines
Quoted 8 lines
/* Make sure there's room in the send buffer */
sndbuf =sk->sk_sndbuf;net/unix/checkpoint.c +1
-1- if (((sk->sk_sndbuf - atomic_read(&sk->sk_wmem_alloc)) < len) && + if (((sk->sk_sndbuf - atomic_read(&sk->sk_wmem_alloc)) < h->lin_len) &&
capable(CAP_NET_ADMIN))net/unix/checkpoint.c +1
-1- sk->sk_sndbuf += len; + sk->sk_sndbuf += h->lin_len;else
sk-> sk_sndbuf = sysctl_wmem_max;
SH> Can you explain what's going on here?
If we're trying to restore a buffer that is larger than the remaining space in the buffer, then one of two things can happen:
1. You're privileged and we make the space you need
2. You're not privileged so we give you the benefit of the doubt and set the buffer limit to the system default
In the case of 2, if that system default still isn't enough then the sendmsg() will fail like it normally would.
The reason for this is that the application could have loaded up its legitimate buffer with data and then set the buffer limit low. That doesn't purge the data it already had buffered, it just limits how much you can add to it. So, in order to not fail a restart of such a legitimate situation, we assume the system default instead of the limit set by the user.
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: danms-r/[email protected]
A message that is not in the archive (sent off-list or not yet received): 87vdhgvi6b.fsf-FLMGYpZoEPULwtHQx/[email protected]
Re: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
Quoted 15 lines · Dan Smith
Quoted 8 lines
Quoted 8 lines
/* Make sure there's room in the send buffer */
sndbuf =sk->sk_sndbuf;net/unix/checkpoint.c +1
-1- if (((sk->sk_sndbuf - atomic_read(&sk->sk_wmem_alloc)) < len) && + if (((sk->sk_sndbuf - atomic_read(&sk->sk_wmem_alloc)) < h->lin_len) &&
capable(CAP_NET_ADMIN))net/unix/checkpoint.c +1
-1- sk->sk_sndbuf += len; + sk->sk_sndbuf += h->lin_len;else
sk->
sk_sndbuf=sysctl_wmem_max;SH> Can you explain what's going on here?
If we're trying to restore a buffer that is larger than the remaining space in the buffer, then one of two things can happen:
Quoted 6 lines
1. You're privileged and we make the space you need
2. You're not privileged so we give you the benefit of the doubt and set the buffer limit to the system defaultIn the case of 2, if that system default still isn't enough then the
sendmsg()will fail like it normally would.
But so should check whether h->len_len < sysctl_wmem_max before doing the capable check? Remember that any check for capable() will set PF_SUPERPRIV on the task, so it's better to not call it if it wasn't definately needed.
Quoted 6 lines
The reason for this is that the application could have loaded up its legitimate buffer with data and then set the buffer limit low. That doesn't purge the data it already had buffered, it just limits how much you can add to it. So, in order to not fail a restart of such a legitimate situation, we assume the system default instead of the limit set by the user.
thanks,
-serge
A message that is not in the archive (sent off-list or not yet received): 20091112021824.GA14646-r/[email protected]
Re: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
SH> But so should check whether h->len_len < sysctl_wmem_max before SH> doing the capable check? Remember that any check for capable() SH> will set PF_SUPERPRIV on the task, so it's better to not call it SH> if it wasn't definately needed.
Okay, sure.
Signature · 3 lines
Dan Smith IBM Linux Technology Center email: danms-r/[email protected]
A message that is not in the archive (sent off-list or not yet received): 87k4xvvc5b.fsf-FLMGYpZoEPULwtHQx/[email protected]
Re: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
Quoted 6 lines · Dan Smith
SH> But so should check whether
h->len_len<sysctl_wmem_maxbefore SH> doing the capable check? Remember that any check forcapable()SH> will setPF_SUPERPRIVon the task, so it's better to not call it SH> if it wasn't definately needed.Okay, sure.
Ok, thanks - I figured if I was misunderstanding, then this suggestion would sound ridiculous and you'd suitably ridicule me :)
thanks,
-serge
Re: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
In reply to a message not in the archiveQuoted 20 lines · Serge E. Hallyn
Quoted 19 lines · Dan Smith
Quoted 8 lines
Quoted 8 lines
/* Make sure there's room in the send buffer */
sndbuf =sk->sk_sndbuf;(quoted diff) +1
-1- if (((sk->sk_sndbuf - atomic_read(&sk->sk_wmem_alloc)) < len) && + if (((sk->sk_sndbuf - atomic_read(&sk->sk_wmem_alloc)) < h->lin_len) &&
capable(CAP_NET_ADMIN))(quoted diff) +1
-1- sk->sk_sndbuf += len; + sk->sk_sndbuf += h->lin_len;else
sk->
sk_sndbuf=sysctl_wmem_max;SH> Can you explain what's going on here?
If we're trying to restore a buffer that is larger than the remaining space in the buffer, then one of two things can happen:
1. You're privileged and we make the space you need
2. You're not privileged so we give you the benefit of the doubt and set the buffer limit to the system default
Quoted 14 lines
Quoted 2 lines
In the case of 2, if that system default still isn't enough then the
sendmsg()will fail like it normally would.But so should check whether
h->len_len<sysctl_wmem_maxbefore doing the capable check? Remember that any check forcapable()will setPF_SUPERPRIVon the task, so it's better to not call it if it wasn't definately needed.Quoted 6 lines
The reason for this is that the application could have loaded up its legitimate buffer with data and then set the buffer limit low. That doesn't purge the data it already had buffered, it just limits how much you can add to it. So, in order to not fail a restart of such a legitimate situation, we assume the system default instead of the limit set by the user.
Maybe also worth beefing up the comment near that code to help future reviewers/developers...
The patch is good.
Oren
Serge E. Hallyn
Acked-byRe: [PATCH 3/4] Update the UNIX buffer restore code to match the new format saved in the image file
In reply to a message not in the archiveQuoted 1 line · Dan Smith
Signed-off-by Dan Smith <danms-r/[email protected]>
Anyway the patch itself seems fine :)
Quoted 102 lines
--- net/unix/checkpoint.c | 54 ++++++++++++++++++++++++++---------------------- 1 files changed, 29 insertions(+), 25 deletions(-)net/unix/checkpoint.c +29
-25@@ -200,7 +200,6 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx,uint8_t peer_shutdown =0 ;void *buf =NULL ;int sndbuf; -int len;int ret =0 ;memset (&msg,0 ,sizeof (msg));@@ -209,14 +208,30 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx,if (IS_ERR (h))return PTR_ERR (h); - len =_ckpt_read_obj_type (ctx,NULL ,0 ,CKPT_HDR_BUFFER ); -if (len <0 ) { - ret = len; +if (h->lin_len >SKB_MAX_ALLOC ) { +ckpt_debug ("socket buffer too big (%llu > %lu)\n" , + h->lin_len,SKB_MAX_ALLOC ); + ret = -EINVAL ; + goto out; + }else if (h->nr_frags !=0 ) { +ckpt_debug ("unix socket claims to have fragments\n" ); + ret = -EINVAL ; goto out; - }else if (len >SKB_MAX_ALLOC ) { -ckpt_debug ("Socket buffer too big (%i > %lu)" , - len,SKB_MAX_ALLOC ); - ret = -ENOSPC ; + } + + buf =kmalloc (h->lin_len,GFP_KERNEL ); +if (!buf) { + ret = -ENOMEM ; + goto out; + } + + kvec.iov_len = h->lin_len; + kvec.iov_base = buf; + ret =_ckpt_read_obj_type (ctx, kvec.iov_base, + h->lin_len,CKPT_HDR_BUFFER ); +ckpt_debug ("read unix socket buffer %llu: %i\n" , h->lin_len, ret); +if (ret < h->lin_len) { + ret = -EINVAL ; goto out; }@@ -245,18 +260,6 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx, } } - kvec.iov_len = len; - buf =kmalloc (len,GFP_KERNEL ); - kvec.iov_base = buf; -if (!buf) { - ret = -ENOMEM ; - goto out; - } - - ret =ckpt_kread (ctx, kvec.iov_base, len); -if (ret <0 ) - goto out; - msg.msg_name = addr; msg.msg_namelen = addrlen;@@ -272,15 +275,16 @@ static int sock_read_buffer_sendmsg(struct ckpt_ctx *ctx,/* Make sure there's room in the send buffer */ sndbuf = sk->sk_sndbuf; -if (((sk->sk_sndbuf -atomic_read (&sk->sk_wmem_alloc)) < len) && +if (((sk->sk_sndbuf -atomic_read (&sk->sk_wmem_alloc)) < h->lin_len) &&capable (CAP_NET_ADMIN )) - sk->sk_sndbuf += len; + sk->sk_sndbuf += h->lin_len;else sk->sk_sndbuf = sysctl_wmem_max; - ret =kernel_sendmsg (sk->sk_socket, &msg, &kvec,1 , len); -ckpt_debug ("kernel_sendmsg(%i,%i): %i\n" , h->sk_objref, len, ret); -if ((ret >0 ) && (ret != len)) + ret =kernel_sendmsg (sk->sk_socket, &msg, &kvec,1 , h->lin_len); +ckpt_debug ("kernel_sendmsg(%i,%llu): %i\n" , + h->sk_objref, h->lin_len, ret); +if ((ret >0 ) && (ret != h->lin_len)) ret = -ENOMEM ; sk->sk_sndbuf = sndbuf;Signature · 6 lines
1.6.3.3 _______________________________________________ Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org https://lists.linux-foundation.org/mailman/listinfo/containers
PATCH 4/4 Add some content to the readme.txt for socket c/r
In reply to a message not in the archive--- Documentation/checkpoint/readme.txt | 21 +++++++++++++++++++++ 1 files changed, 21 insertions(+), 0 deletions(-)
Documentation/checkpoint/readme.txt +21 -0
@@ -341,6 +341,27 @@ we will be forced to more carefully review each of those features. However, this can be controlled with a sysctl-variable. +Sockets +======= + +For AF_UNIX sockets, both endpoints must be within the checkpointed +task set to maintain a connected state after restart. UNIX sockets +that are in the process of passing a descriptor will cause the +checkpoint to fail with -EBUSY indicating a transient state that +cannot be checkpointed. Listening sockets with an unaccepted peer +will also cause an -EBUSY result. + +AF_INET sockets with endpoints outside the checkpointed task set may +remain open if care is taken to avoid TCP timeouts and resets. +Careful use of a virtual IP address can help avoid emission of an RST +to the non-checkpointed endpoint. If desired, the +RESTART_SOCK_LISTENONLY flag may be passed to the restart syscall +which will cause all connected AF_INET sockets to be closed during the +restore process. Listening sockets will still be restored to their +original state, which makes this mode a candidate for something like +an HTTP server. + + Kernel interfaces =================
Signature · 1 line
1.6.3.3
The patches look good, see a few comments per patch.
Before I pull, please (1) use ckpt_err() instead of ckpt_write_err(), and also use ckpt_err() in the restart path where it makes sense.
I should be pushing v19-rc1 by tomorrow, perhaps better wait.
Thanks,
Oren.
Quoted 12 lines · Dan Smith
This set includes the previous INET connected sockets patch with a few trivial changes suggested by Oren, as well as the readme update. The first patch fixes socket buffer checkpoint to include fragments, and includes the changes in the previous first patch that added header mark information. As a result, this set now also includes a fix to the UNIX code to cope with the change.
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
RFC PATCH 2/4 Add c/r support for connected INET sockets (v4)
In reply to Dan SmithThis patch adds basic support for C/R of open INET sockets. I think that all the important bits of the TCP and ICSK socket structures is saved, but I think there is still some additional IPv6 stuff that needs to be handled.
With this patch applied, the following script can be used to demonstrate the functionality:
https://lists.linux-foundation.org/pipermail/containers/2009-October/021239.html
It shows that this enables migration of a sendmail process with open connections from one machine to another without dropping.
We probably need comments from the netdev people about the quality of sanity checking we do on the values in the ckpt_hdr_socket_inet structure on restart.
Note that this still doesn't address lingering sockets yet.
Changes in v4:
- Use the new socket buffer restore functions introduced in the previous patch
- Move listen_sockets list under the restart items in ckpt_ctx
- Rename RESTART_SOCK_LISTENONLY to RESTART_CONN_RESET
Changes in v3:
- Prevent restart from allowing a bind on a <1024 port unless the user is granted that capability
- Add some sanity checking in the inet_precheck() function to make sure the values read from the checkpoint image are within acceptable ranges
- Check the result of sock_restore_header_info() and fail if needed
Changes in v2:
- Restore saddr, rcv_saddr, daddr, sport, and dport from the sockaddr structure instead of saving them separately
- Fix 'sock' naming in sock_cptrst()
- Don't take the queue lock before skb_queue_tail() since it is done for us
- Allow "listen only" restore behavior if RESTART_SOCK_LISTENONLY flag is specified on sys_restart()
- Pull the implementation of the list of listening sockets back into this patch
- Fix dangling printk
- Add some comments around the parent/child restore logic
--- checkpoint/sys.c | 4 + include/linux/checkpoint.h | 5 +- include/linux/checkpoint_hdr.h | 95 +++++++++ include/linux/checkpoint_types.h | 1 + net/checkpoint.c | 27 ++-- net/ipv4/checkpoint.c | 391 ++++++++++++++++++++++++++++++++++---- 6 files changed, 473 insertions(+), 50 deletions(-)
checkpoint/sys.c +4 -0
@@ -221,6 +221,8 @@ static void ckpt_ctx_free(struct ckpt_ctx *ctx)kfree (ctx->pids_arr); +sock_listening_list_free (&ctx->listen_sockets); +kfree (ctx); }
@@ -249,6 +251,8 @@ static struct ckpt_ctx *ckpt_ctx_alloc(int fd, unsigned long uflags,spin_lock_init (&ctx->lock);#endif +INIT_LIST_HEAD (&ctx->listen_sockets); + err = -EBADF ; ctx->file =fget (fd);if (!ctx->file)
include/linux/checkpoint.h +4 -1
@@ -19,6 +19,7 @@#define RESTART_TASKSELF 0x1#define RESTART_FROZEN 0x2#define RESTART_GHOST 0x4 +#define RESTART_CONN_RESET 0x8#ifdef __KERNEL__#ifdef CONFIG_CHECKPOINT
@@ -48,7 +49,8 @@#define RESTART_USER_FLAGS \ (RESTART_TASKSELF | \ RESTART_FROZEN | \ - RESTART_GHOST) + RESTART_GHOST | \ + RESTART_CONN_RESET) extern int walk_task_subtree(struct task_struct *task, int (*func)(struct task_struct *, void *),
@@ -101,6 +103,7 @@ extern int ckpt_sock_getnames(struct ckpt_ctx *ctx, struct sockaddr *loc, unsigned *loc_len, struct sockaddr *rem, unsigned *rem_len);struct sk_buff *sock_restore_skb (struct ckpt_ctx *ctx); +void sock_listening_list_free (struct list_head *head);/* ckpt kflags */ #define ckpt_set_ctx_kflag (__ctx, __kflag) \
include/linux/checkpoint_hdr.h +95 -0
107 lines not on this page · raw message
include/linux/checkpoint_types.h +1 -0
@@ -78,6 +78,7 @@ struct ckpt_ctx {wait_queue_head_t waitq;/* waitqueue for restarting tasks */ wait_queue_head_t ghostq;/* waitqueue for ghost tasks */ struct cred *realcred, *ecred;/* tmp storage for cred at restart */ +struct list_head listen_sockets;/* listening parent sockets */ struct ckpt_stats stats;/* statistics */
net/checkpoint.c +15 -12
@@ -319,6 +319,7 @@ static int __sock_write_skb(struct ckpt_ctx *ctx,static int __sock_write_buffers (struct ckpt_ctx *ctx,struct sk_buff_head *queue, +uint16_t family,int dst_objref) {struct sk_buff *skb;
@@ -331,11 +332,11 @@ static int __sock_write_buffers(struct ckpt_ctx *ctx,return -EBUSY ; } -/* The other ancillary messages are always present -* unlike descriptors. Even though we can't detect -* them and fail the checkpoint, we're not at risk -* because we don't save out (or restore) the control -* information contained in the skb. +/* The other ancillary messages UNIX are always +* present unlike descriptors. Even though we can't +* detect them and fail the checkpoint, we're not at +* risk because we don't restore the control +* information in the UNIX code. */ ret =__sock_write_skb (ctx, skb, dst_objref);
@@ -348,6 +349,7 @@ static int __sock_write_buffers(struct ckpt_ctx *ctx,static int sock_write_buffers (struct ckpt_ctx *ctx,struct sk_buff_head *queue, +uint16_t family,int dst_objref) {struct ckpt_hdr_socket_queue *h;
@@ -367,7 +369,7 @@ static int sock_write_buffers(struct ckpt_ctx *ctx, h->skb_count = ret; ret =ckpt_write_obj (ctx, (struct ckpt_hdr *) h);if (!ret) - ret =__sock_write_buffers (ctx, &tmpq, dst_objref); + ret =__sock_write_buffers (ctx, &tmpq, family, dst_objref); out:ckpt_hdr_put (ctx, h);
@@ -389,12 +391,14 @@ int sock_deferred_write_buffers(void *data)return dst_objref; } - ret =sock_write_buffers (ctx, &dq->sk->sk_receive_queue, dst_objref); + ret =sock_write_buffers (ctx, &dq->sk->sk_receive_queue, + dq->sk->sk_family, dst_objref);ckpt_debug ("write recv buffers: %i\n" , ret);if (ret <0 )return ret; - ret =sock_write_buffers (ctx, &dq->sk->sk_write_queue, dst_objref); + ret =sock_write_buffers (ctx, &dq->sk->sk_write_queue, + dq->sk->sk_family, dst_objref);ckpt_debug ("write send buffers: %i\n" , ret);return ret;
@@ -919,10 +923,9 @@ struct sock *do_sock_restore(struct ckpt_ctx *ctx) goto err;if ((h->sock_common.family ==AF_INET ) && - (h->sock.state !=TCP_LISTEN )) { -/* Temporary hack to enable restore of TCP_LISTEN sockets -* while forcing anything else to a closed state -*/ + (h->sock.state !=TCP_LISTEN ) && + (ctx->uflags &RESTART_CONN_RESET )) { +ckpt_debug ("Forcing open socket closed\n" ); sock->sk->sk_state =TCP_CLOSE ; sock->state =SS_UNCONNECTED ; }
net/ipv4/checkpoint.c +354 -37
443 lines not on this page · raw message
Signature · 1 line
1.6.3.3
Re: [PATCH 2/4] [RFC] Add c/r support for connected INET sockets (v4)
Quoted 45 lines · Dan Smith
This patch adds basic support for C/R of open INET sockets. I think that all the important bits of the TCP and ICSK socket structures is saved, but I think there is still some additional IPv6 stuff that needs to be handled.
With this patch applied, the following script can be used to demonstrate the functionality:
https://lists.linux-foundation.org/pipermail/containers/2009-October/021239.html
It shows that this enables migration of a sendmail process with open connections from one machine to another without dropping.
We probably need comments from the netdev people about the quality of sanity checking we do on the values in the
ckpt_hdr_socket_inetstructure on restart.Note that this still doesn't address lingering sockets yet.
Changes in v4:
- Use the new socket buffer restore functions introduced in the previous patch
- Movelisten_socketslist under the restart items inckpt_ctx
- RenameRESTART_SOCK_LISTENONLYtoRESTART_CONN_RESETChanges in v3:
- Prevent restart from allowing a bind on a <1024 port unless the user is granted that capability
- Add some sanity checking in theinet_precheck()function to make sure the values read from the checkpoint image are within acceptable ranges
- Check the result ofsock_restore_header_info()and fail if neededChanges in v2:
- Restore saddr,rcv_saddr, daddr, sport, and dport from the sockaddr structure instead of saving them separately
- Fix 'sock' naming insock_cptrst()
- Don't take the queue lock beforeskb_queue_tail()since it is done for us
- Allow "listen only" restore behavior ifRESTART_SOCK_LISTENONLYflag is specified onsys_restart()
- Pull the implementation of the list of listening sockets back into this patch
- Fix dangling printk
- Add some comments around the parent/child restore logic
Quoted 3 lines
Acked-by Oren Laadan <[email protected]>Signed-off-by Dan Smith <[email protected]>
There are probably still gremlins in there somewhere, but imo it's past time to get this in the tree so we can be testing it.
-serge