libceph: Partially revert changes to support MSG_SPLICE_PAGES
Subsystems (3)
PATCH v2 1/1 libceph: Partially revert changes to support MSG_SPLICE_PAGES
Fix the mishandling of MSG_DONTWAIT and also reinstates the per-page checking of the source pages (which might have come from a DIO write by userspace) by partially reverting the changes to support MSG_SPLICE_PAGES and doing things a little differently. In messenger_v1:
(1) The ceph_tcp_sendpage() is resurrected and the callers reverted to use that.
(2) The callers now pass MSG_MORE unconditionally. Previously, they were passing in MSG_MORE|MSG_SENDPAGE_NOTLAST and then degrading that to just MSG_MORE on the last call to ->sendpage().
(3) Make ceph_tcp_sendpage() a wrapper around sendmsg() rather than sendpage(), setting MSG_SPLICE_PAGES if sendpage_ok() returns true on the page.
In messenger_v2:
(4) Bring back do_try_sendpage() and make the callers use that.
(5) Make do_try_sendpage() use sendmsg() for both cases and set MSG_SPLICE_PAGES if sendpage_ok() is set.
---
Notes:
ver #2)
- Removed mention of MSG_SENDPAGE_NOTLAST in comments.
- Changed some refs to sendpage to MSG_SPLICE_PAGES in comments.
- Init msg_iter in ceph_tcp_sendpage().
- Move setting of MSG_SPLICE_PAGES in do_try_sendpage() next to comment and adjust how it is cleared.
net/ceph/messenger_v1.c | 58 ++++++++++++++++++++----------- net/ceph/messenger_v2.c | 88 ++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 107 insertions(+), 39 deletions(-)
net/ceph/messenger_v1.c +37 -21
@@ -74,6 +74,39 @@ static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov,return r; } +/* +* @more: MSG_MORE or 0. +*/ +static int ceph_tcp_sendpage (struct socket *sock,struct page *page, +int offset,size_t size,int more) +{ +struct msghdr msg = { + .msg_flags =MSG_DONTWAIT |MSG_NOSIGNAL | more, + }; +struct bio_vec bvec; +int ret; + +/* +* MSG_SPLICE_PAGES cannot properly handle pages with page_count == 0, +* we need to fall back to sendmsg if that's the case. +* +* Same goes for slab pages: skb_can_coalesce() allows +* coalescing neighboring slab objects into a single frag which +* triggers one of hardened usercopy checks. +*/ +if (sendpage_ok (page)) + msg.msg_flags |=MSG_SPLICE_PAGES ; + +bvec_set_page (&bvec, page, size, offset); +iov_iter_bvec (&msg.msg_iter,ITER_DEST , &bvec,1 , size); + + ret =sock_sendmsg (sock, &msg); +if (ret == -EAGAIN ) + ret =0 ; + +return ret; +} +static void con_out_kvec_reset (struct ceph_connection *con) {BUG_ON (con->v1.out_skip);
@@ -450,10 +483,6 @@ static int write_partial_message_data(struct ceph_connection *con)*/ crc = do_datacrc ?le32_to_cpu (msg->footer.data_crc) :0 ;while (cursor->total_resid) { -struct bio_vec bvec; -struct msghdr msghdr = { - .msg_flags =MSG_SPLICE_PAGES , - };struct page *page;size_t page_offset;size_t length;
@@ -465,13 +494,8 @@ static int write_partial_message_data(struct ceph_connection *con) } page =ceph_msg_data_next (cursor, &page_offset, &length); -if (length != cursor->total_resid) - msghdr.msg_flags |=MSG_MORE ; - -bvec_set_page (&bvec, page, length, page_offset); -iov_iter_bvec (&msghdr.msg_iter,ITER_SOURCE , &bvec,1 , length); - - ret =sock_sendmsg (con->sock, &msghdr); + ret =ceph_tcp_sendpage (con->sock, page, page_offset, length, +MSG_MORE );if (ret <=0 ) {if (do_datacrc) msg->footer.data_crc =cpu_to_le32 (crc);
@@ -501,22 +525,14 @@ static int write_partial_message_data(struct ceph_connection *con)*/ static int write_partial_skip (struct ceph_connection *con) { -struct bio_vec bvec; -struct msghdr msghdr = { - .msg_flags =MSG_SPLICE_PAGES |MSG_MORE , - };int ret;dout ("%s %p %d left\n" , __func__, con, con->v1.out_skip);while (con->v1.out_skip >0 ) {size_t size =min (con->v1.out_skip, (int )PAGE_SIZE ); -if (size == con->v1.out_skip) - msghdr.msg_flags &= ~MSG_MORE ; -bvec_set_page (&bvec,ZERO_PAGE (0 ), size,0 ); -iov_iter_bvec (&msghdr.msg_iter,ITER_SOURCE , &bvec,1 , size); - - ret =sock_sendmsg (con->sock, &msghdr); + ret =ceph_tcp_sendpage (con->sock, ceph_zero_page,0 , size, +MSG_MORE );if (ret <=0 ) goto out; con->v1.out_skip -= ret;
net/ceph/messenger_v2.c +70 -18
108 lines not on this page · raw message
Ilya Dryomov
maintainerQuoted 78 lines · David Howells
Fix the mishandling of
MSG_DONTWAITand also reinstates the per-page checking of the source pages (which might have come from a DIO write by userspace) by partially reverting the changes to supportMSG_SPLICE_PAGESand doing things a little differently. Inmessenger_v1:(1) The
ceph_tcp_sendpage()is resurrected and the callers reverted to use that.(2) The callers now pass
MSG_MOREunconditionally. Previously, they were passing in MSG_MORE|MSG_SENDPAGE_NOTLAST and then degrading that to justMSG_MOREon the last call to ->sendpage().(3) Make
ceph_tcp_sendpage()a wrapper aroundsendmsg()rather thansendpage(), settingMSG_SPLICE_PAGESifsendpage_ok()returns true on the page.In
messenger_v2:(4) Bring back
do_try_sendpage()and make the callers use that.(5) Make
do_try_sendpage()usesendmsg()for both cases and setMSG_SPLICE_PAGESifsendpage_ok()is set.Fixes 40a8c17aa770 ("ceph: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage")Fixes fa094ccae1e7 ("ceph: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage()")Reported-by Ilya Dryomov <[email protected]>Link https://lore.kernel.org/r/CAOi1vP9vjLfk3W+AJFeexC93jqPaPUn2dD_4NrzxwoZTbYfOnw@mail.gmail.com/ open hereLink https://lore.kernel.org/r/CAOi1vP_Bn918j24S94MuGyn+Gxk212btw7yWeDrRcW1U8pc_BA@mail.gmail.com/ open hereSigned-off-by David Howells <[email protected]>cc Ilya Dryomov <[email protected]>cc Xiubo Li <[email protected]>cc Jeff Layton <[email protected]>cc "David S. Miller" <[email protected]>cc Eric Dumazet <[email protected]>cc Jakub Kicinski <[email protected]>cc Paolo Abeni <[email protected]>cc Jens Axboe <[email protected]>cc Matthew Wilcox <[email protected]>---
Notes:
ver #2)
- Removed mention ofMSG_SENDPAGE_NOTLASTin comments.
- Changed some refs to sendpage toMSG_SPLICE_PAGESin comments.
- Initmsg_iterinceph_tcp_sendpage().
- Move setting ofMSG_SPLICE_PAGESindo_try_sendpage()next to comment and adjust how it is cleared.net/ceph/messenger_v1.c | 58 ++++++++++++++++++++----------- net/ceph/messenger_v2.c | 88 ++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 107 insertions(+), 39 deletions(-)net/ceph/messenger_v1.c +15
-0@@ -74,6 +74,39 @@ static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov,return r; } +/* +* @more: MSG_MORE or 0. +*/ +static int ceph_tcp_sendpage (struct socket *sock,struct page *page, +int offset,size_t size,int more) +{ +struct msghdr msg = { + .msg_flags =MSG_DONTWAIT |MSG_NOSIGNAL | more, + }; +struct bio_vec bvec; +int ret; + +/* +* MSG_SPLICE_PAGES cannot properly handle pages with page_count == 0, +* we need to fall back to sendmsg if that's the case.
net/ceph/messenger_v1.c static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov, · line 101
Quoted 10 lines
net/ceph/messenger_v1.c +10
-0+* +* Same goes for slab pages: skb_can_coalesce() allows +* coalescing neighboring slab objects into a single frag which +* triggers one of hardened usercopy checks. +*/ +if (sendpage_ok (page)) + msg.msg_flags |=MSG_SPLICE_PAGES ; + +bvec_set_page (&bvec, page, size, offset); +iov_iter_bvec (&msg.msg_iter,ITER_DEST , &bvec,1 , size);
Hi David,
Shouldn't this be ITER_SOURCE?
Thanks,
Ilya