Skip to content

RFC ceph, rbd: Collapse all the I/O types down to something iov_iter-based

David Howells ceph-devel linux-block linux-fsdevel lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest18 patches 2 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/18iov_iter: Add function to see if buffer is all zerosinclude/linux/uio.h lib/iov_iter.c+23 -00 replies
2/18ceph: Rename alignment to offsetfs/ceph/addr.c include/linux/ceph/messenger.h include/linux/ceph/osd_client.h +2 more+26 -260 replies
3/18ceph: Add a new data container type, ceph_databufinclude/linux/ceph/databuf.h include/linux/ceph/messenger.h include/linux/ceph/osd_client.h +3 more+251 -30 replies
4/18ceph: Convert ceph_mds_request::r_pagelist to a databuffs/ceph/acl.c fs/ceph/file.c fs/ceph/inode.c +4 more+96 -1220 replies
5/18rbd: Use ceph_databuf for rbd_obj_read_sync()drivers/block/rbd.c+20 -250 replies
6/18ceph: Change ceph_osdc_call()'s reply to a ceph_databufdrivers/block/rbd.c include/linux/ceph/osd_client.h net/ceph/cls_lock_client.c +1 more+158 -850 replies
7/18ceph: Unexport osd_req_op_cls_request_data_pages()include/linux/ceph/osd_client.h net/ceph/osd_client.c+1 -70 replies
8/18ceph: Remove osd_req_op_cls_response_data_pages()include/linux/ceph/osd_client.h net/ceph/osd_client.c+0 -170 replies
9/18ceph: Convert notify_id_pages to a ceph_databufinclude/linux/ceph/osd_client.h net/ceph/osd_client.c+8 -120 replies
10/18rbd: Switch from using bvec_iter to iov_iterdrivers/block/rbd.c fs/ceph/file.c+127 -4050 replies
11/18ceph: Remove bvec and bio data container typesinclude/linux/ceph/messenger.h include/linux/ceph/osd_client.h net/ceph/messenger.c +1 more+0 -3940 replies
12/18ceph: Convert some page arrays to ceph_databufdrivers/block/rbd.c include/linux/ceph/osd_client.h net/ceph/osd_client.c+55 -340 replies
13/18ceph: Convert users of ceph_pagelist to ceph_databuffs/ceph/locks.c fs/ceph/mds_client.c fs/ceph/super.h +2 more+124 -1180 replies
14/18ceph: Remove ceph_pagelistfs/ceph/locks.c fs/ceph/mds_client.c fs/ceph/xattr.c +7 more+1 -4250 replies
15/18ceph: Convert ceph_osdc_notify() reply to ceph_databufdrivers/block/rbd.c include/linux/ceph/osd_client.h net/ceph/osd_client.c+26 -310 replies
16/18ceph: Remove CEPH_OS_DATA_TYPE_PAGES and its attendant helpersinclude/linux/ceph/osd_client.h net/ceph/osd_client.c+3 -740 replies
17/18ceph: Remove CEPH_MSG_DATA_PAGES and its helpersinclude/linux/ceph/messenger.h net/ceph/messenger.c net/ceph/osd_client.c+5 -1210 replies
18/18ceph: Don't use data_pagesfs/ceph/addr.c fs/ceph/file.c include/linux/ceph/osd_client.h +1 more+23 -441 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 00/18] ceph, rbd: Collapse all the I/O types down to something iov_iter-based

Hi Ilya, Xiubo,

[!] NOTE: This is a preview of a work in progress and doesn't yet fully
compile, let alone actually work!

Here are some patches that (mostly) collapse the different I/O types (PAGES, PAGELIST, BVECS, BIO) down to a single one. I added a new type, ceph_databuf, to make this easier. The page list is attached to that as a bio_vec[] with an iov_iter, but could also be some other type supported by the iov_iter. The iov_iter defines the data or buffer to be used. I have an additional iov_iter type implemented that allows use of a straight folio[] or page[] instead of a bio_vec[] that I can deploy if that proves more useful.

The conversion isn't quite complete:

(1) rbd is done; BVECS and BIO types are replaced with ceph_databuf.

(2) ceph_osd_linger_request::preply_pages needs switching over to a ceph_databuf, but I haven't yet managed to work out how the pages that handle_watch_notify() sticks in there come about.

(3) I haven't altered data transmission in net/ceph/messenger*.c yet. The aim is to reduce it to a single sendmsg() call for each ceph_msg_data struct, using the iov_iter therein.

(4) The data reception routines in net/ceph/messenger*.c also need modifying to pass each ceph_msg_data::iter to recvmsg() in turn.

(5) It might be possible to merge struct ceph_databuf into struct ceph_msg_data and eliminate the former.

(6) fs/ceph/ still needs some work to clean up the use of page arrays.

(7) I would like to change front and middle buffers with a ceph_databuf, vmapping them when we need to access them.

I added a kmap_ceph_databuf_page() macro and used that to get a page and use kmap_local_page() on it to hide the bvec[] inside to make it easier to replace.

Anyway, if anyone has any thoughts...

I've pushed the patches here also:

https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-extract

David

David Howells (18):
iov_iter: Add function to see if buffer is all zeros
ceph: Rename alignment to offset
ceph: Add a new data container type, ceph_databuf
ceph: Convert ceph_mds_request::r_pagelist to a databuf
rbd: Use ceph_databuf for rbd_obj_read_sync()
ceph: Change ceph_osdc_call()'s reply to a ceph_databuf
ceph: Unexport osd_req_op_cls_request_data_pages()
ceph: Remove osd_req_op_cls_response_data_pages()
ceph: Convert notify_id_pages to a ceph_databuf
rbd: Switch from using bvec_iter to iov_iter
ceph: Remove bvec and bio data container types
ceph: Convert some page arrays to ceph_databuf
ceph: Convert users of ceph_pagelist to ceph_databuf
ceph: Remove ceph_pagelist
ceph: Convert ceph_osdc_notify() reply to ceph_databuf
ceph: Remove CEPH_OS_DATA_TYPE_PAGES and its attendant helpers
ceph: Remove CEPH_MSG_DATA_PAGES and its helpers
ceph: Don't use data_pages

 drivers/block/rbd.c             | 645 ++++++++++----------------------
 fs/ceph/acl.c                   |  39 +-
 fs/ceph/addr.c                  |  18 +-
 fs/ceph/file.c                  | 157 ++++----
 fs/ceph/inode.c                 |  85 ++---
 fs/ceph/locks.c                 |  23 +-
 fs/ceph/mds_client.c            | 134 ++++---
 fs/ceph/mds_client.h            |   2 +-
 fs/ceph/super.h                 |   8 +-
 fs/ceph/xattr.c                 |  68 ++--
 include/linux/ceph/databuf.h    |  65 ++++
 include/linux/ceph/messenger.h  | 141 +------
 include/linux/ceph/osd_client.h |  97 ++---
 include/linux/ceph/pagelist.h   |  72 ----
 include/linux/uio.h             |   1 +
 lib/iov_iter.c                  |  22 ++
 net/ceph/Makefile               |   5 +-
 net/ceph/cls_lock_client.c      |  40 +-
 net/ceph/databuf.c              | 149 ++++++++
 net/ceph/messenger.c            | 376 +------------------
 net/ceph/osd_client.c           | 430 +++++++--------------
 net/ceph/pagelist.c             | 171 ---------
 22 files changed, 876 insertions(+), 1872 deletions(-)
 create mode 100644 include/linux/ceph/databuf.h
 delete mode 100644 include/linux/ceph/pagelist.h
 create mode 100644 net/ceph/databuf.c
 delete mode 100644 net/ceph/pagelist.c