Core block IO bits for 4.2
Hi Linus,
This is the core block pull request for 4.2. Nothing really major in here, mostly a collection of smaller optimizations and cleanups, mixed with various fixes. In more detail, this pull request contains:
- Addition of policy specific data to blkcg for block cgroups. From Arianna Avanzini.
- Various cleanups around command types from Christoph.
- Cleanup of the suspend block I/O path from Christoph.
- Plugging updates from Shaohua and Jeff Moyer, for blk-mq.
- Eliminating atomic inc/dec of both remaining IO count and reference count in a bio. From me.
- Fixes for SG gap and chunk size support for data-less (discards) IO, so we can merge these better. From me.
- Small restructuring of blk-mq shared tag support, freeing drivers from iterating hardware queues. From Keith Busch.
- A few cfq-iosched tweaks, from Tahsin Erdogan and me. Makes the IOPS mode the default for non-rotational storage.
Please pull!
git://git.kernel.dk/linux-block.git for-4.2/core
----------------------------------------------------------------
Arianna Avanzini (1):
block, cgroup: implement policy-specific per-blkcg data
Christoph Hellwig (11):
block: rename REQ_TYPE_SPECIAL to REQ_TYPE_DRV_PRIV
block: move REQ_TYPE_ATA_TASKFILE and REQ_TYPE_ATA_PC to ide.h
block: move REQ_TYPE_SENSE to the ide driver
block: remove REQ_TYPE_PM_SHUTDOWN
block: move PM request support to IDE
nbd: stop using req->cmd
block: use an atomic_t for mq_freeze_depth
block: remove BIO_EOPNOTSUPP
block: remove unused BIO_RW_BLOCK and BIO_EOF flags
suspend: simplify block I/O handling
block, dm: don't copy bios for request clones
Jarod Wilson (1):
block: export blkdev_reread_part() and __blkdev_reread_part()
Jeff Moyer (1):
blk-mq: fix plugging in blk_sq_make_request
Jens Axboe (10):
bio: skip atomic inc/dec of ->bi_remaining for non-chains
bio: skip atomic inc/dec of ->bi_cnt for most use cases
block: collapse bio bit space
block: only honor SG gap prevention for merges that contain data
block: don't honor chunk sizes for data-less IO
block: add blk_set_queue_dying() to blkdev.h
cfq-iosched: fix the setting of IOPS mode on SSDs
cfq-iosched: move group scheduling functions under ifdef
cfq-iosched: fix sysfs oops when attempting to read unconfigured weights
cfq-iosched: fix other locations where blkcg_to_cfqgd() can return NULL
Julia Lawall (1):
block: fix returnvar.cocci warnings
Keith Busch (1):
blk-mq: Shared tag enhancements
Mike Snitzer (1):
block: remove management of bi_remaining when restoring original bi_end_io
Ming Lei (1):
block: replace trylock with mutex_lock in blkdev_reread_part()
Shaohua Li (5):
blk: clean up plug
sched: always use blk_schedule_flush_plug in io_schedule_out
blk-mq: avoid re-initialize request which is failed in direct dispatch
blk-mq: do limited block plug for multiple queue case
blk-mq: make plug work for mutiple disks and queues
Steven Rostedt (1):
blktrace: Add blktrace.c to BLOCK LAYER in MAINTAINERS file
Tahsin Erdogan (1):
block: Make CFQ default to IOPS mode on SSDs
MAINTAINERS | 1 + block/bio-integrity.c | 4 +- block/bio.c | 77 +++++++++++-------- block/blk-cgroup.c | 92 ++++++++++++++++++++--- block/blk-cgroup.h | 40 ++++++++-- block/blk-core.c | 136 +++++++++------------------------- block/blk-exec.c | 10 --- block/blk-merge.c | 3 +- block/blk-mq-tag.c | 38 ++++++++++ block/blk-mq-tag.h | 1 + block/blk-mq.c | 160 +++++++++++++++++++++++++--------------- block/blk.h | 5 +- block/bounce.c | 3 - block/cfq-iosched.c | 125 +++++++++++++++++++++++++------ block/elevator.c | 2 + block/ioctl.c | 37 ++++++++-- drivers/block/nbd.c | 50 ++++++------- drivers/block/paride/pd.c | 4 +- drivers/block/sx8.c | 4 +- drivers/block/virtio_blk.c | 6 +- drivers/ide/ide-atapi.c | 10 +-- drivers/ide/ide-cd.c | 10 +-- drivers/ide/ide-cd_ioctl.c | 2 +- drivers/ide/ide-devsets.c | 2 +- drivers/ide/ide-eh.c | 4 +- drivers/ide/ide-floppy.c | 8 +- drivers/ide/ide-io.c | 12 +-- drivers/ide/ide-ioctls.c | 2 +- drivers/ide/ide-park.c | 4 +- drivers/ide/ide-pm.c | 56 ++++++++++---- drivers/ide/ide-tape.c | 6 +- drivers/ide/ide-taskfile.c | 2 +- drivers/md/bcache/io.c | 2 +- drivers/md/bcache/request.c | 2 +- drivers/md/dm-cache-target.c | 6 -- drivers/md/dm-raid1.c | 2 - drivers/md/dm-snap.c | 1 - drivers/md/dm-table.c | 25 ++++--- drivers/md/dm-thin.c | 9 +-- drivers/md/dm-verity.c | 2 +- drivers/md/dm.c | 171 ++++++++++--------------------------------- drivers/md/dm.h | 5 +- fs/btrfs/disk-io.c | 13 +--- fs/btrfs/extent_io.c | 2 - fs/btrfs/volumes.c | 18 ++--- fs/btrfs/volumes.h | 2 - fs/buffer.c | 13 +--- fs/ext4/page-io.c | 1 - fs/nilfs2/segbuf.c | 12 --- fs/xfs/xfs_aops.c | 1 - include/linux/bio.h | 17 ++++- include/linux/blk-mq.h | 4 + include/linux/blk_types.h | 25 ++++--- include/linux/blkdev.h | 45 ++---------- include/linux/elevator.h | 2 + include/linux/fs.h | 3 + include/linux/ide.h | 27 +++++++ include/linux/swap.h | 1 - include/uapi/linux/nbd.h | 2 - kernel/power/Makefile | 3 +- kernel/power/block_io.c | 103 -------------------------- kernel/power/power.h | 9 --- kernel/power/swap.c | 159 ++++++++++++++++++++++++++++++---------- kernel/sched/core.c | 5 +- mm/page_io.c | 2 +- 65 files changed, 853 insertions(+), 757 deletions(-) delete mode 100644 kernel/power/block_io.c
Signature · 1 line
Jens Axboe
Hi Jens,
Quoted 1 line · Jens Axboe
block, dm: don't copy bios for request clones
this change should not be pushed to mainline yet.
Firstly, Christoph has a newer version of the patch that fixes silent data corruption problem:
https://www.redhat.com/archives/dm-devel/2015-May/msg00229.html
And the new version still depends on LLDDs to always complete requests to the end when error happens, while block API doesn't enforce such a requirement. If the assumption is ever broken, the inconsistency between request and bio (e.g. rq->__sector and rq->bio) will cause silent data corruption:
https://www.redhat.com/archives/dm-devel/2015-June/msg00022.html
Signature · 1 line
Jun'ichi Nomura, NEC Corporation
Quoted 7 lines · Junichi Nomura
Hi Jens,
Quoted 1 line · Jens Axboe
block, dm: don't copy bios for request clones
this change should not be pushed to mainline yet.
Hmm. I merged all of Jens' pull requests today, so it's now in my tree. What do?
Linus
Quoted 7 lines · Linus Torvalds
Quoted 4 lines · On Jun 25, 2015 23:17, "Junichi Nomura" <[email protected]…
Quoted 1 line · Jens Axboe
block, dm: don't copy bios for request clones
this change should not be pushed to mainline yet.
Hmm. I merged all of Jens' pull requests today, so it's now in my tree. What do?
Then I think following 3 commits should be reverted.
2d76fff dm: cleanup methods that requeue requests
cbc4e3c dm: do not allocate any mempools for blk-mq request-based DM 5f1b670 block, dm: don't copy bios for request clones
Reverting 5f1b670 involves conflicts in drvers/md/dm.c for these 2 hunks due to diff contexts difference. It's ok to revert those changes ignoring the contexts.
@@ -1089,8 +1038,6 @@ static void free_rq_clone(struct request *clone, bool must_be_mapped)
WARN_ON_ONCE(must_be_mapped && !clone->q);
- blk_rq_unprep_clone(clone);
-
if (md->type == DM_TYPE_MQ_REQUEST_BASED)
/* stacked on blk-mq queue(s) */
tio->ti->type->release_clone_rq(clone); @@ -1973,11 +1889,7 @@ static int map_request(struct dm_rq_target_io *tio, struct request *rq,
}
if (IS_ERR(clone))
return DM_MAPIO_REQUEUE;
- if (setup_clone(clone, rq, tio, GFP_ATOMIC)) {
- /* -ENOMEM */
- ti->type->release_clone_rq(clone);
- return DM_MAPIO_REQUEUE;
- }
+ setup_clone(clone, rq, tio);
}
switch (r) {Signature · 1 line
Jun'ichi Nomura, NEC Corporation
[git pull] device mapper fixes for 4.2-rc1
Quoted 11 lines · Junichi Nomura
Quoted 7 lines · Linus Torvalds
Quoted 4 lines · On Jun 25, 2015 23:17, "Junichi Nomura" <[email protected]…
Quoted 1 line · Jens Axboe
block, dm: don't copy bios for request clones
this change should not be pushed to mainline yet.
Hmm. I merged all of Jens' pull requests today, so it's now in my tree. What do?
Then I think following 3 commits should be reverted.
Quoted 7 lines
2d76fff dm: cleanup methods that requeue requests
cbc4e3c dm: do not allocate any mempools for blk-mq request-based DM 5f1b670 block, dm: don't copy bios for request clonesReverting 5f1b670 involves conflicts in
drvers/md/dm.cfor these 2 hunks due to diff contexts difference. It's ok to revert those changes ignoring the contexts.
That is more than I'd care to put on Linus at this point. I've prepared a DM pull request that fixes things up. Apologies for not pressing this request-based DM partial completion issue further, it was an oversight on my part. We'll have to get it fixed up properly and revisit for a future release.
Linus,
The following changes since commit e262f34741522e0d821642e5449c6eeb512723fc:
dm stats: add support for request-based DM devices (2015-06-17 12:40:41 -0400)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm tags/dm-4.2-fixes
for you to fetch changes up to b5451e456840af027b794afc2c7c84c2a17f569b:
dm cache policy smq: fix "default" version to be 1.4.0 (2015-06-26 10:14:28 -0400)
----------------------------------------------------------------
- Revert block and DM core changes that removed request-based DM's ability to handle partial request completions -- otherwise with the current SCSI LLDs these changes could lead to silent data corruption.
- Fix two DM version bumps that were missing from the initial 4.2 DM pull request (enabled userspace lvm2 to know certain changes have been made).
----------------------------------------------------------------
Mike Snitzer (4):
Revert "dm: do not allocate any mempools for blk-mq request-based DM" Revert "block, dm: don't copy bios for request clones"
dm: bump the ioctl version to 4.32.0
dm cache policy smq: fix "default" version to be 1.4.0
block/blk-core.c | 94 +++++++++++++++-- drivers/md/dm-cache-policy-smq.c | 2 +- drivers/md/dm-table.c | 29 ++---- drivers/md/dm.c | 220 +++++++++++++++++++++++++++------------ drivers/md/dm.h | 5 +- include/linux/blk_types.h | 2 - include/linux/blkdev.h | 6 +- include/uapi/linux/dm-ioctl.h | 4 +- 8 files changed, 256 insertions(+), 106 deletions(-)
Quoted 9 lines · Junichi Nomura
Hi Jens,
Quoted 1 line · Jens Axboe
block, dm: don't copy bios for request clones
this change should not be pushed to mainline yet.
Firstly, Christoph has a newer version of the patch that fixes silent data corruption problem:
Quoted 8 lines
https://www.redhat.com/archives/dm-devel/2015-May/msg00229.htmlAnd the new version still depends on LLDDs to always complete requests to the end when error happens, while block API doesn't enforce such a requirement. If the assumption is ever broken, the inconsistency between request and bio (e.g.
rq->__sectorandrq->bio) will cause silent data corruption:https://www.redhat.com/archives/dm-devel/2015-June/msg00022.html
Why was none of that sent to the block tree?
In any case, Linus, pick up Mike's pull request to fix this up.
Signature · 1 line
Jens Axboe
Quoted 3 lines · Jens Axboe
Why was none of that sent to the block tree?
In any case, Linus, pick up Mike's pull request to fix this up.
I didn't manage to driver the discussion forward, and hadn't really grasped it was in the block tree. For some reason I assumed it only was in Mikes dm WIP tree.