Skip to contentSkip to conversation

PATCH In review Jens Axboe 5 participants 27 messages Oct 24, 2013 – Feb 5, 2015

Subsystems (6)
VFS SCSI Block layer Library code +2 more subsystems
lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

Hi,

I've been working (on and off) on blk-mq for a while now, and I think it's finally time to polish it off and get it posted for review with inclusion for 3.13. It's fairly well contained and these core bits should not impact any drivers. There's a series of prep patches in front of the meat patch.

You can also find this series here:

git://git.kernel.dk/linux-block.git blk-mq/core

and there's a separate repo with some driver updates here:

git://git.kernel.dk/linux-block.git blk-mq/drivers

which is layered on top of the core bits.

See patch 10 for a description of blk-mq itself and the reasoning behind this new queueing model. There's also been a paper published on results run on the code last year. Note that this is somewhat out of date as the code has moved along since, most notably it should be much faster now. The paper can be found here:

http://kernel.dk/systor13-final18.pdf

and was presented at Systor 2013 earlier this year.

author maintainer
#1 lore raw

PATCH 1/11 Export __smp_call_function_single()

The blk-mq core and the blk-mq null driver uses it.

---
 kernel/smp.c | 1 +
 1 file changed, 1 insertion(+)
kernel/smp.c +1 -0
index 0564571..9de461e 100644
@@ -340,6 +340,7 @@ void __smp_call_function_single(int cpu, struct call_single_data *csd,
 	}
 	put_cpu();
 }
+EXPORT_SYMBOL(__smp_call_function_single);
 
 /**
  * smp_call_function_many(): Run a function on a set of other CPUs.
Signature · 1 line
1.8.1.2
#2 lore raw

Re: [PATCH 01/11] Export __smp_call_function_single()

Quoted 1 line · Jens Axboe

The blk-mq core and the blk-mq null driver uses it.

Can't say I really like exporting it as all users will need a major ifdef mess. If at all we'd expose the higher level add to llist and ipi abstraction that both blk-mq and the null_blk driver use.

For now I'd just suggest removing the NULL_IRQ_SOFTIRQ mode from null_blk and decouple that from blk-mq itself. I'll try to cook up such an abstraction in the meantime.

author maintainer
#3 lore raw

Re: [PATCH 01/11] Export __smp_call_function_single()

Quoted 10 lines · Christoph Hellwig
Quoted 1 line · Jens Axboe

The blk-mq core and the blk-mq null driver uses it.

Can't say I really like exporting it as all users will need a major ifdef mess. If at all we'd expose the higher level add to llist and ipi abstraction that both blk-mq and the null_blk driver use.

For now I'd just suggest removing the NULL_IRQ_SOFTIRQ mode from null_blk and decouple that from blk-mq itself. I'll try to cook up such an abstraction in the meantime.

Agree, lets get a proper interface exported for this from kernel/smp.c instead. I've killed this patch and the softirq mode in null_blk for now.

Signature · 1 line
Jens Axboe
Reviewed-by
#4 lore raw

Re: [PATCH 01/11] Export __smp_call_function_single()

Quoted 8 lines · Jens Axboe
Quoted 7 lines · Christoph Hellwig
Quoted 1 line · Jens Axboe

The blk-mq core and the blk-mq null driver uses it.

Can't say I really like exporting it as all users will need a major ifdef mess. If at all we'd expose the higher level add to llist and ipi abstraction that both blk-mq and the null_blk driver use.

Quoted 7 lines
Quoted 3 lines

For now I'd just suggest removing the NULL_IRQ_SOFTIRQ mode from null_blk and decouple that from blk-mq itself. I'll try to cook up such an abstraction in the meantime.

Agree, lets get a proper interface exported for this from kernel/smp.c instead. I've killed this patch and the softirq mode in null_blk for now.

Turns out the proper interface is __smp_call_function_single as long as we make sure it's always available. I've got a series that allows us to use it directly without the llist wrapping, but it probably won't be ready for the next merge window.

So I'll take my earlier objection back, instead please add a:

#5 lore raw

Re: [PATCH 01/11] Export __smp_call_function_single()

In reply to Jens Axboe
Quoted 13 lines · Jens Axboe

The blk-mq core and the blk-mq null driver uses it.

---
 kernel/smp.c | 1 +
 1 file changed, 1 insertion(+)
kernel/smp.c +0 -0
index 0564571..9de461e 100644

kernel/smp.c void __smp_call_function_single(int cpu, struct call_single_data *csd, · line 343

Quoted 4 lines
kernel/smp.c +1 -0
@@ -340,6 +340,7 @@ void __smp_call_function_single(int cpu, struct call_single_data *csd,
 	}
 	put_cpu();
 }
+EXPORT_SYMBOL(__smp_call_function_single);

I agree with exporting it to in-kernel users, but I'd prefer a _GPL export for newly exported symbols that are for in-kernel use.

Thanks,

Ingo

author maintainer
#6 lore raw

Re: [PATCH 01/11] Export __smp_call_function_single()

Quoted 11 lines · Ingo Molnar
Quoted 8 lines · Jens Axboe

The blk-mq core and the blk-mq null driver uses it.

---
 kernel/smp.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 12 lines

kernel/smp.c void __smp_call_function_single(int cpu, struct call_single_data *csd, · line 343

Quoted 9 lines
kernel/smp.c +1 -0
index 0564571..9de461e 100644
@@ -340,6 +340,7 @@ void __smp_call_function_single(int cpu, struct call_single_data *csd,
 	}
 	put_cpu();
 }
+EXPORT_SYMBOL(__smp_call_function_single);

I agree with exporting it to in-kernel users, but I'd prefer a _GPL export for newly exported symbols that are for in-kernel use.

Agree, it should be _GPL. I will update the patch. Can I add your acked-by?

Signature · 1 line
Jens Axboe
Acked-by
#7 lore raw

Re: [PATCH 01/11] Export __smp_call_function_single()

Quoted 25 lines · Jens Axboe
Quoted 11 lines · Ingo Molnar
Quoted 8 lines · Jens Axboe

The blk-mq core and the blk-mq null driver uses it.

---
 kernel/smp.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 12 lines

kernel/smp.c void __smp_call_function_single(int cpu, struct call_single_data *csd, · line 343

Quoted 9 lines
kernel/smp.c +1 -0
index 0564571..9de461e 100644
@@ -340,6 +340,7 @@ void __smp_call_function_single(int cpu, struct call_single_data *csd,
 	}
 	put_cpu();
 }
+EXPORT_SYMBOL(__smp_call_function_single);

I agree with exporting it to in-kernel users, but I'd prefer a _GPL export for newly exported symbols that are for in-kernel use.

Quoted 2 lines

Agree, it should be _GPL. I will update the patch. Can I add your acked-by?

Sure:

Thanks,

Ingo

author maintainer
#8 lore raw

PATCH 2/11 smp: don't warn about csd->flags having CSD_FLAG_LOCK cleared for !wait

In reply to Jens Axboe

blk-mq reuses the request potentially immediately, since the most cache hot is always given out first. This means that rq->csd could be reused between csd->func() being called and csd_unlock() being called. This isn't a problem, since we never use wait == 1 for the smp call function. Add CSD_FLAG_WAIT to be able to tell the difference, retaining the warning for other cases.

---
 kernel/smp.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/smp.c +5 -1
index 9de461e..069d20b 100644
@@ -18,6 +18,7 @@
 #ifdef CONFIG_USE_GENERIC_SMP_HELPERS
 enum {
 	CSD_FLAG_LOCK		= 0x01,
+	CSD_FLAG_WAIT		= 0x02,
 };
 
 struct call_function_data {
@@ -124,7 +125,7 @@ static void csd_lock(struct call_single_data *csd)
 
 static void csd_unlock(struct call_single_data *csd)
 {
-	WARN_ON(!(csd->flags & CSD_FLAG_LOCK));
+	WARN_ON((csd->flags & CSD_FLAG_WAIT) && !(csd->flags & CSD_FLAG_LOCK));
 
 	/*
 	 * ensure we're all done before releasing data:
@@ -146,6 +147,9 @@ void generic_exec_single(int cpu, struct call_single_data *csd, int wait)
 	unsigned long flags;
 	int ipi;
 
+	if (wait)
+		csd->flags |= CSD_FLAG_WAIT;
+
 	raw_spin_lock_irqsave(&dst->lock, flags);
 	ipi = list_empty(&dst->list);
 	list_add_tail(&csd->list, &dst->list);
Signature · 1 line
1.8.1.2
#9 lore raw

Re: [PATCH 02/11] smp: don't warn about csd->flags having CSD_FLAG_LOCK cleared for !wait

Quoted 9 lines · Jens Axboe

blk-mq reuses the request potentially immediately, since the most cache hot is always given out first. This means that rq->csd could be reused between csd->func() being called and csd_unlock() being called. This isn't a problem, since we never use wait == 1 for the smp call function. Add CSD_FLAG_WAIT to be able to tell the difference, retaining the warning for other cases.

Looks fine, but I'd be tempted to just delete the WARN_ON and add a comment similar to the commit message there.

author maintainer
#10 lore raw

PATCH 3/11 block: make rq->cmd_flags be 64-bit

In reply to Jens Axboe

We have officially run out of flags in a 32-bit space. Extend it to 64-bit even on 32-bit archs.

---
 block/blk-core.c          |  4 +--
 drivers/block/floppy.c    |  4 +--
 drivers/scsi/sd.c         |  2 +-
 include/linux/blk_types.h | 66 +++++++++++++++++++++++------------------------
 include/linux/blkdev.h    |  4 +--
 5 files changed, 40 insertions(+), 40 deletions(-)
block/blk-core.c +2 -2
index 0a00e4e..213e9f0 100644
@@ -174,9 +174,9 @@ void blk_dump_rq_flags(struct request *rq, char *msg)
 {
 	int bit;
 
-	printk(KERN_INFO "%s: dev %s: type=%x, flags=%x\n", msg,
+	printk(KERN_INFO "%s: dev %s: type=%x, flags=%llx\n", msg,
 		rq->rq_disk ? rq->rq_disk->disk_name : "?", rq->cmd_type,
-		rq->cmd_flags);
+		(unsigned long long) rq->cmd_flags);
 
 	printk(KERN_INFO "  sector %llu, nr/cnr %u/%u\n",
 	       (unsigned long long)blk_rq_pos(rq),
drivers/block/floppy.c +2 -2
index 04ceb7e..000abe2 100644
@@ -2886,9 +2886,9 @@ static void do_fd_request(struct request_queue *q)
 		return;
 
 	if (WARN(atomic_read(&usage_count) == 0,
-		 "warning: usage count=0, current_req=%p sect=%ld type=%x flags=%x\n",
+		 "warning: usage count=0, current_req=%p sect=%ld type=%x flags=%llx\n",
 		 current_req, (long)blk_rq_pos(current_req), current_req->cmd_type,
-		 current_req->cmd_flags))
+		 (unsigned long long) current_req->cmd_flags))
 		return;
 
 	if (test_and_set_bit(0, &fdc_busy)) {
drivers/scsi/sd.c +1 -1
index e62d17d..dae6133 100644
@@ -1002,7 +1002,7 @@ static int sd_prep_fn(struct request_queue *q, struct request *rq)
 		SCpnt->cmnd[0] = READ_6;
 		SCpnt->sc_data_direction = DMA_FROM_DEVICE;
 	} else {
-		scmd_printk(KERN_ERR, SCpnt, "Unknown command %x\n", rq->cmd_flags);
+		scmd_printk(KERN_ERR, SCpnt, "Unknown command %llx\n", (unsigned long long) rq->cmd_flags);
 		goto out;
 	}
 
include/linux/blk_types.h +33 -33

77 lines not on this page · raw message

include/linux/blkdev.h +2 -2
index 0e6f765..f5c7596 100644
@@ -99,7 +99,7 @@ struct request {
 
 	struct request_queue *q;
 
-	unsigned int cmd_flags;
+	u64 cmd_flags;
 	enum rq_cmd_type_bits cmd_type;
 	unsigned long atomic_flags;
 
@@ -570,7 +570,7 @@ static inline void queue_flag_clear(unsigned int flag, struct request_queue *q)
 
 #define list_entry_rq(ptr)	list_entry((ptr), struct request, queuelist)
 
-#define rq_data_dir(rq)		((rq)->cmd_flags & 1)
+#define rq_data_dir(rq)		(((rq)->cmd_flags & 1) != 0)
 
 static inline unsigned int blk_queue_cluster(struct request_queue *q)
 {
Signature · 1 line
1.8.1.2
author maintainer
#12 lore raw

PATCH 4/11 block: remove request ref_count

In reply to Jens Axboe

From: Christoph Hellwig <[email protected]>

This reference count has been around since before git history, but the only place where it's used is in blk_execute_rq, and ther it is entirely useless as it is incremented before submitting the request and decremented in the end_io handler before waking up the submitter thread.

---
 block/blk-core.c       | 3 ---
 block/blk-exec.c       | 7 -------
 include/linux/blkdev.h | 2 --
 3 files changed, 12 deletions(-)
block/blk-core.c +0 -3
index 213e9f0..18faa7e 100644
@@ -145,7 +145,6 @@ void blk_rq_init(struct request_queue *q, struct request *rq)
 	rq->cmd = rq->__cmd;
 	rq->cmd_len = BLK_MAX_CDB;
 	rq->tag = -1;
-	rq->ref_count = 1;
 	rq->start_time = jiffies;
 	set_start_time_ns(rq);
 	rq->part = NULL;
@@ -1272,8 +1271,6 @@ void __blk_put_request(struct request_queue *q, struct request *req)
 {
 	if (unlikely(!q))
 		return;
-	if (unlikely(--req->ref_count))
-		return;
 
 	blk_pm_put_request(req);
 
block/blk-exec.c +0 -7
index ae4f27d..6b18d82 100644
@@ -24,7 +24,6 @@ static void blk_end_sync_rq(struct request *rq, int error)
 	struct completion *waiting = rq->end_io_data;
 
 	rq->end_io_data = NULL;
-	__blk_put_request(rq->q, rq);
 
 	/*
 	 * complete last, if this is a stack request the process (and thus
@@ -103,12 +102,6 @@ int blk_execute_rq(struct request_queue *q, struct gendisk *bd_disk,
 	int err = 0;
 	unsigned long hang_check;
 
-	/*
-	 * we need an extra reference to the request, so we can look at
-	 * it after io completion
-	 */
-	rq->ref_count++;
-
 	if (!rq->sense) {
 		memset(sense, 0, sizeof(sense));
 		rq->sense = sense;
include/linux/blkdev.h +0 -2
index f5c7596..0a8da96 100644
@@ -160,8 +160,6 @@ struct request {
 
 	unsigned short ioprio;
 
-	int ref_count;
-
 	void *special;		/* opaque pointer available for LLD use */
 	char *buffer;		/* kaddr of the current segment if available */
 
Signature · 1 line
1.8.1.2
author maintainer
#13 lore raw

PATCH 5/11 direct-io: only inc/dec inode->i_dio_count for file systems

In reply to Jens Axboe

We don't need truncate protection for block devices, so add a flag bypassing this cache line dirtying twice for every IO. This easily contributes to 5-10% of the CPU time on high IOPS O_DIRECT testing.

Cc Christoph Hellwig <[email protected]>
Cc Al Viro <[email protected]>
---
 fs/block_dev.c     |  3 ++-
 fs/direct-io.c     | 14 +++++++++++++-
 fs/inode.c         |  7 ++++++-
 include/linux/fs.h |  4 ++++
 4 files changed, 25 insertions(+), 3 deletions(-)
fs/block_dev.c +2 -1
index 1e86823..e20b7c1 100644
@@ -172,7 +172,8 @@ blkdev_direct_IO(int rw, struct kiocb *iocb, const struct iovec *iov,
 	struct inode *inode = file->f_mapping->host;
 
 	return __blockdev_direct_IO(rw, iocb, inode, I_BDEV(inode), iov, offset,
-				    nr_segs, blkdev_get_block, NULL, NULL, 0);
+				    nr_segs, blkdev_get_block, NULL, NULL,
+				    DIO_IGNORE_TRUNCATE);
 }
 
 int __sync_blockdev(struct block_device *bdev, int wait)
fs/direct-io.c +13 -1
index 0e04142..3c96479 100644
@@ -265,7 +265,11 @@ static ssize_t dio_complete(struct dio *dio, loff_t offset, ssize_t ret,
 	if (dio->end_io && dio->result)
 		dio->end_io(dio->iocb, offset, transferred, dio->private);
 
-	inode_dio_done(dio->inode);
+	if (dio->flags & DIO_IGNORE_TRUNCATE)
+		__inode_dio_done(dio->inode);
+	else
+		inode_dio_done(dio->inode);
+
 	if (is_async) {
 		if (dio->rw & WRITE) {
 			int err;
@@ -1194,6 +1198,14 @@ do_blockdev_direct_IO(int rw, struct kiocb *iocb, struct inode *inode,
 	}
 
 	/*
+	 * Will be decremented at I/O completion time. For a block device
+	 * we don't need to protect against truncate, so don't increment
+	 * the inode direct IO count.
+	 */
+	if (!(dio->flags & DIO_IGNORE_TRUNCATE))
+		atomic_inc(&inode->i_dio_count);
+
+	/*
 	 * For file extending writes updating i_size before data
 	 * writeouts complete can expose uninitialized blocks. So
 	 * even for AIO, we need to wait for i/o to complete before
fs/inode.c +6 -1
index b33ba8e..22319db 100644
@@ -1854,6 +1854,11 @@ void inode_dio_wait(struct inode *inode)
 }
 EXPORT_SYMBOL(inode_dio_wait);
 
+void __inode_dio_done(struct inode *inode)
+{
+	wake_up_bit(&inode->i_state, __I_DIO_WAKEUP);
+}
+
 /*
  * inode_dio_done - signal finish of a direct I/O requests
  * @inode: inode the direct I/O happens on
@@ -1864,6 +1869,6 @@ EXPORT_SYMBOL(inode_dio_wait);
 void inode_dio_done(struct inode *inode)
 {
 	if (atomic_dec_and_test(&inode->i_dio_count))
-		wake_up_bit(&inode->i_state, __I_DIO_WAKEUP);
+		__inode_dio_done(inode);
 }
 EXPORT_SYMBOL(inode_dio_done);
include/linux/fs.h +4 -0
index 3f40547..d90a688 100644
@@ -2468,6 +2468,9 @@ enum {
 
 	/* filesystem does not support filling holes */
 	DIO_SKIP_HOLES	= 0x02,
+
+	/* inode/fs/bdev does not need truncate protection */
+	DIO_IGNORE_TRUNCATE = 0x04,
 };
 
 void dio_end_io(struct bio *bio, int error);
@@ -2488,6 +2491,7 @@ static inline ssize_t blockdev_direct_IO(int rw, struct kiocb *iocb,
 #endif
 
 void inode_dio_wait(struct inode *inode);
+void __inode_dio_done(struct inode *inode);
 void inode_dio_done(struct inode *inode);
 
 extern const struct file_operations generic_ro_fops;
Signature · 1 line
1.8.1.2
author maintainer
#14 lore raw

Re: [PATCH 05/11] direct-io: only inc/dec inode->i_dio_count for file systems

Quoted 3 lines · Jens Axboe

We don't need truncate protection for block devices, so add a flag bypassing this cache line dirtying twice for every IO. This easily contributes to 5-10% of the CPU time on high IOPS O_DIRECT testing.

There's a merge error in this patch, please find the correct one below.

fs/block_dev.c +2 -1
index 1e86823..e20b7c1 100644
@@ -172,7 +172,8 @@ blkdev_direct_IO(int rw, struct kiocb *iocb, const struct iovec *iov,
 	struct inode *inode = file->f_mapping->host;
 
 	return __blockdev_direct_IO(rw, iocb, inode, I_BDEV(inode), iov, offset,
-				    nr_segs, blkdev_get_block, NULL, NULL, 0);
+				    nr_segs, blkdev_get_block, NULL, NULL,
+				    DIO_IGNORE_TRUNCATE);
 }
 
 int __sync_blockdev(struct block_device *bdev, int wait)
fs/direct-io.c +10 -3
index 0e04142..acc1471 100644
@@ -265,7 +265,11 @@ static ssize_t dio_complete(struct dio *dio, loff_t offset, ssize_t ret,
 	if (dio->end_io && dio->result)
 		dio->end_io(dio->iocb, offset, transferred, dio->private);
 
-	inode_dio_done(dio->inode);
+	if (dio->flags & DIO_IGNORE_TRUNCATE)
+		__inode_dio_done(dio->inode);
+	else
+		inode_dio_done(dio->inode);
+
 	if (is_async) {
 		if (dio->rw & WRITE) {
 			int err;
@@ -1223,9 +1227,12 @@ do_blockdev_direct_IO(int rw, struct kiocb *iocb, struct inode *inode,
 	}
 
 	/*
-	 * Will be decremented at I/O completion time.
+	 * Will be decremented at I/O completion time. For a block device
+	 * we don't need to protect against truncate, so don't increment
+	 * the inode direct IO count.
 	 */
-	atomic_inc(&inode->i_dio_count);
+	if (!(dio->flags & DIO_IGNORE_TRUNCATE))
+		atomic_inc(&inode->i_dio_count);
 
 	retval = 0;
 	sdio.blkbits = blkbits;
fs/inode.c +6 -1
index b33ba8e..22319db 100644
@@ -1854,6 +1854,11 @@ void inode_dio_wait(struct inode *inode)
 }
 EXPORT_SYMBOL(inode_dio_wait);
 
+void __inode_dio_done(struct inode *inode)
+{
+	wake_up_bit(&inode->i_state, __I_DIO_WAKEUP);
+}
+
 /*
  * inode_dio_done - signal finish of a direct I/O requests
  * @inode: inode the direct I/O happens on
@@ -1864,6 +1869,6 @@ EXPORT_SYMBOL(inode_dio_wait);
 void inode_dio_done(struct inode *inode)
 {
 	if (atomic_dec_and_test(&inode->i_dio_count))
-		wake_up_bit(&inode->i_state, __I_DIO_WAKEUP);
+		__inode_dio_done(inode);
 }
 EXPORT_SYMBOL(inode_dio_done);
include/linux/fs.h +4 -0
index 3f40547..d90a688 100644
@@ -2468,6 +2468,9 @@ enum {
 
 	/* filesystem does not support filling holes */
 	DIO_SKIP_HOLES	= 0x02,
+
+	/* inode/fs/bdev does not need truncate protection */
+	DIO_IGNORE_TRUNCATE = 0x04,
 };
 
 void dio_end_io(struct bio *bio, int error);
@@ -2488,6 +2491,7 @@ static inline ssize_t blockdev_direct_IO(int rw, struct kiocb *iocb,
 #endif
 
 void inode_dio_wait(struct inode *inode);
+void __inode_dio_done(struct inode *inode);
 void inode_dio_done(struct inode *inode);
 
 extern const struct file_operations generic_ro_fops;
Signature · 1 line
Jens Axboe
#15 lore raw

Re: [PATCH 05/11] direct-io: only inc/dec inode->i_dio_count for file systems

Quoted 6 lines · Jens Axboe
Quoted 3 lines · Jens Axboe

We don't need truncate protection for block devices, so add a flag bypassing this cache line dirtying twice for every IO. This easily contributes to 5-10% of the CPU time on high IOPS O_DIRECT testing.

There's a merge error in this patch, please find the correct one below.

Btw, I think this patch should not go in with this series but rather via the VFS tree as it's not related to the blk-mq infrastructure at all.

author maintainer
#16 lore raw

Re: [PATCH 05/11] direct-io: only inc/dec inode->i_dio_count for file systems

Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Jens Axboe
Quoted 3 lines · Jens Axboe

We don't need truncate protection for block devices, so add a flag bypassing this cache line dirtying twice for every IO. This easily contributes to 5-10% of the CPU time on high IOPS O_DIRECT testing.

There's a merge error in this patch, please find the correct one below.

Btw, I think this patch should not go in with this series but rather via the VFS tree as it's not related to the blk-mq infrastructure at all.

That'd be fine with me, if Al could pick it up if he agrees with it. Feel free to add an acked/review-by :-)

Signature · 1 line
Jens Axboe
#17 lore raw

Re: [PATCH 05/11] direct-io: only inc/dec inode->i_dio_count for file systems

Quoted 7 lines · Jens Axboe
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Jens Axboe
Quoted 3 lines · Jens Axboe

We don't need truncate protection for block devices, so add a flag bypassing this cache line dirtying twice for every IO. This easily contributes to 5-10% of the CPU time on high IOPS O_DIRECT testing.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

There's a merge error in this patch, please find the correct one below.

Btw, I think this patch should not go in with this series but rather via the VFS tree as it's not related to the blk-mq infrastructure at all.

That'd be fine with me, if Al could pick it up if he agrees with it. Feel free to add an acked/review-by :-)

This patch never got picked up. Can it be?

author maintainer
#18 lore raw

PATCH 6/11 percpu_counter: make APIs irq safe

In reply to Jens Axboe

From: Shaohua Li <[email protected]>

In my usage, sometimes the percpu APIs are called with irq locked, sometimes not. lockdep complains there is potential deadlock. Let's always use percpucounter lock in irq safe way. There should be no performance penality, as all those are slow code path.

---
 lib/percpu_counter.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
lib/percpu_counter.c +9 -6
index 93c5d5e..7473ee3 100644
@@ -60,14 +60,15 @@ static inline void debug_percpu_counter_deactivate(struct percpu_counter *fbc)
 void percpu_counter_set(struct percpu_counter *fbc, s64 amount)
 {
 	int cpu;
+	unsigned long flags;
 
-	raw_spin_lock(&fbc->lock);
+	raw_spin_lock_irqsave(&fbc->lock, flags);
 	for_each_possible_cpu(cpu) {
 		s32 *pcount = per_cpu_ptr(fbc->counters, cpu);
 		*pcount = 0;
 	}
 	fbc->count = amount;
-	raw_spin_unlock(&fbc->lock);
+	raw_spin_unlock_irqrestore(&fbc->lock, flags);
 }
 EXPORT_SYMBOL(percpu_counter_set);
 
@@ -78,9 +79,10 @@ void __percpu_counter_add(struct percpu_counter *fbc, s64 amount, s32 batch)
 	preempt_disable();
 	count = __this_cpu_read(*fbc->counters) + amount;
 	if (count >= batch || count <= -batch) {
-		raw_spin_lock(&fbc->lock);
+		unsigned long flags;
+		raw_spin_lock_irqsave(&fbc->lock, flags);
 		fbc->count += count;
-		raw_spin_unlock(&fbc->lock);
+		raw_spin_unlock_irqrestore(&fbc->lock, flags);
 		__this_cpu_write(*fbc->counters, 0);
 	} else {
 		__this_cpu_write(*fbc->counters, count);
@@ -97,14 +99,15 @@ s64 __percpu_counter_sum(struct percpu_counter *fbc)
 {
 	s64 ret;
 	int cpu;
+	unsigned long flags;
 
-	raw_spin_lock(&fbc->lock);
+	raw_spin_lock_irqsave(&fbc->lock, flags);
 	ret = fbc->count;
 	for_each_online_cpu(cpu) {
 		s32 *pcount = per_cpu_ptr(fbc->counters, cpu);
 		ret += *pcount;
 	}
-	raw_spin_unlock(&fbc->lock);
+	raw_spin_unlock_irqrestore(&fbc->lock, flags);
 	return ret;
 }
 EXPORT_SYMBOL(__percpu_counter_sum);
Signature · 1 line
1.8.1.2
author maintainer
#19 lore raw

PATCH 7/11 percpu_ida: make percpu_ida percpu size/batch configurable

In reply to Jens Axboe

From: Shaohua Li <[email protected]>

Make percpu_ida percpu size/batch configurable. The block-mq-tag will use it.

After block-mq uses percpu_ida to manage tags, performance is improved. My test is done in a 2 sockets machine, 12 process cross the 2 sockets. So if there is lock contention or ipi, should be stressed heavily. Testing is done for null-blk.

hw_queue_depth nopatch iops patch iops
64 ~800k/s ~1470k/s
2048 ~4470k/s ~4340k/s

---
 include/linux/percpu_ida.h | 18 +++++++++++++++++-
 lib/percpu_ida.c           | 28 +++++++++++-----------------
 2 files changed, 28 insertions(+), 18 deletions(-)
include/linux/percpu_ida.h +17 -1
index 0b23edb..56c1403 100644
@@ -16,6 +16,8 @@ struct percpu_ida {
 	 * percpu_ida_init()
 	 */
 	unsigned			nr_tags;
+	unsigned			percpu_max_size;
+	unsigned			percpu_batch_size;
 
 	struct percpu_ida_cpu __percpu	*tag_cpu;
 
@@ -51,10 +53,24 @@ struct percpu_ida {
 	} ____cacheline_aligned_in_smp;
 };
 
+/*
+ * Number of tags we move between the percpu freelist and the global freelist at
+ * a time
+ */
+#define IDA_DEFAULT_PCPU_BATCH_MOVE	32U
+/* Max size of percpu freelist, */
+#define IDA_DEFAULT_PCPU_SIZE	((IDA_DEFAULT_PCPU_BATCH_MOVE * 3) / 2)
+
 int percpu_ida_alloc(struct percpu_ida *pool, gfp_t gfp);
 void percpu_ida_free(struct percpu_ida *pool, unsigned tag);
 
 void percpu_ida_destroy(struct percpu_ida *pool);
-int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags);
+int __percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags,
+	unsigned long max_size, unsigned long batch_size);
+static inline int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags)
+{
+	return __percpu_ida_init(pool, nr_tags, IDA_DEFAULT_PCPU_SIZE,
+		IDA_DEFAULT_PCPU_BATCH_MOVE);
+}
 
 #endif /* __PERCPU_IDA_H__ */
lib/percpu_ida.c +11 -17
index bab1ba2..a601d42 100644
@@ -30,15 +30,6 @@
 #include <linux/spinlock.h>
 #include <linux/percpu_ida.h>
 
-/*
- * Number of tags we move between the percpu freelist and the global freelist at
- * a time
- */
-#define IDA_PCPU_BATCH_MOVE	32U
-
-/* Max size of percpu freelist, */
-#define IDA_PCPU_SIZE		((IDA_PCPU_BATCH_MOVE * 3) / 2)
-
 struct percpu_ida_cpu {
 	/*
 	 * Even though this is percpu, we need a lock for tag stealing by remote
@@ -78,7 +69,7 @@ static inline void steal_tags(struct percpu_ida *pool,
 	struct percpu_ida_cpu *remote;
 
 	for (cpus_have_tags = cpumask_weight(&pool->cpus_have_tags);
-	     cpus_have_tags * IDA_PCPU_SIZE > pool->nr_tags / 2;
+	     cpus_have_tags * pool->percpu_max_size > pool->nr_tags / 2;
 	     cpus_have_tags--) {
 		cpu = cpumask_next(cpu, &pool->cpus_have_tags);
 
@@ -123,7 +114,7 @@ static inline void alloc_global_tags(struct percpu_ida *pool,
 {
 	move_tags(tags->freelist, &tags->nr_free,
 		  pool->freelist, &pool->nr_free,
-		  min(pool->nr_free, IDA_PCPU_BATCH_MOVE));
+		  min(pool->nr_free, pool->percpu_batch_size));
 }
 
 static inline unsigned alloc_local_tag(struct percpu_ida *pool,
@@ -245,17 +236,17 @@ void percpu_ida_free(struct percpu_ida *pool, unsigned tag)
 		wake_up(&pool->wait);
 	}
 
-	if (nr_free == IDA_PCPU_SIZE) {
+	if (nr_free == pool->percpu_max_size) {
 		spin_lock(&pool->lock);
 
 		/*
 		 * Global lock held and irqs disabled, don't need percpu
 		 * lock
 		 */
-		if (tags->nr_free == IDA_PCPU_SIZE) {
+		if (tags->nr_free == pool->percpu_max_size) {
 			move_tags(pool->freelist, &pool->nr_free,
 				  tags->freelist, &tags->nr_free,
-				  IDA_PCPU_BATCH_MOVE);
+				  pool->percpu_batch_size);
 
 			wake_up(&pool->wait);
 		}
@@ -292,7 +283,8 @@ EXPORT_SYMBOL_GPL(percpu_ida_destroy);
  * Allocation is percpu, but sharding is limited by nr_tags - for best
  * performance, the workload should not span more cpus than nr_tags / 128.
  */
-int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags)
+int __percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags,
+	unsigned long max_size, unsigned long batch_size)
 {
 	unsigned i, cpu, order;
 
@@ -301,6 +293,8 @@ int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags)
 	init_waitqueue_head(&pool->wait);
 	spin_lock_init(&pool->lock);
 	pool->nr_tags = nr_tags;
+	pool->percpu_max_size = max_size;
+	pool->percpu_batch_size = batch_size;
 
 	/* Guard against overflow */
 	if (nr_tags > (unsigned) INT_MAX + 1) {
@@ -319,7 +313,7 @@ int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags)
 	pool->nr_free = nr_tags;
 
 	pool->tag_cpu = __alloc_percpu(sizeof(struct percpu_ida_cpu) +
-				       IDA_PCPU_SIZE * sizeof(unsigned),
+				       pool->percpu_max_size * sizeof(unsigned),
 				       sizeof(unsigned));
 	if (!pool->tag_cpu)
 		goto err;
@@ -332,4 +326,4 @@ err:
 	percpu_ida_destroy(pool);
 	return -ENOMEM;
 }
-EXPORT_SYMBOL_GPL(percpu_ida_init);
+EXPORT_SYMBOL_GPL(__percpu_ida_init);
Signature · 1 line
1.8.1.2
author maintainer
#20 lore raw

PATCH 8/11 percpu_ida: add percpu_ida_for_each_free

In reply to Jens Axboe

From: Shaohua Li <[email protected]>

Add a new API to iterate free ids. blk-mq-tag will use it.

Note, this doesn't guarantee to iterate all free ids restrictly. Caller should be aware of this. blk-mq uses it to do sanity check for request timedout, so can tolerate the limitation.

---
 include/linux/percpu_ida.h |  4 ++++
 lib/percpu_ida.c           | 44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+)
include/linux/percpu_ida.h +4 -0
index 56c1403..63510ae 100644
@@ -73,4 +73,8 @@ static inline int percpu_ida_init(struct percpu_ida *pool, unsigned long nr_tags
 		IDA_DEFAULT_PCPU_BATCH_MOVE);
 }
 
+typedef int (*percpu_ida_cb)(unsigned, void *);
+int percpu_ida_for_each_free(struct percpu_ida *pool, percpu_ida_cb fn,
+	void *data);
+
 #endif /* __PERCPU_IDA_H__ */
lib/percpu_ida.c +44 -0

47 lines not on this page · raw message

Signature · 1 line
1.8.1.2
author maintainer
#21 lore raw

PATCH 9/11 percpu_ida: add an API to return free tags

In reply to Jens Axboe

From: Shaohua Li <[email protected]>

Add an API to return free tags, blk-mq-tag will use it.

Note, this just returns a snapshot of free tags number. blk-mq-tag has two usages of it. One is for info output for diagnosis. The other is to quickly check if there are free tags for request dispatch checking. Neither requires very precise.

---
 include/linux/percpu_ida.h |  1 +
 lib/percpu_ida.c           | 17 +++++++++++++++++
 2 files changed, 18 insertions(+)
include/linux/percpu_ida.h +1 -0
index 63510ae..1900bd0 100644
@@ -77,4 +77,5 @@ typedef int (*percpu_ida_cb)(unsigned, void *);
 int percpu_ida_for_each_free(struct percpu_ida *pool, percpu_ida_cb fn,
 	void *data);
 
+unsigned percpu_ida_free_tags(struct percpu_ida *pool, int cpu);
 #endif /* __PERCPU_IDA_H__ */
lib/percpu_ida.c +17 -0
index 0f51c1b..b0698ea 100644
@@ -371,3 +371,20 @@ out:
 	return err;
 }
 EXPORT_SYMBOL_GPL(percpu_ida_for_each_free);
+
+/**
+ * percpu_ida_free_tags - return free tags number of a specific cpu or global pool
+ * @pool: pool related
+ * @cpu: specific cpu or global pool if @cpu == nr_cpu_ids
+ *
+ * Note: this just returns a snapshot of free tags number.
+ */
+unsigned percpu_ida_free_tags(struct percpu_ida *pool, int cpu)
+{
+	struct percpu_ida_cpu *remote;
+	if (cpu == nr_cpu_ids)
+		return pool->nr_free;
+	remote = per_cpu_ptr(pool->tag_cpu, cpu);
+	return remote->nr_free;
+}
+EXPORT_SYMBOL_GPL(percpu_ida_free_tags);
Signature · 1 line
1.8.1.2
author maintainer
#22 lore raw

PATCH 10/11 blk-mq: new multi-queue block IO queueing mechanism

In reply to Jens Axboe

Linux currently has two models for block devices:

- The classic request_fn based approach, where drivers use struct request units for IO. The block layer provides various helper functionalities to let drivers share code, things like tag management, timeout handling, queueing, etc.

- The "stacked" approach, where a driver squeezes in between the block layer and IO submitter. Since this bypasses the IO stack, driver generally have to manage everything themselves.

With drivers being written for new high IOPS devices, the classic request_fn based driver doesn't work well enough. The design dates back to when both SMP and high IOPS was rare. It has problems with scaling to bigger machines, and runs into scaling issues even on smaller machines when you have IOPS in the hundreds of thousands per device.

The stacked approach is then most often selected as the model for the driver. But this means that everybody has to re-invent everything, and along with that we get all the problems again that the shared approach solved.

This commit introduces blk-mq, block multi queue support. The design is centered around per-cpu queues for queueing IO, which then funnel down into x number of hardware submission queues. We might have a 1:1 mapping between the two, or it might be an N:M mapping. That all depends on what the hardware supports.

blk-mq provides various helper functions, which include:

- Scalable support for request tagging. Most devices need to be able to uniquely identify a request both in the driver and to the hardware. The tagging uses per-cpu caches for freed tags, to enable cache hot reuse.

- Timeout handling without tracking request on a per-device basis. Basically the driver should be able to get a notification, if a request happens to fail.

- Optional support for non 1:1 mappings between issue and submission queues. blk-mq can redirect IO completions to the desired location.

- Support for per-request payloads. Drivers almost always need to associate a request structure with some driver private command structure. Drivers can tell blk-mq this at init time, and then any request handed to the driver will have the required size of memory associated with it.

- Support for merging of IO, and plugging. The stacked model gets neither of these. Even for high IOPS devices, merging sequential IO reduces per-command overhead and thus increases bandwidth.

For now, this is provided as a potential 3rd queueing model, with the hope being that, as it matures, it can replace both the classic and stacked model. That would get us back to having just 1 real model for block devices, leaving the stacked approach to dm/md devices (as it was originally intended).

Contributions in this patch from the following people:

Shaohua Li <[email protected]>
Alexander Gordeev <[email protected]>
Christoph Hellwig <[email protected]>
Mike Christie <[email protected]>
Matias Bjorling <[email protected]>
Jeff Moyer <[email protected]>

---
 block/Makefile            |    5 +-
 block/blk-core.c          |  142 +++--
 block/blk-exec.c          |    7 +
 block/blk-flush.c         |  154 ++++-
 block/blk-mq-cpu.c        |   93 +++
 block/blk-mq-cpumap.c     |  108 ++++
 block/blk-mq-sysfs.c      |  384 ++++++++++++
 block/blk-mq-tag.c        |  204 +++++++
 block/blk-mq-tag.h        |   27 +
 block/blk-mq.c            | 1480 +++++++++++++++++++++++++++++++++++++++++++++
 block/blk-mq.h            |   52 ++
 block/blk-sysfs.c         |   13 +
 block/blk-timeout.c       |   73 ++-
 block/blk.h               |   17 +
 include/linux/bio.h       |    2 +
 include/linux/blk-mq.h    |  182 ++++++
 include/linux/blk_types.h |    2 +
 include/linux/blkdev.h    |   54 +-
 18 files changed, 2890 insertions(+), 109 deletions(-)
 create mode 100644 block/blk-mq-cpu.c
 create mode 100644 block/blk-mq-cpumap.c
 create mode 100644 block/blk-mq-sysfs.c
 create mode 100644 block/blk-mq-tag.c
 create mode 100644 block/blk-mq-tag.h
 create mode 100644 block/blk-mq.c
 create mode 100644 block/blk-mq.h
 create mode 100644 include/linux/blk-mq.h
block/Makefile +3 -2
index 671a83d..20645e8 100644
@@ -5,8 +5,9 @@
 obj-$(CONFIG_BLOCK) := elevator.o blk-core.o blk-tag.o blk-sysfs.o \
 			blk-flush.o blk-settings.o blk-ioc.o blk-map.o \
 			blk-exec.o blk-merge.o blk-softirq.o blk-timeout.o \
-			blk-iopoll.o blk-lib.o ioctl.o genhd.o scsi_ioctl.o \
-			partition-generic.o partitions/
+			blk-iopoll.o blk-lib.o blk-mq.o blk-mq-tag.o \
+			blk-mq-sysfs.o blk-mq-cpu.o blk-mq-cpumap.o ioctl.o \
+			genhd.o scsi_ioctl.o partition-generic.o partitions/
 
 obj-$(CONFIG_BLK_DEV_BSG)	+= bsg.o
 obj-$(CONFIG_BLK_DEV_BSGLIB)	+= bsg-lib.o
block/blk-core.c +84 -58

285 lines not on this page · raw message

block/blk-exec.c +7 -0
index 6b18d82..c3edf9d 100644
@@ -5,6 +5,7 @@
 #include <linux/module.h>
 #include <linux/bio.h>
 #include <linux/blkdev.h>
+#include <linux/blk-mq.h>
 #include <linux/sched/sysctl.h>
 
 #include "blk.h"
@@ -58,6 +59,12 @@ void blk_execute_rq_nowait(struct request_queue *q, struct gendisk *bd_disk,
 
 	rq->rq_disk = bd_disk;
 	rq->end_io = done;
+
+	if (q->mq_ops) {
+		blk_mq_insert_request(q, rq, true);
+		return;
+	}
+
 	/*
 	 * need to check this before __blk_run_queue(), because rq can
 	 * be freed before that returns.
block/blk-flush.c +139 -15

271 lines not on this page · raw message

block/blk-mq-cpu.c new file +93 -0

93 lines not on this page · raw message

block/blk-mq-cpumap.c new file +108 -0

108 lines not on this page · raw message

block/blk-mq-sysfs.c new file +384 -0

384 lines not on this page · raw message

block/blk-mq-tag.c new file +204 -0

204 lines not on this page · raw message

block/blk-mq-tag.h new file +27 -0
new file mode 100644
index 0000000..947ba2c
@@ -0,0 +1,27 @@
+#ifndef INT_BLK_MQ_TAG_H
+#define INT_BLK_MQ_TAG_H
+
+struct blk_mq_tags;
+
+extern struct blk_mq_tags *blk_mq_init_tags(unsigned int nr_tags, unsigned int reserved_tags, int node);
+extern void blk_mq_free_tags(struct blk_mq_tags *tags);
+
+extern unsigned int blk_mq_get_tag(struct blk_mq_tags *tags, gfp_t gfp, bool reserved);
+extern void blk_mq_wait_for_tags(struct blk_mq_tags *tags);
+extern void blk_mq_put_tag(struct blk_mq_tags *tags, unsigned int tag);
+extern void blk_mq_tag_busy_iter(struct blk_mq_tags *tags, void (*fn)(void *data, unsigned long *), void *data);
+extern bool blk_mq_has_free_tags(struct blk_mq_tags *tags);
+extern ssize_t blk_mq_tag_sysfs_show(struct blk_mq_tags *tags, char *page);
+
+enum {
+	BLK_MQ_TAG_CACHE_MIN	= 1,
+	BLK_MQ_TAG_CACHE_MAX	= 64,
+};
+
+enum {
+	BLK_MQ_TAG_FAIL		= -1U,
+	BLK_MQ_TAG_MIN		= BLK_MQ_TAG_CACHE_MIN,
+	BLK_MQ_TAG_MAX		= BLK_MQ_TAG_FAIL - 1,
+};
+
+#endif
block/blk-mq.c new file +1480 -0

1,480 lines not on this page · raw message

block/blk-mq.h new file +52 -0

52 lines not on this page · raw message

block/blk-sysfs.c +13 -0
index 3aa5b19..4f8c4d9 100644
@@ -7,6 +7,7 @@
 #include <linux/bio.h>
 #include <linux/blkdev.h>
 #include <linux/blktrace_api.h>
+#include <linux/blk-mq.h>
 
 #include "blk.h"
 #include "blk-cgroup.h"
@@ -542,6 +543,11 @@ static void blk_release_queue(struct kobject *kobj)
 	if (q->queue_tags)
 		__blk_queue_free_tags(q);
 
+	percpu_counter_destroy(&q->mq_usage_counter);
+
+	if (q->mq_ops)
+		blk_mq_free_queue(q);
+
 	blk_trace_shutdown(q);
 
 	bdi_destroy(&q->backing_dev_info);
@@ -575,6 +581,7 @@ int blk_register_queue(struct gendisk *disk)
 	 * bypass from queue allocation.
 	 */
 	blk_queue_bypass_end(q);
+	queue_flag_set_unlocked(QUEUE_FLAG_INIT_DONE, q);
 
 	ret = blk_trace_init_sysfs(dev);
 	if (ret)
@@ -588,6 +595,9 @@ int blk_register_queue(struct gendisk *disk)
 
 	kobject_uevent(&q->kobj, KOBJ_ADD);
 
+	if (q->mq_ops)
+		blk_mq_register_disk(disk);
+
 	if (!q->request_fn)
 		return 0;
 
@@ -610,6 +620,9 @@ void blk_unregister_queue(struct gendisk *disk)
 	if (WARN_ON(!q))
 		return;
 
+	if (q->mq_ops)
+		blk_mq_unregister_disk(disk);
+
 	if (q->request_fn)
 		elv_unregister_queue(q);
 
block/blk-timeout.c +46 -27

23 lines not on this page · raw message

block/blk.h +17 -0
index e837b8f..c90e1d8 100644

Lines of this file are not shown. View the raw message.

include/linux/bio.h +2 -0
index ec48bac..4c27754 100644

Lines of this file are not shown. View the raw message.

include/linux/blk-mq.h new file +182 -0

182 lines not on this page · raw message

include/linux/blk_types.h +2 -0
index c26801e..238ef0e 100644

Lines of this file are not shown. View the raw message.

include/linux/blkdev.h +47 -7

54 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.8.1.2

Part of the diff is not shown here. View the raw message.

#23 lore raw

Re: [PATCH 10/11] blk-mq: new multi-queue block IO queueing mechanism

block/blk-mq-sysfs.c line 291

Quoted 4 lines · Jens Axboe
block/blk-mq-sysfs.c new file +4 -0
+static struct blk_mq_hw_ctx_sysfs_entry blk_mq_hw_sysfs_ipi = {
+	.attr = {.name = "ipi_redirect", .mode = S_IRUGO | S_IWUGO},
+	.show = blk_mq_hw_sysfs_ipi_show,
+	.store = blk_mq_hw_sysfs_ipi_store,

Is having this world writable a good idea ?

Dave

author maintainer
#24 lore raw

Re: [PATCH 10/11] blk-mq: new multi-queue block IO queueing mechanism

Quoted 7 lines · Dave Jones

block/blk-mq-sysfs.c line 291

Quoted 4 lines · Jens Axboe
block/blk-mq-sysfs.c new file +4 -0
+static struct blk_mq_hw_ctx_sysfs_entry blk_mq_hw_sysfs_ipi = {
+	.attr = {.name = "ipi_redirect", .mode = S_IRUGO | S_IWUGO},
+	.show = blk_mq_hw_sysfs_ipi_show,
+	.store = blk_mq_hw_sysfs_ipi_store,

Is having this world writable a good idea ?

Good catch, that's a typo (should be S_IWUSR). Will fix up.

Signature · 1 line
Jens Axboe
author maintainer
#26 lore raw

PATCH 11/11 null_blk: multi queue aware block test driver

In reply to Jens Axboe

A driver that simply completes IO it receives, it does no transfers. Written to fascilitate testing of the blk-mq code. It supports various module options to use either bio queueing, rq queueing, or mq mode.

---
 drivers/block/Kconfig    |   3 +
 drivers/block/Makefile   |   1 +
 drivers/block/null_blk.c | 638 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 642 insertions(+)
 create mode 100644 drivers/block/null_blk.c
drivers/block/Kconfig +3 -0
index e07a5fd..4682546 100644
@@ -15,6 +15,9 @@ menuconfig BLK_DEV
 
 if BLK_DEV
 
+config BLK_DEV_NULL_BLK
+	tristate "Null test block driver"
+
 config BLK_DEV_FD
 	tristate "Normal floppy disk support"
 	depends on ARCH_MAY_HAVE_PC_FDC
drivers/block/Makefile +1 -0
index ca07399..03b3b4a 100644
@@ -41,6 +41,7 @@ obj-$(CONFIG_BLK_DEV_RBD)     += rbd.o
 obj-$(CONFIG_BLK_DEV_PCIESSD_MTIP32XX)	+= mtip32xx/
 
 obj-$(CONFIG_BLK_DEV_RSXX) += rsxx/
+obj-$(CONFIG_BLK_DEV_NULL_BLK)	+= null_blk.o
 
 nvme-y		:= nvme-core.o nvme-scsi.o
 swim_mod-y	:= swim.o swim_asm.o
drivers/block/null_blk.c new file +638 -0

638 lines not on this page · raw message

Signature · 1 line
1.8.1.2