Skip to contentSkip to conversation

Add support for write life time hints

PATCH Superseded Jens Axboe 5 participants 35 messages Jun 21, 2017 – Jun 27, 2017

Subsystems (9)

v9 of 10 Superseded Latest is v10 →

lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH v9 1/1 Add support for write life time hints

A new iteration of this patchset, previously known as write streams. As before, this patchset aims at enabling applications split up writes into separate streams, based on the perceived life time of the data written. This is useful for a variety of reasons:

- For NVMe, this feature is ratified and released with the NVMe 1.3 spec. Devices implementing Directives can expose multiple streams. Separating data written into streams based on life time can drastically reduce the write amplification. This helps device endurance, and increases performance. Testing just performed internally at Facebook with these patches showed up to a 25% reduction in NAND writes in a RocksDB setup.

- Software caching solutions can make more intelligent decisions on how and where to place data.

Contrary to previous patches, we're not exposing numeric stream values anymore. I've previously advocated for just doing a set of hints that makes sense instead. See the coverage from the LSFMM summit this year:

https://lwn.net/Articles/717755/

This patchset attempts to do that. We add an fcntl(2) interface to get/set these types of hints. We define 4 hints that pertain to data write life times:

RWH_WRITE_LIFE_SHORT Data written with this flag is expected to have
a high overwrite rate, or life time.

RWH_WRITE_LIFE_MEDIUM Longer life time than SHORT

RWH_WRITE_LIFE_LONG Longer life time than MEDIUM

RWH_WRITE_LIFE_EXTREME Longer life time than LONG

The idea is that these are relative values, so an application can use them as they see fit. The underlying device can then place data appropriately, or be free to ignore the hint. It's just a hint.

A branch based on current master can be pulled
from here:

git://git.kernel.dk/linux-block write-stream.9

Changes since v8:

- Add file write hints as well. File hints override inode hints, if both are valid and available.
- Distinguish between "hint not set" or "hint none".
- NVMe: remove global stream allocation and stream parameter
- Rebase on top of new for-4.13/block, to fixup conflicts with the NOWAIT patchset.

Changes since v7:

- NVMe: change 'streams' parameter to be a bool enable/disable. We hardwire the number of streams anyway and use the appropriate amount, so no point in exposing this value.
- NVMe: collapse stream values appropriately, instead of just doing a basic MOD.
- Get rid of pwritev2(2) flags. Just use the fcntl(2) interface.
- Collapse some patches
- Change fcntl(2) interface to get/set values from a user supplied 64-bit pointer.
- Move inode-to-iocb mask setting to iocb_flags().

Changes since v6:

- Rewrite NVMe write stream assignment
- Change NVMe stream assignment to be per-controller, not per-ns. Then we can use the same IDs across name spaces, and we don't have to do lazy setup of streams.
- If streams are enabled on nvme, set io min/opt and discard granularity based on the stream params reported.
- Fixup F_SET_RW_HINT definition, it was 20, should have been 12.

Changes since v5:

- Change enum write_hint to enum rw_hint.
- Change fcntl() interface to be read/write generic
- Bring enum rw_hint all the way to bio/request
- Change references to streams in changelogs and debugfs interface
- Rebase to master to resolve blkdev.h conflict
- Reshuffle patches so the WRITE_LIFE_* hints and type come first. Allowed me to merge two block patches as well.

Changes since v4:

- Add enum write_hint and the WRITE_HINT_* values. This is what we use internally (until transformed to req/bio flags), and what is exposed to user space with the fcntl() interface. Maps directly to the RWF_WRITE_LIFE_* values.
- Add fcntl() interface for getting/setting hint values.
- Get rid of inode ->i_write_hint, encode the 3 bits of hint info in the inode flags intead.
- Allow a write with no hint to clear the old hint. Previously we only changed the hint if a new valid hint was given, not if no hint was passed in.
- Shrink flag space grabbed from 4 to 3 bits for RWF_* and the inode flags.

Changes since v3:

- Change any naming of stream ID to write hint.
- Various little API changes, suggested by Christoph
- Cleanup the NVMe bits, dump the debug info.
- Change NVMe to lazily allocate the streams.
- Various NVMe error handling improvements and command checking.

Changes since v2:

- Get rid of bio->bi_stream and replace with four request/bio flags. These map directly to the RWF_WRITE_* flags that the user passes in.
- Cleanup the NVMe stream setting.
- Drivers now responsible for updating the queue stream write counter, as they determine what stream to map a given flag to.

Changes since v1:

- Guard queue stream stats to ensure we don't mess up memory, if bio_stream() ever were to return a larger value than we support.
- NVMe: ensure we set the stream modulo the name space defined count.
- Cleanup the RWF_ and IOCB_ flags. Set aside 4 bits, and just store the stream value in there. This makes the passing of stream ID from RWF_ space to IOCB_ (and IOCB_ to bio) more efficient, and cleans it up in general.
- Kill the block internal definitions of the stream type, we don't need them anymore. See above.

 block/blk-merge.c          |   16 +++++
 block/blk-mq-debugfs.c     |   24 +++++++
 drivers/nvme/host/core.c   |  142 +++++++++++++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h   |    4 +
 fs/block_dev.c             |    2 
 fs/btrfs/extent_io.c       |    1 
 fs/buffer.c                |   14 ++--
 fs/direct-io.c             |    2 
 fs/ext4/page-io.c          |    2 
 fs/fcntl.c                 |   60 +++++++++++++++++++
 fs/inode.c                 |   11 +++
 fs/iomap.c                 |    1 
 fs/mpage.c                 |    1 
 fs/open.c                  |    1 
 fs/xfs/xfs_aops.c          |    2 
 include/linux/blk_types.h  |   31 +++++++++
 include/linux/blkdev.h     |    3 
 include/linux/fs.h         |   74 ++++++++++++++++++++++-
 include/linux/nvme.h       |   48 +++++++++++++++
 include/uapi/linux/fcntl.h |   16 +++++
 20 files changed, 444 insertions(+), 11 deletions(-)
Signature · 1 line
Jens Axboe
author maintainer
#1 lore raw

PATCH 1/9 fs: add fcntl() interface for setting/getting write life time hints

Define a set of write life time hints:

RWH_WRITE_LIFE_NONE No hints about write life time
RWH_WRITE_LIFE_SHORT Data written has a short life time
RWH_WRITE_LIFE_MEDIUM Data written has a medium life time
RWH_WRITE_LIFE_LONG Data written has a long life time
RWH_WRITE_LIFE_EXTREME Data written has an extremely long life tim

The intent is for these values to be relative to each other, no absolute meaning should be attached to these flag names.

Add an fcntl interface for querying these flags, and also for setting them as well:

F_GET_RW_HINT Returns the read/write hint set.

F_SET_RW_HINT Pass one of the above write hints.

The user passes in a 64-bit pointer to get/set these values, and the interface returns 0/-1 on success/error.

Sample program testing/implementing basic setting/getting of write hints is below.

Add support for storing the write life time hint in the inode flags and in struct file as well, and pass them to the kiocb flags. If both a file and its corresponding inode has a write hint, then we use the one in the file, if available. The file hint can be used for sync/direct IO, for buffered writeback only the inode hint is available.

This is in preparation for utilizing these hints in the block layer, to guide on-media data placement.

/*
* writehint.c: check or set a file/inode write hint */
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdbool.h>
#include <inttypes.h>

#ifndef F_RW_GET_HINT
#define F_LINUX_SPECIFIC_BASE 1024
#define F_RW_GET_HINT (F_LINUX_SPECIFIC_BASE + 11)
#define F_RW_SET_HINT (F_LINUX_SPECIFIC_BASE + 12)
#endif

static char *str[] = { "WRITE_LIFE_NOT_SET", "WRITE_LIFE_NONE",

			"WRITE_LIFE_SHORT", "WRITE_LIFE_MEDIUM",
			"WRITE_LIFE_LONG", "WRITE_LIFE_EXTREME" };

int main(int argc, char *argv[])

{
	uint64_t hint;
	int fd, ret;

	if (argc < 2) {
		fprintf(stderr, "%s: file <hint>\n", argv[0]);
		return 1;
	}

	fd = open(argv[1], O_RDONLY);
	if (fd < 0) {
		perror("open");
		return 2;
	}

	if (argc > 2) {
		hint = atoi(argv[2]);
		ret = fcntl(fd, F_RW_SET_HINT, &hint);
		if (ret < 0) {
			perror("fcntl: F_RW_SET_HINT");
			return 4;
		}
	}

	ret = fcntl(fd, F_RW_GET_HINT, &hint);
	if (ret < 0) {
		perror("fcntl: F_RW_GET_HINT");
		return 3;
	}

	printf("%s: hint %s\n", argv[1], str[hint]);
	close(fd);
	return 0;
}
---
 fs/fcntl.c                 | 60 +++++++++++++++++++++++++++++++++++++
 fs/inode.c                 | 11 +++++++
 fs/open.c                  |  1 +
 include/linux/fs.h         | 74 ++++++++++++++++++++++++++++++++++++++++++++--
 include/uapi/linux/fcntl.h | 16 ++++++++++
 5 files changed, 160 insertions(+), 2 deletions(-)
fs/fcntl.c +60 -0
index f4e7267d117f..7037f0560f36 100644
@@ -243,6 +243,62 @@ static int f_getowner_uids(struct file *filp, unsigned long arg)
 }
 #endif
 
+static long fcntl_rw_hint(struct file *file, unsigned int cmd,
+			  unsigned long arg)
+{
+	struct inode *inode = file_inode(file);
+	enum rw_hint hint, old_hint;
+	long ret = 0;
+
+	switch (cmd) {
+	case F_GET_RW_HINT:
+		if (file->f_write_hint != WRITE_LIFE_NOT_SET)
+			hint = file->f_write_hint;
+		else
+			hint = mask_to_write_hint(inode->i_flags,
+							S_WRITE_LIFE_SHIFT);
+		if (put_user(hint, (u64 __user *) arg))
+			ret = -EFAULT;
+		break;
+	case F_SET_RW_HINT:
+		if (get_user(hint, (u64 __user *) arg)) {
+			ret = -EFAULT;
+			break;
+		}
+		switch (hint) {
+		case WRITE_LIFE_NOT_SET:
+		case WRITE_LIFE_NONE:
+		case WRITE_LIFE_SHORT:
+		case WRITE_LIFE_MEDIUM:
+		case WRITE_LIFE_LONG:
+		case WRITE_LIFE_EXTREME:
+			spin_lock(&file->f_lock);
+			file->f_write_hint = hint;
+			spin_unlock(&file->f_lock);
+
+			/*
+			 * Only propagate hint to inode, if no hint is set,
+			 * or if the hint is being cleared
+			 */
+			old_hint = mask_to_write_hint(inode->i_flags,
+							S_WRITE_LIFE_SHIFT);
+			if (old_hint == WRITE_LIFE_NOT_SET ||
+			    hint == WRITE_LIFE_NOT_SET)
+				inode_set_write_hint(inode, hint);
+			ret = 0;
+			break;
+		default:
+			ret = -EINVAL;
+		}
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
 static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,
 		struct file *filp)
 {
@@ -337,6 +393,10 @@ static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,
 	case F_GET_SEALS:
 		err = shmem_fcntl(filp, cmd, arg);
 		break;
+	case F_GET_RW_HINT:
+	case F_SET_RW_HINT:
+		err = fcntl_rw_hint(filp, cmd, arg);
+		break;
 	default:
 		break;
 	}
fs/inode.c +11 -0
index db5914783a71..defb015a2c6d 100644
@@ -2120,3 +2120,14 @@ struct timespec current_time(struct inode *inode)
 	return timespec_trunc(now, inode->i_sb->s_time_gran);
 }
 EXPORT_SYMBOL(current_time);
+
+void inode_set_write_hint(struct inode *inode, enum rw_hint hint)
+{
+	unsigned int flags = write_hint_to_mask(hint, S_WRITE_LIFE_SHIFT);
+
+	if (flags != mask_to_write_hint(inode->i_flags, S_WRITE_LIFE_SHIFT)) {
+		inode_lock(inode);
+		inode_set_flags(inode, flags, S_WRITE_LIFE_MASK);
+		inode_unlock(inode);
+	}
+}
fs/open.c +1 -0
index cd0c5be8d012..3fe0c4aa7d27 100644
@@ -759,6 +759,7 @@ static int do_dentry_open(struct file *f,
 	     likely(f->f_op->write || f->f_op->write_iter))
 		f->f_mode |= FMODE_CAN_WRITE;
 
+	f->f_write_hint = WRITE_LIFE_NOT_SET;
 	f->f_flags &= ~(O_CREAT | O_EXCL | O_NOCTTY | O_TRUNC);
 
 	file_ra_state_init(&f->f_ra, f->f_mapping->host->i_mapping);
include/linux/fs.h +72 -2

128 lines not on this page · raw message

include/uapi/linux/fcntl.h +16 -0
index 813afd6eee71..defe6e77fc99 100644
@@ -43,6 +43,22 @@
 /* (1U << 31) is reserved for signed error codes */
 
 /*
+ * Set/Get write life time hints.
+ */
+#define F_GET_RW_HINT		(F_LINUX_SPECIFIC_BASE + 11)
+#define F_SET_RW_HINT		(F_LINUX_SPECIFIC_BASE + 12)
+
+/*
+ * Valid hint values for F_{GET,SET}_RW_HINT. 0 is "not set", or can be
+ * used to clear any hints previously set.
+ */
+#define RWH_WRITE_LIFE_NONE	1
+#define RWH_WRITE_LIFE_SHORT	2
+#define RWH_WRITE_LIFE_MEDIUM	3
+#define RWH_WRITE_LIFE_LONG	4
+#define RWH_WRITE_LIFE_EXTREME	5
+
+/*
  * Types of directory notifications that may be requested.
  */
 #define DN_ACCESS	0x00000001	/* File accessed */
Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

Please document the userspace API (added linux-api and linux-man to CC for sugestions), especially including the odd effects of the per-inode settings.

Also I would highly recommend to use different fcntl commands for the file vs inode hints to avoid any strange behavior.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author maintainer
#3 lore raw

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

Quoted 3 lines · Christoph Hellwig

Please document the userspace API (added linux-api and linux-man to CC for sugestions), especially including the odd effects of the per-inode settings.

Of course, I'll send in a diff for the fcntl(2) man page.

Quoted 2 lines

Also I would highly recommend to use different fcntl commands for the file vs inode hints to avoid any strange behavior.

OK, used to have that too... I can add specific _FILE versions.

Signature · 1 line
Jens Axboe
#4 lore raw

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

Quoted 11 lines · Jens Axboe
Quoted 3 lines · Christoph Hellwig

Please document the userspace API (added linux-api and linux-man to CC for sugestions), especially including the odd effects of the per-inode settings.

Of course, I'll send in a diff for the fcntl(2) man page.

Quoted 2 lines

Also I would highly recommend to use different fcntl commands for the file vs inode hints to avoid any strange behavior.

OK, used to have that too... I can add specific _FILE versions.

While you're at it, can you also send in an xfstest or two to check the basic functionality of the fcntl so that we know the code reflects the userspace API ("I set this hint and now I can query it back" and "file hint overrides inode hint") that we want?

--D

Quoted 8 lines
Signature · 7 lines
-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-api" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author maintainer
#5 lore raw

Re: [PATCH 1/9] fs: add fcntl() interface for setting/getting write life time hints

Quoted 11 lines · Darrick J. Wong
Quoted 10 lines · Jens Axboe
Quoted 3 lines · Christoph Hellwig

Please document the userspace API (added linux-api and linux-man to CC for sugestions), especially including the odd effects of the per-inode settings.

Of course, I'll send in a diff for the fcntl(2) man page.

Quoted 2 lines

Also I would highly recommend to use different fcntl commands for the file vs inode hints to avoid any strange behavior.

Quoted 6 lines
Quoted 1 line

OK, used to have that too... I can add specific _FILE versions.

While you're at it, can you also send in an xfstest or two to check the basic functionality of the fcntl so that we know the code reflects the userspace API ("I set this hint and now I can query it back" and "file hint overrides inode hint") that we want?

I definitely can. I already wrote the below to verify that it behaves the way it should.

/*
* test-writehints.c: test file/inode write hint setting/getting */
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdbool.h>
#include <inttypes.h>
#include <assert.h>

#ifndef F_GET_RW_HINT
#define F_LINUX_SPECIFIC_BASE 1024
#define F_GET_RW_HINT (F_LINUX_SPECIFIC_BASE + 11)
#define F_SET_RW_HINT (F_LINUX_SPECIFIC_BASE + 12)
#define F_GET_FILE_RW_HINT (F_LINUX_SPECIFIC_BASE + 13)
#define F_SET_FILE_RW_HINT (F_LINUX_SPECIFIC_BASE + 14)

#define RWF_WRITE_LIFE_NOT_SET 0
#define RWH_WRITE_LIFE_NONE 1
#define RWH_WRITE_LIFE_SHORT 2
#define RWH_WRITE_LIFE_MEDIUM 3
#define RWH_WRITE_LIFE_LONG 4
#define RWH_WRITE_LIFE_EXTREME 5

#endif

static int __get_write_hint(int fd, int cmd)

{
	uint64_t hint;
	int ret;

	ret = fcntl(fd, cmd, &hint);
	if (ret < 0) {
		perror("fcntl: F_GET_RW_FILE_HINT");
		return -1;
	}

	return hint;
}

static int get_file_write_hint(int fd)

{
	return __get_write_hint(fd, F_GET_FILE_RW_HINT);
}

static int get_inode_write_hint(int fd)

{
	return __get_write_hint(fd, F_GET_RW_HINT);
}

static void set_file_write_hint(int fd, uint64_t hint)

{
	uint64_t set_hint = hint;
	int ret;

	ret = fcntl(fd, F_SET_FILE_RW_HINT, &set_hint);
	if (ret < 0) {
		perror("fcntl: F_RW_SET_HINT");
		return;
	}
}

static void set_inode_write_hint(int fd, uint64_t hint)

{
	uint64_t set_hint = hint;
	int ret;

	ret = fcntl(fd, F_SET_RW_HINT, &set_hint);
	if (ret < 0) {
		perror("fcntl: F_RW_SET_HINT");
		return;
	}
}

int main(int argc, char *argv[])

{
	char filename[] = "/tmp/writehintsXXXXXX";
	int ihint, fhint, fd;

	fd = open(filename, O_RDWR | O_CREAT | 0644);
	if (fd < 0) {
		perror("open");
		return 2;
	}

	/*
	 * Default hints for both file and inode should be NOT_SET
	 */
	fhint = get_file_write_hint(fd);
	if (fhint < 0)
		return 0;
	ihint = get_inode_write_hint(fd);
	assert(fhint == ihint);
	assert(fhint == RWF_WRITE_LIFE_NOT_SET);

	/*
	 * Set inode hint, check file hint returns the right hint
	 */
	set_inode_write_hint(fd, RWH_WRITE_LIFE_SHORT);
	fhint = get_file_write_hint(fd);
	ihint = get_inode_write_hint(fd);
	assert(fhint == ihint);
	assert(fhint == RWH_WRITE_LIFE_SHORT);

	/*
	 * Now set file hint, ensure that this is now the hint we get
	 */
	set_file_write_hint(fd, RWH_WRITE_LIFE_LONG);
	fhint = get_file_write_hint(fd);
	ihint = get_inode_write_hint(fd);
	assert(fhint == RWH_WRITE_LIFE_LONG);
	assert(ihint == RWH_WRITE_LIFE_SHORT);

	/*
	 * Clear inode write hint, ensure that file still returns the set hint
	 */
	set_inode_write_hint(fd, RWF_WRITE_LIFE_NOT_SET);
	fhint = get_file_write_hint(fd);
	ihint = get_inode_write_hint(fd);
	assert(fhint == RWH_WRITE_LIFE_LONG);
	assert(ihint == RWF_WRITE_LIFE_NOT_SET);

	/*
	 * Clear file write hint, ensure that now returns cleared
	 */
	set_file_write_hint(fd, RWF_WRITE_LIFE_NOT_SET);
	fhint = get_file_write_hint(fd);
	assert(fhint == RWF_WRITE_LIFE_NOT_SET);

	close(fd);
	unlink(filename);
	return 0;
}
Signature · 1 line
Jens Axboe
author maintainer
#6 lore raw

PATCH 2/9 block: add support for write hints in a bio

In reply to Jens Axboe

No functional changes in this patch, we just set aside 3 bits in the bio/request flags, which can be used to hold a WRITE_LIFE_* life time hint.

Ensure that we don't merge requests that have different life time hints assigned to them.

---
 block/blk-merge.c         | 16 ++++++++++++++++
 include/linux/blk_types.h | 31 +++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)
block/blk-merge.c +16 -0
index cea544ec5d96..17cd0c3f872d 100644
@@ -670,6 +670,14 @@ static struct request *attempt_merge(struct request_queue *q,
 		return NULL;
 
 	/*
+	 * Don't allow merge of different streams, or for a stream with
+	 * non-stream IO.
+	 */
+	if ((req->cmd_flags & REQ_WRITE_LIFE_MASK) !=
+	    (next->cmd_flags & REQ_WRITE_LIFE_MASK))
+		return NULL;
+
+	/*
 	 * If we are allowed to merge, then append bio list
 	 * from next to rq and release next. merge_requests_fn
 	 * will have updated segment counts, update sector
@@ -788,6 +796,14 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio)
 	    !blk_write_same_mergeable(rq->bio, bio))
 		return false;
 
+	/*
+	 * Don't allow merge of different streams, or for a stream with
+	 * non-stream IO.
+	 */
+	if ((rq->cmd_flags & REQ_WRITE_LIFE_MASK) !=
+	    (bio->bi_opf & REQ_WRITE_LIFE_MASK))
+		return false;
+
 	return true;
 }
 
include/linux/blk_types.h +31 -0
index e210da6d14b8..4ce71298e45b 100644
@@ -7,6 +7,7 @@
 
 #include <linux/types.h>
 #include <linux/bvec.h>
+#include <linux/fs.h>
 
 struct bio_set;
 struct bio;
@@ -223,6 +224,10 @@ enum req_flag_bits {
 	__REQ_RAHEAD,		/* read ahead, can fail anytime */
 	__REQ_BACKGROUND,	/* background IO */
 
+	__REQ_WRITE_HINT_SHIFT,	/* 3 bits for life time hint */
+	__REQ_WRITE_HINT_PAD1,
+	__REQ_WRITE_HINT_PAD2,
+
 	/* command specific flags for REQ_OP_WRITE_ZEROES: */
 	__REQ_NOUNMAP,		/* do not free blocks when zeroing */
 
@@ -244,6 +249,13 @@ enum req_flag_bits {
 #define REQ_RAHEAD		(1ULL << __REQ_RAHEAD)
 #define REQ_BACKGROUND		(1ULL << __REQ_BACKGROUND)
 
+#define REQ_WRITE_SHORT		(WRITE_LIFE_SHORT << __REQ_WRITE_HINT_SHIFT)
+#define REQ_WRITE_MEDIUM	(WRITE_LIFE_MEDIUM << __REQ_WRITE_HINT_SHIFT)
+#define REQ_WRITE_LONG		(WRITE_LIFE_LONG << __REQ_WRITE_HINT_SHIFT)
+#define REQ_WRITE_EXTREME	(WRITE_LIFE_EXTREME << __REQ_WRITE_HINT_SHIFT)
+
+#define REQ_WRITE_LIFE_MASK	(0x7 << __REQ_WRITE_HINT_SHIFT)
+ 
 #define REQ_NOUNMAP		(1ULL << __REQ_NOUNMAP)
 #define REQ_NOWAIT		(1ULL << __REQ_NOWAIT)
 
@@ -335,4 +347,23 @@ struct blk_rq_stat {
 	u64 batch;
 };
 
+static inline unsigned int write_hint_to_opf(enum rw_hint hint)
+{
+	return hint << __REQ_WRITE_HINT_SHIFT;
+}
+
+/*
+ * Don't let drivers see WRITE_LIFE_NOT_SET, return NONE for that
+ */
+static inline enum rw_hint opf_to_write_hint(unsigned int opf)
+{
+	enum rw_hint ret;
+
+	ret = (opf & REQ_WRITE_LIFE_MASK) >> __REQ_WRITE_HINT_SHIFT;
+	if (ret == WRITE_LIFE_NOT_SET)
+		ret = WRITE_LIFE_NONE;
+
+	return ret;
+}
+
 #endif /* __LINUX_BLK_TYPES_H */
Signature · 1 line
2.7.4
author maintainer
#8 lore raw

PATCH 3/9 blk-mq: expose stream write hints through debugfs

In reply to Jens Axboe

Useful to verify that things are working the way they should. Reading the file will return number of kb written with each write hint. Writing the file will reset the statistics. No care is taken to ensure that we don't race on updates.

Drivers will write to q->write_hints[] if they handle a given write hint.

---
 block/blk-mq-debugfs.c | 24 ++++++++++++++++++++++++
 include/linux/blkdev.h |  3 +++
 2 files changed, 27 insertions(+)
block/blk-mq-debugfs.c +24 -0
index 9edebbdce0bd..9ebc2945f991 100644
@@ -135,6 +135,29 @@ static void print_stat(struct seq_file *m, struct blk_rq_stat *stat)
 	}
 }
 
+static int queue_write_hint_show(void *data, struct seq_file *m)
+{
+	struct request_queue *q = data;
+	int i;
+
+	for (i = 0; i < BLK_MAX_WRITE_HINTS; i++)
+		seq_printf(m, "hint%d: %llu\n", i, q->write_hints[i]);
+
+	return 0;
+}
+
+static ssize_t queue_write_hint_store(void *data, const char __user *buf,
+				      size_t count, loff_t *ppos)
+{
+	struct request_queue *q = data;
+	int i;
+
+	for (i = 0; i < BLK_MAX_WRITE_HINTS; i++)
+		q->write_hints[i] = 0;
+
+	return count;
+}
+
 static int queue_poll_stat_show(void *data, struct seq_file *m)
 {
 	struct request_queue *q = data;
@@ -730,6 +753,7 @@ static const struct blk_mq_debugfs_attr blk_mq_debugfs_queue_attrs[] = {
 	{"poll_stat", 0400, queue_poll_stat_show},
 	{"requeue_list", 0400, .seq_ops = &queue_requeue_list_seq_ops},
 	{"state", 0600, queue_state_show, queue_state_write},
+	{"write_hints", 0600, queue_write_hint_show, queue_write_hint_store},
 	{},
 };
 
include/linux/blkdev.h +3 -0
index 22cfba64ce81..687394b70924 100644
@@ -586,6 +586,9 @@ struct request_queue {
 
 	size_t			cmd_size;
 	void			*rq_alloc_data;
+
+#define BLK_MAX_WRITE_HINTS	5
+	u64			write_hints[BLK_MAX_WRITE_HINTS];
 };
 
 #define QUEUE_FLAG_QUEUED	1	/* uses generic tag queueing */
Signature · 1 line
2.7.4
author maintainer
#10 lore raw

PATCH 4/9 fs: add O_DIRECT support for sending down write life time hints

In reply to Jens Axboe
---
 fs/block_dev.c | 2 ++
 fs/direct-io.c | 2 ++
 fs/iomap.c     | 1 +
 3 files changed, 5 insertions(+)
fs/block_dev.c +2 -0
index dd91c99e9ba0..30e1fb65c2fa 100644
@@ -183,6 +183,8 @@ static unsigned int dio_bio_write_op(struct kiocb *iocb)
 	/* avoid the need for a I/O completion work item */
 	if (iocb->ki_flags & IOCB_DSYNC)
 		op |= REQ_FUA;
+
+	op |= write_hint_to_opf(iocb_write_hint(iocb));
 	return op;
 }
 
fs/direct-io.c +2 -0
index c87077d1dc33..5fea570551e5 100644
@@ -385,6 +385,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio,
 	else
 		bio->bi_end_io = dio_bio_end_io;
 
+	bio->bi_opf |= write_hint_to_opf(iocb_write_hint(dio->iocb));
+
 	sdio->bio = bio;
 	sdio->logical_offset_in_bio = sdio->cur_page_fs_offset;
 }
fs/iomap.c +1 -0
index c71a64b97fba..9c9f8406018b 100644
@@ -804,6 +804,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 
 		if (dio->flags & IOMAP_DIO_WRITE) {
 			bio_set_op_attrs(bio, REQ_OP_WRITE, REQ_SYNC | REQ_IDLE);
+			bio->bi_opf |= write_hint_to_opf(inode_write_hint(inode));
 			task_io_account_write(bio->bi_iter.bi_size);
 		} else {
 			bio_set_op_attrs(bio, REQ_OP_READ, 0);
Signature · 1 line
2.7.4
#11 lore raw

Re: [PATCH 4/9] fs: add O_DIRECT support for sending down write life time hints

Quoted 14 lines
fs/direct-io.c +2 -0
@@ -385,6 +385,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio,
 	else
 		bio->bi_end_io = dio_bio_end_io;
 
+	bio->bi_opf |= write_hint_to_opf(iocb_write_hint(dio->iocb));
+
 	sdio->bio = bio;
 	sdio->logical_offset_in_bio = sdio->cur_page_fs_offset;
 }
fs/iomap.c +0 -0
index c71a64b97fba..9c9f8406018b 100644

fs/iomap.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 807

Quoted 4 lines
fs/iomap.c +1 -0
@@ -804,6 +804,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 
 		if (dio->flags & IOMAP_DIO_WRITE) {
 			bio_set_op_attrs(bio, REQ_OP_WRITE, REQ_SYNC | REQ_IDLE);
+			bio->bi_opf |= write_hint_to_opf(inode_write_hint(inode));

This should be iocb based as well, otherwise you'll miss out on the per-file hints.

Also please don't add a > 80 char line. (And yes, I should fix the one just above, too)

Quoted 6 lines
fs/iomap.c +0 -0
 			task_io_account_write(bio->bi_iter.bi_size);
 		} else {
 			bio_set_op_attrs(bio, REQ_OP_READ, 0);
Signature · 1 line
2.7.4

---end quoted text---

author maintainer
#12 lore raw

Re: [PATCH 4/9] fs: add O_DIRECT support for sending down write life time hints

Quoted 15 lines · Christoph Hellwig
Quoted 15 lines
fs/direct-io.c +2 -0
@@ -385,6 +385,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio,
 	else
 		bio->bi_end_io = dio_bio_end_io;
 
+	bio->bi_opf |= write_hint_to_opf(iocb_write_hint(dio->iocb));
+
 	sdio->bio = bio;
 	sdio->logical_offset_in_bio = sdio->cur_page_fs_offset;
 }
fs/iomap.c +0 -0
index c71a64b97fba..9c9f8406018b 100644
@@ -804,6 +804,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 
Quoted 6 lines

fs/iomap.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 807

Quoted 3 lines
fs/iomap.c +1 -0
 		if (dio->flags & IOMAP_DIO_WRITE) {
 			bio_set_op_attrs(bio, REQ_OP_WRITE, REQ_SYNC | REQ_IDLE);
+			bio->bi_opf |= write_hint_to_opf(inode_write_hint(inode));

This should be iocb based as well, otherwise you'll miss out on the per-file hints.

Good catch, fixed up.

Quoted 2 lines

Also please don't add a > 80 char line. (And yes, I should fix the one just above, too)

I fixed up them both...

Signature · 1 line
Jens Axboe
author maintainer
#13 lore raw

PATCH 5/9 fs: add support for buffered writeback to pass down write hints

In reply to Jens Axboe
---
 fs/buffer.c | 14 +++++++++-----
 fs/mpage.c  |  1 +
 2 files changed, 10 insertions(+), 5 deletions(-)
fs/buffer.c +9 -5
index 306b720f7383..1259524715c8 100644
@@ -49,7 +49,7 @@
 
 static int fsync_buffers_list(spinlock_t *lock, struct list_head *list);
 static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
-			 struct writeback_control *wbc);
+			 unsigned int stream, struct writeback_control *wbc);
 
 #define BH_ENTRY(list) list_entry((list), struct buffer_head, b_assoc_buffers)
 
@@ -1829,7 +1829,8 @@ int __block_write_full_page(struct inode *inode, struct page *page,
 	do {
 		struct buffer_head *next = bh->b_this_page;
 		if (buffer_async_write(bh)) {
-			submit_bh_wbc(REQ_OP_WRITE, write_flags, bh, wbc);
+			submit_bh_wbc(REQ_OP_WRITE, write_flags, bh,
+					inode_write_hint(inode), wbc);
 			nr_underway++;
 		}
 		bh = next;
@@ -1883,7 +1884,8 @@ int __block_write_full_page(struct inode *inode, struct page *page,
 		struct buffer_head *next = bh->b_this_page;
 		if (buffer_async_write(bh)) {
 			clear_buffer_dirty(bh);
-			submit_bh_wbc(REQ_OP_WRITE, write_flags, bh, wbc);
+			submit_bh_wbc(REQ_OP_WRITE, write_flags, bh,
+					inode_write_hint(inode), wbc);
 			nr_underway++;
 		}
 		bh = next;
@@ -3091,7 +3093,7 @@ void guard_bio_eod(int op, struct bio *bio)
 }
 
 static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
-			 struct writeback_control *wbc)
+			 unsigned int write_hint, struct writeback_control *wbc)
 {
 	struct bio *bio;
 
@@ -3134,6 +3136,8 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
 		op_flags |= REQ_META;
 	if (buffer_prio(bh))
 		op_flags |= REQ_PRIO;
+
+	op_flags |= write_hint_to_opf(write_hint);
 	bio_set_op_attrs(bio, op, op_flags);
 
 	submit_bio(bio);
@@ -3142,7 +3146,7 @@ static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh,
 
 int submit_bh(int op, int op_flags, struct buffer_head *bh)
 {
-	return submit_bh_wbc(op, op_flags, bh, NULL);
+	return submit_bh_wbc(op, op_flags, bh, 0, NULL);
 }
 EXPORT_SYMBOL(submit_bh);
 
fs/mpage.c +1 -0
index 9524fdde00c2..d8a750873bf4 100644
@@ -615,6 +615,7 @@ static int __mpage_writepage(struct page *page, struct writeback_control *wbc,
 			goto confused;
 
 		wbc_init_bio(wbc, bio);
+		bio->bi_opf |= write_hint_to_opf(inode_write_hint(inode));
 	}
 
 	/*
Signature · 1 line
2.7.4
author maintainer
#14 lore raw

PATCH 6/9 ext4: add support for passing in write hints for buffered writes

In reply to Jens Axboe
---
 fs/ext4/page-io.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ext4/page-io.c +2 -0
index 930ca0fc9a0f..92834b702728 100644
@@ -350,6 +350,7 @@ void ext4_io_submit(struct ext4_io_submit *io)
 	if (bio) {
 		int io_op_flags = io->io_wbc->sync_mode == WB_SYNC_ALL ?
 				  REQ_SYNC : 0;
+		io_op_flags |= write_hint_to_opf(inode_write_hint(io->io_end->inode));
 		bio_set_op_attrs(io->io_bio, REQ_OP_WRITE, io_op_flags);
 		submit_bio(io->io_bio);
 	}
@@ -397,6 +398,7 @@ static int io_submit_add_bh(struct ext4_io_submit *io,
 		ret = io_submit_init_bio(io, bh);
 		if (ret)
 			return ret;
+		io->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(inode));
 	}
 	ret = bio_add_page(io->io_bio, page, bh->b_size, bh_offset(bh));
 	if (ret != bh->b_size)
Signature · 1 line
2.7.4
author maintainer
#15 lore raw

PATCH 7/9 xfs: add support for passing in write hints for buffered writes

In reply to Jens Axboe
---
 fs/xfs/xfs_aops.c | 2 ++
 1 file changed, 2 insertions(+)
fs/xfs/xfs_aops.c +2 -0
index 76b6f988e2fa..e4d9d470402c 100644
@@ -506,6 +506,7 @@ xfs_submit_ioend(
 		return status;
 	}
 
+	ioend->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(ioend->io_inode));
 	submit_bio(ioend->io_bio);
 	return 0;
 }
@@ -565,6 +566,7 @@ xfs_chain_bio(
 	bio_chain(ioend->io_bio, new);
 	bio_get(ioend->io_bio);		/* for xfs_destroy_ioend */
 	ioend->io_bio->bi_opf = REQ_OP_WRITE | wbc_to_write_flags(wbc);
+	ioend->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(ioend->io_inode));
 	submit_bio(ioend->io_bio);
 	ioend->io_bio = new;
 }
Signature · 1 line
2.7.4
Reviewed-by on v1 7/9
#16 lore raw

Re: [PATCH 7/9] xfs: add support for passing in write hints for buffered writes

Quoted 2 lines · Jens Axboe

Looks ok to me,

--D

Quoted 27 lines
---
 fs/xfs/xfs_aops.c | 2 ++
 1 file changed, 2 insertions(+)
fs/xfs/xfs_aops.c +2 -0
index 76b6f988e2fa..e4d9d470402c 100644
@@ -506,6 +506,7 @@ xfs_submit_ioend(
 		return status;
 	}
 
+	ioend->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(ioend->io_inode));
 	submit_bio(ioend->io_bio);
 	return 0;
 }
@@ -565,6 +566,7 @@ xfs_chain_bio(
 	bio_chain(ioend->io_bio, new);
 	bio_get(ioend->io_bio);		/* for xfs_destroy_ioend */
 	ioend->io_bio->bi_opf = REQ_OP_WRITE | wbc_to_write_flags(wbc);
+	ioend->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(ioend->io_inode));
 	submit_bio(ioend->io_bio);
 	ioend->io_bio = new;
 }
Signature · 1 line
2.7.4
#18 lore raw

Re: [PATCH 7/9] xfs: add support for passing in write hints for buffered writes

In reply to Jens Axboe

Review

Quoted 1 line
(quoted diff) +1 -0
+	ioend->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(ioend->io_inode));

Too long line again.

I think simply adding an inode_hint_to_opf() flag would make the various opencoded versions of the above call a little easier to read.

author maintainer
#19 lore raw

Re: [PATCH 7/9] xfs: add support for passing in write hints for buffered writes

Quoted 6 lines · Christoph Hellwig

Review

Quoted 1 line
(quoted diff) +1 -0
+	ioend->io_bio->bi_opf |= write_hint_to_opf(inode_write_hint(ioend->io_inode));

Too long line again.

I think simply adding an inode_hint_to_opf() flag would make the various opencoded versions of the above call a little easier to read.

Sure, I'll do that.

Signature · 1 line
Jens Axboe
author maintainer
#20 lore raw

PATCH 8/9 btrfs: add support for passing in write hints for buffered writes

In reply to Jens Axboe
---
 fs/btrfs/extent_io.c | 1 +
 1 file changed, 1 insertion(+)
fs/btrfs/extent_io.c +1 -0
index 19eedf2e630b..3e57cfaa6dd6 100644
@@ -2830,6 +2830,7 @@ static int submit_extent_page(int op, int op_flags, struct extent_io_tree *tree,
 	bio_add_page(bio, page, page_size, offset);
 	bio->bi_end_io = end_io_func;
 	bio->bi_private = tree;
+	op_flags |= write_hint_to_opf(inode_write_hint(page->mapping->host));
 	bio_set_op_attrs(bio, op, op_flags);
 	if (wbc) {
 		wbc_init_bio(wbc, bio);
Signature · 1 line
2.7.4
author maintainer
#21 lore raw

PATCH 9/9 nvme: add support for streams and directives

In reply to Jens Axboe

This adds support for Directives in NVMe, particular for the Streams directive. Support for Directives is a new feature in NVMe 1.3. It allows a user to pass in information about where to store the data, so that it the device can do so most effiently. If an application is managing and writing data with different life times, mixing differently retentioned data onto the same locations on flash can cause write amplification to grow. This, in turn, will reduce performance and life time of the device.

---
 drivers/nvme/host/core.c | 142 +++++++++++++++++++++++++++++++++++++++++++++--
 drivers/nvme/host/nvme.h |   4 ++
 include/linux/nvme.h     |  48 ++++++++++++++++
 3 files changed, 190 insertions(+), 4 deletions(-)
drivers/nvme/host/core.c +138 -4
index aee37b73231d..fcccc1534f7b 100644
@@ -297,6 +297,100 @@ struct request *nvme_alloc_request(struct request_queue *q,
 }
 EXPORT_SYMBOL_GPL(nvme_alloc_request);
 
+static int nvme_enable_streams(struct nvme_ctrl *ctrl)
+{
+	struct nvme_command c;
+
+	memset(&c, 0, sizeof(c));
+
+	c.directive.opcode = nvme_admin_directive_send;
+	c.directive.nsid = cpu_to_le32(0xffffffff);
+	c.directive.doper = NVME_DIR_SND_ID_OP_ENABLE;
+	c.directive.dtype = NVME_DIR_IDENTIFY;
+	c.directive.tdtype = NVME_DIR_STREAMS;
+	c.directive.endir = NVME_DIR_ENDIR;
+
+	return nvme_submit_sync_cmd(ctrl->admin_q, &c, NULL, 0);
+}
+
+static int nvme_get_stream_params(struct nvme_ctrl *ctrl,
+				  struct streams_directive_params *s, u32 nsid)
+{
+	struct nvme_command c;
+
+	memset(&c, 0, sizeof(c));
+	memset(s, 0, sizeof(*s));
+
+	c.directive.opcode = nvme_admin_directive_recv;
+	c.directive.nsid = cpu_to_le32(nsid);
+	c.directive.numd = sizeof(*s);
+	c.directive.doper = NVME_DIR_RCV_ST_OP_PARAM;
+	c.directive.dtype = NVME_DIR_STREAMS;
+
+	return nvme_submit_sync_cmd(ctrl->admin_q, &c, s, sizeof(*s));
+}
+
+static int nvme_configure_directives(struct nvme_ctrl *ctrl)
+{
+	struct streams_directive_params s;
+	int ret;
+
+	if (!(ctrl->oacs & NVME_CTRL_OACS_DIRECTIVES))
+		return 0;
+
+	ret = nvme_enable_streams(ctrl);
+	if (ret)
+		return ret;
+
+	ret = nvme_get_stream_params(ctrl, &s, 0xffffffff);
+	if (ret)
+		return ret;
+
+	ctrl->nssa = le16_to_cpu(s.nssa);
+	ctrl->nr_streams = min_t(unsigned, ctrl->nssa, BLK_MAX_WRITE_HINTS - 1);
+	return 0;
+}
+
+/*
+ * Write hint number to stream mappings
+ */
+static const unsigned int stream_mappings[BLK_MAX_WRITE_HINTS][BLK_MAX_WRITE_HINTS] = {
+	/* 0 or 1 stream, we don't use streams */
+	{ 0, },
+	{ 0, },
+	/* collapse short+medium to short, and long+extreme to medium */
+	{ WRITE_LIFE_NONE, WRITE_LIFE_SHORT, WRITE_LIFE_SHORT,
+		WRITE_LIFE_MEDIUM, WRITE_LIFE_MEDIUM },
+	/* collapse long+extreme to long */
+	{ WRITE_LIFE_NONE, WRITE_LIFE_SHORT, WRITE_LIFE_MEDIUM,
+		WRITE_LIFE_LONG, WRITE_LIFE_LONG },
+	/* 4 streams, no collapsing needed */
+	{ WRITE_LIFE_NONE, WRITE_LIFE_SHORT, WRITE_LIFE_MEDIUM,
+		WRITE_LIFE_LONG, WRITE_LIFE_EXTREME },
+};
+
+/*
+ * Check if 'req' has a write hint associated with it. If it does, assign
+ * a valid namespace stream to the write. If we haven't setup streams yet,
+ * kick off configuration and ignore the hints until that has completed.
+ */
+static void nvme_assign_write_stream(struct nvme_ctrl *ctrl,
+				     struct request *req, u16 *control,
+				     u32 *dsmgmt)
+{
+	enum rw_hint streamid;
+
+	streamid = opf_to_write_hint(req->cmd_flags);
+	if (streamid != WRITE_LIFE_NONE) {
+		streamid = stream_mappings[ctrl->nr_streams][streamid - 1];
+		*control |= NVME_RW_DTYPE_STREAMS;
+		*dsmgmt |= streamid << 16;
+	}
+
+	if (streamid < ARRAY_SIZE(req->q->write_hints))
+		req->q->write_hints[streamid] += blk_rq_bytes(req) >> 9;
+}
+
 static inline void nvme_setup_flush(struct nvme_ns *ns,
 		struct nvme_command *cmnd)
 {
@@ -348,6 +442,7 @@ static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req,
 static inline blk_status_t nvme_setup_rw(struct nvme_ns *ns,
 		struct request *req, struct nvme_command *cmnd)
 {
+	struct nvme_ctrl *ctrl = ns->ctrl;
 	u16 control = 0;
 	u32 dsmgmt = 0;
 
@@ -375,6 +470,9 @@ static inline blk_status_t nvme_setup_rw(struct nvme_ns *ns,
 	cmnd->rw.slba = cpu_to_le64(nvme_block_nr(ns, blk_rq_pos(req)));
 	cmnd->rw.length = cpu_to_le16((blk_rq_bytes(req) >> ns->lba_shift) - 1);
 
+	if (req_op(req) == REQ_OP_WRITE && ctrl->nr_streams)
+		nvme_assign_write_stream(ctrl, req, &control, &dsmgmt);
+
 	if (ns->ms) {
 		switch (ns->pi_type) {
 		case NVME_NS_DPS_PI_TYPE3:
@@ -1094,8 +1192,15 @@ static void nvme_config_discard(struct nvme_ns *ns)
 	BUILD_BUG_ON(PAGE_SIZE / sizeof(struct nvme_dsm_range) <
 			NVME_DSM_MAX_RANGES);
 
-	ns->queue->limits.discard_alignment = logical_block_size;
-	ns->queue->limits.discard_granularity = logical_block_size;
+	if (ctrl->nr_streams && ns->sws && ns->sgs) {
+		unsigned int sz = logical_block_size * ns->sws * ns->sgs;
+
+		ns->queue->limits.discard_alignment = sz;
+		ns->queue->limits.discard_granularity = sz;
+	} else {
+		ns->queue->limits.discard_alignment = logical_block_size;
+		ns->queue->limits.discard_granularity = logical_block_size;
+	}
 	blk_queue_max_discard_sectors(ns->queue, UINT_MAX);
 	blk_queue_max_discard_segments(ns->queue, NVME_DSM_MAX_RANGES);
 	queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, ns->queue);
@@ -1135,6 +1240,7 @@ static int nvme_revalidate_ns(struct nvme_ns *ns, struct nvme_id_ns **id)
 static void __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id)
 {
 	struct nvme_ns *ns = disk->private_data;
+	struct nvme_ctrl *ctrl = ns->ctrl;
 	u16 bs;
 
 	/*
@@ -1149,7 +1255,7 @@ static void __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id)
 
 	blk_mq_freeze_queue(disk->queue);
 
-	if (ns->ctrl->ops->flags & NVME_F_METADATA_SUPPORTED)
+	if (ctrl->ops->flags & NVME_F_METADATA_SUPPORTED)
 		nvme_prep_integrity(disk, id, bs);
 	blk_queue_logical_block_size(ns->queue, bs);
 	if (ns->noiob)
@@ -1161,7 +1267,7 @@ static void __nvme_revalidate_disk(struct gendisk *disk, struct nvme_id_ns *id)
 	else
 		set_capacity(disk, le64_to_cpup(&id->nsze) << (ns->lba_shift - 9));
 
-	if (ns->ctrl->oncs & NVME_CTRL_ONCS_DSM)
+	if (ctrl->oncs & NVME_CTRL_ONCS_DSM)
 		nvme_config_discard(ns);
 	blk_mq_unfreeze_queue(disk->queue);
 }
@@ -1766,6 +1872,7 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 		dev_pm_qos_hide_latency_tolerance(ctrl->device);
 
 	nvme_configure_apst(ctrl);
+	nvme_configure_directives(ctrl);
 
 	ctrl->identified = true;
 
@@ -2158,6 +2265,32 @@ static struct nvme_ns *nvme_find_get_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 	return ret;
 }
 
+static int nvme_setup_streams_ns(struct nvme_ctrl *ctrl, struct nvme_ns *ns)
+{
+	struct streams_directive_params s;
+	int ret;
+
+	if (!ctrl->nr_streams)
+		return 0;
+
+	ret = nvme_get_stream_params(ctrl, &s, ns->ns_id);
+	if (ret)
+		return ret;
+
+	ns->sws = le32_to_cpu(s.sws);
+	ns->sgs = le16_to_cpu(s.sgs);
+
+	if (ns->sws) {
+		unsigned int bs = 1 << ns->lba_shift;
+
+		blk_queue_io_min(ns->queue, bs * ns->sws);
+		if (ns->sgs)
+			blk_queue_io_opt(ns->queue, bs * ns->sws * ns->sgs);
+	}
+
+	return 0;
+}
+
 static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 {
 	struct nvme_ns *ns;
@@ -2187,6 +2320,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 
 	blk_queue_logical_block_size(ns->queue, 1 << ns->lba_shift);
 	nvme_set_queue_limits(ctrl, ns->queue);
+	nvme_setup_streams_ns(ctrl, ns);
 
 	sprintf(disk_name, "nvme%dn%d", ctrl->instance, ns->instance);
 
drivers/nvme/host/nvme.h +4 -0
index ec8c7363934d..f616835afc4c 100644
@@ -147,6 +147,8 @@ struct nvme_ctrl {
 	u16 oncs;
 	u16 vid;
 	u16 oacs;
+	u16 nssa;
+	u16 nr_streams;
 	atomic_t abort_limit;
 	u8 event_limit;
 	u8 vwc;
@@ -199,6 +201,8 @@ struct nvme_ns {
 	unsigned ns_id;
 	int lba_shift;
 	u16 ms;
+	u16 sgs;
+	u32 sws;
 	bool ext;
 	u8 pi_type;
 	unsigned long flags;
include/linux/nvme.h +48 -0

90 lines not on this page · raw message

Signature · 1 line
2.7.4
#22 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Looks mostly good,

but two nit-picks:

- can we keep a module option to disable streams, or in fact for now maybe to explicitly enable it? I expect this to be interesting at least for the first devices that implement it. Also given that it needs to be explicitly enabled I would expect some overhead of just enabling it when never used
- do we even need the < 4 streams fallback now that they are global instead of per-ns instead of just disabling the feature for now?

author maintainer
#23 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 9 lines · Christoph Hellwig

Looks mostly good,

but two nit-picks:

- can we keep a module option to disable streams, or in fact for now maybe to explicitly enable it? I expect this to be interesting at least for the first devices that implement it. Also given that it needs to be explicitly enabled I would expect some overhead of just enabling it when never used

Fine with me, I can add the 'streams' parameter back, but just default it to false.

Quoted 2 lines

- do we even need the < 4 streams fallback now that they are global instead of per-ns instead of just disabling the feature for now?

Maybe the device only supports 2? or 3?

Signature · 1 line
Jens Axboe
#24 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 6 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig

Looks mostly good,

but two nit-picks:

Quoted 8 lines
Quoted 5 lines

- can we keep a module option to disable streams, or in fact for now maybe to explicitly enable it? I expect this to be interesting at least for the first devices that implement it. Also given that it needs to be explicitly enabled I would expect some overhead of just enabling it when never used

Fine with me, I can add the 'streams' parameter back, but just default it to false.

Better would be a parameter to set the default streams count, 0 by default.

Quoted 8 lines
Quoted 2 lines

- do we even need the < 4 streams fallback now that they are global instead of per-ns instead of just disabling the feature for now?

Maybe the device only supports 2? or 3?

--
Jens Axboe

Cheers, Andreas

author maintainer
#25 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 13 lines · Andreas Dilger
Quoted 12 lines · Jens Axboe
Quoted 9 lines · Christoph Hellwig

Looks mostly good,

but two nit-picks:

- can we keep a module option to disable streams, or in fact for now maybe to explicitly enable it? I expect this to be interesting at least for the first devices that implement it. Also given that it needs to be explicitly enabled I would expect some overhead of just enabling it when never used

Quoted 4 lines
Quoted 2 lines

Fine with me, I can add the 'streams' parameter back, but just default it to false.

Better would be a parameter to set the default streams count, 0 by default.

The user should not need to know. If streams is enabled (bool), then it'll ask for as many as we need on the block side right now, and scale down if we have to. So I'd rather keep it as a "use streams or not" bool on the nvme side.

Signature · 1 line
Jens Axboe
#26 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

In reply to Jens Axboe
Quoted 4 lines · Jens Axboe
Quoted 2 lines

- do we even need the < 4 streams fallback now that they are global instead of per-ns instead of just disabling the feature for now?

Maybe the device only supports 2? or 3?

My crystal ball indicates that those are unlikely too see the light. IFF we need to handle them we can still add code for it.

author maintainer
#27 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Jens Axboe
Quoted 2 lines

- do we even need the < 4 streams fallback now that they are global instead of per-ns instead of just disabling the feature for now?

Maybe the device only supports 2? or 3?

My crystal ball indicates that those are unlikely too see the light. IFF we need to handle them we can still add code for it.

But we have to handle it, not doing so would be fragile. So our options are:

1) Keep the stream_mappings[] array. It's simple, and it'll work for any number of streams.

2) Kill stream_mappings[] and just do the MOD again.

I'd strongly lean towards #1. I don't have a lot of faith in crystal balls.

Signature · 1 line
Jens Axboe
#28 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 7 lines · Jens Axboe

But we have to handle it, not doing so would be fragile. So our options are:

1) Keep the stream_mappings[] array. It's simple, and it'll work for any number of streams.

2) Kill stream_mappings[] and just do the MOD again.

3) print a message and tell streams aren't supported on this device.

author maintainer
#29 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 10 lines · Christoph Hellwig
Quoted 7 lines · Jens Axboe

But we have to handle it, not doing so would be fragile. So our options are:

1) Keep the stream_mappings[] array. It's simple, and it'll work for any number of streams.

2) Kill stream_mappings[] and just do the MOD again.

3) print a message and tell streams aren't supported on this device.

Is that your nvme preference - if less than 4 streams, just ignore it? Would seem a shame to lose out with 2 streams, I can think of several hot/cold scenarios that would probably work fine with that. But at the same time, not a big deal to me, if you prefer just turning it off for < 4 streams, that's fine with me.

Signature · 1 line
Jens Axboe
author maintainer
#30 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 10 lines · Jens Axboe
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Jens Axboe

But we have to handle it, not doing so would be fragile. So our options are:

1) Keep the stream_mappings[] array. It's simple, and it'll work for any number of streams.

2) Kill stream_mappings[] and just do the MOD again.

Quoted 7 lines
Quoted 1 line

3) print a message and tell streams aren't supported on this device.

Is that your nvme preference - if less than 4 streams, just ignore it? Would seem a shame to lose out with 2 streams, I can think of several hot/cold scenarios that would probably work fine with that. But at the same time, not a big deal to me, if you prefer just turning it off for < 4 streams, that's fine with me.

http://git.kernel.dk/cgit/linux-block/commit/?h=write-stream&id=98335ae1347a9a08adc831e77e8fefcf06ab8282

Signature · 1 line
Jens Axboe
#31 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

In reply to Christoph Hellwig

Christoph,

Quoted 5 lines

- can we keep a module option to disable streams, or in fact for now maybe to explicitly enable it? I expect this to be interesting at least for the first devices that implement it. Also given that it needs to be explicitly enabled I would expect some overhead of just enabling it when never used

Yeah, based on my experiments we'll need to drive this as an opt-in feature for now. Short term the module option is OK. Once more devices start materializing we probably need a white/blacklist/quirk scheme.

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
author maintainer
#32 lore raw

Re: [PATCH 9/9] nvme: add support for streams and directives

Quoted 9 lines · Martin K. Petersen

Christoph,

Quoted 5 lines

- can we keep a module option to disable streams, or in fact for now maybe to explicitly enable it? I expect this to be interesting at least for the first devices that implement it. Also given that it needs to be explicitly enabled I would expect some overhead of just enabling it when never used

Quoted 3 lines

Yeah, based on my experiments we'll need to drive this as an opt-in feature for now. Short term the module option is OK. Once more devices start materializing we probably need a white/blacklist/quirk scheme.

Completely agree. Might even need quirks for stream allocations too, for instance. But let's hope we can keep it clean.

Signature · 1 line
Jens Axboe
Reviewed-by on v9 1/1
#33 lore raw
In reply to Jens Axboe

Jens,

Quoted 4 lines

A new iteration of this patchset, previously known as write streams. As before, this patchset aims at enabling applications split up writes into separate streams, based on the perceived life time of the data written. This is useful for a variety of reasons:

The grep-wielding streams police noticed several occurrences of the string "stream" in patches 1-3 and 5. Please fix.

Otherwise OK as a baseline.

Signature · 1 line
Martin K. Petersen	Oracle Linux Engineering
author maintainer
#34 lore raw
Quoted 10 lines · Martin K. Petersen

Jens,

Quoted 4 lines

A new iteration of this patchset, previously known as write streams. As before, this patchset aims at enabling applications split up writes into separate streams, based on the perceived life time of the data written. This is useful for a variety of reasons:

The grep-wielding streams police noticed several occurrences of the string "stream" in patches 1-3 and 5. Please fix.

You really hate those streams :-)

I've pruned all references to it, from comments and arguments.

Quoted 3 lines

Otherwise OK as a baseline.

Thanks!

Signature · 1 line
Jens Axboe