Miscellaneous changes for nvme-tcp v2
v2 cover Miscellaneous changes for nvme-tcp v2
A few unrelated changes for NVMoF subsystem, that was found during testing of NVMoF TCP.
patch 2: found during extensive testing on various filesystems patch 3: found during testing on debug kernels with VM_DEBUG enabled
v2
- Moved hint patch as separate patch;
- Fix commit description in stable writes patch;
- Simplify check for SLAB check;
- Remove unnecessary replacments of sendpage calls;
Mike Playle (1):
nvme-tcp: set the STABLE_WRITES flag when data digests are enabled
Mikhail Skorzhinskii (1):
nvme-tcp: don't use sendpage for SLAB pages
drivers/nvme/host/core.c | 5 +++++ drivers/nvme/host/tcp.c | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletion(-)
Signature · 1 line
2.16.4
PATCH v2 1/2 nvme-tcp: set the STABLE_WRITES flag when data digests are enabled
There was a few false alarms sighted on target side about wrong data digest while performing high throughput load to XFS filesystem shared through NVMoF TCP.
This flag tells the rest of the kernel to ensure that the data buffer does not change while the write is in flight. It incurs a performance penalty, so only enable it when it is actually needed, i.e. when we are calculating data digests.
Although even with this change in place, ext2 users can steel experience false positives, as ext2 is not respecting this flag. This may be apply to vfat as well.
---
https://lwn.net/Articles/528031/
https://www.redhat.com/archives/dm-devel/2015-May/msg00158.html
--- drivers/nvme/host/core.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/nvme/host/core.c +5 -0
@@ -11,6 +11,7 @@#include <linux/hdreg.h> #include <linux/kernel.h> #include <linux/module.h> +#include <linux/backing-dev.h> #include <linux/list_sort.h> #include <linux/slab.h> #include <linux/types.h>
@@ -3304,6 +3305,10 @@ static int nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid) goto out_free_ns; } +if (ctrl->opts->data_digest) + ns->queue->backing_dev_info->capabilities + |=BDI_CAP_STABLE_WRITES ; +blk_queue_flag_set (QUEUE_FLAG_NONROT , ns->queue);if (ctrl->ops->flags &NVME_F_PCI_P2PDMA )blk_queue_flag_set (QUEUE_FLAG_PCI_P2PDMA , ns->queue);
Signature · 1 line
2.16.4
Sagi Grimberg
maintainerPATCH v2 1/2 nvme-tcp: set the STABLE_WRITES flag when data digests are enabled
PATCH v2 1/2 nvme-tcp: set the STABLE_WRITES flag when data digests are enabled
In reply to Mikhail SkorzhinskiiThanks,
applied to nvme-5.3.
PATCH v2 2/2 nvme-tcp: don't use sendpage for SLAB pages
In reply to Mikhail SkorzhinskiiAccording to commit a10674bf2406 ("tcp: detecting the misuse of .sendpage for Slab objects") and previous discussion, tcp_sendpage should not be used for pages that is managed by SLAB, as SLAB is not taking page reference counters into consideration.
This change prevents sendpage calls for payload sending too, although this is true only for admin commands, so actual data transfer performance should be untouched.
--- drivers/nvme/host/tcp.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/nvme/host/tcp.c +6 -1
@@ -860,7 +860,12 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)else flags |=MSG_MORE ; - ret =kernel_sendpage (queue->sock, page, offset, len, flags); +/* can't zcopy slab pages */ +if (unlikely (PageSlab (page))) { + ret =sock_no_sendpage (queue->sock, page, offset, len, flags); + }else { + ret =kernel_sendpage (queue->sock, page, offset, len, flags); + }if (ret <=0 )return ret;
Signature · 1 line
2.16.4
Sagi Grimberg
maintainerPATCH v2 2/2 nvme-tcp: don't use sendpage for SLAB pages
Christoph Hellwig
maintainerPATCH v2 2/2 nvme-tcp: don't use sendpage for SLAB pages
In reply to Mikhail SkorzhinskiiQuoted 3 lines · Mikhail Skorzhinskii
This change prevents sendpage calls for payload sending too, although this is true only for admin commands, so actual data transfer performance should be untouched.
XFS can also send slab pages for data.
I've applied to patch to nvme-5.3, but removed the above sentence from the commit log.