Using SGLs for userspace commands
v2 cover Using SGLs for userspace commands
From: Keith Busch <[email protected]>
Changes from previous version:
Merged up to linux 6.12, which had some necessary blk-integrity patches
Dropped all the initial "cleanup" patches. This is instead the minimum patchset to satisfy using the feature.
Fixed setting the queue limit
Fixed the SGLS MPTR bit
Keith Busch (2):
nvme-pci: add support for sgl metadata
nvme-pci: use sgls for all user requests if possible
drivers/nvme/host/core.c | 3 + drivers/nvme/host/nvme.h | 5 ++ drivers/nvme/host/pci.c | 139 ++++++++++++++++++++++++++++++++++----- include/linux/nvme.h | 1 + 4 files changed, 131 insertions(+), 17 deletions(-)
Signature · 1 line
2.43.5
PATCH v2 1/2 nvme-pci: add support for sgl metadata
From: Keith Busch <[email protected]>
Supporting this mode allows merging requests with metadata that wouldn't be possible otherwise, and creating user space requests that straddle physically discontiguous pages.
--- drivers/nvme/host/core.c | 3 + drivers/nvme/host/nvme.h | 5 ++ drivers/nvme/host/pci.c | 136 ++++++++++++++++++++++++++++++++++----- include/linux/nvme.h | 1 + 4 files changed, 129 insertions(+), 16 deletions(-)
drivers/nvme/host/core.c +3 -0
@@ -1985,6 +1985,9 @@ static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl, lim->max_segments =min_t (u32,USHRT_MAX ,min_not_zero (nvme_max_drv_segments (ctrl), ctrl->max_segments)); lim->max_integrity_segments = ctrl->max_integrity_segments; +if (lim->max_integrity_segments >1 && + !nvme_ctrl_meta_sgl_supported (ctrl)) + lim->max_integrity_segments =1 ; lim->virt_boundary_mask =NVME_CTRL_PAGE_SIZE -1 ; lim->max_segment_size =UINT_MAX ; lim->dma_alignment =3 ;
drivers/nvme/host/nvme.h +5 -0
@@ -1126,6 +1126,11 @@ static inline bool nvme_ctrl_sgl_supported(struct nvme_ctrl *ctrl)return ctrl->sgls & ((1 <<0 ) | (1 <<1 )); } +static inline bool nvme_ctrl_meta_sgl_supported (struct nvme_ctrl *ctrl) +{ +return ctrl->sgls &NVME_CTRL_SGLS_MPTR ; +} +#ifdef CONFIG_NVME_HOST_AUTH int __init nvme_init_auth (void ); void __exit nvme_exit_auth(void);
drivers/nvme/host/pci.c +120 -16
241 lines not on this page · raw message
include/linux/nvme.h +1 -0
@@ -389,6 +389,7 @@ enum {NVME_CTRL_CTRATT_PREDICTABLE_LAT =1 <<5 ,NVME_CTRL_CTRATT_NAMESPACE_GRANULARITY =1 <<7 ,NVME_CTRL_CTRATT_UUID_LIST =1 <<9 , + NVME_CTRL_SGLS_MPTR = 1 << 19, }; struct nvme_lbaf {
Signature · 1 line
2.43.5
Christoph Hellwig
maintainerRe: [PATCHv2 1/2] nvme-pci: add support for sgl metadata
Quoted 3 lines · Keith Busch
Supporting this mode allows merging requests with metadata that wouldn't be possible otherwise, and creating user space requests that straddle physically discontiguous pages.
Not just merging, but also creating :) I.e. it allows to transfer multiple non-contiguous metadata segements.
drivers/nvme/host/core.c static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl, · line 1990
Quoted 12 lines
drivers/nvme/host/core.c +3
-0@@ -1985,6 +1985,9 @@ static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl, lim->max_segments =min_t (u32,USHRT_MAX ,min_not_zero (nvme_max_drv_segments (ctrl), ctrl->max_segments)); lim->max_integrity_segments = ctrl->max_integrity_segments; +if (lim->max_integrity_segments >1 && + !nvme_ctrl_meta_sgl_supported (ctrl)) + lim->max_integrity_segments =1 ;
Despite the general mess about the SGLS field in the nvme spec, the meta sgls bit really is a PCIe-only feasture, and this will break metadata support on RDMA. The nvme_ctrl_meta_sgl_supported needs to be in pci.c to set ctrl->max_integrity_segments based on it.
drivers/nvme/host/core.c static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl,
Quoted 4 lines
drivers/nvme/host/core.c +4
-0+static inline bool nvme_ctrl_meta_sgl_supported (struct nvme_ctrl *ctrl) +{ +return ctrl->sgls &NVME_CTRL_SGLS_MPTR ; +}
.. and thus I'd move this to pci.c (or just drop the helper).
drivers/nvme/host/core.c
Quoted 1 line
drivers/nvme/host/core.c +1
-0+ NVME_CTRL_SGLS_MPTR = 1 << 19,
Maybe give the other fields in SGLS a name as well?
Keith Busch
maintainerRe: [PATCHv2 1/2] nvme-pci: add support for sgl metadata
Quoted 33 lines · Christoph Hellwig
Quoted 3 lines · Keith Busch
Supporting this mode allows merging requests with metadata that wouldn't be possible otherwise, and creating user space requests that straddle physically discontiguous pages.
Not just merging, but also creating :) I.e. it allows to transfer multiple non-contiguous metadata segements.
drivers/nvme/host/core.c static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl, · line 1990
Quoted 12 lines
drivers/nvme/host/core.c +3
-0@@ -1985,6 +1985,9 @@ static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl, lim->max_segments =min_t (u32,USHRT_MAX ,min_not_zero (nvme_max_drv_segments (ctrl), ctrl->max_segments)); lim->max_integrity_segments = ctrl->max_integrity_segments; +if (lim->max_integrity_segments >1 && + !nvme_ctrl_meta_sgl_supported (ctrl)) + lim->max_integrity_segments =1 ;Despite the general mess about the SGLS field in the nvme spec, the meta sgls bit really is a PCIe-only feasture, and this will break metadata support on RDMA. The
nvme_ctrl_meta_sgl_supportedneeds to be inpci.cto setctrl->max_integrity_segmentsbased on it.drivers/nvme/host/core.c static void nvme_set_ctrl_limits(struct nvme_ctrl *ctrl,
Quoted 4 lines
drivers/nvme/host/core.c +4
-0+static inline bool nvme_ctrl_meta_sgl_supported (struct nvme_ctrl *ctrl) +{ +return ctrl->sgls &NVME_CTRL_SGLS_MPTR ; +}.. and thus I'd move this to
pci.c(or just drop the helper).
Quoted 3 lines
drivers/nvme/host/core.c
Quoted 1 line
drivers/nvme/host/core.c +1
-0+ NVME_CTRL_SGLS_MPTR = 1 << 19,Maybe give the other fields in SGLS a name as well?
Ack on all suggestions.
PATCH v2 2/2 nvme-pci: use sgls for all user requests if possible
In reply to Keith BuschFrom: Keith Busch <[email protected]>
If the device supports SGLs, use these for all user requests. This format encodes the expected transfer length so it can catch short buffer errors in a user command, whether it occurred accidently or maliciously.
For controllers that support SGL data mode, this is a viable mitigation to CVE-2023-6238.
--- drivers/nvme/host/pci.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/nvme/host/pci.c +3 -2
@@ -515,7 +515,8 @@ static inline bool nvme_pci_metadata_use_sgls(struct nvme_dev *dev, {if (!nvme_ctrl_meta_sgl_supported (&dev->ctrl))return false; -return req->nr_integrity_segments >1 ; +return req->nr_integrity_segments >1 || +nvme_req (req)->flags &NVME_REQ_USERCMD ; } static inline bool nvme_pci_use_sgls(struct nvme_dev *dev, struct request *req,
@@ -533,7 +534,7 @@ static inline bool nvme_pci_use_sgls(struct nvme_dev *dev, struct request *req,if (nvme_pci_metadata_use_sgls (dev, req))return true;if (!sgl_threshold || avg_seg_size < sgl_threshold) -return false; +return nvme_req (req)->flags &NVME_REQ_USERCMD ;return true; }
Signature · 1 line
2.43.5
Christoph Hellwig
maintainerRe: [PATCHv2 2/2] nvme-pci: use sgls for all user requests if possible
Quoted 8 lines · Keith Busch
From: Keith Busch <[email protected]>
If the device supports SGLs, use these for all user requests. This format encodes the expected transfer length so it can catch short buffer errors in a user command, whether it occurred accidently or maliciously.
For controllers that support SGL data mode, this is a viable mitigation to CVE-2023-6238.
The patch itself looks fine, but instead of the handwaivy mitigation, maybe just disable passthrough without SGL support by default to actually fix and not just mitigate the CVE?
Keith Busch
maintainerRe: [PATCHv2 2/2] nvme-pci: use sgls for all user requests if possible
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Keith Busch
From: Keith Busch <[email protected]>
If the device supports SGLs, use these for all user requests. This format encodes the expected transfer length so it can catch short buffer errors in a user command, whether it occurred accidently or maliciously.
Quoted 6 lines
Quoted 2 lines
For controllers that support SGL data mode, this is a viable mitigation to CVE-2023-6238.
The patch itself looks fine, but instead of the handwaivy mitigation, maybe just disable passthrough without SGL support by default to actually fix and not just mitigate the CVE?
SGL is an optional feature that many devices don't implement. Even fewer do it for metadata. Disabling it entirely is "breaking userspace" for users I need to support.
Christoph Hellwig
maintainerRe: [PATCHv2 2/2] nvme-pci: use sgls for all user requests if possible
Quoted 10 lines · Keith Busch
Quoted 6 lines
Quoted 2 lines
For controllers that support SGL data mode, this is a viable mitigation to CVE-2023-6238.
The patch itself looks fine, but instead of the handwaivy mitigation, maybe just disable passthrough without SGL support by default to actually fix and not just mitigate the CVE?
SGL is an optional feature that many devices don't implement. Even fewer do it for metadata. Disabling it entirely is "breaking userspace" for users I need to support.
Well, if that usage creates exploitable behavior we'll need to fix it and not just paper over it. Although this probably only really matters for the non-privileged passthrough.
Keith Busch
maintainerRe: [PATCHv2 2/2] nvme-pci: use sgls for all user requests if possible
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Keith Busch
Quoted 6 lines
Quoted 2 lines
For controllers that support SGL data mode, this is a viable mitigation to CVE-2023-6238.
The patch itself looks fine, but instead of the handwaivy mitigation, maybe just disable passthrough without SGL support by default to actually fix and not just mitigate the CVE?
Quoted 7 lines
Quoted 3 lines
SGL is an optional feature that many devices don't implement. Even fewer do it for metadata. Disabling it entirely is "breaking userspace" for users I need to support.
Well, if that usage creates exploitable behavior we'll need to fix it and not just paper over it. Although this probably only really matters for the non-privileged passthrough.
Only admin users can access this path by default. You have to opt-in for it, so it's not exploitable unless you ask for it. I can't see disabling the interface entirely. In a previous version of this patch, I had the kernel tainted if you tried to do passthrough without SGL support. Would that be a fair compromise if I reintroduce that behavior?
Christoph Hellwig
maintainerRe: [PATCHv2 2/2] nvme-pci: use sgls for all user requests if possible
Quoted 2 lines · Keith Busch
Only admin users can access this path by default. You have to opt-in for it, so it's not exploitable unless you ask for it.
We do allow non-privileged users to send I/O commands and a small whitelist of admin commands by default, limited by the CSE effects.
Quoted 4 lines
I can't see disabling
the interface entirely. In a previous version of this patch, I had the kernel tainted if you tried to do passthrough without SGL support. Would that be a fair compromise if I reintroduce that behavior?
Taint might be a bit too strong, but a one-time message in the log might be useful.
Keith Busch
maintainerRe: [PATCHv2 2/2] nvme-pci: use sgls for all user requests if possible
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Keith Busch
Only admin users can access this path by default. You have to opt-in for it, so it's not exploitable unless you ask for it.
We do allow non-privileged users to send I/O commands and a small whitelist of admin commands by default, limited by the CSE effects.
Not if you can't open the device. The default access permissions don't allow non-priviledged users.
Quoted 7 lines
Quoted 4 lines
I can't see disabling
the interface entirely. In a previous version of this patch, I had the kernel tainted if you tried to do passthrough without SGL support. Would that be a fair compromise if I reintroduce that behavior?Taint might be a bit too strong, but a one-time message in the log might be useful.
Sounds good to me.
Christoph Hellwig
maintainerQuoted 9 lines · Keith Busch
From: Keith Busch <[email protected]>
Changes from previous version:
Merged up to linux 6.12, which had some necessary blk-integrity patches
Dropped all the initial "cleanup" patches. This is instead the minimum patchset to satisfy using the feature.
Refresh my brain, but where was the previous version? A quick grep for " Using SGLs for userspace commands" on my linux-nvme mailbox and an internet search don't find results.
Are the previous cleanups the fixes to the block layer metadata/integrity code to actually properly deal with splitting and merging?
Keith Busch
maintainerQuoted 11 lines · Christoph Hellwig
Quoted 9 lines · Keith Busch
From: Keith Busch <[email protected]>
Changes from previous version:
Merged up to linux 6.12, which had some necessary blk-integrity patches
Dropped all the initial "cleanup" patches. This is instead the minimum patchset to satisfy using the feature.
Quoted 6 lines
Refresh my brain, but where was the previous version? A quick grep for " Using SGLs for userspace commands" on my linux-nvme mailbox and an internet search don't find results.
Are the previous cleanups the fixes to the block layer metadata/integrity code to actually properly deal with splitting and merging?
It's here:
https://lore.kernel.org/linux-nvme/[email protected]/
This series follows on the block integrity series that fixed up the merging and segment accounting.
Much of it is not necessary to get SGL metadata support. It was just cleanups in this path I did along the way.