hw/block/nvme: cmb enhancements and bump to v1.4
PATCH 1/3 hw/block/nvme: cmb enhancements
In reply to Klaus JensenFrom: Padmakar Kalghatgi <[email protected]>
Implement v1.4 logic for configuring the Controller Memory Buffer. The x-legacy-cmb nvme device parameter can be set to revert to v1.3 behavior.
--- hw/block/nvme.h | 1 + include/block/nvme.h | 106 +++++++++++++++++++++++++++++++++++++++---- hw/block/nvme.c | 86 ++++++++++++++++++++++++++++------- 3 files changed, 167 insertions(+), 26 deletions(-)
hw/block/nvme.h +1 -0
@@ -16,6 +16,7 @@ typedef struct NvmeParams {uint32_t aer_max_queued;uint8_t mdts;bool use_intel_id; +bool legacy_cmb; } NvmeParams; typedef struct NvmeAsyncEvent {
include/block/nvme.h +96 -10
158 lines not on this page · raw message
hw/block/nvme.c +70 -16
129 lines not on this page · raw message
Signature · 1 line
2.29.2
PATCH 2/3 hw/block/nvme: bump to v1.4
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
With the new CMB logic in place, bump the implemented specification version to v1.4.
--- hw/block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
hw/block/nvme.c +1 -1
@@ -74,7 +74,7 @@#define NVME_MAX_IOQPAIRS 0xffff#define NVME_DB_SIZE 4 -#define NVME_SPEC_VER 0x00010300 +#define NVME_SPEC_VER 0x00010400#define NVME_CMB_BIR 2#define NVME_PMR_BIR 2#define NVME_TEMPERATURE 0x143
Signature · 1 line
2.29.2
PATCH 3/3 hw/block/nvme: lift cmb restrictions
In reply to Klaus JensenFrom: Klaus Jensen <[email protected]>
The controller now implements v1.4 and we can lift the restrictions on CMB Data Pointer and Command Independent Locations Support (CDPCILS) and CMB Data Pointer Mixed Locations Support (CDPMLS) since the device really does not care about mixed host/cmb pointers in those cases.
--- hw/block/nvme.c | 33 ++------------------------------- 1 file changed, 2 insertions(+), 31 deletions(-)
hw/block/nvme.c +2 -31
@@ -336,7 +336,6 @@ static uint16_t nvme_map_prp(NvmeCtrl *n, uint64_t prp1, uint64_t prp2, trans_len =MIN (len, trans_len);int num_prps = (len >> n->page_bits) +1 ;uint16_t status; -bool prp_list_in_cmb = false;int ret;QEMUSGList *qsg = &req->qsg;
@@ -362,10 +361,6 @@ static uint16_t nvme_map_prp(NvmeCtrl *n, uint64_t prp1, uint64_t prp2,uint32_t nents, prp_trans;int i =0 ; -if (nvme_addr_is_cmb (n, prp2)) { - prp_list_in_cmb = true; - } - nents = (len + n->page_size -1 ) >> n->page_bits; prp_trans =MIN (n->max_prp_ents, nents) *sizeof (uint64_t ); ret =nvme_addr_read (n, prp2, (void *)prp_list, prp_trans);
@@ -382,10 +377,6 @@ static uint16_t nvme_map_prp(NvmeCtrl *n, uint64_t prp1, uint64_t prp2,return NVME_INVALID_PRP_OFFSET |NVME_DNR ; } -if (prp_list_in_cmb !=nvme_addr_is_cmb (n, prp_ent)) { -return NVME_INVALID_USE_OF_CMB |NVME_DNR ; - } - i =0 ; nents = (len + n->page_size -1 ) >> n->page_bits; prp_trans =MIN (n->max_prp_ents, nents) *sizeof (uint64_t );
@@ -519,7 +510,6 @@ static uint16_t nvme_map_sgl(NvmeCtrl *n, QEMUSGList *qsg, QEMUIOVector *iov,uint64_t nsgld;uint32_t seg_len;uint16_t status; -bool sgl_in_cmb = false;hwaddr addr;int ret;
@@ -541,18 +531,6 @@ static uint16_t nvme_map_sgl(NvmeCtrl *n, QEMUSGList *qsg, QEMUIOVector *iov, goto out; } -/* -* If the segment is located in the CMB, the submission queue of the -* request must also reside there. -*/ -if (nvme_addr_is_cmb (n, addr)) { -if (!nvme_addr_is_cmb (n, req->sq->dma_addr)) { -return NVME_INVALID_USE_OF_CMB |NVME_DNR ; - } - - sgl_in_cmb = true; - } -for (;;) {switch (NVME_SGL_TYPE (sgld->type)) {case NVME_SGL_DESCR_TYPE_SEGMENT :
@@ -641,15 +619,6 @@ static uint16_t nvme_map_sgl(NvmeCtrl *n, QEMUSGList *qsg, QEMUIOVector *iov,if (status) { goto unmap; } - -/* -* If the next segment is in the CMB, make sure that the sgl was -* already located there. -*/ -if (sgl_in_cmb !=nvme_addr_is_cmb (n, addr)) { - status =NVME_INVALID_USE_OF_CMB |NVME_DNR ; - goto unmap; - } } out:
@@ -2428,6 +2397,8 @@ static int nvme_start_ctrl(NvmeCtrl *n)static void nvme_cmb_enable_regs (NvmeCtrl *n) { +NVME_CMBLOC_SET_CDPCILS (n->bar.cmbloc,1 ); +NVME_CMBLOC_SET_CDPMLS (n->bar.cmbloc,1 );NVME_CMBLOC_SET_BIR (n->bar.cmbloc,NVME_CMB_BIR );NVME_CMBSZ_SET_SQS (n->bar.cmbsz,1 );
Signature · 1 line
2.29.2
cover hw/block/nvme: cmb enhancements and bump to v1.4
From: Klaus Jensen <[email protected]>
This adds CMB logic from v1.4. To my knowledge, this is the last piece missing for v1.4 compliance, so bump the controller spec version. Please retort if I am jumping the gun.
Since the slow-moving (sorry, super poor attempt of humor) Linux kernel nvme driver does not support v1.4 CMB, this series adds a 'x-legacy-cmb' nvme device parameter that reverts the CMB configuration behavior to v1.3, thus allowing the kernel to continue using the CMB for submission queues.
The question here is if we should rather skip that parameter, only support v1.4 behavior and point the finger at the kernel nvme gang. I'm leaning towards this.
Keith, what is your opinion on this?
Klaus Jensen (2):
hw/block/nvme: bump to v1.4
hw/block/nvme: lift cmb restrictions
Padmakar Kalghatgi (1):
hw/block/nvme: cmb enhancements
hw/block/nvme.h | 1 + include/block/nvme.h | 106 +++++++++++++++++++++++++++++++++---- hw/block/nvme.c | 121 ++++++++++++++++++++++++++----------------- 3 files changed, 170 insertions(+), 58 deletions(-)
Signature · 1 line
2.29.2
Quoted 12 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
This adds CMB logic from v1.4. To my knowledge, this is the last piece missing for v1.4 compliance, so bump the controller spec version. Please retort if I am jumping the gun.
Since the slow-moving (sorry, super poor attempt of humor) Linux kernel nvme driver does not support v1.4 CMB, this series adds a 'x-legacy-cmb' nvme device parameter that reverts the CMB configuration behavior to v1.3, thus allowing the kernel to continue using the CMB for submission queues.
Quoted 5 lines
The question here is if we should rather skip that parameter, only support v1.4 behavior and point the finger at the kernel nvme gang. I'm leaning towards this.
Keith, what is your opinion on this?
The driver may know if it's running in a VM, but it doesn't necessarily know if its device is a passthrough vs. emulated. In the passthrough case, we can't use the CMB at all for <= 1.3 versions since the GPA vs HPA are not being considered. IMO, the kernel driver does need to be updated to suppress CMB usage for pre-1.4, and it should also add the CMBMSC capabilities for 1.4+.
Going forward with the qemu controller, I would remove legacy support for CMB entirely since every host should be able to interop with it without CMB support.