hw/block/nvme: namespace types and zoned namespaces
cover hw/block/nvme: namespace types and zoned namespaces
From: Klaus Jensen <[email protected]>
Hi all,
This series adds support for TP 4056 ("Namespace Types") and TP 4053 ("Zoned Namespaces") and is an alternative implementation to the one submitted by Dmitry[1].
While I don't want this to end up as a discussion about the merits of each version, I want to point out a couple of differences from Dmitry's version. At a glance, my version
* builds on my patch series that adds fairly complete NVMe v1.4 mandatory support, as well as nice-to-have feature such as SGLs, multiple namespaces and mostly just overall clean up. This finally brings the nvme device into a fairly compliant state on which we can add new features. I've tried hard to get these compliance and clean-up patches merged for a long time (in parallel with developing the emulation of NST and ZNS) and I would be really sad to see them by-passed since they have already been through many iterations and already carries Acked- and Reviewed-by's for the bulk of the patches. I think the nvme device is already in a "frankenstate" wrt. the implemented nvme version and the features it currently supports, so I think this kind of cleanup is long overdue.
* uses an attached blockdev and standard blk_aio for persistent zone info. This is the same method used in our patches for Write Uncorrectable and (separate and extended lba) metadata support, but I've left those optional features out for now to ease the review process.
* relies on the universal dulbe support added in ("hw/block/nvme: add support for dulbe") and sparse images for handling reads in gaps (above write pointer and below ZSZE); that is - the size of the underlying blockdev is in terms of ZSZE, not ZCAP
* the controller uses timers to autonomously finish zones (wrt. FRL)
I've been on paternity leave for a month, so I havn't been around to review Dmitry's patches, but I have started that process now. I would also be happy to work with Dmitry & Friends on merging our versions to get the best of both worlds if it makes sense.
This series and all preparatory patch sets (the ones I've been posting yesterday and today) are available on my GitHub[2]. Unfortunately Patchew got screwed up in the middle of me sending patches and it never picked up v2 of "hw/block/nvme: support multiple namespaces" because it was getting late and I made a mistake with the CC's. So my posted series don't apply according to Patchew, but they actually do if you follow the Based-on's (... or just grab [2]).
[1]: Message-Id: <[email protected]> [2]: https://github.com/birkelund/qemu/tree/for-master/nvme
Based-on: <[email protected]> ("[PATCH 0/3] hw/block/nvme: bump to v1.4")
Klaus Jensen (10):
hw/block/nvme: support I/O Command Sets hw/block/nvme: add zns specific fields and types hw/block/nvme: add basic read/write for zoned namespaces hw/block/nvme: add the zone management receive command hw/block/nvme: add the zone management send command hw/block/nvme: add the zone append command hw/block/nvme: track and enforce zone resources hw/block/nvme: allow open to close transitions by controller hw/block/nvme: allow zone excursions hw/block/nvme: support reset/finish recommended limits
block/nvme.c | 6 +- hw/block/nvme-ns.c | 397 +++++++++- hw/block/nvme-ns.h | 148 +++- hw/block/nvme.c | 1676 +++++++++++++++++++++++++++++++++++++++-- hw/block/nvme.h | 76 +- hw/block/trace-events | 43 +- include/block/nvme.h | 252 ++++++- 7 files changed, 2469 insertions(+), 129 deletions(-)
Signature · 1 line
2.27.0
PATCH 1/10 hw/block/nvme: support I/O Command Sets
From: Klaus Jensen <[email protected]>
Implement support for TP 4056 ("Namespace Types"). This adds the 'iocs' (I/O Command Set) device parameter to the nvme-ns device.
--- block/nvme.c | 6 +- hw/block/nvme-ns.c | 24 +++-- hw/block/nvme-ns.h | 11 +- hw/block/nvme.c | 226 +++++++++++++++++++++++++++++++++--------- hw/block/nvme.h | 52 ++++++---- hw/block/trace-events | 6 +- include/block/nvme.h | 53 ++++++++-- 7 files changed, 285 insertions(+), 93 deletions(-)
block/nvme.c +3 -3
@@ -333,7 +333,7 @@ static inline int nvme_translate_error(const NvmeCqe *c) {uint16_t status = (le16_to_cpu (c->status) >>1 ) &0xFF ;if (status) { - trace_nvme_error(le32_to_cpu(c->result), + trace_nvme_error(le32_to_cpu(c->dw0), le16_to_cpu(c->sq_head), le16_to_cpu(c->sq_id), le16_to_cpu(c->cid),
@@ -495,7 +495,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) {BDRVNVMeState *s = bs->opaque;NvmeIdCtrl *idctrl; -NvmeIdNs *idns; +NvmeIdNsNvm *idns;NvmeLBAF *lbaf;uint8_t *resp;uint16_t oncs;
@@ -512,7 +512,7 @@ static void nvme_identify(BlockDriverState *bs, int namespace, Error **errp) goto out; } idctrl = (NvmeIdCtrl *)resp; - idns = (NvmeIdNs *)resp; + idns = (NvmeIdNsNvm *)resp; r =qemu_vfio_dma_map (s->vfio, resp,sizeof (NvmeIdCtrl), true, &iova);if (r) {error_setg (errp,"Cannot map buffer for DMA" );
hw/block/nvme-ns.c +17 -7
@@ -59,8 +59,16 @@ static int nvme_ns_blk_resize(BlockBackend *blk, size_t len, Error **errp)static void nvme_ns_init (NvmeNamespace *ns) { -NvmeIdNs *id_ns = &ns->id_ns; +NvmeIdNsNvm *id_ns; +int unmap =blk_get_flags (ns->blk) &BDRV_O_UNMAP ; + + ns->id_ns[NVME_IOCS_NVM ] =g_new0 (NvmeIdNsNvm,1 ); + id_ns =nvme_ns_id_nvm (ns); + + ns->iocs = ns->params.iocs; + + id_ns->dlfeat = unmap ?0x9 :0x0 ; id_ns->lbaf[0 ].ds = ns->params.lbads; id_ns->nsze =cpu_to_le64 (nvme_ns_nlbas (ns));
@@ -130,8 +138,7 @@ static int nvme_ns_init_blk_state(NvmeNamespace *ns, Error **errp)return 0 ; } -static int nvme_ns_init_blk (NvmeCtrl *n,NvmeNamespace *ns,NvmeIdCtrl *id, -Error **errp) +static int nvme_ns_init_blk (NvmeCtrl *n,NvmeNamespace *ns,Error **errp) {uint64_t perm, shared_perm;
@@ -174,7 +181,8 @@ static int nvme_ns_init_blk(NvmeCtrl *n, NvmeNamespace *ns, NvmeIdCtrl *id,return 0 ; } -static int nvme_ns_check_constraints (NvmeNamespace *ns,Error **errp) +static int nvme_ns_check_constraints (NvmeCtrl *n,NvmeNamespace *ns,Error + **errp) {if (!ns->blk) {error_setg (errp,"block backend not configured" );
@@ -191,11 +199,11 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)int nvme_ns_setup (NvmeCtrl *n,NvmeNamespace *ns,Error **errp) { -if (nvme_ns_check_constraints (ns, errp)) { +if (nvme_ns_check_constraints (n, ns, errp)) {return -1 ; } -if (nvme_ns_init_blk (n, ns, &n->id_ctrl, errp)) { +if (nvme_ns_init_blk (n, ns, errp)) {return -1 ; }
@@ -210,7 +218,8 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error **errp)* With a state file in place we can enable the Deallocated or * Unwritten Logical Block Error feature. */ - ns->id_ns.nsfeat |=0x4 ; +NvmeIdNsNvm *id_ns =nvme_ns_id_nvm (ns); + id_ns->nsfeat |=0x4 ; }if (nvme_register_namespace (n, ns, errp)) {
@@ -239,6 +248,7 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT32 ("nsid" , NvmeNamespace, params.nsid,0 ),DEFINE_PROP_UINT8 ("lbads" , NvmeNamespace, params.lbads,BDRV_SECTOR_BITS ),DEFINE_PROP_DRIVE ("state" , NvmeNamespace, blk_state), +DEFINE_PROP_UINT8 ("iocs" , NvmeNamespace, params.iocs,0x0 ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme-ns.h +9 -2
@@ -21,6 +21,7 @@typedef struct NvmeNamespaceParams {uint32_t nsid; +uint8_t iocs;uint8_t lbads; }NvmeNamespaceParams ;
@@ -30,8 +31,9 @@ typedef struct NvmeNamespace {BlockBackend *blk_state;int32_t bootindex;int64_t size; +uint8_t iocs; -NvmeIdNs id_ns; +void *id_ns[256 ];NvmeNamespaceParams params;unsigned long *utilization;
@@ -50,9 +52,14 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns)return -1 ; } +static inline NvmeIdNsNvm *nvme_ns_id_nvm (NvmeNamespace *ns) +{ +return ns->id_ns[NVME_IOCS_NVM ]; +} +static inline NvmeLBAF *nvme_ns_lbaf (NvmeNamespace *ns) { -NvmeIdNs *id_ns = &ns->id_ns; +NvmeIdNsNvm *id_ns =nvme_ns_id_nvm (ns);return &id_ns->lbaf[NVME_ID_NS_FLBAS_INDEX (id_ns->flbas)]; }
hw/block/nvme.c +178 -48
435 lines not on this page · raw message
hw/block/nvme.h +31 -21
89 lines not on this page · raw message
hw/block/trace-events +4 -2
@@ -51,10 +51,12 @@ pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d" pci_nvme_del_sq(uint16_t qid) "deleting submission queue sqid=%"PRIu16"" pci_nvme_del_cq(uint16_t cqid) "deleted completion queue, cqid=%"PRIu16"" +pci_nvme_identify(uint16_t cid, uint32_t nsid, uint16_t cntid, uint8_t cns, uint8_t csi, uint16_t nvmsetid) "cid %"PRIu16" nsid %"PRIu32" cntid 0x%"PRIx16" cns 0x%"PRIx8" csi 0x%"PRIx8" nvmsetid %"PRIu16"" pci_nvme_identify_ctrl(void) "identify controller" -pci_nvme_identify_ns(uint32_t ns) "nsid %"PRIu32"" -pci_nvme_identify_nslist(uint32_t ns) "nsid %"PRIu32"" +pci_nvme_identify_ns(uint32_t ns, uint8_t csi) "nsid %"PRIu32" csi 0x%"PRIx8"" +pci_nvme_identify_nslist(uint32_t ns, uint8_t csi) "nsid %"PRIu32" csi 0x%"PRIx8"" pci_nvme_identify_ns_descr_list(uint32_t ns) "nsid %"PRIu32"" +pci_nvme_identify_io_cmd_set(uint16_t cid) "cid %"PRIu16"" pci_nvme_get_log(uint16_t cid, uint8_t lid, uint8_t lsp, uint8_t rae, uint32_t len, uint64_t off) "cid %"PRIu16" lid 0x%"PRIx8" lsp 0x%"PRIx8" rae 0x%"PRIx8" len %"PRIu32" off %"PRIu64"" pci_nvme_getfeat(uint16_t cid, uint8_t fid, uint8_t sel, uint32_t cdw11) "cid %"PRIu16" fid 0x%"PRIx8" sel 0x%"PRIx8" cdw11 0x%"PRIx32"" pci_nvme_setfeat(uint16_t cid, uint8_t fid, uint8_t save, uint32_t cdw11) "cid %"PRIu16" fid 0x%"PRIx8" save 0x%"PRIx8" cdw11 0x%"PRIx32""
include/block/nvme.h +43 -10
130 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH 2/10 hw/block/nvme: add zns specific fields and types
In reply to Klaus JensenAdd new fields, types and data structures for TP 4053 ("Zoned Namespaces").
--- include/block/nvme.h | 186 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 180 insertions(+), 6 deletions(-)
include/block/nvme.h +180 -6
@@ -465,7 +465,8 @@ enum NvmeCmbmscMask {#define NVME_CMBSTS_CBAI (cmbsts) (cmsts & 0x1)enum NvmeCommandSet { -NVME_IOCS_NVM =0x0 , +NVME_IOCS_NVM =0x0 , +NVME_IOCS_ZONED =0x2 , };enum NvmeSglDescriptorType {
@@ -552,6 +553,11 @@ enum NvmeIoCommands {NVME_CMD_COMPARE =0x05 ,NVME_CMD_WRITE_ZEROES =0x08 ,NVME_CMD_DSM =0x09 , + +/* Zoned Command Set */ +NVME_CMD_ZONE_MGMT_SEND =0x79 , +NVME_CMD_ZONE_MGMT_RECV =0x7a , + NVME_CMD_ZONE_APPEND = 0x7d, }; typedef struct NvmeDeleteQ {
@@ -664,6 +670,82 @@ enum { NVME_RW_PRINFO_PRCHK_REF = 1 << 10, }; +typedef struct NvmeZoneAppendCmd { +uint8_t opcode; +uint8_t flags; +uint16_t cid; +uint32_t nsid; +uint32_t rsvd8[2 ]; +uint64_t mptr; +NvmeCmdDptr dptr; +uint64_t zslba; +uint16_t nlb; +uint8_t rsvd50; +uint8_t control; +uint32_t ilbrt; +uint16_t lbat; +uint16_t lbatm; +}NvmeZoneAppendCmd ; + +typedef struct NvmeZoneManagementSendCmd { +uint8_t opcode; +uint8_t flags; +uint16_t cid; +uint32_t nsid; +uint32_t rsvd8[4 ]; +NvmeCmdDptr dptr; +uint64_t slba; +uint32_t rsvd48; +uint8_t zsa; +uint8_t zsflags; +uint16_t rsvd54; +uint32_t rsvd56[2 ]; +}NvmeZoneManagementSendCmd ; + +#define NVME_CMD_ZONE_MGMT_SEND_SELECT_ALL (zsflags) ((zsflags) & 0x1) + +typedef enum NvmeZoneManagementSendAction { +NVME_CMD_ZONE_MGMT_SEND_CLOSE =0x1 , +NVME_CMD_ZONE_MGMT_SEND_FINISH =0x2 , +NVME_CMD_ZONE_MGMT_SEND_OPEN =0x3 , +NVME_CMD_ZONE_MGMT_SEND_RESET =0x4 , +NVME_CMD_ZONE_MGMT_SEND_OFFLINE =0x5 , +NVME_CMD_ZONE_MGMT_SEND_SET_ZDE =0x10 , +}NvmeZoneManagementSendAction ; + +typedef struct NvmeZoneManagementRecvCmd { +uint8_t opcode; +uint8_t flags; +uint16_t cid; +uint32_t nsid; +uint8_t rsvd8[16 ]; +NvmeCmdDptr dptr; +uint64_t slba; +uint32_t numdw; +uint8_t zra; +uint8_t zrasp; +uint8_t zrasf; +uint8_t rsvd55[9 ]; +}NvmeZoneManagementRecvCmd ; + +typedef enum NvmeZoneManagementRecvAction { +NVME_CMD_ZONE_MGMT_RECV_REPORT_ZONES =0x0 , +NVME_CMD_ZONE_MGMT_RECV_EXTENDED_REPORT_ZONES =0x1 , +}NvmeZoneManagementRecvAction ; + +typedef enum NvmeZoneManagementRecvActionSpecificField { +NVME_CMD_ZONE_MGMT_RECV_LIST_ALL =0x0 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSE =0x1 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSIO =0x2 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSEO =0x3 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSC =0x4 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSF =0x5 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSRO =0x6 , +NVME_CMD_ZONE_MGMT_RECV_LIST_ZSO =0x7 , +}NvmeZoneManagementRecvActionSpecificField ; + +#define NVME_CMD_ZONE_MGMT_RECEIVE_PARTIAL 0x1 + typedef struct NvmeDsmCmd { uint8_t opcode; uint8_t flags;
@@ -702,13 +784,15 @@ enum NvmeAsyncEventRequest {NVME_AER_INFO_SMART_RELIABILITY =0 ,NVME_AER_INFO_SMART_TEMP_THRESH =1 ,NVME_AER_INFO_SMART_SPARE_THRESH =2 , + NVME_AER_INFO_NOTICE_ZONE_DESCR_CHANGED = 0xef, };typedef struct NvmeAerResult { -uint8_t event_type; -uint8_t event_info; -uint8_t log_page; -uint8_t resv; +uint8_t event_type; +uint8_t event_info; +uint8_t log_page; +uint8_t resv; +uint32_t nsid; }NvmeAerResult ; typedef struct NvmeCqe {
@@ -775,6 +859,14 @@ enum NvmeStatusCodes {NVME_CONFLICTING_ATTRS =0x0180 ,NVME_INVALID_PROT_INFO =0x0181 ,NVME_WRITE_TO_RO =0x0182 , +NVME_ZONE_BOUNDARY_ERROR =0x01b8 , +NVME_ZONE_IS_FULL =0x01b9 , +NVME_ZONE_IS_READ_ONLY =0x01ba , +NVME_ZONE_IS_OFFLINE =0x01bb , +NVME_ZONE_INVALID_WRITE =0x01bc , +NVME_TOO_MANY_ACTIVE_ZONES =0x01bd , +NVME_TOO_MANY_OPEN_ZONES =0x01be , +NVME_INVALID_ZONE_STATE_TRANSITION =0x01bf ,NVME_WRITE_FAULT =0x0280 ,NVME_UNRECOVERED_READ =0x0281 , NVME_E2E_GUARD_ERROR = 0x0282,
@@ -868,6 +960,46 @@ enum { NVME_EFFECTS_UUID_SEL = 1 << 19, }; +typedef enum NvmeZoneType { +NVME_ZT_SEQ =0x2 , +}NvmeZoneType ; + +typedef enum NvmeZoneState { +NVME_ZS_ZSE =0x1 , +NVME_ZS_ZSIO =0x2 , +NVME_ZS_ZSEO =0x3 , +NVME_ZS_ZSC =0x4 , +NVME_ZS_ZSRO =0xd , +NVME_ZS_ZSF =0xe , +NVME_ZS_ZSO =0xf , +}NvmeZoneState ; + +typedef struct NvmeZoneDescriptor { +uint8_t zt; +uint8_t zs; +uint8_t za; +uint8_t rsvd3[5 ]; +uint64_t zcap; +uint64_t zslba; +uint64_t wp; +uint8_t rsvd32[32 ]; +}NvmeZoneDescriptor ; + +#define NVME_ZS (zs) (((zs) >> 4) & 0xf) +#define NVME_ZS_SET (zs, state) ((zs) = ((state) << 4)) + +#define NVME_ZA_ZFC (za) ((za) & (1 << 0)) +#define NVME_ZA_FZR (za) ((za) & (1 << 1)) +#define NVME_ZA_RZR (za) ((za) & (1 << 2)) +#define NVME_ZA_ZDEV (za) ((za) & (1 << 7)) + +#define NVME_ZA_SET_ZFC (za, val) ((za) |= (((val) & 1) << 0)) +#define NVME_ZA_SET_FZR (za, val) ((za) |= (((val) & 1) << 1)) +#define NVME_ZA_SET_RZR (za, val) ((za) |= (((val) & 1) << 2)) +#define NVME_ZA_SET_ZDEV (za, val) ((za) |= (((val) & 1) << 7)) + +#define NVME_ZA_CLEAR (za) ((za) = 0x0) +enum NvmeSmartWarn {NVME_SMART_SPARE =1 <<0 ,NVME_SMART_TEMPERATURE =1 <<1 ,
@@ -899,6 +1031,7 @@ enum NvmeLogIdentifier {NVME_LOG_SMART_INFO =0x02 ,NVME_LOG_FW_SLOT_INFO =0x03 ,NVME_LOG_EFFECTS =0x05 , + NVME_LOG_CHANGED_ZONE_LIST = 0xbf, }; typedef struct NvmePSD {
@@ -1008,6 +1141,10 @@ typedef struct NvmeIdCtrl {uint8_t vs[1024 ]; } NvmeIdCtrl; +enum NvmeIdCtrlOaes { +NVME_OAES_ZDCN =1 <<27 , +}; +enum NvmeIdCtrlOacs {NVME_OACS_SECURITY =1 <<0 ,NVME_OACS_FORMAT =1 <<1 ,
@@ -1048,6 +1185,11 @@ enum NvmeIdCtrlLpa {#define NVME_CTRL_SGLS_MPTR_SGL (0x1 << 19)#define NVME_CTRL_SGLS_ADDR_OFFSET (0x1 << 20) +typedef struct NvmeIdCtrlZns { +uint8_t zasl; +uint8_t rsvd1[4095 ]; +}NvmeIdCtrlZns ; +#define NVME_ARB_AB (arb) (arb & 0x7)#define NVME_ARB_AB_NOLIMIT 0x7#define NVME_ARB_LPW (arb) ((arb >> 8) & 0xff)
@@ -1071,6 +1213,7 @@ enum NvmeIdCtrlLpa {#define NVME_AEC_SMART (aec) (aec & 0xff)#define NVME_AEC_NS_ATTR (aec) ((aec >> 8) & 0x1)#define NVME_AEC_FW_ACTIVATION (aec) ((aec >> 9) & 0x1) +#define NVME_AEC_ZDCN (aec) ((aec >> 27) & 0x1)#define NVME_ERR_REC_TLER (err_rec) (err_rec & 0xffff)#define NVME_ERR_REC_DULBE (err_rec) (err_rec & 0x10000)
@@ -1226,9 +1369,33 @@ enum NvmeIdNsDps { DPS_FIRST_EIGHT = 8, }; +typedef struct NvmeLBAFE { +uint64_t zsze; +uint8_t zdes; +uint8_t rsvd9[7 ]; +}NvmeLBAFE ; + +typedef struct NvmeIdNsZns { +uint16_t zoc; +uint16_t ozcs; +uint32_t mar; +uint32_t mor; +uint32_t rrl; +uint32_t frl; +uint8_t rsvd20[2795 ]; +NvmeLBAFE lbafe[16 ]; +uint8_t rsvd3072[768 ]; +uint8_t vs[256 ]; +}NvmeIdNsZns ; + +#define NVME_ID_NS_ZNS_ZOC_VZC (1 << 0) +#define NVME_ID_NS_ZNS_ZOC_ZAE (1 << 1) + +#define NVME_ID_NS_ZNS_OZCS_RAZB (1 << 0) +static inline void _nvme_check_size (void ) { -QEMU_BUILD_BUG_ON (sizeof (NvmeAerResult) !=4 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeAerResult) !=8 );QEMU_BUILD_BUG_ON (sizeof (NvmeCqe) !=16 );QEMU_BUILD_BUG_ON (sizeof (NvmeDsmRange) !=16 );QEMU_BUILD_BUG_ON (sizeof (NvmeCmd) !=64 );
@@ -1237,17 +1404,24 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeCreateSq) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdentify) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeRwCmd) !=64 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeZoneAppendCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeDsmCmd) !=64 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeZoneManagementSendCmd) !=64 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeZoneManagementRecvCmd) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeRangeType) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeErrorLog) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeFwSlotInfoLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeSmartLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeEnduranceGroupLog) !=512 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdCtrl) !=4096 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeIdCtrlZns) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsNvm) !=4096 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeIdNsZns) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeNvmSetAttributes) !=128 );QEMU_BUILD_BUG_ON (sizeof (NvmeIdNvmSetList) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeBar) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeEffectsLog) !=4096 ); +QEMU_BUILD_BUG_ON (sizeof (NvmeZoneDescriptor) !=64 ); + QEMU_BUILD_BUG_ON(sizeof(NvmeLBAFE) != 16); }#endif
Signature · 1 line
2.27.0
PATCH 3/10 hw/block/nvme: add basic read/write for zoned namespaces
In reply to Klaus JensenThis adds basic read and write for zoned namespaces.
A zoned namespace is created by setting the iocs parameter to 0x2 and supplying a zero-sized blockdev for zone info persistent state (zns.zoneinfo parameter) and the zns.zcap parameter to specify the individual zone capacities. The namespace device will compute the resulting zone size to be the next power of two and fit in as many zones as possible on the underlying namespace blockdev.
If the zone info blockdev pointed to by zns.zoneinfo is non-zero in size it will be assumed to contain existing zone state.
--- hw/block/nvme-ns.c | 227 +++++++++++++++++++++++++- hw/block/nvme-ns.h | 103 ++++++++++++ hw/block/nvme.c | 361 +++++++++++++++++++++++++++++++++++++++--- hw/block/nvme.h | 1 + hw/block/trace-events | 10 ++ 5 files changed, 677 insertions(+), 25 deletions(-)
hw/block/nvme-ns.c +226 -1
@@ -28,6 +28,26 @@#include "nvme.h" #include "nvme-ns.h" +const char *nvme_zs_str (NvmeZone *zone) +{ +return nvme_zs_to_str (nvme_zs (zone)); +} + +const char *nvme_zs_to_str (NvmeZoneState zs) +{ +switch (zs) { +case NVME_ZS_ZSE :return "ZSE" ; +case NVME_ZS_ZSIO :return "ZSIO" ; +case NVME_ZS_ZSEO :return "ZSEO" ; +case NVME_ZS_ZSC :return "ZSC" ; +case NVME_ZS_ZSRO :return "ZSRO" ; +case NVME_ZS_ZSF :return "ZSF" ; +case NVME_ZS_ZSO :return "ZSO" ; + } + +return NULL ; +} +static int nvme_ns_blk_resize (BlockBackend *blk,size_t len,Error **errp) {Error *local_err =NULL ;
@@ -57,6 +77,171 @@ static int nvme_ns_blk_resize(BlockBackend *blk, size_t len, Error **errp)return 0 ; } +static int nvme_ns_init_blk_zoneinfo (NvmeNamespace *ns,size_t len, +Error **errp) +{ +NvmeZone *zone; +NvmeZoneDescriptor *zd; +uint64_t zslba; +int ret; + +BlockBackend *blk = ns->zns.info.blk; + +Error *local_err =NULL ; + +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zslba = i *nvme_ns_zsze (ns); + zone =nvme_ns_get_zone (ns, zslba); + zd = &zone->zd; + + zd->zt =NVME_ZT_SEQ ; +nvme_zs_set (zone,NVME_ZS_ZSE ); + zd->zcap = ns->params.zns.zcap; + zone->wp_staging = zslba; + zd->wp = zd->zslba =cpu_to_le64 (zslba); + } + + ret =nvme_ns_blk_resize (blk, len, &local_err); +if (ret) { +error_propagate_prepend (errp, local_err, +"could not resize zoneinfo blockdev: " ); +return ret; + } + +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zd = &ns->zns.info.zones[i].zd; + + ret =blk_pwrite (blk, i *sizeof (NvmeZoneDescriptor), zd, +sizeof (NvmeZoneDescriptor),0 ); +if (ret <0 ) { +error_setg_errno (errp, -ret,"blk_pwrite: " ); +return ret; + } + } + +return 0 ; +} + +static int nvme_ns_setup_blk_zoneinfo (NvmeNamespace *ns,Error **errp) +{ +NvmeZone *zone; +NvmeZoneDescriptor *zd; +BlockBackend *blk = ns->zns.info.blk; +uint64_t perm, shared_perm; +int64_t len, zoneinfo_len; + +Error *local_err =NULL ; +int ret; + + perm =BLK_PERM_CONSISTENT_READ |BLK_PERM_WRITE ; + shared_perm =BLK_PERM_ALL ; + + ret =blk_set_perm (blk, perm, shared_perm, &local_err); +if (ret) { +error_propagate_prepend (errp, local_err,"blk_set_perm: " ); +return ret; + } + + zoneinfo_len =ROUND_UP (ns->zns.info.num_zones * +sizeof (NvmeZoneDescriptor),BDRV_SECTOR_SIZE ); + + len =blk_getlength (blk); +if (len <0 ) { +error_setg_errno (errp, -len,"blk_getlength: " ); +return len; + } + +if (len) { +if (len != zoneinfo_len) { +error_setg (errp,"zoneinfo size mismatch " +"(expected %" PRIu64" bytes; was %" PRIu64" bytes)" , + zoneinfo_len, len); +error_append_hint (errp,"Did you change the zone size or " +"zone descriptor size?\n" ); +return -1 ; + } + +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zone = &ns->zns.info.zones[i]; + zd = &zone->zd; + + ret =blk_pread (blk, i *sizeof (NvmeZoneDescriptor), zd, +sizeof (NvmeZoneDescriptor)); +if (ret <0 ) { +error_setg_errno (errp, -ret,"blk_pread: " ); +return ret; + }else if (ret !=sizeof (NvmeZoneDescriptor)) { +error_setg (errp,"blk_pread: short read" ); +return -1 ; + } + + zone->wp_staging =nvme_wp (zone); + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSE : +case NVME_ZS_ZSF : +case NVME_ZS_ZSRO : +case NVME_ZS_ZSO : +continue ; + +case NVME_ZS_ZSC : +if (nvme_wp (zone) ==nvme_zslba (zone)) { +nvme_zs_set (zone,NVME_ZS_ZSE ); +continue ; + } + +/* fallthrough */ + +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : +nvme_zs_set (zone,NVME_ZS_ZSF ); +NVME_ZA_SET_ZFC (zd->za,0x1 ); + } + } + +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zd = &ns->zns.info.zones[i].zd; + + ret =blk_pwrite (blk, i *sizeof (NvmeZoneDescriptor), zd, +sizeof (NvmeZoneDescriptor),0 ); +if (ret <0 ) { +error_setg_errno (errp, -ret,"blk_pwrite: " ); +return ret; + } + } + +return 0 ; + } + +if (nvme_ns_init_blk_zoneinfo (ns, zoneinfo_len, &local_err)) { +error_propagate_prepend (errp, local_err, +"could not initialize zoneinfo blockdev: " ); + } + +return 0 ; +} + +static void nvme_ns_init_zoned (NvmeNamespace *ns) +{ +NvmeIdNsNvm *id_ns =nvme_ns_id_nvm (ns); +NvmeIdNsZns *id_ns_zns =nvme_ns_id_zoned (ns); + + id_ns_zns->zoc =cpu_to_le16 (ns->params.zns.zoc); + id_ns_zns->ozcs =cpu_to_le16 (ns->params.zns.ozcs); + +for (int i =0 ; i <= id_ns->nlbaf; i++) { + id_ns_zns->lbafe[i].zsze =cpu_to_le64 (pow2ceil (ns->params.zns.zcap)); + } + + ns->zns.info.num_zones =nvme_ns_nlbas (ns) /nvme_ns_zsze (ns); + ns->zns.info.zones =g_malloc0_n (ns->zns.info.num_zones,sizeof (NvmeZone)); + + id_ns->ncap = ns->zns.info.num_zones * ns->params.zns.zcap; + + id_ns_zns->mar =0xffffffff ; + id_ns_zns->mor =0xffffffff ; +} +static void nvme_ns_init (NvmeNamespace *ns) {NvmeIdNsNvm *id_ns;
@@ -69,12 +254,20 @@ static void nvme_ns_init(NvmeNamespace *ns) ns->iocs = ns->params.iocs; id_ns->dlfeat = unmap ?0x9 :0x0 ; +if (!nvme_ns_zoned (ns)) { + id_ns->dlfeat = unmap ?0x9 :0x0 ; + } id_ns->lbaf[0 ].ds = ns->params.lbads; id_ns->nsze =cpu_to_le64 (nvme_ns_nlbas (ns)); + id_ns->ncap = id_ns->nsze; + +if (ns->iocs ==NVME_IOCS_ZONED ) { + ns->id_ns[NVME_IOCS_ZONED ] =g_new0 (NvmeIdNsZns,1 ); +nvme_ns_init_zoned (ns); + }/* no thin provisioning */ - id_ns->ncap = id_ns->nsze; id_ns->nuse = id_ns->ncap; }
@@ -194,6 +387,28 @@ static int nvme_ns_check_constraints(NvmeCtrl *n, NvmeNamespace *ns, Errorreturn -1 ; } +switch (ns->params.iocs) { +case NVME_IOCS_NVM : +break ; + +case NVME_IOCS_ZONED : +if (!ns->zns.info.blk) { +error_setg (errp,"zone info block backend not configured" ); +return -1 ; + } + +if (!ns->params.zns.zcap) { +error_setg (errp,"zero zone capacity" ); +return -1 ; + } + +break ; + +default : +error_setg (errp,"unsupported I/O command set" ); +return -1 ; + } +return 0 ; }
@@ -222,6 +437,12 @@ int nvme_ns_setup(NvmeCtrl *n, NvmeNamespace *ns, Error **errp) id_ns->nsfeat |=0x4 ; } +if (nvme_ns_zoned (ns)) { +if (nvme_ns_setup_blk_zoneinfo (ns, errp)) { +return -1 ; + } + } +if (nvme_register_namespace (n, ns, errp)) {return -1 ; }
@@ -249,6 +470,10 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT8 ("lbads" , NvmeNamespace, params.lbads,BDRV_SECTOR_BITS ),DEFINE_PROP_DRIVE ("state" , NvmeNamespace, blk_state),DEFINE_PROP_UINT8 ("iocs" , NvmeNamespace, params.iocs,0x0 ), +DEFINE_PROP_DRIVE ("zns.zoneinfo" , NvmeNamespace, zns.info.blk), +DEFINE_PROP_UINT64 ("zns.zcap" , NvmeNamespace, params.zns.zcap,0 ), +DEFINE_PROP_UINT16 ("zns.zoc" , NvmeNamespace, params.zns.zoc,0 ), +DEFINE_PROP_UINT16 ("zns.ozcs" , NvmeNamespace, params.zns.ozcs,0 ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme-ns.h +103 -0
144 lines not on this page · raw message
hw/block/nvme.c +337 -24
501 lines not on this page · raw message
hw/block/nvme.h +1 -0
@@ -7,6 +7,7 @@#define NVME_MAX_NAMESPACES 256 typedef struct NvmeParams { + bool defensive; char *serial; uint32_t num_queues; /* deprecated since 5.1 */ uint32_t max_ioqpairs;
hw/block/trace-events +10 -0
@@ -42,6 +42,8 @@ pci_nvme_req_add_aio(uint16_t cid, void *aio, const char *blkname, uint64_t offs pci_nvme_aio_cb(uint16_t cid, void *aio, const char *blkname, uint64_t offset, const char *opc, void *req) "cid %"PRIu16" aio %p blk \"%s\" offset %"PRIu64" opc \"%s\" req %p" pci_nvme_aio_discard_cb(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_aio_write_cb(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" +pci_nvme_aio_zone_write_cb(uint16_t cid, uint64_t lba, uint32_t nlb, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" wp 0x%"PRIx64"" +pci_nvme_zone_advance_wp(uint16_t cid, uint64_t lba, uint32_t nlb, uint64_t wp_old, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" wp_old 0x%"PRIx64" wp 0x%"PRIx64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode) "cid %"PRIu16" nsid %"PRIu32" sqid %"PRIu16" opc 0x%"PRIx8"" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8"" pci_nvme_rw(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" %s nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64""
@@ -80,6 +82,8 @@ pci_nvme_mmio_write(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PRI pci_nvme_mmio_doorbell_cq(uint16_t cqid, uint16_t new_head) "cqid %"PRIu16" new_head %"PRIu16"" pci_nvme_mmio_doorbell_sq(uint16_t sqid, uint16_t new_tail) "cqid %"PRIu16" new_tail %"PRIu16"" pci_nvme_ns_update_util(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_pending_writes(uint16_t cid, uint64_t zslba, uint64_t wp, uint64_t wp_staging) "cid %"PRIu16" zslba 0x%"PRIx64" wp 0x%"PRIx64" wp_staging 0x%"PRIx64"" +pci_nvme_update_zone_info(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64"" pci_nvme_mmio_intm_set(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask set, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_intm_clr(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask clr, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_cfg(uint64_t data) "wrote MMIO, config controller config=0x%"PRIx64""
@@ -99,6 +103,10 @@ pci_nvme_err_aio(uint16_t cid, void *aio, const char *blkname, uint64_t offset, pci_nvme_err_req_status(uint16_t cid, uint32_t nsid, uint16_t status, uint8_t opc) "cid %"PRIu16" nsid %"PRIu32" status 0x%"PRIx16" opc 0x%"PRIx8"" pci_nvme_err_addr_read(uint64_t addr) "addr 0x%"PRIx64"" pci_nvme_err_addr_write(uint64_t addr) "addr 0x%"PRIx64"" +pci_nvme_err_zone_is_full(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0x%"PRIx64"" +pci_nvme_err_zone_is_read_only(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0x%"PRIx64"" +pci_nvme_err_zone_invalid_write(uint16_t cid, uint64_t slba, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" wp 0x%"PRIx64"" +pci_nvme_err_zone_boundary(uint16_t cid, uint64_t slba, uint32_t nlb, uint64_t zcap) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" zcap 0x%"PRIx64"" pci_nvme_err_invalid_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_num_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_sgl_excess_length(uint16_t cid) "cid %"PRIu16""
@@ -127,6 +135,8 @@ pci_nvme_err_invalid_identify_cns(uint16_t cns) "identify, invalid cns=0x%"PRIx1 pci_nvme_err_invalid_getfeat(int dw10) "invalid get features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_setfeat(uint32_t dw10) "invalid set features, dw10=0x%"PRIx32"" pci_nvme_err_invalid_log_page(uint16_t cid, uint16_t lid) "cid %"PRIu16" lid 0x%"PRIx16"" +pci_nvme_err_invalid_zone(uint16_t cid, uint64_t lba) "cid %"PRIu16" lba 0x%"PRIx64"" +pci_nvme_err_invalid_zone_condition(uint16_t cid, uint64_t zslba, uint8_t condition) "cid %"PRIu16" zslba 0x%"PRIx64" condition 0x%"PRIx8"" pci_nvme_err_startfail_cq(void) "nvme_start_ctrl failed because there are non-admin completion queues" pci_nvme_err_startfail_sq(void) "nvme_start_ctrl failed because there are non-admin submission queues" pci_nvme_err_startfail_nbarasq(void) "nvme_start_ctrl failed because the admin submission queue address is null"
Signature · 1 line
2.27.0
PATCH 4/10 hw/block/nvme: add the zone management receive command
In reply to Klaus JensenAdd the Zone Management Receive command.
--- hw/block/nvme-ns.c | 33 +++++++++-- hw/block/nvme-ns.h | 9 ++- hw/block/nvme.c | 130 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme.h | 6 ++ hw/block/trace-events | 1 + include/block/nvme.h | 5 ++ 6 files changed, 179 insertions(+), 5 deletions(-)
hw/block/nvme-ns.c +29 -4
@@ -99,6 +99,10 @@ static int nvme_ns_init_blk_zoneinfo(NvmeNamespace *ns, size_t len, zd->zcap = ns->params.zns.zcap; zone->wp_staging = zslba; zd->wp = zd->zslba =cpu_to_le64 (zslba); + +if (ns->params.zns.zdes) { + zone->zde =g_malloc0 (nvme_ns_zdes_bytes (ns)); + } } ret =nvme_ns_blk_resize (blk, len, &local_err);
@@ -128,7 +132,7 @@ static int nvme_ns_setup_blk_zoneinfo(NvmeNamespace *ns, Error **errp)NvmeZoneDescriptor *zd;BlockBackend *blk = ns->zns.info.blk;uint64_t perm, shared_perm; -int64_t len, zoneinfo_len; +int64_t len, zoneinfo_len, zone_len;Error *local_err =NULL ;int ret;
@@ -142,8 +146,9 @@ static int nvme_ns_setup_blk_zoneinfo(NvmeNamespace *ns, Error **errp)return ret; } - zoneinfo_len =ROUND_UP (ns->zns.info.num_zones * -sizeof (NvmeZoneDescriptor),BDRV_SECTOR_SIZE ); + zone_len =sizeof (NvmeZoneDescriptor) +nvme_ns_zdes_bytes (ns); + zoneinfo_len =ROUND_UP (ns->zns.info.num_zones * zone_len, +BDRV_SECTOR_SIZE ); len =blk_getlength (blk);if (len <0 ) {
@@ -177,6 +182,23 @@ static int nvme_ns_setup_blk_zoneinfo(NvmeNamespace *ns, Error **errp) zone->wp_staging =nvme_wp (zone); +if (ns->params.zns.zdes) { +uint16_t zde_bytes =nvme_ns_zdes_bytes (ns); +int64_t offset = ns->zns.info.num_zones * +sizeof (NvmeZoneDescriptor); + ns->zns.info.zones[i].zde =g_malloc (zde_bytes); + + ret =blk_pread (blk, offset + i * zde_bytes, + ns->zns.info.zones[i].zde, zde_bytes); +if (ret <0 ) { +error_setg_errno (errp, -ret,"blk_pread: " ); +return ret; + }else if (ret != zde_bytes) { +error_setg (errp,"blk_pread: short read" ); +return -1 ; + } + } +switch (nvme_zs (zone)) {case NVME_ZS_ZSE :case NVME_ZS_ZSF :
@@ -185,7 +207,8 @@ static int nvme_ns_setup_blk_zoneinfo(NvmeNamespace *ns, Error **errp)continue ;case NVME_ZS_ZSC : -if (nvme_wp (zone) ==nvme_zslba (zone)) { +if (nvme_wp (zone) ==nvme_zslba (zone) && + !NVME_ZA_ZDEV (zd->za)) {nvme_zs_set (zone,NVME_ZS_ZSE );continue ; }
@@ -231,6 +254,7 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns)for (int i =0 ; i <= id_ns->nlbaf; i++) { id_ns_zns->lbafe[i].zsze =cpu_to_le64 (pow2ceil (ns->params.zns.zcap)); + id_ns_zns->lbafe[i].zdes = ns->params.zns.zdes; } ns->zns.info.num_zones =nvme_ns_nlbas (ns) /nvme_ns_zsze (ns);
@@ -472,6 +496,7 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT8 ("iocs" , NvmeNamespace, params.iocs,0x0 ),DEFINE_PROP_DRIVE ("zns.zoneinfo" , NvmeNamespace, zns.info.blk),DEFINE_PROP_UINT64 ("zns.zcap" , NvmeNamespace, params.zns.zcap,0 ), +DEFINE_PROP_UINT8 ("zns.zdes" , NvmeNamespace, params.zns.zdes,0 ),DEFINE_PROP_UINT16 ("zns.zoc" , NvmeNamespace, params.zns.zoc,0 ),DEFINE_PROP_UINT16 ("zns.ozcs" , NvmeNamespace, params.zns.ozcs,0 ), DEFINE_PROP_END_OF_LIST(),
hw/block/nvme-ns.h +8 -1
@@ -26,13 +26,15 @@ typedef struct NvmeNamespaceParams {struct {uint64_t zcap; +uint8_t zdes;uint16_t zoc;uint16_t ozcs; } zns; } NvmeNamespaceParams;typedef struct NvmeZone { -NvmeZoneDescriptor zd; +NvmeZoneDescriptor zd; +uint8_t *zde;uint64_t wp_staging; }NvmeZone ;
@@ -152,6 +154,11 @@ static inline void nvme_zs_set(NvmeZone *zone, NvmeZoneState zs) zone->zd.zs = zs <<4 ; } +static inline size_t nvme_ns_zdes_bytes (NvmeNamespace *ns) +{ +return ns->params.zns.zdes <<6 ; +} +static inline bool nvme_ns_zone_wp_valid (NvmeZone *zone) {switch (nvme_zs (zone)) {
hw/block/nvme.c +130 -0
142 lines not on this page · raw message
hw/block/nvme.h +6 -0
@@ -47,6 +47,12 @@ static const NvmeEffectsLog nvme_effects[] = { NVME_EFFECTS_LBCC, }, }, + + [NVME_IOCS_ZONED] = { + .iocs = { + [NVME_CMD_ZONE_MGMT_RECV] = NVME_EFFECTS_CSUPP, + } + }, }; typedef struct NvmeAsyncEvent {
hw/block/trace-events +1 -0
@@ -49,6 +49,7 @@ pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode) "cid %"PRIu16" s pci_nvme_rw(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" %s nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_write_zeroes(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba %"PRIu64" nlb %"PRIu32"" +pci_nvme_zone_mgmt_recv(uint16_t cid, uint32_t nsid, uint64_t slba, uint64_t len, uint8_t zra, uint8_t zrasp, uint8_t zrasf) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" len %"PRIu64" zra 0x%"PRIx8" zrasp 0x%"PRIx8" zrasf 0x%"PRIx8"" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d" pci_nvme_del_sq(uint16_t qid) "deleting submission queue sqid=%"PRIu16""
include/block/nvme.h +5 -0
@@ -746,6 +746,11 @@ typedef enum NvmeZoneManagementRecvActionSpecificField {#define NVME_CMD_ZONE_MGMT_RECEIVE_PARTIAL 0x1 +typedef struct NvmeZoneReportHeader { +uint64_t num_zones; +uint8_t rsvd[56 ]; +}NvmeZoneReportHeader ; + typedef struct NvmeDsmCmd { uint8_t opcode; uint8_t flags;
Signature · 1 line
2.27.0
PATCH 5/10 hw/block/nvme: add the zone management send command
In reply to Klaus JensenAdd the Zone Management Send command.
--- hw/block/nvme.c | 461 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme.h | 4 + hw/block/trace-events | 12 ++ 3 files changed, 477 insertions(+)
hw/block/nvme.c +461 -0
@@ -748,6 +748,11 @@ static void nvme_submit_aio(NvmeAIO *aio) }break ; + +case NVME_AIO_OPC_DISCARD : + aio->aiocb =blk_aio_pdiscard (blk, aio->offset, aio->len, nvme_aio_cb, + aio); +break ; } }
@@ -1142,6 +1147,46 @@ static void nvme_update_zone_info(NvmeNamespace *ns, NvmeRequest *req,nvme_req_add_aio (req, aio); } +static void nvme_update_zone_descr (NvmeNamespace *ns,NvmeRequest *req, +NvmeZone *zone) +{ +uint64_t zslba =-1 ; +QEMUIOVector *iov =g_new0 (QEMUIOVector,1 ); +NvmeAIO *aio =g_new0 (NvmeAIO,1 ); + + *aio = (NvmeAIO ) { + .opc =NVME_AIO_OPC_WRITE , + .blk = ns->zns.info.blk, + .payload = iov, + .offset = ns->zns.info.num_zones *sizeof (NvmeZoneDescriptor), + .req = req, + .flags =NVME_AIO_INTERNAL , + }; + +qemu_iovec_init (iov,1 ); + +if (zone) { + zslba =nvme_zslba (zone); +trace_pci_nvme_update_zone_descr (nvme_cid (req), ns->params.nsid, + zslba); + + aio->offset +=nvme_ns_zone_idx (ns, zslba) *nvme_ns_zdes_bytes (ns); +qemu_iovec_add (iov, zone->zde,nvme_ns_zdes_bytes (ns)); + }else { +trace_pci_nvme_update_zone_descr (nvme_cid (req), ns->params.nsid, + zslba); + +for (int i =0 ; i < ns->zns.info.num_zones; i++) { +qemu_iovec_add (iov, ns->zns.info.zones[i].zde, +nvme_ns_zdes_bytes (ns)); + } + } + + aio->len = iov->size; + +nvme_req_add_aio (req, aio); +} +static void nvme_aio_write_cb (NvmeAIO *aio,void *opaque,int ret) {NvmeRequest *req = aio->req;
@@ -1206,6 +1251,49 @@ static void nvme_rw_cb(NvmeRequest *req, void *opaque)nvme_enqueue_req_completion (cq, req); } +static void nvme_zone_mgmt_send_reset_cb (NvmeRequest *req,void *opaque) +{ +NvmeSQueue *sq = req->sq; +NvmeCtrl *n = sq->ctrl; +NvmeCQueue *cq = n->cq[sq->cqid]; +NvmeNamespace *ns = req->ns; + +trace_pci_nvme_zone_mgmt_send_reset_cb (nvme_cid (req),nvme_nsid (ns)); + +g_free (opaque); + +nvme_enqueue_req_completion (cq, req); +} + +static void nvme_aio_zone_reset_cb (NvmeAIO *aio,void *opaque,int ret) +{ +NvmeRequest *req = aio->req; +NvmeZone *zone = opaque; +NvmeNamespace *ns = req->ns; + +uint64_t zslba =nvme_zslba (zone); +uint64_t zcap =nvme_zcap (zone); + +if (ret) { +return ; + } + +trace_pci_nvme_aio_zone_reset_cb (nvme_cid (req), ns->params.nsid, zslba); + +nvme_zs_set (zone,NVME_ZS_ZSE ); +NVME_ZA_CLEAR (zone->zd.za); + + zone->zd.wp = zone->zd.zslba; + zone->wp_staging = zslba; + +nvme_update_zone_info (ns, req, zone); + +if (ns->blk_state) { +bitmap_clear (ns->utilization, zslba, zcap); +nvme_ns_update_util (ns, zslba, zcap, req); + } +} +static void nvme_aio_cb (void *opaque,int ret) {NvmeAIO *aio = opaque;
@@ -1336,6 +1424,377 @@ static uint16_t nvme_flush(NvmeCtrl *n, NvmeRequest *req)return NVME_NO_COMPLETE ; } +static uint16_t nvme_zone_mgmt_send_close (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeNamespace *ns = req->ns; + +trace_pci_nvme_zone_mgmt_send_close (nvme_cid (req),nvme_nsid (ns), +nvme_zslba (zone),nvme_zs_str (zone)); + + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : +nvme_zs_set (zone,NVME_ZS_ZSC ); + +nvme_update_zone_info (ns, req, zone); + +return NVME_NO_COMPLETE ; + +case NVME_ZS_ZSC : +return NVME_SUCCESS ; + +default : +break ; + } + +trace_pci_nvme_err_invalid_zone_condition (nvme_cid (req),nvme_zslba (zone), +nvme_zs (zone)); +return NVME_INVALID_ZONE_STATE_TRANSITION |NVME_DNR ; +} + +static uint16_t nvme_zone_mgmt_send_finish (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeNamespace *ns = req->ns; + +trace_pci_nvme_zone_mgmt_send_finish (nvme_cid (req),nvme_nsid (ns), +nvme_zslba (zone),nvme_zs_str (zone)); + + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : +case NVME_ZS_ZSC : +case NVME_ZS_ZSE : +nvme_zs_set (zone,NVME_ZS_ZSF ); + +nvme_update_zone_info (ns, req, zone); + +return NVME_NO_COMPLETE ; + +case NVME_ZS_ZSF : +return NVME_SUCCESS ; + +default : +break ; + } + +trace_pci_nvme_err_invalid_zone_condition (nvme_cid (req),nvme_zslba (zone), +nvme_zs (zone)); +return NVME_INVALID_ZONE_STATE_TRANSITION |NVME_DNR ; +} + +static uint16_t nvme_zone_mgmt_send_open (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeNamespace *ns = req->ns; + +trace_pci_nvme_zone_mgmt_send_open (nvme_cid (req),nvme_nsid (ns), +nvme_zslba (zone),nvme_zs_str (zone)); + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSE : +case NVME_ZS_ZSC : +case NVME_ZS_ZSIO : +nvme_zs_set (zone,NVME_ZS_ZSEO ); + +nvme_update_zone_info (ns, req, zone); +return NVME_NO_COMPLETE ; + +case NVME_ZS_ZSEO : +return NVME_SUCCESS ; + +default : +break ; + } + +trace_pci_nvme_err_invalid_zone_condition (nvme_cid (req),nvme_zslba (zone), +nvme_zs (zone)); +return NVME_INVALID_ZONE_STATE_TRANSITION |NVME_DNR ; +} + +static uint16_t nvme_zone_mgmt_send_reset (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeAIO *aio; +NvmeNamespace *ns = req->ns; +uint64_t zslba =nvme_zslba (zone); +uint64_t zcap =nvme_zcap (zone); +uint8_t lbads =nvme_ns_lbads (ns); + +trace_pci_nvme_zone_mgmt_send_reset (nvme_cid (req),nvme_nsid (ns), +nvme_zslba (zone),nvme_zs_str (zone)); + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : +case NVME_ZS_ZSC : +case NVME_ZS_ZSF : + aio =g_new0 (NvmeAIO,1 ); + + *aio = (NvmeAIO ) { + .opc =NVME_AIO_OPC_DISCARD , + .blk = ns->blk, + .offset = zslba << lbads, + .len = zcap << lbads, + .req = req, + .cb = nvme_aio_zone_reset_cb, + .cb_arg = zone, + }; + +nvme_req_add_aio (req, aio); +nvme_req_set_cb (req, nvme_zone_mgmt_send_reset_cb,NULL ); + +return NVME_NO_COMPLETE ; + +case NVME_ZS_ZSE : +return NVME_SUCCESS ; + +case NVME_ZS_ZSRO : +nvme_zs_set (zone,NVME_ZS_ZSO ); + +nvme_update_zone_info (ns, req, zone); + +return NVME_NO_COMPLETE ; + +default : +break ; + } + +trace_pci_nvme_err_invalid_zone_condition (nvme_cid (req),nvme_zslba (zone), +nvme_zs (zone)); +return NVME_INVALID_ZONE_STATE_TRANSITION |NVME_DNR ; +} + +static uint16_t nvme_zone_mgmt_send_offline (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeNamespace *ns = req->ns; + +trace_pci_nvme_zone_mgmt_send_offline (nvme_cid (req),nvme_nsid (ns), +nvme_zslba (zone),nvme_zs_str (zone)); + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSRO : +nvme_zs_set (zone,NVME_ZS_ZSO ); + +nvme_update_zone_info (ns, req, zone); +return NVME_NO_COMPLETE ; + +case NVME_ZS_ZSO : +return NVME_SUCCESS ; + +default : +break ; + } + +trace_pci_nvme_err_invalid_zone_condition (nvme_cid (req),nvme_zslba (zone), +nvme_zs (zone)); +return NVME_INVALID_ZONE_STATE_TRANSITION |NVME_DNR ; +} + +static uint16_t nvme_zone_mgmt_send_set_zde (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeNamespace *ns = req->ns; +uint16_t status; + +trace_pci_nvme_zone_mgmt_send_set_zde (nvme_cid (req),nvme_nsid (ns), +nvme_zslba (zone),nvme_zs_str (zone)); + +if (nvme_zs (zone) !=NVME_ZS_ZSE ) { +trace_pci_nvme_err_invalid_zone_condition (nvme_cid (req), +nvme_zslba (zone), +nvme_zs (zone)); +return NVME_INVALID_ZONE_STATE_TRANSITION |NVME_DNR ; + } + +nvme_zs_set (zone,NVME_ZS_ZSEO ); + + status =nvme_dma (n, zone->zde,nvme_ns_zdes_bytes (ns), +DMA_DIRECTION_TO_DEVICE , req); +if (status) { +return status; + } + +NVME_ZA_SET_ZDEV (zone->zd.za,0x1 ); +nvme_update_zone_descr (ns, req, zone); +nvme_update_zone_info (ns, req, zone); + +return NVME_NO_COMPLETE ; +} + +static uint16_t nvme_zone_mgmt_send_all (NvmeCtrl *n,NvmeRequest *req) +{ +NvmeZoneManagementSendCmd *send = (NvmeZoneManagementSendCmd *) &req->cmd; +NvmeNamespace *ns = req->ns; +NvmeZone *zone; +NvmeZoneState zs; + +uint16_t status =NVME_SUCCESS ; + +trace_pci_nvme_zone_mgmt_send_all (nvme_cid (req),nvme_nsid (ns), send->zsa); + +switch (send->zsa) { +case NVME_CMD_ZONE_MGMT_SEND_SET_ZDE : +return NVME_INVALID_FIELD |NVME_DNR ; + +case NVME_CMD_ZONE_MGMT_SEND_CLOSE : +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zone = &ns->zns.info.zones[i]; + zs =nvme_zs (zone); + +switch (zs) { +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : + status =nvme_zone_mgmt_send_close (n, req, zone); +if (status && status !=NVME_NO_COMPLETE ) { + goto err_out; + } + +default : +continue ; + } + } + +break ; + +case NVME_CMD_ZONE_MGMT_SEND_FINISH : +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zone = &ns->zns.info.zones[i]; + zs =nvme_zs (zone); + +switch (zs) { +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : +case NVME_ZS_ZSC : + status =nvme_zone_mgmt_send_finish (n, req, zone); +if (status && status !=NVME_NO_COMPLETE ) { + goto err_out; + } + +default : +continue ; + } + } + +break ; + +case NVME_CMD_ZONE_MGMT_SEND_OPEN : +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zone = &ns->zns.info.zones[i]; + zs =nvme_zs (zone); + +if (zs ==NVME_ZS_ZSC ) { + status =nvme_zone_mgmt_send_open (n, req, zone); +if (status && status !=NVME_NO_COMPLETE ) { + goto err_out; + } + } + } + +break ; + +case NVME_CMD_ZONE_MGMT_SEND_RESET : +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zone = &ns->zns.info.zones[i]; + zs =nvme_zs (zone); + +switch (zs) { +case NVME_ZS_ZSIO : +case NVME_ZS_ZSEO : +case NVME_ZS_ZSC : +case NVME_ZS_ZSF : + status =nvme_zone_mgmt_send_reset (n, req, zone); +if (status && status !=NVME_NO_COMPLETE ) { + goto err_out; + } + +default : +continue ; + } + } + +break ; + +case NVME_CMD_ZONE_MGMT_SEND_OFFLINE : +for (int i =0 ; i < ns->zns.info.num_zones; i++) { + zone = &ns->zns.info.zones[i]; + zs =nvme_zs (zone); + +if (zs ==NVME_ZS_ZSRO ) { + status =nvme_zone_mgmt_send_offline (n, req, zone); +if (status && status !=NVME_NO_COMPLETE ) { + goto err_out; + } + } + } + +break ; + } + +return status; + +err_out: + req->status = status; + +if (!QTAILQ_EMPTY (&req->aio_tailq)) { +return NVME_NO_COMPLETE ; + } + +return status; +} + +static uint16_t nvme_zone_mgmt_send (NvmeCtrl *n,NvmeRequest *req) +{ +NvmeZoneManagementSendCmd *send = (NvmeZoneManagementSendCmd *) &req->cmd; +NvmeZoneManagementSendAction zsa = send->zsa; +NvmeNamespace *ns = req->ns; +NvmeZone *zone; +uint64_t zslba =le64_to_cpu (send->slba); + +if (!nvme_ns_zoned (ns)) { +return NVME_INVALID_OPCODE |NVME_DNR ; + } + +trace_pci_nvme_zone_mgmt_send (nvme_cid (req), ns->params.nsid, zslba, zsa, + send->zsflags); + +if (NVME_CMD_ZONE_MGMT_SEND_SELECT_ALL (send->zsflags)) { +return nvme_zone_mgmt_send_all (n, req); + } + +if (zslba & (nvme_ns_zsze (ns) -1 )) { +trace_pci_nvme_err_invalid_zslba (nvme_cid (req), zslba); +return NVME_INVALID_FIELD |NVME_DNR ; + } + + zone =nvme_ns_get_zone (ns, zslba); +if (!zone) { +trace_pci_nvme_err_invalid_zone (nvme_cid (req), zslba); +return NVME_INVALID_FIELD |NVME_DNR ; + } + +switch (zsa) { +case NVME_CMD_ZONE_MGMT_SEND_CLOSE : +return nvme_zone_mgmt_send_close (n, req, zone); +case NVME_CMD_ZONE_MGMT_SEND_FINISH : +return nvme_zone_mgmt_send_finish (n, req, zone); +case NVME_CMD_ZONE_MGMT_SEND_OPEN : +return nvme_zone_mgmt_send_open (n, req, zone); +case NVME_CMD_ZONE_MGMT_SEND_RESET : +return nvme_zone_mgmt_send_reset (n, req, zone); +case NVME_CMD_ZONE_MGMT_SEND_OFFLINE : +return nvme_zone_mgmt_send_offline (n, req, zone); +case NVME_CMD_ZONE_MGMT_SEND_SET_ZDE : +return nvme_zone_mgmt_send_set_zde (n, req, zone); + } + +return NVME_INVALID_FIELD |NVME_DNR ; +} +static uint16_t nvme_do_write_zeroes (NvmeCtrl *n,NvmeRequest *req) {NvmeAIO *aio;
@@ -1679,6 +2138,8 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest *req)case NVME_CMD_WRITE :case NVME_CMD_WRITE_ZEROES :return nvme_rwz (n, req); +case NVME_CMD_ZONE_MGMT_SEND : +return nvme_zone_mgmt_send (n, req);case NVME_CMD_ZONE_MGMT_RECV :return nvme_zone_mgmt_recv (n, req);default :
hw/block/nvme.h +4 -0
@@ -51,6 +51,8 @@ static const NvmeEffectsLog nvme_effects[] = { [NVME_IOCS_ZONED ] = { .iocs = { [NVME_CMD_ZONE_MGMT_RECV ] =NVME_EFFECTS_CSUPP , + [NVME_CMD_ZONE_MGMT_SEND ] =NVME_EFFECTS_CSUPP | +NVME_EFFECTS_LBCC , } }, };
@@ -127,6 +129,7 @@ typedef enum NvmeAIOOp {NVME_AIO_OPC_READ =0x2 ,NVME_AIO_OPC_WRITE =0x3 ,NVME_AIO_OPC_WRITE_ZEROES =0x4 , +NVME_AIO_OPC_DISCARD =0x5 , } NvmeAIOOp; typedef enum NvmeAIOFlags {
@@ -164,6 +167,7 @@ static inline const char *nvme_aio_opc_str(NvmeAIO *aio)case NVME_AIO_OPC_READ :return "NVME_AIO_OP_READ" ;case NVME_AIO_OPC_WRITE :return "NVME_AIO_OP_WRITE" ;case NVME_AIO_OPC_WRITE_ZEROES :return "NVME_AIO_OP_WRITE_ZEROES" ; +case NVME_AIO_OPC_DISCARD :return "NVME_AIO_OP_DISCARD" ;default :return "NVME_AIO_OP_UNKNOWN" ; } }
hw/block/trace-events +12 -0
@@ -43,12 +43,22 @@ pci_nvme_aio_cb(uint16_t cid, void *aio, const char *blkname, uint64_t offset, c pci_nvme_aio_discard_cb(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_aio_write_cb(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" nlb %"PRIu32"" pci_nvme_aio_zone_write_cb(uint16_t cid, uint64_t lba, uint32_t nlb, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" wp 0x%"PRIx64"" +pci_nvme_aio_zone_reset_cb(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64"" pci_nvme_zone_advance_wp(uint16_t cid, uint64_t lba, uint32_t nlb, uint64_t wp_old, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" wp_old 0x%"PRIx64" wp 0x%"PRIx64"" pci_nvme_io_cmd(uint16_t cid, uint32_t nsid, uint16_t sqid, uint8_t opcode) "cid %"PRIu16" nsid %"PRIu32" sqid %"PRIu16" opc 0x%"PRIx8"" pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode) "cid %"PRIu16" sqid %"PRIu16" opc 0x%"PRIx8"" pci_nvme_rw(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" %s nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_write_zeroes(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba %"PRIu64" nlb %"PRIu32"" +pci_nvme_zone_mgmt_send(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t zsa, uint8_t zsflags) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zsa 0x%"PRIx8" zsflags 0x%"PRIx8"" +pci_nvme_zone_mgmt_send_all(uint16_t cid, uint32_t nsid, uint8_t za) "cid %"PRIu16" nsid %"PRIu32" za 0x%"PRIx8"" +pci_nvme_zone_mgmt_send_close(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_finish(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_open(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_reset(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_reset_cb(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_mgmt_send_offline(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_zone_mgmt_send_set_zde(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" pci_nvme_zone_mgmt_recv(uint16_t cid, uint32_t nsid, uint64_t slba, uint64_t len, uint8_t zra, uint8_t zrasp, uint8_t zrasf) "cid %"PRIu16" nsid %"PRIu32" slba 0x%"PRIx64" len %"PRIu64" zra 0x%"PRIx8" zrasp 0x%"PRIx8" zrasf 0x%"PRIx8"" pci_nvme_create_sq(uint64_t addr, uint16_t sqid, uint16_t cqid, uint16_t qsize, uint16_t qflags) "create submission queue, addr=0x%"PRIx64", sqid=%"PRIu16", cqid=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16"" pci_nvme_create_cq(uint64_t addr, uint16_t cqid, uint16_t vector, uint16_t size, uint16_t qflags, int ien) "create completion queue, addr=0x%"PRIx64", cqid=%"PRIu16", vector=%"PRIu16", qsize=%"PRIu16", qflags=%"PRIu16", ien=%d"
@@ -85,6 +95,7 @@ pci_nvme_mmio_doorbell_sq(uint16_t sqid, uint16_t new_tail) "cqid %"PRIu16" new_ pci_nvme_ns_update_util(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_zone_pending_writes(uint16_t cid, uint64_t zslba, uint64_t wp, uint64_t wp_staging) "cid %"PRIu16" zslba 0x%"PRIx64" wp 0x%"PRIx64" wp_staging 0x%"PRIx64"" pci_nvme_update_zone_info(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64"" +pci_nvme_update_zone_descr(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64"" pci_nvme_mmio_intm_set(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask set, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_intm_clr(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask clr, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_cfg(uint64_t data) "wrote MMIO, config controller config=0x%"PRIx64""
@@ -138,6 +149,7 @@ pci_nvme_err_invalid_setfeat(uint32_t dw10) "invalid set features, dw10=0x%"PRIx pci_nvme_err_invalid_log_page(uint16_t cid, uint16_t lid) "cid %"PRIu16" lid 0x%"PRIx16"" pci_nvme_err_invalid_zone(uint16_t cid, uint64_t lba) "cid %"PRIu16" lba 0x%"PRIx64"" pci_nvme_err_invalid_zone_condition(uint16_t cid, uint64_t zslba, uint8_t condition) "cid %"PRIu16" zslba 0x%"PRIx64" condition 0x%"PRIx8"" +pci_nvme_err_invalid_zslba(uint16_t cid, uint64_t zslba) "cid %"PRIu16" zslba 0x%"PRIx64"" pci_nvme_err_startfail_cq(void) "nvme_start_ctrl failed because there are non-admin completion queues" pci_nvme_err_startfail_sq(void) "nvme_start_ctrl failed because there are non-admin submission queues" pci_nvme_err_startfail_nbarasq(void) "nvme_start_ctrl failed because the admin submission queue address is null"
Signature · 1 line
2.27.0
PATCH 6/10 hw/block/nvme: add the zone append command
In reply to Klaus JensenAdd the Zone Append command.
--- hw/block/nvme.c | 106 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme.h | 3 ++ hw/block/trace-events | 2 + 3 files changed, 111 insertions(+)
hw/block/nvme.c +106 -0
@@ -1294,6 +1294,12 @@ static void nvme_aio_zone_reset_cb(NvmeAIO *aio, void *opaque, int ret) } } +static void nvme_zone_append_cb (NvmeRequest *req,void *opaque) +{ +trace_pci_nvme_zone_append_cb (nvme_cid (req),le64_to_cpu (req->cqe.qw0)); +nvme_rw_cb (req, opaque); +} +static void nvme_aio_cb (void *opaque,int ret) {NvmeAIO *aio = opaque;
@@ -1424,6 +1430,104 @@ static uint16_t nvme_flush(NvmeCtrl *n, NvmeRequest *req)return NVME_NO_COMPLETE ; } +static uint16_t nvme_do_zone_append (NvmeCtrl *n,NvmeRequest *req, +NvmeZone *zone) +{ +NvmeAIO *aio; +NvmeNamespace *ns = req->ns; + +uint64_t zslba =nvme_zslba (zone); +uint64_t wp = zone->wp_staging; + +size_t len; +uint16_t status; + + req->cqe.qw0 =cpu_to_le64 (wp); + req->slba = wp; + + len = req->nlb <<nvme_ns_lbads (ns); + +trace_pci_nvme_zone_append (nvme_cid (req), zslba, wp, req->nlb); + + status =nvme_check_rw (n, req); +if (status) { + goto invalid; + } + + status =nvme_check_zone_write (n, req->slba, req->nlb, req, zone); +if (status) { + goto invalid; + } + +switch (nvme_zs (zone)) { +case NVME_ZS_ZSE : +case NVME_ZS_ZSC : +nvme_zs_set (zone,NVME_ZS_ZSIO ); +default : +break ; + } + + status =nvme_map (n, len, req); +if (status) { + goto invalid; + } + + aio =g_new0 (NvmeAIO,1 ); + *aio = (NvmeAIO ) { + .opc =NVME_AIO_OPC_WRITE , + .blk = ns->blk, + .offset = req->slba <<nvme_ns_lbads (ns), + .req = req, + .cb = nvme_aio_zone_write_cb, + .cb_arg = zone, + }; + +if (req->qsg.sg) { + aio->len = req->qsg.size; + aio->flags |=NVME_AIO_DMA ; + }else { + aio->len = req->iov.size; + } + +nvme_req_add_aio (req, aio); +nvme_req_set_cb (req, nvme_zone_append_cb, zone); + + zone->wp_staging += req->nlb; + +return NVME_NO_COMPLETE ; + +invalid: +block_acct_invalid (blk_get_stats (ns->blk),BLOCK_ACCT_WRITE ); +return status; +} + +static uint16_t nvme_zone_append (NvmeCtrl *n,NvmeRequest *req) +{ +NvmeZone *zone; +NvmeZoneAppendCmd *zappend = (NvmeZoneAppendCmd *) &req->cmd; +NvmeNamespace *ns = req->ns; +uint64_t zslba =le64_to_cpu (zappend->zslba); + +if (!nvme_ns_zoned (ns)) { +return NVME_INVALID_OPCODE |NVME_DNR ; + } + +if (zslba & (nvme_ns_zsze (ns) -1 )) { +trace_pci_nvme_err_invalid_zslba (nvme_cid (req), zslba); +return NVME_INVALID_FIELD |NVME_DNR ; + } + + req->nlb =le16_to_cpu (zappend->nlb) +1 ; + + zone =nvme_ns_get_zone (ns, zslba); +if (!zone) { +trace_pci_nvme_err_invalid_zone (nvme_cid (req), req->slba); +return NVME_INVALID_FIELD |NVME_DNR ; + } + +return nvme_do_zone_append (n, req, zone); +} +static uint16_t nvme_zone_mgmt_send_close (NvmeCtrl *n,NvmeRequest *req,NvmeZone *zone) {
@@ -2142,6 +2246,8 @@ static uint16_t nvme_io_cmd(NvmeCtrl *n, NvmeRequest *req)return nvme_zone_mgmt_send (n, req);case NVME_CMD_ZONE_MGMT_RECV :return nvme_zone_mgmt_recv (n, req); +case NVME_CMD_ZONE_APPEND : +return nvme_zone_append (n, req);default :trace_pci_nvme_err_invalid_opc (req->cmd.opcode);return NVME_INVALID_OPCODE |NVME_DNR ;
hw/block/nvme.h +3 -0
@@ -53,6 +53,8 @@ static const NvmeEffectsLog nvme_effects[] = { [NVME_CMD_ZONE_MGMT_RECV] = NVME_EFFECTS_CSUPP, [NVME_CMD_ZONE_MGMT_SEND ] =NVME_EFFECTS_CSUPP | NVME_EFFECTS_LBCC, + [NVME_CMD_ZONE_APPEND ] =NVME_EFFECTS_CSUPP | + NVME_EFFECTS_LBCC, } }, };
@@ -177,6 +179,7 @@ static inline bool nvme_req_is_write(NvmeRequest *req)switch (req->cmd.opcode) {case NVME_CMD_WRITE :case NVME_CMD_WRITE_ZEROES : +case NVME_CMD_ZONE_APPEND :return true;default :return false;
hw/block/trace-events +2 -0
@@ -50,6 +50,8 @@ pci_nvme_admin_cmd(uint16_t cid, uint16_t sqid, uint8_t opcode) "cid %"PRIu16" s pci_nvme_rw(uint16_t cid, const char *verb, uint32_t nsid, uint32_t nlb, uint64_t count, uint64_t lba) "cid %"PRIu16" %s nsid %"PRIu32" nlb %"PRIu32" count %"PRIu64" lba 0x%"PRIx64"" pci_nvme_rw_cb(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_write_zeroes(uint16_t cid, uint32_t nsid, uint64_t slba, uint32_t nlb) "cid %"PRIu16" nsid %"PRIu32" slba %"PRIu64" nlb %"PRIu32"" +pci_nvme_zone_append(uint16_t cid, uint64_t zslba, uint64_t wp, uint16_t nlb) "cid %"PRIu16" zslba 0x%"PRIx64" wp 0x%"PRIx64" nlb %"PRIu16"" +pci_nvme_zone_append_cb(uint16_t cid, uint64_t slba) "cid %"PRIu16" slba 0x%"PRIx64"" pci_nvme_zone_mgmt_send(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t zsa, uint8_t zsflags) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zsa 0x%"PRIx8" zsflags 0x%"PRIx8"" pci_nvme_zone_mgmt_send_all(uint16_t cid, uint32_t nsid, uint8_t za) "cid %"PRIu16" nsid %"PRIu32" za 0x%"PRIx8"" pci_nvme_zone_mgmt_send_close(uint16_t cid, uint32_t nsid, uint64_t zslba, const char *zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\""
Signature · 1 line
2.27.0
PATCH 7/10 hw/block/nvme: track and enforce zone resources
In reply to Klaus JensenMove all zone transition rules to a single state machine that also manages zone resources.
--- hw/block/nvme-ns.c | 17 ++- hw/block/nvme-ns.h | 7 ++ hw/block/nvme.c | 304 ++++++++++++++++++++++++++++++++------------- 3 files changed, 242 insertions(+), 86 deletions(-)
hw/block/nvme-ns.c +15 -2
@@ -262,8 +262,13 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) id_ns->ncap = ns->zns.info.num_zones * ns->params.zns.zcap; - id_ns_zns->mar =0xffffffff ; - id_ns_zns->mor =0xffffffff ; + id_ns_zns->mar =cpu_to_le32 (ns->params.zns.mar); + id_ns_zns->mor =cpu_to_le32 (ns->params.zns.mor); + + ns->zns.resources.active = ns->params.zns.mar !=0xffffffff ? + ns->params.zns.mar +1 : ns->zns.info.num_zones; + ns->zns.resources.open = ns->params.zns.mor !=0xffffffff ? + ns->params.zns.mor +1 : ns->zns.info.num_zones; }static void nvme_ns_init (NvmeNamespace *ns)
@@ -426,6 +431,12 @@ static int nvme_ns_check_constraints(NvmeCtrl *n, NvmeNamespace *ns, Errorreturn -1 ; } +if (ns->params.zns.mor > ns->params.zns.mar) { +error_setg (errp,"maximum open resources (MOR) must be less " +"than or equal to maximum active resources (MAR)" ); +return -1 ; + } +break ;default :
@@ -499,6 +510,8 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT8 ("zns.zdes" , NvmeNamespace, params.zns.zdes,0 ),DEFINE_PROP_UINT16 ("zns.zoc" , NvmeNamespace, params.zns.zoc,0 ),DEFINE_PROP_UINT16 ("zns.ozcs" , NvmeNamespace, params.zns.ozcs,0 ), +DEFINE_PROP_UINT32 ("zns.mar" , NvmeNamespace, params.zns.mar,0xffffffff ), +DEFINE_PROP_UINT32 ("zns.mor" , NvmeNamespace, params.zns.mor,0xffffffff ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme-ns.h +7 -0
@@ -29,6 +29,8 @@ typedef struct NvmeNamespaceParams {uint8_t zdes;uint16_t zoc;uint16_t ozcs; +uint32_t mar; +uint32_t mor; } zns; } NvmeNamespaceParams;
@@ -63,6 +65,11 @@ typedef struct NvmeNamespace {uint64_t num_zones;NvmeZone *zones; } info; + +struct { +uint32_t open; +uint32_t active; + } resources; } zns; } NvmeNamespace;
hw/block/nvme.c +220 -84
438 lines not on this page · raw message
Signature · 1 line
2.27.0
PATCH 8/10 hw/block/nvme: allow open to close transitions by controller
In reply to Klaus JensenAllow the controller to release open resources by transitioning implicitly and explicitly opened zones to closed. This is done using a naive "least recently opened" strategy. Some workloads may behave very badly with this, but for the purpose of testing how software deals with this it is acceptable for now.
--- hw/block/nvme-ns.c | 3 + hw/block/nvme-ns.h | 5 ++ hw/block/nvme.c | 176 +++++++++++++++++++++++++++++++----------- hw/block/nvme.h | 5 ++ hw/block/trace-events | 5 ++ 5 files changed, 147 insertions(+), 47 deletions(-)
hw/block/nvme-ns.c +3 -0
@@ -269,6 +269,9 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) ns->params.zns.mar + 1 : ns->zns.info.num_zones; ns->zns.resources.open = ns->params.zns.mor !=0xffffffff ? ns->params.zns.mor +1 : ns->zns.info.num_zones; + +QTAILQ_INIT (&ns->zns.resources.lru_open); +QTAILQ_INIT (&ns->zns.resources.lru_active); }static void nvme_ns_init (NvmeNamespace *ns)
hw/block/nvme-ns.h +5 -0
@@ -39,6 +39,8 @@ typedef struct NvmeZone {uint8_t *zde;uint64_t wp_staging; + +QTAILQ_ENTRY (NvmeZone) lru_entry; } NvmeZone; typedef struct NvmeNamespace {
@@ -69,6 +71,9 @@ typedef struct NvmeNamespace {struct {uint32_t open;uint32_t active; + + QTAILQ_HEAD(, NvmeZone) lru_open; + QTAILQ_HEAD(, NvmeZone) lru_active; } resources; } zns; } NvmeNamespace;
hw/block/nvme.c +129 -47
388 lines not on this page · raw message
hw/block/nvme.h +5 -0
@@ -312,6 +312,11 @@ static inline uint16_t nvme_sqid(NvmeRequest *req) return le16_to_cpu(req->sq->sqid); } +static inline NvmeCtrl *nvme_ctrl (NvmeRequest *req) +{ +return req->sq->ctrl; +} +int nvme_register_namespace (NvmeCtrl *n,NvmeNamespace *ns,Error **errp);#endif /* HW_NVME_H */
hw/block/trace-events +5 -0
@@ -98,6 +98,9 @@ pci_nvme_ns_update_util(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu3 pci_nvme_zone_pending_writes(uint16_t cid, uint64_t zslba, uint64_t wp, uint64_t wp_staging) "cid %"PRIu16" zslba 0x%"PRIx64" wp 0x%"PRIx64" wp_staging 0x%"PRIx64"" pci_nvme_update_zone_info(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64"" pci_nvme_update_zone_descr(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64"" +pci_nvme_zone_zrm_transition(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t from, uint8_t to) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" from 0x%"PRIx8" to 0x%"PRIx8"" +pci_nvme_zone_zrm_candidate(uint16_t cid, uint32_t nsid, uint64_t zslba, uint64_t wp, uint8_t zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" wp 0x%"PRIx64" zc 0x%"PRIx8"" +pci_nvme_zone_zrm_release_open(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_mmio_intm_set(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask set, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_intm_clr(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask clr, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_cfg(uint64_t data) "wrote MMIO, config controller config=0x%"PRIx64""
@@ -121,6 +124,8 @@ pci_nvme_err_zone_is_full(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0x%"PR pci_nvme_err_zone_is_read_only(uint16_t cid, uint64_t slba) "cid %"PRIu16" lba 0x%"PRIx64"" pci_nvme_err_zone_invalid_write(uint16_t cid, uint64_t slba, uint64_t wp) "cid %"PRIu16" lba 0x%"PRIx64" wp 0x%"PRIx64"" pci_nvme_err_zone_boundary(uint16_t cid, uint64_t slba, uint32_t nlb, uint64_t zcap) "cid %"PRIu16" lba 0x%"PRIx64" nlb %"PRIu32" zcap 0x%"PRIx64"" +pci_nvme_err_too_many_active_zones(uint16_t cid) "cid %"PRIu16"" +pci_nvme_err_too_many_open_zones(uint16_t cid) "cid %"PRIu16"" pci_nvme_err_invalid_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_num_sgld(uint16_t cid, uint8_t typ) "cid %"PRIu16" type 0x%"PRIx8"" pci_nvme_err_invalid_sgl_excess_length(uint16_t cid) "cid %"PRIu16""
Signature · 1 line
2.27.0
PATCH 9/10 hw/block/nvme: allow zone excursions
In reply to Klaus JensenAllow the controller to release active resources by transitioning zones to the full state.
--- hw/block/nvme-ns.h | 2 + hw/block/nvme.c | 171 ++++++++++++++++++++++++++++++++++++++---- hw/block/trace-events | 4 + include/block/nvme.h | 10 +++ 4 files changed, 174 insertions(+), 13 deletions(-)
hw/block/nvme-ns.h +2 -0
@@ -75,6 +75,8 @@ typedef struct NvmeNamespace { QTAILQ_HEAD(, NvmeZone) lru_open; QTAILQ_HEAD(, NvmeZone) lru_active; } resources; + +NvmeChangedZoneList changed_list; } zns; } NvmeNamespace;
hw/block/nvme.c +158 -13
278 lines not on this page · raw message
hw/block/trace-events +4 -0
@@ -101,6 +101,10 @@ pci_nvme_update_zone_descr(uint16_t cid, uint32_t nsid, uint64_t zslba) "cid %"P pci_nvme_zone_zrm_transition(uint16_t cid, uint32_t nsid, uint64_t zslba, uint8_t from, uint8_t to) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" from 0x%"PRIx8" to 0x%"PRIx8"" pci_nvme_zone_zrm_candidate(uint16_t cid, uint32_t nsid, uint64_t zslba, uint64_t wp, uint8_t zc) "cid %"PRIu16" nsid %"PRIu32" zslba 0x%"PRIx64" wp 0x%"PRIx64" zc 0x%"PRIx8"" pci_nvme_zone_zrm_release_open(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_zrm_release_active(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_zrm_excursion_not_allowed(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" +pci_nvme_zone_changed(uint32_t nsid, uint64_t zslba) "nsid %"PRIu32" zslba 0x%"PRIx64"" +pci_nvme_zone_excursion(uint32_t nsid, uint64_t zslba, const char *zc) "nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" pci_nvme_mmio_intm_set(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask set, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_intm_clr(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask clr, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_cfg(uint64_t data) "wrote MMIO, config controller config=0x%"PRIx64""
include/block/nvme.h +10 -0
@@ -778,6 +778,7 @@ typedef struct NvmeDsmRange {enum NvmeAsyncEventRequest {NVME_AER_TYPE_ERROR =0 ,NVME_AER_TYPE_SMART =1 , +NVME_AER_TYPE_NOTICE =2 ,NVME_AER_TYPE_IO_SPECIFIC =6 ,NVME_AER_TYPE_VENDOR_SPECIFIC =7 ,NVME_AER_INFO_ERR_INVALID_DB_REGISTER =0 ,
@@ -993,6 +994,14 @@ typedef struct NvmeZoneDescriptor {#define NVME_ZS (zs) (((zs) >> 4) & 0xf)#define NVME_ZS_SET (zs, state) ((zs) = ((state) << 4)) +#define NVME_CHANGED_ZONE_LIST_MAX_IDS 511 + +typedef struct NvmeChangedZoneList { +uint16_t num_ids; +uint8_t rsvd2[6 ]; +uint64_t ids[NVME_CHANGED_ZONE_LIST_MAX_IDS ]; +}NvmeChangedZoneList ; +#define NVME_ZA_ZFC (za) ((za) & (1 << 0))#define NVME_ZA_FZR (za) ((za) & (1 << 1))#define NVME_ZA_RZR (za) ((za) & (1 << 2))
@@ -1428,5 +1437,6 @@ static inline void _nvme_check_size(void)QEMU_BUILD_BUG_ON (sizeof (NvmeEffectsLog) !=4096 );QEMU_BUILD_BUG_ON (sizeof (NvmeZoneDescriptor) !=64 );QEMU_BUILD_BUG_ON (sizeof (NvmeLBAFE) !=16 ); + QEMU_BUILD_BUG_ON(sizeof(NvmeChangedZoneList) != 4096); }#endif
Signature · 1 line
2.27.0
PATCH 10/10 hw/block/nvme: support reset/finish recommended limits
In reply to Klaus JensenAdd the rrl and frl device parameters. The parameters specify the number of seconds before the device may perform an internal operation to "clear" the Reset Zone Recommended and Finish Zone Recommended attributes respectively.
When the attibutes are set is governed by the rrld and frld parameters (Reset/Finish Recomended Limit Delay). The Reset Zone Recommended Delay starts when a zone becomes full. The Finish Zone Recommended Delay starts when the zone is first activated. When the limits are reached, the attributes are cleared again and the process is restarted.
If zone excursions are enabled (they are by default), when the Finish Recommended Limit is reached, the device will finish the zone.
--- hw/block/nvme-ns.c | 105 ++++++++++++++++++++++++++++++++++++++++++ hw/block/nvme-ns.h | 13 ++++++ hw/block/nvme.c | 49 +++++++++++++------- hw/block/nvme.h | 7 +++ hw/block/trace-events | 3 +- 5 files changed, 160 insertions(+), 17 deletions(-)
hw/block/nvme-ns.c +105 -0
@@ -25,6 +25,7 @@#include "hw/qdev-properties.h" #include "hw/qdev-core.h" +#include "trace.h" #include "nvme.h" #include "nvme-ns.h"
@@ -48,6 +49,91 @@ const char *nvme_zs_to_str(NvmeZoneState zs)return NULL ; } +static void nvme_ns_process_timer (void *opaque) +{ +NvmeNamespace *ns = opaque; +BusState *s =qdev_get_parent_bus (&ns->parent_obj); +NvmeCtrl *n =NVME (s->parent); +NvmeZone *zone; + +trace_pci_nvme_ns_process_timer (ns->params.nsid); + +int64_t next_timer =INT64_MAX , now =qemu_clock_get_ns (QEMU_CLOCK_VIRTUAL ); + + QTAILQ_FOREACH(zone, &ns->zns.resources.lru_open, lru_entry) { +int64_t activated_ns = now - zone->stats.activated_ns; +if (activated_ns < ns->zns.frld_ns) { + next_timer =MIN (next_timer, zone->stats.activated_ns + + ns->zns.frld_ns); + +break ; + } + +if (activated_ns < ns->zns.frld_ns + ns->zns.frl_ns) { +NVME_ZA_SET_FZR (zone->zd.za,0x1 ); +nvme_zone_changed (n, ns, zone); + + next_timer =MIN (next_timer, now + ns->zns.frl_ns); + +continue ; + } + +if (zone->wp_staging !=le64_to_cpu (zone->zd.wp)) { + next_timer = now +500 ; +continue ; + } + +nvme_zone_excursion (n, ns, zone,NULL ); + } + + QTAILQ_FOREACH(zone, &ns->zns.resources.lru_active, lru_entry) { +int64_t activated_ns = now - zone->stats.activated_ns; +if (activated_ns < ns->zns.frld_ns) { + next_timer =MIN (next_timer, zone->stats.activated_ns + + ns->zns.frld_ns); + +break ; + } + +if (activated_ns < ns->zns.frld_ns + ns->zns.frl_ns) { +NVME_ZA_SET_FZR (zone->zd.za,0x1 ); +nvme_zone_changed (n, ns, zone); + + next_timer =MIN (next_timer, now + ns->zns.frl_ns); + +continue ; + } + +nvme_zone_excursion (n, ns, zone,NULL ); + } + + QTAILQ_FOREACH(zone, &ns->zns.lru_finished, lru_entry) { +int64_t finished_ns = now - zone->stats.finished_ns; +if (finished_ns < ns->zns.rrld_ns) { + next_timer =MIN (next_timer, zone->stats.finished_ns + + ns->zns.rrld_ns); + +break ; + } + +if (finished_ns < ns->zns.rrld_ns + ns->zns.rrl_ns) { +NVME_ZA_SET_RZR (zone->zd.za,0x1 ); +nvme_zone_changed (n, ns, zone); + + next_timer =MIN (next_timer, now + ns->zns.rrl_ns); + +nvme_zone_changed (n, ns, zone); +continue ; + } + +NVME_ZA_SET_RZR (zone->zd.za,0x0 ); + } + +if (next_timer !=INT64_MAX ) { +timer_mod (ns->zns.timer, next_timer); + } +} +static int nvme_ns_blk_resize (BlockBackend *blk,size_t len,Error **errp) {Error *local_err =NULL ;
@@ -262,6 +348,21 @@ static void nvme_ns_init_zoned(NvmeNamespace *ns) id_ns->ncap = ns->zns.info.num_zones * ns->params.zns.zcap; + id_ns_zns->rrl = ns->params.zns.rrl; + id_ns_zns->frl = ns->params.zns.frl; + +if (ns->params.zns.rrl || ns->params.zns.frl) { + ns->zns.rrl_ns = ns->params.zns.rrl *NANOSECONDS_PER_SECOND ; + ns->zns.rrld_ns = ns->params.zns.rrld *NANOSECONDS_PER_SECOND ; + ns->zns.frl_ns = ns->params.zns.frl *NANOSECONDS_PER_SECOND ; + ns->zns.frld_ns = ns->params.zns.frld *NANOSECONDS_PER_SECOND ; + + ns->zns.timer =timer_new_ns (QEMU_CLOCK_VIRTUAL , + nvme_ns_process_timer, ns); + +QTAILQ_INIT (&ns->zns.lru_finished); + } + id_ns_zns->mar =cpu_to_le32 (ns->params.zns.mar); id_ns_zns->mor =cpu_to_le32 (ns->params.zns.mor);
@@ -515,6 +616,10 @@ static Property nvme_ns_props[] = {DEFINE_PROP_UINT16 ("zns.ozcs" , NvmeNamespace, params.zns.ozcs,0 ),DEFINE_PROP_UINT32 ("zns.mar" , NvmeNamespace, params.zns.mar,0xffffffff ),DEFINE_PROP_UINT32 ("zns.mor" , NvmeNamespace, params.zns.mor,0xffffffff ), +DEFINE_PROP_UINT32 ("zns.rrl" , NvmeNamespace, params.zns.rrl,0 ), +DEFINE_PROP_UINT32 ("zns.frl" , NvmeNamespace, params.zns.frl,0 ), +DEFINE_PROP_UINT32 ("zns.rrld" , NvmeNamespace, params.zns.rrld,0 ), +DEFINE_PROP_UINT32 ("zns.frld" , NvmeNamespace, params.zns.frld,0 ), DEFINE_PROP_END_OF_LIST(), };
hw/block/nvme-ns.h +13 -0
@@ -31,6 +31,10 @@ typedef struct NvmeNamespaceParams {uint16_t ozcs;uint32_t mar;uint32_t mor; +uint32_t rrl; +uint32_t frl; +uint32_t rrld; +uint32_t frld; } zns; } NvmeNamespaceParams;
@@ -40,6 +44,11 @@ typedef struct NvmeZone {uint64_t wp_staging; +struct { +int64_t activated_ns; +int64_t finished_ns; + } stats; +QTAILQ_ENTRY (NvmeZone) lru_entry; } NvmeZone;
@@ -77,6 +86,10 @@ typedef struct NvmeNamespace { } resources;NvmeChangedZoneList changed_list; + + QTAILQ_HEAD(, NvmeZone) lru_finished; +QEMUTimer *timer; +int64_t rrl_ns, rrld_ns, frl_ns, frld_ns; } zns; } NvmeNamespace;
hw/block/nvme.c +33 -16
108 lines not on this page · raw message
hw/block/nvme.h +7 -0
@@ -318,5 +318,12 @@ static inline NvmeCtrl *nvme_ctrl(NvmeRequest *req) }int nvme_register_namespace (NvmeCtrl *n,NvmeNamespace *ns,Error **errp); +uint16_t nvme_zrm_transition (NvmeCtrl *n,NvmeNamespace *ns,NvmeZone *zone, +NvmeZoneState to,NvmeRequest *req); +void nvme_enqueue_event (NvmeCtrl *n,NvmeNamespace *ns,uint8_t event_type, +uint8_t event_info,uint8_t log_page); +void nvme_zone_excursion (NvmeCtrl *n,NvmeNamespace *ns,NvmeZone *zone, +NvmeRequest *req); +void nvme_zone_changed (NvmeCtrl *n,NvmeNamespace *ns,NvmeZone *zone);#endif /* HW_NVME_H */
hw/block/trace-events +2 -1
@@ -85,7 +85,7 @@ pci_nvme_aer(uint16_t cid) "cid %"PRIu16"" pci_nvme_aer_aerl_exceeded(void) "aerl exceeded" pci_nvme_aer_masked(uint8_t type, uint8_t mask) "type 0x%"PRIx8" mask 0x%"PRIx8"" pci_nvme_aer_post_cqe(uint8_t typ, uint8_t info, uint8_t log_page) "type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" -pci_nvme_enqueue_event(uint8_t typ, uint8_t info, uint8_t log_page) "type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" +pci_nvme_enqueue_event(uint32_t nsid, uint8_t typ, uint8_t info, uint8_t log_page) "nsid 0x%"PRIx32" type 0x%"PRIx8" info 0x%"PRIx8" lid 0x%"PRIx8"" pci_nvme_enqueue_event_noqueue(int queued) "queued %d" pci_nvme_enqueue_event_masked(uint8_t typ) "type 0x%"PRIx8"" pci_nvme_no_outstanding_aers(void) "ignoring event; no outstanding AERs"
@@ -105,6 +105,7 @@ pci_nvme_zone_zrm_release_active(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsi pci_nvme_zone_zrm_excursion_not_allowed(uint16_t cid, uint32_t nsid) "cid %"PRIu16" nsid %"PRIu32"" pci_nvme_zone_changed(uint32_t nsid, uint64_t zslba) "nsid %"PRIu32" zslba 0x%"PRIx64"" pci_nvme_zone_excursion(uint32_t nsid, uint64_t zslba, const char *zc) "nsid %"PRIu32" zslba 0x%"PRIx64" zc \"%s\"" +pci_nvme_ns_process_timer(uint32_t nsid) "nsid %"PRIu32"" pci_nvme_mmio_intm_set(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask set, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_intm_clr(uint64_t data, uint64_t new_mask) "wrote MMIO, interrupt mask clr, data=0x%"PRIx64", new_mask=0x%"PRIx64"" pci_nvme_mmio_cfg(uint64_t data) "wrote MMIO, config controller config=0x%"PRIx64""
Signature · 1 line
2.27.0
Quoted 3 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Hi all,
Hello Klaus,
Quoted 28 lines
This series adds support for TP 4056 ("Namespace Types") and TP 4053 ("Zoned Namespaces") and is an alternative implementation to the one submitted by Dmitry[1].
While I don't want this to end up as a discussion about the merits of each version, I want to point out a couple of differences from Dmitry's version. At a glance, my version
* builds on my patch series that adds fairly complete NVMe v1.4 mandatory support, as well as nice-to-have feature such as SGLs, multiple namespaces and mostly just overall clean up. This finally brings the nvme device into a fairly compliant state on which we can add new features. I've tried hard to get these compliance and clean-up patches merged for a long time (in parallel with developing the emulation of NST and ZNS) and I would be really sad to see them by-passed since they have already been through many iterations and already carries Acked- and Reviewed-by's for the bulk of the patches. I think the nvme device is already in a "frankenstate" wrt. the implemented nvme version and the features it currently supports, so I think this kind of cleanup is long overdue.
* uses an attached blockdev and standard
blk_aiofor persistent zone info. This is the same method used in our patches for Write Uncorrectable and (separate and extended lba) metadata support, but I've left those optional features out for now to ease the review process.
Quoted 6 lines
* relies on the universal dulbe support added in ("hw/block/nvme: add support for dulbe") and sparse images for handling reads in gaps (above write pointer and below ZSZE); that is - the size of the underlying blockdev is in terms of ZSZE, not ZCAP
* the controller uses timers to autonomously finish zones (wrt. FRL)
AFAICT, Dmitry's patches does this as well.
Quoted 15 lines
I've been on paternity leave for a month, so I havn't been around to review Dmitry's patches, but I have started that process now. I would also be happy to work with Dmitry & Friends on merging our versions to get the best of both worlds if it makes sense.
This series and all preparatory patch sets (the ones I've been posting yesterday and today) are available on my GitHub[2]. Unfortunately Patchew got screwed up in the middle of me sending patches and it never picked up v2 of "hw/block/nvme: support multiple namespaces" because it was getting late and I made a mistake with the CC's. So my posted series don't apply according to Patchew, but they actually do if you follow the Based-on's (... or just grab [2]).
Quoted 6 lines
[1]: Message-Id: <[email protected]> [2]: https://github.com/birkelund/qemu/tree/for-master/nvme
Based-on: <[email protected]> ("[PATCH 0/3] hw/block/nvme: bump to v1.4")
Is this the only patch series that this series depends on?
In the beginning of the cover letter, you mentioned "NVMe v1.4 mandatory support", "SGLs", "multiple namespaces", and "and mostly just overall clean up".
Quoted 13 lines
Klaus Jensen (10):
hw/block/nvme: support I/O Command Sets hw/block/nvme: add zns specific fields and types hw/block/nvme: add basic read/write for zoned namespaces hw/block/nvme: add the zone management receive command hw/block/nvme: add the zone management send command hw/block/nvme: add the zone append command hw/block/nvme: track and enforce zone resources hw/block/nvme: allow open to close transitions by controller hw/block/nvme: allow zone excursions hw/block/nvme: support reset/finish recommended limits
Quoted 12 lines
block/nvme.c | 6 +- hw/block/nvme-ns.c | 397 +++++++++- hw/block/nvme-ns.h | 148 +++- hw/block/nvme.c | 1676 +++++++++++++++++++++++++++++++++++++++-- hw/block/nvme.h | 76 +- hw/block/trace-events | 43 +- include/block/nvme.h | 252 ++++++- 7 files changed, 2469 insertions(+), 129 deletions(-)Signature · 1 line
2.27.0
I think that you have done a great job getting the NVMe driver out of a frankenstate, and made it compliant with a proper spec (NVMe 1.4).
I'm also a big fan of the refactoring so that the driver handles more than one namespace, and the new bus model.
I know that you first sent your
"nvme: support NVMe v1.3d, SGLs and multiple namespaces" patch series July, last year.
Looking at your outstanding patch series on patchwork:
https://patchwork.kernel.org/project/qemu-devel/list/?submitter=188679
(Feel free to correct me if I have misunderstood anything.)
I see that these are related to your patch series from July last year:
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4
This patch series seems minor and could probably be merged immediately:
hw/block/nvme: handle transient dma errors
This patch series looks a bit weird:
hw/block/nvme: AIO and address mapping refactoring
Since it looks like a V1 post, and was first posted yesterday. However, 2 out of the 17 patches in are Acked-by: Keith. (Perhaps some of your previously posted patches was put inside this new patch series?)
This patch series:
hw/block/nvme: namespace types and zoned namespaces
Which was first posted today. Up until earlier today, we haven't seen any patches from you related to ZNS (only overall NVMe cleanups). Dmitry's ZNS patches have been on the list since 2020-06-16.
Just a friendly suggestion, how about:
1) We get your
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4
patch series merged.
2) We get Dmitry's patch series merged.
Shared 4:th) If there is any feature that you miss in Dmitry's patch series, perhaps you could send patches to add what you are missing.
Shared 4:th) Your other patch series:
hw/block/nvme: AIO and address mapping refactoring could get merged.
Please don't take this suggestion the wrong way, I'm simply trying to come up with a way to move forward from here.
Kind regards,
Niklas
Quoted 158 lines · Niklas Cassel
Quoted 3 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Hi all,
Hello Klaus,
Quoted 28 lines
This series adds support for TP 4056 ("Namespace Types") and TP 4053 ("Zoned Namespaces") and is an alternative implementation to the one submitted by Dmitry[1].
While I don't want this to end up as a discussion about the merits of each version, I want to point out a couple of differences from Dmitry's version. At a glance, my version
* builds on my patch series that adds fairly complete NVMe v1.4 mandatory support, as well as nice-to-have feature such as SGLs, multiple namespaces and mostly just overall clean up. This finally brings the nvme device into a fairly compliant state on which we can add new features. I've tried hard to get these compliance and clean-up patches merged for a long time (in parallel with developing the emulation of NST and ZNS) and I would be really sad to see them by-passed since they have already been through many iterations and already carries Acked- and Reviewed-by's for the bulk of the patches. I think the nvme device is already in a "frankenstate" wrt. the implemented nvme version and the features it currently supports, so I think this kind of cleanup is long overdue.
* uses an attached blockdev and standard
blk_aiofor persistent zone info. This is the same method used in our patches for Write Uncorrectable and (separate and extended lba) metadata support, but I've left those optional features out for now to ease the review process.Quoted 6 lines
* relies on the universal dulbe support added in ("hw/block/nvme: add support for dulbe") and sparse images for handling reads in gaps (above write pointer and below ZSZE); that is - the size of the underlying blockdev is in terms of ZSZE, not ZCAP
* the controller uses timers to autonomously finish zones (wrt. FRL)
AFAICT, Dmitry's patches does this as well.
Quoted 15 lines
I've been on paternity leave for a month, so I havn't been around to review Dmitry's patches, but I have started that process now. I would also be happy to work with Dmitry & Friends on merging our versions to get the best of both worlds if it makes sense.
This series and all preparatory patch sets (the ones I've been posting yesterday and today) are available on my GitHub[2]. Unfortunately Patchew got screwed up in the middle of me sending patches and it never picked up v2 of "hw/block/nvme: support multiple namespaces" because it was getting late and I made a mistake with the CC's. So my posted series don't apply according to Patchew, but they actually do if you follow the Based-on's (... or just grab [2]).
Quoted 6 lines
[1]: Message-Id: <[email protected]> [2]: https://github.com/birkelund/qemu/tree/for-master/nvme
Based-on: <[email protected]> ("[PATCH 0/3] hw/block/nvme: bump to v1.4")
Is this the only patch series that this series depends on?
In the beginning of the cover letter, you mentioned "NVMe v1.4 mandatory support", "SGLs", "multiple namespaces", and "and mostly just overall clean up".
Quoted 13 lines
Klaus Jensen (10):
hw/block/nvme: support I/O Command Sets hw/block/nvme: add zns specific fields and types hw/block/nvme: add basic read/write for zoned namespaces hw/block/nvme: add the zone management receive command hw/block/nvme: add the zone management send command hw/block/nvme: add the zone append command hw/block/nvme: track and enforce zone resources hw/block/nvme: allow open to close transitions by controller hw/block/nvme: allow zone excursions hw/block/nvme: support reset/finish recommended limitsQuoted 12 lines
block/nvme.c | 6 +- hw/block/nvme-ns.c | 397 +++++++++- hw/block/nvme-ns.h | 148 +++- hw/block/nvme.c | 1676 +++++++++++++++++++++++++++++++++++++++-- hw/block/nvme.h | 76 +- hw/block/trace-events | 43 +- include/block/nvme.h | 252 ++++++- 7 files changed, 2469 insertions(+), 129 deletions(-)Signature · 1 line
2.27.0I think that you have done a great job getting the NVMe driver out of a frankenstate, and made it compliant with a proper spec (NVMe 1.4).
I'm also a big fan of the refactoring so that the driver handles more than one namespace, and the new bus model.
I know that you first sent your
"nvme: support NVMe v1.3d, SGLs and multiple namespaces" patch series July, last year.Looking at your outstanding patch series on patchwork:
https://patchwork.kernel.org/project/qemu-devel/list/?submitter=188679(Feel free to correct me if I have misunderstood anything.)
I see that these are related to your patch series from July last year:
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4This patch series seems minor and could probably be merged immediately:
hw/block/nvme: handle transient dma errorsThis patch series looks a bit weird:
hw/block/nvme: AIO and address mapping refactoringSince it looks like a V1 post, and was first posted yesterday. However, 2 out of the 17 patches in are Acked-by: Keith. (Perhaps some of your previously posted patches was put inside this new patch series?)
This patch series:
hw/block/nvme: namespace types and zoned namespacesWhich was first posted today. Up until earlier today, we haven't seen any patches from you related to ZNS (only overall NVMe cleanups). Dmitry's ZNS patches have been on the list since 2020-06-16.
Just a friendly suggestion, how about:
1) We get your
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4patch series merged.
2) We get Dmitry's patch series merged.
Shared 4:th) If there is any feature that you miss in Dmitry's patch series, perhaps you could send patches to add what you are missing.
Quoted 6 lines
Shared 4:th) Your other patch series:
hw/block/nvme: AIO and address mapping refactoring could get merged.Please don't take this suggestion the wrong way, I'm simply trying to come up with a way to move forward from here.
Few months ago Klaus sent a bomb series with ~80 patches, we asked him to split in digestible series of ~20 patches.
Earlier in this cover Klaus provided a link to his git repository with all the patches sorted [2]:
https://github.com/birkelund/qemu/tree/for-master/nvme
This seems enough to get the big picture.
Niklas Cassel, it would be helpful if you or Dmitry can review Klaus patches. I see Klaus is already reviewing Dmitry ones.
Both Keith and Kevin are quite busy recently.
To help them I suggest once you reviewed your patches each other, one of you could send the big series with all patches together.
Anyway soft-freeze is next week, so you have to decide what is critical.
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3
[3] https://www.mail-archive.com/[email protected]/msg718086.html
Quoted 5 lines
Kind regards,
Niklas
Quoted 4 lines · Philippe Mathieu-Daudé
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3
These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
Quoted 9 lines · Keith Busch
Quoted 4 lines · Philippe Mathieu-Daudé
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
I think Philippe's "hw/block/nvme: Fix I/O BAR structure" series is a no-brainer. It just needs to get in asap.
The "hw/block/nvme: handle transient dma errors" series would really benefit from most of the patches in my "hw/block/nvme: AIO and address mapping refactoring" series. The elephant in the room is the AIO part (the "hw/block/nvme: allow multiple aios per command" patch), so I will get rid of it and leave the cleanup patches there and post it as a new series together with the "handle transient dma errors" fixes. This would make it a series of around ~17-18 patches, but I think they are all quite reviewable.
The bump to v1.3 should also pretty much be ready for merging.
nvme emulation merge process (was: Re: [PATCH 00/10] hw/block/nvme: namespace types and zoned namespaces)
Am 30.06.2020 um 22:36 hat Klaus Jensen geschrieben:
Quoted 8 lines · Keith Busch
Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 4 lines
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3
Quoted 5 lines
Quoted 2 lines
These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
I think Philippe's "hw/block/nvme: Fix I/O BAR structure" series is a no-brainer. It just needs to get in asap.
I think we need to talk about how nvme patches are supposed to get merged. I'm not familiar with the hardware nor the code, so the model was that I just blindly merge patches that Keith has reviewed/acked, just to spare him the work to prepare a pull request. But obviously, we started doing things this way when there was a lot less activity around the nvme emulation.
If we find that this doesn't scale any more, maybe we need to change something. Depending on how much time Keith can spend on review in the near future and how much control he wants to keep over the development, I could imagine adding Klaus to MAINTAINERS, either as a co-maintainer or as a reviewer. Then I could rely on reviews/acks from either of you for merging series.
Of course, the patches don't necessarily have to go through my tree either if this only serves to complicate things these days. If sending separate pull requests directly to Peter would make things easier, I certainly wouldn't object.
Kevin
Re: nvme emulation merge process (was: Re: [PATCH 00/10] hw/block/nvme: namespace types and zoned namespaces)
Quoted 22 lines · Kevin Wolf
Am 30.06.2020 um 22:36 hat Klaus Jensen geschrieben:
Quoted 8 lines · Keith Busch
Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 4 lines
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3Quoted 5 lines
Quoted 2 lines
These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
I think Philippe's "hw/block/nvme: Fix I/O BAR structure" series is a no-brainer. It just needs to get in asap.
I think we need to talk about how nvme patches are supposed to get merged. I'm not familiar with the hardware nor the code, so the model was that I just blindly merge patches that Keith has reviewed/acked, just to spare him the work to prepare a pull request. But obviously, we started doing things this way when there was a lot less activity around the nvme emulation.
Quoted 2 lines
If we find that this doesn't scale any more, maybe we need to change something.
Honestly, I do not think the current model has worked very well for some time; especially for larger series where I, for one, has felt that my work was largely ignored due to a lack of designated reviewers. Things only picked up when Beata, Maxim and Philippe started reviewing my series - maybe out of pity or because I was bombing the list, I don't know ;)
We've also seen good patches from Andrzej linger on the list for quite a while, prompting a number of RESENDs. I only recently allocated more time and upped my review game, but I hope that contributors feel that stuff gets reviewed in a timely fashion by now.
Please understand that this is in NO WAY a criticism of Keith who already made it very clear to me that he did not have a lot time to review, but only ack the odd patch.
Quoted 6 lines
Depending on how much time Keith can spend on review in the near future and how much control he wants to keep over the development, I could imagine adding Klaus to
MAINTAINERS, either as a co-maintainer or as a reviewer. Then I could rely on reviews/acks from either of you for merging series.
I would be happy to step up (officially) to help maintain the device with Keith and review on a daily basis, and my position can support this.
Quoted 5 lines
Of course, the patches don't necessarily have to go through my tree either if this only serves to complicate things these days. If sending separate pull requests directly to Peter would make things easier, I certainly wouldn't object.
I don't think there is any reason to by-pass your tree. I think the volume would need to increase even further for that to make sense.
Re: nvme emulation merge process (was: Re: [PATCH 00/10] hw/block/nvme: namespace types and zoned namespaces)
Quoted 52 lines · Klaus Jensen
Quoted 21 lines · Kevin Wolf
Am 30.06.2020 um 22:36 hat Klaus Jensen geschrieben:
Quoted 7 lines · Keith Busch
Quoted 6 lines · Philippe Mathieu-Daudé
Quoted 4 lines
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3Quoted 5 lines
Quoted 2 lines
These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
I think Philippe's "hw/block/nvme: Fix I/O BAR structure" series is a no-brainer. It just needs to get in asap.
I think we need to talk about how nvme patches are supposed to get merged. I'm not familiar with the hardware nor the code, so the model was that I just blindly merge patches that Keith has reviewed/acked, just to spare him the work to prepare a pull request. But obviously, we started doing things this way when there was a lot less activity around the nvme emulation.
Quoted 2 lines
If we find that this doesn't scale any more, maybe we need to change something.
Honestly, I do not think the current model has worked very well for some time; especially for larger series where I, for one, has felt that my work was largely ignored due to a lack of designated reviewers. Things only picked up when Beata, Maxim and Philippe started reviewing my series - maybe out of pity or because I was bombing the list, I don't know ;)
We've also seen good patches from Andrzej linger on the list for quite a while, prompting a number of RESENDs. I only recently allocated more time and upped my review game, but I hope that contributors feel that stuff gets reviewed in a timely fashion by now.
Please understand that this is in NO WAY a criticism of Keith who already made it very clear to me that he did not have a lot time to review, but only ack the odd patch.
Quoted 6 lines
Depending on how much time Keith can spend on review in the near future and how much control he wants to keep over the development, I could imagine adding Klaus to
MAINTAINERS, either as a co-maintainer or as a reviewer. Then I could rely on reviews/acks from either of you for merging series.I would be happy to step up (officially) to help maintain the device with Keith and review on a daily basis, and my position can support this.
Quoted 9 lines
Quoted 5 lines
Of course, the patches don't necessarily have to go through my tree either if this only serves to complicate things these days. If sending separate pull requests directly to Peter would make things easier, I certainly wouldn't object.
I don't think there is any reason to by-pass your tree. I think the volume would need to increase even further for that to make sense.
It my fault as well - I need to get back to reviewing these. (I'll review few of them today I hope)
Best regards,
Maxim Levitsky
Re: nvme emulation merge process (was: Re: [PATCH 00/10] hw/block/nvme: namespace types and zoned namespaces)
In reply to Klaus JensenQuoted 26 lines · Klaus Jensen
Quoted 22 lines · Kevin Wolf
Am 30.06.2020 um 22:36 hat Klaus Jensen geschrieben:
Quoted 8 lines · Keith Busch
Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 4 lines
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3Quoted 5 lines
Quoted 2 lines
These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
I think Philippe's "hw/block/nvme: Fix I/O BAR structure" series is a no-brainer. It just needs to get in asap.
I think we need to talk about how nvme patches are supposed to get merged. I'm not familiar with the hardware nor the code, so the model was that I just blindly merge patches that Keith has reviewed/acked, just to spare him the work to prepare a pull request. But obviously, we started doing things this way when there was a lot less activity around the nvme emulation.
Quoted 2 lines
If we find that this doesn't scale any more, maybe we need to change something.
Quoted 6 lines
Honestly, I do not think the current model has worked very well for some time; especially for larger series where I, for one, has felt that my work was largely ignored due to a lack of designated reviewers. Things only picked up when Beata, Maxim and Philippe started reviewing my series - maybe out of pity or because I was bombing the list, I don't know ;)
I have no interest in the NVMe device emulation, but one of the first thing I notice when I look at the wiki the time I wanted to send my first patch, is the "Return the favor" paragraph:
https://wiki.qemu.org/Contribute/SubmitAPatch#Return_the_favor
"Peer review only works if everyone chips in a bit of review time.
If everyone submitted more patches than they reviewed, we would have a patch backlog. A good goal is to try to review at least as many patches from others as what you submit. Don't worry if you don't know the code base as well as a maintainer; it's perfectly fine to admit when your review is weak because you are unfamiliar with the code."
So as some reviewed my patches, I try to return the favor to the community, in particular when I see someone is stuck waiting for review, and the patch topic is some area I can understand.
I don't see that as an "out of pity" reaction.
Note, it is true bomb series scares reviewers. You learned it the bad way. But you can see, after resending the first part of your "bomb", even if it took 10 versions, the result is a great improvement!
Quoted 16 lines
We've also seen good patches from Andrzej linger on the list for quite a while, prompting a number of RESENDs. I only recently allocated more time and upped my review game, but I hope that contributors feel that stuff gets reviewed in a timely fashion by now.
Please understand that this is in NO WAY a criticism of Keith who already made it very clear to me that he did not have a lot time to review, but only ack the odd patch.
Quoted 6 lines
Depending on how much time Keith can spend on review in the near future and how much control he wants to keep over the development, I could imagine adding Klaus to
MAINTAINERS, either as a co-maintainer or as a reviewer. Then I could rely on reviews/acks from either of you for merging series.
Quoted 3 lines
I would be happy to step up (officially) to help maintain the device with Keith and review on a daily basis, and my position can support this.
Sounds good to me, but it is up to Keith Busch to accept.
It would be nice to have at least one developer from WDC listed as designated reviewer too.
Maxim is candidate for designated reviewer but I think he doesn't have the time.
It would also nice to have Andrzej Jakowski listed, if he is interested.
Quoted 10 lines
Quoted 6 lines
Of course, the patches don't necessarily have to go through my tree either if this only serves to complicate things these days. If sending separate pull requests directly to Peter would make things easier, I certainly wouldn't object.
I don't think there is any reason to by-pass your tree. I think the volume would need to increase even further for that to make sense.
Re: nvme emulation merge process (was: Re: [PATCH 00/10] hw/block/nvme: namespace types and zoned namespaces)
Quoted 17 lines · Philippe Mathieu-Daudé
Quoted 16 lines · Klaus Jensen
We've also seen good patches from Andrzej linger on the list for quite a while, prompting a number of RESENDs. I only recently allocated more time and upped my review game, but I hope that contributors feel that stuff gets reviewed in a timely fashion by now.
Please understand that this is in NO WAY a criticism of Keith who already made it very clear to me that he did not have a lot time to review, but only ack the odd patch.
Quoted 6 lines
Depending on how much time Keith can spend on review in the near future and how much control he wants to keep over the development, I could imagine adding Klaus to
MAINTAINERS, either as a co-maintainer or as a reviewer. Then I could rely on reviews/acks from either of you for merging series.
Quoted 5 lines
Quoted 3 lines
I would be happy to step up (officially) to help maintain the device with Keith and review on a daily basis, and my position can support this.
Sounds good to me, but it is up to Keith Busch to accept.
I definitely want to continue at least having the opprotunity to review, though you may have noticed I am a bit low on time for more thorough maintenance this project deserves. The recent development pace for nvme would benefit from having its own tree, so I'm open to either co-maintenance, or handing off this to others. Please allow me to send a few queries off-list today to check-in with potentially interested parties.
Quoted 7 lines
It would be nice to have at least one developer from WDC listed as designated reviewer too.
Maxim is candidate for designated reviewer but I think he doesn't have the time.
It would also nice to have Andrzej Jakowski listed, if he is interested.
Re: nvme emulation merge process
In reply to Philippe Mathieu-DaudéQuoted 80 lines · Philippe Mathieu-Daudé
Quoted 26 lines · Klaus Jensen
Quoted 22 lines · Kevin Wolf
Am 30.06.2020 um 22:36 hat Klaus Jensen geschrieben:
Quoted 8 lines · Keith Busch
Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 4 lines
What I see doable for the following days is:
- hw/block/nvme: Fix I/O BAR structure [3]
- hw/block/nvme: handle transient dma errors
- hw/block/nvme: bump to v1.3Quoted 5 lines
Quoted 2 lines
These look like sensible patches to rebase future work on, IMO. The 1.3 updates had been prepared a while ago, at least.
I think Philippe's "hw/block/nvme: Fix I/O BAR structure" series is a no-brainer. It just needs to get in asap.
I think we need to talk about how nvme patches are supposed to get merged. I'm not familiar with the hardware nor the code, so the model was that I just blindly merge patches that Keith has reviewed/acked, just to spare him the work to prepare a pull request. But obviously, we started doing things this way when there was a lot less activity around the nvme emulation.
Quoted 2 lines
If we find that this doesn't scale any more, maybe we need to change something.
Quoted 6 lines
Honestly, I do not think the current model has worked very well for some time; especially for larger series where I, for one, has felt that my work was largely ignored due to a lack of designated reviewers. Things only picked up when Beata, Maxim and Philippe started reviewing my series - maybe out of pity or because I was bombing the list, I don't know ;)
I have no interest in the NVMe device emulation, but one of the first thing I notice when I look at the wiki the time I wanted to send my first patch, is the "Return the favor" paragraph:
https://wiki.qemu.org/Contribute/SubmitAPatch#Return_the_favor"Peer review only works if everyone chips in a bit of review time.
If everyone submitted more patches than they reviewed, we would have a patch backlog. A good goal is to try to review at least as many patches from others as what you submit. Don't worry if you don't know the code base as well as a maintainer; it's perfectly fine to admit when your review is weak because you are unfamiliar with the code."So as some reviewed my patches, I try to return the favor to the community, in particular when I see someone is stuck waiting for review, and the patch topic is some area I can understand.
I don't see that as an "out of pity" reaction.
Note, it is true bomb series scares reviewers. You learned it the bad way. But you can see, after resending the first part of your "bomb", even if it took 10 versions, the result is a great improvement!
Quoted 16 lines
We've also seen good patches from Andrzej linger on the list for quite a while, prompting a number of RESENDs. I only recently allocated more time and upped my review game, but I hope that contributors feel that stuff gets reviewed in a timely fashion by now.
Please understand that this is in NO WAY a criticism of Keith who already made it very clear to me that he did not have a lot time to review, but only ack the odd patch.
Quoted 6 lines
Depending on how much time Keith can spend on review in the near future and how much control he wants to keep over the development, I could imagine adding Klaus to
MAINTAINERS, either as a co-maintainer or as a reviewer. Then I could rely on reviews/acks from either of you for merging series.Quoted 3 lines
I would be happy to step up (officially) to help maintain the device with Keith and review on a daily basis, and my position can support this.
Sounds good to me, but it is up to Keith Busch to accept.
Quoted 7 lines
It would be nice to have at least one developer from WDC listed as designated reviewer too.
Maxim is candidate for designated reviewer but I think he doesn't have the time.
It would also nice to have Andrzej Jakowski listed, if he is interested.
Thx! Of course I am interested in helping and I think it is actually great idea to have couple of designated maintainers/reviewers as it would be easier for folks to receive feedback vs requesting it in polling manner :) And please don't get me wrong -- I'm not complaining about anything -- I think it is just reality that everybody is stretched out into multiple directions struggling to allocate time for multiple things. Having many people will actually increase likelihood of introducing high quality improvements.
Also, +1 on separate tree for nvme emulation.
Quoted 12 lines
Quoted 11 lines
Quoted 7 lines
Of course, the patches don't necessarily have to go through my tree either if this only serves to complicate things these days. If sending separate pull requests directly to Peter would make things easier, I certainly wouldn't object.
I don't think there is any reason to by-pass your tree. I think the volume would need to increase even further for that to make sense.
Re: nvme emulation merge process
Quoted 10 lines · Andrzej Jakowski
Thx! Of course I am interested in helping and I think it is actually great idea to have couple of designated maintainers/reviewers as it would be easier for folks to receive feedback vs requesting it in polling manner :) And please don't get me wrong -- I'm not complaining about anything -- I think it is just reality that everybody is stretched out into multiple directions struggling to allocate time for multiple things. Having many people will actually increase likelihood of introducing high quality improvements.
Also, +1 on separate tree for nvme emulation.
Thanks for your help.
Klaus and I will be setting up an external tree for qemu-nvme development (tentatively on git.infradead.org) and pull-request. I'm just waiting for the server admin to upload our public keys. If I don't hear back by Monday, I will use an alternate server in the interim.
Quoted 7 lines · Niklas Cassel
Quoted 3 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Hi all,
Hello Klaus,
Hi Niklas,
Quoted 5 lines
Quoted 2 lines
* the controller uses timers to autonomously finish zones (wrt. FRL)
AFAICT, Dmitry's patches does this as well.
Hmm, yeah. Something is going on at least. It's not really clear to me why it works or what is happening with that admin completion queue timer, but I'll dig through it.
Quoted 19 lines
Quoted 19 lines
I've been on paternity leave for a month, so I havn't been around to review Dmitry's patches, but I have started that process now. I would also be happy to work with Dmitry & Friends on merging our versions to get the best of both worlds if it makes sense.
This series and all preparatory patch sets (the ones I've been posting yesterday and today) are available on my GitHub[2]. Unfortunately Patchew got screwed up in the middle of me sending patches and it never picked up v2 of "hw/block/nvme: support multiple namespaces" because it was getting late and I made a mistake with the CC's. So my posted series don't apply according to Patchew, but they actually do if you follow the Based-on's (... or just grab [2]).
[1]: Message-Id: <[email protected]> [2]: https://github.com/birkelund/qemu/tree/for-master/nvme
Quoted 9 lines
Quoted 2 lines
Based-on: <[email protected]> ("[PATCH 0/3] hw/block/nvme: bump to v1.4")
Is this the only patch series that this series depends on?
In the beginning of the cover letter, you mentioned "NVMe v1.4 mandatory support", "SGLs", "multiple namespaces", and "and mostly just overall clean up".
No, its a string of series that all has a Based-on tag (that is, "[PATCH 0/3] hw/block/nvme: bump to v1.4" has another Based-on tag that points to the dependency of that). The point was to have patchew nicely apply everything, but it broke midway...
As Philippe pointed out, all of the patch sets are integrated in the GitHub tree, applied to QEMU master.
Quoted 8 lines
I think that you have done a great job getting the NVMe driver out of a frankenstate, and made it compliant with a proper spec (NVMe 1.4).
I'm also a big fan of the refactoring so that the driver handles more than one namespace, and the new bus model.
Well, thanks! :)
Quoted 7 lines
I know that you first sent your
"nvme: support NVMe v1.3d, SGLs and multiple namespaces" patch series July, last year.Looking at your outstanding patch series on patchwork:
https://patchwork.kernel.org/project/qemu-devel/list/?submitter=188679
Quoted 8 lines
(Feel free to correct me if I have misunderstood anything.)
I see that these are related to your patch series from July last year:
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4
Yeah this stuff has been around for a while so the history on patchwork is a mess.
Quoted 4 lines
This patch series seems minor and could probably be merged immediately:
hw/block/nvme: handle transient dma errors
Sure, but it's nicer in combination with the previous series ("hw/block/nvme: AIO and address mapping refactoring"). What I /can/ do is rip out "hw/block/nvme: allow multiple aios per command" as that patch might require more time for reviews. The rest of that series are clean ups and a couple of bug fixes.
Quoted 9 lines
This patch series looks a bit weird:
hw/block/nvme: AIO and address mapping refactoringSince it looks like a V1 post, and was first posted yesterday. However, 2 out of the 17 patches in are Acked-by: Keith. (Perhaps some of your previously posted patches was put inside this new patch series?)
Yes that and reviewers requested a lot of separation, so basically the patch set ballooned.
Quoted 8 lines
This patch series:
hw/block/nvme: namespace types and zoned namespacesWhich was first posted today. Up until earlier today, we haven't seen any patches from you related to ZNS (only overall NVMe cleanups). Dmitry's ZNS patches have been on the list since 2020-06-16.
Yeah, as I mentioned in my cover letter, I was on leave, so I wasn't around for the big ZNS release day either. But, honestly, I think this is irrelevant - code should be merged based on technical reasons (not technicalities).
Quoted 12 lines
Just a friendly suggestion, how about:
1) We get your
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4patch series merged.
Blowing my own horn here, but yeah, it seems like everyone would like to see this merged.
Quoted 5 lines
2) We get Dmitry's patch series merged.
Shared 4:th) If there is any feature that you miss in Dmitry's patch series, perhaps you could send patches to add what you are missing.
Looks like the two version are pretty much on par in terms of features.
Quoted 3 lines
Shared 4:th) Your other patch series:
hw/block/nvme: AIO and address mapping refactoring could get merged.
As stated above I think its only a single commit ("hw/block/nvme: allow multiple aios per command") that is controversial in that series.
Quoted 4 lines
Please don't take this suggestion the wrong way, I'm simply trying to come up with a way to move forward from here.
Absolutely - I totally get that you want to move forward with Dmitry's series, but I'd like to finish my review before committing to anything.
Cheers,
Klaus
Quoted 176 lines
-----Original Message-----
From: Klaus Jensen <[email protected]>
Sent: Tuesday, June 30, 2020 4:30 PM
To: Niklas Cassel <[email protected]>
Cc: [email protected]; Klaus Jensen <[email protected]>; [email protected]; Keith Busch <[email protected]>; Max Reitz <[email protected]>; Kevin Wolf <[email protected]>; Javier Gonzalez <[email protected]>; Maxim Levitsky <[email protected]>; Philippe Mathieu-Daudé <[email protected]>; Dmitry Fomichev <[email protected]>; Damien Le Moal <[email protected]>; Matias Bjorling <[email protected]>
Subject: Re: [PATCH 00/10] hw/block/nvme: namespace types and zoned namespacesQuoted 7 lines · Niklas Cassel
Quoted 3 lines · Klaus Jensen
From: Klaus Jensen <[email protected]>
Hi all,
Hello Klaus,
Hi Niklas,
Quoted 5 lines
Quoted 2 lines
* the controller uses timers to autonomously finish zones (wrt. FRL)
AFAICT, Dmitry's patches does this as well.
Hmm, yeah. Something is going on at least. It's not really clear to me why it works or what is happening with that admin completion queue timer, but I'll dig through it.
Quoted 9 lines
Quoted 9 lines
I've been on paternity leave for a month, so I havn't been around to review Dmitry's patches, but I have started that process now. I would also be happy to work with Dmitry & Friends on merging our versions to get the best of both worlds if it makes sense.
This series and all preparatory patch sets (the ones I've been posting yesterday and today) are available on my GitHub[2]. Unfortunately Patchew got screwed up in the middle of me sending patches and it
never
Quoted 1 line
Quoted 1 line
picked up v2 of "hw/block/nvme: support multiple namespaces" because
it
Quoted 9 lines
Quoted 9 lines
was getting late and I made a mistake with the CC's. So my posted series don't apply according to Patchew, but they actually do if you follow the Based-on's (... or just grab [2]).
[1]: Message-Id: <[email protected]> [2]: https://github.com/birkelund/qemu/tree/for-master/nvme
Quoted 9 lines
Quoted 2 lines
Based-on: <[email protected]> ("[PATCH 0/3] hw/block/nvme: bump to v1.4")
Is this the only patch series that this series depends on?
In the beginning of the cover letter, you mentioned "NVMe v1.4 mandatory support", "SGLs", "multiple namespaces", and "and mostly just overall clean up".
No, its a string of series that all has a Based-on tag (that is, "[PATCH 0/3] hw/block/nvme: bump to v1.4" has another Based-on tag that points to the dependency of that). The point was to have patchew nicely apply everything, but it broke midway...
As Philippe pointed out, all of the patch sets are integrated in the GitHub tree, applied to QEMU master.
Quoted 8 lines
I think that you have done a great job getting the NVMe driver out of a frankenstate, and made it compliant with a proper spec (NVMe 1.4).
I'm also a big fan of the refactoring so that the driver handles more than one namespace, and the new bus model.
Well, thanks! :)
Quoted 7 lines
I know that you first sent your
"nvme: support NVMe v1.3d, SGLs and multiple namespaces" patch series July, last year.Looking at your outstanding patch series on patchwork:
https://patchwork.kernel.org/project/qemu-devel/list/?submitter=188679Quoted 8 lines
(Feel free to correct me if I have misunderstood anything.)
I see that these are related to your patch series from July last year:
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4Yeah this stuff has been around for a while so the history on patchwork is a mess.
Quoted 4 lines
This patch series seems minor and could probably be merged immediately:
hw/block/nvme: handle transient dma errorsSure, but it's nicer in combination with the previous series ("hw/block/nvme: AIO and address mapping refactoring"). What I /can/ do is rip out "hw/block/nvme: allow multiple aios per command" as that patch might require more time for reviews. The rest of that series are clean ups and a couple of bug fixes.
Quoted 9 lines
This patch series looks a bit weird:
hw/block/nvme: AIO and address mapping refactoringSince it looks like a V1 post, and was first posted yesterday. However, 2 out of the 17 patches in are Acked-by: Keith. (Perhaps some of your previously posted patches was put inside this new patch series?)
Yes that and reviewers requested a lot of separation, so basically the patch set ballooned.
Quoted 8 lines
This patch series:
hw/block/nvme: namespace types and zoned namespacesWhich was first posted today. Up until earlier today, we haven't seen any patches from you related to ZNS (only overall NVMe cleanups). Dmitry's ZNS patches have been on the list since 2020-06-16.
Yeah, as I mentioned in my cover letter, I was on leave, so I wasn't around for the big ZNS release day either. But, honestly, I think this is irrelevant - code should be merged based on technical reasons (not technicalities).
Quoted 12 lines
Just a friendly suggestion, how about:
1) We get your
hw/block/nvme: bump to v1.3
hw/block/nvme: support scatter gather lists
hw/block/nvme: support multiple namespaces
hw/block/nvme: bump to v1.4patch series merged.
Blowing my own horn here, but yeah, it seems like everyone would like to see this merged.
Quoted 5 lines
2) We get Dmitry's patch series merged.
Shared 4:th) If there is any feature that you miss in Dmitry's patch series, perhaps you could send patches to add what you are missing.
Looks like the two version are pretty much on par in terms of features.
Quoted 3 lines
Shared 4:th) Your other patch series:
hw/block/nvme: AIO and address mapping refactoring could get merged.As stated above I think its only a single commit ("hw/block/nvme: allow multiple aios per command") that is controversial in that series.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
Please don't take this suggestion the wrong way, I'm simply trying to come up with a way to move forward from here.
Absolutely - I totally get that you want to move forward with Dmitry's series, but I'd like to finish my review before committing to anything.
Klaus,
I see that ZNS series from WDC is pretty much orthogonal to most of your patches from "bump to 1.3/bump to 1.4" series... with a few notable exceptions - I had to add support for Get Log Page command, Command Effects Log and AERs to fulfill ZNS protocol requirements. I understand that you've had that functionality already staged in your "bump to 1.3" patchset and beyond and I don't insist that the implementations of these features from our ZNS series are necessarily to be used. I think Niklas' plan appears to be quite productive - for us to rebase on top of "bump to 1.3..1.4" and review these patches in the process, apply our series and then incorporate some of the ideas from your ZNS/NSTypes patchset on top of that. This way, very little work will go to waste.
Cheers,
Dmitry
Quoted 3 lines
Cheers,
Klaus