Skip to contentSkip to conversation

hw/block/nvme: support multi-path for ctrl/ns

RFC PATCH Superseded Minwoo Im 4 participants 21 messages Jan 19, 2021 – Jan 21, 2021 Arm/Amlogic Meson SoC support

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Minwoo Im

author
#0 lore raw

v3 cover hw/block/nvme: support multi-path for ctrl/ns

Hello,

This patch series is third one to support multi-controller and namespace sharing in multi-path. This series introduced subsystem scheme to manage controller(s) and namespace(s) in the subsystem.

This series has new patches from the V2: 'detached' parameter has been added to the nvme-ns device. This will decide whether to attach the namespace to controller(s) in the current subsystem or not. If it's given with true, then it will be just allocated in the subsystem, but not attaching to any controllers in the subsystem. Otherwise, it will automatically attach to all the controllers in the subsystem. The other t hing is that the last patch implemented Identify Active Namespace ID List command handler apart from the Allocated Namespace ID List.

Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2

nvme-cli:

  root@vm:~/work# nvme list -v                                                                                                      
  NVM Express Subsystems                                                                                                 

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

  nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------

  nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0
  nvme1    foo                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:07.0   nvme-subsys1
  nvme2    bar                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:08.0   nvme-subsys1
  nvme3    baz                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:09.0   nvme-subsys1 nvme1n1

NVM Express Namespaces

Device NSID Usage Format Controllers
------------ -------- -------------------------- ---------------- ----------------

  nvme0n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme0
  nvme1n1      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme3

Now we have [allocated|not-allocated]/[attached/detached] scheme for namespaces and controllers. The next series is going to be to support namespace management and attachment with few Identify command handlers.

Please review.

Thanks!

Since RFC V2:
- Rebased on nvme-next branch with trivial patches from the previous version(V2) applied. (Klaus)
- Fix enumeration type name convention with NvmeIdNs prefix. (Klaus)
- Put 'cntlid' to NvmeCtrl instance in nvme_init_ctrl() which was missed in V2.
- Added 'detached' parameter to nvme-ns device to decide whether to attach or not to controller(s) in the subsystem. (Klaus)
- Implemented Identify Active Namespace ID List aprt from Identify Allocated Namespace ID List by removing fall-thru statement.

Since RFC V1:
- Updated namespace sharing scheme to be based on nvme-subsys hierarchy.

Minwoo Im (8):
hw/block/nvme: introduce nvme-subsys device hw/block/nvme: support to map controller to a subsystem hw/block/nvme: add CMIC enum value for Identify Controller hw/block/nvme: support for multi-controller in subsystem hw/block/nvme: add NMIC enum value for Identify Namespace hw/block/nvme: support for shared namespace in subsystem hw/block/nvme: add 'detached' param not to attach namespace hw/block/nvme: Add Identify Active Namespace ID List

 hw/block/meson.build   |   2 +-
 hw/block/nvme-ns.c     |  32 ++++++++--
 hw/block/nvme-ns.h     |  13 ++++
 hw/block/nvme-subsys.c | 111 +++++++++++++++++++++++++++++++++
 hw/block/nvme-subsys.h |  32 ++++++++++
 hw/block/nvme.c        | 137 ++++++++++++++++++++++++++++++++++++++---
 hw/block/nvme.h        |  19 ++++++
 include/block/nvme.h   |   8 +++
 8 files changed, 340 insertions(+), 14 deletions(-)
 create mode 100644 hw/block/nvme-subsys.c
 create mode 100644 hw/block/nvme-subsys.h
Signature · 1 line
2.17.1
author
#1 lore raw

RFC PATCH v3 1/8 hw/block/nvme: introduce nvme-subsys device

To support multi-path in QEMU NVMe device model, We need to have NVMe subsystem hierarchy to map controllers and namespaces to a NVMe subsystem.

This patch introduced a simple nvme-subsys device model. The subsystem will be prepared with subsystem NQN with <subsys_id> provided in nvme-subsys device:

ex) -device nvme-subsys,id=subsys0: nqn.2019-08.org.qemu:subsys0

---
 hw/block/meson.build   |  2 +-
 hw/block/nvme-subsys.c | 63 ++++++++++++++++++++++++++++++++++++++++++
 hw/block/nvme-subsys.h | 25 +++++++++++++++++
 hw/block/nvme.c        |  3 ++
 4 files changed, 92 insertions(+), 1 deletion(-)
 create mode 100644 hw/block/nvme-subsys.c
 create mode 100644 hw/block/nvme-subsys.h
hw/block/meson.build +1 -1
index 602ca6c8541d..83ea2d37978d 100644
@@ -13,7 +13,7 @@ softmmu_ss.add(when: 'CONFIG_SSI_M25P80', if_true: files('m25p80.c'))
 softmmu_ss.add(when: 'CONFIG_SWIM', if_true: files('swim.c'))
 softmmu_ss.add(when: 'CONFIG_XEN', if_true: files('xen-block.c'))
 softmmu_ss.add(when: 'CONFIG_SH4', if_true: files('tc58128.c'))
-softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('nvme.c', 'nvme-ns.c'))
+softmmu_ss.add(when: 'CONFIG_NVME_PCI', if_true: files('nvme.c', 'nvme-ns.c', 'nvme-subsys.c'))
 
 specific_ss.add(when: 'CONFIG_VIRTIO_BLK', if_true: files('virtio-blk.c'))
 specific_ss.add(when: 'CONFIG_VHOST_USER_BLK', if_true: files('vhost-user-blk.c'))
hw/block/nvme-subsys.c new file +63 -0

63 lines not on this page · raw message

hw/block/nvme-subsys.h new file +25 -0
new file mode 100644
index 000000000000..40f06a4c7db0
@@ -0,0 +1,25 @@
+/*
+ * QEMU NVM Express Subsystem: nvme-subsys
+ *
+ * Copyright (c) 2021 Minwoo Im <[email protected]>
+ *
+ * This code is licensed under the GNU GPL v2.  Refer COPYING.
+ */
+
+#ifndef NVME_SUBSYS_H
+#define NVME_SUBSYS_H
+
+#define TYPE_NVME_SUBSYS "nvme-subsys"
+#define NVME_SUBSYS(obj) \
+    OBJECT_CHECK(NvmeSubsystem, (obj), TYPE_NVME_SUBSYS)
+
+#define NVME_SUBSYS_MAX_CTRLS   32
+
+typedef struct NvmeCtrl NvmeCtrl;
+typedef struct NvmeNamespace NvmeNamespace;
+typedef struct NvmeSubsystem {
+    DeviceState parent_obj;
+    uint8_t     subnqn[256];
+} NvmeSubsystem;
+
+#endif /* NVME_SUBSYS_H */
hw/block/nvme.c +3 -0
index 309c26db8ff7..4644937a5c50 100644
@@ -25,6 +25,7 @@
  *              mdts=<N[optional]>,zoned.append_size_limit=<N[optional]> \
  *      -device nvme-ns,drive=<drive_id>,bus=<bus_name>,nsid=<nsid>,\
  *              zoned=<true|false[optional]>
+ *      -device nvme-subsys,id=<subsys_id>
  *
  * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at
  * offset 0 in BAR2 and supports only WDS, RDS and SQS for now.
@@ -37,6 +38,8 @@
  * -object memory-backend-file,id=<mem_id>,share=on,mem-path=<file_path>, \
  *  size=<size> .... -device nvme,...,pmrdev=<mem_id>
  *
+ * To place controller(s) and namespace(s) to a subsystem, then provide
+ * nvme-subsys device as above.
  *
  * nvme device parameters
  * ~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.17.1
author
#2 lore raw

RFC PATCH v3 2/8 hw/block/nvme: support to map controller to a subsystem

In reply to Minwoo Im

nvme controller(nvme) can be mapped to a NVMe subsystem(nvme-subsys). This patch maps a controller to a subsystem by adding a parameter 'subsys' to the nvme device.

To map a controller to a subsystem, we need to put nvme-subsys first and then maps the subsystem to the controller:

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0

If 'subsys' property is not given to the nvme controller, then subsystem NQN will be created with serial (e.g., 'foo' in above example), Otherwise, it will be based on subsys id (e.g., 'subsys0' in above example).

---
 hw/block/nvme.c | 30 ++++++++++++++++++++++++++----
 hw/block/nvme.h |  3 +++
 2 files changed, 29 insertions(+), 4 deletions(-)
hw/block/nvme.c +26 -4
index 4644937a5c50..3e3b5451ea3d 100644
@@ -23,6 +23,7 @@
  *              max_ioqpairs=<N[optional]>, \
  *              aerl=<N[optional]>, aer_max_queued=<N[optional]>, \
  *              mdts=<N[optional]>,zoned.append_size_limit=<N[optional]> \
+ *              ,subsys=<subsys_id> \
  *      -device nvme-ns,drive=<drive_id>,bus=<bus_name>,nsid=<nsid>,\
  *              zoned=<true|false[optional]>
  *      -device nvme-subsys,id=<subsys_id>
@@ -43,6 +44,13 @@
  *
  * nvme device parameters
  * ~~~~~~~~~~~~~~~~~~~~~~
+ * - `subsys`
+ *   NVM Subsystem device. If given, a subsystem NQN will be initialized with
+ *   <subsys_id> given. Otherwise, <serial> will be taken for subsystem NQN.
+ *   Also, it will enable multi controller capability represented in Identify
+ *   Controller data structure in CMIC (Controller Multi-path I/O and Namesapce
+ *   Sharing Capabilities), if given.
+ *
  * - `aerl`
  *   The Asynchronous Event Request Limit (AERL). Indicates the maximum number
  *   of concurrently outstanding Asynchronous Event Request commands support
@@ -4281,11 +4289,25 @@ static int nvme_init_pci(NvmeCtrl *n, PCIDevice *pci_dev, Error **errp)
     return 0;
 }
 
+static void nvme_init_subnqn(NvmeCtrl *n)
+{
+    NvmeSubsystem *subsys = n->subsys;
+    NvmeIdCtrl *id = &n->id_ctrl;
+    char *subnqn;
+
+    if (!subsys) {
+        subnqn = g_strdup_printf("nqn.2019-08.org.qemu:%s", n->params.serial);
+        strpadcpy((char *)id->subnqn, sizeof(id->subnqn), subnqn, '\0');
+        g_free(subnqn);
+    } else {
+        pstrcpy((char *)id->subnqn, sizeof(id->subnqn), (char*)subsys->subnqn);
+    }
+}
+
 static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev)
 {
     NvmeIdCtrl *id = &n->id_ctrl;
     uint8_t *pci_conf = pci_dev->config;
-    char *subnqn;
 
     id->vid = cpu_to_le16(pci_get_word(pci_conf + PCI_VENDOR_ID));
     id->ssvid = cpu_to_le16(pci_get_word(pci_conf + PCI_SUBSYSTEM_VENDOR_ID));
@@ -4331,9 +4353,7 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev)
     id->sgls = cpu_to_le32(NVME_CTRL_SGLS_SUPPORT_NO_ALIGN |
                            NVME_CTRL_SGLS_BITBUCKET);
 
-    subnqn = g_strdup_printf("nqn.2019-08.org.qemu:%s", n->params.serial);
-    strpadcpy((char *)id->subnqn, sizeof(id->subnqn), subnqn, '\0');
-    g_free(subnqn);
+    nvme_init_subnqn(n);
 
     id->psd[0].mp = cpu_to_le16(0x9c4);
     id->psd[0].enlat = cpu_to_le32(0x10);
@@ -4417,6 +4437,8 @@ static void nvme_exit(PCIDevice *pci_dev)
 
 static Property nvme_props[] = {
     DEFINE_BLOCK_PROPERTIES(NvmeCtrl, namespace.blkconf),
+    DEFINE_PROP_LINK("subsys", NvmeCtrl, subsys, TYPE_NVME_SUBSYS,
+                     NvmeSubsystem *),
     DEFINE_PROP_LINK("pmrdev", NvmeCtrl, pmrdev, TYPE_MEMORY_BACKEND,
                      HostMemoryBackend *),
     DEFINE_PROP_STRING("serial", NvmeCtrl, params.serial),
hw/block/nvme.h +3 -0
index 347c149e7905..3fa0e0a15539 100644
@@ -2,6 +2,7 @@
 #define HW_NVME_H
 
 #include "block/nvme.h"
+#include "nvme-subsys.h"
 #include "nvme-ns.h"
 
 #define NVME_MAX_NAMESPACES 256
@@ -158,6 +159,8 @@ typedef struct NvmeCtrl {
 
     uint8_t     zasl;
 
+    NvmeSubsystem   *subsys;
+
     NvmeNamespace   namespace;
     NvmeNamespace   *namespaces[NVME_MAX_NAMESPACES];
     NvmeSQueue      **sq;
Signature · 1 line
2.17.1
author
#3 lore raw

RFC PATCH v3 3/8 hw/block/nvme: add CMIC enum value for Identify Controller

In reply to Minwoo Im

Added Controller Multi-path I/O and Namespace Sharing Capabilities (CMIC) field to support multi-controller in the following patches.

This field is in Identify Controller data structure in [76].

---
 include/block/nvme.h | 4 ++++
 1 file changed, 4 insertions(+)
include/block/nvme.h +4 -0
index 45b2678db1f0..733fb35fedde 100644
@@ -941,6 +941,10 @@ enum NvmeIdCtrlLpa {
     NVME_LPA_EXTENDED = 1 << 2,
 };
 
+enum NvmeIdCtrlCmic {
+    NVME_CMIC_MULTI_CTRL    = 1 << 1,
+};
+
 #define NVME_CTRL_SQES_MIN(sqes) ((sqes) & 0xf)
 #define NVME_CTRL_SQES_MAX(sqes) (((sqes) >> 4) & 0xf)
 #define NVME_CTRL_CQES_MIN(cqes) ((cqes) & 0xf)
Signature · 1 line
2.17.1
author
#4 lore raw

RFC PATCH v3 4/8 hw/block/nvme: support for multi-controller in subsystem

In reply to Minwoo Im

We have nvme-subsys and nvme devices mapped together. To support multi-controller scheme to this setup, controller identifier(id) has to be managed. Earlier, cntlid(controller id) used to be always 0 because we didn't have any subsystem scheme that controller id matters.

This patch introduced 'cntlid' attribute to the nvme controller instance(NvmeCtrl) and make it allocated by the nvme-subsys device mapped to the controller. If nvme-subsys is not given to the controller, then it will always be 0 as it was.

Added 'ctrls' array in the nvme-subsys instance to manage attached controllers to the subsystem with a limit(32). This patch didn't take list for the controllers to make it seamless with nvme-ns device.

---
 hw/block/nvme-subsys.c | 21 +++++++++++++++++++++
 hw/block/nvme-subsys.h |  4 ++++
 hw/block/nvme.c        | 34 ++++++++++++++++++++++++++++++++--
 hw/block/nvme.h        |  1 +
 4 files changed, 58 insertions(+), 2 deletions(-)
hw/block/nvme-subsys.c +21 -0
index f1dc71d588d9..a01003136b12 100644
@@ -22,6 +22,27 @@
 #include "nvme.h"
 #include "nvme-subsys.h"
 
+int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp)
+{
+    NvmeSubsystem *subsys = n->subsys;
+    int cntlid;
+
+    for (cntlid = 0; cntlid < ARRAY_SIZE(subsys->ctrls); cntlid++) {
+        if (!subsys->ctrls[cntlid]) {
+            break;
+        }
+    }
+
+    if (cntlid == ARRAY_SIZE(subsys->ctrls)) {
+        error_setg(errp, "no more free controller id");
+        return -1;
+    }
+
+    subsys->ctrls[cntlid] = n;
+
+    return cntlid;
+}
+
 static void nvme_subsys_setup(NvmeSubsystem *subsys)
 {
     char *subnqn;
hw/block/nvme-subsys.h +4 -0
index 40f06a4c7db0..4eba50d96a1d 100644
@@ -20,6 +20,10 @@ typedef struct NvmeNamespace NvmeNamespace;
 typedef struct NvmeSubsystem {
     DeviceState parent_obj;
     uint8_t     subnqn[256];
+
+    NvmeCtrl    *ctrls[NVME_SUBSYS_MAX_CTRLS];
 } NvmeSubsystem;
 
+int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp);
+
 #endif /* NVME_SUBSYS_H */
hw/block/nvme.c +32 -2
index 3e3b5451ea3d..9f8a739fcd8f 100644
@@ -4304,16 +4304,21 @@ static void nvme_init_subnqn(NvmeCtrl *n)
     }
 }
 
-static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev)
+static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev, uint16_t cntlid)
 {
     NvmeIdCtrl *id = &n->id_ctrl;
     uint8_t *pci_conf = pci_dev->config;
 
+    n->cntlid = cntlid;
+
     id->vid = cpu_to_le16(pci_get_word(pci_conf + PCI_VENDOR_ID));
     id->ssvid = cpu_to_le16(pci_get_word(pci_conf + PCI_SUBSYSTEM_VENDOR_ID));
     strpadcpy((char *)id->mn, sizeof(id->mn), "QEMU NVMe Ctrl", ' ');
     strpadcpy((char *)id->fr, sizeof(id->fr), "1.0", ' ');
     strpadcpy((char *)id->sn, sizeof(id->sn), n->params.serial, ' ');
+
+    id->cntlid = cntlid;
+
     id->rab = 6;
     id->ieee[0] = 0x00;
     id->ieee[1] = 0x02;
@@ -4359,6 +4364,10 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev)
     id->psd[0].enlat = cpu_to_le32(0x10);
     id->psd[0].exlat = cpu_to_le32(0x4);
 
+    if (n->subsys) {
+        id->cmic |= NVME_CMIC_MULTI_CTRL;
+    }
+
     NVME_CAP_SET_MQES(n->bar.cap, 0x7ff);
     NVME_CAP_SET_CQR(n->bar.cap, 1);
     NVME_CAP_SET_TO(n->bar.cap, 0xf);
@@ -4371,11 +4380,28 @@ static void nvme_init_ctrl(NvmeCtrl *n, PCIDevice *pci_dev)
     n->bar.intmc = n->bar.intms = 0;
 }
 
+static int nvme_init_subsys(NvmeCtrl *n, Error **errp)
+{
+    int cntlid;
+
+    if (!n->subsys) {
+        return 0;
+    }
+
+    cntlid = nvme_subsys_register_ctrl(n, errp);
+    if (cntlid < 0) {
+        return -1;
+    }
+
+    return cntlid;
+}
+
 static void nvme_realize(PCIDevice *pci_dev, Error **errp)
 {
     NvmeCtrl *n = NVME(pci_dev);
     NvmeNamespace *ns;
     Error *local_err = NULL;
+    int cntlid;
 
     nvme_check_constraints(n, &local_err);
     if (local_err) {
@@ -4391,7 +4417,11 @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
         return;
     }
 
-    nvme_init_ctrl(n, pci_dev);
+    cntlid = nvme_init_subsys(n, errp);
+    if (cntlid < 0) {
+        return;
+    }
+    nvme_init_ctrl(n, pci_dev, cntlid);
 
     /* setup a namespace if the controller drive property was given */
     if (n->namespace.blkconf.blk) {
hw/block/nvme.h +1 -0
index 3fa0e0a15539..c158cc873b59 100644
@@ -133,6 +133,7 @@ typedef struct NvmeCtrl {
     NvmeBus      bus;
     BlockConf    conf;
 
+    uint16_t    cntlid;
     bool        qs_created;
     uint32_t    page_size;
     uint16_t    page_bits;
Signature · 1 line
2.17.1
author
#5 lore raw

RFC PATCH v3 5/8 hw/block/nvme: add NMIC enum value for Identify Namespace

In reply to Minwoo Im

Added Namespace Multi-path I/O and Namespace Sharing Capabilities (NMIC) field to support shared namespace from controller(s).

This field is in Identify Namespace data structure in [30].

---
 include/block/nvme.h | 4 ++++
 1 file changed, 4 insertions(+)
include/block/nvme.h +4 -0
index 733fb35fedde..28404a62b9d2 100644
@@ -1110,6 +1110,10 @@ enum NvmeNsIdentifierType {
     NVME_NIDT_CSI               = 0x04,
 };
 
+enum NvmeIdNsNmic {
+    NVME_NMIC_NS_SHARED         = 1 << 0,
+};
+
 enum NvmeCsi {
     NVME_CSI_NVM                = 0x00,
     NVME_CSI_ZONED              = 0x02,
Signature · 1 line
2.17.1
author
#6 lore raw

RFC PATCH v3 6/8 hw/block/nvme: support for shared namespace in subsystem

In reply to Minwoo Im

nvme-ns device is registered to a nvme controller device during the initialization in nvme_register_namespace() in case that 'bus' property is given which means it's mapped to a single controller.

This patch introduced a new property 'subsys' just like the controller device instance did to map a namespace to a NVMe subsystem.

If 'subsys' property is given to the nvme-ns device, it will belong to the specified subsystem and will be attached to all controllers in that subsystem by enabling shared namespace capability in NMIC(Namespace Multi-path I/O and Namespace Capabilities) in Identify Namespace.

Usage:

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=<drv>,nsid=1,subsys=subsys0 # Shared
-device nvme-ns,id=ns2,drive=<drv>,nsid=2,bus=nvme2 # Non-shared

In the above example, 'ns1' will be shared to 'nvme0' and 'nvme1' in the same subsystem. On the other hand, 'ns2' will be attached to the 'nvme2' only as a private namespace in that subsystem.

All the namespace with 'subsys' parameter will attach all controllers in the subsystem to the namespace by default.

---
 hw/block/nvme-ns.c     | 23 ++++++++++++++++++-----
 hw/block/nvme-ns.h     |  7 +++++++
 hw/block/nvme-subsys.c | 25 +++++++++++++++++++++++++
 hw/block/nvme-subsys.h |  3 +++
 hw/block/nvme.c        | 10 +++++++++-
 5 files changed, 62 insertions(+), 6 deletions(-)
hw/block/nvme-ns.c +18 -5
index c8b75fa02138..073f65e49cac 100644
@@ -63,6 +63,10 @@ static int nvme_ns_init(NvmeNamespace *ns, Error **errp)
 
     id_ns->npda = id_ns->npdg = npdg - 1;
 
+    if (nvme_ns_shared(ns)) {
+        id_ns->nmic |= NVME_NMIC_NS_SHARED;
+    }
+
     return 0;
 }
 
@@ -358,16 +362,25 @@ static void nvme_ns_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (nvme_register_namespace(n, ns, errp)) {
-        error_propagate_prepend(errp, local_err,
-                                "could not register namespace: ");
-        return;
+    if (ns->subsys) {
+        if (nvme_subsys_register_ns(ns, errp)) {
+            error_propagate_prepend(errp, local_err,
+                                    "could not setup namespace to subsys: ");
+            return;
+        }
+    } else {
+        if (nvme_register_namespace(n, ns, errp)) {
+            error_propagate_prepend(errp, local_err,
+                                    "could not register namespace: ");
+            return;
+        }
     }
-
 }
 
 static Property nvme_ns_props[] = {
     DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf),
+    DEFINE_PROP_LINK("subsys", NvmeNamespace, subsys, TYPE_NVME_SUBSYS,
+                     NvmeSubsystem *),
     DEFINE_PROP_UINT32("nsid", NvmeNamespace, params.nsid, 0),
     DEFINE_PROP_UUID("uuid", NvmeNamespace, params.uuid),
     DEFINE_PROP_BOOL("zoned", NvmeNamespace, params.zoned, false),
hw/block/nvme-ns.h +7 -0
index 293ac990e3f6..929e78861903 100644
@@ -47,6 +47,8 @@ typedef struct NvmeNamespace {
     const uint32_t *iocs;
     uint8_t      csi;
 
+    NvmeSubsystem   *subsys;
+
     NvmeIdNsZoned   *id_ns_zoned;
     NvmeZone        *zone_array;
     QTAILQ_HEAD(, NvmeZone) exp_open_zones;
@@ -77,6 +79,11 @@ static inline uint32_t nvme_nsid(NvmeNamespace *ns)
     return -1;
 }
 
+static inline bool nvme_ns_shared(NvmeNamespace *ns)
+{
+    return !!ns->subsys;
+}
+
 static inline NvmeLBAF *nvme_ns_lbaf(NvmeNamespace *ns)
 {
     NvmeIdNs *id_ns = &ns->id_ns;
hw/block/nvme-subsys.c +25 -0
index a01003136b12..e7efdcae7d0d 100644
@@ -43,6 +43,31 @@ int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp)
     return cntlid;
 }
 
+int nvme_subsys_register_ns(NvmeNamespace *ns, Error **errp)
+{
+    NvmeSubsystem *subsys = ns->subsys;
+    NvmeCtrl *n;
+    int i;
+
+    if (subsys->namespaces[nvme_nsid(ns)]) {
+        error_setg(errp, "namespace %d already registerd to subsy %s",
+                   nvme_nsid(ns), subsys->parent_obj.id);
+        return -1;
+    }
+
+    subsys->namespaces[nvme_nsid(ns)] = ns;
+
+    for (i = 0; i < ARRAY_SIZE(subsys->ctrls); i++) {
+        n = subsys->ctrls[i];
+
+        if (n && nvme_register_namespace(n, ns, errp)) {
+            return -1;
+        }
+    }
+
+    return 0;
+}
+
 static void nvme_subsys_setup(NvmeSubsystem *subsys)
 {
     char *subnqn;
hw/block/nvme-subsys.h +3 -0
index 4eba50d96a1d..ccf6a71398d3 100644
@@ -14,6 +14,7 @@
     OBJECT_CHECK(NvmeSubsystem, (obj), TYPE_NVME_SUBSYS)
 
 #define NVME_SUBSYS_MAX_CTRLS   32
+#define NVME_SUBSYS_MAX_NAMESPACES  32
 
 typedef struct NvmeCtrl NvmeCtrl;
 typedef struct NvmeNamespace NvmeNamespace;
@@ -22,8 +23,10 @@ typedef struct NvmeSubsystem {
     uint8_t     subnqn[256];
 
     NvmeCtrl    *ctrls[NVME_SUBSYS_MAX_CTRLS];
+    NvmeNamespace *namespaces[NVME_SUBSYS_MAX_NAMESPACES];
 } NvmeSubsystem;
 
 int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp);
+int nvme_subsys_register_ns(NvmeNamespace *ns, Error **errp);
 
 #endif /* NVME_SUBSYS_H */
hw/block/nvme.c +9 -1
index 9f8a739fcd8f..06bccf1b9e9e 100644
@@ -25,7 +25,8 @@
  *              mdts=<N[optional]>,zoned.append_size_limit=<N[optional]> \
  *              ,subsys=<subsys_id> \
  *      -device nvme-ns,drive=<drive_id>,bus=<bus_name>,nsid=<nsid>,\
- *              zoned=<true|false[optional]>
+ *              zoned=<true|false[optional]>, \
+ *              subsys=<subsys_id>
  *      -device nvme-subsys,id=<subsys_id>
  *
  * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at
@@ -69,6 +70,13 @@
  *   data size being in effect. By setting this property to 0, users can make
  *   ZASL to be equal to MDTS. This property only affects zoned namespaces.
  *
+ * nvme namespace device parameters
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ * - `subsys`
+ *   NVM Subsystem device.  If given, this namespace will be attached to all
+ *   controllers in the subsystem. Otherwise, `bus` must be given to attach
+ *   this namespace to a specified single controller as a non-shared namespace.
+ *
  * Setting `zoned` to true selects Zoned Command Set at the namespace.
  * In this case, the following namespace properties are available to configure
  * zoned operation:
Signature · 1 line
2.17.1
author
#7 lore raw

RFC PATCH v3 7/8 hw/block/nvme: add 'detached' param not to attach namespace

In reply to Minwoo Im

Introduced 'detached' parameter to nvme-ns device. If given, the namespace will not be attached to controller(s) in the subsystem. If 'subsys' is not given with this option, it should be provided with 'bus' which is for private namespace.

This patch also introduced 'ctrls_bitmap' in NvmeNamespace instance to represent which controler id(cntlid) is attached to this namespace device. A namespace can be attached to multiple controllers in a subsystem so that this bitmap maps those two relationships.

The ctrls_bitmap bitmap should not be accessed directly, but through the helpers introduced in this patch: nvme_ns_is_attached(), nvme_ns_attach(), nvme_ns_detach().

Note that this patch made identify namespace list data not hold non-attached namespace ID in nvme_identify_nslist. Currently, this command handler is for CNS 0x2(Active) and 0x10(Allocated) both. The next patch will introduce a handler for later on.

---
 hw/block/nvme-ns.c     |  9 +++++++++
 hw/block/nvme-ns.h     |  6 ++++++
 hw/block/nvme-subsys.c |  2 ++
 hw/block/nvme.c        | 31 ++++++++++++++++++++++++++++++-
 hw/block/nvme.h        | 15 +++++++++++++++
 5 files changed, 62 insertions(+), 1 deletion(-)
hw/block/nvme-ns.c +9 -0
index 073f65e49cac..70d42c24065c 100644
@@ -17,6 +17,7 @@
 #include "qemu/cutils.h"
 #include "qemu/log.h"
 #include "qemu/error-report.h"
+#include "qemu/hbitmap.h"
 #include "hw/block/block.h"
 #include "hw/pci/pci.h"
 #include "sysemu/sysemu.h"
@@ -304,6 +305,11 @@ static int nvme_ns_check_constraints(NvmeNamespace *ns, Error **errp)
     return 0;
 }
 
+static void nvme_ns_init_state(NvmeNamespace *ns)
+{
+    ns->ctrls_bitmap = hbitmap_alloc(NVME_SUBSYS_MAX_CTRLS, 0);
+}
+
 int nvme_ns_setup(NvmeNamespace *ns, Error **errp)
 {
     if (nvme_ns_check_constraints(ns, errp)) {
@@ -314,6 +320,8 @@ int nvme_ns_setup(NvmeNamespace *ns, Error **errp)
         return -1;
     }
 
+    nvme_ns_init_state(ns);
+
     if (nvme_ns_init(ns, errp)) {
         return -1;
     }
@@ -381,6 +389,7 @@ static Property nvme_ns_props[] = {
     DEFINE_BLOCK_PROPERTIES(NvmeNamespace, blkconf),
     DEFINE_PROP_LINK("subsys", NvmeNamespace, subsys, TYPE_NVME_SUBSYS,
                      NvmeSubsystem *),
+    DEFINE_PROP_BOOL("detached", NvmeNamespace, params.detached, false),
     DEFINE_PROP_UINT32("nsid", NvmeNamespace, params.nsid, 0),
     DEFINE_PROP_UUID("uuid", NvmeNamespace, params.uuid),
     DEFINE_PROP_BOOL("zoned", NvmeNamespace, params.zoned, false),
hw/block/nvme-ns.h +6 -0
index 929e78861903..ad2f55931d1b 100644
@@ -26,6 +26,7 @@ typedef struct NvmeZone {
 } NvmeZone;
 
 typedef struct NvmeNamespaceParams {
+    bool     detached;
     uint32_t nsid;
     QemuUUID uuid;
 
@@ -48,6 +49,11 @@ typedef struct NvmeNamespace {
     uint8_t      csi;
 
     NvmeSubsystem   *subsys;
+    /*
+     * Whether this namespace is attached to a controller or not.  This bitmap
+     * is based on controller id.  This is valid only in case 'subsys' != NULL.
+     */
+    HBitmap         *ctrls_bitmap;
 
     NvmeIdNsZoned   *id_ns_zoned;
     NvmeZone        *zone_array;
hw/block/nvme-subsys.c +2 -0
index e7efdcae7d0d..32ad8ef2825a 100644
@@ -11,6 +11,7 @@
 #include "qemu/uuid.h"
 #include "qemu/iov.h"
 #include "qemu/cutils.h"
+#include "qemu/hbitmap.h"
 #include "qapi/error.h"
 #include "hw/qdev-properties.h"
 #include "hw/qdev-core.h"
@@ -20,6 +21,7 @@
 #include "sysemu/sysemu.h"
 #include "hw/pci/pci.h"
 #include "nvme.h"
+#include "nvme-ns.h"
 #include "nvme-subsys.h"
 
 int nvme_subsys_register_ctrl(NvmeCtrl *n, Error **errp)
hw/block/nvme.c +30 -1
index 06bccf1b9e9e..2b2c07b36c2b 100644
@@ -26,7 +26,7 @@
  *              ,subsys=<subsys_id> \
  *      -device nvme-ns,drive=<drive_id>,bus=<bus_name>,nsid=<nsid>,\
  *              zoned=<true|false[optional]>, \
- *              subsys=<subsys_id>
+ *              subsys=<subsys_id>,detached=<true|false[optional]
  *      -device nvme-subsys,id=<subsys_id>
  *
  * Note cmb_size_mb denotes size of CMB in MB. CMB is assumed to be at
@@ -77,6 +77,13 @@
  *   controllers in the subsystem. Otherwise, `bus` must be given to attach
  *   this namespace to a specified single controller as a non-shared namespace.
  *
+ * - `detached`
+ *   Not to attach the namespace device to controllers in the NVMe subsystem
+ *   during boot-up.  If not given, namespaces are all attached to all
+ *   controllers in the subsystem by default.
+ *   It's mutual exclusive with 'bus' paraemter.  It's only valid in case
+ *   'subsys' is provided.
+ *
  * Setting `zoned` to true selects Zoned Command Set at the namespace.
  * In this case, the following namespace properties are available to configure
  * zoned operation:
@@ -2906,6 +2913,11 @@ static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
         if (ns->params.nsid <= min_nsid) {
             continue;
         }
+
+        if (!nvme_ns_is_attached(n, ns)) {
+            continue;
+        }
+
         list_ptr[j++] = cpu_to_le32(ns->params.nsid);
         if (j == data_len / sizeof(uint32_t)) {
             break;
@@ -4146,6 +4158,19 @@ static void nvme_init_state(NvmeCtrl *n)
     n->aer_reqs = g_new0(NvmeRequest *, n->params.aerl + 1);
 }
 
+static int nvme_attach_namespace(NvmeCtrl *n, NvmeNamespace *ns, Error **errp)
+{
+    if (nvme_ns_is_attached(n, ns)) {
+        error_setg(errp,
+                   "namespace %d is already attached to controller %d",
+                   nvme_nsid(ns), n->cntlid);
+        return -1;
+    }
+
+    nvme_ns_attach(n, ns);
+    return 0;
+}
+
 int nvme_register_namespace(NvmeCtrl *n, NvmeNamespace *ns, Error **errp)
 {
     uint32_t nsid = nvme_nsid(ns);
@@ -4179,6 +4204,10 @@ int nvme_register_namespace(NvmeCtrl *n, NvmeNamespace *ns, Error **errp)
 
     n->namespaces[nsid - 1] = ns;
 
+    if (!ns->params.detached) {
+        return nvme_attach_namespace(n, ns, errp);
+    }
+
     return 0;
 }
 
hw/block/nvme.h +15 -0
index c158cc873b59..582e6d4e8c40 100644
@@ -181,6 +181,21 @@ static inline NvmeNamespace *nvme_ns(NvmeCtrl *n, uint32_t nsid)
     return n->namespaces[nsid - 1];
 }
 
+static inline bool nvme_ns_is_attached(NvmeCtrl *n, NvmeNamespace *ns)
+{
+    return hbitmap_get(ns->ctrls_bitmap, n->cntlid);
+}
+
+static inline void nvme_ns_attach(NvmeCtrl *n, NvmeNamespace *ns)
+{
+    hbitmap_set(ns->ctrls_bitmap, n->cntlid, 1);
+}
+
+static inline void nvme_ns_detach(NvmeCtrl *n, NvmeNamespace *ns)
+{
+    hbitmap_reset(ns->ctrls_bitmap, n->cntlid, 1);
+}
+
 static inline NvmeCQueue *nvme_cq(NvmeRequest *req)
 {
     NvmeSQueue *sq = req->sq;
Signature · 1 line
2.17.1
#8 lore raw

Re: [RFC PATCH V3 7/8] hw/block/nvme: add 'detached' param not to attach namespace

Quoted 33 lines · Minwoo Im

Introduced 'detached' parameter to nvme-ns device. If given, the namespace will not be attached to controller(s) in the subsystem. If 'subsys' is not given with this option, it should be provided with 'bus' which is for private namespace.

This patch also introduced 'ctrls_bitmap' in NvmeNamespace instance to represent which controler id(cntlid) is attached to this namespace device. A namespace can be attached to multiple controllers in a subsystem so that this bitmap maps those two relationships.

The ctrls_bitmap bitmap should not be accessed directly, but through the helpers introduced in this patch: nvme_ns_is_attached(), nvme_ns_attach(), nvme_ns_detach().

Note that this patch made identify namespace list data not hold non-attached namespace ID in nvme_identify_nslist. Currently, this command handler is for CNS 0x2(Active) and 0x10(Allocated) both. The next patch will introduce a handler for later on.

---
 hw/block/nvme-ns.c     |  9 +++++++++
 hw/block/nvme-ns.h     |  6 ++++++
 hw/block/nvme-subsys.c |  2 ++
 hw/block/nvme.c        | 31 ++++++++++++++++++++++++++++++-
 hw/block/nvme.h        | 15 +++++++++++++++
 5 files changed, 62 insertions(+), 1 deletion(-)
hw/block/nvme-ns.c +0 -0
index 073f65e49cac..70d42c24065c 100644

hw/block/nvme-ns.c line 20

Quoted 4 lines
hw/block/nvme-ns.c +1 -0
@@ -17,6 +17,7 @@
 #include "qemu/cutils.h"
 #include "qemu/log.h"
 #include "qemu/error-report.h"
+#include "qemu/hbitmap.h"

Isn't the HBitmap slightly overkill? Can qemu/bitmap.h suffice?

Quoted 3 lines
hw/block/nvme-ns.c +0 -0
 #include "hw/block/block.h"
 #include "hw/pci/pci.h"
 #include "sysemu/sysemu.h"
author
#9 lore raw

Re: [RFC PATCH V3 7/8] hw/block/nvme: add 'detached' param not to attach namespace

Quoted 1 line

Isn't the HBitmap slightly overkill? Can qemu/bitmap.h suffice?

Definitely, yes, I think. Current patch series supoprt up to 32 controllers so I think qemu/bitmap.h is enough for us.

Will update the bitmap operations in the next series.

#10 lore raw
In reply to Minwoo Im
Quoted 32 lines · Minwoo Im

Hello,

This patch series is third one to support multi-controller and namespace sharing in multi-path. This series introduced subsystem scheme to manage controller(s) and namespace(s) in the subsystem.

This series has new patches from the V2: 'detached' parameter has been added to the nvme-ns device. This will decide whether to attach the namespace to controller(s) in the current subsystem or not. If it's given with true, then it will be just allocated in the subsystem, but not attaching to any controllers in the subsystem. Otherwise, it will automatically attach to all the controllers in the subsystem. The other t hing is that the last patch implemented Identify Active Namespace ID List command handler apart from the Allocated Namespace ID List.

Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2

nvme-cli:

  root@vm:~/work# nvme list -v                                                                                                      
  NVM Express Subsystems                                                                                                 

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

Quoted 8 lines
  nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0

Shouldn't nvme0n1 be listed under Namespaces for nvme0?

Quoted 60 lines
  nvme1    foo                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:07.0   nvme-subsys1
  nvme2    bar                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:08.0   nvme-subsys1
  nvme3    baz                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:09.0   nvme-subsys1 nvme1n1

NVM Express Namespaces

Device NSID Usage Format Controllers
------------ -------- -------------------------- ---------------- ----------------

  nvme0n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme0
  nvme1n1      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme3

Now we have [allocated|not-allocated]/[attached/detached] scheme for namespaces and controllers. The next series is going to be to support namespace management and attachment with few Identify command handlers.

Please review.

Thanks!

Since RFC V2:
- Rebased on nvme-next branch with trivial patches from the previous version(V2) applied. (Klaus)
- Fix enumeration type name convention with NvmeIdNs prefix. (Klaus)
- Put 'cntlid' to NvmeCtrl instance in nvme_init_ctrl() which was missed in V2.
- Added 'detached' parameter to nvme-ns device to decide whether to attach or not to controller(s) in the subsystem. (Klaus)
- Implemented Identify Active Namespace ID List aprt from Identify Allocated Namespace ID List by removing fall-thru statement.

Since RFC V1:
- Updated namespace sharing scheme to be based on nvme-subsys hierarchy.

Minwoo Im (8):
hw/block/nvme: introduce nvme-subsys device hw/block/nvme: support to map controller to a subsystem hw/block/nvme: add CMIC enum value for Identify Controller hw/block/nvme: support for multi-controller in subsystem hw/block/nvme: add NMIC enum value for Identify Namespace hw/block/nvme: support for shared namespace in subsystem hw/block/nvme: add 'detached' param not to attach namespace hw/block/nvme: Add Identify Active Namespace ID List

 hw/block/meson.build   |   2 +-
 hw/block/nvme-ns.c     |  32 ++++++++--
 hw/block/nvme-ns.h     |  13 ++++
 hw/block/nvme-subsys.c | 111 +++++++++++++++++++++++++++++++++
 hw/block/nvme-subsys.h |  32 ++++++++++
 hw/block/nvme.c        | 137 ++++++++++++++++++++++++++++++++++++++---
 hw/block/nvme.h        |  19 ++++++
 include/block/nvme.h   |   8 +++
 8 files changed, 340 insertions(+), 14 deletions(-)
 create mode 100644 hw/block/nvme-subsys.c
 create mode 100644 hw/block/nvme-subsys.h
Signature · 1 line
2.17.1
Signature · 1 line
One of us - No more doubt, silence or taboo about mental illness.
#11 lore raw
Quoted 21 lines · Klaus Jensen
Quoted 20 lines · Minwoo Im

Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2

nvme-cli:

  root@vm:~/work# nvme list -v                                                                                                      
  NVM Express Subsystems                                                                                                 

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

  nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3
Quoted 7 lines
Quoted 5 lines

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0

Shouldn't nvme0n1 be listed under Namespaces for nvme0?

Minwoo, try pulling the most current nvme-cli. There was a sysfs scanning bug for non-mpath drives that should be fixed now.

author
#12 lore raw
Quoted 2 lines

Minwoo, try pulling the most current nvme-cli. There was a sysfs scanning bug for non-mpath drives that should be fixed now.

Thank you, Keith! I've posted list result based on the latest one :)

author
#13 lore raw
In reply to Klaus Jensen
Quoted 36 lines · Klaus Jensen
Quoted 35 lines · Minwoo Im

Hello,

This patch series is third one to support multi-controller and namespace sharing in multi-path. This series introduced subsystem scheme to manage controller(s) and namespace(s) in the subsystem.

This series has new patches from the V2: 'detached' parameter has been added to the nvme-ns device. This will decide whether to attach the namespace to controller(s) in the current subsystem or not. If it's given with true, then it will be just allocated in the subsystem, but not attaching to any controllers in the subsystem. Otherwise, it will automatically attach to all the controllers in the subsystem. The other t hing is that the last patch implemented Identify Active Namespace ID List command handler apart from the Allocated Namespace ID List.

Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2

nvme-cli:

  root@vm:~/work# nvme list -v                                                                                                      
  NVM Express Subsystems                                                                                                 

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

  nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3
Quoted 7 lines
Quoted 5 lines

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0

Shouldn't nvme0n1 be listed under Namespaces for nvme0?

Oh, I missed that one from the output. As Keith mentioned, I ran the list command again based on the latest nvme-cli.git:

Please refer the following result. I think it's okay not to send the cover letter again :)

# nvme --version
nvme version 1.13.48.g33c6
# nvme list -v
NVM Express Subsystems

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------

nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0 nvme0n1
nvme1    foo                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:07.0   nvme-subsys1 
nvme2    bar                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:08.0   nvme-subsys1 
nvme3    baz                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:09.0   nvme-subsys1 nvme1c3n1

NVM Express Namespaces

Device NSID Usage Format Controllers
------------ -------- -------------------------- ---------------- ----------------

nvme0n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme0
nvme1n1      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme3
#14 lore raw
Quoted 71 lines · Minwoo Im
Quoted 36 lines · Klaus Jensen
Quoted 35 lines · Minwoo Im

Hello,

This patch series is third one to support multi-controller and namespace sharing in multi-path. This series introduced subsystem scheme to manage controller(s) and namespace(s) in the subsystem.

This series has new patches from the V2: 'detached' parameter has been added to the nvme-ns device. This will decide whether to attach the namespace to controller(s) in the current subsystem or not. If it's given with true, then it will be just allocated in the subsystem, but not attaching to any controllers in the subsystem. Otherwise, it will automatically attach to all the controllers in the subsystem. The other t hing is that the last patch implemented Identify Active Namespace ID List command handler apart from the Allocated Namespace ID List.

Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2

nvme-cli:

  root@vm:~/work# nvme list -v                                                                                                      
  NVM Express Subsystems                                                                                                 

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

  nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3
Quoted 7 lines
Quoted 5 lines

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0

Shouldn't nvme0n1 be listed under Namespaces for nvme0?

Oh, I missed that one from the output. As Keith mentioned, I ran the list command again based on the latest nvme-cli.git:

Please refer the following result. I think it's okay not to send the cover letter again :)

# nvme --version
nvme version 1.13.48.g33c6
# nvme list -v
NVM Express Subsystems

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------

nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0 nvme0n1
nvme1    foo                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:07.0   nvme-subsys1 
nvme2    bar                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:08.0   nvme-subsys1 
nvme3    baz                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:09.0   nvme-subsys1 nvme1c3n1
Quoted 6 lines

NVM Express Namespaces

Device NSID Usage Format Controllers
------------ -------- -------------------------- ---------------- ----------------

nvme0n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme0
nvme1n1      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme3

That looks better, but hmm. Shouldnt the namespace be named nvme1c3n1 here has well? Is that also an issue with nvme-cli?

author
#15 lore raw
Quoted 74 lines · Klaus Jensen
Quoted 73 lines · Minwoo Im
Quoted 36 lines · Klaus Jensen
Quoted 35 lines · Minwoo Im

Hello,

This patch series is third one to support multi-controller and namespace sharing in multi-path. This series introduced subsystem scheme to manage controller(s) and namespace(s) in the subsystem.

This series has new patches from the V2: 'detached' parameter has been added to the nvme-ns device. This will decide whether to attach the namespace to controller(s) in the current subsystem or not. If it's given with true, then it will be just allocated in the subsystem, but not attaching to any controllers in the subsystem. Otherwise, it will automatically attach to all the controllers in the subsystem. The other t hing is that the last patch implemented Identify Active Namespace ID List command handler apart from the Allocated Namespace ID List.

Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3

-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2

nvme-cli:

  root@vm:~/work# nvme list -v                                                                                                      
  NVM Express Subsystems                                                                                                 

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

  nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
  nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3
Quoted 7 lines
Quoted 5 lines

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0

Shouldn't nvme0n1 be listed under Namespaces for nvme0?

Oh, I missed that one from the output. As Keith mentioned, I ran the list command again based on the latest nvme-cli.git:

Please refer the following result. I think it's okay not to send the cover letter again :)

# nvme --version
nvme version 1.13.48.g33c6
# nvme list -v
NVM Express Subsystems

Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------

nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3

NVM Express Controllers

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------

nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0 nvme0n1
nvme1    foo                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:07.0   nvme-subsys1 
nvme2    bar                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:08.0   nvme-subsys1 
nvme3    baz                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:09.0   nvme-subsys1 nvme1c3n1

NVM Express Namespaces

Quoted 7 lines
Quoted 4 lines

Device NSID Usage Format Controllers
------------ -------- -------------------------- ---------------- ----------------

nvme0n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme0
nvme1n1      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme3

That looks better, but hmm. Shouldnt the namespace be named nvme1c3n1 here has well? Is that also an issue with nvme-cli?

No it isn't in this series. The 'nvme3' controller has not been provided with 'subsys' parameter which means it does not support CMIC multi-controller as nvme_init_ctrl() does not set the flag for NVME_CMIC_MULTI_CTRL:

-device nvme,serial=qux,id=nvme3

If this has been given with 'subsys' which means it supports for multi-controller, then it will be given like:

Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 nvme0c0n1

In short, if 'nvme' device controller is not given with 'subsys', then it will not support multi-controller and it will never have GENHD_FL_HIDDEN (nvmeXcYnZ convention).

author
#16 lore raw

RFC PATCH v3 8/8 hw/block/nvme: Add Identify Active Namespace ID List

In reply to Minwoo Im

Spec v1.4b 6.1.4 "Active and Inactive NSID Types" says:

"Active NSIDs for a controller refer to namespaces that are attached to that controller. Allocated NSIDs that are inactive for a controller refer to namespaces that are not attached to that controller."

This patch introduced for Identify Active Namespace ID List (CNS 02h).

---
 hw/block/nvme.c | 39 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)
hw/block/nvme.c +34 -5
index 2b2c07b36c2b..7247167b0ee6 100644
@@ -2883,6 +2883,39 @@ static uint16_t nvme_identify_ns_csi(NvmeCtrl *n, NvmeRequest *req)
     return NVME_INVALID_FIELD | NVME_DNR;
 }
 
+static uint16_t nvme_identify_nslist_active(NvmeCtrl *n, NvmeRequest *req)
+{
+    NvmeNamespace *ns;
+    NvmeIdentify *c = (NvmeIdentify *)&req->cmd;
+    uint32_t min_nsid = le32_to_cpu(c->nsid);
+    uint8_t list[NVME_IDENTIFY_DATA_SIZE] = {};
+    static const int data_len = sizeof(list);
+    uint32_t *list_ptr = (uint32_t *)list;
+    int i, j = 0;
+
+    if (min_nsid >= NVME_NSID_BROADCAST - 1) {
+        return NVME_INVALID_NSID | NVME_DNR;
+    }
+
+    for (i = 1; i <= n->num_namespaces; i++) {
+        ns = nvme_ns(n, i);
+        if (!ns || ns->params.nsid <= min_nsid) {
+            continue;
+        }
+
+        if (!nvme_ns_is_attached(n, ns)) {
+            continue;
+        }
+
+        list_ptr[j++] = cpu_to_le32(ns->params.nsid);
+        if (j == data_len / sizeof(uint32_t)) {
+            break;
+        }
+    }
+
+    return nvme_dma(n, list, data_len, DMA_DIRECTION_FROM_DEVICE, req);
+}
+
 static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
 {
     NvmeNamespace *ns;
@@ -2914,10 +2947,6 @@ static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
             continue;
         }
 
-        if (!nvme_ns_is_attached(n, ns)) {
-            continue;
-        }
-
         list_ptr[j++] = cpu_to_le32(ns->params.nsid);
         if (j == data_len / sizeof(uint32_t)) {
             break;
@@ -3045,7 +3074,7 @@ static uint16_t nvme_identify(NvmeCtrl *n, NvmeRequest *req)
     case NVME_ID_CNS_CS_CTRL:
         return nvme_identify_ctrl_csi(n, req);
     case NVME_ID_CNS_NS_ACTIVE_LIST:
-         /* fall through */
+         return nvme_identify_nslist_active(n, req);
     case NVME_ID_CNS_NS_PRESENT_LIST:
         return nvme_identify_nslist(n, req);
     case NVME_ID_CNS_CS_NS_ACTIVE_LIST:
Signature · 1 line
2.17.1
#17 lore raw

Re: [RFC PATCH V3 8/8] hw/block/nvme: Add Identify Active Namespace ID List

Quoted 81 lines · Minwoo Im

Spec v1.4b 6.1.4 "Active and Inactive NSID Types" says:

"Active NSIDs for a controller refer to namespaces that are attached to that controller. Allocated NSIDs that are inactive for a controller refer to namespaces that are not attached to that controller."

This patch introduced for Identify Active Namespace ID List (CNS 02h).

---
 hw/block/nvme.c | 39 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)
hw/block/nvme.c +34 -5
index 2b2c07b36c2b..7247167b0ee6 100644
@@ -2883,6 +2883,39 @@ static uint16_t nvme_identify_ns_csi(NvmeCtrl *n, NvmeRequest *req)
     return NVME_INVALID_FIELD | NVME_DNR;
 }
 
+static uint16_t nvme_identify_nslist_active(NvmeCtrl *n, NvmeRequest *req)
+{
+    NvmeNamespace *ns;
+    NvmeIdentify *c = (NvmeIdentify *)&req->cmd;
+    uint32_t min_nsid = le32_to_cpu(c->nsid);
+    uint8_t list[NVME_IDENTIFY_DATA_SIZE] = {};
+    static const int data_len = sizeof(list);
+    uint32_t *list_ptr = (uint32_t *)list;
+    int i, j = 0;
+
+    if (min_nsid >= NVME_NSID_BROADCAST - 1) {
+        return NVME_INVALID_NSID | NVME_DNR;
+    }
+
+    for (i = 1; i <= n->num_namespaces; i++) {
+        ns = nvme_ns(n, i);
+        if (!ns || ns->params.nsid <= min_nsid) {
+            continue;
+        }
+
+        if (!nvme_ns_is_attached(n, ns)) {
+            continue;
+        }
+
+        list_ptr[j++] = cpu_to_le32(ns->params.nsid);
+        if (j == data_len / sizeof(uint32_t)) {
+            break;
+        }
+    }
+
+    return nvme_dma(n, list, data_len, DMA_DIRECTION_FROM_DEVICE, req);
+}
+
 static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
 {
     NvmeNamespace *ns;
@@ -2914,10 +2947,6 @@ static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
             continue;
         }
 
-        if (!nvme_ns_is_attached(n, ns)) {
-            continue;
-        }
-
         list_ptr[j++] = cpu_to_le32(ns->params.nsid);
         if (j == data_len / sizeof(uint32_t)) {
             break;
@@ -3045,7 +3074,7 @@ static uint16_t nvme_identify(NvmeCtrl *n, NvmeRequest *req)
     case NVME_ID_CNS_CS_CTRL:
         return nvme_identify_ctrl_csi(n, req);
     case NVME_ID_CNS_NS_ACTIVE_LIST:
-         /* fall through */
+         return nvme_identify_nslist_active(n, req);
     case NVME_ID_CNS_NS_PRESENT_LIST:
         return nvme_identify_nslist(n, req);
     case NVME_ID_CNS_CS_NS_ACTIVE_LIST:
Signature · 1 line
2.17.1

Hello Minwoo,

By introducing a detached parameter,
you are also implicitly making the following
NVMe commands no longer be spec compliant:

NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST

When these commands are called on a detached namespace, they should usually return a zero filled data struct.

Dmitry and I had a patch on V8 on the ZNS series that tried to fix some the existing NVMe commands to be spec compliant, by handling detached namespaces properly. In the end, in order to make it easier to get the ZNS series accepted, we decided to drop the detached related stuff from the series.

Feel free to look at that patch for inspiration:
https://github.com/dmitry-fomichev/qemu/commit/251c0ffee5149c739b1347811fa7e32a1c36bf7c

I'm not sure if you want to modify all the functions that our patch modifies, but I think that you should at least modify the following nvme functions:

nvme_identify_ns()
nvme_identify_ns_csi()
nvme_identify_nslist()
nvme_identify_nslist_csi()

So they handle detached namespaces correctly for both:
NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST as well as for:
NVME_ID_CNS_NS_PRESENT, NVME_ID_CNS_CS_NS_PRESENT, NVME_ID_CNS_NS_PRESENT_LIST, NVME_ID_CNS_CS_NS_PRESENT_LIST

Kind regards,
Niklas

#18 lore raw

Re: [RFC PATCH V3 8/8] hw/block/nvme: Add Identify Active Namespace ID List

Quoted 114 lines · Niklas Cassel
Quoted 81 lines · Minwoo Im

Spec v1.4b 6.1.4 "Active and Inactive NSID Types" says:

"Active NSIDs for a controller refer to namespaces that are attached to that controller. Allocated NSIDs that are inactive for a controller refer to namespaces that are not attached to that controller."

This patch introduced for Identify Active Namespace ID List (CNS 02h).

---
 hw/block/nvme.c | 39 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)
hw/block/nvme.c +34 -5
index 2b2c07b36c2b..7247167b0ee6 100644
@@ -2883,6 +2883,39 @@ static uint16_t nvme_identify_ns_csi(NvmeCtrl *n, NvmeRequest *req)
     return NVME_INVALID_FIELD | NVME_DNR;
 }
 
+static uint16_t nvme_identify_nslist_active(NvmeCtrl *n, NvmeRequest *req)
+{
+    NvmeNamespace *ns;
+    NvmeIdentify *c = (NvmeIdentify *)&req->cmd;
+    uint32_t min_nsid = le32_to_cpu(c->nsid);
+    uint8_t list[NVME_IDENTIFY_DATA_SIZE] = {};
+    static const int data_len = sizeof(list);
+    uint32_t *list_ptr = (uint32_t *)list;
+    int i, j = 0;
+
+    if (min_nsid >= NVME_NSID_BROADCAST - 1) {
+        return NVME_INVALID_NSID | NVME_DNR;
+    }
+
+    for (i = 1; i <= n->num_namespaces; i++) {
+        ns = nvme_ns(n, i);
+        if (!ns || ns->params.nsid <= min_nsid) {
+            continue;
+        }
+
+        if (!nvme_ns_is_attached(n, ns)) {
+            continue;
+        }
+
+        list_ptr[j++] = cpu_to_le32(ns->params.nsid);
+        if (j == data_len / sizeof(uint32_t)) {
+            break;
+        }
+    }
+
+    return nvme_dma(n, list, data_len, DMA_DIRECTION_FROM_DEVICE, req);
+}
+
 static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
 {
     NvmeNamespace *ns;
@@ -2914,10 +2947,6 @@ static uint16_t nvme_identify_nslist(NvmeCtrl *n, NvmeRequest *req)
             continue;
         }
 
-        if (!nvme_ns_is_attached(n, ns)) {
-            continue;
-        }
-
         list_ptr[j++] = cpu_to_le32(ns->params.nsid);
         if (j == data_len / sizeof(uint32_t)) {
             break;
@@ -3045,7 +3074,7 @@ static uint16_t nvme_identify(NvmeCtrl *n, NvmeRequest *req)
     case NVME_ID_CNS_CS_CTRL:
         return nvme_identify_ctrl_csi(n, req);
     case NVME_ID_CNS_NS_ACTIVE_LIST:
-         /* fall through */
+         return nvme_identify_nslist_active(n, req);
     case NVME_ID_CNS_NS_PRESENT_LIST:
         return nvme_identify_nslist(n, req);
     case NVME_ID_CNS_CS_NS_ACTIVE_LIST:
Signature · 1 line
2.17.1

Hello Minwoo,

By introducing a detached parameter,
you are also implicitly making the following
NVMe commands no longer be spec compliant:

NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST

When these commands are called on a detached namespace, they should usually return a zero filled data struct.

Dmitry and I had a patch on V8 on the ZNS series that tried to fix some the existing NVMe commands to be spec compliant, by handling detached namespaces properly. In the end, in order to make it easier to get the ZNS series accepted, we decided to drop the detached related stuff from the series.

Feel free to look at that patch for inspiration:
https://github.com/dmitry-fomichev/qemu/commit/251c0ffee5149c739b1347811fa7e32a1c36bf7c

I'm not sure if you want to modify all the functions that our patch modifies, but I think that you should at least modify the following nvme functions:

nvme_identify_ns()
nvme_identify_ns_csi()
nvme_identify_nslist()
nvme_identify_nslist_csi()

Quoted 7 lines

So they handle detached namespaces correctly for both:
NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST as well as for:
NVME_ID_CNS_NS_PRESENT, NVME_ID_CNS_CS_NS_PRESENT, NVME_ID_CNS_NS_PRESENT_LIST, NVME_ID_CNS_CS_NS_PRESENT_LIST

Definitely - it makes sense to reintroduce your patch here, with a replacement of ns->attached with nvme_ns_is_attach(). Looks like that should be sufficient.

author
#19 lore raw

Re: [RFC PATCH V3 8/8] hw/block/nvme: Add Identify Active Namespace ID List

In reply to Niklas Cassel
Quoted 11 lines

Hello Minwoo,

By introducing a detached parameter,
you are also implicitly making the following
NVMe commands no longer be spec compliant:

NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST

When these commands are called on a detached namespace, they should usually return a zero filled data struct.

Agreed.

Quoted 9 lines

Dmitry and I had a patch on V8 on the ZNS series that tried to fix some the existing NVMe commands to be spec compliant, by handling detached namespaces properly. In the end, in order to make it easier to get the ZNS series accepted, we decided to drop the detached related stuff from the series.

Feel free to look at that patch for inspiration:
https://github.com/dmitry-fomichev/qemu/commit/251c0ffee5149c739b1347811fa7e32a1c36bf7c

I've seen this patch and as Klaus said, only thing patches need go with is to put some of nvme_ns_is_attached() helper among the Identify handlers.

Quoted 8 lines

I'm not sure if you want to modify all the functions that our patch modifies, but I think that you should at least modify the following nvme functions:

nvme_identify_ns()
nvme_identify_ns_csi()
nvme_identify_nslist()
nvme_identify_nslist_csi()

Yes, pretty makes sense to update them. But now it seems like 'attach/detach' scheme should have been separated out of this series which just introduced the multi-path for controllers and namespace sharing. I will drop this 'detach' scheme out of this series and make another series to support all of the Identify you mentioned above cleanly.

Quoted 10 lines

So they handle detached namespaces correctly for both:
NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST as well as for:
NVME_ID_CNS_NS_PRESENT, NVME_ID_CNS_CS_NS_PRESENT, NVME_ID_CNS_NS_PRESENT_LIST, NVME_ID_CNS_CS_NS_PRESENT_LIST

Kind regards,
Niklas

#20 lore raw

Re: [RFC PATCH V3 8/8] hw/block/nvme: Add Identify Active Namespace ID List

Quoted 37 lines · Minwoo Im
Quoted 11 lines

Hello Minwoo,

By introducing a detached parameter,
you are also implicitly making the following
NVMe commands no longer be spec compliant:

NVME_ID_CNS_NS, NVME_ID_CNS_CS_NS, NVME_ID_CNS_NS_ACTIVE_LIST, NVME_ID_CNS_CS_NS_ACTIVE_LIST

When these commands are called on a detached namespace, they should usually return a zero filled data struct.

Agreed.

Quoted 9 lines

Dmitry and I had a patch on V8 on the ZNS series that tried to fix some the existing NVMe commands to be spec compliant, by handling detached namespaces properly. In the end, in order to make it easier to get the ZNS series accepted, we decided to drop the detached related stuff from the series.

Feel free to look at that patch for inspiration:
https://github.com/dmitry-fomichev/qemu/commit/251c0ffee5149c739b1347811fa7e32a1c36bf7c

I've seen this patch and as Klaus said, only thing patches need go with is to put some of nvme_ns_is_attached() helper among the Identify handlers.

Quoted 8 lines

I'm not sure if you want to modify all the functions that our patch modifies, but I think that you should at least modify the following nvme functions:

nvme_identify_ns()
nvme_identify_ns_csi()
nvme_identify_nslist()
nvme_identify_nslist_csi()

Quoted 6 lines

Yes, pretty makes sense to update them. But now it seems like 'attach/detach' scheme should have been separated out of this series which just introduced the multi-path for controllers and namespace sharing. I will drop this 'detach' scheme out of this series and make another series to support all of the Identify you mentioned above cleanly.

Hello Minwoo,

thank you for putting in work on this!

Kind regards,
Niklas