mismatch between chardev and blkdev node names
Hi, NVMe people.
I'd like to get some advices for the reported issue from nvme-cli:
https://github.com/linux-nvme/nvme-cli/issues/510
As all you guys know, the controller chardev is created with controller instance id which is from ctrl->instance. (e.g. /dev/nvme0) For the namespace blkdev node is set to either instance id or subsystem id when ns->head->disk is not there.
This policy can make the blkdev and chardev are pretty different view to the users like who faced this issue.
For the following case, the controller and the namespaces are completly looking different:
root at target:~# nvme list-subsys
nvme-subsys0 - NQN=nqn.2014.08.org.nvmexpress:80861af4bar QEMU NVMe Ctrl
\
+- nvme1 pcie 0000:00:05.0 live nvme-subsys1 - NQN=nqn.2014.08.org.nvmexpress:80861af4foo QEMU NVMe Ctrl \ +- nvme0 pcie 0000:00:04.0 live nvme-subsys2 - NQN=nqn.2014.08.org.nvmexpress:80861af4zoo QEMU NVMe Ctrl \ +- nvme2 pcie 0000:00:06.0 live
root at target:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1 bar QEMU NVMe Ctrl 1 536.87 MB / 536.87 MB 512 B + 0 B 1.0 /dev/nvme1n1 foo QEMU NVMe Ctrl 1 536.87 MB / 536.87 MB 512 B + 0 B 1.0 /dev/nvme2n1 zoo QEMU NVMe Ctrl 1 536.87 MB / 536.87 MB 512 B + 0 B 1.0
This example shows that if we want to get the same result for the id-ctrl, then we should:
# nvme id-ctrl /dev/nvme1 # nvme id-ctrl /dev/nvme0n1
Should users with multipath enabled know this situation well and do something with perfect understanding this policy because they might be administrator for the system? or should the driver provide the same index at least right next to "nvme" for the controller and namespaces?
I hope you guys can give some advices here :)
Thanks,
(+ Added maintainers to CCs)
Quoted 4 lines · Minwoo Im
Should users with multipath enabled know this situation well and do something with perfect understanding this policy because they might be administrator for the system? or should the driver provide the same index at least right next to "nvme" for the controller and namespaces?
You can't just use the controller instance for the namespace names when multipathing: you can't consistently name the namespace that is accessible through two controllers that way. You have to use an identifier that is common to all controllers with a shared namespace access, which is the subsystem instance.
You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.
Quoted 11 lines · Keith Busch
Quoted 4 lines · Minwoo Im
Should users with multipath enabled know this situation well and do something with perfect understanding this policy because they might be administrator for the system? or should the driver provide the same index at least right next to "nvme" for the controller and namespaces?
You can't just use the controller instance for the namespace names when multipathing: you can't consistently name the namespace that is accessible through two controllers that way. You have to use an identifier that is common to all controllers with a shared namespace access, which is the subsystem instance.
Yeah, I can see things in multipath code just like what you mentioned.
Quoted 4 lines
You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.
Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.
Thanks, Keith.
Sagi Grimberg
maintainerQuoted 7 lines
Quoted 4 lines
You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.
Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.
Or run:
nvme list-subsys (with or without passing in a namespace device node).
For example: nvme list-subsys /dev/nvme0n1 will show you exactly which controllers it spans and which subsystem it belongs to.
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.
Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.
Quoted 6 lines
Or run:
nvme list-subsys (with or without passing in a namespace device node).
For example: nvme list-subsys
/dev/nvme0n1will show you exactly which controllers it spans and which subsystem it belongs to.
Yeah, Sagi.
I just have posted the comment to the issue in Github just like what you've mentioned. we can figure out the relationship for the subsystems via "nvme list-subsys".
Thanks,
Hello,
if we look at the UDEV DEVPATH propertys of a particular nvme block device, for example:
DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ
Would it be correct to say that "nvmeX" is the controller instance for this particular block device (or one of the instances)?
Thanks,
Alex.
-----Original Message-----
From: Minwoo Im
Sent: Tuesday, June 11, 2019 8:14 AM
To: Sagi Grimberg ; Minwoo Im ; Keith Busch ; Minwoo Im
Cc: linux-nvme
Subject: Re: [RFC] mismatch between chardev and blkdev node names
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.
Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.
Quoted 6 lines
Or run:
nvme list-subsys (with or without passing in a namespace device node).
For example: nvme list-subsys
/dev/nvme0n1will show you exactly which controllers it spans and which subsystem it belongs to.
Yeah, Sagi.
I just have posted the comment to the issue in Github just like what you've mentioned. we can figure out the relationship for the subsystems via "nvme list-subsys".
Thanks,
_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 9 lines · Alex Lyakas
Hello,
if we look at the UDEV DEVPATH propertys of a particular nvme block device, for example:
DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ
Would it be correct to say that "nvmeX" is the controller instance for this particular block device (or one of the instances)?
Yes, for the above path, The corresponding blkdev of the nvmeX subsystem is nvmeYnZ. We just need to be aware if the MULTIPATH is enabled or not. If so, the X will be a subsystem instance which can be checked by nvme list-subsys. If not, the X is the controller instance name.
Thanks,
Quoted 6 lines · Minwoo Im
Quoted 5 lines · Alex Lyakas
Hello,
if we look at the UDEV DEVPATH propertys of a particular nvme block device, for example:
Quoted 7 lines
Quoted 4 lines
DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ
Would it be correct to say that "nvmeX" is the controller instance for this particular block device (or one of the instances)?
Yes, for the above path, The corresponding blkdev of the nvmeX subsystem is nvmeYnZ. We just need to be aware if the MULTIPATH is enabled or
I mean if subsystem has multiple namespaces, then nvmeYnZ is one of them.
Quoted 4 lines
not. If so, the X will be a subsystem instance which can be checked by nvme list-subsys. If not, the X is the controller instance name.
Thanks,
Minwoo Im,
Thanks for the explanation.
Alex.
-----Original Message-----
From: Minwoo Im
Sent: Sunday, June 23, 2019 2:57 PM
To: Alex Lyakas
Cc: Sagi Grimberg ; Keith Busch ; Minwoo Im ; linux-nvme ; Minwoo Im
Subject: Re: [RFC] mismatch between chardev and blkdev node names
Quoted 7 lines · Minwoo Im
Quoted 6 lines · Alex Lyakas
Hello,
if we look at the UDEV DEVPATH propertys of a particular nvme block device,
for example:
Quoted 8 lines
Quoted 5 lines
DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ
Would it be correct to say that "nvmeX" is the controller instance for this
particular block device (or one of the instances)?Yes, for the above path, The corresponding blkdev of the nvmeX subsystem is nvmeYnZ. We just need to be aware if the MULTIPATH is enabled or
I mean if subsystem has multiple namespaces, then nvmeYnZ is one of them.
Quoted 4 lines
not. If so, the X will be a subsystem instance which can be checked by nvme list-subsys. If not, the X is the controller instance name.
Thanks,