Skip to contentSkip to conversation

mismatch between chardev and blkdev node names

RFC Minwoo Im 5 participants 10 messages Jun 10, 2019 – Jun 24, 2019 NVMe

lore
Share & tools
lore

Minwoo Im

author
#0 lore raw

Hi, NVMe people.

I'd like to get some advices for the reported issue from nvme-cli:
https://github.com/linux-nvme/nvme-cli/issues/510

As all you guys know, the controller chardev is created with controller instance id which is from ctrl->instance. (e.g. /dev/nvme0) For the namespace blkdev node is set to either instance id or subsystem id when ns->head->disk is not there.

This policy can make the blkdev and chardev are pretty different view to the users like who faced this issue.

For the following case, the controller and the namespaces are completly looking different:

root at target:~# nvme list-subsys
nvme-subsys0 - NQN=nqn.2014.08.org.nvmexpress:80861af4bar QEMU NVMe Ctrl
\

 +- nvme1 pcie 0000:00:05.0 live                                                        
nvme-subsys1 - NQN=nqn.2014.08.org.nvmexpress:80861af4foo                 QEMU NVMe Ctrl
\                                                                                       
 +- nvme0 pcie 0000:00:04.0 live                                                        
nvme-subsys2 - NQN=nqn.2014.08.org.nvmexpress:80861af4zoo                 QEMU NVMe Ctrl
\                                                                                       
 +- nvme2 pcie 0000:00:06.0 live                                                        

root at target:~# nvme list
Node SN Model Namespace Usage Format FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------

/dev/nvme0n1     bar                  QEMU NVMe Ctrl                           1         536.87  MB / 536.87  MB    512   B +  0 B   1.0     
/dev/nvme1n1     foo                  QEMU NVMe Ctrl                           1         536.87  MB / 536.87  MB    512   B +  0 B   1.0     
/dev/nvme2n1     zoo                  QEMU NVMe Ctrl                           1         536.87  MB / 536.87  MB    512   B +  0 B   1.0     

This example shows that if we want to get the same result for the id-ctrl, then we should:

  # nvme id-ctrl /dev/nvme1
  # nvme id-ctrl /dev/nvme0n1

Should users with multipath enabled know this situation well and do something with perfect understanding this policy because they might be administrator for the system? or should the driver provide the same index at least right next to "nvme" for the controller and namespaces?

I hope you guys can give some advices here :)

Thanks,

#2 lore raw
In reply to Minwoo Im
Quoted 4 lines · Minwoo Im

Should users with multipath enabled know this situation well and do something with perfect understanding this policy because they might be administrator for the system? or should the driver provide the same index at least right next to "nvme" for the controller and namespaces?

You can't just use the controller instance for the namespace names when multipathing: you can't consistently name the namespace that is accessible through two controllers that way. You have to use an identifier that is common to all controllers with a shared namespace access, which is the subsystem instance.

You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.

author
#3 lore raw
Quoted 11 lines · Keith Busch
Quoted 4 lines · Minwoo Im

Should users with multipath enabled know this situation well and do something with perfect understanding this policy because they might be administrator for the system? or should the driver provide the same index at least right next to "nvme" for the controller and namespaces?

You can't just use the controller instance for the namespace names when multipathing: you can't consistently name the namespace that is accessible through two controllers that way. You have to use an identifier that is common to all controllers with a shared namespace access, which is the subsystem instance.

Yeah, I can see things in multipath code just like what you mentioned.

Quoted 4 lines

You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.

Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.

Thanks, Keith.

maintainer
#4 lore raw
Quoted 7 lines
Quoted 4 lines

You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.

Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.

Or run:

nvme list-subsys (with or without passing in a namespace device node).

For example: nvme list-subsys /dev/nvme0n1 will show you exactly which controllers it spans and which subsystem it belongs to.

#5 lore raw
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.

Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.

Quoted 6 lines

Or run:

nvme list-subsys (with or without passing in a namespace device node).

For example: nvme list-subsys /dev/nvme0n1 will show you exactly which controllers it spans and which subsystem it belongs to.

Yeah, Sagi.

I just have posted the comment to the issue in Github just like what you've mentioned. we can figure out the relationship for the subsystems via "nvme list-subsys".

Thanks,

#6 lore raw

Hello,

if we look at the UDEV DEVPATH propertys of a particular nvme block device, for example:

DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ

Would it be correct to say that "nvmeX" is the controller instance for this particular block device (or one of the instances)?

Thanks,
Alex.

-----Original Message-----
From: Minwoo Im
Sent: Tuesday, June 11, 2019 8:14 AM
To: Sagi Grimberg ; Minwoo Im ; Keith Busch ; Minwoo Im
Cc: linux-nvme
Subject: Re: [RFC] mismatch between chardev and blkdev node names

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

You just need to know this if you're accessing direct namespace handles, and the controller-namespace relationship is accurately represented in sysfs for those wanting to view it.

Then the user should exactly know the relationship bettwen the controller and namespaces via the sysfs structures.

Quoted 6 lines

Or run:

nvme list-subsys (with or without passing in a namespace device node).

For example: nvme list-subsys /dev/nvme0n1 will show you exactly which controllers it spans and which subsystem it belongs to.

Yeah, Sagi.

I just have posted the comment to the issue in Github just like what you've mentioned. we can figure out the relationship for the subsystems via "nvme list-subsys".

Thanks,

_______________________________________________
Linux-nvme mailing list
Linux-nvme at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

author
#7 lore raw
Quoted 9 lines · Alex Lyakas

Hello,

if we look at the UDEV DEVPATH propertys of a particular nvme block device, for example:

DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ

Would it be correct to say that "nvmeX" is the controller instance for this particular block device (or one of the instances)?

Yes, for the above path, The corresponding blkdev of the nvmeX subsystem is nvmeYnZ. We just need to be aware if the MULTIPATH is enabled or not. If so, the X will be a subsystem instance which can be checked by nvme list-subsys. If not, the X is the controller instance name.

Thanks,

author
#8 lore raw
Quoted 6 lines · Minwoo Im
Quoted 5 lines · Alex Lyakas

Hello,

if we look at the UDEV DEVPATH propertys of a particular nvme block device, for example:

Quoted 7 lines
Quoted 4 lines

DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ

Would it be correct to say that "nvmeX" is the controller instance for this particular block device (or one of the instances)?

Yes, for the above path, The corresponding blkdev of the nvmeX subsystem is nvmeYnZ. We just need to be aware if the MULTIPATH is enabled or

I mean if subsystem has multiple namespaces, then nvmeYnZ is one of them.

Quoted 4 lines

not. If so, the X will be a subsystem instance which can be checked by nvme list-subsys. If not, the X is the controller instance name.

Thanks,

#9 lore raw

Minwoo Im,

Thanks for the explanation.

Alex.

-----Original Message-----
From: Minwoo Im
Sent: Sunday, June 23, 2019 2:57 PM
To: Alex Lyakas
Cc: Sagi Grimberg ; Keith Busch ; Minwoo Im ; linux-nvme ; Minwoo Im
Subject: Re: [RFC] mismatch between chardev and blkdev node names

Quoted 7 lines · Minwoo Im
Quoted 6 lines · Alex Lyakas

Hello,

if we look at the UDEV DEVPATH propertys of a particular nvme block device,
for example:

Quoted 8 lines
Quoted 5 lines

DEVPATH=/devices/virtual/nvme-fabrics/ctl/nvmeX/nvmeYnZ

Would it be correct to say that "nvmeX" is the controller instance for this
particular block device (or one of the instances)?

Yes, for the above path, The corresponding blkdev of the nvmeX subsystem is nvmeYnZ. We just need to be aware if the MULTIPATH is enabled or

I mean if subsystem has multiple namespaces, then nvmeYnZ is one of them.

Quoted 4 lines

not. If so, the X will be a subsystem instance which can be checked by nvme list-subsys. If not, the X is the controller instance name.

Thanks,