incorrect sysfs reported by realpath for nvme disk when nvme-core:multipath enabled
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hello
When I use 'realpath' to get the nvme sysfs on latest 5.3.0-rc8 for NVMe SSD, seems it reported incorrectly sysfs, could anyone help check it?
[1] nvme-core multipath=Y
# lspci | grep -i non 86:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller 172Xa/172Xb (rev 01) af:00.0 Non-Volatile memory controller: Intel Corporation NVMe DC SSD [3DNAND, Beta Rock Controller]
# nvme list Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 BTLE8190003X2P0IGN Dell Express Flash NVMe P4600 2.0TB AIC 1 2.00 TB / 2.00 TB 4 KiB + 0 B QDV1DP15 /dev/nvme1n1 S39WNA0K201139 Dell Express Flash PM1725a 1.6TB AIC 1 1.60 TB / 1.60 TB 512 B + 0 B 1.1.1
# cat /sys/module/nvme_core/parameters/multipath Y
# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 222.5G 0 disk ├─sda1 8:1 0 600M 0 part /boot/efi ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 220.9G 0 part ├─rhel_rdma--perf--07-root 253:0 0 50G 0 lvm / ├─rhel_rdma--perf--07-swap 253:1 0 4G 0 lvm [SWAP] └─rhel_rdma--perf--07-home 253:2 0 166.9G 0 lvm /home nvme0n1 259:0 0 1.8T 0 disk nvme1n1 259:2 0 1.5T 0 disk
# realpath /sys/dev/block/259:0 /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme1/nvme0n1 --------> should it be /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme0/nvme0n1 ?
# realpath /sys/dev/block/259:2 /sys/devices/virtual/nvme-subsystem/nvme-subsys1/nvme1n1 --------> No pci address returned, bug?
[2] nvme-core multipath=N
# rmmod nvme # rmmod nvme-core # nvme list # modprobe nvme-core multipath=N # cat /sys/module/nvme_core/parameters/multipath N [root@rdma-perf-07 ~]$ modprobe nvme NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 222.5G 0 disk ├─sda1 8:1 0 600M 0 part /boot/efi ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 220.9G 0 part ├─rhel_rdma--perf--07-root 253:0 0 50G 0 lvm / ├─rhel_rdma--perf--07-swap 253:1 0 4G 0 lvm [SWAP] └─rhel_rdma--perf--07-home 253:2 0 166.9G 0 lvm /home nvme1n1 259:0 0 1.8T 0 disk nvme0n1 259:2 0 1.5T 0 disk
[root@rdma-perf-07 ~]$ realpath /sys/dev/block/259:0 /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme1/nvme1n1
[root@rdma-perf-07 ~]$ realpath /sys/dev/block/259:2 /sys/devices/pci0000:85/0000:85:02.0/0000:86:00.0/nvme/nvme0/nvme0n1
Best Regards,
Yi Zhang
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 2 lines · Yi Zhang
# realpath /sys/dev/block/259:0 /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme1/nvme0n1 --------> should it be /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme0/nvme0n1 ?
The sysfs links are correct. The controller-namespacenaming is just not related, which is what you're observing.
This was confusing to enough people that we've since changed the naming to something people expect to see, but that's staged in 5.4. Given the number of people who find this confusing, this might be worth making a stable patch.
Quoted 2 lines
# realpath /sys/dev/block/259:2 /sys/devices/virtual/nvme-subsystem/nvme-subsys1/nvme1n1 --------> No pci address returned, bug?
For mulitpath capable controllers, the parent of a namespace is the subsystem rather than a controller. The subsystem is a virtual device that doesn't have a physical pci address.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 11 lines
Quoted 3 lines
# realpath /sys/dev/block/259:0 /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme1/nvme0n1 --------> should it be /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme0/nvme0n1 ?The sysfs links are correct. The controller-namespacenaming is just not related, which is what you're observing.
This was confusing to enough people that we've since changed the naming to something people expect to see, but that's staged in 5.4. Given the number of people who find this confusing, this might be worth making a stable patch.
I agree here.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 6 lines · Sagi Grimberg
Quoted 4 lines
This was confusing to enough people that we've since changed the naming to something people expect to see, but that's staged in 5.4. Given the number of people who find this confusing, this might be worth making a stable patch.
I agree here.
Okay, I'll forward the commit to stable once merged to mainline. I don't think we can count on the stable bot to pick this one up. We only need this back to 4.19 as all the previous stables with this naming scheme have be EOL'ed.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 6 lines · Keith Busch
Quoted 5 lines · Sagi Grimberg
Quoted 4 lines
This was confusing to enough people that we've since changed the naming to something people expect to see, but that's staged in 5.4. Given the number of people who find this confusing, this might be worth making a stable patch.
I agree here.
Hi Keith
Thanks for the info, I just test the nvme-5.4, and it works as expect.
[root@rdma-perf-07 ~]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 222.5G 0 disk
├─sda1 8:1 0 600M 0 part /boot/efi ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 220.9G 0 part ├─rhel_rdma--perf--07-root 253:0 0 50G 0 lvm / ├─rhel_rdma--perf--07-swap 253:1 0 4G 0 lvm [SWAP] └─rhel_rdma--perf--07-home 253:2 0 166.9G 0 lvm /home
nvme1n1 259:0 0 1.8T 0 disk └─nvme1n1p1 259:1 0 1.8T 0 part nvme0n1 259:3 0 1.5T 0 disk [root@rdma-perf-07 ~]$ realpath /sys/dev/block/259:0 /sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/nvme/nvme1/nvme1n1 [root@rdma-perf-07 ~]$ realpath /sys/dev/block/259:3 /sys/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1
Thanks
Yi
Quoted 9 lines
Okay, I'll forward the commit to stable once merged to mainline. I don't think we can count on the stable bot to pick this one up. We only need this back to 4.19 as all the previous stables with this naming scheme have be EOL'ed.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme