Skip to contentSkip to conversation

dm: Call proper helper to determine dax support

PATCH In review Jan Kara 4 participants 13 messages Sep 16, 2020 – Sep 18, 2020

Subsystems (4)
lore
Share & tools
lore

Jan Kara

author reviewer
#0 lore raw

PATCH 1/1 dm: Call proper helper to determine dax support

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +4 -0
index e5767c83ea23..b6284c5cae0a 100644
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
 		int blocksize, sector_t start, sector_t len)
 {
+	if (!dax_dev)
+		return false;
+
 	if (!dax_alive(dax_dev))
 		return false;
 
 	return dax_dev->ops->dax_supported(dax_dev, bdev, blocksize, start, len);
 }
+EXPORT_SYMBOL_GPL(dax_supported);
 
 size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
 		size_t bytes, struct iov_iter *i)
drivers/md/dm-table.c +1 -2
index 5edc3079e7c1..bed1ff0744ec 100644
@@ -862,8 +862,7 @@ int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
 {
 	int blocksize = *(int *) data;
 
-	return generic_fsdax_supported(dev->dax_dev, dev->bdev, blocksize,
-				       start, len);
+	return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
 }
 
 /* Check devices support synchronous DAX */
include/linux/dax.h +9 -2
index 6904d4e0b2e0..9f916326814a 100644
@@ -130,6 +130,8 @@ static inline bool generic_fsdax_supported(struct dax_device *dax_dev,
 	return __generic_fsdax_supported(dax_dev, bdev, blocksize, start,
 			sectors);
 }
+bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
+		int blocksize, sector_t start, sector_t len);
 
 static inline void fs_put_dax(struct dax_device *dax_dev)
 {
@@ -157,6 +159,13 @@ static inline bool generic_fsdax_supported(struct dax_device *dax_dev,
 	return false;
 }
 
+static inline bool dax_supported(struct dax_device *dax_dev,
+		struct block_device *bdev, int blocksize, sector_t start,
+		sector_t len)
+{
+	return false;
+}
+
 static inline void fs_put_dax(struct dax_device *dax_dev)
 {
 }
@@ -195,8 +204,6 @@ bool dax_alive(struct dax_device *dax_dev);
 void *dax_get_private(struct dax_device *dax_dev);
 long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages,
 		void **kaddr, pfn_t *pfn);
-bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
-		int blocksize, sector_t start, sector_t len);
 size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
 		size_t bytes, struct iov_iter *i);
 size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
Signature · 4 lines
2.16.4
_______________________________________________
Linux-nvdimm mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Acked-by on v1 1/1
#1 lore raw
Quoted 7 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

Quoted 8 lines

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")

Looked good:

This fix should Cc stable@ right?

Quoted 8 lines
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

Sure, but there really isn't a dependency right?

Dan, will you be picking these up to send to Linux for 5.9-rc?

Thanks,
Mike

Quoted 74 lines
drivers/dax/super.c +4 -0
index e5767c83ea23..b6284c5cae0a 100644
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
 		int blocksize, sector_t start, sector_t len)
 {
+	if (!dax_dev)
+		return false;
+
 	if (!dax_alive(dax_dev))
 		return false;
 
 	return dax_dev->ops->dax_supported(dax_dev, bdev, blocksize, start, len);
 }
+EXPORT_SYMBOL_GPL(dax_supported);
 
 size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
 		size_t bytes, struct iov_iter *i)
drivers/md/dm-table.c +1 -2
index 5edc3079e7c1..bed1ff0744ec 100644
@@ -862,8 +862,7 @@ int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
 {
 	int blocksize = *(int *) data;
 
-	return generic_fsdax_supported(dev->dax_dev, dev->bdev, blocksize,
-				       start, len);
+	return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
 }
 
 /* Check devices support synchronous DAX */
include/linux/dax.h +9 -2
index 6904d4e0b2e0..9f916326814a 100644
@@ -130,6 +130,8 @@ static inline bool generic_fsdax_supported(struct dax_device *dax_dev,
 	return __generic_fsdax_supported(dax_dev, bdev, blocksize, start,
 			sectors);
 }
+bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
+		int blocksize, sector_t start, sector_t len);
 
 static inline void fs_put_dax(struct dax_device *dax_dev)
 {
@@ -157,6 +159,13 @@ static inline bool generic_fsdax_supported(struct dax_device *dax_dev,
 	return false;
 }
 
+static inline bool dax_supported(struct dax_device *dax_dev,
+		struct block_device *bdev, int blocksize, sector_t start,
+		sector_t len)
+{
+	return false;
+}
+
 static inline void fs_put_dax(struct dax_device *dax_dev)
 {
 }
@@ -195,8 +204,6 @@ bool dax_alive(struct dax_device *dax_dev);
 void *dax_get_private(struct dax_device *dax_dev);
 long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages,
 		void **kaddr, pfn_t *pfn);
-bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
-		int blocksize, sector_t start, sector_t len);
 size_t dax_copy_from_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
 		size_t bytes, struct iov_iter *i);
 size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
Signature · 1 line
2.16.4

_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

author reviewer
#2 lore raw
Quoted 15 lines · Mike Snitzer
Quoted 12 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Quoted 7 lines
Quoted 3 lines
Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")

Looked good:

Thanks!

Quoted 1 line

This fix should Cc stable@ right?

Yes, it should go to stable.

Quoted 9 lines
Quoted 7 lines
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

Sure, but there really isn't a dependency right?

Yes, it isn't a context or strict functional dependency. But without this patch Adrian's patch just trades one set of warnings for another set of warnings...

Honza

Signature · 5 lines
Jan Kara <[email protected]>
SUSE Labs, CR
_______________________________________________
Linux-nvdimm mailing list -- [email protected]
To unsubscribe send an email to [email protected]
#3 lore raw
In reply to Jan Kara
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +7 -2
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;
+       int blocksize = *(int *) data, id;
+       bool rc;
 
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

author reviewer
#4 lore raw
Quoted 56 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +0 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 10 lines
drivers/md/dm-table.c +7 -1
+       int blocksize = *(int *) data, id;
+       bool rc;
 
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Honza

Signature · 5 lines
Jan Kara <[email protected]>
SUSE Labs, CR
_______________________________________________
Linux-nvdimm mailing list -- [email protected]
To unsubscribe send an email to [email protected]
#5 lore raw
Quoted 61 lines · Jan Kara
Quoted 59 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +2 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;
+       int blocksize = *(int *) data, id;
+       bool rc;
 
Quoted 12 lines

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 7 lines
drivers/md/dm-table.c +5 -1
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Hold on. This patch hit another regression when I ran the full test of the lvm2-testsuite tool today.

After checking the test log, the task was blocked for more than 120 seconds when running the command 'pvmove --abort' of the script 'pvmove-abort-all.sh'.

Note: Still no lock after applying Dan's fixup. It shows the same call trace with/without Dan's fixup.

[  375.623646] INFO: task lvm:12573 blocked for more than 122 seconds.
[  375.630685]       Not tainted 5.9.0-rc5+ #25
[  375.635467] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[  375.644223] task:lvm             state:D stack:    0 pid:12573
ppid:     1 flags:0x00000080
[  375.653561] Call Trace:
[  375.656342]  __schedule+0x278/0x740
[  375.660261]  ? ttwu_do_wakeup+0x19/0x150
[  375.664653]  schedule+0x40/0xb0
[  375.668173]  schedule_timeout+0x25f/0x300
[  375.672665]  ? __queue_work+0x13a/0x3e0
[  375.676950]  wait_for_completion+0x8d/0xf0
[  375.681538]  __synchronize_srcu.part.21+0x81/0xb0
[  375.686804]  ? __bpf_trace_rcu_utilization+0x10/0x10
[  375.692356]  ? synchronize_srcu+0x59/0xf0
[  375.696877]  __dm_suspend+0x56/0x1d0 [dm_mod]
[  375.701759]  ? table_load+0x2e0/0x2e0 [dm_mod]
[  375.706735]  dm_suspend+0xa5/0xd0 [dm_mod]
[  375.711324]  dev_suspend+0x14d/0x290 [dm_mod]
[  375.716202]  ctl_ioctl+0x1af/0x420 [dm_mod]
[  375.720892]  ? iomap_write_begin+0x4c0/0x4c0
[  375.725675]  dm_ctl_ioctl+0xa/0x10 [dm_mod]
[  375.730352]  __x64_sys_ioctl+0x84/0xb1
[  375.734551]  do_syscall_64+0x33/0x40
[  375.738557]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  375.744212] RIP: 0033:0x7f0106eee87b
[  375.748206] Code: Bad RIP value.
[  375.751822] RSP: 002b:00007ffe10ff4498 EFLAGS: 00000206 ORIG_RAX:
0000000000000010
[  375.760286] RAX: ffffffffffffffda RBX: 0000564c3aebb260 RCX: 00007f0106eee87b
[  375.768262] RDX: 0000564c3c4220c0 RSI: 00000000c138fd06 RDI: 0000000000000004
[  375.776237] RBP: 0000564c3af694fe R08: 0000564c3c3461f0 R09: 00007f0108cfd980
[  375.784206] R10: 000000000000000a R11: 0000000000000206 R12: 0000000000000000
[  375.792181] R13: 0000564c3c4220f0 R14: 0000564c3c4220c0 R15: 0000564c3c4148c0

-- Adrian
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

#6 lore raw
Quoted 3 lines · Huang Adrian

Note: Still no lock after applying Dan's fixup. It shows the same call trace with/without Dan's fixup.

Sorry, fat-finger. Should be 'Still no *luck*'

-- Adrian
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

#7 lore raw
In reply to Huang Adrian
Quoted 76 lines · Huang Adrian
Quoted 61 lines · Jan Kara
Quoted 59 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +2 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;
+       int blocksize = *(int *) data, id;
+       bool rc;
 
Quoted 12 lines

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 7 lines
drivers/md/dm-table.c +5 -1
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Quoted 2 lines

Hold on. This patch hit another regression when I ran the full test of the lvm2-testsuite tool today.

Are you sure it's this patch?

The dax_read_lock() should have zero interaction with the synchronize_srcu() that __dm_suspend() performs. The too srcu domains should not conflict... I don't even see a dax_read_lock() in this path.
_______________________________________________
Linux-nvdimm mailing list -- [email protected]
To unsubscribe send an email to [email protected]

#8 lore raw
Quoted 81 lines · Dan Williams
Quoted 76 lines · Huang Adrian
Quoted 61 lines · Jan Kara
Quoted 59 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +2 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;
+       int blocksize = *(int *) data, id;
+       bool rc;
 
Quoted 12 lines

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 7 lines
drivers/md/dm-table.c +5 -1
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Quoted 2 lines

Hold on. This patch hit another regression when I ran the full test of the lvm2-testsuite tool today.

Quoted 6 lines

Are you sure it's this patch?

The dax_read_lock() should have zero interaction with the synchronize_srcu() that __dm_suspend() performs. The too srcu domains should not conflict... I don't even see a dax_read_lock() in this path.

Confirmed. I hit the hang in the lvm2-testsuite on vanilla 5.9-rc5. So, I'm going to proceed with Jan's patch plus my fixup.
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

#9 lore raw
Quoted 85 lines · Dan Williams
Quoted 83 lines · Dan Williams
Quoted 76 lines · Huang Adrian
Quoted 61 lines · Jan Kara
Quoted 59 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +2 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;
+       int blocksize = *(int *) data, id;
+       bool rc;
 
Quoted 12 lines

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 7 lines
drivers/md/dm-table.c +5 -1
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Quoted 2 lines

Hold on. This patch hit another regression when I ran the full test of the lvm2-testsuite tool today.

Are you sure it's this patch?

Quoted 7 lines
Quoted 4 lines

The dax_read_lock() should have zero interaction with the synchronize_srcu() that __dm_suspend() performs. The too srcu domains should not conflict... I don't even see a dax_read_lock() in this path.

Confirmed. I hit the hang in the lvm2-testsuite on vanilla 5.9-rc5. So, I'm going to proceed with Jan's patch plus my fixup.

Interestingly, the command lvm2-testsuite --only pvmove-abort-all.sh passed on vanilla 5.9-rc5 (on my two boxes).
Is it the same symptom (call trace) with my reported one?

Could you please run the above-mentioned command on your box (w/wo Jan's patch plus my fixup)?

-- Adrian
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

#10 lore raw
Quoted 95 lines · Huang Adrian
Quoted 85 lines · Dan Williams
Quoted 83 lines · Dan Williams
Quoted 76 lines · Huang Adrian
Quoted 61 lines · Jan Kara
Quoted 59 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +2 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;
+       int blocksize = *(int *) data, id;
+       bool rc;
 
Quoted 12 lines

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 7 lines
drivers/md/dm-table.c +5 -1
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Quoted 2 lines

Hold on. This patch hit another regression when I ran the full test of the lvm2-testsuite tool today.

Are you sure it's this patch?

Quoted 7 lines
Quoted 4 lines

The dax_read_lock() should have zero interaction with the synchronize_srcu() that __dm_suspend() performs. The too srcu domains should not conflict... I don't even see a dax_read_lock() in this path.

Confirmed. I hit the hang in the lvm2-testsuite on vanilla 5.9-rc5. So, I'm going to proceed with Jan's patch plus my fixup.

Quoted 6 lines

Interestingly, the command lvm2-testsuite --only pvmove-abort-all.sh passed on vanilla 5.9-rc5 (on my two boxes).
Is it the same symptom (call trace) with my reported one?

Could you please run the above-mentioned command on your box (w/wo Jan's patch plus my fixup)?

Sorry, it should be "w/wo Jan's patch plus Dan's fixup" (I just copied and pasted without modification).

-- Adrian
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

#11 lore raw
In reply to Huang Adrian

On Fri, Sep 18, 2020 at 5:41 AM Huang Adrian <[email protected]> wrote:
[..]

Quoted 6 lines

Interestingly, the command lvm2-testsuite --only pvmove-abort-all.sh passed on vanilla 5.9-rc5 (on my two boxes).
Is it the same symptom (call trace) with my reported one?

Could you please run the above-mentioned command on your box (w/wo Jan's patch plus my fixup)?

Thanks for the exact reproduction details. I was hitting a different hang, but I was able to reproduce this regression / latent bug. Fix is here:

https://lore.kernel.org/linux-nvdimm/160045867590.25663.7548541079217827340.stgit@dwillia2-desk3.amr.corp.intel.com/T/#u
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]

#12 lore raw
In reply to Dan Williams
Quoted 78 lines · Dan Williams
Quoted 76 lines · Huang Adrian
Quoted 73 lines · Jan Kara
Quoted 56 lines · Dan Williams
Quoted 30 lines · Jan Kara

DM was calling generic_fsdax_supported() to determine whether a device
referenced in the DM table supports DAX. However this is a helper for "leaf" device drivers so that they don't have to duplicate common generic checks. High level code
should call dax_supported() helper which that calls into appropriate
helper for the particular device. This problem manifested itself as
kernel messages:

dm-3: error: dax access failed (-95)

when lvm2-testsuite run in cases where a DM device was stacked on top of another DM device.

Fixes 7bf7eac8d648 ("dax: Arrange for dax_supported check to span multiple devices")
---
 drivers/dax/super.c   |  4 ++++
 drivers/md/dm-table.c |  3 +--
 include/linux/dax.h   | 11 +++++++++--
 3 files changed, 14 insertions(+), 4 deletions(-)

This patch should go in together with Adrian's
https://lore.kernel.org/linux-nvdimm/[email protected]

drivers/dax/super.c +0 -0
index e5767c83ea23..b6284c5cae0a 100644

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 330

Quoted 6 lines
drivers/dax/super.c +3 -0
@@ -325,11 +325,15 @@ EXPORT_SYMBOL_GPL(dax_direct_access);
 bool dax_supported(struct dax_device *dax_dev, struct block_device *bdev,
                int blocksize, sector_t start, sector_t len)
 {
+       if (!dax_dev)
+               return false;
+

Hi Jan, Thanks for this.

drivers/dax/super.c EXPORT_SYMBOL_GPL(dax_direct_access); · line 332

Quoted 2 lines
drivers/dax/super.c +0 -0
        if (!dax_alive(dax_dev))
                return false;

One small fixup to quiet lockdep because dax_supported() calls dax_alive() it expects that dax_read_lock() is held. So I'm testing with this incremental change:

drivers/md/dm-table.c +0 -1
index bed1ff0744ec..229f461e7def 100644
@@ -860,9 +860,14 @@ EXPORT_SYMBOL_GPL(dm_table_set_type);
 int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
                        sector_t start, sector_t len, void *data)
 {
-       int blocksize = *(int *) data;

drivers/md/dm-table.c EXPORT_SYMBOL_GPL(dm_table_set_type); · line 871

Quoted 10 lines
drivers/md/dm-table.c +7 -1
+       int blocksize = *(int *) data, id;
+       bool rc;
 
-       return dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       id = dax_read_lock();
+       rc = dax_supported(dev->dax_dev, dev->bdev, blocksize, start, len);
+       dax_read_unlock(id);
+
+       return rc;
 }

Yeah, thanks for this! I was actually looking into this when writing the patch and somehow convinced myself we will always be called through bdev_dax_supported() which does dax_read_lock() for us. But apparently I was wrong...

Quoted 4 lines
Quoted 2 lines

Hold on. This patch hit another regression when I ran the full test of the lvm2-testsuite tool today.

Are you sure it's this patch?

I'm pretty sure I applied this patch with your fixup. I tested it for three times:
1. lvm2-testsuite --only pvmove-abort-all.sh is always passed
without the patch.
2. lvm2-testsuite --only pvmove-abort-all.sh is always blocked
with the patch.

Quoted 4 lines

The dax_read_lock() should have zero interaction with the synchronize_srcu() that __dm_suspend() performs. The too srcu domains should not conflict... I don't even see a dax_read_lock() in this path.

Yup, I understand your observation. The call trace didn't show the dax_read_lock().

-- Adrian
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]