Skip to contentSkip to conversation

blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

PATCH Superseded Mikulas Patocka 3 participants 5 messages Nov 19, 2020 – Nov 20, 2020 6 messages not in the archive

Subsystems (2)
Block layer Device mapper
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): alpine.LRH.2.02.2011190810001.32672@file01.intranet.prod.int.rdu2.redhat.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): alpine.LRH.2.02.2011191337180.588@file01.intranet.prod.int.rdu2.redhat.com

Mikulas Patocka

author maintainer
#6 lore raw

PATCH 1/1 blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

We get these I/O errors when we run md-raid1 on the top of dm-integrity on the top of ramdisk:
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The dm-integrity device uses logical_block_size 4096 and it doesn't affect the "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it will split the bios on 255-sector boundary, making the bios unaligned on logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by Mikulas Patocka <[email protected]>
Cc [email protected]
---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
block/blk-settings.c +10 -0
Index: linux-2.6/block/blk-settings.c
===================================================================
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
 		ret = -1;
 	}
 
+	t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
+	if (t->max_sectors < PAGE_SIZE / 512)
+		t->max_sectors = PAGE_SIZE / 512;
+	t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512);
+	if (t->max_sectors < PAGE_SIZE / 512)
+		t->max_hw_sectors = PAGE_SIZE / 512;
+	t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512);
+	if (t->max_sectors < PAGE_SIZE / 512)
+		t->max_dev_sectors = PAGE_SIZE / 512;
+
 	/* Discard alignment and granularity */
 	if (b->discard_granularity) {
 		alignment = queue_limit_discard_alignment(b, start);
#7 lore raw

Greetings;

Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >> SECTOR_SHIFT instead of / 512.

I don't understand the three conditionals. I believe max_sectors is supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't understand why max_sectors being small should adjust max_hw_sectors and max_dev_sectors. Are the conditions perhaps supposed to be different, adjusting each max_*sectors up to at least PAGE_SIZE / SECTOR_SIZE? Perhaps, like e.g. blk_queue_max_hw_sectors(), the conditionals should log if they are adjusting max_*sectors up to the minimum.

Thanks!

John Dorminy

Quoted 54 lines · Mikulas Patocka

We get these I/O errors when we run md-raid1 on the top of dm-integrity on the top of ramdisk:
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The dm-integrity device uses logical_block_size 4096 and it doesn't affect the "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it will split the bios on 255-sector boundary, making the bios unaligned on logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by Mikulas Patocka <[email protected]>
Cc [email protected]
---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

Index: linux-2.6/block/blk-settings.c
===================================================================

block/blk-settings.c 2020-11-19 21:20:18.000000000 +0100 renamed from block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100 +10 -0
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
                ret = -1;
        }
 
+       t->max_sectors = round_down(t->max_sectors, t->logical_block_size
 / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_sectors = PAGE_SIZE / 512;
+       t->max_hw_sectors = round_down(t->max_hw_sectors,
 t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_hw_sectors = PAGE_SIZE / 512;
+       t->max_dev_sectors = round_down(t->max_dev_sectors,
 t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_dev_sectors = PAGE_SIZE / 512;
+

        /* Discard alignment and granularity */
        if (b->discard_granularity) {
                alignment = queue_limit_discard_alignment(b, start);
Quoted 6 lines

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

#8 lore raw
In reply to Mikulas Patocka

Greetings;

Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >> SECTOR_SHIFT instead of / 512.

I don't understand the three conditionals. I believe max_sectors is supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't understand why max_sectors being small should adjust max_hw_sectors and max_dev_sectors. Are the conditions perhaps supposed to be different, adjusting each max_*sectors up to at least PAGE_SIZE / SECTOR_SIZE? Perhaps, like e.g. blk_queue_max_hw_sectors(), the conditionals should log if they are adjusting max_*sectors up to the minimum.

Thanks!

John Dorminy

Quoted 57 lines · Mikulas Patocka

We get these I/O errors when we run md-raid1 on the top of dm-integrity on the top of ramdisk:
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The dm-integrity device uses logical_block_size 4096 and it doesn't affect the "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it will split the bios on 255-sector boundary, making the bios unaligned on logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by Mikulas Patocka <[email protected]>
Cc [email protected]
---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

Index: linux-2.6/block/blk-settings.c
===================================================================

block/blk-settings.c 2020-11-19 21:20:18.000000000 +0100 renamed from block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100 +10 -1
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
                ret = -1;
        }
 
+       t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_sectors = PAGE_SIZE / 512;
+       t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_hw_sectors = PAGE_SIZE / 512;
+       t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_dev_sectors = PAGE_SIZE / 512;
+
        /* Discard alignment and granularity */
        if (b->discard_granularity) {
                alignment = queue_limit_discard_alignment(b, start);
 
--

dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

author maintainer
#9 lore raw
Quoted 4 lines · John Dorminy

Greetings;

Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >> SECTOR_SHIFT instead of / 512.

Yes, that's a good point.

Quoted 6 lines

I don't understand the three conditionals. I believe max_sectors is supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't understand why max_sectors being small should adjust max_hw_sectors and max_dev_sectors. Are the conditions perhaps supposed to be different, adjusting each max_*sectors up to at least PAGE_SIZE / SECTOR_SIZE?

I copied this pattern from blk_queue_max_hw_sectors. Perhaps, we could use:

	t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
	if (!t->max_sectors)
		t->max_sectors = t->logical_block_size / 512;

instead.

Jens, what do you think is better?

Mikulas

Quoted 67 lines

Perhaps, like e.g. blk_queue_max_hw_sectors(), the conditionals should log if they are adjusting max_*sectors up to the minimum.

Thanks!

John Dorminy

Quoted 57 lines · Mikulas Patocka

We get these I/O errors when we run md-raid1 on the top of dm-integrity on the top of ramdisk:
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The dm-integrity device uses logical_block_size 4096 and it doesn't affect the "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it will split the bios on 255-sector boundary, making the bios unaligned on logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by Mikulas Patocka <[email protected]>
Cc [email protected]
---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

Index: linux-2.6/block/blk-settings.c
===================================================================

block/blk-settings.c 2020-11-19 21:20:18.000000000 +0100 renamed from block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100 +10 -1
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
                ret = -1;
        }
 
+       t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_sectors = PAGE_SIZE / 512;
+       t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_hw_sectors = PAGE_SIZE / 512;
+       t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512);
+       if (t->max_sectors < PAGE_SIZE / 512)
+               t->max_dev_sectors = PAGE_SIZE / 512;
+
        /* Discard alignment and granularity */
        if (b->discard_granularity) {
                alignment = queue_limit_discard_alignment(b, start);
 
--

dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

author maintainer
#10 lore raw

PATCH v2 1/1 blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary.

In reply to Mikulas Patocka

We get I/O errors when we run md-raid1 on the top of dm-integrity on the top of ramdisk.
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff
device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb

The ramdisk device has logical_block_size 512 and max_sectors 255. The dm-integrity device uses logical_block_size 4096 and it doesn't affect the "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have a device with max_sectors not aligned on logical_block_size.

The md-raid device sees that the underlying leg has max_sectors 255 and it will split the bios on 255-sector boundary, making the bios unaligned on logical_block_size.

In order to fix the bug, we round down max_sectors to logical_block_size.

Signed-off-by Mikulas Patocka <[email protected]>
Cc [email protected]
---
 block/blk-settings.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
block/blk-settings.c +10 -0
Index: linux-2.6/block/blk-settings.c
===================================================================
@@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits
 		ret = -1;
 	}
 
+	t->max_sectors = round_down(t->max_sectors, t->logical_block_size >> SECTOR_SHIFT);
+	if (t->max_sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		t->max_sectors = PAGE_SIZE >> SECTOR_SHIFT;
+	t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size >> SECTOR_SHIFT);
+	if (t->max_hw_sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		t->max_hw_sectors = PAGE_SIZE >> SECTOR_SHIFT;
+	t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size >> SECTOR_SHIFT);
+	if (t->max_dev_sectors < PAGE_SIZE >> SECTOR_SHIFT)
+		t->max_dev_sectors = PAGE_SIZE >> SECTOR_SHIFT;
+
 	/* Discard alignment and granularity */
 	if (b->discard_granularity) {
 		alignment = queue_limit_discard_alignment(b, start);

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel