LVM on md0: raid0_make_request bug: can't convert block across chunks or bigger than 64k

DISCUSSION Karl Vogel 3 participants 9 messages Oct 20, 2003 – Oct 28, 2003 Device mapper

Share & tools

Karl Vogel

author

2003-10-20 20:35 UTC #0 lore raw

I'm getting the following kernel messages on V2.6.0-test8-mm1 (I've also tried plain -test7 and some kernels before that) when copying moderately sized files from a raid-0/LVM volume:

--- snip ---
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24081064 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080656 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080784 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080928 64
--- snip ---

The data was created on a V2.4 kernel. The partition contains 1 LVM volume, which is using a RAID-0 md0 as the physical volume:

# ./lvm lvdisplay
  --- Logical volume ---
  LV Name                /dev/system/root
  VG Name                system
  LV UUID                000000-0000-0000-0000-0000-0000-000000
  LV Write Access        read/write
  LV Status              available
  # open                 2
  LV Size                11.58 GB
  Current LE             2964
  Segments               1
  Allocation             next free
  Read ahead sectors     10000
  Block device           254:0

# ./lvm pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               system
  PV Size               11.58 GB / not usable 4.31 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              2964
  Free PE               0
  Allocated PE          2964
  PV UUID               uYHQYU-TmrF-I3Ws-674P-v3Rx-4CPz-mOe5Tu

# ./lvm vgdisplay
  --- Volume group ---
  VG Name               system
  System ID             kvo.local.org1047770061
  Format                lvm1
  VG Access             read/write
  VG Status             resizable
  MAX LV                256
  Cur LV                1
  Open LV               0
  Max PV                256
  Cur PV                1
  Act PV                1
  VG Size               11.58 GB
  PE Size               4.00 MB
  Total PE              2964
  Alloc PE / Size       2964 / 11.58 GB
  Free  PE / Size       0 / 0
  VG UUID               AE3SH5-7pbd-itL5-firC-Sxis-0IXE-qMa5dD

The RAID-0 is setup as:

# cat /etc/raidtab
raiddev /dev/md0
        raid-level              0
        nr-raid-disks           2
        persistent-superblock   0
        chunk-size              8

        device                  /dev/hdd1
        raid-disk               0
        device                  /dev/hdc2
        raid-disk               1

The boot message of the md0 initializing looks like:

Oct 20 20:44:55 kvo kernel: md: Autodetecting RAID arrays.
Oct 20 20:44:55 kvo kernel: md: autorun ...
Oct 20 20:44:55 kvo kernel: md: considering hdd1 ...
Oct 20 20:44:55 kvo kernel: md:  adding hdd1 ...
Oct 20 20:44:55 kvo kernel: md:  adding hdc2 ...
Oct 20 20:44:55 kvo kernel: md: created md0
Oct 20 20:44:55 kvo kernel: md: bind<hdc2>
Oct 20 20:44:55 kvo kernel: md: bind<hdd1>
Oct 20 20:44:55 kvo kernel: md: running: <hdd1><hdc2>
Oct 20 20:44:55 kvo kernel: md0: setting max_sectors to 128, segment
boundary to 32767
Oct 20 20:44:55 kvo kernel: raid0: looking at hdd1
Oct 20 20:44:55 kvo kernel: raid0:   comparing hdd1(6345600) with
hdd1(6345600)
Oct 20 20:44:55 kvo kernel: raid0:   END
Oct 20 20:44:55 kvo syslog: klogd startup succeeded
Oct 20 20:44:55 kvo kernel: raid0:   ==> UNIQUE
Oct 20 20:44:55 kvo kernel: raid0: 1 zones
Oct 20 20:44:55 kvo kernel: raid0: looking at hdc2
Oct 20 20:44:55 kvo kernel: raid0:   comparing hdc2(5799360) with
hdd1(6345600)
Oct 20 20:44:55 kvo kernel: raid0:   NOT EQUAL
Oct 20 20:44:55 kvo kernel: raid0:   comparing hdc2(5799360) with
hdc2(5799360)
Oct 20 20:44:55 kvo kernel: raid0:   END
Oct 20 20:44:55 kvo kernel: raid0:   ==> UNIQUE
Oct 20 20:44:55 kvo kernel: raid0: 2 zones
Oct 20 20:44:55 kvo kernel: raid0: FINAL 2 zones
Oct 20 20:44:55 kvo kernel: raid0: zone 1
Oct 20 20:44:55 kvo kernel: raid0: checking hdc2 ... nope.
Oct 20 20:44:55 kvo kernel: raid0: checking hdd1 ... contained as device
0
Oct 20 20:44:55 kvo kernel:   (6345600) is smallest!.
Oct 20 20:44:55 kvo kernel: raid0: zone->nb_dev: 1, size: 546240
Oct 20 20:44:55 kvo kernel: raid0: current zone offset: 6345600
Oct 20 20:44:55 kvo kernel: raid0: done.
Oct 20 20:44:55 kvo kernel: raid0 : md_size is 12144960 blocks.
Oct 20 20:44:55 kvo kernel: raid0 : conf->hash_spacing is 11598720
blocks.
Oct 20 20:44:55 kvo kernel: raid0 : nb_zone is 2.
Oct 20 20:44:55 kvo kernel: raid0 : Allocating 8 bytes for hash.
Oct 20 20:44:55 kvo kernel: md: ... autorun DONE.

I'm using LVM2.1.95.15 with the V1 interface (am getting errors when compiling the latest LVM2 tools, but that's a different story :)

Please use reply-all if you require more details.

Regards,
Karl

Kevin Corry

2003-10-20 21:12 UTC #1 lore raw

Quoted 4 lines · Karl Vogel

I'm getting the following kernel messages on V2.6.0-test8-mm1 (I've also tried plain -test7 and some kernels before that) when copying moderately sized files from a raid-0/LVM volume:

Quoted 9 lines

--- snip ---
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24081064 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080656 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080784 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080928 64

Looks like this was just recently fixed on the linux-raid list.

http://marc.theaimsgroup.com/?l=linux-raid&m=106661294929434

Signature · 3 lines

Kevin Corry
[email protected]
http://evms.sourceforge.net/

Karl Vogel

author

2003-10-20 21:53 UTC #2 lore raw

Quoted 15 lines · Kevin Corry

Quoted 4 lines · Karl Vogel
I'm getting the following kernel messages on V2.6.0-test8-mm1 (I've also tried plain -test7 and some kernels before that) when copying moderately sized files from a raid-0/LVM volume:
Quoted 9 lines
--- snip ---
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24081064 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080656 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080784 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080928 64

Quoted 3 lines

Looks like this was just recently fixed on the linux-raid list.
http://marc.theaimsgroup.com/?l=linux-raid&m=106661294929434

Applied the patch on 2.6.0-test8-mm1 but it made no difference.

Somebody else referred to this posting:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103369952814053&w=2

but that patch doesn't apply cleanly anymore and I'm not familiar with the code to be confident to fix it up myself. (that post was from almost exactly 1 year ago, so alot changed probably :)

Neil Brown

2003-10-21 00:09 UTC #3 lore raw

Quoted 16 lines · On Monday October 20, [email protected] wrote:

Quoted 15 lines · Kevin Corry
Quoted 13 lines · Karl Vogel
I'm getting the following kernel messages on V2.6.0-test8-mm1 (I've also tried plain -test7 and some kernels before that) when copying moderately sized files from a raid-0/LVM volume:
--- snip ---
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24081064 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080656 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080784 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080928 64

Quoted 5 lines

Quoted 3 lines
Looks like this was just recently fixed on the linux-raid list.
http://marc.theaimsgroup.com/?l=linux-raid&m=106661294929434
Applied the patch on 2.6.0-test8-mm1 but it made no difference.

no, thats a completely different problem.

Quoted 7 lines

Somebody else referred to this posting:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103369952814053&w=2
but that patch doesn't apply cleanly anymore and I'm not familiar with the code to be confident to fix it up myself. (that post was from almost exactly 1 year ago, so alot changed probably :)

That patch is already included.

The problem is that dm is not honouring the merge_bvec_fn that raid0 has set.

This patch might fix it, but I'm not very familiar with the dm code, so I make no promises.

(I wonder why you are running LVM on top of raid0 given that lvm contains raid0 functionality).

NeilBrown

----------- Diffstat output ------------

 ./drivers/md/dm-table.c |    5 +++++
 1 files changed, 5 insertions(+)

drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +5 -0

diff ./drivers/md/dm-table.c~current~ ./drivers/md/dm-table.c
@@ -489,6 +489,11 @@ int dm_get_device(struct dm_target *ti, 
 		rs->max_sectors =
 			min_not_zero(rs->max_sectors, q->max_sectors);
 
+		if (q->merge_bvec_fn)
+			rs->max_sectors =
+				min_not_zero(rs->max_sectors, PAGE_SIZE>>9);
+			
+
 		rs->max_phys_segments =
 			min_not_zero(rs->max_phys_segments,
 				     q->max_phys_segments);

Kevin Corry

2003-10-21 13:42 UTC #4 lore raw

Quoted 17 lines · Neil Brown

Quoted 16 lines · On Monday October 20, [email protected] wrote:
Quoted 15 lines · Kevin Corry
Quoted 13 lines · Karl Vogel
I'm getting the following kernel messages on V2.6.0-test8-mm1 (I've also tried plain -test7 and some kernels before that) when copying moderately sized files from a raid-0/LVM volume:
--- snip ---
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24081064 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080656 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080784 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080928 64

Quoted 7 lines

Quoted 5 lines
Quoted 3 lines
Looks like this was just recently fixed on the linux-raid list.
http://marc.theaimsgroup.com/?l=linux-raid&m=106661294929434
Applied the patch on 2.6.0-test8-mm1 but it made no difference.
no, thats a completely different problem.

Sorry for the confusion. Karl's error messages looked just like ones that were reported on the evms-devel list, which were supposedly fixed by the above patch.

Quoted 15 lines

The problem is that dm is not honouring the merge_bvec_fn that raid0 has set.
This patch might fix it, but I'm not very familiar with the dm code, so I make no promises.
----------- Diffstat output ------------
 ./drivers/md/dm-table.c |    5 +++++
 1 files changed, 5 insertions(+)
diff ./drivers/md/dm-table.c~current~ ./drivers/md/dm-table.c
drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +0 -0
@@ -489,6 +489,11 @@ int dm_get_device(struct dm_target *ti,
 		rs->max_sectors =

drivers/md/dm-table.c int dm_get_device(struct dm_target *ti, · line 499

Quoted 10 lines

drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +5 -0

 			min_not_zero(rs->max_sectors, q->max_sectors);
 
+		if (q->merge_bvec_fn)
+			rs->max_sectors =
+				min_not_zero(rs->max_sectors, PAGE_SIZE>>9);
+
+
 		rs->max_phys_segments =
 			min_not_zero(rs->max_phys_segments,
 				     q->max_phys_segments);

This will probably work, as long as raid0 can split a one-page request that spans a chunk boundary. I'll be interested to see if this solves Karl's problem.

Joe Thornber was telling me about an idea he had to solve the Device-Mapper-on-top-of-MD problem using the dm-io code in his latest Device-Mapper patchset
(http://people.sistina.com/~thornber/patches/2.6/2.6.0-test6/2.6.0-t6-mm1-dm2.tar.bz2).
I believe he's away for this week, but I'll ask him about it again the next time I talk to him.

Signature · 3 lines

Kevin Corry
[email protected]
http://evms.sourceforge.net/

Karl Vogel

author

2003-10-21 16:29 UTC #5 lore raw

Quoted 47 lines · Kevin Corry

Quoted 17 lines · Neil Brown
Quoted 16 lines · On Monday October 20, [email protected] wrote:
Quoted 15 lines · Kevin Corry
Quoted 13 lines · Karl Vogel
I'm getting the following kernel messages on V2.6.0-test8-mm1 (I've also tried plain -test7 and some kernels before that) when copying moderately sized files from a raid-0/LVM volume:
--- snip ---
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24081064 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080656 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080784 64
raid0_make_request bug: can't convert block across chunks or bigger than 64k 24080928 64
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Looks like this was just recently fixed on the linux-raid list.
http://marc.theaimsgroup.com/?l=linux-raid&m=106661294929434
Applied the patch on 2.6.0-test8-mm1 but it made no difference.
no, thats a completely different problem.
Sorry for the confusion. Karl's error messages looked just like ones that were reported on the evms-devel list, which were supposedly fixed by the above patch.
Quoted 17 lines
The problem is that dm is not honouring the merge_bvec_fn that raid0 has set.
This patch might fix it, but I'm not very familiar with the dm code, so I make no promises.
----------- Diffstat output ------------
 ./drivers/md/dm-table.c |    5 +++++
 1 files changed, 5 insertions(+)
diff ./drivers/md/dm-table.c~current~ ./drivers/md/dm-table.c
drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +0 -0
@@ -489,6 +489,11 @@ int dm_get_device(struct dm_target *ti,
 		rs->max_sectors =
 			min_not_zero(rs->max_sectors, q->max_sectors);
 

Quoted 12 lines

drivers/md/dm-table.c int dm_get_device(struct dm_target *ti, · line 499
Quoted 8 lines
drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +5 -0
+		if (q->merge_bvec_fn)
+			rs->max_sectors =
+				min_not_zero(rs->max_sectors, PAGE_SIZE>>9);
+
+
 		rs->max_phys_segments =
 			min_not_zero(rs->max_phys_segments,
 				     q->max_phys_segments);
This will probably work, as long as raid0 can split a one-page request that spans a chunk boundary. I'll be interested to see if this solves Karl's problem.

Good news... it solves the problem with my setup. I was able to copy files off the logical volume (did an md5sum compare to make sure I got the complete files.)

Quoted 6 lines

Joe Thornber was telling me about an idea he had to solve the Device-Mapper-on-top-of-MD problem using the dm-io code in his latest Device-Mapper patchset
(http://people.sistina.com/~thornber/patches/2.6/2.6.0-test6/2.6.0-t6-mm1-dm2.tar.bz2).
I believe he's away for this week, but I'll ask him about it again the next time I talk to him.

Karl Vogel

author

2003-10-21 16:41 UTC #6 lore raw

Quoted 24 lines

Quoted 11 lines
Quoted 11 lines
----------- Diffstat output ------------
 ./drivers/md/dm-table.c |    5 +++++
 1 files changed, 5 insertions(+)
diff ./drivers/md/dm-table.c~current~ ./drivers/md/dm-table.c
drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +0 -0
@@ -489,6 +489,11 @@ int dm_get_device(struct dm_target *ti,
 		rs->max_sectors =
 			min_not_zero(rs->max_sectors, q->max_sectors);
 
Quoted 12 lines
drivers/md/dm-table.c int dm_get_device(struct dm_target *ti, · line 499
Quoted 8 lines
drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +5 -0
+		if (q->merge_bvec_fn)
+			rs->max_sectors =
+				min_not_zero(rs->max_sectors, PAGE_SIZE>>9);
+
+
 		rs->max_phys_segments =
 			min_not_zero(rs->max_phys_segments,
 				     q->max_phys_segments);
This will probably work, as long as raid0 can split a one-page request that spans a chunk boundary. I'll be interested to see if this solves Karl's problem.

Quoted 3 lines

Good news... it solves the problem with my setup. I was able to copy files off the logical volume (did an md5sum compare to make sure I got the complete files.)

Forgot to mention that it generates a compiler warning:

CC drivers/md/dm-table.o
drivers/md/dm-table.c: In function `dm_get_device':
drivers/md/dm-table.c:494: warning: comparison of distinct pointer types lacks a cast

Using:

$ gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.3.1/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --with-system-zlib --enable-__cxa_atexit
--host=i386-redhat-linux
Thread model: posix
gcc version 3.3.1 20030930 (Red Hat Linux 3.3.1-6)

Karl Vogel

author

2003-10-25 20:24 UTC #7 lore raw

Re: LVM on md0: raid0_make_request bug: can't convert block acros s chunks or bigger than 64k

In reply to Neil Brown

Quoted 33 lines · Neil Brown

The problem is that dm is not honouring the merge_bvec_fn that raid0 has set.
This patch might fix it, but I'm not very familiar with the dm code, so I make no promises.
(I wonder why you are running LVM on top of raid0 given that lvm contains raid0 functionality).
NeilBrown
----------- Diffstat output ------------
 ./drivers/md/dm-table.c |    5 +++++
 1 files changed, 5 insertions(+)
diff ./drivers/md/dm-table.c~current~ ./drivers/md/dm-table.c
--- ./drivers/md/dm-table.c~current~ 2003-10-21 10:05:29.000000000
+1000
+++ ./drivers/md/dm-table.c 2003-10-21 10:06:27.000000000 +1000
drivers/md/dm-table.c renamed from drivers/md/dm-table.c~current~ +5 -0
@@ -489,6 +489,11 @@ int dm_get_device(struct dm_target *ti, 
 		rs->max_sectors =
 			min_not_zero(rs->max_sectors, q->max_sectors);
 
+		if (q->merge_bvec_fn)
+			rs->max_sectors =
+				min_not_zero(rs->max_sectors,
 PAGE_SIZE>>9);
+			
+
 		rs->max_phys_segments =
 			min_not_zero(rs->max_phys_segments,

Quoted 1 line

q->max_phys_segments);

I noticed in the 2.6.0-test9 notes the following:

---
Neil Brown:
o md - Use sector rather than block numbers when splitting raid0 requests
---

I'm not sure if this is related to the problem I was experiencing?! Anyway this doesn't fix the problem I was having. I still get the errors with -test9. Above patch to dm-table.c works for me.

Just thought I'd mention this..

Neil Brown

2003-10-28 01:08 UTC #8 lore raw

Re: LVM on md0: raid0_make_request bug: can't convert block acros s chunks or bigger than 64k

Quoted 10 lines · On Saturday October 25, [email protected] wrote:

I noticed in the 2.6.0-test9 notes the following:
---
Neil Brown:
o md - Use sector rather than block numbers when splitting raid0 requests
---

Quoted 6 lines

I'm not sure if this is related to the problem I was experiencing?! Anyway this doesn't fix the problem I was having. I still get the errors with -test9. Above patch to dm-table.c works for me.
Just thought I'd mention this..

This addresses a different problem: raid0 would not correctly handle some requests that are not 1K aligned. XFS does some non-1K-aligned requests where reading it's journal and caught this bug. It has nothing to do with LVM or your problem.

NeilBrown