Skip to contentSkip to conversation

2.4: "access beyond end of device" after ext2 mount

DISCUSSION Mario Holbe 3 participants 10 messages Jan 15, 2005 – Jan 18, 2005

lore
Share & tools
lore
#0 lore raw

Hello,

mounting an ext2 (ext3 as well) filesystem seems to modify the block device's EOF behaviour: before the mount the device returned EOF, after the mount it doesn't anymore:

[on a fresh booted system]

root@darkside:~# uname -a
Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 87,157394 seconds (58699453 bytes/sec)
root@darkside:~# mke2fs /dev/hdg7
...
Block size=4096 (log=2)
Fragment size=4096 (log=2)
625248 inodes, 1249045 blocks
...
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 87,439332 seconds (58510184 bytes/sec)
root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
root@darkside:~# umount /dev/hdg7
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 87,145443 seconds (58707468 bytes/sec)
root@darkside:~# bc
1249045 * 4
4996180
1249045 * 4 * 2
9992360

Could somebody please explain this to me? Is this intentional? I could partly imagine the ext2 mount shrinking the block device's boundaries to the filesystem boundaries - for security reasons perhaps, even if I personally think this isn't the best idea at all, since it would violate layers encapsulation.
However, I have no idea why a) if so, this is not reverted at least on umount and why b) the EOF behaviour of the block device changes - before there were an EOF sent to the application (dd) while after there isn't, but an I/O error instead.

Kernel is Debian's kernel-source-2.4.27 + kernel-patch-2.4-i2c + kernel-patch-2.4-lm-sensors.

It seems there were already some mails regarding this issue suggesting this could have shown up between 2.4.24 and 2.4.25, but it seems they were misunderstood, for example:

From: "ProNIC Solutions" <[email protected]>
Subject: 2.4.25: attempt to access beyond end of device
Date: Tue, 16 Mar 2004 09:06:31 -0500
Message-ID: <005201c40b5f$e21d34a0$0600a8c0@p17>

From: "Peter S. Mazinger" <[email protected]>
Subject: BUG in 2.4.25-rc1: attempt to access beyond end of device
Date: Fri, 6 Feb 2004 13:53:40 +0100 (CET)
Message-ID: <[email protected]>

regards
Mario

Signature · 4 lines
Independence Day: Fortunately, the alien computer operating system works just
fine with the laptop. This proves an important point which Apple enthusiasts
have known for years. While the evil empire of Microsoft may dominate the
computers of Earth people, more advanced life forms clearly prefer Mac's.
#1 lore raw
Quoted 38 lines · Mario Holbe

Hello,

mounting an ext2 (ext3 as well) filesystem seems to modify the block device's EOF behaviour: before the mount the device returned EOF, after the mount it doesn't anymore:

[on a fresh booted system]

root@darkside:~# uname -a
Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 87,157394 seconds (58699453 bytes/sec)
root@darkside:~# mke2fs /dev/hdg7
...
Block size=4096 (log=2)
Fragment size=4096 (log=2)
625248 inodes, 1249045 blocks
...
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 87,439332 seconds (58510184 bytes/sec)
root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
root@darkside:~# umount /dev/hdg7
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 87,145443 seconds (58707468 bytes/sec)
root@darkside:~# bc
1249045 * 4
4996180
1249045 * 4 * 2
9992360
Quoted 1 line

Could somebody please explain this to me? Is this intentional?

No

Quoted 8 lines

I could partly imagine the ext2 mount shrinking the block device's boundaries to the filesystem boundaries - for security reasons perhaps, even if I personally think this isn't the best idea at all, since it would violate layers encapsulation.
However, I have no idea why a) if so, this is not reverted at least on umount and why b) the EOF behaviour of the block device changes - before there were an EOF sent to the application (dd) while after there isn't, but an I/O error instead.

Its indeed strange.

Quoted 12 lines

Kernel is Debian's kernel-source-2.4.27 + kernel-patch-2.4-i2c + kernel-patch-2.4-lm-sensors.

It seems there were already some mails regarding this issue suggesting this could have shown up between 2.4.24 and 2.4.25, but it seems they were misunderstood, for example:

From: "ProNIC Solutions" <[email protected]>
Subject: 2.4.25: attempt to access beyond end of device
Date: Tue, 16 Mar 2004 09:06:31 -0500
Message-ID: <005201c40b5f$e21d34a0$0600a8c0@p17>

Quoted 4 lines

From: "Peter S. Mazinger" <[email protected]>
Subject: BUG in 2.4.25-rc1: attempt to access beyond end of device
Date: Fri, 6 Feb 2004 13:53:40 +0100 (CET)
Message-ID: <[email protected]>

Actually these were fixed during v2.4.26-pre.

Can you please turn readahead off (hdparm -a 0 /dev/hdg) and repeat the tests?

The kernel is trying to read one block after EOD.

Quoted 2 lines

attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183

So maybe an off-by-one read-ahead?

Also I suggest a dump_stack() to see where the read is coming from.

Any other ideas?

#2 lore raw
Quoted 3 lines · Marcelo Tosatti
Quoted 1 line · Mario Holbe

Could somebody please explain this to me? Is this intentional?

No

Thanks :)

Quoted 2 lines

Can you please turn readahead off (hdparm -a 0 /dev/hdg) and repeat the tests? The kernel is trying to read one block after EOD.

The same happens.

root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
5116091392 bytes transferred in 117,573365 seconds (43514034 bytes/sec)
root@darkside:~# hdparm -a 0 /dev/hdg

/dev/hdg:
setting fs readahead to 0
readahead = 0 (off)

root@darkside:~# mount -t ext3 -o ro /dev/hdg7 /mnt
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
root@darkside:~# umount /dev/hdg7
root@darkside:~# hdparm -a 0 /dev/hdg

/dev/hdg:
setting fs readahead to 0
readahead = 0 (off)

root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 100,455251 seconds (50929028 bytes/sec)
root@darkside:~#
Quoted 1 line

So maybe an off-by-one read-ahead?

Please also note dd's output: 9992366 vs. 9992360 records.

root@darkside:~# cat /proc/partitions
...
  34     7    4996183 hdg7 156620 6088613 19984760 368240 3 0 24 0 0 190720 368240
...
root@darkside:~# bc
4996183 * 2
9992366
root@darkside:~# tune2fs -l /dev/hdg7
...
Block count:              1249045
...
root@darkside:~# bc
4096 * 1249045 / 512
9992360

The block device (the partition) is 4996183 kB or 9992366 sectors, while the ext3 fs is 9992360 sectors. Exactly these 9992360 sectors could be read by dd in the second pass.

root@darkside:~# dd if=/dev/hdg7 of=/dev/null bs=512
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
5116088320 bytes transferred in 92,603241 seconds (55247400 bytes/sec)
root@darkside:~#

Fixing dd's blocksize to 512 doesn't help either.

Quoted 2 lines

Also I suggest a dump_stack() to see where the read is coming from.

I have no idea, how and where to do a dump_stack().

regards
Mario

Signature · 2 lines
() Ascii Ribbon Campaign
/\ Support plain text e-mail
#3 lore raw
In reply to Marcelo Tosatti
Quoted 26 lines · Marcelo Tosatti
Quoted 24 lines · Mario Holbe

mounting an ext2 (ext3 as well) filesystem seems to modify the block device's EOF behaviour: before the mount the device returned EOF, after the mount it doesn't anymore:

[on a fresh booted system]

root@darkside:~# uname -a
Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
root@darkside:~# umount /dev/hdg7
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
root@darkside:~# bc
1249045 * 4
4996180
1249045 * 4 * 2
9992360
Quoted 5 lines
Quoted 1 line

Could somebody please explain this to me? Is this intentional?

No

Its indeed strange.

I suppose that what happens is the following:
mounting sets the blocksize to 4096.
After reading 9992360 sectors, reading the next block means reading the next 8 sectors and that fails because only 6 sectors are left.

Test that this is what happens using blockdev --getbsz.

If you want to restore the device to full size, use blockdev --setbsz 512.

Andries

#4 lore raw
Quoted 5 lines · Andries Brouwer

I suppose that what happens is the following:
mounting sets the blocksize to 4096.
After reading 9992360 sectors, reading the next block means reading the next 8 sectors and that fails because only 6 sectors are left. Test that this is what happens using blockdev --getbsz.

Yes! This was the command I was looking for ;)

root@darkside:~# blockdev --getbsz /dev/hdg7
4096
root@darkside:~# blockdev --getbsz /dev/hdg6
1024
root@darkside:~# mount -t ext3 -o ro /dev/hdg6 /mnt
root@darkside:~# umount /dev/hdg6 
root@darkside:~# blockdev --getbsz /dev/hdg6
4096
Quoted 2 lines

If you want to restore the device to full size, use blockdev --setbsz 512.

Does that in any way hurt, if a filesystem is just mounted? I mean, what would happen, if I mount /dev/hdg7 and then set the block size back to 1024? Would that perhaps corrupt my filesystem or something like that?

Mario

Signature · 4 lines
<jv> Oh well, config
<jv> one actually wonders what force in the universe is holding it
<jv> and makes it working
<Beeth> chances and accidents :)
#5 lore raw
Quoted 4 lines · Mario Holbe
Quoted 2 lines

If you want to restore the device to full size, use blockdev --setbsz 512.

Does that in any way hurt, if a filesystem is just mounted?

It is a bad idea to change the blocksize of a mounted filesystem.

#6 lore raw
In reply to Andries Brouwer

Hi Andries,

Quoted 29 lines · Andries Brouwer
Quoted 28 lines · Marcelo Tosatti
Quoted 25 lines · Mario Holbe

mounting an ext2 (ext3 as well) filesystem seems to modify the block device's EOF behaviour: before the mount the device returned EOF, after the mount it doesn't anymore:

[on a fresh booted system]

root@darkside:~# uname -a
Linux darkside 2.4.27 #1 Sat Jan 15 17:07:20 CET 2005 i686 GNU/Linux
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
9992366+0 records in
9992366+0 records out
root@darkside:~# mount -t ext2 -o ro /dev/hdg7 /mnt
root@darkside:~# umount /dev/hdg7
root@darkside:~# dd if=/dev/hdg7 of=/dev/null
attempt to access beyond end of device
22:07: rw=0, want=4996184, limit=4996183
dd: reading `/dev/hdg7': Input/output error
9992360+0 records in
9992360+0 records out
root@darkside:~# bc
1249045 * 4
4996180
1249045 * 4 * 2
9992360

Could somebody please explain this to me? Is this intentional?

Quoted 8 lines
Quoted 3 lines

No

Its indeed strange.

I suppose that what happens is the following:
mounting sets the blocksize to 4096.
After reading 9992360 sectors, reading the next block means reading the next 8 sectors and that fails because only 6 sectors are left.

So this is either not a Linux error and not a disk error, its just that the "use with filesystem" then "direct access" is a unfortunate combination.

What would be the correct fix for this for this, if any?

v2.6 should suffer from the same issues?

Quoted 4 lines

Test that this is what happens using blockdev --getbsz.

If you want to restore the device to full size, use blockdev --setbsz 512.

#7 lore raw
Quoted 3 lines · Marcelo Tosatti

So this is either not a Linux error and not a disk error, its just that the "use with filesystem" then "direct access" is a unfortunate combination. What would be the correct fix for this for this, if any?

Well, I personally think at least the EOF behaviour should be fixed somehow.
I understand the point that the device's blocksize affects the device's length... obviously a block device can only consist of full blocks, not half a block or something like that.
However, if that's right, a block device's length should IMHO also be affected by it's blocksize - thus, the block device should return an EOF after the last block was read.

In my case this would mean, it should return an EOF if blocksize is 1024 and 4996183 blocks (9992366 sectors) are read or better, if the 4996184th block is attempted to read and it should return an EOF if blocksize is 4096 and 1249045 blocks (9992360 sectors) are read or better, if the 1249046th block is attempted to read.

Another approach could be to allow 'fractions of blocks' :) Then, the device could have a blocksize of 4096 and consist of 1249045 full blocks and a 6/8 block. However, I'm not sure, if this is possible at all.

Quoted 1 line

v2.6 should suffer from the same issues?

I don't know, but if it doesn't, one could just backport 2.6's solution :)

Mario

Signature · 4 lines
Independence Day: Fortunately, the alien computer operating system works just
fine with the laptop. This proves an important point which Apple enthusiasts
have known for years. While the evil empire of Microsoft may dominate the
computers of Earth people, more advanced life forms clearly prefer Mac's.
#8 lore raw
In reply to Marcelo Tosatti
Quoted 7 lines · Marcelo Tosatti
Quoted 4 lines

I suppose that what happens is the following:
mounting sets the blocksize to 4096.
After reading 9992360 sectors, reading the next block means reading the next 8 sectors and that fails because only 6 sectors are left.

So this is either not a Linux error and not a disk error, its just that the "use with filesystem" then "direct access" is a unfortunate combination.

It is not a disk error, but I consider it a Linux error.

Quoted 1 line

What would be the correct fix for this for this, if any?

For 2.4 my reaction would be to say that it is a known property of the system, possibly less fortunate, an unimportant flaw. Of course a fix is possible if this is deemed important for some reason.

Quoted 1 line

v2.6 should suffer from the same issues?

I don't think so. But 2.6 details are rather different.

Andries

#9 lore raw
Quoted 11 lines · Andries Brouwer
Quoted 7 lines · Marcelo Tosatti
Quoted 4 lines

I suppose that what happens is the following:
mounting sets the blocksize to 4096.
After reading 9992360 sectors, reading the next block means reading the next 8 sectors and that fails because only 6 sectors are left.

So this is either not a Linux error and not a disk error, its just that the "use with filesystem" then "direct access" is a unfortunate combination.

It is not a disk error, but I consider it a Linux error.

OK.

Quoted 4 lines
Quoted 1 line

What would be the correct fix for this for this, if any?

For 2.4 my reaction would be to say that it is a known property of the system, possibly less fortunate, an unimportant flaw.

This seems to be harmless, so, better do nothing about it.

Quoted 5 lines

Of course a fix is possible if this is deemed important for some reason.

Quoted 1 line

v2.6 should suffer from the same issues?

I don't think so. But 2.6 details are rather different.

OK!