ext4: fix infinite loop at ext4_da_writepages with the terminal extent block of too big file
PATCH RESEND 1/1 ext4: fix infinite loop at ext4_da_writepages with the terminal extent block of too big file
From: Toshiyuki Okajima <[email protected]>
On linux-2.6.36-rc2, if we execute the following script, we can encounter the hangup of '/bin/sync' command:
================================================================================
#!/bin/sh
echo -n "HANG UP TEST: "/bin/dd if=/dev/zero of=/tmp/img bs=1k count=1 seek=1M 2> /dev/null /sbin/mkfs.ext4 -Fq /tmp/img/bin/mount -o loop -t ext4 /tmp/img /mnt/bin/dd if=/dev/zero of=/mnt/file bs=1 count=1 \
seek=$((16*1024*1024*1024*1024-4096)) 2> /dev/null /bin/sync/bin/umount /mnt
echo "DONE"
exit 0
================================================================================
We can see the following backtrace if we get the kdump when this hangup occurs:
================================================================================kthread()
=> bdi_writeback_thread()
=> wb_do_writeback()
=> wb_writeback()
=> writeback_inodes_wb()
=> writeback_sb_inodes()
=> writeback_single_inode()
=> ext4_da_writepages() ---+
^ infinite |
| loop |
+-------------+================================================================================
The reason why this hangup happens is described as follows:
1) We write the last extent block of the file whose size is the filesystem maximum size.
2) "BH_Delay" flag is set on the buffer_head of its block.
3) - the member, "m_lblk" of struct mpage_da_data is 4294967295 (UINT_MAX)
- the member, "m_len" of struct mpage_da_data is 1
mpage_put_bnr_to_bhs() which is called via ext4_da_writepages() cannot clear "BH_Delay" flag of the buffer_head because the type of m_lblk is ext4_lblk_t and then m_lblk + m_len is overflow.
Therefore an infinite loop occurs because ext4_da_writepages() cannot write the page (which corresponds to the block) since "BH_Delay" flag isn't cleared.
--------------------------------------------------------------------------------
static void mpage_put_bnr_to_bhs(struct mpage_da_data *mpd,
struct ext4_map_blocks *map)
{
...
int blocks = map->m_len;
...
do {
// cur_logical = 4294967295
// map->m_lblk = 4294967295
// blocks = 1
// *** map->m_lblk + blocks == 0 (OVERFLOW!) ***
// (cur_logical >= map->m_lblk + blocks) => true
if (cur_logical >= map->m_lblk + blocks)
break;
if (buffer_delay(bh) || buffer_unwritten(bh)) {
BUG_ON(bh->b_bdev != inode->i_sb->s_bdev);
if (buffer_delay(bh)) {
// *** cannot reach here! ***
clear_buffer_delay(bh);
bh->b_blocknr = pblock;
} else {
clear_buffer_unwritten(bh);
BUG_ON(bh->b_blocknr != pblock);
}
} else if (buffer_mapped(bh))
BUG_ON(bh->b_blocknr != pblock);
if (map->m_flags & EXT4_MAP_UNINIT)
set_buffer_uninit(bh);
cur_logical++;
pblock++;
} while ((bh = bh->b_this_page) != head);--------------------------------------------------------------------------------
Therefore, in order to fix this hangup easily, we change the following judgment only.(This change can prevent an overflow from occurring)
--------------------------------------------------------------------------------
if (cur_logical >= map->m_lblk + blocks)
| |
v v
if (cur_logical > map->m_lblk + blocks - 1)--------------------------------------------------------------------------------
NOTE: However, if we mount with nodelalloc option, we cannot experience this hangup while this script is running.
--- fs/ext4/inode.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
fs/ext4/inode.c +1 -1
@@ -2103,7 +2103,7 @@ static void mpage_put_bnr_to_bhs(struct mpage_da_data *mpd, }while ((bh = bh->b_this_page) != head); do { - if (cur_logical >= map->m_lblk + blocks) + if (cur_logical > map->m_lblk + blocks - 1) break; if (buffer_delay(bh) || buffer_unwritten(bh)) {
Signature · 1 line
1.5.5.6
Quoted 7 lines · Toshiyuki Okajima
From: Toshiyuki Okajima <[email protected]>
On linux-2.6.36-rc2, if we execute the following script, we can encounter the hangup of '
/bin/sync' command:
================================================================================
#!/bin/sh
Quoted 10 lines
echo -n "HANG UP TEST: "
/bin/ddif=/dev/zero of=/tmp/img bs=1k count=1 seek=1M 2>/dev/null/sbin/mkfs.ext4-Fq/tmp/img/bin/mount-o loop -t ext4/tmp/img/mnt/bin/ddif=/dev/zero of=/mnt/file bs=1 count=1 \
seek=$((16*1024*1024*1024*1024-4096)) 2>/dev/null/bin/sync/bin/umount/mnt
echo "DONE"
exit 0
On older kernels this testcase result in BUG_ON triggering at fs/ext4/mballoc.c:3229 ext4_mb_normalize_request()
Quoted 93 lines
================================================================================
We can see the following backtrace if we get the kdump when this hangup occurs:
================================================================================kthread()
=>bdi_writeback_thread()
=>wb_do_writeback()=> wb_writeback() => writeback_inodes_wb() => writeback_sb_inodes() => writeback_single_inode() => ext4_da_writepages() ---+ ^ infinite | | loop | +-------------+================================================================================
The reason why this hangup happens is described as follows:
1) We write the last extent block of the file whose size is the filesystem maximum size.
2) "BH_Delay" flag is set on thebuffer_headof its block.
3) - the member, "m_lblk" of structmpage_da_datais 4294967295 (UINT_MAX)
- the member, "m_len" of structmpage_da_datais 1
mpage_put_bnr_to_bhs()which is called viaext4_da_writepages()cannot clear "BH_Delay" flag of thebuffer_headbecause the type ofm_lblkisext4_lblk_tand thenm_lblk+m_lenis overflow.Therefore an infinite loop occurs because
ext4_da_writepages()cannot write the page (which corresponds to the block) since "BH_Delay" flag isn't cleared.
--------------------------------------------------------------------------------
static void mpage_put_bnr_to_bhs(structmpage_da_data*mpd,
structext4_map_blocks*map)
{
...int blocks = map->m_len; ... do { // cur_logical = 4294967295 // map->m_lblk = 4294967295 // blocks = 1 // *** map->m_lblk + blocks == 0 (OVERFLOW!) *** // (cur_logical >= map->m_lblk + blocks) => true if (cur_logical >= map->m_lblk + blocks) break; if (buffer_delay(bh) || buffer_unwritten(bh)) { BUG_ON(bh->b_bdev != inode->i_sb->s_bdev); if (buffer_delay(bh)) { // *** cannot reach here! *** clear_buffer_delay(bh); bh->b_blocknr = pblock; } else { clear_buffer_unwritten(bh); BUG_ON(bh->b_blocknr != pblock); } } else if (buffer_mapped(bh)) BUG_ON(bh->b_blocknr != pblock); if (map->m_flags & EXT4_MAP_UNINIT) set_buffer_uninit(bh); cur_logical++; pblock++; } while ((bh = bh->b_this_page) != head);--------------------------------------------------------------------------------
Therefore, in order to fix this hangup easily, we change the following judgment only.(This change can prevent an overflow from occurring)
--------------------------------------------------------------------------------if (cur_logical >= map->m_lblk + blocks) | | v v if (cur_logical > map->m_lblk + blocks - 1)--------------------------------------------------------------------------------
NOTE: However, if we mount with nodelalloc option, we cannot experience this hangup while this script is running.
Signed-off-by Toshiyuki Okajima <[email protected]>--- fs/ext4/inode.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/ext4/inode.c +1
-1@@ -2103,7 +2103,7 @@ static void mpage_put_bnr_to_bhs(struct mpage_da_data *mpd, }while ((bh = bh->b_this_page) != head); do { - if (cur_logical >= map->m_lblk + blocks) + if (cur_logical > map->m_lblk + blocks - 1) break; if (buffer_delay(bh) || buffer_unwritten(bh)) {
Quoted 19 lines · Dmitry Monakhov
Quoted 7 lines · Toshiyuki Okajima
From: Toshiyuki Okajima <[email protected]>
On linux-2.6.36-rc2, if we execute the following script, we can encounter the hangup of '
/bin/sync' command:
================================================================================
#!/bin/shQuoted 10 lines
echo -n "HANG UP TEST: "
/bin/ddif=/dev/zero of=/tmp/img bs=1k count=1 seek=1M 2>/dev/null/sbin/mkfs.ext4-Fq/tmp/img/bin/mount-o loop -t ext4/tmp/img/mnt/bin/ddif=/dev/zero of=/mnt/file bs=1 count=1 \
seek=$((16*1024*1024*1024*1024-4096)) 2>/dev/null/bin/sync/bin/umount/mnt
echo "DONE"
exit 0
Quoted 2 lines
On older kernels this testcase result in
BUG_ONtriggering atfs/ext4/mballoc.c:3229ext4_mb_normalize_request()
If anyone feels up to the task, extending xfstest 071
("# Exercise IO at large file offsets.") to work on generic kernels, it would have caught this earlier. It needs work, however, configure expected max offsets for various filesystems...
-Eric
Thanks, applied, with a slightly edited commit description.
- Ted
ext4: fix potential infinite loop in ext4_da_writepages()
From: Toshiyuki Okajima <[email protected]>
On linux-2.6.36-rc2, if we execute the following script, we can hang the system when the /bin/sync command is executed:
========================================================================
#!/bin/sh
echo -n "HANG UP TEST: "/bin/dd if=/dev/zero of=/tmp/img bs=1k count=1 seek=1M 2> /dev/null /sbin/mkfs.ext4 -Fq /tmp/img/bin/mount -o loop -t ext4 /tmp/img /mnt/bin/dd if=/dev/zero of=/mnt/file bs=1 count=1 \
seek=$((16*1024*1024*1024*1024-4096)) 2> /dev/null /bin/sync/bin/umount /mnt
echo "DONE"
exit 0
========================================================================
We can see the following backtrace if we get the kdump when this hangup occurs:
======================================================================kthread()
=> bdi_writeback_thread()
=> wb_do_writeback()
=> wb_writeback()
=> writeback_inodes_wb()
=> writeback_sb_inodes()
=> writeback_single_inode()
=> ext4_da_writepages() ---+
^ infinite |
| loop |
+-------------+======================================================================
The reason why this hangup happens is described as follows:
1) We write the last extent block of the file whose size is the filesystem maximum size.
2) "BH_Delay" flag is set on the buffer_head of its block.
3) - the member, "m_lblk" of struct mpage_da_data is 4294967295 (UINT_MAX)
- the member, "m_len" of struct mpage_da_data is 1
mpage_put_bnr_to_bhs() which is called via ext4_da_writepages() cannot clear "BH_Delay" flag of the buffer_head because the type of m_lblk is ext4_lblk_t and then m_lblk + m_len is overflow.
Therefore an infinite loop occurs because ext4_da_writepages() cannot write the page (which corresponds to the block) since "BH_Delay" flag isn't cleared.
----------------------------------------------------------------------
static void mpage_put_bnr_to_bhs(struct mpage_da_data *mpd,
struct ext4_map_blocks *map)
{
...
int blocks = map->m_len;
...
do {
// cur_logical = 4294967295
// map->m_lblk = 4294967295
// blocks = 1
// *** map->m_lblk + blocks == 0 (OVERFLOW!) ***
// (cur_logical >= map->m_lblk + blocks) => true
if (cur_logical >= map->m_lblk + blocks)
break;----------------------------------------------------------------------
NOTE: Mounting with the nodelalloc option will avoid this codepath, and thus, avoid this hang