nilfs2: fix data loss with mmap()
cover nilfs2: fix data loss with mmap()
Andrew,
Please apply the following bugfix and send it to upstream.
It fixes a regression of nilfs2's mmap which was brought by the commit 136e8770cd5d1fe3 "nilfs2: fix issue of nilfs_set_page_dirty() for page at EOF boundary". Andreas Rohner recently found that the commit was not perfect and causing reproducible silent data loss issue.
Thanks in advance,
Ryusuke Konishi
--
Andreas Rohner (1):
nilfs2: fix data loss with mmap()
fs/nilfs2/inode.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
PATCH 1/1 nilfs2: fix data loss with mmap()
From: Andreas Rohner <[email protected]>
This bug leads to reproducible silent data loss, despite the use of msync(), sync() and a clean unmount of the file system. It is easily reproducible with the following script:
----------------[BEGIN SCRIPT]-------------------- mkfs.nilfs2 -f /dev/sdb
mount /dev/sdb /mnt
dd if=/dev/zero bs=1M count=30 of=/mnt/testfile
umount /mnt
mount /dev/sdb /mnt
CHECKSUM_BEFORE="$(md5sum /mnt/testfile)"
/root/mmaptest/mmaptest /mnt/testfile 30 10 5
sync
CHECKSUM_AFTER="$(md5sum /mnt/testfile)"
umount /mnt
mount /dev/sdb /mnt
CHECKSUM_AFTER_REMOUNT="$(md5sum /mnt/testfile)"
umount /mnt
echo "BEFORE MMAP:\t$CHECKSUM_BEFORE"
echo "AFTER MMAP:\t$CHECKSUM_AFTER"
echo "AFTER REMOUNT:\t$CHECKSUM_AFTER_REMOUNT"
----------------[END SCRIPT]--------------------
The mmaptest tool looks something like this (very simplified, with error checking removed):
----------------[BEGIN mmaptest]-------------------- data = mmap(NULL, file_size - file_offset, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, file_offset);
for (i = 0; i < write_count; ++i) {
memcpy(data + i * 4096, buf, sizeof(buf));
msync(data, file_size - file_offset, MS_SYNC))
}----------------[END mmaptest]--------------------
The output of the script looks something like this:
BEFORE MMAP: 281ed1d5ae50e8419f9b978aab16de83 /mnt/testfile AFTER MMAP: 6604a1c31f10780331a6850371b3a313 /mnt/testfile AFTER REMOUNT: 281ed1d5ae50e8419f9b978aab16de83 /mnt/testfile
So it is clear, that the changes done using mmap() do not survive a remount. This can be reproduced a 100% of the time. The problem was introduced with the following commit:
136e877 nilfs2: fix issue of nilfs_set_page_dirty() for page at EOF boundary
If the page was read with mpage_readpage() or mpage_readpages() for example, then it has no buffers attached to it. In that case page_has_buffers(page) in nilfs_set_page_dirty() will be false. Therefore nilfs_set_file_dirty() is never called and the pages are never collected and never written to disk.
This patch fixes the problem by also calling nilfs_set_file_dirty() if the page has no buffers attached to it.
--- fs/nilfs2/inode.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
fs/nilfs2/inode.c +5 -1
@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)static int nilfs_set_page_dirty (struct page *page) { +struct inode *inode = page->mapping->host;int ret =__set_page_dirty_nobuffers (page);if (page_has_buffers (page)) { -struct inode *inode = page->mapping->host;unsigned nr_dirty =0 ;struct buffer_head *bh, *head;
@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)if (nr_dirty)nilfs_set_file_dirty (inode, nr_dirty); + }else if (ret) { +unsigned nr_dirty =1 << (PAGE_SHIFT - inode->i_blkbits); + +nilfs_set_file_dirty (inode, nr_dirty); }return ret; }
Signature · 6 lines
1.7.9.4 -- To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 23 lines · On Thu, 18 Sep 2014 23:56:25 +0900 Ryusuke Konishi…
From: Andreas Rohner <[email protected]>
This bug leads to reproducible silent data loss, despite the use of
msync(),sync()and a clean unmount of the file system. It is easily reproducible with the following script:...
fs/nilfs2/inode.c +1
-1@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)static int nilfs_set_page_dirty (struct page *page) { +struct inode *inode = page->mapping->host;int ret =__set_page_dirty_nobuffers (page);if (page_has_buffers (page)) { -struct inode *inode = page->mapping->host;unsigned nr_dirty =0 ;struct buffer_head *bh, *head;
fs/nilfs2/inode.c static int nilfs_set_page_dirty(struct page *page) · line 249
Quoted 5 lines
fs/nilfs2/inode.c +2
-0@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)if (nr_dirty)nilfs_set_file_dirty (inode, nr_dirty); + }else if (ret) { +unsigned nr_dirty =1 << (PAGE_SHIFT - inode->i_blkbits);
It's quite cosmetic, but it is conventional to use PAGE_CACHE_SHIFT here. --
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 24 lines · Andrew Morton
Quoted 22 lines · On Thu, 18 Sep 2014 23:56:25 +0900 Ryusuke Konishi…
From: Andreas Rohner <[email protected]>
This bug leads to reproducible silent data loss, despite the use of
msync(),sync()and a clean unmount of the file system. It is easily reproducible with the following script:...
fs/nilfs2/inode.c +1
-1@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)static int nilfs_set_page_dirty (struct page *page) { +struct inode *inode = page->mapping->host;int ret =__set_page_dirty_nobuffers (page);if (page_has_buffers (page)) { -struct inode *inode = page->mapping->host;unsigned nr_dirty =0 ;struct buffer_head *bh, *head;
Quoted 8 lines
fs/nilfs2/inode.c static int nilfs_set_page_dirty(struct page *page) · line 249
Quoted 6 lines
fs/nilfs2/inode.c +2
-0@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)if (nr_dirty)nilfs_set_file_dirty (inode, nr_dirty); + }else if (ret) { +unsigned nr_dirty =1 << (PAGE_SHIFT - inode->i_blkbits);It's quite cosmetic, but it is conventional to use
PAGE_CACHE_SHIFThere.
Agreed.
Thanks,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20140919.084107.1306407610758412217.konishi.ryusuke-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org
PATCH v2 1/1 nilfs2: fix data loss with mmap()
This bug leads to reproducible silent data loss, despite the use of msync(), sync() and a clean unmount of the file system. It is easily reproducible with the following script:
----------------[BEGIN SCRIPT]-------------------- mkfs.nilfs2 -f /dev/sdb
mount /dev/sdb /mnt
dd if=/dev/zero bs=1M count=30 of=/mnt/testfile
umount /mnt
mount /dev/sdb /mnt
CHECKSUM_BEFORE="$(md5sum /mnt/testfile)"
/root/mmaptest/mmaptest /mnt/testfile 30 10 5
sync
CHECKSUM_AFTER="$(md5sum /mnt/testfile)"
umount /mnt
mount /dev/sdb /mnt
CHECKSUM_AFTER_REMOUNT="$(md5sum /mnt/testfile)"
umount /mnt
echo "BEFORE MMAP:\t$CHECKSUM_BEFORE"
echo "AFTER MMAP:\t$CHECKSUM_AFTER"
echo "AFTER REMOUNT:\t$CHECKSUM_AFTER_REMOUNT"
----------------[END SCRIPT]--------------------
The mmaptest tool looks something like this (very simplified, with error checking removed):
----------------[BEGIN mmaptest]-------------------- data = mmap(NULL, file_size - file_offset, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, file_offset);
for (i = 0; i < write_count; ++i) {
memcpy(data + i * 4096, buf, sizeof(buf));
msync(data, file_size - file_offset, MS_SYNC))
}----------------[END mmaptest]--------------------
The output of the script looks something like this:
BEFORE MMAP: 281ed1d5ae50e8419f9b978aab16de83 /mnt/testfile AFTER MMAP: 6604a1c31f10780331a6850371b3a313 /mnt/testfile AFTER REMOUNT: 281ed1d5ae50e8419f9b978aab16de83 /mnt/testfile
So it is clear, that the changes done using mmap() do not survive a remount. This can be reproduced a 100% of the time. The problem was introduced with the following commit:
136e877 nilfs2: fix issue of nilfs_set_page_dirty() for page at EOF boundary
If the page was read with mpage_readpage() or mpage_readpages() for example, then it has no buffers attached to it. In that case page_has_buffers(page) in nilfs_set_page_dirty() will be false. Therefore nilfs_set_file_dirty() is never called and the pages are never collected and never written to disk.
This patch fixes the problem by also calling nilfs_set_file_dirty() if the page has no buffers attached to it.
--- fs/nilfs2/inode.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
fs/nilfs2/inode.c +5 -1
@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)static int nilfs_set_page_dirty (struct page *page) { +struct inode *inode = page->mapping->host;int ret =__set_page_dirty_nobuffers (page);if (page_has_buffers (page)) { -struct inode *inode = page->mapping->host;unsigned nr_dirty =0 ;struct buffer_head *bh, *head;
@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)if (nr_dirty)nilfs_set_file_dirty (inode, nr_dirty); + }else if (ret) { +unsigned nr_dirty =1 << (PAGE_CACHE_SHIFT - inode->i_blkbits); + +nilfs_set_file_dirty (inode, nr_dirty); }return ret; }
Signature · 6 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html