Skip to contentSkip to conversation

nilfs2: fix data loss with mmap()

PATCH New Ryusuke Konishi 3 participants 5 messages Sep 18, 2014 – Sep 19, 2014 1 message not in the archive NILFS2 filesystem

lore
Share & tools
lore
#0 lore raw

cover nilfs2: fix data loss with mmap()

Andrew,

Please apply the following bugfix and send it to upstream.

It fixes a regression of nilfs2's mmap which was brought by the commit 136e8770cd5d1fe3 "nilfs2: fix issue of nilfs_set_page_dirty() for page at EOF boundary". Andreas Rohner recently found that the commit was not perfect and causing reproducible silent data loss issue.

Thanks in advance,

Ryusuke Konishi
--
Andreas Rohner (1):
nilfs2: fix data loss with mmap()

 fs/nilfs2/inode.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
#1 lore raw

PATCH 1/1 nilfs2: fix data loss with mmap()

From: Andreas Rohner <[email protected]>

This bug leads to reproducible silent data loss, despite the use of msync(), sync() and a clean unmount of the file system. It is easily reproducible with the following script:

----------------[BEGIN SCRIPT]-------------------- mkfs.nilfs2 -f /dev/sdb
mount /dev/sdb /mnt

dd if=/dev/zero bs=1M count=30 of=/mnt/testfile

umount /mnt
mount /dev/sdb /mnt
CHECKSUM_BEFORE="$(md5sum /mnt/testfile)"

/root/mmaptest/mmaptest /mnt/testfile 30 10 5

sync
CHECKSUM_AFTER="$(md5sum /mnt/testfile)"
umount /mnt
mount /dev/sdb /mnt
CHECKSUM_AFTER_REMOUNT="$(md5sum /mnt/testfile)"
umount /mnt

echo "BEFORE MMAP:\t$CHECKSUM_BEFORE"
echo "AFTER MMAP:\t$CHECKSUM_AFTER"
echo "AFTER REMOUNT:\t$CHECKSUM_AFTER_REMOUNT"
----------------[END SCRIPT]--------------------

The mmaptest tool looks something like this (very simplified, with error checking removed):

----------------[BEGIN mmaptest]-------------------- data = mmap(NULL, file_size - file_offset, PROT_READ | PROT_WRITE,

		MAP_SHARED, fd, file_offset);
for (i = 0; i < write_count; ++i) {
	memcpy(data + i * 4096, buf, sizeof(buf));
	msync(data, file_size - file_offset, MS_SYNC))
}

----------------[END mmaptest]--------------------

The output of the script looks something like this:

BEFORE MMAP:    281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile
AFTER MMAP:     6604a1c31f10780331a6850371b3a313  /mnt/testfile
AFTER REMOUNT:  281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile

So it is clear, that the changes done using mmap() do not survive a remount. This can be reproduced a 100% of the time. The problem was introduced with the following commit:

136e877 nilfs2: fix issue of nilfs_set_page_dirty() for page at EOF boundary

If the page was read with mpage_readpage() or mpage_readpages() for example, then it has no buffers attached to it. In that case page_has_buffers(page) in nilfs_set_page_dirty() will be false. Therefore nilfs_set_file_dirty() is never called and the pages are never collected and never written to disk.

This patch fixes the problem by also calling nilfs_set_file_dirty() if the page has no buffers attached to it.

---
 fs/nilfs2/inode.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
fs/nilfs2/inode.c +5 -1
index 6252b17..9e3c525 100644
@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)
 
 static int nilfs_set_page_dirty(struct page *page)
 {
+	struct inode *inode = page->mapping->host;
 	int ret = __set_page_dirty_nobuffers(page);
 
 	if (page_has_buffers(page)) {
-		struct inode *inode = page->mapping->host;
 		unsigned nr_dirty = 0;
 		struct buffer_head *bh, *head;
 
@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)
 
 		if (nr_dirty)
 			nilfs_set_file_dirty(inode, nr_dirty);
+	} else if (ret) {
+		unsigned nr_dirty = 1 << (PAGE_SHIFT - inode->i_blkbits);
+
+		nilfs_set_file_dirty(inode, nr_dirty);
 	}
 	return ret;
 }
Signature · 6 lines
1.7.9.4

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#2 lore raw
Quoted 23 lines · On Thu, 18 Sep 2014 23:56:25 +0900 Ryusuke Konishi…

From: Andreas Rohner <[email protected]>

This bug leads to reproducible silent data loss, despite the use of msync(), sync() and a clean unmount of the file system. It is easily reproducible with the following script:

...

fs/nilfs2/inode.c +1 -1
@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)
 
 static int nilfs_set_page_dirty(struct page *page)
 {
+	struct inode *inode = page->mapping->host;
 	int ret = __set_page_dirty_nobuffers(page);
 
 	if (page_has_buffers(page)) {
-		struct inode *inode = page->mapping->host;
 		unsigned nr_dirty = 0;
 		struct buffer_head *bh, *head;
 

fs/nilfs2/inode.c static int nilfs_set_page_dirty(struct page *page) · line 249

Quoted 5 lines
fs/nilfs2/inode.c +2 -0
@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)
 
 		if (nr_dirty)
 			nilfs_set_file_dirty(inode, nr_dirty);
+	} else if (ret) {
+		unsigned nr_dirty = 1 << (PAGE_SHIFT - inode->i_blkbits);

It's quite cosmetic, but it is conventional to use PAGE_CACHE_SHIFT here. --
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 24 lines · Andrew Morton
Quoted 22 lines · On Thu, 18 Sep 2014 23:56:25 +0900 Ryusuke Konishi…

From: Andreas Rohner <[email protected]>

This bug leads to reproducible silent data loss, despite the use of msync(), sync() and a clean unmount of the file system. It is easily reproducible with the following script:

...

fs/nilfs2/inode.c +1 -1
@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)
 
 static int nilfs_set_page_dirty(struct page *page)
 {
+	struct inode *inode = page->mapping->host;
 	int ret = __set_page_dirty_nobuffers(page);
 
 	if (page_has_buffers(page)) {
-		struct inode *inode = page->mapping->host;
 		unsigned nr_dirty = 0;
 		struct buffer_head *bh, *head;
 
Quoted 8 lines

fs/nilfs2/inode.c static int nilfs_set_page_dirty(struct page *page) · line 249

Quoted 6 lines
fs/nilfs2/inode.c +2 -0
@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)
 
 		if (nr_dirty)
 			nilfs_set_file_dirty(inode, nr_dirty);
+	} else if (ret) {
+		unsigned nr_dirty = 1 << (PAGE_SHIFT - inode->i_blkbits);

It's quite cosmetic, but it is conventional to use PAGE_CACHE_SHIFT here.

Agreed.

Thanks,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20140919.084107.1306407610758412217.konishi.ryusuke-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org

#5 lore raw

PATCH v2 1/1 nilfs2: fix data loss with mmap()

This bug leads to reproducible silent data loss, despite the use of msync(), sync() and a clean unmount of the file system. It is easily reproducible with the following script:

----------------[BEGIN SCRIPT]-------------------- mkfs.nilfs2 -f /dev/sdb
mount /dev/sdb /mnt

dd if=/dev/zero bs=1M count=30 of=/mnt/testfile

umount /mnt
mount /dev/sdb /mnt
CHECKSUM_BEFORE="$(md5sum /mnt/testfile)"

/root/mmaptest/mmaptest /mnt/testfile 30 10 5

sync
CHECKSUM_AFTER="$(md5sum /mnt/testfile)"
umount /mnt
mount /dev/sdb /mnt
CHECKSUM_AFTER_REMOUNT="$(md5sum /mnt/testfile)"
umount /mnt

echo "BEFORE MMAP:\t$CHECKSUM_BEFORE"
echo "AFTER MMAP:\t$CHECKSUM_AFTER"
echo "AFTER REMOUNT:\t$CHECKSUM_AFTER_REMOUNT"
----------------[END SCRIPT]--------------------

The mmaptest tool looks something like this (very simplified, with error checking removed):

----------------[BEGIN mmaptest]-------------------- data = mmap(NULL, file_size - file_offset, PROT_READ | PROT_WRITE,

		MAP_SHARED, fd, file_offset);
for (i = 0; i < write_count; ++i) {
	memcpy(data + i * 4096, buf, sizeof(buf));
	msync(data, file_size - file_offset, MS_SYNC))
}

----------------[END mmaptest]--------------------

The output of the script looks something like this:

BEFORE MMAP:    281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile
AFTER MMAP:     6604a1c31f10780331a6850371b3a313  /mnt/testfile
AFTER REMOUNT:  281ed1d5ae50e8419f9b978aab16de83  /mnt/testfile

So it is clear, that the changes done using mmap() do not survive a remount. This can be reproduced a 100% of the time. The problem was introduced with the following commit:

136e877 nilfs2: fix issue of nilfs_set_page_dirty() for page at EOF boundary

If the page was read with mpage_readpage() or mpage_readpages() for example, then it has no buffers attached to it. In that case page_has_buffers(page) in nilfs_set_page_dirty() will be false. Therefore nilfs_set_file_dirty() is never called and the pages are never collected and never written to disk.

This patch fixes the problem by also calling nilfs_set_file_dirty() if the page has no buffers attached to it.

---
 fs/nilfs2/inode.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
fs/nilfs2/inode.c +5 -1
index 6252b17..6b0a241 100644
@@ -219,10 +219,10 @@ static int nilfs_writepage(struct page *page, struct writeback_control *wbc)
 
 static int nilfs_set_page_dirty(struct page *page)
 {
+	struct inode *inode = page->mapping->host;
 	int ret = __set_page_dirty_nobuffers(page);
 
 	if (page_has_buffers(page)) {
-		struct inode *inode = page->mapping->host;
 		unsigned nr_dirty = 0;
 		struct buffer_head *bh, *head;
 
@@ -245,6 +245,10 @@ static int nilfs_set_page_dirty(struct page *page)
 
 		if (nr_dirty)
 			nilfs_set_file_dirty(inode, nr_dirty);
+	} else if (ret) {
+		unsigned nr_dirty = 1 << (PAGE_CACHE_SHIFT - inode->i_blkbits);
+
+		nilfs_set_file_dirty(inode, nr_dirty);
 	}
 	return ret;
 }
Signature · 6 lines
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html