Skip to contentSkip to conversation

fat: add fat_fallocate operation

PATCH Superseded Namjae Jeon 2 participants 6 messages Dec 25, 2013 – Feb 14, 2014

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 2/6 fat: add fat_fallocate operation

From: Namjae Jeon <[email protected]>

Implement preallocation via the fallocate syscall on VFAT partitions. This patch is based on an earlier patch of the same name which had some issues detailed below and did not get accepted.
Refer https://lkml.org/lkml/2007/12/22/130.

a) The preallocated space was not persistent when the FALLOC_FL_KEEP_SIZE flag was set. It will deallocate cluster at evict time.

b) There was no need to zero out the clusters when the flag was set Instead of doing an expanding truncate, just allocate clusters and add them to the fat chain. This reduces preallocation time.

Compatibility with windows:
There are no issues when FALLOC_FL_KEEP_SIZE is not set because it just does an expanding truncate. Thus reading from the preallocated area on windows returns null until data is written to it.

When a file with preallocated area using the FALLOC_FL_KEEP_SIZE was written to on windows, the windows driver freed-up the preallocated clusters and allocated new clusters for the new data. The freed up clusters gets reflected in the free space available for the partition which can be seen from the Volume properties.

The windows chkdsk tool also does not report any errors on a disk containing files with preallocated space.

And there is also no issue using linux fat fsck. because discard preallocated clusters at repair time.

---
 fs/fat/file.c  |   74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/fat/inode.c |   19 +++++++++++++++
 2 files changed, 93 insertions(+)
fs/fat/file.c +74 -0
index 0bf0d28..79db8b6 100644
@@ -17,8 +17,12 @@
 #include <linux/blkdev.h>
 #include <linux/fsnotify.h>
 #include <linux/security.h>
+#include <linux/falloc.h>
 #include "fat.h"
 
+static long fat_fallocate(struct file *file, int mode,
+			  loff_t offset, loff_t len);
+
 static int fat_ioctl_get_attributes(struct inode *inode, u32 __user *user_attr)
 {
 	u32 attr;
@@ -182,6 +186,7 @@ const struct file_operations fat_file_operations = {
 #endif
 	.fsync		= fat_file_fsync,
 	.splice_read	= generic_file_splice_read,
+	.fallocate	= fat_fallocate,
 };
 
 static int fat_cont_expand(struct inode *inode, loff_t size)
@@ -220,6 +225,75 @@ out:
 	return err;
 }
 
+/*
+ * Preallocate space for a file. This implements fat's fallocate file
+ * operation, which gets called from sys_fallocate system call. User
+ * space requests len bytes at offset. If FALLOC_FL_KEEP_SIZE is set
+ * we just allocate clusters without zeroing them out. Otherwise we
+ * allocate and zero out clusters via an expanding truncate.
+ */
+static long fat_fallocate(struct file *file, int mode,
+			  loff_t offset, loff_t len)
+{
+	int cluster;
+	int nr_cluster; /* Number of clusters to be allocated */
+	loff_t mm_bytes; /* Number of bytes to be allocated for file */
+	struct inode *inode = file->f_mapping->host;
+	struct super_block *sb = inode->i_sb;
+	struct msdos_sb_info *sbi = MSDOS_SB(sb);
+	int err = 0;
+
+	/* No support for hole punch or other fallocate flags. */
+	if (mode & ~FALLOC_FL_KEEP_SIZE)
+		return -EOPNOTSUPP;
+
+	/* No support for dir */
+	if (!S_ISREG(inode->i_mode))
+		return -EOPNOTSUPP;
+
+	mutex_lock(&inode->i_mutex);
+	if ((offset + len) <= MSDOS_I(inode)->i_disksize)
+		goto error;
+
+	err = inode_newsize_ok(inode, (len + offset));
+	if (err)
+		goto error;
+
+	if (mode & FALLOC_FL_KEEP_SIZE) {
+		/* First compute the number of clusters to be allocated */
+		mm_bytes = offset + len - round_up(MSDOS_I(inode)->mmu_private,
+			sbi->cluster_size);
+		nr_cluster = (mm_bytes + (sbi->cluster_size - 1)) >>
+			sbi->cluster_bits;
+
+		/* Start the allocation.We are not zeroing out the clusters */
+		while (nr_cluster-- > 0) {
+			err = fat_alloc_clusters(inode, &cluster, 1);
+			if (err) {
+				fat_msg(sb, KERN_ERR,
+					"fat_fallocate(): fat_alloc_clusters() error");
+				goto error;
+			}
+			err = fat_chain_add(inode, cluster, 1);
+			if (err) {
+				fat_free_clusters(inode, cluster);
+				goto error;
+			}
+			MSDOS_I(inode)->i_disksize += sbi->cluster_size;
+		}
+	} else {
+		/* This is just an expanding truncate */
+		err = fat_cont_expand(inode, (offset + len));
+		if (err)
+			fat_msg(sb, KERN_ERR,
+				"fat_fallocate(): fat_cont_expand() error");
+	}
+
+error:
+	mutex_unlock(&inode->i_mutex);
+	return err;
+}
+
 /* Free all clusters after the skip'th cluster. */
 static int fat_free(struct inode *inode, int skip)
 {
fs/fat/inode.c +19 -0
index 7e5531a..2674c57 100644
@@ -497,6 +497,25 @@ static void fat_evict_inode(struct inode *inode)
 	if (!inode->i_nlink) {
 		inode->i_size = 0;
 		fat_truncate_blocks(inode, 0);
+	} else {
+		/* Release unwritten fallocated blocks on inode eviction. */
+		if (MSDOS_I(inode)->mmu_private < MSDOS_I(inode)->i_disksize) {
+			int err;
+			fat_truncate_blocks(inode, MSDOS_I(inode)->mmu_private);
+			/* Fallocate results in updating the i_start/iogstart
+			 * for the zero byte file. So, make it return to
+			 * original state during evict and commit it
+			 * synchrnously to avoid any corruption on the next
+			 * access to the cluster chain for the file.
+			 */
+			err = fat_sync_inode(inode);
+			if (err) {
+				fat_msg(inode->i_sb, KERN_WARNING, "Failed to "
+				"update on disk inode for unused fallocated "
+				"blocks, inode could be corrupted. Please run "
+				"fsck");
+			}
+		}
 	}
 	invalidate_inode_buffers(inode);
 	clear_inode(inode);
Signature · 1 line
1.7.9.5
maintainer
#1 lore raw

Sorry for long delay.

fs/fat/file.c out: · line 265

Quoted 4 lines · Namjae Jeon
fs/fat/file.c +4 -0
+	if (mode & FALLOC_FL_KEEP_SIZE) {
+		/* First compute the number of clusters to be allocated */
+		mm_bytes = offset + len - round_up(MSDOS_I(inode)->mmu_private,
+			sbi->cluster_size);

This should use ->i_disksize?

[...]

fs/fat/inode.c static void fat_evict_inode(struct inode *inode) · line 511

Quoted 11 lines
fs/fat/inode.c +11 -0
+		/* Release unwritten fallocated blocks on inode eviction. */
+		if (MSDOS_I(inode)->mmu_private < MSDOS_I(inode)->i_disksize) {
+			int err;
+			fat_truncate_blocks(inode, MSDOS_I(inode)->mmu_private);
+			/* Fallocate results in updating the i_start/iogstart
+			 * for the zero byte file. So, make it return to
+			 * original state during evict and commit it
+			 * synchrnously to avoid any corruption on the next
+			 * access to the cluster chain for the file.
+			 */
+			err = fat_sync_inode(inode);

Ah, good catch. We have to update i_size. I was forgetting about this. Well, sync inode unconditionally would not be good. Maybe, we better to use __fat_write_inode() with inode_needs_sync() or such.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#2 lore raw

2014-02-03, OGAWA Hirofumi <[email protected]>:

Quoted 10 lines

Sorry for long delay.

fs/fat/file.c out: · line 265

Quoted 4 lines · Namjae Jeon
fs/fat/file.c +4 -0
+	if (mode & FALLOC_FL_KEEP_SIZE) {
+		/* First compute the number of clusters to be allocated */
+		mm_bytes = offset + len - round_up(MSDOS_I(inode)->mmu_private,
+			sbi->cluster_size);

Hi OGAWA.

Quoted 1 line

This should use ->i_disksize?

Right, I will fix it.

Quoted 18 lines

[...]

fs/fat/inode.c static void fat_evict_inode(struct inode *inode) · line 511

Quoted 11 lines
fs/fat/inode.c +11 -0
+		/* Release unwritten fallocated blocks on inode eviction. */
+		if (MSDOS_I(inode)->mmu_private < MSDOS_I(inode)->i_disksize) {
+			int err;
+			fat_truncate_blocks(inode, MSDOS_I(inode)->mmu_private);
+			/* Fallocate results in updating the i_start/iogstart
+			 * for the zero byte file. So, make it return to
+			 * original state during evict and commit it
+			 * synchrnously to avoid any corruption on the next
+			 * access to the cluster chain for the file.
+			 */
+			err = fat_sync_inode(inode);

Ah, good catch. We have to update i_size. I was forgetting about this. Well, sync inode unconditionally would not be good. Maybe, we better to use __fat_write_inode() with inode_needs_sync() or such.

Okay, I will change it.

Thanks.

Quoted 3 lines

--
OGAWA Hirofumi <[email protected]>

#3 lore raw
Quoted 17 lines
Quoted 17 lines

[...]

fs/fat/inode.c static void fat_evict_inode(struct inode *inode) · line 511

Quoted 11 lines
fs/fat/inode.c +11 -0
+		/* Release unwritten fallocated blocks on inode eviction. */
+		if (MSDOS_I(inode)->mmu_private < MSDOS_I(inode)->i_disksize) {
+			int err;
+			fat_truncate_blocks(inode, MSDOS_I(inode)->mmu_private);
+			/* Fallocate results in updating the i_start/iogstart
+			 * for the zero byte file. So, make it return to
+			 * original state during evict and commit it
+			 * synchrnously to avoid any corruption on the next
+			 * access to the cluster chain for the file.
+			 */
+			err = fat_sync_inode(inode);

Ah, good catch. We have to update i_size. I was forgetting about this. Well, sync inode unconditionally would not be good. Maybe, we better to use __fat_write_inode() with inode_needs_sync() or such.

Quoted 1 line

Okay, I will change it.

Hi OGAWA

When I checked more, we should wait till inode is sync. Because in the eviction it will leave the inode/buffers being marked dirty.
Not waiting for it get sync over here. It will leave cluster chain corrupted when remounting.
It mean we cannot use __fat_write_inode with inode_needs_sync() conditionally.

Thanks.

Quoted 6 lines

Thanks.

Quoted 3 lines

--
OGAWA Hirofumi <[email protected]>

maintainer
#4 lore raw
Quoted 17 lines · Namjae Jeon
Quoted 17 lines
Quoted 17 lines

[...]

fs/fat/inode.c static void fat_evict_inode(struct inode *inode) · line 511

Quoted 11 lines
fs/fat/inode.c +11 -0
+		/* Release unwritten fallocated blocks on inode eviction. */
+		if (MSDOS_I(inode)->mmu_private < MSDOS_I(inode)->i_disksize) {
+			int err;
+			fat_truncate_blocks(inode, MSDOS_I(inode)->mmu_private);
+			/* Fallocate results in updating the i_start/iogstart
+			 * for the zero byte file. So, make it return to
+			 * original state during evict and commit it
+			 * synchrnously to avoid any corruption on the next
+			 * access to the cluster chain for the file.
+			 */
+			err = fat_sync_inode(inode);

Ah, good catch. We have to update i_size. I was forgetting about this. Well, sync inode unconditionally would not be good. Maybe, we better to use __fat_write_inode() with inode_needs_sync() or such.

Quoted 8 lines
Quoted 1 line

Okay, I will change it.

Hi OGAWA

When I checked more, we should wait till inode is sync. Because in the eviction it will leave the inode/buffers being marked dirty.
Not waiting for it get sync over here. It will leave cluster chain corrupted when remounting.
It mean we cannot use __fat_write_inode with inode_needs_sync() conditionally.

Yeah, this situation bothers us. However, the inode is not marked as dirty after I_FREEING. And in fatfs case, all related dirty buffers should goes into blockdev inode buffers (i.e. metadata only), right?

So, I thought sync is not necessary.

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#5 lore raw

2014-02-14 16:30 GMT+09:00, OGAWA Hirofumi <[email protected]>:

Quoted 29 lines
Quoted 18 lines · Namjae Jeon
Quoted 17 lines
Quoted 17 lines

[...]

fs/fat/inode.c static void fat_evict_inode(struct inode *inode) · line 511

Quoted 11 lines
fs/fat/inode.c +11 -0
+		/* Release unwritten fallocated blocks on inode eviction. */
+		if (MSDOS_I(inode)->mmu_private < MSDOS_I(inode)->i_disksize) {
+			int err;
+			fat_truncate_blocks(inode, MSDOS_I(inode)->mmu_private);
+			/* Fallocate results in updating the i_start/iogstart
+			 * for the zero byte file. So, make it return to
+			 * original state during evict and commit it
+			 * synchrnously to avoid any corruption on the next
+			 * access to the cluster chain for the file.
+			 */
+			err = fat_sync_inode(inode);

Ah, good catch. We have to update i_size. I was forgetting about this. Well, sync inode unconditionally would not be good. Maybe, we better to use __fat_write_inode() with inode_needs_sync() or such.

Quoted 1 line

Okay, I will change it.

Quoted 8 lines

Hi OGAWA

When I checked more, we should wait till inode is sync. Because in the eviction it will leave the inode/buffers being marked dirty. Not waiting for it get sync over here. It will leave cluster chain corrupted when remounting.
It mean we cannot use __fat_write_inode with inode_needs_sync() conditionally.

Quoted 3 lines

Yeah, this situation bothers us. However, the inode is not marked as dirty after I_FREEING. And in fatfs case, all related dirty buffers should goes into blockdev inode buffers (i.e. metadata only), right?

Right.

Quoted 2 lines

So, I thought sync is not necessary.

Yes, I will add it as you pointed.

Thanks for review!

Quoted 5 lines

Thanks.
--
OGAWA Hirofumi <[email protected]>