Skip to contentSkip to conversation

truncate: introduce new sequence

PATCH In review Nick Piggin 3 participants 5 messages Mar 2, 2010 – Mar 17, 2010

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

PATCH 1/2 truncate: introduce new sequence

Al,

Ping again about this. I'm just sending you the core patch plus conversion of simple filesystems and I will send the fs specific patches we already have to their respective maintainers after this gets upstream.

The core patch after your earlier review is now rather simple and shouldn't fundamentally change any existing paths, just adds new helpers really.

It looks like we should be able to get rid of ->truncate() and the legacy helpers rather quickly after this goes upstream if that's what you were worried about.

(The patch is against your #untested branch but there didn't seem to be any conflicts versus mainline.)

Thanks,
Nick
--

truncate: introduce new sequence

Introduce a new truncate calling sequence into fs/mm subsystems. Rather than setattr > vmtruncate > truncate, have filesystems call their truncate sequence from ->setattr if filesystem specific operations are required. vmtruncate is deprecated, and truncate_pagecache and inode_newsize_ok helpers introduced previously should be used.

simple_setattr is introduced for simple in-ram filesystems to implement the new truncate sequence. Eventually all filesystems should be converted to implement a setattr, and the default code in notify_change should go away.

simple_setsize is also introduced to perform just the ATTR_SIZE portion of simple_setattr (ie. changing i_size and trimming pagecache).

To implement the new truncate sequence:
- filesystem specific manipulations (eg freeing blocks) must be done in the setattr method rather than ->truncate.
- vmtruncate can not be used by core code to trim blocks past i_size in the event of write failure after allocation, so this must be performed in the fs code.
- convert usage of helpers block_write_begin, nobh_write_begin, cont_write_begin, and *blockdev_direct_IO* to use _newtrunc postfixed variants. These avoid calling vmtruncate to trim blocks (see previous).
- inode_setattr should not be used. generic_setattr is a new function to be used to copy simple attributes into the generic inode.
- make use of the better opportunity to handle errors with the new sequence.

Big problem with the previous calling sequence: the filesystem is not called until i_size has already changed. This means it is not allowed to fail the call, and also it does not know what the previous i_size was. Also, generic code calling vmtruncate to truncate allocated blocks in case of error had no good way to return a meaningful error (or, for example, atomically handle block deallocation).

Cc Dmitry Monakhov <[email protected]>
Cc Christoph Hellwig <[email protected]>
---
 Documentation/filesystems/vfs.txt |    7 +-
 fs/attr.c                         |   50 ++++++++++++---
 fs/buffer.c                       |  123 ++++++++++++++++++++++++++++++--------
 fs/direct-io.c                    |   61 ++++++++++++------
 fs/libfs.c                        |   83 +++++++++++++++++++++++++
 include/linux/buffer_head.h       |    9 ++
 include/linux/fs.h                |   27 ++++++++
 mm/truncate.c                     |   10 +--
 8 files changed, 307 insertions(+), 63 deletions(-)
Documentation/filesystems/vfs.txt +6 -1
Index: linux-2.6/Documentation/filesystems/vfs.txt
===================================================================
@@ -401,11 +401,16 @@ otherwise noted.
   	started might not be in the page cache at the end of the
   	walk).
 
-  truncate: called by the VFS to change the size of a file.  The
+  truncate: Deprecated. This will not be called if ->setsize is defined.
+	Called by the VFS to change the size of a file.  The
  	i_size field of the inode is set to the desired size by the
  	VFS before this method is called.  This method is called by
  	the truncate(2) system call and related functionality.
 
+	Note: ->truncate and vmtruncate are deprecated. Do not add new
+	instances/calls of these. Filesystems should be converted to do their
+	truncate sequence via ->setattr().
+
   permission: called by the VFS to check for access rights on a POSIX-like
   	filesystem.
 
fs/attr.c +40 -10

77 lines not on this page · raw message

fs/buffer.c +98 -25

183 lines not on this page · raw message

fs/direct-io.c +40 -21

88 lines not on this page · raw message

fs/libfs.c +83 -0

95 lines not on this page · raw message

include/linux/fs.h +26 -1
Index: linux-2.6/include/linux/fs.h
===================================================================
@@ -2244,6 +2244,10 @@ static inline int xip_truncate_page(stru
 #endif
 
 #ifdef CONFIG_BLOCK
+ssize_t __blockdev_direct_IO_newtrunc(int rw, struct kiocb *iocb, struct inode *inode,
+	struct block_device *bdev, const struct iovec *iov, loff_t offset,
+	unsigned long nr_segs, get_block_t get_block, dio_iodone_t end_io,
+	int lock_type);
 ssize_t __blockdev_direct_IO(int rw, struct kiocb *iocb, struct inode *inode,
 	struct block_device *bdev, const struct iovec *iov, loff_t offset,
 	unsigned long nr_segs, get_block_t get_block, dio_iodone_t end_io,
@@ -2257,6 +2261,24 @@ enum {
 	DIO_SKIP_HOLES	= 0x02,
 };
 
+static inline ssize_t blockdev_direct_IO_newtrunc(int rw, struct kiocb *iocb,
+	struct inode *inode, struct block_device *bdev, const struct iovec *iov,
+	loff_t offset, unsigned long nr_segs, get_block_t get_block,
+	dio_iodone_t end_io)
+{
+	return __blockdev_direct_IO_newtrunc(rw, iocb, inode, bdev, iov, offset,
+				    nr_segs, get_block, end_io,
+				    DIO_LOCKING | DIO_SKIP_HOLES);
+}
+
+static inline ssize_t blockdev_direct_IO_no_locking_newtrunc(int rw, struct kiocb *iocb,
+	struct inode *inode, struct block_device *bdev, const struct iovec *iov,
+	loff_t offset, unsigned long nr_segs, get_block_t get_block,
+	dio_iodone_t end_io)
+{
+	return __blockdev_direct_IO_newtrunc(rw, iocb, inode, bdev, iov, offset,
+				nr_segs, get_block, end_io, 0);
+}
 static inline ssize_t blockdev_direct_IO(int rw, struct kiocb *iocb,
 	struct inode *inode, struct block_device *bdev, const struct iovec *iov,
 	loff_t offset, unsigned long nr_segs, get_block_t get_block,
@@ -2327,12 +2349,14 @@ extern int dcache_dir_open(struct inode
 extern int dcache_dir_close(struct inode *, struct file *);
 extern loff_t dcache_dir_lseek(struct file *, loff_t, int);
 extern int dcache_readdir(struct file *, void *, filldir_t);
+extern int simple_setattr(struct dentry *dentry, struct iattr *attr);
 extern int simple_getattr(struct vfsmount *, struct dentry *, struct kstat *);
 extern int simple_statfs(struct dentry *, struct kstatfs *);
 extern int simple_link(struct dentry *, struct inode *, struct dentry *);
 extern int simple_unlink(struct inode *, struct dentry *);
 extern int simple_rmdir(struct inode *, struct dentry *);
 extern int simple_rename(struct inode *, struct dentry *, struct inode *, struct dentry *);
+extern int simple_setsize(struct inode *inode, loff_t newsize);
 extern int simple_sync_file(struct file *, struct dentry *, int);
 extern int simple_empty(struct dentry *);
 extern int simple_readpage(struct file *file, struct page *page);
@@ -2367,7 +2391,8 @@ extern int buffer_migrate_page(struct ad
 
 extern int inode_change_ok(const struct inode *, struct iattr *);
 extern int inode_newsize_ok(const struct inode *, loff_t offset);
-extern int __must_check inode_setattr(struct inode *, struct iattr *);
+extern int __must_check inode_setattr(struct inode *, const struct iattr *);
+extern void generic_setattr(struct inode *inode, const struct iattr *attr);
 
 extern void file_update_time(struct file *file);
 
mm/truncate.c +5 -5
Index: linux-2.6/mm/truncate.c
===================================================================
@@ -547,18 +547,18 @@ EXPORT_SYMBOL(truncate_pagecache);
  * NOTE! We have to be ready to update the memory sharing
  * between the file and the memory map for a potential last
  * incomplete page.  Ugly, but necessary.
+ *
+ * This function is deprecated and simple_setsize or truncate_pagecache
+ * should be used instead.
  */
 int vmtruncate(struct inode *inode, loff_t offset)
 {
-	loff_t oldsize;
 	int error;
 
-	error = inode_newsize_ok(inode, offset);
+	error = simple_setsize(inode, offset);
 	if (error)
 		return error;
-	oldsize = inode->i_size;
-	i_size_write(inode, offset);
-	truncate_pagecache(inode, oldsize, offset);
+
 	if (inode->i_op->truncate)
 		inode->i_op->truncate(inode);
 
include/linux/buffer_head.h +9 -0
Index: linux-2.6/include/linux/buffer_head.h
===================================================================
@@ -203,6 +203,9 @@ int block_write_full_page_endio(struct p
 int block_read_full_page(struct page*, get_block_t*);
 int block_is_partially_uptodate(struct page *page, read_descriptor_t *desc,
 				unsigned long from);
+int block_write_begin_newtrunc(struct file *, struct address_space *,
+				loff_t, unsigned, unsigned,
+				struct page **, void **, get_block_t*);
 int block_write_begin(struct file *, struct address_space *,
 				loff_t, unsigned, unsigned,
 				struct page **, void **, get_block_t*);
@@ -214,6 +217,9 @@ int generic_write_end(struct file *, str
 				struct page *, void *);
 void page_zero_new_buffers(struct page *page, unsigned from, unsigned to);
 int block_prepare_write(struct page*, unsigned, unsigned, get_block_t*);
+int cont_write_begin_newtrunc(struct file *, struct address_space *, loff_t,
+			unsigned, unsigned, struct page **, void **,
+			get_block_t *, loff_t *);
 int cont_write_begin(struct file *, struct address_space *, loff_t,
 			unsigned, unsigned, struct page **, void **,
 			get_block_t *, loff_t *);
@@ -225,6 +231,9 @@ void block_sync_page(struct page *);
 sector_t generic_block_bmap(struct address_space *, sector_t, get_block_t *);
 int block_truncate_page(struct address_space *, loff_t, get_block_t *);
 int file_fsync(struct file *, struct dentry *, int);
+int nobh_write_begin_newtrunc(struct file *, struct address_space *,
+				loff_t, unsigned, unsigned,
+				struct page **, void **, get_block_t*);
 int nobh_write_begin(struct file *, struct address_space *,
 				loff_t, unsigned, unsigned,
 				struct page **, void **, get_block_t*);
#1 lore raw

PATCH 2/2 fs: convert simple fs to new truncate

fs: convert simple fs to new truncate

Convert simple filesystems: ramfs, configfs, sysfs, block_dev to new truncate sequence.

---
 fs/block_dev.c        |    9 +++++----
 fs/configfs/inode.c   |   10 +++++++---
 fs/ramfs/file-mmu.c   |    1 +
 fs/ramfs/file-nommu.c |    7 ++++---
 fs/sysfs/inode.c      |    6 +-----
 5 files changed, 18 insertions(+), 15 deletions(-)
fs/configfs/inode.c +7 -3
Index: linux-2.6/fs/configfs/inode.c
===================================================================
@@ -77,9 +77,13 @@ int configfs_setattr(struct dentry * den
 	if (error)
 		return error;
 
-	error = inode_setattr(inode, iattr);
-	if (error)
-		return error;
+	if (iattr->ia_valid & ATTR_SIZE) {
+		error = simple_setsize(inode, iattr->ia_size);
+		if (error)
+			return error;
+	}
+
+	generic_setattr(inode, iattr);
 
 	if (!sd_iattr) {
 		/* setting attributes for the first time, allocate now */
fs/ramfs/file-mmu.c +1 -0
Index: linux-2.6/fs/ramfs/file-mmu.c
===================================================================
@@ -50,5 +50,6 @@ const struct file_operations ramfs_file_
 };
 
 const struct inode_operations ramfs_file_inode_operations = {
+	.setattr	= simple_setattr,
 	.getattr	= simple_getattr,
 };
fs/ramfs/file-nommu.c +4 -3
Index: linux-2.6/fs/ramfs/file-nommu.c
===================================================================
@@ -145,7 +145,7 @@ static int ramfs_nommu_resize(struct ino
 			return ret;
 	}
 
-	ret = vmtruncate(inode, newsize);
+	ret = simple_setsize(inode, newsize);
 
 	return ret;
 }
@@ -168,7 +168,8 @@ static int ramfs_nommu_setattr(struct de
 
 	/* pick out size-changing events */
 	if (ia->ia_valid & ATTR_SIZE) {
-		loff_t size = i_size_read(inode);
+		loff_t size = inode->i_size;
+
 		if (ia->ia_size != size) {
 			ret = ramfs_nommu_resize(inode, ia->ia_size, size);
 			if (ret < 0 || ia->ia_valid == ATTR_SIZE)
@@ -181,7 +182,7 @@ static int ramfs_nommu_setattr(struct de
 		}
 	}
 
-	ret = inode_setattr(inode, ia);
+	generic_setattr(inode, ia);
  out:
 	ia->ia_valid = old_ia_valid;
 	return ret;
fs/sysfs/inode.c +1 -5
Index: linux-2.6/fs/sysfs/inode.c
===================================================================
@@ -115,11 +115,7 @@ int sysfs_setattr(struct dentry *dentry,
 	if (error)
 		return error;
 
-	iattr->ia_valid &= ~ATTR_SIZE; /* ignore size changes */
-
-	error = inode_setattr(inode, iattr);
-	if (error)
-		return error;
+	generic_setattr(inode, iattr);
 
 	mutex_lock(&sysfs_mutex);
 	error = sysfs_sd_setattr(sd, iattr);
fs/block_dev.c +5 -4
Index: linux-2.6/fs/block_dev.c
===================================================================
@@ -172,8 +172,9 @@ blkdev_direct_IO(int rw, struct kiocb *i
 	struct file *file = iocb->ki_filp;
 	struct inode *inode = file->f_mapping->host;
 
-	return blockdev_direct_IO_no_locking(rw, iocb, inode, I_BDEV(inode),
-				iov, offset, nr_segs, blkdev_get_blocks, NULL);
+	return blockdev_direct_IO_no_locking_newtrunc(rw, iocb, inode,
+				I_BDEV(inode), iov, offset, nr_segs,
+				blkdev_get_blocks, NULL);
 }
 
 int __sync_blockdev(struct block_device *bdev, int wait)
@@ -350,8 +351,8 @@ static int blkdev_write_begin(struct fil
 			struct page **pagep, void **fsdata)
 {
 	*pagep = NULL;
-	return block_write_begin(file, mapping, pos, len, flags, pagep, fsdata,
-				blkdev_get_block);
+	return block_write_begin_newtrunc(file, mapping, pos, len, flags,
+				pagep, fsdata, blkdev_get_block);
 }
 
 static int blkdev_write_end(struct file *file, struct address_space *mapping,
#2 lore raw
In reply to Nick Piggin

What is the patch's destiny, was it accepted to some dev-tree? If no, please clarify when it will be accepted. Basically want to move newsize_ok check out from vmtruncate to inode_change_ok to fix nasty bugs in setattr() and it is depends on Nick's patch.

#3 lore raw
Quoted 5 lines · Dmitry Monakhov

What is the patch's destiny, was it accepted to some dev-tree? If no, please clarify when it will be accepted. Basically want to move newsize_ok check out from vmtruncate to inode_change_ok to fix nasty bugs in setattr() and it is depends on Nick's patch.

I guess Al hasn't had a chance to look at them yet. I don't think they should change much, though. If you want to base patches on top of it, that should be fine. I could submit to -mm in the meantime if you like? Let me know also if you want the new-truncate filesystem conversions (many of them are done).

Thanks,
Nick

#4 lore raw
Quoted 7 lines · Nick Piggin
Quoted 5 lines · Dmitry Monakhov

What is the patch's destiny, was it accepted to some dev-tree? If no, please clarify when it will be accepted. Basically want to move newsize_ok check out from vmtruncate to inode_change_ok to fix nasty bugs in setattr() and it is depends on Nick's patch.

Quoted 6 lines

I guess Al hasn't had a chance to look at them yet. I don't think they should change much, though. If you want to base patches on top of it, that should be fine. I could submit to -mm in the meantime if you like? Let me know also if you want the new-truncate filesystem conversions (many of them are done).

I will please need the latest one you have for ext2. (Is there a public tree with all these?). Could you CC: me on the ext2 patch so I can monitor it's progress, that would be grate.

Quoted 3 lines

Thanks,
Nick

Thanks
Boaz