0/8 Unionfs updates/cleanups/fixes
Subsystems (3)
The following is a series of patches related to Unionfs. The main change here is that we dropped ->sync_page and rewritten ->writepage.
These patches were tested (where appropriate) on Linus's 2.6.24 latest code (as of v2.6.24-rc1-521-g54866f0), as well as the backports to 2.6.{23,22,21,20,19,18,9} on ext2/3/4, xfs, reiserfs, nfs2/3/4, jffs2, ramfs, tmpfs, cramfs, and squashfs (where available). See
http://unionfs.filesystems.org/ to download backported unionfs code.
Please pull from the 'master' branch ofgit://git.kernel.org/pub/scm/linux/kernel/git/ezk/unionfs.git
to receive the following:
Erez Zadok (8):
Unionfs: delete whiteouts in sticky directories
Unionfs: cleanup permission checking code
Unionfs: update usage.txt documentation
Unionfs: mmap updates
Unionfs: avoid a deadlock during branch-management on a pivot_root'ed union
Unionfs: don't bother validating inode if it has no lower branches
Unionfs: don't printk an error if it's due to common copyup Unionfs/VFS: no need to export 2 symbols in security/security.c
Documentation/filesystems/unionfs/usage.txt | 17 +++ fs/unionfs/commonfops.c | 4 fs/unionfs/debug.c | 6 + fs/unionfs/inode.c | 96 ++++++----------- fs/unionfs/mmap.c | 156 ++++++++++------------------ fs/unionfs/rename.c | 2 fs/unionfs/subr.c | 8 + fs/unionfs/union.h | 39 +++++-- security/security.c | 2 9 files changed, 158 insertions(+), 172 deletions(-)
---
Erez Zadok
[email protected]
PATCH 1/8 Unionfs: delete whiteouts in sticky directories
This is needed to maintain Unix semantics.
--- fs/unionfs/inode.c | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)
fs/unionfs/inode.c +17 -0
@@ -90,6 +90,23 @@ static int unionfs_create(struct inode *parent, struct dentry *dentry, err =vfs_unlink (lower_dir_dentry->d_inode, wh_dentry);unlock_dir (lower_dir_dentry); +/* +* Whiteouts are special files and should be deleted +* no matter what (as if they never existed), in +* order to allow this create operation to succeed. +* This is especially important in sticky +* directories: a whiteout may have been created by +* one user, but the newly created file may be +* created by another user. Therefore, in order to +* maintain Unix semantics, if the vfs_unlink above +* ailed, then we have to try to directly unlink the +* whiteout. Note: in the ODF version of unionfs, +* whiteout are handled much more cleanly. +*/ +if (err == -EPERM ) { +struct inode *inode = lower_dir_dentry->d_inode; + err = inode->i_op->unlink (inode, wh_dentry); + }if (err) {printk (KERN_ERR "unionfs: create: could not " "unlink whiteout, err = %d\n" , err);
Signature · 1 line
1.5.2.2
PATCH 2/8 Unionfs: cleanup permission checking code
In reply to Erez ZadokUse vfs helpers and avoid redundant checks performed by the VFS already.
--- fs/unionfs/commonfops.c | 4 --- fs/unionfs/inode.c | 70 +++++++++-------------------------------------- 2 files changed, 13 insertions(+), 61 deletions(-)
fs/unionfs/commonfops.c +0 -4
@@ -669,10 +669,6 @@ static long do_ioctl(struct file *file, unsigned int cmd, unsigned long arg) lower_file =unionfs_lower_file (file); - err =security_file_ioctl (lower_file, cmd, arg); -if (err) - goto out; - err = -ENOTTY ;if (!lower_file || !lower_file->f_op) goto out;
fs/unionfs/inode.c +13 -57
83 lines not on this page · raw message
Signature · 1 line
1.5.2.2
PATCH 3/8 Unionfs: update usage.txt documentation
In reply to Erez Zadok--- Documentation/filesystems/unionfs/usage.txt | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)
Documentation/filesystems/unionfs/usage.txt +17 -0
@@ -74,6 +74,23 @@ to read-write, and change /bar from read-write to read-only: # mount -t unionfs -o remount,mode=/foo=rw,mode=/bar=ro none MOUNTPOINT +Note: in Unionfs 2.x, you cannot set the leftmost branch to readonly because +then Unionfs won't have any writable place for copyups to take place. +Moreover, the VFS can get confused when it tries to modify something in a +file system mounted read-write, but isn't permitted to write to it. +Instead, you should set the whole union as readonly, as described above. +If, however, you must set the leftmost branch as readonly, perhaps so you +can get a snapshot of it at a point in time, then you should insert a new +writable top-level branch, and mark the one you want as readonly. This can +be accomplished as follows, assuming that /foo is your current leftmost +branch: + +# mount -t tmpfs -o size=NNN /new +# mount -t unionfs -o remount,add=/new,mode=/foo=ro none MOUNTPOINT +<do what you want safely in /foo> +# mount -t unionfs -o remount,del=/new,mode=/foo=rw none MOUNTPOINT +<check if there's anything in /new you want to preserve> +# umount /new CACHE CONSISTENCY =================
Signature · 1 line
1.5.2.2
PATCH 4/8 Unionfs: mmap updates
In reply to Erez ZadokRewrite unionfs_writepage to minimize dependence on AOP_WRITEPAGE_ACTIVEATE, handle memory pressure better, and update documentation. Remove unionfs_sync_page because it's not needed.
--- fs/unionfs/mmap.c | 156 ++++++++++++++++++++-------------------------------- 1 files changed, 60 insertions(+), 96 deletions(-)
fs/unionfs/mmap.c +60 -96
@@ -25,83 +25,91 @@ static int unionfs_writepage(struct page *page, struct writeback_control *wbc)struct inode *inode;struct inode *lower_inode;struct page *lower_page; -char *kaddr, *lower_kaddr; +struct address_space *lower_mapping;/* lower inode mapping */ +gfp_t mask; inode = page->mapping->host; lower_inode =unionfs_lower_inode (inode); + lower_mapping = lower_inode->i_mapping;/* * find lower page (returns a locked page) * -* NOTE: we used to call grab_cache_page(), but that was unnecessary -* as it would have tried to create a new lower page if it didn't -* exist, leading to deadlocks (esp. under memory-pressure -* conditions, when it is really a bad idea to *consume* more -* memory). Instead, we assume the lower page exists, and if we can -* find it, then we ->writepage on it; if we can't find it, then it -* couldn't have disappeared unless the kernel already flushed it, -* in which case we're still OK. This is especially correct if -* wbc->sync_mode is WB_SYNC_NONE (as per -* Documentation/filesystems/vfs.txt). If we can't flush our page -* because we can't find a lower page, then at least we re-mark our -* page as dirty, and return AOP_WRITEPAGE_ACTIVATE as the VFS -* expects us to. (Note, if in the future it'd turn out that we -* have to find a lower page no matter what, then we'd have to -* resort to RAIF's page pointer flipping trick.) +* We turn off __GFP_FS while we look for or create a new lower +* page. This prevents a recursion into the file system code, which +* under memory pressure conditions could lead to a deadlock. This +* is similar to how the loop driver behaves (see loop_set_fd in +* drivers/block/loop.c). If we can't find the lower page, we +* redirty our page and return "success" so that the VM will call us +* again in the (hopefully near) future. */ - lower_page =find_lock_page (lower_inode->i_mapping, page->index); + mask =mapping_gfp_mask (lower_mapping) & ~(__GFP_FS); + lower_page =find_or_create_page (lower_mapping, page->index, mask);if (!lower_page) { - err =AOP_WRITEPAGE_ACTIVATE ; + err =0 ;set_page_dirty (page); goto out; } -/* get page address, and encode it */ - kaddr =kmap (page); - lower_kaddr =kmap (lower_page); +/* copy page data from our upper page to the lower page */ +copy_highpage (lower_page, page); -memcpy (lower_kaddr, kaddr,PAGE_CACHE_SIZE ); - -kunmap (page); -kunmap (lower_page); - -BUG_ON (!lower_inode->i_mapping->a_ops->writepage); - -/* call lower writepage (expects locked page) */ -clear_page_dirty_for_io (lower_page);/* emulate VFS behavior */ - err = lower_inode->i_mapping->a_ops->writepage (lower_page, wbc); - -/* b/c find_lock_page locked it and ->writepage unlocks on success */ -if (err) +/* +* Call lower writepage (expects locked page). However, if we are +* called with wbc->for_reclaim, then the VFS/VM just wants to +* reclaim our page. Therefore, we don't need to call the lower +* ->writepage: just copy our data to the lower page (already done +* above), then mark the lower page dirty and unlock it, and return +* success. +*/ +if (wbc->for_reclaim) { +set_page_dirty (lower_page);unlock_page (lower_page); -/* b/c grab_cache_page increased refcnt */ -page_cache_release (lower_page); - + goto out_release; + } +BUG_ON (!lower_mapping->a_ops->writepage); +clear_page_dirty_for_io (lower_page);/* emulate VFS behavior */ + err = lower_mapping->a_ops->writepage (lower_page, wbc);if (err <0 ) {ClearPageUptodate (page); - goto out; + goto out_release; } + +/* +* Lower file systems such as ramfs and tmpfs, may return +* AOP_WRITEPAGE_ACTIVATE so that the VM won't try to (pointlessly) +* write the page again for a while. But those lower file systems +* also set the page dirty bit back again. Since we successfully +* copied our page data to the lower page, then the VM will come +* back to the lower page (directly) and try to flush it. So we can +* save the VM the hassle of coming back to our page and trying to +* flush too. Therefore, we don't re-dirty our own page, and we +* never return AOP_WRITEPAGE_ACTIVATE back to the VM (we consider +* this a success). +* +* We also unlock the lower page if the lower ->writepage returned +* AOP_WRITEPAGE_ACTIVATE. (This "anomalous" behaviour may be +* addressed in future shmem/VM code.) +*/ if (err ==AOP_WRITEPAGE_ACTIVATE ) { -/* -* Lower file systems such as ramfs and tmpfs, may return -* AOP_WRITEPAGE_ACTIVATE so that the VM won't try to -* (pointlessly) write the page again for a while. But -* those lower file systems also set the page dirty bit back -* again. So we mimic that behaviour here. -*/ -if (PageDirty (lower_page)) -set_page_dirty (page); - goto out; + err =0 ; +unlock_page (lower_page); }/* all is well */ SetPageUptodate (page); -/* lower mtimes has changed: update ours */ +/* lower mtimes have changed: update ours */ unionfs_copy_attr_times (inode); -unlock_page (page); - +out_release: +/* b/c find_or_create_page increased refcnt */ +page_cache_release (lower_page); out: +/* +* We unlock our page unconditionally, because we never return +* AOP_WRITEPAGE_ACTIVATE. +*/ +unlock_page (page);return err; }
@@ -119,9 +127,8 @@ static int unionfs_writepages(struct address_space *mapping,if (!mapping_cap_writeback_dirty (lower_inode->i_mapping)) goto out; -/* Note: generic_writepages may return AOP_WRITEPAGE_ACTIVATE */ err =generic_writepages (mapping, wbc); -if (err ==0 ) +if (!err)unionfs_copy_attr_times (inode); out:return err;
@@ -313,53 +320,10 @@ out:return err;/* assume all is ok */ } -static void unionfs_sync_page (struct page *page) -{ -struct inode *inode; -struct inode *lower_inode; -struct page *lower_page; -struct address_space *mapping; - - inode = page->mapping->host; - lower_inode =unionfs_lower_inode (inode); - -/* -* Find lower page (returns a locked page). -* -* NOTE: we used to call grab_cache_page(), but that was unnecessary -* as it would have tried to create a new lower page if it didn't -* exist, leading to deadlocks. All our sync_page method needs to -* do is ensure that pending I/O gets done. -*/ - lower_page =find_lock_page (lower_inode->i_mapping, page->index); -if (!lower_page) { -printk (KERN_ERR "unionfs: find_lock_page failed\n" ); - goto out; - } - -/* do the actual sync */ - mapping = lower_page->mapping; -/* -* XXX: can we optimize ala RAIF and set the lower page to be -* discarded after a successful sync_page? -*/ -if (mapping && mapping->a_ops && mapping->a_ops->sync_page) - mapping->a_ops->sync_page (lower_page); - -/* b/c find_lock_page locked it */ -unlock_page (lower_page); -/* b/c find_lock_page increased refcnt */ -page_cache_release (lower_page); - -out: -return ; -} -struct address_space_operations unionfs_aops = { .writepage = unionfs_writepage, .writepages = unionfs_writepages, .readpage = unionfs_readpage, .prepare_write = unionfs_prepare_write, .commit_write = unionfs_commit_write, - .sync_page = unionfs_sync_page, };
Signature · 1 line
1.5.2.2
PATCH 5/8 Unionfs: avoid a deadlock during branch-management on a pivot_root'ed union
In reply to Erez Zadok--- fs/unionfs/union.h | 39 +++++++++++++++++++++++++++++++++------ 1 files changed, 33 insertions(+), 6 deletions(-)
fs/unionfs/union.h +33 -6
@@ -142,11 +142,18 @@ struct unionfs_sb_info {* This rwsem is used to make sure that a branch management * operation... * 1) will not begin before all currently in-flight operations -* complete +* complete. * 2) any new operations do not execute until the currently -* running branch management operation completes +* running branch management operation completes. +* +* The write_lock_owner records the PID of the task which grabbed +* the rw_sem for writing. If the same task also tries to grab the +* read lock, we allow it. This prevents a self-deadlock when +* branch-management is used on a pivot_root'ed union, because we +* have to ->lookup paths which belong to the same union. */ struct rw_semaphore rwsem; +pid_t write_lock_owner;/* PID of rw_sem owner (write lock) */ int high_branch_id;/* last unique branch ID given */ struct unionfs_data *data; };
@@ -234,10 +241,30 @@ static inline off_t rdstate2offset(struct unionfs_dir_state *buf)return tmp; } -#define unionfs_read_lock (sb) down_read(&UNIONFS_SB(sb)->rwsem) -#define unionfs_read_unlock (sb) up_read(&UNIONFS_SB(sb)->rwsem) -#define unionfs_write_lock (sb) down_write(&UNIONFS_SB(sb)->rwsem) -#define unionfs_write_unlock (sb) up_write(&UNIONFS_SB(sb)->rwsem) +static inline void unionfs_read_lock (struct super_block *sb) +{ +if (UNIONFS_SB (sb)->write_lock_owner && +UNIONFS_SB (sb)->write_lock_owner == current->pid) +return ; +down_read (&UNIONFS_SB (sb)->rwsem); +} +static inline void unionfs_read_unlock (struct super_block *sb) +{ +if (UNIONFS_SB (sb)->write_lock_owner && +UNIONFS_SB (sb)->write_lock_owner == current->pid) +return ; +up_read (&UNIONFS_SB (sb)->rwsem); +} +static inline void unionfs_write_lock (struct super_block *sb) +{ +down_write (&UNIONFS_SB (sb)->rwsem); +UNIONFS_SB (sb)->write_lock_owner = current->pid; +} +static inline void unionfs_write_unlock (struct super_block *sb) +{ +up_write (&UNIONFS_SB (sb)->rwsem); +UNIONFS_SB (sb)->write_lock_owner =0 ; +}static inline void unionfs_double_lock_dentry (struct dentry *d1,struct dentry *d2)
Signature · 1 line
1.5.2.2
PATCH 6/8 Unionfs: don't bother validating inode if it has no lower branches
In reply to Erez Zadok--- fs/unionfs/debug.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
fs/unionfs/debug.c +6 -0
@@ -49,6 +49,9 @@ void __unionfs_check_inode(const struct inode *inode, sb = inode->i_sb; istart =ibstart (inode); iend =ibend (inode); +/* don't check inode if no lower branches */ +if (istart <0 && iend <0 ) +return ;if (unlikely (istart > iend)) {PRINT_CALLER (fname, fxn, line); pr_debug(" Ci0: inode=%p istart/end=%d:%d\n",
@@ -221,6 +224,9 @@ check_inode:return ; istart =ibstart (inode); iend =ibend (inode); +/* don't check inode if no lower branches */ +if (istart <0 && iend <0 ) +return ;BUG_ON (istart > iend);if (unlikely ((istart ==-1 && iend !=-1 ) || (istart !=-1 && iend ==-1 ))) {
Signature · 1 line
1.5.2.2
PATCH 7/8 Unionfs: don't printk an error if it's due to common copyup
In reply to Erez Zadok--- fs/unionfs/inode.c | 9 +++++---- fs/unionfs/rename.c | 2 +- fs/unionfs/subr.c | 8 ++++++-- 3 files changed, 12 insertions(+), 7 deletions(-)
fs/unionfs/inode.c +5 -4
@@ -465,10 +465,11 @@ static int unionfs_symlink(struct inode *dir, struct dentry *dentry,if (!lower_dentry ||IS_ERR (lower_dentry)) {if (IS_ERR (lower_dentry)) err =PTR_ERR (lower_dentry); - -printk (KERN_ERR "unionfs: lower dentry " -"NULL (or error) for bindex = %d\n" , - bindex); +if (!IS_COPYUP_ERR (err)) +printk (KERN_ERR +"unionfs: create_parents for " +"symlink failed: bindex=%d " +"err=%d\n" , bindex, err);continue ; } }
fs/unionfs/rename.c +1 -1
@@ -41,7 +41,7 @@ static int __unionfs_rename(struct inode *old_dir, struct dentry *old_dentry, bindex);if (IS_ERR (lower_new_dentry)) { err =PTR_ERR (lower_new_dentry); -if (err == -EROFS ) +if (IS_COPYUP_ERR (err)) goto out; printk(KERN_ERR "unionfs: error creating directory " "tree for rename, bindex=%d err=%d\n",
fs/unionfs/subr.c +6 -2
@@ -61,8 +61,12 @@ int create_whiteout(struct dentry *dentry, int start) dentry->d_name.name, bindex);if (!lower_dentry ||IS_ERR (lower_dentry)) { -printk (KERN_ERR "unionfs: create_parents " -"failed for bindex = %d\n" , bindex); +int ret =PTR_ERR (lower_dentry); +if (!IS_COPYUP_ERR (ret)) +printk (KERN_ERR +"unionfs: create_parents for " +"whiteout failed: bindex=%d " +"err=%d\n" , bindex, ret);continue ; } }
Signature · 1 line
1.5.2.2
PATCH 8/8 Unionfs/VFS: no need to export 2 symbols in security/security.c
In reply to Erez Zadok--- security/security.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-)
security/security.c +0 -2
@@ -409,7 +409,6 @@ int security_inode_permission(struct inode *inode, int mask, struct nameidata *nreturn 0 ;return security_ops->inode_permission (inode, mask, nd); } -EXPORT_SYMBOL (security_inode_permission);int security_inode_setattr (struct dentry *dentry,struct iattr *attr) {
@@ -519,7 +518,6 @@ int security_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg) {return security_ops->file_ioctl (file, cmd, arg); } -EXPORT_SYMBOL (security_file_ioctl); int security_file_mmap(struct file *file, unsigned long reqprot, unsigned long prot, unsigned long flags,
Signature · 1 line
1.5.2.2