AUFS: merging/stacking several filesystems

Hello fs-developers,

I am developing a stackable unification filesystem which unifies several directories and provides a merged single directory.
I guess most people already knows what it is. When users access a file, the access will be passed/re-directed/converted (sorry, I am not sure which English word is correct) to the real file on the member filesystem. The member filesystem is called 'lower filesytstem' or 'branch' and has a mode 'readonly' and 'readwrite.' And the file deletion is handled as 'whiteout' on the upper writable branch.

On this ML, there have been discussions about UnionMount (Jan Blunck and Bharata B Rao) and Unionfs (Erez Zadok). They took different approaches to implement the merged-view.
The former tries putting it into VFS, and the latter implements as a separate filesystem.
(If I misunderstand about these implementations, please let me know and I shall correct it. Because it is a long time ago when I read their source files last time.)
UnionMount's approach will be able to small, but may be hard to share branches between several UnionMount since the whiteout in it is implemented in the inode on branch filesystem and always shared. According to Bharata's recent post, readdir does not seems to be finished yet.
Unionfs has a longer history. When I got the idea of stacking filesystem (Aug 2005), it already existed. It has virtual super_block, inode, dentry and file objects and they have an array pointing lower same kind objects. After contributing many patches for Unionfs, I re-started my project AUFS (Jun 2006).

In AUFS, the structure of filesystem is simlilar to Unionfs, but I implemented my own ideas, approaches and enhancements in it. Here are some of them and the intention of this post is to get some initial feedback about its design.
You can see the actual details, documents, CVS logs, and how people are using it from
<http://aufs.sf.net>.

Kindly review and let me know your comments.

o file mapping -- mmap and sharing pages
----------------------------------------------------------------------
In AUFS, the file-mapped pages are shared between the lower file and the AUFS's virtual one by overriding vm_operation, particularly ->fault().

In aufs_mmap(),
- get and store vm_ops of the lower file.
- map the file of aufs by generic_file_mmap() and set aufs's vm operations.

In aufs_fault(),
- a race can happen. for instance a multithreaded library.
- get the file of aufs from the passed vma, sleep if needed.
- get the lower file from the aufs file.
- call ->fault() in the previously stored vm_ops with setting the lower file to vm_file.
- restore vm_file and wake_up if someone else got sleep.

When a member filesystem is added to or deleted from the stack (often called union), the same-named file may unveil and its contents will be replaced by the new one when a process read(2) through previously opened file.
(Some users may not want to refresh the filedata. For such users, I have a plan to implement a mount option 'refrof' which decides to refresh the opened files or not.)
In this case, an already mapped file will not be updated since the contents are a part of a process and it should not be changed by AUFS branch management. Of course, in case of the deleting branch has a busy file, it cannot be deleted from the union.

In UnionMount, it won't be matter since it doesn't have its own inode and file object.
In Unionfs, the memory pages mapped to filedata are copied from the lower (real) file into the Unionfs's virtual one and handles it by address_space operations. Recently Unionfs changed it to the one I suggested in last December which AUFS took (since Jul 2006).

o external inode number table and bitmap (XINO/XIB)
----------------------------------------------------------------------
Because aufs has its own virtual inode, it has to manage the inode number. Generally iunique() is used for this purpose, but when a user execute chmod/chown -R to a large directory or rmdir to a dir who has child, a problem may arise. Because chmod/chown -R checks the inode number, it may be changed/re-assigned silently/internally and the command will return an error. In rmdir, dentry_unhash() is called and its child dentry/inode is unhashed. It means the inode number for the child will be changed/re-assigned when then will be accessed again.

To keep the inode number unchanged, aufs has an external inode number table and bitmap (which are called 'xino' and 'xib') per a branch filesystem. The table is a regular file which is created on the first writable branch automatically be default. When several branches exist on the same (real) filesystem, those files will be shared. If xino/xib is unnecessary for user, he can specify 'noxino' mount option and disable it.
Aufs shows the size of these files via sysfs.

Currently these xino/xib are created and deleted at the aufs mount time (the files are still opened), but I have a request from users who are using aufs on NFS server and exporting. So I will implement an option not to delete xino/xib files and re-use it after NFS server reboot.

In UnionMount, it won't be matter since it doesn't have its own inode. In Unionfs, they took iunique() approach and still have above problem. But they already started Unionfs-ODF branch which has another mounted filesystem and delegate the inode number management to it. The ODF approach has some overhead since it requires to create/remove files/dirs on another filesystem.

o cache coherency or user's direct access to branch filesystems (UDBA) -- inotify
----------------------------------------------------------------------
Users may create/delete/change files on branch, bypassing aufs, at anytime (user's direct access, UDBA). Because aufs has its own inode and file objects and they are cached in a generic way, it has to maintain the inode attribute and the directory listing.

In order to implement this, aufs has three levels of detect-test. The most strict test is using inotify(CONFIG_INOTIFY) feature. When a user specifies this test level, aufs will set inotify-watch to all the branch dir in cache. When an aufs dir inode object is created and cached, it will refer the real dirs on branches, and aufs sets inotify-watch to them and will be notified when UDBA occurs. The watch will be cleared when the aufs dir inode is purged from the system inode cache.
When UDBA occurs, aufs registers a function to 'events' thread by schedule_work(), and the function sets some special status to the cached aufs inode private data. When the same file is accessed through aufs, aufs will detect the status and refresh all necessary data.

The other two levels of test don't use inotify. The most simple test level checks nothing. It is for readonly filesystems such as cdrom (Even if the most strict test is specified, aufs doesn't set inotify to such filesystems). The middle level (default) is checking/comparing inode attributes in d_revalidate(). It means this test level may not be effective for a negative dentry.
In most cases, I guess the default level is enough and users can execute 'mount -o remount /aufs' to discard the unused caches. But if a user really want to reflect the UDBA soon, the highest test option will help him/her.

o hardlink over branches, pseudo-link
----------------------------------------------------------------------
When a file on a lower readonly branch is hard-linked (fileA and fileB) and a user modifies fileA, aufs will copy-up it to the upper writable branch and make the originally requested change to fileA on the upper branch. On the writable branch, fileA is not hardlinked. It means fileB on the lower branch still have the old contents.

To address this problem, aufs introduced a 'pseudo-link' (plink) which is a logical hardlink over branches. It maintains the simple inode list on memory and checks the accessed inode is in the list. Finally fileB is handled as if it existed on the writable branch, by referencing fileA's inode on the writable branch as fileB's inode.

Additionally, to support the case of fileA on the writable branch is deleted, aufs creates another hardlink on the writable branch which exists under a special directory to hide it from users.

At remount/umount time, /sbin/{mount,umount}.aufs script checks the pseudo-linked inode list in aufs, re-produces all real hardlinks on the writable branch, and flushes the list on memory (But these script has a potential race problem).

Thank you reading this long and my broken English.

Junjiro Okajima

Hi all.

I am happy to see AuFS author in this list, and I hope there will be some people who review the design and post own comments and ideas, in order to make AuFS even better. :) I am an AuFS user for a long time and what I really appreciate (from the user's point of view) is the following:

- AuFS supports writable branch balancing. That means, you can setup several partitions for writing and AuFS will split all new/modified files between them, based on free disk space, existence of parent directory, randomly, or combinations.

- AuFS supports huge amount of branches. I'm currently using hundreds of branches without just a small slowdown (which is obvious).

- AuFS provides a list of branches through /sys, which doesn't have the limitation like /proc/mounts. For that reason, it works correctly even with thousand of branches (while so much branches would break /proc/mounts at all).

- AuFS implements 'rr' branch mode, it means 'really-readonly'. This is really useful, particularly for ISO images or SquashFS filesystems as a brach, as AuFS doesn't need to re-lookup those filesystems. (You know, a readonly branch 'ro' can be modified from another place, eg. network, so there can occur a 'direct branch access' even for read-only directories and AuFS handles it correctly.)

- last, but not the least, AuFS is really stable in real world situations. I used unionfs in the past, but my second name for it was 'NULL POINTER DEREFERENCE'. I can see those errors still happening in latest unionfs as well, last one I've found is from 27th of May 2008 ... BUG: unable to handle kernel NULL pointer dereference. ... I have absolutely no idea what that means, but the same errors keep appearing in unionfs for years. You won't see anything like that in AuFS. Guess why knoppix and other projects switched to it :)

That's all from me :)
Thanks

Tomas M
slax.org

	:::

o readdir -- virtual dir block on memory (VDIR)
----------------------------------------------------------------------
This is an approach I posted a few months ago replying UnionMount's post. It constructs a virtual dir block on memory. For readdir, aufs calls vfs_readdir() internally for each lower dirs, merges their entries with eliminating the whiteout-ed ones, and gives it the the file (dir) object. So the file object has its entry list until it is closed. The entry list will be updated when the file position is zero and becomes old. This decision is made in aufs automatically.

It may consume rather large memory and cpu cycles. To reduce the number of memory allocations, the implementation became rather tricky .

Some people may call it can be a security hole or DoS attack since the opened and once readdir-ed dir (file object) holds its entry list and becomes a pressure for system memory. But I'd say it is similar to files under /proc or /sys. The virtual files on procfs and sysfs also holds a memory page (generally) while they are opened. When an idea to reduce memory for them is introduced, it will be applied to aufs too.

o policies for selecting one among multiple writable branches, parent-dir, round-robin and most-free-space
----------------------------------------------------------------------
When the number of writable branch is more than one, aufs has to decide the target branch for file creation or copy-up. By default, the highest writable branch which has the parent (or ancestor) dir of the target file is chosen (top-down-parent policy).
By user's request, aufs has some other policies to select the writable branch, round-robin and most-free-space policies for file creation, and top-down-parent, bottom-up-parent and bottom-up policies for copy-up.

As expected, the round-robin policy selects in circular. When you have two writable branches and creates 10 new files, 5 files will be created for each branch. mkdir(2) systemcall is an exception. When you create 10 new directories, all are created on the same branch. And the most-free-space policy selects the one which has most free space among the writable branches. The amount of free space will be checked by aufs internally, and users can specify its time interval.

The policies for copy-up is more simple,
top-down-parent is equivalent to the same named on in create policy, bottom-up-parent selects the writable branch where the parent dir exists and the nearest upper one from the copyup-source, bottom-up selects the nearest upper writable branch from the copyup-source, regardless the existence of the parent dir.

There are some rules or exceptions to apply these policies.
- If there is a readonly branch above the policy-selected branch and the parent dir is marked as opaque (a variation of whiteout), or the target (creating) file is whiteout-ed on the upper readonly branch, then the policy will be ignored and the target file will be created on the nearest upper writable branch than the readonly branch.
- If there is a writable branch above the policy-selected branch and the parent dir is marked as opaque or the target file is whiteouted on the branch, then the policy will be ignored and the target file will be created on the highest one among the upper writable branches who has diropq or whiteout. In case of whiteout, aufs removes it as usual.
- link(2) and rename(2) systemcalls are exceptions in every policy. They try selecting the branch where the source exists as possible since copyup a large file will take long time. If it can't be, ie. the branch where the source exists is readonly, then they will follow the copyup policy.
- There is an exception for rename(2) when the target exists. If the rename target exists, aufs compares the index of the branches where the source and the target exists and selects the higher one. If the selected branch is readonly, then aufs follows the copyup policy.

o revert everything after an error on a branch in a single systemcall, and remove/rename dir -- temporary name and EXDEV
----------------------------------------------------------------------
Since aufs handles several filesystems internally, it is important to revert everything after an error happend on a branch internally, and returns the expected error of systemcall.
To do this, aufs selects only one target writable branch for create/remove operations and didn't change other
branches. Additionally aufs has to pay attention the order of internal operaion to make it revertible at any point. The general rule is here.

For creation,
- lock the real dir on the target branch
- lookup a whiteout for the target
- actual creation of the target
- unlink the whiteout for it, if exists
- d_instantiate()
- unlock the real dir

For removal,
- lock the real dir on the target branch
- create a whiteout for the target, if needed
- actual removal of the target, if it exists on the target branch
- unlock the real dir

Generally rename(2) can handle the destination dir which already exists, and aufs_rename() basically calls vfs_rename() on the writable branch. When an empty dst-dir exists on the lower branch(es), aufs has to make the renamed dir opaque (which is a variation of whiteout and called 'diropq') by creating a special 'diropq' file under the renamed dir.
If aufs cannot create the 'diropq' file, aufs cannot revert the previous vfs_rename().

To address this problem, aufs renames the existing dst-dir to the temporary new whiteout-ed name before the actual vfs_rename(). After all operations succeeded, aufs_rename() passes the temporary name to another kernel thread and returns.
The kernel thread removes the temporary name later.
If aufs cannot create the 'diropq' file, it tries vfs_rename() the src-dir to its old name, and the temporary name to the old dst-dir name.

This approach is implemented in aufs_rmdir() too (except the branch is NFS), and very effective when the target dir has many whiteouts since aufs has to unlink the child whiteouts before calling vfs_rmdir(). It may take long time and user has to wait for the completion of _logically_ empty dir is removed.
With this approach, user don't need to wait so long time. But the number of child whiteout is not so much, nobody likes this overhead. So aufs has an option which specifies the threshold of the number of child whiteouts.

In rename(2), when the target dir has its child on several branches, aufs_rename() returns -EXDEV, since it may cause many/long internal copy-up. Generally mv(1) supports this case and retries create/copy for each children.

Thank you reading this long and my broken English.

Junjiro Okajima

Thanx Tomas,

Tomas M:

Strictly speaking, it is not a limitaion of /proc/mounts but mount(8) or /etc/mtab. I corrected the aufs document and myself a few weeks ago.

Junjiro Okajima

	:::

o readdir -- virtual dir block on memory (VDIR)
----------------------------------------------------------------------
This is an approach I posted a few months ago replying UnionMount's post. It constructs a virtual dir block on memory. For readdir, aufs calls vfs_readdir() internally for each lower dirs, merges their entries with eliminating the whiteout-ed ones, and gives it the the file (dir) object. So the file object has its entry list until it is closed. The entry list will be updated when the file position is zero and becomes old. This decision is made in aufs automatically.

It may consume rather large memory and cpu cycles. To reduce the number of memory allocations, the implementation became rather tricky .

Some people may call it can be a security hole or DoS attack since the opened and once readdir-ed dir (file object) holds its entry list and becomes a pressure for system memory. But I'd say it is similar to files under /proc or /sys. The virtual files on procfs and sysfs also holds a memory page (generally) while they are opened. When an idea to reduce memory for them is introduced, it will be applied to aufs too.

o policies for selecting one among multiple writable branches, parent-dir, round-robin and most-free-space
----------------------------------------------------------------------
When the number of writable branch is more than one, aufs has to decide the target branch for file creation or copy-up. By default, the highest writable branch which has the parent (or ancestor) dir of the target file is chosen (top-down-parent policy).
By user's request, aufs has some other policies to select the writable branch, round-robin and most-free-space policies for file creation, and top-down-parent, bottom-up-parent and bottom-up policies for copy-up.

As expected, the round-robin policy selects in circular. When you have two writable branches and creates 10 new files, 5 files will be created for each branch. mkdir(2) systemcall is an exception. When you create 10 new directories, all are created on the same branch. And the most-free-space policy selects the one which has most free space among the writable branches. The amount of free space will be checked by aufs internally, and users can specify its time interval.

The policies for copy-up is more simple,
top-down-parent is equivalent to the same named on in create policy, bottom-up-parent selects the writable branch where the parent dir exists and the nearest upper one from the copyup-source, bottom-up selects the nearest upper writable branch from the copyup-source, regardless the existence of the parent dir.

There are some rules or exceptions to apply these policies.
- If there is a readonly branch above the policy-selected branch and the parent dir is marked as opaque (a variation of whiteout), or the target (creating) file is whiteout-ed on the upper readonly branch, then the policy will be ignored and the target file will be created on the nearest upper writable branch than the readonly branch.
- If there is a writable branch above the policy-selected branch and the parent dir is marked as opaque or the target file is whiteouted on the branch, then the policy will be ignored and the target file will be created on the highest one among the upper writable branches who has diropq or whiteout. In case of whiteout, aufs removes it as usual.
- link(2) and rename(2) systemcalls are exceptions in every policy. They try selecting the branch where the source exists as possible since copyup a large file will take long time. If it can't be, ie. the branch where the source exists is readonly, then they will follow the copyup policy.
- There is an exception for rename(2) when the target exists. If the rename target exists, aufs compares the index of the branches where the source and the target exists and selects the higher one. If the selected branch is readonly, then aufs follows the copyup policy.

o revert everything after an error on a branch in a single systemcall, and remove/rename dir -- temporary name and EXDEV
----------------------------------------------------------------------
Since aufs handles several filesystems internally, it is important to revert everything after an error happend on a branch internally, and returns the expected error of systemcall.
To do this, aufs selects only one target writable branch for create/remove operations and didn't change other
branches. Additionally aufs has to pay attention the order of internal operaion to make it revertible at any point. The general rule is here.

For creation,
- lock the real dir on the target branch
- lookup a whiteout for the target
- actual creation of the target
- unlink the whiteout for it, if exists
- d_instantiate()
- unlock the real dir

For removal,
- lock the real dir on the target branch
- create a whiteout for the target, if needed
- actual removal of the target, if it exists on the target branch
- unlock the real dir

Generally rename(2) can handle the destination dir which already exists, and aufs_rename() basically calls vfs_rename() on the writable branch. When an empty dst-dir exists on the lower branch(es), aufs has to make the renamed dir opaque (which is a variation of whiteout and called 'diropq') by creating a special 'diropq' file under the renamed dir.
If aufs cannot create the 'diropq' file, aufs cannot revert the previous vfs_rename().

To address this problem, aufs renames the existing dst-dir to the temporary new whiteout-ed name before the actual vfs_rename(). After all operations succeeded, aufs_rename() passes the temporary name to another kernel thread and returns.
The kernel thread removes the temporary name later.
If aufs cannot create the 'diropq' file, it tries vfs_rename() the src-dir to its old name, and the temporary name to the old dst-dir name.

This approach is implemented in aufs_rmdir() too (except the branch is NFS), and very effective when the target dir has many whiteouts since aufs has to unlink the child whiteouts before calling vfs_rmdir(). It may take long time and user has to wait for the completion of _logically_ empty dir is removed.
With this approach, user don't need to wait so long time. But the number of child whiteout is not so much, nobody likes this overhead. So aufs has an option which specifies the threshold of the number of child whiteouts.

In rename(2), when the target dir has its child on several branches, aufs_rename() returns -EXDEV, since it may cause many/long internal copy-up. Generally mv(1) supports this case and retries create/copy for each children.

Thank you reading this long and my broken English.

Junjiro Okajima

	:::

I have posted some of ideas, design or approaches which are implemented in AUFS stackable filesystem about a month before.
While I have a plan to implement some more features still, the current AUFS status is better and used many people for years.
Since I have received requests to submit AUFS into the mainline more than once, Now I'd ask you to include AUFS into mainline. But the source is large (see below).
Should I send all of these files to this ML, or ask you to download them from CVS?
If AUFS was much smaller, I would send files here without asking.

Junjiro Okajima

----------------------------------------------------------------------

$ wc -l fs/aufs25/*.[ch]
    56 fs/aufs25/aufs.h
   109 fs/aufs25/br_fuse.c
   391 fs/aufs25/br_nfs.c
    69 fs/aufs25/br_xfs.c
   932 fs/aufs25/branch.c
   345 fs/aufs25/branch.h
  1043 fs/aufs25/cpup.c
    82 fs/aufs25/cpup.h
   246 fs/aufs25/dcsub.c
    54 fs/aufs25/dcsub.h
   485 fs/aufs25/debug.c
   210 fs/aufs25/debug.h
  1020 fs/aufs25/dentry.c
   384 fs/aufs25/dentry.h
   425 fs/aufs25/dinfo.c
   573 fs/aufs25/dir.c
   146 fs/aufs25/dir.h
   113 fs/aufs25/dlgt.c
   597 fs/aufs25/export.c
   661 fs/aufs25/f_op.c
   826 fs/aufs25/file.c
   246 fs/aufs25/file.h
   185 fs/aufs25/finfo.c
   708 fs/aufs25/hin_or_dlgt.c
   188 fs/aufs25/hinode.h
  1114 fs/aufs25/hinotify.c
   844 fs/aufs25/i_op.c
   828 fs/aufs25/i_op_add.c
   582 fs/aufs25/i_op_del.c
   832 fs/aufs25/i_op_ren.c
   290 fs/aufs25/iinfo.c
   425 fs/aufs25/inode.c
   336 fs/aufs25/inode.h
   307 fs/aufs25/misc.c
   201 fs/aufs25/misc.h
   243 fs/aufs25/module.c
    78 fs/aufs25/module.h
  1489 fs/aufs25/opts.c
   245 fs/aufs25/opts.h
   349 fs/aufs25/plink.c
   111 fs/aufs25/robr.c
   268 fs/aufs25/sbinfo.c
   906 fs/aufs25/super.c
   409 fs/aufs25/super.h
   107 fs/aufs25/sysaufs.c
   150 fs/aufs25/sysaufs.h
   498 fs/aufs25/sysfs.c
   112 fs/aufs25/sysrq.c
   963 fs/aufs25/vdir.c
   653 fs/aufs25/vfsub.c
   493 fs/aufs25/vfsub.h
   693 fs/aufs25/wbr_policy.c
  1058 fs/aufs25/whout.c
   140 fs/aufs25/whout.h
   321 fs/aufs25/wkq.c
   160 fs/aufs25/wkq.h
  1304 fs/aufs25/xino.c
 26603 total
----------------------------------------------------------------------

(http://aufs.sf.net)

Aufs -- Another Unionfs
Junjiro Okajima

# $Id: README,v 1.79 2008/05/04 23:55:14 sfjro Exp $

0. Introduction
----------------------------------------
In the early days, aufs was entirely re-designed and re-implemented Unionfs Version 1.x series. After many original ideas, approaches, improvements and implementations, it becomes totally different from Unionfs while keeping the basic features.
Recently, Unionfs Version 2.x series begin taking some of same approaches to aufs's.
Unionfs is being developed by Professor Erez Zadok at Stony Brook University and his team.
If you don't know Unionfs, I recommend you becoming familiar with it before using aufs. Some terminology in aufs follows Unionfs's.

Bug reports (including my broken English), suggestions, comments and donations are always welcome. Your bug report may help other users, including future users. Especially the bug report which doesn't follow unix/linux filesystem's semantics is important.

1. Features
----------------------------------------
- unite several directories into a single virtual filesystem. The member directory is called as a branch.
- you can specify the permission flags to the branch, which are 'readonly', 'readwrite' and 'whiteout-able.'
- by upper writable branch, internal copyup and whiteout, files/dirs on readonly branch are modifiable logically.
- dynamic branch manipulation, add, del.
- etc... see Unionfs in detail.

Also there are many enhancements in aufs, such as:
- safer and faster
- keep inode number by external inode number table
- keep the timestamps of file/dir in internal copyup operation
- seekable directory, supporting NFS readdir.
- support mmap(2) including /proc/PID/exe symlink, without page-copy
- whiteout is hardlinked in order to reduce the consumption of inodes on branch
- do not copyup, nor create a whiteout when it is unnecessary
- revert a single systemcall when an error occurs in aufs
- remount interface instead of ioctl
- maintain /etc/mtab by an external shell script, /sbin/mount.aufs.
- loopback mounted filesystem as a branch
- kernel thread for removing the dir who has a plenty of whiteouts
- support copyup sparse file (a file which has a 'hole' in it)
- default permission flags for branches
- selectable permission flags for ro branch, whether whiteout can exist or not
- export via NFS.
- support <sysfs>/fs/aufs.
- support multiple writable branches, some policies to select one among multiple writable branches.
- a new semantics for link(2) and rename(2) to support multiple writable branches.
- a delegation of the internal branch access to support task I/O accounting, which also supports Linux Security Modules (LSM) mainly for Suse AppArmor.
- nested mount, i.e. aufs as readonly no-whiteout branch of another aufs.
- copyup-on-open or copyup-on-write
- show-whiteout mode
- show configuration even out of kernel tree
- no glibc changes are required.
- and more... see aufs manual in detail

Aufs is in still development stage, especially:
- pseudo hardlink (hardlink over branches)
- allow a direct access manually to a file on branch, e.g. bypassing aufs. including NFS or remote filesystem branch.
- refine xino and revalidate
- pseudo-link in NFS-exporting

(current work)
- reorder the branch index without del/re-add.
- permanent xino files

(next work)
- an option for refreshing the opened files after add/del branches
- 'move' policy for copy-up between two writable branches, after checking free space.
- ioctl to manipulate file between branches.
- and documentation

(just an idea)
- remount option copy/move between two branches. (unnecessary?)
- O_DIRECT (unnecessary?)
- light version, without branch manipulation. (unnecessary?)
- SMP, because I don't have such machine. But several users reported aufs is working fine on SMP machines.
- copyup in userspace
- inotify in userspace
- xattr, acl

2. Download
----------------------------------------
CVS tree is in aufs project of SourceForge.
Here is simple instructions to get aufs source files. It is recommended to refer to the document about CVS on SourceForge.

	$ mkdir aufs.wcvs
	$ cd aufs.wcvs
	$ cvs -d:pserver:[email protected]:/cvsroot/aufs login
	(CVS password is empty)
	$ cvs -z3 -d:pserver:[email protected]:/cvsroot/aufs co aufs

In order to update files after the first checkout,

	$ cd aufs.wcvs/aufs
	$ cvs update
Do not forget -A option for 'cvs update' if you have ever 'cvs update' with
specifying a file version.

In order to see what the difference between two versions (two dates) is,

	$ cd aufs.wcvs/aufs
	$ cvs diff -D20061212 -D20061219

Usually I am updating CVS tree on every Monday.
I always try putting the stable version in CVS, so you can try CVS instead of SourceForge File Release. And every changes are summarized and reported to aufs-users at lists.sourceforge.net ML. I'd like to recommend you to join this ML.

(snip)

Yup, prepare a patch series and email them out. cc linux-kernel too.

We'd be interested in hearing if aufs addresses any of the shortcomings which reviewers identified in unionfs and if so, how.

Nobody is set up to handle cvs, sorry. It would be worth the time to migrate it to git.

Andrew Morton:

Ok, I will send these patches to two MLs.
I'd ask linux-kernel people to read previous posts too.
http://marc.info/?l=linux-fsdevel&m=120716468102834&w=2
http://marc.info/?l=linux-fsdevel&m=120720593114664&w=2

 Documentation/filesystems/aufs/README      |  374 +++++++
 Documentation/filesystems/aufs/aufs.5      | 1608 ++++++++++++++++++++++++++++
 Documentation/filesystems/aufs/aulchown.c  |   29 +
 Documentation/filesystems/aufs/auplink     |  170 +++
 Documentation/filesystems/aufs/mount.aufs  |  205 ++++
 Documentation/filesystems/aufs/umount.aufs |   33 +
 fs/Kconfig                                 |    2 +
 fs/Makefile                                |    1 +
 fs/aufs/Kconfig                            |  203 ++++
 fs/aufs/Makefile                           |   57 +
 fs/aufs/aufs.h                             |   56 +
 fs/aufs/br_fuse.c                          |  109 ++
 fs/aufs/br_nfs.c                           |  391 +++++++
 fs/aufs/br_xfs.c                           |   69 ++
 fs/aufs/branch.c                           |  933 ++++++++++++++++
 fs/aufs/branch.h                           |  345 ++++++
 fs/aufs/cpup.c                             | 1043 ++++++++++++++++++
 fs/aufs/cpup.h                             |   82 ++
 fs/aufs/dcsub.c                            |  246 +++++
 fs/aufs/dcsub.h                            |   54 +
 fs/aufs/debug.c                            |  485 +++++++++
 fs/aufs/debug.h                            |  210 ++++
 fs/aufs/dentry.c                           | 1020 ++++++++++++++++++
 fs/aufs/dentry.h                           |  384 +++++++
 fs/aufs/dinfo.c                            |  425 ++++++++
 fs/aufs/dir.c                              |  573 ++++++++++
 fs/aufs/dir.h                              |  146 +++
 fs/aufs/dlgt.c                             |  113 ++
 fs/aufs/export.c                           |  597 +++++++++++
 fs/aufs/f_op.c                             |  665 ++++++++++++
 fs/aufs/file.c                             |  822 ++++++++++++++
 fs/aufs/file.h                             |  246 +++++
 fs/aufs/finfo.c                            |  185 ++++
 fs/aufs/hin_or_dlgt.c                      |  708 ++++++++++++
 fs/aufs/hinode.h                           |  188 ++++
 fs/aufs/hinotify.c                         | 1114 +++++++++++++++++++
 fs/aufs/i_op.c                             |  844 +++++++++++++++
 fs/aufs/i_op_add.c                         |  828 ++++++++++++++
 fs/aufs/i_op_del.c                         |  582 ++++++++++
 fs/aufs/i_op_ren.c                         |  832 ++++++++++++++
 fs/aufs/iinfo.c                            |  290 +++++
 fs/aufs/inode.c                            |  425 ++++++++
 fs/aufs/inode.h                            |  336 ++++++
 fs/aufs/misc.c                             |  307 ++++++
 fs/aufs/misc.h                             |  201 ++++
 fs/aufs/module.c                           |  243 +++++
 fs/aufs/module.h                           |   78 ++
 fs/aufs/opts.c                             | 1493 ++++++++++++++++++++++++++
 fs/aufs/opts.h                             |  245 +++++
 fs/aufs/plink.c                            |  349 ++++++
 fs/aufs/robr.c                             |  111 ++
 fs/aufs/sbinfo.c                           |  268 +++++
 fs/aufs/super.c                            |  891 +++++++++++++++
 fs/aufs/super.h                            |  410 +++++++
 fs/aufs/sysaufs.c                          |  104 ++
 fs/aufs/sysaufs.h                          |  150 +++
 fs/aufs/sysfs.c                            |  459 ++++++++
 fs/aufs/sysrq.c                            |  112 ++
 fs/aufs/vdir.c                             |  963 +++++++++++++++++
 fs/aufs/vfsub.c                            |  653 +++++++++++
 fs/aufs/vfsub.h                            |  493 +++++++++
 fs/aufs/wbr_policy.c                       |  693 ++++++++++++
 fs/aufs/whout.c                            | 1058 ++++++++++++++++++
 fs/aufs/whout.h                            |  140 +++
 fs/aufs/wkq.c                              |  321 ++++++
 fs/aufs/wkq.h                              |  160 +++
 fs/aufs/xino.c                             | 1249 +++++++++++++++++++++
 fs/namei.c                                 |    2 +-
 include/linux/aufs_type.h                  |  111 ++
 include/linux/lockdep.h                    |    4 +
 include/linux/namei.h                      |    1 +
 71 files changed, 29296 insertions(+), 1 deletions(-)

Junjiro Okajima

These pathces are against linux-trees.git v2.6.25-mm1.

Junjiro Okajima