File change notification (enhanced dnotify)
Quoted 6 lines · =?ISO-8859-1?Q?R=FCdiger_Klaehn?= <[email protected]> said:
Quoted 4 lines · Horst von Brand
Quoted 3 lines · =?ISO-8859-1?Q?R=FCdiger_Klaehn?= <[email protected]> said:
I am working on a mechanism to let programs watch for file system changes in large directory trees or on the whole system. Since my last post in january I have been trying various approaches.
Quoted 7 lines
Quoted 3 lines
How do you propose to handle the fact that there are changes to _files_, which happen to be pointed to by entries in directories? There is no "change in the directory tree" in Unix...
Of course it is files that change. But as you say each file is pointed to by one or more dentry, so I use the dentry hierarchy to propagate the information about the change. Just like the old dnotify.
dentries just keep the path travelled by hard links to get to the file in memory for fast future access. So if you have, say:
dir1 dir2
| |
. .
. .
. .
\ /somefile
and you referenced somefile by the path through dir1, if you monitor dir2 you won't notice the change. There is no on-disk data to trace back through all the directories that reference the file, and reading all of the filesystem's metadata to find this out is ludicrous (ever seen fsck(8) taking an hour or so to make much the same?).
Signature · 4 lines
Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Quoted 28 lines · Horst von Brand
Quoted 7 lines · =?ISO-8859-1?Q?R=FCdiger_Klaehn?= <[email protected]> said:
Quoted 3 lines · Horst von Brand
Quoted 1 line · =?ISO-8859-1?Q?R=FCdiger_Klaehn?= <[email protected]> said:
I am working on a mechanism to let programs watch for file system changes in large directory trees or on the whole system. Since my last post in january I have been trying various approaches.
Quoted 6 lines
Quoted 3 lines
How do you propose to handle the fact that there are changes to _files_, which happen to be pointed to by entries in directories? There is no "change in the directory tree" in Unix...
Of course it is files that change. But as you say each file is pointed to by one or more dentry, so I use the dentry hierarchy to propagate the information about the change. Just like the old dnotify.
dentries just keep the path travelled by hard links to get to the file in memory for fast future access. So if you have, say:
dir1 dir2 | | . . . . . . \ / somefile
Quoted 5 lines
and you referenced somefile by the path through dir1, if you monitor dir2 you won't notice the change. There is no on-disk data to trace back through all the directories that reference the file, and reading all of the filesystem's metadata to find this out is ludicrous (ever seen fsck(8) taking an hour or so to make much the same?).
I am aware of that. As I mentioned, this approach does not work with hard links, just like the original dnotify.
From the current "Documentation/dnotify.txt":
"In order to make the impact on the file system code as small as possible, the problem of hard links to files has been ignored."
There would be some ways to solve this for hard links. But I don't think that it would be worth it since it would involve a big performance overhead for little gain.
Note that if you watch for changes in the root of a file system, you will get notified exactly once for each file change in the file system regardless of hard links.
In your example if you have one file which can be accessed via two different paths "/dir1/somefile" and "/dir2/somefile" and you watch "/" you would get notified for "/dir1/somefile" or "/dir2/somefile" depending on how the changing program accesses the file.
Figuring out that "/dir1/somefile" and "/dir2/somefile" refer to the same file should IMHO be done in userspace. If inode numbers were unique and persistent on all file systems it might be possible to do this efficiently in kernel space, but unfortunately this is not the case.
My original approach assumed that inode numbers were unique, and it would have worked with hard links. But I think it is much more important to have a mechanism that works for all file systems than to solve the problem of hard links.
best regards,
Rüdiger
By the way: I just made a small website for my enhanced dnotify mechanism. I will post my latest code there. It can be found at
<http://www.lambda-computing.com/~rudi/dnotify/>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Rüdiger Klaehn wrote:
| | My original approach assumed that inode numbers were unique, and it | would have worked with hard links. But I think it is much more important | to have a mechanism that works for all file systems than to solve the | problem of hard links. |
Inode numbers are guaranteed to be unique on a given filesystem other than for hard links.. Where is this assumption broken otherwise?
Mike Waychison
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFAXxdJdQs4kOxk3/MRAjdqAKCIJ20IxRgq0PmBcV7IIKITI9FhRQCggZDm IvcRYGtqB5ss+jhoLNIj2So=
=J6qR
-----END PGP SIGNATURE-----
Quoted 12 lines · Mike Waychison
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Rüdiger Klaehn wrote:
| | My original approach assumed that inode numbers were unique, and it | would have worked with hard links. But I think it is much more important | to have a mechanism that works for all file systems than to solve the | problem of hard links. |
Quoted 3 lines
Inode numbers are guaranteed to be unique on a given filesystem other than for hard links.. Where is this assumption broken otherwise?
To quote jan harkes, who repiled to my original proposal:
"Inode number are not necessarily unique per filesystem. Any filesystem that uses iget4 can have several objects that have the same inode number. For instance, Coda uses 128-bit file-identifiers and the i_ino number is a simple hash that is 'typically' unique. There are also filesystems that invent inode numbers whenever inodes are brought into the cache, but which have no persistency when the inode_cache is pruned. So the next time you see the same object, it could have a different (unique) inode number."
But I think the current approach is much better anyway since it does work for a single directory and its subdirectories instead of globally like my initial attempt. If you want to catch hard link issues you will have to watch the root of the file system, but that is still not that expensive. And on "sane" file systems that use unique inode numbers in userspace to solve the hard link problem.
best regards,
Rüdiger
Quoted 8 lines · Mike Waychison
R�diger Klaehn wrote:
| My original approach assumed that inode numbers were unique, and it | would have worked with hard links. But I think it is much more important | to have a mechanism that works for all file systems than to solve the | problem of hard links.Inode numbers are guaranteed to be unique on a given filesystem other than for hard links.. Where is this assumption broken otherwise?
On some non-Unix filesystems there isn't an equivalent to (invariant) inode numbers.
Signature · 4 lines
Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513