Skip to contentSkip to conversation

Problem with inotify

DISCUSSION David Gómez 6 participants 28 messages Jun 30, 2005 – Jul 5, 2005

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

Hi all ;),

I just patched 2.6.12 kernel with the inotify latest patch (inotify-0.23-rml-2.6.12-14.patch). Inotify is working ok with the test program provided in inotify-utils but... I can no longer mount my IDE cdrom devices :(. Each time i try to mount a disc, the mount proccess get stuck in D state. I don't see what's the relation between inotify and IDE devices, but if i switch back to the unpatched 2.6.12, mounting works again.

Any ideas? Thanks

Signature · 1 line
David Gómez                                      Jabber ID: [email protected]
#1 lore raw
Quoted 6 lines · David Gómez

I just patched 2.6.12 kernel with the inotify latest patch (inotify-0.23-rml-2.6.12-14.patch). Inotify is working ok with the test program provided in inotify-utils but... I can no longer mount my IDE cdrom devices :(. Each time i try to mount a disc, the mount proccess get stuck in D state. I don't see what's the relation between inotify and IDE devices, but if i switch back to the unpatched 2.6.12, mounting works again.

Very weird.

Did everything work with an earlier inotify?

Does wchan show anything useful (ps -ewo user,pid,command,wchan)?

Does it mount successfully once, and then subsequent mounts get suck, or does even the first mount get stuck in D?

Thanks,

Robert Love

#2 lore raw

Hi Robert,

Quoted 8 lines · Robert Love
Quoted 6 lines

I just patched 2.6.12 kernel with the inotify latest patch (inotify-0.23-rml-2.6.12-14.patch). Inotify is working ok with the test program provided in inotify-utils but... I can no longer mount my IDE cdrom devices :(. Each time i try to mount a disc, the mount proccess get stuck in D state. I don't see what's the relation between inotify and IDE devices, but if i switch back to the unpatched 2.6.12, mounting works again.

Very weird.

Indeed.

Quoted 1 line

Did everything work with an earlier inotify?

It's the first notify i've tested, so i cannot compare with previous versions.

Quoted 1 line

Does wchan show anything useful (ps -ewo user,pid,command,wchan)?

I have to reboot to test it. I'll do it and told you if i see anything weird in wchan for the mount process.

Quoted 2 lines

Does it mount successfully once, and then subsequent mounts get suck, or does even the first mount get stuck in D?

The first mount get stuck, and subsequent mounts too. I tried to mount my CD-writer and after that my DVD-writer and i got two mount D processes.

Thanks for the quick answer ;)

Signature · 1 line
David Gómez                                      Jabber ID: [email protected]
#3 lore raw
In reply to Robert Love

Hi Robert,

Quoted 7 lines · Robert Love
Quoted 6 lines

I just patched 2.6.12 kernel with the inotify latest patch (inotify-0.23-rml-2.6.12-14.patch). Inotify is working ok with the test program provided in inotify-utils but... I can no longer mount my IDE cdrom devices :(. Each time i try to mount a disc, the mount proccess get stuck in D state. I don't see what's the relation between inotify and IDE devices, but if i switch back to the unpatched 2.6.12, mounting works again.

Quoted 5 lines

Very weird.

Did everything work with an earlier inotify?

Does wchan show anything useful (ps -ewo user,pid,command,wchan)?

I tested it again, wchan says "inode_wait"...

Signature · 1 line
David Gómez                                      Jabber ID: [email protected]
#4 lore raw

Hi,

Quoted 8 lines · On Thu, 30 Jun 2005, David [utf-8] Gómez wrote:
Quoted 7 lines · Robert Love
Quoted 6 lines

I just patched 2.6.12 kernel with the inotify latest patch (inotify-0.23-rml-2.6.12-14.patch). Inotify is working ok with the test program provided in inotify-utils but... I can no longer mount my IDE cdrom devices :(. Each time i try to mount a disc, the mount proccess get stuck in D state. I don't see what's the relation between inotify and IDE devices, but if i switch back to the unpatched 2.6.12, mounting works again.

Quoted 7 lines
Quoted 5 lines

Very weird.

Did everything work with an earlier inotify?

Does wchan show anything useful (ps -ewo user,pid,command,wchan)?

I tested it again, wchan says "inode_wait"...

Do you have any ntfs volumes mounted by any chance?

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#5 lore raw

Hi Anton,

Quoted 3 lines · Anton Altaparmakov
Quoted 1 line

I tested it again, wchan says "inode_wait"...

Do you have any ntfs volumes mounted by any chance?

No, just ext2

Signature · 1 line
David Gómez                                      Jabber ID: [email protected]
#6 lore raw
Quoted 6 lines · On Thu, 30 Jun 2005, David [utf-8] Gómez wrote:
Quoted 3 lines · Anton Altaparmakov
Quoted 1 line

I tested it again, wchan says "inode_wait"...

Do you have any ntfs volumes mounted by any chance?

No, just ext2

Ok, the ntfs deadlock doesn't affect you then.

So there appears to be a change in the VFS which causes deadlocks associated with both mounting and umounting of filesystems, and this is either introduced by inotify or just made more likely (i.e. 100% of time) by inotify but the change is elsewhere.

Given you see inode_wait it suggests it is a simillar problem to the one I found in ntfs. For a detailed description of how that deadlock comes about see the patch fixing it (appended to end of this email).

So what we probably are somehow getting is that say we have a lock (perhaps i_sem or inode_lock?) and an inode. And we have two processes which do:

Process 1 Process 2
lock the inode (I_LOCK)
take the lock
try to take the lock -> block

				wait_on_inode() -> block

Deadlock.

The question is how and where does that happen?

So far it was only triggering at umount time so it is very interesting that you are seeing it at mount time as well.

ps. Of course it could actually be one single process which takes I_LOCK and then ends up waiting on the inode later on which of course never happens...

Best regards,

Anton

Signature · 226 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

--- This is the patch with description of the deadlock in ntfs. ---

From [email protected] Sun Jun 26 20:33:02 2005
Date: Sun, 26 Jun 2005 12:32:27 -0700
From: [email protected]
To: [email protected], [email protected], [email protected]
Subject: fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation.patch added to -mm tree


The patch titled

     Fix soft lockup due to NTFS: VFS part and explanation

has been added to the -mm tree.  Its filename is

     fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation.patch

Patches currently in -mm which might be from [email protected] are

fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation.patch



From: Anton Altaparmakov <[email protected]>

Something has changed in the core kernel such that we now get concurrent
inode write outs, one e.g via pdflush and one via sys_sync or whatever. 
This causes a nasty deadlock in ntfs.  The only clean solution
unfortunately requires a minor vfs api extension.

First the deadlock analysis:

Prerequisive knowledge: NTFS has a file $MFT (inode 0) loaded at mount
time.  The NTFS driver uses the page cache for storing the file contents as
usual.  More interestingly this file contains the table of on-disk inodes
as a sequence of MFT_RECORDs.  Thus NTFS driver accesses the on-disk inodes
by accessing the MFT_RECORDs in the page cache pages of the loaded inode
$MFT.

The situation: VFS inode X on a mounted ntfs volume is dirty.  For same
inode X, the ntfs_inode is dirty and thus corresponding on-disk inode,
which is as explained above in a dirty PAGE_CACHE_PAGE belonging to the
table of inodes ($MFT, inode 0).

What happens:

Process 1: sys_sync()/umount()/whatever...  calls __sync_single_inode() for
$MFT -> do_writepages() -> write_page for the dirty page containing the
on-disk inode X, the page is now locked -> ntfs_write_mst_block() which
clears PageUptodate() on the page to prevent anyone else getting hold of it
whilst it does the write out (this is necessary as the on-disk inode needs
"fixups" applied before the write to disk which are removed again after the
write and PageUptodate is then set again).  It then analyses the page
looking for dirty on-disk inodes and when it finds one it calls
ntfs_may_write_mft_record() to see if it is safe to write this on-disk
inode.  This then calls ilookup5() to check if the corresponding VFS inode
is in icache().  This in turn calls ifind() which waits on the inode lock
via wait_on_inode whilst holding the global inode_lock.

Process 2: pdflush results in a call to __sync_single_inode for the same
VFS inode X on the ntfs volume.  This locks the inode (I_LOCK) then calls
write-inode -> ntfs_write_inode -> map_mft_record() -> read_cache_page() of
the page (in page cache of table of inodes $MFT, inode 0) containing the
on-disk inode.  This page has PageUptodate() clear because of Process 1
(see above) so read_cache_page() blocks when tries to take the page lock
for the page so it can call ntfs_read_page().

Thus Process 1 is holding the page lock on the page containing the on-disk
inode X and it is waiting on the inode X to be unlocked in ifind() so it
can write the page out and then unlock the page.

And Process 2 is holding the inode lock on inode X and is waiting for the 
page to be unlocked so it can call ntfs_readpage() or discover that 
Process 1 set PageUptodate() again and use the page.

Thus we have a deadlock due to ifind() waiting on the inode lock.

The only sensible solution: NTFS does not care whether the VFS inode is
locked or not when it calls ilookup5() (it doesn't use the VFS inode at
all, it just uses it to find the corresponding ntfs_inode which is of
course attached to the VFS inode (both are one single struct); and it uses
the ntfs_inode which is subject to its own locking so I_LOCK is irrelevant)
hence we want a modified ilookup5_nowait() which is the same as ilookup5()
but it does not wait on the inode lock.

Without such functionality I would have to keep my own ntfs_inode cache in
the NTFS driver just so I can find ntfs_inodes independent of their VFS
inodes which would be slow, memory and cpu cycle wasting, and incredibly
stupid given the icache already exists in the VFS.

Below is a patch that does the ilookup5_nowait() implementation in 
fs/inode.c and exports it.


ilookup5_nowait.diff:

Introduce ilookup5_nowait() which is basically the same as ilookup5() but 
it does not wait on the inode's lock (i.e. it omits the wait_on_inode() 
done in ifind()).

This is needed to avoid a nasty deadlock in NTFS.

Signed-off-by: Anton Altaparmakov <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
---

 fs/inode.c         |   45 +++++++++++++++++++++++++++++++++++++++------
 include/linux/fs.h |    3 +++
 2 files changed, 42 insertions(+), 6 deletions(-)

diff -puN fs/inode.c~fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation fs/inode.c
--- 25/fs/inode.c~fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation	2005-06-26 12:32:21.000000000 -0700
+++ 25-akpm/fs/inode.c	2005-06-26 12:32:21.000000000 -0700
@@ -757,6 +757,7 @@ EXPORT_SYMBOL(igrab);
  * @head:       the head of the list to search
  * @test:	callback used for comparisons between inodes
  * @data:	opaque data pointer to pass to @test
+ * @wait:	if true wait for the inode to be unlocked, if false do not
  *
  * ifind() searches for the inode specified by @data in the inode
  * cache. This is a generalized version of ifind_fast() for file systems where
@@ -771,7 +772,7 @@ EXPORT_SYMBOL(igrab);
  */
 static inline struct inode *ifind(struct super_block *sb,
 		struct hlist_head *head, int (*test)(struct inode *, void *),
-		void *data)
+		void *data, const int wait)
 {
 	struct inode *inode;
 
@@ -780,7 +781,8 @@ static inline struct inode *ifind(struct
 	if (inode) {
 		__iget(inode);
 		spin_unlock(&inode_lock);
-		wait_on_inode(inode);
+		if (likely(wait))
+			wait_on_inode(inode);
 		return inode;
 	}
 	spin_unlock(&inode_lock);
@@ -820,7 +822,7 @@ static inline struct inode *ifind_fast(s
 }
 
 /**
- * ilookup5 - search for an inode in the inode cache
+ * ilookup5_nowait - search for an inode in the inode cache
  * @sb:		super block of file system to search
  * @hashval:	hash value (usually inode number) to search for
  * @test:	callback used for comparisons between inodes
@@ -832,7 +834,38 @@ static inline struct inode *ifind_fast(s
  * identification of an inode.
  *
  * If the inode is in the cache, the inode is returned with an incremented
- * reference count.
+ * reference count.  Note, the inode lock is not waited upon so you have to be
+ * very careful what you do with the returned inode.  You probably should be
+ * using ilookup5() instead.
+ *
+ * Otherwise NULL is returned.
+ *
+ * Note, @test is called with the inode_lock held, so can't sleep.
+ */
+struct inode *ilookup5_nowait(struct super_block *sb, unsigned long hashval,
+		int (*test)(struct inode *, void *), void *data)
+{
+	struct hlist_head *head = inode_hashtable + hash(sb, hashval);
+
+	return ifind(sb, head, test, data, 0);
+}
+
+EXPORT_SYMBOL(ilookup5_nowait);
+
+/**
+ * ilookup5 - search for an inode in the inode cache
+ * @sb:		super block of file system to search
+ * @hashval:	hash value (usually inode number) to search for
+ * @test:	callback used for comparisons between inodes
+ * @data:	opaque data pointer to pass to @test
+ *
+ * ilookup5() uses ifind() to search for the inode specified by @hashval and
+ * @data in the inode cache. This is a generalized version of ilookup() for
+ * file systems where the inode number is not sufficient for unique
+ * identification of an inode.
+ *
+ * If the inode is in the cache, the inode lock is waited upon and the inode is
+ * returned with an incremented reference count.
  *
  * Otherwise NULL is returned.
  *
@@ -843,7 +876,7 @@ struct inode *ilookup5(struct super_bloc
 {
 	struct hlist_head *head = inode_hashtable + hash(sb, hashval);
 
-	return ifind(sb, head, test, data);
+	return ifind(sb, head, test, data, 1);
 }
 
 EXPORT_SYMBOL(ilookup5);
@@ -900,7 +933,7 @@ struct inode *iget5_locked(struct super_
 	struct hlist_head *head = inode_hashtable + hash(sb, hashval);
 	struct inode *inode;
 
-	inode = ifind(sb, head, test, data);
+	inode = ifind(sb, head, test, data, 1);
 	if (inode)
 		return inode;
 	/*
diff -puN include/linux/fs.h~fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation include/linux/fs.h
--- 25/include/linux/fs.h~fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation	2005-06-26 12:32:21.000000000 -0700
+++ 25-akpm/include/linux/fs.h	2005-06-26 12:32:21.000000000 -0700
@@ -1442,6 +1442,9 @@ extern int inode_needs_sync(struct inode
 extern void generic_delete_inode(struct inode *inode);
 extern void generic_drop_inode(struct inode *inode);
 
+extern struct inode *ilookup5_nowait(struct super_block *sb,
+		unsigned long hashval, int (*test)(struct inode *, void *),
+		void *data);
 extern struct inode *ilookup5(struct super_block *sb, unsigned long hashval,
 		int (*test)(struct inode *, void *), void *data);
 extern struct inode *ilookup(struct super_block *sb, unsigned long ino);
_
#7 lore raw
Quoted 4 lines · Anton Altaparmakov

So there appears to be a change in the VFS which causes deadlocks associated with both mounting and umounting of filesystems, and this is either introduced by inotify or just made more likely (i.e. 100% of time) by inotify but the change is elsewhere.

The mount case is baffling. Inotify has no interaction whatsoever with mounting. Unmounting, definitely--see fs/inotify.c ::
inotify_unmount_inodes().

Quoted 3 lines

Given you see inode_wait it suggests it is a simillar problem to the one I found in ntfs. For a detailed description of how that deadlock comes about see the patch fixing it (appended to end of this email).

This does not fix a general problem?

Quoted 4 lines

So what we probably are somehow getting is that say we have a lock (perhaps i_sem or inode_lock?) and an inode. And we have two processes which do:

Quoted 8 lines

Process 1 Process 2
lock the inode (I_LOCK)
take the lock
try to take the lock -> block

				wait_on_inode() -> block

Deadlock.

The question is how and where does that happen?

Well, in the unmount case (the more understable case, and also where the other bug report was triggered) we grab inode_lock and i_prune_sem. But we can sleep (we of course drop inode_lock first), like fs/inode.c ::
invalidate_list() does.

So something else could come in and wreck havoc. But where?

[ Attached is the latest inotify, against 2.6.13-rc1. ]

Robert Love

inotify!

inotify is intended to correct the deficiencies of dnotify, particularly its inability to scale and its terrible user interface:

* dnotify requires the opening of one fd per each directory that you intend to watch. This quickly results in too many open files and pins removable media, preventing unmount.
* dnotify is directory-based. You only learn about changes to directories. Sure, a change to a file in a directory affects the directory, but you are then forced to keep a cache of stat structures.
* dnotify's interface to user-space is awful. Signals?

inotify provides a more usable, simple, powerful solution to file change notification:

* inotify's interface is a device node, not SIGIO. You open a single fd to the device node, which is select()-able.
* inotify has an event that says "the filesystem that the item you were watching is on was unmounted."
* inotify can watch directories or files.

Inotify is currently used by Beagle (a desktop search infrastructure), Gamin (a FAM replacement), and other projects.

See Documentation/filesystems/inotify.txt.

 Documentation/filesystems/inotify.txt |  155 +++++
 fs/Kconfig                            |   13 
 fs/Makefile                           |    1 
 fs/attr.c                             |   33 -
 fs/compat.c                           |   12 
 fs/file_table.c                       |    3 
 fs/inode.c                            |    6 
 fs/inotify.c                          |  984 ++++++++++++++++++++++++++++++++++
 fs/namei.c                            |   30 -
 fs/nfsd/vfs.c                         |    6 
 fs/open.c                             |    3 
 fs/read_write.c                       |   15 
 fs/sysfs/file.c                       |    7 
 fs/xattr.c                            |    5 
 include/linux/fs.h                    |    6 
 include/linux/fsnotify.h              |  257 ++++++++
 include/linux/inotify.h               |  124 ++++
 include/linux/sched.h                 |    4 
 kernel/user.c                         |    4 
 19 files changed, 1604 insertions(+), 64 deletions(-)
Documentation/filesystems/inotify.txt +155 -0
diff -urN linux-2.6.13-rc1/Documentation/filesystems/inotify.txt linux/Documentation/filesystems/inotify.txt
@@ -0,0 +1,155 @@
+				    inotify
+	     a powerful yet simple file change notification system
+
+
+
+Document started 15 Mar 2005 by Robert Love <[email protected]>
+
+(i) User Interface
+
+Inotify is controlled by a device node, /dev/inotify.  If you do not use udev,
+this device may need to be created manually.
+
+First step in using inotify is to open the device file:
+
+	int dev_fd = open ("/dev/inotify", O_RDONLY);
+
+Change events are managed by "watches".  A watch is an (object,mask) pair where
+the object is a file or directory and the mask is a bitmask of one or more
+inotify events that the application wishes to receive.  See <linux/inotify.h>
+for valid events.  A watch is referenced by a watch descriptor, or wd.
+
+Watches are added via a file descriptor to the file.
+
+Watches on a directory will return events on any files inside of the directory.
+
+Adding a watch is simple,
+
+	/* 'wd' represents the watch on fd with mask */
+	struct inotify_request req = { fd, mask };
+	int wd = ioctl (dev_fd, INOTIFY_WATCH, &req);
+
+You can add a large number of files via something like
+
+	for each file to watch {
+		struct inotify_request req;
+		int file_fd;
+
+		file_fd = open (file, O_RDONLY);
+		if (fd < 0) {
+			perror ("open");
+			break;
+		}
+
+		req.fd = file_fd;
+		req.mask = mask;
+
+		wd = ioctl (dev_fd, INOTIFY_WATCH, &req);
+
+		close (fd);
+	}
+
+You can update an existing watch in the same manner, by passing in a new mask.
+
+An existing watch is removed via the INOTIFY_IGNORE ioctl, for example
+
+	ioctl (dev_fd, INOTIFY_IGNORE, wd);
+
+Events are provided in the form of an inotify_event structure that is read(2)
+from /dev/inotify.  The filename is of dynamic length and follows the struct.
+It is of size len.  The filename is padded with null bytes to ensure proper
+alignment.  This padding is reflected in len.
+
+You can slurp multiple events by passing a large buffer, for example
+
+	size_t len = read (fd, buf, BUF_LEN);
+
+Will return as many events as are available and fit in BUF_LEN.
+
+/dev/inotify is also select()- and poll()-able.
+
+You can find the size of the current event queue via the FIONREAD ioctl.
+
+All watches are destroyed and cleaned up on close.
+
+
+(ii) Internal Kernel Implementation
+
+Each open inotify device is associated with an inotify_device structure.
+
+Each watch is associated with an inotify_watch structure.  Watches are chained
+off of each associated device and each associated inode.
+
+See fs/inotify.c for the locking and lifetime rules.
+
+
+(iii) Rationale
+
+Q: What is the design decision behind not tying the watch to the open fd of
+   the watched object?
+
+A: Watches are associated with an open inotify device, not an open file.
+   This solves the primary problem with dnotify: keeping the file open pins
+   the file and thus, worse, pins the mount.  Dnotify is therefore infeasible
+   for use on a desktop system with removable media as the media cannot be
+   unmounted.
+
+Q: What is the design decision behind using an-fd-per-device as opposed to
+   an fd-per-watch?
+
+A: An fd-per-watch quickly consumes more file descriptors than are allowed,
+   more fd's than are feasible to manage, and more fd's than are optimally
+   select()-able.  Yes, root can bump the per-process fd limit and yes, users
+   can use epoll, but requiring both is a silly and extraneous requirement.
+   A watch consumes less memory than an open file, separating the number
+   spaces is thus sensible.  The current design is what user-space developers
+   want: Users open the device, once, and add n watches, requiring but one fd
+   and no twiddling with fd limits.  Opening /dev/inotify two thousand times
+   is silly.  If we can implement user-space's preferences cleanly--and we
+   can, the idr layer makes stuff like this trivial--then we should.
+
+   There are other good arguments.  With a single fd, there is a single
+   item to block on, which is mapped to a single queue of events.  The single
+   fd returns all watch events and also any potential out-of-band data.  If
+   every fd was a separate watch,
+
+   - There would be no way to get event ordering.  Events on file foo and
+     file bar would pop poll() on both fd's, but there would be no way to tell
+     which happened first.  A single queue trivially gives you ordering.  Such
+     ordering is crucial to existing applications such as Beagle.  Imagine
+     "mv a b ; mv b a" events without ordering.
+
+   - We'd have to maintain n fd's and n internal queues with state,
+     versus just one.  It is a lot messier in the kernel.  A single, linear
+     queue is the data structure that makes sense.
+
+   - User-space developers prefer the current API.  The Beagle guys, for
+     example, love it.  Trust me, I asked.  It is not a surprise: Who'd want
+     to manage and block on 1000 fd's via select?
+
+   - You'd have to manage the fd's, as an example: Call close() when you
+     received a delete event.
+
+   - No way to get out of band data.
+
+   - 1024 is still too low.  ;-)
+
+   When you talk about designing a file change notification system that
+   scales to 1000s of directories, juggling 1000s of fd's just does not seem
+   the right interface.  It is too heavy.
+
+Q: Why a device node?
+
+A: The poor user-space interface is the second biggest problem with dnotify.
+   Signals are a terrible, terrible interface for file notification.  Or for
+   anything, for that matter.  The idea solution, from all perspectives, is a
+   file descriptor-based one that allows basic file I/O and poll/select.
+   Obtaining the fd and managing the watches could have been done either via a
+   device file or a family of new system calls.  We decided to implement a
+   device file because adding three or four new system calls that mirrored
+   open, close, and ioctl seemed silly.  A character device makes sense from
+   user-space and was easy to implement inside of the kernel.
+
+   Additionally, it _is_ possible to open /dev/inotify more than once and
+   juggle more than one queue and thus more than one associated fd.
+
fs/attr.c +4 -29
diff -urN linux-2.6.13-rc1/fs/attr.c linux/fs/attr.c
@@ -10,7 +10,7 @@
 #include <linux/mm.h>
 #include <linux/string.h>
 #include <linux/smp_lock.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/fcntl.h>
 #include <linux/quotaops.h>
 #include <linux/security.h>
@@ -107,31 +107,8 @@
 out:
 	return error;
 }
-
 EXPORT_SYMBOL(inode_setattr);
 
-int setattr_mask(unsigned int ia_valid)
-{
-	unsigned long dn_mask = 0;
-
-	if (ia_valid & ATTR_UID)
-		dn_mask |= DN_ATTRIB;
-	if (ia_valid & ATTR_GID)
-		dn_mask |= DN_ATTRIB;
-	if (ia_valid & ATTR_SIZE)
-		dn_mask |= DN_MODIFY;
-	/* both times implies a utime(s) call */
-	if ((ia_valid & (ATTR_ATIME|ATTR_MTIME)) == (ATTR_ATIME|ATTR_MTIME))
-		dn_mask |= DN_ATTRIB;
-	else if (ia_valid & ATTR_ATIME)
-		dn_mask |= DN_ACCESS;
-	else if (ia_valid & ATTR_MTIME)
-		dn_mask |= DN_MODIFY;
-	if (ia_valid & ATTR_MODE)
-		dn_mask |= DN_ATTRIB;
-	return dn_mask;
-}
-
 int notify_change(struct dentry * dentry, struct iattr * attr)
 {
 	struct inode *inode = dentry->d_inode;
@@ -197,11 +174,9 @@
 	if (ia_valid & ATTR_SIZE)
 		up_write(&dentry->d_inode->i_alloc_sem);
 
-	if (!error) {
-		unsigned long dn_mask = setattr_mask(ia_valid);
-		if (dn_mask)
-			dnotify_parent(dentry, dn_mask);
-	}
+	if (!error)
+		fsnotify_change(dentry, ia_valid);
+
 	return error;
 }
 
fs/compat.c +8 -4
diff -urN linux-2.6.13-rc1/fs/compat.c linux/fs/compat.c
@@ -37,7 +37,7 @@
 #include <linux/ctype.h>
 #include <linux/module.h>
 #include <linux/dirent.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/highuid.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/nfsd/nfsd.h>
@@ -1307,9 +1307,13 @@
 out:
 	if (iov != iovstack)
 		kfree(iov);
-	if ((ret + (type == READ)) > 0)
-		dnotify_parent(file->f_dentry,
-				(type == READ) ? DN_ACCESS : DN_MODIFY);
+	if ((ret + (type == READ)) > 0) {
+		struct dentry *dentry = file->f_dentry;
+		if (type == READ)
+			fsnotify_access(dentry);
+		else
+			fsnotify_modify(dentry);
+	}
 	return ret;
 }
 
fs/file_table.c +3 -0
diff -urN linux-2.6.13-rc1/fs/file_table.c linux/fs/file_table.c
@@ -16,6 +16,7 @@
 #include <linux/eventpoll.h>
 #include <linux/mount.h>
 #include <linux/cdev.h>
+#include <linux/fsnotify.h>
 
 /* sysctl tunables... */
 struct files_stat_struct files_stat = {
@@ -126,6 +127,8 @@
 	struct inode *inode = dentry->d_inode;
 
 	might_sleep();
+
+	fsnotify_close(file);
 	/*
 	 * The function eventpoll_release() should be the first called
 	 * in the file cleanup chain.
fs/inode.c +6 -0
diff -urN linux-2.6.13-rc1/fs/inode.c linux/fs/inode.c
@@ -21,6 +21,7 @@
 #include <linux/pagemap.h>
 #include <linux/cdev.h>
 #include <linux/bootmem.h>
+#include <linux/inotify.h>
 
 /*
  * This is needed for the following functions:
@@ -202,6 +203,10 @@
 	INIT_LIST_HEAD(&inode->i_data.i_mmap_nonlinear);
 	spin_lock_init(&inode->i_lock);
 	i_size_ordered_init(inode);
+#ifdef CONFIG_INOTIFY
+	INIT_LIST_HEAD(&inode->inotify_watches);
+	sema_init(&inode->inotify_sem, 1);
+#endif
 }
 
 EXPORT_SYMBOL(inode_init_once);
@@ -346,6 +351,7 @@
 
 	down(&iprune_sem);
 	spin_lock(&inode_lock);
+	inotify_unmount_inodes(&sb->s_inodes);
 	busy = invalidate_list(&sb->s_inodes, &throw_away);
 	spin_unlock(&inode_lock);
 
fs/inotify.c +984 -0

984 lines not on this page · raw message

fs/Kconfig +13 -0
diff -urN linux-2.6.13-rc1/fs/Kconfig linux/fs/Kconfig
@@ -356,6 +356,19 @@
 	  If you don't know whether you need it, then you don't need it:
 	  answer N.
 
+config INOTIFY
+	bool "Inotify file change notification support"
+	default y
+	---help---
+	  Say Y here to enable inotify support and the /dev/inotify character
+	  device.  Inotify is a file change notification system and a
+	  replacement for dnotify.  Inotify fixes numerous shortcomings in
+	  dnotify and introduces several new features.  It allows monitoring
+	  of both files and directories via a single open fd.  Multiple file
+	  events are supported.
+
+	  If unsure, say Y.
+
 config QUOTA
 	bool "Quota support"
 	help
fs/Makefile +1 -0
diff -urN linux-2.6.13-rc1/fs/Makefile linux/fs/Makefile
@@ -12,6 +12,7 @@
 		seq_file.o xattr.o libfs.o fs-writeback.o mpage.o direct-io.o \
 		ioprio.o
 
+obj-$(CONFIG_INOTIFY)		+= inotify.o
 obj-$(CONFIG_EPOLL)		+= eventpoll.o
 obj-$(CONFIG_COMPAT)		+= compat.o
 
fs/namei.c +16 -14
diff -urN linux-2.6.13-rc1/fs/namei.c linux/fs/namei.c
@@ -21,7 +21,7 @@
 #include <linux/namei.h>
 #include <linux/quotaops.h>
 #include <linux/pagemap.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/smp_lock.h>
 #include <linux/personality.h>
 #include <linux/security.h>
@@ -1312,7 +1312,7 @@
 	DQUOT_INIT(dir);
 	error = dir->i_op->create(dir, dentry, mode, nd);
 	if (!error) {
-		inode_dir_notify(dir, DN_CREATE);
+		fsnotify_create(dir, dentry->d_name.name);
 		security_inode_post_create(dir, dentry, mode);
 	}
 	return error;
@@ -1637,7 +1637,7 @@
 	DQUOT_INIT(dir);
 	error = dir->i_op->mknod(dir, dentry, mode, dev);
 	if (!error) {
-		inode_dir_notify(dir, DN_CREATE);
+		fsnotify_create(dir, dentry->d_name.name);
 		security_inode_post_mknod(dir, dentry, mode, dev);
 	}
 	return error;
@@ -1710,7 +1710,7 @@
 	DQUOT_INIT(dir);
 	error = dir->i_op->mkdir(dir, dentry, mode);
 	if (!error) {
-		inode_dir_notify(dir, DN_CREATE);
+		fsnotify_mkdir(dir, dentry->d_name.name);
 		security_inode_post_mkdir(dir,dentry, mode);
 	}
 	return error;
@@ -1801,7 +1801,7 @@
 	}
 	up(&dentry->d_inode->i_sem);
 	if (!error) {
-		inode_dir_notify(dir, DN_DELETE);
+		fsnotify_rmdir(dentry, dentry->d_inode, dir);
 		d_delete(dentry);
 	}
 	dput(dentry);
@@ -1874,9 +1874,10 @@
 
 	/* We don't d_delete() NFS sillyrenamed files--they still exist. */
 	if (!error && !(dentry->d_flags & DCACHE_NFSFS_RENAMED)) {
+		fsnotify_unlink(dentry, dir);
 		d_delete(dentry);
-		inode_dir_notify(dir, DN_DELETE);
 	}
+
 	return error;
 }
 
@@ -1950,7 +1951,7 @@
 	DQUOT_INIT(dir);
 	error = dir->i_op->symlink(dir, dentry, oldname);
 	if (!error) {
-		inode_dir_notify(dir, DN_CREATE);
+		fsnotify_create(dir, dentry->d_name.name);
 		security_inode_post_symlink(dir, dentry, oldname);
 	}
 	return error;
@@ -2023,7 +2024,7 @@
 	error = dir->i_op->link(old_dentry, dir, new_dentry);
 	up(&old_dentry->d_inode->i_sem);
 	if (!error) {
-		inode_dir_notify(dir, DN_CREATE);
+		fsnotify_create(dir, new_dentry->d_name.name);
 		security_inode_post_link(old_dentry, dir, new_dentry);
 	}
 	return error;
@@ -2187,6 +2188,7 @@
 {
 	int error;
 	int is_dir = S_ISDIR(old_dentry->d_inode->i_mode);
+	const char *old_name;
 
 	if (old_dentry->d_inode == new_dentry->d_inode)
  		return 0;
@@ -2208,18 +2210,18 @@
 	DQUOT_INIT(old_dir);
 	DQUOT_INIT(new_dir);
 
+	old_name = fsnotify_oldname_init(old_dentry->d_name.name);
+
 	if (is_dir)
 		error = vfs_rename_dir(old_dir,old_dentry,new_dir,new_dentry);
 	else
 		error = vfs_rename_other(old_dir,old_dentry,new_dir,new_dentry);
 	if (!error) {
-		if (old_dir == new_dir)
-			inode_dir_notify(old_dir, DN_RENAME);
-		else {
-			inode_dir_notify(old_dir, DN_DELETE);
-			inode_dir_notify(new_dir, DN_CREATE);
-		}
+		const char *new_name = old_dentry->d_name.name;
+		fsnotify_move(old_dir, new_dir, old_name, new_name, is_dir);
 	}
+	fsnotify_oldname_free(old_name);
+
 	return error;
 }
 
fs/nfsd/vfs.c +3 -3
diff -urN linux-2.6.13-rc1/fs/nfsd/vfs.c linux/fs/nfsd/vfs.c
@@ -45,7 +45,7 @@
 #endif /* CONFIG_NFSD_V3 */
 #include <linux/nfsd/nfsfh.h>
 #include <linux/quotaops.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/posix_acl.h>
 #include <linux/posix_acl_xattr.h>
 #ifdef CONFIG_NFSD_V4
@@ -860,7 +860,7 @@
 		nfsdstats.io_read += err;
 		*count = err;
 		err = 0;
-		dnotify_parent(file->f_dentry, DN_ACCESS);
+		fsnotify_access(file->f_dentry);
 	} else 
 		err = nfserrno(err);
 out:
@@ -916,7 +916,7 @@
 	set_fs(oldfs);
 	if (err >= 0) {
 		nfsdstats.io_write += cnt;
-		dnotify_parent(file->f_dentry, DN_MODIFY);
+		fsnotify_modify(file->f_dentry);
 	}
 
 	/* clear setuid/setgid flag after write */
fs/open.c +2 -1
diff -urN linux-2.6.13-rc1/fs/open.c linux/fs/open.c
@@ -10,7 +10,7 @@
 #include <linux/file.h>
 #include <linux/smp_lock.h>
 #include <linux/quotaops.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/tty.h>
@@ -951,6 +951,7 @@
 				put_unused_fd(fd);
 				fd = PTR_ERR(f);
 			} else {
+				fsnotify_open(f->f_dentry);
 				fd_install(fd, f);
 			}
 		}
fs/read_write.c +9 -6
diff -urN linux-2.6.13-rc1/fs/read_write.c linux/fs/read_write.c
@@ -10,7 +10,7 @@
 #include <linux/file.h>
 #include <linux/uio.h>
 #include <linux/smp_lock.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/security.h>
 #include <linux/module.h>
 #include <linux/syscalls.h>
@@ -252,7 +252,7 @@
 			else
 				ret = do_sync_read(file, buf, count, pos);
 			if (ret > 0) {
-				dnotify_parent(file->f_dentry, DN_ACCESS);
+				fsnotify_access(file->f_dentry);
 				current->rchar += ret;
 			}
 			current->syscr++;
@@ -303,7 +303,7 @@
 			else
 				ret = do_sync_write(file, buf, count, pos);
 			if (ret > 0) {
-				dnotify_parent(file->f_dentry, DN_MODIFY);
+				fsnotify_modify(file->f_dentry);
 				current->wchar += ret;
 			}
 			current->syscw++;
@@ -539,9 +539,12 @@
 out:
 	if (iov != iovstack)
 		kfree(iov);
-	if ((ret + (type == READ)) > 0)
-		dnotify_parent(file->f_dentry,
-				(type == READ) ? DN_ACCESS : DN_MODIFY);
+	if ((ret + (type == READ)) > 0) {
+		if (type == READ)
+			fsnotify_access(file->f_dentry);
+		else
+			fsnotify_modify(file->f_dentry);
+	}
 	return ret;
 Efault:
 	ret = -EFAULT;
fs/sysfs/file.c +2 -5
diff -urN linux-2.6.13-rc1/fs/sysfs/file.c linux/fs/sysfs/file.c
@@ -3,7 +3,7 @@
  */
 
 #include <linux/module.h>
-#include <linux/dnotify.h>
+#include <linux/fsnotify.h>
 #include <linux/kobject.h>
 #include <linux/namei.h>
 #include <asm/uaccess.h>
@@ -391,9 +391,6 @@
  * sysfs_update_file - update the modified timestamp on an object attribute.
  * @kobj: object we're acting for.
  * @attr: attribute descriptor.
- *
- * Also call dnotify for the dentry, which lots of userspace programs
- * use.
  */
 int sysfs_update_file(struct kobject * kobj, const struct attribute * attr)
 {
@@ -408,7 +405,7 @@
 		if (victim->d_inode && 
 		    (victim->d_parent->d_inode == dir->d_inode)) {
 			victim->d_inode->i_mtime = CURRENT_TIME;
-			dnotify_parent(victim, DN_MODIFY);
+			fsnotify_modify(victim);
 
 			/**
 			 * Drop reference from initial sysfs_get_dentry().
fs/xattr.c +4 -1
diff -urN linux-2.6.13-rc1/fs/xattr.c linux/fs/xattr.c
@@ -16,6 +16,7 @@
 #include <linux/security.h>
 #include <linux/syscalls.h>
 #include <linux/module.h>
+#include <linux/fsnotify.h>
 #include <asm/uaccess.h>
 
 /*
@@ -57,8 +58,10 @@
 		if (error)
 			goto out;
 		error = d->d_inode->i_op->setxattr(d, kname, kvalue, size, flags);
-		if (!error)
+		if (!error) {
+			fsnotify_xattr(d);
 			security_inode_post_setxattr(d, kname, kvalue, size, flags);
+		}
 out:
 		up(&d->d_inode->i_sem);
 	}
include/linux/fs.h +5 -1
diff -urN linux-2.6.13-rc1/include/linux/fs.h linux/include/linux/fs.h
@@ -474,6 +474,11 @@
 	struct dnotify_struct	*i_dnotify; /* for directory notifications */
 #endif
 
+#ifdef CONFIG_INOTIFY
+	struct list_head	inotify_watches; /* watches on this inode */
+	struct semaphore	inotify_sem;	/* protects the watches list */
+#endif
+
 	unsigned long		i_state;
 	unsigned long		dirtied_when;	/* jiffies of first dirtying */
 
@@ -1393,7 +1398,6 @@
 extern int do_remount_sb(struct super_block *sb, int flags,
 			 void *data, int force);
 extern sector_t bmap(struct inode *, sector_t);
-extern int setattr_mask(unsigned int);
 extern int notify_change(struct dentry *, struct iattr *);
 extern int permission(struct inode *, int, struct nameidata *);
 extern int generic_permission(struct inode *, int,
include/linux/fsnotify.h +257 -0

257 lines not on this page · raw message

include/linux/inotify.h +124 -0

124 lines not on this page · raw message

include/linux/sched.h +4 -0
diff -urN linux-2.6.13-rc1/include/linux/sched.h linux/include/linux/sched.h
@@ -410,6 +410,10 @@
 	atomic_t processes;	/* How many processes does this user have? */
 	atomic_t files;		/* How many open files does this user have? */
 	atomic_t sigpending;	/* How many pending signals does this user have? */
+#ifdef CONFIG_INOTIFY
+	atomic_t inotify_watches; /* How many inotify watches does this user have? */
+	atomic_t inotify_devs;	/* How many inotify devs does this user have opened? */
+#endif
 	/* protected by mq_lock	*/
 	unsigned long mq_bytes;	/* How many bytes can be allocated to mqueue? */
 	unsigned long locked_shm; /* How many pages of mlocked shm ? */
kernel/user.c +4 -0
diff -urN linux-2.6.13-rc1/kernel/user.c linux/kernel/user.c
@@ -120,6 +120,10 @@
 		atomic_set(&new->processes, 0);
 		atomic_set(&new->files, 0);
 		atomic_set(&new->sigpending, 0);
+#ifdef CONFIG_INOTIFY
+		atomic_set(&new->inotify_watches, 0);
+		atomic_set(&new->inotify_devs, 0);
+#endif
 
 		new->mq_bytes = 0;
 		new->locked_shm = 0;
#8 lore raw

Hi Robert,

Quoted 6 lines · Robert Love
Quoted 2 lines

either introduced by inotify or just made more likely (i.e. 100% of time) by inotify but the change is elsewhere.

The mount case is baffling. Inotify has no interaction whatsoever with mounting. Unmounting, definitely--see fs/inotify.c ::
inotify_unmount_inodes().

Good news are that it seems unrelated to inotify. I updated to 2.6.13-rc1, and tested it with and without the inotify patch. Now I get the mount process hanged in inode_wait with the vanilla kernel, and working perfectly with the patched one. Weird...

Quoted 1 line

[ Attached is the latest inotify, against 2.6.13-rc1. ]

I tested it with this patch.

bye

Signature · 1 line
David Gómez                                      Jabber ID: [email protected]
#9 lore raw
Quoted 6 lines · David Gómez

Hi Robert,

Quoted 3 lines · Robert Love
Quoted 2 lines

either introduced by inotify or just made more likely (i.e. 100% of time) by inotify but the change is elsewhere.

Quoted 8 lines
Quoted 3 lines

The mount case is baffling. Inotify has no interaction whatsoever with mounting. Unmounting, definitely--see fs/inotify.c ::
inotify_unmount_inodes().

Good news are that it seems unrelated to inotify. I updated to 2.6.13-rc1, and tested it with and without the inotify patch. Now I get the mount process hanged in inode_wait with the vanilla kernel, and working perfectly with the patched one. Weird...

Sorry to hear that your problem continues, but glad to hear that it is not inotify. I was baffled by a lockup in mount; it made no sense.

Good luck finding a fix!

Robert Love

#10 lore raw
In reply to Anton Altaparmakov

Hi Anton,

I'm trying to work around the NTFS lockup issue. Like others, I can reproduce it just by opening nautilus on an NTFS partition (i.e. creating an inotify watch on it) and then unmount at some point after - instant system lockup.

I have tried applying two patches:
fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation.patch from 2.6.13-rc1-mm1 and the "NTFS: Fix a nasty deadlock that appeared in recent kernels" patch from your git tree.

However, I still get the freezing up on unmount. I am using 2.6.12, plus inotify-0.23-15, and the two patches mentioned above. Anything else I can try?

Thanks,
Daniel

#11 lore raw

Hi,

Quoted 11 lines · Daniel Drake

I'm trying to work around the NTFS lockup issue. Like others, I can reproduce it just by opening nautilus on an NTFS partition (i.e. creating an inotify watch on it) and then unmount at some point after - instant system lockup.

I have tried applying two patches:
fix-soft-lockup-due-to-ntfs-vfs-part-and-explanation.patch from 2.6.13-rc1-mm1 and the "NTFS: Fix a nasty deadlock that appeared in recent kernels" patch from your git tree.

However, I still get the freezing up on unmount. I am using 2.6.12, plus inotify-0.23-15, and the two patches mentioned above. Anything else I can try?

Thinking about it some more made me realize that there may be a problem in inotify after all... Could you try the below patch to fs/inotify.c and tell me if it cures the lockup you are seeing? (Note patch compiles but is otherwise untested. But given it locks up without the patch it can't do much worse with it!)

Thanks a lot in advance!

Best regards,

Anton

Signature · 81 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

inotify_unmount_inodes-list-iteration-fix.diff

Patch description: I believe that the inode reference that is being 
dropped by inotify's remove_watch() can cause inodes other than the 
current @inode to be moved away from the per-sb list.  And if this happens 
to be the next inode in the list, i.e. @next_i, then the iteration will 
proceed on the list that @next_i was moved to rather than the per-sb list.  
Thus, the check in the for loop (list_for_each_entry_safe()) for the @head 
being reached will _never_ be true and hence the for loop will keep going 
for ever...  Even worse the memory backing @next_i could be completely 
freed and then completely random results would be obtained.

Basically, I do not believe that using list_for_each_entry_safe() is safe 
at all as it only guards against removal of the current entry but not 
against removal of the next entry.  This patch tries to work around this 
by getting a reference to @next_i whilst the inode_lock is dropped.

Signed-off-by: Anton Altaparmakov <[email protected]>

--- linux-2.6.13-rc1-mm1-vanilla/fs/inotify.c	2005-07-01 14:51:09.000000000 +0100
+++ linux-2.6.13-rc1-mm1/fs/inotify.c	2005-07-02 22:11:11.000000000 +0100
@@ -560,9 +560,10 @@ EXPORT_SYMBOL_GPL(inotify_get_cookie);
  */
 void inotify_unmount_inodes(struct list_head *list)
 {
-	struct inode *inode, *next_i;
+	struct inode *inode, *next_i, *need_iput = NULL;
 
 	list_for_each_entry_safe(inode, next_i, list, i_sb_list) {
+		struct inode *need_iput_tmp;
 		struct inotify_watch *watch, *next_w;
 		struct list_head *watches;
 
@@ -574,8 +575,20 @@ void inotify_unmount_inodes(struct list_
 		if (inode->i_state & (I_CLEAR | I_FREEING))
 			continue;
 
+		need_iput_tmp = need_iput;
+		need_iput = NULL;
+
 		/* In case the remove_watch() drops a reference */
-		__iget(inode);
+		if (inode != need_iput_tmp)
+			__iget(inode);
+		else
+			need_iput_tmp = NULL;
+
+		/* In case the dropping of a reference would nuke next_i. */
+		if (!next_i->i_state & (I_CLEAR | I_FREEING)) {
+			__iget(next_i);
+			need_iput = next_i;
+		}
 
 		/*
 		 * We can safely drop inode_lock here because the per-sb list
@@ -584,6 +597,9 @@ void inotify_unmount_inodes(struct list_
 		 */
 		spin_unlock(&inode_lock);
 
+		if (need_iput_tmp)
+			iput(need_iput_tmp);
+
 		/* for each watch, send IN_UNMOUNT and then remove it */
 		down(&inode->inotify_sem);
 		watches = &inode->inotify_watches;
@@ -599,6 +615,11 @@ void inotify_unmount_inodes(struct list_
 
 		spin_lock(&inode_lock);
 	}
+	if (need_iput) {
+		spin_unlock(&inode_lock);
+		iput(need_iput);
+		spin_lock(&inode_lock);
+	}
 }
 EXPORT_SYMBOL_GPL(inotify_unmount_inodes);
#12 lore raw

Hi Anton,

Quoted 5 lines · Anton Altaparmakov

Thinking about it some more made me realize that there may be a problem in inotify after all... Could you try the below patch to fs/inotify.c and tell me if it cures the lockup you are seeing? (Note patch compiles but is otherwise untested. But given it locks up without the patch it can't do much worse with it!)

Thanks for writing that patch, the effort is much appreciated. Unfortunately it does not help :(

I've done a bit more investigating for you though. I did some tests purely on the console, using inotify-test (from inotify-utils) which is the most simplistic way you can use inotify: it just prints out the recieved events to the screen.

I found out that unmount works perfectly well as long as there are no active inotify watches (this might be quite obvious though!) - i.e. closing inotify-test before unmounting results in a clean unmount. Unmounting while inotify-test is watching the NTFS partition causes the freeze.

When the machine freezes, it still responds to ping, but not to ssh. Sysrq works, so I got a sysrq-p trace:

Pid 8997 comm umount
EIP is at inotify_unmount_inodes+0x38/0x140

stack trace:

invalidate_inodes+0x40/0x90
generic_shutdown_super+0x59/0x140
kill_block_super+0x2d/0x50
deactivate_super+0x5a/0x90
sys_umount+0x3f/0x90
filp_close+0x52/0xa0
sys_oldumount+0x17/0x20
sysenter_past_esp+0x54/0x75

Investigating that function:

(gdb) list *inotify_unmount_inodes+0x38 0x9c8 is in inotify_unmount_inodes (inotify.c:565).
560 */
561 void inotify_unmount_inodes(struct list_head *list)
562 {
563 struct inode *inode, *next_i, *need_iput = NULL;
564
565 list_for_each_entry_safe(inode, next_i, list, i_sb_list) {

566                     struct inode *need_iput_tmp;
567                     struct inotify_watch *watch, *next_w;
568                     struct list_head *watches;

569

I then added a loop counter printk in at line 569 above. It shows that the loop iterates 8 times on a clean unmount, and goes into a seemingly infinite loop (i.e. freeze) when unmounting with inotify watches active.

I don't know much about filesystem internals so I'm pretty stuck here, haven't reached that chapter of Robert's book yet ;)

Please let me know if theres any other info I can provide.

Thanks,
Daniel

#13 lore raw

Hi Daniel,

Quoted 9 lines · Daniel Drake
Quoted 5 lines · Anton Altaparmakov

Thinking about it some more made me realize that there may be a problem in inotify after all... Could you try the below patch to fs/inotify.c and tell me if it cures the lockup you are seeing? (Note patch compiles but is otherwise untested. But given it locks up without the patch it can't do much worse with it!)

Thanks for writing that patch, the effort is much appreciated. Unfortunately it does not help :(

)-:

Quoted 9 lines

I've done a bit more investigating for you though. I did some tests purely on the console, using inotify-test (from inotify-utils) which is the most simplistic way you can use inotify: it just prints out the recieved events to the screen.

I found out that unmount works perfectly well as long as there are no active inotify watches (this might be quite obvious though!) - i.e. closing inotify-test before unmounting results in a clean unmount. Unmounting while inotify-test is watching the NTFS partition causes the freeze.

Great stuff. Thanks! At least my patch appears to have been on the right track! My analysis was that there is an infinite loop and this is what you are observing. (-: It is very interesting that the infinite loop only happens when actual watches are present. It is 1am here so I am going to bed but I will think about this tomorrow/Monday and hopefully cook up a new patch which if you could test it would be great.

Quoted 26 lines

When the machine freezes, it still responds to ping, but not to ssh. Sysrq works, so I got a sysrq-p trace:

Pid 8997 comm umount
EIP is at inotify_unmount_inodes+0x38/0x140

stack trace:

invalidate_inodes+0x40/0x90
generic_shutdown_super+0x59/0x140
kill_block_super+0x2d/0x50
deactivate_super+0x5a/0x90
sys_umount+0x3f/0x90
filp_close+0x52/0xa0
sys_oldumount+0x17/0x20
sysenter_past_esp+0x54/0x75

Investigating that function:

(gdb) list *inotify_unmount_inodes+0x38 0x9c8 is in inotify_unmount_inodes (inotify.c:565).
560 */
561 void inotify_unmount_inodes(struct list_head *list)
562 {
563 struct inode *inode, *next_i, *need_iput = NULL;
564
565 list_for_each_entry_safe(inode, next_i, list, i_sb_list) {

Quoted 8 lines
566                     struct inode *need_iput_tmp;
567                     struct inotify_watch *watch, *next_w;
568                     struct list_head *watches;

569

I then added a loop counter printk in at line 569 above. It shows that the loop iterates 8 times on a clean unmount, and goes into a seemingly infinite loop (i.e. freeze) when unmounting with inotify watches active.

Cool. Thanks for that. Could you modify your printk to output:

printk(KERN_ERR "doing sb 0x%lx, inode 0x%lx, i_state 0x%x\n", (unsigned long)inode->i_sb, inode->i_ino, inode->i_state);

Eeek! I just reread my patch. I am a muppet! Instead of the printk (or in addition if you like), please try the corrected version below. The original version completely fails to do anything at all. It's amazing what the omission of a single parenthesis pair can do to your code logic. )-:

Thanks a lot in advance and many apologies for wasting your time with a bogus patch!

Best regards,

Anton

Signature · 81 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

inotify_unmount_inodes-list-iteration-fix2.diff

Patch description: I believe that the inode reference that is being 
dropped by inotify's remove_watch() can cause inodes other than the 
current @inode to be moved away from the per-sb list.  And if this happens 
to be the next inode in the list, i.e. @next_i, then the iteration will 
proceed on the list that @next_i was moved to rather than the per-sb list.  
Thus, the check in the for loop (list_for_each_entry_safe()) for the @head 
being reached will _never_ be true and hence the for loop will keep going 
for ever...  Even worse the memory backing @next_i could be completely 
freed and then completely random results would be obtained.

Basically, I do not believe that using list_for_each_entry_safe() is safe 
at all as it only guards against removal of the current entry but not 
against removal of the next entry.  This patch tries to work around this 
by getting a reference to @next_i whilst the inode_lock is dropped.

Signed-off-by: Anton Altaparmakov <[email protected]>

--- linux-2.6.13-rc1-mm1-vanilla/fs/inotify.c	2005-07-01 14:51:09.000000000 +0100
+++ linux-2.6.13-rc1-mm1/fs/inotify.c	2005-07-02 22:11:11.000000000 +0100
@@ -560,9 +560,10 @@ EXPORT_SYMBOL_GPL(inotify_get_cookie);
  */
 void inotify_unmount_inodes(struct list_head *list)
 {
-	struct inode *inode, *next_i;
+	struct inode *inode, *next_i, *need_iput = NULL;
 
 	list_for_each_entry_safe(inode, next_i, list, i_sb_list) {
+		struct inode *need_iput_tmp;
 		struct inotify_watch *watch, *next_w;
 		struct list_head *watches;
 
@@ -574,8 +575,20 @@ void inotify_unmount_inodes(struct list_
 		if (inode->i_state & (I_CLEAR | I_FREEING))
 			continue;
 
+		need_iput_tmp = need_iput;
+		need_iput = NULL;
+
 		/* In case the remove_watch() drops a reference */
-		__iget(inode);
+		if (inode != need_iput_tmp)
+			__iget(inode);
+		else
+			need_iput_tmp = NULL;
+
+		/* In case the dropping of a reference would nuke next_i. */
+		if (!(next_i->i_state & (I_CLEAR | I_FREEING))) {
+			__iget(next_i);
+			need_iput = next_i;
+		}
 
 		/*
 		 * We can safely drop inode_lock here because the per-sb list
@@ -584,6 +597,9 @@ void inotify_unmount_inodes(struct list_
 		 */
 		spin_unlock(&inode_lock);
 
+		if (need_iput_tmp)
+			iput(need_iput_tmp);
+
 		/* for each watch, send IN_UNMOUNT and then remove it */
 		down(&inode->inotify_sem);
 		watches = &inode->inotify_watches;
@@ -599,6 +615,11 @@ void inotify_unmount_inodes(struct list_
 
 		spin_lock(&inode_lock);
 	}
+	if (need_iput) {
+		spin_unlock(&inode_lock);
+		iput(need_iput);
+		spin_lock(&inode_lock);
+	}
 }
 EXPORT_SYMBOL_GPL(inotify_unmount_inodes);
#14 lore raw
Quoted 5 lines · Anton Altaparmakov

Eeek! I just reread my patch. I am a muppet! Instead of the printk (or in addition if you like), please try the corrected version below. The original version completely fails to do anything at all. It's amazing what the omission of a single parenthesis pair can do to your code logic. )-:

I reverted the patch you sent earlier (inotify_unmount_inodes-list-iteration-fix.diff) and applied the one you attached here (inotify_unmount_inodes-list-iteration-fix2.diff).

The good news is that the hang is gone. The bad news is that you cured the hang by introducing an oops :(

Unable to handle kernel NULL pointer dereference at virtual address 00000024
 printing eip:
c01751d9
*pde = 00000000
Oops: 0000 [#1]
PREEMPT
Modules linked in: ntfs snd_pcm_oss snd_mixer_oss snd_seq_oss
snd_seq_midi_event snd_seq snd_seq_device snd_intel8x0 snd_ac97_codec snd_pcm
snd_timer snd snd_page_alloc
CPU:    0
EIP:    0060:[<c01751d9>]    Tainted: P      VLI
EFLAGS: 00010283   (2.6.12-gentoo-r3)

EIP is at iput+0x19/0x90
eax: 00000000 ebx: deb82e64 ecx: deb82e64 edx: deb82e64
esi: dd01ff00 edi: deb82e64 ebp: dd01ff00 esp: df7b9eb8
ds: 007b es: 007b ss: 0068
Process umount (pid: 9201, threadinfo=df7b8000 task=dd656a60)

Stack: dd01fe20 c03feb20 df7b8000 c0182337 deb82e64 00000000 00000000 00000000
       00000000 df7b8000 dd01ff08 deb82e64 dd01f0dc df7b8000 deb82e74 df7b9f08
       df7b8000 c0174050 deb82e74 dffe0040 df7b9f08 df7b9f08 dda12274 deb82e00
Call Trace:
 [<c0182337>] inotify_unmount_inodes+0x187/0x1e0
 [<c0174050>] invalidate_inodes+0x40/0x90
 [<c015f579>] generic_shutdown_super+0x59/0x140
 [<c01601dd>] kill_block_super+0x2d/0x50
 [<c015f40a>] deactivate_super+0x5a/0x90
 [<c01773cf>] sys_umount+0x3f/0x90
 [<c0157e42>] filp_close+0x52/0xa0
 [<c0177437>] sys_oldumount+0x17/0x20
 [<c010326b>] sysenter_past_esp+0x54/0x75
Code: ff ff 89 44 24 04 e9 47 fe ff ff 8d b4 26 00 00 00 00 53 83 ec 08 8b 5c

24 10 85 db 74 58 83 bb 24 01 00 00 20 8b 83 94 00 00 00 <8b> 40 24 74 5a 85 c0 74 07 8b 50 14 85 d2 75 47 8d 43 24 c7 44

(gdb) list *iput+0x19
0x1789 is in iput (inode.c:1131).
1126 * Consequently, iput() can sleep.
1127 */
1128 void iput(struct inode *inode)
1129 {
1130 if (inode) {
1131 struct super_operations *op = inode->i_sb->s_op;
1132
1133 BUG_ON(inode->i_state == I_CLEAR);
1134
1135 if (op && op->put_inode)

Note that this exact oops happens even when inotify has not been used, i.e. to reproduce this oops, I can just do:

mount /mnt/ntfs ; umount /mnt/ntfs

What next? :)

Thanks,
Daniel

#15 lore raw
Quoted 6 lines · Daniel Drake

I reverted the patch you sent earlier (inotify_unmount_inodes-list-iteration-fix.diff) and applied the one you attached here (inotify_unmount_inodes-list-iteration-fix2.diff).

The good news is that the hang is gone. The bad news is that you cured the hang by introducing an oops :(

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Thanks a lot!

Anton

Signature · 66 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

--- linux-2.6.13-rc1-mm1-vanilla/fs/inotify.c	2005-07-01 14:51:09.000000000 +0100
+++ linux-2.6.13-rc1-mm1/fs/inotify.c	2005-07-04 15:18:14.000000000 +0100
@@ -560,22 +560,45 @@ EXPORT_SYMBOL_GPL(inotify_get_cookie);
  */
 void inotify_unmount_inodes(struct list_head *list)
 {
-	struct inode *inode, *next_i;
+	struct inode *inode, *next_i, *need_iput = NULL;
 
 	list_for_each_entry_safe(inode, next_i, list, i_sb_list) {
+		struct inode *need_iput_tmp;
 		struct inotify_watch *watch, *next_w;
 		struct list_head *watches;
 
 		/*
+		 * If i_count is zero, the inode cannot have any watches and
+		 * doing an __iget/iput with MS_ACTIVE clear would actually
+		 * evict all inodes from icache which is unnecessarily violent.
+		 */
+		if (!atomic_read(&inode->i_count))
+			continue;
+		/*
 		 * We cannot __iget() an inode in state I_CLEAR or I_FREEING,
 		 * which is fine becayse by that point the inode cannot have
 		 * any associated watches.
 		 */
-		if (inode->i_state & (I_CLEAR | I_FREEING))
+		if (inode->i_state & (I_CLEAR | I_FREEING | I_WILL_FREE))
 			continue;
 
+		need_iput_tmp = need_iput;
+		need_iput = NULL;
+
 		/* In case the remove_watch() drops a reference */
-		__iget(inode);
+		if (inode != need_iput_tmp)
+			__iget(inode);
+		else
+			need_iput_tmp = NULL;
+
+		/* In case the dropping of a reference would nuke next_i. */
+		if ((&next_i->i_sb_list != list) &&
+				atomic_read(&next_i->i_count) &&
+				!(next_i->i_state & (I_CLEAR | I_FREEING |
+				I_WILL_FREE))) {
+			__iget(next_i);
+			need_iput = next_i;
+		}
 
 		/*
 		 * We can safely drop inode_lock here because the per-sb list
@@ -584,6 +607,9 @@ void inotify_unmount_inodes(struct list_
 		 */
 		spin_unlock(&inode_lock);
 
+		if (need_iput_tmp)
+			iput(need_iput_tmp);
+
 		/* for each watch, send IN_UNMOUNT and then remove it */
 		down(&inode->inotify_sem);
 		watches = &inode->inotify_watches;
#16 lore raw
Quoted 10 lines · Anton Altaparmakov
Quoted 6 lines · Daniel Drake

I reverted the patch you sent earlier (inotify_unmount_inodes-list-iteration-fix.diff) and applied the one you attached here (inotify_unmount_inodes-list-iteration-fix2.diff).

The good news is that the hang is gone. The bad news is that you cured the hang by introducing an oops :(

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

I forgot to say that this patch is a replacement for the previous (inotify_unmount_inodes-list-iteration-fix{,2}.diff}.

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#17 lore raw
Quoted 6 lines · Anton Altaparmakov
Quoted 5 lines · Anton Altaparmakov
Quoted 4 lines · Daniel Drake

I reverted the patch you sent earlier (inotify_unmount_inodes-list-iteration-fix.diff) and applied the one you attached here (inotify_unmount_inodes-list-iteration-fix2.diff).

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

The good news is that the hang is gone. The bad news is that you cured the hang by introducing an oops :(

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

I forgot to say that this patch is a replacement for the previous (inotify_unmount_inodes-list-iteration-fix{,2}.diff}.

Note I seem to remember you are using 2.6.12? If so I am not sure when the I_WILL_FREE stuff came into existence, so should inotify.c not compile after applying this patch just delete the " | I_WILL_FREE " in the two places it occurs introduced by the patch and all should be fine.

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#18 lore raw

Anton,

I had used the 2.6.12 kernel with the latest Inotify. There was no "I_WILL_FREE" in the any place. And, there was no problem in compilation.

I believe Inotify is very useful and should be included in the next versions of the kernel. Are there any ongoing plans for this?

Thanks,
Gautam Singaraju
SIS Dept., UNC Charlotte

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Anton Altaparmakov
Sent: Monday, July 04, 2005 11:12 AM
To: Daniel Drake
Cc: David Gómez; Robert Love; John McCutchan; Linux-kernel
Subject: Re: Problem with inotify

Quoted 4 lines · Anton Altaparmakov
Quoted 3 lines · Anton Altaparmakov
Quoted 2 lines · Daniel Drake

I reverted the patch you sent earlier (inotify_unmount_inodes-list-iteration-fix.diff) and applied the one

you

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

attached here (inotify_unmount_inodes-list-iteration-fix2.diff).

The good news is that the hang is gone. The bad news is that you cured

the

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

hang by introducing an oops :(

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

I forgot to say that this patch is a replacement for the previous (inotify_unmount_inodes-list-iteration-fix{,2}.diff}.

Note I seem to remember you are using 2.6.12? If so I am not sure when the I_WILL_FREE stuff came into existence, so should inotify.c not compile after applying this patch just delete the " | I_WILL_FREE " in the two places it occurs introduced by the patch and all should be fine.

Best regards,

Anton

Signature · 10 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#19 lore raw

Gautam,

Quoted 2 lines · Gautam Singaraju

I had used the 2.6.12 kernel with the latest Inotify. There was no "I_WILL_FREE" in the any place. And, there was no problem in compilation.

Er, yes, obviously. You are not using my patch on top of inotify and original inotify does not use I_WILL_FREE. You are answering somewhat out of context...

Quoted 2 lines

I believe Inotify is very useful and should be included in the next versions of the kernel. Are there any ongoing plans for this?

I cannot answer this. Robert/John? However there appear to be issues with it at the moment (the umount problems with ntfs volumes) so probably not quite yet unless we figure out the problems...

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#20 lore raw
In reply to Anton Altaparmakov
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Thanks a million!

Daniel

#21 lore raw
Quoted 6 lines · Daniel Drake
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Great! Thanks a lot for testing! I will send a patch to Robert and Andrew in a minute with more comments added.

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#22 lore raw
Quoted 10 lines · Anton Altaparmakov
Quoted 6 lines · Daniel Drake
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Great! Thanks a lot for testing! I will send a patch to Robert and Andrew in a minute with more comments added.

Nice work, I am going to have a closer look at the patch soon. Could you post the final patch at http://bugzilla.kernel.org/show_bug.cgi?id=4796

Thanks,

Signature · 1 line
John McCutchan <[email protected]>
#23 lore raw
Quoted 6 lines · John McCutchan
Quoted 5 lines · Anton Altaparmakov
Quoted 4 lines · Daniel Drake
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Great! Thanks a lot for testing! I will send a patch to Robert and Andrew in a minute with more comments added.

Nice work, I am going to have a closer look at the patch soon. Could you post the final patch at http://bugzilla.kernel.org/show_bug.cgi?id=4796

Thanks. Now done. But I am not sure about the white space. I can't get anything sensible out of IE on Mac OS 9 which I am on at the moment.

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#24 lore raw
Quoted 10 lines · Anton Altaparmakov
Quoted 9 lines · John McCutchan
Quoted 8 lines · Anton Altaparmakov
Quoted 6 lines · Daniel Drake
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

Great! Thanks a lot for testing! I will send a patch to Robert and Andrew in a minute with more comments added.

Nice work, I am going to have a closer look at the patch soon. Could you post the final patch at http://bugzilla.kernel.org/show_bug.cgi?id=4796

Thanks. Now done. But I am not sure about the white space. I can't get anything sensible out of IE on Mac OS 9 which I am on at the moment.

Originally inotify had 2 functions that handled this. One that would build up a list of inodes to call remove_watch on, the other function would do the actual calling of remove_watch. This mirrored the other unmount paths. I'm wondering if it wouldn't be cleaner to revert back to that way?

John

#25 lore raw
Quoted 17 lines · John McCutchan
Quoted 16 lines · Anton Altaparmakov
Quoted 14 lines · John McCutchan
Quoted 10 lines · Anton Altaparmakov
Quoted 6 lines · Daniel Drake
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Great! Thanks a lot for testing! I will send a patch to Robert and Andrew in a minute with more comments added.

Nice work, I am going to have a closer look at the patch soon. Could you post the final patch at http://bugzilla.kernel.org/show_bug.cgi?id=4796

Quoted 8 lines
Quoted 2 lines

Thanks. Now done. But I am not sure about the white space. I can't get anything sensible out of IE on Mac OS 9 which I am on at the moment.

Originally inotify had 2 functions that handled this. One that would build up a list of inodes to call remove_watch on, the other function would do the actual calling of remove_watch. This mirrored the other unmount paths. I'm wondering if it wouldn't be cleaner to revert back to that way?

That is certainly possible. Out of curiosity, how did you anchor the inodes to your private list? Or did you just have a dynamically allocated array of pointers to inodes?

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#26 lore raw
Quoted 27 lines · Anton Altaparmakov
Quoted 17 lines · John McCutchan
Quoted 16 lines · Anton Altaparmakov
Quoted 14 lines · John McCutchan
Quoted 10 lines · Anton Altaparmakov
Quoted 6 lines · Daniel Drake
Quoted 2 lines · Anton Altaparmakov

)-: I have addressed the only things I can think off that could cause the oops and below is the resulting patch. Could you please test it?

Yeah!! After removing I_WILL_FREE stuff, that fixed both the oops *and* the hang. Everything works nicely now.

Great! Thanks a lot for testing! I will send a patch to Robert and Andrew in a minute with more comments added.

Nice work, I am going to have a closer look at the patch soon. Could you post the final patch at http://bugzilla.kernel.org/show_bug.cgi?id=4796

Quoted 8 lines
Quoted 2 lines

Thanks. Now done. But I am not sure about the white space. I can't get anything sensible out of IE on Mac OS 9 which I am on at the moment.

Originally inotify had 2 functions that handled this. One that would build up a list of inodes to call remove_watch on, the other function would do the actual calling of remove_watch. This mirrored the other unmount paths. I'm wondering if it wouldn't be cleaner to revert back to that way?

Quoted 4 lines

That is certainly possible. Out of curiosity, how did you anchor the inodes to your private list? Or did you just have a dynamically allocated array of pointers to inodes?

The only reason I suggested it, is I'm afraid that maybe we are still missing a corner case. Even with your fix.

We just added a new list_head in the inode struct. If you look at older versions of inotify, maybe around 0.15 you can see for yourself.

John

#27 lore raw
Quoted 12 lines · John McCutchan
Quoted 11 lines · Anton Altaparmakov
Quoted 9 lines · John McCutchan
Quoted 2 lines · John McCutchan

Nice work, I am going to have a closer look at the patch soon. Could you post the final patch at http://bugzilla.kernel.org/show_bug.cgi?id=4796

Originally inotify had 2 functions that handled this. One that would build up a list of inodes to call remove_watch on, the other function would do the actual calling of remove_watch. This mirrored the other unmount paths. I'm wondering if it wouldn't be cleaner to revert back to that way?

Quoted 6 lines
Quoted 3 lines

That is certainly possible. Out of curiosity, how did you anchor the inodes to your private list? Or did you just have a dynamically allocated array of pointers to inodes?

The only reason I suggested it, is I'm afraid that maybe we are still missing a corner case. Even with your fix.

Oh, are you thinking of anything in particular? I don't think you are missing anything now...

You cannot get any new watches which makes things a lot easier (see analysis I just posted in the other thread on LKML: Re: [-mm patch] Fix inotify umount hangs). Thus you can ignore anything that cannot possibly have a watch and that is what my patch does. Both i_count of zero and the three i_states I_CLEAR, I_FREEING, and I_WILL_FREE, which imply an i_count of zero mean that no watches can be present since each watch holds a reference via i_count.

Quoted 2 lines

We just added a new list_head in the inode struct. If you look at older versions of inotify, maybe around 0.15 you can see for yourself.

That is far too space wasting IMHO given you only use it at umount time. That means you are wasting 8 or 16 bytes for each and every inode on the system. If you are going down that route, IMHO it would be better if you just do an array of inodes and then walk that. For scalability you could use a two dimensional array of pages each holding inodes for example. It doesn't really matter what you do given umount is not exactly a hot path so the overhead of doing the necessary page allocations and then freeing them is not important.

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/