Skip to contentSkip to conversation

ieee1394: raw1394: fix possible deadlock in multithreaded clients

PATCH In review Stefan Richter 2 participants 4 messages Oct 26, 2008 – Oct 27, 2008 2 messages not in the archive

lore
Share & tools
lore
#2 lore raw

PATCH 1/1 ieee1394: raw1394: fix possible deadlock in multithreaded clients

In reply to a message not in the archive

Regression in 2.6.28-rc1: When I added the new state_mutex which prevents corruption of raw1394's internal state when accessed by multithreaded client applications, the following possible though highly unlikely deadlock slipped in:

Thread A: Thread B:

 - acquire mmap_sem         - raw1394_write() or raw1394_ioctl()
 - raw1394_mmap()           - acquire state_mutex
 - acquire state_mutex      - copy_to/from_user(), possible page fault:

acquire mmap_sem

The simplest fix is to use mutex_trylock() instead of mutex_lock() in raw1394_mmap(). This changes the behavior under contention in a way which is visible to userspace clients. However, since multithreaded access was entirely buggy before state_mutex was added and libraw1394's documentation advised application programmers to use a handle only in a single thread, this change in behaviour should not be an issue in practice at all.

Since we have to use mutex_trylock() in raw1394_mmap() regardless whether /dev/raw1394 was opened with O_NONBLOCK or not, we now use mutex_trylock() unconditionally everywhere for state_mutex, just to have consistent behavior.

---

Background: That new state_mutex went only in because raw1394_ioctl() already head some weak protection by the Big Kernel Lock, which I removed for the general reasons pro BKL removal (get better performance with local locks; make the locking clearer, easier to debug, more reliable).

 drivers/ieee1394/raw1394.c |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/ieee1394/raw1394.c +6 -3
Index: linux/drivers/ieee1394/raw1394.c
===================================================================
@@ -2268,7 +2268,8 @@ static ssize_t raw1394_write(struct file
 		return -EFAULT;
 	}
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	switch (fi->state) {
 	case opened:
@@ -2548,7 +2549,8 @@ static int raw1394_mmap(struct file *fil
 	struct file_info *fi = file->private_data;
 	int ret;
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	if (fi->iso_state == RAW1394_ISO_INACTIVE)
 		ret = -EINVAL;
@@ -2669,7 +2671,8 @@ static long raw1394_ioctl(struct file *f
 		break;
 	}
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	switch (fi->iso_state) {
 	case RAW1394_ISO_INACTIVE:
Signature · 3 lines
Stefan Richter
-=====-==--- =-=- ==-=-
http://arcgraph.de/sr/
#3 lore raw
Quoted 62 lines · Stefan Richter

Regression in 2.6.28-rc1: When I added the new state_mutex which prevents corruption of raw1394's internal state when accessed by multithreaded client applications, the following possible though highly unlikely deadlock slipped in:

Thread A: Thread B:

 - acquire mmap_sem         - raw1394_write() or raw1394_ioctl()
 - raw1394_mmap()           - acquire state_mutex
 - acquire state_mutex      - copy_to/from_user(), possible page fault:

acquire mmap_sem

The simplest fix is to use mutex_trylock() instead of mutex_lock() in raw1394_mmap(). This changes the behavior under contention in a way which is visible to userspace clients. However, since multithreaded access was entirely buggy before state_mutex was added and libraw1394's documentation advised application programmers to use a handle only in a single thread, this change in behaviour should not be an issue in practice at all.

Since we have to use mutex_trylock() in raw1394_mmap() regardless whether /dev/raw1394 was opened with O_NONBLOCK or not, we now use mutex_trylock() unconditionally everywhere for state_mutex, just to have consistent behavior.

---

Background: That new state_mutex went only in because raw1394_ioctl() already head some weak protection by the Big Kernel Lock, which I removed for the general reasons pro BKL removal (get better performance with local locks; make the locking clearer, easier to debug, more reliable).

 drivers/ieee1394/raw1394.c |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

Index: linux/drivers/ieee1394/raw1394.c
===================================================================

drivers/ieee1394/raw1394.c +4 -2
@@ -2268,7 +2268,8 @@ static ssize_t raw1394_write(struct file
 		return -EFAULT;
 	}
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	switch (fi->state) {
 	case opened:
@@ -2548,7 +2549,8 @@ static int raw1394_mmap(struct file *fil
 	struct file_info *fi = file->private_data;
 	int ret;
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	if (fi->iso_state == RAW1394_ISO_INACTIVE)
 		ret = -EINVAL;

drivers/ieee1394/raw1394.c static long raw1394_ioctl(struct file *f · line 2675

Quoted 6 lines
drivers/ieee1394/raw1394.c +2 -1
@@ -2669,7 +2671,8 @@ static long raw1394_ioctl(struct file *f
 		break;
 	}
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;

So we can return a spurious -EAGAIN to user-space, if the state_mutex is taken briefly by some other context?

Ingo

#4 lore raw
Quoted 15 lines · Ingo Molnar
Quoted 13 lines · Stefan Richter

Regression in 2.6.28-rc1: When I added the new state_mutex which prevents corruption of raw1394's internal state when accessed by multithreaded client applications, the following possible though highly unlikely deadlock slipped in:

Thread A: Thread B:

 - acquire mmap_sem         - raw1394_write() or raw1394_ioctl()
 - raw1394_mmap()           - acquire state_mutex
 - acquire state_mutex      - copy_to/from_user(), possible page fault:

acquire mmap_sem

The simplest fix is to use mutex_trylock() instead of mutex_lock() in raw1394_mmap().

[...]

Quoted 4 lines
Quoted 4 lines

Since we have to use mutex_trylock() in raw1394_mmap() regardless whether /dev/raw1394 was opened with O_NONBLOCK or not, we now use mutex_trylock() unconditionally everywhere for state_mutex, just to have consistent behavior.

[...]

Quoted 37 lines
Quoted 37 lines

Background: That new state_mutex went only in because raw1394_ioctl() already head some weak protection by the Big Kernel Lock, which I removed for the general reasons pro BKL removal (get better performance with local locks; make the locking clearer, easier to debug, more reliable).

 drivers/ieee1394/raw1394.c |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

Index: linux/drivers/ieee1394/raw1394.c
===================================================================

drivers/ieee1394/raw1394.c +4 -2
@@ -2268,7 +2268,8 @@ static ssize_t raw1394_write(struct file
 		return -EFAULT;
 	}
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	switch (fi->state) {
 	case opened:
@@ -2548,7 +2549,8 @@ static int raw1394_mmap(struct file *fil
 	struct file_info *fi = file->private_data;
 	int ret;
 
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;
 
 	if (fi->iso_state == RAW1394_ISO_INACTIVE)
 		ret = -EINVAL;
@@ -2669,7 +2671,8 @@ static long raw1394_ioctl(struct file *f
 		break;
 	}
 
Quoted 6 lines

drivers/ieee1394/raw1394.c static ssize_t raw1394_write(struct file · line 2272

Quoted 3 lines
drivers/ieee1394/raw1394.c +2 -1
-	mutex_lock(&fi->state_mutex);
+	if (!mutex_trylock(&fi->state_mutex))
+		return -EAGAIN;

So we can return a spurious -EAGAIN to user-space, if the state_mutex is taken briefly by some other context?

.write() and .mmap() were not serialized against each other and against .ioctl() at all in raw1394 before 2.6.28-rc1. Only .ioctl() was (somewhat) serialized against itself due to traditionally being implemented as .ioctl() instead of .unlocked_ioctl(). If there were ever concurrent userspace contexts accessing the same file handle, they would have corrupted raw1394's internal state eventually.

Application developers have been advised all the time to access the file handle only from a single thread, ever.

So, before: Concurrent access resulted in hard to debug driver malfunction, without clear error indication.

After: Concurrent access results in -EAGAIN.

I.e. since concurrent access was so broken before that hopefully no raw1394/libraw1394 client exists which did use a single handle in more than one thread, we are quite free how we fix that.

Or at least that's how I understand the situation. Would be good to hear opinions from those who are more involved with libraw1394.

Signature · 3 lines
Stefan Richter
-=====-==--- =-=- ==-=-
http://arcgraph.de/sr/
#5 lore raw
Quoted 2 lines · I wrote:

.write() and .mmap() were not serialized against each other and against .ioctl() at all in raw1394 before 2.6.28-rc1.

PS: There is a need for serialization to some degree because the client registers itself with a controller via .write() (among many other things that are implemented through .write()), manages isochronous I/O contexts on this controller via .ioctl() and maps DMA buffers for isochronous I/O via .mmap().

The raw1394 driver tracks respective state by means of two state variables and some other variables, and accesses of the state variables is not reentrant within one opener of /dev/raw1394. AFAICS the issue exists between .write() and .write(), and independently of that between .ioctl() and .ioctl() and between .ioctl() and .mmap().

Local mutex protection is the simplest way to fix that --- except that there is this obscure issue of locking order between the driver's mutex and the mmap semaphore outside the driver.

Signature · 3 lines
Stefan Richter
-=====-==--- =-=- ==-=-
http://arcgraph.de/sr/