Skip to contentSkip to conversation

counter: drop chrdev_lock

PATCH In review David Lechner 3 participants 18 messages Oct 17, 2021 – Oct 20, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 counter: drop chrdev_lock

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

---
 drivers/counter/counter-chrdev.c |  6 ------
 drivers/counter/counter-sysfs.c  | 13 +++----------
 include/linux/counter.h          |  7 -------
 3 files changed, 3 insertions(+), 23 deletions(-)
drivers/counter/counter-chrdev.c +0 -6
index 967c94ae95bb..b747dc81cfc6 100644
@@ -384,10 +384,6 @@ static int counter_chrdev_open(struct inode *inode, struct file *filp)
 							    typeof(*counter),
 							    chrdev);
 
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
-
 	get_device(&counter->dev);
 	filp->private_data = counter;
 
@@ -419,7 +415,6 @@ static int counter_chrdev_release(struct inode *inode, struct file *filp)
 	mutex_unlock(&counter->ops_exist_lock);
 
 	put_device(&counter->dev);
-	atomic_dec(&counter->chrdev_lock);
 
 	return ret;
 }
@@ -445,7 +440,6 @@ int counter_chrdev_add(struct counter_device *const counter)
 	mutex_init(&counter->events_lock);
 
 	/* Initialize character device */
-	atomic_set(&counter->chrdev_lock, 0);
 	cdev_init(&counter->chrdev, &counter_fops);
 
 	/* Allocate Counter events queue */
drivers/counter/counter-sysfs.c +3 -10
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
 					   u64 val)
 {
 	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
 	/* Allocate new events queue */
 	err = kfifo_alloc(&events, val, GFP_KERNEL);
 	if (err)
-		goto exit_early;
+		return err;
 
 	/* Swap in new events queue */
 	kfifo_free(&counter->events);
 	counter->events.kfifo = events.kfifo;
 
-exit_early:
-	atomic_dec(&counter->chrdev_lock);
-
-	return err;
+	return 0;
 }
 
 static struct counter_comp counter_num_signals_comp =
include/linux/counter.h +0 -7
index 22b14a552b1d..0fd99e255a50 100644
@@ -297,7 +297,6 @@ struct counter_ops {
  * @events:		queue of detected Counter events
  * @events_wait:	wait queue to allow blocking reads of Counter events
  * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
  * @ops_exist_lock:	lock to prevent use during removal
  */
 struct counter_device {
@@ -325,12 +324,6 @@ struct counter_device {
 	DECLARE_KFIFO_PTR(events, struct counter_event);
 	wait_queue_head_t events_wait;
 	struct mutex events_lock;
-	/*
-	 * chrdev_lock is locked by counter_chrdev_open() and unlocked by
-	 * counter_chrdev_release(), so a mutex is not possible here because
-	 * chrdev_lock will invariably be held when returning to user space
-	 */
-	atomic_t chrdev_lock;
 	struct mutex ops_exist_lock;
 };
 
Signature · 1 line
2.25.1
#1 lore raw
Quoted 5 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Much better, thanks!

One remaining question:

include/linux/counter.h struct counter_ops { · line 300

Quoted 8 lines
include/linux/counter.h +0 -1
@@ -297,7 +297,6 @@ struct counter_ops {
  * @events:		queue of detected Counter events
  * @events_wait:	wait queue to allow blocking reads of Counter events
  * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
  * @ops_exist_lock:	lock to prevent use during removal

Why do you still need 2 locks for the same structure?

thanks,

greg k-h

#2 lore raw
Quoted 22 lines · Greg KH
Quoted 5 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Much better, thanks!

One remaining question:

include/linux/counter.h struct counter_ops { · line 300

Quoted 8 lines
include/linux/counter.h +0 -1
@@ -297,7 +297,6 @@ struct counter_ops {
  * @events:		queue of detected Counter events
  * @events_wait:	wait queue to allow blocking reads of Counter events
  * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
  * @ops_exist_lock:	lock to prevent use during removal

Why do you still need 2 locks for the same structure?

Quoted 3 lines

thanks,

greg k-h

Originally there was only the events_lock mutex. Initially I tried using it to also limit the chrdev to a single open, but then came across a "lock held when returning to user space" warning:
https://lore.kernel.org/linux-arm-kernel/YOq19zTsOzKA8v7c@shinobu/T/#m6072133d418d598a5f368bb942c945e46cfab9a5

Instead of losing the benefits of a mutex lock for protecting the events, I ultimately implemented the chrdev_lock separately as an atomic_t. If the chrdev_lock is removed, then we'll use events_lock solely from now on for this structure.

William Breathitt Gray

#3 lore raw
Quoted 32 lines · William Breathitt Gray
Quoted 22 lines · Greg KH
Quoted 5 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Much better, thanks!

One remaining question:

include/linux/counter.h struct counter_ops { · line 300

Quoted 8 lines
include/linux/counter.h +0 -1
@@ -297,7 +297,6 @@ struct counter_ops {
  * @events:		queue of detected Counter events
  * @events_wait:	wait queue to allow blocking reads of Counter events
  * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
  * @ops_exist_lock:	lock to prevent use during removal

Why do you still need 2 locks for the same structure?

Quoted 3 lines

thanks,

greg k-h

Originally there was only the events_lock mutex. Initially I tried using it to also limit the chrdev to a single open, but then came across a "lock held when returning to user space" warning:
https://lore.kernel.org/linux-arm-kernel/YOq19zTsOzKA8v7c@shinobu/T/#m6072133d418d598a5f368bb942c945e46cfab9a5

Quoted 4 lines

Instead of losing the benefits of a mutex lock for protecting the events, I ultimately implemented the chrdev_lock separately as an atomic_t. If the chrdev_lock is removed, then we'll use events_lock solely from now on for this structure.

chrdev_lock should be removed, it doesn't really do what you think it does, as per the thread yesterday :)

So does this mean you can also drop the ops_exist_lock?

thanks,

greg k-h

#4 lore raw
Quoted 38 lines · Greg KH
Quoted 32 lines · William Breathitt Gray
Quoted 22 lines · Greg KH
Quoted 5 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Much better, thanks!

One remaining question:

include/linux/counter.h struct counter_ops { · line 300

Quoted 8 lines
include/linux/counter.h +0 -1
@@ -297,7 +297,6 @@ struct counter_ops {
  * @events:		queue of detected Counter events
  * @events_wait:	wait queue to allow blocking reads of Counter events
  * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
  * @ops_exist_lock:	lock to prevent use during removal

Why do you still need 2 locks for the same structure?

Quoted 3 lines

thanks,

greg k-h

Originally there was only the events_lock mutex. Initially I tried using it to also limit the chrdev to a single open, but then came across a "lock held when returning to user space" warning:
https://lore.kernel.org/linux-arm-kernel/YOq19zTsOzKA8v7c@shinobu/T/#m6072133d418d598a5f368bb942c945e46cfab9a5

Quoted 4 lines

Instead of losing the benefits of a mutex lock for protecting the events, I ultimately implemented the chrdev_lock separately as an atomic_t. If the chrdev_lock is removed, then we'll use events_lock solely from now on for this structure.

Quoted 8 lines

chrdev_lock should be removed, it doesn't really do what you think it does, as per the thread yesterday :)

So does this mean you can also drop the ops_exist_lock?

thanks,

greg k-h

When counter_unregister is called, the ops member is set to NULL to indicate that the driver will be removed and that no new device operations should occur (because the ops callbacks will no longer be valid). The ops_exist_lock is used to allow existing ops callback dereferences to complete before the driver is removed so that we do not suffer a page fault.

I don't believe we can remove this protection (or can we?) but perhaps we can merge the three mutex locks (n_events_list_lock, events_lock, and ops_exist_lock) into a single "counter_lock" that handles all mutex locking for this structure.

William Breathitt Gray

#5 lore raw
Quoted 55 lines · William Breathitt Gray
Quoted 38 lines · Greg KH
Quoted 32 lines · William Breathitt Gray
Quoted 22 lines · Greg KH
Quoted 5 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Much better, thanks!

One remaining question:

include/linux/counter.h struct counter_ops { · line 300

Quoted 8 lines
include/linux/counter.h +0 -1
@@ -297,7 +297,6 @@ struct counter_ops {
   * @events:		queue of detected Counter events
   * @events_wait:	wait queue to allow blocking reads of Counter events
   * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
   * @ops_exist_lock:	lock to prevent use during removal

Why do you still need 2 locks for the same structure?

Quoted 3 lines

thanks,

greg k-h

Originally there was only the events_lock mutex. Initially I tried using it to also limit the chrdev to a single open, but then came across a "lock held when returning to user space" warning:
https://lore.kernel.org/linux-arm-kernel/YOq19zTsOzKA8v7c@shinobu/T/#m6072133d418d598a5f368bb942c945e46cfab9a5

Quoted 4 lines

Instead of losing the benefits of a mutex lock for protecting the events, I ultimately implemented the chrdev_lock separately as an atomic_t. If the chrdev_lock is removed, then we'll use events_lock solely from now on for this structure.

Quoted 8 lines

chrdev_lock should be removed, it doesn't really do what you think it does, as per the thread yesterday :)

So does this mean you can also drop the ops_exist_lock?

thanks,

greg k-h

When counter_unregister is called, the ops member is set to NULL to indicate that the driver will be removed and that no new device operations should occur (because the ops callbacks will no longer be valid). The ops_exist_lock is used to allow existing ops callback dereferences to complete before the driver is removed so that we do not suffer a page fault.

Quoted 5 lines

I don't believe we can remove this protection (or can we?) but perhaps we can merge the three mutex locks (n_events_list_lock, events_lock, and ops_exist_lock) into a single "counter_lock" that handles all mutex locking for this structure.

The different mutexes protect individual parts of the counter struct rather than the struct as a whole (a linked list, kfifo reads, and callback ops), so I think it makes the code clearer having individual mutexes for each rather than having a global mutex for unrelated actions.

#6 lore raw
Quoted 62 lines · David Lechner
Quoted 55 lines · William Breathitt Gray
Quoted 38 lines · Greg KH
Quoted 32 lines · William Breathitt Gray
Quoted 22 lines · Greg KH
Quoted 5 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Much better, thanks!

One remaining question:

include/linux/counter.h struct counter_ops { · line 300

Quoted 8 lines
include/linux/counter.h +0 -1
@@ -297,7 +297,6 @@ struct counter_ops {
   * @events:		queue of detected Counter events
   * @events_wait:	wait queue to allow blocking reads of Counter events
   * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
   * @ops_exist_lock:	lock to prevent use during removal

Why do you still need 2 locks for the same structure?

Quoted 3 lines

thanks,

greg k-h

Originally there was only the events_lock mutex. Initially I tried using it to also limit the chrdev to a single open, but then came across a "lock held when returning to user space" warning:
https://lore.kernel.org/linux-arm-kernel/YOq19zTsOzKA8v7c@shinobu/T/#m6072133d418d598a5f368bb942c945e46cfab9a5

Quoted 4 lines

Instead of losing the benefits of a mutex lock for protecting the events, I ultimately implemented the chrdev_lock separately as an atomic_t. If the chrdev_lock is removed, then we'll use events_lock solely from now on for this structure.

Quoted 8 lines

chrdev_lock should be removed, it doesn't really do what you think it does, as per the thread yesterday :)

So does this mean you can also drop the ops_exist_lock?

thanks,

greg k-h

When counter_unregister is called, the ops member is set to NULL to indicate that the driver will be removed and that no new device operations should occur (because the ops callbacks will no longer be valid). The ops_exist_lock is used to allow existing ops callback dereferences to complete before the driver is removed so that we do not suffer a page fault.

Quoted 5 lines

I don't believe we can remove this protection (or can we?) but perhaps we can merge the three mutex locks (n_events_list_lock, events_lock, and ops_exist_lock) into a single "counter_lock" that handles all mutex locking for this structure.

Quoted 5 lines

The different mutexes protect individual parts of the counter struct rather than the struct as a whole (a linked list, kfifo reads, and callback ops), so I think it makes the code clearer having individual mutexes for each rather than having a global mutex for unrelated actions.

That's a fair point. Because these operations are independent of each other, keeping the mutexes separate would allow users to configure the next events watch list without necessarily stalling the current read() of events. I doubt the difference is significant, but if the purpose of these locks are well enough defined then independent mutex locks may not be a problem.

William Breathitt Gray

#7 lore raw
In reply to David Lechner
Quoted 8 lines · David Lechner

This removes the chrdev_lock from the counter subsystem. This was intended to prevent opening the chrdev more than once. However, this doesn't work in practice since userspace can duplicate file descriptors and pass file descriptors to other processes. Since this protection can't be relied on, it is best to just remove it.

Hi David,

If userspace can bypass this protection then we might as well remove the code as moot. In agree in principle to this change, but I do have some comments inline below.

Quoted 30 lines
---
 drivers/counter/counter-chrdev.c |  6 ------
 drivers/counter/counter-sysfs.c  | 13 +++----------
 include/linux/counter.h          |  7 -------
 3 files changed, 3 insertions(+), 23 deletions(-)
drivers/counter/counter-chrdev.c +0 -5
index 967c94ae95bb..b747dc81cfc6 100644
@@ -384,10 +384,6 @@ static int counter_chrdev_open(struct inode *inode, struct file *filp)
 							    typeof(*counter),
 							    chrdev);
 
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
-
 	get_device(&counter->dev);
 	filp->private_data = counter;
 
@@ -419,7 +415,6 @@ static int counter_chrdev_release(struct inode *inode, struct file *filp)
 	mutex_unlock(&counter->ops_exist_lock);
 
 	put_device(&counter->dev);
-	atomic_dec(&counter->chrdev_lock);
 
 	return ret;
 }

drivers/counter/counter-chrdev.c int counter_chrdev_add(struct counter_device *const counter) · line 442

Quoted 3 lines
drivers/counter/counter-chrdev.c +0 -0
@@ -445,7 +440,6 @@ int counter_chrdev_add(struct counter_device *const counter)
 	mutex_init(&counter->events_lock);
 
 	/* Initialize character device */

We can remove this comment because the purpose of calling cdev_init() is obvious enough from the name.

Quoted 18 lines
drivers/counter/counter-chrdev.c +0 -1
-	atomic_set(&counter->chrdev_lock, 0);
 	cdev_init(&counter->chrdev, &counter_fops);
 
 	/* Allocate Counter events queue */
drivers/counter/counter-sysfs.c +1 -5
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
 					   u64 val)
 {
 	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 10 lines
drivers/counter/counter-sysfs.c +1 -1
 
 	/* Allocate new events queue */
 	err = kfifo_alloc(&events, val, GFP_KERNEL);
 	if (err)
-		goto exit_early;
+		return err;
 
 	/* Swap in new events queue */
 	kfifo_free(&counter->events);
 	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

William Breathitt Gray

Quoted 37 lines
drivers/counter/counter-sysfs.c +1 -4
-exit_early:
-	atomic_dec(&counter->chrdev_lock);
-
-	return err;
+	return 0;
 }
 
 static struct counter_comp counter_num_signals_comp =
include/linux/counter.h +0 -7
index 22b14a552b1d..0fd99e255a50 100644
@@ -297,7 +297,6 @@ struct counter_ops {
  * @events:		queue of detected Counter events
  * @events_wait:	wait queue to allow blocking reads of Counter events
  * @events_lock:	lock to protect Counter events queue read operations
- * @chrdev_lock:	lock to limit chrdev to a single open at a time
  * @ops_exist_lock:	lock to prevent use during removal
  */
 struct counter_device {
@@ -325,12 +324,6 @@ struct counter_device {
 	DECLARE_KFIFO_PTR(events, struct counter_event);
 	wait_queue_head_t events_wait;
 	struct mutex events_lock;
-	/*
-	 * chrdev_lock is locked by counter_chrdev_open() and unlocked by
-	 * counter_chrdev_release(), so a mutex is not possible here because
-	 * chrdev_lock will invariably be held when returning to user space
-	 */
-	atomic_t chrdev_lock;
 	struct mutex ops_exist_lock;
 };
 
Signature · 1 line
2.25.1
#8 lore raw
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

#9 lore raw
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

As a side note, you can also remove the atomic.h includes from these files now as well because we won't be working with an atomic_t anymore.

William Breathitt Gray

#10 lore raw
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

thanks,

greg k-h

#11 lore raw
Quoted 73 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

Quoted 7 lines

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

Thanks,

William Breathitt Gray

#12 lore raw
Quoted 78 lines · William Breathitt Gray
Quoted 77 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

Quoted 7 lines
Quoted 3 lines

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

Wait, no, you need one patch to remove the atomic lock for the open "protection" and then another one for the other locks. The original patch here was fine, but can be part of a patch series, don't lump them all together into one huge change.

thanks,

greg k-h

#13 lore raw
Quoted 87 lines · Greg KH
Quoted 78 lines · William Breathitt Gray
Quoted 77 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

Quoted 7 lines
Quoted 3 lines

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

Quoted 8 lines

Wait, no, you need one patch to remove the atomic lock for the open "protection" and then another one for the other locks. The original patch here was fine, but can be part of a patch series, don't lump them all together into one huge change.

thanks,

greg k-h

Understood. I'll provide my ack for this patch here then.

#14 lore raw
Quoted 93 lines · William Breathitt Gray
Quoted 92 lines · Greg KH
Quoted 78 lines · William Breathitt Gray
Quoted 77 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
  					   u64 val)
  {
  	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
  	/* Allocate new events queue */
  	err = kfifo_alloc(&events, val, GFP_KERNEL);
  	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
  	/* Swap in new events queue */
  	kfifo_free(&counter->events);
  	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

Quoted 7 lines
Quoted 3 lines

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

Wait, no, you need one patch to remove the atomic lock for the open "protection" and then another one for the other locks. The original patch here was fine, but can be part of a patch series, don't lump them all together into one huge change.

Quoted 7 lines
Quoted 3 lines

thanks,

greg k-h

Understood. I'll provide my ack for this patch here then.

Thanks, now queued up!

greg k-h

#15 lore raw
In reply to William Breathitt Gray
Quoted 78 lines · William Breathitt Gray
Quoted 77 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
   					   u64 val)
   {
   	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
   	/* Allocate new events queue */
   	err = kfifo_alloc(&events, val, GFP_KERNEL);
   	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
   	/* Swap in new events queue */
   	kfifo_free(&counter->events);
   	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

Quoted 8 lines
Quoted 3 lines

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

We can't use a spin lock for everything since there are operations that can sleep that need to be in the critical sections. Likewise, we can't use a mutex for everything since some critical sections are in interrupt handlers. But, I suppose we can try combining the existing mutexes. Since the kfifo is accessed from both contexts, it seems like it still needs more consideration than just a mutex or a spin lock, e.g. if events are enabled, don't allow swapping out the kfifo buffer.

#16 lore raw
Quoted 88 lines · David Lechner
Quoted 78 lines · William Breathitt Gray
Quoted 77 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
   					   u64 val)
   {
   	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
   	/* Allocate new events queue */
   	err = kfifo_alloc(&events, val, GFP_KERNEL);
   	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
   	/* Swap in new events queue */
   	kfifo_free(&counter->events);
   	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

Quoted 8 lines
Quoted 3 lines

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

Quoted 8 lines

We can't use a spin lock for everything since there are operations that can sleep that need to be in the critical sections. Likewise, we can't use a mutex for everything since some critical sections are in interrupt handlers. But, I suppose we can try combining the existing mutexes. Since the kfifo is accessed from both contexts, it seems like it still needs more consideration than just a mutex or a spin lock, e.g. if events are enabled, don't allow swapping out the kfifo buffer.

I think there is a deadlock case if we combine the ops_exists_lock with the n_events_list_lock, so this will need further thought. However, at the very least the swap occuring in counter_events_queue_size_write() and the kfifo_in() in counter_push_events() require some sort of locking; it is trivial to cause a page fault with the code in its current state.

I think this can be fixed if just events_lock is changed to spinlock for now without modifying the other locks. We can try to combine the remaining locks in a subsequent patch, if they are capable of being combined.

William Breathitt Gray

#17 lore raw
Quoted 105 lines · William Breathitt Gray
Quoted 88 lines · David Lechner
Quoted 78 lines · William Breathitt Gray
Quoted 77 lines · Greg KH
Quoted 67 lines · William Breathitt Gray
Quoted 36 lines · David Lechner
Quoted 22 lines · William Breathitt Gray
Quoted 21 lines · David Lechner
drivers/counter/counter-sysfs.c +2 -6
index 1ccd771da25f..7bf8882ff54d 100644
@@ -796,25 +796,18 @@ static int counter_events_queue_size_write(struct counter_device *counter,
   					   u64 val)
   {
   	DECLARE_KFIFO_PTR(events, struct counter_event);
-	int err = 0;
-
-	/* Ensure chrdev is not opened more than 1 at a time */
-	if (!atomic_add_unless(&counter->chrdev_lock, 1, 1))
-		return -EBUSY;
+	int err;
 
   	/* Allocate new events queue */
   	err = kfifo_alloc(&events, val, GFP_KERNEL);
   	if (err)
-		goto exit_early;
+		return err;
 
Quoted 9 lines

drivers/counter/counter-sysfs.c static int counter_events_queue_size_write(struct counter_device *counter, · line 808

Quoted 3 lines
drivers/counter/counter-sysfs.c +0 -0
   	/* Swap in new events queue */
   	kfifo_free(&counter->events);
   	counter->events.kfifo = events.kfifo;

Do we need to hold the events_lock mutex here for this swap in case counter_chrdev_read() is in the middle of reading the kfifo to userspace, or do the kfifo macros already protect us from a race condition here?

Another possibility might be to disallow changing the size while events are enabled. Otherwise, we also need to protect against write after free.

Quoted 6 lines

I considered this:

	swap(counter->events.kfifo, events.kfifo);
	kfifo_free(&events);

But I'm not sure that would be safe enough.

I think it depends on whether it's safe to call kfifo_free() while other kfifo_*() calls are executing. I suspect it is not safe because I don't think kfifo_free() waits until all kfifo read/write operations are finished before freeing -- but if I'm wrong here please let me know.

Because of that, will need to hold the counter->events_lock afterall so that we don't modify the events fifo while a kfifo read/write is going on, lest we suffer an address fault. This can happen regardless of whether you swap before or after the kfifo_free() because the old fifo address could still be in use within those uncompleted kfifo_*() calls if they were called before the swap but don't complete before the kfifo_free().

So we have a problem now that I think you have already noticed: the kfifo_in() call in counter_push_events() also needs protection, but it's executing within an interrupt context so we can't try to lock a mutex lest we end up sleeping.

One option we have is as you suggested: we disallow changing size while events are enabled. However, that will require us to keep track of when events are disabled and implement a spinlock to ensure that we don't disable events in the middle of a kfifo_in().

Quoted 4 lines

Alternatively, we could change events_lock to a spinlock and use it to protect all these operations on the counter->events fifo. Would this alternative be a better option so that we avoid creating another separate lock?

I would recommend just having a single lock here if at all possible, until you determine that there a performance problem that can be measured that would require it to be split up.

Quoted 8 lines
Quoted 3 lines

thanks,

greg k-h

All right let's go with a single events_lock spinlock then. David, if you make those changes and submit a v2, I'll be okay with this patch and can provide my ack for it.

Quoted 8 lines

We can't use a spin lock for everything since there are operations that can sleep that need to be in the critical sections. Likewise, we can't use a mutex for everything since some critical sections are in interrupt handlers. But, I suppose we can try combining the existing mutexes. Since the kfifo is accessed from both contexts, it seems like it still needs more consideration than just a mutex or a spin lock, e.g. if events are enabled, don't allow swapping out the kfifo buffer.

I think there is a deadlock case if we combine the ops_exists_lock with the n_events_list_lock, so this will need further thought. However, at the very least the swap occuring in counter_events_queue_size_write() and the kfifo_in() in counter_push_events() require some sort of locking; it is trivial to cause a page fault with the code in its current state.

Quoted 6 lines

I think this can be fixed if just events_lock is changed to spinlock for now without modifying the other locks. We can try to combine the remaining locks in a subsequent patch, if they are capable of being combined.

William Breathitt Gray

After considering this further, kfifo_to_user() could possibly sleep so we can't use a spinlock here afterall. As such, events_lock should remain as a mutex and instead we'll only allow swapping out the kfifo buffer when events are disabled.

William Breathitt Gray