Skip to contentSkip to conversation

remove_proc_entry and read_proc

DISCUSSION Alexey Dobriyan 2 participants 4 messages Feb 1, 2007 – Feb 5, 2007

lore
Share & tools
lore
#0 lore raw
Quoted 21 lines · Duncan Sands
Quoted 10 lines · On Wednesday 31 January 2007 19:42:51 Alexey Dobriyan wrote:
Quoted 8 lines · Duncan Sands

Can read_proc still be executing when remove_proc_entry returns?

In my driver [*] I allocate some data and create a proc entry using create_proc_entry. My read method reads from my allocated data. When shutting down, I call remove_proc_entry and immediately free the data. If some call to read_proc is still executing at this point then it will be accessing freed memory. Can this happen? I've been rummaging around in fs/proc to see what prevents it, but didn't find anything yet.

Quoted 4 lines

This should be fixed by the following patch (in -mm currently):
http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20-rc6/2.6.20-rc6-mm3/broken-out/fix-rmmod-read-write-races-in-proc-entries.patch

Tell me if you're unsure it will.

Excellent! But tell me,

(quoted diff) +2 -0
+	atomic_inc(&dp->pde_users);
+	if (!dp->proc_fops)
Quoted 2 lines

don't you need a memory barrier between these two? Also a corresponding one where proc_fops is set to NULL.

I believe, barriers not needed, not now.

This scheme relies on the fact that remove_proc_entry() will be the only place that will clear ->proc_fops and, once cleared, ->proc_fops will never be resurrected. Clearing of ->proc_fops will eventually propagate to CPU doing first check, thus preveting refcount bumps from this CPU. What can be missed is some "rogue" readers or writers¹. Big deal.

Quoted 16 lines
(quoted diff) +11 -0
+		/*
+		 * Stop accepting new readers/writers. If you're dynamically
+		 * allocating ->proc_fops, save a pointer somewhere.
+		 */
+		de->proc_fops = NULL;
+		/* Wait until all readers/writers are done. */
+		if (atomic_read(&de->pde_users) > 0) {
+			spin_unlock(&proc_subdir_lock);
+			msleep(1);
+			goto again;
+		}

I don't understand how this is supposed to work. Consider

CPU1 CPU2

Quoted 7 lines

atomic_inc(&dp->pde_users);
if (dp->proc_fops)

					de->proc_fops = NULL;
	use_proc_fops <= BOOM
					if (atomic_read(&de->pde_users) > 0) {

what prevents dereference of a NULL proc_fops value?

¹ Sigh, modules should do removals of proc entries first. And I should check
for that.

#1 lore raw

Hi Alexey,

Quoted 7 lines

I believe, barriers not needed, not now.

This scheme relies on the fact that remove_proc_entry() will be the only place that will clear ->proc_fops and, once cleared, ->proc_fops will never be resurrected. Clearing of ->proc_fops will eventually propagate to CPU doing first check, thus preveting refcount bumps from this CPU. What can be missed is some "rogue" readers or writers¹. Big deal.

I don't understand you. Without memory barriers, remove_proc_entry will most of the time, but not all of the time, wait for all readers and writers to finish before exiting. Since the whole point of your patch was to ensure that all readers and writers finish before remove_proc_entry exits, I don't understand why you don't just put the memory barriers in and make it correct.

Also, I do consider it a big deal:

Quoted 2 lines

¹ Sigh, modules should do removals of proc entries first. And I should check for that.

Modules should of course call remove_proc_entry before exiting. However right now, even with your patch, a read or write method can still be running when remove_proc_entry returns [1], so could still be running when the module is removed (if they sleep; I guess this applies mostly to write methods). This is very bad - why not put in memory barriers and fix it? Also, plenty of proc read and write methods access private data that is allocated before calling create_proc_entry and freed after calling remove_proc_entry. If a read or write method is still running after remove_proc_entry returns, then it can access freed memory - very bad.

Ciao,

Duncan.

[1] proc_get_inode does a try_module_get, so it is possible that module unloading is not a problem - not sure.

#2 lore raw
Quoted 8 lines · Duncan Sands
Quoted 7 lines

I believe, barriers not needed, not now.

This scheme relies on the fact that remove_proc_entry() will be the only place that will clear ->proc_fops and, once cleared, ->proc_fops will never be resurrected. Clearing of ->proc_fops will eventually propagate to CPU doing first check, thus preveting refcount bumps from this CPU. What can be missed is some "rogue" readers or writers¹. Big deal.

Quoted 5 lines

I don't understand you. Without memory barriers, remove_proc_entry will most of the time, but not all of the time, wait for all readers and writers to finish before exiting. Since the whole point of your patch was to ensure that all readers and writers finish before remove_proc_entry exits, I don't understand why you don't just put the memory barriers in and make it correct.

Gee, thanks. I sat and wrote code side-by-side, and it looks like, even barriers won't fix anything, because they don't affect other CPUs. There will be new patch soon.

	->proc_fops is valid			->proc_fops is valid
	->pde_users is 0			->pde_users is 0
	------------------------------------------------------------


						if (!pde->proc_fops)
							goto out;

	->proc_fops = NULL;
	if (atomic_read(->pde_users) > 0)
		goto again;

		|
		|				atomic_inc(->pde_users);
		|
		|
		|
		V
Quoted 15 lines

Also, I do consider it a big deal:

Quoted 2 lines

¹ Sigh, modules should do removals of proc entries first. And I should check for that.

Modules should of course call remove_proc_entry before exiting. However right now, even with your patch, a read or write method can still be running when remove_proc_entry returns [1], so could still be running when the module is removed (if they sleep; I guess this applies mostly to write methods). This is very bad - why not put in memory barriers and fix it? Also, plenty of proc read and write methods access private data that is allocated before calling create_proc_entry and freed after calling remove_proc_entry. If a read or write method is still running after remove_proc_entry returns, then it can access freed memory - very bad.

Quoted 2 lines

[1] proc_get_inode does a try_module_get, so it is possible that module unloading is not a problem - not sure.

Modules forget to set ->owner sometimes. Also, it's still racy, because of the typical

	pde = create_proc_entry();
	/*
	 *
	 * ->owner is NULL here, effectively, PDE without ->owner.
	 *
	 */
	if (pde)
		pde->owner = THIS_MODULE;
#3 lore raw
Quoted 2 lines

Gee, thanks. I sat and wrote code side-by-side, and it looks like, even barriers won't fix anything, because they don't affect other CPUs.

?! The whole point of memory barriers is that they affect other CPUs. Maybe you are thinking of compiler barriers?

Quoted 10 lines
	->proc_fops is valid			->proc_fops is valid
	->pde_users is 0			->pde_users is 0
	------------------------------------------------------------


						if (!pde->proc_fops)
							goto out;

	->proc_fops = NULL;
	if (atomic_read(->pde_users) > 0)
Quoted 8 lines
		goto again;

		|
		|				atomic_inc(->pde_users);
		|
		|
		|
		V

The proc_fops check *before* the atomic_inc is indeed pointless (notice how I removed it in the patch I sent?). It's the one after the atomic_inc that prevents this race, but only if there is a memory barrier between the atomic_inc and the check... because otherwise they could be reordered (i.e. seen in reverse order by another CPU) giving the race.

Quoted 11 lines

Modules forget to set ->owner sometimes. Also, it's still racy, because of the typical

	pde = create_proc_entry();
	/*
	 *
	 * ->owner is NULL here, effectively, PDE without ->owner.
	 *
	 */
	if (pde)
		pde->owner = THIS_MODULE;

As long as the module calls remove_proc_entry before being unloaded, this should be ok.

Best wishes,

Duncan.