remove_proc_entry and read_proc
Quoted 21 lines · Duncan Sands
Quoted 10 lines · On Wednesday 31 January 2007 19:42:51 Alexey Dobriyan wrote:
Quoted 8 lines · Duncan Sands
Can
read_procstill be executing whenremove_proc_entryreturns?In my driver [*] I allocate some data and create a proc entry using
create_proc_entry. My read method reads from my allocated data. When shutting down, I callremove_proc_entryand immediately free the data. If some call toread_procis still executing at this point then it will be accessing freed memory. Can this happen? I've been rummaging around in fs/proc to see what prevents it, but didn't find anything yet.Quoted 4 lines
This should be fixed by the following patch (in -mm currently):
http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20-rc6/2.6.20-rc6-mm3/broken-out/fix-rmmod-read-write-races-in-proc-entries.patchTell me if you're unsure it will.
Excellent! But tell me,
(quoted diff) +2
-0+ atomic_inc(&dp->pde_users); + if (!dp->proc_fops)
Quoted 2 lines
don't you need a memory barrier between these two? Also a corresponding one where
proc_fopsis set to NULL.
I believe, barriers not needed, not now.
This scheme relies on the fact that remove_proc_entry() will be the only place that will clear ->proc_fops and, once cleared, ->proc_fops will never be resurrected. Clearing of ->proc_fops will eventually propagate to CPU doing first check, thus preveting refcount bumps from this CPU. What can be missed is some "rogue" readers or writers¹. Big deal.
Quoted 16 lines
(quoted diff) +11
-0+ /* + * Stop accepting new readers/writers. If you're dynamically + * allocating ->proc_fops, save a pointer somewhere. + */ + de->proc_fops = NULL; + /* Wait until all readers/writers are done. */ + if (atomic_read(&de->pde_users) > 0) { + spin_unlock(&proc_subdir_lock); + msleep(1); + goto again; + }I don't understand how this is supposed to work. Consider
CPU1 CPU2
Quoted 7 lines
atomic_inc(&dp->pde_users);
if (dp->proc_fops)de->proc_fops = NULL; use_proc_fops <= BOOM if (atomic_read(&de->pde_users) > 0) {what prevents dereference of a NULL
proc_fopsvalue?
¹ Sigh, modules should do removals of proc entries first. And I should check
for that.
Hi Alexey,
Quoted 7 lines
I believe, barriers not needed, not now.
This scheme relies on the fact that
remove_proc_entry()will be the only place that will clear ->proc_fops and, once cleared, ->proc_fops will never be resurrected. Clearing of ->proc_fops will eventually propagate to CPU doing first check, thus preveting refcount bumps from this CPU. What can be missed is some "rogue" readers or writers¹. Big deal.
I don't understand you. Without memory barriers, remove_proc_entry will most of the time, but not all of the time, wait for all readers and writers to finish before exiting. Since the whole point of your patch was to ensure that all readers and writers finish before remove_proc_entry exits, I don't understand why you don't just put the memory barriers in and make it correct.
Also, I do consider it a big deal:
Quoted 2 lines
¹ Sigh, modules should do removals of proc entries first. And I should check for that.
Modules should of course call remove_proc_entry before exiting. However right now, even with your patch, a read or write method can still be running when remove_proc_entry returns [1], so could still be running when the module is removed (if they sleep; I guess this applies mostly to write methods). This is very bad - why not put in memory barriers and fix it? Also, plenty of proc read and write methods access private data that is allocated before calling create_proc_entry and freed after calling remove_proc_entry. If a read or write method is still running after remove_proc_entry returns, then it can access freed memory - very bad.
Ciao,
Duncan.
[1] proc_get_inode does a try_module_get, so it is possible that module unloading is not a problem - not sure.
Quoted 8 lines · Duncan Sands
Quoted 7 lines
I believe, barriers not needed, not now.
This scheme relies on the fact that
remove_proc_entry()will be the only place that will clear ->proc_fops and, once cleared, ->proc_fops will never be resurrected. Clearing of ->proc_fops will eventually propagate to CPU doing first check, thus preveting refcount bumps from this CPU. What can be missed is some "rogue" readers or writers¹. Big deal.
Quoted 5 lines
I don't understand you. Without memory barriers,
remove_proc_entrywill most of the time, but not all of the time, wait for all readers and writers to finish before exiting. Since the whole point of your patch was to ensure that all readers and writers finish beforeremove_proc_entryexits, I don't understand why you don't just put the memory barriers in and make it correct.
Gee, thanks. I sat and wrote code side-by-side, and it looks like, even barriers won't fix anything, because they don't affect other CPUs. There will be new patch soon.
->proc_fops is valid ->proc_fops is valid ->pde_users is 0 ->pde_users is 0 ------------------------------------------------------------ if (!pde->proc_fops) goto out; ->proc_fops = NULL; if (atomic_read(->pde_users) > 0) goto again; | | atomic_inc(->pde_users); | | | V
Quoted 15 lines
Also, I do consider it a big deal:
Quoted 2 lines
¹ Sigh, modules should do removals of proc entries first. And I should check for that.
Modules should of course call
remove_proc_entrybefore exiting. However right now, even with your patch, a read or write method can still be running whenremove_proc_entryreturns [1], so could still be running when the module is removed (if they sleep; I guess this applies mostly to write methods). This is very bad - why not put in memory barriers and fix it? Also, plenty of proc read and write methods access private data that is allocated before callingcreate_proc_entryand freed after callingremove_proc_entry. If a read or write method is still running afterremove_proc_entryreturns, then it can access freed memory - very bad.
Quoted 2 lines
[1]
proc_get_inodedoes atry_module_get, so it is possible that module unloading is not a problem - not sure.
Modules forget to set ->owner sometimes. Also, it's still racy, because of the typical
pde = create_proc_entry(); /* * * ->owner is NULL here, effectively, PDE without ->owner. * */ if (pde) pde->owner = THIS_MODULE;
Quoted 2 lines
Gee, thanks. I sat and wrote code side-by-side, and it looks like, even barriers won't fix anything, because they don't affect other CPUs.
?! The whole point of memory barriers is that they affect other CPUs. Maybe you are thinking of compiler barriers?
Quoted 10 lines
->proc_fops is valid ->proc_fops is valid ->pde_users is 0 ->pde_users is 0 ------------------------------------------------------------ if (!pde->proc_fops) goto out; ->proc_fops = NULL; if (atomic_read(->pde_users) > 0)
Quoted 8 lines
goto again; | | atomic_inc(->pde_users); | | | V
The proc_fops check *before* the atomic_inc is indeed pointless (notice how I removed it in the patch I sent?). It's the one after the atomic_inc that prevents this race, but only if there is a memory barrier between the atomic_inc and the check... because otherwise they could be reordered (i.e. seen in reverse order by another CPU) giving the race.
Quoted 11 lines
Modules forget to set ->owner sometimes. Also, it's still racy, because of the typical
pde = create_proc_entry(); /* * * ->owner is NULL here, effectively, PDE without ->owner. * */ if (pde) pde->owner = THIS_MODULE;
As long as the module calls remove_proc_entry before being unloaded, this should be ok.
Best wishes,
Duncan.