perf tools: avoid possible race condition in copyfile
PATCH 1/1 perf tools: avoid possible race condition in copyfile
Use unique temporary files when copying to buildid dir to prevent races in case multiple instances are trying to copy same file. This is done by
- creating template in form <path>/.<filename>.XXXXXX where the suffix is used by mkstemp() to create unique file
- change file mode
- copy content
- if successful link temp file to target file
- unlink temp file
At this point the only file left at target path should be the desired one either created by us or other instance if we raced. This should also prevent not yet fully copied files to be visible to to other perf instances that could try to parse them.
On top of that slow_copyfile no longer needs to deal with file mode when creating file since temporary file is already created and mode is set.
--- tools/perf/util/util.c | 48 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 16 deletions(-)
tools/perf/util/util.c +32 -16
@@ -72,20 +72,17 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } -static int slow_copyfile (const char *from,const char *to,mode_t mode) +static int slow_copyfile (const char *from,const char *to) {int err =-1 ;char *line =NULL ;size_t n;FILE *from_fp =fopen (from,"r" ), *to_fp; -mode_t old_umask;if (from_fp ==NULL ) goto out; - old_umask =umask (mode ^0777 ); to_fp =fopen (to,"w" ); -umask (old_umask);if (to_fp ==NULL ) goto out_fclose_from;
@@ -108,36 +105,55 @@ int copyfile_mode(const char *from, const char *to, mode_t mode)struct stat st;void *addr;int err =-1 ; +char *tmp =NULL , *ptr =NULL ;if (stat (from, &st)) goto out; -if (st.st_size ==0 )/* /proc? do it slowly... */ -return slow_copyfile (from, to, mode); - - fromfd =open (from,O_RDONLY ); -if (fromfd <0 ) +/* extra 'x' at the end is to reserve space for '.' */ +if (asprintf (&tmp,"%s.XXXXXXx" , to) <0 ) { + tmp =NULL ; goto out; + } + ptr =strrchr (tmp,'/' ); +if (!ptr) + goto out; + ptr =memmove (ptr +1 , ptr,strlen (ptr) -1 ); + *ptr ='.' ; - tofd =creat (to, mode); + tofd =mkstemp (tmp);if (tofd <0 ) - goto out_close_from; + goto out; + +if (fchmod (tofd, mode)) + goto out_close_to; + +if (st.st_size ==0 ) {/* /proc? do it slowly... */ + err =slow_copyfile (from, tmp); + goto out_close_to; + } + + fromfd =open (from,O_RDONLY ); +if (fromfd <0 ) + goto out_close_to; addr =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fromfd,0 );if (addr ==MAP_FAILED ) - goto out_close_to; + goto out_close_from;if (write (tofd, addr, st.st_size) == st.st_size) err =0 ;munmap (addr, st.st_size); -out_close_to: -close (tofd); -if (err) -unlink (to); out_close_from:close (fromfd); +out_close_to: +close (tofd); +if (!err) + err =link (tmp, to); +unlink (tmp); out: +free (tmp);return err; }
Signature · 1 line
2.4.2
Quoted 15 lines · Milos Vyletel
Use unique temporary files when copying to buildid dir to prevent races in case multiple instances are trying to copy same file. This is done by
- creating template in form <path>/.<filename>.XXXXXX where the suffix is used by
mkstemp()to create unique file
- change file mode
- copy content
- if successful link temp file to target file
- unlink temp fileAt this point the only file left at target path should be the desired one either created by us or other instance if we raced. This should also prevent not yet fully copied files to be visible to to other perf instances that could try to parse them.
Quoted 4 lines
On top of that
slow_copyfileno longer needs to deal with file mode when creating file since temporary file is already created and mode is set.Signed-off-by Milos Vyletel <[email protected]>
Ok, that looks nice!
Assuming it passes testing you can add my ack to it:
Is there any other place in tools/perf where we are using file locking or racy shared access to the same file(s)?
Thanks,
Ingo
Quoted 18 lines · Ingo Molnar
Quoted 15 lines · Milos Vyletel
Use unique temporary files when copying to buildid dir to prevent races in case multiple instances are trying to copy same file. This is done by
- creating template in form <path>/.<filename>.XXXXXX where the suffix is used by
mkstemp()to create unique file
- change file mode
- copy content
- if successful link temp file to target file
- unlink temp fileAt this point the only file left at target path should be the desired one either created by us or other instance if we raced. This should also prevent not yet fully copied files to be visible to to other perf instances that could try to parse them.
Quoted 6 lines
Quoted 4 lines
On top of that
slow_copyfileno longer needs to deal with file mode when creating file since temporary file is already created and mode is set.Signed-off-by Milos Vyletel <[email protected]>Ok, that looks nice!
Thanks.
Quoted 3 lines
Assuming it passes testing you can add my ack to it:
I did some testing myself by running perf record, archive and reading the data on other system as well as running perf buildid-cache on perf binary itself. I did revert fix from 0635b0f that to expose the race with EEXIST and my recreator test passed sucessfully. I've also added debug printfs in the code to really make sure the temporary files are created, exist and then are moved back.
Everything worked as expected but this can surely use some more testing.
Quoted 5 lines
Acked-by Ingo Molnar <[email protected]>Is there any other place in
tools/perfwhere we are using file locking or racy shared access to the same file(s)?
There are two places in code I'm aware of which copy files and the both end up calling copyfile_mode that this patch modifies. Those places are
build_id_cache__add_kcore
kcore_copy
kcore_copy__copy_file
copyfile_mode
build_id_cache__add_s
copyfile
copyfile_mode
unless I'm missing something we should cover all cases.
Milos
Arnaldo Carvalho de Melo
maintainerEm Thu, Jun 04, 2015 at 11:11:00AM +0200, Ingo Molnar escreveu:
Quoted 11 lines · Milos Vyletel
Quoted 4 lines
On top of that
slow_copyfileno longer needs to deal with file mode when creating file since temporary file is already created and mode is set.Signed-off-by Milos Vyletel <[email protected]>Ok, that looks nice!
Assuming it passes testing you can add my ack to it:
Acked-by Ingo Molnar <[email protected]>
Can you please try applying it on tip/perf/core instead? I think it is clashing with:
commit 9c9f5a2f1944e8b6bf2b618d04b31e1c1760637e
Author: Namhyung Kim <[email protected]> Date: Mon May 18 09:30:18 2015 +0900
perf tools: Introduce copyfile_offset() function
- Arnaldo
Quoted 13 lines · Arnaldo Carvalho de Melo
Em Thu, Jun 04, 2015 at 11:11:00AM +0200, Ingo Molnar escreveu:
Quoted 11 lines · Milos Vyletel
Quoted 4 lines
On top of that
slow_copyfileno longer needs to deal with file mode when creating file since temporary file is already created and mode is set.Signed-off-by Milos Vyletel <[email protected]>Ok, that looks nice!
Assuming it passes testing you can add my ack to it:
Acked-by Ingo Molnar <[email protected]>
Quoted 9 lines
Can you please try applying it on tip/perf/core instead? I think it is clashing with:
commit
9c9f5a2f1944e8b6bf2b618d04b31e1c1760637eAuthor: Namhyung Kim <[email protected]> Date: Mon May 18 09:30:18 2015 +0900perf tools: Introduce
copyfile_offset()function
And also with
commit 0b1de0be1eac7b23e89cb43c17b02d38ead6b6c8
Author: Namhyung Kim <[email protected]> Date: Mon May 18 09:30:17 2015 +0900
perf tools: Add rm_rf() utility function
Will resend version based on tip/perf/core after I rerun my testing.
Milos
Arnaldo Carvalho de Melo
maintainerEm Mon, Jun 08, 2015 at 03:58:43PM +0200, Milos Vyletel escreveu:
Quoted 12 lines · Arnaldo Carvalho de Melo
Quoted 8 lines
Can you please try applying it on tip/perf/core instead? I think it is clashing with:
commit
9c9f5a2f1944e8b6bf2b618d04b31e1c1760637eAuthor: Namhyung Kim <[email protected]> Date: Mon May 18 09:30:18 2015 +0900perf tools: Introduce
copyfile_offset()functionAnd also with
Quoted 7 lines
commit
0b1de0be1eac7b23e89cb43c17b02d38ead6b6c8Author: Namhyung Kim <[email protected]> Date: Mon May 18 09:30:17 2015 +0900perf tools: Add
rm_rf()utility functionWill resend version based on tip/perf/core after I rerun my testing.
Thanks a lot!
- Arnaldo