make raid5 checksums preempt-safe
PATCH 1/1 make raid5 checksums preempt-safe
Linus,
The raid5 xor checksums use MMX/SSE state and are not preempt-safe.
Attached patch disables preemption in FPU_SAVE and XMMS_SAVE and restores it in FPU_RESTORE and XMMS_RESTORE - preventing preemption while in fp mode.
Please, apply.
Robert Love
include/asm-i386/xor.h +5 -1
@@ -20,6 +20,7 @@#define FPU_SAVE \ do { \ + preempt_disable(); \ if (!test_thread_flag(TIF_USEDFPU)) \ __asm__ __volatile__ (" clts;\n"); \ __asm__ __volatile__ ("fsave %0; fwait": "=m"(fpu_save[0])); \
@@ -30,6 +31,7 @@ __asm__ __volatile__ ("frstor %0" : :"m" (fpu_save[0 ])); \if (!test_thread_flag (TIF_USEDFPU )) \stts (); \ +preempt_enable (); \ } while (0)#define LD (x,y) " movq 8*("#x")(%1), %%mm"#y" ;\n"
@@ -543,6 +545,7 @@*/ #define XMMS_SAVE \ + preempt_disable(); \ __asm__ __volatile__ ( \ "movl %%cr0,%0 ;\n\t" \ "clts ;\n\t" \
@@ -564,7 +567,8 @@ "movl %0,%%cr0 ;\n\t" \ : \ : "r" (cr0), "r" (xmm_save) \ - : "memory") + : "memory") \ +preempt_enable ();#define ALIGN16 __attribute__((aligned(16)))
include/asm-x86_64/xor.h +4 -2
@@ -38,7 +38,8 @@/* Doesn't use gcc to save the XMM registers, because there is no easy way to tell it to do a clts before the register saving. */ #define XMMS_SAVE \ - asm volatile ( \ + preempt_disable(); \ + asm volatile ( \ "movq %%cr0,%0 ;\n\t" \ "clts ;\n\t" \ "movups %%xmm0,(%1) ;\n\t" \
@@ -59,7 +60,8 @@ "movq %0,%%cr0 ;\n\t" \ : \ : "r" (cr0), "r" (xmm_save) \ - : "memory") + : "memory") \ +preempt_enable ();#define OFFS (x) "16*("#x")"#define PF_OFFS (x) "256+16*("#x")"
Hi,
Quoted 22 lines · Robert Love
include/asm-i386/xor.h +5
-2@@ -543,6 +545,7 @@#define XMMS_SAVE \ + preempt_disable(); \ __asm__ __volatile__ ( \ "movl %%cr0,%0 ;\n\t" \ "clts ;\n\t" \@@ -564,7 +567,8 @@ "movl %0,%%cr0 ;\n\t" \ : \ : "r" (cr0), "r" (xmm_save) \ - : "memory") + : "memory") \ +preempt_enable ();@@ -38,7 +38,8 @@#define XMMS_SAVE \ - asm volatile ( \ + preempt_disable(); \ + asm volatile ( \ "movq %%cr0,%0 ;\n\t" \ "clts ;\n\t" \ "movups %%xmm0,(%1) ;\n\t" \
include/asm-i386/xor.h line 64
Quoted 6 lines
include/asm-i386/xor.h +2
-1@@ -59,7 +60,8 @@ "movq %0,%%cr0 ;\n\t" \ : \ : "r" (cr0), "r" (xmm_save) \ - : "memory") + : "memory") \ +preempt_enable ();
These will suck when on if, I guess... Anyway, will this compile at all? There seems no semicolon after the asm volatile ()
Thunder
Signature · 4 lines
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .- --/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..- .- -/---/--/---/.-./.-./---/.--/.-.-.- --./.-/-.../.-./.././.-../.-.-.-
Quoted 1 line · Thunder from the hill
These will suck when on if, I guess...
hm?
Quoted 2 lines
Anyway, will this compile at all? There seems no semicolon after the asm volatile ()
Ah yes, curses. Thanks.
Robert Love
include/asm-i386/xor.h +5 -1
@@ -20,6 +20,7 @@#define FPU_SAVE \ do { \ + preempt_disable(); \ if (!test_thread_flag(TIF_USEDFPU)) \ __asm__ __volatile__ (" clts;\n"); \ __asm__ __volatile__ ("fsave %0; fwait": "=m"(fpu_save[0])); \
@@ -30,6 +31,7 @@ __asm__ __volatile__ ("frstor %0" : :"m" (fpu_save[0 ])); \if (!test_thread_flag (TIF_USEDFPU )) \stts (); \ +preempt_enable (); \ } while (0)#define LD (x,y) " movq 8*("#x")(%1), %%mm"#y" ;\n"
@@ -543,6 +545,7 @@*/ #define XMMS_SAVE \ + preempt_disable(); \ __asm__ __volatile__ ( \ "movl %%cr0,%0 ;\n\t" \ "clts ;\n\t" \
@@ -564,7 +567,8 @@ "movl %0,%%cr0 ;\n\t" \ : \ : "r" (cr0), "r" (xmm_save) \ - : "memory") + : "memory"); \ + preempt_enable()#define ALIGN16 __attribute__((aligned(16)))
include/asm-x86_64/xor.h +4 -2
@@ -38,7 +38,8 @@/* Doesn't use gcc to save the XMM registers, because there is no easy way to tell it to do a clts before the register saving. */ #define XMMS_SAVE \ - asm volatile ( \ + preempt_disable(); \ + asm volatile ( \ "movq %%cr0,%0 ;\n\t" \ "clts ;\n\t" \ "movups %%xmm0,(%1) ;\n\t" \
@@ -59,7 +60,8 @@ "movq %0,%%cr0 ;\n\t" \ : \ : "r" (cr0), "r" (xmm_save) \ - : "memory") + : "memory"); \ + preempt_enable()#define OFFS (x) "16*("#x")"#define PF_OFFS (x) "256+16*("#x")"
Quoted 10 lines · Robert Love
Quoted 1 line · Thunder from the hill
These will suck when on if, I guess...
hm?
Quoted 2 lines
Anyway, will this compile at all? There seems no semicolon after the asm volatile ()
Ah yes, curses. Thanks.
Also, kindly place "do { ... } while(0)" around XMMS_SAVE, if only for symmetry.
Signature · 4 lines
Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Hi,
Quoted 2 lines · Horst von Brand
Also, kindly place "do { ... } while(0)" around
XMMS_SAVE, if only for symmetry.
We've already had that talked about. Will appear in one of the following patches.
Thunder
Signature · 4 lines
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .- --/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..- .- -/---/--/---/.-./.-./---/.--/.-.-.- --./.-/-.../.-./.././.-../.-.-.-
How can a process easily get a list of all it's open fd?
In reply to Horst von BrandI need to close all the none std[in|out|err] open fd's.
I've been told to do it like so:
{
int i;
for (i=3; i<OPEN_MAX; i++)
close(i);
}This is very slow, plus I have discovered that I can have open fd's with values greater than OPEN_MAX.
I thought about getting the max fd from rlimit, but that doesn't work either. Say I have a rlimit of 1024 open fd's, and I open numbers 3 thru 1023, then I close 3 thru 1022, then I set the rlimit down to
16. rlimit then returns 16, but the largest open fd is still 1023.
So that doesn't work.
And I still have the problem that looping between 3 and whatever I pick as the top and calling close on each in turn is very slow.
So what's the "right way" to do it?
I would *love* for there to be an ioctl or some syscall that I could pass a pointer to an int and a pointer to an int array, and it would come back telling me how many open fd's I've got, and fill in the array with those fd's.
Signature · 3 lines
Mark Atwood | Well done is better than well said. [email protected] | http://www.pobox.com/~mra
Re: How can a process easily get a list of all it's open fd?
tricky. You can use /proc/<getpid>/fd, and close all handles listed here, but this has some caveats:
it's _very_ slow if you have many open files. it's not portable.
it's not safe if you have a thread/signal handler running.
i never heard of a right way to do this.
-alex
int close_all_fd()
{
char fdpath[PATH_MAX];
DIR * dp;
struct dirent * de;
int fd;
sprintf(fdpath, "/proc/%d/fd", getpid());
dp = opendir(fdpath);
if ( !dp )
return -errno;
while ( (de = readdir(dp)) )
{
if ( !strcmp(de->d_name, ".") || !strcmp(de->d_name, "..") )
continue;
fd = strtol(de->d_name, 0, 10);
if ( fd == dirfd(dp) || fd == 0 || fd == 1 || fd == 2 )
continue;
if ( close(fd) < 0 )
fprintf(stderr, "%s: %s\n", de->d_name, strerror(errno));
}
closedir(dp);
return 0;
}Quoted 36 lines · Mark Atwood
I need to close all the none std[in|out|err] open fd's.
I've been told to do it like so:
{ int i; for (i=3; i<OPEN_MAX; i++) close(i); }This is very slow, plus I have discovered that I can have open fd's with values greater than
OPEN_MAX.I thought about getting the max fd from rlimit, but that doesn't work either. Say I have a rlimit of 1024 open fd's, and I open numbers 3 thru 1023, then I close 3 thru 1022, then I set the rlimit down to
16. rlimit then returns 16, but the largest open fd is still 1023.So that doesn't work.
And I still have the problem that looping between 3 and whatever I pick as the top and calling close on each in turn is very slow.
So what's the "right way" to do it?
I would *love* for there to be an ioctl or some syscall that I could pass a pointer to an int and a pointer to an int array, and it would come back telling me how many open fd's I've got, and fill in the array with those fd's.
Signature · 4 lines
Mark Atwood | Well done is better than well said. [email protected] | http://www.pobox.com/~mra -
Re: How can a process easily get a list of all it's open fd?
Quoted 11 lines · Alex Riesen
tricky. You can use /proc/<getpid>/fd, and close all handles listed here, but this has some caveats:
it's _very_ slow if you have many open files. it's not portable.
it's not safe if you have a thread/signal handler running.i never heard of a right way to do this.
-alex
int
close_all_fd()
Quoted 8 lines
{
char fdpath[PATH_MAX];
DIR * dp;
struct dirent * de;
int fd;
sprintf(fdpath, "/proc/%d/fd", getpid());
dp = opendir(fdpath);This can just be:
dp = opendir("/proc/self/fd/");then you don't need fdpath.
You can use sigprocmask() if you're worried about signals coming in during this operation.
miket
Re: How can a process easily get a list of all it's open fd?
Quoted 18 lines · Mike Touloumtzis
Quoted 17 lines · Alex Riesen
tricky. You can use /proc/<getpid>/fd, and close all handles listed here, but this has some caveats:
it's _very_ slow if you have many open files. it's not portable.
it's not safe if you have a thread/signal handler running.i never heard of a right way to do this.
-alex
int
close_all_fd(){ char fdpath[PATH_MAX]; DIR * dp; struct dirent * de; int fd;
Quoted 8 lines
Quoted 2 lines
sprintf(fdpath, "/proc/%d/fd", getpid()); dp = opendir(fdpath);This can just be:
dp = opendir("/proc/self/fd/");then you don't need fdpath.
Oh, indeed.
Quoted 2 lines
You can use
sigprocmask()if you're worried about signals coming in during this operation.
I, personally, strongly dislike this option.
You never know which signals are to be blocked.
Besides, it's still not portable and not safe agains threads.
-alex
Re: How can a process easily get a list of all it's open fd?
In reply to Mark AtwoodHi Mark :)
Quoted 1 line
So what's the "right way" to do it?
AFAIK, the for loop, with getdtablesize() instead of 'OPEN_MAX'.
I do it that way, but I don't really know if it is the 'right way'(tm).
Quoted 4 lines
I would *love* for there to be an ioctl or some syscall that I could pass a pointer to an int and a pointer to an int array, and it would come back telling me how many open fd's I've got, and fill in the array with those fd's.
The array should be allocated by the kernel, or the syscall won't
work as expected ;) If you have 2000 fd open and the array whose address you pass to the ioctl has an smaller size... Anyway, you can call the ioctl a few times ;)
Your proposal seems reasonable (unless there is any other way of
doing this portably), but portability will be an issue...
Raúl
Quoted 11 lines · Robert Love
Linus,
The raid5 xor checksums use MMX/SSE state and are not preempt-safe.
Attached patch disables preemption in
FPU_SAVEandXMMS_SAVEand restores it inFPU_RESTOREandXMMS_RESTORE- preventing preemption while in fp mode.Please, apply.
Robert Love
Use kernel_fpu_begin() and kernel_fpu_end() instead of reinventing the wheel. The current code is broken wrt SSE as well.
--
Brian Gerst