Avoid truncating to 'long' in ALIGN() macro
PATCH 1/1 Avoid truncating to 'long' in ALIGN() macro
Commit 4c8bd7ee ("Do not truncate to 'int' in ALIGN() macro.") was merged to fix the case of code like the following:
unsigned long addr; unsigned int alignment; addr = ALIGN(addr, alignment);
The original ALIGN macro calculated a mask as ~(alignment - 1), and when alignment is just an int, this creates an int mask. If alignment is also unsigned, then this mask will not be sign extended when promoted to a long, which leads to the code above chopping off the top half of addr when long is 64 bits.
However, the changed ALIGN macro, which computes the mask as ~(alignment - 1UL) actually breaks code like the following when long is 32 bits:
u64 addr;
int alignment;
addr = ALIGN(addr, alignment);The reason this breaks is pretty much the same as the original bug that the change was supposed to fix: ~(alignment - 1UL) creates a mask that is an unsigned long, which is unsigned and therefore not sign extended when promoted to u64 (if long is 32 bits).
I think the right fix is to use 1L instead of 1UL, to avoid the possibility of turning the mask unsigned by accident. This will still fix the original problem, without breaking code that uses ALIGN() the second way.
This second construct is actually used in the amso1100 driver, so that driver does not work on 32-bit architectures right now. Unfortunately almost everyone using it runs 64-bit kernels, so this regression was not noticed until now.
---
If it's too late to merge this for 2.6.19, I can work around this in the amso1100 driver by doing ALIGN(addr, (u64) alignment) instead. Let me know if I should send that patch.
However I think this should go in for 2.6.20 at least, since I see only danger with no benefit from having the constant 1 be unsigned.
include/linux/kernel.h +1 -1
@@ -31,7 +31,7 @@ #define ULLONG_MAX (~0ULL)#define STACK_MAGIC 0xdeadbeef#define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0])) -#define ALIGN (x,a) (((x)+(a)-1UL)&~((a)-1UL)) +#define ALIGN (x,a) (((x)+(a)-1L)&~((a)-1L))#define FIELD_SIZEOF (t, f) (sizeof(((t*)0)->f))#define DIV_ROUND_UP (n,d) (((n) + (d) - 1) / (d))#define roundup (x, y) ((((x) + ((y) - 1)) / (y)) * (y))
From: Roland Dreier <[email protected]>
Date: Fri, 24 Nov 2006 21:40:14 -0800
include/linux/kernel.h #define ULLONG_MAX (~0ULL) · line 34
Quoted 10 lines
include/linux/kernel.h +1
-1@@ -31,7 +31,7 @@ #define ULLONG_MAX (~0ULL)#define STACK_MAGIC 0xdeadbeef#define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0])) -#define ALIGN (x,a) (((x)+(a)-1UL)&~((a)-1UL)) +#define ALIGN (x,a) (((x)+(a)-1L)&~((a)-1L))
Perhaps a better way to fix this is to usetypeof() like other similar macros do.
Quoted 2 lines
Perhaps a better way to fix this is to use
typeof()like other similar macros do.
I tried doing
#define ALIGN(x,a) \
({ \
typeof(x) _a = (a); \
((x) + _a - 1) & ~(_a - 1); \
})but that won't compile because of <net/neighbour.h>:
unsigned char ha[ALIGN(MAX_ADDR_LEN, sizeof(unsigned long))];
gcc says:
/scratch/Ksrc/linux-merge/include/net/neighbour.h:104: error: braced-group within expression allowed only inside a function
I guess that could be fixed by changing that declaration but now this is starting to feel like early 2.6.20 material.
- R.
From: Roland Dreier <[email protected]>
Date: Sat, 25 Nov 2006 14:56:22 -0800
Quoted 5 lines
Quoted 2 lines
Perhaps a better way to fix this is to use
typeof()like other similar macros do.I tried doing
Quoted 7 lines
#define ALIGN(x,a) \ ({ \ typeof(x) _a = (a); \ ((x) + _a - 1) & ~(_a - 1); \ })but that won't compile because of <
net/neighbour.h>:
You would need to also cast the constants with typeof() to.
But yes, given the array sizing case in the neighbour code, perhaps we can use your original patch for now. Feel free to push that to Linus.
Quoted 1 line
You would need to also cast the constants with
typeof()to.
Why? I'm not much of a C language lawyer, but I would have thought that in something like
(x) + _a - 1
the "1" will be promoted to the type of the rest of the expression without any explicit cast. I tested the unsigned long/unsigned int and u64/int cases of ALIGN(), and my macro with typeof() works for both of those cases at least.
Quoted 3 lines
But yes, given the array sizing case in the neighbour code, perhaps we can use your original patch for now. Feel free to push that to Linus.
akpm is CC'ed on this thread. Andrew, are you going to pick this up?
Thanks,
Roland
Quoted 5 lines · On Sat, 25 Nov 2006 15:09:38 -0800 Roland Dreier <[email protected]> wrote:
Quoted 3 lines
But yes, given the array sizing case in the neighbour code, perhaps we can use your original patch for now. Feel free to push that to Linus.
akpm is CC'ed on this thread. Andrew, are you going to pick this up?
Changes this late in the piece rather hurt.
Your proposed change is still wrong for long longs, isn't it?
Quoted 14 lines · David Miller
From: Roland Dreier <[email protected]>
Date: Sat, 25 Nov 2006 14:56:22 -0800Quoted 11 lines
Quoted 2 lines
Perhaps a better way to fix this is to use
typeof()like other similar macros do.I tried doing
#define ALIGN(x,a) \ ({ \ typeof(x) _a = (a); \ ((x) + _a - 1) & ~(_a - 1); \ })
Quoted 3 lines
Quoted 1 line
but that won't compile because of <
net/neighbour.h>:You would need to also cast the constants with
typeof()to.
Oh, for fsck sake...
(typeof(x))((x + a - 1) & ~(a - 1ULL))
Quoted 1 line
(typeof(x))((x + a - 1) & ~(a - 1ULL))
Yes I was being stupid thinking I needed a temporary variable to use typeof. But what does the cast to typeof(x) accomplish if we write things the way you suggested above? It seems that the right things is really just
(((x) + (a) - 1) & ~((typeof(x)) (a) - 1))
- R.
Quoted 8 lines · Roland Dreier
Quoted 1 line
(typeof(x))((x + a - 1) & ~(a - 1ULL))Yes I was being stupid thinking I needed a temporary variable to use typeof. But what does the cast to typeof(x) accomplish if we write things the way you suggested above? It seems that the right things is really just
(((x) + (a) - 1) & ~((typeof(x)) (a) - 1))
Better have type of result independent of that of a.
Quoted 1 line
Changes this late in the piece rather hurt.
Fair enough. This was a really close call to me but let's leave it for 2.6.19. I'll ask Linus to merge the patch below to fix amso1100. I'm a little worried that other such uses might be lurking in the tree but I guess no one has complained...
Quoted 1 line
Your proposed change is still wrong for long longs, isn't it?
Yes, it would fail for aligning long long with an unsigned long alignment on 32-bits. But that's broken in the current tree too. I'll post a patch that should work for everything -- how about if you queue that for 2.6.20-early?
- R.
drivers/infiniband/hw/amso1100/c2_provider.c +2 -2
@@ -368,7 +368,7 @@ static struct ib_mr *c2_reg_phys_mr(stru total_len += buffer_list[i].size; pbl_depth +=ALIGN (buffer_list[i].size, - (1 << page_shift)) >> page_shift; + (1ull << page_shift)) >> page_shift; } page_list =vmalloc (sizeof (u64) * pbl_depth);
@@ -383,7 +383,7 @@ static struct ib_mr *c2_reg_phys_mr(struint naddrs; naddrs =ALIGN (buffer_list[i].size, - (1 << page_shift)) >> page_shift; + (1ull << page_shift)) >> page_shift;for (k =0 ; k < naddrs; k++) page_list[j++] = (buffer_list[i].addr + (k << page_shift));
Commit 4c8bd7ee ("Do not truncate to 'int' in ALIGN() macro.") was merged to fix the case of code like the following:
unsigned long addr; unsigned int alignment; addr = ALIGN(addr, alignment);
The original ALIGN macro calculated a mask as ~(alignment - 1), and when alignment is just an int, this creates an int mask. If alignment is also unsigned, then this mask will not be sign extended when promoted to a long, which leads to the code above chopping off the top half of addr when long is 64 bits.
However, the changed ALIGN macro, which computes the mask as ~(alignment - 1UL) actually breaks code like the following when long is 32 bits:
u64 addr;
int alignment;
addr = ALIGN(addr, alignment);The reason this breaks is pretty much the same as the original bug that the change was supposed to fix: ~(alignment - 1UL) creates a mask that is an unsigned long, which is not sign extended when promoted to u64 (if long is 32 bits).
As suggested by Dave Miller and Al Viro, I fixed this by having the ALIGN macro make sure the alignment is promoted to the same type as the value being aligned before doing the negation.
This second construct is actually used in the amso1100 driver, so that driver does not work on 32-bit architectures right now. Unfortunately almost everyone using it runs 64-bit kernels, so this regression was not noticed until now.
---
Patch updated as suggested by Dave Miller and Al Viro...
can we merge this for 2.6.20?
include/linux/kernel.h +1 -1
@@ -31,7 +31,7 @@ #define ULLONG_MAX (~0ULL)#define STACK_MAGIC 0xdeadbeef#define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0])) -#define ALIGN (x,a) (((x)+(a)-1UL)&~((a)-1UL)) +#define ALIGN (x,a) ((typeof(x)) (((x) + (a) - 1) & ~((typeof(x)) (a) - 1)))#define FIELD_SIZEOF (t, f) (sizeof(((t*)0)->f))#define DIV_ROUND_UP (n,d) (((n) + (d) - 1) / (d))#define roundup (x, y) ((((x) + ((y) - 1)) / (y)) * (y))
Quoted 47 lines · On Sun, 26 Nov 2006 11:10:23 -0800 Roland Dreier <[email protected]> wrote:
Commit
4c8bd7ee("Do not truncate to 'int' inALIGN()macro.") was merged to fix the case of code like the following:unsigned long addr; unsigned int alignment; addr = ALIGN(addr, alignment);The original ALIGN macro calculated a mask as ~(alignment - 1), and when alignment is just an int, this creates an int mask. If alignment is also unsigned, then this mask will not be sign extended when promoted to a long, which leads to the code above chopping off the top half of addr when long is 64 bits.
However, the changed ALIGN macro, which computes the mask as ~(alignment - 1UL) actually breaks code like the following when long is 32 bits:
u64 addr; int alignment; addr = ALIGN(addr, alignment);The reason this breaks is pretty much the same as the original bug that the change was supposed to fix: ~(alignment - 1UL) creates a mask that is an unsigned long, which is not sign extended when promoted to u64 (if long is 32 bits).
As suggested by Dave Miller and Al Viro, I fixed this by having the ALIGN macro make sure the alignment is promoted to the same type as the value being aligned before doing the negation.
This second construct is actually used in the amso1100 driver, so that driver does not work on 32-bit architectures right now. Unfortunately almost everyone using it runs 64-bit kernels, so this regression was not noticed until now.
Signed-off-by Roland Dreier <[email protected]>---
Patch updated as suggested by Dave Miller and Al Viro...can we merge this for 2.6.20?
include/linux/kernel.h +0
-0
include/linux/kernel.h #define ULLONG_MAX (~0ULL) · line 37
Quoted 8 lines
include/linux/kernel.h +1
-1@@ -31,7 +31,7 @@ #define ULLONG_MAX (~0ULL)#define STACK_MAGIC 0xdeadbeef#define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0])) -#define ALIGN (x,a) (((x)+(a)-1UL)&~((a)-1UL)) +#define ALIGN (x,a) ((typeof(x)) (((x) + (a) - 1) & ~((typeof(x)) (a) - 1)))#define FIELD_SIZEOF (t, f) (sizeof(((t*)0)->f))#define DIV_ROUND_UP (n,d) (((n) + (d) - 1) / (d))#define roundup (x, y) ((((x) + ((y) - 1)) / (y)) * (y))
I'd be inclined to merge it for 2.6.19. Is everyone OK with it?
Quoted 1 line
I'd be inclined to merge it for 2.6.19. Is everyone OK with it?
I'm OK with that -- your previous email made me thing you didn't want to, but I think the risks are rather low, and there's a least a chance that we'll fix some obscure regression.
Quoted 2 lines · Andrew Morton
I'd be inclined to merge it for 2.6.19. Is everyone OK with it?
I'd _much_ rather make it more readable while at it. Something like this instead, which has just _one_ "typeof" cast, and at least to me makes it a lot more obvious what is going on (aka "to align something, use a mask that is of the same type as the thing to be aligned..").
Maybe it's just me, but I prefer to separate out the actual "action" from the "type fluff" around it.
Linus
---
include/linux/kernel.h +3 -1
@@ -30,8 +30,10 @@ extern const char linux_banner[];#define STACK_MAGIC 0xdeadbeef +#define ALIGN (x,a) __ALIGN_MASK(x,(typeof(x))(a)-1) +#define __ALIGN_MASK (x,mask) (((x)+(mask))&~(mask)) +#define ARRAY_SIZE (x) (sizeof(x) / sizeof((x)[0])) -#define ALIGN (x,a) (((x)+(a)-1UL)&~((a)-1UL))#define FIELD_SIZEOF (t, f) (sizeof(((t*)0)->f))#define DIV_ROUND_UP (n,d) (((n) + (d) - 1) / (d))#define roundup (x, y) ((((x) + ((y) - 1)) / (y)) * (y))
include/linux/kernel.h extern const char linux_banner[]; · line 34
Quoted 2 lines
include/linux/kernel.h +2
-0+#define ALIGN (x,a) __ALIGN_MASK(x,(typeof(x))(a)-1) +#define __ALIGN_MASK (x,mask) (((x)+(mask))&~(mask))
Fine by me, but it loses the extra (typeof(x)) cast that Al wanted to make sure that the result of ALIGN() is not wider than x.
- R.
Quoted 6 lines · Roland Dreier
include/linux/kernel.h extern const char linux_banner[]; · line 34
Quoted 3 lines
include/linux/kernel.h +2
-0+#define ALIGN (x,a) __ALIGN_MASK(x,(typeof(x))(a)-1) +#define __ALIGN_MASK (x,mask) (((x)+(mask))&~(mask))Fine by me, but it loses the extra (typeof(x)) cast that Al wanted to make sure that the result of
ALIGN()is not wider than x.
Not a big deal, is it?
#define ALIGN(x,a) (typeof(x))__ALIGN_MASK(x,(typeof(x))(a)-1)
#define __ALIGN_MASK(x,mask) (((x)+(mask))&~(mask))
Jörn
Signature · 4 lines
Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats. -- Howard Aiken quoted by Ken Iverson quoted by Jim Horning quoted by Raph Levien, 1979
Quoted 6 lines · Roland Dreier
include/linux/kernel.h extern const char linux_banner[]; · line 34
Quoted 3 lines
include/linux/kernel.h +2
-0+#define ALIGN (x,a) __ALIGN_MASK(x,(typeof(x))(a)-1) +#define __ALIGN_MASK (x,mask) (((x)+(mask))&~(mask))Fine by me, but it loses the extra (typeof(x)) cast that Al wanted to make sure that the result of
ALIGN()is not wider than x.
Well, since "mask" is now made to be of the same type as "x", every sub-expression actually has the same type, modulo the normal C behaviour of "expand to at least "int".
So arguably, the result is _more_ like a normal C operation this way. Type-wise, the "ALIGN()" macro acts like any other C operation (ie if you feed it an "unsigned char", the end result is an "int" due to the normal C type widening that happens for all C operations).
But I don't care horribly much. Al may have some other reasons to _not_ want the normal C type expansion to happen (ie maybe he does something unnatural with sparse ;)
Linus
Quoted 8 lines · Linus Torvalds
So arguably, the result is _more_ like a normal C operation this way. Type-wise, the "
ALIGN()" macro acts like any other C operation (ie if you feed it an "unsigned char", the end result is an "int" due to the normal C type widening that happens for all C operations).But I don't care horribly much. Al may have some other reasons to _not_ want the normal C type expansion to happen (ie maybe he does something unnatural with sparse ;)
Type expansion will happen as soon as you do any arithmetics (or passing as argument) anyway.
It's actually more of "typeof() has interesting interactions with other gccisms" kind of thing and general dislike of using that beast more than absolutely necessary.
Not the #1 on my list of the worst gccisms we are using (that would be ({...}) with its insane semantics and interesting ways to get gcc puke its guts out), but still pretty high there...
ObFun: what's the type of ({struct {int x,y;} a = {1,2}; a;}) and how comes that we can say
({struct {int x,y;} a = {1,2}; a;}).yand get gcc eat it up and evaluate that to 2? Note that we are doing a very obvious violation of scope rules - WTF _is_ .y in scope where we have no visible declaration of any structure with field that would have such name?
IOW, gcc allows type to leak out of scope it's been defined in (and typeof adds even more fun to the picture). It not only goes against a _lot_ in C, it's actually not thought through by gcc folks. Just try to mix that with variable-length arrays and watch it blow up in interesting ways...