mm/mempolicy: fix copy_nodes_to_user() for odd inputs
Subsystems (3)
PATCH 1/1 mm/mempolicy: fix copy_nodes_to_user() for odd inputs
From: Arnd Bergmann <[email protected]>
The ALIGN(maxnode-1, 64) check has an off-by-one error that leads to rounding down too much, and ends up not copying the last word when userspace asks for one bit over a multiple of 64.
For compat syscalls, the effect is worse, as the actual calculation for the size does include the extra word, but the check for the input size does not take it into account and can leak a word of stack data.
In addition, the size calculation rounds up to multiples of 64 bits instead of multiples of word size as documented in the man page and expected by numactl.
Fortunately numactl always passes multiples of 64 here, so none of this affects userspace we care about. Finally, the 'nbytes' calculation for compat mode makes no sense since that is the size of the kernel's internal representation, not the userspace side.
Fix the 'copy' calculation to address all of the above. Note that the off-by-one error has been in there since the beginning, but commit e130242dc351 ("mm: simplify compat numa syscalls") is where we starting copying the wrong data.
---
Please double-check the calculation and retest this with the original reproducer. The off-by-one error does explain all of this, I'm still worried about getting this version wrong again as everyone including myself seems to have missed this in the past 25 years and there may still be additional subtle issues around it. Apparently we had many instances of the same off-by-one in this file but all (most?) others got fixed inadvertently by other changes in the past.
--- mm/mempolicy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/mempolicy.c +2 -2
@@ -1749,12 +1749,12 @@ static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,static int copy_nodes_to_user (unsigned long __user *mask,unsigned long maxnode,nodemask_t *nodes) { -unsigned long copy =ALIGN (maxnode-1 ,64 ) /8 ; +unsigned long copy =ALIGN (maxnode,BITS_PER_LONG ) /8 ;unsigned int nbytes =BITS_TO_LONGS (nr_node_ids) *sizeof (long );bool compat =in_compat_syscall ();if (compat) - nbytes =BITS_TO_COMPAT_LONGS (nr_node_ids) *sizeof (compat_long_t); + copy =ALIGN (maxnode,BITS_PER_COMPAT_LONG ) /8 ;if (copy > nbytes) {if (copy >PAGE_SIZE )
Signature · 1 line
2.53.0
Quoted 5 lines · Arnd Bergmann
From: Arnd Bergmann <[email protected]>
The
ALIGN(maxnode-1, 64)check has an off-by-one error that leads to rounding down too much, and ends up not copying the last word when userspace asks for one bit over a multiple of 64.
From what I can tell, this isn't an off-by-one, but just the (possibly weird) convention in mempolicy.c. E.g. in get_nodes():
static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,
unsigned long maxnode)
{
--maxnode;
< etc. >
}Quoted 10 lines
For compat syscalls, the effect is worse, as the actual calculation for the size does include the extra word, but the check for the input size does not take it into account and can leak a word of stack data.
In addition, the size calculation rounds up to multiples of 64 bits instead of multiples of word size as documented in the man page and expected by numactl.
Fortunately numactl always passes multiples of 64 here, so none of
(Claude) went looking at numactl source (e.g. via libnuma) and it seems it passes size + 1 matching the convention?
Quoted 32 lines
this affects userspace we care about. Finally, the 'nbytes' calculation for compat mode makes no sense since that is the size of the kernel's internal representation, not the userspace side.
Fix the 'copy' calculation to address all of the above. Note that the off-by-one error has been in there since the beginning, but commit
e130242dc351("mm: simplify compat numa syscalls") is where we starting copying the wrong data.Reported-by larips <[email protected]>Reported-by Gregory Price (Meta) <[email protected]>Fixes e130242dc351 ("mm: simplify compat numa syscalls")Signed-off-by Arnd Bergmann <[email protected]>---
Please double-check the calculation and retest this with the original reproducer. The off-by-one error does explain all of this, I'm still worried about getting this version wrong again as everyone including myself seems to have missed this in the past 25 years and there may still be additional subtle issues around it. Apparently we had many instances of the same off-by-one in this file but all (most?) others got fixed inadvertently by other changes in the past.--- mm/mempolicy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)mm/mempolicy.c +0
-0
mm/mempolicy.c static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask, · line 1752
Quoted 5 lines
mm/mempolicy.c +1
-1@@ -1749,12 +1749,12 @@ static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,static int copy_nodes_to_user (unsigned long __user *mask,unsigned long maxnode,nodemask_t *nodes) { -unsigned long copy =ALIGN (maxnode-1 ,64 ) /8 ; +unsigned long copy =ALIGN (maxnode,BITS_PER_LONG ) /8 ;
This looks like it'll be broken given the convention of passing bits + 1, e.g. ALIGN(1025, 64) / 8 != ALIGN(1024, 64) / 8.
Then later that could case a buffer overflow potentially?
mm/mempolicy.c static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask, · line 1758
Quoted 7 lines
mm/mempolicy.c +1
-1unsigned int nbytes =BITS_TO_LONGS (nr_node_ids) *sizeof (long );bool compat =in_compat_syscall ();if (compat) - nbytes =BITS_TO_COMPAT_LONGS (nr_node_ids) *sizeof (compat_long_t); + copy =ALIGN (maxnode,BITS_PER_COMPAT_LONG ) /8 ;
Claude suggests that this is potentially broken too for similar reasons.
Quoted 5 lines
mm/mempolicy.c +0
-1if (copy > nbytes) {if (copy >PAGE_SIZE ) --2.53.0
Overall it seems the issue is that e130242dc351 (understandably!) passes maxnode rather than maxnode-1.
So a really small fixup would be something like:
mm/mempolicy.c +2 -2
@@ -1733,12 +1733,12 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode,if (clear_user ((char __user *)mask + nbytes, copy - nbytes))return -EFAULT ; copy = nbytes; - maxnode = nr_node_ids; + maxnode = nr_node_ids +1 ; }if (compat)return compat_put_bitmap ((compat_ulong_t __user *)mask, -nodes_addr (*nodes), maxnode); +nodes_addr (*nodes), maxnode -1 );return copy_to_user (mask,nodes_addr (*nodes), copy) ? -EFAULT :0 ; }
Maybe doing that's the easiest route for this?
--
Cheers, Lorenzo
Quoted 6 lines · Lorenzo Stoakes (ARM)
Quoted 5 lines · Arnd Bergmann
From: Arnd Bergmann <[email protected]>
For compat syscalls, the effect is worse, as the actual calculation for the size does include the extra word, but the check for the input size does not take it into account and can leak a word of stack data.
Quoted 8 lines
Quoted 5 lines
In addition, the size calculation rounds up to multiples of 64 bits instead of multiples of word size as documented in the man page and expected by numactl.
Fortunately numactl always passes multiples of 64 here, so none of
(Claude) went looking at numactl source (e.g. via libnuma) and it seems it passes size + 1 matching the convention?
Ok, I see it now as well. The man page again says "maxnode specifies the number of node IDs that can be stored into nodemask——that is, the maximum node ID plus one.", which I'd naïvely interpret as meaning that a system with 64 nodes would have a "maximum node ID" of 63 and require passing one more (64), but clearly the implementation on both sides expects a 65 here.
Quoted 14 lines
mm/mempolicy.c static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask, · line 1752
Quoted 10 lines
mm/mempolicy.c +1
-1@@ -1749,12 +1749,12 @@ static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,static int copy_nodes_to_user (unsigned long __user *mask,unsigned long maxnode,nodemask_t *nodes) { -unsigned long copy =ALIGN (maxnode-1 ,64 ) /8 ; +unsigned long copy =ALIGN (maxnode,BITS_PER_LONG ) /8 ;This looks like it'll be broken given the convention of passing bits + 1, e.g.
ALIGN(1025, 64)/ 8 !=ALIGN(1024, 64)/ 8.
Quoted 1 line
Then later that could case a buffer overflow potentially?
Right, this would then overflow the user buffer.
Quoted 26 lines
Overall it seems the issue is that
e130242dc351(understandably!) passes maxnode rather than maxnode-1.So a really small fixup would be something like:
mm/mempolicy.c +2
-2@@ -1733,12 +1733,12 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode, if (clear_user((char __user *)mask + nbytes, copy - nbytes)) return -EFAULT; copy = nbytes; - maxnode = nr_node_ids; + maxnode = nr_node_ids +1 ; }if (compat)return compat_put_bitmap ((compat_ulong_t __user *)mask, -nodes_addr (*nodes), maxnode); +nodes_addr (*nodes), maxnode -1 );return copy_to_user (mask,nodes_addr (*nodes), copy) ? -EFAULT :0 ;}
Quoted 1 line
Maybe doing that's the easiest route for this?
I would still try to fix the other mistakes (aligning to 64 instead of BITS_PER_LONG/BITS_PER_COMPAT_LONG and the incorrect nbytes). Maybe it's clearer to just pass the adjusted maxnode into the function directly and keep that extra '-1' in one place?
Arnd
mm/mempolicy.c +3 -3
@@ -1749,12 +1749,12 @@ static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,static int copy_nodes_to_user (unsigned long __user *mask,unsigned long maxnode,nodemask_t *nodes) { -unsigned long copy =ALIGN (maxnode-1 ,64 ) /8 ; +unsigned long copy =ALIGN (maxnode,BITS_PER_LONG ) /8 ;unsigned int nbytes =BITS_TO_LONGS (nr_node_ids) *sizeof (long );bool compat =in_compat_syscall ();if (compat) - nbytes =BITS_TO_COMPAT_LONGS (nr_node_ids) *sizeof (compat_long_t); + copy =ALIGN (maxnode,BITS_PER_COMPAT_LONG ) /8 ;if (copy > nbytes) {if (copy >PAGE_SIZE )
@@ -2030,7 +2030,7 @@ static int kernel_get_mempolicy(int __user *policy,return -EFAULT ;if (nmask) - err =copy_nodes_to_user (nmask, maxnode, &nodes); + err =copy_nodes_to_user (nmask, maxnode-1 , &nodes);return err; }
Quoted 16 lines · Arnd Bergmann
Quoted 6 lines · Lorenzo Stoakes (ARM)
Quoted 5 lines · Arnd Bergmann
From: Arnd Bergmann <[email protected]>
For compat syscalls, the effect is worse, as the actual calculation for the size does include the extra word, but the check for the input size does not take it into account and can leak a word of stack data.Quoted 8 lines
Quoted 5 lines
In addition, the size calculation rounds up to multiples of 64 bits instead of multiples of word size as documented in the man page and expected by numactl.
Fortunately numactl always passes multiples of 64 here, so none of
(Claude) went looking at numactl source (e.g. via libnuma) and it seems it passes size + 1 matching the convention?
Quoted 6 lines
Ok, I see it now as well. The man page again says "maxnode specifies the number of node IDs that can be stored into nodemask——that is, the maximum node ID plus one.", which I'd naïvely interpret as meaning that a system with 64 nodes would have a "maximum node ID" of 63 and require passing one more (64), but clearly the implementation on both sides expects a 65 here.
Yeah which is... special isn't it? :))
Quoted 45 lines
Quoted 15 lines
mm/mempolicy.c static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask, · line 1752
Quoted 11 lines
mm/mempolicy.c +1
-1@@ -1749,12 +1749,12 @@ static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,static int copy_nodes_to_user (unsigned long __user *mask,unsigned long maxnode,nodemask_t *nodes) { -unsigned long copy =ALIGN (maxnode-1 ,64 ) /8 ; +unsigned long copy =ALIGN (maxnode,BITS_PER_LONG ) /8 ;This looks like it'll be broken given the convention of passing bits + 1, e.g.
ALIGN(1025, 64)/ 8 !=ALIGN(1024, 64)/ 8.Quoted 1 line
Then later that could case a buffer overflow potentially?
Right, this would then overflow the user buffer.
Quoted 26 lines
Overall it seems the issue is that
e130242dc351(understandably!) passes maxnode rather than maxnode-1.So a really small fixup would be something like:
mm/mempolicy.c +2
-2@@ -1733,12 +1733,12 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode, if (clear_user((char __user *)mask + nbytes, copy - nbytes)) return -EFAULT; copy = nbytes; - maxnode = nr_node_ids; + maxnode = nr_node_ids +1 ; }if (compat)return compat_put_bitmap ((compat_ulong_t __user *)mask, -nodes_addr (*nodes), maxnode); +nodes_addr (*nodes), maxnode -1 );return copy_to_user (mask,nodes_addr (*nodes), copy) ? -EFAULT :0 ;}
Quoted 6 lines
Quoted 1 line
Maybe doing that's the easiest route for this?
I would still try to fix the other mistakes (aligning to 64 instead of BITS_PER_LONG/BITS_PER_COMPAT_LONG and the incorrect nbytes). Maybe it's clearer to just pass the adjusted maxnode into the function directly and keep that extra '-1' in one place?
Yeah it'd be nice to get away from that convention which is... very confusing :) Especially as 'max' implies to me an inclusive bound.
You could have a wrapper get_nodes() that calls __get_nodes() that has a 'lastnode' parameter for instance?
--
Cheers, Lorenzo
Gregory Price (Meta)
maintainerWhat about the following:
--- 1/2 mm/mempolicy: convert maxnode to a bit count in copy_nodes_to_user()
Converts maxnode to a bit count on entry to copy_nodes_to_user() and get_nodes(), fixing the compat KASAN stack over-read and stack leak. Fixes e130242dc351, Cc stable.
Subject: [PATCH 1/2] mm/mempolicy: convert maxnode to a bit count in
copy_nodes_to_user()
Whenever maxnode - 1 is a multiple of 32, compat_put_bitmap() reads one more compat_ulong_t from the kernel nodemask than the caller asked for, and writes it past the end of the caller's bitmap.
When the kernel nodemask ends there, that read goes past the on-stack nodemask_t in kernel_get_mempolicy(), and its low 32 bits are copied to userspace.
With CONFIG_NODES_SHIFT=6, 64 possible nodes and a 32-bit caller passing maxnode=65:
BUG: KASAN: stack-out-of-bounds in compat_put_bitmap+0x122/0x130 Read of size 8 at addr ffff8880150d7ec0 by task mpoltest/241 compat_put_bitmap+0x122/0x130 copy_nodes_to_user+0x14a/0x250 __ia32_sys_get_mempolicy+0x1b0/0x2b0
Convert maxnode to a bit count once, on entry, and use only that in the rest of copy_nodes_to_user(), so the compat path gets the same size as the native one. Do the same in get_nodes().
--- mm/mempolicy.c | 36 ++++++++++++++++++++++-------------- 1 file changed, 22 insertions(+), 14 deletions(-)
mm/mempolicy.c +22 -14
@@ -1673,15 +1673,22 @@ static int get_bitmap(unsigned long *mask, const unsigned long __user *nmask,return 0 ; } -/* Copy a node mask from user space. */ +/* +* Copy a node mask from user space. +* +* The nodemask syscalls take maxnode as the size of the user bitmap in bits +* plus one. get_nodes() and copy_nodes_to_user() take it as passed and +* convert it to the bitmap size, nbits, on entry. +*/ static int get_nodes (nodemask_t *nodes,const unsigned long __user *nmask,unsigned long maxnode) { - --maxnode; +unsigned long nbits = maxnode -1 ; +nodes_clear (*nodes); -if (maxnode ==0 || !nmask) +if (nbits ==0 || !nmask)return 0 ; -if (maxnode >PAGE_SIZE *BITS_PER_BYTE ) +if (nbits >PAGE_SIZE *BITS_PER_BYTE )return -EINVAL ;/*
@@ -1689,31 +1696,32 @@ static int get_nodes(nodemask_t *nodes, const unsigned long __user *nmask,* if the non supported part is all zero, one word at a time, * starting at the end. */ -while (maxnode >MAX_NUMNODES ) { - unsigned long bits = min_t(unsigned long, maxnode, BITS_PER_LONG); +while (nbits >MAX_NUMNODES ) { + unsigned long bits = min_t(unsigned long, nbits, BITS_PER_LONG);unsigned long t; -if (get_bitmap (&t, &nmask[(maxnode -1 ) /BITS_PER_LONG ], bits)) +if (get_bitmap (&t, &nmask[(nbits -1 ) /BITS_PER_LONG ], bits))return -EFAULT ; -if (maxnode - bits >=MAX_NUMNODES ) { - maxnode -= bits; +if (nbits - bits >=MAX_NUMNODES ) { + nbits -= bits; }else { - maxnode =MAX_NUMNODES ; + nbits =MAX_NUMNODES ; t &= ~((1UL << (MAX_NUMNODES %BITS_PER_LONG )) -1 ); }if (t)return -EINVAL ; } -return get_bitmap (nodes_addr (*nodes), nmask, maxnode); +return get_bitmap (nodes_addr (*nodes), nmask, nbits); }/* Copy a kernel node mask to user space */ static int copy_nodes_to_user (unsigned long __user *mask,unsigned long maxnode,nodemask_t *nodes) { -unsigned long copy =ALIGN (maxnode-1 ,64 ) /8 ; +unsigned long nbits = maxnode -1 ; +unsigned long copy =ALIGN (nbits,64 ) /8 ;unsigned int nbytes =BITS_TO_LONGS (nr_node_ids) *sizeof (long );bool compat =in_compat_syscall ();
@@ -1726,12 +1734,12 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode,if (clear_user ((char __user *)mask + nbytes, copy - nbytes))return -EFAULT ; copy = nbytes; - maxnode = nr_node_ids; + nbits = nr_node_ids; }if (compat)return compat_put_bitmap ((compat_ulong_t __user *)mask, -nodes_addr (*nodes), maxnode); +nodes_addr (*nodes), nbits);return copy_to_user (mask,nodes_addr (*nodes), copy) ? -EFAULT :0 ; }
Signature · 33 lines
2.56.1
--- 2/2 mm/mempolicy: size compat get_mempolicy() copies in compat words
Rounds the compat copy size to BITS_PER_COMPAT_LONG, so 32-bit callers
no longer get 4 bytes written past odd-sized masks.
Subject: [PATCH 2/2] mm/mempolicy: size compat get_mempolicy() copies in
compat words
copy_nodes_to_user() rounds the size of the user bitmap up to 64 bits
for compat callers too, whose bitmap is an array of 32-bit
compat_ulong_t.
When the caller's bitmap holds an odd number of compat words and is
larger than the kernel's nodes, the clear_user() of the tail writes
four bytes past its end.
Example: a 32-bit task asking for a 32-bit mask (maxnode=33) on a
2-node system gets 8 bytes written.
Round the compat copy size to BITS_PER_COMPAT_LONG, so it covers exactly
the words the caller passed.
Fixes: e130242dc351 ("mm: simplify compat numa syscalls")
Cc: [email protected]
---
mm/mempolicy.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)mm/mempolicy.c +3 -1
@@ -1725,8 +1725,10 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode,unsigned int nbytes =BITS_TO_LONGS (nr_node_ids) *sizeof (long );bool compat =in_compat_syscall (); -if (compat) +if (compat) { + copy =ALIGN (nbits,BITS_PER_COMPAT_LONG ) /8 ; nbytes =BITS_TO_COMPAT_LONGS (nr_node_ids) *sizeof (compat_long_t); + }if (copy > nbytes) {if (copy >PAGE_SIZE )
Signature · 1 line
2.56.1
Quoted 9 lines · Gregory Price (Meta)
What about the following:
--- 1/2 mm/mempolicy: convert maxnode to a bit count in
copy_nodes_to_user()Converts maxnode to a bit count on entry to
copy_nodes_to_user()andget_nodes(), fixing the compat KASAN stack over-read and stack leak. Fixese130242dc351, Cc stable.
Quoted 4 lines
Subject: [PATCH 1/2] mm/mempolicy: convert maxnode to a bit count in
copy_nodes_to_user()Whenever maxnode - 1 is a multiple of 32,
compat_put_bitmap()reads one
'multiple of 64' I think
Quoted 16 lines
more
compat_ulong_tfrom the kernel nodemask than the caller asked for, and writes it past the end of the caller's bitmap.When the kernel nodemask ends there, that read goes past the on-stack
nodemask_tinkernel_get_mempolicy(), and its low 32 bits are copied to userspace.With
CONFIG_NODES_SHIFT=6, 64 possible nodes and a 32-bit caller passing maxnode=65:BUG: KASAN: stack-out-of-bounds in compat_put_bitmap+0x122/0x130 Read of size 8 at addr ffff8880150d7ec0 by task mpoltest/241 compat_put_bitmap+0x122/0x130 copy_nodes_to_user+0x14a/0x250 __ia32_sys_get_mempolicy+0x1b0/0x2b0
Quoted 5 lines
Convert maxnode to a bit count once, on entry, and use only that in the rest of
copy_nodes_to_user(), so the compat path gets the same size as the native one. Do the same inget_nodes().Fixes e130242dc351 ("mm: simplify compat numa syscalls")
I like the renaming as a cleanup, though it does make it a little harder to backport to stable kernels from context changes, maybe it's better to do do the minimal fix first.
Quoted 8 lines
mm/mempolicy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)mm/mempolicy.c +0
-0
mm/mempolicy.c static int copy_nodes_to_user(unsigned long · line 1732
Quoted 9 lines
mm/mempolicy.c +3
-1@@ -1725,8 +1725,10 @@ static int copy_nodes_to_user(unsigned long __user *mask, unsigned long maxnode, unsigned int nbytes = BITS_TO_LONGS(nr_node_ids) * sizeof(long);bool compat =in_compat_syscall (); -if (compat) +if (compat) { + copy =ALIGN (nbits,BITS_PER_COMPAT_LONG ) /8 ; nbytes =BITS_TO_COMPAT_LONGS (nr_node_ids) *sizeof (compat_long_t); + }
It fixes the bug, but I'd drop the nbytes= assignment here, as it has no benefit: The only configurations where nbytes actually gets a different value is for NODES_SHIFT<6, but as the on-stack bitmask is still a 64-bit word here, the only difference is whether the extra zeroes come from memset() or compat_put_bitmap().
I assume that when I introduced this bug, I simply confused which of the two variables needed to be adjusted.
Arnd