NUMA x86: add constraints check for nid parameters
PATCH RESEND 1/1 NUMA x86: add constraints check for nid parameters
This patch adds constraints checks into __node_distance() and numa_set_distance() functions. If from or to parameters are lower than zero, it results into oops now.
--- arch/x86/mm/numa.c | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-)
arch/x86/mm/numa.c +7 -4
@@ -430,8 +430,9 @@ static int __init numa_alloc_distance(void)* calls are ignored until the distance table is reset with * numa_reset_distance(). * -* If @from or @to is higher than the highest known node at the time of -* table creation or @distance doesn't make sense, the call is ignored. +* If @from or @to is higher than the highest known node or lower than zero +* at the time of table creation or @distance doesn't make sense, the call +* is ignored. * This is to allow simplification of specific NUMA config implementations. */ void __init numa_set_distance (int from,int to,int distance)
@@ -439,7 +440,8 @@ void __init numa_set_distance(int from, int to, int distance)if (!numa_distance &&numa_alloc_distance () <0 )return ; -if (from >= numa_distance_cnt || to >= numa_distance_cnt) { +if (from >= numa_distance_cnt || to >= numa_distance_cnt || + from <0 || to <0 ) {printk_once (KERN_DEBUG "NUMA: Debug: distance out of bound, from=%d to=%d distance=%d\n" , from, to, distance);return ;
@@ -457,7 +459,8 @@ void __init numa_set_distance(int from, int to, int distance)int __node_distance (int from,int to) { -if (from >= numa_distance_cnt || to >= numa_distance_cnt) +if (from <0 || to <0 || + from >= numa_distance_cnt || to >= numa_distance_cnt)return from == to ?LOCAL_DISTANCE :REMOTE_DISTANCE ;return numa_distance[from * numa_distance_cnt + to]; }
Signature · 1 line
1.7.6.4
Quoted 3 lines · On Thu, 1 Dec 2011 12:45:07 +0100 Petr Holasek <[email protected]> wrote:
This patch adds constraints checks into
__node_distance()andnuma_set_distance()functions. If from or to parameters are lower than zero, it results into oops now.
Passing negative numbers into __node_distance() sounds like a bug in the caller, and this patch will remove our means of detecting that bug.
Perhaps we need to be told more about this patch. Is the bug user-triggerable? If so, how? How was this fault triggered? Etcetera.
Quoted 12 lines · Andrew Morton
Date: Thu, 1 Dec 2011 13:34:51 -0800
From: Andrew Morton <[email protected]>
To: Petr Holasek <[email protected]>
Cc: Thomas Gleixner <[email protected]>, Ingo Molnar <[email protected]>,
"H. Peter Anvin" <[email protected]>, [email protected], Anton Arapov <[email protected]>
Subject: Re: [PATCH RESEND] NUMA x86: add constraints check for nid
parametersOn Thu, 1 Dec 2011 12:45:07 +0100
Petr Holasek <[email protected]> wrote:
Quoted 6 lines
Quoted 3 lines
This patch adds constraints checks into
__node_distance()andnuma_set_distance()functions. If from or to parameters are lower than zero, it results into oops now.Passing negative numbers into
__node_distance()sounds like a bug in the caller, and this patch will remove our means of detecting that bug.
That's true, but upper boundary is checked now, so why not to check lower? Seems inconsistent to me - from this point of view even don't check anything would be better for detecting bug in the caller.
Quoted 5 lines
Perhaps we need to be told more about this patch. Is the bug user-triggerable? If so, how? How was this fault triggered? Etcetera.
AFAIK, neither __node_distance() nor numa_set_distance() aren't in any path from user-space inputs. Their paramaters are based on ACPI tables provided by HW vendors.
Quoted 14 lines · On Fri, 2 Dec 2011 00:14:42 +0100 Petr Holasek <[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Date: Thu, 1 Dec 2011 13:34:51 -0800
From: Andrew Morton <[email protected]>
To: Petr Holasek <[email protected]>
Cc: Thomas Gleixner <[email protected]>, Ingo Molnar <[email protected]>,
"H. Peter Anvin" <[email protected]>, [email protected], Anton Arapov <[email protected]>
Subject: Re: [PATCH RESEND] NUMA x86: add constraints check for nid
parametersOn Thu, 1 Dec 2011 12:45:07 +0100
Petr Holasek <[email protected]> wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This patch adds constraints checks into
__node_distance()andnuma_set_distance()functions. If from or to parameters are lower than zero, it results into oops now.Passing negative numbers into
__node_distance()sounds like a bug in the caller, and this patch will remove our means of detecting that bug.That's true, but upper boundary is checked now, so why not to check lower?
Because it adds more code to the kernel and can hide bugs?
Quoted 4 lines
Seems inconsistent to me - from this point of view even don't check anything would be better for detecting bug in the caller.
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Perhaps we need to be told more about this patch. Is the bug user-triggerable? If so, how? How was this fault triggered? Etcetera.
AFAIK, neither
__node_distance()nornuma_set_distance()aren't in any path from user-space inputs. Their paramaters are based on ACPI tables provided by HW vendors.
That didn't answer my questions. Have you observed any problems in this code? If so, please fully describe them. Or was it purely from code inspection?
If what we're doing here is to be defensive against buggy BIOS tables (a good idea) then we should validate the BIOS table values as close as possible to the point where they were read frmo the BIOS. And we should (probably) emit a warning if a bad table entry is detected, rather than silently fixing it up.
Quoted 29 lines · Andrew Morton
Quoted 5 lines · On Fri, 2 Dec 2011 00:14:42 +0100 Petr Holasek <[email protected]> wrote:
Quoted 3 lines · Andrew Morton
On Thu, 1 Dec 2011 12:45:07 +0100
Petr Holasek <[email protected]> wrote:Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This patch adds constraints checks into
__node_distance()andnuma_set_distance()functions. If from or to parameters are lower than zero, it results into oops now.Passing negative numbers into
__node_distance()sounds like a bug in the caller, and this patch will remove our means of detecting that bug.That's true, but upper boundary is checked now, so why not to check lower?
Because it adds more code to the kernel and can hide bugs?
Quoted 9 lines
Seems inconsistent to me - from this point of view even don't check anything would be better for detecting bug in the caller.
Quoted 5 lines
Perhaps we need to be told more about this patch. Is the bug user-triggerable? If so, how? How was this fault triggered? Etcetera.
Quoted 7 lines
Quoted 3 lines
AFAIK, neither
__node_distance()nornuma_set_distance()aren't in any path from user-space inputs. Their paramaters are based on ACPI tables provided by HW vendors.That didn't answer my questions. Have you observed any problems in this code? If so, please fully describe them. Or was it purely from code inspection?
Code inspection only, nothing from real life.
Quoted 6 lines
If what we're doing here is to be defensive against buggy BIOS tables (a good idea) then we should validate the BIOS table values as close as possible to the point where they were read frmo the BIOS. And we should (probably) emit a warning if a bad table entry is detected, rather than silently fixing it up.
numa_set_distance() does exactly what you described above, only emits a warning. I agree with your objections with __node_distance() checks, it really can hide bugs in caller. So silent fix-up is the main problem and we shouldn't check anything so the caller will be advised when using wrong nid by oops with a benefit of less code for us. Do I understand your opinion on this type of code?
Quoted 11 lines · Petr Holasek
Quoted 11 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This patch adds constraints checks into
__node_distance()andnuma_set_distance()functions. If from or to parameters are lower than zero, it results into oops now.Passing negative numbers into
__node_distance()sounds like a bug in the caller, and this patch will remove our means of detecting that bug.That's true, but upper boundary is checked now, so why not to check lower?
Because it adds more code to the kernel and can hide bugs?
The upper bound is checked to ensure that we don't dereference past end of the array that stores the distance table, so it will catch errors for things like memory hotplug when additional nodes are onlined and the data structure isn't updated accordingly.
Quoted 6 lines
Quoted 5 lines
If what we're doing here is to be defensive against buggy BIOS tables (a good idea) then we should validate the BIOS table values as close as possible to the point where they were read frmo the BIOS. And we should (probably) emit a warning if a bad table entry is detected, rather than silently fixing it up.
Quoted 6 lines
numa_set_distance()does exactly what you described above, only emits a warning. I agree with your objections with__node_distance()checks, it really can hide bugs in caller. So silent fix-up is the main problem and we shouldn't check anything so the caller will be advised when using wrong nid by oops with a benefit of less code for us. Do I understand your opinion on this type of code?
I'd have no objection to adding a check to numa_set_distance() to ensure the node ids are non-negative in the same way we check that the distances themselves are non-negative; that can catch errors when pxms are used uninitialized when parsing the SRAT. However, I think adding the check to __node_distance() is unnecessary.
Quoted 12 lines · David Rientjes
Quoted 9 lines · Petr Holasek
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
This patch adds constraints checks into
__node_distance()andnuma_set_distance()functions. If from or to parameters are lower than zero, it results into oops now.Passing negative numbers into
__node_distance()sounds like a bug in the caller, and this patch will remove our means of detecting that bug.That's true, but upper boundary is checked now, so why not to check lower?
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Because it adds more code to the kernel and can hide bugs?
The upper bound is checked to ensure that we don't dereference past end of the array that stores the distance table, so it will catch errors for things like memory hotplug when additional nodes are onlined and the data structure isn't updated accordingly.
Thanks for clarification, I missed this case.
Quoted 14 lines
Quoted 7 lines
Quoted 6 lines
If what we're doing here is to be defensive against buggy BIOS tables (a good idea) then we should validate the BIOS table values as close as possible to the point where they were read frmo the BIOS. And we should (probably) emit a warning if a bad table entry is detected, rather than silently fixing it up.
Quoted 6 lines
numa_set_distance()does exactly what you described above, only emits a warning. I agree with your objections with__node_distance()checks, it really can hide bugs in caller. So silent fix-up is the main problem and we shouldn't check anything so the caller will be advised when using wrong nid by oops with a benefit of less code for us. Do I understand your opinion on this type of code?
Quoted 5 lines
I'd have no objection to adding a check to
numa_set_distance()to ensure the node ids are non-negative in the same way we check that the distances themselves are non-negative; that can catch errors when pxms are used uninitialized when parsing the SRAT. However, I think adding the check to__node_distance()is unnecessary.
Ok, I'll try v2 without __node_distance()
PATCH v2 1/1 NUMA x86: add constraints check for nid parameters
In reply to David RientjesThis patch adds constraints checks for numa_set_distance() function. It emits warning when pxms are used uninitialized when parsing the SRAT.
--- arch/x86/mm/numa.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-)
arch/x86/mm/numa.c +5 -3
@@ -430,8 +430,9 @@ static int __init numa_alloc_distance(void)* calls are ignored until the distance table is reset with * numa_reset_distance(). * -* If @from or @to is higher than the highest known node at the time of -* table creation or @distance doesn't make sense, the call is ignored. +* If @from or @to is higher than the highest known node or lower than zero +* at the time of table creation or @distance doesn't make sense, the call +* is ignored. * This is to allow simplification of specific NUMA config implementations. */ void __init numa_set_distance (int from,int to,int distance)
@@ -439,7 +440,8 @@ void __init numa_set_distance(int from, int to, int distance)if (!numa_distance &&numa_alloc_distance () <0 )return ; -if (from >= numa_distance_cnt || to >= numa_distance_cnt) { +if (from >= numa_distance_cnt || to >= numa_distance_cnt || + from <0 || to <0 ) {printk_once (KERN_DEBUG "NUMA: Debug: distance out of bound, from=%d to=%d distance=%d\n" , from, to, distance);return ;
Signature · 1 line
1.7.6.4
Quoted 4 lines · Petr Holasek
This patch adds constraints checks for
numa_set_distance()function. It emits warning when pxms are used uninitialized when parsing the SRAT.
The changelog also needs to indicate that, in addition to emitting the warning, that this will avoid a store to a negative index in the numa_distance[] array, that's the real thing you're trying to catch. This would happen if pxm_to_node() is returning NUMA_NO_NODE which is what the pxm-to-node mapping is initialized to.
Quoted 22 lines
Signed-off-by Petr Holasek <[email protected]>--- arch/x86/mm/numa.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-)arch/x86/mm/numa.c +3
-2@@ -430,8 +430,9 @@ static int __init numa_alloc_distance(void)* calls are ignored until the distance table is reset with * numa_reset_distance(). * -* If @from or @to is higher than the highest known node at the time of -* table creation or @distance doesn't make sense, the call is ignored. +* If @from or @to is higher than the highest known node or lower than zero +* at the time of table creation or @distance doesn't make sense, the call +* is ignored. * This is to allow simplification of specific NUMA config implementations. */ void __init numa_set_distance (int from,int to,int distance)
arch/x86/mm/numa.c void __init numa_set_distance(int from, int to, int distance) · line 446
Quoted 8 lines
arch/x86/mm/numa.c +2
-1@@ -439,7 +440,8 @@ void __init numa_set_distance(int from, int to, int distance)if (!numa_distance &&numa_alloc_distance () <0 )return ; -if (from >= numa_distance_cnt || to >= numa_distance_cnt) { +if (from >= numa_distance_cnt || to >= numa_distance_cnt || + from <0 || to <0 ) {printk_once (KERN_DEBUG "NUMA: Debug: distance out of bound, from=%d to=%d distance=%d\n" , from, to, distance);
That warning doesn't look appropriate for negative node ids, it should be modified to say the node ids or distance are out of bounds. One more revision?
Quoted 1 line
arch/x86/mm/numa.c +0
-0return ;
PATCH v3 1/1 NUMA x86: add constraints check for nid parameters
This patch adds constraints checks for numa_set_distance() function. It emits warning and avoids a store to a negative index in numa_distance[] array. Negative ids can be passed when pxm-to-nids mapping is not properly filled while parsing the SRAT.
--- arch/x86/mm/numa.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)
arch/x86/mm/numa.c +6 -4
@@ -430,8 +430,9 @@ static int __init numa_alloc_distance(void)* calls are ignored until the distance table is reset with * numa_reset_distance(). * -* If @from or @to is higher than the highest known node at the time of -* table creation or @distance doesn't make sense, the call is ignored. +* If @from or @to is higher than the highest known node or lower than zero +* at the time of table creation or @distance doesn't make sense, the call +* is ignored. * This is to allow simplification of specific NUMA config implementations. */ void __init numa_set_distance (int from,int to,int distance)
@@ -439,8 +440,9 @@ void __init numa_set_distance(int from, int to, int distance)if (!numa_distance &&numa_alloc_distance () <0 )return ; -if (from >= numa_distance_cnt || to >= numa_distance_cnt) { -printk_once (KERN_DEBUG "NUMA: Debug: distance out of bound, from=%d to=%d distance=%d\n" , +if (from >= numa_distance_cnt || to >= numa_distance_cnt || + from <0 || to <0 ) { +pr_warn_once ("NUMA: Warning: node ids are out of bound, from=%d to=%d distance=%d\n" , from, to, distance);return ; }
Signature · 1 line
1.7.6.4
Quoted 6 lines · Petr Holasek
This patch adds constraints checks for
numa_set_distance()function. It emits warning and avoids a store to a negative index in numa_distance[] array. Negative ids can be passed when pxm-to-nids mapping is not properly filled while parsing the SRAT.Signed-off-by Petr Holasek <[email protected]>
Thanks Petr!
Quoted 10 lines · David Rientjes
Quoted 6 lines · Petr Holasek
This patch adds constraints checks for
numa_set_distance()function. It emits warning and avoids a store to a negative index in numa_distance[] array. Negative ids can be passed when pxm-to-nids mapping is not properly filled while parsing the SRAT.Signed-off-by Petr Holasek <[email protected]>Acked-by David Rientjes <[email protected]>
Thanks guys.
Just to double check: there's no known instance of such a bad SRAT in existence, so this commit can wait until v3.3 and does hot have to go into v3.2, right?
Thanks,
Ingo
Quoted 6 lines · Ingo Molnar
Thanks guys.
Just to double check: there's no known instance of such a bad SRAT in existence, so this commit can wait until v3.3 and does hot have to go into v3.2, right?
Andi mentioned buggy SRATs, but it's outside the scope of this patch; this has nothing to do with buggy SRATs.
Ignoring NUMA emulation which plays with these when you boot with numa=fake, this patch is catching any instance where pxm_to_node(x) returns NUMA_NO_NODE, meaning x is not an initialized pxm. For x86, we initialize pxms by grabbing them from the SRAT, picking the first unused node id, and mapping them. The SRAT has nothing to do with this, the ACPI spec has no notion of what we've defined a node to be. So Petr's patch will simply catch any instance of pxm_to_node(x) where x was not initialized and incorrectly referencing
[tip:x86/mm] x86/numa: Add constraints check for nid parameters
In reply to Petr HolasekCommit-ID: 54eed6cb16ec315565aaaf8e34252ca253a68b7b Gitweb: http://git.kernel.org/tip/54eed6cb16ec315565aaaf8e34252ca253a68b7b Author: Petr Holasek <[email protected]> AuthorDate: Thu, 8 Dec 2011 13:16:41 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Dec 2011 08:03:34 +0100
x86/numa: Add constraints check for nid parameters
This patch adds constraint checks to the numa_set_distance() function.
When the check triggers (this should not happen normally) it emits a warning and avoids a store to a negative index in numa_distance[] array - i.e. avoids memory corruption.
Negative ids can be passed when the pxm-to-nids mapping is not properly filled while parsing the SRAT.
--- arch/x86/mm/numa.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)
arch/x86/mm/numa.c +6 -4
@@ -430,8 +430,9 @@ static int __init numa_alloc_distance(void)* calls are ignored until the distance table is reset with * numa_reset_distance(). * -* If @from or @to is higher than the highest known node at the time of -* table creation or @distance doesn't make sense, the call is ignored. +* If @from or @to is higher than the highest known node or lower than zero +* at the time of table creation or @distance doesn't make sense, the call +* is ignored. * This is to allow simplification of specific NUMA config implementations. */ void __init numa_set_distance (int from,int to,int distance)
@@ -439,8 +440,9 @@ void __init numa_set_distance(int from, int to, int distance)if (!numa_distance &&numa_alloc_distance () <0 )return ; -if (from >= numa_distance_cnt || to >= numa_distance_cnt) { -printk_once (KERN_DEBUG "NUMA: Debug: distance out of bound, from=%d to=%d distance=%d\n" , +if (from >= numa_distance_cnt || to >= numa_distance_cnt || + from <0 || to <0 ) { +pr_warn_once ("NUMA: Warning: node ids are out of bound, from=%d to=%d distance=%d\n" , from, to, distance);return ; }
Quoted 3 lines · Petr Holasek
This patch adds constraints checks for
numa_set_distance()function. It emits warning when pxms are used uninitialized when parsing the SRAT.
Can you expand on the motivation for this please? Is this to catch buggy code or to catch buggy SRATs? If the later are they actually out there?
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
Quoted 5 lines · Andi Kleen
Quoted 3 lines
This patch adds constraints checks for
numa_set_distance()function. It emits warning when pxms are used uninitialized when parsing the SRAT.Can you expand on the motivation for this please?
It's adding a debugging check if something erroneously passes a negative node id when setting a distance, perhaps because of buggy code that failed to initialize the pxm-to-node mapping correctly which would result in storing to a negative index of the numa_distance array as indicated in the changelog. It's for robustness and to warn explicitly rather than silently rejecting it unless you're monitoring debug output.
Quoted 8 lines · Andi Kleen
Quoted 3 lines · Petr Holasek
This patch adds constraints checks for
numa_set_distance()function. It emits warning when pxms are used uninitialized when parsing the SRAT.Can you expand on the motivation for this please? Is this to catch buggy code or to catch buggy SRATs?
Both.
Quoted 1 line
If the later are they actually out there?
Not that i know of - they would likely not boot Linux very well today.
Thanks,
Ingo