dt:numa: adding numa node mapping for memory nodes.
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
--- Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 Documentation/devicetree/bindings/numa.txt
Documentation/devicetree/bindings/numa.txt new file +58 -0
@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg". +The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems. + +====================================================== +2 - nid property +====================================================== +Numa node id, "nid" is required property of memory device node for +numa enabled platforms. + +|------------------------------------------------------| +|Property Type | Usage | Value Type | Definition | +|------------------------------------------------------| +| nid | R | <u32> | Numa Node id | +| | | | for this memory | +|------------------------------------------------------| + +======================================================== +4 - Example memory nodes with numa node id mapping +======================================================== + +Example 1 (2 memory nodes, each mapped to a numa node.): + + memory at 00000000 { + device_type = "memory"; + reg = <0x0 0x00000000 0x0 0x80000000>; + nid = <0x0>; + }; + + memory at 10000000000 { + device_type = "memory"; + reg = <0x100 0x00000000 0x0 0x80000000>; + nid = <0x1>; + }; + +Example 2 (multiple memory ranges in each memory node and mapped to numa node): + + memory at 00000000 { + device_type = "memory"; + reg = <0x0 0x00000000 0x0 0x80000000>, + <0x0 0x80000000 0x0 0x80000000>; + nid = <0x0>; + }; + + memory at 10000000000 { + device_type = "memory"; + reg = <0x100 0x00000000 0x0 0x80000000>, + <0x100 0x80000000 0x0 0x80000000>; + nid = <0x1>; + };
Signature · 1 line
1.8.1.4
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
Quoted 8 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Signed-off-by Ganapatrao Kulkarni <[email protected]>?
Adding the PPC guys as they?ve been doing NUMA on IBM Power Servers for years with OF/DT. So we should really try and follow what they?ve done.
Quoted 71 lines
Documentation/devicetree/bindings/numa.txt| 58 ++++++++++++++++++++++++++++++1 file changed, 58 insertions(+)create mode 100644
Documentation/devicetree/bindings/numa.txtDocumentation/devicetree/bindings/numa.txt new file +58
-0@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg". +The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems. + +====================================================== +2 - nid property +====================================================== +Numa node id, "nid" is required property of memory device node for +numa enabled platforms. + +|------------------------------------------------------| +|Property Type | Usage | Value Type | Definition | +|------------------------------------------------------| +| nid | R | <u32> | Numa Node id | +| | | | for this memory | +|------------------------------------------------------| + +======================================================== +4 - Example memory nodes with numa node id mapping +======================================================== + +Example 1 (2 memory nodes, each mapped to a numa node.): + + memory at 00000000 { + device_type = "memory"; + reg = <0x0 0x00000000 0x0 0x80000000>; + nid = <0x0>; + }; + + memory at 10000000000 { + device_type = "memory"; + reg = <0x100 0x00000000 0x0 0x80000000>; + nid = <0x1>; + }; + +Example 2 (multiple memory ranges in each memory node and mapped to numa node): + + memory at 00000000 { + device_type = "memory"; + reg = <0x0 0x00000000 0x0 0x80000000>, + <0x0 0x80000000 0x0 0x80000000>; + nid = <0x0>; + }; + + memory at 10000000000 { + device_type = "memory"; + reg = <0x100 0x00000000 0x0 0x80000000>, + <0x100 0x80000000 0x0 0x80000000>; + nid = <0x1>; + };Signature · 1 line
1.8.1.4
Signature · 2 lines
Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
Quoted 8 lines · Kumar Gala
Quoted 5 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Ganapatrao Kulkarni <[email protected]>?
Adding the PPC guys as they?ve been doing NUMA on IBM Power Servers for years with OF/DT. So we should really try and follow what they?ve done.
Agreed.
Quoted 4 lines
Quoted 4 lines
Documentation/devicetree/bindings/numa.txt| 58 ++++++++++++++++++++++++++++++1 file changed, 58 insertions(+)create mode 100644
Documentation/devicetree/bindings/numa.txt
Quoted 14 lines
Quoted 6 lines
Documentation/devicetree/bindings/numa.txt new file +0
-0Documentation/devicetree/bindings/numa.txt line 8
Quoted 8 lines
Documentation/devicetree/bindings/numa.txt new file +8
-0@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory
Why the quotes?
Quoted 3 lines
Documentation/devicetree/bindings/numa.txt line 11
Quoted 3 lines
Documentation/devicetree/bindings/numa.txt new file +3
-0+device node to map the range of memory addresses as defined in property "reg". +The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems.
What is a "numa node id", exactly, and how is the OS intended to use it?
I don't see how this can be of any use as-is.
Mark.
Quoted 60 lines
Quoted 50 lines
Documentation/devicetree/bindings/numa.txt new file +47
-0+ +====================================================== +2 - nid property +====================================================== +Numa node id, "nid" is required property of memory device node for +numa enabled platforms. + +|------------------------------------------------------| +|Property Type | Usage | Value Type | Definition | +|------------------------------------------------------| +| nid | R | <u32> | Numa Node id | +| | | | for this memory | +|------------------------------------------------------| + +======================================================== +4 - Example memory nodes with numa node id mapping +======================================================== + +Example 1 (2 memory nodes, each mapped to a numa node.): + + memory at 00000000 { + device_type = "memory"; + reg = <0x0 0x00000000 0x0 0x80000000>; + nid = <0x0>; + }; + + memory at 10000000000 { + device_type = "memory"; + reg = <0x100 0x00000000 0x0 0x80000000>; + nid = <0x1>; + }; + +Example 2 (multiple memory ranges in each memory node and mapped to numa node): + + memory at 00000000 { + device_type = "memory"; + reg = <0x0 0x00000000 0x0 0x80000000>, + <0x0 0x80000000 0x0 0x80000000>; + nid = <0x0>; + }; + + memory at 10000000000 { + device_type = "memory"; + reg = <0x100 0x00000000 0x0 0x80000000>, + <0x100 0x80000000 0x0 0x80000000>; + nid = <0x1>; + };Signature · 1 line
1.8.1.4Signature · 7 lines
Employee of Qualcomm Innovation Center, Inc. Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
Quoted 40 lines · Mark Rutland
Quoted 8 lines · Kumar Gala
Quoted 5 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Ganapatrao Kulkarni <[email protected]>?
Adding the PPC guys as they?ve been doing NUMA on IBM Power Servers for years with OF/DT. So we should really try and follow what they?ve done.
Agreed.
Quoted 4 lines
Quoted 4 lines
Documentation/devicetree/bindings/numa.txt| 58 ++++++++++++++++++++++++++++++1 file changed, 58 insertions(+)create mode 100644
Documentation/devicetree/bindings/numa.txtQuoted 14 lines
Quoted 6 lines
Documentation/devicetree/bindings/numa.txt new file +0
-0Documentation/devicetree/bindings/numa.txt line 8
Quoted 8 lines
Documentation/devicetree/bindings/numa.txt new file +8
-0@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memoryWhy the quotes?
Quoted 5 lines
Quoted 3 lines
Documentation/devicetree/bindings/numa.txt line 11
Quoted 3 lines
Documentation/devicetree/bindings/numa.txt new file +3
-0+device node to map the range of memory addresses as defined in property "reg". +The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems.What is a "numa node id", exactly, and how is the OS intended to use it?
I think "Proximity Domain" would be more suitably, processors and memory or IOs in the same domain will have better performance than crossing other domains.
Thanks
Hanjun
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
In reply to Ganapatrao KulkarniQuoted 19 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Signed-off-by Ganapatrao Kulkarni <[email protected]>--- Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 Documentation/devicetree/bindings/numa.txtDocumentation/devicetree/bindings/numa.txt new file +1
-0@@ -0,0 +1,58 @@ +======================================================
Documentation/devicetree/bindings/numa.txt line 11
Quoted 10 lines
Documentation/devicetree/bindings/numa.txt new file +10
-0+numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg". +The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems.
"Local" and "remote" memory are notions that relate to some other resource -- typically a CPU, but also I/O resources on some systems. It seems to me that a useful NUMA binding would at least specify a "nid" property, or something like it, for both cpu and memory nodes. But this document speaks only of memory nodes.
As Kumar said, the device tree on powerpc server systems already has properties that express NUMA information. If you can get hold of a copy of the PAPR (not ePAPR) from power.org, refer to the description of "ibm,associativity" and related properties. I recall that it's a bit more complex than this proposal, though.
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
Quoted 37 lines · Nathan Lynch
Quoted 19 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Signed-off-by Ganapatrao Kulkarni <[email protected]>--- Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 Documentation/devicetree/bindings/numa.txtDocumentation/devicetree/bindings/numa.txt new file +1
-0@@ -0,0 +1,58 @@ +======================================================Documentation/devicetree/bindings/numa.txt line 11
Quoted 10 lines
Documentation/devicetree/bindings/numa.txt new file +10
-0+numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg". +The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems."Local" and "remote" memory are notions that relate to some other resource -- typically a CPU, but also I/O resources on some systems. It seems to me that a useful NUMA binding would at least specify a "nid" property, or something like it, for both cpu and memory nodes. But this document speaks only of memory nodes.
Quoted 5 lines
As Kumar said, the device tree on powerpc server systems already has properties that express NUMA information. If you can get hold of a copy of the PAPR (not ePAPR) from power.org, refer to the description of "ibm,associativity" and related properties. I recall that it's a bit more complex than this proposal, though.
I'm not able to find a link to the actual PAPR (not ePAPR) specification. Anyone has a linky?
I did find some code in tree, but couldn't find the bindings in Documentation/devicetree/.
Seems like we'd care about form1?
-----8<-----
commit 41eab6f88f24124df89e38067b3766b7bef06ddb
Author: Anton Blanchard <[email protected]> Date: Sun May 16 20:22:31 2010 +0000
powerpc/numa: Use form 1 affinity to setup node distance
Form 1 affinity allows multiple entries in
ibm,associativity-reference-points
which represent affinity domains in decreasing order of importance. The Linux concept of a node is always the first entry, but using the other values as an input to node_distance() allows the memory allocator to make better decisions on which node to go first when local memory has been exhausted.
We keep things simple and create an array indexed by NUMA node, capped at 4 entries. Each time we lookup an associativity property we initialise the array which is overkill, but since we should only hit this path during boot it didn't seem worth adding a per node valid bit.
----->8-----
Quoted 5 lines
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
Quoted 29 lines · Zi Shen Lim
Quoted 28 lines · Nathan Lynch
Quoted 27 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Signed-off-by Ganapatrao Kulkarni <[email protected]>--- Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 Documentation/devicetree/bindings/numa.txtDocumentation/devicetree/bindings/numa.txt new file +9
-0@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg".
Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/numa.txt line 11
Quoted 2 lines
Documentation/devicetree/bindings/numa.txt new file +2
-0+The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems."Local" and "remote" memory are notions that relate to some other resource -- typically a CPU, but also I/O resources on some systems. It seems to me that a useful NUMA binding would at least specify a "nid" property, or something like it, for both cpu and memory nodes. But this document speaks only of memory nodes.
IMO, nid can be extended to cpu node also if the arch dont have any other way to do mapping, in fact this can be scaled to any node. for ARM arch, cpu mapping can be fetched from topology parsing and more over cpu mapping is required much later when secondary cpus comes online.
i have added for memory, since this is parsed very early in the boot sequence and there is no support for dt based map at this moment(at least i am not aware off).
numa distance matrix can also be defined, if the architecture has variable node distance other than just local/remote.
Quoted 7 lines
Quoted 6 lines
As Kumar said, the device tree on powerpc server systems already has properties that express NUMA information. If you can get hold of a copy of the PAPR (not ePAPR) from power.org, refer to the description of "ibm,associativity" and related properties. I recall that it's a bit more complex than this proposal, though.
Quoted 5 lines
I'm not able to find a link to the actual PAPR (not ePAPR) specification. Anyone has a linky?
I did find some code in tree, but couldn't find the bindings in
Documentation/devicetree/.
yes, there is nothing in Documentation, if PPC are using since several years, then i humbly request to publish the documentation for numa bindings.
No point in reinventing the wheel. Also i did not find any spec named PAPR, is it open spec, available to all like ePAPR?
Quoted 32 lines
Seems like we'd care about form1?
-----8<-----
commit41eab6f88f24124df89e38067b3766b7bef06ddbAuthor: Anton Blanchard <[email protected]> Date: Sun May 16 20:22:31 2010 +0000powerpc/numa: Use form 1 affinity to setup node distance
Form 1 affinity allows multiple entries in
ibm,associativity-reference-points
which represent affinity domains in decreasing order of importance. The Linux concept of a node is always the first entry, but using the other values as an input tonode_distance()allows the memory allocator to make better decisions on which node to go first when local memory has been exhausted.We keep things simple and create an array indexed by NUMA node, capped at 4 entries. Each time we lookup an associativity property we initialise the array which is overkill, but since we should only hit this path during boot it didn't seem worth adding a per node valid bit.
Signed-off-by Anton Blanchard <[email protected]>Signed-off-by Benjamin Herrenschmidt <[email protected]>----->8-----
Quoted 5 lines
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
looks like previous email not had the email ids added by kumar gala. adding the missing ids and sending again.
Quoted 98 lines · Ganapatrao Kulkarni
Quoted 29 lines · Zi Shen Lim
Quoted 28 lines · Nathan Lynch
Quoted 27 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Signed-off-by Ganapatrao Kulkarni <[email protected]>--- Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 Documentation/devicetree/bindings/numa.txtDocumentation/devicetree/bindings/numa.txt new file +9
-0@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg".Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/numa.txt line 11
Quoted 2 lines
Documentation/devicetree/bindings/numa.txt new file +2
-0+The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems."Local" and "remote" memory are notions that relate to some other resource -- typically a CPU, but also I/O resources on some systems. It seems to me that a useful NUMA binding would at least specify a "nid" property, or something like it, for both cpu and memory nodes. But this document speaks only of memory nodes.
IMO, nid can be extended to cpu node also if the arch dont have any other way to do mapping, in fact this can be scaled to any node. for ARM arch, cpu mapping can be fetched from topology parsing and more over cpu mapping is required much later when secondary cpus comes online.
i have added for memory, since this is parsed very early in the boot sequence and there is no support for dt based map at this moment(at least i am not aware off).
numa distance matrix can also be defined, if the architecture has variable node distance other than just local/remote.Quoted 7 lines
Quoted 6 lines
As Kumar said, the device tree on powerpc server systems already has properties that express NUMA information. If you can get hold of a copy of the PAPR (not ePAPR) from power.org, refer to the description of "ibm,associativity" and related properties. I recall that it's a bit more complex than this proposal, though.
Quoted 5 lines
I'm not able to find a link to the actual PAPR (not ePAPR) specification. Anyone has a linky?
I did find some code in tree, but couldn't find the bindings in
Documentation/devicetree/.yes, there is nothing in Documentation, if PPC are using since several years, then i humbly request to publish the documentation for numa bindings.
No point in reinventing the wheel. Also i did not find any spec named PAPR, is it open spec, available to all like ePAPR?Quoted 32 lines
Seems like we'd care about form1?
-----8<-----
commit41eab6f88f24124df89e38067b3766b7bef06ddbAuthor: Anton Blanchard <[email protected]> Date: Sun May 16 20:22:31 2010 +0000powerpc/numa: Use form 1 affinity to setup node distance
Form 1 affinity allows multiple entries in
ibm,associativity-reference-points
which represent affinity domains in decreasing order of importance. The Linux concept of a node is always the first entry, but using the other values as an input tonode_distance()allows the memory allocator to make better decisions on which node to go first when local memory has been exhausted.We keep things simple and create an array indexed by NUMA node, capped at 4 entries. Each time we lookup an associativity property we initialise the array which is overkill, but since we should only hit this path during boot it didn't seem worth adding a per node valid bit.
Signed-off-by Anton Blanchard <[email protected]>Signed-off-by Benjamin Herrenschmidt <[email protected]>----->8-----
Quoted 5 lines
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
RFC PATCH 1/1 dt:numa: adding numa node mapping for memory nodes.
In reply to Nathan LynchQuoted 28 lines · Nathan Lynch
Quoted 27 lines · Ganapatrao Kulkarni
From: Ganapatrao Kulkarni <[email protected]>
This patch adds property "nid" to memory node to provide the memory range to numa node id mapping.
Signed-off-by Ganapatrao Kulkarni <[email protected]>--- Documentation/devicetree/bindings/numa.txt | 58 ++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 Documentation/devicetree/bindings/numa.txtDocumentation/devicetree/bindings/numa.txt new file +9
-0@@ -0,0 +1,58 @@ +====================================================== +numa id binding description +====================================================== + +====================================================== +1 - Introduction +====================================================== +The device node property "nid(numa node id)" can be added to memory +device node to map the range of memory addresses as defined in property "reg".
Quoted 8 lines
Documentation/devicetree/bindings/numa.txt line 11
Quoted 2 lines
Documentation/devicetree/bindings/numa.txt new file +2
-0+The property "nid" maps the memory range to the numa node id, which is used to +find the local and remory pages on numa aware systems."Local" and "remote" memory are notions that relate to some other resource -- typically a CPU, but also I/O resources on some systems. It seems to me that a useful NUMA binding would at least specify a "nid" property, or something like it, for both cpu and memory nodes. But this document speaks only of memory nodes.
Agreed. and more, I think I/O resources also need such property, it will have performance influence for the proximity domain of I/O devices too.
Thanks
Hanjun