drivers/of: Introduce ARCH_HAS_OWN_OF_NUMA

PATCH In review Oliver O'Halloran 4 participants 7 messages Apr 9, 2018 – Apr 16, 2018

Subsystems (5)

PowerPC SPARC + UltraSPARC (sparc/sparc64) libnvdimm: non-volatile memory device Device tree +1 more subsystems

lore

Share & tools

lore

Oliver O'Halloran

author maintainer

2018-04-09 07:46 UTC #0 lore raw

PATCH 1/1 drivers/of: Introduce ARCH_HAS_OWN_OF_NUMA

Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.

In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.

The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:

a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()

Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.

This patch remedies the problem by introducing the ARCH_HAS_OWN_OF_NUMA Kconfig option which is selected if an architecture provides an implementation of of_node_to_nid(). If a platform does not use it's own, or the generic OF_NUMA, then always use the inline fallback in of.h so we don't need to futz around with exports.

Cc [email protected]

Cc linuxppc-dev-uLR06cmDAlY/[email protected]

Fixes 298535c00a2c ("of, numa: Add NUMA of binding implementation.")

Signed-off-by Oliver O'Halloran <[email protected]>

---
 arch/powerpc/Kconfig | 1 +
 arch/sparc/Kconfig   | 1 +
 drivers/of/Kconfig   | 3 +++
 drivers/of/base.c    | 7 -------
 include/linux/of.h   | 2 +-
 5 files changed, 6 insertions(+), 8 deletions(-)

arch/powerpc/Kconfig +1 -0

index c32a181a7cbb..74ce5f3564ae 100644
@@ -625,6 +625,7 @@ config NUMA
 	bool "NUMA support"
 	depends on PPC64
 	default y if SMP && PPC_PSERIES
+	select ARCH_HAS_OWN_OF_NUMA
 
 config NODES_SHIFT
 	int

arch/sparc/Kconfig +1 -0

index 8767e45f1b2b..f8071f1c3edb 100644
@@ -299,6 +299,7 @@ config GENERIC_LOCKBREAK
 config NUMA
 	bool "NUMA support"
 	depends on SPARC64 && SMP
+	select ARCH_HAS_OWN_OF_NUMA
 
 config NODES_SHIFT
 	int "Maximum NUMA Nodes (as a power of 2)"

drivers/of/Kconfig +3 -0

index ad3fcad4d75b..01c62b747b25 100644
@@ -103,4 +103,7 @@ config OF_OVERLAY
 config OF_NUMA
 	bool
 
+config ARCH_HAS_OWN_OF_NUMA
+	bool
+
 endif # OF

drivers/of/base.c +0 -7

index 848f549164cd..82a9584bb0e2 100644
@@ -84,13 +84,6 @@ int of_n_size_cells(struct device_node *np)
 }
 EXPORT_SYMBOL(of_n_size_cells);
 
-#ifdef CONFIG_NUMA
-int __weak of_node_to_nid(struct device_node *np)
-{
-	return NUMA_NO_NODE;
-}
-#endif
-
 static struct device_node **phandle_cache;
 static u32 phandle_cache_mask;

include/linux/of.h +1 -1

index 4d25e4f952d9..9bb42dac5e65 100644
@@ -942,7 +942,7 @@ static inline int of_cpu_node_to_id(struct device_node *np)
 #define of_node_cmp(s1, s2)		strcasecmp((s1), (s2))
 #endif
 
-#if defined(CONFIG_OF) && defined(CONFIG_NUMA)
+#if defined(CONFIG_OF_NUMA) || defined(CONFIG_ARCH_HAS_OWN_OF_NUMA)
 extern int of_node_to_nid(struct device_node *np);
 #else
 static inline int of_node_to_nid(struct device_node *device)

Signature · 1 line

2.9.5

Christoph Hellwig

2018-04-09 17:49 UTC #1 lore raw

Quoted 19 lines · Oliver O'Halloran

Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.
In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.
The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:
a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()

Quoted 8 lines

Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.
This patch remedies the problem by introducing the ARCH_HAS_OWN_OF_NUMA Kconfig option which is selected if an architecture provides an implementation of of_node_to_nid(). If a platform does not use it's own, or the generic OF_NUMA, then always use the inline fallback in of.h so we don't need to futz around with exports.

I'd rather have a specific kconfig symbol for the 'generic' implementation, especially given that it doesn't appear to be all that generic.

Rob Herring

maintainer

2018-04-09 20:52 UTC #2 lore raw

In reply to Oliver O'Halloran

Quoted 15 lines · Oliver O'Halloran

Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.
In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.
The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:

Quoted 6 lines

a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()
Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.

How specifically do we hit this? The only module I see using of_node_to_nid in mainline is Cell EDAC driver.

Quoted 5 lines

This patch remedies the problem by introducing the ARCH_HAS_OWN_OF_NUMA Kconfig option which is selected if an architecture provides an implementation of of_node_to_nid(). If a platform does not use it's own, or the generic OF_NUMA, then always use the inline fallback in of.h so we don't need to futz around with exports.

I'm more inclined to figure out how to remove the export and provide a non DT specific function if drivers need to know this.

Rob

Dan Williams

2018-04-09 21:05 UTC #3 lore raw

Quoted 20 lines · Rob Herring

Quoted 19 lines · Oliver O'Halloran
Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.
In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.
The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:
a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()

Quoted 5 lines

Quoted 2 lines
Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.
How specifically do we hit this? The only module I see using of_node_to_nid in mainline is Cell EDAC driver.

The of_pmem driver is using it currently pending for a 4.17 pull request. Stephen hit the compile failure in -next.

Rob Herring

maintainer

2018-04-10 01:02 UTC #4 lore raw

Quoted 21 lines · Dan Williams

Quoted 20 lines · Rob Herring
Quoted 19 lines · Oliver O'Halloran
Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.
In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.
The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:
a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.
How specifically do we hit this? The only module I see using of_node_to_nid in mainline is Cell EDAC driver.
The of_pmem driver is using it currently pending for a 4.17 pull request. Stephen hit the compile failure in -next.

You mean the stuff reviewed last week in the middle of the merge window? Sounds like 4.18 material to me.

Rob

Dan Williams

2018-04-10 02:29 UTC #5 lore raw

Quoted 25 lines · Rob Herring

Quoted 24 lines · Dan Williams
Quoted 23 lines · Rob Herring
Quoted 21 lines · Oliver O'Halloran
Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.
In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.
The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:
a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()
Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
How specifically do we hit this? The only module I see using of_node_to_nid in mainline is Cell EDAC driver.
The of_pmem driver is using it currently pending for a 4.17 pull request. Stephen hit the compile failure in -next.
You mean the stuff reviewed last week in the middle of the merge window? Sounds like 4.18 material to me.

It was originally posted for 4.16. The reposting and review came in late this cycle, but outside of a critical issue I'd rather not delay it again. The build error issue is resolved by not allowing modular builds of this driver for now.

Quoted 2 lines

Rob

Rob Herring

maintainer

2018-04-16 16:47 UTC #6 lore raw

In reply to Dan Williams

Quoted 21 lines · Dan Williams

Quoted 20 lines · Rob Herring
Quoted 19 lines · Oliver O'Halloran
Some OF platforms (pseries and some SPARC systems) has their own implementations of NUMA affinity detection rather than using the generic OF_NUMA driver, which mainly exists for arm64. For other platforms one of two fallbacks provided by the base OF driver are used depending on CONFIG_NUMA.
In the CONFIG_NUMA=n case the fallback is an inline function in of.h. In the =y case the fallback is a real function which is defined as a weak symbol so that it may be overwritten by the architecture if desired.
The problem with this arrangement is that the real implementations all export of_node_to_nid(). Unfortunately it's not possible to export the fallback since it would clash with the non-weak version. As a result we get build failures when:
a) CONFIG_NUMA=y && CONFIG_OF=y, and
b) The platform doesn't implement of_node_to_nid(), and
c) A module uses of_node_to_nid()

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
Given b) will be true for most platforms this is fairly easy to hit and has been observed on ia64 and x86.
How specifically do we hit this? The only module I see using of_node_to_nid in mainline is Cell EDAC driver.
The of_pmem driver is using it currently pending for a 4.17 pull request. Stephen hit the compile failure in -next.

I took a look at this. The correct fix here is to use dev_to_node() instead:

drivers/nvdimm/of_pmem.c +1 -1

index 85013bad35de..0a701837dfc0 100644
@@ -67,7 +67,7 @@ static int of_pmem_region_probe(struct platform_device *pdev)
                 */
                memset(&ndr_desc, 0, sizeof(ndr_desc));
                ndr_desc.attr_groups = region_attr_groups;
-               ndr_desc.numa_node = of_node_to_nid(np);
+               ndr_desc.numa_node = dev_to_node(&pdev->dev);
                ndr_desc.res = &pdev->resource[i];
                ndr_desc.of_node = np;
                set_bit(ND_REGION_PAGEMAP, &ndr_desc.flags);

And we should remove the exported symbol.

I'll send a proper patch.

Rob