memblock, numa: Binary search node id

PATCH In review Yinghai Lu 3 participants 7 messages Aug 15, 2013 – Aug 16, 2013

Subsystems (3)

Memory management Page allocator memblock and memory management initialization

lore

Share & tools

lore

Yinghai Lu

author

2013-08-15 05:46 UTC #0 lore raw

PATCH 1/1 memblock, numa: Binary search node id

Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.

We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.

Signed-off-by Yinghai Lu <[email protected]>

---
 include/linux/memblock.h |    2 ++
 mm/memblock.c            |   18 ++++++++++++++++++
 mm/page_alloc.c          |   19 +++++++++----------
 3 files changed, 29 insertions(+), 10 deletions(-)

include/linux/memblock.h +2 -0

Index: linux-2.6/include/linux/memblock.h
===================================================================
@@ -60,6 +60,8 @@ int memblock_reserve(phys_addr_t base, p
 void memblock_trim_memory(phys_addr_t align);
 
 #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn,
+			    unsigned long  *end_pfn);
 void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
 			  unsigned long *out_end_pfn, int *out_nid);

mm/memblock.c +18 -0

Index: linux-2.6/mm/memblock.c
===================================================================
@@ -914,6 +914,24 @@ int __init_memblock memblock_is_memory(p
 	return memblock_search(&memblock.memory, addr) != -1;
 }
 
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+int __init_memblock memblock_search_pfn_nid(unsigned long pfn,
+			 unsigned long *start_pfn, unsigned long *end_pfn)
+{
+	struct memblock_type *type = &memblock.memory;
+	int mid = memblock_search(type, (phys_addr_t)pfn << PAGE_SHIFT);
+
+	if (mid == -1)
+		return -1;
+
+	*start_pfn = type->regions[mid].base >> PAGE_SHIFT;
+	*end_pfn = (type->regions[mid].base + type->regions[mid].size)
+			>> PAGE_SHIFT;
+
+	return type->regions[mid].nid;
+}
+#endif
+
 /**
  * memblock_is_region_memory - check if a region is a subset of memory
  * @base: base of region to check

mm/page_alloc.c +9 ~~-10~~

Index: linux-2.6/mm/page_alloc.c
===================================================================
@@ -4186,7 +4186,7 @@ int __meminit init_currently_empty_zone(
 int __meminit __early_pfn_to_nid(unsigned long pfn)
 {
 	unsigned long start_pfn, end_pfn;
-	int i, nid;
+	int nid;
 	/*
 	 * NOTE: The following SMP-unsafe globals are only used early in boot
 	 * when the kernel is running single-threaded.

@@ -4197,15 +4197,14 @@ int __meminit __early_pfn_to_nid(unsigne
 	if (last_start_pfn <= pfn && pfn < last_end_pfn)
 		return last_nid;
 
-	for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid)
-		if (start_pfn <= pfn && pfn < end_pfn) {
-			last_start_pfn = start_pfn;
-			last_end_pfn = end_pfn;
-			last_nid = nid;
-			return nid;
-		}
-	/* This is a memory hole */
-	return -1;
+	nid = memblock_search_pfn_nid(pfn, &start_pfn, &end_pfn);
+	if (nid != -1) {
+		last_start_pfn = start_pfn;
+		last_end_pfn = end_pfn;
+		last_nid = nid;
+	}
+
+	return nid;
 }
 #endif /* CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID */

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Andrew Morton

maintainer

2013-08-15 20:43 UTC #1 lore raw

Quoted 7 lines · On Wed, 14 Aug 2013 22:46:29 -0700 Yinghai Lu <[email protected]> wrote:

Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.
We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.

Looks nice. I wonder how much difference it makes.

Quoted 28 lines

...

include/linux/memblock.h +2 -0

@@ -60,6 +60,8 @@ int memblock_reserve(phys_addr_t base, p
 void memblock_trim_memory(phys_addr_t align);
 
 #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn,
+			    unsigned long  *end_pfn);
 void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
 			  unsigned long *out_end_pfn, int *out_nid);

Index: linux-2.6/mm/memblock.c
===================================================================

mm/memblock.c +7 -0

@@ -914,6 +914,24 @@ int __init_memblock memblock_is_memory(p
 	return memblock_search(&memblock.memory, addr) != -1;
 }
 
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+int __init_memblock memblock_search_pfn_nid(unsigned long pfn,
+			 unsigned long *start_pfn, unsigned long *end_pfn)
+{
+	struct memblock_type *type = &memblock.memory;
+	int mid = memblock_search(type, (phys_addr_t)pfn << PAGE_SHIFT);
+

mm/memblock.c int __init_memblock memblock_is_memory(p · line 933

Quoted 10 lines

mm/memblock.c +10 -0

+	if (mid == -1)
+		return -1;
+
+	*start_pfn = type->regions[mid].base >> PAGE_SHIFT;
+	*end_pfn = (type->regions[mid].base + type->regions[mid].size)
+			>> PAGE_SHIFT;
+
+	return type->regions[mid].nid;
+}
+#endif

This function will have no callers if CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID=y. That's not too bad as the function is __init_memblock. But this depends on CONFIG_ARCH_DISCARD_MEMBLOCK. Messy :(

Yinghai Lu

author

2013-08-15 21:06 UTC #2 lore raw

Quoted 11 lines · Andrew Morton

Quoted 7 lines · On Wed, 14 Aug 2013 22:46:29 -0700 Yinghai Lu <[email protected]> wrote:
Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.
We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.
Looks nice. I wonder how much difference it makes.

Russ said he would test on his 256 nodes system, but looks he never got chance.

Thanks

Yinghai

Russ Anderson

2013-08-15 21:37 UTC #3 lore raw

Quoted 8 lines · Yinghai Lu

Quoted 6 lines · Andrew Morton
Quoted 4 lines · On Wed, 14 Aug 2013 22:46:29 -0700 Yinghai Lu <[email protected]> wrote:
Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.

Quoted 8 lines

Quoted 5 lines
Quoted 3 lines
We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.
Looks nice. I wonder how much difference it makes.
Russ said he would test on his 256 nodes system, but looks he never got chance.

I reserved time tonight on a couple big systems to measure the performance difference.

Thanks,

Signature · 8 lines

Russ Anderson, OS RAS/Partitioning Project Lead  
SGI - Silicon Graphics Inc          [email protected]

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Russ Anderson

Acked-by

2013-08-16 19:01 UTC #4 lore raw

In reply to Andrew Morton

Quoted 11 lines · Andrew Morton

Quoted 7 lines · On Wed, 14 Aug 2013 22:46:29 -0700 Yinghai Lu <[email protected]> wrote:
Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.
We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.
Looks nice. I wonder how much difference it makes.

Here are the timing differences for several machines.
In each case with the patch less time was spent in __early_pfn_to_nid().

                        3.11-rc5        with patch      difference (%)
                        --------        ----------      --------------
UV1: 256 nodes  9TB:     411.66          402.47         -9.19 (2.23%)
UV2: 255 nodes 16TB:    1141.02         1138.12         -2.90 (0.25%)
UV2:  64 nodes  2TB:     128.15          126.53         -1.62 (1.26%)
UV2:  32 nodes  2TB:     121.87          121.07         -0.80 (0.66%)

Time in seconds.

Acked-by Russ Anderson <[email protected]>

Quoted 44 lines

Quoted 28 lines

...

include/linux/memblock.h +2 -0

@@ -60,6 +60,8 @@ int memblock_reserve(phys_addr_t base, p
 void memblock_trim_memory(phys_addr_t align);
 
 #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn,
+			    unsigned long  *end_pfn);
 void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
 			  unsigned long *out_end_pfn, int *out_nid);

Index: linux-2.6/mm/memblock.c
===================================================================

mm/memblock.c +7 -0

@@ -914,6 +914,24 @@ int __init_memblock memblock_is_memory(p
 	return memblock_search(&memblock.memory, addr) != -1;
 }
 
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+int __init_memblock memblock_search_pfn_nid(unsigned long pfn,
+			 unsigned long *start_pfn, unsigned long *end_pfn)
+{
+	struct memblock_type *type = &memblock.memory;
+	int mid = memblock_search(type, (phys_addr_t)pfn << PAGE_SHIFT);
+

mm/memblock.c int __init_memblock memblock_is_memory(p · line 933

Quoted 10 lines

mm/memblock.c +10 -0

+	if (mid == -1)
+		return -1;
+
+	*start_pfn = type->regions[mid].base >> PAGE_SHIFT;
+	*end_pfn = (type->regions[mid].base + type->regions[mid].size)
+			>> PAGE_SHIFT;
+
+	return type->regions[mid].nid;
+}
+#endif

This function will have no callers if CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID=y. That's not too bad as the function is __init_memblock. But this depends on CONFIG_ARCH_DISCARD_MEMBLOCK. Messy :(

Signature · 8 lines

Russ Anderson, OS RAS/Partitioning Project Lead  
SGI - Silicon Graphics Inc          [email protected]

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Yinghai Lu

author

2013-08-16 19:15 UTC #5 lore raw

Quoted 17 lines · Russ Anderson

Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Wed, 14 Aug 2013 22:46:29 -0700 Yinghai Lu <[email protected]> wrote:
Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.
We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.
Looks nice. I wonder how much difference it makes.
Here are the timing differences for several machines.
In each case with the patch less time was spent in __early_pfn_to_nid().

Quoted 8 lines

                        3.11-rc5        with patch      difference (%)
                        --------        ----------      --------------
UV1: 256 nodes  9TB:     411.66          402.47         -9.19 (2.23%)
UV2: 255 nodes 16TB:    1141.02         1138.12         -2.90 (0.25%)
UV2:  64 nodes  2TB:     128.15          126.53         -1.62 (1.26%)
UV2:  32 nodes  2TB:     121.87          121.07         -0.80 (0.66%)

Time in seconds.

Thanks.

9T one have more entries in memblock.memory?

Yinghai

Russ Anderson

2013-08-16 19:31 UTC #6 lore raw

Quoted 27 lines · Yinghai Lu

Quoted 17 lines · Russ Anderson
Quoted 11 lines · Andrew Morton
Quoted 7 lines · On Wed, 14 Aug 2013 22:46:29 -0700 Yinghai Lu <[email protected]> wrote:
Current early_pfn_to_nid() on arch that support memblock go over memblock.memory one by one, so will take too many try near the end.
We can use existing memblock_search to find the node id for given pfn, that could save some time on bigger system that have many entries memblock.memory array.
Looks nice. I wonder how much difference it makes.
Here are the timing differences for several machines.
In each case with the patch less time was spent in __early_pfn_to_nid().
Quoted 8 lines
                        3.11-rc5        with patch      difference (%)
                        --------        ----------      --------------
UV1: 256 nodes  9TB:     411.66          402.47         -9.19 (2.23%)
UV2: 255 nodes 16TB:    1141.02         1138.12         -2.90 (0.25%)
UV2:  64 nodes  2TB:     128.15          126.53         -1.62 (1.26%)
UV2:  32 nodes  2TB:     121.87          121.07         -0.80 (0.66%)
Time in seconds.

Quoted 3 lines

Thanks.
9T one have more entries in memblock.memory?

Yes. It is older hardware with smaller DIMMs and more of them.

Signature · 8 lines

Russ Anderson, OS RAS/Partitioning Project Lead  
SGI - Silicon Graphics Inc          [email protected]

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>