mm: simplify zone_idx()
PATCH 1/1 mm: simplify zone_idx()
store zone_idx directly in struct zone to simplify and optimize zone_idx()
--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
include/linux/mmzone.h +2 -1
@@ -941,6 +941,7 @@ struct zone {#endif const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /*
@@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id) { return node_id; };/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the ZONE_NORMAL zone, etc. */ -#define zone_idx (zone) ((zone) - (zone)->zone_pgdat->node_zones) +#define zone_idx (zone) ((zone)->zone_idx)#ifdef CONFIG_ZONE_DEVICE static inline bool zone_is_zone_device (struct zone *zone)
mm/mm_init.c +1 -0
@@ -1425,6 +1425,7 @@ static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx,atomic_long_set (&zone->managed_pages, remaining_pages);zone_set_nid (zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat =NODE_DATA (nid);spin_lock_init (&zone->lock);zone_seqlock_init (zone);
Signature · 1 line
2.17.1
Mike Rapoport
maintainerHi,
Quoted 1 line · gaoxu
store
zone_idxdirectly in struct zone to simplify and optimizezone_idx()
Do you see an actual speed up somewhere?
Quoted 41 lines
Signed-off-by gao xu <[email protected]>--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/mmzone.h +2
-1@@ -941,6 +941,7 @@ struct zone {#endif const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /* @@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id) { return node_id; };/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the ZONE_NORMAL zone, etc. */ -#define zone_idx (zone) ((zone) - (zone)->zone_pgdat->node_zones) +#define zone_idx (zone) ((zone)->zone_idx)#ifdef CONFIG_ZONE_DEVICE static inline bool zone_is_zone_device (struct zone *zone)mm/mm_init.c +1
-0@@ -1425,6 +1425,7 @@ static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx,atomic_long_set (&zone->managed_pages, remaining_pages);zone_set_nid (zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat =NODE_DATA (nid);spin_lock_init (&zone->lock);zone_seqlock_init (zone);Signature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours, Mike.
Quoted 7 lines · Mike Rapoport
Hi,
Quoted 1 line · gaoxu
store
zone_idxdirectly in struct zone to simplify and optimizezone_idx()Do you see an actual speed up somewhere?
+1. Curious if there's data indicating zone_idx is a hot path.
Quoted 38 lines
Quoted 38 lines
Signed-off-by gao xu <[email protected]>--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/mmzone.h +2
-1@@ -941,6 +941,7 @@ struct zone {#endif const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /* @@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id) { return node_id; };/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the ZONE_NORMAL zone, etc. */ -#define zone_idx (zone) ((zone) - (zone)->zone_pgdat->node_zones) +#define zone_idx (zone) ((zone)->zone_idx)#ifdef CONFIG_ZONE_DEVICE static inline bool zone_is_zone_device (struct zone *zone)mm/mm_init.c +1
-0@@ -1425,6 +1425,7 @@ static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx,atomic_long_set (&zone->managed_pages, remaining_pages);zone_set_nid (zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat =NODE_DATA (nid);
Quoted 8 lines
Quoted 4 lines
mm/mm_init.c +0
-1spin_lock_init (&zone->lock);zone_seqlock_init (zone); --2.17.1
--
Sincerely yours,
Mike.
Thanks
Barry
Quoted 9 lines · Mike Rapoport
Quoted 7 lines
Hi,
Quoted 1 line · gaoxu
store
zone_idxdirectly in struct zone to simplify and optimizezone_idx()Do you see an actual speed up somewhere?
Almost negligible. my simple code tests showed the patch provides an average improvement of ~0.02%. Thus, in the Android 15-6.6 kernel, I confidently retained the original zone_idx function.
(https://android-review.googlesource.com/c/kernel/common/+/3578322/2/mm/page_alloc.c#770)
This patch only eliminates 2-3 assembly instructions, making it challenging to observe measurable performance benefits.
However, since the zone struct includes CACHELINE_PADDING (reserving unused space), adding a new member variable does not alter the size of zone. This makes the patch effectively zero-cost while achieving a cleaner implementation of zone_idx.
Quoted 2 lines
+1. Curious if there's data indicating
zone_idxis a hot path.
There are several functions in the memory management code that are frequently executed and will call zone_idx:
rmqueue()->wakeup_kswapd()->zone_idx() alloc_pages_bulk_noprof()->__count_zid_vm_events()->zone_idx()
The patch (https://lore.kernel.org/all/[email protected]/)
will add new hotspot paths, with the details as follows:
__zone_watermark_ok()->zone_is_suitable()->zone_idx() zone_watermark_fast()->zone_is_suitable()->zone_idx() get_page_from_freelist()->zone_is_suitable()->zone_idx() __free_one_page()->zone_max_order()->zone_idx()
Although The patch (https://lore.kernel.org/all/[email protected]/)
has not yet merged into the Linux mainline; it is already included in Android 15-6.6.
Quoted 55 lines
Quoted 16 lines
Quoted 16 lines
Signed-off-by gao xu <[email protected]>--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/mmzone.h +0
-0@@ -941,6 +941,7 @@ struct zone {#endif Quoted 6 lines
include/linux/mmzone.h static inline int local_memory_node(int node_id)
Quoted 6 lines
include/linux/mmzone.h +1
-0const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /* @@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id){ return
node_id; };Quoted 2 lines
include/linux/mmzone.h static inline int local_memory_node(int node_id) · line 1538
Quoted 2 lines
include/linux/mmzone.h +0
-0/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the ZONE_NORMAL zone, etc.
Quoted 2 lines
include/linux/mmzone.h static inline int local_memory_node(int node_id) · line 1539
Quoted 2 lines
include/linux/mmzone.h +0
-1*/ -#define zone_idx (zone) ((zone) -(zone)->zone_pgdat->node_zones)
Quoted 9 lines
mm/mm_init.c static void __meminit zone_init_internals(struct
Quoted 9 lines
include/linux/mmzone.h +1
-0+#define zone_idx (zone) ((zone)->zone_idx)#ifdef CONFIG_ZONE_DEVICE static inline bool zone_is_zone_device (struct zone *zone)mm/mm_init.c +0
-0@@ -1425,6 +1425,7 @@ static void __meminit zone_init_internals(structzone *zone, enum
zone_typeidx,Quoted 10 lines
Quoted 9 lines
mm/mm_init.c +1
-1atomic_long_set (&zone->managed_pages, remaining_pages);zone_set_nid (zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat =NODE_DATA (nid);spin_lock_init (&zone->lock);zone_seqlock_init (zone); --2.17.1
Quoted 3 lines
--
Sincerely yours,
Mike.Thanks
Barry
Quoted 14 lines · gaoxu
Quoted 10 lines · Mike Rapoport
Quoted 7 lines
Hi,
Quoted 1 line · gaoxu
store
zone_idxdirectly in struct zone to simplify and optimizezone_idx()Do you see an actual speed up somewhere?
Almost negligible. my simple code tests showed the patch provides an average improvement of ~0.02%. Thus, in the Android 15-6.6 kernel, I confidently retained the original
zone_idxfunction.
(https://android-review.googlesource.com/c/kernel/common/+/3578322/2/mm/page_alloc.c#770)
Quoted 5 lines
This patch only eliminates 2-3 assembly instructions, making it challenging to observe measurable performance benefits.
However, since the zone struct includesCACHELINE_PADDING(reserving unused space), adding a new member variable does not alter the size of zone. This makes the patch effectively zero-cost while achieving a cleaner implementation ofzone_idx.
The struct zone contains many CONFIG_ options to include or exclude certain fields.
If we're confident that our new zone_idx doesn't increase cacheline usage for all those
cases, this seems like a worthwhile cleanup. Have you checked the numbers?
Quoted 14 lines
Quoted 2 lines
+1. Curious if there's data indicating
zone_idxis a hot path.There are several functions in the memory management code that are frequently executed and will call
zone_idx:
rmqueue()->wakeup_kswapd()->zone_idx() alloc_pages_bulk_noprof()->__count_zid_vm_events()->zone_idx()The patch (https://lore.kernel.org/all/[email protected]/)
will add new hotspot paths, with the details as follows:
__zone_watermark_ok()->zone_is_suitable()->zone_idx() zone_watermark_fast()->zone_is_suitable()->zone_idx() get_page_from_freelist()->zone_is_suitable()->zone_idx() __free_one_page()->zone_max_order()->zone_idx()
Quoted 55 lines
Although The patch (https://lore.kernel.org/all/[email protected]/)
has not yet merged into the Linux mainline; it is already included in Android 15-6.6.Quoted 53 lines
Quoted 16 lines
Quoted 16 lines
Signed-off-by gao xu <[email protected]>--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/mmzone.h +0
-0@@ -941,6 +941,7 @@ struct zone {#endif Quoted 6 lines
include/linux/mmzone.h static inline int local_memory_node(int node_id)
Quoted 6 lines
include/linux/mmzone.h +1
-0const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /* @@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id){ return
node_id; };Quoted 2 lines
include/linux/mmzone.h static inline int local_memory_node(int node_id) · line 1538
Quoted 2 lines
include/linux/mmzone.h +0
-0/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the ZONE_NORMAL zone, etc.
Quoted 2 lines
include/linux/mmzone.h static inline int local_memory_node(int node_id) · line 1539
Quoted 2 lines
include/linux/mmzone.h +0
-1*/ -#define zone_idx (zone) ((zone) -(zone)->zone_pgdat->node_zones)
Quoted 9 lines
mm/mm_init.c static void __meminit zone_init_internals(struct
Quoted 9 lines
include/linux/mmzone.h +1
-0+#define zone_idx (zone) ((zone)->zone_idx)#ifdef CONFIG_ZONE_DEVICE static inline bool zone_is_zone_device (struct zone *zone)mm/mm_init.c +0
-0@@ -1425,6 +1425,7 @@ static void __meminit zone_init_internals(structzone *zone, enum
zone_typeidx,Quoted 13 lines
Quoted 9 lines
mm/mm_init.c +1
-1atomic_long_set (&zone->managed_pages, remaining_pages);zone_set_nid (zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat =NODE_DATA (nid);spin_lock_init (&zone->lock);zone_seqlock_init (zone); --2.17.1
--
Sincerely yours,
Mike.
Thanks
Barry
Quoted 25 lines · gaoxu
Quoted 4 lines
Quoted 4 lines · Mike Rapoport
Quoted 1 line
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Hi,
Quoted 2 lines · gaoxu
store
zone_idxdirectly in struct zone to simplify and optimizezone_idx()Do you see an actual speed up somewhere?
Almost negligible. my simple code tests showed the patch provides an average
improvement of ~0.02%.
Quoted 1 line
Thus, in the Android 15-6.6 kernel, I confidently retained the original
zone_idxfunction.
Quoted 8 lines
(https://android-review.googlesource.com/c/kernel/common/+/3578322/2/m
m/page_alloc.c#770)This patch only eliminates 2-3 assembly instructions, making it challenging to observe measurable performance benefits. However, since the zone struct includes
CACHELINE_PADDING(reserving unused space), adding a new member variable does not alter the size of zone. This makes the patch effectively zero-cost while achieving a cleaner
Quoted 7 lines
implementation of
zone_idx.The struct zone contains many
CONFIG_options to include or exclude certain fields.
If we're confident that our newzone_idxdoesn't increase cacheline usage for all those cases, this seems like a worthwhile cleanup. Have you checked the numbers?
The zone info obtained through T32 in the Android 15-6.6 system(arm64):
(struct zone) struct (1664 bytes,
[0] unsigned long [4] _watermark,
[32] unsigned long watermark_boost,
[40] unsigned long nr_reserved_highatomic,
[48] long [5] lowmem_reserve,
[88] struct pglist_data * zone_pgdat,
[96] struct per_cpu_pages * per_cpu_pageset,
[104] struct per_cpu_zonestat * per_cpu_zonestats,
[112] int pageset_high,
[116] int pageset_batch,
[120] unsigned long zone_start_pfn,
[128] atomic_long_t managed_pages,
[136] unsigned long spanned_pages,
[144] unsigned long present_pages,
[152] unsigned long present_early_pages,
[160] unsigned long cma_pages,
[168] const char * name,
[176] unsigned long nr_isolate_pageblock,
[184] seqlock_t span_seqlock,
[192] int order,
[196] int initialized,
[256] struct cacheline_padding _pad1_,
[256] struct free_area [11] free_area, [1400] unsigned long flags,
[1408] spinlock_t lock,
[1472] struct cacheline_padding _pad2_, [1472] unsigned long percpu_drift_mark, [1480] unsigned long compact_cached_free_pfn, [1488] unsigned long [2] compact_cached_migrate_pfn, [1504] unsigned long compact_init_migrate_pfn, [1512] unsigned long compact_init_free_pfn, [1520] unsigned int compact_considered, [1524] unsigned int compact_defer_shift, [1528] int compact_order_failed,
[1532] bool compact_blockskip_flush,
[1533] bool contiguous,
[1536] struct cacheline_padding _pad3_, [1536] atomic_long_t [11] vm_stat,
[1624] atomic_long_t [0] vm_numa_event)
1) It can be observed that there are 56B of free space in CACHELINE_PADDING(pad1);
2) Before the variables in CACHELINE_PADDING(pad1), there are two CONFIGs that are not enabled in Android 15-6.6:
#ifdef CONFIG_NUMA int node; #endif #ifndef CONFIG_SPARSEMEM unsigned long *pageblock_flags; #endif /* CONFIG_SPARSEMEM */ These two CONFIGs occupy 16B.
3) Compared to the latest Linux code, two new variables, unsigned long nr_free_highatomic and int pageset_high_max, occupy an additional 16B; Based on the above analysis, there are still 24B of free space before CACHELINE_PADDING(pad1).
(If I misunderstand, please point it out in a timely manner. Thank you!)
It would be more appropriate to place the newly added variable zone_idx before initialized.
Quoted 78 lines
Quoted 16 lines
Quoted 3 lines
+1. Curious if there's data indicating
zone_idxis a hot path.There are several functions in the memory management code that are frequently executed and will call
zone_idx:
rmqueue()->wakeup_kswapd()->zone_idx() alloc_pages_bulk_noprof()->__count_zid_vm_events()->zone_idx()The patch
(https://lore.kernel.org/all/[email protected]
m/) will add new hotspot paths, with the details as follows:
__zone_watermark_ok()->zone_is_suitable()->zone_idx() zone_watermark_fast()->zone_is_suitable()->zone_idx() get_page_from_freelist()->zone_is_suitable()->zone_idx() __free_one_page()->zone_max_order()->zone_idx()Quoted 59 lines
Although The patch
(https://lore.kernel.org/all/[email protected]
m/) has not yet merged into the Linux mainline; it is already included in Android 15-6.6.Quoted 55 lines
Quoted 15 lines
Quoted 15 lines
Signed-off-by gao xu <[email protected]>--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/mmzone.h +0
-0@@ -941,6 +941,7 @@ struct zone { #endifQuoted 7 lines
include/linux/mmzone.h static inline int local_memory_node(int · line 1537
Quoted 7 lines
include/linux/mmzone.h +1
-0const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /* @@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id){ return
node_id; };Quoted 3 lines
Quoted 3 lines
include/linux/mmzone.h +0
-0/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the
ZONE_NORMALzone, etc.
Quoted 2 lines
Quoted 2 lines
*/
-#definezone_idx(zone)((zone) -(zone)->zone_pgdat->node_zones)
Quoted 10 lines
Quoted 10 lines
+#define
zone_idx(zone)((zone)->zone_idx)#ifdef
CONFIG_ZONE_DEVICE
static inline boolzone_is_zone_device(struct zone *zone)diff
--gita/mm/mm_init.cb/mm/mm_init.cindex9659689b8..a7f7264f1100644mm/mm_init.c +0
-0@@ -1425,6 +1425,7 @@ static void __meminitzone_init_internals(struct
zone *zone, enum
zone_typeidx,Quoted 13 lines
Quoted 9 lines
atomic_long_set(&zone->managed_pages, remaining_pages); zone_set_nid(zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat = NODE_DATA(nid); spin_lock_init(&zone->lock); zone_seqlock_init(zone);--
2.17.1--
Sincerely yours,
Mike.Thanks
Barry
Quoted 85 lines · gaoxu
Quoted 26 lines · gaoxu
Quoted 4 lines
Quoted 4 lines · Mike Rapoport
Quoted 1 line
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Hi,
Quoted 2 lines · gaoxu
store
zone_idxdirectly in struct zone to simplify and optimizezone_idx()Do you see an actual speed up somewhere?
Almost negligible. my simple code tests showed the patch provides an average
improvement of ~0.02%.
Quoted 1 line
Thus, in the Android 15-6.6 kernel, I confidently retained the original
zone_idxfunction.
Quoted 8 lines
(https://android-review.googlesource.com/c/kernel/common/+/3578322/2/m
m/page_alloc.c#770)This patch only eliminates 2-3 assembly instructions, making it challenging to observe measurable performance benefits. However, since the zone struct includes
CACHELINE_PADDING(reserving unused space), adding a new member variable does not alter the size of zone. This makes the patch effectively zero-cost while achieving a cleanerQuoted 7 lines
implementation of
zone_idx.The struct zone contains many
CONFIG_options to include or exclude certain fields.
If we're confident that our newzone_idxdoesn't increase cacheline usage for all those cases, this seems like a worthwhile cleanup. Have you checked the numbers?The zone info obtained through T32 in the Android 15-6.6 system(arm64):
(struct zone) struct (1664 bytes,
[0] unsigned long [4]_watermark,
[32] unsigned longwatermark_boost,
[40] unsigned longnr_reserved_highatomic,
[48] long [5]lowmem_reserve,
[88] structpglist_data*zone_pgdat,
[96] structper_cpu_pages*per_cpu_pageset,
[104] structper_cpu_zonestat*per_cpu_zonestats,
[112] intpageset_high,
[116] intpageset_batch,
[120] unsigned longzone_start_pfn,
[128]atomic_long_tmanaged_pages,
[136] unsigned longspanned_pages,
[144] unsigned longpresent_pages,
[152] unsigned longpresent_early_pages,
[160] unsigned longcma_pages,
[168] const char * name,
[176] unsigned longnr_isolate_pageblock,
[184]seqlock_tspan_seqlock,
[192] int order,
[196] int initialized,
[256] structcacheline_padding_pad1_,
[256] structfree_area[11]free_area, [1400] unsigned long flags,
[1408]spinlock_tlock,
[1472] structcacheline_padding_pad2_, [1472] unsigned longpercpu_drift_mark, [1480] unsigned longcompact_cached_free_pfn, [1488] unsigned long [2]compact_cached_migrate_pfn, [1504] unsigned longcompact_init_migrate_pfn, [1512] unsigned longcompact_init_free_pfn, [1520] unsigned intcompact_considered, [1524] unsigned intcompact_defer_shift, [1528] intcompact_order_failed,
[1532] boolcompact_blockskip_flush,
[1533] bool contiguous,
[1536] structcacheline_padding_pad3_, [1536]atomic_long_t[11]vm_stat,
[1624]atomic_long_t[0]vm_numa_event)1) It can be observed that there are 56B of free space in CACHELINE_PADDING(pad1);
2) Before the variables inCACHELINE_PADDING(pad1), there are two CONFIGs that are not enabled in Android 15-6.6:#ifdef CONFIG_NUMA int node; #endif #ifndef CONFIG_SPARSEMEM unsigned long *pageblock_flags; #endif /* CONFIG_SPARSEMEM */ These two CONFIGs occupy 16B.
Quoted 5 lines
3) Compared to the latest Linux code, two new variables, unsigned long
nr_free_highatomicand intpageset_high_max, occupy an additional 16B; Based on the above analysis, there are still 24B of free space beforeCACHELINE_PADDING(pad1).
(If I misunderstand, please point it out in a timely manner. Thank you!)It would be more appropriate to place the newly added variable
zone_idxbefore initialized.
I don't have a strong opinion on whether we need zone_idx—I'm okay with having it or not. If you'd like to add it, feel free to send out a v2 noting that it doesn't increase the struct size. If no one objects, it might be a nice cleanup.
Quoted 71 lines
Quoted 71 lines
Quoted 71 lines
Quoted 3 lines
+1. Curious if there's data indicating
zone_idxis a hot path.There are several functions in the memory management code that are frequently executed and will call
zone_idx:
rmqueue()->wakeup_kswapd()->zone_idx() alloc_pages_bulk_noprof()->__count_zid_vm_events()->zone_idx()The patch
(https://lore.kernel.org/all/[email protected]
m/) will add new hotspot paths, with the details as follows:
__zone_watermark_ok()->zone_is_suitable()->zone_idx() zone_watermark_fast()->zone_is_suitable()->zone_idx() get_page_from_freelist()->zone_is_suitable()->zone_idx() __free_one_page()->zone_max_order()->zone_idx()Although The patch
(https://lore.kernel.org/all/[email protected]
m/) has not yet merged into the Linux mainline; it is already included in Android 15-6.6.Quoted 51 lines
Quoted 15 lines
Quoted 15 lines
Signed-off-by gao xu <[email protected]>--- include/linux/mmzone.h | 3 ++- mm/mm_init.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)include/linux/mmzone.h +0
-0@@ -941,6 +941,7 @@ struct zone { #endifQuoted 7 lines
include/linux/mmzone.h static inline int local_memory_node(int · line 1537
Quoted 7 lines
include/linux/mmzone.h +1
-0const char *name; +enum zone_type zone_idx;#ifdef CONFIG_MEMORY_ISOLATION /* @@ -1536,7 +1537,7 @@ static inline int local_memory_node(int node_id){ return
node_id; };Quoted 3 lines
Quoted 3 lines
include/linux/mmzone.h +0
-0/* * zone_idx() returns 0 for the ZONE_DMA zone, 1 for the
ZONE_NORMALzone, etc.
Quoted 2 lines
Quoted 2 lines
*/
-#definezone_idx(zone)((zone) -(zone)->zone_pgdat->node_zones)
Quoted 10 lines
Quoted 10 lines
+#define
zone_idx(zone)((zone)->zone_idx)#ifdef
CONFIG_ZONE_DEVICE
static inline boolzone_is_zone_device(struct zone *zone)diff
--gita/mm/mm_init.cb/mm/mm_init.cindex9659689b8..a7f7264f1100644mm/mm_init.c +0
-0@@ -1425,6 +1425,7 @@ static void __meminitzone_init_internals(struct
zone *zone, enum
zone_typeidx,Quoted 10 lines
Quoted 9 lines
atomic_long_set(&zone->managed_pages, remaining_pages); zone_set_nid(zone, nid); zone->name = zone_names[idx]; + zone->zone_idx = idx; zone->zone_pgdat = NODE_DATA(nid); spin_lock_init(&zone->lock); zone_seqlock_init(zone);--
2.17.1
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
--
Sincerely yours,
Mike.
Thanks
Barry
Quoted 8 lines · Barry Song
Quoted 2 lines · gaoxu
The zone info obtained through T32 in the Android 15-6.6 system(arm64):
(struct zone) struct (1664 bytes,I don't have a strong opinion on whether we need
zone_idx—I'm okay with having it or not. If you'd like to add it, feel free to send out a v2 noting that it doesn't increase the struct size. If no one objects, it might be a nice cleanup.
Plus it's already 1664 bytes! And we have, what, 4 zones per NUMA node? Growing it doesn't feel like a big deal. Although "saves two assembly instructions" is also not exactly a big win. If it saved a cacheline reference, that might be more interesting, but it seems like it's more likely to introduce a cacheline reference than save one. Maybe just not worth doing?
Quoted 11 lines · Barry Song
Quoted 8 lines · gaoxu
Quoted 2 lines
The zone info obtained through T32 in the Android 15-6.6 system(arm64):
(struct zone) struct (1664 bytes,I don't have a strong opinion on whether we need
zone_idx—I'm okay with having it or not. If you'd like to add it, feel free to send out a v2 noting that it doesn't increase the struct size. If no one objects, it might be a nice cleanup.
Quoted 5 lines
Plus it's already 1664 bytes! And we have, what, 4 zones per NUMA node? Growing it doesn't feel like a big deal. Although "saves two assembly instructions" is also not exactly a big win. If it saved a cacheline reference, that might be more interesting, but it seems like it's more likely to introduce a cacheline reference than save one. Maybe just not worth doing?
Zone, zone_pgdat, and node_zones are all considered hot data; most of the time, they reside in the cache. In contrast, zone_idx in the patch is not hot data, and executing ((zone)->zone_idx) will add a new cache line.
Am I understanding this correctly?
If the heat of the zone_idx function increases, this modification will become worthwhile. For example, this patch will increase the heat of the zone_idx function. The patch (https://lore.kernel.org/all/[email protected]/)
will add new hotspot paths, with the details as follows:
__zone_watermark_ok()->zone_is_suitable()->zone_idx() zone_watermark_fast()->zone_is_suitable()->zone_idx() get_page_from_freelist()->zone_is_suitable()->zone_idx() __free_one_page()->zone_max_order()->zone_idx()
Quoted 9 lines · gaoxu
Quoted 4 lines
Growing it doesn't feel like a big deal. Although "saves two assembly instructions" is also not exactly a big win. If it saved a cacheline reference, that might be more interesting, but it seems like it's more likely to introduce a cacheline reference than save one. Maybe just not worth doing?
Zone,
zone_pgdat, andnode_zonesare all considered hot data; most of the time, they reside in the cache. In contrast,zone_idxin the patch is not hot data, and executing ((zone)->zone_idx) will add a new cache line.
Am I understanding this correctly?
CPUs are not limited in the number of instructions they can execute today. For example the ARM X4 can execute 8 instructions per clock. Most of the time most of the CPU is idle, waiting on cache misses. A cache miss (all the way to memory) is about 100ns, so if said CPU is clocked at 3.4GHz, that's 2700 instructions that could be executed instead of waiting for that cacheline. Other top-end CPUs have similar numbers; you can find weak CPUs out there which have smaller ratios, but we don't tend to optimise for low-end CPUs.
Therefore it is more important to avoid cacheline misses than it is to reduce the number of instructions executed. You haven't measured any improvement from your patch, so I think we should defaut to not changing anything.