mm: optimize PageHighMem() check
Subsystems (3)
PATCH 1/1 mm: optimize PageHighMem() check
This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.
PageHighMem(page)
is_highmem(page_zone(page))
zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones
Instead use is_highmem_idx() to work on zone_type available in page flags
----- Before -----
80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>
----- After -----
80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>
For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8-------./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56
---->8-------
e.g. For ARC kmap()
--- include/linux/page-flags.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/page-flags.h +1 -1
@@ -252,7 +252,7 @@ PAGEFLAG(Readahead, reclaim) TESTCLEARFLAG(Readahead, reclaim)* Must use a macro here due to header dependency issues. page_zone() is not * available at this point. */ -#define PageHighMem (__p) is_highmem(page_zone(__p)) +#define PageHighMem (__p) is_highmem_idx(page_zonenum(__p))#else PAGEFLAG_FALSE(HighMem)#endif
Signature · 7 lines
1.9.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 52 lines · Vineet Gupta
This came up when implementing HIHGMEM/PAE40 for ARC.
Thekmap()/kmap_atomic()generated code seemed needlessly bloated due to the wayPageHighMem()macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer thezone_type.
The pointer arithmatic in turn generates the code bloat.PageHighMem(page)
is_highmem(page_zone(page))zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zonesInstead use
is_highmem_idx()to work onzone_typeavailable in page flags----- Before ----- 80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>----- After ----- 80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8------- ./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934) function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56---->8-------
e.g. For ARC
kmap()
Quoted 9 lines
Cc Andrew Morton <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Hugh Dickins <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Michal Hocko <[email protected]>Cc Jennifer Herbert <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Signed-off-by Vineet Gupta <[email protected]>
Looks reasonably to me.
Quoted 19 lines
--- include/linux/page-flags.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/linux/page-flags.h +1
-1@@ -252,7 +252,7 @@ PAGEFLAG(Readahead, reclaim) TESTCLEARFLAG(Readahead, reclaim)* Must use a macro here due to header dependency issues. page_zone() is not * available at this point. */ -#define PageHighMem (__p) is_highmem(page_zone(__p)) +#define PageHighMem (__p) is_highmem_idx(page_zonenum(__p))#else PAGEFLAG_FALSE(HighMem)#endif Signature · 1 line
1.9.1
Signature · 2 lines
Michal Hocko SUSE Labs
Andrew Morton
maintainerQuoted 48 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
This came up when implementing HIHGMEM/PAE40 for ARC.
Thekmap()/kmap_atomic()generated code seemed needlessly bloated due to the wayPageHighMem()macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer thezone_type.
The pointer arithmatic in turn generates the code bloat.PageHighMem(page)
is_highmem(page_zone(page))zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zonesInstead use
is_highmem_idx()to work onzone_typeavailable in page flags----- Before ----- 80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>----- After ----- 80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8------- ./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934) function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56
Quoted 4 lines
---->8-------
e.g. For ARC
kmap()
is_highmem() is deranged. Can't we use a bit in zone->flags or something?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 52 lines · Andrew Morton
Quoted 50 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 50 lines
This came up when implementing HIHGMEM/PAE40 for ARC.
Thekmap()/kmap_atomic()generated code seemed needlessly bloated due to the wayPageHighMem()macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer thezone_type.
The pointer arithmatic in turn generates the code bloat.PageHighMem(page)
is_highmem(page_zone(page))zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zonesInstead use
is_highmem_idx()to work onzone_typeavailable in page flags----- Before ----- 80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>----- After ----- 80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8------- ./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934) function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56---->8-------
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
e.g. For ARC
kmap()
is_highmem()is deranged. Can't we use a bit inzone->flagsor something?
It won't be "a" bit since zone_type is an enum. However zone->flags could be split into 2 bitfields to hold enum zone_flags and enum zone_type.
However this patch still is independent of that since we have struct page as starting point and zone_type is available from there directly w/o monkeying around with any zone structs.
-Vineet
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerQuoted 53 lines · On Fri, 2 Oct 2015 12:45:53 +0530 Vineet Gupta <[email protected]> wrote:
Quoted 52 lines · Andrew Morton
Quoted 50 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 50 lines
This came up when implementing HIHGMEM/PAE40 for ARC.
Thekmap()/kmap_atomic()generated code seemed needlessly bloated due to the wayPageHighMem()macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer thezone_type.
The pointer arithmatic in turn generates the code bloat.PageHighMem(page)
is_highmem(page_zone(page))zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zonesInstead use
is_highmem_idx()to work onzone_typeavailable in page flags----- Before ----- 80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>----- After ----- 80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8------- ./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934) function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56---->8-------
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
e.g. For ARC
kmap()
is_highmem()is deranged. Can't we use a bit inzone->flagsor something?It won't be "a" bit since
zone_typeis an enum.
Yes it will!
static inline int is_highmem(struct zone *zone)
{
return test_bit(ZONE_HIGHMEM, &zone->flags);
}Quoted 5 lines
...
However this patch still is independent of that since we have struct page as starting point and
zone_typeis available from there directly w/o monkeying around with any zone structs.
yup.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 59 lines · Andrew Morton
Quoted 57 lines · On Fri, 2 Oct 2015 12:45:53 +0530 Vineet Gupta <[email protected]> wrote:
Quoted 56 lines · Andrew Morton
Quoted 50 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 50 lines
This came up when implementing HIHGMEM/PAE40 for ARC.
Thekmap()/kmap_atomic()generated code seemed needlessly bloated due to the wayPageHighMem()macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer thezone_type.
The pointer arithmatic in turn generates the code bloat.PageHighMem(page)
is_highmem(page_zone(page))zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zonesInstead use
is_highmem_idx()to work onzone_typeavailable in page flags----- Before ----- 80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>----- After ----- 80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8------- ./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934) function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56---->8-------
Quoted 2 lines
Quoted 2 lines
e.g. For ARC
kmap()
is_highmem()is deranged. Can't we use a bit inzone->flagsor something?
Quoted 7 lines
Quoted 1 line
It won't be "a" bit since
zone_typeis an enum.Yes it will!
static inline int
is_highmem(struct zone *zone){ return test_bit(ZONE_HIGHMEM, &zone->flags); }
Point is do we want to fix this specific case, or do we want to improve zone_idx() in general.
#define zone_idx(zone) ((zone) - (zone)->zone_pgdat->node_zones
If former, I can split up zone->flags into 31:1 bit-field. Otherwise I will split it into 24:8 to hold both zone_flags and zone_type
Quoted 6 lines
Quoted 5 lines
...
However this patch still is independent of that since we have struct page as starting point and
zone_typeis available from there directly w/o monkeying around with any zone structs.yup.
Do u want me to carry this as part of the the ARC HIGHMEM/PAE40 patchset or would you rather prefer merging it via mm tree.
Thx,
-Vineet
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 67 lines · Vineet Gupta
Quoted 59 lines · Andrew Morton
Quoted 59 lines · On Fri, 2 Oct 2015 12:45:53 +0530 Vineet Gupta <[email protected]> wrote:
Quoted 58 lines
Quoted 58 lines · Andrew Morton
Quoted 56 lines
Quoted 56 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 51 lines
Quoted 51 lines
Quoted 51 lines
Quoted 51 lines
This came up when implementing HIHGMEM/PAE40 for ARC.
Thekmap()/kmap_atomic()generated code seemed needlessly bloated due to the wayPageHighMem()macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer thezone_type.
The pointer arithmatic in turn generates the code bloat.PageHighMem(page)
is_highmem(page_zone(page))zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zonesInstead use
is_highmem_idx()to work onzone_typeavailable in page flags----- Before ----- 80756348: mov_s r13,r0 8075634a: ld_s r2,[r13,0] 8075634c: lsr_s r2,r2,30 8075634e: mpy r2,r2,0x2a4 80756352: add_s r2,r2,0x80aef880 80756358: ld_s r3,[r2,28] 8075635a: sub_s r2,r2,r3 8075635c: breq r2,0x2a4,80756378 <kmap+0x48> 80756364: breq r2,0x548,80756378 <kmap+0x48>----- After ----- 80756330: mov_s r13,r0 80756332: ld_s r2,[r13,0] 80756334: lsr_s r2,r2,30 80756336: sub_s r2,r2,1 80756338: brlo r2,2,80756348 <kmap+0x30>For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.
---->8------- ./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934) function old new delta saveable_page 162 154 -8 saveable_highmem_page 154 146 -8 skb_gro_reset_offset 147 131 -16 ... ... __change_page_attr_set_clr 1715 1678 -37 setup_data_read 434 394 -40 mon_bin_event 1967 1927 -40 swsusp_save 1148 1105 -43 _set_pages_array 549 493 -56---->8-------
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
e.g. For ARC
kmap()
is_highmem()is deranged. Can't we use a bit inzone->flagsor something?Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
It won't be "a" bit since
zone_typeis an enum.Yes it will!
static inline int
is_highmem(struct zone *zone){ return test_bit(ZONE_HIGHMEM, &zone->flags); }
Quoted 7 lines
Point is do we want to fix this specific case, or do we want to improve
zone_idx()in general.#define
zone_idx(zone)((zone) - (zone)->zone_pgdat->node_zonesIf former, I can split up
zone->flagsinto 31:1 bit-field. Otherwise I will split it into 24:8 to hold bothzone_flagsandzone_type
Andrew, what do u think. Do we need to shoehorn type (idx) and flags into same placeholder or simply add a new zone_idx field.
There will be slight complications with former to avoid any possible collisions when someone calls set_bit() for flags holder bitfield.
IMHO zone struct is already so bloated, adding another word would be OK specially when number of zone structures is not too many !
-Vineet