Skip to contentSkip to conversation

mm: optimize PageHighMem() check

PATCH In review Vineet Gupta 4 participants 7 messages Sep 29, 2015 – Nov 6, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm: optimize PageHighMem() check

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

----- Before -----

80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>

----- After -----

80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)

function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56

---->8-------

e.g. For ARC kmap()

Cc Andrew Morton <[email protected]>
Cc Naoya Horiguchi <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc "Kirill A. Shutemov" <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Jennifer Herbert <[email protected]>
Cc Konstantin Khlebnikov <[email protected]>
---
 include/linux/page-flags.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/page-flags.h +1 -1
index 41c93844fb1d..2953aaa06d67 100644
@@ -252,7 +252,7 @@ PAGEFLAG(Readahead, reclaim) TESTCLEARFLAG(Readahead, reclaim)
  * Must use a macro here due to header dependency issues. page_zone() is not
  * available at this point.
  */
-#define PageHighMem(__p) is_highmem(page_zone(__p))
+#define PageHighMem(__p) is_highmem_idx(page_zonenum(__p))
 #else
 PAGEFLAG_FALSE(HighMem)
 #endif
Signature · 7 lines
1.9.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Acked-by on v1 1/1
#1 lore raw
Quoted 52 lines · Vineet Gupta

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

   ----- Before -----
80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>
   ----- After  -----
80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

   ---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post
add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56

---->8-------

e.g. For ARC kmap()

Quoted 9 lines
Cc Andrew Morton <[email protected]>
Cc Naoya Horiguchi <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc "Kirill A. Shutemov" <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Jennifer Herbert <[email protected]>
Cc Konstantin Khlebnikov <[email protected]>

Looks reasonably to me.

Quoted 19 lines
---
 include/linux/page-flags.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/page-flags.h +1 -1
index 41c93844fb1d..2953aaa06d67 100644
@@ -252,7 +252,7 @@ PAGEFLAG(Readahead, reclaim) TESTCLEARFLAG(Readahead, reclaim)
  * Must use a macro here due to header dependency issues. page_zone() is not
  * available at this point.
  */
-#define PageHighMem(__p) is_highmem(page_zone(__p))
+#define PageHighMem(__p) is_highmem_idx(page_zonenum(__p))
 #else
 PAGEFLAG_FALSE(HighMem)
 #endif
Signature · 1 line
1.9.1
Signature · 2 lines
Michal Hocko
SUSE Labs
maintainer
#2 lore raw
In reply to Vineet Gupta
Quoted 48 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

   ----- Before -----
80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>
   ----- After  -----
80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

   ---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post
add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56
Quoted 4 lines

---->8-------

e.g. For ARC kmap()

is_highmem() is deranged. Can't we use a bit in zone->flags or something?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw
Quoted 52 lines · Andrew Morton
Quoted 50 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 50 lines

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

   ----- Before -----
80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>
   ----- After  -----
80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

   ---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post
add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56

---->8-------

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

e.g. For ARC kmap()

is_highmem() is deranged. Can't we use a bit in zone->flags or something?

It won't be "a" bit since zone_type is an enum. However zone->flags could be split into 2 bitfields to hold enum zone_flags and enum zone_type.
However this patch still is independent of that since we have struct page as starting point and zone_type is available from there directly w/o monkeying around with any zone structs.

-Vineet

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

maintainer
#4 lore raw
Quoted 53 lines · On Fri, 2 Oct 2015 12:45:53 +0530 Vineet Gupta <[email protected]> wrote:
Quoted 52 lines · Andrew Morton
Quoted 50 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 50 lines

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

   ----- Before -----
80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>
   ----- After  -----
80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

   ---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post
add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56

---->8-------

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

e.g. For ARC kmap()

is_highmem() is deranged. Can't we use a bit in zone->flags or something?

It won't be "a" bit since zone_type is an enum.

Yes it will!

static inline int is_highmem(struct zone *zone)

{
	return test_bit(ZONE_HIGHMEM, &zone->flags);
}
Quoted 5 lines

...

However this patch still is independent of that since we have struct page as starting point and zone_type is available from there directly w/o monkeying around with any zone structs.

yup.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#5 lore raw
Quoted 59 lines · Andrew Morton
Quoted 57 lines · On Fri, 2 Oct 2015 12:45:53 +0530 Vineet Gupta <[email protected]> wrote:
Quoted 56 lines · Andrew Morton
Quoted 50 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 50 lines

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

   ----- Before -----
80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>
   ----- After  -----
80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

   ---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post
add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56

---->8-------

Quoted 2 lines
Quoted 2 lines

e.g. For ARC kmap()

is_highmem() is deranged. Can't we use a bit in zone->flags or something?

Quoted 7 lines
Quoted 1 line

It won't be "a" bit since zone_type is an enum.

Yes it will!

static inline int is_highmem(struct zone *zone)

{
	return test_bit(ZONE_HIGHMEM, &zone->flags);
}

Point is do we want to fix this specific case, or do we want to improve zone_idx() in general.

#define zone_idx(zone) ((zone) - (zone)->zone_pgdat->node_zones

If former, I can split up zone->flags into 31:1 bit-field. Otherwise I will split it into 24:8 to hold both zone_flags and zone_type

Quoted 6 lines
Quoted 5 lines

...

However this patch still is independent of that since we have struct page as starting point and zone_type is available from there directly w/o monkeying around with any zone structs.

yup.

Do u want me to carry this as part of the the ARC HIGHMEM/PAE40 patchset or would you rather prefer merging it via mm tree.

Thx,
-Vineet

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw
Quoted 67 lines · Vineet Gupta
Quoted 59 lines · Andrew Morton
Quoted 59 lines · On Fri, 2 Oct 2015 12:45:53 +0530 Vineet Gupta <[email protected]> wrote:
Quoted 58 lines
Quoted 58 lines · Andrew Morton
Quoted 56 lines
Quoted 56 lines · On Tue, 29 Sep 2015 13:24:20 +0530 Vineet Gupta <[email protected]>…
Quoted 51 lines
Quoted 51 lines
Quoted 51 lines
Quoted 51 lines

This came up when implementing HIHGMEM/PAE40 for ARC.
The kmap() / kmap_atomic() generated code seemed needlessly bloated due to the way PageHighMem() macro is implemented.
It derives the exact zone for page and then does pointer subtraction with first zone to infer the zone_type.
The pointer arithmatic in turn generates the code bloat.

PageHighMem(page)
is_highmem(page_zone(page))

     zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones

Instead use is_highmem_idx() to work on zone_type available in page flags

   ----- Before -----
80756348:	mov_s      r13,r0
8075634a:	ld_s       r2,[r13,0]
8075634c:	lsr_s      r2,r2,30
8075634e:	mpy        r2,r2,0x2a4
80756352:	add_s      r2,r2,0x80aef880
80756358:	ld_s       r3,[r2,28]
8075635a:	sub_s      r2,r2,r3
8075635c:	breq       r2,0x2a4,80756378 <kmap+0x48>
80756364:	breq       r2,0x548,80756378 <kmap+0x48>
   ----- After  -----
80756330:	mov_s      r13,r0
80756332:	ld_s       r2,[r13,0]
80756334:	lsr_s      r2,r2,30
80756336:	sub_s      r2,r2,1
80756338:	brlo       r2,2,80756348 <kmap+0x30>

For x86 defconfig build (32 bit only) it saves around 900 bytes. For ARC defconfig with HIGHMEM, it saved around 2K bytes.

   ---->8-------
./scripts/bloat-o-meter x86/vmlinux-defconfig-pre x86/vmlinux-defconfig-post
add/remove: 0/0 grow/shrink: 0/36 up/down: 0/-934 (-934)
function                                     old     new   delta
saveable_page                                162     154      -8
saveable_highmem_page                        154     146      -8
skb_gro_reset_offset                         147     131     -16
...
...
__change_page_attr_set_clr                  1715    1678     -37
setup_data_read                              434     394     -40
mon_bin_event                               1967    1927     -40
swsusp_save                                 1148    1105     -43
_set_pages_array                             549     493     -56

---->8-------

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

e.g. For ARC kmap()

is_highmem() is deranged. Can't we use a bit in zone->flags or something?

Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line

It won't be "a" bit since zone_type is an enum.

Yes it will!

static inline int is_highmem(struct zone *zone)

{
	return test_bit(ZONE_HIGHMEM, &zone->flags);
}
Quoted 7 lines

Point is do we want to fix this specific case, or do we want to improve zone_idx() in general.

#define zone_idx(zone) ((zone) - (zone)->zone_pgdat->node_zones

If former, I can split up zone->flags into 31:1 bit-field. Otherwise I will split it into 24:8 to hold both zone_flags and zone_type

Andrew, what do u think. Do we need to shoehorn type (idx) and flags into same placeholder or simply add a new zone_idx field.
There will be slight complications with former to avoid any possible collisions when someone calls set_bit() for flags holder bitfield.

IMHO zone struct is already so bloated, adding another word would be OK specially when number of zone structures is not too many !

-Vineet