Skip to contentSkip to conversation

Dropping CONFIG_REORDER on x86-64 for 2.6.22

DISCUSSION Andi Kleen 7 participants 9 messages Apr 19, 2007 – Apr 20, 2007

lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

Hallo,

I'm thinking about dropping the x86-64 CONFIG_REORDER for 2.6.22. The function enabled -ffunction-sections and then tries to reorder the executable

While that's in theory a worthy goal to save TLB/icache, in practice it didn't really work out.

Rationale:
- It cannot be enabled in normal builds because all current lds become very slow when they have to handle thousands of sections.

- The list of hot functions it needs - arch/x86_64/kernel/functionlist - hasn't been changed for a long time and is already out of date. I also don't know no sane way to keep it uptodate. Profile feedback at runtime is not practical for the kernel and even if it's questionable we would find a good training workload. Trying to change the manually is imho foolish and likely not too useful anyways (if any hot function is missed the TLB savings will be lost and it is unlikely we can get 100% coverage)

- The workloads of the kernel are too diverse to make this concept really work

So I'm planning to drop the option and arch/x86_64/kernel/functionlist

Comments?

-Andi

#1 lore raw
Quoted 4 lines · Andi Kleen

Rationale:
- It cannot be enabled in normal builds because all current lds become very slow when they have to handle thousands of sections.

afaik this is only ever reported on SuSE; I've not heard it on any other distro...

#2 lore raw
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines · Andi Kleen

Rationale:
- It cannot be enabled in normal builds because all current lds become very slow when they have to handle thousands of sections.

afaik this is only ever reported on SuSE; I've not heard it on any other distro...

It's horribly slow on Fedora.

#3 lore raw
In reply to Arjan van de Ven
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines · Andi Kleen

Rationale:
- It cannot be enabled in normal builds because all current lds become very slow when they have to handle thousands of sections.

afaik this is only ever reported on SuSE; I've not heard it on any other distro...

Even with that option set, the full kernel build with my configuration finishes in one minute flat on my Gentoo box. Could it be that the linker uses enormous amounts of memory? I have 4GB so I wouldn't immediately notice.

Signature · 2 lines
Måns Rullgård
[email protected]
author
#4 lore raw
Quoted 8 lines · On Friday 20 April 2007 10:35:10 Måns Rullgård wrote:
Quoted 6 lines · Arjan van de Ven
Quoted 4 lines · Andi Kleen

Rationale:
- It cannot be enabled in normal builds because all current lds become very slow when they have to handle thousands of sections.

Quoted 7 lines
Quoted 2 lines

afaik this is only ever reported on SuSE; I've not heard it on any other distro...

Even with that option set, the full kernel build with my configuration finishes in one minute flat on my Gentoo box. Could it be that the linker uses enormous amounts of memory? I have 4GB so I wouldn't immediately notice.

What binutils version do you use? Maybe that particular problem is finally fixed. Of course would still leave the other issues I listed.

-Andi

#5 lore raw
Quoted 12 lines · Andi Kleen
Quoted 11 lines · On Friday 20 April 2007 10:35:10 Måns Rullgård wrote:
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines · Andi Kleen

Rationale:
- It cannot be enabled in normal builds because all current lds become very slow when they have to handle thousands of sections.

afaik this is only ever reported on SuSE; I've not heard it on any other distro...

Quoted 8 lines
Quoted 4 lines

Even with that option set, the full kernel build with my configuration finishes in one minute flat on my Gentoo box. Could it be that the linker uses enormous amounts of memory? I have 4GB so I wouldn't immediately notice.

What binutils version do you use? Maybe that particular problem is finally fixed. Of course would still leave the other issues I listed.

I'm using the Gentoo default, which is 2.16.1 plus some patches. A quick scan of the patches suggests that perhaps this one has something to do with it:
http://sources.redhat.com/ml/binutils/2005-03/msg00847.html

Signature · 2 lines
Måns Rullgård
[email protected]
#6 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen

Hallo,

I'm thinking about dropping the x86-64 CONFIG_REORDER for 2.6.22. The function enabled -ffunction-sections and then tries to reorder the executable

Quoted 8 lines

While that's in theory a worthy goal to save TLB/icache, in practice it didn't really work out.

Rationale:
...
So I'm planning to drop the option and arch/x86_64/kernel/functionlist

Comments?

Have there been any benchmarks showing performance improvements with -ffunction-sections?

Quoted 1 line

-Andi

cu
Adrian

Signature · 4 lines
       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed
#7 lore raw
In reply to Andi Kleen

From: Andi Kleen <[email protected]>
Date: Thu, 19 Apr 2007 13:54:52 +0200

Quoted 1 line

So I'm planning to drop the option and arch/x86_64/kernel/functionlist

Please do so, I'm tired of editing that file every time I remove something from the tree.

That file had alloc_skb_from_cache() in it, which nothing in the vanilla kernel ever invoked. How did it even get there? If it was put there for Xen's sake, that stinks because Xen is out of tree.

#8 lore raw
Quoted 4 lines · David Miller

That file had alloc_skb_from_cache() in it, which nothing in the vanilla kernel ever invoked. How did it even get there? If it was put there for Xen's sake, that stinks because Xen is out of tree.

I think it was included because this is a list of all functions, so no conspiracies to see here :)

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt