Skip to contentSkip to conversation

prefetch the mmap_sem in the fault path

PATCH Valerie Henson 1 participant 1 message Mar 1, 2006

lore
Share & tools
lore
#0 lore raw

Sorry for the broken threading...

Quoted 16 lines · Ray Bryant
Quoted 15 lines · Arjan van de Ven
Quoted 14 lines · Jes Sorensen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Arjan" == Arjan van de Ven <arjan@xxxxxxxxxxxxxxx> writes:

Arjan> In a micro-benchmark that stresses the pagefault path, the Arjan> down_read_trylock on the mmap_sem showed up quite high on the Arjan> profile. Turns out this lock is bouncing between cpus quite a Arjan> bit and thus is cache-cold a lot. This patch prefetches the Arjan> lock (for write) as early as possible (and before some other Arjan> somewhat expensive operations). With this patch, the Arjan> down_read_trylock basically fell out of the top of profile.

Out of curiousity, how big was the box used for testing? It might be worth investigating if anything can be done to reduce the number of times that lock is taken in the first place.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

After all, what's a pain on a 4-way tends to be an utter nightmare on a 16-way ;(

most of it was done on a 2 way, but some tests were done on a 4-way.

Could you share your microbenchmark with us (or point to the source) and we can give this a try on larger systems?

I would be ecstatic to share this benchmark; however I just started working at Intel and did not realize how long it would take to open source a program written solely by an Intel employee (me). I'm getting the paperwork done as fast as I can.

A quick description of the benchmark is:

* Allocate memory
* Write a pattern to it
* Spawn sufficient threads to keep your cpus busy

Each thread does:

* Allocate a little more memory and copy part of memory to it
* Search for a key within its copy
* Free the memory
* Repeat

The patches Arjan submitted make a small improvement, but the big win turns out to be in tuning malloc() parameters, which we are currently experimenting with.

-VAL (not subscribed to l-k as yet)