Skip to contentSkip to conversation

TLB evaluation for Linux

DISCUSSION CHADHA,VINEET 2 participants 2 messages Sep 2, 2008

lore
Share & tools
lore
#0 lore raw
Quoted 10 lines · Arjan van de Ven

On Tue, 2 Sep 2008 00:12:03 -0400 (EDT)
"CHADHA,VINEET" <[email protected]> wrote:

note that linux only does an ipi to processors that actually are currently running a thread of the same program (or a kernel thread). Old versions didn't do this (they also IPI'd idle processors), but on modern cpus and modern kernels that's not supposed to happen anymore (the C-states that flush the tlb anyway now do the kernel side bookkeeping as well to avoid the wakeup+useless flush)

Interesting to know about it.

Quoted 4 lines

one of the problems is that invlpg is rather expensive; in long-ago experiments the threshold was like around a handful of pages already. At that point.. all the bookkeeping isn't likely to be a win. Esp since a tlb refill on x86 is quite cheap.

Yeah that is possible. Do you have link to any published work ? It would be still interesting to characterize and compare behavior for new workloads scenarios such as virtual machines.

#1 lore raw
Quoted 27 lines · On Tue, 2 Sep 2008 10:59:25 -0400 (EDT) "CHADHA,VINEET" <[email protected]> wrote:
Quoted 4 lines · Arjan van de Ven

On Tue, 2 Sep 2008 00:12:03 -0400 (EDT)
"CHADHA,VINEET" <[email protected]> wrote:

Quoted 9 lines

note that linux only does an ipi to processors that actually are currently running a thread of the same program (or a kernel thread).
Old versions didn't do this (they also IPI'd idle processors), but
on modern cpus and modern kernels that's not supposed to happen anymore
(the C-states that flush the tlb anyway now do the kernel side bookkeeping as well to avoid the wakeup+useless flush)

Interesting to know about it.

Quoted 6 lines

one of the problems is that invlpg is rather expensive; in long-ago
experiments the threshold was like around a handful of pages already.
At that point.. all the bookkeeping isn't likely to be a win. Esp since a tlb refill on x86 is quite cheap.

Quoted 4 lines

Yeah that is possible. Do you have link to any published work ? It would be still interesting to characterize and compare behavior for new workloads scenarios such as virtual machines.

I don't have a reference, but I'd not be surprised to see on virtual machines for this to be even stronger; each invlpg would be a hypercall, compared to only one for the total flush.