Skip to contentSkip to conversation

looking for ideas on VM related research projects

DISCUSSION Kallol Biswas 3 participants 5 messages Jun 8, 2009 – Jun 9, 2009

lore
Share & tools
lore
#0 lore raw

Hi,
I have a strong team of good, energetic and young kernel engineers
and research scholar and I am looking for ideas on linux virtual memory related research projects for them.

Any input or pointers on future virtual memory projects will be appreciated.

Kallol Biswas
408-718-1864

#1 lore raw
Quoted 8 lines · Kallol Biswas

Hi,
I have a strong team of good, energetic and young kernel engineers
and research scholar and I am looking for ideas on linux virtual memory related research projects for them.

Any input or pointers on future virtual memory projects will be appreciated.

- automatically using and breaking up large pages
- task/vma affinity
- automatically migrating memory when thread/vma affinity cannot be satisfied

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#2 lore raw

Hi,
I am from IO and driver background, and not up to date with
current VM developments in linux.

Are you talking about NUMA architecture? Could you clarify or point me to document that describes the problem with VMA/task affinity? Where do affinity issues arise?

Kallol

Quoted 22 lines · Avi Kivity
Quoted 10 lines · Kallol Biswas

Hi,
  I have a strong team of good, energetic and young kernel engineers and research scholar and I am looking for ideas on linux virtual memory related research projects for them.

Any input or pointers on future virtual memory projects will be appreciated.

- automatically using and breaking up large pages
- task/vma affinity
- automatically migrating memory when thread/vma affinity cannot be satisfied

--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

#3 lore raw
Quoted 5 lines · Kallol Biswas

Hi,
I am from IO and driver background, and not up to date with
current VM developments in linux.

Are you talking about NUMA architecture?

Yes.

Quoted 2 lines

Could you clarify or point me
to document that describes the problem with VMA/task affinity?

Linux will try to allocate memory on the same node that the allocating thread ran on on the time the page was faulted in. However, for long running processes, the thread will eventually migrate to some other cpu with only 1/N chance of actually running on the same node (N being the number of nodes).

If we can specify that a thread is related to some vma, the scheduler can try to keep that thread running on the vma's node, or else it can migrate the vma to a node with available cpu power.

Quoted 2 lines

Where do affinity issues arise?

Large, long-running processes. My specific interest is virtualization with kvm, but I believe HPC and databases can make use of this as well. HPC currently solves the problem by statically allocating memory and cpu resources.

Signature · 1 line
error compiling committee.c: too many arguments to function
#4 lore raw
In reply to Kallol Biswas
Quoted 4 lines · Kallol Biswas

Hi,
I have a strong team of good, energetic and young kernel engineers
and research scholar and I am looking for ideas on linux virtual memory related research projects for them.

There are some recent research projects that have not been integrated into Linux:

http://www.usenix.org/events/hotos09/tech/full_papers/saxena/saxena_html/
http://www.usenix.org/events/osdi08/tech/full_papers/gupta/gupta_html/index.html

I suspect that a combination of difference engine, compressed caching, and flash swapping could really benefit some environments.

Wes Felter - [email protected]