Skip to contentSkip to conversation

provide an API to userspace doing memory snapshot

RFC Wenchao Xia 2 participants 4 messages Apr 15, 2013 – Apr 17, 2013 KVM

lore
Share & tools
lore
#0 lore raw

Hi,
I'd like to add/export an function which allow userspace program
to take snapshot for a region of memory. Since it is not implemented yet I will describe it as C APIs, it is quite simple now and if it is worthy I'll improve the interface later:

Simple prototype:
C API in userspace:
/*
* This function will mark a section of memory as COW, and return
* a new virtual address of it. User space program can dump out the
* content as a snapshot while other thread continue modify the content
* in the region.

 *   @addr: the virtual address to be snapshotted.
 *   @length: the length of it.
 *   This function returns a new virtual address which can be used as

* snapshot. Return NULL on fail.
*/
void *memory_snapshot_create(void *addr, uint64_t length);

/*
* This function will free the memory snapshot.
* @addr: the virtual snapshot addr to be freed, it should be the
* returned one in memory_snapshot_create().
*/
void memory_snapshot_delete(void *addr);

In kernel space:
The pages in those virtual address will be marked as COW. Take a
page with physical addr P0 as example, it will have two virtual addr:
old A0 and new A1. When modified, kernel should create a new page P1 with same contents, and mapping A1 to P1. When NUMA is used, P1 can be a slower page.
It is quite like fork(), but only COW part of pages. Maybe add it
as an ioctl() in kvm.ko, and change the input/output as a structure describe guest memory state.

Why bring it to kernel space:
Compared with fork():
1 take less RAM, less halt time by avoid marking unnecessary pages
COW.
2 take less RAM if API can return a bitmap let qemu consume dirty
page first.
3 much nicer userspace program model, no need to pipe or memcpy(),
brings better performance.
4 optimization space in kernel space, since snapshoted pages
can be set into slower memory in NUMA when change comes.

#1 lore raw
Quoted 4 lines · Wenchao Xia

I'd like to add/export an function which allow userspace program
to take snapshot for a region of memory. Since it is not implemented yet I will describe it as C APIs, it is quite simple now and if it is worthy I'll improve the interface later:

We talked about a simple approach using fork(2) on IRC yesterday.

Is this email outdated?

Stefan

#2 lore raw
Quoted 6 lines · Stefan Hajnoczi
Quoted 4 lines · Wenchao Xia

I'd like to add/export an function which allow userspace program
to take snapshot for a region of memory. Since it is not implemented yet I will describe it as C APIs, it is quite simple now and if it is worthy I'll improve the interface later:

Quoted 6 lines

We talked about a simple approach using fork(2) on IRC yesterday.

Is this email outdated?

Stefan

No, after the discuss on IRC, I agree that fork() is a simpler
method to do it, which can comes to qemu fast, since user wants it.
With a more consideration, still I think a KVM's mem snapshot would
be an long term solution for it:
The source of the problem comes from acceleration module, kvm.ko, when
qemu does not use it, no troubles. This means an acceleration module missed a function while caller requires. My instinct idea is: when acceleration module replace a pure software one, it should try provide all parts or not stop software filling the gap, and doing so brings benefits, so hope to add it.
My API description is old, the core is COW pages, maybe redesign if
reasonable.

#3 lore raw
Quoted 13 lines · Wenchao Xia
Quoted 6 lines · Stefan Hajnoczi
Quoted 4 lines · Wenchao Xia

I'd like to add/export an function which allow userspace program
to take snapshot for a region of memory. Since it is not implemented yet I will describe it as C APIs, it is quite simple now and if it is worthy I'll improve the interface later:

Quoted 6 lines

We talked about a simple approach using fork(2) on IRC yesterday.

Is this email outdated?

Stefan

Quoted 12 lines

No, after the discuss on IRC, I agree that fork() is a simpler
method to do it, which can comes to qemu fast, since user wants it.
With a more consideration, still I think a KVM's mem snapshot would
be an long term solution for it:
The source of the problem comes from acceleration module, kvm.ko, when
qemu does not use it, no troubles. This means an acceleration module missed a function while caller requires. My instinct idea is: when acceleration module replace a pure software one, it should try provide all parts or not stop software filling the gap, and doing so brings benefits, so hope to add it.
My API description is old, the core is COW pages, maybe redesign if
reasonable.

QEMU is a userspace process that has guest RAM mmapped. You want to snapshot that mmap region but there is no Linux system call to do that. Maybe a new mremap(2) flag is what you want.

But I don't see the connection to kvm.ko which you mention. The feature you're wishing for has nothing to do with kvm.ko.

Stefan