Balloon device in qemu?

Marcelo, the balloon was last seen drifting over your territory. Care to brush it up and send it over?

error compiling committee.c: too many arguments to function

Anthony rewrote the backend, I think this is the latest version:

http://www.archivum.info/[email protected]/2008-04/msg00080.html

Is there a way to detect MMU notifiers from userspace? I don't think it's currently safe to madvise unconditionally.

Regards,

Anthony Liguori

There is no way to detect mmu notifiers from userspace (well strictly speaking you could check /proc/kallsyms) but the point is that without mmu notifiers madvise won't be enough (the memory may not be freed without the other ioctl proposed by Marcelo that also flushes sptes).

Especially with the plan to pin pages during memslot allocation (next step in the kvm-userland compatibility effort when built against old kernel) madvise will be a noop because all memory in the memslot will remain pinned.

The safety issue with madvise was then found to be the same issue for all rmap_remove invocations (not just the ones done by Marcelo's ioctl). Whenever the put_page in rmap_remove happens to be the last free of the page, we have a tlb race where we flush the other vcpus tlbs after the page was already freed by rmap_remove. This is usually only an issue for smp guest on smp host. So in short madvise run on a memslot wasn't "less" safe, but still the ioctl proposed to remove sptes to allow memory to be released wasn't capable of fixing this race for madvise.

mmu notifiers on the other hand are allowing madvise to free all memory reliably while fixing this race as well as a whole (not just for madvise but for swapping and all other operations too).

The compatibility plan for the host kernels without mmu notifiers is going to entirely prevent swapping, and in turn it will never allow rmap_remove to invoke the last free of the page will also fix the tlb troubles.

So if you currently just madvise, right now it won't be less safe than without madvise (it's not safe regardless) but without mmu notifier madvise won't remove sptes so it won't be reliable (this is why once Marcelo proposed the ioctl to call after/before madvise, but that ioctl still had the same issues that every other rmap_remove had without mmu notifier). With the next plan of pinning all pages you can still madvise just fine, and it will be safe too (like every other rmap_remove will be safe) but it will be a guaranteed a noop because of the memslot being entirely page-pinning. So madvise on old kernels is never less safe but if we want to provide reliable ballooning to host kernels without mmu notifiers with the next compatibility plan that Avi suggested, we should "trim" the memslot first (to unpin the pages) instead of calling madvise and then you can munmap the range instead of madvise. Or you can add a new call that only drops sptes in the range and _later_ unpin the pages (that's similar to Marcelo ioctl but the unpin event happening later should fix its previous safety issues). However if the sptes removal is done without touching the memslot that will complicate the memslots semantics quite a bit (because all memory mapped by a memslot won't be guaranteed to be entirely pinned anymore and we'll have to track separately the balloned ranges that aren't pinned). So I guess teaching set_memory_region to trim a memslot (currently it can't) sounds one approach that could be used to allow balloning on kernels without mmu notifiers, if we go with Avi's backwards compatibility suggestion of memslot pinning. Then it's up to you if to munamp or madvise, it'll be the same then.

It's a bit complicated if something isn't clear on what the issues are without mmu notifier, don't hesitate to ask more details.

Right. Ideally, I'd like to just disable the balloon device if mmu notifiers *and* kvm support for it are not present. If you're saying there is no way to probe for this now from userspace, then that's something we need to add.

Regards,

Anthony Liguori

I think it is there, IIRC the kvm mmu notifier pull request on lkml had a few patches from Andrea with all the mm magic and one from avi adding a feature flag. Checking ...

#define KVM_CAP_SYNC_MMU 16 /* Changes to host mmap are reflected in guest */

This one I guess ...

cheers,
Gerd

http://kraxel.fedorapeople.org/xenner/

That doesn't currently tell if kvm is using mmu notifiers though, because it's set unconditionally. If we want to use it to tell if kvm is using mmu notifiers we've to wrap it around an #ifdef or to use the awk hack, which is trivial to add.

I'm more worried about the fact that with Anthony's plan ballooning will be unusable for a long while on enterprise distros, as it will take a while before they run on a >=2.6.27 kernel. So I wonder if a compact layer is worth it. But surely initially we can modify KVM_CAP_SYNC_MMU to not be set if kernel isn't compiled with CONFIG_MMU_NOTIFIER=y and we can add the compact layer later if needed.

I added something to hack-module.awk, so KVM_CAP_SYNC_MMU is now reliable.

error compiling committee.c: too many arguments to function

Great, I'll update the balloon backend and resubmit this afternoon.

Regards,

Anthony Liguori