Is guest OS oriented scheduling welcome?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · 刘志建
Hello folks,
In the past, it was said KVM would like to treat the guest OS threads differently in scheduling. However, till now, the qemu thread is regarded as a conventional user thread. Therefore, it is hard to control how much CPU slices one guest OS can utilize. I don't think a computing cloud provider likes this idea. And, what's more, "Xen and Co.: Communication-aware CPU Scheduling for Consolidated Xen-based Hosting Platforms"(http://www.cse.psu.edu/~sgovinda/papers/vee07.pdf) has shown that the standard thread scheduling in Linux might not fit the virtualization environment well. I have ported Xen's credit scheduler to KVM.
Do you mean, to KVM, or to the Linux scheduler?
Quoted 11 lines
With my work, the users can control how much CPU a guest OS can occupy.
The principles are:
1. all the vcpu threads are declared as FIFO RealTime threads;
2. before a vcpu thread enters guest mode, its blackboard is checked first, to see if the scheduler wants it to continue. If the vcpu thread has a bad luck, it will give up cpu and waits for permission.
3. there is a per-cpu scheduler, which is triggered periodically by its timer. The scheduler writes the scheduling results to the interested party's blackboards. To those that get the permission, the scheduler will try to wake them up in case of their sleeping. 4. In order to port Xen's scheduler, the host Linux is treated as a virtual machine too. However, it never checks its blackboard.
5. a user level per-VM credit control mechanism is implemented, to allow users to dynamically adjust a virtual machine's CPU quota.
Although a vcpu might continue to run for a while after the scheduler has decided that it should actually yield the cpu, our experiments show that this brings trival influence.
The interaction-oriented scheduling are going to be ported/developed.If KVM welcomes this function, I will post it out.
I think this is very interesting, but should be integrated with the Linux scheduler, so it could apply to normal processes, and so it can account for host mode, not just guest mode.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 7 lines · Avi Kivity
Quoted 6 lines · ?????????
Hello folks,
In the past, it was said KVM would like to treat the guest OS threads differently in scheduling. However, till now, the qemu thread is regarded as a conventional user thread. Therefore, it is hard to control how much CPU slices one guest OS can utilize. I don't think a computing cloud provider likes this idea.
Although the standard scheduler tunables are per thread, it is possible to put each QEMU process into a separate CGroup, and use the cpu_shares tunable to control scheduling priority of the guest as a whole, instead of individual threads. Not sure it this is sufficient for what you want, but it is one possible option for guest scheduling..
Daniel
Signature · 4 lines
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Quoted 3 lines · 刘志建
Hello folks,
In the past, it was said KVM would like to treat the guest OS threads differently in scheduling. However, till now, the qemu thread is regarded as a conventional user thread. Therefore, it is hard to control how much CPU slices one guest OS can utilize. I don't think a computing cloud provider likes this idea. And, what's more, "Xen and Co.: Communication-aware CPU Scheduling for Consolidated Xen-based Hosting Platforms"(http://www.cse.psu.edu/~sgovinda/papers/vee07.pdf) has shown that the standard thread scheduling in Linux might not fit the virtualization environment well.
By standard thread scheduling, I presume you mean scheduling that doesn't take into account IO? That is, this paper is arguing that in a virtualization environment, you want to provide temporary disproportionate scheduling to favor IO bound workloads over CPU bound workloads.
I don't think you need the credit scheduler to implement this idea in KVM. CFS provides a number of tunables to userspace along with pretty fine grain control ala cgroups. I think that provides you a roughly equivalent interface to userspace that could be used to make scheduling adjustments based on IO consumption.
What are the other motivating factors for wanting to use credit over cfs?
Regards,
Anthony Liguori
Quoted 24 lines · Anthony Liguori
Quoted 10 lines · 刘志建
Hello folks,
In the past, it was said KVM would like to treat the guest OS
threads differently in scheduling. However, till now, the qemu
thread is regarded as a conventional user thread. Therefore, it is
hard to control how much CPU slices one guest OS can utilize. I
don't think a computing cloud provider likes this idea.
And, what's more, "Xen and Co.: Communication-aware CPU Scheduling
for Consolidated Xen-based Hosting Platforms"(http://www.cse.psu.edu/~sgovinda/papers/vee07.pdf ) has shown that the standard thread scheduling in Linux might not
fit the virtualization environment well.By standard thread scheduling, I presume you mean scheduling that doesn't take into account IO? That is, this paper is arguing that in a virtualization environment, you want to provide temporary disproportionate scheduling to favor IO bound workloads over CPU bound workloads.
I don't think you need the credit scheduler to implement this idea in KVM. CFS provides a number of tunables to userspace along with pretty fine grain control ala cgroups. I think that provides you a roughly equivalent interface to userspace that could be used to make scheduling adjustments based on IO consumption.
Quoted 6 lines
What are the other motivating factors for wanting to use credit over cfs?
Regards,
Anthony Liguori
I'm a lurker on this list, but as someone has raised the point of what cloud computing providers will and won't like, I figure I could add my two cents.
If you guys think in terms of "the client of your client is your real client" - i.e. the clients of the cloud providers are your clients - then you need to consider how they would buy the product. Here in Brazil there is still a lot of ignorance surrounding cloud computing and many clients still think in terms of physical machines - that is that they use their previous knowledge of provisioning physical machines to guide their decisions when contracting new machines. As such they think in terms of contracting a certain amount of MHz/GHz.
I'll give you a perfect and real example of this: We use VMware and we are testing Xen/KVM because we ultimately want to go full open-source with our back-end for virtualization. VMware allows us to sell the product to the end client in terms of CPU/RAM/Disk Space, Xen on the other hand works in terms of CPU prioritization, which AFAIK is the correct way to think about how to use the physical resources. However this approach treats all clients as one unit and does the tasks with the greatest priority instead of treating all clients as equals and taking care of all equally (with a guaranteed minimum amount of CPU as per the client's contract). Adopting Xen would force us to completely redesign our product plans with a product model which would be more alien to the end user (i.e. more difficult to compare/consider).
At the end of the day I think virtualization has two basic users: (1) Companies, laboratories, etc where certain tasks are assigned priorities and all computing resources are used to their maximum all the time (timesharing) and (2) Service providers where all clients have a contract guaranteeing minimum resources and where overselling to some degree is taking place (variable resource usage over the course of the day/week).
Please excuse me if I didn't exactly answer the above question. Coming from a business/somewhat technical background, I'm working as hard as I can to immerse myself in the nitty gritty of virtualization. If anyone has any other questions for what a cloud computing provider would or would not like, feel free to contact me directly or post questions to the list.
Andrew de Andrade
Locaweb