KVM call for 2017-03-14
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.html
After discussions on the QEMU Summit, we are going to have always open a KVM call where you can add topics.
Call details:
By popular demand, a google calendar public entry with it
(Let me know if you have any problems with the calendar entry. I just gave up about getting right at the same time CEST, CET, EDT and DST).
If you need phone number details, contact me privately
Thanks, Juan.
Quoted 10 lines · Juan Quintela
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.html
Ah, I'd forgotten that this was on the call agenda. I actually had an interesting conversation with Alex Graf last week about some similar topics, which I guess you could generally summarize as "what are the issues we need to address as a project in order to not become irrelevant in five years time". Since I wrote them up for an internal "what I did on my holi^Wconference trip" report I might as well repost them here:
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record. It wouldn't be too surprising if some time in the next five years somebody writes a replacement in a safer language (perhaps also targeting only the VM support role) and it got enough mindshare and takeup to eclipse QEMU. [Is it too early/daft to think about prototyping being able to write QEMU device emulation in Rust ?]
If the "VM support" usecase moves to another project then QEMU will become a very quiet backwater...
- on the "emulation" side, nobody is clearly articulating a purpose for QEMU, a reason why you should use it rather than other modelling technologies (or rather than using real hardware). As a result the efforts applied to QEMU are somewhat unfocused. Are we trying to be:
. a dev platform before easy h/w availability?
[not easy for QEMU for several reasons]
. a dev tool that provides better introspection into guest
behaviour than running on h/w?
[if so we should put more work into improving our introspection
and guest tracing capabilities!]
. primarily a tool for doing automated CI testing and one-off
developer smoke-testing that's easier to set up and scale than trying to test on real h/w?
. something else?
[your idea goes here!]
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
- there's no easy way for people to use parts of QEMU like the CPU emulation, or to add their own devices without having to write lots of C code (we're firmly in a "one monolithic blob of code" setup right now and disentangling and setting clear API dividing lines will be a lot of work)
[Making QEMU more modular would help with defeating the legacy and back-compat dragons, though]
thanks
-- PMM
Quoted 21 lines · Peter Maydell
Quoted 10 lines · Juan Quintela
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.htmlAh, I'd forgotten that this was on the call agenda. I actually had an interesting conversation with Alex Graf last week about some similar topics, which I guess you could generally summarize as "what are the issues we need to address as a project in order to not become irrelevant in five years time". Since I wrote them up for an internal "what I did on my holi^Wconference trip" report I might as well repost them here:
Quoted 9 lines
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record. It wouldn't be too surprising if some time in the next five years somebody writes a replacement in a safer language (perhaps also targeting only the VM support role) and it got enough mindshare and takeup to eclipse QEMU. [Is it too early/daft to think about prototyping being able to write QEMU device emulation in Rust ?]
I think rust has the potential to be very interesting. I've been watching to remacs project with interest:
https://github.com/Wilfred/remacs
They are attempting to port Emacs to rust in a piecemeal way. Whether this works remains to be seen but it is certainly an interesting approach compared to other "re-write it all" approaches.
Quoted 4 lines
If the "VM support" usecase moves to another project then QEMU will become a very quiet backwater...
- on the "emulation" side, nobody is clearly articulating a purpose for QEMU, a reason why you should use it rather than other modelling
Quoted 8 lines
technologies (or rather than using real hardware). As a result the efforts applied to QEMU are somewhat unfocused. Are we trying to be:
. a dev platform before easy h/w availability?
[not easy for QEMU for several reasons]
. a dev tool that provides better introspection into guest
behaviour than running on h/w?
[if so we should put more work into improving our introspection
and guest tracing capabilities!]
For example now we support atomics in the TCG it would be fairly easy* to port something like the ThreadSanitizer (which has fairly hefty memory requirements) as a tool for debugging system code.
Quoted 5 lines
. primarily a tool for doing automated CI testing and one-off
developer smoke-testing that's easier to set up and scale than trying to test on real h/w?
. something else?
[your idea goes here!]
Quoted 8 lines
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
- there's no easy way for people to use parts of QEMU like the CPU emulation, or to add their own devices without having to write lots of C code (we're firmly in a "one monolithic blob of code" setup right now and disentangling and setting clear API dividing lines will be a lot of work)
I spoke to Edgar about posting his rports patch set as a catalyst for discussion. Certainly for modelling hobby projects or new hardware QEMU could do with support for rapid prototyping. Others have mentioned maybe including a scripting language integration for device modelling - although I still hold the view that what ever language you choose will be the "wrong one" according to the majority of developers.
Quoted 5 lines
[Making QEMU more modular would help with defeating the legacy and back-compat dragons, though]
thanks
-- PMM
--
Alex Bennée
Quoted 21 lines · Peter Maydell
Quoted 10 lines · Juan Quintela
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.htmlAh, I'd forgotten that this was on the call agenda. I actually had an interesting conversation with Alex Graf last week about some similar topics, which I guess you could generally summarize as "what are the issues we need to address as a project in order to not become irrelevant in five years time". Since I wrote them up for an internal "what I did on my holi^Wconference trip" report I might as well repost them here:
Quoted 4 lines
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record.
On a previous life, I have to work on making qemu pass Common Criteria. Not to be able to remove large bits of it that we were not interested was a mess (basically we care about kvm + a bunch of devices, but it was impossible to remove things like TCG).
Quoted 21 lines
It wouldn't be too surprising if
some time in the next five years somebody writes a replacement in a safer language (perhaps also targeting only the VM support role) and it got enough mindshare and takeup to eclipse QEMU. [Is it too early/daft to think about prototyping being able to
write QEMU device emulation in Rust ?]
If the "VM support" usecase moves to another project then QEMU will become a very quiet backwater...- on the "emulation" side, nobody is clearly articulating a purpose for QEMU, a reason why you should use it rather than other modelling technologies (or rather than using real hardware). As a result the efforts applied to QEMU are somewhat unfocused. Are we trying to be:
. a dev platform before easy h/w availability?
[not easy for QEMU for several reasons]
. a dev tool that provides better introspection into guest
behaviour than running on h/w?
[if so we should put more work into improving our introspection
and guest tracing capabilities!]
Quoted 8 lines
. primarily a tool for doing automated CI testing and one-off
developer smoke-testing that's easier to set up and scale than trying to test on real h/w?
. something else?
[your idea goes here!]
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
But how? and When?
If you get some new interfaces and some devices are not ported, what are we going to do?
- require the people that do the new interfaces to update the legacy code (and they will get bored)
- remove the unmaintained code after some time
Both approaches have its advantages and disadvantages.
Quoted 7 lines
- there's no easy way for people to use parts of QEMU like the CPU emulation, or to add their own devices without having to write lots of C code (we're firmly in a "one monolithic blob of code" setup right now and disentangling and setting clear API dividing lines will be a lot of work)
[Making QEMU more modular would help with defeating the legacy and back-compat dragons, though]
That would be ideal, but where to start? Think of something as "simple" asd adding a struct of operations that implement the differences for TCG, xen and kvm. And you see that you get into having to rewrite lot of code after moving to that abstraction just the more simple operations.
Later, Juan.
Quoted 9 lines · Juan Quintela
Quoted 2 lines · Peter Maydell
[Making QEMU more modular would help with defeating the legacy and back-compat dragons, though]
That would be ideal, but where to start? Think of something as "simple" asd adding a struct of operations that implement the differences for TCG, xen and kvm. And you see that you get into having to rewrite lot of code after moving to that abstraction just the more simple operations.
Oh, indeed, it's a huge and painful amount of work that doesn't actually produce a product with more features when you've done it. But if it was simple and easy we'd have already done it and it wouldn't merit listing on a "big problems we need to solve somehow" list :-)
thanks
-- PMM
Quoted 27 lines · Juan Quintela
Quoted 21 lines · Peter Maydell
Quoted 10 lines · Juan Quintela
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.htmlAh, I'd forgotten that this was on the call agenda. I actually had an interesting conversation with Alex Graf last week about some similar topics, which I guess you could generally summarize as "what are the issues we need to address as a project in order to not become irrelevant in five years time". Since I wrote them up for an internal "what I did on my holi^Wconference trip" report I might as well repost them here:
Quoted 4 lines
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record.
Quoted 4 lines
On a previous life, I have to work on making qemu pass Common Criteria. Not to be able to remove large bits of it that we were not interested was a mess (basically we care about kvm + a bunch of devices, but it was impossible to remove things like TCG).
There was a project to move the build system to Kconfig. Today some block drivers can be loaded at run-time instead of built into the QEMU binary.
Actually we don't need Kconfig if we continue developing module support. For example, if accelerators become modules then you simply don't ship or load the TCG and qtest modules.
ISTR the next are to modularize was ui/ so that GTK, SDL, SPICE, etc can be loaded on-demand and do not need to be included in the core QEMU package.
Stefan
Quoted 21 lines · Peter Maydell
Quoted 10 lines · Juan Quintela
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.htmlAh, I'd forgotten that this was on the call agenda. I actually had an interesting conversation with Alex Graf last week about some similar topics, which I guess you could generally summarize as "what are the issues we need to address as a project in order to not become irrelevant in five years time". Since I wrote them up for an internal "what I did on my holi^Wconference trip" report I might as well repost them here:
Quoted 9 lines
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record. It wouldn't be too surprising if some time in the next five years somebody writes a replacement in a safer language (perhaps also targeting only the VM support role) and it got enough mindshare and takeup to eclipse QEMU. [Is it too early/daft to think about prototyping being able to write QEMU device emulation in Rust ?]
We can move to a safer language starting with the device emulation layer. Keep the rest in C for now. Use a language that has good C interoperability or a convenient foreign function interface.
Start writing new device models in the new language. Convert existing devices if they are good candidates, like the e1000 NIC emulation.
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?
3. Is it safer than C even when writing code to operate on guest RAM (i.e. it's no good if you must use unsafe primitives to do the systems programming tasks that QEMU requires)?
4. Is C interoperability convenient and high performance?
Stefan
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
thanks
-- PMM
Quoted 17 lines · Stefan Hajnoczi
Quoted 9 lines
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record. It wouldn't be too surprising if some time in the next five years somebody writes a replacement in a safer language (perhaps also targeting only the VM support role) and it got enough mindshare and takeup to eclipse QEMU. [Is it too early/daft to think about prototyping being able to write QEMU device emulation in Rust ?]
We can move to a safer language starting with the device emulation layer. Keep the rest in C for now. Use a language that has good C interoperability or a convenient foreign function interface.
Start writing new device models in the new language. Convert existing devices if they are good candidates, like the e1000 NIC emulation.
Quoted 7 lines
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?
3. Is it safer than C even when writing code to operate on guest RAM (i.e. it's no good if you must use unsafe primitives to do the systems programming tasks that QEMU requires)?
4. Is C interoperability convenient and high performance?
That is one approach. Other is to move to a "safer" language the parts that are not performance sensitive. Our command line parsing and device object done is C just make things much, much worse for no gain. A language better for dealing with strings and things like that?
Later, Juan.
PD. And before anyone asked, a language that is not able to get right:
a = "hello"
b = "world"
c = a + " " + b
(put any syntax you like) without having to wonder about memory allocation, sizes, etc is not really valid to work with strings IMHO.
Quoted 6 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?
Quoted 10 lines
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
Now more seriously, I can (relatively easy) compile test my pull requests with:
- linux x86 (latest fedora, but I can get an older one if needed)
- linux x86_64 (latest fedor,, but the same)
- mingw64 32bit (latest fedora, but here I have the problem that Peter uses a different crosscompiler than me)
- mingw64 32bit (the same)
But for the rest, I need to wait that somebody told me that it breaks the build. Normally it is things like size_t is 32bit instead of 64bit or some stupid things like that, that are trivial to fix if I can compile there before doing the pull submission.
Later, Juan.
Quoted 29 lines · Peter Maydell
Quoted 9 lines · Juan Quintela
Hi
Please, send any topic that you are interested in covering.
So far the agenda is:
- Direction of QEMU and toolstack in light of Google Cloud blog:
https://cloudplatform.googleblog.com/2017/01/7-ways-we-harden-our-KVM-hypervisor-at-Google-Cloud-security-in-plaintext.htmlAh, I'd forgotten that this was on the call agenda. I actually had an interesting conversation with Alex Graf last week about some similar topics, which I guess you could generally summarize as "what are the issues we need to address as a project in order to not become irrelevant in five years time". Since I wrote them up for an internal "what I did on my holi^Wconference trip" report I might as well repost them here:
- on the "VM support" side, QEMU is more used because it's the only production-quality option in this space, rather than because its users love it. (cf the Google choice to replace it.) It's also got a pretty poor security record. It wouldn't be too surprising if some time in the next five years somebody writes a replacement in a safer language (perhaps also targeting only the VM support role) and it got enough mindshare and takeup to eclipse QEMU. [Is it too early/daft to think about prototyping being able to write QEMU device emulation in Rust ?]
Quoted 8 lines
If the "VM support" usecase moves to another project then QEMU will become a very quiet backwater...
- on the "emulation" side, nobody is clearly articulating a purpose for QEMU, a reason why you should use it rather than other modelling technologies (or rather than using real hardware). As a result the efforts applied to QEMU are somewhat unfocused. Are we trying to be:
. a dev platform before easy h/w availability?
[not easy for QEMU for several reasons]
What reasons exactly do you mean here?
I think that this would be very useful feature, since AFAIK there is no real good open source emulator "Lego" set available yet. We'd "just" need a possibility to build machines on the fly instead of hard-wiring them in the C source code...
Quoted 3 lines
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
... and don't forget all the code that is in "orphan" state since many years... it's often hard to get patches accepted that primarily touches files that nobody feels responsible for...
Maybe it's really time for a "spring-cleaning", break with some compatibility cruft and do a 3.0 release afterwards ;-)
Thomas
Quoted 6 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?
Quoted 10 lines
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
Yes, please.
Am 14.03.2017 um 10:24 hat Thomas Huth geschrieben:
Quoted 10 lines
Quoted 3 lines
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
... and don't forget all the code that is in "orphan" state since many years... it's often hard to get patches accepted that primarily touches files that nobody feels responsible for...
Maybe it's really time for a "spring-cleaning", break with some compatibility cruft and do a 3.0 release afterwards ;-)
If we decide that the situation is bad enough to do this, I'd vote for breaking not just "some" compatibility for a usual release after three months, but to take more time for it, completely break with the old interfaces and declare this a new QEMU that libvirt should have a separate driver for. And then throw out _all_ of the interfaces that don't match the design any more that we have in mind, even if this means a temporary regression in features.
This means one big cut rather than having every release just slightly incompatible with the previous releases, which should actually make it more managable, even though the change is more radical.
Kevin
Quoted 6 lines · Thomas Huth
Quoted 3 lines · Peter Maydell
Are we trying to be:
. a dev platform before easy h/w availability?
[not easy for QEMU for several reasons]What reasons exactly do you mean here?
The main ones I had in mind are:
* to do this you need really to have access to the specs for new hardware features early. If you wait for publication of official specs to start implementation then you won't be done until most of the "window" before h/w is available has already closed
* you need to actually implement new features, which requires more work than we're currently putting in -- for instance for ARM we still haven't finished implementing virtualization emulation, and we haven't even started with v8.1 features yet
* you need to do better testing than just "does it boot Linux?" if you're trying to provide a dev platform for people to use for developing Linux...
thanks
-- PMM
Quoted 5 lines · Stefan Hajnoczi
The minimum requirements for the new language:
3. Is it safer than C even when writing code to operate on guest RAM (i.e. it's no good if you must use unsafe primitives to do the systems programming tasks that QEMU requires)?
My impression is that many of our security vulnerabilities are overflows in local arrays in the device emulation (for instance good old VENOM), so I think that even if a candidate safer language only provided bounds-checking on arrays it knew about and not on raw guest RAM it would still be a significant improvement. (Accesses to guest RAM are often via APIs that we could add bounds-checks to "by hand" anyway.) So I wouldn't consider this as a "minimum requirement", only a "nice to have".
thanks
-- PMM
Paolo Bonzini
maintainerQuoted 10 lines · Peter Maydell
Quoted 3 lines
3. Is it safer than C even when writing code to operate on guest RAM (i.e. it's no good if you must use unsafe primitives to do the systems programming tasks that QEMU requires)?
My impression is that many of our security vulnerabilities are overflows in local arrays in the device emulation (for instance good old VENOM), so I think that even if a candidate safer language only provided bounds-checking on arrays it knew about and not on raw guest RAM it would still be a significant improvement. (Accesses to guest RAM are often via APIs that we could add bounds-checks to "by hand" anyway.)
Right, this was one of the reasons behind the introduction of
MemoryRegionCache: get both speed (like address_space_map) and bounds checking (like address_space_rw).
It looks like it should be easy to wrap it in any language, be it Rust or a scripting language like Lua.
Paolo
Quoted 18 lines · Juan Quintela
Quoted 6 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Quoted 10 lines
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
Quoted 3 lines
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
OSX is actually in the set that's OK because I have a machine I can test on. The ones that are problems are all the BSDs, AIX, Solaris, Haiku, and architectures sparc, mips, ia64, s390.
thanks
-- PMM
Quoted 20 lines · Kevin Wolf
Am 14.03.2017 um 10:24 hat Thomas Huth geschrieben:
Quoted 10 lines
Quoted 3 lines
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
... and don't forget all the code that is in "orphan" state since many years... it's often hard to get patches accepted that primarily touches files that nobody feels responsible for...
Maybe it's really time for a "spring-cleaning", break with some compatibility cruft and do a 3.0 release afterwards ;-)
If we decide that the situation is bad enough to do this, I'd vote for breaking not just "some" compatibility for a usual release after three months, but to take more time for it, completely break with the old interfaces and declare this a new QEMU that libvirt should have a separate driver for. And then throw out _all_ of the interfaces that don't match the design any more that we have in mind, even if this means a temporary regression in features.
Quoted 3 lines
This means one big cut rather than having every release just slightly incompatible with the previous releases, which should actually make it more managable, even though the change is more radical.
Of course, "legacy code and back-compatibility requirements" will start to grow back the minute we release new interfaces. Whether a big cut is worthwhile depends on the seriousness of the current situation and the rate of regrowth. There's hope the weeds will grow more slowly around well-designed new interfaces.
Am 14.03.2017 um 13:20 hat Markus Armbruster geschrieben:
Quoted 26 lines
Quoted 20 lines · Kevin Wolf
Am 14.03.2017 um 10:24 hat Thomas Huth geschrieben:
Quoted 10 lines
Quoted 3 lines
- in all areas our legacy code and back-compatibility requirements are threatening to choke forward progress if we don't make serious efforts to get on top of them
... and don't forget all the code that is in "orphan" state since many years... it's often hard to get patches accepted that primarily touches files that nobody feels responsible for...
Maybe it's really time for a "spring-cleaning", break with some compatibility cruft and do a 3.0 release afterwards ;-)
If we decide that the situation is bad enough to do this, I'd vote for breaking not just "some" compatibility for a usual release after three months, but to take more time for it, completely break with the old interfaces and declare this a new QEMU that libvirt should have a separate driver for. And then throw out _all_ of the interfaces that don't match the design any more that we have in mind, even if this means a temporary regression in features.
Quoted 3 lines
This means one big cut rather than having every release just slightly incompatible with the previous releases, which should actually make it more managable, even though the change is more radical.
Quoted 5 lines
Of course, "legacy code and back-compatibility requirements" will start to grow back the minute we release new interfaces. Whether a big cut is worthwhile depends on the seriousness of the current situation and the rate of regrowth. There's hope the weeds will grow more slowly around well-designed new interfaces.
There is no doubt that this would happen. But we've managed to keep compatibility more or less for 10 years, so if we extrapolate from that, it means that in another 10 years we need another cut. Doesn't sound too bad.
I'm not completely sure whether our situation really requires it, but it's true that we're (or at least I am) spending a lot of time for keeping compatibility with old interfaces that nobody really likes and that simply don't match any more how things really work.
There are more nice-to-have things like consistently spelt option names and monitor commands that will never be important enough to be cleaned up on their own.
So I'm not saying that we should do this big cut, but _if_ we say that 3.0 is to be considered incompatible, then I'd go for the radical approach rather than making things incompatible in hundreds of details, but not really attacking the big problems that slow down development.
Kevin
Quoted 30 lines · * Juan Quintela ([email protected]) wrote:
Quoted 6 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Quoted 10 lines
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
Now more seriously, I can (relatively easy) compile test my pull requests with:
- linux x86 (latest fedora, but I can get an older one if needed)
- linuxx86_64(latest fedor,, but the same)
- mingw64 32bit (latest fedora, but here I have the problem that Peter uses a different crosscompiler than me)
- mingw64 32bit (the same)
Quoted 4 lines
But for the rest, I need to wait that somebody told me that it breaks the build. Normally it is things like
size_tis 32bit instead of 64bit or some stupid things like that, that are trivial to fix if I can compile there before doing the pull submission.
I also do a FreeBSD VM, and grab an aarch64 and/or PPC bigendian host to test on.
(I could grab an ia64 host, but I don't think I could find anything to install on it that would be new enough for the rest of our build requirements).
Dave
Quoted 2 lines
Later, Juan.
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 36 lines · Dr. David Alan Gilbert
Quoted 30 lines · * Juan Quintela ([email protected]) wrote:
Quoted 6 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Quoted 10 lines
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
Now more seriously, I can (relatively easy) compile test my pull requests with:
- linux x86 (latest fedora, but I can get an older one if needed)
- linuxx86_64(latest fedor,, but the same)
- mingw64 32bit (latest fedora, but here I have the problem that Peter uses a different crosscompiler than me)
- mingw64 32bit (the same)Quoted 4 lines
But for the rest, I need to wait that somebody told me that it breaks the build. Normally it is things like
size_tis 32bit instead of 64bit or some stupid things like that, that are trivial to fix if I can compile there before doing the pull submission.
Quoted 6 lines
I also do a FreeBSD VM, and grab an aarch64 and/or PPC bigendian host to test on.
(I could grab an ia64 host, but I don't think I could find anything to install on it that would be new enough for the rest of our build requirements).
Indeed, ia64 is a fully dead as a host architecture at this point, only interesting as a historical curiosity. Paolo already killed ia64 KVM host support in Linux git back in 2014.
Regards,
Daniel
Signature · 3 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
Obsolete QEMU host environments (was: Re: KVM call for 2017-03-14)
Quoted 40 lines · Daniel P. Berrange
Quoted 39 lines · Dr. David Alan Gilbert
Quoted 30 lines · * Juan Quintela ([email protected]) wrote:
Quoted 6 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Quoted 10 lines
Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
Now more seriously, I can (relatively easy) compile test my pull requests with:
- linux x86 (latest fedora, but I can get an older one if needed)
- linuxx86_64(latest fedor,, but the same)
- mingw64 32bit (latest fedora, but here I have the problem that Peter uses a different crosscompiler than me)
- mingw64 32bit (the same)Quoted 4 lines
But for the rest, I need to wait that somebody told me that it breaks the build. Normally it is things like
size_tis 32bit instead of 64bit or some stupid things like that, that are trivial to fix if I can compile there before doing the pull submission.I also do a FreeBSD VM, and grab an aarch64 and/or PPC bigendian host to test on.
Quoted 7 lines
Quoted 3 lines
(I could grab an ia64 host, but I don't think I could find anything to install on it that would be new enough for the rest of our build requirements).
Indeed, ia64 is a fully dead as a host architecture at this point, only interesting as a historical curiosity. Paolo already killed ia64 KVM host support in Linux git back in 2014.
Our ia64 host backend in QEMU (tcg/ia64) is still marked as maintained ... so it's maybe not as dead as you think? Or should we rather get rid of that soon, too?
Thomas
Re: Obsolete QEMU host environments (was: Re: KVM call for 2017-03-14)
Quoted 3 lines · Thomas Huth
Our ia64 host backend in QEMU (tcg/ia64) is still marked as maintained ... so it's maybe not as dead as you think? Or should we rather get rid of that soon, too?
I don't actually mind whether we keep tcg/ia64 or drop it. But if we keep it then we must have a machine we can test it on -- that means one I have access to (and which we can otherwise use for central testing), not just one the maintainer might happen to have.
Also, that MAINTAINERS entry was added in 2011, so it's probably about as out of date as the code :-)
thanks
-- PMM
Paolo Bonzini
maintainerQuoted 10 lines · Daniel P. Berrange
Quoted 6 lines
I also do a FreeBSD VM, and grab an aarch64 and/or PPC bigendian host to test on.
(I could grab an ia64 host, but I don't think I could find anything to install on it that would be new enough for the rest of our build requirements).
Indeed, ia64 is a fully dead as a host architecture at this point, only interesting as a historical curiosity. Paolo already killed ia64 KVM host support in Linux git back in 2014.
QEMU also never supported ia64 KVM, only qemu-kvm ever did.
Paolo
Quoted 1 line · Dr. David Alan Gilbert
I also do a FreeBSD VM
Do you have repro instructions for how to conveniently set this up?
thanks
-- PMM
Quoted 5 lines · * Peter Maydell ([email protected]) wrote:
Quoted 1 line · Dr. David Alan Gilbert
I also do a FreeBSD VM
Do you have repro instructions for how to conveniently set this up?
No, but given that it's the first time I've set up a BSD in ~20 years it can't have been that hard.
It's just a guest with IDE disk, rtl8139 ether.
Then I've got:
https://www.freebsd.org/doc/handbook/pkgng-intro.html
set up, so then you can just do pkg install whatever
Remember you need gmake but other than that it gets as far as moaning about the asm in the optionrom but other than that it compiles fine.
Dave
Quoted 2 lines
thanks
-- PMM
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Re: Obsolete QEMU host environments
In reply to Peter MaydellQuoted 5 lines · Peter Maydell
Quoted 3 lines · Thomas Huth
Our ia64 host backend in QEMU (tcg/ia64) is still marked as maintained ... so it's maybe not as dead as you think? Or should we rather get rid of that soon, too?
Quoted 8 lines
I don't actually mind whether we keep tcg/ia64 or drop it. But if we keep it then we must have a machine we can test it on -- that means one I have access to (and which we can otherwise use for central testing), not just one the maintainer might happen to have.
Also, that
MAINTAINERSentry was added in 2011, so it's probably about as out of date as the code :-)
Red Hat's last ia64 machine died a few years ago, so I haven't been able to test it for quite some time.
I don't know if Aurelien can still test on ia64 or not.
r~
Quoted 10 lines · Dr. David Alan Gilbert
Quoted 5 lines · * Peter Maydell ([email protected]) wrote:
Quoted 1 line · Dr. David Alan Gilbert
I also do a FreeBSD VM
Do you have repro instructions for how to conveniently set this up?
No, but given that it's the first time I've set up a BSD in ~20 years it can't have been that hard.
It's just a guest with IDE disk, rtl8139 ether.
freebsd ships at least virtio-blk and virtio-net drivers too (not sure about virtio-scsi), so no need to go with emulated devices.
cheers,
Gerd
Quoted 19 lines · Peter Maydell
Quoted 18 lines · Juan Quintela
Quoted 16 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
Quoted 8 lines
Quoted 3 lines
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
OSX is actually in the set that's OK because I have a machine I can test on. The ones that are problems are all the BSDs, AIX, Solaris, Haiku, and architectures sparc, mips, ia64, s390.
Peter,
if you need an s390 box, you can register for a virtual machine at
https://developer.ibm.com/linuxone/
If you have an account let me know the details and I will try to find the right people in IBM to extend the 120 day testing period to unlimited.
Re: Obsolete QEMU host environments
In reply to Richard HendersonQuoted 12 lines · Richard Henderson
Quoted 11 lines · Peter Maydell
Quoted 3 lines · Thomas Huth
Our ia64 host backend in QEMU (tcg/ia64) is still marked as maintained ... so it's maybe not as dead as you think? Or should we rather get rid of that soon, too?
I don't actually mind whether we keep tcg/ia64 or drop it. But if we keep it then we must have a machine we can test it on -- that means one I have access to (and which we can otherwise use for central testing), not just one the maintainer might happen to have.
Quoted 5 lines
Quoted 2 lines
Also, that
MAINTAINERSentry was added in 2011, so it's probably about as out of date as the code :-)Red Hat's last ia64 machine died a few years ago, so I haven't been able to test it for quite some time.
AFAIK, HP are the only major vendor who still sell new ia64 machines and that's high end stuff costing many $$$$$. Even they are pushing people to x86_64 unless they need ia64 for legacy reasons. Otherwise ebay and other secondhand marketplaces are the only way to get hold of ia64.
Regards,
Daniel
Signature · 3 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
Re: Obsolete QEMU host environments
Quoted 16 lines · Daniel P. Berrange
Quoted 15 lines · Richard Henderson
Quoted 13 lines · Peter Maydell
Quoted 3 lines · Thomas Huth
Our ia64 host backend in QEMU (tcg/ia64) is still marked as maintained ... so it's maybe not as dead as you think? Or should we rather get rid of that soon, too?
I don't actually mind whether we keep tcg/ia64 or drop it. But if we keep it then we must have a machine we can test it on -- that means one I have access to (and which we can otherwise use for central testing), not just one the maintainer might happen to have.
Also, that
MAINTAINERSentry was added in 2011, so it's probably about as out of date as the code :-)
Quoted 7 lines
Quoted 2 lines
Red Hat's last ia64 machine died a few years ago, so I haven't been able to test it for quite some time.
AFAIK, HP are the only major vendor who still sell new ia64 machines and that's high end stuff costing many $$$$$. Even they are pushing people to
x86_64unless they need ia64 for legacy reasons. Otherwise ebay and other secondhand marketplaces are the only way to get hold of ia64.
Unless Aurelien speaks up and says that it is still maintained, I'd say we add a big fat "ia is deprecated and going to be removed" warning to the configure script once we left the freeze state. If nobody then speaks up within the next two or three releases of QEMU, we remove the ia64 support from QEMU.
Thomas
Quoted 19 lines · On Tue, 14 Mar 2017 11:56:36 +0100 Peter Maydell <[email protected]>…
Quoted 18 lines · Juan Quintela
Quoted 16 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
Quoted 7 lines
Quoted 3 lines
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
OSX is actually in the set that's OK because I have a machine I can test on. The ones that are problems are all the BSDs, AIX, Solaris, Haiku, and architectures
The most relevant links I could find about AIX hosts only mention QEMU 0.9.1:
http://www.perzl.org/aix/index.php?n=Main.Qemu
http://www.vivier.eu/Qemu/
I'm not aware of any effort within IBM to support newer versions of QEMU on an AIX host (Cc'ing Laurent in case he would be aware of a non-IBM initiative).
I can maybe try to grab an AIX system and see what happens. I'm okay to run the deprecation process if we choose to go that way.
Cheers.
--
Greg
Quoted 5 lines
sparc, mips, ia64, s390.
thanks
-- PMM
Quoted 33 lines · Greg Kurz
Quoted 19 lines · On Tue, 14 Mar 2017 11:56:36 +0100 Peter Maydell <[email protected]>…
Quoted 18 lines · Juan Quintela
Quoted 16 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
Quoted 7 lines
Quoted 3 lines
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
OSX is actually in the set that's OK because I have a machine I can test on. The ones that are problems are all the BSDs, AIX, Solaris, Haiku, and architectures
The most relevant links I could find about AIX hosts only mention QEMU 0.9.1:
Quoted 7 lines
http://www.perzl.org/aix/index.php?n=Main.Qemu
http://www.vivier.eu/Qemu/I'm not aware of any effort within IBM to support newer versions of QEMU on an AIX host (Cc'ing Laurent in case he would be aware of a non-IBM initiative).
At this time it was a personal initiative, made possible because I have worked at Bull on the port of GNOME to AIX and had access to some Bull AIX Servers.
Now this effort has been moved to the AIX Toolbox for Linux Applications:
http://www-03.ibm.com/systems/power/software/aix/linux/
Perhaps you can try to contact someone inside IBM about this toolbox?
Laurent
Re: Obsolete QEMU host environments
In reply to Richard HendersonQuoted 12 lines · Richard Henderson
Quoted 11 lines · Peter Maydell
Quoted 3 lines · Thomas Huth
Our ia64 host backend in QEMU (tcg/ia64) is still marked as maintained ... so it's maybe not as dead as you think? Or should we rather get rid of that soon, too?
I don't actually mind whether we keep tcg/ia64 or drop it. But if we keep it then we must have a machine we can test it on -- that means one I have access to (and which we can otherwise use for central testing), not just one the maintainer might happen to have.
Quoted 7 lines
Quoted 2 lines
Also, that
MAINTAINERSentry was added in 2011, so it's probably about as out of date as the code :-)Red Hat's last ia64 machine died a few years ago, so I haven't been able to test it for quite some time.
I don't know if Aurelien can still test on ia64 or not.
I have the same issue, the last QEMU test I have been able to do on an ia64 machine dates from last summer, so something like 9 months ago. I don't think anybody has a lot of interest in ia64 anymore, so I guess it's time to just remove the ia64 host backend.
Aurelien
Signature · 2 lines
Aurelien Jarno GPG: 4096R/1DDD8C9B [email protected] http://www.aurel32.net
Quoted 42 lines · On Wed, 15 Mar 2017 12:25:38 +0100 Laurent Vivier <[email protected]> wrote:
Quoted 33 lines · Greg Kurz
Quoted 19 lines · On Tue, 14 Mar 2017 11:56:36 +0100 Peter Maydell <[email protected]>…
Quoted 18 lines · Juan Quintela
Quoted 16 lines · Peter Maydell
Quoted 4 lines · Stefan Hajnoczi
On Mon, Mar 13, 2017 at 11:02:01AM +0100, Peter Maydell wrote:
The minimum requirements for the new language:
1. Does it support the host operating systems that QEMU runs on?
2. Does it support the host architectures that QEMU runs on?Speaking of this, I was thinking that we should introduce a rule that for any host OS/arch we support we must have a build machine so we can at least do a compile test. For instance if you believe configure we support Solaris and AIX, but I bet they're bit-rotting. The ia64 backend has to be a strong candidate for being dumped too. Demanding "system we can test on or we drop support" would let us more clearly see what we're actually running on and avoid unnecessarily ruling things out because they don't support Itanium or AIX...
Quoted 7 lines
Quoted 3 lines
YES, YES and YES.
I demand an osX build machine NOW!!!! Remote access is ok.
OSX is actually in the set that's OK because I have a machine I can test on. The ones that are problems are all the BSDs, AIX, Solaris, Haiku, and architectures
The most relevant links I could find about AIX hosts only mention QEMU 0.9.1:
Quoted 7 lines
http://www.perzl.org/aix/index.php?n=Main.Qemu
http://www.vivier.eu/Qemu/I'm not aware of any effort within IBM to support newer versions of QEMU on an AIX host (Cc'ing Laurent in case he would be aware of a non-IBM initiative).
Quoted 8 lines
At this time it was a personal initiative, made possible because I have worked at Bull on the port of GNOME to AIX and had access to some Bull AIX Servers.
Now this effort has been moved to the AIX Toolbox for Linux Applications:
Heh :)
Quoted 2 lines
Perhaps you can try to contact someone inside IBM about this toolbox?
Not even sure if it's worth the pain as I could not find any trace of QEMU in the download pages:
http://www-03.ibm.com/systems/power/software/aix/linux/toolbox/alpha.html
https://ftp.software.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/
Cheers.
--
Greg
Quoted 3 lines
Laurent