8/8 Create MMU 2/3 level accessors in the sub-arch layer (i386)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Zachary Amsden
i386 Transparent paravirtualization sub-arch patch #8.
Transparent paravirtualization support for MMU operations.
All operations which update live page table entries have been moved to the sub-architecture layer. Unfortunately, this required yet another parallel set of
pgtable-Nlevel-ops.hfiles, but this avoids the ugliness of having to use #ifdef's all of the code.This is pure code motion. Anything else would be a bug.
I think that patch is really ugly - it makes hacking VM on i386 even more painful than it already is because the convolutes the file structure even more. Hope it is not applied.
-Andi
Quoted 11 lines · Andi Kleen
Quoted 9 lines · Zachary Amsden
i386 Transparent paravirtualization sub-arch patch #8.
Transparent paravirtualization support for MMU operations.
All operations which update live page table entries have been moved to the sub-architecture layer. Unfortunately, this required yet another parallel set of
pgtable-Nlevel-ops.hfiles, but this avoids the ugliness of having to use #ifdef's all of the code.
Quoted 5 lines
Quoted 1 line
This is pure code motion. Anything else would be a bug.
I think that patch is really ugly - it makes hacking VM on i386 even more painful than it already is because the convolutes the file structure even more. Hope it is not applied.
Especially as there's been no user shown for it, similar to all the other ugly patches from vmware.
Quoted 6 lines
Quoted 3 lines
I think that patch is really ugly - it makes hacking VM on i386 even more painful than it already is because the convolutes the file structure even more. Hope it is not applied.
Especially as there's been no user shown for it, similar to all the other ugly patches from vmware.
Well, some of it can be counted as cleanup or even tuning like the excellent switch_to patch. But not that one and some of the more intrusive patches.
-Andi
Quoted 1 line
Quoted 9 lines · Andi Kleen
Quoted 6 lines
Quoted 3 lines
I think that patch is really ugly - it makes hacking VM on i386 even more painful than it already is because the convolutes the file structure even more. Hope it is not applied.
Especially as there's been no user shown for it, similar to all the other ugly patches from vmware.
Well, some of it can be counted as cleanup or even tuning like the excellent
switch_topatch. But not that one and some of the more intrusive patches.
Yeah, I said ugly ones specificly. There's been some nice previous ones, but most in this series (all the move of stuff to subarches) are rather horrible and lack lots of explanation.
Quoted 12 lines · Christoph Hellwig
Quoted 9 lines · Andi Kleen
Quoted 9 lines
Quoted 5 lines
I think that patch is really ugly - it makes hacking VM on i386 even more painful than it already is because the convolutes the file structure even more. Hope it is not applied.
Especially as there's been no user shown for it, similar to all the other ugly patches from vmware.
Quoted 10 lines
Quoted 4 lines
Well, some of it can be counted as cleanup or even tuning like the excellent
switch_topatch. But not that one and some of the more intrusive patches.Yeah, I said ugly ones specificly. There's been some nice previous ones, but most in this series (all the move of stuff to subarches) are rather horrible and lack lots of explanation.
All of my previous patches have been aimed at fixing bugs, improving performance, reliability and maintinability of the i386 architecture. If you found something that didn't fit one of those categories in my previous patches, then it is either not well enough explained or perhaps inadvertently slipped through from one of my more radical trees - or it could be a bug.
There is a simple explanation for all of this series. The goal is to move all privileged instructions, sensitive instructions, and privilege awareness into a layer where it can be overridden by new code without disrupting the default architecture. On x86, there are a lot of instructions - popf, iret, sgdt, and others which behave differently under different privilege levels, but do not trap. These architectural features must be redefined by any architecture which virtualizes the x86, be it Xen, UML, or an alternative approach. Similarly, certain privileged processor data structures (page tables, descriptor tables) must be protected and accessed in a different manner if one is to utilize the principles of paravirtualization to achieve high performance inside of a virtual machine. I believe this series of patches is one almost clean solution to this. There are obvious problems with the MMU patch, and I'm still trying to come up with a way to properly address that.
That said, I am definitely seeking any feedback you have on how to achieve this goal while being as clean and maintainable as possible - if the Linux community is indeed interested in adopting a paravirtualization approach. Looking from the most general view possible, there are a couple of ways to do this in Linux:
1) Create a new architecture. This is the UML approach, and while it has been successful there, it is difficult to maintain closeness to the hardware architecture without introducing a maintenance burden. This closeness is desirable if one is to achieve high performance and take advantage of more processor specific features.
2) Use the sub-architecture strategy of x86. This approach has a relatively small set of code movement to allow a new virtualized sub-architecture to redefine the privileged and sensitive operations of the processor, as well as to implement easily defined architectural hints which employ higher level virtualization strategies.
3) Use #ifdef'd include/asm-i386 header files. While workable, this has flaws - it is ugly, and it causes the hypervisor header files to sneak into include/asm-i386 rather rapidly destroying maintainability for the native code. This has been attempted before, but if someone were to send those patches to LKML, I would expect them to be rapidly tarred and feathered.
4) Clone entire asm-i386 header files and replace them using an include path, potentially in the sub-architecture level. While this avoids any diffs at all to the native asm-i386 headers, it needlessly duplicates a lot of code, and this creates a greater maintenance burden for somebody. Who that body is can be determined later, but this creates a lot of extra work for that unfortunate person that is wasted time that could easily be spent bettering Linux!
5) Use the HAVE_SUBARCH_PTE_ACCESSOR type approach (similar to the way generic optimizable PTE operations are defined in asm-generic). I have not yet investigated the feasibility of this type of approach, but it seems workable. At least for the MMU patch, some combination of this and other techniques might help make things a lot cleaner.
Do you have ideas? I'm open to all suggestions here. The only goal I have is to make high performance virtualization support in Linux the least disruptive event possible for all parties. Although I'm a bit biased towards i386 from a coding point of view, some of these ideas can cross architecture boundaries as well, so I'm open to feedback from all parties.
Thanks,
Zach
Quoted 9 lines · Zachary Amsden
Quoted 6 lines
Yeah, I said ugly ones specificly. There's been some nice previous ones, but most in this series (all the move of stuff to subarches) are rather horrible and lack lots of explanation.
All of my previous patches have been aimed at fixing bugs, improving performance, reliability and maintinability of the i386 architecture.
Yup, with one or two semi-exceptions, all the patches up to this series seem to be good general cleanups - certainly it's good to move all those open-coded asm statements into single-site inlines and macros: people keep on screwing them up.
We do need to wake the Xen poeple up, make sure that these changes suit them as well, or at least don't screw them over (hard to see how it could though).
Quoted 5 lines
If you found something that didn't fit one of those categories in my previous patches, then it is either not well enough explained or perhaps inadvertently slipped through from one of my more radical trees - or it could be a bug.
Quoted 14 lines
There is a simple explanation for all of this series. The goal is to move all privileged instructions, sensitive instructions, and privilege awareness into a layer where it can be overridden by new code without disrupting the default architecture. On x86, there are a lot of instructions - popf, iret, sgdt, and others which behave differently under different privilege levels, but do not trap. These architectural features must be redefined by any architecture which virtualizes the x86, be it Xen, UML, or an alternative approach. Similarly, certain privileged processor data structures (page tables, descriptor tables) must be protected and accessed in a different manner if one is to utilize the principles of paravirtualization to achieve high performance inside of a virtual machine. I believe this series of patches is one almost clean solution to this. There are obvious problems with the MMU patch, and I'm still trying to come up with a way to properly address that.
Yes this later series is in a different category and will require more review/coordination/discussion/waking-up from the Xen team.
Quoted 4 lines · * Andrew Morton ([email protected]) wrote:
Yup, with one or two semi-exceptions, all the patches up to this series seem to be good general cleanups - certainly it's good to move all those open-coded asm statements into single-site inlines and macros: people keep on screwing them up.
I agree.
Quoted 3 lines
We do need to wake the Xen poeple up, make sure that these changes suit them as well, or at least don't screw them over (hard to see how it could though).
I have a series of similar patches that I've done for Xen that I'll post shortly.
thanks,
-chris
Quoted 15 lines · Andrew Morton
Quoted 11 lines · Zachary Amsden
Quoted 7 lines
Yeah, I said ugly ones specificly. There's been some nice previous ones,
but most in this series (all the move of stuff to subarches) are rather horrible and lack lots of explanation.
All of my previous patches have been aimed at fixing bugs, improving performance, reliability and maintinability of the i386 architecture.
Quoted 10 lines
Yup, with one or two semi-exceptions, all the patches up to this series seem to be good general cleanups - certainly it's good to move all those open-coded asm statements into single-site inlines and macros: people keep on screwing them up.
We do need to wake the Xen poeple up, make sure that these changes suit them as well, or at least don't screw them over (hard to see how it could though).
This patch in particular is still quite controversial. I know at least Andi has objections (quite valid) to the way PAE/non-PAE was dissected, and I would definitely like to address these concerns. Although I have no objection to you committing it to the mm tree right now, please be advised that Chris Wright and I will have to converge quite a bit on this patch, and will likely be doing a substantial amount of rework here to work out Xen compatibilty issues as well as general cleanliness. If it is more convenient for you to live without that churn, by all means feel free to, and we can update the patch once everyone is happy.
Zach