paravirt: refactor struct paravirt_ops into smaller pv_*_ops
Subsystems (3)
RFC PATCH 1/2 paravirt: refactor struct paravirt_ops into smaller pv_*_ops
[ I think this is a straight repost this patch, which addresses all the previous comments. I'd like to submit this for .24 as the basis for a unified paravirt_ops. Any objections? ]
This patch refactors the paravirt_ops structure into groups of functionally related ops:
pv_info - random info, rather than function entrypoints pv_init_ops - functions used at boot time (some for module_init too) pv_misc_ops - lazy mode, which didn't fit well anywhere else pv_time_ops - time-related functionspv_cpu_ops - various privileged instruction ops pv_irq_ops - operations for managing interrupt state pv_apic_ops - APIC operationspv_mmu_ops - operations for managing pagetables
There are several motivations for this:
1. Some of these ops will be general to all x86, and some will be i386/x86-64 specific. This makes it easier to share common stuff while allowing separate implementations where needed.
2. At the moment we must export all of paravirt_ops, but modules only need selected parts of it. This allows us to export on a case by case basis (and also choose which export license we want to apply).
3. Functional groupings make things a bit more readable.
struct paravirt_ops still exists as a container for the pv_* structures; this is mainly so that each function pointer has a unique offset for use as a patch-site identifier.
--- arch/i386/kernel/alternative.c | 4 arch/i386/kernel/asm-offsets.c | 12 arch/i386/kernel/entry.S | 2 arch/i386/kernel/paravirt.c | 354 ++++++++++++++------------ arch/i386/kernel/vmi.c | 168 ++++++------ arch/i386/xen/enlighten.c | 152 ++++++----- include/asm-i386/paravirt.h | 433 +++++++++++++++++--------------- include/asm-i386/pgtable-3level-defs.h | 2 8 files changed, 622 insertions(+), 505 deletions(-)
===================================================================
arch/i386/kernel/alternative.c +2 -2
@@ -368,8 +368,8 @@ void apply_paravirt(struct paravirt_patcBUG_ON (p->len >MAX_PATCH_LEN );/* prep the buffer with the original instructions */ memcpy (insnbuf, p->instr, p->len); - used = paravirt_ops.patch (p->instrtype, p->clobbers, insnbuf, - (unsigned long )p->instr, p->len); + used = pv_init_ops.patch (p->instrtype, p->clobbers, insnbuf, + (unsigned long )p->instr, p->len);BUG_ON (used > p->len);
===================================================================
arch/i386/kernel/asm-offsets.c +6 -6
@@ -116,12 +116,12 @@ void foo(void)#ifdef CONFIG_PARAVIRT BLANK (); -OFFSET (PARAVIRT_enabled, paravirt_ops, paravirt_enabled); -OFFSET (PARAVIRT_irq_disable, paravirt_ops, irq_disable); -OFFSET (PARAVIRT_irq_enable, paravirt_ops, irq_enable); -OFFSET (PARAVIRT_irq_enable_sysexit, paravirt_ops, irq_enable_sysexit); -OFFSET (PARAVIRT_iret, paravirt_ops, iret); -OFFSET (PARAVIRT_read_cr0, paravirt_ops, read_cr0); +OFFSET (PARAVIRT_enabled, pv_info, paravirt_enabled); +OFFSET (PARAVIRT_irq_disable, paravirt_ops, pv_irq_ops.irq_disable); +OFFSET (PARAVIRT_irq_enable, paravirt_ops, pv_irq_ops.irq_enable); +OFFSET (PARAVIRT_irq_enable_sysexit, paravirt_ops, pv_cpu_ops.irq_enable_sysexit); +OFFSET (PARAVIRT_iret, paravirt_ops, pv_cpu_ops.iret); +OFFSET (PARAVIRT_read_cr0, paravirt_ops, pv_cpu_ops.read_cr0);#endif #ifdef CONFIG_XEN
===================================================================
arch/i386/kernel/entry.S +1 -1
@@ -434,7 +434,7 @@ ldt_ss: * is still available to implement the setting of the high * 16-bits in the INTERRUPT_RETURN paravirt-op. */ - cmpl $0, paravirt_ops+PARAVIRT_enabled + cmpl $0, pv_info+PARAVIRT_enabled jne restore_nocheck #endif
===================================================================
arch/i386/kernel/paravirt.c +198 -156
425 lines not on this page · raw message
===================================================================
arch/i386/kernel/vmi.c +84 -84
267 lines not on this page · raw message
===================================================================
arch/i386/xen/enlighten.c +88 -64
255 lines not on this page · raw message
===================================================================
include/asm-i386/paravirt.h +242 -191
941 lines not on this page · raw message
===================================================================
include/asm-i386/pgtable-3level-defs.h +1 -1
@@ -2,7 +2,7 @@#define _I386_PGTABLE_3LEVEL_DEFS_H#ifdef CONFIG_PARAVIRT -#define SHARED_KERNEL_PMD (paravirt_ops.shared_kernel_pmd) +#define SHARED_KERNEL_PMD (pv_info.shared_kernel_pmd)#else #define SHARED_KERNEL_PMD 1#endif
Quoted 3 lines · On Wednesday 10 October 2007 04:24:24 Jeremy Fitzhardinge wrote:
[ I think this is a straight repost this patch, which addresses all the previous comments. I'd like to submit this for .24 as the basis for a unified
paravirt_ops. Any objections? ]
Thanks Jeremy, I've actually taken time to finally review this in detail (I'm assuming you'll refactor as necessary after the x86 arch merger).
arch/i386/kernel/asm-offsets.c void foo(void) · line 119
Quoted 1 line
arch/i386/kernel/asm-offsets.c +1
-0+OFFSET (PARAVIRT_enabled, pv_info, paravirt_enabled);
I think this gives the right answer for the wrong reasons?
arch/i386/kernel/paravirt.c line 38
Quoted 2 lines
arch/i386/kernel/paravirt.c +2
-0+struct paravirt_ops paravirt_ops; +
Do you actually need to define this? See below...
Quoted 1 line
+DEF_NATIVE(, ud2a, "ud2a");
Hmm, that's ugly. It was ugly before, but it's uglier now. Maybe just use "unsigned char ud2a[] = { 0x0f, 0x0b };" in paravirt_patch_default?
Quoted 3 lines
}
struct
paravirt_opsparavirt_ops= {
...
arch/i386/kernel/paravirt.c int paravirt_disable_iospace(void) · line 276
Quoted 6 lines
arch/i386/kernel/paravirt.c +6
-0+ .pv_info = { + .name = "bare hardware", + .paravirt_enabled = 0, + .kernel_rpl = 0, + .shared_kernel_pmd = 1, /* Only used when CONFIG_X86_PAE is set */ + },
This is the bit I don't get. Why not just declare struct pv_info pvinfo, etc, and use the declaration of struct paravirt_ops to get your unique offset-based identifiers for patching?
Rest looks fine...
Thanks!
Rusty.
Quoted 5 lines · Rusty Russell
arch/i386/kernel/asm-offsets.c void foo(void) · line 119
Quoted 2 lines
arch/i386/kernel/asm-offsets.c +1
-0+OFFSET (PARAVIRT_enabled, pv_info, paravirt_enabled);I think this gives the right answer for the wrong reasons?
Actually, it was OK. The only use is in entry.S, which uses pv_info + PARAVIRT_enabled.
J
Huh, thought I did a more complete reply to this. Must have farted on it.
Quoted 3 lines · Rusty Russell
Thanks Jeremy, I've actually taken time to finally review this in detail (I'm assuming you'll refactor as necessary after the x86 arch merger).
Yep.
Quoted 7 lines
arch/i386/kernel/paravirt.c line 38
Quoted 3 lines
arch/i386/kernel/paravirt.c +2
-0+struct paravirt_ops paravirt_ops; +Do you actually need to define this? See below...
Quoted 6 lines
Quoted 2 lines
+DEF_NATIVE(, ud2a, "ud2a");
Hmm, that's ugly. It was ugly before, but it's uglier now. Maybe just use "unsigned char ud2a[] = {
0x0f,0x0b};" inparavirt_patch_default?
Yeah, its not pretty. I'll have another go.
Quoted 6 lines
Quoted 4 lines
}
struct
paravirt_opsparavirt_ops= {...
Quoted 12 lines
arch/i386/kernel/paravirt.c int paravirt_disable_iospace(void) · line 276
Quoted 7 lines
arch/i386/kernel/paravirt.c +6
-0+ .pv_info = { + .name = "bare hardware", + .paravirt_enabled = 0, + .kernel_rpl = 0, + .shared_kernel_pmd = 1, /* Only used when CONFIG_X86_PAE is set */ + },This is the bit I don't get. Why not just declare struct
pv_infopvinfo, etc, and use the declaration of structparavirt_opsto get your unique offset-based identifiers for patching?
Given an op id number in .parainstructions, the patching code needs to be able to index into something to get the corresponding function pointer. If each pv_* structure is its own little unrelated structure, then the id has to be a <structure, id> tuple, which just complicates things. If I pack them all into a single structure then it becomes a simple offset calculation.
That said, there's no need for pv_info to be in that structure, since it contains no function pointers. I'll move it out.
J
Quoted 15 lines · On Thursday 11 October 2007 04:02:50 Jeremy Fitzhardinge wrote:
Quoted 4 lines
Quoted 1 line
struct
paravirt_opsparavirt_ops= {...
Quoted 10 lines
arch/i386/kernel/paravirt.c int paravirt_disable_iospace(void) · line 276
Quoted 6 lines
arch/i386/kernel/paravirt.c +6
-0+ .pv_info = { + .name = "bare hardware", + .paravirt_enabled = 0, + .kernel_rpl = 0, + .shared_kernel_pmd = 1, /* Only used when CONFIG_X86_PAE is set */ + },This is the bit I don't get. Why not just declare struct
pv_infopvinfo, etc, and use the declaration of structparavirt_opsto get your unique offset-based identifiers for patching?
Quoted 6 lines
Given an op id number in .parainstructions, the patching code needs to be able to index into something to get the corresponding function pointer. If each pv_* structure is its own little unrelated structure, then the id has to be a <structure, id> tuple, which just complicates things. If I pack them all into a single structure then it becomes a simple offset calculation.
Sure, but this can actually be a temporary thing inside the patch code (or at
least static to that file if it's too big for the stack).
struct paravirt_ops patch_template = { .pv_info = pv_info, .pv_cpu_ops =pv_cpu_ops, ... };
Then you can even rename struct paravirt_ops to "struct patch_template" and
we're well on the way to making this a generic function-call patching mechanism, rather than something paravirt-specific.
Hope that clarifies my thinking...
Rusty.
Quoted 10 lines · Rusty Russell
Sure, but this can actually be a temporary thing inside the patch code (or at
least static to that file if it's too big for the stack).struct
paravirt_opspatch_template= { .pv_info =pv_info, .pv_cpu_ops =pv_cpu_ops, ... };Then you can even rename struct
paravirt_opsto "structpatch_template" and
we're well on the way to making this a generic function-call patching mechanism, rather than something paravirt-specific.
Hm, I see. I'm not quite sure that's the best way to achieve a generic result, but I see your point.
Quoted 1 line
Hope that clarifies my thinking...
Well, I'd agree with making the code more generic if another user appears, but I'd rather not do it prematurely.
Sorry, I forgot to update lguest. I'll do that and repost (but I won't have had a chance to test it).
Are you otherwise happy with the patch in its current form? And are you happy with the lazymode changes?
J
Quoted 2 lines · On Saturday 13 October 2007 05:16:04 Jeremy Fitzhardinge wrote:
Are you otherwise happy with the patch in its current form? And are you happy with the lazymode changes?
No, and yes.
This applies on top of your 1/2. Boots here, runs lguest. It also exports pv_info: lg.ko checks it to avoid lguest-in-lguest.
Cheers,
Rusty.
==
Wean off "struct paravirt_ops" entirely: it's now merely a patching enumeration tool
arch/i386/kernel/asm-offsets.c +7 -5
@@ -117,11 +117,13 @@ void foo(void)#ifdef CONFIG_PARAVIRT BLANK ();OFFSET (PARAVIRT_enabled, pv_info, paravirt_enabled); -OFFSET (PARAVIRT_irq_disable, paravirt_ops, pv_irq_ops.irq_disable); -OFFSET (PARAVIRT_irq_enable, paravirt_ops, pv_irq_ops.irq_enable); -OFFSET (PARAVIRT_irq_enable_sysexit, paravirt_ops, pv_cpu_ops.irq_enable_sysexit); -OFFSET (PARAVIRT_iret, paravirt_ops, pv_cpu_ops.iret); -OFFSET (PARAVIRT_read_cr0, paravirt_ops, pv_cpu_ops.read_cr0); +OFFSET (PARAVIRT_PATCH_pv_cpu_ops, paravirt_patch_template, pv_cpu_ops); +OFFSET (PARAVIRT_PATCH_pv_irq_ops, paravirt_patch_template, pv_irq_ops); +OFFSET (PV_IRQ_irq_disable, pv_irq_ops, irq_disable); +OFFSET (PV_IRQ_irq_enable, pv_irq_ops, irq_enable); +OFFSET (PV_CPU_iret, pv_cpu_ops, iret); +OFFSET (PV_CPU_irq_enable_sysexit, pv_cpu_ops, irq_enable_sysexit); +OFFSET (PV_CPU_read_cr0, pv_cpu_ops, read_cr0);#endif #ifdef CONFIG_XEN
arch/i386/kernel/paravirt.c +135 -145
313 lines not on this page · raw message
drivers/char/hvc_lguest.c +1 -1
@@ -115,7 +115,7 @@ static struct hv_ops lguest_cons = {* (0), and the struct hv_ops containing the put_chars() function. */ static int __init cons_init (void ) { -if (strcmp (paravirt_ops.name,"lguest" ) !=0 ) +if (strcmp (pv_info.name,"lguest" ) !=0 )return 0 ;return hvc_instantiate (0 ,0 , &lguest_cons);
drivers/lguest/core.c +2 -2
@@ -248,8 +248,8 @@ static void unmap_switcher(void) }/*H:130 Our Guest is usually so well behaved; it never tries to do things it -* isn't allowed to. Unfortunately, "struct paravirt_ops" isn't quite -* complete, because it doesn't contain replacements for the Intel I/O +* isn't allowed to. Unfortunately, Linux's paravirtual infrastructure isn't +* quite complete, because it doesn't contain replacements for the Intel I/O * instructions. As a result, the Guest sometimes fumbles across one during * the boot process as it probes for various things which are usually attached * to a PC.
drivers/lguest/lguest.c +7 -7
@@ -23,7 +23,7 @@* * So how does the kernel know it's a Guest? The Guest starts at a special * entry point marked with a magic string, which sets up a few things then -* calls here. We replace the native functions in "struct paravirt_ops" +* calls here. We replace the native functions various "paravirt" structures * with our Guest versions, then boot like normal. :*/ /*
@@ -331,7 +331,7 @@ static void lguest_load_tls(struct threa }/*G:038 That's enough excitement for now, back to ploughing through each of -* the paravirt_ops (we're about 1/3 of the way through). +* the different pv_ops structures (we're about 1/3 of the way through). * * This is the Local Descriptor Table, another weird Intel thingy. Linux only * uses this for some strange applications like Wine. We don't do anything
@@ -558,7 +558,7 @@ static void lguest_set_pte(pte_t *ptep,lazy_hcall (LHCALL_FLUSH_TLB ,1 ,0 ,0 ); } -/* Unfortunately for Lguest, the paravirt_ops for page tables were based on +/* Unfortunately for Lguest, the pv_mmu_ops for page tables were based on * native page table operations. On native hardware you can set a new page * table entry whenever you want, but if you want to remove one you have to do * a TLB flush (a TLB is a little cache of page table entries kept by the CPU).
@@ -902,7 +902,7 @@ static __init char *lguest_memory_setup(/*G:050 * Patching (Powerfully Placating Performance Pedants) * -* We have already seen that "struct paravirt_ops" lets us replace simple +* We have already seen that pv_ops structures let us replace simple * native instructions with calls to the appropriate back end all throughout * the kernel. This allows the same kernel to run as a Guest and as a native * kernel, but it's slow because of all the indirect branches.
@@ -957,9 +957,9 @@ static unsigned lguest_patch(u8 type, u1return insn_len; } -/*G:030 Once we get to lguest_init(), we know we're a Guest. The paravirt_ops -* structure in the kernel provides a single point for (almost) every routine -* we have to override to avoid privileged instructions. */ +/*G:030 Once we get to lguest_init(), we know we're a Guest. The pv_ops +* structures in the kernel provide points for (almost) every routine we have +* to override to avoid privileged instructions. */ __init void lguest_init (void *boot) {/* Copy boot parameters first: the Launcher put the physical location
include/asm-i386/paravirt.h +22 -19
93 lines not on this page · raw message
Quoted 11 lines · Rusty Russell
Quoted 3 lines · On Saturday 13 October 2007 05:16:04 Jeremy Fitzhardinge wrote:
Are you otherwise happy with the patch in its current form? And are you happy with the lazymode changes?
No, and yes.
This applies on top of your 1/2. Boots here, runs lguest. It also exports
pv_info: lg.ko checks it to avoid lguest-in-lguest.
Yeah, OK, I can live with that. I'll fold it into the base patch, and update the lazymode patch accordingly.
J