Xen-paravirt_ops: Xen guest implementation for paravirt_ops interface
Subsystems (18)
cover Xen-paravirt_ops: Xen guest implementation for paravirt_ops interface
Hi Andi,
This patch series implements the Linux Xen guest as a paravirt_ops backend. The features in implemented this patch series are:
* domU only
* UP only (most code is SMP-safe, but there's no way to create a new vcpu)
* writable pagetables, with late pinning/early unpinning
(no shadow pagetable support)
* supports both PAE and non-PAE modes
* xen hvc console (console=hvc0)
* virtual block device (blockfront)
* virtual network device (netfront)
The patch series is in three parts:
1-2 Cleanup patches to various parts of the kernel
3-15 Extensions to the core code and/or paravirt ops, needed to support
Xen. Includes hooks into get/setting pte values and the new patching machinery.
16-26 The Xen paravirt_ops implementation itself.
(Some of the earlier patches in the series have already been posted,
but are included to make the series self-contained.)
I've tried to make each patch as self-explanatory as possible. The series is based on 2.6.21-rc2-git1.
Changes since the previous posting:
- Moved the arch_*_mmap stubs to asm-generic/mm_hooks.h, and included that in asm-*/mmu_context.h (James Bottomley)
- Use unsigned long consistently for addresses (Andi Kleen)
- Moved xenboot_console declaration to its own header. (Andi Kleen)
- Renamed FIX_PARAVIRT to FIX_PARAVIRT_BOOTMAP to emphasize that its intended for bootstrap only (Ingo Molnar)
- Cleaned up headers a fair bit. Moved a lot of the native_* functions to their "home" headers, so that they can be used either for native paravirt boots, or when CONFIG_PARAVIRT is disabled. This cuts down on duplication, and makes paravirt.h simpler.
- Various cleanups from Adrian Bunk
Thanks,
J
PATCH 1/26 Xen-paravirt_ops: Fix typo in sync_constant_test_bit()s name.
--- include/asm-i386/sync_bitops.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
===================================================================
include/asm-i386/sync_bitops.h +1 -1
@@ -130,7 +130,7 @@ static inline int sync_test_and_change_breturn oldbit; } -static __always_inline int sync_const_test_bit(int nr, const volatile unsigned long *addr) +static __always_inline int sync_constant_test_bit(int nr, const volatile unsigned long *addr) {return ((1UL << (nr &31 )) & (((const volatile unsigned int *)addr)[nr >>5 ])) !=0 ;
Re: [patch 01/26] Xen-paravirt_ops: Fix typo in sync_constant_test_bit()s name.
Quoted 1 line · Jeremy Fitzhardinge
Signed-off-by Jeremy Fitzhardinge <[email protected]>
Ingo
PATCH 2/26 Xen-paravirt_ops: ignore vgacon if hardware not present
In reply to Jeremy FitzhardingeAvoid trying to set up vgacon if there's no vga hardware present.
From: Gerd Hoffmann <[email protected]>
--- drivers/video/console/vgacon.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
===================================================================
drivers/video/console/vgacon.c +2 -1
@@ -372,7 +372,8 @@ static const char *vgacon_startup(void) }/* VGA16 modes are not handled by VGACON */ - if ((ORIG_VIDEO_MODE == 0x0D) || /* 320x200/4 */ + if ((ORIG_VIDEO_MODE == 0x00) || /* SCREEN_INFO not initialized */ + (ORIG_VIDEO_MODE == 0x0D) || /* 320x200/4 */ (ORIG_VIDEO_MODE == 0x0E) || /* 640x200/4 */ (ORIG_VIDEO_MODE == 0x10) || /* 640x350/4 */ (ORIG_VIDEO_MODE == 0x12) || /* 640x480/4 */
Re: [patch 02/26] Xen-paravirt_ops: ignore vgacon if hardware not present
Quoted 1 line · Jeremy Fitzhardinge
Avoid trying to set up vgacon if there's no vga hardware present.
Ingo
PATCH 3/26 Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
In reply to Jeremy FitzhardingeAdd a _paravirt_nop function for use as a stub for no-op operations, and paravirt_nop #defined void * version to make using it easier (since all its uses are as a void *).
This is useful to allow the patcher to automatically identify noop operations so it can simply nop out the callsite.
--- arch/i386/kernel/paravirt.c | 22 +++++++++++----------- include/asm-i386/paravirt.h | 3 +++ 2 files changed, 14 insertions(+), 11 deletions(-)
===================================================================
arch/i386/kernel/paravirt.c +11 -11
@@ -34,7 +34,7 @@#include <asm/tlbflush.h> /* nop stub */ -static void native_nop (void ) +void _paravirt_nop (void ) { }
@@ -489,7 +489,7 @@ struct paravirt_ops paravirt_ops = { .patch = native_patch, .banner = default_banner, - .arch_setup = native_nop, + .arch_setup = paravirt_nop, .memory_setup = machine_specific_memory_setup, .get_wallclock = native_get_wallclock, .set_wallclock = native_set_wallclock,
@@ -544,23 +544,23 @@ struct paravirt_ops paravirt_ops = { .setup_boot_clock = setup_boot_APIC_clock, .setup_secondary_clock = setup_secondary_APIC_clock, #endif - .set_lazy_mode = (void *)native_nop, + .set_lazy_mode = paravirt_nop, .flush_tlb_user = native_flush_tlb, .flush_tlb_kernel = native_flush_tlb_global, .flush_tlb_single = native_flush_tlb_single, - .alloc_pt = (void *)native_nop, - .alloc_pd = (void *)native_nop, - .alloc_pd_clone = (void *)native_nop, - .release_pt = (void *)native_nop, - .release_pd = (void *)native_nop, + .alloc_pt = paravirt_nop, + .alloc_pd = paravirt_nop, + .alloc_pd_clone = paravirt_nop, + .release_pt = paravirt_nop, + .release_pd = paravirt_nop, .set_pte = native_set_pte, .set_pte_at = native_set_pte_at, .set_pmd = native_set_pmd, - .pte_update = (void *)native_nop, - .pte_update_defer = (void *)native_nop, + .pte_update = paravirt_nop, + .pte_update_defer = paravirt_nop, #ifdef CONFIG_X86_PAE .set_pte_atomic = native_set_pte_atomic, .set_pte_present = native_set_pte_present,
@@ -572,7 +572,7 @@ struct paravirt_ops paravirt_ops = { .irq_enable_sysexit = native_irq_enable_sysexit, .iret = native_iret, - .startup_ipi_hook = (void *)native_nop, + .startup_ipi_hook = paravirt_nop, };/*
===================================================================
include/asm-i386/paravirt.h +3 -0
@@ -422,6 +422,9 @@ static inline void pmd_clear(pmd_t *pmdp#define arch_enter_lazy_mmu_mode () paravirt_ops.set_lazy_mode(PARAVIRT_LAZY_MMU)#define arch_leave_lazy_mmu_mode () paravirt_ops.set_lazy_mode(PARAVIRT_LAZY_NONE) +void _paravirt_nop (void ); +#define paravirt_nop ((void *)_paravirt_nop) +/* These all sit in the .parainstructions section to tell us what to patch. */ struct paravirt_patch {u8 *instr;/* original instructions */
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 6 lines · Jeremy Fitzhardinge
Add a
_paravirt_nopfunction for use as a stub for no-op operations, andparavirt_nop#defined void * version to make using it easier (since all its uses are as a void *).This is useful to allow the patcher to automatically identify noop operations so it can simply nop out the callsite.
include/asm-i386/paravirt.h static inline void pmd_clear(pmd_t *pmdp · line 426
Quoted 2 lines
include/asm-i386/paravirt.h +2
-0+void _paravirt_nop (void ); +#define paravirt_nop ((void *)_paravirt_nop)
but only as a cleanup of the current open-coded (void *) casts. My problem with this is that it loses the types. Not that there is much to check for, but still, this adds some assumptions about how function calls look like.
Ingo
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 4 lines · Ingo Molnar
but only as a cleanup of the current open-coded (void *) casts. My problem with this is that it loses the types. Not that there is much to check for, but still, this adds some assumptions about how function calls look like.
I agree. I don't generally like this kind of hack, but having a single test for "func == paravirt_nop" to look for nop pv_ops in the patcher is what tipped the balance.
J
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 9 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
Quoted 4 lines · Ingo Molnar
but only as a cleanup of the current open-coded (void *) casts. My problem with this is that it loses the types. Not that there is much to check for, but still, this adds some assumptions about how function calls look like.
I agree. I don't generally like this kind of hack, but having a single test for "func ==
paravirt_nop" to look for noppv_opsin the patcher is what tipped the balance.
how about __paravirt_nop_start < func < __paravirt_nop_end and preserve the types?
thanks,
-chris
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 3 lines · Chris Wright
how about
__paravirt_nop_start< func <__paravirt_nop_endand preserve the types?
Er? The reason for the (void *) cast is to stop gcc complaining about mismatched pointer types.
J
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 7 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
Quoted 3 lines · Chris Wright
how about
__paravirt_nop_start< func <__paravirt_nop_endand preserve the types?Er? The reason for the (void *) cast is to stop gcc complaining about mismatched pointer types.
I mean like this (bunch of work, for a type check that we're really ignoring anwyay, but this is the idea...)
arch/i386/kernel/paravirt.c +70 -20
@@ -35,10 +35,60 @@#include <asm/tlbflush.h> #include <asm/timer.h> -/* nop stub */ -void _paravirt_nop (void ) -{ -} +/* nop stubs */ +void __paravirt_nop pv_nop_arch_setup(void) +{ +} +void __paravirt_nop pv_nop_set_lazy_mode(int mode) +{ +} +void __paravirt_nop pv_nop_alloc_pt(struct mm_struct *mm, u32 pfn) +{ +} +void __paravirt_nop pv_nop_alloc_pd(u32 pfn) +{ +} +void __paravirt_nop pv_nop_alloc_pd_clone(u32 pfn, u32 clonepfn, u32 start, u32 count) +{ +} +void __paravirt_nop pv_nop_release_pt(u32 pfn) +{ +} +void __paravirt_nop pv_nop_release_pd(u32 pfn) +{ +} +void __paravirt_nop pv_nop_pte_update(struct mm_struct *mm, unsigned long addr, pte_t *ptep) +{ +} +void __paravirt_nop pv_nop_pte_update_defer(struct mm_struct *mm, unsigned long addr, pte_t *ptep) +{ +} +void * __paravirt_nop pv_nop_kmap_atomic_pte(struct page *page, enum km_type type) +{ +} +void __paravirt_nop pv_nop_load_tr_desc(void) +{ +} +void __paravirt_nop pv_nop_activate_mm(struct mm_struct *prev, struct mm_struct *next) +{ +} +void __paravirt_nop pv_nop_dup_mmap(struct mm_struct *oldmm, struct mm_struct *mm) +{ +} +void __paravirt_nop pv_nop_exit_mmap(struct mm_struct *mm) +{ +} +void __paravirt_nop pv_nop_startup_ipi_hook(int phys_apicid, unsigned long start_eip, unsigned long start_esp) +{ +} +#ifdef CONFIG_X86_LOCAL_APIC +void __paravirt_nop pv_nop_setup_boot_clock(void) +{ +} +void __paravirt_nop pv_nop_setup_secondary_clock(void) +{ +} +#endif static void __init default_banner (void ) {
@@ -166,7 +216,7 @@ unsigned paravirt_patch_default(u8 type,if (opfunc ==NULL )/* If there's no function, patch it with a ud2a (BUG) */ ret =paravirt_patch_insns (site, len, start_ud2a, end_ud2a); -else if (opfunc == paravirt_nop) +else if (opfunc >= __paravirt_nop_start || opfunc < __paravirt_nop_end)/* If the operation is a nop, then nop the callsite */ ret =paravirt_patch_nop (); else if (type == PARAVIRT_PATCH(iret) ||
@@ -521,7 +571,7 @@ struct paravirt_ops paravirt_ops = { .patch = native_patch, .banner = default_banner, - .arch_setup = paravirt_nop, + .arch_setup = pv_nop_arch_setup, .memory_setup = machine_specific_memory_setup, .get_wallclock = native_get_wallclock, .set_wallclock = native_set_wallclock,
@@ -577,7 +627,7 @@ struct paravirt_ops paravirt_ops = { .setup_boot_clock = setup_boot_APIC_clock, .setup_secondary_clock = setup_secondary_APIC_clock, #endif - .set_lazy_mode = paravirt_nop, + .set_lazy_mode = pv_nop_set_lazy_mode, .pagetable_setup_start = native_pagetable_setup_start, .pagetable_setup_done = native_pagetable_setup_done,
@@ -587,24 +637,24 @@ struct paravirt_ops paravirt_ops = { .flush_tlb_single = native_flush_tlb_single, .flush_tlb_others = native_flush_tlb_others, - .alloc_pt = paravirt_nop, - .alloc_pd = paravirt_nop, - .alloc_pd_clone = paravirt_nop, - .release_pt = paravirt_nop, - .release_pd = paravirt_nop, + .alloc_pt = pv_nop_alloc_pt, + .alloc_pd = pv_nop_alloc_pd, + .alloc_pd_clone = pv_nop_alloc_pd_clone, + .release_pt = pv_nop_release_pt, + .release_pd = pv_nop_release_pd, .set_pte = native_set_pte, .set_pte_at = native_set_pte_at, .set_pmd = native_set_pmd, - .pte_update = paravirt_nop, - .pte_update_defer = paravirt_nop, + .pte_update = pv_nop_pte_update, + .pte_update_defer = pv_nop_pte_update_defer, .ptep_get_and_clear = native_ptep_get_and_clear, #ifdef CONFIG_HIGHPTE .kmap_atomic_pte = native_kmap_atomic_pte, #else - .kmap_atomic_pte = paravirt_nop, + .kmap_atomic_pte = pv_nop_kmap_atomic_pte, #endif #ifdef CONFIG_X86_PAE
@@ -627,11 +677,11 @@ struct paravirt_ops paravirt_ops = { .irq_enable_sysexit = native_irq_enable_sysexit, .iret = native_iret, - .dup_mmap = paravirt_nop, - .exit_mmap = paravirt_nop, - .activate_mm = paravirt_nop, - - .startup_ipi_hook = paravirt_nop, + .dup_mmap = pv_nop_dup_mmap, + .exit_mmap = pv_nop_exit_mmap, + .activate_mm = pv_nop_activate_mm, + + .startup_ipi_hook = pv_nop_startup_ipi_hook, };/*
arch/i386/kernel/vmlinux.lds.S +3 -0
@@ -49,6 +49,9 @@ SECTIONS SCHED_TEXT LOCK_TEXT KPROBES_TEXT + __paravirt_nop_start = .; + *(.paravirtnop) + __paravirt_nop_end = .; *(.fixup) *(.gnu.warning) _etext = .; /* End of text section */
arch/i386/xen/enlighten.c +9 -9
@@ -697,7 +697,7 @@ static const struct paravirt_ops xen_par .iret = (void *)&hypercall_page[__HYPERVISOR_iret], .irq_enable_sysexit = (void *)xen_sti_sysexit, - .load_tr_desc = paravirt_nop, + .load_tr_desc = pv_nop_load_tr_desc, .set_ldt = xen_set_ldt, .load_gdt = xen_load_gdt, .load_idt = xen_load_idt,
@@ -723,8 +723,8 @@ static const struct paravirt_ops xen_par .apic_write = xen_apic_write, .apic_write_atomic = xen_apic_write, .apic_read = xen_apic_read, - .setup_boot_clock = paravirt_nop, - .setup_secondary_clock = paravirt_nop, + .setup_boot_clock = pv_nop_setup_boot_clock, + .setup_secondary_clock = pv_nop_setup_secondary_clock, #endif .flush_tlb_user = xen_flush_tlb,
@@ -732,8 +732,8 @@ static const struct paravirt_ops xen_par .flush_tlb_single = xen_flush_tlb_single, .flush_tlb_others = xen_flush_tlb_others, - .pte_update = paravirt_nop, - .pte_update_defer = paravirt_nop, + .pte_update = pv_nop_pte_update, + .pte_update_defer = pv_nop_pte_update_defer, .pagetable_setup_start = xen_pagetable_setup_start, .pagetable_setup_done = xen_pagetable_setup_done,
@@ -747,9 +747,9 @@ static const struct paravirt_ops xen_par .alloc_pt = xen_alloc_pt_init, .release_pt = xen_release_pt, - .alloc_pd = paravirt_nop, - .alloc_pd_clone = paravirt_nop, - .release_pd = paravirt_nop, + .alloc_pd = pv_nop_alloc_pd, + .alloc_pd_clone = pv_nop_alloc_pd_clone, + .release_pd = pv_nop_release_pd, .kmap_atomic_pte = xen_kmap_atomic_pte,
@@ -773,7 +773,7 @@ static const struct paravirt_ops xen_par#endif /* PAE */ .set_lazy_mode = xen_set_lazy_mode, - .startup_ipi_hook = paravirt_nop, + .startup_ipi_hook = pv_nop_startup_ipi_hook, };#ifdef CONFIG_SMP
include/asm-i386/paravirt.h +24 -3
@@ -202,6 +202,30 @@ int native_write_msr(unsigned int msr, uint native_write_msr (unsigned int msr,unsigned long long val);unsigned long long native_read_tsc (void );unsigned long long native_read_pmc (void ); + +/* Paravirt nops */ +#define __paravirt_nop __attribute__ ((__section__ (".paravirtnop"))) +extern char __paravirt_nop_start[], __paravirt_nop_end[]; + +void __paravirt_nop pv_nop_arch_setup(void); +void __paravirt_nop pv_nop_set_lazy_mode(int mode); +void __paravirt_nop pv_nop_alloc_pt(struct mm_struct *mm, u32 pfn); +void __paravirt_nop pv_nop_alloc_pd(u32 pfn); +void __paravirt_nop pv_nop_alloc_pd_clone(u32 pfn, u32 clonepfn, u32 start, u32 count); +void __paravirt_nop pv_nop_release_pt(u32 pfn); +void __paravirt_nop pv_nop_release_pd(u32 pfn); +void __paravirt_nop pv_nop_pte_update(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +void __paravirt_nop pv_nop_pte_update_defer(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +void * __paravirt_nop pv_nop_kmap_atomic_pte(struct page *page, enum km_type type); +void __paravirt_nop pv_nop_load_tr_desc(void); +void __paravirt_nop pv_nop_activate_mm(struct mm_struct *prev, struct mm_struct *next); +void __paravirt_nop pv_nop_dup_mmap(struct mm_struct *oldmm, struct mm_struct *mm); +void __paravirt_nop pv_nop_exit_mmap(struct mm_struct *mm); +void __paravirt_nop pv_nop_startup_ipi_hook(int phys_apicid, unsigned long start_eip, unsigned long start_esp); +#ifdef CONFIG_X86_LOCAL_APIC +void __paravirt_nop pv_nop_setup_boot_clock(void); +void __paravirt_nop pv_nop_setup_secondary_clock(void); +#endif /* Mark a paravirt probe function. */ #define paravirt_probe (fn) \
@@ -876,9 +900,6 @@ static inline pte_t raw_ptep_get_and_cle#define arch_enter_lazy_mmu_mode () PVOP_VCALL1(set_lazy_mode, PARAVIRT_LAZY_MMU)#define arch_leave_lazy_mmu_mode () PVOP_VCALL1(set_lazy_mode, PARAVIRT_LAZY_NONE) -void _paravirt_nop (void ); -#define paravirt_nop ((void *)_paravirt_nop) -/* These all sit in the .parainstructions section to tell us what to patch. */ struct paravirt_patch_site {u8 *instr;/* original instructions */
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 3 lines · Chris Wright
I mean like this (bunch of work, for a type check that we're really ignoring anwyay, but this is the idea...)
Oh, I see. I think this is the best argument yet for the current arrangement...
J
Re: [patch 03/26] Xen-paravirt_ops: use paravirt_nop to consistently mark no-op operations
Quoted 6 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
Quoted 2 lines · Chris Wright
I mean like this (bunch of work, for a type check that we're really ignoring anwyay, but this is the idea...)
Oh, I see. I think this is the best argument yet for the current arrangement...
Heh, like I said it's a bunch of work for literally nothing ;-)
PATCH 4/26 Xen-paravirt_ops: Add pagetable accessors to pack and unpack pagetable entries
In reply to Jeremy FitzhardingeAdd a set of accessors to pack, unpack and modify page table entries (at all levels). This allows a paravirt implementation to control the contents of pgd/pmd/pte entries. For example, Xen uses this to convert the (pseudo-)physical address into a machine address when populating a pagetable entry, and converting back to pphys address when an entry is read.
--- arch/i386/kernel/paravirt.c | 84 +++++-------------------------------- arch/i386/kernel/vmi.c | 6 +- include/asm-i386/page.h | 71 ++++++++++++++++++++++++++----- include/asm-i386/paravirt.h | 52 +++++++++++++++++----- include/asm-i386/pgtable-2level.h | 28 +++++++++--- include/asm-i386/pgtable-3level.h | 65 +++++++++++++++++----------- include/asm-i386/pgtable.h | 2 7 files changed, 178 insertions(+), 130 deletions(-)
===================================================================
arch/i386/kernel/paravirt.c +12 -72
@@ -398,78 +398,6 @@ static void native_flush_tlb_single(u32 {__native_flush_tlb_single (addr); } - -#ifndef CONFIG_X86_PAE -static void native_set_pte (pte_t *ptep,pte_t pteval) -{ - *ptep = pteval; -} - -static void native_set_pte_at (struct mm_struct *mm,u32 addr,pte_t *ptep,pte_t pteval) -{ - *ptep = pteval; -} - -static void native_set_pmd (pmd_t *pmdp,pmd_t pmdval) -{ - *pmdp = pmdval; -} - -#else /* CONFIG_X86_PAE */ - -static void native_set_pte (pte_t *ptep,pte_t pte) -{ - ptep->pte_high = pte.pte_high; -smp_wmb (); - ptep->pte_low = pte.pte_low; -} - -static void native_set_pte_at (struct mm_struct *mm,u32 addr,pte_t *ptep,pte_t pte) -{ - ptep->pte_high = pte.pte_high; -smp_wmb (); - ptep->pte_low = pte.pte_low; -} - -static void native_set_pte_present (struct mm_struct *mm,unsigned long addr,pte_t *ptep,pte_t pte) -{ - ptep->pte_low =0 ; -smp_wmb (); - ptep->pte_high = pte.pte_high; -smp_wmb (); - ptep->pte_low = pte.pte_low; -} - -static void native_set_pte_atomic (pte_t *ptep,pte_t pteval) -{ -set_64bit ((unsigned long long *)ptep,pte_val (pteval)); -} - -static void native_set_pmd (pmd_t *pmdp,pmd_t pmdval) -{ -set_64bit ((unsigned long long *)pmdp,pmd_val (pmdval)); -} - -static void native_set_pud (pud_t *pudp,pud_t pudval) -{ - *pudp = pudval; -} - -static void native_pte_clear (struct mm_struct *mm,unsigned long addr,pte_t *ptep) -{ - ptep->pte_low =0 ; -smp_wmb (); - ptep->pte_high =0 ; -} - -static void native_pmd_clear (pmd_t *pmd) -{ -u32 *tmp = (u32 *)pmd; - *tmp =0 ; -smp_wmb (); - *(tmp +1 ) =0 ; -} -#endif /* CONFIG_X86_PAE */ /* These are in entry.S */ extern void native_iret (void );
@@ -561,13 +489,25 @@ struct paravirt_ops paravirt_ops = { .set_pmd = native_set_pmd, .pte_update = paravirt_nop, .pte_update_defer = paravirt_nop, + + .ptep_get_and_clear = native_ptep_get_and_clear, + #ifdef CONFIG_X86_PAE .set_pte_atomic = native_set_pte_atomic, .set_pte_present = native_set_pte_present, .set_pud = native_set_pud, .pte_clear = native_pte_clear, .pmd_clear = native_pmd_clear, + + .pmd_val = native_pmd_val, + .make_pmd = native_make_pmd, #endif + + .pte_val = native_pte_val, + .pgd_val = native_pgd_val, + + .make_pte = native_make_pte, + .make_pgd = native_make_pgd, .irq_enable_sysexit = native_irq_enable_sysexit, .iret = native_iret,
===================================================================
arch/i386/kernel/vmi.c +3 -3
@@ -426,13 +426,13 @@ static void vmi_release_pd(u32 pfn) ((level) | (is_current_as (mm, user) ? \ (VMI_PAGE_DEFER | VMI_PAGE_CURRENT_AS | ((addr) & VMI_PAGE_VA_MASK)) : 0)) -static void vmi_update_pte (struct mm_struct *mm,u32 addr,pte_t *ptep) +static void vmi_update_pte (struct mm_struct *mm,unsigned long addr,pte_t *ptep) {vmi_check_page_type (__pa (ptep) >>PAGE_SHIFT ,VMI_PAGE_PTE ); vmi_ops.update_pte (ptep,vmi_flags_addr (mm, addr,VMI_PAGE_PT ,0 )); } -static void vmi_update_pte_defer (struct mm_struct *mm,u32 addr,pte_t *ptep) +static void vmi_update_pte_defer (struct mm_struct *mm,unsigned long addr,pte_t *ptep) {vmi_check_page_type (__pa (ptep) >>PAGE_SHIFT ,VMI_PAGE_PTE ); vmi_ops.update_pte (ptep,vmi_flags_addr_defer (mm, addr,VMI_PAGE_PT ,0 ));
@@ -445,7 +445,7 @@ static void vmi_set_pte(pte_t *ptep, pte vmi_ops.set_pte (pte, ptep,VMI_PAGE_PT ); } -static void vmi_set_pte_at (struct mm_struct *mm,u32 addr,pte_t *ptep,pte_t pte) +static void vmi_set_pte_at (struct mm_struct *mm,unsigned long addr,pte_t *ptep,pte_t pte) {vmi_check_page_type (__pa (ptep) >>PAGE_SHIFT ,VMI_PAGE_PTE ); vmi_ops.set_pte (pte, ptep,vmi_flags_addr (mm, addr,VMI_PAGE_PT ,0 ));
===================================================================
include/asm-i386/page.h +60 -11
105 lines not on this page · raw message
===================================================================
include/asm-i386/paravirt.h +40 -12
105 lines not on this page · raw message
===================================================================
include/asm-i386/pgtable-2level.h +22 -6
@@ -11,11 +11,24 @@* within a page table are directly modified. Thus, the following * hook is made available. */ +static inline void native_set_pte (pte_t *ptep ,pte_t pte) +{ + *ptep = pte; +} +static inline void native_set_pte_at (struct mm_struct *mm,unsigned long addr, +pte_t *ptep ,pte_t pte) +{ +native_set_pte (ptep, pte); +} +static inline void native_set_pmd (pmd_t *pmdp,pmd_t pmd) +{ + *pmdp = pmd; +}#ifndef CONFIG_PARAVIRT -#define set_pte (pteptr, pteval) (*(pteptr) = pteval) -#define set_pte_at (mm,addr,ptep,pteval) set_pte(ptep,pteval) -#define set_pmd (pmdptr, pmdval) (*(pmdptr) = (pmdval)) -#endif +#define set_pte (pteptr, pteval) native_set_pte(pteptr, pteval) +#define set_pte_at (mm,addr,ptep,pteval) native_set_pte_at(mm, addr, ptep, pteval) +#define set_pmd (pmdptr, pmdval) native_set_pmd(pmdptr, pmdval) +#endif /* CONFIG_PARAVIRT */ #define set_pte_atomic (pteptr, pteval) set_pte(pteptr,pteval)#define set_pte_present (mm,addr,ptep,pteval) set_pte_at(mm,addr,ptep,pteval)
@@ -23,11 +36,14 @@#define pte_clear (mm,addr,xp) do { set_pte_at(mm, addr, xp, __pte(0)); } while (0)#define pmd_clear (xp) do { set_pmd(xp, __pmd(0)); } while (0) -#define raw_ptep_get_and_clear (xp) __pte(xchg(&(xp)->pte_low, 0)) +static inline pte_t native_ptep_get_and_clear (pte_t *xp) +{ +return __pte (xchg (&xp->pte_low,0 )); +}#define pte_page (x) pfn_to_page(pte_pfn(x))#define pte_none (x) (!(x).pte_low) -#define pte_pfn (x) ((unsigned long)(((x).pte_low >> PAGE_SHIFT))) +#define pte_pfn (x) (pte_val(x) >> PAGE_SHIFT)#define pfn_pte (pfn, prot) __pte(((pfn) << PAGE_SHIFT) | pgprot_val(prot))#define pfn_pmd (pfn, prot) __pmd(((pfn) << PAGE_SHIFT) | pgprot_val(prot))
===================================================================
include/asm-i386/pgtable-3level.h +39 -26
134 lines not on this page · raw message
===================================================================
include/asm-i386/pgtable.h +2 -0
@@ -263,6 +263,8 @@ static inline pte_t pte_mkhuge(pte_t pte*/ #define pte_update (mm, addr, ptep) do { } while (0)#define pte_update_defer (mm, addr, ptep) do { } while (0) + +#define raw_ptep_get_and_clear (xp) native_ptep_get_and_clear(xp)#endif /*
Re: [patch 04/26] Xen-paravirt_ops: Add pagetable accessors to pack and unpack pagetable entries
Quoted 6 lines · Jeremy Fitzhardinge
Add a set of accessors to pack, unpack and modify page table entries (at all levels). This allows a paravirt implementation to control the contents of pgd/pmd/pte entries. For example, Xen uses this to convert the (pseudo-)physical address into a machine address when populating a pagetable entry, and converting back to pphys address when an entry is read.
looks good.
only some minor style nits:
Quoted 14 lines
include/asm-i386/page.h +14
-0+static inline unsigned long long native_pgd_val (pgd_t pgd) +{ +return pgd.pgd; +} +static inline unsigned long long native_pmd_val (pmd_t pmd) +{ +return pmd.pmd; +} +static inline unsigned long long native_pte_val (pte_t pte) +{ +return pte.pte_low | ((unsigned long long )pte.pte_high <<32 ); +} +static inline pgd_t native_make_pgd (unsigned long long val) +{
include/asm-i386/page.h line 75
Quoted 10 lines
include/asm-i386/page.h +10
-0+return (pgd_t ) { val }; +} +static inline pmd_t native_make_pmd (unsigned long long val) +{ +return (pmd_t ) { val }; +} +static inline pte_t native_make_pte (unsigned long long val) +{ +return (pte_t ) { .pte_low = val, .pte_high = (val >>32 ) } ; +}
missing newlines between inline functions.
include/asm-i386/page.h line 80
Quoted 4 lines
include/asm-i386/page.h +4
-0+#ifndef CONFIG_PARAVIRT +#define pmd_val (x) native_pmd_val(x) +#define __pmd (x) native_make_pmd(x) +#endif /* !CONFIG_PARAVIRT */
no need for the closing !CONFIG_PARAVIRT comment: this define is 2 lines long so it's not that hard to find the start of the block. We typically do the /* !CONFIG_XX */ comment only for larger blocks, and when multiple #endif's intermix.
Quoted 6 lines
include/asm-i386/page.h +6
-0+static inline unsigned long native_pgd_val (pgd_t pgd) +{ +return pgd.pgd; +} +static inline unsigned long native_pte_val (pte_t pte) +{
include/asm-i386/page.h line 105
Quoted 10 lines
include/asm-i386/page.h +10
-0+return pte.pte_low; +} +static inline pgd_t native_make_pgd (unsigned long val) +{ +return (pgd_t ) { val }; +} +static inline pte_t native_make_pte (unsigned long val) +{ +return (pte_t ) { .pte_low = val }; +}
newlines.
include/asm-i386/page.h line 109
Quoted 4 lines
#define
HPAGE_SHIFT22
#include <asm-generic/pgtable-nopmd.h>include/asm-i386/page.h +1
-1-#endif +#endif /* CONFIG_X86_PAE */
(for example here the #endif comment is justified.)
Ingo
Re: [patch 04/26] Xen-paravirt_ops: Add pagetable accessors to pack and unpack pagetable entries
Quoted 12 lines · Ingo Molnar
include/asm-i386/page.h line 75
Quoted 9 lines
include/asm-i386/page.h +8
-0+static inline pmd_t native_make_pmd (unsigned long long val) +{ +return (pmd_t ) { val }; +} +static inline pte_t native_make_pte (unsigned long long val) +{ +return (pte_t ) { .pte_low = val, .pte_high = (val >>32 ) } ; +}missing newlines between inline functions.
OK.
Quoted 11 lines
include/asm-i386/page.h line 80
Quoted 5 lines
include/asm-i386/page.h +4
-0+#ifndef CONFIG_PARAVIRT +#define pmd_val (x) native_pmd_val(x) +#define __pmd (x) native_make_pmd(x) +#endif /* !CONFIG_PARAVIRT */ no need for the closing !CONFIG_PARAVIRT comment: this define is 2 lines long so it's not that hard to find the start of the block. We typically do the /* !CONFIG_XX */ comment only for larger blocks, and when multiple #endif's intermix.
Yeah, I tend to put them there reflexively. Its so easy for an #endif to drift away over time, and suddenly you have no idea what's going on. I agree its overkill in this case.
Quoted 8 lines
include/asm-i386/page.h line 109
Quoted 5 lines
#define
HPAGE_SHIFT22
#include <asm-generic/pgtable-nopmd.h>include/asm-i386/page.h +1
-1-#endif +#endif /* CONFIG_X86_PAE */ (for example here the #endif comment is justified.)
Yeah, and it probably started life much closer to the #ifdef...
J
PATCH 5/26 Xen-paravirt_ops: paravirt_ops: hooks to set up initial pagetable
In reply to Jeremy FitzhardingeThis patch introduces paravirt_ops hooks to control how the kernel's initial pagetable is set up.
In the case of a native boot, the very early bootstrap code creates a simple non-PAE pagetable to map the kernel and physical memory. When the VM subsystem is initialized, it creates a proper pagetable which respects the PAE mode, large pages, etc.
When booting under a hypervisor, there are many possibilities for what paging environment the hypervisor establishes for the guest kernel, so the constructon of the kernel's pagetable depends on the hypervisor.
In the case of Xen, the hypervisor boots the kernel with a fully constructed pagetable, which is already using PAE if necessary. Also, Xen requires particular care when constructing pagetables to make sure all pagetables are always mapped read-only.
In order to make this easier, kernel's initial pagetable construction has been changed to only allocate and initialize a pagetable page if there's no page already present in the pagetable. This allows the Xen paravirt backend to make a copy of the hypervisor-provided pagetable, allowing the kernel to establish any more mappings it needs while keeping the existing ones.
A slightly subtle point which is worth highlighting here is that Xen requires all kernel mappings to share the same pte_t pages between all pagetables, so that updating a kernel page's mapping in one pagetable is reflected in all other pagetables. This makes it possible to allocate a page and attach it to a pagetable without having to explicitly enumerate that page's mapping in all pagetables.
--- arch/i386/kernel/paravirt.c | 3 arch/i386/mm/init.c | 159 +++++++++++++++++++++++++++++-------------- include/asm-i386/paravirt.h | 17 ++++ include/asm-i386/pgtable.h | 16 ++++ 4 files changed, 143 insertions(+), 52 deletions(-)
===================================================================
arch/i386/kernel/paravirt.c +3 -0
@@ -549,6 +549,9 @@ struct paravirt_ops paravirt_ops = {#endif .set_lazy_mode = paravirt_nop, + .pagetable_setup_start = native_pagetable_setup_start, + .pagetable_setup_done = native_pagetable_setup_done, + .flush_tlb_user = native_flush_tlb, .flush_tlb_kernel = native_flush_tlb_global, .flush_tlb_single = native_flush_tlb_single,
===================================================================
arch/i386/mm/init.c +108 -51
218 lines not on this page · raw message
===================================================================
include/asm-i386/paravirt.h +16 -1
@@ -2,10 +2,11 @@#define __ASM_PARAVIRT_H/* Various instructions on x86 need to be replaced for * para-virtualization: those hooks are defined here. */ + +#ifdef CONFIG_PARAVIRT #include <linux/stringify.h> #include <asm/page.h> -#ifdef CONFIG_PARAVIRT /* These are the most performance critical ops, so we want to be able to patch * callers */ #define PARAVIRT_IRQ_DISABLE 0
@@ -48,6 +49,9 @@ struct paravirt_opsvoid (*arch_setup)(void );char *(*memory_setup)(void );void (*init_IRQ)(void ); + +void (*pagetable_setup_start)(pgd_t *pgd_base); +void (*pagetable_setup_done)(pgd_t *pgd_base);void (*banner)(void );
@@ -363,6 +367,17 @@ static inline void setup_secondary_clock }#endif +static inline void paravirt_pagetable_setup_start (pgd_t *base) +{ +if (paravirt_ops.pagetable_setup_start) + (*paravirt_ops.pagetable_setup_start)(base); +} + +static inline void paravirt_pagetable_setup_done (pgd_t *base) +{ +if (paravirt_ops.pagetable_setup_done) + (*paravirt_ops.pagetable_setup_done)(base); +}void native_set_pte (pte_t *ptep,pte_t pteval);void native_set_pte_at (struct mm_struct *mm,u32 addr,pte_t *ptep,pte_t pteval);
===================================================================
include/asm-i386/pgtable.h +16 -0
@@ -497,6 +497,22 @@ do { \* tables contain all the necessary information. */ #define update_mmu_cache (vma,address,pte) do { } while (0) + +void native_pagetable_setup_start (pgd_t *base); +void native_pagetable_setup_done (pgd_t *base); + +#ifndef CONFIG_PARAVIRT +static inline void paravirt_pagetable_setup_start (pgd_t *base) +{ +native_pagetable_setup_start (base); +} + +static inline void paravirt_pagetable_setup_done (pgd_t *base) +{ +native_pagetable_setup_done (base); +} +#endif /* !CONFIG_PARAVIRT */ +#endif /* !__ASSEMBLY__ */ #ifdef CONFIG_FLATMEM
Re: [patch 05/26] Xen-paravirt_ops: paravirt_ops: hooks to set up initial pagetable
Quoted 2 lines · Jeremy Fitzhardinge
This patch introduces
paravirt_opshooks to control how the kernel's initial pagetable is set up.
looks good. Some minor nits:
arch/i386/mm/init.c extern void __init remap_numa_kva(void); · line 370
Quoted 2 lines
arch/i386/mm/init.c +2
-0+/* Make sure kernel address space is empty so that a pagetable +will be allocated for it. */
comment style.
arch/i386/mm/init.c static void __init pagetable_init (void) · line 422
Quoted 4 lines
arch/i386/mm/init.c +4
-0+/* Enable PSE if available */ +if (cpu_has_pse) { +set_in_cr4 (X86_CR4_PSE ); + }
unnecessary braces.
Ingo
Re: [patch 05/26] Xen-paravirt_ops: paravirt_ops: hooks to set up initial pagetable
Quoted 6 lines · Ingo Molnar
arch/i386/mm/init.c extern void __init remap_numa_kva(void); · line 370
Quoted 3 lines
arch/i386/mm/init.c +2
-0+/* Make sure kernel address space is empty so that a pagetable +will be allocated for it. */ comment style.
As you've noticed its a comment style I use quite often. I use it for what's essentially a little local comment, rather than a block comment which really needs to draw attention to itself. In this case, I think using
/*
* Make sure kernel address space is empty so that a pagetable
* will be allocated for it.
*/
is just too loud for what its trying to say. If // comments were deemed acceptable, I'd probably use them here instead.
J
Re: [patch 05/26] Xen-paravirt_ops: paravirt_ops: hooks to set up initial pagetable
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 6 lines · Ingo Molnar
arch/i386/mm/init.c extern void __init remap_numa_kva(void); · line 370
Quoted 3 lines
arch/i386/mm/init.c +2
-0+/* Make sure kernel address space is empty so that a pagetable +will be allocated for it. */ comment style.
Quoted 4 lines
As you've noticed its a comment style I use quite often. I use it for what's essentially a little local comment, rather than a block comment which really needs to draw attention to itself. In this case, I think using
Understood, but then you need to make that comment a single line :)
The way Linux code prefers comments (like the code itself actually prefers it!) is single line comments stay on a single line, and multiline comments are always block.
So it's either:
/* my little comment here */
or
/*
* This comment, although small and to the point, does span multiple
* lines and regardless, needs to be a block comment.
*/
I guess it just makes it easier on the eyes when everything conforms and others need to look at / maintain your code.
-- Steve
Re: [patch 05/26] Xen-paravirt_ops: paravirt_ops: hooks to set up initial pagetable
In reply to Jeremy FitzhardingeQuoted 9 lines · Jeremy Fitzhardinge
Quoted 6 lines · Ingo Molnar
arch/i386/mm/init.c extern void __init remap_numa_kva(void); · line 370
Quoted 3 lines
arch/i386/mm/init.c +2
-0+/* Make sure kernel address space is empty so that a pagetable +will be allocated for it. */ comment style.
As you've noticed its a comment style I use quite often.
Me too, and Ingo and I have had this argument before. These days I try hard to fit simple comments in a single line 8)
Rusty.
PATCH 6/26 Xen-paravirt_ops: paravirt_ops: allocate a fixmap slot
In reply to Jeremy FitzhardingeAllocate a fixmap slot for use by a paravirt_ops implementation. This is intended for early-boot bootstrap mappings. Once the zones and allocator have been set up, it would be better to use get_vm_area() to allocate some virtual space.
Xen uses this to map the hypervisor's shared info page, which doesn't have a pseudo-physical page number, and therefore can't be mapped ordinarily. It is needed early because it contains the vcpu state, including the interrupt mask.
--- include/asm-i386/fixmap.h | 3 +++ 1 file changed, 3 insertions(+)
===================================================================
include/asm-i386/fixmap.h +3 -0
@@ -86,6 +86,9 @@ enum fixed_addresses {#ifdef CONFIG_PCI_MMCONFIG FIX_PCIE_MCFG,#endif +#ifdef CONFIG_PARAVIRT + FIX_PARAVIRT_BOOTMAP, +#endif __end_of_permanent_fixed_addresses,/* temporary boot-time mappings, used before ioremap() is functional */ #define NR_FIX_BTMAPS 16
Re: [patch 06/26] Xen-paravirt_ops: paravirt_ops: allocate a fixmap slot
Quoted 9 lines · Jeremy Fitzhardinge
Allocate a fixmap slot for use by a
paravirt_opsimplementation. This is intended for early-boot bootstrap mappings. Once the zones and allocator have been set up, it would be better to useget_vm_area()to allocate some virtual space.Xen uses this to map the hypervisor's shared info page, which doesn't have a pseudo-physical page number, and therefore can't be mapped ordinarily. It is needed early because it contains the vcpu state, including the interrupt mask.
Ingo
PATCH 7/26 Xen-paravirt_ops: Allow paravirt backend to choose kernel PMD sharing
In reply to Jeremy FitzhardingeNormally when running in PAE mode, the 4th PMD maps the kernel address space, which can be shared among all processes (since they all need the same kernel mappings.
Xen, however, does not allow guests to have the kernel pmd shared between page tables, so parameterize pgtable.c to allow both modes of operation.
There are several side-effects of this. One is that vmalloc will update the kernel address space mappings, and those updates need to be propagated into all processes if the kernel mapping are not intrinsically shared. In the non-PAE case, this is done by maintaining a pgd_list of all processes; this list is used when all process pagetables must be updated. pgd_list is threaded via otherwise unused entries in the page structure for the pgd, which means that the pgd must be page-sized for this to work.
Normally the PAE pgd is only 4x64 byte entries large, but Xen requires the PAE pgd to page aligned anyway, so this patch forces the pgd to be page aligned+sized when the kernel pmd is unshared, to accomodate both these requirements.
Also, since there may be several distinct kernel pmds (if the user/kernel split is below 3G), there's no point in allocating them from a slab cache; they're just allocated with get_free_page and initialized appropriately. (Of course the could be cached if there is just a single kernel pmd - which is the default with a 3G user/kernel split - but it doesn't seem worthwhile to add yet another case into this code).
[ Many thanks to wli for review comments. ]
--- arch/i386/kernel/paravirt.c | 1 arch/i386/mm/fault.c | 6 +- arch/i386/mm/init.c | 18 +++++- arch/i386/mm/pageattr.c | 2 arch/i386/mm/pgtable.c | 84 ++++++++++++++++++++++++++------ include/asm-i386/paravirt.h | 1 include/asm-i386/pgtable-2level-defs.h | 2 include/asm-i386/pgtable-2level.h | 2 include/asm-i386/pgtable-3level-defs.h | 6 ++ include/asm-i386/pgtable-3level.h | 2 include/asm-i386/pgtable.h | 7 ++ 11 files changed, 105 insertions(+), 26 deletions(-)
===================================================================
arch/i386/kernel/paravirt.c +1 -0
@@ -604,6 +604,7 @@ struct paravirt_ops paravirt_ops = { .name = "bare hardware", .paravirt_enabled = 0, .kernel_rpl = 0, + .shared_kernel_pmd = 1, /* Only used when CONFIG_X86_PAE is set */ .patch = native_patch, .banner = default_banner,
===================================================================
arch/i386/mm/fault.c +3 -3
@@ -588,8 +588,7 @@ do_sigbus:force_sig_info_fault (SIGBUS ,BUS_ADRERR , address, tsk); } -#ifndef CONFIG_X86_PAE -void vmalloc_sync_all (void ) +void _vmalloc_sync_all (void ) {/* * Note that races in the updates of insync and start aren't
@@ -600,6 +599,8 @@ void vmalloc_sync_all(void) static DECLARE_BITMAP(insync, PTRS_PER_PGD);static unsigned long start =TASK_SIZE ;unsigned long address; + +BUG_ON (SHARED_KERNEL_PMD );BUILD_BUG_ON (TASK_SIZE & ~PGDIR_MASK );for (address = start; address >=TASK_SIZE ; address +=PGDIR_SIZE ) {
@@ -623,4 +624,3 @@ void vmalloc_sync_all(void) start = address +PGDIR_SIZE ; } } -#endif
===================================================================
arch/i386/mm/init.c +15 -3
@@ -715,6 +715,8 @@ struct kmem_cache *pmd_cache;void __init pgtable_cache_init (void ) { +size_t pgd_size =PTRS_PER_PGD *sizeof (pgd_t); +if (PTRS_PER_PMD >1 ) { pmd_cache = kmem_cache_create("pmd", PTRS_PER_PMD*sizeof(pmd_t),
@@ -724,13 +726,23 @@ void __init pgtable_cache_init(void) NULL);if (!pmd_cache)panic ("pgtable_cache_init(): cannot create pmd cache" ); + +if (!SHARED_KERNEL_PMD ) { +/* If we're in PAE mode and have a non-shared +kernel pmd, then the pgd size must be a +page size. This is because the pgd_list +links through the page structure, so there +can only be one pgd per page for this to +work. */ + pgd_size =PAGE_SIZE ; + } } pgd_cache =kmem_cache_create ("pgd" , -PTRS_PER_PGD *sizeof (pgd_t), -PTRS_PER_PGD *sizeof (pgd_t), + pgd_size, + pgd_size,0 , pgd_ctor, -PTRS_PER_PMD ==1 ? pgd_dtor :NULL ); + (!SHARED_KERNEL_PMD ) ? pgd_dtor :NULL );if (!pgd_cache)panic ("pgtable_cache_init(): Cannot create pgd cache" ); }
===================================================================
arch/i386/mm/pageattr.c +1 -1
@@ -91,7 +91,7 @@ static void set_pmd_pte(pte_t *kpte, unsunsigned long flags;set_pte_atomic (kpte, pte);/* change init_mm */ -if (PTRS_PER_PMD >1 ) +if (SHARED_KERNEL_PMD )return ;spin_lock_irqsave (&pgd_lock, flags);
===================================================================
arch/i386/mm/pgtable.c +69 -15
138 lines not on this page · raw message
===================================================================
include/asm-i386/paravirt.h +1 -0
@@ -34,6 +34,7 @@ struct paravirt_opsstruct paravirt_ops {unsigned int kernel_rpl; +int shared_kernel_pmd;int paravirt_enabled;const char *name;
===================================================================
include/asm-i386/pgtable-2level-defs.h +2 -0
@@ -1,5 +1,7 @@#ifndef _I386_PGTABLE_2LEVEL_DEFS_H#define _I386_PGTABLE_2LEVEL_DEFS_H + +#define SHARED_KERNEL_PMD 0/* * traditional i386 two-level paging structure:
===================================================================
include/asm-i386/pgtable-2level.h +0 -2
@@ -65,6 +65,4 @@ static inline int pte_exec_kernel(pte_t#define __pte_to_swp_entry (pte) ((swp_entry_t) { (pte).pte_low })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -void vmalloc_sync_all (void ); -#endif /* _I386_PGTABLE_2LEVEL_H */
===================================================================
include/asm-i386/pgtable-3level-defs.h +6 -0
@@ -1,5 +1,11 @@#ifndef _I386_PGTABLE_3LEVEL_DEFS_H#define _I386_PGTABLE_3LEVEL_DEFS_H + +#ifdef CONFIG_PARAVIRT +#define SHARED_KERNEL_PMD (paravirt_ops.shared_kernel_pmd) +#else +#define SHARED_KERNEL_PMD 1 +#endif /* * PGDIR_SHIFT determines what a top-level page table entry can map
===================================================================
include/asm-i386/pgtable-3level.h +0 -2
@@ -180,6 +180,4 @@ static inline pmd_t pfn_pmd(unsigned lon#define __pmd_free_tlb (tlb, x) do { } while (0) -#define vmalloc_sync_all () ((void)0) -#endif /* _I386_PGTABLE_3LEVEL_H */
===================================================================
include/asm-i386/pgtable.h +7 -0
@@ -244,6 +244,13 @@ static inline pte_t pte_mkwrite(pte_t ptstatic inline pte_t pte_mkwrite (pte_t pte) { (pte).pte_low |= _PAGE_RW;return pte; }static inline pte_t pte_mkhuge (pte_t pte) { (pte).pte_low |= _PAGE_PSE;return pte; } +extern void _vmalloc_sync_all (void ); +static inline void vmalloc_sync_all (void ) +{ +if (!SHARED_KERNEL_PMD ) +_vmalloc_sync_all (); +} +#ifdef CONFIG_X86_PAE # include <asm/pgtable-3level.h> #else
Re: [patch 07/26] Xen-paravirt_ops: Allow paravirt backend to choose kernel PMD sharing
Quoted 7 lines · Jeremy Fitzhardinge
Normally when running in PAE mode, the 4th PMD maps the kernel address space, which can be shared among all processes (since they all need the same kernel mappings.
Xen, however, does not allow guests to have the kernel pmd shared between page tables, so parameterize
pgtable.cto allow both modes of operation.
Ingo
PATCH 8/26 Xen-paravirt_ops: add hooks to intercept mm creation and destruction
In reply to Jeremy FitzhardingeAdd hooks to allow a paravirt implementation to track the lifetime of an mm.
--- arch/i386/kernel/paravirt.c | 4 ++++ include/asm-i386/mmu_context.h | 8 ++++++-- include/asm-i386/paravirt.h | 24 ++++++++++++++++++++++++ include/linux/sched.h | 6 ++++++ kernel/fork.c | 2 ++ mm/mmap.c | 3 +++ 6 files changed, 45 insertions(+), 2 deletions(-)
===================================================================
arch/i386/kernel/paravirt.c +4 -0
@@ -591,6 +591,10 @@ struct paravirt_ops paravirt_ops = { .irq_enable_sysexit = native_irq_enable_sysexit, .iret = native_iret, + .dup_mmap = paravirt_nop, + .exit_mmap = paravirt_nop, + .activate_mm = paravirt_nop, + .startup_ipi_hook = paravirt_nop, };
===================================================================
include/asm-i386/mmu_context.h +6 -2
@@ -5,6 +5,7 @@#include <asm/atomic.h> #include <asm/pgalloc.h> #include <asm/tlbflush.h> +#include <asm/paravirt.h> /* * Used for LDT copy/destruction.
@@ -65,7 +66,10 @@ static inline void switch_mm(struct mm_s#define deactivate_mm (tsk, mm) \ asm("movl %0,%%gs": :"r" (0)); -#define activate_mm (prev, next) \ - switch_mm((prev),(next),NULL) +#define activate_mm (prev, next) \ + do { \ + arch_activate_mm(prev, next); \ + switch_mm((prev),(next),NULL); \ + } while(0);#endif
===================================================================
include/asm-i386/paravirt.h +24 -0
@@ -118,6 +118,12 @@ struct paravirt_opsvoid (*io_delay)(void );void (*const_udelay)(unsigned long loops); +void (*activate_mm)(struct mm_struct *prev, +struct mm_struct *next); +void (*dup_mmap)(struct mm_struct *oldmm, +struct mm_struct *mm); +void (*exit_mmap)(struct mm_struct *mm); +#ifdef CONFIG_X86_LOCAL_APIC void (*apic_write)(unsigned long reg,unsigned long v);void (*apic_write_atomic)(unsigned long reg,unsigned long v);
@@ -398,6 +404,24 @@ static inline void startup_ipi_hook(int }#endif +#define __HAVE_ARCH_MM_LIFETIME +static inline void arch_activate_mm (struct mm_struct *prev, +struct mm_struct *next) +{ + paravirt_ops.activate_mm (prev, next); +} + +static inline void arch_dup_mmap (struct mm_struct *oldmm, +struct mm_struct *mm) +{ + paravirt_ops.dup_mmap (oldmm, mm); +} + +static inline void arch_exit_mmap (struct mm_struct *mm) +{ + paravirt_ops.exit_mmap (mm); +} +#define __flush_tlb () paravirt_ops.flush_tlb_user()#define __flush_tlb_global () paravirt_ops.flush_tlb_kernel()#define __flush_tlb_single (addr) paravirt_ops.flush_tlb_single(addr)
===================================================================
include/linux/sched.h +6 -0
@@ -374,6 +374,12 @@ struct mm_struct {rwlock_t ioctx_list_lock;struct kioctx *ioctx_list; }; + +#ifndef __HAVE_ARCH_MM_LIFETIME +#define arch_activate_mm (prev, next) do {} while(0) +#define arch_dup_mmap (oldmm, mm) do {} while(0) +#define arch_exit_mmap (mm) do {} while(0) +#endif struct sighand_struct {atomic_t count;
===================================================================
kernel/fork.c +2 -0
@@ -286,6 +286,8 @@ static inline int dup_mmap(struct mm_strif (retval) goto out; } +/* a new mm has just been created */ +arch_dup_mmap (oldmm, mm); retval =0 ; out:up_write (&mm->mmap_sem);
===================================================================
mm/mmap.c +3 -0
@@ -1976,6 +1976,9 @@ void exit_mmap(struct mm_struct *mm)struct vm_area_struct *vma = mm->mmap;unsigned long nr_accounted =0 ;unsigned long end; + +/* mm's last user has gone, and its about to be pulled down */ +arch_exit_mmap (mm);lru_add_drain ();flush_cache_mm (mm);
Re: [patch 08/26] Xen-paravirt_ops: add hooks to intercept mm creation and destruction
Quoted 2 lines · Jeremy Fitzhardinge
Add hooks to allow a paravirt implementation to track the lifetime of an mm.
(i guess this one should be merged with the ARCH-define cleanup patch)
Ingo
Re: [patch 08/26] Xen-paravirt_ops: add hooks to intercept mm creation and destruction
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Jeremy Fitzhardinge
Add hooks to allow a paravirt implementation to track the lifetime of an mm.
(i guess this one should be merged with the ARCH-define cleanup patch)
Already done. In fact, I think I did it before the last patch post. Are you looking at an uptodate series of patches?
PATCH 9/26 Xen-paravirt_ops: remove HAVE_ARCH_MM_LIFETIME, define no-op architecture implementations
In reply to Jeremy Fitzhardingeakpm:
Quoted 2 lines
Can we lose
__HAVE_ARCH_MM_LIFETIME? Just define these (preferably in C, not in cpp) in the appropriateinclude/asm-foo/files?
--- include/asm-alpha/mmu_context.h | 1 + include/asm-arm/mmu_context.h | 1 + include/asm-arm26/mmu_context.h | 2 ++ include/asm-avr32/mmu_context.h | 1 + include/asm-cris/mmu_context.h | 2 ++ include/asm-frv/mmu_context.h | 1 + include/asm-generic/mm_hooks.h | 18 ++++++++++++++++++ include/asm-h8300/mmu_context.h | 1 + include/asm-i386/mmu_context.h | 11 ++++++++++- include/asm-i386/paravirt.h | 5 ++--- include/asm-ia64/mmu_context.h | 1 + include/asm-m32r/mmu_context.h | 1 + include/asm-m68k/mmu_context.h | 1 + include/asm-m68knommu/mmu_context.h | 1 + include/asm-mips/mmu_context.h | 1 + include/asm-parisc/mmu_context.h | 1 + include/asm-powerpc/mmu_context.h | 1 + include/asm-ppc/mmu_context.h | 1 + include/asm-s390/mmu_context.h | 2 ++ include/asm-sh/mmu_context.h | 1 + include/asm-sh64/mmu_context.h | 2 +- include/asm-sparc/mmu_context.h | 2 ++ include/asm-sparc64/mmu_context.h | 1 + include/asm-um/mmu_context.h | 2 ++ include/asm-v850/mmu_context.h | 2 ++ include/asm-x86_64/mmu_context.h | 1 + include/asm-xtensa/mmu_context.h | 1 + include/linux/sched.h | 6 ------ mm/mmap.c | 1 + 29 files changed, 61 insertions(+), 11 deletions(-)
===================================================================
include/asm-alpha/mmu_context.h +1 -0
@@ -10,6 +10,7 @@#include <asm/system.h> #include <asm/machvec.h> #include <asm/compiler.h> +#include <asm-generic/mm_hooks.h> /* * Force a context reload. This is needed when we change the page
===================================================================
include/asm-arm/mmu_context.h +1 -0
@@ -16,6 +16,7 @@#include <linux/compiler.h> #include <asm/cacheflush.h> #include <asm/proc-fns.h> +#include <asm-generic/mm_hooks.h> void __check_kvm_seq (struct mm_struct *mm);
===================================================================
include/asm-arm26/mmu_context.h +2 -0
@@ -12,6 +12,8 @@*/ #ifndef __ASM_ARM_MMU_CONTEXT_H#define __ASM_ARM_MMU_CONTEXT_H + +#include <asm-generic/mm_hooks.h> #define init_new_context (tsk,mm) 0#define destroy_context (mm) do { } while(0)
===================================================================
include/asm-avr32/mmu_context.h +1 -0
@@ -15,6 +15,7 @@#include <asm/tlbflush.h> #include <asm/pgalloc.h> #include <asm/sysreg.h> +#include <asm-generic/mm_hooks.h> /* * The MMU "context" consists of two things:
===================================================================
include/asm-cris/mmu_context.h +2 -0
@@ -1,5 +1,7 @@#ifndef __CRIS_MMU_CONTEXT_H#define __CRIS_MMU_CONTEXT_H + +#include <asm-generic/mm_hooks.h> extern int init_new_context (struct task_struct *tsk,struct mm_struct *mm);extern void get_mmu_context (struct mm_struct *mm);
===================================================================
include/asm-frv/mmu_context.h +1 -0
@@ -15,6 +15,7 @@#include <asm/setup.h> #include <asm/page.h> #include <asm/pgalloc.h> +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-generic/mm_hooks.h new file +18 -0
@@ -0,0 +1,18 @@ +/* +* Define generic no-op hooks for arch_dup_mmap and arch_exit_mmap, to +* be included in asm-FOO/mmu_context.h for any arch FOO which doesn't +* need to hook these. +*/ +#ifndef _ASM_GENERIC_MM_HOOKS_H +#define _ASM_GENERIC_MM_HOOKS_H + +static inline void arch_dup_mmap (struct mm_struct *oldmm, +struct mm_struct *mm) +{ +} + +static inline void arch_exit_mmap (struct mm_struct *mm) +{ +} + +#endif /* _ASM_GENERIC_MM_HOOKS_H */
===================================================================
include/asm-h8300/mmu_context.h +1 -0
@@ -4,6 +4,7 @@#include <asm/setup.h> #include <asm/page.h> #include <asm/pgalloc.h> +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-i386/mmu_context.h +10 -1
@@ -6,6 +6,15 @@#include <asm/pgalloc.h> #include <asm/tlbflush.h> #include <asm/paravirt.h> +#ifndef CONFIG_PARAVIRT +#include <asm-generic/mm_hooks.h> + +static inline void paravirt_activate_mm (struct mm_struct *prev, +struct mm_struct *next) +{ +} +#endif /* !CONFIG_PARAVIRT */ +/* * Used for LDT copy/destruction.
@@ -68,7 +77,7 @@ static inline void switch_mm(struct mm_s#define activate_mm (prev, next) \ do { \ - arch_activate_mm(prev, next); \ + paravirt_activate_mm(prev, next); \ switch_mm((prev),(next),NULL); \ } while(0);
===================================================================
include/asm-i386/paravirt.h +2 -3
@@ -403,9 +403,8 @@ static inline void startup_ipi_hook(int }#endif -#define __HAVE_ARCH_MM_LIFETIME -static inline void arch_activate_mm (struct mm_struct *prev, -struct mm_struct *next) +static inline void paravirt_activate_mm (struct mm_struct *prev, +struct mm_struct *next) { paravirt_ops.activate_mm (prev, next); }
===================================================================
include/asm-ia64/mmu_context.h +1 -0
@@ -29,6 +29,7 @@#include <linux/spinlock.h> #include <asm/processor.h> +#include <asm-generic/mm_hooks.h> struct ia64_ctx {spinlock_t lock;
===================================================================
include/asm-m32r/mmu_context.h +1 -0
@@ -15,6 +15,7 @@#include <asm/pgalloc.h> #include <asm/mmu.h> #include <asm/tlbflush.h> +#include <asm-generic/mm_hooks.h> /* * Cache of MMU context last used.
===================================================================
include/asm-m68k/mmu_context.h +1 -0
@@ -1,6 +1,7 @@#ifndef __M68K_MMU_CONTEXT_H#define __M68K_MMU_CONTEXT_H +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-m68knommu/mmu_context.h +1 -0
@@ -4,6 +4,7 @@#include <asm/setup.h> #include <asm/page.h> #include <asm/pgalloc.h> +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-mips/mmu_context.h +1 -0
@@ -20,6 +20,7 @@#include <asm/mipsmtregs.h> #include <asm/smtc.h> #endif /* SMTC */ +#include <asm-generic/mm_hooks.h> /* * For the fast tlb miss handlers, we keep a per cpu array of pointers
===================================================================
include/asm-parisc/mmu_context.h +1 -0
@@ -5,6 +5,7 @@#include <asm/atomic.h> #include <asm/pgalloc.h> #include <asm/pgtable.h> +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-powerpc/mmu_context.h +1 -0
@@ -10,6 +10,7 @@#include <linux/mm.h> #include <asm/mmu.h> #include <asm/cputable.h> +#include <asm-generic/mm_hooks.h> /* * Copyright (C) 2001 PPC 64 Team, IBM Corp
===================================================================
include/asm-ppc/mmu_context.h +1 -0
@@ -6,6 +6,7 @@#include <asm/bitops.h> #include <asm/mmu.h> #include <asm/cputable.h> +#include <asm-generic/mm_hooks.h> /* * On 32-bit PowerPC 6xx/7xx/7xxx CPUs, we use a set of 16 VSIDs
===================================================================
include/asm-s390/mmu_context.h +2 -0
@@ -10,6 +10,8 @@#define __S390_MMU_CONTEXT_H#include <asm/pgalloc.h> +#include <asm-generic/mm_hooks.h> +/* * get a new mmu context.. S390 don't know about contexts. */
===================================================================
include/asm-sh/mmu_context.h +1 -0
@@ -12,6 +12,7 @@#include <asm/tlbflush.h> #include <asm/uaccess.h> #include <asm/io.h> +#include <asm-generic/mm_hooks.h> /* * The MMU "context" consists of two things:
===================================================================
include/asm-sh64/mmu_context.h +1 -1
@@ -27,7 +27,7 @@ extern unsigned long mmu_context_cache;extern unsigned long mmu_context_cache;#include <asm/page.h> - +#include <asm-generic/mm_hooks.h> /* Current mm's pgd */ extern pgd_t *mmu_pdtp_cache;
===================================================================
include/asm-sparc/mmu_context.h +2 -0
@@ -4,6 +4,8 @@#include <asm/btfixup.h> #ifndef __ASSEMBLY__ + +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-sparc64/mmu_context.h +1 -0
@@ -9,6 +9,7 @@#include <linux/spinlock.h> #include <asm/system.h> #include <asm/spitfire.h> +#include <asm-generic/mm_hooks.h> static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {
===================================================================
include/asm-um/mmu_context.h +2 -0
@@ -5,6 +5,8 @@#ifndef __UM_MMU_CONTEXT_H#define __UM_MMU_CONTEXT_H + +#include <asm-generic/mm_hooks.h> #include "linux/sched.h" #include "choose-mode.h"
===================================================================
include/asm-v850/mmu_context.h +2 -0
@@ -1,5 +1,7 @@#ifndef __V850_MMU_CONTEXT_H__#define __V850_MMU_CONTEXT_H__ + +#include <asm-generic/mm_hooks.h> #define destroy_context (mm) ((void)0)#define init_new_context (tsk,mm) 0
===================================================================
include/asm-x86_64/mmu_context.h +1 -0
@@ -7,6 +7,7 @@#include <asm/pda.h> #include <asm/pgtable.h> #include <asm/tlbflush.h> +#include <asm-generic/mm_hooks.h> /* * possibly do the LDT unload here?
===================================================================
include/asm-xtensa/mmu_context.h +1 -0
@@ -18,6 +18,7 @@#include <asm/pgtable.h> #include <asm/cacheflush.h> #include <asm/tlbflush.h> +#include <asm-generic/mm_hooks.h> #define XCHAL_MMU_ASID_BITS 8
===================================================================
include/linux/sched.h +0 -6
@@ -374,12 +374,6 @@ struct mm_struct {rwlock_t ioctx_list_lock;struct kioctx *ioctx_list; }; - -#ifndef __HAVE_ARCH_MM_LIFETIME -#define arch_activate_mm (prev, next) do {} while(0) -#define arch_dup_mmap (oldmm, mm) do {} while(0) -#define arch_exit_mmap (mm) do {} while(0) -#endif struct sighand_struct {atomic_t count;
===================================================================
mm/mmap.c +1 -0
@@ -29,6 +29,7 @@#include <asm/uaccess.h> #include <asm/cacheflush.h> #include <asm/tlb.h> +#include <asm/mmu_context.h> #ifndef arch_mmap_check#define arch_mmap_check (addr, len, flags) (0)
Re: [patch 09/26] Xen-paravirt_ops: remove HAVE_ARCH_MM_LIFETIME, define no-op architecture implementations
Quoted 3 lines · Jeremy Fitzhardinge
akpm:
Quoted 2 lines
Can we lose
__HAVE_ARCH_MM_LIFETIME? Just define these (preferably in C, not in cpp) in the appropriateinclude/asm-foo/files?
Ingo
PATCH 10/26 Xen-paravirt_ops: rename struct paravirt_patch to paravirt_patch_site for clarity
In reply to Jeremy FitzhardingeRename struct paravirt_patch to paravirt_patch_site, so that it clearly refers to a callsite, and not the patch which may be applied to that callsite.
--- arch/i386/kernel/alternative.c | 8 +++----- arch/i386/kernel/vmi.c | 4 ---- include/asm-i386/alternative.h | 6 +++--- include/asm-i386/paravirt.h | 5 ++++- 4 files changed, 10 insertions(+), 13 deletions(-)
===================================================================
arch/i386/kernel/alternative.c +3 -5
@@ -350,9 +350,9 @@ void alternatives_smp_switch(int smp)#endif #ifdef CONFIG_PARAVIRT -void apply_paravirt (struct paravirt_patch *start,struct paravirt_patch *end) -{ -struct paravirt_patch *p; +void apply_paravirt (struct paravirt_patch_site *start,struct paravirt_patch_site *end) +{ +struct paravirt_patch_site *p;for (p = start; p < end; p++) {unsigned int used;
@@ -379,8 +379,6 @@ void apply_paravirt(struct paravirt_patc/* Sync to be conservative, in case we patched following instructions */ sync_core(); } -extern struct paravirt_patch __start_parainstructions[], - __stop_parainstructions[];#endif /* CONFIG_PARAVIRT */ void __init alternative_instructions (void )
===================================================================
arch/i386/kernel/vmi.c +0 -4
@@ -70,10 +70,6 @@ struct {void (*set_initial_ap_state)(int ,int );void (*halt)(void ); } vmi_ops; - -/* XXX move this to alternative.h */ -extern struct paravirt_patch __start_parainstructions[], - __stop_parainstructions[];/* * VMI patching routines.
===================================================================
include/asm-i386/alternative.h +3 -3
@@ -118,12 +118,12 @@ static inline void alternatives_smp_swit#define LOCK_PREFIX ""#endif -struct paravirt_patch ; +struct paravirt_patch_site ;#ifdef CONFIG_PARAVIRT -void apply_paravirt (struct paravirt_patch *start,struct paravirt_patch *end); +void apply_paravirt (struct paravirt_patch_site *start,struct paravirt_patch_site *end);#else static inline void -apply_paravirt (struct paravirt_patch *start,struct paravirt_patch *end) +apply_paravirt (struct paravirt_patch_site *start,struct paravirt_patch_site *end) {}#define __start_parainstructions NULL#define __stop_parainstructions NULL
===================================================================
include/asm-i386/paravirt.h +4 -1
@@ -501,12 +501,15 @@ void _paravirt_nop(void);#define paravirt_nop ((void *)_paravirt_nop)/* These all sit in the .parainstructions section to tell us what to patch. */ -struct paravirt_patch { +struct paravirt_patch_site {u8 *instr;/* original instructions */ u8 instrtype;/* type of this instruction */ u8 len;/* length of original instruction */ u16 clobbers;/* what registers you may clobber */ }; + +extern struct paravirt_patch_site __start_parainstructions[], + __stop_parainstructions[];#define paravirt_alt (insn_string, typenum, clobber) \ "771:\n\t" insn_string "\n" "772:\n" \
PATCH 11/26 Xen-paravirt_ops: Use patch site IDs computed from offset in paravirt_ops structure
In reply to Jeremy FitzhardingeUse patch type identifiers derived from the offset of the operation in the paravirt_ops structure. This avoids having to maintain a separate enum for patch site types.
This patch also drops the fused save_fl+cli operation, which doesn't really add much and makes things more complex.
--- arch/i386/kernel/asm-offsets.c | 5 + arch/i386/kernel/paravirt.c | 14 ++--- arch/i386/kernel/vmi.c | 38 ++------------ include/asm-i386/paravirt.h | 107 +++++++++++++++++++--------------------- 4 files changed, 70 insertions(+), 94 deletions(-)
===================================================================
arch/i386/kernel/asm-offsets.c +5 -0
@@ -110,5 +110,10 @@ void foo(void)OFFSET (PARAVIRT_irq_enable_sysexit, paravirt_ops, irq_enable_sysexit);OFFSET (PARAVIRT_iret, paravirt_ops, iret);OFFSET (PARAVIRT_read_cr0, paravirt_ops, read_cr0); + +DEFINE (PARAVIRT_IRQ_DISABLE ,PARAVIRT_PATCH (irq_disable)); +DEFINE (PARAVIRT_IRQ_ENABLE ,PARAVIRT_PATCH (irq_enable)); +DEFINE (PARAVIRT_STI_SYSEXIT ,PARAVIRT_PATCH (irq_enable_sysexit)); +DEFINE (PARAVIRT_INTERRUPT_RETURN ,PARAVIRT_PATCH (iret));#endif }
===================================================================
arch/i386/kernel/paravirt.c +6 -8
@@ -57,7 +57,6 @@ DEF_NATIVE(sti, "sti");DEF_NATIVE (sti,"sti" );DEF_NATIVE (popf,"push %eax; popf" );DEF_NATIVE (pushf,"pushf; pop %eax" ); -DEF_NATIVE (pushf_cli,"pushf; pop %eax; cli" );DEF_NATIVE (iret,"iret" );DEF_NATIVE (sti_sysexit,"sti; sysexit" );
@@ -65,13 +64,12 @@ static const struct native_insns {const char *start, *end; } native_insns[] = { - [PARAVIRT_IRQ_DISABLE] = { start_cli, end_cli }, - [PARAVIRT_IRQ_ENABLE] = { start_sti, end_sti }, - [PARAVIRT_RESTORE_FLAGS] = { start_popf, end_popf }, - [PARAVIRT_SAVE_FLAGS] = { start_pushf, end_pushf }, - [PARAVIRT_SAVE_FLAGS_IRQ_DISABLE] = { start_pushf_cli, end_pushf_cli }, - [PARAVIRT_INTERRUPT_RETURN] = { start_iret, end_iret }, - [PARAVIRT_STI_SYSEXIT] = { start_sti_sysexit, end_sti_sysexit }, + [PARAVIRT_PATCH(irq_disable)] = { start_cli, end_cli }, + [PARAVIRT_PATCH(irq_enable)] = { start_sti, end_sti }, + [PARAVIRT_PATCH(restore_fl)] = { start_popf, end_popf }, + [PARAVIRT_PATCH(save_fl)] = { start_pushf, end_pushf }, + [PARAVIRT_PATCH(iret)] = { start_iret, end_iret }, + [PARAVIRT_PATCH(irq_enable_sysexit)] = { start_sti_sysexit, end_sti_sysexit }, };static unsigned native_patch (u8 type,u16 clobbers,void *insns,unsigned len)
===================================================================
arch/i386/kernel/vmi.c +6 -32
@@ -78,11 +78,6 @@ struct {#define MNEM_JMP 0xe9#define MNEM_RET 0xc3 -static char irq_save_disable_callout[] = { -MNEM_CALL ,0 ,0 ,0 ,0 , -MNEM_CALL ,0 ,0 ,0 ,0 , -MNEM_RET -};#define IRQ_PATCH_INT_MASK 0#define IRQ_PATCH_DISABLE 5
@@ -130,33 +125,17 @@ static unsigned vmi_patch(u8 type, u16 cstatic unsigned vmi_patch (u8 type,u16 clobbers,void *insns,unsigned len) {switch (type) { -case PARAVIRT_IRQ_DISABLE : +case PARAVIRT_PATCH (irq_disable):return patch_internal (VMI_CALL_DisableInterrupts, len, insns); -case PARAVIRT_IRQ_ENABLE : +case PARAVIRT_PATCH (irq_enable):return patch_internal (VMI_CALL_EnableInterrupts, len, insns); -case PARAVIRT_RESTORE_FLAGS : +case PARAVIRT_PATCH (restore_fl):return patch_internal (VMI_CALL_SetInterruptMask, len, insns); -case PARAVIRT_SAVE_FLAGS : +case PARAVIRT_PATCH (save_fl):return patch_internal (VMI_CALL_GetInterruptMask, len, insns); -case PARAVIRT_SAVE_FLAGS_IRQ_DISABLE : -if (len >=10 ) { -patch_internal (VMI_CALL_GetInterruptMask, len, insns); -patch_internal (VMI_CALL_DisableInterrupts, len-5 , insns+5 ); -return 10 ; - }else { -/* -* You bastards didn't leave enough room to -* patch save_flags_irq_disable inline. Patch -* to a helper -*/ -BUG_ON (len <5 ); - *(char *)insns =MNEM_CALL ; -patch_offset (insns, irq_save_disable_callout); -return 5 ; - } -case PARAVIRT_INTERRUPT_RETURN : +case PARAVIRT_PATCH (iret):return patch_internal (VMI_CALL_IRET , len, insns); -case PARAVIRT_STI_SYSEXIT : +case PARAVIRT_PATCH (irq_enable_sysexit):return patch_internal (VMI_CALL_SYSEXIT , len, insns);default :break ;
@@ -733,11 +712,6 @@ static inline int __init activate_vmi(vopara_fill (restore_fl, SetInterruptMask);para_fill (irq_disable, DisableInterrupts);para_fill (irq_enable, EnableInterrupts); -/* irq_save_disable !!! sheer pain */ -patch_offset (&irq_save_disable_callout[IRQ_PATCH_INT_MASK ], - (char *)paravirt_ops.save_fl); -patch_offset (&irq_save_disable_callout[IRQ_PATCH_DISABLE ], - (char *)paravirt_ops.irq_disable);#ifndef CONFIG_NO_IDLE_HZ para_fill (safe_halt, Halt);#else
===================================================================
include/asm-i386/paravirt.h +53 -54
161 lines not on this page · raw message
PATCH 12/26 Xen-paravirt_ops: Fix patch site clobbers to include return register
In reply to Jeremy FitzhardingeFix a few clobbers to include the return register. The clobbers set is the set of all registers modified (or may be modified) by the code snippet, regardless of whether it was deliberate or accidental.
--- include/asm-i386/paravirt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
===================================================================
include/asm-i386/paravirt.h +2 -2
@@ -556,7 +556,7 @@ static inline unsigned long __raw_local_ "popl %%edx; popl %%ecx") : "=a"(f): "m"(paravirt_ops.save_fl), paravirt_type(PARAVIRT_PATCH(save_fl)), - paravirt_clobber(CLBR_NONE) + paravirt_clobber(CLBR_EAX) : "memory", "cc");return f; }
@@ -610,7 +610,7 @@ static inline unsigned long __raw_local_ "%c[paravirt_cli_type]", "%c[paravirt_clobber]")#define STI_STRING _paravirt_alt("pushl %%ecx; pushl %%edx;" \ - "call *paravirt_ops+%c[irq_enable];" \ + "call *paravirt_ops+%c[irq_enable];" \ "popl %%edx; popl %%ecx", \ "%c[paravirt_cli_type]", "%c[paravirt_clobber]")
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
Quoted 16 lines · Jeremy Fitzhardinge
Fix a few clobbers to include the return register. The clobbers set is the set of all registers modified (or may be modified) by the code snippet, regardless of whether it was deliberate or accidental.
Signed-off-by Jeremy Fitzhardinge <[email protected]>Cc Rusty Russell <[email protected]>Cc Zachary Amsden <[email protected]>--- include/asm-i386/paravirt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)===================================================================
include/asm-i386/paravirt.h static inline unsigned long __raw_local_ · line 562
Quoted 8 lines
include/asm-i386/paravirt.h +1
-1@@ -556,7 +556,7 @@ static inline unsigned long __raw_local_ "popl %%edx; popl %%ecx") : "=a"(f): "m"(paravirt_ops.save_fl), paravirt_type(PARAVIRT_PATCH(save_fl)), - paravirt_clobber(CLBR_NONE) + paravirt_clobber(CLBR_EAX) : "memory", "cc");return f;
Has this been tested on older gcc's? I seem to recall them barfing over things like this.
Zach
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
Quoted 13 lines · Zachary Amsden
Quoted 12 lines · Jeremy Fitzhardinge
Fix a few clobbers to include the return register. The clobbers set is the set of all registers modified (or may be modified) by the code snippet, regardless of whether it was deliberate or accidental.
Signed-off-by Jeremy Fitzhardinge <[email protected]>Cc Rusty Russell <[email protected]>Cc Zachary Amsden <[email protected]>--- include/asm-i386/paravirt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 15 lines
include/asm-i386/paravirt.h static inline unsigned long __raw_local_ · line 562
Quoted 12 lines
===================================================================
include/asm-i386/paravirt.h +1
-1@@ -556,7 +556,7 @@ static inline unsigned long __raw_local_ "popl %%edx; popl %%ecx") : "=a"(f): "m"(paravirt_ops.save_fl), paravirt_type(PARAVIRT_PATCH(save_fl)), - paravirt_clobber(CLBR_NONE) + paravirt_clobber(CLBR_EAX) : "memory", "cc");return f;Has this been tested on older gcc's? I seem to recall them barfing over things like this.
Things like what? Do you mean the %[foo] asm parameter syntax? I think those versions are no longer supported - Arjan posted a patch a few days ago to convert a pile of asms to this form. Or do you mean something else?
J
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
Quoted 33 lines · Jeremy Fitzhardinge
Quoted 14 lines · Zachary Amsden
Quoted 12 lines · Jeremy Fitzhardinge
Fix a few clobbers to include the return register. The clobbers set is the set of all registers modified (or may be modified) by the code snippet, regardless of whether it was deliberate or accidental.
Signed-off-by Jeremy Fitzhardinge <[email protected]>Cc Rusty Russell <[email protected]>Cc Zachary Amsden <[email protected]>--- include/asm-i386/paravirt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Quoted 16 lines
include/asm-i386/paravirt.h static inline unsigned long __raw_local_ · line 562
Quoted 13 lines
===================================================================
include/asm-i386/paravirt.h +1
-1@@ -556,7 +556,7 @@ static inline unsigned long __raw_local_ "popl %%edx; popl %%ecx") : "=a"(f): "m"(paravirt_ops.save_fl), paravirt_type(PARAVIRT_PATCH(save_fl)), - paravirt_clobber(CLBR_NONE) + paravirt_clobber(CLBR_EAX) : "memory", "cc");return f;Has this been tested on older gcc's? I seem to recall them barfing over things like this.
Quoted 4 lines
Things like what? Do you mean the %[foo] asm parameter syntax? I think those versions are no longer supported - Arjan posted a patch a few days ago to convert a pile of asms to this form. Or do you mean something else?
I meant having an output in the clobber list, I didn't know we were dropping support for older versions already.
Zach
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
Quoted 8 lines · Zachary Amsden
Quoted 6 lines
Things like what? Do you mean the %[foo] asm parameter syntax? I think those versions are no longer supported - Arjan posted a patch a few days ago to convert a pile of asms to this form. Or do you mean something else?
I meant having an output in the clobber list,
There's no output in the clobber list, just "memory" and "cc". The paravirt_clobber() is the stuff which gets put in the .parainstructions section which tells the patcher what registers are expected to be modified at that callsite (including both temp and output registers).
J
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 6 lines · Zachary Amsden
Quoted 6 lines
Things like what? Do you mean the %[foo] asm parameter syntax? I think those versions are no longer supported - Arjan posted a patch a few days ago to convert a pile of asms to this form. Or do you mean something else?
Quoted 8 lines
Quoted 2 lines
I meant having an output in the clobber list,
There's no output in the clobber list, just "memory" and "cc". The
paravirt_clobber()is the stuff which gets put in the .parainstructions section which tells the patcher what registers are expected to be modified at that callsite (including both temp and output registers).
Ah, ok. I didn't think gcc had grown the appropriate antlers yet to allow outputs in the clobber list. But the paravirt-clobbers, yes that makes sense.
Zach
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
In reply to Jeremy FitzhardingeQuoted 3 lines · Jeremy Fitzhardinge
Fix a few clobbers to include the return register. The clobbers set is the set of all registers modified (or may be modified) by the code snippet, regardless of whether it was deliberate or accidental.
i guess we need these paravirt.h clobber fixes for v2.6.21 (VMI) too, correct?
Ingo
Re: [patch 12/26] Xen-paravirt_ops: Fix patch site clobbers to include return register
Quoted 10 lines · Ingo Molnar
Quoted 4 lines · Jeremy Fitzhardinge
Fix a few clobbers to include the return register. The clobbers set is the set of all registers modified (or may be modified) by the code snippet, regardless of whether it was deliberate or accidental.
i guess we need these
paravirt.hclobber fixes for v2.6.21 (VMI) too, correct?
No, I don't think so. These aren't gcc-style clobbers, but a marker in the .parainstruction section about what registers may be clobbered by
the patched in code. VMI doesn't use that info, and/or doesn't clobber
more than is allowed at each site, so it isn't an issue.
J
PATCH 13/26 Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Jeremy FitzhardingeWrap a set of interesting paravirt_ops calls in a wrapper which makes the callsites available for patching. Unfortunately this is pretty ugly because there's no way to get gcc to generate a function call, but also wrap just the callsite itself with the necessary labels.
This patch supports functions with 0-4 arguments, and either void or returning a value. 64-bit arguments must be split into a pair of 32-bit arguments (lower word first). Small structures are returned in registers.
--- include/asm-i386/paravirt.h | 577 ++++++++++++++++++++++++++++++++----------- 1 file changed, 442 insertions(+), 135 deletions(-)
===================================================================
include/asm-i386/paravirt.h +442 -135
@@ -33,6 +33,222 @@#define paravirt_alt (insn_string) \ _paravirt_alt(insn_string, "%c[paravirt_typenum]", "%c[paravirt_clobber]") +#define PVOP_CALL0 (__rettype, __op) \ + ({ \ + __rettype __ret; \ + if (sizeof(__rettype) > sizeof(unsigned long)) { \ + unsigned long long __tmp; \ + unsigned long __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=A" (__tmp), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } else { \ + unsigned long __tmp, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__tmp), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } \ + __ret; \ + }) +#define PVOP_VCALL0 (__op) \ + ({ \ + unsigned long __eax, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__eax), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + }) + +#define PVOP_CALL1 (__rettype, __op, arg1) \ + ({ \ + __rettype __ret; \ + if (sizeof(__rettype) > sizeof(unsigned long)) { \ + unsigned long long __tmp; \ + unsigned long __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=A" (__tmp), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "a" ((u32)(arg1)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } else { \ + unsigned long __tmp, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__tmp), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } \ + __ret; \ + }) +#define PVOP_VCALL1 (__op, arg1) \ + ({ \ + unsigned long __eax, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__eax), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + }) + +#define PVOP_CALL2 (__rettype, __op, arg1, arg2) \ + ({ \ + __rettype __ret; \ + if (sizeof(__rettype) > sizeof(unsigned long)) { \ + unsigned long long __tmp; \ + unsigned long __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=A" (__tmp), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "a" ((u32)(arg1)), \ + "d" ((u32)(arg2)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } else { \ + unsigned long __tmp, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__tmp), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + "1" ((u32)(arg2)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } \ + __ret; \ + }) +#define PVOP_VCALL2 (__op, arg1, arg2) \ + ({ \ + unsigned long __eax, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__eax), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + "1" ((u32)(arg2)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + }) + +#define PVOP_CALL3 (__rettype, __op, arg1, arg2, arg3) \ + ({ \ + __rettype __ret; \ + if (sizeof(__rettype) > sizeof(unsigned long)) { \ + unsigned long long __tmp; \ + unsigned long __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=A" (__tmp), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "a" ((u32)(arg1)), \ + "d" ((u32)(arg2)), \ + "1" ((u32)(arg3)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } else { \ + unsigned long __tmp, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__tmp), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + "1" ((u32)(arg2)), \ + "2" ((u32)(arg3)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } \ + __ret; \ + }) +#define PVOP_VCALL3 (__op, arg1, arg2, arg3) \ + ({ \ + unsigned long __eax, __edx, __ecx; \ + asm volatile(paravirt_alt("call *%[op]") \ + : "=a" (__eax), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + "1" ((u32)(arg2)), \ + "2" ((u32)(arg3)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + }) + +#define PVOP_CALL4 (__rettype, __op, arg1, arg2, arg3, arg4) \ + ({ \ + __rettype __ret; \ + if (sizeof(__rettype) > sizeof(unsigned long)) { \ + unsigned long long __tmp; \ + unsigned long __ecx; \ + asm volatile("push %[_arg4]; " \ + paravirt_alt("call *%[op]") \ + "lea 4(%%esp),%%esp" \ + : "=A" (__tmp), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "a" ((u32)(arg1)), \ + "d" ((u32)(arg2)), \ + "1" ((u32)(arg3)), \ + [_arg4] "mr" ((u32)(arg4)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc",); \ + __ret = (__rettype)__tmp; \ + } else { \ + unsigned long __tmp, __edx, __ecx; \ + asm volatile("push %[_arg4]; " \ + paravirt_alt("call *%[op]") \ + "lea 4(%%esp),%%esp" \ + : "=a" (__tmp), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + "1" ((u32)(arg2)), \ + "2" ((u32)(arg3)), \ + [_arg4]"mr" ((u32)(arg4)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + __ret = (__rettype)__tmp; \ + } \ + __ret; \ + }) +#define PVOP_VCALL4 (__op, arg1, arg2, arg3, arg4) \ + ({ \ + unsigned long __eax, __edx, __ecx; \ + asm volatile("push %[_arg4]; " \ + paravirt_alt("call *%[op]") \ + "lea 4(%%esp),%%esp" \ + : "=a" (__eax), "=d" (__edx), "=c" (__ecx) \ + : [op] "m" (paravirt_ops.__op), \ + "0" ((u32)(arg1)), \ + "1" ((u32)(arg2)), \ + "2" ((u32)(arg3)), \ + [_arg4]"mr" ((u32)(arg4)), \ + paravirt_type(PARAVIRT_PATCH(__op)), \ + paravirt_clobber(CLBR_ANY) \ + : "memory", "cc"); \ + }) +struct thread_struct ;struct Xgt_desc_struct ;struct tss_struct ;
@@ -110,14 +326,10 @@ struct paravirt_opsvoid (*set_ldt)(const void *desc,unsigned entries);unsigned long (*store_tr)(void );void (*load_tls)(struct thread_struct *t,unsigned int cpu); -void (*write_ldt_entry)(void *dt,int entrynum, -u32 low,u32 high); -void (*write_gdt_entry)(void *dt,int entrynum, -u32 low,u32 high); -void (*write_idt_entry)(void *dt,int entrynum, -u32 low,u32 high); -void (*load_esp0)(struct tss_struct *tss, -struct thread_struct *thread); +void (*write_ldt_entry)(void *dt,int entrynum,u32 low,u32 high); +void (*write_gdt_entry)(void *dt,int entrynum,u32 low,u32 high); +void (*write_idt_entry)(void *dt,int entrynum,u32 low,u32 high); +void (*load_esp0)(struct tss_struct *tss,struct thread_struct *thread);void (*set_iopl_mask)(unsigned mask);
@@ -194,151 +406,165 @@ struct paravirt_opsextern struct paravirt_ops paravirt_ops; -#define paravirt_enabled () (paravirt_ops.paravirt_enabled) +static inline int paravirt_enabled (void ) +{ +return paravirt_ops.paravirt_enabled; +}static inline void load_esp0 (struct tss_struct *tss,struct thread_struct *thread) { - paravirt_ops.load_esp0 (tss, thread); -} - -#define ARCH_SETUP paravirt_ops.arch_setup(); +PVOP_VCALL2 (load_esp0, tss, thread); +} + +#define ARCH_SETUP PVOP_VCALL0(arch_setup);static inline unsigned long get_wallclock (void ) { -return paravirt_ops.get_wallclock (); + return PVOP_CALL0(unsigned long, get_wallclock); }static inline int set_wallclock (unsigned long nowtime) { -return paravirt_ops.set_wallclock (nowtime); +return PVOP_CALL1 (int, set_wallclock, nowtime); }static inline void do_time_init (void ) { -return paravirt_ops.time_init (); +PVOP_VCALL0 (time_init); }/* The paravirtualized CPUID instruction. */ static inline void __cpuid (unsigned int *eax,unsigned int *ebx,unsigned int *ecx,unsigned int *edx) { - paravirt_ops.cpuid (eax, ebx, ecx, edx); +PVOP_VCALL4 (cpuid, eax, ebx, ecx, edx); }/* * These special macros can be used to get or set a debugging register */ -#define get_debugreg (var, reg) var = paravirt_ops.get_debugreg(reg) -#define set_debugreg (val, reg) paravirt_ops.set_debugreg(reg, val) - -#define clts () paravirt_ops.clts() - -#define read_cr0 () paravirt_ops.read_cr0() -#define write_cr0 (x) paravirt_ops.write_cr0(x) - -#define read_cr2 () paravirt_ops.read_cr2() -#define write_cr2 (x) paravirt_ops.write_cr2(x) - -#define read_cr3 () paravirt_ops.read_cr3() -#define write_cr3 (x) paravirt_ops.write_cr3(x) - -#define read_cr4 () paravirt_ops.read_cr4() -#define read_cr4_safe (x) paravirt_ops.read_cr4_safe() -#define write_cr4 (x) paravirt_ops.write_cr4(x) - -#define raw_ptep_get_and_clear (xp) (paravirt_ops.ptep_get_and_clear(xp)) +#define get_debugreg (var, reg) var = PVOP_CALL1(unsigned long, get_debugreg, reg) +#define set_debugreg (val, reg) PVOP_VCALL2(set_debugreg, reg, val) + +#define clts () PVOP_VCALL0(clts) + +#define read_cr0 () PVOP_CALL0(unsigned long, read_cr0) +#define write_cr0 (x) PVOP_VCALL1(write_cr0, x) + +#define read_cr2 () PVOP_CALL0(unsigned long, read_cr2) +#define write_cr2 (x) PVOP_VCALL1(write_cr2, x) + +#define read_cr3 () PVOP_CALL0(unsigned long, read_cr3) +#define write_cr3 (x) PVOP_VCALL1(write_cr3, x) + +#define read_cr4 () PVOP_CALL0(unsigned long, read_cr4) +#define read_cr4_safe (x) PVOP_CALL0(unsigned long, read_cr4_safe) +#define write_cr4 (x) PVOP_VCALL1(write_cr4, x)static inline void raw_safe_halt (void ) { - paravirt_ops.safe_halt (); +PVOP_VCALL0 (safe_halt); }static inline void halt (void ) { - paravirt_ops.safe_halt (); -} -#define wbinvd () paravirt_ops.wbinvd() +PVOP_VCALL0 (safe_halt); +} +#define wbinvd () PVOP_VCALL0(wbinvd)#define get_kernel_rpl () (paravirt_ops.kernel_rpl)#define rdmsr (msr,val1,val2) do { \ int _err; \ - u64 _l = paravirt_ops.read_msr(msr,&_err); \ + u64 _l = PVOP_CALL2(u64, read_msr, msr, &_err); \ val1 = (u32)_l; \ val2 = _l >> 32; \ } while(0) -#define wrmsr (msr,val1,val2) do { \ - u64 _l = ((u64)(val2) << 32) | (val1); \ - paravirt_ops.write_msr((msr), _l); \ -} while(0) +#define wrmsr (msr,val1,val2) PVOP_VCALL3(write_msr, msr, val1, val2)#define rdmsrl (msr,val) do { \ int _err; \ - val = paravirt_ops.read_msr((msr),&_err); \ + val = PVOP_CALL2(u64, read_msr, msr, &_err); \ } while(0) -#define wrmsrl (msr,val) (paravirt_ops.write_msr((msr),(val))) -#define wrmsr_safe (msr,a,b) ({ \ - u64 _l = ((u64)(b) << 32) | (a); \ - paravirt_ops.write_msr((msr),_l); \ -}) +#define wrmsrl (msr,val) PVOP_CALL3(int, write_msr, msr, val, 0) +#define wrmsr_safe (msr,a,b) PVOP_CALL3(int, write_msr, msr, a, b)/* rdmsr with exception handling */ #define rdmsr_safe (msr,a,b) ({ \ int _err; \ - u64 _l = paravirt_ops.read_msr(msr,&_err); \ + u64 _l = PVOP_CALL2(u64, read_msr, msr, &_err); \ (*a) = (u32)_l; \ (*b) = _l >> 32; \ _err; })#define rdtsc (low,high) do { \ - u64 _l = paravirt_ops.read_tsc(); \ + u64 _l = PVOP_CALL0(u64, read_tsc); \ low = (u32)_l; \ high = _l >> 32; \ } while(0)#define rdtscl (low) do { \ - u64 _l = paravirt_ops.read_tsc(); \ + u64 _l = PVOP_CALL0(u64, read_tsc); \ low = (int)_l; \ } while(0) -#define rdtscll (val) (val = paravirt_ops.read_tsc()) +#define rdtscll (val) (val = PVOP_CALL0(u64, read_tsc))#define write_tsc (val1,val2) wrmsr(0x10, val1, val2)#define rdpmc (counter,low,high) do { \ - u64 _l = paravirt_ops.read_pmc(); \ + u64 _l = PVOP_CALL0(u64, read_pmc); \ low = (u32)_l; \ high = _l >> 32; \ } while(0) -#define load_TR_desc () (paravirt_ops.load_tr_desc()) -#define load_gdt (dtr) (paravirt_ops.load_gdt(dtr)) -#define load_idt (dtr) (paravirt_ops.load_idt(dtr)) -#define set_ldt (addr, entries) (paravirt_ops.set_ldt((addr), (entries))) -#define store_gdt (dtr) (paravirt_ops.store_gdt(dtr)) -#define store_idt (dtr) (paravirt_ops.store_idt(dtr)) -#define store_tr (tr) ((tr) = paravirt_ops.store_tr()) -#define load_TLS (t,cpu) (paravirt_ops.load_tls((t),(cpu))) -#define write_ldt_entry (dt, entry, low, high) \ - (paravirt_ops.write_ldt_entry((dt), (entry), (low), (high))) -#define write_gdt_entry (dt, entry, low, high) \ - (paravirt_ops.write_gdt_entry((dt), (entry), (low), (high))) -#define write_idt_entry (dt, entry, low, high) \ - (paravirt_ops.write_idt_entry((dt), (entry), (low), (high))) -#define set_iopl_mask (mask) (paravirt_ops.set_iopl_mask(mask)) - -#define __pte (x) paravirt_ops.make_pte(x) -#define __pgd (x) paravirt_ops.make_pgd(x) - -#define pte_val (x) paravirt_ops.pte_val(x) -#define pgd_val (x) paravirt_ops.pgd_val(x) - -#ifdef CONFIG_X86_PAE -#define __pmd (x) paravirt_ops.make_pmd(x) -#define pmd_val (x) paravirt_ops.pmd_val(x) -#endif +static inline void load_TR_desc (void ) +{ +PVOP_VCALL0 (load_tr_desc); +} +static inline void load_gdt (const struct Xgt_desc_struct *dtr) +{ +PVOP_VCALL1 (load_gdt, dtr); +} +static inline void load_idt (const struct Xgt_desc_struct *dtr) +{ +PVOP_VCALL1 (load_idt, dtr); +} +static inline void set_ldt (const void *addr,unsigned entries) +{ +PVOP_VCALL2 (set_ldt, addr, entries); +} +static inline void store_gdt (struct Xgt_desc_struct *dtr) +{ +PVOP_VCALL1 (store_gdt, dtr); +} +static inline void store_idt (struct Xgt_desc_struct *dtr) +{ +PVOP_VCALL1 (store_idt, dtr); +} +#define store_tr (tr) ((tr) = PVOP_CALL0(unsigned long, store_tr)) +static inline void load_TLS (struct thread_struct *t,unsigned cpu) +{ +PVOP_VCALL2 (load_tls, t, cpu); +} +static inline void write_ldt_entry (void *dt,int entry,u32 low,u32 high) +{ +PVOP_VCALL4 (write_ldt_entry, dt, entry, low, high); +} +static inline void write_gdt_entry (void *dt,int entry,u32 low,u32 high) +{ +PVOP_VCALL4 (write_gdt_entry, dt, entry, low, high); +} +static inline void write_idt_entry (void *dt,int entry,u32 low,u32 high) +{ +PVOP_VCALL4 (write_idt_entry, dt, entry, low, high); +} +static inline void set_iopl_mask (unsigned mask) +{ +PVOP_VCALL1 (set_iopl_mask, mask); +}/* The paravirtualized I/O functions */ static inline void slow_down_io (void ) {
@@ -356,27 +582,27 @@ static inline void slow_down_io(void) {*/ static inline void apic_write (unsigned long reg,unsigned long v) { - paravirt_ops.apic_write (reg,v); +PVOP_VCALL2 (apic_write, reg, v); }static inline void apic_write_atomic (unsigned long reg,unsigned long v) { - paravirt_ops.apic_write_atomic (reg,v); +PVOP_VCALL2 (apic_write_atomic, reg, v); }static inline unsigned long apic_read (unsigned long reg) { -return paravirt_ops.apic_read (reg); + return PVOP_CALL1(unsigned long, apic_read, reg); }static inline void setup_boot_clock (void ) { - paravirt_ops.setup_boot_clock (); +PVOP_VCALL0 (setup_boot_clock); }static inline void setup_secondary_clock (void ) { - paravirt_ops.setup_secondary_clock (); +PVOP_VCALL0 (setup_secondary_clock); }#endif
@@ -396,91 +622,172 @@ static inline void startup_ipi_hook(intstatic inline void startup_ipi_hook (int phys_apicid,unsigned long start_eip,unsigned long start_esp) { -return paravirt_ops.startup_ipi_hook (phys_apicid, start_eip, start_esp); +PVOP_VCALL3 (startup_ipi_hook, phys_apicid, start_eip, start_esp); }#endif static inline void paravirt_activate_mm (struct mm_struct *prev,struct mm_struct *next) { - paravirt_ops.activate_mm (prev, next); +PVOP_VCALL2 (activate_mm, prev, next); }static inline void arch_dup_mmap (struct mm_struct *oldmm,struct mm_struct *mm) { - paravirt_ops.dup_mmap (oldmm, mm); +PVOP_VCALL2 (dup_mmap, oldmm, mm); }static inline void arch_exit_mmap (struct mm_struct *mm) { - paravirt_ops.exit_mmap (mm); -} - -#define __flush_tlb () paravirt_ops.flush_tlb_user() -#define __flush_tlb_global () paravirt_ops.flush_tlb_kernel() -#define __flush_tlb_single (addr) paravirt_ops.flush_tlb_single(addr) - -#define paravirt_alloc_pt (pfn) paravirt_ops.alloc_pt(pfn) -#define paravirt_release_pt (pfn) paravirt_ops.release_pt(pfn) - -#define paravirt_alloc_pd (pfn) paravirt_ops.alloc_pd(pfn) +PVOP_VCALL1 (exit_mmap, mm); +} + +#define __flush_tlb () PVOP_VCALL0(flush_tlb_user) +#define __flush_tlb_global () PVOP_VCALL0(flush_tlb_kernel) +#define __flush_tlb_single (addr) PVOP_VCALL1(flush_tlb_single, addr) + +#define paravirt_alloc_pt (pfn) PVOP_VCALL1(alloc_pt, pfn) +#define paravirt_release_pt (pfn) PVOP_VCALL1(release_pt, pfn) + +#define paravirt_alloc_pd (pfn) PVOP_VCALL1(alloc_pd, pfn)#define paravirt_alloc_pd_clone (pfn, clonepfn, start, count) \ - paravirt_ops.alloc_pd_clone(pfn, clonepfn, start, count) -#define paravirt_release_pd (pfn) paravirt_ops.release_pd(pfn) + PVOP_VCALL4(alloc_pd_clone, pfn, clonepfn, start, count) +#define paravirt_release_pd (pfn) PVOP_VCALL1(release_pd, pfn) + +static inline void pte_update (struct mm_struct *mm,unsigned long addr,pte_t *ptep) +{ +PVOP_VCALL3 (pte_update, mm, addr, ptep); +} + +static inline void pte_update_defer (struct mm_struct *mm,unsigned long addr,pte_t *ptep) +{ +PVOP_VCALL3 (pte_update_defer, mm, addr, ptep); +} + +#ifdef CONFIG_X86_PAE +static inline pte_t __pte (unsigned long long val) +{ + unsigned long long ret = PVOP_CALL2(unsigned long long, make_pte, val, val >> 32); +return (pte_t ) { ret, ret >>32 }; +} + +static inline pmd_t __pmd (unsigned long long val) +{ + return (pmd_t) { PVOP_CALL2(unsigned long long, make_pmd, val, val >> 32) }; +} + +static inline pgd_t __pgd (unsigned long long val) +{ + return (pgd_t) { PVOP_CALL2(unsigned long long, make_pgd, val, val >> 32) }; +} + +static inline unsigned long long pte_val (pte_t x) +{ + return PVOP_CALL2(unsigned long long, pte_val, x.pte_low, x.pte_high); +} + +static inline unsigned long long pmd_val (pmd_t x) +{ + return PVOP_CALL2(unsigned long long, pmd_val, x.pmd, x.pmd >> 32); +} + +static inline unsigned long long pgd_val (pgd_t x) +{ + return PVOP_CALL2(unsigned long long, pgd_val, x.pgd, x.pgd >> 32); +}static inline void set_pte (pte_t *ptep,pte_t pteval) { - paravirt_ops.set_pte (ptep, pteval); +PVOP_VCALL3 (set_pte, ptep, pteval.pte_low, pteval.pte_high); }static inline void set_pte_at (struct mm_struct *mm,unsigned long addr,pte_t *ptep,pte_t pteval) { +/* 5 arg words */ paravirt_ops.set_pte_at (mm, addr, ptep, pteval); } +static inline void set_pte_atomic (pte_t *ptep,pte_t pteval) +{ +PVOP_VCALL3 (set_pte_atomic, ptep, pteval.pte_low, pteval.pte_high); +} + +static inline void set_pte_present (struct mm_struct *mm,unsigned long addr, +pte_t *ptep,pte_t pte) +{ +/* 5 arg words */ + paravirt_ops.set_pte_present (mm, addr, ptep, pte); +} +static inline void set_pmd (pmd_t *pmdp,pmd_t pmdval) { - paravirt_ops.set_pmd (pmdp, pmdval); -} - -static inline void pte_update (struct mm_struct *mm,u32 addr,pte_t *ptep) -{ - paravirt_ops.pte_update (mm, addr, ptep); -} - -static inline void pte_update_defer (struct mm_struct *mm,u32 addr,pte_t *ptep) -{ - paravirt_ops.pte_update_defer (mm, addr, ptep); -} - -#ifdef CONFIG_X86_PAE -static inline void set_pte_atomic (pte_t *ptep,pte_t pteval) -{ - paravirt_ops.set_pte_atomic (ptep, pteval); -} - -static inline void set_pte_present (struct mm_struct *mm,unsigned long addr,pte_t *ptep,pte_t pte) -{ - paravirt_ops.set_pte_present (mm, addr, ptep, pte); +PVOP_VCALL3 (set_pmd, pmdp, pmdval.pmd, pmdval.pmd >>32 ); }static inline void set_pud (pud_t *pudp,pud_t pudval) { - paravirt_ops.set_pud (pudp, pudval); +PVOP_VCALL3 (set_pud, pudp, pudval.pgd.pgd, pudval.pgd.pgd >>32 ); }static inline void pte_clear (struct mm_struct *mm,unsigned long addr,pte_t *ptep) { - paravirt_ops.pte_clear (mm, addr, ptep); +PVOP_VCALL3 (pte_clear, mm, addr, ptep); }static inline void pmd_clear (pmd_t *pmdp) { - paravirt_ops.pmd_clear (pmdp); -} -#endif +PVOP_VCALL1 (pmd_clear, pmdp); +} + +static inline pte_t raw_ptep_get_and_clear (pte_t *p) +{ + unsigned long long val = PVOP_CALL1(unsigned long long, ptep_get_and_clear, p); +return (pte_t ) { val, val >>32 }; +} +#else /* !CONFIG_X86_PAE */ +static inline pte_t __pte (unsigned long val) +{ + return (pte_t) { PVOP_CALL1(unsigned long, make_pte, val) }; +} + +static inline pgd_t __pgd (unsigned long val) +{ + return (pgd_t) { PVOP_CALL1(unsigned long, make_pgd, val) }; +} + +static inline unsigned long pte_val (pte_t x) +{ + return PVOP_CALL1(unsigned long, pte_val, x.pte_low); +} + +static inline unsigned long pgd_val (pgd_t x) +{ + return PVOP_CALL1(unsigned long, pgd_val, x.pgd); +} + +static inline void set_pte (pte_t *ptep,pte_t pteval) +{ +PVOP_VCALL2 (set_pte, ptep, pteval.pte_low); +} + +static inline void set_pte_at (struct mm_struct *mm,unsigned long addr, +pte_t *ptep,pte_t pteval) +{ +PVOP_VCALL4 (set_pte_at, mm, addr, ptep, pteval.pte_low); +} + +static inline void set_pmd (pmd_t *pmdp,pmd_t pmdval) +{ +PVOP_VCALL2 (set_pmd, pmdp, pmdval.pud.pgd.pgd); +} + +static inline pte_t raw_ptep_get_and_clear (pte_t *p) +{ + return (pte_t) { PVOP_CALL1(unsigned long, ptep_get_and_clear, p) }; +} +#endif /* CONFIG_X86_PAE */ /* Lazy mode for batching updates / context switch */ #define PARAVIRT_LAZY_NONE 0
@@ -488,12 +795,12 @@ static inline void pmd_clear(pmd_t *pmdp#define PARAVIRT_LAZY_CPU 2#define __HAVE_ARCH_ENTER_LAZY_CPU_MODE -#define arch_enter_lazy_cpu_mode () paravirt_ops.set_lazy_mode(PARAVIRT_LAZY_CPU) -#define arch_leave_lazy_cpu_mode () paravirt_ops.set_lazy_mode(PARAVIRT_LAZY_NONE) +#define arch_enter_lazy_cpu_mode () PVOP_VCALL1(set_lazy_mode, PARAVIRT_LAZY_CPU) +#define arch_leave_lazy_cpu_mode () PVOP_VCALL1(set_lazy_mode, PARAVIRT_LAZY_NONE)#define __HAVE_ARCH_ENTER_LAZY_MMU_MODE -#define arch_enter_lazy_mmu_mode () paravirt_ops.set_lazy_mode(PARAVIRT_LAZY_MMU) -#define arch_leave_lazy_mmu_mode () paravirt_ops.set_lazy_mode(PARAVIRT_LAZY_NONE) +#define arch_enter_lazy_mmu_mode () PVOP_VCALL1(set_lazy_mode, PARAVIRT_LAZY_MMU) +#define arch_leave_lazy_mmu_mode () PVOP_VCALL1(set_lazy_mode, PARAVIRT_LAZY_NONE)void _paravirt_nop (void );#define paravirt_nop ((void *)_paravirt_nop)
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · Jeremy Fitzhardinge
Wrap a set of interesting
paravirt_opscalls in a wrapper which makes the callsites available for patching. Unfortunately this is pretty ugly because there's no way to get gcc to generate a function call, but also wrap just the callsite itself with the necessary labels.This patch supports functions with 0-4 arguments, and either void or returning a value. 64-bit arguments must be split into a pair of 32-bit arguments (lower word first). Small structures are returned in registers.
ugh. This is beyond ugly! Why dont we just compile two images, one for Xen and one for native, do two passes to get those two images and 'merge' them into a single vmlinuz (so that we still have a 'single' kernel unit to deal with on the distro side). This way we avoid all this crazy, limited, fragile patchery business...
Ingo
David Miller
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
From: Ingo Molnar <[email protected]>
Date: Fri, 16 Mar 2007 10:24:45 +0100
Quoted 5 lines
ugh. This is beyond ugly! Why dont we just compile two images, one for Xen and one for native, do two passes to get those two images and 'merge' them into a single vmlinuz (so that we still have a 'single' kernel unit to deal with on the distro side). This way we avoid all this crazy, limited, fragile patchery business...
Perhaps the problem can be dealt with using ELF relocations.
There is another case, discussed yesterday on netdev, where run-time resolution of ELF relocations would be useful (for very-very-very-read-only variables) so if it can solve this problem too it would be nice to have a generic infrastructure for it.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · David Miller
From: Ingo Molnar <[email protected]>
Date: Fri, 16 Mar 2007 10:24:45 +0100Quoted 5 lines
ugh. This is beyond ugly! Why dont we just compile two images, one for Xen and one for native, do two passes to get those two images and 'merge' them into a single vmlinuz (so that we still have a 'single' kernel unit to deal with on the distro side). This way we avoid all this crazy, limited, fragile patchery business...
Quoted 6 lines
Perhaps the problem can be dealt with using ELF relocations.
There is another case, discussed yesterday on netdev, where run-time resolution of ELF relocations would be useful (for very-very-very-read-only variables) so if it can solve this problem too it would be nice to have a generic infrastructure for it.
yeah, and i really think this is very fundamental: we've already got a very nice tool that can do things like detect when we are paravirt and optimize and patch things in a machine-specific way. It can even reorder instructions and simulate the CPU's pipeline state and do very smart optimizations based on that. It's a really neat thing, they call it "GCC".
Limited, instruction-level patching like alternatives.h is fine because that makes it easier to support multiple, incompatible CPU architectures, without having to do a hugely intrusive split at the kernel RPM level.
but the level of 'binary patching' done by the paravirt and Xen goes way beyond that, and the changes here really underscore that we:
_should not emulate the closed source world_
There the only solution is to binary-patch - because they have no source code. But here, we've got all the source code.
splitting the images and simply extending vmlinuz to have a 'multi-image' format would not only eliminate a huge amount of hookery, it would also solve the problem of CONFIG_PARAVIRT bloating the native kernel's codepaths by ~7%.
nobody wants to boot a xen-paravirt kernel from a floppy, so image size is not an issue. In-RAM overhead would in fact be /reduced/, because currently all the paravirt overhead hits both the native and the paravirt kernel. Nor would /all/ of the vmlinuz have to be replicated in the images - it's enough to replicate only those functions that truly differ between the two build methods.
Ingo
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 4 lines · Ingo Molnar
Quoted 2 lines · David Miller
Perhaps the problem can be dealt with using ELF relocations.
Quoted 8 lines
Quoted 4 lines
There is another case, discussed yesterday on netdev, where run-time resolution of ELF relocations would be useful (for very-very-very-read-only variables) so if it can solve this problem too it would be nice to have a generic infrastructure for it.
yeah, and i really think this is very fundamental: [...]
I think what Dave is suggesting is that we use the reloc information the compiler generates to find the patchable callsites rather than have special wrappers. This is an interesting idea.
Quoted 7 lines
Limited, instruction-level patching like
alternatives.his fine because that makes it easier to support multiple, incompatible CPU architectures, without having to do a hugely intrusive split at the kernel RPM level.but the level of 'binary patching' done by the paravirt and Xen goes way beyond that,
Not really. There are only three cases:
1. replace an indirect call with a direct call
2. nop out a callsite
3. patch in a short inline sequence
And as I pointed out, this is used by all pv_op backends, using a common piece of code to implement at least 1 and 2. 3 could be implemented semi-generically by using rules like "if (func == native_sti) { patch("sti"); }", which would cover many cases where a hypervisor doesn't need any special handling for a particular operation.
The goal is to eliminate the cost of the indirect calls with nice predictable indirect calls. There's a 1 byte/callsite overhead, but I don't think that's a horrible overhead.
And, at worst, its only a little more complex than the kinds of transformations.
Ideally, its a mechanism which could be used elsewhere. It applies with you have some kind of ops_vector table which is updated once (or perhaps very rarely), and you don't want to wear the overhead of indirect calls everywhere.
Quoted 7 lines
and the changes here really underscore that we:
_shouldnot emulate the closed sourceworld_There the only solution is to binary-patch - because they have no source code. But here, we've got all the source code.
I don't think this is a relevant comparison. This is purely a matter of optimising out unnecessary indirect calls.
Quoted 6 lines
nobody wants to boot a xen-paravirt kernel from a floppy, so image size is not an issue. In-RAM overhead would in fact be /reduced/, because currently all the paravirt overhead hits both the native and the paravirt kernel. Nor would /all/ of the vmlinuz have to be replicated in the images - it's enough to replicate only those functions that truly differ between the two build methods.
One of the explicit goals of pv_ops was to allow a single kernel to either boot on native hardware or under any one of the supported hypervisors, explicitly to avoid having to manage multiple kernel images. Compiling the kernel N+1 times for N hypervisors, and then bundling them up in some kind of multi-image format doesn't seem like a particularly good tradeoff. The kernel RPM on my machine here is already ~50Mbytes; expanding that to 250Mbytes to support native, Xen, vmi, lguest and kvm doesn't seem reasonable.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to David MillerQuoted 6 lines · David Miller
Perhaps the problem can be dealt with using ELF relocations.
There is another case, discussed yesterday on netdev, where run-time resolution of ELF relocations would be useful (for very-very-very-read-only variables) so if it can solve this problem too it would be nice to have a generic infrastructure for it.
That's an interesting idea. Have you or anyone else looked at what it would take to code up?
For this case, I guess you'd walk the relocs looking for references into the paravirt_ops structure. You'd need to check that was a reference from an indirect jump or call instruction, which would identify a patchable callsite. The offset into the pv_ops structure would identify which operation is involved. That would be enough to use the existing paravirt_ops patch machinery to insert a patch, though it doesn't provide quite as much information as the current scheme. The current scheme also tells us how much space is available for patching over, and what registers the patched-in code can use/clobber.
What are the netdev requirements?
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 6 lines · David Miller
Perhaps the problem can be dealt with using ELF relocations.
There is another case, discussed yesterday on netdev, where run-time resolution of ELF relocations would be useful (for very-very-very-read-only variables) so if it can solve this problem too it would be nice to have a generic infrastructure for it.
Quoted 8 lines
That's an interesting idea. Have you or anyone else looked at what it would take to code up?
For this case, I guess you'd walk the relocs looking for references into the
paravirt_opsstructure. You'd need to check that was a reference from an indirect jump or call instruction, which would identify a patchable callsite. The offset into thepv_opsstructure would identify which operation is involved.
I wrote a whole email on ways to do this, BUT...
Perhaps our patching code can already be vastly simplified to something like:
#define pv_patch(call, args...) \
asm volatile("8888:");
call(args);
asm volatile("8889:"
[ stuff to put 8889, 8888 and call in fixup section ]The patching code doesn't even need to decode between those two: it simply looks for an indirect call insn (0xff 0xd?). If it finds more than one, it doesn't patch. If it only finds one, it patches. We'll probably hit them all just doing that.
Quoted 1 line
What are the netdev requirements?
Reading Ben LaHaise's (very cool!) patch, it's not clear that using reloc postprocessing is going to be clearer than open-coding it as he has done.
Rusty.
David Miller
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
From: Rusty Russell <[email protected]>
Date: Sat, 17 Mar 2007 21:33:58 +1100
Quoted 12 lines · Jeremy Fitzhardinge
Quoted 11 lines
Quoted 6 lines · David Miller
Perhaps the problem can be dealt with using ELF relocations.
There is another case, discussed yesterday on netdev, where run-time resolution of ELF relocations would be useful (for very-very-very-read-only variables) so if it can solve this problem too it would be nice to have a generic infrastructure for it.
That's an interesting idea. Have you or anyone else looked at what it would take to code up?
Quoted 7 lines
Quoted 5 lines
For this case, I guess you'd walk the relocs looking for references into the
paravirt_opsstructure. You'd need to check that was a reference from an indirect jump or call instruction, which would identify a patchable callsite. The offset into thepv_opsstructure would identify which operation is involved.I wrote a whole email on ways to do this, BUT...
The idea is _NOT_ that you go look for references to the paravirt_ops members structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...
Instead you get rid of paravirt ops completely, and you call functions whose symbol name will not resolve in the initial kernel link.
You do an initial link of the kernel, look for the unresolved symbols in the ELF relocation tables (just like the linker does), and put those references into a table that is use to patch things up and you can use standard ELF relocation code to handle this, exactly like code we already have for module loading in the kernel already.
This idea is about 15 years old, sparc32 has been doing exactly this via something called "btfixup" to handle the page table, TLB, and cache differences of 15 different cpu+cache type combinations.
Quoted 5 lines
#define pv_patch(call, args...) \
asm volatile("8888:");
call(args);
asm volatile("8889:"
[ stuff to put 8889, 8888 and call in fixup section ]Please, use ELF and it's powerful and clean existing way to do this please. :-)
Quoted 5 lines
Quoted 1 line
What are the netdev requirements?
Reading Ben LaHaise's (very cool!) patch, it's not clear that using reloc postprocessing is going to be clearer than open-coding it as he has done.
Ben's case can be handled in the same way. Just do not define the symbols, pre-link, look for the references in the relocation tables, and run through that when you do the set_very_readonly() or install_paravirt_ops() thing.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 8 lines · David Miller
The idea is _NOT_ that you go look for references to the
paravirt_opsmembers structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...Instead you get rid of paravirt ops completely, and you call functions whose symbol name will not resolve in the initial kernel link.
Yeah, I came to that conclusion after thinking about it for a while. Thanks for the pointer to the sparc stuff; it looks very interesting.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to David MillerQuoted 4 lines
The idea is _NOT_ that you go look for references to the
paravirt_opsmembers structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...
That wouldn't handle inlines though. At least some of the current paravirtops like cli/sti are critical enough to require inlining.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · Andi Kleen
Quoted 5 lines
The idea is _NOT_ that you go look for references to the
paravirt_opsmembers structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...That wouldn't handle inlines though. At least some of the current paravirtops like cli/sti are critical enough to require inlining.
You could special-case it in the thing handling the relocs; if you're relocating to point to a function which you have an inline substitution, then inline.
The bigger problem is that you don't know what registers you can clobber. For the pv_ops in hand-written asm, that a big pain. The code either has to assume the worst, or the "relocator" has to do something more sophisticated (like look for push/pop pairs surrounding the call, perhaps?).
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 5 lines
The bigger problem is that you don't know what registers you can clobber. For the
pv_opsin hand-written asm, that a big pain. The code either has to assume the worst, or the "relocator" has to do something more sophisticated (like look for push/pop pairs surrounding the call, perhaps?).
You could use the dwarf2 unwind tables. They have exact information what register has what. But it would likely get complicated.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 2 lines · Andi Kleen
You could use the dwarf2 unwind tables. They have exact information what register has what. But it would likely get complicated.
Yes. And would they be accurate for hand-written asm, which is where we have this problem?
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 6 lines · Jeremy Fitzhardinge
Quoted 2 lines · Andi Kleen
You could use the dwarf2 unwind tables. They have exact information what register has what. But it would likely get complicated.
Yes. And would they be accurate for hand-written asm, which is where we have this problem?
Yes. All inline assembly tells gcc what registers are clobbered and it fills in the tables. Hand clobbering in inline assembly cannot be expressed with the current toolchain, so we moved all those out of line.
But again I'm not sure it will work anyways. For once you would need large padding around the calls anyways for inline replacement -- how would you generate that? I expect you would need to put the calls into asm() again and with that a custom annotiation format looks reasonable.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · Andi Kleen
Yes. All inline assembly tells gcc what registers are clobbered and it fills in the tables. Hand clobbering in inline assembly cannot be expressed with the current toolchain, so we moved all those out of line.
But again I'm not sure it will work anyways. For once you would need large padding around the calls anyways for inline replacement -- how would you generate that? I expect you would need to put the calls into
asm()again and with that a custom annotiation format looks reasonable.
Inlining is most important for very small code: sti, cli, pushf;pop eax, etc (in many cases, no-ops). We'd have at least 5 bytes to work in, and maybe more if there are surrounding push/pops to be consumed.
For example, say we wanted to put a general call for sti into entry.S, where its expected it won't touch any registers. In that case, we'd have a sequence like:
push %eax
push %ecx
push %edx
call paravirt_cli
pop %edx
pop %ecx
pop %eax
If we parse the relocs, then we'd find the reference to paravirt_cli. If we look at the byte before and see 0xe8, then we can see if its a call. If we then work out in each direction and see matched push/pops, then we know what registers can be trashed in the call. This also allows us to determine the callsite size, and therefore how much space we need for inlining.
So in this case, we see that there are 5 bytes for the call and a further 6 bytes of push/pops available for inlining.
Of course this is hand-written code anyway, so there's no particular burden to having some extra metadata stashed away in another section. For compiler-generated code, we know that it's already expecting standard C ABI calling conventions. The downside, of course, is that only the 5 byte call space is available for inline patching.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 20 lines · Jeremy Fitzhardinge
Quoted 10 lines · Andi Kleen
Yes. All inline assembly tells gcc what registers are clobbered and it fills in the tables. Hand clobbering in inline assembly cannot be expressed with the current toolchain, so we moved all those out of line.
But again I'm not sure it will work anyways. For once you would need large padding around the calls anyways for inline replacement -- how would you generate that? I expect you would need to put the calls into
asm()again and with that a custom annotiation format looks reasonable.Inlining is most important for very small code: sti, cli, pushf;pop eax, etc (in many cases, no-ops). We'd have at least 5 bytes to work in, and maybe more if there are surrounding push/pops to be consumed.
For example, say we wanted to put a general call for sti into
entry.S, where its expected it won't touch any registers. In that case, we'd have a sequence like:
Quoted 7 lines
push %eax
push %ecx
push %edx
callparavirt_cli
pop %edx
pop %ecx
pop %eax
This cannot right now be expressed as inline assembly in the unwinder at all because there is no way to inject the push/pops into the compiler generated ehframe tables.
[BTW I plan to resubmit the unwinder with some changes]
Quoted 8 lines
If we parse the relocs, then we'd find the reference to
paravirt_cli. If we look at the byte before and see0xe8, then we can see if its a call. If we then work out in each direction and see matched push/pops, then we know what registers can be trashed in the call. This also allows us to determine the callsite size, and therefore how much space we need for inlining.
gcc normally doesn't generate push/pops around directly around the call site, but somewhere else due to the way its register allocator works. It can be anywhere in the function or even not there at all if the register didn't contain anything useful. And they're not necessarily push/pops of course.
So you would need to write it as inline assembly. I'm not sure it would be significantly cleaner than just having tables then.
Quoted 8 lines
So in this case, we see that there are 5 bytes for the call and a further 6 bytes of push/pops available for inlining.
Of course this is hand-written code anyway, so there's no particular burden to having some extra metadata stashed away in another section. For compiler-generated code, we know that it's already expecting standard C ABI calling conventions. The downside, of course, is that only the 5 byte call space is available for inline patching.
It's unlikely you can do much useful in 5 bytes I guess.
Regarding cli/sti: i've been actually thinking about changing it in the non paravirt kernel. IIRC most save_flags/restore_flags are inside spin_lock_irqsave/restore() and that is a separate function anyways so a little larger special case code is ok as long as it is not slower. There is some evidence that at least on P4 a software cli/sti flag without pushf/popf would be faster.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 10 lines · Andi Kleen
Quoted 9 lines
push %eax
push %ecx
push %edx
callparavirt_cli
pop %edx
pop %ecx
pop %eax
Quoted 4 lines
This cannot right now be expressed as inline assembly in the unwinder at all because there is no way to inject the push/pops into the compiler generated ehframe tables.
I was thinking more of entry.S rather than inline asms. The only ones I can think of which would be relevant are the interrupt operations in spinlocks, and even then we get a bit more leeway.
Quoted 6 lines
gcc normally doesn't generate push/pops around directly around the call site, but somewhere else due to the way its register allocator works. It can be anywhere in the function or even not there at all if the register didn't contain anything useful. And they're not necessarily push/pops of course.
Right, I'm not making any assumptions about what gcc generates for calls, other than assuming it uses a standard "call X" direct call instruction.
Quoted 3 lines
So you would need to write it as inline assembly. I'm not sure it would be significantly cleaner than just having tables then.
No, my intention is that the vast majority of pv_ops uses, which are calls from C code, would simply be normal C calls, syntatically and semantically.
Quoted 2 lines
It's unlikely you can do much useful in 5 bytes I guess.
I think the main value is retaining non-PARAVIRT native performance levels. In the native case, the inlined instructions amount to 1 or 2 bytes, and having small patch sites is an advantage because there's less space wasted on nops. My understanding is that a direct call has almost zero overhead on modern processors, because both the call and the return can be completely predicted and prefetched, so the threshhold at which you make inline vs call tradeoff is pretty small.
Quoted 7 lines
Regarding cli/sti: i've been actually thinking about changing it in the non paravirt kernel. IIRC most save_flags/restore_flags are inside spin_lock_irqsave/restore() and that is a separate function anyways so a little larger special case code is ok as long as it is not slower. There is some evidence that at least on P4 a software cli/sti flag without pushf/popf would be faster.
There are also the ones in entry.S. I suppose spinlocks do get used more than syscalls and interrupts.
J
David Miller
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Date: Mon, 19 Mar 2007 11:57:28 +0100
Quoted 14 lines · Jeremy Fitzhardinge
Quoted 5 lines
Andi Kleen wrote:
For example, say we wanted to put a general call for sti intoentry.S, where its expected it won't touch any registers. In that case, we'd have a sequence like:Quoted 7 lines
push %eax
push %ecx
push %edx
callparavirt_cli
pop %edx
pop %ecx
pop %eax
Quoted 5 lines
This cannot right now be expressed as inline assembly in the unwinder at all because there is no way to inject the push/pops into the compiler generated ehframe tables.
[BTW I plan to resubmit the unwinder with some changes]
It's inability to handle sequences like the above sounds to me like a very good argument to _not_ merge the unwinder back into the tree.
To me, that unwinder is nothing but trouble, it severly limits what cases you can use special calling conventions via inline asm (and we have done that on several occaisions) and even ignoring that the unwinder only works half the time.
Please don't subject us to another couple months of hair-pulling only to have Linus yank the thing out again, there are certainly more useful things to spend time on :-)
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 3 lines
It's inability to handle sequences like the above sounds to me like a very good argument to _not_ merge the unwinder back into the tree.
The unwinder can handle it fine, it is just that gcc right now cannot be taught to generate correct unwind tables for it. If paravirt ops is widely used i guess it would be possible to find some workaround for that though.
There is one case it truly cannot handle (it would be possible by switching to dwarf3), but that was relatively easily eliminated and wasn't a problem imho.
Quoted 4 lines
To me, that unwinder is nothing but trouble, it severly limits what cases you can use special calling conventions via inline asm (and we have done that on several occaisions) and even ignoring that the unwinder only works half the time.
Initially we had some bugs that accounted for near all failures, but they were all fixed in the latest version.
Quoted 3 lines
Please don't subject us to another couple months of hair-pulling only to have Linus yank the thing out again, there are certainly more useful things to spend time on :-)
I think better backtraces are a good use of my time. Sorry if you disagree on that.
-Andi
PATCH 1/1 x86_64 : Suppress __jiffies
Hi Andi
I was considering re-submiting patch to group together xtime_lock/xtime/jiffies in the same cache line. Before doing so, I believe the following cleanup patch could prepare the x86_64 port to be similar with other arches.
Thank you
[PATCH] x86_64 : Suppress __jiffies
x86_64 arch currently specially defines __jiffies, as a relic of old vsyscall implementation. It is currently used to validate vgetcpu() user cache as a token. If we use another token, more opaque like the seqlock sequence, we can get rid of special __jiffies handling. As a result, jiffies_64 is no more included in vsyscall page. This patch prepares the introduction of time_data.
arch/x86_64/kernel/vsyscall.c +9 -1
@@ -62,6 +62,14 @@ struct vsyscall_gtod_data_t __vsyscall_g .sysctl_enabled = 1, }; +/* +* vgetcpu_token is an opaque token used to validate vgetcpu() user cache. +* We need something that change like jiffies. It used to be jiffies +* but using seqlock __vsyscall_gtod_data.lock sequence is better because +* we prefer not let user read jiffies. +*/ +#define vgetcpu_token () read_seqbegin(&__vsyscall_gtod_data.lock) +void update_vsyscall (struct timespec *wall_time,struct clocksource *clock) {unsigned long flags;
@@ -180,7 +188,7 @@ vgetcpu(unsigned *cpu, unsigned *node, sWe do this here because otherwise user space would do it on its own in a likely inferior way (no access to jiffies). If you don't like it pass NULL. */ -if (tcache && tcache->blob[0 ] == (j = __jiffies)) { +if (tcache && tcache->blob[0 ] == (j =vgetcpu_token ())) { p = tcache->blob[1 ]; }else if (__vgetcpu_mode ==VGETCPU_RDTSCP ) {/* Load per CPU data from RDTSCP */
arch/x86_64/kernel/time.c +0 -1
@@ -53,7 +53,6 @@ DEFINE_SPINLOCK(rtc_lock);EXPORT_SYMBOL (rtc_lock);DEFINE_SPINLOCK (i8253_lock); -volatile unsigned long __jiffies __section_jiffies =INITIAL_JIFFIES ;unsigned long profile_pc (struct pt_regs *regs) {
arch/x86_64/kernel/vmlinux.lds.S +1 -5
@@ -12,7 +12,7 @@ #undef i386 /* in case the preprocessor OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64", "elf64-x86-64") OUTPUT_ARCH(i386:x86-64) ENTRY(phys_startup_64) -jiffies_64 = jiffies; +jiffies = jiffies_64; _proxy_pda = 0; PHDRS { text PT_LOAD FLAGS(5); /* R_E */
@@ -97,10 +97,6 @@ #define VVIRT(x) (ADDR(x) - VVIRT_OFFSET .vgetcpu_mode : AT(VLOAD(.vgetcpu_mode)) { *(.vgetcpu_mode) } vgetcpu_mode = VVIRT(.vgetcpu_mode); - . = ALIGN(CONFIG_X86_L1_CACHE_BYTES); - .jiffies : AT(VLOAD(.jiffies)) { *(.jiffies) } - jiffies = VVIRT(.jiffies); - .vsyscall_1 ADDR(.vsyscall_0) + 1024: AT(VLOAD(.vsyscall_1)) { *(.vsyscall_1) } .vsyscall_2 ADDR(.vsyscall_0) + 2048: AT(VLOAD(.vsyscall_2))
include/asm-x86_64/vsyscall.h +0 -2
@@ -17,7 +17,6 @@ #ifdef __KERNEL__#include <linux/seqlock.h> #define __section_vgetcpu_mode __attribute__ ((unused, __section__ (".vgetcpu_mode"), aligned(16))) -#define __section_jiffies __attribute__ ((unused, __section__ (".jiffies"), aligned(16)))/* Definitions for CONFIG_GENERIC_TIME definitions */ #define __section_vsyscall_gtod_data __attribute__ \
@@ -31,7 +30,6 @@ #define hpet_readl(a) readl((c#define hpet_writel (d,a) writel(d, (void __iomem *)fix_to_virt(FIX_HPET_BASE) + a)extern int __vgetcpu_mode; -extern volatile unsigned long __jiffies;/* kernel space (writeable) */ extern int vgetcpu_mode;
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Andi KleenQuoted 3 lines · Andi Kleen
Initially we had some bugs that accounted for near all failures, but they were all fixed in the latest version.
No. The real bugs were that the people involved wouldn't even accept that unwinding information was inevitably buggy and/or incomplete.
That much more fundamental bug never got fixed, as far as I know.
I'm not going to merge anything that depends on unwind tables as things stand. The pain just isn't worth it.
Quoted 3 lines
Quoted 3 lines
Please don't subject us to another couple months of hair-pulling only to have Linus yank the thing out again, there are certainly more useful things to spend time on :-)
Good call. Dwarf2 unwinding simply isn't worth doing. But I won't yank it out, I simply won't merge it. It was more than just totally buggy code, it was an inability of the people to understand that even bugfree code isn't enough - you have to be able to also handle buggy data.
Linus
David Miller
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
From: Linus Torvalds <[email protected]>
Date: Mon, 19 Mar 2007 20:18:14 -0700 (PDT)
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Please don't subject us to another couple months of hair-pulling only to have Linus yank the thing out again, there are certainly more useful things to spend time on :-)
Good call. Dwarf2 unwinding simply isn't worth doing. But I won't yank it out, I simply won't merge it. It was more than just totally buggy code, it was an inability of the people to understand that even bugfree code isn't enough - you have to be able to also handle buggy data.
Thank you.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 7 lines · David Miller
From: Linus Torvalds <[email protected]>
Date: Mon, 19 Mar 2007 20:18:14 -0700 (PDT)Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Please don't subject us to another couple months of hair-pulling only to have Linus yank the thing out again, there are certainly more useful things to spend time on :-)
Quoted 6 lines
Quoted 4 lines
Good call. Dwarf2 unwinding simply isn't worth doing. But I won't yank it out, I simply won't merge it. It was more than just totally buggy code, it was an inability of the people to understand that even bugfree code isn't enough - you have to be able to also handle buggy data.
Thank you.
Hmm..
I know the feeling I have had a similar rant about the kexec on panic
code path. The code is still no where near as paranoid about normal
kernel things not working as it could be, but by ranting about it periodically the people doing the work are gradually making it better.
I'm conflicted about the dwarf unwinder. I was off doing other things at the time so I missed the pain, but I do have a distinct recollection of the back traces on x86_64 being distinctly worse the on i386. Lately I haven't seen that so it may be I was misinterpreting what I was seeing, and the compiler optimizations were what gave me such weird back traces.
But if the quality of our backtraces has gone down and dwarf unwinder could give us better back traces it is likely worth pursuing. Of course it would need to start with the assumption that it's tables may be borked (the kernel is busted after all) and be much more careful than Andi's last attempt.
Eric
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
\
Quoted 3 lines
I'm conflicted about the dwarf unwinder. I was off doing other things at the time so I missed the pain, but I do have a distinct recollection of the back traces on
x86_64being distinctly worse the on i386.
The only case were i386 was better was with frame pointers, which was never fully implemented for x86-64. However i find that hilarious:
people are spending a lot of time right here in this thread to squeeze out the best call sequences for the paravirt ops, but then accept losing a full frame pointer register on i386. I never found that acceptable, that is why I prefered the unwinder instead.
This said the big problem with the frame pointers is mostly gone now:
on older CPUs it tended to cause a pipeline stall early in the function. That is now fixed in the latest Intel/upcomming AMD CPUs, but there are still millions and millions of older CPUs out there so I still don't consider it acceptable.
Quoted 4 lines
Lately
I haven't seen that so it may be I was misinterpreting what I was seeing, and the compiler optimizations were what gave me such weird back traces.
The main problem is that subsystems are getting more and more complex and especially callbacks seem to multiply far too quickly.
In 2.4 it was often very reasonable to just sort out the false positives, but with sometimes 20-30+ level deep call chains in 2.6 with many callbacks that just gets far too tenuous.
Quoted 5 lines
But if the quality of our backtraces has gone down and dwarf unwinder could give us better back traces it is likely worth pursuing. Of course it would need to start with the assumption that it's tables may be borked (the kernel is busted after all) and be much more careful than Andi's last attempt.
The latest version validates the stack always. It was only a few lines of change. I doubt it will make much difference though. The few true crashes we had were not actually due the unwinder itself, but the buggy fallback code (which were fixed quickly). But anyways it should satisfy everybody's paranoia now.
Although in future it would be good if people did some more analysis in root causes for failures before let the paranoia take over and revert patches.
We see a good example here of what I call the JFS/ACPI effect: code gets merged too early with some visible problems. It gets a bad name and afterwards people never look objectively at it again and just trust their prejudices.
But that's not a good strategy to get good code in the end I think. If there is enough evidence the early problems were fixed then prejudices should be reevaluated.
I will let it cook some time in -mm* and we will see if it works now or not. I'm pretty confident it will though. And if it does there is no reason not to resubmit it.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 11 lines · Andi Kleen
Quoted 3 lines
I'm conflicted about the dwarf unwinder. I was off doing other things at the time so I missed the pain, but I do have a distinct recollection of the back traces on
x86_64being distinctly worse the on i386.The only case were i386 was better was with frame pointers, which was never fully implemented for x86-64. However i find that hilarious:
people are spending a lot of time right here in this thread to squeeze out the best call sequences for the paravirt ops, but then accept losing a full frame pointer register on i386. I never found that acceptable, that is why I prefered the unwinder instead.
Quoted 5 lines
This said the big problem with the frame pointers is mostly gone now:
on older CPUs it tended to cause a pipeline stall early in the function. That is now fixed in the latest Intel/upcomming AMD CPUs, but there are still millions and millions of older CPUs out there so I still don't consider it acceptable.
What I recall observing is call traces that made no sense. Not just extra noise in the stack trace but things like seeing a function that has exactly one path to it, and not seeing all of the functions on that path in the call trace.
In my later debugging I have been reasonably able to attribute those kinds of things to compiler optimizations like inlining and tail call optimization.
Now I will agree that having fewer or no false positives to weed through is a good thing, if we can do it reliably.
Quoted 5 lines
Quoted 4 lines
Lately
I haven't seen that so it may be I was misinterpreting what I was seeing, and the compiler optimizations were what gave me such weird back traces.
Quoted 7 lines
The main problem is that subsystems are getting more and more complex and especially callbacks seem to multiply far too quickly.
In 2.4 it was often very reasonable to just sort out the false positives, but with sometimes 20-30+ level deep call chains in 2.6 with many callbacks that just
gets far too tenuous.
Hmm. I haven't seen those traces, but I wonder if the size of those stack traces indicates potential stack overflow problems.
Quoted 11 lines
Quoted 5 lines
But if the quality of our backtraces has gone down and dwarf unwinder could give us better back traces it is likely worth pursuing. Of course it would need to start with the assumption that it's tables may be borked (the kernel is busted after all) and be much more careful than Andi's last attempt.
The latest version validates the stack always. It was only a few lines of change. I doubt it will make much difference though. The few true crashes we had were not actually due the unwinder itself, but the buggy fallback code (which were fixed quickly). But anyways it should satisfy everybody's paranoia now.
Do you also validate the unwind data?
Quoted 6 lines
Although in future it would be good if people did some more analysis in root causes for failures before let the paranoia take over and revert patches.
We see a good example here of what I call the JFS/ACPI effect: code gets merged too early with some visible problems. It gets a bad name and afterwards people never look objectively at it again and just trust their prejudices.
I don't know. The impression I got was the root cause analysis stopped when it was observed that the code was unsuitable for solving the problem. When asked about it, it appeared the developer did not understand the question. Therefore the root cause was assumed to be the developer.
At least that is how I have read the few little bits I have seen.
Quoted 3 lines
But that's not a good strategy to get good code in the end I think. If there is enough evidence the early problems were fixed then prejudices should be reevaluated.
Certainly. However if the developer has lost a certain amount of initial trust, the burden becomes much higher.
Eric
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 4 lines · Eric W. Biederman
What I recall observing is call traces that made no sense. Not just extra noise in the stack trace but things like seeing a function that has exactly one path to it, and not seeing all of the functions on that path in the call trace.
That's tail call/sibling call optimization. No unwinder can untangle that because the return address is lost. But it's also an quite important optimization.
Quoted 8 lines
Quoted 5 lines
In 2.4 it was often very reasonable to just sort out the false positives, but with sometimes 20-30+ level deep call chains in 2.6 with many callbacks that just
gets far too tenuous.Hmm. I haven't seen those traces, but I wonder if the size of those stack traces indicates potential stack overflow problems.
Most functions have quite small frames, so 20-30 is still not a problem
Quoted 1 line
Do you also validate the unwind data?
There are many sanity checks in the unwind code and it will fall back to the old unwinder when it gets stuck.
Quoted 10 lines
Quoted 7 lines
Although in future it would be good if people did some more analysis in root causes for failures before let the paranoia take over and revert patches.
We see a good example here of what I call the JFS/ACPI effect: code gets merged too early with some visible problems. It gets a bad name and afterwards people never look objectively at it again and just trust their prejudices.
I don't know. The impression I got was the root cause analysis stopped when it was observed that the code was unsuitable for solving the problem.
No, me and Jan fixed all reported bugs as far as I know.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 2 lines · Andi Kleen
No, me and Jan fixed all reported bugs as far as I know.
No you did not. You didn't fix the ones I reported. Which is why it got removed, and will not get added back until there is another maintainer.
The ones I reported were all about trusting the stack contents implicitly, and assuming that the unwind info was there and valid. Using things like "__get_user()" didn't fix it, because if a WARN_ON() happened while we held the mm semaphore and the unwind info was bogus, it would take a page-fault and deadlock.
Those kinds of things are not acceptable for debugging output. If I cannot use WARN_ON() because I hold the MM lock and I'm afraid there might be kernel corruption, then something is *wrong*!
And I told you guys this. Over *months*. And you ignored me. You told me everything was fine. Each time, somebody else ended up reporting a hang where the unwinder was at fault. And since I couldn't trust the maintainers to fix it, removing the broken feature that only caused more problems than it fixed was the only option.
And you clearly *still* haven't accepted the fact that the code was buggy.
Does anybody wonder why I wouldn't merge it back?
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 1 line · Linus Torvalds
The ones I reported were all about trusting the stack contents implicitly,
The latest version added the full double check you wanted - every new RSP is validated against the current stack or the exception stacks.
Quoted 1 line
and assuming that the unwind info was there and valid. Using things like
The code never did that. In fact many of the problems we had initially especially came out of that -- the fallback code that would handle this case wasn't fully correct.
Quoted 3 lines
"
__get_user()" didn't fix it, because if aWARN_ON()happened while we held the mm semaphore and the unwind info was bogus, it would take a page-fault and deadlock.
That was fixed too even before you dropped it by using probe_kernel_address()
Quoted 3 lines
And I told you guys this. Over *months*. And you ignored me. You told me everything was fine. Each time, somebody else ended up reporting a hang where the unwinder was at fault.
That was because most of the bugs were reported many times duplicated -- and reports from older releases kept streaming in even after the fix went in.
Also frankly often your analysis about what went wrong was just incorrect.
Quoted 1 line
And you clearly *still* haven't accepted the fact that the code was buggy.
There were some bugs, but they were all fixed. Or at least I hope -- we will find out during testing.
Quoted 1 line
Does anybody wonder why I wouldn't merge it back?
So do you have an alternative to the unwinder? Don't tell me "real men decode 30+ entry long stack traces with lots of callbacks by hand". Or do you prefer to use frame pointers everywhere?
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 4 lines · Andi Kleen
The code never did that. In fact many of the problems we had initially especially came out of that -- the fallback code that would handle this case wasn't fully correct.
I don't keep my emails any more, but you *never* fixed the problems in arch/*/kernel/traps.c.
Yes, the kernel/unwind.c issues generally got fixed. The infinite loops in the *callers* never did.
Quoted 2 lines
Also frankly often your analysis about what went wrong was just incorrect.
Still in denial, I see.
Do you still claim that "the fallback position always did the right thing"? Despite the fact that the unwinder had sometimes *corrupted* the incoming information so much that the fallback position was the one that oopsed? And no, you didn't fix that.
And no, IT DID NOT use probe_kernel_address like you still claim.
Anyway, you work for Suse, I don't care what you do to the Suse kernel. Maybe it will get stable some day. Somehow, I doubt it.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 10 lines · Linus Torvalds
Quoted 4 lines · Andi Kleen
The code never did that. In fact many of the problems we had initially especially came out of that -- the fallback code that would handle this case wasn't fully correct.
I don't keep my emails any more, but you *never* fixed the problems in
arch/*/kernel/traps.c.
I fixed that one after you dropped it (hmm, double checking: or at least I thought I had fixed it, but don't see the code right now; will redo then) Basically it was just a one liner anyways - always check against all the stacks that are there.
Quoted 2 lines
Yes, the
kernel/unwind.cissues generally got fixed. The infinite loops in the *callers* never did.
There was later a weaker form that should have caught most loops, but admittedly it wasn't 100% bullet-proof with exception stacks.
Quoted 8 lines
Quoted 3 lines
Also frankly often your analysis about what went wrong was just incorrect.
Still in denial, I see.
Do you still claim that "the fallback position always did the right thing"
No initially it was buggy and that caused several of the crashes.
Quoted 3 lines
Despite the fact that the unwinder had sometimes *corrupted* the incoming information so much that the fallback position was the one that oopsed? And no, you didn't fix that.
No, it oopsed because it was broken by itself.
Anyways that got fixed quickly.
Quoted 2 lines
And no, IT DID NOT use
probe_kernel_addresslike you still claim.
There definitely was a patch that made it use it. You might have not merged it though.
Quoted 2 lines
Anyway, you work for Suse, I don't care what you do to the Suse kernel. Maybe it will get stable some day. Somehow, I doubt it.
So what is your proposed alternative to handle long backtraces? You didn't answer that question. Please do, I'm curious about your thoughts in this area.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 4 lines · Andi Kleen
So what is your proposed alternative to handle long backtraces? You didn't answer that question. Please do, I'm curious about your thoughts in this area.
the thing is, I'd rather see a long backtrace that is hard to decipher but that *never* *ever* causes any additional problems, over a pretty one.
Because that's really the issue: do you want a "pretty" backtrace, or do you want one that is rock solid but has some crud in it.
I'll take the rock solid one any day. Especially as even the pretty one won't fix the most common problem, which is "I don't see the caller" (due to inlining or tail-calls).
In contrast, the ugly backtrace will have some "garbage entries" from previous frames that didn't get overwritten, but there have actually been (admittedly rare) cases where those garbage entries have given hints about what happened just before.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 10 lines · Linus Torvalds
Quoted 4 lines · Andi Kleen
So what is your proposed alternative to handle long backtraces? You didn't answer that question. Please do, I'm curious about your thoughts in this area.
the thing is, I'd rather see a long backtrace that is hard to decipher but that *never* *ever* causes any additional problems, over a pretty one.
Well it causes additional problems. We had some cases where it was really hard to distingush garbage and the true call chain. I can probably dig out some examples if you want.
With lots of call backs (e.g. common with sysfs) it is also frequently not obvious how the call chains are supposed to go.
I would have agreed with you in the 2.4 time frame. It was also always fun to explain this to a newbie with oopses and watch their face when they secretly think you're crazy ;-) But code is getting much and more complex unfortunately and I had a few cases where it was really ugly to sort them out.
Quoted 2 lines
Because that's really the issue: do you want a "pretty" backtrace, or do you want one that is rock solid but has some crud in it.
I just want an as exact backtrace as possible. I also think that we can make the unwinder robust enough.
Quoted 3 lines
I'll take the rock solid one any day. Especially as even the pretty one won't fix the most common problem, which is "I don't see the caller" (due to inlining or tail-calls).
The problem is not one or a few levels of calls. That can be always figured out from the source. The problem is when you have a double
digit number of them with non obvious (dynamic indirect) dependencies. Yes it can be figured out too, but it is a long and very annoying process with lots of grepping etc. and even then you sometimes can't be 100% sure in some cases.
It's also a mechanic process that just cries to be done by a machine instead.
Quoted 2 lines
In contrast, the ugly backtrace will have some "garbage entries" from previous frames that didn't get overwritten, but there have actually been
If it was only a few that would be great ...
-Andi
Re: [Xen-devel] Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 5 lines
Quoted 2 lines
Because that's really the issue: do you want a "pretty" backtrace, or do you want one that is rock solid but has some crud in it.
I just want an as exact backtrace as possible. I also think that we can make the unwinder robust enough.
Any reason you can't put the exact back trace in "[xxx]" and the ones we see on the stack which dont look like call trace as ?xxx? It makes the code a bit trickier but we depend on the quality of traces
Alan
Re: [Xen-devel] Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · Alan Cox
Quoted 5 lines
Quoted 2 lines
Because that's really the issue: do you want a "pretty" backtrace, or do you want one that is rock solid but has some crud in it.
I just want an as exact backtrace as possible. I also think that we can make the unwinder robust enough.
Any reason you can't put the exact back trace in "[xxx]" and the ones we see on the stack which dont look like call trace as ?xxx? It makes the code a bit trickier but we depend on the quality of traces
Linus is worried about the unwinder crashing -- that wouldn't help with that.
What the (now out of tree) unwinder does is to check if it finishes the trace and if not fall back to the old unwinder.
-Andi
Re: [Xen-devel] Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 2 lines · Andi Kleen
Linus is worried about the unwinder crashing -- that wouldn't help with that.
And to make it clear: this is not a theoretical worry. It happened many times over the months the unwinder was in.
It was supposed to help debugging, but it made bugs that *would* have been nicely debuggable without it into nightmares. So the only reason for it existing in the first place was actually the thing that made it not work.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Andi KleenQuoted 8 lines · Andi Kleen
Quoted 3 lines · Linus Torvalds
the thing is, I'd rather see a long backtrace that is hard to decipher but that *never* *ever* causes any additional problems, over a pretty one.
Well it causes additional problems. We had some cases where it was really hard to distingush garbage and the true call chain. I can probably dig out some examples if you want.
Well, by "additional problems" _I_ mean things like "a warning turned into a fatal oops and didn't get logged at all".
That's a lot more serious than "there were a few extra entries in the traceback that caused us some confusion".
And yes, we had exactly that case happen several times.
Quoted 2 lines
With lots of call backs (e.g. common with sysfs) it is also frequently not obvious how the call chains are supposed to go.
With callbacks, it's actually often nice to see the callback data that is on the stack (and it's very obvious from the "<function+0>" ksymtab
explanation: you can't have a <function+0> that is anything but a callback pointer (since it isn't a return address).
Linus
Andrew Morton
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Andi KleenQuoted 2 lines · On Tue, 20 Mar 2007 21:23:52 +0100 Andi Kleen <[email protected]> wrote:
Well it causes additional problems. We had some cases where it was really hard to distingush garbage and the true call chain.
yes, for some reason the naive backtraces seem to have got messier and messier over the years and some of them are really quite hard to piece together nowadays.
An accurate backtrace would have some value, if we can get it bullet-proof.
The fault-injection driver wants it too. And lockdep, I guess.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Eric W. BiedermanQuoted 7 lines · Eric W. Biederman
I'm conflicted about the dwarf unwinder. I was off doing other things at the time so I missed the pain, but I do have a distinct recollection of the back traces on
x86_64being distinctly worse the on i386. Lately I haven't seen that so it may be I was misinterpreting what I was seeing, and the compiler optimizations were what gave me such weird back traces.
Well, if you compile x86_64 with frame pointers it helps a bit because the compiler doesn't tail merge function calls. But the stack backtrace ignores the frame pointers even if they're present, unlike i386 which will use them.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Jeremy FitzhardingeQuoted 13 lines · Jeremy Fitzhardinge
For example, say we wanted to put a general call for sti into
entry.S, where its expected it won't touch any registers. In that case, we'd have a sequence like:push %eax
push %ecx
push %edx
callparavirt_cli
pop %edx
pop %ecx
pop %eax
Quoted 6 lines
If we parse the relocs, then we'd find the reference to
paravirt_cli. If we look at the byte before and see0xe8, then we can see if its a call. If we then work out in each direction and see matched push/pops,
then we know what registers can be trashed in the call. This also
allows us to determine the callsite size, and therefore how much space
we need for inlining.
No, that is a very dangerous suggestion. You absolutely *cannot* do this safely without explicitly marking the start EIP of this code. You *must* use metadata to do that. It is never safe to disassemble backwards or "rewind" EIP for x86 code.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 11 lines · Zachary Amsden
Quoted 5 lines · Jeremy Fitzhardinge
If we then work out in each direction and see matched push/pops,
then we know what registers can be trashed in the call. This also allows us to determine the callsite size, and therefore how much space we need for inlining.No, that is a very dangerous suggestion. You absolutely *cannot* do this safely without explicitly marking the start EIP of this code. You *must* use metadata to do that. It is never safe to disassemble backwards or "rewind" EIP for x86 code.
What do you mean the instruction before is "mov $0x52515000,%eax"?
Yeah, you're right. Oh well.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Andi KleenQuoted 7 lines · Andi Kleen
Quoted 4 lines
The idea is _NOT_ that you go look for references to the
paravirt_opsmembers structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...That wouldn't handle inlines though. At least some of the current paravirtops like cli/sti are critical enough to require inlining.
Well, we'd patch the inline over the call if we have room.
Magic patching would be neat, but the downsides are that (1) we can't expand the patching room and (2) there's no way of attaching clobber info to the call site (doing register liveness analysis is not appealing).
Now, this may not be fatal. 5 bytes is enough for all the native ops to
be patched inline. For lguest this covers popf and pushf, but not cli
and sti (10 bytes): they'd have to be calls.
As for clobber info, it turns out that almost all of the calls can clobber %eax, which is probably enough. We just need to mark the handful of asm ones where this isn't true.
Thoughts?
Rusty.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · Rusty Russell
Quoted 7 lines · Andi Kleen
Quoted 4 lines
The idea is _NOT_ that you go look for references to the
paravirt_opsmembers structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...That wouldn't handle inlines though. At least some of the current paravirtops like cli/sti are critical enough to require inlining.
Quoted 6 lines
Well, we'd patch the inline over the call if we have room.
Magic patching would be neat, but the downsides are that (1) we can't expand the patching room and (2) there's no way of attaching clobber info to the call site (doing register liveness analysis is not appealing).
True. You can use all of the call clobbered registers.
Quoted 7 lines
Now, this may not be fatal. 5 bytes is enough for all the native ops to
be patched inline. For lguest this covers popf and pushf, but not cli
and sti (10 bytes): they'd have to be calls.As for clobber info, it turns out that almost all of the calls can clobber %eax, which is probably enough. We just need to mark the handful of asm ones where this isn't true.
I guess if the code is larger than a function call I'm failing to see the disadvantage in making it a direct function call. Any modern processor ought to be able to predict it perfectly, and processors like the P4 may even optimize the call out of their L1 instruction cache.
If what David is suggesting works, making all of these direct calls
looks easy and very maintainable. At which point patching
instructions inline is quite possibly overkill.
Is it truly critical to inline any of these instructions?
Eric
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 2 lines · Eric W. Biederman
True. You can use all of the call clobbered registers.
Quite often, the biggest single win of inlining is not so much the code size (although if done right, that will be smaller too), but the fact that inlining DOES NOT CLOBBER AS MANY REGISTERS!
The lack of register clobbering, and the freedom for the compiler to choose registers around an inlined function is usually the biggest win! If you can't do that, then inlining generally doesn't actually even help: a call/return to a single instruction isn't all that much slower than just doing the "cli" in the first place.
If we end up with a setup where any inlined instruction needs to act as if it was a function call (just with the "call" instruction papered over with the inlined instruction sequence), then there is no point to this at all.
In short: people here seem to think that inlining is about avoiding the call/ret sequence. Not so. The real advantages of inlining are elsewhere.
So *please* don't believe that you can make it "as cheap" to have some automatic fixup of two sequences, one inlined and one as a "call". It may look so when you look at the single instruction generated, but you're ignoring all the instructions *around* the site.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 5 lines · Linus Torvalds
So *please* don't believe that you can make it "as cheap" to have some automatic fixup of two sequences, one inlined and one as a "call". It may look so when you look at the single instruction generated, but you're ignoring all the instructions *around* the site.
Side note, you can certainly fix things like this at least in theory, but it requires:
- the "call" instruction that is used instead of the inlining should basically have no callee-clobbers. Any paravirt_ops called this way should have a special calling sequence where they simple save and restore all the registers they use.
This is usually not that bad. Just create a per-architecture wrapper function that saves/restores anything that the C calling convention on that architecture says is clobbered by calls.
- if the function has arguments, and the inlined sequence can take the arguments in arbitrary registers, you are going to penalize the inlined sequence anyway (by forcing some fixed arbitrary register allocation policy).This thing is largely unfixable without some really extreme compiler games (like post-processing the assembler output and having different entry-points depending on where the arguments are..)
.. it will obviously depend on how thngs are done whether these things are useful or not. But it does mean that it's always a good idea to just have a config option of "turn off all the paravirt crap, because it *does* add overhead, and replacing instructions on the fly doesn't make that overhead go away".
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Linus TorvaldsQuoted 9 lines · Linus Torvalds
Quoted 2 lines · Eric W. Biederman
True. You can use all of the call clobbered registers.
Quite often, the biggest single win of inlining is not so much the code size (although if done right, that will be smaller too), but the fact that inlining DOES NOT CLOBBER AS MANY REGISTERS!
Yes, that's something that we've been conscious of when designing the pv_ops patching mechanism. As it stands, each time you emit a call to a pv-ops operation, it not only stores the start and end of the patchable area, but also which registers are available for clobbering at that callsite by the patched-in code. For things like sti/cli, where the surrounding code expects nothing to be clobbered, we make sure that the regs are preserved on the pv-ops side, even though there's a call to a normal C function in the middle. That gives in the pvops backend the flexibility to patch over that site with either some inline code or a call out to some function which doesn't necessarily clobber the full caller-save set (or even any registers).
Quoted 3 lines
In short: people here seem to think that inlining is about avoiding the call/ret sequence. Not so. The real advantages of inlining are elsewhere.
Yes. Probably the biggest win of inlining is constant propagation into the inline function, but register-use flexibility is good small-scale win (vs the micro-scale call/ret elimination).
Quoted 12 lines
Side note, you can certainly fix things like this at least in theory, but it requires:
- the "call" instruction that is used instead of the inlining should basically have no callee-clobbers. Any
paravirt_opscalled this way should have a special calling sequence where they simple save and restore all the registers they use.This is usually not that bad. Just create a per-architecture wrapper function that saves/restores anything that the C calling convention on that architecture says is clobbered by calls.
Most of the places we intercept are normal C calls anyway, so this isn't a big issue. Its mainly the places where we replace single instructions that this will have a big effect.
The trouble with this is that we're back to having to have special wrappers around each call to hide the normal C calling convention from the compiler, so that it knows that it has more registers to play with. This was the main complaint about the original version of the patch, because it all looks very ugly.
Quoted 7 lines
- if the function has arguments, and the inlined sequence can take the arguments in arbitrary registers, you are going to penalize the inlined sequence anyway (by forcing some fixed arbitrary register allocation policy).This thing is largely unfixable without some really extreme compiler games (like post-processing the assembler output and having different entry-points depending on where the arguments are..)
Yeah, that doesn't sound like much fun. I think using the normal regparm calling convention will be OK. Aside from some slightly longer instruction encodings, all the registers are more or less interchangeable anyway.
Quoted 5 lines
.. it will obviously depend on how thngs are done whether these things are useful or not. But it does mean that it's always a good idea to just have a config option of "turn off all the paravirt crap, because it *does* add overhead, and replacing instructions on the fly doesn't make that overhead go away".
Yes, there's a big switch to turn all this off. It would be nice if we could get things to the point that it isn't necessary to have (ie, running on bare hardware really is indistinguishable either way), but we're a fair way from being able to prove that. In the meantime, the goal is to try to keep the source-level changes a local as possible so that maintaining the CONFIG_PARAVIRT vs non-PARAVIRT distinction is straightforward.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Linus TorvaldsQuoted 8 lines · Linus Torvalds
Quoted 2 lines · Eric W. Biederman
True. You can use all of the call clobbered registers.
Quite often, the biggest single win of inlining is not so much the code size (although if done right, that will be smaller too), but the fact that inlining DOES NOT CLOBBER AS MANY REGISTERS!
Thanks Linus.
*This* was the reason that the current hand-coded calls only clobber % eax. It was a compromise between native (no clobbers) and others (might need a reg).
Now, since we decided to allow paravirt_ops operations to be normal C (ie. the patching is optional and done late), we actually push and pop % ecx and %edx. This makes the call site 10 bytes long, which is a nice size for patching anyway (enough for a movl $0, <addr>, a-la lguest's cli, or movw $0, %gs:<addr> if we supported SMP).
The current 6 paravirt ops which are patched cover the vast majority of calls (until the Xen patches, then we need ~4 more?). Jeremy chose to expand patching to cover *all* paravirt ops, rather than just the new hot ones, and that's where we tipped over the ugliness threshold.
Cheers,
Rusty.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 8 lines · Rusty Russell
Quoted 6 lines · Linus Torvalds
Quoted 2 lines · Eric W. Biederman
True. You can use all of the call clobbered registers.
Quite often, the biggest single win of inlining is not so much the code size (although if done right, that will be smaller too), but the fact that inlining DOES NOT CLOBBER AS MANY REGISTERS!
For VMI, the default clobber was "cc", and you need a way to allow at least that, because saving and restoring flags is too expensive on x86.
Quoted 6 lines
Thanks Linus.
*This* was the reason that the current hand-coded calls only clobber % eax. It was a compromise between native (no clobbers) and others (might need a reg).
I still don't think this was a good trade. The primary motivation for clobbering %eax was that Xen wanted a free register to use for computing the offset into the shared data in the case of SMP preemptible kernels. Xen no longer needs such a register, they can use the PDA offset instead. And it does hurt native performance by unconditionally stealing a register in the four most commonly invoked paravirt-ops code sequences.
Quoted 6 lines
Now, since we decided to allow
paravirt_opsoperations to be normal C (ie. the patching is optional and done late), we actually push and pop % ecx and %edx. This makes the call site 10 bytes long, which is a nice size for patching anyway (enough for a movl $0, <addr>, a-la lguest's cli, or movw $0, %gs:<addr> if we supported SMP).
You can do it in 11 bytes with no clobbers and normal C semantics by linking to a direct address instead of calling to an indirect, but then you need some gross fixup technology in paravirt_patch:
if (call_addr == (void*)native_sti) {
...
}
I think we should probably try to do it in 12 bytes. Freeing eax to the inline caller is likely to make up the 2 bytes of space more we have to nop.
One thing I always tried to get in VMI was to encapsulate the actual code which went through the business of computing arguments that were not even used in the native case. Unfortunately, that seems impossible in the current design, but I don't think it is an issue because I don't think there is actually a way to express:
SWITCHABLE_CODE_BLOCK_BEGIN {
/* arbitrary C code for native */
} SWITCHABLE_CODE_BLOCK_ALTERNATIVE {
/* arbitrary C code for something else */
}
Dave's linker suggestion is probably the best for things like that.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 6 lines · Zachary Amsden
Quoted 3 lines · Rusty Russell
*This* was the reason that the current hand-coded calls only clobber % eax. It was a compromise between native (no clobbers) and others (might need a reg).
I still don't think this was a good trade.
...
Quoted 1 line
Xen no longer needs such a register
Hmm, well, if VMI is happy, Xen is happy, and lguest is happy, then perhaps we're better off with a cc-only clobber rule? Certainly makes life simpler.
Quoted 6 lines
Quoted 5 lines
Now, since we decided to allow
paravirt_opsoperations to be normal C (ie. the patching is optional and done late), we actually push and pop % ecx and %edx. This makes the call site 10 bytes long, which is a nice size for patching anyway (enough for a movl $0, <addr>, a-la lguest's cli, or movw $0, %gs:<addr> if we supported SMP).
Quoted 7 lines
You can do it in 11 bytes with no clobbers and normal C semantics by linking to a direct address instead of calling to an indirect, but then you need some gross fixup technology in
paravirt_patch:if (
call_addr== (void*)native_sti) {
...
}
Well, I don't think we need such hacks: since we have to use handcoded asm and mark the callsites anyway, marking what they're calling is trivial.
The other idea from "btfixup" is that we can do the patching *much* earlier, so we don't need the initial code to be valid at all if we wanted to: we just need room to patch in a call insn. We could then generate trampolines which do the necessary pushes & pops automatically for backends which want to use C calling conventions.
Perhaps it's time for code and benchmarks?
Rusty.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Zachary AmsdenQuoted 2 lines · Zachary Amsden
For VMI, the default clobber was "cc", and you need a way to allow at least that, because saving and restoring flags is too expensive on x86.
According to lore (Andi, I think), asm() always clobbers cc.
Quoted 7 lines
I still don't think this was a good trade. The primary motivation for clobbering %eax was that Xen wanted a free register to use for computing the offset into the shared data in the case of SMP preemptible kernels. Xen no longer needs such a register, they can use the PDA offset instead. And it does hurt native performance by unconditionally stealing a register in the four most commonly invoked paravirt-ops code sequences.
Actually, it still does need a temp register. The sequence for cli is:
mov %fs:xen_vcpu, %eax
movb $1,1(%eax)
At some point I hope to move the vcpu structure directly into the pda/percpu variables, at which point it will need no temps.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Am Di 20.03.2007 06:54 schrieb Jeremy Fitzhardinge <[email protected]>:
Quoted 7 lines
Quoted 4 lines · Zachary Amsden
For VMI, the default clobber was "cc", and you need a way to allow at
least that, because saving and restoring flags is too expensive on x86.According to lore (Andi, I think),
asm()always clobbers cc.
asm with input and/or output. I'm not sure about asms without that.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Jeremy FitzhardingeQuoted 6 lines · Jeremy Fitzhardinge
Quoted 2 lines · Zachary Amsden
For VMI, the default clobber was "cc", and you need a way to allow at least that, because saving and restoring flags is too expensive on x86.
According to lore (Andi, I think),
asm()always clobbers cc.
On x86, yes. Practically any instruction will clobber cc anyway, so it's the default.
Although the gcc guys have occasionally been suggesting we should set it in our asms.
Quoted 4 lines
Actually, it still does need a temp register. The sequence for cli is:
mov %fs:xen_vcpu, %eax
movb $1,1(%eax)
We should just do this natively. There's been several tests over the years saying that it's much more efficient to do sti/cli as a simple store, and handling the "oops, we got an interrupt while interrupts were disabled" as a special case.
I have this dim memory that ARM has done it that way for a long time because it's so expensive to do a "real" cli/sti.
And I think -rt does it for other reasons. It's just more flexible.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Jeremy Fitzhardinge
Actually, it still does need a temp register. The sequence for cli is:
mov %fs:xen_vcpu, %eax
movb $1,1(%eax)We should just do this natively. There's been several tests over the years saying that it's much more efficient to do sti/cli as a simple store, and handling the "oops, we got an interrupt while interrupts were disabled" as a special case.
Quoted 4 lines
I have this dim memory that ARM has done it that way for a long time because it's so expensive to do a "real" cli/sti.
And I think -rt does it for other reasons. It's just more flexible.
If that is the case. In the normal kernel what would the "the oops, we got an interrupt code do?"
I assume it would leave interrupts disabled when it returns? Like we currently do with the delayed disable of normal interrupts?
I'm trying to understand the proposed semantics.
Looking at the above code snippet. I guess it is about time to merge our per_cpu and pda variables...
Eric
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 5 lines · Eric W. Biederman
If that is the case. In the normal kernel what would the "the oops, we got an interrupt code do?"
I assume it would leave interrupts disabled when it returns? Like we currently do with the delayed disable of normal interrupts?
Yeah, disable interrupts, and set a flag that the fake "sti" can test, and just return without doing anything.
(You may or may not also need to do extra work to Ack the hardware interrupt etc, which may be irq-controller specific. Once the CPU has accepted the interrupt, you may not be able to just leave it dangling)
But I was throwing that out as a long-term thing. I'm not claiming it's trivial, but it should be doable.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 8 lines · Linus Torvalds
Quoted 5 lines · Eric W. Biederman
If that is the case. In the normal kernel what would the "the oops, we got an interrupt code do?"
I assume it would leave interrupts disabled when it returns? Like we currently do with the delayed disable of normal interrupts?
Quoted 7 lines
Yeah, disable interrupts, and set a flag that the fake "sti" can test, and just return without doing anything.
(You may or may not also need to do extra work to Ack the hardware interrupt etc, which may be irq-controller specific. Once the CPU has accepted the interrupt, you may not be able to just leave it dangling)
So it would be something like:
pda.intr_mask = 1; /* disable interrupts */
...
pda.intr_mask = 0; /* enable interrupts */
if (xchg(&pda.intr_pending, 0)) /* check pending */
asm("sti"); /* was pending; isr left cpu interrupts masked */and in the interrupt handler:
if (pda.intr_mask) {
pda.intr_pending = 1;
regs->eflags &= ~IF;
maybe_ack_interrupt_controller();
iret
}?
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 6 lines · Jeremy Fitzhardinge
Quoted 5 lines
Yeah, disable interrupts, and set a flag that the fake "sti" can test, and just return without doing anything.
(You may or may not also need to do extra work to Ack the hardware interrupt etc, which may be irq-controller specific. Once the CPU has accepted the interrupt, you may not be able to just leave it dangling)
Quoted 7 lines
So it would be something like:
pda.intr_mask = 1; /* disable interrupts */ ... pda.intr_mask = 0; /* enable interrupts */ if (xchg(&pda.intr_pending, 0)) /* check pending */
Well, can't do xchg, since it implies #LOCK, and you'll lose more than you gain on the processors where it matters. Cmpxchg is fine, but processor dependent.
Or, just make the interrupt handlers use software resend for IRQs when pda.intr_mask is set to zero. Now, local_irq_save / restore are very pretty:
int local_irq_save(void)
{
int tmp = pda.intr_mask;
pda.intr_mask = 0;
/*
* note there is a window here where local IRQs notice intr_mask == 0
* in that case, they will attempt to resend the IRQ via a tasklet,
* and will succeed, albeit through a slightly longer path */
local_bh_disable();
return tmp;
}
void local_irq_restore(int enabled)
{
pda.intr_mask = enabled;
/*
* note there is a window here where softirqs are not processed by
* the interrupt handler, but that is not a problem, since it will
* get done here in the outer enable of any nested pair. */
if (enabled)
local_bh_enable(); }
I think Ingo's suggestion of using the hardirq tracing is another way that could work, but it seems to be too heavyweight and tied too much to the lockdep verification code - plus it inserts additional raw_irq_disable in places that seem counter to the goal of getting rid of them in the first place. Perhaps I'm misunderstanding the code, though.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 12 lines · Zachary Amsden
void
local_irq_restore(int enabled){ pda.intr_mask = enabled; /* * note there is a window here where softirqs are not processed by * the interrupt handler, but that is not a problem, since it will * get done here in the outer enable of any nested pair. */ if (enabled) local_bh_enable(); }
Actually, this one is more complicated. You also need to actually enable hardware interrupts again if they got disabled by an interrupt actually occurring while the "soft-interrupt" was disabled.
But since it's all a local-cpu issue, you can do things like test cpu-local memory flags for whetehr that has happened or not.
So it *should* be something as simple as
local_irq_disable()
{
pda.irq_enable = 0;
}
handle_interrupt()
{
if (!pda.irq_enable) {
pda.irq_queued = 1;
queue_interrupt();
.. make sure we return with hardirq's now
disabled: just clear IF in the pt_regs ..
return;
}
.. normal ..
}
local_irq_enable()
{
pda.irq_enable = 1;
barrier();
/* Common case - nothing happened while we were fake-disabled.. */
if (!pda.irq_queued)
return;
/* Ok, actually handle the things! */
handle_queued_irqs();
/*
* And enable the hw interrupts again, they got disabled
* when we were queueing stuff..
*/
hardware_sti();
}but I haven't really gone over it in any detail, I may have missed something really obvious.
Anyway, it really *should* be pretty damn simple. No need to disable preemption, there should be no events that can *cause* it, since all interrupts get headed off at the pass.. (the return-from-interrupt thng should already notice that it's returning to an interrupts-disabled section and not try to do any preemption).
What did I miss?
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 15 lines · Linus Torvalds
Quoted 12 lines · Zachary Amsden
void
local_irq_restore(int enabled){ pda.intr_mask = enabled; /* * note there is a window here where softirqs are not processed by * the interrupt handler, but that is not a problem, since it will * get done here in the outer enable of any nested pair. */ if (enabled) local_bh_enable(); }
Quoted 2 lines
Actually, this one is more complicated. You also need to actually enable hardware interrupts again if they got disabled by an interrupt actually occurring while the "soft-interrupt" was disabled.
Actually, I was thinking the irq handlers would just not mess around with eflags on the stack, just call the chip to ack the interrupt and re-enable hardware interrupts when they left, since that is free anyway with the iret. Maybe leaving irqs disabled is better.
Quoted 4 lines
Anyway, it really *should* be pretty damn simple. No need to disable preemption, there should be no events that can *cause* it, since all interrupts get headed off at the pass.. (the return-from-interrupt thng should already notice that it's returning to an interrupts-disabled section and not try to do any preemption).
What did I miss?
I wasn't disabling preemption to actually disable preemption. I was just using bh_disable as a global hammer to stop softirqs (thus the irq replay tasklet) from running during the normal irq_exit path. Then, we can just use the existing software IRQ replay code, and I think barely any new code (queue_irq(), etc) has to be written.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 4 lines · Zachary Amsden
Actually, I was thinking the irq handlers would just not mess around with eflags on the stack, just call the chip to ack the interrupt and re-enable hardware interrupts when they left, since that is free anyway with the iret.
No can do. Think level-triggered. You *need* to disable the interrupt, and disabling it at the CPU is the easiest approach. Even so, you need to worry about SMP and screaming interrupts at all CPU's, but if you don't ack it to the IO-APIC until later, that should be ok (alternatively, you need to just mask-and-ack the irq controller).
Quoted 1 line
Maybe leaving irqs disabled is better.
One of the advantages of doing that is that you only ever have a queue of one single entry, which then makes it easier to do the replay.
Linus
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Zachary Amsden
Actually, I was thinking the irq handlers would just not mess around with eflags on the stack, just call the chip to ack the interrupt and re-enable hardware interrupts when they left, since that is free anyway with the iret.
No can do. Think level-triggered. You *need* to disable the interrupt, and disabling it at the CPU is the easiest approach. Even so, you need to worry about SMP and screaming interrupts at all CPU's, but if you don't ack it to the IO-APIC until later, that should be ok (alternatively, you need to just mask-and-ack the irq controller).
Well, you can keep it masked, but a more important point is that I've entirely neglected local interrupts. This might work for IRQs, but for local timer or thermal or IPIs, using the tasklet based replay simply will not work.
Quoted 2 lines
One of the advantages of doing that is that you only ever have a queue of one single entry, which then makes it easier to do the replay.
Yes. Unfortunately now both do_IRQ and all the smp_foo interrupt handlers need to detect and queue for replay, but fortunately they all have the interrupt number conveniently on the stack.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Jeremy FitzhardingeQuoted 17 lines · Jeremy Fitzhardinge
Quoted 8 lines · Linus Torvalds
Quoted 5 lines · Eric W. Biederman
If that is the case. In the normal kernel what would the "the oops, we got an interrupt code do?"
I assume it would leave interrupts disabled when it returns? Like we currently do with the delayed disable of normal interrupts?Quoted 7 lines
Yeah, disable interrupts, and set a flag that the fake "sti" can test, and just return without doing anything.
(You may or may not also need to do extra work to Ack the hardware interrupt etc, which may be irq-controller specific. Once the CPU has accepted the interrupt, you may not be able to just leave it dangling)
Quoted 7 lines
So it would be something like:
pda.intr_mask = 1; /* disable interrupts */ ... pda.intr_mask = 0; /* enable interrupts */ if (xchg(&pda.intr_pending, 0)) /* check pending */ asm("sti"); /* was pending; isr left cpu interrupts masked */
I don't know that you need an xchg there. If you're still on the same CPU, it should all be nice and causal even across an interrupt handler. So it could be:
pda.intr_mask = 0; /* intr_pending can't get set after this */
if (unlikely(pda.intr_pending)) {
pda.intr_pending = 0;
asm("sti");
}(This would actually need a C barrier, but I'll ignore that as this'd end up being asm...)
But other interesting things could happen. If we never did a real CLI and we get preempted and switched to another CPU between clearing intr_mask and checking intr_pending, we get a little confused.
But perhaps that doesn't matter because we'd by definition have no pending interrupts on either processor?
Is it expensive to do an STI if interrupts are already enabled?
Signature · 1 line
Mathematics is the supreme nostalgia of our time.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 6 lines · Matt Mackall
I don't know that you need an xchg there. If you're still on the same CPU, it should all be nice and causal even across an interrupt handler. So it could be:
pda.intr_mask = 0; /*
intr_pendingcan't get set after this */
Why not? Oh, I see. intr_mask is inverted form of EFLAGS_IF.
Quoted 5 lines
if (unlikely(pda.intr_pending)) {
pda.intr_pending = 0;
asm("sti");
}Quoted 7 lines
(This would actually need a C barrier, but I'll ignore that as this'd end up being asm...)
But other interesting things could happen. If we never did a real CLI and we get preempted and switched to another CPU between clearing
intr_maskand checkingintr_pending, we get a little confused.
I think Jeremy's idea was to have interrupt handlers leave interrupts disabled on exit if pda.intr_mask was set. In which case, they would bypass all work and we could never get preempted. I don't think leaving hardware interrupts disabled for such a long time is good though.
Quoted 5 lines
But perhaps that doesn't matter because we'd by definition have no pending interrupts on either processor?
Is it expensive to do an STI if interrupts are already enabled?
Yes.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 3 lines · Zachary Amsden
I think Jeremy's idea was to have interrupt handlers leave interrupts disabled on exit if pda.intr_mask was set. In which case, they would bypass all work and we could never get preempted.
Yes, I was worried that if we left the isr without actually handling the interrupt, it would still be asserted and we'd just get interrupted again. The idea is that we avoid touching cli/sti for the common case of no interrupts while interrupts are disabled, but we'd still need to fall back to using them if an interrupt becomes pending.
Quoted 2 lines
I don't think leaving hardware interrupts disabled for such a long time is good though.
How long? It would be no longer than now, and possibly less, wouldn't it?
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 14 lines · Jeremy Fitzhardinge
Quoted 4 lines · Zachary Amsden
I think Jeremy's idea was to have interrupt handlers leave interrupts disabled on exit if pda.intr_mask was set. In which case, they would bypass all work and we could never get preempted.
Yes, I was worried that if we left the isr without actually handling the interrupt, it would still be asserted and we'd just get interrupted again. The idea is that we avoid touching cli/sti for the common case of no interrupts while interrupts are disabled, but we'd still need to fall back to using them if an interrupt becomes pending.
Quoted 6 lines
Quoted 3 lines
I don't think leaving hardware interrupts disabled for such a long time is good though.
How long? It would be no longer than now, and possibly less, wouldn't it?
Hmm. Perhaps. Something about the asymmetry bothers me alot though.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Zachary AmsdenQuoted 9 lines · Zachary Amsden
Quoted 6 lines · Matt Mackall
I don't know that you need an xchg there. If you're still on the same CPU, it should all be nice and causal even across an interrupt handler. So it could be:
pda.intr_mask = 0; /*
intr_pendingcan't get set after this */Why not? Oh, I see.
intr_maskis inverted form ofEFLAGS_IF.
It's not even that. There are two things that can happen:
case 1:
intr_mask = 1;
<interrupt occurs and is deferred>
intr_mask = 0;
/* intr_pending is already set and CLI is in effect */
if(intr_pending)case 2:
intr_mask = 1;
intr_mask = 0;
<interrupt occurs and is processed>
/* intr_pending remains cleared */
if(intr_pending)As this is all about local interrupts, it's all on a single CPU and out of order issues aren't visible..
Quoted 2 lines
Quoted 2 lines
(This would actually need a C barrier, but I'll ignore that as this'd end up being asm...)
..unless the compiler is doing the reordering, of course.
Quoted 7 lines
Quoted 3 lines
But other interesting things could happen. If we never did a real CLI and we get preempted and switched to another CPU between clearing
intr_maskand checkingintr_pending, we get a little confused.I think Jeremy's idea was to have interrupt handlers leave interrupts disabled on exit if pda.intr_mask was set. In which case, they would bypass all work and we could never get preempted.
I was actually worrying about the case where the interrupt came in "late". But I don't think it's a problem there either.
Quoted 2 lines
I don't think leaving
hardware interrupts disabled for such a long time is good though.
It can only be worse than the current situation by the amount of time it takes to defer an interrupt once. On average, it'll be a lot better as most critical sections are -tiny-.
Signature · 1 line
Mathematics is the supreme nostalgia of our time.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Matt MackallQuoted 40 lines · Matt Mackall
Quoted 19 lines · Jeremy Fitzhardinge
Quoted 9 lines · Linus Torvalds
Quoted 6 lines · Eric W. Biederman
If that is the case. In the normal kernel what would the "the oops, we got an interrupt code do?"
I assume it would leave interrupts disabled when it returns? Like we currently do with the delayed disable of normal interrupts?Quoted 8 lines
Yeah, disable interrupts, and set a flag that the fake "sti" can test, and just return without doing anything.
(You may or may not also need to do extra work to Ack the hardware interrupt etc, which may be irq-controller specific. Once the CPU has accepted the interrupt, you may not be able to just leave it dangling)
Quoted 8 lines
So it would be something like:
pda.intr_mask = 1; /* disable interrupts */ ... pda.intr_mask = 0; /* enable interrupts */ if (xchg(&pda.intr_pending, 0)) /* check pending */ asm("sti"); /* was pending; isr left cpu interrupts masked */I don't know that you need an xchg there. If you're still on the same CPU, it should all be nice and causal even across an interrupt handler. So it could be:
pda.intr_mask = 0; /* intr_pending can't get set after this */ if (unlikely(pda.intr_pending)) { pda.intr_pending = 0; asm("sti"); }
Quoted 7 lines
(This would actually need a C barrier, but I'll ignore that as this'd end up being asm...)
But other interesting things could happen. If we never did a real CLI and we get preempted and switched to another CPU between clearing
intr_maskand checkingintr_pending, we get a little confused.
Could prevent preempt if pda.intr_mask is set. preemptible() is defined as:
# define preemptible() (preempt_count() == 0 && !irqs_disabled())
anyway, so that would be changed to look at the intr_mask rather than eflags.
(I'm not sure if preemptible() is actually used to determine whether preempt or not).
Alternatively, the intr_mask could be encoded in a bit of preempt_count...
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Linus TorvaldsQuoted 2 lines
But I was throwing that out as a long-term thing. I'm not claiming it's trivial, but it should be doable.
One thing I was pondering was to replace the expensive popfs with
bt $IF,(%rsp)
jnc 1f
sti
1:
But would be mostly a P4 optimization again and I'm not 100% sure it is worth it.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 8 lines · Andi Kleen
One thing I was pondering was to replace the expensive popfs with
bt $IF,(%rsp) jnc 1f sti
1:But would be mostly a P4 optimization again and I'm not 100% sure it is worth it.
Worth it on 32-bit. On AMD64, probably not. On Intel 64-bit, maybe, but less important than in P4 days.
This could change character completely if used at the tail of a function where you now have
sti; 1: ret
Which generates an interrupt holdoff on the ret, an unusual thing to do.
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 2 lines
Worth it on 32-bit. On AMD64, probably not. On Intel 64-bit, maybe, but less important than in P4 days.
Well most of Intel 64bit is P4 -- and Intel is still shipping millions more of them each quarter.
Quoted 6 lines
This could change character completely if used at the tail of a function where you now have
sti; 1: ret
Which generates an interrupt holdoff on the ret, an unusual thing to do.
Unusual yes, but I don't see how it should cause problems. Or do you have anything specific in mind?
Worse is probably that on K8 this case might cause a pipeline stall if there isn't a prefix in front of the ret.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Eric W. BiedermanQuoted 2 lines · Eric W. Biederman
Looking at the above code snippet. I guess it is about time to merge our
per_cpuand pda variables...
Rusty has a nice patch series to do just that; I think he's still looking for a taker.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Eric W. BiedermanQuoted 2 lines · Eric W. Biederman
Looking at the above code snippet. I guess it is about time to merge our
per_cpuand pda variables...
Indeed, thanks for the prod. Now 2.6.21-rc4-mm1 is out, I'll resend the patches.
Cheers,
Rusty.
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Linus TorvaldsQuoted 4 lines · Linus Torvalds
I have this dim memory that ARM has done it that way for a long time because it's so expensive to do a "real" cli/sti.
And I think -rt does it for other reasons. It's just more flexible.
-rt doesnt wrap cli/sti anymore: spin_lock_irq*() doesnt disable irq flags on -rt, thus the amount of real irqs-off sections is very small and reviewable.
But nevertheless an incarnation of the code survived and is upstream already, in the form of TRACE_IRQFLAGS lockdep code ;) This implements a soft hardirq flag _today_: all that would be needed is for Xen to define raw_local_irq_disable() as a NOP, and to use the current->hardirqs_enabled as 'soft IRQ-off flag'.
Note that ->hardirqs_enabled is self-maintained, i.e. it's not just a stupid shadow of the hardirq flag, it's an independently maintained flag that does not rely on the existence of the hard flag.
[ this code even has its own debugging code, so out-of-sync-flags,
double-off and double-on is detected and complained about. So all the hard stuff has already been done as part of lockdep, and it's even long-term maintainable because under a native lockdep kernel we check the soft flag against the hard flag. ]
so i think a soft cli/sti flag support should be merged/unified with the trace_hardirqs_on()/trace_hardirqs_off() code.
Ingo
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Linus TorvaldsQuoted 9 lines · Linus Torvalds
We should just do this natively. There's been several tests over the years saying that it's much more efficient to do sti/cli as a simple store, and handling the "oops, we got an interrupt while interrupts were disabled" as a special case.
I have this dim memory that ARM has done it that way for a long time because it's so expensive to do a "real" cli/sti.
And I think -rt does it for other reasons. It's just more flexible.
64-bit powerpc does this now as well.
Paul.
David Miller
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
From: Paul Mackerras <[email protected]>
Date: Wed, 21 Mar 2007 11:03:14 +1100
Quoted 7 lines
Quoted 5 lines · Linus Torvalds
We should just do this natively. There's been several tests over the years saying that it's much more efficient to do sti/cli as a simple store, and handling the "oops, we got an interrupt while interrupts were disabled" as a special case.
Quoted 6 lines
Quoted 4 lines
I have this dim memory that ARM has done it that way for a long time because it's so expensive to do a "real" cli/sti.
And I think -rt does it for other reasons. It's just more flexible.
64-bit powerpc does this now as well.
I was curious about this so I had a look.
There appears to be three pieces of state used to manage this on powerpc, PACASOFTIRQEN(r13), PACAHARDIRQEN(r13) and the SOFTE() in the stackframe.
Plus there is all of this complicated logic on trap entry and exit to manage these three values properly.
local_irq_restore() doesn't look like a simple piece of code either. Logically it should be simple, update the software binary state, and if enabling see if any interrupts came in while we were disable so we can run them.
Given all of that, is it really cheaper than just flipping the bit in the cpu control register? :-/
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Eric W. BiedermanQuoted 1 line · * Eric W. Biederman ([email protected]) wrote:
Is it truly critical to inline any of these instructions?
I don't have any current measurements. But we'd been aiming at getting irq_{en,dis}able to a simple memory write to pda. But simplicity, maintenance, etc. win over trimming a couple cycles, so still worth real look.
thanks,
-chris
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Eric W. BiedermanQuoted 15 lines · Eric W. Biederman
Quoted 12 lines · Rusty Russell
Quoted 8 lines · Andi Kleen
Quoted 5 lines
The idea is _NOT_ that you go look for references to the
paravirt_opsmembers structure, that would be stupid and you wouldn't be able to use the most efficient addressing mode on a given cpu, you'd be patching up indirect calls and crap like that. Just say no...That wouldn't handle inlines though. At least some of the current paravirtops like cli/sti are critical enough to require inlining.
Well, we'd patch the inline over the call if we have room.
Quoted 8 lines
Quoted 5 lines
Magic patching would be neat, but the downsides are that (1) we can't expand the patching room and (2) there's no way of attaching clobber info to the call site (doing register liveness analysis is not appealing).
True. You can use all of the call clobbered registers.
The trouble is that there are places where we want to intercept things like sti/cli in entry.S in a context where all the registers are in use, so we can't generally make the assumption that caller-saved regs are clobberable. For any call-site in compiler-generated code we can make that assumption, of course.
Quoted 9 lines
Quoted 8 lines
Now, this may not be fatal. 5 bytes is enough for all the native ops to
be patched inline. For lguest this covers popf and pushf, but not cli
and sti (10 bytes): they'd have to be calls.As for clobber info, it turns out that almost all of the calls can clobber %eax, which is probably enough. We just need to mark the handful of asm ones where this isn't true.
Quoted 6 lines
I guess if the code is larger than a function call I'm failing to see the disadvantage in making it a direct function call. Any modern processor ought to be able to predict it perfectly, and processors like the P4 may even optimize the call out of their L1 instruction cache.
For current processors, I think i-cache pollution is really the only downside to a call. But maybe you could put multiple pv-op functions into a cacheline if they're used in close proximity (cli/sti/save_fl/restore_fl would all fit into a single cacheline).
Quoted 4 lines
If what David is suggesting works, making all of these direct calls
looks easy and very maintainable. At which point patching
instructions inline is quite possibly overkill.
I spent some time looking at what it would take to get this going. It would probably look something like:
1. make CONFIG_PARAVIRT select CONFIG_RELOCATABLE
2. generate vmlinux
3. extract relocs, and process the references to paravirt symbols into a table
4. compile and link that table into the kernel again (like the ksyms dance)
5. at boot time, use that table to redirect calls/references into the paravirt backend to the appropriate one
I think if we store the reloc references as section-relative, and the reloc table itself is linked into its own section, then there's no need for multipass linking to combine the reloc table into the kernel image, since adding that table won't affect any existing reloc.
All this is doable; I'd probably end up hacking boot/compressed/relocs.c to generate the appropriate reloc table. My main concern is hacking the kernel build process itself; I'm unsure of what it would actually take to implement all this.
Quoted 2 lines
Is it truly critical to inline any of these instructions?
Possibly not, but I'd like to be able to say with confidence that running a PARAVIRT kernel on bare hardware has no performance loss compared to running a !PARAVIRT kernel. There's the case of small instruction sequences which have been replaced with calls (such as sti/cli/push;popf/etc), and also the case of hooks which are noops on native (like make_pte/pte_val, etc). In those cases it would be nice to patch in a replacement, either a real instruction or just nops.
J
David Miller
maintainerRe: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
From: Jeremy Fitzhardinge <[email protected]>
Date: Mon, 19 Mar 2007 12:10:08 -0700
Quoted 4 lines
All this is doable; I'd probably end up hacking
boot/compressed/relocs.cto generate the appropriate reloc table. My main concern is hacking the kernel build process itself; I'm unsure of what it would actually take to implement all this.
32-bit Sparc's btfixup might be usable as a guide.
Another point worth making is that for function calls you can fix things up lazily if you want.
So you link, build the reloc tables, then link in a *.o file that does provide the functions in the form of stubs. The stubs intercept the call, and patch the callsite, then revector to the real handler.
I don't like this idea actually because it essentially means you either:
1) Only allow one setting of the operations
OR
2) Need to have code which walks the whole reloc table anyways to handle settings after the first so you can revector everyone back to the stubs and lazy reloc properly again
In fact forget I mentioned this idea :)
As another note, I do agree with Linus about the register usage arguments. It is important. I think it's been mentioned but what you could do is save nothing (so that "sti" and "cli" are just that and cost nothing), but the more complicated versions save and restore enough registers to operate.
It all depends upon what you're trying to do. For example, it's easy to use patching to make different PTE layouts be supportable in the same kernel image. We do this on sparc64 since sun4v has a different PTE layout than sun4u, you can see the code in asm-sparc64/pgtable.h for details (search for "sun4v_*_patch")
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 5 lines · David Miller
Another point worth making is that for function calls you can fix things up lazily if you want.
[...]
In fact forget I mentioned this idea :)
OK :) I think we'll only ever want to bind to a hypervisor once, since the underlying hypervisor can't change on the fly (well, in principle it might if you migrate, but you'll have more problems than just dealing with the hook points). And lazy binding doesn't buy much; I think its much better to get it all out of the way at once, and then throw the reloc info away.
Quoted 6 lines
As another note, I do agree with Linus about the register usage arguments. It is important. I think it's been mentioned but what you could do is save nothing (so that "sti" and "cli" are just that and cost nothing), but the more complicated versions save and restore enough registers to operate.
Right, that's pretty much what we do now.
Quoted 6 lines
It all depends upon what you're trying to do. For example, it's easy to use patching to make different PTE layouts be supportable in the same kernel image. We do this on sparc64 since sun4v has a different PTE layout than sun4u, you can see the code in
asm-sparc64/pgtable.hfor details (search for "sun4v_*_patch")
I see. We want to do something conceptually like this, but we need to handle more than just adjusting which of two constants to use as a mask. For example, Xen needs to run pfns through a pfn->mfn (machine frame number) conversion and back when making/unpacking pagetable entries.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Jeremy FitzhardingeQuoted 5 lines
Possibly not, but I'd like to be able to say with confidence that running a PARAVIRT kernel on bare hardware has no performance loss compared to running a !PARAVIRT kernel. There's the case of small instruction sequences which have been replaced with calls (such as sti/cli/push;popf/etc),
My guess is that most critical pushf/popf are in spin_lock_irqsave(). It would be possible to special case that one -- inline it -- and use out of line versions for all the others.
-Andi
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Ingo MolnarQuoted 8 lines · Ingo Molnar
Quoted 5 lines · Jeremy Fitzhardinge
Wrap a set of interesting
paravirt_opscalls in a wrapper which makes the callsites available for patching. Unfortunately this is pretty ugly because there's no way to get gcc to generate a function call, but also wrap just the callsite itself with the necessary labels.
Quoted 7 lines
Quoted 5 lines
This patch supports functions with 0-4 arguments, and either void or returning a value. 64-bit arguments must be split into a pair of 32-bit arguments (lower word first). Small structures are returned in registers.
ugh. This is beyond ugly!
Yes. I can't say I'm proud of this, but I really couldn't come up with a cleaner approach.
Quoted 5 lines
Why dont we just compile two images, one for
Xen and one for native, do two passes to get those two images and 'merge' them into a single vmlinuz (so that we still have a 'single' kernel unit to deal with on the distro side). This way we avoid all this crazy, limited, fragile patchery business...
Well, one thing to make clear is this is absolutely not a Xen-specific patch or piece of code. This is part of the paravirt_ops infrastructure designed to remove the overhead of all the indirect calls which are scattered all over the place. (Perhaps I should post the general paravirt and Xen specific patches in separate patch series to make this clear...).
The idea is to wrap the callsite itself with in the same manner as the other altinstructions so that the general patcher can, at the very least, convert the indirect call to a direct one, or nop it out if its an indirect call. This means that a pv_ops implementation can get about 90% of the benefit of patching without any extra effort.
So, I disagree with your characterisation that its "limited"; this is a pretty general mechanism. The fragile part is in using the PVOP_* macros properly to match the ABI's calling convention, particularly with tricky cases like passed and returned structures and 64-bit args. But that just needs to be done once in one place, and is otherwise self-contained.
I would love a better mechanism. I played with using things like gcc's builtin_apply stuff, and so on, but I could find no way to get gcc to set up the args and then be able to just emit the call itself under asm control.
I haven't looked at Dave's reply in detail, but I saw some mention of using relocs. The idea is intriguing , but I don't quite see how it would all fit together.
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 7 lines · Jeremy Fitzhardinge
Well, one thing to make clear is this is absolutely not a Xen-specific patch or piece of code. This is part of the
paravirt_opsinfrastructure designed to remove the overhead of all the indirect calls which are scattered all over the place. (Perhaps I should post the general paravirt and Xen specific patches in separate patch series to make this clear...).
Quoted 6 lines
The idea is to wrap the callsite itself with in the same manner as the other altinstructions so that the general patcher can, at the very least, convert the indirect call to a direct one, or nop it out if its an indirect call. This means that a
pv_opsimplementation can get about 90% of the benefit of patching without any extra effort.
I like this code very much; although it is unavoidably ugly, it is a nice general mechanism for doing code rewriting. Much more elaboration on this below.
Quoted 7 lines
So, I disagree with your characterisation that its "limited"; this is a pretty general mechanism. The fragile part is in using the PVOP_* macros properly to match the ABI's calling convention, particularly with tricky cases like passed and returned structures and 64-bit args. But that just needs to be done once in one place, and is otherwise self-contained.
You could just use the VMI ABI, then patch everything at runtime to call into the Xen paravirt-ops backend ;)
Quoted 5 lines
I would love a better mechanism. I played with using things like gcc's
builtin_applystuff, and so on, but I could find no way to get gcc to set up the args and then be able to just emit the call itself under asm control.
I fought tooth and nail to get something cleaner than this for VMI back when it was a subarch. In the end, the best I could do was wrap the constraints into prettier macros so the asm volatile stuff wasn't sticking out everywhere. It was pretty, but the macros were so grotesque that I was exiled from my home planet.
static inline void local_irq_restore(const unsigned long flags)
{
vmi_wrap_call(
SetInterruptMask, "pushl %0; popfl",
VMI_NO_OUTPUT,
1, VMI_IREG1 (flags),
XCONC("cc", "memory"));
}So the constraints are obvious and tied to the inline assembly. But Jeremy seems to have done even better with the vcall stuff. Prettier:
+ PVOP_VCALL0(setup_boot_clock);
Quoted 5 lines
I haven't looked at Dave's reply in detail, but I saw some mention of using relocs. The idea is intriguing , but I don't quite see how it would all fit together.
We went through this design exercise, and thought it was pretty promising. Basically, you would reserve a set of "local" relocation types that should never be emitted by the toolchain. Then you can have complex relocations, such as "replace pushf; popf %0 with arbitrary code." You can even leave the arguments unfixed and grant the compiler register allocation, as long as you took care to encode the input / output registers somewhere (in a .reloc section of some sort, or encoded in the relocation type itself).
Now you can make complex decisions at runtime, and apply choice functions to these relocations that can cope with a variety of different circumstances - you could encode not just paravirt-ops as relocations, but all of the alternative instructions, and smp alternatives, and even higher level constructs, such as choices made by the user with the kernel command line - some potential examples:
acpi=noirq
idle=halt
With proper synchronization, using something like stop_machine_run, you can even make these choices dynamically, and then relink the kernel in place to take faster paths. And the technique is universal, so you could use it cross architecture, which would be really helpful for architectures that say, have really slow indirect branches.
Once the technique gains wide acceptance, you could use it for all kernel interfaces which have static function pointers for the post-init lifetime of the kernel. Which might contribute to a global performance improvement of perhaps a couple percent. But the cost is clearly the complexity.
I just had a slightly interesting idea - you could even catch bugs where dynamic assignments to function pointers fail to update the appropriate patch sites by checking for non .init code sections which write through accelerated_fn_ptr_t's using static checking from sparse.
Is that sort of what you were thinking of Dave?
Zach
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 4 lines · Zachary Amsden
I like this code very much; although it is unavoidably ugly, it is a nice general mechanism for doing code rewriting. Much more elaboration on this below.
Thanks.
Quoted 9 lines
static inline void
local_irq_restore(const unsigned long flags){ vmi_wrap_call( SetInterruptMask, "pushl %0; popfl", VMI_NO_OUTPUT, 1, VMI_IREG1 (flags), XCONC("cc", "memory")); }
Quoted 4 lines
So the constraints are obvious and tied to the inline assembly. But Jeremy seems to have done even better with the vcall stuff. Prettier:
+
PVOP_VCALL0(setup_boot_clock);
Yeah, it doesn't try as hard as your example, so its all based around the function call ABI. If you want to inline something, you need to do that elsewhere, which I guess is OK because that's not the common case (only very simple cases can be replaced by inlines, and only a few of those are worth doing).
Quoted 8 lines
We went through this design exercise, and thought it was pretty promising. Basically, you would reserve a set of "local" relocation types that should never be emitted by the toolchain. Then you can have complex relocations, such as "replace pushf; popf %0 with arbitrary code." You can even leave the arguments unfixed and grant the compiler register allocation, as long as you took care to encode the input / output registers somewhere (in a .reloc section of some sort, or encoded in the relocation type itself).
I'm pretty sure that's not what he means. The big objection to the PVOP_* stuff is the fact that there are these massive macros full of inline asm to wrap the calls, which have to be invoked in a fragile type-unsafe way. Adding custom relocs would suffer the same problem, since you'd need inline asm to deal with them, and I'm deathly frightened of whatever binutils would do if you mean real relocs.
I think the suggestion is much simpler. If you convince gcc/binutils to leave the .reloc section in vmlinux, and make that available to the kernel itself, then you can scan all the kernel's relocs to find ones which refer to paravirt_ops, and use those to determine which are callsites that can be patched.
The main upside is that all the callsites are just normal C calls; there's no special syntax or strange macros, and we get the full benefit of typechecking, etc.
But I can see a few downsides compared the current scheme:
1. Identifying the callsites is a somewhat hackish process of looking at a reloc and doing a bit of dissassembly to see what is using the reloc, to identify calls and jumps
2. There's nothing explicit to tell us how much space there is to patch into; we just have to assume sizeof(indirect call/jmp)
3. There's no information about the register environment at the callsite, so we just have to adopt normal C ABI rules. For the patch sites in hand-written asm, this could be tricky.
4. gcc could do strange things which prevent detection of patch sites. For example, it might CSE the value of, say, paravirt_ops.irq_enable, which would be a reasonable optimisation, but prevent any of the resulting indirect calls from being patched. In general it relies on gcc to generate identifiable callsites, which is a bit unpredictable.
5. There's still a moderate amount of binutils hackery to get the relocs into the right form, and there's plenty of scope for it to screw up.
Quoted 1 line
[ Roswell technology deleted ]
J
Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
Quoted 6 lines · Jeremy Fitzhardinge
I think the suggestion is much simpler. If you convince gcc/binutils to leave the .reloc section in vmlinux, and make that available to the kernel itself, then you can scan all the kernel's relocs to find ones which refer to
paravirt_ops, and use those to determine which are callsites that can be patched.
Yes, that is pretty nice.
Quoted 15 lines
The main upside is that all the callsites are just normal C calls; there's no special syntax or strange macros, and we get the full benefit of typechecking, etc.
But I can see a few downsides compared the current scheme:
1. Identifying the callsites is a somewhat hackish process of looking at a reloc and doing a bit of dissassembly to see what is using the reloc, to identify calls and jumps
2. There's nothing explicit to tell us how much space there is to patch into; we just have to assume sizeof(indirect call/jmp)
3. There's no information about the register environment at the callsite, so we just have to adopt normal C ABI rules. For the patch sites in hand-written asm, this could be tricky.
4. gcc could do strange things which prevent detection of patch
Quoted 9 lines
sites. For example, it might CSE the value of, say, paravirt_ops.irq_enable, which would be a reasonable optimisation, but prevent any of the resulting indirect calls from being patched. In general it relies on gcc to generate identifiable callsites, which is a bit unpredictable.
5. There's still a moderate amount of binutils hackery to get the relocs into the right form, and there's plenty of scope for it to screw up.
And yes, those are nasty points. I think I'd be interested in seeing more discussion on it, perhaps those issues could be worked out.
Quoted 2 lines
Quoted 2 lines
[ Roswell technology deleted ]
Nack. Everyone needs Roswell technology.
Actually, that was not a serious proposal. I think the effort and complexity would likely not justify the gain. But I still had to throw it out, since it is what we use on Betelgeuse.
Z
Re: [Xen-devel] Re: [patch 13/26] Xen-paravirt_ops: Consistently wrap paravirt ops callsites to make them patchable
In reply to Ingo MolnarQuoted 12 lines · Ingo Molnar
Quoted 9 lines · Jeremy Fitzhardinge
Wrap a set of interesting
paravirt_opscalls in a wrapper which makes the callsites available for patching. Unfortunately this is pretty ugly because there's no way to get gcc to generate a function call, but also wrap just the callsite itself with the necessary labels.This patch supports functions with 0-4 arguments, and either void or returning a value. 64-bit arguments must be split into a pair of 32-bit arguments (lower word first). Small structures are returned in registers.
Quoted 5 lines
ugh. This is beyond ugly! Why dont we just compile two images, one for Xen and one for native, do two passes to get those two images and 'merge' them into a single vmlinuz (so that we still have a 'single' kernel unit to deal with on the distro side). This way we avoid all this crazy, limited, fragile patchery business...
But with lguest, VMI and kvm I don't think that's a good idea.
For background: the current patching code is ugly too, but it only deals with the 6 most common functions, so it's contained. This gets us pretty close to !CONFIG_PARAVIRT performance. (But slowdown is still measurable).
We get worse with Xen wanting to alter mkpte et al. By the time we patch another 6 functions (each one slightly different), we get pretty close to general coverage anyway, so it's clearer to do them all.
I think the question is, can we do it better than this? My previous attempts (which lead to the current code) ranged from ugly to very ugly 8(
Rusty.
PATCH 14/26 Xen-paravirt_ops: add common patching machinery
In reply to Jeremy FitzhardingeImplement the actual patching machinery. paravirt_patch_default() contains the logic to automatically patch a callsite based on a few simple rules:
- if the paravirt_op function is paravirt_nop, then patch nops
- if the paravirt_op function is a jmp target, then jmp to it
- if the paravirt_op function is callable and doesn't clobber too much for the callsite, call it directly
paravirt_patch_default is suitable as a default implementation of paravirt_ops.patch, will remove most of the expensive indirect calls in favour of either a direct call or a pile of nops.
Backends may implement their own patcher, however. There are several helper functions to help with this:
paravirt_patch_nop nop out a callsiteparavirt_patch_ignore leave the callsite as-isparavirt_patch_call patch a call if the caller and callee
have compatible clobbersparavirt_patch_jmp patch in a jmpparavirt_patch_insns patch some literal instructions over
the callsite, if they fit
This patch also implements more direct patches for the native case, so that when running on native hardware many common operations are implemented inline.
--- arch/i386/kernel/alternative.c | 3 arch/i386/kernel/paravirt.c | 163 ++++++++++++++++++++++++++++++++-------- include/asm-i386/paravirt.h | 11 ++ 3 files changed, 147 insertions(+), 30 deletions(-)
===================================================================
arch/i386/kernel/alternative.c +3 -0
@@ -359,6 +359,9 @@ void apply_paravirt(struct paravirt_patc used = paravirt_ops.patch (p->instrtype, p->clobbers, p->instr, p->len); + +BUG_ON (used > p->len); +#ifdef CONFIG_DEBUG_PARAVIRT {int i;
===================================================================
arch/i386/kernel/paravirt.c +133 -30
173 lines not on this page · raw message
===================================================================
include/asm-i386/paravirt.h +11 -0
@@ -821,6 +821,17 @@ struct paravirt_patch_site {u16 clobbers;/* what registers you may clobber */ }; +unsigned paravirt_patch_nop (void ); +unsigned paravirt_patch_ignore (unsigned len); +unsigned paravirt_patch_call (void *target,u16 tgt_clobbers, +void *site,u16 site_clobbers, +unsigned len); +unsigned paravirt_patch_jmp (void *target,void *site,unsigned len); +unsigned paravirt_patch_default (u8 type,u16 clobbers,void *site,unsigned len); + +unsigned paravirt_patch_insns (void *site,unsigned len, +const char *start,const char *end); +extern struct paravirt_patch_site __start_parainstructions[], __stop_parainstructions[];
Re: [patch 14/26] Xen-paravirt_ops: add common patching machinery
Quoted 4 lines · Jeremy Fitzhardinge
Implement the actual patching machinery.
paravirt_patch_default()contains the logic to automatically patch a callsite based on a few simple rules:
Quoted 8 lines
- if the
paravirt_opfunction isparavirt_nop, then patch nops
- if theparavirt_opfunction is a jmp target, then jmp to it
- if theparavirt_opfunction is callable and doesn't clobber too much for the callsite, call it directly
paravirt_patch_defaultis suitable as a default implementation of paravirt_ops.patch, will remove most of the expensive indirect calls in favour of either a direct call or a pile of nops.
Ingo
Re: [patch 14/26] Xen-paravirt_ops: add common patching machinery
Quoted 11 lines · Ingo Molnar
Quoted 9 lines · Jeremy Fitzhardinge
Implement the actual patching machinery.
paravirt_patch_default()contains the logic to automatically patch a callsite based on a few simple rules:- if the
paravirt_opfunction isparavirt_nop, then patch nops
- if theparavirt_opfunction is a jmp target, then jmp to it
- if theparavirt_opfunction is callable and doesn't clobber too much for the callsite, call it directly
Quoted 5 lines
Quoted 3 lines
paravirt_patch_defaultis suitable as a default implementation of paravirt_ops.patch, will remove most of the expensive indirect calls in favour of either a direct call or a pile of nops.Acked-by Ingo Molnar <[email protected]>
I like this one too, but note that it needs a twist when we change to use direct calls to wrappers for the Great paravirt_ops Unexporting.
Rusty.
PATCH 15/26 Xen-paravirt_ops: Add apply_to_page_range() which applies a function to a pte range.
In reply to Jeremy FitzhardingeAdd a new mm function apply_to_page_range() which applies a given function to every pte in a given virtual address range in a given mm structure. This is a generic alternative to cut-and-pasting the Linux idiomatic pagetable walking code in every place that a sequence of PTEs must be accessed.
Although this interface is intended to be useful in a wide range of situations, it is currently used specifically by several Xen subsystems, for example: to ensure that pagetables have been allocated for a virtual address range, and to construct batched special pagetable update requests to map I/O memory (in ioremap()).
--- include/linux/mm.h | 5 ++ mm/memory.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 99 insertions(+)
===================================================================
include/linux/mm.h +5 -0
@@ -1135,6 +1135,11 @@ struct page *follow_page(struct vm_area_#define FOLL_GET 0x04/* do get_page on page */ #define FOLL_ANON 0x08/* give ZERO_PAGE if no pgtable */ +typedef int (*pte_fn_t )(pte_t *pte,struct page *pmd_page,unsigned long addr, +void *data); +extern int apply_to_page_range (struct mm_struct *mm,unsigned long address, +unsigned long size,pte_fn_t fn,void *data); +#ifdef CONFIG_PROC_FS void vm_stat_account (struct mm_struct *,unsigned long ,struct file *,long );#else
===================================================================
mm/memory.c +94 -0
100 lines not on this page · raw message
Re: [patch 15/26] Xen-paravirt_ops: Add apply_to_page_range() which applies a function to a pte range.
Quoted 5 lines · Jeremy Fitzhardinge
Add a new mm function
apply_to_page_range()which applies a given function to every pte in a given virtual address range in a given mm structure. This is a generic alternative to cut-and-pasting the Linux idiomatic pagetable walking code in every place that a sequence of PTEs must be accessed.
nice one! I suspect we could simplify some of the less performance-critical open-coded pagetable walker loops in the kernel with this? (i'm not even sure it's all that much of a performance loss to pass a function pointer around - would be nice to convert say mprotect() to this and see the performance difference?)
Ingo
Re: [patch 15/26] Xen-paravirt_ops: Add apply_to_page_range() which applies a function to a pte range.
Quoted 9 lines · * Ingo Molnar ([email protected]) wrote:
Quoted 5 lines · Jeremy Fitzhardinge
Add a new mm function
apply_to_page_range()which applies a given function to every pte in a given virtual address range in a given mm structure. This is a generic alternative to cut-and-pasting the Linux idiomatic pagetable walking code in every place that a sequence of PTEs must be accessed.
Quoted 5 lines
nice one! I suspect we could simplify some of the less performance-critical open-coded pagetable walker loops in the kernel with this? (i'm not even sure it's all that much of a performance loss to pass a function pointer around - would be nice to convert say
mprotect()to this and see the performance difference?)
We went through this before, and eventually got kick back to keep it Xen specific. There's still some effort for generalized walkers from the Gelato folks IIRC. And we can certainly adapt.
Re: [patch 15/26] Xen-paravirt_ops: Add apply_to_page_range() which applies a function to a pte range.
In reply to Ingo MolnarQuoted 6 lines · Ingo Molnar
nice one! I suspect we could simplify some of the less performance-critical open-coded pagetable walker loops in the kernel with this? (i'm not even sure it's all that much of a performance loss to pass a function pointer around - would be nice to convert say
mprotect()to this and see the performance difference?)
apply_page_to_range has the side-effect of allocating all the pagetable levels for the address range it walks. Xen uses this to populate pagetables (for example, build out the pagetable, and let the hypervisor plug a mapping into the pte page).
If that's OK for the other uses, then it should be good.
J
PATCH 16/26 Xen-paravirt_ops: Allocate and free vmalloc areas
In reply to Jeremy FitzhardingeAllocate/destroy a 'vmalloc' VM area: alloc_vm_area and free_vm_area The alloc function ensures that page tables are constructed for the region of kernel virtual address space and mapped into init_mm.
--- arch/i386/mm/fault.c | 43 +++++++++++++++++++++++++++++++++++++++++++ include/linux/vmalloc.h | 4 ++++ 2 files changed, 47 insertions(+)
===================================================================
arch/i386/mm/fault.c +43 -0
@@ -23,6 +23,7 @@#include <linux/module.h> #include <linux/kprobes.h> #include <linux/uaccess.h> +#include <linux/vmalloc.h> #include <asm/system.h> #include <asm/desc.h>
@@ -624,3 +625,45 @@ void _vmalloc_sync_all(void) start = address +PGDIR_SIZE ; } } + + +static int f (pte_t *pte,struct page *pmd_page,unsigned long addr,void *data) +{ +/* apply_to_page_range() does all the hard work. */ +return 0 ; +} + +struct vm_struct *alloc_vm_area (unsigned long size) +{ +struct vm_struct *area; + + area =get_vm_area (size,VM_IOREMAP ); +if (area ==NULL ) +return NULL ; + +/* +* This ensures that page tables are constructed for this region +* of kernel virtual address space and mapped into init_mm. +*/ +if (apply_to_page_range (&init_mm, (unsigned long )area->addr, + area->size, f,NULL )) { +free_vm_area (area); +return NULL ; + } + +/* Make sure the pagetables are constructed in process kernel +mappings */ +vmalloc_sync_all (); + +return area; +} +EXPORT_SYMBOL_GPL (alloc_vm_area); + +void free_vm_area (struct vm_struct *area) +{ +struct vm_struct *ret; + ret =remove_vm_area (area->addr); +BUG_ON (ret != area); +kfree (area); +} +EXPORT_SYMBOL_GPL (free_vm_area);
===================================================================
include/linux/vmalloc.h +4 -0
@@ -68,6 +68,10 @@ extern int map_vm_area(struct vm_struct struct page ***pages);extern void unmap_vm_area (struct vm_struct *area); +/* Allocate/destroy a 'vmalloc' VM area. */ +extern struct vm_struct *alloc_vm_area (unsigned long size); +extern void free_vm_area (struct vm_struct *area); +/* * Internals. Dont't use.. */
Re: [patch 16/26] Xen-paravirt_ops: Allocate and free vmalloc areas
Quoted 7 lines · Jeremy Fitzhardinge
Allocate/destroy a 'vmalloc' VM area:
alloc_vm_areaandfree_vm_areaThe alloc function ensures that page tables are constructed for the region of kernel virtual address space and mapped intoinit_mm.---
a/arch/i386/mm/fault.c+struct
vm_struct*alloc_vm_area(unsigned long size)
this should go into mm/vmalloc.c.
Ingo
Re: [patch 16/26] Xen-paravirt_ops: Allocate and free vmalloc areas
Quoted 4 lines · Ingo Molnar
Quoted 2 lines
+struct
vm_struct*alloc_vm_area(unsigned long size)this should go into
mm/vmalloc.c.
The trouble is that it calls the arch-dependent vmalloc_sync_all(), which is important for Xen's use of this function. And without that it's just a pointless wrapper around get_vm_area.
J
PATCH 17/26 Xen-paravirt_ops: Add nosegneg capability to the vsyscall page notes
In reply to Jeremy FitzhardingeAdd the "nosegneg" fake capabilty to the vsyscall page notes. This is used by the runtime linker to select a glibc version which then disables negative-offset accesses to the thread-local segment via %gs. These accesses require emulation in Xen (because segments are truncated to protect the hypervisor address space) and avoiding them provides a measurable performance boost.
--- arch/i386/kernel/vsyscall-note.S | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
===================================================================
arch/i386/kernel/vsyscall-note.S +28 -0
@@ -23,3 +24,31 @@ 3: .balign 4; /* pad out section */ ASM_ELF_NOTE_BEGIN(".note.kernel-version", "a", UTS_SYSNAME, 0) .long LINUX_VERSION_CODE ASM_ELF_NOTE_END + +#ifdef CONFIG_XEN +/* + * Add a special note telling glibc's dynamic linker a fake hardware + * flavor that it will use to choose the search path for libraries in the + * same way it uses real hardware capabilities like "mmx". + * We supply "nosegneg" as the fake capability, to indicate that we + * do not like negative offsets in instructions using segment overrides, + * since we implement those inefficiently. This makes it possible to + * install libraries optimized to avoid those access patterns in someplace + * like /lib/i686/tls/nosegneg. Note that an /etc/ld.so.conf.d/file + * corresponding to the bits here is needed to make ldconfig work right. + * It should contain: + * hwcap 0 nosegneg + * to match the mapping of bit to name that we give here. + */ +#define NOTE_KERNELCAP_BEGIN(ncaps, mask) \ + ASM_ELF_NOTE_BEGIN(".note.kernelcap", "a", "GNU", 2) \ + .long ncaps, mask +#define NOTE_KERNELCAP(bit, name) \ + .byte bit; .asciz name +#define NOTE_KERNELCAP_END ASM_ELF_NOTE_END + +NOTE_KERNELCAP_BEGIN(1, 1) +NOTE_KERNELCAP(1, "nosegneg") +NOTE_KERNELCAP_END +#endif +
Re: [patch 17/26] Xen-paravirt_ops: Add nosegneg capability to the vsyscall page notes
i've Cc:-ed Roland and Ulrich, who should make the call on this one.
Quoted 59 lines · Jeremy Fitzhardinge
Add the "nosegneg" fake capabilty to the vsyscall page notes. This is used by the runtime linker to select a glibc version which then disables negative-offset accesses to the thread-local segment via %gs. These accesses require emulation in Xen (because segments are truncated to protect the hypervisor address space) and avoiding them provides a measurable performance boost.
Signed-off-by Ian Pratt <[email protected]>Signed-off-by Christian Limpach <[email protected]>Signed-off-by Chris Wright <[email protected]>Acked-by Zachary Amsden <[email protected]>--- arch/i386/kernel/vsyscall-note.S | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)===================================================================
arch/i386/kernel/vsyscall-note.S +28
-0@@ -23,3 +24,31 @@ 3: .balign 4; /* pad out section */ ASM_ELF_NOTE_BEGIN(".note.kernel-version", "a", UTS_SYSNAME, 0) .long LINUX_VERSION_CODE ASM_ELF_NOTE_END + +#ifdef CONFIG_XEN +/* + * Add a special note telling glibc's dynamic linker a fake hardware + * flavor that it will use to choose the search path for libraries in the + * same way it uses real hardware capabilities like "mmx". + * We supply "nosegneg" as the fake capability, to indicate that we + * do not like negative offsets in instructions using segment overrides, + * since we implement those inefficiently. This makes it possible to + * install libraries optimized to avoid those access patterns in someplace + * like /lib/i686/tls/nosegneg. Note that an /etc/ld.so.conf.d/file + * corresponding to the bits here is needed to make ldconfig work right. + * It should contain: + * hwcap 0 nosegneg + * to match the mapping of bit to name that we give here. + */ +#define NOTE_KERNELCAP_BEGIN(ncaps, mask) \ + ASM_ELF_NOTE_BEGIN(".note.kernelcap", "a", "GNU", 2) \ + .long ncaps, mask +#define NOTE_KERNELCAP(bit, name) \ + .byte bit; .asciz name +#define NOTE_KERNELCAP_END ASM_ELF_NOTE_END + +NOTE_KERNELCAP_BEGIN(1, 1) +NOTE_KERNELCAP(1, "nosegneg") +NOTE_KERNELCAP_END +#endif +Signature · 7 lines
-- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [patch 17/26] Xen-paravirt_ops: Add nosegneg capability to the vsyscall page notes
Quoted 3 lines
arch/i386/kernel/vsyscall-note.S 3: .balign 4; /* pad out section */ · line 52
Quoted 3 lines
arch/i386/kernel/vsyscall-note.S +3
-0+NOTE_KERNELCAP_BEGIN(1, 1) +NOTE_KERNELCAP(1, "nosegneg") +NOTE_KERNELCAP_END
This should be:
NOTE_KERNELCAP_BEGIN(1, 1)NOTE_KERNELCAP(0, "nosegneg")NOTE_KERNELCAP_END
i.e. 1->0 in the "bit" member. (Note the ld.so.conf.d file must have the matching bit number for ldconfig-based lookups to do the right thing.) Or else:
NOTE_KERNELCAP_BEGIN(1, 2)NOTE_KERNELCAP(0, "nosegneg")NOTE_KERNELCAP_END
i.e. 1->2 in the "mask" member. (The mask value should be 1<<bit.)
Some pre-release glibc's (before 2.4) had a bug in the code that parses this, and would crash parsing the correct note. Using the wrong bit value with nonmatching mask worked around this. IIRC, no glibc release ever included the buggy version of the code. In nonbuggy glibc, the mismatched value causes the "nosegneg" to be omitted from the directory search (under LD_LIBRARY_PATH and default directories), though ldconfig-based lookups will work (the most common case).
Thanks,
Roland
Re: [patch 17/26] Xen-paravirt_ops: Add nosegneg capability to the vsyscall page notes
Quoted 10 lines · Roland McGrath
This should be:
NOTE_KERNELCAP_BEGIN(1, 1)NOTE_KERNELCAP(0, "nosegneg")NOTE_KERNELCAP_ENDi.e. 1->0 in the "bit" member. (Note the ld.so.conf.d file must have the matching bit number for ldconfig-based lookups to do the right thing.) Or else:
Quoted 6 lines
NOTE_KERNELCAP_BEGIN(1, 2)NOTE_KERNELCAP(0, "nosegneg")NOTE_KERNELCAP_ENDi.e. 1->2 in the "mask" member. (The mask value should be 1<<bit.)
Thanks Roland. I've never really understood this stuff, and I just copied this cargo-cultishly.
I'm not quite sure what you're suggesting here though. Do you mean one of:
NOTE_KERNELCAP_BEGIN(1, 1)NOTE_KERNELCAP(0, "nosegneg")NOTE_KERNELCAP_END
or
NOTE_KERNELCAP_BEGIN(1, 2)NOTE_KERNELCAP(1, "nosegneg")NOTE_KERNELCAP_END
is the correct thing to use?
Quoted 8 lines
Some pre-release glibc's (before 2.4) had a bug in the code that parses this, and would crash parsing the correct note. Using the wrong bit value with nonmatching mask worked around this. IIRC, no glibc release ever included the buggy version of the code. In nonbuggy glibc, the mismatched value causes the "nosegneg" to be omitted from the directory search (under
LD_LIBRARY_PATHand default directories), though ldconfig-based lookups will work (the most common case).
Are you saying that one of the corrected forms might cause old glibcs to crash, or just ignore nosegneg?
J
Re: [patch 17/26] Xen-paravirt_ops: Add nosegneg capability to the vsyscall page notes
Quoted 6 lines
I'm not quite sure what you're suggesting here though. Do you mean one of:
NOTE_KERNELCAP_BEGIN(1, 1)NOTE_KERNELCAP(0, "nosegneg")NOTE_KERNELCAP_END
Quoted 7 lines
or
NOTE_KERNELCAP_BEGIN(1, 2)NOTE_KERNELCAP(1, "nosegneg")NOTE_KERNELCAP_ENDis the correct thing to use?
Yes. (Sorry about the typo, 1 and 0 are close enough aren't they? ;-)
Quoted 10 lines
Quoted 7 lines
Some pre-release glibc's (before 2.4) had a bug in the code that parses this, and would crash parsing the correct note. Using the wrong bit value with nonmatching mask worked around this. IIRC, no glibc release ever included the buggy version of the code. In nonbuggy glibc, the mismatched value causes the "nosegneg" to be omitted from the directory search (under
LD_LIBRARY_PATHand default directories), though ldconfig-based lookups will work (the most common case).Are you saying that one of the corrected forms might cause old glibcs to crash, or just ignore nosegneg?
Yes, the affected glibc crashed with the canonical form (the first above). I'm not sure such a glibc exists in the wild today, maybe only in some FC5 test releases (I CC'd Jakub so he can verify that at least as far as FC). Rik van Riel discovered that the s/0/1/ tweak sufficed for common daily use (i.e. ld.so.cache hits), and did that temporarily when he was hacking on Xen Linux kernels for Fedora, but reverted it after we fixed glibc.
The uncorrected form causes current (correct) glibc to ignore nosegneg (for cache misses).
Looking at the buggy version of the code, I think it will not crash with the second form above, just avoid using bit 0. (But I wouldn't swear to it without testing it.) The second form should certainly be fine with the current glibc. Just make sure that "kernelcap 1 nosegneg" is used in the ld.so.conf.d file, to match 1 in the NOTE_KERNELCAP first arg.
Thanks,
Roland
PATCH 18/26 Xen-paravirt_ops: Add XEN config options
In reply to Jeremy FitzhardingeThe XEN config option enables the Xen paravirt_ops interface, which is installed when the kernel finds itself running under Xen.
Xen is no longer a sub-architecture, so the X86_XEN subarch config option has gone.
Xen is currently incompatible with:
- PREEMPT
- HZ: set to 100Hz for now, to cut down on VCPU context switch rate. This will be adapted to use tickless later.
--- arch/i386/Kconfig | 2 ++ arch/i386/xen/Kconfig | 10 ++++++++++ 2 files changed, 12 insertions(+)
===================================================================
arch/i386/Kconfig +2 -0
@@ -216,6 +216,8 @@ config PARAVIRT under a hypervisor, improving performance significantly. However, when run without a hypervisor the kernel is theoretically slower. If in doubt, say N. + +source "arch/i386/xen/Kconfig" config VMI bool "VMI Paravirt-ops support"
===================================================================
arch/i386/xen/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +# +# This Kconfig describes xen options +# + +config XEN + bool "Enable support for Xen hypervisor" + depends on PARAVIRT && HZ_100 && !PREEMPT && !NO_HZ + default y + help + This is the Linux Xen port.
Re: [patch 18/26] Xen-paravirt_ops: Add XEN config options
Quoted 2 lines · Jeremy Fitzhardinge
Xen is currently incompatible with:
- PREEMPT
why? That should be fixed.
Ingo
Re: [patch 18/26] Xen-paravirt_ops: Add XEN config options
Quoted 8 lines · Ingo Molnar
Quoted 3 lines · Jeremy Fitzhardinge
Xen is currently incompatible with:
- PREEMPTwhy? That should be fixed.
Actually I don't think that's true any more. We'll need to work out how to deal with preempt and suspend/resume, but the process freezer seems like the way to do that.
J
PATCH 19/26 Xen-paravirt_ops: Add Xen interface header files
In reply to Jeremy FitzhardingeAdd Xen interface header files. These are taken fairly directly from the Xen tree, but somewhat rearranged to suit the kernel's conventions.
Define macros and inline functions for doing hypercalls into the hypervisor.
--- include/asm-i386/xen/hypercall.h | 413 ++++++++++++++++++++++++++++++ include/asm-i386/xen/hypervisor.h | 72 +++++ include/asm-i386/xen/interface.h | 187 +++++++++++++ include/xen/interface/elfnote.h | 133 +++++++++ include/xen/interface/event_channel.h | 195 ++++++++++++++ include/xen/interface/features.h | 43 +++ include/xen/interface/grant_table.h | 301 ++++++++++++++++++++++ include/xen/interface/io/blkif.h | 96 +++++++ include/xen/interface/io/console.h | 23 + include/xen/interface/io/netif.h | 152 +++++++++++ include/xen/interface/io/ring.h | 260 +++++++++++++++++++ include/xen/interface/io/xenbus.h | 42 +++ include/xen/interface/io/xs_wire.h | 87 ++++++ include/xen/interface/memory.h | 145 ++++++++++ include/xen/interface/physdev.h | 61 ++++ include/xen/interface/sched.h | 77 +++++ include/xen/interface/vcpu.h | 109 ++++++++ include/xen/interface/version.h | 60 ++++ include/xen/interface/xen.h | 445 +++++++++++++++++++++++++++++++++ 19 files changed, 2901 insertions(+)
===================================================================
include/asm-i386/xen/hypercall.h new file +413 -0
@@ -0,0 +1,413 @@ +/****************************************************************************** +* hypercall.h +* +* Linux-specific hypervisor handling. +* +* Copyright (c) 2002-2004, K A Fraser +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License version 2 +* as published by the Free Software Foundation; or, when distributed +* separately from the Linux kernel or incorporated into other +* software packages, subject to the following license: +* +* Permission is hereby granted, free of charge, to any person obtaining a copy +* of this source file (the "Software"), to deal in the Software without +* restriction, including without limitation the rights to use, copy, modify, +* merge, publish, distribute, sublicense, and/or sell copies of the Software, +* and to permit persons to whom the Software is furnished to do so, subject to +* the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +* IN THE SOFTWARE. +*/ + +#ifndef __HYPERCALL_H__ +#define __HYPERCALL_H__ + +#include <linux/errno.h> +#include <linux/string.h> + +#include <xen/interface/xen.h> +#include <xen/interface/sched.h> +#include <xen/interface/physdev.h> + +extern struct {char _entry[32 ]; } hypercall_page[]; + +#define _hypercall0 (type, name) \ +({ \ + long __res; \ + asm volatile ( \ + "call %[call]" \ + : "=a" (__res) \ + : [call] "m" (hypercall_page[__HYPERVISOR_##name]) \ + : "memory" ); \ + (type)__res; \ +}) + +#define _hypercall1 (type, name, a1) \ +({ \ + long __res, __ign1; \ + asm volatile ( \ + "call %[call]" \ + : "=a" (__res), "=b" (__ign1) \ + : "1" ((long)(a1)), \ + [call] "m" (hypercall_page[__HYPERVISOR_##name]) \ + : "memory" ); \ + (type)__res; \ +}) + +#define _hypercall2 (type, name, a1, a2) \ +({ \ + long __res, __ign1, __ign2; \ + asm volatile ( \ + "call %[call]" \ + : "=a" (__res), "=b" (__ign1), "=c" (__ign2) \ + : "1" ((long)(a1)), "2" ((long)(a2)), \ + [call] "m" (hypercall_page[__HYPERVISOR_##name]) \ + : "memory" ); \ + (type)__res; \ +}) + +#define _hypercall3 (type, name, a1, a2, a3) \ +({ \ + long __res, __ign1, __ign2, __ign3; \ + asm volatile ( \ + "call %[call]" \ + : "=a" (__res), "=b" (__ign1), "=c" (__ign2), \ + "=d" (__ign3) \ + : "1" ((long)(a1)), "2" ((long)(a2)), \ + "3" ((long)(a3)), \ + [call] "m" (hypercall_page[__HYPERVISOR_##name]) \ + : "memory" ); \ + (type)__res; \ +}) + +#define _hypercall4 (type, name, a1, a2, a3, a4) \ +({ \ + long __res, __ign1, __ign2, __ign3, __ign4; \ + asm volatile ( \ + "call %[call]" \ + : "=a" (__res), "=b" (__ign1), "=c" (__ign2), \ + "=d" (__ign3), "=S" (__ign4) \ + : "1" ((long)(a1)), "2" ((long)(a2)), \ + "3" ((long)(a3)), "4" ((long)(a4)), \ + [call] "m" (hypercall_page[__HYPERVISOR_##name]) \ + : "memory" ); \ + (type)__res; \ +}) + +#define _hypercall5 (type, name, a1, a2, a3, a4, a5) \ +({ \ + long __res, __ign1, __ign2, __ign3, __ign4, __ign5; \ + asm volatile ( \ + "call %[call]" \ + : "=a" (__res), "=b" (__ign1), "=c" (__ign2), \ + "=d" (__ign3), "=S" (__ign4), "=D" (__ign5) \ + : "1" ((long)(a1)), "2" ((long)(a2)), \ + "3" ((long)(a3)), "4" ((long)(a4)), \ + "5" ((long)(a5)), \ + [call] "m" (hypercall_page[__HYPERVISOR_##name]) \ + : "memory" ); \ + (type)__res; \ +}) + +static inline int +HYPERVISOR_set_trap_table ( +struct trap_info *table) +{ +return _hypercall1 (int, set_trap_table, table); +} + +static inline int +HYPERVISOR_mmu_update ( +struct mmu_update *req,int count,int *success_count,domid_t domid) +{ +return _hypercall4 (int, mmu_update, req, count, success_count, domid); +} + +static inline int +HYPERVISOR_mmuext_op ( +struct mmuext_op *op,int count,int *success_count,domid_t domid) +{ +return _hypercall4 (int, mmuext_op, op, count, success_count, domid); +} + +static inline int +HYPERVISOR_set_gdt ( +unsigned long *frame_list,int entries) +{ +return _hypercall2 (int, set_gdt, frame_list, entries); +} + +static inline int +HYPERVISOR_stack_switch ( +unsigned long ss,unsigned long esp) +{ +return _hypercall2 (int, stack_switch, ss, esp); +} + +static inline int +HYPERVISOR_set_callbacks ( +unsigned long event_selector,unsigned long event_address, +unsigned long failsafe_selector,unsigned long failsafe_address) +{ +return _hypercall4 (int, set_callbacks, + event_selector, event_address, + failsafe_selector, failsafe_address); +} + +static inline int +HYPERVISOR_fpu_taskswitch ( +int set) +{ +return _hypercall1 (int, fpu_taskswitch, set); +} + +static inline int +HYPERVISOR_sched_op ( +int cmd,unsigned long arg) +{ +return _hypercall2 (int, sched_op, cmd, arg); +} + +static inline long +HYPERVISOR_set_timer_op ( +u64 timeout) +{ +unsigned long timeout_hi = (unsigned long )(timeout>>32 ); +unsigned long timeout_lo = (unsigned long )timeout; +return _hypercall2 (long, set_timer_op, timeout_lo, timeout_hi); +} + +static inline int +HYPERVISOR_set_debugreg ( +int reg,unsigned long value) +{ +return _hypercall2 (int, set_debugreg, reg, value); +} + +static inline unsigned long +HYPERVISOR_get_debugreg ( +int reg) +{ + return _hypercall1(unsigned long, get_debugreg, reg); +} + +static inline int +HYPERVISOR_update_descriptor ( +u64 ma,u64 desc) +{ +return _hypercall4 (int, update_descriptor, ma, ma>>32 , desc, desc>>32 ); +} + +static inline int +HYPERVISOR_memory_op ( +unsigned int cmd,void *arg) +{ +return _hypercall2 (int, memory_op, cmd, arg); +} + +static inline int +HYPERVISOR_multicall ( +void *call_list,int nr_calls) +{ +return _hypercall2 (int, multicall, call_list, nr_calls); +} + +static inline int +HYPERVISOR_update_va_mapping ( +unsigned long va,pte_t new_val,unsigned long flags) +{ +unsigned long pte_hi =0 ; +#ifdef CONFIG_X86_PAE + pte_hi = new_val.pte_high; +#endif +return _hypercall4 (int, update_va_mapping, va, + new_val.pte_low, pte_hi, flags); +} + +static inline int +HYPERVISOR_event_channel_op ( +int cmd,void *arg) +{ +int rc =_hypercall2 (int, event_channel_op, cmd, arg); +if (unlikely (rc == -ENOSYS )) { +struct evtchn_op op; + op.cmd = cmd; +memcpy (&op.u, arg,sizeof (op.u)); + rc =_hypercall1 (int, event_channel_op_compat, &op); +memcpy (arg, &op.u,sizeof (op.u)); + } +return rc; +} + +static inline int +HYPERVISOR_xen_version ( +int cmd,void *arg) +{ +return _hypercall2 (int, xen_version, cmd, arg); +} + +static inline int +HYPERVISOR_console_io ( +int cmd,int count,char *str) +{ +return _hypercall3 (int, console_io, cmd, count, str); +} + +static inline int +HYPERVISOR_physdev_op (int cmd,void *arg) +{ +int rc =_hypercall2 (int, physdev_op, cmd, arg); +if (unlikely (rc == -ENOSYS )) { +struct physdev_op op; + op.cmd = cmd; +memcpy (&op.u, arg,sizeof (op.u)); + rc =_hypercall1 (int, physdev_op_compat, &op); +memcpy (arg, &op.u,sizeof (op.u)); + } +return rc; +} + +static inline int +HYPERVISOR_grant_table_op ( +unsigned int cmd,void *uop,unsigned int count) +{ +return _hypercall3 (int, grant_table_op, cmd, uop, count); +} + +static inline int +HYPERVISOR_update_va_mapping_otherdomain ( +unsigned long va,pte_t new_val,unsigned long flags,domid_t domid) +{ +unsigned long pte_hi =0 ; +#ifdef CONFIG_X86_PAE + pte_hi = new_val.pte_high; +#endif +return _hypercall5 (int, update_va_mapping_otherdomain, va, + new_val.pte_low, pte_hi, flags, domid); +} + +static inline int +HYPERVISOR_vm_assist ( +unsigned int cmd,unsigned int type) +{ +return _hypercall2 (int, vm_assist, cmd, type); +} + +static inline int +HYPERVISOR_vcpu_op ( +int cmd,int vcpuid,void *extra_args) +{ +return _hypercall3 (int, vcpu_op, cmd, vcpuid, extra_args); +} + +static inline int +HYPERVISOR_suspend ( +unsigned long srec) +{ +return _hypercall3 (int, sched_op, SCHEDOP_shutdown, + SHUTDOWN_suspend, srec); +} + +static inline int +HYPERVISOR_nmi_op ( +unsigned long op, +unsigned long arg) +{ +return _hypercall2 (int, nmi_op, op, arg); +} + +static inline void +MULTI_update_va_mapping (struct multicall_entry *mcl,unsigned long va, +pte_t new_val,unsigned long flags) +{ + mcl->op = __HYPERVISOR_update_va_mapping; + mcl->args[0 ] = va; +#ifdef CONFIG_X86_PAE + mcl->args[1 ] = new_val.pte_low; + mcl->args[2 ] = new_val.pte_high; +#else + mcl->args[1 ] = new_val.pte_low; + mcl->args[2 ] =0 ; +#endif + mcl->args[3 ] = flags; +} + +static inline void +MULTI_grant_table_op (struct multicall_entry *mcl,unsigned int cmd, +void *uop,unsigned int count) +{ + mcl->op = __HYPERVISOR_grant_table_op; + mcl->args[0 ] = cmd; + mcl->args[1 ] = (unsigned long )uop; + mcl->args[2 ] = count; +} + +static inline void +MULTI_update_va_mapping_otherdomain (struct multicall_entry *mcl,unsigned long va, +pte_t new_val,unsigned long flags,domid_t domid) +{ + mcl->op = __HYPERVISOR_update_va_mapping_otherdomain; + mcl->args[0 ] = va; +#ifdef CONFIG_X86_PAE + mcl->args[1 ] = new_val.pte_low; + mcl->args[2 ] = new_val.pte_high; +#else + mcl->args[1 ] = new_val.pte_low; + mcl->args[2 ] =0 ; +#endif + mcl->args[3 ] = flags; + mcl->args[4 ] = domid; +} + +static inline void +MULTI_update_descriptor (struct multicall_entry *mcl,u64 maddr, +struct desc_struct desc) +{ + mcl->op = __HYPERVISOR_update_descriptor; + mcl->args[0 ] = maddr; + mcl->args[1 ] = maddr >>32 ; + mcl->args[2 ] = desc.a; + mcl->args[3 ] = desc.b; +} + +static inline void +MULTI_memory_op (struct multicall_entry *mcl,unsigned int cmd,void *arg) +{ + mcl->op = __HYPERVISOR_memory_op; + mcl->args[0 ] = cmd; + mcl->args[1 ] = (unsigned long )arg; +} + +static inline void +MULTI_mmu_update (struct multicall_entry *mcl,struct mmu_update *req, +int count,int *success_count,domid_t domid) +{ + mcl->op = __HYPERVISOR_mmu_update; + mcl->args[0 ] = (unsigned long )req; + mcl->args[1 ] = count; + mcl->args[2 ] = (unsigned long )success_count; + mcl->args[3 ] = domid; +} + +static inline void +MULTI_mmuext_op (struct multicall_entry *mcl,struct mmuext_op *op,int count, +int *success_count,domid_t domid) +{ + mcl->op = __HYPERVISOR_mmuext_op; + mcl->args[0 ] = (unsigned long )op; + mcl->args[1 ] = count; + mcl->args[2 ] = (unsigned long )success_count; + mcl->args[3 ] = domid; +} +#endif /* __HYPERCALL_H__ */
===================================================================
include/asm-i386/xen/hypervisor.h new file +72 -0
72 lines not on this page · raw message
===================================================================
include/asm-i386/xen/interface.h new file +187 -0
187 lines not on this page · raw message
===================================================================
include/xen/interface/elfnote.h new file +133 -0
133 lines not on this page · raw message
===================================================================
include/xen/interface/event_channel.h new file +195 -0
195 lines not on this page · raw message
===================================================================
include/xen/interface/features.h new file +43 -0
43 lines not on this page · raw message
===================================================================
include/xen/interface/grant_table.h new file +301 -0
301 lines not on this page · raw message
===================================================================
include/xen/interface/io/blkif.h new file +96 -0
96 lines not on this page · raw message
===================================================================
include/xen/interface/io/console.h new file +23 -0
@@ -0,0 +1,23 @@ +/****************************************************************************** +* console.h +* +* Console I/O interface for Xen guest OSes. +* +* Copyright (c) 2005, Keir Fraser +*/ + +#ifndef __XEN_PUBLIC_IO_CONSOLE_H__ +#define __XEN_PUBLIC_IO_CONSOLE_H__ + +typedef uint32_t XENCONS_RING_IDX ; + +#define MASK_XENCONS_IDX (idx, ring) ((idx) & (sizeof(ring)-1)) + +struct xencons_interface { +char in[1024 ]; +char out[2048 ]; +XENCONS_RING_IDX in_cons, in_prod; +XENCONS_RING_IDX out_cons, out_prod; +}; + +#endif /* __XEN_PUBLIC_IO_CONSOLE_H__ */
===================================================================
include/xen/interface/io/netif.h new file +152 -0
152 lines not on this page · raw message
===================================================================
include/xen/interface/io/ring.h new file +260 -0
260 lines not on this page · raw message
===================================================================
include/xen/interface/io/xenbus.h new file +42 -0
42 lines not on this page · raw message
===================================================================
include/xen/interface/io/xs_wire.h new file +87 -0
87 lines not on this page · raw message
===================================================================
include/xen/interface/memory.h new file +145 -0
145 lines not on this page · raw message
===================================================================
include/xen/interface/physdev.h new file +61 -0
61 lines not on this page · raw message
===================================================================
include/xen/interface/sched.h new file +77 -0
77 lines not on this page · raw message
===================================================================
include/xen/interface/vcpu.h new file +109 -0
109 lines not on this page · raw message
===================================================================
include/xen/interface/version.h new file +60 -0
60 lines not on this page · raw message
===================================================================
include/xen/interface/xen.h new file +445 -0
445 lines not on this page · raw message
PATCH 20/26 Xen-paravirt_ops: Core Xen implementation
In reply to Jeremy FitzhardingeCore Xen Implementation
This patch is a rollup of all the core pieces of the Xen implementation, including booting, memory management, interrupts, time and so on.
This adds a second entry point to head.S, which is jumped to when booted by Xen. This allows startup under Xen to be easily detected.
Because Xen starts the kernel in a fairly sane state, very little setup is needed here; it just needs to jump into xen_start_kernel to init the paravirt_ops structure, and then jump into start_kernel proper.
This also makes a few small adjustments to the gdt tables to make them properly suited to Xen.
This patch also contains an initial set of paravirt ops for Xen, mostly ones which can just be implemented with the generic native_* version. At boot time, the global paravirt_ops structure is populated with the Xen pointers in xen_start_kernel, which then jumps to the standard start_kernel.
Also implements Xen pagetable handling, including the machinery to implement direct pagetables. This means that pte update operations are intercepted so that pseudo-physical addresses can be converted to machine addresses. It also implements late pinning/early unpinning so that pagetables are initialized while they're normal read-write memory as much as possible, yet pinned to make cr3 reloads as efficient as possible.
Xen implements interrupts in terms of event channels. This patch maps an event through an event channel to an irq, and then feeds it through the normal interrupt path, via a Xen irq_chip implementation.
Context switches are optimised with the use of multicalls, so that the multiple hypercalls needed to perform the switch can be implemented with a single hypercall. In order to simplify the source changes, this is done via a generic multicall batching mechanism, which can be used to batch arbitrary series of hypercalls into a multicall.
Time is implemented by using a clocksource which is driven from the hypervisor's nanosecond timebase. Xen implements time by extrapolating from known timestamps using the tsc; the hypervisor is responsible for making sure that the tsc is constant rate and synchronized between vcpus.
--- arch/i386/Makefile | 3 arch/i386/kernel/cpu/common.c | 3 arch/i386/kernel/entry.S | 79 +++ arch/i386/kernel/head.S | 13 arch/i386/kernel/paravirt.c | 48 +- arch/i386/kernel/vmlinux.lds.S | 1 arch/i386/xen/Makefile | 2 arch/i386/xen/enlighten.c | 752 ++++++++++++++++++++++++++++++++++++++ arch/i386/xen/events.c | 481 ++++++++++++++++++++++++ arch/i386/xen/features.c | 29 + arch/i386/xen/mmu.c | 411 ++++++++++++++++++++ arch/i386/xen/mmu.h | 49 ++ arch/i386/xen/multicalls.c | 62 +++ arch/i386/xen/multicalls.h | 27 + arch/i386/xen/setup.c | 95 ++++ arch/i386/xen/time.c | 446 ++++++++++++++++++++++ arch/i386/xen/xen-head.S | 30 + arch/i386/xen/xen-ops.h | 31 + include/asm-i386/irq.h | 1 include/asm-i386/paravirt.h | 39 + include/asm-i386/pda.h | 18 include/asm-i386/xen/hypercall.h | 18 include/xen/events.h | 28 + include/xen/features.h | 26 + include/xen/page.h | 175 ++++++++ 25 files changed, 2841 insertions(+), 26 deletions(-)
===================================================================
arch/i386/Makefile +3 -0
@@ -93,6 +93,9 @@ mcore-$(CONFIG_X86_ES7000) := mach-defau mcore-$(CONFIG_X86_ES7000) := mach-default core-$(CONFIG_X86_ES7000) := arch/i386/mach-es7000/ +# Xen paravirtualization support +core-$(CONFIG_XEN) += arch/i386/xen/ + # default subarch .h files mflags-y += -Iinclude/asm-i386/mach-default
===================================================================
arch/i386/kernel/cpu/common.c +3 -0
@@ -19,6 +19,7 @@#include <mach_apic.h> #endif #include <asm/pda.h> +#include <asm/paravirt.h> #include "cpu.h"
@@ -707,6 +708,8 @@ __cpuinit int init_gdt(int cpu, struct t pda->cpu_number = cpu; pda->pcurrent = idle; +paravirt_init_pda (pda, cpu); +return 1 ; }
===================================================================
arch/i386/kernel/entry.S +79 -0
85 lines not on this page · raw message
===================================================================
arch/i386/kernel/head.S +11 -2
@@ -535,6 +535,10 @@ unhandled_paravirt: ud2 #endif +#ifdef CONFIG_XEN +#include "../xen/xen-head.S" +#endif + /* * Real beginning of normal "text" segment */
@@ -544,8 +548,9 @@ ENTRY(_stext) /* * BSS section */ -.section ".bss.page_aligned","w" +.section ".bss.page_aligned","wa" ENTRY(swapper_pg_dir) + .align PAGE_SIZE_asm .fill 1024,4,0 ENTRY(empty_zero_page) .fill 4096,1,0
@@ -614,7 +619,8 @@ ENTRY(boot_gdt_table) /* * The Global Descriptor Table contains 28 quadwords, per-CPU. */ - .align L1_CACHE_BYTES + .section ".data.page_aligned" + .align PAGE_SIZE_asm ENTRY(cpu_gdt_table) .quad 0x0000000000000000 /* NULL descriptor */ .quad 0x0000000000000000 /* 0x0b reserved */
@@ -663,3 +669,6 @@ ENTRY(cpu_gdt_table) .quad 0x0000000000000000 /* 0xf0 - unused */ .quad 0x0000000000000000 /* 0xf8 - GDT entry 31: double-fault TSS */ + /* Be sure this is zeroed to avoid false validations in Xen */ + .fill PAGE_SIZE_asm / 8 - GDT_ENTRIES,8,0 + .previous
===================================================================
arch/i386/kernel/paravirt.c +24 -24
162 lines not on this page · raw message
===================================================================
arch/i386/kernel/vmlinux.lds.S +1 -0
@@ -95,6 +95,7 @@ SECTIONS . = ALIGN(4096); .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { + *(.data.page_aligned) *(.data.idt) }
===================================================================
arch/i386/xen/Makefile new file +2 -0
@@ -0,0 +1,2 @@ +obj-y := enlighten.o setup.o events.o time.o \ + features.o mmu.o multicalls.o
===================================================================
arch/i386/xen/enlighten.c new file +752 -0
752 lines not on this page · raw message
===================================================================
arch/i386/xen/events.c new file +481 -0
481 lines not on this page · raw message
===================================================================
arch/i386/xen/features.c new file +29 -0
@@ -0,0 +1,29 @@ +/****************************************************************************** +* features.c +* +* Xen feature flags. +* +* Copyright (c) 2006, Ian Campbell, XenSource Inc. +*/ +#include <linux/types.h> +#include <linux/cache.h> +#include <linux/module.h> +#include <asm/xen/hypervisor.h> +#include <xen/features.h> + +u8 xen_features[XENFEAT_NR_SUBMAPS *32 ]__read_mostly ; +EXPORT_SYMBOL_GPL (xen_features); + +void xen_setup_features (void ) +{ +struct xen_feature_info fi; +int i, j; + +for (i =0 ; i <XENFEAT_NR_SUBMAPS ; i++) { + fi.submap_idx = i; +if (HYPERVISOR_xen_version (XENVER_get_features, &fi) <0 ) +break ; +for (j=0 ; j<32 ; j++) + xen_features[i*32 +j] = !!(fi.submap &1 <<j); + } +}
===================================================================
arch/i386/xen/mmu.c new file +411 -0
411 lines not on this page · raw message
===================================================================
arch/i386/xen/mmu.h new file +49 -0
49 lines not on this page · raw message
===================================================================
arch/i386/xen/multicalls.c new file +62 -0
62 lines not on this page · raw message
===================================================================
arch/i386/xen/multicalls.h new file +27 -0
@@ -0,0 +1,27 @@ +#ifndef _XEN_MULTICALLS_H +#define _XEN_MULTICALLS_H + +#include "xen-ops.h" + +/* Multicalls */ +struct multicall_space +{ +struct multicall_entry *mc; +void *args; +}; + +/* Allocate room for a multicall and its args */ +struct multicall_space xen_mc_entry (size_t args); + +/* Flush all pending multicalls */ +int xen_mc_flush (void ); + +/* Issue a multicall if we're not in lazy mode */ +static inline void xen_mc_issue (void ) +{ +if (xen_get_lazy_mode () ==PARAVIRT_LAZY_NONE ) +if (xen_mc_flush ()) +BUG (); +} + +#endif /* _XEN_MULTICALLS_H */
===================================================================
arch/i386/xen/setup.c new file +95 -0
95 lines not on this page · raw message
===================================================================
arch/i386/xen/time.c new file +446 -0
446 lines not on this page · raw message
===================================================================
arch/i386/xen/xen-head.S new file +30 -0
@@ -0,0 +1,30 @@ +/* Xen-specific pieces of head.S, intended to be included in the right + place in head.S */ + +#include <linux/elfnote.h> +#include <asm/boot.h> +#include <xen/interface/elfnote.h> + +ENTRY(startup_xen) + movl %esi,xen_start_info + jmp startup_paravirt + +.pushsection ".bss.page_aligned" +ENTRY(hypercall_page) + .align PAGE_SIZE_asm + .skip 0x1000 +.popsection + + ELFNOTE(Xen, XEN_ELFNOTE_GUEST_OS, .asciz, "linux") + ELFNOTE(Xen, XEN_ELFNOTE_GUEST_VERSION, .asciz, "2.6") + ELFNOTE(Xen, XEN_ELFNOTE_XEN_VERSION, .asciz, "xen-3.0") + ELFNOTE(Xen, XEN_ELFNOTE_VIRT_BASE, .long, __PAGE_OFFSET) + ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, .long, startup_xen) + ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, .long, hypercall_page) + ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .asciz, "!writable_page_tables|pae_pgdir_above_4gb") +#ifdef CONFIG_X86_PAE + ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "yes") +#else + ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "no") +#endif + ELFNOTE(Xen, XEN_ELFNOTE_LOADER, .asciz, "generic")
===================================================================
arch/i386/xen/xen-ops.h new file +31 -0
@@ -0,0 +1,31 @@ +#ifndef XEN_OPS_H +#define XEN_OPS_H + +#include <linux/init.h> + +extern struct start_info *xen_start_info; +extern struct shared_info *HYPERVISOR_shared_info; + +char *__init xen_memory_setup (void ); +void __init xen_arch_setup (void ); +void __init xen_init_IRQ (void ); + +void __init xen_time_init (void ); +unsigned long xen_get_wallclock (void ); +int xen_set_wallclock (unsigned long time); + +void stop_hz_timer (void ); +void start_hz_timer (void ); + +DECLARE_PER_CPU (unsigned, xen_lazy_mode); + +static inline unsigned xen_get_lazy_mode (void ) +{ +unsigned ret =get_cpu_var (xen_lazy_mode); +put_cpu_var (xen_lazy_mode); + +return ret; +} + + +#endif /* XEN_OPS_H */
===================================================================
include/asm-i386/irq.h +1 -0
@@ -43,6 +43,7 @@ extern void fixup_irqs(cpumask_t map);extern void fixup_irqs (cpumask_t map);#endif +unsigned int do_IRQ (struct pt_regs *regs);void init_IRQ (void );void __init native_init_IRQ (void );
===================================================================
include/asm-i386/paravirt.h +39 -0
@@ -253,6 +253,7 @@ struct Xgt_desc_struct;struct Xgt_desc_struct ;struct tss_struct ;struct mm_struct ; +struct i386_pda ;struct paravirt_ops {unsigned int kernel_rpl;
@@ -272,6 +273,7 @@ struct paravirt_opsvoid (*arch_setup)(void );char *(*memory_setup)(void );void (*init_IRQ)(void ); +void (*init_pda)(struct i386_pda *,int cpu);void (*pagetable_setup_start)(pgd_t *pgd_base);void (*pagetable_setup_done)(pgd_t *pgd_base);
@@ -399,6 +401,30 @@ struct paravirt_opsvoid (*startup_ipi_hook)(int phys_apicid,unsigned long start_eip,unsigned long start_esp); }; +/* Non-paravirtualized implementations of various operations for +back-ends which don't need their own version. */ +void native_clts (void ); + +unsigned long native_read_cr0 (void ); +void native_write_cr0 (unsigned long val); + +unsigned long native_read_cr2 (void ); +void native_write_cr2 (unsigned long val); + +unsigned long native_read_cr3 (void ); +void native_write_cr3 (unsigned long val); + +unsigned long native_read_cr4 (void ); +unsigned long native_read_cr4_safe (void ); +void native_write_cr4 (unsigned long val); + +void native_wbinvd (void ); + +unsigned long long native_read_msr (unsigned int msr,int *err); +int native_write_msr (unsigned int msr,unsigned long long val); +unsigned long long native_read_tsc (void ); +unsigned long long native_read_pmc (void ); +/* Mark a paravirt probe function. */ #define paravirt_probe (fn) \ static asmlinkage void (*__paravirtprobe_##fn)(void) __attribute_used__ \
@@ -520,6 +546,13 @@ static inline void halt(void) high = _l >>32 ; \ }while (0 ) +void native_write_ldt_entry(void *dt, int entrynum, u32 low, u32 high); +void native_write_gdt_entry (void *dt,int entrynum,u32 low,u32 high); +void native_write_idt_entry (void *dt,int entrynum,u32 low,u32 high); +void native_store_gdt (struct Xgt_desc_struct *dtr); +void native_store_idt (struct Xgt_desc_struct *dtr); +unsigned long native_store_tr (void ); +static inline void load_TR_desc (void ) {PVOP_VCALL0 (load_tr_desc);
@@ -641,6 +674,12 @@ static inline void arch_exit_mmap(structstatic inline void arch_exit_mmap (struct mm_struct *mm) {PVOP_VCALL1 (exit_mmap, mm); +} + +static inline void paravirt_init_pda (struct i386_pda *pda,int cpu) +{ +if (paravirt_ops.init_pda) + (*paravirt_ops.init_pda)(pda, cpu); }#define __flush_tlb () PVOP_VCALL0(flush_tlb_user)
===================================================================
include/asm-i386/pda.h +18 -0
@@ -16,8 +16,26 @@ struct i386_pdaint cpu_number;struct task_struct *pcurrent;/* current process */ struct pt_regs *irq_regs; + +#ifdef CONFIG_PARAVIRT +union { +#ifdef CONFIG_XEN +struct { +struct vcpu_info *vcpu; +unsigned long cr3; + } xen; +#endif /* CONFIG_XEN */ + }; +#endif /* CONFIG_PARAVIRT */ }; +#ifndef CONFIG_PARAVIRT +static inline void paravirt_init_pda (struct i386_pda *pda,int cpu) +{ +} +#endif + +extern struct i386_pda boot_pda;extern struct i386_pda *_cpu_pda[];#define cpu_pda (i) (_cpu_pda[i])
===================================================================
include/asm-i386/xen/hypercall.h +18 -0
@@ -410,4 +410,22 @@ MULTI_mmuext_op(struct multicall_entry * mcl->args[2 ] = (unsigned long )success_count; mcl->args[3] = domid; } + +static inline void +MULTI_set_gdt (struct multicall_entry *mcl,unsigned long *frames,int entries) +{ + mcl->op = __HYPERVISOR_set_gdt; + mcl->args[0 ] = (unsigned long )frames; + mcl->args[1 ] = entries; +} + +static inline void +MULTI_stack_switch (struct multicall_entry *mcl, +unsigned long ss,unsigned long esp) +{ + mcl->op = __HYPERVISOR_stack_switch; + mcl->args[0 ] = ss; + mcl->args[1 ] = esp; +} +#endif /* __HYPERCALL_H__ */
===================================================================
include/xen/events.h new file +28 -0
@@ -0,0 +1,28 @@ +#ifndef _XEN_EVENTS_H +#define _XEN_EVENTS_H + +#include <linux/irq.h> + +int bind_evtchn_to_irqhandler (unsigned int evtchn, +irqreturn_t (*handler)(int ,void *), +unsigned long irqflags,const char *devname, +void *dev_id); +int bind_virq_to_irqhandler (unsigned int virq,unsigned int cpu, +irqreturn_t (*handler)(int ,void *), +unsigned long irqflags,const char *devname,void *dev_id); + +/* +* Common unbind function for all event sources. Takes IRQ to unbind from. +* Automatically closes the underlying event channel (even for bindings +* made with bind_evtchn_to_irqhandler()). +*/ +void unbind_from_irqhandler (unsigned int irq,void *dev_id); + +static inline void notify_remote_via_evtchn (int port) +{ +struct evtchn_send send = { .port = port }; + (void )HYPERVISOR_event_channel_op (EVTCHNOP_send, &send); +} + +extern void notify_remote_via_irq (int irq); +#endif /* _XEN_EVENTS_H */
===================================================================
include/xen/features.h new file +26 -0
@@ -0,0 +1,26 @@ +/****************************************************************************** +* features.h +* +* Query the features reported by Xen. +* +* Copyright (c) 2006, Ian Campbell +*/ + +#ifndef __XEN_FEATURES_H__ +#define __XEN_FEATURES_H__ + +#include <xen/interface/features.h> + +void xen_setup_features (void ); + +extern u8 xen_features[XENFEAT_NR_SUBMAPS *32 ]; + +static inline int xen_feature (int flag) +{ +switch (flag) { + } + +return xen_features[flag]; +} + +#endif /* __ASM_XEN_FEATURES_H__ */
===================================================================
include/xen/page.h new file +175 -0
94 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 9 lines · Jeremy Fitzhardinge
Core Xen Implementation
This patch is a rollup of all the core pieces of the Xen implementation, including booting, memory management, interrupts, time and so on.
arch/i386/kernel/head.S unhandled_paravirt: · line 540
Quoted 6 lines
arch/i386/kernel/head.S +3
-0@@ -535,6 +535,10 @@ unhandled_paravirt: ud2 #endif +#ifdef CONFIG_XEN +#include "../xen/xen-head.S" +#endif
i'd suggest to remove the #ifdef and push it into xen-head.S.
arch/i386/kernel/head.S ENTRY(_stext) · line 551
Quoted 5 lines
arch/i386/kernel/head.S +1
-1/* * BSS section */ -.section ".bss.page_aligned","w" +.section ".bss.page_aligned","wa"
why is this done?
arch/i386/kernel/head.S ENTRY(_stext) · line 554
Quoted 3 lines
ENTRY(swapper_pg_dir)arch/i386/kernel/head.S +1
-0+ .align PAGE_SIZE_asm .fill 1024,4,0
does the native kernel lose memory here?
arch/i386/kernel/head.S static unsigned long native_store_tr(voi · line 443
Quoted 11 lines
arch/i386/kernel/head.S +3
-3@@ -437,9 +437,9 @@ static unsigned long native_store_tr(voi static void native_load_tls(struct thread_struct *t, unsigned int cpu) { -#define C(i) get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + i] = t->tls_array[i] - C(0); C(1); C(2); -#undef C + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 0] = t->tls_array[0]; + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 1] = t->tls_array[1]; + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 2] = t->tls_array[2]; }
this is a cleanup unrelated to the purpose of the patch.
arch/i386/kernel/head.S static unsigned long native_store_tr(voi
Quoted 8 lines
arch/i386/kernel/head.S +8
-0+static void xen_safe_halt(void) +{ + stop_hz_timer(); + /* Blocking includes an implicit local_irq_enable(). */ + if (HYPERVISOR_sched_op(SCHEDOP_block, 0) != 0) + BUG(); + start_hz_timer(); +}
change this to tick_nohz_stop_sched_tick()/restart_sched_tick() instead.
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +6
-0+static void xen_halt(void) +{ +#if 0 + if (irqs_disabled()) + HYPERVISOR_vcpu_op(VCPUOP_down, smp_processor_id(), NULL); +#endif
hm?
arch/i386/kernel/head.S
Quoted 1 line
arch/i386/kernel/head.S +1
-0+ op->arg1.linear_addr = arbitrary_virt_to_machine((unsigned long)addr).maddr;
80+ chars line. (there are more instances of this throughout the patch)
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +6
-0+ for (va = dtr->address, f = 0; + va < dtr->address + size; + va += PAGE_SIZE, f++) { + frames[f] = virt_to_mfn(va); + make_lowmem_page_readonly((void *)va); + }
coding style.
arch/i386/kernel/head.S
Quoted 2 lines
arch/i386/kernel/head.S +2
-0+ /* This is used very early, so we can't rely on per-cpu data + being set up, so no multicalls */
comment coding style. (there are instances of this throughout the patch)
arch/i386/kernel/head.S
Quoted 2 lines
arch/i386/kernel/head.S +2
-0+ spin_lock(&lock); + for(in = out = 0; in < count; in++) {
missing whitespace.
arch/i386/kernel/head.S
Quoted 1 line
arch/i386/kernel/head.S +1
-0+ if (HYPERVISOR_update_descriptor(virt_to_machine(dt + entry*8).maddr,
80+ chars.
arch/i386/kernel/head.S
Quoted 9 lines
arch/i386/kernel/head.S +9
-0+static void xen_set_iopl_mask(unsigned mask) +{ +#if 0 + struct physdev_set_iopl set_iopl; + + /* Force the change at ring 0. */ + set_iopl.iopl = (mask == 0) ? 1 : (mask >> 12) & 3; + HYPERVISOR_physdev_op(PHYSDEVOP_set_iopl, &set_iopl); +#endif
hm?
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +6
-0+ } + + write_pda(xen.cr3, cr3); + + + {
unnecessary newline.
arch/i386/kernel/head.S
Quoted 8 lines
arch/i386/kernel/head.S +8
-0+static struct irq_chip xen_dynamic_chip __read_mostly = { + .name = "xen-virq", + .mask = disable_dynirq, + .unmask = enable_dynirq, + .ack = ack_dynirq, + .set_affinity = set_affinity_irq, + .retrigger = retrigger_dynirq, +};
nicely done! :-)
Quoted 4 lines
arch/i386/kernel/head.S +4
-0+void xen_set_pte(pte_t *ptep, pte_t pte) +{ +#if 1 + struct mmu_update u;
arch/i386/kernel/head.S
Quoted 10 lines
arch/i386/kernel/head.S +10
-0+ + u.ptr = virt_to_machine(ptep).maddr; + u.val = pte_val_ma(pte); + if (HYPERVISOR_mmu_update(&u, 1, NULL, DOMID_SELF) < 0) + BUG(); +#else + ptep->pte_high = pte.pte_high; + smp_wmb(); + ptep->pte_low = pte.pte_low; +#endif
hm?
arch/i386/kernel/head.S
Quoted 4 lines
arch/i386/kernel/head.S +4
-0+ pgd = swapper_pg_dir + pgd_index(vaddr); + if (pgd_none(*pgd)) { + BUG(); + return;
hm?
arch/i386/kernel/head.S
Quoted 9 lines
arch/i386/kernel/head.S +9
-0+void xen_pte_clear(struct mm_struct *mm, unsigned long addr,pte_t *ptep) +{ +#if 1 + ptep->pte_low = 0; + smp_wmb(); + ptep->pte_high = 0; +#else + set_64bit((u64 *)ptep, 0); +#endif
hm?
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +5
-0+unsigned long xen_pmd_val(pmd_t pmd) +{ + BUG(); + return 0; +}
make it noret.
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +5
-0+pmd_t xen_make_pmd(unsigned long pmd) +{ + BUG(); + return __pmd(0); +}
ditto.
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +4
-0+static void xen_timer_interrupt_hook(void) +{ + runstate->time[RUNSTATE_offline] - + __get_cpu_var(processed_stolen_time);
NACK for now, for the reasons outlined in the 'stolen time' thread. Stolen time accounting is a concept only related to the scheduler tick, it's not a concept that should leak into normal timer interrupt concepts.
arch/i386/kernel/head.S
Quoted 8 lines
arch/i386/kernel/head.S +8
-0+ /* System-wide jiffy work. */ + ticks = 0; + while(delta > NS_PER_TICK) { + delta -= NS_PER_TICK; + processed_system_time += NS_PER_TICK; + ticks++; + } + do_timer(ticks);
that should be updated to clockevents. (i suspect it already is?)
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +5
-0+ +#if 0 + if (unlikely((mfn >> machine_to_phys_order) != 0)) + return max_mapnr; +#endif
hm?
Ingo
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 15 lines · Ingo Molnar
Quoted 12 lines · Jeremy Fitzhardinge
Core Xen Implementation
This patch is a rollup of all the core pieces of the Xen implementation, including booting, memory management, interrupts, time and so on.
arch/i386/kernel/head.S +0
-0@@ -535,6 +535,10 @@ unhandled_paravirt: ud2 #endif
Quoted 5 lines
arch/i386/kernel/head.S unhandled_paravirt: · line 540
Quoted 3 lines
arch/i386/kernel/head.S +3
-0+#ifdef CONFIG_XEN +#include "../xen/xen-head.S" +#endifi'd suggest to remove the #ifdef and push it into
xen-head.S.
Even better would be an explanation of why this needs to be included at all. Life would be much simpler if this was simply a totally separate file.
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
In reply to Ingo MolnarQuoted 14 lines · * Ingo Molnar ([email protected]) wrote:
Quoted 12 lines · Jeremy Fitzhardinge
Core Xen Implementation
This patch is a rollup of all the core pieces of the Xen implementation, including booting, memory management, interrupts, time and so on.
arch/i386/kernel/head.S +0
-0@@ -535,6 +535,10 @@ unhandled_paravirt: ud2 #endif
Quoted 5 lines
arch/i386/kernel/head.S unhandled_paravirt: · line 540
Quoted 3 lines
arch/i386/kernel/head.S +3
-0+#ifdef CONFIG_XEN +#include "../xen/xen-head.S" +#endifi'd suggest to remove the #ifdef and push it into
xen-head.S.
That's been fixed, the two are built as seperate objects now.
Quoted 13 lines
arch/i386/kernel/head.S static unsigned long native_store_tr(voi · line 443
Quoted 11 lines
arch/i386/kernel/head.S +3
-3@@ -437,9 +437,9 @@ static unsigned long native_store_tr(voi static void native_load_tls(struct thread_struct *t, unsigned int cpu) { -#define C(i) get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + i] = t->tls_array[i] - C(0); C(1); C(2); -#undef C + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 0] = t->tls_array[0]; + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 1] = t->tls_array[1]; + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 2] = t->tls_array[2]; }this is a cleanup unrelated to the purpose of the patch.
Sure, will split out.
Quoted 10 lines
arch/i386/kernel/head.S static unsigned long native_store_tr(voi
Quoted 8 lines
arch/i386/kernel/head.S +8
-0+static void xen_safe_halt(void) +{ + stop_hz_timer(); + /* Blocking includes an implicit local_irq_enable(). */ + if (HYPERVISOR_sched_op(SCHEDOP_block, 0) != 0) + BUG(); + start_hz_timer(); +}change this to tick_nohz_stop_sched_tick()/restart_sched_tick() instead.
I don't think we want to do that. It's dropped now with proper NO_HZ support, as the core does that for us.
Quoted 8 lines
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +6
-0+static void xen_halt(void) +{ +#if 0 + if (irqs_disabled()) + HYPERVISOR_vcpu_op(VCPUOP_down, smp_processor_id(), NULL); +#endifhm?
That's already been fixed.
Quoted 18 lines
arch/i386/kernel/head.S
Quoted 1 line
arch/i386/kernel/head.S +1
-0+ op->arg1.linear_addr = arbitrary_virt_to_machine((unsigned long)addr).maddr;80+ chars line. (there are more instances of this throughout the patch)
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +6
-0+ for (va = dtr->address, f = 0; + va < dtr->address + size; + va += PAGE_SIZE, f++) { + frames[f] = virt_to_mfn(va); + make_lowmem_page_readonly((void *)va); + }coding style.
arch/i386/kernel/head.S
Quoted 2 lines
arch/i386/kernel/head.S +2
-0+ /* This is used very early, so we can't rely on per-cpu data + being set up, so no multicalls */comment coding style. (there are instances of this throughout the patch)
Quoted 8 lines
arch/i386/kernel/head.S
Quoted 2 lines
arch/i386/kernel/head.S +2
-0+ spin_lock(&lock); + for(in = out = 0; in < count; in++) {missing whitespace.
arch/i386/kernel/head.S
Quoted 1 line
arch/i386/kernel/head.S +1
-0+ if (HYPERVISOR_update_descriptor(virt_to_machine(dt + entry*8).maddr,80+ chars.
cleaning those
Quoted 12 lines
arch/i386/kernel/head.S
Quoted 9 lines
arch/i386/kernel/head.S +9
-0+static void xen_set_iopl_mask(unsigned mask) +{ +#if 0 + struct physdev_set_iopl set_iopl; + + /* Force the change at ring 0. */ + set_iopl.iopl = (mask == 0) ? 1 : (mask >> 12) & 3; + HYPERVISOR_physdev_op(PHYSDEVOP_set_iopl, &set_iopl); +#endifhm?
Quoted 8 lines
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +6
-0+ } + + write_pda(xen.cr3, cr3); + + + {unnecessary newline.
dropped
Quoted 11 lines
arch/i386/kernel/head.S
Quoted 8 lines
arch/i386/kernel/head.S +8
-0+static struct irq_chip xen_dynamic_chip __read_mostly = { + .name = "xen-virq", + .mask = disable_dynirq, + .unmask = enable_dynirq, + .ack = ack_dynirq, + .set_affinity = set_affinity_irq, + .retrigger = retrigger_dynirq, +};nicely done! :-)
Quoted 16 lines
Quoted 4 lines
arch/i386/kernel/head.S +4
-0+void xen_set_pte(pte_t *ptep, pte_t pte) +{ +#if 1 + struct mmu_update u;arch/i386/kernel/head.S
Quoted 10 lines
arch/i386/kernel/head.S +10
-0+ + u.ptr = virt_to_machine(ptep).maddr; + u.val = pte_val_ma(pte); + if (HYPERVISOR_mmu_update(&u, 1, NULL, DOMID_SELF) < 0) + BUG(); +#else + ptep->pte_high = pte.pte_high; + smp_wmb(); + ptep->pte_low = pte.pte_low; +#endifhm?
that's already been fixed
Quoted 6 lines
arch/i386/kernel/head.S
Quoted 4 lines
arch/i386/kernel/head.S +4
-0+ pgd = swapper_pg_dir + pgd_index(vaddr); + if (pgd_none(*pgd)) { + BUG(); + return;hm?
that's already been fixed
Quoted 11 lines
arch/i386/kernel/head.S
Quoted 9 lines
arch/i386/kernel/head.S +9
-0+void xen_pte_clear(struct mm_struct *mm, unsigned long addr,pte_t *ptep) +{ +#if 1 + ptep->pte_low = 0; + smp_wmb(); + ptep->pte_high = 0; +#else + set_64bit((u64 *)ptep, 0); +#endifhm?
that's already been fixed
Quoted 7 lines
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +5
-0+unsigned long xen_pmd_val(pmd_t pmd) +{ + BUG(); + return 0; +}make it noret.
It's actually used now, so it's not noret-able (if that's a word ;-)
Quoted 8 lines
arch/i386/kernel/head.S
Quoted 6 lines
arch/i386/kernel/head.S +5
-0+pmd_t xen_make_pmd(unsigned long pmd) +{ + BUG(); + return __pmd(0); +}ditto.
ditto
Quoted 10 lines
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +4
-0+static void xen_timer_interrupt_hook(void) +{ + runstate->time[RUNSTATE_offline] - + __get_cpu_var(processed_stolen_time);NACK for now, for the reasons outlined in the 'stolen time' thread. Stolen time accounting is a concept only related to the scheduler tick, it's not a concept that should leak into normal timer interrupt concepts.
As Jeremey mentioned in that thread, it's just used as a place to tigger the caluculations periodically.
Quoted 10 lines
arch/i386/kernel/head.S
Quoted 8 lines
arch/i386/kernel/head.S +8
-0+ /* System-wide jiffy work. */ + ticks = 0; + while(delta > NS_PER_TICK) { + delta -= NS_PER_TICK; + processed_system_time += NS_PER_TICK; + ticks++; + } + do_timer(ticks);that should be updated to clockevents. (i suspect it already is?)
Yes it is.
Quoted 7 lines
arch/i386/kernel/head.S
Quoted 5 lines
arch/i386/kernel/head.S +5
-0+ +#if 0 + if (unlikely((mfn >> machine_to_phys_order) != 0)) + return max_mapnr; +#endifhm?
Good point, I don't recall the condition where that would get triggered.
thanks,
-chris
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 17 lines · Chris Wright
Quoted 15 lines · * Ingo Molnar ([email protected]) wrote:
Quoted 12 lines · Jeremy Fitzhardinge
Core Xen Implementation
This patch is a rollup of all the core pieces of the Xen implementation, including booting, memory management, interrupts, time and so on.
arch/i386/kernel/head.S +0
-0@@ -535,6 +535,10 @@ unhandled_paravirt: ud2 #endif
Quoted 9 lines
Quoted 6 lines
arch/i386/kernel/head.S unhandled_paravirt:
Quoted 4 lines
arch/i386/kernel/head.S +3
-0+#ifdef CONFIG_XEN +#include "../xen/xen-head.S" +#endifi'd suggest to remove the #ifdef and push it into
xen-head.S.That's been fixed, the two are built as seperate objects now.
Actually, we tried it but it causes bad kernel images with some binutils, so it has to be included for now.
Quoted 15 lines
Quoted 14 lines
arch/i386/kernel/head.S static unsigned long native_store_tr(voi · line 444
Quoted 12 lines
arch/i386/kernel/head.S +3
-3@@ -437,9 +437,9 @@ static unsigned long native_store_tr(voi static void native_load_tls(struct thread_struct *t, unsigned int cpu) { -#define C(i) get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + i] = t->tls_array[i] - C(0); C(1); C(2); -#undef C + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 0] = t->tls_array[0]; + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 1] = t->tls_array[1]; + get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + 2] = t->tls_array[2]; }this is a cleanup unrelated to the purpose of the patch.
Quoted 2 lines
Sure, will split out.
Already done in Rusty's cleanup patch.
J
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 5 lines · * Jeremy Fitzhardinge ([email protected]) wrote:
Quoted 1 line · Chris Wright
That's been fixed, the two are built as seperate objects now.
Actually, we tried it but it causes bad kernel images with some binutils, so it has to be included for now.
Ah, missed that.
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
arch/i386/kernel/paravirt.c static unsigned long native_store_tr(voi · line 444
Quoted 12 lines
arch/i386/kernel/paravirt.c +3
-3@@ -437,9 +437,9 @@ static unsigned long native_store_tr(voistatic void native_load_tls (struct thread_struct *t,unsigned int cpu) { -#define C (i) get_cpu_gdt_table(cpu)[GDT_ENTRY_TLS_MIN + i] = t->tls_array[i] -C (0 );C (1 );C (2 ); -#undef C +get_cpu_gdt_table (cpu)[GDT_ENTRY_TLS_MIN +0 ] = t->tls_array[0 ]; +get_cpu_gdt_table (cpu)[GDT_ENTRY_TLS_MIN +1 ] = t->tls_array[1 ]; +get_cpu_gdt_table (cpu)[GDT_ENTRY_TLS_MIN +2 ] = t->tls_array[2 ]; }this is a cleanup unrelated to the purpose of the patch.
Quoted 4 lines
Quoted 2 lines
Sure, will split out.
Already done in Rusty's cleanup patch.
Hmm, not here, I'll fix it up.
thanks,
-chris
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
In reply to Ingo MolnarQuoted 5 lines · * Ingo Molnar ([email protected]) wrote:
arch/i386/kernel/head.S ENTRY(_stext) · line 554
Quoted 3 lines
ENTRY(swapper_pg_dir)arch/i386/kernel/head.S +1
-0+ .align PAGE_SIZE_asm .fill 1024,4,0does the native kernel lose memory here?
Not in my builds.
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 8 lines · Chris Wright
Quoted 5 lines · * Ingo Molnar ([email protected]) wrote:
arch/i386/kernel/head.S ENTRY(_stext) · line 554
Quoted 3 lines
ENTRY(swapper_pg_dir)arch/i386/kernel/head.S +1
-0+ .align PAGE_SIZE_asm .fill 1024,4,0does the native kernel lose memory here?
Not in my builds.
Shouldn't the align be before the label. Otherwise padding would be inserted between the label and the data.
Eric
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 7 lines · Eric W. Biederman
Quoted 5 lines
Quoted 5 lines
arch/i386/kernel/head.S ENTRY(_stext) · line 554
Quoted 4 lines
ENTRY(swapper_pg_dir)arch/i386/kernel/head.S +1
-0+ .align PAGE_SIZE_asm .fill 1024,4,0Shouldn't the align be before the label. Otherwise padding would be inserted between the label and the data.
Good point.
Thanks,
J
Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
In reply to Eric W. BiedermanQuoted 9 lines · * Eric W. Biederman ([email protected]) wrote:
Quoted 7 lines · Chris Wright
Quoted 5 lines · * Ingo Molnar ([email protected]) wrote:
arch/i386/kernel/head.S ENTRY(_stext) · line 554
Quoted 3 lines
ENTRY(swapper_pg_dir)arch/i386/kernel/head.S +1
-0+ .align PAGE_SIZE_asm .fill 1024,4,0does the native kernel lose memory here?
Quoted 4 lines
Quoted 1 line
Not in my builds.
Shouldn't the align be before the label. Otherwise padding would be inserted between the label and the data.
Yes, I think you're right. Thanks
-chris
Re: [Xen-devel] Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
In reply to Ingo MolnarQuoted 7 lines · Ingo Molnar
arch/i386/xen/mmu.c line 244
Quoted 5 lines
arch/i386/xen/mmu.c new file +5
-0+unsigned long xen_pmd_val (pmd_t pmd) +{ +BUG (); +return 0 ; +}make it noret.
OK, I missed this one. How?
Wondering if I've missed a trick here...
Rusty.
Re: [Xen-devel] Re: [patch 20/26] Xen-paravirt_ops: Core Xen implementation
Quoted 11 lines · Rusty Russell
Quoted 8 lines · Ingo Molnar
arch/i386/xen/mmu.c line 244
Quoted 6 lines
arch/i386/xen/mmu.c new file +5
-0+unsigned long xen_pmd_val (pmd_t pmd) +{ +BUG (); +return 0 ; +}make it noret.
Quoted 3 lines
OK, I missed this one. How?
Wondering if I've missed a trick here...
No, I don't think its terribly useful to make it noret; noret is an interface annotation to let the caller know the function won't return. If I mark this noret, then the compiler will complain that it does appear to return.
J
PATCH 21/26 Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Jeremy FitzhardingeImplement a Xen back-end for hvc console.
From: Gerd Hoffmann <[email protected]>
--- arch/i386/xen/Kconfig | 1 arch/i386/xen/events.c | 3 - drivers/Makefile | 3 + drivers/xen/Makefile | 1 drivers/xen/hvc-console.c | 134 +++++++++++++++++++++++++++++++++++++++++++++ include/xen/events.h | 1 6 files changed, 142 insertions(+), 1 deletion(-)
===================================================================
arch/i386/xen/Kconfig +1 -0
@@ -5,6 +5,7 @@ config XEN config XEN bool "Enable support for Xen hypervisor" depends on PARAVIRT && HZ_100 && !PREEMPT && !NO_HZ + select HVC_DRIVER default y help This is the Linux Xen port.
===================================================================
arch/i386/xen/events.c +2 -1
@@ -209,7 +209,7 @@ static int find_unbound_irq(void)return irq; } -static int bind_evtchn_to_irq (unsigned int evtchn) +int bind_evtchn_to_irq (unsigned int evtchn) {int irq;
@@ -233,6 +233,7 @@ static int bind_evtchn_to_irq(unsigned ireturn irq; } +EXPORT_SYMBOL_GPL (bind_evtchn_to_irq);static int bind_virq_to_irq (unsigned int virq,unsigned int cpu) {
===================================================================
drivers/Makefile +3 -0
@@ -14,6 +14,9 @@ obj-$(CONFIG_ACPI) += acpi/ # was used and do nothing if so obj-$(CONFIG_PNP) += pnp/ obj-$(CONFIG_ARM_AMBA) += amba/ + +# Xen is the default console when running as a guest +obj-$(CONFIG_XEN) += xen/ # char/ comes before serial/ etc so that the VT console is the boot-time # default.
===================================================================
drivers/xen/Makefile new file +1 -0
@@ -0,0 +1,1 @@ +obj-y += hvc-console.o
===================================================================
drivers/xen/hvc-console.c new file +134 -0
134 lines not on this page · raw message
===================================================================
include/xen/events.h +1 -0
@@ -3,6 +3,7 @@#include <linux/irq.h> +int bind_evtchn_to_irq (unsigned int evtchn); int bind_evtchn_to_irqhandler(unsigned int evtchn, irqreturn_t (*handler)(int, void *), unsigned long irqflags, const char *devname,
Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 6 lines · Jeremy Fitzhardinge
Implement a Xen back-end for hvc console.
drivers/xen/hvc-console.c new file +4
-0+ cons = intf->out_cons; + prod = intf->out_prod; +mb (); +BUG_ON ((prod - cons) >sizeof (intf->out));
drivers/xen/hvc-console.c line 82
Quoted 10 lines
drivers/xen/hvc-console.c new file +9
-0+ +while ((sent < len) && ((prod - cons) <sizeof (intf->out))) + intf->out[MASK_XENCONS_IDX (prod++, intf->out)] = data[sent++]; + +wmb (); + intf->out_prod = prod; + prod = intf->in_prod; +mb (); +BUG_ON ((prod - cons) >sizeof (intf->in));
such mb()'s are typically a sign of "i have no clear idea what SMP serialization rules apply here, but something is needed because otherwise it breaks" ?
Ingo
Re: Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 7 lines · Ingo Molnar
Quoted 3 lines
+ prod =
intf->in_prod;
+mb();
+BUG_ON((prod - cons) > sizeof(intf->in));such mb()'s are typically a sign of "i have no clear idea what SMP serialization rules apply here, but something is needed because otherwise it breaks" ?
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
All our ring protocols (and we have a few) require memory barriers. Those that use our standard set of ring macros have the memory barriers hidden from view, but they're still there.
-- Keir
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 6 lines · Keir Fraser
Quoted 4 lines · Ingo Molnar
Quoted 3 lines
+ prod =
intf->in_prod;
+mb();
+BUG_ON((prod - cons) > sizeof(intf->in));
Quoted 8 lines
Quoted 3 lines
such mb()'s are typically a sign of "i have no clear idea what SMP serialization rules apply here, but something is needed because otherwise it breaks" ?
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
then use rmb()/wmb(). Rarely does a ring protocol truly need mb().
Ingo
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 6 lines · Ingo Molnar
Quoted 4 lines
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
then use rmb()/wmb(). Rarely does a ring protocol truly need
mb().
It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through xenbus_comms.c I think all the barriers are correct except there is a spurious extra mb() in xb_read(), where there is a later rmb() which is sufficient by itself. All the others have a purpose.
-- Keir
Andrew Morton
maintainerRe: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 7 lines · On Fri, 16 Mar 2007 10:31:49 +0000 Keir Fraser <[email protected]> wrote:
Quoted 5 lines · Ingo Molnar
Quoted 4 lines
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
Quoted 8 lines
Quoted 1 line
then use rmb()/wmb(). Rarely does a ring protocol truly need
mb().It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through
xenbus_comms.cI think all the barriers are correct except there is a spurious extramb()inxb_read(), where there is a laterrmb()which is sufficient by itself. All the others have a purpose.
If Ingo couldn't work this out from reading the code then nobody else can, and we have a maintainability problem which can only be solved with adequate commenting.
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 10 lines · Andrew Morton
Quoted 6 lines
It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through
xenbus_comms.cI think all the barriers are correct except there is a spurious extramb()inxb_read(), where there is a laterrmb()which is sufficient by itself. All the others have a purpose.If Ingo couldn't work this out from reading the code then nobody else can, and we have a maintainability problem which can only be solved with adequate commenting.
Agreed. I've fixed up the commenting, removed the spurious mb(), and I just need Jeremy to merge that change into the Xen pv_ops patchset. :-)
-- Keir
RE: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
But it is shorter than from JFK or Newark.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Keir Fraser
Sent: Friday, March 16, 2007 6:58 AM
To: Andrew Morton; Keir Fraser
Cc: [email protected]; Andi Kleen; [email protected]; Keir Fraser; Wright; [email protected]; Ingo Molnar; [email protected]
Subject: Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
Quoted 1 line · Andrew Morton
Quoted 1 line
It's needed for writing data /after/ reading the consumer index that
shows
Quoted 1 line
Quoted 1 line
you have space to write. Looking through
xenbus_comms.cI think all
the
Quoted 1 line
Quoted 1 line
barriers are correct except there is a spurious extra
mb()in
xb_read(),
Quoted 1 line
Quoted 1 line
where there is a later
rmb()which is sufficient by itself. All the
others
Quoted 4 lines
Quoted 2 lines
have a purpose.
If Ingo couldn't work this out from reading the code then nobody else
can,
Quoted 2 lines
and we have a maintainability problem which can only be solved with adequate commenting.
Agreed. I've fixed up the commenting, removed the spurious mb(), and I just
need Jeremy to merge that change into the Xen pv_ops patchset. :-)
-- Keir
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Andrew MortonQuoted 10 lines · Andrew Morton
Quoted 6 lines
It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through
xenbus_comms.cI think all the barriers are correct except there is a spurious extramb()inxb_read(), where there is a laterrmb()which is sufficient by itself. All the others have a purpose.If Ingo couldn't work this out from reading the code then nobody else can, and we have a maintainability problem which can only be solved with adequate commenting.
Agreed. I've fixed up the commenting, removed the spurious mb(), and I just need Jeremy to merge that change into the Xen pv_ops patchset. :-)
-- Keir
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Andrew MortonQuoted 10 lines · Andrew Morton
Quoted 6 lines
It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through
xenbus_comms.cI think all the barriers are correct except there is a spurious extramb()inxb_read(), where there is a laterrmb()which is sufficient by itself. All the others have a purpose.If Ingo couldn't work this out from reading the code then nobody else can, and we have a maintainability problem which can only be solved with adequate commenting.
Agreed. I've fixed up the commenting, removed the spurious mb(), and I just need Jeremy to merge that change into the Xen pv_ops patchset. :-)
-- Keir
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Ingo MolnarQuoted 6 lines · Ingo Molnar
Quoted 4 lines
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
then use rmb()/wmb(). Rarely does a ring protocol truly need
mb().
It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through xenbus_comms.c I think all the barriers are correct except there is a spurious extra mb() in xb_read(), where there is a later rmb() which is sufficient by itself. All the others have a purpose.
-- Keir
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Ingo MolnarQuoted 6 lines · Ingo Molnar
Quoted 4 lines
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
then use rmb()/wmb(). Rarely does a ring protocol truly need
mb().
It's needed for writing data /after/ reading the consumer index that shows you have space to write. Looking through xenbus_comms.c I think all the barriers are correct except there is a spurious extra mb() in xb_read(), where there is a later rmb() which is sufficient by itself. All the others have a purpose.
-- Keir
Re: [Xen-devel] Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Ingo MolnarQuoted 7 lines · Ingo Molnar
Quoted 3 lines
+ prod =
intf->in_prod;
+mb();
+BUG_ON((prod - cons) > sizeof(intf->in));such mb()'s are typically a sign of "i have no clear idea what SMP serialization rules apply here, but something is needed because otherwise it breaks" ?
These mb()'s are pretty standard for lock-free producer/consumer rings. Write descriptor /then/ write the updated producer. Read the producer /then/ read any descriptors revealed by this new producer value.
All our ring protocols (and we have a few) require memory barriers. Those that use our standard set of ring macros have the memory barriers hidden from view, but they're still there.
-- Keir
Re: [patch 21/26] Xen-paravirt_ops: Use the hvc console infrastructure for Xen console
In reply to Ingo MolnarQuoted 8 lines · Ingo Molnar
drivers/xen/hvc-console.c line 82
Quoted 4 lines
drivers/xen/hvc-console.c new file +3
-0+ prod = intf->in_prod; +mb (); +BUG_ON ((prod - cons) >sizeof (intf->in));such mb()'s are typically a sign of "i have no clear idea what SMP serialization rules apply here, but something is needed because otherwise it breaks" ?
Hm, in this case its because it's sharing the memory with Xen, so there's a particular ordering protocol. It needs some comments.
J
PATCH 22/26 Xen-paravirt_ops: Add early printk support via hvc console
In reply to Jeremy FitzhardingeAdd early printk support via hvc console, enable using "earlyprintk=xen" on the kernel command line.
From: Gerd Hoffmann <[email protected]>
--- arch/x86_64/kernel/early_printk.c | 5 +++++ drivers/xen/hvc-console.c | 25 +++++++++++++++++++++++++ include/xen/hvc-console.h | 6 ++++++ 3 files changed, 36 insertions(+)
===================================================================
arch/x86_64/kernel/early_printk.c +5 -0
@@ -6,6 +6,7 @@#include <asm/io.h> #include <asm/processor.h> #include <asm/fcntl.h> +#include <xen/hvc-console.h> /* Simple VGA output */
@@ -243,6 +244,10 @@ static int __init setup_early_printk(chasimnow_init (buf +6 ); early_console = &simnow_console; keep_early =1 ; +#ifdef CONFIG_XEN + }else if (!strncmp (buf,"xen" ,3 )) { + early_console = &xenboot_console; +#endif }register_console (early_console);return 0 ;
===================================================================
drivers/xen/hvc-console.c +25 -0
@@ -28,6 +28,7 @@#include <xen/page.h> #include <xen/events.h> #include <xen/interface/io/console.h> +#include <xen/hvc-console.h> #include "../char/hvc_console.h"
@@ -132,3 +133,27 @@ module_init(xen_init);module_init (xen_init);module_exit (xen_fini);console_initcall (xen_cons_init); + +static void xenboot_write_console (struct console *console,const char *string, +unsigned len) +{ +unsigned int linelen, off =0 ; +const char *pos; + +while (off < len &&NULL != (pos =strchr (string+off,'\n' ))) { + linelen = pos-string+off; +if (off + linelen > len) +break ; +write_console (0 , string+off, linelen); +write_console (0 ,"\r\n" ,2 ); + off += linelen +1 ; + } +if (off < len) +write_console (0 , string+off, len-off); +} + +struct console xenboot_console = { + .name ="xenboot" , + .write = xenboot_write_console, + .flags =CON_PRINTBUFFER |CON_BOOT , +};
===================================================================
include/xen/hvc-console.h new file +6 -0
@@ -0,0 +1,6 @@ +#ifndef XEN_HVC_CONSOLE_H +#define XEN_HVC_CONSOLE_H + +extern struct console xenboot_console; + +#endif /* XEN_HVC_CONSOLE_H */
Re: [patch 22/26] Xen-paravirt_ops: Add early printk support via hvc console
Quoted 5 lines · Jeremy Fitzhardinge
Add early printk support via hvc console, enable using "earlyprintk=xen" on the kernel command line.
From: Gerd Hoffmann <[email protected]>
Signed-off-by Jeremy Fitzhardinge <[email protected]>
Ingo
PATCH 23/26 Xen-paravirt_ops: Add Xen grant table support
In reply to Jeremy FitzhardingeAdd Xen 'grant table' driver which allows granting of access to selected local memory pages by other virtual machines and, symmetrically, the mapping of remote memory pages which other virtual machines have granted access to.
This driver is a prerequisite for many of the Xen virtual device drivers, which grant the 'device driver domain' restricted and temporary access to only those memory pages that are currently involved in I/O operations.
--- drivers/xen/Makefile | 1 drivers/xen/grant-table.c | 454 +++++++++++++++++++++++++++++++++++++++++++++ include/xen/grant_table.h | 107 ++++++++++ 3 files changed, 562 insertions(+)
===================================================================
drivers/xen/Makefile +1 -0
@@ -1,1 +1,2 @@ obj-y += hvc-console.o +obj-y += grant-table.o obj-y += hvc-console.o
===================================================================
drivers/xen/grant-table.c new file +454 -0
454 lines not on this page · raw message
===================================================================
include/xen/grant_table.h new file +107 -0
107 lines not on this page · raw message
Re: [patch 23/26] Xen-paravirt_ops: Add Xen grant table support
drivers/xen/grant-table.c line 366
Quoted 13 lines · Jeremy Fitzhardinge
Add Xen 'grant table' driver which allows granting of access to selected local memory pages by other virtual machines and, symmetrically, the mapping of remote memory pages which other virtual machines have granted access to.
This driver is a prerequisite for many of the Xen virtual device drivers, which grant the 'device driver domain' restricted and temporary access to only those memory pages that are currently involved in I/O operations.
drivers/xen/grant-table.c new file +3
-0+ +#ifndef __ia64__ + {
introduce a proper arch method instead.
Quoted 4 lines
(quoted diff) +4
-0+ unsigned long flags; + int ref; + grant_ref_t head; + spin_lock_irqsave(&gnttab_list_lock, flags);
Review
Quoted 10 lines
(quoted diff) +7
-0+ unsigned long flags; + spin_lock_irqsave(&gnttab_list_lock, flags); + unsigned long flags; + int count = 1; + if (head == GNTTAB_LIST_END) + grant_ref_t g = *private_head; + if (unlikely(g == GNTTAB_LIST_END))
coding style problems.
Ingo
Re: [patch 23/26] Xen-paravirt_ops: Add Xen grant table support
Quoted 7 lines · Ingo Molnar
drivers/xen/grant-table.c line 366
Quoted 4 lines
drivers/xen/grant-table.c new file +3
-0+ +#ifndef __ia64__ + {introduce a proper arch method instead.
I'll clean out the ia64 stuff. We can add it back properly when (if?) the time comes.
Quoted 5 lines
drivers/xen/grant-table.c line 314
Quoted 3 lines
drivers/xen/grant-table.c new file +2
-0+grant_ref_t g = *private_head; +if (unlikely (g ==GNTTAB_LIST_END ))coding style problems.
Yup.
PATCH 24/26 Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
In reply to Jeremy FitzhardingeThis communicates with the machine control software via a registry residing in a controlling virtual machine. This allows dynamic creation, destruction and modification of virtual device configurations (network devices, block devices and CPUS, to name some examples).
--- arch/i386/xen/Kconfig | 5 drivers/xen/Makefile | 1 drivers/xen/xenbus/Makefile | 9 drivers/xen/xenbus/xenbus_client.c | 563 ++++++++++++++++++ drivers/xen/xenbus/xenbus_comms.c | 218 +++++++ drivers/xen/xenbus/xenbus_comms.h | 44 + drivers/xen/xenbus/xenbus_probe.c | 891 +++++++++++++++++++++++++++++ drivers/xen/xenbus/xenbus_probe.h | 72 ++ drivers/xen/xenbus/xenbus_probe_backend.c | 269 ++++++++ drivers/xen/xenbus/xenbus_xs.c | 840 +++++++++++++++++++++++++++ include/asm-i386/xen/hypervisor.h | 1 include/xen/xenbus.h | 208 ++++++ 12 files changed, 3121 insertions(+)
===================================================================
arch/i386/xen/Kconfig +5 -0
@@ -9,3 +9,8 @@ config XEN default y help This is the Linux Xen port. + +config XEN_BACKEND + bool + depends XEN + default n
===================================================================
drivers/xen/Makefile +1 -0
@@ -1,2 +1,3 @@ obj-y += grant-table.o obj-y += grant-table.o obj-y += hvc-console.o +obj-y += xenbus/
===================================================================
drivers/xen/xenbus/Makefile new file +9 -0
@@ -0,0 +1,9 @@ +obj-y += xenbus.o + +xenbus-objs = +xenbus-objs += xenbus_client.o +xenbus-objs += xenbus_comms.o +xenbus-objs += xenbus_xs.o +xenbus-objs += xenbus_probe.o + +obj-$(CONFIG_XEN_BACKEND) += xenbus_probe_backend.o
===================================================================
drivers/xen/xenbus/xenbus_client.c new file +563 -0
563 lines not on this page · raw message
===================================================================
drivers/xen/xenbus/xenbus_comms.c new file +218 -0
218 lines not on this page · raw message
===================================================================
drivers/xen/xenbus/xenbus_comms.h new file +44 -0
44 lines not on this page · raw message
===================================================================
drivers/xen/xenbus/xenbus_probe.c new file +891 -0
891 lines not on this page · raw message
===================================================================
drivers/xen/xenbus/xenbus_probe.h new file +72 -0
72 lines not on this page · raw message
===================================================================
drivers/xen/xenbus/xenbus_probe_backend.c new file +269 -0
269 lines not on this page · raw message
===================================================================
drivers/xen/xenbus/xenbus_xs.c new file +840 -0
840 lines not on this page · raw message
===================================================================
include/asm-i386/xen/hypervisor.h +1 -0
@@ -42,6 +42,7 @@#include <asm/ptrace.h> #include <asm/page.h> +#include <asm/desc.h> #if defined(__i386__)# ifdef CONFIG_X86_PAE # include <asm-generic/pgtable-nopud.h>
===================================================================
include/xen/xenbus.h new file +208 -0
76 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Re: [patch 24/26] Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
Quoted 5 lines · Jeremy Fitzhardinge
This communicates with the machine control software via a registry residing in a controlling virtual machine. This allows dynamic creation, destruction and modification of virtual device configurations (network devices, block devices and CPUS, to name some examples).
should be reviewed by the driver-core folks too i guess?
drivers/xen/xenbus/xenbus_probe_backend.c
Quoted 8 lines
drivers/xen/xenbus/xenbus_probe_backend.c new file +5
-0+//#include <asm/maddr.h> +//#include <xen/xen_proc.h> +//#include <xen/evtchn.h> +//#include <xen/hvm.h> +// .shutdown = xenbus_dev_shutdown,
hm?
Ingo
Re: [patch 24/26] Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
Quoted 10 lines · * Ingo Molnar ([email protected]) wrote:
Quoted 5 lines · Jeremy Fitzhardinge
This communicates with the machine control software via a registry residing in a controlling virtual machine. This allows dynamic creation, destruction and modification of virtual device configurations (network devices, block devices and CPUS, to name some examples).
should be reviewed by the driver-core folks too i guess?
Greg has gone through it a couple times. Any more feedback would be useful.
Quoted 10 lines
drivers/xen/xenbus/xenbus_probe_backend.c
Quoted 8 lines
drivers/xen/xenbus/xenbus_probe_backend.c new file +5
-0+//#include <asm/maddr.h> +//#include <xen/xen_proc.h> +//#include <xen/evtchn.h> +//#include <xen/hvm.h> +// .shutdown = xenbus_dev_shutdown, hm?
fixing that up, not sure where that snuck in.
Re: [patch 24/26] Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
Quoted 10 lines · Chris Wright
Quoted 9 lines · * Ingo Molnar ([email protected]) wrote:
Quoted 5 lines · Jeremy Fitzhardinge
This communicates with the machine control software via a registry residing in a controlling virtual machine. This allows dynamic creation, destruction and modification of virtual device configurations (network devices, block devices and CPUS, to name some examples).
Quoted 4 lines
Quoted 1 line
should be reviewed by the driver-core folks too i guess?
Greg has gone through it a couple times. Any more feedback would be useful.
please indicate any such review feedback (and approval) via signed-off-by lines done by that person.
Ingo
Re: [patch 24/26] Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
Quoted 2 lines · * Ingo Molnar ([email protected]) wrote:
please indicate any such review feedback (and approval) via signed-off-by lines done by that person.
Certainly, Greg didn't sign off, he simply gave review feedback.
thanks,
-chris
Re: [patch 24/26] Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
Quoted 5 lines · Chris Wright
Quoted 2 lines · * Ingo Molnar ([email protected]) wrote:
please indicate any such review feedback (and approval) via signed-off-by lines done by that person.
Certainly, Greg didn't sign off, he simply gave review feedback.
Yeah, I did that a while ago, but I don't want to give my acked-by or signed-off for this code. While I can not find anything specfically wrong with it, it seems a bit "odd", but that might just be because I really don't understand what they are doing here, or why.
And no, I really don't want to understand it either at the moment due to being busy with other things :)
thanks,
greg k-h
Re: [patch 24/26] Xen-paravirt_ops: Add the Xenbus sysfs and virtual device hotplug driver.
In reply to Ingo MolnarQuoted 22 lines · Ingo Molnar
Quoted 6 lines · Jeremy Fitzhardinge
This communicates with the machine control software via a registry residing in a controlling virtual machine. This allows dynamic creation, destruction and modification of virtual device configurations (network devices, block devices and CPUS, to name some examples).
should be reviewed by the driver-core folks too i guess?
Quoted 7 lines
drivers/xen/xenbus/xenbus_probe_backend.c new file +3
-0+//#include <asm/maddr.h> +//#include <xen/xen_proc.h> +//#include <xen/evtchn.h>
Quoted 9 lines
Review
Quoted 6 lines
(quoted diff) +2
-0+//#include <xen/hvm.h> +// .shutdown = xenbus_dev_shutdown,hm?
Yeah, gunk. I'll clean that out.
J
PATCH 25/26 Xen-paravirt_ops: Add Xen virtual block device driver.
In reply to Jeremy FitzhardingeThe block device frontend driver allows the kernel to access block devices exported exported by a virtual machine containing a physical block device driver.
--- drivers/block/Kconfig | 1 drivers/block/Makefile | 1 drivers/block/xen/Kconfig | 14 drivers/block/xen/Makefile | 5 drivers/block/xen/blkfront.c | 844 ++++++++++++++++++++++++++++++++++++++++++ drivers/block/xen/block.h | 135 ++++++ drivers/block/xen/vbd.c | 230 +++++++++++ include/linux/major.h | 2 8 files changed, 1232 insertions(+)
===================================================================
drivers/block/Kconfig +1 -0
@@ -461,6 +461,7 @@ config CDROM_PKTCDVD_WCACHE don't do deferred write error handling yet. source "drivers/s390/block/Kconfig" +source "drivers/block/xen/Kconfig" config ATA_OVER_ETH tristate "ATA over Ethernet support"
===================================================================
drivers/block/Makefile +1 -0
@@ -29,3 +29,4 @@ obj-$(CONFIG_BLK_DEV_SX8) += sx8.o obj-$(CONFIG_BLK_DEV_SX8) += sx8.o obj-$(CONFIG_BLK_DEV_UB) += ub.o +obj-$(CONFIG_XEN) += xen/
===================================================================
drivers/block/xen/Kconfig new file +14 -0
@@ -0,0 +1,14 @@ +menu "Xen block device drivers" + depends on XEN + +config XEN_BLKDEV_FRONTEND + tristate "Block device frontend driver" + depends on XEN + default y + help + The block device frontend driver allows the kernel to access block + devices exported from a device driver virtual machine. Unless you + are building a dedicated device driver virtual machine, then you + almost certainly want to say Y here. + +endmenu
===================================================================
drivers/block/xen/Makefile new file +5 -0
@@ -0,0 +1,5 @@ + +obj-$(CONFIG_XEN_BLKDEV_FRONTEND) := xenblk.o + +xenblk-objs := blkfront.o vbd.o +
===================================================================
drivers/block/xen/blkfront.c new file +844 -0
844 lines not on this page · raw message
===================================================================
drivers/block/xen/block.h new file +135 -0
135 lines not on this page · raw message
===================================================================
drivers/block/xen/vbd.c new file +230 -0
230 lines not on this page · raw message
===================================================================
include/linux/major.h +2 -0
@@ -156,6 +156,8 @@#define VXSPEC_MAJOR 200/* VERITAS volume config driver */ #define VXDMP_MAJOR 201/* VERITAS volume multipath driver */ +#define XENVBD_MAJOR 202/* Xen virtual block device */ +#define MSR_MAJOR 202#define CPUID_MAJOR 203
PATCH 26/26 Xen-paravirt_ops: Add the Xen virtual network device driver.
In reply to Jeremy FitzhardingeThe network device frontend driver allows the kernel to access network devices exported exported by a virtual machine containing a physical network device driver.
--- drivers/net/Kconfig | 12 drivers/net/Makefile | 2 drivers/net/xen-netfront.c | 2066 ++++++++++++++++++++++++++++++++++++++++++++ include/xen/events.h | 2 4 files changed, 2082 insertions(+)
===================================================================
drivers/net/Kconfig +12 -0
@@ -2525,6 +2525,18 @@ source "drivers/atm/Kconfig" source "drivers/s390/net/Kconfig" +config XEN_NETDEV_FRONTEND + tristate "Xen network device frontend driver" + depends on XEN + default y + help + The network device frontend driver allows the kernel to + access network devices exported exported by a virtual + machine containing a physical network device driver. The + frontend driver is intended for unprivileged guest domains; + if you are compiling a kernel for a Xen guest, you almost + certainly want to enable this. + config ISERIES_VETH tristate "iSeries Virtual Ethernet driver support" depends on PPC_ISERIES
===================================================================
drivers/net/Makefile +2 -0
@@ -218,3 +218,5 @@ obj-$(CONFIG_FS_ENET) += fs_enet/ obj-$(CONFIG_FS_ENET) += fs_enet/ obj-$(CONFIG_NETXEN_NIC) += netxen/ + +obj-$(CONFIG_XEN_NETDEV_FRONTEND) += xen-netfront.o
===================================================================
drivers/net/xen-netfront.c new file +2066 -0
2,066 lines not on this page · raw message
===================================================================
include/xen/events.h +2 -0
@@ -2,6 +2,8 @@#define _XEN_EVENTS_H#include <linux/irq.h> +#include <xen/interface/event_channel.h> +#include <asm/xen/hypercall.h> int bind_evtchn_to_irq (unsigned int evtchn); int bind_evtchn_to_irqhandler(unsigned int evtchn,
Re: [patch 26/26] Xen-paravirt_ops: Add the Xen virtual network device driver.
Quoted 30 lines · On Thu, 01 Mar 2007 15:25:09 -0800 Jeremy Fitzhardinge <[email protected]> wrote:
The network device frontend driver allows the kernel to access network devices exported exported by a virtual machine containing a physical network device driver.
Signed-off-by Ian Pratt <[email protected]>Signed-off-by Christian Limpach <[email protected]>Signed-off-by Chris Wright <[email protected]>Signed-off-by Jeremy Fitzhardinge <[email protected]>Cc Jeff Garzik <[email protected]>--- drivers/net/Kconfig | 12 drivers/net/Makefile | 2 drivers/net/xen-netfront.c | 2066 ++++++++++++++++++++++++++++++++++++++++++++ include/xen/events.h | 2 4 files changed, 2082 insertions(+)===================================================================
drivers/net/Kconfig +5
-0@@ -2525,6 +2525,18 @@ source "drivers/atm/Kconfig" source "drivers/s390/net/Kconfig" +config XEN_NETDEV_FRONTEND + tristate "Xen network device frontend driver" + depends on XEN + default y + help
drivers/net/Kconfig source "drivers/atm/Kconfig" · line 2542
Quoted 10 lines
drivers/net/Kconfig +7
-0+ The network device frontend driver allows the kernel to + access network devices exported exported by a virtual + machine containing a physical network device driver. The + frontend driver is intended for unprivileged guest domains; + if you are compiling a kernel for a Xen guest, you almost + certainly want to enable this. + config ISERIES_VETH tristate "iSeries Virtual Ethernet driver support" depends on PPC_ISERIES
Might make more sense earlier in list (near other virtual devices).
===================================================================
drivers/net/Kconfig
Quoted 9 lines
drivers/net/Kconfig +8
-0+/* + * Mutually-exclusive module options to select receive data path: + * rx_copy : Packets are copied by network backend into local memory + * rx_flip : Page containing packet data is transferred to our ownership + * For fully-virtualised guests there is no option - copying must be used. + * For paravirtualised guests, flipping is the default. + */ +#ifdef CONFIG_XEN
Hey I thought this driver depended on CONFIG_XEN already?
drivers/net/Kconfig
Quoted 10 lines
drivers/net/Kconfig +10
-0+static int MODPARM_rx_copy = 0; +module_param_named(rx_copy, MODPARM_rx_copy, bool, 0); +MODULE_PARM_DESC(rx_copy, "Copy packets from network card (rather than flip)"); +static int MODPARM_rx_flip = 0; +module_param_named(rx_flip, MODPARM_rx_flip, bool, 0); +MODULE_PARM_DESC(rx_flip, "Flip packets from network card (rather than copy)"); +#else +static const int MODPARM_rx_copy = 1; +static const int MODPARM_rx_flip = 0; +#endif
No MIXED case variable names please.
Why have two mutually exclusive values instead of just one value with three states: 0 = normal, 1 = copy, 2 = flip?
drivers/net/Kconfig
Quoted 7 lines
drivers/net/Kconfig +7
-0+#define DPRINTK(fmt, args...) \ + pr_debug("netfront (%s:%d) " fmt, \ + __FUNCTION__, __LINE__, ##args) +#define IPRINTK(fmt, args...) \ + printk(KERN_INFO "netfront: " fmt, ##args) +#define WPRINTK(fmt, args...) \ + printk(KERN_WARNING "netfront: " fmt, ##args)
Could you use dev_dbg, dev_info, dev_warn instead of these macros?
drivers/net/Kconfig
Quoted 7 lines
drivers/net/Kconfig +7
-0+ +/** Send a packet on a net device to encourage switches to learn the + * MAC. We send a fake ARP request. + * + * @param dev device + * @return 0 on success, error code otherwise + */
Why the sudden urge to use docbook format on one internal function.
Quoted 11 lines
drivers/net/Kconfig +11
-0+static int send_fake_arp(struct net_device *dev) +{ + struct sk_buff *skb; + u32 src_ip, dst_ip; + + dst_ip = INADDR_BROADCAST; + src_ip = inet_select_addr(dev, dst_ip, RT_SCOPE_LINK); + + /* No IP? Then nothing to do. */ + if (src_ip == 0) + return 0;
drivers/net/Kconfig
Quoted 10 lines
drivers/net/Kconfig +10
-0+ + skb = arp_create(ARPOP_REPLY, ETH_P_ARP, + dst_ip, dev, src_ip, + /*dst_hw*/ NULL, /*src_hw*/ NULL, + /*target_hw*/ dev->dev_addr); + if (skb == NULL) + return -ENOMEM; + + return dev_queue_xmit(skb); +}
This should probably be done in generic (non driver code). It creates lots of dependencies here.
Quoted 6 lines
drivers/net/Kconfig +6
-0+/* + * We use this notifier to send out a fake ARP reply to reset switches and + * router ARP caches when an IP interface is brought up on a VIF. + */ +static int +inetdev_notify(struct notifier_block *this, unsigned long event, void *ptr)
drivers/net/Kconfig
Quoted 10 lines
drivers/net/Kconfig +10
-0+{ + struct in_ifaddr *ifa = (struct in_ifaddr *)ptr; + struct net_device *dev = ifa->ifa_dev->dev; + + /* UP event and is it one of our devices? */ + if (event == NETDEV_UP && dev->open == network_open) + (void)send_fake_arp(dev); + + return NOTIFY_DONE; +}
Shouldn't just be a global kernel option for gratuitous ARP. Doesn't seem to be unique to this driver.
With sysctl to enable it.
Signature · 1 line
Stephen Hemminger <[email protected]>
Re: [patch 26/26] Xen-paravirt_ops: Add the Xen virtual network device driver.
Quoted 39 lines · Stephen Hemminger
Quoted 35 lines · On Thu, 01 Mar 2007 15:25:09 -0800 Jeremy Fitzhardinge <[email protected]> wrote:
The network device frontend driver allows the kernel to access network devices exported exported by a virtual machine containing a physical network device driver.
Signed-off-by Ian Pratt <[email protected]>Signed-off-by Christian Limpach <[email protected]>Signed-off-by Chris Wright <[email protected]>Signed-off-by Jeremy Fitzhardinge <[email protected]>Cc Jeff Garzik <[email protected]>--- drivers/net/Kconfig | 12 drivers/net/Makefile | 2 drivers/net/xen-netfront.c | 2066 ++++++++++++++++++++++++++++++++++++++++++++ include/xen/events.h | 2 4 files changed, 2082 insertions(+)===================================================================
drivers/net/Kconfig +10
-0@@ -2525,6 +2525,18 @@ source "drivers/atm/Kconfig" source "drivers/s390/net/Kconfig" +config XEN_NETDEV_FRONTEND + tristate "Xen network device frontend driver" + depends on XEN + default y + help + The network device frontend driver allows the kernel to + access network devices exported exported by a virtual + machine containing a physical network device driver. The + frontend driver is intended for unprivileged guest domains; + if you are compiling a kernel for a Xen guest, you almost
Quoted 9 lines
drivers/net/Kconfig source "drivers/atm/Kconfig" · line 2542
Quoted 6 lines
drivers/net/Kconfig +2
-0+ certainly want to enable this. + config ISERIES_VETH tristate "iSeries Virtual Ethernet driver support" depends on PPC_ISERIESMight make more sense earlier in list (near other virtual devices).
OK.
Quoted 3 lines
Hey I thought this driver depended on
CONFIG_XENalready?
You're right.
Quoted 6 lines
Quoted 6 lines
drivers/net/Kconfig +6
-0+static int MODPARM_rx_copy = 0; +module_param_named(rx_copy, MODPARM_rx_copy, bool, 0); +MODULE_PARM_DESC(rx_copy, "Copy packets from network card (rather than flip)"); +static int MODPARM_rx_flip = 0; +module_param_named(rx_flip, MODPARM_rx_flip, bool, 0); +MODULE_PARM_DESC(rx_flip, "Flip packets from network card (rather than copy)");
Quoted 9 lines
drivers/net/Kconfig
Quoted 5 lines
drivers/net/Kconfig +4
-0+#else +static const int MODPARM_rx_copy = 1; +static const int MODPARM_rx_flip = 0; +#endifNo MIXED case variable names please.
OK.
Quoted 3 lines
Why have two mutually exclusive values instead of just one value with three states: 0 = normal, 1 = copy, 2 = flip?
That does seem odd. I'll fix it up.
Quoted 12 lines
drivers/net/Kconfig
Quoted 8 lines
drivers/net/Kconfig +7
-0+#define DPRINTK(fmt, args...) \ + pr_debug("netfront (%s:%d) " fmt, \ + __FUNCTION__, __LINE__, ##args) +#define IPRINTK(fmt, args...) \ + printk(KERN_INFO "netfront: " fmt, ##args) +#define WPRINTK(fmt, args...) \ + printk(KERN_WARNING "netfront: " fmt, ##args)Could you use
dev_dbg,dev_info,dev_warninstead of these macros?
Yes. I'd done that conversion elsewhere, but overlooked it here.
Quoted 10 lines
drivers/net/Kconfig
Quoted 8 lines
drivers/net/Kconfig +7
-0+ +/** Send a packet on a net device to encourage switches to learn the + * MAC. We send a fake ARP request. + * + * @param dev device + * @return 0 on success, error code otherwise + */Why the sudden urge to use docbook format on one internal function.
It probably got copied from somewhere. I'll clean it up.
Quoted 21 lines
Quoted 21 lines
drivers/net/Kconfig +21
-0+static int send_fake_arp(struct net_device *dev) +{ + struct sk_buff *skb; + u32 src_ip, dst_ip; + + dst_ip = INADDR_BROADCAST; + src_ip = inet_select_addr(dev, dst_ip, RT_SCOPE_LINK); + + /* No IP? Then nothing to do. */ + if (src_ip == 0) + return 0; + + skb = arp_create(ARPOP_REPLY, ETH_P_ARP, + dst_ip, dev, src_ip, + /*dst_hw*/ NULL, /*src_hw*/ NULL, + /*target_hw*/ dev->dev_addr); + if (skb == NULL) + return -ENOMEM; + + return dev_queue_xmit(skb); +}
Quoted 9 lines
Quoted 1 line
This should probably be done in generic (non driver code). It creates lots of dependencies here.
[...]
Shouldn't just be a global kernel option for gratuitous ARP. Doesn't seem to be unique to this driver.
With sysctl to enable it.
I agree it would be nice to not have to do this in the driver. The specific requirement here is to make it send a packet after resume (which includes arriving after a migration) so that a switch can quickly work that the machine is there. Is there a generic mechanism for doing this?
J
RFC 1/1 Arp announce (for Xen)
What about implementing the unused arp_announce flag on the inetdevice? Something like the following. Totally untested...
Looks like it either was there (and got removed) or was planned but never implemented.
net/ipv4/devinet.c +10 -0
@@ -1089,6 +1089,16 @@ static int inetdev_event(struct notifier } }ip_mc_up (in_dev); +/* fallthru */ + +case NETDEV_CHANGEADDR : +/* Send gratuitous ARP in case of address change or new device */ +if (IN_DEV_ARP_ANNOUNCE (in_dev)) +arp_send (ARPOP_REQUEST ,ETH_P_ARP , + in_dev->ifa_list->ifa_address, dev, + in_dev->ifa_list->ifa_address,NULL , + dev->dev_addr,NULL ); +break ;case NETDEV_DOWN :ip_mc_down (in_dev);
Re: [RFC] Arp announce (for Xen)
Quoted 5 lines · Stephen Hemminger
What about implementing the unused
arp_announceflag on the inetdevice? Something like the following. Totally untested...Looks like it either was there (and got removed) or was planned but never implemented.
This would be acceptable. It only needs a simple script to poke the correct value, and that could be easily integrated into vendor scripts. The general idea of being able to have a callback to userspace is okay but we'd like something lighter weight for live migration, which is typically done only within the scope of a single layer-2 network and hence all that is really required is the gratuitous ARP to kick the switch hardware.
-- Keir
Re: [RFC] Arp announce (for Xen)
In reply to Stephen HemmingerQuoted 5 lines · Stephen Hemminger
What about implementing the unused
arp_announceflag on the inetdevice? Something like the following. Totally untested...Looks like it either was there (and got removed) or was planned but never implemented.
If something like this goes in, it wouldn't hurt to do similar with IPv6 (RFC2461 section 7.2.6).
There are very popular hardware-based routers which refresh their NDP caches only every 24 hours or 20 minutes (depending on the software version). Sending unsolicited NAs would eliminate traffic blackholing.
Quoted 27 lines
net/ipv4/devinet.c +10
-1@@ -1089,6 +1089,16 @@ static int inetdev_event(struct notifier } }ip_mc_up (in_dev); +/* fallthru */ + +case NETDEV_CHANGEADDR : +/* Send gratuitous ARP in case of address change or new device */ +if (IN_DEV_ARP_ANNOUNCE (in_dev)) +arp_send (ARPOP_REQUEST ,ETH_P_ARP , + in_dev->ifa_list->ifa_address, dev, + in_dev->ifa_list->ifa_address,NULL , + dev->dev_addr,NULL ); +break ;case NETDEV_DOWN :ip_mc_down (in_dev); -To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 3 lines
Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
Re: [RFC] Arp announce (for Xen)
Quoted 6 lines · Pekka Savola
Quoted 5 lines · Stephen Hemminger
What about implementing the unused
arp_announceflag on the inetdevice? Something like the following. Totally untested...Looks like it either was there (and got removed) or was planned but never implemented.
IN_DEV_ARP_ANNOUNCE is in 2.6.18, at least..used in arp_solicit in arp.c
I really hope this didn't get removed because I find it very useful!
But, you could certainly add another sysctl...
Thanks,
Ben
Signature · 1 line
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Re: [RFC] Arp announce (for Xen)
Quoted 7 lines · On Fri, 02 Mar 2007 10:29:53 -0800 Ben Greear <[email protected]> wrote:
Quoted 6 lines · Pekka Savola
Quoted 5 lines · Stephen Hemminger
What about implementing the unused
arp_announceflag on the inetdevice? Something like the following. Totally untested...Looks like it either was there (and got removed) or was planned but never implemented.
Quoted 9 lines
IN_DEV_ARP_ANNOUNCEis in 2.6.18, at least..used inarp_solicitinarp.cI really hope this didn't get removed because I find it very useful!
But, you could certainly add another sysctl...
Thanks,
Ben
yeah, something new like arp_notify? or arp_gratiutous
There are other drivers that do their own arp, they need to be fixed.
Signature · 1 line
Stephen Hemminger <[email protected]>
Re: [RFC] Arp announce (for Xen)
In reply to Stephen HemmingerQuoted 5 lines · Stephen Hemminger
What about implementing the unused
arp_announceflag on the inetdevice? Something like the following. Totally untested...Looks like it either was there (and got removed) or was planned but never implemented.
Seems like the right solution to me (if it works)
-Andi
Re: [RFC] Arp announce (for Xen)
Quoted 1 line · Andi Kleen
Seems like the right solution to me (if it works)
I like it as well given the simple change.
These are the kind of optimizations that justify offloading things to the kernel (instead of user space) i.e very little lines of code, covers a large number of user-desires, and can be turned off if you want to do something complex. _Almost_ elegant Stephen;->
cheers,
jamal
Re: [RFC] Arp announce (for Xen)
In reply to Stephen Hemmingernet/ipv4/devinet.c static int inetdev_event(struct notifier · line 1096
Quoted 3 lines · * Stephen Hemminger ([email protected]) wrote:
net/ipv4/devinet.c +3
-0+case NETDEV_CHANGEADDR : +/* Send gratuitous ARP in case of address change or new device */ +if (IN_DEV_ARP_ANNOUNCE (in_dev))
Conceptually right on, but it looks like improper hijacking of arp_announce sysctl. Could introduce another
thanks,
-chris
David Miller
maintainerRe: [RFC] Arp announce (for Xen)
In reply to Stephen HemmingerFrom: Stephen Hemminger <[email protected]>
Date: Thu, 1 Mar 2007 17:30:30 -0800
Quoted 5 lines
What about implementing the unused
arp_announceflag on the inetdevice? Something like the following. Totally untested...Looks like it either was there (and got removed) or was planned but never implemented.
This idea is fine. But:
net/ipv4/devinet.c static int inetdev_event(struct notifier · line 1100
Quoted 7 lines
net/ipv4/devinet.c +7
-0+case NETDEV_CHANGEADDR : +/* Send gratuitous ARP in case of address change or new device */ +if (IN_DEV_ARP_ANNOUNCE (in_dev)) +arp_send (ARPOP_REQUEST ,ETH_P_ARP , + in_dev->ifa_list->ifa_address, dev, + in_dev->ifa_list->ifa_address,NULL , + dev->dev_addr,NULL );
We'll need to make sure the appropriate 'arp_anounce' address selection is employed here.
One idea is to change arp_solicit() such that it can be invoked in this context, or provide a new helper function which will do the source address selection rules of 'arp_announce' and then invoke arp_send() as appropriate for us.
RFC 1/1 ARP notify option
This adds another inet device option to enable gratuitous ARP when device is brought up or address change. This is handy for clusters or virtualization.
Tested on a normal device (not Xen).
--- Documentation/networking/ip-sysctl.txt | 6 ++++++ include/linux/inetdevice.h | 2 ++ include/linux/sysctl.h | 1 + net/ipv4/devinet.c | 16 ++++++++++++++++ 4 files changed, 25 insertions(+)
Documentation/networking/ip-sysctl.txt +6 -0
@@ -732,6 +732,12 @@ The max value from conf/{all,interface}/arp_ignore is used when ARP request is received on the {interface} +arp_notify - BOOLEAN + Define mode for notification of address and device changes. + 0 - (default): do nothing + 1 - Generate gratuitous arp replies when device is brought up + or hardware address changes. + arp_accept - BOOLEAN Define behavior when gratuitous arp replies are received: 0 - drop gratuitous arp frames
include/linux/inetdevice.h +2 -0
@@ -26,6 +26,7 @@int arp_announce;int arp_ignore;int arp_accept; +int arp_notify;int medium_id;int no_xfrm;int no_policy;
@@ -84,6 +85,7 @@#define IN_DEV_ARPFILTER (in_dev) (ipv4_devconf.arp_filter || (in_dev)->cnf.arp_filter)#define IN_DEV_ARP_ANNOUNCE (in_dev) (max(ipv4_devconf.arp_announce, (in_dev)->cnf.arp_announce))#define IN_DEV_ARP_IGNORE (in_dev) (max(ipv4_devconf.arp_ignore, (in_dev)->cnf.arp_ignore)) +#define IN_DEV_ARP_NOTIFY (in_dev) (ipv4_devconf.arp_notify || (in_dev)->cnf.arp_notify)struct in_ifaddr {
include/linux/sysctl.h +1 -0
@@ -495,6 +495,7 @@NET_IPV4_CONF_ARP_IGNORE =19 ,NET_IPV4_CONF_PROMOTE_SECONDARIES =20 ,NET_IPV4_CONF_ARP_ACCEPT =21 , +NET_IPV4_CONF_ARP_NOTIFY =22 , __NET_IPV4_CONF_MAX };
net/ipv4/devinet.c +16 -0
@@ -1089,6 +1089,14 @@ } }ip_mc_up (in_dev); +/* fall through */ +case NETDEV_CHANGEADDR : +if (IN_DEV_ARP_NOTIFY (in_dev)) +arp_send (ARPOP_REQUEST ,ETH_P_ARP , + in_dev->ifa_list->ifa_address, + dev, + in_dev->ifa_list->ifa_address, +NULL , dev->dev_addr,NULL );break ;case NETDEV_DOWN :ip_mc_down (in_dev);
@@ -1495,6 +1503,14 @@ .proc_handler = &proc_dointvec, }, { + .ctl_name = NET_IPV4_CONF_ARP_NOTIFY, + .procname = "arp_notify", + .data = &ipv4_devconf.arp_notify, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec, + }, + { .ctl_name = NET_IPV4_CONF_NOXFRM, .procname = "disable_xfrm", .data = &ipv4_devconf.no_xfrm,
Re: [RFC] ARP notify option
Quoted 4 lines · Stephen Hemminger
This adds another inet device option to enable gratuitous ARP when device is brought up or address change. This is handy for clusters or virtualization.
Thanks Stephen. Haven't tested this yet, but it definitely cleans up a warty corner of netfront.
J
Re: [RFC] ARP notify option
In reply to Stephen HemmingerQuoted 3 lines · * Stephen Hemminger ([email protected]) wrote:
This adds another inet device option to enable gratuitous ARP when device is brought up or address change. This is handy for clusters or virtualization.
This looks good. I'll test with Xen. What about the source addr selection?
thanks,
-chris
Re: [RFC] ARP notify option
In reply to Stephen HemmingerDocumentation/networking/ip-sysctl.txt line 739
Quoted 5 lines · Stephen Hemminger
Documentation/networking/ip-sysctl.txt +5
-0+arp_notify - BOOLEAN + Define mode for notification of address and device changes. + 0 - (default): do nothing + 1 - Generate gratuitous arp replies when device is brought up + or hardware address changes.
Did you consider using gratuitous arp requests instead? I remember reading about some hardware that updated its arp cache on gratuitous requests but not gratuitous replies.
Chris
Re: [RFC] ARP notify option
Quoted 12 lines · On Tue, 06 Mar 2007 15:18:07 -0600 "Chris Friesen" <[email protected]> wrote:
Documentation/networking/ip-sysctl.txt line 739
Quoted 5 lines · Stephen Hemminger
Documentation/networking/ip-sysctl.txt +5
-0+arp_notify - BOOLEAN + Define mode for notification of address and device changes. + 0 - (default): do nothing + 1 - Generate gratuitous arp replies when device is brought up + or hardware address changes.Did you consider using gratuitous arp requests instead? I remember reading about some hardware that updated its arp cache on gratuitous requests but not gratuitous replies.
Quoted 1 line
Chris
I copied the ARP generation from other places that were doing gratuitous ARP already: Xen and irlan.
Our local switch used REPLY's to do the same thing.
One could imagine making it a ternary value and having 2 generate REQUEST's.
Signature · 1 line
Stephen Hemminger <[email protected]>
Re: [RFC] ARP notify option
In reply to Chris FriesenQuoted 10 lines · Chris Friesen
Quoted 5 lines · Stephen Hemminger
+arp_notify - BOOLEAN
+ Define mode for notification of address and device changes.
+ 0 - (default): do nothing
+ 1 - Generate gratuitous arp replies when device is brought up
+ or hardware address changes.Did you consider using gratuitous arp requests instead? I remember reading about some hardware that updated its arp cache on gratuitous requests but not gratuitous replies.
You might be interested in taking a look at:
http://tools.ietf.org/id/draft-cheshire-ipv4-acd
There has been some follow-up discussion on this in the thread starting at:
http://www1.ietf.org/mail-archive/web/int-area/current/msg00611.html
In particular, you may be interested in this comment about ARP request and ARP reply for gratuitous ARP:
http://www1.ietf.org/mail-archive/web/int-area/current/msg00669.html
Signature · 3 lines
Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
Re: [RFC] ARP notify option
Quoted 19 lines · On Wed, 7 Mar 2007 08:42:39 +0200 (EET) Pekka Savola <[email protected]> wrote:
Quoted 10 lines · Chris Friesen
Quoted 5 lines · Stephen Hemminger
+arp_notify - BOOLEAN
+ Define mode for notification of address and device changes.
+ 0 - (default): do nothing
+ 1 - Generate gratuitous arp replies when device is brought up
+ or hardware address changes.Did you consider using gratuitous arp requests instead? I remember reading about some hardware that updated its arp cache on gratuitous requests but not gratuitous replies.
You might be interested in taking a look at:
http://tools.ietf.org/id/draft-cheshire-ipv4-acd
There has been some follow-up discussion on this in the thread starting at:
Quoted 6 lines
http://www1.ietf.org/mail-archive/web/int-area/current/msg00611.html
In particular, you may be interested in this comment about ARP request and ARP reply for gratuitous ARP:
http://www1.ietf.org/mail-archive/web/int-area/current/msg00669.html
Looks like REQUESTS make more sense. (See below). I will rework this patch.
5. Why are ARP Announcements performed using ARP Request packets and not ARP Reply packets?
During IETF deliberation of IPv4 Address Conflict Detection from 2000 to 2005, a question that kept being asked repeatedly was, "Shouldn't ARP Announcements be performed using gratuitous ARP Reply packets?"
On the face of it, this seems reasonable. A conventional ARP Reply is an answer to a question. If in fact no question had been asked, then it would be reasonable to describe such a reply as gratuitous. This description would seem to apply perfectly to an ARP
Announcement: an answer to an implied question that in fact no one asked.
However reasonable this may seem in principle, there are two reasons why in practice ARP Request packets are the better choice. One is historical precedent, and the other is practicality.
Expires 11th January 2006 Cheshire [Page 14]
Internet Draft IPv4 Address Conflict Detection 11th July 2005
The historical precedent is that, as described above in Section 4, Gratuitous ARP is described in Stevens Networking [Ste94] as using ARP Request packets. BSD Unix, Windows, Mac OS 9, Mac OS X, etc. all use ARP Request packets as described in Stevens. At this stage, trying to mandate that they all switch to using ARP Reply packets would be futile.
The practical reason is that ARP Request packets are more likely to work correctly with more existing ARP implementations, some of which may not implement RFC 826 correctly. The Packet Reception rules in RFC 826 state that the opcode is the last thing to check in packet processing, so it really shouldn't matter, but there may be "creative" implementations that have different packet processing depending on the ar$op field, and there are several reasons why these are more likely to accept gratuitous ARP Requests than gratuitous ARP Replies:
* An incorrect ARP implementation may expect that ARP Replies are only sent via unicast. RFC 826 does not say this, but an incorrect implementation may assume it, and the "principle of least surprise" dictates that where there are two or more ways to solve a networking problem that are otherwise equally good, the one with the fewest unusual properties is the one likely to have the fewest interoperability problems with existing implementations. An ARP Announcement needs to broadcast information to all hosts on the link. Since ARP Request packets are always broadcast, and ARP Reply packets are not, receiving an ARP Request packet via broadcast is less surprising than receiving an ARP Reply packet via broadcast.
* An incorrect ARP implementation may expect that ARP Replies are only received in response to ARP Requests that have been issued recently by that implementation. Unexpected unsolicited Replies may be ignored.
* An incorrect ARP implementation may ignore ARP Replies where ar$tha doesn't match its hardware address.
* An incorrect ARP implementation may ignore ARP Replies where ar$tpa doesn't match its IP address.
In summary, there are more ways that an incorrect ARP implementation might plausibly reject an ARP Reply (which usually occurs as a result of being solicited by the client) than an ARP Request (which is already expected to occur unsolicited).
Re: [patch 26/26] Xen-paravirt_ops: Add the Xen virtual network device driver.
In reply to Stephen HemmingerQuoted 8 lines · Stephen Hemminger
drivers/net/xen-netfront.c line 489
Quoted 7 lines
drivers/net/xen-netfront.c new file +7
-0+ +/** Send a packet on a net device to encourage switches to learn the +* MAC. We send a fake ARP request. +* +* @param dev device +* @return 0 on success, error code otherwise +*/ Why the sudden urge to use docbook format on one internal function.
And it's not even proper kerneldoc but looks more like javadoc. If you want to write kerneldoc comments please also verify them by running the tools over it.
Quoted 2 lines
This should probably be done in generic (non driver code). It creates lots of dependencies here.
Actually the right way to do it is in userspace, as all clustering solutions do. That's whay everyone told the Xen folks by the just refuse to rip this junk out.
Re: [patch 26/26] Xen-paravirt_ops: Add the Xen virtual network device driver.
Quoted 3 lines · * Christoph Hellwig ([email protected]) wrote:
Actually the right way to do it is in userspace, as all clustering solutions do. That's whay everyone told the Xen folks by the just refuse to rip this junk out.
I'm ripping it out.
Quoted 2 lines · Jeremy Fitzhardinge
* virtual block device (blockfront)
* virtual network device (netfront)
note, these drivers should be submitted through the proper block drivers and network drivers review process - not via the x86_64 tree.
Ingo
Quoted 10 lines · Ingo Molnar
Quoted 3 lines · Jeremy Fitzhardinge
* virtual block device (blockfront)
* virtual network device (netfront)note, these drivers should be submitted through the proper block drivers and network drivers review process - not via the
x86_64tree.
Yes; who should look at them? I posted netfront to net-dev, and Stephen Hemminger commented on it on the last repost, but I'd love them to get more scrutiny.
Obviously they're not actually useful without Xen (and vice-versa), so there's not much point in committing them separately.
J
Quoted 2 lines · Jeremy Fitzhardinge
This patch series implements the Linux Xen guest as a
paravirt_opsbackend. [...]
19/26 seems to be missing (lkml size limit?) so i couldnt review it.
Ingo
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Jeremy Fitzhardinge
This patch series implements the Linux Xen guest as a
paravirt_opsbackend. [...]19/26 seems to be missing (lkml size limit?) so i couldnt review it.
Shouldn't think so; its a fairly small patch. Here it is.
J
Subject: add kmap_atomic_pte for mapping highpte pages
Xen and VMI both have special requirements when mapping a highmem pte page into the kernel address space. These can be dealt with by adding a new kmap_atomic_pte() function for mapping highptes, and hooking it into the paravirt_ops infrastructure.
Xen specifically wants to map the pte page RO, so this patch exposes a helper function, _kmap_atomic, which maps the page with the specified page protections.
This also adds a kmap_flush_unused() function to clear out the cached kmap mappings. Xen needs this to clear out any potential stray RW mappings of pages which will become part of a pagetable.
--- arch/i386/kernel/paravirt.c | 7 +++++++ arch/i386/mm/highmem.c | 9 +++++++-- include/asm-i386/highmem.h | 11 +++++++++++ include/asm-i386/paravirt.h | 14 +++++++++++++- include/asm-i386/pgtable.h | 4 ++-- include/linux/highmem.h | 6 ++++++ mm/highmem.c | 9 +++++++++ 7 files changed, 55 insertions(+), 5 deletions(-)
===================================================================
arch/i386/kernel/paravirt.c +7 -0
@@ -20,6 +20,7 @@#include <linux/efi.h> #include <linux/bcd.h> #include <linux/start_kernel.h> +#include <linux/highmem.h> #include <asm/bug.h> #include <asm/paravirt.h>
@@ -600,6 +601,12 @@ struct paravirt_ops paravirt_ops = { .ptep_get_and_clear = native_ptep_get_and_clear, +#ifdef CONFIG_HIGHPTE + .kmap_atomic_pte = native_kmap_atomic_pte, +#else + .kmap_atomic_pte = paravirt_nop, +#endif +#ifdef CONFIG_X86_PAE .set_pte_atomic = native_set_pte_atomic, .set_pte_present = native_set_pte_present,
===================================================================
arch/i386/mm/highmem.c +7 -2
@@ -26,7 +26,7 @@ void kunmap(struct page *page)* However when holding an atomic kmap is is not legal to sleep, so atomic * kmaps are appropriate for short, tight code paths only. */ -void *kmap_atomic (struct page *page,enum km_type type) +void *_kmap_atomic (struct page *page,enum km_type type,pgprot_t prot) {enum fixed_addresses idx;unsigned long vaddr;
@@ -41,9 +41,14 @@ void *kmap_atomic(struct page *page, enureturn page_address (page); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx); -set_pte (kmap_pte-idx,mk_pte (page, kmap_prot)); +set_pte (kmap_pte-idx,mk_pte (page, prot));return (void *) vaddr; +} + +void *kmap_atomic (struct page *page,enum km_type type) +{ +return _kmap_atomic (page, type, kmap_prot); }void kunmap_atomic (void *kvaddr,enum km_type type)
===================================================================
include/asm-i386/highmem.h +11 -0
@@ -24,6 +24,7 @@#include <linux/threads.h> #include <asm/kmap_types.h> #include <asm/tlbflush.h> +#include <asm/paravirt.h> /* declarations for highmem.c */ extern unsigned long highstart_pfn, highend_pfn;
@@ -67,10 +68,20 @@ extern void FASTCALL(kunmap_high(structvoid *kmap (struct page *page);void kunmap (struct page *page); +void *_kmap_atomic (struct page *page,enum km_type type,pgprot_t prot);void *kmap_atomic (struct page *page,enum km_type type);void kunmap_atomic (void *kvaddr,enum km_type type);void *kmap_atomic_pfn (unsigned long pfn,enum km_type type);struct page *kmap_atomic_to_page (void *ptr); + +static inline void *native_kmap_atomic_pte (struct page *page,enum km_type type) +{ +return kmap_atomic (page, type); +} + +#ifndef CONFIG_PARAVIRT +#define kmap_atomic_pte (page, type) kmap_atomic(page, type) +#endif #define flush_cache_kmaps () do { } while (0)
===================================================================
include/asm-i386/paravirt.h +13 -1
@@ -16,11 +16,14 @@ #ifndef __ASSEMBLY__ #include <linux/types.h> #include <linux/cpumask.h> - +#include <asm/kmap_types.h> + +struct page; struct thread_struct; struct Xgt_desc_struct; struct tss_struct; struct mm_struct; + struct paravirt_ops { unsigned int kernel_rpl;
@@ -138,6 +141,8 @@ struct paravirt_opsvoid (*pte_update_defer)(struct mm_struct *mm,unsigned long addr,pte_t *ptep);pte_t (*ptep_get_and_clear)(pte_t *ptep); + +void *(*kmap_atomic_pte)(struct page *page,enum km_type type);#ifdef CONFIG_X86_PAE void (*set_pte_atomic)(pte_t *ptep,pte_t pteval);
@@ -667,6 +672,13 @@ static inline void flush_tlb_others(cpum PVOP_VCALL4(alloc_pd_clone, pfn, clonepfn, start, count)#define paravirt_release_pd (pfn) PVOP_VCALL1(release_pd, pfn) +static inline void *kmap_atomic_pte (struct page *page,enum km_type type) +{ +unsigned long ret; + ret = PVOP_CALL2(unsigned long, kmap_atomic_pte, page, type); +return (void *)ret; +} +static inline void pte_update (struct mm_struct *mm,unsigned long addr,pte_t *ptep) {PVOP_VCALL3 (pte_update, mm, addr, ptep);
===================================================================
include/asm-i386/pgtable.h +2 -2
@@ -479,9 +479,9 @@ extern pte_t *lookup_address(unsigned lo#if defined(CONFIG_HIGHPTE )#define pte_offset_map (dir, address) \ - ((pte_t *)kmap_atomic(pmd_page(*(dir)),KM_PTE0) + pte_index(address)) + ((pte_t *)kmap_atomic_pte(pmd_page(*(dir)),KM_PTE0) + pte_index(address))#define pte_offset_map_nested (dir, address) \ - ((pte_t *)kmap_atomic(pmd_page(*(dir)),KM_PTE1) + pte_index(address)) + ((pte_t *)kmap_atomic_pte(pmd_page(*(dir)),KM_PTE1) + pte_index(address))#define pte_unmap (pte) kunmap_atomic(pte, KM_PTE0)#define pte_unmap_nested (pte) kunmap_atomic(pte, KM_PTE1)#else
===================================================================
include/linux/highmem.h +6 -0
@@ -27,6 +27,8 @@ unsigned int nr_free_highpages(void);unsigned int nr_free_highpages (void );extern unsigned long totalhigh_pages; +void kmap_flush_unused (void ); +#else /* CONFIG_HIGHMEM */ static inline unsigned int nr_free_highpages (void ) {return 0 ; }
@@ -44,9 +46,13 @@ static inline void *kmap(struct page *pa#define kmap_atomic (page, idx) \ ({ pagefault_disable(); page_address(page); }) +#define _kmap_atomic (page, idx, prot) kmap_atomic(page, idx) +#define kunmap_atomic (addr, idx) do { pagefault_enable(); } while (0)#define kmap_atomic_pfn (pfn, idx) kmap_atomic(pfn_to_page(pfn), (idx))#define kmap_atomic_to_page (ptr) virt_to_page(ptr) + +#define kmap_flush_unused () do {} while(0)#endif #endif /* CONFIG_HIGHMEM */
===================================================================
mm/highmem.c +9 -0
@@ -97,6 +97,15 @@ static void flush_all_zero_pkmaps(void)set_page_address (page,NULL ); }flush_tlb_kernel_range (PKMAP_ADDR (0 ),PKMAP_ADDR (LAST_PKMAP )); +} + +/* Flush all unused kmap mappings in order to remove stray +mappings. */ +void kmap_flush_unused (void ) +{ +spin_lock (&kmap_lock); +flush_all_zero_pkmaps (); +spin_unlock (&kmap_lock); }static inline unsigned long map_new_virtual (struct page *page)
arch/i386/kernel/paravirt.c struct paravirt_ops paravirt_ops = { · line 608
Quoted 5 lines · Jeremy Fitzhardinge
arch/i386/kernel/paravirt.c +5
-0+#ifdef CONFIG_HIGHPTE + .kmap_atomic_pte = native_kmap_atomic_pte, +#else + .kmap_atomic_pte = paravirt_nop, +#endif
This is ifdefing is quite ugly. Shouldn't native_kmap_atomic_pte just be a noop in the !CONFIG_HIGHPTE case?
arch/i386/mm/highmem.c void kunmap(struct page *page) · line 29
Quoted 2 lines
arch/i386/mm/highmem.c +1
-1-void *kmap_atomic (struct page *page,enum km_type type) +void *_kmap_atomic (struct page *page,enum km_type type,pgprot_t prot)
We normally call our "secial" function __foo, not _foo. But in this case it really should have a more meaningfull name like kmap_atomic_prot anyway.
arch/i386/mm/highmem.c void *kmap_atomic(struct page *page, enu · line 51
Quoted 3 lines
arch/i386/mm/highmem.c +3
-0+void *kmap_atomic (struct page *page,enum km_type type) +{ +return _kmap_atomic (page, type, kmap_prot);
And this one should probably be an inline.
include/asm-i386/highmem.h extern void FASTCALL(kunmap_high(struct · line 84
Quoted 8 lines
include/asm-i386/highmem.h +8
-0+static inline void *native_kmap_atomic_pte (struct page *page,enum km_type type) +{ +return kmap_atomic (page, type); +} + +#ifndef CONFIG_PARAVIRT +#define kmap_atomic_pte (page, type) kmap_atomic(page, type) +#endif
This is all getting rather ugly just for your pagetable hackery.
Quoted 12 lines · Christoph Hellwig
arch/i386/kernel/paravirt.c struct paravirt_ops paravirt_ops = { · line 608
Quoted 6 lines · Jeremy Fitzhardinge
arch/i386/kernel/paravirt.c +5
-0+#ifdef CONFIG_HIGHPTE + .kmap_atomic_pte = native_kmap_atomic_pte, +#else + .kmap_atomic_pte = paravirt_nop, +#endif This is ifdefing is quite ugly. Shouldn't
native_kmap_atomic_ptejust be a noop in the !CONFIG_HIGHPTE case?
Yes, but the trouble is that asm/highmem.h simply isn't included in the !HIGHMEM case, so I can't put anything in there, and putting anything pv_ops related into linux/highmem.h isn't appropriate either.
Quoted 8 lines
arch/i386/mm/highmem.c void kunmap(struct page *page) · line 29
Quoted 3 lines
arch/i386/mm/highmem.c +1
-1-void *kmap_atomic (struct page *page,enum km_type type) +void *_kmap_atomic (struct page *page,enum km_type type,pgprot_t prot)We normally call our "secial" function
__foo, not_foo. But in this case it really should have a more meaningfull name likekmap_atomic_protanyway.
OK.
Quoted 7 lines
arch/i386/mm/highmem.c void *kmap_atomic(struct page *page, enu · line 51
Quoted 4 lines
arch/i386/mm/highmem.c +3
-0+void *kmap_atomic (struct page *page,enum km_type type) +{ +return _kmap_atomic (page, type, kmap_prot);And this one should probably be an inline.
OK, if you think it makes a difference.
Quoted 12 lines
include/asm-i386/highmem.h extern void FASTCALL(kunmap_high(struct · line 84
Quoted 9 lines
include/asm-i386/highmem.h +8
-0+static inline void *native_kmap_atomic_pte (struct page *page,enum km_type type) +{ +return kmap_atomic (page, type); +} + +#ifndef CONFIG_PARAVIRT +#define kmap_atomic_pte (page, type) kmap_atomic(page, type) +#endif This is all getting rather ugly just for your pagetable hackery.
Well, I could promote kmap_atomic_pte to a first-class interface, but it seems like overkill.
J