xen/pvh: Dom0 support
Subsystems (3)
cover xen/pvh: Dom0 support
Hello,
The following patches enable booting a PVH Dom0. So far I've only tested them in my test box, so I expect more issues will show up as this gets further testing. In any case, this is the bare minimum to get a PVH Dom0 working.
In order to test them Xen 4.11 RC or plain master/staging should be used. Compile a Linux kernel with this patches and PVH support and add dom0=pvh to the Xen command line. The following message will be displayed during boot by the Linux kernel if PVH mode is used:
"Booting paravirtualized kernel on Xen PVH"
Thanks, Roger.
Roger Pau Monne (3):
xen/pvh: enable and set default MTRR type xen/store: do not store local values in xen_start_info
xen: share start flags between PV and PVH
arch/x86/xen/enlighten.c | 7 +++++++ arch/x86/xen/enlighten_pv.c | 1 + arch/x86/xen/enlighten_pvh.c | 4 ++++ drivers/xen/xenbus/xenbus_probe.c | 5 ++--- include/xen/xen.h | 4 +++- 5 files changed, 17 insertions(+), 4 deletions(-)
Signature · 1 line
2.17.0
PATCH 1/3 xen/pvh: enable and set default MTRR type
On PVH MTRR is not initialized by the firmware (because there's no firmware), so the kernel is started with MTRR disabled which means all memory accesses are UC.
So far there have been no issues (ie: slowdowns) caused by this because PVH only supported DomU mode without passed-through devices, so Xen was using WB as the default memory type instead of UC.
Fix this by enabling MTRR and setting the default type to WB. Linux will use PAT to set the actual memory cache attributes.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: [email protected]
--- arch/x86/xen/enlighten_pvh.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/xen/enlighten_pvh.c +3 -0
@@ -6,6 +6,7 @@#include <asm/io_apic.h> #include <asm/hypervisor.h> #include <asm/e820/api.h> +#include <asm/mtrr.h> #include <asm/x86_init.h> #include <asm/xen/interface.h>
@@ -98,6 +99,8 @@ void __init xen_prepare_pvh(void) xen_pvh =1 ; +wrmsr_safe (MSR_MTRRdefType,0x800 |MTRR_TYPE_WRBACK ,0 ); + msr =cpuid_ebx (xen_cpuid_base () +2 ); pfn =__pa (hypercall_page);wrmsr_safe (msr, (u32 )pfn, (u32)(pfn >>32 ));
Signature · 7 lines
2.17.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/3 xen/pvh: enable and set default MTRR type
In reply to Roger Pau MonneOn PVH MTRR is not initialized by the firmware (because there's no firmware), so the kernel is started with MTRR disabled which means all memory accesses are UC.
So far there have been no issues (ie: slowdowns) caused by this because PVH only supported DomU mode without passed-through devices, so Xen was using WB as the default memory type instead of UC.
Fix this by enabling MTRR and setting the default type to WB. Linux will use PAT to set the actual memory cache attributes.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: [email protected]
--- arch/x86/xen/enlighten_pvh.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/xen/enlighten_pvh.c +3 -0
@@ -6,6 +6,7 @@#include <asm/io_apic.h> #include <asm/hypervisor.h> #include <asm/e820/api.h> +#include <asm/mtrr.h> #include <asm/x86_init.h> #include <asm/xen/interface.h>
@@ -98,6 +99,8 @@ void __init xen_prepare_pvh(void) xen_pvh =1 ; +wrmsr_safe (MSR_MTRRdefType,0x800 |MTRR_TYPE_WRBACK ,0 ); + msr =cpuid_ebx (xen_cpuid_base () +2 ); pfn =__pa (hypercall_page);wrmsr_safe (msr, (u32 )pfn, (u32)(pfn >>32 ));
Signature · 1 line
2.17.0
PATCH 2/3 xen/store: do not store local values in xen_start_info
In reply to Roger Pau MonneThere's no need to store the xenstore page or event channel in xen_start_info if they are locally initialized.
This also fixes PVH local xenstore initialization due to the lack of xen_start_info in that case.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: [email protected]
--- drivers/xen/xenbus/xenbus_probe.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/xen/xenbus/xenbus_probe.c +2 -3
@@ -710,7 +710,7 @@ static int __init xenstored_local_init(void)if (!page) goto out_err; - xen_store_gfn = xen_start_info->store_mfn =virt_to_gfn ((void *)page); + xen_store_gfn =virt_to_gfn ((void *)page);/* Next allocate a local port which xenstored can bind to */ alloc_unbound.dom =DOMID_SELF ;
@@ -722,8 +722,7 @@ static int __init xenstored_local_init(void) goto out_err;BUG_ON (err); - xen_store_evtchn = xen_start_info->store_evtchn = - alloc_unbound.port; + xen_store_evtchn = alloc_unbound.port;return 0 ;
Signature · 7 lines
2.17.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/3 xen/store: do not store local values in xen_start_info
In reply to Roger Pau MonneThere's no need to store the xenstore page or event channel in xen_start_info if they are locally initialized.
This also fixes PVH local xenstore initialization due to the lack of xen_start_info in that case.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: [email protected]
--- drivers/xen/xenbus/xenbus_probe.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/xen/xenbus/xenbus_probe.c +2 -3
@@ -710,7 +710,7 @@ static int __init xenstored_local_init(void)if (!page) goto out_err; - xen_store_gfn = xen_start_info->store_mfn =virt_to_gfn ((void *)page); + xen_store_gfn =virt_to_gfn ((void *)page);/* Next allocate a local port which xenstored can bind to */ alloc_unbound.dom =DOMID_SELF ;
@@ -722,8 +722,7 @@ static int __init xenstored_local_init(void) goto out_err;BUG_ON (err); - xen_store_evtchn = xen_start_info->store_evtchn = - alloc_unbound.port; + xen_store_evtchn = alloc_unbound.port;return 0 ;
Signature · 1 line
2.17.0
PATCH 3/3 xen: share start flags between PV and PVH
In reply to Roger Pau MonneUse a global variable to store the start flags for both PV and PVH. This allows the xen_initial_domain macro to work properly on PVH.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: [email protected]
--- arch/x86/xen/enlighten.c | 7 +++++++ arch/x86/xen/enlighten_pv.c | 1 + arch/x86/xen/enlighten_pvh.c | 1 + include/xen/xen.h | 4 +++- 4 files changed, 12 insertions(+), 1 deletion(-)
arch/x86/xen/enlighten.c +7 -0
@@ -64,6 +64,13 @@ struct shared_info xen_dummy_shared_info;__read_mostly int xen_have_vector_callback;EXPORT_SYMBOL_GPL (xen_have_vector_callback); +/* +* NB: needs to live in .data because it's used by xen_prepare_pvh which runs +* before clearing the bss. +*/ +uint32_t xen_start_flags __attribute__((section(".data"))) = 0; +EXPORT_SYMBOL_GPL (xen_start_flags); +/* * Point at some empty memory to start with. We map the real shared_info * page as soon as fixmap is up and running.
arch/x86/xen/enlighten_pv.c +1 -0
@@ -1227,6 +1227,7 @@ asmlinkage __visible void __init xen_start_kernel(void)return ; xen_domain_type =XEN_PV_DOMAIN ; + xen_start_flags = xen_start_info->flags;xen_setup_features ();
arch/x86/xen/enlighten_pvh.c +1 -0
@@ -98,6 +98,7 @@ void __init xen_prepare_pvh(void) } xen_pvh =1 ; + xen_start_flags = pvh_start_info.flags;wrmsr_safe (MSR_MTRRdefType,0x800 |MTRR_TYPE_WRBACK ,0 );
include/xen/xen.h +3 -1
@@ -25,12 +25,14 @@ extern bool xen_pvh;#define xen_hvm_domain () (xen_domain_type == XEN_HVM_DOMAIN)#define xen_pvh_domain () (xen_pvh) +extern uint32_t xen_start_flags; +#ifdef CONFIG_XEN_DOM0 #include <xen/interface/xen.h> #include <asm/xen/hypervisor.h> #define xen_initial_domain () (xen_domain() && \ - xen_start_info && xen_start_info->flags & SIF_INITDOMAIN) + (xen_start_flags & SIF_INITDOMAIN))#else /* !CONFIG_XEN_DOM0 */ #define xen_initial_domain () (0)#endif /* CONFIG_XEN_DOM0 */
Signature · 7 lines
2.17.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/3 xen: share start flags between PV and PVH
In reply to Roger Pau MonneUse a global variable to store the start flags for both PV and PVH. This allows the xen_initial_domain macro to work properly on PVH.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: [email protected]
--- arch/x86/xen/enlighten.c | 7 +++++++ arch/x86/xen/enlighten_pv.c | 1 + arch/x86/xen/enlighten_pvh.c | 1 + include/xen/xen.h | 4 +++- 4 files changed, 12 insertions(+), 1 deletion(-)
arch/x86/xen/enlighten.c +7 -0
@@ -64,6 +64,13 @@ struct shared_info xen_dummy_shared_info;__read_mostly int xen_have_vector_callback;EXPORT_SYMBOL_GPL (xen_have_vector_callback); +/* +* NB: needs to live in .data because it's used by xen_prepare_pvh which runs +* before clearing the bss. +*/ +uint32_t xen_start_flags __attribute__((section(".data"))) = 0; +EXPORT_SYMBOL_GPL (xen_start_flags); +/* * Point at some empty memory to start with. We map the real shared_info * page as soon as fixmap is up and running.
arch/x86/xen/enlighten_pv.c +1 -0
@@ -1227,6 +1227,7 @@ asmlinkage __visible void __init xen_start_kernel(void)return ; xen_domain_type =XEN_PV_DOMAIN ; + xen_start_flags = xen_start_info->flags;xen_setup_features ();
arch/x86/xen/enlighten_pvh.c +1 -0
@@ -98,6 +98,7 @@ void __init xen_prepare_pvh(void) } xen_pvh =1 ; + xen_start_flags = pvh_start_info.flags;wrmsr_safe (MSR_MTRRdefType,0x800 |MTRR_TYPE_WRBACK ,0 );
include/xen/xen.h +3 -1
@@ -25,12 +25,14 @@ extern bool xen_pvh;#define xen_hvm_domain () (xen_domain_type == XEN_HVM_DOMAIN)#define xen_pvh_domain () (xen_pvh) +extern uint32_t xen_start_flags; +#ifdef CONFIG_XEN_DOM0 #include <xen/interface/xen.h> #include <asm/xen/hypervisor.h> #define xen_initial_domain () (xen_domain() && \ - xen_start_info && xen_start_info->flags & SIF_INITDOMAIN) + (xen_start_flags & SIF_INITDOMAIN))#else /* !CONFIG_XEN_DOM0 */ #define xen_initial_domain () (0)#endif /* CONFIG_XEN_DOM0 */
Signature · 1 line
2.17.0
kbuild test robot
build botRe: [PATCH 3/3] xen: share start flags between PV and PVH
Thank you for the patch! Yet something to improve:
Hi Roger,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v4.17-rc3 next-20180503]
[cannot apply to xen-tip/linux-next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Roger-Pau-Monne/xen-pvh-Dom0-support/20180504-102629
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All errors (new ones prefixed by >>):
In file included from arch/arm64/xen/../../arm/xen/enlighten.c:1:0:
>> include/xen/xen.h:28:8: error: unknown type name 'uint32_t'
extern uint32_t xen_start_flags;
^~~~~~~~vim +/uint32_t +28 include/xen/xen.h
27
> 28 extern uint32_t xen_start_flags;
29 ---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
kbuild test robot
build botRe: [PATCH 3/3] xen: share start flags between PV and PVH
In reply to Roger Pau MonneThank you for the patch! Yet something to improve:
Hi Roger,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v4.17-rc3 next-20180503]
[cannot apply to xen-tip/linux-next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Roger-Pau-Monne/xen-pvh-Dom0-support/20180504-102629
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All errors (new ones prefixed by >>):
In file included from arch/arm64/xen/../../arm/xen/enlighten.c:1:0:
>> include/xen/xen.h:28:8: error: unknown type name 'uint32_t'
extern uint32_t xen_start_flags;
^~~~~~~~vim +/uint32_t +28 include/xen/xen.h
27
> 28 extern uint32_t xen_start_flags;
29 ---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel