Skip to contentSkip to conversation

xen: Fix possible user space selector corruption

PATCH In review Frediano Ziglio 4 participants 17 messages Oct 3, 2013 – Oct 7, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen: Fix possible user space selector corruption

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}
Signature · 1 line
1.7.10.4
#1 lore raw
Quoted 21 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

Looks okay to me but I don't really understand it...

David

#2 lore raw
In reply to Frediano Ziglio
Quoted 21 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

Looks okay to me but I don't really understand it...

David

Reviewed-by on v1 1/1
#3 lore raw
In reply to Frediano Ziglio
Quoted 21 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Quoted 27 lines
---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}
#4 lore raw
Quoted 30 lines · Andrew Cooper
Quoted 21 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

Quoted 9 lines

In terms of the correctness of the fix,

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Suggestion for the comment ??

Frediano

Quoted 28 lines
Quoted 27 lines
---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}
#5 lore raw
Quoted 26 lines · Frediano Ziglio
Quoted 25 lines · Andrew Cooper
Quoted 24 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

Should this also go in stable tree?

Quoted 41 lines
Quoted 6 lines

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Suggestion for the comment ??

Frediano

Quoted 28 lines
Quoted 27 lines
---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}
#6 lore raw
Quoted 4 lines · Konrad Rzeszutek Wilk
Quoted 3 lines · Frediano Ziglio
Quoted 2 lines · Andrew Cooper
Quoted 1 line · Frediano Ziglio
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

Should this also go in stable tree?

Very much so. The change which exposed it for us was from 3.7 iirc, but I believe it has been a latent bug for as long as the native early boot code uses __USER_DS.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Suggestion for the comment ??

Frediano

My suggestion was to omit the comment entirely, or simplify it to just:

/* Xen starts us with XEN_FLAT_RING1_DS, but linux code expects __USER_DS */

Anyone who wants the full explanation can read the patch description.

~Andrew

#7 lore raw
In reply to Konrad Rzeszutek Wilk
Quoted 4 lines · Konrad Rzeszutek Wilk
Quoted 3 lines · Frediano Ziglio
Quoted 2 lines · Andrew Cooper
Quoted 1 line · Frediano Ziglio
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

Should this also go in stable tree?

Very much so. The change which exposed it for us was from 3.7 iirc, but I believe it has been a latent bug for as long as the native early boot code uses __USER_DS.

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Suggestion for the comment ??

Frediano

My suggestion was to omit the comment entirely, or simplify it to just:

/* Xen starts us with XEN_FLAT_RING1_DS, but linux code expects __USER_DS */

Anyone who wants the full explanation can read the patch description.

~Andrew

#8 lore raw
Quoted 23 lines · Andrew Cooper
Quoted 4 lines · Konrad Rzeszutek Wilk
Quoted 3 lines · Frediano Ziglio
Quoted 2 lines · Andrew Cooper
Quoted 1 line · Frediano Ziglio
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

Should this also go in stable tree?

Very much so. The change which exposed it for us was from 3.7 iirc, but I believe it has been a latent bug for as long as the native early boot code uses __USER_DS.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

Suggestion for the comment ??

Frediano

My suggestion was to omit the comment entirely, or simplify it to just:

Suggesting comments should be omitted because other code is poorly commented seems odd to me.

Quoted 1 line

/* Xen starts us with XEN_FLAT_RING1_DS, but linux code expects __USER_DS */

I do think this comment better summarizes the reason for loading the segment registers so I'd prefer this.

David

#9 lore raw

PATCH v2 1/1 xen: Fix possible user space selector corruption

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

---
 arch/x86/xen/smp.c |    9 +++++++++
 1 file changed, 9 insertions(+)

Just changed comment on source code as suggested by Andrew Cooper.

arch/x86/xen/smp.c +9 -0
index d99cae8..6d89fcc 100644
@@ -245,6 +245,15 @@ static void __init xen_smp_prepare_boot_cpu(void)
 	   old memory can be recycled */
 	make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+	/*
+	 * Xen starts us with XEN_FLAT_RING1_DS, but linux code
+	 * expects __USER_DS
+	 */
+	loadsegment(ds, __USER_DS);
+	loadsegment(es, __USER_DS);
+#endif
+
 	xen_filter_cpu_maps();
 	xen_setup_vcpu_info_placement();
 }
Signature · 1 line
1.7.10.4
Reviewed-by on v2 1/1
#10 lore raw
Quoted 20 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

Quoted 28 lines
---
 arch/x86/xen/smp.c |    9 +++++++++
 1 file changed, 9 insertions(+)

Just changed comment on source code as suggested by Andrew Cooper.

arch/x86/xen/smp.c +9 -0
index d99cae8..6d89fcc 100644
@@ -245,6 +245,15 @@ static void __init xen_smp_prepare_boot_cpu(void)
 	   old memory can be recycled */
 	make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+	/*
+	 * Xen starts us with XEN_FLAT_RING1_DS, but linux code
+	 * expects __USER_DS
+	 */
+	loadsegment(ds, __USER_DS);
+	loadsegment(es, __USER_DS);
+#endif
+
 	xen_filter_cpu_maps();
 	xen_setup_vcpu_info_placement();
 }
Reviewed-by on v2 1/1
#11 lore raw
In reply to Frediano Ziglio
Quoted 20 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

Quoted 28 lines
---
 arch/x86/xen/smp.c |    9 +++++++++
 1 file changed, 9 insertions(+)

Just changed comment on source code as suggested by Andrew Cooper.

arch/x86/xen/smp.c +9 -0
index d99cae8..6d89fcc 100644
@@ -245,6 +245,15 @@ static void __init xen_smp_prepare_boot_cpu(void)
 	   old memory can be recycled */
 	make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+	/*
+	 * Xen starts us with XEN_FLAT_RING1_DS, but linux code
+	 * expects __USER_DS
+	 */
+	loadsegment(ds, __USER_DS);
+	loadsegment(es, __USER_DS);
+#endif
+
 	xen_filter_cpu_maps();
 	xen_setup_vcpu_info_placement();
 }
#12 lore raw

PATCH v2 1/1 xen: Fix possible user space selector corruption

In reply to David Vrabel

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

---
 arch/x86/xen/smp.c |    9 +++++++++
 1 file changed, 9 insertions(+)

Just changed comment on source code as suggested by Andrew Cooper.

arch/x86/xen/smp.c +9 -0
index d99cae8..6d89fcc 100644
@@ -245,6 +245,15 @@ static void __init xen_smp_prepare_boot_cpu(void)
 	   old memory can be recycled */
 	make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+	/*
+	 * Xen starts us with XEN_FLAT_RING1_DS, but linux code
+	 * expects __USER_DS
+	 */
+	loadsegment(ds, __USER_DS);
+	loadsegment(es, __USER_DS);
+#endif
+
 	xen_filter_cpu_maps();
 	xen_setup_vcpu_info_placement();
 }
Signature · 1 line
1.7.10.4
#13 lore raw
In reply to Andrew Cooper
Quoted 23 lines · Andrew Cooper
Quoted 4 lines · Konrad Rzeszutek Wilk
Quoted 3 lines · Frediano Ziglio
Quoted 2 lines · Andrew Cooper
Quoted 1 line · Frediano Ziglio
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

Should this also go in stable tree?

Very much so. The change which exposed it for us was from 3.7 iirc, but I believe it has been a latent bug for as long as the native early boot code uses __USER_DS.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

Suggestion for the comment ??

Frediano

My suggestion was to omit the comment entirely, or simplify it to just:

Suggesting comments should be omitted because other code is poorly commented seems odd to me.

Quoted 1 line

/* Xen starts us with XEN_FLAT_RING1_DS, but linux code expects __USER_DS */

I do think this comment better summarizes the reason for loading the segment registers so I'd prefer this.

David

#14 lore raw
In reply to Frediano Ziglio
Quoted 26 lines · Frediano Ziglio
Quoted 25 lines · Andrew Cooper
Quoted 24 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

Should this also go in stable tree?

Quoted 41 lines
Quoted 6 lines

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Suggestion for the comment ??

Frediano

Quoted 28 lines
Quoted 27 lines
---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}
#15 lore raw
In reply to Andrew Cooper
Quoted 30 lines · Andrew Cooper
Quoted 21 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

Quoted 9 lines

In terms of the correctness of the fix,

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Suggestion for the comment ??

Frediano

Quoted 28 lines
Quoted 27 lines
---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}
Reviewed-by on v1 1/1
#16 lore raw
In reply to Frediano Ziglio
Quoted 21 lines · Frediano Ziglio

Due to the way kernel is initialized under Xen is possible that the ring1 selector used by the kernel for the boot cpu end up to be copied to userspace leading to segmentation fault in the userspace.

Xen code in the kernel initialize no-boot cpus with correct selectors (ds and es set to __USER_DS) but the boot one keep the ring1 (passed by Xen). On task context switch (switch_to) we assume that ds, es and cs already point to __USER_DS and __KERNEL_CSso these selector are not changed.

If processor is an Intel that support sysenter instruction sysenter/sysexit is used so ds and es are not restored switching back from kernel to userspace. In the case the selectors point to a ring1 instead of __USER_DS the userspace code will crash on first memory access attempt (to be precise Xen on the emulated iret used to do sysexit will detect and set ds and es to zero which lead to GPF anyway).

Now if an userspace process call kernel using sysenter and get rescheduled (for me it happen on a specific init calling wait4) could happen that the ring1 selector is set to ds and es.

Quoted 7 lines

This is quite hard to detect cause after a while these selectors are fixed (__USER_DS seems sticky).

Bisecting the code commit 7076aada1040de4ed79a5977dbabdb5e5ea5e249 appears to be the first one that have this issue.

In terms of the correctness of the fix,

However, I am not sure the comment is necessary. The prevailing style is for no justification of loads of segment selectors on boot, and the comment itself refers simply to an interaction issue of 32bit on Xen when making use of sysenter.

Quoted 27 lines
---
 arch/x86/xen/smp.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/xen/smp.c +12 -0
index d1e4777..2a47241 100644
@@ -278,6 +278,18 @@ static void __init xen_smp_prepare_boot_cpu(void)
 		   old memory can be recycled */
 		make_lowmem_page_readwrite(xen_initial_gdt);
 
+#ifdef CONFIG_X86_32
+		/*
+		 * Assure we use segments with user level access.
+		 * During switching of task these segments got not reloaded
+		 * so it could happen that userspace tasks get Xen ring1
+		 * selector causing exit with sysenter failures on next
+		 * userspace memory operation.
+		 */
+		loadsegment(ds, __USER_DS);
+		loadsegment(es, __USER_DS);
+#endif
+
 		xen_filter_cpu_maps();
 		xen_setup_vcpu_info_placement();
 	}