Skip to contentSkip to conversation

Running X server in domU?

PATCH In review Mark Hurenkamp 3 participants 3 messages Jul 8, 2005 – Jul 9, 2005

lore
Share & tools
lore
#0 lore raw

Hi Sean,

I've been strugling along that path, but have had little time to work on it recently, so I don't have any progress to report.

I did manage to get one step beyond where you are now, meaning, I got a version of X patched to work without a VT.
(recent debian based distro's seem to include the patch by default, I've moved to using Ubuntu Hoary, which uses xorg 6.8.2-10 and has said patch included).

Next problem I ran into was some IO access problem. I have not had the chance to investigate that further, but I got a hint from Ian about patching Xen to allow this IO (not sure how safe that is though). I was also thinking that, since X does not really have a need for the IO access, perhaps X could be patched (was planning to try with fbdev driver, since that should not have any need for other IO).
See this post:
http://article.gmane.org/gmane.comp.emulators.xen.devel/3013/match=hurenkamp

I don't know when I will have more time to investigate, it can be a very time consuming issue to track, since it probably involves several rebuilds of Xorg and/or Xen.

Anyways, I wish you good luck, and am very interested if you have something of interest to report :-).
When I have some more info, I'll also report back here.

Mark.

Quoted 3 lines

Hi,

I posted a query about my problems running X servers from user domains

in Xen back in April:

Quoted 4 lines

http://lists.xensource.com/archives/html/xen-devel/2005-04/msg00651.html

As I recall X was being actively developed at that time to remove its

unnecessary assumptions about having direct PCI access which was thought to be the key to progress, with patches apparently available. Does anybody know the current status or have any further experience of this please?

Quoted 4 lines

Cheers,

Sean.

#1 lore raw

PATCH 1/1 per-cpu time changes

Here is another set of patches for the per-cpu timer changes to xen and linux. The patch is against today's xen-unstable-src.tgz. This fixes the occasional panic I was seeing on booting a slower system, and the constant panic booting a fast system. It has been tested UP and SMP on an IBM summit system, and an IntelliStation. The 'time went backwards' messages are no longer seen.

A previous version of this patch was tested by
[email protected].

xen/common/schedule.c renamed from xen/common/schedule.c.orig +18 -0
@@ -53,6 +53,7 @@ string_param("sched", opt_sched);
 /* Various timer handlers. */
 static void s_timer_fn(void *unused);
 static void t_timer_fn(void *unused);
+static void tsc_timer_fn(void *unused);
 static void dom_timer_fn(void *data);
 
 /* This is global for now so that private implementations can reach it */
@@ -76,6 +77,7 @@ static struct scheduler ops;
 
 /* Per-CPU periodic timer sends an event to the currently-executing domain. */
 static struct ac_timer t_timer[NR_CPUS]; 
+static struct ac_timer tsc_timer[NR_CPUS]; 
 
 void free_domain_struct(struct domain *d)
 {
@@ -531,6 +533,7 @@ int idle_cpu(int cpu)
  * Timers: the scheduler utilises a number of timers
  * - s_timer: per CPU timer for preemption and scheduling decisions
  * - t_timer: per CPU periodic timer to send timer interrupt to current dom
+ * - tsc_timer: per CPU periodic timer to update time bases
  * - dom_timer: per domain timer to specifiy timeout values
  ****************************************************************************/
 
@@ -560,6 +563,17 @@ static void t_timer_fn(void *unused)
     set_ac_timer(&t_timer[cpu], NOW() + MILLISECS(10));
 }
 
+/* Periodic tick timer: update time bases for per-cpu timing. */
+static void tsc_timer_fn(void *unused)
+{
+    unsigned int        cpu = current->processor;
+
+    extern void percpu_ticks(void);
+    percpu_ticks();
+
+    set_ac_timer(&tsc_timer[cpu], NOW() + MILLISECS(250));
+}
+
 /* Domain timer function, sends a virtual timer interrupt to domain */
 static void dom_timer_fn(void *data)
 {
@@ -581,6 +595,7 @@ void __init scheduler_init(void)
         spin_lock_init(&schedule_data[i].schedule_lock);
         init_ac_timer(&schedule_data[i].s_timer, s_timer_fn, NULL, i);
         init_ac_timer(&t_timer[i], t_timer_fn, NULL, i);
+        init_ac_timer(&tsc_timer[i], tsc_timer_fn, NULL, i);
     }
 
     schedule_data[0].curr = idle_task[0];
@@ -610,6 +625,9 @@ void schedulers_start(void) 
 {   
     t_timer_fn(0);
     smp_call_function((void *)t_timer_fn, NULL, 1, 1);
+
+    tsc_timer_fn(0);
+    smp_call_function((void *)tsc_timer_fn, NULL, 1, 1);
 }
 
 void dump_runq(unsigned char key)
xen/arch/x86/smpboot.c renamed from xen/arch/x86/smpboot.c.orig +2 -0
@@ -433,6 +433,7 @@ void __init start_secondary(void *unused
 
 	extern void percpu_traps_init(void);
 	extern void cpu_init(void);
+	extern void setup_percpu_time(void);
 
 	set_current(idle_task[cpu]);
 	set_processor_id(cpu);
@@ -454,6 +455,7 @@ void __init start_secondary(void *unused
 
 	setup_secondary_APIC_clock();
 	enable_APIC_timer();
+	setup_percpu_time();
 
 	/*
 	 * low-memory mappings have been cleared, flush them from
linux-2.6.11-xen-sparse/arch/xen/i386/kernel/time.c renamed from linux-2.6.11-xen-sparse/arch/xen/i386/kernel/time.c.orig +44 -47

243 lines not on this page · raw message

linux-2.6.11-xen-sparse/arch/xen/i386/kernel/timers/timer_tsc.c renamed from linux-2.6.11-xen-sparse/arch/xen/i386/kernel/timers/timer_tsc.c.orig +26 -9
@@ -10,6 +10,7 @@
 #include <linux/cpufreq.h>
 #include <linux/string.h>
 #include <linux/jiffies.h>
+#include <linux/percpu.h>
 
 #include <asm/timer.h>
 #include <asm/io.h>
@@ -35,8 +36,8 @@ extern spinlock_t i8253_lock;
 
 static int use_tsc;
 
-static unsigned long long monotonic_base;
-static u32 monotonic_offset;
+static DEFINE_PER_CPU(unsigned long long, monotonic_base);
+static DEFINE_PER_CPU(u64, monotonic_offset);
 static seqlock_t monotonic_lock = SEQLOCK_UNLOCKED;
 
 /* convert from cycles(64bits) => nanoseconds (64bits)
@@ -75,18 +76,32 @@ static inline unsigned long long cycles_
 static unsigned long fast_gettimeoffset_quotient;
 
 extern u32 shadow_tsc_stamp;
-extern u64 shadow_system_time;
+extern DEFINE_PER_CPU(u64, shadow_tsc_stamp);
+extern DEFINE_PER_CPU(u64, shadow_system_time);
+
+unsigned long long get_full_tsc_offset(void)
+{
+	unsigned long long tsc;
+
+	/* Read the Time Stamp Counter */
+	rdtscll(tsc);
+
+	tsc -= per_cpu(shadow_tsc_stamp, smp_processor_id());
+
+	return cycles_2_ns(tsc);
+}
 
 static unsigned long get_offset_tsc(void)
 {
 	register unsigned long eax, edx;
+unsigned long long tsc;
 
 	/* Read the Time Stamp Counter */
 
-	rdtsc(eax,edx);
+	rdtscll(tsc);
 
 	/* .. relative to previous jiffy (32 bits is enough) */
-	eax -= shadow_tsc_stamp;
+	eax = tsc - per_cpu(shadow_tsc_stamp, smp_processor_id());
 
 	/*
          * Time offset = (tsc_low delta) * fast_gettimeoffset_quotient
@@ -110,12 +125,13 @@ static unsigned long long monotonic_cloc
 {
 	unsigned long long last_offset, this_offset, base;
 	unsigned seq;
+	int cpu = smp_processor_id();
 	
 	/* atomically read monotonic base & last_offset */
 	do {
 		seq = read_seqbegin(&monotonic_lock);
-		last_offset = monotonic_offset;
-		base = monotonic_base;
+		last_offset = per_cpu(monotonic_offset, cpu);
+		base = per_cpu(monotonic_base, cpu);
 	} while (read_seqretry(&monotonic_lock, seq));
 
 	/* Read the Time Stamp Counter */
@@ -152,11 +168,12 @@ unsigned long long sched_clock(void)
 
 static void mark_offset_tsc(void)
 {
+	int cpu = smp_processor_id();
 
 	/* update the monotonic base value */
 	write_seqlock(&monotonic_lock);
-	monotonic_base = shadow_system_time;
-	monotonic_offset = shadow_tsc_stamp;
+	per_cpu(monotonic_base, cpu) = per_cpu(shadow_system_time, cpu);
+	per_cpu(monotonic_offset, cpu) = per_cpu(shadow_tsc_stamp, cpu);
 	write_sequnlock(&monotonic_lock);
 }
 
xen/arch/x86/time.c renamed from xen/arch/x86/time.c.orig +72 -2

140 lines not on this page · raw message

xen/include/public/xen.h renamed from xen/include/public/xen.h.orig +16 -0
@@ -330,6 +330,21 @@ typedef struct vcpu_info {
 } vcpu_info_t;
 
 /*
+ * Xen/kernel shared data
+ *  per cpu timing information.
+ */
+typedef struct time_info_st
+{
+    u32			time_version1;
+    u32			time_version2;
+    tsc_timestamp_t	tsc_timestamp;	/* TSC at last update */
+    u64			system_time;	/* time, in nanoseconds, since boot */
+    u64			cpu_freq;	/* CPU frequency (Hz) */
+    u32			wc_sec;		/* Secs  00:00:00 UTC,  Jan 1, 1970. */
+    u32			wc_usec;	/* Usecs 00:00:00 UTC,  Jan 1, 1970. */
+} time_info_t;
+
+/*
  * Xen/kernel shared data -- pointer provided in start_info.
  * NB. We expect that this struct is smaller than a page.
  */
@@ -403,6 +418,7 @@ typedef struct shared_info {
 
     arch_shared_info_t arch;
 
+    time_info_t vcpu_time[MAX_VIRT_CPUS];
 } shared_info_t;
 
 /*
Signature · 2 lines
Don Fry
[email protected]
#2 lore raw

Re: [PATCH] per-cpu time changes

Quoted 6 lines · Don Fry

Here is another set of patches for the per-cpu timer changes to xen and linux. The patch is against today's xen-unstable-src.tgz. This fixes the occasional panic I was seeing on booting a slower system, and the constant panic booting a fast system. It has been tested UP and SMP on an IBM summit system, and an IntelliStation. The 'time went backwards' messages are no longer seen.

Great! I'm cooking up a time patch to check in over the next few days -- it should handle the one remaining problem I see with your patch, that although 'guest time' will be monotonic on each individual cpu, they will diverge over time because percpu_ticks does not update local time w.r.t. a shared timebase. I think solving that necessarily means some bigger changes to how time is handled in Xen, and the interface to guests, and I'm currently merging those changes into the xenlinux tree (the Xen side is done and I can post an interim patch if that's interesting for anyone).

-- Keir