Credit2: fix per-socket runqueue setup

Hi everyone,

Code is already there in the credit2 scheduler, for setting up one runqueue per each socket found in the system, despite the fact that comments in sched_credit2.c still report this (well, something like this )to be a TODO item.

However, such code is not functional. In fact, on the following host:

cpu_topology :
cpu: core socket node

  0:       0        1        0
  1:       0        1        0
  2:       1        1        0
  3:       1        1        0
  4:       2        1        0
  5:       2        1        0
  6:       8        1        0
  7:       8        1        0
  8:       9        1        0
  9:       9        1        0
 10:      10        1        0
 11:      10        1        0
 12:       0        0        1
 13:       0        0        1
 14:       1        0        1
 15:       1        0        1
 16:       2        0        1
 17:       2        0        1
 18:       8        0        1
 19:       8        0        1
 20:       9        0        1
 21:       9        0        1
 22:      10        0        1
 23:      10        0        1
numa_info              :
node:    memsize    memfree    distances
   0:     18432      17488      10,20
   1:     19200      17932      20,10

What I see during boot and scheduler initialization is as follows (for all the 24 pCPUs):

(XEN) init_pcpu: Strange, cpu 1 already initialized! (XEN) init_pcpu: Strange, cpu 2 already initialized!
...
(XEN) init_pcpu: Strange, cpu 23 already initialized!

Also, once the system booted, here's what happened:

root@tg03:~# xl dmesg |grep -i runqueue
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Adding cpu 1 to runqueue 0
(XEN) Adding cpu 2 to runqueue 0
...
(XEN) Adding cpu 23 to runqueue 0

I.e., only one runqueue was created, despite the host having 2 sockets, and all the pCPUs have been assigned to it.

This series fixes that by doing the following three things:
- anticipate the identification of the topology of the boot CPU to happen _before_ the scheduler's initialization. This is necessary to make sure that, when credit2 will try to figure out on which socket CPU 0 it, it will find the correct info (patch 1);
- properly initialize the cpuinfo_x86 data structure, and more specifically the phys_proc_id field, that hosts the socket, to something invalid (-1). In fact, having that field in all elements set to 0 would induce credit2 to think that the pCPU have already been initialized, and that all are on socket 0 (patch 1);
- use the info made available by early boot CPU identification in credit2 (patch 2).

After this series is applied, no more "Strange, cpu X already initialized!" can be found in the logs. Also, here's how the scheduler is being setup:

root@tg03:~# xl dmesg |grep -i runqueue
(XEN) Adding cpu 0 to runqueue 1
(XEN)  First cpu on runqueue, activating
(XEN) Adding cpu 1 to runqueue 1
(XEN) Adding cpu 2 to runqueue 1
(XEN) Adding cpu 3 to runqueue 1
(XEN) Adding cpu 4 to runqueue 1
(XEN) Adding cpu 5 to runqueue 1
(XEN) Adding cpu 6 to runqueue 1
(XEN) Adding cpu 7 to runqueue 1
(XEN) Adding cpu 8 to runqueue 1
(XEN) Adding cpu 9 to runqueue 1
(XEN) Adding cpu 10 to runqueue 1
(XEN) Adding cpu 11 to runqueue 1
(XEN) Adding cpu 12 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Adding cpu 13 to runqueue 0
(XEN) Adding cpu 14 to runqueue 0
(XEN) Adding cpu 15 to runqueue 0
(XEN) Adding cpu 16 to runqueue 0
(XEN) Adding cpu 17 to runqueue 0
(XEN) Adding cpu 18 to runqueue 0
(XEN) Adding cpu 19 to runqueue 0
(XEN) Adding cpu 20 to runqueue 0
(XEN) Adding cpu 21 to runqueue 0
(XEN) Adding cpu 22 to runqueue 0
(XEN) Adding cpu 23 to runqueue 0

Which makes a lot more sense. :-)

Regards,
Dario

---
Dario Faggioli (2):
x86: during boot, anticipate identifying the boot cpu
sched: credit2: use boot CPU info for CPU #0

 xen/arch/x86/setup.c            |    8 ++++++--
 xen/arch/x86/smpboot.c          |    3 ++-
 xen/common/sched_credit2.c      |   12 +++++-------
 xen/include/asm-x86/processor.h |    6 ++++--
 4 files changed, 17 insertions(+), 12 deletions(-)

--
s happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)