2.5.17 fix for running a SMP kernel on a UP box

I can't seem to run a SMP 2.5.17 kernel on a UP machine, it locks up during the boot process. In talking to Jack Vogel, he suggested I make the following patch, which seems to solve the problem for me. In looking at the code, I have no idea of why this seems to work, so there probably is a better fix out there.

Any suggestions?

thanks,

greg k-h

diff -Nru a/arch/i386/kernel/io_apic.c b/arch/i386/kernel/io_apic.c
@@ -205,7 +205,7 @@
 } ____cacheline_aligned irq_balance_t;
 
 static irq_balance_t irq_balance[NR_IRQS] __cacheline_aligned
-			= { [ 0 ... NR_IRQS-1 ] = { 1, 0 } };
+			= { [ 0 ... NR_IRQS-1 ] = { 0, 0 } };
 
 extern unsigned long irq_affinity [NR_IRQS];
 

I was looking at this with Jack Vogel and I can't figure out how it goes wrong, either. However, the code in move() that uses the cpu number is a bit strange. Entering loops in their middles is generally considered bug-prone by programming style books. What about eliminating the goto by using something like the attached patch?

James Cleverdon
IBM xSeries Linux Solutions
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot com

*** linux/arch/i386/kernel/io_apic.c.df Mon May 20 22:07:36 2002
--- linux/arch/i386/kernel/io_apic.c Tue May 21 18:41:54 2002
***************
*** 203,213 ****
unsigned int cpu;
unsigned long timestamp;
} ____cacheline_aligned irq_balance_t;

static irq_balance_t irq_balance[NR_IRQS] __cacheline_aligned
! = { [ 0 ... NR_IRQS-1 ] = { 1, 0 } };

  extern unsigned long irq_affinity [NR_IRQS];
  
  #endif

--- 203,213 ----
unsigned int cpu;
unsigned long timestamp;
} ____cacheline_aligned irq_balance_t;

static irq_balance_t irq_balance[NR_IRQS] __cacheline_aligned
! = { [ 0 ... NR_IRQS-1 ] = { 0, 0 } };

  extern unsigned long irq_affinity [NR_IRQS];
  
  #endif

***************
*** 220,248 ****
static unsigned long move(int curr_cpu, unsigned long allowed_mask, unsigned long now, int direction) {
int search_idle = 1;
int cpu = curr_cpu;

! goto inside;
!
! do {
! if (unlikely(cpu == curr_cpu))
! search_idle = 0;
! inside:
! if (direction == 1) {
! cpu++;
! if (cpu >= smp_num_cpus)
cpu = 0;
} else {
! cpu--;
! if (cpu == -1)
! cpu = smp_num_cpus-1;
}
! } while (!IRQ_ALLOWED(cpu,allowed_mask) ||
! (search_idle && !IDLE_ENOUGH(cpu,now)));
!
! return cpu;
}

static inline void balance_irq(int irq)
{
#if CONFIG_SMP
--- 220,242 ----
static unsigned long move(int curr_cpu, unsigned long allowed_mask, unsigned long now, int direction) {
int search_idle = 1;
int cpu = curr_cpu;

! for (;;) {
! if (direction) {
! if (++cpu >= smp_num_cpus)
cpu = 0;
} else {
! if (--cpu < 0)
! cpu = smp_num_cpus - 1;
}
! if (IRQ_ALLOWED(cpu, allowed_mask) && (!search_idle || IDLE_ENOUGH(cpu, now)))
! return cpu;
! if (unlikely(cpu == curr_cpu))
! search_idle = 0;
! }
}

  static inline void balance_irq(int irq)
  {
  #if CONFIG_SMP

I should add one bit of information. I originally saw this problem on a UP IBM Netvista machine, its the same box Greg had it happen on.

However, I have a 933Mhz PIII HP box at home and it does not have the problem.

Since we are limited in the variety of machines to test on I am not sure about this, but I believe its only going to occur on UP systems with an IOAPIC.

If you apply the irq_balance patch to a 2.4.* kernel you can recreate the same hang, in fact it was on 2.4.18 that i first ran into it.

I realize its kinda a corner case, running an SMP kernel on a subset of UP machines, but hey, I figure its supposed to work :)

Jack F. Vogel
IBM  Linux Solutions
[email protected]  (work)
[email protected] (home)