CONFIG_PREEMPT_REALTIME, 'Fully Preemptible Kernel', VP-2.6.9-rc4-mm1-T4

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T4

I would have to say this is "very rough" at this point. I had the following problems in the build:
[1] kernel/ksyms.c - undefined symbols
[2] kernel/mutex.c - obvious cut / paste problems
[3] XFS has incompatible mutex definition
[4] suspicious warnings
[5] missing symbols for modules
Details at the end.

I booted w/ SMP and the machine threw a lot of error messages about sleeping in
an invalid context. For example:

include/linux/rwsem.h:43
in_atomic():1 [00010001], irqs_disabled():1

[<c011f0ea>] __might_sleep+0xca/0xe0
[<c01390d4>] rw_mutex_read_lock+0x34/0x50
[<c0122dbd>] profile_hook+0x1d/0x50
[<c0123338>] profile_tick+0x68/0x70
[<c01150ad>] smp_apic_timer_interrupt+0x5d/0xf0
[<c0105820>] default_idle+0x0/0x40
[<c010854a>] apic_timer_interrupt+0x1a/0x20
[<c0105820>] default_idle+0x0/0x40
[<c011007b>] dmi_get_system_info+0xb/0x20
[<c010585a>] default_idle+0x3a/0x40
[<c03b4a4d>] start_kernel+0x19d/0x1e0
[<c03b4440>] unknown_bootoption+0x0/0x180

(somehow managed to stop the scrolling console with the above message displayed...)

Finally died with a kernel BUG
kernel BUG at kernel/latenc.c:419!
invalid operand: 0000 [#1]
PREEMPT SMP
Modules linked in microcode dm_mod uhci_hcd ext3 jbd

CPU:     1
EIP:    0001:[<00000000>]    Not tainted VLI
EFLAGS: c1663f38  (2.6.9-rc4-mm1-VP-T4)

EIP is at 0x0
eax: 00000000 ebx: c1663f54 ecx: c0109a8c edx: c1663f78
esi: 0000000c edi: c1663f28 ebp: c1663f3c esp: c1663f78
ds: 007b es: 07b ss: 4f03 preempt: 00000001
Process swapper (pid: 0, threadinfo=c1662000 task=c165e550)
<0> Kernel panic - not syncing: Attempted to kill the idle task!

Rebooting with num_cpus=1 and appeared to make it farther but then the console scrolled like crazy and finally said "console shuts up ..." and the machine appeared to be hung. Could not scroll the window up or down to see the full message. Had to power off / on to get the machine back up. Going back to -T3 until I see some fixes.

If the machine managed to record some good data in /var/log/messages I will send them separately to you for reference.

--Mark H Johnson
<mailto:[email protected]>

Details on build problems / work arounds follow:

[1] ksyms.c - I commented these lines out, to get a complete build, but there appears to
be code that expects rtc_lock to be defined. See #5.

arch/i386/kernel/i386_ksyms.c:166 error: `rtc_lock' undeclared here (not in a function)
arch/i386/kernel/i386_ksyms.c:166 error: initializer element is not constant
arch/i386/kernel/i386_ksyms.c:166 error: (near initialization for `__ksymtab_rtc_lock.value')
followed by a similar error for atomic_dec_and_lock at line 177.

[2] mutex.c - several symbols were defined twice, fixed by changing the names to
the functions preceeding them. See lines 108, 201, 213, 297.

[3] XFS compile failed as follows:
CC [M] fs/xfs/quota/xfs_dquot.o
In file included from fs/xfs/linux-2.6/xfs_linux.h:63,

                 from fs/xfs/xfs.h:35,
                 from fs/xfs/quota/xfs_dquot.c:33:
fs/xfs/linux-2.6/mutex.h:45: error: conflicting types for `mutex_t'
include/asm/spinlock.h:79: error: previous declaration of `mutex_t'
In file included from fs/xfs/linux-2.6/xfs_linux.h:102,
                 from fs/xfs/xfs.h:35,
                 from fs/xfs/quota/xfs_dquot.c:33:

fs/xfs/linux-2.6/xfs_vnode.h:578:30: macro "mutex_lock" requires 2 arguments, but only 1 given fs/xfs/linux-2.6/xfs_vnode.h:585:30: macro "mutex_lock" requires 2 arguments, but only 1 given
fs/xfs/quota/xfs_dquot.c:1327:23: macro "mutex_lock" requires 2 arguments, but only 1 given
fs/xfs/quota/xfs_dquot.c:1390:41: macro "mutex_lock" requires 2 arguments, but only 1 given
fs/xfs/linux-2.6/xfs_vnode.h: In function `vn_flagset':
fs/xfs/linux-2.6/xfs_vnode.h:578: warning: statement with no effect fs/xfs/linux-2.6/xfs_vnode.h: In function `vn_flagclr':
fs/xfs/linux-2.6/xfs_vnode.h:585: warning: statement with no effect Turned off XFS in the build.

[4] I considered the following warnings to be "suspicious" but am not sure if they are really problems or not.

CC security/selinux/ss/policydb.o
fs/dcache.c: In function `prune_dcache':
fs/dcache.c:391: warning: passing arg 1 of `cond_resched_lock' from incompatible pointer type
CC security/selinux/ss/services.o
CC fs/inode.o
fs/inode.c: In function `invalidate_list':
fs/inode.c:317: warning: passing arg 1 of `cond_resched_lock' from incompatible pointer type

[5] Several modules had undefined symbols. The messages were...

Kernel: arch/i386/boot/bzImage is ready
*** Warning: "mutex_trylock_bh" [drivers/net/ppp_synctty.ko] undefined!
*** Warning: "del_mtd_partitions" [drivers/mtd/maps/scx200_docflash.ko] undefined!
*** Warning: "add_mtd_partitions" [drivers/mtd/maps/scx200_docflash.ko] undefined!
*** Warning: "i2o_msg_in_to_virt" [drivers/message/i2o/i2o_scsi.ko] undefined!
*** Warning: "i2o_msg_out_to_virt" [drivers/message/i2o/i2o_core.ko] undefined!
*** Warning: "i2o_msg_in_to_virt" [drivers/message/i2o/i2o_core.ko] undefined!
*** Warning: "i2o_msg_in_to_virt" [drivers/message/i2o/i2o_block.ko] undefined!
*** Warning: "rtc_lock" [drivers/char/nvram.ko] undefined!
*** Warning: "rtc_lock" [drivers/char/mwave/mwave.ko] undefined!
*** Warning: "rtc_lock" [drivers/block/floppy.ko] undefined!
...
if [ -r System.map ]; then /sbin/depmod -ae -F System.map -b /var/tmp/kernel-2.6.9rc4mm1VPT4-root -r 2.6.9-rc4-mm1-VP-T4; fi
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/net/ppp_synctty.ko
needs unknown symbol mutex_trylock_bh
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/mtd/maps/scx200_docflash.ko
needs unknown symbol del_mtd_partitions
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/mtd/maps/scx200_docflash.ko
needs unknown symbol add_mtd_partitions
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/message/i2o/i2o_scsi.ko
needs unknown symbol i2o_msg_in_to_virt
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/message/i2o/i2o_core.ko
needs unknown symbol i2o_msg_in_to_virt
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/message/i2o/i2o_core.ko
needs unknown symbol i2o_msg_out_to_virt
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/message/i2o/i2o_block.ko
needs unknown symbol i2o_msg_in_to_virt
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/char/nvram.ko
needs unknown symbol rtc_lock
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/char/mwave/mwave.ko
needs unknown symbol rtc_lock
WARNING: /var/tmp/kernel-2.6.9 rc4mm1VPT4-root/lib/modules/2.6.9-rc4-mm1-VP-T4/kernel/drivers/block/floppy.ko
needs unknown symbol rtc_lock

i've uploaded -T5 which should fix most of the build issues:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T5

CONFIG_PREEMPT_REALTIME is still an experimental feature and defaults to 'n'.

-T5 will likely not fix the exit.c warnings, which, unless they are accompanied by real crashes, should be mostly harmless. (famous last words.) (The zombie task and self-reaping thread handling is a really hard nut to crack, and i have nobody but me to blame for that code ...)

Ingo

hi,

i still can't build it. Fist i reverse applied T4, then applied T5 and tried a make bzImage. I'll try from scratch though to make sure, cause these errors look identical to the T4 ones.

  CLEAN   .tmp_versions
  CHK     include/linux/version.h
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/basic/split-include
  HOSTCC  scripts/basic/docproc
  HOSTCC  scripts/genksyms/genksyms.o
  HOSTCC  scripts/genksyms/lex.o
  HOSTCC  scripts/genksyms/parse.o
  HOSTLD  scripts/genksyms/genksyms
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/modpost.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/conmakehash
  HOSTCC  scripts/bin2c
  CC      arch/i386/kernel/asm-offsets.s
In file included from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/spinlock.h:419: error: parse error before '*' token include/linux/spinlock.h:419: warning: function declaration isn't a prototype include/linux/spinlock.h:420: error: parse error before '*' token include/linux/spinlock.h:420: warning: function declaration isn't a prototype include/linux/spinlock.h:421: error: parse error before '*' token include/linux/spinlock.h:421: warning: function declaration isn't a prototype include/linux/spinlock.h:422: error: parse error before '*' token include/linux/spinlock.h:422: warning: function declaration isn't a prototype include/linux/spinlock.h:423: error: parse error before '*' token include/linux/spinlock.h:423: warning: function declaration isn't a prototype include/linux/spinlock.h:424: error: parse error before '*' token include/linux/spinlock.h:424: warning: function declaration isn't a prototype include/linux/spinlock.h:425: error: parse error before '*' token include/linux/spinlock.h:425: warning: function declaration isn't a prototype include/linux/spinlock.h:426: error: parse error before '*' token include/linux/spinlock.h:426: warning: function declaration isn't a prototype include/linux/spinlock.h:427: error: parse error before '*' token include/linux/spinlock.h:427: warning: function declaration isn't a prototype include/linux/spinlock.h:428: error: parse error before '*' token include/linux/spinlock.h:428: warning: function declaration isn't a prototype include/linux/spinlock.h:429: error: parse error before '*' token include/linux/spinlock.h:429: warning: function declaration isn't a prototype include/linux/spinlock.h:430: error: parse error before '*' token include/linux/spinlock.h:430: warning: function declaration isn't a prototype include/linux/spinlock.h:431: error: parse error before '*' token include/linux/spinlock.h:431: warning: function declaration isn't a prototype include/linux/spinlock.h:467: error: parse error before '*' token include/linux/spinlock.h:467: warning: function declaration isn't a prototype include/linux/spinlock.h:468: error: parse error before '*' token include/linux/spinlock.h:468: warning: function declaration isn't a prototype include/linux/spinlock.h:469: error: parse error before '*' token include/linux/spinlock.h:469: warning: function declaration isn't a prototype include/linux/spinlock.h:470: error: parse error before '*' token include/linux/spinlock.h:470: warning: function declaration isn't a prototype include/linux/spinlock.h:471: error: parse error before '*' token include/linux/spinlock.h:471: warning: function declaration isn't a prototype include/linux/spinlock.h:472: error: parse error before '*' token include/linux/spinlock.h:472: warning: function declaration isn't a prototype include/linux/spinlock.h:473: error: parse error before '*' token include/linux/spinlock.h:473: warning: function declaration isn't a prototype include/linux/spinlock.h:474: error: parse error before '*' token include/linux/spinlock.h:474: warning: function declaration isn't a prototype include/linux/spinlock.h:475: error: parse error before '*' token include/linux/spinlock.h:475: warning: function declaration isn't a prototype include/linux/spinlock.h:476: error: parse error before '*' token include/linux/spinlock.h:476: warning: function declaration isn't a prototype include/linux/spinlock.h:477: error: parse error before '*' token include/linux/spinlock.h:477: warning: function declaration isn't a prototype include/linux/spinlock.h:478: error: parse error before '*' token include/linux/spinlock.h:478: warning: function declaration isn't a prototype include/linux/spinlock.h:479: error: parse error before '*' token include/linux/spinlock.h:479: warning: function declaration isn't a prototype include/linux/spinlock.h:480: error: parse error before '*' token include/linux/spinlock.h:480: warning: function declaration isn't a prototype include/linux/spinlock.h:481: error: parse error before '*' token include/linux/spinlock.h:481: warning: function declaration isn't a prototype include/linux/spinlock.h:482: error: parse error before '*' token include/linux/spinlock.h:482: warning: function declaration isn't a prototype include/linux/spinlock.h:483: error: parse error before '*' token include/linux/spinlock.h:483: warning: function declaration isn't a prototype include/linux/spinlock.h:484: error: parse error before '*' token include/linux/spinlock.h:484: warning: function declaration isn't a prototype include/linux/spinlock.h:485: error: parse error before '*' token include/linux/spinlock.h:485: warning: function declaration isn't a prototype include/linux/spinlock.h:486: error: parse error before '*' token include/linux/spinlock.h:486: warning: function declaration isn't a prototype In file included from include/linux/capability.h:45,

                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/spinlock.h:543:1: warning: "spin_lock_init" redefined include/linux/spinlock.h:208:1: warning: this is the location of the previous definition include/linux/spinlock.h:548:1: warning: "rwlock_init" redefined include/linux/spinlock.h:226:1: warning: this is the location of the previous definition include/linux/spinlock.h:553:1: warning: "spin_is_locked" redefined include/linux/spinlock.h:210:1: warning: this is the location of the previous definition include/linux/spinlock.h:563:1: warning: "spin_unlock_wait" redefined include/linux/spinlock.h:212:1: warning: this is the location of the previous definition In file included from include/linux/capability.h:45,

                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/linux/spinlock.h:669: error: parse error before "raw_spinlock_t"
include/linux/spinlock.h:669: warning: function declaration isn't a prototype
In file included from include/linux/time.h:7,
                 from include/linux/timex.h:58,
                 from include/linux/sched.h:11,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/seqlock.h:35: error: parse error before "raw_spinlock_t" include/linux/seqlock.h:35: warning: no semicolon at end of struct or union include/linux/seqlock.h:36: warning: type defaults to int' in declaration of seqlock_t' include/linux/seqlock.h:36: warning: data definition has no type or storage class include/linux/seqlock.h:50: error: parse error before '*' token include/linux/seqlock.h:51: warning: function declaration isn't a prototype include/linux/seqlock.h: In function `write_seqlock':
include/linux/seqlock.h:52: error: `sl' undeclared (first use in this function) include/linux/seqlock.h:52: error: (Each undeclared identifier is reported only once include/linux/seqlock.h:52: error: for each function it appears in.) include/linux/seqlock.h:52: error: parse error before "raw_spinlock_t" include/linux/seqlock.h:52: error: `raw_spinlock_t' undeclared (first use in this function) include/linux/seqlock.h:52: error: parse error before ')' token include/linux/seqlock.h: At top level:
include/linux/seqlock.h:52: error: parse error before "while" include/linux/seqlock.h:57: error: parse error before '*' token include/linux/seqlock.h:58: warning: function declaration isn't a prototype include/linux/seqlock.h: In function `write_sequnlock':
include/linux/seqlock.h:60: error: `sl' undeclared (first use in this function) include/linux/seqlock.h:61: error: parse error before "raw_spinlock_t" include/linux/seqlock.h: At top level:
include/linux/seqlock.h:61: error: parse error before "while" include/linux/seqlock.h:64: error: parse error before '*' token include/linux/seqlock.h:65: warning: function declaration isn't a prototype include/linux/seqlock.h: In function `write_tryseqlock':
include/linux/seqlock.h:66: error: sl' undeclared (first use in this function) include/linux/seqlock.h:66: error: parse error before "raw_spinlock_t" include/linux/seqlock.h:66: warning: unused variable __ret' include/linux/seqlock.h:66: error: parse error before "while" include/linux/seqlock.h:66: error: `raw_spinlock_t' undeclared (first use in this function) include/linux/seqlock.h:66: error: parse error before ')' token include/linux/seqlock.h:66: warning: left-hand operand of comma expression has no effect include/linux/seqlock.h:66: warning: unused variable `ret' include/linux/seqlock.h:66: warning: no return statement in function returning non-void include/linux/seqlock.h: At top level:
include/linux/seqlock.h:66: error: parse error before ')' token include/linux/seqlock.h:66: warning: type defaults to int' in declaration of __ret' include/linux/seqlock.h:66: warning: data definition has no type or storage class include/linux/seqlock.h:66: error: parse error before '}' token include/linux/seqlock.h:76: warning: type defaults to int' in declaration of seqlock_t' include/linux/seqlock.h:76: error: parse error before '*' token include/linux/seqlock.h:77: warning: function declaration isn't a prototype include/linux/seqlock.h: In function `read_seqbegin':
include/linux/seqlock.h:78: error: `sl' undeclared (first use in this function) include/linux/seqlock.h: At top level:
include/linux/seqlock.h:91: warning: type defaults to int' in declaration of seqlock_t' include/linux/seqlock.h:91: error: parse error before '*' token include/linux/seqlock.h:92: warning: function declaration isn't a prototype include/linux/seqlock.h: In function `read_seqretry':
include/linux/seqlock.h:94: error: iv' undeclared (first use in this function) include/linux/seqlock.h:94: error: sl' undeclared (first use in this function) In file included from include/linux/timex.h:58,

                 from include/linux/sched.h:11,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/time.h: At top level:
include/linux/time.h:83: error: parse error before "xtime_lock" include/linux/time.h:83: warning: type defaults to int' in declaration of xtime_lock' include/linux/time.h:83: warning: data definition has no type or storage class In file included from include/asm/semaphore.h:41,

                 from include/linux/sched.h:19,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/wait.h:82: error: parse error before '*' token include/linux/wait.h:83: warning: function declaration isn't a prototype include/linux/wait.h: In function `init_waitqueue_head':
include/linux/wait.h:84: error: q' undeclared (first use in this function) include/linux/wait.h:84: error: RAW_SPIN_LOCK_UNLOCKED' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:103: error: parse error before '*' token include/linux/wait.h:104: warning: function declaration isn't a prototype include/linux/wait.h: In function `waitqueue_active':
include/linux/wait.h:105: error: `q' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:117: error: parse error before '*' token include/linux/wait.h:117: warning: function declaration isn't a prototype include/linux/wait.h:118: error: parse error before '*' token include/linux/wait.h:118: warning: function declaration isn't a prototype include/linux/wait.h:119: error: parse error before '*' token include/linux/wait.h:119: warning: function declaration isn't a prototype include/linux/wait.h:121: error: parse error before '*' token include/linux/wait.h:122: warning: function declaration isn't a prototype include/linux/wait.h: In function `__add_wait_queue':
include/linux/wait.h:123: error: new' undeclared (first use in this function) include/linux/wait.h:123: error: head' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:129: error: parse error before '*' token include/linux/wait.h:131: warning: function declaration isn't a prototype include/linux/wait.h: In function `__add_wait_queue_tail':
include/linux/wait.h:132: error: new' undeclared (first use in this function) include/linux/wait.h:132: error: head' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:135: error: parse error before '*' token include/linux/wait.h:137: warning: function declaration isn't a prototype include/linux/wait.h: In function `__remove_wait_queue':
include/linux/wait.h:138: error: `old' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:141: error: parse error before '*' token include/linux/wait.h:141: warning: function declaration isn't a prototype include/linux/wait.h:142: error: parse error before '*' token include/linux/wait.h:142: warning: function declaration isn't a prototype include/linux/wait.h:143: error: parse error before '*' token include/linux/wait.h:143: warning: function declaration isn't a prototype include/linux/wait.h:144: error: parse error before '*' token include/linux/wait.h:144: warning: function declaration isn't a prototype include/linux/wait.h:145: error: parse error before '*' token include/linux/wait.h:145: error: `__wait_on_bit' declared as function returning a function include/linux/wait.h:145: warning: function declaration isn't a prototype include/linux/wait.h:145: error: parse error before "unsigned" include/linux/wait.h:146: error: parse error before '*' token include/linux/wait.h:146: error: `__wait_on_bit_lock' declared as function returning a function include/linux/wait.h:146: warning: function declaration isn't a prototype include/linux/wait.h:146: error: parse error before "unsigned" include/linux/wait.h:150: error: parse error before '*' token include/linux/wait.h:150: warning: type defaults to int' in declaration of bit_waitqueue' include/linux/wait.h:150: warning: data definition has no type or storage class include/linux/wait.h:288: error: parse error before '*' token include/linux/wait.h:290: warning: function declaration isn't a prototype include/linux/wait.h: In function `add_wait_queue_exclusive_locked':
include/linux/wait.h:291: error: wait' undeclared (first use in this function) include/linux/wait.h:292: error: q' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:298: error: parse error before '*' token include/linux/wait.h:300: warning: function declaration isn't a prototype include/linux/wait.h: In function `remove_wait_queue_locked':
include/linux/wait.h:301: error: q' undeclared (first use in this function) include/linux/wait.h:301: error: wait' undeclared (first use in this function) include/linux/wait.h: At top level:
include/linux/wait.h:309: error: parse error before '*' token include/linux/wait.h:309: warning: function declaration isn't a prototype include/linux/wait.h:310: error: parse error before '*' token include/linux/wait.h:310: warning: function declaration isn't a prototype include/linux/wait.h:312: error: parse error before '*' token include/linux/wait.h:312: warning: function declaration isn't a prototype include/linux/wait.h:313: error: parse error before '*' token include/linux/wait.h:313: warning: function declaration isn't a prototype include/linux/wait.h:319: error: parse error before '*' token include/linux/wait.h:319: warning: function declaration isn't a prototype include/linux/wait.h:321: error: parse error before '*' token include/linux/wait.h:321: warning: function declaration isn't a prototype include/linux/wait.h:323: error: parse error before '*' token include/linux/wait.h:323: warning: function declaration isn't a prototype
In file included from include/asm/rwsem.h:42,

                 from include/linux/rwsem.h:27,
                 from include/asm/semaphore.h:42,
                 from include/linux/sched.h:19,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm/spinlock.h:26: error: `raw_spinlock_t' used prior to declaration
In file included from include/asm/rwsem.h:42,
                 from include/linux/rwsem.h:27,
                 from include/asm/semaphore.h:42,
                 from include/linux/sched.h:19,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm/spinlock.h:80:1: warning: "SPIN_LOCK_UNLOCKED" redefined
In file included from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/spinlock.h:199:1: warning: this is the location of the previous definition include/asm/spinlock.h:86: error: conflicting types for spinlock_t' include/linux/spinlock.h:198: error: previous declaration of spinlock_t' In file included from include/asm/rwsem.h:42,

                 from include/linux/rwsem.h:27,
                 from include/asm/semaphore.h:42,
                 from include/linux/sched.h:19,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm/spinlock.h:93:1: warning: "RW_LOCK_UNLOCKED" redefined
In file included from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/spinlock.h:220:1: warning: this is the location of the previous definition include/asm/spinlock.h:99: error: conflicting types for rwlock_t' include/linux/spinlock.h:219: error: previous declaration of rwlock_t' include/asm/spinlock.h:165: error: parse error before "do" include/asm/spinlock.h:197: error: parse error before "void" include/asm/spinlock.h:207: error: parse error before "do" include/asm/spinlock.h:259: error: parse error before "do" include/asm/spinlock.h:267: error: parse error before "do"
In file included from include/asm/rwsem.h:42,

                 from include/linux/rwsem.h:27,
                 from include/asm/semaphore.h:42,
                 from include/linux/sched.h:19,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm/spinlock.h:275:1: warning: "_raw_read_unlock" redefined
In file included from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/linux/spinlock.h:228:1: warning: this is the location of the previous definition
In file included from include/asm/rwsem.h:42,
                 from include/linux/rwsem.h:27,
                 from include/asm/semaphore.h:42,
                 from include/linux/sched.h:19,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm/spinlock.h:276:1: warning: "_raw_write_unlock" redefined
In file included from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:

include/linux/spinlock.h:230:1: warning: this is the location of the previous definition include/asm/spinlock.h:278: error: parse error before '{' token include/asm/spinlock.h:281: warning: type defaults to int' in declaration of atomic_dec' include/asm/spinlock.h:281: warning: parameter names (without types) in function declaration include/asm/spinlock.h:281: error: conflicting types for atomic_dec' include/asm/atomic.h:116: error: previous declaration of atomic_dec' include/asm/spinlock.h:281: warning: data definition has no type or storage class include/asm/spinlock.h:282: error: parse error before "if" include/asm/spinlock.h:284: warning: type defaults to int' in declaration of atomic_inc' include/asm/spinlock.h:284: warning: parameter names (without types) in function declaration include/asm/spinlock.h:284: error: conflicting types for atomic_inc' include/asm/atomic.h:102: error: previous declaration of atomic_inc' include/asm/spinlock.h:284: warning: data definition has no type or storage class include/asm/spinlock.h:285: error: parse error before "return" include/asm/spinlock.h:288: error: parse error before '{' token include/asm/spinlock.h:293: error: parse error before numeric constant include/asm/spinlock.h:293: warning: type defaults to int' in declaration of atomic_add' include/asm/spinlock.h:293: warning: function declaration isn't a prototype include/asm/spinlock.h:293: error: conflicting types for atomic_add' include/asm/atomic.h:53: error: previous declaration of atomic_add' include/asm/spinlock.h:293: warning: data definition has no type or storage class In file included from arch/i386/kernel/asm-offsets.c:7:
include/linux/sched.h:847:56: macro "_spin_lock_irqsave" requires 2 arguments, but only 1 given In file included from arch/i386/kernel/asm-offsets.c:7:
include/linux/sched.h: In function `dequeue_signal_lock':
include/linux/sched.h:847: error: _spin_lock_irqsave' undeclared (first use in this function) include/linux/seqlock.h: In function write_tryseqlock':
include/linux/seqlock.h:66: warning: statement with no effect
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [arch/i386/kernel/asm-offsets.s] Error 2

same errors.. Both with the preemptible real time thingy and without..

flo

Try building for SMP. I suspect this is a UP build problem.

Lee

I got same errors...
Struct mutex_t is defined in include/asm-i386/spinlock.h. It's only included in include/linux/spinlock.h if CONFIG_SMP is set, but mutex_t is used at include/linux/spinlock.h:419. Set CONFIG_SMP=y then kernel builds successfully here.

Best Regards,
Wen-chien Jesse Sung

could you send me your .config? Had to do some wacky include file magic to be able to use semaphores in spinlocks, but could easily have missed some .config variations.

Ingo

Sure,

you released T6 already, but here's my T5 config anyways (this one w/o CONFIG_PREEMPT_REALTIME, same result with it enabled though) Gonna try building T6 now:

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.9-rc4-mm1-VP-T5
# Tue Oct 12 01:13:25 2004
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_BROKEN_ON_SMP=y

#
# General setup
#
CONFIG_LOCALVERSION="-LT"
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=14
# CONFIG_HOTPLUG is not set
CONFIG_KOBJECT_UEVENT=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_PREEMPT_TIMING=y
CONFIG_LATENCY_TRACE=y
CONFIG_MCOUNT=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SHMEM=y
# CONFIG_TINY_SHMEM is not set

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
CONFIG_MK7=y
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_USE_3DNOW=y
# CONFIG_SMP is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_PREEMPT_VOLUNTARY=y
CONFIG_PREEMPT_SOFTIRQS=y
CONFIG_PREEMPT_HARDIRQS=y
# CONFIG_PREEMPT_REALTIME is not set
# CONFIG_X86_UP_APIC is not set
CONFIG_X86_TSC=y
# CONFIG_X86_MCE is not set
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
CONFIG_HAVE_DEC_LOCK=y

#
# Performance-monitoring counters support
#
# CONFIG_PERFCTR is not set
# CONFIG_KEXEC is not set

#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SOFTWARE_SUSPEND is not set

#
# ACPI (Advanced Configuration and Power Interface) Support #
# CONFIG_ACPI is not set CONFIG_ACPI_BLACKLIST_YEAR=0

#
# APM (Advanced Power Management) BIOS Support
#
CONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not set
# CONFIG_APM_CPU_IDLE is not set
# CONFIG_APM_DISPLAY_BLANK is not set
# CONFIG_APM_RTC_IS_GMT is not set
# CONFIG_APM_ALLOW_INTS is not set
CONFIG_APM_REAL_MODE_POWER_OFF=y

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_LEGACY_PROC is not set
CONFIG_PCI_NAMES=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_MISC=y

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_DEBUG_DRIVER is not set

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play support
#

#
# Block devices
#
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
# CONFIG_LBD is not set
# CONFIG_CDROM_PKTCDVD is not set

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives #
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set CONFIG_BLK_DEV_IDESCSI=m
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
CONFIG_BLK_DEV_SIS5513=y
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
CONFIG_SCSI=m
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=m

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs #
# CONFIG_SCSI_MULTI_LUN is not set CONFIG_SCSI_CONSTANTS=y
# CONFIG_SCSI_LOGGING is not set

#
# SCSI Transport Attributes
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set

#
# SCSI low-level drivers
#
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=32
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
# CONFIG_AIC7XXX_REG_PRETTY_PRINT is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_SCSI_SATA is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLOGIC_1280_1040 is not set
CONFIG_SCSI_QLA2XXX=m
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set

#
# Multi-device support (RAID and LVM)
#
# CONFIG_MD is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set

#
# Networking support
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_MMAP is not set
CONFIG_NETLINK_DEV=y
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
# CONFIG_IP_MULTIPLE_TABLES is not set
# CONFIG_IP_ROUTE_MULTIPATH is not set
# CONFIG_IP_ROUTE_VERBOSE is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_TUNNEL is not set

#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set

#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
# CONFIG_IP_NF_CT_ACCT is not set
# CONFIG_IP_NF_CT_PROTO_SCTP is not set
# CONFIG_IP_NF_FTP is not set
# CONFIG_IP_NF_IRC is not set
# CONFIG_IP_NF_TFTP is not set
# CONFIG_IP_NF_AMANDA is not set
# CONFIG_IP_NF_QUEUE is not set
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_LIMIT=m
CONFIG_IP_NF_MATCH_IPRANGE=m
CONFIG_IP_NF_MATCH_MAC=m
CONFIG_IP_NF_MATCH_PKTTYPE=m
CONFIG_IP_NF_MATCH_MARK=m
CONFIG_IP_NF_MATCH_MULTIPORT=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH_ESP=m
CONFIG_IP_NF_MATCH_LENGTH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_TCPMSS=m
CONFIG_IP_NF_MATCH_HELPER=m
CONFIG_IP_NF_MATCH_STATE=m
CONFIG_IP_NF_MATCH_CONNTRACK=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_REALM=m
# CONFIG_IP_NF_MATCH_SCTP is not set
# CONFIG_IP_NF_MATCH_COMMENT is not set
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
# CONFIG_IP_NF_NAT_LOCAL is not set
# CONFIG_IP_NF_NAT_SNMP_BASIC is not set
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_MARK=m
CONFIG_IP_NF_TARGET_CLASSIFY=m
# CONFIG_IP_NF_RAW is not set
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
# CONFIG_IP_NF_COMPAT_IPCHAINS is not set
# CONFIG_IP_NF_COMPAT_IPFWADM is not set

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_HW_FLOWCONTROL is not set

#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
# CONFIG_NET_SCH_CLK_CPU is not set
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
# CONFIG_NET_SCH_HFSC is not set
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
# CONFIG_NET_SCH_NETEM is not set
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
# CONFIG_CLS_U32_PERF is not set
# CONFIG_NET_CLS_IND is not set
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
# CONFIG_NET_CLS_ACT is not set
CONFIG_NET_CLS_POLICE=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_KGDBOE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NETPOLL_RX is not set
# CONFIG_NETPOLL_TRAP is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_ETHERTAP is not set

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_NET_VENDOR_3COM is not set

#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
CONFIG_SIS900=m
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set

#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set

#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPPOE=m
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set

#
# Input I/O drivers
#
CONFIG_GAMEPORT=m
CONFIG_SOUND_GAMEPORT=m
# CONFIG_GAMEPORT_NS558 is not set
# CONFIG_GAMEPORT_L4 is not set
# CONFIG_GAMEPORT_EMU10K1 is not set
# CONFIG_GAMEPORT_VORTEX is not set
# CONFIG_GAMEPORT_FM801 is not set
# CONFIG_GAMEPORT_CS461x is not set
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=y
# CONFIG_INPUT_UINPUT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_CONSOLE is not set
CONFIG_SERIAL_8250_NR_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
CONFIG_NVRAM=y
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=m
# CONFIG_AGP_ALI is not set
# CONFIG_AGP_ATI is not set
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD64 is not set
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_INTEL_MCH is not set
# CONFIG_AGP_NVIDIA is not set
CONFIG_AGP_SIS=m
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_AGP_EFFICEON is not set
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HANGCHECK_TIMER=m

#
# I2C support
#
CONFIG_I2C=m
CONFIG_I2C_CHARDEV=m

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCF=m
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_I810=m
CONFIG_I2C_ISA=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_PIIX4=m
CONFIG_I2C_PROSAVAGE=m
CONFIG_I2C_SAVAGE4=m
CONFIG_SCx200_ACB=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
# CONFIG_I2C_STUB is not set
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
CONFIG_I2C_VOODOO3=m
# CONFIG_I2C_PCA_ISA is not set

#
# Hardware Sensors Chip support
#
CONFIG_I2C_SENSOR=m
CONFIG_SENSORS_ADM1021=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_FSCHER=m
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_MAX1619=m
# CONFIG_SENSORS_SMSC47M1 is not set
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83627HF=m

#
# Other I2C Chip support
#
CONFIG_SENSORS_EEPROM=m
CONFIG_SENSORS_PCF8574=m
CONFIG_SENSORS_PCF8591=m
CONFIG_SENSORS_RTC8564=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set

#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set

#
# Graphics support
#
# CONFIG_FB is not set
CONFIG_VIDEO_SELECT=y

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y

#
# Sound
#
CONFIG_SOUND=m

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=m
# CONFIG_SND_VERBOSE_PRINTK is not set
CONFIG_SND_DEBUG=y
# CONFIG_SND_DEBUG_MEMORY is not set
# CONFIG_SND_DEBUG_DETECT is not set

#
# Generic devices
#
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_VIRMIDI is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set

#
# PCI devices
#
CONFIG_SND_AC97_CODEC=m
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
CONFIG_SND_CS46XX=m
CONFIG_SND_CS46XX_NEW_DSP=y
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
# CONFIG_SND_INTEL8X0 is not set
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VX222 is not set

#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set

#
# USB support
#
# CONFIG_USB is not set

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
# CONFIG_EXT2_FS_POSIX_ACL is not set
# CONFIG_EXT2_FS_SECURITY is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
# CONFIG_EXT3_FS_POSIX_ACL is not set
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISER4_FS is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
CONFIG_ROMFS_FS=y
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=y

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not set
CONFIG_DEVPTS_FS_XATTR=y
# CONFIG_DEVPTS_FS_SECURITY is not set
CONFIG_TMPFS=y
# CONFIG_TMPFS_XATTR is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
CONFIG_CRAMFS=m
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
# CONFIG_NFS_FS is not set
# CONFIG_NFSD is not set
# CONFIG_EXPORTFS is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
CONFIG_AFS_FS=m
CONFIG_RXRPC=m

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
CONFIG_NLS_CODEPAGE_850=y
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
CONFIG_NLS_CODEPAGE_1250=y
# CONFIG_NLS_CODEPAGE_1251 is not set
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=y
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_UTF8 is not set

#
# Profiling support
#
# CONFIG_PROFILING is not set

#
# Kernel hacking
#
CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
CONFIG_DEBUG_PREEMPT=y
# CONFIG_DEBUG_INFO is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_KPROBES is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PAGEALLOC is not set
# CONFIG_4KSTACKS is not set
# CONFIG_SCHEDSTATS is not set
# CONFIG_KGDB is not set

#
# Security options
#
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
# CONFIG_SECURITY_NETWORK is not set
CONFIG_SECURITY_CAPABILITIES=m
# CONFIG_SECURITY_SECLVL is not set
# CONFIG_SECURITY_SELINUX is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
# CONFIG_CRYPTO_HMAC is not set
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_MD4 is not set
# CONFIG_CRYPTO_MD5 is not set
# CONFIG_CRYPTO_SHA1 is not set
# CONFIG_CRYPTO_SHA256 is not set
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_WP512 is not set
# CONFIG_CRYPTO_DES is not set
# CONFIG_CRYPTO_BLOWFISH is not set
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_SERPENT is not set
# CONFIG_CRYPTO_AES_586 is not set
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
# CONFIG_CRYPTO_TEA is not set
# CONFIG_CRYPTO_ARC4 is not set
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_DEFLATE is not set
# CONFIG_CRYPTO_MICHAEL_MIC is not set
# CONFIG_CRYPTO_CRC32C is not set
# CONFIG_CRYPTO_TEST is not set

#
# Library routines
#
CONFIG_CRC_CCITT=m
CONFIG_CRC32=m
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m
CONFIG_GENERIC_HARDIRQS=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_PC=y

Sharp roughness is the name of the game ;)

Built fine here, for P4 SMP/HT, and CONFIG_PREEMPT_REALTIME=y.

But, not surprisingly, booting/initing gives me lots of fireworks. This is just some (tiny) sample taken from syslog:

Oct 12 00:15:26 gamma-suse1 kernel: Debug: sleeping function called from
invalid context pdflush(120) at kernel/mutex.c:23
Oct 12 00:15:26 gamma-suse1 kernel: in_atomic():1 [00000001],
irqs_disabled():1
Oct 12 00:15:26 gamma-suse1 kernel:  [__might_sleep+193/212]
__might_sleep+0xc1/0xd4
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0118fca>] __might_sleep+0xc1/0xd4
Oct 12 00:15:26 gamma-suse1 kernel:  [_mutex_lock+39/96]
_mutex_lock+0x27/0x60
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0130d0c>] _mutex_lock+0x27/0x60
Oct 12 00:15:26 gamma-suse1 kernel:  [_mutex_lock_irqsave+22/26]
_mutex_lock_irqsave+0x16/0x1a
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0130d7d>]
_mutex_lock_irqsave+0x16/0x1a
Oct 12 00:15:26 gamma-suse1 kernel:  [page_address+78/158]
page_address+0x4e/0x9e
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0149c97>] page_address+0x4e/0x9e
Oct 12 00:15:26 gamma-suse1 kernel:  [bio_hw_segments+12/48]
bio_hw_segments+0xc/0x30
Oct 12 00:15:26 gamma-suse1 kernel:  [<c01617ca>] bio_hw_segments+0xc/0x30
Oct 12 00:15:26 gamma-suse1 kernel:  [__make_request+1010/1251]
__make_request+0x3f2/0x4e3
Oct 12 00:15:26 gamma-suse1 kernel:  [<c023db69>] __make_request+0x3f2/0x4e3
Oct 12 00:15:26 gamma-suse1 kernel:  [generic_make_request+298/662]
generic_make_request+0x12a/0x296
Oct 12 00:15:26 gamma-suse1 kernel:  [<c023dee4>]

generic_make_request+0x12a/0x296
...

Oct 12 00:15:26 gamma-suse1 kernel: Debug: sleeping function called from
invalid context modprobe(1666) at kernel/mutex.c:23
Oct 12 00:15:26 gamma-suse1 kernel: in_atomic():1 [00000001],
irqs_disabled():1
Oct 12 00:15:26 gamma-suse1 kernel:  [__might_sleep+193/212]
__might_sleep+0xc1/0xd4
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0118fca>] __might_sleep+0xc1/0xd4
Oct 12 00:15:26 gamma-suse1 kernel:  [_mutex_lock+39/96]
_mutex_lock+0x27/0x60
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0130d0c>] _mutex_lock+0x27/0x60
Oct 12 00:15:26 gamma-suse1 kernel:  [_mutex_lock_irqsave+22/26]
_mutex_lock_irqsave+0x16/0x1a
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0130d7d>]
_mutex_lock_irqsave+0x16/0x1a
Oct 12 00:15:26 gamma-suse1 kernel:  [page_address+78/158]
page_address+0x4e/0x9e
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0149c97>] page_address+0x4e/0x9e
Oct 12 00:15:26 gamma-suse1 kernel:  [bio_hw_segments+12/48]
bio_hw_segments+0xc/0x30
Oct 12 00:15:26 gamma-suse1 kernel:  [<c01617ca>] bio_hw_segments+0xc/0x30
Oct 12 00:15:26 gamma-suse1 kernel:  [__make_request+1010/1251]
__make_request+0x3f2/0x4e3
Oct 12 00:15:26 gamma-suse1 kernel:  [<c023db69>] __make_request+0x3f2/0x4e3
Oct 12 00:15:26 gamma-suse1 kernel:  [generic_make_request+298/662]
generic_make_request+0x12a/0x296
Oct 12 00:15:26 gamma-suse1 kernel:  [<c023dee4>]

generic_make_request+0x12a/0x296
...

Oct 12 00:15:26 gamma-suse1 kernel: Debug: sleeping function called from
invalid context blogd(1851) at kernel/mutex.c:23
Oct 12 00:15:26 gamma-suse1 kernel: in_atomic():1 [00000001],
irqs_disabled():1
Oct 12 00:15:26 gamma-suse1 kernel:  [__might_sleep+193/212]
__might_sleep+0xc1/0xd4
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0118fca>] __might_sleep+0xc1/0xd4
Oct 12 00:15:26 gamma-suse1 kernel:  [_mutex_lock+39/96]
_mutex_lock+0x27/0x60
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0130d0c>] _mutex_lock+0x27/0x60
Oct 12 00:15:26 gamma-suse1 kernel:  [_mutex_lock_irqsave+22/26]
_mutex_lock_irqsave+0x16/0x1a
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0130d7d>]
_mutex_lock_irqsave+0x16/0x1a
Oct 12 00:15:26 gamma-suse1 kernel:  [page_address+78/158]
page_address+0x4e/0x9e
Oct 12 00:15:26 gamma-suse1 kernel:  [<c0149c97>] page_address+0x4e/0x9e
Oct 12 00:15:26 gamma-suse1 kernel:  [bio_hw_segments+12/48]
bio_hw_segments+0xc/0x30
Oct 12 00:15:26 gamma-suse1 kernel:  [<c01617ca>] bio_hw_segments+0xc/0x30
Oct 12 00:15:26 gamma-suse1 kernel:  [__make_request+1010/1251]
__make_request+0x3f2/0x4e3
Oct 12 00:15:26 gamma-suse1 kernel:  [<c023db69>] __make_request+0x3f2/0x4e3
Oct 12 00:15:26 gamma-suse1 kernel:  [generic_make_request+298/662]
generic_make_request+0x12a/0x296
Oct 12 00:15:26 gamma-suse1 kernel:  [<c023dee4>]

generic_make_request+0x12a/0x296
...

An so on, and so on...

rncbc aka Rui Nuno Capela
[email protected]

Ingo, are any of the VP patches known to work on x64? Here is thewade's latest report:

--

I applied the patch and the kernel built, but like 2.6.9-mm2-VP-S9 it crashed before it could load. The last bit of the message was a lot of what I guess are frame pointers, but there was a few lines that had info.
For example an RIP message having to do with add_preempt_count+16

But it all ended with Aieee...

I have yet to get any VP kernel to run on my x86_64. I suppose I should try just the mm3 or mm4 patches without the VP portion, so that is what I will do.

--

Lee

i have two x64 boxes which i tested with S* -VP and i also did a quick testboot of -T3 on one of them but YMMV. (There's one caveat: latency tracing must not be enabled, x64 gcc has a nastiness that makes -pg unusable for tracing purposes on x64.)

-T4 and upwards will likely not even compile on x64 - i'll fix it up once the rate of PREEMPT_REALTIME changes calms down.

Ingo

This fixed the build problems for me (SMP). I did get one unresolved symbol when building this with REALTIME enabled. Also got error messages scrolling up the screen when I tried to boot it (looked very much like Mark's problem with T4) and it never made it. :( If I had to guess, it might be related to APICs? I always have to use "noapic" boot parameter. Ingo what are you running this on? I don't have the exact error messages, but I'm rebuilding it now to try to get those. Without RT Preemption it seems to be running very nicely.

kr

(which symbol was this?)

dont worry about it not booting on your setup with PREEMPT_REALTIME, as long as it boots with !PREEMPT_REALTIME - i only really converted my testsystems which are basically IDE + e100/e1000/rtl8139, ext3 and the bare minimum that is needed to run Fedora. It might be useful to send me a bootlog if you have any easy way to capture it - if not it's not a big problem either.

Ingo

WARNING: /lib/modules/2.6.9-rc4-mm1-VP-T5-RT/kernel/drivers/net/ppp_synctty.ko needs unknown symbol _mutex_trylock_bh

I shouldn't have even had ppp enabled and then wouldn't have noticed it, but...

thx - fix will be in -T7.

Ingo

i've uploaded -T6:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T6

this should fix the UP build issues reported by many. -T6 also brings back the ->break_lock framework and converts a few more locks to raw.

SMP is still expected to be flaky due to the zombie-task problem(s). But UP is not out of the 'extremely experimental' status either.

to create a -T6 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T6

Ingo

UP build is still failed:
arch/i386/kernel/vm86.c:707: error: `__RAW_SPIN_LOCK_UNLOCKED'
undeclared here (not in a function)

Best Regards,
Wen-chien Jesse Sung

ok, fixed this one too and re-uploaded -T6 - please check whether it builds for you now.

Ingo

Yes, it works now! Thanks a lot! :)

Best Regards,
Wen-chien Jesse Sung

one more warning wrt. PREEMPT_REALTIME: if this option is enabled then it is not safe to make interrupts non-threaded via /proc/irq/*/*/threaded. If you need to turn an interrupt into a high-prio event then its irq thread should be set to RT priority via 'chrt'. (-T7 will turn off /proc/irq/*/*/threaded altogether, to make sure it's not set accidentally.)

Ingo

in fact i've re-uploaded a new version of the -T6 patch to disable direct interrupts under PREEMPT_REALTIME kernels. The only exception is IRQ1 on PCs (the keyboard irq), which can be useful for debugging purposes (SysRq, etc.). I turned the keyboard related locks into raw spinlocks to make this safe.

Ingo

i've uploaded -T7:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T7

Changes since -T6:

- further stabilization of PREEMPT_REALTIME: fixed the task-reaping problem by moving TASK_ZOMBIE out of p->state and thus completely separating preemption from the child-exit mechanism. This got rid of the 'Badness in exit.c' warnings on my SMP testbox (and related crashes).

- fixed the _mutex_trylock_bh missing symbol problem reported by K.R. Foley and Florian Schmidt.

- turned the sysrq lock into a raw spinlock, to enable direct keyboard irqs.

PREEMPT_REALTIME is still experimental, but it's already looking much better on my testboxes.

to create a -T7 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T7

Ingo

OK. 2.6.9-rc4-mm1-T7 builds and runs on my laptop (P4/UP), apparently fine. I know it's probably too early to complain, but I'm sending a couple of dmesg's that I took right after init, showing some badness going on.

.config file is aldo attached.

Cheers.

rncbc aka Rui Nuno Capela
[email protected]

does the patch below ontop of T7 fix these messages for you?

Ingo

@@ -261,8 +261,8 @@ struct snd_stru_ali {
 	unsigned short	ac97_ext_id;
 	unsigned short	ac97_ext_status;
 
-	spinlock_t	reg_lock;
-	spinlock_t	voice_alloc;
+	raw_spinlock_t	reg_lock;
+	raw_spinlock_t	voice_alloc;
 
 #ifdef CONFIG_PM
 	ali_image_t *image;

@@ -541,7 +541,7 @@ int send_sigurg(struct fown_struct *fown
 	return ret;
 }
 
-static rwlock_t fasync_lock = RW_LOCK_UNLOCKED;
+static DECLARE_RAW_RWLOCK(fasync_lock);
 static kmem_cache_t *fasync_cache;
 
 /*

OK. This one builds just fine here. Again I tried booting preempt realtime. We were going along fine and then all hell broke loose on the console. Pressed Ctrl-s to stop the scrolling and it then bit the dust. It did manage to get into the logs this time and I am attaching that. This is a different SMP system that I use as a workstation at a client site. Dual 2.6GHz Xeons (with HT)
512MB

kr

Oct 12 09:52:58 swdev14 syslogd 1.4.1: restart.
Oct 12 09:52:58 swdev14 syslog: syslogd startup succeeded
Oct 12 09:52:59 swdev14 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Oct 12 09:52:59 swdev14 syslog: klogd startup succeeded
Oct 12 09:52:59 swdev14 kernel:  sys_init_module+0x68/0x1ce
Oct 12 09:52:59 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:52:59 swdev14 kernel: scheduling while atomic: usb.agent/0x04010000/1298
Oct 12 09:52:59 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:52:59 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:52:59 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:52:59 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:52:59 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:52:59 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:52:59 swdev14 irqbalance: irqbalance startup succeeded
Oct 12 09:52:59 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:52:59 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:52:59 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:52:59 swdev14 kernel: EXT3 FS on hda6, internal journal
Oct 12 09:52:59 swdev14 kernel: device-mapper: 4.1.0-ioctl (2003-12-10) initialised: [email protected]
Oct 12 09:52:59 swdev14 kernel: Adding 2048216k swap on /dev/hda5.  Priority:-1 extents:1
Oct 12 09:52:59 swdev14 kernel: scheduling while atomic: rc.sysinit/0x04010001/1561
Oct 12 09:52:59 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:52:59 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:52:59 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:52:59 swdev14 portmap: portmap startup succeeded
Oct 12 09:52:59 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:52:59 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:52:59 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:52:59 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:52:59 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:52:59 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:52:59 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 09:52:59 swdev14 kernel:  [<c029a35d>] _spin_unlock_irqrestore+0x10/0x36
Oct 12 09:52:59 swdev14 rpc.statd[2649]: Version 1.0.6 Starting
Oct 12 09:52:59 swdev14 kernel:  [<c0118895>] try_to_wake_up+0x1e8/0x270
Oct 12 09:52:59 swdev14 kernel:  [<c0118940>] wake_up_process+0x23/0x27
Oct 12 09:52:59 swdev14 kernel:  [<c011923d>] sched_migrate_task+0x7e/0x9d
Oct 12 09:52:59 swdev14 nfslock: rpc.statd startup succeeded
Oct 12 09:52:59 swdev14 kernel:  [<c01192e6>] sched_exec+0x8a/0xd4
Oct 12 09:52:59 swdev14 kernel:  [<c01192fb>] sched_exec+0x9f/0xd4
Oct 12 09:52:59 swdev14 kernel:  [<c0166f3b>] do_execve+0x3e/0x249
Oct 12 09:52:59 swdev14 kernel:  [<c0168881>] getname+0x91/0xbc
Oct 12 09:52:59 swdev14 kernel:  [<c0104d5d>] sys_execve+0x47/0x9a
Oct 12 09:52:59 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:52:59 swdev14 kernel: scheduling while atomic: grep/0x04010000/1569
Oct 12 09:52:59 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:52:59 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:52:59 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:52:59 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:00 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:00 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:00 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:00 swdev14 rpcidmapd: rpc.idmapd startup succeeded
Oct 12 09:53:00 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:00 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:00 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:00 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 09:53:00 swdev14 kernel:  [<c02996c7>] cond_resched+0x14/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0133822>] _mutex_lock+0x19/0x3f
Oct 12 09:53:00 swdev14 kernel:  [<c014d347>] handle_mm_fault+0x54/0x18a
Oct 12 09:53:00 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:00 swdev14 random: Initializing random number generator:  succeeded
Oct 12 09:53:00 swdev14 kernel:  [<c0116fa8>] do_page_fault+0x20b/0x662
Oct 12 09:53:00 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:00 swdev14 kernel:  [<c02996c1>] cond_resched+0xe/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c014e08f>] sys_brk+0x28/0x10f
Oct 12 09:53:00 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:00 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c014e08f>] sys_brk+0x28/0x10f
Oct 12 09:53:00 swdev14 kernel:  [<c014fd3a>] sys_munmap+0x59/0x7b
Oct 12 09:53:00 swdev14 rc: Starting pcmcia:  succeeded
Oct 12 09:53:00 swdev14 kernel:  [<c0116d9d>] do_page_fault+0x0/0x662
Oct 12 09:53:00 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 09:53:00 swdev14 kernel: scheduling while atomic: cat/0x04010000/1752
Oct 12 09:53:00 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:00 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:00 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:00 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:00 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:00 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:00 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:00 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:00 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:00 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:00 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 09:53:00 swdev14 kernel:  [<c015b55f>] vfs_read+0x0/0x134
Oct 12 09:53:00 swdev14 kernel:  [<c015b8ed>] sys_read+0x50/0x7a
Oct 12 09:53:00 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:00 swdev14 kernel: schedulingro0x603>] er_i> [ic_<4c0134_pre<4> pr<c01tr88>] checkpt1fd9>
Oct 12 09:53:00 swdev14 kernel:  [<c0299] 49/0x1pr19136>] pt_ [<mc4> _spore+0xb[<c7
Oct 12 09:53:00 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:00 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:00 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:00 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:00 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:00 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:00 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:00 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:00 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:00 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:00 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:00 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:00 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:00 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:00 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:00 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:00 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:01 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:01 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:01 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:01 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:01 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:01 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:01 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:01 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel: ]97a>]6>0dae+0>+0c0+0xc0_wxumi5e+] v/t[x010xdux0>0c0>on> [o0c0/41cre46/nd_x8he> _mu+pd/ro03>e [i
Oct 12 09:53:01 swdev14 kernel:  [<]9/0r1936pt> [m4>_sre+[07
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:01 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:01 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:01 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:01 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:01 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:01 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:01 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:01 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:01 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:01 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:01 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:01 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:01 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:01 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:01 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:01 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:01 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:01 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 last message repeated 2 times
Oct 12 09:53:01 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:01 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:01 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:01 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:01 swdev14 kernel:  [<c01a9bd8>] dummy_file_permiansm026chec> [_skb<4>x3c
Oct 12 09:53:01 swdev14 kernel: 3>] tc4a8eg+0[<nt+54sg+04aft_
Oct 12 09:53:01 swdev14 kernel:   su7
Oct 12 09:53:01 swdev14 kernel: ] x97
Oct 12 09:53:01 swdev14 kernel:  ref>] c01348_pr
Oct 12 09:53:01 swdev14 kernel:   pr<c01tra88>] check_pt1f9d9>7
Oct 12 09:53:01 swdev14 kernel:  [<c02992] 496/0x1pr19136>]pt_ [<mc4> _spore+0x[<7
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x1>] ] dunc[<cwn_<c013.lo338ck_ [omec514d>c0unl02404f3>] qest[<c0nt+qdisc56/0>] de+outd1e
Oct 12 09:53:01 swdev14 kernel: ] 0x239>[<cout>] suntsub_pree+0xpin_unlox34 subt+0xpree2/0a2c7ck+134eem29a2clocc013nt+265end
Oct 12 09:53:01 swdev14 kernel: <4 tc [unttcp_v4_s+0x5fmit6>x3c>] +0x[<c025sen8d
Oct 12 09:53:01 swdev14 kernel: ] a65>] t+0x134ampt0134emp34apt_caio_x13>]x50a>e+0d4>0x1d>] e+01a9bile_cricrit<4> [<mco[<cwri10617_pa3>scle 2/27is /0x2674> [ondc0pre
Oct 12 09:53:01 swdev14 kernel: <4] c [<ck_
Oct 12 09:53:01 swdev14 kernel: <029<4con<4> [<c_rw+0c01/fde
Oct 12 09:53:01 swdev14 kernel: < smnter4
Oct 12 09:53:01 swdev14 kernel: <er_in/0xe_con9/fb2>]12cheing1e1d6edce+08>] _ti106fck+] 0xn+0x8a/[<ctimi9
Oct 12 09:53:01 swdev14 kernel: <_pr0x_mcount+0x11
Oct 12 09:53:01 swdev14 kernel: <] _sqrec0298491>] __d+0+0<c04def0x0/029+0x01ock.46
Oct 12 09:53:01 swdev14 kernel: mutexve+ boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:01 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:01 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:01 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:01 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:01 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:01 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:01 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:01 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:01 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:01 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:02 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:02 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:02 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:02 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:02 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:02 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:02 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:02 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:02 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 last message repeated 2 times
Oct 12 09:53:02 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:02 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:02 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:02 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:02 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:02 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:02 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:02 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:02 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:02 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:02 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:02 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:02 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:02 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:02 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:02 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:02 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:02 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:02 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:02 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:02 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:02 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:02 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:02 swdev14 kernel:  [<c011f0b2>] release_console_sem+0x59/0xcf
Oct 12 09:53:02 swdev14 kernel:  [<c011efb2>] vprintk+0x128/0x16f
Oct 12 09:53:02 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:02 swdev14 kernel:  [<c011ee86>] printk+0x1d/0x21
Oct 12 09:53:02 swdev14 kernel:  [<c0106edc>] show_trace+0x4e/0x8d
Oct 12 09:53:02 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:02 swdev14 kernel:  [<c0106fd9>] dump_stack+0x23/0x27
Oct 12 09:53:02 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:02 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:02 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:02 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:02 swdev14 kernel:  [<c01347
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:02 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:02 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:02 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:02 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:02 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:02 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:02 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:02 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:02 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:02 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:02 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:02 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:02 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:02 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:02 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:02 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:02 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:02 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:02 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:02 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:02 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:02 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:02 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:02 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:02 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:02 swdev14 last message repeated 2 times
Oct 12 09:53:02 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:02 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:02 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:02 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:02 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:02 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:03 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:03 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:03 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:03 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:03 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:03 swdev14 kernel: c0c>o4> [ou0c0/41c0r4d_xche> w_mu+pdo03>er i< ><4 c0t88>] cp1fd97
Oct 12 09:53:03 swdev14 kernel:  [<]49/0r136>p [m<4_sore+[7
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:03 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:03 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:03 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:03 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:03 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:03 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:03 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:03 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:03 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:03 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:03 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:03 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:03 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:03 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:03 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:03 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:03 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:03 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:03 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0254sg4at<4 97
Oct 12 09:53:03 swdev14 kernel: >]97a6>xde+0>0c0xc_wx0mmi5e  p<c01tra88>] checkpt_x
Oct 12 09:53:03 swdev14 kernel:  [<] s96/0x1pr19936>]pt_> [<mc<4> _spore+0x[<07
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:03 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:03 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:03 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:03 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:03 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:03 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:03 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:03 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:03 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:03 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:03 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:03 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:03 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:03 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:03 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:03 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:03 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:03 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:03 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:03 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:03 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:03 swdev14 last message repeated 2 times
Oct 12 09:53:03 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:03 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:03 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:03 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:03 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:03 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:03 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:03 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:03 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:03 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:03 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:04 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:04 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:04 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:04 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:04 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:04 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:04 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:04 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:04 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:04 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:04 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:04 swdev14 kernel:  [ro0x603>] er_> [<ic_
Oct 12 09:53:04 swdev14 kernel:  recf>] <c013488_pr<4>  pr<c010tra888>] check_pt_x1f9d9>
Oct 12 09:53:04 swdev14 kernel:  [<c029926] s496/0xr19136>] pt [<mco<4> _spre+0xb[<7
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x1] dunc[<cwn_<c013.lo338ck_> [omec514dc0unl02404f3>] qest[<c0nt56/0>] de+0xecfa>outdst_oux2f4/0x18148
Oct 12 09:53:04 swdev14 kernel: <4 nf_11e
Oct 12 09:53:04 swdev14 kernel: 4>] 0x239>[<out>] suuntsub_pree+0xpin_unlox34 subt+0pre2/0a2c7ck+134eemp29a2clocc013nt+265end
Oct 12 09:53:04 swdev14 kernel: <4 tcp [unttcp_v4_s+0x5fmit>x3c>] +0x<c025sen8d
Oct 12 09:53:04 swdev14 kernel: ] a65>] t+0x34mpt13emp34apt_ciox1]x50>e+040x13d>] e+0>] aue_fritcri<4> [<mc<cwri1061_pa3>scle 2/27is /0267> ond0pre
Oct 12 09:53:04 swdev14 kernel: <] c [<ck_
Oct 12 09:53:04 swdev14 kernel: <4con<4> [<c_r0c0/fde
Oct 12 09:53:04 swdev14 kernel: < smter4
Oct 12 09:53:04 swdev14 kernel: <er_in/0_con9/efb2>]12cheing1e1d6edce+08>] _ti106fck] 0xb+0x8a/[<ctimi9
Oct 12 09:53:04 swdev14 kernel: _pr0x4__mcount+0x1
Oct 12 09:53:04 swdev14 kernel: <] _sqre0298491>] __+0x+0<c04de+0x0/029+0x01ck.46
Oct 12 09:53:04 swdev14 kernel: mutexve] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:04 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:04 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:04 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:04 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:04 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:04 swdev14 kernel:  [<cg[<nt4sgat<4 s7
Oct 12 09:53:04 swdev14 kernel: <>97a>6>]0xdte+0>0c00xce_wx0mi15e /0n[<x01xd0>+0c4>o> [ouc6/481cr46d_xhed>_mu+pd/ro03>e i< c>]c01_p
Oct 12 09:53:04 swdev14 kernel: <4 <c0tr81fd9
Oct 12 09:53:04 swdev14 kernel:  [<]9/0r1936pt [m4>_re+[<7
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:04 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:04 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:04 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:04 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:04 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:04 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:04 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:04 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:04 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:04 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:04 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:04 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:04 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:04 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:04 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:04 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:04 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:04 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:04 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:04 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:04 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+at<4 7
Oct 12 09:53:04 swdev14 kernel: <>x97a>6>xde+0>0c0xc_wx0ummio15e v/t[<x00xdx0>cc04>o4> [ou0c02/41cr4d_x8che>w_mu+pd/o> [<ic_
Oct 12 09:53:04 swdev14 kernel:  rec>]c01348_pre<4>  pr<c01tra88>] checkptx1f9d9>7
Oct 12 09:53:04 swdev14 kernel:  [<c02992] s49/0x10r191936>]pt [mc4>_re+[<07
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:04 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:04 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:04 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:04 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:04 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:04 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:04 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:04 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:04 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:04 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:04 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:04 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:04 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:04 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:04 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:05 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:05 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:05 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:05 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:05 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:05 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0254a65>] tcp_sgat<4> s97
Oct 12 09:53:05 swdev14 kernel: <>]97a>]6>xde+0> 0c00xc0mi5e] v/t[x010xd0>0c0>o> [o0c02/4x1cr46d_xched> _mut+pd/o03>er ic
Oct 12 09:53:05 swdev14 kernel: < f>c013_p<4 <c0t88>] cpx1d9
Oct 12 09:53:05 swdev14 kernel:  [<c]9/0136>p [mc4_sre+[7
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:05 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:05 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:05 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:05 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:05 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c5<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:05 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:05 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:05 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:05 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:05 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:05 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 last message repeated 2 times
Oct 12 09:53:05 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:05 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:05 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:05 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:05 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:05 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:05 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:05 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:05 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:05 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:05 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:05 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:05 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:05 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:05 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:05 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:05 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:05 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:05 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:05 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:05 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:05 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:05 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:05 swdev14 kernel:  [<c011f0b2>] release_console_sem+0x59/0xcf
Oct 12 09:53:05 swdev14 kernel:  [<c011efb2>] vprintk+0x128/0x16f
Oct 12 09:53:05 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:05 swdev14 kernel:  [<c011ee86>] printk+0x1d/0x21
Oct 12 09:53:05 swdev14 kernel:  [<c0106edc>] show_trace+0x4e/0x8d
Oct 12 09:53:05 swdev14 kernel: 134g+0 [<k+<c0<4__c01348_p91134ng[<21>] rqre6
Oct 12 09:53:05 swdev14 kernel: x85/001> [<cdo> [<cef1c
Oct 12 09:53:05 swdev14 kernel: >] <4.te0x5/001save> [rt_xeb _1
Oct 12 09:53:05 swdev14 kernel: ck+0xb/[<c0x132<c/0x15171e6
Oct 12 09:53:05 swdev14 kernel:  [<dd>it+0> [_output+0xd5/6
Oct 12 09:53:05 swdev14 kernel: </06d402pu023bf_s<c02ut> [<_q9e<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:05 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:05 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:05 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:05 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:05 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:05 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:05 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:05 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:05 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:05 swdev14 last message repeated 2 times
Oct 12 09:53:05 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:05 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:05 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:05 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:05 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:05 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:05 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:05 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:05 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:05 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:05 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:05 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:05 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:05 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:06 swdev14 kernel:  [<c0134936>] touch_preempt_ti0x9 _spina/0] chimi
Oct 12 09:53:06 swdev14 kernel:  _s34
Oct 12 09:53:06 swdev14 kernel: <4d><c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:06 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:06 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:06 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:06 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:06 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:06 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 last message repeated 2 times
Oct 12 09:53:06 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:06 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:06 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:06 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:06 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:06 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:06 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:06 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:06 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:06 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:06 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:06 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:06 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:06 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:06 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:06 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:06 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:06 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:06 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:06 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:06 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:06 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:06 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:06 swdev14 kernel:  [<c011f0b2>] release_console_sem+0x59/0xcf
Oct 12 09:53:06 swdev14 kernel:  [<c011efb2>] vprintk+0x128/0x16f
Oct 12 09:53:06 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:06 swdev14 kernel:  [<c011ee86>] printk+0x1d/0x21
Oct 12 09:53:06 swdev14 kernel:  [<c0106edc>] show_trace+0x4e/0x8d
Oct 12 09:53:06 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:06 swdev14 kernel:  [<c0106fd9>] dump_stack+0x23/0x27
Oct 12 09:53:06 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:06 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:06 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:06 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:06 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:06 swdev14 kernel:  [<c029a358>] _spin_unlock_irqrestore+0xb/0x36
Oct 12 09:53:06 swdev14 kernel:  [<c0298491>] __down+0x85/0x107
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:06 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:06 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:06 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:06 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:06 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:06 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:06 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:06 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:06 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:06 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:06 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mco[<c_obfow+while atfti<c02 i83
Oct 12 09:53:06 swdev14 kernel: t+0 schedule+0xxbe<4>t+0>]
Oct 12 09:53:06 swdev14 kernel:  cond_resed+out<c0ti[<cpreem2/0> co26/1>]
Oct 12 09:53:06 swdev14 kernel: pt<c0134<cing+0x191/0x1f9 _spin_unlock+0x1a/0x34
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c0134936>] [<c0134af1>] touch_preempt_timing+0x46/0x4a sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c02996d9>] [<c0134af1>] cond_resched+0x26/0x83 sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c029a2c7>] [<c0299716>] _spin_unlock+0x1a/0x34 cond_resched+0x63/0x83
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c0134888>] [<c0133b8e>] check_preempt_timing+0x191/0x1f9 _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c029a2c7>] [<c011f8f1>] _spin_unlock+0x1a/0x34 profile_hook+0x1d/0x47
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c013414d>] [<c011fde3>] __mcount+0x1d/0x21 profile_tick+0x63/0x65
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c0265d8c>] [<c0113d03>] tcp_v4_send_check+0xe/0xe2 smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c025fbf1>] [<c0106bb6>] tcp_transmit_skb+0x432/0x852 apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] [<c029007b>] mcount+0x14/0x18 packet_sendmsg+0x210/0x280
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c0265d8c>] [<c0299fdb>] tcp_v4_send_check+0xe/0xe2 _spin_lock+0x56/0x78
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c025fc9d>] [<c02405a7>] tcp_transmit_skb+0x4de/0x852 dev_watchdog+0x0/0xb7
Oct 12 09:53:06 swdev14 kernel: 
Oct 12 09:53:06 swdev14 kernel:  [<c01b2056>] [<c02405c3>] memcpy+x2eb [3c59x]
Oct 12 09:53:06 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:06 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:06 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:06 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:06 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:06 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:06 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:06 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:06 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:06 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:06 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:06 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:06 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:06 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:06 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:06 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:06 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:06 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:07 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:07 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:07 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:07 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:07 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:07 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:07 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:07 swdev14 last message repeated 2 times
Oct 12 09:53:07 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:07 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:07 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:07 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:07 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:07 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:07 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:07 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:07 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:07 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:07 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:07 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:07 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:07 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:07 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:07 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:07 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:07 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:07 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:07 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:07 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:07 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:07 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:07 swdev14 kernel:  [<c011f0b2>] release_console_sem+0x59/0xcf
Oct 12 09:53:07 swdev14 kernel:  [<c011efb2>] vprintk+0x128/0x16f
Oct 12 09:53:07 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:07 swdev14 kernel:  [<c011ee86>] printk+0x1d/0x21
Oct 12 09:53:07 swdev14 kernel:  [<c0106edc>] show_trace+0x4e/0x8d
Oct 12 09:53:07 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:07 swdev14 kernel:  [<c0106fd9>] dump_stack+0x23/0x27
Oct 12 09:53:07 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:07 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:07 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:07 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:07 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:07 swdev14 kernel:  [<c029a358>] _spin_unlock_irqrestore+0xb/0x36
Oct 12 09:53:07 swdev14 kernel:  [<c0298491>] __down+0x85/0x107
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:07 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:07 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:07 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:07 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:07 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:07 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:07 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:07 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:07 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:07 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:07 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:07 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:07 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:07 swdev14 kernel:  [>] de+0xecfa>outpdst_ox2f4/0x18
Oct 12 09:53:07 swdev14 kernel: <148> [<c02_hoe
Oct 12 09:53:07 swdev14 kernel: <4] d539>it+> [t_o [<_pr134afmpt4> [spi
Oct 12 09:53:07 swdev14 kernel: +0x8c01/02c7> [<c0134eck0x<c04
Oct 12 09:53:07 swdev14 kernel: ] 1>] eck+0[<rans/0x14/0026che> [_skb<43c3>] 4a8g+0[<nt54sg+4at_c
Oct 12 09:53:07 swdev14 kernel:   su97
Oct 12 09:53:07 swdev14 kernel: >] x97
Oct 12 09:53:07 swdev14 kernel: <4a>36
Oct 12 09:53:07 swdev14 kernel: >] 0x5da>te+0x12> [0xc01+0xac01e_wakx0/ummyion15e+0] vf[<c0x500100x5dulx0>c+0xc02>on4> [<c0ouc+0c06/488x1c01pree46nd_rex8hed+0> w_mutex+0 prd/0ro0x603>] er_ [<ic_
Oct 12 09:53:07 swdev14 kernel:  rcf>]<c013488_pr
Oct 12 09:53:07 swdev14 kernel:   p<c010tra88>] check_pptx1f9d9>7
Oct 12 09:53:07 swdev14 kernel:  [<c0299267] s49/0xp36>]pt_> [<cmco4> _spiore+0xb[<07
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:07 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:07 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:07 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:07 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:07 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:07 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:07 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:07 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:07 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:07 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:07 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:07 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:07 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:07 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:07 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:07 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:07 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:08 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:08 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:08 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:08 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:08 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:08 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:08 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 last message repeated 2 times
Oct 12 09:53:08 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:08 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:08 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:08 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:08 swdev14 kernel: ummyio15e+0] vf/0xnt[<0x50100x5dux0>c+0c02>on> [<c0ou0c026/48x1c01re46/nd_rex8ched+> _mute+0 pd/0o0x03>] er [ic
Oct 12 09:53:08 swdev14 kernel: <4 rcf>]<c0134_pr
Oct 12 09:53:08 swdev14 kernel:   p<c01tra88>] check_pt_1fd97
Oct 12 09:53:08 swdev14 kernel:  [<c029996/0xpr191936>]pt_> [<mc<4> _spore+0x[<7
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:08 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:08 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:08 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:08 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:08 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:08 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:08 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:08 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:08 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:08 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:08 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:08 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:08 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:08 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:08 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:08 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:08 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:08 swdev14 kernel:  [<c025fbf1>] tcp_transmit_skb+0x432/0x852
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>cp_transmit_skb+0x432/0x852
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c0265d8c>] tcp_v4_send_check+0xe/0xe2
Oct 12 09:53:08 swdev14 kernel:  [<c025fc9d>] tcp_transmit_skb+0x4de/0x852
Oct 12 09:53:08 swdev14 kernel:  [<c01b2056>] memcpy+0x12/0x3c
Oct 12 09:53:08 swdev14 kernel:  [<c0260a83>] tcp_write_xmit+0x149/0x2c0
Oct 12 09:53:08 swdev14 kernel:  [<c0254a8e>] tcp_sendmsg+0x4ff/0x108d
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c0254a65>] tcp_sendmsg+0x4d6/0x108d
Oct 12 09:53:08 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:08 swdev14 last message repeated 2 times
Oct 12 09:53:08 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:08 swdev14 kernel:  [<c02764e0>] inet_sendmsg+0x50/0x5b
Oct 12 09:53:08 swdev14 kernel:  [<c0227bda>] sock_aio_write+0x124/0x136
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c015b73d>] do_sync_write+0xaa/0xd6
Oct 12 09:53:08 swdev14 kernel:  [<c01333e3>] autoremove_wake_function+0x0/0x57
Oct 12 09:53:08 swdev14 kernel:  [<c01a9bd8>] dummy_file_permission+0x8/0xc
Oct 12 09:53:08 swdev14 kernel:  [<c015b80b>] vfs_write+0xa2/0x134
Oct 12 09:53:08 swdev14 kernel:  [<c015b869>] vfs_write+0x100/0x134
Oct 12 09:53:08 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:08 swdev14 kernel:  [<c015b967>] sys_write+0x50/0x7a
Oct 12 09:53:08 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 09:53:08 swdev14 kernel: scheduling while atomic: mount/0x04010002/2732
Oct 12 09:53:08 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:08 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:08 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:08 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:08 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0prx02> d91p[l>c.4av/p4> p> [/0x48e0x1a/023c/0x11514
Oct 12 09:53:08 swdev14 kernel:   i>st> b_p82/0134cou4> ock> [reem/0x_pre82/7+0xchecingc7+0x4d>d/0v4_/0ra/0 m
Oct 12 09:53:08 swdev14 kernel: <4ck<c0t_skb+0
Oct 12 09:53:08 swdev14 kernel: <4em4>i4>x4<c+0<c6/0x01x82/013pt_co97
Oct 12 09:53:08 swdev14 kernel: mpx97_a0xe0>]g+0>] m18
Oct 12 09:53:08 swdev14 kernel: wr4>ake_fun0x5_fil0x vfs0xrite+0x100/
Oct 12 09:53:08 swdev14 kernel: <4 mco
Oct 12 09:53:08 swdev14 kernel: <4ite+0x50/0x7a
Oct 12 09:53:08 swdev14 kernel:  [52/lin10lerx63/0299/0x99ed [<preemp46d_resched+0x26/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:08 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:08 swdev14 kernel:  [<c02996d9>] cond_resched+0x26/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0299716>] cond_resched+0x63/0x83
Oct 12 09:53:08 swdev14 kernel:  [<c0133b8e>] _rw_mutex_read_lock+0x24/0x39
Oct 12 09:53:08 swdev14 kernel:  [<c011f8f1>] profile_hook+0x1d/0x47
Oct 12 09:53:08 swdev14 kernel:  [<c011fde3>] profile_tick+0x63/0x65
Oct 12 09:53:08 swdev14 kernel:  [<c0113d03>] smp_apic_timer_interrupt+0x60/0xe4
Oct 12 09:53:08 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 09:53:08 swdev14 kernel:  [<c011f0b2>] release_console_sem+0x59/0xcf
Oct 12 09:53:08 swdev14 kernel:  [<c011efb2>] vprintk+0x128/0x16f
Oct 12 09:53:08 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:08 swdev14 kernel:  [<c011ee86>] printk+0x1d/0x21
Oct 12 09:53:09 swdev14 kernel:  [<c0106edc>] show_trace+0x4e/0x8d
Oct 12 09:53:09 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:09 swdev14 kernel:  [<c0106fd9>] dump_stack+0x23/0x27
Oct 12 09:53:09 swdev14 kernel:  [<c0299267>] schedule+0xbaf/0xbe2
Oct 12 09:53:09 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:09 swdev14 kernel:  [<c0134888>] check_preempt_timing+0x191/0x1f9
Oct 12 09:53:09 swdev14 kernel:  [<c0134936>] touch_preempt_timing+0x46/0x4a
Oct 12 09:53:09 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:09 swdev14 kernel:  [<c029a358>] _spin_unlock_irqrestore+0xb/0x36
Oct 12 09:53:09 swdev14 kernel:  [<c0298491>] __down+0x85/0x107
Oct 12 09:53:09 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:09 swdev14 kernel:  [<c0298496>] __down+0x8a/0x107
Oct 12 09:53:09 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 09:53:09 swdev14 kernel:  [<c029865c>] __down_failed+0x8/0xc
Oct 12 09:53:09 swdev14 kernel:  [<c0133ee3>] .text.lock.mutex+0x5/0x146
Oct 12 09:53:09 swdev14 kernel:  [<c0133884>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 09:53:09 swdev14 kernel:  [<e0840abe>] boomerang_start_xmit+0x123/0x2eb [3c59x]
Oct 12 09:53:09 swdev14 kernel:  [<c013414d>] __mcount+0x1d/0x21
Oct 12 09:53:09 swdev14 kernel:  [<c029a2b8>] _spin_unlock+0xb/0x34
Oct 12 09:53:09 swdev14 kernel:  [<c02404f3>] qdisc_restart+0x132/0x1e6
Oct 12 09:53:09 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:09 swdev14 kernel:  [<c0240517>] qdisc_restart+0x156/0x1e6
Oct 12 09:53:09 swdev14 kernel:  [<c02313dd>] dev_queue_xmit+0x239/0x2d9
Oct 12 09:53:09 swdev14 kernel:  [<c024ecfa>] ip_finish_output+0xd5/0x216
Oct 12 09:53:09 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:09 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 09:53:09 swdev14 kernel:  [<c025148e>] dst_output+0x1a/0x2f
Oct 12 09:53:09 swdev14 kernel:  [<c023bf2c>] nf_hook_slow+0xec/0x11e
Oct 12 09:53:09 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:09 swdev14 kernel:  [<c024f539>] ip_queue_xmit+0x495/0x59e
Oct 12 09:53:09 swdev14 kernel:  [<c0251474>] dst_output+0x0/0x2f
Oct 12 09:53:09 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:09 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:53:09 swdev14 kernel:  [<c029a2c7>] _spin_unlock+0x1a/0x34
Oct 12 09:53:09 swdev14 kernel:  [<c0134af1>] sub_preempt_count+0x82/0x97
Oct 12 09:56:00 swdev14 syslogd 1.4.1: restart.

does the patch below make your system bootable? It should fix the two most common messages you got.

Ingo

@@ -169,7 +169,7 @@ int profile_event_unregister(enum profil
 }
 
 static struct notifier_block * profile_listeners;
-static rwlock_t profile_lock = RW_LOCK_UNLOCKED;
+static raw_rwlock_t profile_lock = RAW_RW_LOCK_UNLOCKED;
  
 int register_profile_notifier(struct notifier_block * nb)
 {

@@ -832,8 +832,8 @@ struct vortex_private {
 	u16 deferred;						/* Resend these interrupts when we
 										 * bale from the ISR */
 	u16 io_size;						/* Size of PCI region (for release_region) */
-	spinlock_t lock;					/* Serialise access to device & its vortex_private */
-	spinlock_t mdio_lock;				/* Serialise access to mdio hardware */
+	raw_spinlock_t lock;					/* Serialise access to device & its vortex_private */
+	raw_spinlock_t mdio_lock;				/* Serialise access to mdio hardware */
 	struct mii_if_info mii;				/* MII lib hooks/info */
 };
 

No. Attached log from this boot. Also worth noting: I have no keyboard while trying to boot this. This doesn't really surprise me, but I am seeing hit or miss on the keyboard (more times than not the keyboard is dead) with the T3 patch also. Doesn't seem to be an issue on my 933 SMP system at home. Is this a regression? Ideas?

kr

Oct 12 11:46:34 swdev14 syslogd 1.4.1: restart.
Oct 12 11:46:34 swdev14 syslog: syslogd startup succeeded
Oct 12 11:46:34 swdev14 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Oct 12 11:46:34 swdev14 syslog: klogd startup succeeded
Oct 12 11:46:34 swdev14 kernel: 26/0x83>
Oct 12 11:46:34 swdev14 kernel:  => ended at:   <cond_resched+0x26/0x83>
Oct 12 11:46:34 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:34 swdev14 kernel:  [<c013484c>] check_preempt_timing+0x161/0x1f9
Oct 12 11:46:34 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:34 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:34 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:34 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:34 swdev14 kernel:  [<c0133816>] _mutex_lock+0x19/0x3f
Oct 12 11:46:34 swdev14 kernel:  [<c0133878>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 11:46:34 swdev14 kernel:  [<c01cbe88>] tty_register_ldisc+0x37/0xa4
Oct 12 11:46:34 swdev14 kernel:  [<c036be3e>] console_init+0x27/0x4a
Oct 12 11:46:34 swdev14 kernel:  [<c035487a>] start_kernel+0xd7/0x1c6
Oct 12 11:46:34 swdev14 kernel:  [<c03543b0>] unknown_bootoption+0x0/0x15d
Oct 12 11:46:34 swdev14 irqbalance: irqbalance startup succeeded
Oct 12 11:46:34 swdev14 kernel: Console: colour VGA+ 80x25
Oct 12 11:46:34 swdev14 kernel: Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Oct 12 11:46:34 swdev14 kernel: Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Oct 12 11:46:34 swdev14 kernel: Memory: 513500k/523712k available (1645k kernel code, 9608k reserved, 726k data, 272k init, 0k highmem)
Oct 12 11:46:34 swdev14 kernel: Checking if this processor honours the WP bit even in supervisor mode... Ok.
Oct 12 11:46:34 swdev14 kernel: Security Scaffold v1.0.0 initialized
Oct 12 11:46:34 swdev14 kernel: Capability LSM initialized
Oct 12 11:46:34 swdev14 kernel: Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Oct 12 11:46:34 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 11:46:34 swdev14 portmap: portmap startup succeeded
Oct 12 11:46:34 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 11:46:34 swdev14 kernel: CPU: Physical Processor ID: 0
Oct 12 11:46:34 swdev14 kernel: Intel machine check architecture supported.
Oct 12 11:46:34 swdev14 kernel: Intel machine check reporting enabled on CPU#0.
Oct 12 11:46:34 swdev14 kernel: CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 11:46:34 swdev14 kernel: Enabling fast FPU save and restore... done.
Oct 12 11:46:34 swdev14 kernel: Enabling unmasked SIMD FPU exception support... done.
Oct 12 11:46:34 swdev14 kernel: Checking 'hlt' instruction... OK.
Oct 12 11:46:34 swdev14 kernel: CPU0: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 11:46:34 swdev14 kernel: per-CPU timeslice cutoff: 1462.71 usecs.
Oct 12 11:46:34 swdev14 kernel: task migration cache decay timeout: 2 msecs.
Oct 12 11:46:34 swdev14 kernel: Booting processor 1/1 eip 2000
Oct 12 11:46:34 swdev14 kernel: Initializing CPU#1
Oct 12 11:46:34 swdev14 rpc.statd[2649]: Version 1.0.6 Starting
Oct 12 11:46:34 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 11:46:34 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 11:46:34 swdev14 nfslock: rpc.statd startup succeeded
Oct 12 11:46:34 swdev14 kernel: CPU: Physical Processor ID: 0
Oct 12 11:46:34 swdev14 kernel: Intel machine check architecture supported.
Oct 12 11:46:34 swdev14 kernel: Intel machine check reporting enabled on CPU#1.
Oct 12 11:46:34 swdev14 kernel: CPU1: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 11:46:34 swdev14 kernel: CPU1: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 11:46:34 swdev14 kernel: Booting processor 2/6 eip 2000
Oct 12 11:46:34 swdev14 kernel: Initializing CPU#2
Oct 12 11:46:34 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 11:46:34 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 11:46:34 swdev14 kernel: CPU: Physical Processor ID: 3
Oct 12 11:46:35 swdev14 kernel: Intel machine check architecture supported.
Oct 12 11:46:35 swdev14 kernel: Intel machine check reporting enabled on CPU#2.
Oct 12 11:46:35 swdev14 kernel: CPU2: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 11:46:35 swdev14 kernel: CPU2: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 11:46:35 swdev14 kernel: Booting processor 3/7 eip 2000
Oct 12 11:46:35 swdev14 kernel: Initializing CPU#3
Oct 12 11:46:35 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 11:46:35 swdev14 rpcidmapd: rpc.idmapd startup succeeded
Oct 12 11:46:35 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 11:46:35 swdev14 kernel: CPU: Physical Processor ID: 3
Oct 12 11:46:35 swdev14 kernel: Intel machine check architecture supported.
Oct 12 11:46:35 swdev14 kernel: Intel machine check reporting enabled on CPU#3.
Oct 12 11:46:35 swdev14 kernel: CPU3: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 11:46:35 swdev14 kernel: CPU3: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 11:46:35 swdev14 kernel: Total of 4 processors activated (20611.07 BogoMIPS).
Oct 12 11:46:35 swdev14 kernel: checking TSC synchronization across 4 CPUs: passed.
Oct 12 11:46:35 swdev14 kernel: ksoftirqd started up.
Oct 12 11:46:35 swdev14 last message repeated 2 times
Oct 12 11:46:35 swdev14 kernel: Brought up 4 CPUs
Oct 12 11:46:35 swdev14 random: Initializing random number generator:  succeeded
Oct 12 11:46:35 swdev14 kernel: ksoftirqd started up.
Oct 12 11:46:35 swdev14 rc: Starting pcmcia:  succeeded
Oct 12 11:46:35 swdev14 kernel: checking if image is initramfs...it isn't (no cpio magic); looks like an initrd
Oct 12 11:46:35 swdev14 kernel: Freeing initrd memory: 207k freed
Oct 12 11:46:35 swdev14 kernel: NET: Registered protocol family 16
Oct 12 11:46:35 swdev14 kernel: PCI: PCI BIOS revision 2.10 entry at 0xfd915, last bus=5
Oct 12 11:46:35 swdev14 kernel: PCI: Using configuration type 1
Oct 12 11:46:35 swdev14 kernel: mtrr: v2.0 (20020519)
Oct 12 11:46:35 swdev14 kernel: Linux Plug and Play Support v0.97 (c) Adam Belay
Oct 12 11:46:35 swdev14 kernel: PCI: Probing PCI hardware
Oct 12 11:46:35 swdev14 kernel: PCI: Probing PCI hardware (bus 00)
Oct 12 11:46:35 swdev14 kernel: PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1
Oct 12 11:46:35 swdev14 kernel: PCI: Transparent bridge - 0000:00:1e.0
Oct 12 11:46:35 swdev14 kernel: Simple Boot Flag at 0x36 set to 0x1
Oct 12 11:46:35 swdev14 kernel: apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
Oct 12 11:46:35 swdev14 kernel: apm: disabled - APM is not SMP safe.
Oct 12 11:46:35 swdev14 kernel: VFS: Disk quotas dquot_6.5.1
Oct 12 11:46:35 swdev14 kernel: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Oct 12 11:46:35 swdev14 kernel: Initializing Cryptographic API
Oct 12 11:46:35 swdev14 kernel: vesafb: probe of vesafb0 failed with error -6
Oct 12 11:46:35 swdev14 kernel: isapnp: Scanning for PnP cards...
Oct 12 11:46:35 swdev14 kernel: isapnp: No Plug & Play device found
Oct 12 11:46:35 swdev14 kernel: scheduling while atomic: swapper/0x04000001/1
Oct 12 11:46:35 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:35 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:35 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:35 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:35 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:36 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:36 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:36 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:36 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:36 swdev14 kernel:  [<c0133b82>] _rw_mutex_read_lock+0x24/0x39
Oct 12 11:46:36 swdev14 kernel:  [<c011f62c>] profile_handoff_task+0x1a/0x52
Oct 12 11:46:36 swdev14 netfs: Mounting NFS filesystems:  succeeded
Oct 12 11:46:36 swdev14 kernel:  [<c011c508>] __put_task_struct+0x66/0x119
Oct 12 11:46:36 swdev14 kernel:  [<c0298a0b>] schedule+0x35f/0xbe2
Oct 12 11:46:36 swdev14 kernel:  [<c029a392>] _spin_unlock_irq+0x1b/0x35
Oct 12 11:46:36 swdev14 netfs: Mounting other filesystems:  succeeded
Oct 12 11:46:36 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:36 swdev14 kernel:  [<c0134ae5>] sub_preempt_count+0x82/0x97
Oct 12 11:46:36 swdev14 kernel:  [<c029a392>] _spin_unlock_irq+0x1b/0x35
Oct 12 11:46:36 swdev14 kernel:  [<c029938c>] wait_for_completion+0x84/0xe3
Oct 12 11:46:36 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:36 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:36 swdev14 kernel:  [<c012eab2>] queue_work+0x72/0xa0
Oct 12 11:46:36 swdev14 kernel:  [<c012e9cd>] call_usermodehelper+0xc7/0xce
Oct 12 11:46:36 swdev14 kernel:  [<c012e89d>] __call_usermodehelper+0x0/0x69
Oct 12 11:46:36 swdev14 kernel:  [<c01e54d5>] class_hotplug+0x0/0x44
Oct 12 11:46:36 swdev14 kernel:  [<c01aeeb4>] kobject_hotplug+0x27e/0x2e2
Oct 12 11:46:36 swdev14 kernel:  [<c01ae0f0>] create_dir+0x3e/0x4e
Oct 12 11:46:36 swdev14 autofs: automount startup succeeded
Oct 12 11:46:36 swdev14 kernel:  [<c01ae34e>] kobject_add+0x8c/0xfa
Oct 12 11:46:36 swdev14 kernel:  [<c01e56b7>] class_device_add+0x8d/0x15e
Oct 12 11:46:36 swdev14 kernel:  [<c01e5d0b>] class_simple_device_add+0xa3/0x104
Oct 12 11:46:36 swdev14 kernel:  [<c01cfb4d>] tty_register_device+0x73/0xdd
Oct 12 11:46:36 swdev14 kernel:  [<c01e64a8>] kobj_map+0xa0/0x136
Oct 12 11:46:36 swdev14 kernel:  [<c0164ba4>] cdev_add+0x4b/0x4f
Oct 12 11:46:36 swdev14 kernel:  [<c01cfe6a>] tty_register_driver+0x14c/0x243
Oct 12 11:46:36 swdev14 smartd[2807]: smartd version 5.21 Copyright (C) 2002-3 Bruce Allen 
Oct 12 11:46:36 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 11:46:36 swdev14 smartd[2807]: Home page is http://smartmontools.sourceforge.net/  
Oct 12 11:46:36 swdev14 kernel:  [<c036c34d>] legacy_pty_init+0x28a/0x2c8
Oct 12 11:46:36 swdev14 smartd[2807]: Opened configuration file /etc/smartd.conf 
Oct 12 11:46:36 swdev14 kernel:  [<c036c657>] pty_init+0xd/0x16
Oct 12 11:46:36 swdev14 smartd[2807]: Configuration file /etc/smartd.conf parsed. 
Oct 12 11:46:36 swdev14 kernel:  [<c03549b2>] do_initcalls+0x30/0xbd
Oct 12 11:46:36 swdev14 smartd[2807]: Device: /dev/hda, opened 
Oct 12 11:46:36 swdev14 kernel:  [<c0100541>] init+0x87/0x19a
Oct 12 11:46:36 swdev14 kernel:  [<c01004ba>] init+0x0/0x19a
Oct 12 11:46:36 swdev14 smartd[2807]: Device: /dev/hda, not found in smartd database. 
Oct 12 11:46:36 swdev14 kernel:  [<c01042c9>] kernel_thread_helper+0x5/0xb
Oct 12 11:46:36 swdev14 smartd[2807]: Device: /dev/hda, is SMART capable. Adding to "monitor" list. 
Oct 12 11:46:36 swdev14 kernel: scheduling while atomic: swapper/0x04000001/1
Oct 12 11:46:36 swdev14 smartd[2807]: Monitoring 1 ATA and 0 SCSI devices 
Oct 12 11:46:36 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:37 swdev14 smartd[2809]: smartd has fork()ed into background mode. New PID=2809. 
Oct 12 11:46:37 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:37 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:37 swdev14 smartd: smartd startup succeeded
Oct 12 11:46:37 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:37 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:37 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:37 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:37 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:37 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:37 swdev14 kernel:  [<c0133b82>] _rw_mutex_read_lock+0x24/0x39
Oct 12 11:46:37 swdev14 kernel:  [<c011f62c>] profile_handoff_task+0x1a/0x52
Oct 12 11:46:37 swdev14 kernel:  [<c011c508>] __put_task_struct+0x66/0x119
Oct 12 11:46:37 swdev14 kernel:  [<c0298a0b>] schedule+0x35f/0xbe2
Oct 12 11:46:37 swdev14 kernel:  [<c029a392>] _spin_unlock_irq+0x1b/0x35
Oct 12 11:46:37 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:37 swdev14 sshd:  succeeded
Oct 12 11:46:37 swdev14 kernel:  [<c0134ae5>] sub_preempt_count+0x82/0x97
Oct 12 11:46:37 swdev14 kernel:  [<c029a392>] _spin_unlock_irq+0x1b/0x35
Oct 12 11:46:37 swdev14 kernel:  [<c029938c>] wait_for_completion+0x84/0xe3
Oct 12 11:46:37 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:37 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:37 swdev14 kernel:  [<c012eab2>] queue_work+0x72/0xa0
Oct 12 11:46:37 swdev14 kernel:  [<c012e9cd>] call_usermodehelper+0xc7/0xce
Oct 12 11:46:37 swdev14 kernel:  [<c012e89d>] __call_usermodehelper+0x0/0x69
Oct 12 11:46:37 swdev14 kernel:  [<c01e54d5>] class_hotplug+0x0/0x44
Oct 12 11:46:37 swdev14 kernel:  [<c01aeeb4>.20
Oct 12 11:46:37 swdev14 xinetd: xinetd startup succeeded
Oct 12 11:46:37 swdev14 kernel: ip_tables: (C) 2000-2002 Netfilter core team
Oct 12 11:46:37 swdev14 kernel: NET: Registered protocol family 10
Oct 12 11:46:37 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:37 swdev14 kernel: caller is raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:37 swdev14 ntpdate[2873]: can't find host wizard 
Oct 12 11:46:37 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:37 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:37 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:37 swdev14 kernel:  [<e0b1d282>] inet6_create+0x26f/0x315 [ipv6]
Oct 12 11:46:37 swdev14 kernel:  [<c02284aa>] __sock_create+0xd5/0x19d
Oct 12 11:46:37 swdev14 kernel:  [<c02285dc>] sock_create_kern+0x33/0x37
Oct 12 11:46:37 swdev14 kernel:  [<e09bc824>] icmpv6_init+0xc4/0x110 [ipv6]
Oct 12 11:46:37 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:37 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:37 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc856>] icmpv6_init+0xf6/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b1d282>] inet6_create+0x26f/0x315 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c02284aa>] __sock_create+0xd5/0x19d
Oct 12 11:46:38 swdev14 kernel:  [<c02285dc>] sock_create_kern+0x33/0x37
Oct 12 11:46:38 swdev14 kernel:  [<e09bc824>] icmpv6_init+0xc4/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc856>] icmpv6_init+0xf6/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b1d282>] inet6_create+0x26f/0x315 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c02284aa>] __sock_create+0xd5/0x19d
Oct 12 11:46:38 swdev14 kernel:  [<c02285dc>] sock_create_kern+0x33/0x37
Oct 12 11:46:38 swdev14 kernel:  [<e09bc824>] icmpv6_init+0xc4/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc856>] icmpv6_init+0xf6/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b1d282>] inet6_create+0x26f/0x315 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c02284aa>] __sock_create+0xd5/0x19d
Oct 12 11:46:38 swdev14 kernel:  [<c02285dc>] sock_create_kern+0x33/0x37
Oct 12 11:46:38 swdev14 kernel:  [<e09bc824>] icmpv6_init+0xc4/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc856>] icmpv6_init+0xf6/0x110 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc14b>] inet6_init+0xb9/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32f56>] raw_v6_hash+0x62/0x85 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b1d282>] inet6_create+0x26f/0x315 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c02284aa>] __sock_create+0xd5/0x19d
Oct 12 11:46:38 swdev14 kernel:  [<c02285dc>] sock_create_kern+0x33/0x37
Oct 12 11:46:38 swdev14 kernel:  [<e09bc60d>] ndisc_init+0x32/0xe9 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc161>] inet6_init+0xcf/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: modprobe/2819
Oct 12 11:46:38 swdev14 kernel: caller is raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c011a2aa>] smp_processor_id+0xa8/0xb9
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e0b32fda>] raw_v6_unhash+0x61/0xad [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc646>] ndisc_init+0x6b/0xe9 [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<e09bc161>] inet6_init+0xcf/0x20c [ipv6]
Oct 12 11:46:38 swdev14 kernel:  [<c0138c03>] sys_init_module+0x15c/0x1ce
Oct 12 11:46:38 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:38 swdev14 kernel: IPv6 over IPv4 tunneling driver
Oct 12 11:46:38 swdev14 kernel: ------------[ cut here ]------------
Oct 12 11:46:38 swdev14 kernel: kernel BUG at kernel/mutex.c:185!
Oct 12 11:46:38 swdev14 kernel: invalid operand: 0000 [#1]
Oct 12 11:46:38 swdev14 kernel: PREEMPT SMP 
Oct 12 11:46:38 swdev14 kernel: Modules linked in: ipv6 autofs4 nfs lockd sunrpc iptable_filter ip_tables ide_cd cdrom 3c59x mii tg3 floppy sg scsi_mod parport_pc parport microcode dm_mod evdev usbhid uhci_hcd usbcore ext3 jbd
Oct 12 11:46:38 swdev14 kernel: CPU:    2
Oct 12 11:46:38 swdev14 kernel: EIP:    0060:[<c0133bbc>]    Not tainted VLI
Oct 12 11:46:38 swdev14 kernel: EFLAGS: 00010246   (2.6.9-rc4-mm1-VP-T7) 
Oct 12 11:46:38 swdev14 kernel: EIP is at _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:38 swdev14 kernel: eax: 00000000   ebx: 00000000   ecx: c0350e00   edx: c1466b80
Oct 12 11:46:38 swdev14 kernel: esi: ddef3ac8   edi: dd878814   ebp: de5d7f18   esp: de5d7f18
Oct 12 11:46:38 swdev14 kernel: ds: 007b   es: 007b   ss: 0068   preempt: 00000002
Oct 12 11:46:38 swdev14 kernel: Process sshd (pid: 2849, threadinfo=de5d6000 task=de4a7a80)
Oct 12 11:46:38 swdev14 kernel: Stack: de5d7f44 c02537ec c0350e00 00000016 c029a3c6 dfe12a80 ddef3c94 ddef3bfc 
Oct 12 11:46:38 swdev14 kernel:        dfe12a80 ddef3a80 ffffffea de5d7f5c c0275b18 ddef3a80 00000005 dfe12a80 
Oct 12 11:46:38 swdev14 kernel:        00000003 de5d7f78 c022889e dfe12a80 00000005 00000000 00000004 08090bf0 
Oct 12 11:46:38 swdev14 kernel: Call Trace:
Oct 12 11:46:38 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:38 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:38 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:39 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:39 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:39 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:39 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:39 swdev14 kernel: Code: 75 fc 89 ec 5d c3 55 89 e5 e8 21 fb fd ff 8b 4d 08 8b 01 85 c0 74 14 8d 41 04 ba ff ff 00 00 f0 0f c1 10 0f 85 6e 03 00 00 5d c3 <0f> 0b b9 00 1a c7 2a c0 eb e2 55 89 e5 e8 f2 fa fd ff 8b 4d 08 
Oct 12 11:46:39 swdev14 kernel:  <3>scheduling while atomic: sshd/0x04000001/2849
Oct 12 11:46:39 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c011f5d4>] profile_task_exit+0x18/0x56
Oct 12 11:46:39 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 11:46:39 swdev14 kernel:  [<c01216f8>] do_exit+0x1f/0x3bd
Oct 12 11:46:39 swdev14 kernel:  [<c029929f>] preempt_schedule+0x11/0x7a
Oct 12 11:46:39 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:39 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:39 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:39 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:39 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:39 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:39 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:39 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:39 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:39 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:39 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:39 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:39 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:39 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:39 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:39 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:39 swdev14 kernel: note: sshd[2849] exited with preempt_count 1
Oct 12 11:46:39 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2849
Oct 12 11:46:39 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c014b81d>] unmap_vmas+0x190/0x29b
Oct 12 11:46:39 swdev14 kernel:  [<c015006f>] exit_mmap+0xb2/0x1cc
Oct 12 11:46:39 swdev14 kernel:  [<c011c979>] mmput+0x3b/0xb9
Oct 12 11:46:39 swdev14 kernel:  [<c0121807>] do_exit+0x12e/0x3bd
Oct 12 11:46:39 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:39 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:39 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:39 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:39 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:39 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:39 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:39 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:39 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:39 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:39 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:39 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:39 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:39 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:39 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:39 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:39 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2849
Oct 12 11:46:39 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c0133816>] _mutex_lock+0x19/0x3f
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c014dff0>] remove_vm_struct+0x34/0x9f
Oct 12 11:46:39 swdev14 kernel:  [<c015012f>] exit_mmap+0x172/0x1cc
Oct 12 11:46:39 swdev14 kernel:  [<c011c979>] mmput+0x3b/0xb9
Oct 12 11:46:39 swdev14 kernel:  [<c0121807>] do_exit+0x12e/0x3bd
Oct 12 11:46:39 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:39 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:39 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:39 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:39 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:39 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:39 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:39 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:39 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:39 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:39 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:39 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:39 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:39 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:39 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:39 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:39 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2849
Oct 12 11:46:39 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:39 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:39 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:39 swdev14 kernel:  [<c0133816>] _mutex_lock+0x19/0x3f
Oct 12 11:46:39 swdev14 kernel:  [<c015197a>] anon_vma_unlink+0x23/0x90
Oct 12 11:46:39 swdev14 kernel:  [<c015c50e>] fput+0xe/0x1f
Oct 12 11:46:39 swdev14 kernel:  [<c014e032>] remove_vm_struct+0x76/0x9f
Oct 12 11:46:39 swdev14 kernel:  [<c015012f>] exit_mmap+0x172/0x1cc
Oct 12 11:46:39 swdev14 kernel:  [<c011c979>] mmput+0x3b/0xb9
Oct 12 11:46:39 swdev14 kernel:  [<c0121807>] do_exit+0x12e/0x3bd
Oct 12 11:46:39 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:39 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:39 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:40 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:40 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:40 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:40 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:40 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:40 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:40 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:40 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2849
Oct 12 11:46:40 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:40 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:40 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:40 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:40 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c014dfdb>] remove_vm_struct+0x1f/0x9f
Oct 12 11:46:40 swdev14 kernel:  [<c015012f>] exit_mmap+0x172/0x1cc
Oct 12 11:46:40 swdev14 kernel:  [<c011c979>] mmput+0x3b/0xb9
Oct 12 11:46:40 swdev14 kernel:  [<c0121807>] do_exit+0x12e/0x3bd
Oct 12 11:46:40 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:40 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:40 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:40 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:40 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:40 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:40 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:40 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:40 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:40 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2849
Oct 12 11:46:40 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:40 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:40 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:40 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:40 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:40 swdev14 kernel:  [<c014dfdb>] remove_vm_struct+0x1f/0x9f
Oct 12 11:46:40 swdev14 kernel:  [<c015012f>] exit_mmap+0x172/0x1cc
Oct 12 11:46:40 swdev14 kernel:  [<c011c979>] mmput+0x3b/0xb9
Oct 12 11:46:40 swdev14 kernel:  [<c0121807>] do_exit+0x12e/0x3bd
Oct 12 11:46:40 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:40 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:40 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:40 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:40 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:40 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:40 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:40 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:40 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:40 swdev14 kernel: scheduling while atomic: sshd/0x00000001/2849
Oct 12 11:46:40 swdev14 kernel: caller is __down+0x8a/0x107
Oct 12 11:46:40 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:40 swdev14 kernel:  [<c029848a>] __down+0x8a/0x107
Oct 12 11:46:40 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:40 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:40 swdev14 kernel:  [<c029a34c>] _spin_unlock_irqrestore+0xb/0x36
Oct 12 11:46:40 swdev14 kernel:  [<c0298485>] __down+0x85/0x107
Oct 12 11:46:40 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 11:46:40 swdev14 kernel:  [<c029848a>] __down+0x8a/0x107
Oct 12 11:46:40 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:40 swdev14 kernel:  [<c0298650>] __down_failed+0x8/0xc
Oct 12 11:46:40 swdev14 kernel:  [<c011c3d8>] .text.lock.sched+0x5/0x15
Oct 12 11:46:40 swdev14 kernel:  [<c01ccbc1>] disassociate_ctty+0x1d/0x16d
Oct 12 11:46:40 swdev14 kernel:  [<c01218f1>] do_exit+0x218/0x3bd
Oct 12 11:46:40 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 11:46:40 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:40 swdev14 kernel:  [<c029a1aa>] _write_lock+0x1b/0x76
Oct 12 11:46:40 swdev14 kernel:  [<c0264422>] tcp_listen_wlock+0x16/0xac
Oct 12 11:46:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 11:46:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 11:46:40 swdev14 kernel:  [<c0133bbc>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 11:46:40 swdev14 kernel:  [<c02537ec>] tcp_listen_start+0x175/0x1d1
Oct 12 11:46:40 swdev14 kernel:  [<c029a3c6>] _spin_unlock_bh+0x1a/0x34
Oct 12 11:46:40 swdev14 kernel:  [<c0275b18>] inet_listen+0x65/0x7a
Oct 12 11:46:40 swdev14 kernel:  [<c022889e>] sys_listen+0x5c/0x74
Oct 12 11:46:40 swdev14 kernel:  [<c0229558>] sys_socketcall+0xb1/0x239
Oct 12 11:46:40 swdev14 kernel:  [<c015aeb9>] sys_close+0x75/0x91
Oct 12 11:46:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:46 swdev14 ntpdate[2873]: step time server 159.82.80.104 offset -0.498445 sec
Oct 12 11:46:46 swdev14 ntpd:  succeeded
Oct 12 11:46:46 swdev14 ntpd[2877]: ntpd [email protected] Thu Mar 11 11:46:39 EST 2004 (1)
Oct 12 11:46:46 swdev14 ntpd: ntpd startup succeeded
Oct 12 11:46:46 swdev14 ntpd[2877]: precision = 1.000 usec
Oct 12 11:46:46 swdev14 ntpd[2877]: kernel time sync status 0040
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "opus" unknown, line ignored
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "hal" unknown, line ignored
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "wizard" unknown, line ignored
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "time1.utc.com" unknown, line ignored
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "time2.utc.com" unknown, line ignored
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "time3.utc.com" unknown, line ignored
Oct 12 11:46:46 swdev14 vsftpd: vsftpd vsftpd succeeded
Oct 12 11:46:46 swdev14 ntpd[2877]: frequency initialized 124.193 PPM from /var/lib/ntp/drift
Oct 12 11:46:46 swdev14 ntpd[2877]: configure: keyword "authenticate" unknown, line ignored
Oct 12 11:46:46 swdev14 gpm[2896]: *** info [startup.c(95)]: 
Oct 12 11:46:46 swdev14 gpm[2896]: Started gpm successfully. Entered daemon mode.
Oct 12 11:46:46 swdev14 gpm[2896]: *** info [mice.c(1766)]: 
Oct 12 11:46:46 swdev14 gpm[2896]: imps2: Auto-detected intellimouse PS/2
Oct 12 11:46:47 swdev14 gpm: gpm startup succeeded
Oct 12 11:46:47 swdev14 crond: crond startup succeeded
Oct 12 11:46:49 swdev14 kernel: lp0: using parport0 (polling).
Oct 12 11:46:49 swdev14 kernel: lp0: console ready
Oct 12 11:46:49 swdev14 kernel: scheduling while atomic: serial/0x04000001/2947
Oct 12 11:46:49 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:49 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:49 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:49 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:49 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:49 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:49 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:49 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:49 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:49 swdev14 kernel:  [<c0133b82>] _rw_mutex_read_lock+0x24/0x39
Oct 12 11:46:49 swdev14 kernel:  [<c011f62c>] profile_handoff_task+0x1a/0x52
Oct 12 11:46:49 swdev14 kernel:  [<c011c508>] __put_task_struct+0x66/0x119
Oct 12 11:46:49 swdev14 kernel:  [<c0298a0b>] schedule+0x35f/0xbe2
Oct 12 11:46:49 swdev14 kernel:  [<c029a392>] _spin_unlock_irq+0x1b/0x35
Oct 12 11:46:49 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:49 swdev14 kernel:  [<c0134ae5>] sub_preempt_count+0x82/0x97
Oct 12 11:46:49 swdev14 kernel:  [<c029a392>] _spin_unlock_irq+0x1b/0x35
Oct 12 11:46:49 swdev14 kernel:  [<c029938c>] wait_for_completion+0x84/0xe3
Oct 12 11:46:49 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:49 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:49 swdev14 kernel:  [<c012ead4>] queue_work+0x94/0xa0
Oct 12 11:46:49 swdev14 kernel:  [<c012e9cd>] call_usermodehelper+0xc7/0xce
Oct 12 11:46:49 swdev14 kernel:  [<c012e89d>] __call_usermodehelper+0x0/0x69
Oct 12 11:46:49 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 11:46:49 swdev14 kernel:  [<c012e6e1>] request_module+0xa1/0xe5
Oct 12 11:46:49 swdev14 kernel:  [<c0134141>] __mcount+0x1d/0x21
Oct 12 11:46:49 swdev14 kernel:  [<c0164d2c>] base_probe+0xe/0x52
Oct 12 11:46:49 swdev14 kernel:  [<c01e672b>] kobj_lookup+0xf8/0x105
Oct 12 11:46:49 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 11:46:49 swdev14 kernel:  [<c0164d50>] base_probe+0x32/0x52
Oct 12 11:46:49 swdev14 kernel:  [<c01e672b>] kobj_lookup+0xf8/0x105
Oct 12 11:46:49 swdev14 kernel:  [<c0164d1e>] base_probe+0x0/0x52
Oct 12 11:46:49 swdev14 kernel:  [<c01649ff>] chrdev_open+0xf2/0x16f
Oct 12 11:46:49 swdev14 kernel:  [<c016490d>] chrdev_open+0x0/0x16f
Oct 12 11:46:49 swdev14 kernel:  [<c015aafa>] dentry_open+0x106/0x180
Oct 12 11:46:49 swdev14 kernel:  [<c015a9f2>] filp_open+0x62/0x64
Oct 12 11:46:49 swdev14 kernel:  [<c013388c>] _mutex_unlock+0xe/0x5e
Oct 12 11:46:49 swdev14 kernel:  [<c01b1bfa>] find_next_zero_bit+0x14/0xa6
Oct 12 11:46:49 swdev14 kernel:  [<c015ac04>] get_unused_fd+0x90/0xe4
Oct 12 11:46:49 swdev14 kernel:  [<c015ad58>] sys_open+0x4b/0x88
Oct 12 11:46:49 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 11:46:50 swdev14 kernel: scheduling while atomic: khelper/0x04000001/14
Oct 12 11:46:50 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 11:46:50 swdev14 kernel:  [<c029925b>] schedule+0xbaf/0xbe2
Oct 12 11:46:50 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:50 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:50 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:50 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:50 swdev14 kernel:  [<c013492a>] touch_preempt_timing+0x46/0x4a
Oct 12 11:46:50 swdev14 kernel:  [<c02996cd>] cond_resched+0x26/0x83
Oct 12 11:46:50 swdev14 kernel:  [<c029970a>] cond_resched+0x63/0x83
Oct 12 11:46:50 swdev14 kernel:  [<c0133b82>] _rw_mutex_read_lock+0x24/0x39
Oct 12 11:46:50 swdev14 kernel:  [<c011f62c>] profile_handoff_task+0x1a/0x52
Oct 12 11:46:50 swdev14 kernel:  [<c011c508>] __put_task_struct+0x66/0x119
Oct 12 11:46:50 swdev14 kernel:  [<c0298a0b>] schedule+0x35f/0xbe2
Oct 12 11:46:50 swdev14 kernel:  [<c029a35d>] _spin_unlock_irqrestore+0x1c/0x36
Oct 12 11:46:50 swdev14 kernel:  [<c013487c>] check_preempt_timing+0x191/0x1f9
Oct 12 11:46:50 swdev14 kernel:  [<c0134ae5>] sub_preempt_count+0x82/0x97
Oct 12 11:46:50 swdev14 kernel:  [<c029a35d>] _spin_unlock_irqrestore+0x1c/0x36
Oct 12 11:46:50 swdev14 kernel:  [<c012edbf>] worker_thread+0x20c/0x22a
Oct 12 11:46:50 swdev14 kernel:  [<c012e89d>] __call_usermodehelper+0x0/0x69
Oct 12 11:46:50 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:50 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 11:46:50 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 11:46:50 swdev14 kernel:  [<c0132f2b>] kthread+0xbc/0xc1
Oct 12 11:46:50 swdev14 kernel:  [<c012ebb3>] worker_thread+0x0/0x22a
Oct 12 11:46:50 swdev14 kernel:  [<c0132e6f>] kthread+0x0/0xc1
Oct 12 11:46:50 swdev14 kernel:  [<c01042c9>] kernel_thread_helper+0x5/0xb
Oct 12 11:48:12 swdev14 syslogd 1.4.1: restart.

i've uploaded the -T8 VP patch:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T8

lots of stabilization of CONFIG_PREEMPT_REALTIME. It's still in experimental status but general stability is improving.

Changes since -T7:

- fixed a nasty category of bugs that were introduced by the use of rwsems for rwlocks. rwsems are not read-recursive, while rwlocks are. Fortunately it was not too hard to identify & fix recursive users of tasklist_lock, in fact each of these also qualifies as a cleanup. The symptom of this bug was a soft-deadlocking of the system.

- fixed profiler locks, i believe this could resolve the bootup crash reported by K.R. Foley.

- fixed VP / XFS namespace collision reported by Mark H. Johnson

- fix one more final detail of the new zombie-task handling code

- fixed 3c59x.c, fasync-handling, ipv6, module-loader runtime warnings reported by K.R. Foley.

- fixed the ali5451 locking

to create a -T8 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T8

Ingo

Booted part way this time but then soft locked. Mouse was still working but not KB. Any idea why the KB is not working with these? I am rebuilding with all of the pwr managment off because I am still getting acpi messages in the boot :-/ I have attached the log from the boot.

kr

Oct 12 15:33:34 swdev14 syslogd 1.4.1: restart.
Oct 12 15:33:34 swdev14 syslog: syslogd startup succeeded
Oct 12 15:33:34 swdev14 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Oct 12 15:33:34 swdev14 syslog: klogd startup succeeded
Oct 12 15:33:34 swdev14 kernel: Linux version 2.6.9-rc4-mm1-VP-T8 ([email protected]) (gcc version 3.3.3 20040412 (Red Hat Linux 3.3.3-7)) #11 SMP Tue Oct 12 15:25:18 CDT 2004
Oct 12 15:33:34 swdev14 kernel: BIOS-provided physical RAM map:
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 0000000000100000 - 000000001ff70000 (usable)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 000000001ff70000 - 000000001ff77000 (ACPI data)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 000000001ff77000 - 000000001ff80000 (ACPI NVS)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 000000001ff80000 - 0000000020000000 (reserved)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
Oct 12 15:33:34 swdev14 kernel:  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
Oct 12 15:33:35 swdev14 kernel:  BIOS-e820: 00000000ff800000 - 00000000ffc00000 (reserved)
Oct 12 15:33:35 swdev14 kernel:  BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
Oct 12 15:33:35 swdev14 kernel: 0MB HIGHMEM available.
Oct 12 15:33:35 swdev14 irqbalance: irqbalance startup succeeded
Oct 12 15:33:35 swdev14 kernel: 511MB LOWMEM available.
Oct 12 15:33:35 swdev14 kernel: found SMP MP-table at 000f6b00
Oct 12 15:33:35 swdev14 kernel: DMI present.
Oct 12 15:33:35 swdev14 portmap: portmap startup succeeded
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Oct 12 15:33:35 swdev14 kernel: Processor #0 15:2 APIC version 20
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled)
Oct 12 15:33:35 swdev14 kernel: Processor #6 15:2 APIC version 20
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Oct 12 15:33:35 swdev14 kernel: Processor #1 15:2 APIC version 20
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
Oct 12 15:33:35 swdev14 kernel: Processor #7 15:2 APIC version 20
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
Oct 12 15:33:35 swdev14 kernel: ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
Oct 12 15:33:35 swdev14 kernel: Using ACPI for processor (LAPIC) configuration information
Oct 12 15:33:35 swdev14 kernel: Intel MultiProcessor Specification v1.4
Oct 12 15:33:35 swdev14 kernel:     Virtual Wire compatibility mode.
Oct 12 15:33:35 swdev14 kernel: OEM ID:   Product ID: PLACER CRB   APIC at: 0xFEE00000
Oct 12 15:33:35 swdev14 kernel: I/O APIC #2 Version 32 at 0xFEC00000.
Oct 12 15:33:35 swdev14 kernel: I/O APIC #3 Version 32 at 0xFEC80000.
Oct 12 15:33:35 swdev14 kernel: I/O APIC #4 Version 32 at 0xFEC80100.
Oct 12 15:33:35 swdev14 rpc.statd[2649]: Version 1.0.6 Starting
Oct 12 15:33:35 swdev14 kernel: Enabling APIC mode:  Flat.  Using 3 I/O APICs
Oct 12 15:33:35 swdev14 kernel: Processors: 4
Oct 12 15:33:35 swdev14 kernel: Built 1 zonelists
Oct 12 15:33:35 swdev14 kernel: Initializing CPU#0
Oct 12 15:33:35 swdev14 nfslock: rpc.statd startup succeeded
Oct 12 15:33:35 swdev14 kernel: Kernel command line: ro root=LABEL=/ noapic
Oct 12 15:33:35 swdev14 kernel: (swapper/0): new 324744 us maximum-latency critical section.
Oct 12 15:33:35 swdev14 kernel:  => started at: <start_kernel+0x48/0x1c6>
Oct 12 15:33:35 swdev14 kernel:  => ended at:   <cond_resched+0x26/0x83>
Oct 12 15:33:35 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:35 swdev14 kernel:  [<c0134834>] check_preempt_timing+0x161/0x1f9
Oct 12 15:33:35 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:35 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:35 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:35 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:35 swdev14 kernel:  [<c0135c0c>] register_cpu_notifier+0x18/0x58
Oct 12 15:33:36 swdev14 kernel:  [<c01301b4>] rcu_cpu_notify+0x36/0x38
Oct 12 15:33:36 swdev14 kernel:  [<c03646c7>] rcu_init+0x70/0x74
Oct 12 15:33:36 swdev14 kernel:  [<c035485c>] start_kernel+0xb9/0x1c6
Oct 12 15:33:36 swdev14 kernel:  [<c03543b0>] unknown_bootoption+0x0/0x15d
Oct 12 15:33:36 swdev14 kernel: PID hash table entries: 2048 (order: 11, 32768 bytes)
Oct 12 15:33:36 swdev14 kernel: (swapper/0): new 354276 us maximum-latency critical section.
Oct 12 15:33:36 swdev14 rpcidmapd: rpc.idmapd startup succeeded
Oct 12 15:33:36 swdev14 kernel:  => started at: <cond_resched+0x26/0x83>
Oct 12 15:33:36 swdev14 kernel:  => ended at:   <cond_resched+0x26/0x83>
Oct 12 15:33:36 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:36 swdev14 kernel:  [<c0134834>] check_preempt_timing+0x161/0x1f9
Oct 12 15:33:36 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:36 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:36 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:36 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:36 swdev14 random: Initializing random number generator:  succeeded
Oct 12 15:33:36 swdev14 kernel:  [<c0135c0c>] register_cpu_notifier+0x18/0x58
Oct 12 15:33:36 swdev14 kernel:  [<c0128433>] timer_cpu_notify+0x25/0x27
Oct 12 15:33:36 swdev14 kernel:  [<c03643cb>] init_timers+0x34/0x54
Oct 12 15:33:36 swdev14 kernel:  [<c035486b>] start_kernel+0xc8/0x1c6
Oct 12 15:33:36 swdev14 kernel:  [<c03543b0>] unknown_bootoption+0x0/0x15d
Oct 12 15:33:36 swdev14 kernel: Detected 2592.193 MHz processor.
Oct 12 15:33:36 swdev14 kernel: Using tsc for high-res timesource
Oct 12 15:33:36 swdev14 kernel: (swapper/0): new 705817 us maximum-latency critical section.
Oct 12 15:33:36 swdev14 rc: Starting pcmcia:  succeeded
Oct 12 15:33:36 swdev14 kernel:  => started at: <cond_resched+0x26/0x83>
Oct 12 15:33:36 swdev14 kernel:  => ended at:   <cond_resched+0x26/0x83>
Oct 12 15:33:36 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:36 swdev14 kernel:  [<c0134834>] check_preempt_timing+0x161/0x1f9
Oct 12 15:33:36 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:36 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:36 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:36 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:36 swdev14 kernel:  [<c01337fe>] _mutex_lock+0x19/0x3f
Oct 12 15:33:36 swdev14 kernel:  [<c0133860>] _mutex_lock_irqsave+0x16/0x1c
Oct 12 15:33:36 swdev14 kernel:  [<c01cbdd4>] tty_register_ldisc+0x37/0xa4
Oct 12 15:33:36 swdev14 kernel:  [<c036be3e>] console_init+0x27/0x4a
Oct 12 15:33:36 swdev14 kernel:  [<c035487a>] start_kernel+0xd7/0x1c6
Oct 12 15:33:36 swdev14 kernel:  [<c03543b0>] unknown_bootoption+0x0/0x15d
Oct 12 15:33:36 swdev14 kernel: Console: colour VGA+ 80x25
Oct 12 15:33:36 swdev14 kernel: Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Oct 12 15:33:36 swdev14 kernel: Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Oct 12 15:33:36 swdev14 kernel: Memory: 513504k/523712k available (1645k kernel code, 9604k reserved, 726k data, 272k init, 0k highmem)
Oct 12 15:33:36 swdev14 netfs: Mounting NFS filesystems:  succeeded
Oct 12 15:33:36 swdev14 kernel: Checking if this processor honours the WP bit even in supervisor mode... Ok.
Oct 12 15:33:36 swdev14 kernel: Security Scaffold v1.0.0 initialized
Oct 12 15:33:36 swdev14 netfs: Mounting other filesystems:  succeeded
Oct 12 15:33:36 swdev14 kernel: Capability LSM initialized
Oct 12 15:33:36 swdev14 kernel: Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Oct 12 15:33:37 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 15:33:37 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 15:33:37 swdev14 kernel: CPU: Physical Processor ID: 0
Oct 12 15:33:37 swdev14 kernel: Intel machine check architecture supported.
Oct 12 15:33:37 swdev14 kernel: Intel machine check reporting enabled on CPU#0.
Oct 12 15:33:37 swdev14 kernel: CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 15:33:37 swdev14 kernel: Enabling fast FPU save and restore... done.
Oct 12 15:33:37 swdev14 kernel: Enabling unmasked SIMD FPU exception support... done.
Oct 12 15:33:37 swdev14 kernel: Checking 'hlt' instruction... OK.
Oct 12 15:33:37 swdev14 kernel: CPU0: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 15:33:37 swdev14 kernel: per-CPU timeslice cutoff: 1462.71 usecs.
Oct 12 15:33:37 swdev14 kernel: task migration cache decay timeout: 2 msecs.
Oct 12 15:33:37 swdev14 kernel: Booting processor 1/1 eip 2000
Oct 12 15:33:37 swdev14 autofs: automount startup succeeded
Oct 12 15:33:37 swdev14 kernel: Initializing CPU#1
Oct 12 15:33:37 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 15:33:37 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 15:33:37 swdev14 smartd[2807]: smartd version 5.21 Copyright (C) 2002-3 Bruce Allen 
Oct 12 15:33:37 swdev14 kernel: CPU: Physical Processor ID: 0
Oct 12 15:33:37 swdev14 smartd[2807]: Home page is http://smartmontools.sourceforge.net/  
Oct 12 15:33:37 swdev14 smartd[2807]: Opened configuration file /etc/smartd.conf 
Oct 12 15:33:37 swdev14 kernel: Intel machine check architecture supported.
Oct 12 15:33:37 swdev14 smartd[2807]: Configuration file /etc/smartd.conf parsed. 
Oct 12 15:33:37 swdev14 smartd[2807]: Device: /dev/hda, opened 
Oct 12 15:33:37 swdev14 kernel: Intel machine check reporting enabled on CPU#1.
Oct 12 15:33:37 swdev14 kernel: CPU1: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 15:33:37 swdev14 kernel: CPU1: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 15:33:37 swdev14 kernel: Booting processor 2/6 eip 2000
Oct 12 15:33:37 swdev14 smartd[2807]: Device: /dev/hda, not found in smartd database. 
Oct 12 15:33:37 swdev14 kernel: Initializing CPU#2
Oct 12 15:33:37 swdev14 smartd[2807]: Device: /dev/hda, is SMART capable. Adding to "monitor" list. 
Oct 12 15:33:37 swdev14 smartd[2807]: Monitoring 1 ATA and 0 SCSI devices 
Oct 12 15:33:37 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 15:33:37 swdev14 smartd[2809]: smartd has fork()ed into background mode. New PID=2809. 
Oct 12 15:33:37 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 15:33:37 swdev14 smartd: smartd startup succeeded
Oct 12 15:33:37 swdev14 kernel: CPU: Physical Processor ID: 3
Oct 12 15:33:38 swdev14 kernel: Intel machine check architecture supported.
Oct 12 15:33:38 swdev14 kernel: Intel machine check reporting enabled on CPU#2.
Oct 12 15:33:38 swdev14 kernel: CPU2: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 15:33:38 swdev14 kernel: CPU2: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 15:33:38 swdev14 kernel: Booting processor 3/7 eip 2000
Oct 12 15:33:38 swdev14 kernel: Initializing CPU#3
Oct 12 15:33:38 swdev14 kernel: CPU: Trace cache: 12K uops, L1 D cache: 8K
Oct 12 15:33:38 swdev14 sshd:  succeeded
Oct 12 15:33:38 swdev14 kernel: CPU: L2 cache: 512K
Oct 12 15:33:38 swdev14 kernel: CPU: Physical Processor ID: 3
Oct 12 15:33:38 swdev14 kernel: Intel machine check architecture supported.
Oct 12 15:33:38 swdev14 kernel: Intel machine check reporting enabled on CPU#3.
Oct 12 15:33:38 swdev14 kernel: CPU3: Intel P4/Xeon Extended MCE MSRs (12) available
Oct 12 15:33:38 swdev14 kernel: CPU3: Intel(R) Xeon(TM) CPU 2.60GHz stepping 07
Oct 12 15:33:38 swdev14 kernel: Total of 4 processors activated (20594.68 BogoMIPS).
Oct 12 15:33:38 swdev14 kernel: checking TSC synchronization across 4 CPUs: 
Oct 12 15:33:38 swdev14 kernel: CPU#0 had 0 usecs TSC skew, fixed it up.
Oct 12 15:33:38 swdev14 kernel: CPU#2 had 0 usecs TSC skeFlag at 0x36 set to 0x1
Oct 12 15:33:38 swdev14 kernel: apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
Oct 12 15:33:38 swdev14 xinetd: xinetd startup succeeded
Oct 12 15:33:38 swdev14 kernel: apm: disabled - APM is not SMP safe.
Oct 12 15:33:38 swdev14 kernel: VFS: Disk quotas dquot_6.5.1
Oct 12 15:33:38 swdev14 kernel: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Oct 12 15:33:38 swdev14 kernel: Initializing Cryptographic API
Oct 12 15:33:38 swdev14 kernel: vesafb: probe of vesafb0 failed with error -6
Oct 12 15:33:38 swdev14 kernel: isapnp: Scanning for PnP cards...
Oct 12 15:33:38 swdev14 kernel: isapnp: No Plug & Play device found
Oct 12 15:33:38 swdev14 kernel: requesting new irq thread for IRQ8...
Oct 12 15:33:38 swdev14 kernel: Real Time Clock Driver v1.12
Oct 12 15:33:38 swdev14 kernel: requesting new irq thread for IRQ12...
Oct 12 15:33:38 swdev14 kernel: Failed to disable AUX port, but continuing anyway... Is this a SiS?
Oct 12 15:33:38 swdev14 kernel: If AUX port is really absent please use the 'i8042.noaux' option.
Oct 12 15:33:38 swdev14 kernel: serio: i8042 AUX port at 0x60,0x64 irq 12
Oct 12 15:33:38 swdev14 kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Oct 12 15:33:38 swdev14 ntpdate[2873]: can't find host wizard 
Oct 12 15:33:38 swdev14 kernel: io scheduler noop registered
Oct 12 15:33:39 swdev14 kernel: io scheduler anticipatory registered
Oct 12 15:33:39 swdev14 kernel: io scheduler deadline registered
Oct 12 15:33:39 swdev14 kernel: io scheduler cfq registered
Oct 12 15:33:39 swdev14 kernel: RAMDISK driver initialized: 16 RAM disks of 8192K size 1024 blocksize
Oct 12 15:33:39 swdev14 kernel: Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
Oct 12 15:33:39 swdev14 kernel: ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Oct 12 15:33:39 swdev14 kernel: ICH4: IDE controller at PCI slot 0000:00:1f.1
Oct 12 15:33:39 swdev14 kernel: PCI: Enabling device 0000:00:1f.1 (0005 -> 0007)
Oct 12 15:33:39 swdev14 kernel: ICH4: chipset revision 2
Oct 12 15:33:39 swdev14 kernel: ICH4: not 100%% native mode: will probe irqs later
Oct 12 15:33:39 swdev14 kernel:     ide0: BM-DMA at 0x1460-0x1467, BIOS settings: hda:DMA, hdb:pio
Oct 12 15:33:39 swdev14 kernel:     ide1: BM-DMA at 0x1468-0x146f, BIOS settings: hdc:DMA, hdd:pio
Oct 12 15:33:39 swdev14 kernel: hda: WDC WD800BB-75CAA0, ATA DISK drive
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ14...
Oct 12 15:33:39 swdev14 kernel: elevator: using anticipatory as default io scheduler
Oct 12 15:33:39 swdev14 kernel: ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Oct 12 15:33:39 swdev14 kernel: hdc: SONY DVD RW DW-U18A, ATAPI CD/DVD-ROM drive
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ15...
Oct 12 15:33:39 swdev14 kernel: ide1 at 0x170-0x177,0x376 on irq 15
Oct 12 15:33:39 swdev14 kernel: hda: max request size: 128KiB
Oct 12 15:33:39 swdev14 kernel: IRQ#14 thread started up.
Oct 12 15:33:39 swdev14 kernel: hda: Host Protected Area detected.
Oct 12 15:33:39 swdev14 kernel: ^Icurrent capacity is 156250000 sectors (80000 MB)
Oct 12 15:33:39 swdev14 kernel: ^Inative  capacity is 156301488 sectors (80026 MB)
Oct 12 15:33:39 swdev14 kernel: hda: Host Protected Area disabled.
Oct 12 15:33:39 swdev14 kernel: hda: 156301488 sectors (80026 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
Oct 12 15:33:39 swdev14 kernel:  hda: hda1 hda2 < hda5 hda6 hda7 >
Oct 12 15:33:39 swdev14 kernel: mice: PS/2 mouse device common for all mice
Oct 12 15:33:39 swdev14 kernel: IRQ#12 thread started up.
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ1...
Oct 12 15:33:39 swdev14 kernel: IRQ#1 thread started up.
Oct 12 15:33:39 swdev14 kernel: input: AT Translated Set 2 keyboard on isa0060/serio0
Oct 12 15:33:39 swdev14 kernel: md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
Oct 12 15:33:39 swdev14 kernel: NET: Registered protocol family 2
Oct 12 15:33:39 swdev14 kernel: IP: routing cache hash table of 2048 buckets, 32Kbytes
Oct 12 15:33:39 swdev14 kernel: TCP: Hash tables configured (established 16384 bind 21845)
Oct 12 15:33:39 swdev14 kernel: NET: Registered protocol family 1
Oct 12 15:33:39 swdev14 kernel: NET: Registered protocol family 17
Oct 12 15:33:39 swdev14 kernel: NET: Registered protocol family 8
Oct 12 15:33:39 swdev14 kernel: NET: Registered protocol family 20
Oct 12 15:33:39 swdev14 kernel: Starting balanced_irq
Oct 12 15:33:39 swdev14 kernel: md: Autodetecting RAID arrays.
Oct 12 15:33:39 swdev14 kernel: md: autorun ...
Oct 12 15:33:39 swdev14 kernel: md: ... autorun DONE.
Oct 12 15:33:39 swdev14 kernel: RAMDISK: Compressed image found at block 0
Oct 12 15:33:39 swdev14 kernel: VFS: Mounted root (ext2 filesystem).
Oct 12 15:33:39 swdev14 kernel: kjournald starting.  Commit interval 5 seconds
Oct 12 15:33:39 swdev14 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 12 15:33:39 swdev14 kernel: Freeing unused kernel memory: 272k freed
Oct 12 15:33:39 swdev14 kernel: IRQ#8 thread started up.
Oct 12 15:33:39 swdev14 kernel: usbcore: registered new driver usbfs
Oct 12 15:33:39 swdev14 kernel: usbcore: registered new driver hub
Oct 12 15:33:39 swdev14 kernel: USB Universal Host Controller Interface driver v2.2
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.0: Intel Corp. 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #1
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ11...
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.0: irq 11, io base 0x1400
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 1
Oct 12 15:33:39 swdev14 kernel: hub 1-0:1.0: USB hub found
Oct 12 15:33:39 swdev14 kernel: hub 1-0:1.0: 2 ports detected
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.1: Intel Corp. 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #2
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ10...
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.1: irq 10, io base 0x1420
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 2
Oct 12 15:33:39 swdev14 kernel: hub 2-0:1.0: USB hub found
Oct 12 15:33:39 swdev14 kernel: hub 2-0:1.0: 2 ports detected
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.2: Intel Corp. 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #3
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ5...
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.2: irq 5, io base 0x1440
Oct 12 15:33:39 swdev14 kernel: uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 3
Oct 12 15:33:39 swdev14 kernel: hub 3-0:1.0: USB hub found
Oct 12 15:33:39 swdev14 kernel: hub 3-0:1.0: 2 ports detected
Oct 12 15:33:39 swdev14 kernel: usb 1-2: new low speed USB device using address 2
Oct 12 15:33:39 swdev14 kernel: IRQ#11 thread started up.
Oct 12 15:33:39 swdev14 kernel: input: USB HID v1.00 Mouse [Microsoft Microsoft IntelliMouse� Optical] on usb-0000:00:1d.0-2
Oct 12 15:33:39 swdev14 kernel: usbcore: registered new driver usbhid
Oct 12 15:33:39 swdev14 kernel: drivers/usb/input/hid-core.c: v2.0:USB HID core driver
Oct 12 15:33:39 swdev14 kernel: EXT3 FS on hda6, internal journal
Oct 12 15:33:39 swdev14 kernel: device-mapper: 4.1.0-ioctl (2003-12-10) initialised: [email protected]
Oct 12 15:33:39 swdev14 kernel: Adding 2048216k swap on /dev/hda5.  Priority:-1 extents:1
Oct 12 15:33:39 swdev14 kernel: kjournald starting.  Commit interval 5 seconds
Oct 12 15:33:39 swdev14 kernel: EXT3 FS on hda1, internal journal
Oct 12 15:33:39 swdev14 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 12 15:33:39 swdev14 kernel: kjournald starting.  Commit interval 5 seconds
Oct 12 15:33:39 swdev14 kernel: EXT3 FS on hda7, internal journal
Oct 12 15:33:39 swdev14 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 12 15:33:39 swdev14 kernel: IA-32 Microcode Update Driver: v1.14 <[email protected]>
Oct 12 15:33:39 swdev14 kernel: microcode: No suitable data for CPU0
Oct 12 15:33:39 swdev14 kernel: microcode: No suitable data for CPU2
Oct 12 15:33:39 swdev14 kernel: microcode: No suitable data for CPU3
Oct 12 15:33:39 swdev14 kernel: microcode: No suitable data for CPU1
Oct 12 15:33:39 swdev14 kernel: parport0: PC-style at 0x378 (0x778) [PCSPP,TRISTATE,EPP]
Oct 12 15:33:39 swdev14 kernel: parport0: irq 7 detected
Oct 12 15:33:39 swdev14 kernel: SCSI subsystem initialized
Oct 12 15:33:39 swdev14 kernel: inserting floppy driver for 2.6.9-rc4-mm1-VP-T8
Oct 12 15:33:39 swdev14 kernel: Floppy drive(s): fd0 is 1.44M
Oct 12 15:33:39 swdev14 kernel: requesting new irq thread for IRQ6...
Oct 12 15:33:39 swdev14 kernel: IRQ#6 thread started up.
Oct 12 15:33:39 swdev14 kernel: FDC 0 is a National Semiconductor PC87306
Oct 12 15:33:39 swdev14 kernel: tg3.c:v3.10 (September 14, 2004)
Oct 12 15:33:39 swdev14 kernel: eth0: Tigon3 [partno(BCM95702A20) rev 1002 PHY(5703)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:50:45:00:9b:33
Oct 12 15:33:39 swdev14 kernel: eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] 
Oct 12 15:33:39 swdev14 kernel: 3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
Oct 12 15:33:39 swdev14 kernel: 0000:05:01.0: 3Com PCI 3c905C Tornado at 0x2000. Vers LK1.1.19
Oct 12 15:33:39 swdev14 kernel: IRQ#15 thread started up.
Oct 12 15:33:39 swdev14 kernel: hdc: ATAPI 40X DVD-ROM DVD-R CD-R/RW drive, 2048kB Cache, UDMA(33)
Oct 12 15:33:39 swdev14 kernel: Uniform CD-ROM driver Revision: 3.20
Oct 12 15:33:39 swdev14 kernel: ip_tables: (C) 2000-2002 Netfilter core team
Oct 12 15:33:39 swdev14 kernel: NET: Registered protocol family 10
Oct 12 15:33:39 swdev14 kernel: IPv6 over IPv4 tunneling driver
Oct 12 15:33:40 swdev14 kernel: ------------[ cut here ]------------
Oct 12 15:33:40 swdev14 kernel: kernel BUG at kernel/mutex.c:185!
Oct 12 15:33:40 swdev14 kernel: invalid operand: 0000 [#1]
Oct 12 15:33:40 swdev14 kernel: PREEMPT SMP 
Oct 12 15:33:40 swdev14 kernel: Modules linked in: ipv6 autofs4 nfs lockd sunrpc iptable_filter ip_tables ide_cd cdrom 3c59x mii tg3 floppy sg scsi_mod parport_pc parport microcode dm_mod evdev usbhid uhci_hcd usbcore ext3 jbd
Oct 12 15:33:40 swdev14 kernel: CPU:    0
Oct 12 15:33:40 swdev14 kernel: EIP:    0060:[<c0133ba4>]    Not tainted VLI
Oct 12 15:33:40 swdev14 kernel: EFLAGS: 00010246   (2.6.9-rc4-mm1-VP-T8) 
Oct 12 15:33:40 swdev14 kernel: EIP is at _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel: eax: 00000000   ebx: 00000000   ecx: c0350f80   edx: c1406b80
Oct 12 15:33:40 swdev14 kernel: esi: da261ac8   edi: da7b1814   ebp: de661f18   esp: de661f18
Oct 12 15:33:40 swdev14 kernel: ds: 007b   es: 007b   ss: 0068   preempt: 00000002
Oct 12 15:33:40 swdev14 kernel: Process sshd (pid: 2835, threadinfo=de660000 task=debaea80)
Oct 12 15:33:40 swdev14 kernel: Stack: de661f44 c0253738 c0350f80 00000016 c029a33a c16caa80 da261c94 da261bfc 
Oct 12 15:33:40 swdev14 kernel:        c16caa80 da261a80 ffffffea de661f5c c0275a8c da261a80 00000005 c16caa80 
Oct 12 15:33:40 swdev14 kernel:        00000003 de661f78 c02287ea c16caa80 00000005 00000000 00000004 08090bf0 
Oct 12 15:33:40 swdev14 kernel: Call Trace:
Oct 12 15:33:40 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:40 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:40 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:40 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:40 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:40 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:40 swdev14 kernel: Code: 75 fc 89 ec 5d c3 55 89 e5 e8 39 fb fd ff 8b 4d 08 8b 01 85 c0 74 14 8d 41 04 ba ff ff 00 00 f0 0f c1 10 0f 85 6e 03 00 00 5d c3 <0f> 0b b9 00 9a c6 2a c0 eb e2 55 89 e5 e8 0a fb fd ff 8b 4d 08 
Oct 12 15:33:40 swdev14 kernel:  <3>scheduling while atomic: sshd/0x04000001/2835
Oct 12 15:33:40 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c02991cf>] schedule+0xbaf/0xbe2
Oct 12 15:33:40 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:40 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c0134864>] check_preempt_timing+0x191/0x1f9
Oct 12 15:33:40 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:40 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c011f5c8>] profile_task_exit+0x18/0x56
Oct 12 15:33:40 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 15:33:40 swdev14 kernel:  [<c01216e0>] do_exit+0x1f/0x3bd
Oct 12 15:33:40 swdev14 kernel:  [<c0299213>] preempt_schedule+0x11/0x7a
Oct 12 15:33:40 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 15:33:40 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 15:33:40 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 15:33:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 15:33:40 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:40 swdev14 kernel:  [<c029a11e>] _write_lock+0x1b/0x76
Oct 12 15:33:40 swdev14 kernel:  [<c026436e>] tcp_listen_wlock+0x16/0xac
Oct 12 15:33:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 15:33:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 15:33:40 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:40 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:40 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:40 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:40 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:40 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:40 swdev14 kernel: note: sshd[2835] exited with preempt_count 1
Oct 12 15:33:40 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2835
Oct 12 15:33:40 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c02991cf>] schedule+0xbaf/0xbe2
Oct 12 15:33:40 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:40 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c0134864>] check_preempt_timing+0x191/0x1f9
Oct 12 15:33:40 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:40 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c014b7e5>] unmap_vmas+0x190/0x29b
Oct 12 15:33:40 swdev14 kernel:  [<c0150037>] exit_mmap+0xb2/0x1cc
Oct 12 15:33:40 swdev14 kernel:  [<c011c96d>] mmput+0x3b/0xb9
Oct 12 15:33:40 swdev14 kernel:  [<c01217ef>] do_exit+0x12e/0x3bd
Oct 12 15:33:40 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 15:33:40 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 15:33:40 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 15:33:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 15:33:40 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:40 swdev14 kernel:  [<c029a11e>] _write_lock+0x1b/0x76
Oct 12 15:33:40 swdev14 kernel:  [<c026436e>] tcp_listen_wlock+0x16/0xac
Oct 12 15:33:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 15:33:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 15:33:40 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:40 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:40 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:40 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:40 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:40 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:40 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:40 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2835
Oct 12 15:33:40 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c02991cf>] schedule+0xbaf/0xbe2
Oct 12 15:33:40 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:40 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c0134864>] check_preempt_timing+0x191/0x1f9
Oct 12 15:33:40 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:40 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:40 swdev14 kernel:  [<c014b7e5>] unmap_vmas+0x190/0x29b
Oct 12 15:33:40 swdev14 kernel:  [<c0150037>] exit_mmap+0xb2/0x1cc
Oct 12 15:33:40 swdev14 kernel:  [<c011c96d>] mmput+0x3b/0xb9
Oct 12 15:33:40 swdev14 kernel:  [<c01217ef>] do_exit+0x12e/0x3bd
Oct 12 15:33:40 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 15:33:40 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 15:33:40 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 15:33:40 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 15:33:40 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:40 swdev14 kernel:  [<c029a11e>] _write_lock+0x1b/0x76
Oct 12 15:33:40 swdev14 kernel:  [<c026436e>] tcp_listen_wlock+0x16/0xac
Oct 12 15:33:40 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 15:33:40 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 15:33:40 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:40 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:41 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:41 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:41 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:41 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:41 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:41 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:41 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2835
Oct 12 15:33:41 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c02991cf>] schedule+0xbaf/0xbe2
Oct 12 15:33:41 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:41 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c0134864>] check_preempt_timing+0x191/0x1f9
Oct 12 15:33:41 swdev14 kernel:  [<c0134912>] touch_preempt_timing+0x46/0x4a
Oct 12 15:33:41 swdev14 kernel:  [<c0299641>] cond_resched+0x26/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c014b7e5>] unmap_vmas+0x190/0x29b
Oct 12 15:33:41 swdev14 kernel:  [<c0150037>] exit_mmap+0xb2/0x1cc
Oct 12 15:33:41 swdev14 kernel:  [<c011c96d>] mmput+0x3b/0xb9
Oct 12 15:33:41 swdev14 kernel:  [<c01217ef>] do_exit+0x12e/0x3bd
Oct 12 15:33:41 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 15:33:41 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 15:33:41 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 15:33:41 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 15:33:41 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:41 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:41 swdev14 kernel:  [<c029a11e>] _write_lock+0x1b/0x76
Oct 12 15:33:41 swdev14 kernel:  [<c026436e>] tcp_listen_wlock+0x16/0xac
Oct 12 15:33:41 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 15:33:41 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 15:33:41 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:41 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:41 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:41 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:41 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:41 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:41 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:41 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:41 swdev14 kernel: scheduling while atomic: sshd/0x04000001/2835
Oct 12 15:33:41 swdev14 kernel: caller is cond_resched+0x63/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c02991cf>] schedule+0xbaf/0xbe2
Oct 12 15:33:41 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c0113d1c>] smp_apic_timer_interrupt+0x79/0xe4
Oct 12 15:33:41 swdev14 kernel:  [<c0134914>] touch_preempt_timing+0x48/0x4a
Oct 12 15:33:41 swdev14 kernel:  [<c0106bb6>] apic_timer_interrupt+0x1a/0x20
Oct 12 15:33:41 swdev14 kernel:  [<c0134914>] touch_preempt_timing+0x48/0x4a
Oct 12 15:33:41 swdev14 kernel:  [<c029967e>] cond_resched+0x63/0x83
Oct 12 15:33:41 swdev14 kernel:  [<c014dfa3>] remove_vm_struct+0x1f/0x9f
Oct 12 15:33:41 swdev14 kernel:  [<c01500f7>] exit_mmap+0x172/0x1cc
Oct 12 15:33:41 swdev14 kernel:  [<c011c96d>] mmput+0x3b/0xb9
Oct 12 15:33:41 swdev14 kernel:  [<c01217ef>] do_exit+0x12e/0x3bd
Oct 12 15:33:41 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 15:33:41 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 15:33:41 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 15:33:41 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 15:33:41 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:41 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:41 swdev14 kernel:  [<c029a11e>] _write_lock+0x1b/0x76
Oct 12 15:33:41 swdev14 kernel:  [<c026436e>] tcp_listen_wlock+0x16/0xac
Oct 12 15:33:41 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 15:33:41 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 15:33:41 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:41 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:41 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:41 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:41 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:41 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:41 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:41 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:41 swdev14 kernel: scheduling while atomic: sshd/0x00000001/2835
Oct 12 15:33:41 swdev14 kernel: caller is __down+0x8a/0x107
Oct 12 15:33:41 swdev14 kernel:  [<c02991cf>] schedule+0xbaf/0xbe2
Oct 12 15:33:41 swdev14 kernel:  [<c02983fe>] __down+0x8a/0x107
Oct 12 15:33:41 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:41 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:41 swdev14 kernel:  [<c029a2c0>] _spin_unlock_irqrestore+0xb/0x36
Oct 12 15:33:41 swdev14 kernel:  [<c02983f9>] __down+0x85/0x107
Oct 12 15:33:41 swdev14 kernel:  [<c01136d4>] mcount+0x14/0x18
Oct 12 15:33:41 swdev14 kernel:  [<c02983fe>] __down+0x8a/0x107
Oct 12 15:33:41 swdev14 kernel:  [<c011a368>] default_wake_function+0x0/0x1c
Oct 12 15:33:41 swdev14 kernel:  [<c02985c4>] __down_failed+0x8/0xc
Oct 12 15:33:41 swdev14 kernel:  [<c011c3cc>] .text.lock.sched+0x5/0x15
Oct 12 15:33:41 swdev14 kernel:  [<c01ccb0d>] disassociate_ctty+0x1d/0x16d
Oct 12 15:33:41 swdev14 kernel:  [<c01218d9>] do_exit+0x218/0x3bd
Oct 12 15:33:41 swdev14 kernel:  [<c0107780>] do_invalid_op+0x0/0x10b
Oct 12 15:33:41 swdev14 kernel:  [<c01073e2>] do_divide_error+0x0/0x131
Oct 12 15:33:41 swdev14 kernel:  [<c0117898>] fixup_exception+0x1c/0x38
Oct 12 15:33:41 swdev14 kernel:  [<c0107889>] do_invalid_op+0x109/0x10b
Oct 12 15:33:41 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:41 swdev14 kernel:  [<c0134129>] __mcount+0x1d/0x21
Oct 12 15:33:41 swdev14 kernel:  [<c029a11e>] _write_lock+0x1b/0x76
Oct 12 15:33:41 swdev14 kernel:  [<c026436e>] tcp_listen_wlock+0x16/0xac
Oct 12 15:33:41 swdev14 kernel:  [<c0106c31>] error_code+0x2d/0x38
Oct 12 15:33:41 swdev14 kernel:  [<c011007b>] generic_set_mtrr+0x68/0x9c
Oct 12 15:33:41 swdev14 kernel:  [<c0133ba4>] _rw_mutex_write_unlock+0x25/0x2f
Oct 12 15:33:41 swdev14 kernel:  [<c0253738>] tcp_listen_start+0x175/0x1d1
Oct 12 15:33:41 swdev14 kernel:  [<c029a33a>] _spin_unlock_bh+0x1a/0x34
Oct 12 15:33:41 swdev14 kernel:  [<c0275a8c>] inet_listen+0x65/0x7a
Oct 12 15:33:41 swdev14 kernel:  [<c02287ea>] sys_listen+0x5c/0x74
Oct 12 15:33:41 swdev14 kernel:  [<c02294a4>] sys_socketcall+0xb1/0x239
Oct 12 15:33:41 swdev14 kernel:  [<c015ae81>] sys_close+0x75/0x91
Oct 12 15:33:41 swdev14 kernel:  [<c0106175>] sysenter_past_esp+0x52/0x71
Oct 12 15:33:47 swdev14 ntpdate[2873]: step time server 159.82.80.54 offset -0.490683 sec
Oct 12 15:33:47 swdev14 ntpd:  succeeded
Oct 12 15:33:47 swdev14 ntpd[2877]: ntpd [email protected] Thu Mar 11 11:46:39 EST 2004 (1)
Oct 12 15:33:47 swdev14 ntpd: ntpd startup succeeded
Oct 12 15:33:47 swdev14 ntpd[2877]: precision = 1.000 usec
Oct 12 15:33:47 swdev14 ntpd[2877]: kernel time sync status 0040
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "opus" unknown, line ignored
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "hal" unknown, line ignored
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "wizard" unknown, line ignored
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "time1.utc.com" unknown, line ignored
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "time2.utc.com" unknown, line ignored
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "time3.utc.com" unknown, line ignored
Oct 12 15:33:47 swdev14 vsftpd: vsftpd vsftpd succeeded
Oct 12 15:33:47 swdev14 ntpd[2877]: frequency initialized 117.670 PPM from /var/lib/ntp/drift
Oct 12 15:33:47 swdev14 ntpd[2877]: configure: keyword "authenticate" unknown, line ignored
Oct 12 15:33:47 swdev14 gpm[2896]: *** info [startup.c(95)]: 
Oct 12 15:33:47 swdev14 gpm[2896]: Started gpm successfully. Entered daemon mode.
Oct 12 15:33:47 swdev14 gpm[2896]: *** info [mice.c(1766)]: 
Oct 12 15:33:47 swdev14 gpm[2896]: imps2: Auto-detected intellimouse PS/2
Oct 12 15:33:48 swdev14 gpm: gpm startup succeeded
Oct 12 15:33:48 swdev14 crond: crond startup succeeded
Oct 12 15:33:50 swdev14 kernel: lp0: using parport0 (polling).
Oct 12 15:33:50 swdev14 kernel: lp0: console ready
Oct 12 15:36:04 swdev14 syslogd 1.4.1: restart.

the soft lock it seems was due to a mutex assert in the TCP code. Now i see that you have the ipv6 module loaded. Would it be possible to disable ipv6 briefly, to see whether there are other issues?

hm, the keyboard should be working - i have not seen any specific keyboard problems with PREEMPT_REALTIME enabled (except in the very early stages of this feature). So it must be something else. Is this a PS2 or an USB keyboard?

this assert says that a write_unlock() was done before the mutex was initialized - which is a no-no. Note that the stock kernel does not do this checking and there's a chance that it has a true bug here which it ignores silently. The more likely scenario is that the kernel mutex code somewhere changed an assumption which broke the code. I'll try to reproduce this.

Ingo

Yes. I don't need and will disable when building T9.

PS2 keyboard. For clarification, I have intermittent problems with the keyboard on this machine with anything to do with preempt. Sometimes it works on boot sometimes it does not. Actually since it is so unpredictable, I can't even say for sure that it has to do with preempt, but it hasn't failed yet on rc4-mm1. I have tried to find a pattern and have yet to do so. Possibly pertinent info:

Iwill MB dual 2.6G Xeon 512 ram

00:00.0 Host bridge: Intel Corp. E7505 Memory Controller Hub (rev 03)
00:00.1 Class ff00: Intel Corp. E7000 Series RAS Controller (rev 03)
00:01.0 PCI bridge: Intel Corp. E7000 Series Processor to AGP Controller (rev 03) 00:02.0 PCI bridge: Intel Corp. E7000 Series Hub Interface B PCI-to-PCI Bridge (rev 03) 00:1d.0 USB Controller: Intel Corp. 82801DB (ICH4) USB UHCI #1 (rev 02)
00:1d.1 USB Controller: Intel Corp. 82801DB (ICH4) USB UHCI #2 (rev 02)
00:1d.2 USB Controller: Intel Corp. 82801DB (ICH4) USB UHCI #3 (rev 02)
00:1d.7 USB Controller: Intel Corp. 82801DB (ICH4) USB2 EHCI Controller (rev 02) 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB/EB/ER Hub interface to PCI Bridge (rev 82) 00:1f.0 ISA bridge: Intel Corp. 82801DB (ICH4) LPC Bridge (rev 02)
00:1f.1 IDE interface: Intel Corp. 82801DB (ICH4) Ultra ATA 100 Storage Controller (rev 02) 00:1f.3 SMBus: Intel Corp. 82801DB/DBM (ICH4) SMBus Controller (rev 02)
01:00.0 VGA compatible controller: nVidia Corporation NV17 [GeForce4 MX 440] (rev a3) 02:1c.0 PIC: Intel Corp. 82870P2 P64H2 I/OxAPIC (rev 03)
02:1d.0 PCI bridge: Intel Corp. 82870P2 P64H2 Hub PCI Bridge (rev 03)
02:1e.0 PIC: Intel Corp. 82870P2 P64H2 I/OxAPIC (rev 03)
02:1f.0 PCI bridge: Intel Corp. 82870P2 P64H2 Hub PCI Bridge (rev 03)
04:04.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5702X Gigabit Ethernet (rev 02) 05:01.0 Ethernet controller: 3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)

i've uploaded the -T9 VP patch:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T9

this is a bugfixes-only release that should fix the highmem-related issues reported by K.R. Foley and Mark H. Johnson: 3 more locks had to be converted to raw.

to create a -T9 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T9

Ingo

Built and running on my laptop (P4/UP). Usual dmesg info goes attached, where some exposed traces need a look.

OTOH, my desktop (P4/SMP/HT) refuses to boot/init all the way up. I guess you know the picture ;) Couldn't get anything out of the mess. Will try later.

Cheers.

rncbc aka Rui Nuno Capela
[email protected]

OK. This one actually boots all the way into X and even shuts down cleanly (no errors either way). Still no keyboard, which is why I had to shut it down. :) Does this indicate that the keyboard is actually being detected or no?

Oct 13 09:29:59 swdev14 kernel: requesting new irq thread for IRQ1...
Oct 13 09:29:59 swdev14 kernel: IRQ#1 thread started up.
Oct 13 09:29:59 swdev14 kernel: input: AT Translated Set 2 keyboard on isa0060/serio0

We're getting there. This is with ipv6 disabled, btw.

kr

There are lots of similar warnings in Reiser4:
CC fs/reiser4/plugin/node/node.o
In file included from include/linux/spinlock.h:16,

                 from include/linux/wait.h:25,
                 from include/linux/fs.h:12,
                 from fs/reiser4/plugin/node/../../reiser4.h:13,
                 from fs/reiser4/plugin/node/../../debug.h:9,
                 from fs/reiser4/plugin/node/node.c:47:

include/asm/mutex.h:75:5: warning: "RWSEM_DEBUG" is not defined

Also, there is an easy to fix compile error - redefinition of inode_lock in fs/reiser4/plugin/object.c

i'm pleased to announce a significantly improved version of the Real-Time Preemption (PREEMPT_REALTIME) feature that i have been working towards in the past couple of weeks.

the patch (against 2.6.9-rc4-mm1) can be downloaded from:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U0

this is i believe the first correct conversion of the Linux kernel to a fully preemptible (fully mutex-based) preemption model, while still keeping all locking properties of Linux.

I also think that this feature can and should be integrated into the upstream kernel sometime in the future. It will need improvements and fixes and lots of testing, but i believe the basic concept is sound and inclusion is manageable and desirable. So comments, testing and feedback is more than welcome!

to recap the conceptual issues that needed solving: the previous patch already converted a fair portion of spinlocks to mutexes, but that in a fully preemptible kernel model the following locking primitives are especially problematic:

- RCU locking
- per-cpu counters and per-cpu variables
- tlb gather/release logic
- seqlocks
- atomic-kmaps
- pte mapping

note that while we tend to think about these as SMP locking primitives, in a fully preemptible model these locking rules are necessary for correctness, even on a single-CPU embedded box.

Unfortunately none of the existing preemption patches solve these
problems: they concentrate on UP but these locking rules must not be ignored on UP either!

(Bill Huey's mutex patch he just released is the one notable exception i know about, which is also a correct implementation i believe, but it doesnt attack these locking rules [yet]. Bill's locking hierarchy is i believe quite similar to the -T9 patch - this i believe is roughly the best one can get when only using spinlocks as a vehicle.)

In the previous (-T9) version of the Real-Time Preemption patch, the above locking primitives kept large portions of kernel code non-preemptable, causing a 'spreadout' of raw spinlocks and non-preemptible sections to various kernel subsystems.

Another, even more problematic effect was that both network drivers and block IO drivers got 'contaminated' by raw spinlocks, triggering lots of per-driver changes and thus limiting testability. It is basically an opt-in model to correctness which is bad from a maintainance and upstream acceptance point of view. The -T9 patch was a prime example of the problems the Linux locking primitives cause in a fully preemptible kernel model.

To solve all these fundamental problems, i improved/fixed/changed all of these locking methods to be preemption-friendly. Most of the time it was necessary to introduce an additional API variant because e.g. rcu_read_lock() is anonymous (it doesnt identify the data protected), so i introduced a variant that takes the write-lock as an argument. In the PREEMPT_REALTIME case we can thus properly serialize on that lock.

For per-cpu variables i introduced a new API variant that creates a spinlock-array for the per-cpu-variable, and users must make sure the cpu field doesnt change. Migration to another CPU can happen within the critical section, but 'statistically' the variable is still per-CPU and update correctness is fully preserved.

TLB shootdown was the source of another nasty type of critical section:
it keeps preemption disabled during much of the pagetable zapping and also relies on a per-CPU variable to keep TLB state. The fix for PREEMPT_REALTIME (on x86) was to implement a simpler but preemptible TLB flushing method. This necessiated two minor API changes to the generic TLB shootdown code - pretty straightforward ones.

Atomic kmaps were another source of non-preemptible sections:
pte_map/pte_unmap both used nontrivial functions within and ran for a long time, creating a lock dependency and a latency problem as well. I wrapped them via normal kmaps, which thus become preemptible. (The main reason for atomic kmaps were non-preemptability constraints - but those are not present in a fully preemptible model.)

seqlocks (used within the VFS and other places) were another problem:
the are now preemptible by default, the same auto-type-detection logic applies to them as to preemptible/raw spinlocks: switching between a preemptible and a non-preemptible type is done by changing the prototype, the locking APIs stay the same.

The improvements to locking allowed the gradual 'pulling out' of all raw-spinlocks from various kernel subsystems. In the -U0 patch i have almost completely finished this 'pullout', and as a result the following kernel subsystems are now completely spinlock-free and 100% mutex-based:

- networking
- IO subsystem
- VFS and lowlevel filesystems
- memory management (except the SLAB allocator lock)
- signal code
- all of drivers/* and sound/*

note: it is important that when PREEMPT_REALTIME is disabled, the old locking rules apply and there is no performance impact whatsoever. So what this implements is in essence a compile-time multi-tier locking architecture enabling 4 types of preemption models:

  - stock                      (casual voluntary kernel preemption)
  - CONFIG_PREEMPT_VOLUNTARY   (lots of cond_resched() points)
  - CONFIG_PREEMPT             (involuntary preemption plus spinlocks)
  - CONFIG_PREEMPT_REALTIME    (everything is a mutex)

these models cover all the variants people are interested in: servers with almost no latency worries, desktops with ~1msec needs and hard-RT applications needing both microsecond-range latencies and provable maximum latencies.

to quantitatively see the effects of these changes to the locking landscape, here's the output of a script that disassembles the kernel image and counts the number of actual spin lock/unlock function calls done versus mutex lock/unlocks:

With PREEMPT_REALTIME disabled, all locks are spinlocks and old semaphores:

        spinlock API calls:      5359  (71.6%)
        | old mutex API calls:   2120  (28.3%)
        | new mutex API calls:      2  (0%)
        all mutex API calls:     2122  (28.3%)
        --------------------------------------
        lock API calls:          7481 (100.0%)

with the -T9 kernel, which had the 'spread out' locking model, a considerable portion of spinlocks were replaced by mutexes, but more than 20% of usage was still spinlocks:

        spinlock API calls:      1835  (23.1%)
        | old mutex API calls:   2142  (26.9%)
        | new mutex API calls:   3961  (49.8%)
        all mutex API calls:     6103  (76.8%)
        --------------------------------------
        lock API calls:          7938 (100.0%)

here are some fresh numbers from Bill Huey's mmLinux kernel:

        spinlock API calls:      2452  (30.3%)
        all mutex API calls:     5614  (69.6%)
        --------------------------------------
        lock API calls:          8066 (100.0%)

(his mutex implementation directly falls back to up()/down() so the new mutexes become part of the old mutexes.)

while i believe that the locking design is fundamentally incomplete in the MontaVista kernel and thus is not directly comparable to PREEMPT_REALTIME nor the mmLinux kernel, here are the stats from it using a similar .config:

        spinlock API calls:      1444  (26.1%)
        | old mutex API calls:   2095  (37.9%)
        | new mutex API calls:   1981  (35.8%)
        all mutex API calls:     4076  (73.8%)
        --------------------------------------
        lock API calls:          5520 (100.0%)

(here it is visible that apparently a significant amount of [i believe necessary] locking is missing from this kernel.)

finally, here are the stats from the new PREEMPT_REALTIME -U0 kernel:

        spinlock API calls:       491  (6.0%)
        | old mutex API calls:   2142  (26.2%)
        | new mutex API calls:   5536  (67.7%)
        all mutex API calls:     7678  (93.9%)
        --------------------------------------
        lock API calls:          8169 (100.0%)

note that almost all of the remaining spinlocks are held for a short amount of time and have a finite maximum duration. They involve hardware access, scheduling and interrupt handling and timers - almost all of that code has O(1) characteristics.

what this means is that we are approaching hard-real-time domains ... using what is in essence the stock Linux kernel!

note that priority inheritance is still not part of this patch, but that effort can now be centralized to the two basic Linux semaphore types, solving the full spectrum of priority inheritance problems!

the code is still x86-only but only for practical reasons - other architectures will be covered in the future as well.

to create a -U0 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U0

Ingo

Tried this, it died.

During boot, I saw a message about scheduling while atomic for postgres, and a stack dump. It then continued, and got all the way into x. I started to type 'ssh-'(add), and then userspace locked(ping still worked from remote.

It *was* able to save data to kern.log, which is helpful.

Also, I noticed that the swapper/0 had a high-latency critical section.

The bug that caused it to crash was in mm/highmem.c.

Attached you will find the kernel log, ksymoops output, and config. I've been running 2.6.9-rc4 since it came out earlier this week. This is my first mm or preempt kernel, however.

(note: I had to lowercase the version number, because make-kpkg doesn't like uppercase)

(swapper/0): new 485720 us maximum-latency critical section.
=> started at: <start_kernel+0x32/0x1b0>
=> ended at: <cond_resched+0x19/0x70>
[check_preempt_timing+200/304] check_preempt_timing+0xc8/0x130 [touch_preempt_timing+43/48] touch_preempt_timing+0x2b/0x30 [cond_resched+25/112] cond_resched+0x19/0x70 [_mutex_lock+20/64] _mutex_lock+0x14/0x40 [get_cmos_time+17/464] get_cmos_time+0x11/0x1d0 [pidhash_init+193/256] pidhash_init+0xc1/0x100 [time_init+8/128] time_init+0x8/0x80 [timer_cpu_notify+23/32] timer_cpu_notify+0x17/0x20 [softirq_init+17/32] softirq_init+0x11/0x20 [start_kernel+179/432] start_kernel+0xb3/0x1b0 [unknown_bootoption+0/352] unknown_bootoption+0x0/0x160
(swapper/0): new 608732 us maximum-latency critical section.
=> started at: <cond_resched+0x19/0x70>
=> ended at: <cond_resched+0x19/0x70>
[check_preempt_timing+200/304] check_preempt_timing+0xc8/0x130 [touch_preempt_timing+43/48] touch_preempt_timing+0x2b/0x30 [cond_resched+25/112] cond_resched+0x19/0x70 [_mutex_lock+32/64] _mutex_lock+0x20/0x40 [get_cmos_time+17/464] get_cmos_time+0x11/0x1d0 [pidhash_init+193/256] pidhash_init+0xc1/0x100 [time_init+8/128] time_init+0x8/0x80 [timer_cpu_notify+23/32] timer_cpu_notify+0x17/0x20 [softirq_init+17/32] softirq_init+0x11/0x20 [start_kernel+179/432] start_kernel+0xb3/0x1b0 [unknown_bootoption+0/352] unknown_bootoption+0x0/0x160
scheduling while atomic: postmaster/0x04000002/905
caller is cond_resched+0x53/0x70
[schedule+1329/1392] schedule+0x531/0x570 [check_preempt_timing+26/304] check_preempt_timing+0x1a/0x130 [check_preempt_timing+26/304] check_preempt_timing+0x1a/0x130 [cond_resched+83/112] cond_resched+0x53/0x70 [_mutex_lock+20/64] _mutex_lock+0x14/0x40 [_mutex_lock_irqsave+5/16] _mutex_lock_irqsave+0x5/0x10 [avc_has_perm_noaudit+42/384] avc_has_perm_noaudit+0x2a/0x180 [kmap_high+280/400] kmap_high+0x118/0x190 [cond_resched+25/112] cond_resched+0x19/0x70 [_mutex_lock+32/64] _mutex_lock+0x20/0x40 [page_address+153/176] page_address+0x99/0xb0 [avc_has_perm+58/120] avc_has_perm+0x3a/0x78 [common_interrupt+24/32] common_interrupt+0x18/0x20 [ipc_has_perm+112/144] ipc_has_perm+0x70/0x90 [ipcperms+114/160] ipcperms+0x72/0xa0 [semctl_main+119/944] semctl_main+0x77/0x3b0 [check_preempt_timing+26/304] check_preempt_timing+0x1a/0x130 [cond_resched+25/112] cond_resched+0x19/0x70 [_rw_mutex_read_lock+25/48] _rw_mutex_read_lock+0x19/0x30 [pg0+945317965/1070072832] dm_table_any_congested+0xd/0x50 [dm_mod] [cond_resched+25/112] cond_resched+0x19/0x70 [check_preempt_timing+26/304] check_preempt_timing+0x1a/0x130 [check_preempt_timing+26/304] check_preempt_timing+0x1a/0x130 [cond_resched+25/112] cond_resched+0x19/0x70 [_mutex_lock+32/64] _mutex_lock+0x20/0x40 [page_address+153/176] page_address+0x99/0xb0 [kmap_high+280/400] kmap_high+0x118/0x190 [cond_resched+25/112] cond_resched+0x19/0x70 [_mutex_lock+32/64] _mutex_lock+0x20/0x40 [page_address+153/176] page_address+0x99/0xb0 [kunmap_high+80/160] kunmap_high+0x50/0xa0 [do_anonymous_page+63/384] do_anonymous_page+0x3f/0x180 [do_no_page+97/800] do_no_page+0x61/0x320 [cond_resched+25/112] cond_resched+0x19/0x70 [handle_mm_fault+203/336] handle_mm_fault+0xcb/0x150 [do_page_fault+436/1416] do_page_fault+0x1b4/0x588 [unmap_vmas+229/368] unmap_vmas+0xe5/0x170 [cond_resched+25/112] cond_resched+0x19/0x70 [cond_resched+25/112] cond_resched+0x19/0x70 [copy_to_user+64/96] copy_to_user+0x40/0x60 [sys_semctl+189/192] sys_semctl+0xbd/0xc0 [sys_ipc+188/640] sys_ipc+0xbc/0x280 [do_page_fault+0/1416] do_page_fault+0x0/0x588

------------[ cut here ]------------
kernel BUG at mm/highmem.c:189!
invalid operand: 0000 [#1]
PREEMPT 
Modules linked in: nfsd exportfs snd_cmipci snd_opl3_lib snd_hwdep snd_mpu401_uart snd_rawmidi snd_seq_device 8250 serial_core nfs lockd sunrpc raid5 xor md dm_mirror dm_mod aic7xxx sg sr_mod cdrom sd_mod scsi_mod snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd snd_page_alloc soundcore 3c59x mii sis900 crc32 siimage
CPU:    0
EIP:    0060:[kunmap_high+30/160]    Not tainted VLI
EFLAGS: 00010246   (2.6.9-rc4-mm1-vp-u0) 

EIP is at kunmap_high+0x1e/0xa0
eax: 00000000 ebx: c2806360 ecx: c036586c edx: 00000001
esi: f6b36000 edi: f68c97e0 ebp: f68c9818 esp: f68cec48
ds: 007b es: 007b ss: 0068 preempt: 00000001
Process FvwmTaskBar (pid: 1249, threadinfo=f68ce000 task=f68cda80)

Stack: 00000001 00350025 c0144e53 f68c9818 00000001 f6888d2c f6b36000 00000000 
       f6888d2c f68c9818 c0144fe1 f68d2b68 00000001 b6800007 c027b329 f68c9818 
       00000000 00000000 b6800007 f68c97e0 00000001 f68c97e0 f68d2b68 b6800007 
Call Trace:
 [do_anonymous_page+83/384] do_anonymous_page+0x53/0x180
 [do_no_page+97/800] do_no_page+0x61/0x320
 [cond_resched+25/112] cond_resched+0x19/0x70
 [handle_mm_fault+203/336] handle_mm_fault+0xcb/0x150
 [do_page_fault+436/1416] do_page_fault+0x1b4/0x588
 [__pagevec_lru_add+165/192] __pagevec_lru_add+0xa5/0xc0
 [read_pages+315/336] read_pages+0x13b/0x150
 [ext3_get_block+0/160] ext3_get_block+0x0/0xa0
 [check_preempt_timing+26/304] check_preempt_timing+0x1a/0x130
 [cond_resched+25/112] cond_resched+0x19/0x70
 [cond_resched+25/112] cond_resched+0x19/0x70
 [do_page_fault+0/1416] do_page_fault+0x0/0x588
 [error_code+45/56] error_code+0x2d/0x38
 [file_read_actor+65/224] file_read_actor+0x41/0xe0
 [do_generic_mapping_read+368/1248] do_generic_mapping_read+0x170/0x4e0
 [__generic_file_aio_read+479/528] __generic_file_aio_read+0x1df/0x210
 [file_read_actor+0/224] file_read_actor+0x0/0xe0
 [generic_file_aio_read+71/112] generic_file_aio_read+0x47/0x70
 [do_sync_read+161/224] do_sync_read+0xa1/0xe0
 [autoremove_wake_function+0/80] autoremove_wake_function+0x0/0x50
 [vfs_read+205/288] vfs_read+0xcd/0x120
 [sys_read+71/128] sys_read+0x47/0x80
 [syscall_call+7/11] syscall_call+0x7/0xb
Code: fe ff ff 8d 76 00 8d bc 27 00 00 00 00 83 ec 08 89 5c 24 04 89 c3 b8 44 42 36 c0 e8 ad 9f fe ff 89 d8 e8 c6 05 00 00 85 c0 75 08 <0f> 0b bd 00 c0 a3 28 c0 05 00 00 80 00 31 db c1 e8 0c 8b 14 85 

ksymoops 2.4.9 on i686 2.6.9-rc4. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.6.9-rc4-mm1-vp-u0/ (specified) -m /boot/System.map-2.6.9-rc4-mm1-vp-u0 (specified)

Error (regular_file): read_ksyms stat /proc/ksyms failed No modules in ksyms, skipping objects
No ksyms, skipping lsmod
kernel BUG at mm/highmem.c:189!

invalid operand: 0000 [#1]
CPU:    0
EIP:    0060:[kunmap_high+30/160]    Not tainted VLI
EFLAGS: 00010246   (2.6.9-rc4-mm1-vp-u0) 
eax: 00000000   ebx: c2806360   ecx: c036586c   edx: 00000001
esi: f6b36000   edi: f68c97e0   ebp: f68c9818   esp: f68cec48
ds: 007b   es: 007b   ss: 0068   preempt: 00000001
Stack: 00000001 00350025 c0144e53 f68c9818 00000001 f6888d2c f6b36000 00000000 
       f6888d2c f68c9818 c0144fe1 f68d2b68 00000001 b6800007 c027b329 f68c9818 
       00000000 00000000 b6800007 f68c97e0 00000001 f68c97e0 f68d2b68 b6800007 
Call Trace:

Warning (Oops_read): Code line not seen, dumping what data is available

Code: fe ff ff 8d 76 00 8d bc 27 00 00 00 00 83 ec 08 89 5c 24 04 89 c3 b8 44 42 36 c0 e8 ad 9f fe ff 89 d8 e8 c6 05 00 00 85 c0 75 08 <0f> 0b bd 00 c0 a3 28 c0 05 00 00 80 00 31 db c1 e8 0c 8b 14 85 

Using defaults from ksymoops -t elf32-i386 -a i386

Code; ffffffd5 <__kernel_rt_sigreturn+1b95/????>
00000000 <_EIP>:
Code; ffffffd5 <__kernel_rt_sigreturn+1b95/????>
0: fe (bad)
Code; ffffffd6 <__kernel_rt_sigreturn+1b96/????>
1: ff (bad)
Code; ffffffd7 <__kernel_rt_sigreturn+1b97/????>
2: ff 8d 76 00 8d bc decl 0xbc8d0076(%ebp)
Code; ffffffdd <__kernel_rt_sigreturn+1b9d/????>
8: 27 daa
Code; ffffffde <__kernel_rt_sigreturn+1b9e/????>
9: 00 00 add %al,(%eax)
Code; ffffffe0 <__kernel_rt_sigreturn+1ba0/????>
b: 00 00 add %al,(%eax)
Code; ffffffe2 <__kernel_rt_sigreturn+1ba2/????>
d: 83 ec 08 sub $0x8,%esp
Code; ffffffe5 <__kernel_rt_sigreturn+1ba5/????>
10: 89 5c 24 04 mov %ebx,0x4(%esp)
Code; ffffffe9 <__kernel_rt_sigreturn+1ba9/????>
14: 89 c3 mov %eax,%ebx
Code; ffffffeb <__kernel_rt_sigreturn+1bab/????>
16: b8 44 42 36 c0 mov $0xc0364244,%eax
Code; fffffff0 <__kernel_rt_sigreturn+1bb0/????>
1b: e8 ad 9f fe ff call fffe9fcd <_EIP+0xfffe9fcd>
Code; fffffff5 <__kernel_rt_sigreturn+1bb5/????>
20: 89 d8 mov %ebx,%eax
Code; fffffff7 <__kernel_rt_sigreturn+1bb7/????>
22: e8 c6 05 00 00 call 5ed <_EIP+0x5ed>
Code; fffffffc <__kernel_rt_sigreturn+1bbc/????>
27: 85 c0 test %eax,%eax
Code; fffffffe <__kernel_rt_sigreturn+1bbe/????>
29: 75 08 jne 33 <_EIP+0x33>
Code; 00000000 Before first symbol
2b: 0f 0b ud2a
Code; 00000002 Before first symbol
2d: bd 00 c0 a3 28 mov $0x28a3c000,%ebp
Code; 00000007 Before first symbol
32: c0 05 00 00 80 00 31 rolb $0x31,0x800000
Code; 0000000e Before first symbol
39: db c1 fcmovnb %st(1),%st
Code; 00000010 Before first symbol
3b: e8 0c 8b 14 85 call 85148b4c <_EIP+0x85148b4c>

1 warning and 1 error issued. Results may not be reliable.

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.9-rc4-mm1-vp-u0
# Wed Oct 13 19:45:17 2004
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_BROKEN_ON_SMP=y

#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_LOG_BUF_SHIFT=14
# CONFIG_HOTPLUG is not set
CONFIG_KOBJECT_UEVENT=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_PREEMPT_TIMING=y
# CONFIG_LATENCY_TRACE is not set
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SHMEM=y
# CONFIG_TINY_SHMEM is not set

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
CONFIG_MK7=y
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_USE_3DNOW=y
# CONFIG_SMP is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_PREEMPT_VOLUNTARY=y
CONFIG_PREEMPT_SOFTIRQS=y
CONFIG_PREEMPT_HARDIRQS=y
CONFIG_PREEMPT_REALTIME=y
CONFIG_X86_UP_APIC=y
CONFIG_X86_UP_IOAPIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_TSC=y
# CONFIG_X86_MCE is not set
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=m

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_HIGHMEM=y
CONFIG_HIGHPTE=y
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_REGPARM=y

#
# Performance-monitoring counters support
#
# CONFIG_PERFCTR is not set
# CONFIG_KEXEC is not set

#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SOFTWARE_SUSPEND is not set

#
# ACPI (Advanced Configuration and Power Interface) Support #
# CONFIG_ACPI is not set CONFIG_ACPI_BLACKLIST_YEAR=0

#
# APM (Advanced Power Management) BIOS Support
#
CONFIG_APM=y
CONFIG_APM_IGNORE_USER_SUSPEND=y
CONFIG_APM_DO_ENABLE=y
CONFIG_APM_CPU_IDLE=y
# CONFIG_APM_DISPLAY_BLANK is not set
CONFIG_APM_RTC_IS_GMT=y
CONFIG_APM_ALLOW_INTS=y
CONFIG_APM_REAL_MODE_POWER_OFF=y

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_MSI is not set
CONFIG_PCI_LEGACY_PROC=y
CONFIG_PCI_NAMES=y
CONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_AOUT is not set
CONFIG_BINFMT_MISC=m

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play support
#
# CONFIG_PNP is not set

#
# Block devices
#
CONFIG_BLK_DEV_FD=y
# CONFIG_BLK_DEV_XD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
# CONFIG_LBD is not set
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives #
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECD=m
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
CONFIG_BLK_DEV_SIIMAGE=m
CONFIG_BLK_DEV_SIS5513=y
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not set
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
CONFIG_SCSI=m
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=m

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs #
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set

#
# SCSI Transport Attributes
#
CONFIG_SCSI_SPI_ATTRS=m
# CONFIG_SCSI_FC_ATTRS is not set

#
# SCSI low-level drivers
#
CONFIG_BLK_DEV_3W_XXXX_RAID=m
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_7000FASST is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
# CONFIG_SCSI_AACRAID is not set
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=32
CONFIG_AIC7XXX_RESET_DELAY_MS=15000
CONFIG_AIC7XXX_DEBUG_ENABLE=y
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_IN2000 is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_SVW is not set
# CONFIG_SCSI_ATA_PIIX is not set
# CONFIG_SCSI_SATA_NV is not set
# CONFIG_SCSI_SATA_PROMISE is not set
# CONFIG_SCSI_SATA_SX4 is not set
CONFIG_SCSI_SATA_SIL=m
# CONFIG_SCSI_SATA_SIS is not set
# CONFIG_SCSI_SATA_ULI is not set
# CONFIG_SCSI_SATA_VIA is not set
# CONFIG_SCSI_SATA_VITESSE is not set
CONFIG_SCSI_BUSLOGIC=m
# CONFIG_SCSI_OMIT_FLASHPOINT is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
CONFIG_SCSI_GDTH=m
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_GENERIC_NCR5380_MMIO is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_NCR53C406A is not set
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
# CONFIG_SCSI_SYM53C8XX_IOMAPPED is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_PSI240I is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLOGIC_1280_1040 is not set
CONFIG_SCSI_QLA2XXX=m
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_SYM53C416 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set

#
# Old CD-ROM drivers (not SCSI, not IDE)
#
# CONFIG_CD_NO_IDESCSI is not set

#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID5=m
CONFIG_MD_RAID6=m
# CONFIG_MD_MULTIPATH is not set
CONFIG_BLK_DEV_DM=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
CONFIG_DM_ZERO=m

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set

#
# Networking support
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
# CONFIG_NETLINK_DEV is not set
CONFIG_UNIX=y
CONFIG_NET_KEY=m
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
# CONFIG_NET_IPGRE is not set
CONFIG_IP_MROUTE=y
# CONFIG_IP_PIMSM_V1 is not set
# CONFIG_IP_PIMSM_V2 is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_TUNNEL=m

#
# IP: Virtual Server Configuration
#
CONFIG_IP_VS=m
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12

#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y

#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m

#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_BRIDGE_NETFILTER=y

#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_CT_ACCT=y
# CONFIG_IP_NF_CT_PROTO_SCTP is not set
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_LIMIT=m
CONFIG_IP_NF_MATCH_IPRANGE=m
CONFIG_IP_NF_MATCH_MAC=m
CONFIG_IP_NF_MATCH_PKTTYPE=m
CONFIG_IP_NF_MATCH_MARK=m
CONFIG_IP_NF_MATCH_MULTIPORT=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH_ESP=m
CONFIG_IP_NF_MATCH_LENGTH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_TCPMSS=m
CONFIG_IP_NF_MATCH_HELPER=m
CONFIG_IP_NF_MATCH_STATE=m
CONFIG_IP_NF_MATCH_CONNTRACK=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_PHYSDEV=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_REALM=m
# CONFIG_IP_NF_MATCH_SCTP is not set
CONFIG_IP_NF_MATCH_COMMENT=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
CONFIG_IP_NF_NAT_LOCAL=y
CONFIG_IP_NF_NAT_SNMP_BASIC=m
CONFIG_IP_NF_NAT_IRC=m
CONFIG_IP_NF_NAT_FTP=m
CONFIG_IP_NF_NAT_TFTP=m
CONFIG_IP_NF_NAT_AMANDA=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_MARK=m
CONFIG_IP_NF_TARGET_CLASSIFY=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_TARGET_NOTRACK=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
# CONFIG_IP_NF_COMPAT_IPCHAINS is not set
# CONFIG_IP_NF_COMPAT_IPFWADM is not set

#
# Bridge: Netfilter Configuration
#
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_XFRM=y
CONFIG_XFRM_USER=m

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
CONFIG_BRIDGE=m
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_HW_FLOWCONTROL is not set

#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
# CONFIG_NET_SCH_CLK_CPU is not set
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
# CONFIG_CLS_U32_PERF is not set
# CONFIG_NET_CLS_IND is not set
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
# CONFIG_NET_CLS_ACT is not set
CONFIG_NET_CLS_POLICE=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_KGDBOE is not set
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_RX is not set
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=m

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
CONFIG_NET_VENDOR_3COM=y
# CONFIG_EL1 is not set
# CONFIG_EL2 is not set
# CONFIG_ELPLUS is not set
# CONFIG_EL16 is not set
# CONFIG_EL3 is not set
# CONFIG_3C515 is not set
CONFIG_VORTEX=m
# CONFIG_TYPHOON is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set

#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_CS89x0 is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
CONFIG_SIS900=m
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_NET_POCKET is not set

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set

#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set

#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPPOE=m
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
CONFIG_NETCONSOLE=y

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=y
# CONFIG_INPUT_TSDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
# CONFIG_SERIO_CT82C710 is not set
CONFIG_SERIO_PCIPS2=y
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=m
CONFIG_SERIAL_8250_NR_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=m
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
CONFIG_HW_RANDOM=y
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=y
# CONFIG_AGP_ALI is not set
CONFIG_AGP_ATI=m
CONFIG_AGP_AMD=y
CONFIG_AGP_AMD64=m
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_INTEL_MCH is not set
CONFIG_AGP_NVIDIA=m
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_AGP_EFFICEON is not set
CONFIG_DRM=y
CONFIG_DRM_TDFX=m
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HANGCHECK_TIMER is not set

#
# I2C support
#
CONFIG_I2C=m
CONFIG_I2C_CHARDEV=m

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_ELEKTOR is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
# CONFIG_I2C_ISA is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
CONFIG_I2C_VOODOO3=m
# CONFIG_I2C_PCA_ISA is not set

#
# Hardware Sensors Chip support
#
# CONFIG_I2C_SENSOR is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set

#
# Other I2C Chip support
#
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia devices
#
CONFIG_VIDEO_DEV=m

#
# Video For Linux
#

#
# Video Adapters
#
# CONFIG_VIDEO_BT848 is not set
# CONFIG_VIDEO_PMS is not set
# CONFIG_VIDEO_CPIA is not set
# CONFIG_VIDEO_SAA5246A is not set
# CONFIG_VIDEO_SAA5249 is not set
# CONFIG_TUNER_3036 is not set
# CONFIG_VIDEO_STRADIS is not set
# CONFIG_VIDEO_ZORAN is not set
# CONFIG_VIDEO_SAA7134 is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_DPC is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_CX88 is not set
# CONFIG_VIDEO_OVCAMCHIP is not set

#
# Radio Adapters
#
# CONFIG_RADIO_CADET is not set
# CONFIG_RADIO_RTRACK is not set
# CONFIG_RADIO_RTRACK2 is not set
# CONFIG_RADIO_AZTECH is not set
# CONFIG_RADIO_GEMTEK is not set
# CONFIG_RADIO_GEMTEK_PCI is not set
# CONFIG_RADIO_MAXIRADIO is not set
# CONFIG_RADIO_MAESTRO is not set
# CONFIG_RADIO_SF16FMI is not set
# CONFIG_RADIO_SF16FMR2 is not set
# CONFIG_RADIO_TERRATEC is not set
# CONFIG_RADIO_TRUST is not set
# CONFIG_RADIO_TYPHOON is not set
# CONFIG_RADIO_ZOLTRIX is not set

#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set

#
# Graphics support
#
# CONFIG_FB is not set
CONFIG_VIDEO_SELECT=y

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y

#
# Sound
#
CONFIG_SOUND=m

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_HWDEP=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
# CONFIG_SND_SEQ_DUMMY is not set
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=m
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set

#
# Generic devices
#
CONFIG_SND_MPU401_UART=m
CONFIG_SND_OPL3_LIB=m
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_VIRMIDI is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set

#
# ISA devices
#
# CONFIG_SND_AD1848 is not set
# CONFIG_SND_CS4231 is not set
# CONFIG_SND_CS4232 is not set
# CONFIG_SND_CS4236 is not set
# CONFIG_SND_ES1688 is not set
# CONFIG_SND_ES18XX is not set
# CONFIG_SND_GUSCLASSIC is not set
# CONFIG_SND_GUSEXTREME is not set
# CONFIG_SND_GUSMAX is not set
# CONFIG_SND_INTERWAVE is not set
# CONFIG_SND_INTERWAVE_STB is not set
# CONFIG_SND_OPTI92X_AD1848 is not set
# CONFIG_SND_OPTI92X_CS4231 is not set
# CONFIG_SND_OPTI93X is not set
# CONFIG_SND_SB8 is not set
# CONFIG_SND_SB16 is not set
# CONFIG_SND_SBAWE is not set
# CONFIG_SND_WAVEFRONT is not set
# CONFIG_SND_CMI8330 is not set
# CONFIG_SND_OPL3SA2 is not set
# CONFIG_SND_SGALAXY is not set
# CONFIG_SND_SSCAPE is not set

#
# PCI devices
#
CONFIG_SND_AC97_CODEC=m
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
CONFIG_SND_BT87X=m
# CONFIG_SND_BT87X_OVERCLOCK is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_ALS4000 is not set
CONFIG_SND_CMIPCI=m
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
CONFIG_SND_INTEL8X0=m
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VX222 is not set

#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set

#
# USB support
#
# CONFIG_USB is not set

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=m

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
# CONFIG_ZISOFS is not set
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not set
CONFIG_DEVPTS_FS_XATTR=y
CONFIG_DEVPTS_FS_SECURITY=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_XATTR is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_RPCSEC_GSS_SPKM3 is not set
CONFIG_SMB_FS=m
CONFIG_SMB_NLS_DEFAULT=y
CONFIG_SMB_NLS_REMOTE="cp437"
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
# CONFIG_NCP_FS is not set
CONFIG_CODA_FS=m
# CONFIG_CODA_FS_OLD_API is not set
# CONFIG_AFS_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=m

#
# Profiling support
#
# CONFIG_PROFILING is not set

#
# Kernel hacking
#
# CONFIG_DEBUG_KERNEL is not set
CONFIG_DEBUG_PREEMPT=y
# CONFIG_FRAME_POINTER is not set
CONFIG_EARLY_PRINTK=y
CONFIG_4KSTACKS=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y

#
# Security options
#
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_CAPABILITIES=y
# CONFIG_SECURITY_SECLVL is not set
CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
# CONFIG_SECURITY_SELINUX_DISABLE is not set
CONFIG_SECURITY_SELINUX_DEVELOP=y
# CONFIG_SECURITY_SELINUX_MLS is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_AES_586=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
# CONFIG_CRYPTO_TEST is not set

#
# Library routines
#
CONFIG_CRC_CCITT=m
CONFIG_CRC32=m
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=m
CONFIG_ZLIB_DEFLATE=m
CONFIG_GENERIC_HARDIRQS=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_PC=y

could you disable HIGHMEM (or at least HIGHPTE) and try again? Some last-minute bug slipped into that code.

Ingo

Well, it's a little better, but it still died. Just took longer.

However, this time, my kern.log got corrupted. I saw 2 scheduling while atomic errors in dmesg(before it locked up), but only one in kern.log, and a bunch of random data(using ext3 data=writeback). Symptoms this time around were laggy keyboard handling, zombie processes(this may have been caused by the scheduling while atomic problem), and ctrl-c not working.

I'll try again tomorrow, and hopefully get more data.

this is normal during bootup - we spend many seconds with preemption disabled - it's fair at that stage. After it has booted up you can reset the maximum-latency searching via:

echo 50 > /proc/sys/kernel/preempt_max_latency

i have that line in my rc.local.

Ingo

I have this built and running on my SMP system now and thus far it looks very promising. Numbers to follow.

kr

Don't try to debug spinlocks as in CONFIG_DEBUG_SPINLOCK and CONFIG_PREEMPT_REALTIME at the same time. The two are currently incompatible.

kr

Cool :)

Say, does it still apply that one should not use unthreaded IRQ handlers for all IRQ's when using PREEMPT_REALTIME (Except maybe for the keyboard)?

flo

yes - and this kernel simply does not allow the un-threading of interrupt handlers anymore, so you cannot accidentally misconfigure it. (Not even the keyboard interrupt is an exception, it would have lock-ripple-effects elsewhere.)

so the preferred (and only) interface to mark interrupts 'high prio' is via process priorities. Starting from the -U1 kernel it will be possible to do this:

chrt -f 60 -p ps -C 'IRQ:1' -o pid=
chrt -f 60 -p ps -C 'IRQ:8' -o pid=

this sets the keyboard and the RT-timer interrupt to FIFO:60.

In -U0 this is not possible because 'ps -C' does not handle kernel threads with a space in their name. So there you'd need some wacky thing like:

   chrt -f 60 -p `ps ax -o pid= -o comm= | grep "IRQ 1$" | cut -dI -f1`
   chrt -f 60 -p `ps ax -o pid= -o comm= | grep "IRQ 8$" | cut -dI -f1`

(someone should fix procps - or does it intentionally break with whitespace command-strings?)

Ingo

Hi,

thanks for the infos.

btw: i use:

chrt -f -p 99 pidof "IRQ 5"

for example (chrt commandline parsing is kinda braindead). It seems to work:

~$ ps -cmL pidof "IRQ 5"

  PID   LWP CLS PRI TTY      STAT   TIME COMMAND
  110     - -     - ?        -      0:00 [IRQ 5]
    -   110 FF  139 -        S<     0:00 -

flo

ah, indeed, pidof is more robust. Ok, i changed the format from IRQ:5 back to "IRQ 5" in my tree to not break scripts.

Ingo

CC'ed jackit-devel mailing list, cause this might be interesting for them, too.

Ah, btw: U0 booted fine here.. Seems to run allright, too (for everything non jackd). Only thing is:

When starting jackd i get a floating point exception. Dunno where that comes from:

~$ jackd -d alsa -p 512
jackd 0.99.0
Copyright 2001-2003 Paul Davis and others.
jackd comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it under certain conditions; see the file COPYING for details

loading driver ..
creating alsa driver ... hw:0|hw:0|512|2|48000|0|0|nomon|swmeter|-|32bit control device hw:0
configuring for 48000Hz, period = 512 frames, buffer = 2 periods Couldn't open hw:0 for 32bit samples trying 24bit instead Couldn't open hw:0 for 24bit samples trying 16bit instead Couldn't open hw:0 for 32bit samples trying 24bit instead Couldn't open hw:0 for 24bit samples trying 16bit instead Floating point exception

running jackd in gdb locks up the gdb process i think [i'm not too experienced in debugging stuff].

here;s partial strace and ltrace logs (only the end). I have no idea if this is a jack bug exposed by your kernrel patches or a bug in your kernel patches exposed by jackd :) But it seems to be a mutex/futex issue...

strace:

....

sched_get_priority_max(SCHED_FIFO)      = 99
sched_get_priority_max(SCHED_FIFO)      = 99
sched_get_priority_min(SCHED_FIFO)      = 1

mmap2(NULL, 8388608, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6d42000
mprotect(0xb6d42000, 4096, PROT_NONE) = 0
clone(child_stack=0xb7541b48, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7541bf8, {entry_number:6, base_addr:0xb7541bb0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7541bf8) = 1546 sched_setscheduler(1546, SCHED_OTHER, { 0 }) = 0
sched_setscheduler(1546, SCHED_FIFO, { 20 }) = 0

futex(0xb7541d94, FUTEX_WAKE, 1)        = 1
ioctl(7, 0x4140, 0x1)                   = 0
ioctl(7, 0x4142, 0x1)                   = 0
sched_get_priority_max(SCHED_FIFO)      = 99
sched_get_priority_min(SCHED_FIFO)      = 1

mmap2(NULL, 8388608, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6542000
mprotect(0xb6542000, 4096, PROT_NONE) = 0
clone(child_stack=0xb6d41b48, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6d41bf8, {entry_number:6, base_addr:0xb6d41bb0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6d41bf8) = 1547 sched_setscheduler(1547, SCHED_OTHER, { 0 }) = 0
sched_setscheduler(1547, SCHED_FIFO, { 10 }) = 0
futex(0xb6d41d94, FUTEX_WAKE, 1) = 1
+++ killed by SIGFPE +++

ltrace:

....
jack_client_alloc_internal(0x8057800, 0x8063a78, 0xbfffe310, 0x80538b1, 0xbffff758) = 0x8064d20 pthread_mutex_unlock(0x8063abc, 0x8063a78, 0xbfffe310, 0x80538b1, 0xbffff758) = 0 creating alsa driver ... hw:0|hw:0|512|2|48000|0|0|nomon|swmeter|-|32bit
control device hw:0
configuring for 48000Hz, period = 512 frames, buffer = 2 periods
Couldn't open hw:0 for 32bit samples trying 24bit instead
Couldn't open hw:0 for 24bit samples trying 16bit instead
Couldn't open hw:0 for 32bit samples trying 24bit instead
Couldn't open hw:0 for 24bit samples trying 16bit instead
free(0x8058658) = <void>
snprintf("/jck-[32 bit float mono audio]", 64, "/jck-[%s]", "32 bit float mono audio") = 30 jack_shmalloc(0xbfffd140, 262144, 0x8063b80, 0xb7fdc21c, 0) = 0 jack_attach_shm(0x8063b80, 262144, 0x8063b80, 0xb7fdc21c, 0) = 0 pthread_mutex_lock(0x8063b00, 0x8063b80, 0x8063b00, 262144, 2048) = 0

malloc(1024)                                     = 0x806e830
malloc(8)                                        = 0x806dd80
malloc(8)                                        = 0x8058658

malloc(8 <unfinished ...>
+++ killed by SIGTRAP +++

This does not happen on my laptop.

Testing also 2.6.9-rc4-mm1-U0, but a slightly custom jack 0.99.5 (cvs) patched with "my" max_delayed_usecs suite.

And jackd it's pumping while I'm writing this lines: jackd -R -d alsa, against bundled crapsound (ali5451).

My laptop is a P4 2.53Ghz, running on Mandrake 10.1c (gcc 3.4.1, glibc 2.3.3 NPTL).

Take care.

rncbc aka Rui Nuno Capela
[email protected]

Hi,

hmm, it could, of course, be again debian's infamous glibc which bites me in the ass (as it did with ignoring pthread attributes (which still isn't fixed afaics)). Which direction should i go with investigating this further? I will build cvs jackd for a start.

flo

P.S.: attached is my .config

Or maybe this:

#
# Security options
#
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
# CONFIG_SECURITY_NETWORK is not set
CONFIG_SECURITY_CAPABILITIES=m

For me, anything that needs to use setcap/getcap fails if I don't compile in security capabilities ie. CONFIG_SECURITY_CAPABILITIES=y. Don't know if this is your problem or not.

kr

nah, this worked very well for all other versions of the VP patches. right now i'm building U0 w/o PREEMPT_REALTIME, to see if i still get the exception.

flo

jackd seems to work fine w/o PREEMPT_REALTIME (otherwise identical config).

flo

jackd seems to work fine w/o PREEMPT_REALTIME (otherwise identical config).

flo

err, sorry, shouldn't have cc'ed to jackit-devel as it's subscribers only. sorry for that.. Ah, there's nothing better than making a complete ass out of oneself :)

flo

In article <[email protected]>,

Why not use pgrep -x 'IRQ 1' . It's part of procps (at least the version debian, even woody, is using), some kind of standard (solaris has it too), and works.

Mike.

"In times of universal deceit, telling the truth becomes
 a revolutionary act." -- George Orwell.

i have released the -U1 PREEMPT_REALTIME patch:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U1

this is a strict bugfixes-only release. With -U1 i cannot reproduce any of the bugs on my testsystems anymore, but take care nevertheless, this is still experimental code.

Changes since -U0:

- bugfix: fixed the highmem related crash reported by Adam Heath and i think this could also fix the crash reported by Mark H Johnson.

- bugfix: fixed a number of networking related soft-lockups, caused by a deadlock scenarios in the ipv4, netfilter and net-xmit locking code. This could fix the lockup reported by Lorenzo Allegrucci.

- bugfix: enable interrupts in the int3 handler - gdb will otherwise trigger a kernel debug message.

- cleanup: reworked the RCU API wrappers, we now have the following variants:

rcu_read_[un]lock_spin(&spinlock)
rcu_read_[un]lock_bh_spin(&spinlock)
rcu_read_[un]lock_sem(&semaphore)

this change was necessary for the network locking fixes.

- debugging helper: SysRq-T will now print the stacktrace of currently running tasks too. (They might be a bit unreliable occasionally but very useful to debug deadlocks.)

- configurability fix: disabled the /proc/kernel/softirq_preemption and hardirq_preemption runtime flags (and the softirq-preempt= and hardirq-preempt= boot flags) if PREEMPT_REALTIME is enabled - in the fully preemptible model these must always be on.

there are no known bugs at this moment, so please re-report any issues you might still encounter.

to create a -U1 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U1

Ingo

This was during NFS startup in init.

using smp_processor_id() in preemptible [00000001] code:
rpc.rquotad/2158
caller is ipt_do_table+0x7b/0x3a0

 [<c011aa15>] smp_processor_id+0x95/0xa0
 [<c038cbfb>] ipt_do_table+0x7b/0x3a0
 [<c038aa8b>] ip_ct_refresh_acct+0xb/0x80
 [<c038f1d4>] ipt_local_hook+0x74/0xc0
 [<c034d73a>] nf_iterate+0x5a/0xa0
 [<c035af00>] dst_output+0x0/0x40
 [<c034da3c>] nf_hook_slow+0x5c/0x100
 [<c035af00>] dst_output+0x0/0x40
 [<c035aaf4>] ip_push_pending_frames+0x414/0x480
 [<c035af00>] dst_output+0x0/0x40
 [<c0377c88>] udp_push_pending_frames+0x148/0x260
 [<c0378178>] udp_sendmsg+0x378/0x6e0
 [<c0134c73>] __mcount+0x13/0x20
 [<c037f7bc>] inet_sendmsg+0x3c/0x60
 [<c03397d8>] sock_sendmsg+0xb8/0xe0
 [<c0134c73>] __mcount+0x13/0x20
 [<c0134c73>] __mcount+0x13/0x20
 [<c0113d30>] mcount+0x14/0x18
 [<c020172a>] __copy_from_user_ll+0xa/0x40
 [<c0133d00>] autoremove_wake_function+0x0/0x60
 [<c03391ef>] move_addr_to_kernel+0x2f/0x60
 [<c033ab36>] sys_sendto+0xd6/0x100
 [<c033d144>] sock_common_setsockopt+0x24/0x40
 [<c0134c73>] __mcount+0x13/0x20
 [<c020172a>] __copy_from_user_ll+0xa/0x40
 [<c0201803>] copy_from_user+0x43/0x80
 [<c0113d30>] mcount+0x14/0x18
 [<c020172a>] __copy_from_user_ll+0xa/0x40
 [<c033b297>] sys_socketcall+0xf7/0x180
 [<c01176a0>] do_page_fault+0x0/0x62a
 [<c0105357>] syscall_call+0x7/0xb

ugh, this is a nasty one - if you look at the TABLE_OFFSET trickery in ipt_do_table it's basically an open-coded per-CPU variable in essence. (probably predating percpu.h so it's fair.) Could you try the quick hack below? (it compiles but is otherwise untested)

The proper solution would be to change the code to use per-cpu variables (and get that patch accepted upstream) and then trivially convert it to get_cpu_var_locked().

Ingo

@@ -287,10 +287,14 @@ ipt_do_table(struct sk_buff **pskb,
 	 * match it. */
 	offset = ntohs(ip->frag_off) & IP_OFFSET;
 
+#ifdef CONFIG_PREEMPT_REALTIME
+	write_lock_bh(&table->lock);
+#else
 	read_lock_bh(&table->lock);
+#endif
 	IP_NF_ASSERT(table->valid_hooks & (1 << hook));
 	table_base = (void *)table->private->entries
-		+ TABLE_OFFSET(table->private, smp_processor_id());
+		+ TABLE_OFFSET(table->private, _smp_processor_id());
 	e = get_entry(table_base, table->private->hook_entry[hook]);
 
 #ifdef CONFIG_NETFILTER_DEBUG

@@ -397,7 +401,11 @@ ipt_do_table(struct sk_buff **pskb,
 #ifdef CONFIG_NETFILTER_DEBUG
 	((struct ipt_entry *)table_base)->comefrom = 0xdead57ac;
 #endif
+#ifdef CONFIG_PREEMPT_REALTIME
+	write_unlock_bh(&table->lock);
+#else
 	read_unlock_bh(&table->lock);
+#endif
 
 #ifdef DEBUG_ALLOW_ALL
 	return NF_ACCEPT;

This fixed it..

Daniel

Looks like that latency trace is mostly printk overhead from the scheduling while atomic errors. In general, if you are still getting lots of printks in your logs due to bugs, the latency traces are not very useful.

Lee

gzipped latecy_trace this time, sorry.

Yes, -U1 seems to have fixed it for me.

BTW, I'm getting a lot of "scheduling while atomic" messages running LTP's runalltests.sh -x 200.
Attached is the kern.log and the latency trace.

I route therefore you are

Got this one running on my SMP workstation FINALLY. The problems that I have been having with the keyboard not working goes away if I compile in CONFIG_SERIO_PCIPS2 instead of making it a module. :-/ Anyway I am getting some "using smp_processor_id messages" all of which seem to be getting generated from ip_tables as in:

Oct 14 15:35:52 swdev14 kernel: using smp_processor_id() in preemptible [00000001] code: thunderbird-bin/3933
Oct 14 15:35:52 swdev14 kernel: caller is ipt_do_table+0x79/0x335 [ip_tables]
Oct 14 15:35:52 swdev14 kernel:  [<c011878e>] smp_processor_id+0xa8/0xb9
Oct 14 15:35:52 swdev14 kernel:  [<e08550a8>] ipt_do_table+0x79/0x335 [ip_tables]
Oct 14 15:35:52 swdev14 kernel:  [<e08550a8>] ipt_do_table+0x79/0x335 [ip_tables]
Oct 14 15:35:52 swdev14 kernel:  [<e09ba0b5>] ipt_local_out_hook+0x76/0x79 [iptable_filter]
Oct 14 15:35:52 swdev14 kernel:  [<c02394ea>] nf_iterate+0x70/0xa1
Oct 14 15:35:52 swdev14 kernel:  [<c024ef4f>] dst_output+0x0/0x2f
Oct 14 15:35:52 swdev14 kernel:  [<c023982d>] nf_hook_slow+0x79/0x126
Oct 14 15:35:52 swdev14 kernel:  [<c024ef4f>] dst_output+0x0/0x2f
Oct 14 15:35:52 swdev14 kernel:  [<c024cfe2>] ip_queue_xmit+0x495/0x59e
Oct 14 15:35:52 swdev14 kernel:  [<c024ef4f>] dst_output+0x0/0x2f
Oct 14 15:35:52 swdev14 kernel:  [<c01325d5>] __mcount+0x1d/0x21
Oct 14 15:35:52 swdev14 kernel:  [<c0298716>] _spin_unlock_irq+0xb/0x35
Oct 14 15:35:52 swdev14 kernel:  [<c01171e7>] finish_task_switch+0x3c/0x85
Oct 14 15:35:52 swdev14 kernel:  [<c0111d1c>] mcount+0x14/0x18
Oct 14 15:35:52 swdev14 kernel:  [<c01325d5>] __mcount+0x1d/0x21
Oct 14 15:35:52 swdev14 kernel:  [<c0263c48>] tcp_v4_send_check+0xe/0xe2
Oct 14 15:35:52 swdev14 kernel:  [<c025d95b>] tcp_transmit_skb+0x435/0x85b
Oct 14 15:35:52 swdev14 kernel:  [<c0111d1c>] mcount+0x14/0x18
Oct 14 15:35:52 swdev14 kernel:  [<c0263c48>] tcp_v4_send_check+0xe/0xe2
Oct 14 15:35:52 swdev14 kernel:  [<c025da07>] tcp_transmit_skb+0x4e1/0x85b
Oct 14 15:35:52 swdev14 kernel:  [<c01af5ee>] memcpy+0x12/0x3c
Oct 14 15:35:52 swdev14 kernel:  [<c025e830>] tcp_write_xmit+0x14c/0x2c6
Oct 14 15:35:52 swdev14 kernel:  [<c0252608>] tcp_sendmsg+0x50d/0x10a7
Oct 14 15:35:52 swdev14 kernel:  [<c0111d1c>] mcount+0x14/0x18
Oct 14 15:35:52 swdev14 kernel:  [<c02525dc>] tcp_sendmsg+0x4e1/0x10a7
Oct 14 15:35:52 swdev14 kernel:  [<c0224f0e>] sock_sendmsg+0xfa/0xfc
Oct 14 15:35:52 swdev14 kernel:  [<c0274665>] inet_sendmsg+0x50/0x5b
Oct 14 15:35:52 swdev14 kernel:  [<c0224f0e>] sock_sendmsg+0xfa/0xfc
Oct 14 15:35:52 swdev14 kernel:  [<c01325d5>] __mcount+0x1d/0x21
Oct 14 15:35:52 swdev14 kernel:  [<c01af10e>] find_next_bit+0x16/0x92
Oct 14 15:35:52 swdev14 kernel:  [<c0117b37>] find_busiest_group+0xd4/0x2e0
Oct 14 15:35:52 swdev14 kernel:  [<c0131d20>] _mutex_unlock+0xe/0x5e
Oct 14 15:35:52 swdev14 kernel:  [<c0111d1c>] mcount+0x14/0x18
Oct 14 15:35:52 swdev14 kernel:  [<c0131d20>] _mutex_unlock+0xe/0x5e
Oct 14 15:35:52 swdev14 kernel:  [<c0131cba>] _mutex_lock+0x29/0x3f
Oct 14 15:35:52 swdev14 kernel:  [<c013186b>] autoremove_wake_function+0x0/0x57
Oct 14 15:35:52 swdev14 kernel:  [<c0224c69>] sockfd_lookup+0x1f/0x74
Oct 14 15:35:52 swdev14 kernel:  [<c0111d1c>] mcount+0x14/0x18
Oct 14 15:35:52 swdev14 kernel:  [<c0226493>] sys_sendto+0xed/0x10c
Oct 14 15:35:52 swdev14 kernel:  [<c0173221>] inode_times_differ+0x9/0x4a
Oct 14 15:35:52 swdev14 kernel:  [<c017332a>] update_atime+0xc8/0xcd
Oct 14 15:35:52 swdev14 kernel:  [<c01325d5>] __mcount+0x1d/0x21
Oct 14 15:35:52 swdev14 kernel:  [<c02264bd>] sys_send+0xb/0x3f
Oct 14 15:35:52 swdev14 kernel:  [<c0226d5e>] sys_socketcall+0x12e/0x239
Oct 14 15:35:52 swdev14 kernel:  [<c0111d1c>] mcount+0x14/0x18
Oct 14 15:35:52 swdev14 kernel:  [<c02264ed>] sys_send+0x3b/0x3f
Oct 14 15:35:52 swdev14 kernel:  [<c0226d5e>] sys_socketcall+0x12e/0x239
Oct 14 15:35:52 swdev14 kernel:  [<c0158cdd>] sys_read+0x78/0x7a
Oct 14 15:35:52 swdev14 kernel:  [<c0106161>] sysenter_past_esp+0x52/0x71

I've reenabled highmem(4g).

Seems to be working fine. Has been running 11 minutes, without problems.

ps: Something that irks me. During bootup, I get the high-latency traces for
swapper/0. These fill up the dmesg ring buffer, so the early messages get dropped. Is there anything that can be done to fix that?

Got my first message.

scheduling while atomic: kswapd0/0x04000001/10
caller is cond_resched+0x53/0x70

 [<c027ad31>] schedule+0x531/0x570
 [<c027b2a3>] cond_resched+0x53/0x70
 [<c012c604>] _mutex_lock+0x14/0x40
 [<c0149521>] page_lock_anon_vma+0x31/0x60
 [<c0149725>] page_referenced_anon+0x15/0x80
 [<c01498ba>] page_referenced+0x7a/0x80
 [<c0141635>] refill_inactive_zone+0x435/0x4b0
 [<c01408a3>] shrink_slab+0x143/0x160
 [<c0141728>] shrink_zone+0x78/0xc0
 [<c0141b7a>] balance_pgdat+0x23a/0x2f0
 [<c0141ced>] kswapd+0xbd/0xf0
 [<c012c140>] autoremove_wake_function+0x0/0x50
 [<c01056d2>] ret_from_fork+0x6/0x14
 [<c012c140>] autoremove_wake_function+0x0/0x50
 [<c0141c30>] kswapd+0x0/0xf0
 [<c0103a2d>] kernel_thread_helper+0x5/0x18

Config is as before, with highmem enabled being the only difference.

i'm working on this one currently, it's a bit tricky.

Ingo

"scheduling while atomic" messages in Reiser4 mentioned at -U0 thread also appear in this version, but less often.

i have released the -U2 PREEMPT_REALTIME patch:

http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U2

this too is a bugfixes-only release, and it is still experimental code.

Changes since -U1:

- bugfix: fix page_lock_anon_vma() crash reported by Adam Heath and Lorenzo Allegrucci.

- bugfix: fix selinux atomic-schedule warning messages, reported by Mark H Johnson.

- bugfix: ip_tables atomic-schedule fix, fixes the messages reported by Daniel Walker and K.R. Foley.

- bugfix: fix warnings/deadlocks in inet_create(), reported by Mark H Johnson.

- bugfix: fixed a crash-in-shmfs-during-heavy-swapout bug

- bugfix: enable preemption while doing mmdrop() in the scheduler - it may schedule.

- debugging feature: when PREEMPT_TIMING is enabled then the code also keeps a trace/stack of preemption enabler EIPs. (if LATENCY_TRACE is enabled as well then the parent EIP is recorded as well.) Whenever a stack trace due to atomicity violations is printed, the preemption stack is printed as well. This makes it much easier to identify the place that did the illegal preemption-disabling.

to create a -U2 tree from scratch the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
+ http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
+ http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
+ http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-U2

Ingo

kernel/latency.c: In function `add_preempt_count':
kernel/latency.c:390: error: structure has no member named preempt_trace_eip' kernel/latency.c:394: error: structure has no member named preempt_trace_parent_eip'

Here's a patch:

@@ -387,11 +387,9 @@
 	if (val <= 10) {
 		unsigned int idx = preempt_count() & PREEMPT_MASK;
 		if (idx < MAX_PREEMPT_TRACE) {
-			current->preempt_trace_eip[idx] = eip;
 #ifdef CONFIG_LATENCY_TRACE
+			current->preempt_trace_eip[idx] = eip;
 			current->preempt_trace_parent_eip[idx] = parent_eip;
-#else
-			current->preempt_trace_parent_eip[idx] = 0;
 #endif
 		}
 	}

--

please try the patch below instead - it will keep the tracer working even with !LATENCY_TRACE.

Ingo

@@ -706,7 +706,7 @@ struct task_struct {
 
 #define MAX_PREEMPT_TRACE 16
 
-#ifdef CONFIG_LATENCY_TRACE
+#ifdef CONFIG_PREEMPT_TIMING
 	unsigned long preempt_trace_eip[MAX_PREEMPT_TRACE];
 	unsigned long preempt_trace_parent_eip[MAX_PREEMPT_TRACE];
 #endif

@@ -33,6 +33,11 @@ static inline void *kmap(struct page *pa
 #define kmap_atomic_pfn(pfn, idx)	page_address(pfn_to_page(pfn))
 #define kmap_atomic_to_page(ptr)	virt_to_page(ptr)
 
+#define kmap_atomic_rt kmap_atomic
+#define kmap_atomic_pfn_rt kmap_atomic_pfn
+#define kunmap_atomic_rt kunmap_atomic
+#define kmap_atomic_to_page_rt(kvaddr) kmap_atomic_to_page(kvaddr)
+
 #endif /* CONFIG_HIGHMEM */
 
 /* when CONFIG_HIGHMEM is not set these will be plain clear/copy_page */

mm/shmem.c: In function `shmem_dir_map':
mm/shmem.c:103: warning: implicit declaration of function kmap_atomic_rt' mm/shmem.c:103: error: KM_USER0' undeclared (first use in this function) mm/shmem.c:103: error: (Each undeclared identifier is reported only once mm/shmem.c:103: error: for each function it appears in.)
mm/shmem.c: In function `shmem_dir_unmap':
mm/shmem.c:108: warning: implicit declaration of function kunmap_atomic_rt' mm/shmem.c:108: error: KM_USER0' undeclared (first use in this function) mm/shmem.c: In function `shmem_swp_map':
mm/shmem.c:113: error: KM_USER1' undeclared (first use in this function) mm/shmem.c: In function shmem_swp_balance_unmap':
mm/shmem.c:125: error: KM_USER1' undeclared (first use in this function) mm/shmem.c: In function shmem_swp_unmap':
mm/shmem.c:130: error: KM_USER1' undeclared (first use in this function) mm/shmem.c: In function shmem_swp_set':
mm/shmem.c:333: warning: implicit declaration of function kmap_atomic_to_page_rt' mm/shmem.c:333: error: invalid type argument of ->'
mm/shmem.c: In function `shmem_file_write':
mm/shmem.c:1362: error: KM_USER0' undeclared (first use in this function) mm/shmem.c:1362: warning: assignment makes pointer from integer without a cast mm/shmem.c: In function shmem_symlink':
mm/shmem.c:1719: error: `KM_USER0' undeclared (first use in this function) mm/shmem.c:1719: warning: assignment makes pointer from integer without a cast make[1]: *** [mm/shmem.o] Error 1
make: *** [mm] Error 2
root@nietzsche> /home/bhuey/linux-2.6.8% 17# make tags

....

I've got kgdb targetted next and I'm trying to figure out how to write a rw/semaphore with priority inheritance.

bill