2.6.0-test3-mm1: scheduling while atomic (ext3?)

Aug 13 06:47:48 middle -- MARK --
Aug 13 06:53:03 middle kernel:  printing eip:
Aug 13 06:53:03 middle kernel: c016c14a
Aug 13 06:53:03 middle kernel: Oops: 0000 [#1]
Aug 13 06:53:03 middle kernel: PREEMPT 
Aug 13 06:53:03 middle kernel: CPU:    0
Aug 13 06:53:03 middle kernel: EIP:    0060:[<c016c14a>]    Not tainted VLI
Aug 13 06:53:03 middle kernel: EFLAGS: 00010286
Aug 13 06:53:03 middle kernel: EIP is at find_inode_fast+0x1a/0x60
Aug 13 06:53:03 middle kernel: eax: f7b7e000   ebx: 000d5ff4   ecx: e68e9a48   edx: 00000000
Aug 13 06:53:03 middle kernel: esi: f7b7e000   edi: c1a50d80   ebp: f2f41e14   esp: f2f41e04
Aug 13 06:53:03 middle kernel: ds: 007b   es: 007b   ss: 0068
Aug 13 06:53:03 middle kernel: Process make (pid: 9500, threadinfo=f2f40000 task=eb0966a0)
Aug 13 06:53:03 middle kernel: Stack: f0c05cc0 f2f40000 f0271cc0 000d5ff4 f2f41e38 c016c7c0 f7b7e000 c1a50d80 
Aug 13 06:53:03 middle kernel:        000d5ff4 c1a50d80 000d5ff4 f0271cc0 f7b7e000 f2f41e58 c018fc92 f7b7e000 
Aug 13 06:53:03 middle kernel:        000d5ff4 c3600234 fffffff4 dddd2a74 dddd2a08 f2f41e7c c0160b10 dddd2a08 
Aug 13 06:53:03 middle kernel: Call Trace:
Aug 13 06:53:03 middle kernel:  [<c016c7c0>] iget_locked+0x50/0xc0
Aug 13 06:53:03 middle kernel:  [<c018fc92>] ext3_lookup+0x62/0xd0
Aug 13 06:53:03 middle kernel:  [<c0160b10>] real_lookup+0xc0/0xf0
Aug 13 06:53:03 middle kernel:  [<c0160d84>] do_lookup+0x84/0x90
Aug 13 06:53:03 middle kernel:  [<c0161211>] link_path_walk+0x481/0x870
Aug 13 06:53:03 middle kernel:  [<c0161abe>] __user_walk+0x3e/0x60
Aug 13 06:53:03 middle kernel:  [<c015cdce>] vfs_stat+0x1e/0x60
Aug 13 06:53:03 middle kernel:  [<c015d43b>] sys_stat64+0x1b/0x40
Aug 13 06:53:03 middle kernel:  [<c03ca78f>] syscall_call+0x7/0xb
Aug 13 06:53:03 middle kernel: 
Aug 13 06:53:03 middle kernel: Code: 75 ca eb c6 8d b6 00 00 00 00 8d bc 27 00 00 00 00 55 89 e5 57 56 53 83 ec 04 8b 5d 10 8b 7d 0c 8b 75 08 8b 0f 85 c9 74 13 8b 11 <0f> 18 02 90 39 59 18 89 c8 74 10 85 d2 89 d1 75 ed 31 c0 83 c4 
Aug 13 06:53:03 middle kernel:  <6>note: make[9500] exited with preempt_count 1
Aug 13 06:53:03 middle kernel: Call Trace:
Aug 13 06:53:03 middle kernel:  [<c011c29e>] schedule+0x58e/0x5a0
Aug 13 06:53:03 middle kernel:  [<c0143f31>] unmap_page_range+0x41/0x70
Aug 13 06:53:03 middle kernel:  [<c014411f>] unmap_vmas+0x1bf/0x220
Aug 13 06:53:03 middle kernel:  [<c0147e39>] exit_mmap+0x79/0x190
Aug 13 06:53:03 middle kernel:  [<c011dd9a>] mmput+0x7a/0xe0
Aug 13 06:53:03 middle kernel:  [<c0121a88>] do_exit+0x118/0x3f0
Aug 13 06:53:03 middle kernel:  [<c011a1e0>] do_page_fault+0x0/0x46b
Aug 13 06:53:03 middle kernel:  [<c010b6c9>] die+0xf9/0x100
Aug 13 06:53:03 middle kernel:  [<c011a30d>] do_page_fault+0x12d/0x46b
Aug 13 06:53:03 middle kernel:  [<c0155b38>] __getblk+0x28/0x50
Aug 13 06:53:03 middle kernel:  [<c018b8e2>] ext3_getblk+0x92/0x290
Aug 13 06:53:03 middle kernel:  [<c01542b1>] wake_up_buffer+0x11/0x30
Aug 13 06:53:03 middle kernel:  [<c01542fe>] unlock_buffer+0x2e/0x50
Aug 13 06:53:03 middle kernel:  [<c0157afd>] ll_rw_block+0x4d/0x80
Aug 13 06:53:03 middle kernel:  [<c018f957>] ext3_find_entry+0x307/0x3c0
Aug 13 06:53:03 middle kernel:  [<c011a1e0>] do_page_fault+0x0/0x46b
Aug 13 06:53:03 middle kernel:  [<c03cb19b>] error_code+0x2f/0x38
Aug 13 06:53:03 middle kernel:  [<c016c14a>] find_inode_fast+0x1a/0x60
Aug 13 06:53:03 middle kernel:  [<c016c7c0>] iget_locked+0x50/0xc0
Aug 13 06:53:03 middle kernel:  [<c018fc92>] ext3_lookup+0x62/0xd0
Aug 13 06:53:03 middle kernel:  [<c0160b10>] real_lookup+0xc0/0xf0
Aug 13 06:53:03 middle kernel:  [<c0160d84>] do_lookup+0x84/0x90
Aug 13 06:53:03 middle kernel:  [<c0161211>] link_path_walk+0x481/0x870
Aug 13 06:53:03 middle kernel:  [<c0161abe>] __user_walk+0x3e/0x60
Aug 13 06:53:03 middle kernel:  [<c015cdce>] vfs_stat+0x1e/0x60
Aug 13 06:53:03 middle kernel:  [<c015d43b>] sys_stat64+0x1b/0x40
Aug 13 06:53:03 middle kernel:  [<c03ca78f>] syscall_call+0x7/0xb
Aug 13 06:53:03 middle kernel: 

Kind regards,
Jurriaan

All lies all lies all schemes all schemes
Every winner means a loser in the western dream.
	News Model Army - Western Dream
Debian (Unstable) GNU/Linux 2.6.0-test3-mm1 4276 bogomips load av: 0.00 0.26 0.26

You oopsed here:

Code; c016c144 No symbols available
25: 85 c9 test %ecx,%ecx
Code; c016c146 No symbols available
27: 74 13 je 3c <_EIP+0x3c>
Code; c016c148 No symbols available
29: 8b 11 mov (%ecx),%edx

This decode from eip onwards should be reliable

Code; c016c14a No symbols available
00000000 <_EIP>:
Code; c016c14a No symbols available <=====
0: 0f 18 02 prefetchnta (%edx) <=====
Code; c016c14d No symbols available
3: 90 nop
Code; c016c14e No symbols available
4: 39 59 18 cmp %ebx,0x18(%ecx)
Code; c016c151 No symbols available

And indeed, your %edx is zero.

But if a prefetch of zero oopses then we should be oopsing in there all the time.

hlist_for_each() is completely assuming that prefetch(0) is safe, and you undoubtedly oopsed doing it.

Colour me confused, and let me Cc lots of x86 guys ;)

Exactly what sort of CPU are you using?

AMD Athlon XP2400+ on a VIA KT400 chipset, single CPU-system.

Kind regards,
Jurriaan

OK, thanks. The word is that Athlons will, very occasionally, take a fault when prefetching from an unmapped address.

 include/linux/list.h |   12 ++++++++----
 1 files changed, 8 insertions(+), 4 deletions(-)

diff -puN include/linux/list.h~hlist_for_each-fix include/linux/list.h
@@ -504,11 +504,15 @@ static __inline__ void hlist_add_after(s
 
 #define hlist_entry(ptr, type, member) container_of(ptr,type,member)
 
-/* Cannot easily do prefetch unfortunately */
-#define hlist_for_each(pos, head) \
-	for (pos = (head)->first; pos && ({ prefetch(pos->next); 1; }); \
-	     pos = pos->next) 
+#define hlist_for_each(pos, head)					\
+	for (	pos = (head)->first;					\
+		likely(pos) && ({					\
+				if (likely(pos->next))			\
+					prefetch(pos->next);		\
+				1; });					\
+		pos = pos->next)
 
+/* Cannot easily do prefetch unfortunately */
 #define hlist_for_each_safe(pos, n, head) \
 	for (pos = (head)->first; n = pos ? pos->next : 0, pos; \
 	     pos = n)

_

Page zero in the kernel is mapped in 4Mb paging mode (which is what the Athlon uses). Also your likely(pos) pretty much wiped out the point of prefetching and punishes other processors because it is in the wrong place. For that matter we could add a LIST_NULL that pointed somewhere safe and wasn't NULL per se in 2.7.

Put the likely(pos) in the asm/prefetch for Athlon until someone can figure out what is going on with some specific Athlons, 2.6 and certain kernels (notably 4G/4G).

Long term we really do need to start supporting a zero page mapped at 0->64K when not debugging the kernel, then you can let the compiler do NULL dereferences which is a _huge_ win because you can move stuff around a lot of natural C conditionals to get better unrolling and instruction scheduling.

The alternative is to start doing multipointer lists which is messier and uses more memory (ie each node has next, prev, "several nodes on")

Alan

<riffles through random config options>

Like this?

What happens if someone runs a K6 kernel on a K7?

Or various other CPU types? What is the matrix here?

I don't like the way this is headed...

@@ -568,6 +568,10 @@ static inline void rep_nop(void)
 #define ARCH_HAS_PREFETCH
 extern inline void prefetch(const void *x)
 {
+#ifdef CONFIG_MK7
+	if (unlikely(x == NULL))
+		return;		/* athlons like to oops in prefetch(0) */
+#endif
 	alternative_input(ASM_NOP4,
 			  "prefetchnta (%1)",
 			  X86_FEATURE_XMM,

_

Beats me, but then the prefetch code in 2.6 seems broken from 5 seconds of inspection anyway. We are testing the XMM feature and using prefetchnta for Athlon, thats wrong for lots of athlon processors that dont have XMM but do have prefetch/prefetchw, (which btw also seem to work properly on all these processors
while prefetchnta seems to do funky things)

Perhaps someone should fix prefetch() before they worry about the rest of the mess ?

For Athlon we should be testing 3Dnow, and using prefetch/prefetchw for Intel cases we want to go for prefetchnta if XMM is set (PIII, PIV)

Alan

Hi!

You break things :-(.

Also prefetch with test for null does probably more harm than good. What about simply assuming K7 can not do prefetch?

Pavel

When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

If it's an Opteron then it's a known errata (#91 iirc). Update your BIOS in this case.

The x86-64 kernel port also has a workaround for this (adding exception handling to the prefetches)

-Andi