Skip to contentSkip to conversation

2.4.20-pre10aa1 oops report (was Re: Linux-2.4.20-pre8-aa2 oops report. [solved])

DISCUSSION harisri 3 participants 4 messages Oct 15, 2002 – Oct 16, 2002

lore
Share & tools
lore

harisri

author
#0 lore raw

Hello Andrea,

Quoted 6 lines

this smells like a problem with one of your modules. Please make 100% sure you use exactly the same .config for both 2.4.20pre10 and 2.4.20pre10aa1 and please try to find which is the module that is crashing the kernel after it's being loaded. Expect always different kind of crashes and oopses. You can also try to turn on the slab debugging option in the kernel hacking menu.

Yes I am using the same .config file from 2.4.20-pre10 on 2.4.20-pre10aa1 (of course I run make oldconfig, and accept the default setting that shows up on 2.4.20-pre10aa1)

I think you are right, it has something to do with the kernel modules.

Quoted 4 lines
Quoted 1 line

Code; c01e55e2 <fast_clear_page+12/50>

you also may want to configure the kernel as i686 instead of K7 so fast_clear_page won't be used to see if it makes any difference.

Ok. That didn't really help. Kernel compiled for i386 even crashes, but the k7 optimised kernel crashes at the Athlon speed :-)

Quoted 10 lines

the place where the oops happens is most certainly not the problem, either something is wrong with fast_clear_page for whatever hardware reason, or more likely the moduled by modprobe is corrupting the freelist and alloc_pages returned garbage.

btw, how much memory do you have? If you've more than 800M it could be a
broken driver using pte_offset by hand, try to reproduce with mem=800m in such case. To fix this you should find which is the module that is destabilizing the kernel.

My computer has 512 MB RAM. No highmem.

I am able to trigger the issue (after 3 attempts [1]) with, CONFIG_AGP m
CONFIG_AGP_AMD y
CONFIG_DRM y
CONFIG_DRM_RADEON m

While I couldn't trigger the issue (after 5 attempts [1]) without them. Hence I suspect it may be something to do with them. But it takes a lot of time to test these all, I think I will have good answers in couple of days time considering the amount of time it takes to perform the tests.

[1]
1. Login to XFree86/Gnome
2. Start Mozilla, Evolution, OpenOffice Writer/Calc/Impress, Konqueror, KMail. And exit them all.
3. mke2fs -j /dev/hdc9; mount /dev/hdc9 /test;cd /test;dd if=/dev/zero of=zero bs=1024 count=2097152;cd /
4. Redo the step 2
5. Log out and log in and redo step 2
6. Unmount /test

Repeat the above test cycle few times (on 3rd attempt or so) the system oops (when I had AGP/AMD/DRM/Radeon stuff).

Thanks for your help.

Hari
[email protected]

#1 lore raw

Hello Andrea,

Quoted 6 lines
Quoted 6 lines

this smells like a problem with one of your modules. Please make 100% sure you use exactly the same .config for both 2.4.20pre10 and 2.4.20pre10aa1 and please try to find which is the module that is crashing the kernel after it's being loaded. Expect always different kind of crashes and oopses. You can also try to turn on the slab debugging option in the kernel hacking menu.

That precisely is the reason. The bad news is that system crashes when agpgart and radeon are compiled as modules, and the good news is that I am unable to crash it when they are not.

Mainline (2.4.20-pre10) is stable when agpgart and radeon are compiled as modules.

The problem is much easier to reproduce than I thought, just log in and log out of XFree86/Gnome few times (3 or more times in my case) is more than adequate to crash it.

Here is the .config which is stable in -aa1:
CONFIG_AGP=y
CONFIG_AGP_AMD=y
CONFIG_DRM=y
CONFIG_DRM_NEW=y
CONFIG_DRM_RADEON=y

Here is the .config which destabilises the -aa1 kernel:
CONFIG_AGP=m
CONFIG_AGP_AMD=y
CONFIG_DRM=y
CONFIG_DRM_NEW=y
CONFIG_DRM_RADEON=m

Unfortunately system just reboots without leaving oops information in the system logs. If you want I can try few older versions of -aa to find from when it started happening.

Thanks for your help.

Signature · 2 lines
Hari
[email protected]
#2 lore raw

Hello,

Quoted 3 lines

That precisely is the reason. The bad news is that system crashes when agpgart and radeon are compiled as modules, and the good news is that I am unable to crash it when they are not.

My goodness, I have spoken too early I guess. The -aa kernel crashes whether agpgart and radeon are modules or not.

Quoted 2 lines

Mainline (2.4.20-pre10) is stable when agpgart and radeon are compiled as modules.

That holds true still.

Quoted 3 lines

The problem is much easier to reproduce than I thought, just log in and log out of XFree86/Gnome few times (3 or more times in my case) is more than adequate to crash it.

That is still the case.

Signature · 2 lines
Hari
[email protected]
#3 lore raw

Re: 2.4.20-pre10aa1 oops report (was Re: Linux-2.4.20-pre8-aa2 oops report. [solved]) [solved2? ac97]

Quoted 8 lines · Srihari Vijayaraghavan

Hello,

Quoted 3 lines

That precisely is the reason. The bad news is that system crashes when agpgart and radeon are compiled as modules, and the good news is that I am unable to crash it when they are not.

My goodness, I have spoken too early I guess. The -aa kernel crashes whether agpgart and radeon are modules or not.

I'm running this kernel for 5 days now very often under heavy load (also with thousand of tasks with volanomark in background and aio and flood of writes from /dev/zero), and there's no sign of instability (besides a rare tcp race that is been reported for 2.4.19 on l-k too, not fatal, it only deadlocks the tcp connection and you've to kill the task because readmsg will never return until it gets a signal, I tried to debug it but with no luck yet, but that is also most certainly a mainline issue too and it triggers only during heavy load).

You probably did something incidentally (not part of your regression test loop) that corrupted memory. The regression test is a workload that will show you if the corruption has happened in the past or not, but the regression test loop is not the thing that is generating the corruption. The regression test loop is what gets _harmed_ by the corruption, it's not the culprit.

My crystall ball is telling me that you could reproduce it easily on my tree because when you feel finally stable and that you can restart doing your usual work without worrying about oopses, you enjoy yourself playing some music to relax. And you instead don't play music while you try to reproduce the problem because you're busy looking at stressing the kernel and in turn you can't reproduce the bug. Is she right? ;)

Please try with CONFIG_SOUND=n and make sure to run:

	rm -r /lib/modules/2.4.20-pre10aa1

before "make modules_install" to avoid running stale modules (also enable modversions just in case).

I see a pile of oopses all showing ac97 loaded into the kernel, some also for 2.4.19, but they may be unrelated problems of course. A number of reports showing definitive random mm corruption like yours on top of 2.4.20-pre vanilla (not -aa) are most certainly been affected too by the ac97 bug (I'm CC'ing the other affected testers, they can try as well the same as you). I never tried ac97 (I've a couple of boxes that could handle it, but I never attempted to play sound on those yet and the chipset may be different so it may not trigger for me after all even if I could load that module).

Hint: in the past I found easier to reproduce various module bugs with a loop like this:

while :; do insmod ac97_codec.o; rmmod ac97_codec.o; done

you can try the above and see if it trigger in seconds.

Quoted 1 line

From the l-k db grepping it seems the bug is been introduced in 2.4.19.

So I would suggest you to try to reproduce after a:

	rm -r 2.4.20pre10aa1/drivers/sound
	cp -a 2.4.18/drivers/sound 2.4.20pre10aa1/drivers
	cd 2.4.20pre10aa1; make oldconfig ...

(of course you can replace 2.4.20pre10aa1 with 2.4.20pre11 vanilla or 2.4.20pre10ac2)

and see if the instability goes away?

Marcelo also included some further ac97 patch in pre11, maybe 2.4.20pre11aa1 will fix it, you may want to give it a try too when I release it (OTOH, I'm fixing what seems to be a design bug in the o1 scheduler that is apparently generatating an huge cpu waste, so I don't guarantee that the very first release with these changes will be as solid as 2.4.20pre10aa1 ;)

Thanks for all the reports,

Andrea