kernel oops in cfi_cmdset_0002.c:do_write_one()...

Hi there,

on my embedded PPC system with linux-2.425 I have two flash devices. The first one (a new MBM29LV652UE) is directly connected to the MPC8xx and used to store bootloader, kernel and rootfs (jffs2). It's detected via CFI by do_map_probe("cfi_probe"...).

The second one (MBM29LV800BA) is connected via some glue logic via an FPGA. Accesses to this one are very slow. For some reason it does not react on the CFI sequence and thus not detected. So I use do_map_probe("jedec_probe"...) instead.

...
Amd/Fujitsu Extended Query Table v1.1 at 0x0040
number of CFI chips: 1
cfi_cmdset_0002: Disabling fast programming due to code brokenness.
init_dab4k_mtd: bank 1, name: DAB4K 0, size:8388608 bytes Search for id:(04 225b) interleave(1) type(2)
Found: Fujitsu MBM29LV800BA
DAB4K 1: Found 1 x16 devices at 0x0 in 16-bit mode number of JEDEC chips: 1
init_dab4k_mtd: bank 2, name: DAB4K 1, size:1048576 bytes DAB4K flash0: Using Static image partition definition Creating 5 MTD partitions on "DAB4K 0":
0x00120000-0x00800000 : "JFFS2"
0x00000000-0x00030000 : "U-Boot"
0x00030000-0x00040000 : "Environment" 0x00040000-0x00050000 : "FPGA"
0x00050000-0x00120000 : "Kernel"
DAB4K flash1: Using Static image partition definition Creating 2 MTD partitions on "DAB4K 1":
0x00000000-0x00004000 : "DADSP Boot" 0x00030000-0x00080000 : "Selfstart"

Everything seems fine. "eraseall /dev/mtd6" works. But when I try to write data to the second flash

	dd if=/dev/urandom of=/dev/mtd6 bs=64k count=1

the kernel crashes with

Warning: DQ5 raised while program operation was in progress, however operation completed OK
Warning: DQ5 raised while program operation was in progress, however operation completed OK
Warning: DQ5 raised while program operation was in progress, however operation completed OK Waiting for write to complete timed out in do_write_oneword.

Oops: kernel access of bad area, sig: 11
NIP: 0FDF96B8 XER: 00000000 LR: 1006DA30 SP: 7FFFFC90 REGS: c0b31f50 TRAP: 0400
   Not tainted

MSR: 4000d032 EE: 1 PR: 1 FP: 0 ME: 1 IR/DR: 11
TASK = c0b30000[97] 'dd' Last syscall: 4
last math 00000000 last altivec 00000000
GPR00: 1006DAAC 7FFFFC90 00000000 0FEC09E0 7FFFFD40 00010000 1018B000 1017A1B8
GPR08: 0FE0D744 100A0000 00000000 7FFFFC50 42000002 100A6B24 00000000 00000001
GPR16: 00000000 00000000 00000000 00000000 00000001 00000000 00000001 00000000
GPR24: 7FFFFF88 7FFFFF78 00010000 1007C170 7FFFFD40 7FFFFD40 00000005 1007C170 Call backtrace:
0FECCB20 1006DAAC 1006C504 10011A18 10004770 10004360 0FDAFD14 00000000

Kernel panic: Aiee, killing interrupt handler!

In interrupt handler - not syncing
<0>Rebooting in 10 seconds..

When I increase the timeout in drivers/mtd/chips/cfi_cmdset_0002.c:do_write_oneword()

	/* Polling toggle bits instead of reading back many times
	   This ensures that write operation is really completed,
	   or tells us why it failed. */
	dq6 = CMD(1<<6);
	dq5 = CMD(1<<5);

- timeo = jiffies + (HZ/1000); /* setting timeout to 1ms for now */
+ timeo = jiffies + (10 * HZ/1000); /* setting timeout to 10ms for now */

then I still get the warnings about "DQ5 raised" but not about the timeout. Thus the system does not crash. So I suspect that something in the lines:

		} else {
			printk(KERN_WARNING "Waiting for write to complete timed out in do_write_oneword.");

			chip->state = FL_READY;
			wake_up(&chip->wq);
			cfi_spin_unlock(chip->mutex);
			DISABLE_VPP(map);
			ret = -EIO;
		}

causes the crash...

Ideas?

Steven Scholz

imc Measurement & Control               imc Meßsysteme GmbH
Voltastr. 5                             Voltastr. 5
13355 Berlin                            13355 Berlin
Germany                                 Deutschland

These error messages are from an ancient version of cfi_cmdset_0002.c. They come from over-efficient software design that didn't consider the quirks of how a flash chip works. Have you tried a recent version?

Thayne Harbaugh
Linux Networx

Be aware that the more recent version from MTD CVS are broken for interleaved chips.

http://lists.infradead.org/pipermail/linux-mtd/2004-March/009467.html

David Vrabel

What does "are broken" mean?
Just that autodetect of the flash type might not work? Or will read and write operations fail once the flash is detected correctly?

IOW: Could I use the MTD CVS with "interleaved chips" when I make sure my chips is correctly detected?

Thanks,

Steven

Ehm. I am using a 2.4.25 kernel. I thought the MTD code in there should be quiet recent...

How can I check the version of the MTD code in my kernel tree?

Steven Scholz

imc Measurement & Control               imc Meßsysteme GmbH
Voltastr. 5                             Voltastr. 5
13355 Berlin                            13355 Berlin
Germany                                 Deutschland

I just notice that in my old 2.4.20 kernel tree I have a

* $Id: cfi_cmdset_0002.c,v 1.67 2003/04/15 22:32:13 thayne Exp $

and in a _new_ 2.4.25 tree I have

* $Id: cfi_cmdset_0002.c,v 1.62 2003/01/24 23:30:13 dwmw2 Exp $

How could that be !?!?

Is the MTD code not pushed up into the main kernel sources???

Steven

Steven Scholz

imc Measurement & Control               imc Meßsysteme GmbH
Voltastr. 5                             Voltastr. 5
13355 Berlin                            13355 Berlin
Germany                                 Deutschland

It is, whenever David finds the time for it.

Don't blame him, this is the result of using CVS, which is designed around the assumption of a single central repository. If current sync speed is to slow for you, try to make it easier for him to send patches forward.

Jörn

Invincibility is in oneself, vulnerability is in the opponent.
-- Sun Tzu

I don't know, if he is really happy, if everybody starts to send patches upstream. He does a good job on this and he focussed more on 2.6 in the last months. But I'm sure that he can be convinced to bring 2.4 to the same state as 2.6 at least. On the other hand we live very well with the MTD patches since long, so what :) I never trust the kernel stuff of subsystems, which have a active development team :)

Thomas
________________________________________________________________________
"Free software" is a matter of liberty, not price. To understand the concept,
you should think of "free" as in "free speech,'' not as in "free beer".
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: [email protected]

That was not what I meant (although it did sound confusing, sorry).

You can prepare patches and send them to David. Preparing those patches is stupid and boring work, I guess David likes it as little as I do. Then he can see if they are good enough and send them forward.

Or, alternatively, you can accept the current situation as the best you can get for your money, which is for free.

Jörn

My second remark is that our intellectual powers are rather geared to
master static relations and that our powers to visualize processes
evolving in time are relatively poorly developed.
-- Edsger W. Dijkstra

Well, switching from CVS wouldn't necessarily make any difference. If I were using something like BitKeeper then there'd still only be one repository which I actually _care_ about, and the effect would be much the same.

I'd be quite interested in setting up a CVS->BK gateway like the ALSA one though. That might make my life easier.

I've been fairly busy recently; I've updated the code in Linus' tree a little while ago but it could do with another update. I'm very much slower to update 2.4 unless it's actually broken. The 2.4 branch is old and dead now; all development should be on 2.6

The way the update goes is something like this...

1. Wait till the MTD CVS code is actually stable. It's not at the moment; cfi_cmdset_0002.c is known to be broken and I can't send it off while it's like that.

2. Look at the current version of the kernel I'm updating. For each file in drivers/mtd and include/linux/mtd, take the version from CVS with matching $Id$ tag -- the version from CVS which was last sent to Linus or Marcelo for inclusion.

3. Generate a patch between those versions in my CVS, and the version currently in the mainstream kernel -- those are the patches which were included in an upstream kernel but didn't go through me. Read each change carefully, and see if it's applicable to _current_ CVS code. If so, apply it. If not, discard it. If it's discarded because the problem was already fixed, that's fine. If it's discarded because it's just a bogus change, make a mental note of it; reversion of upstream patches needs to be justified (or at least noted) in your covering email because it's often done by accident when updating Linus' tree from an external CVS, and it's _very_ much frowned upon.

4. Now you have all the upstream changes merged into current CVS. Drop current CVS into a current Linux tree, take a diff between the two. Read it. All of it. Make cosmetic changes in CVS to reduce noise in the patch you're planning to send.

5. Read it again.

6. Build it. Test it if you're feeling conscientious.

7. Read it again. Identify separate change sets from the whole thing; 'bk citool' is useful for this. Commit each set of changes separately with an appropriate changelog.

8. Stick it in a BK tree somewhere for Linus or Marcelo to pull from.

You can, of course, replace #7 and #8 with splitting it up into separate patches and mailbombing if you have religious objections to BK.

But generally, we try to keep CVS working and stable. It's rare that it is non-working for a prolonged period of time. You _should_ be able to just drop it in to a recent 2.4 or 2.6 kernel and expect it to 'just work'.

If I had hardware with interleaved AMD chips to hand, I'd have a look at it (or at least wander round the corner to Arcom and take away David's excuse :)

dwmw2

Right. That I see the problem doesn't mean that I also see a solution. :(

2, 3 and 4 involve some manual work that could be automated, not sure if you've already done that. If you don't a human "patch assistant" could help you, although I don't suspect many volunteers. ;)

7 is a bit philosophical. It would be easier if patches were kept seperate from the beginning, something CVS doesn't allow to do. But then you need to deal with dependences between them, and - worse - make sure that seperate patches remain seperate. It would make merging easier, but the time before merging will be different, much harder unless everyone has the discipline for it.

Propably this wouldn't work in your current mode of "everyone dump into CVS", you'd have more work reading and refusing patches. That work is currently distributed, which makes your life easier somewhere else.

Jörn

And spam is a useful source of entropy for /dev/random too!
-- Jasmine Strong

The point is that there isn't a 'solution', and that the problem isn't CVS.

The 'problem' is the task at hand -- maintaining a kernel subsystem that only a few people care deeply about.

I think it needs to be done outside Linus' tree, for the following reasons:
- Patches are of varying quality and I don't have time to vet or rewrite them all before timely inclusion.
- People don't use the current kernels, even though they should.
- Having a development branch which gets fairly widespead testing is useful.

Letting people commit directly to CVS seems to work fairly well. It gets stuff tested without me having to do everything myself, and makes it nice and easy for people to take over 'maintenance' of individual drivers. I think we manage to keep a fairly reasonable balance of stability in CVS too -- it's not often broken.

The downside is that it does take a while for Linus' tree to get updated, and that's partly because the procedure involves real human intervention to vet all the changes that have gone in.

But that vetting has to be done _whatever_ happens. The question is do we restrict the number of people who can commit to CVS, and make people submit patches for vetting before they're committed, or do we let them commit first and then improve it over time if such is necessary?

Again, I think we have the balance right. People commit reasonable patches to CVS, and occasionally I may revert them if I _really_ object. But they don't have to go through me or other people first, and there's not a bottleneck during _submission_.

I'm always open to suggestions for how to improve the process -- but I don't have any ideas. If someone's offended that a particular feature hasn't made it into Linus' tree, the best course of action I can recommend is to go through my update procedure themselves, merging changes from Linus' tree into CVS in a way which works in both 2.4 and 2.6, and then producing a _clean_ patch which Linus will accept, and which updates his tree to match what's now in CVS. Then mail it to me and I'll look at it. If it's all fine, all I need to do is look, and I can say "OK, Linus please apply this patch".

It's not a lot of manual work. Lots of 'grep $Id: | cut ... xargs cvs' but that's not really 'manual'.

It's not so hard. I rarely want to make changes to any given file in more than one changeset. Before BK I could just make a single big 'update' patch then open it in emacs and cut'n'paste into separate files. With BK I use citool to commit selected files in different csets.

Indeed -- and it makes other peoples lives easier too because they can actually move on without waiting for me (or other trustworthy people to whom I could delegate the task -- they exist, but they're probably not blessed with any more copious quantities of free time than I am).

dwmw2

Revert cfi_cmdset_0002.c back to 1.94? 1.95 is the change that appears to have broken interleaved chips.

David