UBIFS: power cut test on 2.6.35 + NOR

Hi All,
I'd need a hint on the best way to go on with my development.

I'm working on a custom target based on 2.6.35.3 kernel with NOR device.

UBIFS R/W partitions seem not robust against power cut tests: I'm using integck 1.4.9 and I can have unrecoverable errors on tested volume when performing real power cuts.

First of all, I've done several MTD stress tests to validate the low
level driver.
I've checked git repos, and
http://git.infradead.org/users/dedekind/ubifs-v2.6.35.git/ <http://git.infradead.org/users/dedekind/ubifs-v2.6.35.git/shortlog>
is updated to 08/2012 (that is approx 3.7.1 kernel revision) and no more
supported.

From several discussions on the list I think the best way to solve (or
at least to have a better behaviour with ubi/ubifs) could be to get the patches from this repo and merge them with my sources.

Do you agree or is it better to do more investigation on current issues?

Thanks in advance for your support,
Enrico

yes, use http://git.infradead.org/users/dedekind/ubifs-v2.6.35.git to update.

it has several "NOR power cut" fixes, for example:

http://git.infradead.org/users/dedekind/ubifs-v2.6.35.git/commit/152780c61d65b0cc5cd5cc974e926a9c47534d5e
http://git.infradead.org/users/dedekind/ubifs-v2.6.35.git/commit/9258edaca1d7b36abd0e23661579c35fe9c4f00f
http://git.infradead.org/users/dedekind/ubifs-v2.6.35.git/commit/e5d7669e2fa0797c0c110fa9bf3e51b013ba399b

which I doubt are in 2.6.35.3

Btw. we used the (no longer available) http://git.infradead.org/users/dedekind/ubifs-v2.6.29.git
(that has been kept up-to-date with mainline till June 2011) to update our sources, and we see no NOR power cut issues.

---
N. van Bolhuis.

Hi Enrico,

I'm using 2.6.35.14 (last 2.6.35) with all UBIFS patches from the repo you mentioned above. I have also added/ported all patches from mainline 3.9 stable (excluding those only dealing with debug prints etc).

Still I have some minor (I hope) issues after running integck power-cut tests for several days. They manifest in errors on xattr nodes. There are some related patches in discussion, see:
http://thread.gmane.org/gmane.linux.kernel.lsm/18746
http://thread.gmane.org/gmane.linux.drivers.mtd/47021

BR // Mats

Hi Mats, Norbert,
thanks for your kind feedbacks.

Mats, are you working on a NOR-based system or are you referring to NAND?

I'm going to merge from 2.6.35 back-port repo and I'll let you know.

Regards,
E

NOR

Hi All,
I've merged down all 2.6.35 git repo patches and started again power cut tests with integck.

After a whirl, I had an unrecoverable error during volume mount:

[   18.060413] UBI: attaching mtd7 to ubi0
[   18.064335] UBI: physical eraseblock size:   131072 bytes (128 KiB)
[   18.070608] UBI: logical eraseblock size:    130944 bytes
[   18.076025] UBI: smallest flash I/O unit:    1
[   18.080472] UBI: VID header offset:          64 (aligned 64)
[   18.086145] UBI: data offset:                128
[   18.121404] UBI: attached mtd7 to ubi0
[   18.125167] UBI: MTD device name:            "wfs"
[   18.129962] UBI: MTD device size:            10 MiB
[   18.137534] UBI: number of good PEBs:        80
[   18.142129] UBI: number of bad PEBs:         0
[   18.146576] UBI: max. allowed volumes:       128
[   18.151196] UBI: wear-leveling threshold:    4096
[   18.155916] UBI: number of internal volumes: 1
[   18.160362] UBI: number of user volumes:     1
[   18.164818] UBI: available PEBs:             0
[   18.169264] UBI: total number of reserved PEBs: 80
[   18.174071] UBI: number of PEBs reserved for bad PEB handling: 0
[   18.180083] UBI: max/mean erase counter: 527/487
[   18.184741] UBI: image sequence number: 0
[   18.188766] UBI: background thread "ubi_bgt0d" started, PID 1219
UBI device number 0, total 80 LEBs (10475520 bytes, 10.0 MiB), available
0 LEBs (0 bytes), LEB size 130944 bytes (127.9 KiB)
[   20.391279] MTD do_write_buffer(): software timeout
[   20.452081] UBI error: nor_erase_prepare: cannot invalidate PEB 25,

write returned -5 read returned 2

Digging the web, I've found several discussions on write-timeout with
NOR devices (cfr.http://lists.infradead.org/pipermail/linux-mtd/2013-June/047177.html), but this seems not to be the root cause.

Disabling background thread I was still not able to mount the volume,
but this patch (http://git.infradead.org/ubifs-2.6.git/commit/6fb4374f6b1b3932f3acfe9d353568d3d8599cad)
solved and at least mounting is ok.

As soon as write operations are performed on the filesystem again there's a failure: it seems sector erasure is not properly handled/requested.

Flash dump shows PEB 25 this way (= partially erased), so it turns out write operation can't work:

------------------------------------------------------------
00320040 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
003201e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00320220 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
003203e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00320820 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
------------------------------------------------------------

Please note that I'm using a 'slow' device (M29EW), with a maximum sector erasure time of 4 secs, and this seems to stress asynchronous operations with ubifs and integck test.

I'll go on with commit log analysis to check if this kind of issue is already solved/discussed.

Full partition erasure has solved mount/fs issue, as expected.

Regards,
E

Hi,
at least two commits for M29EW NOR chips within MTD repo...trying to merge and then check again.

E

(Adding Huang, who has also debugged (different) problems with this NOR driver)

Hi Enrico,

I'm curious: what makes you think this has a different root cause?

I'm still working on debugging the do_write_buffer timeout on my systems. I'd recommend trying to increase the timeout in do_write_buffer(), just to see if your kernel isn't waiting long enough. (In my case, I can pretty well show that while the code *says* it's waiting for a whole jiffy -- at least 1ms -- it is in fact waiting less than that.)

I believe Huang's resolution for his problem was actually a problem with the flash part itself.

Brian

yes.
For my timeout, Micron confirmed that there is a silicon bug in this type of NOR.

thanks
Huang Shijie