data loss on jffs2 filesystem on dataflash

Hi,
I loose files, which I write on a jffs2 filesystem.

I start with a freshly erased mtd partition. Then I start the following sequence:

mount /conf
echo aaa > /conf/aaa
echo aaa > /conf/aaa
umount /conf
mount /conf

After remounting, the file gets lost (I think because the GC removes the inode). I use kernel 2.6.13 with mtd out of cvs of today. The machine is a AT91RM9200 with a serial dataflash. I used already
jffs2 (most time readonly) as rootfs here for months without problems. The only specialty is, that the dataflash has a unfamiliar erase size of 8448 bytes.

Maybe someone can give me a pointer, what might be wrong.

Best regards
Peter

Here a log of the problem:
(The lines PM: are write and erase logs at the mtd device)

mount /conf
JFFS2 write-buffering enabled (8448)
[JFFS2 DBG] (109) __jffs2_dbg_dump_block_lists_nolock: dump JFFS2 blocks lists:
[JFFS2 DBG] (109) __jffs2_dbg_dump_block_lists_nolock: dump JFFS2 blocks lists:
[JFFS2 DBG] (109) jffs2_do_read_inode: read inode #1
[JFFS2 DBG] (109) jffs2_do_read_inode: creating inocache for root inode
[JFFS2 DBG] (109) jffs2_add_ino_cache: add c0e213c4 (ino #1)
[JFFS2 DBG] (109) jffs2_do_read_inode_internal: ino #1 nlink is 1
[JFFS2 DBG] (109) jffs2_get_inode_nodes: ino #1
[JFFS2 DBG] (109) jffs2_get_inode_nodes: nodes of inode #1 were read, the highest version is 0, latest_mctime 3221420188, mctime_ver 0. [root@armbox /root]$echo aaa > /conf/aaa
[JFFS2 DBG] (94) jffs2_add_ino_cache: add c0e213dc (ino #2)
[JFFS2 DBG] (94) jffs2_add_fd_to_list: add dirent "aaa", ino #2
[JFFS2 DBG] (111) jffs2_add_full_dnode_to_inode: adding node 0x00-0x04 @0x006a1770 on flash, newfrag *c0f343b8 [root@armbox /root]$
jffs2: No clean, dirty _or_ erasable blocks to GC from! Where are they all?
jffs2: Couldn't find erase block to garbage collect!
PM: dataflash_write: 8574720 .. 8583168

[root@armbox /root]$echo aaa > /conf/aaa
[JFFS2 DBG] (94) jffs2_truncate_fragtree: truncating fragtree to 0x00000000 bytes
[JFFS2 DBG] (112) jffs2_add_full_dnode_to_inode: adding node 0x00-0x04 @0x006a17fc on flash, newfrag *c0f343b8 [root@armbox /root]$jffs2: No clean, dirty _or_ erasable blocks to GC from! Where are they all?
jffs2: Couldn't find erase block to garbage collect!
PM: dataflash_write: 8574720 .. 8583168

[root@armbox /root]$umount /conf
[JFFS2 DBG] (113) jffs2_kill_fragtree: killing
[root@armbox /root]$mount /conf
JFFS2 write-buffering enabled (8448)
[JFFS2 DBG] (114) jffs2_scan_eraseblock: no summary found in jeb 0x006a1700. Apply original scan.
[JFFS2 DBG] (114) jffs2_add_ino_cache: add c0f933c4 (ino #2)
[JFFS2 DBG] (114) __jffs2_dbg_dump_block_lists_nolock: dump JFFS2 blocks lists:
[JFFS2 DBG] (114) __jffs2_dbg_dump_block_lists_nolock: dump JFFS2 blocks lists:
[JFFS2 DBG] (114) jffs2_do_read_inode: read inode #1
[JFFS2 DBG] (114) jffs2_do_read_inode: creating inocache for root inode
[JFFS2 DBG] (114) jffs2_add_ino_cache: add c0f933ac (ino #1)
[JFFS2 DBG] (114) jffs2_do_read_inode_internal: ino #1 nlink is 1
[JFFS2 DBG] (114) jffs2_get_inode_nodes: ino #1
[JFFS2 DBG] (114) jffs2_get_inode_nodes: nodes of inode #1 were read, the highest version is 0, latest_mctime 3221420188, mctime_ver 0. [JFFS2 DBG] (8) jffs2_del_ino_cache: del c0f933c4 (ino #2)

PM: dataflash_erase: addr=8574720 len=8448

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Do you have sumary enabled ? If yes, try to disable it.

Frankly, I do not know DataFlash specificity but I have several basic questions bellow :)

[snip]

Why I don't see messages from scan.c? Strange.

[snip]

[snip]

two nodes are written to the same place ? That's odd...

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

...

It's not enabled.

...

My fault, new log attached

...

Yes, that's what I thought, too.

new log...

[42949436.200000] jffs2_get_sb(): dev_name "/dev/mtdblock/2"
[42949436.200000] jffs2_get_sb(): path_lookup() returned 0, inode c0fcde94
[42949436.200000] jffs2_get_sb_mtd(): New superblock for device 2 ("etcfs")
[42949436.200000] JFFS2 write-buffering enabled (8448)
[42949436.200000] Allocating readbuf of 4096 bytes
[42949436.200000] jffs2_scan_eraseblock(): Scanning block at 0x0
[42949436.200000] Block at 0x00000000 is empty (erased)
[42949436.200000] jffs2_scan_eraseblock(): Scanning block at 0x2100
[42949436.210000] Block at 0x00002100 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0x4200
[42949436.210000] Block at 0x00004200 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0x6300
[42949436.210000] Block at 0x00006300 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0x8400
[42949436.210000] Block at 0x00008400 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0xa500
[42949436.210000] Block at 0x0000a500 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0xc600
[42949436.210000] Block at 0x0000c600 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0xe700
[42949436.210000] Block at 0x0000e700 is empty (erased)
[42949436.210000] jffs2_scan_eraseblock(): Scanning block at 0x10800
[42949436.220000] Block at 0x00010800 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x12900
[42949436.220000] Block at 0x00012900 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x14a00
[42949436.220000] Block at 0x00014a00 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x16b00
[42949436.220000] Block at 0x00016b00 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x18c00
[42949436.220000] Block at 0x00018c00 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x1ad00
[42949436.220000] Block at 0x0001ad00 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x1ce00
[42949436.220000] Block at 0x0001ce00 is empty (erased)
[42949436.220000] jffs2_scan_eraseblock(): Scanning block at 0x1ef00
[42949436.220000] Block at 0x0001ef00 is empty (erased)
[42949436.220000] Scanned flash completely
[42949436.220000] [JFFS2 DBG] (102) __jffs2_dbg_dump_block_lists_nolock: dump JFFS2 blocks lists:
[42949436.230000] flash_size: 0x021000
[42949436.230000] used_size: 0x000000
[42949436.230000] dirty_size: 0x000000
[42949436.230000] wasted_size: 0x000000
[42949436.230000] unchecked_size: 0x000000
[42949436.230000] free_size: 0x021000
[42949436.230000] erasing_size: 0x000000
[42949436.230000] bad_size: 0x000000
[42949436.230000] sector_size: 0x002100
[42949436.230000] jffs2_reserved_blocks size: 0x000000
[42949436.230000] nextblock: NULL
[42949436.230000] gcblock: NULL
[42949436.230000] clean_list: empty
[42949436.230000] very_dirty_list: empty
[42949436.230000] dirty_list: empty
[42949436.230000] erasable_list: empty
[42949436.230000] erasing_list: empty
[42949436.230000] erase_pending_list: empty
[42949436.230000] erasable_pending_wbuf_list: empty
[42949436.230000] bad_list: empty
[42949436.230000] bad_used_list: empty
[42949436.230000] Pass 1 complete
[42949436.230000] Pass 2 starting
[42949436.230000] Pass 2a starting
[42949436.230000] Pass 2 complete
[42949436.230000] Pass 3 complete
[42949436.230000] [JFFS2 DBG] (102) __jffs2_dbg_dump_block_lists_nolock: dump JFFS2 blocks lists:
[42949436.230000] flash_size: 0x021000
[42949436.230000] used_size: 0x000000
[42949436.230000] dirty_size: 0x000000
[42949436.230000] wasted_size: 0x000000
[42949436.230000] unchecked_size: 0x000000
[42949436.230000] free_size: 0x021000
[42949436.230000] erasing_size: 0x000000
[42949436.230000] bad_size: 0x000000
[42949436.230000] sector_size: 0x002100
[42949436.230000] jffs2_reserved_blocks size: 0x000000
[42949436.230000] nextblock: NULL
[42949436.230000] gcblock: NULL
[42949436.230000] clean_list: empty
[42949436.230000] very_dirty_list: empty
[42949436.230000] dirty_list: empty
[42949436.230000] erasable_list: empty
[42949436.230000] erasing_list: empty
[42949436.230000] erase_pending_list: empty
[42949436.230000] erasable_pending_wbuf_list: empty
[42949436.230000] bad_list: empty
[42949436.230000] bad_used_list: empty
[42949436.230000] Not rotating empty clean_list
[42949436.230000] Not rotating empty very_dirty_list
[42949436.230000] Not rotating empty dirty_list
[42949436.230000] Not rotating empty erasable_list
[42949436.230000] Not rotating empty erase_pending_list
[42949436.230000] Rotating free_list by 7
[42949436.230000] Erase block at front of free_list is at 0000e700
[42949436.230000] JFFS2 trigger levels (size 132 KiB, block size 8 KiB, 16 blocks)
[42949436.230000] Blocks required to allow deletion:    2 (16 KiB)
[42949436.230000] Blocks required to allow writes:      3 (24 KiB)
[42949436.230000] Blocks required to quiesce GC thread: 4 (33 KiB)
[42949436.230000] Blocks required to allow GC merges:   3 (24 KiB)
[42949436.230000] Blocks required to GC bad blocks:     0 (0 KiB)
[42949436.230000] Amount of dirty space required to GC: 9799 bytes
[42949436.230000] jffs2_do_fill_super(): Getting root inode
[42949436.230000] jffs2_read_inode(): inode->i_ino == 1
[42949436.230000] [JFFS2 DBG] (102) jffs2_do_read_inode: read inode #1
[42949436.230000] [JFFS2 DBG] (102) jffs2_do_read_inode: creating inocache for root inode
[42949436.230000] [JFFS2 DBG] (102) jffs2_add_ino_cache: add c0ebc3c4 (ino #1)
[42949436.230000] [JFFS2 DBG] (102) jffs2_do_read_inode_internal: ino #1 nlink is 1
[42949436.230000] [JFFS2 DBG] (102) jffs2_get_inode_nodes: ino #1
[42949436.230000] [JFFS2 DBG] (102) jffs2_get_inode_nodes: nodes of inode #1 were read, the highest version is 0, latest_mctime 3221420412, mctime_ver 0.
[42949436.230000] jffs2_read_inode() returning
[42949436.230000] jffs2_do_fill_super(): d_alloc_root()
[42949436.230000] JFFS2: Garbage collect thread is pid 103
[42949436.230000] jffs2_thread_should_wake(): nr_free_blocks 16, nr_erasing_blocks 0, dirty_size 0x0: no
[42949436.230000] jffs2_garbage_collect_thread sleeping...

[root@armbox /root]$echo aaa > /conf/aaa
[root@armbox /root]$d[42949498.310000] jffs2: No clean, dirty _or_ erasable blocks to GC from! Where are they all?

[42949498.310000] jffs2: Couldn't find erase block to garbage collect!
[42949498.320000] PM: dataflash_write: 1022208 .. 1030656
mesg -c
[42949495.620000] jffs2_lookup()
[42949495.620000] jffs2_create()
[42949495.620000] jffs2_new_inode(): dir_i 1, mode 0x81a4
[42949495.620000] [JFFS2 DBG] (94) jffs2_add_ino_cache: add c0ebc3dc (ino #2)
[42949495.620000] jffs2_do_new_inode(): Assigned ino# 2
[42949495.620000] jffs2_reserve_space(): Requested 0x44 bytes
[42949495.620000] jffs2_reserve_space(): alloc sem got
[42949495.620000] jffs2_find_nextblock(): new nextblock = 0x0000e700
[42949495.620000] jffs2_do_reserve_space(): Giving 0x2100 bytes at 0xe700
[42949495.620000] jffs2_do_create(): reserved 0x2100 bytes
[42949495.620000] jffs2_add_physical_node_ref(): Node at 0xe700(2), size 0x44
[42949495.620000] jffs2_write_dnode wrote node at 0x0000e700(2) with dsize 0x0, csize 0x0, node_crc 0xbb60398c, data_crc 0x00000000, totlen 0x00000044
[42949495.620000] jffs2_do_create created file with mode 0x81a4
[42949495.620000] jffs2_complete_reservation()
[42949495.620000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949495.620000] jffs2_reserve_space(): Requested 0x2c bytes
[42949495.620000] jffs2_reserve_space(): alloc sem got
[42949495.620000] jffs2_do_reserve_space(): Giving 0x20bc bytes at 0xe744
[42949495.620000] jffs2_write_dirent(ino #1, name at *0xc1dcb970 "aaa"->ino #2, name_crc 0x0f46aa3f)
[42949495.620000] jffs2_add_physical_node_ref(): Node at 0xe744(2), size 0x2c
[42949495.620000] [JFFS2 DBG] (94) jffs2_add_fd_to_list: add dirent "aaa", ino #2
[42949495.620000] jffs2_complete_reservation()
[42949495.620000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949495.620000] jffs2_create: Created ino #2 with mode 100644, nlink 1(1). nrpages 0
[42949495.640000] jffs2_prepare_write()
[42949495.640000] jffs2_read_inode_range: ino #2, range 0x00000000-0x00001000
[42949495.640000] Filling non-frag hole from 0-4096
[42949495.640000] end prepare_write(). pg->flags 209
[42949495.640000] jffs2_commit_write(): ino #2, page at 0x0, range 0-4, flags 209
[42949495.640000] jffs2_write_inode_range(): Ino #2, ofs 0x0, len 0x4
[42949495.640000] jffs2_reserve_space(): Requested 0xc4 bytes
[42949495.640000] jffs2_reserve_space(): alloc sem got
[42949495.640000] jffs2_do_reserve_space(): Giving 0x2090 bytes at 0xe770
[42949495.640000] jffs2_add_physical_node_ref(): Node at 0xe770(2), size 0x48
[42949495.640000] jffs2_write_dnode wrote node at 0x0000e770(2) with dsize 0x4, csize 0x4, node_crc 0x33de5e85, data_crc 0x56bc8289, totlen 0x00000048
[42949495.640000] [JFFS2 DBG] (105) jffs2_add_full_dnode_to_inode: adding node 0x00-0x04 @0x0000e770 on flash, newfrag *c0eac3b8
[42949495.640000] Obsoleting node at 0x0000e700 of len 0x44: <7>Wasting
[42949495.640000] jffs2_complete_reservation()
[42949495.640000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949495.640000] increasing writtenlen by 4
[42949495.640000] jffs2_commit_write() returning 4
[42949498.310000] jffs2_write_super()
[42949498.310000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949498.310000] jffs2_erase_pending_blocks completed
[42949498.310000] jffs2_flush_wbuf_gc() called for ino #0...
[42949498.310000] jffs2_flush_wbuf_gc() calls gc pass
[42949498.310000] jffs2: No clean, dirty _or_ erasable blocks to GC from! Where are they all?
[42949498.310000] jffs2: Couldn't find erase block to garbage collect!
[42949498.320000] PM: dataflash_write: 1022208 .. 1030656
[42949498.460000] jffs2_flush_wbuf_gc() ends...

[root@armbox /root]$echo aaa > /conf/aaa
[root@armbox /root]$[42949518.460000] jffs2: No clean, dirty _or_ erasable blocks to GC from! Where are they all?

[42949518.460000] jffs2: Couldn't find erase block to garbage collect!
[42949518.470000] PM: dataflash_write: 1022208 .. 1030656

[root@armbox /root]$dmesg -c

[42949515.170000] jffs2_setattr(): ino #2
[42949515.170000] jffs2_reserve_space(): Requested 0x44 bytes
[42949515.170000] jffs2_reserve_space(): alloc sem got
[42949515.170000] jffs2_do_reserve_space(): Giving 0x2048 bytes at 0xe7b8
[42949515.170000] jffs2_add_physical_node_ref(): Node at 0xe7b8(2), size 0x44
[42949515.170000] jffs2_write_dnode wrote node at 0x0000e7b8(2) with dsize 0x0, csize 0x0, node_crc 0x247c3272, data_crc 0x00000000, totlen 0x00000044
[42949515.170000] [JFFS2 DBG] (94) jffs2_truncate_fragtree: truncating fragtree to 0x00000000 bytes
[42949515.170000] Obsoleting node at 0x0000e770 of len 0x48: <7>Wasting
[42949515.170000] jffs2_complete_reservation()
[42949515.170000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949515.180000] jffs2_prepare_write()
[42949515.180000] jffs2_read_inode_range: ino #2, range 0x00000000-0x00001000
[42949515.190000] Filling non-frag hole from 0-4096
[42949515.190000] end prepare_write(). pg->flags 209
[42949515.190000] jffs2_commit_write(): ino #2, page at 0x0, range 0-4, flags 209
[42949515.190000] jffs2_write_inode_range(): Ino #2, ofs 0x0, len 0x4
[42949515.190000] jffs2_reserve_space(): Requested 0xc4 bytes
[42949515.190000] jffs2_reserve_space(): alloc sem got
[42949515.190000] jffs2_do_reserve_space(): Giving 0x2004 bytes at 0xe7fc
[42949515.190000] jffs2_add_physical_node_ref(): Node at 0xe7fc(2), size 0x48
[42949515.190000] jffs2_write_dnode wrote node at 0x0000e7fc(2) with dsize 0x4, csize 0x4, node_crc 0x8f17e80f, data_crc 0x56bc8289, totlen 0x00000048
[42949515.190000] [JFFS2 DBG] (107) jffs2_add_full_dnode_to_inode: adding node 0x00-0x04 @0x0000e7fc on flash, newfrag *c0eac3b8
[42949515.190000] Obsoleting node at 0x0000e7b8 of len 0x44: <7>Wasting
[42949515.190000] jffs2_complete_reservation()
[42949515.190000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949515.190000] increasing writtenlen by 4
[42949515.190000] jffs2_commit_write() returning 4
[42949518.460000] jffs2_write_super()
[42949518.460000] jffs2_thread_should_wake(): nr_free_blocks 15, nr_erasing_blocks 0, dirty_size 0x0: no
[42949518.460000] jffs2_erase_pending_blocks completed
[42949518.460000] jffs2_flush_wbuf_gc() called for ino #0...
[42949518.460000] jffs2_flush_wbuf_gc() calls gc pass
[42949518.460000] jffs2: No clean, dirty _or_ erasable blocks to GC from! Where are they all?
[42949518.460000] jffs2: Couldn't find erase block to garbage collect!
[42949518.470000] PM: dataflash_write: 1022208 .. 1030656
[42949518.610000] jffs2_flush_wbuf_gc() ends...

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Peter Menzebach wrote:

[snip]

[snip]

JFFS2 writes a node at offset 0x0000e770.

[snip]

[snip]

After the second echo, it obsolets the old data node.

[snip]

And writes the new node at 0x0000e7fc

But your messages show that the write goes to the same place.

May be (guessing):
1. Your messages are invalid?
2. Something goes on with JFFS2 wbuf and it is committed to the same page (unlikely)?
3. Your flash driver is incorrect?

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Thanks for your help!

I found (at least partly) out, whats happening: Some parts of code are relying still on the assumption, that the erase size/sector size is fixed to a power of 2.

Example nodelist.c:
len = ofs & (c->wbuf_pagesize - 1)

So the jffs2 code is broken, if it *should* support erase sizes which are not 2^X.

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Ah!
What's your erasesize?

I'll fix this.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

What's your c->wbuf_pagesize ?

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Hello Peter,

I've returned to explore your problem. Do you have any success?
I have a vague idea what happens, need to think a little. Unfortunately, I have no DataFlash to just debug this.

Please, write me your flash name - I want to glance at its data sheet.

Thanks,

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Peter, please, let we be public. This may be helpful for others. Ok, I'll write you what I think later.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

I glanced at the manual. Uhh, DataFlash is very specific beast. It suppoers page program with built-in erase command... So DataFlash effectively may be considered as a block device. Then you may use any FS on it providing you have wrote proper driver? Why do you need JFFS2 then :-) ?

JFFS2 orients to "classical" flashes. They have no "write page with built-in erase" operation.

Didn't read the manual carefully, what do they refer by "Main memory array"?

BTW, having 8*1056 write buffer is not perfect ides, better make it as small as possible, i.e., 1056 bytes.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Don't. I'm actually glad about some flash with sizes not exactly matching a power of two. It causes you some pain, but generally helps to find bugs.

Still can't. Block devices have the attribute that writing AAA... to a block containing BBB... gives you one of three possible results in case of power failure:

1. BBB...BBB all written
2. AAA...AAA nothing written
3. AAA...BBB partially written.

Flash doesn't have 3, but two more cases:
4. FFF...FFF erased, nothing written
5. AAA...FFF erased, partially written

Plus the really obnoxious
6. FFF...FFF partially erased. Looks fine but some bits may flip randomly, writes may not stick, etc.

Now try finding a filesystem that is robust if 4-6 happens. ;)

What does this thing do?

Definitely.

Jörn

You can't tell where a program is going to spend its time. Bottlenecks
occur in surprising places, so don't try to second guess and put in a
speed hack until you've proven that's where the bottleneck is.
-- Rob Pike

Don't underastand this. If you mean the atomicity, CRC may help here. And no problems. Or may be you missed the the fact that we have eraseblock size = writeblock size?

It erases individual page, then writes there. To put it differently, in your terminology, eraseblock size = writeblock size.

P.S. I actually missed the mailing list, this should have gone to the MTD ML. So let's move there please.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Image this block contains the used block bitmap of ext2. All blocks handled with this bitmap are free, so the complete block contains zeroes. Now we allocate a single block, writing a single one to the bitmap. When writing, a power failure happens right after the erase:

Before:
00000000000

After:
FFFFFFFFFF

Gee, every single block is used now. Good thing this happened to the block bitmap, so we only lost some space on the fs. Somewhere else, we'd have data corruption.

Any hard disk filesystem will suffer some kind of corruption in such a case. They heavily depend on the non-existance of 4-6.

Neat.

As always. You fuck it up, I get to fix it. ;)

Jörn

Homo Sapiens is a goal, not a description.
-- unknown

Oh OK, Got it, thanks.

hehe :-)

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Hi!

ext2 and anything that does not do journalling?

I do not thing behaviour on powerfail is part of block device definition.

Pavel

64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         

Pavel, AFAIU,

Joern meant that if HDD starts a block write operation, it will accomplish it even if power-fail happens (probably there are some capacitors there). So, it is impossible, say, that HDD has written one half of a sector and has not written the other half.

And he wanted to say that DataFlash HW does not guarante this. But, perhaps, adding a special HW, this is implementable.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Hard drives contain capacitors to prevent writing of runt sectors on a powerfail? Didn't we go around this a while ago and decide it's mostly urban legend, and that plenty of people have seen runt/bad sectors?

No idea. But theoretically it should be so, at least "good" drives should. May be a competent person will comment on this, that's quite interesting.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

The only significant energy storage that hard disks contain is the inertia of the rotating disk assembly. Since the platter motor is not a generator it doesn't help. Those tiny bypass capacitors you see can't store enough energy to do anything useful during a power failure.

BUT... The PC/AT power supplies store a lot of energy and they run for many milliseconds after a power fail. 2-100 uF in series = 50 uF @ 300 v.
J = 1/2 CV^2

J = 50uF * 300^2 / 2 = 2.25 joules (lots of energy).

If the power-fail line is properly connected and if the power fail line operates at the correct time, the CPU will be halted while there is still enough energy available to complete any write that has gotten to the disk-drives sector buffer. This does not protect data, but it should certainly protect the sectors which might now contain header, good data or junk, and a proper CRC. IOW a good sector.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips). Warning : 98.36% of all statistics are fiction.

****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Actually, 3 years ago, I researched this in my company's lab. There is about a 1 in 180 probability of the power to the read/write head failing before the sector write is complete.

The end result is a sector that fails ECC check. The linux kernel reports this and your I/O operation returns with an error.

Not easy, but if you force a rewrite of the sector, the problem cures itself. Not easy to do through the VFS though. I was only able to make it happen by using dd if=/dev/zero on the unmounted device node. After that, the sector read just fine.

-Scott Kasten-

--- "linux-os (Dick Johnson)" <[email protected]>

****************************************************************

Yep. I did _not_ say anything about finishing to write a sector. What I said was that there is only one case of a started and unfinished sector: it contains partially old and partially new data and nothing else.

And the difference (one of them, at least) between hard disks and flash is the "and nothing else" part. Flash may contain other information as well or even be in a partially erased state, randomly flipping bits in the future or not accepting writes.

Jörn

When in doubt, use brute force.
-- Ken Thompson

Noone bothered defining it, but most everyone is happy about it being as it is. Non-journalling filesystems would have severe corruption on unclean umounts. lost+found would fill up much faster than people are used to, if 4-6 was common for hard disks.

Journalling filesystems actually would be robust against 4-6, as long as their block size was large enough. The journal must contain the complete erase block from flash - which is commonly in the area of 64k or 128k. Ext3 blocks are much smaller, so the fs would still corrupt.

Well - DataFlash appears to have very small block sizes. So yes, it would be possible to use ext3 on it. But then there's still the problem of limited per-block flash lifetime and ext3 doesn't do wear levelling.

Jörn

You cannot suppose that Moliere ever troubled himself to be original in the
matter of ideas. You cannot suppose that the stories he tells in his plays
have never been told before. They were culled, as you very well know.
-- Andre-Louis Moreau in Scarabouche

Worse, actually. Corruption will also happen for file data, which may pass fsck just fine. Your data is gone and noone told you about it. ;)

Jörn

A defeated army first battles and then seeks victory.
-- Sun Tzu

Ups, I sent the mail to LKML instead of MTD ML by accident :-)

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

I do need some wear leveling (and a filesystem which supports block sizes of 1056 bytes ;) )

The main memory array is the flash itself with pages of 1056 bytes

At the moment, the write buffer = reported erase block size. Is this neccessary that this is equal?

my reasons for setting erase block size to 8(16)*1056:
1. my lazyness -> I have to patch the mkfs.jff2 every time, since it makes a guess, that -e 1056 is not a very good idea (at lest the versions some times ago)

2. It seemed to me, that jffs2 boots faster, if the number of blocks is smaller.

3. A guess without proof: jffs2 likes blocks which are not too small

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Ok.

Ok, just strange name. On the picture they call it "Flash memory array".

-e specifies the *erasesize*. In your case, make it = DataFlash block size = 8*1056. So, all is OK. Although you may make it 1056 (DataFlash allows this?), it is bad idea.

But In your logs, I saw that you have *write buffer* size = 8*1056! Write buffer size is another thing. It is the minimal flash IO unit. JFFS2 assumes that it cannot write 1 byte or 100 bytes, it assumes that it can only write 'write buffer size' bytes. And the goal of the write buffer is to accumulate many small JFFS2 writes in RAM, and when the write buffer becomes full, it is flushed to flash.

So, in your case, make write buffer = Data Flash page size = 1056.

I guess we have got confused with terminology. Please, let use this:

eraseblock - the minimal erase unit
IO block - the minimal IO unit = write buffer size

eraseblocks are usually larger then IO blocks.

The larger is the IO block, the more data you loose due to unclean reboots in case of JFFS2. So, don't make it larger then 1056.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

...

How can this reported from the flash device to the mtd layer and then to jffs2?
In the code, I see at the moment only, that in the device driver mtd_info.erasesize is set, later on in jffs2 I see, that this has become the sector_size, which becomes then the wbuf_pagesize.

Is here mtd_info.ooblock to be set?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

I see, in jffs2_dataflash_setup(): c->wbuf_pagesize = c->sector_size; I don't know why Andrew Victor set wbuf size to Sector Size. May be you'll ask him?

Ok, theoretically this should work as well. But having smaller wbuf size is better.

Yes, I in case of NAND we use this, I guess you may use it as well ... May be it is time to introduce a 'writesize' field to mtd_info, instead of 'oobblock' ... ?

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Ok, wirh the answer from Andrew, the conclusion is for me at the moment:

At the moment the mtd layer only reports one usable "tile" value to jffs2, which is the erasesize.

So the erase_size is the only sensful value, which can be used as sector_size and wbuf_pagesize.

I can fix the mkfs.jffs2 (if needed), that it can generate correct images for a erase_size of 1056.

Open questions:

I think, the original problem with the lost files is not solved, if I reduce the erase size?

Make sector_size=erase_size=wbuf_size=1056 sense?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

hi,

Via mtd_info.erasesize

To reduce the number of erase blocks, JFFS2 may also concatenate a number of erase-blocks in a 'virtual block'. That is what sector_size and wbuf_pagesize might get set to.

No. All bytes on a DataFlash device are directly addressable, so there is not really an OOB area.

Regards,
Andrew Victor

Not any more. That strategy can cause data loss and other fun things which we don't want.

Jörn

Good warriors cause others to come to them and do not go to others.
-- Sun Tzu

Hi Andrew,

Yes, right. Albeit this is fixed, this doesn't matter.

The question is *why you set wbuf size = eraseblock size?* In case of DataFlash, I assume, wbuf_size = DataFlash page size (the minimal writing unit). Eraseblock size must be = DataFlash block size.

Yes, it is theoretically correct to set write buffer size = eraseblock size, but this is *really* bad. This is wasteful (imagine, any sync will will add lots of padding and waste the whole eraseblock).

And, the size wbuf has nothing to do with "concatenate number of eraseblocks", whare is the relation?

Stop, mtd_info.ooblock is the size of NAND page in case of NAND. I would read it as 'the size of block which contains OOB'. Agree, not very good name.

So, I offered to fill mtd_info.ooblock by the size of DataFlash page, and set.

Comments? Thanks.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

hi Artem,

I don't. I set the wbuf_pagesize to the sector_size.
(Which at that time was 1 or more eraseblocks. And I think sector_size is/was the smallest unit of flash that JFFS2 would ever erase/write).

Maybe there is some confusion here. Dataflash has both 'pages' and 'blocks'. With a 'page' being the smallest unit that can be erased/written.

The MTD-device's erasesize is therefore set to the smallest unit - the Dataflash page size (528 or 1056, depending on device)

Dataflash does not use Padding nodes.

Maybe it would be more optimal to set the wbuf_size to eraseblocksize? I don't know.

I do know that JFFS2-on-Dataflash was working fine when I committed the changes in February.

Regards,
Andrew Victor

Andrew,

You used to think or still think :-) ?
eraseblock, which is of size = c->sector_size is the smallest JFFS2 erase unit.
The smallest write unit is usually *smaller* then eraseblock and = c->wbuf_pagesize. Your DataFlash port has c->wbuf_pagesize = c->sector_size which is wasteful. This leads to much faster Flash wear. This makes unneeded loads to JFFS2. This should work, but not the best approach.

Yes, but it is impossible to make JFFS2 eraseblock = DataFlash page size. Why? Because the page is too small. JFFS2's nodes cannot overlap eraseblock's borders. So, we cannot fit large 4K JFFS2 nodes to such small (1056 bytes) eraseblocks. Thus, the most logical approach is to make eraseblock size = DataFlash page size.

Well, may be this is right that *MTD* reports erasesize = DataFlash page size. But JFFS2 then must realize that it works on top of DataFlash and use blocks as erase units.

I'm about JFFS2 - it pads wbuf. But well, it does not matter anyway. If your wbuf is 8K, you write there 100 byters then do sync, you wate (8K - 100) bytes. It doen't matter whether you write padding or not. You waste the space, period. :-)

Grrr. No! We don't understand each other at all :-)

Now it doesn't :-) Sure, you don't have to fix it :-) We were just curious. I guess Peter should deal with this.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Ok guys,
I will try, to get it to work, but at the moment I'm a little bit lost.

So here, what I saw in the data sheet, code and your answers:

Dataflash:

The dataflash has a smallest unit = dataflash page, which can be written and erased (528/1056 bytes)

The dataflash has blocks, which are 8 * pagesize. This units may be used for programming/erasing, but is not used in the driver

The dataflash has sectors, which are 256 pages (sector0+1 are special). These sectors are only relevant in terms of wear leveling, since each page of a sector must be reprogrammed at least once each 10.000 writes.

jffs2:
jffs2 nodes are 4 K
the wbuf_size should be the smallest size, which you can re-program

My assumptions now:

We have to set the jffs2->sector_size to 8 * dataflash pagesize We have to set the jffs2->wbuf_size to dataflash pagesize (528/1056) We have to set the mtd->erasesize to 8 * dataflash pagesize

And still I didn't get the point, why it isn't working today....

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Yes, IMO.

For JFFS2 it is OK. But for other purposes it may be not OK. So, that's up to you. MTD is NOR/NAND-centric at the moment. You may fix this.

Me too so far :-) If you send me a DataFlash-ed board - I'll dig this :-)

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

I mean, MTD probably must returm both - page and block sized. JFFS2 should use block.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Ok, sector_size now 8*1056, wbuf_size = 1056 erase_size=8*1056 -> kernel oops
-> ok, found SECTOR_ADDR was defined wrong
-> after fixing stable and not working correctly as before ;)
...

Ok, Ok, now I am digging a bit....
with some more printfs, I now looked a bit clearer, and have another question:
when jffs2_flush_wbuf is called, how is this area now markes as written.
Sometimes later jffs2_do_reserve_space is called an returns the same address, which was written to flash....

Any hints?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

The space is accounted by corresponding fields in jffs2_eraseblock structure: free_size, wasted_size, used_size.

Jffs2 always writes to c->sector_size - jeb->free_size.

The corresponding place in jffs2_flush_wbuf, where the parameters a adjusted is wbuf.c:506.

But.. Argh! Look at line 488: if (pad && !jffs2_dataflash(c))

Why !jffs2_dataflash(c)??? I bet this is the bug.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Nope, it's OK there, only when padding.

In my logs, wbuf_len is always 0.

Then when I write the flash,
free_size, wasted_size, used_size is adjusted with 0.

How is the intention of the write buffer?
- I write 10 bytes at the beginning
- Then after 2 seconds, the buffer is flushed
- I it correct, to adjust with wbuf_len, or must it be wbuf_size?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

OK, its late here and my head is working only on 2%. But anyway, today I insist this is a bug. :-)

I don't understand what is the pad parameter at all. AFAICS, it is just an old rudiment. What is padding? If the write buffer is not full, we fill the rest of it by padding and flush it. Why we may not want this? I have no idea. This is something odd. Why DataFlash is something special??? May be I'm too stupid today to understand this? Later.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

You are right (both):
It's late, and it was the bug....
Tomorrow is another day....

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Ok, so far one step again...
but I don't find the place, where jeb->free_size and c->free_size is adjusted for the *data* (wbuf_len). I find this only for the padding. Any ideas?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Hmm, at the very end of __jffs2_flush_wbuf() I suppose.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Well, I don't think this is the right approach, this is rather a dirty'n'ugly approach...

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

I suppose if you change this on
len = ofs % c->wbuf_pagesize;

everything should be fine.

There is a high chance that this is the only place like this.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Oh, but you said JFFS2 worked with your dataflash for ages? How could it?

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Andrew Victor have had some patches for fix, and I worked with this old version since last week: http://maxim.org.za/AT91RM9200/

He committed his changes to the mtd cvs at the beginning of this year. And I now have upgraded to 2.6.13, and found some problems in the recent version.

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

[snip]

[snip]

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Ok, one step forward. With these patches I can now copy a generated rootfs to flash, mount it, and read it without problems.

But with writing to an empty flash, there are still these problems. I can confirm, that the mtd interface works properly when reading, writing, erasing flash. For testing, I have replaced only the jffs2 with an old version, leaving the mtd part the same. There everything works fine.

My guess at the moment is, that the write buffering has some problems somewhere. I can see, that everytime a whole pagesize is written to flash. Is that intended?

Any ideas?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

This is OK. The whole goal of the write-buffer is to accumulate small writes and to flush the whole page at once. It is assumed that the minimal flash IO unit is c->wbuf_pagesize.

Could you please reproduce the problem, gather the log and send it again, but with 2 notes:

1. please, output messages synchronously to a serial console (try dmesg -n 8). When you use dmesg you see only the last output and loose everything which does not fit the internal Linux output ring buffer.

2. AFAIU, after remount files disappear. Please, also send the log of the second mount/

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Do you put this into cvs or shall I submit it later again?

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

No I didn't. I would like to commit this lated, together with other DataFlash fixes, unless this is really important to you to commit this now.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

BTW, try to disable EBS. It is a new feature and may have a bug.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

OK guys,
here we are with a patch for it. I made several tests now with wbuf->pagesize of 1056 and 8*1056 and erase size of 8*1056 and it looks good so far.

Open is now the question, if we should implement a possibility to adjust the wbuf_pagesize independently from the jffs2 sectorsize. For my purposes I can live with a wbuf_size = jffs2 sector_size = 8*1056, but if we find a nice solution, I would implement it...

IMHO the cleanest solution would be, that the dataflash/mtd layer reports it's true erase_size = smallest write size = 528/1056 bytes, and should not make a further guess about later use.

JFFS2 now recognizes, that it can't work with this parameters, and should adjust it to multiples of the size internally.

Please comment to patch and how to proceed...

Best regards
Peter

P.S. Sry I can't manage to send the patch as attachment. (it get's rejected by the mailing list)

diff -urN mtda/fs/jffs2/nodelist.c mtdb/fs/jffs2/nodelist.c
@@ -412,7 +412,7 @@
 
 	/* Calculate how many bytes were already checked */
 	ofs = ref_offset(ref) + sizeof(struct jffs2_raw_inode);
-	len = ofs & (c->wbuf_pagesize - 1);
+	len = ofs % c->wbuf_pagesize;
 	if (likely(len))
 		len = c->wbuf_pagesize - len;
 

diff -urN mtda/fs/jffs2/os-linux.h mtdb/fs/jffs2/os-linux.h
@@ -65,8 +65,12 @@
 
 #define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY)
 
-#ifndef CONFIG_JFFS2_FS_WRITEBUFFER
+/*
 #define SECTOR_ADDR(x) ( ((unsigned long)(x) & ~(c->sector_size-1)) )
+*/
+#ifndef CONFIG_JFFS2_FS_WRITEBUFFER
+#define SECTOR_ADDR(x) ( (((unsigned long)(x) / c->sector_size) * c->sector_size) )
+
 
 #ifdef CONFIG_JFFS2_SUMMARY
 #define jffs2_can_mark_obsolete(c) (0)

diff -urN mtda/fs/jffs2/scan.c mtdb/fs/jffs2/scan.c
@@ -233,12 +233,12 @@
 		c->nextblock->dirty_size = 0;
 	}
 #ifdef CONFIG_JFFS2_FS_WRITEBUFFER
-	if (!jffs2_can_mark_obsolete(c) && c->nextblock && (c->nextblock->free_size & (c->wbuf_pagesize-1))) {
+	if (!jffs2_can_mark_obsolete(c) && c->nextblock && (c->nextblock->free_size % c->wbuf_pagesize)) {
 		/* If we're going to start writing into a block which already
 		   contains data, and the end of the data isn't page-aligned,
 		   skip a little and align it. */
 
-		uint32_t skip = c->nextblock->free_size & (c->wbuf_pagesize-1);
+		uint32_t skip = c->nextblock->free_size % c->wbuf_pagesize;
 
 		D1(printk(KERN_DEBUG "jffs2_scan_medium(): Skipping %d bytes in 

nextblock to ensure page alignment\n",
skip));

diff -urN mtda/fs/jffs2/wbuf.c mtdb/fs/jffs2/wbuf.c
@@ -28,12 +28,12 @@
 static unsigned char *brokenbuf;
 #endif
 
+#define PAGE_DIV(x) ( ((unsigned long)(x) / (unsigned long)(c->wbuf_pagesize)) * (unsigned long)(c->wbuf_pagesize) )
+#define PAGE_MOD(x) ( (unsigned long)(x) % (unsigned long)(c->wbuf_pagesize) )
+
 /* max. erase failures before we mark a block bad */
 #define MAX_ERASE_FAILURES 	2
 
-/* two seconds timeout for timed wbuf-flushing */
-#define WBUF_FLUSH_TIMEOUT	2 * HZ
-
 struct jffs2_inodirty {
 	uint32_t ino;
 	struct jffs2_inodirty *next;

@@ -434,7 +434,7 @@
 	   if we have a switch to next page, we will not have
 	   enough remaining space for this.
 	*/
-	if (pad && !jffs2_dataflash(c)) {
+	if (pad ) {
 		c->wbuf_len = PAD(c->wbuf_len);
 
 		/* Pad with JFFS2_DIRTY_BITMASK initially.  this helps out ECC'd NOR

@@ -483,9 +483,9 @@
 	}
 
 	spin_lock(&c->erase_completion_lock);
-
+		
 	/* Adjust free size of the block if we padded. */
-	if (pad && !jffs2_dataflash(c)) {
+	if (pad) {
 		struct jffs2_eraseblock *jeb;
 
 		jeb = &c->blocks[c->wbuf_ofs / c->sector_size];

@@ -602,15 +602,6 @@
 
 	return ret;
 }
-
-#ifdef CONFIG_JFFS2_FS_WRITEBUFFER
-#define PAGE_DIV(x) ( ((unsigned long)(x) / (unsigned long)(c->wbuf_pagesize)) * (unsigned long)(c->wbuf_pagesize) )
-#define PAGE_MOD(x) ( (unsigned long)(x) % (unsigned long)(c->wbuf_pagesize) )
-#else
-#define PAGE_DIV(x) ( (x) & (~(c->wbuf_pagesize - 1)) )
-#define PAGE_MOD(x) ( (x) & (c->wbuf_pagesize - 1) )
-#endif
-
 int jffs2_flash_writev(struct jffs2_sb_info *c, const struct kvec 

*invecs, unsigned long count, loff_t to, size_t *retlen, uint32_t ino)
{
struct kvec outvecs[3];
@@ -654,7 +645,7 @@
erase block. Anything else, and you die.
New block starts at xxx000c (0-b = block header)
*/
- if (SECTOR_ADDR(to) != SECTOR_ADDR(c->wbuf_ofs)) {
+ if (PAGE_DIV(to) != PAGE_DIV(c->wbuf_ofs)) {
/* It's a write to a new block */
if (c->wbuf_len) {
D1(printk(KERN_DEBUG "jffs2_flash_writev() to 0x%lx causes flush of
wbuf at 0x%08x\n", (unsigned long)to, c->wbuf_ofs));
@@ -905,7 +896,7 @@
goto exit;

/* if we read in a different block, return */
- if (SECTOR_ADDR(ofs) != SECTOR_ADDR(c->wbuf_ofs))
+ if (PAGE_DIV(ofs) != PAGE_DIV(c->wbuf_ofs))
goto exit;

if (ofs >= c->wbuf_ofs) {
@@ -1104,7 +1095,7 @@
return 0;

if (!c->mtd->block_markbad)
- return 1; // What else can we do?
+ return 1;

D1(printk(KERN_WARNING "jffs2_write_nand_badblock(): Marking bad
block at %08x\n", bad_offset));
ret = c->mtd->block_markbad(c->mtd, bad_offset);

diff -urN mtda/drivers/mtd/mtdpart.c mtdb/drivers/mtd/mtdpart.c
@@ -465,8 +465,9 @@
 		if (slave->offset == MTDPART_OFS_APPEND)
 			slave->offset = cur_offset;
 		if (slave->offset == MTDPART_OFS_NXTBLK) {
-			u_int32_t emask = master->erasesize-1;
-			slave->offset = (cur_offset + emask) & ~emask;
+			slave->offset = cur_offset;
+			if ((cur_offset % master->erasesize) != 0)
+				slave->offset = ((cur_offset / master->erasesize) + 1) * master->erasesize;
 			if (slave->offset != cur_offset) {
 				printk(KERN_NOTICE "Moving partition %d: "
 				       "0x%08x -> 0x%08x\n", i,

You can leave. But do you want to have more CPU load, more Flash IO, faster flash wear, more lost data after Unclean reboot?

If no, make wbuf_size = DataFlash page size. And IMO, it is *MUST* for CVS commit.

I thinks the cleanest if it reports *both* sizes. Let the upper (above MTD) layers decide what erasesize to use.

That's usual thing, unfortunately.

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

But in this case mtd_info structure has to be changed/extended. Otherwise the information does not get to the jffs2 layer. How should/could this be managed?

Problem is here, how does the user get the information about the erase size used by jffs2?

Are these two values typically existing? In the dataflash case, it's only one value.

If they make sense, we have a write_chunk_size, and an erasesize size...

gets out (I commented out instead of removing)

c->sector_size != c->wbuf_ofs

C++ comment

OK

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Yes. As a hacky solution you may use mtd->oobblock to inform about DataFlash page size.

He detects that the flash is DataFlash (mtd->type == MTD_DATAFLASH). Afterwards he knows, that
1. mtd->erasesize is the size of DataFlash block
2. mtd->oobblock is the size of DataFlash page

May be this is not the best solution. Consult tglx1.

Sorry, didn't get it. I mean, MTD provide information about both DataFlash block size and DataFlash page size. That's obvious.

Of course.
c->sector_size = eraseblock size
c->wbuf_ofs - the absolute offset from the beginning of the partition where wbuf will be written.

The if() just check if JFFS2 switched to another eraseblock (because we are at the end of eraseblock and there is anyway too few space to write any node). So, SECTOR_ADDR() is absolutely correct. What's wrong?

Well, it does not relate to the DataFlash stuff. It is generally good idea to send this as a distinct patch. But don't want to be an ass, so, let it be :-) Although dwmw2 is usually hit the celling when I edit his comments and I would prefer if you just change it to /* */ ... :-)

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

hi,

No. You can't change mtd->erasesize.
There are other existing applications that access the raw MTD layer and want the real erasesize.

This magic minimum block size could always be calculated in JFFS2 (in jffs2_dataflash_setup). It is a JFFS2-specific value. Just calculate an 'N' where:
(N * mtd->erasesize) >= jffs2_minimum_block_size
and
(mtd->size % (N * mtd->erasesize)) == 0

Regards,
Andrew Victor

Ok guys, I don't actually know the best solution. Just think up something. I don't care about MTD as much, sorry :-) You may joint the IRC and talk to tglx1 and others.

My objections are:
1. jffs2_dataflash_setup() should set c->wbuf_pagesize = DataFlash page
2. c->sector_size should be set to DataFlash block size
3. The way how you find out DataFlash page and DataFlash block size must work on any other DataFlash.
4. When JFFS2 erases an eraseblock, it must erase c->sector_size bytes, not c->wbuf_pagesize bytes/

I think that's fair, isn't it?

Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Hi,

I agree. The erase size reported to the mtd layer should be always the physical erase size (528/1056).

That would possible.
Set jffs2->wbuf_size to erasesize and calculate the jffs2->sector size as you outlined...

Best regards
Peter

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

How about this in jffs2_dataflash_setup:

@@ -1204,14 +1197,28 @@
 	
 	/* Initialize write buffer */
 	init_rwsem(&c->wbuf_sem);
-	c->wbuf_pagesize = c->sector_size;
+	c->wbuf_pagesize = c->mtd->erasesize;
+
+	c->sector_size = ((JFFS2_MIN_SECTOR_SIZE + c->mtd->erasesize - 1) /
+			  c->mtd->erasesize) * c->mtd->erasesize;
+
+	while ((5*c->sector_size < c->mtd->size) &&
+		((c->mtd->size % c->sector_size) != 0))
+	{
+		c->sector_size+=c->mtd->erasesize;
+	}
+	if ((c->mtd->size % c->sector_size) != 0) {
+		printk(KERN_WARNING "JFFS2 no suitable sector size found\n");
+		return (1);
+	};
+	
 	c->wbuf_ofs = 0xFFFFFFFF;
 
 	c->wbuf = kmalloc(c->wbuf_pagesize, GFP_KERNEL);
 	if (!c->wbuf)
 		return -ENOMEM;
 
-	printk(KERN_INFO "JFFS2 write-buffering enabled (%i)\n", c->wbuf_pagesize);
+	printk(KERN_INFO "JFFS2 write-buffering enabled (%i) erasesize (%i)\n", c->wbuf_pagesize, c->sector_size);
 
 	return 0;
 }

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1

Peter Menzebach wrote:
forget the last patch, I made up my mind. If I do not find a good sector size, I should resize the size of flash for jffs2. See below.

The question is, if the call for jffs2_flash_setup should be moved before the size checks in jffs2_do_fill_super?

Best regards
Peter

@@ -1200,18 +1193,37 @@
 }
 
 int jffs2_dataflash_setup(struct jffs2_sb_info *c) {
+	int min_sector_size;
 	c->cleanmarker_size = 0;		/* No cleanmarkers needed */
 	
 	/* Initialize write buffer */
 	init_rwsem(&c->wbuf_sem);
-	c->wbuf_pagesize = c->sector_size;
+	c->wbuf_pagesize = c->mtd->erasesize;
+
+	min_sector_size = ((JFFS2_MIN_SECTOR_SIZE + c->mtd->erasesize - 1) / c->mtd->erasesize) * c->mtd->erasesize;
+
+	c->sector_size = min_sector_size;
+
+	c->flash_size = c->mtd->size;
+	
+	while ((c->sector_size < (2 * min_sector_size)) && (c->flash_size % c->sector_size != 0)) {
+		c->sector_size+=c->mtd->erasesize;
+	}
+
+	if ((c->flash_size % c->sector_size) != 0) {
+		c->sector_size = min_sector_size;
+		c->flash_size = (c->flash_size / min_sector_size) * min_sector_size;
+		printk(KERN_INFO "JFFS2 flash size and erase size adjusted size %dKiB erasesize %dB\n", c->flash_size/1024, c->sector_size);
+	};
+	
+	
 	c->wbuf_ofs = 0xFFFFFFFF;
 
 	c->wbuf = kmalloc(c->wbuf_pagesize, GFP_KERNEL);
 	if (!c->wbuf)
 		return -ENOMEM;
 
-	printk(KERN_INFO "JFFS2 write-buffering enabled (%i)\n", c->wbuf_pagesize);
+	printk(KERN_INFO "JFFS2 write-buffering enabled (%i) erasesize (%i)\n", c->wbuf_pagesize, c->sector_size);
 
 	return 0;
 }

Peter Menzebach
Menzebach und Wolff IT-Consulting GbR
Phone +49 751 355 387 1