jffs2 copy too long after mounting
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi all
I think I find the reason why there are so many similar logs("Check
potential deletion dirent").
It seems in jffs2_garbage_collect_deletion_dirent(), jffs2 tries to
find "whether this deletion dirent is still needed to actively delete a 'real' dirent with the same name that's still somewhere else on the flash."
I wonder why there is such a necessary handling procedure? There must
be a newest valid direntry. Why we need to remove the older one and "write out a new deletion dirent to replace it"? What if jffs2 just did not search the older deletion dirent allover the flash? Thanks.
B.R.
Jia
Quoted 78 lines · hejianet
Hi Artem, thanks for your reply
Now I can reproduce it after 2-3 days stress tests:
1.just do lots of small writing(4k bytes each time) radomly in one of 1000
thousand file
2.remove the 1000 files,now the jffs2 fs is *very* dirty.
3.I dynamically increase the thresholdnr_free_blocksto trigger the gc injffs2_thread_should_wake():if (c->nr_free_blocks + c->nr_erasing_blocks < c->resv_blocks_gctrigger && (dirty > c->nospc_dirty_size)) ret = 1;4. I watched after a while, there were tens of thousands of lines (I enable the jffs2 log level to be 1)
Check potential deletion dirent at008ddd24
Check potential deletion dirent at008ddcf4
Check potential deletion dirent at008ddcc4
Check potential deletion dirent at008ddc68
Check potential deletion dirent at008dca34
Check potential deletion dirent at008dca04
Check potential deletion dirent at0088d77c
Check potential deletion dirent at0088d74c
Check potential deletion dirent at0088d71c
Check potential deletion dirent at0088d6ec
Check potential deletion dirent at0088d6bc
Check potential deletion dirent at0088c648
Check potential deletion dirent at0088c618
Check potential deletion dirent at0088c2a8
Check potential deletion dirent at0088c278
Check potential deletion dirent at0088c248
Check potential deletion dirent at0088c218
Check potential deletion dirent at0088c1e8
Check potential deletion dirent at00886b50
Check potential deletion dirent at 00885388
Check potential deletion dirent at 00885358
Check potential deletion dirent at008852a8
Check potential deletion dirent at 00885278
5. I did find that the free blocks number of jffs2 is very slowly increasedSo could you give me any ideas, it is normal or abnormal? Thanks a lot
Quoted 23 lines · Artem Bityutskiy
Quoted 8 lines · hejianet
hi All
my kernel is 2.6.16
after mounting the nand flash with jffs2 partition, the first copy operation for a 15M file(from jffs2 partition to jffs2 partition) take 9 minutes,but the
second copy operation takes 1 minutes.
the disk usage(df result) is over 85%.
I wonder is there any patch ported to such old kernel?His is because how JFFS2 is designed. When the mount operation is finished, JFFS2 does not really finish mounting. There is the background process which keeps working - it basically reads all files in the system and checks the CRC, which is needed to find out which nodes are new and which are obsolete.
So what happens is that you mount JFFS2, the background thread keeps working (see 'top' for example).
While the background thread works, you can read files. But you cannot write - writes will be blocked. Well, it is a bit more complex - you can write a little, but from some point your writes will be blocked until the background thread finishes. Then the write will continue and finish.
Quoted 7 lines
In you case, you mount, start writing your 15MiB file, write gets blocked, waits for the background thread, then continues write. The background thread takes ages to finish on big partitions.
The second write is fast because you do not wait for the background thread any longer.
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
Sorry,maybe my question is long and bothering:(
I try to make it more easily and clearly
in jffs2_garbage_collect_deletion_dirent(),
what does it mean, why in nand, we can't actually mark nodes obsolete pernamently
/* On a medium where we can't actually mark nodes obsolete
pernamently, such as NAND flash, ..... */
Quoted 105 lines · hejianet
Hi all
I think I find the reason why there are so many similar logs("Check
potential deletion dirent").
It seems injffs2_garbage_collect_deletion_dirent(), jffs2 tries to
find "whether this deletion dirent is still needed to actively delete a 'real' dirent with the same name that's still somewhere else on the flash."
I wonder why there is such a necessary handling procedure? There must
be a newest valid direntry. Why we need to remove the older one and "write out a new deletion dirent to replace it"? What if jffs2 just did not search the older deletion dirent allover the flash? Thanks.
B.R.
JiaQuoted 67 lines · hejianet
Hi Artem, thanks for your reply
Now I can reproduce it after 2-3 days stress tests:
1.just do lots of small writing(4k bytes each time) radomly in one of 1000
thousand file
2.remove the 1000 files,now the jffs2 fs is *very* dirty.
3.I dynamically increase the thresholdnr_free_blocksto trigger the gc injffs2_thread_should_wake():if (c->nr_free_blocks + c->nr_erasing_blocks < c->resv_blocks_gctrigger && (dirty > c->nospc_dirty_size)) ret = 1;4. I watched after a while, there were tens of thousands of lines (I enable the jffs2 log level to be 1)
Check potential deletion dirent at008ddd24
Check potential deletion dirent at008ddcf4
Check potential deletion dirent at008ddcc4
Check potential deletion dirent at008ddc68
Check potential deletion dirent at008dca34
Check potential deletion dirent at008dca04
Check potential deletion dirent at0088d77c
Check potential deletion dirent at0088d74c
Check potential deletion dirent at0088d71c
Check potential deletion dirent at0088d6ec
Check potential deletion dirent at0088d6bc
Check potential deletion dirent at0088c648
Check potential deletion dirent at0088c618
Check potential deletion dirent at0088c2a8
Check potential deletion dirent at0088c278
Check potential deletion dirent at0088c248
Check potential deletion dirent at0088c218
Check potential deletion dirent at0088c1e8
Check potential deletion dirent at00886b50
Check potential deletion dirent at 00885388
Check potential deletion dirent at 00885358
Check potential deletion dirent at008852a8
Check potential deletion dirent at 00885278
5. I did find that the free blocks number of jffs2 is very slowly increasedSo could you give me any ideas, it is normal or abnormal? Thanks a lot
Quoted 27 lines · Artem Bityutskiy
Quoted 8 lines · hejianet
hi All
my kernel is 2.6.16
after mounting the nand flash with jffs2 partition, the first copy operation for a 15M file(from jffs2 partition to jffs2 partition) take 9 minutes,but the
second copy operation takes 1 minutes.
the disk usage(df result) is over 85%.
I wonder is there any patch ported to such old kernel?His is because how JFFS2 is designed. When the mount operation is finished, JFFS2 does not really finish mounting. There is the background process which keeps working - it basically reads all files in the system and checks the CRC, which is needed to find out which nodes are new and which are obsolete.
So what happens is that you mount JFFS2, the background thread keeps working (see 'top' for example).
While the background thread works, you can read files. But you cannot write - writes will be blocked. Well, it is a bit more complex - you can write a little, but from some point your writes will be blocked until the background thread finishes. Then the write will continue and finish.
In you case, you mount, start writing your 15MiB file, write gets blocked, waits for the background thread, then continues write. The background thread takes ages to finish on big partitions.
Quoted 10 lines
Quoted 3 lines
The second write is fast because you do not wait for the background thread any longer.
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
Quoted 7 lines · hejianet
Sorry,maybe my question is long and bothering:(
I try to make it more easily and clearly
injffs2_garbage_collect_deletion_dirent(),
what does it mean, why in nand, we can't actually mark nodes obsolete pernamently
/* On a medium where we can't actually mark nodes obsolete
pernamently, such as NAND flash, ..... */
Because in NAND you cannot change the contents of a NAND page which has already been written, because you have ECC for the data in the OOB. So you cannot just mark a node as obsolete "in-place". In NOR you can do this.
Signature · 2 lines
Best Regards, Artem Bityutskiy