GC operation
Hello,
a question just for better understanding of JFFS2...
When and how is GC called?jffs2_garbage_collect_pass() is called from GC thread AND after each writing to flash. But does a single call of jffs2_garbage_collect_pass() (or jffs2_garbage_collect_trigger() or SIGHUP) proceed only one node at a time? For each node / for each block / ..... some loop?
Ah, another one: Are blocks on the erase_pending_list erased beside the GC cycles?
Hm, sorry for the annoying questions, but garbage collection can be an ugly and confusing thing if one is not used to it :-) Thanks for your assistance!
Signature · 1 line
Bernhard
Quoted 1 line · Bernhard Priewasser
When and how is GC called?
From the GC thread and when there is no (or few) free space to write. In the latter case the writing process is blocked and waits until GC has freed some space.
Quoted 1 line
(or
jffs2_garbage_collect_trigger()or SIGHUP) proceed only one node at a time? For each node / for each block / ..... some loop?
jffs2_garbage_collect_pass() moves one node at a time.
Quoted 1 line
Ah, another one: Are blocks on the
erase_pending_listerased beside the GC cycles?
They are erased later, yes.
Quoted 1 line
Hm, sorry for the annoying questions, but garbage collection can be an ugly and confusing thing if one is not used to it :-)
Brr, didn't get it.. GC may be ugly if what?
Signature · 3 lines
Best Regards, Artem B. Bityutskiy, St.-Petersburg, Russia.
Hi Artem,
Quoted 1 line
Brr, didn't get it.. GC may be ugly if what?
If someone wants to understand how it works in detail :-)
Quoted 4 lines
Quoted 1 line
When and how is GC called?
From the GC thread and when there is no (or few) free space to write. In the latter case the writing process is blocked and waits until GC has freed some space.
E.g. if it is considered as neccessary either by jffs2_reserve_space() or jffs2_thread_should_wake().
Something about the blocking topic... If there is almost no free space and a write command issued, can it be blocked until the whole partition is GC'd (worst case)?? What a latency time... Are there mechanisms to avoid/control this? What about the "erase suspend" thing?
Quoted 3 lines
Quoted 2 lines
Ah, another one: Are blocks on the
erase_pending_listerased beside the GC cycles?They are erased later, yes.
jffs2_erase_pending_blocks(), am I right? When is it called? I can only find it in jffs2_write_super() with count=0 and jffs2_find_nextblock() with count=1.
Many thanks,
Bernhard
Hi Bernhard,
Quoted 2 lines · Bernhard Priewasser
Quoted 1 line
Brr, didn't get it.. GC may be ugly if what?
If someone wants to understand how it works in detail :-)
So, the problem is that you think the code is not understandable? Offer your changes then.
Quoted 11 lines
Quoted 5 lines
Quoted 2 lines
When and how is GC called?
From the GC thread and when there is no (or few) free space to write. In the latter case the writing process is blocked and waits until GC has freed some space.
E.g. if it is considered as neccessary either by
jffs2_reserve_space()orjffs2_thread_should_wake().
Something about the blocking topic... If there is almost no free space and a write command issued, can it be blocked until the whole partition is GC'd (worst case)?? What a latency time... Are there mechanisms to avoid/control this? What about the "erase suspend" thing?
Hmm.
Suppose we have an almost full partition. You write N bytes. There is only M free space, M < N. If there is N-M of dirty space, GC is started. Otherwise, -ENOSPC. GC will work until there is enough space to write. No need to GC the whole partition... Only if the dirty space is distributed over it, then yes. And yes, the more data is on the FS, the less is the GC performance and the greater is the average latency...
Or, since blocks to GC are picked randomly, we may end up with GC of the whole partition, hypothetically, but unlikely.
Quoted 3 lines
jffs2_erase_pending_blocks(), am I right? When is it called? I can only find it injffs2_write_super()with count=0 andjffs2_find_nextblock()with count=1.
So, you've answered your question :-)
HTH,
Artem.
Signature · 3 lines
Best Regards, Artem B. Bityutskiy, St.-Petersburg, Russia.
Hi Artem,
Quoted 1 line
So, the problem is that you think the code is not understandable? Offer your changes then.
No, the problem is that it's not always understandable for ME, that's a big difference :-)
Quoted 3 lines
Suppose we have an almost full partition. You write N bytes. There is only M free space, M < N. If there is N-M of dirty space, GC is started. Otherwise,
-ENOSPC. GC will work until there is enough space to write. No need to GC the whole partition... Only if the dirty space is distributed over it, then yes. And yes, the more data is on the FS, the less is the GC performance and the greater is the average latency...Or, since blocks to GC are picked randomly, we may end up with GC of the whole partition, hypothetically, but unlikely.
OK. Things become more clearly. A write request to an almost full partition always issues garbage collecting just as much space (nodes) which is neccessary for writing the new data? And full GC is performed only at mount? (And if c->resv_blocks_gctrigger is reached?)
Quoted 4 lines
Quoted 3 lines
jffs2_erase_pending_blocks(), am I right? When is it called? I can only find it injffs2_write_super()with count=0 andjffs2_find_nextblock()with count=1.So, you've answered your question :-)
I meant the way it is called by kupdated (which I don't know at all).
Quoted 1 line
HTH,
It did! Thank you for patience.
Signature · 1 line
Bernhard
Quoted 6 lines · Bernhard Priewasser
Quoted 5 lines
Quoted 4 lines
jffs2_erase_pending_blocks(), am I right? When is it called? I can only find it injffs2_write_super()with count=0 andjffs2_find_nextblock()with count=1.So, you've answered your question :-)
I meant the way it is called by kupdated (which I don't know at all).
The kupdated thread wakes up every 5 seconds or so and calls sync_supers, which will eventually call jffs2_write_super() and hence jffs2_erase_pending_blocks is called.
josh
Quoted 5 lines · Bernhard Priewasser
Quoted 3 lines
Suppose we have an almost full partition. You write N bytes. There is only M free space, M < N. If there is N-M of dirty space, GC is started. Otherwise,
-ENOSPC. GC will work until there is enough space to write. No need to GC the whole partition... Only if the dirty space is distributed over it, then yes. And yes, the more data is on the FS, the less is the GC performance and the greater is the average latency...Or, since blocks to GC are picked randomly, we may end up with GC of the whole partition, hypothetically, but unlikely.
OK. Things become more clearly. A write request to an almost full partition always issues garbage collecting just as much space (nodes) which is neccessary for writing the new data? And full GC is performed only at mount? (And if
c->resv_blocks_gctriggeris reached?)
Not almost full partition. Here are some statements:
1. GC starts if there are no *free eraseblocks*.
2. GC makes free space in units of an eraseblock.
3. No GC is needed on mount.
JFFS2 does *checking* on mount, i.e., checks all CRCs. It is done in the context of the GC thread, but does not actually relate to the Garbage Collection. This is just how it is implemented.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
jffs2_erase_pending_blocks(), am I right? When is it called? I can only find it injffs2_write_super()with count=0 andjffs2_find_nextblock()with count=1.So, you've answered your question :-)
I meant the way it is called by kupdated (which I don't know at all).
Will just add to what Josh said.
jffs2_write_super() is called when the JFFS2 superblock is dirty (JFFS2's sb->s_dirt is set). JFFS2 marks its "struct superblock" structure as dirty when something is added to the erase_pending_list or written to the wbuf.
When sb->s_dirt, kupdated will call jffs2_write_super() in some configurable time and JFFS2 will erase pending eraseblocks and flush the wbuf.
I'm not sure why JFFS2 does not use its own timer for this. But it seems it used to.
Signature · 3 lines
Best Regards, Artem B. Bityutskiy, St.-Petersburg, Russia.
Quoted 7 lines
Not almost full partition. Here are some statements:
1. GC starts if there are no *free eraseblocks*.
2. GC makes free space in units of an eraseblock.
3. No GC is needed on mount.JFFS2 does *checking* on mount, i.e., checks all CRCs. It is done in the context of the GC thread, but does not actually relate to the Garbage Collection. This is just how it is implemented.
Blocking time is rather important for us, e.g. the GC and erase_pending stuff is too. Thank you for bringing some light into it, perhaps it wouldn't be bad if there was some documentation. I'm doing my Master's Thesis right now on JFFS2, that might be a good documention.
Quoted 3 lines
jffs2_write_super()is called when the JFFS2 superblock is dirty (JFFS2'ssb->s_dirtis set). JFFS2 marks its "struct superblock" structure as dirty when something is added to theerase_pending_listor written to the wbuf.When
sb->s_dirt, kupdated will calljffs2_write_super()in some configurable time and JFFS2 will erase pending eraseblocks and flush the wbuf.
If anybody else is intersted: See http://lists.infradead.org/pipermail/linux-mtd/2005-November/014271.html
and follow-up.
Signature · 1 line
Bernhard