Question: background GC on UBIFS
Hi, folks,
We did some performance test of writing data to a ubifs volume by dd. The block size of dd is 1MB.
When the volume has more than 20% free space, the speed of writing is about 1.4MB/s ~ 1.2MB/s.
However, when the free space is less and less, the speed decreases to about 83KB/s. We think the overhead is caused by GC, since GC tries to find a free space and move valid data to another place to get a free block.
Although we already have a kthread to collect freeable blocks, it doesn't touch blocks which have both valid and invalid nodes. My question is why don't we do such consolidation/defragmentation periodicly, so that the overhead could be decreased when there is not enough free space?
thanks.
Richard Weinberger
maintainerAm 31.03.2016 um 08:25 schrieb Wang Fangpeng:
Quoted 8 lines
Hi, folks,
We did some performance test of writing data to a ubifs volume by dd. The block size of dd is 1MB.
When the volume has more than 20% free space, the speed of writing is about 1.4MB/s ~ 1.2MB/s.
Quoted 8 lines
However, when the free space is less and less, the speed decreases to about 83KB/s. We think the overhead is caused by GC, since GC tries to find a free space and move valid data to another place to get a free block.
Although we already have a kthread to collect freeable blocks, it doesn't touch blocks which have both valid and invalid nodes. My question is why don't we do such consolidation/defragmentation periodicly, so that the overhead could be decreased when there is not enough free space?
AFAIK UBIFS does this on demand to be friendly to the flash. Otherwise it would move blocks around for nothing (if there is no writer).
Doing it periodically is doable but how often and when? Only when the system is completely idle?
I'm not sure if there will be a huge performance gain. Basically you shift the IO load to another location.
But you can give it a try. Add a new debugfs file to trigger an aggressive GC run. Currently GC tries to produce as much space as we budget. You can change GC to produce as much space as possible.
Thanks,
//richard
Quoted 3 lines · Richard Weinberger
Doing it periodically is doable but how often and when? Only when the system
is completely idle?
I would say - we should give a knob (mechanism) and let the userland decide when and how much to run it (policy).
Quoted 2 lines
But you can give it a try. Add a new debugfs file to trigger an aggressive GC run.
I'd say the trigger should be - GC one eraseblock. Or better, clean some dirt, or make some free space. Internally this will be GC one eraseblock, but later on this may change. UBIFS will pick the best eraseblock to GC automatically. Userland just triggers it at the right poing. I guess this should be a blocking API - userland blocks until GC is done. Then it can force more GC.
You can play with this knob, see if it helps.
However, fundamentally, UBIFS will always get slower when the space is short, by its design. You can make it a bit faster when you have little space, but you can't make it as fast as when there is more space, unfortunately. Not easily, at least.
Artem.