jffs2 Erase Block Summary overhead
Hi,
after doing some test on a 64MB nand flash, we see that only 50 MB can be stored on the filesystem [1]. The extra 10 MB are taken by jffs2 summary. If we disable jffs2 sumary in the kernel, the 10MB become available.
Is there a way to reduce this overhead ?
20 % seems really big.
May be Centralized Summary [3] give better result ? What is its status ?
Matthieu
[1]
# time mount /dev/mtdblock3 -t jffs2 /tmp/mnt/ ; time cat /tmp/mnt/root/file.1 > /dev/null ; du -sh /tmp/mnt; df real 0m 3.05s user 0m 0.00s sys 0m 3.05s real 0m 0.34s user 0m 0.01s sys 0m 0.33s 50.6M /tmp/mnt Filesystem Size Used Available Use% Mounted on /dev/mtdblock3 61.5M 60.7M 832.0k 99% /tmp/mnt
[2]
# time mount /dev/mtdblock3 -t jffs2 /tmp/mnt/ ; time cat /tmp/mnt/root/file.1 > /dev/null ; du -sh /tmp/mnt; df real 0m 11.30s user 0m 0.01s sys 0m 11.28s real 0m 0.35s user 0m 0.00s sys 0m 0.33s 50.6M /tmp/mnt /dev/mtdblock3 61.5M 51.2M 10.3M 83% /tmp/mnt
Quoted 6 lines · Matthieu CASTET
after doing some test on a 64MB nand flash, we see that only 50 MB can be stored on the filesystem [1]. The extra 10 MB are taken by jffs2 summary. If we disable jffs2 sumary in the kernel, the 10MB become available.
Is there a way to reduce this overhead ?
20 % seems really big.
There's a lot of redundant information in the summary; I think we could halve it fairly easily. I don't believe your 20% figure though -- that's too high. You really can't rely on the output of df.
Signature · 1 line
dwmw2
Hi,
Quoted 3 lines · David Woodhouse
There's a lot of redundant information in the summary; I think we could halve it fairly easily.
That would be great :)
Quoted 3 lines
I don't believe your 20% figure though -- that's too high. You really can't rely on the output of df.
But the jffs2 layer say "no space left on device", so the it is really full.
For more information on the test procedure, I fill the jffs2 filesystem with someting like :
"i=0; while true; do dd if=/dev/urandom of=file.$i count=1024; i=$(($i+1));done"
And I can create only 93 complete files, a incomplete file of 430 KB (and there should be 4 MB taken file others files).
May be I am on a worse test case ?
If I remove "Erase Block Summary", I can really fill the jffs2 up to 60MB [1] instead of 51MB.
Thanks,
Matthieu
[1]
# time mount /dev/mtdblock3 -t jffs2 /tmp/mnt/ ; time cat /tmp/mnt/root/file.1 > /dev/null ; du -sh /tmp/mnt; df real 0m 11.91s user 0m 0.00s sys 0m 11.91s real 0m 0.36s user 0m 0.01s sys 0m 0.35s 59.8M /tmp/mnt Filesystem Size Used Available Use% Mounted on /dev/ram0 4.4M 4.4M 0 100% / none 7.0M 16.0k 7.0M 0% /tmp none 7.0M 16.0k 7.0M 0% /tmp tmpfs 7.0M 16.0k 7.0M 0% /tmp /dev/mtdblock3 61.5M 60.9M 640.0k 99% /tmp/mnt
Quoted 10 lines · Matthieu CASTET
Hi,
Quoted 2 lines · David Woodhouse
There's a lot of redundant information in the summary; I think we could halve it fairly easily.
That would be great :)
Quoted 3 lines
I don't believe your 20% figure though -- that's too high. You really can't rely on the output of df.
After an accurate benchmark it is more 15 % than 20%, but it still big.
Quoted 33 lines
But the jffs2 layer say "no space left on device", so the it is really full.
For more information on the test procedure, I fill the jffs2 filesystem with someting like :
"i=0; while true; do dd if=/dev/urandom of=file.$i count=1024; i=$(($i+1));done"
And I can create only 93 complete files, a incomplete file of 430 KB (and there should be 4 MB taken file others files).May be I am on a worse test case ?
If I remove "Erase Block Summary", I can really fill the jffs2 up to 60MB [1] instead of 51MB.
Thanks,
Matthieu
[1]
# time mount /dev/mtdblock3 -t jffs2 /tmp/mnt/ ; time cat /tmp/mnt/root/file.1 > /dev/null ; du -sh /tmp/mnt; df real 0m 11.91s user 0m 0.00s sys 0m 11.91s real 0m 0.36s user 0m 0.01s sys 0m 0.35s 59.8M /tmp/mnt Filesystem Size Used Available Use% Mounted on /dev/ram0 4.4M 4.4M 0 100% / none 7.0M 16.0k 7.0M 0% /tmp none 7.0M 16.0k 7.0M 0% /tmp tmpfs 7.0M 16.0k 7.0M 0% /tmp /dev/mtdblock3 61.5M 60.9M 640.0k 99% /tmp/mnt
Ühel kenal päeval (neljapäev 15 veebruar 2007 5:30 pm) kirjutas Matthieu CASTET:
Quoted 6 lines
Hi,
after doing some test on a 64MB nand flash, we see that only 50 MB can be stored on the filesystem [1]. The extra 10 MB are taken by jffs2 summary. If we disable jffs2 sumary in the kernel, the 10MB become available.
This is interesting. I have a Puppy Linux build within 52MB (ps 512B/ebs 16KB) and sumtool grows it to 57MB.
JFFS2/device with ps of 2048B/ebs 128KB has much much better compression level and sum adds ca. 3-4% for this case.
You can play with "dry" mkfs.jffs2 and different page;block sizes/file counts/file types to get the whole picture. Or read the code ;) Somebody could put a nice summary table into FAQ with estimated results.
cheers,
Indrek
Quoted 40 lines
Is there a way to reduce this overhead ?
20 % seems really big.May be Centralized Summary [3] give better result ? What is its status ?
Matthieu
[1]
# time mount /dev/mtdblock3 -t jffs2 /tmp/mnt/ ; time cat /tmp/mnt/root/file.1 > /dev/null ; du -sh /tmp/mnt; df real 0m 3.05s user 0m 0.00s sys 0m 3.05s real 0m 0.34s user 0m 0.01s sys 0m 0.33s 50.6M /tmp/mnt Filesystem Size Used Available Use% Mounted on /dev/mtdblock3 61.5M 60.7M 832.0k 99% /tmp/mnt[2]
# time mount /dev/mtdblock3 -t jffs2 /tmp/mnt/ ; time cat /tmp/mnt/root/file.1 > /dev/null ; du -sh /tmp/mnt; df real 0m 11.30s user 0m 0.01s sys 0m 11.28s real 0m 0.35s user 0m 0.00s sys 0m 0.33s 50.6M /tmp/mnt /dev/mtdblock3 61.5M 51.2M 10.3M 83% /tmp/mnt[3]
http://www.inf.u-szeged.hu/jffs2/mount.php______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
Quoted 10 lines · Indrek Kruusa
Ühel kenal päeval (neljapäev 15 veebruar 2007 5:30 pm) kirjutas Matthieu CASTET:
Quoted 6 lines
Hi,
after doing some test on a 64MB nand flash, we see that only 50 MB can be stored on the filesystem [1]. The extra 10 MB are taken by jffs2 summary. If we disable jffs2 sumary in the kernel, the 10MB become available.
This is interesting. I have a Puppy Linux build within 52MB (ps 512B/ebs 16KB) and sumtool grows it to 57MB. JFFS2/device with ps of 2048B/ebs 128KB has much much better compression level and sum adds ca. 3-4% for this case.
Mine is a ps 512B/ebs 4KB.
Quoted 1 line
You can play with "dry" mkfs.jffs2 and different page;block sizes/file counts/file types to get the whole picture.
I don't get the same result with image generated with mkfs.jffs2 and on the fly written file :
- with mkfs.jffs2 + sumtool give me only 7 % overhead
- on the fly write give me 15 % overhead
Matthieu