Per-file compression
Quoted 10 lines · Theodore Ts'o
Quoted 7 lines · Richard Weinberger
Quoted 6 lines · Tom Marshall
We are running into space issues when upgrading Android devices to the latest version. This issue is particularly troublesome for 64-bit devices that initially shipped without 64-bit support, as the multi-lib files nearly double the space requirements. As it happens, many system files are quite compressible -- namely shared libs and apk files. So implementing filesystem compression is a natural solution.
Quoted 6 lines
Tom, have you verified whether or not this approach actually is sufficient to address the problem you are trying to solve? Specifically, (a) can you get enough space compression that it will make things fit? Given that apk files are already compressed using zip, I'm a bit surprised you would get enough of a compression savings that it's going to allow to make things fit.
Yes it works surprisingly well. Both .so files and .apk files compress by roughly 40%. I'm also a bit surprised at the .apk results, but it works great.
Quoted 3 lines
And secondly (b) after
you decompress the shared libraries, is there enough memory that the resulting system functions well.
Yes, on our particular device, it actually performs at least as well as before, possibly better. I haven't done an extensive survey of memory usage though.
The majority of compression comes from .so and .apk files. Shared libs are typically mmap'd and most of the contents accessed. apk files are not referenced much after initial dexopt.
Quoted 7 lines
Often times these older Android
devices don't have a huge amount of memory, and so either with or without the overhead of needing to decompress the entire file (which can be solved with chunking, although this tends to reduce the compression efficiency), is the system going to function well using 64-bit binaries that take up a lot more room than the older 32-bit binaries?
Our test device is not so much an older device, just a mid-range device (msm8916). It has mediocre emmc speed and decent CPU (though thermal throttling is pretty aggressive).
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
So, I wrote a thing called 'zfile' that hooks into the VFS layer and intercepts
file_operationsto do file (de)compression on the fly. When a file is opened, it reads and decompresses the data into memory. The file may be read, written, and mmaped in the usual way. If the contents are changed, the data is compressed and written back.
Quoted 6 lines
Do you really need to be able to rewrite the files? Life gets much easier if you can assume that the compressed files are read-only; especially once you start trying to use chunking, it *definitely* becomes easier if you don't need to support modifying the compressed files. And in the case of the system partition, this might be a safe assumption,yes?
You are correct. Writing is not really a requirement for us, just nice to have.