what's cooking in zram for 4.1
Hello Karel,
There will be some user-space visible changes in zram 4.1 we'd love to let you know about.
1) new sysfs node -- /sys/block/zramX/compact
triggers zram memory compaction.
2) zram has deprecated some of the existing stat sysfs attributes. we will consolidate zramX device's stats in 3 files, rather than having N files (per-stat).
The idea is:
-- the existing RW sysfs device nodes will be downgraded to WO nodes (in linux 4.11) -- deprecated RO sysfs nodes will eventually be removed (in linux 4.11)
User-space is advised to use the following files:
-- /sys/block/zram<id>/stat
Represents block layer statistics (read Documentation/block/stat.txt for details).
-- /sys/block/zram<id>/io_stat
The stat file represents device's I/O statistics not accounted by block layer and, thus, not available in zram<id>/stat file. It consists of a single line of text and contains the following stats separated by whitespace:
failed_reads
failed_writes
invalid_io
notify_free
-- /sys/block/zram<id>/mm_stat
The stat file represents device's mm statistics. It consists of a single line of text and contains the following stats separated by whitespace:
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
deprecated nodes will be around up until linux 4.11 (approx 2 years from now). in the meantime, zram will warn (once) should any user space app access any of the deprecated attrs:
"zram: 30788 (cat) Attribute num_reads (and others) will be removed. See zram documentation."
-ss
Karel Zak
maintainerQuoted 2 lines · Sergey Senozhatsky
There will be some user-space visible changes in zram 4.1 we'd love to let you know about.
Thanks.
Quoted 2 lines
2) zram has deprecated some of the existing stat sysfs attributes. we will consolidate zramX device's stats in 3 files, rather than having N files (per-stat).
sysfs attributes used by zramctl(8) are:
disksize
orig_data_size
compr_data_size
comp_algorithm
max_comp_streams
mem_used_total
reset
Quoted 4 lines
-- /sys/block/zram<id>/mm_stat
The stat file represents device's mm statistics. It consists of a single line of text and contains the following stats separated by whitespace:
hmm... frankly, the reason why I love /sys and why hate /proc is value-per-file. You do not need special parsers to read from /sys (usually).
Quoted 7 lines
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
Why do you need all in one file? ... to provide consistent statistics?
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 12 lines · Karel Zak
hmm... frankly, the reason why I love /sys and why hate /proc is value-per-file. You do not need special parsers to read from /sys (usually).
Quoted 7 lines
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
Quoted 2 lines
Why do you need all in one file? ... to provide consistent statistics?
yes, that's the main reason.
-ss
Hello
Quoted 13 lines · Sergey Senozhatsky
Quoted 12 lines · Karel Zak
hmm... frankly, the reason why I love /sys and why hate /proc is value-per-file. You do not need special parsers to read from /sys (usually).
Quoted 7 lines
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
Quoted 4 lines
Quoted 2 lines
Why do you need all in one file? ... to provide consistent statistics?
yes, that's the main reason.
In my side, other main reason was to reduce the number of system call to see statistics. It is not only syscall overhead itself but also causes slightly high-order allocation for kernel internal data structure via slab allocation which is bad on low memory situation where is frequent in zram-swap. Slab allocation could be fallback with 0-order pages but it could cause excessive page reclaim seriously since compaction didn't work.
Yes, it's a one of problem of current VM but there is no reason to hesitate if we can avoid such problems and support consistent statistic as well.
Signature · 2 lines
Kind regards, Minchan Kim
Quoted 5 lines · Minchan Kim
Quoted 4 lines
Quoted 2 lines
Why do you need all in one file? ... to provide consistent statistics?
yes, that's the main reason.
Quoted 10 lines
In my side, other main reason was to reduce the number of system call to see statistics. It is not only syscall overhead itself but also causes slightly high-order allocation for kernel internal data structure via slab allocation which is bad on low memory situation where is frequent in zram-swap. Slab allocation could be fallback with 0-order pages but it could cause excessive page reclaim seriously since compaction didn't work.
Yes, it's a one of problem of current VM but there is no reason to hesitate if we can avoid such problems and support consistent statistic as well.
true.
syscalls and corresponding error handling for each one of them were on a list:
https://www.marc.info/?l=linux-kernel&m=142586445420298&w=2
Karel has a remarkably good codebase, so error handling would not be an issue :-) that's why consistent stats moved forward.
in general, increasing consistency and reducing the syscall pressure are good enough to move on.
-ss
Karel Zak
maintainerQuoted 2 lines · Sergey Senozhatsky
There will be some user-space visible changes in zram 4.1 we'd love to let you know about.
...
Quoted 11 lines
-- /sys/block/zram<id>/mm_stat
The stat file represents device's mm statistics. It consists of a single line of text and contains the following stats separated by whitespace:
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
OK, zramctl(8) code in util-linux git tree supports mm_stat file now. It's able to fallback to the original /sys/block/zram<id>/<attr> if mm_stat does not exist. The change will be available in v2.27 (~Aug 2015).
I have also added support for MEM-LIMIT, MEM-USED and MIGRATED columns.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 18 lines · Karel Zak
Quoted 2 lines · Sergey Senozhatsky
There will be some user-space visible changes in zram 4.1 we'd love to let you know about.
...
Quoted 11 lines
-- /sys/block/zram<id>/mm_stat
The stat file represents device's mm statistics. It consists of a single line of text and contains the following stats separated by whitespace:
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
Quoted 6 lines
OK, zramctl(8) code in util-linux git tree supports
mm_statfile now. It's able to fallback to the original /sys/block/zram<id>/<attr> ifmm_statdoes not exist. The change will be available in v2.27 (~Aug 2015).I have also added support for MEM-LIMIT, MEM-USED and MIGRATED columns.
great, thanks!
-ss
Quoted 19 lines · Sergey Senozhatsky
Quoted 18 lines · Karel Zak
Quoted 2 lines · Sergey Senozhatsky
There will be some user-space visible changes in zram 4.1 we'd love to let you know about.
...
Quoted 11 lines
-- /sys/block/zram<id>/mm_stat
The stat file represents device's mm statistics. It consists of a single line of text and contains the following stats separated by whitespace:
orig_data_size
compr_data_size
mem_used_total
mem_limit
mem_used_max
zero_pages
num_migrated
Quoted 8 lines
Quoted 6 lines
OK, zramctl(8) code in util-linux git tree supports
mm_statfile now. It's able to fallback to the original /sys/block/zram<id>/<attr> ifmm_statdoes not exist. The change will be available in v2.27 (~Aug 2015).I have also added support for MEM-LIMIT, MEM-USED and MIGRATED columns.
great, thanks!
Thanks, guys!
Signature · 2 lines
Kind regards, Minchan Kim