Add a document describing the resource counter abstraction
PATCH 1/1 Add a document describing the resource counter abstraction
The resource counter is supposed to facilitate the resource accounting of arbitrary resource (and it already does this for memory controller).
However, it is about to be used in other resources controllers (swap, kernel memory, networking, etc), so provide a doc describing how to work with it. This will eliminate all the possible future duplications in the appropriate controllers' docs.
---
Documentation/controllers/resource_counter.txt new file +173 -0
@@ -0,0 +1,173 @@ + + The Resource Counter + +The resource counter, declared at include/linux/res_counter.h +is supposed to facilitate the resource management by controllers +by providing common stuff for accounting. + +This "stuff" includes the res_counter structure and routines +to work with it. + + + +1. Crucial parts of the res_counter structure + + a. unsigned long long usage + + The usage value shows the amount of a resource that is consumed + by a group at a given time. The units of measurement should be + determined by the controller, that uses this counter. E.g. it can + be bytes, items or any other unit the controller operates on. + + b. unsigned long long max_usage + + The maximal value of the usage over the time. + + This value is useful when gathering a statistical information about + the particular group, as it shows the actual resource requirements + for a particular group, not just some usage snapshot. + + c. unsigned long long limit + + The maximal allowed amount of resource to consume by the group. In + case the group requests for more resources, so that the usage value + would exceed the limit, the resource allocation is rejected (see + the next section) + + d. unsigned long long failcnt + + The failcnt stands for "failures counter". This is the number of + resource allocation attempts, that failed. + + c. spinlock_t lock + + Protects changes of the above values. + + + +2. Basic accounting routines + + a. void res_counter_init(struct res_counter *rc) + + Initializes the resource counter. As usual, should be the first + routine called for a new counter. + + b. int res_counter_charge[_locked] + (struct res_counter *rc, unsigned long val) + + When a resource is about to be allocated it has to be accounted + with the appropriate resource counter (controller should determine + which one to use by its own). This operation is called "charging". + + c. void res_counter_uncharge[_locked] + (struct res_counter *rc, unsigned long val) + + When a resource is released (freed) is should be de-accounted + from the resource counter it was accounted to. This is called + "uncharging". + + The _locked routines imply that the res_counter->lock is taken. + + + 2.1 Other accounting routines + + There are more routines that may help you with the common need, like + checking whether the limit is reached or re-setting the max_usage + value. They are all declared in include/linux/res_counter.h + + + +3. Analyzing the resource counter registrations + + a. If the failcnt value constantly grows this means, that the counter's + limit is too tight. Either the group is misbehaving and consumes too + many resources, or the configuration is not suitable for the group + and the limit should be increased. + + b. The max_usage value can be used to quickly tune the group. One may + set the limits to maximal values and either load the container with + a common patterns or leave one for a while. After this the max_usage + value shows the amount of memory the container would require during + its common activity. + + Setting the limit a bit above this value gives a pretty good + configuration, that works in most of the cases. + + c. If the max_usage is much less than the limit, but the failcnt value + is growing, then the group tries to allocate a big chunk of resource + at once. + + d. If the max_usage is much less than the limit, but the failcnt value + is 0, then this group is given too high limit, that it does not + require. It is better to lower the limit a bit leaving more resource + for other groups. + + + +4. Communication with the control groups subsystem (cgroups) + +All the resource controllers, that are using cgroups and resource +counters should provide files to work with the resource counter fields. +They are recommended to adhere to the following rules: + + a. File names + + Field name File name + --------------------------------------------------- + usage usage_in_<unit_of_measurement> + max_usage max_usage_in_<unit_of_measurement> + limit limit_in_<unit_of_measurement> + failcnt failcnt + lock no file :) + + b. Reading from file should show the according field value in the + appropriate format. + + c. Writing to file + + Field Expected behavior + ---------------------------------- + usage prohibited + max_usage reset to usage + limit set the limit + failcnt reset to zero + + + +5. Usage example + + a. Declare a task group (take a look at cgroups subsystem for this) and + fold a res_counter into it + + struct my_group { + struct res_counter res; + + <other fields> + } + + b. Put hooks in resource allocation/release paths + + int alloc_something(...) + { + if (res_counter_charge(res_counter_ptr, amount) < 0) + return -ENOMEM; + + <allocate the resource and return to the caller> + } + + void release_something(...) + { + res_counter_uncharge(res_counter_ptr, amount); + + <release the resource> + } + + In order to keep the usage value self-consistent, both the + "res_counter_ptr" and the "amount" in release_something() should be + the same as they were in the alloc_something() when the releasing + resource was allocated. + + c. Provide the way to read res_counter values and set them (the cgroups + still can help with it). + + c. Compile and run :)
include/linux/res_counter.h +2 -0
@@ -9,6 +9,8 @@* * Author: Pavel Emelianov <[email protected]> * +* See Documentation/controllers/resource_counter.txt for more +* info about what this counter is. */ #include <linux/cgroup.h>
Quoted 18 lines · On Wed, 12 Mar 2008 13:25:20 +0300 Pavel Emelyanov wrote:
The resource counter is supposed to facilitate the resource accounting of arbitrary resource (and it already does this for memory controller).
However, it is about to be used in other resources controllers (swap, kernel memory, networking, etc), so provide a doc describing how to work with it. This will eliminate all the possible future duplications in the appropriate controllers' docs.
Signed-off-by Pavel Emelyanov <[email protected]>---
Documentation/controllers/resource_counter.txt new file +0
-0
Documentation/controllers/resource_counter.txt line 4
Quoted 4 lines
Documentation/controllers/resource_counter.txt new file +4
-0@@ -0,0 +1,173 @@ + + The Resource Counter + +The resource counter, declared at include/linux/res_counter.h
include/linux/res_counter.h,
Quoted 5 lines
Documentation/controllers/resource_counter.txt new file +5
-0+is supposed to facilitate the resource management by controllers +by providing common stuff for accounting. + +This "stuff" includes the res_counter structure and routines +to work with it.
Documentation/controllers/resource_counter.txt line 19
Quoted 10 lines
Documentation/controllers/resource_counter.txt new file +10
-0+ + + +1. Crucial parts of the res_counter structure + + a. unsigned long long usage + + The usage value shows the amount of a resource that is consumed + by a group at a given time. The units of measurement should be + determined by the controller, that uses this counter. E.g. it can
controller that
Documentation/controllers/resource_counter.txt line 24
Quoted 5 lines
Documentation/controllers/resource_counter.txt new file +5
-0+ be bytes, items or any other unit the controller operates on. + + b. unsigned long long max_usage + + The maximal value of the usage over the time.
over time.
Documentation/controllers/resource_counter.txt line 26
Quoted 2 lines
Documentation/controllers/resource_counter.txt new file +2
-0+ + This value is useful when gathering a statistical information about
gathering statistical information
Documentation/controllers/resource_counter.txt line 35
Quoted 9 lines
Documentation/controllers/resource_counter.txt new file +9
-0+ the particular group, as it shows the actual resource requirements + for a particular group, not just some usage snapshot. + + c. unsigned long long limit + + The maximal allowed amount of resource to consume by the group. In + case the group requests for more resources, so that the usage value + would exceed the limit, the resource allocation is rejected (see + the next section)
section).
Documentation/controllers/resource_counter.txt line 40
Quoted 5 lines
Documentation/controllers/resource_counter.txt new file +5
-0+ + d. unsigned long long failcnt + + The failcnt stands for "failures counter". This is the number of + resource allocation attempts, that failed.
attempts that failed.
Quoted 10 lines
Documentation/controllers/resource_counter.txt new file +10
-0+ + c. spinlock_t lock + + Protects changes of the above values. + + + +2. Basic accounting routines + + a. void res_counter_init(struct res_counter *rc)
Documentation/controllers/resource_counter.txt line 60
Quoted 10 lines
Documentation/controllers/resource_counter.txt new file +10
-0+ + Initializes the resource counter. As usual, should be the first + routine called for a new counter. + + b. int res_counter_charge[_locked] + (struct res_counter *rc, unsigned long val) + + When a resource is about to be allocated it has to be accounted + with the appropriate resource counter (controller should determine + which one to use by its own). This operation is called "charging".
on its own).
This description implies (at least to me) that res_counter_charge[_locked]() should be called for accounting before the actual allocation is performed. Right? So if the allocation fails, then the uncharge function below should also be called...
Quoted 4 lines
Documentation/controllers/resource_counter.txt new file +4
-0+ + c. void res_counter_uncharge[_locked] + (struct res_counter *rc, unsigned long val) +
Documentation/controllers/resource_counter.txt line 74
Quoted 10 lines
Documentation/controllers/resource_counter.txt new file +10
-0+ When a resource is released (freed) is should be de-accounted + from the resource counter it was accounted to. This is called + "uncharging". + + The _locked routines imply that the res_counter->lock is taken. + + + 2.1 Other accounting routines + + There are more routines that may help you with the common need, like
with common needs, like
Documentation/controllers/resource_counter.txt line 75
Quoted 1 line
Documentation/controllers/resource_counter.txt new file +1
-0+ checking whether the limit is reached or re-setting the max_usage
resetting
Documentation/controllers/resource_counter.txt line 76
Quoted 1 line
Documentation/controllers/resource_counter.txt new file +1
-0+ value. They are all declared in include/linux/res_counter.h
end with period.
Documentation/controllers/resource_counter.txt line 82
Quoted 6 lines
Documentation/controllers/resource_counter.txt new file +6
-0+ + + +3. Analyzing the resource counter registrations + + a. If the failcnt value constantly grows this means, that the counter's
If the failcnt value constantly grows, this means that the counter's
Documentation/controllers/resource_counter.txt line 89
Quoted 7 lines
Documentation/controllers/resource_counter.txt new file +7
-0+ limit is too tight. Either the group is misbehaving and consumes too + many resources, or the configuration is not suitable for the group + and the limit should be increased. + + b. The max_usage value can be used to quickly tune the group. One may + set the limits to maximal values and either load the container with + a common patterns or leave one for a while. After this the max_usage
pattern
Documentation/controllers/resource_counter.txt line 94
Quoted 5 lines
Documentation/controllers/resource_counter.txt new file +5
-0+ value shows the amount of memory the container would require during + its common activity. + + Setting the limit a bit above this value gives a pretty good + configuration, that works in most of the cases.
No comma.
Quoted 5 lines
Documentation/controllers/resource_counter.txt new file +5
-0+ + c. If the max_usage is much less than the limit, but the failcnt value + is growing, then the group tries to allocate a big chunk of resource + at once. +
Documentation/controllers/resource_counter.txt line 109
Quoted 10 lines
Documentation/controllers/resource_counter.txt new file +10
-0+ d. If the max_usage is much less than the limit, but the failcnt value + is 0, then this group is given too high limit, that it does not + require. It is better to lower the limit a bit leaving more resource + for other groups. + + + +4. Communication with the control groups subsystem (cgroups) + +All the resource controllers, that are using cgroups and resource
No comma.
Documentation/controllers/resource_counter.txt line 110
Quoted 1 line
Documentation/controllers/resource_counter.txt new file +1
-0+counters should provide files to work with the resource counter fields.
Is this in some cgroup filesystem or in sysfs?
Documentation/controllers/resource_counter.txt line 123
Quoted 13 lines
Documentation/controllers/resource_counter.txt new file +13
-0+They are recommended to adhere to the following rules: + + a. File names + + Field name File name + --------------------------------------------------- + usage usage_in_<unit_of_measurement> + max_usage max_usage_in_<unit_of_measurement> + limit limit_in_<unit_of_measurement> + failcnt failcnt + lock no file :) + + b. Reading from file should show the according field value in the
s/according/corresponding/ ?
Line ends with a space. :(
Please check the entire file for that.
Documentation/controllers/resource_counter.txt line 133
Quoted 10 lines
Documentation/controllers/resource_counter.txt new file +10
-0+ appropriate format. + + c. Writing to file + + Field Expected behavior + ---------------------------------- + usage prohibited + max_usage reset to usage + limit set the limit + failcnt reset to zero
---
~Randy