Introduce new attribute "priority" to control group
cover Introduce new attribute "priority" to control group
From: Yulei Zhang <[email protected]>
This patch is the init patch of a series which we want to present the idea of prioritized tasks management. As the cloud computing introduces intricate configurations to provide customized infrasturctures and friendly user experiences, in order to maximum utilization of sources and improve the efficiency of arrangement, we add the new attribute "priority" to control group, which could be used as graded factor by subssystems to manipulate the behaviors of processes.
Base on the order of priority, we could apply different resource configuration strategies, sometimes it will be more accuracy instead of fine tuning in each subsystem. And of course to set fundamental rules, for example, high priority cgroups could seize the resource from cgroups with lower priority all the time.
The default value of "priority" is set to 0 which means the highest priority, and the totally levels of priority is defined by CGROUP_PRIORITY_MAX. Each subsystem could register callback to receive the priority change notification for their own purposes.
We would like to send out the corresponding features in the coming weeks, which are relaying on the priority settings. For example, the prioritized oom, memory reclaiming and cpu schedule strategy.
Lei Chen (1):
cgroup: add support for cgroup priority
include/linux/cgroup-defs.h | 2 + include/linux/cgroup.h | 2 + kernel/cgroup/cgroup.c | 90 +++++++++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+)
Signature · 1 line
2.28.0
A message that is not in the archive (sent off-list or not yet received): cover.1617355387.git.yuleixzhang-1Nz4purKYjRBDgjK7y7TUQ@public.gmane.org
RFC 1/1 cgroup: add support for cgroup priority
From: Lei Chen <[email protected]>
Introduce new attribute "priority" to control group, which could be used as scale by subssytem to manipulate the behaviors of processes.
The default value of "priority" is set to 0 which means the highest priority, and the totally levels of priority is defined by CGROUP_PRIORITY_MAX.
--- include/linux/cgroup-defs.h | 2 + include/linux/cgroup.h | 2 + kernel/cgroup/cgroup.c | 90 +++++++++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+)
include/linux/cgroup-defs.h +2 -0
@@ -417,6 +417,7 @@ struct cgroup {u16 subtree_ss_mask;u16 old_subtree_control;u16 old_subtree_ss_mask; +u16 priority;/* Private pointers for each registered subsystem */ struct cgroup_subsys_state __rcu *subsys[CGROUP_SUBSYS_COUNT ];
@@ -640,6 +641,7 @@ struct cgroup_subsys {void (*exit)(struct task_struct *task);void (*release)(struct task_struct *task);void (*bind)(struct cgroup_subsys_state *root_css); +int (*css_priority_change)(struct cgroup_subsys_state *css,u16 old,u16 new); bool early_init:1;
include/linux/cgroup.h +2 -0
@@ -47,6 +47,7 @@ struct kernel_clone_args;/* internal flags */ #define CSS_TASK_ITER_SKIPPED (1U << 16) +#define CGROUP_PRIORITY_MAX 8/* a css_task_iter should be treated as an opaque object */ struct css_task_iter {
@@ -957,5 +958,6 @@ static inline void cgroup_bpf_get(struct cgroup *cgrp) {}static inline void cgroup_bpf_put (struct cgroup *cgrp) {}#endif /* CONFIG_CGROUP_BPF */ +ssize_t cgroup_priority (struct cgroup_subsys_state *css);#endif /* _LINUX_CGROUP_H */
kernel/cgroup/cgroup.c +90 -0
114 lines not on this page · raw message
Signature · 1 line
2.28.0
kernel test robot
build botRe: [RFC 1/1] cgroup: add support for cgroup priority
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on cgroup/for-next]
[also build test ERROR on v5.12-rc5 next-20210401]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/yulei-kernel-gmail-com/Introduce-new-attribute-priority-to-control-group/20210404-225313 base: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next config: s390-randconfig-r025-20210404 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 30df6d5d6a8537d3ec7d8fe4299289a4c5a74d5c)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install s390 cross compiling tool for clang build
# apt-get install binutils-s390x-linux-gnu
# https://github.com/0day-ci/linux/commit/8000447113bfea873921c098f0033aba76d206ae
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review yulei-kernel-gmail-com/Introduce-new-attribute-priority-to-control-group/20210404-225313 git checkout 8000447113bfea873921c098f0033aba76d206ae
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=s390
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
#define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
^
include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
___constant_swab32(x) : \
^
include/uapi/linux/swab.h:20:12: note: expanded from macro '___constant_swab32'
(((__u32)(x) & (__u32)0x0000ff00UL) << 8) | \
^
In file included from kernel/cgroup/cgroup.c:35:
In file included from include/linux/init_task.h:18:
In file included from include/net/net_namespace.h:39:
In file included from include/linux/skbuff.h:31:
In file included from include/linux/dma-mapping.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:80:
include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
#define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
^
include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
___constant_swab32(x) : \
^
include/uapi/linux/swab.h:21:12: note: expanded from macro '___constant_swab32'
(((__u32)(x) & (__u32)0x00ff0000UL) >> 8) | \
^
In file included from kernel/cgroup/cgroup.c:35:
In file included from include/linux/init_task.h:18:
In file included from include/net/net_namespace.h:39:
In file included from include/linux/skbuff.h:31:
In file included from include/linux/dma-mapping.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:80:
include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
#define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
^
include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32'
___constant_swab32(x) : \
^
include/uapi/linux/swab.h:22:12: note: expanded from macro '___constant_swab32'
(((__u32)(x) & (__u32)0xff000000UL) >> 24)))
^
In file included from kernel/cgroup/cgroup.c:35:
In file included from include/linux/init_task.h:18:
In file included from include/net/net_namespace.h:39:
In file included from include/linux/skbuff.h:31:
In file included from include/linux/dma-mapping.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:80:
include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu'
#define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
^
include/uapi/linux/swab.h:120:12: note: expanded from macro '__swab32'
__fswab32(x))
^
In file included from kernel/cgroup/cgroup.c:35:
In file included from include/linux/init_task.h:18:
In file included from include/net/net_namespace.h:39:
In file included from include/linux/skbuff.h:31:
In file included from include/linux/dma-mapping.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:80:
include/asm-generic/io.h:501:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
__raw_writeb(value, PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:511:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
__raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:521:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
__raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:609:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
readsb(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:617:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
readsw(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:625:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
readsl(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:634:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
writesb(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:643:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
writesw(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:652:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
writesl(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
>> kernel/cgroup/cgroup.c:4831:26: error: incompatible pointer types passing 'u16 *' (aka 'unsigned short *') to parameter of type 'int *' [-Werror,-Wincompatible-pointer-types]
ret = kstrtoint(buf, 0, &prio);
^~~~~
include/linux/kernel.h:245:67: note: passing argument to parameter 'res' here
int __must_check kstrtoint(const char *s, unsigned int base, int *res);
^20 warnings and 1 error generated.
vim +4831 kernel/cgroup/cgroup.c
4822
4823 static ssize_t cgroup_priority_write(struct kernfs_open_file *of,
4824 char *buf, size_t nbytes, loff_t off)
4825 {
4826 struct cgroup *cgrp, *parent;
4827 ssize_t ret;
4828 u16 prio, orig;
4829
4830 buf = strstrip(buf);
> 4831 ret = kstrtoint(buf, 0, &prio);
4832 if (ret)
4833 return ret;
4834
4835 if (prio < 0 || prio >= CGROUP_PRIORITY_MAX)
4836 return -ERANGE;
4837
4838 cgrp = cgroup_kn_lock_live(of->kn, false);
4839 if (!cgrp)
4840 return -ENOENT;
4841 parent = cgroup_parent(cgrp);
4842 if (parent && prio < parent->priority) {
4843 ret = -EINVAL;
4844 goto unlock_out;
4845 }
4846 orig = cgrp->priority;
4847 if (prio == orig)
4848 goto unlock_out;
4849
4850 cgroup_set_priority(cgrp, prio);
4851 cgroup_priority_propagate(cgrp);
4852 unlock_out:
4853 cgroup_kn_unlock(of->kn);
4854
4855 return ret ?: nbytes;
4856 }
4857
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Quoted 15 lines · On Sun, Apr 04, 2021 at 10:51:53PM +0800…
From: Yulei Zhang <[email protected]>
This patch is the init patch of a series which we want to present the idea of prioritized tasks management. As the cloud computing introduces intricate configurations to provide customized infrasturctures and friendly user experiences, in order to maximum utilization of sources and improve the efficiency of arrangement, we add the new attribute "priority" to control group, which could be used as graded factor by subssystems to manipulate the behaviors of processes.
Base on the order of priority, we could apply different resource configuration strategies, sometimes it will be more accuracy instead of fine tuning in each subsystem. And of course to set fundamental rules, for example, high priority cgroups could seize the resource from cgroups with lower priority all the time.
Quoted 8 lines
The default value of "priority" is set to 0 which means the highest priority, and the totally levels of priority is defined by
CGROUP_PRIORITY_MAX. Each subsystem could register callback to receive the priority change notification for their own purposes.We would like to send out the corresponding features in the coming weeks, which are relaying on the priority settings. For example, the prioritized oom, memory reclaiming and cpu schedule strategy.
We've been trying really hard to give each interface semantics which is logical and describable independent of the implementation details. This runs precisely against that.
Thanks.
Signature · 1 line
tejun
Quoted 26 lines · Tejun Heo
Quoted 15 lines · On Sun, Apr 04, 2021 at 10:51:53PM +0800…
From: Yulei Zhang <[email protected]>
This patch is the init patch of a series which we want to present the idea of prioritized tasks management. As the cloud computing introduces intricate configurations to provide customized infrasturctures and friendly user experiences, in order to maximum utilization of sources and improve the efficiency of arrangement, we add the new attribute "priority" to control group, which could be used as graded factor by subssystems to manipulate the behaviors of processes.
Base on the order of priority, we could apply different resource configuration strategies, sometimes it will be more accuracy instead of fine tuning in each subsystem. And of course to set fundamental rules, for example, high priority cgroups could seize the resource from cgroups with lower priority all the time.
Quoted 8 lines
The default value of "priority" is set to 0 which means the highest priority, and the totally levels of priority is defined by
CGROUP_PRIORITY_MAX. Each subsystem could register callback to receive the priority change notification for their own purposes.We would like to send out the corresponding features in the coming weeks, which are relaying on the priority settings. For example, the prioritized oom, memory reclaiming and cpu schedule strategy.
Quoted 8 lines
We've been trying really hard to give each interface semantics which is logical and describable independent of the implementation details. This runs precisely against that.
Thanks.
--
tejun
Thanks for the feedback. I am afraid that I didn't express myself clearly about the idea of the priority attribute. We don't want to overwrite the semantics
for each interface in cgroup, just hope to introduce another factor that could help us apply the management strategy. For example, In our production environment
K8s has its own priority class to implement the Qos, and it will be very helpful
if control group could provide corresponding priority to assist the implementation.
-Yulei
A message that is not in the archive (sent off-list or not yet received): CACZOiM0xA+6kAeM2sk3SfVV9Vu+5dOzC7APoNmB0Zw3jQKbg+w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 37 lines · yulei zhang
Quoted 26 lines · Tejun Heo
Quoted 15 lines · On Sun, Apr 04, 2021 at 10:51:53PM +0800…
From: Yulei Zhang <[email protected]>
This patch is the init patch of a series which we want to present the idea of prioritized tasks management. As the cloud computing introduces intricate configurations to provide customized infrasturctures and friendly user experiences, in order to maximum utilization of sources and improve the efficiency of arrangement, we add the new attribute "priority" to control group, which could be used as graded factor by subssystems to manipulate the behaviors of processes.
Base on the order of priority, we could apply different resource configuration strategies, sometimes it will be more accuracy instead of fine tuning in each subsystem. And of course to set fundamental rules, for example, high priority cgroups could seize the resource from cgroups with lower priority all the time.
Quoted 8 lines
The default value of "priority" is set to 0 which means the highest priority, and the totally levels of priority is defined by
CGROUP_PRIORITY_MAX. Each subsystem could register callback to receive the priority change notification for their own purposes.We would like to send out the corresponding features in the coming weeks, which are relaying on the priority settings. For example, the prioritized oom, memory reclaiming and cpu schedule strategy.
Quoted 8 lines
We've been trying really hard to give each interface semantics which is logical and describable independent of the implementation details. This runs precisely against that.
Thanks.
--
tejun
Quoted 10 lines
Thanks for the feedback. I am afraid that I didn't express myself clearly about the idea of the priority attribute. We don't want to overwrite the semantics
for each interface in cgroup, just hope to introduce another factor that could help us apply the management strategy. For example, In our production environment
K8s has its own priority class to implement the Qos, and it will be very helpful
if control group could provide corresponding priority to assist the implementation.
IMHO the 'priority' is a high level concept mainly to define policies and should not be part of user API. The meaning of priority changes with each use-case and it can be more than one dimension. For example I can have two jobs running on a machine. One is a batch and the second is a latency sensitive job, let's say web server. For oom-kill use-case, I might prefer to kill the web server as there will be multiple instances running and the load balancer will redirect the load. However for memory reclaim, I would prefer to reclaim from batch as the cost of refaults in the web server is visible to end customers.
Basically we should have mechanisms in the kernel which can be used to define and enforce high level priorities. I can set memory.low of the web server to prefer reclaim from batch job. For oom-kill, Android's lmkd and fb's oomd are the examples of user space oom-killer where policies are in user space.