A issue when migrating a process's memory using cgroup

Hi all,

I met a problem when I migrate a process's memory using cgroup.
Two processes, process A and Process B. Process A has 20G memory, I migrate it from node0 to node1, by writing "1" to "cpuset.mems". It will consume 20s time. At the time, I write process B's "cpuset.cpus" to bind the cpus. But setting "cpuset.cpus" is block untill the first operation is finished.

So I read the cpuset code in kernel. In the function of "cpuset_write_resmask", Common handling for a write to a "cpus" or "mems" file, It grabs the "cgroup_mutex" first, and release it until finish writing "cpus" or "mems". But if migrate large size memory of a process using cpuset, it will consume much time. At the time, other handling for write to "cpuset.cpus" and "cpuset.mems" will blocked because of "cgroup_mutex".

In this case, I think the "cgroup_mutex" in "cpuset_write_resmask" should be replace by littler granularity lock.

Quoted 13 lines · xiexiangyou

Hi all,

I met a problem when I migrate a process's memory using cgroup.
Two processes, process A and Process B. Process A has 20G memory, I migrate it from node0 to node1, by writing "1" to "cpuset.mems". It will consume 20s time. At the time, I write process B's "cpuset.cpus" to bind the cpus. But setting "cpuset.cpus" is block untill the first operation is finished.

So I read the cpuset code in kernel. In the function of "cpuset_write_resmask", Common handling for a write to a "cpus" or "mems" file, It grabs the "cgroup_mutex" first, and release it until finish writing "cpus" or "mems". But if migrate large size memory of a process using cpuset, it will consume much time. At the time, other handling for write to "cpuset.cpus" and "cpuset.mems" will blocked because of "cgroup_mutex".

Quoted 2 lines

In this case, I think the "cgroup_mutex" in "cpuset_write_resmask" should be replace by littler granularity lock.

Yeah, in newer kernels we no longer grab cgroup_mutex in cpuset_write_resmask().