Skip to contentSkip to conversation

ipc: use a work queue to free_ipc

PATCH Superseded Giuseppe Scrivano 3 participants 5 messages Feb 17, 2020 – Feb 24, 2020 RCU

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 ipc: use a work queue to free_ipc

it avoids blocking on synchronize_rcu() in kern_umount().

the code:

\#define _GNU_SOURCE
\#include <sched.h>
\#include <error.h>
\#include <errno.h>
\#include <stdlib.h>
int main()
{
int i;
for (i = 0; i < 1000; i++)
if (unshare (CLONE_NEWIPC) < 0)
error (EXIT_FAILURE, errno, "unshare");
}

gets from:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.06
	Percent of CPU this job got: 0%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.05

to:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.02
	Percent of CPU this job got: 96%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03

---
v2:
- comment added in free_ipc_ns()

v1: https://lkml.org/lkml/2020/2/11/692

 include/linux/ipc_namespace.h |  2 ++
 ipc/namespace.c               | 20 ++++++++++++++++++--
 2 files changed, 20 insertions(+), 2 deletions(-)
include/linux/ipc_namespace.h +2 -0
index c309f43bde45..a06a78c67f19 100644
@@ -68,6 +68,8 @@ struct ipc_namespace {
 	struct user_namespace *user_ns;
 	struct ucounts *ucounts;
 
+	struct llist_node mnt_llist;
+
 	struct ns_common ns;
 } __randomize_layout;
 
ipc/namespace.c +18 -2
index b3ca1476ca51..7b9922244891 100644
@@ -117,6 +117,10 @@ void free_ipcs(struct ipc_namespace *ns, struct ipc_ids *ids,
 
 static void free_ipc_ns(struct ipc_namespace *ns)
 {
+	/* mq_put_mnt() waits for a grace period as kern_unmount()
+	 * uses synchronize_rcu().
+	 */
+	mq_put_mnt(ns);
 	sem_exit_ns(ns);
 	msg_exit_ns(ns);
 	shm_exit_ns(ns);
@@ -127,6 +131,17 @@ static void free_ipc_ns(struct ipc_namespace *ns)
 	kfree(ns);
 }
 
+static LLIST_HEAD(free_ipc_list);
+static void free_ipc(struct work_struct *unused)
+{
+	struct llist_node *node = llist_del_all(&free_ipc_list);
+	struct ipc_namespace *n, *t;
+
+	llist_for_each_entry_safe(n, t, node, mnt_llist)
+		free_ipc_ns(n);
+}
+static DECLARE_WORK(free_ipc_work, free_ipc);
+
 /*
  * put_ipc_ns - drop a reference to an ipc namespace.
  * @ns: the namespace to put
@@ -148,8 +163,9 @@ void put_ipc_ns(struct ipc_namespace *ns)
 	if (refcount_dec_and_lock(&ns->count, &mq_lock)) {
 		mq_clear_sbinfo(ns);
 		spin_unlock(&mq_lock);
-		mq_put_mnt(ns);
-		free_ipc_ns(ns);
+
+		if (llist_add(&ns->mnt_llist, &free_ipc_list))
+			schedule_work(&free_ipc_work);
 	}
 }
 
Signature · 1 line
2.24.1
maintainer Reviewed-by on v2 1/1
#1 lore raw
Quoted 33 lines · Giuseppe Scrivano

it avoids blocking on synchronize_rcu() in kern_umount().

the code:

\#define _GNU_SOURCE
\#include <sched.h>
\#include <error.h>
\#include <errno.h>
\#include <stdlib.h>
int main()
{
int i;
for (i = 0; i < 1000; i++)
if (unshare (CLONE_NEWIPC) < 0)
error (EXIT_FAILURE, errno, "unshare");
}

gets from:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.06
	Percent of CPU this job got: 0%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.05

to:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.02
	Percent of CPU this job got: 96%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03
Quoted 4 lines

---
v2:
- comment added in free_ipc_ns()

Much better, thank you!

Quoted 67 lines

v1: https://lkml.org/lkml/2020/2/11/692

 include/linux/ipc_namespace.h |  2 ++
 ipc/namespace.c               | 20 ++++++++++++++++++--
 2 files changed, 20 insertions(+), 2 deletions(-)
include/linux/ipc_namespace.h +2 -0
index c309f43bde45..a06a78c67f19 100644
@@ -68,6 +68,8 @@ struct ipc_namespace {
 	struct user_namespace *user_ns;
 	struct ucounts *ucounts;
 
+	struct llist_node mnt_llist;
+
 	struct ns_common ns;
 } __randomize_layout;
 
ipc/namespace.c +18 -2
index b3ca1476ca51..7b9922244891 100644
@@ -117,6 +117,10 @@ void free_ipcs(struct ipc_namespace *ns, struct ipc_ids *ids,
 
 static void free_ipc_ns(struct ipc_namespace *ns)
 {
+	/* mq_put_mnt() waits for a grace period as kern_unmount()
+	 * uses synchronize_rcu().
+	 */
+	mq_put_mnt(ns);
 	sem_exit_ns(ns);
 	msg_exit_ns(ns);
 	shm_exit_ns(ns);
@@ -127,6 +131,17 @@ static void free_ipc_ns(struct ipc_namespace *ns)
 	kfree(ns);
 }
 
+static LLIST_HEAD(free_ipc_list);
+static void free_ipc(struct work_struct *unused)
+{
+	struct llist_node *node = llist_del_all(&free_ipc_list);
+	struct ipc_namespace *n, *t;
+
+	llist_for_each_entry_safe(n, t, node, mnt_llist)
+		free_ipc_ns(n);
+}
+static DECLARE_WORK(free_ipc_work, free_ipc);
+
 /*
  * put_ipc_ns - drop a reference to an ipc namespace.
  * @ns: the namespace to put
@@ -148,8 +163,9 @@ void put_ipc_ns(struct ipc_namespace *ns)
 	if (refcount_dec_and_lock(&ns->count, &mq_lock)) {
 		mq_clear_sbinfo(ns);
 		spin_unlock(&mq_lock);
-		mq_put_mnt(ns);
-		free_ipc_ns(ns);
+
+		if (llist_add(&ns->mnt_llist, &free_ipc_list))
+			schedule_work(&free_ipc_work);
 	}
 }
 
Signature · 1 line
2.24.1
#2 lore raw
In reply to Giuseppe Scrivano
Quoted 25 lines · Giuseppe Scrivano

it avoids blocking on synchronize_rcu() in kern_umount().

the code:

\#define _GNU_SOURCE
\#include <sched.h>
\#include <error.h>
\#include <errno.h>
\#include <stdlib.h>
int main()
{
int i;
for (i = 0; i < 1000; i++)
if (unshare (CLONE_NEWIPC) < 0)
error (EXIT_FAILURE, errno, "unshare");
}

gets from:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.06
	Percent of CPU this job got: 0%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.05
Quoted 7 lines

to:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.02
	Percent of CPU this job got: 96%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03

I have a question. You create 1000 namespaces in a single process and then free them. So I expect that single process is busy waiting for that kern_umount 1000 types, and waiting for 1000 synchronize_rcu's.

Does this ever show up in a real world work-load?

Is the cost of a single synchronize_rcu a problem?

The code you are working to avoid is this.

void kern_unmount(struct vfsmount *mnt)

{
	/* release long term mount so mount point can be released */
	if (!IS_ERR_OR_NULL(mnt)) {
		real_mount(mnt)->mnt_ns = NULL;
		synchronize_rcu();	/* yecchhh... */
		mntput(mnt);
	}
}

Which makes me wonder if perhaps there might be a simpler solution involving just that code. But I do realize such a solution would require analyzing all of the code after kern_unmount to see if any of it depends upon the synchronize_rcu.

In summary, I see no correctness problems with your code. Code that runs faster is always nice. In this case I just see the cost being shifted somewhere else not eliminated. I also see a slight increase in complexity.

So I am wondering if this was an exercise to speed up a toy benchmark or if this is an effort to speed of real world code.

At the very least some version of the motivation needs to be recorded so that the next time some one comes in an reworks the code they can look in the history and figure out what they need to do to avoid introducing a regeression.

Eric

Quoted 68 lines

---
v2:
- comment added in free_ipc_ns()

v1: https://lkml.org/lkml/2020/2/11/692

 include/linux/ipc_namespace.h |  2 ++
 ipc/namespace.c               | 20 ++++++++++++++++++--
 2 files changed, 20 insertions(+), 2 deletions(-)
include/linux/ipc_namespace.h +2 -0
index c309f43bde45..a06a78c67f19 100644
@@ -68,6 +68,8 @@ struct ipc_namespace {
 	struct user_namespace *user_ns;
 	struct ucounts *ucounts;
 
+	struct llist_node mnt_llist;
+
 	struct ns_common ns;
 } __randomize_layout;
 
ipc/namespace.c +18 -2
index b3ca1476ca51..7b9922244891 100644
@@ -117,6 +117,10 @@ void free_ipcs(struct ipc_namespace *ns, struct ipc_ids *ids,
 
 static void free_ipc_ns(struct ipc_namespace *ns)
 {
+	/* mq_put_mnt() waits for a grace period as kern_unmount()
+	 * uses synchronize_rcu().
+	 */
+	mq_put_mnt(ns);
 	sem_exit_ns(ns);
 	msg_exit_ns(ns);
 	shm_exit_ns(ns);
@@ -127,6 +131,17 @@ static void free_ipc_ns(struct ipc_namespace *ns)
 	kfree(ns);
 }
 
+static LLIST_HEAD(free_ipc_list);
+static void free_ipc(struct work_struct *unused)
+{
+	struct llist_node *node = llist_del_all(&free_ipc_list);
+	struct ipc_namespace *n, *t;
+
+	llist_for_each_entry_safe(n, t, node, mnt_llist)
+		free_ipc_ns(n);
+}
+static DECLARE_WORK(free_ipc_work, free_ipc);
+
 /*
  * put_ipc_ns - drop a reference to an ipc namespace.
  * @ns: the namespace to put
@@ -148,8 +163,9 @@ void put_ipc_ns(struct ipc_namespace *ns)
 	if (refcount_dec_and_lock(&ns->count, &mq_lock)) {
 		mq_clear_sbinfo(ns);
 		spin_unlock(&mq_lock);
-		mq_put_mnt(ns);
-		free_ipc_ns(ns);
+
+		if (llist_add(&ns->mnt_llist, &free_ipc_list))
+			schedule_work(&free_ipc_work);
 	}
 }
#3 lore raw
Quoted 35 lines · [email protected] (Eric W. Biederman) writes:
Quoted 25 lines · Giuseppe Scrivano

it avoids blocking on synchronize_rcu() in kern_umount().

the code:

\#define _GNU_SOURCE
\#include <sched.h>
\#include <error.h>
\#include <errno.h>
\#include <stdlib.h>
int main()
{
int i;
for (i = 0; i < 1000; i++)
if (unshare (CLONE_NEWIPC) < 0)
error (EXIT_FAILURE, errno, "unshare");
}

gets from:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.06
	Percent of CPU this job got: 0%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.05
Quoted 7 lines

to:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.02
	Percent of CPU this job got: 96%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03
Quoted 7 lines

I have a question. You create 1000 namespaces in a single process and then free them. So I expect that single process is busy waiting for that kern_umount 1000 types, and waiting for 1000 synchronize_rcu's.

Does this ever show up in a real world work-load?

Is the cost of a single synchronize_rcu a problem?

yes exactly, creating 1000 namespaces is not a real world use case (at least in my experience) but I've used it only to show the impact of the patch.

The cost of the single synchronize_rcu is the issue.

Most containers run in their own IPC namespace, so this is a constant cost for each container.

Quoted 18 lines

The code you are working to avoid is this.

void kern_unmount(struct vfsmount *mnt)

{
	/* release long term mount so mount point can be released */
	if (!IS_ERR_OR_NULL(mnt)) {
		real_mount(mnt)->mnt_ns = NULL;
		synchronize_rcu();	/* yecchhh... */
		mntput(mnt);
	}
}

Which makes me wonder if perhaps there might be a simpler solution involving just that code. But I do realize such a solution would require analyzing all of the code after kern_unmount to see if any of it depends upon the synchronize_rcu.

Quoted 7 lines

In summary, I see no correctness problems with your code. Code that runs faster is always nice. In this case I just see the cost being shifted somewhere else not eliminated. I also see a slight increase in complexity.

So I am wondering if this was an exercise to speed up a toy benchmark or if this is an effort to speed of real world code.

I've seen the issue while profiling real world work loads.

Quoted 4 lines

At the very least some version of the motivation needs to be recorded so that the next time some one comes in an reworks the code they can look in the history and figure out what they need to do to avoid introducing a regeression.

Is it enough in the git commit message or should it be an inline comment?

Thanks,
Giuseppe

#4 lore raw
Quoted 41 lines · Giuseppe Scrivano
Quoted 39 lines · [email protected] (Eric W. Biederman) writes:
Quoted 25 lines · Giuseppe Scrivano

it avoids blocking on synchronize_rcu() in kern_umount().

the code:

\#define _GNU_SOURCE
\#include <sched.h>
\#include <error.h>
\#include <errno.h>
\#include <stdlib.h>
int main()
{
int i;
for (i = 0; i < 1000; i++)
if (unshare (CLONE_NEWIPC) < 0)
error (EXIT_FAILURE, errno, "unshare");
}

gets from:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.06
	Percent of CPU this job got: 0%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.05
Quoted 7 lines

to:

	Command being timed: "./ipc-namespace"
	User time (seconds): 0.00
	System time (seconds): 0.02
	Percent of CPU this job got: 96%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03

I have a question. You create 1000 namespaces in a single process and then free them. So I expect that single process is busy waiting for that kern_umount 1000 types, and waiting for 1000 synchronize_rcu's.

Quoted 7 lines
Quoted 3 lines

Does this ever show up in a real world work-load?

Is the cost of a single synchronize_rcu a problem?

yes exactly, creating 1000 namespaces is not a real world use case (at least in my experience) but I've used it only to show the impact of the patch.

I know running 1000 containers is a real use case, and I would not be surprised if their are configurations that go higher.

Quoted 4 lines

The cost of the single synchronize_rcu is the issue.

Most containers run in their own IPC namespace, so this is a constant cost for each container.

Agreed.

Quoted 23 lines
Quoted 23 lines

The code you are working to avoid is this.

void kern_unmount(struct vfsmount *mnt)

{
	/* release long term mount so mount point can be released */
	if (!IS_ERR_OR_NULL(mnt)) {
		real_mount(mnt)->mnt_ns = NULL;
		synchronize_rcu();	/* yecchhh... */
		mntput(mnt);
	}
}

Which makes me wonder if perhaps there might be a simpler solution involving just that code. But I do realize such a solution would require analyzing all of the code after kern_unmount to see if any of it depends upon the synchronize_rcu.

In summary, I see no correctness problems with your code. Code that runs faster is always nice. In this case I just see the cost being shifted somewhere else not eliminated. I also see a slight increase in complexity.

Quoted 4 lines
Quoted 2 lines

So I am wondering if this was an exercise to speed up a toy benchmark or if this is an effort to speed of real world code.

I've seen the issue while profiling real world work loads.

So the question is how to remove this delay.

Quoted 7 lines
Quoted 4 lines

At the very least some version of the motivation needs to be recorded so that the next time some one comes in an reworks the code they can look in the history and figure out what they need to do to avoid introducing a regeression.

Is it enough in the git commit message or should it be an inline comment?

The git commit message should be enough to record the motivation.

A comment in the code that about the work queue that says something like "used to avoid the cost of synchronize_rcu in kern_unmount" would also be nice.

Eric