Skip to contentSkip to conversation

perf lock contention: Do not use BPF task local storage

PATCH Applied Namhyung Kim 4 participants 8 messages Nov 18, 2022 – Jan 10, 2023

Subsystems (2)
lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

PATCH 1/1 perf lock contention: Do not use BPF task local storage

It caused some troubles when a lock inside kmalloc is contended because task local storage would allocate memory using kmalloc. It'd create a recusion and even crash in my system.

There could be a couple of workarounds but I think the simplest one is to use a pre-allocated hash map. We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

---
 tools/perf/util/bpf_lock_contention.c         |  1 +
 .../perf/util/bpf_skel/lock_contention.bpf.c  | 34 ++++++++++++-------
 2 files changed, 23 insertions(+), 12 deletions(-)
tools/perf/util/bpf_lock_contention.c +1 -0
index 0deec1178778..4db9ad3d50c4 100644
@@ -39,6 +39,7 @@ int lock_contention_prepare(struct lock_contention *con)
 	bpf_map__set_value_size(skel->maps.stacks, con->max_stack * sizeof(u64));
 	bpf_map__set_max_entries(skel->maps.stacks, con->map_nr_entries);
 	bpf_map__set_max_entries(skel->maps.lock_stat, con->map_nr_entries);
+	bpf_map__set_max_entries(skel->maps.tstamp, con->map_nr_entries);
 
 	if (target__has_cpu(target))
 		ncpus = perf_cpu_map__nr(evlist->core.user_requested_cpus);
tools/perf/util/bpf_skel/lock_contention.bpf.c +22 -12
index 1bb8628e7c9f..9681cb59b0df 100644
@@ -40,10 +40,10 @@ struct {
 
 /* maintain timestamp at the beginning of contention */
 struct {
-	__uint(type, BPF_MAP_TYPE_TASK_STORAGE);
-	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__uint(type, BPF_MAP_TYPE_HASH);
 	__type(key, int);
 	__type(value, struct tstamp_data);
+	__uint(max_entries, MAX_ENTRIES);
 } tstamp SEC(".maps");
 
 /* actual lock contention statistics */
@@ -103,18 +103,28 @@ static inline int can_record(void)
 SEC("tp_btf/contention_begin")
 int contention_begin(u64 *ctx)
 {
-	struct task_struct *curr;
+	__u32 pid;
 	struct tstamp_data *pelem;
 
 	if (!enabled || !can_record())
 		return 0;
 
-	curr = bpf_get_current_task_btf();
-	pelem = bpf_task_storage_get(&tstamp, curr, NULL,
-				     BPF_LOCAL_STORAGE_GET_F_CREATE);
-	if (!pelem || pelem->lock)
+	pid = bpf_get_current_pid_tgid();
+	pelem = bpf_map_lookup_elem(&tstamp, &pid);
+	if (pelem && pelem->lock)
 		return 0;
 
+	if (pelem == NULL) {
+		struct tstamp_data zero = {};
+
+		bpf_map_update_elem(&tstamp, &pid, &zero, BPF_ANY);
+		pelem = bpf_map_lookup_elem(&tstamp, &pid);
+		if (pelem == NULL) {
+			lost++;
+			return 0;
+		}
+	}
+
 	pelem->timestamp = bpf_ktime_get_ns();
 	pelem->lock = (__u64)ctx[0];
 	pelem->flags = (__u32)ctx[1];
@@ -128,7 +138,7 @@ int contention_begin(u64 *ctx)
 SEC("tp_btf/contention_end")
 int contention_end(u64 *ctx)
 {
-	struct task_struct *curr;
+	__u32 pid;
 	struct tstamp_data *pelem;
 	struct contention_key key;
 	struct contention_data *data;
@@ -137,8 +147,8 @@ int contention_end(u64 *ctx)
 	if (!enabled)
 		return 0;
 
-	curr = bpf_get_current_task_btf();
-	pelem = bpf_task_storage_get(&tstamp, curr, NULL, 0);
+	pid = bpf_get_current_pid_tgid();
+	pelem = bpf_map_lookup_elem(&tstamp, &pid);
 	if (!pelem || pelem->lock != ctx[0])
 		return 0;
 
@@ -156,7 +166,7 @@ int contention_end(u64 *ctx)
 		};
 
 		bpf_map_update_elem(&lock_stat, &key, &first, BPF_NOEXIST);
-		pelem->lock = 0;
+		bpf_map_delete_elem(&tstamp, &pid);
 		return 0;
 	}
 
@@ -169,7 +179,7 @@ int contention_end(u64 *ctx)
 	if (data->min_time > duration)
 		data->min_time = duration;
 
-	pelem->lock = 0;
+	bpf_map_delete_elem(&tstamp, &pid);
 	return 0;
 }
 
Signature · 1 line
2.38.1.584.g0f3c55d4c2-goog
reviewer Acked-by on v1 1/1
#1 lore raw
Quoted 6 lines · Namhyung Kim

It caused some troubles when a lock inside kmalloc is contended because task local storage would allocate memory using kmalloc. It'd create a recusion and even crash in my system.

There could be a couple of workarounds but I think the simplest one is to use a pre-allocated hash map.

Quoted 2 lines

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path. I am also looking into this direction on using the BPF allocator.

maintainer Applied
#2 lore raw

Em Mon, Nov 21, 2022 at 09:32:56AM -0800, Martin KaFai Lau escreveu:

Quoted 9 lines · Namhyung Kim
Quoted 6 lines

It caused some troubles when a lock inside kmalloc is contended because task local storage would allocate memory using kmalloc. It'd create a recusion and even crash in my system.

There could be a couple of workarounds but I think the simplest one is to use a pre-allocated hash map.

Thanks, applied.

- Arnaldo

Quoted 5 lines
Quoted 2 lines

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path. I am also looking into this direction on using the BPF allocator.

Signature · 1 line
- Arnaldo
author maintainer
#3 lore raw
In reply to Martin KaFai Lau

Hello,

Quoted 7 lines · Martin KaFai Lau
Quoted 2 lines · Namhyung Kim

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path. I am also looking into this direction on using the BPF allocator.

Any progress on this? Are there any concerns about the change?

Thanks,
Namhyung

#4 lore raw
Quoted 4 lines · Namhyung Kim

Hello,

Quoted 1 line · Martin KaFai Lau
Quoted 8 lines
Quoted 6 lines · Namhyung Kim
Quoted 2 lines

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path. I am also looking into this direction on using the BPF allocator.

Any progress on this? Are there any concerns about the change?

Yep, I am working on it. It is not a direct replacement from kzalloc to bpf_mem_cache_alloc. eg. Some changes in the bpf mem allocator is needed to ensure the free list cannot be reused before the rcu grace period. There is a similar RFC patchset going into this direction that I am trying with.

author maintainer
#5 lore raw
Quoted 13 lines · Martin KaFai Lau
Quoted 11 lines · Namhyung Kim

Hello,

Quoted 7 lines · Martin KaFai Lau
Quoted 2 lines · Namhyung Kim

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path. I am also looking into this direction on using the BPF allocator.

Quoted 6 lines
Quoted 1 line

Any progress on this? Are there any concerns about the change?

Yep, I am working on it. It is not a direct replacement from kzalloc to bpf_mem_cache_alloc. eg. Some changes in the bpf mem allocator is needed to ensure the free list cannot be reused before the rcu grace period. There is a similar RFC patchset going into this direction that I am trying with.

I see. Thanks for the update!

Namhyung

#6 lore raw
In reply to Martin KaFai Lau

Hi Martin,

Quoted 12 lines · Martin KaFai Lau
Quoted 11 lines · Namhyung Kim

Hello,

Quoted 7 lines · Martin KaFai Lau
Quoted 2 lines · Namhyung Kim

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path.  I am also looking into this direction on using the BPF allocator.

Quoted 8 lines
Quoted 1 line

Any progress on this?  Are there any concerns about the change?

Yep, I am working on it. It is not a direct replacement from kzalloc to bpf_mem_cache_alloc. eg. Some changes in the bpf mem allocator is needed to ensure the free list cannot be reused before the rcu grace period. There is a similar RFC patchset going into this direction that I am trying with.

.

Do you mean "[RFC PATCH bpf-next 0/6] bpf: Handle reuse in bpf memory alloc" [0], right ? The main concern [1] for the proposal is the possibility of OOM will increase when RCU tasks trace grace period is slow, because the immediate reuse is disabled and the reuse is only possible after one RCU tasks trace grace period. Using a memory cgroup and setting a hard-limit on the cgroup may reduce the influence of the OOM problem, but it is not good enough. So do you have other ways to mitigate the potential OOM problem ?

[0]: https://lore.kernel.org/bpf/[email protected]/
[1]:
https://lore.kernel.org/bpf/CAADnVQ+z-Y6Yv2i-icAUy=Uyh9yiN4S1AOrLd=K8mu32TXORkw@mail.gmail.com/

#7 lore raw
Quoted 17 lines · Hou Tao

Hi Martin,

Quoted 14 lines · Martin KaFai Lau
Quoted 12 lines · Namhyung Kim

Hello,

Quoted 7 lines · Martin KaFai Lau
Quoted 2 lines · Namhyung Kim

We could fix the task local storage to use the safe BPF allocator, but it takes time so let's change this until it happens actually.

I also got another report on the kfree_rcu path.  I am also looking into this direction on using the BPF allocator.

Any progress on this?  Are there any concerns about the change?

Quoted 8 lines
Quoted 6 lines

Yep, I am working on it. It is not a direct replacement from kzalloc to bpf_mem_cache_alloc. eg. Some changes in the bpf mem allocator is needed to ensure the free list cannot be reused before the rcu grace period. There is a similar RFC patchset going into this direction that I am trying with.

.

Do you mean "[RFC PATCH bpf-next 0/6] bpf: Handle reuse in bpf memory alloc" [0], right ?

Yes, that is the RFC patch I was referring :). I was planning to comment after looking at the patch in details. I have shared some of my quick thoughts in that thread for the local storage usages.

Quoted 10 lines

The main concern [1] for the proposal is the possibility of OOM will increase when RCU tasks trace grace period is slow, because the immediate reuse is disabled and the reuse is only possible after one RCU tasks trace grace period. Using a memory cgroup and setting a hard-limit on the cgroup may reduce the influence of the OOM problem, but it is not good enough. So do you have other ways to mitigate the potential OOM problem ?

[0]: https://lore.kernel.org/bpf/[email protected]/
[1]:
https://lore.kernel.org/bpf/CAADnVQ+z-Y6Yv2i-icAUy=Uyh9yiN4S1AOrLd=K8mu32TXORkw@mail.gmail.com/