bpf: Mitigate latency spikes associated with freeing non-preallocated htab
PATCH 1/1 bpf: Mitigate latency spikes associated with freeing non-preallocated htab
Following the recent upgrade of one of our BPF programs, we encountered significant latency spikes affecting other applications running on the same host. After thorough investigation, we identified that these spikes were primarily caused by the prolonged duration required to free a non-preallocated htab with approximately 2 million keys.
Notably, our kernel configuration lacks the presence of CONFIG_PREEMPT. In scenarios where kernel execution extends excessively, other threads might be starved of CPU time, resulting in latency issues across the system. To mitigate this, we've adopted a proactive approach by incorporating cond_resched() calls within the kernel code. This ensures that during lengthy kernel operations, the scheduler is invoked periodically to provide opportunities for other threads to execute.
--- kernel/bpf/hashtab.c | 1 + 1 file changed, 1 insertion(+)
kernel/bpf/hashtab.c +1 -0
@@ -1489,6 +1489,7 @@ static void delete_all_elements(struct bpf_htab *htab)hlist_nulls_for_each_entry_safe (l, n, head, hash_node) {hlist_nulls_del_rcu (&l->hash_node);htab_elem_free (htab, l); +cond_resched (); } }migrate_enable ();
Signature · 1 line
2.30.1 (Apple Git-130)
Yonghong Song
reviewerQuoted 24 lines · Yafang Shao
Following the recent upgrade of one of our BPF programs, we encountered significant latency spikes affecting other applications running on the same host. After thorough investigation, we identified that these spikes were primarily caused by the prolonged duration required to free a non-preallocated htab with approximately 2 million keys.
Notably, our kernel configuration lacks the presence of
CONFIG_PREEMPT. In scenarios where kernel execution extends excessively, other threads might be starved of CPU time, resulting in latency issues across the system. To mitigate this, we've adopted a proactive approach by incorporatingcond_resched()calls within the kernel code. This ensures that during lengthy kernel operations, the scheduler is invoked periodically to provide opportunities for other threads to execute.Signed-off-by Yafang Shao <[email protected]>--- kernel/bpf/hashtab.c | 1 + 1 file changed, 1 insertion(+)kernel/bpf/hashtab.c +0
-0
kernel/bpf/hashtab.c static void delete_all_elements(struct bpf_htab *htab) · line 1493
Quoted 5 lines
kernel/bpf/hashtab.c +1
-0@@ -1489,6 +1489,7 @@ static void delete_all_elements(struct bpf_htab *htab)hlist_nulls_for_each_entry_safe (l, n, head, hash_node) {hlist_nulls_del_rcu (&l->hash_node);htab_elem_free (htab, l); +cond_resched (); }
should we put cond_resched() here inside the top 'for' loop, but outside the bucket loop? Do you really have a long link list for a particular bucket? Otherwise, the patch looks good to me. In hashtab.c, we have cond_resched() in some other places to mitigate similar issues.
Quoted 2 lines
kernel/bpf/hashtab.c +0
-0}migrate_enable ();
Quoted 22 lines · Yonghong Song
Quoted 19 lines · Yafang Shao
Following the recent upgrade of one of our BPF programs, we encountered significant latency spikes affecting other applications running on the same host. After thorough investigation, we identified that these spikes were primarily caused by the prolonged duration required to free a non-preallocated htab with approximately 2 million keys.
Notably, our kernel configuration lacks the presence of
CONFIG_PREEMPT. In scenarios where kernel execution extends excessively, other threads might be starved of CPU time, resulting in latency issues across the system. To mitigate this, we've adopted a proactive approach by incorporatingcond_resched()calls within the kernel code. This ensures that during lengthy kernel operations, the scheduler is invoked periodically to provide opportunities for other threads to execute.Signed-off-by Yafang Shao <[email protected]>--- kernel/bpf/hashtab.c | 1 + 1 file changed, 1 insertion(+)
Quoted 13 lines
kernel/bpf/hashtab.c static void delete_all_elements(struct bpf_htab *htab) · line 1493
Quoted 10 lines
kernel/bpf/hashtab.c +1
-0@@ -1489,6 +1489,7 @@ static void delete_all_elements(struct bpf_htab *htab)hlist_nulls_for_each_entry_safe (l, n, head, hash_node) {hlist_nulls_del_rcu (&l->hash_node);htab_elem_free (htab, l); +cond_resched (); }should we put
cond_resched()here inside the top 'for' loop, but outside the bucket loop? Do you really have a long link list for a particular bucket?
We haven't monitored a long list for a particular bucket. Will change it. Thanks for your suggestion.
Quoted 5 lines
Otherwise, the patch looks good to me. In
hashtab.c, we havecond_resched()in some other places to mitigate similar issues.Quoted 2 lines
kernel/bpf/hashtab.c +0
-0}migrate_enable ();
Signature · 2 lines
Regards Yafang