BCC: python: support fmod_ret
v2 cover BCC: python: support fmod_ret
Support fmod_ret, and add a useful tool for mptcp, also the interface can be verified with this function.
------
Changelog:
v2:
- change the name of the tool to 'mptcpify'
- fix the code style problems in mptcpify.py
- modify 'support_fmod_ret' in __init__.py, add some comment and ask for suggestions to BCC devs.
Hi Matt:
So nice to recieve your reply. Do you think we can create a PR with v2 patch to BCC now? The description of PR is attached below, Can you help me to take a look?
'''
Multipath TCP (MPTCP) serves as an enhancement to the conventional TCP protocol, enabling a single transport-layer connection to leverage multiple network interfaces. This capability makes MPTCP advantageous for applications requiring bandwidth consolidation, seamless failover mechanisms, and more robust connectivity solutions.
Linux kernel starts to support MPTCP since v5.6, and it provides a fmod_ret interface 'update_socket_protocol' to force applications using MPTCP instead of TCP without modifyiing its code.
So these patches provide a tool named 'mptcpify' which can achieve this. Using python-BCC is a more easy way for future development, so it is so important to support 'fmod_ret' in python-BCC.
@matttbe from the MPTCP kernel team had a look at the new tool. '''
Gang Yan (2):
BCC: Python: Support 'fmod_ret' method for eBPF
BCC: python: add a useful tool for mptcp
src/python/bcc/__init__.py | 34 ++++++++++++++++ tools/mptcpify.py | 80 ++++++++++++++++++++++++++++++++++++++ tools/mptcpify.txt | 22 +++++++++++ 3 files changed, 136 insertions(+) create mode 100644 tools/mptcpify.py create mode 100644 tools/mptcpify.txt
Signature · 1 line
2.25.1
PATCH v2 1/2 BCC: Python: Support 'fmod_ret' method for eBPF
In kernel, there exists a lot of 'fmod_ret' functions, such as 'update_socket_protocol'. But it cannot attached in BCC-python directly, so this patch provides an interface for python to use 'fmod_ret' attaching method.
But there exists a question about how to check 'fmod_ret' support in kernel, do you have any suggestion? I already considered checking the kernel version which seems not suitable.
--- src/python/bcc/__init__.py | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
src/python/bcc/__init__.py +34 -0
@@ -461,6 +461,7 @@ class BPF(object): self.raw_tracepoint_fds = {} self.kfunc_entry_fds = {} self.kfunc_exit_fds = {} + self.fmod_ret_fds = {} self.lsm_fds = {} self.perf_buffers = {} self.open_perf_events = {}
@@ -1157,6 +1158,12 @@ class BPF(object): return True return False + @staticmethod + def support_fmod_ret(): + # It is not clear what can be used to check if 'fmod_ret' + # is supported by the kernel. Assuming checking kfunc is enough + return BPF.support_kfunc() + def detach_kfunc(self, fn_name=b""): fn_name = _assert_is_bytes(fn_name) fn_name = BPF.add_prefix(b"kfunc__", fn_name)
@@ -1166,6 +1173,15 @@ class BPF(object): os.close(self.kfunc_entry_fds[fn_name]) del self.kfunc_entry_fds[fn_name] + def detach_fmod_ret(self, fn_name=b""): + fn_name = _assert_is_bytes(fn_name) + fn_name = BPF.add_prefix(b"kmod_ret__", fn_name) + + if fn_name not in self.fmod_ret_fds: + raise Exception("Fmod_ret func %s is not attached" % fn_name) + os.close(self.fmod_ret_fds[fn_name]) + del self.fmod_ret_fds[fn_name] + def detach_kretfunc(self, fn_name=b""): fn_name = _assert_is_bytes(fn_name) fn_name = BPF.add_prefix(b"kretfunc__", fn_name)
@@ -1189,6 +1205,22 @@ class BPF(object): self.kfunc_entry_fds[fn_name] = fd return self + def attach_fmod_ret(self, fn_name=b""): + fn_name = _assert_is_bytes(fn_name) + fn_name = BPF.add_prefix(b"kmod_ret__", fn_name) + + if fn_name in self.fmod_ret_fds: + raise Exception("Fmod_ret func %s has been attached" % fn_name) + + fn = self.load_func(fn_name, BPF.TRACING) + fd = lib.bpf_attach_kfunc(fn.fd) + + if fd < 0: + raise Exception("Failed to attach BPF to fmod_ret kernel func") + self.fmod_ret_fds[fn_name] = fd + + return self + def attach_kretfunc(self, fn_name=b""): fn_name = _assert_is_bytes(fn_name) fn_name = BPF.add_prefix(b"kretfunc__", fn_name)
@@ -1824,6 +1856,8 @@ class BPF(object): self.detach_kretfunc(k) for k, v in list(self.lsm_fds.items()): self.detach_lsm(k) + for k, v in list(self.fmod_ret_fds.items()): + self.detach_fmod_ret(k) # Clean up opened perf ring buffer and perf events table_keys = list(self.tables.keys())
Signature · 1 line
2.25.1
PATCH v2 2/2 BCC: python: add a useful tool for mptcp
In reply to Gang YanMultipath TCP (MPTCP) is an extension of the standard TCP protocol that allows a single transport connection to use multiple network interfaces or paths. MPTCP is useful for applications like bandwidth aggregation, failover, and more resilient connections.
Linux kernel starts to support MPTCP since v5.6, this patch provides a method which can easily force applications use MPTCP socket without modifing its code.
--- tools/mptcpify.py | 80 ++++++++++++++++++++++++++++++++++++++++++++++ tools/mptcpify.txt | 22 +++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 tools/mptcpify.py create mode 100644 tools/mptcpify.txt
tools/mptcpify.py new file +80 -0
@@ -0,0 +1,80 @@ +#!/usr/bin/env python +# +# mptcpify Make the applications to use MPTCP. +# For Linux, uses BCC, eBPF. Embedded C. +# +# USAGE: mptcpify -t +# +# Copyright 2025 Kylin Software, Inc. +# Licensed under the Apache License, Version 2.0 (the "License") +# +# 05-Apr-2025 Gang Yan Created this. + +import ctypes as ct +import argparse +import signal +import time + +from bcc import BPF + +#arguments +parser = argparse.ArgumentParser( + description="mptcpify try to force applications to use MPTCP instead of TCP") +parser.add_argument("-t", "--targets", required=True, type=str, + help="use ',' for multi targets, eg: 'iperf3,rsync'") + +args_str = parser.parse_args() +args_list = [t.strip() for t in args_str.targets.split(',')] + +if (not BPF.support_fmod_ret()): + print("Your kernel version is too old," + " fmod_ret method is only support kernel v5.7 and later.") + exit() + +TASK_COMM_LEN = 18 + +class app_name(ct.Structure): + _fields_ = [("str", ct.c_char * TASK_COMM_LEN)] + +# define BPF program +prog = """ +#include <linux/net.h> +#include <uapi/linux/in.h> +#include <linux/string.h> + +struct app_name { + char name[TASK_COMM_LEN]; +}; + +BPF_HASH(support_apps, struct app_name); + +KMOD_RET(update_socket_protocol, int family, int type, int protocol, int ret) +{ + struct app_name target = {}; + bpf_get_current_comm(&target.name, TASK_COMM_LEN); + + if ((family == AF_INET || family == AF_INET6) && + type == SOCK_STREAM && + (!protocol || protocol == IPPROTO_TCP) && + support_apps.lookup(&target)) + return IPPROTO_MPTCP; + + return protocol; + +} + +""" + +b = BPF(text=prog) +b.attach_fmod_ret("update_socket_protocol") + +support_apps = b.get_table("support_apps") +for i in args_list: + app = i.encode() + name = app_name() + name.str = app[:TASK_COMM_LEN-1].ljust(TASK_COMM_LEN, b'\0') + support_apps[name] = ct.c_uint32(1) + +print("MPTCP is been forced for ", args_list); +signal.pause() +
tools/mptcpify.txt new file +22 -0
@@ -0,0 +1,22 @@ +Demonstrations of mptcpify, the Linux eBPF/bcc version. + + +mptcpify forces the application to use to MPTCP instead of TCP. + +mptcpify has been verified with iperf3 and rsync[TCP module]. It can be used +for incresing the speed of transferring data with rsync. + +The MPTCP configuration is decribed in +https://www.mptcp.dev/pm.html + +USAGE message: + +usage: sudo python ./mptcpify.py [-h] -t TARGETS + +mptcpify try to force applications to use MPTCP instead of TCP + +options: + -h, --help show this help message and exit + -t TARGETS, --targets TARGETS + use ',' for multi targets, eg: 'iperf3,rsync' +
Signature · 1 line
2.25.1
Matthieu Baerts
maintainerHi Gang,
Quoted 11 lines · Gang Yan
Support
fmod_ret, and add a useful tool for mptcp, also the interface can be verified with this function.------
Changelog:
v2:
- change the name of the tool to 'mptcpify'
- fix the code style problems inmptcpify.py
- modify 'support_fmod_ret' in__init__.py, add some comment and ask for suggestions to BCC devs.
Quoted 5 lines
Hi Matt:
So nice to recieve your reply. Do you think we can create a PR with v2 patch to BCC now? The description of PR is attached below, Can you help me to take a look?
Thank you for the v2. I have just some small comments, but it is not blocking, the rest looks good to me.
Then yes, please create the PR, no need to send a v3.
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Matthieu Baerts
maintainerRe: [Patch, v2, 1/2] BCC: Python: Support 'fmod_ret' method for eBPF
In reply to Gang YanHi Gang,
Quoted 18 lines · Gang Yan
In kernel, there exists a lot of '
fmod_ret' functions, such as 'update_socket_protocol'. But it cannot attached in BCC-python directly, so this patch provides an interface for python to use 'fmod_ret' attaching method.But there exists a question about how to check '
fmod_ret' support in kernel, do you have any suggestion? I already considered checking the kernel version which seems not suitable.Signed-off-by Gang Yan <[email protected]>--- src/python/bcc/__init__.py | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)src/python/bcc/__init__.py +0
-0
(...)
src/python/bcc/__init__.py class BPF(object): · line 1213
Quoted 10 lines
src/python/bcc/__init__.py +6
-0@@ -1189,6 +1205,22 @@ class BPF(object): self.kfunc_entry_fds[fn_name] = fd return self + def attach_fmod_ret(self, fn_name=b""): + fn_name = _assert_is_bytes(fn_name) + fn_name = BPF.add_prefix(b"kmod_ret__", fn_name) + + if fn_name in self.fmod_ret_fds: + raise Exception("Fmod_ret func %s has been attached" % fn_name)
Detail: maybe clearer with this:
"Fmod_ret func %s is already attached"
(...)
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Matthieu Baerts
maintainerRe: [Patch, v2, 2/2] BCC: python: add a useful tool for mptcp
In reply to Gang YanHi Gang,
Quoted 37 lines · Gang Yan
Multipath TCP (MPTCP) is an extension of the standard TCP protocol that allows a single transport connection to use multiple network interfaces or paths. MPTCP is useful for applications like bandwidth aggregation, failover, and more resilient connections.
Linux kernel starts to support MPTCP since v5.6, this patch provides a method which can easily force applications use MPTCP socket without modifing its code.
Signed-off-by Gang Yan <[email protected]>--- tools/mptcpify.py | 80 ++++++++++++++++++++++++++++++++++++++++++++++ tools/mptcpify.txt | 22 +++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 tools/mptcpify.py create mode 100644 tools/mptcpify.txttools/mptcpify.py new file +14
-0@@ -0,0 +1,80 @@ +#!/usr/bin/env python +# +# mptcpify Make the applications to use MPTCP. +# For Linux, uses BCC, eBPF. Embedded C. +# +# USAGE: mptcpify -t +# +# Copyright 2025 Kylin Software, Inc. +# Licensed under the Apache License, Version 2.0 (the "License") +# +# 05-Apr-2025 Gang Yan Created this. + +import ctypes as ct +import argparse
tools/mptcpify.py line 24
Quoted 10 lines
tools/mptcpify.py new file +10
-0+import signal +import time + +from bcc import BPF + +#arguments +parser = argparse.ArgumentParser( + description="mptcpify try to force applications to use MPTCP instead of TCP") +parser.add_argument("-t", "--targets", required=True, type=str, + help="use ',' for multi targets, eg: 'iperf3,rsync'")
If it is easy for you to have this argument optional, don't hesitate to add this support: I think it will be useful to have this mode "everything is forced to use MPTCP".
(...)
tools/mptcpify.py line 78
Quoted 11 lines
tools/mptcpify.py new file +11
-0+b = BPF(text=prog) +b.attach_fmod_ret("update_socket_protocol") + +support_apps = b.get_table("support_apps") +for i in args_list: + app = i.encode() + name = app_name() + name.str = app[:TASK_COMM_LEN-1].ljust(TASK_COMM_LEN, b'\0') + support_apps[name] = ct.c_uint32(1) + +print("MPTCP is been forced for ", args_list);
Detail: maybe better with "is being forced".
Cheers,
Matt
Signature · 1 line
Sponsored by the NGI0 Core fund.
Re: [Patch, v2, 1/2] BCC: Python: Support 'fmod_ret' method for eBPF
In reply to Matthieu BaertsOn Wed, Apr 09, 2025 at 10:26:23AM +0200, Matthieu Baerts wrote:
Hi Matt,
Quoted 36 lines
Hi Gang,
Quoted 18 lines · Gang Yan
In kernel, there exists a lot of '
fmod_ret' functions, such as 'update_socket_protocol'. But it cannot attached in BCC-python directly, so this patch provides an interface for python to use 'fmod_ret' attaching method.But there exists a question about how to check '
fmod_ret' support in kernel, do you have any suggestion? I already considered checking the kernel version which seems not suitable.Signed-off-by Gang Yan <[email protected]>--- src/python/bcc/__init__.py | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)src/python/bcc/__init__.py +0
-0(...)
src/python/bcc/__init__.py class BPF(object): · line 1213
Quoted 10 lines
src/python/bcc/__init__.py +6
-0@@ -1189,6 +1205,22 @@ class BPF(object): self.kfunc_entry_fds[fn_name] = fd return self + def attach_fmod_ret(self, fn_name=b""): + fn_name = _assert_is_bytes(fn_name) + fn_name = BPF.add_prefix(b"kmod_ret__", fn_name) + + if fn_name in self.fmod_ret_fds: + raise Exception("Fmod_ret func %s has been attached" % fn_name)Detail: maybe clearer with this:
Quoted 4 lines
"
Fmod_retfunc %s is already attached"(...)
In BCC's attach_kfunc:
if fn_name in self.kfunc_entry_fds:
raise Exception("Kernel entry func %s has been attached" % fn_name)
So I think it's better to keep the same with it.
I agree with your view that mptcpify.py should, by default, enforce the use of MPTCP for all applications, and this feature will be incorporated before it is integrated into BCC tools.
The PR link:
https://github.com/iovisor/bcc/pull/5274
Thanks!
Best wishes!
Gang
Quoted 6 lines
Cheers,
MattSignature · 1 line
Sponsored by the NGI0 Core fund.