tcp: Add TCP ROCCET congestion control module.
Subsystems (4)
PATCH v5 1/1 tcp: Add TCP ROCCET congestion control module.
TCP ROCCET is an new congestion control algorithm based on TCP CUBIC that improves its overall performance in cellular networks. By its mode of function, CUBIC causes bufferbloat while it tries to detect the available throughput of a network path. This is particularly a problem with large buffers in mobile networks. A more detailed description and analysis of this problem caused by TCP CUBIC can be found in [1]. TCP ROCCET addresses the bufferbloat problem by adding two additional metrics to detect bufferbloat. The first metric is the relative increase in RTT from its minimum, the srRTT. Here, bufferbloat can be detected when RTTs increase due to buffer filling. The second metric is the acknowledgment arrival rate sampled over 100ms intervals. If CUBIC increases the send rate or congestion window, and the acknowledgment arrival rate stays on the same level, the connection is limited by the bottleneck link's capacity. In such cases, ROCCET reduces the send rate to prevent bufferbloat. ROCCET uses a modified version of slow start rather than HyStart because HyStart is known to enter the congestion avoidance phase too early when used in cellular networks [2]. Therefore, ROCCET uses a combination of srRTT and the acknowledgment arrival rate to determine when to exit slow start. For the congestion avoidance phase, ROCCET relies on the srRTT and monitoring the send and received Bytes to detect the filling of the bottleneck buffer.
In real-world mobile 5G NR measurements, TCP ROCCET achieves better performance than CUBIC and BBRv3, by maintaining similar throughput while reducing the latency. In stationary 5G NR scenarios, the performance is similar to that of BBRv3. More information about TCP ROCCET and measurement evaluations can be found here [3].
For the version (2026-07-21) we provide additional performance evaluation regarding throughput, latency and bandwidth share [4].
[1] https://doi.org/10.1109/VTC2023-Fall60731.2023.10333357
[2] https://doi.org/10.1109/WMNC.2016.7543932
[3] https://doi.org/10.23919/WONS68803.2026.11501781
[4] http://go.lu-h.de/roccet-2026-07-21
---
Changes since v1:
* Adjust some comments & Rework commit message
* Fix Kconfig format
* Add div-by-0 check variables
* Fix ack summation from +=1 to +=acked (counting now ACKs instead of ACK events)
* Fix wrapping in time comparisons
* Remove most module_params, including 'ignore_loss'
* Remove check that always evaluated to 'true' regarding 'bw_limit_detect'
Changes since v2:
* Improve initialization of roccettcp struct
* Always react to ECE bits and reset flag
* Detect plateaus in ACK-rate
* Fix potential overflow in ACK-rate tracking for high-speed connections
* Reset ACK-counter correctly on new interval
* Fix potential integer overflow in rRTT calculation
Changes since v3:
* Changes in comments & Kconfig help text
* Adjust commit message, adding reference to new performance evaluation
* Add minimum RTT probing
* Use cong_control callback instead of cong_avoid
* Move CA_EVENT_TX_START event check to dedicated callback (cwnd_event_tx_start)
* Add monitoring of send and receive rate as mentioned in the original paper
* Refactor of the code
* Remove kfunc exports
* Fix potential overflow in rRTT calculation by using 64-bit arithmetic
* Fix potential division-by-zero in srRTT calculation due to RTT value
* Initialize ack_rate timestamp to current time
* Fix potential double penalty when receiving ECN (ECE) mark
* Reset ack counting when connection goes idle
* Add module-parameter bounds check on register
Changes since v4:
* Remove __always_inline annotations & unused function
* Fix module parameter bounds check desync
* Add idle-check to fix connection freezing
* Exchange .ssthresh callback from recalc_ssthresh to a ssthresh-getter, fixing double penalties
* Remove unnecessary seq-no wrapping check & Remove 0 as 'uninitialized' seq-no value
* Wait for first control round to finish before starting to monitor potential slow-start exit conditions
--- net/ipv4/Kconfig | 12 + net/ipv4/Makefile | 1 + net/ipv4/tcp_roccet.c | 1050 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 1063 insertions(+) create mode 100644 net/ipv4/tcp_roccet.c
net/ipv4/Kconfig +12 -0
@@ -663,6 +663,18 @@ config TCP_CONG_CDG delay gradients." In Networking 2011. Preprint: http://caia.swin.edu.au/cv/dahayes/content/networking2011-cdg-preprint.pdf +config TCP_CONG_ROCCET + tristate "ROCCET TCP" + default n + help + TCP ROCCET (RTT oriented CUBIC congestion control exTension) is a + sender-side-only congestion control algorithm based on the TCP CUBIC + protocol stack/TCP CUBIC congestion control algorithm that optimizes + its performance. Especially for networks with large buffers + (wireless, cellular networks), TCP ROCCET has improved performance by + maintaining a similar throughput as CUBIC while reducing latency. + For more information, see: https://arxiv.org/abs/2510.25281 + config TCP_CONG_BBR tristate "BBR TCP" default n
net/ipv4/Makefile +1 -0
@@ -45,6 +45,7 @@ obj-$(CONFIG_INET_TCP_DIAG) += tcp_diag.o obj-$(CONFIG_INET_UDP_DIAG) += udp_diag.o obj-$(CONFIG_INET_RAW_DIAG) += raw_diag.o obj-$(CONFIG_TCP_CONG_BBR) += tcp_bbr.o +obj-$(CONFIG_TCP_CONG_ROCCET) += tcp_roccet.o obj-$(CONFIG_TCP_CONG_BIC) += tcp_bic.o obj-$(CONFIG_TCP_CONG_CDG) += tcp_cdg.o obj-$(CONFIG_TCP_CONG_CUBIC) += tcp_cubic.o
net/ipv4/tcp_roccet.c new file +1050 -0
1,050 lines not on this page · raw message
base-commit: 1bb784eb6e38fd73143f021608e4ef3095d0c0d7
Signature · 1 line
2.43.0
Quoted 31 lines · Tim Fuechsel
TCP ROCCET is an new congestion control algorithm based on TCP CUBIC that improves its overall performance in cellular networks. By its mode of function, CUBIC causes bufferbloat while it tries to detect the available throughput of a network path. This is particularly a problem with large buffers in mobile networks. A more detailed description and analysis of this problem caused by TCP CUBIC can be found in [1]. TCP ROCCET addresses the bufferbloat problem by adding two additional metrics to detect bufferbloat. The first metric is the relative increase in RTT from its minimum, the srRTT. Here, bufferbloat can be detected when RTTs increase due to buffer filling. The second metric is the acknowledgment arrival rate sampled over 100ms intervals. If CUBIC increases the send rate or congestion window, and the acknowledgment arrival rate stays on the same level, the connection is limited by the bottleneck link's capacity. In such cases, ROCCET reduces the send rate to prevent bufferbloat. ROCCET uses a modified version of slow start rather than HyStart because HyStart is known to enter the congestion avoidance phase too early when used in cellular networks [2]. Therefore, ROCCET uses a combination of srRTT and the acknowledgment arrival rate to determine when to exit slow start. For the congestion avoidance phase, ROCCET relies on the srRTT and monitoring the send and received Bytes to detect the filling of the bottleneck buffer.
In real-world mobile 5G NR measurements, TCP ROCCET achieves better performance than CUBIC and BBRv3, by maintaining similar throughput while reducing the latency. In stationary 5G NR scenarios, the performance is similar to that of BBRv3. More information about TCP ROCCET and measurement evaluations can be found here [3].
For the version (2026-07-21) we provide additional performance evaluation regarding throughput, latency and bandwidth share [4].
Quoted 7 lines
[1] https://doi.org/10.1109/VTC2023-Fall60731.2023.10333357
[2] https://doi.org/10.1109/WMNC.2016.7543932
[3] https://doi.org/10.23919/WONS68803.2026.11501781
[4] http://go.lu-h.de/roccet-2026-07-21Signed-off-by Lukas Prause <[email protected]>Signed-off-by Tim Fuechsel <[email protected]>
Hi Lukas, Tim,
Here is a AI review of your patch. There are several architectural issues, race conditions, and logic bugs that need to be addressed.
1. Concurrency data race on module parameters in socket init
------------------------------------------------------------
In roccettcp_init():
param_check(true);
param_precompute();Both functions mutate global static variables (beta, bic_scale, beta_scale, cube_rtt_scale, cube_factor) that are marked __read_mostly. Calling this in roccettcp_init() on every socket creation causes unsynchronized concurrent writes across CPUs. Furthermore, param_check() calls pr_err() / pr_info(), which will spam the kernel log on every connection creation if an invalid parameter was configured.
Global scale factors should only be precomputed at module init time (roccettcp_register), or validated using kernel_param_ops.
2. Bypassing PRR and broken loss recovery via .cong_control
----------------------------------------------------------
By defining .cong_control = roccet_control, TCP core completely delegates congestion control and skips PRR (tcp_cwnd_reduction()).
However, in roccet_control():
if (!ca->is_in_recovery)
roccettcp_cong_avoid(sk, ack, rs->acked_sacked);During recovery (TCP_CA_Recovery), ROCCET does nothing to regulate cwnd or clock out retransmissions according to RFC 6937. Unless you have a full custom recovery engine (like BBR), you should stick to .cong_avoid so the stack handles PRR and loss recovery properly.
3. Broken .ssthresh callback and ignored ECN (TCP_CA_CWR)
---------------------------------------------------------roccettcp_ssthresh() simply returns tp->snd_ssthresh without recalculation:
static u32 roccettcp_ssthresh(struct sock *sk)
{
return tcp_sk(sk)->snd_ssthresh;
}The recalculation was moved to roccettcp_state(..., TCP_CA_Recovery). This breaks the TCP stack contract:
a) tcp_init_cwnd_reduction() initializes PRR state using the return value of icsk_ca_ops->ssthresh(sk). Returning the unreduced ssthresh corrupts PRR.
b) When ECN CE marks arrive, tcp_enter_cwr() transitions to TCP_CA_CWR and calls icsk_ca_ops->ssthresh(sk). Since roccettcp_state() only checks for Loss and Recovery, TCP_CA_CWR is completely ignored. Consequently, neither ssthresh nor cwnd is ever reduced on ECN.
4. Congestion avoidance starvation in ORBITER
---------------------------------------------
In roccettcp_cong_avoid():
send = tp->snd_nxt - ca->interval_snd_seq_start;
received = tp->snd_una - ca->interval_una_seq_start;
send_more_than_acked =
send > received + ((tcp_snd_cwnd(tp) * tp->mss_cache) / 100);
...
if (!tcp_is_cwnd_limited(sk) || send_more_than_acked)
return;During steady-state transmission, the in-flight byte count (send - received) is approximately cwnd * mss. This is ~100x larger than the 1% guard space ((cwnd * mss) / 100). Thus, send_more_than_acked evaluates to true on almost every ACK, returning before bictcp_update() or tcp_cong_avoid_ai() can run. Window growth in congestion avoidance is completely starved.
5. curr_min_rtt poisoning and premature slow-start exit
-------------------------------------------------------
In roccettcp_reset(), ca->curr_rtt is 0 and ca->curr_min_rtt is ~0U. In update_min_rtt():
if (ca->curr_rtt < ca->curr_min_rtt) {
ca->curr_min_rtt = max(ca->curr_rtt, 1);
...If an ACK arrives before an RTT sample is collected (e.g. sample->rtt_us <= 0), ca->curr_rtt is 0. ca->curr_min_rtt is set to max(0, 1) = 1 us. Because 1 us is smaller than any real path RTT, ca->curr_min_rtt remains stuck at 1 us.
Then in update_srrtt():
u32 rrtt = div_u64(100 * (u64)(ca->curr_rtt - ca->curr_min_rtt),
ca->curr_min_rtt);For a 20 ms RTT (20000 us), rrtt evaluates to 1,999,900. curr_srrtt immediately exceeds sr_rtt_upper_bound and terminates slow start on the very first sample.
6. Perpetual min-RTT probe loop
-------------------------------
In roccet_min_rtt_probe(), when the probe finishes:
ca->probe_min_rtt_until = 0;
ca->state = ORBITER;ca->next_min_rtt_probe is not updated when exiting the probe. If no new lower RTT was found during the probe, next_min_rtt_probe remains in the past. On the next ACK, after(now, ca->next_min_rtt_probe) immediately evaluates to true, trapping the flow in a continuous probe loop and repeatedly halving cwnd down to TCP_INIT_CWND.
7. 71-minute timestamp wrap and permanent DRAIN state
-----------------------------------------------------
now is calculated via jiffies_to_usecs(tcp_jiffies32), which wraps every ~71.5 minutes (2^32 us).
In roccet_control():
else if ((s32)now - ca->roccet_last_event_time_us <= 100 * USEC_PER_MSEC)
ca->state = DRAIN;ca->roccet_last_event_time_us is initialized to 0. When bit 31 of now is set (for 35.7 minutes out of every 71.5 minutes), (s32)now is negative, so (s32)now - 0 is negative and always <= 100000. Newly created connections during that 35-minute window will immediately enter and remain permanently stuck in DRAIN state.
Also, before() and after() from <net/tcp.h> are sequence number comparison macros (32-bit TCP seqno), not time comparison macros.
8. Re-implementing tcp_update_pacing_rate()
-------------------------------------------roccet_control() duplicates the pacing rate calculation from tcp_input.c and hardcodes 100% pacing instead of honoring sysctl_tcp_pacing_ca_ratio. If you switch to .cong_avoid, the core stack will manage pacing updates automatically.
9. Coding style & formatting
----------------------------
- Declarations after statements (ISO C90 violations) in roccettcp_init(), update_srrtt(), roccettcp_acked(), and roccettcp_register().
- C++ style comments (//) instead of /* ... */ in multiple places.
- Commented-out dead code: //WRITE_ONCE(sk->sk_pacing_rate, 0);
- Typos in comments ("sRrTT", "probeing", "has was idle", "defer", etc.) and references to non-existent struct fields ("curr_min_rtt_timed.rtt").
Thanks.
Hi Eric,
thank you for the review. On a few points, I’d have some comments. We think that this AI review is, at some point, a little bit misleading. We are using the cong_control functionality to implement stricter packet pacing than the default value.
With this, we aim to implement our own alternative to PRR.
Quoted 7 lines
1. Concurrency data race on module parameters in socket init
------------------------------------------------------------
Inroccettcp_init():param_check(true); param_precompute();Both functions mutate global static variables (beta,
bic_scale,
beta_scale,
Quoted 2 lines
cube_rtt_scale,cube_factor) that are marked__read_mostly. Calling this inroccettcp_init()on every socket creation causes
unsynchronized
Quoted 2 lines
concurrent writes across CPUs. Furthermore,
param_check()callspr_err()/pr_info(), which will spam the kernel log on every connection creation
if an
Quoted 4 lines
invalid parameter was configured.
Global scale factors should only be precomputed at module init time (
roccettcp_register), or validated usingkernel_param_ops.
I agree with you. We will fix that.
Quoted 5 lines
2. Bypassing PRR and broken loss recovery via .cong_control
----------------------------------------------------------
By defining .cong_control =roccet_control, TCP core completely delegates congestion control and skips PRR (tcp_cwnd_reduction()).
Quoted 8 lines
However, in
roccet_control():if (!ca->is_in_recovery) roccettcp_cong_avoid(sk, ack, rs->acked_sacked);During recovery (
TCP_CA_Recovery), ROCCET does nothing to regulate cwnd or clock out retransmissions according to RFC 6937. Unless you have a full custom recovery engine (like BBR), you should stick to .cong_avoid so the stack handles PRR and loss recovery properly.
If we are in recovery we still do packet pacing to 100 percent of the cwnd instead of 120.
Quoted 3 lines
3. Broken .ssthresh callback and ignored ECN (
TCP_CA_CWR)
---------------------------------------------------------roccettcp_ssthresh()simply returnstp->snd_ssthreshwithout
recalculation:
Quoted 8 lines
static u32 roccettcp_ssthresh(struct sock *sk) { return tcp_sk(sk)->snd_ssthresh; }The recalculation was moved to
roccettcp_state(..., TCP_CA_Recovery). This breaks the TCP stack contract:
a)tcp_init_cwnd_reduction()initializes PRR state using the return
value of
Quoted 1 line
icsk_ca_ops->ssthresh(sk). Returning the unreduced ssthresh
corrupts PRR.
Quoted 1 line
b) When ECN CE marks arrive,
tcp_enter_cwr()transitions toTCP_CA_CWR
and calls
Quoted 1 line
icsk_ca_ops->ssthresh(sk). Since roccettcp_state() only checks for
Loss and
Quoted 1 line
Recovery,
TCP_CA_CWRis completely ignored. Consequently, neither
ssthresh
Quoted 1 line
nor cwnd is ever reduced on ECN.
We are not using the PRR from the TCP stack, because we are implementing cong_control.
I agree that ECNs are ignored here. We will fix that.
Quoted 11 lines
4. Congestion avoidance starvation in ORBITER
---------------------------------------------
Inroccettcp_cong_avoid():send = tp->snd_nxt - ca->interval_snd_seq_start; received = tp->snd_una - ca->interval_una_seq_start; send_more_than_acked = send > received + ((tcp_snd_cwnd(tp) * tp->mss_cache) / 100); ... if (!tcp_is_cwnd_limited(sk) || send_more_than_acked) return;
Quoted 1 line
During steady-state transmission, the in-flight byte count (send -
received)
Quoted 2 lines
is approximately cwnd * mss. This is ~100x larger than the 1% guard space ((cwnd * mss) / 100). Thus,
send_more_than_ackedevaluates to true on
almost
Quoted 1 line
every ACK, returning before
bictcp_update()ortcp_cong_avoid_ai()can
run.
Quoted 1 line
Window growth in congestion avoidance is completely starved.
That is not true. Here we are monitoring the bytes sent and received over a time interval of 5 RTT
Quoted 9 lines
5.
curr_min_rttpoisoning and premature slow-start exit
-------------------------------------------------------
Inroccettcp_reset(),ca->curr_rttis 0 andca->curr_min_rttis ~0U. Inupdate_min_rtt():if (ca->curr_rtt < ca->curr_min_rtt) { ca->curr_min_rtt = max(ca->curr_rtt, 1); ...If an ACK arrives before an RTT sample is collected (e.g.
sample->rtt_us <= 0),
Quoted 2 lines
ca->curr_rttis 0.ca->curr_min_rttis set tomax(0, 1)= 1 us. Because 1 us is smaller than any real path RTT,ca->curr_min_rtt
remains stuck
Quoted 5 lines
at 1 us.
Then inupdate_srrtt():u32 rrtt = div_u64(100 * (u64)(ca->curr_rtt - ca->curr_min_rtt), ca->curr_min_rtt);For a 20 ms RTT (20000 us), rrtt evaluates to 1,999,900.
curr_srrtt
immediately
Quoted 1 line
exceeds
sr_rtt_upper_boundand terminates slow start on the very first
sample.
That could be a problem, and we will implement a fix for it.
Quoted 7 lines
6. Perpetual min-RTT probe loop
-------------------------------
Inroccet_min_rtt_probe(), when the probe finishes:ca->probe_min_rtt_until = 0; ca->state = ORBITER;
ca->next_min_rtt_probeis not updated when exiting the probe. If no
new lower
Quoted 2 lines
RTT was found during the probe,
next_min_rtt_proberemains in the past. On the next ACK,after(now, ca->next_min_rtt_probe)immediately
evaluates to
Quoted 1 line
true, trapping the flow in a continuous probe loop and repeatedly
halving cwnd
Quoted 1 line
down to
TCP_INIT_CWND.
That is not true. Every time a new minimum RTT is found, we calculate ca->next_min_rtt_probe, and before we enter probing, the current minimum RTT is set to u32 max.
This means the next RTT is the new minimum RTT.
Quoted 7 lines
7. 71-minute timestamp wrap and permanent DRAIN state
-----------------------------------------------------
now is calculated viajiffies_to_usecs(tcp_jiffies32), which wraps every ~71.5 minutes (2^32 us).In
roccet_control():else if ((s32)now - ca->roccet_last_event_time_us <= 100 *
USEC_PER_MSEC)
Quoted 3 lines
ca->state = DRAIN;
ca->roccet_last_event_time_usis initialized to 0. When bit 31 of now
is set
Quoted 2 lines
(for 35.7 minutes out of every 71.5 minutes), (s32)now is negative, so (s32)now - 0 is negative and always <= 100000. Newly created
connections during
Quoted 1 line
that 35-minute window will immediately enter and remain permanently
stuck in
Quoted 4 lines
DRAIN state.
Also,
before()andafter()from <net/tcp.h> are sequence number comparison macros (32-bit TCP seqno), not time comparison macros.
This is a problem, and we will fix it.
In the past, we got a hint that we should use the before and after macros here.
Quoted 3 lines
8. Re-implementing
tcp_update_pacing_rate()
-------------------------------------------roccet_control()duplicates the pacing rate calculation from
tcp_input.c and
Quoted 1 line
hardcodes 100% pacing instead of honoring
sysctl_tcp_pacing_ca_ratio.
If you
Quoted 1 line
switch to .cong_avoid, the core stack will manage pacing updates
automatically.
We do this as an alternative to PRR with stricter pacing.
Quoted 8 lines
9. Coding style & formatting
----------------------------
- Declarations after statements (ISO C90 violations) inroccettcp_init(),update_srrtt(),roccettcp_acked(), androccettcp_register().
- C++ style comments (//) instead of /* ... */ in multiple places.
- Commented-out dead code: //WRITE_ONCE(sk->sk_pacing_rate, 0);
- Typos in comments ("sRrTT", "probeing", "has was idle", "defer", etc.) and references to non-existent struct fields ("curr_min_rtt_timed.rtt").
Of course, we will correct that.
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider: - [Critical] Concurrent socket initialization mutates global configuration parameters without…
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
- [Critical] Concurrent socket initialization mutates global configuration parameters without synchronization, leading to severe data races and corruption of cube_factor.
- [Critical] The curr_rtt field is implicitly initialized to 0, which permanently corrupts the curr_min_rtt calculation if evaluated before a valid RTT sample is received.
- [High] The .ssthresh callback erroneously returns the unmodified threshold, bypassing TCP's standard Retransmission Timeout (RTO) backoff and causing congestion floods.
- [High] ROCCET completely fails to reduce its congestion window upon receiving Explicit Congestion Notification (ECN) CE marks, violating RFC 3168.
- [High] ROCCET bypasses and breaks TCP's Proportional Rate Reduction (PRR) fast recovery by blindly overriding cwnd assignments during TCP_CA_Recovery.
- [High] Fast convergence logic incorrectly calculates last_max_cwnd using the artificially slashed probe window instead of the true network capacity.
- [Medium] Integer truncation of the acknowledged packet count allows attackers to spoof a low ACK rate and bypass congestion control.
--
commit 8c2a501e71c2cbc59259d9d2e53dda94d869cfdb
Author: Tim Fuechsel <[email protected]>
Subject: tcp: Add TCP ROCCET congestion control module.
This commit adds TCP ROCCET, a new congestion control algorithm based on TCP CUBIC. It aims to improve overall performance and reduce bufferbloat in cellular networks by utilizing srRTT and acknowledgment arrival rates.
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +0
-0
[ ... ]
Quoted 5 lines
net/ipv4/tcp_roccet.c new file +5
-0+static void roccettcp_reset (struct roccettcp *ca) +{ +memset (ca,0 ,sizeof (struct roccettcp )); + ca->next_srrtt_check =0 ; + ca->curr_min_rtt = ~0U ;
[ ... ]
net/ipv4/tcp_roccet.c line 197
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+ ca->initial_round_completed = false; +} + +static void update_min_rtt (struct sock *sk) +{ +struct roccettcp *ca =inet_csk_ca (sk); + +/* Check if new lower min RTT was found. If so, set it directly */ +if (ca->curr_rtt < ca->curr_min_rtt) { + ca->curr_min_rtt =max (ca->curr_rtt,1 );
[Severity: Critical]
Does this initialization logic safely handle early ACKs without valid RTT samples?
roccettcp_reset() initializes the struct with zeroes, making curr_rtt = 0 and curr_min_rtt = ~0U. If an early ACK lacks a valid RTT sample (for example, a pure duplicate ACK where sample->rtt_us < 0), curr_rtt remains
0. The comparison 0 < ~0U will succeed in update_min_rtt(), locking curr_min_rtt to 1. Future valid RTTs will never be smaller than 1.
[ ... ]
net/ipv4/tcp_roccet.c line 250
Quoted 7 lines
net/ipv4/tcp_roccet.c new file +7
-0+ }else { + ca->ack_rate_last_rate_time = now; + ca->ack_rate_last_rate = ca->ack_rate_curr_rate; + ca->ack_rate_curr_rate = ca->ack_rate_cnt; + ca->ack_rate_cnt = + acked;// start counting for the new interval + }
[Severity: Medium]
Could this direct assignment truncate the 32-bit acked parameter?
The struct definition restricts ack_rate_cnt to only 16 bits. Without a bounding check like min_t(u32, acked, U16_MAX) here, a malicious receiver utilizing large windows or LRO/GRO aggregation could acknowledge more than 65535 packets at once. This would cause the counter to silently wrap around and spoof a low ACK rate.
[ ... ]
Quoted 2 lines
net/ipv4/tcp_roccet.c new file +2
-0+static void roccet_min_rtt_probe (struct sock *sk,u32 now) +{
[ ... ]
net/ipv4/tcp_roccet.c line 374
Quoted 7 lines
net/ipv4/tcp_roccet.c new file +7
-0+ }else if (before (now, ca->refill_until)) { +/* Reset cwnd and refill the pipe. */ +if (ca->state !=RTT_PROBE_REFILL ) { +tcp_snd_cwnd_set (tp, ca->cwnd_before_min_rtt_probe); +tcp_sk (sk)->snd_ssthresh =tcp_snd_cwnd (tp); + ca->state =RTT_PROBE_REFILL ; + }
[Severity: High]
What happens if this min_rtt probe timer expires while the connection is mid-recovery?
Abruptly inflating the cwnd back to the pre-probe value during fast recovery would bypass the PRR pacing mechanism and inflate the window.
[ ... ]
Quoted 2 lines
net/ipv4/tcp_roccet.c new file +2
-0+static void param_precompute (void ) +{
[ ... ]
net/ipv4/tcp_roccet.c line 463
Quoted 11 lines
net/ipv4/tcp_roccet.c new file +11
-0+/* divide by bic_scale and by constant Srtt (100ms) */ +do_div (cube_factor, bic_scale *10 ); +} + +static void roccettcp_init (struct sock *sk) +{ +/* Check & precompute on `init` in order to use the newest +* available params. +*/ +param_check (true); +param_precompute ();
[Severity: Critical]
Does calling param_precompute() per socket initialization create a data race on the global cube_factor variable?
Since roccettcp_init() is invoked dynamically for every newly created socket, the unprotected in-place division do_div(cube_factor, ...) mutates shared global state. If concurrent sockets are created, threads will subject the variable to multiple divisions, resulting in a miscalculation of the cwnd target in bictcp_update().
[ ... ]
net/ipv4/tcp_roccet.c line 798
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +4
-0+static u32 roccettcp_ssthresh (struct sock *sk) +{ +return tcp_sk (sk)->snd_ssthresh; +}
[Severity: High]
Does returning the unmodified snd_ssthresh bypass TCP's required threshold reductions?
When a severe congestion event like an RTO timeout occurs, or during an ECN CE-mark response in TCP_CA_CWR, the core stack queries the .ssthresh callback to shrink the slow-start threshold. Returning the current threshold unmodified prevents PRR from pacing properly and allows the window to exponentially grow back to massive sizes.
[ ... ]
Quoted 2 lines
net/ipv4/tcp_roccet.c new file +2
-0+static u32 roccettcp_recalc_ssthresh (struct sock *sk) +{
[ ... ]
Quoted 7 lines
net/ipv4/tcp_roccet.c new file +7
-0+if (ca->state ==RTT_PROBE ) { +/* Handle ECN as cubic congestion event in min +* RTT probe. +*/ + ca->ece_received = false; + + ca->epoch_start =0 ;/* end of epoch */
net/ipv4/tcp_roccet.c line 836
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+ +/* Wmax and fast convergence */ +if (cwnd < ca->last_max_cwnd && fast_convergence) + ca->last_max_cwnd = + (cwnd * (BICTCP_BETA_SCALE + beta)) / + (2 *BICTCP_BETA_SCALE ); +else + ca->last_max_cwnd = cwnd; + + cwnd = ca->cwnd_before_min_rtt_probe;
[Severity: High]
Is this fast convergence calculation using the correct window size?
The active cwnd is artificially halved during the min_rtt probe. By calculating the new last_max_cwnd using this slashed probe window before restoring the true cwnd, the historical maximum window (which anchors the CUBIC curve origin) would become permanently understated.
[ ... ]
net/ipv4/tcp_roccet.c line 848
Quoted 6 lines
net/ipv4/tcp_roccet.c new file +6
-0+/* Handle ECN as ROCCET congestion event. */ +if (ca->ece_received) { + ca->ece_received = false; +roccet_congestion_event (sk,jiffies_to_usecs (tcp_jiffies32)); +return tcp_snd_cwnd (tp); + }
[Severity: High]
Is this ECN handling code unreachable?
This intended ECN handling code sits inside roccettcp_recalc_ssthresh(), which appears to only be invoked by roccettcp_state() during TCP_CA_Recovery. Since ECN marks normally trigger TCP_CA_CWR (which roccettcp_state() ignores), this logic will never execute during actual ECN events, violating protocol requirements.
[ ... ]
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+static void roccettcp_state (struct sock *sk,u8 new_state) +{ +struct roccettcp *ca =inet_csk_ca (sk); +struct tcp_sock *tp =tcp_sk (sk); + + ca->is_in_recovery = false; + +if (new_state == TCP_CA_Loss) { +roccettcp_reset (ca); + }else if (new_state == TCP_CA_Recovery) {
net/ipv4/tcp_roccet.c line 891
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+ ca->is_in_recovery = true; + +/* Here we set the cwnd and ssthresh to the same value so +* the TCP state machine knows we are in cong. avoid and +* not in slow start. +*/ +tcp_sk (sk)->snd_ssthresh =roccettcp_recalc_ssthresh (sk); +tcp_snd_cwnd_set (tp,tcp_sk (sk)->snd_ssthresh); + } +}
[Severity: High]
Does instantly setting the cwnd here bypass TCP's Proportional Rate Reduction (PRR) fast recovery?
Blindly overriding the cwnd assignment upon entering TCP_CA_Recovery instantly shrinks the window and destroys the core stack's PRR pacing mechanism, which leads to retransmission bursts.
Additionally, the TCP_CA_Loss branch misses the manual ssthresh reduction applied to Recovery, cementing the RTO backoff bypass.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/apWz9rPcGMgYCKOR@volt-roccet-vm?part=1
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 21 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 21 potential issue(s) to consider:
Critical: 0 · High: 5 · Medium: 8 · Low: 8
- [High] ROCCET performs no congestion response to ECN.
- [High] ROCCET removes the standard congestion response to loss/RTO, and the commit message does not disclose it.
- [High] update_min_rtt() (net/ipv4/tcp_roccet.c:191-206) treats an absent RTT sample (ca->curr_rtt == 0) as a valid measurement and…
- [High] param_check() only rejects beta_param outside (0, 1024).
- [High] roccettcp_init() (net/ipv4/tcp_roccet.c:457-463) — called for every new TCP socket that selects ROCCET, i.e. by any unprivileged…
- [Medium] param_check(true) is invoked from roccettcp_init() (net/ipv4/tcp_roccet.c:462) for every new socket, and it emits…
- [Medium] The writable beta_param interface (net/ipv4/tcp_roccet.c:144, module_param(beta_param, int, 0644) at…
- [Medium] roccettcp_reset() (net/ipv4/tcp_roccet.c:168) memsets the whole per-socket CA state, zeroing…
- [Medium] initial_round_completed is documented as a full-round guard ('Set to true after the initial roccet-control round has completed',…
- [Medium] The LAUNCH exit path in roccettcp_cong_avoid() computes tcp_sk(sk)->snd_ssthresh = tcp_snd_cwnd(tp) / 2…
- [Medium] roccet_min_rtt_probe() reduces cwnd and relies on a later call, while ca->state is still RTT_PROBE/RTT_PROBE_REFILL and while…
- [Medium] In roccettcp_recalc_ssthresh(), the RTT_PROBE branch updates ca->last_max_cwnd from the local cwnd, which at that point still…
- [Medium] update_srrtt() truncates the 64-bit rRTT result into a u32 and then evaluates the EWMA entirely in 32-bit arithmetic: `u32 rrtt =…
- [Low] Unfinished-submission artifacts in the new file: leftover commented-out code //WRITE_ONCE(sk->sk_pacing_rate, 0); in…
- [Low] The main commit description omits ROCCET's periodic forced congestion-window reduction for minimum-RTT probing.
- [Low] Two comments describe artifacts that do not exist or state the wrong unit: update_srrtt() explains its invariant in terms of…
- [Low] param_check()'s comment (net/ipv4/tcp_roccet.c:382-385) claims it exists 'to check the roccet params in advance in order to avoid…
- [Low] In update_ack_rate(), the 'new interval' branch assigns the u32 parameter acked directly into the u16 field ca->ack_rate_cnt…
- [Low] In roccettcp_cong_avoid(), the LAUNCH branch does acked = tcp_slow_start(tp, acked); if (!acked) return;…
- [Low] The ORBITER send/received monitoring interval cannot account for more than 4 GiB of data: `send = tp->snd_nxt -…
- [Low] sr_rtt_upper_bound and ack_rate_diff_ss are module_param(..., 0644) and are read from the packet-processing hot…
--
tcp: Add TCP ROCCET congestion control module.
This adds a new loadable congestion control module, ROCCET, which builds on the CUBIC window growth function and adds an srRTT metric, an ACK arrival rate metric and a send/receive byte comparison to detect bufferbloat. It replaces HyStart with its own slow start (LAUNCH) and drives everything from a .cong_control callback.
A few questions about the new file follow.
The commit body describes the congestion avoidance phase as purely passive:
"For the congestion avoidance phase, ROCCET relies on the srRTT and
monitoring the send and received Bytes to detect the filling of the bottleneck buffer."
[Severity: Low]
Could the commit message also describe the active minimum RTT probing? roccet_min_rtt_probe() cuts cwnd to max(cwnd/2, TCP_INIT_CWND), or max(cwnd/3, TCP_INIT_CWND) when tcp_is_cwnd_limited() is false, pins snd_ssthresh to that value for at least 200ms or one RTT, resets curr_min_rtt and later restores the previous window. It is armed every time the five second deadline in ca->next_min_rtt_probe expires. Only the revision history mentions it ("Add minimum RTT probing"), while the body reads as if no forced window reduction happens.
Quoted 6 lines
net/ipv4/tcp_roccet.c new file +0
-0@@ -0,0 +1,1050 @@
[ ... ]
Quoted 2 lines
net/ipv4/tcp_roccet.c new file +2
-0+/* min RTT probe period in ms */ +#define ROCCET_NEXT_MIN_RTT_PROBE 5000
[ ... ]
Quoted 14 lines
net/ipv4/tcp_roccet.c new file +14
-0+/* Parameters that are specific to the ROCCET-Algorithm */ + +// = 717/1024 (BICTCP_BETA_SCALE) +#define BETA_PARAM_DEFAULT 717 +#define BIC_SCALE_PARAM_DEFAULT 41 + +static uint sr_rtt_upper_bound__read_mostly =100 ; +static int ack_rate_diff_ss__read_mostly =10 ; + +module_param (sr_rtt_upper_bound, uint,0644 ); +MODULE_PARM_DESC (sr_rtt_upper_bound,"ROCCET's upper bound for srRTT." ); +module_param (ack_rate_diff_ss, int,0644 ); +MODULE_PARM_DESC (ack_rate_diff_ss, +"ROCCET's threshold to exit slow start if ACK-rate defer by given amount of segments." );
[ ... ]
net/ipv4/tcp_roccet.c line 159
Quoted 5 lines
net/ipv4/tcp_roccet.c new file +5
-0+/* Note parameters that are used for precomputing scale factors are read-only */ +module_param (fast_convergence, int,0644 ); +MODULE_PARM_DESC (fast_convergence,"turn on/off fast convergence" ); +module_param (beta_param, int,0644 ); +MODULE_PARM_DESC (beta_param,"beta for multiplicative increase" );
[Severity: Medium]
The comment above says precompute parameters are read-only, but beta_param is registered with mode 0644 while bic_scale_param uses 0444. Since there is no module_param_cb setter, a write to beta_param goes straight into the global with no validation and no recomputation of beta, beta_scale or cube_factor. Those are refreshed only later, when some unrelated socket runs roccettcp_init().
If the written value is out of range, param_check(true) puts the default into the hidden beta but leaves beta_param reporting the rejected value, so /sys/module/tcp_roccet/parameters/beta_param permanently disagrees with the beta actually in use. Should beta_param be 0444 like bic_scale_param, or use a setter that validates and recomputes at write time?
[ ... ]
net/ipv4/tcp_roccet.c line 170
Quoted 3 lines
net/ipv4/tcp_roccet.c new file +3
-0+static void roccettcp_reset (struct roccettcp *ca) +{ +memset (ca,0 ,sizeof (struct roccettcp ));
[Severity: Medium]
The memset also zeroes ca->interval_snd_seq_start and ca->interval_una_seq_start. roccettcp_init() seeds them explicitly:
ca->interval_snd_seq_start = tcp_sk(sk)->snd_nxt; ca->interval_una_seq_start = tcp_sk(sk)->snd_una;
but the TCP_CA_Loss branch of roccettcp_state() calls roccettcp_reset() without re-seeding them. Does ORBITER then compare absolute sequence numbers?
send = tp->snd_nxt - ca->interval_snd_seq_start; received = tp->snd_una - ca->interval_una_seq_start;
With both baselines at 0 this reduces to "bytes in flight > 1% of cwnd * mss", which looks true for as long as the flow is sending, so the later
if (!tcp_is_cwnd_limited(sk) || send_more_than_acked) return;
blocks all window growth until the first srRTT evaluation re-seeds the baselines, and that same evaluation can authorize a congestion event.
net/ipv4/tcp_roccet.c line 197
Quoted 7 lines
net/ipv4/tcp_roccet.c new file +7
-0+static void update_min_rtt (struct sock *sk) +{ +struct roccettcp *ca =inet_csk_ca (sk); + +/* Check if new lower min RTT was found. If so, set it directly */ +if (ca->curr_rtt < ca->curr_min_rtt) { + ca->curr_min_rtt =max (ca->curr_rtt,1 );
[Severity: High]
Can this latch a 1us minimum RTT when no RTT sample has been taken yet?
After roccettcp_reset() or roccettcp_init(), ca->curr_rtt is 0 and ca->curr_min_rtt is ~0U. roccettcp_acked() returns early for invalid samples:
/* Some calls are for duplicates without timestamps */ if (sample->rtt_us < 0) return;
and tcp_clean_rtx_queue() calls pkts_acked unconditionally with sample.rtt_us taken from rate->rtt_us, which is -1 when no RTT could be measured. roccet_control() then still runs update_min_rtt() and update_srrtt() with curr_rtt == 0, so 0 < ~0U is true and curr_min_rtt becomes max(0, 1) == 1.
Reachable cases would be a SACK-only or duplicate first ACK when the first packet of the initial window is lost, and the first ACK after an RTO on a connection without TCP timestamps.
Once curr_min_rtt is 1, update_srrtt() computes
rrtt = 100 * (curr_rtt - 1) / 1, i.e. millions, so curr_srrtt stays far above sr_rtt_upper_bound (default 100) and the bufferbloat detector fires continuously until the next minimum RTT probe five seconds later. Would it help to distinguish "no sample yet" from a real 1us RTT here?
Quoted 3 lines
net/ipv4/tcp_roccet.c new file +3
-0+/* Probe for the min RTT in ROCCET_NEXT_MIN_RTT_PROBE seconds +* if no other update occurs. +*/
[ ... ]
net/ipv4/tcp_roccet.c line 257
Quoted 11 lines
net/ipv4/tcp_roccet.c new file +11
-0+ ca->ack_rate_curr_rate = ca->ack_rate_cnt; + ca->ack_rate_cnt = + acked;// start counting for the new interval + } + + ca->was_idle = false; + }else { +// Cap the ack count to avoid overflow + ca->ack_rate_cnt =min_t (u32, ca->ack_rate_cnt + acked, +U16_MAX ); + }
[Severity: Low]
The new-interval branch assigns the u32 argument acked straight into the u16 ca->ack_rate_cnt with no clamp, while the sibling branch a few lines below saturates with min_t(u32, ..., U16_MAX). Should the first assignment saturate too? An ACK covering more than 65535 segments at an interval boundary truncates (70000 becomes 4464) and feeds a wrong value into get_ack_rate_diff().
[ ... ]
net/ipv4/tcp_roccet.c line 286
Quoted 5 lines
net/ipv4/tcp_roccet.c new file +5
-0+/* Calculate the new rRTT (Scaled by 100). +* 100 * ((sRTT - sRTT_min) / sRTT_min). +* +* curr_min_rtt_timed.rtt is always <= than curr_rtt, +* since this is the minimum of the rtt.
[Severity: Low]
There is no curr_min_rtt_timed member in struct roccettcp; the field is the flat u32 curr_min_rtt. Same kind of thing in update_min_rtt(), where the comment says the next probe happens "in ROCCET_NEXT_MIN_RTT_PROBE seconds" while the macro is documented as milliseconds and is multiplied by USEC_PER_MSEC, i.e. five seconds and not 5000 seconds.
net/ipv4/tcp_roccet.c line 297
Quoted 11 lines
net/ipv4/tcp_roccet.c new file +11
-0+* +* 0 is a valid value for rrtt. +*/ +u32 rrtt =div_u64 (100 * (u64)(ca->curr_rtt - ca->curr_min_rtt), + ca->curr_min_rtt); + +// (1 - alpha) * srRTT + alpha * rRTT + ca->curr_srrtt = ((100 -ROCCET_ALPHA_TIMES_100 ) * ca->curr_srrtt + +ROCCET_ALPHA_TIMES_100 * rrtt) / +100 ; +}
[Severity: Medium]
The div_u64() numerator is widened to 64 bits, but the result is stored in a u32 and the EWMA below is evaluated entirely in 32-bit arithmetic. Does 80 * ca->curr_srrtt wrap once curr_srrtt exceeds roughly 53.6M?
With curr_min_rtt floored to 1us (see the update_min_rtt() question above), a 540ms RTT sample already gives rrtt above 53.6M, and curr_srrtt converges toward it. curr_srrtt is the only bufferbloat signal gating roccet_congestion_event() and the LAUNCH exit, so a wrap there either defeats the reaction or fires it spuriously.
The v2/v3 changelog says "Fix potential integer overflow in rRTT calculation" and "Fix potential overflow in rRTT calculation by using 64-bit arithmetic", but only the numerator was widened.
[ ... ]
Quoted 3 lines
net/ipv4/tcp_roccet.c new file +3
-0+ probe_cwnd =max (tcp_snd_cwnd (tp) /2 ,TCP_INIT_CWND ); +if (!tcp_is_cwnd_limited (sk)) + probe_cwnd =max (tcp_snd_cwnd (tp) /3 ,TCP_INIT_CWND );
[ ... ]
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +4
-0+ ca->refill_until = ca->probe_min_rtt_until + interval; + }else if (before (now, ca->refill_until)) { +/* Reset cwnd and refill the pipe. */ +if (ca->state !=RTT_PROBE_REFILL ) {
net/ipv4/tcp_roccet.c line 380
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+tcp_snd_cwnd_set (tp, ca->cwnd_before_min_rtt_probe); +tcp_sk (sk)->snd_ssthresh =tcp_snd_cwnd (tp); + ca->state =RTT_PROBE_REFILL ; + } + }else { +/* End min RTT probing phase. */ + ca->probe_min_rtt_until =0 ; + ca->state =ORBITER ; + } +}
[Severity: Medium]
The cwnd restore depends on ca->state still being RTT_PROBE or RTT_PROBE_REFILL and on an ACK landing inside the refill window. Can other code paths clear the state and leave the reduced window in place?
roccettcp_recalc_ssthresh() starts with:
if (ca->state == RTT_PROBE_REFILL) ca->state = ORBITER;
and roccet_control() overwrites ca->state in the two higher priority branches:
if (tcp_in_slow_start(tp)) {
ca->state = LAUNCH;
} else if ((s32)now - ca->roccet_last_event_time_us <=
100 * USEC_PER_MSEC) {
ca->state = DRAIN;In all of these, probe_min_rtt_until and refill_until stay non-zero, so the restore branch is never taken. The sender then keeps the 2x-3x reduced cwnd with snd_ssthresh pinned to it, and when the deadline finally passes the terminal else only clears probe_min_rtt_until, making that probe round a no-op. The same skip happens if no ACK arrives inside the refill window.
Quoted 4 lines
+/* Used to check the roccet params in advance in order to avoid
+ * invalid-param-attacks. This validates the provided params and
+ * then saves valid copies of the params.
+ */
[Severity: Low]
This describes all of the parameters, but only beta_param and bic_scale_param are validated and copied. sr_rtt_upper_bound, ack_rate_diff_ss, initial_ssthresh, fast_convergence and tcp_friendliness are all 0644 and are read live with no bounds check and no validated copy, including initial_ssthresh which is assigned directly to tp->snd_ssthresh in roccettcp_init(). Could the comment be narrowed to what is actually checked?
Quoted 11 lines
net/ipv4/tcp_roccet.c new file +11
-0+static int param_check (bool use_defaults) +{ +int ret =0 ; + +/* +* Validate parameters to avoid division by zero errors. +*/ +if (beta_param <=0 || beta_param >=BICTCP_BETA_SCALE ) { +pr_err ("roccet: beta must be between 0 and %d\n" , +BICTCP_BETA_SCALE ); +
net/ipv4/tcp_roccet.c line 406
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+if (use_defaults) { +pr_info ("Using default value of %d for beta.\n" , +BETA_PARAM_DEFAULT ); + beta =BETA_PARAM_DEFAULT ; + }else { + ret = -EINVAL ; + } + }else { + beta = beta_param; + }
[Severity: Medium]param_check(true) is called from roccettcp_init(), i.e. once per socket. Can this flood the log? The out-of-range path never rewrites beta_param itself, only the hidden beta, so once an invalid value is stored every subsequent connection that selects roccet prints two to four lines, indefinitely. Would pr_*_once() or ratelimited variants, or moving the diagnostics to module init and a param setter, avoid that?
[ ... ]
Quoted 9 lines
net/ipv4/tcp_roccet.c new file +9
-0+static void param_precompute (void ) +{ +/* Precompute a bunch of the scaling factors that are used per-packet +* based on SRTT of 100ms +*/ + beta_scale = +8 * (BICTCP_BETA_SCALE + beta) /3 / (BICTCP_BETA_SCALE - beta); + + cube_rtt_scale = (bic_scale *10 );/* 1024*c/rtt */
[ ... ]
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +4
-0+/* 1/c * 2^2*bictcp_HZ * srtt */ + cube_factor =1ull << (10 +3 *BICTCP_HZ );/* 2^40 */ + +/* divide by bic_scale and by constant Srtt (100ms) */
net/ipv4/tcp_roccet.c line 463
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+do_div (cube_factor, bic_scale *10 ); +} + +static void roccettcp_init (struct sock *sk) +{ +/* Check & precompute on `init` in order to use the newest +* available params. +*/ +param_check (true); +param_precompute ();
[Severity: High]
Is it safe to write the module-wide globals from a per-socket callback? roccettcp_init() runs for every new socket that selects roccet, and param_check()/param_precompute() write beta, bic_scale, beta_scale, cube_rtt_scale and cube_factor with no lock, RCU or atomic annotation, while every other live flow reads them locklessly from the ACK path (bictcp_update(), roccet_congestion_event(), roccettcp_recalc_ssthresh()).
cube_factor is also published in an intermediate state:
cube_factor = 1ull << (10 + 3 * BICTCP_HZ); /* 2^40 */ do_div(cube_factor, bic_scale * 10);
so a concurrent bictcp_update() can multiply the undivided 2^40, and two initializers racing here can apply do_div() twice to the same variable, shrinking cube_factor by another factor of ~410 for the rest of the module's life.
param_check() also reads beta_param twice, once for the range test and once for "beta = beta_param;". If the second read picks up a concurrently written 1024, param_precompute() divides by (BICTCP_BETA_SCALE - beta) == 0.
Upstream CUBIC does this precomputation once in __init cubictcp_register() and keeps bic_scale read-only. Could ROCCET do the same?
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +4
-0+ +struct roccettcp *ca =inet_csk_ca (sk); + +roccettcp_reset (ca);
[ ... ]
net/ipv4/tcp_roccet.c line 477
Quoted 3 lines
net/ipv4/tcp_roccet.c new file +3
-0+cmpxchg (&sk->sk_pacing_status,SK_PACING_NONE ,SK_PACING_NEEDED ); +//WRITE_ONCE(sk->sk_pacing_rate, 0); +}
[Severity: Low]
This isn't a bug, but the commented-out WRITE_ONCE() looks like leftover debug code and should probably be dropped. There are also several C99 // comments in the file (next to BETA_PARAM_DEFAULT, in update_ack_rate() and in update_srrtt()) which checkpatch flags as errors.
And in roccet_control() the pacing comment contradicts both the code and the comment right below it:
* In Congestion Avoidance phase, set it to 120 % the current rate. ... /* Pacing rate of 100% * (instead of ipv4.sysctl_tcp_pacing_ca_ratio) */ rate *= 100;
[ ... ]
net/ipv4/tcp_roccet.c line 641
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+tcp_friendliness: +/* TCP Friendly */ +if (tcp_friendliness) { +u32 scale = beta_scale; + + delta = (cwnd * scale) >>3 ; +while (ca->ack_cnt > delta) {/* update tcp cwnd */ + ca->ack_cnt -= delta; + ca->tcp_cwnd++; + }
[Severity: High]
Can this loop spin forever when delta is 0? ca->ack_cnt is non-zero here because "ca->ack_cnt += acked;" runs at the top of bictcp_update(), and subtracting 0 never changes it, so there is no exit condition.
param_check() only rejects beta_param outside (0, 1024), and the surviving values truncate beta_scale down far enough that (cwnd * scale) >> 3 is 0 for small cwnd:
beta_param = 300 -> beta_scale 4 -> delta 0 at cwnd 1 beta_param = 200 -> beta_scale 3 -> delta 0 at cwnd <= 2 beta_param = 1 -> beta_scale 2 -> delta 0 at cwnd <= 3
cwnd of 2 is reachable because roccet_congestion_event() clamps to a floor of 2. This runs in softirq context on the ACK path, so the CPU would stop processing softirqs.
Unlike upstream CUBIC, which computes beta_scale once in __init cubictcp_register(), this module recomputes it from beta_param on every new socket, so a runtime write to the 0644 parameter reaches live traffic. Should the accepted range be tightened, or delta == 0 guarded?
[ ... ]
Quoted 11 lines
net/ipv4/tcp_roccet.c new file +11
-0+if ((ca->curr_srrtt > sr_rtt_upper_bound && +get_ack_rate_diff (ca) <= ack_rate_diff_ss) || + (!tcp_is_cwnd_limited (sk) && + ca->initial_round_completed)) { + ca->epoch_start =0 ; + +/* Handle initial slow start. +* Most bufferbloat occurs here +*/ +if (tp->snd_ssthresh ==TCP_INFINITE_SSTHRESH ) { +tcp_sk (sk)->snd_ssthresh =tcp_snd_cwnd (tp)
net/ipv4/tcp_roccet.c line 701
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+ / 2; +/* since this is the initial slow start, +* the min cwnd won't be 1, so the window +* can't be set to 0 by accident. +* Halfing the cwnd will undo the previous step +* of slow start. Which is fine since the pipe +* is already full. +*/ +tcp_snd_cwnd_set (tp,max (tcp_snd_cwnd (tp) /2 , +TCP_INIT_CWND ));
[Severity: Medium]
Is the assumption in that comment always true? cwnd == 1 together with snd_ssthresh == TCP_INFINITE_SSTHRESH looks reachable:
tcp_enter_loss()
WRITE_ONCE(tp->snd_ssthresh, icsk->icsk_ca_ops->ssthresh(sk));
-> roccettcp_ssthresh() returns the unchanged TCP_INFINITE_SSTHRESH
tcp_snd_cwnd_set(tp, tcp_packets_in_flight(tp) + 1); /* == 1 */
tcp_set_ca_state(sk, TCP_CA_Loss)
roccettcp_state() -> roccettcp_reset() -> state = LAUNCHTaking this branch with cwnd == 1 stores snd_ssthresh = 0 while cwnd is raised to TCP_INIT_CWND. After that tcp_in_slow_start() can never be true again (cwnd is never < 0), so roccet_control() never selects LAUNCH, tcp_slow_start() is unreachable for the rest of the connection, and the pacing branch "tcp_snd_cwnd(tp) < tp->snd_ssthresh / 2" is dead. cwnd of 2 or 3 gives snd_ssthresh 1. Should there be a floor here?
Quoted 1 line
net/ipv4/tcp_roccet.c new file +1
-0+ }else {
[ ... ]
net/ipv4/tcp_roccet.c line 717
Quoted 5 lines
net/ipv4/tcp_roccet.c new file +5
-0+ acked =tcp_slow_start (tp, acked); +if (!acked) +return ; + + }else if (ca->state ==ORBITER ) {
[Severity: Low]
The non-zero leftover returned by tcp_slow_start() means cwnd just reached snd_ssthresh and the remaining credit should be applied in congestion avoidance, which is what stock CUBIC does with bictcp_update() plus tcp_cong_avoid_ai(). Here the LAUNCH branch just falls out of the if/else chain, and ca->state only becomes ORBITER on the next roccet_control() call, so the credit is dropped. Was that intended?
[ ... ]
net/ipv4/tcp_roccet.c line 740
Quoted 2 lines
net/ipv4/tcp_roccet.c new file +2
-0+ send = tp->snd_nxt - ca->interval_snd_seq_start; + received = tp->snd_una - ca->interval_una_seq_start;
[Severity: Low]
These are 32-bit sequence deltas, and the interval is five RTTs (ca->next_srrtt_check = now + 5 * ca->curr_rtt). If a flow advances more than 4GiB within that interval the modulo totals invert the send_more_than_acked comparison. That needs roughly 34Gbit/s at a 200ms RTT, so it is an extreme case, but should the totals be accumulated in u64?
[ ... ]
net/ipv4/tcp_roccet.c line 768
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +4
-0+ roccet_xj =div_u64 ((u64 )jitter *100 , ca->curr_min_rtt) + + sr_rtt_upper_bound; +if (roccet_xj < sr_rtt_upper_bound) + roccet_xj = sr_rtt_upper_bound;
[Severity: Low]sr_rtt_upper_bound is a 0644 module parameter read here twice as a plain load, and the second read is the wrap guard for the first. Since sysfs writes only take the module param mutex, which this softirq-context reader does not hold, the compiler is free to reload it and defeat the guard. Would a single READ_ONCE() snapshot into a local be better? The LAUNCH exit test reads sr_rtt_upper_bound and ack_rate_diff_ss the same way.
[ ... ]
net/ipv4/tcp_roccet.c line 798
Quoted 4 lines
net/ipv4/tcp_roccet.c new file +4
-0+static u32 roccettcp_ssthresh (struct sock *sk) +{ +return tcp_sk (sk)->snd_ssthresh; +}
[Severity: High]
With .ssthresh pointing at this getter, is there any multiplicative decrease left on loss or RTO?
tcp_enter_loss() does:
WRITE_ONCE(tp->snd_ssthresh, icsk->icsk_ca_ops->ssthresh(sk));
which stores the same value back, so snd_ssthresh is never reduced on RTO. roccettcp_state() then calls roccettcp_reset() for TCP_CA_Loss, which memsets the state, wiping ca->last_max_cwnd (CUBIC's W_max) and returning the flow to LAUNCH. roccettcp_recalc_ssthresh() returns cwnd unchanged while tcp_in_slow_start(), and because .cong_control is provided tcp_cong_control() returns before tcp_cwnd_reduction(), so PRR does not run either.
After an RTO the flow slow-starts straight back to the same unreduced ssthresh. On a lossy or shallow-buffered path, where RTT never inflates and the srRTT heuristic never fires, that leaves no back-off on loss at all. Only the file header mentions that LAUNCH ignores loss; could the commit message describe the RTO and ORBITER behaviour as well?
Quoted 5 lines
net/ipv4/tcp_roccet.c new file +5
-0+static u32 roccettcp_recalc_ssthresh (struct sock *sk) +{ +const struct tcp_sock *tp =tcp_sk (sk); +struct roccettcp *ca =inet_csk_ca (sk); +u32 cwnd =tcp_snd_cwnd (tp);
[ ... ]
Quoted 1 line
net/ipv4/tcp_roccet.c new file +1
-0+if (ca->state ==RTT_PROBE ) {
[ ... ]
net/ipv4/tcp_roccet.c line 836
Quoted 9 lines
net/ipv4/tcp_roccet.c new file +9
-0+/* Wmax and fast convergence */ +if (cwnd < ca->last_max_cwnd && fast_convergence) + ca->last_max_cwnd = + (cwnd * (BICTCP_BETA_SCALE + beta)) / + (2 *BICTCP_BETA_SCALE ); +else + ca->last_max_cwnd = cwnd; + + cwnd = ca->cwnd_before_min_rtt_probe;
[Severity: Medium]
At this point cwnd still holds tcp_snd_cwnd(tp), which during a probe is the deliberately deflated probe window; the assignment from ca->cwnd_before_min_rtt_probe happens only afterwards and is used just to recompute cwnd_before_min_rtt_probe. Doesn't that contradict the comment above ("we use the cwnd before the probing interval to calculate the cwnd reduction ... it is very likely that congestion was caused by the cwnd value before min RTT probing")?
With last_max_cwnd taken from the small probe window, once probing ends and cwnd is restored to the larger pre-probe value, bictcp_update() sees ca->last_max_cwnd <= cwnd and takes the
ca->bic_K = 0; ca->bic_origin_point = cwnd;
path, skipping the concave region and growing convexly right after a congestion event.
[ ... ]
net/ipv4/tcp_roccet.c line 930
Quoted 10 lines
net/ipv4/tcp_roccet.c new file +10
-0+static void roccet_in_ack_event (struct sock *sk,u32 flags) +{ +struct roccettcp *ca =inet_csk_ca (sk); + +/* Handle ECE bit. +* Processing of ECE events is done in roccettcp_recalc_ssthresh() +*/ +if (flags &CA_ACK_ECE ) + ca->ece_received = true; +}
[Severity: High]
Is roccettcp_recalc_ssthresh() actually reachable from the ECE path? It is only called from the TCP_CA_Recovery branch of roccettcp_state(), while a pure ECN mark goes through:
tcp_try_to_open()
tcp_enter_cwr()
tcp_init_cwnd_reduction()
WRITE_ONCE(tp->snd_ssthresh, ca_ops->ssthresh(sk));
tcp_set_ca_state(sk, TCP_CA_CWR);ca_ops->ssthresh is roccettcp_ssthresh(), which returns snd_ssthresh unchanged, and roccettcp_state() has no TCP_CA_CWR branch. Since .cong_control is set, tcp_cong_control() also returns before tcp_cwnd_reduction(), so PRR does not reduce cwnd either.
On an ECN-marked path without loss, does anything reduce ssthresh or cwnd? The latched ece_received then appears to be applied much later as a stale reduction at an unrelated Recovery event. The v3 changelog says "Always react to ECE bits and reset flag".
[ ... ]
Quoted 3 lines
net/ipv4/tcp_roccet.c new file +3
-0+/* current rate is (cwnd * mss) / srtt +* In Slow Start [1], set sk_pacing_rate to 200 % the current rate. +* In Congestion Avoidance phase, set it to 120 % the current rate.
[ ... ]
net/ipv4/tcp_roccet.c line 1011
Quoted 2 lines
net/ipv4/tcp_roccet.c new file +2
-0+ ca->initial_round_completed = true; +}
[Severity: Medium]
This is set unconditionally at the end of the first roccet_control() call that has rs->acked_sacked != 0, but the field is documented as "Set to true after the initial roccet-control round has completed" and roccettcp_cong_avoid() relies on that meaning:
* This condition is checked only after the initial round of
* LAUNCH has completed. This is done in order to avoid false
* positives, which can occur when the tracked outstanding
* packets have not yet caught up to the initial cwnd.
Since there is no RTT or sequence-space round boundary test, the "!tcp_is_cwnd_limited(sk) && ca->initial_round_completed" term can fire from the second ACK onwards. With cwnd 10 and only three packets in flight, tcp_is_cwnd_limited() is false in slow start (10 < 2 * 3), so the exit runs and leaves the flow permanently out of slow start at cwnd 10, growing only via the cubic path afterwards. Should the flag be set on a round boundary instead?
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/apWz9rPcGMgYCKOR%40volt-roccet-vm