Skip to contentSkip to conversation

Flowtables ignore timeout settings in recent kernels

DISCUSSION Vladimir Smelhaus 3 participants 4 messages Oct 15, 2023 – Oct 25, 2023 Regressions

lore
Share & tools
lore
#0 lore raw

Netfilter ignores the timeout settings for a flowtable

# sysctl -a -r flowtable
net.netfilter.nf_flowtable_tcp_timeout = 30
net.netfilter.nf_flowtable_udp_timeout = 30

Situation. A long udp connection (tunnel) with some data flowing through a router. The connection is sent to a flowtable on the router. It's a few packets per second, more here and there, a pause here and there, and so on over and over. The pauses are minimal and are also limited by the tunnel settings to be no longer than 25 seconds. Everything is satisfying to make the connection last continuously in the flowtable and not reappear in forward. However, the connection keeps dropping out of the flowtable. It stays in the flowtable (offloaded) for a second at most and then it is kicked out, back to forward.

In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.

Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.

Regards

Vladimir Smelhaus

#1 lore raw
Quoted 16 lines · Vladimir Smelhaus

Netfilter ignores the timeout settings for a flowtable

# sysctl -a -r flowtable
net.netfilter.nf_flowtable_tcp_timeout = 30
net.netfilter.nf_flowtable_udp_timeout = 30

Situation. A long udp connection (tunnel) with some data flowing through a router. The connection is sent to a flowtable on the router. It's a few packets per second, more here and there, a pause here and there, and so on over and over. The pauses are minimal and are also limited by the tunnel settings to be no longer than 25 seconds. Everything is satisfying to make the connection last continuously in the flowtable and not reappear in forward. However, the connection keeps dropping out of the flowtable. It stays in the flowtable (offloaded) for a second at most and then it is kicked out, back to forward.

Quoted 4 lines

In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.

Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.

Then please perform bisection to find a culprit that introduces your regression (see Documentation/admin-guide/bug-bisect.rst in the kernel sources for reference). Also, it'd been great if you also post the reproducer script inline (within your email) instead, as some MUAs (like mutt that I'm using now) may ignore the attachment.

Anyway, thanks for the regression report. I'm adding it to regzbot:

#regzbot ^introduced: v5.10..v6.5

Signature · 1 line
An old man doll... just what I always wanted! - Clara
#2 lore raw
Quoted 28 lines · Bagas Sanjaya
Quoted 16 lines · Vladimir Smelhaus

Netfilter ignores the timeout settings for a flowtable

# sysctl -a -r flowtable
net.netfilter.nf_flowtable_tcp_timeout = 30
net.netfilter.nf_flowtable_udp_timeout = 30

Situation. A long udp connection (tunnel) with some data flowing through a router. The connection is sent to a flowtable on the router. It's a few packets per second, more here and there, a pause here and there, and so on over and over. The pauses are minimal and are also limited by the tunnel settings to be no longer than 25 seconds. Everything is satisfying to make the connection last continuously in the flowtable and not reappear in forward. However, the connection keeps dropping out of the flowtable. It stays in the flowtable (offloaded) for a second at most and then it is kicked out, back to forward.

Quoted 4 lines

In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.

Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.

Then please perform bisection to find a culprit that introduces your regression (see Documentation/admin-guide/bug-bisect.rst in the kernel sources for reference). Also, it'd been great if you also post the reproducer script inline (within your email) instead, as some MUAs (like mutt that I'm using now) may ignore the attachment.

Quoted 3 lines

Anyway, thanks for the regression report. I'm adding it to regzbot:

#regzbot ^introduced: v5.10..v6.5

Fix here:

https://patchwork.ozlabs.org/project/netfilter-devel/patch/[email protected]/

it is a bug from Jun 2023, regression was introduced in the v6.5 development cycle.

Thanks for reporting.

#3 lore raw
Quoted 31 lines · Pablo Neira Ayuso
Quoted 30 lines · Bagas Sanjaya
Quoted 16 lines · Vladimir Smelhaus

Netfilter ignores the timeout settings for a flowtable

# sysctl -a -r flowtable
net.netfilter.nf_flowtable_tcp_timeout = 30
net.netfilter.nf_flowtable_udp_timeout = 30

Situation. A long udp connection (tunnel) with some data flowing through a router. The connection is sent to a flowtable on the router. It's a few packets per second, more here and there, a pause here and there, and so on over and over. The pauses are minimal and are also limited by the tunnel settings to be no longer than 25 seconds. Everything is satisfying to make the connection last continuously in the flowtable and not reappear in forward. However, the connection keeps dropping out of the flowtable. It stays in the flowtable (offloaded) for a second at most and then it is kicked out, back to forward.

Quoted 4 lines

In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.

Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.

Then please perform bisection to find a culprit that introduces your regression (see Documentation/admin-guide/bug-bisect.rst in the kernel sources for reference). Also, it'd been great if you also post the reproducer script inline (within your email) instead, as some MUAs (like mutt that I'm using now) may ignore the attachment.

Anyway, thanks for the regression report. I'm adding it to regzbot:

Quoted 9 lines
Quoted 1 line

#regzbot ^introduced: v5.10..v6.5

Fix here:

https://patchwork.ozlabs.org/project/netfilter-devel/patch/[email protected]/

it is a bug from Jun 2023, regression was introduced in the v6.5 development cycle.

Telling regzbot:

#regzbot fix: netfilter: nf_flow_table: GC pushes back packets to classic path

Thanks.

Signature · 1 line
An old man doll... just what I always wanted! - Clara