See complete description here:
https://bugzilla.kernel.org/show_bug.cgi?id=72561
Summary - some ICMP redirects seem to be missing.
Alan suggests I gather some traces, but I'm am unsure of what to trace. Suggestions please?

Reproduce the actual issue and post a simplified testcase, without transparent proxying, etc.

Per Jessen

author

2014-06-10 18:27 UTC #2 lore raw

Quoted 11 lines · Lukas Tribus

Quoted 8 lines
See complete description here:
https://bugzilla.kernel.org/show_bug.cgi?id=72561
Summary - some ICMP redirects seem to be missing.
Alan suggests I gather some traces, but I'm am unsure of what to trace. Suggestions please?
Reproduce the actual issue and post a simplified testcase, without transparent proxying, etc.

Okay, I have devised a way to reproduce the problem:

I have a test setup of three machines:

“client”, “firewall” and “server”. All on the same network.

Client:
Set up default route via “firewall”.

Server:
Assign 10.232.1.1-2-3-4-...-15/24 to an interface. Run a tcp echo service (port 7).

Firewall:
Create routing table “test99”. /etc/iproute2/rt_tables.
ip route add default via <server> dev eth0 table test99
ip rule add fwmark 5 table test99
iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 7 -j MARK --set-mark 5

(this setup is what will produce the ICMP redirects).

On “firewall”, run tcpdump to document (missing) redirects:
tcpdump -n -i eth0 proto \\icmp

On “client”, create some test input:

cat <<XXX >test.input
klop
alpha
nothing
tagi
line1
line2
line3
line4
XXX

Create a script
cat <<XXX >doit
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
do

    telnet 10.232.1.$i 7 <test.input &

done

When you run “sh doit”, all of the telnet requests to 10.232.1.x should be redirected, but the tcpdump running on "firewall" will only show some of them.

Dmitry Popov

2014-06-10 20:06 UTC #3 lore raw

Quoted 10 lines · On Tue, 10 Jun 2014 20:10:23 +0200 Per Jessen <[email protected]> wrote:

Quoted 6 lines · Lukas Tribus
Quoted 6 lines
See complete description here:
https://bugzilla.kernel.org/show_bug.cgi?id=72561
Summary - some ICMP redirects seem to be missing.
Okay, I have devised a way to reproduce the problem:
...

It's probably not a bug, but intended behavior. See ip_rt_send_redirect in net/ipv4/route.c. The kernel has a built-in per-source-address limiter for icmp redirects.

You can disable it for example with
echo -1 > /proc/sys/net/ipv4/route/redirect_silence Though I am not sure if it is ok in wild networks.

Per Jessen

author

2014-06-11 07:19 UTC #4 lore raw

Quoted 8 lines · Dmitry Popov

Quoted 5 lines · On Tue, 10 Jun 2014 20:10:23 +0200 Per Jessen <[email protected]> wrote:
Quoted 4 lines · Lukas Tribus
Quoted 4 lines
See complete description here:
https://bugzilla.kernel.org/show_bug.cgi?id=72561

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Summary - some ICMP redirects seem to be missing.
Okay, I have devised a way to reproduce the problem:
...
It's probably not a bug, but intended behavior. See ip_rt_send_redirect in net/ipv4/route.c. The kernel has a built-in per-source-address limiter for icmp redirects.

Hi Dmitry

Any possibility that this limiter may have been inactive/faulty in e.g. 2.6.34?
We only hit this issue when we upgraded the firewall to a newer openSUSE (for
ebtables) which came with kernel 3.11.6, but prior to that we had no issue with missing redirects.

Quoted 2 lines

You can disable it for example with echo -1 > /proc/sys/net/ipv4/route/redirect_silence Though I am not sure if it is ok in wild networks.

Okay, that worked on the first attempt.
Should I also be looking at the other redirect settings?

/proc/sys/net/ipv4/route/redirect_load
/proc/sys/net/ipv4/route/redirect_number

thanks
Per

Dmitry Popov

2014-06-11 15:04 UTC #5 lore raw

Quoted 7 lines · On Wed, 11 Jun 2014 09:19:22 +0200 Per Jessen <[email protected]> wrote:

Quoted 6 lines · Dmitry Popov
Quoted 3 lines · On Tue, 10 Jun 2014 20:10:23 +0200 Per Jessen <[email protected]> wrote:
Quoted 2 lines · Lukas Tribus
Quoted 2 lines
See complete description here:

Quoted 8 lines

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
https://bugzilla.kernel.org/show_bug.cgi?id=72561
Summary - some ICMP redirects seem to be missing.
It's probably not a bug, but intended behavior. See ip_rt_send_redirect in net/ipv4/route.c. The kernel has a built-in per-source-address limiter for icmp redirects.
Any possibility that this limiter may have been inactive/faulty in e.g. 2.6.34?

Hi!

This limiter worked per-destination-address before kernel v3.6. More precisely, commit 1d861aa4b3(inet: Minimize use of cached route inetpeer) changed that behavior. I don't know the reasons behind it (both approaches have pros and cons), I think you should ask David if you're curious.

* upd: it's not a limiter for icmp redirects only, it is used for other icmp errors too.

Quoted 9 lines

Quoted 3 lines
You can disable it for example with
echo -1 > /proc/sys/net/ipv4/route/redirect_silence Though I am not sure if it is ok in wild networks.
Okay, that worked on the first attempt.
Should I also be looking at the other redirect settings?
/proc/sys/net/ipv4/route/redirect_load
/proc/sys/net/ipv4/route/redirect_number

redirect_silence is enough, you can look at the algorithm (it's simple enough) in net/ipv4/route.c:ip_rt_send_redirect if you're curious how it works and what those settings mean.