New: Network connections hang going through an OpenBSD firewall]
Subsystems (2)
Can you reproduce on the latest kernel, please?
Is the OpenBSD mangling the packet in any way?
Can anyone tell me if this smells like something
recently fixed (MTU issues)? Doesn't sound like
the windowscaling problem but could be related.
thanks,
Nivedita
-------- Original Message --------
Subject: [Bug 3397] New: Network connections hang going through an OpenBSD firewall
Date: Tue, 14 Sep 2004 09:38:24 -0700
From: [email protected]
To: [email protected]
http://bugme.osdl.org/show_bug.cgi?id=3397
Summary: Network connections hang going through an OpenBSD
firewall
Kernel Version: 2.6.6+
Status: NEW
Severity: blocking
Owner: [email protected]
Submitter: [email protected]
Distribution: Fedora Core 2, Gentoo
Hardware Environment: All
Software Environment: All
Problem Description:
We have seen a number of issues with people accessing our website
http://www.cs.unm.edu/ using kernels 2.6.6+. Doing some network sniffing we can
see that data request is received by the web server and the web server responds
however after certain amount of bytes it simply stops.
I have played with MTU sizes and if Ethereal is to be believed the transfer stops after MTU + 77 bytes.
This has been reported to us by a number of different people running different
distributions ie. Fedora Core 2, Gentoo. For example 2.6.5 kernel that comes with FC2 works. Secure IMAP and Secure POP don't seem to work either using 2.6.6+.
What is even stranger is that SSH connections don't exhibit this kind of a problem ie. you can SSH withouth a hitch.
A data point is that we are using transparent (in-line) OpenBSD firewall with
Packetfilter.
Steps to reproduce:
Boot into 2.6.6+ kernel try pulling up http://www.cs.unm.edu/ through a web browser. It won't show up. Boot into a 2.6.5 and below and the page will show up.
Any clues will be appreciated.
------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
Quoted 6 lines · On Tue, 14 Sep 2004 14:00:26 -0700 Nivedita Singhvi <[email protected]> wrote:
Can you reproduce on the latest kernel, please?
Is the OpenBSD mangling the packet in any way?Can anyone tell me if this smells like something
recently fixed (MTU issues)? Doesn't sound like
the windowscaling problem but could be related.
OpenBSD pf is easy to configure to break window scaling. The developer claims its not a bug. Basically stateless TCP connection tracking will never work, it's a bad idea.
Quoted 10 lines · Daniel Hartmeier
The problem arises when the user creates a complicated ruleset that passes the first SYN of a connection without creating a state entry. You might argue that this shouldn't be possible, but some forms of stateless filtering are being used. The man page warns against this, too.
In this particular case, the ruleset tells pf to pass the initial SYN without creating state (and therefore without any place to note the window option and first scale factor). When, later, the SYN+ACK creates state, the state just doesn't contain the information to follow the scaled windows.
Quoted 1 line · Nivedita Singhvi
Can you reproduce on the latest kernel, please?
I will try it with the latest kernel. I have tried this with kernel 2.6.8-1.521 from
It doesn't work with it. Should I download 2.6.8-1 from kernel.org and compile it by hand ?
Quoted 1 line
Is the OpenBSD mangling the packet in any way?
It is certainly possible but everything works fine with all other OSes and kernels 2.6.5 and below which leads me to believe something was changed in 2.6.6 that broke it.
Are you able to reproduce it ?
Thanks,
Vladimir Vuksan
Quoted 7 lines
Can anyone tell me if this smells like something
recently fixed (MTU issues)? Doesn't sound like
the windowscaling problem but could be related.
David S. Miller
maintainerQuoted 7 lines · On Tue, 14 Sep 2004 15:09:50 -0600 Vladimir <[email protected]> wrote:
Quoted 1 line
Is the OpenBSD mangling the packet in any way?
It is certainly possible but everything works fine with all other OSes and kernels 2.6.5 and below which leads me to believe something was changed in 2.6.6 that broke it.
Are you able to reproduce it ?
OpenBSD packet filter is busted, and the maintainer of it claims this is not a bug.
That changes in 2.6.6 didn't "break" things, it enabled a feature in TCP that OpenBSD stateless TCP connection tracking cannot handle, and old TCP feature in fact, window scaling.
See here for more info:
Quoted 1 line · Vladimir
It doesn't work with it. Should I download 2.6.8-1 from kernel.org and compile it by hand ?
If you can, download the latest (linux-2.6.9-rc2). You can get the patch against linux-2.6.8.1 vanilla from kernel.org.
Quoted 3 lines
It is certainly possible but everything works fine with all other OSes and kernels 2.6.5 and below which leads me to believe something was changed in 2.6.6 that broke it.
Are you able to reproduce it ?
Unfortunately, I have zero cycles right now (and in the near future) to address any issues, I'm afraid. Someone on netdev will likely be able to help you or have run into it already. See Stephen's post from a few minutes ago..
thanks,
Nivedita
Quoted 11 lines · David S. Miller
OpenBSD packet filter is busted, and the maintainer of it claims this is not a bug.
That changes in 2.6.6 didn't "break" things, it enabled a feature in TCP that OpenBSD stateless TCP connection tracking cannot handle, and old TCP feature in fact, window scaling.
See here for more info:
Thanks. We were able to fix our firewall so things work properly now. The problem is that this is "insidious" since it is not immediately apparent what the problem is especially since it tends to work with all other OSes except Linux with 2.6.6+.
I will note this on the bug I submitted and close it.
Thanks a lot,
Vladimir Vuksan
Quoted 3 lines · Nivedita Singhvi
If you can, download the latest (linux-2.6.9-rc2). You can get the patch against linux-2.6.8.1 vanilla from kernel.org.
Patch against 2.6.8. It won't apply clean on 2.6.8.1.
Signature · 2 lines
Tomasz Torcz Morality must always be based on practicality. [email protected] -- Baron Vladimir Harkonnen
Quoted 13 lines · On Tue, 14 Sep 2004 14:21:41 -0700 Nivedita Singhvi <[email protected]> wrote:
Quoted 2 lines · Vladimir
It doesn't work with it. Should I download 2.6.8-1 from kernel.org and compile it by hand ?
If you can, download the latest (linux-2.6.9-rc2). You can get the patch against linux-2.6.8.1 vanilla from kernel.org.
Quoted 4 lines
It is certainly possible but everything works fine with all other OSes and kernels 2.6.5 and below which leads me to believe something was changed in 2.6.6 that broke it.
Quoted 6 lines
Quoted 1 line
Are you able to reproduce it ?
Unfortunately, I have zero cycles right now (and in the near future) to address any issues, I'm afraid. Someone on netdev will likely be able to help you or have run into it already. See Stephen's post from a few minutes ago..
Since the latest kernel chooses window scaling factor automatically, the value chosen by default is small enough that OpenBSD
busted pf will just make performance suck. If you want test it, you have to increase tcp_rmem[2] or tcp_rmem_max to force a window scale > 2.