quite big breakthrough in the BAD network performance, which mm6 did not fix
hey, i have had a breakthrough in the investigation... it turns out that some sites does not load.. but you know all about that, and a "fix" with sysctl fixes some of it.
networking was generally slow - or not!
it seems that its only HTTP transfers going insanely slow. which also probably is those ipv4 issues, so now we just need to figure out what changed, and what we need to change to fix it, so that we again can get all sites loading, and HTTP protocol fully functionel again.
hope someone has some ideas.
Signature · 2 lines
Regards, Redeeman [email protected]
Ok first take benchmarks ( use wget ), and secondly results from the internet vary day by day , hour to hour , minute by minute. Don't expect all sites on the internet to be the same speed, or even stay the same speed for that matter. For more accurate benchmark results setup a personal server on your own private network and benchmark http trasnfers using different kernels.
Matt H.
Quoted 11 lines · Redeeman
hey, i have had a breakthrough in the investigation... it turns out that some sites does not load.. but you know all about that, and a "fix" with sysctl fixes some of it.
networking was generally slow - or not!
it seems that its only HTTP transfers going insanely slow. which also probably is those ipv4 issues, so now we just need to figure out what changed, and what we need to change to fix it, so that we again can get all sites loading, and HTTP protocol fully functionel again.hope someone has some ideas.
Quoted 5 lines · Matt Heler
Ok first take benchmarks ( use wget ), and secondly results from the internet vary day by day , hour to hour , minute by minute. Don't expect all sites on the internet to be the same speed, or even stay the same speed for that matter. For more accurate benchmark results setup a personal server on your own private network and benchmark http trasnfers using different kernels.
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/s
also, the gnu ftp, where i took gcc3.4.1, it gave me 200kb/s
Quoted 23 lines
Matt H.
Quoted 11 lines · Redeeman
hey, i have had a breakthrough in the investigation... it turns out that some sites does not load.. but you know all about that, and a "fix" with sysctl fixes some of it.
networking was generally slow - or not!
it seems that its only HTTP transfers going insanely slow. which also probably is those ipv4 issues, so now we just need to figure out what changed, and what we need to change to fix it, so that we again can get all sites loading, and HTTP protocol fully functionel again.hope someone has some ideas.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 4 lines · Redeeman
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/s
That could be easily explained by the fact that the www.kernel.org ftp and http services are handled by different programs (vsftpd vs. Apache).
Erik
Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 -- | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
Quoted 9 lines · Erik Mouw
Quoted 4 lines · Redeeman
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/sThat could be easily explained by the fact that the www.kernel.org ftp and http services are handled by different programs (vsftpd vs. Apache).
yeah it could.. however it isnt. because 2.6.5 can easily take 200kb/s from kernel.org http, and it sound strange too, that with 2.6.7 ALL http adresses only give 50kb/s, and with 2.6.5 it gives 200 :>
Quoted 4 lines
Erik
Again, this isn't that sufficient enough in proving your case. Like I said before, please provide benchmarks from an http ( apache ) server on your private network to validate theese claims.
Matt H.
Quoted 22 lines · Redeeman
Quoted 9 lines · Erik Mouw
Quoted 4 lines · Redeeman
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/sThat could be easily explained by the fact that the www.kernel.org ftp and http services are handled by different programs (vsftpd vs. Apache).
yeah it could.. however it isnt. because 2.6.5 can easily take 200kb/s from kernel.org http, and it sound strange too, that with 2.6.7 ALL http adresses only give 50kb/s, and with 2.6.5 it gives 200 :>
Quoted 1 line
Erik
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 11 lines · Redeeman
Quoted 6 lines · Matt Heler
Ok first take benchmarks ( use wget ), and secondly results from the internet vary day by day , hour to hour , minute by minute. Don't expect all sites on the internet to be the same speed, or even stay the same speed for that matter. For more accurate benchmark results setup a personal server on your own private network and benchmark http trasnfers using different kernels.
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
How do you know that?
Quoted 2 lines
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/s
Trafic shaping somewhere along the route? Much more load on HTTP than FTP? Are they the very same machines? Under the exact same load? Are the servers written with the same care? Are the clients?
Quoted 1 line
also, the gnu ftp, where i took gcc3.4.1, it gave me 200kb/s
Ditto.
Unless you set up something where there aren't dozens of unknown variables and a hundred or so that you have got no chance at all to even guess what their values/effects are...
Signature · 4 lines
Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Please follow these instructions:
1. STFU 2. Log in as root 3. emerge apache 4. Set up a local web server with a large file on it 5. wget -c the file over your network with each kernel, from another
computer on the local network.
Erik's said several times now to use a local server. He's right. You're not. Probability is against him; but you're still in the box with schrodinger's cat, so you can't give any sort of guarantee either. ~ His method *does* give a controlled experiment which supplies said guarantee.
Redeeman wrote:
| On Tue, 2004-07-06 at 15:53 +0200, Erik Mouw wrote: | |>On Tue, Jul 06, 2004 at 03:25:30PM +0200, Redeeman wrote: |> |>>i am aware of this, however, what i use to benchmark is kernel.org, as i |>>can see they have alot bandwith free. |>>if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch |>>with 200kb/s |> |>That could be easily explained by the fact that the www.kernel.org ftp |>and http services are handled by different programs (vsftpd vs. |>Apache). | | yeah it could.. however it isnt. because 2.6.5 can easily take 200kb/s | from kernel.org http, and it sound strange too, that with 2.6.7 ALL http | adresses only give 50kb/s, and with 2.6.5 it gives 200 :> | |> |>Erik |> | | | - | To unsubscribe from this list: send the line "unsubscribe linux-kernel" in | the body of a message to [email protected] | More majordomo info at http://vger.kernel.org/majordomo-info.html | Please read the FAQ at http://www.tux.org/lkml/ |
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFA6wa9hDd4aOud5P8RAm1/AJ4i+9BY9x00QaRZKbB/jBSCV9AbHgCdG7bj 23bKs8ptiLOA32B816Y13vk=
=1oBo
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
John Richard Moser wrote:
| Please follow these instructions: | | 1. STFU | 2. Log in as root | 3. emerge apache | 4. Set up a local web server with a large file on it | 5. wget -c the file over your network with each kernel, from another | computer on the local network. |
Sorry, Matt and Erik, I'm busy doing other things and not paying attention.
| Erik's said several times now to use a local server. He's right. | You're not. Probability is against him; but you're still in the box | with schrodinger's cat, so you can't give any sort of guarantee either. | ~ His method *does* give a controlled experiment which supplies said | guarantee.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFA6wmihDd4aOud5P8RAkeVAKCOMdbrXX7QRbjrnEULVUASKDElFACcCvZO 2rAccJJKbD3LSv55blUPoQg=
=rZKk
-----END PGP SIGNATURE-----
Quoted 10 lines · Horst von Brand
Quoted 8 lines · Redeeman
Quoted 6 lines · Matt Heler
Ok first take benchmarks ( use wget ), and secondly results from the internet vary day by day , hour to hour , minute by minute. Don't expect all sites on the internet to be the same speed, or even stay the same speed for that matter. For more accurate benchmark results setup a personal server on your own private network and benchmark http trasnfers using different kernels.
Quoted 4 lines
Quoted 2 lines
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
How do you know that?
how i know? i dont think anyone in the matter of seconds begin to use the spare ~800mbit/s of bandwith they do not use when i try, (according to info from bwbar on kernel.org)
Quoted 15 lines
Quoted 3 lines
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/s
Trafic shaping somewhere along the route? Much more load on HTTP than FTP? Are they the very same machines? Under the exact same load? Are the servers written with the same care? Are the clients?
Quoted 1 line
also, the gnu ftp, where i took gcc3.4.1, it gave me 200kb/s
Ditto.
Unless you set up something where there aren't dozens of unknown variables and a hundred or so that you have got no chance at all to even guess what their values/effects are...
Not to sound mean about this. But either you prove your claim with benchmarks in a controlled enviroment ( that means in a private network ), or you stop trolling and complaining. The linux kernel is a free piece of software, if you don't like one version of it, then feel free to use some earlier version. Otherwise please stop.
Matt H.
Quoted 39 lines · Redeeman
Quoted 9 lines · Horst von Brand
Quoted 8 lines · Redeeman
Quoted 6 lines · Matt Heler
Ok first take benchmarks ( use wget ), and secondly results from the internet vary day by day , hour to hour , minute by minute. Don't expect all sites on the internet to be the same speed, or even stay the same speed for that matter. For more accurate benchmark results setup a personal server on your own private network and benchmark http trasnfers using different kernels.
Quoted 4 lines
Quoted 2 lines
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
How do you know that?
how i know? i dont think anyone in the matter of seconds begin to use the spare ~800mbit/s of bandwith they do not use when i try, (according to info from bwbar on kernel.org)
Quoted 7 lines
Quoted 2 lines
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/s
Trafic shaping somewhere along the route? Much more load on HTTP than FTP? Are they the very same machines? Under the exact same load? Are the servers written with the same care? Are the clients?
Quoted 7 lines
Quoted 1 line
also, the gnu ftp, where i took gcc3.4.1, it gave me 200kb/s
Ditto.
Unless you set up something where there aren't dozens of unknown variables and a hundred or so that you have got no chance at all to even guess what their values/effects are...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 7 lines · Matt Heler
Not to sound mean about this. But either you prove your claim with benchmarks in a controlled enviroment ( that means in a private network ), or you stop trolling and complaining. The linux kernel is a free piece of software, if you don't like one version of it, then feel free to use some earlier version. Otherwise please stop.
Matt H.
I've got basicly the same problem (only worse). 2.6.7(,mm5) get ~10 bytes/sec from seattletimes.com and 25 Kbytes/sec when I "echo 0 > tcp_default_win_scale; \ echo 0 > tcp_moderate_rcvbuf;". This may be open-source where you ask users to test, but asking users to setup non-trivial HARDWARE and NETWORKS seems to be asking a bit much. YMMV, etc.
I'll get intrested if it doesn't fix its self by rc time.
Thomas (who could test on a "private network", but is a lazy bastard.)
this must be some misunderstanding, i do not want to complain, and i dont hope people get that impression, i am trying to do feedback, so that issues can be fixed.
the thing about testing against a local apache, i did that, and its fast, however, i still take that with a grain of salt, because, as said before, even though that internet speed may vary from time to time, i can see that kernel.org has plenty bandwith, and when 2.6.5 then downloads with 200kb/s from http://kernel.org, and 2.6.7 only 50kb/s, this should be able to prove its some issues with 2.6.7, but thats just my opinion
Quoted 55 lines · Matt Heler
Not to sound mean about this. But either you prove your claim with benchmarks in a controlled enviroment ( that means in a private network ), or you stop trolling and complaining. The linux kernel is a free piece of software, if you don't like one version of it, then feel free to use some earlier version. Otherwise please stop.
Matt H.
Quoted 34 lines · Redeeman
Quoted 9 lines · Horst von Brand
Quoted 8 lines · Redeeman
Quoted 6 lines · Matt Heler
Ok first take benchmarks ( use wget ), and secondly results from the internet vary day by day , hour to hour , minute by minute. Don't expect all sites on the internet to be the same speed, or even stay the same speed for that matter. For more accurate benchmark results setup a personal server on your own private network and benchmark http trasnfers using different kernels.
Quoted 4 lines
Quoted 2 lines
i am aware of this, however, what i use to benchmark is kernel.org, as i can see they have alot bandwith free.
How do you know that?
how i know? i dont think anyone in the matter of seconds begin to use the spare ~800mbit/s of bandwith they do not use when i try, (according to info from bwbar on kernel.org)
Quoted 7 lines
Quoted 2 lines
if i use kernel.org http i get 50kb/s, if i use ftp, i can easily fetch with 200kb/s
Trafic shaping somewhere along the route? Much more load on HTTP than FTP? Are they the very same machines? Under the exact same load? Are the servers written with the same care? Are the clients?
Quoted 7 lines
Quoted 1 line
also, the gnu ftp, where i took gcc3.4.1, it gave me 200kb/s
Ditto.
Unless you set up something where there aren't dozens of unknown variables and a hundred or so that you have got no chance at all to even guess what their values/effects are...
Quoted 5 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 3 lines · Redeeman
this must be some misunderstanding, i do not want to complain, and i dont hope people get that impression, i am trying to do feedback, so that issues can be fixed.
Redeeman - your firewall is broken, or somebody's firewall.
Look at /proc/sys/net/ipv4/tcp_default_win_scale , if it currently contains 7, do:
echo 1 > /proc/sys/net/ipv4/tcp_default_win_scale
and retry.
Quoted 3 lines
downloads with 200kb/s from http://kernel.org, and 2.6.7 only 50kb/s, this should be able to prove its some issues with 2.6.7, but thats just my opinion
Things can be more complicated than they appear. Currently all evidence for these changes points to firewalls messing with TCP options, TCP options which used to have more default versions in older kernels.
Regards,
bert
Signature · 2 lines
http://www.PowerDNS.com Open source, database driven DNS Software http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
Quoted 6 lines · bert hubert
Quoted 3 lines · Redeeman
this must be some misunderstanding, i do not want to complain, and i dont hope people get that impression, i am trying to do feedback, so that issues can be fixed.
Redeeman - your firewall is broken, or somebody's firewall.
i dont have a firewall, but i am afraid my isp probably is doing something, after reading another thread :(
Quoted 5 lines
Look at
/proc/sys/net/ipv4/tcp_default_win_scale, if it currently contains 7, do:echo 1 >
/proc/sys/net/ipv4/tcp_default_win_scale
its 1 as default, using the tcp patch from another thread fixes so that i can connect to sites. (packages.gentoo.org etc) where before that patch came, i echo'ed 0 into it, and it worked aswell,
however i didnt get more than 50kb/s either :|
Quoted 9 lines
and retry.
Quoted 3 lines
downloads with 200kb/s from http://kernel.org, and 2.6.7 only 50kb/s, this should be able to prove its some issues with 2.6.7, but thats just my opinion
Things can be more complicated than they appear. Currently all evidence for these changes points to firewalls messing with TCP options, TCP options which used to have more default versions in older kernels.
yes, i just realised that :(
Quoted 5 lines
Regards,
bert
okay, no i tested using a hardware router as gateway and it works fine, but this could aswell be because the hardware router is having the same stuff as default has linux 2.6.5, i dont know,
what more shall i test? :)
Quoted 26 lines · bert hubert
Quoted 3 lines · Redeeman
this must be some misunderstanding, i do not want to complain, and i dont hope people get that impression, i am trying to do feedback, so that issues can be fixed.
Redeeman - your firewall is broken, or somebody's firewall.
Look at
/proc/sys/net/ipv4/tcp_default_win_scale, if it currently contains 7, do:echo 1 >
/proc/sys/net/ipv4/tcp_default_win_scaleand retry.
Quoted 3 lines
downloads with 200kb/s from http://kernel.org, and 2.6.7 only 50kb/s, this should be able to prove its some issues with 2.6.7, but thats just my opinion
Things can be more complicated than they appear. Currently all evidence for these changes points to firewalls messing with TCP options, TCP options which used to have more default versions in older kernels.
Regards,
bert
Quoted 3 lines · Redeeman
its 1 as default, using the tcp patch from another thread fixes so that i can connect to sites. (packages.gentoo.org etc) where before that patch came, i echo'ed 0 into it, and it worked aswell,
however i didnt get more than 50kb/s either :|
Redeeman, from your trace to outpost.ds9a.nl:10000 I note that something in your path removes the wscale option, or that you have turned off window scaling entirely. Can you check /proc/sys/net/ipv4/tcp_window_scaling ?
43.909623 redeeman.33083 > 213.244.168.210.10000: S 4031970603:4031970603(0)
win 5840 <mss 1322,sackOK,timestamp 23502 0> 43.909678 213.244.168.210.10000 > redeeman.33083: S 634167324:634167324(0) ack 4031970604 win 5792 <mss 1460,sackOK,timestamp 2136531455 23502> (DF) 43.951129 redeeman.33083 > 213.244.168.210.10000: . ack 1 win 5840 <nop,nop,timestamp 23543 2136531455>
I also note that you most probably have tcp_default_win_scale set to 0.
Can you confirm for me that with 2.6.7-mm6 (and exactly that version)
- you have no TCP connectivity to packages.gentoo.org by default
- you can access packages.gentoo.org with /proc/sys/net/ipv4/tcp_default_win_scale at both 1 and 0
- that speed, even with tcp_default_win_scale set to 0, is significantly lower than with stock 2.6.7, that is, if you download some big files, and measure that, and then reboot immediately to 2.6.7, things get lots faster
Alternatively, can you reboot to a kernel with the problem ("can't connect to packages.gentoo.org") and try to connect to http://outpost.ds9a.nl:10000
and tcpdump that and send me the dump (if it does in fact not work).
Regards,
bert
Signature · 2 lines
http://www.PowerDNS.com Open source, database driven DNS Software http://lartc.org Linux Advanced Routing & Traffic Control HOWTO
just entered the site again with window scaling set to 1.
i have patched my kernel with the patch from the tcp_default_win_scale thread, should i try without?
Quoted 17 lines · bert hubert
Quoted 3 lines · Redeeman
its 1 as default, using the tcp patch from another thread fixes so that i can connect to sites. (packages.gentoo.org etc) where before that patch came, i echo'ed 0 into it, and it worked aswell,
however i didnt get more than 50kb/s either :|Redeeman, from your trace to outpost.ds9a.nl:10000 I note that something in your path removes the wscale option, or that you have turned off window scaling entirely. Can you check
/proc/sys/net/ipv4/tcp_window_scaling?43.909623 redeeman.33083 > 213.244.168.210.10000: S 4031970603:4031970603(0)
win 5840 <mss 1322,sackOK,timestamp 23502 0> 43.909678 213.244.168.210.10000 > redeeman.33083: S 634167324:634167324(0) ack 4031970604 win 5792 <mss 1460,sackOK,timestamp 2136531455 23502> (DF) 43.951129 redeeman.33083 > 213.244.168.210.10000: . ack 1 win 5840 <nop,nop,timestamp 23543 2136531455>
Quoted 4 lines
I also note that you most probably have
tcp_default_win_scaleset to 0.Can you confirm for me that with 2.6.7-mm6 (and exactly that version)
- you have no TCP connectivity to packages.gentoo.org by default
yes
Quoted 3 lines
- you can access packages.gentoo.org with
/proc/sys/net/ipv4/tcp_default_win_scaleat both 1 and 0
yes
Quoted 5 lines
- that speed, even with
tcp_default_win_scaleset to 0, is significantly lower than with stock 2.6.7, that is, if you download some big files, and measure that, and then reboot immediately to 2.6.7, things get lots faster
yes
Quoted 9 lines
Alternatively, can you reboot to a kernel with the problem ("can't connect to packages.gentoo.org") and try to connect to http://outpost.ds9a.nl:10000
and tcpdump that and send me the dump (if it does in fact not work).Regards,
bert