e1000e: reset of tx_queue_len
Subsystems (2)
Hi all,
while working on a new server, I noticed that tx_queue_len is reset to the default of 1000 while ADDRCONF(NETDEV_CHANGE). This happens with 2.6.30.8, but I could not see any obvious differences to the current state of e1000e, while trying to find the problem in the code.
The interface has been configured including a customized qlen via /etc/network/interfaces. The ports are not connected at startup so obviously the Kernel reports
ADDRCONF(NETDEV_UP): eth2: link is not ready
while ifconfig properly displays the customized qlen.
When plugging the server tx_queue_len is reset to 1000.
Any help is appreciated. Thanks.
Franco
Quoted 5 lines · Franco Fichtner
while working on a new server, I noticed that
tx_queue_lenis reset to the default of 1000 whileADDRCONF(NETDEV_CHANGE). This happens with 2.6.30.8, but I could not see any obvious differences to the current state of e1000e, while trying to find the problem in the code.
The driver tries to set the queuelen back to 1000 if the link changes speed, because the driver tries to shorten the txqueuelen to 100 to reduce latency when the link changes to 100/10 Mb/s
Quoted 5 lines
The interface has been configured including a customized qlen via
/etc/network/interfaces. The ports are not connected at startup so obviously the Kernel reports
ADDRCONF(NETDEV_UP): eth2: link is not ready
do your scripts try to configure the queuelen when link is not up? Why are the ports not connected at startup?
Quoted 6 lines
while ifconfig properly displays the customized qlen.
When plugging the server
tx_queue_lenis reset to 1000.Any help is appreciated. Thanks.
you can safely comment out the code in netdev.c on lines 3532 through 3549, as well as any other lines of code in the driver that modify or use the variable netdev->tx_queue_len
Its possible we should just remove this code from the in-kernel driver, as I don't think many other drivers mess with the tx_queue_len either. We did introduce the code to fix some users complaints about the amount of work that can get queued up in the hardware with tx_queue_len = 1000 when at 10/100 speeds.
Hope this helps!
Quoted 11 lines · Jesse
Quoted 6 lines · Franco Fichtner
while working on a new server, I noticed that
tx_queue_lenis reset to the default of 1000 whileADDRCONF(NETDEV_CHANGE). This happens with 2.6.30.8, but I could not see any obvious differences to the current state of e1000e, while trying to find the problem in the code.The driver tries to set the queuelen back to 1000 if the link changes speed, because the driver tries to shorten the txqueuelen to 100 to reduce latency when the link changes to 100/10 Mb/s
Thanks, that explains it. Since we don't connect ports at startup, the plugging causes a speed "change".
Quoted 9 lines
Quoted 6 lines
The interface has been configured including a customized qlen via
/etc/network/interfaces. The ports are not connected at startup so obviously the Kernel reports
ADDRCONF(NETDEV_UP): eth2: link is not readydo your scripts try to configure the queuelen when link is not up? Why are the ports not connected at startup?
I have a modified up command that sets appropriate values of mtu and qlen via ip. The device is brought up at startup automatically with the correct values. At least that's what ifconfig shows even though the port is not yet connected.
There has been a lot of testing back and forth with single hosts and much larger quantities of traffic. This got really messy when testing single host first (connected via 100 Mb/s, so qlen to 100) and then switching to 1000 Mb/s. qlen would still be 100, causing lots of packet drops. This took a few days to track down, unfortunately.
Quoted 1 line
Hope this helps
Greatly appreciated, thanks again. :)
Franco
Hi Jesse,
Quoted 2 lines
Its possible we should just remove this code from the in-kernel driver, as I don't think many other drivers mess with the
tx_queue_leneither. We did introduce the code to fix some users complaints about the amount of work that can get queued up in the hardware withtx_queue_len= 1000 when at 10/100 speeds.
I've tested the changes you suggested and now tx_queue_len is no longer reset. I have also prepared a patch for net-next for you to sign or ack (or nack) because I'd like to see this go, also.
Thanks again,
Franco