Skip to contentSkip to conversation

ADSL/ATM linklayer tc shaping regression fix commits for stable

DISCUSSION Jesper Dangaard Brouer 5 participants 11 messages Aug 20, 2013 – Aug 23, 2013 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Greg (and google search),

ADSL/ATM linklayer shaping were broken in kernel release from 3.8 to 3.10 by commit 56b765b79 ("htb: improved accuracy at high rates").

I can see that 3.9-stable is marked EOL, and 3.8-stable also seems closed. I guess it is up to Greg, how far the commits described below can be backported. Below is the commit needed for a 3.9 backport.

This was not the only regression introduced by the regression commit, several fixes were required:

(Refactor/change commit for easy 3.9 backport step (in v3.10-rc1)):
commit 6906f4ed6f8 (htb: add HTB_DIRECT_QLEN attribute)

The following commit fixes are in 3.10-rc5:
commit 01cb71d2d47 (net_sched: restore "overhead xxx" handling) commit 5343a7f8be1 (net_sched: htb: do not mix 1ns and 64ns time units)

The following commit fixes are in 3.10-rc6:
commit 40edeff6e1c (net_sched: qdisc_get_rtab() must check data[] array) (this fix goes further back, but needed for linklayer atm, see below[1])

Refactor improvements (v3.11-rc1):
commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

The linklayer ATM/ADSL fix, reached 3.11-rc6:
commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

[1] Eric Dumazet also found a general "linklayer atm" regression (dating way-back), which could cause rate-tables to get wrongly shared between the same rates with-and-without linklayer atm settings. Addressed/fixed in (3.10-rc6:):
- commit 40edeff6e1c (net_sched: qdisc_get_rtab() must check data[] array)
When configuring two completely equal rates, the kernel detects that these two equal rates can share the same rate-table. But the kernel didn't check if the rate-table data had been modified, which is done in the linklayer atm case.
In practice, this often isn't a problem, as overhead parameter is usually combined with the linklayer parameter.

The regression commit 56b765b79 ("htb: improved accuracy at high rates"), is not all bad, it did provided a significantly improved rate shaping accuracy. The old rate-table based system, which this commit removed, would get inaccurate at rates above 100Mbit/s.

(ps. I'm the original author of http://www.adsl-optimizer.dk/ and
http://sourceforge.net/projects/adsl-optimizer/)

Signature · 5 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer
#1 lore raw
Quoted 8 lines · Jesper Dangaard Brouer

Hi Greg (and google search),

ADSL/ATM linklayer shaping were broken in kernel release from 3.8 to 3.10 by commit 56b765b79 ("htb: improved accuracy at high rates").

I can see that 3.9-stable is marked EOL, and 3.8-stable also seems closed. I guess it is up to Greg, how far the commits described below can be backported. Below is the commit needed for a 3.9 backport.

As you say, 3.8 and 3.9 are end-of-life, so there's nothing I can, or even want, to do with them, sorry.

If there are issues with 3.10, that's a different story.

Quoted 5 lines

Refactor improvements (v3.11-rc1):
commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

The linklayer ATM/ADSL fix, reached 3.11-rc6:
commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

David sends me the networking patches for the stable tree, and if he thinks these are applicable, then I'll take them.

thanks,

greg k-h

#2 lore raw
Quoted 1 line · On Tue, 20 Aug 2013 08:16:50 -0700 Greg Kroah-Hartman…

On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:

[...]

Quoted 4 lines

If there are issues with 3.10, that's a different story.

Quoted 2 lines

Refactor improvements (v3.11-rc1):
commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

Needed because it fixes the accuracy of the rate calc.

Quoted 2 lines
Quoted 2 lines

The linklayer ATM/ADSL fix, reached 3.11-rc6:
commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

Needed because we broke userspace interface.

Quoted 2 lines

David sends me the networking patches for the stable tree, and if he thinks these are applicable, then I'll take them.

DaveM, please? What do you want me to do, submit this req/patches somehow?

(Can see that is not currently on your stable queue via:
http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

Signature · 5 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer
#4 lore raw
In reply to a message not in the archive
Quoted 19 lines · Jesper Dangaard Brouer
Quoted 1 line · On Tue, 20 Aug 2013 08:16:50 -0700 Greg Kroah-Hartman…

On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:

[...]

Quoted 4 lines

If there are issues with 3.10, that's a different story.

Quoted 2 lines

Refactor improvements (v3.11-rc1):
commit 130d3d68b52 (net_sched: psched_ratecfg_precompute()

improvements)

Needed because it fixes the accuracy of the rate calc.

Quoted 2 lines
Quoted 2 lines

The linklayer ATM/ADSL fix, reached 3.11-rc6:
commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

Needed because we broke userspace interface.

Quoted 9 lines
Quoted 2 lines

David sends me the networking patches for the stable tree, and if he thinks these are applicable, then I'll take them.

DaveM, please? What do you want me to do, submit this req/patches somehow?

(Can see that is not currently on your stable queue via:
http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

+1 Dealing with the busted DSL stuff since 3.8 has been no fun and it would be nice if these last fixes made it into 3.10.X at least so router/fw distros can pick them up more easily. The difference in dsl behavior since the bustage in 3.8 to the saner now is remarkable.

Probably not meeting the criteria for 3.10-stable inclusion (as they aren't exactly bugfixes) are the tiny improvements to the flow dissector made in 3.11 for better ipip, ipv6 encapsulation, and 802.1ad support, but I thought I'd mention them so other fq_codel folk pick them up that need them.

Not ready for any tree and increasingly doubtful for 3.12 are some improvements under test to codel and cisco's sch_pie, if anyone wants to play with them, let me know off list.

Quoted 7 lines

--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer

Signature · 4 lines
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

_______________________________________________
Bloat mailing list Bloat-JXvr2/[email protected]
https://lists.bufferbloat.net/listinfo/bloat

maintainer
#5 lore raw
In reply to Jesper Dangaard Brouer

From: Jesper Dangaard Brouer <[email protected]>
Date: Wed, 21 Aug 2013 14:03:50 +0200

Quoted 18 lines
Quoted 1 line · On Tue, 20 Aug 2013 08:16:50 -0700 Greg Kroah-Hartman…

On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:

[...]

Quoted 4 lines

If there are issues with 3.10, that's a different story.

Quoted 2 lines

Refactor improvements (v3.11-rc1):
commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

Needed because it fixes the accuracy of the rate calc.

Quoted 2 lines
Quoted 2 lines

The linklayer ATM/ADSL fix, reached 3.11-rc6:
commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

Needed because we broke userspace interface.

Quoted 7 lines
Quoted 2 lines

David sends me the networking patches for the stable tree, and if he thinks these are applicable, then I'll take them.

DaveM, please? What do you want me to do, submit this req/patches somehow?

(Can see that is not currently on your stable queue via:
http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

I'll put it there, you just have to formally ask me to do it that's all.

Asking Greg to do it directly is a complete waste of time. :-)

#6 lore raw
Quoted 24 lines · On Wed, 21 Aug 2013 11:20:23 -0700 (PDT) David Miller <[email protected]>…

From: Jesper Dangaard Brouer <[email protected]>
Date: Wed, 21 Aug 2013 14:03:50 +0200

Quoted 21 lines
Quoted 1 line · On Tue, 20 Aug 2013 08:16:50 -0700 Greg Kroah-Hartman…

On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:

[...]

Quoted 4 lines

If there are issues with 3.10, that's a different story.

Quoted 2 lines

Refactor improvements (v3.11-rc1):
commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

Needed because it fixes the accuracy of the rate calc.

Quoted 2 lines
Quoted 2 lines

The linklayer ATM/ADSL fix, reached 3.11-rc6:
commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

Needed because we broke userspace interface.

Quoted 2 lines

David sends me the networking patches for the stable tree, and if he thinks these are applicable, then I'll take them.

Quoted 7 lines
Quoted 4 lines

DaveM, please? What do you want me to do, submit this req/patches somehow?

(Can see that is not currently on your stable queue via:
http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

I'll put it there, you just have to formally ask me to do it that's all.

Thanks, I assume that this was formally enough and you are taking the patches.

So, for future reference:

Stable patches for the networking tree, I should:
1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
to see if my patch is already on your stable queue
2) if not, ask you and cc [email protected]
3) you will handle the interaction with Greg and [email protected]

Correct?

Quoted 1 line

Asking Greg to do it directly is a complete waste of time. :-)

I'll remember that. I was just following Documentation/stable_kernel_rules.txt, I guess I should have known better as all my previous stable patches have automagically been picked up and processed by you :-)

Signature · 5 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer
maintainer
#7 lore raw

From: Jesper Dangaard Brouer <[email protected]>
Date: Thu, 22 Aug 2013 10:04:26 +0200

Quoted 9 lines

So, for future reference:

Stable patches for the networking tree, I should:
1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
to see if my patch is already on your stable queue
2) if not, ask you and cc [email protected]
3) you will handle the interaction with Greg and [email protected]

Correct?

Yes, and that's how it's been with the networking for years.

#8 lore raw
Quoted 11 lines · On Thu, 22 Aug 2013 01:23:48 -0700 (PDT) David Miller <[email protected]>…

From: Jesper Dangaard Brouer <[email protected]>
Date: Thu, 22 Aug 2013 10:04:26 +0200

Quoted 8 lines

So, for future reference:

Stable patches for the networking tree, I should:
1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
to see if my patch is already on your stable queue
2) if not, ask you and cc [email protected]
3) you will handle the interaction with Greg and [email protected]

Quoted 3 lines
Quoted 1 line

Correct?

Yes, and that's how it's been with the networking for years.

Yes, and in the future, documentation of this will be available here:
Documentation/networking/netdev-FAQ.txt

Right now is only available via net-next:
https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/tree/Documentation/networking/netdev-FAQ.txt
or
https://lwn.net/Articles/559211/

Guess, we should add a pointer to this new doc from Documentation/stable_kernel_rules.txt once it hits Linus'es tree.

Now I cannot find any more ways to spell-it-out, for people searching google ;-)

Signature · 5 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer
#9 lore raw
In reply to David Miller

Hi David,

Quoted 11 lines · David Miller

From: Jesper Dangaard Brouer <[email protected]>
Date: Thu, 22 Aug 2013 10:04:26 +0200

Quoted 8 lines

So, for future reference:

Stable patches for the networking tree, I should:
1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
to see if my patch is already on your stable queue
2) if not, ask you and cc [email protected]
3) you will handle the interaction with Greg and [email protected]

Quoted 4 lines
Quoted 1 line

Correct?

Yes, and that's how it's been with the networking for years.

How about these two stable requests?

http://article.gmane.org/gmane.linux.kernel.stable/61247/match=weidong
http://article.gmane.org/gmane.linux.kernel.stable/61248/match=weidong

#11 lore raw
In reply to David Miller

Hi David,

Quoted 11 lines · David Miller

From: Jesper Dangaard Brouer <[email protected]>
Date: Thu, 22 Aug 2013 10:04:26 +0200

Quoted 8 lines

So, for future reference:

Stable patches for the networking tree, I should:
1) check http://patchwork.ozlabs.org/bundle/davem/stable/?state=*
to see if my patch is already on your stable queue
2) if not, ask you and cc [email protected]
3) you will handle the interaction with Greg and [email protected]

Quoted 4 lines
Quoted 1 line

Correct?

Yes, and that's how it's been with the networking for years.

How about these two stable requests?

http://article.gmane.org/gmane.linux.kernel.stable/61247
http://article.gmane.org/gmane.linux.kernel.stable/61248