HFSC
Quoted 4 lines · Patrick McHardy
This is currently all there is. If you have some specific questions, just ask (but please CC lartc). If anyone wants to write some documentation I'd be happy to help, but I don't have time for it myself.
I am not sure if the original poster has specific questions, but I sure do.
I just recently got into this HFSC mess myself, so I'm a bit fuzzy in all the terms and differences in implementation. I read the paper (SIGCOM97) on HFSC and I think I understood most of it. But there are some things in the implementation that I couldn't really realize.
I'll quote the usage here for reference:
,----
| Usage: ... hfsc [ rt SC ] [ ls SC ] [ ul SC ] | | SC := [ [ m1 BPS ] [ d SEC ] m2 BPS | | m1 : slope of first segment | d : x-coordinate of intersection | m2 : slope of second segment
`----
Okay, the SC parameters I think I understand rather well - they are there to define the service curve itself. But the way hfsc takes three optional parameters of service curves puzzles me.
I believe 'rt' referes to 'Real-Time Service Curve', 'ls' to 'Link Sharing Service Curve' and 'ul' to 'Upper Limit Service Curve'. If I understand correctly, the SIGCOM97 paper mentioned that the link sharing selection need not be the same as the real time selection, but in examples assumed for simplicity that they were. Also, from the source I conclude that 'Upper Limit Service Curve' cannot be specified without 'Link Sharing Service Curve'.
So, this, all in all, baffles me :-)
The possible combinations I can make here are:
Real-Time
Link-Sharing
Real-Time, Link-Sharing
Link-Sharing, Upper-Limit
Real-Time, Link-Sharing, Upper-Limit
How do these behave? If I specify *only* the real-time curve, what is used for the link-sharing part? Or does that mean that there is no sharing? Or if I only specify the link-sharing curve, does that mean that no specific deadlines are for packets, just that they are sent based on the link-sharing model? And what actually is the upper-limit service curve? I take it that it is some kind of a packet drop curve, but I don't know how it would behave - nor why it would require the link-sharing curve.
So, any pointers on these would be helpful, or if you manage to get the time to explain it specifically.
I will probably cook up atleast an example script using HFSC for normal QoS if I manage to understand how it works, perhaps even some documentation.
-- Naked
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 32 lines · Nuutti Kotivuori
Quoted 4 lines · Patrick McHardy
This is currently all there is. If you have some specific questions, just ask (but please CC lartc). If anyone wants to write some documentation I'd be happy to help, but I don't have time for it myself.
I am not sure if the original poster has specific questions, but I sure do.
I just recently got into this HFSC mess myself, so I'm a bit fuzzy in all the terms and differences in implementation. I read the paper (SIGCOM97) on HFSC and I think I understood most of it. But there are some things in the implementation that I couldn't really realize.
I'll quote the usage here for reference:
,----
| Usage: ... hfsc [ rt SC ] [ ls SC ] [ ul SC ] | | SC := [ [ m1 BPS ] [ d SEC ] m2 BPS | | m1 : slope of first segment | d : x-coordinate of intersection | m2 : slope of second segment`----
Okay, the SC parameters I think I understand rather well - they are there to define the service curve itself. But the way hfsc takes three optional parameters of service curves puzzles me.
Quoted 7 lines
I believe 'rt' referes to 'Real-Time Service Curve', 'ls' to 'Link Sharing Service Curve' and 'ul' to 'Upper Limit Service Curve'. If I understand correctly, the SIGCOM97 paper mentioned that the link sharing selection need not be the same as the real time selection, but in examples assumed for simplicity that they were. Also, from the source I conclude that 'Upper Limit Service Curve' cannot be specified without 'Link Sharing Service Curve'.
Correct so far.
Quoted 10 lines
So, this, all in all, baffles me :-)
The possible combinations I can make here are:
Real-Time
Link-Sharing
Real-Time, Link-Sharing
Link-Sharing, Upper-Limit
Real-Time, Link-Sharing, Upper-Limit
Quoted 8 lines
How do these behave? If I specify *only* the real-time curve, what is used for the link-sharing part? Or does that mean that there is no sharing? Or if I only specify the link-sharing curve, does that mean that no specific deadlines are for packets, just that they are sent based on the link-sharing model? And what actually is the upper-limit service curve? I take it that it is some kind of a packet drop curve, but I don't know how it would behave - nor why it would require the link-sharing curve.
The combinations you list are correct. Real-time curves are only valid for leaf-classes, whereas link-sharing and upper-limit curves are valid for all classes in the heirarchy. When multiple curves are used, the following must hold: rt <= ls <= ul
To understand why there are two (forgetting about upper-limit curves for now) different curves your need to know that scheduling in HFSC is based on two criteria: the real-time criterion which ensures that the guarantees of leaf-classes are met and the link-sharing criterion which tries to satisfy the service curves of intermediate classes and distributes excess bandwidth fairly. The reason why there are two different criteria is that in the Fair Service Link-sharing model that is approximated by HFSC it is not always possible to guarantee the service of all classes simultaneously at all times (with non-linear service curves). HFSC chooses to guarantee the service curves of (real-time) leaf-classes (because only leaves carry packets), and uses the link-sharing criterion to minimize the discrepancy between the actual service received and the service defined by the Fair Service Link-sharing Model.
The upper-limit curve is used to limit the link-sharing curve. Without an upper-limit curve, packets are dequeued at the speed the underlying device is capable of. For example in the case of software devices, this is not very desireable, so you can limit the total output rate.
For your other questions:
- If you specify only a real-time service curve the class will not participate in link-sharing. This means it can only send at it's configured rate. The difference two a link-share+upper-limit curve is that the service is guaranteed.
- If you specify only a link-share curve their are no deadlines and no guarantees can be given.
Some other notes you might find helpful:
- The sum of all real-time curves must not exceed 100%.
- The actual bandwidth assigned to link-sharing classes doesn't matter, only the relative difference between sibling-classes is important
Quoted 7 lines
So, any pointers on these would be helpful, or if you manage to get the time to explain it specifically.
I will probably cook up atleast an example script using HFSC for normal QoS if I manage to understand how it works, perhaps even some documentation.
Great! I started some documentation last year, but never got to finishing it. I can send it to you in private, but I don't want to publish it as long as it's unfinished.
Regards
Patrick
Quoted 3 lines
-- Naked
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Thanks for your answers Patrick.
now I'm able to start testing :)
btw, i would appreciate if you could send me the documentation u started.
Thanks in advance.
Signature · 3 lines
_______________________________________________ LARTC mailing list / [email protected] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 3 lines · Patrick McHardy
The combinations you list are correct. Real-time curves are only valid for leaf-classes, whereas link-sharing and upper-limit curves are valid for all classes in the heirarchy.
Right, after a bit of experimentation and thinking, I realized this.
Quoted 2 lines
When multiple curves are used, the following must hold:
rt <= ls <= ul
If this is for all t, in practise this means:
if d > 0
m1(rt) <= m1(ls) <= m1(ul)
m2(rt) <= m2(ls) <= m2(ul)
if m1 < m2
d(rt) >= d(ls) >= d(ul)
elsif m1 > m2
d(rt) <= d(ls> <= d(ul)
else
d irrelevant
else
m1 irrelevant
m2(rt) <= m2(ls) <= m2(ul)Am I correct? What happens if these values are violated? Are any errors signalled?
Also, I have very little clue why these must hold as such.
Obviously if a link-sharing curve is smaller than the real-time curve, then when the class participates in link-sharing, it would have to have sent less than it already has. But if this is so, does the algorithm break totally, or does that only mean that the class does not participate in link-sharing before the excess bandwidth share for the class based on the relative link-sharing service curve goes above the real-time service curve? The latter would not necessarily an unwanted behaviour.
Then if the upper-limit service curve is smaller than the link-sharing curve, what would this cause? Naive assumptions would lead me to think that it would merely mean that the class participates in link-sharing based on the relative service curve it has, but never ends up taking more than what the upper-limit service curve dictates. Eg. in a case with a relatively large link-sharing service curve and a smaller upper-limit service curve the class would get a big share out of a small amount of excess bandwidth shared, but as bandwidth to share is increased, upper-limit service curve will limit it to a constant limit. Or again, does the algorithm break somehow?
And I am even more confused when I think what this means for the interior classes and their service curves. Apparently the service curves for parent classes are respected, but does that mean that the service curve for a parent class would have to be equal to or larger than the sum of all child classes service curves? If so, how does one calculate this? It would be an n-ary curve if calculated exactly. Or if not so, then what happens if the service curves of child classes exceed the service curve of the parent class? Obviously here we are talking only about link-sharing service curves (and upper-limit service curves) as real-time service curves are always fulfilled and only handled by leaf classes.
Quoted 6 lines
To understand why there are two (forgetting about upper-limit curves for now) different curves your need to know that scheduling in HFSC is based on two criteria: the real-time criterion which ensures that the guarantees of leaf-classes are met and the link-sharing criterion which tries to satisfy the service curves of intermediate classes and distributes excess bandwidth fairly. The reason why
Quoted 8 lines
there are two different criteria is that in the Fair Service Link-sharing model that is approximated by HFSC it is not always possible to guarantee the service of all classes simultaneously at all times (with non-linear service curves). HFSC chooses to guarantee the service curves of (real-time) leaf-classes (because only leaves carry packets), and uses the link-sharing criterion to minimize the discrepancy between the actual service received and the service defined by the Fair Service Link-sharing Model.
Right. I think I understand this, but I am not so certain of the implications for actual use.
Quoted 5 lines
The upper-limit curve is used to limit the link-sharing curve. Without an upper-limit curve, packets are dequeued at the speed the underlying device is capable of. For example in the case of software devices, this is not very desireable, so you can limit the total output rate.
I came to this conclusion by experimentation. So upper-limit service curve can be used to shape link-sharing usage - but real-time service curves are fulfilled regardless of it? So the end result would be that a class with only a real-time service curve throttles itself to the rate, with a link-sharing service curve becomes work-conserving and with an upper-limit service curve it throttles itself again.
Quoted 8 lines
For your other questions:
- If you specify only a real-time service curve the class will not participate in link-sharing. This means it can only send at it's configured rate. The difference two a link-share+upper-limit curve is that the service is guaranteed.- If you specify only a link-share curve their are no deadlines and no guarantees can be given.
Right. After a while I realized this. But again I am somewhat uncertain of the ramifications.
If we assume classes that have real-time service curves equal to link-sharing service curves, and compare those to classes that have only link-sharing service curves, in an environment where there is excess bandwidth to share, how will the results be different? If there is a difference, is it because the link-sharing algorithm might choose to not fulfill the service curve of a class immediately in the interest of exact fairness, where the real-time constraint would have forced to abandon fairness for a moment to fulfill the service curve?
I am really bad at explaining what I mean in these things, which just shows my uncertainty in the subject.
Quoted 2 lines
Some other notes you might find helpful:
- The sum of all real-time curves must not exceed 100%.
100% of what? Actual link capacity? And does this mean for any time t in the service curve? How do I confirm that a bunch of service curves with different m1, d and m2 values never end up exceeding 100% anywhere?
And what happens if they do? Obviously there's no bandwidth shared, but does the algorithm still try to fulfill the requirements as well as it can, picking the real-time class that is behind the schedule the most? Or will some class(es) end up dominating the link? Or will the whole algorithm break down because some variable grows negative or something?
Quoted 3 lines
- The actual bandwidth assigned to link-sharing classes doesn't matter, only the relative difference between sibling-classes is important
If only relative difference matters, why must link-sharing service curves be larger than real-time service curves? And smaller than upper-limit service curves?
Quoted 3 lines
Great! I started some documentation last year, but never got to finishing it. I can send it to you in private, but I don't want to publish it as long as it's unfinished.
Large parts of the HFSC algorithm seem to be still unclear to me, especially in all the edge cases - the common case seems rather straightforward and I have even gotten it to work, at times.
I also seem to have a lot of difficulties in trying to simulate the behaviour of the qdisc. I managed to build 'tcsim' with support for HFSC, but I have been unable to get any reasonable results with it with any qdisc at all. Even 'examples/sfq' which is very trivial seems to produce almost random results when I change the 'perturb' parameter, even though it should hardly make a difference, unless one is unlucky enough to get two flows hashed in the same bucket, which is extremely unlikely with only 3 flows. I have attempted to use a couple other tools as well, and haven't been too successful.
Also, something as trivial as this:
tc qdisc add dev $DEV root handle 1: hfsc default 1 tc class add dev $DEV parent 1: classid 1:1 hfsc rt m2 100kbps
seems to work for 'eth0' but not 'lo' interface, where as for example the 'tbf' qdisc does work for 'lo' as well. If I run those commands on 'lo', every packet shows up as dropped by the qdisc.
In any case, I am not going to write anything about HFSC before I can get even the basics cleared up to myself :-)
Thanks for sorting this thing out.
-- Naked
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Late reply but here it is ;)
Quoted 21 lines · Nuutti Kotivuori
Quoted 2 lines · Patrick McHardy
When multiple curves are used, the following must hold:
rt <= ls <= ulIf this is for all t, in practise this means:
if d > 0
m1(rt) <= m1(ls) <= m1(ul) m2(rt) <= m2(ls) <= m2(ul) if m1 < m2 d(rt) >= d(ls) >= d(ul) elsif m1 > m2 d(rt) <= d(ls> <= d(ul) else d irrelevant else m1 irrelevant m2(rt) <= m2(ls) <= m2(ul)
Quoted 2 lines
Am I correct? What happens if these values are violated? Are any errors signalled?
I think it can be expressed easier like this:
b(t) {
m1 * t t <= d
m1 * d + m2 * (t - d) t > d
}b_rt(t) <= b_ls(t) <= b_ul(t) for all t >= 0
No error is signalled when these are violated.
Quoted 10 lines
Also, I have very little clue why these must hold as such.
Obviously if a link-sharing curve is smaller than the real-time curve, then when the class participates in link-sharing, it would have to have sent less than it already has. But if this is so, does the algorithm break totally, or does that only mean that the class does not participate in link-sharing before the excess bandwidth share for the class based on the relative link-sharing service curve goes above the real-time service curve? The latter would not necessarily an unwanted behaviour.
The later is correct, the class will participate in link-sharing, but will only be selected by the real-time criterion under full load. It will also be punished later wrt. excess bandwidth as long as the parent constantly stays active.
Quoted 10 lines
Then if the upper-limit service curve is smaller than the link-sharing curve, what would this cause? Naive assumptions would lead me to think that it would merely mean that the class participates in link-sharing based on the relative service curve it has, but never ends up taking more than what the upper-limit service curve dictates. Eg. in a case with a relatively large link-sharing service curve and a smaller upper-limit service curve the class would get a big share out of a small amount of excess bandwidth shared, but as bandwidth to share is increased, upper-limit service curve will limit it to a constant limit. Or again, does the algorithm break somehow?
It will still respect the upper-limit curve, but I'm not sure about the consequences for sibling-classes and the parent's active child list, I need to think about this some more. In any case it's not advisable to do so.
Quoted 11 lines
And I am even more confused when I think what this means for the interior classes and their service curves. Apparently the service curves for parent classes are respected, but does that mean that the service curve for a parent class would have to be equal to or larger than the sum of all child classes service curves? If so, how does one calculate this? It would be an n-ary curve if calculated exactly. Or if not so, then what happens if the service curves of child classes exceed the service curve of the parent class? Obviously here we are talking only about link-sharing service curves (and upper-limit service curves) as real-time service curves are always fulfilled and only handled by leaf classes.
The sum of all realtime service curves must not be bigger than the service curve of the link itself, otherwise the service can't be guaranteed. For link-sharing curves, it's actually not important that they don't exceed their parent because they only define a share, not an absolute amount of service. Only the relative differences between siblings matter.
Adding n curves gives you (in the worst case) a (n+1)-ary curve, you can calculate it like this:
sc1: m1 = 100kbit, d = 1s, m2 = 200kbit
sc2: m1 = 50kbit, d = 0.25s, m2 = 300kbit
sc3: m1 = 200kbit, d = 1.5s, m2 = 500kbit
-----------------------------------------
m {
350kbit d <= 0.25s
600kbit 0.25s < d <= 1s
700kbit 1s < d <= 1.5s
1000kbit d > 1.5s
}Quoted 7 lines
Quoted 5 lines
The upper-limit curve is used to limit the link-sharing curve. Without an upper-limit curve, packets are dequeued at the speed the underlying device is capable of. For example in the case of software devices, this is not very desireable, so you can limit the total output rate.
Quoted 6 lines
I came to this conclusion by experimentation. So upper-limit service curve can be used to shape link-sharing usage - but real-time service curves are fulfilled regardless of it? So the end result would be that a class with only a real-time service curve throttles itself to the rate, with a link-sharing service curve becomes work-conserving and with an upper-limit service curve it throttles itself again.
Exactly.
Quoted 13 lines
Quoted 8 lines
For your other questions:
- If you specify only a real-time service curve the class will not participate in link-sharing. This means it can only send at it's configured rate. The difference two a link-share+upper-limit curve is that the service is guaranteed.- If you specify only a link-share curve their are no deadlines and no guarantees can be given.
Right. After a while I realized this. But again I am somewhat uncertain of the ramifications.
Quoted 8 lines
If we assume classes that have real-time service curves equal to link-sharing service curves, and compare those to classes that have only link-sharing service curves, in an environment where there is excess bandwidth to share, how will the results be different? If there is a difference, is it because the link-sharing algorithm might choose to not fulfill the service curve of a class immediately in the interest of exact fairness, where the real-time constraint would have forced to abandon fairness for a moment to fulfill the service curve?
If it is possible to fulfill all demands with the available excess-bandwidth than there is no difference. The real difference is of a different kind. A parents link-sharing service curve might be violated by the real-time criterion used for one of it's children. The parents' siblings will suffer from this as well (link-sharing wise) because they share the same parent and part of the service given to all siblings of this parent has been used in violation of link-sharing, so link-sharing only leaves will suffer. An example for this is in the HFSC paper on page 6.
Quoted 6 lines
Quoted 1 line
- The sum of all real-time curves must not exceed 100%.
100% of what? Actual link capacity? And does this mean for any time t in the service curve? How do I confirm that a bunch of service curves with different m1, d and m2 values never end up exceeding 100% anywhere?
Yes, actually link capacity. Already explained above.
Quoted 6 lines
And what happens if they do? Obviously there's no bandwidth shared, but does the algorithm still try to fulfill the requirements as well as it can, picking the real-time class that is behind the schedule the most? Or will some class(es) end up dominating the link? Or will the whole algorithm break down because some variable grows negative or something?
Nothing bad will happen, only the guarantees can't be met anymore. It will still pick the class with the smallest deadline.
Quoted 9 lines
Quoted 4 lines
- The actual bandwidth assigned to link-sharing classes doesn't matter, only the relative difference between sibling-classes is important
If only relative difference matters, why must link-sharing service curves be larger than real-time service curves? And smaller than upper-limit service curves?
They don't. It just makes it simpler to assure that the service given to a class is at least the amount defined by the real-time curve, which is usually what you want.
Quoted 9 lines
I also seem to have a lot of difficulties in trying to simulate the behaviour of the qdisc. I managed to build 'tcsim' with support for HFSC, but I have been unable to get any reasonable results with it with any qdisc at all. Even 'examples/sfq' which is very trivial seems to produce almost random results when I change the 'perturb' parameter, even though it should hardly make a difference, unless one is unlucky enough to get two flows hashed in the same bucket, which is extremely unlikely with only 3 flows. I have attempted to use a couple other tools as well, and haven't been too successful.
Have you applied the patches from trash.net/~kaber/hfsc/tcsim ? With HZ00 and PSCHED_GETTIMEOFDAY as clocksource I got very good results.
Quoted 8 lines
Also, something as trivial as this:
tc qdisc add dev $DEV root handle 1: hfsc default 1 tc class add dev $DEV parent 1: classid 1:1 hfsc rt m2 100kbps
seems to work for 'eth0' but not 'lo' interface, where as for example the 'tbf' qdisc does work for 'lo' as well. If I run those commands on 'lo', every packet shows up as dropped by the qdisc.
Works ok here .. do you mean inside tcsim ?
Regards
Patrick
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 1 line · Patrick McHardy
Late reply but here it is ;)
No worries - it wasn't exactly brief and I did have other stuff to spend my time on.
BTW the Blue scheduler patch for 2.6 seems to be working nicely - but I haven't had the time to run the tests on it that I wished to, so I haven't posted anything further about it.
Quoted 2 lines
Quoted 1 line · Nuutti Kotivuori
Patrick McHardy wrote:
[...]
Quoted 8 lines
I think it can be expressed easier like this:
b(t) > {
m1 * t t <= d
m1 * d + m2 * (t - d) t > d
}
b_rt(t)<=b_ls(t)<=b_ul(t)for all t >= 0
Yes, certainly - I just wished to eliminate t from it all.
Quoted 1 line
No error is signalled when these are violated.
Right.
Quoted 4 lines
The later is correct, the class will participate in link-sharing, but will only be selected by the real-time criterion under full load. It will also be punished later wrt. excess bandwidth as long as the parent constantly stays active.
Ah, yes. Makes perfect sense.
Quoted 4 lines
It will still respect the upper-limit curve, but I'm not sure about the consequences for sibling-classes and the parent's active child list, I need to think about this some more. In any case it's not advisable to do so.
Okay.
Quoted 3 lines
The sum of all realtime service curves must not be bigger than the service curve of the link itself, otherwise the service can't be guaranteed.
Nod.
Quoted 4 lines
For link-sharing curves, it's actually not important that they don't exceed their parent because they only define a share, not an absolute amount of service. Only the relative differences between siblings matter.
Makes sense.
Quoted 7 lines
Adding n curves gives you (in the worst case) a (n+1)-ary curve, you can calculate it like this:
sc1: m1 = 100kbit, d = 1s, m2 = 200kbit
sc2: m1 = 50kbit, d = 0.25s, m2 = 300kbit
sc3: m1 = 200kbit, d = 1.5s, m2 = 500kbit
-----------------------------------------
Quoted 6 lines
m > {
350kbit d <= 0.25s
600kbit 0.25s < d <= 1s
700kbit 1s < d <= 1.5s
1000kbit d > 1.5s
}Right. I think there's a need for a small tool to make these calculations - and perhaps even to automatically be able to scale other curves to maintain the restrictions. But that is for the future, manual calculation will do for now.
Quoted 9 lines
If it is possible to fulfill all demands with the available excess-bandwidth than there is no difference. The real difference is of a different kind. A parents link-sharing service curve might be violated by the real-time criterion used for one of it's children. The parents' siblings will suffer from this as well (link-sharing wise) because they share the same parent and part of the service given to all siblings of this parent has been used in violation of link-sharing, so link-sharing only leaves will suffer. An example for this is in the HFSC paper on page 6.
Right, I think I understand this now.
Quoted 1 line
Quoted 1 line
Quoted 1 line
- The sum of all real-time curves must not exceed 100%.
[...]
Quoted 3 lines
Yes, actually link capacity. Already explained above.
Quoted 1 line
And what happens if they do?
[...]
Quoted 2 lines
Nothing bad will happen, only the guarantees can't be met anymore. It will still pick the class with the smallest deadline.
Okay, this is good to know.
Quoted 7 lines
Quoted 3 lines
If only relative difference matters, why must link-sharing service curves be larger than real-time service curves? And smaller than upper-limit service curves?
They don't. It just makes it simpler to assure that the service given to a class is at least the amount defined by the real-time curve, which is usually what you want.
Exactly.
Quoted 2 lines
Quoted 2 lines
I also seem to have a lot of difficulties in trying to simulate the behaviour of the qdisc.
[...]
Quoted 3 lines
Have you applied the patches from trash.net/~kaber/hfsc/tcsim ? With HZ00 and
PSCHED_GETTIMEOFDAYas clocksource I got very good results.
I tried HZ00 and HZ0 both, and the results were odd. But I think I didn't touch the clocksource at all. I will try later on with PSCHED_GETTIMEOFDAY as well.
Quoted 8 lines
Quoted 6 lines
Also, something as trivial as this:
tc qdisc add dev $DEV root handle 1: hfsc default 1
tc class add dev $DEV parent 1: classid 1:1 hfsc rt m2 100kbps seems to work for 'eth0' but not 'lo' interface, where as for example the 'tbf' qdisc does work for 'lo' as well. If I run those commands on 'lo', every packet shows up as dropped by the qdisc.Works ok here .. do you mean inside tcsim ?
No, I don't mean inside tcsim. Here is a full transcript:
*****
shiro:~# export DEV=lo shiro:~# tc -s -d qdisc show dev $DEV shiro:~# ping localhost -c 4 PING shiro.i.naked.iki.fi (127.0.0.1) 56(84) bytes of data. 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=1 ttld time=0.062 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=2 ttld time=0.059 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=3 ttld time=0.059 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=4 ttld time=0.060 ms
--- shiro.i.naked.iki.fi ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms rtt min/avg/max/mdev = 0.059/0.060/0.062/0.001 ms
shiro:~# tc qdisc add dev $DEV root handle 1: hfsc default 1 shiro:~# tc class add dev $DEV parent 1: classid 1:1 hfsc rt m2 100kbps shiro:~# ping localhost -c 4 PING shiro.i.naked.iki.fi (127.0.0.1) 56(84) bytes of data. ^C --- shiro.i.naked.iki.fi ping statistics --- 0 packets transmitted, 0 received
shiro:~# tc -s -d qdisc show dev $DEV qdisc hfsc 1: default 1 Sent 0 bytes 0 pkts (dropped 17, overlimits 0)
shiro:~# tc -s -d class show dev $DEV class hfsc 1: root Sent 0 bytes 0 pkts (dropped 0, overlimits 0) period 0 level 1
class hfsc 1:1 parent 1: rt m1 0bps d 0us m2 800Kbit
Sent 0 bytes 0 pkts (dropped 17, overlimits 0) period 0 level 0
shiro:~# tc qdisc del dev $DEV root shiro:~# ping localhost -c 4 PING shiro.i.naked.iki.fi (127.0.0.1) 56(84) bytes of data. 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=1 ttld time=0.060 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=2 ttld time=0.060 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=3 ttld time=0.057 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=4 ttld time=0.059 ms
--- shiro.i.naked.iki.fi ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms rtt min/avg/max/mdev = 0.057/0.059/0.060/0.001 ms
shiro:~# uname -a Linux shiro 2.6.3-shiro-1 #1 Sat Mar 6 21:03:38 EET 2004 i686 GNU/Linux *****
So, I don't know what's up with that. I will debug further.
Anyhow, I will try now to come up with a working setup of HFSC for my personal use, and in the process I will try to document a sane method for coming up with the service curves and setting up the whole thing.
If that works out, more comprehensive documentation can come later.
This has been very helpful, thank you!
-- Naked
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 5 lines · Nuutti Kotivuori
Patrick McHardy wrote:
BTW the Blue scheduler patch for 2.6 seems to be working nicely - but I haven't had the time to run the tests on it that I wished to, so I haven't posted anything further about it.
I have in the mean time read up on SFB, maybe I'll extend blue when I find the time.
Quoted 15 lines
Quoted 7 lines
Adding n curves gives you (in the worst case) a (n+1)-ary curve, you can calculate it like this:
sc1: m1 = 100kbit, d = 1s, m2 = 200kbit
sc2: m1 = 50kbit, d = 0.25s, m2 = 300kbit
sc3: m1 = 200kbit, d = 1.5s, m2 = 500kbit
-----------------------------------------Quoted 6 lines
m >>{ 350kbit d <= 0.25s 600kbit 0.25s < d <= 1s 700kbit 1s < d <= 1.5s 1000kbit d > 1.5s }
Quoted 4 lines
Right. I think there's a need for a small tool to make these calculations - and perhaps even to automatically be able to scale other curves to maintain the restrictions. But that is for the future, manual calculation will do for now.
Yes that would certainly be better.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
I also seem to have a lot of difficulties in trying to simulate the behaviour of the qdisc.
[...]
Quoted 8 lines
Quoted 3 lines
Have you applied the patches from trash.net/~kaber/hfsc/tcsim ? With HZ00 and
PSCHED_GETTIMEOFDAYas clocksource I got very good results.I tried HZ00 and HZ0 both, and the results were odd. But I think I didn't touch the clocksource at all. I will try later on with
PSCHED_GETTIMEOFDAYas well.
PSCHED_GETTIMEOFDAY (or PSCHED_CPU in case of the kernel) are important for HFSC to work properly, PSCHED_JIFFIES has too low resolution.
Quoted 51 lines
Quoted 1 line
Works ok here .. do you mean inside tcsim ?
No, I don't mean inside tcsim. Here is a full transcript:
*****
shiro:~# export DEV=lo shiro:~# tc -s -d qdisc show dev $DEV shiro:~# ping localhost -c 4 PING shiro.i.naked.iki.fi (127.0.0.1) 56(84) bytes of data. 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=1 ttld time=0.062 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=2 ttld time=0.059 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=3 ttld time=0.059 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=4 ttld time=0.060 ms--- shiro.i.naked.iki.fi ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms rtt min/avg/max/mdev = 0.059/0.060/0.062/0.001 msshiro:~# tc qdisc add dev $DEV root handle 1: hfsc default 1 shiro:~# tc class add dev $DEV parent 1: classid 1:1 hfsc rt m2 100kbps shiro:~# ping localhost -c 4 PING shiro.i.naked.iki.fi (127.0.0.1) 56(84) bytes of data. ^C --- shiro.i.naked.iki.fi ping statistics --- 0 packets transmitted, 0 receivedshiro:~# tc -s -d qdisc show dev $DEV qdisc hfsc 1: default 1 Sent 0 bytes 0 pkts (dropped 17, overlimits 0)shiro:~# tc -s -d class show dev $DEV class hfsc 1: root Sent 0 bytes 0 pkts (dropped 0, overlimits 0) period 0 level 1class hfsc 1:1 parent 1: rt m1 0bps d 0us m2 800Kbit
Sent 0 bytes 0 pkts (dropped 17, overlimits 0) period 0 level 0shiro:~# tc qdisc del dev $DEV root shiro:~# ping localhost -c 4 PING shiro.i.naked.iki.fi (127.0.0.1) 56(84) bytes of data. 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=1 ttld time=0.060 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=2 ttld time=0.060 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=3 ttld time=0.057 ms 64 bytes from shiro.i.naked.iki.fi (127.0.0.1): icmp_seq=4 ttld time=0.059 ms--- shiro.i.naked.iki.fi ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms rtt min/avg/max/mdev = 0.057/0.059/0.060/0.001 ms
Quoted 5 lines
shiro:~# uname -a Linux shiro 2.6.3-shiro-1 #1 Sat Mar 6 21:03:38 EET 2004 i686 GNU/Linux *****So, I don't know what's up with that. I will debug further.
I know what the problem is. Try: ip link set lo txqueuelen 1 or upgrade to 2.6.4. The problem got introduces when fixing an off-by-one in pfifo_fast, before it would enqueue one packet with a txqueuelen of 0. In 2.6.4 this behaviour is restored, although it's a misconfiguration anyway to use leaf-queues with a limit of 1 for anything but well-formed flows.
Quoted 6 lines
Anyhow, I will try now to come up with a working setup of HFSC for my personal use, and in the process I will try to document a sane method for coming up with the service curves and setting up the whole thing.
If that works out, more comprehensive documentation can come later.
Great, I'm looking forward to it.
Regards
Patrick
Quoted 5 lines
This has been very helpful, thank you!
-- Naked
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 9 lines · Patrick McHardy
Quoted 5 lines · Nuutti Kotivuori
Patrick McHardy wrote:
BTW the Blue scheduler patch for 2.6 seems to be working nicely - but I haven't had the time to run the tests on it that I wished to, so I haven't posted anything further about it.
I have in the mean time read up on SFB, maybe I'll extend blue when I find the time.
That reminds me - I have an "extension" to Blue I'd like to try and cook up, if I ever manage to get the time.
Ingress Blue. Basically just having a token bucket on ingress, just like traffic policing has - but using Blue on that. Running out of tokens means packet drop, so increase probability. Bucket overflowing with tokens means link idle, so decrease probability.
I have a feeling something like that might work well when trying to reduce packet queueing at the ISP on a slow inbound link - better than the usual strict ingress police or using IMQ with RED and such.
Quoted 3 lines
PSCHED_GETTIMEOFDAY(orPSCHED_CPUin case of the kernel) are important for HFSC to work properly,PSCHED_JIFFIEShas too low resolution.
That might be what was messing up my other simulations as well - thanks for the heads up, I will see what comes out of that.
[...]
Quoted 1 line
I know what the problem is. Try: ip link set lo txqueuelen 1
Works like a dream!
Quoted 5 lines
or upgrade to 2.6.4. The problem got introduces when fixing an off-by-one in
pfifo_fast, before it would enqueue one packet with a txqueuelen of 0. In 2.6.4 this behaviour is restored, although it's a misconfiguration anyway to use leaf-queues with a limit of 1 for anything but well-formed flows.
Figures... :-) So I just got bitten by the default configuration of txqueuelen of 0 for lo, that I didn't even happen to think about. Time to upgrade to 2.6.4 as well it seems.
-- Naked
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/