: [DCCP] CCID2: Tell DCCP to quickly check whether cwnd is available
PATCH 6/10 : [DCCP] CCID2: Tell DCCP to quickly check whether cwnd is available
If not enough cwnd is available, tell the sender to check again as soon as possible. This will increase CPU utilization (polling frequently for cwnd) but will improve network performance. That is, the sender will need to wait less before detecting the increase of cwnd. A better architecture would be for the CCID to call-back (or dequeue) from DCCP when it is able to transmit traffic -- not the other way around as it currently occurs.
------------------------------------------------------------------------------
ccid2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
------------------------------------------------------------------------------
net/dccp/ccids/ccid2.c +1 -1
@@ -122,7 +122,7 @@ static int ccid2_hc_tx_send_packet(struc } } -return 100 ;/* XXX */ +return 1 ;/* XXX CCID should dequeue when ready instead of polling */ }static void ccid2_change_l_ack_ratio (struct sock *sk,int val)
Re: [PATCH 6/10]: [DCCP] CCID2: Tell DCCP to quickly check whether
From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, 14 Sep 2006 14:12:07 -0300
Quoted 9 lines
If not enough cwnd is available, tell the sender to check again as soon as possible. This will increase CPU utilization (polling frequently for cwnd) but will improve network performance. That is, the sender will need to wait less before detecting the increase of cwnd. A better architecture would be for the CCID to call-back (or dequeue) from DCCP when it is able to transmit traffic -- not the other way around as it currently occurs.
Signed-off-by Andrea Bittau <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>
Applied, but this is expensive.
Consider having 100,000 DCCP sockets, some large percentage of which are polling in this manner, it simply doesn't scale.
And if anything this return value should be jiffies_to_msecs(1) or something like that.
Quoted 9 lines · David Miller
From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, 14 Sep 2006 14:12:07 -0300Quoted 6 lines
If not enough cwnd is available, tell the sender to check again as soon as possible. This will increase CPU utilization (polling frequently for cwnd) but will improve network performance. That is, the sender will need to wait less before detecting the increase of cwnd. A better architecture would be for the CCID to call-back (or dequeue) from DCCP when it is able to transmit traffic -- not the other way around as it currently occurs.
Remember Dave merged my tx queueing recently so this is available now
Quoted 5 lines
Applied, but this is expensive.
Consider having 100,000 DCCP sockets, some large percentage of which are polling in this manner, it simply doesn't scale.
Agree.
Signature · 7 lines
Ian McDonald Web: http://wand.net.nz/~iam4 Blog: http://imcdnzl.blogspot.com WAND Network Research Group Department of Computer Science University of Waikato New Zealand
Quoted 1 line · Ian McDonald
Remember Dave merged my tx queueing recently so this is available now
I don't think tx queueing immedeately "helps" solving this problem. It's an architectural change that I'm proposing. In the current API, DCCP is the "master" and asks the CCID if it can send. The CCID either says yes, or says "ask me again in X time". I don't think this is great because some CCIDs don't really have a concept of time. In rate based protocols, perhaps there is a way to say "OK I can send after X time" but in window based protocols it's "upon the reception of the next ack". OK one can give an esitmate of time [rtt/cwnd?] but it still isn't great.
By reversing the architecture, I think this problem is solved quite neatly. In this case, it is the CCID which drives DCCP. The API would be: CCID tells DCCP "hey, you can send" and DCCP sends happilly. Not a poll & push model as before, but rather a pull model from CCID's perspective.
To make this even more efficient, CCID could say, "hey you can send X packets". This way, with a single notification from CCID, DCCP may drain its tx queue. There needs to be a mechanism for CCID to stop "pulling" from DCCP. For example, when DCCP doesn't want to send anything, CCID shouldn't say "hey u can send X packets". CCID can either infer this by noting that DCCP didn't send anything after a notification, or DCCP may explicitly turn notifications off when it's queue size is 0. Perhaps the latter is better.
To summarize, the API would be as follows. DCCP would implement:
void pull(int x); /* Called by CCID, indicating that DCCP may send x packets */
CCID would implement:
void notify(int true); /* if true, CCID will pull from DCCP, else not */
Quoted 4 lines · Andrea Bittau
Quoted 1 line · Ian McDonald
Remember Dave merged my tx queueing recently so this is available now
I don't think tx queueing immedeately "helps" solving this problem. It's an
Understand totally. I wasn't saying tx queueing would solve world hunger. It just helps you with the proper approach - doesn't solve it as there is a whole lot more to do...
Quoted 2 lines
To make this even more efficient, CCID could say, "hey you can send X packets". This way, with a single notification from CCID, DCCP may drain its tx queue.
If tx queueing doesn't help, how do you drain your tx queue when you don't have one :-)
Signature · 7 lines
Ian McDonald Web: http://wand.net.nz/~iam4 Blog: http://imcdnzl.blogspot.com WAND Network Research Group Department of Computer Science University of Waikato New Zealand
Quoted 2 lines · Ian McDonald
Understand totally. I wasn't saying tx queueing would solve world hunger. It just helps you with the proper approach - doesn't solve it
I'm sure DCCP will one day solve world hunger---TCP failed.
Quoted 2 lines
If tx queueing doesn't help, how do you drain your tx queue when you don't have one :-)
I said that on purpose to make you "happy" =D but I didn't add a reference---you spotted it anyway.
One thing I want to experiment with [now that we have tx queueing] is a "writev" style API for sending multiple packets with one syscall. Not quite writev because that is scatter gather i/o but i call it writev anyway =D. Perhaps we can add a flag to sendmsg() which tells it to use the iov as multiple packets rather than a single packet scattered about. The idea is that you should be able to send across bulk data with less context switches. The combination of writev, tx queueing and ccid pull should lower CPU utilization, and I'm wondering by how much [i.e. is it worth the effort?]
Quoted 8 lines · Andrea Bittau
One thing I want to experiment with [now that we have tx queueing] is a "writev" style API for sending multiple packets with one syscall. Not quite writev because that is scatter gather i/o but i call it writev anyway =D. Perhaps we can add a flag to
sendmsg()which tells it to use the iov as multiple packets rather than a single packet scattered about. The idea is that you should be able to send across bulk data with less context switches. The combination of writev, tx queueing and ccid pull should lower CPU utilization, and I'm wondering by how much [i.e. is it worth the effort?]
I agree that APIs can improve and after I've done my current research I'm going to look at that some more.
The sendmsg code can certainly be modified - I've done this for my bestpacketnext code. If you want have a look at it on my website - it is real ugly but it works.
Signature · 7 lines
Ian McDonald Web: http://wand.net.nz/~iam4 Blog: http://imcdnzl.blogspot.com WAND Network Research Group Department of Computer Science University of Waikato New Zealand