Skip to contentSkip to conversation

: [DCCP] CCID2: Tell DCCP to quickly check whether cwnd is available

PATCH Applied Arnaldo Carvalho de Melo 4 participants 7 messages Sep 14, 2006 – Sep 19, 2006 Networking

lore
Share & tools
lore
#0 lore raw

PATCH 6/10 : [DCCP] CCID2: Tell DCCP to quickly check whether cwnd is available

If not enough cwnd is available, tell the sender to check again as soon as possible. This will increase CPU utilization (polling frequently for cwnd) but will improve network performance. That is, the sender will need to wait less before detecting the increase of cwnd. A better architecture would be for the CCID to call-back (or dequeue) from DCCP when it is able to transmit traffic -- not the other way around as it currently occurs.

------------------------------------------------------------------------------

 ccid2.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

------------------------------------------------------------------------------

net/dccp/ccids/ccid2.c +1 -1
index 699a566..e0acd1b 100644
@@ -122,7 +122,7 @@ static int ccid2_hc_tx_send_packet(struc
 		}
 	}
 
-	return 100; /* XXX */
+	return 1; /* XXX CCID should dequeue when ready instead of polling */
 }
 
 static void ccid2_change_l_ack_ratio(struct sock *sk, int val)
maintainer Applied
#1 lore raw

Re: [PATCH 6/10]: [DCCP] CCID2: Tell DCCP to quickly check whether

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, 14 Sep 2006 14:12:07 -0300

Quoted 9 lines

If not enough cwnd is available, tell the sender to check again as soon as possible. This will increase CPU utilization (polling frequently for cwnd) but will improve network performance. That is, the sender will need to wait less before detecting the increase of cwnd. A better architecture would be for the CCID to call-back (or dequeue) from DCCP when it is able to transmit traffic -- not the other way around as it currently occurs.

Applied, but this is expensive.

Consider having 100,000 DCCP sockets, some large percentage of which are polling in this manner, it simply doesn't scale.

And if anything this return value should be jiffies_to_msecs(1) or something like that.

#2 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 9 lines · David Miller

From: Arnaldo Carvalho de Melo <[email protected]>
Date: Thu, 14 Sep 2006 14:12:07 -0300

Quoted 6 lines

If not enough cwnd is available, tell the sender to check again as soon as possible. This will increase CPU utilization (polling frequently for cwnd) but will improve network performance. That is, the sender will need to wait less before detecting the increase of cwnd. A better architecture would be for the CCID to call-back (or dequeue) from DCCP when it is able to transmit traffic -- not the other way around as it currently occurs.

Remember Dave merged my tx queueing recently so this is available now

Quoted 5 lines

Applied, but this is expensive.

Consider having 100,000 DCCP sockets, some large percentage of which are polling in this manner, it simply doesn't scale.

Agree.

Signature · 7 lines
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.com
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
#3 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 1 line · Ian McDonald

Remember Dave merged my tx queueing recently so this is available now

I don't think tx queueing immedeately "helps" solving this problem. It's an architectural change that I'm proposing. In the current API, DCCP is the "master" and asks the CCID if it can send. The CCID either says yes, or says "ask me again in X time". I don't think this is great because some CCIDs don't really have a concept of time. In rate based protocols, perhaps there is a way to say "OK I can send after X time" but in window based protocols it's "upon the reception of the next ack". OK one can give an esitmate of time [rtt/cwnd?] but it still isn't great.

By reversing the architecture, I think this problem is solved quite neatly. In this case, it is the CCID which drives DCCP. The API would be: CCID tells DCCP "hey, you can send" and DCCP sends happilly. Not a poll & push model as before, but rather a pull model from CCID's perspective.

To make this even more efficient, CCID could say, "hey you can send X packets". This way, with a single notification from CCID, DCCP may drain its tx queue. There needs to be a mechanism for CCID to stop "pulling" from DCCP. For example, when DCCP doesn't want to send anything, CCID shouldn't say "hey u can send X packets". CCID can either infer this by noting that DCCP didn't send anything after a notification, or DCCP may explicitly turn notifications off when it's queue size is 0. Perhaps the latter is better.

To summarize, the API would be as follows. DCCP would implement:
void pull(int x); /* Called by CCID, indicating that DCCP may send x packets */

CCID would implement:
void notify(int true); /* if true, CCID will pull from DCCP, else not */

#4 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 4 lines · Andrea Bittau
Quoted 1 line · Ian McDonald

Remember Dave merged my tx queueing recently so this is available now

I don't think tx queueing immedeately "helps" solving this problem. It's an

Understand totally. I wasn't saying tx queueing would solve world hunger. It just helps you with the proper approach - doesn't solve it as there is a whole lot more to do...

Quoted 2 lines

To make this even more efficient, CCID could say, "hey you can send X packets". This way, with a single notification from CCID, DCCP may drain its tx queue.

If tx queueing doesn't help, how do you drain your tx queue when you don't have one :-)

Signature · 7 lines
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.com
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand
#5 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 2 lines · Ian McDonald

Understand totally. I wasn't saying tx queueing would solve world hunger. It just helps you with the proper approach - doesn't solve it

I'm sure DCCP will one day solve world hunger---TCP failed.

Quoted 2 lines

If tx queueing doesn't help, how do you drain your tx queue when you don't have one :-)

I said that on purpose to make you "happy" =D but I didn't add a reference---you spotted it anyway.

One thing I want to experiment with [now that we have tx queueing] is a "writev" style API for sending multiple packets with one syscall. Not quite writev because that is scatter gather i/o but i call it writev anyway =D. Perhaps we can add a flag to sendmsg() which tells it to use the iov as multiple packets rather than a single packet scattered about. The idea is that you should be able to send across bulk data with less context switches. The combination of writev, tx queueing and ccid pull should lower CPU utilization, and I'm wondering by how much [i.e. is it worth the effort?]

#6 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 8 lines · Andrea Bittau

One thing I want to experiment with [now that we have tx queueing] is a "writev" style API for sending multiple packets with one syscall. Not quite writev because that is scatter gather i/o but i call it writev anyway =D. Perhaps we can add a flag to sendmsg() which tells it to use the iov as multiple packets rather than a single packet scattered about. The idea is that you should be able to send across bulk data with less context switches. The combination of writev, tx queueing and ccid pull should lower CPU utilization, and I'm wondering by how much [i.e. is it worth the effort?]

I agree that APIs can improve and after I've done my current research I'm going to look at that some more.

The sendmsg code can certainly be modified - I've done this for my bestpacketnext code. If you want have a look at it on my website - it is real ugly but it works.

Signature · 7 lines
Ian McDonald
Web: http://wand.net.nz/~iam4
Blog: http://imcdnzl.blogspot.com
WAND Network Research Group
Department of Computer Science
University of Waikato
New Zealand