Rough VJ Channel Implementation - vj_core.patch
Subsystems (5)
PATCH 1/3 Rough VJ Channel Implementation - vj_core.patch
Hey guys... I've been working with Rusty on a VJ Channel implementation. Noting Dave's recent release of his implementation, we thought we'd better get this "out there" so we can do some early comparison/combining and come up with the best possible implementation.
There are three patches in total:
1) vj_core.patch - core files for VJ to userspace
2) vj_udp.patch - badly hacked up UDP receive implementation - basically just to test what logic may be like!
3) vj_ne2k.patch - modified NE2K and 8390 used for testing on QEMU
Notes:
* channels can have global or local buffers (local for userspace. Could be used directly by intelligent NIC)
* UDP receive breaks real UDP - doesn't talk anything except VJ Channels anymore. Needs integration with normal sources.
* Userspace test app (below) uses VJ protocol family to mmap space for local buffers, if it receives buffers in kernel space sends a request for that buffer to be copied to local buffer.
* Default channel converts to skb and feeds through normal receive path.
TODO:
* send not yet implemented
* integrate non vj
* LOTS of fixmes
Cheers,
Kelly
Test userspace app:
/* Van Jacobson net channels implementation for Linux
Copyright (C) 2006 Kelly Daly <[email protected]> IBM Corporation
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/mman.h>
#include <sys/poll.h>
#include <netinet/in.h>
#include "linux-2.6.16/include/linux/types.h"
#include "linux-2.6.16/include/linux/vjchan.h"
//flowid
#define SADDR 0
#define DADDR 0
#define SPORT 0
#define DPORT 60000
#define IFINDEX 0
#define PF_VJCHAN 27
static struct vj_buffer *get_buffer(struct vj_channel_ring *ring, int desc_num)
{
printf("desc_num %i\n", desc_num);
return (void *)ring + (desc_num + 1) * getpagesize();
}
/* return the next buffer, but do not move on */
static struct vj_buffer *vj_peek_next_buffer(struct vj_channel_ring *ring)
{
if (ring->c.head == ring->p.tail)
return NULL;
return get_buffer(ring, ring->q[ring->c.head]);
}/* move on to next buffer */
static void vj_done_with_buffer(struct vj_channel_ring *ring)
{
ring->c.head = (ring->c.head+1)%VJ_NET_CHANNEL_ENTRIES;
printf("done_with_buffer\n\n");
}int main(int argc, char *argv[])
{
int sk, cls, bnd, pll;
void * mmapped;
struct vj_flowid flowid;
struct vj_channel_ring *ring;
struct vj_buffer *buf;
struct pollfd pfd;
printf("\nstart of vjchannel socket test app\n");
sk = socket(PF_VJCHAN, SOCK_DGRAM, IPPROTO_UDP);
if (sk == -1) {
perror("Unable to open socket!");
return -1;
}
printf("socket open with ret code %i\n\n", sk);//create flowid!!!
flowid.saddr = SADDR;
flowid.daddr = DADDR;
flowid.sport = SPORT;
flowid.dport = htons(DPORT);
flowid.ifindex = IFINDEX;
flowid.proto = IPPROTO_UDP;
printf("flowid created\n");
bnd = bind(sk, (struct sockaddr *)&flowid, sizeof(struct vj_flowid));
if (bnd == -1) {
perror("Unable to bind socket!");
return -1;
}
printf("socket bound with ret code %i\n\n", bnd);
ring = mmap(0, (getpagesize() * (VJ_NET_CHANNEL_ENTRIES+1)), PROT_READ|PROT_WRITE, MAP_SHARED, sk, 0);
if (ring == MAP_FAILED) {
perror ("Unable to mmap socket!");
return -1;
}
printf("socket mmapped to address %lu\n\n", (unsigned long)mmapped);
pfd.fd = sk;
pfd.events = POLLIN;
for (;;) {
pll = poll(&pfd, 1, -1);
if (pll < 0) {
perror("polling failed!");
return -1;
}//consume
buf = vj_peek_next_buffer(ring);
printf("buf %p\n", buf);//print data, not headers
printf(" Buffer Length = %i\n", buf->data_len);
printf(" Header Length = %i\n", buf->header_len);
printf(" Buffer Data: '%.*s'\n", buf->data_len - 28, buf->data + buf->header_len + 28);
vj_done_with_buffer(ring);
}
cls = close(sk);
if (cls != 0) {
perror("Unable to close socket!");
return -2;
}
printf("socket closed with ret code %i\n\n", cls);
return 0;
}-------------------------
Basic infrastructure for Van Jacobson net channels: lockless ringbuffer for buffer transport. Entries in ring buffer are descriptors for global or local buffers: ring and local buffers are mmapped into userspace. Channels are registered with the core by flowid, and a thread services the default channel for any non-matching packets. Drivers get (global) buffers from vj_get_buffer, and dispatch them through vj_netif_rx. As userspace mmap cannot reach global buffers, select() copies global buffers into local buffers if required.
include/linux/socket.h Mon Apr 24 19:50:46 2006 renamed from include/linux/socket.h Thu Mar 23 06:32:12 2006 +2 -0
@@ -186,6 +187,7 @@ #define AF_PPPOX 24 /* PPPoX sockets */ #define AF_WANPIPE 25 /* Wanpipe API Sockets */ #define AF_LLC 26 /* Linux LLC */ +#define AF_VJCHAN 27 /* VJ Channel */ #define AF_TIPC 30 /* TIPC sockets */ #define AF_BLUETOOTH 31 /* Bluetooth sockets */ #define AF_MAX 32 /* For now.. */
@@ -219,7 +221,8 @@ #define PF_PPPOX AF_PPPOX #define PF_WANPIPE AF_WANPIPE #define PF_LLC AF_LLC +#define PF_VJCHAN AF_VJCHAN #define PF_TIPC AF_TIPC #define PF_BLUETOOTH AF_BLUETOOTH #define PF_MAX AF_MAX
net/Kconfig Mon Apr 24 19:50:46 2006 renamed from net/Kconfig Thu Mar 23 06:32:12 2006 +6 -0
@@ -65,6 +65,12 @@ source "net/ipv6/Kconfig" endif # if INET + +config VJCHAN + bool "Van Jacobson Net Channel Support (EXPERIMENTAL)" + depends on EXPERIMENTAL + ---help--- + This adds a userspace-accessible packet receive interface. Say N. menuconfig NETFILTER bool "Network packet filtering (replaces ipchains)"
net/Makefile Mon Apr 24 19:50:46 2006 renamed from net/Makefile Thu Mar 23 06:32:12 2006 +1 -0
@@ -46,6 +46,7 @@ obj-$(CONFIG_IP_SCTP) += sctp/ obj-$(CONFIG_IEEE80211) += ieee80211/ obj-$(CONFIG_TIPC) += tipc/ +obj-$(CONFIG_VJCHAN) += vjchan/ ifeq ($(CONFIG_NET),y) obj-$(CONFIG_SYSCTL) += sysctl_net.o
include/linux/vjchan.h Mon Apr 24 19:50:46 2006 renamed from dev/null Thu Mar 23 06:32:12 2006 +79 -0
79 lines not on this page · raw message
net/vjchan/Makefile Mon Apr 24 19:50:46 2006 renamed from dev/null Thu Mar 23 06:32:12 2006 +3 -0
@@ -0,0 +1,3 @@ +#obj-m += vjtest.o +obj-y += vjnet.o +obj-y += af_vjchan.o
net/vjchan/af_vjchan.c Mon Apr 24 19:50:46 2006 renamed from dev/null Thu Mar 23 06:32:12 2006 +198 -0
198 lines not on this page · raw message
net/vjchan/vjnet.c Mon Apr 24 19:50:46 2006 renamed from dev/null Thu Mar 23 06:32:12 2006 +550 -0
550 lines not on this page · raw message
David S. Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Wed, 26 Apr 2006 11:47:34 +0000
Quoted 4 lines
Noting Dave's recent release of his implementation, we thought we'd better get this "out there" so we can do some early comparison/combining and come up with the best possible implementation.
Thanks for publishing your work.
I'm actually not that upset that I duplicated the work a little bit because trying to start implementing things forced me to think in a more focued way about this stuff.
I'll look over your patches, thanks.
Hi Dave,
Thanks for your response. =)
Quoted 5 lines · you
Ok I have comments already just glancing at the initial patch.
With the 32-bit descriptors in the channel, you indeed end up with a fixed sized pool with a lot of hard-to-finesse sizing and lookup problems to solve.
It should be quite trivial to resize this pool using RCU.
Quoted 4 lines
So what I wanted to do was finesse the entire issue by simply side-stepping it initially. Use a normal buffer with a tail descriptor, when you enqueue you give a tail descriptor pointer.
The tail pointers are an excellent idea - and they certainly fix a lot of compatibility issues that we side-stepped (we were going for the "make it work" approach rather than the "make it right" - figured we could get to that bit later =P ).
Quoted 6 lines
I really dislike the pools of buffers, partly because they are fixed size (or dynamically sized and even more expensive to implement), but moreso because there is all of this absolutely stupid state management you eat just to get at the real data. That's pointless, we're trying to make this as light as possible. Just use real pointers and describe the packet with a tail descriptor.
We approached this from the understanding that an intelligent NIC will be able to transition directly to userspace, which is a major win. 0 copies to userspace would be sweet. I think we can still achieve this using your scheme without *too* much pain.
Quoted 7 lines
Next, you can't even begin to work on the protocol channels before you do one very important piece of work. Integration of all of the ipv4 and ipv6 protocol hash tables into a central code, it's a total prerequisite. Then you modify things to use a generic inet_{,listen_}lookup() or inet6_{,listen_}lookup() that takes a protocol number as well as saddr/daddr/sport/dport and searches from a central table.
Understood. And agreed. Once again was side-stepped just to try to get a "working model". Will look into this immediately.
Quoted 2 lines
So I think I'll continue working on my implementation, it's more transitional and that's how we have to do this kind of work.
Thanks again for your comments =) (and thanks to everyone else who took the time to respond to this)
Kelly
David S. Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Thu, 27 Apr 2006 13:31:37 +1000
Quoted 1 line
It should be quite trivial to resize this pool using RCU.
Yes, a lot of this stuff can use RCU, in particular the channel demux is a prime candidate.
There are some non-trivial issues wrt. synchronizing the net channel lookup state with socket state changes (socket moves to close or whatever). This reminds me that we had some nice TCP hash table RCU patches that Ben LaHaise posted at one point and that slipped through the cracks. That took care of all the event ordering issues, it seemed at the time, and is something we need to get back on track with.
Quoted 4 lines
The tail pointers are an excellent idea - and they certainly fix a lot of compatibility issues that we side-stepped (we were going for the "make it work" approach rather than the "make it right" - figured we could get to that bit later =P ).
Start simple, we can keep mucking with the interfaces over and over again as we move from simple netif_receive_skb() channels out to the more complex socket demux style channel.
This is a big and long project, there are no style points for trying to go all the way in the first pass :-)
Quoted 4 lines
We approached this from the understanding that an intelligent NIC will be able to transition directly to userspace, which is a major win. 0 copies to userspace would be sweet. I think we can still achieve this using your scheme without *too* much pain.
Understood. What's your basic idea? Just make the buffers in the pool large enough to fit the SKB encapsulation at the end?
Note that this will change a lot of the assumptions currently in your buffer handling code about buffer reuse and such.
So the idea in your scheme is to give the buffer pools to the NIC in a per-channel way via a simple descriptor table? And the u32's are arbitrary keys that index into this descriptor table, right?
I would suggest just sticking to the simple global input queue. Solve the easy problems and the buffering model first. Then we can port drivers and people can bang on the basic infrastructure. Take my SKB encapsulator in my vj-2.6 tree once you've transformed your buffer pools to accomodate.
I'll actually sit back and let you do that, I'm actually coming around more to your scheme in some regards :-) I'll sit and think about some of the heavier issues we'll hit in the next phase and once you have a cut at the current phase I'll work on a tg3 driver port.
Thanks!
Quoted 7 lines · David S. Miller
Quoted 4 lines
We approached this from the understanding that an intelligent NIC will be able to transition directly to userspace, which is a major win. 0 copies to userspace would be sweet. I think we can still achieve this using your scheme without *too* much pain.
Understood. What's your basic idea? Just make the buffers in the pool large enough to fit the SKB encapsulation at the end?
There are some caveats here found while developing zero-copy sniffer [1]. Project's goal was to remap skbs into userspace in real-time. While absolute numbers (posted to netdev@) were really high, it is only applicable to read-only application. As was shown in IOAT thread, data must be warmed in caches, so reading from mapped area will be as fast as memcpy() (read+write), and copy_to_user() actually almost equal to memcpy() (benchmarks were posted to netdev@). And we must add remapping overhead.
If we want to dma data from nic into premapped userspace area, this will strike with message sizes/misalignment/slow read and so on, so preallocation has even more problems.
This change also requires significant changes in application, at least until recv/send are changed, which is not the best thing to do.
So I think that mapping itself can be done as some additional socket option or something not turnedon by default.
I do think that significant win in VJ's tests belongs not to remapping and cache-oriented changes, but to move all protocol processing into process' context.
I fully agree with Dave that it must be implemented step-by-step, and the most significant, IMHO, is moving protocol processing into socket's "place". This will force to netfilter changes, but I do think that for the proof-of-concept code we can turn it off.
I will start to work in this direction next week after aio_sendfile() is completed.
So, we will have three attempts to write incompatible stacks - and that is good :) No one need an excuse to rewrite something, as I read in Rusty's blog...
Thanks.
[1]. http://tservice.net.ru/~s0mbre/old/?section=projects&item=af_tlb
Signature · 1 line
Evgeniy Polyakov
David S. Miller
maintainerFrom: Evgeniy Polyakov <[email protected]>
Date: Thu, 27 Apr 2006 15:51:26 +0400
Quoted 8 lines
There are some caveats here found while developing zero-copy sniffer [1]. Project's goal was to remap skbs into userspace in real-time. While absolute numbers (posted to netdev@) were really high, it is only applicable to read-only application. As was shown in IOAT thread, data must be warmed in caches, so reading from mapped area will be as fast as
memcpy()(read+write), andcopy_to_user()actually almost equal tomemcpy()(benchmarks were posted to netdev@). And we must add remapping overhead.
Yes, all of these issues are related quite strongly. Thanks for making the connection explicit.
But, the mapping overhead is zero for this net channel stuff, at least as it is implemented and designed by Kelly. Ring buffer is setup ahead of time into the user's address space, and a ring of buffers into that area are given to the networking card.
We remember the translations here, so no get_user_pages() on each transfer and garbage like that. And yes this all harks back to the issues that are discussed in Chapter 5 of Networking Algorithmics. But the core thing to understand is that by defining a new API and setting up the buffer pool ahead of time, we avoid all of the get_user_pages() overhead while retaining full kernel/user protection.
Evgeniy, the difference between this and your work is that you did not have an intelligent piece of hardware that could be told to recognize flows, and only put packets for a specific flow into that's flow's buffer pool.
Quoted 3 lines
If we want to dma data from nic into premapped userspace area, this will strike with message sizes/misalignment/slow read and so on, so preallocation has even more problems.
I do not really think this is an issue, we put the full packet into user space and teach it where the offset is to the actual data. We'll do the same things we do today to try and get the data area aligned. User can do whatever is logical and relevant on his end to deal with strange cases.
In fact we can specify that card has to take some care to get data area of packet aligned on say an 8 byte boundary or something like that. When we don't have hardware assist, we are going to be doing copies.
Quoted 2 lines
This change also requires significant changes in application, at least until recv/send are changed, which is not the best thing to do.
This is exactly the point, we can only do a good job and receive zero copy if we can change the interfaces, and that's exactly what we're doing here.
Quoted 3 lines
I do think that significant win in VJ's tests belongs not to remapping and cache-oriented changes, but to move all protocol processing into process' context.
I partly disagree. The biggest win is eliminating all of the control overhead (all of "softint RX + protocol demux + IP route lookup + socket lookup" is turned into single flow demux), and the SMP safe data structure which makes it realistic enough to always move the bulk of the packet work to the socket's home cpu.
I do not think userspace protocol implementation buys enough to justify it. We have to do the protection switch in and out of kernel space anyways, so why not still do the protected protocol processing work in the kernel? It is still being done on the user's behalf, contributes to his time slice, and avoids all of the terrible issues of userspace protocol implementations.
So in my mind, the optimal situation from both a protection preservation and also a performance perspective is net channels to kernel socket protocol processing, buffers DMA'd directly into userspace if hardware assist is present.
Quoted 4 lines
I fully agree with Dave that it must be implemented step-by-step, and the most significant, IMHO, is moving protocol processing into socket's "place". This will force to netfilter changes, but I do think that for the proof-of-concept code we can turn it off.
And I also want to note that even if the whole idea explodes and cannot be made to work, there are good arguments for transitioning to SKB'less drivers for their own sake. So work will really not be lost.
Let's have 100 different implementations of net channels! :-)
Quoted 4 lines · David S. Miller
Evgeniy, the difference between this and your work is that you did not have an intelligent piece of hardware that could be told to recognize flows, and only put packets for a specific flow into that's flow's buffer pool.
There are the most "intellegent" NICs which use MMIO copy like Realtek 8139 :) which were used in receiving zero-copy [1] project.
There was special alorithm researched for receiving zero-copy [1] to allow to put not page-aligned TCP frames into pages, but there was other problem when page was committed, since no byte commit is allowed in VFS.
In this case we do not have that problem, but instead we must force userspace to be very smart when dealing with mapped buffers, instead of simple recv(). And for sending it must be even smarter, since data must be properly aligned. And what about crappy hardware which can DMA only into limited memory area, or NIC that can not do sg? Or do we need remapping for NIC that can not do checksum calculation?
Quoted 10 lines
Quoted 3 lines
If we want to dma data from nic into premapped userspace area, this will strike with message sizes/misalignment/slow read and so on, so preallocation has even more problems.
I do not really think this is an issue, we put the full packet into user space and teach it where the offset is to the actual data. We'll do the same things we do today to try and get the data area aligned. User can do whatever is logical and relevant on his end to deal with strange cases.
Quoted 4 lines
In fact we can specify that card has to take some care to get data area of packet aligned on say an 8 byte boundary or something like that. When we don't have hardware assist, we are going to be doing copies.
Userspace must be too smart, and as we saw with various java tests, it can not be so even now.
And what if pages are shared and several threads are trying to write into the same remapped area? Will we use COW and be blamed like Mach and FreeBSD developers? :)
Quoted 10 lines
Quoted 3 lines
I do think that significant win in VJ's tests belongs not to remapping and cache-oriented changes, but to move all protocol processing into process' context.
I partly disagree. The biggest win is eliminating all of the control overhead (all of "softint RX + protocol demux + IP route lookup + socket lookup" is turned into single flow demux), and the SMP safe data structure which makes it realistic enough to always move the bulk of the packet work to the socket's home cpu.
Quoted 6 lines
I do not think userspace protocol implementation buys enough to justify it. We have to do the protection switch in and out of kernel space anyways, so why not still do the protected protocol processing work in the kernel? It is still being done on the user's behalf, contributes to his time slice, and avoids all of the terrible issues of userspace protocol implementations.
After hard irq softirq is scheduled, then later userspace is scheduled, at least 2 context switch just to move a packet, and "slow" userspace code is interrupted by both irqs again...
I run some tests on ppc32 embedded boards which showed that rescheduling latency tend to have milliseconds delay sometimes (about 4 running processes on 200mhz cpu), although we do not have some real-time requirements here it is not a good sign...
Quoted 6 lines
And I also want to note that even if the whole idea explodes and cannot be made to work, there are good arguments for transitioning to SKB'less drivers for their own sake. So work will really not be lost.
Let's have 100 different implementations of net channels! :-)
:)
Signature · 1 line
Evgeniy Polyakov
Quoted 4 lines · you
So the idea in your scheme is to give the buffer pools to the NIC in a per-channel way via a simple descriptor table? And the u32's are arbitrary keys that index into this descriptor table, right?
yeah - it _was_... Although since having a play with coding it into your implementation we've come up with the following:
Using the descriptor table adds excess complexity for kernel buffers, and is really only useful for userspace. So instead of using descriptor tables for everything we've come up with a dynamic descriptor table scheme instead where they are used only for userspace.
The move to skb-ising the buffers has made it more difficult to keep track of buffer lifetimes. Previously we were leaving the buffers in the ring until completely finished with them. The producer could reuse the buffer once the consumer head had moved on. With the graft to skb we can no longer do this unless the packets are processed serially (which is ok for socket channels, but not realistic for the default).
We DID write an infrastructure to resolve this issue, although it is more complex than the dynamic descriptor scheme for userspace. And we want to keep this simple - right?
Cheers,
K
David S. Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Thu, 4 May 2006 12:59:23 +1000
Quoted 3 lines
We DID write an infrastructure to resolve this issue, although it is more complex than the dynamic descriptor scheme for userspace. And we want to keep this simple - right?
Yes.
I wonder if it is possible to manage the buffer pool just like a SLAB cache to deal with the variable lifetimes. The system has a natural "working set" size of networking buffers at a given point in time and even the default net channel can grow to accomodate that with some kind of limit.
This is kind of what I was alluding to in the past, in that we now have globals limits on system TCP socket memory when really what we want to do is have a set of global generic system packet memory limits.
These two things can tie in together.
Note that this means we need a callback in the SKB to free the memory up. For direct net channels to a socket, you don't need any callbacks of course because as you mentioned you know the buffer lifetimes.
People want such a callback anyways in order to experiment with SKB recycling in drivers.
Note that some kind of "shrink" callback would need to be implemented. It would only be needed for the default channel. We need to seriously avoid needing something like this over the socket net channels because that is serious complexity.
Finally... if we go the global packet memory route, we will need hard and soft limits. There is a danger in such a scheme of not being able to get critical control packets out (ACKs, etc.). Also, there are all kinds of classification and drop algorithms (see RED) which could be used to handle overload situations gracefully.
So, are you still sure you want to do away with the descriptors for the default channel? Is the scheme I have outlined above doable or is there some critical barrier or some complexity issue which makes it undesirable?
Quoted 15 lines · David S. Miller
From: Kelly Daly <[email protected]>
Date: Thu, 4 May 2006 12:59:23 +1000Quoted 3 lines
We DID write an infrastructure to resolve this issue, although it is more complex than the dynamic descriptor scheme for userspace. And we want to keep this simple - right?
Yes.
I wonder if it is possible to manage the buffer pool just like a SLAB cache to deal with the variable lifetimes. The system has a natural "working set" size of networking buffers at a given point in time and even the default net channel can grow to accomodate that with some kind of limit.
Quoted 6 lines
This is kind of what I was alluding to in the past, in that we now have globals limits on system TCP socket memory when really what we want to do is have a set of global generic system packet memory limits.
These two things can tie in together.
Hi Dave,
We kept a simple "used" bitmap, but to avoid the consumer touching it,
also put a "I am masquerading as an SKB" bit in the trailer, like so:
include/linux/skbuff.h +2 -1
@@ -133,7 +133,8 @@ struct skb_frag_struct {*/ struct skb_shared_info {atomic_t dataref; -unsigned short nr_frags; +unsigned short nr_frags :15 ; +unsigned int chan_as_skb :1 ;unsigned short tso_size;unsigned short tso_segs;unsigned short ufo_size;
net/core/skbuff.c +5 -1
@@ -289,6 +289,7 @@ struct sk_buff *skb_netchan_graft(structskb_shinfo (skb)->ufo_size =0 ;skb_shinfo (skb)->ip6_frag_id =0 ;skb_shinfo (skb)->frag_list =NULL ; +skb_shinfo (skb)->chan_as_skb =1 ;return skb; }
@@ -328,7 +329,10 @@ void skb_release_data(struct sk_buff *skif (skb_shinfo (skb)->frag_list)skb_drop_fraglist (skb); -kfree (skb->head); +if (skb_shinfo (skb)->chan_as_skb) +skb_shinfo (skb)->chan_as_skb =0 ; +else +kfree (skb->head); } }
Buffer allocation would be: find_first_bit, check that it's not actually inside an skb, or otherwise find_next_bit. Assuming most buffers do not go down default channel, this is efficient.
Problems:
1) it's still not cache-friendly with producers on multiple CPUs. We could divide up the bitmap into per-cpu regions to try first to improve cache behaviour.
2) In addition, we had every buffer one page large. This isn't sufficient for jumbo frames, and wasteful for ethernet. So if we statically assign descriptors -> buffers, we need to have multiple sizes.
3) OTOH, if descriptor table is dynamic, we have cache issues again as multiple people are writing to it, and it's not clear what we really gain over direct pointers.
4) Grow/shrink can be done, but needs stop_machine, or maybe tricky RCU.
5) The killer for me: we can't use our scheme straight-to-userspace anyway, since we can't trust the (user-writable) ringbuffer in deciding what buffers to release. Since we need to store this somewhere, we need a test in netchannel_enqueue. At which point, we might as well translate to "descriptors" at that point, anyway (since descriptors are only really needed for userspace). Something like:
tail = np->netchan_tail; if (tail == np->netchan_head) return -ENOMEM;
+ /* Write to userspace? They can't deref ptr anyway. */
+ if (np->shadow_ring && !netchan_local_buf(bp)) {
+ np->shadow_ring[tail] = bp;
+ bp = (void *)-1;
+ }
np->netchan_queue[tail++] = bp; if (tail == NET_CHANNEL_ENTRIES)
(We don't have local buffers yet, but I'm assuming we'll use v. low pointers for them). Userspace goes "desc number is in range, we can access directly" or "desc number isn't, call into kernel to copy them for us".
Quoted 4 lines
So, are you still sure you want to do away with the descriptors for the default channel? Is the scheme I have outlined above doable or is there some critical barrier or some complexity issue which makes it undesirable?
I think it's simpler to build global alloc limiters on what we have. The slab already has the nice lifetime and cache-friendly properties we want, so we just have to write the limiting code. There's enough work there to keep us busy 8)
Cheers,
Rusty.
Signature · 1 line
ccontrol: http://ozlabs.org/~rusty/ccontrol
David S. Miller
maintainerOk I have comments already just glancing at the initial patch.
With the 32-bit descriptors in the channel, you indeed end up with a fixed sized pool with a lot of hard-to-finesse sizing and lookup problems to solve.
So what I wanted to do was finesse the entire issue by simply side-stepping it initially. Use a normal buffer with a tail descriptor, when you enqueue you give a tail descriptor pointer.
Yes, it's weirder to handle this in hardware, but it's not impossible and using real pointers means two things:
1) You can design a simple netif_receive_skb() channel that works today, encapsulation of channel buffers into an SKB is like 15 lines of code and no funny lookups.
2) People can start porting the input path of drivers right now and retain full functionality and test anything they want. This is important for getting the drivers stable as fast as possible.
And it also means we can tackle the buffer pool issue of the 32-bit descriptors later, if we actually want to do things that way, I think we probably don't.
To be honest, I don't think using a 32-bit descriptor is so critical even from a hardware implementation perspective. Yes, on 64-bit you're dealing with a 64-bit quantity so the number of entries in the channel are halfed from what a 32-bit arch uses.
Yes I say this for 2 reasons:
1) We have no idea whether it's critical to have "~512" entries in the channel which is about what a u32 queue entry type affords you on x86 with 4096 byte page size.
2) Furthermore, it is sized by page size, and most 64-bit platforms use an 8K base page size anyways, so the number of queue entries ends of being the same. Yes, I know some 64-bit platforms use a 4K page size, please see #1 :-)
I really dislike the pools of buffers, partly because they are fixed size (or dynamically sized and even more expensive to implement), but moreso because there is all of this absolutely stupid state management you eat just to get at the real data. That's pointless, we're trying to make this as light as possible. Just use real pointers and describe the packet with a tail descriptor.
We can use a u64 or whatever in a hardware implementation.
Next, you can't even begin to work on the protocol channels before you do one very important piece of work. Integration of all of the ipv4 and ipv6 protocol hash tables into a central code, it's a total prerequisite. Then you modify things to use a generic inet_{,listen_}lookup() or inet6_{,listen_}lookup() that takes a protocol number as well as saddr/daddr/sport/dport and searches from a central table.
So I think I'll continue working on my implementation, it's more transitional and that's how we have to do this kind of work.
Quoted 7 lines · David S. Miller
Next, you can't even begin to work on the protocol channels before you do one very important piece of work. Integration of all of the ipv4 and ipv6 protocol hash tables into a central code, it's a total prerequisite. Then you modify things to use a generic inet_{,listen_}lookup() or inet6_{,listen_}lookup() that takes a protocol number as well as saddr/daddr/sport/dport and searches from a central table.
Back here again ;)
Is this on the right track (see patch below)?
K
_____________________
include/net/inet_hashtables.h +2 -0
@@ -418,4 +418,6 @@ static inline struct sock *inet_lookup(sextern int inet_hash_connect (struct inet_timewait_death_row *death_row,struct sock *sk); + +extern struct inet_hashinfo *inet_hashes[256 ];#endif /* _INET_HASHTABLES_H */
include/net/sock.h +1 -0
@@ -196,6 +196,7 @@ struct sock {unsigned short sk_type;int sk_rcvbuf;socket_lock_t sk_lock; +struct netchannel *sk_channel;wait_queue_head_t *sk_sleep;struct dst_entry *sk_dst_cache;struct xfrm_policy *sk_policy[2 ];
net/core/dev.c +37 -0
@@ -116,6 +116,7 @@#include <net/iw_handler.h> #include <asm/current.h> #include <linux/audit.h> +#include <net/inet_hashtables.h> /* * The list of packet types we will receive (as opposed to discard)
@@ -190,6 +191,8 @@ static inline struct hlist_head *dev_indreturn &dev_index_head[ifindex & ((1 <<NETDEV_HASHBITS )-1 )]; } +static struct netchannel default_netchannel; +/* * Our notifier list */
@@ -1907,6 +1910,37 @@ struct netchannel_buftrailer *__netchann }EXPORT_SYMBOL_GPL (__netchannel_dequeue); + +/* Find the channel for a packet, or return default channel. */ +struct netchannel *find_netchannel (const struct netchannel_buftrailer *np) +{ +struct sock *sk =NULL ; +unsigned long dlen = np->netchan_buf_len - np->netchan_buf_offset; +void *data = (void *)np - dlen; + +switch (np->netchan_buf_proto) { +case __constant_htons (ETH_P_IP ): { +struct iphdr *ip = data; +int iphl = ip->ihl *4 ; + +if (dlen >= (iphl +4 ) && iphl ==sizeof (struct iphdr )) { +u16 *ports = (u16 *)(ip +1 ); + +if (inet_hashes[ip->protocol]) { + sk =inet_lookup (inet_hashes[ip->protocol], + ip->saddr, ports[0 ], + ip->daddr, ports[1 ], + np->netchan_buf_dev->ifindex); + } +break ; + } + } + } +if (sk && sk->sk_channel) +return sk->sk_channel; +return &default_netchannel; +} +static gifconf_func_t * gifconf_list [NPROTO ];/**
@@ -3421,6 +3455,9 @@ static int __init net_dev_init(void)hotcpu_notifier (dev_cpu_callback,0 );dst_init ();dev_mcast_init (); + +/* FIXME: This should be attached to thread/threads. */ +netchannel_init (&default_netchannel,NULL ,NULL ); rc =0 ; out:return rc;
net/ipv4/inet_hashtables.c +2 -0
@@ -337,3 +337,5 @@ out: }EXPORT_SYMBOL_GPL (inet_hash_connect); + +struct inet_hashinfo *inet_hashes[256 ];
net/ipv4/tcp.c +1 -0
@@ -2173,6 +2173,7 @@ void __init tcp_init(void) tcp_hashinfo.ehash_size << 1, tcp_hashinfo.bhash_size);tcp_register_congestion_control (&tcp_reno); + inet_hashes[IPPROTO_TCP ] = &tcp_hashinfo; }EXPORT_SYMBOL (tcp_close);
David S. Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Thu, 4 May 2006 17:28:27 +1000
Quoted 9 lines · David S. Miller
Quoted 7 lines
Next, you can't even begin to work on the protocol channels before you do one very important piece of work. Integration of all of the ipv4 and ipv6 protocol hash tables into a central code, it's a total prerequisite. Then you modify things to use a generic inet_{,listen_}lookup() or inet6_{,listen_}lookup() that takes a protocol number as well as saddr/daddr/sport/dport and searches from a central table.
Quoted 3 lines
Back here again ;)
Is this on the right track (see patch below)?
It is on the right track.
I very much fear abuse of the inet_hashes[] array. So I'd rather hide it behind a programmatic interface, something like:
extern struct sock *inet_lookup_proto(u16 protocol, u32 saddr, u16 sport,
u32 daddr, u16 dport, int ifindex);
and export that from inet_hashtables.c
Then you have registry and unregistry functions in inet_hashtables.c that setup the static inet_hashes[] array. So TCP would go:
inet_hash_register(IPPROTO_TCP, &tcp_hashinfo);
instead of the direct assignment to inet_hashes[] it makes right now in your patch.
Thanks!
Quoted 2 lines · David S. Miller
I very much fear abuse of the inet_hashes[] array. So I'd rather hide it behind a programmatic interface, something like:
done! I will continue with implementation of default netchannel for now.
Quoted 1 line
Thanks!
anytime =)
Cheers,
K
______________________
include/net/inet_hashtables.h +3 -0
@@ -418,4 +418,7 @@ static inline struct sock *inet_lookup(sextern int inet_hash_connect (struct inet_timewait_death_row *death_row,struct sock *sk); +extern void inet_hash_register (u8 proto,struct inet_hashinfo *hashinfo); +extern struct sock *inet_lookup_proto (u8 protocol,u32 saddr,u16 sport,u32 daddr,u16 dport,int ifindex); +#endif /* _INET_HASHTABLES_H */
include/net/sock.h +1 -0
@@ -196,6 +196,7 @@ struct sock {unsigned short sk_type;int sk_rcvbuf;socket_lock_t sk_lock; +struct netchannel *sk_channel;wait_queue_head_t *sk_sleep;struct dst_entry *sk_dst_cache;struct xfrm_policy *sk_policy[2 ];
net/core/dev.c +34 -0
@@ -116,6 +116,7 @@#include <net/iw_handler.h> #include <asm/current.h> #include <linux/audit.h> +#include <net/inet_hashtables.h> /* * The list of packet types we will receive (as opposed to discard)
@@ -190,6 +191,8 @@ static inline struct hlist_head *dev_indreturn &dev_index_head[ifindex & ((1 <<NETDEV_HASHBITS )-1 )]; } +static struct netchannel default_netchannel; +/* * Our notifier list */
@@ -1907,6 +1910,34 @@ struct netchannel_buftrailer *__netchann }EXPORT_SYMBOL_GPL (__netchannel_dequeue); + +/* Find the channel for a packet, or return default channel. */ +struct netchannel *find_netchannel (const struct netchannel_buftrailer *np) +{ +struct sock *sk =NULL ; +unsigned long dlen = np->netchan_buf_len - np->netchan_buf_offset; +void *data = (void *)np - dlen; + +switch (np->netchan_buf_proto) { +case __constant_htons (ETH_P_IP ): { +struct iphdr *ip = data; +int iphl = ip->ihl *4 ; + +if (dlen >= (iphl +4 ) && iphl ==sizeof (struct iphdr )) { +u16 *ports = (u16 *)(ip +1 ); + sk =inet_lookup_proto (ip->protocol, + ip->saddr, ports[0 ], + ip->daddr, ports[1 ], + np->netchan_buf_dev->ifindex); +break ; + } + } + } +if (sk && sk->sk_channel) +return sk->sk_channel; +return &default_netchannel; +} +static gifconf_func_t * gifconf_list [NPROTO ];/**
@@ -3421,6 +3452,9 @@ static int __init net_dev_init(void)hotcpu_notifier (dev_cpu_callback,0 );dst_init ();dev_mcast_init (); + +/* FIXME: This should be attached to thread/threads. */ +netchannel_init (&default_netchannel,NULL ,NULL ); rc =0 ; out:return rc;
net/ipv4/inet_hashtables.c +22 -0
@@ -337,3 +337,25 @@ out: }EXPORT_SYMBOL_GPL (inet_hash_connect); + +static struct inet_hashinfo *inet_hashes[256 ]; + +void inet_hash_register (u8 proto,struct inet_hashinfo *hashinfo) +{ +BUG_ON (inet_hashes[proto]); + inet_hashes[proto] = hashinfo; +} +EXPORT_SYMBOL (inet_hash_register); + +struct sock *inet_lookup_proto (u8 protocol,u32 saddr,u16 sport,u32 daddr,u16 dport,int ifindex) +{ +struct sock *sk =NULL ; +if (inet_hashes[protocol]) { + sk =inet_lookup (inet_hashes[protocol], + saddr, sport, + daddr, dport, + ifindex); + } +return sk; +} +EXPORT_SYMBOL (inet_lookup_proto);
net/ipv4/tcp.c +1 -0
@@ -2173,6 +2173,7 @@ void __init tcp_init(void) tcp_hashinfo.ehash_size << 1, tcp_hashinfo.bhash_size);tcp_register_congestion_control (&tcp_reno); +inet_hash_register (IPPROTO_TCP , &tcp_hashinfo); }EXPORT_SYMBOL (tcp_close);
Quoted 1 line · Kelly Daly
done! I will continue with implementation of default netchannel for now.
___________________
include/net/inet_hashtables.h +3 -0
@@ -418,4 +418,7 @@ static inline struct sock *inet_lookup(sextern int inet_hash_connect (struct inet_timewait_death_row *death_row,struct sock *sk); +extern void inet_hash_register (u8 proto,struct inet_hashinfo *hashinfo); +extern struct sock *inet_lookup_proto (u8 protocol,u32 saddr,u16 sport,u32 daddr,u16 dport,int ifindex); +#endif /* _INET_HASHTABLES_H */
include/net/sock.h +1 -0
@@ -196,6 +196,7 @@ struct sock {unsigned short sk_type;int sk_rcvbuf;socket_lock_t sk_lock; +struct netchannel *sk_channel;wait_queue_head_t *sk_sleep;struct dst_entry *sk_dst_cache;struct xfrm_policy *sk_policy[2 ];
net/core/dev.c +69 -1
109 lines not on this page · raw message
net/ipv4/inet_hashtables.c +22 -0
@@ -337,3 +337,25 @@ out: }EXPORT_SYMBOL_GPL (inet_hash_connect); + +static struct inet_hashinfo *inet_hashes[256 ]; + +void inet_hash_register (u8 proto,struct inet_hashinfo *hashinfo) +{ +BUG_ON (inet_hashes[proto]); + inet_hashes[proto] = hashinfo; +} +EXPORT_SYMBOL (inet_hash_register); + +struct sock *inet_lookup_proto (u8 protocol,u32 saddr,u16 sport,u32 daddr,u16 dport,int ifindex) +{ +struct sock *sk =NULL ; +if (inet_hashes[protocol]) { + sk =inet_lookup (inet_hashes[protocol], + saddr, sport, + daddr, dport, + ifindex); + } +return sk; +} +EXPORT_SYMBOL (inet_lookup_proto);
net/ipv4/tcp.c +1 -0
@@ -2173,6 +2173,7 @@ void __init tcp_init(void) tcp_hashinfo.ehash_size << 1, tcp_hashinfo.bhash_size);tcp_register_congestion_control (&tcp_reno); +inet_hash_register (IPPROTO_TCP , &tcp_hashinfo); }EXPORT_SYMBOL (tcp_close);
David S. Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Tue, 16 May 2006 11:02:29 +1000
Quoted 2 lines · Kelly Daly
Quoted 1 line
done! I will continue with implementation of default netchannel for now.
Some context? It's been a week since we were discussing this, so I'd like to know what we're looking at here in this patch :)
Quoted 9 lines · David S. Miller
From: Kelly Daly <[email protected]>
Date: Tue, 16 May 2006 11:02:29 +1000Quoted 3 lines · Kelly Daly
Quoted 2 lines
done! I will continue with implementation of default netchannel for now.
Some context? It's been a week since we were discussing this, so I'd like to know what we're looking at here in this patch :)
the implementation of the default netchannel =)
Quoted 4 lines
-
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
David S. Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Tue, 16 May 2006 11:02:29 +1000
Quoted 13 lines
net/core/dev.c +13
-0+/* handles default chan buffers that nobody else wants */ +static int default_netchannel_thread (void *unused) +{ +wait_queue_t wait; +struct netchannel_buftrailer *bp; +struct sk_buff *skbp; + + wait.private = current; + wait.func = default_wake_function;; +INIT_LIST_HEAD (&wait.task_list); + +add_wait_queue (&default_netchannel_wq, &wait); +set_current_state (TASK_UNINTERRUPTIBLE );
net/core/dev.c softnet_break: · line 1890
Quoted 10 lines
net/core/dev.c +10
-0+while (!kthread_should_stop ()) { + bp =__netchannel_dequeue (&default_netchannel); + skbp =skb_netchan_graft (bp,GFP_ATOMIC ); +netif_receive_skb (skbp); + } +remove_wait_queue (&default_netchannel_wq, &wait); +__set_current_state (TASK_RUNNING ); +return 0 ; +} +
When does this thread ever go to sleep? Seems like it will loop forever and not block when the default_netchannel queue is empty. :-)
net/core/dev.c struct netchannel_buftrailer *__netchann · line 1950
Quoted 1 line
net/core/dev.c +1
-0+unsigned long dlen = np->netchan_buf_len - np->netchan_buf_offset;
Probably deserves a "netchan_buf_len(bp)" inline in linux/netchannel.h
net/ipv4/inet_hashtables.c
Quoted 3 lines
diff -urp
davem_orig/net/ipv4/inet_hashtables.ckelly/net/ipv4/inet_hashtables.cnet/ipv4/inet_hashtables.c +0
-0
The hash table bits look good, just as they did last time :-) So I'll put this part into my vj-2.6 tree now, thanks.
Quoted 2 lines
The hash table bits look good, just as they did last time :-) So I'll put this part into my vj-2.6 tree now, thanks.
Rockin' - thanks...
Sorry for the massive delay - here's the next attempt.
-------
include/linux/netchannel.h +7 -0
@@ -19,6 +19,7 @@ struct netchannel {void (*netchan_callb)(struct netchannel *);void *netchan_callb_data;unsigned long netchan_head; +wait_queue_head_t wq; }; extern void netchannel_init(struct netchannel *,
@@ -56,6 +57,11 @@ static inline unsigned char *netchan_bufreturn netchan_buf_base (bp) + bp->netchan_buf_offset; } +static inline int netchan_data_len (const struct netchannel_buftrailer *bp) +{ +return bp->netchan_buf_len - bp->netchan_buf_offset; +} +extern int netchannel_enqueue (struct netchannel *,struct netchannel_buftrailer *);extern struct netchannel_buftrailer *__netchannel_dequeue (struct netchannel *);static inline struct netchannel_buftrailer *netchannel_dequeue (struct netchannel *np)
@@ -65,6 +71,7 @@ static inline struct netchannel_buftrail return __netchannel_dequeue(np); } +extern struct netchannel *find_netchannel (const struct netchannel_buftrailer *bp);extern struct sk_buff *skb_netchan_graft (struct netchannel_buftrailer *,gfp_t );#endif /* _LINUX_NETCHANNEL_H */
include/net/inet_hashtables.h +3 -0
@@ -418,4 +418,7 @@ static inline struct sock *inet_lookup(sextern int inet_hash_connect (struct inet_timewait_death_row *death_row,struct sock *sk); +extern void inet_hash_register (u8 proto,struct inet_hashinfo *hashinfo); +extern struct sock *inet_lookup_proto (u8 protocol,u32 saddr,u16 sport,u32 daddr,u16 dport,int ifindex); +#endif /* _INET_HASHTABLES_H */
include/net/sock.h +1 -0
@@ -196,6 +196,7 @@ struct sock {unsigned short sk_type;int sk_rcvbuf;socket_lock_t sk_lock; +struct netchannel *sk_channel;wait_queue_head_t *sk_sleep;struct dst_entry *sk_dst_cache;struct xfrm_policy *sk_policy[2 ];
net/core/dev.c +86 -0
124 lines not on this page · raw message
net/core/dev.c struct netchannel_buftrailer *__netchann · line 1934
Quoted 2 lines · Kelly Daly
net/core/dev.c +2
-0+switch (bp->netchan_buf_proto) { +case __constant_htons (ETH_P_IP ): {
__constant_htons and friends should not be used in runtime code, only for data being initialized at compile time.
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 7 lines · James Morris
net/core/dev.c struct netchannel_buftrailer *__netchann · line 1934
Quoted 2 lines · Kelly Daly
net/core/dev.c +2
-0+switch (bp->netchan_buf_proto) { +case __constant_htons (ETH_P_IP ): {
__constant_htonsand friends should not be used in runtime code, only for data being initialized at compile time.
... because they generate the same code, so, to make source code less cluttered ... :-)
- Arnaldo
In article <[email protected]> (at Wed, 21 Jun 2006 23:58:56 -0400 (EDT)), James Morris <[email protected]> says:
Quoted 7 lines · Kelly Daly
net/core/dev.c struct netchannel_buftrailer *__netchann · line 1934
Quoted 3 lines
net/core/dev.c +2
-0+switch (bp->netchan_buf_proto) { +case __constant_htons (ETH_P_IP ): {
__constant_htonsand friends should not be used in runtime code, only for data being initialized at compile time.
I disagree. For "case," use __constant_{hton,ntoh}{s,l}(), please.
--yoshfuji
David Miller
maintainerFrom: Kelly Daly <[email protected]>
Date: Thu, 22 Jun 2006 12:05:35 +1000
Quoted 5 lines
Quoted 2 lines
The hash table bits look good, just as they did last time :-) So I'll put this part into my vj-2.6 tree now, thanks.
Rockin' - thanks...
Sorry for the massive delay - here's the next attempt.
My review delay was just as bad if not worse :-)
net/core/dev.c struct netchannel_buftrailer *__netchann · line 1968
Quoted 12 lines
net/core/dev.c +12
-0+static int sock_add_netchannel (struct sock *sk) +{ +struct netchannel *np; + + np =kmalloc (sizeof (struct netchannel ),GFP_KERNEL ); +if (!np) +return -ENOMEM ; +netchannel_init (np, netchannel_wake, (void *)np); + sk->sk_channel = np; + +return 0 ; +}
This function is unreferenced entirely? It's marked static, so don't bother including it unless it is being used.
Fix this, give me a good changelog and signed-off-by line and I'll stick this into the vj-2.6 tree
Thanks!
PATCH 1/1 netchannel subsystem.
In reply to Kelly DalyLet me also bring attention to another netchannel implementation.
Some design notes [blog copypasts, sorry if it is out of sync sometimes].
First of all, do not use sockets. Just forget that such interface exists.
New receiving channel abstraction will be created by special syscall, which allows to specify in creation time source IP range, or it can be wildcarded to IPADDR_ANY.
Interrupt handler (or netif_receive_skb()) will check skb and try to get netchannel, then skb will be linked into netchannel receive queue if memory limits allow it and that is all. No protocol processing. If hardware allows to split header from data it will be possible to create zero-copy technique: when netchannel is selected in netif_receive_skb() it will have ->alloc_data() callback which would allow, for example, to allocate data from userspace mapped area. When receiver is awakened after data has been added to netchannel, it will call netchannel's ->get_data() callback which would allow to remap data pages from skb into process' VMA.
Unified network cache will hold information about source and destination IP addresses, or it's hashes for IPv6, information about ports and well-defined IP protocols.
It is built as two dimensional array of default 8 order (i.e. 64k entries) with XOR hash. Comparison of Jenkins hash with XOR hash used in TCP socket selection code can be found at [1].
Main problem here is routing cache. Linux is great in routing, it supports several cache algorithms, which are extremely fast in appropriate environments, but main question is "do we need routing for netchannels"?. So it looks like netchannel either must include that route interface or, if it will be designed as userspace-only communication channel, implement own trivial pseudo-route algorithm.
Most of the drivers do not set skb->pkt_type, so it stays default PACKET_HOST, which means that every packet, received by interface must be checked to be valid for the system, which is a point to use standard Linux routing, but from the other point of view, it is quite easy to create own callback for IP address setup, which would update netchannel routing table.
No one need an excuse to rewrite anything, so I will create own cache for netchannels, which will be updated when IP addresses are changed.
Ok, basic interfaces have been created:
* unified cache to store netchannels (IPv4 and stub for IPv6 hashes, TCP and UDP)
* skb queueing mechanism
* netchannel creation/removing commands
* netchannel's callback to allocate/free pages (for example to get data from mapped area) not only from SLAB cache
There are only couple of things left (today maybe):
* create netchannel's callback to move/copy data to userspace
* create read data command
* test new netchannel interface :) (it boots and works ok with netchannel interface turned on for usual sockets)
If you have read all above, here is link to hash comparison [1].
[1]. Compared Jenkins hash with XOR hash used in TCP socket selection code.
http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14
And proof-of-concept patch itself.
arch/i386/kernel/syscall_table.S +1 -0
@@ -315,3 +315,4 @@ ENTRY(sys_call_table) .long sys_splice .long sys_sync_file_range .long sys_tee /* 315 */ + .long sys_netchannel_control
arch/x86_64/ia32/ia32entry.S +1 -0
@@ -696,4 +696,5 @@ ia32_sys_call_table: .quad sys_sync_file_range .quad sys_tee .quad compat_sys_vmsplice + .quad sys_netchannel_control ia32_syscall_end:
include/asm-x86_64/unistd.h +3 -1
@@ -617,8 +617,10 @@ __SYSCALL(__NR_tee, sys_tee) __SYSCALL(__NR_sync_file_range, sys_sync_file_range)#define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice) +#define __NR_netchannel_control 279 +__SYSCALL(__NR_vmsplice, sys_netchannel_control) -#define __NR_syscall_max __NR_vmsplice +#define __NR_syscall_max __NR_netchannel_control#ifndef __NO_STUBS
include/linux/netchannel.h new file +73 -0
73 lines not on this page · raw message
include/linux/netdevice.h +9 -0
@@ -684,6 +684,15 @@ extern void dev_queue_xmit_nit(struct sextern void dev_init (void ); +#ifdef CONFIG_NETCHANNEL +extern int netchannel_recv (struct sk_buff *skb); +#else +static int netchannel_recv (struct sk_buff *skb) +{ +return -1 ; +} +#endif +extern int netdev_nit;extern int netdev_budget;
include/linux/skbuff.h +12 -0
@@ -301,6 +301,7 @@ struct sk_buff {* Handling routines are only of interest to the kernel */ #include <linux/slab.h> +#include <linux/netchannel.h> #include <asm/system.h>
@@ -314,6 +315,17 @@ static inline struct sk_buff *alloc_skb(return __alloc_skb (size, priority,0 ); } +#ifdef CONFIG_NETCHANNEL +extern struct sk_buff *netchannel_alloc (struct unetchannel *unc,unsigned int header_size, +unsigned int total_size,gfp_t gfp_mask); +#else +static struct sk_buff *netchannel_alloc (struct unetchannel *unc,unsigned int header_size, +unsigned int total_size,gfp_t gfp_mask) +{ +return NULL ; +} +#endif +static inline struct sk_buff *alloc_skb_fclone (unsigned int size,gfp_t priority) {
include/linux/syscalls.h +2 -0
@@ -582,4 +582,6 @@ asmlinkage long sys_tee(int fdin, int fdasmlinkage long sys_sync_file_range (int fd,loff_t offset,loff_t nbytes,unsigned int flags); +asmlinkage int sys_netchannel_control(void __user *arg); +#endif
kernel/sys_ni.c +2 -0
@@ -132,3 +132,5 @@ cond_syscall(sys_mincore);cond_syscall (sys_madvise);cond_syscall (sys_mremap);cond_syscall (sys_remap_file_pages); + +cond_syscall (sys_netchannel_control);
net/Kconfig +8 -0
@@ -66,6 +66,14 @@ source "net/ipv6/Kconfig" endif # if INET +config NETCHANNEL + bool "Network channels" + ---help--- + Network channels are peer-to-peer abstraction, which allows to create + high performance communications. + Main advantages are unified address cache, protocol processing moved + to userspace, receiving zero-copy support and other interesting features. + menuconfig NETFILTER bool "Network packet filtering (replaces ipchains)" ---help---
net/core/Makefile +1 -0
@@ -16,3 +16,4 @@ obj-$(CONFIG_NET_DIVERT) += dv.o obj-$(CONFIG_NET_PKTGEN) += pktgen.o obj-$(CONFIG_WIRELESS_EXT) += wireless.o obj-$(CONFIG_NETPOLL) += netpoll.o +obj-$(CONFIG_NETCHANNEL) += netchannel.o
net/core/dev.c +4 -0
@@ -1712,6 +1712,10 @@ int netif_receive_skb(struct sk_buff *sk } } + ret =netchannel_recv (skb); +if (!ret) + goto out; +#ifdef CONFIG_NET_CLS_ACT if (pt_prev) { ret =deliver_skb (skb, pt_prev, orig_dev);
net/core/netchannel.c new file +583 -0
583 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov
David S. Miller
maintainerRe: [1/1] netchannel subsystem.
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 16 May 2006 10:19:09 +0400
Quoted 2 lines
net/core/netchannel.c new file +2
-0+static int netchannel_convert_skb_ipv4 (struct sk_buff *skb,struct unetchannel *unc) +{
...
Quoted 2 lines
net/core/netchannel.c new file +2
-0+switch (unc->proto) { +case IPPROTO_TCP :
...
net/core/netchannel.c line 199
Quoted 1 line
net/core/netchannel.c new file +1
-0+case IPPROTO_UDP :
...
Why do people write code like this?
Port location is protocol agnostic, there are always 2 16-bit ports at beginning of header without exception.
Without this, ICMP would be useless :-)
Re: [1/1] netchannel subsystem.
Quoted 12 lines · David S. Miller
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 16 May 2006 10:19:09 +0400Quoted 2 lines
net/core/netchannel.c new file +2
-0+static int netchannel_convert_skb_ipv4 (struct sk_buff *skb,struct unetchannel *unc) +{...
Quoted 2 lines
net/core/netchannel.c new file +2
-0+switch (unc->proto) { +case IPPROTO_TCP :...
Quoted 1 line
net/core/netchannel.c new file +1
-0+case IPPROTO_UDP :...
Quoted 6 lines
Why do people write code like this?
Port location is protocol agnostic, there are always 2 16-bit ports at beginning of header without exception.
Without this, ICMP would be useless :-)
And what if we use ESP which would place it's hashed sequence number as port?
Signature · 1 line
Evgeniy Polyakov
Re: [1/1] netchannel subsystem.
Quoted 15 lines · Evgeniy Polyakov
Quoted 14 lines · David S. Miller
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 16 May 2006 10:19:09 +0400Quoted 2 lines
(quoted diff) +2
-0+static int netchannel_convert_skb_ipv4(struct sk_buff *skb, struct unetchannel *unc) +{...
Quoted 2 lines
(quoted diff) +2
-0+ switch (unc->proto) { + case IPPROTO_TCP:...
Review
Quoted 1 line
(quoted diff) +1
-0+ case IPPROTO_UDP:...
Why do people write code like this?
Quoted 7 lines
Quoted 4 lines
Port location is protocol agnostic, there are always 2 16-bit ports at beginning of header without exception.
Without this, ICMP would be useless :-)
And what if we use ESP which would place it's hashed sequence number as port?
Actually it should be one big hash, no matter if it is ipv4/tcp or ipv6/esp, src/dst/sport/dport/proto were created just to allow easier debug in ipv4 environment.
Attached patch for userspace copy:
tmp/netchannel.2 renamed from tmp/netchannel.1 +68 -1
@@ -35,10 +35,10 @@ diff --git a/include/linux/netchannel.h b/include/linux/netchannel.h
@@ -100,6 +100,7 @@ + + struct page * (*nc_alloc_page)(unsigned int size); + void (*nc_free_page)(struct page *page); ++ int (*nc_read_data)(struct netchannel *, unsigned int *len, void __user *arg); + + struct sk_buff_head list; +};
@@ -228,10 +229,10 @@ ret = deliver_skb(skb, pt_prev, orig_dev); diff --git a/net/core/netchannel.c b/net/core/netchannel.c --- /dev/null +++ b/net/core/netchannel.c -@@ -0,0 +1,583 @@ +@@ -0,0 +1,649 @@ +/* + * netchannel.c + *
@@ -578,6 +579,40 @@ + return err; +} + ++/* ++ * Actually it should be something like recvmsg(). ++ */ ++static int netchannel_copy_to_user(struct netchannel *nc, unsigned int *len, void __user *arg) ++{ ++ unsigned int copied; ++ struct sk_buff *skb; ++ struct iovec to; ++ int err = -EINVAL; ++ ++ to.iov_base = arg; ++ to.iov_len = *len; ++ ++ skb = skb_dequeue(&nc->list); ++ if (!skb) ++ return -EAGAIN; ++ ++ copied = skb->len; ++ if (copied > *len) ++ copied = *len; ++ ++ if (skb->ip_summed==CHECKSUM_UNNECESSARY) { ++ err = skb_copy_datagram_iovec(skb, 0, &to, copied); ++ } else { ++ err = skb_copy_and_csum_datagram_iovec(skb,0, &to); ++ } ++ ++ *len = (err == 0)?copied:0; ++ ++ kfree_skb(skb); ++ ++ return err; ++} ++ +static int netchannel_create(struct unetchannel *unc) +{ + struct netchannel *nc;
@@ -612,6 +647,8 @@ + atomic_set(&nc->refcnt, 1); + memcpy(&nc->unc, unc, sizeof(struct unetchannel)); + ++ nc->nc_read_data = &netchannel_copy_to_user; ++ + hlist_add_head_rcu(&nc->node, &bucket->head); + +out_unlock:
@@ -658,6 +695,30 @@ + return 0; +} + ++static int netchannel_recv_data(struct unetchannel_control *ctl, void __user *data) ++{ ++ int ret = -ENODEV; ++ struct netchannel_cache_head *bucket; ++ struct netchannel *nc; ++ ++ bucket = netchannel_bucket(&ctl->unc); ++ ++ mutex_lock(&bucket->mutex); ++ ++ nc = netchannel_check_full(&ctl->unc, bucket); ++ if (!nc) ++ nc = netchannel_check_dest(&ctl->unc, bucket); ++ ++ if (!nc) ++ goto out_unlock; ++ ++ ret = nc->nc_read_data(nc, &ctl->len, data); ++ ++out_unlock: ++ mutex_unlock(&bucket->mutex); ++ return ret; ++} ++ +asmlinkage int sys_netchannel_control(void __user *arg) +{ + struct unetchannel_control ctl;
@@ -677,10 +738,16 @@ + case NETCHANNEL_REMOVE: + ret = netchannel_remove(&ctl.unc); + break; ++ case NETCHANNEL_READ: ++ ret = netchannel_recv_data(&ctl, arg + sizeof(struct unetchannel_control)); ++ break; + default: + ret = -EINVAL; + break; + } ++ ++ if (copy_to_user(arg, &ctl, sizeof(struct unetchannel_control))) ++ return -ERESTARTSYS; + + return ret; +}
Signature · 1 line
Evgeniy Polyakov
David S. Miller
maintainerRe: [1/1] netchannel subsystem.
In reply to Evgeniy PolyakovFrom: Evgeniy Polyakov <[email protected]>
Date: Tue, 16 May 2006 10:59:23 +0400
Quoted 2 lines
And what if we use ESP which would place it's hashed sequence number as port?
If it makes you happy put something like:
case TCP: case UDP: case SCTP: case DCCP: ... default: ...
Re: [1/1] netchannel subsystem.
Quoted 8 lines · David S. Miller
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 16 May 2006 10:59:23 +0400Quoted 2 lines
And what if we use ESP which would place it's hashed sequence number as port?
If it makes you happy put something like:
Quoted 8 lines
case TCP: case UDP: case SCTP: case DCCP: ... default: ...
That makes sence, but from the other point we loose ability to drop packet early while checking it's header for correct checksum, sizes and so on...
Signature · 1 line
Evgeniy Polyakov
PATCH 1/1 Netchannel subsyste.
In reply to David S. MillerReceiving support.
As proof-of-concept code I created simple copy_to_user() based getting data callback. Next step is to implement netchannels data allocation callbacks to get data from mapped userspace area and make get data callback be similar to ->recvmsg() so all protocol processing happens in userspace (TCP should start working, now only UDP case) if there is some interest in it. Patch attached.
There are brief description of netchannel design and implementation, patches and userspace utility at project's homepage [1].
Thank you.
1. Netchannel project homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
Receiving netchannel implementation.
arch/i386/kernel/syscall_table.S +2 -0
@@ -315,3 +315,5 @@ ENTRY(sys_call_table) .long sys_splice .long sys_sync_file_range .long sys_tee /* 315 */ + .long sys_vmsplice + .long sys_netchannel_control
arch/x86_64/ia32/ia32entry.S +1 -0
@@ -696,4 +696,5 @@ ia32_sys_call_table: .quad sys_sync_file_range .quad sys_tee .quad compat_sys_vmsplice + .quad sys_netchannel_control ia32_syscall_end:
include/asm-i386/unistd.h +2 -1
@@ -322,8 +322,9 @@#define __NR_sync_file_range 314#define __NR_tee 315#define __NR_vmsplice 316 +#define __NR_netchannel_control 317 -#define NR_syscalls 317 +#define NR_syscalls 318/* * user-visible error numbers are in the range -1 - -128: see
include/asm-x86_64/unistd.h +3 -1
@@ -617,8 +617,10 @@ __SYSCALL(__NR_tee, sys_tee) __SYSCALL(__NR_sync_file_range, sys_sync_file_range)#define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice) +#define __NR_netchannel_control 279 +__SYSCALL(__NR_vmsplice, sys_netchannel_control) -#define __NR_syscall_max __NR_vmsplice +#define __NR_syscall_max __NR_netchannel_control#ifndef __NO_STUBS
include/linux/netchannel.h new file +75 -0
75 lines not on this page · raw message
include/linux/netdevice.h +9 -0
@@ -684,6 +684,15 @@ extern void dev_queue_xmit_nit(struct sextern void dev_init (void ); +#ifdef CONFIG_NETCHANNEL +extern int netchannel_recv (struct sk_buff *skb); +#else +static int netchannel_recv (struct sk_buff *skb) +{ +return -1 ; +} +#endif +extern int netdev_nit;extern int netdev_budget;
include/linux/skbuff.h +12 -0
@@ -301,6 +301,7 @@ struct sk_buff {* Handling routines are only of interest to the kernel */ #include <linux/slab.h> +#include <linux/netchannel.h> #include <asm/system.h>
@@ -314,6 +315,17 @@ static inline struct sk_buff *alloc_skb(return __alloc_skb (size, priority,0 ); } +#ifdef CONFIG_NETCHANNEL +extern struct sk_buff *netchannel_alloc (struct unetchannel *unc,unsigned int header_size, +unsigned int total_size,gfp_t gfp_mask); +#else +static struct sk_buff *netchannel_alloc (struct unetchannel *unc,unsigned int header_size, +unsigned int total_size,gfp_t gfp_mask) +{ +return NULL ; +} +#endif +static inline struct sk_buff *alloc_skb_fclone (unsigned int size,gfp_t priority) {
include/linux/syscalls.h +2 -0
@@ -582,4 +582,6 @@ asmlinkage long sys_tee(int fdin, int fdasmlinkage long sys_sync_file_range (int fd,loff_t offset,loff_t nbytes,unsigned int flags); +asmlinkage long sys_netchannel_control (void __user *arg); +#endif
kernel/sys_ni.c +2 -0
@@ -132,3 +132,5 @@ cond_syscall(sys_mincore);cond_syscall (sys_madvise);cond_syscall (sys_mremap);cond_syscall (sys_remap_file_pages); + +cond_syscall (sys_netchannel_control);
net/Kconfig +8 -0
@@ -66,6 +66,14 @@ source "net/ipv6/Kconfig" endif # if INET +config NETCHANNEL + bool "Network channels" + ---help--- + Network channels are peer-to-peer abstraction, which allows to create + high performance communications. + Main advantages are unified address cache, protocol processing moved + to userspace, receiving zero-copy support and other interesting features. + menuconfig NETFILTER bool "Network packet filtering (replaces ipchains)" ---help---
net/core/Makefile +1 -0
@@ -16,3 +16,4 @@ obj-$(CONFIG_NET_DIVERT) += dv.o obj-$(CONFIG_NET_PKTGEN) += pktgen.o obj-$(CONFIG_WIRELESS_EXT) += wireless.o obj-$(CONFIG_NETPOLL) += netpoll.o +obj-$(CONFIG_NETCHANNEL) += netchannel.o
net/core/dev.c +4 -0
@@ -1712,6 +1712,10 @@ int netif_receive_skb(struct sk_buff *sk } } + ret =netchannel_recv (skb); +if (!ret) + goto out; +#ifdef CONFIG_NET_CLS_ACT if (pt_prev) { ret =deliver_skb (skb, pt_prev, orig_dev);
net/core/netchannel.c new file +691 -0
691 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov
Netchannel subsystem update.
This updates brings new features to the following supported:
* unified cache to store netchannels (IPv4 and stub for fied cache to store netchannels (IPv4 and stub for IPv6 hashes, TCP and UDP)
* skb queueing mechanism
* netchannel creation/removing/reading commands
* netchannel's callback to allocate/free pages (for example to get data from mapped area) not only from SLAB cache
* netchannel's callback to move/copy data to userspace
Added:
* memory limits (soft limits, since update is not protected).
* blocking reading.
* two types of data reading backends (copy_to_user(), copy to (could be mapped) area).
Patch against previous release is attached.
Userspace application, design and implementation notes, full patchsets can be found at project homepage [1].
1. Network channel homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
I would like to rise a question about how netchannel object should be handled by system in general, i.e. should netchannels be associated with process or they should live by themselfs, i.e. like routes? My implementation allows netchannels to be setup permanently, so process can exit and then new one can bind to existing netchannel and read it's data, but it requires some tricks to create mapping of it's pages into process' context...
Also if netchannel is created, but no process is associated with it, who will process protocol state machine?
include/linux/netchannel.h +28 -4
@@ -32,13 +32,20 @@ enum netchannel_commands { NETCHANNEL_DUMP, }; +enum netchannel_type { +NETCHANNEL_COPY_USER =0 , +NETCHANNEL_MMAP , +NETCHANEL_VM_HACK , +}; +struct unetchannel {__u32 src, dst;/* source/destination hashes */ __u16 sport, dport;/* source/destination ports */ __u8 proto;/* IP protocol number */ -__u8 listen; -__u8 reserved[2 ]; +__u8 type;/* Netchannel type */ +__u8 memory_limit_order;/* Memor limit order */ +__u8 reserved; };struct unetchannel_control
@@ -46,6 +53,8 @@ struct unetchannel_controlstruct unetchannel unc;__u32 cmd;__u32 len; +__u32 flags; +__u32 timeout; };#ifdef __KERNEL__
@@ -60,9 +69,14 @@ struct netchannelstruct page * (*nc_alloc_page)(unsigned int size);void (*nc_free_page)(struct page *page); -int (*nc_read_data)(struct netchannel *,unsigned int *len,void __user *arg); +int (*nc_read_data)(struct netchannel *,unsigned int *timeout,unsigned int *len,void *arg); + +struct sk_buff_head recv_queue; +wait_queue_head_t wait; + +unsigned int qlen; -struct sk_buff_head list; +void *priv; };struct netchannel_cache_head
@@ -71,5 +85,15 @@ struct netchannel_cache_headstruct mutex mutex; }; +#define NETCHANNEL_MAX_ORDER 32 +#define NETCHANNEL_MIN_ORDER PAGE_SHIFT + +struct netchannel_mmap +{ +struct page **page; +unsigned int pnum; +unsigned int poff; +}; +#endif /* __KERNEL__ */ #endif /* __NETCHANNEL_H */
include/linux/skbuff.h +2 -2
@@ -301,7 +301,6 @@ struct sk_buff {* Handling routines are only of interest to the kernel */ #include <linux/slab.h> -#include <linux/netchannel.h> #include <asm/system.h>
@@ -316,10 +315,11 @@ static inline struct sk_buff *alloc_skb( }#ifdef CONFIG_NETCHANNEL +struct unetchannel ;extern struct sk_buff *netchannel_alloc (struct unetchannel *unc,unsigned int header_size,unsigned int total_size,gfp_t gfp_mask);#else -static struct sk_buff *netchannel_alloc (struct unetchannel *unc,unsigned int header_size, +static struct sk_buff *netchannel_alloc (void *unc,unsigned int header_size,unsigned int total_size,gfp_t gfp_mask) {return NULL ;
net/core/netchannel.c +336 -35
465 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov
Re: Netchannel subsystem update.
The more I think about TCP processing in netchannels, the more I get close to the following ideas:
* map netchannel to socket.
* implement own TCP (receiving for now) state machine.
So I would like to ask people, what do we want for netchannels
* existing Linux TCP stack
* fairly simple minimalistic RFC compliant stack
While developing first apporoach I've found that input TCP processing sometimes refers to dst_entry which can only be obtained through the input routing code. You can find appropriate changes in attached incremental patch. Full netchannel patch can be found at homepage [1].
Implementations is fairly proof-of-concept,
since I do not like the idea to bind netchannel to socket.<br/> All TCP state machine is handled inside socket code, so userspace must create listening socket, wait until new connection is created, accept it and the bind netchannel to the newly created socket for established connection. All further data flow is handled inside netchannels, but actually it is not working as expected yet.
So question is how to process TCP state machine for netchannels: bind them to socket and use existing code, or create small netchannel TCP state machine?
1. Netchannel homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
Initial TCP support for netchannels. Incremental patch. Proof-of-concept only.
include/linux/netchannel.h +3 -0
@@ -55,6 +55,7 @@ struct unetchannel_control__u32 len;__u32 flags;__u32 timeout; +unsigned int fd; };#ifdef __KERNEL__
@@ -77,6 +78,8 @@ struct netchannelunsigned int qlen;void *priv; + +struct inode *inode; };struct netchannel_cache_head
net/core/netchannel.c +150 -33
254 lines not on this page · raw message
net/ipv4/tcp_ipv4.c +11 -2
@@ -727,7 +727,10 @@ int tcp_v4_conn_request(struct sock *sk,#endif /* Never answer to SYNs send to broadcast or multicast */ -if (((struct rtable *)skb->dst)->rt_flags & +if (!skb->dst) { +if (MULTICAST (daddr)) + goto drop; + }else if (((struct rtable *)skb->dst)->rt_flags & (RTCF_BROADCAST |RTCF_MULTICAST )) goto drop;
@@ -924,15 +927,21 @@ static struct sock *tcp_v4_hnd_req(strucstruct iphdr *iph = skb->nh.iph;struct sock *nsk;struct request_sock **prev; +int iif;/* Find possible connection requests. */ struct request_sock *req =inet_csk_search_req (sk, &prev, th->source, iph->saddr, iph->daddr);if (req)return tcp_check_req (sk, skb, req, prev); +if (!skb->dst) + iif =0 ; +else + iif =inet_iif (skb); + nsk =__inet_lookup_established (&tcp_hashinfo, skb->nh.iph->saddr, th->source, skb->nh.iph->daddr, -ntohs (th->dest),inet_iif (skb)); +ntohs (th->dest), iif);if (nsk) {if (nsk->sk_state !=TCP_TIME_WAIT ) {
Signature · 1 line
Evgeniy Polyakov
David S. Miller
maintainerRe: Netchannel subsystem update.
From: Evgeniy Polyakov <[email protected]>
Date: Sat, 20 May 2006 19:52:02 +0400
Quoted 3 lines
While developing first apporoach I've found that input TCP processing sometimes refers to
dst_entrywhich can only be obtained through the input routing code. You can find appropriate changes in attached incremental patch.
It would be no trouble to cache the input route in the socket when netchannel is enabled.
[Netchannel] Full TCP receiving support.
Hello, developers.
Attached patch implements full TCP input processing for netchannels [1]. It is based on socket processing code and is fairly hairy for now. Main idea is to queue skbs into netchannels private queue in interrupt time and then remove skbs and process them in process' context. To make TCP works userspace procesing code should only perform several simple steps similar to how backlog is processed in socket code.
Attached patch against previously posted netchannel patches which mostly implements netchannel_copy_to_user_tcp() function which performs TCP processing and copies dat ato userspace. As you can see it is quite trivial.
Current state is quite proof-of-concept, since there are some ugliness in the code and various uninteresting debugs, so I plan to clean this up and run some tests to show if such approach works or not.
Full patch and userspace application are available from netchannel homepage [1].
Thank you.
1. Netchannel homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
net/core/netchannel.c +65 -37
@@ -34,6 +34,7 @@#include <linux/in.h> #include <linux/ip.h> #include <linux/tcp.h> +#include <net/tcp.h> #include <linux/udp.h> #include <linux/netdevice.h>
@@ -221,6 +222,13 @@ static int netchannel_convert_skb_ipv4(sif (skb->len < len || len < (iph->ihl*4 )) goto inhdr_error; +if (pskb_trim_rcsum (skb, len)) + goto inhdr_error; + +if (iph->ihl >5 ) +printk ("netchannel: IP options: %u.%u.%u.%u -> %u.%u.%u.%u, ihl: %u.\n" , +NIPQUAD (iph->saddr),NIPQUAD (iph->daddr), iph->ihl); + unc->dst = iph->daddr; unc->src = iph->saddr; unc->proto = iph->protocol;
@@ -388,9 +396,12 @@ int netchannel_recv(struct sk_buff *skb) goto unlock; } -skb_queue_tail (&nc->recv_queue, skb); nc->qlen += skb->len; +skb_queue_tail (&nc->recv_queue, skb);wake_up (&nc->wait); + +if (nc->inode &&SOCKET_I (nc->inode)->sk) +wake_up (SOCKET_I (nc->inode)->sk->sk_sleep); unlock:rcu_read_unlock ();
@@ -454,58 +465,75 @@ static int netchannel_copy_to_user_tcp(sstruct socket *sock;struct sock *sk;struct sk_buff *skb; - - skb =netchannel_get_skb (nc, timeout, &err); -if (!skb) -return err; +struct iovec iov; +struct msghdr msg; +unsigned flags =MSG_DONTWAIT ;if (!nc->inode) - goto err_out_free; + goto err_out; sock =SOCKET_I (nc->inode);if (!sock || !sock->sk) - goto err_out_free; + goto err_out; sk = sock->sk; -__skb_pull (skb, skb->nh.iph->ihl*4 ); +do { + msg.msg_control=NULL ; + msg.msg_controllen=0 ; + msg.msg_iovlen=1 ; + msg.msg_iov=&iov; + msg.msg_name=NULL ; + msg.msg_namelen=0 ; + msg.msg_flags = flags; + iov.iov_len=*len; + iov.iov_base=arg; - skb->h.raw = skb->data; + err =sock_recvmsg (sock, &msg, iov.iov_len, flags); - th = skb->h.th; +printk ("netchannel: TCP: len: %u, err: %d.\n" , *len, err); -printk ("netchannel: TCP: syn: %u, fin: %u, rst: %u, psh: %u, ack: %u, urg: %u, ece: %u, cwr: %u, res1: %u, doff: %u.\n" , - th->syn, th->fin, th->rst, th->psh, th->ack, th->urg, th->ece, th->cwr, th->res1, th->doff); - -if (sk->sk_state ==TCP_ESTABLISHED ) { -struct iovec to; -unsigned int copied; - - to.iov_base = arg; - to.iov_len = *len; +if (err >0 ) { + *len = err; +return 0 ; + }else if (err && err != -EAGAIN ) +return err; - copied = skb->len; -if (copied > *len) - copied = *len; + err =0 ; -if (skb->ip_summed ==CHECKSUM_UNNECESSARY ) { - err =skb_copy_datagram_iovec (skb,0 , &to, copied); - }else { - err =skb_copy_and_csum_datagram_iovec (skb,0 , &to); - } + skb =netchannel_get_skb (nc, timeout, &err); +if (!skb) +return err; + +__skb_pull (skb, skb->nh.iph->ihl*4 ); + + skb->h.raw = skb->data; + + th = skb->h.th; +TCP_SKB_CB (skb)->seq =ntohl (th->seq); +TCP_SKB_CB (skb)->end_seq = (TCP_SKB_CB (skb)->seq + th->syn + th->fin + + skb->len - th->doff *4 ); +TCP_SKB_CB (skb)->ack_seq =ntohl (th->ack_seq); +TCP_SKB_CB (skb)->when =0 ; +TCP_SKB_CB (skb)->flags = skb->nh.iph->tos; +TCP_SKB_CB (skb)->sacked =0 ; - *len = (err ==0 )?copied:0 ; - } - - nc->qlen -= skb->len; +printk ("netchannel: TCP: syn: %u, fin: %u, rst: %u, psh: %u, ack: %u, urg: %u, ece: %u, cwr: %u, res1: %u, doff: %u.\n" , + th->syn, th->fin, th->rst, th->psh, th->ack, th->urg, th->ece, th->cwr, th->res1, th->doff); + + nc->qlen -= skb->len; - err = sk->sk_backlog_rcv (sk, skb); -printk ("netchannel: TCP: sk_backlog_rcv() ret: %d.\n" , err); -return err; + err = sk->sk_backlog_rcv (sk, skb); + +printk ("netchannel: TCP: seq=%u, ack=%u, sk_state=%u, backlog_err: %d, sock_qlen: %u.\n" , + th->seq, th->ack_seq, sk->sk_state, err,skb_queue_len (&sk->sk_receive_queue)); + +if (err) +return err; + }while (!err); -err_out_free: - nc->qlen -= skb->len; -kfree_skb (skb); +return 0 ; +err_out:return err; }
Signature · 1 line
Evgeniy Polyakov
Re: [Netchannel] Full TCP receiving support.
Hello, developers.
Initial TCP benchmark.
After tweaking some stuff I run netchannel vs socket TCP benchmark. Unfortunately currently I'm unable to run 1Gbit test, since my test machine is slightly broken... So only 100Mbit for now.
Performance graph attached.
Speed is the same, CPU usage is the same
(socket CPU usage is 1% (or 20% :) more than netchannel one, but let's throw this away as some experimental error).
Netchannel was setup in copy_to_user mode, so it was not expected to be faster or eat less CPU, but main purpose of this test was to show that netchannels with _all_ protocol processing moved to process' context can be at least that fast as when it is splitted into pieces.
I've found some links in the web and blogs where some developers completely disagree with VJ idea of moving stuff into process context... Well, this should break theirs mostly theoretical arguments.
It is clear that using memcpy setup CPU usage numbers (at least) for netchannel are noticebly better than for sockets (see previous posted benchmarks). I will start changing core networking code to accept different copying "actor" methods which will allow to use netchannels preallocated mapped area instead of copy_to_user().
Full patch and userspace application are available from netchannel homepage [1].
Thank you.
1. Netchannel homepage.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
Signature · 1 line
Evgeniy Polyakov