Kernel hooks just to get rid of copy_[to/from]_user() and syscall overhead?

Hi,

I have to implement a system which grabs udp packets off a gigabit connection, take some basic action based on what they contain, repack their data with a custom protocol header and send them through a gigabit ethernet interface on broadcast.

I know how to do this in userspace, but I need to know if doing everyting in the kernel would show a considerable speed improvement due to removing syscall and memory copy overhead. The system will be quite stressed, having to deal with around 15-20000 packets/second.

I didn't want to start this e-mail with an excuse, so I delayed it until here :). I appologise if this isn't the right place to ask, it seemed that way to me. I wasn't able to find sufficient and coherent information about this issue on the internet or on this mailing list's archives, so I decided to ask you people directly.

Thanks for your time,
Mihnea Balta

I think you should do everything in kernel space, with a user-mode interface for non-realtime control, i.e., what characteristics of UDP packets are being "filtered". You just make a module that contains what you need, with an ioctl() hook to control it.

That way, data from your gigabit interface(s) never has to get to "user-space" at all.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips). Why is the government concerned about the lunatic fringe? Think about it.

mmap() packet socket interface eliminates the need for system calls when traffic is coming in at a high rate. The kernel -> user copy is also eliminated, but its just replaced with a kernel -> kernel copy :P

You could perhaps also use linux socket filters to minimize the number of packets you need to evaluate...

Check out this sample code: http://www.scaramanga.co.uk/code-fu/lincap.c

HTH

// Gianni Tedesco (gianni at scaramanga dot co dot uk)
lynx --source www.scaramanga.co.uk/gianni-at-ecsc.asc | gpg --import
8646BE7D: 6D9F 2287 870E A2C9 8F60 3A3C 91B5 7669 8646 BE7D

Thank you to everybody who answered, I'll try the mmap()ed descriptor userspace solution, as it seems good enough.

Following the indications I got here, I did the packet recieving stuff using a mmaped packet socket. From what I understand, that is a recieve-only interface, so it seems that I'm stuck with old sendto() for putting the packets back on the wire. I'd like to know if sendto() can do 20000 (small) packets/second on a fast x86 MP machine (dual or maybe quad) which doesn't do much besides this routing process. If it can't, please tell me if there's any feasible way of implementing a kind of buffer, i.e. keeping recieved packets in a local buffer and sending them alltoghether when the buffer fills or after a timer expires (I'm thinking DMA to the NIC or something simmilar, as the buffer will contain the complete packets, with all the required packets).

Thanks,
Mihnea

PS: feasible == not spending 1 month development time for a 1 microsecond/packet improvement.