Interconnect virtual device?

Hello!

I am considering writing a kernel module to provide a very simple virtual interface. This interface would be attached to another interface, and would not be directly associated with hardware. It can have IP addresses & routing tables, and in almost every way appear to be a regular ethernet interface.

The idea is that when you tx on this virtual interface, it will cause a packet to be received on another interface. And, when a packet is received on the interface, it will tx on the other interface.

I believe I can use this to create some interesting effects with routing and bridging on a limitted number of physical interfaces.

Quoted 8 lines · Ben Greear

Hello!

I am considering writing a kernel module to provide a very simple virtual interface. This interface would be attached to another interface, and would not be directly associated with hardware. It can have IP addresses & routing tables, and in almost every way appear to be a regular ethernet interface.

Quoted 5 lines

The idea is that when you tx on this virtual interface, it will cause a packet to be received on another interface. And, when a packet is received on the interface, it will tx on the other interface.

In other words, it redirects to another device - I take it based on some rules. Sounds to me like the mirred action already does this (and in addition can mirror).

jamal wrote:

Quoted 3 lines

In other words, it redirects to another device - I take it based on some rules. Sounds to me like the mirred action already does this (and in addition can mirror).

Actually, I was thinking that either user-space or perhaps a kernel module could use the standard methods of receiving all pkts in promisc mode to do the bridging portion. If I force the real ethernet interface to have no IP address, and put it into promisc mode, then I can effectively bridge in user-space.

When I re-insert the pkts into the stack by writing to the virtual device, the (potentially modified) packet can be
routed and firewalled, etc just like a normal packet received from the network. I may need to have two virtual inter-connects, one w/out an IP that does the bridging portion, and one with an IP that actually communicates with the rest of the kernel. In this case, the two virtuals would be connected to each other.

I think this could provide a way to do very customized actions on raw packets without having to hack the kernel on an ongoing basis.

There are two ways to do this:

a) You could redirect to a packet socket - a small extension needed to the redirect action (mostly mechanical details involved like keeping state of which sockets are open etc).

b) My preference is to push this gentleman's PF_RING
(http://www.ntop.org/ntop.html) netdevice into the kernel. He has replicated unfortunately a lot of the stuff already done by MMAPED packet socket - but i think we can forgive him since solution a) would require hacking packet socket.

Reinjection of packets still needs working for that device - just as much as a few cleanups here and there. The problem is the guy is not very responsive - I have a lot of notes on his stuff if you are willing to chase him around.
You can then get redirection to this device for free (for either incoming or outgoing packets); something like:

tc filter add dev eth0 .... \
match ip src 10.0.0.1/32 \
action mirred egress redirect dev ring0

Assuming you have a program running on user space you should receive all packets incoming and/or outgoing on eth0.

And no, you dont need the eth device to have a ip address attached.

Quoted 5 lines · jamal

There are two ways to do this:

a) You could redirect to a packet socket - a small extension needed to the redirect action (mostly mechanical details involved like keeping state of which sockets are open etc).

I'd rather not take this approach, as I'd like to have this functionality available in a kernel module as well as user-space. Netdevices are easy to work with in both user-space and kernel-space.

Quoted 8 lines

tc filter add dev eth0 .... \
match ip src 10.0.0.1/32 \
action mirred egress redirect dev ring0

Assuming you have a program running on user space you should receive all packets incoming and/or outgoing on eth0.

And no, you dont need the eth device to have a ip address attached.

Just mirror-ing will not meet my goal.

I may also wish to drop packets
entirely, before they ever reach any of the protocol stacks.

That said, a brief glance at the ntop page leads me to believe that his packet socket might be interesting for other reasons. But, I have enough fun trying to push my own stuff into the kernel... probably won't bother trying to push his stuff in too :)

b) My preference is to push this gentleman's PF_RING
(http://www.ntop.org/ntop.html) netdevice into the kernel. He has replicated unfortunately a lot of the stuff already done by MMAPED packet socket - but i think we can forgive him since solution a) would require hacking packet socket.

Reinjection of packets still needs working for that device - just as much as a few cleanups here and there. The problem is the guy is not very responsive - I have a lot of notes on his stuff if you are willing to chase him around.
You can then get redirection to this device for free (for either incoming or outgoing packets); something like:

tc filter add dev eth0 .... \
match ip src 10.0.0.1/32 \
action mirred egress redirect dev ring0

I think we talked about this once already and I do like the idea but the reinjection is at least of the same importance to me. What I'm thinking of basically gets down to two ring buffers both holding mmaped buffers.

The action enqueues skbs to the first ring buffer
and userspace dequeues it from there. The skb gets completely lost at this point by having it shot or just cloned if mirroring is requested.

Userspace may reinject the skb again by putting it onto the second ring buffer for a kernel thread to pick up and reinject it at netif_receive_skb. Userspace may do whathever it likes as long as the checksum gets corrected, it may also fragment packets and reinject more than it originally received. Obviously for the redirect to userspace the skb must fullfil quite a lot of requirements only achievable on ingress but it would open up possibilities quite a lot.

Quoted 4 lines · Thomas Graf

I think we talked about this once already and I do like the idea but the reinjection is at least of the same importance to me. What I'm thinking of basically gets down to two ring buffers both holding mmaped buffers.

The ring device already has the recv side direction ring, Whats needed is tx side.

Quoted 3 lines

The action enqueues skbs to the first ring buffer
and userspace dequeues it from there. The skb gets completely lost at this point by having it shot or just cloned if mirroring is requested.

This will happen by default i.e the mirred action will take care of buffer management and hopefully the ring device will worry about reusing those skbs.

Quoted 7 lines

Userspace may reinject the skb again by putting it onto the second ring buffer for a kernel thread to pick up and reinject it at netif_receive_skb. Userspace may do whathever it likes as long as the checksum gets corrected, it may also fragment packets and reinject more than it originally received. Obviously for the redirect to userspace the skb must fullfil quite a lot of requirements only achievable on ingress but it would open up possibilities quite a lot.

One thing the PF_RING device needs is some form of metadata that can be passed to user space. PF_PACKET with MMAP has a few, but we need to do a lot more such as exposing most if not all of the skb metadata. Similarly, the direction from user space will contain details of where this packet is going to go (ingress vs egress) - PF_PACKET assumes egress only at the moment.