Network Event Notifier Mechanism
Subsystems (3)
cover Network Event Notifier Mechanism
This patch implements a mechanism that allows interested clients to register for notification of certain network events. The intended use is to allow RDMA devices (linux/drivers/infiniband) to be notified of neighbour updates, ICMP redirects, path MTU changes, and route changes.
The reason these devices need update events is because they typically cache this information in hardware and need to be notified when this information has been updated.
The key events of interest are:
- neighbour mac address change
- routing redirect (the next hop neighbour changes for a dst_entry)
- path mtu change (the patch mtu for a dst_entry changes).
- route add/deletes
This approach is one of many possibilities and may be preferred because it uses an existing notification mechanism that has precedent in the stack. An alternative would be to add a netdev method to notify affect devices of these events.
This code does not yet implement path MTU change because the number of places in which this value is updated is large and if this mechanism seems reasonable, it would be probably be best to funnel these updates through a single function.
We would like to get this or similar functionality included in 2.6.19 and request comments.
This patchset consists of 2 patches:
1) New files implementing the Network Event Notifier
2) Core network changes to generate network event notifications
PATCH 1/2 Network Event Notifier Mechanism.
This patch uses notifier blocks to implement a network event notifier mechanism.
Clients register their callback function by calling register_netevent_notifier() like this:
static struct notifier_block nb = {
.notifier_call = my_callback_func
};
...
register_netevent_notifier(&nb);
---
include/net/netevent.h | 41 +++++++++++++++++++++++++++++ net/core/netevent.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+), 0 deletions(-)
include/net/netevent.h new file +41 -0
@@ -0,0 +1,41 @@ +#ifndef _NET_EVENT_H +#define _NET_EVENT_H + +/* +* Generic netevent notifiers +* +* Authors: +* Tom Tucker <[email protected]> +* +* Changes: +*/ + +#ifdef __KERNEL__ + +#include <net/dst.h> + +struct netevent_redirect { +struct dst_entry *old; +struct dst_entry *new; +}; + +struct netevent_route_change { +int event; +struct fib_info *fib_info; +}; + +enum netevent_notif_type { +NETEVENT_NEIGH_UPDATE =1 ,/* arg is * struct neighbour */ +NETEVENT_ROUTE_UPDATE ,/* arg is * netevent_route_change */ +NETEVENT_PMTU_UPDATE , +NETEVENT_REDIRECT ,/* arg is * struct netevent_redirect */ +}; + +extern int register_netevent_notifier (struct notifier_block *nb); +extern int unregister_netevent_notifier (struct notifier_block *nb); +extern int call_netevent_notifiers (unsigned long val,void *v); + +#endif +#endif + +
net/core/netevent.c new file +67 -0
67 lines not on this page · raw message
David Miller
maintainerRe: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:19:28 -0500
include/net/netevent.h line 25
Quoted 4 lines
include/net/netevent.h new file +4
-0+struct netevent_route_change { +int event; +struct fib_info *fib_info; +};
It's not generic if you're putting ipv4 FIB route objects in the datastructure.
This is so much unnecessary syntactic sugar for what you're trying to accomplish.
Just pass a "void *", and have the type of the object defined by which notifier it is. This is exactly how all notifiers work today.
Then you don't need any new infrastructure at all. The existing notifier bits handle this kind of scheme already, no need to invent new stuff.
Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
Quoted 14 lines · David Miller
From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:19:28 -0500include/net/netevent.h line 25
Quoted 4 lines
include/net/netevent.h new file +4
-0+struct netevent_route_change { +int event; +struct fib_info *fib_info; +};It's not generic if you're putting ipv4 FIB route objects in the datastructure.
This is so much unnecessary syntactic sugar for what you're trying to accomplish.
Quoted 6 lines
Just pass a "void *", and have the type of the object defined by which notifier it is. This is exactly how all notifiers work today.
Then you don't need any new infrastructure at all. The existing notifier bits handle this kind of scheme already, no need to invent new stuff.
Ok.
But what about the redirect event? It really needs to pass two bits of
info: The old dst_entry ptr and the new dst_entry ptr. This is needed so the rdma driver can figure out which connections now need to use the new dst_entry...
David Miller
maintainerRe: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:34:55 -0500
Quoted 4 lines
But what about the redirect event? It really needs to pass two bits of
info: The olddst_entryptr and the newdst_entryptr. This is needed so the rdma driver can figure out which connections now need to use the newdst_entry...
struct foo_event_data {
struct whatever a;
struct thistoo b;
};struct foo_event_data x; x->a = xxx; x->b = yyy; event->data = (void *) &x; send_event(&event);
Why do you need typed data to do this?
Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
----- Original Message -----
From: "David Miller" <[email protected]>
To: <[email protected]>
Cc: <[email protected]>
Sent: Friday, June 23, 2006 3:56 PM
Subject: Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
| From: Steve Wise <[email protected]> | Date: Fri, 23 Jun 2006 15:34:55 -0500 | | > But what about the redirect event? It really needs to pass two bits
of
| > info: The old dst_entry ptr and the new dst_entry ptr. This is needed
| > so the rdma driver can figure out which connections now need to use the
| > new dst_entry...
|
| struct foo_event_data {
| struct whatever a;
| struct thistoo b;
| };
|
| struct foo_event_data x;
|
| x->a = xxx;
| x->b = yyy;
|
| event->data = (void *) &x;
| send_event(&event);
|
| Why do you need typed data to do this?
|You don't. This is fine with me...
Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
In reply to David MillerQuoted 11 lines · David Miller
From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:19:28 -0500include/net/netevent.h line 25
Quoted 4 lines
include/net/netevent.h new file +4
-0+struct netevent_route_change { +int event; +struct fib_info *fib_info; +};It's not generic if you're putting ipv4 FIB route objects in the datastructure.
True.
I guess what I think we should do is pass the fib_info * when its a IPv4 route add/del, and a rt6_info * when its a IPv6 add/del. This avoids having to create some new family independent struct. What I'll have to do, however, is have specific notifier event enums for each:
NETEVENT_IPV4_ROUTE_ADDNETEVENT_IPV4_ROUTE_DELNETEVENT_IPV6_ROUTE_ADDNETEVENT_IPV6_ROUTE_DEL
This keeps it simple IMO...
Does that sound reasonable to you?
Steve.
David Miller
maintainerRe: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
From: Steve Wise <[email protected]>
Date: Mon, 26 Jun 2006 10:26:11 -0500
Quoted 5 lines
I guess what I think we should do is pass the
fib_info* when its a IPv4 route add/del, and art6_info* when its a IPv6 add/del. This avoids having to create some new family independent struct. What I'll have to do, however, is have specific notifier event enums for each:
Quoted 8 lines
NETEVENT_IPV4_ROUTE_ADDNETEVENT_IPV4_ROUTE_DELNETEVENT_IPV6_ROUTE_ADDNETEVENT_IPV6_ROUTE_DELThis keeps it simple IMO...
Does that sound reasonable to you?
You can avoid creating an event per address family by defining your data as:
struct netevent_route_info {
u16 family;
void *data;
};and passing that through the notifier.
Then you just need NETEVENT_ROUTE_{ADD,DEL}
Just an idea...
Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.
Quoted 25 lines · David Miller
From: Steve Wise <[email protected]>
Date: Mon, 26 Jun 2006 10:26:11 -0500Quoted 5 lines
I guess what I think we should do is pass the
fib_info* when its a IPv4 route add/del, and art6_info* when its a IPv6 add/del. This avoids having to create some new family independent struct. What I'll have to do, however, is have specific notifier event enums for each:Quoted 8 lines
NETEVENT_IPV4_ROUTE_ADDNETEVENT_IPV4_ROUTE_DELNETEVENT_IPV6_ROUTE_ADDNETEVENT_IPV6_ROUTE_DELThis keeps it simple IMO...
Does that sound reasonable to you?
You can avoid creating an event per address family by defining your data as:
struct netevent_route_info { u16 family; void *data; };
Quoted 5 lines
and passing that through the notifier.
Then you just need NETEVENT_ROUTE_{ADD,DEL}
Just an idea...
Seems reasonable.
Thanks,
Steve.
PATCH 2/2 Core network changes to support network event notification.
In reply to Steve WiseThis patch adds event calls for neighbour change, route update, and routing redirect events.
TODO: PMTU change events.
---
net/core/Makefile | 2 +- net/core/neighbour.c | 8 ++++++++ net/ipv4/fib_semantics.c | 7 +++++++ net/ipv4/route.c | 6 ++++++ 4 files changed, 22 insertions(+), 1 deletions(-)
net/core/Makefile +1 -1
@@ -7,7 +7,7 @@ obj-y := sock.o request_sock.o skbuff.o obj-$(CONFIG_SYSCTL) += sysctl_net_core.o -obj-y += dev.o ethtool.o dev_mcast.o dst.o \ +obj-y += dev.o ethtool.o dev_mcast.o dst.o netevent.o \ neighbour.o rtnetlink.o utils.o link_watch.o filter.o obj-$(CONFIG_XFRM) += flow.o
net/core/neighbour.c +8 -0
@@ -30,9 +30,11 @@ #include <linux/times.h>#include <net/neighbour.h> #include <net/dst.h> #include <net/sock.h> +#include <net/netevent.h> #include <linux/rtnetlink.h> #include <linux/random.h> #include <linux/string.h> +#include <linux/notifier.h> #define NEIGH_DEBUG 1
@@ -755,6 +757,7 @@ #endif neigh->nud_state =NUD_STALE ; neigh->updated = jiffies;neigh_suspect (neigh); +call_netevent_notifiers (NETEVENT_NEIGH_UPDATE , neigh); } }else if (state &NUD_DELAY ) { if (time_before_eq(now,
@@ -763,6 +766,7 @@ #endif neigh->nud_state =NUD_REACHABLE ; neigh->updated = jiffies;neigh_connect (neigh); +call_netevent_notifiers (NETEVENT_NEIGH_UPDATE , neigh); next = neigh->confirmed + neigh->parms->reachable_time; }else {NEIGH_PRINTK2 ("neigh %p is probed.\n" , neigh);
@@ -783,6 +787,7 @@ #endif neigh->nud_state =NUD_FAILED ; neigh->updated = jiffies; notify =1 ; +call_netevent_notifiers (NETEVENT_NEIGH_UPDATE , neigh);NEIGH_CACHE_STAT_INC (neigh->tbl, res_failed);NEIGH_PRINTK2 ("neigh %p is failed.\n" , neigh);
@@ -1056,6 +1061,9 @@ out: (neigh->flags | NTF_ROUTER) : (neigh->flags & ~NTF_ROUTER ); } + +call_netevent_notifiers (NETEVENT_NEIGH_UPDATE , neigh); +write_unlock_bh (&neigh->lock);#ifdef CONFIG_ARPD if (notify && neigh->parms->app_probes)
net/ipv4/fib_semantics.c +7 -0
@@ -45,6 +45,7 @@ #include <net/tcp.h>#include <net/sock.h> #include <net/ip_fib.h> #include <net/ip_mp_alg.h> +#include <net/netevent.h> #include "fib_lookup.h"
@@ -278,9 +279,15 @@ void rtmsg_fib(int event, u32 key, struc struct nlmsghdr *n, struct netlink_skb_parms *req) { struct sk_buff *skb; +struct netevent_route_change rev; +u32 pid = req ? req->pid : n->nlmsg_pid;int size =NLMSG_SPACE (sizeof (struct rtmsg )+256 ); + rev.event = event; + rev.fib_info = fa->fa_info; +call_netevent_notifiers (NETEVENT_ROUTE_UPDATE , &rev); + skb =alloc_skb (size,GFP_KERNEL );if (!skb)return ;
net/ipv4/route.c +6 -0
@@ -105,6 +105,7 @@ #include <net/tcp.h>#include <net/icmp.h> #include <net/xfrm.h> #include <net/ip_mp_alg.h> +#include <net/netevent.h> #ifdef CONFIG_SYSCTL #include <linux/sysctl.h> #endif
@@ -1120,6 +1121,7 @@ void ip_rt_redirect(u32 old_gw, u32 daddstruct rtable *rth, **rthp;u32 skeys[2 ] = { saddr,0 };int ikeys[2 ] = { dev->ifindex,0 }; +struct netevent_redirect netevent;if (!in_dev)return ;
@@ -1211,6 +1213,10 @@ void ip_rt_redirect(u32 old_gw, u32 daddrt_drop (rt); goto do_next; } + + netevent.old = &rth->u.dst; + netevent.new = &rt->u.dst; +call_netevent_notifiers (NETEVENT_REDIRECT , &netevent);rt_del (hash, rth);if (!rt_intern_hash (hash, rt, &rt))
Quoted 10 lines · Steve Wise
This patch implements a mechanism that allows interested clients to register for notification of certain network events. The intended use is to allow RDMA devices (linux/drivers/infiniband) to be notified of neighbour updates, ICMP redirects, path MTU changes, and route changes.
The reason these devices need update events is because they typically cache this information in hardware and need to be notified when this information has been updated.
Quoted 6 lines
The key events of interest are:
- neighbour mac address change
- routing redirect (the next hop neighbour changes for adst_entry)
- path mtu change (the patch mtu for adst_entrychanges).
- route add/deletes
I'd like to know more about what the RDMA device is going to do with this information. I thought RDMA was for receiving packets? Most of the info here pertains to transmission.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtQuoted 5 lines
I'd like to know more about what the RDMA device is going to do with this information. I thought RDMA was for receiving packets? Most of the info here pertains to transmission.
RDMA Ethernet devices adhere to a set of protocols defined by the IETF. See the RDDP WG (http://www.ietf.org/html.charters/rddp-charter.html)
for the Internet Drafts that define the protocols.
Steve.
Quoted 8 lines · Steve Wise
Quoted 4 lines
I'd like to know more about what the RDMA device is going to do with this information. I thought RDMA was for receiving packets? Most of the info here pertains to transmission.
RDMA Ethernet devices adhere to a set of protocols defined by the IETF. See the RDDP WG (http://www.ietf.org/html.charters/rddp-charter.html)
for the Internet Drafts that define the protocols.
Would it be possible for you to give us a quick summary of the relevant points?
Thanks,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt