Skip to contentSkip to conversation

Network Event Notifier Mechanism

RFC PATCH In review Steve Wise 3 participants 13 messages Jun 23, 2006 – Jun 27, 2006

Subsystems (3)
lore
Share & tools
lore

Steve Wise

author
#0 lore raw

cover Network Event Notifier Mechanism

This patch implements a mechanism that allows interested clients to register for notification of certain network events. The intended use is to allow RDMA devices (linux/drivers/infiniband) to be notified of neighbour updates, ICMP redirects, path MTU changes, and route changes.

The reason these devices need update events is because they typically cache this information in hardware and need to be notified when this information has been updated.

The key events of interest are:

- neighbour mac address change
- routing redirect (the next hop neighbour changes for a dst_entry)
- path mtu change (the patch mtu for a dst_entry changes).
- route add/deletes

This approach is one of many possibilities and may be preferred because it uses an existing notification mechanism that has precedent in the stack. An alternative would be to add a netdev method to notify affect devices of these events.

This code does not yet implement path MTU change because the number of places in which this value is updated is large and if this mechanism seems reasonable, it would be probably be best to funnel these updates through a single function.

We would like to get this or similar functionality included in 2.6.19 and request comments.

This patchset consists of 2 patches:

1) New files implementing the Network Event Notifier
2) Core network changes to generate network event notifications

author
#1 lore raw

PATCH 1/2 Network Event Notifier Mechanism.

This patch uses notifier blocks to implement a network event notifier mechanism.

Clients register their callback function by calling register_netevent_notifier() like this:

static struct notifier_block nb = {
.notifier_call = my_callback_func
};

...

register_netevent_notifier(&nb);
---

 include/net/netevent.h |   41 +++++++++++++++++++++++++++++
 net/core/netevent.c    |   67 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 108 insertions(+), 0 deletions(-)
include/net/netevent.h new file +41 -0
new file mode 100644
index 0000000..9ceab27
@@ -0,0 +1,41 @@
+#ifndef _NET_EVENT_H
+#define _NET_EVENT_H
+
+/*
+ *	Generic netevent notifiers
+ *
+ *	Authors:
+ *      Tom Tucker              <[email protected]>
+ *
+ * 	Changes:
+ */
+
+#ifdef __KERNEL__
+
+#include <net/dst.h>
+
+struct netevent_redirect {
+	struct dst_entry *old;
+	struct dst_entry *new;
+};
+
+struct netevent_route_change {
+        int event;
+        struct fib_info *fib_info;
+};
+
+enum netevent_notif_type {
+	NETEVENT_NEIGH_UPDATE = 1, /* arg is * struct neighbour */
+	NETEVENT_ROUTE_UPDATE,	   /* arg is * netevent_route_change */
+	NETEVENT_PMTU_UPDATE,
+	NETEVENT_REDIRECT,	   /* arg is * struct netevent_redirect */
+};
+
+extern int register_netevent_notifier(struct notifier_block *nb);
+extern int unregister_netevent_notifier(struct notifier_block *nb);
+extern int call_netevent_notifiers(unsigned long val, void *v);
+
+#endif
+#endif
+
+
net/core/netevent.c new file +67 -0

67 lines not on this page · raw message

maintainer
#2 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:19:28 -0500

include/net/netevent.h line 25

Quoted 4 lines
include/net/netevent.h new file +4 -0
+struct netevent_route_change {
+        int event;
+        struct fib_info *fib_info;
+};

It's not generic if you're putting ipv4 FIB route objects in the datastructure.

This is so much unnecessary syntactic sugar for what you're trying to accomplish.

Just pass a "void *", and have the type of the object defined by which notifier it is. This is exactly how all notifiers work today.

Then you don't need any new infrastructure at all. The existing notifier bits handle this kind of scheme already, no need to invent new stuff.

author
#3 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

Quoted 14 lines · David Miller

From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:19:28 -0500

include/net/netevent.h line 25

Quoted 4 lines
include/net/netevent.h new file +4 -0
+struct netevent_route_change {
+        int event;
+        struct fib_info *fib_info;
+};

It's not generic if you're putting ipv4 FIB route objects in the datastructure.

This is so much unnecessary syntactic sugar for what you're trying to accomplish.

Quoted 6 lines

Just pass a "void *", and have the type of the object defined by which notifier it is. This is exactly how all notifiers work today.

Then you don't need any new infrastructure at all. The existing notifier bits handle this kind of scheme already, no need to invent new stuff.

Ok.

But what about the redirect event? It really needs to pass two bits of
info: The old dst_entry ptr and the new dst_entry ptr. This is needed so the rdma driver can figure out which connections now need to use the new dst_entry...

maintainer
#4 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:34:55 -0500

Quoted 4 lines

But what about the redirect event? It really needs to pass two bits of
info: The old dst_entry ptr and the new dst_entry ptr. This is needed so the rdma driver can figure out which connections now need to use the new dst_entry...

struct foo_event_data {
	struct whatever a;
	struct thistoo b;
};
	struct foo_event_data x;

	x->a = xxx;
	x->b = yyy;

	event->data = (void *) &x;
	send_event(&event);

Why do you need typed data to do this?

author
#5 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

----- Original Message -----
From: "David Miller" <[email protected]>
To: <[email protected]>
Cc: <[email protected]>
Sent: Friday, June 23, 2006 3:56 PM
Subject: Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

| From: Steve Wise <[email protected]>
| Date: Fri, 23 Jun 2006 15:34:55 -0500
|
| > But what about the redirect event?  It really needs to pass two bits 

of
| > info: The old dst_entry ptr and the new dst_entry ptr. This is needed
| > so the rdma driver can figure out which connections now need to use the

| > new dst_entry...
|
| struct foo_event_data {
| struct whatever a;
| struct thistoo b;
| };
|
| struct foo_event_data x;
|
| x->a = xxx;
| x->b = yyy;
|
| event->data = (void *) &x;
| send_event(&event);
|
| Why do you need typed data to do this?
|

You don't. This is fine with me...

author
#6 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

In reply to David Miller
Quoted 11 lines · David Miller

From: Steve Wise <[email protected]>
Date: Fri, 23 Jun 2006 15:19:28 -0500

include/net/netevent.h line 25

Quoted 4 lines
include/net/netevent.h new file +4 -0
+struct netevent_route_change {
+        int event;
+        struct fib_info *fib_info;
+};

It's not generic if you're putting ipv4 FIB route objects in the datastructure.

True.

I guess what I think we should do is pass the fib_info * when its a IPv4 route add/del, and a rt6_info * when its a IPv6 add/del. This avoids having to create some new family independent struct. What I'll have to do, however, is have specific notifier event enums for each:

NETEVENT_IPV4_ROUTE_ADD
NETEVENT_IPV4_ROUTE_DEL
NETEVENT_IPV6_ROUTE_ADD
NETEVENT_IPV6_ROUTE_DEL

This keeps it simple IMO...

Does that sound reasonable to you?

Steve.

maintainer
#7 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

From: Steve Wise <[email protected]>
Date: Mon, 26 Jun 2006 10:26:11 -0500

Quoted 5 lines

I guess what I think we should do is pass the fib_info * when its a IPv4 route add/del, and a rt6_info * when its a IPv6 add/del. This avoids having to create some new family independent struct. What I'll have to do, however, is have specific notifier event enums for each:

Quoted 8 lines

NETEVENT_IPV4_ROUTE_ADD
NETEVENT_IPV4_ROUTE_DEL
NETEVENT_IPV6_ROUTE_ADD
NETEVENT_IPV6_ROUTE_DEL

This keeps it simple IMO...

Does that sound reasonable to you?

You can avoid creating an event per address family by defining your data as:

	struct netevent_route_info {
		u16 family;
		void *data;
	};

and passing that through the notifier.

Then you just need NETEVENT_ROUTE_{ADD,DEL}

Just an idea...

author
#8 lore raw

Re: [PATCH REPOST 1/2] Network Event Notifier Mechanism.

Quoted 25 lines · David Miller

From: Steve Wise <[email protected]>
Date: Mon, 26 Jun 2006 10:26:11 -0500

Quoted 5 lines

I guess what I think we should do is pass the fib_info * when its a IPv4 route add/del, and a rt6_info * when its a IPv6 add/del. This avoids having to create some new family independent struct. What I'll have to do, however, is have specific notifier event enums for each:

Quoted 8 lines

NETEVENT_IPV4_ROUTE_ADD
NETEVENT_IPV4_ROUTE_DEL
NETEVENT_IPV6_ROUTE_ADD
NETEVENT_IPV6_ROUTE_DEL

This keeps it simple IMO...

Does that sound reasonable to you?

You can avoid creating an event per address family by defining your data as:

	struct netevent_route_info {
		u16 family;
		void *data;
	};
Quoted 5 lines

and passing that through the notifier.

Then you just need NETEVENT_ROUTE_{ADD,DEL}

Just an idea...

Seems reasonable.

Thanks,

Steve.

author
#9 lore raw

PATCH 2/2 Core network changes to support network event notification.

In reply to Steve Wise

This patch adds event calls for neighbour change, route update, and routing redirect events.

TODO: PMTU change events.
---

 net/core/Makefile        |    2 +-
 net/core/neighbour.c     |    8 ++++++++
 net/ipv4/fib_semantics.c |    7 +++++++
 net/ipv4/route.c         |    6 ++++++
 4 files changed, 22 insertions(+), 1 deletions(-)
net/core/Makefile +1 -1
index e9bd246..2645ba4 100644
@@ -7,7 +7,7 @@ obj-y := sock.o request_sock.o skbuff.o 
 
 obj-$(CONFIG_SYSCTL) += sysctl_net_core.o
 
-obj-y		     += dev.o ethtool.o dev_mcast.o dst.o \
+obj-y		     += dev.o ethtool.o dev_mcast.o dst.o netevent.o \
 			neighbour.o rtnetlink.o utils.o link_watch.o filter.o
 
 obj-$(CONFIG_XFRM) += flow.o
net/core/neighbour.c +8 -0
index 50a8c73..c637897 100644
@@ -30,9 +30,11 @@ #include <linux/times.h>
 #include <net/neighbour.h>
 #include <net/dst.h>
 #include <net/sock.h>
+#include <net/netevent.h>
 #include <linux/rtnetlink.h>
 #include <linux/random.h>
 #include <linux/string.h>
+#include <linux/notifier.h>
 
 #define NEIGH_DEBUG 1
 
@@ -755,6 +757,7 @@ #endif
 			neigh->nud_state = NUD_STALE;
 			neigh->updated = jiffies;
 			neigh_suspect(neigh);
+			call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
 		}
 	} else if (state & NUD_DELAY) {
 		if (time_before_eq(now, 
@@ -763,6 +766,7 @@ #endif
 			neigh->nud_state = NUD_REACHABLE;
 			neigh->updated = jiffies;
 			neigh_connect(neigh);
+			call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
 			next = neigh->confirmed + neigh->parms->reachable_time;
 		} else {
 			NEIGH_PRINTK2("neigh %p is probed.\n", neigh);
@@ -783,6 +787,7 @@ #endif
 		neigh->nud_state = NUD_FAILED;
 		neigh->updated = jiffies;
 		notify = 1;
+		call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
 		NEIGH_CACHE_STAT_INC(neigh->tbl, res_failed);
 		NEIGH_PRINTK2("neigh %p is failed.\n", neigh);
 
@@ -1056,6 +1061,9 @@ out:
 			(neigh->flags | NTF_ROUTER) :
 			(neigh->flags & ~NTF_ROUTER);
 	}
+
+	call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
+
 	write_unlock_bh(&neigh->lock);
 #ifdef CONFIG_ARPD
 	if (notify && neigh->parms->app_probes)
net/ipv4/fib_semantics.c +7 -0
index 0f4145b..67a30af 100644
@@ -45,6 +45,7 @@ #include <net/tcp.h>
 #include <net/sock.h>
 #include <net/ip_fib.h>
 #include <net/ip_mp_alg.h>
+#include <net/netevent.h>
 
 #include "fib_lookup.h"
 
@@ -278,9 +279,15 @@ void rtmsg_fib(int event, u32 key, struc
 	       struct nlmsghdr *n, struct netlink_skb_parms *req)
 {
 	struct sk_buff *skb;
+	struct netevent_route_change rev;
+
 	u32 pid = req ? req->pid : n->nlmsg_pid;
 	int size = NLMSG_SPACE(sizeof(struct rtmsg)+256);
 
+	rev.event = event;
+	rev.fib_info = fa->fa_info;
+	call_netevent_notifiers(NETEVENT_ROUTE_UPDATE, &rev);
+
 	skb = alloc_skb(size, GFP_KERNEL);
 	if (!skb)
 		return;
net/ipv4/route.c +6 -0
index cc9423d..e9ba831 100644
@@ -105,6 +105,7 @@ #include <net/tcp.h>
 #include <net/icmp.h>
 #include <net/xfrm.h>
 #include <net/ip_mp_alg.h>
+#include <net/netevent.h>
 #ifdef CONFIG_SYSCTL
 #include <linux/sysctl.h>
 #endif
@@ -1120,6 +1121,7 @@ void ip_rt_redirect(u32 old_gw, u32 dadd
 	struct rtable *rth, **rthp;
 	u32  skeys[2] = { saddr, 0 };
 	int  ikeys[2] = { dev->ifindex, 0 };
+	struct netevent_redirect netevent;
 
 	if (!in_dev)
 		return;
@@ -1211,6 +1213,10 @@ void ip_rt_redirect(u32 old_gw, u32 dadd
 					rt_drop(rt);
 					goto do_next;
 				}
+				
+				netevent.old = &rth->u.dst;
+				netevent.new = &rt->u.dst;
+				call_netevent_notifiers(NETEVENT_REDIRECT, &netevent);
 
 				rt_del(hash, rth);
 				if (!rt_intern_hash(hash, rt, &rt))
#10 lore raw
In reply to Steve Wise
Quoted 10 lines · Steve Wise

This patch implements a mechanism that allows interested clients to register for notification of certain network events. The intended use is to allow RDMA devices (linux/drivers/infiniband) to be notified of neighbour updates, ICMP redirects, path MTU changes, and route changes.

The reason these devices need update events is because they typically cache this information in hardware and need to be notified when this information has been updated.

Quoted 6 lines

The key events of interest are:

- neighbour mac address change
- routing redirect (the next hop neighbour changes for a dst_entry)
- path mtu change (the patch mtu for a dst_entry changes).
- route add/deletes

I'd like to know more about what the RDMA device is going to do with this information. I thought RDMA was for receiving packets? Most of the info here pertains to transmission.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author
#11 lore raw
Quoted 5 lines

I'd like to know more about what the RDMA device is going to do with this information. I thought RDMA was for receiving packets? Most of the info here pertains to transmission.

RDMA Ethernet devices adhere to a set of protocols defined by the IETF. See the RDDP WG (http://www.ietf.org/html.charters/rddp-charter.html)
for the Internet Drafts that define the protocols.

Steve.

#12 lore raw
Quoted 8 lines · Steve Wise
Quoted 4 lines

I'd like to know more about what the RDMA device is going to do with this information. I thought RDMA was for receiving packets? Most of the info here pertains to transmission.

RDMA Ethernet devices adhere to a set of protocols defined by the IETF. See the RDDP WG (http://www.ietf.org/html.charters/rddp-charter.html)
for the Internet Drafts that define the protocols.

Would it be possible for you to give us a quick summary of the relevant points?

Thanks,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt