Add NTF_EXT_AGED to control FDB ageing in SW or HW
Subsystems (5)
cover Add NTF_EXT_AGED to control FDB ageing in SW or HW
From: Scott Feldman <[email protected]>
Add a new NTF_EXT_FLAG to mark an FDB as externally aged, for example by
offload hardware. Switchdev driver/devices can set this flag when learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its ageing task. If flag is set, the driver/device is responsible for calling call_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...) when entry expires.
This give the flexibility for driver/device to decide ageing policy based on its capabilities. For devices managing many FDB entries, it is desireable for the device to aged out its own entries. Devices not capable of aged entries can rely of SW to age out the entries.
Scott Feldman (5):
neighbour: add external aged flag
switchdev: add ntf_flags to FDB notifier
bridge: call external learn add if adding FDB entry with
NTF_EXT_LEARNED set
bridge: let HW control FDB ageing by setting NTF_EXT_AGED
rocker: explicitly set SW ageing for rocker
drivers/net/ethernet/rocker/rocker.c | 9 +++++---- include/net/switchdev.h | 1 + include/uapi/linux/neighbour.h | 1 + net/bridge/br.c | 18 ++++++++++-------- net/bridge/br_fdb.c | 16 +++++++++++++--- net/bridge/br_private.h | 8 +++++--- 6 files changed, 35 insertions(+), 18 deletions(-)
Signature · 1 line
1.7.10.4
RFC PATCH 1/5 neighbour: add external aged flag
From: Scott Feldman <[email protected]>
NFT_EXT_AGED marks neighbor (FDB) entries as externally aged, typically in hardware. Software can skip these entries when aging entries.
--- include/uapi/linux/neighbour.h | 1 + 1 file changed, 1 insertion(+)
include/uapi/linux/neighbour.h +1 -0
@@ -40,6 +40,7 @@ enum {#define NTF_MASTER 0x04#define NTF_PROXY 0x08/* == ATF_PUBL */ #define NTF_EXT_LEARNED 0x10 +#define NTF_EXT_AGED 0x20#define NTF_ROUTER 0x80/*
Signature · 1 line
1.7.10.4
RFC PATCH 2/5 switchdev: add ntf_flags to FDB notifier
In reply to [email protected]From: Scott Feldman <[email protected]>
Let switchdev driver set NTF_* flags per FDB entry. Driver can set NTF_EXT_AGED to indicate driver/device will aged out entry. Driver can set NTF_EXT_LEARNED to indicate driver/device learned this entry.
--- include/net/switchdev.h | 1 + 1 file changed, 1 insertion(+)
include/net/switchdev.h +1 -0
@@ -26,6 +26,7 @@ struct netdev_switch_notifier_fdb_info {struct netdev_switch_notifier_info info;/* must be first */ const unsigned char *addr;u16 vid; +u8 ntf_flags; }; static inline struct net_device *
Signature · 1 line
1.7.10.4
RFC PATCH 3/5 bridge: call external learn add if adding FDB entry with NTF_EXT_LEARNED set
In reply to [email protected]From: Scott Feldman <[email protected]>
Only call into bridge driver to add externally learned FDB entry if driver/ device indicates entry with NTF_EXT_LEARNED.
--- net/bridge/br.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
net/bridge/br.c +10 -8
@@ -140,17 +140,19 @@ static int br_netdev_switch_event(struct notifier_block *unused,switch (event) {case NETDEV_SWITCH_FDB_ADD : fdb_info = ptr; - err =br_fdb_external_learn_add (br, p, fdb_info->addr, - fdb_info->vid); -if (err) - err =notifier_from_errno (err); +if (fdb_info->ntf_flags &NTF_EXT_LEARNED ) { + err =br_fdb_external_learn_add (br, p, fdb_info); +if (err) + err =notifier_from_errno (err); + }break ;case NETDEV_SWITCH_FDB_DEL : fdb_info = ptr; - err =br_fdb_external_learn_del (br, p, fdb_info->addr, - fdb_info->vid); -if (err) - err =notifier_from_errno (err); +if (fdb_info->ntf_flags &NTF_EXT_LEARNED ) { + err =br_fdb_external_learn_del (br, p, fdb_info); +if (err) + err =notifier_from_errno (err); + }break ; }
Signature · 1 line
1.7.10.4
RFC PATCH 4/5 bridge: let HW control FDB ageing by setting NTF_EXT_AGED
In reply to [email protected]From: Scott Feldman <[email protected]>
During bridge's ageing function, exclude FDB entries marked with NTF_EXT_AGED. The external driver/device which set NTF_EXT_AGED flag will age out these entries.
--- net/bridge/br_fdb.c | 16 +++++++++++++--- net/bridge/br_private.h | 8 +++++--- 2 files changed, 18 insertions(+), 6 deletions(-)
net/bridge/br_fdb.c +13 -3
@@ -24,6 +24,7 @@#include <linux/atomic.h> #include <asm/unaligned.h> #include <linux/if_vlan.h> +#include <net/switchdev.h> #include "br_private.h" static struct kmem_cache *br_fdb_cache__read_mostly ;
@@ -280,7 +281,7 @@ void br_fdb_cleanup(unsigned long _data)hlist_for_each_entry_safe (f, n, &br->hash[i], hlist) {unsigned long this_timer; -if (f->is_static) +if (f->is_static || f->is_external_aged)continue ; this_timer = f->updated + delay;if (time_before_eq (this_timer, jiffies))
@@ -482,6 +483,7 @@ static struct net_bridge_fdb_entry *fdb_create(struct hlist_head *head, fdb->is_static =0 ; fdb->added_by_user =0 ; fdb->added_by_external_learn =0 ; + fdb->is_external_aged =0 ; fdb->updated = fdb->used = jiffies;hlist_add_head_rcu (&fdb->hlist, head); }
@@ -615,6 +617,7 @@ static int fdb_fill_info(struct sk_buff *skb, const struct net_bridge *br, ndm->ndm_pad1 =0 ; ndm->ndm_pad2 =0 ; ndm->ndm_flags = fdb->added_by_external_learn ?NTF_EXT_LEARNED :0 ; + ndm->ndm_flags |= fdb->is_external_aged ?NTF_EXT_AGED :0 ; ndm->ndm_type =0 ; ndm->ndm_ifindex = fdb->dst ? fdb->dst->dev->ifindex : br->dev->ifindex; ndm->ndm_state =fdb_to_nud (fdb);
@@ -990,10 +993,13 @@ void br_fdb_unsync_static(struct net_bridge *br, struct net_bridge_port *p) }int br_fdb_external_learn_add (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid) +struct netdev_switch_notifier_fdb_info *fdb_info) {struct hlist_head *head;struct net_bridge_fdb_entry *fdb; +const unsigned char *addr = fdb_info->addr; +u16 vid = fdb_info->vid; +bool is_external_aged = !!(fdb_info->ntf_flags &NTF_EXT_AGED );int err =0 ;ASSERT_RTNL ();
@@ -1008,6 +1014,7 @@ int br_fdb_external_learn_add(struct net_bridge *br, struct net_bridge_port *p, goto err_unlock; } fdb->added_by_external_learn =1 ; + fdb->is_external_aged = is_external_aged;fdb_notify (br, fdb,RTM_NEWNEIGH ); }else if (fdb->added_by_external_learn) {/* Refresh entry */
@@ -1015,6 +1022,7 @@ int br_fdb_external_learn_add(struct net_bridge *br, struct net_bridge_port *p, }else if (!fdb->added_by_user) {/* Take over SW learned entry */ fdb->added_by_external_learn =1 ; + fdb->is_external_aged = is_external_aged; fdb->updated = jiffies;fdb_notify (br, fdb,RTM_NEWNEIGH ); }
@@ -1026,10 +1034,12 @@ err_unlock: }int br_fdb_external_learn_del (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid) +struct netdev_switch_notifier_fdb_info *fdb_info) {struct hlist_head *head;struct net_bridge_fdb_entry *fdb; +const unsigned char *addr = fdb_info->addr; +u16 vid = fdb_info->vid;int err =0 ;ASSERT_RTNL ();
net/bridge/br_private.h +5 -3
@@ -18,6 +18,7 @@#include <linux/netpoll.h> #include <linux/u64_stats_sync.h> #include <net/route.h> +#include <net/switchdev.h> #include <linux/if_vlan.h> #define BR_HASH_BITS 8
@@ -101,7 +102,8 @@ struct net_bridge_fdb_entry unsigned char is_local:1, is_static:1, added_by_user:1, - added_by_external_learn:1; + added_by_external_learn:1, + is_external_aged:1;__u16 vlan_id; };
@@ -403,9 +405,9 @@ int br_fdb_dump(struct sk_buff *skb, struct netlink_callback *cb,int br_fdb_sync_static (struct net_bridge *br,struct net_bridge_port *p);void br_fdb_unsync_static (struct net_bridge *br,struct net_bridge_port *p);int br_fdb_external_learn_add (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid); +struct netdev_switch_notifier_fdb_info *fdb_info);int br_fdb_external_learn_del (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid); +struct netdev_switch_notifier_fdb_info *fdb_info);/* br_forward.c */ void br_deliver (const struct net_bridge_port *to,struct sk_buff *skb);
Signature · 1 line
1.7.10.4
David Miller
maintainerRe: [PATCH net-next RFC 4/5] bridge: let HW control FDB ageing by setting NTF_EXT_AGED
From: [email protected]
Date: Thu, 19 Feb 2015 23:09:54 -0800
Quoted 7 lines
net/bridge/br_fdb.c +1
-1@@ -990,10 +993,13 @@ void br_fdb_unsync_static(struct net_bridge *br, struct net_bridge_port *p) }int br_fdb_external_learn_add (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid) +struct netdev_switch_notifier_fdb_info *fdb_info) {
...
Quoted 7 lines
net/bridge/br_fdb.c +1
-1@@ -1026,10 +1034,12 @@ err_unlock: }int br_fdb_external_learn_del (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid) +struct netdev_switch_notifier_fdb_info *fdb_info) {
...
net/bridge/br_private.h int br_fdb_dump(struct sk_buff *skb, struct netlink_callback *cb, · line 410
Quoted 9 lines
net/bridge/br_private.h +2
-2@@ -403,9 +405,9 @@ int br_fdb_dump(struct sk_buff *skb, struct netlink_callback *cb,int br_fdb_sync_static (struct net_bridge *br,struct net_bridge_port *p);void br_fdb_unsync_static (struct net_bridge *br,struct net_bridge_port *p);int br_fdb_external_learn_add (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid); +struct netdev_switch_notifier_fdb_info *fdb_info);int br_fdb_external_learn_del (struct net_bridge *br,struct net_bridge_port *p, -const unsigned char *addr,u16 vid); +struct netdev_switch_notifier_fdb_info *fdb_info);
You probably want to mark the fdb_info argument as 'const'.
RFC PATCH 5/5 rocker: explicitly set SW ageing for rocker
In reply to [email protected]From: Scott Feldman <[email protected]>
Rocker (for now) will rely on SW (bridge) ageing. Other drivers/devices may wish to set NTF_EXT_AGED to turn off SW (bridge) ageing.
--- drivers/net/ethernet/rocker/rocker.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/net/ethernet/rocker/rocker.c +5 -4
@@ -3026,10 +3026,11 @@ static void rocker_port_fdb_learn_work(struct work_struct *work) container_of(work, struct rocker_fdb_learn_work, work);bool removing = (lw->flags &ROCKER_OP_FLAG_REMOVE );bool learned = (lw->flags &ROCKER_OP_FLAG_LEARNED ); -struct netdev_switch_notifier_fdb_info info; - - info.addr = lw->addr; - info.vid = lw->vid; +struct netdev_switch_notifier_fdb_info info = { + .addr = lw->addr, + .vid = lw->vid, + .ntf_flags =NTF_EXT_LEARNED , + };if (learned && removing) call_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL,
Signature · 1 line
1.7.10.4
Jiri Pirko
maintainerRe: [PATCH net-next RFC 5/5] rocker: explicitly set SW ageing for rocker
Quoted 17 lines · Fri, Feb 20, 2015 at 08:09:55AM CET, [email protected] wrote:
From: Scott Feldman <[email protected]>
Rocker (for now) will rely on SW (bridge) ageing. Other
drivers/devicesmay wish to setNTF_EXT_AGEDto turn off SW (bridge) ageing.Signed-off-by Scott Feldman <[email protected]>---
drivers/net/ethernet/rocker/rocker.c| 9 +++++----1 file changed, 5 insertions(+), 4 deletions(-)drivers/net/ethernet/rocker/rocker.c +0
-0@@ -3026,10 +3026,11 @@ static void rocker_port_fdb_learn_work(struct work_struct *work) container_of(work, struct rocker_fdb_learn_work, work);bool removing = (lw->flags &ROCKER_OP_FLAG_REMOVE );
drivers/net/ethernet/rocker/rocker.c static void rocker_port_fdb_learn_work(struct work_struct *work) · line 3033
Quoted 10 lines
drivers/net/ethernet/rocker/rocker.c +5
-4bool learned = (lw->flags &ROCKER_OP_FLAG_LEARNED ); -struct netdev_switch_notifier_fdb_info info; - - info.addr = lw->addr; - info.vid = lw->vid; +struct netdev_switch_notifier_fdb_info info = { + .addr = lw->addr, + .vid = lw->vid, + .ntf_flags =NTF_EXT_LEARNED , + };
I think that this patch should be before "bridge: call external learn add if adding FDB entry with NTF_EXT_LEARNED set" in order to not break functionality during bisect.
Quoted 6 lines
drivers/net/ethernet/rocker/rocker.c +0
-0if (learned && removing) call_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL,Signature · 1 line
1.7.10.4
Re: [PATCH net-next RFC 5/5] rocker: explicitly set SW ageing for rocker
Quoted 24 lines · Jiri Pirko
Quoted 23 lines · Fri, Feb 20, 2015 at 08:09:55AM CET, [email protected] wrote:
From: Scott Feldman <[email protected]>
Rocker (for now) will rely on SW (bridge) ageing. Other
drivers/devicesmay wish to setNTF_EXT_AGEDto turn off SW (bridge) ageing.Signed-off-by Scott Feldman <[email protected]>---
drivers/net/ethernet/rocker/rocker.c| 9 +++++----1 file changed, 5 insertions(+), 4 deletions(-)drivers/net/ethernet/rocker/rocker.c +1
-4@@ -3026,10 +3026,11 @@ static void rocker_port_fdb_learn_work(struct work_struct *work) container_of(work, struct rocker_fdb_learn_work, work);bool removing = (lw->flags &ROCKER_OP_FLAG_REMOVE );bool learned = (lw->flags &ROCKER_OP_FLAG_LEARNED ); -struct netdev_switch_notifier_fdb_info info; - - info.addr = lw->addr; - info.vid = lw->vid; +struct netdev_switch_notifier_fdb_info info = {
Quoted 8 lines
drivers/net/ethernet/rocker/rocker.c static void rocker_port_fdb_learn_work(struct work_struct *work) · line 3033
Quoted 4 lines
drivers/net/ethernet/rocker/rocker.c +4
-0+ .addr = lw->addr, + .vid = lw->vid, + .ntf_flags = NTF_EXT_LEARNED, + };I think that this patch should be before "bridge: call external learn add if adding FDB entry with
NTF_EXT_LEARNEDset" in order to not break functionality during bisect.
Ok, I reordered 'em. Thanks.
Quoted 8 lines · On 2/19/15, 11:09 PM, [email protected] wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for example by
offload hardware. Switchdev driver/devices can set this flag when learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.
Quoted 5 lines
This give the flexibility for driver/device to decide ageing policy based on its capabilities. For devices managing many FDB entries, it is desireable for the device to aged out its own entries. Devices not capable of aged entries can rely of SW to age out the entries.
scott, patches look good. However, I am not sure yet if there is a need to make it a per fdb entry flag.
At some point we will also need the hw ageing parameter to be configurable.
So other approach could be,
- ageing parameter on bridge gets offloaded the hw
- so, by default hw and kernel age their own entries using the same bridge device default timer
- User can explicitly disable HW ageing by using self (this needs some more thought because now the call is on the bridge device)
Thanks,
Roopa
Quoted 21 lines · roopa
Quoted 12 lines · On 2/19/15, 11:09 PM, [email protected] wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for example by
offload hardware. Switchdev driver/devices can set this flag when
learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its
ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.Quoted 8 lines
This give the flexibility for driver/device to decide ageing policy based on
its capabilities. For devices managing many FDB entries, it is desireable for
the device to aged out its own entries. Devices not capable of aged entries
can rely of SW to age out the entries.
Quoted 2 lines
scott, patches look good. However, I am not sure yet if there is a need to make it a per fdb entry flag.
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
Quoted 8 lines
At some point we will also need the hw ageing parameter to be configurable. So other approach could be,
- ageing parameter on bridge gets offloaded the hw
- so, by default hw and kernel age their own entries using the same bridge device default timer
- User can explicitly disable HW ageing by using self (this needs some more thought because now the call is on the bridge device)
I think we want to be careful with both SW and HW aging entries since a host CPU's clock might be suspended/drifting etc.. at least, we probably want one or the other to take precedence other the other one?
Signature · 1 line
Florian
Quoted 22 lines · Florian Fainelli
Quoted 21 lines · roopa
Quoted 20 lines · On 2/19/15, 11:09 PM, [email protected] wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for example by
offload hardware. Switchdev driver/devices can set this flag when
learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its
ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.This give the flexibility for driver/device to decide ageing policy based on
its capabilities. For devices managing many FDB entries, it is desireable for
the device to aged out its own entries. Devices not capable of aged entries
can rely of SW to age out the entries.
Quoted 5 lines
Quoted 2 lines
scott, patches look good. However, I am not sure yet if there is a need to make it a per fdb entry flag.
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for
Same here (for all the Marvell switches I am aware of). There is only a single timer with a granularity of 15 seconds (or maybe 16 seconds on some chips).
Quoted 2 lines
higher end switches, or even maybe per-FDB entry?
For the Marvell switches, the only per-fdb entry is a flag to indicate static vs. dynamic.
Quoted 9 lines
Quoted 8 lines
At some point we will also need the hw ageing parameter to be configurable. So other approach could be,
- ageing parameter on bridge gets offloaded the hw
- so, by default hw and kernel age their own entries using the same bridge device default timer
- User can explicitly disable HW ageing by using self (this needs some more thought because now the call is on the bridge device)
Quoted 3 lines
I think we want to be careful with both SW and HW aging entries since a host CPU's clock might be suspended/drifting etc.. at least, we probably want one or the other to take precedence other the other one?
I may be missing something, but I don't immediately see how this patch set helps me solve any of the problems I am seeing when integrating the Marvell switch code. Even if the switch internally does hardware aging, it still seems to me that we'll need software aging on top of that, even if the bridge code in the kernel has the same addresses in its fdb as the switch. I see no feasible means to keep the fdb in the switch synchronized with the fdb in the kernel.
How would this work with Broadcom switch chips ?
Thanks,
Guenter
Quoted 30 lines
-----Original Message-----
From: Florian Fainelli [mailto:[email protected]]
Sent: Saturday, February 21, 2015 12:44 AM
To: roopa; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Viswanath Bandaru; [email protected]
Subject: Re: [PATCH net-next RFC 0/5] AddNTF_EXT_AGEDto control FDB ageing in SW or HWQuoted 4 lines · roopa
Quoted 3 lines · On 2/19/15, 11:09 PM, [email protected] wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for exampleby
Quoted 13 lines
Quoted 13 lines
offload hardware. Switchdev driver/devices can set this flag when
learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.This give the flexibility for driver/device to decide ageing policy based on its capabilities. For devices managing many FDB entries, it is desireable for the device to aged out its own entries. Devices not capable of aged entries can rely of SW to age out the entries.
Quoted 7 lines
Quoted 2 lines
scott, patches look good. However, I am not sure yet if there is a need to make it a per fdb entry flag.
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
The decision to age an entry is on a per-FDB entry, So if the flag is not available in the fdb-entry, then it has to be obtained from associated netdev. I think the per-FDB may be more convenient than keeping it on netdev.
Quoted 5 lines
Quoted 2 lines
At some point we will also need the hw ageing parameter to be
configurable.
Quoted 2 lines
So other approach could be,
- ageing parameter on bridge gets offloaded the hw
Like everybody said, the ageing is really the switch's property. If there are two bridges with ports from same switching device, it becomes tricky to configure ageing time on the switch. ( I think this is what you may be referring to be saying it needs more thought).
Quoted 10 lines
Quoted 4 lines
- so, by default hw and kernel age their own entries using the same bridge device default timer
- User can explicitly disable HW ageing by using self (this needs some more thought because now the call is on the bridge device)I think we want to be careful with both SW and HW aging entries since a host CPU's clock might be suspended/drifting etc.. at least, we probably want one or the other to take precedence other the other one?
--
Florian
Quoted 3 lines · Viswanath Bandaru
Like everybody said, the ageing is really the switch's property. If there are two bridges with ports from same switching device, it becomes tricky to configure ageing time on the switch. ( I think this is what you may be referring to be saying it needs more thought).
There is only so much we can do. One thing we can not do is to change the existing switch hardware. Maybe we can ask switch vendors to consider adding the ability to provide aging time per bridge group or per port, but even if vendors agree to do that, it won't happen quickly, and it won't fix existing chips.
Asking the switch to set its aging time to min(aging for all ports) is better than to keep the aging time at the default of 5 minutes for all ports, no matter what.
Guenter
Quoted 29 lines · Viswanath Bandaru
Quoted 29 lines
-----Original Message-----
From: Florian Fainelli [mailto:[email protected]]
Sent: Saturday, February 21, 2015 12:44 AM
To: roopa; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Viswanath Bandaru; [email protected]
Subject: Re: [PATCH net-next RFC 0/5] AddNTF_EXT_AGEDto control FDB ageing in SW or HWQuoted 4 lines · roopa
Quoted 3 lines · On 2/19/15, 11:09 PM, [email protected] wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for exampleby
Quoted 13 lines
Quoted 13 lines
offload hardware. Switchdev driver/devices can set this flag when
learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.This give the flexibility for driver/device to decide ageing policy based on its capabilities. For devices managing many FDB entries, it is desireable for the device to aged out its own entries. Devices not capable of aged entries can rely of SW to age out the entries.
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
scott, patches look good. However, I am not sure yet if there is a need to make it a per fdb entry flag.
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
The decision to age an entry is on a per-FDB entry, So if the flag is not available in the fdb-entry, then it has to be obtained from associated netdev. I think the per-FDB may be more convenient than keeping it on netdev.
I was not saying netdev, but the bridge could have it...something like the below:
net/bridge/br_fdb.c +2 -1
@@ -280,7 +280,8 @@ void br_fdb_cleanup(unsigned long _data)hlist_for_each_entry_safe (f, n, &br->hash[i], hlist) {unsigned long this_timer; -if (f->is_static) +if (f->is_static || (f->added_by_external_learn && + br->hw_aging_enabled))continue ; this_timer = f->updated + delay;if (time_before_eq (this_timer, jiffies))
net/bridge/br_private.h +1 -0
@@ -246,6 +246,7 @@ struct net_bridgeunsigned char topology_change;unsigned char topology_change_detected; +unsigned char hw_aging_enabled;#ifdef CONFIG_BRIDGE_IGMP_SNOOPING unsigned char multicast_router;
I have no objections to the current patchset (they clearly fix the problem). I was only wondering if the notion of aging can continue to stay in the bridge and not be carried into every fdb entry. which seems cleaner to me.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
At some point we will also need the hw ageing parameter to be
configurable.
Quoted 2 lines
So other approach could be,
- ageing parameter on bridge gets offloaded the hwLike everybody said, the ageing is really the switch's property. If there are two bridges with ports from same switching device, it becomes tricky to configure ageing time on the switch. ( I think this is what you may be referring to be saying it needs more thought).
Regarding the two bridges with ports from the same switching device, I think the switch driver should handle that. Maybe min value etc as discussed further in this thread.
Today given that most people are going to use the vlan filtering bridge and most likely prefer having a single vlan filtering bridge represent their swicth (we do in most cases), I see this being a non critical problem to solve.
Thanks,
Roopa
A message that is not in the archive (sent off-list or not yet received): CAE4R7bAPS1GZKaC4M6x9cqfTOkVju1+Po4KzanfSniEFX9oi1w@mail.gmail.com
Quoted 10 lines
Geunter had this question in the thread:
"I may be missing something, but I don't immediately see how this patch set helps me solve any of the problems I am seeing when integrating the Marvell switch code. Even if the switch internally does hardware aging, it still seems to me that we'll need software aging on top of that, even if the bridge code in the kernel has the same addresses in its fdb as the switch. I see no feasible means to keep the fdb in the switch synchronized with the fdb in the kernel".
Quoted 4 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Andrew
Quoted 15 lines · Andrew Lunn
Quoted 10 lines
Geunter had this question in the thread:
"I may be missing something, but I don't immediately see how this patch set helps me solve any of the problems I am seeing when integrating the Marvell switch code. Even if the switch internally does hardware aging, it still seems to me that we'll need software aging on top of that, even if the bridge code in the kernel has the same addresses in its fdb as the switch. I see no feasible means to keep the fdb in the switch synchronized with the fdb in the kernel".
Quoted 4 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.
Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Agreed.
Some of the Marvell chip support accessing its registers through Ethernet, so that may be an option. That is not supported on all chips, so it would not be a generic solution, but it may be worthwhile looking into.
Guenter
Quoted 1 line · Guenter Roeck
<snip>
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Quoted 9 lines
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
I do agree that reading all of FDB into CPU is a pain. Given the table size of 1K or 8K, I am (probably incorrectly) speculating that the device may be a router primarily. Also, not having means of learn-notifications and/or a quick (hw) interface to get all the table, I (again probably incorrectly) speculate that there are not many use cases associated with FDB and end-user/CPU for this silicon.
So I am thinking why would you want to read FDB to CPU ? You can disable learning on the bridge and have the driver not send any learning notifications to kernel, while the silicon continues to learn and forward. The end user may not be able see the FDB on a command, but is this a requirement for you ?
I may be missing some use cases here, so would you mind mentioning ?
Quoted 11 lines
Agreed.
Some of the Marvell chip support accessing its registers through Ethernet, so that may be an option. That is not supported on all chips, so it would not be a generic solution, but it may be worthwhile looking into.
Guenter
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 27 lines · B Viswanath
Quoted 1 line · Guenter Roeck
<snip>
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Quoted 9 lines
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Quoted 3 lines
I do agree that reading all of FDB into CPU is a pain. Given the table size of 1K or 8K, I am (probably incorrectly) speculating that the device may be a router primarily. Also, not having means of
No. I don't think any of the Marvell or Broadcom entry level switch chips supports L3.
See http://www.marvell.com/switching/link-street/ and
http://community.broadcom.com/docs/DOC-1724.
Forwarding table size in those chips is from 1k all the way up to 16k.
It is correct to assume that some of those chips are _used_ in routers/L3 switches, but that would not be used for L3 data but to interrconnect the various cards in the system, primarily for internal management traffic.
Quoted 4 lines
learn-notifications and/or a quick (hw) interface to get all the table, I (again probably incorrectly) speculate that there are not many use cases associated with FDB and end-user/CPU for this silicon.
Quoted 8 lines
So I am thinking why would you want to read FDB to CPU ? You can disable learning on the bridge and have the driver not send any learning notifications to kernel, while the silicon continues to learn and forward. The end user may not be able see the FDB on a command, but is this a requirement for you ?
I may be missing some use cases here, so would you mind mentioning ?
A bridge can span multiple switch chips as well as some local interfaces. In that case, it would be beneficial if the switch would be able to share its fdb with the CPU, but I don't think it is mandatory. I may be missing something, though.
Guenter
Quoted 38 lines · Guenter Roeck
Quoted 27 lines · B Viswanath
Quoted 1 line · Guenter Roeck
<snip>
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Quoted 9 lines
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Quoted 3 lines
I do agree that reading all of FDB into CPU is a pain. Given the table size of 1K or 8K, I am (probably incorrectly) speculating that the device may be a router primarily. Also, not having means of
No. I don't think any of the Marvell or Broadcom entry level switch chips supports L3.
See http://www.marvell.com/switching/link-street/ and
http://community.broadcom.com/docs/DOC-1724.
Quoted 6 lines
Forwarding table size in those chips is from 1k all the way up to 16k.
It is correct to assume that some of those chips are _used_ in routers/L3 switches, but that would not be used for L3 data but to interrconnect the various cards in the system, primarily for internal management traffic.
I didn't mean that the chips support L3 in hardware, as you said they don't. I meant that these chips are used in routers which have routing done by CPU. Typically the OPENWRT class devices, SOHO or similar devices.
For these routers, exposing FDB to CPU is not really a requirement. So the hardware is not built to have learn/age-notifications or efficient access to FDB. This is the reason I asked you whether the driver you are developing really needs to expose FDB to CPU.
Quoted 11 lines
Quoted 11 lines
learn-notifications and/or a quick (hw) interface to get all the table, I (again probably incorrectly) speculate that there are not many use cases associated with FDB and end-user/CPU for this silicon.
So I am thinking why would you want to read FDB to CPU ? You can disable learning on the bridge and have the driver not send any learning notifications to kernel, while the silicon continues to learn and forward. The end user may not be able see the FDB on a command, but is this a requirement for you ?
Quoted 6 lines
Quoted 2 lines
I may be missing some use cases here, so would you mind mentioning ?
A bridge can span multiple switch chips as well as some local interfaces. In that case, it would be beneficial if the switch would be able to share its fdb with the CPU, but I don't think it is mandatory. I may be missing something, though.
This is a general usecase and is usually with chips that can support learning notifications and other CPU controls on FDB. It can be implemented via bridge controlling the FDB. But I suspect this usecase won't be applicable for the device you are attempting to port the driver to.
Quoted 2 lines
Guenter
Quoted 46 lines · B Viswanath
Quoted 38 lines · Guenter Roeck
Quoted 27 lines · B Viswanath
Quoted 1 line · Guenter Roeck
<snip>
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Quoted 9 lines
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Quoted 3 lines
I do agree that reading all of FDB into CPU is a pain. Given the table size of 1K or 8K, I am (probably incorrectly) speculating that the device may be a router primarily. Also, not having means of
No. I don't think any of the Marvell or Broadcom entry level switch chips supports L3.
See http://www.marvell.com/switching/link-street/ and
http://community.broadcom.com/docs/DOC-1724.Quoted 6 lines
Forwarding table size in those chips is from 1k all the way up to 16k.
It is correct to assume that some of those chips are _used_ in routers/L3 switches, but that would not be used for L3 data but to interrconnect the various cards in the system, primarily for internal management traffic.
Quoted 5 lines
I didn't mean that the chips support L3 in hardware, as you said they don't. I meant that these chips are used in routers which have routing done by CPU. Typically the OPENWRT class devices, SOHO or similar devices.
No, that is not the use case in any of the use cases I am aware of and have been involved with.
Quoted 23 lines
For these routers, exposing FDB to CPU is not really a requirement. So the hardware is not built to have learn/age-notifications or efficient access to FDB. This is the reason I asked you whether the driver you are developing really needs to expose FDB to CPU.
Quoted 11 lines
Quoted 11 lines
learn-notifications and/or a quick (hw) interface to get all the table, I (again probably incorrectly) speculate that there are not many use cases associated with FDB and end-user/CPU for this silicon.
So I am thinking why would you want to read FDB to CPU ? You can disable learning on the bridge and have the driver not send any learning notifications to kernel, while the silicon continues to learn and forward. The end user may not be able see the FDB on a command, but is this a requirement for you ?
Quoted 6 lines
Quoted 2 lines
I may be missing some use cases here, so would you mind mentioning ?
A bridge can span multiple switch chips as well as some local interfaces. In that case, it would be beneficial if the switch would be able to share its fdb with the CPU, but I don't think it is mandatory. I may be missing something, though.
Quoted 6 lines
This is a general usecase and is usually with chips that can support learning notifications and other CPU controls on FDB. It can be implemented via bridge controlling the FDB. But I suspect this usecase won't be applicable for the device you are attempting to port the driver to.
What do you mean with "bridge controlling the FDB" ? You mean by software ? That would not scale for the reasons mentioned earlier; the switch chips need to be able to populate their own FDBs.
Note that I am not that much concerned about "my" use cases, at least not for now. For those cases, there will in general only be one port connected to the CPU, and a bridge group is not shared across multiple ethernet ports conneted to the CPU and some of the switch ports. But that doesn't mean that we should ignore that possibility.
Thanks,
Guenter
Quoted 47 lines · Guenter Roeck
Quoted 46 lines · B Viswanath
Quoted 44 lines · Guenter Roeck
Quoted 27 lines · B Viswanath
Quoted 1 line · Guenter Roeck
<snip>
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Quoted 9 lines
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Quoted 3 lines
I do agree that reading all of FDB into CPU is a pain. Given the table size of 1K or 8K, I am (probably incorrectly) speculating that the device may be a router primarily. Also, not having means of
No. I don't think any of the Marvell or Broadcom entry level switch chips supports L3.
See http://www.marvell.com/switching/link-street/ and
http://community.broadcom.com/docs/DOC-1724.Forwarding table size in those chips is from 1k all the way up to 16k.
It is correct to assume that some of those chips are _used_ in routers/L3 switches, but that would not be used for L3 data but to interrconnect the various cards in the system, primarily for internal management traffic.
Quoted 7 lines
Quoted 5 lines
I didn't mean that the chips support L3 in hardware, as you said they don't. I meant that these chips are used in routers which have routing done by CPU. Typically the OPENWRT class devices, SOHO or similar devices.
No, that is not the use case in any of the use cases I am aware of and have been involved with.
Ok.
Quoted 31 lines
Quoted 24 lines
For these routers, exposing FDB to CPU is not really a requirement. So the hardware is not built to have learn/age-notifications or efficient access to FDB. This is the reason I asked you whether the driver you are developing really needs to expose FDB to CPU.
Quoted 11 lines
Quoted 11 lines
learn-notifications and/or a quick (hw) interface to get all the table, I (again probably incorrectly) speculate that there are not many use cases associated with FDB and end-user/CPU for this silicon.
So I am thinking why would you want to read FDB to CPU ? You can disable learning on the bridge and have the driver not send any learning notifications to kernel, while the silicon continues to learn and forward. The end user may not be able see the FDB on a command, but is this a requirement for you ?
Quoted 6 lines
Quoted 2 lines
I may be missing some use cases here, so would you mind mentioning ?
A bridge can span multiple switch chips as well as some local interfaces. In that case, it would be beneficial if the switch would be able to share its fdb with the CPU, but I don't think it is mandatory. I may be missing something, though.
Quoted 6 lines
This is a general usecase and is usually with chips that can support learning notifications and other CPU controls on FDB. It can be implemented via bridge controlling the FDB. But I suspect this usecase won't be applicable for the device you are attempting to port the driver to.
Quoted 3 lines
What do you mean with "bridge controlling the FDB" ? You mean by software ? That would not scale for the reasons mentioned earlier; the switch chips need to be able to populate their own FDBs.
Yes, I mean software. I think the scalability issue is not applicable for chips (in the end-devices) which don't have the need to have CPU looking at FDB. Those chips can maintain their own FDB but don't need to inform CPU, therefore bypassing the scalability issue. Please also see my note below.
Quoted 6 lines
Note that I am not that much concerned about "my" use cases, at least not for now. For those cases, there will in general only be one port connected to the CPU, and a bridge group is not shared across multiple ethernet ports conneted to the CPU and some of the switch ports. But that doesn't mean that we should ignore that possibility.
The general usecases are those that deal with CPU looking at the FDB and/or somehow trying to manipulate. I guess I am trying to say that these usecases are really applicable to devices/silicon where FDB addition/ageing notifications are supported by the silicon and where FDB can be read quickly (than MDIO/SPI/I2C) by CPU. Therefore the drivers can afford to learn the FDB additions/deletions and update the bridge, thus the bridge and the hardware FDB can be in sync.
For those devices/silicon which don't offer these capabilities, the use cases themselves are not applicable. The driver can afford to not read FDB from silicon. Therefore, the scalability issues (reading huge data from MDIO/SPI/I2C) don't really come into play.
Quoted 3 lines
Thanks,
Guenter
Quoted 113 lines · B Viswanath
Quoted 47 lines · Guenter Roeck
Quoted 46 lines · B Viswanath
Quoted 44 lines · Guenter Roeck
Quoted 27 lines · B Viswanath
Quoted 1 line · Guenter Roeck
<snip>
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
You'll want to turn learning off on the bridge, and enable learning (and
learning_sync) in hardware. The hw driver will install an FDB entry in the bridge's FDB and mark it "external". The entry will also appear in the device's FDB.Quoted 7 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Quoted 9 lines
Which, coincidentially, is the case in my application. The newer Marvell switches support up to 8k forwarding table entries, so that would be really awkward.
Quoted 5 lines
We probably need a design for devices where we can efficiently get access to the hardware table, and use it in the software bridge. But we also need a design where the SW and HW bridges have independently tables.
Quoted 3 lines
I do agree that reading all of FDB into CPU is a pain. Given the table size of 1K or 8K, I am (probably incorrectly) speculating that the device may be a router primarily. Also, not having means of
No. I don't think any of the Marvell or Broadcom entry level switch chips supports L3.
See http://www.marvell.com/switching/link-street/ and
http://community.broadcom.com/docs/DOC-1724.Forwarding table size in those chips is from 1k all the way up to 16k.
It is correct to assume that some of those chips are _used_ in routers/L3 switches, but that would not be used for L3 data but to interrconnect the various cards in the system, primarily for internal management traffic.
Quoted 7 lines
Quoted 5 lines
I didn't mean that the chips support L3 in hardware, as you said they don't. I meant that these chips are used in routers which have routing done by CPU. Typically the OPENWRT class devices, SOHO or similar devices.
No, that is not the use case in any of the use cases I am aware of and have been involved with.
Ok.
Quoted 31 lines
Quoted 24 lines
For these routers, exposing FDB to CPU is not really a requirement. So the hardware is not built to have learn/age-notifications or efficient access to FDB. This is the reason I asked you whether the driver you are developing really needs to expose FDB to CPU.
Quoted 11 lines
Quoted 11 lines
learn-notifications and/or a quick (hw) interface to get all the table, I (again probably incorrectly) speculate that there are not many use cases associated with FDB and end-user/CPU for this silicon.
So I am thinking why would you want to read FDB to CPU ? You can disable learning on the bridge and have the driver not send any learning notifications to kernel, while the silicon continues to learn and forward. The end user may not be able see the FDB on a command, but is this a requirement for you ?
Quoted 6 lines
Quoted 2 lines
I may be missing some use cases here, so would you mind mentioning ?
A bridge can span multiple switch chips as well as some local interfaces. In that case, it would be beneficial if the switch would be able to share its fdb with the CPU, but I don't think it is mandatory. I may be missing something, though.
Quoted 6 lines
This is a general usecase and is usually with chips that can support learning notifications and other CPU controls on FDB. It can be implemented via bridge controlling the FDB. But I suspect this usecase won't be applicable for the device you are attempting to port the driver to.
Quoted 3 lines
What do you mean with "bridge controlling the FDB" ? You mean by software ? That would not scale for the reasons mentioned earlier; the switch chips need to be able to populate their own FDBs.
Yes, I mean software. I think the scalability issue is not applicable for chips (in the end-devices) which don't have the need to have CPU looking at FDB. Those chips can maintain their own FDB but don't need to inform CPU, therefore bypassing the scalability issue. Please also see my note below.
Quoted 6 lines
Note that I am not that much concerned about "my" use cases, at least not for now. For those cases, there will in general only be one port connected to the CPU, and a bridge group is not shared across multiple ethernet ports conneted to the CPU and some of the switch ports. But that doesn't mean that we should ignore that possibility.
The general usecases are those that deal with CPU looking at the FDB and/or somehow trying to manipulate. I guess I am trying to say that these usecases are really applicable to devices/silicon where FDB addition/ageing notifications are supported by the silicon and where FDB can be read quickly (than MDIO/SPI/I2C) by CPU. Therefore the drivers can afford to learn the FDB additions/deletions and update the bridge, thus the bridge and the hardware FDB can be in sync.
Quoted 5 lines
For those devices/silicon which don't offer these capabilities, the use cases themselves are not applicable. The driver can afford to not read FDB from silicon. Therefore, the scalability issues (reading huge data from MDIO/SPI/I2C) don't really come into play.
Looks like we are in violent agreement.
Thanks,
Guenter
David Miller
maintainerFrom: Andrew Lunn <[email protected]>
Date: Wed, 25 Feb 2015 15:31:13 +0100
Quoted 6 lines
I don't think this is going to work. There is no efficient way to get the hardware tables out of the hardware. We don't get notification of additions or removals. We can only read the whole table. And it can be expensive to read the whole table, since it can be 1K or more entries, going over an MDIO bus, which in the worst case can be bit banging on gpio lines.
Agreed, for most chips, this is how things seem to be structured.
Jiri Pirko
maintainerQuoted 22 lines · Fri, Feb 20, 2015 at 08:13:46PM CET, [email protected] wrote:
Quoted 21 lines · roopa
Quoted 20 lines · On 2/19/15, 11:09 PM, [email protected] wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for example by
offload hardware. Switchdev driver/devices can set this flag when
learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its
ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.This give the flexibility for driver/device to decide ageing policy based on
its capabilities. For devices managing many FDB entries, it is desireable for
the device to aged out its own entries. Devices not capable of aged entries
can rely of SW to age out the entries.
Quoted 6 lines
Quoted 2 lines
scott, patches look good. However, I am not sure yet if there is a need to make it a per fdb entry flag.
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
I'm currently not aware of any hw which does not have global age timer. But I believe that they will appear. The model that we have now, to propagate aging setting of bridge down is more general and should be ok.
Drivers should probably take care of multi bridge setup with different aging setup. Maybe to find minimal time and print a warning?
Quoted 15 lines
Quoted 9 lines
At some point we will also need the hw ageing parameter to be configurable. So other approach could be,
- ageing parameter on bridge gets offloaded the hw
- so, by default hw and kernel age their own entries using the same bridge device default timer
- User can explicitly disable HW ageing by using self (this needs some more thought because now the call is on the bridge device)I think we want to be careful with both SW and HW aging entries since a host CPU's clock might be suspended/drifting etc.. at least, we probably want one or the other to take precedence other the other one?
Signature · 1 line
Florian
Quoted 6 lines
-----Original Message-----
Quoted 4 lines
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
Quoted 7 lines
I'm currently not aware of any hw which does not have global age timer. But I believe that they will appear. The model that we have now, to propagate aging setting of bridge down is more general and should be ok.
Drivers should probably take care of multi bridge setup with different aging setup. Maybe to find minimal time and print a warning?
Setting up the minimal time in such a scenario is good.
Should we also consider the possibility bridges containing ports from different devices (and therefore different drivers) ? If that is a possibility, I think the bridge module should take responsibility of finding out the minimal time to pushing to all involved drivers.
Jiri Pirko
maintainerQuoted 10 lines · Sat, Feb 21, 2015 at 12:29:38PM CET, [email protected] wrote:
Quoted 10 lines
-----Original Message-----
Quoted 4 lines
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
I'm currently not aware of any hw which does not have global age timer. But I believe that they will appear. The model that we have now, to propagate aging setting of bridge down is more general and should be ok.
Quoted 7 lines
Quoted 3 lines
Drivers should probably take care of multi bridge setup with different aging setup. Maybe to find minimal time and print a warning?
Setting up the minimal time in such a scenario is good.
Should we also consider the possibility bridges containing ports from different devices (and therefore different drivers) ? If that is a possibility, I think the bridge module should take responsibility of finding out the minimal time to pushing to all involved drivers.
It is certainly possible to bridge ports from multiple switch devices. But that should not be a problem, because 1 bridge has 1 aging setup which will be passed to all port drivers.
I believe that the only case which need to be resolved is multiple bridges over single switch device. And I believe that it should be handled in drivers because only drivers know what the hw is capable of (if it supports aging setup per port/bridge/global).
Quoted 2 lines
Quoted 28 lines
-----Original Message-----
From: Jiri Pirko [mailto:[email protected]]
Sent: Saturday, February 21, 2015 9:20 PM
To: Viswanath Bandaru
Cc: Florian Fainelli; roopa; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH net-next RFC 0/5] AddNTF_EXT_AGEDto control FDB ageing in SW or HWQuoted 16 lines · Sat, Feb 21, 2015 at 12:29:38PM CET, [email protected] wrote:
Quoted 6 lines
-----Original Message-----
Quoted 4 lines
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
Quoted 7 lines
I'm currently not aware of any hw which does not have global age timer. But I believe that they will appear. The model that we have now, to propagate aging setting of bridge down is more general and should be ok.
Drivers should probably take care of multi bridge setup with different aging setup. Maybe to find minimal time and print a warning?
Setting up the minimal time in such a scenario is good.
Quoted 8 lines
Quoted 1 line
Should we also consider the possibility bridges containing ports from
different devices (and therefore different drivers) ? If that is a possibility, I think the bridge module should take responsibility of finding out the minimal time to pushing to all involved drivers.
It is certainly possible to bridge ports from multiple switch devices. But that should not be a problem, because 1 bridge has 1 aging setup which will be passed to all port drivers.
I apologize for confusion, I actually meant multiple bridge devices (which was the context) each having ports from different switch devices. Is this expected to be expected in reality ?
Quoted 8 lines
I believe that the only case which need to be resolved is multiple bridges over single switch device. And I believe that it should be handled in drivers because only drivers know what the hw is capable of (if it supports aging setup per port/bridge/global).
Quoted 2 lines
Quoted 15 lines · Jiri Pirko
Quoted 8 lines · Sat, Feb 21, 2015 at 12:29:38PM CET, [email protected] wrote:
Quoted 8 lines
-----Original Message-----
Quoted 3 lines
I agree, in fact, most of the HW I have access to only has a global age timer configuration knob. Is this configurable on a per-port basis for higher end switches, or even maybe per-FDB entry?
I'm currently not aware of any hw which does not have global age timer. But I believe that they will appear. The model that we have now, to propagate aging setting of bridge down is more general and should be ok.
Quoted 6 lines
Quoted 3 lines
Drivers should probably take care of multi bridge setup with different aging setup. Maybe to find minimal time and print a warning?
Setting up the minimal time in such a scenario is good.
Should we also consider the possibility bridges containing ports from different devices (and therefore different drivers) ? If that is a possibility, I think the bridge module should take responsibility of finding out the minimal time to pushing to all involved drivers.
Quoted 10 lines
It is certainly possible to bridge ports from multiple switch devices. But that should not be a problem, because 1 bridge has 1 aging setup which will be passed to all port drivers.
I believe that the only case which need to be resolved is multiple bridges over single switch device. And I believe that it should be handled in drivers because only drivers know what the hw is capable of (if it supports aging setup per port/bridge/global).
Quoted 1 line
I agree.
Quoted 29 lines · On Fri, Feb 20, 2015 at 12:39 PM, <[email protected]> wrote:
From: Scott Feldman <[email protected]>
Add a new
NTF_EXT_FLAGto mark an FDB as externally aged, for example by
offload hardware. Switchdev driver/devices can set this flag when learning a
new FDB entry and SW (the bridge driver) will skip this entry when running its ageing task. If flag is set, the driver/device is responsible for callingcall_netdev_switch_notifiers(NETDEV_SWITCH_FDB_DEL, ...)when entry expires.This give the flexibility for driver/device to decide ageing policy based on its capabilities. For devices managing many FDB entries, it is desireable for the device to aged out its own entries. Devices not capable of aged entries can rely of SW to age out the entries.
Scott Feldman (5):
neighbour: add external aged flag
switchdev: addntf_flagsto FDB notifier
bridge: call external learn add if adding FDB entry with
NTF_EXT_LEARNEDset
bridge: let HW control FDB ageing by settingNTF_EXT_AGED
rocker: explicitly set SW ageing for rockerdrivers/net/ethernet/rocker/rocker.c | 9 +++++---- include/net/switchdev.h | 1 + include/uapi/linux/neighbour.h | 1 + net/bridge/br.c | 18 ++++++++++-------- net/bridge/br_fdb.c | 16 +++++++++++++--- net/bridge/br_private.h | 8 +++++--- 6 files changed, 35 insertions(+), 18 deletions(-)
Quoted 3 lines
--
1.7.10.4
Patches look good. With these, my earlier patch
(http://patchwork.ozlabs.org/patch/435597/) is no longer required.
Signature · 2 lines
Regards, Siva Mannem.