Improve batch times by caching link lookups
Subsystems (3)
cover Improve batch times by caching link lookups
From: David Ahern <[email protected]>
Many commands convert device names to an index using ll_name_to_index. ll_name_to_index calls if_nametoindex which is ioctl based and the result is not cached. When using a batch file this means the same device lookups can be done repeatedly adding unnecessary overhead (socket + ioctl + close for each device lookup).
This series adds a new function, ll_link_get, to send a netlink based RTM_GETLINK. If successful, the result is cached in idx_head and name_head so future lookups can re-use the entry. ll_name_to_index is updated to use ll_link_get over if_nametoindex.
The first 2 patches add a means to drop an entry from the cache and updates iplink_modify to use that new function to drop entries on device renames.
David Ahern (3):
ll_map: Add function to remove link cache entry by index ip link: Drop cache entry on name changes Improve batch times by caching link lookups
include/ll_map.h | 1 + ip/ip_common.h | 3 ++- ip/iplink.c | 10 ++++++++-- ip/iplink_vxcan.c | 3 ++- ip/link_veth.c | 3 ++- lib/ll_map.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 6 files changed, 72 insertions(+), 6 deletions(-)
Signature · 1 line
2.11.0
PATCH 1/3 ll_map: Add function to remove link cache entry by index
From: David Ahern <[email protected]>
Add ll_drop_by_index to remove an entry from the link cache.
--- include/ll_map.h | 1 + lib/ll_map.c | 14 ++++++++++++++ 2 files changed, 15 insertions(+)
include/ll_map.h +1 -0
@@ -9,6 +9,7 @@ unsigned ll_name_to_index(const char *name);const char *ll_index_to_name (unsigned idx);int ll_index_to_type (unsigned idx);int ll_index_to_flags (unsigned idx); +void ll_drop_by_index (unsigned index);unsigned namehash (const char *str);const char *ll_idx_n2a (unsigned int idx);
lib/ll_map.c +14 -0
@@ -210,6 +210,20 @@ unsigned ll_name_to_index(const char *name)return idx; } +void ll_drop_by_index (unsigned index) +{ +struct ll_cache *im; + + im =ll_get_by_index (index); +if (!im) +return ; + +hlist_del (&im->idx_hash); +hlist_del (&im->name_hash); + +free (im); +} +void ll_init_map (struct rtnl_handle *rth) {static int initialized;
Signature · 1 line
2.11.0
PATCH 2/3 ip link: Drop cache entry on name changes
In reply to David AhernFrom: David Ahern <[email protected]>
If a link's name is changed remove any entry from the link cache.
--- ip/ip_common.h | 3 ++- ip/iplink.c | 10 ++++++++-- ip/iplink_vxcan.c | 3 ++- ip/link_veth.c | 3 ++- 4 files changed, 14 insertions(+), 5 deletions(-)
ip/ip_common.h +2 -1
@@ -123,7 +123,8 @@ struct link_util {struct link_util *get_link_kind (const char *kind); -int iplink_parse (int argc,char **argv,struct iplink_req *req,char **type); +int iplink_parse (int argc,char **argv,struct iplink_req *req,char **type, +bool *name_change);/* iplink_bridge.c */ void br_dump_bridge_id (const struct ifla_bridge_id *id,char *buf,size_t len);
ip/iplink.c +8 -2
@@ -573,7 +573,8 @@ static int iplink_parse_vf(int vf, int *argcp, char ***argvp,return 0 ; } -int iplink_parse (int argc,char **argv,struct iplink_req *req,char **type) +int iplink_parse (int argc,char **argv,struct iplink_req *req,char **type, +bool *name_change) {char *name =NULL ;char *dev =NULL ;
@@ -943,6 +944,8 @@ int iplink_parse(int argc, char **argv, struct iplink_req *req, char **type) dev = name; else if (!strcmp(name, dev)) name = dev; + else if (name_change) + *name_change = true;if (dev && addr_len) {int halen =nl_get_ll_addr_len (dev);
@@ -1030,9 +1033,10 @@ static int iplink_modify(int cmd, unsigned int flags, int argc, char **argv) .n.nlmsg_type = cmd, .i.ifi_family = preferred_family, }; +bool name_change = false;int ret; - ret =iplink_parse (argc, argv, &req, &type); + ret =iplink_parse (argc, argv, &req, &type, &name_change);if (ret <0 )return ret;
@@ -1082,6 +1086,8 @@ static int iplink_modify(int cmd, unsigned int flags, int argc, char **argv)if (rtnl_talk (&rth, &req.n,NULL ) <0 )return -2 ; +else if (name_change) +ll_drop_by_index (req.i.ifi_index);return 0 ; }
ip/iplink_vxcan.c +2 -1
@@ -56,7 +56,8 @@ static int vxcan_parse_opt(struct link_util *lu, int argc, char **argv, n->nlmsg_len +=sizeof (struct ifinfomsg ); - err =iplink_parse (argc -1 , argv +1 , (struct iplink_req *)n, &type); + err =iplink_parse (argc -1 , argv +1 , (struct iplink_req *)n, &type, +NULL );if (err <0 )return err;
ip/link_veth.c +2 -1
@@ -54,7 +54,8 @@ static int veth_parse_opt(struct link_util *lu, int argc, char **argv, n->nlmsg_len +=sizeof (struct ifinfomsg ); - err =iplink_parse (argc -1 , argv +1 , (struct iplink_req *)n, &type); + err =iplink_parse (argc -1 , argv +1 , (struct iplink_req *)n, &type, +NULL );if (err <0 )return err;
Signature · 1 line
2.11.0
PATCH 3/3 Improve batch times by caching link lookups
In reply to David AhernFrom: David Ahern <[email protected]>
ip route (for example) uses ll_name_to_index to convert the user given device name to an index. At the moment ll_name_to_index uses if_nametoindex which is ioctl based and does not cache the result. When using a batch file this means the same device lookups can be done repeatedly adding unnecessary overhead (socket + ioctl + close for each device lookup).
Add a new function, ll_link_get, to send a netlink based RTM_GETLINK. If successful, cache the result in idx_head and name_head so future lookups can re-use the entry. Update ll_name_to_index to use ll_link_get over if_nametoindex.
With this change the time to install 720,022 routes with 2 ecmp nexthops where the nexthop device is given is reduced from 30.7 seconds to 17.6 seconds.
--- lib/ll_map.c | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-)
lib/ll_map.c +43 -1
@@ -192,6 +192,46 @@ int ll_index_to_flags(unsigned idx)return im ? im->flags :-1 ; } +static int ll_link_get (const char *name) +{ +struct { +struct nlmsghdr n; +struct ifinfomsg ifm; +char buf[1024 ]; + } req = { + .n.nlmsg_len =NLMSG_LENGTH (sizeof (struct ifinfomsg )), + .n.nlmsg_flags =NLM_F_REQUEST , + .n.nlmsg_type =RTM_GETLINK , + }; +__u32 filt_mask =RTEXT_FILTER_VF |RTEXT_FILTER_SKIP_STATS ; +struct rtnl_handle rth = {}; +struct nlmsghdr *answer; +int rc =0 ; + +if (rtnl_open (&rth,0 ) <0 ) +return 0 ; + +addattr32 (&req.n,sizeof (req),IFLA_EXT_MASK , filt_mask); +addattr_l (&req.n,sizeof (req),IFLA_IFNAME , name, +strlen (name) +1 ); + +if (rtnl_talk (&rth, &req.n, &answer) <0 ) + goto out; + +/* add entry to cache */ + rc =ll_remember_index (answer,NULL ); +if (!rc) { +struct ifinfomsg *ifm =NLMSG_DATA (answer); + + rc = ifm->ifi_index; + } + +free (answer); +out: +rtnl_close (&rth); +return rc; +} +unsigned ll_name_to_index (const char *name) {const struct ll_cache *im;
@@ -204,7 +244,9 @@ unsigned ll_name_to_index(const char *name)if (im)return im->index; - idx =if_nametoindex (name); + idx =ll_link_get (name); +if (idx ==0 ) + idx =if_nametoindex (name);if (idx ==0 ) idx =ll_idx_a2n (name);return idx;
Signature · 1 line
2.11.0
lib/ll_map.c unsigned ll_name_to_index(const char *name) · line 249
Quoted 3 lines · On Mon, 7 Jan 2019 14:55:52 -0800 David Ahern <[email protected]> wrote:
lib/ll_map.c +3
-0+ idx =ll_link_get (name); +if (idx ==0 ) + idx =if_nametoindex (name);
What is advantage of using netlink, other than not having to open a socket.
Re: [PATCH iproute2-next 2/3] ip link: Drop cache entry on name changes
In reply to David Ahernip/ip_common.h struct link_util { · line 127
Quoted 2 lines · On Mon, 7 Jan 2019 14:55:51 -0800 David Ahern <[email protected]> wrote:
ip/ip_common.h +2
-0+int iplink_parse (int argc,char **argv,struct iplink_req *req,char **type, +bool *name_change);
Not a real fan of adding another by reference return value flag. It makes the logic flow more complex.
Is there another way? Caching in general is dicey anyway.
Re: [PATCH iproute2-next 2/3] ip link: Drop cache entry on name changes
Quoted 11 lines · Stephen Hemminger
ip/ip_common.h struct link_util { · line 127
Quoted 2 lines · On Mon, 7 Jan 2019 14:55:51 -0800 David Ahern <[email protected]> wrote:
ip/ip_common.h +2
-0+int iplink_parse (int argc,char **argv,struct iplink_req *req,char **type, +bool *name_change);Not a real fan of adding another by reference return value flag. It makes the logic flow more complex.
Is there another way? Caching in general is dicey anyway.
The details of the link change are buried inside of the req argument. It does not make sense to decode the message to see if the devices referenced by ifi_index == IFLA_NAME.
The alternative is to just negate caching of ifi_index regardless of what the request is doing.
If the caching only gained 5 or 10% we would not be having this discussion - I would not have sent patches. The gain here is significant and the caching aligns with the whole intent of batch files.
Quoted 10 lines · Stephen Hemminger
lib/ll_map.c unsigned ll_name_to_index(const char *name) · line 249
Quoted 3 lines · On Mon, 7 Jan 2019 14:55:52 -0800 David Ahern <[email protected]> wrote:
lib/ll_map.c +3
-0+ idx =ll_link_get (name); +if (idx ==0 ) + idx =if_nametoindex (name);What is advantage of using netlink, other than not having to open a socket.
As mentioned in my response to Eric, use of netlink is to keep the cache entries in sync. The existing cache is populated by netlink messages; this function should follow suit. if_nametoindex does not allow setting the flags or type fields.