net: configurable device name hash
Subsystems (2)
PATCH 1/1 net: configurable device name hash
include/net/net_namespace.h +1 -2
@@ -29,8 +29,7 @@ struct net_generic;struct sock ; -#define NETDEV_HASHBITS 8 -#define NETDEV_HASHENTRIES (1 << NETDEV_HASHBITS) +#define NETDEV_HASHENTRIES (1 << CONFIG_NETDEV_HASHBITS)struct net {atomic_t count;/* To decided when the network
net/Kconfig +7 -0
@@ -45,6 +45,13 @@ config COMPAT_NETLINK_MESSAGES menu "Networking options" +config NETDEV_HASHBITS + int "Network device hash size" + range 8 20 + default 8 + help + Select network device hash size as a power of 2. + source "net/packet/Kconfig" source "net/unix/Kconfig" source "net/xfrm/Kconfig"
David Miller
maintainerFrom: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:16:14 +0200
Quoted 1 line
Signed-off-by Octavian Purdila <[email protected]>
We're not doing this sorry.
Dynamically size it at boot time or something, but a config option is out of the question.
Quoted 10 lines · On Wednesday 11 November 2009 21:21:20 you wrote:
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:16:14 +0200Quoted 1 line
Signed-off-by Octavian Purdila <[email protected]>We're not doing this sorry.
Dynamically size it at boot time or something, but a config option is out of the question.
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
Would it be acceptable to grow it at runtime, in list_netdevice for instance?
Quoted 7 lines · Octavian Purdila
Quoted 6 lines · On Wednesday 11 November 2009 21:21:20 you wrote:
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:16:14 +0200Quoted 1 line
Signed-off-by Octavian Purdila <[email protected]>We're not doing this sorry.
Quoted 8 lines
Quoted 3 lines
Dynamically size it at boot time or something, but a config option is out of the question.
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
Would it be acceptable to grow it at runtime, in
list_netdevicefor instance?
It will be really hard, now we use RCU lookups...
What workload could reasonably need 1.000.000 hash slots, and 16.000.000 netdevices ?
Quoted 12 lines · On Wednesday 11 November 2009 22:08:31 you wrote:
Quoted 11 lines · Octavian Purdila
Quoted 9 lines · On Wednesday 11 November 2009 21:21:20 you wrote:
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:16:14 +0200Quoted 1 line
Signed-off-by Octavian Purdila <[email protected]>We're not doing this sorry.
Dynamically size it at boot time or something, but a config option is out of the question.
Quoted 8 lines
Quoted 5 lines
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
Would it be acceptable to grow it at runtime, in
list_netdevicefor instance?It will be really hard, now we use RCU lookups...
OK, I've forgot about that :)
Quoted 3 lines
What workload could reasonably need 1.000.000 hash slots, and 16.000.000
netdevices ?
And yes, I clearly get ahead of myself with that 20 bits.
Lets say we will max it to 14 for machines with over 1G of memory, would it be acceptable to consume 64K out of that even if in most of the usecases we will only have a handful of interfaces?
So, on second thought, perhaps is better to leave this alone and have those few users who need it to change NETDEV_HASHBITS themselves - its not like its a too heavy patch to carry around.
David Miller
maintainerFrom: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:38:44 +0200
Quoted 3 lines
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
We have no idea how many sockets will be used by the system yet we dynamically size the socket hash tables.
Please do some research and see how we handle this elsewhere in the networking.
Quoted 7 lines · On Wed, 11 Nov 2009 12:42:35 -0800 (PST) David Miller <[email protected]>…
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:38:44 +0200Quoted 3 lines
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
Quoted 5 lines
We have no idea how many sockets will be used by the system yet we dynamically size the socket hash tables.
Please do some research and see how we handle this elsewhere in the networking.
dcache also sizes hash bits at boot time on available memory. See alloc_large_system_hash().
Quoted 10 lines · On Wednesday 11 November 2009 23:33:42 you wrote:
On Wed, 11 Nov 2009 12:42:35 -0800 (PST)
Quoted 7 lines · David Miller
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:38:44 +0200Quoted 3 lines
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
Quoted 9 lines
Quoted 5 lines
We have no idea how many sockets will be used by the system yet we dynamically size the socket hash tables.
Please do some research and see how we handle this elsewhere in the networking.
dcache also sizes hash bits at boot time on available memory. See
alloc_large_system_hash().
Thanks Stephen.
I was actually taking a look at that but I see that the device hash is allocated per net namespace which means we can't use alloc_large_system_hash().
We could use a similar function that will work in the per namespace initialization context, but this might upset net namespace folks since we will get a large hash for every namespace.
Not sure what can be done to address that problem now except using a boot parameter to override the defaults. A better solution would be to be able to use "namespace create" parameters but it appears we don't have this possibility, yet.
Quoted 27 lines · On Wed, 11 Nov 2009 23:47:41 +0200 Octavian Purdila <[email protected]>…
Quoted 10 lines · On Wednesday 11 November 2009 23:33:42 you wrote:
On Wed, 11 Nov 2009 12:42:35 -0800 (PST)
Quoted 7 lines · David Miller
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 21:38:44 +0200Quoted 3 lines
I don't think we can dynamically size it at boot time since it depends on the usage pattern which is impossible to determine at boot time, right?
Quoted 9 lines
Quoted 5 lines
We have no idea how many sockets will be used by the system yet we dynamically size the socket hash tables.
Please do some research and see how we handle this elsewhere in the networking.
dcache also sizes hash bits at boot time on available memory. See
alloc_large_system_hash().Thanks Stephen.
I was actually taking a look at that but I see that the device hash is allocated per net namespace which means we can't use
alloc_large_system_hash().
Quoted 9 lines
We could use a similar function that will work in the per namespace initialization context, but this might upset net namespace folks since we will get a large hash for every namespace.
Not sure what can be done to address that problem now except using a boot parameter to override the defaults. A better solution would be to be able to use "namespace create" parameters but it appears we don't have this possibility, yet.
Remember though that really hash sizes really don't buy that much more speed. Going from 256 to 1024 gives a 4x benefit but with 10,000 devices that just means scanning 10 vs. 40 names. It is not like the file system cache where name lookup is a major component of overhead.
You can still use alloc_large_system_hash, but just constrain it to a maximum of order 10 or something.
David Miller
maintainerFrom: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 23:47:41 +0200
Quoted 3 lines
We could use a similar function that will work in the per namespace initialization context, but this might upset net namespace folks since we will get a large hash for every namespace.
Use kzalloc(), that's sufficient for a 64K or so hash table which is way more than you ever will need.
Use the GFP_* flags that will silently (ie. without a log message) fail, and divide by two until you successfully allocate the table if you're worried about memory fragmentation at allocation time.
This is so straightforward, I can't believe we're talking so much about how to implement this, it's a 15 minute hack :-)
Quoted 10 lines · On Wed, 11 Nov 2009 18:36:26 -0800 (PST) David Miller <[email protected]>…
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 23:47:41 +0200Quoted 3 lines
We could use a similar function that will work in the per namespace initialization context, but this might upset net namespace folks since we will get a large hash for every namespace.
Use
kzalloc(), that's sufficient for a 64K or so hash table which is way more than you ever will need.
Quoted 6 lines
Use the GFP_* flags that will silently (ie. without a log message) fail, and divide by two until you successfully allocate the table if you're worried about memory fragmentation at allocation time.
This is so straightforward, I can't believe we're talking so much about how to implement this, it's a 15 minute hack :-)
Yes, but sadly, sometimes there is too much history(!) to be able to be fully aware of it. "suck-it-and-see" type patches are possibly a quicker way to find out what people are thinking right now!
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 16 lines · Mark Smith
Quoted 13 lines · On Wed, 11 Nov 2009 18:36:26 -0800 (PST) David Miller <[email protected]>…
From: Octavian Purdila <[email protected]>
Date: Wed, 11 Nov 2009 23:47:41 +0200Quoted 3 lines
We could use a similar function that will work in the per namespace initialization context, but this might upset net namespace folks since we will get a large hash for every namespace.
Use
kzalloc(), that's sufficient for a 64K or so hash table which is way more than you ever will need.Use the GFP_* flags that will silently (ie. without a log message) fail, and divide by two until you successfully allocate the table if you're worried about memory fragmentation at allocation time.
Quoted 7 lines
Quoted 2 lines
This is so straightforward, I can't believe we're talking so much about how to implement this, it's a 15 minute hack :-)
Yes, but sadly, sometimes there is too much history(!) to be able to be fully aware of it. "suck-it-and-see" type patches are possibly a quicker way to find out what people are thinking right now!
Before extending hash tables, we should make sure existing algos are going to scale with millions of netdevices, and they dont scale that much for the moment. We still have many for_each_netdev() loops...
It's easy to change a constant somewhere in an include file, its less easy to make real scalability changes :(