device.h: pack struct dev_links_info
Subsystems (4)
Greg Kroah-Hartman
maintainerPATCH 1/2 device.h: pack struct dev_links_info
The dev_links_info structure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.
--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/device.h +1 -1
@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;/** * struct device - The basic device structure
Signature · 1 line
2.21.0
Greg Kroah-Hartman
maintainerPATCH 2/2 device.h: reorganize struct device
struct device is big, around 760 bytes on x86_64. It's not a critical structure, but it is embedded everywhere, so making it smaller is always a good thing.
With a recent patch that moved a field from struct device to the private structure, some benchmarks showed a very odd regression, despite this structure having nothing to do with those benchmarks. That caused me to look into the layout of the structure. Using 'pahole', it showed a number of holes and ways that the structure could be reordered in order to align some cachelines better, as well as reduce the size of the overall structure.
This patch removes 16 bytes from 'struct device' on a 64bit system, just by moving things around. Given we know there are systems with at least 30k devices in memory at once, every little byte counts, and this change could be a savings of 480k of kernel memory for them. On "normal" systems the overall memory savings would be much less.
--- include/linux/device.h | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
include/linux/device.h +11 -11
@@ -967,18 +967,14 @@ struct dev_links_info {* a higher-level representation of the device. */ struct device { +struct kobject kobj;struct device *parent;struct device_private *p; -struct kobject kobj;const char *init_name;/* initial name of the device */ const struct device_type *type; -struct mutex mutex;/* mutex to synchronize calls to -* its driver. -*/ -struct bus_type *bus;/* type of bus device is on */ struct device_driver *driver;/* which driver has allocated this device */
@@ -986,7 +982,14 @@ struct device {core doesn't touch it */ void *driver_data;/* Driver data, set and get with dev_set/get_drvdata */ +struct mutex mutex;/* mutex to synchronize calls to +* its driver. +*/ +struct dev_links_info links; +spinlock_t devres_lock; +struct list_head devres_head; +struct dev_pm_info power;struct dev_pm_domain *pm_domain;
@@ -1000,9 +1003,6 @@ struct device {struct list_head msi_list;#endif -#ifdef CONFIG_NUMA -int numa_node;/* NUMA node this device is close to */ -#endif const struct dma_map_ops *dma_ops;u64 *dma_mask;/* dma mask (if dma'able device) */ u64 coherent_dma_mask;/* Like dma_mask, but for
@@ -1032,9 +1032,6 @@ struct device {dev_t devt;/* dev_t, creates the sysfs "dev" */ u32 id;/* device instance */ -spinlock_t devres_lock; -struct list_head devres_head; -struct klist_node knode_class;struct class *class;const struct attribute_group **groups;/* optional groups */
@@ -1043,6 +1040,9 @@ struct device {struct iommu_group *iommu_group;struct iommu_fwspec *iommu_fwspec; +#ifdef CONFIG_NUMA +int numa_node;/* NUMA node this device is close to */ +#endif bool offline_disabled:1; bool offline:1; bool of_node_reused:1;
Signature · 1 line
2.21.0
Quoted 7 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>
Quoted 17 lines
--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;/** * struct device - The basic device structure
Johan Hovold
maintainerQuoted 16 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/linux/device.h +0
-0
include/linux/device.h struct dev_links_info { · line 887
Quoted 5 lines
include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;
This seems like a bad idea. You're changing the alignment of these fields to one byte, something which may cause the compiler to generate less efficient code to deal with unaligned accesses (even if they happen to currently be naturally aligned in struct device).
I don't think we should mess with __packed unless for things that actually require it (e.g. data going on to the wire) even if it means wasting 4 bytes on 64-bit archs.
Johan
Greg Kroah-Hartman
maintainerQuoted 12 lines · Johan Hovold
Quoted 11 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines
include/linux/device.h struct dev_links_info { · line 887
Quoted 10 lines
include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;This seems like a bad idea. You're changing the alignment of these fields to one byte, something which may cause the compiler to generate less efficient code to deal with unaligned accesses (even if they happen to currently be naturally aligned in struct device).
No, all this changes is the trailing "space" is gone. The alignment of the fields did not change at all as they are all naturally aligned (list_head is just 2 pointers).
Here's the pahole output before and after this patch:
Before:
struct dev_links_info {
struct list_head suppliers; /* 0 16 */
struct list_head consumers; /* 16 16 */
enum dl_dev_state status; /* 32 4 */ /* size: 40, cachelines: 1, members: 3 */
/* padding: 4 */
/* last cacheline: 40 bytes */
};After:
struct dev_links_info {
struct list_head suppliers; /* 0 16 */
struct list_head consumers; /* 16 16 */
enum dl_dev_state status; /* 32 4 */ /* size: 36, cachelines: 1, members: 3 */
/* last cacheline: 36 bytes */
};So this allows us to save 4 bytes in struct device by putting something in that trailing "hole" that can be aligned with it better (i.e. an integer or something else).
thanks,
greg k-h
Johan Hovold
maintainerQuoted 29 lines · Greg Kroah-Hartman
Quoted 12 lines · Johan Hovold
Quoted 11 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
include/linux/device.h struct dev_links_info { · line 887
Quoted 10 lines
include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;This seems like a bad idea. You're changing the alignment of these fields to one byte, something which may cause the compiler to generate less efficient code to deal with unaligned accesses (even if they happen to currently be naturally aligned in struct device).
Quoted 3 lines
No, all this changes is the trailing "space" is gone. The alignment of the fields did not change at all as they are all naturally aligned (
list_headis just 2 pointers).
Yes, currently and in struct device, but given a pointer to a struct dev_links_info the compiler must assume it is unaligned and act accordingly for example.
Quoted 3 lines
So this allows us to save 4 bytes in struct device by putting something in that trailing "hole" that can be aligned with it better (i.e. an integer or something else).
I understand that, but I don't think it is worth to start using packed liked this for internal structures as it may have subtle and unintended consequences.
Johan
Greg Kroah-Hartman
maintainerQuoted 30 lines · Johan Hovold
Quoted 29 lines · Greg Kroah-Hartman
Quoted 27 lines · Johan Hovold
Quoted 16 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/linux/device.h +0
-0include/linux/device.h struct dev_links_info { · line 887
Quoted 5 lines
include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;This seems like a bad idea. You're changing the alignment of these fields to one byte, something which may cause the compiler to generate less efficient code to deal with unaligned accesses (even if they happen to currently be naturally aligned in struct device).
Quoted 7 lines
Quoted 3 lines
No, all this changes is the trailing "space" is gone. The alignment of the fields did not change at all as they are all naturally aligned (
list_headis just 2 pointers).Yes, currently and in struct device, but given a pointer to a struct
dev_links_infothe compiler must assume it is unaligned and act accordingly for example.
Packing the structure doesn't mean that the addressing of it is not also aligned, that should just depend on the location of the pointer in the first place, right?
Surely compilers are not that foolish :)
And accessing this field should not be an issue of "slow", hopefully the memory savings would offset any compiler mess.
Quoted 7 lines
Quoted 3 lines
So this allows us to save 4 bytes in struct device by putting something in that trailing "hole" that can be aligned with it better (i.e. an integer or something else).
I understand that, but I don't think it is worth to start using packed liked this for internal structures as it may have subtle and unintended consequences.
I'm not understanding what the consequences are here, sorry. Does the compiler output change given that the structure is still aligned properly in the "parent" structure? I can't see any output changed here, but maybe I am not looking properly?
thanks,
greg k-h
Johan Hovold
maintainerQuoted 35 lines · Greg Kroah-Hartman
Quoted 34 lines · Johan Hovold
Quoted 32 lines · Greg Kroah-Hartman
Quoted 12 lines · Johan Hovold
Quoted 11 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
include/linux/device.h struct dev_links_info { · line 887
Quoted 10 lines
include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;This seems like a bad idea. You're changing the alignment of these fields to one byte, something which may cause the compiler to generate less efficient code to deal with unaligned accesses (even if they happen to currently be naturally aligned in struct device).
No, all this changes is the trailing "space" is gone. The alignment of the fields did not change at all as they are all naturally aligned (
list_headis just 2 pointers).
Quoted 7 lines
Quoted 3 lines
Yes, currently and in struct device, but given a pointer to a struct
dev_links_infothe compiler must assume it is unaligned and act accordingly for example.Packing the structure doesn't mean that the addressing of it is not also aligned, that should just depend on the location of the pointer in the first place, right?
Packing a structure per definition means changing the alignment requirement of each field of the struct to 1-byte alignment.
Another example of unintended consequences would obviously be that if someone later adds a short field, say 1-byte, field before the dev_links_info struct, all its fields would be non-naturally aligned also in struct device.
Sure that can be avoided by inspection (and refusal to add new holes), but again, not obvious when the link structure is defined elsewhere.
Quoted 4 lines
Surely compilers are not that foolish :)
And accessing this field should not be an issue of "slow", hopefully the memory savings would offset any compiler mess.
There are other subtleties like atomicity that may come into play.
And even if any penalties are deemed acceptable in this case, you're also setting a precedent for others. Note that we do not seem to use __packed this way currently
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
So this allows us to save 4 bytes in struct device by putting something in that trailing "hole" that can be aligned with it better (i.e. an integer or something else).
Quoted 8 lines
Quoted 3 lines
I understand that, but I don't think it is worth to start using packed liked this for internal structures as it may have subtle and unintended consequences.
I'm not understanding what the consequences are here, sorry. Does the compiler output change given that the structure is still aligned properly in the "parent" structure? I can't see any output changed here, but maybe I am not looking properly?
It's all arch dependent, and you won't see any difference on x86-64.
The following example produces additional instructions even on 32-bit arm here:
struct a1 {
void *p;
void *q;
int i;
} __attribute__((__packed__));struct a2 {
void *p;
void *q;
int i;
};int f(struct a1 *a)
{
return a->i;
}int g(struct a2 *a)
{
return a->i;
}Johan
Greg Kroah-Hartman
maintainerQuoted 57 lines · Johan Hovold
Quoted 35 lines · Greg Kroah-Hartman
Quoted 34 lines · Johan Hovold
Quoted 32 lines · Greg Kroah-Hartman
Quoted 12 lines · Johan Hovold
Quoted 11 lines · Greg Kroah-Hartman
The
dev_links_infostructure has 4 bytes of padding at the end of it when embedded in struct device (which is the only place it lives). To help reduce the size of struct device pack this structure so we can take advantage of the hole with later structure reorganizations.Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>--- include/linux/device.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 15 lines
include/linux/device.h struct dev_links_info { · line 887
Quoted 10 lines
include/linux/device.h +1
-1@@ -884,7 +884,7 @@ struct dev_links_info {struct list_head suppliers;struct list_head consumers;enum dl_dev_state status; -}; +} __packed;This seems like a bad idea. You're changing the alignment of these fields to one byte, something which may cause the compiler to generate less efficient code to deal with unaligned accesses (even if they happen to currently be naturally aligned in struct device).
No, all this changes is the trailing "space" is gone. The alignment of the fields did not change at all as they are all naturally aligned (
list_headis just 2 pointers).Quoted 7 lines
Quoted 3 lines
Yes, currently and in struct device, but given a pointer to a struct
dev_links_infothe compiler must assume it is unaligned and act accordingly for example.Packing the structure doesn't mean that the addressing of it is not also aligned, that should just depend on the location of the pointer in the first place, right?
Packing a structure per definition means changing the alignment requirement of each field of the struct to 1-byte alignment.
Another example of unintended consequences would obviously be that if someone later adds a short field, say 1-byte, field before the
dev_links_infostruct, all its fields would be non-naturally aligned also in struct device.Sure that can be avoided by inspection (and refusal to add new holes), but again, not obvious when the link structure is defined elsewhere.
Quoted 2 lines
Surely compilers are not that foolish :)
Quoted 8 lines
Quoted 2 lines
And accessing this field should not be an issue of "slow", hopefully the memory savings would offset any compiler mess.
There are other subtleties like atomicity that may come into play.
And even if any penalties are deemed acceptable in this case, you're also setting a precedent for others. Note that we do not seem to use
__packedthis way currently
Yeah, that is a good point, normally we use packed to keep padding from the middle of the structure from happening.
I just don't like that 4 bytes sitting there doing nothing :)
Quoted 31 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
So this allows us to save 4 bytes in struct device by putting something in that trailing "hole" that can be aligned with it better (i.e. an integer or something else).
Quoted 8 lines
Quoted 3 lines
I understand that, but I don't think it is worth to start using packed liked this for internal structures as it may have subtle and unintended consequences.
I'm not understanding what the consequences are here, sorry. Does the compiler output change given that the structure is still aligned properly in the "parent" structure? I can't see any output changed here, but maybe I am not looking properly?
It's all arch dependent, and you won't see any difference on x86-64.
The following example produces additional instructions even on 32-bit arm here:
struct a1 { void *p; void *q; int i; } __attribute__((__packed__));struct a2 { void *p; void *q; int i; };int
f(struct a1 *a)
Quoted 8 lines
{ return a->i; }int
g(struct a2 *a){ return a->i; }
Ok, fair enough, I'll leave this alone.
But, in thinking about this, there is no real reason that I can see that this structure even is in struct device. It should be able to be in the private "internal" structure.
The patch below moves it out of struct device entirely. Overall there is no memory savings, but it could give us the chance to only create this structure if we really need it later on, as very few things use links at this point in time.
Rafael, there is one logic change below, the link structure is not initialized until device_add() happens, instead of device_initialize(). Will that affect anything that you can think of? Does anyone do anything with links before device_add() is called?
I only test-built this patch, I didn't boot anything with it to see how bad it explodes :)
thanks,
greg k-h
drivers/base/base.h +13 -0
@@ -53,6 +53,18 @@ struct driver_private { };#define to_driver (obj) container_of(obj, struct driver_private, kobj) +/** +* struct dev_links_info - Device data related to device links. +* @suppliers: List of links to supplier devices. +* @consumers: List of links to consumer devices. +* @status: Driver status information. +*/ +struct dev_links_info { +struct list_head suppliers; +struct list_head consumers; +enum dl_dev_state status; +}; +/** * struct device_private - structure to hold the private to the driver core portions of the device structure. *
@@ -76,6 +88,7 @@ struct device_private {struct klist_node knode_bus;struct list_head deferred_probe;struct device *device; +struct dev_links_info links; };#define to_device_private_parent (obj) \ container_of(obj, struct device_private, knode_parent)
drivers/base/core.c +26 -26
189 lines not on this page · raw message
drivers/base/power/main.c +4 -4
@@ -259,7 +259,7 @@ static void dpm_wait_for_suppliers(struct device *dev, bool async)* callbacks freeing the link objects for the links in the list we're * walking. */ -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node)if (READ_ONCE (link->status) !=DL_STATE_DORMANT )dpm_wait (link->supplier, async);
@@ -288,7 +288,7 @@ static void dpm_wait_for_consumers(struct device *dev, bool async)* continue instead of trying to continue in parallel with its * unregistration). */ -list_for_each_entry_rcu (link, &dev->links.consumers, s_node) +list_for_each_entry_rcu (link, &dev->p->links.consumers, s_node)if (READ_ONCE (link->status) !=DL_STATE_DORMANT )dpm_wait (link->consumer, async);
@@ -1214,7 +1214,7 @@ static void dpm_superior_set_must_resume(struct device *dev) idx =device_links_read_lock (); -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node) link->supplier->power.must_resume = true;device_links_read_unlock (idx);
@@ -1688,7 +1688,7 @@ static void dpm_clear_superiors_direct_complete(struct device *dev) idx =device_links_read_lock (); -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) { +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node) {spin_lock_irq (&link->supplier->power.lock); link->supplier->power.direct_complete = false;spin_unlock_irq (&link->supplier->power.lock);
drivers/base/power/runtime.c +5 -5
@@ -256,7 +256,7 @@ static int rpm_get_suppliers(struct device *dev) {struct device_link *link; -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) { +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node) {int retval;if (!(link->flags &DL_FLAG_PM_RUNTIME ))
@@ -281,7 +281,7 @@ static void rpm_put_suppliers(struct device *dev) {struct device_link *link; -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node)if (link->rpm_active &&READ_ONCE (link->status) !=DL_STATE_SUPPLIER_UNBIND ) {pm_runtime_put (link->supplier);
@@ -1557,7 +1557,7 @@ void pm_runtime_clean_up_links(struct device *dev) idx =device_links_read_lock (); -list_for_each_entry_rcu (link, &dev->links.consumers, s_node) { +list_for_each_entry_rcu (link, &dev->p->links.consumers, s_node) {if (link->flags &DL_FLAG_STATELESS )continue ;
@@ -1581,7 +1581,7 @@ void pm_runtime_get_suppliers(struct device *dev) idx =device_links_read_lock (); -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node)if (link->flags &DL_FLAG_PM_RUNTIME )pm_runtime_get_sync (link->supplier);
@@ -1599,7 +1599,7 @@ void pm_runtime_put_suppliers(struct device *dev) idx =device_links_read_lock (); -list_for_each_entry_rcu (link, &dev->links.suppliers, c_node) +list_for_each_entry_rcu (link, &dev->p->links.suppliers, c_node)if (link->flags &DL_FLAG_PM_RUNTIME )pm_runtime_put (link->supplier);
include/linux/device.h +0 -13
@@ -874,18 +874,6 @@ enum dl_dev_state { DL_DEV_UNBINDING, }; -/** -* struct dev_links_info - Device data related to device links. -* @suppliers: List of links to supplier devices. -* @consumers: List of links to consumer devices. -* @status: Driver status information. -*/ -struct dev_links_info { -struct list_head suppliers; -struct list_head consumers; -enum dl_dev_state status; -}; -/** * struct device - The basic device structure * @parent: The device's "parent" device, the device to which it is attached.
@@ -986,7 +974,6 @@ struct device {core doesn't touch it */ void *driver_data;/* Driver data, set and get with dev_set/get_drvdata */ -struct dev_links_info links;struct dev_pm_info power;struct dev_pm_domain *pm_domain;
Johan Hovold
maintainerQuoted 6 lines · Greg Kroah-Hartman
On Wed, Feb 27, 2019 at 11:59:51AM +0100, Johan Hovold wrote:
Yeah, that is a good point, normally we use packed to keep padding from the middle of the structure from happening.
I just don't like that 4 bytes sitting there doing nothing :)
You could perhaps put them directly in struct device if the dev_links_info struct is just used a separator there and this really bothers you. :)
Quoted 9 lines
But, in thinking about this, there is no real reason that I can see that this structure even is in struct device. It should be able to be in the private "internal" structure.
The patch below moves it out of struct device entirely. Overall there is no memory savings, but it could give us the chance to only create this structure if we really need it later on, as very few things use links at this point in time.
Quoted 4 lines
Rafael, there is one logic change below, the link structure is not initialized until
device_add()happens, instead ofdevice_initialize(). Will that affect anything that you can think of? Does anyone do anything with links beforedevice_add()is called?
I think device_add() may be too late.
The earliest point in time when device links can be added is after :c:func:device_add() has been called for the supplier and :c:func:device_initialize() has been called for the consumer.
Johan
Greg Kroah-Hartman
maintainerQuoted 11 lines · Johan Hovold
Quoted 6 lines · Greg Kroah-Hartman
On Wed, Feb 27, 2019 at 11:59:51AM +0100, Johan Hovold wrote:
Yeah, that is a good point, normally we use packed to keep padding from the middle of the structure from happening.
I just don't like that 4 bytes sitting there doing nothing :)
You could perhaps put them directly in struct device if the
dev_links_infostruct is just used a separator there and this really bothers you. :)
True :)
Quoted 14 lines
Quoted 9 lines
But, in thinking about this, there is no real reason that I can see that this structure even is in struct device. It should be able to be in the private "internal" structure.
The patch below moves it out of struct device entirely. Overall there is no memory savings, but it could give us the chance to only create this structure if we really need it later on, as very few things use links at this point in time.
Quoted 4 lines
Rafael, there is one logic change below, the link structure is not initialized until
device_add()happens, instead ofdevice_initialize(). Will that affect anything that you can think of? Does anyone do anything with links beforedevice_add()is called?
Quoted 6 lines
I think
device_add()may be too late.The earliest point in time when device links can be added is after :c:func:
device_add()has been called for the supplier and :c:func:device_initialize()has been called for the consumer.
That is true today due to the way the code is set up, but it would be good to figure out if anyone actually does call it this early.
thanks,
greg k-h
Quoted 25 lines · Greg Kroah-Hartman
Quoted 9 lines · Johan Hovold
Quoted 5 lines · Greg Kroah-Hartman
On Wed, Feb 27, 2019 at 11:59:51AM +0100, Johan Hovold wrote:
Yeah, that is a good point, normally we use packed to keep padding from the middle of the structure from happening.I just don't like that 4 bytes sitting there doing nothing :)
You could perhaps put them directly in struct device if the
dev_links_infostruct is just used a separator there and this really bothers you. :)True :)
Quoted 13 lines
Quoted 13 lines
But, in thinking about this, there is no real reason that I can see that this structure even is in struct device. It should be able to be in the private "internal" structure.
The patch below moves it out of struct device entirely. Overall there is no memory savings, but it could give us the chance to only create this structure if we really need it later on, as very few things use links at this point in time.
Rafael, there is one logic change below, the link structure is not initialized until
device_add()happens, instead ofdevice_initialize(). Will that affect anything that you can think of? Does anyone do anything with links beforedevice_add()is called?
Quoted 8 lines
Quoted 6 lines
I think
device_add()may be too late.The earliest point in time when device links can be added is after :c:func:
device_add()has been called for the supplier and :c:func:device_initialize()has been called for the consumer.That is true today due to the way the code is set up, but it would be good to figure out if anyone actually does call it this early.
ISTR a use case where it was needed in the IOMMU subsystem, but I'm not sure if it has been used that way eventually.
The only way to really find out would be to audit all of the device_link_add() callers I'm afraid.
Cheers,
Rafael
Greg Kroah-Hartman
maintainerPATCH v2 1/1 device.h: reorganize struct device
In reply to Greg Kroah-Hartmanstruct device is big, around 760 bytes on x86_64. It's not a critical structure, but it is embedded everywhere, so making it smaller is always a good thing.
With a recent patch that moved a field from struct device to the private structure, some benchmarks showed a very odd regression, despite this structure having nothing to do with those benchmarks. That caused me to look into the layout of the structure. Using 'pahole', it showed a number of holes and ways that the structure could be reordered in order to align some cachelines better, as well as reduce the size of the overall structure.
Move 'struct kobj' to the start of the structure, to keep that access in the first cacheline, and try to organize things a bit more compactly where possible
By doing these few moves, the result removes at least 8 bytes from 'struct device' on a 64bit system. Given we know there are systems with at least 30k devices in memory at once, every little byte counts, and this change could be a savings of 240k of kernel memory for them. On "normal" systems the overall memory savings would be much less.
---
v2: drop the first patch, and make this one a bit simpler to try to take
advantage where we can. It's not as much savings, but it's better than nothing.
include/linux/device.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
include/linux/device.h +8 -8
@@ -967,18 +967,14 @@ struct dev_links_info {* a higher-level representation of the device. */ struct device { +struct kobject kobj;struct device *parent;struct device_private *p; -struct kobject kobj;const char *init_name;/* initial name of the device */ const struct device_type *type; -struct mutex mutex;/* mutex to synchronize calls to -* its driver. -*/ -struct bus_type *bus;/* type of bus device is on */ struct device_driver *driver;/* which driver has allocated this device */
@@ -986,6 +982,10 @@ struct device {core doesn't touch it */ void *driver_data;/* Driver data, set and get with dev_set/get_drvdata */ +struct mutex mutex;/* mutex to synchronize calls to +* its driver. +*/ +struct dev_links_info links;struct dev_pm_info power;struct dev_pm_domain *pm_domain;
@@ -1000,9 +1000,6 @@ struct device {struct list_head msi_list;#endif -#ifdef CONFIG_NUMA -int numa_node;/* NUMA node this device is close to */ -#endif const struct dma_map_ops *dma_ops;u64 *dma_mask;/* dma mask (if dma'able device) */ u64 coherent_dma_mask;/* Like dma_mask, but for
@@ -1029,6 +1026,9 @@ struct device {struct device_node *of_node;/* associated device tree node */ struct fwnode_handle *fwnode;/* firmware device node */ +#ifdef CONFIG_NUMA +int numa_node;/* NUMA node this device is close to */ +#endif dev_t devt;/* dev_t, creates the sysfs "dev" */ u32 id;/* device instance */
Signature · 1 line
2.21.0