resource: Use list_head to link sibling resource
Subsystems (22)
v7 cover resource: Use list_head to link sibling resource
This patchset is doing:
1) Move reparent_resources() to kernel/resource.c to clean up duplicated code in arch/microblaze/pci/pci-common.c and arch/powerpc/kernel/pci-common.c .
2) Replace struct resource's sibling list from singly linked list to list_head. Clearing out those pointer operation within singly linked list for better code readability.
2) Based on list_head replacement, add a new function walk_system_ram_res_rev() which can does reversed iteration on iomem_resource's siblings.
3) Change kexec_file loading to search system RAM top down for kernel loadin, using walk_system_ram_res_rev().
Note:
This patchset only passed testing on x86_64 arch with network enabling. The thing we need pay attetion to is that a root resource's child member need be initialized specifically with LIST_HEAD_INIT() if statically defined or INIT_LIST_HEAD() for dynamically definition. Here Just like we do for iomem_resource/ioport_resource, or the change in get_pci_domain_busn_res().
v6:
http://lkml.kernel.org/r/20180704041038.8190-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
v5:
http://lkml.kernel.org/r/20180612032831.29747-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
v4:
http://lkml.kernel.org/r/20180507063224.24229-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
v3:
http://lkml.kernel.org/r/20180419001848.3041-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
v2:
http://lkml.kernel.org/r/20180408024724.16812-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
v1:
http://lkml.kernel.org/r/20180322033722.9279-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Changelog:v6->v7:
Fix code bugs that test robot reported on mips and ia64.
Add error code description in reparent_resources() according to Andy's comment, and fix minor log typo.v5->v6:
Fix code style problems in reparent_resources() and use existing error codes, according to Andy's suggestion.
Fix bugs test robot reported.
v4->v5:
Add new patch 0001 to move duplicated reparent_resources() to kernel/resource.c to make it be shared by different ARCH-es.
Fix several code bugs reported by test robot on ARCH powerpc and microblaze.v3->v4:
Fix several bugs test robot reported. Rewrite cover letter and patch log according to reviewer's comment.
v2->v3:
Rename resource functions first_child() and sibling() to resource_first_chils() and resource_sibling(). Dan suggested this.
Move resource_first_chils() and resource_sibling() to linux/ioport.h and make them as inline function. Rob suggested this. Accordingly add linux/list.h including in linux/ioport.h, please help review if this bring efficiency degradation or code redundancy.
The change on struct resource {} bring two pointers of size increase, mention this in git log to make it more specifically, Rob suggested this.
v1->v2:
Use list_head instead to link resource siblings. This is suggested by Andrew.
Rewrite walk_system_ram_res_rev() after list_head is taken to link resouce siblings.
Baoquan He (4):
resource: Move reparent_resources() to kernel/resource.c and make it
public
resource: Use list_head to link sibling resource
resource: add walk_system_ram_res_rev()
kexec_file: Load kernel at top of system RAM if required
arch/arm/plat-samsung/pm-check.c | 6 +- arch/ia64/sn/kernel/io_init.c | 2 +- arch/microblaze/pci/pci-common.c | 41 +---- arch/mips/pci/pci-rc32434.c | 12 +- arch/powerpc/kernel/pci-common.c | 39 +--- arch/sparc/kernel/ioport.c | 2 +- arch/xtensa/include/asm/pci-bridge.h | 4 +- drivers/eisa/eisa-bus.c | 2 + drivers/gpu/drm/drm_memory.c | 3 +- drivers/gpu/drm/gma500/gtt.c | 5 +- drivers/hv/vmbus_drv.c | 52 +++--- drivers/input/joystick/iforce/iforce-main.c | 4 +- drivers/nvdimm/namespace_devs.c | 6 +- drivers/nvdimm/nd.h | 5 +- drivers/of/address.c | 4 +- drivers/parisc/lba_pci.c | 4 +- drivers/pci/controller/vmd.c | 8 +- drivers/pci/probe.c | 2 + drivers/pci/setup-bus.c | 2 +- include/linux/ioport.h | 21 ++- kernel/kexec_file.c | 2 + kernel/resource.c | 266 ++++++++++++++++++---------- 22 files changed, 260 insertions(+), 232 deletions(-)
Signature · 1 line
2.13.6
PATCH v7 1/4 resource: Move reparent_resources() to kernel/resource.c and make it public
reparent_resources() is duplicated in arch/microblaze/pci/pci-common.c and arch/powerpc/kernel/pci-common.c, so move it to kernel/resource.c so that it's shared.
--- arch/microblaze/pci/pci-common.c | 37 ----------------------------------- arch/powerpc/kernel/pci-common.c | 35 --------------------------------- include/linux/ioport.h | 1 + kernel/resource.c | 42 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 43 insertions(+), 72 deletions(-)
arch/microblaze/pci/pci-common.c +0 -37
@@ -619,43 +619,6 @@ int pcibios_add_device(struct pci_dev *dev)EXPORT_SYMBOL (pcibios_add_device);/* -* Reparent resource children of pr that conflict with res -* under res, and make res replace those children. -*/ -static int __init reparent_resources (struct resource *parent, -struct resource *res) -{ -struct resource *p, **pp; -struct resource **firstpp =NULL ; - -for (pp = &parent->child; (p = *pp) !=NULL ; pp = &p->sibling) { -if (p->end < res->start) -continue ; -if (res->end < p->start) -break ; -if (p->start < res->start || p->end > res->end) -return -1 ;/* not completely contained */ -if (firstpp ==NULL ) - firstpp = pp; - } -if (firstpp ==NULL ) -return -1 ;/* didn't find any conflicting entries? */ - res->parent = parent; - res->child = *firstpp; - res->sibling = *pp; - *firstpp = res; - *pp =NULL ; -for (p = res->child; p !=NULL ; p = p->sibling) { - p->parent = res; -pr_debug ("PCI: Reparented %s [%llx..%llx] under %s\n" , - p->name, - (unsigned long long )p->start, - (unsigned long long )p->end, res->name); - } -return 0 ; -} - -/* * Handle resources of PCI devices. If the world were perfect, we could * just allocate all the resource regions and do nothing more. It isn't. * On the other hand, we cannot just re-allocate all devices, as it would
arch/powerpc/kernel/pci-common.c +0 -35
@@ -1088,41 +1088,6 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,EXPORT_SYMBOL (pcibios_align_resource);/* -* Reparent resource children of pr that conflict with res -* under res, and make res replace those children. -*/ -static int reparent_resources (struct resource *parent, -struct resource *res) -{ -struct resource *p, **pp; -struct resource **firstpp =NULL ; - -for (pp = &parent->child; (p = *pp) !=NULL ; pp = &p->sibling) { -if (p->end < res->start) -continue ; -if (res->end < p->start) -break ; -if (p->start < res->start || p->end > res->end) -return -1 ;/* not completely contained */ -if (firstpp ==NULL ) - firstpp = pp; - } -if (firstpp ==NULL ) -return -1 ;/* didn't find any conflicting entries? */ - res->parent = parent; - res->child = *firstpp; - res->sibling = *pp; - *firstpp = res; - *pp =NULL ; -for (p = res->child; p !=NULL ; p = p->sibling) { - p->parent = res; -pr_debug ("PCI: Reparented %s %pR under %s\n" , - p->name, p, res->name); - } -return 0 ; -} - -/* * Handle resources of PCI devices. If the world were perfect, we could * just allocate all the resource regions and do nothing more. It isn't. * On the other hand, we cannot just re-allocate all devices, as it would
include/linux/ioport.h +1 -0
@@ -192,6 +192,7 @@ extern int allocate_resource(struct resource *root, struct resource *new,struct resource *lookup_resource (struct resource *root,resource_size_t start);int adjust_resource (struct resource *res,resource_size_t start,resource_size_t size); +int reparent_resources (struct resource *parent,struct resource *res);resource_size_t resource_alignment (struct resource *res);static inline resource_size_t resource_size (const struct resource *res) {
kernel/resource.c +42 -0
48 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [PATCH v7 1/4] resource: Move reparent_resources() to kernel/resource.c and make it public
Quoted 3 lines · Baoquan He
reparent_resources()is duplicated inarch/microblaze/pci/pci-common.candarch/powerpc/kernel/pci-common.c, so move it tokernel/resource.cso that it's shared.
Some minor stuff.
Quoted 7 lines
+/**
+ *reparent_resources- reparent resource children of parent that res covers
+ * @parent: parent resource descriptor
+ * @res: resource descriptor desired by caller
+ *
+ * Returns 0 on success,-ENOTSUPPif child resource is not completely
+ * contained by 'res',-ECANCELEDif no any conflicting entry found.
'res' -> @res
Quoted 2 lines
+ *
+ * Reparent resource children of 'parent' that conflict with 'res'
Ditto + 'parent' -> @parent
Quoted 1 line
+ * under 'res', and make 'res' replace those children.
Ditto.
Quoted 18 lines
+ */
kernel/resource.c +17
-0+int reparent_resources (struct resource *parent,struct resource *res) +{ +struct resource *p, **pp; +struct resource **firstpp =NULL ; + +for (pp = &parent->child; (p = *pp) !=NULL ; pp = &p->sibling) { +if (p->end < res->start) +continue ; +if (res->end < p->start) +break ; +if (p->start < res->start || p->end > res->end) +return -ENOTSUPP ;/* not completely contained */ +if (firstpp ==NULL ) + firstpp = pp; + } +if (firstpp ==NULL ) +return -ECANCELED ;/* didn't find any conflicting entries? */
kernel/resource.c int adjust_resource(struct resource *res, resource_size_t start, · line 1022
Quoted 10 lines
kernel/resource.c +9
-0+ res->parent = parent; + res->child = *firstpp; + res->sibling = *pp; + *firstpp = res; + *pp =NULL ; +for (p = res->child; p !=NULL ; p = p->sibling) { + p->parent = res; +pr_debug ("PCI: Reparented %s %pR under %s\n" , + p->name, p, res->name);
Now, PCI is a bit confusing here.
Quoted 11 lines
kernel/resource.c +5
-1+ } +return 0 ; +} +EXPORT_SYMBOL (reparent_resources); +static void __init __reserve_region_with_split (struct resource *root,resource_size_t start,resource_size_t end,const char *name) --2.13.6
Signature · 2 lines
With Best Regards, Andy Shevchenko
A message that is not in the archive (sent off-list or not yet received): CAHp75VdO88ydJQ9GHdaDUmAmzL6QHR=US6JiXZ1R_EEA-xWR1Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v7 1/4] resource: Move reparent_resources() to kernel/resource.c and make it public
Quoted 7 lines · Andy Shevchenko
Quoted 6 lines · Baoquan He
reparent_resources()is duplicated inarch/microblaze/pci/pci-common.candarch/powerpc/kernel/pci-common.c, so move it tokernel/resource.cso that it's shared.+ * Returns 0 on success,
-ENOTSUPPif child resource is not completely
+ * contained by 'res',-ECANCELEDif no any conflicting entry found.
You also can refer to constants by prefixing them with %, e.g. %-ENOTSUPP. But this is up to you completely.
Signature · 2 lines
With Best Regards, Andy Shevchenko
A message that is not in the archive (sent off-list or not yet received): CAHp75Vf2yEwHhEhhQH2XN+pOQ=-skiAHZ=FgLnfVV8vcm59qeQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH v7 1/4] resource: Move reparent_resources() to kernel/resource.c and make it public
Quoted 7 lines · Andy Shevchenko
Quoted 4 lines · Andy Shevchenko
Quoted 3 lines · Baoquan He
reparent_resources()is duplicated inarch/microblaze/pci/pci-common.candarch/powerpc/kernel/pci-common.c, so move it tokernel/resource.cso that it's shared.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
+ * Returns 0 on success,
-ENOTSUPPif child resource is not completely
+ * contained by 'res',-ECANCELEDif no any conflicting entry found.You also can refer to constants by prefixing them with %, e.g. %-ENOTSUPP. But this is up to you completely.
Thanks, will fix when repost.
PATCH v7 2/4 resource: Use list_head to link sibling resource
In reply to Baoquan HeThe struct resource uses singly linked list to link siblings, implemented by pointer operation. Replace it with list_head for better code readability.
Based on this list_head replacement, it will be very easy to do reverse iteration on iomem_resource's sibling list in later patch.
Besides, type of member variables of struct resource, sibling and child, are changed from 'struct resource *' to 'struct list_head'. This brings two pointers of size increase.
--- arch/arm/plat-samsung/pm-check.c | 6 +- arch/ia64/sn/kernel/io_init.c | 2 +- arch/microblaze/pci/pci-common.c | 4 +- arch/mips/pci/pci-rc32434.c | 12 +- arch/powerpc/kernel/pci-common.c | 4 +- arch/sparc/kernel/ioport.c | 2 +- arch/xtensa/include/asm/pci-bridge.h | 4 +- drivers/eisa/eisa-bus.c | 2 + drivers/gpu/drm/drm_memory.c | 3 +- drivers/gpu/drm/gma500/gtt.c | 5 +- drivers/hv/vmbus_drv.c | 52 +++---- drivers/input/joystick/iforce/iforce-main.c | 4 +- drivers/nvdimm/namespace_devs.c | 6 +- drivers/nvdimm/nd.h | 5 +- drivers/of/address.c | 4 +- drivers/parisc/lba_pci.c | 4 +- drivers/pci/controller/vmd.c | 8 +- drivers/pci/probe.c | 2 + drivers/pci/setup-bus.c | 2 +- include/linux/ioport.h | 17 ++- kernel/resource.c | 206 ++++++++++++++-------------- 21 files changed, 183 insertions(+), 171 deletions(-)
arch/arm/plat-samsung/pm-check.c +3 -3
@@ -46,8 +46,8 @@ typedef u32 *(run_fn_t)(struct resource *ptr, u32 *arg);static void s3c_pm_run_res (struct resource *ptr,run_fn_t fn,u32 *arg) {while (ptr !=NULL ) { -if (ptr->child !=NULL ) -s3c_pm_run_res (ptr->child, fn, arg); +if (!list_empty (&ptr->child)) +s3c_pm_run_res (resource_first_child (&ptr->child), fn, arg);if ((ptr->flags &IORESOURCE_SYSTEM_RAM ) ==IORESOURCE_SYSTEM_RAM ) {
@@ -57,7 +57,7 @@ static void s3c_pm_run_res(struct resource *ptr, run_fn_t fn, u32 *arg) arg = (fn)(ptr, arg); } - ptr = ptr->sibling; + ptr =resource_sibling (ptr); } }
arch/ia64/sn/kernel/io_init.c +1 -1
@@ -192,7 +192,7 @@ sn_io_slot_fixup(struct pci_dev *dev)* if it's already in the device structure, remove it before * inserting */ -if (res->parent && res->parent->child) +if (res->parent && !list_empty (&res->parent->child))release_resource (res);if (res->flags &IORESOURCE_IO )
arch/microblaze/pci/pci-common.c +3 -1
@@ -533,7 +533,9 @@ void pci_process_bridge_OF_ranges(struct pci_controller *hose, res->flags = range.flags; res->start = range.cpu_addr; res->end = range.cpu_addr + range.size -1 ; - res->parent = res->child = res->sibling =NULL ; + res->parent =NULL ; +INIT_LIST_HEAD (&res->child); +INIT_LIST_HEAD (&res->sibling); } }
arch/mips/pci/pci-rc32434.c +8 -4
@@ -53,8 +53,8 @@ static struct resource rc32434_res_pci_mem1 = { .start = 0x50000000, .end = 0x5FFFFFFF, .flags = IORESOURCE_MEM, - .sibling = NULL, - .child = &rc32434_res_pci_mem2 + .sibling = LIST_HEAD_INIT(rc32434_res_pci_mem1.sibling), + .child = LIST_HEAD_INIT(rc32434_res_pci_mem1.child), };static struct resource rc32434_res_pci_mem2 = {
@@ -63,8 +63,8 @@ static struct resource rc32434_res_pci_mem2 = { .end = 0x6FFFFFFF, .flags = IORESOURCE_MEM, .parent = &rc32434_res_pci_mem1, - .sibling = NULL, - .child = NULL + .sibling = LIST_HEAD_INIT(rc32434_res_pci_mem2.sibling), + .child = LIST_HEAD_INIT(rc32434_res_pci_mem2.child), };static struct resource rc32434_res_pci_io1 = {
@@ -72,6 +72,8 @@ static struct resource rc32434_res_pci_io1 = { .start = 0x18800000, .end = 0x188FFFFF, .flags = IORESOURCE_IO, + .sibling = LIST_HEAD_INIT(rc32434_res_pci_io1.sibling), + .child = LIST_HEAD_INIT(rc32434_res_pci_io1.child), };extern struct pci_ops rc32434_pci_ops;
@@ -208,6 +210,8 @@ static int __init rc32434_pci_init(void)pr_info ("PCI: Initializing PCI\n" ); +list_add (&rc32434_res_pci_mem2.sibling, &rc32434_res_pci_mem1.child); + ioport_resource.start = rc32434_res_pci_io1.start; ioport_resource.end = rc32434_res_pci_io1.end;
arch/powerpc/kernel/pci-common.c +3 -1
@@ -761,7 +761,9 @@ void pci_process_bridge_OF_ranges(struct pci_controller *hose, res->flags = range.flags; res->start = range.cpu_addr; res->end = range.cpu_addr + range.size -1 ; - res->parent = res->child = res->sibling =NULL ; + res->parent =NULL ; +INIT_LIST_HEAD (&res->child); +INIT_LIST_HEAD (&res->sibling); } } }
arch/sparc/kernel/ioport.c +1 -1
@@ -669,7 +669,7 @@ static int sparc_io_proc_show(struct seq_file *m, void *v)struct resource *root = m->private, *r;const char *nm; -for (r = root->child; r !=NULL ; r = r->sibling) { +list_for_each_entry (r, &root->child, sibling) {if ((nm = r->name) ==NULL ) nm ="???" ; seq_printf(m, "%016llx-%016llx: %s\n", (unsigned long long)r->start,
arch/xtensa/include/asm/pci-bridge.h +2 -2
@@ -71,8 +71,8 @@ static inline void pcibios_init_resource(struct resource *res, res->flags = flags; res->name = name; res->parent =NULL ; - res->sibling =NULL ; - res->child =NULL ; +INIT_LIST_HEAD (&res->child); +INIT_LIST_HEAD (&res->sibling); }
drivers/eisa/eisa-bus.c +2 -0
@@ -408,6 +408,8 @@ static struct resource eisa_root_res = { .start = 0, .end = 0xffffffff, .flags = IORESOURCE_IO, + .sibling = LIST_HEAD_INIT(eisa_root_res.sibling), + .child = LIST_HEAD_INIT(eisa_root_res.child), };static int eisa_bus_count;
drivers/gpu/drm/drm_memory.c +1 -2
@@ -155,9 +155,8 @@ u64 drm_get_max_iomem(void)struct resource *tmp;resource_size_t max_iomem =0 ; -for (tmp = iomem_resource.child; tmp; tmp = tmp->sibling) { +list_for_each_entry (tmp, &iomem_resource.child, sibling) max_iomem =max (max_iomem, tmp->end); - }return max_iomem; }
drivers/gpu/drm/gma500/gtt.c +2 -3
@@ -565,7 +565,7 @@ int psb_gtt_init(struct drm_device *dev, int resume)int psb_gtt_restore (struct drm_device *dev) {struct drm_psb_private *dev_priv = dev->dev_private; -struct resource *r = dev_priv->gtt_mem->child; +struct resource *r;struct gtt_range *range;unsigned int restored =0 , total =0 , size =0 ;
@@ -573,14 +573,13 @@ int psb_gtt_restore(struct drm_device *dev)mutex_lock (&dev_priv->gtt_mutex);psb_gtt_init (dev,1 ); -while (r !=NULL ) { +list_for_each_entry (r, &dev_priv->gtt_mem->child, sibling) { range = container_of(r, struct gtt_range, resource);if (range->pages) {psb_gtt_insert (dev, range,1 ); size += range->resource.end - range->resource.start; restored++; } - r = r->sibling; total++; }mutex_unlock (&dev_priv->gtt_mutex);
drivers/hv/vmbus_drv.c +22 -30
99 lines not on this page · raw message
drivers/input/joystick/iforce/iforce-main.c +2 -2
@@ -305,8 +305,8 @@ int iforce_init_device(struct iforce *iforce) iforce->device_memory.end =200 ; iforce->device_memory.flags =IORESOURCE_MEM ; iforce->device_memory.parent =NULL ; - iforce->device_memory.child =NULL ; - iforce->device_memory.sibling =NULL ; +INIT_LIST_HEAD (&iforce->device_memory.child); +INIT_LIST_HEAD (&iforce->device_memory.sibling);/* * Wait until device ready - until it sends its first response.
drivers/nvdimm/namespace_devs.c +3 -3
@@ -637,7 +637,7 @@ static resource_size_t scan_allocate(struct nd_region *nd_region, retry: first =0 ;for_each_dpa_resource (ndd, res) { -struct resource *next = res->sibling, *new_res =NULL ; +struct resource *next =resource_sibling (res), *new_res =NULL ;resource_size_t allocate, available =0 ;enum alloc_loc loc =ALLOC_ERR ;const char *action;
@@ -763,7 +763,7 @@ static resource_size_t scan_allocate(struct nd_region *nd_region,* an initial "pmem-reserve pass". Only do an initial BLK allocation * when none of the DPA space is reserved. */ -if ((is_pmem || !ndd->dpa.child) && n == to_allocate) +if ((is_pmem ||list_empty (&ndd->dpa.child)) && n == to_allocate)return init_dpa_allocation (label_id, nd_region, nd_mapping, n);return n; }
@@ -779,7 +779,7 @@ static int merge_dpa(struct nd_region *nd_region, retry:for_each_dpa_resource (ndd, res) {int rc; -struct resource *next = res->sibling; +struct resource *next =resource_sibling (res);resource_size_t end = res->start +resource_size (res); if (!next || strcmp(res->name, label_id->id) != 0
drivers/nvdimm/nd.h +2 -3
@@ -102,11 +102,10 @@ unsigned sizeof_namespace_label(struct nvdimm_drvdata *ndd); (unsigned long long) (res ? res->start : 0), ##arg)#define for_each_dpa_resource (ndd, res) \ - for (res = (ndd)->dpa.child; res; res = res->sibling) + list_for_each_entry(res, &(ndd)->dpa.child, sibling)#define for_each_dpa_resource_safe (ndd, res, next) \ - for (res = (ndd)->dpa.child, next = res ? res->sibling : NULL; \ - res; res = next, next = next ? next->sibling : NULL) + list_for_each_entry_safe(res, next, &(ndd)->dpa.child, sibling)struct nd_percpu_lane {int count;
drivers/of/address.c +3 -1
@@ -330,7 +330,9 @@ int of_pci_range_to_resource(struct of_pci_range *range, {int err; res->flags = range->flags; - res->parent = res->child = res->sibling =NULL ; + res->parent =NULL ; +INIT_LIST_HEAD (&res->child); +INIT_LIST_HEAD (&res->sibling); res->name = np->full_name;if (res->flags &IORESOURCE_IO ) {
drivers/parisc/lba_pci.c +2 -2
@@ -170,8 +170,8 @@ lba_dump_res(struct resource *r, int d)for (i = d; i ; --i)printk (" " );printk (KERN_DEBUG "%p [%lx,%lx]/%lx\n" , r, (long )r->start, (long )r->end, r->flags); -lba_dump_res (r->child, d+2 ); -lba_dump_res (r->sibling, d); +lba_dump_res (resource_first_child (&r->child), d+2 ); +lba_dump_res (resource_sibling (r), d); }
drivers/pci/controller/vmd.c +4 -4
@@ -542,14 +542,14 @@ static struct pci_ops vmd_ops = {static void vmd_attach_resources (struct vmd_dev *vmd) { - vmd->dev->resource[VMD_MEMBAR1 ].child = &vmd->resources[1 ]; - vmd->dev->resource[VMD_MEMBAR2 ].child = &vmd->resources[2 ]; +list_add (&vmd->resources[1 ].sibling, &vmd->dev->resource[VMD_MEMBAR1 ].child); +list_add (&vmd->resources[2 ].sibling, &vmd->dev->resource[VMD_MEMBAR2 ].child); }static void vmd_detach_resources (struct vmd_dev *vmd) { - vmd->dev->resource[VMD_MEMBAR1 ].child =NULL ; - vmd->dev->resource[VMD_MEMBAR2 ].child =NULL ; +INIT_LIST_HEAD (&vmd->dev->resource[VMD_MEMBAR1 ].child); +INIT_LIST_HEAD (&vmd->dev->resource[VMD_MEMBAR2 ].child); }/*
drivers/pci/probe.c +2 -0
@@ -59,6 +59,8 @@ static struct resource *get_pci_domain_busn_res(int domain_nr) r->res.start =0 ; r->res.end =0xff ; r->res.flags =IORESOURCE_BUS |IORESOURCE_PCI_FIXED ; +INIT_LIST_HEAD (&r->res.child); +INIT_LIST_HEAD (&r->res.sibling);list_add_tail (&r->list, &pci_domain_busn_res_list);
drivers/pci/setup-bus.c +1 -1
@@ -2107,7 +2107,7 @@ int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)continue ;/* Ignore BARs which are still in use */ -if (res->child) +if (!list_empty (&res->child))continue ; ret =add_to_list (&saved, bridge, res,0 ,0 );
include/linux/ioport.h +15 -2
@@ -12,6 +12,7 @@#ifndef __ASSEMBLY__#include <linux/compiler.h> #include <linux/types.h> +#include <linux/list.h> /* * Resources are tree-like, allowing * nesting etc..
@@ -22,7 +23,8 @@ struct resource {const char *name;unsigned long flags;unsigned long desc; -struct resource *parent, *sibling, *child; +struct list_head child, sibling; +struct resource *parent; };/*
@@ -216,7 +218,6 @@ static inline bool resource_contains(struct resource *r1, struct resource *r2)return r1->start <= r2->start && r1->end >= r2->end; } -/* Convenience shorthand with allocation */ #define request_region (start,n,name) __request_region(&ioport_resource, (start), (n), (name), 0)#define request_muxed_region (start,n,name) __request_region(&ioport_resource, (start), (n), (name), IORESOURCE_MUXED)
@@ -287,6 +288,18 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)return (r1->start <= r2->end && r1->end >= r2->start); } +static inline struct resource *resource_sibling (struct resource *res) +{ +if (res->parent && !list_is_last (&res->sibling, &res->parent->child)) +return list_next_entry (res, sibling); +return NULL ; +} + +static inline struct resource *resource_first_child (struct list_head *head) +{ + return list_first_entry_or_null(head, struct resource, sibling); +} +#endif /* __ASSEMBLY__ */ #endif /* _LINUX_IOPORT_H */
kernel/resource.c +101 -105
476 lines not on this page · raw message
Signature · 6 lines
2.13.6 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
PATCH v7 3/4 resource: add walk_system_ram_res_rev()
In reply to Baoquan HeThis function, being a variant of walk_system_ram_res() introduced in commit 8c86e70acead ("resource: provide new functions to walk through resources"), walks through a list of all the resources of System RAM in reversed order, i.e., from higher to lower.
It will be used in kexec_file code.
--- include/linux/ioport.h | 3 +++ kernel/resource.c | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+)
include/linux/ioport.h +3 -0
@@ -279,6 +279,9 @@ extern int walk_system_ram_res(u64 start, u64 end, void *arg, int (*func)(struct resource *, void *));extern int +walk_system_ram_res_rev (u64 start,u64 end,void *arg, +int (*func)(struct resource *,void *)); +extern int walk_iomem_res_desc (unsigned long desc,unsigned long flags,u64 start,u64 end,void *arg,int (*func)(struct resource *,void *));
kernel/resource.c +40 -0
@@ -23,6 +23,8 @@#include <linux/pfn.h> #include <linux/mm.h> #include <linux/resource_ext.h> +#include <linux/string.h> +#include <linux/vmalloc.h> #include <asm/io.h>
@@ -443,6 +445,44 @@ int walk_system_ram_res(u64 start, u64 end, void *arg, }/* +* This function, being a variant of walk_system_ram_res(), calls the @func +* callback against all memory ranges of type System RAM which are marked as +* IORESOURCE_SYSTEM_RAM and IORESOUCE_BUSY in reversed order, i.e., from +* higher to lower. +*/ +int walk_system_ram_res_rev (u64 start,u64 end,void *arg, +int (*func)(struct resource *,void *)) +{ +unsigned long flags; +struct resource *res; +int ret =-1 ; + + flags =IORESOURCE_SYSTEM_RAM |IORESOURCE_BUSY ; + +read_lock (&resource_lock); +list_for_each_entry_reverse (res, &iomem_resource.child, sibling) { +if (start >= end) +break ; +if ((res->flags & flags) != flags) +continue ; +if (res->desc !=IORES_DESC_NONE ) +continue ; +if (res->end < start) +break ; + +if ((res->end >= start) && (res->start < end)) { + ret = (*func)(res, arg); +if (ret) +break ; + } + end = res->start -1 ; + + } +read_unlock (&resource_lock); +return ret; +} + +/* * This function calls the @func callback against all memory ranges, which * are ranges marked as IORESOURCE_MEM and IORESOUCE_BUSY. */
Signature · 6 lines
2.13.6 _______________________________________________ Linux-nvdimm mailing list [email protected] https://lists.01.org/mailman/listinfo/linux-nvdimm
PATCH v7 4/4 kexec_file: Load kernel at top of system RAM if required
In reply to Baoquan HeFor kexec_file loading, if kexec_buf.top_down is 'true', the memory which is used to load kernel/initrd/purgatory is supposed to be allocated from top to down. This is what we have been doing all along in the old kexec loading interface and the kexec loading is still default setting in some distributions. However, the current kexec_file loading interface doesn't do like this. The function arch_kexec_walk_mem() it calls ignores checking kexec_buf.top_down, but calls walk_system_ram_res() directly to go through all resources of System RAM from bottom to up, to try to find memory region which can contain the specific kexec buffer, then call locate_mem_hole_callback() to allocate memory in that found memory region from top to down. This brings confusion especially when KASLR is widely supported , users have to make clear why kexec/kdump kernel loading position is different between these two interfaces in order to exclude unnecessary noises. Hence these two interfaces need be unified on behaviour.
Here add checking if kexec_buf.top_down is 'true' in arch_kexec_walk_mem(), if yes, call the newly added walk_system_ram_res_rev() to find memory region from top to down to load kernel.
--- kernel/kexec_file.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/kexec_file.c +2 -0
@@ -518,6 +518,8 @@ int __weak arch_kexec_walk_mem(struct kexec_buf *kbuf,IORESOURCE_SYSTEM_RAM |IORESOURCE_BUSY , crashk_res.start, crashk_res.end, kbuf, func); + else if (kbuf->top_down) + return walk_system_ram_res_rev(0, ULONG_MAX, kbuf, func); else return walk_system_ram_res(0, ULONG_MAX, kbuf, func); }
Signature · 1 line
2.13.6
Andrew Morton
maintainerRe: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 6 lines · On Wed, 18 Jul 2018 10:49:44 +0800 Baoquan He <[email protected]> wrote:
For
kexec_fileloading, if kexec_buf.top_down is 'true', the memory which is used to loadkernel/initrd/purgatoryis supposed to be allocated from top to down. This is what we have been doing all along in the old kexec loading interface and the kexec loading is still default setting in some distributions. However, the currentkexec_fileloading interface doesn't do like this. The functionarch_kexec_walk_mem()it calls ignores checking
Quoted 8 lines
kexec_buf.top_down, but calls
walk_system_ram_res()directly to go through all resources of System RAM from bottom to up, to try to find memory region which can contain the specific kexec buffer, then calllocate_mem_hole_callback()to allocate memory in that found memory region from top to down. This brings confusion especially when KASLR is widely supported , users have to make clear why kexec/kdump kernel loading position is different between these two interfaces in order to exclude unnecessary noises. Hence these two interfaces need be unified on behaviour.
As far as I can tell, the above is the whole reason for the patchset, yes? To avoid confusing users.
Is that sufficient? Can we instead simplify their lives by providing better documentation or informative printks or better Kconfig text, etc?
And who *are* the people who are performing this configuration? Random system administrators? Linux distro engineers? If the latter then they presumably aren't easily confused!
In other words, I'm trying to understand how much benefit this patchset will provide to our users as a whole.
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Hi Andrew,
Quoted 17 lines · Andrew Morton
Quoted 6 lines · On Wed, 18 Jul 2018 10:49:44 +0800 Baoquan He…
For
kexec_fileloading, if kexec_buf.top_down is 'true', the memory which is used to loadkernel/initrd/purgatoryis supposed to be allocated from top to down. This is what we have been doing all along in the old kexec loading interface and the kexec loading is still default setting in some distributions. However, the currentkexec_fileloading interface doesn't do like this. The functionarch_kexec_walk_mem()it calls ignores checkingQuoted 8 lines
kexec_buf.top_down, but calls
walk_system_ram_res()directly to go through all resources of System RAM from bottom to up, to try to find memory region which can contain the specific kexec buffer, then calllocate_mem_hole_callback()to allocate memory in that found memory region from top to down. This brings confusion especially when KASLR is widely supported , users have to make clear why kexec/kdump kernel loading position is different between these two interfaces in order to exclude unnecessary noises. Hence these two interfaces need be unified on behaviour.
Quoted 2 lines
As far as I can tell, the above is the whole reason for the patchset, yes? To avoid confusing users.
In fact, it's not just trying to avoid confusing users. Kexec loading and kexec_file loading are just do the same thing in essence. Just we need do kernel image verification on uefi system, have to port kexec loading code to kernel.
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the kexec_file loading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.
And at the first post, I just posted below with AKASHI's walk_system_ram_res_rev() version. Later you suggested to use list_head to link child sibling of resource, see what the code change looks like.
http://lkml.kernel.org/r/20180322033722.9279-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Then I posted v2
http://lkml.kernel.org/r/20180408024724.16812-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rob Herring mentioned that other components which has this tree struct have planned to do the same thing, replacing the singly linked list with list_head to link resource child sibling. Just quote Rob's words as below. I think this could be another reason.
~~~~~ From Rob
The DT struct device_node also has the same tree structure with parent, child, sibling pointers and converting to list_head had been on the todo list for a while. ACPI also has some tree walking functions (drivers/acpi/acpica/pstree.c). Perhaps there should be a common tree struct and helpers defined either on top of list_head or a
~~~~~
new struct if that saves some size.
Quoted 8 lines
Is that sufficient? Can we instead simplify their lives by providing better documentation or informative printks or better
Kconfigtext, etc?And who *are* the people who are performing this configuration? Random system administrators? Linux distro engineers? If the latter then they presumably aren't easily confused!
Kexec was invented for kernel developer to speed up their kernel rebooting. Now high end sever admin, kernel developer and QE are also keen to use it to reboot large box for faster feature testing, bug debugging. Kernel dev could know this well, about kernel loading position, admin or QE might not be aware of it very well.
Quoted 3 lines
In other words, I'm trying to understand how much benefit this patchset will provide to our users as a whole.
Understood. The list_head replacing patch truly involes too many code changes, it's risky. I am willing to try any idea from reviewers, won't persuit they have to be accepted finally. If don't have a try, we don't know what it looks like, and what impact it may have. I am fine to take AKASHI's simple version of walk_system_ram_res_rev() to lower risk, even though it could be a little bit low efficient.
Thanks
Baoquan
Andrew Morton
maintainerRe: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 51 lines · On Thu, 19 Jul 2018 23:17:53 +0800 Baoquan He…
Hi Andrew,
Quoted 17 lines · Andrew Morton
Quoted 6 lines · On Wed, 18 Jul 2018 10:49:44 +0800 Baoquan He…
For
kexec_fileloading, if kexec_buf.top_down is 'true', the memory which is used to loadkernel/initrd/purgatoryis supposed to be allocated from top to down. This is what we have been doing all along in the old kexec loading interface and the kexec loading is still default setting in some distributions. However, the currentkexec_fileloading interface doesn't do like this. The functionarch_kexec_walk_mem()it calls ignores checkingQuoted 8 lines
kexec_buf.top_down, but calls
walk_system_ram_res()directly to go through all resources of System RAM from bottom to up, to try to find memory region which can contain the specific kexec buffer, then calllocate_mem_hole_callback()to allocate memory in that found memory region from top to down. This brings confusion especially when KASLR is widely supported , users have to make clear why kexec/kdump kernel loading position is different between these two interfaces in order to exclude unnecessary noises. Hence these two interfaces need be unified on behaviour.Quoted 2 lines
As far as I can tell, the above is the whole reason for the patchset, yes? To avoid confusing users.
In fact, it's not just trying to avoid confusing users. Kexec loading and
kexec_fileloading are just do the same thing in essence. Just we need do kernel image verification on uefi system, have to port kexec loading code to kernel.Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.And at the first post, I just posted below with AKASHI's
walk_system_ram_res_rev()version. Later you suggested to uselist_headto link child sibling of resource, see what the code change looks like.
http://lkml.kernel.org/r/20180322033722.9279-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.orgThen I posted v2
http://lkml.kernel.org/r/20180408024724.16812-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rob Herring mentioned that other components which has this tree struct have planned to do the same thing, replacing the singly linked list withlist_headto link resource child sibling. Just quote Rob's words as below. I think this could be another reason.
Quoted 8 lines
~~~~~ From Rob
The DT structdevice_nodealso has the same tree structure with parent, child, sibling pointers and converting tolist_headhad been on the todo list for a while. ACPI also has some tree walking functions (drivers/acpi/acpica/pstree.c). Perhaps there should be a common tree struct and helpers defined either on top oflist_heador a
~~~~~
new struct if that saves some size.
Please let's get all this into the changelogs?
Quoted 19 lines
Quoted 8 lines
Is that sufficient? Can we instead simplify their lives by providing better documentation or informative printks or better
Kconfigtext, etc?And who *are* the people who are performing this configuration? Random system administrators? Linux distro engineers? If the latter then they presumably aren't easily confused!
Kexec was invented for kernel developer to speed up their kernel rebooting. Now high end sever admin, kernel developer and QE are also keen to use it to reboot large box for faster feature testing, bug debugging. Kernel dev could know this well, about kernel loading position, admin or QE might not be aware of it very well.
Quoted 3 lines
In other words, I'm trying to understand how much benefit this patchset will provide to our users as a whole.
Quoted 6 lines
Understood. The
list_headreplacing patch truly involes too many code changes, it's risky. I am willing to try any idea from reviewers, won't persuit they have to be accepted finally. If don't have a try, we don't know what it looks like, and what impact it may have. I am fine to take AKASHI's simple version ofwalk_system_ram_res_rev()to lower risk, even though it could be a little bit low efficient.
The larger patch produces a better result. We can handle it ;)
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Hi Andrew,
Quoted 41 lines · Andrew Morton
Quoted 31 lines · On Thu, 19 Jul 2018 23:17:53 +0800 Baoquan He…
Quoted 2 lines
As far as I can tell, the above is the whole reason for the patchset, yes? To avoid confusing users.
In fact, it's not just trying to avoid confusing users. Kexec loading and
kexec_fileloading are just do the same thing in essence. Just we need do kernel image verification on uefi system, have to port kexec loading code to kernel.Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.And at the first post, I just posted below with AKASHI's
walk_system_ram_res_rev()version. Later you suggested to uselist_headto link child sibling of resource, see what the code change looks like.
http://lkml.kernel.org/r/20180322033722.9279-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.orgThen I posted v2
http://lkml.kernel.org/r/20180408024724.16812-1-bhe-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rob Herring mentioned that other components which has this tree struct have planned to do the same thing, replacing the singly linked list withlist_headto link resource child sibling. Just quote Rob's words as below. I think this could be another reason.Quoted 8 lines
~~~~~ From Rob
The DT structdevice_nodealso has the same tree structure with parent, child, sibling pointers and converting tolist_headhad been on the todo list for a while. ACPI also has some tree walking functions (drivers/acpi/acpica/pstree.c). Perhaps there should be a common tree struct and helpers defined either on top oflist_heador a
~~~~~
new struct if that saves some size.
Quoted 1 line
Please let's get all this into the changelogs?
Sorry for late reply because of some urgent customer hotplug issues.
I am rewriting all change logs, and cover letter. Then found I was wrong about the 2nd reason. The current kexec_file_load calls kexec_locate_mem_hole() to go through all system RAM region, if one region is larger than the size of kernel or initrd, it will search a position in that region from top to down. Since kexec will jump to 2nd kernel and don't need to care the 1st kernel's data, we can always find a usable space to load kexec kernel/initrd under 4G.
So the only reason for this patch is keeping consistent with kexec_load and avoid confusion.
And since x86 5-level paging mode has been added, we have another issue for top-down searching in the whole system RAM. That is we support dynamic 4-level to 5-level changing. Namely a kernel compiled with 5-level support, we can add 'no5lvl' to force 4-level. Then jumping from a 5-level kernel to 4-level kernel, e.g we load kernel at the top of system RAM in 5-level paging mode which might be bigger than 64TB, then try to jump to 4-level kernel with the upper limit of 64TB. For this case, we need add limit for kexec kernel loading if in 5-level kernel.
All this mess makes me hesitate to choose a deligate method. Maybe I should drop this patchset.
Quoted 20 lines
Quoted 20 lines
Quoted 9 lines
Is that sufficient? Can we instead simplify their lives by providing better documentation or informative printks or better
Kconfigtext, etc?And who *are* the people who are performing this configuration? Random system administrators? Linux distro engineers? If the latter then they presumably aren't easily confused!
Kexec was invented for kernel developer to speed up their kernel rebooting. Now high end sever admin, kernel developer and QE are also keen to use it to reboot large box for faster feature testing, bug debugging. Kernel dev could know this well, about kernel loading position, admin or QE might not be aware of it very well.
Quoted 3 lines
In other words, I'm trying to understand how much benefit this patchset will provide to our users as a whole.
Quoted 8 lines
Quoted 6 lines
Understood. The
list_headreplacing patch truly involes too many code changes, it's risky. I am willing to try any idea from reviewers, won't persuit they have to be accepted finally. If don't have a try, we don't know what it looks like, and what impact it may have. I am fine to take AKASHI's simple version ofwalk_system_ram_res_rev()to lower risk, even though it could be a little bit low efficient.The larger patch produces a better result. We can handle it ;)
For this issue, if we stop changing the kexec top down searching code, I am not sure if we should post this replacing with list_head patches separately.
Thanks
Baoquan
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
In reply to Baoquan HeQuoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See memblock_find_in_range_node
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.
Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_node
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct kexec_segment, for the old kexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it calls kimage_load_normal_segment() to copy user space buffer to intermediate pages which are allocated with flag GFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding, GFP_KERNEL will make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.
#define KEXEC_CONTROL_MEMORY_GFP (GFP_KERNEL | __GFP_NORETRY)
struct kexec_segment {
/*
* This pointer can point to user memory if kexec_load() system
* call is used or will point to kernel memory if
* kexec_file_load() system call is used.
*
* Use ->buf when expecting to deal with user memory and use ->kbuf
* when expecting to deal with kernel memory.
*/
union {
void __user *buf;
void *kbuf;
};
size_t bufsz;
unsigned long mem;
size_t memsz;
};Thanks
Baoquan
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 16 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_node
Quoted 16 lines
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.
I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
The memblock code warns exactly for that reason.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 34 lines · Michal Hocko
Quoted 16 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_nodeQuoted 16 lines
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.
Quoted 4 lines
I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove in 1st kernel, it does impact the kernel which kexec jump into if kernel is at top of system RAM and the top RAM is in movable node.
Quoted 5 lines
The memblock code warns exactly for that reason.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 35 lines · Baoquan He
Quoted 34 lines · Michal Hocko
Quoted 32 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_nodeKexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.
Quoted 8 lines
Quoted 4 lines
I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove in 1st kernel, it does impact the kernel which kexec jump into if kernel is at top of system RAM and the top RAM is in movable node.
It will affect the 1st kernel (which does the memblock allocation top-down) as well. For reasons mentioned above.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 41 lines · Michal Hocko
Quoted 40 lines · Baoquan He
Quoted 38 lines · Michal Hocko
Quoted 16 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_nodeQuoted 16 lines
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
Quoted 6 lines
Quoted 3 lines
OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove in 1st kernel, it does impact the kernel which kexec jump into if kernel is at top of system RAM and the top RAM is in movable node.
It will affect the 1st kernel (which does the memblock allocation top-down) as well. For reasons mentioned above.
And btw. in the ideal world, we would restrict the memblock allocation top-down from the non-movable nodes. But I do not think we have that information ready at the time when the reservation is done.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 46 lines · Michal Hocko
Quoted 45 lines · Michal Hocko
Quoted 43 lines · Baoquan He
Quoted 34 lines · Michal Hocko
Quoted 16 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_nodeQuoted 16 lines
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.Quoted 4 lines
I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove in 1st kernel, it does impact the kernel which kexec jump into if kernel is at top of system RAM and the top RAM is in movable node.
Quoted 6 lines
Quoted 2 lines
It will affect the 1st kernel (which does the memblock allocation top-down) as well. For reasons mentioned above.
And btw. in the ideal world, we would restrict the memblock allocation top-down from the non-movable nodes. But I do not think we have that information ready at the time when the reservation is done.
Oh, you could mix kexec loading up with kdump kernel loading. For kdump kernel, we need reserve memory region during bootup with memblock allocator. For kexec loading, we just operate after system up, and do not need to reserve any memmory region. About memory used to load them, it's quite different way.
Thanks
Baoquan
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 54 lines · Baoquan He
Quoted 46 lines · Michal Hocko
Quoted 45 lines · Michal Hocko
Quoted 43 lines · Baoquan He
Quoted 34 lines · Michal Hocko
Quoted 16 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_nodeQuoted 16 lines
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.Quoted 4 lines
I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove in 1st kernel, it does impact the kernel which kexec jump into if kernel is at top of system RAM and the top RAM is in movable node.
Quoted 6 lines
Quoted 2 lines
It will affect the 1st kernel (which does the memblock allocation top-down) as well. For reasons mentioned above.
And btw. in the ideal world, we would restrict the memblock allocation top-down from the non-movable nodes. But I do not think we have that information ready at the time when the reservation is done.
Quoted 5 lines
Oh, you could mix kexec loading up with kdump kernel loading. For kdump kernel, we need reserve memory region during bootup with memblock allocator. For kexec loading, we just operate after system up, and do not need to reserve any memmory region. About memory used to load them, it's quite different way.
I didn't know about that. I thought both use the same underlying reservation mechanism. My bad and sorry for the noise.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v7 4/4] kexec_file: Load kernel at top of system RAM if required
Quoted 55 lines · Michal Hocko
Quoted 54 lines · Baoquan He
Quoted 52 lines · Michal Hocko
Quoted 41 lines · Michal Hocko
Quoted 40 lines · Baoquan He
Quoted 38 lines · Michal Hocko
Quoted 16 lines · Baoquan He
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Baoquan He
Kexec has been a formal feature in our distro, and customers owning those kind of very large machine can make use of this feature to speed up the reboot process. On uefi machine, the
kexec_fileloading will search place to put kernel under 4G from top to down. As we know, the 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume it. It may have possibility to not be able to find a usable space for kernel/initrd. From the top down of the whole memory space, we don't have this worry.Quoted 4 lines
I do not have the full context here but let me note that you should be careful when doing top-down reservation because you can easily get into hotplugable memory and break the hotremove usecase. We even warn when this is done. See
memblock_find_in_range_nodeQuoted 16 lines
Kexec read kernel/initrd file into buffer, just search usable positions for them to do the later copying. You can see below struct
kexec_segment, for the oldkexec_load, kernel/initrd are read into user space buffer, the @buf stores the user space buffer address, @mem stores the position where kernel/initrd will be put. In kernel, it callskimage_load_normal_segment()to copy user space buffer to intermediate pages which are allocated with flagGFP_KERNEL. These intermediate pages are recorded as entries, later when user execute "kexec -e" to trigger kexec jumping, it will do the final copying from the intermediate pages to the real destination pages which @mem pointed. Because we can't touch the existed data in 1st kernel when do kexec kernel loading. With my understanding,GFP_KERNELwill make those intermediate pages be allocated inside immovable area, it won't impact hotplugging. But the @mem we searched in the whole system RAM might be lost along with hotplug. Hence we need do kexec kernel again when hotplug event is detected.I am not sure I am following. If @mem is placed at movable node then the memory hotremove simply won't work, because we are seeing reserved pages and do not know what to do about them. They are not migrateable. Allocating intermediate pages from other nodes doesn't really help.
Quoted 6 lines
Quoted 3 lines
OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove in 1st kernel, it does impact the kernel which kexec jump into if kernel is at top of system RAM and the top RAM is in movable node.
It will affect the 1st kernel (which does the memblock allocation top-down) as well. For reasons mentioned above.
And btw. in the ideal world, we would restrict the memblock allocation top-down from the non-movable nodes. But I do not think we have that information ready at the time when the reservation is done.
Quoted 8 lines
Quoted 5 lines
Oh, you could mix kexec loading up with kdump kernel loading. For kdump kernel, we need reserve memory region during bootup with memblock allocator. For kexec loading, we just operate after system up, and do not need to reserve any memmory region. About memory used to load them, it's quite different way.
I didn't know about that. I thought both use the same underlying reservation mechanism. My bad and sorry for the noise.
Not at all. It's truly confusing. I often need take time to recall those details.