mm: introduce new field "managed_pages" to struct zone

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 5 lines · Simon Jeons

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Yeah, why bootmem doesn't touch highmem pages? The patch is buggy. :(

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Regards!
Gerry

Quoted 6 lines · Simon Jeons

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Quoted 4 lines

Regards!
Gerry

Quoted 7 lines · Simon Jeons

Quoted 6 lines

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 6 lines · Simon Jeons

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,

According to my understanding, bootmem allocator does only manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                         u64 goal, u64 limit)
{
         void *ptr;
         u64 addr;

         if (limit > memblock.current_limit)
                 limit = memblock.current_limit;

         addr = memblock_find_in_range_node(goal, limit, size, align, nid);
         if (!addr)
                 return NULL;
}

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 10 lines

Quoted 10 lines

Regards!
Gerry

Quoted 6 lines · Simon Jeons

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 6 lines · Simon Jeons

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,
According to my understanding, bootmem allocator does only manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                         u64 goal, u64 limit)
{
         void *ptr;
         u64 addr;

         if (limit > memblock.current_limit)
                 limit = memblock.current_limit;

         addr = memblock_find_in_range_node(goal, limit, size, align, nid);
         if (!addr)
                 return NULL;
}

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 10 lines

Quoted 10 lines

Regards!
Gerry

Quoted 6 lines · Simon Jeons

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 30 lines · Jiang Liu

Quoted 6 lines · Simon Jeons

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,
According to my understanding, bootmem allocator does only manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                         u64 goal, u64 limit)
{
         void *ptr;
         u64 addr;

         if (limit > memblock.current_limit)
                 limit = memblock.current_limit;

         addr = memblock_find_in_range_node(goal, limit, size, align, nid);
         if (!addr)
                 return NULL;
}

Quoted 3 lines

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

highmem is just used for x86, correct? ppc doesn't have highmem I think.

Quoted 11 lines

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 3 lines

Quoted 3 lines

Regards!
Gerry

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines · Simon Jeons

Quoted 6 lines

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 6 lines · Jiang Liu

Quoted 5 lines · Simon Jeons

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Quoted 2 lines

Quoted 1 line

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,

Hi Jiang,

The comments of max_pfn_mapped is "highest direct mapped pfn over 4GB", so if both bootmem allocator and memblock just manage direct mapping pages? BTW, could you show me where you can figure out traditional bootmem allocator manages directly mapping pages?

Quoted 36 lines

According to my understanding, bootmem allocator does only manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                         u64 goal, u64 limit)
{
         void *ptr;
         u64 addr;

         if (limit > memblock.current_limit)
                 limit = memblock.current_limit;

         addr = memblock_find_in_range_node(goal, limit, size, align, nid);
         if (!addr)
                 return NULL;
}

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 3 lines

Quoted 3 lines

Regards!
Gerry

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines · Simon Jeons

Quoted 6 lines

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 9 lines · Simon Jeons

Quoted 8 lines · Jiang Liu

Quoted 6 lines · Simon Jeons

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,

Quoted 4 lines

Hi Jiang,

The comments of max_pfn_mapped is "highest direct mapped pfn over 4GB", so if both bootmem allocator and memblock just manage direct mapping pages? BTW, could you show me where you can figure out traditional bootmem allocator manages directly mapping pages?

Hi Simon,
Bootmem allocator only manages directly mapped pages, but memblock could manage all pages.
For traditional bootmem allocator, you could trace back callers of init_bootmem_node() and init_bootmem() to get the idea.

Regards!
Gerry

Quoted 42 lines

Quoted 42 lines

According to my understanding, bootmem allocator does only manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                          u64 goal, u64 limit)
{
          void *ptr;
          u64 addr;

          if (limit > memblock.current_limit)
                  limit = memblock.current_limit;

          addr = memblock_find_in_range_node(goal, limit, size, align, nid);
          if (!addr)
                  return NULL;
}

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 10 lines

Quoted 10 lines

Regards!
Gerry

Quoted 6 lines · Simon Jeons

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 13 lines · Jiang Liu

Quoted 12 lines · Simon Jeons

Quoted 9 lines · Jiang Liu

Quoted 7 lines · Simon Jeons

Hi Jiang,

Quoted 4 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory
only, and it does not manage highmem pages because kernel can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,

Hi Jiang,

Quoted 7 lines

Quoted 2 lines

The comments of max_pfn_mapped is "highest direct mapped pfn over 4GB", so if both bootmem allocator and memblock just manage direct mapping pages? BTW, could you show me where you can figure out traditional bootmem allocator manages directly mapping pages?

Hi Simon,
Bootmem allocator only manages directly mapped pages, but
memblock could manage all pages.
For traditional bootmem allocator, you could trace back callers of init_bootmem_node() and init_bootmem() to get the idea.

Hi Jiang,

I track the callset of init_bootmem() against openrisc architecture(arch/openrisc/kernel/setup.c), it seems that it manages all the memory instead of low memory you mentioned. BTW, I didn't read x86_64 direct mapping codes before, if has enough big memory, what's the range of direct mapping?

Quoted 47 lines

Regards!
Gerry

Quoted 44 lines

Quoted 44 lines

According to my understanding, bootmem allocator does only
manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                          u64 goal, u64 limit)
{
          void *ptr;
          u64 addr;

          if (limit > memblock.current_limit)
                  limit = memblock.current_limit;

          addr = memblock_find_in_range_node(goal, limit, size, 
align, nid);
          if (!addr)
                  return NULL;
}

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 10 lines

Quoted 10 lines

Regards!
Gerry

Quoted 6 lines · Simon Jeons

Hi Jiang,

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

Quoted 14 lines · Jiang Liu

Quoted 13 lines · Simon Jeons

Quoted 8 lines · Jiang Liu

Quoted 6 lines · Simon Jeons

Hi Jiang,

Quoted 3 lines · Jiang Liu

Hi Simon,
Bootmem allocator is used to managed DMA and Normal memory only, and it does not manage highmem pages because kernel
can't directly access highmem pages.

Why you say so? Could you point out where you figure out bootmem allocator doesn't handle highmem pages? In my understanding, it doesn't distinguish low memory or high memory.

Hi Simon,

Hi Jiang,

The comments of max_pfn_mapped is "highest direct mapped pfn over 4GB", so if both bootmem allocator and memblock just manage direct mapping pages? BTW, could you show me where you can figure out traditional bootmem allocator manages directly mapping pages?

Quoted 4 lines

Hi Simon,
Bootmem allocator only manages directly mapped pages, but memblock could manage all pages.
For traditional bootmem allocator, you could trace back callers of init_bootmem_node() and init_bootmem() to get the idea.

Hi Jiang,

I track the callset of init_bootmem() against openrisc architecture(arch/openrisc/kernel/setup.c), it seems that it manages all the memory instead of low memory you mentioned. BTW, I didn't read x86_64 direct mapping codes before, if has enough big memory, what's the range of direct mapping?

Quoted 42 lines

Regards!
Gerry

Quoted 38 lines

Quoted 38 lines

According to my understanding, bootmem allocator does only manages lowmem pages.
For traditional bootmem allocator in mm/bootmem.c, it could only manages directly mapped lowmem pages. For new bootmem allocator in mm/nobootmem.c, it depends on memblock to do the real work. Let's take x86 as an example:
1) following code set memblock.current_limit to max_low_pfn.
arch/x86/kernel/setup.c: memblock.current_limit = get_max_mapped();
2) the core of bootmem allocator in nobootmem.c is function __alloc_memory_core_early(), which has following code to avoid allocate highmem pages:
static void * __init __alloc_memory_core_early(int nid, u64 size, u64 align,

                                          u64 goal, u64 limit)
{
          void *ptr;
          u64 addr;

          if (limit > memblock.current_limit)
                  limit = memblock.current_limit;

          addr = memblock_find_in_range_node(goal, limit, size, align, nid);
          if (!addr)
                  return NULL;
}

I guess it's the same for other architectures. On the other hand, some other architectures may allocate highmem pages during boot by directly using memblock interfaces. For example, ppc use memblock interfaces to allocate highmem pages for giagant hugetlb pages.

I'm working a patch set to fix those cases.

Regards!
Gerry

Quoted 6 lines

Quoted 6 lines

Regards!
Gerry

Quoted 2 lines · Simon Jeons

Hi Jiang,

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

https://patchwork.kernel.org/patch/1781291/

You said that the bootmem allocator doesn't touch *highmem pages*, so highmem zones' managed_pages is set to the accurate value "spanned_pages - absent_pages" in function free_area_init_core() and won't be updated anymore. Why it doesn't touch *highmem pages*? Could you point out where you figure out this?

.