Skip to contentSkip to conversation

2.6.0 test3 does not boot on ia64 NUMA

DISCUSSION Xavier Bru 4 participants 13 messages Aug 26, 2003 – Sep 8, 2003

lore
Share & tools
lore

Xavier Bru

author
#0 lore raw

... but 2.5.72 was OK :-)

Booting on a 4 nodes Itanium machine, with
CONFIG_NUMA=y
CONFIG_ACPI_NUMA=y
CONFIG_DISCONTIGMEM=y
# CONFIG_VIRTUAL_MEM_MAP is not set

system hangs after a message:
"On node 1 totalpages: <a very very BIG number that looks garbage>"

Having a look to the code, it seems that problem is due to memory initialisation changes:
. unlike 2.5.72, find_memory() is now called before
acpi_numa_init(), and supposes that numnodes = 1. So only 1 bootmem_data_t struct is initialised.
. acpi_numa_init() then finds in SRAT that 4 nodes exist. . when paging_init calls discontig_paging_init only bootmem_data_t
for node 0 is initialized, and garbage is found for nodes 1 to 3.

I tried to put find_memory() after acpi_numa_init() as in 2.5.72, but now we get:
bootmem alloc of 100 bytes failed! in acpi_table_init()
that now uses the bootmem allocator.

I wonder what should be the right order for initialisation, and if there are ia64 platforms running 2.6.0 with CONFIG_NUMA. Any help is apreciated.
Thanks in advance.
Xavier.

Signature · 8 lines
 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
#1 lore raw
Quoted 9 lines · Xavier Bru

I tried to put find_memory() after acpi_numa_init() as in 2.5.72, but now we get:
bootmem alloc of 100 bytes failed! in acpi_table_init()
that now uses the bootmem allocator.

I wonder what should be the right order for initialisation, and if there are ia64 platforms running 2.6.0 with CONFIG_NUMA. Any help is apreciated.

Jesse Barnes has an unreleased patch to make SGI SN2 work, which does move the find_memory() after acpi_numa_init(). There is another patch, which I've attached, which removes the bootmem alloc of memory in ACPI.

I haven't played with this recently, so your mileage may vary. Jesse should be back from vacation next week, and should release updated SN2 patches then.

mh

Signature · 55 lines
Wild Open Source Inc.                  [email protected]


# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.1395  -> 1.1396 
#	drivers/acpi/tables.c	1.14    -> 1.15   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 03/07/30	[email protected]	1.1396
# don't use alloc_bootmem in acpi table init
# --------------------------------------------
#
diff -Nru a/drivers/acpi/tables.c b/drivers/acpi/tables.c
--- a/drivers/acpi/tables.c	Wed Jul 30 11:45:27 2003
+++ b/drivers/acpi/tables.c	Wed Jul 30 11:45:27 2003
@@ -69,7 +69,8 @@
 
 static unsigned long		sdt_pa;		/* Physical Address */
 static unsigned long		sdt_count;	/* Table count */
-static struct acpi_table_sdt	*sdt_entry;
+
+static struct acpi_table_sdt	sdt_entry[ACPI_MAX_TABLES];
 
 void
 acpi_table_print (
@@ -413,12 +414,6 @@
 			sdt_count = ACPI_MAX_TABLES;
 		}
 
-		sdt_entry = alloc_bootmem(sdt_count * sizeof(struct acpi_table_sdt));
-		if (!sdt_entry) {
-			printk(KERN_ERR "ACPI: Could not allocate mem for SDT entries!\n");
-			return -ENOMEM;
-		}
-
 		for (i = 0; i < sdt_count; i++)
 			sdt_entry[i].pa = (unsigned long) mapped_xsdt->entry[i];
 	}
@@ -463,12 +458,6 @@
 			printk(KERN_WARNING PREFIX "Truncated %lu RSDT entries\n",
 				(sdt_count - ACPI_MAX_TABLES));
 			sdt_count = ACPI_MAX_TABLES;
-		}
-
-		sdt_entry = alloc_bootmem(sdt_count * sizeof(struct acpi_table_sdt));
-		if (!sdt_entry) {
-			printk(KERN_ERR "ACPI: Could not allocate mem for SDT entries!\n");
-			return -ENOMEM;
 		}
 
 		for (i = 0; i < sdt_count; i++)
author
#2 lore raw
In reply to Xavier Bru

Martin,

Thanks for answering my mail. I applied Jesse's patch ([PATCH] latest discontig mail dated Fri, 8 Aug 2003) but this does'nt build with:
CONFIG_NUMA=y
CONFIG_ACPI_NUMA=y
CONFIG_DISCONTIGMEM=y
# CONFIG_VIRTUAL_MEM_MAP is not set
I get:
include/linux/mm.h: In function `lowmem_page_address':
include/linux/mm.h:349: `mem_map' undeclared (first use in this function)

It seems that this only works on Altix, unless I missed something.

BTW, applying your patch for removing alloc_bootmem and moving back find_memory() to its previous place, system boots OK. Thanks again for your help.

Xavier
-----------------------------------------------------------------------------
diff --exclude-from /users/xb/proc/diff.exclude -r 0t4/arch/ia64/kernel/setup.c linux-2.6.0-test4/arch/ia64/kernel/setup.c 374a375

Quoted 1 line
	find_memory();

388,389d388
< find_memory();
<
diff --exclude-from /users/xb/proc/diff.exclude -r 0t4/drivers/acpi/tables.c linux-2.6.0-test4/drivers/acpi/tables.c 72,73c72
<
< static struct acpi_table_sdt sdt_entry[ACPI_MAX_TABLES];
---

Quoted 1 line

static struct acpi_table_sdt *sdt_entry;

428a428,433

Quoted 6 lines
		sdt_entry = alloc_bootmem(sdt_count * sizeof(struct acpi_table_sdt));
		if (!sdt_entry) {
			printk(KERN_ERR "ACPI: Could not allocate mem for SDT entries!\n");
			return -ENOMEM;
		}

474a480,485

Quoted 6 lines
		sdt_entry = alloc_bootmem(sdt_count * sizeof(struct acpi_table_sdt));
		if (!sdt_entry) {
			printk(KERN_ERR "ACPI: Could not allocate mem for SDT entries!\n");
			return -ENOMEM;
		}
Signature · 8 lines
 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
Quoted 83 lines · Martin Hicks
Quoted 9 lines · Xavier Bru

I tried to put find_memory() after acpi_numa_init() as in 2.5.72, but now we get:
bootmem alloc of 100 bytes failed! in acpi_table_init()
that now uses the bootmem allocator.

I wonder what should be the right order for initialisation, and if there are ia64 platforms running 2.6.0 with CONFIG_NUMA. Any help is apreciated.

Jesse Barnes has an unreleased patch to make SGI SN2 work, which does move the find_memory() after acpi_numa_init(). There is another patch, which I've attached, which removes the bootmem alloc of memory in ACPI.

I haven't played with this recently, so your mileage may vary. Jesse should be back from vacation next week, and should release updated SN2 patches then.

mh

Signature · 17 lines
Wild Open Source Inc.                  [email protected]


# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.1395  -> 1.1396 
#	drivers/acpi/tables.c	1.14    -> 1.15   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 03/07/30	[email protected]	1.1396
# don't use alloc_bootmem in acpi table init
# --------------------------------------------
#
diff -Nru a/drivers/acpi/tables.c b/drivers/acpi/tables.c
drivers/acpi/tables.c +2 -14
@@ -69,7 +69,8 @@
 
 static unsigned long		sdt_pa;		/* Physical Address */
 static unsigned long		sdt_count;	/* Table count */
-static struct acpi_table_sdt	*sdt_entry;
+
+static struct acpi_table_sdt	sdt_entry[ACPI_MAX_TABLES];
 
 void
 acpi_table_print (
@@ -413,12 +414,6 @@
 			sdt_count = ACPI_MAX_TABLES;
 		}
 
-		sdt_entry = alloc_bootmem(sdt_count * sizeof(struct acpi_table_sdt));
-		if (!sdt_entry) {
-			printk(KERN_ERR "ACPI: Could not allocate mem for SDT entries!\n");
-			return -ENOMEM;
-		}
-
 		for (i = 0; i < sdt_count; i++)
 			sdt_entry[i].pa = (unsigned long) mapped_xsdt->entry[i];
 	}
@@ -463,12 +458,6 @@
 			printk(KERN_WARNING PREFIX "Truncated %lu RSDT entries\n",
 				(sdt_count - ACPI_MAX_TABLES));
 			sdt_count = ACPI_MAX_TABLES;
-		}
-
-		sdt_entry = alloc_bootmem(sdt_count * sizeof(struct acpi_table_sdt));
-		if (!sdt_entry) {
-			printk(KERN_ERR "ACPI: Could not allocate mem for SDT entries!\n");
-			return -ENOMEM;
 		}
 
 		for (i = 0; i < sdt_count; i++)
-

To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
In reply to Xavier Bru
Quoted 13 lines · Xavier Bru

Martin,

Thanks for answering my mail. I applied Jesse's patch ([PATCH] latest discontig mail dated Fri, 8 Aug 2003) but this does'nt build with:
CONFIG_NUMA=y
CONFIG_ACPI_NUMA=y
CONFIG_DISCONTIGMEM=y
# CONFIG_VIRTUAL_MEM_MAP is not set
I get:
include/linux/mm.h: In function `lowmem_page_address':
include/linux/mm.h:349: `mem_map' undeclared (first use in this function)

Quoted 1 line

It seems that this only works on Altix, unless I missed something.

I'm pretty sure that you need CONFIG_VIRTUAL_MEM_MAP=y if you have CONFIG_DISCONTIGMEM=y

mh

Signature · 1 line
Wild Open Source Inc.                  [email protected]
author
#4 lore raw
In reply to Xavier Bru

Martin,

Thanks for the information (Until now CONFIG_DISCONTIGMEM was exclusive with CONFIG_VIRTUAL_MEM_MAP).

System now builds OK with Jesse's patch + the acpi alloc_bootmem patch.

Unfortunately, it dies somewhere after entering paging_init(), and before printing "On node %d totalpages:":

ifa=0xf000ff54f7c5722b
iip=0xa0000001007780d0
0xa0000001007780d0 <__alloc_bootmem_core+592>: [MII] ld8 r17=[r19],8

last traces are:

Mca related initilaization done
kernel unaligned access to 0xf000ff54f000eefb ip=0xa0000001007780d0 kernel unaligned access to 0xf000ff54f000ef03 ip=0xa0000001007780d0 kernel unaligned access to 0xf000ff54f000ef0b ip=0xa0000001007780d0

I will try to look at this a bit more if I find some more time.

Thanks again.

Xavier

Signature · 8 lines
 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
Quoted 25 lines · Martin Hicks
Quoted 13 lines · Xavier Bru

Martin,

Thanks for answering my mail. I applied Jesse's patch ([PATCH] latest discontig mail dated Fri, 8 Aug 2003) but this does'nt build with:
CONFIG_NUMA=y
CONFIG_ACPI_NUMA=y
CONFIG_DISCONTIGMEM=y
# CONFIG_VIRTUAL_MEM_MAP is not set
I get:
include/linux/mm.h: In function `lowmem_page_address':
include/linux/mm.h:349: `mem_map' undeclared (first use in this function)

Quoted 1 line

It seems that this only works on Altix, unless I missed something.

I'm pretty sure that you need CONFIG_VIRTUAL_MEM_MAP=y if you have CONFIG_DISCONTIGMEM=y

mh

Signature · 1 line
Wild Open Source Inc.                  [email protected]
#5 lore raw
In reply to Xavier Bru
Quoted 9 lines · Xavier Bru

printing "On node %d totalpages:":

ifa=0xf000ff54f7c5722b
iip=0xa0000001007780d0
0xa0000001007780d0 <__alloc_bootmem_core+592>: [MII] ld8 r17=[r19],8

I will try to look at this a bit more if I find some more time.

Do you have this changeset in your tree? It sounds like a similar problem to what I was seeing.

[email protected], 2003-08-25 12:23:20-07:00, [email protected]
[PATCH] ia64: paddr_to_nid fixup

Here is a small patch for paddr_to_nid(). This fix is already in 2.4 and is used in the case where a NUMA kernel is running on a machine without a SRAT ACPI table. Without this patch the node info is not correctly located.

mh

Signature · 1 line
Wild Open Source Inc.                  [email protected]
author
#6 lore raw
In reply to Xavier Bru

Martin,

I checked that the code for changeset 1.1292.1.3 is already present in 2.6.0 test4.

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1292.1.2 -> 1.1292.1.3
# arch/ia64/mm/numa.c 1.1 -> 1.2
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 03/08/25 [email protected] 1.1292.1.3
# [PATCH] ia64: paddr_to_nid fixup
#
# Here is a small patch for paddr_to_nid(). This fix is already in 2.4

# and is used in the case where a NUMA kernel is running on a machine
# without a SRAT ACPI table.  Without this patch the node info is not
# correctly located.
# --------------------------------------------
#
diff -Nru a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
--- a/arch/ia64/mm/numa.c	Wed Aug 27 00:40:34 2003
+++ b/arch/ia64/mm/numa.c	Wed Aug 27 00:40:34 2003
@@ -42,5 +42,5 @@
 		    paddr < node_memblk[i].start_paddr + node_memblk[i].size)
 			break;

- return (i < num_memblks) ? node_memblk[i].nid : -1;
+ return (i < num_memblks) ? node_memblk[i].nid : (num_memblks ? -1 : 0);
}

Thanks for your help.
Xavier

Signature · 8 lines
 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
Quoted 33 lines · Martin Hicks
Quoted 9 lines · Xavier Bru

printing "On node %d totalpages:":

ifa=0xf000ff54f7c5722b
iip=0xa0000001007780d0
0xa0000001007780d0 <__alloc_bootmem_core+592>: [MII] ld8 r17=[r19],8

I will try to look at this a bit more if I find some more time.

Do you have this changeset in your tree? It sounds like a similar problem to what I was seeing.

[email protected], 2003-08-25 12:23:20-07:00, [email protected]
[PATCH] ia64: paddr_to_nid fixup

Here is a small patch for paddr_to_nid(). This fix is already in 2.4 and is used in the case where a NUMA kernel is running on a machine without a SRAT ACPI table. Without this patch the node info is not correctly located.

mh

Signature · 5 lines
Wild Open Source Inc.                  [email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#7 lore raw
In reply to Xavier Bru

Hello Martin,

I finally found the reason for crashing at init time:
On node 0, our test configuration has:
2 GB of memory at address 0
2 GB of memory at address 6 GB (due to PCI hole).

Current code for acpi_numa_memory_affinity_init ignores physical memory bank if the hole (4GB) is bigger than the bank (2 GB). As the node_memblk is not present for address 6 GB, paddr_to_nid returns -1 and alloc_bootmem_pages_node crashes with a Null pointer.

As we now have CONFIG_VIRTUAL_MEM_MAP=y, I suppose we should also use sparse memory in same node. (Am I right ?)

Now 2.6.0 test4 boots OK in NUMA with:

. Jesse's discontig patch
. Tony's trim patch
. alloc_bootmem patch
. and this small one :-)

arch/ia64/kernel/acpi.c +1 -2
diff --exclude-from /users/xb/proc/diff.exclude -Nur linux-2.6.0-test4/arch/ia64/kernel/acpi.c 0t4/arch/ia64/kernel/acpi.c
@@ -423,9 +423,8 @@
 
 	if (min_hole_size) {
 		if (min_hole_size > size) {
-			printk(KERN_ERR "Too huge memory hole. Ignoring %ld MBytes at %lx\n",
+			printk(KERN_WARNING "Huge memory hole. Using %ld MBytes at %lx\n",
 			       size/(1024*1024), paddr);
-			return;
 		}
 	}
 

Thanks again for your help.

Xavier

Signature · 8 lines
 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
Quoted 34 lines
Quoted 33 lines · Martin Hicks
Quoted 9 lines · Xavier Bru

printing "On node %d totalpages:":

ifa=0xf000ff54f7c5722b
iip=0xa0000001007780d0
0xa0000001007780d0 <__alloc_bootmem_core+592>: [MII] ld8 r17=[r19],8

I will try to look at this a bit more if I find some more time.

Do you have this changeset in your tree? It sounds like a similar problem to what I was seeing.

[email protected], 2003-08-25 12:23:20-07:00, [email protected]
[PATCH] ia64: paddr_to_nid fixup

Here is a small patch for paddr_to_nid(). This fix is already in 2.4 and is used in the case where a NUMA kernel is running on a machine without a SRAT ACPI table. Without this patch the node info is not correctly located.

mh

Signature · 5 lines
Wild Open Source Inc.                  [email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#8 lore raw
In reply to Xavier Bru

Thanks Xavier, I've included this in the latest discontig patch, which I'll post again next week I think (with the fixes David wanted for reentrance).

Jesse

Quoted 88 lines · Xavier Bru

Hello Martin,

I finally found the reason for crashing at init time:
On node 0, our test configuration has:
2 GB of memory at address 0
2 GB of memory at address 6 GB (due to PCI hole).

Current code for acpi_numa_memory_affinity_init ignores physical memory bank if the hole (4GB) is bigger than the bank (2 GB). As the node_memblk is not present for address 6 GB, paddr_to_nid returns -1 and alloc_bootmem_pages_node crashes with a Null pointer.

As we now have CONFIG_VIRTUAL_MEM_MAP=y, I suppose we should also use sparse memory in same node. (Am I right ?)

Now 2.6.0 test4 boots OK in NUMA with:

. Jesse's discontig patch
. Tony's trim patch
. alloc_bootmem patch
. and this small one :-)

diff --exclude-from /users/xb/proc/diff.exclude -Nur linux-2.6.0-test4/arch/ia64/kernel/acpi.c 0t4/arch/ia64/kernel/acpi.c

arch/ia64/kernel/acpi.c +1 -2
@@ -423,9 +423,8 @@
 
 	if (min_hole_size) {
 		if (min_hole_size > size) {
-			printk(KERN_ERR "Too huge memory hole. Ignoring %ld MBytes at %lx\n",
+			printk(KERN_WARNING "Huge memory hole. Using %ld MBytes at %lx\n",
 			       size/(1024*1024), paddr);
-			return;
 		}
 	}
 

Thanks again for your help.

Xavier

Signature · 8 lines
 Sinc?res salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
Quoted 26 lines
Quoted 25 lines · Martin Hicks
Quoted 9 lines · Xavier Bru

printing "On node %d totalpages:":

ifa=0xf000ff54f7c5722b
iip=0xa0000001007780d0
0xa0000001007780d0 <__alloc_bootmem_core+592>: [MII] ld8 r17=[r19],8

I will try to look at this a bit more if I find some more time.

Do you have this changeset in your tree? It sounds like a similar problem to what I was seeing.

[email protected], 2003-08-25 12:23:20-07:00, [email protected]
[PATCH] ia64: paddr_to_nid fixup

Here is a small patch for paddr_to_nid(). This fix is already in 2.4 and is used in the case where a NUMA kernel is running on a machine without a SRAT ACPI table. Without this patch the node info is not correctly located.

Quoted 8 lines
Quoted 8 lines

mh

Signature · 5 lines
Wild Open Source Inc.                  [email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw
In reply to Xavier Bru
Quoted 42 lines

Thanks Xavier, I've included this in the latest discontig patch, which I'll post again next week I think (with the fixes David wanted for reentrance).

Jesse

Quoted 7 lines · Xavier Bru

Hello Martin,

I finally found the reason for crashing at init time:
On node 0, our test configuration has:
2 GB of memory at address 0
2 GB of memory at address 6 GB (due to PCI hole).

Quoted 6 lines

Current code for acpi_numa_memory_affinity_init ignores physical memory bank if the hole (4GB) is bigger than the bank (2 GB). As the node_memblk is not present for address 6 GB, paddr_to_nid returns -1 and alloc_bootmem_pages_node crashes with a Null pointer.

As we now have CONFIG_VIRTUAL_MEM_MAP=y, I suppose we

should also use

Quoted 10 lines

sparse memory in same node. (Am I right ?)

Now 2.6.0 test4 boots OK in NUMA with:

. Jesse's discontig patch
. Tony's trim patch
. alloc_bootmem patch
. and this small one :-)

diff --exclude-from /users/xb/proc/diff.exclude -Nur

linux-2.6.0-test4/arch/ia64/kernel/acpi.c 0t4/arch/ia64/kernel/acpi.c

Quoted 1 line

--- linux-2.6.0-test4/arch/ia64/kernel/acpi.c

2003-08-23 01:55:43.000000000 +0200
Quoted 1 line

+++ 0t4/arch/ia64/kernel/acpi.c 2003-09-02

15:37:17.000000000 +0200

Review line 426

Quoted 5 lines
(quoted diff) +0 -1
@@ -423,9 +423,8 @@
 
 	if (min_hole_size) {
 		if (min_hole_size > size) {
-			printk(KERN_ERR "Too huge memory hole. 

Ignoring %ld MBytes at %lx\n",

Quoted 6 lines

Review line 426

Quoted 1 line
(quoted diff) +1 -0
+			printk(KERN_WARNING "Huge memory hole. 

Using %ld MBytes at %lx\n",

Review line 429

Quoted 4 lines
(quoted diff) +0 -1
 			       size/(1024*1024), paddr);
-			return;
 		}
 	}

What are the remaining issues with sparse memory within a node? CONFIG_VIRTUAL_MEM_MAP should be able to cope with this without wasting memory on "struct page" for non-existent pages in the holes.

Presumably there are some bootmem bitmap size issues if the gaps within nodes are too huge. But a few GB shouldn't be a problem (with a 16K page size, each GB of memory/hole only takes 8K of bitmap).

Is there anything else that blows up?

If not, then could we just drop the printk altogether?

-Tony

#10 lore raw
In reply to Xavier Bru
Quoted 3 lines · Tony

What are the remaining issues with sparse memory within a node? CONFIG_VIRTUAL_MEM_MAP should be able to cope with this without wasting memory on "struct page" for non-existent pages in the holes.

Yeah, that's what we're using the latest discontig patches.

Quoted 3 lines

Presumably there are some bootmem bitmap size issues if the gaps within nodes are too huge. But a few GB shouldn't be a problem (with a 16K page size, each GB of memory/hole only takes 8K of bitmap).

Right, until we get really huge holes, that'll be ok, and there are already some optimizations to keep track of the last place a successful bootmem allocation came from so that you don't keep walking through the holes, which helped zx1 boots.

Quoted 1 line

Is there anything else that blows up?

Not that I know of...

Quoted 1 line

If not, then could we just drop the printk altogether?

Sure, sounds good to me.

Jesse

author
#11 lore raw
In reply to Xavier Bru
Quoted 7 lines · Jesse Barnes
Quoted 3 lines · Tony

What are the remaining issues with sparse memory within a node? CONFIG_VIRTUAL_MEM_MAP should be able to cope with this without wasting memory on "struct page" for non-existent pages in the holes.

Yeah, that's what we're using the latest discontig patches.

Quoted 8 lines
Quoted 3 lines

Presumably there are some bootmem bitmap size issues if the gaps within nodes are too huge. But a few GB shouldn't be a problem (with a 16K page size, each GB of memory/hole only takes 8K of bitmap).

Right, until we get really huge holes, that'll be ok, and there are already some optimizations to keep track of the last place a successful bootmem allocation came from so that you don't keep walking through the holes, which helped zx1 boots.

This allows supporting our platform without specific modifications.

Quoted 8 lines
Quoted 2 lines

Is there anything else that blows up?

Not that I know of...

Quoted 1 line

If not, then could we just drop the printk altogether?

Sure, sounds good to me.

About code cleaning, it seems that the IA64_NODESIZE_* options are still present in Kconfig but no more used, at least for DIG platforms.

Thanks again for taking in account our problem.
Xavier

Signature · 8 lines
 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:[email protected]
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________
#12 lore raw
In reply to Xavier Bru
Quoted 3 lines · Xavier Bru

About code cleaning, it seems that the IA64_NODESIZE_* options are still present in Kconfig but no more used, at least for DIG platforms.

I've removed these in the patch as well, I'll post a new one for testing soon.

Thanks,
Jesse