Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1m5x4w-0007YH-3u for mharc-grub-devel@gnu.org; Tue, 20 Jul 2021 17:15:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5x4u-0007Vb-EW for grub-devel@gnu.org; Tue, 20 Jul 2021 17:15:00 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:54620) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5x4r-0001uW-JI for grub-devel@gnu.org; Tue, 20 Jul 2021 17:15:00 -0400 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16KL2NLd110458; Tue, 20 Jul 2021 17:14:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=OefqlnteOFSLmejSCTX8T/nV+08y5HitQsJj1XaFPrc=; b=Rb3kVpAyuMab+9VKMGWXeFXMHbcVplstYBW0ARucZEkFJf3B1tfxrdKSO9Q89NSDvL0y WcvFFPDtabsWEopttWhOuwHr8zaewTR23TjNHDGtVMUKhQ+Dp3+wyLQxscTfpc5Zvzcs gcY79lEdYEAddYAYUJQPpTM0aekJreKpWuTnGaDHRK47SGhY4WXtHJiKIUTINhs+Crm5 acJEgM4EB2dMRyD+jg7wRpKyX0lySDqNIab8dxQPIBhC9Scd8J0KU8yTyj42kq32UXv0 yMbxtMocA+sJZQ8m7EdjeZuU6rDX76miBkNLBKAULvpBTfPMlpbZDD3CnhFH5xkRttxn OA== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 39wy71xq3t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Jul 2021 17:14:53 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 16KLDgad009130; Tue, 20 Jul 2021 21:14:52 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma03wdc.us.ibm.com with ESMTP id 39vqduqt9w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 20 Jul 2021 21:14:52 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 16KLEpSD11862568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 20 Jul 2021 21:14:51 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 93158C6063; Tue, 20 Jul 2021 21:14:51 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 44C75C6059; Tue, 20 Jul 2021 21:14:51 +0000 (GMT) Received: from sbct-2.. (unknown [9.47.158.152]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Tue, 20 Jul 2021 21:14:51 +0000 (GMT) From: Stefan Berger To: grub-devel@gnu.org Cc: dkiper@net-space.pl, Daniel Axtens , Stefan Berger Subject: [PATCH v2 2/4] ieee1275: claim more memory Date: Tue, 20 Jul 2021 17:14:47 -0400 Message-Id: <20210720211449.572356-3-stefanb@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210720211449.572356-1-stefanb@linux.vnet.ibm.com> References: <20210720211449.572356-1-stefanb@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: kAcXmX5kUztho3cYSBodcMznYh50Hb6R X-Proofpoint-ORIG-GUID: kAcXmX5kUztho3cYSBodcMznYh50Hb6R X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-07-20_13:2021-07-19, 2021-07-20 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 suspectscore=0 mlxlogscore=999 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 impostorscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107200131 Received-SPF: none client-ip=148.163.156.1; envelope-from=stefanb@linux.vnet.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2021 21:15:00 -0000 From: Daniel Axtens On powerpc-ieee1275, we are running out of memory trying to verify anything. This is because: - we have to load an entire file into memory to verify it. This is extremely difficult to change with appended signatures. - We only have 32MB of heap. - Distro kernels are now often around 30MB. So we want to claim more memory from OpenFirmware for our heap. There are some complications: - The grub mm code isn't the only thing that will make claims on memory from OpenFirmware: * PFW/SLOF will have claimed some for their own use. * The ieee1275 loader will try to find other bits of memory that we haven't claimed to place the kernel and initrd when we go to boot. * Once we load Linux, it will also try to claim memory. It claims memory without any reference to /memory/available, it just starts at min(top of RMO, 768MB) and works down. So we need to avoid this area. See arch/powerpc/kernel/prom_init.c as of v5.11. - The smallest amount of memory a ppc64 KVM guest can have is 256MB. It doesn't work with distro kernels but can work with custom kernels. We should maintain support for that. (ppc32 can boot with even less, and we shouldn't break that either.) - Even if a VM has more memory, the memory OpenFirmware makes available as Real Memory Area can be restricted. A freshly created LPAR on a PowerVM machine is likely to have only 256MB available to OpenFirmware even if it has many gigabytes of memory allocated. EFI systems will attempt to allocate 1/4th of the available memory, clamped to between 1M and 1600M. That seems like a good sort of approach, we just need to figure out if 1/4 is the right fraction for us. We don't know in advance how big the kernel and initrd are going to be, which makes figuring out how much memory we can take a bit tricky. To figure out how much memory we should leave unused, I looked at: - an Ubuntu 20.04.1 ppc64le pseries KVM guest: vmlinux: ~30MB initrd: ~50MB - a RHEL8.2 ppc64le pseries KVM guest: vmlinux: ~30MB initrd: ~30MB Ubuntu VMs struggle to boot with just 256MB under SLOF. RHEL likewise has a higher minimum supported memory figure. So lets first consider a distro kernel and 512MB of addressible memory. (This is the default case for anything booting under PFW.) Say we lose 131MB to PFW (based on some tests). This leaves us 381MB. 1/4 of 381MB is ~95MB. That should be enough to verify a 30MB vmlinux and should leave plenty of space to load Linux and the initrd. If we consider 256MB of RMA under PFW, we have just 125MB remaining. 1/4 of that is a smidge under 32MB, which gives us very poor odds of verifying a distro-sized kernel. However, if we need 80MB just to put the kernel and initrd in memory, we can't claim any more than 45MB anyway. So 1/4 will do. We'll come back to this later. grub is always built as a 32-bit binary, even if it's loading a ppc64 kernel. So we can't address memory beyond 4GB. This gives a natural cap of 1GB for powerpc-ieee1275. Also apply this 1/4 approach to i386-ieee1275, but keep the 32MB cap. make check still works for both i386 and powerpc and I've booted powerpc grub with this change under SLOF and PFW. Signed-off-by: Daniel Axtens Tested-by: Stefan Berger --- docs/grub-dev.texi | 6 ++- grub-core/kern/ieee1275/init.c | 81 +++++++++++++++++++++++++++------- 2 files changed, 69 insertions(+), 18 deletions(-) diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi index 6c629a23e..c11f1ac46 100644 --- a/docs/grub-dev.texi +++ b/docs/grub-dev.texi @@ -1047,7 +1047,9 @@ space is limited to 4GiB. GRUB allocates pages from EFI for its heap, at most 1.6 GiB. On i386-ieee1275 and powerpc-ieee1275 GRUB uses same stack as IEEE1275. -It allocates at most 32MiB for its heap. + +On i386-ieee1275, GRUB allocates at most 32MiB for its heap. On +powerpc-ieee1275, GRUB allocates up to 1GiB. On sparc64-ieee1275 stack is 256KiB and heap is 2MiB. @@ -1075,7 +1077,7 @@ In short: @item i386-qemu @tab 60 KiB @tab < 4 GiB @item *-efi @tab ? @tab < 1.6 GiB @item i386-ieee1275 @tab ? @tab < 32 MiB -@item powerpc-ieee1275 @tab ? @tab < 32 MiB +@item powerpc-ieee1275 @tab ? @tab < 1 GiB @item sparc64-ieee1275 @tab 256KiB @tab 2 MiB @item arm-uboot @tab 256KiB @tab 2 MiB @item mips(el)-qemu_mips @tab 2MiB @tab 253 MiB diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c index c5d091689..4162b5949 100644 --- a/grub-core/kern/ieee1275/init.c +++ b/grub-core/kern/ieee1275/init.c @@ -45,11 +45,12 @@ #include #endif -/* The maximum heap size we're going to claim */ +/* The maximum heap size we're going to claim. Not used by sparc. + We allocate 1/4 of the available memory under 4G, up to this limit. */ #ifdef __i386__ #define HEAP_MAX_SIZE (unsigned long) (64 * 1024 * 1024) -#else -#define HEAP_MAX_SIZE (unsigned long) (32 * 1024 * 1024) +#else // __powerpc__ +#define HEAP_MAX_SIZE (unsigned long) (1 * 1024 * 1024 * 1024) #endif extern char _start[]; @@ -145,16 +146,45 @@ grub_claim_heap (void) + GRUB_KERNEL_MACHINE_STACK_SIZE), 0x200000); } #else -/* Helper for grub_claim_heap. */ +/* Helper for grub_claim_heap on powerpc. */ +static int +heap_size (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, + void *data) +{ + grub_uint32_t total = *(grub_uint32_t *)data; + + if (type != GRUB_MEMORY_AVAILABLE) + return 0; + + /* Do not consider memory beyond 4GB */ + if (addr > 0xffffffffUL) + return 0; + + if (addr + len > 0xffffffffUL) + len = 0xffffffffUL - addr; + + total += len; + *(grub_uint32_t *)data = total; + + return 0; +} + static int heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, void *data) { - unsigned long *total = data; + grub_uint32_t total = *(grub_uint32_t *)data; if (type != GRUB_MEMORY_AVAILABLE) return 0; + /* Do not consider memory beyond 4GB */ + if (addr > 0xffffffffUL) + return 0; + + if (addr + len > 0xffffffffUL) + len = 0xffffffffUL - addr; + if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_NO_PRE1_5M_CLAIM)) { if (addr + len <= 0x180000) @@ -168,10 +198,6 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, } len -= 1; /* Required for some firmware. */ - /* Never exceed HEAP_MAX_SIZE */ - if (*total + len > HEAP_MAX_SIZE) - len = HEAP_MAX_SIZE - *total; - /* In theory, firmware should already prevent this from happening by not listing our own image in /memory/available. The check below is intended as a safeguard in case that doesn't happen. However, it doesn't protect @@ -183,6 +209,18 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, len = 0; } + /* If this block contains 0x30000000 (768MB), do not claim below that. + Linux likes to claim memory at min(RMO top, 768MB) and works down + without reference to /memory/available. */ + if ((addr < 0x30000000) && ((addr + len) > 0x30000000)) + { + len = len - (0x30000000 - addr); + addr = 0x30000000; + } + + if (len > total) + len = total; + if (len) { grub_err_t err; @@ -191,10 +229,12 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, if (err) return err; grub_mm_init_region ((void *) (grub_addr_t) addr, len); + total -= len; } - *total += len; - if (*total >= HEAP_MAX_SIZE) + *(grub_uint32_t *)data = total; + + if (total == 0) return 1; return 0; @@ -203,13 +243,22 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, static void grub_claim_heap (void) { - unsigned long total = 0; + grub_uint32_t total = 0; if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM)) - heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, - 1, &total); - else - grub_machine_mmap_iterate (heap_init, &total); + { + heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, + 1, &total); + return; + } + + grub_machine_mmap_iterate (heap_size, &total); + + total = total / 4; + if (total > HEAP_MAX_SIZE) + total = HEAP_MAX_SIZE; + + grub_machine_mmap_iterate (heap_init, &total); } #endif -- 2.25.1