Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1mLKA3-0005Ep-TG for mharc-grub-devel@gnu.org; Wed, 01 Sep 2021 02:55:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55954) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLKA0-0005BY-VE for grub-devel@gnu.org; Wed, 01 Sep 2021 02:55:50 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]:39608) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mLK9y-0001wM-EX for grub-devel@gnu.org; Wed, 01 Sep 2021 02:55:48 -0400 Received: by mail-pf1-x434.google.com with SMTP id e16so1286878pfc.6 for ; Tue, 31 Aug 2021 23:55:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axtens.net; s=google; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=ZmFefVZr+ao8uiIYoMbiFfV8zCLjLvn/d1+nK8pc/yM=; b=Lf95hYX/BkJysBGPORjgoLMxk0crL62xHQkxVQsbIEfSurhhUQp9O0BBVL5FEEsqSJ gU49qek1umK95cgFLrwkxBMr0RYrNQcit7RRKmivmO7jvbrc3teH0PDxFAFmBTqdG/nX 87Nl9WCeCp/qqvuec+C7kyexSgZE1QIX+EM60= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=ZmFefVZr+ao8uiIYoMbiFfV8zCLjLvn/d1+nK8pc/yM=; b=inDG6vZTMfSb1v11ZDy31qNcBveZChNsMTV2cJ4zfE9oy4z3aMY1RC660xIZlK1toW 1mT0KuQNEeITTrhNixi3JxhaKrJmGyCB1FjS4ANmIweyMG9hDmJuIKHQ9QdPBbpfG0+Y yL803YgCvXhtAy8QKfAyxFEHAXTKV7SaNsseqhxQQV9WQ/Zw7cYOi//YUpweaAI5k8UG 1sc/Uct2SFXu1mT4twMlFTcPdUf6GWafU+5MLw0AN9vaKxXZv0KVmugBwk5nKIzElZ5/ sX6/0Se2A7JchDwmCYE8wtek43SVWQwtvvoh+Bqi+2z5i270M7iLmqq2Mxyn8ocPsyEP i4FA== X-Gm-Message-State: AOAM532gU3JbVNutv8HJ+hCR1lpoWhjDCkSpUiPWAob2fr8eEHbRgZkr nwQQKYEtl8P/TKrlkzGlsEl8nw== X-Google-Smtp-Source: ABdhPJwxHiWrZI4AcCW239ompixpp6faPR6/fY/yy0BMlm97c+nTrMe1q/as2jYvMqLPRKe07LMwQQ== X-Received: by 2002:a62:2904:0:b0:405:2cb8:8cfb with SMTP id p4-20020a622904000000b004052cb88cfbmr7408433pfp.41.1630479344582; Tue, 31 Aug 2021 23:55:44 -0700 (PDT) Received: from localhost ([2001:4479:e200:df00:603:4bfe:470d:37b1]) by smtp.gmail.com with ESMTPSA id j20sm22179529pgb.2.2021.08.31.23.55.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Aug 2021 23:55:44 -0700 (PDT) From: Daniel Axtens To: Stefan Berger , grub-devel@gnu.org Cc: dkiper@net-space.pl, Stefan Berger Subject: Re: [PATCH v2 2/4] ieee1275: claim more memory In-Reply-To: <20210720211449.572356-3-stefanb@linux.vnet.ibm.com> References: <20210720211449.572356-1-stefanb@linux.vnet.ibm.com> <20210720211449.572356-3-stefanb@linux.vnet.ibm.com> Date: Wed, 01 Sep 2021 16:55:41 +1000 Message-ID: <87wno06az6.fsf@dja-thinkpad.axtens.net> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=dja@axtens.net; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Sep 2021 06:55:50 -0000 Stefan Berger writes: > From: Daniel Axtens > > On powerpc-ieee1275, we are running out of memory trying to verify > anything. This is because: > > - we have to load an entire file into memory to verify it. This is > extremely difficult to change with appended signatures. > - We only have 32MB of heap. > - Distro kernels are now often around 30MB. > > So we want to claim more memory from OpenFirmware for our heap. > > There are some complications: > > - The grub mm code isn't the only thing that will make claims on > memory from OpenFirmware: > > * PFW/SLOF will have claimed some for their own use. > > * The ieee1275 loader will try to find other bits of memory that we > haven't claimed to place the kernel and initrd when we go to boot. > > * Once we load Linux, it will also try to claim memory. It claims > memory without any reference to /memory/available, it just starts > at min(top of RMO, 768MB) and works down. So we need to avoid this > area. See arch/powerpc/kernel/prom_init.c as of v5.11. > > - The smallest amount of memory a ppc64 KVM guest can have is 256MB. > It doesn't work with distro kernels but can work with custom kernels. > We should maintain support for that. (ppc32 can boot with even less, > and we shouldn't break that either.) > > - Even if a VM has more memory, the memory OpenFirmware makes available > as Real Memory Area can be restricted. A freshly created LPAR on a > PowerVM machine is likely to have only 256MB available to OpenFirmware > even if it has many gigabytes of memory allocated. > > EFI systems will attempt to allocate 1/4th of the available memory, > clamped to between 1M and 1600M. That seems like a good sort of > approach, we just need to figure out if 1/4 is the right fraction > for us. > > We don't know in advance how big the kernel and initrd are going to be, > which makes figuring out how much memory we can take a bit tricky. > > To figure out how much memory we should leave unused, I looked at: > > - an Ubuntu 20.04.1 ppc64le pseries KVM guest: > vmlinux: ~30MB > initrd: ~50MB > > - a RHEL8.2 ppc64le pseries KVM guest: > vmlinux: ~30MB > initrd: ~30MB > > Ubuntu VMs struggle to boot with just 256MB under SLOF. > RHEL likewise has a higher minimum supported memory figure. > So lets first consider a distro kernel and 512MB of addressible memory. > (This is the default case for anything booting under PFW.) Say we lose > 131MB to PFW (based on some tests). This leaves us 381MB. 1/4 of 381MB > is ~95MB. That should be enough to verify a 30MB vmlinux and should > leave plenty of space to load Linux and the initrd. > > If we consider 256MB of RMA under PFW, we have just 125MB remaining. 1/4 > of that is a smidge under 32MB, which gives us very poor odds of verifying > a distro-sized kernel. However, if we need 80MB just to put the kernel > and initrd in memory, we can't claim any more than 45MB anyway. So 1/4 > will do. We'll come back to this later. > > grub is always built as a 32-bit binary, even if it's loading a ppc64 > kernel. So we can't address memory beyond 4GB. This gives a natural cap > of 1GB for powerpc-ieee1275. > > Also apply this 1/4 approach to i386-ieee1275, but keep the 32MB cap. > > make check still works for both i386 and powerpc and I've booted > powerpc grub with this change under SLOF and PFW. > > Signed-off-by: Daniel Axtens > Tested-by: Stefan Berger > --- > docs/grub-dev.texi | 6 ++- > grub-core/kern/ieee1275/init.c | 81 +++++++++++++++++++++++++++------- > 2 files changed, 69 insertions(+), 18 deletions(-) > > diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi > index 6c629a23e..c11f1ac46 100644 > --- a/docs/grub-dev.texi > +++ b/docs/grub-dev.texi > @@ -1047,7 +1047,9 @@ space is limited to 4GiB. GRUB allocates pages from EFI for its heap, at most > 1.6 GiB. > > On i386-ieee1275 and powerpc-ieee1275 GRUB uses same stack as IEEE1275. > -It allocates at most 32MiB for its heap. > + > +On i386-ieee1275, GRUB allocates at most 32MiB for its heap. On > +powerpc-ieee1275, GRUB allocates up to 1GiB. > > On sparc64-ieee1275 stack is 256KiB and heap is 2MiB. > > @@ -1075,7 +1077,7 @@ In short: > @item i386-qemu @tab 60 KiB @tab < 4 GiB > @item *-efi @tab ? @tab < 1.6 GiB > @item i386-ieee1275 @tab ? @tab < 32 MiB > -@item powerpc-ieee1275 @tab ? @tab < 32 MiB > +@item powerpc-ieee1275 @tab ? @tab < 1 GiB > @item sparc64-ieee1275 @tab 256KiB @tab 2 MiB > @item arm-uboot @tab 256KiB @tab 2 MiB > @item mips(el)-qemu_mips @tab 2MiB @tab 253 MiB > diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c > index c5d091689..4162b5949 100644 > --- a/grub-core/kern/ieee1275/init.c > +++ b/grub-core/kern/ieee1275/init.c > @@ -45,11 +45,12 @@ > #include > #endif > > -/* The maximum heap size we're going to claim */ > +/* The maximum heap size we're going to claim. Not used by sparc. > + We allocate 1/4 of the available memory under 4G, up to this limit. */ > #ifdef __i386__ > #define HEAP_MAX_SIZE (unsigned long) (64 * 1024 * 1024) > -#else > -#define HEAP_MAX_SIZE (unsigned long) (32 * 1024 * 1024) > +#else // __powerpc__ > +#define HEAP_MAX_SIZE (unsigned long) (1 * 1024 * 1024 * 1024) > #endif > > extern char _start[]; > @@ -145,16 +146,45 @@ grub_claim_heap (void) > + GRUB_KERNEL_MACHINE_STACK_SIZE), 0x200000); > } > #else > -/* Helper for grub_claim_heap. */ > +/* Helper for grub_claim_heap on powerpc. */ > +static int > +heap_size (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > + void *data) > +{ > + grub_uint32_t total = *(grub_uint32_t *)data; > + > + if (type != GRUB_MEMORY_AVAILABLE) > + return 0; > + > + /* Do not consider memory beyond 4GB */ > + if (addr > 0xffffffffUL) > + return 0; > + > + if (addr + len > 0xffffffffUL) > + len = 0xffffffffUL - addr; > + > + total += len; > + *(grub_uint32_t *)data = total; > + > + return 0; > +} > + > static int > heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > void *data) > { > - unsigned long *total = data; > + grub_uint32_t total = *(grub_uint32_t *)data; > > if (type != GRUB_MEMORY_AVAILABLE) > return 0; > > + /* Do not consider memory beyond 4GB */ > + if (addr > 0xffffffffUL) > + return 0; > + > + if (addr + len > 0xffffffffUL) > + len = 0xffffffffUL - addr; > + > if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_NO_PRE1_5M_CLAIM)) > { > if (addr + len <= 0x180000) > @@ -168,10 +198,6 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > } > len -= 1; /* Required for some firmware. */ > > - /* Never exceed HEAP_MAX_SIZE */ > - if (*total + len > HEAP_MAX_SIZE) > - len = HEAP_MAX_SIZE - *total; > - > /* In theory, firmware should already prevent this from happening by not > listing our own image in /memory/available. The check below is intended > as a safeguard in case that doesn't happen. However, it doesn't protect > @@ -183,6 +209,18 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > len = 0; > } > > + /* If this block contains 0x30000000 (768MB), do not claim below that. > + Linux likes to claim memory at min(RMO top, 768MB) and works down > + without reference to /memory/available. */ > + if ((addr < 0x30000000) && ((addr + len) > 0x30000000)) > + { > + len = len - (0x30000000 - addr); > + addr = 0x30000000; > + } > + > + if (len > total) > + len = total; (looking back over my code to test the runtime claim model on powerpc) This change accidentally breaks the fixed claim mode. We should keep this change, but ... > + > if (len) > { > grub_err_t err; > @@ -191,10 +229,12 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > if (err) > return err; > grub_mm_init_region ((void *) (grub_addr_t) addr, len); > + total -= len; > } > > - *total += len; > - if (*total >= HEAP_MAX_SIZE) > + *(grub_uint32_t *)data = total; > + > + if (total == 0) > return 1; > > return 0; > @@ -203,13 +243,22 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type, > static void > grub_claim_heap (void) > { > - unsigned long total = 0; > + grub_uint32_t total = 0; > > if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM)) > - heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, > - 1, &total); ... here we should initialise total to GRUB_IEEE1275_STATIC_HEAP_LEN first. I'll do this change when I next do a revision of the series, but Stefan if you send something with my patches attached before I send the next revision please include this. (We should have a discussion elsewhere about dropping some of the ancient platforms we support here. FORCE_CLAIM is only used for OpenHackware, which was last modified in 2014 and had qemu support deprecated in qemu v3.1 and removed in Feb 2020. FORCE_CLAIM requires special case code in at least 4 places. This seems like poor value for complexity.) > - else > - grub_machine_mmap_iterate (heap_init, &total); > + { > + heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN, > + 1, &total); > + return; > + } > + > + grub_machine_mmap_iterate (heap_size, &total); > + > + total = total / 4; > + if (total > HEAP_MAX_SIZE) > + total = HEAP_MAX_SIZE; > + > + grub_machine_mmap_iterate (heap_init, &total); > } > #endif > > -- > 2.25.1