Implement a grub loader for RISC-V LINUX
cover Implement a grub loader for RISC-V LINUX
Implement an initial version of riscv loader and related commands to load and run linux kernel and initrd on RISC-V. I tested this series based on the following configuration:
- QEMU 4.2.50 (machine: virt)
- OpenSBI v0.5-51
- U-Boot 2020.01-rc5
- grub 2.04
- linux-kernel v5.4
- openSUSE-Tumbleweed-20191103
Chester Lin (2):
RISC-V: Correct linux headers' definitions
RISC-V: Implement linux image loader
grub-core/loader/riscv/linux.c | 311 ++++++++++++++++++++++++++++++++- include/grub/riscv32/linux.h | 4 +- include/grub/riscv64/linux.h | 6 +- 3 files changed, 312 insertions(+), 9 deletions(-)
Signature · 1 line
2.24.0
Hi Chester,
Quoted 10 lines · Chester Lin
Implement an initial version of riscv loader and related commands to load and run linux kernel and initrd on RISC-V. I tested this series based on the following configuration:
- QEMU 4.2.50 (machine: virt)
- OpenSBI v0.5-51
- U-Boot 2020.01-rc5
- grub 2.04
- linux-kernel v5.4
- openSUSE-Tumbleweed-20191103
Thanks a lot for tackling this problem - it's been on the back burner for way too long :). Unfortunately this patch set loads grub via UEFI, but then does not execute Linux using the UEFI protocol. While that's a nice hack for starters, it severely limits the extensibility of the boot flow going forward.
IIRC either Anup or Atish wanted to work on a UEFI boot stub for Linux. We could then just unify the ARM and RISC-V UEFI boot paths in grub and use that common code to run Linux via the UEFI stub.
Thanks,
Alex
Sent from my iPhone
Quoted 17 lines · Alexander Graf
Hi Chester,
Quoted 11 lines · Chester Lin
Implement an initial version of riscv loader and related commands to load and run linux kernel and initrd on RISC-V. I tested this series based on the following configuration:
- QEMU 4.2.50 (machine: virt)
- OpenSBI v0.5-51
- U-Boot 2020.01-rc5
- grub 2.04
- linux-kernel v5.4
- openSUSE-Tumbleweed-20191103
Quoted 5 lines
Thanks a lot for tackling this problem - it's been on the back burner for way too long :). Unfortunately this patch set loads grub via UEFI, but then does not execute Linux using the UEFI protocol. While that's a nice hack for starters, it severely limits the extensibility of the boot flow going forward.
IIRC either Anup or Atish wanted to work on a UEFI boot stub for Linux. We could then just unify the ARM and RISC-V UEFI boot paths in grub and use that common code to run Linux via the UEFI stub.
Yes. I am working on it. In fact, I got Linux kernel booting via bootefi command last week. I have tried to use as much as ARM stub code possible which will help in unifying them in future.
I am yet to add UEFI run time services. But I was thinking to post a RFC series with EFI stub code first and work on run time services after that. Let me know if you think that’s not a good idea.
Quoted 5 lines
Thanks,
Alex
Quoted 19 lines · Atish Patra
Sent from my iPhone
Quoted 16 lines · Alexander Graf
Hi Chester,
Quoted 11 lines · Chester Lin
Implement an initial version of riscv loader and related commands to load and run linux kernel and initrd on RISC-V. I tested this series based on the following configuration:
- QEMU 4.2.50 (machine: virt)
- OpenSBI v0.5-51
- U-Boot 2020.01-rc5
- grub 2.04
- linux-kernel v5.4
- openSUSE-Tumbleweed-20191103
Quoted 8 lines
Quoted 5 lines
Thanks a lot for tackling this problem - it's been on the back burner for way too long :). Unfortunately this patch set loads grub via UEFI, but then does not execute Linux using the UEFI protocol. While that's a nice hack for starters, it severely limits the extensibility of the boot flow going forward.
IIRC either Anup or Atish wanted to work on a UEFI boot stub for Linux. We could then just unify the ARM and RISC-V UEFI boot paths in grub and use that common code to run Linux via the UEFI stub.
Yes. I am working on it. In fact, I got Linux kernel booting via bootefi command last week. I have tried to use as much as ARM stub code possible which will help in unifying them in future.
I am yet to add UEFI run time services. But I was thinking to post a RFC series with EFI stub code first and work on run time services after that. Let me know if you think that’s not a good idea.
I think that's a great idea. It will also unblock any work to move this patch to boot using the UEFI protocol.
Alex
Hi Alex, Atish and Anup,
Quoted 28 lines · Alexander Graf
Quoted 26 lines · Atish Patra
Sent from my iPhone
Quoted 16 lines · Alexander Graf
Hi Chester,
Quoted 11 lines · Chester Lin
Implement an initial version of riscv loader and related commands to load and run linux kernel and initrd on RISC-V. I tested this series based on the following configuration:
- QEMU 4.2.50 (machine: virt)
- OpenSBI v0.5-51
- U-Boot 2020.01-rc5
- grub 2.04
- linux-kernel v5.4
- openSUSE-Tumbleweed-20191103Quoted 5 lines
Thanks a lot for tackling this problem - it's been on the back burner for way too long :). Unfortunately this patch set loads grub via UEFI, but then does not execute Linux using the UEFI protocol. While that's a nice hack for starters, it severely limits the extensibility of the boot flow going forward.
IIRC either Anup or Atish wanted to work on a UEFI boot stub for Linux. We could then just unify the ARM and RISC-V UEFI boot paths in grub and use that common code to run Linux via the UEFI stub.
Yes. I am working on it. In fact, I got Linux kernel booting via bootefi command last week. I have tried to use as much as ARM stub code possible which will help in unifying them in future.
Quoted 9 lines
Quoted 1 line
I am yet to add UEFI run time services. But I was thinking to post a RFC series with EFI stub code first and work on run time services after that. Let me know if you think that’s not a good idea.
I think that's a great idea. It will also unblock any work to move this patch to boot using the UEFI protocol.
Alex
It's a huge progress! thank you for letting me know about this great news. BTW, it seems that u-boot uses a specific approach to unblock non-boot harts which are looped by the wfi command. I wonder if we have any plan to move this operation to the OS side? For example, let non-boot harts keep waiting until linux kernel unblocks them. It seems that HSM (Hart State Managment Extension) is still under development, would we rely on this extension to implement CPU online/offline functions for RISC-V?
I raise this question because I think it could be complicated for grub to manage non-boot harts which are blocked in u-boot stage if no general data structure or boottime service provided by the previous bootloader [e.g, u-boot or UEFI FW]. Otherwise the efi-call start_image should specifically handle it but that also means UEFI or other kinds of bootloader must follow it as well.
Thanks for your patience,
Chester
Quoted 54 lines · Chester Lin
Hi Alex, Atish and Anup,
Quoted 43 lines · Alexander Graf
Quoted 33 lines · Atish Patra
Sent from my iPhone
Quoted 25 lines
On Jan 16, 2020, at 10:06 PM, Alexander Graf <[email protected]> wrote:
Hi Chester,
Quoted 7 lines · Chester Lin
Implement an initial version of riscv loader and related commands to load
and run linux kernel and initrd on RISC-V. I tested this series based on
the following configuration:Quoted 6 lines
- QEMU 4.2.50 (machine: virt)
- OpenSBI v0.5-51
- U-Boot 2020.01-rc5
- grub 2.04
- linux-kernel v5.4
- openSUSE-Tumbleweed-20191103Thanks a lot for tackling this problem - it's been on the back burner for way too long :). Unfortunately this patch set loads grub via UEFI, but then does not execute Linux using the UEFI protocol. While that's a nice hack for starters, it severely limits the extensibility of the boot flow going forward.
Quoted 6 lines
IIRC either Anup or Atish wanted to work on a UEFI boot stub for Linux. We could then just unify the ARM and RISC-V UEFI boot paths in grub and use that common code to run Linux via the UEFI stub.
Quoted 8 lines
Yes. I am working on it. In fact, I got Linux kernel booting via bootefi command last week. I have tried to use as much as ARM stub code possible which will help in unifying them in future.
I am yet to add UEFI run time services. But I was thinking to post a RFC series with EFI stub code first and work on run time services after that. Let me know if you think that’s not a good idea.
Quoted 7 lines
I think that's a great idea. It will also unblock any work to move this
patch to boot using the UEFI protocol.Alex
Quoted 14 lines
It's a huge progress! thank you for letting me know about this great news. BTW,
it seems that u-boot uses a specific approach to unblock non-boot harts which are
looped by the wfi command. I wonder if we have any plan to move this operation
to the OS side? For example, let non-boot harts keep waiting until linux kernel
unblocks them. It seems that HSM (Hart State Managment Extension) is still under
development, would we rely on this extension to implement CPU online/offline
functions for RISC-V?
Yes. I am currently working on HSM extension in Linux. OpenSBI implementation is done.
Quoted 10 lines
I raise this question because I think it could be complicated for grub to manage
non-boot harts which are blocked in u-boot stage if no general data structure or
boottime service provided by the previous bootloader [e.g, u-boot or UEFI FW].
Otherwise the efi-callstart_imageshould specifically handle it but that also
means UEFI or other kinds of bootloader must follow it as well.
With HSM, single core will boot all the way to Linux and bring up all the other harts via hsm calls from OpenSBI. There will be only single hart available in U-Boot during booting. As a result, grub won't need to manage non-boot harts.
My current plan is to send HSM patches first and send EFI stub patches once I test them on top of HSM series to boot all the harts.
Quoted 2 lines
Thanks for your patience,
Chester
Signature · 2 lines
Regards, Atish
Quoted 10 lines · Atish Patra
Quoted 1 line
Thanks a lot for tackling this problem - it's been on the back
burner for way too long :). Unfortunately this patch set loads grub via UEFI, but then does not execute Linux using the UEFI protocol. While that's a nice hack for starters, it severely limits the extensibility of the boot flow going forward.
Quoted 3 lines
IIRC either Anup or Atish wanted to work on a UEFI boot stub for Linux. We could then just unify the ARM and RISC-V UEFI boot paths in grub and use that common code to run Linux via the UEFI stub.
Quoted 7 lines
Yes. I am working on it. In fact, I got Linux kernel booting via bootefi command last week. I have tried to use as much as ARM stub code possible which will help in unifying them in future.
I am yet to add UEFI run time services. But I was thinking to post a RFC series with EFI stub code first and work on run time services after that. Let me know if you think that’s not a good idea.
Absolutely. Could you cc me when you send that to linux-efi list?
FWIW, I intend to get back to unifying the RISV-V linux loader back into the one that started as the arm64 one. But I have recently changed jobs, and there is some paperwork to sort out with the FSF.
Best Regards,
Leif
PATCH 1/2 RISC-V: Correct linux headers' definitions
In reply to Chester LinAdjust the image hedaers based on the current definition in RISC-V LINUX.
--- include/grub/riscv32/linux.h | 4 ++-- include/grub/riscv64/linux.h | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-)
include/grub/riscv32/linux.h +2 -2
@@ -19,7 +19,7 @@#ifndef GRUB_RISCV32_LINUX_HEADER #define GRUB_RISCV32_LINUX_HEADER 1 -#define GRUB_LINUX_RISCV_MAGIC_SIGNATURE 0x52534356/* 'RSCV' */ +#define GRUB_LINUX_RISCV_MAGIC_SIGNATURE 0x05435352/* 'RSC\x5' */ /* From linux/Documentation/riscv/booting.txt */ struct linux_riscv_kernel_header
@@ -32,7 +32,7 @@ struct linux_riscv_kernel_headergrub_uint64_t res2;/* reserved */ grub_uint64_t res3;/* reserved */ grub_uint64_t res4;/* reserved */ -grub_uint32_t magic;/* Magic number, little endian, "RSCV" */ +grub_uint32_t magic;/* Magic number, little endian, "RSC\x05" */ grub_uint32_t hdr_offset;/* Offset of PE/COFF header */ };
include/grub/riscv64/linux.h +3 -3
@@ -19,7 +19,7 @@#ifndef GRUB_RISCV64_LINUX_HEADER #define GRUB_RISCV64_LINUX_HEADER 1 -#define GRUB_LINUX_RISCV_MAGIC_SIGNATURE 0x52534356/* 'RSCV' */ +#define GRUB_LINUX_RISCV_MAGIC_SIGNATURE 0x05435352/* 'RSC\x5' */ #define GRUB_EFI_PE_MAGIC 0x5A4D
@@ -32,9 +32,9 @@ struct linux_riscv_kernel_headergrub_uint64_t res0;/* reserved */ grub_uint64_t res1;/* reserved */ grub_uint64_t res2;/* reserved */ -grub_uint64_t res3;/* reserved */ +grub_uint64_t res3;/* reserved */ grub_uint64_t res4;/* reserved */ -grub_uint32_t magic;/* Magic number, little endian, "RSCV" */ +grub_uint32_t magic;/* Magic number, little endian, "RSC\x05" */ grub_uint32_t hdr_offset;/* Offset of PE/COFF header */ };
Signature · 1 line
2.24.0
PATCH 2/2 RISC-V: Implement linux image loader
In reply to Chester LinAn image loader for booting RISC-V linux. The major idea is based on loader/arm64/linux.c but still follows the the current boot procedure of RISC-V Linux. [boot-entry = dram-base + text-offset]
--- grub-core/loader/riscv/linux.c | 311 ++++++++++++++++++++++++++++++++- 1 file changed, 307 insertions(+), 4 deletions(-)
grub-core/loader/riscv/linux.c +307 -4
@@ -15,21 +15,240 @@* You should have received a copy of the GNU General Public License * along with GRUB. If not, see <http://www.gnu.org/licenses/>. */ - +#include <grub/charset.h> #include <grub/command.h> -#include <grub/dl.h> +#include <grub/err.h> +#include <grub/file.h> +#include <grub/fdt.h> +#include <grub/linux.h> +#include <grub/loader.h> +#include <grub/mm.h> +#include <grub/types.h> +#include <grub/cpu/linux.h> +#include <grub/efi/efi.h> +#include <grub/efi/fdtload.h> +#include <grub/efi/memory.h> +#include <grub/efi/pe32.h> +#include <grub/i18n.h> #include <grub/lib/cmdline.h> +#include <grub/verify.h> GRUB_MOD_LICENSE ("GPLv3+" ); +static grub_dl_t my_mod; +static int loaded; + +static void *kernel_addr; +static grub_size_t kernel_size; + +static char *linux_args; +static grub_size_t cmdline_size; + +static grub_addr_t initrd_start; +static grub_addr_t initrd_end; + +#if __riscv_xlen ==64 +#define INITRD_MAX_ADDRESS_OFFSET (32ULL * 1024 * 1024 * 1024) +#else +#define INITRD_MAX_ADDRESS_OFFSET (512U * 1024 * 1024) +#endif + +#define get_hartid (__v) \ +{ \ + __asm__ __volatile__ ("mv %0, tp" \ + : "=r" (__v) : \ + : "memory"); \ +} + +typedef void (*kernel_entry_t ) (int ,void *); + +grub_err_t +grub_arch_efi_linux_check_image (struct linux_arch_kernel_header * lh) +{ +if (lh->magic !=GRUB_LINUX_RISCV_MAGIC_SIGNATURE ) +return grub_error (GRUB_ERR_BAD_OS ,"invalid magic number" ); + +grub_dprintf ("linux" ,"UEFI stub kernel:\n" ); +grub_dprintf ("linux" ,"PE/COFF header @ %08x\n" , lh->hdr_offset); + +return GRUB_ERR_NONE ; +} + +static grub_err_t +prepare_fdt (void **fdt_addr) +{ +int node, retval; +void *fdt; + + fdt =grub_fdt_load (GRUB_EFI_LINUX_FDT_EXTRA_SPACE ); + +if (!fdt) + goto failure; + + node =grub_fdt_find_subnode (fdt,0 ,"chosen" ); +if (node <0 ) + node =grub_fdt_add_subnode (fdt,0 ,"chosen" ); + +if (node <1 ) + goto failure; + +/* Set initrd info */ +if (initrd_start && initrd_end > initrd_start) + { +grub_dprintf ("linux" ,"Initrd @ %p-%p\n" , + (void *) initrd_start, (void *) initrd_end); + + retval =grub_fdt_set_prop64 (fdt, node,"linux,initrd-start" , + initrd_start); +if (retval) + goto failure; + + retval =grub_fdt_set_prop64 (fdt, node,"linux,initrd-end" , + initrd_end); +if (retval) + goto failure; + } + +if (grub_fdt_install () !=GRUB_ERR_NONE ) + goto failure; + + *fdt_addr = fdt; + +return GRUB_ERR_NONE ; + +failure: +grub_fdt_unload (); +return grub_error (GRUB_ERR_BAD_OS ,"failed to install/update FDT" ); +} + +static grub_err_t +grub_linux_boot (void ) +{ + registerunsigned long hartid; +kernel_entry_t boot_kernel; +void *fdt_addr =NULL ; + +if (!kernel_addr) +return grub_error (GRUB_ERR_BAD_OS ,"kernel image is not loaded" ); + + boot_kernel = (kernel_entry_t ) kernel_addr; + +if (prepare_fdt (&fdt_addr) !=GRUB_ERR_NONE ) + goto failure; + +get_hartid (hartid); + +grub_dprintf ("linux" ,"Boot Hart: %lu\n" , hartid); + +boot_kernel (hartid, fdt_addr); + +failure: +return grub_error (GRUB_ERR_BAD_OS ,"failed to boot kernel" ); +} + + + +static grub_err_t +grub_linux_unload (void ) +{ +grub_dl_unref (my_mod); + loaded =0 ; +if (initrd_start) +grub_efi_free_pages ((grub_efi_physical_address_t ) initrd_start, +GRUB_EFI_BYTES_TO_PAGES (initrd_end - initrd_start)); + initrd_start = initrd_end =0 ; +grub_free (linux_args); +if (kernel_addr) +grub_efi_free_pages ((grub_addr_t ) kernel_addr, +GRUB_EFI_BYTES_TO_PAGES (kernel_size)); +grub_fdt_unload (); +return GRUB_ERR_NONE ; +} + +/* +* This function returns a pointer to a legally allocated initrd buffer, +* or NULL if unsuccessful +*/ +static void * +allocate_kernel_mem (int kernel_pages,grub_addr_t offset) +{ +grub_addr_t max_addr; + +if (grub_efi_get_ram_base (&max_addr) !=GRUB_ERR_NONE ) +return NULL ; + + max_addr += offset; + +return grub_efi_allocate_pages_real (max_addr, kernel_pages, +GRUB_EFI_ALLOCATE_ADDRESS , +GRUB_EFI_LOADER_DATA ); +} + +static void * +allocate_initrd_mem (int initrd_pages) +{ +grub_addr_t max_addr; + +if (grub_efi_get_ram_base (&max_addr) !=GRUB_ERR_NONE ) +return NULL ; + + max_addr +=INITRD_MAX_ADDRESS_OFFSET -1 ; + +return grub_efi_allocate_pages_real (max_addr, initrd_pages, +GRUB_EFI_ALLOCATE_MAX_ADDRESS , +GRUB_EFI_LOADER_DATA ); +} +static grub_err_t grub_cmd_initrd (grub_command_t cmd __attribute__ ((unused)),int argc __attribute__ ((unused)),char *argv[] __attribute__ ((unused))) { -grub_error (GRUB_ERR_NOT_IMPLEMENTED_YET ,N_ ("Linux not supported yet" )); +struct grub_linux_initrd_context initrd_ctx = {0 ,0 ,0 }; +int initrd_size, initrd_pages; +void *initrd_mem =NULL ; + +if (argc ==0 ) + { +grub_error (GRUB_ERR_BAD_ARGUMENT ,N_ ("filename expected" )); + goto fail; + } + +if (!loaded) + { +grub_error (GRUB_ERR_BAD_ARGUMENT , +N_ ("you need to load the kernel first" )); + goto fail; + } + +if (grub_initrd_init (argc, argv, &initrd_ctx)) + goto fail; + + initrd_size =grub_get_initrd_size (&initrd_ctx); +grub_dprintf ("linux" ,"Loading initrd\n" ); + + initrd_pages = (GRUB_EFI_BYTES_TO_PAGES (initrd_size)); + initrd_mem =allocate_initrd_mem (initrd_pages); + +if (!initrd_mem) + { +grub_error (GRUB_ERR_OUT_OF_MEMORY ,N_ ("out of memory" )); + goto fail; + } + +if (grub_initrd_load (&initrd_ctx, argv, initrd_mem)) + goto fail; + + initrd_start = (grub_addr_t ) initrd_mem; + initrd_end = initrd_start + initrd_size; + + fail: +grub_initrd_close (&initrd_ctx); +if (initrd_mem && !initrd_start) +grub_efi_free_pages ((grub_addr_t ) initrd_mem, initrd_pages);return grub_errno; + } static grub_err_t
@@ -37,9 +256,92 @@ grub_cmd_linux (grub_command_t cmd __attribute__ ((unused)), int argc __attribute__ ((unused)), char *argv[] __attribute__ ((unused))) { -grub_error (GRUB_ERR_NOT_IMPLEMENTED_YET ,N_ ("Linux not supported yet" )); +grub_file_t file =0 ; +struct linux_arch_kernel_header lh; +grub_err_t err; + +grub_dl_ref (my_mod); + +if (argc ==0 ) + { +grub_error (GRUB_ERR_BAD_ARGUMENT ,N_ ("filename expected" )); + goto fail; + } + + file =grub_file_open (argv[0 ],GRUB_FILE_TYPE_LINUX_KERNEL ); +if (!file) + goto fail; + + kernel_size =grub_file_size (file); + +if (grub_file_read (file, &lh,sizeof (lh)) < (long )sizeof (lh)) +return grub_errno; + +if (grub_arch_efi_linux_check_image (&lh) !=GRUB_ERR_NONE ) + goto fail; + +grub_loader_unset (); + + kernel_addr =allocate_kernel_mem ( +GRUB_EFI_BYTES_TO_PAGES (kernel_size), lh.text_offset); + +if (!kernel_addr) + { +grub_error (GRUB_ERR_OUT_OF_MEMORY ,N_ ("out of memory" )); + goto fail; + } + +grub_file_seek (file,0 ); +if (grub_file_read (file, kernel_addr, kernel_size) + < (grub_ssize_t ) kernel_size) + { +if (!grub_errno) +grub_error (GRUB_ERR_BAD_OS ,N_ ("premature end of file %s" ), argv[0 ]); + goto fail; + } + +grub_dprintf ("linux" ,"kernel @ %p\n" , kernel_addr); + + cmdline_size =grub_loader_cmdline_size (argc, argv) +sizeof (LINUX_IMAGE ); + linux_args =grub_malloc (cmdline_size); +if (!linux_args) + { +grub_error (GRUB_ERR_OUT_OF_MEMORY ,N_ ("out of memory" )); + goto fail; + } +grub_memcpy (linux_args,LINUX_IMAGE ,sizeof (LINUX_IMAGE )); + err =grub_create_loader_cmdline (argc, argv, + linux_args +sizeof (LINUX_IMAGE ) -1 , + cmdline_size, +GRUB_VERIFY_KERNEL_CMDLINE ); +if (err) + goto fail; + +if (grub_errno ==GRUB_ERR_NONE ) + { +grub_loader_set (grub_linux_boot, grub_linux_unload,0 ); + loaded =1 ; + } + +fail: +if (file) +grub_file_close (file); + +if (grub_errno !=GRUB_ERR_NONE ) + { +grub_dl_unref (my_mod); + loaded =0 ; + } + +if (linux_args && !loaded) +grub_free (linux_args); + +if (kernel_addr && !loaded) +grub_efi_free_pages ((grub_addr_t ) kernel_addr, +GRUB_EFI_BYTES_TO_PAGES (kernel_size));return grub_errno; + }static grub_command_t cmd_linux, cmd_initrd;
@@ -50,6 +352,7 @@ GRUB_MOD_INIT (linux) N_("Load Linux.")); cmd_initrd =grub_register_command ("initrd" , grub_cmd_initrd,0 ,N_ ("Load initrd." )); + my_mod = mod; }GRUB_MOD_FINI (linux)
Signature · 1 line
2.24.0