Skip to contentSkip to conversation

Add minimal RISC-V Xen build and build testing

PATCH Superseded Oleksii Kurochko 2 participants 7 messages Dec 30, 2022 – Jan 5, 2023 RISC-V

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#1 lore raw

PATCH v2 1/2 arch/riscv: initial RISC-V support to build/run minimal Xen

In reply to Oleksii Kurochko

The patch provides a minimal amount of changes to start build and run minimal Xen binary at GitLab CI&CD that will allow continuous checking of the build status of RISC-V Xen.

Except introduction of new files the following changes were done:
* Redefinition of ALIGN define from '.algin 2' to '.align 4' as RISC-V implementations (except for with C extension) enforce 32-bit instruction address alignment. With C extension, 16-bit and 32-bit are both allowed.
* ALL_OBJ-y and ALL_LIBS-y were temporary overwritted to produce a minimal hypervisor image otherwise it will be required to push huge amount of headers and stubs for common, drivers, libs etc which aren't necessary for now.
* Section changed from .text to .text.header for start function to make it the first one executed.
* Rework riscv64/Makefile logic to rebase over changes since the first RISC-V commit.

RISC-V Xen can be built by the following instructions:

  $ CONTAINER=riscv64 ./automation/scripts/containerize \
       make XEN_TARGET_ARCH=riscv64 tiny64_defconfig
  $ CONTAINER=riscv64 ./automation/scripts/containerize \
       make XEN_TARGET_ARCH=riscv64 -C xen build

RISC-V Xen can be run as:

  $ qemu-system-riscv64 -M virt -smp 1 -nographic -m 2g \
       -kernel xen/xen

To run in debug mode should be done the following instructions:

 $ qemu-system-riscv64 -M virt -smp 1 -nographic -m 2g \
        -kernel xen/xen -s -S
 # In separate terminal:
 $ riscv64-buildroot-linux-gnu-gdb
 $ target remote :1234
 $ add-symbol-file <xen_src>/xen/xen-syms 0x80200000
 $ hb *0x80200000
 $ c # it should stop at instruction j 0x80200000 <start>

---
Changes in V2:
- Update commit message:
- Add explanation why ALIGN define was changed.
- Add explanation why section of 'start' function was changed.
- Rework xen.lds.S linker script. It is mostly based on ARM except ARM-specific sections which have been removed.
- Rework in riscv64/Makefile rule $(TARGET)-syms
- Remove asm/types.h header as after reworking of riscv64/Makefile it is not needed now.
- Remove unneeded define SYMBOLS_DUMMY_OBJ.

---
 xen/arch/riscv/Makefile             |  16 +++
 xen/arch/riscv/arch.mk              |   4 +
 xen/arch/riscv/include/asm/config.h |   9 +-
 xen/arch/riscv/riscv64/Makefile     |   2 +-
 xen/arch/riscv/riscv64/head.S       |   2 +-
 xen/arch/riscv/xen.lds.S            | 158 ++++++++++++++++++++++++++++
 6 files changed, 188 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/riscv/xen.lds.S
xen/arch/riscv/Makefile +16 -0
index 942e4ffbc1..74386beb85 100644
@@ -1,2 +1,18 @@
+obj-$(CONFIG_RISCV_64) += riscv64/
+
+$(TARGET): $(TARGET)-syms
+	$(OBJCOPY) -O binary -S $< $@
+
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(NM) -pa --format=sysv $(@D)/$(@F) \
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
+		>$(@D)/$(@F).map
+
+$(obj)/xen.lds: $(src)/xen.lds.S FORCE
+	$(call if_changed_dep,cpp_lds_S)
+
+clean-files := $(objtree)/.xen-syms.[0-9]*
+
 .PHONY: include
 include:
xen/arch/riscv/arch.mk +4 -0
index ae8fe9dec7..012dc677c3 100644
@@ -11,3 +11,7 @@ riscv-march-$(CONFIG_RISCV_ISA_C)       := $(riscv-march-y)c
 # -mcmodel=medlow would force Xen into the lower half.
 
 CFLAGS += -march=$(riscv-march-y) -mstrict-align -mcmodel=medany
+
+# TODO: Drop override when more of the build is working
+override ALL_OBJS-y = arch/$(TARGET_ARCH)/built_in.o
+override ALL_LIBS-y =
xen/arch/riscv/include/asm/config.h +8 -1
index e2ae21de61..e10e13ba53 100644
@@ -1,6 +1,9 @@
 #ifndef __RISCV_CONFIG_H__
 #define __RISCV_CONFIG_H__
 
+#include <xen/const.h>
+#include <xen/page-size.h>
+
 #if defined(CONFIG_RISCV_64)
 # define LONG_BYTEORDER 3
 # define ELFSIZE 64
@@ -28,7 +31,7 @@
 
 /* Linkage for RISCV */
 #ifdef __ASSEMBLY__
-#define ALIGN .align 2
+#define ALIGN .align 4
 
 #define ENTRY(name)                                \
   .globl name;                                     \
@@ -36,6 +39,10 @@
   name:
 #endif
 
+#define XEN_VIRT_START  _AT(UL,0x00200000)
+
+#define SMP_CACHE_BYTES (1 << 6)
+
 #endif /* __RISCV_CONFIG_H__ */
 /*
  * Local variables:
xen/arch/riscv/riscv64/Makefile +1 -1
index 15a4a65f66..3340058c08 100644
@@ -1 +1 @@
-extra-y += head.o
+obj-y += head.o
xen/arch/riscv/riscv64/head.S +1 -1
index 0dbc27ba75..0330b29c01 100644
@@ -1,6 +1,6 @@
 #include <asm/config.h>
 
-        .text
+        .section .text.header, "ax", %progbits
 
 ENTRY(start)
         j  start
xen/arch/riscv/xen.lds.S new file +158 -0

158 lines not on this page · raw message

Signature · 1 line
2.38.1
#4 lore raw

PATCH v2 2/2 automation: add RISC-V 64 cross-build tests for Xen

In reply to Oleksii Kurochko

Add build jobs to cross-compile Xen-only for RISC-V 64.

---
Changes in V2:
- Add HYPERVISOR_ONLY to RISCV jobs because after rebase on top of the patch series "CI: Fixes/cleanup in preparation for RISCV" it is required to set HYPERVISOR_ONLY in build.yaml

---
 automation/gitlab-ci/build.yaml | 45 +++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)
automation/gitlab-ci/build.yaml +45 -0
index e6a9357de3..11eb1c6b82 100644
@@ -172,6 +172,33 @@
   variables:
     <<: *gcc
 
+.riscv64-cross-build-tmpl:
+  <<: *build
+  variables:
+    XEN_TARGET_ARCH: riscv64
+  tags:
+    - x86_64
+
+.riscv64-cross-build:
+  extends: .riscv64-cross-build-tmpl
+  variables:
+    debug: n
+
+.riscv64-cross-build-debug:
+  extends: .riscv64-cross-build-tmpl
+  variables:
+    debug: y
+
+.gcc-riscv64-cross-build:
+  extends: .riscv64-cross-build
+  variables:
+    <<: *gcc
+
+.gcc-riscv64-cross-build-debug:
+  extends: .riscv64-cross-build-debug
+  variables:
+    <<: *gcc
+
 # Jobs below this line
 
 archlinux-gcc:
@@ -617,6 +644,21 @@ alpine-3.12-gcc-debug-arm64-boot-cpupools:
     EXTRA_XEN_CONFIG: |
       CONFIG_BOOT_TIME_CPUPOOLS=y
 
+# RISC-V 64 cross-build
+riscv64-cross-gcc:
+  extends: .gcc-riscv64-cross-build
+  variables:
+    CONTAINER: archlinux:riscv64
+    KBUILD_DEFCONFIG: tiny64_defconfig
+    HYPERVISOR_ONLY: y
+
+riscv64-cross-gcc-debug:
+  extends: .gcc-riscv64-cross-build-debug
+  variables:
+    CONTAINER: archlinux:riscv64
+    KBUILD_DEFCONFIG: tiny64_defconfig
+    HYPERVISOR_ONLY: y
+
 ## Test artifacts common
 
 .test-jobs-artifact-common:
@@ -692,3 +734,6 @@ kernel-5.10.74-export:
       - binaries/bzImage
   tags:
     - x86_64
+
+# # RISC-V 64 test artificats
+# # TODO: add RISC-V 64 test artitifacts
Signature · 1 line
2.38.1
#0 lore raw

v2 cover Add minimal RISC-V Xen build and build testing

The patch series introduces the following:
- provide a minimal amount of changes to add initial RISC-V support to make Xen binary buildable and runnable for RISC-V architecture which can be used for future development and testing.
- add RISC-V 64 cross-compile build jobs to check if any new changes break RISC-V build.

The patch series is based on "CI: Fixes/cleanup in preparation for RISCV" patch series which can be found at
https://lists.xenproject.org/archives/html/xen-devel/2022-12/msg01761.html

Changes in V2:
- Remove the patch "automation: add cross-compiler support
for the build script" because it was reworked as a part of the patch series "CI: Fixes/cleanup in preparation for RISCV".
- Remove the patch "automation: add python3 package for riscv64.dockerfile" because it is not necessary for RISCV Xen binary build now.
- Rework the patch "arch/riscv: initial RISC-V support to build/run minimal Xen" according to the comments about v1 of the patch series.
- Add HYPERVISOR_ONLY to RISCV jobs in build.yaml after rebasing on "CI: Fixes/cleanup in preparation for RISCV" patch series.

Oleksii Kurochko (2):
arch/riscv: initial RISC-V support to build/run minimal Xen
automation: add RISC-V 64 cross-build tests for Xen

 automation/gitlab-ci/build.yaml     |  45 ++++++++
 xen/arch/riscv/Makefile             |  16 +++
 xen/arch/riscv/arch.mk              |   4 +
 xen/arch/riscv/include/asm/config.h |   9 +-
 xen/arch/riscv/riscv64/Makefile     |   2 +-
 xen/arch/riscv/riscv64/head.S       |   2 +-
 xen/arch/riscv/xen.lds.S            | 158 ++++++++++++++++++++++++++++
 7 files changed, 233 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/riscv/xen.lds.S
Signature · 1 line
2.38.1
#2 lore raw

Re: [XEN PATCH v2 1/2] arch/riscv: initial RISC-V support to build/run minimal Xen

In reply to Oleksii Kurochko
Quoted 6 lines · Oleksii Kurochko

The patch provides a minimal amount of changes to start build and run minimal Xen binary at GitLab CI&CD that will allow continuous checking of the build status of RISC-V Xen.

Except introduction of new files the following changes were done:
* Redefinition of ALIGN define from '.algin 2' to '.align 4' as

"align"

Quoted 1 line

RISC-V implementations (except for with C extension) enforce 32-bit

While the C extension might mean things to RISC-V experts, it is better to say the "compression" extension here so it's clear to non-RISC-V experts reading the commit message too.

But, do we actually care about C?

ENTRY() needs to be 4 byte aligned because one of the few things it is going to be used for is {m,s}tvec which requires 4-byte alignment even with an IALIGN of 2.

I'd drop all talk about C and just say that 2 was an incorrect choice previously.

Quoted 11 lines

instruction address alignment. With C extension, 16-bit and 32-bit are both allowed.
* ALL_OBJ-y and ALL_LIBS-y were temporary overwritted to produce a minimal hypervisor image otherwise it will be required to push huge amount of headers and stubs for common, drivers, libs etc which aren't necessary for now.
* Section changed from .text to .text.header for start function to make it the first one executed.
* Rework riscv64/Makefile logic to rebase over changes since the first RISC-V commit.

Quoted 3 lines

RISC-V Xen can be built by the following instructions:

  $ CONTAINER=riscv64 ./automation/scripts/containerize \
       make XEN_TARGET_ARCH=riscv64 tiny64_defconfig

This needs a -C xen in this rune.

Quoted 10 lines
xen/arch/riscv/Makefile +5 -0
index 942e4ffbc1..74386beb85 100644
@@ -1,2 +1,18 @@
+obj-$(CONFIG_RISCV_64) += riscv64/
+
+$(TARGET): $(TARGET)-syms
+	$(OBJCOPY) -O binary -S $< $@
+

xen/arch/riscv/Makefile line 15

Quoted 10 lines
xen/arch/riscv/Makefile +10 -0
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(NM) -pa --format=sysv $(@D)/$(@F) \
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
+		>$(@D)/$(@F).map
+
+$(obj)/xen.lds: $(src)/xen.lds.S FORCE
+	$(call if_changed_dep,cpp_lds_S)
+
+clean-files := $(objtree)/.xen-syms.[0-9]*

We don't need clean-files now that the main link has been simplified.

xen/arch/riscv/include/asm/config.h line 42

Quoted 9 lines
xen/arch/riscv/include/asm/config.h +1 -0
index e2ae21de61..e10e13ba53 100644
@@ -36,6 +39,10 @@
   name:
 #endif
 
+#define XEN_VIRT_START  _AT(UL,0x00200000)

Space after the comma.

Otherwise, LGTM.

~Andrew

#5 lore raw

Re: [XEN PATCH v2 2/2] automation: add RISC-V 64 cross-build tests for Xen

In reply to Oleksii Kurochko
Quoted 13 lines · Oleksii Kurochko
automation/gitlab-ci/build.yaml +5 -0
index e6a9357de3..11eb1c6b82 100644
@@ -617,6 +644,21 @@ alpine-3.12-gcc-debug-arm64-boot-cpupools:
     EXTRA_XEN_CONFIG: |
       CONFIG_BOOT_TIME_CPUPOOLS=y
 
+# RISC-V 64 cross-build
+riscv64-cross-gcc:
+  extends: .gcc-riscv64-cross-build
+  variables:
+    CONTAINER: archlinux:riscv64

automation/gitlab-ci/build.yaml alpine-3.12-gcc-debug-arm64-boot-cpupools: · line 661

Quoted 10 lines
automation/gitlab-ci/build.yaml +10 -0
+    KBUILD_DEFCONFIG: tiny64_defconfig
+    HYPERVISOR_ONLY: y
+
+riscv64-cross-gcc-debug:
+  extends: .gcc-riscv64-cross-build-debug
+  variables:
+    CONTAINER: archlinux:riscv64
+    KBUILD_DEFCONFIG: tiny64_defconfig
+    HYPERVISOR_ONLY: y
+

Judging by the Kconfig which gets written out, I suggest inserting the two RANDCONFIG jobs right now.

automation/gitlab-ci/build.yaml kernel-5.10.74-export: · line 739

Quoted 10 lines
automation/gitlab-ci/build.yaml +3 -0
 ## Test artifacts common
 
 .test-jobs-artifact-common:
@@ -692,3 +734,6 @@ kernel-5.10.74-export:
       - binaries/bzImage
   tags:
     - x86_64
+
+# # RISC-V 64 test artificats
+# # TODO: add RISC-V 64 test artitifacts

Drop this hunk.  All you're going to be doing is deleting it in the next series...

~Andrew

author
#3 lore raw

Re: [XEN PATCH v2 1/2] arch/riscv: initial RISC-V support to build/run minimal Xen

In reply to Andrew Cooper
Quoted 18 lines · Andrew Cooper
Quoted 6 lines · Oleksii Kurochko

The patch provides a minimal amount of changes to start build and run minimal Xen binary at GitLab CI&CD that will allow continuous checking of the build status of RISC-V Xen.

Except introduction of new files the following changes were done:
* Redefinition of ALIGN define from '.algin 2' to '.align 4' as

"align"

Quoted 2 lines

  RISC-V implementations (except for with C extension) enforce 32- bit

While the C extension might mean things to RISC-V experts, it is better
to say the "compression" extension here so it's clear to non-RISC-V experts reading the commit message too.

Quoted 8 lines

But, do we actually care about C?

ENTRY() needs to be 4 byte aligned because one of the few things it is
going to be used for is {m,s}tvec which requires 4-byte alignment even
with an IALIGN of 2.

We don't care about C (at least now).

Quoted 62 lines

I'd drop all talk about C and just say that 2 was an incorrect choice previously.

Quoted 14 lines

  instruction address alignment.  With C extension, 16-bit and 32- bit
  are both allowed.
* ALL_OBJ-y and ALL_LIBS-y were temporary overwritted to produce a minimal hypervisor image otherwise it will be required to push huge amount of headers and stubs for common, drivers, libs etc which
  aren't necessary for now.
* Section changed from .text to .text.header for start function to make it the first one executed.
* Rework riscv64/Makefile logic to rebase over changes since the first
  RISC-V commit.

Quoted 3 lines

RISC-V Xen can be built by the following instructions:

  $ CONTAINER=riscv64 ./automation/scripts/containerize \
       make XEN_TARGET_ARCH=riscv64 tiny64_defconfig

This needs a -C xen in this rune.

Quoted 17 lines
xen/arch/riscv/Makefile +7 -0
index 942e4ffbc1..74386beb85 100644
@@ -1,2 +1,18 @@
+obj-$(CONFIG_RISCV_64) += riscv64/
+
+$(TARGET): $(TARGET)-syms
+       $(OBJCOPY) -O binary -S $< $@
+
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
+       $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $<

$(build_id_linker) -o $@
+       $(NM) -pa --format=sysv $(@D)/$(@F) \
+               | $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
+               >$(@D)/$(@F).map

xen/arch/riscv/Makefile line 15

Quoted 5 lines
xen/arch/riscv/Makefile +5 -0
+
+$(obj)/xen.lds: $(src)/xen.lds.S FORCE
+       $(call if_changed_dep,cpp_lds_S)
+
+clean-files := $(objtree)/.xen-syms.[0-9]*

We don't need clean-files now that the main link has been simplified.

Quoted 10 lines
xen/arch/riscv/include/asm/config.h +0 -0
index e2ae21de61..e10e13ba53 100644
@@ -36,6 +39,10 @@

   name:
 #endif

+#define XEN_VIRT_START  _AT(UL,0x00200000)

Space after the comma.

Quoted 2 lines

Otherwise, LGTM.

Thanks for the comments.
They were fixed in patch series v4:
[PATCH v4 0/2] Add minimal RISC-V Xen build and build testing

~ Oleksii

Quoted 1 line

~Andrew

author
#6 lore raw

Re: [XEN PATCH v2 2/2] automation: add RISC-V 64 cross-build tests for Xen

In reply to Andrew Cooper
Quoted 30 lines · Andrew Cooper
Quoted 14 lines · Oleksii Kurochko
automation/gitlab- renamed from automation/gitlab-ci/build.yaml +0 -0

ci/build.yaml
index e6a9357de3..11eb1c6b82 100644

automation/gitlab-ci/build.yaml +0 -0
@@ -617,6 +644,21 @@ alpine-3.12-gcc-debug-arm64-boot-cpupools:

     EXTRA_XEN_CONFIG: |
       CONFIG_BOOT_TIME_CPUPOOLS=y

automation/gitlab-ci/build.yaml +5 -0
+# RISC-V 64 cross-build
+riscv64-cross-gcc:
+  extends: .gcc-riscv64-cross-build
+  variables:
+    CONTAINER: archlinux:riscv64

automation/gitlab-ci/build.yaml alpine-3.12-gcc-debug-arm64-boot-cpupools: · line 661

Quoted 10 lines
automation/gitlab-ci/build.yaml +10 -0
+    KBUILD_DEFCONFIG: tiny64_defconfig
+    HYPERVISOR_ONLY: y
+
+riscv64-cross-gcc-debug:
+  extends: .gcc-riscv64-cross-build-debug
+  variables:
+    CONTAINER: archlinux:riscv64
+    KBUILD_DEFCONFIG: tiny64_defconfig
+    HYPERVISOR_ONLY: y
+

Judging by the Kconfig which gets written out, I suggest inserting the
two RANDCONFIG jobs right now.

Quoted 15 lines

automation/gitlab-ci/build.yaml kernel-5.10.74-export: · line 739

Quoted 10 lines

 ## Test artifacts common

 .test-jobs-artifact-common:

automation/gitlab-ci/build.yaml +0 -0
@@ -692,3 +734,6 @@ kernel-5.10.74-export:

       - binaries/bzImage
   tags:
     - x86_64

automation/gitlab-ci/build.yaml +3 -0
+
+# # RISC-V 64 test artificats
+# # TODO: add RISC-V 64 test artitifacts

Drop this hunk.  All you're going to be doing is deleting it in the next
series...

Thanks for the comments.
They were fixed in patch series v4:
[PATCH v4 2/2] automation: add RISC-V 64 cross-build tests for Xen

~ Oleksii

Quoted 1 line

~Andrew