Skip to contentSkip to conversation

Initial support for Power

PATCH Superseded Shawn Anastasio 2 participants 8 messages Jun 12, 2023 – Jun 13, 2023

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Initial support for Power

Hello all,

This patch series adds support for building a minimal image (head.o-only) for Power ISA 2.07B+ (POWER8+) systems. The first patch boots to an infinite loop and the second adds early serial console support on pseries VMs, with bare metal support planned next.

Since Xen previously had support for a much older version of the ISA in version 3.2.3, we were able to carry over some headers and support routines from that version. Unlike that initial port though, this effort focuses solely on POWER8+ CPUs that are capable of running in Little Endian mode.

With an appropriate powerpc64le-linux-gnu cross-toolchain, the minimal image can be built with:

$ make XEN_TARGET_ARCH=ppc64 -C xen openpower_defconfig
$ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen TARGET=ppc64/head.o

The resulting head.o can then be booted in a standard QEMU/pseries VM:

$ qemu-system-ppc64 -M pseries-5.2 -m 256M -kernel xen/ppc64/head.o \
	-vga none -serial mon:stdio -nographic

Thanks,
Shawn

--
Changes from v2:
- Add ppc64le cross-build container patch
- Add ppc64le cross build CI job patch
- Drop serial output patch (will be in future patch series)
- Drop setup.c and unneeded headers from minimal build patch
- Fixed ordering of MAINTAINERS patch + add F: line
- Fix config/ppc64.mk option names
- Clarify Kconfig Baseline ISA option help strings

Shawn Anastasio (4):
automation: Add container for ppc64le builds
xen: Add files needed for minimal ppc64le build
automation: Add ppc64le cross-build jobs
maintainers: Add ppc64 maintainer

 MAINTAINERS                                   |   4 +
 .../build/debian/bullseye-ppc64le.dockerfile  |  28 +++
 automation/gitlab-ci/build.yaml               |  60 ++++++
 automation/scripts/containerize               |   1 +
 config/ppc64.mk                               |   5 +
 xen/Makefile                                  |   5 +-
 xen/arch/ppc/Kconfig                          |  42 +++++
 xen/arch/ppc/Kconfig.debug                    |   0
 xen/arch/ppc/Makefile                         |  16 ++
 xen/arch/ppc/Rules.mk                         |   0
 xen/arch/ppc/arch.mk                          |  11 ++
 xen/arch/ppc/configs/openpower_defconfig      |  13 ++
 xen/arch/ppc/include/asm/config.h             |  63 +++++++
 xen/arch/ppc/include/asm/page-bits.h          |   7 +
 xen/arch/ppc/ppc64/Makefile                   |   1 +
 xen/arch/ppc/ppc64/asm-offsets.c              |   0
 xen/arch/ppc/ppc64/head.S                     |  27 +++
 xen/arch/ppc/xen.lds.S                        | 173 ++++++++++++++++++
 18 files changed, 454 insertions(+), 2 deletions(-)
 create mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile
 create mode 100644 config/ppc64.mk
 create mode 100644 xen/arch/ppc/Kconfig
 create mode 100644 xen/arch/ppc/Kconfig.debug
 create mode 100644 xen/arch/ppc/Makefile
 create mode 100644 xen/arch/ppc/Rules.mk
 create mode 100644 xen/arch/ppc/arch.mk
 create mode 100644 xen/arch/ppc/configs/openpower_defconfig
 create mode 100644 xen/arch/ppc/include/asm/config.h
 create mode 100644 xen/arch/ppc/include/asm/page-bits.h
 create mode 100644 xen/arch/ppc/ppc64/Makefile
 create mode 100644 xen/arch/ppc/ppc64/asm-offsets.c
 create mode 100644 xen/arch/ppc/ppc64/head.S
 create mode 100644 xen/arch/ppc/xen.lds.S
Signature · 1 line
2.30.2
#1 lore raw

PATCH v2 1/4 automation: Add container for ppc64le builds

Add a container for cross-compiling xen for ppc64le.

---
 .../build/debian/bullseye-ppc64le.dockerfile  | 28 +++++++++++++++++++
 automation/scripts/containerize               |  1 +
 2 files changed, 29 insertions(+)
 create mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile
automation/build/debian/bullseye-ppc64le.dockerfile new file +28 -0
new file mode 100644
index 0000000000..8a87631b52
@@ -0,0 +1,28 @@
+FROM debian:bullseye-slim
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="[email protected]"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+# Add compiler path
+ENV CROSS_COMPILE powerpc64le-linux-gnu-
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes --no-install-recommends install \
+        bison \
+        build-essential \
+        checkpolicy \
+        flex \
+        gawk \
+        gcc-powerpc64le-linux-gnu \
+        make \
+        python3-minimal \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
automation/scripts/containerize +1 -0
index 5476ff0ea1..6d46f63665 100755
@@ -33,6 +33,7 @@ case "_${CONTAINER}" in
     _focal) CONTAINER="${BASE}/ubuntu:focal" ;;
     _jessie) CONTAINER="${BASE}/debian:jessie" ;;
     _jessie-i386) CONTAINER="${BASE}/debian:jessie-i386" ;;
+    _bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;;
     _stretch|_) CONTAINER="${BASE}/debian:stretch" ;;
     _stretch-i386) CONTAINER="${BASE}/debian:stretch-i386" ;;
     _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
Signature · 1 line
2.30.2
#2 lore raw

PATCH v2 2/4 xen: Add files needed for minimal ppc64le build

In reply to Shawn Anastasio

Add the build system changes required to build for ppc64le (POWER8+). As of now the resulting image simply boots to an infinite loop.

$ make XEN_TARGET_ARCH=ppc64 -C xen openpower_defconfig
$ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen build

This port targets POWER8+ CPUs running in Little Endian mode specifically, and does not boot on older machines. Additionally, this initial skeleton only implements the PaPR/pseries boot protocol which allows it to be booted in a standard QEMU virtual machine:

$ qemu-system-ppc64 -M pseries-5.2 -m 256M -kernel xen/xen
---
 config/ppc64.mk                          |   5 +
 xen/Makefile                             |   5 +-
 xen/arch/ppc/Kconfig                     |  42 ++++++
 xen/arch/ppc/Kconfig.debug               |   0
 xen/arch/ppc/Makefile                    |  16 +++
 xen/arch/ppc/Rules.mk                    |   0
 xen/arch/ppc/arch.mk                     |  11 ++
 xen/arch/ppc/configs/openpower_defconfig |  13 ++
 xen/arch/ppc/include/asm/config.h        |  63 +++++++++
 xen/arch/ppc/include/asm/page-bits.h     |   7 +
 xen/arch/ppc/ppc64/Makefile              |   1 +
 xen/arch/ppc/ppc64/asm-offsets.c         |   0
 xen/arch/ppc/ppc64/head.S                |  27 ++++
 xen/arch/ppc/xen.lds.S                   | 173 +++++++++++++++++++++++
 14 files changed, 361 insertions(+), 2 deletions(-)
 create mode 100644 config/ppc64.mk
 create mode 100644 xen/arch/ppc/Kconfig
 create mode 100644 xen/arch/ppc/Kconfig.debug
 create mode 100644 xen/arch/ppc/Makefile
 create mode 100644 xen/arch/ppc/Rules.mk
 create mode 100644 xen/arch/ppc/arch.mk
 create mode 100644 xen/arch/ppc/configs/openpower_defconfig
 create mode 100644 xen/arch/ppc/include/asm/config.h
 create mode 100644 xen/arch/ppc/include/asm/page-bits.h
 create mode 100644 xen/arch/ppc/ppc64/Makefile
 create mode 100644 xen/arch/ppc/ppc64/asm-offsets.c
 create mode 100644 xen/arch/ppc/ppc64/head.S
 create mode 100644 xen/arch/ppc/xen.lds.S
config/ppc64.mk new file +5 -0
new file mode 100644
index 0000000000..597f0668c3
@@ -0,0 +1,5 @@
+CONFIG_PPC := y
+CONFIG_PPC64 := y
+CONFIG_PPC_$(XEN_OS) := y
+
+CONFIG_XEN_INSTALL_SUFFIX :=
xen/Makefile +3 -2
index e89fc461fc..db5454fb58 100644
@@ -38,7 +38,7 @@ EFI_MOUNTPOINT ?= $(BOOT_DIR)/efi
 ARCH=$(XEN_TARGET_ARCH)
 SRCARCH=$(shell echo $(ARCH) | \
           sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
-              -e s'/riscv.*/riscv/g')
+              -e s'/riscv.*/riscv/g' -e s'/ppc.*/ppc/g')
 export ARCH SRCARCH
 
 # Allow someone to change their config file
@@ -244,7 +244,7 @@ include $(XEN_ROOT)/Config.mk
 export TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
 export TARGET_ARCH     := $(shell echo $(XEN_TARGET_ARCH) | \
                             sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
-                                -e s'/riscv.*/riscv/g')
+                                -e s'/riscv.*/riscv/g' -e s'/ppc.*/ppc/g')
 
 export CONFIG_SHELL := $(SHELL)
 export CC CXX LD NM OBJCOPY OBJDUMP ADDR2LINE
@@ -563,6 +563,7 @@ _clean:
 	$(Q)$(MAKE) $(clean)=xsm
 	$(Q)$(MAKE) $(clean)=crypto
 	$(Q)$(MAKE) $(clean)=arch/arm
+	$(Q)$(MAKE) $(clean)=arch/ppc
 	$(Q)$(MAKE) $(clean)=arch/riscv
 	$(Q)$(MAKE) $(clean)=arch/x86
 	$(Q)$(MAKE) $(clean)=test
xen/arch/ppc/Kconfig new file +42 -0

42 lines not on this page · raw message

xen/arch/ppc/Kconfig.debug new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
xen/arch/ppc/Makefile new file +16 -0
new file mode 100644
index 0000000000..10b101cf9c
@@ -0,0 +1,16 @@
+obj-$(CONFIG_PPC64) += ppc64/
+
+$(TARGET): $(TARGET)-syms
+	cp -f $< $@
+
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(NM) -pa --format=sysv $(@D)/$(@F) \
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
+		>$(@D)/$(@F).map
+
+$(obj)/xen.lds: $(src)/xen.lds.S FORCE
+	$(call if_changed_dep,cpp_lds_S)
+
+.PHONY: include
+include:
xen/arch/ppc/Rules.mk new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
xen/arch/ppc/arch.mk new file +11 -0
new file mode 100644
index 0000000000..c185a5028a
@@ -0,0 +1,11 @@
+########################################
+# Power-specific definitions
+
+ppc-march-$(CONFIG_POWER_ISA_2_07B) := power8
+ppc-march-$(CONFIG_POWER_ISA_3_00) := power9
+
+CFLAGS += -mcpu=$(ppc-march-y) -mstrict-align -mcmodel=large -mabi=elfv2 -mno-altivec -mno-vsx
+
+# TODO: Drop override when more of the build is working
+override ALL_OBJS-y = arch/$(TARGET_ARCH)/built_in.o
+override ALL_LIBS-y =
xen/arch/ppc/configs/openpower_defconfig new file +13 -0
new file mode 100644
index 0000000000..8783eb3488
@@ -0,0 +1,13 @@
+# CONFIG_SCHED_CREDIT is not set
+# CONFIG_SCHED_RTDS is not set
+# CONFIG_SCHED_NULL is not set
+# CONFIG_SCHED_ARINC653 is not set
+# CONFIG_TRACEBUFFER is not set
+# CONFIG_HYPFS is not set
+# CONFIG_GRANT_TABLE is not set
+# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
+
+CONFIG_PPC64=y
+CONFIG_DEBUG=y
+CONFIG_DEBUG_INFO=y
+CONFIG_EXPERT=y
xen/arch/ppc/include/asm/config.h new file +63 -0

63 lines not on this page · raw message

xen/arch/ppc/include/asm/page-bits.h new file +7 -0
new file mode 100644
index 0000000000..4c01bf9716
@@ -0,0 +1,7 @@
+#ifndef __PPC_PAGE_BITS_H__
+#define __PPC_PAGE_BITS_H__
+
+#define PAGE_SHIFT              16 /* 64 KiB Pages */
+#define PADDR_BITS              48
+
+#endif /* __PPC_PAGE_BITS_H__ */
xen/arch/ppc/ppc64/Makefile new file +1 -0
new file mode 100644
index 0000000000..3340058c08
@@ -0,0 +1 @@
+obj-y += head.o
xen/arch/ppc/ppc64/asm-offsets.c new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
xen/arch/ppc/ppc64/head.S new file +27 -0
new file mode 100644
index 0000000000..e24331f95c
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+.section .text.header, "ax", %progbits
+
+ENTRY(start)
+    /*
+     * Depending on how we were booted, the CPU could be running in either
+     * Little Endian or Big Endian mode. The following trampoline from Linux
+     * cleverly uses an instruction that encodes to a NOP if the CPU's
+     * endianness matches the assumption of the assembler (LE, in our case)
+     * or a branch to code that performs the endian switch in the other case.
+     */
+    tdi 0, 0, 0x48    /* Reverse endian of b . + 8          */
+    b $ + 44          /* Skip trampoline if endian is good  */
+    .long 0xa600607d  /* mfmsr r11                          */
+    .long 0x01006b69  /* xori r11,r11,1                     */
+    .long 0x00004039  /* li r10,0                           */
+    .long 0x6401417d  /* mtmsrd r10,1                       */
+    .long 0x05009f42  /* bcl 20,31,$+4                      */
+    .long 0xa602487d  /* mflr r10                           */
+    .long 0x14004a39  /* addi r10,r10,20                    */
+    .long 0xa6035a7d  /* mtsrr0 r10                         */
+    .long 0xa6037b7d  /* mtsrr1 r11                         */
+    .long 0x2400004c  /* rfid                               */
+
+    /* Now that the endianness is confirmed, continue */
+1:  b 1b
xen/arch/ppc/xen.lds.S new file +173 -0

173 lines not on this page · raw message

Signature · 1 line
2.30.2
#3 lore raw

PATCH v2 3/4 automation: Add ppc64le cross-build jobs

In reply to Shawn Anastasio

Add build jobs to cross-compile Xen for ppc64le.

---
 automation/gitlab-ci/build.yaml | 60 +++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
automation/gitlab-ci/build.yaml +60 -0
index 420ffa5acb..bd8c7332db 100644
@@ -183,6 +183,33 @@
   variables:
     <<: *gcc
 
+.ppc64le-cross-build-tmpl:
+  <<: *build
+  variables:
+    XEN_TARGET_ARCH: ppc64
+  tags:
+    - x86_64
+
+.ppc64le-cross-build:
+  extends: .ppc64le-cross-build-tmpl
+  variables:
+    debug: n
+
+.ppc64le-cross-build-debug:
+  extends: .ppc64le-cross-build-tmpl
+  variables:
+    debug: y
+
+.gcc-ppc64le-cross-build:
+  extends: .ppc64le-cross-build
+  variables:
+    <<: *gcc
+
+.gcc-ppc64le-cross-build-debug:
+  extends: .ppc64le-cross-build-debug
+  variables:
+    <<: *gcc
+
 .yocto-test:
   stage: build
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
@@ -516,6 +543,39 @@ archlinux-current-gcc-riscv64-debug-randconfig:
     EXTRA_FIXED_RANDCONFIG:
       CONFIG_COVERAGE=n
 
+# Power cross-build
+debian-bullseye-gcc-ppc64le:
+  extends: .gcc-ppc64le-cross-build
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    HYPERVISOR_ONLY: y
+
+debian-bullseye-gcc-ppc64le-debug:
+  extends: .gcc-ppc64le-cross-build-debug
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    HYPERVISOR_ONLY: y
+
+debian-bullseye-gcc-ppc64le-randconfig:
+  extends: .gcc-ppc64le-cross-build
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    RANDCONFIG: y
+    EXTRA_FIXED_RANDCONFIG:
+      CONFIG_COVERAGE=n
+
+debian-bullseye-gcc-ppc64le-debug-randconfig:
+  extends: .gcc-ppc64le-cross-build-debug
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    RANDCONFIG: y
+    EXTRA_FIXED_RANDCONFIG:
+      CONFIG_COVERAGE=n
+
 # Yocto test jobs
 yocto-qemuarm64:
   extends: .yocto-test-arm64
Signature · 1 line
2.30.2
#4 lore raw

PATCH v2 4/4 maintainers: Add ppc64 maintainer

In reply to Shawn Anastasio
---
 MAINTAINERS | 4 ++++
 1 file changed, 4 insertions(+)
MAINTAINERS +4 -0
index 1bb7a6a839..8966175400 100644
@@ -460,6 +460,10 @@ X:	xen/arch/x86/acpi/lib.c
 F:	xen/drivers/cpufreq/
 F:	xen/include/acpi/cpufreq/
 
+PPC64
+M:	Shawn Anastasio <[email protected]>
+F:  xen/arch/ppc
+
 PUBLIC I/O INTERFACES AND PV DRIVERS DESIGNS
 M:	Juergen Gross <[email protected]>
 S:	Supported
Signature · 1 line
2.30.2
#5 lore raw

Re: [PATCH v2 4/4] maintainers: Add ppc64 maintainer

MAINTAINERS X: xen/arch/x86/acpi/lib.c · line 465

Quoted 9 lines · Shawn Anastasio
MAINTAINERS +3 -0
@@ -460,6 +460,10 @@ X:	xen/arch/x86/acpi/lib.c
 F:	xen/drivers/cpufreq/
 F:	xen/include/acpi/cpufreq/
 
+PPC64
+M:	Shawn Anastasio <[email protected]>
+F:  xen/arch/ppc

I'm sorry, but two nits again: This lack a trailing slash, and padding is done using spaces on the 2nd line instead of a tab.

Jan

#6 lore raw

Re: [PATCH v2 4/4] maintainers: Add ppc64 maintainer

Quoted 7 lines · Jan Beulich
Quoted 6 lines · Shawn Anastasio
MAINTAINERS +0 -0
@@ -460,6 +460,10 @@ X:	xen/arch/x86/acpi/lib.c
 F:	xen/drivers/cpufreq/
 F:	xen/include/acpi/cpufreq/
 
Quoted 8 lines

MAINTAINERS X: xen/arch/x86/acpi/lib.c · line 465

Quoted 3 lines
MAINTAINERS +3 -0
+PPC64
+M:	Shawn Anastasio <[email protected]>
+F:  xen/arch/ppc

I'm sorry, but two nits again: This lack a trailing slash, and padding is done using spaces on the 2nd line instead of a tab.

Jan

Oops, sorry about that. Will fix in v3.

Thanks,
Shawn

#7 lore raw
In reply to Shawn Anastasio
Quoted 5 lines · Shawn Anastasio

With an appropriate powerpc64le-linux-gnu cross-toolchain, the minimal image can be built with:

$ make XEN_TARGET_ARCH=ppc64 -C xen openpower_defconfig
$ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen TARGET=ppc64/head.o

Minor clarification to this cover letter, the manual TARGET= override is not necessary. All that is needed is:

$ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen build

Thanks,
Shawn