Skip to contentSkip to conversation

Initial support for Power

PATCH In review Shawn Anastasio 3 participants 7 messages Jun 20, 2023

lore
Share & tools
lore
#0 lore raw

v5 cover Initial support for Power

Hello all,

This patch series adds support for building a minimal image for Power ISA 2.07B+ (POWER8+) systems. In addition to a patch adding support to the build system and a simple infinite loop at the entrypoint, patches to add ppc64le support to the CI as well as a MAINTAINERS update are included.

Since Xen previously had support for a much older version of the ISA in version 3.2.3, we were able to carry over some headers and support routines from that version. Unlike that initial port though, this effort focuses solely on POWER8+ CPUs that are capable of running in Little Endian mode.

With an appropriate powerpc64le-linux-gnu cross-toolchain, the minimal image can be built with:

$ make XEN_TARGET_ARCH=ppc64 -C xen build

The resulting binary can then be booted in a standard QEMU/pseries VM:

$ qemu-system-ppc64 -M pseries-5.2 -m 256M -kernel xen/xen \
	-vga none -serial mon:stdio -nographic

Thanks,
Shawn

--
Changes from v5:
- Rename openpower_defconfig to ppc64_defconfig
- Clarify Kconfig ISA baseline help strings
- Add ELF metadata annotations to head.S
- Add dummy .got/.got.plt sections plus assertions to ld script
- Change .align to .p2align in config.h/ENTRY()

Changes from v4:
- Change '$(@D)/$(@F)' to '$@' in ppc/Makefile
- Add -m64, -mlittle-endian to CFLAGS to allow using ppc{32,64be} toolchains
- Change IBM-specific '$' to '.' in head.S
- Drop unnecessary ALIGN() in linker script in .bss

Changes from v3:
- Fix formatting of MAINTAINERS patch

Changes from v2:
- Add ppc64le cross-build container patch
- Add ppc64le cross build CI job patch
- Drop serial output patch (will be in future patch series)
- Drop setup.c and unneeded headers from minimal build patch
- Fixed ordering of MAINTAINERS patch + add F: line
- Fix config/ppc64.mk option names
- Clarify Kconfig Baseline ISA option help strings

Shawn Anastasio (3):
xen: Add files needed for minimal ppc64le build
automation: Add ppc64le cross-build jobs
maintainers: Add ppc64 maintainer

 MAINTAINERS                          |   4 +
 automation/gitlab-ci/build.yaml      |  60 +++++++++
 config/ppc64.mk                      |   4 +
 xen/Makefile                         |   5 +-
 xen/arch/ppc/Kconfig                 |  42 ++++++
 xen/arch/ppc/Kconfig.debug           |   0
 xen/arch/ppc/Makefile                |  16 +++
 xen/arch/ppc/Rules.mk                |   0
 xen/arch/ppc/arch.mk                 |  12 ++
 xen/arch/ppc/configs/ppc64_defconfig |  13 ++
 xen/arch/ppc/include/asm/config.h    |  63 +++++++++
 xen/arch/ppc/include/asm/page-bits.h |   7 +
 xen/arch/ppc/ppc64/Makefile          |   1 +
 xen/arch/ppc/ppc64/asm-offsets.c     |   0
 xen/arch/ppc/ppc64/head.S            |  30 +++++
 xen/arch/ppc/xen.lds.S               | 188 +++++++++++++++++++++++++++
 16 files changed, 443 insertions(+), 2 deletions(-)
 create mode 100644 config/ppc64.mk
 create mode 100644 xen/arch/ppc/Kconfig
 create mode 100644 xen/arch/ppc/Kconfig.debug
 create mode 100644 xen/arch/ppc/Makefile
 create mode 100644 xen/arch/ppc/Rules.mk
 create mode 100644 xen/arch/ppc/arch.mk
 create mode 100644 xen/arch/ppc/configs/ppc64_defconfig
 create mode 100644 xen/arch/ppc/include/asm/config.h
 create mode 100644 xen/arch/ppc/include/asm/page-bits.h
 create mode 100644 xen/arch/ppc/ppc64/Makefile
 create mode 100644 xen/arch/ppc/ppc64/asm-offsets.c
 create mode 100644 xen/arch/ppc/ppc64/head.S
 create mode 100644 xen/arch/ppc/xen.lds.S

--
2.30.2

#1 lore raw

PATCH v5 1/3 xen: Add files needed for minimal ppc64le build

Add the build system changes required to build for ppc64le (POWER8+). As of now the resulting image simply boots to an infinite loop.

$ make XEN_TARGET_ARCH=ppc64 -C xen build

This port targets POWER8+ CPUs running in Little Endian mode specifically, and does not boot on older machines. Additionally, this initial skeleton only implements the PaPR/pseries boot protocol which allows it to be booted in a standard QEMU virtual machine:

$ qemu-system-ppc64 -M pseries-5.2 -m 256M -kernel xen/xen
---
 config/ppc64.mk                      |   4 +
 xen/Makefile                         |   5 +-
 xen/arch/ppc/Kconfig                 |  42 ++++++
 xen/arch/ppc/Kconfig.debug           |   0
 xen/arch/ppc/Makefile                |  16 +++
 xen/arch/ppc/Rules.mk                |   0
 xen/arch/ppc/arch.mk                 |  12 ++
 xen/arch/ppc/configs/ppc64_defconfig |  13 ++
 xen/arch/ppc/include/asm/config.h    |  63 +++++++++
 xen/arch/ppc/include/asm/page-bits.h |   7 +
 xen/arch/ppc/ppc64/Makefile          |   1 +
 xen/arch/ppc/ppc64/asm-offsets.c     |   0
 xen/arch/ppc/ppc64/head.S            |  30 +++++
 xen/arch/ppc/xen.lds.S               | 188 +++++++++++++++++++++++++++
 14 files changed, 379 insertions(+), 2 deletions(-)
 create mode 100644 config/ppc64.mk
 create mode 100644 xen/arch/ppc/Kconfig
 create mode 100644 xen/arch/ppc/Kconfig.debug
 create mode 100644 xen/arch/ppc/Makefile
 create mode 100644 xen/arch/ppc/Rules.mk
 create mode 100644 xen/arch/ppc/arch.mk
 create mode 100644 xen/arch/ppc/configs/ppc64_defconfig
 create mode 100644 xen/arch/ppc/include/asm/config.h
 create mode 100644 xen/arch/ppc/include/asm/page-bits.h
 create mode 100644 xen/arch/ppc/ppc64/Makefile
 create mode 100644 xen/arch/ppc/ppc64/asm-offsets.c
 create mode 100644 xen/arch/ppc/ppc64/head.S
 create mode 100644 xen/arch/ppc/xen.lds.S
config/ppc64.mk new file +4 -0
new file mode 100644
index 0000000000..05f74bb306
@@ -0,0 +1,4 @@
+CONFIG_PPC := y
+CONFIG_PPC64 := y
+
+CONFIG_XEN_INSTALL_SUFFIX :=
xen/Makefile +3 -2
index e89fc461fc..db5454fb58 100644
@@ -38,7 +38,7 @@ EFI_MOUNTPOINT ?= $(BOOT_DIR)/efi
 ARCH=$(XEN_TARGET_ARCH)
 SRCARCH=$(shell echo $(ARCH) | \
           sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
-              -e s'/riscv.*/riscv/g')
+              -e s'/riscv.*/riscv/g' -e s'/ppc.*/ppc/g')
 export ARCH SRCARCH
 
 # Allow someone to change their config file
@@ -244,7 +244,7 @@ include $(XEN_ROOT)/Config.mk
 export TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
 export TARGET_ARCH     := $(shell echo $(XEN_TARGET_ARCH) | \
                             sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
-                                -e s'/riscv.*/riscv/g')
+                                -e s'/riscv.*/riscv/g' -e s'/ppc.*/ppc/g')
 
 export CONFIG_SHELL := $(SHELL)
 export CC CXX LD NM OBJCOPY OBJDUMP ADDR2LINE
@@ -563,6 +563,7 @@ _clean:
 	$(Q)$(MAKE) $(clean)=xsm
 	$(Q)$(MAKE) $(clean)=crypto
 	$(Q)$(MAKE) $(clean)=arch/arm
+	$(Q)$(MAKE) $(clean)=arch/ppc
 	$(Q)$(MAKE) $(clean)=arch/riscv
 	$(Q)$(MAKE) $(clean)=arch/x86
 	$(Q)$(MAKE) $(clean)=test
xen/arch/ppc/Kconfig new file +42 -0

42 lines not on this page · raw message

xen/arch/ppc/Kconfig.debug new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
xen/arch/ppc/Makefile new file +16 -0
new file mode 100644
index 0000000000..98220648af
@@ -0,0 +1,16 @@
+obj-$(CONFIG_PPC64) += ppc64/
+
+$(TARGET): $(TARGET)-syms
+	cp -f $< $@
+
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
+	$(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+	$(NM) -pa --format=sysv $@ \
+		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort \
+		> [email protected]
+
+$(obj)/xen.lds: $(src)/xen.lds.S FORCE
+	$(call if_changed_dep,cpp_lds_S)
+
+.PHONY: include
+include:
xen/arch/ppc/Rules.mk new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
xen/arch/ppc/arch.mk new file +12 -0
new file mode 100644
index 0000000000..36830457c6
@@ -0,0 +1,12 @@
+########################################
+# Power-specific definitions
+
+ppc-march-$(CONFIG_POWER_ISA_2_07B) := power8
+ppc-march-$(CONFIG_POWER_ISA_3_00) := power9
+
+CFLAGS += -m64 -mlittle-endian -mcpu=$(ppc-march-y)
+CFLAGS += -mstrict-align -mcmodel=large -mabi=elfv2 -mno-altivec -mno-vsx
+
+# TODO: Drop override when more of the build is working
+override ALL_OBJS-y = arch/$(TARGET_ARCH)/built_in.o
+override ALL_LIBS-y =
xen/arch/ppc/configs/ppc64_defconfig new file +13 -0
new file mode 100644
index 0000000000..8783eb3488
@@ -0,0 +1,13 @@
+# CONFIG_SCHED_CREDIT is not set
+# CONFIG_SCHED_RTDS is not set
+# CONFIG_SCHED_NULL is not set
+# CONFIG_SCHED_ARINC653 is not set
+# CONFIG_TRACEBUFFER is not set
+# CONFIG_HYPFS is not set
+# CONFIG_GRANT_TABLE is not set
+# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
+
+CONFIG_PPC64=y
+CONFIG_DEBUG=y
+CONFIG_DEBUG_INFO=y
+CONFIG_EXPERT=y
xen/arch/ppc/include/asm/config.h new file +63 -0

63 lines not on this page · raw message

xen/arch/ppc/include/asm/page-bits.h new file +7 -0
new file mode 100644
index 0000000000..4c01bf9716
@@ -0,0 +1,7 @@
+#ifndef __PPC_PAGE_BITS_H__
+#define __PPC_PAGE_BITS_H__
+
+#define PAGE_SHIFT              16 /* 64 KiB Pages */
+#define PADDR_BITS              48
+
+#endif /* __PPC_PAGE_BITS_H__ */
xen/arch/ppc/ppc64/Makefile new file +1 -0
new file mode 100644
index 0000000000..3340058c08
@@ -0,0 +1 @@
+obj-y += head.o
xen/arch/ppc/ppc64/asm-offsets.c new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
xen/arch/ppc/ppc64/head.S new file +30 -0
new file mode 100644
index 0000000000..7be949aa57
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+.section .text.header, "ax", %progbits
+
+ENTRY(start)
+    /*
+     * Depending on how we were booted, the CPU could be running in either
+     * Little Endian or Big Endian mode. The following trampoline from Linux
+     * cleverly uses an instruction that encodes to a NOP if the CPU's
+     * endianness matches the assumption of the assembler (LE, in our case)
+     * or a branch to code that performs the endian switch in the other case.
+     */
+    tdi 0, 0, 0x48    /* Reverse endian of b . + 8          */
+    b . + 44          /* Skip trampoline if endian is good  */
+    .long 0xa600607d  /* mfmsr r11                          */
+    .long 0x01006b69  /* xori r11,r11,1                     */
+    .long 0x00004039  /* li r10,0                           */
+    .long 0x6401417d  /* mtmsrd r10,1                       */
+    .long 0x05009f42  /* bcl 20,31,$+4                      */
+    .long 0xa602487d  /* mflr r10                           */
+    .long 0x14004a39  /* addi r10,r10,20                    */
+    .long 0xa6035a7d  /* mtsrr0 r10                         */
+    .long 0xa6037b7d  /* mtsrr1 r11                         */
+    .long 0x2400004c  /* rfid                               */
+
+    /* Now that the endianness is confirmed, continue */
+1:  b 1b
+
+.size start, . - start
+.type start, %function
xen/arch/ppc/xen.lds.S new file +188 -0

188 lines not on this page · raw message

Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH v5 1/3] xen: Add files needed for minimal ppc64le build

Quoted 5 lines · Shawn Anastasio
xen/arch/ppc/xen.lds.S new file +0 -0
new file mode 100644
index 0000000000..f75d9be4ed

xen/arch/ppc/xen.lds.S line 25

Quoted 9 lines
xen/arch/ppc/xen.lds.S new file +8 -0
<snip>
+/**
+ * OF's base load address is 0x400000 (XEN_VIRT_START).
+ * By defining sections this way, we can keep our virtual address base at 0x400000
+ * while keeping the physical base at 0x0.
+ *
+ * Without this, OF incorrectly loads .text at 0x400000 + 0x400000 = 0x800000.
+ * Taken from x86/xen.lds.S
+ */

Sorry, one last thing.

What's OF?  Searching around suggests it might be OpenFirmware, but I'm not certain if that's applicable in this context either?

~Andrew

#3 lore raw

Re: [PATCH v5 1/3] xen: Add files needed for minimal ppc64le build

----- Original Message -----

Quoted 27 lines

From: "Andrew Cooper" <[email protected]>
To: "Shawn Anastasio" <[email protected]>, [email protected]
Cc: "Timothy Pearson" <[email protected]>, "George Dunlap" <[email protected]>, "Jan Beulich" <[email protected]>, "Julien Grall" <[email protected]>, "Stefano Stabellini" <[email protected]>, "Wei Liu" <[email protected]>
Sent: Tuesday, June 20, 2023 4:21:22 PM
Subject: Re: [PATCH v5 1/3] xen: Add files needed for minimal ppc64le build

Quoted 9 lines · Shawn Anastasio
xen/arch/ppc/xen.lds.S new file +3 -0
new file mode 100644
index 0000000000..f75d9be4ed
<snip>
+/**
+ * OF's base load address is 0x400000 (XEN_VIRT_START).
+ * By defining sections this way, we can keep our virtual address base at
Quoted 6 lines

0x400000
+ * while keeping the physical base at 0x0.
+ *
+ * Without this, OF incorrectly loads .text at 0x400000 + 0x400000 = 0x800000.
+ * Taken from x86/xen.lds.S
+ */

Sorry, one last thing.

Quoted 4 lines

What's OF?  Searching around suggests it might be OpenFirmware, but I'm not certain if that's applicable in this context either?

~Andrew

OF is indeed Open Firmware. It was mainly used on old Apple machines, but its legacy lives on in SLOF (SlimLine Open Firmware) which is the QEMU bootloader for the pSeries machine type.

#4 lore raw

Re: [PATCH v5 1/3] xen: Add files needed for minimal ppc64le build

Quoted 18 lines · Timothy Pearson
Quoted 18 lines · Shawn Anastasio
Quoted 9 lines
xen/arch/ppc/xen.lds.S new file +3 -0
new file mode 100644
index 0000000000..f75d9be4ed
<snip>
+/**
+ * OF's base load address is 0x400000 (XEN_VIRT_START).
+ * By defining sections this way, we can keep our virtual address base at
Quoted 6 lines

0x400000
+ * while keeping the physical base at 0x0.
+ *
+ * Without this, OF incorrectly loads .text at 0x400000 + 0x400000 = 0x800000.
+ * Taken from x86/xen.lds.S
+ */

Sorry, one last thing.

Quoted 5 lines
Quoted 4 lines

What's OF?  Searching around suggests it might be OpenFirmware, but I'm not certain if that's applicable in this context either?

~Andrew

OF is indeed Open Firmware. It was mainly used on old Apple machines, but its legacy lives on in SLOF (SlimLine Open Firmware) which is the QEMU bootloader for the pSeries machine type.

Ok.  Seeing as everything else appears to be in order, I'll expand this abbreviation on commit for the benefit of others reading the comment.

~Andrew

#5 lore raw

PATCH v5 2/3 automation: Add ppc64le cross-build jobs

In reply to Shawn Anastasio

Add build jobs to cross-compile Xen for ppc64le.

---
 automation/gitlab-ci/build.yaml | 60 +++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
automation/gitlab-ci/build.yaml +60 -0
index 420ffa5acb..bd8c7332db 100644
@@ -183,6 +183,33 @@
   variables:
     <<: *gcc
 
+.ppc64le-cross-build-tmpl:
+  <<: *build
+  variables:
+    XEN_TARGET_ARCH: ppc64
+  tags:
+    - x86_64
+
+.ppc64le-cross-build:
+  extends: .ppc64le-cross-build-tmpl
+  variables:
+    debug: n
+
+.ppc64le-cross-build-debug:
+  extends: .ppc64le-cross-build-tmpl
+  variables:
+    debug: y
+
+.gcc-ppc64le-cross-build:
+  extends: .ppc64le-cross-build
+  variables:
+    <<: *gcc
+
+.gcc-ppc64le-cross-build-debug:
+  extends: .ppc64le-cross-build-debug
+  variables:
+    <<: *gcc
+
 .yocto-test:
   stage: build
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
@@ -516,6 +543,39 @@ archlinux-current-gcc-riscv64-debug-randconfig:
     EXTRA_FIXED_RANDCONFIG:
       CONFIG_COVERAGE=n
 
+# Power cross-build
+debian-bullseye-gcc-ppc64le:
+  extends: .gcc-ppc64le-cross-build
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    HYPERVISOR_ONLY: y
+
+debian-bullseye-gcc-ppc64le-debug:
+  extends: .gcc-ppc64le-cross-build-debug
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    HYPERVISOR_ONLY: y
+
+debian-bullseye-gcc-ppc64le-randconfig:
+  extends: .gcc-ppc64le-cross-build
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    RANDCONFIG: y
+    EXTRA_FIXED_RANDCONFIG:
+      CONFIG_COVERAGE=n
+
+debian-bullseye-gcc-ppc64le-debug-randconfig:
+  extends: .gcc-ppc64le-cross-build-debug
+  variables:
+    CONTAINER: debian:bullseye-ppc64le
+    KBUILD_DEFCONFIG: openpower_defconfig
+    RANDCONFIG: y
+    EXTRA_FIXED_RANDCONFIG:
+      CONFIG_COVERAGE=n
+
 # Yocto test jobs
 yocto-qemuarm64:
   extends: .yocto-test-arm64
Signature · 1 line
2.30.2
#6 lore raw

PATCH v5 3/3 maintainers: Add ppc64 maintainer

In reply to Shawn Anastasio
---
 MAINTAINERS | 4 ++++
 1 file changed, 4 insertions(+)
MAINTAINERS +4 -0
index 7a07949e1d..64c9cc89ed 100644
@@ -458,6 +458,10 @@ X:	xen/arch/x86/acpi/lib.c
 F:	xen/drivers/cpufreq/
 F:	xen/include/acpi/cpufreq/
 
+PPC64
+M:	Shawn Anastasio <[email protected]>
+F:	xen/arch/ppc/
+
 PUBLIC I/O INTERFACES AND PV DRIVERS DESIGNS
 M:	Juergen Gross <[email protected]>
 S:	Supported
Signature · 1 line
2.30.2