Skip to contentSkip to conversation

s390x: Add snippet support

RFC In review Janosch Frank 4 participants 18 messages May 20, 2021 – Jun 21, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover s390x: Add snippet support

The SIE support allows us to run guests and test the hypervisor's (V)SIE implementation. However it requires that the guest instructions are binary which limits the complexity of the guest code.

The snippet support provides a way to write guest code as ASM or C and simply memcpy it into guest memory. Some of the KVM-unit-test library can be re-used which further speeds up guest code development.

The included mvpg-sie test helped us to deliver the KVM mvpg fixes which Claudio posted a short while ago. In the future I'll post Secure Execution snippet support patches which was my initial goal with this series anyway.

I heard you liked tests so I put tests inside tests so you can test while you test.

Janosch Frank (2):
s390x: Add guest snippet support
s390x: mvpg: Add SIE mvpg test

 .gitignore                      |   2 +
 s390x/Makefile                  |  29 ++++++-
 s390x/mvpg-sie.c                | 139 ++++++++++++++++++++++++++++++++
 s390x/snippets/c/cstart.S       |  13 +++
 s390x/snippets/c/flat.lds       |  51 ++++++++++++
 s390x/snippets/c/mvpg-snippet.c |  33 ++++++++
 s390x/unittests.cfg             |   3 +
 7 files changed, 267 insertions(+), 3 deletions(-)
 create mode 100644 s390x/mvpg-sie.c
 create mode 100644 s390x/snippets/c/cstart.S
 create mode 100644 s390x/snippets/c/flat.lds
 create mode 100644 s390x/snippets/c/mvpg-snippet.c
Signature · 1 line
2.30.2
#1 lore raw

RFC 1/2 s390x: Add guest snippet support

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
 .gitignore                |  2 ++
 s390x/Makefile            | 28 ++++++++++++++++++---
 s390x/snippets/c/cstart.S | 13 ++++++++++
 s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 91 insertions(+), 3 deletions(-)
 create mode 100644 s390x/snippets/c/cstart.S
 create mode 100644 s390x/snippets/c/flat.lds
.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
 /api/dirty-log
 /api/dirty-log-perf
 /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +25 -3
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
 asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
 FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =
+
+snippets = $(snippets-a)$(snippets-c)
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
 	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
 		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
 	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)
 	$(RM) $(@:.elf=.aux.o)
 	@chmod a-x $@
 
@@ -93,7 +115,7 @@ FLATLIBS = $(libcflat)
 	$(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --no-verify --image $< -o $@
 
 arch_clean: asm_offsets_clean
-	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d lib/s390x/.*.d
+	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(SNIPPET_DIR)/c/*.{o,elf,bin,gbin} $(SNIPPET_DIR)/.*.d $(TEST_DIR)/.*.d lib/s390x/.*.d
 
 generated-files = $(asm-offsets)
 $(tests:.elf=.o) $(asmlib) $(cflatobjs): $(generated-files)
s390x/snippets/c/cstart.S new file +13 -0
new file mode 100644
index 00000000..02a3338b
@@ -0,0 +1,13 @@
+#include <asm/sigp.h>
+
+.section .init
+	.globl start
+start:
+	/* XOR all registers with themselves to clear them fully. */
+	.irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
+	xgr \i,\i
+	.endr
+	/* 0x3000 is the stack page for now */
+	lghi	%r15, 0x4000
+	brasl	%r14, main
+	sigp    %r1, %r0, SIGP_STOP
s390x/snippets/c/flat.lds new file +51 -0

51 lines not on this page · raw message

Signature · 1 line
2.30.2
#2 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 33 lines · On Thu, 20 May 2021 09:47:29 +0000 Janosch Frank <[email protected]> wrote:

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
 .gitignore                |  2 ++
 s390x/Makefile            | 28 ++++++++++++++++++---
 s390x/snippets/c/cstart.S | 13 ++++++++++
 s390x/snippets/c/flat.lds | 51

+++++++++++++++++++++++++++++++++++++++ 4 files changed, 91 insertions(+), 3 deletions(-) create mode 100644 s390x/snippets/c/cstart.S create mode 100644 s390x/snippets/c/flat.lds

.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
 /api/dirty-log
 /api/dirty-log-perf
 /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -1
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
 asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
 FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 10 lines
s390x/Makefile +10 -0
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =
+
+snippets = $(snippets-a)$(snippets-c)

↑↑
I'm not a Makefile expert, but, don't you need a space between the two variable expansions?

s390x/Makefile OBJDIRS += lib/s390x · line 100

Quoted 13 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)

I would keep the %.o as the first in the list

s390x/Makefile OBJDIRS += lib/s390x

Quoted 5 lines
s390x/Makefile +1 -1
 	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
 		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
 	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets)

so all the snippets are always baked in every test?

Quoted 91 lines
s390x/Makefile +0 -2
 $(@:.elf=.aux.o) $(RM) $(@:.elf=.aux.o)
 	@chmod a-x $@
 
@@ -93,7 +115,7 @@ FLATLIBS = $(libcflat)
 	$(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT)
--no-verify --image $< -o $@ 
 arch_clean: asm_offsets_clean
-	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d

lib/s390x/.*.d

s390x/Makefile +1 -0
+	$(RM) $(TEST_DIR)/*.{o,elf,bin}

$(SNIPPET_DIR)/c/*.{o,elf,bin,gbin} $(SNIPPET_DIR)/.*.d $(TEST_DIR)/.*.d lib/s390x/.*.d generated-files = $(asm-offsets)
$(tests:.elf=.o) $(asmlib) $(cflatobjs): $(generated-files)

s390x/snippets/c/cstart.S new file +13 -0
new file mode 100644
index 00000000..02a3338b
@@ -0,0 +1,13 @@
+#include <asm/sigp.h>
+
+.section .init
+	.globl start
+start:
+	/* XOR all registers with themselves to clear them fully. */
+	.irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
+	xgr \i,\i
+	.endr
+	/* 0x3000 is the stack page for now */
+	lghi	%r15, 0x4000
+	brasl	%r14, main
+	sigp    %r1, %r0, SIGP_STOP
s390x/snippets/c/flat.lds new file +5 -0
new file mode 100644
index 00000000..5e707325
@@ -0,0 +1,51 @@
+SECTIONS
+{
+	.lowcore : {
+		/*
+		 * Initial short psw for disk boot, with 31 bit

addressing for

s390x/snippets/c/flat.lds new file +1 -0
+		 * non z/Arch environment compatibility and the

instruction

s390x/snippets/c/flat.lds new file +45 -0
+		 * address 0x10000 (cstart64.S .init).
+		 */
+		. = 0;
+		 LONG(0x00080000)
+		 LONG(0x80004000)
+		 /* Restart new PSW for booting via PSW restart. */
+		 . = 0x1a0;
+		 QUAD(0x0000000180000000)
+		 QUAD(0x0000000000004000)
+	}
+	. = 0x4000;
+	.text : {
+		*(.init)
+		*(.text)
+		*(.text.*)
+	}
+	. = ALIGN(64K);
+	etext = .;
+	.opd : { *(.opd) }
+	. = ALIGN(16);
+	.dynamic : {
+		dynamic_start = .;
+		*(.dynamic)
+	}
+	.dynsym : {
+		dynsym_start = .;
+		*(.dynsym)
+	}
+	.rela.dyn : { *(.rela*) }
+	. = ALIGN(16);
+	.data : {
+		*(.data)
+		*(.data.rel*)
+	}
+	. = ALIGN(16);
+	.rodata : { *(.rodata) *(.rodata.*) }
+	. = ALIGN(16);
+	__bss_start = .;
+	.bss : { *(.bss) }
+	__bss_end = .;
+	. = ALIGN(64K);
+	edata = .;
+	. += 64K;
+	. = ALIGN(64K);
+}
#3 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 34 lines · Claudio Imbrenda
Quoted 31 lines · On Thu, 20 May 2021 09:47:29 +0000 Janosch Frank <[email protected]> wrote:

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
 .gitignore                |  2 ++
 s390x/Makefile            | 28 ++++++++++++++++++---
 s390x/snippets/c/cstart.S | 13 ++++++++++
 s390x/snippets/c/flat.lds | 51

+++++++++++++++++++++++++++++++++++++++ 4 files changed, 91 insertions(+), 3 deletions(-) create mode 100644 s390x/snippets/c/cstart.S create mode 100644 s390x/snippets/c/flat.lds

.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
 /api/dirty-log
 /api/dirty-log-perf
 /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
 asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 16 lines

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 12 lines
s390x/Makefile +10 -1
 FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =
+
+snippets = $(snippets-a)$(snippets-c)

↑↑
I'm not a Makefile expert, but, don't you need a space between the two variable expansions?

Yup, I already fixed that one

Quoted 15 lines

s390x/Makefile OBJDIRS += lib/s390x · line 100

Quoted 13 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)

I would keep the %.o as the first in the list>

Quoted 7 lines

s390x/Makefile OBJDIRS += lib/s390x

Quoted 5 lines
s390x/Makefile +1 -1
 	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
 		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
 	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets)

so all the snippets are always baked in every test?

Currently yes.
Do you have better ideas?

Quoted 93 lines
Quoted 92 lines
s390x/Makefile +0 -2
 $(@:.elf=.aux.o) $(RM) $(@:.elf=.aux.o)
 	@chmod a-x $@
 
@@ -93,7 +115,7 @@ FLATLIBS = $(libcflat)
 	$(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT)
--no-verify --image $< -o $@ 
 arch_clean: asm_offsets_clean
-	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d

lib/s390x/.*.d

s390x/Makefile +1 -0
+	$(RM) $(TEST_DIR)/*.{o,elf,bin}

$(SNIPPET_DIR)/c/*.{o,elf,bin,gbin} $(SNIPPET_DIR)/.*.d $(TEST_DIR)/.*.d lib/s390x/.*.d generated-files = $(asm-offsets)
$(tests:.elf=.o) $(asmlib) $(cflatobjs): $(generated-files)

s390x/snippets/c/cstart.S new file +13 -0
new file mode 100644
index 00000000..02a3338b
@@ -0,0 +1,13 @@
+#include <asm/sigp.h>
+
+.section .init
+	.globl start
+start:
+	/* XOR all registers with themselves to clear them fully. */
+	.irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
+	xgr \i,\i
+	.endr
+	/* 0x3000 is the stack page for now */
+	lghi	%r15, 0x4000
+	brasl	%r14, main
+	sigp    %r1, %r0, SIGP_STOP
s390x/snippets/c/flat.lds new file +5 -0
new file mode 100644
index 00000000..5e707325
@@ -0,0 +1,51 @@
+SECTIONS
+{
+	.lowcore : {
+		/*
+		 * Initial short psw for disk boot, with 31 bit

addressing for

s390x/snippets/c/flat.lds new file +1 -0
+		 * non z/Arch environment compatibility and the

instruction

s390x/snippets/c/flat.lds new file +45 -0
+		 * address 0x10000 (cstart64.S .init).
+		 */
+		. = 0;
+		 LONG(0x00080000)
+		 LONG(0x80004000)
+		 /* Restart new PSW for booting via PSW restart. */
+		 . = 0x1a0;
+		 QUAD(0x0000000180000000)
+		 QUAD(0x0000000000004000)
+	}
+	. = 0x4000;
+	.text : {
+		*(.init)
+		*(.text)
+		*(.text.*)
+	}
+	. = ALIGN(64K);
+	etext = .;
+	.opd : { *(.opd) }
+	. = ALIGN(16);
+	.dynamic : {
+		dynamic_start = .;
+		*(.dynamic)
+	}
+	.dynsym : {
+		dynsym_start = .;
+		*(.dynsym)
+	}
+	.rela.dyn : { *(.rela*) }
+	. = ALIGN(16);
+	.data : {
+		*(.data)
+		*(.data.rel*)
+	}
+	. = ALIGN(16);
+	.rodata : { *(.rodata) *(.rodata.*) }
+	. = ALIGN(16);
+	__bss_start = .;
+	.bss : { *(.bss) }
+	__bss_end = .;
+	. = ALIGN(64K);
+	edata = .;
+	. += 64K;
+	. = ALIGN(64K);
+}
#4 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

In reply to Janosch Frank
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
  .gitignore                |  2 ++
  s390x/Makefile            | 28 ++++++++++++++++++---
  s390x/snippets/c/cstart.S | 13 ++++++++++
  s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
  4 files changed, 91 insertions(+), 3 deletions(-)

create mode 100644 s390x/snippets/c/cstart.S
create mode 100644 s390x/snippets/c/flat.lds

.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
  /api/dirty-log
  /api/dirty-log-perf
  /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
  asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
  FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 1 line
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Quoted 8 lines
s390x/Makefile +8 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \

s390x/Makefile OBJDIRS += lib/s390x · line 104

Quoted 10 lines
s390x/Makefile +6 -1
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
  	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
  		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
  	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Quoted 21 lines
s390x/Makefile +1 -1
  	$(RM) $(@:.elf=.aux.o)
  	@chmod a-x $@
 
@@ -93,7 +115,7 @@ FLATLIBS = $(libcflat)
  	$(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --no-verify --image $< -o $@
 
  arch_clean: asm_offsets_clean
-	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d lib/s390x/.*.d
+	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(SNIPPET_DIR)/c/*.{o,elf,bin,gbin} $(SNIPPET_DIR)/.*.d $(TEST_DIR)/.*.d lib/s390x/.*.d
 
  generated-files = $(asm-offsets)
  $(tests:.elf=.o) $(asmlib) $(cflatobjs): $(generated-files)
s390x/snippets/c/cstart.S new file +3 -0
new file mode 100644
index 00000000..02a3338b
@@ -0,0 +1,13 @@
+#include <asm/sigp.h>
+
+.section .init

s390x/snippets/c/cstart.S line 13

Quoted 10 lines
s390x/snippets/c/cstart.S new file +10 -0
+	.globl start
+start:
+	/* XOR all registers with themselves to clear them fully. */
+	.irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
+	xgr \i,\i
+	.endr
+	/* 0x3000 is the stack page for now */
+	lghi	%r15, 0x4000
+	brasl	%r14, main
+	sigp    %r1, %r0, SIGP_STOP

I think you should clear r0 before using it here?

Thomas

#5 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 32 lines · Thomas Huth
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
  .gitignore                |  2 ++
  s390x/Makefile            | 28 ++++++++++++++++++---
  s390x/snippets/c/cstart.S | 13 ++++++++++
  s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
  4 files changed, 91 insertions(+), 3 deletions(-)

create mode 100644 s390x/snippets/c/cstart.S
create mode 100644 s390x/snippets/c/flat.lds

.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
  /api/dirty-log
  /api/dirty-log-perf
  /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
  asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 13 lines

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
  FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

Sure

Quoted 4 lines

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 2 lines
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Yes, already fixed that a long while ago
I thought I had sent out a new version already, maybe that was an illusion as I can't seem to find it right now.

Quoted 14 lines
Quoted 14 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
Quoted 8 lines

s390x/Makefile OBJDIRS += lib/s390x · line 104

Quoted 5 lines
s390x/Makefile +1 -1
  	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
  		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
  	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Yes it does.
I'd like to avoid having to specify a makefile rule for every test that uses snippets as we already have more than the mvpg one in the queue.

So I'm having Steffen looking into a solution for this problem. My first idea was to bring the used snippets into the unittests.cfg but I disliked that we then would have compile instructions in another file. Maybe there's a way to include that into the makefile in a clever way?

Quoted 26 lines
Quoted 26 lines
s390x/Makefile +1 -1
  	$(RM) $(@:.elf=.aux.o)
  	@chmod a-x $@
 
@@ -93,7 +115,7 @@ FLATLIBS = $(libcflat)
  	$(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --no-verify --image $< -o $@
 
  arch_clean: asm_offsets_clean
-	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d lib/s390x/.*.d
+	$(RM) $(TEST_DIR)/*.{o,elf,bin} $(SNIPPET_DIR)/c/*.{o,elf,bin,gbin} $(SNIPPET_DIR)/.*.d $(TEST_DIR)/.*.d lib/s390x/.*.d
 
  generated-files = $(asm-offsets)
  $(tests:.elf=.o) $(asmlib) $(cflatobjs): $(generated-files)
s390x/snippets/c/cstart.S new file +7 -0
new file mode 100644
index 00000000..02a3338b
@@ -0,0 +1,13 @@
+#include <asm/sigp.h>
+
+.section .init
+	.globl start
+start:
+	/* XOR all registers with themselves to clear them fully. */
+	.irp i, 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
Quoted 8 lines

s390x/snippets/c/cstart.S line 13

Quoted 6 lines
s390x/snippets/c/cstart.S new file +6 -0
+	xgr \i,\i
+	.endr
+	/* 0x3000 is the stack page for now */
+	lghi	%r15, 0x4000
+	brasl	%r14, main
+	sigp    %r1, %r0, SIGP_STOP

I think you should clear r0 before using it here?

Right. will fix

Quoted 3 lines

Thomas

#6 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 81 lines · Janosch Frank
Quoted 32 lines · Thomas Huth
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
   .gitignore                |  2 ++
   s390x/Makefile            | 28 ++++++++++++++++++---
   s390x/snippets/c/cstart.S | 13 ++++++++++
   s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
   4 files changed, 91 insertions(+), 3 deletions(-)

create mode 100644 s390x/snippets/c/cstart.S
create mode 100644 s390x/snippets/c/flat.lds

.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
   /api/dirty-log
   /api/dirty-log-perf
   /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
   asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 13 lines

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
   FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

Sure

Quoted 4 lines

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 2 lines
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Yes, already fixed that a long while ago
I thought I had sent out a new version already, maybe that was an illusion as I can't seem to find it right now.

Quoted 14 lines
Quoted 14 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
Quoted 8 lines

s390x/Makefile OBJDIRS += lib/s390x · line 104

Quoted 5 lines
s390x/Makefile +1 -1
   	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
   		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
   	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Quoted 8 lines

Yes it does.
I'd like to avoid having to specify a makefile rule for every test that uses snippets as we already have more than the mvpg one in the queue.

So I'm having Steffen looking into a solution for this problem. My first idea was to bring the used snippets into the unittests.cfg but I disliked that we then would have compile instructions in another file. Maybe there's a way to include that into the makefile in a clever way?

I haven't tried, but maybe you could replace the $(snippets) in the last line with

$(wildcard snippets/[email protected])

or something similar?

Thomas

#7 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 91 lines · Thomas Huth
Quoted 81 lines · Janosch Frank
Quoted 32 lines · Thomas Huth
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
   .gitignore                |  2 ++
   s390x/Makefile            | 28 ++++++++++++++++++---
   s390x/snippets/c/cstart.S | 13 ++++++++++
   s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
   4 files changed, 91 insertions(+), 3 deletions(-)

create mode 100644 s390x/snippets/c/cstart.S
create mode 100644 s390x/snippets/c/flat.lds

.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
   /api/dirty-log
   /api/dirty-log-perf
   /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
   asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 13 lines

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
   FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

Sure

Quoted 4 lines

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 2 lines
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Yes, already fixed that a long while ago
I thought I had sent out a new version already, maybe that was an illusion as I can't seem to find it right now.

Quoted 14 lines
Quoted 14 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
Quoted 8 lines

s390x/Makefile OBJDIRS += lib/s390x · line 104

Quoted 5 lines
s390x/Makefile +1 -1
   	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
   		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
   	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Quoted 8 lines

Yes it does.
I'd like to avoid having to specify a makefile rule for every test that uses snippets as we already have more than the mvpg one in the queue.

So I'm having Steffen looking into a solution for this problem. My first idea was to bring the used snippets into the unittests.cfg but I disliked that we then would have compile instructions in another file. Maybe there's a way to include that into the makefile in a clever way?

Quoted 6 lines

I haven't tried, but maybe you could replace the $(snippets) in the last line with

  $(wildcard snippets/[email protected])

or something similar?

That starts falling apart when multiple tests use the same snippet, no?

Quoted 3 lines

Thomas

#8 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 92 lines · Janosch Frank
Quoted 91 lines · Thomas Huth
Quoted 89 lines · Janosch Frank
Quoted 32 lines · Thomas Huth
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
    .gitignore                |  2 ++
    s390x/Makefile            | 28 ++++++++++++++++++---
    s390x/snippets/c/cstart.S | 13 ++++++++++
    s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
    4 files changed, 91 insertions(+), 3 deletions(-)
    create mode 100644 s390x/snippets/c/cstart.S
    create mode 100644 s390x/snippets/c/flat.lds
.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
    /api/dirty-log
    /api/dirty-log-perf
    /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
    asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 13 lines

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
    FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

Sure

Quoted 4 lines

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 2 lines
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Yes, already fixed that a long while ago
I thought I had sent out a new version already, maybe that was an illusion as I can't seem to find it right now.

Quoted 14 lines
Quoted 14 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
Quoted 8 lines

s390x/Makefile OBJDIRS += lib/s390x · line 104

Quoted 5 lines
s390x/Makefile +1 -1
    	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
    		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
    	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Yes it does.
I'd like to avoid having to specify a makefile rule for every test that uses snippets as we already have more than the mvpg one in the queue.

So I'm having Steffen looking into a solution for this problem. My first idea was to bring the used snippets into the unittests.cfg but I disliked that we then would have compile instructions in another file. Maybe there's a way to include that into the makefile in a clever way?

Quoted 8 lines
Quoted 6 lines

I haven't tried, but maybe you could replace the $(snippets) in the last line with

   $(wildcard snippets/[email protected])

or something similar?

That starts falling apart when multiple tests use the same snippet, no?

That's true ... Maybe something like:

$(filter %.gbin,$^)

?

Thomas

#9 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 98 lines · Thomas Huth
Quoted 97 lines · Janosch Frank
Quoted 96 lines · Thomas Huth
Quoted 81 lines · Janosch Frank
Quoted 32 lines · Thomas Huth
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
    .gitignore                |  2 ++
    s390x/Makefile            | 28 ++++++++++++++++++---
    s390x/snippets/c/cstart.S | 13 ++++++++++
    s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
    4 files changed, 91 insertions(+), 3 deletions(-)
    create mode 100644 s390x/snippets/c/cstart.S
    create mode 100644 s390x/snippets/c/flat.lds
.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
    /api/dirty-log
    /api/dirty-log-perf
    /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
    asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 13 lines

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
    FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

Sure

Quoted 4 lines

s390x/Makefile OBJDIRS += lib/s390x · line 87

Quoted 2 lines
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Yes, already fixed that a long while ago
I thought I had sent out a new version already, maybe that was an illusion as I can't seem to find it right now.

Quoted 14 lines
Quoted 14 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
Quoted 8 lines

s390x/Makefile OBJDIRS += lib/s390x · line 104

Quoted 5 lines
s390x/Makefile +1 -1
    	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
    		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
    	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Quoted 8 lines

Yes it does.
I'd like to avoid having to specify a makefile rule for every test that uses snippets as we already have more than the mvpg one in the queue.

So I'm having Steffen looking into a solution for this problem. My first idea was to bring the used snippets into the unittests.cfg but I disliked that we then would have compile instructions in another file. Maybe there's a way to include that into the makefile in a clever way?

I haven't tried, but maybe you could replace the $(snippets) in the last line with

   $(wildcard snippets/[email protected])
Quoted 7 lines
Quoted 3 lines
Quoted 1 line

or something similar?

That starts falling apart when multiple tests use the same snippet, no?

That's true ... Maybe something like:

  $(filter %.gbin,$^)

That filters all files that are not gbins from the prereqs, right? In what way is that different to linking all the snippets? After all they are currently a prereq for %.elf or am I missing something here?

Just to expand our use-case a bit more with an example since the code is still too rough to post right now:

We test diagnose handling for Secure Execution / PV in one host test where we have a gbin each for most diagnose codes 0x44, 0x9c, 0x288, 0x500 (0x308 needs a whole test and possible multiple gbins on its own due to its complexity).
Other test bundles are PV IO, misc (stsi, sthyi, ...) and of course other VSIE instruction handling tests in the future.

Quoted 5 lines

?

Thomas

#10 lore raw

Re: [kvm-unit-tests RFC 1/2] s390x: Add guest snippet support

Quoted 103 lines · Janosch Frank
Quoted 102 lines · Thomas Huth
Quoted 100 lines · Janosch Frank
Quoted 91 lines · Thomas Huth
Quoted 81 lines · Janosch Frank
Quoted 32 lines · Thomas Huth
Quoted 31 lines · Janosch Frank

Snippets can be used to easily write and run guest (SIE) tests. The snippet is linked into the test binaries and can therefore be accessed via a ptr.

---
     .gitignore                |  2 ++
     s390x/Makefile            | 28 ++++++++++++++++++---
     s390x/snippets/c/cstart.S | 13 ++++++++++
     s390x/snippets/c/flat.lds | 51 +++++++++++++++++++++++++++++++++++++++
     4 files changed, 91 insertions(+), 3 deletions(-)
     create mode 100644 s390x/snippets/c/cstart.S
     create mode 100644 s390x/snippets/c/flat.lds
.gitignore +2 -0
index 784cb2dd..29d3635b 100644
@@ -22,3 +22,5 @@ cscope.*
     /api/dirty-log
     /api/dirty-log-perf
     /s390x/*.bin
+/s390x/snippets/*/*.bin
+/s390x/snippets/*/*.gbin
s390x/Makefile +0 -0
index 8de926ab..fe267011 100644
@@ -75,11 +75,33 @@ OBJDIRS += lib/s390x
     asmlib = $(TEST_DIR)/cstart64.o $(TEST_DIR)/cpu.o
 
Quoted 13 lines

s390x/Makefile OBJDIRS += lib/s390x · line 85

Quoted 10 lines
s390x/Makefile +8 -1
     FLATLIBS = $(libcflat)
-%.elf: %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
+
+SNIPPET_DIR = $(TEST_DIR)/snippets
+
+# C snippets that need to be linked
+snippets-c =
+
+# ASM snippets that are directly compiled and converted to a *.gbin
+snippets-a =

Could you please call this snippets-s instead of ...-a ? The -a suffix looks like an archive to me otherwise.

Sure

Quoted 4 lines

s390x/Makefile OBJDIRS += lib/s390x · line 86

Quoted 2 lines
s390x/Makefile +1 -0
+snippets = $(snippets-a)$(snippets-c)

Shouldn't there be a space between the two?

Yes, already fixed that a long while ago
I thought I had sent out a new version already, maybe that was an illusion as I can't seem to find it right now.

Quoted 14 lines
Quoted 14 lines
s390x/Makefile +13 -0
+snippets-o += $(patsubst %.gbin,%.o,$(snippets))
+
+$(snippets-a): $(snippets-o) $(FLATLIBS)
+	$(OBJCOPY) -O binary $(patsubst %.gbin,%.o,$@) $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+$(snippets-c): $(snippets-o) $(SNIPPET_DIR)/c/cstart.o  $(FLATLIBS)
+	$(CC) $(LDFLAGS) -o $@ -T $(SNIPPET_DIR)/c/flat.lds \
+		$(filter %.o, $^) $(FLATLIBS)
+	$(OBJCOPY) -O binary $@ $@
+	$(OBJCOPY) -I binary -O elf64-s390 -B "s390:64-bit" $@ $@
+
+%.elf: $(snippets) %.o $(FLATLIBS) $(SRCDIR)/s390x/flat.lds $(asmlib)
Quoted 8 lines

s390x/Makefile OBJDIRS += lib/s390x · line 103

Quoted 5 lines
s390x/Makefile +1 -1
     	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) \
     		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$@\"
     	$(CC) $(LDFLAGS) -o $@ -T $(SRCDIR)/s390x/flat.lds \
-		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
+		$(filter %.o, $^) $(FLATLIBS) $(snippets) $(@:.elf=.aux.o)

Does this link the snippets into all elf files? ... wouldn't it be better to restrict it somehow to the files that really need them?

Quoted 8 lines

Yes it does.
I'd like to avoid having to specify a makefile rule for every test that uses snippets as we already have more than the mvpg one in the queue.

So I'm having Steffen looking into a solution for this problem. My first idea was to bring the used snippets into the unittests.cfg but I disliked that we then would have compile instructions in another file. Maybe there's a way to include that into the makefile in a clever way?

Quoted 6 lines

I haven't tried, but maybe you could replace the $(snippets) in the last line with

    $(wildcard snippets/[email protected])

or something similar?

That starts falling apart when multiple tests use the same snippet, no?

Quoted 7 lines
Quoted 3 lines

That's true ... Maybe something like:

   $(filter %.gbin,$^)

That filters all files that are not gbins from the prereqs, right? In what way is that different to linking all the snippets? After all they are currently a prereq for %.elf or am I missing something here?

Sorry, I meant we could then add the prereqs manually for each tests that require it, e.g. add a single line like:

mvpg-sie.elf: snippets/mvpg.gbin

(without specifying a rule, just the dependency)

... not sure whether that works, though...

Thomas

#11 lore raw

RFC 2/2 s390x: mvpg: Add SIE mvpg test

In reply to Janosch Frank

Let's also check the PEI values to make sure our VSIE implementation is correct.

---
 s390x/Makefile                  |   3 +-
 s390x/mvpg-sie.c                | 139 ++++++++++++++++++++++++++++++++
 s390x/snippets/c/mvpg-snippet.c |  33 ++++++++
 s390x/unittests.cfg             |   3 +
 4 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 s390x/mvpg-sie.c
 create mode 100644 s390x/snippets/c/mvpg-snippet.c
s390x/Makefile +2 -1
index fe267011..6692cf73 100644
@@ -22,6 +22,7 @@ tests += $(TEST_DIR)/uv-guest.elf
 tests += $(TEST_DIR)/sie.elf
 tests += $(TEST_DIR)/mvpg.elf
 tests += $(TEST_DIR)/uv-host.elf
+tests += $(TEST_DIR)/mvpg-sie.elf
 
 tests_binary = $(patsubst %.elf,%.bin,$(tests))
 ifneq ($(HOST_KEY_DOCUMENT),)
@@ -79,7 +80,7 @@ FLATLIBS = $(libcflat)
 SNIPPET_DIR = $(TEST_DIR)/snippets
 
 # C snippets that need to be linked
-snippets-c =
+snippets-c = $(SNIPPET_DIR)/c/mvpg-snippet.gbin
 
 # ASM snippets that are directly compiled and converted to a *.gbin
 snippets-a =
s390x/mvpg-sie.c new file +139 -0

139 lines not on this page · raw message

s390x/snippets/c/mvpg-snippet.c new file +33 -0
new file mode 100644
index 00000000..96b70c9c
@@ -0,0 +1,33 @@
+#include <libcflat.h>
+
+static inline void force_exit(void)
+{
+	asm volatile("	diag	0,0,0x44\n");
+}
+
+static inline int mvpg(unsigned long r0, void *dest, void *src)
+{
+	register unsigned long reg0 asm ("0") = r0;
+	int cc;
+
+	asm volatile("	mvpg    %1,%2\n"
+		     "	ipm     %0\n"
+		     "	srl     %0,28"
+		     : "=&d" (cc) : "a" (dest), "a" (src), "d" (reg0)
+		     : "memory", "cc");
+	return cc;
+}
+
+static void test_mvpg_real(void)
+{
+	mvpg(0, (void *)0x5000, (void *)0x6000);
+	force_exit();
+}
+
+__attribute__((section(".text"))) int main(void)
+{
+	test_mvpg_real();
+	test_mvpg_real();
+	test_mvpg_real();
+	return 0;
+}
s390x/unittests.cfg +3 -0
index 9f81a608..8634b1b1 100644
@@ -103,3 +103,6 @@ file = sie.elf
 [mvpg]
 file = mvpg.elf
 timeout = 10
+
+[mvpg-sie]
+file = mvpg-sie.elf
Signature · 1 line
2.30.2
#12 lore raw

Re: [kvm-unit-tests RFC 2/2] s390x: mvpg: Add SIE mvpg test

Quoted 61 lines · On Thu, 20 May 2021 09:47:30 +0000 Janosch Frank <[email protected]> wrote:

Let's also check the PEI values to make sure our VSIE implementation is correct.

---
 s390x/Makefile                  |   3 +-
 s390x/mvpg-sie.c                | 139
++++++++++++++++++++++++++++++++ s390x/snippets/c/mvpg-snippet.c |
33 ++++++++ s390x/unittests.cfg             |   3 +
 4 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 s390x/mvpg-sie.c
 create mode 100644 s390x/snippets/c/mvpg-snippet.c
s390x/Makefile +2 -1
index fe267011..6692cf73 100644
@@ -22,6 +22,7 @@ tests += $(TEST_DIR)/uv-guest.elf
 tests += $(TEST_DIR)/sie.elf
 tests += $(TEST_DIR)/mvpg.elf
 tests += $(TEST_DIR)/uv-host.elf
+tests += $(TEST_DIR)/mvpg-sie.elf
 
 tests_binary = $(patsubst %.elf,%.bin,$(tests))
 ifneq ($(HOST_KEY_DOCUMENT),)
@@ -79,7 +80,7 @@ FLATLIBS = $(libcflat)
 SNIPPET_DIR = $(TEST_DIR)/snippets
 
 # C snippets that need to be linked
-snippets-c =
+snippets-c = $(SNIPPET_DIR)/c/mvpg-snippet.gbin
 
 # ASM snippets that are directly compiled and converted to a *.gbin
 snippets-a =
s390x/mvpg-sie.c new file +21 -0
new file mode 100644
index 00000000..a617704b
@@ -0,0 +1,139 @@
+#include <libcflat.h>
+#include <asm/asm-offsets.h>
+#include <asm-generic/barrier.h>
+#include <asm/interrupt.h>
+#include <asm/pgtable.h>
+#include <mmu.h>
+#include <asm/page.h>
+#include <asm/facility.h>
+#include <asm/mem.h>
+#include <asm/sigp.h>
+#include <smp.h>
+#include <alloc_page.h>
+#include <bitops.h>
+#include <vm.h>
+#include <sclp.h>
+#include <sie.h>
+
+static u8 *guest;
+static u8 *guest_instr;
+static struct vm vm;
+

s390x/mvpg-sie.c line 31

Quoted 10 lines
s390x/mvpg-sie.c new file +10 -0
+static uint8_t *src;
+static uint8_t *dst;
+
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_start[];
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_end[];
+int binary_size;
+
+static void handle_validity(struct vm *vm)
+{
+	report(0, "VALIDITY: %x", vm->sblk->ipb >> 16);

I think an assert would be better. This should not happen, and if it happens something went very wrong and we have no guarantee that we will be able to continue

Quoted 22 lines
s390x/mvpg-sie.c new file +19 -0
+}
+
+static void sie(struct vm *vm)
+{
+	/* Reset icptcode so we don't trip below */
+	vm->sblk->icptcode = 0;
+
+	while (vm->sblk->icptcode == 0) {
+		sie64a(vm->sblk, &vm->save_area);
+		if (vm->sblk->icptcode == ICPT_VALIDITY)
+			handle_validity(vm);
+	}
+	vm->save_area.guest.grs[14] = vm->sblk->gg14;
+	vm->save_area.guest.grs[15] = vm->sblk->gg15;
+}
+
+static void test_mvpg_pei(void)
+{
+	uint64_t **pei_dst = (uint64_t **)((uintptr_t) vm.sblk +

0xc0);

s390x/mvpg-sie.c new file +1 -0
+	uint64_t **pei_src = (uint64_t **)((uintptr_t) vm.sblk +

0xc8); +

s390x/mvpg-sie.c line 53

Quoted 1 line
s390x/mvpg-sie.c new file +1 -0
+	report_prefix_push("pei");

maybe clear the destination buffer...

Quoted 8 lines
s390x/mvpg-sie.c new file +3 -0
+	protect_page(guest + 0x6000, PAGE_ENTRY_I);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PARTEXEC, "Partial

execution");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_src == ((uintptr_t)vm.sblk->mso) +

0x6000 + PAGE_ENTRY_I, "PEI_SRC correct");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_dst == vm.sblk->mso + 0x5000,

"PEI_DST correct");

... and check that the page was not copied

s390x/mvpg-sie.c

Quoted 2 lines
s390x/mvpg-sie.c new file +2 -0
+	/* Jump over the diag44 */
+	sie(&vm);

I would check if you really got a diag44

Quoted 22 lines
s390x/mvpg-sie.c new file +6 -0
+	/* Clear PEI data for next check */
+	memset((uint64_t *)((uintptr_t) vm.sblk + 0xc0), 0, 16);
+	unprotect_page(guest + 0x6000, PAGE_ENTRY_I);
+	protect_page(guest + 0x5000, PAGE_ENTRY_I);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PARTEXEC, "Partial

execution");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_src == vm.sblk->mso + 0x6000,

"PEI_SRC correct");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_dst == vm.sblk->mso + 0x5000 +

PAGE_ENTRY_I, "PEI_DST correct"); +

s390x/mvpg-sie.c new file +6 -0
+	report_prefix_pop();
+}
+
+static void test_mvpg(void)
+{
+	int binary_size =

((uintptr_t)_binary_s390x_snippets_c_mvpg_snippet_gbin_end - + (uintptr_t)_binary_s390x_snippets_c_mvpg_snippet_gbin_start); +

s390x/mvpg-sie.c new file +1 -0
+	memcpy(guest,

_binary_s390x_snippets_c_mvpg_snippet_gbin_start, binary_size);

s390x/mvpg-sie.c

Quoted 5 lines
s390x/mvpg-sie.c new file +5 -0
+	memset(src, 0x42, PAGE_SIZE);
+	memset(dst, 0x43, PAGE_SIZE);
+	sie(&vm);
+	mb();
+	report(!memcmp(src, dst, PAGE_SIZE) && *dst == 0x42, "Page

or maybe you can clear the destination buffer here, if you prefer

Quoted 108 lines

moved"); +}

s390x/mvpg-sie.c new file +16 -0
+
+static void setup_guest(void)
+{
+	setup_vm();
+
+	/* Allocate 1MB as guest memory */
+	guest = alloc_pages(8);
+	/* The first two pages are the lowcore */
+	guest_instr = guest + PAGE_SIZE * 2;
+
+	vm.sblk = alloc_page();
+
+	vm.sblk->cpuflags = CPUSTAT_ZARCH | CPUSTAT_RUNNING;
+	vm.sblk->prefix = 0;
+	/*
+	 * Pageable guest with the same ASCE as the test programm,

but

s390x/mvpg-sie.c new file +4 -0
+	 * the guest memory 0x0 is offset to start at the allocated
+	 * guest pages and end after 1MB.
+	 *
+	 * It's not pretty but faster and easier than managing guest

ASCEs.

s390x/mvpg-sie.c new file +11 -0
+	 */
+	vm.sblk->mso = (u64)guest;
+	vm.sblk->msl = (u64)guest;
+	vm.sblk->ihcpu = 0xffff;
+
+	vm.sblk->crycbd = (uint64_t)alloc_page();
+
+	vm.sblk->gpsw.addr = PAGE_SIZE * 4;
+	vm.sblk->gpsw.mask = 0x0000000180000000ULL;
+	vm.sblk->ictl = ICTL_OPEREXC | ICTL_PINT;
+	/* Enable MVPG interpretation as we want to test KVM and not

ourselves */

s390x/mvpg-sie.c new file +23 -0
+	vm.sblk->eca = ECA_MVPGI;
+
+	src = guest + PAGE_SIZE * 6;
+	dst = guest + PAGE_SIZE * 5;
+}
+
+int main(void)
+{
+	report_prefix_push("mvpg-sie");
+	if (!sclp_facilities.has_sief2) {
+		report_skip("SIEF2 facility unavailable");
+		goto done;
+	}
+
+	setup_guest();
+	test_mvpg();
+	test_mvpg_pei();
+
+done:
+	report_prefix_pop();
+	return report_summary();
+
+}
(quoted diff) +0 -0

b/s390x/snippets/c/mvpg-snippet.c new file mode 100644 index 00000000..96b70c9c

s390x/snippets/c/mvpg-snippet.c new file +33 -0
@@ -0,0 +1,33 @@
+#include <libcflat.h>
+
+static inline void force_exit(void)
+{
+	asm volatile("	diag	0,0,0x44\n");
+}
+
+static inline int mvpg(unsigned long r0, void *dest, void *src)
+{
+	register unsigned long reg0 asm ("0") = r0;
+	int cc;
+
+	asm volatile("	mvpg    %1,%2\n"
+		     "	ipm     %0\n"
+		     "	srl     %0,28"
+		     : "=&d" (cc) : "a" (dest), "a" (src), "d" (reg0)
+		     : "memory", "cc");
+	return cc;
+}
+
+static void test_mvpg_real(void)
+{
+	mvpg(0, (void *)0x5000, (void *)0x6000);
+	force_exit();
+}
+
+__attribute__((section(".text"))) int main(void)
+{
+	test_mvpg_real();
+	test_mvpg_real();
+	test_mvpg_real();
+	return 0;
+}
s390x/unittests.cfg +3 -0
index 9f81a608..8634b1b1 100644
@@ -103,3 +103,6 @@ file = sie.elf
 [mvpg]
 file = mvpg.elf
 timeout = 10
+
+[mvpg-sie]
+file = mvpg-sie.elf
#13 lore raw

Re: [kvm-unit-tests RFC 2/2] s390x: mvpg: Add SIE mvpg test

Quoted 70 lines · Claudio Imbrenda
Quoted 67 lines · On Thu, 20 May 2021 09:47:30 +0000 Janosch Frank <[email protected]> wrote:

Let's also check the PEI values to make sure our VSIE implementation is correct.

---
 s390x/Makefile                  |   3 +-
 s390x/mvpg-sie.c                | 139
++++++++++++++++++++++++++++++++ s390x/snippets/c/mvpg-snippet.c |
33 ++++++++ s390x/unittests.cfg             |   3 +
 4 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 s390x/mvpg-sie.c
 create mode 100644 s390x/snippets/c/mvpg-snippet.c
s390x/Makefile +2 -1
index fe267011..6692cf73 100644
@@ -22,6 +22,7 @@ tests += $(TEST_DIR)/uv-guest.elf
 tests += $(TEST_DIR)/sie.elf
 tests += $(TEST_DIR)/mvpg.elf
 tests += $(TEST_DIR)/uv-host.elf
+tests += $(TEST_DIR)/mvpg-sie.elf
 
 tests_binary = $(patsubst %.elf,%.bin,$(tests))
 ifneq ($(HOST_KEY_DOCUMENT),)
@@ -79,7 +80,7 @@ FLATLIBS = $(libcflat)
 SNIPPET_DIR = $(TEST_DIR)/snippets
 
 # C snippets that need to be linked
-snippets-c =
+snippets-c = $(SNIPPET_DIR)/c/mvpg-snippet.gbin
 
 # ASM snippets that are directly compiled and converted to a *.gbin
 snippets-a =
s390x/mvpg-sie.c new file +27 -0
new file mode 100644
index 00000000..a617704b
@@ -0,0 +1,139 @@
+#include <libcflat.h>
+#include <asm/asm-offsets.h>
+#include <asm-generic/barrier.h>
+#include <asm/interrupt.h>
+#include <asm/pgtable.h>
+#include <mmu.h>
+#include <asm/page.h>
+#include <asm/facility.h>
+#include <asm/mem.h>
+#include <asm/sigp.h>
+#include <smp.h>
+#include <alloc_page.h>
+#include <bitops.h>
+#include <vm.h>
+#include <sclp.h>
+#include <sie.h>
+
+static u8 *guest;
+static u8 *guest_instr;
+static struct vm vm;
+
+static uint8_t *src;
+static uint8_t *dst;
+
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_start[];
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_end[];
+int binary_size;
Quoted 8 lines

s390x/mvpg-sie.c line 31

Quoted 4 lines
s390x/mvpg-sie.c new file +4 -0
+
+static void handle_validity(struct vm *vm)
+{
+	report(0, "VALIDITY: %x", vm->sblk->ipb >> 16);

I think an assert would be better. This should not happen, and if it happens something went very wrong and we have no guarantee that we will be able to continue

That depends on the test. If you use multiple gbins/snippets in a test, then a validity for one of those does not necessarily break the others. Especially when you think about PV tests where the destroy will clean out all the state anyway.

Quoted 27 lines
Quoted 23 lines
s390x/mvpg-sie.c new file +19 -0
+}
+
+static void sie(struct vm *vm)
+{
+	/* Reset icptcode so we don't trip below */
+	vm->sblk->icptcode = 0;
+
+	while (vm->sblk->icptcode == 0) {
+		sie64a(vm->sblk, &vm->save_area);
+		if (vm->sblk->icptcode == ICPT_VALIDITY)
+			handle_validity(vm);
+	}
+	vm->save_area.guest.grs[14] = vm->sblk->gg14;
+	vm->save_area.guest.grs[15] = vm->sblk->gg15;
+}
+
+static void test_mvpg_pei(void)
+{
+	uint64_t **pei_dst = (uint64_t **)((uintptr_t) vm.sblk +

0xc0);

s390x/mvpg-sie.c new file +1 -0
+	uint64_t **pei_src = (uint64_t **)((uintptr_t) vm.sblk +

0xc8); +

s390x/mvpg-sie.c line 53

Quoted 1 line
s390x/mvpg-sie.c new file +1 -0
+	report_prefix_push("pei");

maybe clear the destination buffer...

Quoted 10 lines
Quoted 8 lines
s390x/mvpg-sie.c new file +3 -0
+	protect_page(guest + 0x6000, PAGE_ENTRY_I);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PARTEXEC, "Partial

execution");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_src == ((uintptr_t)vm.sblk->mso) +

0x6000 + PAGE_ENTRY_I, "PEI_SRC correct");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_dst == vm.sblk->mso + 0x5000,

"PEI_DST correct");

... and check that the page was not copied

Sure

Quoted 5 lines

s390x/mvpg-sie.c

Quoted 3 lines
s390x/mvpg-sie.c new file +2 -0
+	/* Jump over the diag44 */
+	sie(&vm);

I would check if you really got a diag44

Sure, reporting doesn't make sense to me but an assert should be fine

Quoted 22 lines
Quoted 22 lines
s390x/mvpg-sie.c new file +6 -0
+	/* Clear PEI data for next check */
+	memset((uint64_t *)((uintptr_t) vm.sblk + 0xc0), 0, 16);
+	unprotect_page(guest + 0x6000, PAGE_ENTRY_I);
+	protect_page(guest + 0x5000, PAGE_ENTRY_I);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PARTEXEC, "Partial

execution");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_src == vm.sblk->mso + 0x6000,

"PEI_SRC correct");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_dst == vm.sblk->mso + 0x5000 +

PAGE_ENTRY_I, "PEI_DST correct"); +

s390x/mvpg-sie.c new file +6 -0
+	report_prefix_pop();
+}
+
+static void test_mvpg(void)
+{
+	int binary_size =

((uintptr_t)_binary_s390x_snippets_c_mvpg_snippet_gbin_end - + (uintptr_t)_binary_s390x_snippets_c_mvpg_snippet_gbin_start); +

s390x/mvpg-sie.c new file +1 -0
+	memcpy(guest,
Quoted 8 lines

s390x/mvpg-sie.c

Quoted 6 lines

_binary_s390x_snippets_c_mvpg_snippet_gbin_start, binary_size);

s390x/mvpg-sie.c new file +5 -0
+	memset(src, 0x42, PAGE_SIZE);
+	memset(dst, 0x43, PAGE_SIZE);
+	sie(&vm);
+	mb();
+	report(!memcmp(src, dst, PAGE_SIZE) && *dst == 0x42, "Page

or maybe you can clear the destination buffer here, if you prefer

I'll have a look, thanks for the review!

Quoted 110 lines
Quoted 109 lines

moved"); +}

s390x/mvpg-sie.c new file +16 -0
+
+static void setup_guest(void)
+{
+	setup_vm();
+
+	/* Allocate 1MB as guest memory */
+	guest = alloc_pages(8);
+	/* The first two pages are the lowcore */
+	guest_instr = guest + PAGE_SIZE * 2;
+
+	vm.sblk = alloc_page();
+
+	vm.sblk->cpuflags = CPUSTAT_ZARCH | CPUSTAT_RUNNING;
+	vm.sblk->prefix = 0;
+	/*
+	 * Pageable guest with the same ASCE as the test programm,

but

s390x/mvpg-sie.c new file +4 -0
+	 * the guest memory 0x0 is offset to start at the allocated
+	 * guest pages and end after 1MB.
+	 *
+	 * It's not pretty but faster and easier than managing guest

ASCEs.

s390x/mvpg-sie.c new file +11 -0
+	 */
+	vm.sblk->mso = (u64)guest;
+	vm.sblk->msl = (u64)guest;
+	vm.sblk->ihcpu = 0xffff;
+
+	vm.sblk->crycbd = (uint64_t)alloc_page();
+
+	vm.sblk->gpsw.addr = PAGE_SIZE * 4;
+	vm.sblk->gpsw.mask = 0x0000000180000000ULL;
+	vm.sblk->ictl = ICTL_OPEREXC | ICTL_PINT;
+	/* Enable MVPG interpretation as we want to test KVM and not

ourselves */

s390x/mvpg-sie.c new file +23 -0
+	vm.sblk->eca = ECA_MVPGI;
+
+	src = guest + PAGE_SIZE * 6;
+	dst = guest + PAGE_SIZE * 5;
+}
+
+int main(void)
+{
+	report_prefix_push("mvpg-sie");
+	if (!sclp_facilities.has_sief2) {
+		report_skip("SIEF2 facility unavailable");
+		goto done;
+	}
+
+	setup_guest();
+	test_mvpg();
+	test_mvpg_pei();
+
+done:
+	report_prefix_pop();
+	return report_summary();
+
+}
(quoted diff) +0 -0

b/s390x/snippets/c/mvpg-snippet.c new file mode 100644 index 00000000..96b70c9c

s390x/snippets/c/mvpg-snippet.c new file +33 -0
@@ -0,0 +1,33 @@
+#include <libcflat.h>
+
+static inline void force_exit(void)
+{
+	asm volatile("	diag	0,0,0x44\n");
+}
+
+static inline int mvpg(unsigned long r0, void *dest, void *src)
+{
+	register unsigned long reg0 asm ("0") = r0;
+	int cc;
+
+	asm volatile("	mvpg    %1,%2\n"
+		     "	ipm     %0\n"
+		     "	srl     %0,28"
+		     : "=&d" (cc) : "a" (dest), "a" (src), "d" (reg0)
+		     : "memory", "cc");
+	return cc;
+}
+
+static void test_mvpg_real(void)
+{
+	mvpg(0, (void *)0x5000, (void *)0x6000);
+	force_exit();
+}
+
+__attribute__((section(".text"))) int main(void)
+{
+	test_mvpg_real();
+	test_mvpg_real();
+	test_mvpg_real();
+	return 0;
+}
s390x/unittests.cfg +3 -0
index 9f81a608..8634b1b1 100644
@@ -103,3 +103,6 @@ file = sie.elf
 [mvpg]
 file = mvpg.elf
 timeout = 10
+
+[mvpg-sie]
+file = mvpg-sie.elf
#14 lore raw

Re: [kvm-unit-tests RFC 2/2] s390x: mvpg: Add SIE mvpg test

In reply to Claudio Imbrenda
Quoted 116 lines · Claudio Imbrenda
Quoted 61 lines · On Thu, 20 May 2021 09:47:30 +0000 Janosch Frank <[email protected]> wrote:

Let's also check the PEI values to make sure our VSIE implementation is correct.

---
 s390x/Makefile                  |   3 +-
 s390x/mvpg-sie.c                | 139
++++++++++++++++++++++++++++++++ s390x/snippets/c/mvpg-snippet.c |
33 ++++++++ s390x/unittests.cfg             |   3 +
 4 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 s390x/mvpg-sie.c
 create mode 100644 s390x/snippets/c/mvpg-snippet.c
s390x/Makefile +2 -1
index fe267011..6692cf73 100644
@@ -22,6 +22,7 @@ tests += $(TEST_DIR)/uv-guest.elf
 tests += $(TEST_DIR)/sie.elf
 tests += $(TEST_DIR)/mvpg.elf
 tests += $(TEST_DIR)/uv-host.elf
+tests += $(TEST_DIR)/mvpg-sie.elf
 
 tests_binary = $(patsubst %.elf,%.bin,$(tests))
 ifneq ($(HOST_KEY_DOCUMENT),)
@@ -79,7 +80,7 @@ FLATLIBS = $(libcflat)
 SNIPPET_DIR = $(TEST_DIR)/snippets
 
 # C snippets that need to be linked
-snippets-c =
+snippets-c = $(SNIPPET_DIR)/c/mvpg-snippet.gbin
 
 # ASM snippets that are directly compiled and converted to a *.gbin
 snippets-a =
s390x/mvpg-sie.c new file +21 -0
new file mode 100644
index 00000000..a617704b
@@ -0,0 +1,139 @@
+#include <libcflat.h>
+#include <asm/asm-offsets.h>
+#include <asm-generic/barrier.h>
+#include <asm/interrupt.h>
+#include <asm/pgtable.h>
+#include <mmu.h>
+#include <asm/page.h>
+#include <asm/facility.h>
+#include <asm/mem.h>
+#include <asm/sigp.h>
+#include <smp.h>
+#include <alloc_page.h>
+#include <bitops.h>
+#include <vm.h>
+#include <sclp.h>
+#include <sie.h>
+
+static u8 *guest;
+static u8 *guest_instr;
+static struct vm vm;
+

s390x/mvpg-sie.c line 31

Quoted 10 lines
s390x/mvpg-sie.c new file +10 -0
+static uint8_t *src;
+static uint8_t *dst;
+
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_start[];
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_end[];
+int binary_size;
+
+static void handle_validity(struct vm *vm)
+{
+	report(0, "VALIDITY: %x", vm->sblk->ipb >> 16);

I think an assert would be better. This should not happen, and if it happens something went very wrong and we have no guarantee that we will be able to continue

Quoted 22 lines
s390x/mvpg-sie.c new file +19 -0
+}
+
+static void sie(struct vm *vm)
+{
+	/* Reset icptcode so we don't trip below */
+	vm->sblk->icptcode = 0;
+
+	while (vm->sblk->icptcode == 0) {
+		sie64a(vm->sblk, &vm->save_area);
+		if (vm->sblk->icptcode == ICPT_VALIDITY)
+			handle_validity(vm);
+	}
+	vm->save_area.guest.grs[14] = vm->sblk->gg14;
+	vm->save_area.guest.grs[15] = vm->sblk->gg15;
+}
+
+static void test_mvpg_pei(void)
+{
+	uint64_t **pei_dst = (uint64_t **)((uintptr_t) vm.sblk +

0xc0);

s390x/mvpg-sie.c new file +1 -0
+	uint64_t **pei_src = (uint64_t **)((uintptr_t) vm.sblk +

0xc8); +

s390x/mvpg-sie.c line 53

Quoted 1 line
s390x/mvpg-sie.c new file +1 -0
+	report_prefix_push("pei");

maybe clear the destination buffer...

Quoted 8 lines
s390x/mvpg-sie.c new file +3 -0
+	protect_page(guest + 0x6000, PAGE_ENTRY_I);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PARTEXEC, "Partial

execution");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_src == ((uintptr_t)vm.sblk->mso) +

0x6000 + PAGE_ENTRY_I, "PEI_SRC correct");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_dst == vm.sblk->mso + 0x5000,

"PEI_DST correct");

... and check that the page was not copied

Quoted 8 lines

s390x/mvpg-sie.c

Quoted 2 lines
s390x/mvpg-sie.c new file +2 -0
+	/* Jump over the diag44 */
+	sie(&vm);

I would check if you really got a diag44

s390x/mvpg-sie.c line 64

Quoted 3 lines
s390x/mvpg-sie.c new file +3 -0
+	/* Clear PEI data for next check */
+	memset((uint64_t *)((uintptr_t) vm.sblk + 0xc0), 0, 16);
+	unprotect_page(guest + 0x6000, PAGE_ENTRY_I);

I'll also move these over to src/dst so we only use the magic constants once.

Quoted 136 lines
Quoted 19 lines
s390x/mvpg-sie.c new file +3 -0
+	protect_page(guest + 0x5000, PAGE_ENTRY_I);
+	sie(&vm);
+	report(vm.sblk->icptcode == ICPT_PARTEXEC, "Partial

execution");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_src == vm.sblk->mso + 0x6000,

"PEI_SRC correct");

s390x/mvpg-sie.c new file +1 -0
+	report((uintptr_t)**pei_dst == vm.sblk->mso + 0x5000 +

PAGE_ENTRY_I, "PEI_DST correct"); +

s390x/mvpg-sie.c new file +6 -0
+	report_prefix_pop();
+}
+
+static void test_mvpg(void)
+{
+	int binary_size =

((uintptr_t)_binary_s390x_snippets_c_mvpg_snippet_gbin_end - + (uintptr_t)_binary_s390x_snippets_c_mvpg_snippet_gbin_start); +

s390x/mvpg-sie.c new file +1 -0
+	memcpy(guest,

_binary_s390x_snippets_c_mvpg_snippet_gbin_start, binary_size);

s390x/mvpg-sie.c

Quoted 5 lines
s390x/mvpg-sie.c new file +5 -0
+	memset(src, 0x42, PAGE_SIZE);
+	memset(dst, 0x43, PAGE_SIZE);
+	sie(&vm);
+	mb();
+	report(!memcmp(src, dst, PAGE_SIZE) && *dst == 0x42, "Page

or maybe you can clear the destination buffer here, if you prefer

Quoted 108 lines

moved"); +}

s390x/mvpg-sie.c new file +16 -0
+
+static void setup_guest(void)
+{
+	setup_vm();
+
+	/* Allocate 1MB as guest memory */
+	guest = alloc_pages(8);
+	/* The first two pages are the lowcore */
+	guest_instr = guest + PAGE_SIZE * 2;
+
+	vm.sblk = alloc_page();
+
+	vm.sblk->cpuflags = CPUSTAT_ZARCH | CPUSTAT_RUNNING;
+	vm.sblk->prefix = 0;
+	/*
+	 * Pageable guest with the same ASCE as the test programm,

but

s390x/mvpg-sie.c new file +4 -0
+	 * the guest memory 0x0 is offset to start at the allocated
+	 * guest pages and end after 1MB.
+	 *
+	 * It's not pretty but faster and easier than managing guest

ASCEs.

s390x/mvpg-sie.c new file +11 -0
+	 */
+	vm.sblk->mso = (u64)guest;
+	vm.sblk->msl = (u64)guest;
+	vm.sblk->ihcpu = 0xffff;
+
+	vm.sblk->crycbd = (uint64_t)alloc_page();
+
+	vm.sblk->gpsw.addr = PAGE_SIZE * 4;
+	vm.sblk->gpsw.mask = 0x0000000180000000ULL;
+	vm.sblk->ictl = ICTL_OPEREXC | ICTL_PINT;
+	/* Enable MVPG interpretation as we want to test KVM and not

ourselves */

s390x/mvpg-sie.c new file +23 -0
+	vm.sblk->eca = ECA_MVPGI;
+
+	src = guest + PAGE_SIZE * 6;
+	dst = guest + PAGE_SIZE * 5;
+}
+
+int main(void)
+{
+	report_prefix_push("mvpg-sie");
+	if (!sclp_facilities.has_sief2) {
+		report_skip("SIEF2 facility unavailable");
+		goto done;
+	}
+
+	setup_guest();
+	test_mvpg();
+	test_mvpg_pei();
+
+done:
+	report_prefix_pop();
+	return report_summary();
+
+}
(quoted diff) +0 -0

b/s390x/snippets/c/mvpg-snippet.c new file mode 100644 index 00000000..96b70c9c

s390x/snippets/c/mvpg-snippet.c new file +33 -0
@@ -0,0 +1,33 @@
+#include <libcflat.h>
+
+static inline void force_exit(void)
+{
+	asm volatile("	diag	0,0,0x44\n");
+}
+
+static inline int mvpg(unsigned long r0, void *dest, void *src)
+{
+	register unsigned long reg0 asm ("0") = r0;
+	int cc;
+
+	asm volatile("	mvpg    %1,%2\n"
+		     "	ipm     %0\n"
+		     "	srl     %0,28"
+		     : "=&d" (cc) : "a" (dest), "a" (src), "d" (reg0)
+		     : "memory", "cc");
+	return cc;
+}
+
+static void test_mvpg_real(void)
+{
+	mvpg(0, (void *)0x5000, (void *)0x6000);
+	force_exit();
+}
+
+__attribute__((section(".text"))) int main(void)
+{
+	test_mvpg_real();
+	test_mvpg_real();
+	test_mvpg_real();
+	return 0;
+}
s390x/unittests.cfg +3 -0
index 9f81a608..8634b1b1 100644
@@ -103,3 +103,6 @@ file = sie.elf
 [mvpg]
 file = mvpg.elf
 timeout = 10
+
+[mvpg-sie]
+file = mvpg-sie.elf
#15 lore raw

Re: [kvm-unit-tests RFC 2/2] s390x: mvpg: Add SIE mvpg test

In reply to Janosch Frank
Quoted 62 lines · Janosch Frank

Let's also check the PEI values to make sure our VSIE implementation is correct.

---
  s390x/Makefile                  |   3 +-
  s390x/mvpg-sie.c                | 139 ++++++++++++++++++++++++++++++++
  s390x/snippets/c/mvpg-snippet.c |  33 ++++++++
  s390x/unittests.cfg             |   3 +
  4 files changed, 177 insertions(+), 1 deletion(-)

create mode 100644 s390x/mvpg-sie.c
create mode 100644 s390x/snippets/c/mvpg-snippet.c

s390x/Makefile +2 -1
index fe267011..6692cf73 100644
@@ -22,6 +22,7 @@ tests += $(TEST_DIR)/uv-guest.elf
  tests += $(TEST_DIR)/sie.elf
  tests += $(TEST_DIR)/mvpg.elf
  tests += $(TEST_DIR)/uv-host.elf
+tests += $(TEST_DIR)/mvpg-sie.elf
 
  tests_binary = $(patsubst %.elf,%.bin,$(tests))
  ifneq ($(HOST_KEY_DOCUMENT),)
@@ -79,7 +80,7 @@ FLATLIBS = $(libcflat)
  SNIPPET_DIR = $(TEST_DIR)/snippets
  report_abort
  # C snippets that need to be linked
-snippets-c =
+snippets-c = $(SNIPPET_DIR)/c/mvpg-snippet.gbin
 
  # ASM snippets that are directly compiled and converted to a *.gbin
  snippets-a =
s390x/mvpg-sie.c renamed from dev/nullreport_abort +22 -0
new file mode 100644
index 00000000..a617704b
@@ -0,0 +1,139 @@
+#include <libcflat.h>
+#include <asm/asm-offsets.h>
+#include <asm-generic/barrier.h>
+#include <asm/interrupt.h>
+#include <asm/pgtable.h>
+#include <mmu.h>
+#include <asm/page.h>
+#include <asm/facility.h>
+#include <asm/mem.h>
+#include <asm/sigp.h>
+#include <smp.h>
+#include <alloc_page.h>
+#include <bitops.h>
+#include <vm.h>
+#include <sclp.h>
+#include <sie.h>
+
+static u8 *guest;
+static u8 *guest_instr;
+static struct vm vm;
+
+static uint8_t *src;

s390x/mvpg-sie.c line 32

Quoted 10 lines
s390x/mvpg-sie.c renamed from dev/nullreport_abort +10 -0
+static uint8_t *dst;
+
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_start[];
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_end[];
+int binary_size;
+
+static void handle_validity(struct vm *vm)
+{
+	report(0, "VALIDITY: %x", vm->sblk->ipb >> 16);
+}

Maybe rather use report_abort() in that case? Or does it make sense to continue running the other tests afterwards?

Thomas

#16 lore raw

Re: [kvm-unit-tests RFC 2/2] s390x: mvpg: Add SIE mvpg test

Quoted 77 lines · Thomas Huth
Quoted 62 lines · Janosch Frank

Let's also check the PEI values to make sure our VSIE implementation is correct.

---
  s390x/Makefile                  |   3 +-
  s390x/mvpg-sie.c                | 139 ++++++++++++++++++++++++++++++++
  s390x/snippets/c/mvpg-snippet.c |  33 ++++++++
  s390x/unittests.cfg             |   3 +
  4 files changed, 177 insertions(+), 1 deletion(-)

create mode 100644 s390x/mvpg-sie.c
create mode 100644 s390x/snippets/c/mvpg-snippet.c

s390x/Makefile +2 -1
index fe267011..6692cf73 100644
@@ -22,6 +22,7 @@ tests += $(TEST_DIR)/uv-guest.elf
  tests += $(TEST_DIR)/sie.elf
  tests += $(TEST_DIR)/mvpg.elf
  tests += $(TEST_DIR)/uv-host.elf
+tests += $(TEST_DIR)/mvpg-sie.elf
 
  tests_binary = $(patsubst %.elf,%.bin,$(tests))
  ifneq ($(HOST_KEY_DOCUMENT),)
@@ -79,7 +80,7 @@ FLATLIBS = $(libcflat)
  SNIPPET_DIR = $(TEST_DIR)/snippets
  report_abort
  # C snippets that need to be linked
-snippets-c =
+snippets-c = $(SNIPPET_DIR)/c/mvpg-snippet.gbin
 
  # ASM snippets that are directly compiled and converted to a *.gbin
  snippets-a =
s390x/mvpg-sie.c renamed from dev/nullreport_abort +22 -0
new file mode 100644
index 00000000..a617704b
@@ -0,0 +1,139 @@
+#include <libcflat.h>
+#include <asm/asm-offsets.h>
+#include <asm-generic/barrier.h>
+#include <asm/interrupt.h>
+#include <asm/pgtable.h>
+#include <mmu.h>
+#include <asm/page.h>
+#include <asm/facility.h>
+#include <asm/mem.h>
+#include <asm/sigp.h>
+#include <smp.h>
+#include <alloc_page.h>
+#include <bitops.h>
+#include <vm.h>
+#include <sclp.h>
+#include <sie.h>
+
+static u8 *guest;
+static u8 *guest_instr;
+static struct vm vm;
+
+static uint8_t *src;

s390x/mvpg-sie.c line 32

Quoted 10 lines
s390x/mvpg-sie.c renamed from dev/nullreport_abort +10 -0
+static uint8_t *dst;
+
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_start[];
+extern const char _binary_s390x_snippets_c_mvpg_snippet_gbin_end[];
+int binary_size;
+
+static void handle_validity(struct vm *vm)
+{
+	report(0, "VALIDITY: %x", vm->sblk->ipb >> 16);
+}

Maybe rather use report_abort() in that case? Or does it make sense to continue running the other tests afterwards?

Quoted 2 lines

Thomas

Hmm, right this is not yet in the SIE lib.

#17 lore raw
In reply to Janosch Frank
Quoted 8 lines · Janosch Frank

The SIE support allows us to run guests and test the hypervisor's (V)SIE implementation. However it requires that the guest instructions are binary which limits the complexity of the guest code.

The snippet support provides a way to write guest code as ASM or C and simply memcpy it into guest memory. Some of the KVM-unit-test library can be re-used which further speeds up guest code development.

Quoted 7 lines

The included mvpg-sie test helped us to deliver the KVM mvpg fixes which Claudio posted a short while ago. In the future I'll post Secure Execution snippet support patches which was my initial goal with this series anyway.

I heard you liked tests so I put tests inside tests so you can test while you test.

The idea sounds sane to me.

Signature · 3 lines
Thanks,

David / dhildenb