Skip to contentSkip to conversation

xen: add support for crash dump analysis with xen.efi

PATCH Superseded Juergen Gross 2 participants 5 messages Apr 21, 2023 – Apr 24, 2023

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen: add support for crash dump analysis with xen.efi

Today it is not possible to analyse crash dumps of a system in hypervisor mode when it had been booted via EFI, as the crash utility doesn't understand the file format of xen.efi.

This can easily be solved by creating an ELF file from xen.efi via objcopy. Using that file as name list for crash enables the user to analyse the dump in hypervisor mode.

---
 xen/Kconfig.debug     | 5 ++++-
 xen/Makefile          | 4 ++++
 xen/arch/x86/Makefile | 3 +++
 3 files changed, 11 insertions(+), 1 deletion(-)
xen/Kconfig.debug +4 -1
index 94e818ee09..4aec0fd3aa 100644
@@ -138,6 +138,9 @@ config DEBUG_INFO
 	  the EFI boot partition (look for "INSTALL_EFI_STRIP" in
 	  docs/misc/efi.pandoc for more information - when not using
 	  "make install-xen" for installing xen.efi, stripping needs to be
-	  done outside the Xen build environment).
+	  done outside the Xen build environment). Note that stripping
+	  xen.efi using "INSTALL_EFI_STRIP" will disable the building of
+	  xen.efi.elf, which is needed for "crash" dump analysis of systems
+	  booted via EFI.
 
 endmenu
xen/Makefile +4 -0
index 2710d7327e..db50dcb502 100644
@@ -505,6 +505,9 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
 		if [ -e $(TARGET).efi.map ]; then \
 			$(INSTALL_DATA) $(TARGET).efi.map $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.map; \
 		fi; \
+		if [ -e $(TARGET).efi.elf ]; then \
+			$(INSTALL_DATA) $(TARGET).efi.elf $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.elf; \
+		fi; \
 		ln -sf $(T)-$(XEN_FULLVERSION).efi $(D)$(EFI_DIR)/$(T)-$(XEN_VERSION).$(XEN_SUBVERSION).efi; \
 		ln -sf $(T)-$(XEN_FULLVERSION).efi $(D)$(EFI_DIR)/$(T)-$(XEN_VERSION).efi; \
 		ln -sf $(T)-$(XEN_FULLVERSION).efi $(D)$(EFI_DIR)/$(T).efi; \
@@ -539,6 +542,7 @@ _uninstall:
 	rm -f $(D)$(DEBUG_DIR)/$(T)-syms-$(XEN_FULLVERSION).map
 	rm -f $(D)$(EFI_DIR)/$(T)-$(XEN_FULLVERSION).efi
 	rm -f $(D)$(EFI_DIR)/$(T)-$(XEN_VERSION).$(XEN_SUBVERSION).efi
+	rm -f $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.elf
 	rm -f $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.map
 	rm -f $(D)$(EFI_DIR)/$(T)-$(XEN_VERSION).efi
 	rm -f $(D)$(EFI_DIR)/$(T).efi
xen/arch/x86/Makefile +3 -0
index fc9487aa40..87ce00addf 100644
@@ -224,6 +224,9 @@ endif
 	$(MAKE) $(build)=$(@D) .$(@F).1r.o .$(@F).1s.o
 	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T $(obj)/efi.lds -N $< \
 	      $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(orphan-handling-y) $(note_file_option) -o $@
+ifeq ($(CONFIG_DEBUG_INFO),y)
+	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),,$(OBJCOPY) -O elf64-x86-64 $@ [email protected])
+endif
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
 	rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*
Signature · 1 line
2.35.3
#1 lore raw
Quoted 20 lines · Juergen Gross

Today it is not possible to analyse crash dumps of a system in hypervisor mode when it had been booted via EFI, as the crash utility doesn't understand the file format of xen.efi.

This can easily be solved by creating an ELF file from xen.efi via objcopy. Using that file as name list for crash enables the user to analyse the dump in hypervisor mode.

---
 xen/Kconfig.debug     | 5 ++++-
 xen/Makefile          | 4 ++++
 xen/arch/x86/Makefile | 3 +++
 3 files changed, 11 insertions(+), 1 deletion(-)
xen/Kconfig.debug +0 -0
index 94e818ee09..4aec0fd3aa 100644

xen/Kconfig.debug config DEBUG_INFO · line 144

Quoted 8 lines
xen/Kconfig.debug +4 -1
@@ -138,6 +138,9 @@ config DEBUG_INFO
 	  the EFI boot partition (look for "INSTALL_EFI_STRIP" in
 	  docs/misc/efi.pandoc for more information - when not using
 	  "make install-xen" for installing xen.efi, stripping needs to be
-	  done outside the Xen build environment).
+	  done outside the Xen build environment). Note that stripping
+	  xen.efi using "INSTALL_EFI_STRIP" will disable the building of
+	  xen.efi.elf, which is needed for "crash" dump analysis of systems
+	  booted via EFI.

I'm afraid I don't understand this: INSTALL_EFI_STRIP only affects what may (optionally) be placed on the EFI partition. The normally installed xen.efi should be unaffected by this; an intermediate xen.efi.stripped is created instead.

xen/Makefile _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX) · line 510

Quoted 9 lines
xen/Makefile +3 -0
@@ -505,6 +505,9 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
 		if [ -e $(TARGET).efi.map ]; then \
 			$(INSTALL_DATA) $(TARGET).efi.map $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.map; \
 		fi; \
+		if [ -e $(TARGET).efi.elf ]; then \
+			$(INSTALL_DATA) $(TARGET).efi.elf $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.elf; \
+		fi; \

To avoid the risk of the two going out of sync (as also to limit redundancy), could you wrap the earlier "if" in a "for x in map elf; do ...; done" loop?

xen/arch/x86/Makefile endif · line 232

Quoted 12 lines
xen/arch/x86/Makefile +3 -0
@@ -224,6 +224,9 @@ endif
 	$(MAKE) $(build)=$(@D) .$(@F).1r.o .$(@F).1s.o
 	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T $(obj)/efi.lds -N $< \
 	      $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(orphan-handling-y) $(note_file_option) -o $@
+ifeq ($(CONFIG_DEBUG_INFO),y)
+	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),,$(OBJCOPY) -O elf64-x86-64 $@ [email protected])
+endif
 	$(NM) -pa --format=sysv $(@D)/$(@F) \
 		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
 	rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*

Personally I think that - in case of build problems - having the map file created is more important (and less likely to fail) than the ELF one, so I'd prefer if the two steps could be ordered the other way around.

Further I vaguely recall possible issues with entirely blank make rule lines. Plus having some trace of the command in a verbose log perhaps also wouldn't hurt. IOW maybe better add another use of the colon command here (we already have some, at least through $(MKRELOC)):

ifeq ($(CONFIG_DEBUG_INFO),y)

	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),:)$(OBJCOPY) -O elf64-x86-64 $@ [email protected]

endif

?

Finally - do you really need to copy all the non-debug sections as well? Might --only-keep-debug be helpful here (provided it works for a PE/COFF -> ELF copy operation; it looks to do so in my simple experiments)?

Jan

#2 lore raw
Quoted 26 lines · Jan Beulich
Quoted 25 lines · Juergen Gross

Today it is not possible to analyse crash dumps of a system in hypervisor mode when it had been booted via EFI, as the crash utility doesn't understand the file format of xen.efi.

This can easily be solved by creating an ELF file from xen.efi via objcopy. Using that file as name list for crash enables the user to analyse the dump in hypervisor mode.

---
  xen/Kconfig.debug     | 5 ++++-
  xen/Makefile          | 4 ++++
  xen/arch/x86/Makefile | 3 +++
  3 files changed, 11 insertions(+), 1 deletion(-)
xen/Kconfig.debug +1 -1
index 94e818ee09..4aec0fd3aa 100644
@@ -138,6 +138,9 @@ config DEBUG_INFO
  	  the EFI boot partition (look for "INSTALL_EFI_STRIP" in
  	  docs/misc/efi.pandoc for more information - when not using
  	  "make install-xen" for installing xen.efi, stripping needs to be
-	  done outside the Xen build environment).
+	  done outside the Xen build environment). Note that stripping
Quoted 8 lines

xen/Kconfig.debug config DEBUG_INFO · line 144

Quoted 3 lines
xen/Kconfig.debug +3 -0
+	  xen.efi using "INSTALL_EFI_STRIP" will disable the building of
+	  xen.efi.elf, which is needed for "crash" dump analysis of systems
+	  booted via EFI.

I'm afraid I don't understand this: INSTALL_EFI_STRIP only affects what may (optionally) be placed on the EFI partition. The normally installed xen.efi should be unaffected by this; an intermediate xen.efi.stripped is created instead.

Oh, obviously I misinterpreted something. I'll drop this hunk.

Quoted 14 lines

xen/Makefile _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX) · line 510

Quoted 10 lines
xen/Makefile +3 -0
@@ -505,6 +505,9 @@ _install: $(TARGET)$(CONFIG_XEN_INSTALL_SUFFIX)
  		if [ -e $(TARGET).efi.map ]; then \
  			$(INSTALL_DATA) $(TARGET).efi.map $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.map; \
  		fi; \
+		if [ -e $(TARGET).efi.elf ]; then \
+			$(INSTALL_DATA) $(TARGET).efi.elf $(D)$(DEBUG_DIR)/$(T)-$(XEN_FULLVERSION).efi.elf; \
+		fi; \

To avoid the risk of the two going out of sync (as also to limit redundancy), could you wrap the earlier "if" in a "for x in map elf; do ...; done" loop?

Sure, will do.

Quoted 17 lines

xen/arch/x86/Makefile endif · line 232

Quoted 13 lines
xen/arch/x86/Makefile +3 -0
@@ -224,6 +224,9 @@ endif
  	$(MAKE) $(build)=$(@D) .$(@F).1r.o .$(@F).1s.o
  	$(LD) $(call EFI_LDFLAGS,$(VIRT_BASE)) -T $(obj)/efi.lds -N $< \
  	      $(@D)/.$(@F).1r.o $(@D)/.$(@F).1s.o $(orphan-handling-y) $(note_file_option) -o $@
+ifeq ($(CONFIG_DEBUG_INFO),y)
+	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),,$(OBJCOPY) -O elf64-x86-64 $@ [email protected])
+endif
  	$(NM) -pa --format=sysv $(@D)/$(@F) \
  		| $(objtree)/tools/symbols --all-symbols --xensyms --sysv --sort >$(@D)/$(@F).map
  	rm -f $(@D)/.$(@F).[0-9]* $(@D)/..$(@F).[0-9]*

Personally I think that - in case of build problems - having the map file created is more important (and less likely to fail) than the ELF one, so I'd prefer if the two steps could be ordered the other way around.

Fine with me.

Quoted 10 lines

Further I vaguely recall possible issues with entirely blank make rule lines. Plus having some trace of the command in a verbose log perhaps also wouldn't hurt. IOW maybe better add another use of the colon command here (we already have some, at least through $(MKRELOC)):

ifeq ($(CONFIG_DEBUG_INFO),y)

	$(if $(filter --strip-debug,$(EFI_LDFLAGS)),:)$(OBJCOPY) -O elf64-x86-64 $@ [email protected]

endif

?

Okay.

Quoted 4 lines

Finally - do you really need to copy all the non-debug sections as well? Might --only-keep-debug be helpful here (provided it works for a PE/COFF -> ELF copy operation; it looks to do so in my simple experiments)?

No, doesn't work (objcopy does, but not crash with that file):

crash: xen.efi.elf: no text and data contents

Juergen

#3 lore raw
Quoted 8 lines · Juergen Gross
Quoted 3 lines · Jan Beulich

Finally - do you really need to copy all the non-debug sections as well? Might --only-keep-debug be helpful here (provided it works for a PE/COFF -> ELF copy operation; it looks to do so in my simple experiments)?

No, doesn't work (objcopy does, but not crash with that file):

crash: xen.efi.elf: no text and data contents

Oh, wow. What use do they have for the .text / .data contents? I very much hope they use the in-memory Xen image for analysis, not what they may read out of the image.

In any event, please add half a sentence to the description to mention this aspect.

Jan

#4 lore raw
Quoted 6 lines · Jan Beulich
Quoted 5 lines · Juergen Gross
Quoted 3 lines · Jan Beulich

Finally - do you really need to copy all the non-debug sections as well? Might --only-keep-debug be helpful here (provided it works for a PE/COFF -> ELF copy operation; it looks to do so in my simple experiments)?

Quoted 7 lines
Quoted 3 lines

No, doesn't work (objcopy does, but not crash with that file):

crash: xen.efi.elf: no text and data contents

Oh, wow. What use do they have for the .text / .data contents? I very much hope they use the in-memory Xen image for analysis, not what they may read out of the image.

I think crash is fine in this regard. At least __start_xen() seems to be all zeroes in my test dump, which is fine for an .init function after the system is up and running. :-)

Quoted 2 lines

In any event, please add half a sentence to the description to mention this aspect.

Okay.

Juergen