Skip to contentSkip to conversation

kexec-tools: ia64: icache flush should align to 32-bytes

PATCH In review Terry Loftin 2 participants 3 messages Oct 11, 2006 – Oct 12, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kexec-tools: ia64: icache flush should align to 32-bytes

Hi,
I've run into problems testing kexec/kdump on a Montecito revision C processor. In purgatory, __dummy_efi_function is copied onto the end of the command line boot parameter (command_line + command_line_len) and this address is used to replace the EFI call to set_virtual_address_map(). The copied range is then icache flushed.

The destination address is aligned to 16-bytes (in kexec-elf-ia64.c), but the fc.i instruction flushes a 32-byte range "associated" with that address. When my command line length is 16-byte aligned but not 32-byte aligned, this results in the first 16-bytes of __dummy_efi_function getting flushed (and the 16 bytes prior to that), but the second half of the function (the part with the br.ret) does not get flushed. kdump then hangs in purgatory. By adding a few spaces to my command line, it becomes both 16 and 32-byte aligned, and kdump works.

This patch makes icache_flush_range() align the start address to 32-bytes and account for the difference. The patch is against Horms kexec-tools-testing tree. As a side note, you could also fix this by just adding 32 to the length passed to flush_icache_range() but that hides the dependent behavior.

Thanks,
-T

purgatory/arch/ia64/purgatory-ia64.c 2006-10-11 16:05:06.000000000 -0600 renamed from purgatory/arch/ia64/purgatory-ia64.c 2006-10-06 13:40:50.000000000 -0600 +3 -1
@@ -222,7 +222,9 @@
  void
  flush_icache_range(char *start, unsigned long len)
  {
-	unsigned long i;
+	unsigned long i,addr;
+	addr = (unsigned long)start & ~31UL;
+	len += (unsigned long)start - addr;
  	for (i = 0;i < len; i += 32)
  	  asm volatile("fc.i %0"::"r"(start + i):"memory");
	asm volatile (";;sync.i;;":::"memory");
Acked-by on v1 1/1
#1 lore raw
Quoted 24 lines

-----Original Message-----
From: Terry Loftin [mailto:[email protected]]
Sent: 2006Äê10ÔÂ12ÈÕ 6:41
To: [email protected]; [email protected]
Cc: Zou, Nanhai; [email protected]
Subject: [PATCH] kexec-tools: ia64: icache flush should align to 32-bytes

Hi,
I've run into problems testing kexec/kdump on a Montecito revision C processor. In purgatory, __dummy_efi_function is copied onto the end of the command line boot parameter (command_line + command_line_len) and this address is used to replace the EFI call to set_virtual_address_map(). The copied range is then icache flushed.

The destination address is aligned to 16-bytes (in kexec-elf-ia64.c), but the fc.i instruction flushes a 32-byte range "associated" with that address. When my command line length is 16-byte aligned but not 32-byte aligned, this results in the first 16-bytes of __dummy_efi_function getting flushed (and the 16 bytes prior to that), but the second half of the function (the part with the br.ret) does not get flushed. kdump then hangs in purgatory. By adding a few spaces to my command line, it becomes both 16 and 32-byte aligned, and kdump works.

Quoted 7 lines

This patch makes icache_flush_range() align the start address to 32-bytes and account for the difference. The patch is against Horms kexec-tools-testing tree. As a side note, you could also fix this by just adding 32 to the length passed to flush_icache_range() but that hides the dependent behavior.

Thanks,
-T

It seems I was always testing with command line more than 16 bytes length.....

Thanks.

#2 lore raw
In reply to Terry Loftin
Quoted 12 lines
Quoted 12 lines

The destination address is aligned to 16-bytes (in kexec-elf-ia64.c), but the fc.i instruction flushes a 32-byte range "associated" with that address. When my command line length is 16-byte aligned but not 32-byte aligned, this results in the first 16-bytes of __dummy_efi_function getting flushed (and the 16 bytes prior to that), but the second half of the function (the part with the br.ret) does not get flushed. kdump then hangs in purgatory. By adding a few spaces to my command line, it becomes both 16 and 32-byte aligned, and kdump works.

Quoted 6 lines
Quoted 2 lines

Thanks,
-T

It seems I was always testing with command line more than 16 bytes length.....

Maybe an example would help my explanation a bit:

A line in kexec-elf-ia64.c sets command line length
to (strlen(command_line)+15) & ~15 thus aligning
it to 16-bytes, the needed alignment for an instruction bundle. (2 bundles or 32-bytes of code are copied).

The usual command line length is about 92 chars, so
command_line_len is (92+15) & ~15 = 96 which is
both 16-byte aligned and 32-byte aligned. This case works fine.

But we always add " machvec=dig" to the command line to avoid problems with the HP ioc. Thus we have a command line length of: (103+15) & ~15 = 112 which is 16-byte aligned but NOT 32-byte aligned. In this case, our kdump kernel always hangs in purgatory at the call to EFI set_virtual_address_map(). This is because the icache flush flushes on 32-byte boundaries, so only half of the code was flushed out of the cache. The bit of code with the return instruction was not flushed.

By inserting 10 spaces into the command line, the new length is (113+15) & ~15 = 128 which is both 16 and
32-byte aligned. In this case, kdump works fine. This
is probably why your tests always passed (you used a Tiger Montecito - you don't need "machvec=dig" and our tests (on HP Montecito) always failed.

With this fix and the iosapic fix, we've completed over 200 consective kdumps without failure using our buncho test. (Do a lot of disk and net stuff, and then panic in an IPI handler). So far, it looks pretty good.

-T