Skip to contentSkip to conversation

RAMOOPS: few questions

PATCH Maxim Levitsky 2 participants 3 messages Jun 1, 2010 – Jun 2, 2010 Library code

lore
Share & tools
lore
#0 lore raw

Hi,

I want to switch over my own patch that does more or less the same that this driver.

As I see this driver uses ioremap to reserve the memory. However, I have a question about how to tell kernel that specific memory region is reserved.

Well, I use now mem=... to do so, but this wastes space, because the region I need can't be near end of the RAM.

I tried to use 'memmap=20M$0x70000000' on kernel command line, but that just makes kernel reboot.
I think its a bug though.

Also, does this driver dupms all kernel log or only panic/oops message? My patch actually makes kernel store actual printk buffer in fixed location, thus it is very robust. It was rejected though in favour of solution you eventually implemented.

Also I would like to extend your driver to dump old contents of the ram buffer to kernel log, so one wouldn't worry about erasing in on next boot after crash.
Don't know when I do that though.

Best regards,
Maxim Levitsky.

For reference my patch attached.

commit 58316c061f10839dfd5ebfd6ed4def6370de7b62

Author: Maxim Levitsky <[email protected]>
Date:   Thu May 27 22:49:01 2010 +0300

printk: Allow to store log buffer into video memory

This patch adds new kernel parameter printk_address= that will allow it to store the printk buffer in arbitary (I/O) memory address.

If you own a system that has discrete video ram, and it isn't cleared automaticly by bios on reboot, you can use this as a black box recorder of crashes.

If debugfs is enabled, log of last boot is copied into system ram, and can be accessed via debugfs, for example cat /sys/kernel/debug/printk/crash_dmesg

kernel/printk.c +102 -0
index 444b770..68df9fd 100644
@@ -39,6 +39,7 @@
 #include <linux/syslog.h>
 
 #include <asm/uaccess.h>
+#include <linux/debugfs.h>
 
 /*
  * for_each_console() allows you to iterate on each console
@@ -146,6 +147,7 @@ static char *log_buf = __log_buf;
 static int log_buf_len = __LOG_BUF_LEN;
 static unsigned logged_chars; /* Number of chars produced since last read+clear operation */
 static int saved_console_loglevel = -1;
+static unsigned int printk_phys_address;
 
 #ifdef CONFIG_KEXEC
 /*
@@ -170,6 +172,9 @@ static int __init log_buf_len_setup(char *str)
 	unsigned size = memparse(str, &str);
 	unsigned long flags;
 
+	if (printk_phys_address)
+		return 1;
+
 	if (size)
 		size = roundup_pow_of_two(size);
 	if (size > log_buf_len) {
@@ -206,6 +211,103 @@ out:
 
 __setup("log_buf_len=", log_buf_len_setup);
 
+
+#ifdef CONFIG_HWMEM_PRINTK
+
+#define EARLY_LOG_PAGES (8 * PAGE_SIZE)
+char *old_log_buf;
+struct debugfs_blob_wrapper crash_dmesg_wrapper;
+
+static int __init printk_phys_address_setup(char *str)
+{
+	unsigned long flags;
+	void *mem_address;
+
+	get_option(&str, &printk_phys_address);
+	if (!printk_phys_address)
+		return 0;
+
+	/* temporarly map first few pages of log memory */
+	mem_address = early_ioremap(printk_phys_address,  EARLY_LOG_PAGES);
+	if (!mem_address)
+		return 0;
+
+	printk(KERN_INFO "Logging kernel messages into HW memory at %08x\n",
+								printk_phys_address);
+	/* allocate saved log buffer, and save the log memory that we
+	  will otherwise overwrite */
+	old_log_buf = alloc_bootmem(__LOG_BUF_LEN);
+	if (old_log_buf)
+		memcpy(old_log_buf, mem_address, EARLY_LOG_PAGES);
+
+	/* clear log memory now */
+	memset(mem_address, 0, EARLY_LOG_PAGES);
+
+	/* copy current printk buffer to log memory, and switch to new buffer */
+	spin_lock_irqsave(&logbuf_lock, flags);
+	memcpy(mem_address, log_buf, EARLY_LOG_PAGES);
+	log_buf = mem_address;
+	log_buf_len = EARLY_LOG_PAGES;
+	spin_unlock_irqrestore(&logbuf_lock, flags);
+
+	return 1;
+}
+__setup("printk_address=", printk_phys_address_setup);
+
+static int printk_phys_address_late(void)
+{
+
+	char *mem_address, *early_mem_address;
+	unsigned long flags;
+	struct dentry *dbgfs_dir;
+
+	if (!printk_phys_address)
+		return 0;
+
+	/* now do late initialization */
+	mem_address = ioremap(printk_phys_address,  __LOG_BUF_LEN);
+
+	if (!mem_address) {
+		printk(KERN_ALERT "Can't fully map hardware kernel log memory."
+			" Log buffer limited to %lu KB\n", EARLY_LOG_PAGES);
+		return  0;
+	}
+
+	/* save the rest of log memory, and publish it */
+	if (old_log_buf) {
+		memcpy(old_log_buf + EARLY_LOG_PAGES,
+			mem_address + EARLY_LOG_PAGES,
+				__LOG_BUF_LEN - EARLY_LOG_PAGES);
+
+		crash_dmesg_wrapper.data = old_log_buf;
+		crash_dmesg_wrapper.size = __LOG_BUF_LEN;
+
+		dbgfs_dir = debugfs_create_dir("printk", NULL);
+
+		if (dbgfs_dir > 0)
+			debugfs_create_blob("crash_dmesg", S_IRUSR, dbgfs_dir,
+				&crash_dmesg_wrapper);
+	}
+
+	/* clear rest of the log memory now */
+	memset(mem_address + EARLY_LOG_PAGES , 0,
+		__LOG_BUF_LEN - EARLY_LOG_PAGES);
+
+
+	/* switch to the full log memory now */
+	spin_lock_irqsave(&logbuf_lock, flags);
+	early_mem_address = log_buf;
+	log_buf = mem_address;
+	log_buf_len = __LOG_BUF_LEN;
+	spin_unlock_irqrestore(&logbuf_lock, flags);
+
+	/* free temp mapping of the log memory */
+	early_iounmap(early_mem_address, EARLY_LOG_PAGES);
+	return 1;
+}
+postcore_initcall(printk_phys_address_late);
+#endif
+
 #ifdef CONFIG_BOOT_PRINTK_DELAY
 
 static unsigned int boot_delay; /* msecs delay after each printk during bootup */
lib/Kconfig.debug +31 -0
index e722e9d..aa714ce 100644
@@ -758,6 +758,37 @@ config BOOT_PRINTK_DELAY
 	  BOOT_PRINTK_DELAY also may cause DETECT_SOFTLOCKUP to detect
 	  what it believes to be lockup conditions.
 
+config HWMEM_PRINTK
+	bool "Log printk message buffer into video ram (DANGEROUS)"
+	depends on DEBUG_KERNEL && PRINTK
+	help
+	  This option allows to place kernel log buffer into pre-defined
+	  area, somewhere in memory space.
+	  It is intended to place this buffer into video ram assuming it
+	  isn't cleared on reboot.
+	  This creates some sort of black box recorder and can be very useful
+	  to debug several problems, especially 'panics' that happen while you
+	  use the X window system.
+
+	  To use, first ensure that you aren't using X, or that you tell video
+	  driver not to use all the video ram
+	  (easy way is to use the 'vesa' X driver)
+
+	  Then, pick an address within the video memory,
+	  (best somewhere in middle), and boot kernel with
+	  printk_address=$ADDRESS
+
+	  If you also select debugfs support, you can easily look at
+	  kernel log of failed boot at:
+	  /sys/kernel/debug/printk/crash_dmesg
+
+	  (Assuming you mounted debugfs on /sys/kernel/debug)
+
+	  Misuse of this option can be DANGEROUS, as it makes kernel write at
+	  arbitary (selected by you) hardware memory range.
+
+	  It is only intended for debbuging, so say 'no' if not sure
+
 config RCU_TORTURE_TEST
 	tristate "torture tests for RCU"
 	depends on DEBUG_KERNEL
#1 lore raw

Il 01/06/2010 21:32, Maxim Levitsky ha scritto:

Quoted 5 lines

Hi,

I want to switch over my own patch that does more or less the same that this driver.

Quoted 7 lines

As I see this driver uses ioremap to reserve the memory. However, I have a question about how to tell kernel that specific memory region is reserved.

Well, I use now mem=... to do so, but this wastes space, because the region I need can't be near end of the RAM.

Have you got any particular constraints?

Quoted 5 lines

I tried to use 'memmap=20M$0x70000000' on kernel command line, but that just makes kernel reboot.
I think its a bug though.

Also, does this driver dupms all kernel log or only panic/oops message?

It dumps all, but there's a fixed size for each dump, so you can see the panic/oops information and the latest prints of the kernel before panic/oops.

Quoted 9 lines

My patch actually makes kernel store actual printk buffer in fixed location, thus it is very robust. It was rejected though in favour of solution you eventually implemented.

Also I would like to extend your driver to dump old contents of the ram buffer to kernel log, so one wouldn't worry about erasing in on next boot after crash.
Don't know when I do that though.

ramoops uses a circular buffer of fixed size to store more information. The number of "dumps" depends on the region size of memory you use.

Regards,

Marco

#2 lore raw
Quoted 10 lines · Marco Stornelli

Il 01/06/2010 21:32, Maxim Levitsky ha scritto:

Quoted 9 lines

Hi,

I want to switch over my own patch that does more or less the same that this driver.

As I see this driver uses ioremap to reserve the memory. However, I have a question about how to tell kernel that specific memory region is reserved.

Quoted 5 lines
Quoted 3 lines

Well, I use now mem=... to do so, but this wastes space, because the region I need can't be near end of the RAM.

Have you got any particular constraints?

The trick I use, and works on most systems is that bios doesn't bother to clear RAM on reboot. It still uses it for its own purposes.

Quoted 10 lines
Quoted 6 lines

I tried to use 'memmap=20M$0x70000000' on kernel command line, but that just makes kernel reboot.
I think its a bug though.

Also, does this driver dupms all kernel log or only panic/oops message?

It dumps all, but there's a fixed size for each dump, so you can see the panic/oops information and the latest prints of the kernel before panic/oops.

Thats fine.

Quoted 5 lines
Quoted 5 lines

My patch actually makes kernel store actual printk buffer in fixed location, thus it is very robust. It was rejected though in favour of solution you eventually implemented.

Quoted 8 lines
Quoted 5 lines

Also I would like to extend your driver to dump old contents of the ram buffer to kernel log, so one wouldn't worry about erasing in on next boot after crash.
Don't know when I do that though.

ramoops uses a circular buffer of fixed size to store more information. The number of "dumps" depends on the region size of memory you use.

You mean it is smart enough not to overwrite the reserved region on next boot after the crash provided that I managed to tell kernel not to use it?

This definitely looks promising. I was thinking do to the same thing when I get free time.

There are still issues though comparing to mine implementation.

Namely, my approach allows to look at kernel log of last kernel even if there was no panic/oops, and this comes handy when system has reset button, and your graphical card decides to hang....
However the dump mechanism isn't suitable for that task, but rather a console.

And second problem is that your driver kicks in quite late stage, while I managed to start the logging quite early.
Maybe its possible to switch from module_init to some early initcall. Note that then I had to use ioremap_early.

And last thing that that I suffer from too, is inability to reserve a region from kernel command line.

memmap=20M$0x70000000 doesn't work.

Thanks for your driver.

Best regards,
Maxim Levitsky