Add a hash value for each line in /dev/kmsg
Subsystems (9)
cover Add a hash value for each line in /dev/kmsg
Hello,
I resend patches as there was no reply. Any comments are welcome.
Changes from the previous post:
- Rebased to 3.11-rc2
- Update the explanation in the cover mail a bit
Brief description
=================
This patch series adds hash values of printk format strings into each line of /dev/kmsg outputs as follows:
6,154,325061,-,b7db707c@kernel/smp.c:554;Brought up 4 CPUs
These hash values allow use space to identify messages easily and do valuable things. There is no guarantee that the hash values are unique. However, user space can know which one collides to others from a message-and-hash catalog generated at build time, so finally it should be able to identify the messages. User space can also use filename:lineno information to do that, although the information is not portable among various kernel versions.
Background
==========
We sometimes want to identify each kernel message automatically because of the following purposes:
(1) Show more detailed information about the message by cooperating with external system
(2) Take actions automatically depending on the message, for example trigger a fail over or collect related information to analyze the problem
However, most of printk messages have dynamically generated parts, so it is difficult to identify the messages. Even if you can, it will be expensive.
To solve this issue, this patch series add a hash value of each printk format string. A catalog file which consists of hash, location, and format string lines is also generated at the kernel build time. User space tools can easily identify kernel messages by utilizing the catalog file.
Design
======
Hash values are displayed only when reading /dev/kmsg. Console and /proc/kmsg outputs are not affected by this feature.
The format in /dev/kmsg output is like this:
6,154,325061,-,b7db707c@kernel/smp.c:554;Brought up 4 CPUs
The 5th field is added, and it means "<32-bit hash>@<filename>:<lineno>". If you disable this feature (CONFIG_KMSG_HASH=n) or any cases the hash value is not calculated, the 5th field shows '-'.
The function printk() is replaced with macro version to create a metadata table which is used to calculate and keep hash values. Hash value calculation is done at the kernel build time, but not execution time. scripts/msghash tools scan the metadata table in vmlinux, calculate hash values, and put them back into the table. At the same time, msghash creates a catalog file named "<objfile>.msglist". Here is an example of its contents:
...
b6be0058@init/main.c:180,"Parameter %s is obsolete, ignored"
72dfe336@init/main.c:403,"Malformed early option '%s'"
85788d36@init/do_mounts.c:573,"Waiting for root device %s..."
...The format is <32-bit hash>@<filename>:<lineno>,"<message format>". Similar to /dev/kmsg, unprintable characters in message formats are escaped and a trailing newline is dropped.
Restrictions
============
Currently, only printk, pr_xxx, dev_printk, and dev_xxx are supported except for pr_debug with dynamic printk. Other printk variants won't show meaningful hash values. I'm going to support some of them in the future patches.
KERN_CONT cases are not dealt with well. Only the first printk's hash is output and continued printks are ignored.
Kernel module support has not been implemented yet. I'm going to do this in the future patches.
Note
====
When CONFIG_KMSG_HASH=y, a metadata table and additional file name strings are allocated in object files, and it increases the kernel size. Below is the difference of CONFIG_KMSG_HASH=n/y in some configuration. The total size increases about 2%.
text data bss dec hex filename
8135284 2395144 2211840 12742268 c26e7c vmlinux_hash_disabled
8255079 2519368 2211840 12986287 c627af vmlinux_hash_enabled
Thanks,
---
Hidehiro Kawai (5):
printk: Add msghash support for dev_printk
msghash: Add userland msghash tool tools/include: Add jhash.h
printk: add message hash values in /dev/kmsg output
printk: make printk a macro
.gitignore | 1 Makefile | 7 + drivers/base/core.c | 57 ++------- drivers/usb/storage/debug.c | 2 include/asm-generic/vmlinux.lds.h | 5 + include/linux/device.h | 73 +++++++---- include/linux/kmsghash.h | 19 +++ include/linux/printk.h | 31 ++++- kernel/printk.c | 81 +++++++++--- lib/Kconfig.debug | 22 +++ lib/dynamic_debug.c | 4 - net/core/dev.c | 2 scripts/Makefile | 1 scripts/link-vmlinux.sh | 5 + scripts/msghash/.gitignore | 1 scripts/msghash/Makefile | 7 + scripts/msghash/msghash.c | 241 +++++++++++++++++++++++++++++++++++++ scripts/msghash/msghash.sh | 45 +++++++ tools/include/tools/jhash.h | 192 +++++++++++++++++++++++++++++ 19 files changed, 698 insertions(+), 98 deletions(-) create mode 100644 include/linux/kmsghash.h create mode 100644 scripts/msghash/.gitignore create mode 100644 scripts/msghash/Makefile create mode 100644 scripts/msghash/msghash.c create mode 100755 scripts/msghash/msghash.sh create mode 100644 tools/include/tools/jhash.h
Signature · 3 lines
Hidehiro Kawai Hitach, Yokohama Research Laboratory Linux Technology Center
RFC PATCH 1/5 printk: make printk a macro
To allow building up metadata transparently for each printk(), make printk() a macro while keeping printk()'s API. Then, printk() calls _printk() which implements the original printk() function.
printk() is used from assembly sources, but they don't include printk.h and don't use this new printk() macro. This is addressed by assigning the printk symbol to _printk in the linker script.
---
include/asm-generic/vmlinux.lds.h | 1 + include/linux/printk.h | 4 +++- kernel/printk.c | 12 ++++++++---- 3 files changed, 12 insertions(+), 5 deletions(-)
include/asm-generic/vmlinux.lds.h +1 -0
@@ -393,6 +393,7 @@ *(.text.hot) \ *(.text) \ *(.ref.text) \ +VMLINUX_SYMBOL (printk) =VMLINUX_SYMBOL (_printk); \ MEM_KEEP(init.text) \MEM_KEEP (exit.text) \ *(.text.unlikely)
include/linux/printk.h +3 -1
@@ -121,7 +121,9 @@ asmlinkage int printk_emit(int facility, int level, const char *fmt, ...); asmlinkage __printf(1, 2) __cold -int printk(const char *fmt, ...); +int _printk(const char *fmt, ...); + +#define printk(fmt, args...) _printk(fmt, ## args)/* * Special printk facility for scheduler use only, _DO_NOT_USE_ !
kernel/printk.c +8 -4
@@ -1658,10 +1658,14 @@ asmlinkage int printk_emit(int facility, int level,EXPORT_SYMBOL (printk_emit);/** -* printk - print a kernel message +* _printk - print a kernel message * @fmt: format string * -* This is printk(). It can be called from any context. We want it to work. +* Now, printk() is a macro defined in include/linux/printk.h, and this +* function is its body. It can be called from any context, but please +* use printk() macro. Directly calling _printk() is not recommended. +* In assembly sources, you can call printk as in the past because the +* linker program links the printk symbol in assembly sources to _printk. * * We try to grab the console_lock. If we succeed, it's easy - we log the * output and call the console drivers. If we fail to get the semaphore, we
@@ -1678,7 +1682,7 @@ EXPORT_SYMBOL(printk_emit);* * See the vsnprintf() documentation for format string extensions over C99. */ -asmlinkage int printk(const char *fmt, ...) +asmlinkage int _printk(const char *fmt, ...) {va_list args;int r;
@@ -1697,7 +1701,7 @@ asmlinkage int printk(const char *fmt, ...) return r; } -EXPORT_SYMBOL (printk); +EXPORT_SYMBOL (_printk);#else /* CONFIG_PRINTK */
RFC PATCH 2/5 printk: add message hash values in /dev/kmsg output
In reply to Hidehiro KawaiThis patch adds the hash value for each printk line when reading /dev/kmsg as bellow:
6,154,325061,-,b7db707c@kernel/smp.c:554;Brought up 4 CPUs
Here, the 5th field is the 32-bit hash value followed by @<filename>:<lineno>. If this feature is disabled, i.e. CONFIG_KMSG_HASH=n, or the hash value hasn't be calculated, the 5th field simply shows '-'. Hash values don't appear in other than /dev/kmsg, thus they are not't displayed on the console and won't be written to /var/log/messages.
A metadata table which includes pointers to the format strings and location info of printk is placed between __start___kmsg_meta to __stop___kmsg_meta. Since the kernel doesn't calculate hash values dynamically, userland tools need to calculate hash values and place them into the table before running the kernel. To save the space, the pointers to the format strings are replaced with the calculated hash values.
In the current implementation, KERN_CONT cases are not dealt with well. Only a hash value of the first printk in the sequence is printed.
---
drivers/base/core.c | 3 +- include/asm-generic/vmlinux.lds.h | 4 ++ include/linux/kmsghash.h | 19 ++++++++++ include/linux/printk.h | 29 ++++++++++++++-- kernel/printk.c | 69 ++++++++++++++++++++++++++++--------- lib/Kconfig.debug | 17 +++++++++ 6 files changed, 122 insertions(+), 19 deletions(-) create mode 100644 include/linux/kmsghash.h
drivers/base/core.c +2 -1
@@ -2153,7 +2153,8 @@ int dev_vprintk_emit(int level, const struct device *dev, hdrlen =create_syslog_header (dev, hdr,sizeof (hdr)); -return vprintk_emit (0 , level, hdrlen ? hdr :NULL , hdrlen, fmt, args); +return vprintk_emit (0 , level, hdrlen ? hdr :NULL , hdrlen, + fmt, args,NULL ); }EXPORT_SYMBOL (dev_vprintk_emit);
include/asm-generic/vmlinux.lds.h +4 -0
@@ -188,6 +188,10 @@ VMLINUX_SYMBOL(__start___verbose) = .; \ *(__verbose ) \ VMLINUX_SYMBOL(__stop___verbose) = .; \ + . = ALIGN(8); \ + VMLINUX_SYMBOL(__start___kmsg_meta) = .; \ + *(__kmsg_meta ) \ + VMLINUX_SYMBOL(__stop___kmsg_meta) = .; \ LIKELY_PROFILE() \ BRANCH_PROFILE() \TRACE_PRINTKS ()
include/linux/kmsghash.h new file +19 -0
@@ -0,0 +1,19 @@ +#ifndef _LINUX_KMSGHASH_H +#define _LINUX_KMSGHASH_H + +/* kernel message metadata flags */ +#define KMSG_FLAGS_HASH_VALID (1<<0) +#define KMSG_FLAGS_DEFAULT 0 + +struct kmsg_meta { +const char *filename; +union { +/* format is replaced with hash at build time */ +const char *format; +u32 hash; + } u; +unsigned int lineno:18 ; +unsigned int flags:8 ; +} __attribute__((aligned (8 ))); + +#endif
include/linux/printk.h +27 -2
@@ -5,6 +5,7 @@#include <linux/init.h> #include <linux/kern_levels.h> #include <linux/linkage.h> +#include <linux/kmsghash.h> extern const char linux_banner[];extern const char linux_proc_banner[];
@@ -110,7 +111,7 @@ void early_printk(const char *s, ...) { } asmlinkage __printf(5, 0) int vprintk_emit(int facility, int level, const char *dict, size_t dictlen, - const char *fmt, va_list args); + const char *fmt, va_list args, struct kmsg_meta *meta); asmlinkage __printf(1, 0) int vprintk(const char *fmt, va_list args);
@@ -123,7 +124,31 @@ asmlinkage int printk_emit(int facility, int level, asmlinkage __printf(1, 2) __cold int _printk(const char *fmt, ...); -#define printk(fmt, args...) _printk(fmt, ## args) +__printf(2, 3) __cold +int __printk(struct kmsg_meta *meta, const char *fmt, ...); + +#ifdef CONFIG_KMSG_HASH +#define DEFINE_KMSG_METADATA(name, fmt) \ + static struct kmsg_meta __aligned(8) \ + __attribute__((section("__kmsg_meta"))) name = { \ + .filename = __FILE__, \ + .u.format = __builtin_constant_p(fmt) ? (fmt) : 0, \ + .lineno = __LINE__, \ + .flags = KMSG_FLAGS_DEFAULT, \ + } + +#define printk_(fmt, args...) \ +({ \ + DEFINE_KMSG_METADATA(_meta, (fmt)); \ + __printk(&_meta, (fmt), ##args); \ +}) + +#define printk(fmt, args...) printk_(fmt, ##args) + +#else + +#define printk(fmt, args...) __printk(0, fmt, ##args) +#endif /* CONFIG_KMSG_HASH */ /* * Special printk facility for scheduler use only, _DO_NOT_USE_ !
kernel/printk.c +53 -16
181 lines not on this page · raw message
lib/Kconfig.debug +17 -0
@@ -15,6 +15,23 @@ config PRINTK_TIME The behavior is also controlled by the kernel command line parameter printk.time=1. See Documentation/kernel-parameters.txt +config KMSG_HASH + bool "Show the hash value of each printk message format" + depends on PRINTK + default n + help + If you say y here, the hash value of each printk message + format is displayed when reading /dev/kmsg as bellow: + + 6,154,325061,-,b7db707c@kernel/smp.c:554;Brought up 4 CPUs + + Here, the 5th field is the 32-bit hash value followed by + @<location>. If this feature is disabled or the hash value + couldn't be calculated, the 5th field simply shows '-'. + + Please note that the kernel image size gets larger by + hundres Kbytes when this feature is enabled. + config DEFAULT_MESSAGE_LOGLEVEL int "Default message log level (1-7)" range 1 7
A message that is not in the archive (sent off-list or not yet received): CAEBb_QC3FgBuvnihvj4+xUYKCQX-KnLtW3203GDv-ByiOGsa-g@mail.gmail.com
Re: [RESEND RFC PATCH 2/5] printk: add message hash values in /dev/kmsg output
Hello,
(2013/07/26 0:01), Kazuo Ito wrote:> Hello, again.
Quoted 13 lines
Quoted 9 lines · Hidehiro Kawai
drivers/base/core.c +0
-0@@ -2153,7 +2153,8 @@ int dev_vprintk_emit(int level, const struct device *dev, hdrlen =create_syslog_header (dev, hdr,sizeof (hdr));
Quoted 14 lines
Quoted 7 lines
drivers/base/core.c +2
-1-return vprintk_emit (0 , level, hdrlen ? hdr :NULL , hdrlen, fmt, args); +return vprintk_emit (0 , level, hdrlen ? hdr :NULL , hdrlen, + fmt, args,NULL ); }
EXPORT_SYMBOL(dev_vprintk_emit);Position of struct
kmsg_meta* arguments can be made more consistent. Because it can occur withva_listwhich comes last by convention, it should be brought to front of these lists of arguments.
You are right. It should be consistent.
So I will place kmsg_meta * in the front of fmt in the next version. It would be preferred that the format string immediately followed by variable arguments.
Regards,
Signature · 3 lines
Hidehiro Kawai Hitachi, Yokohama Research Laboratory Linux Technology Center
RFC PATCH 3/5 tools/include: Add jhash.h
In reply to Hidehiro KawaiCopy include/linux/jhash.h to tools/include/tools/ for userland tools. The original jhash.h includes a couple of header files which is not suitable for userland. So unfold the only necessary part in those header files to the copied jhash.h
---
tools/include/tools/jhash.h | 192 +++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 192 insertions(+), 0 deletions(-) create mode 100644 tools/include/tools/jhash.h
tools/include/tools/jhash.h new file +192 -0
@@ -0,0 +1,192 @@ +#ifndef _TOOLS_JHASH_H +#define _TOOLS_JHASH_H + +/* jhash.h: Jenkins hash support. +* +* Copyright (C) 2006. Bob Jenkins ([email protected]) +* +* http://burtleburtle.net/bob/hash/ +* +* These are the credits from Bob's sources: +* +* lookup3.c, by Bob Jenkins, May 2006, Public Domain. +* +* These are functions for producing 32-bit hashes for hash table lookup. +* hashword(), hashlittle(), hashlittle2(), hashbig(), mix(), and final() +* are externally useful functions. Routines to test the hash are included +* if SELF_TEST is defined. You can use this free for any purpose. It's in +* the public domain. It has no warranty. +* +* Copyright (C) 2009-2010 Jozsef Kadlecsik ([email protected]) +* +* I've modified Bob's hash to be useful in the Linux kernel, and +* any bugs present are my fault. +* Jozsef +*/ + +#define __packed __attribute__((packed)) + +/* +* __get_unaligned_cpu32 was copied from +* include/linux/unaligned/packed_struct.h +*/ +struct __una_u32 {u32 x; } __packed; +static inline u32 __get_unaligned_cpu32 (const void *p) +{ +const struct __una_u32 *ptr = (const struct __una_u32 *)p; +return ptr->x; +} + +/* rol32 was copied from include/linux/bitops.h */ +/** +* rol32 - rotate a 32-bit value left +* @word: value to rotate +* @shift: bits to roll +*/ +static inline u32 rol32 (u32 word,unsigned int shift) +{ +return (word << shift) | (word >> (32 - shift)); +} + +/* Best hash sizes are of power of two */ +#define jhash_size (n) ((u32)1<<(n)) +/* Mask the hash value, i.e (value & jhash_mask(n)) instead of (value % n) */ +#define jhash_mask (n) (jhash_size(n)-1) + +/* __jhash_mix -- mix 3 32-bit values reversibly. */ +#define __jhash_mix (a, b, c) \ +{ \ + a -= c; a ^= rol32(c, 4); c += b; \ + b -= a; b ^= rol32(a, 6); a += c; \ + c -= b; c ^= rol32(b, 8); b += a; \ + a -= c; a ^= rol32(c, 16); c += b; \ + b -= a; b ^= rol32(a, 19); a += c; \ + c -= b; c ^= rol32(b, 4); b += a; \ +} + +/* __jhash_final - final mixing of 3 32-bit values (a,b,c) into c */ +#define __jhash_final (a, b, c) \ +{ \ + c ^= b; c -= rol32(b, 14); \ + a ^= c; a -= rol32(c, 11); \ + b ^= a; b -= rol32(a, 25); \ + c ^= b; c -= rol32(b, 16); \ + a ^= c; a -= rol32(c, 4); \ + b ^= a; b -= rol32(a, 14); \ + c ^= b; c -= rol32(b, 24); \ +} + +/* An arbitrary initial parameter */ +#define JHASH_INITVAL 0xdeadbeef + +/* jhash - hash an arbitrary key +* @k: sequence of bytes as key +* @length: the length of the key +* @initval: the previous hash, or an arbitray value +* +* The generic version, hashes an arbitrary sequence of bytes. +* No alignment or length assumptions are made about the input key. +* +* Returns the hash value of the key. The result depends on endianness. +*/ +static inline u32 jhash (const void *key,u32 length,u32 initval) +{ +u32 a, b, c; +const u8 *k = key; + +/* Set up the internal state */ + a = b = c =JHASH_INITVAL + length + initval; + +/* All but the last block: affect some 32 bits of (a,b,c) */ +while (length >12 ) { + a +=__get_unaligned_cpu32 (k); + b +=__get_unaligned_cpu32 (k +4 ); + c +=__get_unaligned_cpu32 (k +8 ); +__jhash_mix (a, b, c); + length -=12 ; + k +=12 ; + } +/* Last block: affect all 32 bits of (c) */ +/* All the case statements fall through */ +switch (length) { +case 12 : c += (u32 )k[11 ]<<24 ; +case 11 : c += (u32 )k[10 ]<<16 ; +case 10 : c += (u32 )k[9 ]<<8 ; +case 9 : c += k[8 ]; +case 8 : b += (u32 )k[7 ]<<24 ; +case 7 : b += (u32 )k[6 ]<<16 ; +case 6 : b += (u32 )k[5 ]<<8 ; +case 5 : b += k[4 ]; +case 4 : a += (u32 )k[3 ]<<24 ; +case 3 : a += (u32 )k[2 ]<<16 ; +case 2 : a += (u32 )k[1 ]<<8 ; +case 1 : a += k[0 ]; +__jhash_final (a, b, c); +case 0 :/* Nothing left to add */ +break ; + } + +return c; +} + +/* jhash2 - hash an array of u32's +* @k: the key which must be an array of u32's +* @length: the number of u32's in the key +* @initval: the previous hash, or an arbitray value +* +* Returns the hash value of the key. +*/ +static inline u32 jhash2 (const u32 *k,u32 length,u32 initval) +{ +u32 a, b, c; + +/* Set up the internal state */ + a = b = c =JHASH_INITVAL + (length<<2 ) + initval; + +/* Handle most of the key */ +while (length >3 ) { + a += k[0 ]; + b += k[1 ]; + c += k[2 ]; +__jhash_mix (a, b, c); + length -=3 ; + k +=3 ; + } + +/* Handle the last 3 u32's: all the case statements fall through */ +switch (length) { +case 3 : c += k[2 ]; +case 2 : b += k[1 ]; +case 1 : a += k[0 ]; +__jhash_final (a, b, c); +case 0 :/* Nothing left to add */ +break ; + } + +return c; +} + + +/* jhash_3words - hash exactly 3, 2 or 1 word(s) */ +static inline u32 jhash_3words (u32 a,u32 b,u32 c,u32 initval) +{ + a +=JHASH_INITVAL ; + b +=JHASH_INITVAL ; + c += initval; + +__jhash_final (a, b, c); + +return c; +} + +static inline u32 jhash_2words (u32 a,u32 b,u32 initval) +{ +return jhash_3words (a, b,0 , initval); +} + +static inline u32 jhash_1word (u32 a,u32 initval) +{ +return jhash_3words (a,0 ,0 , initval); +} + +#endif /* _TOOLS_JHASH_H */
RFC PATCH 4/5 msghash: Add userland msghash tool
In reply to Hidehiro KawaiThis patch adds scripts/msghash/msghash. msghash scans a kmsg_meta table between __start___kmsg_meta to __stop___kmsg_meta in the target object file (vmlinux), computes a hash value for each message format string, then replaces the format string with the hash value. Valid flags are also set to entries in the kmsg_meta table. When computing the hash value, prefixed loglevel characters are ignored.
Even if configured with CONFIG_KMSG_HASH=y, without running msghash, no hash values are output via /dev/kmsg. So msghash is executed during the build process, just after generating vmlinux.
msghash also generate a Hash-and-message catalog named "<objfile>.msglist". Each line in the .msglist file is output in the form of:
<hash in hex>@<file>:<lineno>,<message format>
.msglist file is useful to match /dev/kmsg messages with messages you concern.
Here is the example of .msglist:
...
b6be0058@init/main.c:180,"Parameter %s is obsolete, ignored" 72dfe336@init/main.c:403,"Malformed early option '%s'" 85788d36@init/do_mounts.c:573,"Waiting for root device %s..."
...
Currently, only vmlinux is supported. msghash can't update the kmsg_meta tables of kernel modules.
---
.gitignore | 1 Makefile | 7 + lib/Kconfig.debug | 5 + scripts/Makefile | 1 scripts/link-vmlinux.sh | 5 + scripts/msghash/.gitignore | 1 scripts/msghash/Makefile | 7 + scripts/msghash/msghash.c | 241 ++++++++++++++++++++++++++++++++++++++++++++ scripts/msghash/msghash.sh | 45 ++++++++ 9 files changed, 312 insertions(+), 1 deletions(-) create mode 100644 scripts/msghash/.gitignore create mode 100644 scripts/msghash/Makefile create mode 100644 scripts/msghash/msghash.c create mode 100755 scripts/msghash/msghash.sh
.gitignore +1 -0
@@ -32,6 +32,7 @@ modules.builtin *.lzo *.patch *.gcno +*.msglist # # Top-level generic files
Makefile +6 -1
@@ -766,6 +766,10 @@ vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) cmd_link-vmlinux = $(CONFIG_SHELL) $< $(LD) $(LDFLAGS) $(LDFLAGS_vmlinux) quiet_cmd_link-vmlinux = LINK $@ +# Remove old .msglist + cmd_rm-msglist = echo rm -f [email protected] +quiet_cmd_rm-msglist = CLEAN [email protected] + # Include targets which we want to # execute if the rest of the kernel build went well. vmlinux: scripts/link-vmlinux.sh $(vmlinux-deps) FORCE
@@ -1274,7 +1278,8 @@ clean: $(clean-dirs) -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \ -o -name '*.symtypes' -o -name 'modules.order' \ -o -name modules.builtin -o -name '.tmp_*.o.*' \ - -o -name '*.gcno' \) -type f -print | xargs rm -f + -o -name '*.gcno' -o -name '*.msglist' \) -type f -print |\ + xargs rm -f # Generate tags for editors # ---------------------------------------------------------------------------
lib/Kconfig.debug +5 -0
@@ -12,6 +12,11 @@ config PRINTK_TIME to /dev/kmsg. This flag just specifies if the timestamp should be included, not that the timestamp is recorded. + Hash values are calculated during the build process and + put into a table in object files. Hash-and-message catalogs + named <objfile>.msglist are also generated for convenience + in message handling in userland. + The behavior is also controlled by the kernel command line parameter printk.time=1. See Documentation/kernel-parameters.txt
scripts/Makefile +1 -0
@@ -37,6 +37,7 @@ subdir-$(CONFIG_MODVERSIONS) += genksyms subdir-y += mod subdir-$(CONFIG_SECURITY_SELINUX) += selinux subdir-$(CONFIG_DTC) += dtc +subdir-$(CONFIG_KMSG_HASH) += msghash # Let clean descend into subdirs subdir- += basic kconfig package selinux
scripts/link-vmlinux.sh +5 -0
@@ -228,5 +228,10 @@ if [ -n "${CONFIG_KALLSYMS}" ]; then fi fi +if [ -n "${CONFIG_KMSG_HASH}" ]; then + rm -f vmlinux.msglist + ${srctree}/scripts/msghash/msghash.sh vmlinux || exit 1 +fi + # We made a new kernel - delete old version file rm -f .old_version
scripts/msghash/.gitignore new file +1 -0
@@ -0,0 +1 @@ +msghash
scripts/msghash/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +hostprogs-y := msghash + +HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include + +always := $(hostprogs-y) +clean-files := $(hostprogs-y) +
scripts/msghash/msghash.c new file +241 -0
241 lines not on this page · raw message
scripts/msghash/msghash.sh new file +45 -0
45 lines not on this page · raw message
RFC PATCH 5/5 printk: Add msghash support for dev_printk
In reply to Hidehiro KawaiAdd msghash support for dev_printk and its variants (dev_xxx). Without this patch, no kmsg_meta data are passed to vprintk_emit, and this means no hash value is output.
To pass the metadata, this patch changes dev_printk and its variants to macros which construct the metadata in the similar way of printk. No API of those functions are changed.
---
drivers/base/core.c | 56 +++++++++------------------------ drivers/usb/storage/debug.c | 2 + include/linux/device.h | 73 +++++++++++++++++++++++++------------------ lib/dynamic_debug.c | 4 +- net/core/dev.c | 2 + 5 files changed, 62 insertions(+), 75 deletions(-)
drivers/base/core.c +15 -41
@@ -2145,8 +2145,8 @@ create_syslog_header(const struct device *dev, char *hdr, size_t hdrlen) }EXPORT_SYMBOL (create_syslog_header); -int dev_vprintk_emit (int level,const struct device *dev, -const char *fmt,va_list args) +int dev_vprintk_emit (struct kmsg_meta *meta,int level, +const struct device *dev,const char *fmt,va_list args) {char hdr[128 ];size_t hdrlen;
@@ -2154,18 +2154,19 @@ int dev_vprintk_emit(int level, const struct device *dev, hdrlen =create_syslog_header (dev, hdr,sizeof (hdr));return vprintk_emit (0 , level, hdrlen ? hdr :NULL , hdrlen, - fmt, args,NULL ); + fmt, args, meta); }EXPORT_SYMBOL (dev_vprintk_emit); -int dev_printk_emit (int level,const struct device *dev,const char *fmt, ...) +int dev_printk_emit (struct kmsg_meta *meta,int level, +const struct device *dev,const char *fmt, ...) {va_list args;int r;va_start (args, fmt); - r =dev_vprintk_emit (level, dev, fmt, args); + r =dev_vprintk_emit (meta, level, dev, fmt, args);va_end (args);
@@ -2173,19 +2174,20 @@ int dev_printk_emit(int level, const struct device *dev, const char *fmt, ...) }EXPORT_SYMBOL (dev_printk_emit); -static int __dev_printk (const char *level,const struct device *dev, -struct va_format *vaf) +static int __dev_printk (struct kmsg_meta *meta,const char *level, +const struct device *dev,struct va_format *vaf) {if (!dev) -return printk ("%s(NULL device *): %pV" , level, vaf); +return __printk (meta,"%s(NULL device *): %pV" , level, vaf); -return dev_printk_emit (level[1 ] -'0' , dev, +return dev_printk_emit (meta, level[1 ] -'0' , dev,"%s %s: %pV" ,dev_driver_string (dev),dev_name (dev), vaf); } -int dev_printk (const char *level,const struct device *dev, -const char *fmt, ...) +__printf(4, 5) +int _dev_printk (struct kmsg_meta *meta,const char *level, +const struct device *dev,const char *fmt, ...) {struct va_format vaf;va_list args;
@@ -2196,40 +2198,12 @@ int dev_printk(const char *level, const struct device *dev, vaf.fmt = fmt; vaf.va = &args; - r =__dev_printk (level, dev, &vaf); + r =__dev_printk (meta, level, dev, &vaf);va_end (args); return r; } -EXPORT_SYMBOL (dev_printk); - -#define define_dev_printk_level (func, kern_level) \ -int func(const struct device *dev, const char *fmt, ...) \ -{ \ - struct va_format vaf; \ - va_list args; \ - int r; \ - \ - va_start(args, fmt); \ - \ - vaf.fmt = fmt; \ - vaf.va = &args; \ - \ - r = __dev_printk(kern_level, dev, &vaf); \ - \ - va_end(args); \ - \ - return r; \ -} \ -EXPORT_SYMBOL(func); - -define_dev_printk_level (dev_emerg,KERN_EMERG ); -define_dev_printk_level (dev_alert,KERN_ALERT ); -define_dev_printk_level (dev_crit,KERN_CRIT ); -define_dev_printk_level (dev_err,KERN_ERR ); -define_dev_printk_level (dev_warn,KERN_WARNING ); -define_dev_printk_level (dev_notice,KERN_NOTICE ); -define_dev_printk_level (_dev_info,KERN_INFO ); +EXPORT_SYMBOL (_dev_printk);#endif
drivers/usb/storage/debug.c +1 -1
@@ -186,7 +186,7 @@ int usb_stor_dbg(const struct us_data *us, const char *fmt, ...)va_start (args, fmt); - r =dev_vprintk_emit (7 , &us->pusb_dev->dev, fmt, args); + r =dev_vprintk_emit (NULL ,7 , &us->pusb_dev->dev, fmt, args);va_end (args);
include/linux/device.h +43 -30
85 lines not on this page · raw message
lib/dynamic_debug.c +2 -2
@@ -578,7 +578,7 @@ int __dynamic_dev_dbg(struct _ddebug *descriptor, }else {char buf[PREFIX_SIZE ]; - res =dev_printk_emit (7 , dev,"%s%s %s: %pV" , + res =dev_printk_emit (NULL ,7 , dev,"%s%s %s: %pV" ,dynamic_emit_prefix (descriptor, buf),dev_driver_string (dev),dev_name (dev), &vaf);
@@ -610,7 +610,7 @@ int __dynamic_netdev_dbg(struct _ddebug *descriptor,if (dev && dev->dev.parent) {char buf[PREFIX_SIZE ]; - res = dev_printk_emit(7, dev->dev.parent, + res = dev_printk_emit(NULL, 7, dev->dev.parent, "%s%s %s %s: %pV", dynamic_emit_prefix(descriptor, buf), dev_driver_string(dev->dev.parent),
net/core/dev.c +1 -1
@@ -6258,7 +6258,7 @@ static int __netdev_printk(const char *level, const struct net_device *dev,int r;if (dev && dev->dev.parent) { - r = dev_printk_emit(level[1] - '0', + r = dev_printk_emit(NULL, level[1] - '0', dev->dev.parent, "%s %s %s: %pV", dev_driver_string(dev->dev.parent),
Quoted 20 lines · Hidehiro Kawai
================
This patch series adds hash values of printk format strings into each line of/dev/kmsgoutputs as follows:6,154,325061,-,b7db707c@kernel/smp.c:554;Brought up 4 CPUsThese hash values allow use space to identify messages easily and do valuable things. There is no guarantee that the hash values are unique. However, user space can know which one collides to others from a message-and-hash catalog generated at build time, so finally it should be able to identify the messages. User space can also use filename:lineno information to do that, although the information is not portable among various kernel versions.
Background
==========
We sometimes want to identify each kernel message automatically because of the following purposes:
Quoted 5 lines
(1) Show more detailed information about the message by cooperating with external system
(2) Take actions automatically depending on the message, for example trigger a fail over or collect related information to analyze the problem
If there are really "take action" uses, perhaps it'd be better to mark the specific areas of code where this needs to be done and add a notifier function there instead of depending on post effect processing.
Hello,
Quoted 26 lines · Hidehiro Kawai
Quoted 20 lines
================
This patch series adds hash values of printk format strings into each line of/dev/kmsgoutputs as follows:6,154,325061,-,b7db707c@kernel/smp.c:554;Brought up 4 CPUsThese hash values allow use space to identify messages easily and do valuable things. There is no guarantee that the hash values are unique. However, user space can know which one collides to others from a message-and-hash catalog generated at build time, so finally it should be able to identify the messages. User space can also use filename:lineno information to do that, although the information is not portable among various kernel versions.
Background
==========
We sometimes want to identify each kernel message automatically because of the following purposes:Quoted 5 lines
(1) Show more detailed information about the message by cooperating with external system
(2) Take actions automatically depending on the message, for example trigger a fail over or collect related information to analyze the problem
Quoted 5 lines
If there are really "take action" uses, perhaps it'd be better to mark the specific areas of code where this needs to be done and add a notifier function there instead of depending on post effect processing.
It is one of the solution if we can add such notifiers easily, but I think it's not true. Adding a notifier means that it gives the portion of the codes fixed meaning. This is similar to adding a tracepoint. Maintainer and developers have to keep the meaning consistent, but they wouldn't like to do that so much. Moreover, events users want to handle differ depending on users and system vendors. As the result, we would have to add many notifiers into many places.
On the other hand, printk message handling is a flexible, readily available and multipurpose way.
Regards,
Signature · 3 lines
Hidehiro Kawai Hitachi, Yokohama Research Laboratory Linux Technology Center
A message that is not in the archive (sent off-list or not yet received): CAEBb_QCkTc++pOpv83xdgnZgeUDa1-QK8McY6yqwe2wZN3zt5g@mail.gmail.com
Hello,
(2013/07/25 23:56), Kazuo Ito wrote:> Hello,
Quoted 33 lines
Quoted 19 lines · Hidehiro Kawai
.gitignore | 1 Makefile | 7 + drivers/base/core.c | 57 ++------- drivers/usb/storage/debug.c | 2 include/asm-generic/vmlinux.lds.h | 5 + include/linux/device.h | 73 +++++++---- include/linux/kmsghash.h | 19 +++ include/linux/printk.h | 31 ++++- kernel/printk.c | 81 +++++++++--- lib/Kconfig.debug | 22 +++ lib/dynamic_debug.c | 4 - net/core/dev.c | 2 scripts/Makefile | 1 scripts/link-vmlinux.sh | 5 + scripts/msghash/.gitignore | 1 scripts/msghash/Makefile | 7 + scripts/msghash/msghash.c | 241 +++++++++++++++++++++++++++++++++++++ scripts/msghash/msghash.sh | 45 +++++++Quoted 9 lines
tools/include/tools/jhash.h | 192 +++++++++++++++++++++++++++++ 19 files changed, 698 insertions(+), 98 deletions(-) create mode 100644 include/linux/kmsghash.h create mode 100644 scripts/msghash/.gitignore create mode 100644 scripts/msghash/Makefile create mode 100644 scripts/msghash/msghash.c create mode 100755 scripts/msghash/msghash.sh create mode 100644 tools/include/tools/jhash.h
Quoted 4 lines
As for one that once tried to implement a similar idea (in vein at LinuxCon Japan a couple of years ago), I heartily welcome this patch while feeling amount of changes might be perceived to be too much, requiring some tradeoff analysis with other implementation options.
I agreed. The current implementation has advantages and drawbacks, and there are some implementation options. We have to find a convincing way.
Quoted 7 lines
This patch precalculates hash values and wraps
callers of printk around in macros so that it hardly ever causes runtime overhead, it could avoid hash collisions if needs be, and many creative uses of printk
can be dealt with, in particular nested formats in %pV, which was my downfall... The cost is numbers of required changes.
Quoted 7 lines
Calculating hash values in vprink_* would make spreading changes across all the printk-derived functions and macros unnecessary, but would preclude use of elaborate hash functions and, in cases like
dev_printk(), would require a way to find or extract an appropriate format string. The latter requires complex pattern matching, lots of guess-work, has to deal with %pV, and will never be perfect.
Actually, in my first prototype implementation (but not published), hash values are calculated dynamically when vprintk_emit is called. Changed lines are less than this implementation, and it can also handle %pV cases. But as you say, extracting messages and calculating hashes preliminarily is the difficult part. I thought this difficulty will make people not use this feature. So I took a way manipulating metadata in printk macros.
Regards,
--
Hidehiro Kawai
Hitachi, Yokohama Research Laboratory
Linux Technology Center