printk: move dictionaries to meta data
Subsystems (7)
John Ogness
maintainercover printk: move dictionaries to meta data
Hello,
Here is a series to move dictionary properties (currently only SUBSYSTEM and DEVICE exist) into the meta data of a record, thus eliminating the need for the dict ring. This change affects how the dictionaries are stored, but does not affect how they are presented to userspace.
The main purpose of the change is to address concerns [0] about the reliability of dictionary properties as well as allowing to efficiently expand the type and number of properties available [1].
This series is based heavily on the proof of concept [2] from Petr Mladek. (Petr, feel free to add Co-developed-by tags.)
The series is based on the printk-rework branch of the printk git tree:
f5f022e53b87 ("printk: reimplement log_cont using record extension")
John Ogness
[0] https://lkml.kernel.org/r/20200904151336.GC20558@alley
[1] https://lkml.kernel.org/r/[email protected]
[2] https://lkml.kernel.org/r/20200911095035.GI3864@alley
John Ogness (3):
printk: move printk_info into separate array
printk: move dictionary keys to dev_printk_info
printk: remove dict ring
Documentation/admin-guide/kdump/gdbmacros.txt | 73 ++--- drivers/base/core.c | 46 +-- include/linux/dev_printk.h | 8 + include/linux/printk.h | 6 +- kernel/printk/internal.h | 4 +- kernel/printk/printk.c | 209 ++++++------- kernel/printk/printk_ringbuffer.c | 292 ++++++++---------- kernel/printk/printk_ringbuffer.h | 95 ++---- kernel/printk/printk_safe.c | 2 +- scripts/gdb/linux/dmesg.py | 16 +- 10 files changed, 336 insertions(+), 415 deletions(-)
Signature · 7 lines
2.20.1 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
John Ogness
maintainerPATCH 1/3 printk: move printk_info into separate array
The majority of the size of a descriptor is taken up by meta data, which is often not of interest to the ringbuffer (for example, when performing state checks). Since descriptors are often temporarily stored on the stack, keeping their size minimal will help reduce stack pressure.
Rather than embedding the printk_info into the descriptor, create a separate printk_info array. The index of a descriptor in the descriptor array corresponds to the printk_info with the same index in the printk_info array. The rules for validity of a printk_info match the existing rules for the data blocks: the descriptor must be in a consistent state.
--- kernel/printk/printk.c | 17 +++- kernel/printk/printk_ringbuffer.c | 145 +++++++++++++++++++----------- kernel/printk/printk_ringbuffer.h | 29 +++--- 3 files changed, 125 insertions(+), 66 deletions(-)
kernel/printk/printk.c +15 -2
@@ -959,11 +959,11 @@ void log_buf_vmcoreinfo_setup(void)VMCOREINFO_STRUCT_SIZE (prb_desc_ring);VMCOREINFO_OFFSET (prb_desc_ring, count_bits);VMCOREINFO_OFFSET (prb_desc_ring, descs); +VMCOREINFO_OFFSET (prb_desc_ring, infos);VMCOREINFO_OFFSET (prb_desc_ring, head_id);VMCOREINFO_OFFSET (prb_desc_ring, tail_id);VMCOREINFO_STRUCT_SIZE (prb_desc); -VMCOREINFO_OFFSET (prb_desc, info);VMCOREINFO_OFFSET (prb_desc, state_var);VMCOREINFO_OFFSET (prb_desc, text_blk_lpos);VMCOREINFO_OFFSET (prb_desc, dict_blk_lpos);
@@ -1097,6 +1097,7 @@ static char setup_dict_buf[CONSOLE_EXT_LOG_MAX] __initdata;void __init setup_log_buf (int early) { +struct printk_info *new_infos;unsigned int new_descs_count;struct prb_desc *new_descs;struct printk_info info;
@@ -1156,6 +1157,17 @@ void __init setup_log_buf(int early)return ; } + new_descs_size = new_descs_count *sizeof (struct printk_info ); + new_infos =memblock_alloc (new_descs_size,LOG_ALIGN ); +if (unlikely (!new_infos)) { +pr_err ("log_buf_len: %zu info bytes not available\n" , + new_descs_size); +memblock_free (__pa (new_descs), new_log_buf_len); +memblock_free (__pa (new_dict_buf), new_log_buf_len); +memblock_free (__pa (new_log_buf), new_log_buf_len); +return ; + } +prb_rec_init_rd (&r, &info, &setup_text_buf[0 ],sizeof (setup_text_buf), &setup_dict_buf[0 ],sizeof (setup_dict_buf));
@@ -1163,7 +1175,8 @@ void __init setup_log_buf(int early)prb_init (&printk_rb_dynamic, new_log_buf,ilog2 (new_log_buf_len), new_dict_buf,ilog2 (new_log_buf_len), - new_descs,ilog2 (new_descs_count)); + new_descs,ilog2 (new_descs_count), + new_infos);logbuf_lock_irqsave (flags);
kernel/printk/printk_ringbuffer.c +94 -51
387 lines not on this page · raw message
kernel/printk/printk_ringbuffer.h +16 -13
@@ -58,7 +58,6 @@ struct prb_data_blk_lpos {* @state_var: A bitwise combination of descriptor ID and descriptor state. */ struct prb_desc { -struct printk_info info;atomic_long_t state_var;struct prb_data_blk_lpos text_blk_lpos;struct prb_data_blk_lpos dict_blk_lpos;
@@ -76,6 +75,7 @@ struct prb_data_ring {struct prb_desc_ring {unsigned int count_bits;struct prb_desc *descs; +struct printk_info *infos;atomic_long_t head_id;atomic_long_t tail_id; };
@@ -237,19 +237,8 @@ enum desc_state { static char _##name##_dict[1U << ((avgdictbits) + (descbits))] \ __aligned(__alignof__(unsigned long)); \ static struct prb_desc _##name##_descs[_DESCS_COUNT(descbits)] = { \ - /* this will be the first record reserved by a writer */ \ - [0] = { \ - .info = { \ -/* will be incremented to 0 on the first reservation */ \ - .seq = -(u64)_DESCS_COUNT(descbits), \ - }, \ - }, \ /* the initial head and tail */ \ [_DESCS_COUNT(descbits) - 1] = { \ - .info = { \ -/* reports the first seq value during the bootstrap phase */ \ - .seq = 0, \ - }, \/* reusable */ \ .state_var = ATOMIC_INIT(DESC0_SV(descbits)), \ /* no associated data block */ \
@@ -257,10 +246,23 @@ static struct prb_desc _##name##_descs[_DESCS_COUNT(descbits)] = { \ .dict_blk_lpos = FAILED_BLK_LPOS, \ }, \ }; \ +static struct printk_info _##name##_infos[_DESCS_COUNT(descbits)] = { \ +/* this will be the first record reserved by a writer */ \ + [0 ] = { \ +/* will be incremented to 0 on the first reservation */ \ + .seq = -(u64 )_DESCS_COUNT (descbits), \ + }, \ +/* the initial head and tail */ \ + [_DESCS_COUNT (descbits) -1 ] = { \ +/* reports the first seq value during the bootstrap phase */ \ + .seq =0 , \ + }, \ +}; \ static struct printk_ringbuffer name = { \ .desc_ring = { \ .count_bits = descbits, \ .descs = &_##name##_descs[0], \ + .infos = &_##name##_infos[0], \ .head_id = ATOMIC_INIT(DESC0_ID(descbits)), \ .tail_id = ATOMIC_INIT(DESC0_ID(descbits)), \ }, \
@@ -336,7 +338,8 @@ void prb_final_commit(struct prb_reserved_entry *e);void prb_init (struct printk_ringbuffer *rb,char *text_buf,unsigned int text_buf_size,char *dict_buf,unsigned int dict_buf_size, -struct prb_desc *descs,unsigned int descs_count_bits); +struct prb_desc *descs,unsigned int descs_count_bits, +struct printk_info *infos);unsigned int prb_record_text_space (struct prb_reserved_entry *e);/* Reader Interface */
Signature · 7 lines
2.20.1 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Petr Mladek
maintainerRe: [PATCH printk 1/3] printk: move printk_info into separate array
Quoted 6 lines · John Ogness
The majority of the size of a descriptor is taken up by meta data, which is often not of interest to the ringbuffer (for example, when performing state checks). Since descriptors are often temporarily stored on the stack, keeping their size minimal will help reduce stack pressure.
Quoted 6 lines
Rather than embedding the
printk_infointo the descriptor, create a separateprintk_infoarray. The index of a descriptor in the descriptor array corresponds to theprintk_infowith the same index in theprintk_infoarray. The rules for validity of aprintk_infomatch the existing rules for the data blocks: the descriptor must be in a consistent state.
I like this approach.
IMHO, it better separates dict_ring- and printk-specific metadata.
The three printk-specific values (seq, caller_id, len) are still accessed by the ring buffer code. It is not ideal. But it helps to hide tricky operations in the ring buffer API, keep the code more safe and sane. These exceptions are actually better visible now.
Quoted 11 lines
kernel/printk/printk.c +1
-0@@ -1097,6 +1097,7 @@ static char setup_dict_buf[CONSOLE_EXT_LOG_MAX] __initdata;void __init setup_log_buf (int early) { +struct printk_info *new_infos;unsigned int new_descs_count;struct prb_desc *new_descs;struct printk_info info;
kernel/printk/printk.c void __init setup_log_buf(int early) · line 1160
Quoted 4 lines
kernel/printk/printk.c +1
-0@@ -1156,6 +1157,17 @@ void __init setup_log_buf(int early)return ; } + new_descs_size = new_descs_count *sizeof (struct printk_info );
Must be stored into new variable, e.g. new_infos_size.=
kernel/printk/printk.c void __init setup_log_buf(int early) · line 1166
Quoted 6 lines
kernel/printk/printk.c +6
-0+ new_infos =memblock_alloc (new_descs_size,LOG_ALIGN ); +if (unlikely (!new_infos)) { +pr_err ("log_buf_len: %zu info bytes not available\n" , + new_descs_size); +memblock_free (__pa (new_descs), new_log_buf_len); +memblock_free (__pa (new_dict_buf), new_log_buf_len);
The above two calls have wrong size.
The same problem is there also in the error path when new_descs allocation fail. It might be better to handle this using some goto err_* tagrets.
Please, fix the old problem in a separate patch.
Quoted 12 lines
kernel/printk/printk.c +4
-0+memblock_free (__pa (new_log_buf), new_log_buf_len); +return ; + } +prb_rec_init_rd (&r, &info, &setup_text_buf[0 ],sizeof (setup_text_buf), &setup_dict_buf[0 ],sizeof (setup_dict_buf));
kernel/printk/printk_ringbuffer.c static bool copy_data(struct prb_data_ring *data_ring, · line 1767
Quoted 6 lines
kernel/printk/printk_ringbuffer.c +3
-0@@ -1726,12 +1762,12 @@ static bool copy_data(struct prb_data_ring *data_ring,/* * Actual cannot be less than expected. It can be more than expected * because of the trailing alignment padding. +* +* Note that invalid @len values can occur because the caller loads +* the value during an allowed data race.
I hope that this will not bite us in the future. The fact is that copying the entire struct printk_info in get_desc() is ugly and copy_data() has to be careful anyway.
kernel/printk/printk_ringbuffer.c static bool copy_data(struct prb_data_ring *data_ring, · line 1773
Quoted 12 lines
kernel/printk/printk_ringbuffer.c +1
-4*/ -if (WARN_ON_ONCE (data_size < (unsigned int )len)) { -pr_warn_once ("wrong data size (%u, expecting >=%hu) for data: %.*s\n" , - data_size, len, data_size, data); +if (data_size < (unsigned int )len)return false; - }kernel/printk/printk_ringbuffer.c +0
-0/* Caller interested in the line count? */ if (line_count)
Otherwise, I like this patch.
Best Regards,
Petr
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
John Ogness
maintainerRe: [PATCH printk 1/3] printk: move printk_info into separate array
Quoted 14 lines · Petr Mladek
Quoted 14 lines
kernel/printk/printk.c +1
-0@@ -1097,6 +1097,7 @@ static char setup_dict_buf[CONSOLE_EXT_LOG_MAX] __initdata;void __init setup_log_buf (int early) { +struct printk_info *new_infos;unsigned int new_descs_count;struct prb_desc *new_descs;struct printk_info info;@@ -1156,6 +1157,17 @@ void __init setup_log_buf(int early)return ; }
Quoted 3 lines
kernel/printk/printk.c void __init setup_log_buf(int early) · line 1160
Quoted 1 line
kernel/printk/printk.c +1
-0+ new_descs_size = new_descs_count *sizeof (struct printk_info );Must be stored into new variable, e.g. new_infos_size.=
Ack.
Quoted 9 lines
kernel/printk/printk.c void __init setup_log_buf(int early) · line 1166
Quoted 6 lines
kernel/printk/printk.c +6
-0+ new_infos =memblock_alloc (new_descs_size,LOG_ALIGN ); +if (unlikely (!new_infos)) { +pr_err ("log_buf_len: %zu info bytes not available\n" , + new_descs_size); +memblock_free (__pa (new_descs), new_log_buf_len); +memblock_free (__pa (new_dict_buf), new_log_buf_len);The above two calls have wrong size.
Quoted 5 lines
The same problem is there also in the error path when
new_descsallocation fail. It might be better to handle this using some goto err_* tagrets.Please, fix the old problem in a separate patch.
The "old problem" didn't exist. The problem is introduced with this series. I will fix it with appropriate goto err_* targets for v2.
Quoted 5 lines
Quoted 5 lines
kernel/printk/printk_ringbuffer.c +0
-0@@ -1726,12 +1762,12 @@ static bool copy_data(struct prb_data_ring *data_ring,/* * Actual cannot be less than expected. It can be more than expected
Quoted 8 lines
kernel/printk/printk_ringbuffer.c static bool copy_data(struct prb_data_ring *data_ring, · line 1767
Quoted 4 lines
kernel/printk/printk_ringbuffer.c +3
-0* because of the trailing alignment padding. +* +* Note that invalid @len values can occur because the caller loads +* the value during an allowed data race. I hope that this will not bite us in the future. The fact is that copying the entire struct
printk_infoinget_desc()is ugly andcopy_data()has to be careful anyway.
It isn't an issue because the state is verified again at the end of prb_read(). I added the comment because if all you are looking at is copy_data(), you may not know that @len was read on a data-race. Whereas inside of prb_read(), it is obvious that the memcpy() is a data-race.
John Ogness
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
John Ogness
maintainerPATCH 2/3 printk: move dictionary keys to dev_printk_info
In reply to John OgnessDictionaries are only used for SUBSYSTEM and DEVICE properties. The current implementation stores the property names each time they are used. This requires more space than otherwise necessary. Also, because the dictionary entries are currently considered optional, it cannot be relied upon that they are always available, even if the writer wanted to store them. These issues will increase should new dictionary properties be introduced.
Rather than storing the subsystem and device properties in the dict ring, introduce a struct dev_printk_info with separate fields to store only the property values. Embed this struct within the struct printk_info to provide guaranteed availability.
--- Documentation/admin-guide/kdump/gdbmacros.txt | 73 ++++---- drivers/base/core.c | 46 ++--- include/linux/dev_printk.h | 8 + include/linux/printk.h | 6 +- kernel/printk/internal.h | 4 +- kernel/printk/printk.c | 161 +++++++++--------- kernel/printk/printk_ringbuffer.h | 3 + kernel/printk/printk_safe.c | 2 +- scripts/gdb/linux/dmesg.py | 16 +- 9 files changed, 159 insertions(+), 160 deletions(-)
Documentation/admin-guide/kdump/gdbmacros.txt +33 -40
@@ -172,13 +172,13 @@ end define dump_record set var $desc = $arg0 - if ($argc > 1) - set var $prev_flags = $arg1 + set var $info = $arg1 + if ($argc > 2) + set var $prev_flags = $arg2 else set var $prev_flags = 0 end - set var $info = &$desc->info set var $prefix = 1 set var $newline = 1
@@ -237,44 +237,36 @@ define dump_record # handle dictionary data - set var $begin = $desc->dict_blk_lpos.begin % (1U << prb->dict_data_ring.size_bits) - set var $next = $desc->dict_blk_lpos.next % (1U << prb->dict_data_ring.size_bits) - - # handle data-less record - if ($begin & 1) - set var $dict_len = 0 - set var $dict = "" - else - # handle wrapping data block - if ($begin > $next) - set var $begin = 0 - end - - # skip over descriptor id - set var $begin = $begin + sizeof(long) - - # handle truncated message - if ($next - $begin < $info->dict_len) - set var $dict_len = $next - $begin - else - set var $dict_len = $info->dict_len + set var $dict = &$info->dev_info.subsystem[0] + set var $dict_len = sizeof($info->dev_info.subsystem) + if ($dict[0] != '\0') + printf " SUBSYSTEM=" + set var $idx = 0 + while ($idx < $dict_len) + set var $c = $dict[$idx] + if ($c == '\0') + loop_break + else + if ($c < ' ' || $c >= 127 || $c == '\\') + printf "\\x%02x", $c + else + printf "%c", $c + end + end + set var $idx = $idx + 1 end - - set var $dict = &prb->dict_data_ring.data[$begin] + printf "\n" end - if ($dict_len > 0) + set var $dict = &$info->dev_info.device[0] + set var $dict_len = sizeof($info->dev_info.device) + if ($dict[0] != '\0') + printf " DEVICE=" set var $idx = 0 - set var $line = 1 while ($idx < $dict_len) - if ($line) - printf " " - set var $line = 0 - end set var $c = $dict[$idx] if ($c == '\0') - printf "\n" - set var $line = 1 + loop_break else if ($c < ' ' || $c >= 127 || $c == '\\') printf "\\x%02x", $c
@@ -288,10 +280,10 @@ define dump_record end end document dump_record - Dump a single record. The first parameter is the descriptor - sequence number, the second is optional and specifies the - previous record's flags, used for properly formatting - continued lines. + Dump a single record. The first parameter is the descriptor, + the second parameter is the info, the third parameter is + optional and specifies the previous record's flags, used for + properly formatting continued lines. end define dmesg
@@ -311,12 +303,13 @@ define dmesg while (1) set var $desc = &prb->desc_ring.descs[$id % $desc_count] + set var $info = &prb->desc_ring.infos[$id % $desc_count] # skip non-committed record set var $state = 3 & ($desc->state_var.counter >> $desc_flags_shift) if ($state == $desc_committed || $state == $desc_finalized) - dump_record $desc $prev_flags - set var $prev_flags = $desc->info.flags + dump_record $desc $info $prev_flags + set var $prev_flags = $info->flags end set var $id = ($id + 1) & $id_mask
drivers/base/core.c +16 -30
79 lines not on this page · raw message
include/linux/dev_printk.h +8 -0
@@ -21,6 +21,14 @@struct device ; +#define PRINTK_INFO_SUBSYSTEM_LEN 16 +#define PRINTK_INFO_DEVICE_LEN 48 + +struct dev_printk_info { +char subsystem[PRINTK_INFO_SUBSYSTEM_LEN ]; +char device[PRINTK_INFO_DEVICE_LEN ]; +}; +#ifdef CONFIG_PRINTK __printf (3 ,0 )__cold
include/linux/printk.h +4 -2
@@ -158,10 +158,12 @@ static inline void printk_nmi_direct_enter(void) { }static inline void printk_nmi_direct_exit (void ) { }#endif /* PRINTK_NMI */ +struct dev_printk_info ; +#ifdef CONFIG_PRINTK -asmlinkage __printf(5, 0) +asmlinkage __printf(4, 0) int vprintk_emit(int facility, int level, - const char *dict, size_t dictlen, + const struct dev_printk_info *dev_info, const char *fmt, va_list args); asmlinkage __printf(1, 0)
kernel/printk/internal.h +2 -2
@@ -14,9 +14,9 @@extern raw_spinlock_t logbuf_lock; -__printf(5, 0) +__printf(4, 0)int vprintk_store (int facility,int level, -const char *dict,size_t dictlen, +const struct dev_printk_info *dev_info,const char *fmt,va_list args); __printf(1, 0) int vprintk_default(const char *fmt, va_list args);
kernel/printk/printk.c +81 -80
357 lines not on this page · raw message
kernel/printk/printk_ringbuffer.h +3 -0
@@ -4,6 +4,7 @@#define _KERNEL_PRINTK_RINGBUFFER_H#include <linux/atomic.h> +#include <linux/dev_printk.h> /* * Meta information about each stored message.
@@ -21,6 +22,8 @@ struct printk_info { u8 flags:5; /* internal record flags */ u8 level:3; /* syslog level */u32 caller_id;/* thread id or processor id */ + +struct dev_printk_info dev_info; };/*
kernel/printk/printk_safe.c +1 -1
@@ -375,7 +375,7 @@ __printf(1, 0) int vprintk_func(const char *fmt, va_list args) raw_spin_trylock(&logbuf_lock)) {int len; - len =vprintk_store (0 ,LOGLEVEL_DEFAULT ,NULL ,0 , fmt, args); + len =vprintk_store (0 ,LOGLEVEL_DEFAULT ,NULL , fmt, args);raw_spin_unlock (&logbuf_lock);defer_console_output ();return len;
scripts/gdb/linux/dmesg.py +11 -5
@@ -52,6 +52,12 @@ class LxDmesg(gdb.Command): addr = utils.read_ulong(desc_ring, off) descs = utils.read_memoryview(inf, addr, desc_sz * desc_ring_count).tobytes() + # read in info array + info_sz = printk_info_type.get_type().sizeof + off = prb_desc_ring_type.get_type()['infos'].bitpos // 8 + addr = utils.read_ulong(desc_ring, off) + infos = utils.read_memoryview(inf, addr, info_sz * desc_ring_count).tobytes() + # read in text data ring structure off = printk_ringbuffer_type.get_type()['text_data_ring'].bitpos // 8 addr = prb_addr + off
@@ -73,9 +79,8 @@ class LxDmesg(gdb.Command): begin_off = off + (prb_data_blk_lpos_type.get_type()['begin'].bitpos // 8) next_off = off + (prb_data_blk_lpos_type.get_type()['next'].bitpos // 8) - off = prb_desc_type.get_type()['info'].bitpos // 8 - ts_off = off + printk_info_type.get_type()['ts_nsec'].bitpos // 8 - len_off = off + printk_info_type.get_type()['text_len'].bitpos // 8 + ts_off = printk_info_type.get_type()['ts_nsec'].bitpos // 8 + len_off = printk_info_type.get_type()['text_len'].bitpos // 8 # definitions from kernel/printk/printk_ringbuffer.h desc_committed = 1
@@ -95,6 +100,7 @@ class LxDmesg(gdb.Command): while True: ind = did % desc_ring_count desc_off = desc_sz * ind + info_off = info_sz * ind # skip non-committed record state = 3 & (utils.read_u64(descs, desc_off + sv_off +
@@ -119,7 +125,7 @@ class LxDmesg(gdb.Command): # skip over descriptor id text_start = begin + utils.get_long_type().sizeof - text_len = utils.read_u16(descs, desc_off + len_off) + text_len = utils.read_u16(infos, info_off + len_off) # handle truncated message if end - text_start < text_len:
@@ -128,7 +134,7 @@ class LxDmesg(gdb.Command): text = text_data[text_start:text_start + text_len].decode( encoding='utf8', errors='replace') - time_stamp = utils.read_u64(descs, desc_off + ts_off) + time_stamp = utils.read_u64(infos, info_off + ts_off) for line in text.splitlines(): msg = u"[{time:12.6f}] {line}\n".format(
Signature · 7 lines
2.20.1 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk 2/3] printk: move dictionary keys to dev_printk_info
drivers/base/core.c void device_shutdown(void) · line 3832
Quoted 12 lines · John Ogness
drivers/base/core.c +2
-4if (dev->class) subsys = dev->class->name;else if (dev->bus) subsys = dev->bus->name;else -return 0 ; +return ; - pos +=snprintf (hdr + pos, hdrlen - pos,"SUBSYSTEM=%s" , subsys); -if (pos >= hdrlen) - goto overflow; +snprintf (dev_info->subsystem,sizeof (dev_info->subsystem), subsys);
It's unlikely that subsys would contain a %, but this will be yet another place to spend brain cycles ignoring if doing static analysis. So can we not do this. Either of strXcpy() for X=s,l will do the same thing, and likely faster.
Rasmus
Petr Mladek
maintainerRe: [PATCH printk 2/3] printk: move dictionary keys to dev_printk_info
Quoted 10 lines · Rasmus Villemoes
Quoted 8 lines · John Ogness
drivers/base/core.c +1
-1if (dev->class) subsys = dev->class->name;else if (dev->bus) subsys = dev->bus->name;else -return 0 ; +return ;
Quoted 9 lines
drivers/base/core.c void device_shutdown(void) · line 3832
Quoted 4 lines
drivers/base/core.c +1
-3- pos +=snprintf (hdr + pos, hdrlen - pos,"SUBSYSTEM=%s" , subsys); -if (pos >= hdrlen) - goto overflow; +snprintf (dev_info->subsystem,sizeof (dev_info->subsystem), subsys);It's unlikely that subsys would contain a %, but this will be yet another place to spend brain cycles ignoring if doing static analysis. So can we not do this. Either of
strXcpy()for X=s,l will do the same thing, and likely faster.
Good point! Better be on the safe size in a generic printk() API.
Well, I am afraid that this would be only small drop in a huge lake. class->name and bus->name seems to be passed to %s in so many *print*() calls all over the kernel code.
IMHO, this is not the right place to prevent the problem. Dangerous names must be prevented when a new bus, class, device is added.
Best Rergards,
Petr
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk 2/3] printk: move dictionary keys to dev_printk_info
Quoted 21 lines · Petr Mladek
Quoted 10 lines · Rasmus Villemoes
Quoted 8 lines · John Ogness
drivers/base/core.c +1
-1if (dev->class) subsys = dev->class->name;else if (dev->bus) subsys = dev->bus->name;else -return 0 ; +return ;Quoted 9 lines
drivers/base/core.c void device_shutdown(void) · line 3832
Quoted 4 lines
drivers/base/core.c +1
-3- pos +=snprintf (hdr + pos, hdrlen - pos,"SUBSYSTEM=%s" , subsys); -if (pos >= hdrlen) - goto overflow; +snprintf (dev_info->subsystem,sizeof (dev_info->subsystem), subsys);It's unlikely that subsys would contain a %, but this will be yet another place to spend brain cycles ignoring if doing static analysis. So can we not do this. Either of
strXcpy()for X=s,l will do the same thing, and likely faster.
Quoted 5 lines
Good point! Better be on the safe size in a generic
printk()API.Well, I am afraid that this would be only small drop in a huge lake.
class->nameandbus->nameseems to be passed to %s in so many *print*() calls all over the kernel code.
So what? printf("%s", some_string_that_might_contain_percent_chars) is not a problem.
printf(some_string_that_might_contain_percent_chars) is.
And yes, one could do
snprintf(dev_info->subsystem, sizeof(dev_info->subsystem), "%s", subsys);
but one might as well avoid the snprintf overhead and use one of the strX functions that have the exact same semantics (copy as much as there's room for, ensure nul-termination).
Rasmus
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Petr Mladek
maintainerRe: [PATCH printk 2/3] printk: move dictionary keys to dev_printk_info
Quoted 22 lines · Rasmus Villemoes
Quoted 21 lines · Petr Mladek
Quoted 19 lines · Rasmus Villemoes
drivers/base/core.c void device_shutdown(void) · line 3832
Quoted 12 lines · John Ogness
drivers/base/core.c +2
-4if (dev->class) subsys = dev->class->name;else if (dev->bus) subsys = dev->bus->name;else -return 0 ; +return ; - pos +=snprintf (hdr + pos, hdrlen - pos,"SUBSYSTEM=%s" , subsys); -if (pos >= hdrlen) - goto overflow; +snprintf (dev_info->subsystem,sizeof (dev_info->subsystem), subsys);It's unlikely that subsys would contain a %, but this will be yet another place to spend brain cycles ignoring if doing static analysis. So can we not do this. Either of
strXcpy()for X=s,l will do the same thing, and likely faster.
Quoted 8 lines
Quoted 5 lines
Good point! Better be on the safe size in a generic
printk()API.Well, I am afraid that this would be only small drop in a huge lake.
class->nameandbus->nameseems to be passed to %s in so many *print*() calls all over the kernel code.So what?
printf("%s", some_string_that_might_contain_percent_chars)is not a problem.
Grr, shame on me. I have completely messed this. The combination of Friday afternoon and noisy kids did not help me much to get it right.
Quoted 1 line
printf(some_string_that_might_contain_percent_chars)is.
I fully agree that passing unknown string as "fmt" is dangerous and must be used carefully. It is not needed here.
Quoted 7 lines
And yes, one could do
snprintf(dev_info->subsystem, sizeof(dev_info->subsystem), "%s", subsys);but one might as well avoid the snprintf overhead and use one of the strX functions that have the exact same semantics (copy as much as there's room for, ensure nul-termination).
Yes, we should use either snprinf() with %s or strXcpy().
Best Regards,
Petr
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk 2/3] printk: move dictionary keys to dev_printk_info
In reply to John OgnessQuoted 8 lines · John Ogness
Dictionaries are only used for SUBSYSTEM and DEVICE properties. The current implementation stores the property names each time they are used. This requires more space than otherwise necessary. Also, because the dictionary entries are currently considered optional, it cannot be relied upon that they are always available, even if the writer wanted to store them. These issues will increase should new dictionary properties be introduced.
Quoted 6 lines
Rather than storing the subsystem and device properties in the dict ring, introduce a struct
dev_printk_infowith separate fields to store only the property values. Embed this struct within the structprintk_infoto provide guaranteed availability.Signed-off-by John Ogness <[email protected]>
Best Regards,
Petr
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
John Ogness
maintainerPATCH 3/3 printk: remove dict ring
In reply to John OgnessSince there is no code that will ever store anything into the dict ring, remove it. If any future dictionary properties are to be added, these should be added to the struct printk_info.
--- kernel/printk/printk.c | 45 +++------ kernel/printk/printk_ringbuffer.c | 155 +++++++----------------------- kernel/printk/printk_ringbuffer.h | 63 +++--------- 3 files changed, 63 insertions(+), 200 deletions(-)
kernel/printk/printk.c +13 -32
@@ -427,7 +427,6 @@ static u32 log_buf_len = __LOG_BUF_LEN;* Define the average message size. This only affects the number of * descriptors that will be available. Underestimating is better than * overestimating (too many available descriptors is better than not enough). -* The dictionary buffer will be the same size as the text buffer. */ #define PRB_AVGBITS 5/* 32 character average length */
@@ -435,7 +434,7 @@ static u32 log_buf_len = __LOG_BUF_LEN;#error CONFIG_LOG_BUF_SHIFT value too small.#endif _DEFINE_PRINTKRB (printk_rb_static,CONFIG_LOG_BUF_SHIFT -PRB_AVGBITS , -PRB_AVGBITS ,PRB_AVGBITS , &__log_buf[0 ]); +PRB_AVGBITS , &__log_buf[0 ]);static struct printk_ringbuffer printk_rb_dynamic;
@@ -502,12 +501,12 @@ static int log_store(u32 caller_id, int facility, int level,struct printk_record r;u16 trunc_msg_len =0 ; -prb_rec_init_wr (&r, text_len,0 ); +prb_rec_init_wr (&r, text_len);if (!prb_reserve (&e, prb, &r)) {/* truncate the message if it is too long for empty buffer */ truncate_msg (&text_len, &trunc_msg_len); -prb_rec_init_wr (&r, text_len + trunc_msg_len,0 ); +prb_rec_init_wr (&r, text_len + trunc_msg_len);/* survive when the log buffer is too small for trunc_msg */ if (!prb_reserve (&e, prb, &r))return 0 ;
@@ -897,8 +896,7 @@ static int devkmsg_open(struct inode *inode, struct file *file)mutex_init (&user->lock);prb_rec_init_rd (&user->record, &user->info, - &user->text_buf[0 ],sizeof (user->text_buf), -NULL ,0 ); + &user->text_buf[0 ],sizeof (user->text_buf));logbuf_lock_irq (); user->seq =prb_first_valid_seq (prb);
@@ -954,7 +952,6 @@ void log_buf_vmcoreinfo_setup(void)VMCOREINFO_STRUCT_SIZE (printk_ringbuffer);VMCOREINFO_OFFSET (printk_ringbuffer, desc_ring);VMCOREINFO_OFFSET (printk_ringbuffer, text_data_ring); -VMCOREINFO_OFFSET (printk_ringbuffer, dict_data_ring);VMCOREINFO_OFFSET (printk_ringbuffer, fail);VMCOREINFO_STRUCT_SIZE (prb_desc_ring);
@@ -967,7 +964,6 @@ void log_buf_vmcoreinfo_setup(void)VMCOREINFO_STRUCT_SIZE (prb_desc);VMCOREINFO_OFFSET (prb_desc, state_var);VMCOREINFO_OFFSET (prb_desc, text_blk_lpos); -VMCOREINFO_OFFSET (prb_desc, dict_blk_lpos);VMCOREINFO_STRUCT_SIZE (prb_data_blk_lpos);VMCOREINFO_OFFSET (prb_data_blk_lpos, begin);
@@ -977,7 +973,6 @@ void log_buf_vmcoreinfo_setup(void)VMCOREINFO_OFFSET (printk_info, seq);VMCOREINFO_OFFSET (printk_info, ts_nsec);VMCOREINFO_OFFSET (printk_info, text_len); -VMCOREINFO_OFFSET (printk_info, dict_len);VMCOREINFO_OFFSET (printk_info, caller_id);VMCOREINFO_OFFSET (printk_info, dev_info);
@@ -1076,7 +1071,7 @@ static unsigned int __init add_to_rb(struct printk_ringbuffer *rb,struct prb_reserved_entry e;struct printk_record dest_r; -prb_rec_init_wr (&dest_r, r->info->text_len,0 ); +prb_rec_init_wr (&dest_r, r->info->text_len);if (!prb_reserve (&e, rb, &dest_r))return 0 ;
@@ -1106,7 +1101,6 @@ void __init setup_log_buf(int early)struct printk_record r;size_t new_descs_size;unsigned long flags; -char *new_dict_buf;char *new_log_buf;unsigned int free;u64 seq;
@@ -1141,20 +1135,11 @@ void __init setup_log_buf(int early)return ; } - new_dict_buf =memblock_alloc (new_log_buf_len,LOG_ALIGN ); -if (unlikely (!new_dict_buf)) { -pr_err ("log_buf_len: %lu dict bytes not available\n" , - new_log_buf_len); -memblock_free (__pa (new_log_buf), new_log_buf_len); -return ; - } - new_descs_size = new_descs_count *sizeof (struct prb_desc ); new_descs =memblock_alloc (new_descs_size,LOG_ALIGN );if (unlikely (!new_descs)) {pr_err ("log_buf_len: %zu desc bytes not available\n" , new_descs_size); -memblock_free (__pa (new_dict_buf), new_log_buf_len);memblock_free (__pa (new_log_buf), new_log_buf_len);return ; }
@@ -1165,18 +1150,14 @@ void __init setup_log_buf(int early)pr_err ("log_buf_len: %zu info bytes not available\n" , new_descs_size);memblock_free (__pa (new_descs), new_log_buf_len); -memblock_free (__pa (new_dict_buf), new_log_buf_len);memblock_free (__pa (new_log_buf), new_log_buf_len);return ; } -prb_rec_init_rd (&r, &info, - &setup_text_buf[0 ],sizeof (setup_text_buf), -NULL ,0 ); +prb_rec_init_rd (&r, &info, &setup_text_buf[0 ],sizeof (setup_text_buf));prb_init (&printk_rb_dynamic, new_log_buf,ilog2 (new_log_buf_len), - new_dict_buf,ilog2 (new_log_buf_len), new_descs,ilog2 (new_descs_count), new_infos);
@@ -1455,7 +1436,7 @@ static int syslog_print(char __user *buf, int size)if (!text)return -ENOMEM ; -prb_rec_init_rd (&r, &info, text,LOG_LINE_MAX +PREFIX_MAX ,NULL ,0 ); +prb_rec_init_rd (&r, &info, text,LOG_LINE_MAX +PREFIX_MAX );while (size >0 ) {size_t n;
@@ -1542,7 +1523,7 @@ static int syslog_print_all(char __user *buf, int size, bool clear) len -=get_record_print_text_size (&info, line_count, true, time); } -prb_rec_init_rd (&r, &info, text,LOG_LINE_MAX +PREFIX_MAX ,NULL ,0 ); +prb_rec_init_rd (&r, &info, text,LOG_LINE_MAX +PREFIX_MAX ); len =0 ;prb_for_each_record (seq, prb, seq, &r) {
@@ -1912,7 +1893,7 @@ static size_t log_output(int facility, int level, enum log_flags lflags,struct prb_reserved_entry e;struct printk_record r; -prb_rec_init_wr (&r, text_len,0 ); +prb_rec_init_wr (&r, text_len);if (prb_reserve_in_last (&e, prb, &r, caller_id)) {memcpy (&r.text_buf[r.info->text_len], text, text_len); r.info->text_len += text_len;
@@ -2400,7 +2381,7 @@ void console_unlock(void)return ; } -prb_rec_init_rd (&r, &info, text,sizeof (text),NULL ,0 ); +prb_rec_init_rd (&r, &info, text,sizeof (text));/* * Console drivers are called with interrupts disabled, so
@@ -3258,7 +3239,7 @@ bool kmsg_dump_get_line_nolock(struct kmsg_dumper *dumper, bool syslog,size_t l =0 ;bool ret = false; -prb_rec_init_rd (&r, &info, line, size,NULL ,0 ); +prb_rec_init_rd (&r, &info, line, size);if (!dumper->active) goto out;
@@ -3349,7 +3330,7 @@ bool kmsg_dump_get_buffer(struct kmsg_dumper *dumper, bool syslog,bool ret = false;bool time = printk_time; -prb_rec_init_rd (&r, &info, buf, size,NULL ,0 ); +prb_rec_init_rd (&r, &info, buf, size);if (!dumper->active || !buf || !size) goto out;
@@ -3397,7 +3378,7 @@ bool kmsg_dump_get_buffer(struct kmsg_dumper *dumper, bool syslog, l +=record_print_text (&r, syslog, time);/* adjust record to store to remaining buffer space */ -prb_rec_init_rd (&r, &info, buf + l, size - l,NULL ,0 ); +prb_rec_init_rd (&r, &info, buf + l, size - l); seq = r.info->seq +1 ; }
kernel/printk/printk_ringbuffer.c +37 -118
365 lines not on this page · raw message
kernel/printk/printk_ringbuffer.h +13 -50
165 lines not on this page · raw message
Signature · 7 lines
2.20.1 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk 3/3] printk: remove dict ring
Quoted 4 lines · John Ogness
Since there is no code that will ever store anything into the dict ring, remove it. If any future dictionary properties are to be added, these should be added to the struct
printk_info.
Ack. I'm very happy to see the dictionary stuff go. It was one of those over-designed things in the printk rewrite years and years ago.
Linus
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH printk 3/3] printk: remove dict ring
In reply to John OgnessQuoted 5 lines · John Ogness
Since there is no code that will ever store anything into the dict ring, remove it. If any future dictionary properties are to be added, these should be added to the struct
printk_info.Signed-off-by John Ogness <[email protected]>
I like the result. It simplified the code and removed some twists.
Best Regards,
Petr
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec