printk: cleanup buffer handling
John Ogness
maintainerv4 cover printk: cleanup buffer handling
Hi,
This is v4 of a series to cleanup console buffer handling and prepare for code sharing with the upcoming threaded/atomic consoles. v3 is here [0].
The main purpose of the series is to introduce two new lockless functions to handle reading and formatting of printk messages. These functions can then be used from any context, which is important for the upcoming threaded/atomic consoles. The series also helps to cleanup some of the internal printk interfaces and cleanly separate formatting code from outputting code.
Some changes in this version were not part of the feedback. However, from the discussions and examples I decided that some of the names I had previously chosen were not appropriate. Particularly, structs, variables, and functions should only use the "console_" prefix if it is console-specific. Things that are used for general printk processing should use a "printk_" prefix. This makes the diff to v3 rather large, even though the "real code" has only minor changes.
@Petr: Like with v3, this version uses a wrapper struct for the message metadata to avoid clobbering. Making the message formatting code robust enough to handle metadata clobbering was too complex (particularly with the dropped tracking).
Changes since v3:
- Provide a detailed explanation in the commit message about why message metadata is put into a wrapper struct instead of the buffer struct.
- Reorder stack variable definitions so that static variables are at the top and separated from non-static variables. IMHO it is important to clearly see which of the variables are static.
- Drop a previous coding-style change from a line not related to this series.
- In console_prepend_dropped() make sure the buffer is large enough for the dropped message and at least PREFIX_MAX of the current message.
- Define the PREFIX_MAX macro for !CONFIG_PRINTK in internal.h because console_prepend_dropped() now needs it.
- Rename various functions, structs, fields, and macros:
console_get_next_message() -> printk_get_next_message()
struct console_buffers -> struct printk_buffers
struct console_message -> struct printk_message
console_message.cbufs -> printk_message.pbufs
console_message.outbuf_seq -> printk_message.seq
LOG_LINE_MAX -> PRINTKRB_RECORD_MAX
PREFIX_MAX -> PRINTK_PREFIX_MAX
CONSOLE_LOG_MAX and
CONSOLE_EXT_LOG_MAX -> PRINTK_MESSAGE_MAX- Adjust the values of string limit macros. This is explained in detail in the commit message.
- Replace @buf and @text_buf in struct devkmsg_user with struct printk_buffers.
- Replace message formatting code in devkmsg_read() with printk_get_next_message().
- Define all printk_message structs on the stack.
John Ogness
[0] https://lore.kernel.org/lkml/[email protected]
John Ogness (6):
printk: move size limit macros into internal.h
printk: introduce struct printk_buffers
printk: introduce printk_get_next_message() and printk_message
printk: introduce console_prepend_dropped() for dropped messages
printk: use printk_buffers for devkmsg
printk: adjust string limit macros
Thomas Gleixner (2):
console: Use BIT() macros for @flags values
console: Document struct console
include/linux/console.h | 100 +++++++++---- include/linux/printk.h | 2 - kernel/printk/internal.h | 45 ++++++ kernel/printk/printk.c | 293 +++++++++++++++++++++++---------------- 4 files changed, 288 insertions(+), 152 deletions(-)
base-commit: 6b2b0d839acaa84f05a77184370f793752e786e9
Signature · 1 line
2.30.2
John Ogness
maintainerPATCH v4 1/8 printk: move size limit macros into internal.h
The size limit macros are located further down in printk.c and behind ifdef conditionals. This complicates their usage for upcoming changes. Move the macros into internal.h so that they are still invisible outside of printk, but easily accessible for printk.
Also, the maximum size of formatted extended messages does not need to be known by any code outside of printk, so move it to internal.h as well. And like CONSOLE_LOG_MAX, for !CONFIG_PRINTK set CONSOLE_EXT_LOG_MAX to 0 to reduce the static memory footprint.
--- include/linux/printk.h | 2 -- kernel/printk/internal.h | 24 ++++++++++++++++++++++++ kernel/printk/printk.c | 17 ----------------- 3 files changed, 24 insertions(+), 19 deletions(-)
include/linux/printk.h +0 -2
@@ -44,8 +44,6 @@ static inline const char *printk_skip_headers(const char *buffer)return buffer; } -#define CONSOLE_EXT_LOG_MAX 8192 -/* printk's without a loglevel use this.. */ #define MESSAGE_LOGLEVEL_DEFAULT CONFIG_MESSAGE_LOGLEVEL_DEFAULT
kernel/printk/internal.h +24 -0
@@ -14,6 +14,24 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write,#ifdef CONFIG_PRINTK +#ifdef CONFIG_PRINTK_CALLER +#define PREFIX_MAX 48 +#else +#define PREFIX_MAX 32 +#endif + +/* the maximum size of a formatted record (i.e. with prefix added per line) */ +#define CONSOLE_LOG_MAX 1024 + +/* the maximum size of a formatted extended record */ +#define CONSOLE_EXT_LOG_MAX 8192 + +/* the maximum size for a dropped text message */ +#define DROPPED_TEXT_MAX 64 + +/* the maximum size allowed to be reserved for a record */ +#define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX) +/* Flags for a single printk record. */ enum printk_info_flags { LOG_NEWLINE = 2, /* text ended with a newline */
@@ -48,6 +66,12 @@ u16 printk_parse_prefix(const char *text, int *level, enum printk_info_flags *flags);#else +#define PREFIX_MAX 0 +#define CONSOLE_LOG_MAX 0 +#define CONSOLE_EXT_LOG_MAX 0 +#define DROPPED_TEXT_MAX 0 +#define LOG_LINE_MAX 0 +/* * In !PRINTK builds we still export console_sem * semaphore and some of console functions (console_unlock()/etc.), so
kernel/printk/printk.c +0 -17
@@ -465,21 +465,6 @@ static struct latched_seq clear_seq = { .val[1] = 0, }; -#ifdef CONFIG_PRINTK_CALLER -#define PREFIX_MAX 48 -#else -#define PREFIX_MAX 32 -#endif - -/* the maximum size of a formatted record (i.e. with prefix added per line) */ -#define CONSOLE_LOG_MAX 1024 - -/* the maximum size for a dropped text message */ -#define DROPPED_TEXT_MAX 64 - -/* the maximum size allowed to be reserved for a record */ -#define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX) -#define LOG_LEVEL (v) ((v) & 0x07)#define LOG_FACILITY (v) ((v) >> 3 & 0xff)
@@ -2387,8 +2372,6 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre#else /* CONFIG_PRINTK */ -#define CONSOLE_LOG_MAX 0 -#define DROPPED_TEXT_MAX 0#define printk_time false#define prb_read_valid (rb, seq, r) false
Signature · 1 line
2.30.2
Re: [PATCH printk v4 1/8] printk: move size limit macros into internal.h
Quoted 6 lines · John Ogness
The size limit macros are located further down in
printk.cand behind ifdef conditionals. This complicates their usage for upcoming changes. Move the macros intointernal.hso that they are still invisible outside of printk, but easily accessible for printk.
Quoted 7 lines
Also, the maximum size of formatted extended messages does not need to be known by any code outside of printk, so move it to
internal.has well. And likeCONSOLE_LOG_MAX, for !CONFIG_PRINTK setCONSOLE_EXT_LOG_MAXto 0 to reduce the static memory footprint.Signed-off-by John Ogness <[email protected]>
Best Regards,
Petr
John Ogness
maintainerPATCH v4 2/8 console: Use BIT() macros for @flags values
In reply to John OgnessFrom: Thomas Gleixner <[email protected]>
Rather than manually calculating powers of 2, use the BIT() macros. Also take this opportunatity to cleanup and restructure the value comments into proper kerneldoc comments.
--- include/linux/console.h | 46 ++++++++++++++++++++++++++++++++--------- 1 file changed, 36 insertions(+), 10 deletions(-)
include/linux/console.h +36 -10
@@ -15,6 +15,7 @@#define _LINUX_CONSOLE_H_ 1#include <linux/atomic.h> +#include <linux/bits.h> #include <linux/rculist.h> #include <linux/types.h>
@@ -125,18 +126,43 @@ static inline int con_debug_leave(void)/* * The interface for a console, or any other device that wants to capture * console messages (printer driver?) -* -* If a console driver is marked CON_BOOT then it will be auto-unregistered -* when the first real console is registered. This is for early-printk drivers. */ -#define CON_PRINTBUFFER (1) -#define CON_CONSDEV (2)/* Preferred console, /dev/console */ -#define CON_ENABLED (4) -#define CON_BOOT (8) -#define CON_ANYTIME (16)/* Safe to call when cpu is offline */ -#define CON_BRL (32)/* Used for a braille device */ -#define CON_EXTENDED (64)/* Use the extended output format a la /dev/kmsg */ +/** +* cons_flags - General console flags +* @CON_PRINTBUFFER: Used by newly registered consoles to avoid duplicate +* output of messages that were already shown by boot +* consoles or read by userspace via syslog() syscall. +* @CON_CONSDEV: Indicates that the console driver is backing +* /dev/console. +* @CON_ENABLED: Indicates if a console is allowed to print records. If +* false, the console also will not advance to later +* records. +* @CON_BOOT: Marks the console driver as early console driver which +* is used during boot before the real driver becomes +* available. It will be automatically unregistered +* when the real console driver is registered unless +* "keep_bootcon" parameter is used. +* @CON_ANYTIME: A misnomed historical flag which tells the core code +* that the legacy @console::write callback can be invoked +* on a CPU which is marked OFFLINE. That is misleading as +* it suggests that there is no contextual limit for +* invoking the callback. The original motivation was +* readiness of the per-CPU areas. +* @CON_BRL: Indicates a braille device which is exempt from +* receiving the printk spam for obvious reasons. +* @CON_EXTENDED: The console supports the extended output format of +* /dev/kmesg which requires a larger output buffer. +*/ +enum cons_flags { +CON_PRINTBUFFER =BIT (0 ), +CON_CONSDEV =BIT (1 ), +CON_ENABLED =BIT (2 ), +CON_BOOT =BIT (3 ), +CON_ANYTIME =BIT (4 ), +CON_BRL =BIT (5 ), +CON_EXTENDED =BIT (6 ), +};struct console {char name[16 ];
Signature · 1 line
2.30.2
John Ogness
maintainerPATCH v4 3/8 console: Document struct console
In reply to John OgnessFrom: Thomas Gleixner <[email protected]>
Add kerneldoc comments to struct console.
--- include/linux/console.h | 54 ++++++++++++++++++++++++++++------------- 1 file changed, 37 insertions(+), 17 deletions(-)
include/linux/console.h +37 -17
@@ -164,24 +164,44 @@ enum cons_flags { CON_EXTENDED = BIT(6), }; +/** +* struct console - The console descriptor structure +* @name: The name of the console driver +* @write: Write callback to output messages (Optional) +* @read: Read callback for console input (Optional) +* @device: The underlying TTY device driver (Optional) +* @unblank: Callback to unblank the console (Optional) +* @setup: Callback for initializing the console (Optional) +* @exit: Callback for teardown of the console (Optional) +* @match: Callback for matching a console (Optional) +* @flags: Console flags. See enum cons_flags +* @index: Console index, e.g. port number +* @cflag: TTY control mode flags +* @ispeed: TTY input speed +* @ospeed: TTY output speed +* @seq: Sequence number of the next ringbuffer record to print +* @dropped: Number of unreported dropped ringbuffer records +* @data: Driver private data +* @node: hlist node for the console list +*/ struct console { -char name[16 ]; -void (*write)(struct console *,const char *,unsigned ); -int (*read)(struct console *,char *,unsigned ); -struct tty_driver *(*device)(struct console *,int *); -void (*unblank)(void ); -int (*setup)(struct console *,char *); -int (*exit)(struct console *); -int (*match)(struct console *,char *name,int idx,char *options); -short flags; -short index; -int cflag; -uint ispeed; -uint ospeed; -u64 seq; -unsigned long dropped; -void *data; -struct hlist_node node; +char name[16 ]; +void (*write)(struct console *co,const char *s,unsigned int count); +int (*read)(struct console *co,char *s,unsigned int count); +struct tty_driver *(*device)(struct console *co,int *index); +void (*unblank)(void ); +int (*setup)(struct console *co,char *options); +int (*exit)(struct console *co); +int (*match)(struct console *co,char *name,int idx,char *options); +short flags; +short index; +int cflag; +uint ispeed; +uint ospeed; +u64 seq; +unsigned long dropped; +void *data; +struct hlist_node node; };#ifdef CONFIG_LOCKDEP
Signature · 1 line
2.30.2
John Ogness
maintainerPATCH v4 4/8 printk: introduce struct printk_buffers
In reply to John OgnessIntroduce a new struct printk_buffers to contain all the buffers needed to read and format a printk message for output. Putting the buffers inside a struct reduces the number of buffer pointers that need to be tracked. Also, it allows usage of the sizeof() macro for the buffer sizes, rather than expecting certain sized buffers being passed in.
Note that since the output buffer for normal consoles is now CONSOLE_EXT_LOG_MAX instead of CONSOLE_LOG_MAX, multi-line messages that may have been previously truncated will now be printed in full. This should be considered a feature and not a bug since the CONSOLE_LOG_MAX restriction was about limiting static buffer usage rather than limiting printed text.
--- kernel/printk/internal.h | 10 +++++++ kernel/printk/printk.c | 57 ++++++++++++++++++---------------------- 2 files changed, 36 insertions(+), 31 deletions(-)
kernel/printk/internal.h +10 -0
@@ -82,3 +82,13 @@ u16 printk_parse_prefix(const char *text, int *level,static inline bool printk_percpu_data_ready (void ) {return false; }#endif /* CONFIG_PRINTK */ + +/** +* struct printk_buffers - Buffers to read/format/output printk messages. +* @outbuf: After formatting, contains text to output. +* @scratchbuf: Used as temporary ringbuffer reading and string-print space. +*/ +struct printk_buffers { +char outbuf[CONSOLE_EXT_LOG_MAX ]; +char scratchbuf[LOG_LINE_MAX ]; +};
kernel/printk/printk.c +26 -31
102 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH printk v4 4/8] printk: introduce struct printk_buffers
Quoted 7 lines · John Ogness
Introduce a new struct
printk_buffersto contain all the buffers needed to read and format a printk message for output. Putting the buffers inside a struct reduces the number of buffer pointers that need to be tracked. Also, it allows usage of thesizeof()macro for the buffer sizes, rather than expecting certain sized buffers being passed in.
Quoted 8 lines
Note that since the output buffer for normal consoles is now
CONSOLE_EXT_LOG_MAXinstead ofCONSOLE_LOG_MAX, multi-line messages that may have been previously truncated will now be printed in full. This should be considered a feature and not a bug since theCONSOLE_LOG_MAXrestriction was about limiting static buffer usage rather than limiting printed text.Signed-off-by John Ogness <[email protected]>
Best Regards,
Petr
John Ogness
maintainerPATCH v4 5/8 printk: introduce printk_get_next_message() and printk_message
In reply to John OgnessCode for performing the console output is intermixed with code that is formatting the output for that console. Introduce a new helper function printk_get_next_message() to handle the reading and formatting of the printk text. The helper does not require any locking so that in the future it can be used for other printing contexts as well.
This also introduces a new struct printk_message to wrap the struct printk_buffers, adding metadata about its contents. This allows users of printk_get_next_message() to receive all relevant information about the message that was read and formatted.
Why is struct printk_message a wrapper struct?
It is intentional that a wrapper struct is introduced instead of adding the metadata directly to struct printk_buffers. The upcoming atomic consoles support multiple printing contexts per CPU. This means that while a CPU is formatting a message, it can be interrupted and the interrupting context may also format a (possibly different) message. Since the printk buffers are rather large, there will only be one struct printk_buffers per CPU and it must be shared by the possible contexts of that CPU.
If the metadata was part of struct printk_buffers, interrupting contexts would clobber the metadata being prepared by the interrupted context. This could be handled by robustifying the message formatting functions to cope with metadata unexpectedly changing. However, this would require significant amounts of extra data copying, also adding significant complexity to the code.
Instead, the metadata can live on the stack of the formatting context and the message formatting functions do not need to be concerned about the metadata changing underneath them.
Note that the message formatting functions can handle unexpected text buffer changes. So it is perfectly OK if a shared text buffer is clobbered by an interrupting context. The atomic console implementation will recognize the interruption and avoid printing the (probably garbage) text buffer.
--- kernel/printk/internal.h | 16 ++++++ kernel/printk/printk.c | 115 +++++++++++++++++++++++++++------------ 2 files changed, 96 insertions(+), 35 deletions(-)
kernel/printk/internal.h +16 -0
@@ -92,3 +92,19 @@ struct printk_buffers {char outbuf[CONSOLE_EXT_LOG_MAX ];char scratchbuf[LOG_LINE_MAX ]; }; + +/** +* struct printk_message - Container for a prepared printk message. +* @pbufs: printk buffers used to prepare the message. +* @outbuf_len: The length of prepared text in @pbufs->outbuf to output. This +* does not count the terminator. A value of 0 means there is +* nothing to output and this record should be skipped. +* @seq: The sequence number of the record used for @pbufs->outbuf. +* @dropped: The number of dropped records from reading @seq. +*/ +struct printk_message { +struct printk_buffers *pbufs; +unsigned int outbuf_len; +u64 seq; +unsigned long dropped; +};
kernel/printk/printk.c +80 -35
157 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH printk v4 5/8] printk: introduce printk_get_next_message() and printk_message
Quoted 34 lines · John Ogness
Code for performing the console output is intermixed with code that is formatting the output for that console. Introduce a new helper function
printk_get_next_message()to handle the reading and formatting of the printk text. The helper does not require any locking so that in the future it can be used for other printing contexts as well.This also introduces a new struct
printk_messageto wrap the structprintk_buffers, adding metadata about its contents. This allows users ofprintk_get_next_message()to receive all relevant information about the message that was read and formatted.Why is struct
printk_messagea wrapper struct?It is intentional that a wrapper struct is introduced instead of adding the metadata directly to struct
printk_buffers. The upcoming atomic consoles support multiple printing contexts per CPU. This means that while a CPU is formatting a message, it can be interrupted and the interrupting context may also format a (possibly different) message. Since the printk buffers are rather large, there will only be one structprintk_buffersper CPU and it must be shared by the possible contexts of that CPU.If the metadata was part of struct
printk_buffers, interrupting contexts would clobber the metadata being prepared by the interrupted context. This could be handled by robustifying the message formatting functions to cope with metadata unexpectedly changing. However, this would require significant amounts of extra data copying, also adding significant complexity to the code.Instead, the metadata can live on the stack of the formatting context and the message formatting functions do not need to be concerned about the metadata changing underneath them.
Quoted 5 lines
Note that the message formatting functions can handle unexpected text buffer changes. So it is perfectly OK if a shared text buffer is clobbered by an interrupting context. The atomic console implementation will recognize the interruption and avoid printing the (probably garbage) text buffer.
Great description!
Quoted 1 line
Signed-off-by John Ogness <[email protected]>
Best Regards,
Petr
John Ogness
maintainerPATCH v4 6/8 printk: introduce console_prepend_dropped() for dropped messages
In reply to John OgnessCurrently "dropped messages" are separately printed immediately before printing the printk message. Since normal consoles are now using an output buffer that is much larger than previously, the "dropped message" could be prepended to the printk message and then output everything in a single write() call.
Introduce a helper function console_prepend_dropped() to prepend an existing message with a "dropped message". This simplifies the code by allowing all message formatting to be handled together and then only requires a single write() call to output the full message. And since this helper does not require any locking, it can be used in the future for other console printing contexts as well.
--- kernel/printk/internal.h | 4 -- kernel/printk/printk.c | 86 ++++++++++++++++++++++++++-------------- 2 files changed, 57 insertions(+), 33 deletions(-)
kernel/printk/internal.h +0 -4
@@ -26,9 +26,6 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write,/* the maximum size of a formatted extended record */ #define CONSOLE_EXT_LOG_MAX 8192 -/* the maximum size for a dropped text message */ -#define DROPPED_TEXT_MAX 64 -/* the maximum size allowed to be reserved for a record */ #define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX)
@@ -69,7 +66,6 @@ u16 printk_parse_prefix(const char *text, int *level,#define PREFIX_MAX 0#define CONSOLE_LOG_MAX 0#define CONSOLE_EXT_LOG_MAX 0 -#define DROPPED_TEXT_MAX 0#define LOG_LINE_MAX 0/*
kernel/printk/printk.c +57 -29
122 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH printk v4 6/8] printk: introduce console_prepend_dropped() for dropped messages
Quoted 14 lines · John Ogness
Currently "dropped messages" are separately printed immediately before printing the printk message. Since normal consoles are now using an output buffer that is much larger than previously, the "dropped message" could be prepended to the printk message and then output everything in a single
write()call.Introduce a helper function
console_prepend_dropped()to prepend an existing message with a "dropped message". This simplifies the code by allowing all message formatting to be handled together and then only requires a singlewrite()call to output the full message. And since this helper does not require any locking, it can be used in the future for other console printing contexts as well.
Quoted 1 line
Signed-off-by John Ogness <[email protected]>
A comment below.
Quoted 30 lines
kernel/printk/printk.c +24
-0@@ -2724,6 +2699,52 @@ static void __console_unlock(void)up_console_sem (); } +/* +* Prepend the message in @pmsg->pbufs->outbuf with a "dropped message". This +* is achieved by shifting the existing message over and inserting the dropped +* message. +* +* @pmsg is the printk message to prepend. +* +* @dropped is the dropped count to report in the dropped message. +* +* If the message text in @pmsg->pbufs->outbuf does not have enough space for +* the dropped message, the message text will be sufficiently truncated. +* +* If @pmsg->pbufs->outbuf is modified, @pmsg->outbuf_len is updated. +*/ +static void console_prepend_dropped (struct printk_message *pmsg,unsigned long dropped) +{ +struct printk_buffers *pbufs = pmsg->pbufs; +const size_t scratchbuf_sz =sizeof (pbufs->scratchbuf); +const size_t outbuf_sz =sizeof (pbufs->outbuf); +char *scratchbuf = &pbufs->scratchbuf[0 ]; +char *outbuf = &pbufs->outbuf[0 ]; +size_t len; + + len = snprintf(scratchbuf, scratchbuf_sz,
kernel/printk/printk.c static void __console_unlock(void) · line 2735
Quoted 10 lines
kernel/printk/printk.c +10
-0+ "** %lu printk messages dropped **\n", dropped); + +/* +* Make sure outbuf is sufficiently large before prepending. +* Keep at least the prefix when the message must be truncated. +* It is a rather theoretical problem when someone tries to +* use a minimalist buffer. +*/ +if (WARN_ON_ONCE (len +PREFIX_MAX >= outbuf_sz)) +return ;
I guess that this will always trigger the compiler warning when CONFIG_PRINTK is disabled. See the report for v3 at
https://lore.kernel.org/r/[email protected]
Hmm, we might want to fix this warning so that it does not break build with -Werror.
IMHO, the proper solution would be to define this function only when CONFIG_PRINTK is defined. But it might require bigger changes and define many more console functions only when CONFIG_PRINTK is defined. This is out-of-scope of this patchset.
I wonder if the following would work as an "intermediate" workaround:
if (!IS_ENABLED(CONFIG_PRINTK) || WARN_ON_ONCE(len + PREFIX_MAX >= outbuf_sz)) return;
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v4 6/8] printk: introduce console_prepend_dropped() for dropped messages
Quoted 6 lines · Petr Mladek
kernel/printk/printk.c static void __console_unlock(void) · line 2735
Quoted 2 lines
kernel/printk/printk.c +2
-0+if (WARN_ON_ONCE (len +PREFIX_MAX >= outbuf_sz)) +return ;I guess that this will always trigger the compiler warning when
CONFIG_PRINTKis disabled. See the report for v3 at
https://lore.kernel.org/r/[email protected]
That report is actually complaining about the check after this one. It is the same "problem".
Quoted 8 lines
Hmm, we might want to fix this warning so that it does not break build with -Werror.
IMHO, the proper solution would be to define this function only when
CONFIG_PRINTKis defined. But it might require bigger changes and define many more console functions only whenCONFIG_PRINTKis defined. This is out-of-scope of this patchset.
Quoted 5 lines
I wonder if the following would work as an "intermediate" workaround:
if (!IS_ENABLED(CONFIG_PRINTK) || WARN_ON_ONCE(len + PREFIX_MAX >= outbuf_sz)) return;
The whole CONFIG_PRINTK stuff is a total mess right now. We should definitely get that cleaned up at some point. As an intermediate workaround, it might make more sense to just put the whole function inside an "#ifdef CONFIG_PRINTK". It doesn't return anything anyway.
#ifdef CONFIG_PRINTK
static void console_prepend_dropped(struct printk_message *pmsg, unsigned long dropped) {
...
}
#else
#define console_prepend_dropped(pmsg, dropped)
#endif
There are already places in the code that look like this (for example, print_caller()).
Otherwise, if you want to use IS_ENABLED(), you will need it for both checks.
John
Petr Mladek
maintainerRe: [PATCH printk v4 6/8] printk: introduce console_prepend_dropped() for dropped messages
Quoted 10 lines · John Ogness
Quoted 6 lines · Petr Mladek
kernel/printk/printk.c static void __console_unlock(void) · line 2735
Quoted 2 lines
kernel/printk/printk.c +2
-0+if (WARN_ON_ONCE (len +PREFIX_MAX >= outbuf_sz)) +return ;I guess that this will always trigger the compiler warning when
CONFIG_PRINTKis disabled. See the report for v3 at
https://lore.kernel.org/r/[email protected]That report is actually complaining about the check after this one. It is the same "problem".
I see.
Quoted 19 lines
Quoted 8 lines
Hmm, we might want to fix this warning so that it does not break build with -Werror.
IMHO, the proper solution would be to define this function only when
CONFIG_PRINTKis defined. But it might require bigger changes and define many more console functions only whenCONFIG_PRINTKis defined. This is out-of-scope of this patchset.Quoted 5 lines
I wonder if the following would work as an "intermediate" workaround:
if (!IS_ENABLED(CONFIG_PRINTK) || WARN_ON_ONCE(len + PREFIX_MAX >= outbuf_sz)) return;The whole
CONFIG_PRINTKstuff is a total mess right now. We should definitely get that cleaned up at some point. As an intermediate workaround, it might make more sense to just put the whole function inside an "#ifdefCONFIG_PRINTK". It doesn't return anything anyway.
Quoted 8 lines
#ifdef
CONFIG_PRINTK
static voidconsole_prepend_dropped(struct printk_message *pmsg, unsigned long dropped){
...
}
#else
#defineconsole_prepend_dropped(pmsg, dropped)
#endif
Looks good.
The question is if we want to do it in this patch or as a follow up fix.
I slightly prefer to do it as a follow up fix to avoid another respin. Also we will not need to complicate this patch with explaining why this is needed.
Best Regards,
Petr
John Ogness
maintainerPATCH v4 7/8 printk: use printk_buffers for devkmsg
In reply to John OgnessReplace the individual @text_buf and @buf buffers in struct devkmsg_user with a struct printk_buffers. This reduces the number of buffers to keep track of. Also, since @text_buf is only used for ringbuffer reading and @buf is only used for output, it maps perfectly to the purpose and fields of struct printk_buffers.
As a side-effect, @text_buf was 8kB large, even though it only needed to be the max size of a ringbuffer record. By switching to struct printk_buffers, ~7kB less memory is allocated when opening /dev/kmsg.
And since struct printk_buffers will be used now, reduce duplicate code by calling printk_get_next_message() to handle the record reading and formatting.
--- kernel/printk/printk.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-)
kernel/printk/printk.c +14 -18
@@ -695,15 +695,16 @@ static ssize_t msg_print_ext_body(char *buf, size_t size,return len; } +static bool printk_get_next_message (struct printk_message *pmsg,u64 seq,bool is_extended); +/* /dev/kmsg - userspace message inject/listen interface */ struct devkmsg_user {atomic64_t seq;struct ratelimit_state rs;struct mutex lock; -char buf[CONSOLE_EXT_LOG_MAX ]; +struct printk_buffers pbufs;struct printk_info info; -char text_buf[CONSOLE_EXT_LOG_MAX ];struct printk_record record; };
@@ -787,7 +788,10 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf, {struct devkmsg_user *user = file->private_data;struct printk_record *r = &user->record; -size_t len; +char *outbuf = &user->pbufs.outbuf[0 ]; +struct printk_message pmsg = { + .pbufs = &user->pbufs, + };ssize_t ret;if (!user)
@@ -797,7 +801,7 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,if (ret)return ret; -if (!prb_read_valid (prb,atomic64_read (&user->seq), r)) { +if (!printk_get_next_message (&pmsg,atomic64_read (&user->seq), true)) {if (file->f_flags &O_NONBLOCK ) { ret = -EAGAIN ; goto out;
@@ -820,30 +824,25 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf, goto out; } -if (r->info->seq !=atomic64_read (&user->seq)) { +if (pmsg.dropped) {/* our last seen message is gone, return error and reset */ -atomic64_set (&user->seq, r->info->seq); +atomic64_set (&user->seq, pmsg.seq); ret = -EPIPE ; goto out; } - len =info_print_ext_header (user->buf,sizeof (user->buf), r->info); - len +=msg_print_ext_body (user->buf + len,sizeof (user->buf) - len, - &r->text_buf[0 ], r->info->text_len, - &r->info->dev_info); - -atomic64_set (&user->seq, r->info->seq +1 ); +atomic64_set (&user->seq, pmsg.seq +1 ); -if (len > count) { +if (pmsg.outbuf_len > count) { ret = -EINVAL ; goto out; } -if (copy_to_user (buf, user->buf, len)) { +if (copy_to_user (buf, outbuf, pmsg.outbuf_len)) { ret = -EFAULT ; goto out; } - ret = len; + ret = pmsg.outbuf_len; out:mutex_unlock (&user->lock);return ret;
@@ -937,9 +936,6 @@ static int devkmsg_open(struct inode *inode, struct file *file)mutex_init (&user->lock); -prb_rec_init_rd (&user->record, &user->info, - &user->text_buf[0 ],sizeof (user->text_buf)); -atomic64_set (&user->seq,prb_first_valid_seq (prb)); file->private_data = user;
Signature · 1 line
2.30.2
John Ogness
maintainerRe: [PATCH printk v4 7/8] printk: use printk_buffers for devkmsg
kernel/printk/printk.c static ssize_t msg_print_ext_body(char *buf, size_t size, · line 709
Quoted 12 lines · John Ogness
/*
/dev/kmsg- userspace message inject/listen interface */ structdevkmsg_user{kernel/printk/printk.c +1
-2atomic64_t seq;struct ratelimit_state rs;struct mutex lock; -char buf[CONSOLE_EXT_LOG_MAX ]; +struct printk_buffers pbufs;struct printk_info info; -char text_buf[CONSOLE_EXT_LOG_MAX ];struct printk_record record; };
There is also no need for the printk_info and printk_record structs. printk_get_next_message() uses its own on the stack.
kernel/printk/printk.c static ssize_t devkmsg_read(struct file *file, char __user *buf, · line 804
Quoted 6 lines
kernel/printk/printk.c +1
-1@@ -797,7 +801,7 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,if (ret)return ret; -if (!prb_read_valid (prb,atomic64_read (&user->seq), r)) { +if (!printk_get_next_message (&pmsg,atomic64_read (&user->seq), true)) {
This needs to be a while loop because printk_get_next_message() should be the only function reading from the ringbuffer. The wait_event_interruptible() in this loop should just wait for the existence of the new record. It should not be the one reading it.
I will reply to this message with the correct patch to replace this patch.
John Ogness
John Ogness
maintainerPATCH v2 7/8 printk: use printk_buffers for devkmsg
Replace the buffers in struct devkmsg_user with a struct printk_buffers. This reduces the number of buffers to track.
As a side-effect, @text_buf was 8kB large, even though it only needed to be the max size of a ringbuffer record. By switching to struct printk_buffers, ~7kB less memory is allocated when opening /dev/kmsg.
And since struct printk_buffers will be used now, reduce duplicate code by calling printk_get_next_message() to handle the record reading and formatting.
--- kernel/printk/printk.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-)
kernel/printk/printk.c +16 -24
@@ -695,16 +695,14 @@ static ssize_t msg_print_ext_body(char *buf, size_t size,return len; } +static bool printk_get_next_message (struct printk_message *pmsg,u64 seq,bool is_extended); +/* /dev/kmsg - userspace message inject/listen interface */ struct devkmsg_user {atomic64_t seq;struct ratelimit_state rs;struct mutex lock; -char buf[CONSOLE_EXT_LOG_MAX ]; - -struct printk_info info; -char text_buf[CONSOLE_EXT_LOG_MAX ]; -struct printk_record record; +struct printk_buffers pbufs; }; static __printf(3, 4) __cold
@@ -786,8 +784,10 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { struct devkmsg_user *user = file->private_data; -struct printk_record *r = &user->record; -size_t len; +char *outbuf = &user->pbufs.outbuf[0 ]; +struct printk_message pmsg = { + .pbufs = &user->pbufs, + };ssize_t ret;if (!user)
@@ -797,7 +797,7 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,if (ret)return ret; -if (!prb_read_valid (prb,atomic64_read (&user->seq), r)) { +while (!printk_get_next_message (&pmsg,atomic64_read (&user->seq), true)) {if (file->f_flags &O_NONBLOCK ) { ret = -EAGAIN ; goto out;
@@ -814,36 +814,31 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,* This pairs with __wake_up_klogd:A. */ ret =wait_event_interruptible (log_wait, -prb_read_valid (prb, -atomic64_read (&user->seq), r));/* LMM(devkmsg_read:A) */ +prb_read_valid (prb,atomic64_read (&user->seq), +NULL ));/* LMM(devkmsg_read:A) */ if (ret) goto out; } -if (r->info->seq !=atomic64_read (&user->seq)) { +if (pmsg.dropped) {/* our last seen message is gone, return error and reset */ -atomic64_set (&user->seq, r->info->seq); +atomic64_set (&user->seq, pmsg.seq); ret = -EPIPE ; goto out; } - len =info_print_ext_header (user->buf,sizeof (user->buf), r->info); - len +=msg_print_ext_body (user->buf + len,sizeof (user->buf) - len, - &r->text_buf[0 ], r->info->text_len, - &r->info->dev_info); +atomic64_set (&user->seq, pmsg.seq +1 ); -atomic64_set (&user->seq, r->info->seq +1 ); - -if (len > count) { +if (pmsg.outbuf_len > count) { ret = -EINVAL ; goto out; } -if (copy_to_user (buf, user->buf, len)) { +if (copy_to_user (buf, outbuf, pmsg.outbuf_len)) { ret = -EFAULT ; goto out; } - ret = len; + ret = pmsg.outbuf_len; out:mutex_unlock (&user->lock);return ret;
@@ -937,9 +932,6 @@ static int devkmsg_open(struct inode *inode, struct file *file)mutex_init (&user->lock); -prb_rec_init_rd (&user->record, &user->info, - &user->text_buf[0 ],sizeof (user->text_buf)); -atomic64_set (&user->seq,prb_first_valid_seq (prb)); file->private_data = user;
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v4 7/8 v2] printk: use printk_buffers for devkmsg
Quoted 55 lines · John Ogness
Replace the buffers in struct
devkmsg_userwith a structprintk_buffers. This reduces the number of buffers to track.As a side-effect, @text_buf was 8kB large, even though it only needed to be the max size of a ringbuffer record. By switching to struct
printk_buffers, ~7kB less memory is allocated when opening/dev/kmsg.And since struct
printk_bufferswill be used now, reduce duplicate code by callingprintk_get_next_message()to handle the record reading and formatting.Signed-off-by John Ogness <[email protected]>--- kernel/printk/printk.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-)kernel/printk/printk.c +7
-7@@ -695,16 +695,14 @@ static ssize_t msg_print_ext_body(char *buf, size_t size,return len; } +static bool printk_get_next_message (struct printk_message *pmsg,u64 seq,bool is_extended); +/* /dev/kmsg - userspace message inject/listen interface */ struct devkmsg_user {atomic64_t seq;struct ratelimit_state rs;struct mutex lock; -char buf[CONSOLE_EXT_LOG_MAX ]; - -struct printk_info info; -char text_buf[CONSOLE_EXT_LOG_MAX ]; -struct printk_record record; +struct printk_buffers pbufs; }; static __printf(3, 4) __cold@@ -786,8 +784,10 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { struct devkmsg_user *user = file->private_data; -struct printk_record *r = &user->record; -size_t len; +char *outbuf = &user->pbufs.outbuf[0 ]; +struct printk_message pmsg = { + .pbufs = &user->pbufs, + };ssize_t ret;if (!user)
kernel/printk/printk.c static ssize_t devkmsg_read(struct file *file, char __user *buf, · line 800
Quoted 5 lines
kernel/printk/printk.c +1
-1@@ -797,7 +797,7 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,if (ret)return ret; -if (!prb_read_valid (prb,atomic64_read (&user->seq), r)) { +while (!printk_get_next_message (&pmsg,atomic64_read (&user->seq), true)) {
A problem is that printk_get_next_message() does not format the message when it shoud get supressed on the console.
I would solve it be adding a parameter to printk_get_next_message() that will tell whether to suppress or not, e.g. @can_suppress.
kernel/printk/printk.c static ssize_t devkmsg_read(struct file *file, char __user *buf, · line 818
Quoted 11 lines
kernel/printk/printk.c +2
-2if (file->f_flags &O_NONBLOCK ) { ret = -EAGAIN ; goto out;@@ -814,36 +814,31 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,* This pairs with __wake_up_klogd:A. */ ret =wait_event_interruptible (log_wait, -prb_read_valid (prb, -atomic64_read (&user->seq), r));/* LMM(devkmsg_read:A) */ +prb_read_valid (prb,atomic64_read (&user->seq), +NULL ));/* LMM(devkmsg_read:A) */
The above change from "if" to "while" could be avoided if we use printk_get_next_message() here as well. It looks slightly more strightfoward to me. Or do I miss something, please?
kernel/printk/printk.c static ssize_t devkmsg_read(struct file *file, char __user *buf, · line 822
Quoted 4 lines
kernel/printk/printk.c +0
-0if (ret) goto out; }
Otherwise, I like this change. It simplifies the code.
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v4 7/8 v2] printk: use printk_buffers for devkmsg
Quoted 5 lines · Petr Mladek
kernel/printk/printk.c static ssize_t devkmsg_read(struct file *file, char __user *buf, · line 800
Quoted 2 lines
kernel/printk/printk.c +1
-1-if (!prb_read_valid (prb,atomic64_read (&user->seq), r)) { +while (!printk_get_next_message (&pmsg,atomic64_read (&user->seq), true)) {A problem is that
printk_get_next_message()does not format the message when it shoud get supressed on the console.
Nice catch. I missed that.
Quoted 2 lines
I would solve it be adding a parameter to
printk_get_next_message()that will tell whether to suppress or not, e.g. @can_suppress.
OK.
Quoted 7 lines
Quoted 7 lines
kernel/printk/printk.c +0
-0if (file->f_flags &O_NONBLOCK ) { ret = -EAGAIN ; goto out;@@ -814,36 +814,31 @@ static ssize_t devkmsg_read(struct file *file, char __user *buf,* This pairs with __wake_up_klogd:A. */ ret = wait_event_interruptible(log_wait,
Quoted 8 lines
kernel/printk/printk.c static ssize_t devkmsg_read(struct file *file, char __user *buf, · line 818
Quoted 4 lines
kernel/printk/printk.c +2
-2-prb_read_valid (prb, - atomic64_read(&user->seq), r)); /* LMM(devkmsg_read:A) */ +prb_read_valid (prb,atomic64_read (&user->seq), + NULL)); /* LMM(devkmsg_read:A) */The above change from "if" to "while" could be avoided if we use
printk_get_next_message()here as well. It looks slightly more strightfoward to me.
Yes, that is better. A loop is overkill here.
John
John Ogness
maintainerPATCH v4 8/8 printk: adjust string limit macros
In reply to John OgnessThe various internal size limit macros have names and/or values that do not fit well to their current usage.
Rename the macros so that their purpose is clear and, if needed, provide a more appropriate value. In general, the new macros and values will lead to less memory usage. The new macros are...
PRINTK_MESSAGE_MAX:
This is the maximum size for a formatted message on a console, devkmsg, or syslog. It does not matter which format the message has (normal or extended). It replaces the use of CONSOLE_EXT_LOG_MAX for console and devkmsg. It replaces the use of CONSOLE_LOG_MAX for syslog.
Historically, normal messages have been allowed to print up to 1kB, whereas extended messages have been allowed to print up to 8kB. However, the difference in lengths of these message types is not significant and in multi-line records, normal messages are probably larger. Also, because 1kB is only slightly above the allowed record size, multi-line normal messages could be easily truncated during formatting.
This new macro should be significantly larger than the allowed record size to allow sufficient space for extended or multi-line prefix text. A value of 2kB should be plenty of space. For normal messages this represents a doubling of the historically allowed amount. For extended messages it reduces the excessive 8kB size, thus reducing memory usage needed for message formatting.
PRINTK_PREFIX_MAX:
This is the maximum size allowed for a record prefix (used by console and syslog). It replaces PREFIX_MAX. The value is left unchanged.
PRINTKRB_RECORD_MAX:
This is the maximum size allowed to be reserved for a record in the ringbuffer. It is used by all readers and writers with the printk ringbuffer. It replaces LOG_LINE_MAX.
Previously this was set to "1kB - PREFIX_MAX", which makes some sense if 1kB is the limit for normal message output and prefixes are enabled. However, with the allowance of larger output and the existence of multi-line records, the value is rather bizarre.
Round the value up to 1kB.
--- kernel/printk/internal.h | 27 +++++++++++++-------------- kernel/printk/printk.c | 24 ++++++++++++------------ 2 files changed, 25 insertions(+), 26 deletions(-)
kernel/printk/internal.h +13 -14
@@ -15,19 +15,19 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write,#ifdef CONFIG_PRINTK #ifdef CONFIG_PRINTK_CALLER -#define PREFIX_MAX 48 +#define PRINTK_PREFIX_MAX 48#else -#define PREFIX_MAX 32 +#define PRINTK_PREFIX_MAX 32#endif -/* the maximum size of a formatted record (i.e. with prefix added per line) */ -#define CONSOLE_LOG_MAX 1024 - -/* the maximum size of a formatted extended record */ -#define CONSOLE_EXT_LOG_MAX 8192 +/* +* the maximum size of a formatted record (i.e. with prefix added +* per line and dropped messages or in extended message format) +*/ +#define PRINTK_MESSAGE_MAX 2048/* the maximum size allowed to be reserved for a record */ -#define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX) +#define PRINTKRB_RECORD_MAX 1024/* Flags for a single printk record. */ enum printk_info_flags {
@@ -63,10 +63,9 @@ u16 printk_parse_prefix(const char *text, int *level, enum printk_info_flags *flags);#else -#define PREFIX_MAX 0 -#define CONSOLE_LOG_MAX 0 -#define CONSOLE_EXT_LOG_MAX 0 -#define LOG_LINE_MAX 0 +#define PRINTK_PREFIX_MAX 0 +#define PRINTK_MESSAGE_MAX 0 +#define PRINTKRB_RECORD_MAX 0/* * In !PRINTK builds we still export console_sem
@@ -85,8 +84,8 @@ static inline bool printk_percpu_data_ready(void) { return false; }* @scratchbuf: Used as temporary ringbuffer reading and string-print space. */ struct printk_buffers { -char outbuf[CONSOLE_EXT_LOG_MAX ]; -char scratchbuf[LOG_LINE_MAX ]; +char outbuf[PRINTK_MESSAGE_MAX ]; +char scratchbuf[PRINTKRB_RECORD_MAX ]; };/**
kernel/printk/printk.c +12 -12
@@ -731,7 +731,7 @@ static ssize_t devkmsg_write(struct kiocb *iocb, struct iov_iter *from)size_t len =iov_iter_count (from);ssize_t ret = len; -if (!user || len >LOG_LINE_MAX ) +if (!user || len >PRINTKRB_RECORD_MAX )return -EINVAL ;/* Ignore when user logging is disabled. */
@@ -1130,7 +1130,7 @@ static unsigned int __init add_to_rb(struct printk_ringbuffer *rb,return prb_record_text_space (&e); } -static char setup_text_buf[LOG_LINE_MAX ]__initdata ; +static char setup_text_buf[PRINTKRB_RECORD_MAX ]__initdata ;void __init setup_log_buf (int early) {
@@ -1396,7 +1396,7 @@ static size_t record_print_text(struct printk_record *r, bool syslog,size_t text_len = r->info->text_len;size_t buf_size = r->text_buf_size;char *text = r->text_buf; -char prefix[PREFIX_MAX ]; +char prefix[PRINTK_PREFIX_MAX ];bool truncated = false;size_t prefix_len;size_t line_len;
@@ -1495,7 +1495,7 @@ static size_t get_record_print_text_size(struct printk_info *info,unsigned int line_count, bool syslog, bool time) { - char prefix[PREFIX_MAX]; + char prefix[PRINTK_PREFIX_MAX];size_t prefix_len; prefix_len =info_print_prefix (info, syslog, time, prefix);
@@ -1561,11 +1561,11 @@ static int syslog_print(char __user *buf, int size)int len =0 ;u64 seq; - text =kmalloc (CONSOLE_LOG_MAX ,GFP_KERNEL ); + text =kmalloc (PRINTK_MESSAGE_MAX ,GFP_KERNEL );if (!text)return -ENOMEM ; -prb_rec_init_rd (&r, &info, text,CONSOLE_LOG_MAX ); +prb_rec_init_rd (&r, &info, text,PRINTK_MESSAGE_MAX );mutex_lock (&syslog_lock);
@@ -1666,7 +1666,7 @@ static int syslog_print_all(char __user *buf, int size, bool clear)u64 seq;bool time; - text =kmalloc (CONSOLE_LOG_MAX ,GFP_KERNEL ); + text =kmalloc (PRINTK_MESSAGE_MAX ,GFP_KERNEL );if (!text)return -ENOMEM ;
@@ -1678,7 +1678,7 @@ static int syslog_print_all(char __user *buf, int size, bool clear) seq =find_first_fitting_seq (latched_seq_read_nolock (&clear_seq),-1 , size, true, time); -prb_rec_init_rd (&r, &info, text,CONSOLE_LOG_MAX ); +prb_rec_init_rd (&r, &info, text,PRINTK_MESSAGE_MAX ); len =0 ;prb_for_each_record (seq, prb, seq, &r) {
@@ -2201,8 +2201,8 @@ int vprintk_store(int facility, int level, reserve_size =vsnprintf (&prefix_buf[0 ],sizeof (prefix_buf), fmt, args2) +1 ;va_end (args2); -if (reserve_size >LOG_LINE_MAX ) - reserve_size =LOG_LINE_MAX ; +if (reserve_size >PRINTKRB_RECORD_MAX ) + reserve_size =PRINTKRB_RECORD_MAX ;/* Extract log level or control flags. */ if (facility ==0 )
@@ -2216,7 +2216,7 @@ int vprintk_store(int facility, int level,if (flags &LOG_CONT ) {prb_rec_init_wr (&r, reserve_size); -if (prb_reserve_in_last (&e, prb, &r, caller_id,LOG_LINE_MAX )) { +if (prb_reserve_in_last (&e, prb, &r, caller_id,PRINTKRB_RECORD_MAX )) { text_len =printk_sprint (&r.text_buf[r.info->text_len], reserve_size, facility, &flags, fmt, args); r.info->text_len += text_len;
@@ -2727,7 +2727,7 @@ static void console_prepend_dropped(struct printk_message *pmsg, unsigned long d* It is a rather theoretical problem when someone tries to * use a minimalist buffer. */ -if (WARN_ON_ONCE (len +PREFIX_MAX >= outbuf_sz)) +if (WARN_ON_ONCE (len +PRINTK_PREFIX_MAX >= outbuf_sz))return ;if (pmsg->outbuf_len + len >= outbuf_sz) {
Signature · 1 line
2.30.2
Re: [PATCH printk v4 8/8] printk: adjust string limit macros
Quoted 42 lines · John Ogness
The various internal size limit macros have names and/or values that do not fit well to their current usage.
Rename the macros so that their purpose is clear and, if needed, provide a more appropriate value. In general, the new macros and values will lead to less memory usage. The new macros are...
PRINTK_MESSAGE_MAX:This is the maximum size for a formatted message on a console, devkmsg, or syslog. It does not matter which format the message has (normal or extended). It replaces the use of
CONSOLE_EXT_LOG_MAXfor console and devkmsg. It replaces the use ofCONSOLE_LOG_MAXfor syslog.Historically, normal messages have been allowed to print up to 1kB, whereas extended messages have been allowed to print up to 8kB. However, the difference in lengths of these message types is not significant and in multi-line records, normal messages are probably larger. Also, because 1kB is only slightly above the allowed record size, multi-line normal messages could be easily truncated during formatting.
This new macro should be significantly larger than the allowed record size to allow sufficient space for extended or multi-line prefix text. A value of 2kB should be plenty of space. For normal messages this represents a doubling of the historically allowed amount. For extended messages it reduces the excessive 8kB size, thus reducing memory usage needed for message formatting.
PRINTK_PREFIX_MAX:This is the maximum size allowed for a record prefix (used by console and syslog). It replaces
PREFIX_MAX. The value is left unchanged.
PRINTKRB_RECORD_MAX:This is the maximum size allowed to be reserved for a record in the ringbuffer. It is used by all readers and writers with the printk ringbuffer. It replaces
LOG_LINE_MAX.
Quoted 8 lines
Previously this was set to "1kB -
PREFIX_MAX", which makes some sense if 1kB is the limit for normal message output and prefixes are enabled. However, with the allowance of larger output and the existence of multi-line records, the value is rather bizarre.Round the value up to 1kB.
Signed-off-by John Ogness <[email protected]>
I like this.
Best Regards,
Petr
Petr Mladek
maintainerQuoted 13 lines · John Ogness
Hi,
This is v4 of a series to cleanup console buffer handling and prepare for code sharing with the upcoming threaded/atomic consoles. v3 is here [0].
The main purpose of the series is to introduce two new lockless functions to handle reading and formatting of printk messages. These functions can then be used from any context, which is important for the upcoming threaded/atomic consoles. The series also helps to cleanup some of the internal printk interfaces and cleanly separate formatting code from outputting code.
Quoted 5 lines
John Ogness (6):
printk: move size limit macros intointernal.h
printk: introduce structprintk_buffers
printk: introduceprintk_get_next_message()andprintk_message
printk: introduceconsole_prepend_dropped()for dropped messages
This patch would need a followup fix to prevent the compiler warning.
Quoted 1 line
printk: use
printk_buffersfor devkmsg
This one would need respin to fix the problem with suppressed messages.
Quoted 5 lines
printk: adjust string limit macros
Thomas Gleixner (2):
console: UseBIT()macros for @flags values
console: Document struct console
The rest looks ready for linux-next.
I see three ways how to move forward:
1. Respin the entire patchset.
2. Respin only 7th patch + send the follow up fix seperately
3. Push first 6 patches and handle the rest separately.
What would be more convenient for you, please?
Honestly, I do not have any real preference.
Best Regards,
Petr
John Ogness
maintainerQuoted 6 lines · Petr Mladek
On Thu 2023-01-05 11:43:27, John Ogness wrote:
I see three ways how to move forward:1. Respin the entire patchset.
2. Respin only 7th patch + send the follow up fix seperately
3. Push first 6 patches and handle the rest separately.
I will respin the entire patchset. That is easist for me and makes things quite straight forward for you.
I still am looking into and will to respond to your comments on patch 7/8 before I do the respin.
John