implement threaded console printing
Subsystems (7)
John Ogness
maintainercover implement threaded console printing
This is the next part of our printk-rework effort (points 2 and 6 of the LPC 2019 summary [0]).
This series implements a kthread for each registered console. The kthreads locklessly retrieve the records from the printk ringbuffer and also do not cause any lock contention between one another. This allows consoles to run at full speed. For example, a netconsole is able to dump records much faster than a serial or vt console.
There are situations where kthread printing is not sufficient. For example, during early boot before kthreads exist or during panic situations, where the kthreads may not get a chance to schedule. In these cases, the current method of printing directly within the printk() caller context is used. New functions printk_direct_enter() and printk_direct_exit() are made available to mark areas of the kernel where direct printing should be used.
This series also introduces pr_flush(): a might_sleep() function that will block until all active printing threads have caught up to the latest record at the time of the pr_flush() call. This function is useful, for example, to wait until pending records are flushed to consoles before suspending.
And finally, this series introduces a new console flag CON_MIGHT_SLEEP that can be set by consoles that only want their write() callback called in sleepable contexts. This was requested by Daniel Vetter [1] due to the many and growing number of deadlock situations in the fbcon code.
Note that this series does *not* increase the reliability of console printing. Rather it focuses on the non-interference aspect of printk() by decoupling printk() callers from printing (during normal operation). Nonetheless, the reliability aspect should not worsen due to this series (with perhaps the exception of the fbcon, which will only print from sleepable contexts).
[0] https://lore.kernel.org/lkml/[email protected]
[1] https://lore.kernel.org/all/[email protected]
John Ogness (13):
printk: rename cpulock functions
printk: cpu sync always disable interrupts
printk: use percpu flag instead of cpu_online()
printk: get caller_id/timestamp after migration disable
printk: call boot_delay_msec() in printk_delay()
printk: refactor and rework printing logic
printk: move buffer definitions into console_emit_next_record() caller
printk: add pr_flush()
printk: add functions to allow direct printing
printk: add kthread console printers
printk: reimplement console_lock for proper kthread support
printk: remove @console_locked
console: introduce CON_MIGHT_SLEEP for vt
drivers/tty/sysrq.c | 2 + drivers/tty/vt/vt.c | 2 +- include/linux/console.h | 22 +- include/linux/printk.h | 70 ++-- kernel/hung_task.c | 11 +- kernel/printk/printk.c | 903 +++++++++++++++++++++++++++++----------- kernel/rcu/tree_stall.h | 2 + kernel/reboot.c | 14 +- kernel/watchdog.c | 4 + kernel/watchdog_hld.c | 4 + lib/dump_stack.c | 6 +- lib/nmi_backtrace.c | 6 +- 12 files changed, 765 insertions(+), 281 deletions(-)
base-commit: a229327733b86aa585effdb0d27a87b12aa51597
Signature · 1 line
2.30.2
John Ogness
maintainerPATCH 1/13 printk: rename cpulock functions
Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to cpu_sync. The main functions then become:
printk_cpu_sync_get_irqsave(flags);
printk_cpu_sync_put_irqrestore(flags);--- include/linux/printk.h | 42 ++++++++++++------------- kernel/printk/printk.c | 71 +++++++++++++++++++++--------------------- lib/dump_stack.c | 4 +-- lib/nmi_backtrace.c | 4 +-- 4 files changed, 61 insertions(+), 60 deletions(-)
include/linux/printk.h +21 -21
@@ -281,43 +281,43 @@ static inline void printk_trigger_flush(void)#endif #ifdef CONFIG_SMP -extern int __printk_cpu_trylock (void ); -extern void __printk_wait_on_cpu_lock (void ); -extern void __printk_cpu_unlock (void ); +extern int __printk_cpu_sync_try_get (void ); +extern void __printk_cpu_sync_wait (void ); +extern void __printk_cpu_sync_put (void );/** -* printk_cpu_lock_irqsave() - Acquire the printk cpu-reentrant spinning -* lock and disable interrupts. +* printk_cpu_sync_get_irqsave() - Acquire the printk cpu-reentrant spinning +* lock and disable interrupts. * @flags: Stack-allocated storage for saving local interrupt state, -* to be passed to printk_cpu_unlock_irqrestore(). +* to be passed to printk_cpu_sync_put_irqrestore(). * * If the lock is owned by another CPU, spin until it becomes available. * Interrupts are restored while spinning. */ -#define printk_cpu_lock_irqsave (flags) \ - for (;;) { \ - local_irq_save(flags); \ - if (__printk_cpu_trylock()) \ - break; \ - local_irq_restore(flags); \ - __printk_wait_on_cpu_lock(); \ +#define printk_cpu_sync_get_irqsave (flags) \ + for (;;) { \ + local_irq_save(flags); \ + if (__printk_cpu_sync_try_get()) \ + break; \ + local_irq_restore(flags); \ + __printk_cpu_sync_wait(); \ }/** -* printk_cpu_unlock_irqrestore() - Release the printk cpu-reentrant spinning -* lock and restore interrupts. -* @flags: Caller's saved interrupt state, from printk_cpu_lock_irqsave(). +* printk_cpu_sync_put_irqrestore() - Release the printk cpu-reentrant spinning +* lock and restore interrupts. +* @flags: Caller's saved interrupt state, from printk_cpu_sync_get_irqsave(). */ -#define printk_cpu_unlock_irqrestore (flags) \ +#define printk_cpu_sync_put_irqrestore (flags) \ do { \ - __printk_cpu_unlock(); \ + __printk_cpu_sync_put(); \ local_irq_restore(flags); \ - } while (0) \ + } while (0)#else -#define printk_cpu_lock_irqsave (flags) ((void)flags) -#define printk_cpu_unlock_irqrestore (flags) ((void)flags) +#define printk_cpu_sync_get_irqsave (flags) ((void)flags) +#define printk_cpu_sync_put_irqrestore (flags) ((void)flags)#endif /* CONFIG_SMP */
kernel/printk/printk.c +36 -35
148 lines not on this page · raw message
lib/dump_stack.c +2 -2
@@ -102,9 +102,9 @@ asmlinkage __visible void dump_stack_lvl(const char *log_lvl)* Permit this cpu to perform nested stack dumps while serialising * against other CPUs */ -printk_cpu_lock_irqsave (flags); +printk_cpu_sync_get_irqsave (flags);__dump_stack (log_lvl); -printk_cpu_unlock_irqrestore (flags); +printk_cpu_sync_put_irqrestore (flags); }EXPORT_SYMBOL (dump_stack_lvl);
lib/nmi_backtrace.c +2 -2
@@ -99,7 +99,7 @@ bool nmi_cpu_backtrace(struct pt_regs *regs)* Allow nested NMI backtraces while serializing * against other CPUs. */ -printk_cpu_lock_irqsave (flags); +printk_cpu_sync_get_irqsave (flags);if (!READ_ONCE (backtrace_idle) && regs &&cpu_in_idle (instruction_pointer (regs))) {pr_warn ("NMI backtrace for cpu %d skipped: idling at %pS\n" , cpu, (void *)instruction_pointer (regs));
@@ -110,7 +110,7 @@ bool nmi_cpu_backtrace(struct pt_regs *regs)else dump_stack (); } -printk_cpu_unlock_irqrestore (flags); +printk_cpu_sync_put_irqrestore (flags);cpumask_clear_cpu (cpu,to_cpumask (backtrace_mask));return true; }
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 8 lines · John Ogness
Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to
cpu_sync. The main functions then become:printk_cpu_sync_get_irqsave(flags); printk_cpu_sync_put_irqrestore(flags);
It is possible that I will understand the motivation later when reading the entire patchset. But my initial reaction is confusion ;-)
From mo POV, it is a lock. It tries to get exclusive access and has to wait until the current owner releases it.
As you say: "its usage must be carefully considered and most likely will require programming locklessly." I guess that it is related to:
+ There is a risk of deadlocks that are typically associated with locks. After all the word "lock" is part of "deadlock".
+ It requires lockless programming because it is supposed to be terminal lock. It means that no other locks should be taken under it.
Is there any other API using this naming scheme, please?
I have get() and put() associated with reference counting. But it has an opposite meaning. It usually guards an object from freeing as long as there is at least one user. And it allows to have many users.
Regarding the reentrancy. It seems that "_nested" suffix is used for this type of locks, for example, mutex_lock_nested(), spin_lock_nested().
It might be enough to add "_nested" suffix and explain why it has to be used carefully (terminal lock) in a comment.
But I might miss something.
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 7 lines · Petr Mladek
Quoted 6 lines · John Ogness
Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to
cpu_sync. The main functions then become:
Quoted 5 lines
Quoted 2 lines
printk_cpu_sync_get_irqsave(flags); printk_cpu_sync_put_irqrestore(flags);It is possible that I will understand the motivation later when reading the entire patchset. But my initial reaction is confusion ;-)
Actually, the motivation comes from a discussion we had during the RT Track at Plumbers 2021 [0]. It isn't a lock and so we didn't want to call it a lock. (More below.)
Quoted 2 lines
From mo POV, it is a lock. It tries to get exclusive access and has to wait until the current owner releases it.
It is only exclusive for a CPU. If another context on that CPU tries to get the "lock" it will succeed. For example:
process context lock() -> success
--- INTERRUPT ---
irq context lock() -> success
--- NMI ---
nmi context lock() -> success
None of these contexts can assume that they have synchronized access because clearly they have all interrupted each other. If an object does not provide synchronized access to data, then "lock" is probably not a good name for that object.
Quoted 9 lines
As you say: "its usage must be carefully considered and most likely will require programming locklessly." I guess that it is related to:
+ There is a risk of deadlocks that are typically associated with locks. After all the word "lock" is part of "deadlock".
+ It requires lockless programming because it is supposed to be terminal lock. It means that no other locks should be taken under it.
It is because (as in the example above), taking this "lock" does not provide synchronization to data. It is only synchronizing between CPUs. It was Steven's suggestion to call the thing a cpu_sync object and nobody in the RT Track seemed to disagree.
Quoted 1 line
Is there any other API using this naming scheme, please?
No.
Quoted 3 lines
I have
get()andput()associated with reference counting. But it has an opposite meaning. It usually guards an object from freeing as long as there is at least one user. And it allows to have many users.
This _is_ reference counting. In fact, if you look at the implementation you see:
atomic_inc(&printk_cpu_sync_nested);
It is allowing multiple users (from the same CPU).
Quoted 6 lines
Regarding the reentrancy. It seems that "
_nested" suffix is used for this type of locks, for example,mutex_lock_nested(),spin_lock_nested().It might be enough to add "
_nested" suffix and explain why it has to be used carefully (terminal lock) in a comment.
The internal counter is called "_nested" to make it clear to us printk developers. IMO the common _get and _put semantics are appropriate here. The important thing is that the word "lock" is removed. It is not a lock.
John
Steven Rostedt
reviewerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 4 lines · On Fri, 11 Feb 2022 15:48:08 +0106 John Ogness <[email protected]>…
It is because (as in the example above), taking this "lock" does not provide synchronization to data. It is only synchronizing between CPUs. It was Steven's suggestion to call the thing a
cpu_syncobject and nobody in the RT Track seemed to disagree.
I love causing trouble ;-)
Actually, it wasn't just my suggestion. IIRC, I believe Peter Zijlstra was against calling it a lock (Peter, you can use lore to see the context here).
And besides calling it "goldie locks" I suggested calling it something that has to do with "synchronization" instead of true exclusiveness. And name it something to do with keeping one CPU from stepping on another CPU. And the entire group agreed to not call it a lock, but instead a CPU synchronization object.
Linux Plumbers RT MC summary:
https://lpc.events/event/11/page/104-accepted-microconferences#summary-rt
Printk topic video:
https://youtu.be/5jkvV4A2kT8?list=PLVsQ_xZBEyN3wA8Ej4BUjudXFbXuxhnfc&t=9217
Printk topic slides:
https://linuxplumbersconf.org/event/11/contributions/1060/attachments/829/1600/lpc2021_rtmc_ogness.pdf
(This is why I pushed all the MCs to provide summaries!)
-- Steve
Re: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 8 lines · Steven Rostedt
Quoted 4 lines · On Fri, 11 Feb 2022 15:48:08 +0106 John Ogness <[email protected]>…
It is because (as in the example above), taking this "lock" does not provide synchronization to data. It is only synchronizing between CPUs. It was Steven's suggestion to call the thing a
cpu_syncobject and nobody in the RT Track seemed to disagree.
Quoted 4 lines
I love causing trouble ;-)
Actually, it wasn't just my suggestion. IIRC, I believe Peter Zijlstra was against calling it a lock (Peter, you can use lore to see the context here).
All I remember is that it was in a room and I was late, I can't even remember what City we were all in at the time. Was this Lisbon?
Anyway, as Steve said, it isn't really a strict exclusion thing, it only avoids the most egregious inter-cpu interleaving. I'm down with goldi-locks, something has to have that name :-)
Petr Mladek
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 14 lines · Peter Zijlstra
Quoted 8 lines · Steven Rostedt
Quoted 4 lines · On Fri, 11 Feb 2022 15:48:08 +0106 John Ogness <[email protected]>…
It is because (as in the example above), taking this "lock" does not provide synchronization to data. It is only synchronizing between CPUs. It was Steven's suggestion to call the thing a
cpu_syncobject and nobody in the RT Track seemed to disagree.Quoted 4 lines
I love causing trouble ;-)
Actually, it wasn't just my suggestion. IIRC, I believe Peter Zijlstra was against calling it a lock (Peter, you can use lore to see the context here).
Quoted 6 lines
All I remember is that it was in a room and I was late, I can't even remember what City we were all in at the time. Was this Lisbon?
Anyway, as Steve said, it isn't really a strict exclusion thing, it only avoids the most egregious inter-cpu interleaving. I'm down with goldi-locks, something has to have that name :-)
You troublemakers :-)
OK, I know, I am the troublemaker here.
Best Regards,
Petr
Petr Mladek
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
In reply to John OgnessQuoted 11 lines · John Ogness
Quoted 10 lines · Petr Mladek
Quoted 8 lines · John Ogness
Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to
cpu_sync. The main functions then become:printk_cpu_sync_get_irqsave(flags); printk_cpu_sync_put_irqrestore(flags);
Quoted 6 lines
Quoted 2 lines
It is possible that I will understand the motivation later when reading the entire patchset. But my initial reaction is confusion ;-)
Actually, the motivation comes from a discussion we had during the RT Track at Plumbers 2021 [0]. It isn't a lock and so we didn't want to call it a lock. (More below.)
Thanks for the link. I have listened to the discussion. And I am still not persuaded ;-)
Quoted 12 lines
Quoted 2 lines
From mo POV, it is a lock. It tries to get exclusive access and has to wait until the current owner releases it.
It is only exclusive for a CPU. If another context on that CPU tries to get the "lock" it will succeed. For example:
process context
lock()-> success
--- INTERRUPT ---
irq contextlock()-> success
--- NMI ---
nmi contextlock()-> success
Quoted 4 lines
None of these contexts can assume that they have synchronized access because clearly they have all interrupted each other. If an object does not provide synchronized access to data, then "lock" is probably not a good name for that object.
All _nested_ locks have these limits. In fact, _all_ locks have these limits. This is why it is common to take many locks (chain of locks) if you really want to serialize some things. This is why there are ABBA problems.
Quoted 6 lines
Quoted 6 lines
As you say: "its usage must be carefully considered and most likely will require programming locklessly." I guess that it is related to:
+ There is a risk of deadlocks that are typically associated with locks. After all the word "lock" is part of "deadlock".
Quoted 8 lines
Quoted 3 lines
+ It requires lockless programming because it is supposed to be terminal lock. It means that no other locks should be taken under it.
It is because (as in the example above), taking this "lock" does not provide synchronization to data. It is only synchronizing between CPUs. It was Steven's suggestion to call the thing a
cpu_syncobject and nobody in the RT Track seemed to disagree.
IMHO, the main task of this API is to synchronize CPUs. It is normal that a lock does not protect all objects that are accessed under the lock.
Quoted 10 lines
Quoted 3 lines
I have
get()andput()associated with reference counting. But it has an opposite meaning. It usually guards an object from freeing as long as there is at least one user. And it allows to have many users.This _is_ reference counting. In fact, if you look at the implementation you see:
atomic_inc(&printk_cpu_sync_nested);It is allowing multiple users (from the same CPU).
Yes. My point is that reference counting prevents releasing of an object. It does not prevent parallel access. The parallel access is prevented by locks.
From my POV, the main task of this API is to prevent parallel printing from other CPUs. Even Steven Rostedt wrote in the chat "This makes it a lock" see the recording[0] around the time 2:46:14.
Quoted 11 lines
Quoted 6 lines
Regarding the reentrancy. It seems that "
_nested" suffix is used for this type of locks, for example,mutex_lock_nested(),spin_lock_nested().It might be enough to add "
_nested" suffix and explain why it has to be used carefully (terminal lock) in a comment.The internal counter is called "
_nested" to make it clear to us printk developers. IMO the common_getand_putsemantics are appropriate here. The important thing is that the word "lock" is removed. It is not a lock.
Why is it so important to get rid of the word "lock", please?
Well, I probably understand it. The API must be used carefully. This whole discussion is about how to make the risks more obvious.
My main fear are deadlocks caused when someone tries to get this "cpu_sync" thing. This is why I would like to call it a lock.
I guess that you are more concerned about races between different contexts when implementing atomic consoles. This is why you prefer to avoid the word "lock".
OK, this discussion helped me to improve my mental model about this API. So, the name is getting less important for me. I would still slightly prefer to keep "lock". But I am fine with the renaming to "put/get".
[0] https://youtu.be/cZUzc0U1jJ4?t=12946
Best Regards,
Petr
Sergey Senozhatsky
reviewerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
In reply to Petr MladekQuoted 7 lines · Petr Mladek
Quoted 6 lines · John Ogness
Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to
cpu_sync. The main functions then become:
Quoted 8 lines
Quoted 2 lines
printk_cpu_sync_get_irqsave(flags); printk_cpu_sync_put_irqrestore(flags);It is possible that I will understand the motivation later when reading the entire patchset. But my initial reaction is confusion ;-)
From mo POV, it is a lock. It tries to get exclusive access and has to wait until the current owner releases it.
printk has been using enter/exit naming for a while now (starting with nmi enter/exit, then printk_safe enter/exit and soon direct enter/exit); so may be we can follow suit here and use printk_cpu_sync_enter() and printk_cpu_sync_exit()?
Quoted 1 line
After all the word "lock" is part of "deadlock".
That's a good one.
John Ogness
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 4 lines · Sergey Senozhatsky
printk has been using enter/exit naming for a while now (starting with nmi enter/exit, then
printk_safeenter/exit and soon direct enter/exit); so may be we can follow suit here and useprintk_cpu_sync_enter()andprintk_cpu_sync_exit()?
I like this suggestion. So it would be:
printk_cpu_sync_enter_irqsave()printk_cpu_sync_exit_irqrestore()
Petr Mladek
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
In reply to Sergey SenozhatskyQuoted 14 lines · Sergey Senozhatsky
Quoted 13 lines · Petr Mladek
Quoted 8 lines · John Ogness
Since the printk cpulock is CPU-reentrant and since it is used in all contexts, its usage must be carefully considered and most likely will require programming locklessly. To avoid mistaking the printk cpulock as a typical lock, rename it to
cpu_sync. The main functions then become:printk_cpu_sync_get_irqsave(flags); printk_cpu_sync_put_irqrestore(flags);It is possible that I will understand the motivation later when reading the entire patchset. But my initial reaction is confusion ;-)
Quoted 7 lines
Quoted 2 lines
From mo POV, it is a lock. It tries to get exclusive access and has to wait until the current owner releases it.
printk has been using enter/exit naming for a while now (starting with nmi enter/exit, then
printk_safeenter/exit and soon direct enter/exit); so may be we can follow suit here and useprintk_cpu_sync_enter()andprintk_cpu_sync_exit()?
Interesting idea.
Honestly, I do not like it much. The existing enter/exit API is not blocking but "cpu_sync" API is.
This patch is about how to make it more obvious that this API has to be used carefully. I see the following main risks when using this API:
+ it might cause deadlocks, especially in panic
+ it is supposed to be tail lock (no other locks allowed in this context)
+ it is re-entrant
+ it does not prevent parallel (nested) access on the same CPU
"get/put" are more acceptable for me. They create at lest some feeling that it tries to get something and it might take some time. Even thought many "get/put" APIs are not blocking.
BTW: The API disables IRQ. So the nested access is limited to two
levels: normal/IRQ and nested NMI contexts.
Best Regards,
Petr
Sergey Senozhatsky
reviewerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
Quoted 8 lines · Petr Mladek
Quoted 4 lines
printk has been using enter/exit naming for a while now (starting with nmi enter/exit, then
printk_safeenter/exit and soon direct enter/exit); so may be we can follow suit here and useprintk_cpu_sync_enter()andprintk_cpu_sync_exit()?Interesting idea.
Honestly, I do not like it much.
:) OK
Quoted 11 lines
The existing enter/exit API is not blocking but "
cpu_sync" API is.This patch is about how to make it more obvious that this API has to be used carefully. I see the following main risks when using this API:
+ it might cause deadlocks, especially in panic
+ it is supposed to be tail lock (no other locks allowed in this context)
Quoted 8 lines
+ it is re-entrant
+ it does not prevent parallel (nested) access on the same CPU
"get/put" are more acceptable for me. They create at lest some feeling that it tries to get something and it might take some time. Even thought many "get/put" APIs are not blocking.
I don't mind "get/put".
The thing that looks nice in enter/exit is that enter/exit don't expose any implementation details: is there a lock behind or not, is there a ref-counter or not, and so on.
John Ogness
maintainerRe: [PATCH printk v1 01/13] printk: rename cpulock functions
In reply to Petr MladekQuoted 4 lines · Petr Mladek
This patch is about how to make it more obvious that this API has to be used carefully. I see the following main risks when using this API:
Quoted 8 lines
+ it might cause deadlocks, especially in panic
+ it is supposed to be tail lock (no other locks allowed in this context)
+ it is re-entrant
+ it does not prevent parallel (nested) access on the same CPU
I can add more information about the risks to the function documentation (include/linux/printk.h).
Quoted 3 lines
"get/put" are more acceptable for me. They create at lest some feeling that it tries to get something and it might take some time. Even thought many "get/put" APIs are not blocking.
OK. So I will keep the naming for the series v2.
Quoted 2 lines
BTW: The API disables IRQ. So the nested access is limited to two
levels: normal/IRQ and nested NMI contexts.
And there are reentrant call chains. For example, we already have:
nmi_cpu_backtrace()
printk_cpu_sync_get_irqsave(flags);
dump_stack();
printk_cpu_sync_get_irqsave(flags);
__dump_stack(log_lvl);
printk_cpu_sync_put_irqrestore(flags);
printk_cpu_sync_put_irqrestore(flags);John
John Ogness
maintainerPATCH 2/13 printk: cpu sync always disable interrupts
In reply to John OgnessThe CPU sync functions are a NOP for !CONFIG_SMP. But for !CONFIG_SMP they still need to disable interrupts in order to preserve context within the CPU sync sections.
--- include/linux/printk.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
include/linux/printk.h +9 -9
@@ -285,9 +285,16 @@ extern int __printk_cpu_sync_try_get(void);extern void __printk_cpu_sync_wait (void );extern void __printk_cpu_sync_put (void ); +#else + +#define __printk_cpu_sync_try_get () true +#define __printk_cpu_sync_wait () +#define __printk_cpu_sync_put () +#endif /* CONFIG_SMP */ +/** -* printk_cpu_sync_get_irqsave() - Acquire the printk cpu-reentrant spinning -* lock and disable interrupts. +* printk_cpu_sync_get_irqsave() - Disable interrupts and acquire the printk +* cpu-reentrant spinning lock. * @flags: Stack-allocated storage for saving local interrupt state, * to be passed to printk_cpu_sync_put_irqrestore(). *
@@ -314,13 +321,6 @@ extern void __printk_cpu_sync_put(void);local_irq_restore (flags); \ }while (0 ) -#else - -#define printk_cpu_sync_get_irqsave (flags) ((void)flags) -#define printk_cpu_sync_put_irqrestore (flags) ((void)flags) - -#endif /* CONFIG_SMP */ - extern int kptr_restrict;/**
Signature · 1 line
2.30.2
Re: [PATCH printk v1 02/13] printk: cpu sync always disable interrupts
Quoted 5 lines · John Ogness
The CPU sync functions are a NOP for !CONFIG_SMP. But for !CONFIG_SMP they still need to disable interrupts in order to preserve context within the CPU sync sections.
Signed-off-by John Ogness <[email protected]>
Good catch!
It is interesting that the original code did not disable interrupts around __dump_stack() on non-SMP, see the commit 766c268bc6d39b8124 ("lib/dump_stack: move cpu lock to printk.c"). But it was rather a bug. The patch makes perfect sense.
Best Regards,
Petr
Re: [PATCH printk v1 02/13] printk: cpu sync always disable interrupts
Quoted 7 lines · Petr Mladek
Quoted 5 lines · John Ogness
The CPU sync functions are a NOP for !CONFIG_SMP. But for !CONFIG_SMP they still need to disable interrupts in order to preserve context within the CPU sync sections.
Signed-off-by John Ogness <[email protected]>
Quoted 8 lines
Good catch!
It is interesting that the original code did not disable interrupts around
__dump_stack()on non-SMP, see the commit766c268bc6d39b8124("lib/dump_stack: move cpu lock toprintk.c"). But it was rather a bug. The patch makes perfect sense.Reviewed-by Petr Mladek <[email protected]>
John Ogness
maintainerPATCH 3/13 printk: use percpu flag instead of cpu_online()
In reply to John OgnessThe CON_ANYTIME console flag is used to label consoles that will work correctly before percpu resources are allocated. To check the condition, cpu_online(raw_smp_processor_id()) was used. However, this is odd because CPUs can go offline at a later point. Also, the function is forced to use the raw_ variant because migration is not disabled.
Since commit ab6f762f0f53 ("printk: queue wake_up_klogd irq_work only if per-CPU areas are ready") there is a variable to identify if percpu resources have been allocated. Use that variable instead of cpu_online(raw_smp_processor_id()).
--- include/linux/console.h | 2 +- kernel/printk/printk.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
include/linux/console.h +1 -1
@@ -133,7 +133,7 @@ static inline int con_debug_leave(void)#define CON_CONSDEV (2)/* Preferred console, /dev/console */ #define CON_ENABLED (4)#define CON_BOOT (8) -#define CON_ANYTIME (16)/* Safe to call when cpu is offline */ +#define CON_ANYTIME (16)/* Safe to call before per-cpu resources ready */ #define CON_BRL (32)/* Used for a braille device */ #define CON_EXTENDED (64)/* Use the extended output format a la /dev/kmsg */
kernel/printk/printk.c +2 -2
@@ -2577,11 +2577,11 @@ static int have_callable_console(void)* * Console drivers may assume that per-cpu resources have been allocated. So * unless they're explicitly marked as being able to cope (CON_ANYTIME) don't -* call them until this CPU is officially up. +* call them until per-cpu resources have been allocated. */ static inline int can_use_console (void ) { -return cpu_online (raw_smp_processor_id ()) ||have_callable_console (); +return (printk_percpu_data_ready () ||have_callable_console ()); }/**
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 7 lines · John Ogness
The
CON_ANYTIMEconsole flag is used to label consoles that will work correctly before percpu resources are allocated. To check the condition, cpu_online(raw_smp_processor_id()) was used. However, this is odd because CPUs can go offline at a later point. Also, the function is forced to use theraw_variant because migration is not disabled.
Quoted 6 lines
Since commit
ab6f762f0f53("printk: queuewake_up_klogdirq_workonly if per-CPU areas are ready") there is a variable to identify if percpu resources have been allocated. Use that variable instead of cpu_online(raw_smp_processor_id()).Signed-off-by John Ogness <[email protected]>
I dug more into the history to be more confident about the change. I am not sure that it is safe.
CON_ANYTIME really seems to be related to per-CPU variables and early boot, see the commit 76a8ad293912cd2f01eca ("[PATCH] Make printk work for really early debugging").
I am not 100% sure. I think that the support for per-CPU variables is enabled by setup_per_cpu_areas(). It is done for_each_possible_cpu() So that per-CPU variables should always be available after printk_percpu_data_ready(). So far, so good.
The problem is the commit ac25575203c11145066ea ("[PATCH] CPU hotplug printk fix"). It suggests that per-CPU data of slab are freed during hotplug.
There are many other things that are manipulated during cpu hotplug. And there are the two notifiers "printk:dead" and "printk:online", see printk_late_init(). Maybe, we should use them to decide whether the non-trivial consoles are callable during CPU hotplug.
Best Regards,
Petr
Sergey Senozhatsky
reviewerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 6 lines · Petr Mladek
I dug more into the history to be more confident about the change. I am not sure that it is safe.
CON_ANYTIMEreally seems to be related to per-CPU variables and early boot, see the commit76a8ad293912cd2f01eca("[PATCH] Make printk work for really early debugging").
Hmm... This looks a bit tricky. It seems that the commit in question is concerned with per-CPU variables access indeed; not printk() per-CPU access though but per-CPU access from con->write(). I guess console_unlock() did not use to access per-CPU variables before, it does now, so that can_use_console() in fact protects both printk (which certainly happens) and con->write() (do we even have it?) per-CPU access
console_unlock()
{
if (!can_use_console())
return;
for (;;) {
printk_safe_enter_irqsave() -> __this_cpu()
printk_safe_exit_irqrestore() -> __this_cpu()
}
}Sergey Senozhatsky
reviewerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
In reply to Petr MladekQuoted 1 line · Petr Mladek
On Mon 2022-02-07 20:49:13, John Ogness wrote:
[..]
Quoted 8 lines
The problem is the commit
ac25575203c11145066ea("[PATCH] CPU hotplug printk fix"). It suggests that per-CPU data of slab are freed during hotplug.There are many other things that are manipulated during cpu hotplug. And there are the two notifiers "printk:dead" and "printk:online", see
printk_late_init(). Maybe, we should use them to decide whether the non-trivial consoles are callable during CPU hotplug.
Great findings. Looks like we only set __printk_percpu_data_ready to true and never set it back to false, relying on cpu_online() in such cases. But here's the thing: we have printk_percpu_data_ready() in __printk_recursion_counter() and in wake_up_klogd() and in defer_console_output(), but why we never check __printk_percpu_data_ready in __down_trylock_console_sem()/__up_console_sem() and more importantly in console_trylock_spinning() and those do access this_cpu() in printk safe enter/exit. Am I missing something?
Petr Mladek
maintainerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 12 lines · Sergey Senozhatsky
Quoted 1 line · Petr Mladek
On Mon 2022-02-07 20:49:13, John Ogness wrote:
[..]
Quoted 8 lines
The problem is the commit
ac25575203c11145066ea("[PATCH] CPU hotplug printk fix"). It suggests that per-CPU data of slab are freed during hotplug.There are many other things that are manipulated during cpu hotplug. And there are the two notifiers "printk:dead" and "printk:online", see
printk_late_init(). Maybe, we should use them to decide whether the non-trivial consoles are callable during CPU hotplug.
Quoted 8 lines
Great findings. Looks like we only set
__printk_percpu_data_readyto true and never set it back to false, relying oncpu_online()in such cases. But here's the thing: we haveprintk_percpu_data_ready()in__printk_recursion_counter()and inwake_up_klogd()and indefer_console_output(), but why we never check__printk_percpu_data_readyin __down_trylock_console_sem()/__up_console_sem() and more importantly inconsole_trylock_spinning()and those do accessthis_cpu()in printk safe enter/exit. Am I missing something?
Great point!
I am not 100% sure. But it seems that static per-CPU variables might actually be used since the boot.
This is from mm/percpu.c
* There is special consideration for the first chunk which must handle
* the static percpu variables in the kernel image as allocation services
* are not online yet. In short, the first chunk is structured like so:
*
* <Static | [Reserved] | Dynamic>
*
* The static data is copied from the original section managed by the
* linker. The reserved section, if non-zero, primarily manages static
* percpu variables from kernel modules. Finally, the dynamic section
* takes care of normal allocations.
I thought that it might work only for CPU0. But it seems that it probably works for each possible cpu, see:
bool __is_kernel_percpu_address(unsigned long addr, unsigned long *can_addr) {
#ifdef CONFIG_SMP
const size_t static_size = __per_cpu_end - __per_cpu_start;
void __percpu *base = __addr_to_pcpu_ptr(pcpu_base_addr);
unsigned int cpu;
for_each_possible_cpu(cpu) {
void *start = per_cpu_ptr(base, cpu);
void *va = (void *)addr;
if (va >= start && va < start + static_size) {[...]
}
and
/**
* is_kernel_percpu_address - test whether address is from static percpu area
* @addr: address to test
*
* Test whether @addr belongs to in-kernel static percpu area. Module
* static percpu areas are not considered. For those, use
* is_module_percpu_address().
*
* RETURNS:
* %true if @addr is from in-kernel static percpu area, %false otherwise. */
bool is_kernel_percpu_address(unsigned long addr)
{
return __is_kernel_percpu_address(addr, NULL);
}Most likely, only dynamically allocated per-cpu variables have to wait until the per-cpu areas are initialized.
This might explain why there is no genericare_per_cpu_variables_ready() callback.
We should probably revisit the code and remove the fallback to normal static variables.
Best Regards,
Petr
Sergey Senozhatsky
reviewerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 22 lines · Petr Mladek
Quoted 12 lines · Sergey Senozhatsky
Quoted 1 line · Petr Mladek
On Mon 2022-02-07 20:49:13, John Ogness wrote:
[..]
Quoted 8 lines
The problem is the commit
ac25575203c11145066ea("[PATCH] CPU hotplug printk fix"). It suggests that per-CPU data of slab are freed during hotplug.There are many other things that are manipulated during cpu hotplug. And there are the two notifiers "printk:dead" and "printk:online", see
printk_late_init(). Maybe, we should use them to decide whether the non-trivial consoles are callable during CPU hotplug.Quoted 8 lines
Great findings. Looks like we only set
__printk_percpu_data_readyto true and never set it back to false, relying oncpu_online()in such cases. But here's the thing: we haveprintk_percpu_data_ready()in__printk_recursion_counter()and inwake_up_klogd()and indefer_console_output(), but why we never check__printk_percpu_data_readyin __down_trylock_console_sem()/__up_console_sem() and more importantly inconsole_trylock_spinning()and those do accessthis_cpu()in printk safe enter/exit. Am I missing something?
Quoted 4 lines
Great point!
I am not 100% sure. But it seems that static per-CPU variables might actually be used since the boot.
Wow, this is great to learn. Thanks!
Quoted 51 lines
This is from
mm/percpu.c* There is special consideration for the first chunk which must handle
* the static percpu variables in the kernel image as allocation services
* are not online yet. In short, the first chunk is structured like so:
*
* <Static | [Reserved] | Dynamic>
*
* The static data is copied from the original section managed by the
* linker. The reserved section, if non-zero, primarily manages static
* percpu variables from kernel modules. Finally, the dynamic section
* takes care of normal allocations.I thought that it might work only for CPU0. But it seems that it probably works for each possible cpu, see:
bool
__is_kernel_percpu_address(unsigned long addr, unsigned long *can_addr){#ifdef CONFIG_SMP const size_t static_size = __per_cpu_end - __per_cpu_start; void __percpu *base = __addr_to_pcpu_ptr(pcpu_base_addr); unsigned int cpu; for_each_possible_cpu(cpu) { void *start = per_cpu_ptr(base, cpu); void *va = (void *)addr; if (va >= start && va < start + static_size) {[...]
}and
/**
*is_kernel_percpu_address- test whether address is from static percpu area
* @addr: address to test
*
* Test whether @addr belongs to in-kernel static percpu area. Module
* static percpu areas are not considered. For those, use
*is_module_percpu_address().
*
* RETURNS:
* %true if @addr is from in-kernel static percpu area, %false otherwise. */
boolis_kernel_percpu_address(unsigned long addr){ return __is_kernel_percpu_address(addr, NULL); }
Quoted 8 lines
Most likely, only dynamically allocated per-cpu variables have to wait until the per-cpu areas are initialized.
This might explain why there is no generic
are_per_cpu_variables_ready()callback.We should probably revisit the code and remove the fallback to normal static variables.
Agreed.
John Ogness
maintainerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
In reply to Petr MladekI have taken some time to investigate the percpu implementation so that I could provide clear answers here.
Quoted 2 lines · Petr Mladek
I am not 100% sure. But it seems that static per-CPU variables might actually be used since the boot.
You are correct, but until per-cpu areas are setup, they all point to CPU0. Normally that is not a problem since usually code is using this_cpu_ptr() (which will always be the CPU0 in early boot), rather than specifying foreign CPUs with per_cpu_ptr().
Quoted 2 lines
Most likely, only dynamically allocated per-cpu variables have to wait until the per-cpu areas are initialized.
It is also important to wait if data will be stored that is no longer valid after per-cpu areas are setup. setup_per_cpu_areas() copies the static CPU0 per-cpu value to all the newly setup per-cpu areas.
This is actually the cause for the mystery [0] of failing irq_work when printk_deferred was added with commit 15341b1dd409749fa5625e4b632013b6ba81609b ("char/random: silence a lockdep splat with printk()". The problem was that the irq work was being queued and _claimed_ before per-cpu areas were setup. But the actual work functions are not called there. So when the irq_work was copied to all the new per-cpu areas and the actual work was carried out, the work item that was cleared was the old static variable and not the new per-cpu area variable. This means all the irq_work flags were left in the claimed state (PENDING|BUSY) and could no longer be claimed.
By avoiding queueing irq_work before setup_per_cpu_areas(), we correctly avoided this problem. (I considered sending a patch so that irq_work_claim() will fail if a global @percpu_complete is not yet set. But for now, our set_percpu_data_ready() solution is at least good enough for the printk subsystem.)
Quoted 2 lines
We should probably revisit the code and remove the fallback to normal static variables.
Definitely. Now it is clear that @printk_count and @printk_count_nmi do not need early variants.
John
Petr Mladek
maintainerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 15 lines · John Ogness
I have taken some time to investigate the percpu implementation so that I could provide clear answers here.
Quoted 2 lines · Petr Mladek
I am not 100% sure. But it seems that static per-CPU variables might actually be used since the boot.
You are correct, but until per-cpu areas are setup, they all point to CPU0. Normally that is not a problem since usually code is using
this_cpu_ptr()(which will always be the CPU0 in early boot), rather than specifying foreign CPUs withper_cpu_ptr().Quoted 2 lines
Most likely, only dynamically allocated per-cpu variables have to wait until the per-cpu areas are initialized.
Quoted 8 lines
It is also important to wait if data will be stored that is no longer valid after per-cpu areas are setup.
setup_per_cpu_areas()copies the static CPU0 per-cpu value to all the newly setup per-cpu areas.This is actually the cause for the mystery [0] of failing
irq_workwhenprintk_deferredwas added with commit15341b1dd409749fa5625e4b632013b6ba81609b("char/random: silence a lockdep splat withprintk()".
Just for record, the right commit ID in the mainline is 1b710b1b10eff9d466. It used printk_deferred() in _warn_unseeded_randomness():
drivers/char/random.c +3 -2
@@ -1687,8 +1687,9 @@ static void _warn_unseeded_randomness(const char *func_name, void *caller, print_once = true;#endif if (__ratelimit (&unseeded_warning)) -pr_notice ("random: %s called from %pS with crng_init=%d\n" , - func_name, caller, crng_init); +printk_deferred (KERN_NOTICE "random: %s called from %pS " +"with crng_init=%d\n" , func_name, caller, + crng_init); }/*
Quoted 4 lines
By avoiding queueing
irq_workbeforesetup_per_cpu_areas(), we correctly avoided this problem. (I considered sending a patch so thatirq_work_claim()will fail if a global @percpu_complete is not yet set.
That is a great TODO :-)
Quoted 2 lines
But for now, our
set_percpu_data_ready()solution is at least good enough for the printk subsystem.)
Yes but it is most likely not needed for CON_ANYTIME consoles, see below.
Quoted 5 lines
Quoted 2 lines
We should probably revisit the code and remove the fallback to normal static variables.
Definitely. Now it is clear that @printk_count and @printk_count_nmi do not need early variants.
Note the ordering:
asmlinkage __visible void __init __no_sanitize_address start_kernel(void) {
[...]
early_security_init();
[...]
setup_per_cpu_areas();
[...]
parse_early_param();
after_dashes = parse_args("Booting kernel",My guess is that _warn_unseeded_randomness() was called from a code before early_security_init(). It was bad because per-CPU variables were not ready yet.
The early consoles are enabled by parse_early_param(). It happens after setup_per_cpu_areas(). It means that all console drivers should be on the safe side.
Best Regards,
Petr
Jason A. Donenfeld
maintainerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Hi Petr,
Quoted 6 lines · Petr Mladek
Just for record, the right commit ID in the mainline is
1b710b1b10eff9d466. It usedprintk_deferred()in_warn_unseeded_randomness():
drivers/char/random.c static void _warn_unseeded_randomness(const char *func_name, void *caller, · line 1693
Quoted 9 lines
drivers/char/random.c +3
-2@@ -1687,8 +1687,9 @@ static void _warn_unseeded_randomness(const char *func_name, void *caller, print_once = true;#endif if (__ratelimit (&unseeded_warning)) -pr_notice ("random: %s called from %pS with crng_init=%d\n" , - func_name, caller, crng_init); +printk_deferred (KERN_NOTICE "random: %s called from %pS " +"with crng_init=%d\n" , func_name, caller, + crng_init); }
Are we able to revert this yet? Or is it still required because of locking issues? Would gladly take a patch to revert that if the non-deferred function is fine for 5.18.
Jason
Petr Mladek
maintainerRe: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 7 lines · Jason A. Donenfeld
Hi Petr,
Quoted 3 lines · Petr Mladek
Just for record, the right commit ID in the mainline is
1b710b1b10eff9d466. It usedprintk_deferred()in_warn_unseeded_randomness():
Quoted 16 lines
drivers/char/random.c static void _warn_unseeded_randomness(const char *func_name, void *caller, · line 1693
Quoted 12 lines
drivers/char/random.c +3
-2@@ -1687,8 +1687,9 @@ static void _warn_unseeded_randomness(const char *func_name, void *caller, print_once = true;#endif if (__ratelimit (&unseeded_warning)) -pr_notice ("random: %s called from %pS with crng_init=%d\n" , - func_name, caller, crng_init); +printk_deferred (KERN_NOTICE "random: %s called from %pS " +"with crng_init=%d\n" , func_name, caller, + crng_init); }Are we able to revert this yet? Or is it still required because of locking issues? Would gladly take a patch to revert that if the non-deferred function is fine for 5.18.
Unfortunately, printk_deferred() will still be needed in 5.18 here.
One thing is that this patchset introducing console kthreads will not be ready for 5.18.
But more importantly, the kthreads will not be enough to remove the cyclic dependency. The legacy synchronous printing will still be used during early boot, panic, etc.
Honestly, I am still not sure if we will be able to get rid of printk_deferred(). I hope so but some more magic will be necessary. Anyway, the kthreads should help to achieve this.
Best Regards,
Petr
Petr Mladek
maintainertwo locations: was: Re: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
In reply to John OgnessQuoted 20 lines · John Ogness
The
CON_ANYTIMEconsole flag is used to label consoles that will work correctly before percpu resources are allocated. To check the condition, cpu_online(raw_smp_processor_id()) was used. However, this is odd because CPUs can go offline at a later point. Also, the function is forced to use theraw_variant because migration is not disabled.Since commit
ab6f762f0f53("printk: queuewake_up_klogdirq_workonly if per-CPU areas are ready") there is a variable to identify if percpu resources have been allocated. Use that variable instead of cpu_online(raw_smp_processor_id()).include/linux/console.h +0
-0kernel/printk/printk.c +0
-0@@ -2577,11 +2577,11 @@ static int have_callable_console(void)*
kernel/printk/printk.c static int have_callable_console(void) · line 2585
Quoted 10 lines
kernel/printk/printk.c +2
-2* Console drivers may assume that per-cpu resources have been allocated. So * unless they're explicitly marked as being able to cope (CON_ANYTIME) don't -* call them until this CPU is officially up. +* call them until per-cpu resources have been allocated. */ static inline int can_use_console (void ) { -return cpu_online (raw_smp_processor_id ()) ||have_callable_console (); +return (printk_percpu_data_ready () ||have_callable_console ()); }
cpu_online(raw_smp_processor_id()) check is used also in call_console_drivers(). The same logic should be used in both locations.
I found this when reviewing 6th patch that replaced both checks with a single one.
Note that I am still not sure if this change is correct at all. It will allow to always call the console during CPU hotplug and I am not sure if it is safe. IMHO, it might cause problems when a console driver uses, for example, CPU-bound workqueues.
Best Regards,
Petr
John Ogness
maintainerRe: two locations: was: Re: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 21 lines · Petr Mladek
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -2577,11 +2577,11 @@ static int have_callable_console(void)* kernel/printk/printk.c static int have_callable_console(void) · line 2585
Quoted 10 lines
kernel/printk/printk.c +2
-2* Console drivers may assume that per-cpu resources have been allocated. So * unless they're explicitly marked as being able to cope (CON_ANYTIME) don't -* call them until this CPU is officially up. +* call them until per-cpu resources have been allocated. */ static inline int can_use_console (void ) { -return cpu_online (raw_smp_processor_id ()) ||have_callable_console (); +return (printk_percpu_data_ready () ||have_callable_console ()); }cpu_online(raw_smp_processor_id()) check is used also in
call_console_drivers(). The same logic should be used in both locations.
Quoted 7 lines
I found this when reviewing 6th patch that replaced both checks with a single one.
Note that I am still not sure if this change is correct at all. It will allow to always call the console during CPU hotplug and I am not sure if it is safe. IMHO, it might cause problems when a console driver uses, for example, CPU-bound workqueues.
You are correct. We must take hotplug into account for !CON_ANYTIME consoles. There may be some hotplug callbacks that make memory unavailable for the console.
However, I will add the use of printk_percpu_data_ready() in the check. !CON_ANYTIME consoles also should not be called until the per-cpu areas are ready. For example, it would be bad if a console queued irq_work before per-cpu areas are setup (cpu_online() is true during this time).
One of my main concerns was that raw_smp_processor_id() was used for the check. It is conceptually wrong to exclude certain consoles based on a current CPU when migration is still enabled. I understand that the use of can_use_console() is an optimization to avoid doing extra work where there are no consoles available. But the task could be preemptible there and _conceptually_, could get moved to another CPU before its write() callback is called. The cpu_online() check belongs in code where preemption is disabled.
If the context is preemptible, I do not think it will ever see !cpu_online(). So I think if the cpu_online() check is limited to unlocking when console_trylock() was used, it will be correct.
In the current implementation of printk(), it would be odd to do this conditional check (perhaps by passing @do_cond_resched to can_use_console()). But my series does significant refactoring and actually does need to distinguish between console_lock() and console_trylock() due to the kthreads and supporting the handover. So it should work well that the cpu_online() check for !CON_ANYTIME is only performed when !preemptible.
Regardless, my v2 will keep cpu_online() checks since they are necessary for hotplug support.
John
Petr Mladek
maintainerRe: two locations: was: Re: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 49 lines · John Ogness
Quoted 21 lines · Petr Mladek
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -2577,11 +2577,11 @@ static int have_callable_console(void)* kernel/printk/printk.c static int have_callable_console(void) · line 2585
Quoted 10 lines
kernel/printk/printk.c +2
-2* Console drivers may assume that per-cpu resources have been allocated. So * unless they're explicitly marked as being able to cope (CON_ANYTIME) don't -* call them until this CPU is officially up. +* call them until per-cpu resources have been allocated. */ static inline int can_use_console (void ) { -return cpu_online (raw_smp_processor_id ()) ||have_callable_console (); +return (printk_percpu_data_ready () ||have_callable_console ()); }cpu_online(raw_smp_processor_id()) check is used also in
call_console_drivers(). The same logic should be used in both locations.Quoted 7 lines
I found this when reviewing 6th patch that replaced both checks with a single one.
Note that I am still not sure if this change is correct at all. It will allow to always call the console during CPU hotplug and I am not sure if it is safe. IMHO, it might cause problems when a console driver uses, for example, CPU-bound workqueues.
You are correct. We must take hotplug into account for !CON_ANYTIME consoles. There may be some hotplug callbacks that make memory unavailable for the console.
However, I will add the use of
printk_percpu_data_ready()in the check. !CON_ANYTIME consoles also should not be called until the per-cpu areas are ready. For example, it would be bad if a console queuedirq_workbefore per-cpu areas are setup (cpu_online()is true during this time).One of my main concerns was that
raw_smp_processor_id()was used for the check. It is conceptually wrong to exclude certain consoles based on a current CPU when migration is still enabled. I understand that the use ofcan_use_console()is an optimization to avoid doing extra work where there are no consoles available. But the task could be preemptible there and _conceptually_, could get moved to another CPU before itswrite()callback is called. Thecpu_online()check belongs in code where preemption is disabled.
Quoted 3 lines
If the context is preemptible, I do not think it will ever see !cpu_online(). So I think if the
cpu_online()check is limited to unlocking whenconsole_trylock()was used, it will be correct.
This would require calling console_lock()/console_unlock() in a hotplug code when cpu_online() already returns false. Do I get it correctly?
I agree that it should not happen. console_lock() must be called in a preemptible context. And CPU should not be in a pre-emptible context when cpu_online() returns false. To be honest, I did not check the code. It just does not make much sense.
Quoted 10 lines
In the current implementation of
printk(), it would be odd to do this conditional check (perhaps by passing @do_cond_resched tocan_use_console()). But my series does significant refactoring and actually does need to distinguish betweenconsole_lock()andconsole_trylock()due to the kthreads and supporting the handover. So it should work well that thecpu_online()check for !CON_ANYTIME is only performed when !preemptible.Regardless, my v2 will keep
cpu_online()checks since they are necessary for hotplug support.
Yes, I would do it to stay on the safe side.
Best Regards,
Petr
John Ogness
maintainerRe: two locations: was: Re: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
Quoted 7 lines · Petr Mladek
Quoted 3 lines
If the context is preemptible, I do not think it will ever see !cpu_online(). So I think if the
cpu_online()check is limited to unlocking whenconsole_trylock()was used, it will be correct.This would require calling console_lock()/console_unlock() in a hotplug code when
cpu_online()already returns false. Do I get it correctly?
Yes, but there is also the other side: Are there preemptible code paths that are calling console_trylock()? And if yes, then why are they not using console_lock()?
There is only a handfull of console_trylock() users (about 10 sites). Most of them appear to be differentiating from preemptible and non-preemptible contexts. I will investigate them in detail.
If the raw_smp_processor_id() is changed to smp_processor_id() within the console_trylock()/console_unlock() call path, then at least the kernel's lock debugging would identify if there are preemptible contexts that are using console_trylock().
John
Petr Mladek
maintainerRe: two locations: was: Re: [PATCH printk v1 03/13] printk: use percpu flag instead of cpu_online()
In reply to John OgnessQuoted 49 lines · John Ogness
Quoted 21 lines · Petr Mladek
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -2577,11 +2577,11 @@ static int have_callable_console(void)* kernel/printk/printk.c static int have_callable_console(void) · line 2585
Quoted 10 lines
kernel/printk/printk.c +2
-2* Console drivers may assume that per-cpu resources have been allocated. So * unless they're explicitly marked as being able to cope (CON_ANYTIME) don't -* call them until this CPU is officially up. +* call them until per-cpu resources have been allocated. */ static inline int can_use_console (void ) { -return cpu_online (raw_smp_processor_id ()) ||have_callable_console (); +return (printk_percpu_data_ready () ||have_callable_console ()); }cpu_online(raw_smp_processor_id()) check is used also in
call_console_drivers(). The same logic should be used in both locations.Quoted 7 lines
I found this when reviewing 6th patch that replaced both checks with a single one.
Note that I am still not sure if this change is correct at all. It will allow to always call the console during CPU hotplug and I am not sure if it is safe. IMHO, it might cause problems when a console driver uses, for example, CPU-bound workqueues.
You are correct. We must take hotplug into account for !CON_ANYTIME consoles. There may be some hotplug callbacks that make memory unavailable for the console.
However, I will add the use of
printk_percpu_data_ready()in the check. !CON_ANYTIME consoles also should not be called until the per-cpu areas are ready. For example, it would be bad if a console queuedirq_workbefore per-cpu areas are setup (cpu_online()is true during this time).One of my main concerns was that
raw_smp_processor_id()was used for the check. It is conceptually wrong to exclude certain consoles based on a current CPU when migration is still enabled. I understand that the use ofcan_use_console()is an optimization to avoid doing extra work where there are no consoles available. But the task could be preemptible there and _conceptually_, could get moved to another CPU before itswrite()callback is called. Thecpu_online()check belongs in code where preemption is disabled.
Quoted 3 lines
If the context is preemptible, I do not think it will ever see !cpu_online(). So I think if the
cpu_online()check is limited to unlocking whenconsole_trylock()was used, it will be correct.
I investigated the cpu hotplug code and found the following:
1. In the cpu_down() code path, @cpu_online mask is cleared by this call chain:
+ take_cpu_down()
+ __cpu_disable()
+ smp_ops.cpu_disable()
+ native_cpu_disable() # x86
+ cpu_disable_common()
+ remove_cpu_from_maps()
+ set_cpu_online(cpu, false)
, where take_cpu_down() is called via:
+ .teardown.single() calback for CPUHP_TEARDOWN_CPU state
+ takedown_cpu()
+ stop_machine_cpuslocked()
+ stop_cpus()
+ __stop_cpus()
+ queue_stop_cpus_work()
+ cpu_stop_queue_work()
, which queues the work in cpu_stopper thread that is bound
to the CPU:
+ cpu_stop_init()
+ smpboot_register_percpu_thread()
+ __smpboot_create_thread()
+ kthread_create_on_cpu()
Summary: @cpu_online mask is cleared on the affected CPU in
cpu_stopper thread that is bound to the same CPU. It happens when handling CPUHP_TEARDOWN_CPU.
2. The CPU hotplug states are split into three groups:
+ code running on control CPU (another CPU)
+ low level code running on the hotplugged CPU
+ code running in the hotplug thread on the hotplugged CPU
It is described in include/linux/cpuhotplug.h:
/* PREPARE section invoked on a control CPU */ CPUHP_OFFLINE = 0, [...] /* * STARTING section invoked on the hotplugged CPU in low level * bringup and teardown code. */ CPUHP_AP_IDLE_DEAD, [...] CPUHP_AP_ONLINE, CPUHP_TEARDOWN_CPU, /* Online section invoked on the hotplugged CPU from the hotplug thread */ CPUHP_AP_ONLINE_IDLE, CPUHP_AP_SCHED_WAIT_EMPTY, [...] CPUHP_ONLINE,
, where sched_cpu_wait_empty() is the .teardown.single callback for
CPUHP_AP_SCHED_WAIT_EMPTY. After this callback, another tasks should not be scheduled on this CPU. Any attempt should be catched and handled by sched_cpu_dying().
Note that CPUHP_AP_SCHED_WAIT_EMPTY is called before CPUHP_TEARDOWN_CPU when the CPU goes down.
Summary: @cpu_only mask is cleared for the CPU when other tasks
could not longer be sheduled there.
Result: cpu_online(raw_smp_processor_id()) should be safe for our
purpose. It will return false only the task could not longer migrate from the CPU.
I have to admit that it is far from obvious and tricky like hell.
OK, cpu_online(raw_smp_processor_id()) check is not racy. Another question is whether it is a good check whether the consoles are usable or not.
I found the following:
1. My understanding is that affinity of IRQs is disabled right after clearing @cpu_online mask:
void cpu_disable_common(void)
{
[...]
remove_cpu_from_maps(cpu);
[...]
fixup_irqs();
[...]
}2. Timers must not be used close after clearing @cpu_online mask, see see include/linux/cpuhotplug.h:
/* Must be the last timer callback */ CPUHP_AP_DUMMY_TIMER_STARTING, [...] CPUHP_AP_ONLINE, CPUHP_TEARDOWN_CPU,
Result: From my POV, cpu_online(raw_smp_processor_id()) marks
reasonable place in the CPU hotplug code when the conosles start/stop being usable.
But again, it is far from obvious and tricky like hell.
Summary: We need to keep cpu_online(raw_smp_processor_id()) check
to make the consoles safe during CPU hotplug.
IMHO, it is not about per-CPU variables. It is more about timers, interrupts. The hotplugged CPU is not ready call console code at these early hotplug stages.
Quoted 2 lines
Regardless, my v2 will keep
cpu_online()checks since they are necessary for hotplug support.
Yes, please. We should also somehow document this. But it can be done separately. It is not necessarily in the scope of your patchset.
Best Regards,
Petr
John Ogness
maintainerPATCH 4/13 printk: get caller_id/timestamp after migration disable
In reply to John OgnessCurrently the local CPU timestamp and caller_id for the record are collected while migration is enabled. Since this information is CPU-specific, it should be collected with migration disabled.
Migration is disabled immediately after collecting this information anyway, so just move the information collection to after the migration disabling.
--- kernel/printk/printk.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
kernel/printk/printk.c +6 -4
@@ -2017,7 +2017,7 @@ static inline void printk_delay(void)static inline u32 printk_caller_id (void ) {return in_task () ?task_pid_nr (current) : -0x80000000 +raw_smp_processor_id (); +0x80000000 +smp_processor_id (); }/**
@@ -2099,7 +2099,6 @@ int vprintk_store(int facility, int level,const struct dev_printk_info *dev_info, const char *fmt, va_list args) { - const u32 caller_id = printk_caller_id(); struct prb_reserved_entry e;enum printk_info_flags flags =0 ;struct printk_record r;
@@ -2109,10 +2108,14 @@ int vprintk_store(int facility, int level,u8 *recursion_ptr;u16 reserve_size;va_list args2; +u32 caller_id;u16 text_len;int ret =0 ;u64 ts_nsec; +if (!printk_enter_irqsave (recursion_ptr, irqflags)) +return 0 ; +/* * Since the duration of printk() can vary depending on the message * and state of the ringbuffer, grab the timestamp now so that it is
@@ -2121,8 +2124,7 @@ int vprintk_store(int facility, int level,*/ ts_nsec =local_clock (); -if (!printk_enter_irqsave (recursion_ptr, irqflags)) -return 0 ; + caller_id =printk_caller_id ();/* * The sprintf needs to come first since the syslog prefix might be
Signature · 1 line
2.30.2
Re: [PATCH printk v1 04/13] printk: get caller_id/timestamp after migration disable
Quoted 9 lines · John Ogness
Currently the local CPU timestamp and
caller_idfor the record are collected while migration is enabled. Since this information is CPU-specific, it should be collected with migration disabled.Migration is disabled immediately after collecting this information anyway, so just move the information collection to after the migration disabling.
Signed-off-by John Ogness <[email protected]>
Re: [PATCH printk v1 04/13] printk: get caller_id/timestamp after migration disable
In reply to John OgnessQuoted 9 lines · John Ogness
Currently the local CPU timestamp and
caller_idfor the record are collected while migration is enabled. Since this information is CPU-specific, it should be collected with migration disabled.Migration is disabled immediately after collecting this information anyway, so just move the information collection to after the migration disabling.
Signed-off-by John Ogness <[email protected]>
Great catch!
Best Regards,
Petr
John Ogness
maintainerPATCH 5/13 printk: call boot_delay_msec() in printk_delay()
In reply to John Ognessboot_delay_msec() is always called immediately before printk_delay() so just call it from within printk_delay().
--- kernel/printk/printk.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/printk/printk.c +4 -3
@@ -2002,8 +2002,10 @@ static u8 *__printk_recursion_counter(void)int printk_delay_msec__read_mostly ; -static inline void printk_delay (void ) +static inline void printk_delay (int level) { +boot_delay_msec (level); +if (unlikely (printk_delay_msec)) {int m = printk_delay_msec;
@@ -2224,8 +2226,7 @@ asmlinkage int vprintk_emit(int facility, int level, in_sched = true; } -boot_delay_msec (level); -printk_delay (); +printk_delay (level); printed_len =vprintk_store (facility, level, dev_info, fmt, args);
Signature · 1 line
2.30.2
Re: [PATCH printk v1 05/13] printk: call boot_delay_msec() in printk_delay()
Quoted 4 lines · John Ogness
boot_delay_msec()is always called immediately beforeprintk_delay()so just call it from withinprintk_delay().Signed-off-by John Ogness <[email protected]>
Quoted 9 lines
kernel/printk/printk.c +3
-1-static inline void printk_delay (void ) +static inline void printk_delay (int level) { +boot_delay_msec (level); +if (unlikely (printk_delay_msec)) {int m = printk_delay_msec;
kernel/printk/printk.c asmlinkage int vprintk_emit(int facility, int level, · line 2229
Quoted 6 lines
kernel/printk/printk.c +1
-2@@ -2224,8 +2226,7 @@ asmlinkage int vprintk_emit(int facility, int level, in_sched = true; } -boot_delay_msec (level); -printk_delay (); +printk_delay (level);
As a side note, I wonder why do boot_delay_msec() and printk_delay() touch only NMI watchdog. Shouldn't they also touch softlockup, WQ, RCU, etc. watchdogs?
Petr Mladek
maintainerRe: [PATCH printk v1 05/13] printk: call boot_delay_msec() in printk_delay()
Quoted 20 lines · Sergey Senozhatsky
Quoted 4 lines · John Ogness
boot_delay_msec()is always called immediately beforeprintk_delay()so just call it from withinprintk_delay().Signed-off-by John Ogness <[email protected]>Reviewed-by Sergey Senozhatsky <[email protected]>Quoted 12 lines
kernel/printk/printk.c +3
-1-static inline void printk_delay (void ) +static inline void printk_delay (int level) { +boot_delay_msec (level); +if (unlikely (printk_delay_msec)) {int m = printk_delay_msec;@@ -2224,8 +2226,7 @@ asmlinkage int vprintk_emit(int facility, int level, in_sched = true; }
Quoted 7 lines
kernel/printk/printk.c asmlinkage int vprintk_emit(int facility, int level, · line 2229
Quoted 3 lines
kernel/printk/printk.c +1
-2-boot_delay_msec (level); -printk_delay (); +printk_delay (level);As a side note, I wonder why do
boot_delay_msec()andprintk_delay()touch only NMI watchdog. Shouldn't they also touch softlockup, WQ, RCU, etc. watchdogs?
It seems that touch_nmi_watchdog() actually calls touch_softlockup_watchdog() that calls wq_watchdog_touch().
Best Regards,
Petr
Sergey Senozhatsky
reviewerRe: [PATCH printk v1 05/13] printk: call boot_delay_msec() in printk_delay()
Quoted 20 lines · Petr Mladek
Quoted 12 lines
Quoted 12 lines
kernel/printk/printk.c +3
-1-static inline void printk_delay (void ) +static inline void printk_delay (int level) { +boot_delay_msec (level); +if (unlikely (printk_delay_msec)) {int m = printk_delay_msec;@@ -2224,8 +2226,7 @@ asmlinkage int vprintk_emit(int facility, int level, in_sched = true; }Quoted 7 lines
kernel/printk/printk.c asmlinkage int vprintk_emit(int facility, int level, · line 2229
Quoted 3 lines
kernel/printk/printk.c +1
-2-boot_delay_msec (level); -printk_delay (); +printk_delay (level);As a side note, I wonder why do
boot_delay_msec()andprintk_delay()touch only NMI watchdog. Shouldn't they also touch softlockup, WQ, RCU, etc. watchdogs?
Quoted 2 lines
It seems that
touch_nmi_watchdog()actually callstouch_softlockup_watchdog()that callswq_watchdog_touch().
Oh, you are right, good point.
Well, we still have RCU stall detector which can trigger (in theory).
Re: [PATCH printk v1 05/13] printk: call boot_delay_msec() in printk_delay()
In reply to John OgnessQuoted 4 lines · John Ogness
boot_delay_msec()is always called immediately beforeprintk_delay()so just call it from withinprintk_delay().Signed-off-by John Ogness <[email protected]>
Best Regards,
Petr
John Ogness
maintainerPATCH 6/13 printk: refactor and rework printing logic
In reply to John OgnessRefactor/rework printing logic in order to prepare for moving to threaded console printing.
- Move @console_seq into struct console so that the current "position" of each console can be tracked individually.
- Move @console_dropped into struct console so that the current drop count of each console can be tracked individually.
- Modify printing logic so that each console independently loads, prepares, prints, and delays its next record.
- Remove exclusive_console logic. Since console positions are handled independently, replaying past records occurs naturally.
--- include/linux/console.h | 2 + kernel/printk/printk.c | 388 ++++++++++++++++++++-------------------- 2 files changed, 198 insertions(+), 192 deletions(-)
include/linux/console.h +2 -0
@@ -151,6 +151,8 @@ struct console {int cflag;uint ispeed;uint ospeed; +u64 seq; +unsigned long dropped;void *data;struct console *next; };
kernel/printk/printk.c +196 -192
497 lines not on this page · raw message
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 06/13] printk: refactor and rework printing logic
Quoted 6 lines · John Ogness
Refactor/rework printing logic in order to prepare for moving to threaded console printing.
- Move @console_seq into struct console so that the current "position" of each console can be tracked individually.
Quoted 8 lines
- Move @console_dropped into struct console so that the current drop count of each console can be tracked individually.
- Modify printing logic so that each console independently loads, prepares, prints, and delays its next record.
- Remove
exclusive_consolelogic. Since console positions are handled independently, replaying past records occurs naturally.
It would be great to say if it has any behavior change.
There is one change caused by moving printk_delay(). I suggest to do it in a separate patch. See below for more details.
Another change is that console replaying the log (former exclusive console) does not longer block other consoles. New messages appear on other consoles while the newly added console is still replaying.
Otherwise it should not change the existing behavior.
Quoted 36 lines
kernel/printk/printk.c +9
-10@@ -2226,8 +2194,6 @@ asmlinkage int vprintk_emit(int facility, int level, in_sched = true; } -printk_delay (level); - printed_len =vprintk_store (facility, level, dev_info, fmt, args);/* If called from the scheduler, we can not call up(). */ @@ -2560,31 +2524,167 @@ int is_console_locked(void)EXPORT_SYMBOL (is_console_locked);/* -* Check if we have any console that is capable of printing while cpu is -* booting or shutting down. Requires console_sem. +* Check if the given console is currently capable and allowed to print +* records. +* +* Requires the console_lock. */ -static int have_callable_console (void ) +static inline bool console_is_usable (struct console *con) { -struct console *con; +if (!(con->flags &CON_ENABLED )) +return false; -for_each_console (con) -if ((con->flags &CON_ENABLED ) && - (con->flags &CON_ANYTIME )) -return 1 ; +if (!con->write) +return false;
kernel/printk/printk.c int is_console_locked(void) · line 2548
Quoted 10 lines
kernel/printk/printk.c +9
-1-return 0 ; +/* +* Console drivers may assume that per-cpu resources have been +* allocated. So unless they're explicitly marked as being able to +* cope (CON_ANYTIME) don't call them until per-cpu resources have +* been allocated. +*/ +if (!printk_percpu_data_ready () && + !(con->flags &CON_ANYTIME )) +return false;
Just for record. I am not completely sure about this check. It is being discussed in the 3rd patch.
Quoted 74 lines
kernel/printk/printk.c +74
-0+ +return true; +} + +static void __console_unlock (void ) +{ + console_locked =0 ; +up_console_sem (); +} + +/* +* Print one record for the given console. The record printed is whatever +* record is the next available record for the given console. +* +* Requires the console_lock. +* +* Returns false if the given console has no next record to print, otherwise +* true. +* +* @handover will be set to true if a printk waiter has taken over the +* console_lock, in which case the caller is no longer holding the +* console_lock. Otherwise it is set to false. +*/ +static bool console_emit_next_record (struct console *con,bool *handover) +{ +static char ext_text[CONSOLE_EXT_LOG_MAX ]; +static char text[CONSOLE_LOG_MAX ]; +struct printk_info info; +struct printk_record r; +unsigned long flags; +char *write_text; +size_t len; + +prb_rec_init_rd (&r, &info, text,sizeof (text)); + + *handover = false; + +if (!prb_read_valid (prb, con->seq, &r)) +return false; + +if (con->seq != r.info->seq) { + con->dropped += r.info->seq - con->seq; + con->seq = r.info->seq; + } + +/* Skip record that has level above the console loglevel. */ +if (suppress_message_printing (r.info->level)) { + con->seq++; + goto skip; + } + +if (con->flags &CON_EXTENDED ) { + write_text = &ext_text[0 ]; + len =info_print_ext_header (ext_text,sizeof (ext_text), r.info); + len +=msg_print_ext_body (ext_text + len,sizeof (ext_text) - len, + &r.text_buf[0 ], r.info->text_len, &r.info->dev_info); + }else { + write_text = &text[0 ]; + len =record_print_text (&r, console_msg_format &MSG_FORMAT_SYSLOG , printk_time); + } + +/* +* While actively printing out messages, if another printk() +* were to occur on another CPU, it may wait for this one to +* finish. This task can not be preempted if there is a +* waiter waiting to take over. +* +* Interrupts are disabled because the hand over to a waiter +* must not be interrupted until the hand over is completed +* (@console_waiter is cleared). +*/ +printk_safe_enter_irqsave (flags); +console_lock_spinning_enable (); +
kernel/printk/printk.c int is_console_locked(void) · line 2632
Quoted 10 lines
kernel/printk/printk.c +10
-0+stop_critical_timings ();/* don't trace print latency */ +call_console_driver (con, write_text, len); +start_critical_timings (); + + con->seq++; + + *handover =console_lock_spinning_disable_and_check (); +printk_safe_exit_irqrestore (flags); + +printk_delay (r.info->level);
This is the desired behavior when the messages are printed by kthreads. Though, it will delay the output more times when more consoles are registered and the messages are printed synchronously from console_unlock().
It is probably not super important. The delay is used only for debugging and people probably adjust it by a personal taste.
Anyway, we should not hide this behavior change in this hude patch. IMHO, it is perfectly fine to keep the delay in vprintk_emit() for now. We could move it into console_flush_all() and kthread loops in a separate patch later.
Quoted 54 lines
kernel/printk/printk.c +42
-6+skip: +return true; }/* -* Can we actually use the console at this time on this cpu? +* Print out all remaining records to all consoles. +* +* Requires the console_lock. +* +* Returns true if a console was available for flushing, otherwise false. * -* Console drivers may assume that per-cpu resources have been allocated. So -* unless they're explicitly marked as being able to cope (CON_ANYTIME) don't -* call them until per-cpu resources have been allocated. +* @next_seq is set to the highest sequence number of all of the consoles that +* were flushed. +* +* @handover will be set to true if a printk waiter has taken over the +* console_lock, in which case the caller is no longer holding the +* console_lock. Otherwise it is set to false. */ -static inline int can_use_console (void ) +static bool console_flush_all (bool do_cond_resched,u64 *next_seq,bool *handover) { -return (printk_percpu_data_ready () ||have_callable_console ()); +bool any_usable = false; +struct console *con; +bool any_progress; + + *next_seq =0 ; + *handover = false; + + do { + any_progress = false; + + for_each_console(con) { + bool progress; + + if (!console_is_usable(con)) + continue; + any_usable = true; + + progress = console_emit_next_record(con, handover); + if (*handover) + return true; + + /* Track the highest seq flushed. */ + if (con->seq > *next_seq) + *next_seq = con->seq; + + if (!progress) + continue; + any_progress = true;
kernel/printk/printk.c int is_console_locked(void) · line 2690
Quoted 10 lines
kernel/printk/printk.c +7
-0+ +if (do_cond_resched) +cond_resched (); + } + }while (any_progress); + +return any_usable; }/**
I do not see any other problem. And I like this approach.
console_flush_all() might be a bit complicated for people that see it for the first time. But console_unlock() complicated as well. From my POV, the new code looks better than the previous one.
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 06/13] printk: refactor and rework printing logic
Quoted 12 lines · Petr Mladek
Quoted 12 lines
Refactor/rework printing logic in order to prepare for moving to threaded console printing.
- Move @console_seq into struct console so that the current "position" of each console can be tracked individually.
- Move @console_dropped into struct console so that the current drop count of each console can be tracked individually.
- Modify printing logic so that each console independently loads, prepares, prints, and delays its next record.
Quoted 7 lines
Quoted 2 lines
- Remove
exclusive_consolelogic. Since console positions are handled independently, replaying past records occurs naturally.It would be great to say if it has any behavior change.
There is one change caused by moving
printk_delay(). I suggest to do it in a separate patch. See below for more details.
OK. I will do it in a separate patch.
Quoted 5 lines
Another change is that console replaying the log (former exclusive console) does not longer block other consoles. New messages appear on other consoles while the newly added console is still replaying.
Otherwise it should not change the existing behavior.
OK, I will mention behavior changes.
Quoted 27 lines
Quoted 27 lines
kernel/printk/printk.c +9
-8@@ -2560,31 +2524,167 @@ int is_console_locked(void)EXPORT_SYMBOL (is_console_locked);/* -* Check if we have any console that is capable of printing while cpu is -* booting or shutting down. Requires console_sem. +* Check if the given console is currently capable and allowed to print +* records. +* +* Requires the console_lock. */ -static int have_callable_console (void ) +static inline bool console_is_usable (struct console *con) { -struct console *con; +if (!(con->flags &CON_ENABLED )) +return false; -for_each_console (con) -if ((con->flags &CON_ENABLED ) && - (con->flags &CON_ANYTIME )) -return 1 ; +if (!con->write) +return false;
Quoted 13 lines
kernel/printk/printk.c int is_console_locked(void) · line 2548
Quoted 10 lines
kernel/printk/printk.c +9
-1-return 0 ; +/* +* Console drivers may assume that per-cpu resources have been +* allocated. So unless they're explicitly marked as being able to +* cope (CON_ANYTIME) don't call them until per-cpu resources have +* been allocated. +*/ +if (!printk_percpu_data_ready () && + !(con->flags &CON_ANYTIME )) +return false;Just for record. I am not completely sure about this check. It is being discussed in the 3rd patch.
Yes. In that discussion I mention that I will change it for v2.
Quoted 90 lines
Quoted 74 lines
kernel/printk/printk.c +74
-0+ +return true; +} + +static void __console_unlock (void ) +{ + console_locked =0 ; +up_console_sem (); +} + +/* +* Print one record for the given console. The record printed is whatever +* record is the next available record for the given console. +* +* Requires the console_lock. +* +* Returns false if the given console has no next record to print, otherwise +* true. +* +* @handover will be set to true if a printk waiter has taken over the +* console_lock, in which case the caller is no longer holding the +* console_lock. Otherwise it is set to false. +*/ +static bool console_emit_next_record (struct console *con,bool *handover) +{ +static char ext_text[CONSOLE_EXT_LOG_MAX ]; +static char text[CONSOLE_LOG_MAX ]; +struct printk_info info; +struct printk_record r; +unsigned long flags; +char *write_text; +size_t len; + +prb_rec_init_rd (&r, &info, text,sizeof (text)); + + *handover = false; + +if (!prb_read_valid (prb, con->seq, &r)) +return false; + +if (con->seq != r.info->seq) { + con->dropped += r.info->seq - con->seq; + con->seq = r.info->seq; + } + +/* Skip record that has level above the console loglevel. */ +if (suppress_message_printing (r.info->level)) { + con->seq++; + goto skip; + } + +if (con->flags &CON_EXTENDED ) { + write_text = &ext_text[0 ]; + len =info_print_ext_header (ext_text,sizeof (ext_text), r.info); + len +=msg_print_ext_body (ext_text + len,sizeof (ext_text) - len, + &r.text_buf[0 ], r.info->text_len, &r.info->dev_info); + }else { + write_text = &text[0 ]; + len =record_print_text (&r, console_msg_format &MSG_FORMAT_SYSLOG , printk_time); + } + +/* +* While actively printing out messages, if another printk() +* were to occur on another CPU, it may wait for this one to +* finish. This task can not be preempted if there is a +* waiter waiting to take over. +* +* Interrupts are disabled because the hand over to a waiter +* must not be interrupted until the hand over is completed +* (@console_waiter is cleared). +*/ +printk_safe_enter_irqsave (flags); +console_lock_spinning_enable (); +kernel/printk/printk.c int is_console_locked(void) · line 2632
Quoted 10 lines
kernel/printk/printk.c +10
-0+stop_critical_timings ();/* don't trace print latency */ +call_console_driver (con, write_text, len); +start_critical_timings (); + + con->seq++; + + *handover =console_lock_spinning_disable_and_check (); +printk_safe_exit_irqrestore (flags); + +printk_delay (r.info->level);This is the desired behavior when the messages are printed by kthreads. Though, it will delay the output more times when more consoles are registered and the messages are printed synchronously from
console_unlock().
Quoted 8 lines
It is probably not super important. The delay is used only for debugging and people probably adjust it by a personal taste.
Anyway, we should not hide this behavior change in this hude patch. IMHO, it is perfectly fine to keep the delay in
vprintk_emit()for now. We could move it intoconsole_flush_all()and kthread loops in a separate patch later.
OK, I will move it in a separate patch. And I will move it into the kthreads and console_flush_all() so that the delay is only once per successful output.
John
John Ogness
maintainerPATCH 7/13 printk: move buffer definitions into console_emit_next_record() caller
In reply to John OgnessExtended consoles print extended messages and do not print messages about dropped records.
Non-extended consoles print "normal" messages as well as extra messages about dropped records.
Currently the buffers for these various message types are defined within the functions that might use them and their usage is based upon the CON_EXTENDED flag. This will be a problem when moving to kthread printers because each printer must be able to provide its own buffers.
Move all the message buffer definitions outside of console_emit_next_record(). The caller knows if extended or dropped messages should be printed and can specify the appropriate buffers to use. The console_emit_next_record() and call_console_driver() functions can know what to print based on whether specified buffers are non-NULL.
With this change, buffer definition/allocation/specification is separated from the code that does the various types of string printing.
--- kernel/printk/printk.c | 60 ++++++++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 17 deletions(-)
kernel/printk/printk.c +43 -17
@@ -380,6 +380,9 @@ static struct latched_seq clear_seq = {/* the maximum size of a formatted record (i.e. with prefix added per line) */ #define CONSOLE_LOG_MAX 1024 +/* the maximum size for a dropped text message */ +#define DROPPED_TEXT_MAX 64 +/* the maximum size allowed to be reserved for a record */ #define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX)
@@ -1877,18 +1880,18 @@ static int console_trylock_spinning(void)/* * Call the specified console driver, asking it to write out the specified -* text and length. For non-extended consoles, if any records have been +* text and length. If @dropped_text is non-NULL and any records have been * dropped, a dropped message will be written out first. */ -static void call_console_driver (struct console *con,const char *text,size_t len) +static void call_console_driver (struct console *con,const char *text,size_t len, +char *dropped_text) { -static char dropped_text[64 ];size_t dropped_len;trace_console_rcuidle (text, len); -if (con->dropped && !(con->flags &CON_EXTENDED )) { - dropped_len =snprintf (dropped_text,sizeof (dropped_text), +if (con->dropped && dropped_text) { + dropped_len =snprintf (dropped_text,DROPPED_TEXT_MAX ,"** %lu printk messages dropped **\n" , con->dropped); con->dropped =0 ;
@@ -2241,6 +2244,7 @@ EXPORT_SYMBOL(_printk);#else /* CONFIG_PRINTK */ #define CONSOLE_LOG_MAX 0 +#define DROPPED_TEXT_MAX 0#define printk_time false#define prb_read_valid (rb, seq, r) false
@@ -2264,7 +2268,10 @@ static ssize_t msg_print_ext_body(char *buf, size_t size, struct dev_printk_info *dev_info) { return 0; }static void console_lock_spinning_enable (void ) { }static int console_lock_spinning_disable_and_check (void ) {return 0 ; } -static void call_console_driver (struct console *con,const char *text,size_t len) {} +static void call_console_driver (struct console *con,const char *text,size_t len, +char *dropped_text) +{ +}static bool suppress_message_printing (int level) {return false; }static void printk_delay (int level) {}
@@ -2560,6 +2567,14 @@ static void __console_unlock(void)* Print one record for the given console. The record printed is whatever * record is the next available record for the given console. * +* @text is a buffer of size CONSOLE_LOG_MAX. +* +* If extended messages should be printed, @ext_text is a buffer of size +* CONSOLE_EXT_LOG_MAX. Otherwise @ext_text must be NULL. +* +* If dropped messages should be printed, @dropped_text is a buffer of size +* DROPPED_TEXT_MAX. Otherwise @dropped_text must be NULL. +* * Requires the console_lock. * * Returns false if the given console has no next record to print, otherwise
@@ -2569,17 +2584,16 @@ static void __console_unlock(void)* console_lock, in which case the caller is no longer holding the * console_lock. Otherwise it is set to false. */ -static bool console_emit_next_record (struct console *con,bool *handover) +static bool console_emit_next_record (struct console *con,char *text,char *ext_text, +char *dropped_text,bool *handover) { -static char ext_text[CONSOLE_EXT_LOG_MAX ]; -static char text[CONSOLE_LOG_MAX ];struct printk_info info;struct printk_record r;unsigned long flags;char *write_text;size_t len; -prb_rec_init_rd (&r, &info, text,sizeof (text)); +prb_rec_init_rd (&r, &info, text,CONSOLE_LOG_MAX ); *handover = false;
@@ -2597,13 +2611,13 @@ static bool console_emit_next_record(struct console *con, bool *handover) goto skip; } -if (con->flags &CON_EXTENDED ) { - write_text = &ext_text[0 ]; - len =info_print_ext_header (ext_text,sizeof (ext_text), r.info); - len +=msg_print_ext_body (ext_text + len,sizeof (ext_text) - len, +if (ext_text) { + write_text = ext_text; + len =info_print_ext_header (ext_text,CONSOLE_EXT_LOG_MAX , r.info); + len +=msg_print_ext_body (ext_text + len,CONSOLE_EXT_LOG_MAX - len, &r.text_buf[0 ], r.info->text_len, &r.info->dev_info); }else { - write_text = &text[0 ]; + write_text = text; len =record_print_text (&r, console_msg_format &MSG_FORMAT_SYSLOG , printk_time); }
@@ -2621,7 +2635,7 @@ static bool console_emit_next_record(struct console *con, bool *handover)console_lock_spinning_enable ();stop_critical_timings ();/* don't trace print latency */ -call_console_driver (con, write_text, len); +call_console_driver (con, write_text, len, dropped_text);start_critical_timings (); con->seq++;
@@ -2650,6 +2664,9 @@ static bool console_emit_next_record(struct console *con, bool *handover)*/ static bool console_flush_all (bool do_cond_resched,u64 *next_seq,bool *handover) { +static char dropped_text[DROPPED_TEXT_MAX ]; +static char ext_text[CONSOLE_EXT_LOG_MAX ]; +static char text[CONSOLE_LOG_MAX ];bool any_usable = false;struct console *con;bool any_progress;
@@ -2667,7 +2684,16 @@ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handovecontinue ; any_usable = true; - progress =console_emit_next_record (con, handover); +if (con->flags &CON_EXTENDED ) { +/* Extended consoles do not print "dropped messages". */ + progress =console_emit_next_record (con, &text[0 ], + &ext_text[0 ],NULL , + handover); + }else { + progress =console_emit_next_record (con, &text[0 ], +NULL , &dropped_text[0 ], + handover); + }if (*handover)return true;
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 07/13] printk: move buffer definitions into console_emit_next_record() caller
Quoted 31 lines · John Ogness
Extended consoles print extended messages and do not print messages about dropped records.
Non-extended consoles print "normal" messages as well as extra messages about dropped records.
Currently the buffers for these various message types are defined within the functions that might use them and their usage is based upon the
CON_EXTENDEDflag. This will be a problem when moving to kthread printers because each printer must be able to provide its own buffers.Move all the message buffer definitions outside of
console_emit_next_record(). The caller knows if extended or dropped messages should be printed and can specify the appropriate buffers to use. Theconsole_emit_next_record()andcall_console_driver()functions can know what to print based on whether specified buffers are non-NULL.With this change, buffer definition/allocation/specification is separated from the code that does the various types of string printing.
kernel/printk/printk.c +0
-3@@ -2597,13 +2611,13 @@ static bool console_emit_next_record(struct console *con, bool *handover) goto skip; } -if (con->flags &CON_EXTENDED ) { - write_text = &ext_text[0 ]; - len =info_print_ext_header (ext_text,sizeof (ext_text), r.info);
kernel/printk/printk.c static bool console_emit_next_record(struct console *con, bool *handover) · line 2621
Quoted 10 lines
kernel/printk/printk.c +5
-2- len +=msg_print_ext_body (ext_text + len,sizeof (ext_text) - len, +if (ext_text) { + write_text = ext_text; + len =info_print_ext_header (ext_text,CONSOLE_EXT_LOG_MAX , r.info); + len +=msg_print_ext_body (ext_text + len,CONSOLE_EXT_LOG_MAX - len, &r.text_buf[0 ], r.info->text_len, &r.info->dev_info); }else { - write_text = &text[0 ]; + write_text = text; len =record_print_text (&r, console_msg_format &MSG_FORMAT_SYSLOG , printk_time);
@text and @ext_text buffers are never used at the same time. It might be enough to use a single text[CONSOLE_EXT_LOG_MAX] buffer. It would even slightly simplify the code.
Quoted 12 lines
kernel/printk/printk.c +1
-1}@@ -2621,7 +2635,7 @@ static bool console_emit_next_record(struct console *con, bool *handover)console_lock_spinning_enable ();stop_critical_timings ();/* don't trace print latency */ -call_console_driver (con, write_text, len); +call_console_driver (con, write_text, len, dropped_text);start_critical_timings (); con->seq++;
kernel/printk/printk.c static bool console_emit_next_record(struct console *con, bool *handover) · line 2669
Quoted 6 lines
kernel/printk/printk.c +3
-0@@ -2650,6 +2664,9 @@ static bool console_emit_next_record(struct console *con, bool *handover)*/ static bool console_flush_all (bool do_cond_resched,u64 *next_seq,bool *handover) { +static char dropped_text[DROPPED_TEXT_MAX ]; +static char ext_text[CONSOLE_EXT_LOG_MAX ]; +static char text[CONSOLE_LOG_MAX ];
These buffers are for printing from console_unlock(). The same buffers will need to be allocated for each console in the kthreads.
It might make sense to allocate these buffers in register_console() and store the pointers in struct console.
Well, we might need extra buffers for atomic console drivers and diffent contexts that would be used during panic. But maybe they can be allocated in register_console() as well.
kernel/printk/printk.c static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handove · line 2689
Quoted 11 lines
kernel/printk/printk.c +3
-1bool any_usable = false;struct console *con;bool any_progress;@@ -2667,7 +2684,16 @@ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handovecontinue ; any_usable = true; - progress =console_emit_next_record (con, handover); +if (con->flags &CON_EXTENDED ) { +/* Extended consoles do not print "dropped messages". */ + progress = console_emit_next_record(con, &text[0],
IMHO, &text[0] buffer is not used for extended consoles.
kernel/printk/printk.c static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handove · line 2698
Quoted 9 lines
kernel/printk/printk.c +7
-0+ &ext_text[0 ],NULL , + handover); + }else { + progress =console_emit_next_record (con, &text[0 ], +NULL , &dropped_text[0 ], + handover); + }if (*handover)return true;
I do not resist on allocating the buffers in register_console(). I am not sure if it would really makes things easier.
But I would really like to use the same buffer for normal and extended consoles if possible.
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 07/13] printk: move buffer definitions into console_emit_next_record() caller
Quoted 17 lines · Petr Mladek
Quoted 17 lines
kernel/printk/printk.c +4
-4@@ -2597,13 +2611,13 @@ static bool console_emit_next_record(struct console *con, bool *handover) goto skip; } -if (con->flags &CON_EXTENDED ) { - write_text = &ext_text[0 ]; - len =info_print_ext_header (ext_text,sizeof (ext_text), r.info); - len +=msg_print_ext_body (ext_text + len,sizeof (ext_text) - len, +if (ext_text) { + write_text = ext_text; + len =info_print_ext_header (ext_text,CONSOLE_EXT_LOG_MAX , r.info); + len +=msg_print_ext_body (ext_text + len,CONSOLE_EXT_LOG_MAX - len, &r.text_buf[0 ], r.info->text_len, &r.info->dev_info);
Quoted 8 lines
kernel/printk/printk.c static bool console_emit_next_record(struct console *con, bool *handover) · line 2621
Quoted 4 lines
kernel/printk/printk.c +1
-1}else { - write_text = &text[0 ]; + write_text = text; len =record_print_text (&r, console_msg_format &MSG_FORMAT_SYSLOG , printk_time);@text and @ext_text buffers are never used at the same time. It might be enough to use a single text[CONSOLE_EXT_LOG_MAX] buffer. It would even slightly simplify the code.
No, they _are_ used at the same time.
r.text_buf is @text. msg_print_ext_body() takes @ext_text and &r.text_buf[0]. Unfortunately msg_print_ext_body() does not work "in place" like record_print_text() does.
Quoted 11 lines
kernel/printk/printk.c static bool console_emit_next_record(struct console *con, bool *handover) · line 2669
Quoted 7 lines
kernel/printk/printk.c +3
-0@@ -2650,6 +2664,9 @@ static bool console_emit_next_record(struct console *con, bool *handover)*/ static bool console_flush_all (bool do_cond_resched,u64 *next_seq,bool *handover) { +static char dropped_text[DROPPED_TEXT_MAX ]; +static char ext_text[CONSOLE_EXT_LOG_MAX ]; +static char text[CONSOLE_LOG_MAX ];These buffers are for printing from
console_unlock(). The same buffers will need to be allocated for each console in the kthreads.
Quoted 6 lines
It might make sense to allocate these buffers in
register_console()and store the pointers in struct console.Well, we might need extra buffers for atomic console drivers and diffent contexts that would be used during panic. But maybe they can be allocated in
register_console()as well.
register_console() happens quite early. But my plan for v2 is to make them global static variables and allocate them on the first register_console().
Quoted 7 lines
Quoted 7 lines
kernel/printk/printk.c +0
-0bool any_usable = false;struct console *con;bool any_progress;@@ -2667,7 +2684,16 @@ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handovecontinue ; any_usable = true;
Quoted 6 lines
kernel/printk/printk.c static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handove · line 2689
Quoted 4 lines
kernel/printk/printk.c +3
-1- progress =console_emit_next_record (con, handover); +if (con->flags &CON_EXTENDED ) { +/* Extended consoles do not print "dropped messages". */ + progress = console_emit_next_record(con, &text[0],IMHO, &text[0] buffer is not used for extended consoles.
Yes. msg_print_ext_body() needs it.
Quoted 4 lines
Quoted 4 lines
kernel/printk/printk.c +4
-0+ &ext_text[0 ],NULL , + handover); + }else { + progress = console_emit_next_record(con, &text[0],
Quoted 8 lines
kernel/printk/printk.c static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handove · line 2698
Quoted 5 lines
kernel/printk/printk.c +3
-0+NULL , &dropped_text[0 ], + handover); + }if (*handover)return true;I do not resist on allocating the buffers in
register_console(). I am not sure if it would really makes things easier.
I'll give it a try for v2.
John
John Ogness
maintainerPATCH 8/13 printk: add pr_flush()
In reply to John OgnessProvide a might-sleep function to allow waiting for threaded console printers to catch up to the latest logged message.
Use pr_flush() whenever it is desirable to get buffered messages printed before continuing: suspend_console(), resume_console(), console_stop(), console_start(), console_unblank().
--- include/linux/printk.h | 7 ++++ kernel/printk/printk.c | 73 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 79 insertions(+), 1 deletion(-)
include/linux/printk.h +7 -0
@@ -170,6 +170,8 @@ extern void __printk_safe_exit(void);#define printk_deferred_enter __printk_safe_enter#define printk_deferred_exit __printk_safe_exit +extern bool pr_flush (int timeout_ms,bool reset_on_progress); +/* * Please don't use printk_ratelimit(), because it shares ratelimiting state * with all other unrelated printk_ratelimit() callsites. Instead use
@@ -224,6 +226,11 @@ static inline void printk_deferred_exit(void) { } +static inline bool pr_flush (int timeout_ms,bool reset_on_progress) +{ +return true; +} +static inline int printk_ratelimit (void ) {return 0 ;
kernel/printk/printk.c +72 -1
110 lines not on this page · raw message
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 08/13] printk: add pr_flush()
Quoted 28 lines · John Ogness
Provide a might-sleep function to allow waiting for threaded console printers to catch up to the latest logged message.
Use
pr_flush()whenever it is desirable to get buffered messages printed before continuing:suspend_console(),resume_console(),console_stop(),console_start(),console_unblank().kernel/printk/printk.c +2
-0@@ -2449,6 +2449,7 @@ void suspend_console(void)if (!console_suspend_enabled)return ;pr_info ("Suspending console(s) (use no_console_suspend to debug)\n" ); +pr_flush (1000 , true);console_lock (); console_suspended =1 ;up_console_sem ();@@ -2461,6 +2462,7 @@ void resume_console(void)down_console_sem (); console_suspended =0 ;console_unlock (); +pr_flush (1000 , true); }/**
kernel/printk/printk.c void console_unblank(void) · line 2808
Quoted 6 lines
kernel/printk/printk.c +2
-1@@ -2802,8 +2804,10 @@ void console_unblank(void)if (oops_in_progress) {if (down_trylock_console_sem () !=0 )return ; - }else + }else { +pr_flush (1000 , true);
It would make more sense to flush the consoles after they are unblanked. I mean to move this to the end of the function.
Also it is not obvious why this is not called when oops_in_progress is set. I guess that it is because trylock is needed in this case. It should be handled inside pr_flush().
I mean that pr_flush() should internally use trylock when @oops_in_progress is set. It will make it safe even in this mode.
kernel/printk/printk.c struct tty_driver *console_device(int *index) · line 2876
Quoted 10 lines
kernel/printk/printk.c +2
-0console_lock (); + } console_locked =1 ; console_may_schedule =0 ;@@ -2869,6 +2873,7 @@ struct tty_driver *console_device(int *index)*/ void console_stop (struct console *console) { +pr_flush (1000 , true);
It would be enough to flush just the given @console.
It might be possible to take over the job from the related kthread and flush it in this context. Well, I am not sure if it is a good idea.
kernel/printk/printk.c void console_start(struct console *console) · line 2888
Quoted 8 lines
kernel/printk/printk.c +1
-0console_lock (); console->flags &= ~CON_ENABLED ;console_unlock ();@@ -2880,6 +2885,7 @@ void console_start(struct console *console)console_lock (); console->flags |=CON_ENABLED ;console_unlock (); +pr_flush (1000 , true);
Same here.
kernel/printk/printk.c static int __init printk_late_init(void) · line 3260
Quoted 10 lines
kernel/printk/printk.c +3
-0}EXPORT_SYMBOL (console_start);@@ -3249,6 +3255,71 @@ static int __init printk_late_init(void)late_initcall (printk_late_init);#if definedCONFIG_PRINTK +/** +* pr_flush() - Wait for printing threads to catch up. +*
Alternative solution would be to take over the job from the kthreads and flush the consoles in this context. Well, I am not sure if it is a good idea or not.
Quoted 51 lines
kernel/printk/printk.c +51
-0+* @timeout_ms: The maximum time (in ms) to wait. +* @reset_on_progress: Reset the timeout if forward progress is seen. +* +* A value of 0 for @timeout_ms means no waiting will occur. A value of -1 +* represents infinite waiting. +* +* If @reset_on_progress is true, the timeout will be reset whenever any +* printer has been seen to make some forward progress. +* +* Context: Process context. May sleep while acquiring console lock. +* Return: true if all enabled printers are caught up. +*/ +bool pr_flush (int timeout_ms,bool reset_on_progress) +{ +int remaining = timeout_ms; +struct console *con; +u64 last_diff =0 ; +u64 printk_seq; +u64 diff; +u64 seq; + +might_sleep (); + + seq =prb_next_seq (prb); + +for (;;) { + diff =0 ; + +console_lock (); +for_each_console (con) { +if (!console_is_usable (con)) +continue ; + printk_seq = con->seq; +if (printk_seq < seq) + diff += seq - printk_seq; + } +console_unlock (); + +if (diff != last_diff && reset_on_progress) + remaining = timeout_ms; + +if (diff ==0 || remaining ==0 ) +break ; + +if (remaining <0 ) { +/* no timeout limit */ +msleep (100 ); + }else if (remaining <100 ) { +msleep (remaining); + remaining =0 ; + }else {
kernel/printk/printk.c static int __init printk_late_init(void) · line 3321
Quoted 10 lines
kernel/printk/printk.c +10
-0+msleep (100 ); + remaining -=100 ; + } + + last_diff = diff; + } + +return (diff ==0 ); +} +EXPORT_SYMBOL (pr_flush);
Summary:
The pr_flush() API and the optional timeout look reasonable to me.
Please, handle oops_in_progress in pr_flush() and make it safe in this mode. It will allow to move it at the end of console_unblank() where it makes more sense.
I do not resist on flushing only the given consoles in console_stop() and console_start(). It is nice to have and can be done later.
Also I do not resist on doing the flush in the context of the caller. I am not even sure if it is a good idea. We could play with it later when there are some problems with the current approach in practice.
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 08/13] printk: add pr_flush()
Quoted 6 lines · Petr Mladek
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -2802,8 +2804,10 @@ void console_unblank(void)if (oops_in_progress) {
Quoted 8 lines
kernel/printk/printk.c void console_unblank(void) · line 2808
Quoted 5 lines
kernel/printk/printk.c +2
-1if (down_trylock_console_sem () !=0 )return ; - }else + }else { +pr_flush (1000 , true);It would make more sense to flush the consoles after they are unblanked. I mean to move this to the end of the function.
Agreed.
Quoted 7 lines
Also it is not obvious why this is not called when
oops_in_progressis set. I guess that it is because trylock is needed in this case. It should be handled insidepr_flush().I mean that
pr_flush()should internally use trylock when @oops_in_progress is set. It will make it safe even in this mode.
pr_flush() is a might_sleep() function. We agreed on this at LPC2019.
Creating a pr_flush() that will directly push out the messages (or busy-wait) in non-preemptible contexts is complicated. It might be something to attempt for the future, but I would prefer to avoid it at this stage.
Quoted 12 lines
kernel/printk/printk.c struct tty_driver *console_device(int *index) · line 2876
Quoted 10 lines
kernel/printk/printk.c +2
-0console_lock (); + } console_locked =1 ; console_may_schedule =0 ;@@ -2869,6 +2873,7 @@ struct tty_driver *console_device(int *index)*/ void console_stop (struct console *console) { +pr_flush (1000 , true);It would be enough to flush just the given @console.
For v2 I will create an internal __pr_flush() to allow specifying that only a single console is flushed. The high level pr_flush() will then call __pr_flush() specifying all consoles.
Quoted 3 lines
It might be possible to take over the job from the related kthread and flush it in this context. Well, I am not sure if it is a good idea.
I agree that it might not be a good idea. Let's keep things simple for now.
Quoted 10 lines
kernel/printk/printk.c void console_start(struct console *console) · line 2888
Quoted 8 lines
kernel/printk/printk.c +1
-0console_lock (); console->flags &= ~CON_ENABLED ;console_unlock ();@@ -2880,6 +2885,7 @@ void console_start(struct console *console)console_lock (); console->flags |=CON_ENABLED ;console_unlock (); +pr_flush (1000 , true);Same here.
OK.
Quoted 6 lines
Quoted 6 lines
kernel/printk/printk.c +0
-0}EXPORT_SYMBOL (console_start);@@ -3249,6 +3255,71 @@ static int __init printk_late_init(void)late_initcall (printk_late_init);
Quoted 8 lines
kernel/printk/printk.c static int __init printk_late_init(void) · line 3260
Quoted 4 lines
kernel/printk/printk.c +3
-0#if definedCONFIG_PRINTK +/** +* pr_flush() - Wait for printing threads to catch up. +* Alternative solution would be to take over the job from the kthreads and flush the consoles in this context. Well, I am not sure if it is a good idea or not.
Since pr_flush() is might_sleep() this would be relatively simple. Just grab the console mutex and go. My concern is that this task may have different scheduling parameters that could negatively affect the system. For normal operation, I really would prefer that the designated kthreads do the work. If "waiting for the kthreads" turns out to be problematic, then maybe we could go down this path.
Quoted 64 lines
Quoted 51 lines
kernel/printk/printk.c +51
-0+* @timeout_ms: The maximum time (in ms) to wait. +* @reset_on_progress: Reset the timeout if forward progress is seen. +* +* A value of 0 for @timeout_ms means no waiting will occur. A value of -1 +* represents infinite waiting. +* +* If @reset_on_progress is true, the timeout will be reset whenever any +* printer has been seen to make some forward progress. +* +* Context: Process context. May sleep while acquiring console lock. +* Return: true if all enabled printers are caught up. +*/ +bool pr_flush (int timeout_ms,bool reset_on_progress) +{ +int remaining = timeout_ms; +struct console *con; +u64 last_diff =0 ; +u64 printk_seq; +u64 diff; +u64 seq; + +might_sleep (); + + seq =prb_next_seq (prb); + +for (;;) { + diff =0 ; + +console_lock (); +for_each_console (con) { +if (!console_is_usable (con)) +continue ; + printk_seq = con->seq; +if (printk_seq < seq) + diff += seq - printk_seq; + } +console_unlock (); + +if (diff != last_diff && reset_on_progress) + remaining = timeout_ms; + +if (diff ==0 || remaining ==0 ) +break ; + +if (remaining <0 ) { +/* no timeout limit */ +msleep (100 ); + }else if (remaining <100 ) { +msleep (remaining); + remaining =0 ; + }else {kernel/printk/printk.c static int __init printk_late_init(void) · line 3321
Quoted 10 lines
kernel/printk/printk.c +10
-0+msleep (100 ); + remaining -=100 ; + } + + last_diff = diff; + } + +return (diff ==0 ); +} +EXPORT_SYMBOL (pr_flush);Summary:
Quoted 5 lines
The
pr_flush()API and the optional timeout look reasonable to me.Please, handle
oops_in_progressinpr_flush()and make it safe in this mode. It will allow to move it at the end ofconsole_unblank()where it makes more sense.
I will add another oops_in_progress check at the end. pr_flush() will not be made safe for oops_in_progress. Keep in mind that when oops_in_progress is set, direct printing will be active, so there should be no need for pr_flush() anyway.
Quoted 2 lines
I do not resist on flushing only the given consoles in
console_stop()andconsole_start(). It is nice to have and can be done later.
It is a simple change. I will do it for v2.
Quoted 4 lines
Also I do not resist on doing the flush in the context of the caller. I am not even sure if it is a good idea. We could play with it later when there are some problems with the current approach in practice.
For now, let's keep it as a waiting function.
John
Petr Mladek
maintainerRe: [PATCH printk v1 08/13] printk: add pr_flush()
Quoted 22 lines · John Ogness
Quoted 6 lines · Petr Mladek
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -2802,8 +2804,10 @@ void console_unblank(void)if (oops_in_progress) {Quoted 8 lines
kernel/printk/printk.c void console_unblank(void) · line 2808
Quoted 5 lines
kernel/printk/printk.c +2
-1if (down_trylock_console_sem () !=0 )return ; - }else + }else { +pr_flush (1000 , true);It would make more sense to flush the consoles after they are unblanked. I mean to move this to the end of the function.
Agreed.
Quoted 4 lines
Also it is not obvious why this is not called when
oops_in_progressis set. I guess that it is because trylock is needed in this case. It should be handled insidepr_flush().
Quoted 6 lines
Quoted 3 lines
I mean that
pr_flush()should internally use trylock when @oops_in_progress is set. It will make it safe even in this mode.
pr_flush()is amight_sleep()function. We agreed on this at LPC2019.
I see.
Quoted 4 lines
Creating a
pr_flush()that will directly push out the messages (or busy-wait) in non-preemptible contexts is complicated. It might be something to attempt for the future, but I would prefer to avoid it at this stage.
Sure. It is not needed now, definitely.
Quoted 13 lines
Quoted 12 lines
kernel/printk/printk.c struct tty_driver *console_device(int *index) · line 2876
Quoted 10 lines
kernel/printk/printk.c +2
-0console_lock (); + } console_locked =1 ; console_may_schedule =0 ;@@ -2869,6 +2873,7 @@ struct tty_driver *console_device(int *index)*/ void console_stop (struct console *console) { +pr_flush (1000 , true);It would be enough to flush just the given @console.
Quoted 3 lines
For v2 I will create an internal
__pr_flush()to allow specifying that only a single console is flushed. The high levelpr_flush()will then call__pr_flush()specifying all consoles.
Feel free to keep it as is when it gets to complicated. We could always optimize it later.
Quoted 6 lines
Quoted 3 lines
It might be possible to take over the job from the related kthread and flush it in this context. Well, I am not sure if it is a good idea.
I agree that it might not be a good idea. Let's keep things simple for now.
I agree.
Quoted 28 lines
Quoted 10 lines
kernel/printk/printk.c void console_start(struct console *console) · line 2888
Quoted 8 lines
kernel/printk/printk.c +1
-0console_lock (); console->flags &= ~CON_ENABLED ;console_unlock ();@@ -2880,6 +2885,7 @@ void console_start(struct console *console)console_lock (); console->flags |=CON_ENABLED ;console_unlock (); +pr_flush (1000 , true);Same here.
OK.
Quoted 6 lines
Quoted 6 lines
kernel/printk/printk.c +0
-0}EXPORT_SYMBOL (console_start);@@ -3249,6 +3255,71 @@ static int __init printk_late_init(void)late_initcall (printk_late_init);Quoted 8 lines
kernel/printk/printk.c static int __init printk_late_init(void) · line 3260
Quoted 4 lines
kernel/printk/printk.c +3
-0#if definedCONFIG_PRINTK +/** +* pr_flush() - Wait for printing threads to catch up. +* Alternative solution would be to take over the job from the kthreads and flush the consoles in this context. Well, I am not sure if it is a good idea or not.
Quoted 6 lines
Since
pr_flush()ismight_sleep()this would be relatively simple. Just grab the console mutex and go. My concern is that this task may have different scheduling parameters that could negatively affect the system. For normal operation, I really would prefer that the designated kthreads do the work. If "waiting for the kthreads" turns out to be problematic, then maybe we could go down this path.
The different scheduling parameters were actually my concern. I though about a high priority task waiting for normal priority kthreads.
Anyway, let's keep it simple now. Only practice will show if a more complex solution is needed.
Quoted 66 lines
Quoted 66 lines
Quoted 51 lines
kernel/printk/printk.c +51
-0+* @timeout_ms: The maximum time (in ms) to wait. +* @reset_on_progress: Reset the timeout if forward progress is seen. +* +* A value of 0 for @timeout_ms means no waiting will occur. A value of -1 +* represents infinite waiting. +* +* If @reset_on_progress is true, the timeout will be reset whenever any +* printer has been seen to make some forward progress. +* +* Context: Process context. May sleep while acquiring console lock. +* Return: true if all enabled printers are caught up. +*/ +bool pr_flush (int timeout_ms,bool reset_on_progress) +{ +int remaining = timeout_ms; +struct console *con; +u64 last_diff =0 ; +u64 printk_seq; +u64 diff; +u64 seq; + +might_sleep (); + + seq =prb_next_seq (prb); + +for (;;) { + diff =0 ; + +console_lock (); +for_each_console (con) { +if (!console_is_usable (con)) +continue ; + printk_seq = con->seq; +if (printk_seq < seq) + diff += seq - printk_seq; + } +console_unlock (); + +if (diff != last_diff && reset_on_progress) + remaining = timeout_ms; + +if (diff ==0 || remaining ==0 ) +break ; + +if (remaining <0 ) { +/* no timeout limit */ +msleep (100 ); + }else if (remaining <100 ) { +msleep (remaining); + remaining =0 ; + }else {kernel/printk/printk.c static int __init printk_late_init(void) · line 3321
Quoted 10 lines
kernel/printk/printk.c +10
-0+msleep (100 ); + remaining -=100 ; + } + + last_diff = diff; + } + +return (diff ==0 ); +} +EXPORT_SYMBOL (pr_flush);Summary:
The
pr_flush()API and the optional timeout look reasonable to me.
Quoted 8 lines
Quoted 3 lines
Please, handle
oops_in_progressinpr_flush()and make it safe in this mode. It will allow to move it at the end ofconsole_unblank()where it makes more sense.I will add another
oops_in_progresscheck at the end.pr_flush()will not be made safe foroops_in_progress. Keep in mind that whenoops_in_progressis set, direct printing will be active, so there should be no need forpr_flush()anyway.
Sounds good.
Best Regards,
Petr
John Ogness
maintainerPATCH 9/13 printk: add functions to allow direct printing
In reply to John OgnessOnce kthread printing is introduced, console printing will no longer occur in the context of the printk caller. However, there are some special contexts where it is desirable for the printk caller to directly print out kernel messages. Using pr_flush() to wait for threaded printers is only possible if the caller is in a sleepable context and the kthreads are active. That is not always the case.
Introduce printk_direct_enter() and printk_direct_exit() functions to explicitly (and globally) activate/deactivate direct console printing.
Activate direct printing for:
- sysrq
- emergency reboot/shutdown
- cpu and rcu stalls
- hard and soft lockups
- hung tasks
- stack dumps
--- drivers/tty/sysrq.c | 2 ++ include/linux/printk.h | 11 +++++++++++ kernel/hung_task.c | 11 ++++++++++- kernel/printk/printk.c | 39 ++++++++++++++++++++++++++++++++++++++- kernel/rcu/tree_stall.h | 2 ++ kernel/reboot.c | 14 +++++++++++++- kernel/watchdog.c | 4 ++++ kernel/watchdog_hld.c | 4 ++++ lib/dump_stack.c | 2 ++ lib/nmi_backtrace.c | 2 ++ 10 files changed, 88 insertions(+), 3 deletions(-)
drivers/tty/sysrq.c +2 -0
@@ -594,9 +594,11 @@ void __handle_sysrq(int key, bool check_mask)* should not) and is the invoked operation enabled? */ if (!check_mask ||sysrq_on_mask (op_p->enable_mask)) { +printk_direct_enter ();pr_info ("%s\n" , op_p->action_msg); console_loglevel = orig_log_level; op_p->handler (key); +printk_direct_exit (); }else {pr_info ("This sysrq operation is disabled.\n" ); console_loglevel = orig_log_level;
include/linux/printk.h +11 -0
@@ -170,6 +170,9 @@ extern void __printk_safe_exit(void);#define printk_deferred_enter __printk_safe_enter#define printk_deferred_exit __printk_safe_exit +extern void printk_direct_enter (void ); +extern void printk_direct_exit (void ); +extern bool pr_flush (int timeout_ms,bool reset_on_progress);/*
@@ -226,6 +229,14 @@ static inline void printk_deferred_exit(void) { } +static inline void printk_direct_enter (void ) +{ +} + +static inline void printk_direct_exit (void ) +{ +} +static inline bool pr_flush (int timeout_ms,bool reset_on_progress) {return true;
kernel/hung_task.c +10 -1
@@ -125,6 +125,8 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout)* complain: */ if (sysctl_hung_task_warnings) { +printk_direct_enter (); +if (sysctl_hung_task_warnings >0 ) sysctl_hung_task_warnings--; pr_err("INFO: task %s:%d blocked for more than %ld seconds.\n",
@@ -140,6 +142,8 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout)if (sysctl_hung_task_all_cpu_backtrace) hung_task_show_all_bt = true; + +printk_direct_exit (); }touch_nmi_watchdog ();
@@ -202,12 +206,17 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout) } unlock:rcu_read_unlock (); -if (hung_task_show_lock) +if (hung_task_show_lock) { +printk_direct_enter ();debug_show_all_locks (); +printk_direct_exit (); + }if (hung_task_show_all_bt) { hung_task_show_all_bt = false; +printk_direct_enter ();trigger_all_cpu_backtrace (); +printk_direct_exit (); }if (hung_task_call_panic)
kernel/printk/printk.c +38 -1
@@ -348,6 +348,31 @@ static int console_msg_format = MSG_FORMAT_DEFAULT;static DEFINE_MUTEX (syslog_lock);#ifdef CONFIG_PRINTK +static atomic_t printk_direct =ATOMIC_INIT (0 ); + +/** +* printk_direct_enter - cause console printing to occur in the context of +* printk() callers +* +* This globally effects all printk() callers. +* +* Context: Any context. +*/ +void printk_direct_enter (void ) +{ +atomic_inc (&printk_direct); +} + +/** +* printk_direct_exit - restore console printing behavior from direct +* +* Context: Any context. +*/ +void printk_direct_exit (void ) +{ +atomic_dec (&printk_direct); +} +DECLARE_WAIT_QUEUE_HEAD (log_wait);/* All 3 protected by @syslog_lock. */ /* the next printk record to read by syslog(READ) or /proc/kmsg */
@@ -3325,6 +3350,7 @@ EXPORT_SYMBOL(pr_flush);*/ #define PRINTK_PENDING_WAKEUP 0x01#define PRINTK_PENDING_OUTPUT 0x02 +#define PRINTK_DIRECT_OUTPUT 0x04 static DEFINE_PER_CPU(int, printk_pending);
@@ -3333,9 +3359,15 @@ static void wake_up_klogd_work_func(struct irq_work *irq_work)int pending =__this_cpu_xchg (printk_pending,0 );if (pending &PRINTK_PENDING_OUTPUT ) { +if (pending &PRINTK_DIRECT_OUTPUT ) +printk_direct_enter (); +/* If trylock fails, someone else is doing the printing */ if (console_trylock ())console_unlock (); + +if (pending &PRINTK_DIRECT_OUTPUT ) +printk_direct_exit (); }if (pending &PRINTK_PENDING_WAKEUP )
@@ -3360,11 +3392,16 @@ void wake_up_klogd(void)void defer_console_output (void ) { +int val =PRINTK_PENDING_OUTPUT ; +if (!printk_percpu_data_ready ())return ; +if (atomic_read (&printk_direct)) + val |=PRINTK_DIRECT_OUTPUT ; +preempt_disable (); -__this_cpu_or (printk_pending,PRINTK_PENDING_OUTPUT ); +__this_cpu_or (printk_pending, val);irq_work_queue (this_cpu_ptr (&wake_up_klogd_work));preempt_enable (); }
kernel/rcu/tree_stall.h +2 -0
@@ -610,6 +610,7 @@ static void print_cpu_stall(unsigned long gps)* See Documentation/RCU/stallwarn.rst for info on how to debug * RCU CPU stall warnings. */ +printk_direct_enter ();trace_rcu_stall_warning (rcu_state.name,TPS ("SelfDetected" ));pr_err ("INFO: %s self-detected stall on CPU\n" , rcu_state.name);raw_spin_lock_irqsave_rcu_node (rdp->mynode, flags);
@@ -644,6 +645,7 @@ static void print_cpu_stall(unsigned long gps)*/ set_tsk_need_resched (current);set_preempt_need_resched (); +printk_direct_exit (); }static void check_cpu_stall (struct rcu_data *rdp)
kernel/reboot.c +13 -1
@@ -447,9 +447,11 @@ static int __orderly_reboot(void) ret =run_cmd (reboot_cmd);if (ret) { +printk_direct_enter ();pr_warn ("Failed to start orderly reboot: forcing the issue\n" );emergency_sync ();kernel_restart (NULL ); +printk_direct_exit (); }return ret;
@@ -462,6 +464,7 @@ static int __orderly_poweroff(bool force) ret =run_cmd (poweroff_cmd);if (ret && force) { +printk_direct_enter ();pr_warn ("Failed to start orderly shutdown: forcing the issue\n" );/*
@@ -471,6 +474,7 @@ static int __orderly_poweroff(bool force)*/ emergency_sync ();kernel_power_off (); +printk_direct_exit (); }return ret;
@@ -528,6 +532,8 @@ EXPORT_SYMBOL_GPL(orderly_reboot);*/ static void hw_failure_emergency_poweroff_func (struct work_struct *work) { +printk_direct_enter (); +/* * We have reached here after the emergency shutdown waiting period has * expired. This means orderly_poweroff has not been able to shut off
@@ -544,6 +550,8 @@ static void hw_failure_emergency_poweroff_func(struct work_struct *work)*/ pr_emerg ("Hardware protection shutdown failed. Trying emergency restart\n" );emergency_restart (); + +printk_direct_exit (); } static DECLARE_DELAYED_WORK(hw_failure_emergency_poweroff_work,
@@ -582,11 +590,13 @@ void hw_protection_shutdown(const char *reason, int ms_until_forced) {static atomic_t allow_proceed =ATOMIC_INIT (1 ); +printk_direct_enter (); +pr_emerg ("HARDWARE PROTECTION shutdown (%s)\n" , reason);/* Shutdown should be initiated only once. */ if (!atomic_dec_and_test (&allow_proceed)) -return ; + goto out;/* * Queue a backup emergency shutdown in the event of
@@ -594,6 +604,8 @@ void hw_protection_shutdown(const char *reason, int ms_until_forced)*/ hw_failure_emergency_poweroff (ms_until_forced);orderly_poweroff (true); +out: +printk_direct_exit (); }EXPORT_SYMBOL_GPL (hw_protection_shutdown);
kernel/watchdog.c +4 -0
@@ -424,6 +424,8 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)/* Start period for the next softlockup warning. */ update_report_ts (); +printk_direct_enter (); +pr_emerg ("BUG: soft lockup - CPU#%d stuck for %us! [%s:%d]\n" ,smp_processor_id (), duration, current->comm,task_pid_nr (current));
@@ -442,6 +444,8 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)add_taint (TAINT_SOFTLOCKUP ,LOCKDEP_STILL_OK );if (softlockup_panic)panic ("softlockup: hung tasks" ); + +printk_direct_exit (); }return HRTIMER_RESTART ;
kernel/watchdog_hld.c +4 -0
@@ -135,6 +135,8 @@ static void watchdog_overflow_callback(struct perf_event *event,if (__this_cpu_read (hard_watchdog_warn) == true)return ; +printk_direct_enter (); +pr_emerg ("Watchdog detected hard LOCKUP on cpu %d\n" , this_cpu);print_modules ();
@@ -155,6 +157,8 @@ static void watchdog_overflow_callback(struct perf_event *event,if (hardlockup_panic)nmi_panic (regs,"Hard LOCKUP" ); +printk_direct_exit (); +__this_cpu_write (hard_watchdog_warn, true);return ; }
lib/dump_stack.c +2 -0
@@ -102,9 +102,11 @@ asmlinkage __visible void dump_stack_lvl(const char *log_lvl)* Permit this cpu to perform nested stack dumps while serialising * against other CPUs */ +printk_direct_enter ();printk_cpu_sync_get_irqsave (flags);__dump_stack (log_lvl);printk_cpu_sync_put_irqrestore (flags); +printk_direct_exit (); }EXPORT_SYMBOL (dump_stack_lvl);
lib/nmi_backtrace.c +2 -0
@@ -99,6 +99,7 @@ bool nmi_cpu_backtrace(struct pt_regs *regs)* Allow nested NMI backtraces while serializing * against other CPUs. */ +printk_direct_enter ();printk_cpu_sync_get_irqsave (flags);if (!READ_ONCE (backtrace_idle) && regs &&cpu_in_idle (instruction_pointer (regs))) { pr_warn("NMI backtrace for cpu %d skipped: idling at %pS\n",
@@ -111,6 +112,7 @@ bool nmi_cpu_backtrace(struct pt_regs *regs)dump_stack (); }printk_cpu_sync_put_irqrestore (flags); +printk_direct_exit ();cpumask_clear_cpu (cpu,to_cpumask (backtrace_mask));return true; }
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 09/13] printk: add functions to allow direct printing
Quoted 10 lines · John Ogness
Once kthread printing is introduced, console printing will no longer occur in the context of the printk caller. However, there are some special contexts where it is desirable for the printk caller to directly print out kernel messages. Using
pr_flush()to wait for threaded printers is only possible if the caller is in a sleepable context and the kthreads are active. That is not always the case.Introduce
printk_direct_enter()andprintk_direct_exit()functions to explicitly (and globally) activate/deactivate direct console printing.
We should make it more clear what direct console printing means.
It is just the best effort to print messages on consoles when they are unused at the moment. By other words, it is the legacy mode that uses trylock to get access to consoles.
Quoted 7 lines
Activate direct printing for:
- sysrq
- emergency reboot/shutdown
- cpu and rcu stalls
- hard and soft lockups
- hung tasks
- stack dumps
It would be great to mention what rules of thumb were used to choose these locations, see below.
Quoted 18 lines
Signed-off-by John Ogness <[email protected]>--- drivers/tty/sysrq.c | 2 ++ include/linux/printk.h | 11 +++++++++++ kernel/hung_task.c | 11 ++++++++++- kernel/printk/printk.c | 39 ++++++++++++++++++++++++++++++++++++++- kernel/rcu/tree_stall.h | 2 ++ kernel/reboot.c | 14 +++++++++++++- kernel/watchdog.c | 4 ++++ kernel/watchdog_hld.c | 4 ++++ lib/dump_stack.c | 2 ++ lib/nmi_backtrace.c | 2 ++ 10 files changed, 88 insertions(+), 3 deletions(-)drivers/tty/sysrq.c +0
-0@@ -594,9 +594,11 @@ void __handle_sysrq(int key, bool check_mask)* should not) and is the invoked operation enabled?
drivers/tty/sysrq.c void __handle_sysrq(int key, bool check_mask) · line 604
Quoted 10 lines
drivers/tty/sysrq.c +2
-0*/ if (!check_mask ||sysrq_on_mask (op_p->enable_mask)) { +printk_direct_enter ();pr_info ("%s\n" , op_p->action_msg); console_loglevel = orig_log_level; op_p->handler (key); +printk_direct_exit (); }else {pr_info ("This sysrq operation is disabled.\n" ); console_loglevel = orig_log_level;
We should handle all messages the same way, including "This sysrq operation is disabled" and "HELP:" section. @suppress_printk is disabled for all these messages as well.
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -3333,9 +3359,15 @@ static void wake_up_klogd_work_func(struct irq_work *irq_work)int pending =__this_cpu_xchg (printk_pending,0 );if (pending &PRINTK_PENDING_OUTPUT ) {
kernel/printk/printk.c static void wake_up_klogd_work_func(struct irq_work *irq_work) · line 3371
Quoted 10 lines
kernel/printk/printk.c +6
-0+if (pending &PRINTK_DIRECT_OUTPUT ) +printk_direct_enter (); +/* If trylock fails, someone else is doing the printing */ if (console_trylock ())console_unlock (); + +if (pending &PRINTK_DIRECT_OUTPUT ) +printk_direct_exit (); }
I want to be sure that we are on the same page.
This is a good example where the direct output is used in a non-preemtive context. It means that it might cause soft or live-lockups. And it might break RT guarantees.
It might be worth it when the system already is in big troubles and RT guarantees are already broken. This might be true for:
+ emergency reboot/shutdown
We probably should use it also when @suppress_printk is already disabled:
+ sysrq
But I am not sure about situations where only a particular process or CPU is in troubles:
+ cpu and rcu stalls
+ hard and soft lockups
+ hung tasks
+ "generic" stack dumps
The risks:
On one hand, I like that it is a conservative approach. It reduces the risk of bad user experience with switching to printk kthreads.
On the other hand, it means that the problem with soft/live-lockups will still be there. Also the direct output is global and affects any messages. It means that we will still need to keep and maintain printk_deferred() for scheduler, NMI, console drivers, ...
My preferences:
I prefer to do the changes proposed by this patch (with updated sysrq handling) in the 1st stage. It is more conservative. The switch to kthreads in most other situations will be big enough step.
But I think that we might go even further in the future. After all, the plan was to use the direct printing only when the system is about to die. This might eventually allow to remove printk_deferred() and all the possible soft/live-lockups.
Best Regards,
Petr
RE: [PATCH printk v1 09/13] printk: add functions to allow direct printing
Quoted 5 lines · John Ogness
Quoted 4 lines
Once kthread printing is introduced, console printing will no longer occur in the context of the printk caller. However, there are some special contexts where it is desirable for the printk caller to directly print out kernel messages.
Yes - most of them.
It is going to be pretty impossible to do 'hard' kernel debugging if printk get deferred.
Typically there already is a deferral method. The prints aren't done by the kernel, but by syslogd.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Petr Mladek
maintainerRe: [PATCH printk v1 09/13] printk: add functions to allow direct printing
Quoted 10 lines · David Laight
Quoted 5 lines · John Ogness
Quoted 4 lines
Once kthread printing is introduced, console printing will no longer occur in the context of the printk caller. However, there are some special contexts where it is desirable for the printk caller to directly print out kernel messages.
Yes - most of them.
It is going to be pretty impossible to do 'hard' kernel debugging if printk get deferred.
What do you exactly mean by 'hard' kernel debugging, please? What exactly won't be possible, please?
It is really important to talk about real scenarios because we do our best to address them. This patchset disables the deferred console handling in several situations, especially during panic, reboot, kexec. The consoles are flushed during suspend, etc.
The patchset solves real problems, soft and live-lockups and even deadlocks.
It will actually allow debugging when too many messages caused softlockups and the system died. I saw it many times during boot of systems with many network interfaces or disks. Backtraces from all CPUs and processes, detailed Oops reports, and similar extensive output was able to kill the system.
In each case, the legacy "direct" mode will still be around.
Quoted 2 lines
Typically there already is a deferral method. The prints aren't done by the kernel, but by syslogd.
syslogd helps only when userspace works and the messages can be stored on a filesystems.
And it is perfectly fine and much more safe to defer consoles most of the time. I have seen many reports where consoles put the system into knees. And sometime printk() was the root of all problems.
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 09/13] printk: add functions to allow direct printing
In reply to Petr MladekQuoted 12 lines · Petr Mladek
Quoted 10 lines · John Ogness
Once kthread printing is introduced, console printing will no longer occur in the context of the printk caller. However, there are some special contexts where it is desirable for the printk caller to directly print out kernel messages. Using
pr_flush()to wait for threaded printers is only possible if the caller is in a sleepable context and the kthreads are active. That is not always the case.Introduce
printk_direct_enter()andprintk_direct_exit()functions to explicitly (and globally) activate/deactivate direct console printing.
Quoted 5 lines
We should make it more clear what direct console printing means.
It is just the best effort to print messages on consoles when they are unused at the moment. By other words, it is the legacy mode that uses trylock to get access to consoles.
OK, I will clarify that.
Quoted 10 lines
Quoted 7 lines
Activate direct printing for:
- sysrq
- emergency reboot/shutdown
- cpu and rcu stalls
- hard and soft lockups
- hung tasks
- stack dumpsIt would be great to mention what rules of thumb were used to choose these locations, see below.
OK.
Quoted 10 lines
Quoted 10 lines
drivers/tty/sysrq.c +1
-0@@ -594,9 +594,11 @@ void __handle_sysrq(int key, bool check_mask)* should not) and is the invoked operation enabled? */ if (!check_mask ||sysrq_on_mask (op_p->enable_mask)) { +printk_direct_enter ();pr_info ("%s\n" , op_p->action_msg); console_loglevel = orig_log_level; op_p->handler (key);
Quoted 8 lines
drivers/tty/sysrq.c void __handle_sysrq(int key, bool check_mask) · line 604
Quoted 4 lines
drivers/tty/sysrq.c +1
-0+printk_direct_exit (); }else {pr_info ("This sysrq operation is disabled.\n" ); console_loglevel = orig_log_level;We should handle all messages the same way, including "This sysrq operation is disabled" and "HELP:" section. @suppress_printk is disabled for all these messages as well.
OK.
Quoted 19 lines
Quoted 6 lines
kernel/printk/printk.c +0
-0@@ -3333,9 +3359,15 @@ static void wake_up_klogd_work_func(struct irq_work *irq_work)int pending =__this_cpu_xchg (printk_pending,0 );if (pending &PRINTK_PENDING_OUTPUT ) {kernel/printk/printk.c static void wake_up_klogd_work_func(struct irq_work *irq_work) · line 3371
Quoted 10 lines
kernel/printk/printk.c +6
-0+if (pending &PRINTK_DIRECT_OUTPUT ) +printk_direct_enter (); +/* If trylock fails, someone else is doing the printing */ if (console_trylock ())console_unlock (); + +if (pending &PRINTK_DIRECT_OUTPUT ) +printk_direct_exit (); }I want to be sure that we are on the same page.
Quoted 3 lines
This is a good example where the direct output is used in a non-preemtive context. It means that it might cause soft or live-lockups. And it might break RT guarantees.
The idea here is that somewhere something was supposed to be direct printed, but the printing was deferred to irq_work. Then probably it should be direct printed here.
Also note that I implemented it incorrectly. Really wake_up_klogd_work_func() should have been changed to:
static void wake_up_klogd_work_func(struct irq_work *irq_work)
{
int pending = __this_cpu_xchg(printk_pending, 0);
if (pending & PRINTK_PENDING_OUTPUT &&
pending & PRINTK_DIRECT_OUTPUT) {
/* If trylock fails, someone else is doing the printing */
if (console_trylock())
console_unlock();
}
if (pending & PRINTK_PENDING_WAKEUP)
wake_up_interruptible(&log_wait);
}For v2 I will _not_ create a new PRINTK_DIRECT_OUTPUT flag value nor will I modify wake_up_klogd_work_func(). Instead I will modify defer_console_output() like this:
@@ -3256,11 +3659,16 @@ void wake_up_klogd(void)
void defer_console_output(void)
{
+ int val = PRINTK_PENDING_WAKEUP;
+
if (!printk_percpu_data_ready())
return;
+ if (atomic_read(&printk_direct))
+ val |= PRINTK_PENDING_OUTPUT;
+
preempt_disable();
- __this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT);
+ __this_cpu_or(printk_pending, val);
irq_work_queue(this_cpu_ptr(&wake_up_klogd_work)); preempt_enable();
}
So PRINTK_PENDING_WAKEUP will always be sent and PRINTK_PENDING_OUTPUT will be used for direct output.
Quoted 20 lines
It might be worth it when the system already is in big troubles and RT guarantees are already broken. This might be true for:
+ emergency reboot/shutdown
We probably should use it also when @suppress_printk is already disabled:
+ sysrq
But I am not sure about situations where only a particular process or CPU is in troubles:
+ cpu and rcu stalls
+ hard and soft lockups
+ hung tasks
+ "generic" stack dumpsThe risks:
Quoted 7 lines
On one hand, I like that it is a conservative approach. It reduces the risk of bad user experience with switching to printk kthreads.
On the other hand, it means that the problem with soft/live-lockups will still be there. Also the direct output is global and affects any messages. It means that we will still need to keep and maintain
printk_deferred()for scheduler, NMI, console drivers, ...
The reason I was so conservative is because we do not yet have atomic consoles. Right now direct printing is the only way to achieve some reliability. I am concerned that putting too much on the threads now might cause some bad taste for users and put the kthread approach at risk. All the above scenarios are things that should not occur on a normally functioning system.
Quoted 5 lines
My preferences:
I prefer to do the changes proposed by this patch (with updated sysrq handling) in the 1st stage. It is more conservative. The switch to kthreads in most other situations will be big enough step.
Agreed.
Quoted 4 lines
But I think that we might go even further in the future. After all, the plan was to use the direct printing only when the system is about to die. This might eventually allow to remove
printk_deferred()and all the possible soft/live-lockups.
Yes. Reliability will come from the planned atomic consoles. For systems that will have atomic consoles available, direct printing should be safe from all contexts. For systems without atomic consoles, we may need to keep the "legacy" direct printing alive.
John
John Ogness
maintainerPATCH 10/13 printk: add kthread console printers
In reply to John OgnessCreate a kthread for each console to perform console printing. During normal operation (@system_state == SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.
During non-normal operation, console printing is done as it has been:
within the context of the printk caller or within irq work triggered by the printk caller.
Console printers synchronize against each other and against console lockers by taking the console lock for each message that is printed.
--- include/linux/console.h | 2 + kernel/printk/printk.c | 159 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 159 insertions(+), 2 deletions(-)
include/linux/console.h +2 -0
@@ -153,6 +153,8 @@ struct console {uint ospeed;u64 seq;unsigned long dropped; +struct task_struct *thread; +void *data;struct console *next; };
kernel/printk/printk.c +157 -2
237 lines not on this page · raw message
Signature · 1 line
2.30.2
Petr Mladek
maintainerearly start: was: Re: [PATCH printk v1 10/13] printk: add kthread console printers
Quoted 19 lines · John Ogness
Create a kthread for each console to perform console printing. During normal operation (@system_state ==
SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.During non-normal operation, console printing is done as it has been:
within the context of the printk caller or within irq work triggered by the printk caller.Console printers synchronize against each other and against console lockers by taking the console lock for each message that is printed.
--- include/linux/console.h | 2 + kernel/printk/printk.c | 159 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 159 insertions(+), 2 deletions(-)
kernel/printk/printk.c void printk_direct_exit(void) · line 386
Quoted 7 lines
kernel/printk/printk.c +4
-0@@ -373,6 +380,14 @@ void printk_direct_exit(void)atomic_dec (&printk_direct); } +static inline bool allow_direct_printing (void ) +{ + return (!kthreads_started || + system_state != SYSTEM_RUNNING ||
It would be great to use printk kthreads as early as possible.
I have seen many soft-lockups when the system initialized many devices, for example, many disks or network interfaces. I am not completely sure. But I think that it might happen already when initcalls are proceed.
Many messages might be printed when people are debugging something.
kthreads are usable when "kthreadd" kernel thread is started in rest_init(). It means that they might be usable also in SYSTEM_SCHEDULING and SYSTEM_FREEING_INITMEM.
Quoted 8 lines
kernel/printk/printk.c +4
-0+ oops_in_progress || + atomic_read(&printk_direct)); +} +DECLARE_WAIT_QUEUE_HEAD (log_wait);/* All 3 protected by @syslog_lock. */ /* the next printk record to read by syslog(READ) or /proc/kmsg */
kernel/printk/printk.c static int __init printk_late_init(void) · line 3321
Quoted 9 lines
kernel/printk/printk.c +6
-0@@ -3275,6 +3313,13 @@ static int __init printk_late_init(void) ret =cpuhp_setup_state_nocalls (CPUHP_AP_ONLINE_DYN ,"printk:online" , console_cpu_notify,NULL );WARN_ON (ret <0 ); + +console_lock (); +for_each_console (con) +start_printk_kthread (con); + kthreads_started = true; +console_unlock ();
I would do this in core_initcall() so that the kthreads are used as early as possible.
kernel/printk/printk.c static int __init printk_late_init(void) · line 3325
Quoted 4 lines
kernel/printk/printk.c +1
-0+return 0 ; }late_initcall (printk_late_init);
Best Regards,
Petr
Petr Mladek
maintainerstart&stop: was: Re: [PATCH printk v1 10/13] printk: add kthread console printers
In reply to John OgnessQuoted 19 lines · John Ogness
Create a kthread for each console to perform console printing. During normal operation (@system_state ==
SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.During non-normal operation, console printing is done as it has been:
within the context of the printk caller or within irq work triggered by the printk caller.Console printers synchronize against each other and against console lockers by taking the console lock for each message that is printed.
--- include/linux/console.h | 2 + kernel/printk/printk.c | 159 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 159 insertions(+), 2 deletions(-)
kernel/printk/printk.c void register_console(struct console *newcon) · line 3149
Quoted 5 lines
kernel/printk/printk.c +2
-0@@ -3114,6 +3145,8 @@ void register_console(struct console *newcon)/* Begin with next message. */ newcon->seq =prb_next_seq (prb); } +if (kthreads_started) +start_printk_kthread (newcon);
It is a corner case but we should handle possible error here.
The easiest solution is to stop all other printk kthreads and use direct printing all the time. It should be enough because it is a corner case. If kthreads could not be created then the system is in big troubles anyway.
kernel/printk/printk.c int unregister_console(struct console *console) · line 3207
Quoted 9 lines
kernel/printk/printk.c +2
-0console_unlock ();console_sysfs_notify ();@@ -3170,6 +3203,11 @@ int unregister_console(struct console *console) } } +if (console->thread) { +kthread_stop (console->thread);
I think that must not be called under console_lock or any other locks that might be used by the kthread.
kthread_stop() waits until the kthread really exits. It means that kthread_func() must check kthread_should_stop() and return.
There might be a deadlock when kthread_func() need to take some locks in the main loop between kthread_should_stop() checks.
Quoted 8 lines
kernel/printk/printk.c +3
-0+ console->thread =NULL ; + } +if (res) goto out_disable_unlock;@@ -3345,6 +3390,116 @@ bool pr_flush(int timeout_ms, bool reset_on_progress)
[...]
Quoted 6 lines
kernel/printk/printk.c +6
-0+} + +static int printk_kthread_func (void *data) +{ +struct console *con = data; +char *dropped_text =NULL ;
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3427
Quoted 10 lines
kernel/printk/printk.c +10
-0+char *ext_text =NULL ; +bool progress; +bool handover; +u64 seq =0 ; +char *text; +int error; + +pr_info ("%sconsole [%s%d]: printing thread started\n" , + (con->flags &CON_BOOT ) ?"boot" :"" , + con->name, con->index);
This should be printed after the allocations succeeded before entering the main loop.
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3441
Quoted 13 lines
kernel/printk/printk.c +13
-0+ text =kmalloc (CONSOLE_LOG_MAX ,GFP_KERNEL ); +if (!text) + goto out; + +if (con->flags &CON_EXTENDED ) { + ext_text =kmalloc (CONSOLE_EXT_LOG_MAX ,GFP_KERNEL ); +if (!ext_text) + goto out; + }else { + dropped_text =kmalloc (DROPPED_TEXT_MAX ,GFP_KERNEL ); +if (!dropped_text) + goto out; + }
here
Quoted 10 lines
kernel/printk/printk.c +10
-0+for (;;) { + error =wait_event_interruptible (log_wait,printer_should_wake (con, seq)); + +if (kthread_should_stop ()) +break ; + +if (error) +continue ; + + do {
[...]
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3499
Quoted 12 lines
kernel/printk/printk.c +12
-0+ +/* Must be called within console_lock(). */ +static void start_printk_kthread (struct console *con) +{ + con->thread =kthread_run (printk_kthread_func, con, +"pr/%s%d" , con->name, con->index); +if (IS_ERR (con->thread)) { + con->thread =NULL ; +pr_err ("%sconsole [%s%d]: unable to start printing thread\n" , + (con->flags &CON_BOOT ) ?"boot" :"" , + con->name, con->index); +return ;
We need to pass the error down. The system has to switch to direct printing off when any printk kthread could not be started. Otherwise, the related console would not work.
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3505
Quoted 6 lines
kernel/printk/printk.c +3
-0+ } +} +/* * Delayed printk version, for scheduler-internal messages: */
Best Regards,
Petr
Petr Mladek
maintainermain loop: was: Re: [PATCH printk v1 10/13] printk: add kthread console printers
In reply to John OgnessQuoted 25 lines · John Ogness
Create a kthread for each console to perform console printing. During normal operation (@system_state ==
SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.During non-normal operation, console printing is done as it has been:
within the context of the printk caller or within irq work triggered by the printk caller.Console printers synchronize against each other and against console lockers by taking the console lock for each message that is printed.
--- include/linux/console.h | 2 + kernel/printk/printk.c | 159 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 159 insertions(+), 2 deletions(-)kernel/printk/printk.c +3
-0@@ -3345,6 +3390,116 @@ bool pr_flush(int timeout_ms, bool reset_on_progress) }EXPORT_SYMBOL (pr_flush); +static bool printer_should_wake (struct console *con,u64 seq) +{ +short flags;
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3405
Quoted 10 lines
kernel/printk/printk.c +10
-0+ +if (kthread_should_stop ()) +return true; + +if (console_suspended) +return false; + +/* +* This is an unsafe read to con->flags, but false positives +* are not an issue as long as they are rare.
This is not convincing. Anyway, it looks like an optimization. I suggest to do the check under console_lock in the main loop. printk kthread is a slow path anyway.
Also we should call there the entire console_is_usable(). It should do the same checks as in console_flush_all() and the direct mode.
Quoted 23 lines
kernel/printk/printk.c +23
-0+*/ + flags =data_race (READ_ONCE (con->flags)); +if (!(flags &CON_ENABLED )) +return false; + +return prb_read_valid (prb, seq,NULL ); +} + +static int printk_kthread_func (void *data) +{ +struct console *con = data; +char *dropped_text =NULL ; +char *ext_text =NULL ; +bool progress; +bool handover; +u64 seq =0 ; +char *text; +int error; + +pr_info ("%sconsole [%s%d]: printing thread started\n" , + (con->flags &CON_BOOT ) ?"boot" :"" , + con->name, con->index); +
[...]
Quoted 7 lines
kernel/printk/printk.c +7
-0+ +for (;;) { + error =wait_event_interruptible (log_wait,printer_should_wake (con, seq)); + +if (kthread_should_stop ()) +break ; +
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3458
Quoted 10 lines
kernel/printk/printk.c +10
-0+if (error) +continue ; + + do { + console_lock(); + if (console_suspended) { + console_unlock(); + break; + } +
We should check here also console_is_usable() to be in sync with console_flush_all().
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3467
Quoted 9 lines
kernel/printk/printk.c +9
-0+/* +* Even though the printk kthread is always preemptible, it is +* still not allowed to call cond_resched() from within +* console drivers. The task may become non-preemptible in the +* console driver call chain. For example, vt_console_print() +* takes a spinlock and then can call into fbcon_redraw(), +* which can conditionally invoke cond_resched(). +*/ + console_may_schedule =0 ;
I do not understand the reason. How is this different from console_unlock() and console_flush_all(), please?
Quoted 11 lines
kernel/printk/printk.c +11
-0+ progress =console_emit_next_record (con, text, ext_text, + dropped_text, &handover); +if (handover) +break ; + + seq = con->seq; + +/* Unlock console without invoking direct printing. */ +__console_unlock (); + }while (progress); + }
kernel/printk/printk.c bool pr_flush(int timeout_ms, bool reset_on_progress) · line 3488
Quoted 10 lines
kernel/printk/printk.c +10
-0+out: +kfree (dropped_text); +kfree (ext_text); +kfree (text); +pr_info ("%sconsole [%s%d]: printing thread stopped\n" , + (con->flags &CON_BOOT ) ?"boot" :"" , + con->name, con->index); +return 0 ; +} +
Best Regards,
Petr
Petr Mladek
maintainerwake_up_all: was: Re: [PATCH printk v1 10/13] printk: add kthread console printers
In reply to John OgnessQuoted 19 lines · John Ogness
Create a kthread for each console to perform console printing. During normal operation (@system_state ==
SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.During non-normal operation, console printing is done as it has been:
within the context of the printk caller or within irq work triggered by the printk caller.Console printers synchronize against each other and against console lockers by taking the console lock for each message that is printed.
--- include/linux/console.h | 2 + kernel/printk/printk.c | 159 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 159 insertions(+), 2 deletions(-)
kernel/printk/printk.c static void wake_up_klogd_work_func(struct irq_work *irq_work) · line 3529
Quoted 5 lines
kernel/printk/printk.c +1
-1@@ -3371,7 +3526,7 @@ static void wake_up_klogd_work_func(struct irq_work *irq_work) }if (pending &PRINTK_PENDING_WAKEUP ) -wake_up_interruptible (&log_wait); +wake_up_interruptible_all (&log_wait);
Good catch. I am curious how this worked so far. It looks like only the first waiter was waken and others had to wait for another new message.
The typical problem with wake_up_interruptible_all() is that it wakes many processes that needs to take the same lock and block each other. There is even a name for this situation but I do not recall it.
My understanding is that wake_up_interruptible() requires that the waken process wakes the next waiter when done. But I do not see it guaranteed for @log_wait wait queue.
Best Regards,
Petr
John Ogness
maintainerPATCH 11/13 printk: reimplement console_lock for proper kthread support
In reply to John OgnessWith non-threaded console printers preemption is disabled while holding the console lock in order to avoid the situation where the console printer is scheduled away and no other task can lock the console (for printing or otherwise). Disabling preemption is necessary because the console lock is implemented purely as a semaphore, which has no owner.
Like non-threaded console printers, kthread printers use the console lock to synchronize during printing. However, since they use console_lock() instead of a best-effort console_trylock(), it is not possible to disable preemption upon locking. Therefore an alternative for synchronizing and avoiding the above mentioned situation is needed.
The kthread printers do not need to synchronize against each other, but they do need to synchronize against console_lock() callers. To provide this synchronization, introduce a per-console mutex. The mutex is taken by the kthread printer during printing and is also taken by console_lock() callers. Since mutexes have owners, when calling console_lock(), the scheduler is able to schedule any kthread printers that may have been preempted while printing.
Rather than console_lock() callers holding the per-console mutex for the duration of the console lock, the per-console mutex is only taken in order to set a new CON_PAUSED flag, which is checked by the kthread printers. This avoids any issues due to nested locking between the various per-console mutexes.
The kthread printers must also synchronize against console_trylock() callers. Since console_trylock() is non-blocking, a global atomic counter will be used to identify if any kthread printers are active. The kthread printers will also check the atomic counter to identify if the console has been locked by another task via console_trylock().
A locking overview for console_lock(), console_trylock(), and the kthread printers is as follows (pseudo code):
console_lock()
{
down(&console_sem);
for_each_console(con) {
mutex_lock(&con->lock);
con->flags |= CON_PAUSED;
mutex_unlock(&con->lock);
}
}console_trylock()
{
assert(down_trylock(&console_sem));
assert(atomic_cmpxchg(&console_lock_count, 0, -1));
}kthread_printer()
{
mutex_lock(&con->lock);
assert(con->flags & CON_PAUSED);
assert(atomic_inc_unless_negative(&console_lock_count));
con->write();
atomic_dec(&console_lock_count);
mutex_unlock(&con->lock);
}Also note that the console owner and waiter logic now only applies between contexts that have both taken the console lock via console_trylock(). This is for 2 reasons:
1. Contexts that have taken the console lock via console_lock() require a sleepable context when unlocking to unpause the kthread printers. But a waiter context has used console_trylock() and may not be sleepable.
2. The kthread printers no longer acquire the console lock, so it is not possible to handover the console lock.
This also has implications for console_unlock(), which attempts a console_trylock() before returning. Introduce console_trylock_sched() to allow console_unlock() to specify if it is in a sleepable context.
--- include/linux/console.h | 15 ++++ kernel/printk/printk.c | 190 +++++++++++++++++++++++++++++++--------- 2 files changed, 166 insertions(+), 39 deletions(-)
include/linux/console.h +15 -0
@@ -16,6 +16,7 @@#include <linux/atomic.h> #include <linux/types.h> +#include <linux/mutex.h> struct vc_data ;struct console_font_op ;
@@ -136,6 +137,7 @@ static inline int con_debug_leave(void)#define CON_ANYTIME (16)/* Safe to call before per-cpu resources ready */ #define CON_BRL (32)/* Used for a braille device */ #define CON_EXTENDED (64)/* Use the extended output format a la /dev/kmsg */ +#define CON_PAUSED (128)/* Sleep while console is locked */ struct console {char name[16 ];
@@ -155,6 +157,19 @@ struct console {unsigned long dropped;struct task_struct *thread; +/* +* The per-console lock is used by printing kthreads to synchronize +* this console with callers of console_lock(). This is necessary in +* order to allow printing kthreads to run in parallel to each other, +* while each safely accessing their own @flags and synchronizing +* against direct printing via console_lock/console_unlock. +* +* Note: For synchronizing against direct printing via +* console_trylock/console_unlock, see the static global +* variable @console_lock_count. +*/ +struct mutex lock; +void *data;struct console *next; };
kernel/printk/printk.c +151 -39
321 lines not on this page · raw message
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Hi,
the solution is very interesting and I am sure that it has many advantages. But it is also quite complex. I want to be sure about the motivations before I start discussing details.
Quoted 4 lines · John Ogness
With non-threaded console printers preemption is disabled while holding the console lock in order to avoid the situation where the console printer is scheduled away and no other task can lock the console (for printing or otherwise).
I guess that you are talking about that vprintk_emit() disables the preemtion. console_lock() does not disable preemption.
Quoted 2 lines
Disabling preemption is necessary because the console lock is implemented purely as a semaphore, which has no owner.
I do not understand this explanation.
The preemtion was added into vprintk_emit() by the commit fd5f7cde1b85d4c8e09c ("printk: Never set console_may_schedule in console_trylock()").
It is not really necessary. It actually increases the risk of softlockups. The motivation was to increase the chance that messages will get printed. Nobody prints the messages when the owner is sleeping. We hoped that the console owner steeling will be efficient enough to prevent the softlockups.
Quoted 4 lines
Like non-threaded console printers, kthread printers use the console lock to synchronize during printing. However, since they use
console_lock()instead of a best-effortconsole_trylock(), it is not possible to disable preemption upon locking.
^^^^^^^^^^^^^^^
IMHO, it is possible to disable preemtion. But we propably do not want to do it, especially on RT.
Quoted 2 lines
Therefore an alternative for synchronizing and avoiding the above mentioned situation is needed.
I do not think that preemtion is the reason for an alternative soluiton.
IMHO, the reason is to handle different consoles independently. It will allow to handle more consoles in parallel. And more importantly they will not block each other.
Quoted 1 line
The kthread printers do not need to synchronize against each other,
yes
Quoted 1 line
but they do need to synchronize against
console_lock()callers.
We should make it more clear why. I see the following reasons:
+ it is needed for switching between direct printing and offload to kthreads.
+ another operations on all consoles, e.g. suspend, updating list of registered consoles
+ console_lock() has several other external users and their dependencies are complicated.
Quoted 3 lines
To provide this synchronization, introduce a per-console mutex. The mutex is taken by the kthread printer during printing and is also taken by
console_lock()callers.
ok
Quoted 3 lines
Since mutexes have owners, when calling
console_lock(), the scheduler is able to schedule any kthread printers that may have been preempted while printing.
I do not userstand the relation between mutex owners and scheduling kthreads. The mutex can be taken by any process.
Quoted 5 lines
Rather than
console_lock()callers holding the per-console mutex for the duration of the console lock, the per-console mutex is only taken in order to set a newCON_PAUSEDflag, which is checked by the kthread printers. This avoids any issues due to nested locking between the various per-console mutexes.
The flag adds some complexity. The logic will use two locks + one flag instead of just two locks. IMHO, it deserves a more clear explanation why it is needed. When can nested locking happen? Will it make the nested locking safe?
The obvious ordering is to always take con->mutex under console_sem.
Are you afraid that some console driver might need to take console_sem from con->write() callback? IMHO, this might cause deadlock even when using CON_PAUSED flag.
CPU0: CPU1:
console_lock()
down(&console_sem);
for_each_console(con)
kthread_printer()
mutex_lock(&con->lock);
con->write()
down(&console_sem)
mutex_lock(&con->lock);DEADLOCK!!!
Quoted 6 lines
The kthread printers must also synchronize against
console_trylock()callers. Sinceconsole_trylock()is non-blocking, a global atomic counter will be used to identify if any kthread printers are active. The kthread printers will also check the atomic counter to identify if the console has been locked by another task viaconsole_trylock().
This is another complexity. An easier solution would be to do:
console_trylock()
{
if (down_trylock(&console_sem))
return failed;
for_each_console(con) {
if (mutext_trylock(&con->lock))
goto fail;
}
// sucessfail:
for_each_console(con_fail) {
if (con_fail == con)
break;
mutex_unlock(&con->lock);
}Of course, the atomic counter is interesting. But it is also another extra complexity. Is it worth it?
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 3 lines · Petr Mladek
the solution is very interesting and I am sure that it has many advantages. But it is also quite complex. I want to be sure about the motivations before I start discussing details.
I am jumping ahead to this email so that we can start discussing details. This patch is really the key to make the whole series work correctly.
Quoted 8 lines · John Ogness
Quoted 4 lines
With non-threaded console printers preemption is disabled while holding the console lock in order to avoid the situation where the console printer is scheduled away and no other task can lock the console (for printing or otherwise).
I guess that you are talking about that
vprintk_emit()disables the preemtion.console_lock()does not disable preemption.
Yes. I did not mean to imply that console_lock() is disabling preemption. But preemption is disabled at some point _within_ the console_lock()/console_unlock() window. I consider console_lock()/console_unlock() to be the main API functions for direct printing and I wanted to point out that calling that pair will cause preemption disabling.
Quoted 9 lines
Quoted 2 lines
Disabling preemption is necessary because the console lock is implemented purely as a semaphore, which has no owner.
I do not understand this explanation.
The preemtion was added into
vprintk_emit()by the commitfd5f7cde1b85d4c8e09c("printk: Never setconsole_may_scheduleinconsole_trylock()").
Quoted 5 lines
It is not really necessary. It actually increases the risk of softlockups. The motivation was to increase the chance that messages will get printed. Nobody prints the messages when the owner is sleeping. We hoped that the console owner steeling will be efficient enough to prevent the softlockups.
Yes, I am referring to this case where the context taking the semaphore goes to sleep. If another context tries to take the semaphore, the scheduler does not know which task it can schedule to resolve the situation because there is no true lock contention.
With a mutex there is an owner. When another task tries to lock a mutex, the scheduler knows which task must be scheduled to resolve this true lock contention. (There are also other benefits relating to priority inheritance, but I chose not to mention this.)
Quoted 8 lines
Quoted 4 lines
Like non-threaded console printers, kthread printers use the console lock to synchronize during printing. However, since they use
console_lock()instead of a best-effortconsole_trylock(), it is not possible to disable preemption upon locking.^^^^^^^^^^^^^^^
IMHO, it is possible to disable preemtion. But we propably do not want to do it, especially on RT.
As stated in the commit message of fd5f7cde1b85 ("printk: Never set console_may_schedule in console_trylock()"), enabling preemption is "a mistake". That is why I said "not possible". Perhaps I should say "bad idea".
Quoted 5 lines
Quoted 2 lines
Therefore an alternative for synchronizing and avoiding the above mentioned situation is needed.
I do not think that preemtion is the reason for an alternative soluiton.
Well, if enabling preemption is "a mistake" then we _need_ an alternative.
Quoted 3 lines
IMHO, the reason is to handle different consoles independently. It will allow to handle more consoles in parallel. And more importantly they will not block each other.
This is an additional benefit, but it was not my primary motivation when designing the alternative. Honestly, I do not care how we phrase the motivation and benefits. I only wanted to make it clear that disabling preemption is not an acceptable option.
Quoted 8 lines
Quoted 1 line
The kthread printers do not need to synchronize against each other,
yes
Quoted 1 line
but they do need to synchronize against
console_lock()callers.We should make it more clear why. I see the following reasons:
Quoted 8 lines
+ it is needed for switching between direct printing and offload to kthreads.
+ another operations on all consoles, e.g. suspend, updating list of registered consoles
+
console_lock()has several other external users and their dependencies are complicated.
OK. Those are also the reasons I see.
Quoted 4 lines
Quoted 3 lines
To provide this synchronization, introduce a per-console mutex. The mutex is taken by the kthread printer during printing and is also taken by
console_lock()callers.
Quoted 8 lines
ok
Quoted 3 lines
Since mutexes have owners, when calling
console_lock(), the scheduler is able to schedule any kthread printers that may have been preempted while printing.I do not userstand the relation between mutex owners and scheduling kthreads. The mutex can be taken by any process.
As mentioned above, it is about creating true lock contention between the kthreads and the console_lock() callers.
Quoted 11 lines
Quoted 5 lines
Rather than
console_lock()callers holding the per-console mutex for the duration of the console lock, the per-console mutex is only taken in order to set a newCON_PAUSEDflag, which is checked by the kthread printers. This avoids any issues due to nested locking between the various per-console mutexes.The flag adds some complexity. The logic will use two locks + one flag instead of just two locks. IMHO, it deserves a more clear explanation why it is needed. When can nested locking happen? Will it make the nested locking safe?
Quoted 4 lines
The obvious ordering is to always take
con->mutexunderconsole_sem.Are you afraid that some console driver might need to take
console_semfromcon->write()callback?
No. By nested locking I mean:
for_each_console(c)
mutex_lock(&c->lock);
/* .... do something .... */
for_each_console_reversed(c)
mutex_unlock(&c->lock);Aside from this looking ugly and being a potential lockdep problem if the console ordering were to change, it would also mean that all the console mutexes are locked while suspended. That is not necessarily an issue, but I just did not like the idea of holding so many locks while suspended. And I still do not feel so comfortable about the somewhat arbitrary ordering of console locking.
If you think the above construct is good, then we do not need the flag approach.
Quoted 21 lines
Quoted 6 lines
The kthread printers must also synchronize against
console_trylock()callers. Sinceconsole_trylock()is non-blocking, a global atomic counter will be used to identify if any kthread printers are active. The kthread printers will also check the atomic counter to identify if the console has been locked by another task viaconsole_trylock().This is another complexity. An easier solution would be to do:
console_trylock(){ if (down_trylock(&console_sem)) return failed; for_each_console(con) { if (mutext_trylock(&con->lock)) goto fail; } // sucess
Quoted 6 lines
fail:
for_each_console(con_fail) { if (con_fail == con) break; mutex_unlock(&con->lock); }
No, this cannot work. Semaphores are the _only_ synchonization object in the kernel that is allowed to perform a trylock from any context (because they don't have owners).
Quoted 2 lines
Of course, the atomic counter is interesting. But it is also another extra complexity. Is it worth it?
I could find no alternative. But I am open for ideas if anyone thinks it can be done with less complexity.
John
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 21 lines · John Ogness
Quoted 14 lines
This is another complexity. An easier solution would be to do:
console_trylock(){ if (down_trylock(&console_sem)) return failed; for_each_console(con) { if (mutext_trylock(&con->lock)) goto fail; } // sucessQuoted 6 lines
fail:
for_each_console(con_fail) { if (con_fail == con) break; mutex_unlock(&con->lock); }
Quoted 3 lines
No, this cannot work. Semaphores are the _only_ synchonization object in the kernel that is allowed to perform a trylock from any context (because they don't have owners).
Sorry, I need to clarify this bold statement.
Semaphores are the only synchonization object in the kernel that is allowed to perform a trylock from any context and does not disable preemption upon locking. (This is important because console lock holders expect preemption to be unaffected when taking the lock.)
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
In reply to John OgnessHi,
first, to make it clear. The changes make sense and I am going to review the code. But I would still like to clarify the commit message. I do not understand it myself. Future generations should know why we did this ;-)
Quoted 18 lines · John Ogness
Quoted 3 lines · Petr Mladek
the solution is very interesting and I am sure that it has many advantages. But it is also quite complex. I want to be sure about the motivations before I start discussing details.
I am jumping ahead to this email so that we can start discussing details. This patch is really the key to make the whole series work correctly.
Quoted 8 lines · John Ogness
Quoted 4 lines
With non-threaded console printers preemption is disabled while holding the console lock in order to avoid the situation where the console printer is scheduled away and no other task can lock the console (for printing or otherwise).
I guess that you are talking about that
vprintk_emit()disables the preemtion.console_lock()does not disable preemption.
Quoted 6 lines
Yes. I did not mean to imply that
console_lock()is disabling preemption. But preemption is disabled at some point _within_ the console_lock()/console_unlock() window. I consider console_lock()/console_unlock() to be the main API functions for direct printing and I wanted to point out that calling that pair will cause preemption disabling.
But console_unlock() should not call consoles when kthreads are usable. It should just wake up them. The kthreads might still use console_lock() and __console_unlock().
I do not understand why this should be a reason to introduce per-console locks.
Quoted 20 lines
Quoted 9 lines
Quoted 2 lines
Disabling preemption is necessary because the console lock is implemented purely as a semaphore, which has no owner.
I do not understand this explanation.
The preemtion was added into
vprintk_emit()by the commitfd5f7cde1b85d4c8e09c("printk: Never setconsole_may_scheduleinconsole_trylock()").Quoted 5 lines
It is not really necessary. It actually increases the risk of softlockups. The motivation was to increase the chance that messages will get printed. Nobody prints the messages when the owner is sleeping. We hoped that the console owner steeling will be efficient enough to prevent the softlockups.
Yes, I am referring to this case where the context taking the semaphore goes to sleep. If another context tries to take the semaphore, the scheduler does not know which task it can schedule to resolve the situation because there is no true lock contention.
Quoted 4 lines
With a mutex there is an owner. When another task tries to lock a mutex, the scheduler knows which task must be scheduled to resolve this true lock contention. (There are also other benefits relating to priority inheritance, but I chose not to mention this.)
This sounds interesting. Does scheduler wake up or prioritize mutex owners?
I do not see this in the code. There is the optimistic spin but it is used only when the current owner is just running on another CPU.
Again, I do not see how this could be a motivation to introduce per-console mutexes.
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Like non-threaded console printers, kthread printers use the console lock to synchronize during printing. However, since they use
console_lock()instead of a best-effortconsole_trylock(), it is not possible to disable preemption upon locking.^^^^^^^^^^^^^^^
Quoted 7 lines
Quoted 2 lines
IMHO, it is possible to disable preemtion. But we propably do not want to do it, especially on RT.
As stated in the commit message of
fd5f7cde1b85("printk: Never setconsole_may_scheduleinconsole_trylock()"), enabling preemption is "a mistake". That is why I said "not possible". Perhaps I should say "bad idea".
"bad idea" is better. I am sorry that I am too picky. But the disabled preemption was a big mystery when I started working on printk code ages ago. And it was hard to find any explanation if it was required or just nice to have. Clear commit message or comment would have helped a lot.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Therefore an alternative for synchronizing and avoiding the above mentioned situation is needed.
I do not think that preemtion is the reason for an alternative soluiton.
Well, if enabling preemption is "a mistake" then we _need_ an alternative.
The alternative is the offload to kthreads that were introduced in another patch. This patch introduces per-console locks and the kthreads are usable even without them.
This is why I am confused by this commit messages.
Quoted 8 lines
Quoted 3 lines
IMHO, the reason is to handle different consoles independently. It will allow to handle more consoles in parallel. And more importantly they will not block each other.
This is an additional benefit, but it was not my primary motivation when designing the alternative. Honestly, I do not care how we phrase the motivation and benefits. I only wanted to make it clear that disabling preemption is not an acceptable option.
I am sorry but I still do not understand the primary motivation. I do not see how the per-console mutexes help with preemtion:
+ console_lock()/console_unlock() is fast when kthreads are usable
+ console_lock()/console_unlock() behaves the same as before when kthreads are not usable, for example, during early boot, or panic.
Idea: Did you want to prevent situation when the console_lock()
is blocked because any printk kthread is sleeping with the lock?
But it is still the case. console_lock() must still take each per-console mutex and can be blocked by any sleeping printk kthread.
BTW: The possibility to process printk kthreads in parallel is
a good enough reason from my POV. But I would still like to understand your primary motivation. There must be some important aspect that I do not understand. And it would be great to document it.
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
To provide this synchronization, introduce a per-console mutex. The mutex is taken by the kthread printer during printing and is also taken by
console_lock()callers.ok
Quoted 3 lines
Since mutexes have owners, when calling
console_lock(), the scheduler is able to schedule any kthread printers that may have been preempted while printing.
Quoted 5 lines
Quoted 2 lines
I do not userstand the relation between mutex owners and scheduling kthreads. The mutex can be taken by any process.
As mentioned above, it is about creating true lock contention between the kthreads and the
console_lock()callers.
I understand that it solves synchronization between the global console_lock() and per-console mutexes.
But I still do not understand the effect of mutex owners and that scheduler might wake them. I do not see any code for this. Will this behavior be added by some followup patchset?
Quoted 26 lines
Quoted 11 lines
Quoted 5 lines
Rather than
console_lock()callers holding the per-console mutex for the duration of the console lock, the per-console mutex is only taken in order to set a newCON_PAUSEDflag, which is checked by the kthread printers. This avoids any issues due to nested locking between the various per-console mutexes.The flag adds some complexity. The logic will use two locks + one flag instead of just two locks. IMHO, it deserves a more clear explanation why it is needed. When can nested locking happen? Will it make the nested locking safe?
Quoted 4 lines
The obvious ordering is to always take
con->mutexunderconsole_sem.Are you afraid that some console driver might need to take
console_semfromcon->write()callback?No. By nested locking I mean:
for_each_console(c) mutex_lock(&c->lock); /* .... do something .... */ for_each_console_reversed(c) mutex_unlock(&c->lock);
Quoted 6 lines
Aside from this looking ugly and being a potential lockdep problem if the console ordering were to change, it would also mean that all the console mutexes are locked while suspended. That is not necessarily an issue, but I just did not like the idea of holding so many locks while suspended. And I still do not feel so comfortable about the somewhat arbitrary ordering of console locking.
It makes perfect sense. I see why we need the flags now.
Quoted 31 lines
If you think the above construct is good, then we do not need the flag approach.
Quoted 21 lines
Quoted 6 lines
The kthread printers must also synchronize against
console_trylock()callers. Sinceconsole_trylock()is non-blocking, a global atomic counter will be used to identify if any kthread printers are active. The kthread printers will also check the atomic counter to identify if the console has been locked by another task viaconsole_trylock().This is another complexity. An easier solution would be to do:
console_trylock(){ if (down_trylock(&console_sem)) return failed; for_each_console(con) { if (mutext_trylock(&con->lock)) goto fail; } // sucessQuoted 6 lines
fail:
for_each_console(con_fail) { if (con_fail == con) break; mutex_unlock(&con->lock); }
Quoted 3 lines
No, this cannot work. Semaphores are the _only_ synchonization object in the kernel that is allowed to perform a trylock from any context (because they don't have owners).
Ah, mutex_trylock() can't be used in atomic context. I was not aware of this.
It means that also the atomic counter makes perfect sense. And it is very nice trick.
Best Regards,
Petr
PS: I am going to review the code.
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 15 lines · Petr Mladek
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
With non-threaded console printers preemption is disabled while holding the console lock in order to avoid the situation where the console printer is scheduled away and no other task can lock the console (for printing or otherwise).
I guess that you are talking about that
vprintk_emit()disables the preemtion.console_lock()does not disable preemption.Quoted 6 lines
Yes. I did not mean to imply that
console_lock()is disabling preemption. But preemption is disabled at some point _within_ the console_lock()/console_unlock() window. I consider console_lock()/console_unlock() to be the main API functions for direct printing and I wanted to point out that calling that pair will cause preemption disabling.
Quoted 3 lines
But
console_unlock()should not call consoles when kthreads are usable. It should just wake up them. The kthreads might still useconsole_lock()and__console_unlock().
I am only talking about when direct printing should be performed. Direct printing depends on more than just the kthreads. See allow_direct_printing().
Quoted 2 lines
I do not understand why this should be a reason to introduce per-console locks.
I see your point and you are correct. The reason for per-console locks is purely to allow the console printers to run in parallel.
Quoted 20 lines
Quoted 20 lines
Quoted 9 lines
Quoted 2 lines
Disabling preemption is necessary because the console lock is implemented purely as a semaphore, which has no owner.
I do not understand this explanation.
The preemtion was added into
vprintk_emit()by the commitfd5f7cde1b85d4c8e09c("printk: Never setconsole_may_scheduleinconsole_trylock()").Quoted 5 lines
It is not really necessary. It actually increases the risk of softlockups. The motivation was to increase the chance that messages will get printed. Nobody prints the messages when the owner is sleeping. We hoped that the console owner steeling will be efficient enough to prevent the softlockups.
Yes, I am referring to this case where the context taking the semaphore goes to sleep. If another context tries to take the semaphore, the scheduler does not know which task it can schedule to resolve the situation because there is no true lock contention.
Quoted 7 lines
Quoted 4 lines
With a mutex there is an owner. When another task tries to lock a mutex, the scheduler knows which task must be scheduled to resolve this true lock contention. (There are also other benefits relating to priority inheritance, but I chose not to mention this.)
This sounds interesting. Does scheduler wake up or prioritize mutex owners?
Sorry, the only example of this is priority inheritance. But for non-PREEMPT_RT there is no priority inheritance. The lock would need to be an rtmutex for this ability, which probably doesn't make sense for printk.
And besides, the priority inheritance only works in 1 direction. With the use of the flags approach, there is no way for printing kthreads to boost console_lock owners. So even mentioning priority inheritance is misleading.
Quoted 5 lines
I do not see this in the code. There is the optimistic spin but it is used only when the current owner is just running on another CPU.
Again, I do not see how this could be a motivation to introduce per-console mutexes.
You are correct.
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Like non-threaded console printers, kthread printers use the console lock to synchronize during printing. However, since they use
console_lock()instead of a best-effortconsole_trylock(), it is not possible to disable preemption upon locking.^^^^^^^^^^^^^^^
Quoted 7 lines
Quoted 2 lines
IMHO, it is possible to disable preemtion. But we propably do not want to do it, especially on RT.
As stated in the commit message of
fd5f7cde1b85("printk: Never setconsole_may_scheduleinconsole_trylock()"), enabling preemption is "a mistake". That is why I said "not possible". Perhaps I should say "bad idea".
Quoted 5 lines
"bad idea" is better. I am sorry that I am too picky. But the disabled preemption was a big mystery when I started working on printk code ages ago. And it was hard to find any explanation if it was required or just nice to have. Clear commit message or comment would have helped a lot.
AFAICT the issue only applies to the printk() call chain, because the caller may be holding other locks that are critical to the system. (In Tetsuo's case [0], it was @oom_lock.)
[0] https://lore.kernel.org/linux-mm/[email protected]/
That means the reasoning in my commit message about kthreaded printing and disabling preemption is moot. I will clean that up. Thanks for pointing out my handwaving.
Quoted 8 lines
I do not see how the per-console mutexes help with preemtion:
+ console_lock()/console_unlock() is fast when kthreads are usable
+ console_lock()/console_unlock() behaves the same as before when kthreads are not usable, for example, during early boot, or panic.
Quoted 5 lines
BTW: The possibility to process printk kthreads in parallel is
a good enough reason from my POV. But I would still like to understand your primary motivation. There must be some important aspect that I do not understand. And it would be great to document it.
You are right. The v2 commit message will focus on:
- the motivation for per-console locks is parallel printing
- explain about how disabling preemption is only necessary for direct printing via printk() because the caller may be holding system-critical and/or timing-sensitive locks (and also to allow the console owner/waiter logic to work correctly)
- correctly clarifying why the various types (semaphore/mutex/flag/atomic) were chosen to implement the printing sychronization between atomic-direct, non-atomic-direct, and kthreads (and I will explicitly remind the audience that mutex_trylock() cannot be used in atomic context)
Thanks, Petr!
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 6 lines · John Ogness
Quoted 5 lines · Petr Mladek
Quoted 4 lines
With a mutex there is an owner. When another task tries to lock a mutex, the scheduler knows which task must be scheduled to resolve this true lock contention. (There are also other benefits relating to priority inheritance, but I chose not to mention this.)
Quoted 7 lines
Quoted 2 lines
This sounds interesting. Does scheduler wake up or prioritize mutex owners?
Sorry, the only example of this is priority inheritance. But for non-PREEMPT_RT there is no priority inheritance. The lock would need to be an rtmutex for this ability, which probably doesn't make sense for printk.
Good to know.
Quoted 9 lines
The v2 commit message will focus on:
- the motivation for per-console locks is parallel printing
- explain about how disabling preemption is only necessary for direct printing via
printk()because the caller may be holding system-critical and/or timing-sensitive locks (and also to allow the console owner/waiter logic to work correctly)
Quoted 5 lines
- correctly clarifying why the various types (semaphore/mutex/flag/atomic) were chosen to implement the printing sychronization between atomic-direct, non-atomic-direct, and kthreads (and I will explicitly remind the audience that
mutex_trylock()cannot be used in atomic context)
Sounds great.
Thanks a lot for the hard work. It is very appreciated.
Best Regards,
Petr
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
In reply to John OgnessQuoted 19 lines · John Ogness
Rather than
console_lock()callers holding the per-console mutex for the duration of the console lock, the per-console mutex is only taken in order to set a newCON_PAUSEDflag, which is checked by the kthread printers. This avoids any issues due to nested locking between the various per-console mutexes.include/linux/console.h +1
-0@@ -16,6 +16,7 @@#include <linux/atomic.h> #include <linux/types.h> +#include <linux/mutex.h> struct vc_data ;struct console_font_op ;
include/linux/console.h static inline int con_debug_leave(void) · line 140
Quoted 4 lines
include/linux/console.h +1
-0@@ -136,6 +137,7 @@ static inline int con_debug_leave(void)#define CON_ANYTIME (16)/* Safe to call before per-cpu resources ready */ #define CON_BRL (32)/* Used for a braille device */ #define CON_EXTENDED (64)/* Use the extended output format a la /dev/kmsg */ +#define CON_PAUSED (128)/* Sleep while console is locked */
I was few times confused by the name. It sounds like that the console does not show any messages as all. I mean that it is something as console_suspend().
It is not easy to find a good short name explaining that it is only about kthreads. What about CON_DIRECT?
Quoted 15 lines
include/linux/console.h +0
-0struct console {char name[16 ];kernel/printk/printk.c +4
-0@@ -214,6 +214,26 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write,/* Number of registered extended console drivers. */ static int nr_ext_console_drivers; +/* +* Used to synchronize printing kthreads against direct printing via +* console_trylock/console_unlock. +*
kernel/printk/printk.c int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write, · line 230
Quoted 10 lines
kernel/printk/printk.c +10
-0+* Values: +* -1 = console locked (via trylock), kthreads will not print +* 0 = no kthread printing, console not locked (via trylock) +* >0 = kthread(s) actively printing +* +* Note: For synchronizing against direct printing via +* console_lock/console_unlock, see the @lock variable in +* struct console. +*/ +static atomic_t console_lock_count =ATOMIC_INIT (0 );
I am getting lost in all the variables called lock.
What about calling this "console_kthreads_active"? The counter is incremented when a kthread is actively printing.
kernel/printk/printk.c int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write, · line 235
Quoted 4 lines
kernel/printk/printk.c +4
-0+#define console_excl_trylock () (atomic_cmpxchg(&console_lock_count, 0, -1) == 0) +#define console_excl_unlock () atomic_cmpxchg(&console_lock_count, -1, 0) +#define console_printer_tryenter () atomic_inc_unless_negative(&console_lock_count) +#define console_printer_exit () atomic_dec(&console_lock_count)
Similar here. What about the following
#define console_kthreads_atomic_tryblock() (atomic_cmpxchg(&console_kthreads_active, 0, -1) == 0) #define console_kthreads_atomic_unblock() atomic_cmpxchg(&console_kthreads_active, -1, 0) #define console_kthread_tryenter() atomic_inc_unless_negative(&console_kthreads_active) #define console_kthread_exit() atomic_dec(&console_kthreads_active)
It is more clear that it is an "atomic" way to block/pause the "kthreads".
I do not resist on the word "block". I somehow like it more than "pause". But "pause" is good as well when used together with "kthreads".
Quoted 11 lines
kernel/printk/printk.c +4
-0+/* * Helper macros to handle lockdep when locking/unlocking console_sem. We use * macros instead of functions so that _RET_IP_ contains useful information. @@ -256,6 +276,37 @@ static void __up_console_sem(unsigned long ip) }#define up_console_sem () __up_console_sem(_RET_IP_) +/* +* Tracks whether kthread printers are all paused. A value of true implies +* that the console is locked via console_lock() or the console is suspended.
kernel/printk/printk.c static void __up_console_sem(unsigned long ip) · line 291
Quoted 10 lines
kernel/printk/printk.c +10
-0+* Reading and writing to this variable requires holding @console_sem. +*/ +static bool consoles_paused; + +/* +* Pause or unpause all kthread printers. +* +* Requires the console_lock. +*/ +static void __pause_all_consoles (bool do_pause)
Again. It feels like no consoles will work anymore. What about
console_kthreads_block().
Quoted 46 lines
kernel/printk/printk.c +33
-0+{ +struct console *con; + +for_each_console (con) { +mutex_lock (&con->lock); +if (do_pause) + con->flags |=CON_PAUSED ; +else + con->flags &= ~CON_PAUSED ; +mutex_unlock (&con->lock); + } + + consoles_paused = do_pause; +} + +#define pause_all_consoles () __pause_all_consoles(true) +#define unpause_all_consoles () __pause_all_consoles(false) +/* * This is used for debugging the mess that is the VT code by * keeping track if we have the console semaphore held. It's @@ -2568,15 +2616,45 @@ int console_trylock(void)up_console_sem ();return 0 ; } +if (!console_excl_trylock ()) { +up_console_sem (); +return 0 ; + } console_locked =1 ; console_may_schedule =0 ;return 1 ; }EXPORT_SYMBOL (console_trylock); +/* +* A variant of console_trylock() that allows specifying if the context may +* sleep. If yes, a trylock on @console_sem is attempted and if successful, +* the threaded printers are paused. This is important to ensure that +* sleepable contexts do not become involved in console_lock handovers and +* will call cond_resched() during the printing loop. +*/ +static int console_trylock_sched (bool may_schedule) +{ +if (!may_schedule) +return console_trylock ();
kernel/printk/printk.c int console_trylock(void) · line 2649
Quoted 10 lines
kernel/printk/printk.c +10
-0+ +might_sleep (); + +if (down_trylock_console_sem ()) +return 0 ; +if (console_suspended) { +up_console_sem (); +return 0 ; + } +pause_all_consoles ();
This is weird. Any trylock function should be fast and non-blocking. But pause_all_consoles() uses mutex_lock().
My expectation would be that console_trylock_sched() behaves excatly the same as console_trylock() except that it will set console_may_schedule by the given parameter.
I would do it the other way. Rename console_trylock() and implement:
int console_trylock(void)
{
return console_trylock_sched(false);
}LATER: I got it. It is used by console_trylock_sched() called
in console_unlock() when "do_cond_resched == true". In this case, the trylock might wait for the mutexes. It will prevent transfering console_lock from schedulable to atomic context by the check in console_emit_next_record().
Hmm, I would still prefer to keep console_trylock_sched() behavior sane: non-blocking in all situations. It means that we actually do not need it and console_trylock() is enough.
It will allow to steal console_lock() from schedulable context. But it is not a regression. And it is only a corner case when console_unlock() re-takes the semaphore after releasing it.
We could do the same optimization in console_unlock() by calling console_emit_next_record() with NULL handover pointer when do_cond_resched == true. But we should do it as a separate patch later.
Quoted 15 lines
kernel/printk/printk.c +6
-1+ console_locked =1 ; + console_may_schedule =1 ; +return 1 ; +} +int is_console_locked (void ) { -return console_locked; +return (console_locked ||atomic_read (&console_lock_count)); }EXPORT_SYMBOL (is_console_locked);@@ -2610,6 +2688,19 @@ static inline bool console_is_usable(struct console *con)static void __console_unlock (void ) {
kernel/printk/printk.c static inline bool console_is_usable(struct console *con) · line 2699
Quoted 10 lines
kernel/printk/printk.c +9
-0console_locked =0 ; + +/* +* Depending on whether console_lock() or console_trylock() was used, +* appropriately allow the kthread printers to continue. +*/ +if (consoles_paused) +unpause_all_consoles (); +else +console_excl_unlock ();
This is a good example when the meaning of console_excl_lock() is not obvious. IHMO, the following is easier to understand:
if (console_kthreads_blocked) console_kthreads_unblock(); else console_kthreads_atomic_unblock();
Quoted 41 lines
kernel/printk/printk.c +9
-6+ +/* Wake the kthread printers. */ +wake_up_klogd (); +up_console_sem (); }@@ -2632,7 +2723,8 @@ static void __console_unlock(void)* * @handover will be set to true if a printk waiter has taken over the * console_lock, in which case the caller is no longer holding the -* console_lock. Otherwise it is set to false. +* console_lock. Otherwise it is set to false. A NULL pointer may be provided +* to disable allowing the console_lock to be taken over by a printk waiter. */ static bool console_emit_next_record (struct console *con,char *text,char *ext_text,char *dropped_text,bool *handover)@@ -2640,12 +2732,14 @@ static bool console_emit_next_record(struct console *con, char *text, char *ext_struct printk_info info;struct printk_record r;unsigned long flags; +bool allow_handover;char *write_text;size_t len;prb_rec_init_rd (&r, &info, text,CONSOLE_LOG_MAX ); - *handover = false; +if (handover) + *handover = false;if (!prb_read_valid (prb, con->seq, &r))return false;@@ -2671,18 +2765,23 @@ static bool console_emit_next_record(struct console *con, char *text, char *ext_ len =record_print_text (&r, console_msg_format &MSG_FORMAT_SYSLOG , printk_time); } -/* -* While actively printing out messages, if another printk() -* were to occur on another CPU, it may wait for this one to -* finish. This task can not be preempted if there is a
kernel/printk/printk.c static bool console_emit_next_record(struct console *con, char *text, char *ext_ · line 2769
Quoted 10 lines
kernel/printk/printk.c +2
-8-* waiter waiting to take over. -* -* Interrupts are disabled because the hand over to a waiter -* must not be interrupted until the hand over is completed -* (@console_waiter is cleared). -*/ -printk_safe_enter_irqsave (flags); -console_lock_spinning_enable (); +/* Handovers may only happen between trylock contexts. */ + allow_handover = (handover &&atomic_read (&console_lock_count) ==-1 );
It is far from obvious why the check for console_lock_count is needed. I found the reason only after removing the check and seeing the warning about using mutex_unlock() in atomic context.
I suggest something like:
/*
* Allow to handover the lock only from atomic context. The handover
* is used to transfer console owner between more printk() callers.
* They are not allowed to unblock the console kthreads by mutexes
* in atomic context.
*/
allow_handover = (handover && console_kthreads_atomically_blocked());
Quoted 28 lines
kernel/printk/printk.c +16
-1+ +if (allow_handover) { +/* +* While actively printing out messages, if another printk() +* were to occur on another CPU, it may wait for this one to +* finish. This task can not be preempted if there is a +* waiter waiting to take over. +* +* Interrupts are disabled because the hand over to a waiter +* must not be interrupted until the hand over is completed +* (@console_waiter is cleared). +*/ +printk_safe_enter_irqsave (flags); +console_lock_spinning_enable (); + }stop_critical_timings ();/* don't trace print latency */ call_console_driver (con, write_text, len, dropped_text);@@ -2825,7 +2926,7 @@ void console_unlock(void)* Re-check if there is a new record to flush. If the trylock * fails, another context is already handling the printing. */ - }while (prb_read_valid (prb, next_seq,NULL ) &&console_trylock ()); + }while (prb_read_valid (prb, next_seq,NULL ) &&console_trylock_sched (do_cond_resched)); }EXPORT_SYMBOL (console_unlock);
kernel/printk/printk.c void console_unblank(void) · line 2963
Quoted 7 lines
kernel/printk/printk.c +4
-0@@ -2856,6 +2957,10 @@ void console_unblank(void)if (oops_in_progress) {if (down_trylock_console_sem () !=0 )return ; +if (!console_excl_trylock ()) { +up_console_sem (); +return ; + }
It would be better to use
if (oops_in_progress) {
if (!console_trylock())
return;Quoted 12 lines
kernel/printk/printk.c +0
-3}else {pr_flush (1000 , true);console_lock ();@@ -3397,16 +3502,17 @@ static bool printer_should_wake(struct console *con, u64 seq)if (kthread_should_stop ())return true; -if (console_suspended) -return false; -/* * This is an unsafe read to con->flags, but false positives
kernel/printk/printk.c static bool printer_should_wake(struct console *con, u64 seq) · line 3515
Quoted 10 lines
kernel/printk/printk.c +5
-1* are not an issue as long as they are rare. */ flags =data_race (READ_ONCE (con->flags)); -if (!(flags &CON_ENABLED )) + +if (!(flags &CON_ENABLED ) || + (flags &CON_PAUSED ) || +atomic_read (&console_lock_count) ==-1 ) {return false; + }
I would remove also this and do the after waking up. I think that it is actually already being discussed in another patch.
kernel/printk/printk.c static bool printer_should_wake(struct console *con, u64 seq) · line 3518
Quoted 3 lines
kernel/printk/printk.c +0
-0return prb_read_valid (prb, seq,NULL ); }
Otherwise, it looks good.
This must have been hard to invent. Great job!
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 37 lines · Petr Mladek
Quoted 11 lines
kernel/printk/printk.c +11
-0+/* +* A variant of console_trylock() that allows specifying if the context may +* sleep. If yes, a trylock on @console_sem is attempted and if successful, +* the threaded printers are paused. This is important to ensure that +* sleepable contexts do not become involved in console_lock handovers and +* will call cond_resched() during the printing loop. +*/ +static int console_trylock_sched (bool may_schedule) +{ +if (!may_schedule) +return console_trylock ();kernel/printk/printk.c int console_trylock(void) · line 2649
Quoted 10 lines
kernel/printk/printk.c +10
-0+ +might_sleep (); + +if (down_trylock_console_sem ()) +return 0 ; +if (console_suspended) { +up_console_sem (); +return 0 ; + } +pause_all_consoles ();This is weird. Any trylock function should be fast and non-blocking. But
pause_all_consoles()usesmutex_lock().My expectation would be that
console_trylock_sched()behaves excatly the same asconsole_trylock()except that it will setconsole_may_scheduleby the given parameter.I would do it the other way. Rename
console_trylock()and implement:int
console_trylock(void){ return console_trylock_sched(false); }
Quoted 5 lines
LATER: I got it. It is used by
console_trylock_sched()called
inconsole_unlock()when "do_cond_resched== true". In this case, the trylock might wait for the mutexes. It will prevent transferingconsole_lockfrom schedulable to atomic context by the check inconsole_emit_next_record().
Yes!
Quoted 9 lines
Hmm, I would still prefer to keep
console_trylock_sched()behavior sane: non-blocking in all situations. It means that we actually do not need it andconsole_trylock()is enough.It will allow to steal
console_lock()from schedulable context. But it is not a regression. And it is only a corner case whenconsole_unlock()re-takes the semaphore after releasing it.
A console waiter must not wait on a schedulable context. The console waiter is burning the CPU waiting for a transfer. If the console owner gets scheduled away while still holding the console lock, that is bad.
Quoted 4 lines
We could do the same optimization in
console_unlock()by callingconsole_emit_next_record()with NULL handover pointer whendo_cond_resched== true. But we should do it as a separate patch later.
It is not an optimization, it is needed. Passing a NULL handover pointer when do_cond_resched == true would handle it correctly, but this feels like a workaround to me.
The reason for adding console_trylock_sched() is because a context that previously acquired the console lock via console_lock() wants to try to reacquire it. If it reacquires the console lock using the kthread mutexes, the locking scenario returns to the same as it was before... all kthreads are blocked via their mutex.
You are suggesting that a console_lock() context later tries to reacquire the console lock, but using the console_trylock() method (atomic counter) and keeping console_may_schedule=1.
IMHO, _this_ is a weird variant that requires passing in a NULL handover pointer as a workaround. It introduces a third locking scenario where a schedulable context is using functions created for atomic use.
Also, as I mentioned in the percpu thread [0], I think we need to avoid console_trylock() usage in schedulable contexts. Functions need to be aware in what contexts they are running and call the appropriate functions for it.
Quoted 11 lines
kernel/printk/printk.c void console_unblank(void) · line 2963
Quoted 8 lines
kernel/printk/printk.c +4
-0@@ -2856,6 +2957,10 @@ void console_unblank(void)if (oops_in_progress) {if (down_trylock_console_sem () !=0 )return ; +if (!console_excl_trylock ()) { +up_console_sem (); +return ; + }It would be better to use
Quoted 4 lines
if (oops_in_progress) {
if (!console_trylock())
return;Well that is slightly different. It would mean that @console_suspended is now also considered. I will investigate if that matters, but currently it is not considered.
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 51 lines · John Ogness
Quoted 37 lines · Petr Mladek
Quoted 11 lines
kernel/printk/printk.c +11
-0+/* +* A variant of console_trylock() that allows specifying if the context may +* sleep. If yes, a trylock on @console_sem is attempted and if successful, +* the threaded printers are paused. This is important to ensure that +* sleepable contexts do not become involved in console_lock handovers and +* will call cond_resched() during the printing loop. +*/ +static int console_trylock_sched (bool may_schedule) +{ +if (!may_schedule) +return console_trylock ();kernel/printk/printk.c int console_trylock(void) · line 2649
Quoted 10 lines
kernel/printk/printk.c +10
-0+ +might_sleep (); + +if (down_trylock_console_sem ()) +return 0 ; +if (console_suspended) { +up_console_sem (); +return 0 ; + } +pause_all_consoles ();This is weird. Any trylock function should be fast and non-blocking. But
pause_all_consoles()usesmutex_lock().My expectation would be that
console_trylock_sched()behaves excatly the same asconsole_trylock()except that it will setconsole_may_scheduleby the given parameter.I would do it the other way. Rename
console_trylock()and implement:int
console_trylock(void){ return console_trylock_sched(false); }Quoted 5 lines
LATER: I got it. It is used by
console_trylock_sched()called
inconsole_unlock()when "do_cond_resched== true". In this case, the trylock might wait for the mutexes. It will prevent transferingconsole_lockfrom schedulable to atomic context by the check inconsole_emit_next_record().Yes!
Quoted 5 lines
Hmm, I would still prefer to keep
console_trylock_sched()behavior sane: non-blocking in all situations. It means that we actually do not need it andconsole_trylock()is enough.
Quoted 8 lines
Quoted 4 lines
It will allow to steal
console_lock()from schedulable context. But it is not a regression. And it is only a corner case whenconsole_unlock()re-takes the semaphore after releasing it.A console waiter must not wait on a schedulable context. The console waiter is burning the CPU waiting for a transfer. If the console owner gets scheduled away while still holding the console lock, that is bad.
There must be a confusion. Of course, preemption must be disabled when the current console_lock() owner allows to take over the lock. It is currently achieved by disabling interrupts:
printk_safe_enter_irqsave(flags); console_lock_spinning_enable(); call_console_driver(con, write_text, len, dropped_text); *handover = console_lock_spinning_disable_and_check(); printk_safe_exit_irqrestore(flags);
I wanted to say that it is safe even when the preemption is otherwise enabled around this code.
Also I wanted to say that it is not ideal when the current owner is called with preemption enabled and the new owner will continue handling consoles with preemption disabled. But the original code worked this way.
It would be nice to avoid moving the lock from a preemptive context to non-preemptive one. But I would prefer to do it separately because the proposed console_trylock_sched() is really controversial. IMHO, it is not worth delaying this patchset because of this.
Quoted 19 lines
Quoted 4 lines
We could do the same optimization in
console_unlock()by callingconsole_emit_next_record()with NULL handover pointer whendo_cond_resched== true. But we should do it as a separate patch later.It is not an optimization, it is needed. Passing a NULL handover pointer when
do_cond_resched== true would handle it correctly, but this feels like a workaround to me.The reason for adding
console_trylock_sched()is because a context that previously acquired the console lock viaconsole_lock()wants to try to reacquire it. If it reacquires the console lock using the kthread mutexes, the locking scenario returns to the same as it was before... all kthreads are blocked via their mutex.You are suggesting that a
console_lock()context later tries to reacquire the console lock, but using theconsole_trylock()method (atomic counter) and keeping console_may_schedule=1.
Quoted 8 lines
IMHO, _this_ is a weird variant that requires passing in a NULL handover pointer as a workaround. It introduces a third locking scenario where a schedulable context is using functions created for atomic use.
Also, as I mentioned in the percpu thread [0], I think we need to avoid
console_trylock()usage in schedulable contexts. Functions need to be aware in what contexts they are running and call the appropriate functions for it.
Let's put the NULL parameter aside. It is an implementation detail. It can be handled by another parameter or so.
IMHO, the main disagreement is:
+ I do not like the proposed console_trylock_sched() API because it uses mutex_lock().
From my POV, it can obviously cause deadlock. It is something that nobody would expect from any *_trylock() API. IMHO, it is a ticking bomb.
+ You do not like that console_lock()/console_unlock() might re-take the lock using another approach (atomic counter vs. con->flags.
From your POV, using non-blocking console_trylock() in console_unlock() would be 3rd locking scheme. Also you say that that the caller should know the context and use the appropriate function.
Let me try to persuade you ;-)
1. Regarding the 3rd locking scheme:
It is exactly the opposite from my POV. console_trylock_sched() looks like a 3rd locking variant to me. We would have blocking console_lock(), non-blocking console_trylock(), and semi-blocking console_trylock_sched().
And it is even worse. console_trylock_sched() behaves differently according to the parameter. It makes is even more hard to understand the behavior.
2. Regarding using appropriate function:
Let's see how it looks in the code:
printk()
preempt_disable()
console_trylock()
console_unlock()
// printing
up_console_sem()
retry = console_trylock_sched();It feels like using sleeping lock in a non-preemptive context. OK, we could make it more obvious:
console_unlock()
{
[...]
if (may_schedule)
retry = console_trylock_sched();
else
retry = console_trylock();}
But it is yet another complexity in console_unlock(). Is it really necessary?
3. Implementation detail:
From my POV, the way how console_lock()/console_trylock() takes the lock is an implementation detail.
console_lock() uses con->flags because it can and actually must wait/sleep. console_trylock() uses the atomic counter because it must not block.
We hide this implementation detail in __console_unlock(). It releases the lock the same way as it was taken.
4. may_schedule vs. handover
The information whether console_unlock() may schedule is stored in @console_may_schedule variable. console_lock() allows scheduling and console_trylock() disables scheduling.
console_emit_next_record() allows handover according to the atomic counter. It is set by the way how console_lock was taken. console_lock() disables handover. console_trylock() enables handover.
Now, console_trylock_sched() is needed to re-take the lock the same way as it was taken originally. Otherwise, the atomic counter will be used even when scheduling was enabled. As a result, console_emit_next_record() would work differently.
By other word, console_trylock_sched() is used to encode @console_may_schedule into @console_lock_count.
It looks like a hack similar to passing NULL *handover. But from my POV, it is much worse. The price is console_trylock_sched() API with very strange behavior. It is super tricky. It took me hours to understand the motivation and behavior.
From, my POV, the most clear solution is to pass "may_schedule" parameter to console_emit_next_record().
Does it make sense, please?
Do I miss anything, please?
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 8 lines · Petr Mladek
console_unlock()
{
[...]
if (may_schedule)
retry = console_trylock_sched();
else
retry = console_trylock();
}I believe the main confusion comes from the function name I chose and the poor function description. Using your above code idea and changing to a more fitting name, I would suggest:
console_unlock()
{
[...]
if (may_schedule)
retry = console_lock_reacquire();
else
retry = console_trylock();
}This console_lock_reacquire() acquires the console lock the same way that console_lock() does it. The only reason we don't just use console_lock() is because we want to perform a try on @console_sem. But if we are successful, in the end, we have re-taken the console lock exactly as console_lock() did before: @console_sem locked, kthreads blocked by mutex.
You say this creates deadlock potential, but I do not see how that could be. We are in the same context and locking the same way we did before.
But my primary concern is not the naming or workarounds or confusing APIs. So we should not let ourselves be diverted by that aspect.
My primary concern is the technical difference when a schedulable context reacquires via atomic counter (which fails if any kthread is active) vs. reacquiring via mutex (which never fails).
The reason for the reacquire is because (during direct printing) we see that a new record appeared and we need to make sure it gets printed (because other direct printers may have aborted, expecting us to print it).
This scenario is only interesting if kthread printers exist because otherwise @console_sem is enough to handle the direct printing.
So the questions are:
1. Is it OK to assume the new record will be printed if any kthread is active? If yes, then it is enough to use the atomic counter.
2. Or, since we are responsible for direct printing, do we want to be certain that the record is printed by printing it ourselves? If yes, then we must block all the kthreads and perform the printing directly to all the consoles. This requires the mutex approach.
IMHO #1 will relies heavily on kthreads waking up and printing (even though the printk caller requested direct printing), whereas #2 will cause direct printers to more actively print (possibly printing more than was requested).
I prefer to go the route of #2 because it is more conservative. IMHO, when direct printing becomes active, we really should make a strong effort to direct print.
Having now stated the issues (hopefully clearly), I will not fight for #2. If you say it is enough to rely on the kthreads in this scenario, then I will implement the atomic counter solution for my v2.
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 30 lines · John Ogness
Quoted 8 lines · Petr Mladek
console_unlock() { [...] if (may_schedule) retry = console_trylock_sched(); else retry = console_trylock(); }I believe the main confusion comes from the function name I chose and the poor function description. Using your above code idea and changing to a more fitting name, I would suggest:
console_unlock() {(quoted diff) +0
-0if (may_schedule) retry = console_lock_reacquire(); else retry = console_trylock(); }This
console_lock_reacquire()acquires the console lock the same way thatconsole_lock()does it. The only reason we don't just useconsole_lock()is because we want to perform a try on @console_sem. But if we are successful, in the end, we have re-taken the console lock exactly asconsole_lock()did before: @console_sem locked, kthreads blocked by mutex.
Quoted 2 lines
You say this creates deadlock potential, but I do not see how that could be. We are in the same context and locking the same way we did before.
Yeah, it looks safe when using to re-acquire the lock in console_unlock().
I thought about it as a generic API. If it exists, it might be used anywhere. And it has pretty special behavior that one has to keep in mind.
Quoted 11 lines
But my primary concern is not the naming or workarounds or confusing APIs. So we should not let ourselves be diverted by that aspect.
My primary concern is the technical difference when a schedulable context reacquires via atomic counter (which fails if any kthread is active) vs. reacquiring via mutex (which never fails).
The reason for the reacquire is because (during direct printing) we see that a new record appeared and we need to make sure it gets printed (because other direct printers may have aborted, expecting us to print it).
I see. I missed this POV.
Quoted 13 lines
This scenario is only interesting if kthread printers exist because otherwise @console_sem is enough to handle the direct printing.
So the questions are:
1. Is it OK to assume the new record will be printed if any kthread is active? If yes, then it is enough to use the atomic counter.
2. Or, since we are responsible for direct printing, do we want to be certain that the record is printed by printing it ourselves? If yes, then we must block all the kthreads and perform the printing directly to all the consoles. This requires the mutex approach.
Quoted 4 lines
IMHO #1 will relies heavily on kthreads waking up and printing (even though the printk caller requested direct printing), whereas #2 will cause direct printers to more actively print (possibly printing more than was requested).
OK, it means that the main problem here _is not_ the scheduling context, console_lock() vs. console_trylock(). The main problem _is_ the direct printing vs. the offload to kthreads.
Of course, the context is important. It affects how we could re-take the lock. But the main problem is the printing mode. We must make sure that:
1. someone is printing pending messages when the direct mode is needed
2. kthreads are woken and can enter the printing mode when the direct mode is disabled.
Will console_trylock_sched()/console_trylock_reacquire() really help here?
The API theoretically helps in direct mode when the lock was taken via console_lock(). But it does not help when the lock was taken via console_trylock() from printk(). It might mean that the forward progress might not be guaranteed in the direct mode (early boot, panic, ...).
Hmm, the forward progress seems to be guaranteed in the direct mode most of the time. console_trylock() can take over the atomic counter because console kthreads are not allowed to enter the printing mode in this case.
I used "most of the time" because there might be races when the mode is switched. "printk_direct" is an atomic variable. CON_DIRECT is set under con->mutex but console_trylock() does not take the mutex...
There are also races when the offload to consoles kthreads is allowed. For example, console_trylock() might block console_kthread_printing_tryenter().
Sigh, I am afraid that we have non-trivial problems to guarantee that all messages will be printed:
+ races when switching between direct mode
and offload to kthreads. It might cause
stall in both modes.
+ console_trylock() races with console_kthread_printing_tryenter(). It might put kthread into a sleep even when it is supposed to print the message.
IMHO, console_trylock_sched() does not help much here. We need to solve console_trylock() path anyway.
I think that the solution might be:
+ make sure that the state of "printk_direct" atomic variable is enough to distinguish about the mode.
+ always wakeup() console kthreads after console_trylock() to handle the possible race with console_kthread_printing_tryenter()
I have to think more about it.
Does it make any sense, please?
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 9 lines · Petr Mladek
Quoted 9 lines
console_unlock() {(quoted diff) +0
-0if (may_schedule) retry = console_lock_reacquire(); else retry = console_trylock(); }
[...]
Quoted 9 lines
OK, it means that the main problem here
_isnot_the scheduling context,console_lock()vs.console_trylock(). The main problem _is_ the direct printing vs. the offload to kthreads.Of course, the context is important. It affects how we could re-take the lock. But the main problem is the printing mode. We must make sure that:
1. someone is printing pending messages when the direct mode is needed
console_trylock() causes difficulties here because it will fail if any kthread is active. It is an example of direct mode failure. But is that really any different than current mainline console_trylock() failing because a console_lock() context is active (and possibly not scheduled on a CPU)?
Quoted 2 lines
2. kthreads are woken and can enter the printing mode when the direct mode is disabled.
This happens at the end of vprintk_emit() and within __console_unlock(), regardless if the printk() was running in direct mode or not.
Quoted 4 lines
Will
console_lock_reacquire()really help here?The API theoretically helps in direct mode when the lock was taken via
console_lock().
console_lock_reacquire() only exists for the console_lock() case.
Quoted 4 lines
But it does not help when the lock was taken
viaconsole_trylock()fromprintk(). It might mean that the forward progress might not be guaranteed in the direct mode (early boot, panic, ...).
How is the console_trylock() case different from current mainline now? As I mentioned above, the kthreads can block console_trylock(), but so can a console_lock() currently in mainline.
Quoted 9 lines
Hmm, the forward progress seems to be guaranteed in the direct mode most of the time.
console_trylock()can take over the atomic counter because console kthreads are not allowed to enter the printing mode in this case.I used "most of the time" because there might be races when the mode is switched. "
printk_direct" is an atomic variable.CON_DIRECTis set undercon->mutexbutconsole_trylock()does not take the mutex...
You are mixing issues here. If CON_DIRECT is set, there is already a console_lock() in progress, so console_trylock() fails on @console_sem.
Quoted 3 lines
There are also races when the offload to consoles kthreads is allowed. For example,
console_trylock()might blockconsole_kthread_printing_tryenter().
I do not see how that is a problem. If any context has the console lock (no matter how it got that lock) then the kthreads are blocked.
If direct printing is enabled (from @printk_direct or @oops_in_progress or @system_state != SYSTEM_RUNNING), the task with the console lock will print out *all* the remaining records.
If direct printing is not enabled, the kthreads are woken in __console_unlock().
Quoted 6 lines
Sigh, I am afraid that we have non-trivial problems to guarantee that all messages will be printed:
+ races when switching between direct mode
and offload to kthreads. It might cause
stall in both modes.
Scheduable contexts holding locks can cause stalls. We have that same problem with console_lock() in mainline now. The kernel provides mechanisms to avoid such stalls (niceness, priorities, policies, priority inheritance), but this is all problem-specific and must be fine-tuned by the user if they are running workloads that are causing problems. kthreads are not solving the reliability problem (and they never will).
Quoted 4 lines
+
console_trylock()races withconsole_kthread_printing_tryenter(). It might put kthread into a sleep even when it is supposed to print the message.
kthread is never _supposed_ to print a message. It is there to offload direct printing. If console_trylock() (direct printing) wins, then that is the context that does the printing.
Quoted 2 lines
IMHO,
console_lock_reacquire()does not help much here. We need to solveconsole_trylock()path anyway.
It preserves a consistent locking scenario for the console_lock() path. That is all it is intended to do.
Quoted 4 lines
I think that the solution might be:
+ make sure that the state of "
printk_direct" atomic variable is enough to distinguish about the mode.
The printk subsystem does not have absolute control over which task/context is doing the actual printing. Currently in mainline there are printers that handoff to waiters and even some that do not print at all because there is already waiters. Adding kthreads introduces a new task that can print. But there still is no real control about who prints. The important thing is that there is some context or runnable task that is running the printing code (whether scheduled or not).
Quoted 3 lines
+ always
wakeup()console kthreads afterconsole_trylock()to handle the possible race withconsole_kthread_printing_tryenter()
I do not understand. If console_trylock() wins, it already wakes the kthreads on __console_unlock(). If console_trylock() loses, the kthreads are already running.
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 9 lines · John Ogness
Quoted 4 lines · Petr Mladek
But it does not help when the lock was taken
viaconsole_trylock()fromprintk(). It might mean that the forward progress might not be guaranteed in the direct mode (early boot, panic, ...).How is the
console_trylock()case different from current mainline now? As I mentioned above, the kthreads can blockconsole_trylock(), but so can aconsole_lock()currently in mainline.
CPU0 CPU1 CPU2
printk()
// direct mode allowed
console_trylock()
console_unlock()
console_flush_all()
printk_direct_enter()
allows_direct_printing() -> false;
break;
__console_unlock()
wakeup_klogd()
// woken printk_khread
console_thread_printing_enter()
printk_direct_exit()
console_trylock()
atomic_tryblock()
//fails because thread activereturn;
printk_direct_enter() console_thread_printing_exit() // sleep because atomic_read(&printk_direct) is not zero
Result: nobody prints
Note: The thread will actually not sleep because printk_should_wake()
does not check atomic_read(&printk_direct).
But it is a bug. Active thread should check it and allow entering direct mode.
Note2: Even when one printk thread flushes the messages. There might
be other thread that will never get scheduled a nobody would printk the messages on the related console.
This is the race that I see with console_trylock(). IMHO, if we solve this race then we do not need console_trylock_sched().
Well, I might be wrong. It is Friday evening. I still do not have the entire picture. I probably should have waited for Monday.
I am sure that I misunderstood several things, including wait_event() API or CON_DIRECT flag. But the above race looks real. And it somehow supports my feeling about that there are some races that can't be solved by console_trylock_sched().
Best Regards,
Petr
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Hi Petr,
I do not think the example below is showing what you think it does, but it does shed light on an important point.
Quoted 4 lines · Petr Mladek
CPU0 CPU1 CPU2
printk()
// direct mode allowed
OK, so @printk_direct is >= 1.
Quoted 5 lines
console_trylock()
console_unlock()
console_flush_all()
printk_direct_enter()
@printk_direct is now >= 2.
Quoted 2 lines
allows_direct_printing() -> false;I do not understand why you say it returns false. @printk_direct is not zero. This returns _true_ and will print all records currently in the ringbuffer.
Quoted 4 lines
break;
__console_unlock()
wakeup_klogd()Note that all kthreads wake here. To keep it simple, let us assume there is only 1.
Quoted 1 line
// woken printk_khread
The kthread will only go into its printing loop if new records have appeared after the above __console_unlock(). If any did appear, _that_ printk() will attempt to print them because direct printing is active. But let us assume that the kthread on CPU2 woke up before the new printk() context could attempt a console_lock() or console_trylock().
Quoted 1 line
console_thread_printing_enter()
@console_lock_count is now 1. Now console_lock() and console_trylock() will block or fail, respectively.
Quoted 1 line
printk_direct_exit()
@printk_direct is now >= 1.
Quoted 3 lines
console_trylock()
atomic_tryblock()
//fails because thread activeCorrect. (Note that if CPU0 had been in a console_lock() context, it would _not_ fail because console_lock_reacquire() would wait for the active kthreads to finish their current record.)
Quoted 3 lines
return; printk_direct_enter()
@printk_direct is now >= 2.
Quoted 1 line
console_thread_printing_exit()
Note that CPU2 would have printed a single record.
Quoted 3 lines
// sleep because atomic_read(&printk_direct) is not zero
No, the kthread does not sleep. It will continue printing until it is blocked via console_lock()/console_trylock() or until all the records are printed.
Quoted 1 line
Result: nobody prints
Sorry, but I have no idea how you came to that conclusion.
Quoted 2 lines
Note: The thread will actually not sleep because
printk_should_wake()
does not checkatomic_read(&printk_direct).
Exactly.
Quoted 2 lines
But it is a bug. Active thread should check it and allow entering direct mode.
We are in direct mode this entire example. I do not understand what you mean by "allow entering direct mode". Perhaps you mean "allow console_trylock() to succeed".
If that is what you mean, then you are suggesting that the console_trylock() spins until all the kthreads have finished their current record. This could be a new variant of console_trylock().
And then rather than continuing their printing loop, the kthreads all stay asleep as long as @printk_direct is non-zero (in addition to the existing sleep conditions).
This could work if the kthreads disable preemption for the console_thread_printing_enter()/_exit() window.
Although it should be noted that this still will not help if a console_lock() context is scheduled out.
Quoted 3 lines
Note2: Even when one printk thread flushes the messages. There might
be other thread that will never get scheduled a nobody would printk the messages on the related console.
That is what atomic consoles are for. Until atomic consoles are available, that situation is covered by @oops_in_progress and console_flush_on_panic().
Quoted 2 lines
This is the race that I see with
console_trylock(). IMHO, if we solve this race then we do not needconsole_lock_reacquire().
I do not understand why you continue to mix console_trylock() and console_lock_reacquire(). console_lock_reacquire() is only for the console_lock() context.
Quoted 2 lines
Well, I might be wrong. It is Friday evening. I still do not have the entire picture. I probably should have waited for Monday.
I believe that you sense a danger with direct printing and console_trylock(). It allows for scenarios like your example that end up relying on kthreads to finish the printing (if there is no panic). Mainline already has this issue because console_lock() can also be scheduled away and console_trylock() has no chance to print. This really is the same issue and ultimately relies on @oops_in_progress and console_flush_on_panic() to get the messages out.
I believe you are hinting at the worst-case scenario: a kthread getting scheduled out while printing and never seeing a CPU again because the system is so busy. Assuming the system does not panic, no printing would be seen on that console anymore, even if direct printing is enabled.
The only solution I see (aside from atomic consoles) is to disable preemption during printing. Perhaps for non-atomic consoles, this is what we need to do. That, together with a new console_trylock() variant, should avoid this concern. Do you agree? Do we want to go that route?
Disabling preemption would be a problem moving forward for the fbcon because, for the future, I really would like fbcon to live in a sleepable context. I already have some new ideas about this. But that is not a topic for this series.
John
P.S. By "new console_trylock() variant" I mean something for the console_trylock() context like:
1. grab @console_sem with down_trylock_console_sem()
2. spin on atomic_tryblock()
If console_thread_printing_enter() disables preemption, this should work about as well as the current owner/waiter logic for @console_sem. I can do some prototype testing to see if there is some detail I am missing.
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 13 lines · John Ogness
Hi Petr,
I do not think the example below is showing what you think it does, but it does shed light on an important point.
Quoted 4 lines · Petr Mladek
CPU0 CPU1 CPU2
printk()
// direct mode allowedOK, so @printk_direct is >= 1.
Quoted 7 lines
Quoted 5 lines
console_trylock()
console_unlock()
console_flush_all()
printk_direct_enter()@printk_direct is now >= 2.
I am sorry. I made a mistake in the example. I hope that the following shows what I wanted to show:
CPU0 CPU1 CPU2
printk_direct_enter() @printk_direct == 1.
printk()
// direct mode allowed
console_trylock()
console_unlock()
printk_direct_exit()
@printk_direct == 0
console_flush_all()
allows_direct_printing() -> false;
break;
__console_unlock()
wakeup_klogd()
// woken printk_ktread
console_thread_printing_enter()
console_trylock()
atomic_tryblock()
//fails because thread activereturn;
printk_direct_enter() @printk_direct == 1 console_thread_printing_exit()
My expectation: printk_kthread will go into sleep because the system
is in direct mode. In this case, nobody would be scheduled to continue the printing.
Quoted 3 lines
No, the kthread does not sleep. It will continue printing until it is blocked via console_lock()/console_trylock() or until all the records are printed.
I see. In the patch, the kthread does not check @printk_direct at all. I have missed this :-(
From my POV, it is a very non-intuitive behavior. But it seems that you did it intentionally.
Quoted 3 lines
If that is what you mean, then you are suggesting that the
console_trylock()spins until all the kthreads have finished their current record. This could be a new variant ofconsole_trylock().
No, console_trylock() must not wait. My expectation would be something like:
static bool printer_should_wake(struct console *con, u64 seq) {
[...]
/*
* Bail out when direct more is requested. Make sure
* that someone tries to continue printing in
* the direct mode.
*/
if (atomic_read(&printk_direct)) {
defer_console_output();
return false;
}[...]
}
Quoted 10 lines
That is what atomic consoles are for. Until atomic consoles are available, that situation is covered by @oops_in_progress and
console_flush_on_panic().Quoted 2 lines
This is the race that I see with
console_trylock(). IMHO, if we solve this race then we do not needconsole_lock_reacquire().I do not understand why you continue to mix
console_trylock()andconsole_lock_reacquire().console_lock_reacquire()is only for theconsole_lock()context.
Because the code path in console_unlock() is the same for console_lock() and console_trylock(). I am sorry but I still do not see any convincing argument why it is so important to handle the trylock different way according to how it was taken originally.
The result of console_trylock() and console_reackquire() is exactly the same:
+ console_sem is taken
+ kthreads are blocked by a single value (flag or counter)
My view, is that the console_trylock_reacquire() is a weird function that is not necessary. If we really need it to avoid any race then then the logic is too complex and we have to make it easier. console_trylock() must always be safe to use!
Simple logic:
If the atomic console_trylock() fails it means that someone else is already printing. It does not matter if it is another console_lock owner or kthreads.
If console_trylock() is enough in one code path then it should be enough also in the other.
Also:
If the console lock is released then anything could happen, especially the kthreads might start printing. I do not see any point in stopping them again.
They will start working _only_ when the direct mode is _not_ requested. If they did not start working then we could re-take the lock atomically and we do not need to set CON_DIRECT flag.
My view is that console_kthreads_atomic_tryblock() is a fast path how to block the kthreads. IMHO, it is perfectly fine to use it even in console_lock(). The complicated and sleeping console_kthreads_block() is needed only when any kthread is printing and we really need to wait.
OK, you wrote:
<John>
The reason for the reacquire is because (during direct printing) we see that a new record appeared and we need to make sure it gets printed (because other direct printers may have aborted, expecting us to print it).
</John>
Let's make sure that we understand it the same way. When are we expected to print the pending records directly?
+ When console_lock() was used? My answer is "No".
+ When @printk_direct > 0 My answer is "Yes".
Quoted 10 lines
Quoted 2 lines
Well, I might be wrong. It is Friday evening. I still do not have the entire picture. I probably should have waited for Monday.
I believe that you sense a danger with direct printing and
console_trylock(). It allows for scenarios like your example that end up relying on kthreads to finish the printing (if there is no panic). Mainline already has this issue becauseconsole_lock()can also be scheduled away andconsole_trylock()has no chance to print. This really is the same issue and ultimately relies on @oops_in_progress andconsole_flush_on_panic()to get the messages out.
Sure.
Quoted 4 lines
I believe you are hinting at the worst-case scenario: a kthread getting scheduled out while printing and never seeing a CPU again because the system is so busy. Assuming the system does not panic, no printing would be seen on that console anymore, even if direct printing is enabled.
We could not do much about this. As you say, it happens already in current mainline. If the current console_lock owner is sleeping, it might never release the console lock. The fallback is in console_flush_panic().
IMHO, it is worse with the kthreads. Any printk kthread might block switching to the direct mode and printing messages on other consoles. We should allow switching to the direct mode ASAP. This is why I suggest that kthreads should check @printk_direct counter and bail out. It is similar to
https://lore.kernel.org/all/[email protected]/
It makes the logic symmetric and easier:
+ console_lock() and printk_direct_enter() block kthreads. The kthreads will bail out on the next record when printing.
+ console_unlock() and printk_direct_exit() allow using kthreads.
+ console_unlock() will stop direct printing when @printk_direct == 0.
+ printk_kthread() will get blocked when @printk_direct > 0
+ vprintk_emit() and console_unlock() will do the same decision according to @printk_direct counter. They will either try to printk directly or wake up kthreads.
Slightly unrelated:
+ __console_emit_next_record() should allow handover by a parameter passed from the caller. Not by guessing from a global state.
Quoted 9 lines
The only solution I see (aside from atomic consoles) is to disable preemption during printing. Perhaps for non-atomic consoles, this is what we need to do. That, together with a new
console_trylock()variant, should avoid this concern. Do you agree? Do we want to go that route?Disabling preemption would be a problem moving forward for the fbcon because, for the future, I really would like fbcon to live in a sleepable context. I already have some new ideas about this. But that is not a topic for this series.
I though about this as well. But let's keep the preemption enabled in the kthreads for now. We could always disable it when it causes problems in the real life.
Best Regards,
Petr
PS: It seems like we are going in circles. My intention is to keep
the logic as simple and as clear as possible:
+ if we need lock then use lock
+ if we need trylock then use trylock
+ if we want direct mode then block kthreads and try enter the direct mode ASAP.
+ if kthreads mode is allowed then do nothing in console_unlock() and leave the job to kthreads.
+ console_lock() temporarily blocks kthreads but it handle messages only when direct mode is enforced.
John Ogness
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 6 lines · Petr Mladek
My intention is to keep the logic as simple and as clear as possible:
+ if we need lock then use lock
+ if we need trylock then use trylock
Quoted 8 lines
+ if we want direct mode then block kthreads and try enter the direct mode ASAP.
+ if kthreads mode is allowed then do nothing in
console_unlock()and leave the job to kthreads.+
console_lock()temporarily blocks kthreads but it handle messages only when direct mode is enforced.
Thank you for your examples, detailed analysis, insight, and summaries.
This particular review became quite complicated because offline you sent me a heavily revised version. Several of your comments are criticizing your version and not the actual series I posted. For v2 we need to handle it better so that the list has a chance at following our discussion. ;-)
I will post a v2 that attempts to address your concerns and try to frame the naming and structures to align with your suggestions.
John
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
Quoted 16 lines · John Ogness
Quoted 6 lines · Petr Mladek
My intention is to keep the logic as simple and as clear as possible:
+ if we need lock then use lock
+ if we need trylock then use trylock
Quoted 8 lines
+ if we want direct mode then block kthreads and try enter the direct mode ASAP.
+ if kthreads mode is allowed then do nothing in
console_unlock()and leave the job to kthreads.+
console_lock()temporarily blocks kthreads but it handle messages only when direct mode is enforced.
Quoted 7 lines
Thank you for your examples, detailed analysis, insight, and summaries.
This particular review became quite complicated because offline you sent me a heavily revised version. Several of your comments are criticizing your version and not the actual series I posted. For v2 we need to handle it better so that the list has a chance at following our discussion. ;-)
I am really sorry for this. And some my mails also were a bit misleading because I missed something in the code.
One small plus is that most of the new names were mentioned in the public https://lore.kernel.org/r/YhYKP/UuSKENGwfj@alley
But I agree that I probably made it more confusing than needed.
Quoted 2 lines
I will post a v2 that attempts to address your concerns and try to frame the naming and structures to align with your suggestions.
Yes, sending v2 and continuing the discussion sounds like a good idea.
Best Regards,
Petr
Petr Mladek
maintainerRe: [PATCH printk v1 11/13] printk: reimplement console_lock for proper kthread support
In reply to John OgnessQuoted 21 lines · John Ogness
Quoted 9 lines · Petr Mladek
Quoted 9 lines
console_unlock() {(quoted diff) +0
-0if (may_schedule) retry = console_lock_reacquire(); else retry = console_trylock(); }[...]
Quoted 8 lines
OK, it means that the main problem here
_isnot_the scheduling context,console_lock()vs.console_trylock(). The main problem _is_ the direct printing vs. the offload to kthreads.Of course, the context is important. It affects how we could re-take the lock. But the main problem is the printing mode. We must make sure that:
Quoted 7 lines
Quoted 1 line
1. someone is printing pending messages when the direct mode is needed
console_trylock()causes difficulties here because it will fail if any kthread is active. It is an example of direct mode failure. But is that really any different than current mainlineconsole_trylock()failing because aconsole_lock()context is active (and possibly not scheduled on a CPU)?
I see. Well, console_lock() does not mean that we are in direct mode. Yes, it stops kthreads. But it might be called by register_console() or tty driver. console_unlock() might do nothing when allow_direct_printing() returns false in console_flush_all().
Quoted 13 lines
Quoted 2 lines
2. kthreads are woken and can enter the printing mode when the direct mode is disabled.
This happens at the end of
vprintk_emit()and within__console_unlock(), regardless if theprintk()was running in direct mode or not.Quoted 4 lines
Will
console_lock_reacquire()really help here?The API theoretically helps in direct mode when the lock was taken via
console_lock().
console_lock_reacquire()only exists for theconsole_lock()case.
Quoted 8 lines
Quoted 4 lines
But it does not help when the lock was taken
viaconsole_trylock()fromprintk(). It might mean that the forward progress might not be guaranteed in the direct mode (early boot, panic, ...).How is the
console_trylock()case different from current mainline now? As I mentioned above, the kthreads can blockconsole_trylock(), but so can aconsole_lock()currently in mainline.
It is very different. Failing console_trylock() in the current mainline means that someone else is responsible for printing all the pending messages. All should be on the safe side.
Failing console_trylock() in this patch means that some printk kthread is active but nothing else. The kthread might handle one message and then go into sleep because someone requested the direct mode using atomic_inc(). Other kthreads might be sleeping and ignore everything. The failing console_trylock() will mean that nobody will handle the rest of the messages. The race is described in the other reply.
Honestly, handling console_trylock() case looks more important to me because it is used by printk(). It is the main usecase. While console_lock() code path should be used rarely, during console registration. Well, it is used also by tty code but I am not sure how often it is really called there.
Quoted 5 lines
Quoted 5 lines
Hmm, the forward progress seems to be guaranteed in the direct mode most of the time.
console_trylock()can take over the atomic counter because console kthreads are not allowed to enter the printing mode in this case.
Quoted 7 lines
Quoted 4 lines
I used "most of the time" because there might be races when the mode is switched. "
printk_direct" is an atomic variable.CON_DIRECTis set undercon->mutexbutconsole_trylock()does not take the mutex...You are mixing issues here. If
CON_DIRECTis set, there is already aconsole_lock()in progress, soconsole_trylock()fails on @console_sem.
Yes, I am sorry for the confusion. I forgot the meaning of the flag. I thought that it was set when the direct mode was requested. But it is not true. It only means that console_lock is taken and it could do the direct printing.
It would make sense to rename it. It might be part of all the confusion here. The name creates false feeling about that the direct mode is requested. I think that I have actually suggested renaming the flag in an earlier mail.
Best Regards,
Petr
John Ogness
maintainerPATCH 12/13 printk: remove @console_locked
In reply to John OgnessThe static global variable @console_locked is used to help debug VT code to make sure that certain code paths are running with the console_lock held. However, this information is also available with the static global variable @consoles_paused (for locking via console_lock()), and the static global variable @console_lock_count (for locking via console_trylock()).
Remove @console_locked and update is_console_locked() to use the alternative variables.
--- kernel/printk/printk.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-)
kernel/printk/printk.c +14 -16
@@ -307,15 +307,7 @@ static void __pause_all_consoles(bool do_pause)#define pause_all_consoles () __pause_all_consoles(true)#define unpause_all_consoles () __pause_all_consoles(false) -/* -* This is used for debugging the mess that is the VT code by -* keeping track if we have the console semaphore held. It's -* definitely not the perfect debug tool (we don't know if _WE_ -* hold it and are racing, but it helps tracking those weird code -* paths in the console code where we end up in places I want -* locked without the console semaphore held). -*/ -static int console_locked, console_suspended; +static int console_suspended;/* * Array of consoles built from command line options (console=)
@@ -2595,7 +2587,6 @@ void console_lock(void)if (console_suspended)return ;pause_all_consoles (); - console_locked =1 ; console_may_schedule =1 ; }EXPORT_SYMBOL (console_lock);
@@ -2620,7 +2611,6 @@ int console_trylock(void)up_console_sem ();return 0 ; } - console_locked =1 ; console_may_schedule =0 ;return 1 ; }
@@ -2647,14 +2637,25 @@ static int console_trylock_sched(bool may_schedule)return 0 ; }pause_all_consoles (); - console_locked =1 ; console_may_schedule =1 ;return 1 ; } +/* +* This is used to help to make sure that certain paths within the VT code are +* running with the console lock held. It is definitely not the perfect debug +* tool (it is not known if the VT code is the task holding the console lock), +* but it helps tracking those weird code paths in the console code such as +* when the console is suspended: where the console is not locked but no +* console printing may occur. +* +* Note: This returns true when the console is suspended but is not locked. +* This is intentional because the VT code must consider that situation +* the same as if the console was locked. +*/ int is_console_locked (void ) { -return (console_locked ||atomic_read (&console_lock_count)); +return (consoles_paused ||atomic_read (&console_lock_count)); }EXPORT_SYMBOL (is_console_locked);
@@ -2687,8 +2688,6 @@ static inline bool console_is_usable(struct console *con)static void __console_unlock (void ) { - console_locked =0 ; -/* * Depending on whether console_lock() or console_trylock() was used, * appropriately allow the kthread printers to continue.
@@ -2966,7 +2965,6 @@ void console_unblank(void)console_lock (); } - console_locked =1 ; console_may_schedule =0 ;for_each_console (c)if ((c->flags &CON_ENABLED ) && c->unblank)
Signature · 1 line
2.30.2
Re: [PATCH printk v1 12/13] printk: remove @console_locked
Quoted 9 lines · John Ogness
The static global variable @console_locked is used to help debug VT code to make sure that certain code paths are running with the
console_lockheld. However, this information is also available with the static global variable @consoles_paused (for locking viaconsole_lock()), and the static global variable @console_lock_count (for locking viaconsole_trylock()).Remove @console_locked and update
is_console_locked()to use the alternative variables.
It is great that we are also removing something ;-)
Best Regards,
Petr
John Ogness
maintainerPATCH 13/13 console: introduce CON_MIGHT_SLEEP for vt
In reply to John OgnessDeadlocks and the framebuffer console have been a recurring issue that is getting worse. Daniel Vetter suggested [0] that fbcon->write() should no longer be called from an atomic context.
Introduce a new console flag CON_MIGHT_SLEEP for a console driver to specify that it is only called from sleepable contexts. Set the fbcon to use this new flag.
[0] https://lore.kernel.org/all/[email protected]
--- drivers/tty/vt/vt.c | 2 +- include/linux/console.h | 1 + kernel/printk/printk.c | 2 ++ 3 files changed, 4 insertions(+), 1 deletion(-)
drivers/tty/vt/vt.c +1 -1
@@ -3161,7 +3161,7 @@ static struct console vt_console_driver = { .write = vt_console_print, .device = vt_console_device, .unblank = unblank_screen, - .flags = CON_PRINTBUFFER, + .flags = CON_PRINTBUFFER|CON_MIGHT_SLEEP, .index = -1, };#endif
include/linux/console.h +1 -0
@@ -138,6 +138,7 @@ static inline int con_debug_leave(void)#define CON_BRL (32)/* Used for a braille device */ #define CON_EXTENDED (64)/* Use the extended output format a la /dev/kmsg */ #define CON_PAUSED (128)/* Sleep while console is locked */ +#define CON_MIGHT_SLEEP (256)/* Can only be called from sleepable context */ struct console {char name[16 ];
kernel/printk/printk.c +2 -0
@@ -2836,6 +2836,8 @@ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handoveif (!console_is_usable (con))continue ; +if ((con->flags &CON_MIGHT_SLEEP ) && !do_cond_resched) +continue ; any_usable = true;if (con->flags &CON_EXTENDED ) {
Signature · 1 line
2.30.2
Petr Mladek
maintainerRe: [PATCH printk v1 13/13] console: introduce CON_MIGHT_SLEEP for vt
Quoted 3 lines · John Ogness
Deadlocks and the framebuffer console have been a recurring issue that is getting worse. Daniel Vetter suggested [0] that
fbcon->write()should no longer be called from an atomic context.
We should make it clear that people will not longer see kernel messages on ttyX during early boot, panic, and in some other situations when printk kthreads are not available.
Or do I miss something?
Do we really want this?
Do the tty maintainers really want to give up on supporting modes when processes/kthreads do not work?
Maybe, it is inevitable. Maybe, people are not using tty for debugging too much.
Anyway, this change has to be approved by tty guys.
Best Regards,
Petr
Quoted 43 lines
Introduce a new console flag
CON_MIGHT_SLEEPfor a console driver to specify that it is only called from sleepable contexts. Set the fbcon to use this new flag.[0] https://lore.kernel.org/all/[email protected]
Signed-off-by John Ogness <[email protected]>--- drivers/tty/vt/vt.c | 2 +- include/linux/console.h | 1 + kernel/printk/printk.c | 2 ++ 3 files changed, 4 insertions(+), 1 deletion(-)drivers/tty/vt/vt.c +1
-1@@ -3161,7 +3161,7 @@ static struct console vt_console_driver = { .write = vt_console_print, .device = vt_console_device, .unblank = unblank_screen, - .flags = CON_PRINTBUFFER, + .flags = CON_PRINTBUFFER|CON_MIGHT_SLEEP, .index = -1, };#endif include/linux/console.h +1
-0@@ -138,6 +138,7 @@ static inline int con_debug_leave(void)#define CON_BRL (32)/* Used for a braille device */ #define CON_EXTENDED (64)/* Use the extended output format a la /dev/kmsg */ #define CON_PAUSED (128)/* Sleep while console is locked */ +#define CON_MIGHT_SLEEP (256)/* Can only be called from sleepable context */ struct console {char name[16 ];kernel/printk/printk.c +0
-0
kernel/printk/printk.c static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handove · line 2840
Quoted 5 lines
kernel/printk/printk.c +2
-0@@ -2836,6 +2836,8 @@ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handoveif (!console_is_usable (con))continue ; +if ((con->flags &CON_MIGHT_SLEEP ) && !do_cond_resched) +continue ;
This means that ttyX will be able to show the messages only from a process context. It it will not longer show the messages during, early boot, panic, suspend, and some other situations.
Do we really want this?
Do the tty maintainers really want to give up support in these modes when processes/kthreads do not work?
Maybe, it is inevitable. Maybe, people are not using tty for debugging too much.
Anyway, this has to be
Quoted 5 lines
kernel/printk/printk.c +0
-0any_usable = true;if (con->flags &CON_EXTENDED ) {Signature · 1 line
2.30.2
Greg Kroah-Hartman
maintainerRe: [PATCH printk v1 13/13] console: introduce CON_MIGHT_SLEEP for vt
Quoted 9 lines · Petr Mladek
Quoted 3 lines · John Ogness
Deadlocks and the framebuffer console have been a recurring issue that is getting worse. Daniel Vetter suggested [0] that
fbcon->write()should no longer be called from an atomic context.We should make it clear that people will not longer see kernel messages on ttyX during early boot, panic, and in some other situations when printk kthreads are not available.
Quoted 6 lines
Or do I miss something?
Do we really want this?
Do the tty maintainers really want to give up on supporting modes when processes/kthreads do not work?
I would love that, but:
Quoted 2 lines
Maybe, it is inevitable. Maybe, people are not using tty for debugging too much.
People are using tty consoles for debugging all the time. We can not break that.
thanks,
greg k-h
A message that is not in the archive (sent off-list or not yet received): [email protected]
John Ogness
maintainerRe: [PATCH printk v1 10/13] printk: add kthread console printers
Quoted 8 lines · Hillf Danton
Quoted 4 lines · On Mon, 7 Feb 2022 20:49:20 +0106 John Ogness wrote:
Create a kthread for each console to perform console printing. During normal operation (@system_state ==
SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.The code becomes simpler if the kthread is replaced with a workqueue work, given the system workers.
We do not want multiple console printers blocking each other. We also would not want the consoles blocking non-printk work items for extended periods of time. So we would likely need a dedicated worker per console. I'm not convinced the code would look simpler just by changing the deferred work API. But perhaps the kernel community wants to get away from manual kthread management in general. (??)
I am now pondering if there is some technical advantages to workqueues. I suppose with dedicated workers, you could still easily adjust CPU affinities and scheduling policies/priorites from userspace, like with kthreads. But is there some advantage over kthreads?
John Ogness
Petr Mladek
maintainerRe: [PATCH printk v1 10/13] printk: add kthread console printers
Quoted 17 lines · John Ogness
Quoted 8 lines · Hillf Danton
Quoted 4 lines · On Mon, 7 Feb 2022 20:49:20 +0106 John Ogness wrote:
Create a kthread for each console to perform console printing. During normal operation (@system_state ==
SYSTEM_RUNNING), the kthread printers are responsible for all printing on their respective consoles.The code becomes simpler if the kthread is replaced with a workqueue work, given the system workers.
We do not want multiple console printers blocking each other. We also would not want the consoles blocking non-printk work items for extended periods of time. So we would likely need a dedicated worker per console. I'm not convinced the code would look simpler just by changing the deferred work API. But perhaps the kernel community wants to get away from manual kthread management in general. (??)
Quoted 4 lines
I am now pondering if there is some technical advantages to workqueues. I suppose with dedicated workers, you could still easily adjust CPU affinities and scheduling policies/priorites from userspace, like with kthreads. But is there some advantage over kthreads?
It is true that custom kthread implementation is a bit tricky. Especially it is easy to create races. Also they are often not optimal from the resources POV.
But I do not see any big advantage in this case. It would likely bring more problems than it would solve.
Note that many people rely on debugging kernel via printk() and consoles. Non-working consoles switch the system into a black box.
My concern is:
+ Workqueues code is very tricky and a lot of operations require pool->lock. There is even a hook from the scheduler. It would be yet another hard-to-debug layer when messages do not reach console. Another source of problems when handling consoles during panic. Another source of recursive messages.
+ As John already suggested. We would require rescue kthreads. It means that it won't safe any resources. Also rescuers are woken via the last idle worker and timer, using another tricky code.
+ There might be real problems to see messages on overloaded system. Workqueues have very limited support how to deal with it. There are basically only few possibilities: per-CPU workers vs. workers limited by a CPU-mask. Normal vs. hi-priority workers (nothing in between). The priority could not be modified at runtime.
A more realistic approach would be using the kthread_worker API. It was designed for users that need dedicated kthreads, custom scheduling policy and priority. It provides a generic loop. The work items are queued the same way as in the workqueues API.
My mine concern is that the kthread_worker API still uses an internal lock. And it is yet another layer that might be hard to debug when printk() does not work.
I would prefer using a custom kthread for now. The main reason is that it is waken using wake_up_process(). We will have to deal with very similar problems as with console_lock (semaphore locking) that uses wake_up_process() as well.
Best Regards,
Petr
Sergey Senozhatsky
reviewerRe: [PATCH printk v1 10/13] printk: add kthread console printers
Quoted 3 lines · Petr Mladek
My mine concern is that the
kthread_workerAPI still uses an internal lock. And it is yet another layer that might be hard to debug whenprintk()does not work.
Isn't this also true for kthreads? Looks like we add "couple of spin_locks" to the picture - rq, etc. - one way of the other.
Petr Mladek
maintainerRe: [PATCH printk v1 10/13] printk: add kthread console printers
Quoted 7 lines · Sergey Senozhatsky
Quoted 3 lines · Petr Mladek
My mine concern is that the
kthread_workerAPI still uses an internal lock. And it is yet another layer that might be hard to debug whenprintk()does not work.Isn't this also true for kthreads? Looks like we add "couple of
spin_locks" to the picture - rq, etc. - one way of the other.
kthread_worker API is built on top of kthreads. It means one more lock, one more layer to deal with.
If anyone wants to propose workqueues or kthread_worker API, please, come up with a code and proof that it makes things easier.
It is fine to mention this possibility. It is fine to consider it. It is fine to keep it in mind. But I do not want to spent too much time on theoretic discussions at the moment. We have a code now. John has spent non-trivial time on it. Let's concentrate on review.
Best Regards,
Petr