Skip to contentSkip to conversation

printk: Skip console drivers on PREEMPT_RT.

PATCH Superseded Sebastian Andrzej Siewior 3 participants 15 messages Jul 20, 2022 – Jul 26, 2022 Printk

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 printk: Skip console drivers on PREEMPT_RT.

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This is behaviour is not compatible with PREEMPT_RT.

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

---
 kernel/printk/printk.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/printk/printk.c +10 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		up_console_sem();
+		return;
+	}
+
+	/*
 	 * Console drivers are called with interrupts disabled, so
 	 * @console_may_schedule should be cleared before; however, we may
 	 * end up dumping a lot of lines, for example, if called from
maintainer
#1 lore raw

Hi Sebastian,

Quoted 16 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This is behaviour is not compatible with PREEMPT_RT.

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

---
 kernel/printk/printk.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

kernel/printk/printk.c void console_unlock(void) · line 2851

Quoted 9 lines
kernel/printk/printk.c +6 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		up_console_sem();

This should be:

                __console_unlock();

kernel/printk/printk.c void console_unlock(void) · line 2853

Quoted 2 lines
kernel/printk/printk.c +2 -0
+		return;
+	}

Note that if @console_may_schedule is 1, then we are in a sleepable context and could print. But since that is not very often, it is probably better to just have it off all the time as you propose.

John Ogness

#2 lore raw
Quoted 1 line · John Ogness

Hi Sebastian,

Hi,

Quoted 15 lines

kernel/printk/printk.c void console_unlock(void) · line 2851

Quoted 12 lines
kernel/printk/printk.c +6 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		up_console_sem();

This should be:

Quoted 1 line
                __console_unlock();

Why?

Quoted 6 lines

kernel/printk/printk.c void console_unlock(void) · line 2853

Quoted 2 lines
kernel/printk/printk.c +2 -0
+		return;
+	}

Note that if @console_may_schedule is 1, then we are in a sleepable context and could print. But since that is not very often, it is probably better to just have it off all the time as you propose.

No we can't coz printk disables interrupts before invoking the console drivers.

Quoted 1 line

John Ogness

Sebastian

maintainer
#3 lore raw
Quoted 15 lines · Sebastian Andrzej Siewior
Quoted 15 lines

kernel/printk/printk.c void console_unlock(void) · line 2851

Quoted 12 lines
kernel/printk/printk.c +6 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		up_console_sem();

This should be:

Quoted 3 lines
Quoted 1 line
                __console_unlock();

Why?

Because that is the official function that performs the console unlocking. With upcoming patches, that function does more. But even without those patches, by directly calling up_console_sem() you are leaving @console_locked set. The @console_locked variable is used by the vt code to validate correct console locking.

I realize you are just copy/pasting from the condition above (checking @console_suspended), but that condition is special. When consoles are suspended, we still allow @console_sem to be locked/unlocked, but without changing the locked status of the console. See suspend_console() and resume_console() if you really want to see the gory details.

John

#4 lore raw

PATCH v2 1/1 printk: Skip console drivers on PREEMPT_RT.

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This is behaviour is not compatible with PREEMPT_RT.

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

---
v1…v2:
- Use __console_unlock() as suggested by John.

 kernel/printk/printk.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/printk/printk.c +10 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		__console_unlock();
+		return;
+	}
+
+	/*
 	 * Console drivers are called with interrupts disabled, so
 	 * @console_may_schedule should be cleared before; however, we may
 	 * end up dumping a lot of lines, for example, if called from
maintainer
#5 lore raw
Quoted 3 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This is behaviour is not compatible with PREEMPT_RT.

Maybe I do not understand it correctly. It sounds like we could not disable interrupts when interrupts or preemption is already disabled. Like nested disablement of interrupts is bad.

Is this a generic rule? Is is about the nesting?

Or is is somehow specific to the console drivers called from printk() directly? Do you always want to disable here because it might be an atomic context and they might take too long?

I guess that the sentence "additionally disables interrupts before it invokes the console drivers" is not really important" and it confused me.

Quoted 3 lines

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

I guess that this is for RT tree because the kthreads and the atomic consoles are still not in the mainline.

Quoted 4 lines
kernel/printk/printk.c +0 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}

kernel/printk/printk.c void console_unlock(void) · line 2853

Quoted 10 lines
kernel/printk/printk.c +8 -0
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		__console_unlock();
+		return;
+	}

Do you want this even in panic() or early boot?

AFAIK, only the serial console has atomic write() callback in the RT tree. Is this the only console used by RT kernel users in practice?

kernel/printk/printk.c void console_unlock(void) · line 2858

Quoted 4 lines
kernel/printk/printk.c +1 -0
+	/*
 	 * Console drivers are called with interrupts disabled, so
 	 * @console_may_schedule should be cleared before; however, we may
 	 * end up dumping a lot of lines, for example, if called from

Best Regards,
Petr

#6 lore raw
Quoted 10 lines · Petr Mladek
Quoted 3 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This is behaviour is not compatible with PREEMPT_RT.

Maybe I do not understand it correctly. It sounds like we could not disable interrupts when interrupts or preemption is already disabled. Like nested disablement of interrupts is bad.

Is this a generic rule? Is is about the nesting?

You must not invoke the console drivers with disabled interrupts. This is bad. So even if the context you were called from has interrupts enabled then in console_emit_next_record() you have printk_safe_enter_irqsave() before call_console_driver() which disables interrupts and this is bad. More below…

Quoted 3 lines

Or is is somehow specific to the console drivers called from printk() directly? Do you always want to disable here because it might be an atomic context and they might take too long?

You can't acquire a sleeping lock with disabled interrupts and or preemption. Therefore the console drivers must not be invoked because they need to acquire a sleeping lock(s).

Quoted 2 lines

I guess that the sentence "additionally disables interrupts before it invokes the console drivers" is not really important" and it confused me.

This refers to printk_safe_enter_irqsave(). You could argue that it is safe to invoke the console drivers if the context, in which printk() is invoked, is safe. However this is not possible because printk disables interrupts prio invoking the console drivers as just explained. Therefore I don't see a way how to invoke the console drivers on RT as of v5.19-rc7.

Quoted 7 lines
Quoted 4 lines

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

I guess that this is for RT tree because the kthreads and the atomic consoles are still not in the mainline.

I would like to have this applied to the v5.20 upstream tree and then revoked once the missing bits have been merged. Based on what I see, there shouldn't be any road blocks.

Quoted 5 lines
Quoted 5 lines
kernel/printk/printk.c +0 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
Quoted 11 lines

kernel/printk/printk.c void console_unlock(void) · line 2853

Quoted 9 lines
kernel/printk/printk.c +8 -0
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		__console_unlock();
+		return;
+	}

Do you want this even in panic() or early boot?

yes. I don't see a way to invoke the console drivers without the print-kthread.

Quoted 2 lines

AFAIK, only the serial console has atomic write() callback in the RT tree. Is this the only console used by RT kernel users in practice?

The atomic console is made for emergencies. Everything else should be written using the printk thread.

Quoted 8 lines

kernel/printk/printk.c void console_unlock(void) · line 2858

Quoted 5 lines
kernel/printk/printk.c +1 -0
+	/*
 	 * Console drivers are called with interrupts disabled, so
 	 * @console_may_schedule should be cleared before; however, we may
 	 * end up dumping a lot of lines, for example, if called from

Best Regards,
Petr

Sebastian

maintainer
#7 lore raw
Quoted 6 lines · Sebastian Andrzej Siewior
Quoted 5 lines · Petr Mladek
Quoted 3 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This is behaviour is not compatible with PREEMPT_RT.

Quoted 6 lines
Quoted 4 lines

Maybe I do not understand it correctly.

Or is is somehow specific to the console drivers called from printk() directly?

You can't acquire a sleeping lock with disabled interrupts and or

This is what I have missed. It might be obvious for people living by RT kernel. But spinlocks do not sleep in normal kernel. So I did not get that the spinlocks are the culprit. Please, make it more obvious in the commit message, for example:

--- cut ---
Console drivers use spinlocks that might sleep in PREEMPT_RT. As a result they must not be called with interrupts enabled... --- cut ---

Quoted 10 lines
Quoted 6 lines
Quoted 3 lines

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

I guess that this is for RT tree because the kthreads and the atomic consoles are still not in the mainline.

I would like to have this applied to the v5.20 upstream tree and then revoked once the missing bits have been merged. Based on what I see, there shouldn't be any road blocks.

Huh, I do not think that it is a good idea. There are neither atomic consoles nor printk kthreads in upstream. The patch effectively completely disables the consoles in PREEMPT_RT. All consoles will be _empty all the time_.

Also the consoles were never tested with interrupts enabled. I am pretty sure that interrupts has to be disabled in some locations, for example, when sending some data sequences on the ports. Are we sure that they are always explicitly disabled inside the drivers? This is one reason why we reverted the console kthreads in 5.19-rc4. AFAIK, John is doing some inspection of all drivers.

That said, I am going to leave the decision on John. I am not sure what are the expectations of RT users in mainline.

From my point, this patch does not make much sense. IMHO, it will not make mainline usable with PREEMPT_RT. Any serious RT user will need to revert it and apply a better printk solution from the out-of-tree RT patchset.

Best Regards,
Petr

maintainer
#8 lore raw

Hi Petr,

Quoted 4 lines · Petr Mladek

From my point, this patch does not make much sense. IMHO, it will not make mainline usable with PREEMPT_RT. Any serious RT user will need to revert it and apply a better printk solution from the out-of-tree RT patchset.

The problem is that direct console printing cannot work with PREEMPT_RT, even in a panic situation. And it never will (using "normal" console drivers).

PREEMPT_RT is basically mainline now except for the kthread printers and atomic consoles. But these features will not be available in mainline so soon. (Atomic consoles have not gone through the LKML review process at all yet.)

We see value in allowing PREEMPT_RT to be available now, even if it means no console printing. You claim "any serious RT user" needs console printing, but for production RT systems, the console is probably disabled anyway.

We could solve this with kconfig tricks, making console kconfig options depend on !PREEMPT_RT. But that would cause the console config of users to be disabled, making it more inconvenient for them to turn it back on once these features are available.

A software switch (as implemented by Sebastian's patch) may provide the simplest transition. A 5.20 PREEMPT_RT would not have console printing, but a later version (using the same config) would.

John

#9 lore raw
In reply to Petr Mladek
Quoted 9 lines · Petr Mladek

This is what I have missed. It might be obvious for people living by RT kernel. But spinlocks do not sleep in normal kernel. So I did not get that the spinlocks are the culprit. Please, make it more obvious in the commit message, for example:

--- cut ---
Console drivers use spinlocks that might sleep in PREEMPT_RT. As a result they must not be called with interrupts enabled... --- cut ---

It is not only the sleeping lock by itself but also the fact that polling on 115200 or 9600 to print a line is simply takes too much time. Even if the line consists of four letters.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

Disable console printing until the return of atomic consoles and the printing thread. This allows to retrieve the log buffer from user space which is not possible by disable printk.

I guess that this is for RT tree because the kthreads and the atomic consoles are still not in the mainline.

Quoted 8 lines
Quoted 3 lines

I would like to have this applied to the v5.20 upstream tree and then revoked once the missing bits have been merged. Based on what I see, there shouldn't be any road blocks.

Huh, I do not think that it is a good idea. There are neither atomic consoles nor printk kthreads in upstream. The patch effectively completely disables the consoles in PREEMPT_RT. All consoles will be _empty all the time_.

I am aware of that, I tested that patch (as in I didn't just send it). This is what I see on bare metal's UART:

|  Booting `Debian GNU/Linux'
|
| Loading Linux RT ...
| Loading initial ramdisk ...
| No EFI environment detected.
| early console in extract_kernel
| input_data: 0x00000000031d12e0
| input_len: 0x0000000000c283e1
| output: 0x0000000001000000
| output_len: 0x00000000025f49f8
| kernel_total_size: 0x0000000002e26000
| needed_size: 0x0000000003000000
| trampoline_32bit: 0x0000000000099000
| 
| Decompressing Linux... Parsing ELF... done.
| Booting the kernel.
| Loading, please wait...
| Starting version 251.3-1
| Begin: Loading essential drivers ... done.

…

and this is okay for me at this point. This means that v5.20 could have RT enabled for x86 and people can start test it without additional patches. They can enable it and if it boots they can check dmesg's output for anything odd. If it doesn't boot they can either wait for the next release or grab the patchset with the missing bits for debugging. This is way better than letting printk depend on !RT which would provide no insight at all. The RT option is still hidden behind EXPERT.

Quoted 4 lines

Also the consoles were never tested with interrupts enabled. I am pretty sure that interrupts has to be disabled in some locations, for example, when sending some data sequences on the ports. Are we sure that they are always explicitly disabled inside the drivers?

The 8250 did disable interrupts via local_irq_disable() and this is not desired on RT since it *really* disables interrupts. This was long ago addressed in commit
ebade5e833eda ("serial: 8250: Clean up the locking for -rt")

The interrupts are enabled in general on RT and they have to. I did not remove this pattern (local_irq_disable() + try_lock()) from all console drivers because five years ago we were talking about explicit entry points for magic-sysrq and oops_in_progress and I didn't want to patch around until it is done. I don't know what John's plans here are but I did mention this to him a while ago.
A few other (ARM related drivers) are patched as part of the RT queue.

Quoted 2 lines

This is one reason why we reverted the console kthreads in 5.19-rc4. AFAIK, John is doing some inspection of all drivers.

The console drivers (or drivers in general) never *really* disable interrupts on RT via spin_lock_irq*() and this is intended.

Quoted 7 lines

That said, I am going to leave the decision on John. I am not sure what are the expectations of RT users in mainline.

From my point, this patch does not make much sense. IMHO, it will not make mainline usable with PREEMPT_RT. Any serious RT user will need to revert it and apply a better printk solution from the out-of-tree RT patchset.

The lack of the console probably makes RT not production ready as of v5.20 but I was debugging systems without a UART so.
It might not be production ready without a console. Also ARM and PowerPC can not be enabled so there will be a queue for a while unless people device that they don't care about what is left. However it will hopefully will attract x86 people to give it a spin. Then we may receive bug reports and or patches which we did not before and so get better.

Quoted 2 lines

Best Regards,
Petr

Sebastian

maintainer
#10 lore raw
Quoted 6 lines · Sebastian Andrzej Siewior
Quoted 5 lines · Petr Mladek

This is what I have missed. It might be obvious for people living by RT kernel. But spinlocks do not sleep in normal kernel. So I did not get that the spinlocks are the culprit. Please, make it more obvious in the commit message, for example:

Quoted 8 lines
Quoted 4 lines

--- cut ---
Console drivers use spinlocks that might sleep in PREEMPT_RT. As a result they must not be called with interrupts enabled... --- cut ---

It is not only the sleeping lock by itself but also the fact that polling on 115200 or 9600 to print a line is simply takes too much time. Even if the line consists of four letters.

I see.

Quoted 19 lines
Quoted 4 lines

Huh, I do not think that it is a good idea. There are neither atomic consoles nor printk kthreads in upstream. The patch effectively completely disables the consoles in PREEMPT_RT. All consoles will be _empty all the time_.

I am aware of that, I tested that patch (as in I didn't just send it). This is what I see on bare metal's UART:

|  Booting `Debian GNU/Linux'
|
| Loading Linux RT ...
| Loading initial ramdisk ...
| No EFI environment detected.
| early console in extract_kernel
| input_data: 0x00000000031d12e0
| input_len: 0x0000000000c283e1
| output: 0x0000000001000000
| output_len: 0x00000000025f49f8
| kernel_total_size: 0x0000000002e26000
Quoted 8 lines
| needed_size: 0x0000000003000000
| trampoline_32bit: 0x0000000000099000
| 
| Decompressing Linux... Parsing ELF... done.
| Booting the kernel.
| Loading, please wait...
| Starting version 251.3-1
| Begin: Loading essential drivers ... done.

I see. These messages are pushed to the serial console directly.

Quoted 7 lines

and this is okay for me at this point. This means that v5.20 could have RT enabled for x86 and people can start test it without additional patches. They can enable it and if it boots they can check dmesg's output for anything odd. If it doesn't boot they can either wait for the next release or grab the patchset with the missing bits for debugging. This is way better than letting printk depend on !RT which would provide no insight at all. The RT option is still hidden behind EXPERT.

Makes sense.

Quoted 9 lines
Quoted 4 lines

Also the consoles were never tested with interrupts enabled. I am pretty sure that interrupts has to be disabled in some locations, for example, when sending some data sequences on the ports. Are we sure that they are always explicitly disabled inside the drivers?

The 8250 did disable interrupts via local_irq_disable() and this is not desired on RT since it *really* disables interrupts. This was long ago addressed in commit
ebade5e833eda ("serial: 8250: Clean up the locking for -rt")

Good to know.

Quoted 7 lines

The lack of the console probably makes RT not production ready as of v5.20 but I was debugging systems without a UART so.
It might not be production ready without a console. Also ARM and PowerPC can not be enabled so there will be a queue for a while unless people device that they don't care about what is left. However it will hopefully will attract x86 people to give it a spin. Then we may receive bug reports and or patches which we did not before and so get better.

OK, it all makes sense now. Could you please send v3 with an updated commit message?

It should explain why it is not acceptable to call console drivers with disabled interrupts in RT (spinlocks are sleeping locks, pooling takes too long).

It should make clear the effect of the patch. printk() messages will never reach console. They will be accessible only from userspace (dmesg).

Finally, it should mention that this is only a temporary solution. It allows to boot PREEMPT_RT mainline kernel without extra patches. The consoles will later get enabled again by calling console drivers with interrupts enabled from dedicated kthreads. Also it will be possible to call consoles directly by atomic consoles drivers.

If it gets acked by John then I could queue it for 5.20.

Thanks for explaining all the details.

Best Regards,
Petr

#11 lore raw

PATCH v3 1/1 printk: Skip console drivers on PREEMPT_RT.

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This behaviour is not desired on PREEMPT_RT:
- The console driver are using spinlock_t based locking which become sleeping locks on PREEMPT_RT and must not be acquired with disabled interrupts (or preemption).

- The locks within the console drivers must remain sleeping locks and they must not disable interrupts. Printing (and polling for its completion) at 115200 baud on an UART takes too long for PREEMPT_RT in general and so raises the latency of the IRQ-off time of the system beyond acceptable levels.

Skip printing to the console as temporary workaround until the printing threads and atomic consoles have been introduced or another solution which is compatible with the PREEMPT_RT approach.
With this change, the user will not see any kernel message printed to the console but can retrieve the printk buffer from userland (via the dmesg command). This allows enable PREEMPT_RT as a whole without disabling printk and loosing all kernel output.

Disable console printing on PREEMPT_RT.

---
v2…v3:
- Reword commit message by adding a few details/ explanations.

v1…v2:
- Use __console_unlock() as suggested by John.

 kernel/printk/printk.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/printk/printk.c +10 -0
@@ -2843,6 +2843,16 @@ void console_unlock(void)
 	}
 
 	/*
+	 * On PREEMPT_RT it is not possible to invoke console drivers with
+	 * disabled interrupts and or preemption. Therefore all drivers are
+	 * skipped and the output can be retrieved from the buffer.
+	 */
+	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		__console_unlock();
+		return;
+	}
+
+	/*
 	 * Console drivers are called with interrupts disabled, so
 	 * @console_may_schedule should be cleared before; however, we may
 	 * end up dumping a lot of lines, for example, if called from
maintainer Reviewed-by on v3 1/1
#12 lore raw
Quoted 20 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This behaviour is not desired on PREEMPT_RT:
- The console driver are using spinlock_t based locking which become sleeping locks on PREEMPT_RT and must not be acquired with disabled interrupts (or preemption).

- The locks within the console drivers must remain sleeping locks and they must not disable interrupts. Printing (and polling for its completion) at 115200 baud on an UART takes too long for PREEMPT_RT in general and so raises the latency of the IRQ-off time of the system beyond acceptable levels.

Skip printing to the console as temporary workaround until the printing threads and atomic consoles have been introduced or another solution which is compatible with the PREEMPT_RT approach.
With this change, the user will not see any kernel message printed to the console but can retrieve the printk buffer from userland (via the dmesg command). This allows enable PREEMPT_RT as a whole without disabling printk and loosing all kernel output.

Quoted 3 lines

Disable console printing on PREEMPT_RT.

Thanks a lot for updating the commit message. It looks good to me now.

Best Regards,
Petr

PS: John, if you ack it then I'll queue it for 5.20.

maintainer Reviewed-by on v3 1/1
#13 lore raw
In reply to Sebastian Andrzej Siewior
Quoted 12 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This behaviour is not desired on PREEMPT_RT:
- The console driver are using spinlock_t based locking which become sleeping locks on PREEMPT_RT and must not be acquired with disabled interrupts (or preemption).

- The locks within the console drivers must remain sleeping locks and they must not disable interrupts. Printing (and polling for its completion) at 115200 baud on an UART takes too long for PREEMPT_RT in general and so raises the latency of the IRQ-off time of the system beyond acceptable levels.

Quoted 7 lines

Skip printing to the console as temporary workaround until the printing threads and atomic consoles have been introduced or another solution which is compatible with the PREEMPT_RT approach.
With this change, the user will not see any kernel message printed to the console but can retrieve the printk buffer from userland (via the dmesg command). This allows enable PREEMPT_RT as a whole without disabling printk and loosing all kernel output.

Note that "the dmesg command" is not the only userspace tool to access the kernel logs. Logging daemons (using /proc/kmsg or /dev/kmsg) also have full access.

Quoted 3 lines

Disable console printing on PREEMPT_RT.

maintainer
#14 lore raw
Quoted 21 lines · John Ogness
Quoted 12 lines · Sebastian Andrzej Siewior

printk might be invoked in a context with disabled interrupts and or preemption and additionally disables interrupts before it invokes the console drivers. This behaviour is not desired on PREEMPT_RT:
- The console driver are using spinlock_t based locking which become sleeping locks on PREEMPT_RT and must not be acquired with disabled interrupts (or preemption).

- The locks within the console drivers must remain sleeping locks and they must not disable interrupts. Printing (and polling for its completion) at 115200 baud on an UART takes too long for PREEMPT_RT in general and so raises the latency of the IRQ-off time of the system beyond acceptable levels.

Quoted 7 lines

Skip printing to the console as temporary workaround until the printing threads and atomic consoles have been introduced or another solution which is compatible with the PREEMPT_RT approach.
With this change, the user will not see any kernel message printed to the console but can retrieve the printk buffer from userland (via the dmesg command). This allows enable PREEMPT_RT as a whole without disabling printk and loosing all kernel output.

Quoted 3 lines

Note that "the dmesg command" is not the only userspace tool to access the kernel logs. Logging daemons (using /proc/kmsg or /dev/kmsg) also have full access.

I have updated the message when comitting, see
https://git.kernel.org/pub/scm/linux/kernel/git/printk/linux.git/commit/?h=rework/kthreads&id=c01c1c784a02aaa216524977b294b8834d0ee907

Quoted 5 lines
Quoted 3 lines

Disable console printing on PREEMPT_RT.

JFYI, the patch has been committed into printk/linux.git, branch rework/kthreads. I am going to add it into the pull request for 5.20.

Best Regards,
Petr