Skip to contentSkip to conversation

Late KS topics

Ksummit-discuss Hannes Reinecke 6 participants 8 messages Oct 26, 2015 – Oct 27, 2015 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

(I just realized that I should've sent it to ksummit-discuss, too.
So here it is.)

Hi Ted,

here are the two things I'd like to bring up:

- Asynchronous probing
Currently, the driver model has the simple match/probe mechanism by
which individual drivers can attach to device. As per design, these device are enumerated by the 'bus', and the match/probe pair is executed for each device. As this is a simple loop, each match/probe pair is executed sequentially.
This creates quite a significant slowdown on large installations, where eg. the PCI bus is scanned sequentially, and probe() is called for every device, although in fact each device is totally independent, so there isn't really a need for the probe function to be serialized. I would like to discuss the implications of moving to asynchronous probing in general, either per bus in with the driver core in general.

The topic will be presented by me an Luis Rodriguez; possible slot would be 11 or 14.

The other topic is a rather short one, which possibly could be move to a lightning talk:
- printk usecases
Currently, printk is used for two largely different usecases:
1) Really urgent, 'ship me out at any cost now' messages and
2) Rather longish, 'and incidentally I always wanted to tell you what I did on my recent holidays' type of logging entries. As it so happens, this really makes it hard to do any improvements for printk as it inevitably messes up one or the other side. (This whole idea came up as we've had some instances which simply could not boot as the serial console took too long to print out all the stuff, so systemd timeout kicked in and killed the udev process which should have handled the boot device.)
The talk/proposal should be used to come to a consensus about restricting printk() to high-priority, small volume messages, and implement a different call (like log_printk()) for the low-priority, high-volume traffic.

Cheers,

Hannes

#1 lore raw
Quoted 13 lines · Hannes Reinecke

The other topic is a rather short one, which possibly could be move to
a lightning talk:
- printk usecases
Currently, printk is used for two largely different usecases:
1) Really urgent, 'ship me out at any cost now' messages and
2) Rather longish, 'and incidentally I always wanted to tell you what I did on my recent holidays' type of logging entries. As it so happens, this really makes it hard to do any improvements for printk as it inevitably messes up one or the other side.

Quoted 8 lines

(This whole idea came up as we've had some instances which simply
could not boot as the serial console took too long to print out all the stuff, so systemd timeout kicked in and killed the udev process which should have handled the boot device.)
The talk/proposal should be used to come to a consensus about restricting printk() to high-priority, small volume messages, and implement a different call (like log_printk()) for the low-priority, high-volume traffic.

Currently, the former doesn't work. As lamented recently, there's no synchronous printk even over serial console any more. It's lossy, and you can't even do the traditional printk debugging any more with
printk(); is_this_the_bit_that_kills_the_machine(); printk();
because the first printk might not be *seen* before the machine dies.

Signature · 1 line
dwmw2
#2 lore raw
In reply to Hannes Reinecke
Quoted 4 lines · Hannes Reinecke

The talk/proposal should be used to come to a consensus about restricting printk() to high-priority, small volume messages, and implement a different call (like log_printk()) for the low-priority, high-volume traffic.

The printk() troubles are not only about the "high volume" vs. "very important", but also about the context of the caller.

The optimistic comment above printk() says that it can be called from any context, but that's not true for quite some time already.
So if printk() is going to be redesigned to be more lightweight, this aspect definitely needs to be taken into consideration as well.

Signature · 2 lines
Jiri Kosina
SUSE Labs
#4 lore raw
In reply to Hannes Reinecke
Quoted 4 lines

(This whole idea came up as we've had some instances which simply
could not boot as the serial console took too long to print out all the stuff, so systemd timeout kicked in and killed the udev process which should have handled the boot device.)

What is the default value for that timeout? Is there a configuration file where I can easily change it?

I think I may have seen this when I added some verbose debugging printk() messages.

-Tony

#6 lore raw
In reply to Hannes Reinecke
Quoted 6 lines · Hannes Reinecke

- Asynchronous probing
Currently, the driver model has the simple match/probe mechanism by
which individual drivers can attach to device. As per design, these device are enumerated by the 'bus', and the match/probe pair is executed for each device. As this is a simple loop, each match/probe pair is executed sequentially.

There's also the deferred probing and general device ordering thing I mailed Ted about yesterday - it's a different issue to async probing, there's a big thread on the lists about one proposal for a subset of these issues many of the participants of which are here.

#5 lore raw
In reply to Luck, Tony
Quoted 8 lines · Tony
Quoted 4 lines

(This whole idea came up as we've had some instances which simply
could not boot as the serial console took too long to print out all the stuff, so systemd timeout kicked in and killed the udev process which should have handled the boot device.)

What is the default value for that timeout? Is there a configuration file where I can easily change it?

That's the udev worker timeout, which can be modified with later versions of udev IIRC.

Quoted 3 lines

I think I may have seen this when I added some verbose debugging printk() messages.

Oh, that's quite possible. Speaking from painful experience :-)

Cheers,

Hannes

#8 lore raw

[Ksummit-discuss] Deferred probing

In reply to a message not in the archive
Quoted 3 lines · tytso

I didn't see that message, sorry. I found it after spending many minutes searching an overflowing inbox, but it's not something which is scalable.

Please provide clearer information on the process that you are expecting people to follow :(

Quoted 3 lines

If you have specific requests, please use a separate subject line (so people who are massively behind on their e-mail during the conference have some *hope* of finding messages with real-time requests), and

I was hoping that replying to your message asking for new topics and sending to the (different to normal) e-mail address you were requesting was a good way of indicating a new topic.

Quoted 7 lines

please don't reference "subject XXX on the lists" and expect folks to have to read a largish thread and to try to figure out what issues you want to talk about.

Please send a 2-3 sentence summary of the issues you want to discuss --- I'm not sure I understand what needs to be discussed about "deferred probing and general device ordering" this week.

Currently we don't do anything useful to capture inter-device dependencies outside the parent/child relationship. This means that during boot we can get a very largee number of probe attempts which gets messy from a logging point of view and means that during suspend and resume we don't do anything to make sure we're suspending and resuming things in a useful order.

Currently it's all hacked with link order and is very fragile.

#3 lore raw
In reply to Jiri Kosina
Quoted 9 lines · Jiri Kosina
Quoted 4 lines · Hannes Reinecke

The talk/proposal should be used to come to a consensus about restricting printk() to high-priority, small volume messages, and implement a different call (like log_printk()) for the low-priority, high-volume traffic.

The printk() troubles are not only about the "high volume" vs. "very important", but also about the context of the caller.

sometimes "high volume" and 'unlucky' printk_deferred() dynamic duo can do dramatic things. printk_deferred() queues a irq work - wake_up_klogd_work(), which does

wake_up_klogd_work_func():

	..
	console_trylock()
		console_unlock()

so we eventually end up in

	for (;;) {
		..
		call_console_drivers()
		..
	}

	-ss
Quoted 4 lines

The optimistic comment above printk() says that it can be called from any context, but that's not true for quite some time already.
So if printk() is going to be redesigned to be more lightweight, this aspect definitely needs to be taken into consideration as well.