Skip to contentSkip to conversation

+ clocksource-increase-initcall-priority.patch added to -mm tree

DISCUSSION Thomas Gleixner 2 participants 20 messages Oct 8, 2006 – Oct 9, 2006 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 3 lines · On Fri, 2006-10-06 at 18:53 -0700, [email protected] wrote:

Since it's likely that this interface would get used during bootup I moved all the clocksource registration into the postcore initcall. This also eliminated some clocksource shuffling during bootup.

We had the init call in postcore already. John moved it to module init to eliminate trouble with unsynced / unstable TSCs, IIRC.

John, can you please comment on this.

tglx

#2 lore raw
Quoted 9 lines · Thomas Gleixner
Quoted 3 lines · On Fri, 2006-10-06 at 18:53 -0700, [email protected] wrote:

Since it's likely that this interface would get used during bootup I moved all the clocksource registration into the postcore initcall. This also eliminated some clocksource shuffling during bootup.

We had the init call in postcore already. John moved it to module init to eliminate trouble with unsynced / unstable TSCs, IIRC.

John, can you please comment on this.

It also breaks pmtimer.

tglx

#3 lore raw
Quoted 6 lines · Thomas Gleixner
Quoted 5 lines · Thomas Gleixner
Quoted 3 lines · On Fri, 2006-10-06 at 18:53 -0700, [email protected] wrote:

Since it's likely that this interface would get used during bootup I moved all the clocksource registration into the postcore initcall. This also eliminated some clocksource shuffling during bootup.

Quoted 6 lines
Quoted 4 lines

We had the init call in postcore already. John moved it to module init to eliminate trouble with unsynced / unstable TSCs, IIRC.

John, can you please comment on this.

It also breaks pmtimer.

OGAWA reported this already. It breaks the case when there is a verified read needed, instead of the fast read. I'll fix it.

Daniel

#4 lore raw
Quoted 10 lines · Daniel Walker
Quoted 9 lines · Thomas Gleixner
Quoted 8 lines · Thomas Gleixner
Quoted 3 lines · On Fri, 2006-10-06 at 18:53 -0700, [email protected] wrote:

Since it's likely that this interface would get used during bootup I moved all the clocksource registration into the postcore initcall. This also eliminated some clocksource shuffling during bootup.

We had the init call in postcore already. John moved it to module init to eliminate trouble with unsynced / unstable TSCs, IIRC.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

John, can you please comment on this.

It also breaks pmtimer.

OGAWA reported this already. It breaks the case when there is a verified read needed, instead of the fast read. I'll fix it.

I'd like to know, why we need to move that and you did not explain _why_ it is likely that it is used during bootup.

tglx

#5 lore raw
Quoted 13 lines · Thomas Gleixner
Quoted 12 lines · Daniel Walker
Quoted 11 lines · Thomas Gleixner
Quoted 9 lines · Thomas Gleixner
Quoted 3 lines · On Fri, 2006-10-06 at 18:53 -0700, [email protected] wrote:

Since it's likely that this interface would get used during bootup I moved all the clocksource registration into the postcore initcall. This also eliminated some clocksource shuffling during bootup.

We had the init call in postcore already. John moved it to module init to eliminate trouble with unsynced / unstable TSCs, IIRC.

John, can you please comment on this.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

It also breaks pmtimer.

OGAWA reported this already. It breaks the case when there is a verified read needed, instead of the fast read. I'll fix it.

I'd like to know, why we need to move that and you did not explain _why_ it is likely that it is used during bootup.

If the clocksources are registered at the same time as the clocksource users then you end up with users frequently switching clocks during boot up. The original clocksource code solved this by not allowing a real clocksource lookup until after the system fully booted.

However, if you put all the clocksources into postcore initcall, with that being known in advance, and all the users are in lower priority initcalls then you don't need extra code to prevent churn during bootup.

The reason that I think this will get used during boot up is because some of the target users will be instrumentation, and (my prediction anyway) is that some will need to use the interface early. Still even postcore may not be early enough.

Daniel

#6 lore raw
Quoted 7 lines · Daniel Walker
Quoted 2 lines

I'd like to know, why we need to move that and you did not explain _why_ it is likely that it is used during bootup.

If the clocksources are registered at the same time as the clocksource users then you end up with users frequently switching clocks during boot up. The original clocksource code solved this by not allowing a real clocksource lookup until after the system fully booted.

No, that was not the reason. It does not hurt anything, when the clocksource changes during bootup. You simply replace one read out device by another one.

The reason was that we wanted to select clocksources as late as possible, as we wanted to have late init clocksources ready and TSC problems outruled.

Quoted 8 lines

However, if you put all the clocksources into postcore initcall, with that being known in advance, and all the users are in lower priority initcalls then you don't need extra code to prevent churn during bootup.

The reason that I think this will get used during boot up is because some of the target users will be instrumentation, and (my prediction anyway) is that some will need to use the interface early. Still even postcore may not be early enough.

Early bootup Instrumentation is really not a good argument to make that fragile time related stuff even more complex. There is no problem to register reliable clocksources in early bootup, but do not make this mandatory. Not every system is an ARM SoC, where you can and must rely on the one source which is available usually right when the CPU comes up.

Early bootup instrumentation can very well go with none or coarse grained time information, if there is a need to have early boot intstrumentation at all.

tglx

#7 lore raw
Quoted 9 lines · Thomas Gleixner
Quoted 7 lines · Daniel Walker
Quoted 2 lines

I'd like to know, why we need to move that and you did not explain _why_ it is likely that it is used during bootup.

If the clocksources are registered at the same time as the clocksource users then you end up with users frequently switching clocks during boot up. The original clocksource code solved this by not allowing a real clocksource lookup until after the system fully booted.

Quoted 7 lines

No, that was not the reason. It does not hurt anything, when the clocksource changes during bootup. You simply replace one read out device by another one.

The reason was that we wanted to select clocksources as late as possible, as we wanted to have late init clocksources ready and TSC problems outruled.

I think we're going to need to wait for John to comment. His last comment on this patch was in agreement as long as it solved the churn issue (as of Aug. 4 on LKML).

Quoted 9 lines
Quoted 8 lines

However, if you put all the clocksources into postcore initcall, with that being known in advance, and all the users are in lower priority initcalls then you don't need extra code to prevent churn during bootup.

The reason that I think this will get used during boot up is because some of the target users will be instrumentation, and (my prediction anyway) is that some will need to use the interface early. Still even postcore may not be early enough.

Quoted 6 lines

Early bootup Instrumentation is really not a good argument to make that fragile time related stuff even more complex. There is no problem to register reliable clocksources in early bootup, but do not make this mandatory. Not every system is an ARM SoC, where you can and must rely on the one source which is available usually right when the CPU comes up.

It's not mandatory, it's just preferred.. As I said above, to avoid churn. I don't like the churn at boot up, and I tried to make sure there was none added in the patch set.

Daniel

#8 lore raw
Quoted 10 lines · Daniel Walker
Quoted 6 lines

Early bootup Instrumentation is really not a good argument to make that fragile time related stuff even more complex. There is no problem to register reliable clocksources in early bootup, but do not make this mandatory. Not every system is an ARM SoC, where you can and must rely on the one source which is available usually right when the CPU comes up.

It's not mandatory, it's just preferred.. As I said above, to avoid churn. I don't like the churn at boot up, and I tried to make sure there was none added in the patch set.

What churn at bootup ? The clocksources _can_ be switched and it does not matter, when this is done. We had the trouble with the early registration a couple of month ago, and there is no reason to reintroduce it. On systems which have exactly one clocksource, you can register them early in bootup, but please do not touch the x86 setup for no good reason.

tglx

#9 lore raw
Quoted 12 lines · Thomas Gleixner
Quoted 10 lines · Daniel Walker
Quoted 6 lines

Early bootup Instrumentation is really not a good argument to make that fragile time related stuff even more complex. There is no problem to register reliable clocksources in early bootup, but do not make this mandatory. Not every system is an ARM SoC, where you can and must rely on the one source which is available usually right when the CPU comes up.

It's not mandatory, it's just preferred.. As I said above, to avoid churn. I don't like the churn at boot up, and I tried to make sure there was none added in the patch set.

Quoted 6 lines

What churn at bootup ? The clocksources _can_ be switched and it does not matter, when this is done. We had the trouble with the early registration a couple of month ago, and there is no reason to reintroduce it. On systems which have exactly one clocksource, you can register them early in bootup, but please do not touch the x86 setup for no good reason.

There was a special case inside kernel/time/clocksource.c to prevent clock switching during boot up. If you remove that (which I have) then you will end up with clock switching happening a few times during bootup (whenever a new highest rated clock is registered), that's the churn I'm referring to.

The churn is not optimal. I've used postcore to prevent it, and make the API usable earlier. So there is a reason for the change.

Daniel

#10 lore raw
Quoted 8 lines · Daniel Walker

There was a special case inside kernel/time/clocksource.c to prevent clock switching during boot up. If you remove that (which I have) then you will end up with clock switching happening a few times during bootup (whenever a new highest rated clock is registered), that's the churn I'm referring to.

The churn is not optimal. I've used postcore to prevent it, and make the API usable earlier. So there is a reason for the change.

Yes, a bad one. The disabling had a totally different reason and you are not listening at all.

You just introduce a problem again, because it does not happen on your machines and you think, that some not yet available instrumentation code needs high resolution time stamps.

The reason why this was delayed into late boot is simply that the unstable, unsynchronized TSC's made way too much trouble and the pmtimer can not be initialized early.

I'm not going to accept that. Your change might work on 5 machines you have tested on, but we start over with the same breakage we solved already.

This early boot instrumentation code can work with low resolution time information quite well and none of the boot code does need any high res time information. Boot code is different from a running system and has different requirements.

This is a solution for a nonexisting problem, which just brings back already solved ones.

tglx

#11 lore raw
Quoted 7 lines · Thomas Gleixner
Quoted 6 lines · Daniel Walker

There was a special case inside kernel/time/clocksource.c to prevent clock switching during boot up. If you remove that (which I have) then you will end up with clock switching happening a few times during bootup (whenever a new highest rated clock is registered), that's the churn I'm referring to.

Quoted 5 lines
Quoted 2 lines

The churn is not optimal. I've used postcore to prevent it, and make the API usable earlier. So there is a reason for the change.

Yes, a bad one. The disabling had a totally different reason and you are not listening at all.

I am listening, Thomas ..

Quoted 3 lines

You just introduce a problem again, because it does not happen on your machines and you think, that some not yet available instrumentation code needs high resolution time stamps.

Ok.

Quoted 7 lines

The reason why this was delayed into late boot is simply that the unstable, unsynchronized TSC's made way too much trouble and the pmtimer can not be initialized early.

I'm not going to accept that. Your change might work on 5 machines you have tested on, but we start over with the same breakage we solved already.

Your going to have to explain the breakage further. I don't recall seeing any discussion on this.

Quoted 7 lines

This early boot instrumentation code can work with low resolution time information quite well and none of the boot code does need any high res time information. Boot code is different from a running system and has different requirements.

This is a solution for a nonexisting problem, which just brings back already solved ones.

There is at least one existing problem which it does solve. How about we discuss the early TSC boot breakage, which you mention above, so I can properly fix this situation.

Daniel

#12 lore raw
Quoted 10 lines · Daniel Walker
Quoted 7 lines

The reason why this was delayed into late boot is simply that the unstable, unsynchronized TSC's made way too much trouble and the pmtimer can not be initialized early.

I'm not going to accept that. Your change might work on 5 machines you have tested on, but we start over with the same breakage we solved already.

Your going to have to explain the breakage further. I don't recall seeing any discussion on this.

See above, but I'm happy to copy it.

Quoted 3 lines
Quoted 3 lines

The reason why this was delayed into late boot is simply that the unstable, unsynchronized TSC's made way too much trouble and the pmtimer can not be initialized early.

I don't have the bug reports handy, but they are in the LKML archives.

Quoted 9 lines
Quoted 7 lines

This early boot instrumentation code can work with low resolution time information quite well and none of the boot code does need any high res time information. Boot code is different from a running system and has different requirements.

This is a solution for a nonexisting problem, which just brings back already solved ones.

There is at least one existing problem which it does solve.

Which one exactly? I'm not aware of a problem with the existing code at all.

Quoted 3 lines

How about we
discuss the early TSC boot breakage, which you mention above, so I can properly fix this situation.

See above. Also it interferes probably with the highres/dyntick code, as we might switch over way too early. Have not looked into that yet.

tglx

#13 lore raw
Quoted 6 lines · Thomas Gleixner

See above, but I'm happy to copy it.

Quoted 3 lines
Quoted 3 lines

The reason why this was delayed into late boot is simply that the unstable, unsynchronized TSC's made way too much trouble and the pmtimer can not be initialized early.

I don't have the bug reports handy, but they are in the LKML archives.

Like I said I need more detail. I'm assuming John will know what your talking about, or do you have a rough time frame?

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

This early boot instrumentation code can work with low resolution time information quite well and none of the boot code does need any high res time information. Boot code is different from a running system and has different requirements.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

This is a solution for a nonexisting problem, which just brings back already solved ones.

There is at least one existing problem which it does solve.

Which one exactly? I'm not aware of a problem with the existing code at all.

Clock shuffling.

Quoted 6 lines
Quoted 3 lines

How about we
discuss the early TSC boot breakage, which you mention above, so I can properly fix this situation.

See above. Also it interferes probably with the highres/dyntick code, as we might switch over way too early. Have not looked into that yet.

I would be surprised if it caused problems. The clock switching is done at the same time as always. I tried not to alter generic time functionality in any way.

Daniel

#14 lore raw
Quoted 5 lines · Daniel Walker
Quoted 3 lines

Which one exactly? I'm not aware of a problem with the existing code at all.

Clock shuffling.

What's the problem with that ? It replaces clocks. Where _is_ the problem ?

tglx

#15 lore raw
Quoted 9 lines · Thomas Gleixner
Quoted 5 lines · Daniel Walker
Quoted 3 lines

Which one exactly? I'm not aware of a problem with the existing code at all.

Clock shuffling.

What's the problem with that ? It replaces clocks. Where _is_ the problem ?

The problem is that it's not optimal to have clocks switching furiously. This is something John notes as an issue in the unchanged kernel/time/clocksource.c file.

Daniel

#16 lore raw
Quoted 6 lines · Daniel Walker
Quoted 5 lines · Thomas Gleixner
Quoted 4 lines · Daniel Walker
Quoted 3 lines

Which one exactly? I'm not aware of a problem with the existing code at all.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Clock shuffling.

What's the problem with that ? It replaces clocks. Where _is_ the problem ?

The problem is that it's not optimal to have clocks switching furiously. This is something John notes as an issue in the unchanged kernel/time/clocksource.c file.

I don't see that behaviour on my machines and nobody complains about that. I don't care about stale comments. Point me to a bug report instead of your perception of what's optimal and not.

Working is not necessary optimal, but your vision of optimal is not necessarily working either.

tglx

#17 lore raw
Quoted 4 lines · Thomas Gleixner

I don't see that behaviour on my machines and nobody complains about that. I don't care about stale comments. Point me to a bug report instead of your perception of what's optimal and not.

Let both do this. Lets discuss empirical behavior. Otherwise we aren't making any progress.

Quoted 2 lines

Working is not necessary optimal, but your vision of optimal is not necessarily working either.

I'm looking for the TSC issue you mentioned.. I tested it as best I could, so I need more information to say it's not working.

Daniel

#18 lore raw
Quoted 9 lines · Daniel Walker
Quoted 4 lines · Thomas Gleixner

I don't see that behaviour on my machines and nobody complains about that. I don't care about stale comments. Point me to a bug report instead of your perception of what's optimal and not.

Let both do this. Lets discuss empirical behavior. Otherwise we aren't making any progress.

Go, grep the LKML archives and let those who had problems test your modifications. Come back when they confirm that it does not change anything.

You want to change behaviour of the current code, so it's your job to verify that it does not break anything.

I have been there and done that with the ARM interrupt code
http://www.linutronix.de/index.php?page=testing

I know what I'm talking about.

tglx

#19 lore raw
Quoted 8 lines · Thomas Gleixner
Quoted 7 lines · Daniel Walker
Quoted 4 lines · Thomas Gleixner

I don't see that behaviour on my machines and nobody complains about that. I don't care about stale comments. Point me to a bug report instead of your perception of what's optimal and not.

Quoted 6 lines
Quoted 2 lines

Let both do this. Lets discuss empirical behavior. Otherwise we aren't making any progress.

Go, grep the LKML archives and let those who had problems test your modifications. Come back when they confirm that it does not change anything.

This is why I want this to go into -mm .. To flush out the corner cases that it's impossible for me to find on my own.

Quoted 2 lines

You want to change behaviour of the current code, so it's your job to verify that it does not break anything.

I have, within my ability to do so.

Quoted 4 lines

I have been there and done that with the ARM interrupt code
http://www.linutronix.de/index.php?page=testing

I know what I'm talking about.

I think you know what your talking about, and I respect your opinion.. I think you have some points, that I will follow up on. However most of what your saying comes off to me like, "Reverse _all_ your changes!" , "Makes this patch set go away for 2 months", and I don't understand where that is coming from.

Daniel

#20 lore raw
Quoted 5 lines · Daniel Walker

I think you know what your talking about, and I respect your opinion.. I think you have some points, that I will follow up on. However most of what your saying comes off to me like, "Reverse _all_ your changes!" , "Makes this patch set go away for 2 months", and I don't understand where that is coming from.

I did nowhere say, that you should revert _ALL_ your changes ! If you need 2 month to fixup the issues I pointed at , I can't help it.

tglx