SMP processor rework help needed
Hi,
I have this unique situation where cpu 1 has less features (like fxsr) then cpu 0.
However the linux kernel only looks at the cpu 0 and assumes the all the other cpu's have the same capabilities. Which can cause it to oops. Several of you have been so kind to help me find this problem. Now I can manually fit it in processor.h to change #define cpu_has_fxsr 0. I would like to rewrite this bit so it tests all the cpu's if it has a certain capability and only then activates this feature, I think this would be a more clean way of handling things.
Anyway I have been looking through processor.h and bitops.h and figured it does a some asm to extract this information.
So again I am stuck as my limited knowledge stops here, so I was hoping that someone could give me a helping hand or point me in a direction.
Ps 2.4.12-ac2 works fine with my little modification on my aic78xxx lx smp system in case you wanted to know.
In article <000b01c154ee$1d6a2610$6400a8c0@it0>,
Quoted 3 lines · Tommy Faasen
Hi,
I have this unique situation where cpu 1 has less features (like fxsr) then cpu 0.
I used to have such an AMP machine too: a dual PII-300 with one Katmai and one Deschutes. It's technically a violation of the specs; the Intel SMP spec requires that the non boot cpus need to have a superset of the features of the boot CPU. One CPU died, so it is symmetric now.
For most capabilities it should already work in 2.4 after hpa's cpu set rewrite, but FXSAVE is unfortunately a bit of a special case because it is used in the scheduler context switch and that is required early in the initialization for SMP bootup and changing it would be very intrusive.
In the 2.2 SuSE kernel it was fixed instead by adding a new kernel command line option nofxsave that overrides the FXSAVE bit on the first CPU. That is ok because such setup is very rare and is only generated by people who build their own boxes; and these should also know how to pass kernel command line arguments.
Such an option should also be easy to add to 2.4.
-Andi
Quoted 17 lines · Andi Kleen
In article <000b01c154ee$1d6a2610$6400a8c0@it0>,
Quoted 3 lines · Tommy Faasen
Hi,
I have this unique situation where cpu 1 has less features (like fxsr) then cpu 0.I used to have such an AMP machine too: a dual PII-300 with one Katmai and one Deschutes. It's technically a violation of the specs; the Intel SMP spec requires that the non boot cpus need to have a superset of the features of the boot CPU. One CPU died, so it is symmetric now.
For most capabilities it should already work in 2.4 after hpa's cpu set rewrite, but FXSAVE is unfortunately a bit of a special case because it is used in the scheduler context switch and that is required early in the initialization for SMP bootup and changing it would be very intrusive.
Quoted 5 lines
In the 2.2 SuSE kernel it was fixed instead by adding a new kernel command line option nofxsave that overrides the FXSAVE bit on the first CPU. That is ok because such setup is very rare and is only generated by people who build their own boxes; and these should also know how to pass kernel command line arguments.
This may sound like a dumb question but wouldn't simply swapping the CPUs have the same affect?
Gerhard
--
Gerhard Mack
<>< As a computer I find your faith in technology amusing.
Yes it would however in my case, the cooler,which I can't get off, is too big for the first slot!
It's just 1 more way linux can be more flexible with hardware. ----- Original Message -----
From: "Gerhard Mack" <[email protected]>
To: "Andi Kleen" <[email protected]>
Cc: "Tommy Faasen" <[email protected]>; <[email protected]>
Sent: Sunday, October 14, 2001 10:50 PM
Subject: Re: SMP processor rework help needed
Quoted 6 lines · Andi Kleen
Quoted 5 lines
In article <000b01c154ee$1d6a2610$6400a8c0@it0>,
Quoted 2 lines · Tommy Faasen
Hi,
I have this unique situation where cpu 1 has less features (like fxsr)
then
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
cpu 0.
I used to have such an AMP machine too: a dual PII-300 with one Katmai
and one
Quoted 36 lines
Quoted 10 lines
Deschutes. It's technically a violation of the specs; the Intel SMP spec requires that the non boot cpus need to have a superset of the features of the boot CPU. One CPU died, so it is symmetric now.
For most capabilities it should already work in 2.4 after hpa's cpu set rewrite, but FXSAVE is unfortunately a bit of a special case because it is used in the scheduler context switch and that is required early in the initialization for SMP bootup and changing it would be very intrusive.
Quoted 5 lines
In the 2.2 SuSE kernel it was fixed instead by adding a new kernel command line option nofxsave that overrides the FXSAVE bit on the first CPU. That is ok because such setup is very rare and is only generated by people who build their own boxes; and these should also know how to pass kernel command line arguments.
This may sound like a dumb question but wouldn't simply swapping the CPUs have the same affect?
Gerhard
--
Gerhard Mack<>< As a computer I find your faith in technology amusing.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines · Gerhard Mack
This may sound like a dumb question but wouldn't simply swapping the CPUs have the same affect?
In theory yes, assuming the determination of the boot cpu is fully deterministic. the spec says it is the one with the lowest apic number; but who knows if that is true in every weird board.
-Andi
Quoted 4 lines · Andi Kleen
I used to have such an AMP machine too: a dual PII-300 with one Katmai and one Deschutes. It's technically a violation of the specs; the Intel SMP spec requires that the non boot cpus need to have a superset of the features of the boot CPU. One CPU died, so it is symmetric now.
Errrrm, excuse me, but why not simply swap the two CPUs so that the one with less features becomes the boot CPU?
Signature · 3 lines
Ciao, Pascal -<[ [email protected], netmail 2:241/215.72, home http://cobol.cjb.net/) ]>-
Quoted 7 lines · Andi Kleen
Quoted 2 lines · Gerhard Mack
This may sound like a dumb question but wouldn't simply swapping the CPUs have the same affect?
In theory yes, assuming the determination of the boot cpu is fully deterministic. the spec says it is the one with the lowest apic number; but who knows if that is true in every weird board.
I do recall that the apics have programmable numbers. We even test that as part of our cpu initialization. So that means little.
For intel the initial determination is made having the cpus race on the apic bus. The cpu that sends a message first gets the lowest apicid. Though I need to see how the P4 Xeon does it, as the apic bus is actually unused.
Also many boards have logic so that allows the second cpu to become the boot cpu if the first cpu fails to boot. This logic might be as simple as round-robin, so even a deterministic may make this difficult.
So the only reliable way to force the boot cpu is with software that runs before the operating system, usually the firmware.
I'll keep this in mind for linuxBIOS, as that would be an ideal place to implement something like that.
Eric
Quoted 11 lines · Andi Kleen
I used to have such an AMP machine too: a dual PII-300 with one Katmai and one Deschutes. It's technically a violation of the specs; the Intel SMP spec requires that the non boot cpus need to have a superset of the features of the boot CPU. One CPU died, so it is symmetric now.
For most capabilities it should already work in 2.4 after hpa's cpu set rewrite, but FXSAVE is unfortunately a bit of a special case because it is used in the scheduler context switch and that is required early in the initialization for SMP bootup and changing it would be very intrusive.
Quoted 7 lines
In the 2.2 SuSE kernel it was fixed instead by adding a new kernel command line option nofxsave that overrides the FXSAVE bit on the first CPU. That is ok because such setup is very rare and is only generated by people who build their own boxes; and these should also know how to pass kernel command line arguments.
Such an option should also be easy to add to 2.4.
Given that this is an unlikely case, and init code is released after boot, if this were going to be done at all it might be desirable to make it a build option and check all capabilities on all CPUs. While I can't identify a current set of CPUs with at least one feature missing from each, given Linux running on Intel and AMD processors, and many other type as well (still multiple SPARC vendors, I think), just checking all the things which might be broken or missing would be the safest way.
No, I have no idea how hard it really is, one quick look at the code makes it look less than hard and more than non-trivial for iA86.
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.