Skip to contentSkip to conversation

x86: Add PCI extended config space access for AMD Barcelona

PATCH Andi Kleen 4 participants 12 messages Sep 1, 2007 – Sep 4, 2007 3 messages not in the archive

lore
Share & tools
lore

Andi Kleen

author
#2 lore raw
In reply to a message not in the archive
Quoted 4 lines · On Thursday 30 August 2007 19:43:14 Robert Richter wrote:

This patch implements PCI extended configuration space access for AMD's Barcelona CPUs. It extends the method using CF8/CFC IO addresses. An x86 capability bit has been introduced that is set for CPUs supporting PCI extended config space accesses.

We shouldn't need this because extended config space should work here and Linux should use it
(especially after we added the ugly Barcelona workaround into that code)

The only exception would be if the user disables MMCONFIG in CONFIG, but that's their own fault then.

Ok there might be buggy BIOS around with no or no usable MCFG table, but since extended config space is not really a critical feature that's not a big issue.

So I don't think we want this special case code at all and should just rely on MMCONFIG.

-Andi

author
#14 lore raw

Re: [patch 1/5] x86: Add AMD64 Barcelona PMU MSR definitions

In reply to a message not in the archive
Quoted 2 lines · On Thursday 30 August 2007 19:43:12 Robert Richter wrote:

Already added to the -mm tree. Its filename is
i386-add-amd64-barcelona-pmu-msr-definitions.patch

Patch doesn't apply to -rc5 + ff tree.

Hunk #1 FAILED at 73.
Hunk #2 FAILED at 107.
2 out of 2 hunks FAILED -- rejects in file include/asm-i386/msr-index.h

If you want to do random white space changes and code movement please do it in a separate patch in the future.

-Andi

#3 lore raw
In reply to Andi Kleen
Quoted 9 lines · Andi Kleen
Quoted 4 lines · On Thursday 30 August 2007 19:43:14 Robert Richter wrote:

This patch implements PCI extended configuration space access for AMD's Barcelona CPUs. It extends the method using CF8/CFC IO addresses. An x86 capability bit has been introduced that is set for CPUs supporting PCI extended config space accesses.

We shouldn't need this because extended config space should work here and Linux should use it
(especially after we added the ugly Barcelona workaround into that code)

This patch is needed for all buggy BIOSes that don't correctly set up MCFG.

Quoted 2 lines

The only exception would be if the user disables MMCONFIG in CONFIG, but that's their own fault then.

No, as you stated yourself there are two exceptions. And the more serious one is the BIOS issue.

Quoted 2 lines

Ok there might be buggy BIOS around with no or no usable MCFG table, but since extended config space is not really a critical feature that's not a big issue.

Not sure why you assume extended config space (ECS) is not critical. Ok, for a lot of stuff it does not matter.
But it is needed for some devices for full functionality.
E.g. for perfmon (family 0x10/extended inerrupts) extended config space access is a prerequisite.

Quoted 2 lines

So I don't think we want this special case code at all and should just rely on MMCONFIG.

Rely on something unreliable (due to buggy/incomplete BIOS)? I don't think we should do this.

IMHO it is best to try to use MMCONFIG if it's working and to use a fallback (e.g. CF8 ECS access for family 0x10) if available.

Regards,

Andreas

Signature · 5 lines
Operating | AMD Saxony Limited Liability Company & Co. KG,
  System  | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 Research | Register Court Dresden: HRA 4896, General Partner authorized
  Center  | to represent: AMD Saxony LLC (Wilmington, Delaware, US)
  (OSRC)  | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
author
#4 lore raw
Quoted 2 lines

But it is needed for some devices for full functionality.

Examples? I can only think of PCI express error reporting, which few drivers implement anyways and isn't really a show stopper if it doesn't work. Besides I would be surprised if it even works on the cheap desktop boards which have MCFG less BIOS.

Quoted 2 lines

E.g. for perfmon (family 0x10/extended inerrupts) extended config space access is a prerequisite.

How so?

Quoted 3 lines

IMHO it is best to try to use MMCONFIG if it's working and to use a fallback (e.g. CF8 ECS access for family 0x10) if available.

We only put in workarounds if there is a serious problem otherwise (e.g. not booting etc.). I just don't see this here.

-Andi

#5 lore raw

Andi,

Quoted 6 lines · Andi Kleen
Quoted 1 line

But it is needed for some devices for full functionality.

Examples? I can only think of PCI express error reporting, which few drivers implement anyways and isn't really a show stopper if it doesn't work. Besides I would be surprised if it even works on the cheap desktop boards which have MCFG less BIOS.

As you say, there are BIOSs that do not support MMCONFIG. Thus, CF8 access is not only a workaround to boot a system. There are systems that really use CF8 access. Also, MMCONFIG depends on many conditions that must match. Cfg space must be mapped, the memory must be E820 reserved, MCFG table must be set, access to MMCONFIG must be enabled for the device. Many things that may fail and can partly not be fixed by the OS.

Quoted 4 lines
Quoted 2 lines

E.g. for perfmon (family 0x10/extended inerrupts) extended config space access is a prerequisite.

How so?

Recent (10h) and upcomming CPU families make heavy use of PCI ext cfg space for certain CPU features. Setup of the extended interupt local vector table for IBS (used with Perfmon2) is one example. CPU designers do not take care anymore if a feature is in the base or extended config space. So access to PCI ECS is essential.

Quoted 5 lines
Quoted 2 lines

IMHO it is best to try to use MMCONFIG if it's working and to use a fallback (e.g. CF8 ECS access for family 0x10) if available.

We only put in workarounds if there is a serious problem otherwise (e.g. not booting etc.). I just don't see this here.

As said above, I do not see CF8 access as a workaround. I expect my system to work in the same way also if MMCONFIG is not available.

-Robert

Signature · 3 lines
AMD Saxony, Dresden, Germany
Operating System Research Center
email: [email protected]
#11 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen
Quoted 2 lines

But it is needed for some devices for full functionality.

Examples? I can only think of PCI express error reporting, which few drivers implement anyways and isn't really a show stopper if it doesn't work. Besides I would be surprised if it even works on the cheap desktop boards which have MCFG less BIOS.

Sure, AER is one example.
I don't see why all other stuff in ECS is not worth reading or writing. And I am not sure whether all server boards set up MCFG in the correct way.

Quoted 4 lines
Quoted 2 lines

E.g. for perfmon (family 0x10/extended inerrupts) extended config space access is a prerequisite.

How so?

E.g. access to the IBS control register needs ECS access on Barcelona. I guess we have to wait until Robert sends his patches which contain all details.

Quoted 6 lines
Quoted 3 lines

IMHO it is best to try to use MMCONFIG if it's working and to use a fallback (e.g. CF8 ECS access for family 0x10) if available.

We only put in workarounds if there is a serious problem otherwise (e.g. not booting etc.). I just don't see this here.

The serious problem is you can't access PCI ECS if the BIOS does not take care to (correctly) set up MCFG.
And unfortunately this is too often the case.

See for instance Robert Hancock's patch http://lkml.org/lkml/2007/5/30/2
to enable MMCONFIG access in certain cases where BIOS did not correctly set up MCFG. Why are people working on such stuff if it is not serious enough?

Barcelona just adds another way to access PCI ECS (besides MMCONFIG) and I can't understand why this shouldn't be supported by Linux.

Do you have any suggestion how else to add support for PCI ECS access via IO instructions for Barcelona?

Regards,

Andreas

Signature · 5 lines
Operating | AMD Saxony Limited Liability Company & Co. KG,
  System  | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 Research | Register Court Dresden: HRA 4896, General Partner authorized
  Center  | to represent: AMD Saxony LLC (Wilmington, Delaware, US)
  (OSRC)  | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
author
#12 lore raw
Quoted 2 lines

And unfortunately this is too often the case.

On Barcelona systems?

Quoted 4 lines

See for instance Robert Hancock's patch http://lkml.org/lkml/2007/5/30/2
to enable MMCONFIG access in certain cases where BIOS did not correctly set up MCFG. Why are people working on such stuff if it is not serious enough?

I don't know why. I'm just not aware of any serious problems that are solved by extended config space access.

Originally we had trouble because MCFG was wrong and the systems would not boot. Also there is one class of systems (some x86 Apples) where cf8 doesn't work, but MMCONFIG does.
But that's all specific to older systems.

Quoted 2 lines

Barcelona just adds another way to access PCI ECS (besides MMCONFIG) and I can't understand why this shouldn't be supported by Linux.

It will be supported when the BIOS gets it right.

Quoted 2 lines

Do you have any suggestion how else to add support for PCI ECS access via IO instructions for Barcelona?

My suggestion is to rely on MMCONFIG for this.

-Andi

author
#6 lore raw
In reply to Robert Richter
Quoted 10 lines · Robert Richter
Quoted 6 lines · Andi Kleen
Quoted 1 line

But it is needed for some devices for full functionality.

Examples? I can only think of PCI express error reporting, which few drivers implement anyways and isn't really a show stopper if it doesn't work. Besides I would be surprised if it even works on the cheap desktop boards which have MCFG less BIOS.

As you say, there are BIOSs that do not support MMCONFIG. Thus, CF8 access is not only a workaround to boot a system.

We're talking about accessing the extended part of config spaces. I'm not aware of any case where that is required to boot a system.

Quoted 5 lines

Recent (10h) and upcomming CPU families make heavy use of PCI ext cfg space for certain CPU features. Setup of the extended interupt local vector table for IBS (used with Perfmon2) is one example. CPU designers do not take care anymore if a feature is in the base or extended config space. So access to PCI ECS is essential.

I don't think it's a big issue if IBS doesn't work on a few buggy BIOS (which should hopefully become fewer anyways because Vista is out which actually uses MCFG)

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

IMHO it is best to try to use MMCONFIG if it's working and to use a fallback (e.g. CF8 ECS access for family 0x10) if available.

We only put in workarounds if there is a serious problem otherwise (e.g. not booting etc.). I just don't see this here.

As said above, I do not see CF8 access as a workaround. I expect my system to work in the same way also if MMCONFIG is not available.

It should boot sure, but exotic stuff not working is not a major issue

-Andi

#7 lore raw

Andi,

Quoted 4 lines · Andi Kleen
Quoted 2 lines

As said above, I do not see CF8 access as a workaround. I expect my system to work in the same way also if MMCONFIG is not available.

It should boot sure, but exotic stuff not working is not a major issue

It is not only about booting the system without MMCONFIG, it is also about using the system. For this PCI ECS CF8 access is needed. So why not adding support for this? Excluding major parts of CPU registers when using CF8 base access only will cause writing code to workaround this. Not very nice to handle.

-Robert

Signature · 3 lines
AMD Saxony, Dresden, Germany
Operating System Research Center
email: [email protected]
author
#8 lore raw
Quoted 10 lines · Robert Richter

Andi,

Quoted 4 lines · Andi Kleen
Quoted 2 lines

As said above, I do not see CF8 access as a workaround. I expect my system to work in the same way also if MMCONFIG is not available.

It should boot sure, but exotic stuff not working is not a major issue

It is not only about booting the system without MMCONFIG, it is also about using the system

There are limits on how many BIOS bug workaround (and all this patch kit is nothing more than a elaborate BIOS bug workaround) we can do. IMHO you're far exceeding the threshold.

Better would be to invest that energy to get the BIOSes fixed.

Quoted 1 line

. For this PCI ECS CF8 access is needed.

AER and IBS are all totally optional features and the systems will be completely usable without them.

-Andi

#9 lore raw
In reply to Andi Kleen
Quoted 10 lines · Andi Kleen
Quoted 8 lines · Robert Richter
Quoted 6 lines · Andi Kleen
Quoted 1 line

But it is needed for some devices for full functionality.

Examples? I can only think of PCI express error reporting, which few drivers implement anyways and isn't really a show stopper if it doesn't work. Besides I would be surprised if it even works on the cheap desktop boards which have MCFG less BIOS.

Quoted 5 lines
Quoted 2 lines

As you say, there are BIOSs that do not support MMCONFIG. Thus, CF8 access is not only a workaround to boot a system.

We're talking about accessing the extended part of config spaces. I'm not aware of any case where that is required to boot a system.

in the bios for the new cpu, CF8 is used for ext conf space accessing...(for ht and mem init).

for setting mmio to accessing pci conf space, need to set sth on MSR. So the BIOS need to make it right..., otherwise OS need to set that when booting every cpu...to workaround it to make MMCONFIG working.

YH

author
#10 lore raw
Quoted 4 lines

for setting mmio to accessing pci conf space, need to set sth on MSR. So the BIOS need to make it right..., otherwise OS need to set that when booting every cpu...to workaround it to make MMCONFIG working.

Yes the BIOS has to get some things right. After all that's the BIOS job. And the BIOS knows much more about the hardware than the generic kernel. x86 Linux is not trying to be a BIOS too.

-Andi