PCI Express
Hi,
Is MSI/MSIX support available for ia64?
Are PCIE devices supported in ia64?
A pointer to any documentations with regards to PCIE and ia64 will be very much appreciated.
Thanks.
colin
Quoted 1 line · Colin Ngam
Is MSI/MSIX support available for ia64?
Yes
Quoted 1 line
Are PCIE devices supported in ia64?
I'm having trouble with the word "supported". You have a PCI-E Bridge and firmware that can talk to it hooked up to an ia64 box?
Anyway, linux has some PCI-E support in the generic PCI code.
Quoted 2 lines
A pointer to any documentations with regards to PCIE and ia64 will be very much appreciated.
I don't think anything ia64 or linux specific exists.
grant
Quoted 8 lines · Grant Grundler
Quoted 1 line · Colin Ngam
Are PCIE devices supported in ia64?
I'm having trouble with the word "supported". You have a PCI-E Bridge and firmware that can talk to it hooked up to an ia64 box?
Anyway, linux has some PCI-E support in the generic PCI code.
... and I wrote the ia64-specific bits to access extended PCI config space through the SAL PCI accessors. See PCI_SAL_EXT_ADDRESS().
Various PCIe features are in the process of being added. It all depends what you mean when you say "supported".
Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon the nation that is attacked, and every man will be glad of those conscience-soothing falsities, and will diligently study them, and refuse to examine any refutations of them; and thus he will by and by convince himself that the war is just, and will thank God for the better sleep he enjoys after this process of grotesque self-deception." -- Mark Twain
Quoted 8 lines · Matthew Wilcox
Quoted 7 lines · Grant Grundler
Quoted 1 line · Colin Ngam
Are PCIE devices supported in ia64?
I'm having trouble with the word "supported". You have a PCI-E Bridge and firmware that can talk to it hooked up to an ia64 box?
Quoted 8 lines
Quoted 1 line
Anyway, linux has some PCI-E support in the generic PCI code.
... and I wrote the ia64-specific bits to access extended PCI config space through the SAL PCI accessors. See
PCI_SAL_EXT_ADDRESS().Various PCIe features are in the process of being added. It all depends what you mean when you say "supported".
Are there specific lists (other than this one) to be watching for PCIe discussions?
thanks
Mark
Quoted 6 lines · Mark Maule
Quoted 2 lines · Matthew Wilcox
Various PCIe features are in the process of being added. It all depends what you mean when you say "supported".
Are there specific lists (other than this one) to be watching for PCIe discussions?
There's [email protected] and some stuff also gets discussed on l-k. There's not a whole lot of ia64-specific content, so I would only expect it to come up rarely on this list.
Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon the nation that is attacked, and every man will be glad of those conscience-soothing falsities, and will diligently study them, and refuse to examine any refutations of them; and thus he will by and by convince himself that the war is just, and will thank God for the better sleep he enjoys after this process of grotesque self-deception." -- Mark Twain
Grant Grundler wrote:
Hi Grant,
Thanks for your respond.
Quoted 9 lines · Colin Ngam
Quoted 5 lines
Is MSI/MSIX support available for ia64?
Yes
I looked at Documentation/MSI-HOWTO.txt(linux-ia64-release-2.6.11) and it mentioned that CONFIG_X86_LOCAL_APIC has to be configured. I did not find this configured on in any sample ia64 config files, but it does exist in defconfig for i386. Is this just an ia32 config option?
Quoted 12 lines
Quoted 7 lines
Are PCIE devices supported in ia
I'm having trouble with the word "supported". You have a PCI-E Bridge and firmware that can talk to it hooked up to an ia64 box?
Pretty soon.
Is there any ia64 box with MSI and PCIE actually "activated and running"?
Thanks.
colin
Quoted 19 lines
Anyway, linux has some PCI-E support in the generic PCI code.
Quoted 4 lines
A pointer to any documentations with regards to PCIE and ia64 will be very much appreciated.
I don't think anything ia64 or linux specific exists.
grant
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 3 lines · Colin Ngam
I looked at Documentation/MSI-HOWTO.txt(linux-ia64-release-2.6.11) and
it mentioned that
CONFIG_X86_LOCAL_APIChas to be configured. I did
not
Quoted 2 lines
find this configured on in any sample ia64 config files, but it does exist in defconfig for i386. Is this just an ia32 config option?
IA64 uses SAPIC instead of IOxAPIC. SAPIC, which is somewhat similar to IOxAPIC, is included in IA64 kernel. MSI/MSI-X support configuration option therefore is always available in IA64.
Thanks,
Long
Quoted 5 lines · Tom L
Quoted 3 lines · Colin Ngam
I looked at Documentation/MSI-HOWTO.txt(linux-ia64-release-2.6.11) and
it mentioned that
CONFIG_X86_LOCAL_APIChas to be configured. I did
Quoted 8 lines
not
Quoted 2 lines
find this configured on in any sample ia64 config files, but it does exist in defconfig for i386. Is this just an ia32 config option?
IA64 uses SAPIC instead of IOxAPIC. SAPIC, which is somewhat similar to IOxAPIC, is included in IA64 kernel. MSI/MSI-X support configuration option therefore is always available in IA64.
So the MSIs are programmed to point at the processor SAPIC block? I think that'll work for us on Altix, but if they're pointed at an external Intel (or compatible) interrupt controller, we'll have to write new code for Altix.
Jesse
Quoted 4 lines · Colin Ngam
I looked at Documentation/MSI-HOWTO.txt(linux-ia64-release-2.6.11) and it mentioned that
CONFIG_X86_LOCAL_APIChas to be configured. I did not find this configured on in any sample ia64 config files, but it does exist in defconfig for i386. Is this just an ia32 config option?
yes
Quoted 5 lines
Quoted 3 lines
I'm having trouble with the word "supported". You have a PCI-E Bridge and firmware that can talk to it hooked up to an ia64 box?
Pretty soon.
Is there any ia64 box with MSI and PCIE actually "activated and running"?
I've not seen anything published yet.
*Some* x86-64 boxes have working MSI (and PCI-E).
Recent PCI quirks have been submitted to flag the AMD chipsets that don't support MSI or MSI-X.
grant
Quoted 3 lines · Jesse Barnes
Quoted 2 lines · Tom L
Quoted 1 line · Colin Ngam
I looked at Documentation/MSI-HOWTO.txt(linux-ia64-release-2.6.11)
and
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
it mentioned that
CONFIG_X86_LOCAL_APIChas to be configured. I
did
Quoted 4 lines
Quoted 4 lines
not
Quoted 1 line
find this configured on in any sample ia64 config files, but it
does
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
exist in defconfig for i386. Is this just an ia32 config option?
IA64 uses SAPIC instead of IOxAPIC. SAPIC, which is somewhat similar
to
Quoted 4 lines
Quoted 2 lines
IOxAPIC, is included in IA64 kernel. MSI/MSI-X support configuration option therefore is always available in IA64.
So the MSIs are programmed to point at the processor SAPIC block? I
think
Quoted 1 line
that'll work for us on Altix, but if they're pointed at an external
Intel >(or
Quoted 1 line
compatible) interrupt controller, we'll have to write new code for
Altix.
MSI/MSI-X message address is programmed to point at specific processor, not an external Intel interrupt controller, as a target.
Thanks,
Long
Quoted 3 lines · Jesse Barnes
So the MSIs are programmed to point at the processor SAPIC block? I think that'll work for us on Altix, but if they're pointed at an external Intel (or compatible) interrupt controller, we'll have to write new code for Altix.
Assuming your "Processor Interrupt Block" (IIRC) is at 0xfee00000 it should just work.
grant
Quoted 8 lines · Grant Grundler
Quoted 4 lines · Jesse Barnes
So the MSIs are programmed to point at the processor SAPIC block? I think that'll work for us on Altix, but if they're pointed at an external Intel (or compatible) interrupt controller, we'll have to write new code for Altix.
Assuming your "Processor Interrupt Block" (IIRC) is at
0xfee00000it should just work.
Cool, I was hoping it just pointed at the same place we use for IPIs. It *should* work on Altix then, assuming the MSI allocation routines don't have platform knowledge in them (like ACPI bits or something).
Thanks,
Jesse
Quoted 3 lines · Jesse Barnes
Cool, I was hoping it just pointed at the same place we use for IPIs. It *should* work on Altix then, assuming the MSI allocation routines don't have platform knowledge in them (like ACPI bits or something).
I submitted a patch to add MSI support to tg3:
ftp://ftp.parisc-linux.org/patches/diff-2.6.10-tg3_MSI-03
It was rejected because of HW bugs in bcm57xx chips. But it seems to work fine for me on the HP ZX1 platforms. Maybe it's worth trying on the altix boxes as well.
grant
Quoted 22 lines · Tom L
Quoted 3 lines · Jesse Barnes
Quoted 2 lines · Tom L
Quoted 1 line · Colin Ngam
I looked at Documentation/MSI-HOWTO.txt(linux-ia64-release-2.6.11)
and
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
it mentioned that
CONFIG_X86_LOCAL_APIChas to be configured. Idid
Quoted 4 lines
Quoted 4 lines
not
Quoted 1 line
find this configured on in any sample ia64 config files, but it
does
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
exist in defconfig for i386. Is this just an ia32 config option?
IA64 uses SAPIC instead of IOxAPIC. SAPIC, which is somewhat similar
to
Quoted 4 lines
Quoted 2 lines
IOxAPIC, is included in IA64 kernel. MSI/MSI-X support configuration option therefore is always available in IA64.
So the MSIs are programmed to point at the processor SAPIC block? I
Quoted 8 lines
think
Quoted 1 line
that'll work for us on Altix, but if they're pointed at an external
Intel >(or
Quoted 1 line
compatible) interrupt controller, we'll have to write new code for
Altix.
MSI/MSI-X message address is programmed to point at specific processor, not an external Intel interrupt controller, as a target.
Hi Long/All,
Thank you (all) very much for the information.
It's been a long time since we tested MSI/MSIX on our Altix boxes. It has been longer since I looked at the code. It works and we are looking forward to hooking them up with the current Infrastructure available on ia64 - pci_enable|disable_msi().
On Altix systems, we have a set of "Interrupt Registers" in Memory Address Space that is initialized to target specific CPUs. The way we initialize a card's MSI is:
1. The Target Address is One of these "Interrupt Registers"
2. The Data Payload is the IRQ plus some special Altix bits.
This memory write causes the "Interrupt Chipset" to generate a LINTR message to the configured targeted cpu with the IRQ. Ofcourse, these registers are Altix Platform specific. Moreover, we have chunks of these registers all over the place.
Is there a more direct mechanism to generate an interrupt(LINTR Message) to a Processor? 1 of the Special bits that I mentioned in Item 2 above causes our hardware to flush all posted DMA buffers before allowing the LINTR Message to be generated to the cpu.
With Altix, it's always interesting, with respect to "just work" with typical ia64 code base. Hopefully, it will work cleaner compared to some of our other efforts.
Are you the maintainer for the MSI code?
Thanks.
colin
Quoted 4 lines
Thanks,
Long
Quoted 9 lines · Colin Ngam
It's been a long time since we tested MSI/MSIX on our Altix boxes. It has been longer since I looked at the code. It works and we are looking forward to hooking them up with the current Infrastructure available on ia64 - pci_enable|disable_msi().
On Altix systems, we have a set of "Interrupt Registers" in Memory Address Space that is initialized to target specific CPUs. The way we initialize a card's MSI is:
Quoted 7 lines
1. The Target Address is One of these "Interrupt Registers"
2. The Data Payload is the IRQ plus some special Altix bits.This memory write causes the "Interrupt Chipset" to generate a LINTR message to the configured targeted cpu with the IRQ. Ofcourse, these registers are Altix Platform specific. Moreover, we have chunks of these registers all over the place.
That's one way to do it, but is obviously very Altix specific.
Quoted 4 lines
Is there a more direct mechanism to generate an interrupt(LINTR Message) to a Processor? 1 of the Special bits that I mentioned in Item 2 above causes our hardware to flush all posted DMA buffers before allowing the LINTR Message to be generated to the cpu.
Yes, see Grant's earlier message about the processor interrupt block. If we target MSIs at that, they'll behave the same way as IPIs, which are already platform independent (i.e. they work on Altix, zx1, tiger, etc.) since the processor interrupt block is builtin to the CPU. See Section 5.8.4 of Vol. 2 of the Itanium Arch. Software Developer's Manual (the blue books).
Jesse
Quoted 1 line · Colin
On Altix systems, we have a set of "Interrupt Registers" in Memory
Address
Quoted 9 lines
Space that is initialized to target specific CPUs. The way we initialize a card's MSI is:
1. The Target Address is One of these "Interrupt Registers"
2. The Data Payload is the IRQ plus some special Altix bits.This memory write causes the "Interrupt Chipset" to generate a LINTR message to the configured targeted cpu with the IRQ. Ofcourse, these registers are Altix Platform specific. Moreover, we have chunks of
these
Quoted 1 line
registers all over the place.
Your solution is Altix chipset specific. MSI affects other architectures too. General solution would almost certainly be a better one.
Quoted 1 line
Is there a more direct mechanism to generate an interrupt(LINTR
Message)
Quoted 1 line
to a Processor?
I agree with Jesse's comment.
Quoted 5 lines
With Altix, it's always interesting, with respect to "just work" with typical ia64 code base. Hopefully, it will work cleaner compared to some of our other efforts.
Are you the maintainer for the MSI code?
I suggest you move any discussion or any proposed changes to LKML because MSI affects other architectures too. Again, general solution would almost certainly be a better one.
Thanks,
Long
Quoted 9 lines · Tom L
Quoted 1 line · Colin
On Altix systems, we have a set of "Interrupt Registers" in Memory
Address
Quoted 6 lines
Space that is initialized to target specific CPUs. The way we initialize a card's MSI is:
1. The Target Address is One of these "Interrupt Registers"
2. The Data Payload is the IRQ plus some special Altix bits.
Quoted 8 lines
Quoted 3 lines
This memory write causes the "Interrupt Chipset" to generate a LINTR message to the configured targeted cpu with the IRQ. Ofcourse, these registers are Altix Platform specific. Moreover, we have chunks of
these
Quoted 1 line
registers all over the place.
Your solution is Altix chipset specific. MSI affects other architectures too. General solution would almost certainly be a better one.
Hi All,
Yes it is and we are looking for a generic solution.
Quoted 7 lines
Quoted 3 lines
Is there a more direct mechanism to generate an interrupt(LINTR
Message)
Quoted 1 line
to a Processor?
I agree with Jesse's comment.
Well, unfortunately, we do not send IPI by using the Processor Interrupt Block. We actually
target a Special Altix Chipset "IPI Interrupt" register that ends up generating an IPI. That is why, we
have Platform Specific "send ipi" calls on ia64:
platform_send_ipi()
File Line 0 machvec.h 113 #define platform_send_ipi ia64_mv.send_ipi 1 machvec.h 282 #define platform_send_ipi ia64_send_ipi
2 machvec_sn2.h 86 #define platform_send_ipi sn2_send_IPI
3 machvec.h 95 #define platform_send_ipi ia64_mv.send_ipi
4 machvec.h 255 #define platform_send_ipi ia64_send_ipi
5 machvec_sn2.h 83 #define platform_send_ipi sn2_send_IPI
I do not know why we do not use the Processor Interrupt Block, but I will find out. But this does
not mean that we cannot use the PIB for MSI. Ofcourse, it may not be relocated at0x0000 0000 FEE0 0000.
Quoted 11 lines
Quoted 7 lines
With Altix, it's always interesting, with respect to "just work" with typical ia64 code base. Hopefully, it will work cleaner compared to some of our other efforts.
Are you the maintainer for the MSI code?
I suggest you move any discussion or any proposed changes to LKML because MSI affects other architectures too. Again, general solution would almost certainly be a better one.
We will. We are looking for a general solution.
We will start looking through the MSI code see how best we can implement this functionality to support Altix platform. At that point we will post to lkml for guidance.
Thanks.
colin
Quoted 8 lines
Thanks,
Long
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 15 lines · Colin Ngam
Well, unfortunately, we do not send IPI by using the Processor Interrupt Block. We actually
target a Special Altix Chipset "IPI Interrupt" register that ends up generating an IPI. That is why, we
have Platform Specific "send ipi" calls on ia64:
platform_send_ipi()File Line 0 machvec.h 113 #define platform_send_ipi ia64_mv.send_ipi 1 machvec.h 282 #define platform_send_ipi ia64_send_ipi2
machvec_sn2.h86 #defineplatform_send_ipisn2_send_IPI
3machvec.h95 #defineplatform_send_ipiia64_mv.send_ipi
4machvec.h255 #defineplatform_send_ipiia64_send_ipi
5machvec_sn2.h83 #defineplatform_send_ipisn2_send_IPI
Quoted 5 lines
I do not know why we do not use the Processor Interrupt Block, but I will find out. But this does
not mean that we cannot use the PIB for MSI. Ofcourse, it may not be relocated at0x00000000 FEE0 0000.
That's beside the point though--MSIs don't use IPIs, they just use the processor interrupt block, which is also used by IPIs (on non-sn2 platforms). If the processor interrupt block works for us (pretty sure it does) MSIs targetted there should also work. The problem for us is if a processor tries to IPI a processor on a different node with the processor interrupt block, which I don't think MSIs will try to do.
Jesse
Quoted 16 lines · Jesse Barnes
Quoted 15 lines · Colin Ngam
Well, unfortunately, we do not send IPI by using the Processor Interrupt Block. We actually
target a Special Altix Chipset "IPI Interrupt" register that ends up generating an IPI. That is why, we
have Platform Specific "send ipi" calls on ia64:
platform_send_ipi()File Line 0 machvec.h 113 #define platform_send_ipi ia64_mv.send_ipi 1 machvec.h 282 #define platform_send_ipi ia64_send_ipi2
machvec_sn2.h86 #defineplatform_send_ipisn2_send_IPI
3machvec.h95 #defineplatform_send_ipiia64_mv.send_ipi
4machvec.h255 #defineplatform_send_ipiia64_send_ipi
5machvec_sn2.h83 #defineplatform_send_ipisn2_send_IPI
Quoted 5 lines
Quoted 5 lines
I do not know why we do not use the Processor Interrupt Block, but I will find out. But this does
not mean that we cannot use the PIB for MSI. Ofcourse, it may not be relocated at0x00000000 FEE0 0000.
Hi Jesse,
Quoted 5 lines
That's beside the point though--MSIs don't use IPIs, they just use the processor interrupt block, which is also used by IPIs (on non-sn2 platforms). If the processor interrupt block works for us (pretty sure it does) MSIs
That's what I will find out before deciding to use it. It is not used for IPIs on our platform, probably for very good reasons. If we can and the address is the same than it is a no brainer.
Quoted 4 lines
targetted there should also work. The problem for us is if a processor tries to IPI a processor on a different node with the processor interrupt block, which I don't think MSIs will try to do.
Perhaps you can elaborate. I am not sure what you are trying to say here.
Thanks.
colin
Quoted 3 lines
Jesse
Quoted 41 lines · Colin Ngam
Quoted 16 lines · Jesse Barnes
Quoted 15 lines · Colin Ngam
Well, unfortunately, we do not send IPI by using the Processor Interrupt Block. We actually
target a Special Altix Chipset "IPI Interrupt" register that ends up generating an IPI. That is why, we
have Platform Specific "send ipi" calls on ia64:
platform_send_ipi()File Line 0 machvec.h 113 #define platform_send_ipi ia64_mv.send_ipi 1 machvec.h 282 #define platform_send_ipi ia64_send_ipi2
machvec_sn2.h86 #defineplatform_send_ipisn2_send_IPI
3machvec.h95 #defineplatform_send_ipiia64_mv.send_ipi
4machvec.h255 #defineplatform_send_ipiia64_send_ipi
5machvec_sn2.h83 #defineplatform_send_ipisn2_send_IPIQuoted 5 lines
Quoted 5 lines
I do not know why we do not use the Processor Interrupt Block, but I will find out. But this does
not mean that we cannot use the PIB for MSI. Ofcourse, it may not be relocated at0x00000000 FEE0 0000.Hi Jesse,
Quoted 5 lines
That's beside the point though--MSIs don't use IPIs, they just use the processor interrupt block, which is also used by IPIs (on non-sn2 platforms). If the processor interrupt block works for us (pretty sure it does) MSIs
That's what I will find out before deciding to use it. It is not used for IPIs on our platform, probably for very good reasons. If we can and the address is the same than it is a no brainer.
Quoted 4 lines
targetted there should also work. The problem for us is if a processor tries to IPI a processor on a different node with the processor interrupt block, which I don't think MSIs will try to do.
Quoted 9 lines
Perhaps you can elaborate. I am not sure what you are trying to say here.
Thanks.
colin
Quoted 3 lines
Jesse
Hi Jesse,
I beleive the reason why we do not use the Processor Interrupt Block for generating interrupts is because it does not have the range to reach all the nodes that we can have in the system.
Thanks.
colin
Quoted 6 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines · Colin Ngam
That's what I will find out before deciding to use it. It is not used for IPIs on our platform, probably for very good reasons.
Yep, because IPIs generated from the local block to a distant processor won't work otherwise (afaik).
Quoted 8 lines
If we can and the
address is the same than it is a no brainer.Quoted 3 lines
targetted there should also work. The problem for us is if a processor tries to IPI a processor on a different node with the processor interrupt block, which I don't think MSIs will try to do.
Perhaps you can elaborate. I am not sure what you are trying to say here.
The SHub manual is a little vague here, but my understanding is that in order to send remote IPIs we need to use the SHub, whereas local IPIs (i.e. processor to itself or to the other CPU on the same FSB) will work fine.
An MSI should behave like a processor sending an IPI to itself since its address can be targeted at the processor's interrupt block and set to generate a local interrupt. Is that right, Tom & Grant?
If this won't work for us, no biggie, we just have to abstract things a little more. We could make the MSI into a platform specific cookie that we can store a SHub/PIC/TIO address in and other platforms can use to target the processor interrupt block. Hopefully we won't have to do that though, since it would require a few more changes.
Jesse
Quoted 24 lines · Jesse Barnes
Quoted 2 lines · Colin Ngam
That's what I will find out before deciding to use it. It is not used for IPIs on our platform, probably for very good reasons.
Yep, because IPIs generated from the local block to a distant processor won't work otherwise (afaik).
Quoted 8 lines
If we can and the
address is the same than it is a no brainer.Quoted 3 lines
targetted there should also work. The problem for us is if a processor tries to IPI a processor on a different node with the processor interrupt block, which I don't think MSIs will try to do.
Perhaps you can elaborate. I am not sure what you are trying to say here.
The SHub manual is a little vague here, but my understanding is that in order to send remote IPIs we need to use the SHub, whereas local IPIs (i.e. processor to itself or to the other CPU on the same FSB) will work fine.
An MSI should behave like a processor sending an IPI to itself since its address can be targeted at the processor's interrupt block and set to generate a local interrupt. Is that right, Tom & Grant?
Quoted 5 lines
If this won't work for us, no biggie, we just have to abstract things a little more. We could make the MSI into a platform specific cookie that we can store a SHub/PIC/TIO address in and other platforms can use to target the processor interrupt block. Hopefully we won't have to do that though, since it would require a few more changes.
May be the way we go. Needs more perusal of the MSI code.
Thanks.
colin
Quoted 7 lines
Jesse
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Mar 08, 2005 at 05:29:13PM -0800, Jesse Barnes wrote:
...
Quoted 3 lines
An MSI should behave like a processor sending an IPI to itself since its address can be targeted at the processor's interrupt block and set to generate a local interrupt. Is that right, Tom & Grant?
Yes - that's how I understand it too.
Quoted 4 lines
If this won't work for us, no biggie, we just have to abstract things a little more. We could make the MSI into a platform specific cookie that we can store a SHub/PIC/TIO address in and other platforms can use to target the processor interrupt block.
Couple more options:
o add hooks for MSI quirk/fixup for *after* the MSI has been assigned by generic code.
o make sure all MSI only go to "node local" processors. I thought platform code has some control over the CPU EID and data portion of the MSI. But I haven't looked over that in a few monthes.
grant
Grant Grundler wrote:
Hi Grant,
Thanks for the info.
Bottomline - Processor Interrupt Block not implemented on Altix.
Thanks.
colin
Quoted 26 lines · Jesse Barnes
...
Quoted 3 lines
An MSI should behave like a processor sending an IPI to itself since its address can be targeted at the processor's interrupt block and set to generate a local interrupt. Is that right, Tom & Grant?
Yes - that's how I understand it too.
Quoted 4 lines
If this won't work for us, no biggie, we just have to abstract things a little more. We could make the MSI into a platform specific cookie that we can store a SHub/PIC/TIO address in and other platforms can use to target the processor interrupt block.
Couple more options:
o add hooks for MSI quirk/fixup for *after* the MSI has been assigned by generic code.o make sure all MSI only go to "node local" processors. I thought platform code has some control over the CPU EID and data portion of the MSI. But I haven't looked over that in a few monthes.
grant
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Mar 08, 2005 at 05:29:13PM -0800, Jesse Barnes wrote:
...
Quoted 1 line
An MSI should behave like a processor sending an IPI to itself since
its
Quoted 2 lines
address can be targeted at the processor's interrupt block and set to generate a local interrupt. Is that right, Tom & Grant?
MSI address is programmed at 0xfeex_xxxx (based on PCI Specs) to generate a memory write to FSB. This address can be configured to target a platform interrupt controller (like IOAPIC for example), which in turn generates a memory write to a FSB, or be configured to send a memory write directly to FSB. Existing MSI support implements a direct memory-write mechanism (generic solution without platform dependency and limitation to MSI-X support as an example). With existing direct memory-write mechanism, a device's MSI address is configured with current running CPU as a target. MSI support also implements MSI SMP affinity, which supports IRQ rebalance and allows user to select which CPU target through /proc/irq/...
I hope it helps.
Thanks,
Long
Quoted 5 lines · Tom L
Existing MSI support implements a direct
memory-write mechanism (generic solution without platform dependency and limitation to MSI-X support as an example). With existing direct memory-write mechanism, a device's MSI address is configured with current running CPU as a target.
At some point, PARISC (and Alpha?) will also want to support MSI using "direct memory write". But like Altix, does not implement 0xfeeXXXXX (1) PIB since PARISC arch predates PCI by a decade or so.
We'll have to figure out which platform specific hooks would enable parisc/alpha/altix to use existing PCI MSI/MSI-X support.
PA-RISC IPI is as straight forward as Jesse described - just write to a per CPU address with a data value (vector).
thanks,
grant
(1) the most recent two or three PA-RISC chipsets do implement 0xfeeXXXXX interrupt block. But firmware hands us the CPU "ID/EID" values to use when programming the IO SAPIC IRT. See drivers/parisc/iosapic.c iosapic_set_irt_data() for more details.
Bottomline is, even if we know it's implemented in HW, we don't know if we are using 0xfeeXXXXX or not for IO SAPIC today. platform hooks into MSI code would have to sort that out.
Quoted 11 lines · Grant Grundler
Quoted 7 lines · Tom L
Existing MSI support implements a direct
memory-write mechanism (generic solution without platform dependency and limitation to MSI-X support as an example). With existing direct memory-write mechanism, a device's MSI address is configured with current running CPU as a target.
Quoted 7 lines
At some point, PARISC (and Alpha?) will also want to support MSI using "direct memory write". But like Altix, does not implement 0xfeeXXXXX (1) PIB since PARISC arch predates PCI by a decade or so.
We'll have to figure out which platform specific hooks would enable parisc/alpha/altix to use existing PCI MSI/MSI-X support.
Hi Grant,
Sounds great.
Thanks.
colin
Quoted 17 lines
PA-RISC IPI is as straight forward as Jesse described - just write to a per CPU address with a data value (vector).
thanks,
grant(1) the most recent two or three PA-RISC chipsets do implement 0xfeeXXXXX interrupt block. But firmware hands us the CPU "ID/EID" values to use when programming the IO SAPIC IRT. See
drivers/parisc/iosapic.ciosapic_set_irt_data()for more details.Bottomline is, even if we know it's implemented in HW, we don't know if we are using 0xfeeXXXXX or not for IO SAPIC today. platform hooks into MSI code would have to sort that out.
Quoted 4 lines · Colin Ngam
Quoted 2 lines
We'll have to figure out which platform specific hooks would enable parisc/alpha/altix to use existing PCI MSI/MSI-X support.
Sounds great.
Indeed. :^)
I forgot to mention I've had this on my wish list for ~5 years...
http://lists.parisc-linux.org/pipermail/parisc-linux/1999-December/008066.html
grant
Quoted 10 lines · Tom L
On Tue, Mar 08, 2005 at 05:29:13PM -0800, Jesse Barnes wrote:
...Quoted 3 lines
An MSI should behave like a processor sending an IPI to itself since
its
Quoted 7 lines
Quoted 4 lines
address can be targeted at the processor's interrupt block and set to generate a local interrupt. Is that right, Tom & Grant?
MSI address is programmed at 0xfeex_xxxx (based on PCI Specs) to
Hi Tom,
The MSI address portion allows 64bits for the address. It should be an address whereby a specific Platform has configured with sufficient information to generate an Interrupt to the targeted "cpu" with the payload - IRQ.
I am very much interested in your assertion that the MSI address is defined by PCI Spec to be 0xfeex_xxxx. Can you kindly point me to the relevent section? Perhaps if you have an online copy, can you cut and paste the relevent section?
Thanks.
colin
Quoted 17 lines
generate a memory write to FSB. This address can be configured to target a platform interrupt controller (like IOAPIC for example), which in turn generates a memory write to a FSB, or be configured to send a memory write directly to FSB. Existing MSI support implements a direct memory-write mechanism (generic solution without platform dependency and limitation to MSI-X support as an example). With existing direct memory-write mechanism, a device's MSI address is configured with current running CPU as a target. MSI support also implements MSI SMP affinity, which supports IRQ rebalance and allows user to select which CPU target through
/proc/irq/...I hope it helps.
Thanks,
Long
Quoted 1 line · Colin Ngam
The MSI address portion allows 64bits for the address.
Yes - *allows*. It also allows devices to implement 32-bit MSI address. MSI "Control" bits indicate what is implemented.
See PCI2.2 spec or later and search for MSI.
Quoted 2 lines
I am very much interested in your assertion that the MSI address is defined by PCI Spec to be 0xfeex_xxxx.
...
Quoted 1 line
Can you kindly point me to the relevent section?
It's not and it doesn't belong there.
IA64 specs defines Processor Interrupt Block and where it lives. See "Intel® Itanium® Software Developer's Manual"
Volume 2, Part 1, Section 5.8.4 Volume 2, Part 1, Section 11.9.3
grant
Quoted 11 lines · Grant Grundler
Quoted 2 lines
I am very much interested in your assertion that the MSI address is defined by PCI Spec to be 0xfeex_xxxx.
...
Quoted 1 line
Can you kindly point me to the relevent section?
It's not and it doesn't belong there.
IA64 specs defines Processor Interrupt Block and where it lives. See "Intel(r) Itanium(r) Software Developer's Manual"
Volume 2, Part 1, Section 5.8.4 Volume 2, Part 1, Section 11.9.3
I apologize that it is not in PCI Spec. It is also in Section 8.11 of "IA-32 Intel Architecture Software Developer's Manual (volume 3)."
Thanks,
Long
Quoted 23 lines · Tom L
Quoted 10 lines · Grant Grundler
Quoted 4 lines
I am very much interested in your assertion that the MSI address is defined by PCI Spec to be 0xfeex_xxxx.
...
Quoted 3 lines
Can you kindly point me to the relevent section?
Quoted 8 lines
It's not and it doesn't belong there.
IA64 specs defines Processor Interrupt Block and where it lives. See "Intel(r) Itanium(r) Software Developer's Manual"
Volume 2, Part 1, Section 5.8.4 Volume 2, Part 1, Section 11.9.3
Quoted 3 lines
I apologize that it is not in PCI Spec. It is also in Section 8.11 of "IA-32 Intel Architecture Software Developer's Manual (volume 3)."
Hi,
Thanks Tom.
I believe we are all are in agreement that the MSI Address needs to be Platform "Tunable" :-)
Thank you.
colin
Quoted 9 lines
Thanks,
Long
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html