NTB driver support in haswell platform?
Hi Jon,
I have a Intel Haswell platform in hand, and our team want to use NTB in this platform.
I checked the current intel NTB driver in Linux kernel, I found the Haswell NTB pci device id is not contained in ntb_pci_tbl[]. I want to know whether current kernel ntb driver can support the ntb device in Haswell platform ?
Haswell NTB device id:
Quoted 1 line
From Haswell EDS 7.4.2
did
Bus: 0 Device: 3 Function: 0 Offset: 2
Bit Attr Default Description
15:0 RO-V 2F08h Device_Identification_Number — Device ID values vary from function to function. Bits 15:8 are equal to 0x2F. The following list is a breakdown of the function groups.
0x2F00 - 0x2F1F : PCI Express and DMI2
0x2F20 - 0x2F3F : Integrated I/O Features
0x2F40 - 0x2F5F : Performance Monitors
0x2F80 - 0x2F9F : Intel QPI
0x2FA0 - 0x2FBF : Home Agent/Memory Controller
0x2FC0 - 0x2FDF : Power Management
0x2FE0 - 0x2FFF : Cbo/Ring
Default value may vary based on bus, device, and function of this CSR location.
Current ntb_pci_tbl[] in Linux:
#define PCI_DEVICE_ID_INTEL_NTB_B2B_JSF 0x3725
#define PCI_DEVICE_ID_INTEL_NTB_PS_JSF 0x3726
#define PCI_DEVICE_ID_INTEL_NTB_SS_JSF 0x3727
#define PCI_DEVICE_ID_INTEL_NTB_B2B_SNB 0x3C0D
#define PCI_DEVICE_ID_INTEL_NTB_PS_SNB 0x3C0E
#define PCI_DEVICE_ID_INTEL_NTB_SS_SNB 0x3C0F
#define PCI_DEVICE_ID_INTEL_NTB_B2B_IVT 0x0E0D
#define PCI_DEVICE_ID_INTEL_NTB_PS_IVT 0x0E0E
#define PCI_DEVICE_ID_INTEL_NTB_SS_IVT 0x0E0F
#define PCI_DEVICE_ID_INTEL_NTB_B2B_HSX 0x2F0D
#define PCI_DEVICE_ID_INTEL_NTB_PS_HSX 0x2F0E
#define PCI_DEVICE_ID_INTEL_NTB_SS_HSX 0x2F0F
#define PCI_DEVICE_ID_INTEL_NTB_B2B_BWD 0x0C4E
So we should modify the default device id to 0x2F0D, 0x2F0E or 0x2F0F ? What's the difference between them?
Thanks!
Yijing.
Signature · 2 lines
Thanks! Yijing
Quoted 5 lines · Yijing Wang
Hi Jon,
I have a Intel Haswell platform in hand, and our team want to use NTB in this platform.
I checked the current intel NTB driver in Linux kernel, I found the Haswell NTB pci device id is not contained in ntb_pci_tbl[]. I want to know whether current kernel ntb driver can support the ntb device in Haswell platform ?
Yes, it does support Haswell and the Device IDs are in there. PCI_DEVICE_ID_INTEL_NTB_B2B_HSX, PCI_DEVICE_ID_INTEL_NTB_PS_HSX, and PCI_DEVICE_ID_INTEL_NTB_SS_HSX are the relevant dev ids for Haswell.
Quoted 35 lines
Haswell NTB device id:
From Haswell EDS 7.4.2
did
Bus: 0 Device: 3 Function: 0 Offset: 2
Bit Attr Default Description
15:0 RO-V 2F08hDevice_Identification_Number— Device ID values vary from function to function. Bits 15:8 are equal to0x2F. The following list is a breakdown of the function groups.
0x2F00-0x2F1F: PCI Express and DMI2
0x2F20-0x2F3F: Integrated I/O Features
0x2F40-0x2F5F: Performance Monitors
0x2F80-0x2F9F: Intel QPI
0x2FA0-0x2FBF: Home Agent/Memory Controller
0x2FC0-0x2FDF: Power Management
0x2FE0-0x2FFF: Cbo/Ring
Default value may vary based on bus, device, and function of this CSR location.Current ntb_pci_tbl[] in Linux:
#define
PCI_DEVICE_ID_INTEL_NTB_B2B_JSF0x3725
#definePCI_DEVICE_ID_INTEL_NTB_PS_JSF0x3726
#definePCI_DEVICE_ID_INTEL_NTB_SS_JSF0x3727
#definePCI_DEVICE_ID_INTEL_NTB_B2B_SNB0x3C0D
#definePCI_DEVICE_ID_INTEL_NTB_PS_SNB0x3C0E
#definePCI_DEVICE_ID_INTEL_NTB_SS_SNB0x3C0F
#definePCI_DEVICE_ID_INTEL_NTB_B2B_IVT0x0E0D
#definePCI_DEVICE_ID_INTEL_NTB_PS_IVT0x0E0E
#definePCI_DEVICE_ID_INTEL_NTB_SS_IVT0x0E0F
#definePCI_DEVICE_ID_INTEL_NTB_B2B_HSX0x2F0D
#definePCI_DEVICE_ID_INTEL_NTB_PS_HSX0x2F0E
#definePCI_DEVICE_ID_INTEL_NTB_SS_HSX0x2F0F
#definePCI_DEVICE_ID_INTEL_NTB_B2B_BWD0x0C4E
Quoted 1 line
So we should modify the default device id to
0x2F0D,0x2F0Eor0x2F0F?
The device IDs are present above: PCI_DEVICE_ID_INTEL_NTB_B2B_HSX, PCI_DEVICE_ID_INTEL_NTB_PS_HSX, and PCI_DEVICE_ID_INTEL_NTB_SS_HSX.
Quoted 1 line
What's the difference between them?
The last 3 letters are the name of the CPU where NTB is found. HSX is Haswell Xeon. The 2-3 letters before that are the configuration type of the NTB device. B2B is for "Back-to-back" configurations, aka "NTB-NTB".
B2B
[CPU]---[NTB]===[NTB]---[CPU]
PS/SS is for NTB-RP configurations. PS is "Primary Side" and SS is "Secondary Side".
[CPU]---[SS|PS]---[CPU]
I have an NTB wiki on my github account
(https://github.com/jonmason/ntb/wiki) describing the configuration, etc. Also on the wiki is a link to a doc (not written by me, and contains references to a driver that was not made public) that has some graphics that might be useful. Specifically, pages 10 and 17. To save time, the URL is
http://download.intel.com/design/intarch/papers/323328.pdf
Let me know if you have any questions or issues, and I'll be happy to walk you through it.
Thanks,
Jon
Quoted 11 lines
Thanks!
Yijing.Signature · 2 lines
Thanks! Yijing
Hi Jon, Thanks for your detailed explanation. Now I have a clearer understanding of it.
Thanks! :)
Yijing.
Quoted 99 lines · Jon Mason
Quoted 5 lines · Yijing Wang
Hi Jon,
I have a Intel Haswell platform in hand, and our team want to use NTB in this platform.
I checked the current intel NTB driver in Linux kernel, I found the Haswell NTB pci device id is not contained in ntb_pci_tbl[]. I want to know whether current kernel ntb driver can support the ntb device in Haswell platform ?Yes, it does support Haswell and the Device IDs are in there.
PCI_DEVICE_ID_INTEL_NTB_B2B_HSX,PCI_DEVICE_ID_INTEL_NTB_PS_HSX, andPCI_DEVICE_ID_INTEL_NTB_SS_HSXare the relevant dev ids for Haswell.Quoted 35 lines
Haswell NTB device id:
From Haswell EDS 7.4.2
did
Bus: 0 Device: 3 Function: 0 Offset: 2
Bit Attr Default Description
15:0 RO-V 2F08hDevice_Identification_Number— Device ID values vary from function to function. Bits 15:8 are equal to0x2F. The following list is a breakdown of the function groups.
0x2F00-0x2F1F: PCI Express and DMI2
0x2F20-0x2F3F: Integrated I/O Features
0x2F40-0x2F5F: Performance Monitors
0x2F80-0x2F9F: Intel QPI
0x2FA0-0x2FBF: Home Agent/Memory Controller
0x2FC0-0x2FDF: Power Management
0x2FE0-0x2FFF: Cbo/Ring
Default value may vary based on bus, device, and function of this CSR location.Current ntb_pci_tbl[] in Linux:
#define
PCI_DEVICE_ID_INTEL_NTB_B2B_JSF0x3725
#definePCI_DEVICE_ID_INTEL_NTB_PS_JSF0x3726
#definePCI_DEVICE_ID_INTEL_NTB_SS_JSF0x3727
#definePCI_DEVICE_ID_INTEL_NTB_B2B_SNB0x3C0D
#definePCI_DEVICE_ID_INTEL_NTB_PS_SNB0x3C0E
#definePCI_DEVICE_ID_INTEL_NTB_SS_SNB0x3C0F
#definePCI_DEVICE_ID_INTEL_NTB_B2B_IVT0x0E0D
#definePCI_DEVICE_ID_INTEL_NTB_PS_IVT0x0E0E
#definePCI_DEVICE_ID_INTEL_NTB_SS_IVT0x0E0F
#definePCI_DEVICE_ID_INTEL_NTB_B2B_HSX0x2F0D
#definePCI_DEVICE_ID_INTEL_NTB_PS_HSX0x2F0E
#definePCI_DEVICE_ID_INTEL_NTB_SS_HSX0x2F0F
#definePCI_DEVICE_ID_INTEL_NTB_B2B_BWD0x0C4EQuoted 1 line
So we should modify the default device id to
0x2F0D,0x2F0Eor0x2F0F?The device IDs are present above:
PCI_DEVICE_ID_INTEL_NTB_B2B_HSX,PCI_DEVICE_ID_INTEL_NTB_PS_HSX, andPCI_DEVICE_ID_INTEL_NTB_SS_HSX.Quoted 1 line
What's the difference between them?
The last 3 letters are the name of the CPU where NTB is found. HSX is Haswell Xeon. The 2-3 letters before that are the configuration type of the NTB device. B2B is for "Back-to-back" configurations, aka "NTB-NTB".
B2B
[CPU]---[NTB]===[NTB]---[CPU]PS/SS is for NTB-RP configurations. PS is "Primary Side" and SS is "Secondary Side".
[CPU]---[SS|PS]---[CPU]
I have an NTB wiki on my github account
(https://github.com/jonmason/ntb/wiki) describing the configuration, etc. Also on the wiki is a link to a doc (not written by me, and contains references to a driver that was not made public) that has some graphics that might be useful. Specifically, pages 10 and 17. To save time, the URL is
http://download.intel.com/design/intarch/papers/323328.pdfLet me know if you have any questions or issues, and I'll be happy to walk you through it.
Thanks,
JonQuoted 11 lines
Thanks!
Yijing.Signature · 2 lines
Thanks! Yijing--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/.
Signature · 2 lines
Thanks! Yijing