Dual Athlon, AGP, and PCI

Here's the latest on my endeavors to get everything working full speed on my Tyan K7 Thunder system. First, a little background on the applicable hardware and software:

Dual Athlon 1.4GHz CPUs (*NOT* MP chips)
AMD-760 MP chipset, latest revision (as per AMD Rev. documents D/L last week)
- AMD-762 Northbridge Rev. 11
- AMD-766 Southbridge Rev. 01
Mainboard Adaptec 7899P Dual Channel SCSI 160
256MB ECC Reg. DDR
Asus V8200 Deluxe Geforce 3
IBM ATA100 30GB drive
IBM Ultra 160 36GB drive

I've been dealing with both 2.4.7 and 2.4.2 kernels, but I have mainly been using 2.4.2 because I started with that before I realized 2.4.7 was available. I also have the NVidia driver, version 1.0-1251.

I'm focusing mainly on AGP right now (I spent some good $$ on this GeForce 3 card and I want it running full speed :) so this is where I've made the most progress. I've modified Jeff's agpgart code to recognize the AMD-762 chip so the module loads without having to use "insmod agpgart agp_try_unsupported=1". By default the 762 comes up with 4x enabled, SBA and FW off and there are no setting in the system BIOS for AGP other than Aperture Memory size (which I set to 64MB). Re-compiling the NVidia driver after compiling the SMP kernel enables the Asus card to operate such:

[root@keroon /root]# cat /proc/nv/card0
----- Driver Info -----
NVRM Version: 1.0-1251
------ Card Info ------

Model:        NVIDIA card
IRQ:          17
------ AGP Info -------
AGP status:   Enabled
AGP Driver:   AGPGART
Bridge:       Generic AMD
SBA:          Supported [disabled]
FW:           Supported [disabled]
Rates:        4x 2x 1x  [4x]
Registers:    0x0f000217:0x00000104

[root@keroon /root]#

So, AGP 4x is enabled and things work OK, but the system still locks occasionally while playing Quake 3.

The NVidia driver, as advertised, will use its own AGP driver, Jeff's agpgart driver, or fall back to PCI mode. According to NVidia, you can enable the different modes by setting NVagp in XF86Config-4 to a number. I set it to use agpgart. Here's the rub.

Using the same software I used to generate the Kernel source documentation on my web server, I took a look at the NVidia driver. I found a lot of useful DEBUG code and I found some conditional code that allows the NVidia driver to use agpgart. Neither code section was being compiled either by the Makefile or the RPM .spec file. So, I modified the Makefile for NVdriver (-DDEBUG and -DAGPGART), compiled and installed NVdriver, and restarted X. Black screen and Ctrl+Alt+Del.

Examining /var/log/messages I found the following:

1. Enabling the agpgart support code results in the following errors:

Aug  2 00:53:44 keroon kernel: NVRM: AGPGART: AMD Irongate chipset
Aug  2 00:53:44 keroon kernel: NVRM: AGPGART: unable to set MTRR
write-combining
Aug  2 00:53:44 keroon kernel: NVRM: AGPGART: aperture: 64M @ 0xf4000000
Aug  2 00:53:44 keroon kernel: NVRM: AGPGART: aperture mapped from
0xf4000000 to 0xd59d9000
Aug  2 00:53:44 keroon kernel: NVRM: AGPGART: mode 4x
Aug  2 00:53:44 keroon kernel: NV0: ioctl(50, 0xbffffa38)
Aug  2 00:53:44 keroon kernel: NV0: ioctl(39, 0xbffff9f0)
Aug  2 00:53:44 keroon kernel: NVRM: AGPGART: allocated 16 pages
Aug  2 00:53:44 keroon kernel: NV0: mmap([0x48604000-0x48614000]
off=0x80000000
Aug  2 00:53:44 keroon kernel: NVctl: ioctl(36, 0xbffff9f8)
Aug  2 00:53:44 keroon kernel: NV0: ioctl(38, 0xbffff9ec)
Aug  2 00:53:44 keroon kernel: NV0: mmap([0x48614000-0x48624000]
off=0x800000
Aug  2 00:53:44 keroon kernel: NVctl: ioctl(43, 0xbffffa14)
Aug  2 00:53:44 keroon kernel: NVctl: ioctl(36, 0xbffffa24)
Aug  2 00:53:44 keroon kernel: NV0: could not find map for vm 0x40604000
Aug  2 00:53:44 keroon kernel: NV: vtop: 0x40604000 to p 0xf8000000
Aug  2 00:53:44 keroon kernel: NV0: ioctl(39, 0xbffffa1c)
Aug  2 00:53:44 keroon kernel: NV0: mmap([0x4002c000-0x4002d000]
off=0x40000000
Aug  2 00:53:44 keroon kernel: NVctl: ioctl(36, 0xbffffa24)
Aug  2 00:53:44 keroon kernel: NV0: could not find map for vm 0x4002c000

Note that early on in the kernel boot process, mtrr reports:

. . . CPU1<T0:2654864,T1:884944,D:4,S:884958,C:2654875> checking TSC synchronization across CPUs: passed. Setting commenced=1, go go go
mtrr: your CPUs had inconsistent fixed MTRR settings
mtrr: probably your BIOS does not setup all CPUs . . .

and later:

mtrr: type mismatch for f8000000,4000000 old: write-back new:
write-combining

2. NVidia's own AGP support code uses the kernel PCI functions. Because the kernel PCI code does not recognize the AMD-762 or AMD-766 chip IDs, a probe for an AGP device fails and generic support is assumed. AGP still works because NVDriver reads the capabilities registers which report AGP 4x is supported and enabled, but the extra features (FW and SBA) can not be enabled.

As for IDE, I can make it work in PIO Mode 4, Ultra DMA Mode 2 (66MHz), 32-bit transfers, and enable write caching (pushing it to 25MB/sec) using hdparm, but again the IDE (AMD-766) is not recognised and the IDE driver defaults to generic IDE ot boot (16-bit, no DMA, no cache, 2.5MB/sec).

The SCSI controllers work like a champ as do the onboard NICs (3c980 TX).

PGA

Paul G. Allen
UNIX Admin II/Network Security
Akamai Technologies, Inc.
www.akamai.com

Upgrade your kernel. 2.4.2 had broken mtrr support on Athlons.

Head Developer           | "...and we have C(n) = (n (n + 1))/ 2. Easy as pie.
Big Net, Inc.            |  Actually easier, Pi = Sum 8 / ((4n + 1)(4n + 3))."
                         |                   - Donald E. KNUTH

My mainboard is Abit KT7A, when I was using kernel 2.2.x I could use NV internal AGP support, after upgrading kernel to 2.4.x I have to use AGPGART, otherwise it will fall back to PCI mode.

You don't need to to do this, it is in NVdia README

Option "NvAGP" "integer"
Configure AGP support. Integer argument can be one of:
0 : disable agp
1 : use NVIDIA's internal AGP support, if possible 2 : use AGPGART, if possible
3 : use any agp support (try AGPGART, then NVIDIA's AGP)
Default: 1.

There are other options you can use.
see Appendix D NVGLX readme.

Oh well binary drivers sux, but better than setting aside another partition just for running Quake3 under Window$ :)

+-R-| Netscape Communicator 4.x |-H-| Powered by Linux 2.4.x |-7-+
|/v\ Agus Budy Wuysang                   MIS Department          |
| |  Phone:  +62-21-344-1316 ext 317     GSM: +62-816-1972-051   |
+--------| http://www.rad.net.id/users/personal/s/supes |--------+

Hello,

I just want to ask if there is a facility in the kernel that load
balance to different default routes, since i'm using this routes for uplink purposes only (my downlink is via satellite, it doesn't matter where i send my packets uplink as long as it reaches the internet backbone).

for example

	in my box, I have routes as describe below

	destination	gateway		netmask
	default         isp1 		0.0.0.0
	default		isp2		0.0.0.0

Want i want is for the kernel to load balance (e.g round robin) uplink packets to isp1 and isp2. If this in is not possible in the current kernel, where in the kernel source files can i start hacking to make this possible?

TIA

Rodel T. Viado
System Administrator
Iligan Global Access Network Inc.

You can look at my resume and Online Certification at:
http://www.brainbench.com/transcript.jsp?pid=1404199

Hi rtviado,

Friday, August 03, 2001, 6:02:48 PM, you wrote:

r> Hello,

r> I just want to ask if there is a facility in the kernel that load
r> balance to different default routes, since i'm using this routes for r> uplink purposes only (my downlink is via satellite, it doesn't matter r> where i send my packets uplink as long as it reaches the internet r> backbone).

r> for example

r> in my box, I have routes as describe below

r>         destination     gateway         netmask
r>         default         isp1            0.0.0.0
r>         default         isp2            0.0.0.0

r> Want i want is for the kernel to load balance (e.g round robin) uplink r> packets to isp1 and isp2. If this in is not possible in the current r> kernel, where in the kernel source files can i start hacking to make this r> possible?

r> TIA
As i know, there is a networking driver called bonding ,which can load balance sending packets through several ethernet connections, both 2.2.X and 2.4.X kernel have this driver.

Best regards,
Michael Chen                           mailto:[email protected]

In article <[email protected]>,

2.2+ support multipath routing with load balancing per route. You can configure it by specifying multiple nexthops with iproute2.

-Andi

[SNIP]

I'm not shure why. You stated that AGPGART worked with the try_unsopported=1 parameter (and that you had hacked the code to accept it anyways).

You do not need C experience to look at that code. It just states all possible module parameters in C form, and has a comment to them all.

Ill create a patch for it this weekend.

Regards
Anders Fugmann.

(Alan - please read below for the IDs you requested)

[SNIP]

Will not work on my system, but I haven't spent a lot of time on it yet.

Works. I changed the default in the code to 1 to enable it all the time (it can still be set to 0 [disabled] by setting it with insmod).

I looked and saw nothing else that might help me out. I did quickly scan the rest of the code and added the AMD-762 ID and description (BTW, anyone know what the REAL name of the MP chips are? I used Irongate MP), and changed the description for the GeForce ID to something more meaningful, to produce the following:

[root@keroon /root]# cat /proc/nv/card0
----- Driver Info -----
NVRM Version: 1.0-1251
------ Card Info ------

Model:        GeForce3 DDR
IRQ:          17
------ AGP Info -------
AGP status:   Enabled
AGP Driver:   AGPGART
Bridge:       AMD Irongate MP
SBA:          Supported [enabled]
FW:           Supported [disabled]
Rates:        4x 2x 1x  [4x]
Registers:    0x0f000217:0x00000304

[root@keroon /root]#

I also managed to get it to work with the NVidia AGP driver (as opposed to agpgart), but still without FW. I swear FW works on my A7V133 with this same video card.

Alan,

I created a new file - amd7411.c - based upon the previous file for
the AMD7409. I added IDs to pci_ids.h, pci.h, etc. in an effort to support the different AMD devices. My kernel loads the AMD7411 driver now, but does not enable DMA, etc., although using hdparm to enable everything seems more reliable now.

Below is the output from lspci -vvv and lspci -x for my system as it sits now. (Hmmm, looking at it I think I mis-named a couple of the devices. Oh well - too much work, too little sleep [thank you SirCam, Code Red, et. al] :)

[root@keroon /root]# lspci -vvv | more
00:00.0 Host bridge: Advanced Micro Devices [AMD] AMD-762 [Irongate MP] System Controller (rev 11)
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort+ >SERR- <PERR-

	Latency: 64
	Region 0: Memory at f4000000 (32-bit, prefetchable) [size=64M]
	Region 1: Memory at f0004000 (32-bit, prefetchable) [size=4K]
	Region 2: I/O ports at 1c30 [disabled] [size=4]
	Capabilities: [a0] AGP version 2.0
		Status: RQ=15 SBA+ 64bit- FW+ Rate=x1,x2
		Command: RQ=0 SBA+ AGP+ 64bit- FW- Rate=<none>

00:01.0 PCI bridge: Advanced Micro Devices [AMD] AMD-766 [Irongate MP] PCI Bridge (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

	Latency: 99
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=68
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: f2000000-f2ffffff
	Prefetchable memory behind bridge: f8000000-fc0fffff
	BridgeCtl: Parity- SERR- NoISA+ VGA+ MAbort- >Reset- FastB2B-

00:07.0 ISA bridge: Advanced Micro Devices [AMD] AMD-766 [Cobra MP] ISA (rev 02)
Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0

00:07.1 IDE interface: Advanced Micro Devices [AMD] AMD-766 [Cobra MP] IDE (rev 01) (prog-if 8a [Master SecP PriP])
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

	Latency: 64
	Region 4: I/O ports at f000 [size=16]

00:07.3 Bridge: Advanced Micro Devices [AMD] AMD-766 [Cobra MP] Bridge (rev 01)
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-

-------------------------

[root@keroon /root]# lspci -x | more
00:00.0 Host bridge: Advanced Micro Devices [AMD] AMD-762 [Irongate MP] System Controller (rev 11)
00: 22 10 0c 70 06 00 10 22 11 00 00 06 00 40 00 00
10: 08 00 00 f4 08 40 00 f0 31 1c 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 a0 00 00 00 00 00 00 00 00 00 00 00

00:01.0 PCI bridge: Advanced Micro Devices [AMD] AMD-766 [Irongate MP] PCI Bridge
00: 22 10 0d 70 07 00 20 02 00 00 04 06 00 63 01 00
10: 00 00 00 00 00 00 00 00 00 01 01 44 f1 01 20 22
20: 00 f2 f0 f2 00 f8 00 fc 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 ff 00 0c 00

00:07.0 ISA bridge: Advanced Micro Devices [AMD] AMD-766 [Cobra MP] ISA (rev 02)00: 22 10 10 74 0f 00 00 02 02 00 01 06 00 00 80 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:07.1 IDE interface: Advanced Micro Devices [AMD] AMD-766 [Cobra MP] IDE (rev 01)
00: 22 10 11 74 05 00 00 02 01 8a 01 01 00 40 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 01 f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:07.3 Bridge: Advanced Micro Devices [AMD] AMD-766 [Cobra MP] Bridge (rev 01)
00: 22 10 13 74 00 00 80 02 01 00 80 06 00 40 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

---------------------------------

PGA

--
Paul G. Allen
UNIX Admin II/Network Security
Akamai Technologies, Inc.
www.akamai.com

It's better (to me) to have it in the code than to have to add it at module load time. Also, I compiled agpgart into the kernel, not as a module.

Sorry for the confusion.

I have plenty of C/C++ experience, but anyone looking at 2M+ lines of sparsely commented source code and trying to get a quick understanding of how things work needs help.

Understand for C++ is a C/C++ source code browser. It parses a C/C++ source tree and creates documentation that allows easy browsing of call trees, include trees, declarations, global object usage, and more. It allowed me to quickly find the applicable code and see what was happening. I can even show you a complete graphic of the entire kernel start sequence from (I hope I remember the function name here) start_kernel to nearly any function thereafter.

The only caveat is sometimes in order to create all this you have to tell Understand which directory to find a header/include file in, and if you're not sure which is the correct one (in the case of multiple headers of the same name) you could get a little lost when looking at the resulting documentation.

In the case of the above - PCI IDs, AGP support, etc. - I have never looked at any kernel source before and I had no idea even where to start. After running it through Understand, a couple clicks of a mouse and I can find nearl any object I want and know where it's used and why.

Thanks,

PGA

Paul G. Allen
UNIX Admin II/Programmer
Akamai Technologies, Inc.
www.akamai.com
Work: (858)909-3630
Cell: (858)395-5043

The name of this chip is "Viper Plus".

Vojtech Pavlik
SuSE Labs

Thank you. :)

Paul G. Allen
UNIX Admin II/Network Security
Akamai Technologies, Inc.
www.akamai.com