Skip to contentSkip to conversation

aacraid and large memory problem (2.6.0-test11)

DISCUSSION Kevin Fenzi 3 participants 9 messages Dec 2, 2003 – Dec 11, 2003

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

Booting 2.6.0-test11 on a machine with 8GB memory and using the aacraid driver results in a hang on boot. Passing mem=2048M causes it to boot normally. 4GB also hangs. 2.6.0-test8 booted normally on this same hardware.

8GB memory, dual xeon 3.06mhz with hyperthreading, RedHat 9 on it currently.

Happy to provide details on setup/software, etc.

Perhaps this patch in 2.6.0-test9 is the culprit?
http://www.linuxhq.com/kernel/v2.6/0-test9/drivers/scsi/aacraid/comminit.c

On normal boot:

SCSI subsystem initialized
Red Hat/Adaptec aacraid driver (1.1.2 Nov 26 2003)
AAC0: kernel 4.0.4 build 6008
AAC0: monitor 4.0.4 build 6008
AAC0: bios 4.0.0 build 6008
AAC0: serial b7e06ffafaf001
AAC0: 64 Bit PAE enabled
scsi0 : aacraid
Vendor: ADAPTEC Model: Adaptec RAID5 Rev: V1.0
Type: Direct-Access ANSI SCSI revision: 02

On the hangs:

SCSI subsystem initialized
Loading sd_mod.ko module
Loading aacraid.ko module
Red Hat/Adaptec aacraid driver (1.1.2 Nov 26 2003)
aacraid: NMI ISR: NMI_BUS_INTERFACE_UNIT_ERROR
aacraid: <...repeats 7 more times>
aacraid: NormPrioCommand was received with Fib StructType = 0xff <hangs>

Thanks for any ideas.

kevin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD4DBQE/zOl43imCezTjY0ERAsH1AKCQ/xzLEEysLF+ewdEKXr5AGWplrgCWOs8f rXbYWCZSyPGSGrKQEq6z6w==
=xAVH
-----END PGP SIGNATURE-----

#1 lore raw
Quoted 11 lines · Kevin Fenzi

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings,

Booting 2.6.0-test11 on a machine with 8GB memory and using the aacraid driver results in a hang on boot. Passing mem=2048M causes it to boot normally. 4GB also hangs. 2.6.0-test8 booted normally on this same hardware.

Quoted 7 lines

8GB memory, dual xeon 3.06mhz with hyperthreading, RedHat 9 on it currently.

Happy to provide details on setup/software, etc.

Perhaps this patch in 2.6.0-test9 is the culprit?
http://www.linuxhq.com/kernel/v2.6/0-test9/drivers/scsi/aacraid/comminit.c

This patch is what made aacraid work with over 4 gig of memory for me. I have an 8 proc system with 16gig of memory and without this patch I get data corruption in high memory.

I don't boot on the aacraid though.

Signature · 1 line
Mark Haverkamp <[email protected]>
#2 lore raw

In article <[email protected]>, Mark Haverkamp <[email protected]> wrote:

| On Tue, 2003-12-02 at 11:35, Kevin Fenzi wrote:
| > -----BEGIN PGP SIGNED MESSAGE-----
| > Hash: SHA1
| > 
| > 
| > Greetings, 
| > 
| > Booting 2.6.0-test11 on a machine with 8GB memory and using the
| > aacraid driver results in a hang on boot. Passing mem=2048M causes it
| > to boot normally. 4GB also hangs. 2.6.0-test8 booted normally on this
| > same hardware. 
| > 
| > 8GB memory, dual xeon 3.06mhz with hyperthreading, RedHat 9 on it
| > currently. 
| > 
| > Happy to provide details on setup/software, etc. 
| > 
| > Perhaps this patch in 2.6.0-test9 is the culprit?
| > http://www.linuxhq.com/kernel/v2.6/0-test9/drivers/scsi/aacraid/comminit.c
| 
| This patch is what made aacraid work with over 4 gig of memory for me. 
| I have an 8 proc system with 16gig of memory and without this patch I
| get data corruption in high memory.
| 
| I don't boot on the aacraid though.

It would be interesting to know what memory model is being used in each case. Both CONFIG_HIGHMEM* and maybe user/kernel split might play.

Based on one boot with one machine, 4G RAM, it didn't hang. Unfortunately a production machine, I was playing following some "unscheduled maintenence."

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#3 lore raw
In reply to Mark Haverkamp

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mark" == Mark Haverkamp <[email protected]> writes:

Quoted 10 lines · Kevin Fenzi
Quoted 10 lines

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Greetings,

Booting 2.6.0-test11 on a machine with 8GB memory and using the aacraid driver results in a hang on boot. Passing mem=2048M causes it to boot normally. 4GB also hangs. 2.6.0-test8 booted normally on this same hardware.

Quoted 7 lines
Quoted 7 lines

8GB memory, dual xeon 3.06mhz with hyperthreading, RedHat 9 on it currently.

Happy to provide details on setup/software, etc.

Perhaps this patch in 2.6.0-test9 is the culprit?
http://www.linuxhq.com/kernel/v2.6/0-test9/drivers/scsi/aacraid/comminit.c

Mark> This patch is what made aacraid work with over 4 gig of memory Mark> for me. I have an 8 proc system with 16gig of memory and without Mark> this patch I get data corruption in high memory.

Mark> I don't boot on the aacraid though.

Is there any way you can try booting from it and see if it's a boot issue for you as well?

I can try booting the one here from something else and see if it works with that.

kevin

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQE/zkzd3imCezTjY0ERAtdRAJ9NIp56DWRFI6zxpbgyLtKQzkYcIACfYKil Z6XcmnrXQ9Qsiy24d7ac044=
=a+PX
-----END PGP SIGNATURE-----

#4 lore raw
Quoted 24 lines · Kevin Fenzi

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mark" == Mark Haverkamp <[email protected]> writes:

Quoted 10 lines · Kevin Fenzi
Quoted 10 lines

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Greetings,

Booting 2.6.0-test11 on a machine with 8GB memory and using the aacraid driver results in a hang on boot. Passing mem=2048M causes it to boot normally. 4GB also hangs. 2.6.0-test8 booted normally on this same hardware.

Quoted 7 lines
Quoted 7 lines

8GB memory, dual xeon 3.06mhz with hyperthreading, RedHat 9 on it currently.

Happy to provide details on setup/software, etc.

Perhaps this patch in 2.6.0-test9 is the culprit?
http://www.linuxhq.com/kernel/v2.6/0-test9/drivers/scsi/aacraid/comminit.c

Quoted 8 lines

Mark> This patch is what made aacraid work with over 4 gig of memory Mark> for me. I have an 8 proc system with 16gig of memory and without Mark> this patch I get data corruption in high memory.

Mark> I don't boot on the aacraid though.

Is there any way you can try booting from it and see if it's a boot issue for you as well?

I set up my machine to boot on the aacraid disk and it booted OK for me. Maybe its a problem with a particular model?

lspci on mine says:

02:04.0 RAID bus controller: Digital Equipment Corporation DECchip 21554 (rev 01)
Subsystem: Adaptec Adaptec 5400S

Quoted 6 lines

I can try booting the one here from something else and see if it works with that.

kevin

Signature · 1 line
Mark Haverkamp <[email protected]>
#5 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mark" == Mark Haverkamp <[email protected]> writes:

Mark> I set up my machine to boot on the aacraid disk and it booted OK Mark> for me. Maybe its a problem with a particular model?

Mark> lspci on mine says:

Mark> 02:04.0 RAID bus controller: Digital Equipment Corporation Mark> DECchip 21554 (rev 01) Subsystem: Adaptec Adaptec 5400S

This one says:

05:01.0 RAID bus controller: Adaptec AAC-RAID (rev 01)
Subsystem: Adaptec AAC-RAID
Flags: bus master, fast Back2Back, 66Mhz, slow devsel, latency 64, IRQ 96 Memory at f8000000 (32-bit, prefetchable) [size=64M] Expansion ROM at <unassigned> [disabled] [size=64K]
Capabilities: [80] Power Management version 2

It's a 2200S controller.
bios version 6008

Mark> -- Mark Haverkamp <[email protected]>

kevin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQE/zmOY3imCezTjY0ERAtOeAKCCCos81FiV0oy89ojnJQiTCK212QCgkO8/ 1rARggxxE7xP8YVggM2f6vI=
=+t3d
-----END PGP SIGNATURE-----

#6 lore raw

In article <[email protected]>, Kevin Fenzi <[email protected]> wrote:

| 
| >>>>> "Mark" == Mark Haverkamp <[email protected]> writes:
| 
| Mark> I set up my machine to boot on the aacraid disk and it booted OK
| Mark> for me.  Maybe its a problem with a particular model?
| 
| Mark> lspci on mine says:
| 
| Mark> 02:04.0 RAID bus controller: Digital Equipment Corporation
| Mark> DECchip 21554 (rev 01) Subsystem: Adaptec Adaptec 5400S
| 
| This one says: 
| 
| 05:01.0 RAID bus controller: Adaptec AAC-RAID (rev 01)
|         Subsystem: Adaptec AAC-RAID
|         Flags: bus master, fast Back2Back, 66Mhz, slow devsel, latency 64, IRQ 96
|         Memory at f8000000 (32-bit, prefetchable) [size=64M]
|         Expansion ROM at <unassigned> [disabled] [size=64K]
|         Capabilities: [80] Power Management version 2
| 
| It's a 2200S controller. 
| bios version 6008
| 
| Mark> -- Mark Haverkamp <[email protected]>
| 
| kevin

02:03.0 RAID bus controller: Digital Equipment Corporation DECchip 21554 (rev 01)
Subsystem: Adaptec Dell PowerEdge RAID Controller 2
Flags: bus master, medium devsel, latency 32, IRQ 18
Memory at fe200000 (32-bit, non-prefetchable) [size=8K]
I/O ports at f800 [size=256]
Expansion ROM at fd000000 [disabled] [size=128K]
Capabilities: <available only to root>

Sorry - it's back on 2.4, just thought this might be useful.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#7 lore raw
In reply to Kevin Fenzi
Quoted 22 lines · Kevin Fenzi

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mark" == Mark Haverkamp <[email protected]> writes:

Mark> I set up my machine to boot on the aacraid disk and it booted OK Mark> for me. Maybe its a problem with a particular model?

Mark> lspci on mine says:

Mark> 02:04.0 RAID bus controller: Digital Equipment Corporation Mark> DECchip 21554 (rev 01) Subsystem: Adaptec Adaptec 5400S

This one says:

05:01.0 RAID bus controller: Adaptec AAC-RAID (rev 01)
Subsystem: Adaptec AAC-RAID
Flags: bus master, fast Back2Back, 66Mhz, slow devsel, latency 64, IRQ 96 Memory at f8000000 (32-bit, prefetchable) [size=64M] Expansion ROM at <unassigned> [disabled] [size=64K]
Capabilities: [80] Power Management version 2

Quoted 3 lines

It's a 2200S controller.
bios version 6008

Could you try this patch? I took the code from the adaptec version of the driver. It fiddles with the dma mask on the 2200S controller among others.

Mark.

===== drivers/scsi/aacraid/aachba.c 1.20 vs edited =====

drivers/scsi/aacraid/aachba.c +8 -0
@@ -525,6 +525,14 @@
 	if(dev->pae_support != 0) {
 		printk(KERN_INFO"%s%d: 64 Bit PAE enabled\n", dev->name, dev->id);
 		pci_set_dma_mask(dev->pdev, (dma_addr_t)0xFFFFFFFFFFFFFFFFULL);
+	} else {
+		/* 
+		 * Reset if Quirk 31 was used, since data 
+		 * transfers are ok.
+		 */
+		if (pci_set_dma_mask(dev->pdev, 0xFFFFFFFFULL)) {
+			printk(KERN_INFO"aacraid: Can't reset DMA mask.\n");
+		}
 	}
 
 	fib_complete(fibptr);

===== drivers/scsi/aacraid/aacraid.h 1.9 vs edited =====

drivers/scsi/aacraid/aacraid.h +2 -0
@@ -529,6 +529,8 @@
 	char *	vname;
 	char *	model;
 	u16	channels;
+	int	quirks;
+#define AAC_QUIRK_31BIT         1
 };
 
 /*

===== drivers/scsi/aacraid/linit.c 1.22 vs edited =====

drivers/scsi/aacraid/linit.c +48 -28

97 lines not on this page · raw message

Signature · 1 line
Mark Haverkamp <[email protected]>
#8 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mark" == Mark Haverkamp <[email protected]> writes:

Quoted 4 lines · Kevin Fenzi
Quoted 4 lines

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Mark" == Mark Haverkamp <[email protected]> writes:

Mark> I set up my machine to boot on the aacraid disk and it booted OK Mark> for me. Maybe its a problem with a particular model?

Quoted 1 line
Quoted 1 line

Mark> lspci on mine says:

Quoted 1 line
Quoted 1 line

Mark> 02:04.0 RAID bus controller: Digital Equipment Corporation Mark> DECchip 21554 (rev 01) Subsystem: Adaptec Adaptec 5400S

Quoted 10 lines
Quoted 10 lines

This one says:

05:01.0 RAID bus controller: Adaptec AAC-RAID (rev 01) Subsystem:
Adaptec AAC-RAID Flags: bus master, fast Back2Back, 66Mhz, slow devsel, latency 64, IRQ 96 Memory at f8000000 (32-bit, prefetchable) [size=64M] Expansion ROM at <unassigned> [disabled] [size=64K] Capabilities: [80] Power Management version 2

It's a 2200S controller. bios version 6008

Mark> Could you try this patch? I took the code from the adaptec Mark> version of the driver. It fiddles with the dma mask on the Mark> 2200S controller among others.

Sorry it took me so long to try it. ;)

Just applied to a 2.6.0-test11 kernel and rebooted and it worked! Boot came up fine and it sees all 8GB of memory.

free
total used free shared buffers cached
Mem: 8290996 712380 7578616 0 20956 569352
- -/+ buffers/cache: 122072 8168924
Swap: 8385920 0 8385920

Red Hat/Adaptec aacraid driver (1.1.2 Dec 10 2003)
AAC0: kernel 4.0.4 build 6008
AAC0: monitor 4.0.4 build 6008
AAC0: bios 4.0.0 build 6008
AAC0: serial b7e06ffafaf001
AAC0: 64 Bit PAE enabled
scsi0 : aacraid

This looks like a good patch here.

Mark> Mark.

kevin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQE/2K8l3imCezTjY0ERAr0aAJkBLLGSLT5f4XygLR7uTwvQ6bQWSwCeIpUQ GNFgepA9aOlk0MMUGDNHGmY=
=wOp5
-----END PGP SIGNATURE-----