Skip to contentSkip to conversation

RAID corruption with kernel 3.2?

DISCUSSION H. Peter Anvin 3 participants 6 messages Feb 12, 2012 MD RAID

lore
Share & tools
lore
#0 lore raw

Has anyone seen RAID-5 or -6 corruption with kernel 3.2 on systems previously stable? If so, please let me know your hardware configuration... we're trying to track down a problem, and RAID *might* be part of it...

-hpa

Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.
#1 lore raw
Quoted 7 lines · H. Peter Anvin

Has anyone seen RAID-5 or -6 corruption with kernel 3.2 on systems previously stable?  If so, please let me know your hardware configuration... we're trying to track down a problem, and RAID *might* be part of it...

       -hpa

Quoted 8 lines

--
H. Peter Anvin, Intel Open Source Technology Center I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Details? My >3.2 RAID6 still works fine.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
Quoted 3 lines · Mathias Burén

Details? My>3.2 RAID6 still works fine.

What is your hardware configuration? Especially your CPU?

The current suspicion -- but we don't know yet -- is that this is a problem on Sandy Bridge and some Penryn CPUs which have the XSAVE instruction, but we might very well be completely wrong on that.

-hpa
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 6 lines · H. Peter Anvin
Quoted 4 lines · Mathias Burén

Details? My>3.2 RAID6 still works fine.

Quoted 7 lines

What is your hardware configuration?  Especially your CPU?

The current suspicion -- but we don't know yet -- is that this is a problem on Sandy Bridge and some Penryn CPUs which have the XSAVE instruction, but we might very well be completely wrong on that.

       -hpa

The CPU is an old Intel Atom, so nothing special. 3.2.5 with CK patches.

Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to H. Peter Anvin

Hello,

Quoted 9 lines · H. Peter Anvin
Quoted 3 lines · Mathias Burén

Details? My>3.2 RAID6 still works fine.

What is your hardware configuration? Especially your CPU?

The current suspicion -- but we don't know yet -- is that this is a problem on Sandy Bridge and some Penryn CPUs which have the XSAVE instruction, but we might very well be completely wrong on that.

I have NOT seen any corruptions. Have a dual CPU (8 cores) system with the following CPU:

cat /proc/cpuinfo

   processor       : 0
   vendor_id       : GenuineIntel
   cpu family      : 6
   model           : 23
   model name      : Intel(R) Xeon(R) CPU           X5460  @ 3.16GHz
   stepping        : 10
   microcode       : 0xa07
   cpu MHz         : 1999.000
   cache size      : 6144 KB
   physical id     : 0
   siblings        : 4
   core id         : 0
   cpu cores       : 4
   apicid          : 0
   initial apicid  : 0
   fpu             : yes
   fpu_exception   : yes
   cpuid level     : 13
   wp              : yes
   flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
   bogomips        : 6317.90
   clflush size    : 64

cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:

Has the following softraids:

cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md3 : active raid1 sdf2[1] sdc2[0]

         70613632 blocks [2/2] [UU]

md1 : active raid1 sdd2[1] sda2[0]

         70613632 blocks [2/2] [UU]

md2 : active raid1 sdb2[0] sde2[1]

         70613632 blocks [2/2] [UU]

md4 : active raid6 sdn1[7] sdm1[6] sdk1[4] sdl1[5] sdh1[1] sdj1[3] sdi1[2] sdg1[0]

         1754480640 blocks level 6, 2048k chunk, algorithm 2 [8/8] [UUUUUUUU]

md0 : active raid1 sdb1[1] sdd1[3] sdf1[5] sde1[4] sdc1[2] sda1[0]

         521984 blocks [6/6] [UUUUUU]

The devices under md4 are hardware raid1 on two external SAS boxes. Kernel currently is 3.2.1 with some security fixes. This system distributes daily more then 5 TiB (9 million files) and so far no one has reported any problems to me. I have several other systems with the same CPU (lower clock) and software Raid6 but with local attached disks. They also have no problems. Kernels there are 3.2.5.

Regards,
Holger

#5 lore raw
Quoted 29 lines · Holger Kiehl
Quoted 5 lines

The current suspicion -- but we don't know yet -- is that this is a problem on Sandy Bridge and some Penryn CPUs which have the XSAVE instruction, but we might very well be completely wrong on that.

I have NOT seen any corruptions. Have a dual CPU (8 cores) system with the following CPU:

cat /proc/cpuinfo

   processor       : 0
   vendor_id       : GenuineIntel
   cpu family      : 6
   model           : 23
   model name      : Intel(R) Xeon(R) CPU           X5460  @ 3.16GHz
   stepping        : 10
   microcode       : 0xa07
   cpu MHz         : 1999.000
   cache size      : 6144 KB
   physical id     : 0
   siblings        : 4
   core id         : 0
   cpu cores       : 4
   apicid          : 0
   initial apicid  : 0
   fpu             : yes
   fpu_exception   : yes
   cpuid level     : 13
   wp              : yes
   flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
Quoted 10 lines

pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
bogomips : 6317.90
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:

Okay, so that's Penryn, it has XSAVE but not XSAVEOPT. Very interesting data point, I appreciate the info!

-hpa

Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.