Skip to contentSkip to conversation

LVM System

linux-lvm Anthony W. Marino 7 participants 25 messages Mar 4, 2002

lore
Share & tools
lore
#0 lore raw

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Thank You,
Anthony

#1 lore raw

-- "Anthony W. Marino" <[email protected]> on 03/04/02 16:39:10 -0500

Quoted 8 lines

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Striped LVM on RAID5 is a nice, flexable way to go. If you set the individual raid stripes at an I/O page (normally 4K on linux) and then stripe across the PV's the I/O gets spread nicely across plenty of paths. Using a stripe == I/O size avoids the "write penalty" in RAID5, the Raid Controller and drives do a nice job of buffering and the whole thing should work nicely.

--
Steven Lembark 2930 W. Palmer
Workhorse Computing Chicago, IL 60647

                                           +1 800 762 1582
#2 lore raw
Quoted 11 lines · On Monday 04 March 2002 04:49 pm, [email protected] wrote:

-- "Anthony W. Marino" <[email protected]> on 03/04/02 16:39:10 -0500

Quoted 8 lines

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 7 lines

Striped LVM on RAID5 is a nice, flexable way to go. If you set the individual raid stripes at an I/O page (normally 4K on linux) and then stripe across the PV's the I/O gets spread nicely across plenty of paths. Using a stripe == I/O size avoids the "write penalty" in RAID5, the Raid Controller and drives do a nice job of buffering and the whole thing should work nicely.

Do you recommend any particular kernel release?
Where could I find good instructions on putting it all together?

Thank You,
Anthony

#3 lore raw
In reply to Anthony W. Marino
Quoted 8 lines · Anthony W. Marino

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

My only comment is that my new setup (specs below) has worked nicely on a production machine for about a month now.

LVM 1.0.1-rc4
3Ware 7410 RAID Controller
2x IBM 60GB hard drives
XFS Filesystem
RedHat 7.2

Signature · 3 lines
Randomly Generated Tagline:
#define NULL 0           /* silly thing is, we don't even use this */
              -- Larry Wall in perl.c from the perl source code
#4 lore raw
Quoted 10 lines · Theo Van Dinter
Quoted 8 lines · Anthony W. Marino

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

My only comment is that my new setup (specs below) has worked nicely on a production machine for about a month now.

LVM 1.0.1-rc4
3Ware 7410 RAID Controller
2x IBM 60GB hard drives
XFS Filesystem
RedHat 7.2

When I went to 3ware for 7800 driver for SuSE I didn't find one for that particular release. I found one for SuSE 7.0 and wonder if I could use that one for my SuSE 7.3 system.

#5 lore raw
Quoted 20 lines · Anthony W. Marino
Quoted 10 lines · Theo Van Dinter
Quoted 8 lines · Anthony W. Marino

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

My only comment is that my new setup (specs below) has worked nicely on a production machine for about a month now.

LVM 1.0.1-rc4
3Ware 7410 RAID Controller
2x IBM 60GB hard drives
XFS Filesystem
RedHat 7.2

Quoted 4 lines

When I went to 3ware for 7800 driver for SuSE I didn't find one for that particular release. I found one for SuSE 7.0 and wonder if I could use that one for my SuSE 7.3 system.

I think I found an reply to an old message from me that states it would work
just fine. How do you like the 7410?

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#6 lore raw
In reply to Anthony W. Marino
Quoted 8 lines · Anthony W. Marino

When I went to 3ware for 7800 driver for SuSE I didn't find one for that particular release. I found one for SuSE 7.0 and wonder if I could use that one for my SuSE 7.3 system.

I think I found an reply to an old message from me that states it would work
just fine. How do you like the 7410?

What RAID configuration are you running?

It's the 3w-xxxx kernel module. It seems to be standard in the 2.4.x series kernels, I didn't have to do any compiling to get it to work. I don't know what kernel the SuSE distros use though.

The 7410 works nicely. I only planned to do RAID1, so I didn't go for the extra cache 7450. The only thing I don't like about it so far is the RAID manager tool that you get from 3ware has a really terribly installation script (let's figure out which distro I'm running on based on the kernel version I'm running... yeah! <G>) So I haven't installed it yet. It's also looking like they only have a GUI tool, which means it'll be difficult getting a monitor setup to watch for failed disks. :(

Signature · 2 lines
Randomly Generated Tagline:
"I hear you collect plants.  So does my wife."      - Napoleon
#7 lore raw
Quoted 5 lines · Theo Van Dinter
Quoted 4 lines · Anthony W. Marino

When I went to 3ware for 7800 driver for SuSE I didn't find one for that particular release. I found one for SuSE 7.0 and wonder if I could use that one for my SuSE 7.3 system.

Quoted 9 lines
Quoted 4 lines

I think I found an reply to an old message from me that states it would
work just fine. How do you like the 7410?

What RAID configuration are you running?

It's the 3w-xxxx kernel module. It seems to be standard in the 2.4.x series kernels, I didn't have to do any compiling to get it to work. I don't know what kernel the SuSE distros use though.

I don't understand. Are you saying the all the software I need is contained within my kernel or do I still need to get a driver from 3Ware? I'm running SuSE Mantel 2.4.18 which the changes text states it has 2.4.19-pre1 as well as XFS.

Quoted 7 lines

The 7410 works nicely. I only planned to do RAID1, so I didn't go for the extra cache 7450. The only thing I don't like about it so far is the RAID manager tool that you get from 3ware has a really terribly installation script (let's figure out which distro I'm running on based on the kernel version I'm running... yeah! <G>) So I haven't installed it yet. It's also looking like they only have a GUI tool, which means it'll be difficult getting a monitor setup to watch for failed disks. :(

#8 lore raw
Quoted 4 lines · Anthony W. Marino

I don't understand. Are you saying the all the software I need is contained within my kernel or do I still need to get a driver from 3Ware? I'm running SuSE Mantel 2.4.18 which the changes text states it has 2.4.19-pre1 as well as XFS.

When I installed the SGI 1.0.2 kernel on my machine (2.4.9), it already had the latest (1.02.00.008) 3w-xxxx module in place. I doubt the SGI folks added it, so I'm assuming it's a standard module in the 2.4.x series kernels.

If you want the "manager" software (which really just lets you see status/configuration), that's a seperate piece you can download.

Signature · 5 lines
Randomly Generated Tagline:
There's often more than one correct thing.
 There's often more than one right thing.
 There's often more than one obvious thing.
              -- Larry Wall in <[email protected]>
#9 lore raw
Quoted 6 lines · Theo Van Dinter
Quoted 4 lines · Anthony W. Marino

I don't understand. Are you saying the all the software I need is contained within my kernel or do I still need to get a driver from 3Ware? I'm running SuSE Mantel 2.4.18 which the changes text states it has 2.4.19-pre1 as well as XFS.

Quoted 7 lines

When I installed the SGI 1.0.2 kernel on my machine (2.4.9), it already had the latest (1.02.00.008) 3w-xxxx module in place. I doubt the SGI folks added it, so I'm assuming it's a standard module in the 2.4.x series kernels.

If you want the "manager" software (which really just lets you see status/configuration), that's a seperate piece you can download.

Thank You,
Anthony

#10 lore raw
In reply to Theo Van Dinter
Quoted 10 lines · Theo Van Dinter
Quoted 8 lines · Anthony W. Marino

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

My only comment is that my new setup (specs below) has worked nicely on a production machine for about a month now.

LVM 1.0.1-rc4
3Ware 7410 RAID Controller
2x IBM 60GB hard drives
XFS Filesystem
RedHat 7.2

What RAID configuration are you running?

#11 lore raw
In reply to Anthony W. Marino

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

Quoted 27 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#12 lore raw
Quoted 9 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

It was recommended to me that all large file systems have best performance with XFS.

Quoted 32 lines
Quoted 18 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining the quality on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#13 lore raw

Well, thats possible. The best thing to do is run benchmarks (I like iozone) on the proposed
system and see what works best.
The tests I did were just on small filesystems (1G volumes, 64M, 128M and 196M RAM). Filesizes I tested were only up to 500M or so.
I found that while most tested filesystems (reiser, XFS, ext3 and vfat) did drop off in performance as filesize increased, XFS dropped off the most rapidly in that dimension. ext3 was the most stable (in performance as filesize increased), though its small filesize performance was the worst. XFS looked like kinda the opposite!
:)
Reiser had the worst write performance of all of them, but admirable read.

YMMV.

Quoted 69 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 12:50 p.m.
To: [email protected]; Steve Wray
Subject: Re: [linux-lvm] LVM System

Quoted 9 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

It was recommended to me that all large file systems have best performance
with XFS.

Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines
Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining

the quality

Quoted 18 lines
Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#14 lore raw

Hmm..that could be so, but I'm not sure if it's JUST xfs or if it's XFS/LVM. My largefile performance is excellent, as is my small file perfromance.

Quoted 96 lines · Steve Wray

Well, thats possible. The best thing to do is run benchmarks (I like iozone) on the proposed
system and see what works best.
The tests I did were just on small filesystems (1G volumes, 64M, 128M and 196M RAM). Filesizes I tested were only up to 500M or so.
I found that while most tested filesystems (reiser, XFS, ext3 and vfat) did drop off in performance as filesize increased, XFS dropped off the most rapidly in that dimension. ext3 was the most stable (in performance as filesize increased), though its small filesize performance was the worst. XFS looked like kinda the opposite!
:)
Reiser had the worst write performance of all of them, but admirable read.

YMMV.

Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 61 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 12:50 p.m.
To: [email protected]; Steve Wray
Subject: Re: [linux-lvm] LVM System

Quoted 9 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

It was recommended to me that all large file systems have best performance
with XFS.

Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines
Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining

the quality

Quoted 18 lines
Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 6 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Signature · 8 lines
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin it."
Latin Proverb
#15 lore raw

Fair enough; I have to admit that I did *no* benchmarking of a simple partitioned filesystem, only LVM. Its so *easy* to just make a new volume for benchmarking :)
I was lazy... my bad!

Quoted 125 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Austin Gonyou
Sent: Tuesday, 5 March 2002 1:12 p.m.
To: [email protected]
Subject: RE: [linux-lvm] LVM System

Hmm..that could be so, but I'm not sure if it's JUST xfs or if it's XFS/LVM. My largefile performance is excellent, as is my small file perfromance.

Quoted 91 lines · Steve Wray

Well, thats possible. The best thing to do is run benchmarks (I like iozone) on the proposed
system and see what works best.
The tests I did were just on small filesystems (1G volumes, 64M, 128M and 196M RAM). Filesizes I tested were only up to 500M or so.
I found that while most tested filesystems (reiser, XFS, ext3 and vfat) did drop off in performance as filesize increased, XFS dropped off the most rapidly in that dimension. ext3 was the most stable (in performance as filesize increased), though its small filesize performance was the worst. XFS looked like kinda the opposite!
:)
Reiser had the worst write performance of all of them, but admirable read.

YMMV.

Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 61 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 12:50 p.m.
To: [email protected]; Steve Wray
Subject: Re: [linux-lvm] LVM System

Quoted 9 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

It was recommended to me that all large file systems have best performance
with XFS.

Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines
Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining

the quality

Quoted 18 lines
Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 6 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

--
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin it." Latin Proverb

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#16 lore raw

Wooohoo!! That's for sure.

BTW..are you using IDE or SCSI?

That is an ENOURMOUS difference, I've found, to LVM/XFS performance Benchmarking.

Quoted 143 lines · Steve Wray

Fair enough; I have to admit that I did *no* benchmarking of a simple partitioned filesystem, only LVM. Its so *easy* to just make a new volume for benchmarking :)
I was lazy... my bad!

Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 6 lines

Behalf Of Austin Gonyou
Sent: Tuesday, 5 March 2002 1:12 p.m.
To: [email protected]
Subject: RE: [linux-lvm] LVM System

Quoted 37 lines

Hmm..that could be so, but I'm not sure if it's JUST xfs or if it's XFS/LVM. My largefile performance is excellent, as is my small file perfromance.

Quoted 32 lines · Steve Wray

Well, thats possible. The best thing to do is run benchmarks (I like iozone) on the proposed
system and see what works best.
The tests I did were just on small filesystems (1G volumes, 64M, 128M and 196M RAM). Filesizes I tested were only up to 500M or so.
I found that while most tested filesystems (reiser, XFS, ext3 and vfat) did drop off in performance as filesize increased, XFS dropped off the most rapidly in that dimension. ext3 was the most stable (in performance as filesize increased), though its small filesize performance was the worst. XFS looked like kinda the opposite!
:)
Reiser had the worst write performance of all of them, but admirable read.

YMMV.

Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 9 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 12:50 p.m.
To: [email protected]; Steve Wray
Subject: Re: [linux-lvm] LVM System

Quoted 2 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better

with

Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
Quoted 7 lines

striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

It was recommended to me that all large file systems have best performance
with XFS.

Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines
Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining

the quality

Quoted 9 lines
Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at

http://www.sistina.com/lvm/Pages/howto.html

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at

http://www.sistina.com/lvm/Pages/howto.html

Quoted 13 lines
Quoted 8 lines
Quoted 7 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 8 lines

--
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin

it."

Quoted 8 lines

Latin Proverb

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Signature · 8 lines
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin it."
Latin Proverb
#17 lore raw

My benchmarking was done on IDE, with an offboard controller and 4 drives as masters.
There should be some more info in the list archive; I posted some burble here not long ago, describing just what I was doing. I described it as 'poor mans RAID'.

Quoted 175 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Austin Gonyou
Sent: Tuesday, 5 March 2002 1:32 p.m.
To: [email protected]
Subject: RE: [linux-lvm] LVM System

Wooohoo!! That's for sure.

BTW..are you using IDE or SCSI?

That is an ENOURMOUS difference, I've found, to LVM/XFS performance Benchmarking.

Quoted 138 lines · Steve Wray

Fair enough; I have to admit that I did *no* benchmarking of a simple partitioned filesystem, only LVM. Its so *easy* to just make a new volume for benchmarking :)
I was lazy... my bad!

Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 6 lines

Behalf Of Austin Gonyou
Sent: Tuesday, 5 March 2002 1:12 p.m.
To: [email protected]
Subject: RE: [linux-lvm] LVM System

Quoted 37 lines

Hmm..that could be so, but I'm not sure if it's JUST xfs or if it's XFS/LVM. My largefile performance is excellent, as is my small file perfromance.

Quoted 32 lines · Steve Wray

Well, thats possible. The best thing to do is run benchmarks (I like iozone) on the proposed
system and see what works best.
The tests I did were just on small filesystems (1G volumes, 64M, 128M and 196M RAM). Filesizes I tested were only up to 500M or so.
I found that while most tested filesystems (reiser, XFS, ext3 and vfat) did drop off in performance as filesize increased, XFS dropped off the most rapidly in that dimension. ext3 was the most stable (in performance as filesize increased), though its small filesize performance was the worst. XFS looked like kinda the opposite!
:)
Reiser had the worst write performance of all of them, but admirable read.

YMMV.

Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 9 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 12:50 p.m.
To: [email protected]; Steve Wray
Subject: Re: [linux-lvm] LVM System

Quoted 2 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better

with

Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
Quoted 7 lines

striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

It was recommended to me that all large file systems have best performance
with XFS.

Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines
Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining

the quality

Quoted 9 lines
Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at

http://www.sistina.com/lvm/Pages/howto.html

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at

http://www.sistina.com/lvm/Pages/howto.html

Quoted 13 lines
Quoted 8 lines
Quoted 7 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 8 lines

--
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin

it."

Quoted 8 lines

Latin Proverb

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

--
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin it." Latin Proverb

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#18 lore raw
In reply to Austin Gonyou
Quoted 9 lines · Austin Gonyou

Wooohoo!! That's for sure.

BTW..are you using IDE or SCSI?

That is an ENOURMOUS difference, I've found, to LVM/XFS performance Benchmarking.

Quoted 1 line · Steve Wray
Quoted 1 line

[ 158 lines janked ]

<rant>
Well, I know that it's november for ever (since AOL did connect to the internet, usenet actually).

But people, please learn to quote.
It was impossible to figure out WHAT you where refering, too.

see e.g. http://www.inet.bg/faq/netiquete.html
</rant>

Sorry, please keep up the good work with LVM,
Goetz.

#19 lore raw
Quoted 6 lines · Goetz Bock
Quoted 4 lines · Austin Gonyou

Wooohoo!! That's for sure.

BTW..are you using IDE or SCSI?

Quoted 8 lines
Quoted 5 lines

That is an ENOURMOUS difference, I've found, to LVM/XFS performance Benchmarking.

Quoted 1 line · Steve Wray
Quoted 1 line

[ 158 lines janked ]

<rant>
Well, I know that it's november for ever (since AOL did connect to the internet, usenet actually).

ITYM September.
http://www.tuxedo.org/~esr/jargon/html/entry/September-that-never-ended.html

Quoted 1 line

But people, please learn to quote.

Hear Hear!.

Signature · 1 line
Share and Enjoy. 
#20 lore raw
In reply to Steve Wray
Quoted 9 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 6 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

I've only got a couple of things to say in the regard about XFS and performance Falling off.

XFS's performance doesn't fall off in the respects you've spoken about. It's a misconception when using LVM, or evms, etc along with XFS.

I ran into a similar issue, and was convinced it was XFS or LVM, etc. This is in the relation of using XFS and LVM, vs. XFS with no LVM, and then testing increasing filesizes with iozone.

What I found was that dbench, and iozone both did not perform very well with respect to software raid or striping, but that in fact the filesystems themselves performed about 100% faster than without LVM striping or LVM Concat.

So much faster that my database guys noticed the enourmous increase in speed on a test setup I created for them. One was using LVM, one was not, with XFS. The one which was NOT using LVM had a hardware stripe setup with write-back caching and adaptive read-ahead. The controller has 128MB pc100Dimm. Using a 3 drive RAID 0 hardware, vs 3 drives striped with LVM was an enourmous downgrade when benchmarking..but actual use, db inserts, file create, etc, ran >100% faster than with Hardware striping.

I did the same test with Reiser and Ext2/3, none fared as well, but they all did better with LVM and striping than without.

Quoted 37 lines
Quoted 3 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining the

quality

Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 9 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

Signature · 8 lines
Austin Gonyou
Systems Architect, CCNA
Coremetrics, Inc.
Phone: 512-698-7250
email: [email protected]

"It is the part of a good shepherd to shear his flock, not to skin it."
Latin Proverb
#21 lore raw
Quoted 40 lines · Austin Gonyou
Quoted 9 lines · Steve Wray

I'd recommend ext3 with data journalling for sensitive filesystems. Its slower than XFS but (seems to) scale better with striping and large files. XFS performance (seems to) fall off very rapidly as file size exceeds buffer size.

XFS only journals metadata. So on event of a crash, the filesystem structure will (likely) be sound, but theres no guarantee that the data blocks will be uncorrupted!!!

Quoted 5 lines

XFS is very nice tho, in that volumes and filesystems can be grown with no downtime at all (without even unmounting).

You might want to do some benchmarking before committing to a build (thats recently saved my butt actually).

I've only got a couple of things to say in the regard about XFS and performance Falling off.

XFS's performance doesn't fall off in the respects you've spoken about. It's a misconception when using LVM, or evms, etc along with XFS.

I ran into a similar issue, and was convinced it was XFS or LVM, etc. This is in the relation of using XFS and LVM, vs. XFS with no LVM, and then testing increasing filesizes with iozone.

What I found was that dbench, and iozone both did not perform very well with respect to software raid or striping, but that in fact the filesystems themselves performed about 100% faster than without LVM striping or LVM Concat.

So much faster that my database guys noticed the enourmous increase in speed on a test setup I created for them. One was using LVM, one was not, with XFS. The one which was NOT using LVM had a hardware stripe setup with write-back caching and adaptive read-ahead. The controller has 128MB pc100Dimm. Using a 3 drive RAID 0 hardware, vs 3 drives striped with LVM was an enourmous downgrade when benchmarking..but actual use, db inserts, file create, etc, ran >100% faster than with Hardware striping.

Quoted 3 lines

I did the same test with Reiser and Ext2/3, none fared as well, but they all did better with LVM and striping than without.

What kernel release and XFS version are you using? Anthony

Quoted 38 lines
Quoted 38 lines
Quoted 2 lines

-----Original Message-----
From: [email protected]

[mailto:[email protected]]On

Quoted 7 lines

Behalf Of Anthony W. Marino
Sent: Tuesday, 5 March 2002 10:39 a.m.
To: [email protected]
Subject: [linux-lvm] LVM System

Any thoughts or articles that would be usefull in determining the

quality

Quoted 9 lines

on the following combination would be greatly appreciated:

LVM 1.x
3Ware 7800 Raid Controller
Maxtor 40GB harddrives
XFS Journalling FS
SuSE 2.4.18+ Linux

Quoted 8 lines

Thank You,
Anthony

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#22 lore raw
In reply to Anthony W. Marino
Quoted 1 line · Anthony W. Marino

LVM 1.x

good idea

Quoted 1 line

3Ware 7800 Raid Controller

what do you want to go for? RAID5

Quoted 1 line

Maxtor 40GB harddrives

All IDE drives are about the same, but I'm running softraid5 with 4x60GB and 6x80GB, all Maxtor

Quoted 1 line

XFS Journalling FS

I'm using ReiserFS and ext3 (both in production), and am about to move from reiserFS to ext3. Except for special purpose (e.g. my MP3 base, and an thousends off small files (1-10k) box)

Quoted 1 line

SuSE 2.4.18+ Linux

IMHO, don't use SuSE, pick one of the real RPM based distributions. If you want everything provided go for RedHat or Mandrake, if you want a real server (and know how to compile your kernel) go for trustix.

Some more comments:
- don't use striping on RAID5. It is not going to work (well, if you have two RAID5 sets it will, but that's a waste of disk space)
- this ony appiles to my bad expirience with the 6xxx 3ware controllers:
test wether softRAID with the linux kernel is not faster than the hardware RAID
(well, you can not boot from softRAID as comfortable as you can from
hardware RAID)
- install the 3ware tool, the 3dm is a deamon that is configurateable from a web page (only runs on localhost, and has (for sure) some security holes). It can send mails on errors, too.
(AFAIR the install script is realy broken, don't run it, do it by
hand)
- with a 2.4.xx kernel you do not need a special driver
- I've a 6800 for sale. i switched my home filebase over to 3 promise controllers. They can do powersaving while I'm at work.

Cu,
Goetz.

#23 lore raw
Quoted 8 lines · Goetz Bock
Quoted 1 line · Anthony W. Marino

LVM 1.x

good idea

Quoted 1 line

3Ware 7800 Raid Controller

what do you want to go for? RAID5

RAID5

Quoted 11 lines
Quoted 1 line

Maxtor 40GB harddrives

All IDE drives are about the same, but I'm running softraid5 with 4x60GB and 6x80GB, all Maxtor

Quoted 1 line

XFS Journalling FS

I'm using ReiserFS and ext3 (both in production), and am about to move from reiserFS to ext3. Except for special purpose (e.g. my MP3 base, and an thousends off small files (1-10k) box)

Quoted 5 lines
Quoted 1 line

SuSE 2.4.18+ Linux

IMHO, don't use SuSE, pick one of the real RPM based distributions. If you want everything provided go for RedHat or Mandrake, if you want a real server (and know how to compile your kernel) go for trustix.

Why don't you think that SuSE isn't a real server OS???

Quoted 8 lines

Some more comments:
- don't use striping on RAID5. It is not going to work (well, if you have two RAID5 sets it will, but that's a waste of disk space)
- this ony appiles to my bad expirience with the 6xxx 3ware controllers:
test wether softRAID with the linux kernel is not faster than the hardware RAID
(well, you can not boot from softRAID as comfortable as you can from
hardware RAID)

I have 4x40GB Maxtor drives and haven't determined what my final configuration will be.

Quoted 9 lines

- install the 3ware tool, the 3dm is a deamon that is configurateable from a web page (only runs on localhost, and has (for sure) some security holes). It can send mails on errors, too. (AFAIR the install script is realy broken, don't run it, do it by hand)
- with a 2.4.xx kernel you do not need a special driver
- I've a 6800 for sale. i switched my home filebase over to 3 promise controllers. They can do powersaving while I'm at work.

Where do I find this 3ware tool?

Thanks,
Anthony

Quoted 8 lines

Cu,
Goetz.

_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

#24 lore raw
Quoted 1 line · Anthony W. Marino

Why don't you think that SuSE isn't a real server OS???

It was just MHO, use what ever you're comfortable with, but see my private email

Quoted 2 lines

I have 4x40GB Maxtor drives and haven't determined what my final configuration will be.

As others have said, do some test runs.

Quoted 1 line

Where do I find this 3ware tool?

It's in the tools package you can download from the 3ware side (last I checked the side required javascipt, so I can/will not look it up for you)

Cu,
Goetz.