Skip to contentSkip to conversation

Estimate xfs_repair run time

DISCUSSION Gionatan Danti 4 participants 10 messages Aug 19, 2017 – Aug 20, 2017

lore
Share & tools
lore
#0 lore raw

Hi list,
using XFS on relatively big filesystems (> 8TB) I was wondering if it is possible to estimate how much time an emergency "xfs_repair" would take.

Some specific questions:
- will total time depend on how much the filesystem is filled (I think so...)?
- will total time depend on how data are layed on the physical disks (ie: fragmented vs sequential)?
- will total time scale down with increasing spindle count (ie: single disk vs 4-way RAID10)?

On a related question: I generally use LVM to segregate/isolate my virtual machine images. In this manner, even a completely blowed up filesystem on one LV can not affect other LV.

How do you feel using a single big LV + XFS + preallocated RAW disk images? Can a problem on the main XFS filesystem be contained on only some VM files, or it really risks to destroy the entire filesystem?

Thank you all.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
#1 lore raw

Sorry, reposting to CC me...

Il 19-08-2017 12:05 Gionatan Danti ha scritto:

Quoted 22 lines

Hi list,
using XFS on relatively big filesystems (> 8TB) I was wondering if it is possible to estimate how much time an emergency "xfs_repair" would take.

Some specific questions:
- will total time depend on how much the filesystem is filled (I think so...)?
- will total time depend on how data are layed on the physical disks (ie: fragmented vs sequential)?
- will total time scale down with increasing spindle count (ie: single disk vs 4-way RAID10)?

On a related question: I generally use LVM to segregate/isolate my virtual machine images. In this manner, even a completely blowed up filesystem on one LV can not affect other LV.

How do you feel using a single big LV + XFS + preallocated RAW disk images? Can a problem on the main XFS filesystem be contained on only some VM files, or it really risks to destroy the entire filesystem?

Thank you all.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
#2 lore raw
Quoted 23 lines · Gionatan Danti

Sorry, reposting to CC me...

Il 19-08-2017 12:05 Gionatan Danti ha scritto:

Quoted 20 lines

Hi list,
using XFS on relatively big filesystems (> 8TB) I was wondering if it is possible to estimate how much time an emergency "xfs_repair" would take.

Some specific questions:
- will total time depend on how much the filesystem is filled (I think so...)?
- will total time depend on how data are layed on the physical disks (ie: fragmented vs sequential)?
- will total time scale down with increasing spindle count (ie: single disk vs 4-way RAID10)?

On a related question: I generally use LVM to segregate/isolate my virtual machine images. In this manner, even a completely blowed up filesystem on one LV can not affect other LV.

How do you feel using a single big LV + XFS + preallocated RAW disk images? Can a problem on the main XFS filesystem be contained on only some VM files, or it really risks to destroy the entire filesystem?

Should work fine; be sure to set the extent size hint to reduce fragmentation.

--D

Quoted 13 lines
Quoted 2 lines

Thank you all.

Signature · 9 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#3 lore raw

Il 19-08-2017 17:48 Darrick J. Wong ha scritto:

Quoted 1 line

Should work fine; be sure to set the extent size hint to reduce fragmentation.

Hi, do you mean tuning the "allocsize" mount option? If I remember correctly, this parameter should be auto-tuned by XFS during normal operation, right?

Any thoughts on the xfs_repair matter?
Thanks.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
#4 lore raw
Quoted 7 lines · Gionatan Danti

Il 19-08-2017 17:48 Darrick J. Wong ha scritto:

Quoted 2 lines

Should work fine; be sure to set the extent size hint to reduce fragmentation.

Hi, do you mean tuning the "allocsize" mount option? If I remember correctly, this parameter should be auto-tuned by XFS during normal operation, right?

I meant the per-inode extent size hint. See 'extsize' in the xfs_io manpage.

Quoted 1 line

Any thoughts on the xfs_repair matter?

{yes, yes, depends on which spindle(s) metadata end up on and how many files cross AGs} in that order. :)

--D

Quoted 12 lines

Thanks.

Signature · 9 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#5 lore raw

Il 19-08-2017 18:20 Darrick J. Wong ha scritto:

Quoted 1 line

I meant the per-inode extent size hint. See 'extsize' in the xfs_io manpage.

I did not know about per-file extent hint. Thank you very much for pointing me that!

Quoted 2 lines

{yes, yes, depends on which spindle(s) metadata end up on and how many files cross AGs} in that order. :)

Mmm ok, as expected :)
Do you have any direct experience of xfs_repair run times on big and full XFS filesystems?

Thanks.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
#6 lore raw
Quoted 6 lines · Gionatan Danti
Quoted 2 lines

{yes, yes, depends on which spindle(s) metadata end up on and how many files cross AGs} in that order. :)

Mmm ok, as expected :)
Do you have any direct experience of xfs_repair run times on big and full XFS filesystems?

Yes. And there's no hard and fast number we can give you because runtime will depends on the amount and density of the metadata, the speed of the underlying storage, how much RAM you have and how badly corrupted the filesystem is.

IOWs, runtime can vary from a few minutes to a few days as you vary those parameters, and there's really no way to tell in advance where in that massive variance any specific production filesystem will land.

Doing a test run (xfs_repair -n) to get a ballpark figure during a planned downtime period is the only way you'll get any idea of the *best case* repair duration on a *clean* filesystem. But the moment repair finds corruptions, all guesses you can make about runtime go out the window...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#8 lore raw
In reply to Gionatan Danti

As for this question:

Quoted 1 line · Gionatan Danti

Hi list,

...

Quoted 1 line

How do you feel using a single big LV + XFS + preallocated RAW disk images? Can a problem on the main XFS filesystem be contained on only some VM files, or it really risks to destroy the entire filesystem?

It's pretty rare to have large-scale, filesystem-wide corruption. The only times I've ever seen filesystems really get wiped out by corruption was when the underlying storage went haywire; a scrambled raid, etc. The sort of thing no filesystem could cope with ...

-Eric

#7 lore raw
In reply to Dave Chinner

Hi Dave,

Il 20-08-2017 04:04 Dave Chinner ha scritto:

Quoted 11 lines

Yes. And there's no hard and fast number we can give you because runtime will depends on the amount and density of the metadata, the speed of the underlying storage, how much RAM you have and how badly corrupted the filesystem is.

IOWs, runtime can vary from a few minutes to a few days as you vary those parameters, and there's really no way to tell in advance where in that massive variance any specific production filesystem will land.

Quoted 5 lines

Doing a test run (xfs_repair -n) to get a ballpark figure during a planned downtime period is the only way you'll get any idea of the *best case* repair duration on a *clean* filesystem. But the moment repair finds corruptions, all guesses you can make about runtime go out the window...

ok, so I at least need to try & see how well xfs_repair does on a clean filesystem.

Thanks.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
#9 lore raw
In reply to Eric Sandeen

Hi Eric,

Il 20-08-2017 04:10 Eric Sandeen ha scritto:

Quoted 4 lines

It's pretty rare to have large-scale, filesystem-wide corruption. The only times I've ever seen filesystems really get wiped out by corruption was when the underlying storage went haywire; a scrambled raid, etc. The sort of thing no filesystem could cope with ...

this is reassuring! ;)

Thank you for reporting.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8