Skip to contentSkip to conversation

XFS over device mapper dm-thin device

DISCUSSION Todd Gill 4 participants 12 messages Feb 2, 2018 – Feb 9, 2018

lore
Share & tools
lore

Todd Gill

author
#0 lore raw

Hi,

I'd like to be able to take a snapshot of an XFS filesystem on top of dm-thin.

The goal for the snapshot is:

- fast to complete (couple seconds or better)
- independent of origin (UUID updated?)
- read/write

To satisfy the independent goal, I'm updating the UUID on the snapshot. Currently "xfs_admin -U xxx" takes in the ballpark of 14 seconds on my test system.

Is there a better approach to achieve the goals? Or can the UUID update be changed to be faster?

I realize the filesystem can be mounted with "nouuid" option, but would prefer to avoid the possible trouble of duplicate UUIDs.

This is a test script I've used to illustrate the time of "xfs_admin -U":

https://pastebin.com/PFA5YRuH

Thanks for any help,
Todd

#1 lore raw
Quoted 8 lines · Todd Gill

Hi,

I'd like to be able to take a snapshot of an XFS filesystem on top of dm-thin.

The goal for the snapshot is:

Quoted 7 lines

- fast to complete (couple seconds or better)
- independent of origin (UUID updated?)
- read/write

To satisfy the independent goal, I'm updating the UUID on the snapshot. Currently "xfs_admin -U xxx" takes in the ballpark of 14 seconds on my test system.

There's a very good chance that this is caused by dm-thinp doing COW for the storage underlying the superblocks modified by the UUID change.

Best to start by measuring how much IO is going on when you change the UUID. e.g. iostat -x -d -m 1. An strace showing how long the xfs_db read() and write() calls to the dm-thinp device would lso be instructive.

Quoted 1 line

Is there a better approach to achieve the goals?

No, you are doing the right thing.

Quoted 2 lines

Or can the UUID update
be changed to be faster?

Not sure - there's a fixed amount of modification we need to make to change the UUID so I'm not sure there is any optimisations we can make here.

If it's the speed of the underlying snapshot device doing COW after a snapshot that is causing this issue, then there's probably nothing we can do about it at the filesystem layer, anyway....

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
Quoted 13 lines · Dave Chinner
Quoted 12 lines · Todd Gill

Hi,

I'd like to be able to take a snapshot of an XFS filesystem on top of dm-thin.

The goal for the snapshot is:

- fast to complete (couple seconds or better)
- independent of origin (UUID updated?)
- read/write

Quoted 7 lines
Quoted 3 lines

To satisfy the independent goal, I'm updating the UUID on the snapshot. Currently "xfs_admin -U xxx" takes in the ballpark of 14 seconds on my test system.

There's a very good chance that this is caused by dm-thinp doing COW for the storage underlying the superblocks modified by the UUID change.

It's actually because we rewrite the whole log, because it has the user-facing superblock stamped in it.

I looked at this a bit, and didn't immediately see an easy way around xfs_db's need to rewrite the whole log.

uuid_f()

                /* clear the log (setting uuid) if it's not dirty */
                if (!sb_logzero(&uu))
                        return 0;

so I think that's taking the bulk of the time. I haven't looked closely enough at all the mount-time log parsing to see if we could maybe write less of it and still be valid.

Quoted 7 lines

Best to start by measuring how much IO is going on when you change the UUID. e.g. iostat -x -d -m 1. An strace showing how long the xfs_db read() and write() calls to the dm-thinp device would lso be instructive.

Quoted 1 line

Is there a better approach to achieve the goals?

Quoted 8 lines

No, you are doing the right thing.

Quoted 2 lines

Or can the UUID update
be changed to be faster?

Not sure - there's a fixed amount of modification we need to make to change the UUID so I'm not sure there is any optimisations we can make here.

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I'm not sure there's a nice backwards-compatible way to un-ring that bell, though.

-eric

#3 lore raw
Quoted 30 lines · Eric Sandeen
Quoted 13 lines · Dave Chinner
Quoted 12 lines · Todd Gill

Hi,

I'd like to be able to take a snapshot of an XFS filesystem on top of dm-thin.

The goal for the snapshot is:

- fast to complete (couple seconds or better)
- independent of origin (UUID updated?)
- read/write

Quoted 7 lines
Quoted 3 lines

To satisfy the independent goal, I'm updating the UUID on the snapshot. Currently "xfs_admin -U xxx" takes in the ballpark of 14 seconds on my test system.

There's a very good chance that this is caused by dm-thinp doing COW for the storage underlying the superblocks modified by the UUID change.

It's actually because we rewrite the whole log, because it has the user-facing superblock stamped in it.

I looked at this a bit, and didn't immediately see an easy way around xfs_db's need to rewrite the whole log.

Quoted 8 lines

uuid_f()

                /* clear the log (setting uuid) if it's not dirty */
                if (!sb_logzero(&uu))
                        return 0;

so I think that's taking the bulk of the time. I haven't looked closely enough at all the mount-time log parsing to see if we could maybe write less of it and still be valid.

Ah, I forgot we added that. Originally the uuid in the log was only used to identify external log devices (or Irix based filesystems).

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Or can the UUID update
be changed to be faster?

Not sure - there's a fixed amount of modification we need to make to change the UUID so I'm not sure there is any optimisations we can make here.

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#4 lore raw
Quoted 7 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Cheers,

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

-Eric

#5 lore raw
Quoted 9 lines · Eric Sandeen
Quoted 6 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Quoted 5 lines
Quoted 1 line

Cheers,

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#6 lore raw
Quoted 10 lines · Dave Chinner
Quoted 9 lines · Eric Sandeen
Quoted 6 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Cheers,

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Sure, if we're happy burning an incompat bit for this, sounds good.

-Eric

#7 lore raw
Quoted 15 lines · Eric Sandeen
Quoted 12 lines · Dave Chinner
Quoted 11 lines · Eric Sandeen
Quoted 7 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Cheers,

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Sure, if we're happy burning an incompat bit for this, sounds good.

Or a log incompat bit?

--D

Quoted 6 lines

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
Quoted 20 lines · Darrick J. Wong
Quoted 19 lines · Eric Sandeen
Quoted 16 lines · Dave Chinner
Quoted 14 lines · Eric Sandeen
Quoted 7 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Cheers,

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Sure, if we're happy burning an incompat bit for this, sounds good.

Or a log incompat bit?

That's what I suggested :P

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#9 lore raw
Quoted 21 lines · Dave Chinner
Quoted 20 lines · Darrick J. Wong
Quoted 19 lines · Eric Sandeen
Quoted 16 lines · Dave Chinner
Quoted 14 lines · Eric Sandeen
Quoted 7 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Cheers,

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Sure, if we're happy burning an incompat bit for this, sounds good.

Or a log incompat bit?

That's what I suggested :P

And that's what I /meant/, sorry for the lack of specificity ;)

-Eric

#10 lore raw
Quoted 27 lines · Eric Sandeen
Quoted 24 lines · Dave Chinner
Quoted 23 lines · Darrick J. Wong
Quoted 22 lines · Eric Sandeen
Quoted 18 lines · Dave Chinner
Quoted 9 lines · Eric Sandeen
Quoted 6 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Quoted 5 lines
Quoted 1 line

Cheers,

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Sure, if we're happy burning an incompat bit for this, sounds good.

Or a log incompat bit?

That's what I suggested :P

And that's what I /meant/, sorry for the lack of specificity ;)

So we are all happy to burn a log incompat bit for this?

/me wonders how many one liners we can string this out into :)

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#11 lore raw
Quoted 32 lines · Dave Chinner
Quoted 31 lines · Eric Sandeen
Quoted 28 lines · Dave Chinner
Quoted 26 lines · Darrick J. Wong
Quoted 15 lines · Eric Sandeen
Quoted 12 lines · Dave Chinner
Quoted 11 lines · Eric Sandeen
Quoted 7 lines · Dave Chinner
Quoted 2 lines

In retrospect I think we (I) should have left the "metadata" uuid as the one to match the log, not the user-facing one.

I suspect we can just change the log recovery code to accept either sb_uuid or sb_metauuid if the metauuid feature bit is set and change xfs_db to not rewrite the log when metauuid is used....

Cheers,

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

That was my first thought, but then newer userspace UUID changes
won't be mountable on older kernels. I don't think we can just
change it w/o some thought re: compatibility...

So maybe we should use a log incompat bit and provide an alternative command for the old way of changing the UUID in the log?

Sure, if we're happy burning an incompat bit for this, sounds good.

Or a log incompat bit?

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

That's what I suggested :P

And that's what I /meant/, sorry for the lack of specificity ;)

So we are all happy to burn a log incompat bit for this?

/me wonders how many one liners we can string this out into :)

I thought we were doing haiku to approve patch ideas now. :(

--D

Quoted 8 lines

-Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html