Skip to contentSkip to conversation

Btrfs stable updates for v3.16

DISCUSSION Chris Mason 4 participants 6 messages Sep 3, 2014 – Sep 6, 2014 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi everyone,

For 3.16, please pull these into stable, I've cherry picked and tested them here. For 3.15 and earlier there are a few conflicts, so I'll make a git tree with things to pull.

8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+
38c1c2e44bacb37efd68b90b3f70386a8ee370ee v3.11+
f6dc45c7a93a011dff6eb9b2ffda59c390c7705a v3.15+
9e0af23764344f7f1b68e4eefbe7dc865018b63d v3.15+

-chris

#1 lore raw
Quoted 10 lines · Chris Mason

Hi everyone,

For 3.16, please pull these into stable, I've cherry picked and tested them here. For 3.15 and earlier there are a few conflicts, so I'll make a git tree with things to pull.

8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+
38c1c2e44bacb37efd68b90b3f70386a8ee370ee v3.11+
f6dc45c7a93a011dff6eb9b2ffda59c390c7705a v3.15+
9e0af23764344f7f1b68e4eefbe7dc865018b63d v3.15+

Now applied to the trees I manage.

thanks,

greg k-h

#2 lore raw
In reply to Chris Mason
Quoted 7 lines · Chris Mason

Hi everyone,

For 3.16, please pull these into stable, I've cherry picked and tested them here. For 3.15 and earlier there are a few conflicts, so I'll make a git tree with things to pull.

8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+

This ("fix filemap_flush call in btrfs_file_release") is the only one that requires some work for 3.14.

There is one conflict in ordered.data.c - just a sligh work queue submission change - and the second in transaction.c where the patch does not delete enough from btrfs_flush_all_pending_stuffs(), since 3.14 still has the old qgroup calls in place. I removed it wholesale and that makes everything fit.

The followup ("fix filemap_flush call in btrfs_file_release") then also applies.

Should they also go into the next 3.14.x stable cycle? This rename deadlock sounds like a possible problem with rsync, which seems like a popular use case, and I guess nobody will complain about slightly better performance either.

Holger

#3 lore raw
Quoted 19 lines · Holger Hoffstätte
Quoted 7 lines · Chris Mason

Hi everyone,

For 3.16, please pull these into stable, I've cherry picked and tested them here. For 3.15 and earlier there are a few conflicts, so I'll make a git tree with things to pull.

8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+

This ("fix filemap_flush call in btrfs_file_release") is the only one that requires some work for 3.14.

There is one conflict in ordered.data.c - just a sligh work queue submission change - and the second in transaction.c where the patch does not delete enough from btrfs_flush_all_pending_stuffs(), since 3.14 still has the old qgroup calls in place. I removed it wholesale and that makes everything fit.

Quoted 7 lines

The followup ("fix filemap_flush call in btrfs_file_release") then also applies.

Should they also go into the next 3.14.x stable cycle? This rename deadlock sounds like a possible problem with rsync, which seems like a popular use case, and I guess nobody will complain about slightly better performance either.

Right, the btrfs_flush_all_pending_stuffs function can just be deleted.
But, Liu Bo's patch isn't required on 3.14 (since the regression he
fixed came with 3.15).

And these changes are big enough that I like to test them a little here before sending out. I did mark that patch as 3.15+, but really that deadlock has been there forever. We only started seeing it with 3.15+ because other waitqueue problems made it stand out.

-chris

#4 lore raw
In reply to Holger Hoffstätte

(removed stable@ for now)

Just to follow up on this -

Quoted 6 lines · Holger Hoffstätte

[on backporting both:
8d875f95da43c6a8f18f77869f2ef26e9594fecc and f6dc45c7a93a011dff6eb9b2ffda59c390c7705a to 3.14]

...and I guess nobody will complain about slightly better performance either.

So "slightly" is an understatement: rename/replace/truncate without full immediate flushing - as done by rsync's copy+rename - is now indeed *much* faster, to the point that backups with many small files finish in a fraction of the time. Wonderful!

Please do get this into the next 3.14 cycle, I'll gladly help and test.

Thanks!

Holger

#5 lore raw
In reply to Chris Mason
Quoted 7 lines · Chris Mason

Hi everyone,

For 3.16, please pull these into stable, I've cherry picked and tested them here. For 3.15 and earlier there are a few conflicts, so I'll make a git tree with things to pull.

8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+

Hi Chris,

Would this patch you submitted be likely to fix the chrome profile corruption needing restore from earlier snapshot problem we discussed a few weeks back?

If so, yeah and thanks! :)
(compiling 3.16.2 now)

btrfs: disable strict file flushes for renames and truncates

commit 8d875f95da43c6a8f18f77869f2ef26e9594fecc upstream.

Truncates and renames are often used to replace old versions of a file with new versions. Applications often expect this to be an atomic replacement, even if they haven't done anything to make sure the new version is fully on disk.

Btrfs has strict flushing in place to make sure that renaming over an old file with a new file will fully flush out the new file before allowing the transaction commit with the rename to complete.

This ordering means the commit code needs to be able to lock file pages, and there are a few paths in the filesystem where we will try to end a transaction with the page lock held. It's rare, but these things can deadlock.

This patch removes the ordered flushes and switches to a best effort filemap_flush like ext4 uses. It's not perfect, but it should fix the deadlocks.

Signature · 4 lines
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901