Skip to content

btrfs: used compressed_bio structure for read and write

Qu Wenruo linux-btrfs 5 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest9 patches 1 replyIn review
  2. v49 patches 4 repliesSuperseded
  3. v39 patches 6 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v29 patches3 repliesSuperseded
v19 patches2 repliesSuperseded

Changes in v5 from the cover letter

- Implement an internal compress_bio_last_folio()
Upstream is removing bio_last_bvec_all(), and the recent stable page bouncing for direct IO is going to make that function no longer reliable.

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9btrfs: introduce lzo_compress_bio() helperfs/btrfs/compression.h fs/btrfs/lzo.c+262 -00 replies
2/9btrfs: introduce zstd_compress_bio() helperfs/btrfs/compression.h fs/btrfs/zstd.c+186 -00 replies
3/9btrfs: introduce zlib_compress_bio() helperfs/btrfs/compression.h fs/btrfs/zlib.c+194 -00 replies
4/9btrfs: introduce btrfs_compress_bio() helperfs/btrfs/compression.c fs/btrfs/compression.h+81 -00 replies
5/9btrfs: switch to btrfs_compress_bio() interface for compressed writesfs/btrfs/compression.c fs/btrfs/inode.c+108 -940 replies
6/9btrfs: remove the old btrfs_compress_folios() infrastructuresfs/btrfs/compression.c fs/btrfs/compression.h fs/btrfs/inode.c +3 more+1 -6490 replies
7/9btrfs: get rid of compressed_folios[] usage for compressed readfs/btrfs/compression.c+22 -200 replies
8/9btrfs: get rid of compressed_folios[] usage for encoded writesfs/btrfs/compression.c fs/btrfs/compression.h fs/btrfs/inode.c+59 -630 replies
9/9btrfs: get rid of compressed_bio::compressed_folios[]fs/btrfs/compression.c fs/btrfs/compression.h+0 -70 replies

Reviews trailers per patch

All 9 patches have a Reviewed-by or Acked-by.

PatchBoris Burkov
1/9R-b
2/9R-b
3/9R-b
4/9R-b
5/9R-b
6/9R-b
7/9R-b
8/9R-b
9/9R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v5 thread

Cover letter [PATCH v5 0/9] btrfs: used compressed_bio structure for read and write

[CHANGELOG]
v5:
- Implement an internal compress_bio_last_folio()
Upstream is removing bio_last_bvec_all(), and the recent stable page bouncing for direct IO is going to make that function no longer reliable.

Thus we need to implement it locally, with some extra ASSERT()s to make sure the compressed bio have all folio sizes fixed to min_folio_size, so that we can use offset_in_folio(folio, bi_size) to calculate the last position.

- Fix several spelling errors
jut -> just in patch 5 and chaning -> changing in patch 7.

v4:
- Fix a missing kunmap_local() introduced in encoded write changes Which only affects 32bit systems.

- Fix an incorrect unlikely() usage introduced in encoded write changes Which will not cause any observable problem but completely flips the intention.

- Fix an allocation loop bug for bs > ps cases in encoded write changes The encoded write has migrated to use btrfs_alloc_compr_folio() which respects the minimal folio size.
This means the old PAGE_SIZE based loop condition must also be changed.

v3:
- Rebased to the latest for-next branch
There is a minor conflict with the format of parameter list of get_current_folio().

- Call btrfs_free_compr_folio() during cleanup_compressed_bio()

- Make it more consistent regarding using btrfs_alloc/free_compr_folio() The old compressed read path is using btrfs_alloc_folio_array() but freeing them using btrfs_free_compr_folio().

This is fine but never consistent, unify it to use btrfs_free_compr_folio() for all compressed_bio.

v2:
- Fix an error in error path of of compress_file_range()
If btrfs_compress_bio() returned an error, we should reset @cb to NULL before doing error handling, or cleanup_compressed_bio() can be called on an error pointer.

- Fix several bugs in zstd_compress_bio() which causes compression failure
There are several different bugs in the mostly copy-n-pasted code:

* Uncommon @start and @len usage
The old code allows @start and @len to be modified, which is against the more common practice nowadays.

* Fix a incorrect input buffer check
Which cause us to incorrectly end the compression early and fallback to uncompressed write.

I was never a huge fan of the current btrfs_compress_folios() interface:

- Complex and duplicated parameter list

* A folio array to hold all folios
Which means extra error handling.

* A @nr_folios pointer
That pointer is both input and output, representing the number of max folios, but also the number of compressed folios.

The number of input folios is not really necessary, it's always no larger than DIV_ROUND_UP(len, PAGE_SIZE) in the first place.

* A @total_in pointer
Again an pointer as both input and output, representing the filemap range length, and how many bytes are compressed in this run.

However if we failed to compress the full range, all supported algorithms will return an error, thus fallback to uncompressed path.

Thus there is no need to use it as an output pointer.

* A @total_compressed point
Again an pointer as both input and output, representing the max number of compressed size, and the final compressed size.

However we do not need it as an input at all, we always error out if the compressed size is larger than the original size.

- Extra error cleanup handling

We need to cleanup the compressed_folios[] array during error handling.

Replace the old btrfs_compress_folios() interface with btrfs_compress_bio(), which has the following benefits:

- Simplified parameter list

* inode
* start
* len
* compress_type
* compress_level
* write_flags

No parameter is sharing input and output members, and all are very straightforward (except the last write_flags, which is just an extra bio flag).

- Directly return a compressed_bio structure

With minor modifications, that pointer can be passed to btrfs_submit_bio().

The caller still needs to do proper round up and fill the proper disk_bytenr/num_bytes before submission.

And for error handling, simply call cleanup_compressed_bio() then everything is cleaned up properly (at least I hope so).

- No more extra folios array passing and error handling

Qu Wenruo (9):
btrfs: introduce lzo_compress_bio() helper
btrfs: introduce zstd_compress_bio() helper
btrfs: introduce zlib_compress_bio() helper
btrfs: introduce btrfs_compress_bio() helper
btrfs: switch to btrfs_compress_bio() interface for compressed writes
btrfs: remove the old btrfs_compress_folios() infrastructures
btrfs: get rid of compressed_folios[] usage for compressed read
btrfs: get rid of compressed_folios[] usage for encoded writes
btrfs: get rid of compressed_bio::compressed_folios[]

 fs/btrfs/compression.c | 208 ++++++++++++++++------------------
 fs/btrfs/compression.h |  40 ++++---
 fs/btrfs/inode.c       | 251 ++++++++++++++++++++++-------------------
 fs/btrfs/lzo.c         | 223 ++++++++++++++++++++++++------------
 fs/btrfs/zlib.c        |  64 ++++++-----
 fs/btrfs/zstd.c        |  98 ++++++++--------
 6 files changed, 482 insertions(+), 402 deletions(-)
Signature · 1 line
2.52.0