btrfs: compression: migrate to folio interfaces
Subsystems (2)
cover btrfs: compression: migrate to folio interfaces
This is the conversion for btrfs compression paths to use folio interfaces.
For now, it's a pure intrefaces change, just with some variable names also changed from "page" to "folio".
There is no utilization of possible larger folio size yet, thus we're still using a lot of PAGE_SIZE/PAGE_SHIFT in the existing code.
But it's still a good first step towards large folio for btrfs data.
The first patch is in fact independent from the series, to slightly enhance the page cache missing error handling, but all later patches relies on it, to make later folio change a little smoother.
The third patch is also a good cleanup, as it allows we to pass a single page to inline creation path.
Although during tests, it turns out that under heavy race we can try to insert an empty inline extent, but since the old code can handle it well, I just added one comment for it.
The remaining but the last one are some preparation before the final conversion.
And the final patch is the core conversion, as we have several structure relying on page array, it's impossible to just convert one algorithm to folio meanwhile keep all the other using pages.
Qu Wenruo (6):
btrfs: compression: add error handling for missed page cache
btrfs: compression: convert page allocation to folio interfaces
btrfs: make insert_inline_extent() to accept one page directly
btrfs: migrate insert_inline_extent() to folio interfaces
btrfs: introduce btrfs_alloc_folio_array()
btrfs: compression: migrate compression/decompression paths to folios
fs/btrfs/compression.c | 119 +++++++++++++++++++------------- fs/btrfs/compression.h | 41 ++++++------ fs/btrfs/extent_io.c | 31 +++++++++ fs/btrfs/extent_io.h | 2 + fs/btrfs/inode.c | 149 +++++++++++++++++++++-------------------- fs/btrfs/lzo.c | 87 ++++++++++++------------ fs/btrfs/zlib.c | 110 ++++++++++++++++-------------- fs/btrfs/zstd.c | 79 ++++++++++++---------- 8 files changed, 349 insertions(+), 269 deletions(-)
Signature · 1 line
2.43.0
PATCH 1/6 btrfs: compression: add error handling for missed page cache
For all the supported compression algorithms, the compression path would always need to grab the page cache, then do the compression.
Normally we would get a page reference without any problem, since the write path should have already locked the pages in the write range. Just for the sake of error handling, we should handle the page cache miss case.
This patch adds a common wrapper, btrfs_compress_find_get_page(), which calls find_get_page(), and do the error handling along with an error message with an ASSERT().
Callers inside compression path would only need to call btrfs_compress_find_get_page(), and error out if it returned any error.
--- fs/btrfs/compression.c | 26 ++++++++++++++++++++++++++ fs/btrfs/compression.h | 3 +++ fs/btrfs/lzo.c | 5 +++-- fs/btrfs/zlib.c | 14 ++++++++++---- fs/btrfs/zstd.c | 9 +++++++-- 5 files changed, 49 insertions(+), 8 deletions(-)
fs/btrfs/compression.c +26 -0
@@ -974,6 +974,32 @@ static unsigned int btrfs_compress_set_level(int type, unsigned level)return level; } +/* A wrapper around find_get_page(), with extra error message. */ +int btrfs_compress_find_get_page (struct address_space *mapping,u64 start, +struct page **in_page_ret) +{ +struct page *in_page; + +/* +* The compressed write path should have the page locked already, +* thus we only need to grab one reference of the page cache. +*/ + in_page =find_get_page (mapping, start >>PAGE_SHIFT ); +if (unlikely (!in_page)) { +struct btrfs_inode *binode =BTRFS_I (mapping->host); +struct btrfs_fs_info *fs_info = binode->root->fs_info; + +btrfs_crit (fs_info, +"failed to get page cache, root %lld ino %llu file offset %llu" , + binode->root->root_key.objectid,btrfs_ino (binode), + start); +ASSERT (0 ); +return -ENOENT ; + } + *in_page_ret = in_page; +return 0 ; +} +/* * Given an address space and start and length, compress the bytes into @pages * that are allocated on demand.
fs/btrfs/compression.h +3 -0
@@ -143,6 +143,9 @@ bool btrfs_compress_is_valid_type(const char *str, size_t len);int btrfs_compress_heuristic (struct inode *inode,u64 start,u64 end); +int btrfs_compress_find_get_page (struct address_space *mapping,u64 start, +struct page **in_page_ret); +int zlib_compress_pages (struct list_head *ws,struct address_space *mapping,u64 start,struct page **pages,unsigned long *out_pages,unsigned long *total_in,unsigned long *total_out);
fs/btrfs/lzo.c +3 -2
@@ -244,8 +244,9 @@ int lzo_compress_pages(struct list_head *ws, struct address_space *mapping,/* Get the input page first */ if (!page_in) { - page_in =find_get_page (mapping, cur_in >>PAGE_SHIFT ); -ASSERT (page_in); + ret =btrfs_compress_find_get_page (mapping, cur_in, &page_in); +if (ret <0 ) + goto out; }/* Compress at most one sector of data each time */
fs/btrfs/zlib.c +10 -4
@@ -151,9 +151,12 @@ int zlib_compress_pages(struct list_head *ws, struct address_space *mapping,if (data_in) {kunmap_local (data_in);put_page (in_page); + data_in =NULL ; } - in_page =find_get_page (mapping, - start >>PAGE_SHIFT ); + ret =btrfs_compress_find_get_page (mapping, + start, &in_page); +if (ret <0 ) + goto out; data_in =kmap_local_page (in_page);copy_page (workspace->buf + i *PAGE_SIZE , data_in);
@@ -164,9 +167,12 @@ int zlib_compress_pages(struct list_head *ws, struct address_space *mapping,if (data_in) {kunmap_local (data_in);put_page (in_page); + data_in =NULL ; } - in_page =find_get_page (mapping, - start >>PAGE_SHIFT ); + ret =btrfs_compress_find_get_page (mapping, + start, &in_page); +if (ret <0 ) + goto out; data_in =kmap_local_page (in_page); start +=PAGE_SIZE ; workspace->strm.next_in = data_in;
fs/btrfs/zstd.c +7 -2
@@ -404,7 +404,9 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, }/* map in the first page of input data */ - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); + ret =btrfs_compress_find_get_page (mapping, start, &in_page); +if (ret <0 ) + goto out; workspace->in_buf.src =kmap_local_page (in_page); workspace->in_buf.pos =0 ; workspace->in_buf.size =min_t (size_t, len,PAGE_SIZE );
@@ -477,10 +479,13 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping,if (workspace->in_buf.pos == workspace->in_buf.size) { tot_in +=PAGE_SIZE ;kunmap_local (workspace->in_buf.src); + workspace->in_buf.src =NULL ;put_page (in_page); start +=PAGE_SIZE ; len -=PAGE_SIZE ; - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); + ret =btrfs_compress_find_get_page (mapping, start, &in_page); +if (ret <0 ) + goto out; workspace->in_buf.src =kmap_local_page (in_page); workspace->in_buf.pos =0 ; workspace->in_buf.size =min_t (size_t, len,PAGE_SIZE );
Signature · 1 line
2.43.0
PATCH 2/6 btrfs: compression: convert page allocation to folio interfaces
In reply to Qu WenruoCurrently we have two wrappers to allocate and free a page for compression usage:
- btrfs_alloc_compr_page()
- btrfs_free_compr_page()
The allocator would try to grab a page from the pool, and only allocate a new page if the pool is empty.
The reclaimer would check if the pool is full, and if not full it would put the page into the pool.
This patch would convert both helpers to use folio interfaces, and allowing further conversion of compression path to folios.
--- fs/btrfs/compression.c | 24 ++++++++++++------------ fs/btrfs/compression.h | 4 ++-- fs/btrfs/inode.c | 4 ++-- fs/btrfs/lzo.c | 4 ++-- fs/btrfs/zlib.c | 6 +++--- fs/btrfs/zstd.c | 6 +++--- 6 files changed, 24 insertions(+), 24 deletions(-)
fs/btrfs/compression.c +12 -12
@@ -161,7 +161,7 @@ static int compression_decompress(int type, struct list_head *ws,static void btrfs_free_compressed_pages (struct compressed_bio *cb) {for (unsigned int i =0 ; i < cb->nr_pages; i++) -btrfs_free_compr_page (cb->compressed_pages[i]); +btrfs_free_compr_folio (page_folio (cb->compressed_pages[i]));kfree (cb->compressed_pages); }
@@ -223,25 +223,25 @@ static unsigned long btrfs_compr_pool_scan(struct shrinker *sh, struct shrink_co/* * Common wrappers for page allocation from compression wrappers */ -struct page *btrfs_alloc_compr_page (void ) +struct folio *btrfs_alloc_compr_folio (void ) { -struct page *page =NULL ; +struct folio *folio =NULL ;spin_lock (&compr_pool.lock);if (compr_pool.count >0 ) { - page = list_first_entry(&compr_pool.list, struct page, lru); -list_del_init (&page->lru); + folio = list_first_entry(&compr_pool.list, struct folio, lru); +list_del_init (&folio->lru); compr_pool.count--; }spin_unlock (&compr_pool.lock); -if (page) -return page; +if (folio) +return folio; -return alloc_page (GFP_NOFS ); +return folio_alloc (GFP_NOFS ,0 ); } -void btrfs_free_compr_page (struct page *page) +void btrfs_free_compr_folio (struct folio *folio) {bool do_free = false;
@@ -249,7 +249,7 @@ void btrfs_free_compr_page(struct page *page)if (compr_pool.count > compr_pool.thresh) { do_free = true; }else { -list_add (&page->lru, &compr_pool.list); +list_add (&folio->lru, &compr_pool.list); compr_pool.count++; }spin_unlock (&compr_pool.lock);
@@ -257,8 +257,8 @@ void btrfs_free_compr_page(struct page *page)if (!do_free)return ; -ASSERT (page_ref_count (page) ==1 ); -put_page (page); +ASSERT (folio_ref_count (folio) ==1 ); +folio_put (folio); }static void end_bbio_comprssed_read (struct btrfs_bio *bbio)
fs/btrfs/compression.h +2 -2
@@ -98,8 +98,8 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio);unsigned int btrfs_compress_str2level (unsigned int type,const char *str); -struct page *btrfs_alloc_compr_page (void ); -void btrfs_free_compr_page (struct page *page); +struct folio *btrfs_alloc_compr_folio (void ); +void btrfs_free_compr_folio (struct folio *folio);enum btrfs_compression_type {BTRFS_COMPRESS_NONE =0 ,
fs/btrfs/inode.c +2 -2
@@ -1044,7 +1044,7 @@ static void compress_file_range(struct btrfs_work *work)if (pages) {for (i =0 ; i < nr_pages; i++) {WARN_ON (pages[i]->mapping); -btrfs_free_compr_page (pages[i]); +btrfs_free_compr_folio (page_folio (pages[i])); }kfree (pages); }
@@ -1059,7 +1059,7 @@ static void free_async_extent_pages(struct async_extent *async_extent)for (i =0 ; i < async_extent->nr_pages; i++) {WARN_ON (async_extent->pages[i]->mapping); -btrfs_free_compr_page (async_extent->pages[i]); +btrfs_free_compr_folio (page_folio (async_extent->pages[i])); }kfree (async_extent->pages); async_extent->nr_pages =0 ;
fs/btrfs/lzo.c +2 -2
@@ -152,7 +152,7 @@ static int copy_compressed_data_to_page(char *compressed_data, cur_page = out_pages[*cur_out /PAGE_SIZE ];/* Allocate a new page */ if (!cur_page) { - cur_page =btrfs_alloc_compr_page (); + cur_page =folio_page (btrfs_alloc_compr_folio (),0 );if (!cur_page)return -ENOMEM ; out_pages[*cur_out /PAGE_SIZE ] = cur_page;
@@ -178,7 +178,7 @@ static int copy_compressed_data_to_page(char *compressed_data, cur_page = out_pages[*cur_out /PAGE_SIZE ];/* Allocate a new page */ if (!cur_page) { - cur_page =btrfs_alloc_compr_page (); + cur_page =folio_page (btrfs_alloc_compr_folio (),0 );if (!cur_page)return -ENOMEM ; out_pages[*cur_out /PAGE_SIZE ] = cur_page;
fs/btrfs/zlib.c +3 -3
@@ -121,7 +121,7 @@ int zlib_compress_pages(struct list_head *ws, struct address_space *mapping, workspace->strm.total_in =0 ; workspace->strm.total_out =0 ; - out_page =btrfs_alloc_compr_page (); + out_page =folio_page (btrfs_alloc_compr_folio (),0 );if (out_page ==NULL ) { ret = -ENOMEM ; goto out;
@@ -206,7 +206,7 @@ int zlib_compress_pages(struct list_head *ws, struct address_space *mapping, ret = -E2BIG ; goto out; } - out_page =btrfs_alloc_compr_page (); + out_page =folio_page (btrfs_alloc_compr_folio (),0 );if (out_page ==NULL ) { ret = -ENOMEM ; goto out;
@@ -242,7 +242,7 @@ int zlib_compress_pages(struct list_head *ws, struct address_space *mapping, ret = -E2BIG ; goto out; } - out_page =btrfs_alloc_compr_page (); + out_page =folio_page (btrfs_alloc_compr_folio (),0 );if (out_page ==NULL ) { ret = -ENOMEM ; goto out;
fs/btrfs/zstd.c +3 -3
@@ -412,7 +412,7 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, workspace->in_buf.size =min_t (size_t, len,PAGE_SIZE );/* Allocate and map in the output buffer */ - out_page =btrfs_alloc_compr_page (); + out_page =folio_page (btrfs_alloc_compr_folio (),0 );if (out_page ==NULL ) { ret = -ENOMEM ; goto out;
@@ -457,7 +457,7 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, ret = -E2BIG ; goto out; } - out_page =btrfs_alloc_compr_page (); + out_page =folio_page (btrfs_alloc_compr_folio (),0 );if (out_page ==NULL ) { ret = -ENOMEM ; goto out;
@@ -517,7 +517,7 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, ret = -E2BIG ; goto out; } - out_page =btrfs_alloc_compr_page (); + out_page =folio_page (btrfs_alloc_compr_folio (),0 );if (out_page ==NULL ) { ret = -ENOMEM ; goto out;
Signature · 1 line
2.43.0
PATCH 3/6 btrfs: make insert_inline_extent() to accept one page directly
In reply to Qu WenruoSince our inline extent can not accept anything larger than a sector, there is really no need to pass all the compressed pages to insert_inline_extent().
And just in case, expand the ASSERT()s to make sure we only try inline with compressed size no larger than sectorsize.
--- fs/btrfs/inode.c | 47 +++++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 22 deletions(-)
fs/btrfs/inode.c +25 -22
@@ -512,12 +512,13 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans, struct btrfs_inode *inode, bool extent_inserted, size_t size, size_t compressed_size, int compress_type, - struct page **compressed_pages, + struct page *compressed_page, bool update_i_size) { struct btrfs_root *root = inode->root;struct extent_buffer *leaf;struct page *page =NULL ; +const u32 sectorsize = trans->fs_info->sectorsize;char *kaddr;unsigned long ptr;struct btrfs_file_extent_item *ei;
@@ -525,10 +526,23 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans,size_t cur_size = size;u64 i_size; -ASSERT ((compressed_size >0 && compressed_pages) || - (compressed_size ==0 && !compressed_pages)); +/* +* The decompressed size must still be no larger than a sector. +* Under heavy race, we can have size == 0 passed in, but that +* shouldn't be a big deal and we can continue the insertion. +*/ +ASSERT (size <= sectorsize); -if (compressed_size && compressed_pages) +/* +* The compressed size also need to be no larger than a sector. +* That's also why we only need one page as the parameter. +*/ +if (compressed_page) +ASSERT (compressed_size <= sectorsize); +else +ASSERT (compressed_size ==0 ); + +if (compressed_size && compressed_page) cur_size = compressed_size;if (!extent_inserted) {
@@ -556,21 +570,10 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans, ptr =btrfs_file_extent_inline_start (ei);if (compress_type !=BTRFS_COMPRESS_NONE ) { -struct page *cpage; -int i =0 ; -while (compressed_size >0 ) { - cpage = compressed_pages[i]; - cur_size = min_t(unsigned long, compressed_size, -PAGE_SIZE ); + kaddr =kmap_local_page (compressed_page); +write_extent_buffer (leaf, kaddr, ptr, compressed_size); +kunmap_local (kaddr); - kaddr =kmap_local_page (cpage); -write_extent_buffer (leaf, kaddr, ptr, cur_size); -kunmap_local (kaddr); - - i++; - ptr += cur_size; - compressed_size -= cur_size; - }btrfs_set_file_extent_compression (leaf, ei, compress_type); }else {
@@ -620,7 +623,7 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans, static noinline int cow_file_range_inline(struct btrfs_inode *inode, u64 size,size_t compressed_size,int compress_type, -struct page **compressed_pages, +struct page *compressed_page,bool update_i_size) {struct btrfs_drop_extents_args drop_args = {0 };
@@ -668,7 +671,7 @@ static noinline int cow_file_range_inline(struct btrfs_inode *inode, u64 size, ret =insert_inline_extent (trans, path, inode, drop_args.extent_inserted, size, compressed_size, compress_type, - compressed_pages, update_i_size); + compressed_page, update_i_size);if (ret && ret != -ENOSPC ) {btrfs_abort_transaction (trans, ret); goto out;
@@ -975,7 +978,7 @@ static void compress_file_range(struct btrfs_work *work) }else { ret =cow_file_range_inline (inode, actual_end, total_compressed, - compress_type, pages, + compress_type, pages[0 ], false); }if (ret <=0 ) {
@@ -10385,7 +10388,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,if (start ==0 && encoded->unencoded_len == encoded->len && encoded->unencoded_offset ==0 ) { ret =cow_file_range_inline (inode, encoded->len, orig_count, - compression, pages, true); + compression, pages[0 ], true);if (ret <=0 ) {if (ret ==0 ) ret = orig_count;
Signature · 1 line
2.43.0
PATCH 4/6 btrfs: migrate insert_inline_extent() to folio interfaces
In reply to Qu WenruoSince insert_inline_extent() now only accepts a single page, it's much easier to convert it to use folio interfaces.
--- fs/btrfs/inode.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
fs/btrfs/inode.c +10 -8
@@ -512,7 +512,7 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans, struct btrfs_inode *inode, bool extent_inserted, size_t size, size_t compressed_size, int compress_type, - struct page *compressed_page, + struct folio *compressed_folio, bool update_i_size) { struct btrfs_root *root = inode->root;
@@ -537,12 +537,12 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans,* The compressed size also need to be no larger than a sector. * That's also why we only need one page as the parameter. */ -if (compressed_page) +if (compressed_folio)ASSERT (compressed_size <= sectorsize);else ASSERT (compressed_size ==0 ); -if (compressed_size && compressed_page) +if (compressed_size && compressed_folio) cur_size = compressed_size;if (!extent_inserted) {
@@ -570,7 +570,7 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans, ptr =btrfs_file_extent_inline_start (ei);if (compress_type !=BTRFS_COMPRESS_NONE ) { - kaddr =kmap_local_page (compressed_page); + kaddr =kmap_local_folio (compressed_folio,0 );write_extent_buffer (leaf, kaddr, ptr, compressed_size);kunmap_local (kaddr);
@@ -623,7 +623,7 @@ static int insert_inline_extent(struct btrfs_trans_handle *trans, static noinline int cow_file_range_inline(struct btrfs_inode *inode, u64 size,size_t compressed_size,int compress_type, -struct page *compressed_page, +struct folio *compressed_folio,bool update_i_size) {struct btrfs_drop_extents_args drop_args = {0 };
@@ -671,7 +671,7 @@ static noinline int cow_file_range_inline(struct btrfs_inode *inode, u64 size, ret =insert_inline_extent (trans, path, inode, drop_args.extent_inserted, size, compressed_size, compress_type, - compressed_page, update_i_size); + compressed_folio, update_i_size);if (ret && ret != -ENOSPC ) {btrfs_abort_transaction (trans, ret); goto out;
@@ -978,7 +978,8 @@ static void compress_file_range(struct btrfs_work *work) }else { ret =cow_file_range_inline (inode, actual_end, total_compressed, - compress_type, pages[0 ], + compress_type, +page_folio (pages[0 ]), false); }if (ret <=0 ) {
@@ -10388,7 +10389,8 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,if (start ==0 && encoded->unencoded_len == encoded->len && encoded->unencoded_offset ==0 ) { ret =cow_file_range_inline (inode, encoded->len, orig_count, - compression, pages[0 ], true); + compression,page_folio (pages[0 ]), + true);if (ret <=0 ) {if (ret ==0 ) ret = orig_count;
Signature · 1 line
2.43.0
PATCH 5/6 btrfs: introduce btrfs_alloc_folio_array()
In reply to Qu WenruoThe new helper would do the same thing as btrfs_alloc_page_array(), but with folios.
One extra difference is, there is no extra helper for bulk allocation, thus it may not be as efficient as the page version.
--- fs/btrfs/extent_io.c | 31 +++++++++++++++++++++++++++++++ fs/btrfs/extent_io.h | 2 ++ 2 files changed, 33 insertions(+)
fs/btrfs/extent_io.c +31 -0
@@ -667,6 +667,37 @@ static void end_bbio_data_read(struct btrfs_bio *bbio)bio_put (bio); } +/* +* Populate every free slot in a provided array with folios. +* +* @nr_folios: number of foliosto allocate +* @folio_array: the array to fill with folios; any existing non-null entries in +* the array will be skipped +* @extra_gfp: the extra GFP flags for the allocation. +* +* Return: 0 if all folios were able to be allocated; +* -ENOMEM otherwise, the partially allocated folios would be freed and +* the array slots zeroed +*/ +int btrfs_alloc_folio_array (unsigned int nr_folios,struct folio **folio_array, +gfp_t extra_gfp) +{ +for (int i =0 ; i < nr_folios; i++) { +if (folio_array[i]) +continue ; + folio_array[i] =folio_alloc (GFP_NOFS | extra_gfp,0 ); +if (!folio_array[i]) + goto error; + } +return 0 ; +error: +for (int i =0 ; i < nr_folios; i++) { +if (folio_array[i]) +folio_put (folio_array[i]); + } +return -ENOMEM ; +} +/* * Populate every free slot in a provided array with pages. *
fs/btrfs/extent_io.h +2 -0
@@ -346,6 +346,8 @@ void btrfs_clear_buffer_dirty(struct btrfs_trans_handle *trans,int btrfs_alloc_page_array (unsigned int nr_pages,struct page **page_array,gfp_t extra_gfp); +int btrfs_alloc_folio_array (unsigned int nr_folios,struct folio **folio_array, +gfp_t extra_gfp);#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS bool find_lock_delalloc_range(struct inode *inode,
Signature · 1 line
2.43.0
PATCH 6/6 btrfs: compression: migrate compression/decompression paths to folios
In reply to Qu WenruoFor both compression and decompression paths, btrfs always requires a "struct page **pages" and "unsigned long nr_pages", this involves quite some part of the btrfs compression paths:
- All the compression entrances
- compressed_bio structure
This affects both compression and decompression.
- async_extent structure
Unfortunately with all those involved parts, there is no good way to split the conversion into smaller patches while still passes compiling.
So this patch would go a large conversion just in one go.
Please note this is just pure page->folio conversion, no change on the page sized folio requirement yet.
--- fs/btrfs/compression.c | 89 ++++++++++++++++----------------- fs/btrfs/compression.h | 38 +++++++-------- fs/btrfs/inode.c | 108 ++++++++++++++++++++--------------------- fs/btrfs/lzo.c | 84 ++++++++++++++++---------------- fs/btrfs/zlib.c | 104 +++++++++++++++++++-------------------- fs/btrfs/zstd.c | 74 ++++++++++++++-------------- 6 files changed, 249 insertions(+), 248 deletions(-)
fs/btrfs/compression.c +45 -44
@@ -90,20 +90,20 @@ bool btrfs_compress_is_valid_type(const char *str, size_t len) }static int compression_compress_pages (int type,struct list_head *ws, -struct address_space *mapping,u64 start,struct page **pages, -unsigned long *out_pages,unsigned long *total_in, -unsigned long *total_out) +struct address_space *mapping,u64 start,struct folio **folios, +unsigned long *out_folios,unsigned long *total_in, +unsigned long *total_out) {switch (type) {case BTRFS_COMPRESS_ZLIB : -return zlib_compress_pages (ws, mapping, start, pages, - out_pages, total_in, total_out); +return zlib_compress_folios (ws, mapping, start, folios, + out_folios, total_in, total_out);case BTRFS_COMPRESS_LZO : -return lzo_compress_pages (ws, mapping, start, pages, - out_pages, total_in, total_out); +return lzo_compress_folios (ws, mapping, start, folios, + out_folios, total_in, total_out);case BTRFS_COMPRESS_ZSTD : -return zstd_compress_pages (ws, mapping, start, pages, - out_pages, total_in, total_out); +return zstd_compress_folios (ws, mapping, start, folios, + out_folios, total_in, total_out);case BTRFS_COMPRESS_NONE :default :/*
@@ -115,7 +115,7 @@ static int compression_compress_pages(int type, struct list_head *ws,* Not a big deal, just need to inform caller that we * haven't allocated any pages yet. */ - *out_pages =0 ; + *out_folios =0 ;return -E2BIG ; } }
@@ -158,11 +158,11 @@ static int compression_decompress(int type, struct list_head *ws, } } -static void btrfs_free_compressed_pages (struct compressed_bio *cb) +static void btrfs_free_compressed_folios (struct compressed_bio *cb) { -for (unsigned int i =0 ; i < cb->nr_pages; i++) -btrfs_free_compr_folio (page_folio (cb->compressed_pages[i])); -kfree (cb->compressed_pages); +for (unsigned int i =0 ; i < cb->nr_folios; i++) +btrfs_free_compr_folio (cb->compressed_folios[i]); +kfree (cb->compressed_folios); }static int btrfs_decompress_bio (struct compressed_bio *cb);
@@ -269,7 +269,7 @@ static void end_bbio_comprssed_read(struct btrfs_bio *bbio)if (!status) status =errno_to_blk_status (btrfs_decompress_bio (cb)); -btrfs_free_compressed_pages (cb); +btrfs_free_compressed_folios (cb);btrfs_bio_end_io (cb->orig_bbio, status);bio_put (&bbio->bio); }
@@ -323,7 +323,7 @@ static void btrfs_finish_compressed_write_work(struct work_struct *work)end_compressed_writeback (cb);/* Note, our inode could be gone now */ -btrfs_free_compressed_pages (cb); +btrfs_free_compressed_folios (cb);bio_put (&cb->bbio.bio); }
@@ -342,17 +342,18 @@ static void end_bbio_comprssed_write(struct btrfs_bio *bbio)queue_work (fs_info->compressed_write_workers, &cb->write_end_work); } -static void btrfs_add_compressed_bio_pages (struct compressed_bio *cb) +static void btrfs_add_compressed_bio_folios (struct compressed_bio *cb) {struct bio *bio = &cb->bbio.bio;u32 offset =0 ;while (offset < cb->compressed_len) { +int ret;u32 len =min_t (u32, cb->compressed_len - offset,PAGE_SIZE );/* Maximum compressed extent is smaller than bio size limit. */ -__bio_add_page (bio, cb->compressed_pages[offset >>PAGE_SHIFT ], - len,0 ); + ret =bio_add_folio (bio, cb->compressed_folios[offset >>PAGE_SHIFT ], len,0 ); +ASSERT (ret); offset += len; } }
@@ -367,8 +368,8 @@ static void btrfs_add_compressed_bio_pages(struct compressed_bio *cb)* the end io hooks. */ void btrfs_submit_compressed_write (struct btrfs_ordered_extent *ordered, -struct page **compressed_pages, -unsigned int nr_pages, +struct folio **compressed_folios, +unsigned int nr_folios,blk_opf_t write_flags,bool writeback) {
@@ -384,14 +385,14 @@ void btrfs_submit_compressed_write(struct btrfs_ordered_extent *ordered, end_bbio_comprssed_write); cb->start = ordered->file_offset; cb->len = ordered->num_bytes; - cb->compressed_pages = compressed_pages; + cb->compressed_folios = compressed_folios; cb->compressed_len = ordered->disk_num_bytes; cb->writeback = writeback;INIT_WORK (&cb->write_end_work, btrfs_finish_compressed_write_work); - cb->nr_pages = nr_pages; + cb->nr_folios = nr_folios; cb->bbio.bio.bi_iter.bi_sector = ordered->disk_bytenr >>SECTOR_SHIFT ; cb->bbio.ordered = ordered; -btrfs_add_compressed_bio_pages (cb); +btrfs_add_compressed_bio_folios (cb);btrfs_submit_bio (&cb->bbio,0 ); }
@@ -599,14 +600,14 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)free_extent_map (em); - cb->nr_pages =DIV_ROUND_UP (compressed_len,PAGE_SIZE ); - cb->compressed_pages =kcalloc (cb->nr_pages,sizeof (struct page *),GFP_NOFS ); -if (!cb->compressed_pages) { + cb->nr_folios =DIV_ROUND_UP (compressed_len,PAGE_SIZE ); + cb->compressed_folios =kcalloc (cb->nr_folios,sizeof (struct page *),GFP_NOFS ); +if (!cb->compressed_folios) { ret =BLK_STS_RESOURCE ; goto out_free_bio; } - ret2 =btrfs_alloc_page_array (cb->nr_pages, cb->compressed_pages,0 ); + ret2 =btrfs_alloc_folio_array (cb->nr_folios, cb->compressed_folios,0 );if (ret2) { ret =BLK_STS_RESOURCE ; goto out_free_compressed_pages;
@@ -618,7 +619,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)/* include any pages we added in add_ra-bio_pages */ cb->len = bbio->bio.bi_iter.bi_size; cb->bbio.bio.bi_iter.bi_sector = bbio->bio.bi_iter.bi_sector; -btrfs_add_compressed_bio_pages (cb); +btrfs_add_compressed_bio_folios (cb);if (memstall)psi_memstall_leave (&pflags);
@@ -627,7 +628,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio)return ; out_free_compressed_pages: -kfree (cb->compressed_pages); +kfree (cb->compressed_folios); out_free_bio:bio_put (&cb->bbio.bio); out:
@@ -974,18 +975,18 @@ static unsigned int btrfs_compress_set_level(int type, unsigned level)return level; } -/* A wrapper around find_get_page(), with extra error message. */ -int btrfs_compress_find_get_page (struct address_space *mapping,u64 start, -struct page **in_page_ret) +/* A wrapper around filemap_get_folio(), with extra error message. */ +int btrfs_compress_filemap_get_folio (struct address_space *mapping,u64 start, +struct folio **in_folio_ret) { -struct page *in_page; +struct folio *in_folio;/* -* The compressed write path should have the page locked already, -* thus we only need to grab one reference of the page cache. +* The compressed write path should have the folio locked already, +* thus we only need to grab one reference. */ - in_page =find_get_page (mapping, start >>PAGE_SHIFT ); -if (unlikely (!in_page)) { + in_folio =filemap_get_folio (mapping, start >>PAGE_SHIFT ); +if (IS_ERR (in_folio)) {struct btrfs_inode *binode =BTRFS_I (mapping->host);struct btrfs_fs_info *fs_info = binode->root->fs_info;
@@ -996,7 +997,7 @@ int btrfs_compress_find_get_page(struct address_space *mapping, u64 start,ASSERT (0 );return -ENOENT ; } - *in_page_ret = in_page; + *in_folio_ret = in_folio;return 0 ; }
@@ -1020,9 +1021,9 @@ int btrfs_compress_find_get_page(struct address_space *mapping, u64 start,* @total_out is an in/out parameter, must be set to the input length and will * be also used to return the total number of compressed bytes */ -int btrfs_compress_pages (unsigned int type_level,struct address_space *mapping, -u64 start,struct page **pages, -unsigned long *out_pages, +int btrfs_compress_folios (unsigned int type_level,struct address_space *mapping, +u64 start,struct folio **folios, +unsigned long *out_folios,unsigned long *total_in,unsigned long *total_out) {
@@ -1033,8 +1034,8 @@ int btrfs_compress_pages(unsigned int type_level, struct address_space *mapping, level =btrfs_compress_set_level (type, level); workspace =get_workspace (type, level); - ret =compression_compress_pages (type, workspace, mapping, start, pages, - out_pages, total_in, total_out); + ret =compression_compress_pages (type, workspace, mapping, start, folios, + out_folios, total_in, total_out);put_workspace (type, workspace);return ret; }
fs/btrfs/compression.h +19 -19
@@ -35,11 +35,11 @@ static_assert((BTRFS_MAX_COMPRESSED % PAGE_SIZE) == 0);struct page ;struct compressed_bio { -/* Number of compressed pages in the array */ -unsigned int nr_pages; +/* Number of compressed folios in the array */ +unsigned int nr_folios; -/* the pages with the compressed data on them */ -struct page **compressed_pages; +/* the folios with the compressed data on them */ +struct folio **compressed_folios;/* starting offset in the inode for our pages */ u64 start;
@@ -79,9 +79,9 @@ static inline unsigned int btrfs_compress_level(unsigned int type_level)int __init btrfs_init_compress (void );void __cold btrfs_exit_compress (void ); -int btrfs_compress_pages (unsigned int type_level,struct address_space *mapping, -u64 start,struct page **pages, -unsigned long *out_pages, +int btrfs_compress_folios (unsigned int type_level,struct address_space *mapping, +u64 start,struct folio **folios, +unsigned long *out_folios,unsigned long *total_in,unsigned long *total_out); int btrfs_decompress(int type, const u8 *data_in, struct page *dest_page,
@@ -90,10 +90,10 @@ int btrfs_decompress_buf2page(const char *buf, u32 buf_len, struct compressed_bio *cb, u32 decompressed);void btrfs_submit_compressed_write (struct btrfs_ordered_extent *ordered, -struct page **compressed_pages, -unsigned int nr_pages, -blk_opf_t write_flags, -bool writeback); +struct folio **compressed_folios, +unsigned int nr_folios, +blk_opf_t write_flags, +bool writeback);void btrfs_submit_compressed_read (struct btrfs_bio *bbio);unsigned int btrfs_compress_str2level (unsigned int type,const char *str);
@@ -143,11 +143,11 @@ bool btrfs_compress_is_valid_type(const char *str, size_t len);int btrfs_compress_heuristic (struct inode *inode,u64 start,u64 end); -int btrfs_compress_find_get_page (struct address_space *mapping,u64 start, -struct page **in_page_ret); +int btrfs_compress_filemap_get_folio (struct address_space *mapping,u64 start, +struct folio **in_folio_ret); -int zlib_compress_pages (struct list_head *ws,struct address_space *mapping, -u64 start,struct page **pages,unsigned long *out_pages, +int zlib_compress_folios (struct list_head *ws,struct address_space *mapping, +u64 start,struct folio **folios,unsigned long *out_folios,unsigned long *total_in,unsigned long *total_out);int zlib_decompress_bio (struct list_head *ws,struct compressed_bio *cb); int zlib_decompress(struct list_head *ws, const u8 *data_in,
@@ -157,8 +157,8 @@ struct list_head *zlib_alloc_workspace(unsigned int level);void zlib_free_workspace (struct list_head *ws);struct list_head *zlib_get_workspace (unsigned int level); -int lzo_compress_pages (struct list_head *ws,struct address_space *mapping, -u64 start,struct page **pages,unsigned long *out_pages, +int lzo_compress_folios (struct list_head *ws,struct address_space *mapping, +u64 start,struct folio **folios,unsigned long *out_folios,unsigned long *total_in,unsigned long *total_out);int lzo_decompress_bio (struct list_head *ws,struct compressed_bio *cb); int lzo_decompress(struct list_head *ws, const u8 *data_in,
@@ -167,8 +167,8 @@ int lzo_decompress(struct list_head *ws, const u8 *data_in,struct list_head *lzo_alloc_workspace (unsigned int level);void lzo_free_workspace (struct list_head *ws); -int zstd_compress_pages (struct list_head *ws,struct address_space *mapping, -u64 start,struct page **pages,unsigned long *out_pages, +int zstd_compress_folios (struct list_head *ws,struct address_space *mapping, +u64 start,struct folio **folios,unsigned long *out_folios,unsigned long *total_in,unsigned long *total_out);int zstd_decompress_bio (struct list_head *ws,struct compressed_bio *cb); int zstd_decompress(struct list_head *ws, const u8 *data_in,
fs/btrfs/inode.c +53 -55
240 lines not on this page · raw message
fs/btrfs/lzo.c +42 -42
198 lines not on this page · raw message
fs/btrfs/zlib.c +52 -52
206 lines not on this page · raw message
fs/btrfs/zstd.c +38 -36
174 lines not on this page · raw message
Signature · 1 line
2.43.0
Quoted 29 lines · Qu Wenruo
This is the conversion for btrfs compression paths to use folio interfaces.
For now, it's a pure intrefaces change, just with some variable names also changed from "page" to "folio".
There is no utilization of possible larger folio size yet, thus we're still using a lot of PAGE_SIZE/PAGE_SHIFT in the existing code.
But it's still a good first step towards large folio for btrfs data.
The first patch is in fact independent from the series, to slightly enhance the page cache missing error handling, but all later patches relies on it, to make later folio change a little smoother.
The third patch is also a good cleanup, as it allows we to pass a single page to inline creation path.
Although during tests, it turns out that under heavy race we can try to insert an empty inline extent, but since the old code can handle it well, I just added one comment for it.The remaining but the last one are some preparation before the final conversion.
And the final patch is the core conversion, as we have several structure relying on page array, it's impossible to just convert one algorithm to folio meanwhile keep all the other using pages.
Quoted 7 lines
Qu Wenruo (6):
btrfs: compression: add error handling for missed page cache
btrfs: compression: convert page allocation to folio interfaces
btrfs: makeinsert_inline_extent()to accept one page directly
btrfs: migrateinsert_inline_extent()to folio interfaces
btrfs: introducebtrfs_alloc_folio_array()
btrfs: compression: migrate compression/decompression paths to folios
I added this patchset to my misc-next and it was in linux-next until now. The bug that was a blocker for folio conversions is now fixed, also thanks to you, so we can continue with the conversions. As this patchset is 2 months old I'm not sure if it would be helpful to start commenting and do the normal iteration round, I did a review and style fixup round and moved it to for-next. Please have a look and let me know if you find something wrong. I did mostly whitespace changes, though I did remove the ASSERT(0), if there's btrfs_crit message it's quite noticeable, and removed the local variable for fs_info in the first one.
The conversions are all direct and seem safe to me, we won't do multi-page folios yet, so the intemediate steps are the right way to go. Thanks.
Quoted 48 lines · David Sterba
Quoted 29 lines · Qu Wenruo
This is the conversion for btrfs compression paths to use folio interfaces.
For now, it's a pure intrefaces change, just with some variable names also changed from "page" to "folio".
There is no utilization of possible larger folio size yet, thus we're still using a lot of PAGE_SIZE/PAGE_SHIFT in the existing code.
But it's still a good first step towards large folio for btrfs data.
The first patch is in fact independent from the series, to slightly enhance the page cache missing error handling, but all later patches relies on it, to make later folio change a little smoother.
The third patch is also a good cleanup, as it allows we to pass a single page to inline creation path.
Although during tests, it turns out that under heavy race we can try to insert an empty inline extent, but since the old code can handle it well, I just added one comment for it.The remaining but the last one are some preparation before the final conversion.
And the final patch is the core conversion, as we have several structure relying on page array, it's impossible to just convert one algorithm to folio meanwhile keep all the other using pages.
Quoted 7 lines
Qu Wenruo (6):
btrfs: compression: add error handling for missed page cache
btrfs: compression: convert page allocation to folio interfaces
btrfs: makeinsert_inline_extent()to accept one page directly
btrfs: migrateinsert_inline_extent()to folio interfaces
btrfs: introducebtrfs_alloc_folio_array()
btrfs: compression: migrate compression/decompression paths to foliosI added this patchset to my misc-next and it was in linux-next until now. The bug that was a blocker for folio conversions is now fixed, also thanks to you, so we can continue with the conversions. As this patchset is 2 months old I'm not sure if it would be helpful to start commenting and do the normal iteration round, I did a review and style fixup round and moved it to for-next. Please have a look and let me know if you find something wrong. I did mostly whitespace changes, though I did remove the ASSERT(0), if there's
btrfs_critmessage it's quite noticeable, and removed the local variable forfs_infoin the first one.
Quoted 4 lines
The conversions are all direct and seem safe to me, we won't do multi-page folios yet, so the intemediate steps are the right way to go. Thanks.
Well, since I'm reading through all the commits in misc-next, I found some typos from myself:
- btrfs: prefer to allocate larger folio for metadata
* No longer no smaller, must be an exact fit. No larger no smaller, ...
* So here we go a different workaround, allocate a order 2 folio first
... allocate an order 2 ...
And these patches must be dropped:
- btrfs: defrag: prepare defrag for larger data folio size
- btrfs: introduce cached folio size
These two introduced and utilize fs_info->folio_shift, for filemap code, which is completely wrong. As filemap always expect an index with PAGE_SHIFT.
Otherwise the commits look fine to me.
Thanks,
Qu
Quoted 31 lines · Qu Wenruo
Quoted 23 lines
Quoted 5 lines
And the final patch is the core conversion, as we have several structure relying on page array, it's impossible to just convert one algorithm to folio meanwhile keep all the other using pages.
Quoted 7 lines
Qu Wenruo (6):
btrfs: compression: add error handling for missed page cache
btrfs: compression: convert page allocation to folio interfaces
btrfs: makeinsert_inline_extent()to accept one page directly
btrfs: migrateinsert_inline_extent()to folio interfaces
btrfs: introducebtrfs_alloc_folio_array()
btrfs: compression: migrate compression/decompression paths to foliosI added this patchset to my misc-next and it was in linux-next until now. The bug that was a blocker for folio conversions is now fixed, also thanks to you, so we can continue with the conversions. As this patchset is 2 months old I'm not sure if it would be helpful to start commenting and do the normal iteration round, I did a review and style fixup round and moved it to for-next. Please have a look and let me know if you find something wrong. I did mostly whitespace changes, though I did remove the ASSERT(0), if there's
btrfs_critmessage it's quite noticeable, and removed the local variable forfs_infoin the first one.Quoted 4 lines
The conversions are all direct and seem safe to me, we won't do multi-page folios yet, so the intemediate steps are the right way to go. Thanks.
Well, since I'm reading through all the commits in misc-next, I found some typos from myself:
Quoted 6 lines
- btrfs: prefer to allocate larger folio for metadata
* No longer no smaller, must be an exact fit. No larger no smaller, ...* So here we go a different workaround, allocate a order 2 folio first
... allocate an order 2 ...
I have this patch only for testing and did not do a typo-fixing pass as updates are still expected. Also I found some problems with this patch in page migration, I've sent the logs to slack.
Quoted 8 lines
And these patches must be dropped:
- btrfs: defrag: prepare defrag for larger data folio size
- btrfs: introduce cached folio sizeThese two introduced and utilize
fs_info->folio_shift, for filemap code, which is completely wrong. As filemap always expect an index withPAGE_SHIFT.
Ok, will drop them.