btrfs: per-fs compression workspace manager
Subsystems (2)
cover btrfs: per-fs compression workspace manager
Currently btrfs utilizes the following components at a per-module basis:
- Folios pool
A per-module pool for compressed data.
All folios in the pool are page sized.
- Workspace
A workspace is a compression algorithm specific structure, providing things like extra memory buffer and compression level handling.
- Workspace manager
The workspace manager is managing the above workspaces for each algorithm.
All the folio pool/workspaces are using the fixed PAGE_SIZE buffer size, this is fine for now as even for block size (bs) < page size (ps) cases, a larger buffer size won't cause huge problems except wasting memories.
However if we're going to support bs > ps, this fixed PAGE_SIZE buffer and per-module shared folios pool/workspaces will not work at all.
To address this problem, this series will move the workspace and workspace manager into a per-fs basis, so that different fses (with different block size) can have their own workspaces.
This brings a small memory usage reduce for bs < ps cases. Now zlib/lzo/zstd will only allocate buffer using block size.
This is especially useful for lzo compression algorithm, as lzo is an one-short compression algorithm, it doesn't support multi-shot (aka, swapping input/output buffer halfway) compress/decompress.
Thus btrfs goes per-block compression for LZO, and compressed result will never go larger than a block (or btrfs will just give up). In that case, a 64K page sized buffer will waste 7/8th of the buffer.
This is part 1 of the preparation for btrfs bs > ps support.
Qu Wenruo (7):
btrfs: add an fs_info parameter for compression workspace manager
btrfs: add workspace manager initialization for zstd
btrfs: add generic workspace manager initialization
btrfs: migrate to use per-fs workspace manager
btrfs: cleanup the per-module workspace managers
btrfs: rename btrfs_compress_op to btrfs_compress_levels
btrfs: reduce workspace buffer space to block size
fs/btrfs/compression.c | 186 ++++++++++++++++++++++++----------------- fs/btrfs/compression.h | 49 +++++------ fs/btrfs/disk-io.c | 4 + fs/btrfs/fs.h | 13 +++ fs/btrfs/lzo.c | 25 +++--- fs/btrfs/zlib.c | 16 ++-- fs/btrfs/zstd.c | 143 ++++++++++++++++--------------- 7 files changed, 248 insertions(+), 188 deletions(-)
Signature · 1 line
2.50.1
PATCH 1/7 btrfs: add an fs_info parameter for compression workspace manager
[BACKGROUND]
Currently btrfs shares workspaces and their managers for all filesystems, this is mostly fine as all those workspaces are using page size based buffers, and btrfs only support block size (bs) <= page size (ps).
This means even if bs < ps, we at most waste some buffer space in the workspace, but everything will still work fine.
The problem here is that is limiting our support for bs > ps cases.
As now a workspace now may need larger buffer to handle bs > ps cases, but since the pool has no way to distinguish different workspaces, a regular workspace (which is still using buffer size based on ps) can be passed to a btrfs whose bs > ps.
In that case the buffer is not large enough, and will cause various problems.
[ENHANCEMENT]
To prepare for the per-fs workspace migration, add an fs_info parameter to all workspace related functions.
For now this new fs_info parameter is not yet utilized.
--- fs/btrfs/compression.c | 73 ++++++++++++++++++++++-------------------- fs/btrfs/compression.h | 23 +++++++------ fs/btrfs/lzo.c | 2 +- fs/btrfs/zlib.c | 6 ++-- fs/btrfs/zstd.c | 12 +++---- 5 files changed, 62 insertions(+), 54 deletions(-)
fs/btrfs/compression.c +38 -35
@@ -702,7 +702,7 @@ static void free_heuristic_ws(struct list_head *ws)kfree (workspace); } -static struct list_head *alloc_heuristic_ws (void ) +static struct list_head *alloc_heuristic_ws (struct btrfs_fs_info *fs_info) {struct heuristic_ws *ws;
@@ -741,13 +741,13 @@ static const struct btrfs_compress_op * const btrfs_compress_op[] = { &btrfs_zstd_compress, }; -static struct list_head *alloc_workspace (int type,int level) +static struct list_head *alloc_workspace (struct btrfs_fs_info *fs_info,int type,int level) {switch (type) { -case BTRFS_COMPRESS_NONE :return alloc_heuristic_ws (); -case BTRFS_COMPRESS_ZLIB :return zlib_alloc_workspace (level); -case BTRFS_COMPRESS_LZO :return lzo_alloc_workspace (); -case BTRFS_COMPRESS_ZSTD :return zstd_alloc_workspace (level); +case BTRFS_COMPRESS_NONE :return alloc_heuristic_ws (fs_info); +case BTRFS_COMPRESS_ZLIB :return zlib_alloc_workspace (fs_info, level); +case BTRFS_COMPRESS_LZO :return lzo_alloc_workspace (fs_info); +case BTRFS_COMPRESS_ZSTD :return zstd_alloc_workspace (fs_info, level);default :/* * This can't happen, the type is validated several times
@@ -773,7 +773,7 @@ static void free_workspace(int type, struct list_head *ws) } } -static void btrfs_init_workspace_manager (int type) +static void btrfs_init_workspace_manager (struct btrfs_fs_info *fs_info,int type) {struct workspace_manager *wsm;struct list_head *workspace;
@@ -788,9 +788,9 @@ static void btrfs_init_workspace_manager(int type)* Preallocate one workspace for each compression type so we can * guarantee forward progress in the worst case */ - workspace =alloc_workspace (type,0 ); + workspace =alloc_workspace (fs_info, type,0 );if (IS_ERR (workspace)) { -btrfs_warn (NULL , +btrfs_warn (fs_info,"cannot preallocate compression workspace, will try later" ); }else {atomic_set (&wsm->total_ws,1 );
@@ -819,7 +819,7 @@ static void btrfs_cleanup_workspace_manager(int type)* Preallocation makes a forward progress guarantees and we do not return * errors. */ -struct list_head *btrfs_get_workspace (int type,int level) +struct list_head *btrfs_get_workspace (struct btrfs_fs_info *fs_info,int type,int level) {struct workspace_manager *wsm;struct list_head *workspace;
@@ -867,7 +867,7 @@ struct list_head *btrfs_get_workspace(int type, int level)* context of btrfs_compress_bio/btrfs_compress_pages */ nofs_flag =memalloc_nofs_save (); - workspace =alloc_workspace (type, level); + workspace =alloc_workspace (fs_info, type, level);memalloc_nofs_restore (nofs_flag);if (IS_ERR (workspace)) {
@@ -890,7 +890,7 @@ struct list_head *btrfs_get_workspace(int type, int level) /* no burst */ 1);if (__ratelimit (&_rs)) -btrfs_warn (NULL , +btrfs_warn (fs_info,"no compression workspaces, low memory, retrying" ); } goto again;
@@ -898,13 +898,13 @@ struct list_head *btrfs_get_workspace(int type, int level)return workspace; } -static struct list_head *get_workspace (int type,int level) +static struct list_head *get_workspace (struct btrfs_fs_info *fs_info,int type,int level) {switch (type) { -case BTRFS_COMPRESS_NONE :return btrfs_get_workspace (type, level); -case BTRFS_COMPRESS_ZLIB :return zlib_get_workspace (level); -case BTRFS_COMPRESS_LZO :return btrfs_get_workspace (type, level); -case BTRFS_COMPRESS_ZSTD :return zstd_get_workspace (level); +case BTRFS_COMPRESS_NONE :return btrfs_get_workspace (fs_info, type, level); +case BTRFS_COMPRESS_ZLIB :return zlib_get_workspace (fs_info, level); +case BTRFS_COMPRESS_LZO :return btrfs_get_workspace (fs_info, type, level); +case BTRFS_COMPRESS_ZSTD :return zstd_get_workspace (fs_info, level);default :/* * This can't happen, the type is validated several times
@@ -918,7 +918,7 @@ static struct list_head *get_workspace(int type, int level)* put a workspace struct back on the list or free it if we have enough * idle ones sitting around */ -void btrfs_put_workspace (int type,struct list_head *ws) +void btrfs_put_workspace (struct btrfs_fs_info *fs_info,int type,struct list_head *ws) {struct workspace_manager *wsm;struct list_head *idle_ws;
@@ -949,13 +949,13 @@ void btrfs_put_workspace(int type, struct list_head *ws)cond_wake_up (ws_wait); } -static void put_workspace (int type,struct list_head *ws) +static void put_workspace (struct btrfs_fs_info *fs_info,int type,struct list_head *ws) {switch (type) { -case BTRFS_COMPRESS_NONE :return btrfs_put_workspace (type, ws); -case BTRFS_COMPRESS_ZLIB :return btrfs_put_workspace (type, ws); -case BTRFS_COMPRESS_LZO :return btrfs_put_workspace (type, ws); -case BTRFS_COMPRESS_ZSTD :return zstd_put_workspace (ws); +case BTRFS_COMPRESS_NONE :return btrfs_put_workspace (fs_info, type, ws); +case BTRFS_COMPRESS_ZLIB :return btrfs_put_workspace (fs_info, type, ws); +case BTRFS_COMPRESS_LZO :return btrfs_put_workspace (fs_info, type, ws); +case BTRFS_COMPRESS_ZSTD :return zstd_put_workspace (fs_info, ws);default :/* * This can't happen, the type is validated several times
@@ -1038,29 +1038,31 @@ int btrfs_compress_folios(unsigned int type, int level, struct btrfs_inode *inod u64 start, struct folio **folios, unsigned long *out_folios, unsigned long *total_in, unsigned long *total_out) { + struct btrfs_fs_info *fs_info = inode->root->fs_info;const unsigned long orig_len = *total_out;struct list_head *workspace;int ret; level =btrfs_compress_set_level (type, level); - workspace =get_workspace (type, level); + workspace =get_workspace (fs_info, type, level); ret =compression_compress_pages (type, workspace, inode, start, folios, out_folios, total_in, total_out);/* The total read-in bytes should be no larger than the input. */ ASSERT (*total_in <= orig_len); -put_workspace (type, workspace); +put_workspace (fs_info, type, workspace);return ret; }static int btrfs_decompress_bio (struct compressed_bio *cb) { +struct btrfs_fs_info *fs_info =cb_to_fs_info (cb);struct list_head *workspace;int ret;int type = cb->compress_type; - workspace =get_workspace (type,0 ); + workspace =get_workspace (fs_info, type,0 ); ret =compression_decompress_bio (workspace, cb); -put_workspace (type, workspace); +put_workspace (fs_info, type, workspace);if (!ret)zero_fill_bio (&cb->orig_bbio->bio);
@@ -1087,10 +1089,10 @@ int btrfs_decompress(int type, const u8 *data_in, struct folio *dest_folio,*/ ASSERT (dest_pgoff + destlen <=PAGE_SIZE && destlen <= sectorsize); - workspace =get_workspace (type,0 ); + workspace =get_workspace (fs_info, type,0 ); ret =compression_decompress (type, workspace, data_in, dest_folio, dest_pgoff, srclen, destlen); -put_workspace (type, workspace); +put_workspace (fs_info, type, workspace);return ret; }
@@ -1106,10 +1108,10 @@ int __init btrfs_init_compress(void)if (!compr_pool.shrinker)return -ENOMEM ; -btrfs_init_workspace_manager (BTRFS_COMPRESS_NONE ); -btrfs_init_workspace_manager (BTRFS_COMPRESS_ZLIB ); -btrfs_init_workspace_manager (BTRFS_COMPRESS_LZO ); -zstd_init_workspace_manager (); +btrfs_init_workspace_manager (NULL ,BTRFS_COMPRESS_NONE ); +btrfs_init_workspace_manager (NULL ,BTRFS_COMPRESS_ZLIB ); +btrfs_init_workspace_manager (NULL ,BTRFS_COMPRESS_LZO ); +zstd_init_workspace_manager (NULL );spin_lock_init (&compr_pool.lock);INIT_LIST_HEAD (&compr_pool.list);
@@ -1543,7 +1545,8 @@ static void heuristic_collect_sample(struct inode *inode, u64 start, u64 end,*/ int btrfs_compress_heuristic (struct btrfs_inode *inode,u64 start,u64 end) { -struct list_head *ws_list =get_workspace (0 ,0 ); +struct btrfs_fs_info *fs_info = inode->root->fs_info; +struct list_head *ws_list =get_workspace (fs_info,0 ,0 );struct heuristic_ws *ws;u32 i;u8 byte;
@@ -1612,7 +1615,7 @@ int btrfs_compress_heuristic(struct btrfs_inode *inode, u64 start, u64 end) } out: -put_workspace (0 , ws_list); +put_workspace (fs_info,0 , ws_list);return ret; }
fs/btrfs/compression.h +14 -9
@@ -75,6 +75,11 @@ struct compressed_bio {struct btrfs_bio bbio; }; +static inline struct btrfs_fs_info *cb_to_fs_info (const struct compressed_bio *cb) +{ +return cb->bbio.fs_info; +} +/* @range_end must be exclusive. */ static inline u32 btrfs_calc_input_length (struct folio *folio,u64 range_end,u64 cur) {
@@ -128,8 +133,8 @@ struct workspace_manager {wait_queue_head_t ws_wait; }; -struct list_head *btrfs_get_workspace (int type,int level); -void btrfs_put_workspace (int type,struct list_head *ws); +struct list_head *btrfs_get_workspace (struct btrfs_fs_info *fs_info,int type,int level); +void btrfs_put_workspace (struct btrfs_fs_info *fs_info,int type,struct list_head *ws);struct btrfs_compress_op {struct workspace_manager *workspace_manager;
@@ -162,9 +167,9 @@ int zlib_decompress_bio(struct list_head *ws, struct compressed_bio *cb);int zlib_decompress (struct list_head *ws,const u8 *data_in,struct folio *dest_folio,unsigned long dest_pgoff,size_t srclen,size_t destlen); -struct list_head *zlib_alloc_workspace (unsigned int level); +struct list_head *zlib_alloc_workspace (struct btrfs_fs_info *fs_info,unsigned int level);void zlib_free_workspace (struct list_head *ws); -struct list_head *zlib_get_workspace (unsigned int level); +struct list_head *zlib_get_workspace (struct btrfs_fs_info *fs_info,unsigned int level); int lzo_compress_folios(struct list_head *ws, struct btrfs_inode *inode, u64 start, struct folio **folios, unsigned long *out_folios,
@@ -173,7 +178,7 @@ int lzo_decompress_bio(struct list_head *ws, struct compressed_bio *cb);int lzo_decompress (struct list_head *ws,const u8 *data_in,struct folio *dest_folio,unsigned long dest_pgoff,size_t srclen,size_t destlen); -struct list_head *lzo_alloc_workspace (void ); +struct list_head *lzo_alloc_workspace (struct btrfs_fs_info *fs_info);void lzo_free_workspace (struct list_head *ws); int zstd_compress_folios(struct list_head *ws, struct btrfs_inode *inode,
@@ -183,11 +188,11 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb);int zstd_decompress (struct list_head *ws,const u8 *data_in,struct folio *dest_folio,unsigned long dest_pgoff,size_t srclen,size_t destlen); -void zstd_init_workspace_manager (void ); +void zstd_init_workspace_manager (struct btrfs_fs_info *fs_info);void zstd_cleanup_workspace_manager (void ); -struct list_head *zstd_alloc_workspace (int level); +struct list_head *zstd_alloc_workspace (struct btrfs_fs_info *fs_info,int level);void zstd_free_workspace (struct list_head *ws); -struct list_head *zstd_get_workspace (int level); -void zstd_put_workspace (struct list_head *ws); +struct list_head *zstd_get_workspace (struct btrfs_fs_info *fs_info,int level); +void zstd_put_workspace (struct btrfs_fs_info *fs_info,struct list_head *ws);#endif
fs/btrfs/lzo.c +1 -1
@@ -80,7 +80,7 @@ void lzo_free_workspace(struct list_head *ws)kfree (workspace); } -struct list_head *lzo_alloc_workspace (void ) +struct list_head *lzo_alloc_workspace (struct btrfs_fs_info *fs_info) {struct workspace *workspace;
fs/btrfs/zlib.c +3 -3
@@ -36,9 +36,9 @@ struct workspace {static struct workspace_manager wsm; -struct list_head *zlib_get_workspace (unsigned int level) +struct list_head *zlib_get_workspace (struct btrfs_fs_info *fs_info,unsigned int level) { -struct list_head *ws =btrfs_get_workspace (BTRFS_COMPRESS_ZLIB , level); +struct list_head *ws =btrfs_get_workspace (fs_info,BTRFS_COMPRESS_ZLIB , level); struct workspace *workspace = list_entry(ws, struct workspace, list); workspace->level = level;
@@ -55,7 +55,7 @@ void zlib_free_workspace(struct list_head *ws)kfree (workspace); } -struct list_head *zlib_alloc_workspace (unsigned int level) +struct list_head *zlib_alloc_workspace (struct btrfs_fs_info *fs_info,unsigned int level) {struct workspace *workspace;int workspacesize;
fs/btrfs/zstd.c +6 -6
@@ -182,7 +182,7 @@ static void zstd_calc_ws_mem_sizes(void) } } -void zstd_init_workspace_manager (void ) +void zstd_init_workspace_manager (struct btrfs_fs_info *fs_info) {struct list_head *ws;int i;
@@ -198,7 +198,7 @@ void zstd_init_workspace_manager(void)for (i =0 ; i <ZSTD_BTRFS_MAX_LEVEL ; i++)INIT_LIST_HEAD (&wsm.idle_ws[i]); - ws =zstd_alloc_workspace (ZSTD_BTRFS_MAX_LEVEL ); + ws =zstd_alloc_workspace (fs_info,ZSTD_BTRFS_MAX_LEVEL );if (IS_ERR (ws)) {btrfs_warn (NULL ,"cannot preallocate zstd compression workspace" ); }else {
@@ -276,7 +276,7 @@ static struct list_head *zstd_find_workspace(int level)* attempt to allocate a new workspace. If we fail to allocate one due to * memory pressure, go to sleep waiting for the max level workspace to free up. */ -struct list_head *zstd_get_workspace (int level) +struct list_head *zstd_get_workspace (struct btrfs_fs_info *fs_info,int level) {struct list_head *ws;unsigned int nofs_flag;
@@ -291,7 +291,7 @@ struct list_head *zstd_get_workspace(int level)return ws; nofs_flag =memalloc_nofs_save (); - ws =zstd_alloc_workspace (level); + ws =zstd_alloc_workspace (fs_info, level);memalloc_nofs_restore (nofs_flag);if (IS_ERR (ws)) {
@@ -318,7 +318,7 @@ struct list_head *zstd_get_workspace(int level)* isn't set, it is also set here. Only the max level workspace tries and wakes * up waiting workspaces. */ -void zstd_put_workspace (struct list_head *ws) +void zstd_put_workspace (struct btrfs_fs_info *fs_info,struct list_head *ws) {struct workspace *workspace =list_to_workspace (ws);
@@ -357,7 +357,7 @@ void zstd_free_workspace(struct list_head *ws)kfree (workspace); } -struct list_head *zstd_alloc_workspace (int level) +struct list_head *zstd_alloc_workspace (struct btrfs_fs_info *fs_info,int level) {struct workspace *workspace;
Signature · 1 line
2.50.1
PATCH 3/7 btrfs: add generic workspace manager initialization
In reply to Qu WenruoThis involves:
- Add generic_(alloc|free)_workspace_manager helpers.
These are the helper to alloc/free workspace_manager structure, which will allocate a workspace_manager structure, initialize it, and pre-allocate one workspace for it.
- Call generic_alloc_workspace_manager() inside
btrfs_alloc_compress_wsm()
For none, zlib and lzo compression algorithms.
- Call generic_alloc_workspace_manager() inside
btrfs_free_compress_wsm()
For none, zlib and lzo compression algorithms.
For now the generic per-fs workspace managers won't really have any effect, and all compression is still going through the global workspace manager.
--- fs/btrfs/compression.c | 66 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+)
fs/btrfs/compression.c +66 -0
@@ -773,6 +773,40 @@ static void free_workspace(int type, struct list_head *ws) } } +static int generic_alloc_workspace_manager (struct btrfs_fs_info *fs_info, +enum btrfs_compression_type type) +{ +struct workspace_manager *gwsm; +struct list_head *workspace; + +ASSERT (fs_info->compr_wsm[type] ==NULL ); + gwsm =kzalloc (sizeof (*gwsm),GFP_KERNEL ); +if (!gwsm) +return -ENOMEM ; + +INIT_LIST_HEAD (&gwsm->idle_ws); +spin_lock_init (&gwsm->ws_lock); +atomic_set (&gwsm->total_ws,0 ); +init_waitqueue_head (&gwsm->ws_wait); + fs_info->compr_wsm[type] = gwsm; + +/* +* Preallocate one workspace for each compression type so we can +* guarantee forward progress in the worst case +*/ + workspace =alloc_workspace (fs_info, type,0 ); +if (IS_ERR (workspace)) { +btrfs_warn (fs_info, +"cannot preallocate compression workspace for %s, will try later" , +btrfs_compress_type2str (type)); + }else { +atomic_set (&gwsm->total_ws,1 ); + gwsm->free_ws =1 ; +list_add (workspace, &gwsm->idle_ws); + } +return 0 ; +} +static void btrfs_init_workspace_manager (struct btrfs_fs_info *fs_info,int type) {struct workspace_manager *wsm;
@@ -799,6 +833,26 @@ static void btrfs_init_workspace_manager(struct btrfs_fs_info *fs_info, int type } } +static void generic_free_workspace_manager (struct btrfs_fs_info *fs_info, +enum btrfs_compression_type type) +{ +struct list_head *ws; +struct workspace_manager *gwsm = fs_info->compr_wsm[type]; + +/* ZSTD uses its own workspace manager, should enter here. */ +ASSERT (type !=BTRFS_COMPRESS_ZSTD && type <BTRFS_NR_COMPRESS_TYPES ); +if (!gwsm) +return ; + fs_info->compr_wsm[type] =NULL ; +while (!list_empty (&gwsm->idle_ws)) { + ws = gwsm->idle_ws.next; +list_del (ws); +free_workspace (type, ws); +atomic_dec (&gwsm->total_ws); + } +kfree (gwsm); +} +static void btrfs_cleanup_workspace_manager (int type) {struct workspace_manager *wsman;
@@ -1101,6 +1155,15 @@ int btrfs_alloc_compress_wsm(struct btrfs_fs_info *fs_info) {int ret; + ret =generic_alloc_workspace_manager (fs_info,BTRFS_COMPRESS_NONE ); +if (ret <0 ) + goto error; + ret =generic_alloc_workspace_manager (fs_info,BTRFS_COMPRESS_ZLIB ); +if (ret <0 ) + goto error; + ret =generic_alloc_workspace_manager (fs_info,BTRFS_COMPRESS_LZO ); +if (ret <0 ) + goto error; ret =zstd_alloc_workspace_manager (fs_info);if (ret <0 ) goto error;
@@ -1112,6 +1175,9 @@ int btrfs_alloc_compress_wsm(struct btrfs_fs_info *fs_info)void btrfs_free_compress_wsm (struct btrfs_fs_info *fs_info) { +generic_free_workspace_manager (fs_info,BTRFS_COMPRESS_NONE ); +generic_free_workspace_manager (fs_info,BTRFS_COMPRESS_ZLIB ); +generic_free_workspace_manager (fs_info,BTRFS_COMPRESS_LZO );zstd_free_workspace_manager (fs_info); }
Signature · 1 line
2.50.1
PATCH 2/7 btrfs: add workspace manager initialization for zstd
In reply to Qu WenruoThis involves:
- Add zstd_alloc_workspace_manager() and zstd_free_workspace_manager() Those two functions will accept an fs_info pointer, and alloc/free fs_info->compr_wsm[BTRFS_COMPRESS_ZSTD] pointer.
- Add btrfs_alloc_compress_wsm() and btrfs_free_compress_wsm() Those are helpers allocating the workspace managers for all algorithms.
For now only zstd is supported, and the timing is a little unusual, the btrfs_alloc_compress_wsm() should only be called after the sectorsize being initialized.
Meanwhile btrfs_free_fs_info_compress() is called in btrfs_free_fs_info().
- Move the definition of btrfs_compression_type to "fs.h" The reason is that "compression.h" has already included "fs.h", thus we can not just include "compression.h" to get the definition of BTRFS_NR_COMPRESS_TYPES to define fs_info::compr_wsm[].
For now the per-fs zstd workspace manager won't really have any effect, and all compression is still going through the global workspace manager.
--- fs/btrfs/compression.c | 18 ++++++++++++++ fs/btrfs/compression.h | 15 ++++-------- fs/btrfs/disk-io.c | 4 ++++ fs/btrfs/fs.h | 13 +++++++++++ fs/btrfs/zstd.c | 53 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 93 insertions(+), 10 deletions(-)
fs/btrfs/compression.c +18 -0
@@ -1097,6 +1097,24 @@ int btrfs_decompress(int type, const u8 *data_in, struct folio *dest_folio,return ret; } +int btrfs_alloc_compress_wsm (struct btrfs_fs_info *fs_info) +{ +int ret; + + ret =zstd_alloc_workspace_manager (fs_info); +if (ret <0 ) + goto error; +return 0 ; +error: +btrfs_free_compress_wsm (fs_info); +return ret; +} + +void btrfs_free_compress_wsm (struct btrfs_fs_info *fs_info) +{ +zstd_free_workspace_manager (fs_info); +} +int __init btrfs_init_compress (void ) { if (bioset_init(&btrfs_compressed_bioset, BIO_POOL_SIZE,
fs/btrfs/compression.h +5 -10
@@ -89,6 +89,9 @@ static inline u32 btrfs_calc_input_length(struct folio *folio, u64 range_end, u6return min (range_end,folio_end (folio)) - cur; } +int btrfs_alloc_compress_wsm (struct btrfs_fs_info *fs_info); +void btrfs_free_compress_wsm (struct btrfs_fs_info *fs_info); +int __init btrfs_init_compress (void );void __cold btrfs_exit_compress (void );
@@ -112,16 +115,6 @@ int btrfs_compress_str2level(unsigned int type, const char *str);struct folio *btrfs_alloc_compr_folio (void );void btrfs_free_compr_folio (struct folio *folio); -enum btrfs_compression_type { -BTRFS_COMPRESS_NONE =0 , -BTRFS_COMPRESS_ZLIB =1 , -BTRFS_COMPRESS_LZO =2 , -BTRFS_COMPRESS_ZSTD =3 , -BTRFS_NR_COMPRESS_TYPES =4 , - -BTRFS_DEFRAG_DONT_COMPRESS , -}; -struct workspace_manager {struct list_head idle_ws;spinlock_t ws_lock;
@@ -188,6 +181,8 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb);int zstd_decompress (struct list_head *ws,const u8 *data_in,struct folio *dest_folio,unsigned long dest_pgoff,size_t srclen,size_t destlen); +int zstd_alloc_workspace_manager (struct btrfs_fs_info *fs_info); +void zstd_free_workspace_manager (struct btrfs_fs_info *fs_info);void zstd_init_workspace_manager (struct btrfs_fs_info *fs_info);void zstd_cleanup_workspace_manager (void );struct list_head *zstd_alloc_workspace (struct btrfs_fs_info *fs_info,int level);
fs/btrfs/disk-io.c +4 -0
@@ -1248,6 +1248,7 @@ void btrfs_free_fs_info(struct btrfs_fs_info *fs_info)if (fs_info->fs_devices)btrfs_close_devices (fs_info->fs_devices); +btrfs_free_compress_wsm (fs_info);percpu_counter_destroy (&fs_info->stats_read_blocks);percpu_counter_destroy (&fs_info->dirty_metadata_bytes);percpu_counter_destroy (&fs_info->delalloc_bytes);
@@ -3411,6 +3412,9 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device*/ fs_info->max_inline =min_t (u64, fs_info->max_inline, fs_info->sectorsize); + ret =btrfs_alloc_compress_wsm (fs_info); +if (ret) + goto fail_sb_buffer; ret =btrfs_init_workqueues (fs_info);if (ret) goto fail_sb_buffer;
fs/btrfs/fs.h +13 -0
@@ -305,6 +305,16 @@ enum {#define BTRFS_WARNING_COMMIT_INTERVAL (300)#define BTRFS_DEFAULT_MAX_INLINE (2048) +enum btrfs_compression_type { +BTRFS_COMPRESS_NONE =0 , +BTRFS_COMPRESS_ZLIB =1 , +BTRFS_COMPRESS_LZO =2 , +BTRFS_COMPRESS_ZSTD =3 , +BTRFS_NR_COMPRESS_TYPES =4 , + +BTRFS_DEFRAG_DONT_COMPRESS , +}; +struct btrfs_dev_replace {/* See #define above */ u64 replace_state;
@@ -507,6 +517,9 @@ struct btrfs_fs_info {u64 last_trans_log_full_commit;unsigned long long mount_opt; +/* Compress related structures. */ +void *compr_wsm[BTRFS_NR_COMPRESS_TYPES ]; +int compress_type;int compress_level;u32 commit_interval;
fs/btrfs/zstd.c +53 -0
65 lines not on this page · raw message
Signature · 1 line
2.50.1
PATCH 4/7 btrfs: migrate to use per-fs workspace manager
In reply to Qu WenruoThere are several interfaces involved for each algorithm:
- alloc workspace
All algorithms allocate a workspace without the need for workspace manager.
So no change needs to be done.
- get workspace
This involves checking the workspace manager to find a free one, and if not, allocate a new one.
For none and lzo, they share the same generic btrfs_get_workspace() helper, only needs to update that function to use the per-fs manager.
For zlib it uses a wrapper around btrfs_get_workspace(), so no special work needed.
For zstd, update zstd_find_workspace() and zstd_get_workspace() to utilize the per-fs manager.
- put workspace
For none/zlib/lzo they share the same btrfs_put_workspace(), update that function to use the per-fs manager.
For zstd, it's zstd_put_workspace(), the same update.
- zstd specific timer
This is the timer to reclaim workspace, change it to grab the per-fs workspace manager instead.
Now all workspace are managed by the per-fs manager.
--- fs/btrfs/compression.c | 18 +++++------ fs/btrfs/zstd.c | 71 +++++++++++++++++++++++------------------- 2 files changed, 48 insertions(+), 41 deletions(-)
fs/btrfs/compression.c +9 -9
@@ -875,7 +875,7 @@ static void btrfs_cleanup_workspace_manager(int type)*/ struct list_head *btrfs_get_workspace (struct btrfs_fs_info *fs_info,int type,int level) { -struct workspace_manager *wsm; +struct workspace_manager *wsm = fs_info->compr_wsm[type];struct list_head *workspace;int cpus =num_online_cpus ();unsigned nofs_flag;
@@ -885,7 +885,7 @@ struct list_head *btrfs_get_workspace(struct btrfs_fs_info *fs_info, int type, iwait_queue_head_t *ws_wait;int *free_ws; - wsm = btrfs_compress_op[type]->workspace_manager; +ASSERT (wsm); idle_ws = &wsm->idle_ws; ws_lock = &wsm->ws_lock; total_ws = &wsm->total_ws;
@@ -974,19 +974,19 @@ static struct list_head *get_workspace(struct btrfs_fs_info *fs_info, int type,*/ void btrfs_put_workspace (struct btrfs_fs_info *fs_info,int type,struct list_head *ws) { -struct workspace_manager *wsm; +struct workspace_manager *gwsm = fs_info->compr_wsm[type];struct list_head *idle_ws;spinlock_t *ws_lock;atomic_t *total_ws;wait_queue_head_t *ws_wait;int *free_ws; - wsm = btrfs_compress_op[type]->workspace_manager; - idle_ws = &wsm->idle_ws; - ws_lock = &wsm->ws_lock; - total_ws = &wsm->total_ws; - ws_wait = &wsm->ws_wait; - free_ws = &wsm->free_ws; +ASSERT (gwsm); + idle_ws = &gwsm->idle_ws; + ws_lock = &gwsm->ws_lock; + total_ws = &gwsm->total_ws; + ws_wait = &gwsm->ws_wait; + free_ws = &gwsm->free_ws;spin_lock (ws_lock);if (*free_ws <=num_online_cpus ()) {
fs/btrfs/zstd.c +39 -32
158 lines not on this page · raw message
Signature · 1 line
2.50.1
PATCH 6/7 btrfs: rename btrfs_compress_op to btrfs_compress_levels
In reply to Qu WenruoSince all workspace managers are per-fs, there is no need nor no way to store them inside btrfs_compress_op::wsm anymore.
With that said, we can do the following modifications:
- Remove zstd_workspace_mananger::ops
Zstd always grab the global btrfs_compress_op[].
- Remove btrfs_compress_op::wsm member
- Rename btrfs_compress_op to btrfs_compress_levels
This should make it more clear that btrfs_*_compress structures are only to indicate the levels of each compress algorithm.
--- fs/btrfs/compression.c | 14 +++++++------- fs/btrfs/compression.h | 11 +++++------ fs/btrfs/lzo.c | 2 +- fs/btrfs/zlib.c | 2 +- fs/btrfs/zstd.c | 4 +--- 5 files changed, 15 insertions(+), 18 deletions(-)
fs/btrfs/compression.c +7 -7
@@ -727,9 +727,9 @@ static struct list_head *alloc_heuristic_ws(struct btrfs_fs_info *fs_info)return ERR_PTR (-ENOMEM ); } -const struct btrfs_compress_op btrfs_heuristic_compress = {0 }; +const struct btrfs_compress_levels btrfs_heuristic_compress = {0 }; -static const struct btrfs_compress_op *const btrfs_compress_op[] = { +static const struct btrfs_compress_levels *const btrfs_compress_levels[] = {/* The heuristic is represented as compression type 0 */ &btrfs_heuristic_compress, &btrfs_zlib_compress,
@@ -981,12 +981,12 @@ static void put_workspace(struct btrfs_fs_info *fs_info, int type, struct list_h*/ static int btrfs_compress_set_level (unsigned int type,int level) { -const struct btrfs_compress_op *ops = btrfs_compress_op[type]; +const struct btrfs_compress_levels *levels = btrfs_compress_levels[type];if (level ==0 ) - level = ops->default_level; + level = levels->default_level;else - level =clamp (level, ops->min_level, ops->max_level); + level =clamp (level, levels->min_level, levels->max_level);return level; }
@@ -996,9 +996,9 @@ static int btrfs_compress_set_level(unsigned int type, int level)*/ bool btrfs_compress_level_valid (unsigned int type,int level) { -const struct btrfs_compress_op *ops = btrfs_compress_op[type]; +const struct btrfs_compress_levels *levels = btrfs_compress_levels[type]; -return ops->min_level <= level && level <= ops->max_level; +return levels->min_level <= level && level <= levels->max_level; }/* Wrapper around find_get_page(), with extra error message. */
fs/btrfs/compression.h +5 -6
@@ -129,8 +129,7 @@ struct workspace_manager {struct list_head *btrfs_get_workspace (struct btrfs_fs_info *fs_info,int type,int level);void btrfs_put_workspace (struct btrfs_fs_info *fs_info,int type,struct list_head *ws); -struct btrfs_compress_op { -struct workspace_manager *workspace_manager; +struct btrfs_compress_levels {/* Maximum level supported by the compression algorithm */ int min_level;int max_level;
@@ -140,10 +139,10 @@ struct btrfs_compress_op {/* The heuristic workspaces are managed via the 0th workspace manager */ #define BTRFS_NR_WORKSPACE_MANAGERS BTRFS_NR_COMPRESS_TYPES -extern const struct btrfs_compress_op btrfs_heuristic_compress; -extern const struct btrfs_compress_op btrfs_zlib_compress; -extern const struct btrfs_compress_op btrfs_lzo_compress; -extern const struct btrfs_compress_op btrfs_zstd_compress; +extern const struct btrfs_compress_levels btrfs_heuristic_compress; +extern const struct btrfs_compress_levels btrfs_zlib_compress; +extern const struct btrfs_compress_levels btrfs_lzo_compress; +extern const struct btrfs_compress_levels btrfs_zstd_compress;const char *btrfs_compress_type2str (enum btrfs_compression_type type);bool btrfs_compress_is_valid_type (const char *str,size_t len);
fs/btrfs/lzo.c +1 -1
@@ -486,7 +486,7 @@ int lzo_decompress(struct list_head *ws, const u8 *data_in,return ret; } -const struct btrfs_compress_op btrfs_lzo_compress = { +const struct btrfs_compress_levels btrfs_lzo_compress = { .max_level =1 , .default_level =1 , };
fs/btrfs/zlib.c +1 -1
@@ -480,7 +480,7 @@ int zlib_decompress(struct list_head *ws, const u8 *data_in,return ret; } -const struct btrfs_compress_op btrfs_zlib_compress = { +const struct btrfs_compress_levels btrfs_zlib_compress = { .min_level =1 , .max_level =9 , .default_level =BTRFS_ZLIB_DEFAULT_LEVEL ,
fs/btrfs/zstd.c +1 -3
@@ -77,7 +77,6 @@ struct workspace {*/ struct zstd_workspace_manager { -const struct btrfs_compress_op *ops;spinlock_t lock;struct list_head lru_list;struct list_head idle_ws[ZSTD_BTRFS_MAX_LEVEL ];
@@ -190,7 +189,6 @@ int zstd_alloc_workspace_manager(struct btrfs_fs_info *fs_info)if (!zwsm)return -ENOMEM ;zstd_calc_ws_mem_sizes (); - zwsm->ops = &btrfs_zstd_compress;spin_lock_init (&zwsm->lock);init_waitqueue_head (&zwsm->wait);timer_setup (&zwsm->timer, zstd_reclaim_timer_fn,0 );
@@ -727,7 +725,7 @@ int zstd_decompress(struct list_head *ws, const u8 *data_in,return ret; } -const struct btrfs_compress_op btrfs_zstd_compress = { +const struct btrfs_compress_levels btrfs_zstd_compress = { .min_level =ZSTD_BTRFS_MIN_LEVEL , .max_level =ZSTD_BTRFS_MAX_LEVEL , .default_level =ZSTD_BTRFS_DEFAULT_LEVEL ,
Signature · 1 line
2.50.1
PATCH 5/7 btrfs: cleanup the per-module workspace managers
In reply to Qu WenruoSince all workspaces are handled by the per-fs workspace managers, we can safely remove the old per-module managers.
--- fs/btrfs/compression.c | 55 +----------------------------------------- fs/btrfs/compression.h | 2 -- fs/btrfs/lzo.c | 3 --- fs/btrfs/zlib.c | 3 --- fs/btrfs/zstd.c | 49 ------------------------------------- 5 files changed, 1 insertion(+), 111 deletions(-)
fs/btrfs/compression.c +1 -54
@@ -688,8 +688,6 @@ struct heuristic_ws {struct list_head list; }; -static struct workspace_manager heuristic_wsm; -static void free_heuristic_ws (struct list_head *ws) {struct heuristic_ws *workspace;
@@ -729,9 +727,7 @@ static struct list_head *alloc_heuristic_ws(struct btrfs_fs_info *fs_info)return ERR_PTR (-ENOMEM ); } -const struct btrfs_compress_op btrfs_heuristic_compress = { - .workspace_manager = &heuristic_wsm, -}; +const struct btrfs_compress_op btrfs_heuristic_compress = {0 };static const struct btrfs_compress_op *const btrfs_compress_op[] = {/* The heuristic is represented as compression type 0 */
@@ -807,32 +803,6 @@ static int generic_alloc_workspace_manager(struct btrfs_fs_info *fs_info,return 0 ; } -static void btrfs_init_workspace_manager (struct btrfs_fs_info *fs_info,int type) -{ -struct workspace_manager *wsm; -struct list_head *workspace; - - wsm = btrfs_compress_op[type]->workspace_manager; -INIT_LIST_HEAD (&wsm->idle_ws); -spin_lock_init (&wsm->ws_lock); -atomic_set (&wsm->total_ws,0 ); -init_waitqueue_head (&wsm->ws_wait); - -/* -* Preallocate one workspace for each compression type so we can -* guarantee forward progress in the worst case -*/ - workspace =alloc_workspace (fs_info, type,0 ); -if (IS_ERR (workspace)) { -btrfs_warn (fs_info, -"cannot preallocate compression workspace, will try later" ); - }else { -atomic_set (&wsm->total_ws,1 ); - wsm->free_ws =1 ; -list_add (workspace, &wsm->idle_ws); - } -} -static void generic_free_workspace_manager (struct btrfs_fs_info *fs_info,enum btrfs_compression_type type) {
@@ -853,20 +823,6 @@ static void generic_free_workspace_manager(struct btrfs_fs_info *fs_info,kfree (gwsm); } -static void btrfs_cleanup_workspace_manager (int type) -{ -struct workspace_manager *wsman; -struct list_head *ws; - - wsman = btrfs_compress_op[type]->workspace_manager; -while (!list_empty (&wsman->idle_ws)) { - ws = wsman->idle_ws.next; -list_del (ws); -free_workspace (type, ws); -atomic_dec (&wsman->total_ws); - } -} -/* * This finds an available workspace or allocates a new one. * If it's not possible to allocate a new one, waits until there's one.
@@ -1192,11 +1148,6 @@ int __init btrfs_init_compress(void)if (!compr_pool.shrinker)return -ENOMEM ; -btrfs_init_workspace_manager (NULL ,BTRFS_COMPRESS_NONE ); -btrfs_init_workspace_manager (NULL ,BTRFS_COMPRESS_ZLIB ); -btrfs_init_workspace_manager (NULL ,BTRFS_COMPRESS_LZO ); -zstd_init_workspace_manager (NULL ); -spin_lock_init (&compr_pool.lock);INIT_LIST_HEAD (&compr_pool.list); compr_pool.count =0 ;
@@ -1217,10 +1168,6 @@ void __cold btrfs_exit_compress(void)btrfs_compr_pool_scan (NULL ,NULL );shrinker_free (compr_pool.shrinker); -btrfs_cleanup_workspace_manager (BTRFS_COMPRESS_NONE ); -btrfs_cleanup_workspace_manager (BTRFS_COMPRESS_ZLIB ); -btrfs_cleanup_workspace_manager (BTRFS_COMPRESS_LZO ); -zstd_cleanup_workspace_manager ();bioset_exit (&btrfs_compressed_bioset); }
fs/btrfs/compression.h +0 -2
@@ -183,8 +183,6 @@ int zstd_decompress(struct list_head *ws, const u8 *data_in, size_t destlen);int zstd_alloc_workspace_manager (struct btrfs_fs_info *fs_info);void zstd_free_workspace_manager (struct btrfs_fs_info *fs_info); -void zstd_init_workspace_manager (struct btrfs_fs_info *fs_info); -void zstd_cleanup_workspace_manager (void );struct list_head *zstd_alloc_workspace (struct btrfs_fs_info *fs_info,int level);void zstd_free_workspace (struct list_head *ws);struct list_head *zstd_get_workspace (struct btrfs_fs_info *fs_info,int level);
fs/btrfs/lzo.c +0 -3
@@ -68,8 +68,6 @@ struct workspace {struct list_head list; }; -static struct workspace_manager wsm; -void lzo_free_workspace (struct list_head *ws) { struct workspace *workspace = list_entry(ws, struct workspace, list);
@@ -489,7 +487,6 @@ int lzo_decompress(struct list_head *ws, const u8 *data_in, }const struct btrfs_compress_op btrfs_lzo_compress = { - .workspace_manager = &wsm, .max_level =1 , .default_level =1 , };
fs/btrfs/zlib.c +0 -3
@@ -34,8 +34,6 @@ struct workspace {int level; }; -static struct workspace_manager wsm; -struct list_head *zlib_get_workspace (struct btrfs_fs_info *fs_info,unsigned int level) {struct list_head *ws =btrfs_get_workspace (fs_info,BTRFS_COMPRESS_ZLIB , level);
@@ -483,7 +481,6 @@ int zlib_decompress(struct list_head *ws, const u8 *data_in, }const struct btrfs_compress_op btrfs_zlib_compress = { - .workspace_manager = &wsm, .min_level =1 , .max_level =9 , .default_level =BTRFS_ZLIB_DEFAULT_LEVEL ,
fs/btrfs/zstd.c +0 -49
73 lines not on this page · raw message
Signature · 1 line
2.50.1
PATCH 7/7 btrfs: reduce workspace buffer space to block size
In reply to Qu WenruoCurrently the compression workspace buffer size is always based on PAGE_SIZE, but btrfs has support subpage sized block size for some time.
This means for one-shot compression algorithm like lzo, we're wasting quite some memory if the block size is smaller than page size, as the LZO only works on one block (thus one-shot).
On 64K page sized systems with 4K block size, it means we only need at most 8K buffer space for lzo, but in reality we're allocating 64K buffer.
So to reduce the memory usage, change all workspace buffer to base its size based on block size other than page size.
--- fs/btrfs/lzo.c | 20 +++++++++++++------- fs/btrfs/zlib.c | 5 +++-- fs/btrfs/zstd.c | 6 ++++-- 3 files changed, 20 insertions(+), 11 deletions(-)
fs/btrfs/lzo.c +13 -7
@@ -58,9 +58,6 @@* 0x1000 | SegHdr N+1| Data payload N+1 ... | */ -#define WORKSPACE_BUF_LENGTH (lzo1x_worst_compress(PAGE_SIZE)) -#define WORKSPACE_CBUF_LENGTH (lzo1x_worst_compress(PAGE_SIZE)) -struct workspace {void *mem;void *buf;/* where decompressed data goes */
@@ -68,6 +65,15 @@ struct workspace {struct list_head list; }; +static u32 workspace_buf_length (const struct btrfs_fs_info *fs_info) +{ +return lzo1x_worst_compress (fs_info->sectorsize); +} +static u32 workspace_cbuf_length (const struct btrfs_fs_info *fs_info) +{ +return lzo1x_worst_compress (fs_info->sectorsize); +} +void lzo_free_workspace (struct list_head *ws) { struct workspace *workspace = list_entry(ws, struct workspace, list);
@@ -87,8 +93,8 @@ struct list_head *lzo_alloc_workspace(struct btrfs_fs_info *fs_info)return ERR_PTR (-ENOMEM ); workspace->mem =kvmalloc (LZO1X_MEM_COMPRESS ,GFP_KERNEL | __GFP_NOWARN); - workspace->buf =kvmalloc (WORKSPACE_BUF_LENGTH ,GFP_KERNEL | __GFP_NOWARN); - workspace->cbuf =kvmalloc (WORKSPACE_CBUF_LENGTH ,GFP_KERNEL | __GFP_NOWARN); + workspace->buf =kvmalloc (workspace_buf_length (fs_info),GFP_KERNEL | __GFP_NOWARN); + workspace->cbuf =kvmalloc (workspace_cbuf_length (fs_info),GFP_KERNEL | __GFP_NOWARN);if (!workspace->mem || !workspace->buf || !workspace->cbuf) goto fail;
@@ -384,7 +390,7 @@ int lzo_decompress_bio(struct list_head *ws, struct compressed_bio *cb)kunmap_local (kaddr); cur_in +=LZO_LEN ; -if (unlikely (seg_len >WORKSPACE_CBUF_LENGTH )) { +if (unlikely (seg_len >workspace_cbuf_length (fs_info))) {struct btrfs_inode *inode = cb->bbio.inode;/*
@@ -444,7 +450,7 @@ int lzo_decompress(struct list_head *ws, const u8 *data_in,const u32 sectorsize = fs_info->sectorsize;size_t in_len;size_t out_len; -size_t max_segment_len =WORKSPACE_BUF_LENGTH ; +size_t max_segment_len =workspace_buf_length (fs_info);int ret =0 ;if (srclen <LZO_LEN || srclen > max_segment_len +LZO_LEN *2 )
fs/btrfs/zlib.c +3 -2
@@ -55,6 +55,7 @@ void zlib_free_workspace(struct list_head *ws)struct list_head *zlib_alloc_workspace (struct btrfs_fs_info *fs_info,unsigned int level) { +const u32 blocksize = fs_info->sectorsize;struct workspace *workspace;int workspacesize;
@@ -78,8 +79,8 @@ struct list_head *zlib_alloc_workspace(struct btrfs_fs_info *fs_info, unsigned i workspace->buf_size =ZLIB_DFLTCC_BUF_SIZE ; }if (!workspace->buf) { - workspace->buf =kmalloc (PAGE_SIZE ,GFP_KERNEL ); - workspace->buf_size =PAGE_SIZE ; + workspace->buf =kmalloc (blocksize,GFP_KERNEL ); + workspace->buf_size = blocksize; }if (!workspace->strm.workspace || !workspace->buf) goto fail;
fs/btrfs/zstd.c +4 -2
@@ -370,6 +370,7 @@ void zstd_free_workspace(struct list_head *ws)struct list_head *zstd_alloc_workspace (struct btrfs_fs_info *fs_info,int level) { +const u32 blocksize = fs_info->sectorsize;struct workspace *workspace; workspace =kzalloc (sizeof (*workspace),GFP_KERNEL );
@@ -382,7 +383,7 @@ struct list_head *zstd_alloc_workspace(struct btrfs_fs_info *fs_info, int level) workspace->req_level = level; workspace->last_used = jiffies; workspace->mem =kvmalloc (workspace->size,GFP_KERNEL | __GFP_NOWARN); - workspace->buf =kmalloc (PAGE_SIZE ,GFP_KERNEL ); + workspace->buf =kmalloc (blocksize,GFP_KERNEL );if (!workspace->mem || !workspace->buf) goto fail;
@@ -590,6 +591,7 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb)size_t srclen = cb->compressed_len;zstd_dstream *stream;int ret =0 ; +const u32 blocksize =cb_to_fs_info (cb)->sectorsize;unsigned long folio_in_index =0 ;unsigned long total_folios_in =DIV_ROUND_UP (srclen,PAGE_SIZE );unsigned long buf_start;
@@ -613,7 +615,7 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb) workspace->out_buf.dst = workspace->buf; workspace->out_buf.pos =0 ; - workspace->out_buf.size =PAGE_SIZE ; + workspace->out_buf.size = blocksize;while (1 ) {size_t ret2;
Signature · 1 line
2.50.1
Re: [PATCH 2/7] btrfs: add workspace manager initialization for zstd
In reply to Qu WenruoQuoted 13 lines · Qu Wenruo
This involves:
- Add
zstd_alloc_workspace_manager()andzstd_free_workspace_manager()Those two functions will accept anfs_infopointer, and alloc/free fs_info->compr_wsm[BTRFS_COMPRESS_ZSTD] pointer.- Add
btrfs_alloc_compress_wsm()andbtrfs_free_compress_wsm()Those are helpers allocating the workspace managers for all algorithms.
For now only zstd is supported, and the timing is a little unusual, thebtrfs_alloc_compress_wsm()should only be called after the sectorsize being initialized.
Quoted 7 lines
Meanwhile
btrfs_free_fs_info_compress()is called inbtrfs_free_fs_info().- Move the definition of
btrfs_compression_typeto "fs.h" The reason is that "compression.h" has already included "fs.h", thus we can not just include "compression.h" to get the definition ofBTRFS_NR_COMPRESS_TYPESto define fs_info::compr_wsm[].
This is a bit unfortunate, I'd like to keep the subystems in their own files but we'd have to add another kind of indirection to resolve that. Either a new header or another structure for the compression types that is not embedded in fs_info. But the type definitions are short and used in a lot of code anyway so no big deal.
Re: [PATCH 3/7] btrfs: add generic workspace manager initialization
In reply to Qu WenruoQuoted 28 lines · Qu Wenruo
This involves:
- Add generic_(alloc|free)_workspace_manager helpers.
These are the helper to alloc/freeworkspace_managerstructure, which will allocate aworkspace_managerstructure, initialize it, and pre-allocate one workspace for it.- Call
generic_alloc_workspace_manager()inside
btrfs_alloc_compress_wsm()
For none, zlib and lzo compression algorithms.- Call
generic_alloc_workspace_manager()inside
btrfs_free_compress_wsm()
For none, zlib and lzo compression algorithms.For now the generic per-fs workspace managers won't really have any effect, and all compression is still going through the global workspace manager.
Signed-off-by Qu Wenruo <[email protected]>--- fs/btrfs/compression.c | 66 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+)fs/btrfs/compression.c +0
-0
fs/btrfs/compression.c static void free_workspace(int type, struct list_head *ws) · line 777
Quoted 5 lines
fs/btrfs/compression.c +2
-0@@ -773,6 +773,40 @@ static void free_workspace(int type, struct list_head *ws) } } +static int generic_alloc_workspace_manager (struct btrfs_fs_info *fs_info, +enum btrfs_compression_type type)
The helpers are static so you may drop the "generic_" prefix, we have other unprefixed helpers like alloc_workspace etc.
Quoted 38 lines · Qu Wenruo
Currently btrfs utilizes the following components at a per-module basis:
- Folios pool
A per-module pool for compressed data.
All folios in the pool are page sized.- Workspace
A workspace is a compression algorithm specific structure, providing things like extra memory buffer and compression level handling.- Workspace manager
The workspace manager is managing the above workspaces for each algorithm.All the folio pool/workspaces are using the fixed
PAGE_SIZEbuffer size, this is fine for now as even for block size (bs) < page size (ps) cases, a larger buffer size won't cause huge problems except wasting memories.However if we're going to support bs > ps, this fixed
PAGE_SIZEbuffer and per-module shared folios pool/workspaces will not work at all.To address this problem, this series will move the workspace and workspace manager into a per-fs basis, so that different fses (with different block size) can have their own workspaces.
This brings a small memory usage reduce for bs < ps cases. Now zlib/lzo/zstd will only allocate buffer using block size.
This is especially useful for lzo compression algorithm, as lzo is an one-short compression algorithm, it doesn't support multi-shot (aka, swapping input/output buffer halfway) compress/decompress.
Thus btrfs goes per-block compression for LZO, and compressed result will never go larger than a block (or btrfs will just give up). In that case, a 64K page sized buffer will waste 7/8th of the buffer.
This is part 1 of the preparation for btrfs bs > ps support.
Quoted 8 lines
Qu Wenruo (7):
btrfs: add anfs_infoparameter for compression workspace manager
btrfs: add workspace manager initialization for zstd
btrfs: add generic workspace manager initialization
btrfs: migrate to use per-fs workspace manager
btrfs: cleanup the per-module workspace managers
btrfs: renamebtrfs_compress_optobtrfs_compress_levels
btrfs: reduce workspace buffer space to block size
Re: [PATCH 2/7] btrfs: add workspace manager initialization for zstd
In reply to David SterbaQuoted 22 lines · David Sterba
Quoted 13 lines · Qu Wenruo
This involves:
- Add
zstd_alloc_workspace_manager()andzstd_free_workspace_manager()Those two functions will accept anfs_infopointer, and alloc/free fs_info->compr_wsm[BTRFS_COMPRESS_ZSTD] pointer.- Add
btrfs_alloc_compress_wsm()andbtrfs_free_compress_wsm()Those are helpers allocating the workspace managers for all algorithms.
For now only zstd is supported, and the timing is a little unusual, thebtrfs_alloc_compress_wsm()should only be called after the sectorsize being initialized.Quoted 7 lines
Meanwhile
btrfs_free_fs_info_compress()is called inbtrfs_free_fs_info().- Move the definition of
btrfs_compression_typeto "fs.h" The reason is that "compression.h" has already included "fs.h", thus we can not just include "compression.h" to get the definition ofBTRFS_NR_COMPRESS_TYPESto define fs_info::compr_wsm[].
Quoted 5 lines
This is a bit unfortunate, I'd like to keep the subystems in their own files but we'd have to add another kind of indirection to resolve that. Either a new header or another structure for the compression types that is not embedded in
fs_info. But the type definitions are short and used in a lot of code anyway so no big deal.
Yeah, I'm fine with the change as a temporary fix.
For the long time, I'm always wondering why we didn't put this into on-disk format.
It's part of the btrfs_file_extent_item::compression definition, thus on-disk format seems like a better fit (for the 4 supported value + NR).
Thanks,
Qu