Skip to contentSkip to conversation

f2fs: add compress_mode mount option

PATCH In review Daeho Jeong 3 participants 10 messages Dec 1, 2020 – Dec 8, 2020 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 f2fs: add compress_mode mount option

From: Daeho Jeong <[email protected]>

We will add a new "compress_mode" mount option to control file compression mode. This supports "fs" and "user". In "fs" mode (default), f2fs does automatic compression on the compression enabled files. In "user" mode, f2fs disables the automaic compression and gives the user discretion of choosing the target file and the timing. It means the user can do manual compression/decompression on the compression enabled files using ioctls.

---
v2: changed mount option name and added more explanation of mount option

---
 Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
 fs/f2fs/compress.c                 |  2 +-
 fs/f2fs/data.c                     |  2 +-
 fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
 fs/f2fs/segment.c                  |  2 +-
 fs/f2fs/super.c                    | 23 ++++++++++++++++++++
 6 files changed, 91 insertions(+), 3 deletions(-)
Documentation/filesystems/f2fs.rst +35 -0
index b8ee761c9922..5eb8d63439ec 100644
@@ -260,6 +260,13 @@ compress_extension=%s	 Support adding specified extension, so that f2fs can enab
 			 For other files, we can still enable compression via ioctl.
 			 Note that, there is one reserved special extension '*', it
 			 can be set to enable compression for all files.
+compress_mode=%s	 Control file compression mode. This supports "fs" and "user"
+			 modes. In "fs" mode (default), f2fs does automatic compression
+			 on the compression enabled files. In "user" mode, f2fs disables
+			 the automaic compression and gives the user discretion of
+			 choosing the target file and the timing. The user can do manual
+			 compression/decompression on the compression enabled files using
+			 ioctls.
 inlinecrypt		 When possible, encrypt/decrypt the contents of encrypted
 			 files using the blk-crypto framework rather than
 			 filesystem-layer encryption. This allows the use of
@@ -810,6 +817,34 @@ Compress metadata layout::
 	| data length | data chksum | reserved |      compressed data       |
 	+-------------+-------------+----------+----------------------------+
 
+Compression mode
+--------------------------
+
+f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
+With this option, f2fs provides a choice to select the way how to compress the
+compression enabled files (refer to "Compression implementation" section for how to
+enable compression on a regular inode).
+
+1) compress_mode=fs
+This is the default option. f2fs does automatic compression in the writeback of the
+compression enabled files.
+
+2) compress_mode=user
+This disables the automaic compression and gives the user discretion of choosing the
+target file and the timing. The user can do manual compression/decompression on the
+compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
+ioctls like the below.
+
+To decompress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
+
+To compress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
+
 NVMe Zoned Namespace devices
 ----------------------------
 
fs/f2fs/compress.c +1 -1
index 3957a84a185e..87090da8693d 100644
@@ -926,7 +926,7 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
 
 static bool cluster_may_compress(struct compress_ctx *cc)
 {
-	if (!f2fs_compressed_file(cc->inode))
+	if (!f2fs_need_compress_data(cc->inode))
 		return false;
 	if (f2fs_is_atomic_file(cc->inode))
 		return false;
fs/f2fs/data.c +1 -1
index be4da52604ed..42254d3859c7 100644
@@ -3164,7 +3164,7 @@ static inline bool __should_serialize_io(struct inode *inode,
 	if (IS_NOQUOTA(inode))
 		return false;
 
-	if (f2fs_compressed_file(inode))
+	if (f2fs_need_compress_data(inode))
 		return true;
 	if (wbc->sync_mode != WB_SYNC_ALL)
 		return true;
fs/f2fs/f2fs.h +30 -0
index e0826779a101..94d16bde5e24 100644
@@ -149,6 +149,7 @@ struct f2fs_mount_info {
 	unsigned char compress_algorithm;	/* algorithm type */
 	unsigned compress_log_size;		/* cluster log size */
 	unsigned char compress_ext_cnt;		/* extension count */
+	int compress_mode;			/* compression mode */
 	unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN];	/* extensions */
 };
 
@@ -677,6 +678,7 @@ enum {
 	FI_VERITY_IN_PROGRESS,	/* building fs-verity Merkle tree */
 	FI_COMPRESSED_FILE,	/* indicate file's data can be compressed */
 	FI_MMAP_FILE,		/* indicate file was mmapped */
+	FI_ENABLE_COMPRESS,	/* enable compression in "user" compression mode */
 	FI_MAX,			/* max flag, never be used */
 };
 
@@ -1243,6 +1245,18 @@ enum fsync_mode {
 	FSYNC_MODE_NOBARRIER,	/* fsync behaves nobarrier based on posix */
 };
 
+enum {
+	COMPR_MODE_FS,		/*
+				 * automatically compress compression
+				 * enabled files
+				 */
+	COMPR_MODE_USER,	/*
+				 * automatical compression is disabled.
+				 * user can control the file compression
+				 * using ioctls
+				 */
+};
+
 /*
  * this value is set in page as a private data which indicate that
  * the page is atomically written, and it is in inmem_pages list.
@@ -2752,6 +2766,22 @@ static inline int f2fs_compressed_file(struct inode *inode)
 		is_inode_flag_set(inode, FI_COMPRESSED_FILE);
 }
 
+static inline bool f2fs_need_compress_data(struct inode *inode)
+{
+	int compress_mode = F2FS_OPTION(F2FS_I_SB(inode)).compress_mode;
+
+	if (!f2fs_compressed_file(inode))
+		return false;
+
+	if (compress_mode == COMPR_MODE_FS)
+		return true;
+	else if (compress_mode == COMPR_MODE_USER &&
+			is_inode_flag_set(inode, FI_ENABLE_COMPRESS))
+		return true;
+
+	return false;
+}
+
 static inline unsigned int addrs_per_inode(struct inode *inode)
 {
 	unsigned int addrs = CUR_ADDRS_PER_INODE(inode) -
fs/f2fs/segment.c +1 -1
index 1596502f7375..5d6c9d6f237b 100644
@@ -3254,7 +3254,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
 			else
 				return CURSEG_COLD_DATA;
 		}
-		if (file_is_cold(inode) || f2fs_compressed_file(inode))
+		if (file_is_cold(inode) || f2fs_need_compress_data(inode))
 			return CURSEG_COLD_DATA;
 		if (file_is_hot(inode) ||
 				is_inode_flag_set(inode, FI_HOT_DATA) ||
fs/f2fs/super.c +23 -0
index 87f7a6e86370..cc6eb4ed80a2 100644
@@ -146,6 +146,7 @@ enum {
 	Opt_compress_algorithm,
 	Opt_compress_log_size,
 	Opt_compress_extension,
+	Opt_compress_mode,
 	Opt_atgc,
 	Opt_err,
 };
@@ -214,6 +215,7 @@ static match_table_t f2fs_tokens = {
 	{Opt_compress_algorithm, "compress_algorithm=%s"},
 	{Opt_compress_log_size, "compress_log_size=%u"},
 	{Opt_compress_extension, "compress_extension=%s"},
+	{Opt_compress_mode, "compress_mode=%s"},
 	{Opt_atgc, "atgc"},
 	{Opt_err, NULL},
 };
@@ -934,10 +936,25 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount)
 			F2FS_OPTION(sbi).compress_ext_cnt++;
 			kfree(name);
 			break;
+		case Opt_compress_mode:
+			name = match_strdup(&args[0]);
+			if (!name)
+				return -ENOMEM;
+			if (!strcmp(name, "fs")) {
+				F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
+			} else if (!strcmp(name, "user")) {
+				F2FS_OPTION(sbi).compress_mode = COMPR_MODE_USER;
+			} else {
+				kfree(name);
+				return -EINVAL;
+			}
+			kfree(name);
+			break;
 #else
 		case Opt_compress_algorithm:
 		case Opt_compress_log_size:
 		case Opt_compress_extension:
+		case Opt_compress_mode:
 			f2fs_info(sbi, "compression options not supported");
 			break;
 #endif
@@ -1523,6 +1540,11 @@ static inline void f2fs_show_compress_options(struct seq_file *seq,
 		seq_printf(seq, ",compress_extension=%s",
 			F2FS_OPTION(sbi).extensions[i]);
 	}
+
+	if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_FS)
+		seq_printf(seq, ",compress_mode=%s", "fs");
+	else if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_USER)
+		seq_printf(seq, ",compress_mode=%s", "user");
 }
 
 static int f2fs_show_options(struct seq_file *seq, struct dentry *root)
@@ -1672,6 +1694,7 @@ static void default_options(struct f2fs_sb_info *sbi)
 	F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
 	F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
 	F2FS_OPTION(sbi).compress_ext_cnt = 0;
+	F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
 	F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
 
 	sbi->sb->s_flags &= ~SB_INLINECRYPT;
Signature · 8 lines
2.29.2.454.gaff20da3a2-goog



_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
#1 lore raw

PATCH v2 2/2 f2fs: add F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE

From: Daeho Jeong <[email protected]>

Added two ioctl to decompress/compress explicitly the compression enabled file in "compress_mode=user" mount option.

Using these two ioctls, the users can make a control of compression and decompression of their files.

---
v2: reformed codes based on comments and put gradual flush routine

---
 fs/f2fs/file.c            | 185 ++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/f2fs.h |   2 +
 2 files changed, 187 insertions(+)
fs/f2fs/file.c +185 -0
index be8db06aca27..300355fe25f0 100644
@@ -4026,6 +4026,185 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
 	return ret;
 }
 
+static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
+{
+	DEFINE_READAHEAD(ractl, NULL, inode->i_mapping, page_idx);
+	struct address_space *mapping = inode->i_mapping;
+	struct page *page;
+	pgoff_t redirty_idx = page_idx;
+	int i, page_len = 0, ret = 0;
+
+	page_cache_ra_unbounded(&ractl, len, 0);
+
+	for (i = 0; i < len; i++, page_idx++) {
+		page = read_cache_page(mapping, page_idx, NULL, NULL);
+		if (IS_ERR(page)) {
+			ret = PTR_ERR(page);
+			break;
+		}
+		page_len++;
+	}
+
+	for (i = 0; i < page_len; i++, redirty_idx++) {
+		page = find_lock_page(mapping, redirty_idx);
+		if (!page)
+			ret = -ENOENT;
+		set_page_dirty(page);
+		f2fs_put_page(page, 1);
+		f2fs_put_page(page, 0);
+	}
+
+	return ret;
+}
+
+static int f2fs_ioc_decompress_file(struct file *filp, unsigned long arg)
+{
+	struct inode *inode = file_inode(filp);
+	struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+	struct f2fs_inode_info *fi = F2FS_I(inode);
+	pgoff_t page_idx = 0, last_idx;
+	unsigned int blk_per_seg = sbi->blocks_per_seg, dirtied = 0;
+	int cluster_size = F2FS_I(inode)->i_cluster_size;
+	int count, ret;
+
+	if (!f2fs_sb_has_compression(sbi) ||
+			F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
+		return -EOPNOTSUPP;
+
+	if (!(filp->f_mode & FMODE_WRITE))
+		return -EBADF;
+
+	if (!f2fs_compressed_file(inode))
+		return -EINVAL;
+
+	if (!f2fs_is_compress_backend_ready(inode))
+		return -ENOPKG;
+
+	f2fs_balance_fs(F2FS_I_SB(inode), true);
+
+	file_start_write(filp);
+	inode_lock(inode);
+
+	if (f2fs_is_mmap_file(inode)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
+	if (ret)
+		goto out;
+
+	if (!atomic_read(&fi->i_compr_blocks))
+		goto out;
+
+	last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+
+	count = last_idx - page_idx;
+	while (count) {
+		int len = min(cluster_size, count);
+
+		ret = redirty_blocks(inode, page_idx, len);
+		if (ret < 0)
+			break;
+
+		count -= len;
+		page_idx += len;
+		dirtied += len;
+		if (dirtied >= blk_per_seg) {
+			filemap_fdatawrite(inode->i_mapping);
+			dirtied = 0;
+		}
+	}
+
+	if (!ret)
+		ret = filemap_write_and_wait_range(inode->i_mapping, 0,
+							LLONG_MAX);
+
+	if (ret)
+		f2fs_warn(sbi, "%s: The file might be partially decompressed "
+				"(errno=%d). Please delete the file.\n",
+				__func__, ret);
+out:
+	inode_unlock(inode);
+	file_end_write(filp);
+
+	return ret;
+}
+
+static int f2fs_ioc_compress_file(struct file *filp, unsigned long arg)
+{
+	struct inode *inode = file_inode(filp);
+	struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+	pgoff_t page_idx = 0, last_idx;
+	unsigned int blk_per_seg = sbi->blocks_per_seg, dirtied = 0;
+	int cluster_size = F2FS_I(inode)->i_cluster_size;
+	int count, ret;
+
+	if (!f2fs_sb_has_compression(sbi) ||
+			F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
+		return -EOPNOTSUPP;
+
+	if (!(filp->f_mode & FMODE_WRITE))
+		return -EBADF;
+
+	if (!f2fs_compressed_file(inode))
+		return -EINVAL;
+
+	if (!f2fs_is_compress_backend_ready(inode))
+		return -ENOPKG;
+
+	f2fs_balance_fs(F2FS_I_SB(inode), true);
+
+	file_start_write(filp);
+	inode_lock(inode);
+
+	if (f2fs_is_mmap_file(inode)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
+	if (ret)
+		goto out;
+
+	set_inode_flag(inode, FI_ENABLE_COMPRESS);
+
+	last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+
+	count = last_idx - page_idx;
+	while (count) {
+		int len = min(cluster_size, count);
+
+		ret = redirty_blocks(inode, page_idx, len);
+		if (ret < 0)
+			break;
+
+		count -= len;
+		page_idx += len;
+		dirtied += len;
+		if (dirtied >= blk_per_seg) {
+			filemap_fdatawrite(inode->i_mapping);
+			dirtied = 0;
+		}
+	}
+
+	if (!ret)
+		ret = filemap_write_and_wait_range(inode->i_mapping, 0,
+							LLONG_MAX);
+
+	clear_inode_flag(inode, FI_ENABLE_COMPRESS);
+
+	if (ret)
+		f2fs_warn(sbi, "%s: The file might be partially compressed "
+				"(errno=%d). Please delete the file.\n",
+				__func__, ret);
+out:
+	inode_unlock(inode);
+	file_end_write(filp);
+
+	return ret;
+}
+
 static long __f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 {
 	switch (cmd) {
@@ -4113,6 +4292,10 @@ static long __f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
 		return f2fs_ioc_get_compress_option(filp, arg);
 	case F2FS_IOC_SET_COMPRESS_OPTION:
 		return f2fs_ioc_set_compress_option(filp, arg);
+	case F2FS_IOC_DECOMPRESS_FILE:
+		return f2fs_ioc_decompress_file(filp, arg);
+	case F2FS_IOC_COMPRESS_FILE:
+		return f2fs_ioc_compress_file(filp, arg);
 	default:
 		return -ENOTTY;
 	}
@@ -4352,6 +4535,8 @@ long f2fs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 	case F2FS_IOC_SEC_TRIM_FILE:
 	case F2FS_IOC_GET_COMPRESS_OPTION:
 	case F2FS_IOC_SET_COMPRESS_OPTION:
+	case F2FS_IOC_DECOMPRESS_FILE:
+	case F2FS_IOC_COMPRESS_FILE:
 		break;
 	default:
 		return -ENOIOCTLCMD;
include/uapi/linux/f2fs.h +2 -0
index f00199a2e38b..352a822d4370 100644
@@ -40,6 +40,8 @@
 						struct f2fs_comp_option)
 #define F2FS_IOC_SET_COMPRESS_OPTION	_IOW(F2FS_IOCTL_MAGIC, 22,	\
 						struct f2fs_comp_option)
+#define F2FS_IOC_DECOMPRESS_FILE	_IO(F2FS_IOCTL_MAGIC, 23)
+#define F2FS_IOC_COMPRESS_FILE		_IO(F2FS_IOCTL_MAGIC, 24)
 
 /*
  * should be same as XFS_IOC_GOINGDOWN.
Signature · 8 lines
2.29.2.454.gaff20da3a2-goog



_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
#2 lore raw

Re: [f2fs-dev] [PATCH v2 2/2] f2fs: add F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE

Quoted 67 lines · Daeho Jeong

From: Daeho Jeong <[email protected]>

Added two ioctl to decompress/compress explicitly the compression enabled file in "compress_mode=user" mount option.

Using these two ioctls, the users can make a control of compression and decompression of their files.

---
v2: reformed codes based on comments and put gradual flush routine

---
  fs/f2fs/file.c            | 185 ++++++++++++++++++++++++++++++++++++++
  include/uapi/linux/f2fs.h |   2 +
  2 files changed, 187 insertions(+)
fs/f2fs/file.c +43 -0
index be8db06aca27..300355fe25f0 100644
@@ -4026,6 +4026,185 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
  	return ret;
  }
 
+static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
+{
+	DEFINE_READAHEAD(ractl, NULL, inode->i_mapping, page_idx);
+	struct address_space *mapping = inode->i_mapping;
+	struct page *page;
+	pgoff_t redirty_idx = page_idx;
+	int i, page_len = 0, ret = 0;
+
+	page_cache_ra_unbounded(&ractl, len, 0);
+
+	for (i = 0; i < len; i++, page_idx++) {
+		page = read_cache_page(mapping, page_idx, NULL, NULL);
+		if (IS_ERR(page)) {
+			ret = PTR_ERR(page);
+			break;
+		}
+		page_len++;
+	}
+
+	for (i = 0; i < page_len; i++, redirty_idx++) {
+		page = find_lock_page(mapping, redirty_idx);
+		if (!page)
+			ret = -ENOENT;
+		set_page_dirty(page);
+		f2fs_put_page(page, 1);
+		f2fs_put_page(page, 0);
+	}
+
+	return ret;
+}
+
+static int f2fs_ioc_decompress_file(struct file *filp, unsigned long arg)
+{
+	struct inode *inode = file_inode(filp);
+	struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+	struct f2fs_inode_info *fi = F2FS_I(inode);
+	pgoff_t page_idx = 0, last_idx;
+	unsigned int blk_per_seg = sbi->blocks_per_seg, dirtied = 0;
+	int cluster_size = F2FS_I(inode)->i_cluster_size;
+	int count, ret;
+
+	if (!f2fs_sb_has_compression(sbi) ||
+			F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)

fs/f2fs/file.c static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg) · line 4081

Quoted 10 lines
fs/f2fs/file.c +10 -0
+		return -EOPNOTSUPP;
+
+	if (!(filp->f_mode & FMODE_WRITE))
+		return -EBADF;
+
+	if (!f2fs_compressed_file(inode))
+		return -EINVAL;
+
+	if (!f2fs_is_compress_backend_ready(inode))
+		return -ENOPKG;

Oh, it looks there will be a race case with f2fs_ioc_set_compress_option(), it needs to relocate this condition under inode lock.

In all other places, it will return -EOPNOTSUPP, how about keeping in line with them?

Quoted 21 lines
fs/f2fs/file.c +21 -0
+
+	f2fs_balance_fs(F2FS_I_SB(inode), true);
+
+	file_start_write(filp);
+	inode_lock(inode);
+
+	if (f2fs_is_mmap_file(inode)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
+	if (ret)
+		goto out;
+
+	if (!atomic_read(&fi->i_compr_blocks))
+		goto out;
+
+	last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+
+	count = last_idx - page_idx;

fs/f2fs/file.c static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg) · line 4112

Quoted 10 lines
fs/f2fs/file.c +10 -0
+	while (count) {
+		int len = min(cluster_size, count);
+
+		ret = redirty_blocks(inode, page_idx, len);
+		if (ret < 0)
+			break;
+
+		count -= len;
+		page_idx += len;
+		dirtied += len;

use get_dirty_pages(inode) will be a little bit more accurate? as kworker is running to writeback dirty pages in background.

Quoted 32 lines
fs/f2fs/file.c +32 -0
+		if (dirtied >= blk_per_seg) {
+			filemap_fdatawrite(inode->i_mapping);
+			dirtied = 0;
+		}
+	}
+
+	if (!ret)
+		ret = filemap_write_and_wait_range(inode->i_mapping, 0,
+							LLONG_MAX);
+
+	if (ret)
+		f2fs_warn(sbi, "%s: The file might be partially decompressed "
+				"(errno=%d). Please delete the file.\n",
+				__func__, ret);
+out:
+	inode_unlock(inode);
+	file_end_write(filp);
+
+	return ret;
+}
+
+static int f2fs_ioc_compress_file(struct file *filp, unsigned long arg)
+{
+	struct inode *inode = file_inode(filp);
+	struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+	pgoff_t page_idx = 0, last_idx;
+	unsigned int blk_per_seg = sbi->blocks_per_seg, dirtied = 0;
+	int cluster_size = F2FS_I(inode)->i_cluster_size;
+	int count, ret;
+
+	if (!f2fs_sb_has_compression(sbi) ||
+			F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)

fs/f2fs/file.c static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg) · line 4154

Quoted 10 lines
fs/f2fs/file.c +10 -0
+		return -EOPNOTSUPP;
+
+	if (!(filp->f_mode & FMODE_WRITE))
+		return -EBADF;
+
+	if (!f2fs_compressed_file(inode))
+		return -EINVAL;
+
+	if (!f2fs_is_compress_backend_ready(inode))
+		return -ENOPKG;

Ditto,

Quoted 20 lines
fs/f2fs/file.c +20 -0
+
+	f2fs_balance_fs(F2FS_I_SB(inode), true);
+
+	file_start_write(filp);
+	inode_lock(inode);
+
+	if (f2fs_is_mmap_file(inode)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
+	if (ret)
+		goto out;
+
+	set_inode_flag(inode, FI_ENABLE_COMPRESS);
+
+	last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+
+	count = last_idx - page_idx;

fs/f2fs/file.c static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg) · line 4184

Quoted 10 lines
fs/f2fs/file.c +10 -0
+	while (count) {
+		int len = min(cluster_size, count);
+
+		ret = redirty_blocks(inode, page_idx, len);
+		if (ret < 0)
+			break;
+
+		count -= len;
+		page_idx += len;
+		dirtied += len;

Ditto,

Quoted 52 lines
fs/f2fs/file.c +29 -0
+		if (dirtied >= blk_per_seg) {
+			filemap_fdatawrite(inode->i_mapping);
+			dirtied = 0;
+		}
+	}
+
+	if (!ret)
+		ret = filemap_write_and_wait_range(inode->i_mapping, 0,
+							LLONG_MAX);
+
+	clear_inode_flag(inode, FI_ENABLE_COMPRESS);
+
+	if (ret)
+		f2fs_warn(sbi, "%s: The file might be partially compressed "
+				"(errno=%d). Please delete the file.\n",
+				__func__, ret);
+out:
+	inode_unlock(inode);
+	file_end_write(filp);
+
+	return ret;
+}
+
  static long __f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
  {
  	switch (cmd) {
@@ -4113,6 +4292,10 @@ static long __f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
  		return f2fs_ioc_get_compress_option(filp, arg);
  	case F2FS_IOC_SET_COMPRESS_OPTION:
  		return f2fs_ioc_set_compress_option(filp, arg);
+	case F2FS_IOC_DECOMPRESS_FILE:
+		return f2fs_ioc_decompress_file(filp, arg);
+	case F2FS_IOC_COMPRESS_FILE:
+		return f2fs_ioc_compress_file(filp, arg);
  	default:
  		return -ENOTTY;
  	}
@@ -4352,6 +4535,8 @@ long f2fs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
  	case F2FS_IOC_SEC_TRIM_FILE:
  	case F2FS_IOC_GET_COMPRESS_OPTION:
  	case F2FS_IOC_SET_COMPRESS_OPTION:
+	case F2FS_IOC_DECOMPRESS_FILE:
+	case F2FS_IOC_COMPRESS_FILE:
  		break;
  	default:
  		return -ENOIOCTLCMD;
include/uapi/linux/f2fs.h +0 -0
index f00199a2e38b..352a822d4370 100644

include/uapi/linux/f2fs.h line 47

Quoted 8 lines
include/uapi/linux/f2fs.h +2 -0
@@ -40,6 +40,8 @@
  						struct f2fs_comp_option)
  #define F2FS_IOC_SET_COMPRESS_OPTION	_IOW(F2FS_IOCTL_MAGIC, 22,	\
  						struct f2fs_comp_option)
+#define F2FS_IOC_DECOMPRESS_FILE	_IO(F2FS_IOCTL_MAGIC, 23)
+#define F2FS_IOC_COMPRESS_FILE		_IO(F2FS_IOCTL_MAGIC, 24)
 
  /*
   * should be same as XFS_IOC_GOINGDOWN.

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#3 lore raw
In reply to Daeho Jeong
Quoted 168 lines · Daeho Jeong

From: Daeho Jeong <[email protected]>

We will add a new "compress_mode" mount option to control file compression mode. This supports "fs" and "user". In "fs" mode (default), f2fs does automatic compression on the compression enabled files. In "user" mode, f2fs disables the automaic compression and gives the user discretion of choosing the target file and the timing. It means the user can do manual compression/decompression on the compression enabled files using ioctls.

---
v2: changed mount option name and added more explanation of mount option

---
  Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
  fs/f2fs/compress.c                 |  2 +-
  fs/f2fs/data.c                     |  2 +-
  fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
  fs/f2fs/segment.c                  |  2 +-
  fs/f2fs/super.c                    | 23 ++++++++++++++++++++
  6 files changed, 91 insertions(+), 3 deletions(-)
Documentation/filesystems/f2fs.rst +35 -0
index b8ee761c9922..5eb8d63439ec 100644
@@ -260,6 +260,13 @@ compress_extension=%s	 Support adding specified extension, so that f2fs can enab
  			 For other files, we can still enable compression via ioctl.
  			 Note that, there is one reserved special extension '*', it
  			 can be set to enable compression for all files.
+compress_mode=%s	 Control file compression mode. This supports "fs" and "user"
+			 modes. In "fs" mode (default), f2fs does automatic compression
+			 on the compression enabled files. In "user" mode, f2fs disables
+			 the automaic compression and gives the user discretion of
+			 choosing the target file and the timing. The user can do manual
+			 compression/decompression on the compression enabled files using
+			 ioctls.
  inlinecrypt		 When possible, encrypt/decrypt the contents of encrypted
  			 files using the blk-crypto framework rather than
  			 filesystem-layer encryption. This allows the use of
@@ -810,6 +817,34 @@ Compress metadata layout::
  	| data length | data chksum | reserved |      compressed data       |
  	+-------------+-------------+----------+----------------------------+
 
+Compression mode
+--------------------------
+
+f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
+With this option, f2fs provides a choice to select the way how to compress the
+compression enabled files (refer to "Compression implementation" section for how to
+enable compression on a regular inode).
+
+1) compress_mode=fs
+This is the default option. f2fs does automatic compression in the writeback of the
+compression enabled files.
+
+2) compress_mode=user
+This disables the automaic compression and gives the user discretion of choosing the
+target file and the timing. The user can do manual compression/decompression on the
+compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
+ioctls like the below.
+
+To decompress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
+
+To compress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
+
  NVMe Zoned Namespace devices
  ----------------------------
 
fs/f2fs/compress.c +1 -1
index 3957a84a185e..87090da8693d 100644
@@ -926,7 +926,7 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
 
  static bool cluster_may_compress(struct compress_ctx *cc)
  {
-	if (!f2fs_compressed_file(cc->inode))
+	if (!f2fs_need_compress_data(cc->inode))
  		return false;
  	if (f2fs_is_atomic_file(cc->inode))
  		return false;
fs/f2fs/data.c +1 -1
index be4da52604ed..42254d3859c7 100644
@@ -3164,7 +3164,7 @@ static inline bool __should_serialize_io(struct inode *inode,
  	if (IS_NOQUOTA(inode))
  		return false;
 
-	if (f2fs_compressed_file(inode))
+	if (f2fs_need_compress_data(inode))
  		return true;
  	if (wbc->sync_mode != WB_SYNC_ALL)
  		return true;
fs/f2fs/f2fs.h +30 -0
index e0826779a101..94d16bde5e24 100644
@@ -149,6 +149,7 @@ struct f2fs_mount_info {
  	unsigned char compress_algorithm;	/* algorithm type */
  	unsigned compress_log_size;		/* cluster log size */
  	unsigned char compress_ext_cnt;		/* extension count */
+	int compress_mode;			/* compression mode */
  	unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN];	/* extensions */
  };
 
@@ -677,6 +678,7 @@ enum {
  	FI_VERITY_IN_PROGRESS,	/* building fs-verity Merkle tree */
  	FI_COMPRESSED_FILE,	/* indicate file's data can be compressed */
  	FI_MMAP_FILE,		/* indicate file was mmapped */
+	FI_ENABLE_COMPRESS,	/* enable compression in "user" compression mode */
  	FI_MAX,			/* max flag, never be used */
  };
 
@@ -1243,6 +1245,18 @@ enum fsync_mode {
  	FSYNC_MODE_NOBARRIER,	/* fsync behaves nobarrier based on posix */
  };
 
+enum {
+	COMPR_MODE_FS,		/*
+				 * automatically compress compression
+				 * enabled files
+				 */
+	COMPR_MODE_USER,	/*
+				 * automatical compression is disabled.
+				 * user can control the file compression
+				 * using ioctls
+				 */
+};
+
  /*
   * this value is set in page as a private data which indicate that
   * the page is atomically written, and it is in inmem_pages list.
@@ -2752,6 +2766,22 @@ static inline int f2fs_compressed_file(struct inode *inode)
  		is_inode_flag_set(inode, FI_COMPRESSED_FILE);
  }
 
+static inline bool f2fs_need_compress_data(struct inode *inode)
+{
+	int compress_mode = F2FS_OPTION(F2FS_I_SB(inode)).compress_mode;
+
+	if (!f2fs_compressed_file(inode))
+		return false;
+
+	if (compress_mode == COMPR_MODE_FS)
+		return true;
+	else if (compress_mode == COMPR_MODE_USER &&
+			is_inode_flag_set(inode, FI_ENABLE_COMPRESS))
+		return true;
+
+	return false;
+}
+
  static inline unsigned int addrs_per_inode(struct inode *inode)
  {
  	unsigned int addrs = CUR_ADDRS_PER_INODE(inode) -
fs/f2fs/segment.c +0 -0
index 1596502f7375..5d6c9d6f237b 100644

fs/f2fs/segment.c static int __get_segment_type_6(struct f2fs_io_info *fio) · line 3257

Quoted 5 lines
fs/f2fs/segment.c +1 -1
@@ -3254,7 +3254,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
  			else
  				return CURSEG_COLD_DATA;
  		}
-		if (file_is_cold(inode) || f2fs_compressed_file(inode))
+		if (file_is_cold(inode) || f2fs_need_compress_data(inode))

Can we keep consistent about writting compressed file to code data area?

Or there is other concern about this?

Thanks,

Quoted 63 lines
fs/f2fs/segment.c +0 -0
  			return CURSEG_COLD_DATA;
  		if (file_is_hot(inode) ||
  				is_inode_flag_set(inode, FI_HOT_DATA) ||
fs/f2fs/super.c +22 -0
index 87f7a6e86370..cc6eb4ed80a2 100644
@@ -146,6 +146,7 @@ enum {
  	Opt_compress_algorithm,
  	Opt_compress_log_size,
  	Opt_compress_extension,
+	Opt_compress_mode,
  	Opt_atgc,
  	Opt_err,
  };
@@ -214,6 +215,7 @@ static match_table_t f2fs_tokens = {
  	{Opt_compress_algorithm, "compress_algorithm=%s"},
  	{Opt_compress_log_size, "compress_log_size=%u"},
  	{Opt_compress_extension, "compress_extension=%s"},
+	{Opt_compress_mode, "compress_mode=%s"},
  	{Opt_atgc, "atgc"},
  	{Opt_err, NULL},
  };
@@ -934,10 +936,25 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount)
  			F2FS_OPTION(sbi).compress_ext_cnt++;
  			kfree(name);
  			break;
+		case Opt_compress_mode:
+			name = match_strdup(&args[0]);
+			if (!name)
+				return -ENOMEM;
+			if (!strcmp(name, "fs")) {
+				F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
+			} else if (!strcmp(name, "user")) {
+				F2FS_OPTION(sbi).compress_mode = COMPR_MODE_USER;
+			} else {
+				kfree(name);
+				return -EINVAL;
+			}
+			kfree(name);
+			break;
  #else
  		case Opt_compress_algorithm:
  		case Opt_compress_log_size:
  		case Opt_compress_extension:
+		case Opt_compress_mode:
  			f2fs_info(sbi, "compression options not supported");
  			break;
  #endif
@@ -1523,6 +1540,11 @@ static inline void f2fs_show_compress_options(struct seq_file *seq,
  		seq_printf(seq, ",compress_extension=%s",
  			F2FS_OPTION(sbi).extensions[i]);
  	}
+
+	if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_FS)
+		seq_printf(seq, ",compress_mode=%s", "fs");
+	else if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_USER)
+		seq_printf(seq, ",compress_mode=%s", "user");
  }
 
  static int f2fs_show_options(struct seq_file *seq, struct dentry *root)

fs/f2fs/super.c static void default_options(struct f2fs_sb_info *sbi) · line 1700

Quoted 7 lines
fs/f2fs/super.c +1 -0
@@ -1672,6 +1694,7 @@ static void default_options(struct f2fs_sb_info *sbi)
  	F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
  	F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
  	F2FS_OPTION(sbi).compress_ext_cnt = 0;
+	F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
  	F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
 
  	sbi->sb->s_flags &= ~SB_INLINECRYPT;

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

A message that is not in the archive (sent off-list or not yet received): CACOAw_xafxyUZe1g-nXjTrkisbUqCvSk7js7amhqw3zADAq22Q@mail.gmail.com

#5 lore raw
Quoted 2 lines · Daeho Jeong

We might use compress_extension=*,compress_option=user. In this option, we're gonna allocate all the writes in cold zone.

Oh, so all files in data partition will be tagged as compressed file, but the compressing time will be controlled by user, then, only blocks belong to in-compress-process inode will go to cold area, right?

Thanks,

Quoted 247 lines

2020년 12월 2일 (수) 오후 3:31, Chao Yu <[email protected] <mailto:[email protected]>>님이 작성:

    On 2020/12/1 12:08, Daeho Jeong wrote:
     > From: Daeho Jeong <[email protected] <mailto:[email protected]>>
     >
     > We will add a new "compress_mode" mount option to control file
     > compression mode. This supports "fs" and "user". In "fs" mode (default),
     > f2fs does automatic compression on the compression enabled files.
     > In "user" mode, f2fs disables the automaic compression and gives the
     > user discretion of choosing the target file and the timing. It means
     > the user can do manual compression/decompression on the compression
     > enabled files using ioctls.
     >
     > Signed-off-by: Daeho Jeong <[email protected] <mailto:[email protected]>>
     > ---
     > v2: changed mount option name and added more explanation of mount option
     > ---
     >   Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
     >   fs/f2fs/compress.c                 |  2 +-
     >   fs/f2fs/data.c                     |  2 +-
     >   fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
     >   fs/f2fs/segment.c                  |  2 +-
     >   fs/f2fs/super.c                    | 23 ++++++++++++++++++++
     >   6 files changed, 91 insertions(+), 3 deletions(-)
     >
     > diff --git a/Documentation/filesystems/f2fs.rst b/Documentation/filesystems/f2fs.rst
     > index b8ee761c9922..5eb8d63439ec 100644
     > --- a/Documentation/filesystems/f2fs.rst
     > +++ b/Documentation/filesystems/f2fs.rst
     > @@ -260,6 +260,13 @@ compress_extension=%s     Support adding specified extension, so that f2fs can enab
     >                        For other files, we can still enable compression via ioctl.
     >                        Note that, there is one reserved special extension '*', it
     >                        can be set to enable compression for all files.
     > +compress_mode=%s      Control file compression mode. This supports "fs" and "user"
     > +                      modes. In "fs" mode (default), f2fs does automatic compression
     > +                      on the compression enabled files. In "user" mode, f2fs disables
     > +                      the automaic compression and gives the user discretion of
     > +                      choosing the target file and the timing. The user can do manual
     > +                      compression/decompression on the compression enabled files using
     > +                      ioctls.
     >   inlinecrypt          When possible, encrypt/decrypt the contents of encrypted
     >                        files using the blk-crypto framework rather than
     >                        filesystem-layer encryption. This allows the use of
     > @@ -810,6 +817,34 @@ Compress metadata layout::
     >       | data length | data chksum | reserved |      compressed data       |
     >       +-------------+-------------+----------+----------------------------+
     >
     > +Compression mode
     > +--------------------------
     > +
     > +f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
     > +With this option, f2fs provides a choice to select the way how to compress the
     > +compression enabled files (refer to "Compression implementation" section for how to
     > +enable compression on a regular inode).
     > +
     > +1) compress_mode=fs
     > +This is the default option. f2fs does automatic compression in the writeback of the
     > +compression enabled files.
     > +
     > +2) compress_mode=user
     > +This disables the automaic compression and gives the user discretion of choosing the
     > +target file and the timing. The user can do manual compression/decompression on the
     > +compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
     > +ioctls like the below.
     > +
     > +To decompress a file,
     > +
     > +fd = open(filename, O_WRONLY, 0);
     > +ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
     > +
     > +To compress a file,
     > +
     > +fd = open(filename, O_WRONLY, 0);
     > +ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
     > +
     >   NVMe Zoned Namespace devices
     >   ----------------------------
     >
     > diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
     > index 3957a84a185e..87090da8693d 100644
     > --- a/fs/f2fs/compress.c
     > +++ b/fs/f2fs/compress.c
     > @@ -926,7 +926,7 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
     >
     >   static bool cluster_may_compress(struct compress_ctx *cc)
     >   {
     > -     if (!f2fs_compressed_file(cc->inode))
     > +     if (!f2fs_need_compress_data(cc->inode))
     >               return false;
     >       if (f2fs_is_atomic_file(cc->inode))
     >               return false;
     > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
     > index be4da52604ed..42254d3859c7 100644
     > --- a/fs/f2fs/data.c
     > +++ b/fs/f2fs/data.c
     > @@ -3164,7 +3164,7 @@ static inline bool __should_serialize_io(struct inode *inode,
     >       if (IS_NOQUOTA(inode))
     >               return false;
     >
     > -     if (f2fs_compressed_file(inode))
     > +     if (f2fs_need_compress_data(inode))
     >               return true;
     >       if (wbc->sync_mode != WB_SYNC_ALL)
     >               return true;
     > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
     > index e0826779a101..94d16bde5e24 100644
     > --- a/fs/f2fs/f2fs.h
     > +++ b/fs/f2fs/f2fs.h
     > @@ -149,6 +149,7 @@ struct f2fs_mount_info {
     >       unsigned char compress_algorithm;       /* algorithm type */
     >       unsigned compress_log_size;             /* cluster log size */
     >       unsigned char compress_ext_cnt;         /* extension count */
     > +     int compress_mode;                      /* compression mode */
     >       unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN]; /* extensions */
     >   };
     >
     > @@ -677,6 +678,7 @@ enum {
     >       FI_VERITY_IN_PROGRESS,  /* building fs-verity Merkle tree */
     >       FI_COMPRESSED_FILE,     /* indicate file's data can be compressed */
     >       FI_MMAP_FILE,           /* indicate file was mmapped */
     > +     FI_ENABLE_COMPRESS,     /* enable compression in "user" compression mode */
     >       FI_MAX,                 /* max flag, never be used */
     >   };
     >
     > @@ -1243,6 +1245,18 @@ enum fsync_mode {
     >       FSYNC_MODE_NOBARRIER,   /* fsync behaves nobarrier based on posix */
     >   };
     >
     > +enum {
     > +     COMPR_MODE_FS,          /*
     > +                              * automatically compress compression
     > +                              * enabled files
     > +                              */
     > +     COMPR_MODE_USER,        /*
     > +                              * automatical compression is disabled.
     > +                              * user can control the file compression
     > +                              * using ioctls
     > +                              */
     > +};
     > +
     >   /*
     >    * this value is set in page as a private data which indicate that
     >    * the page is atomically written, and it is in inmem_pages list.
     > @@ -2752,6 +2766,22 @@ static inline int f2fs_compressed_file(struct inode *inode)
     >               is_inode_flag_set(inode, FI_COMPRESSED_FILE);
     >   }
     >
     > +static inline bool f2fs_need_compress_data(struct inode *inode)
     > +{
     > +     int compress_mode = F2FS_OPTION(F2FS_I_SB(inode)).compress_mode;
     > +
     > +     if (!f2fs_compressed_file(inode))
     > +             return false;
     > +
     > +     if (compress_mode == COMPR_MODE_FS)
     > +             return true;
     > +     else if (compress_mode == COMPR_MODE_USER &&
     > +                     is_inode_flag_set(inode, FI_ENABLE_COMPRESS))
     > +             return true;
     > +
     > +     return false;
     > +}
     > +
     >   static inline unsigned int addrs_per_inode(struct inode *inode)
     >   {
     >       unsigned int addrs = CUR_ADDRS_PER_INODE(inode) -
     > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
     > index 1596502f7375..5d6c9d6f237b 100644
     > --- a/fs/f2fs/segment.c
     > +++ b/fs/f2fs/segment.c
     > @@ -3254,7 +3254,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
     >                       else
     >                               return CURSEG_COLD_DATA;
     >               }
     > -             if (file_is_cold(inode) || f2fs_compressed_file(inode))
     > +             if (file_is_cold(inode) || f2fs_need_compress_data(inode))

    Can we keep consistent about writting compressed file to code data area?

    Or there is other concern about this?

    Thanks,

     >                       return CURSEG_COLD_DATA;
     >               if (file_is_hot(inode) ||
     >                               is_inode_flag_set(inode, FI_HOT_DATA) ||
     > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
     > index 87f7a6e86370..cc6eb4ed80a2 100644
     > --- a/fs/f2fs/super.c
     > +++ b/fs/f2fs/super.c
     > @@ -146,6 +146,7 @@ enum {
     >       Opt_compress_algorithm,
     >       Opt_compress_log_size,
     >       Opt_compress_extension,
     > +     Opt_compress_mode,
     >       Opt_atgc,
     >       Opt_err,
     >   };
     > @@ -214,6 +215,7 @@ static match_table_t f2fs_tokens = {
     >       {Opt_compress_algorithm, "compress_algorithm=%s"},
     >       {Opt_compress_log_size, "compress_log_size=%u"},
     >       {Opt_compress_extension, "compress_extension=%s"},
     > +     {Opt_compress_mode, "compress_mode=%s"},
     >       {Opt_atgc, "atgc"},
     >       {Opt_err, NULL},
     >   };
     > @@ -934,10 +936,25 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount)
     >                       F2FS_OPTION(sbi).compress_ext_cnt++;
     >                       kfree(name);
     >                       break;
     > +             case Opt_compress_mode:
     > +                     name = match_strdup(&args[0]);
     > +                     if (!name)
     > +                             return -ENOMEM;
     > +                     if (!strcmp(name, "fs")) {
     > +                             F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
     > +                     } else if (!strcmp(name, "user")) {
     > +                             F2FS_OPTION(sbi).compress_mode = COMPR_MODE_USER;
     > +                     } else {
     > +                             kfree(name);
     > +                             return -EINVAL;
     > +                     }
     > +                     kfree(name);
     > +                     break;
     >   #else
     >               case Opt_compress_algorithm:
     >               case Opt_compress_log_size:
     >               case Opt_compress_extension:
     > +             case Opt_compress_mode:
     >                       f2fs_info(sbi, "compression options not supported");
     >                       break;
     >   #endif
     > @@ -1523,6 +1540,11 @@ static inline void f2fs_show_compress_options(struct seq_file *seq,
     >               seq_printf(seq, ",compress_extension=%s",
     >                       F2FS_OPTION(sbi).extensions[i]);
     >       }
     > +
     > +     if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_FS)
     > +             seq_printf(seq, ",compress_mode=%s", "fs");
     > +     else if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_USER)
     > +             seq_printf(seq, ",compress_mode=%s", "user");
     >   }
     >
     >   static int f2fs_show_options(struct seq_file *seq, struct dentry *root)
     > @@ -1672,6 +1694,7 @@ static void default_options(struct f2fs_sb_info *sbi)
Quoted 9 lines
     >       F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
     >       F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
     >       F2FS_OPTION(sbi).compress_ext_cnt = 0;
     > +     F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
     >       F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
     >
     >       sbi->sb->s_flags &= ~SB_INLINECRYPT;
     >

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#6 lore raw

Yep, you're right~ :)

2020년 12월 3일 (목) 오전 10:16, Chao Yu <[email protected]>님이 작성:

Quoted 267 lines · Daeho Jeong
Quoted 2 lines

We might use compress_extension=*,compress_option=user. In this option, we're gonna allocate all the writes in cold zone.

Oh, so all files in data partition will be tagged as compressed file, but the compressing time will be controlled by user, then, only blocks belong to in-compress-process inode will go to cold area, right?

Thanks,

Quoted 256 lines

2020년 12월 2일 (수) 오후 3:31, Chao Yu <[email protected] <mailto:[email protected]>>님이 작성:

    On 2020/12/1 12:08, Daeho Jeong wrote:
     > From: Daeho Jeong <[email protected] <mailto:[email protected]>>
     >
     > We will add a new "compress_mode" mount option to control file
     > compression mode. This supports "fs" and "user". In "fs" mode (default),
     > f2fs does automatic compression on the compression enabled files.
     > In "user" mode, f2fs disables the automaic compression and gives the
     > user discretion of choosing the target file and the timing. It means
     > the user can do manual compression/decompression on the compression
     > enabled files using ioctls.
     >
     > Signed-off-by: Daeho Jeong <[email protected] <mailto:[email protected]>>
     > ---
     > v2: changed mount option name and added more explanation of mount option
     > ---
     >   Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
     >   fs/f2fs/compress.c                 |  2 +-
     >   fs/f2fs/data.c                     |  2 +-
     >   fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
     >   fs/f2fs/segment.c                  |  2 +-
     >   fs/f2fs/super.c                    | 23 ++++++++++++++++++++
     >   6 files changed, 91 insertions(+), 3 deletions(-)
     >
     > diff --git a/Documentation/filesystems/f2fs.rst b/Documentation/filesystems/f2fs.rst
     > index b8ee761c9922..5eb8d63439ec 100644
     > --- a/Documentation/filesystems/f2fs.rst
     > +++ b/Documentation/filesystems/f2fs.rst
     > @@ -260,6 +260,13 @@ compress_extension=%s     Support adding specified extension, so that f2fs can enab
     >                        For other files, we can still enable compression via ioctl.
     >                        Note that, there is one reserved special extension '*', it
     >                        can be set to enable compression for all files.
     > +compress_mode=%s      Control file compression mode. This supports "fs" and "user"
     > +                      modes. In "fs" mode (default), f2fs does automatic compression
     > +                      on the compression enabled files. In "user" mode, f2fs disables
     > +                      the automaic compression and gives the user discretion of
     > +                      choosing the target file and the timing. The user can do manual
     > +                      compression/decompression on the compression enabled files using
     > +                      ioctls.
     >   inlinecrypt          When possible, encrypt/decrypt the contents of encrypted
     >                        files using the blk-crypto framework rather than
     >                        filesystem-layer encryption. This allows the use of
     > @@ -810,6 +817,34 @@ Compress metadata layout::
     >       | data length | data chksum | reserved |      compressed data       |
     >       +-------------+-------------+----------+----------------------------+
     >
     > +Compression mode
     > +--------------------------
     > +
     > +f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
     > +With this option, f2fs provides a choice to select the way how to compress the
     > +compression enabled files (refer to "Compression implementation" section for how to
     > +enable compression on a regular inode).
     > +
     > +1) compress_mode=fs
     > +This is the default option. f2fs does automatic compression in the writeback of the
     > +compression enabled files.
     > +
     > +2) compress_mode=user
     > +This disables the automaic compression and gives the user discretion of choosing the
     > +target file and the timing. The user can do manual compression/decompression on the
     > +compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
     > +ioctls like the below.
     > +
     > +To decompress a file,
     > +
     > +fd = open(filename, O_WRONLY, 0);
     > +ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
     > +
     > +To compress a file,
     > +
     > +fd = open(filename, O_WRONLY, 0);
     > +ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
     > +
     >   NVMe Zoned Namespace devices
     >   ----------------------------
     >
     > diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
     > index 3957a84a185e..87090da8693d 100644
     > --- a/fs/f2fs/compress.c
     > +++ b/fs/f2fs/compress.c
     > @@ -926,7 +926,7 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
     >
     >   static bool cluster_may_compress(struct compress_ctx *cc)
     >   {
     > -     if (!f2fs_compressed_file(cc->inode))
     > +     if (!f2fs_need_compress_data(cc->inode))
     >               return false;
     >       if (f2fs_is_atomic_file(cc->inode))
     >               return false;
     > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
     > index be4da52604ed..42254d3859c7 100644
     > --- a/fs/f2fs/data.c
     > +++ b/fs/f2fs/data.c
     > @@ -3164,7 +3164,7 @@ static inline bool __should_serialize_io(struct inode *inode,
     >       if (IS_NOQUOTA(inode))
     >               return false;
     >
     > -     if (f2fs_compressed_file(inode))
     > +     if (f2fs_need_compress_data(inode))
     >               return true;
     >       if (wbc->sync_mode != WB_SYNC_ALL)
     >               return true;
     > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
     > index e0826779a101..94d16bde5e24 100644
     > --- a/fs/f2fs/f2fs.h
     > +++ b/fs/f2fs/f2fs.h
     > @@ -149,6 +149,7 @@ struct f2fs_mount_info {
     >       unsigned char compress_algorithm;       /* algorithm type */
     >       unsigned compress_log_size;             /* cluster log size */
     >       unsigned char compress_ext_cnt;         /* extension count */
     > +     int compress_mode;                      /* compression mode */
     >       unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN]; /* extensions */
     >   };
     >
     > @@ -677,6 +678,7 @@ enum {
     >       FI_VERITY_IN_PROGRESS,  /* building fs-verity Merkle tree */
     >       FI_COMPRESSED_FILE,     /* indicate file's data can be compressed */
     >       FI_MMAP_FILE,           /* indicate file was mmapped */
     > +     FI_ENABLE_COMPRESS,     /* enable compression in "user" compression mode */
     >       FI_MAX,                 /* max flag, never be used */
     >   };
     >
     > @@ -1243,6 +1245,18 @@ enum fsync_mode {
     >       FSYNC_MODE_NOBARRIER,   /* fsync behaves nobarrier based on posix */
     >   };
     >
     > +enum {
     > +     COMPR_MODE_FS,          /*
     > +                              * automatically compress compression
     > +                              * enabled files
     > +                              */
     > +     COMPR_MODE_USER,        /*
     > +                              * automatical compression is disabled.
     > +                              * user can control the file compression
     > +                              * using ioctls
     > +                              */
     > +};
     > +
     >   /*
     >    * this value is set in page as a private data which indicate that
     >    * the page is atomically written, and it is in inmem_pages list.
     > @@ -2752,6 +2766,22 @@ static inline int f2fs_compressed_file(struct inode *inode)
     >               is_inode_flag_set(inode, FI_COMPRESSED_FILE);
     >   }
     >
     > +static inline bool f2fs_need_compress_data(struct inode *inode)
     > +{
     > +     int compress_mode = F2FS_OPTION(F2FS_I_SB(inode)).compress_mode;
     > +
     > +     if (!f2fs_compressed_file(inode))
     > +             return false;
     > +
     > +     if (compress_mode == COMPR_MODE_FS)
     > +             return true;
     > +     else if (compress_mode == COMPR_MODE_USER &&
     > +                     is_inode_flag_set(inode, FI_ENABLE_COMPRESS))
     > +             return true;
     > +
     > +     return false;
     > +}
     > +
     >   static inline unsigned int addrs_per_inode(struct inode *inode)
     >   {
     >       unsigned int addrs = CUR_ADDRS_PER_INODE(inode) -
     > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
     > index 1596502f7375..5d6c9d6f237b 100644
     > --- a/fs/f2fs/segment.c
     > +++ b/fs/f2fs/segment.c
     > @@ -3254,7 +3254,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
     >                       else
     >                               return CURSEG_COLD_DATA;
     >               }
     > -             if (file_is_cold(inode) || f2fs_compressed_file(inode))
     > +             if (file_is_cold(inode) || f2fs_need_compress_data(inode))

    Can we keep consistent about writting compressed file to code data area?

    Or there is other concern about this?

    Thanks,

     >                       return CURSEG_COLD_DATA;
     >               if (file_is_hot(inode) ||
     >                               is_inode_flag_set(inode, FI_HOT_DATA) ||
     > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
     > index 87f7a6e86370..cc6eb4ed80a2 100644
     > --- a/fs/f2fs/super.c
     > +++ b/fs/f2fs/super.c
     > @@ -146,6 +146,7 @@ enum {
     >       Opt_compress_algorithm,
     >       Opt_compress_log_size,
     >       Opt_compress_extension,
     > +     Opt_compress_mode,
     >       Opt_atgc,
     >       Opt_err,
     >   };
     > @@ -214,6 +215,7 @@ static match_table_t f2fs_tokens = {
     >       {Opt_compress_algorithm, "compress_algorithm=%s"},
     >       {Opt_compress_log_size, "compress_log_size=%u"},
     >       {Opt_compress_extension, "compress_extension=%s"},
     > +     {Opt_compress_mode, "compress_mode=%s"},
     >       {Opt_atgc, "atgc"},
     >       {Opt_err, NULL},
     >   };
     > @@ -934,10 +936,25 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount)
     >                       F2FS_OPTION(sbi).compress_ext_cnt++;
     >                       kfree(name);
     >                       break;
     > +             case Opt_compress_mode:
     > +                     name = match_strdup(&args[0]);
     > +                     if (!name)
     > +                             return -ENOMEM;
     > +                     if (!strcmp(name, "fs")) {
     > +                             F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
     > +                     } else if (!strcmp(name, "user")) {
     > +                             F2FS_OPTION(sbi).compress_mode = COMPR_MODE_USER;
     > +                     } else {
     > +                             kfree(name);
     > +                             return -EINVAL;
     > +                     }
     > +                     kfree(name);
     > +                     break;
     >   #else
     >               case Opt_compress_algorithm:
     >               case Opt_compress_log_size:
     >               case Opt_compress_extension:
     > +             case Opt_compress_mode:
     >                       f2fs_info(sbi, "compression options not supported");
     >                       break;
     >   #endif
     > @@ -1523,6 +1540,11 @@ static inline void f2fs_show_compress_options(struct seq_file *seq,
     >               seq_printf(seq, ",compress_extension=%s",
     >                       F2FS_OPTION(sbi).extensions[i]);
     >       }
     > +
     > +     if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_FS)
     > +             seq_printf(seq, ",compress_mode=%s", "fs");
     > +     else if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_USER)
     > +             seq_printf(seq, ",compress_mode=%s", "user");
     >   }
     >
     >   static int f2fs_show_options(struct seq_file *seq, struct dentry *root)
     > @@ -1672,6 +1694,7 @@ static void default_options(struct f2fs_sb_info *sbi)
     >       F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
     >       F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
     >       F2FS_OPTION(sbi).compress_ext_cnt = 0;
     > +     F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
     >       F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
     >
     >       sbi->sb->s_flags &= ~SB_INLINECRYPT;
     >

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#7 lore raw

Alright, that makes sense. :)

Quoted 270 lines · Daeho Jeong

Yep, you're right~ :)

2020년 12월 3일 (목) 오전 10:16, Chao Yu <[email protected]>님이 작성:

Quoted 267 lines · Daeho Jeong
Quoted 2 lines

We might use compress_extension=*,compress_option=user. In this option, we're gonna allocate all the writes in cold zone.

Oh, so all files in data partition will be tagged as compressed file, but the compressing time will be controlled by user, then, only blocks belong to in-compress-process inode will go to cold area, right?

Thanks,

Quoted 256 lines

2020년 12월 2일 (수) 오후 3:31, Chao Yu <[email protected] <mailto:[email protected]>>님이 작성:

     On 2020/12/1 12:08, Daeho Jeong wrote:
      > From: Daeho Jeong <[email protected] <mailto:[email protected]>>
      >
      > We will add a new "compress_mode" mount option to control file
      > compression mode. This supports "fs" and "user". In "fs" mode (default),
      > f2fs does automatic compression on the compression enabled files.
      > In "user" mode, f2fs disables the automaic compression and gives the
      > user discretion of choosing the target file and the timing. It means
      > the user can do manual compression/decompression on the compression
      > enabled files using ioctls.
      >
      > Signed-off-by: Daeho Jeong <[email protected] <mailto:[email protected]>>
      > ---
      > v2: changed mount option name and added more explanation of mount option
      > ---
      >   Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
      >   fs/f2fs/compress.c                 |  2 +-
      >   fs/f2fs/data.c                     |  2 +-
      >   fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
      >   fs/f2fs/segment.c                  |  2 +-
      >   fs/f2fs/super.c                    | 23 ++++++++++++++++++++
      >   6 files changed, 91 insertions(+), 3 deletions(-)
      >
      > diff --git a/Documentation/filesystems/f2fs.rst b/Documentation/filesystems/f2fs.rst
      > index b8ee761c9922..5eb8d63439ec 100644
      > --- a/Documentation/filesystems/f2fs.rst
      > +++ b/Documentation/filesystems/f2fs.rst
      > @@ -260,6 +260,13 @@ compress_extension=%s     Support adding specified extension, so that f2fs can enab
      >                        For other files, we can still enable compression via ioctl.
      >                        Note that, there is one reserved special extension '*', it
      >                        can be set to enable compression for all files.
      > +compress_mode=%s      Control file compression mode. This supports "fs" and "user"
      > +                      modes. In "fs" mode (default), f2fs does automatic compression
      > +                      on the compression enabled files. In "user" mode, f2fs disables
      > +                      the automaic compression and gives the user discretion of
      > +                      choosing the target file and the timing. The user can do manual
      > +                      compression/decompression on the compression enabled files using
      > +                      ioctls.
      >   inlinecrypt          When possible, encrypt/decrypt the contents of encrypted
      >                        files using the blk-crypto framework rather than
      >                        filesystem-layer encryption. This allows the use of
      > @@ -810,6 +817,34 @@ Compress metadata layout::
      >       | data length | data chksum | reserved |      compressed data       |
      >       +-------------+-------------+----------+----------------------------+
      >
      > +Compression mode
      > +--------------------------
      > +
      > +f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
      > +With this option, f2fs provides a choice to select the way how to compress the
      > +compression enabled files (refer to "Compression implementation" section for how to
      > +enable compression on a regular inode).
      > +
      > +1) compress_mode=fs
      > +This is the default option. f2fs does automatic compression in the writeback of the
      > +compression enabled files.
      > +
      > +2) compress_mode=user
      > +This disables the automaic compression and gives the user discretion of choosing the
      > +target file and the timing. The user can do manual compression/decompression on the
      > +compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
      > +ioctls like the below.
      > +
      > +To decompress a file,
      > +
      > +fd = open(filename, O_WRONLY, 0);
      > +ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
      > +
      > +To compress a file,
      > +
      > +fd = open(filename, O_WRONLY, 0);
      > +ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
      > +
      >   NVMe Zoned Namespace devices
      >   ----------------------------
      >
      > diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
      > index 3957a84a185e..87090da8693d 100644
      > --- a/fs/f2fs/compress.c
      > +++ b/fs/f2fs/compress.c
      > @@ -926,7 +926,7 @@ int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
      >
      >   static bool cluster_may_compress(struct compress_ctx *cc)
      >   {
      > -     if (!f2fs_compressed_file(cc->inode))
      > +     if (!f2fs_need_compress_data(cc->inode))
      >               return false;
      >       if (f2fs_is_atomic_file(cc->inode))
      >               return false;
      > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
      > index be4da52604ed..42254d3859c7 100644
      > --- a/fs/f2fs/data.c
      > +++ b/fs/f2fs/data.c
      > @@ -3164,7 +3164,7 @@ static inline bool __should_serialize_io(struct inode *inode,
      >       if (IS_NOQUOTA(inode))
      >               return false;
      >
      > -     if (f2fs_compressed_file(inode))
      > +     if (f2fs_need_compress_data(inode))
      >               return true;
      >       if (wbc->sync_mode != WB_SYNC_ALL)
      >               return true;
      > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
      > index e0826779a101..94d16bde5e24 100644
      > --- a/fs/f2fs/f2fs.h
      > +++ b/fs/f2fs/f2fs.h
      > @@ -149,6 +149,7 @@ struct f2fs_mount_info {
      >       unsigned char compress_algorithm;       /* algorithm type */
      >       unsigned compress_log_size;             /* cluster log size */
      >       unsigned char compress_ext_cnt;         /* extension count */
      > +     int compress_mode;                      /* compression mode */
      >       unsigned char extensions[COMPRESS_EXT_NUM][F2FS_EXTENSION_LEN]; /* extensions */
      >   };
      >
      > @@ -677,6 +678,7 @@ enum {
      >       FI_VERITY_IN_PROGRESS,  /* building fs-verity Merkle tree */
      >       FI_COMPRESSED_FILE,     /* indicate file's data can be compressed */
      >       FI_MMAP_FILE,           /* indicate file was mmapped */
      > +     FI_ENABLE_COMPRESS,     /* enable compression in "user" compression mode */
      >       FI_MAX,                 /* max flag, never be used */
      >   };
      >
      > @@ -1243,6 +1245,18 @@ enum fsync_mode {
      >       FSYNC_MODE_NOBARRIER,   /* fsync behaves nobarrier based on posix */
      >   };
      >
      > +enum {
      > +     COMPR_MODE_FS,          /*
      > +                              * automatically compress compression
      > +                              * enabled files
      > +                              */
      > +     COMPR_MODE_USER,        /*
      > +                              * automatical compression is disabled.
      > +                              * user can control the file compression
      > +                              * using ioctls
      > +                              */
      > +};
      > +
      >   /*
      >    * this value is set in page as a private data which indicate that
      >    * the page is atomically written, and it is in inmem_pages list.
      > @@ -2752,6 +2766,22 @@ static inline int f2fs_compressed_file(struct inode *inode)
      >               is_inode_flag_set(inode, FI_COMPRESSED_FILE);
      >   }
      >
      > +static inline bool f2fs_need_compress_data(struct inode *inode)
      > +{
      > +     int compress_mode = F2FS_OPTION(F2FS_I_SB(inode)).compress_mode;
      > +
      > +     if (!f2fs_compressed_file(inode))
      > +             return false;
      > +
      > +     if (compress_mode == COMPR_MODE_FS)
      > +             return true;
      > +     else if (compress_mode == COMPR_MODE_USER &&
      > +                     is_inode_flag_set(inode, FI_ENABLE_COMPRESS))
      > +             return true;
      > +
      > +     return false;
      > +}
      > +
      >   static inline unsigned int addrs_per_inode(struct inode *inode)
      >   {
      >       unsigned int addrs = CUR_ADDRS_PER_INODE(inode) -
      > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
      > index 1596502f7375..5d6c9d6f237b 100644
      > --- a/fs/f2fs/segment.c
      > +++ b/fs/f2fs/segment.c
      > @@ -3254,7 +3254,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
      >                       else
      >                               return CURSEG_COLD_DATA;
      >               }
      > -             if (file_is_cold(inode) || f2fs_compressed_file(inode))
      > +             if (file_is_cold(inode) || f2fs_need_compress_data(inode))

     Can we keep consistent about writting compressed file to code data area?

     Or there is other concern about this?

     Thanks,

      >                       return CURSEG_COLD_DATA;
      >               if (file_is_hot(inode) ||
      >                               is_inode_flag_set(inode, FI_HOT_DATA) ||
      > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
      > index 87f7a6e86370..cc6eb4ed80a2 100644
      > --- a/fs/f2fs/super.c
      > +++ b/fs/f2fs/super.c
      > @@ -146,6 +146,7 @@ enum {
      >       Opt_compress_algorithm,
      >       Opt_compress_log_size,
      >       Opt_compress_extension,
      > +     Opt_compress_mode,
      >       Opt_atgc,
      >       Opt_err,
      >   };
      > @@ -214,6 +215,7 @@ static match_table_t f2fs_tokens = {
      >       {Opt_compress_algorithm, "compress_algorithm=%s"},
      >       {Opt_compress_log_size, "compress_log_size=%u"},
      >       {Opt_compress_extension, "compress_extension=%s"},
      > +     {Opt_compress_mode, "compress_mode=%s"},
      >       {Opt_atgc, "atgc"},
      >       {Opt_err, NULL},
      >   };
      > @@ -934,10 +936,25 @@ static int parse_options(struct super_block *sb, char *options, bool is_remount)
      >                       F2FS_OPTION(sbi).compress_ext_cnt++;
      >                       kfree(name);
      >                       break;
      > +             case Opt_compress_mode:
      > +                     name = match_strdup(&args[0]);
      > +                     if (!name)
      > +                             return -ENOMEM;
      > +                     if (!strcmp(name, "fs")) {
      > +                             F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
      > +                     } else if (!strcmp(name, "user")) {
      > +                             F2FS_OPTION(sbi).compress_mode = COMPR_MODE_USER;
      > +                     } else {
      > +                             kfree(name);
      > +                             return -EINVAL;
      > +                     }
      > +                     kfree(name);
      > +                     break;
      >   #else
      >               case Opt_compress_algorithm:
      >               case Opt_compress_log_size:
      >               case Opt_compress_extension:
      > +             case Opt_compress_mode:
      >                       f2fs_info(sbi, "compression options not supported");
      >                       break;
      >   #endif
      > @@ -1523,6 +1540,11 @@ static inline void f2fs_show_compress_options(struct seq_file *seq,
      >               seq_printf(seq, ",compress_extension=%s",
      >                       F2FS_OPTION(sbi).extensions[i]);
      >       }
      > +
      > +     if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_FS)
      > +             seq_printf(seq, ",compress_mode=%s", "fs");
      > +     else if (F2FS_OPTION(sbi).compress_mode == COMPR_MODE_USER)
      > +             seq_printf(seq, ",compress_mode=%s", "user");
      >   }
      >
      >   static int f2fs_show_options(struct seq_file *seq, struct dentry *root)
      > @@ -1672,6 +1694,7 @@ static void default_options(struct f2fs_sb_info *sbi)
      >       F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
      >       F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
      >       F2FS_OPTION(sbi).compress_ext_cnt = 0;
      > +     F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
      >       F2FS_OPTION(sbi).bggc_mode = BGGC_MODE_ON;
      >
      >       sbi->sb->s_flags &= ~SB_INLINECRYPT;
      >
Quoted 2 lines

.

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by on v2 1/2
#8 lore raw
In reply to Daeho Jeong
Quoted 11 lines · Daeho Jeong

From: Daeho Jeong <[email protected]>

We will add a new "compress_mode" mount option to control file compression mode. This supports "fs" and "user". In "fs" mode (default), f2fs does automatic compression on the compression enabled files. In "user" mode, f2fs disables the automaic compression and gives the user discretion of choosing the target file and the timing. It means the user can do manual compression/decompression on the compression enabled files using ioctls.

Thanks,

#9 lore raw
In reply to Daeho Jeong
Quoted 62 lines · Daeho Jeong

From: Daeho Jeong <[email protected]>

We will add a new "compress_mode" mount option to control file compression mode. This supports "fs" and "user". In "fs" mode (default), f2fs does automatic compression on the compression enabled files. In "user" mode, f2fs disables the automaic compression and gives the user discretion of choosing the target file and the timing. It means the user can do manual compression/decompression on the compression enabled files using ioctls.

---
v2: changed mount option name and added more explanation of mount option

---
 Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
 fs/f2fs/compress.c                 |  2 +-
 fs/f2fs/data.c                     |  2 +-
 fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
 fs/f2fs/segment.c                  |  2 +-
 fs/f2fs/super.c                    | 23 ++++++++++++++++++++
 6 files changed, 91 insertions(+), 3 deletions(-)
Documentation/filesystems/f2fs.rst +25 -0
index b8ee761c9922..5eb8d63439ec 100644
@@ -260,6 +260,13 @@ compress_extension=%s	 Support adding specified extension, so that f2fs can enab
 			 For other files, we can still enable compression via ioctl.
 			 Note that, there is one reserved special extension '*', it
 			 can be set to enable compression for all files.
+compress_mode=%s	 Control file compression mode. This supports "fs" and "user"
+			 modes. In "fs" mode (default), f2fs does automatic compression
+			 on the compression enabled files. In "user" mode, f2fs disables
+			 the automaic compression and gives the user discretion of
+			 choosing the target file and the timing. The user can do manual
+			 compression/decompression on the compression enabled files using
+			 ioctls.
 inlinecrypt		 When possible, encrypt/decrypt the contents of encrypted
 			 files using the blk-crypto framework rather than
 			 filesystem-layer encryption. This allows the use of
@@ -810,6 +817,34 @@ Compress metadata layout::
 	| data length | data chksum | reserved |      compressed data       |
 	+-------------+-------------+----------+----------------------------+
 
+Compression mode
+--------------------------
+
+f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
+With this option, f2fs provides a choice to select the way how to compress the
+compression enabled files (refer to "Compression implementation" section for how to
+enable compression on a regular inode).
+
+1) compress_mode=fs
+This is the default option. f2fs does automatic compression in the writeback of the
+compression enabled files.
+
+2) compress_mode=user
+This disables the automaic compression and gives the user discretion of choosing the
+target file and the timing. The user can do manual compression/decompression on the
+compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
+ioctls like the below.
+

Documentation/filesystems/f2fs.rst Compress metadata layout:: · line 847

Quoted 10 lines
Documentation/filesystems/f2fs.rst +10 -0
+To decompress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
+
+To compress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
+

Why doesn't compress_mode=user just cause regular files to not inherit the compression flag? Then users could set or clear the compression flag using FS_IOC_SETFLAGS, without any need for these new ioctls.

- Eric

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#10 lore raw

compression enabled regular files have different formats of node metadata on disk. So, using the "compress_mode=user,compress_extension=*" mount option, we want to make the metadata of files ready for compression and make them compressed whenever the user wants using new ioctls.

2020년 12월 8일 (화) 오후 12:51, Eric Biggers <[email protected]>님이 작성:

Quoted 80 lines · Daeho Jeong
Quoted 62 lines

From: Daeho Jeong <[email protected]>

We will add a new "compress_mode" mount option to control file compression mode. This supports "fs" and "user". In "fs" mode (default), f2fs does automatic compression on the compression enabled files. In "user" mode, f2fs disables the automaic compression and gives the user discretion of choosing the target file and the timing. It means the user can do manual compression/decompression on the compression enabled files using ioctls.

---
v2: changed mount option name and added more explanation of mount option

---
 Documentation/filesystems/f2fs.rst | 35 ++++++++++++++++++++++++++++++
 fs/f2fs/compress.c                 |  2 +-
 fs/f2fs/data.c                     |  2 +-
 fs/f2fs/f2fs.h                     | 30 +++++++++++++++++++++++++
 fs/f2fs/segment.c                  |  2 +-
 fs/f2fs/super.c                    | 23 ++++++++++++++++++++
 6 files changed, 91 insertions(+), 3 deletions(-)
Documentation/filesystems/f2fs.rst +25 -0
index b8ee761c9922..5eb8d63439ec 100644
@@ -260,6 +260,13 @@ compress_extension=%s     Support adding specified extension, so that f2fs can enab
                       For other files, we can still enable compression via ioctl.
                       Note that, there is one reserved special extension '*', it
                       can be set to enable compression for all files.
+compress_mode=%s      Control file compression mode. This supports "fs" and "user"
+                      modes. In "fs" mode (default), f2fs does automatic compression
+                      on the compression enabled files. In "user" mode, f2fs disables
+                      the automaic compression and gives the user discretion of
+                      choosing the target file and the timing. The user can do manual
+                      compression/decompression on the compression enabled files using
+                      ioctls.
 inlinecrypt           When possible, encrypt/decrypt the contents of encrypted
                       files using the blk-crypto framework rather than
                       filesystem-layer encryption. This allows the use of
@@ -810,6 +817,34 @@ Compress metadata layout::
      | data length | data chksum | reserved |      compressed data       |
      +-------------+-------------+----------+----------------------------+
 
+Compression mode
+--------------------------
+
+f2fs supports "fs" and "user" compression modes with "compression_mode" mount option.
+With this option, f2fs provides a choice to select the way how to compress the
+compression enabled files (refer to "Compression implementation" section for how to
+enable compression on a regular inode).
+
+1) compress_mode=fs
+This is the default option. f2fs does automatic compression in the writeback of the
+compression enabled files.
+
+2) compress_mode=user
+This disables the automaic compression and gives the user discretion of choosing the
+target file and the timing. The user can do manual compression/decompression on the
+compression enabled files using F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE
+ioctls like the below.
+

Documentation/filesystems/f2fs.rst Compress metadata layout:: · line 847

Quoted 10 lines
Documentation/filesystems/f2fs.rst +10 -0
+To decompress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_DECOMPRESS_FILE);
+
+To compress a file,
+
+fd = open(filename, O_WRONLY, 0);
+ret = ioctl(fd, F2FS_IOC_COMPRESS_FILE);
+

Why doesn't compress_mode=user just cause regular files to not inherit the compression flag? Then users could set or clear the compression flag using FS_IOC_SETFLAGS, without any need for these new ioctls.

- Eric