shmem: Add user and group quota support for tmpfs

PATCH Superseded [email protected] 2 participants 10 messages Apr 26, 2023 – May 17, 2023

Subsystems (5)

Memory management Documentation VFS tmpfs (shmem filesystem) +1 more subsystems

lore

Share & tools

lore

[email protected]

author

2023-04-26 10:20 UTC #0 lore raw

v4 cover shmem: Add user and group quota support for tmpfs

From: Carlos Maiolino <[email protected]>

Hello folks.

This is the final version of the quota support from tmpfs, with all the issues addressed, and now including RwB tags on all patches, and should be ready for merge. Details are within each patch, and the original cover-letter below.

Hi folks. this work has been done originally by Lukas, but he left the company, so I'm taking over his work from where he left it of. This series is virtually done, and he had updated it with comments from the last version, but, I'm initially posting it as a RFC because it's been a while since he posted the last version.
Most of what I did here was rebase his last work on top of current Linus's tree.

Honza, there is one patch from you in this series, which I believe you had it suggested to Lukas on a previous version.

The original cover-letter follows...

people have been asking for quota support in tmpfs many times in the past mostly to avoid one malicious user, or misbehaving user/program to consume all of the system memory. This has been partially solved with the size mount option, but some problems still prevail.

One of the problems is the fact that /dev/shm is still generally unprotected with this and another is administration overhead of managing multiple tmpfs mounts and lack of more fine grained control.

Quota support can solve all these problems in a somewhat standard way people are already familiar with from regular file systems. It can give us more fine grained control over how much memory user/groups can consume. Additionally it can also control number of inodes and with special quota mount options introduced with a second patch we can set global limits allowing us to replace the size mount option with quota entirely.

Currently the standard userspace quota tools (quota, xfs_quota) are only using quotactl ioctl which is expecting a block device. I patched quota [1] and xfs_quota [2] to use quotactl_fd in case we want to run the tools on mount point directory to work nicely with tmpfs.

The implementation was tested on patched version of xfstests [3].

[1] https://github.com/lczerner/quota/tree/quotactl_fd_support
[2] https://github.com/lczerner/xfsprogs/tree/quotactl_fd_support
[3] https://github.com/lczerner/xfstests/tree/tmpfs_quota_support

Jan Kara (1):
quota: Check presence of quota operation structures instead of

    ->quota_read and ->quota_write callbacks

Lukas Czerner (5):
shmem: make shmem_inode_acct_block() return error
shmem: make shmem_get_inode() return ERR_PTR instead of NULL
shmem: prepare shmem quota infrastructure
shmem: quota support
Add default quota limit mount options

 Documentation/filesystems/tmpfs.rst |  31 ++
 fs/Kconfig                          |  12 +
 fs/quota/dquot.c                    |   2 +-
 include/linux/shmem_fs.h            |  28 ++
 include/uapi/linux/quota.h          |   1 +
 mm/Makefile                         |   2 +-
 mm/shmem.c                          | 465 +++++++++++++++++++++-------
 mm/shmem_quota.c                    | 350 +++++++++++++++++++++
 8 files changed, 783 insertions(+), 108 deletions(-)
 create mode 100644 mm/shmem_quota.c

Signed-off-by Carlos Maiolino <[email protected]>

Signature · 1 line

2.30.2

[email protected]

author

2023-04-26 10:20 UTC #1 lore raw

PATCH v4 1/6 shmem: make shmem_inode_acct_block() return error

From: Lukas Czerner <[email protected]>

Make shmem_inode_acct_block() return proper error code instead of bool. This will be useful later when we introduce quota support.

There should be no functional change.

Signed-off-by Lukas Czerner <[email protected]>

Signed-off-by Carlos Maiolino <[email protected]>

Reviewed-by Jan Kara <[email protected]>

---
 mm/shmem.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

mm/shmem.c +10 -8

index 448f393d8ab2b..b1b3b826f6189 100644
@@ -198,13 +198,14 @@ static inline void shmem_unacct_blocks(unsigned long flags, long pages)
 		vm_unacct_memory(pages * VM_ACCT(PAGE_SIZE));
 }
 
-static inline bool shmem_inode_acct_block(struct inode *inode, long pages)
+static inline int shmem_inode_acct_block(struct inode *inode, long pages)
 {
 	struct shmem_inode_info *info = SHMEM_I(inode);
 	struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
+	int err = -ENOSPC;
 
 	if (shmem_acct_block(info->flags, pages))
-		return false;
+		return err;
 
 	if (sbinfo->max_blocks) {
 		if (percpu_counter_compare(&sbinfo->used_blocks,

@@ -213,11 +214,11 @@ static inline bool shmem_inode_acct_block(struct inode *inode, long pages)
 		percpu_counter_add(&sbinfo->used_blocks, pages);
 	}
 
-	return true;
+	return 0;
 
 unacct:
 	shmem_unacct_blocks(info->flags, pages);
-	return false;
+	return err;
 }
 
 static inline void shmem_inode_unacct_blocks(struct inode *inode, long pages)

@@ -369,7 +370,7 @@ bool shmem_charge(struct inode *inode, long pages)
 	struct shmem_inode_info *info = SHMEM_I(inode);
 	unsigned long flags;
 
-	if (!shmem_inode_acct_block(inode, pages))
+	if (shmem_inode_acct_block(inode, pages))
 		return false;
 
 	/* nrpages adjustment first, then shmem_recalc_inode() when balanced */

@@ -1583,13 +1584,14 @@ static struct folio *shmem_alloc_and_acct_folio(gfp_t gfp, struct inode *inode,
 	struct shmem_inode_info *info = SHMEM_I(inode);
 	struct folio *folio;
 	int nr;
-	int err = -ENOSPC;
+	int err;
 
 	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
 		huge = false;
 	nr = huge ? HPAGE_PMD_NR : 1;
 
-	if (!shmem_inode_acct_block(inode, nr))
+	err = shmem_inode_acct_block(inode, nr);
+	if (err)
 		goto failed;
 
 	if (huge)

@@ -2433,7 +2435,7 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,
 	int ret;
 	pgoff_t max_off;
 
-	if (!shmem_inode_acct_block(inode, 1)) {
+	if (shmem_inode_acct_block(inode, 1)) {
 		/*
 		 * We may have got a page, returned -ENOENT triggering a retry,
 		 * and now we find ourselves with -ENOMEM. Release the page, to

Signature · 1 line

2.30.2

[email protected]

author

2023-04-26 10:20 UTC #2 lore raw

PATCH v4 2/6 shmem: make shmem_get_inode() return ERR_PTR instead of NULL

In reply to [email protected]

From: Lukas Czerner <[email protected]>

Make shmem_get_inode() return ERR_PTR instead of NULL on error. This will be useful later when we introduce quota support.

There should be no functional change.

Signed-off-by Lukas Czerner <[email protected]>

Signed-off-by Carlos Maiolino <[email protected]>

Reviewed-by Jan Kara <[email protected]>

---
 mm/shmem.c | 206 ++++++++++++++++++++++++++++++-----------------------
 1 file changed, 116 insertions(+), 90 deletions(-)

mm/shmem.c +116 ~~-90~~

index b1b3b826f6189..09a44f1b2c954 100644
@@ -2354,65 +2354,71 @@ static struct inode *shmem_get_inode(struct mnt_idmap *idmap, struct super_block
 	struct shmem_inode_info *info;
 	struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
 	ino_t ino;
+	int err;
+
+	err = shmem_reserve_inode(sb, &ino);
+	if (err)
+		return ERR_PTR(err);
 
-	if (shmem_reserve_inode(sb, &ino))
-		return NULL;
 
 	inode = new_inode(sb);
-	if (inode) {
-		inode->i_ino = ino;
-		inode_init_owner(idmap, inode, dir, mode);
-		inode->i_blocks = 0;
-		inode->i_atime = inode->i_mtime = inode->i_ctime = current_time(inode);
-		inode->i_generation = get_random_u32();
-		info = SHMEM_I(inode);
-		memset(info, 0, (char *)inode - (char *)info);
-		spin_lock_init(&info->lock);
-		atomic_set(&info->stop_eviction, 0);
-		info->seals = F_SEAL_SEAL;
-		info->flags = flags & VM_NORESERVE;
-		info->i_crtime = inode->i_mtime;
-		info->fsflags = (dir == NULL) ? 0 :
-			SHMEM_I(dir)->fsflags & SHMEM_FL_INHERITED;
-		if (info->fsflags)
-			shmem_set_inode_flags(inode, info->fsflags);
-		INIT_LIST_HEAD(&info->shrinklist);
-		INIT_LIST_HEAD(&info->swaplist);
-		simple_xattrs_init(&info->xattrs);
-		cache_no_acl(inode);
-		mapping_set_large_folios(inode->i_mapping);
-
-		switch (mode & S_IFMT) {
-		default:
-			inode->i_op = &shmem_special_inode_operations;
-			init_special_inode(inode, mode, dev);
-			break;
-		case S_IFREG:
-			inode->i_mapping->a_ops = &shmem_aops;
-			inode->i_op = &shmem_inode_operations;
-			inode->i_fop = &shmem_file_operations;
-			mpol_shared_policy_init(&info->policy,
-						 shmem_get_sbmpol(sbinfo));
-			break;
-		case S_IFDIR:
-			inc_nlink(inode);
-			/* Some things misbehave if size == 0 on a directory */
-			inode->i_size = 2 * BOGO_DIRENT_SIZE;
-			inode->i_op = &shmem_dir_inode_operations;
-			inode->i_fop = &simple_dir_operations;
-			break;
-		case S_IFLNK:
-			/*
-			 * Must not load anything in the rbtree,
-			 * mpol_free_shared_policy will not be called.
-			 */
-			mpol_shared_policy_init(&info->policy, NULL);
-			break;
-		}
 
-		lockdep_annotate_inode_mutex_key(inode);
-	} else
+	if (!inode) {
 		shmem_free_inode(sb);
+		return ERR_PTR(-ENOSPC);
+	}
+
+	inode->i_ino = ino;
+	inode_init_owner(idmap, inode, dir, mode);
+	inode->i_blocks = 0;
+	inode->i_atime = inode->i_mtime = inode->i_ctime = current_time(inode);
+	inode->i_generation = get_random_u32();
+	info = SHMEM_I(inode);
+	memset(info, 0, (char *)inode - (char *)info);
+	spin_lock_init(&info->lock);
+	atomic_set(&info->stop_eviction, 0);
+	info->seals = F_SEAL_SEAL;
+	info->flags = flags & VM_NORESERVE;
+	info->i_crtime = inode->i_mtime;
+	info->fsflags = (dir == NULL) ? 0 :
+		SHMEM_I(dir)->fsflags & SHMEM_FL_INHERITED;
+	if (info->fsflags)
+		shmem_set_inode_flags(inode, info->fsflags);
+	INIT_LIST_HEAD(&info->shrinklist);
+	INIT_LIST_HEAD(&info->swaplist);
+	simple_xattrs_init(&info->xattrs);
+	cache_no_acl(inode);
+	mapping_set_large_folios(inode->i_mapping);
+
+	switch (mode & S_IFMT) {
+	default:
+		inode->i_op = &shmem_special_inode_operations;
+		init_special_inode(inode, mode, dev);
+		break;
+	case S_IFREG:
+		inode->i_mapping->a_ops = &shmem_aops;
+		inode->i_op = &shmem_inode_operations;
+		inode->i_fop = &shmem_file_operations;
+		mpol_shared_policy_init(&info->policy,
+					 shmem_get_sbmpol(sbinfo));
+		break;
+	case S_IFDIR:
+		inc_nlink(inode);
+		/* Some things misbehave if size == 0 on a directory */
+		inode->i_size = 2 * BOGO_DIRENT_SIZE;
+		inode->i_op = &shmem_dir_inode_operations;
+		inode->i_fop = &simple_dir_operations;
+		break;
+	case S_IFLNK:
+		/*
+		 * Must not load anything in the rbtree,
+		 * mpol_free_shared_policy will not be called.
+		 */
+		mpol_shared_policy_init(&info->policy, NULL);
+		break;
+	}
+
+	lockdep_annotate_inode_mutex_key(inode);
 	return inode;
 }

@@ -2927,27 +2933,30 @@ shmem_mknod(struct mnt_idmap *idmap, struct inode *dir,
 	    struct dentry *dentry, umode_t mode, dev_t dev)
 {
 	struct inode *inode;
-	int error = -ENOSPC;
+	int error;
 
 	inode = shmem_get_inode(idmap, dir->i_sb, dir, mode, dev, VM_NORESERVE);
-	if (inode) {
-		error = simple_acl_create(dir, inode);
-		if (error)
-			goto out_iput;
-		error = security_inode_init_security(inode, dir,
-						     &dentry->d_name,
-						     shmem_initxattrs, NULL);
-		if (error && error != -EOPNOTSUPP)
-			goto out_iput;
 
-		error = 0;
-		dir->i_size += BOGO_DIRENT_SIZE;
-		dir->i_ctime = dir->i_mtime = current_time(dir);
-		inode_inc_iversion(dir);
-		d_instantiate(dentry, inode);
-		dget(dentry); /* Extra count - pin the dentry in core */
-	}
+	if (IS_ERR(inode))
+		return PTR_ERR(inode);
+
+	error = simple_acl_create(dir, inode);
+	if (error)
+		goto out_iput;
+	error = security_inode_init_security(inode, dir,
+					     &dentry->d_name,
+					     shmem_initxattrs, NULL);
+	if (error && error != -EOPNOTSUPP)
+		goto out_iput;
+
+	error = 0;
+	dir->i_size += BOGO_DIRENT_SIZE;
+	dir->i_ctime = dir->i_mtime = current_time(dir);
+	inode_inc_iversion(dir);
+	d_instantiate(dentry, inode);
+	dget(dentry); /* Extra count - pin the dentry in core */
 	return error;
+
 out_iput:
 	iput(inode);
 	return error;

@@ -2958,20 +2967,26 @@ shmem_tmpfile(struct mnt_idmap *idmap, struct inode *dir,
 	      struct file *file, umode_t mode)
 {
 	struct inode *inode;
-	int error = -ENOSPC;
+	int error;
 
 	inode = shmem_get_inode(idmap, dir->i_sb, dir, mode, 0, VM_NORESERVE);
-	if (inode) {
-		error = security_inode_init_security(inode, dir,
-						     NULL,
-						     shmem_initxattrs, NULL);
-		if (error && error != -EOPNOTSUPP)
-			goto out_iput;
-		error = simple_acl_create(dir, inode);
-		if (error)
-			goto out_iput;
-		d_tmpfile(file, inode);
+
+	if (IS_ERR(inode)) {
+		error = PTR_ERR(inode);
+		goto err_out;
 	}
+
+	error = security_inode_init_security(inode, dir,
+					     NULL,
+					     shmem_initxattrs, NULL);
+	if (error && error != -EOPNOTSUPP)
+		goto out_iput;
+	error = simple_acl_create(dir, inode);
+	if (error)
+		goto out_iput;
+	d_tmpfile(file, inode);
+
+err_out:
 	return finish_open_simple(file, error);
 out_iput:
 	iput(inode);

@@ -3146,8 +3161,9 @@ static int shmem_symlink(struct mnt_idmap *idmap, struct inode *dir,
 
 	inode = shmem_get_inode(idmap, dir->i_sb, dir, S_IFLNK | 0777, 0,
 				VM_NORESERVE);
-	if (!inode)
-		return -ENOSPC;
+
+	if (IS_ERR(inode))
+		return PTR_ERR(inode);
 
 	error = security_inode_init_security(inode, dir, &dentry->d_name,
 					     shmem_initxattrs, NULL);

@@ -3762,12 +3778,13 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc)
 	struct shmem_options *ctx = fc->fs_private;
 	struct inode *inode;
 	struct shmem_sb_info *sbinfo;
+	int error = -ENOMEM;
 
 	/* Round up to L1_CACHE_BYTES to resist false sharing */
 	sbinfo = kzalloc(max((int)sizeof(struct shmem_sb_info),
 				L1_CACHE_BYTES), GFP_KERNEL);
 	if (!sbinfo)
-		return -ENOMEM;
+		return error;
 
 	sb->s_fs_info = sbinfo;

@@ -3829,8 +3846,10 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc)
 
 	inode = shmem_get_inode(&nop_mnt_idmap, sb, NULL, S_IFDIR | sbinfo->mode, 0,
 				VM_NORESERVE);
-	if (!inode)
+	if (IS_ERR(inode)) {
+		error = PTR_ERR(inode);
 		goto failed;
+	}
 	inode->i_uid = sbinfo->uid;
 	inode->i_gid = sbinfo->gid;
 	sb->s_root = d_make_root(inode);

@@ -3840,7 +3859,7 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc)
 
 failed:
 	shmem_put_super(sb);
-	return -ENOMEM;
+	return error;
 }
 
 static int shmem_get_tree(struct fs_context *fc)

@@ -4209,10 +4228,16 @@ EXPORT_SYMBOL_GPL(shmem_truncate_range);
 #define shmem_vm_ops				generic_file_vm_ops
 #define shmem_anon_vm_ops			generic_file_vm_ops
 #define shmem_file_operations			ramfs_file_operations
-#define shmem_get_inode(idmap, sb, dir, mode, dev, flags) ramfs_get_inode(sb, dir, mode, dev)
 #define shmem_acct_size(flags, size)		0
 #define shmem_unacct_size(flags, size)		do {} while (0)
 
+static inline struct inode *shmem_get_inode(struct mnt_idmap *idmap, struct super_block *sb, struct inode *dir,
+					    umode_t mode, dev_t dev, unsigned long flags)
+{
+	struct inode *inode = ramfs_get_inode(sb, dir, mode, dev);
+	return inode ? inode : ERR_PTR(-ENOSPC);
+}
+
 #endif /* CONFIG_SHMEM */
 
 /* common code */

@@ -4237,9 +4262,10 @@ static struct file *__shmem_file_setup(struct vfsmount *mnt, const char *name, l
 
 	inode = shmem_get_inode(&nop_mnt_idmap, mnt->mnt_sb, NULL,
 				S_IFREG | S_IRWXUGO, 0, flags);
-	if (unlikely(!inode)) {
+
+	if (IS_ERR(inode)) {
 		shmem_unacct_size(flags, size);
-		return ERR_PTR(-ENOSPC);
+		return ERR_CAST(inode);
 	}
 	inode->i_flags |= i_flags;
 	inode->i_size = size;

Signature · 1 line

2.30.2

[email protected]

author

2023-04-26 10:20 UTC #3 lore raw

PATCH v4 3/6 quota: Check presence of quota operation structures instead of ->quota_read and ->quota_write callbacks

In reply to [email protected]

From: Jan Kara <[email protected]>

Currently we check whether superblock has ->quota_read and ->quota_write operations to check whether filesystem supports quotas. However for example for shmfs we will not read or write dquots so check whether quota operations are set in the superblock instead.

Signed-off-by Jan Kara <[email protected]>

Reviewed-by Carlos Maiolino <[email protected]>

---
 fs/quota/dquot.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

fs/quota/dquot.c +1 -1

index a6357f728034a..81563a83b609d 100644
@@ -2367,7 +2367,7 @@ int dquot_load_quota_sb(struct super_block *sb, int type, int format_id,
 
 	if (!fmt)
 		return -ESRCH;
-	if (!sb->s_op->quota_write || !sb->s_op->quota_read ||
+	if (!sb->dq_op || !sb->s_qcop ||
 	    (type == PRJQUOTA && sb->dq_op->get_projid == NULL)) {
 		error = -EINVAL;
 		goto out_fmt;

Signature · 1 line

2.30.2

[email protected]

author

2023-04-26 10:20 UTC #4 lore raw

PATCH v4 4/6 shmem: prepare shmem quota infrastructure

In reply to [email protected]

From: Lukas Czerner <[email protected]>

Add new shmem quota format, its quota_format_ops together with dquot_operations

Signed-off-by Lukas Czerner <[email protected]>

Signed-off-by Carlos Maiolino <[email protected]>

Reviewed-by Jan Kara <[email protected]>

---
 fs/Kconfig                 |  12 ++
 include/linux/shmem_fs.h   |  12 ++
 include/uapi/linux/quota.h |   1 +
 mm/Makefile                |   2 +-
 mm/shmem_quota.c           | 318 +++++++++++++++++++++++++++++++++++++
 5 files changed, 344 insertions(+), 1 deletion(-)
 create mode 100644 mm/shmem_quota.c

fs/Kconfig +12 -0

index e99830c650336..ba809882f1961 100644
@@ -233,6 +233,18 @@ config TMPFS_INODE64
 
 	  If unsure, say N.
 
+config TMPFS_QUOTA
+	bool "Tmpfs quota support"
+	depends on TMPFS
+	select QUOTA
+	help
+	  Quota support allows to set per user and group limits for tmpfs
+	  usage.  Say Y to enable quota support. Once enabled you can control
+	  user and group quota enforcement with quota, usrquota and grpquota
+	  mount options.
+
+	  If unsure, say N.
+
 config ARCH_SUPPORTS_HUGETLBFS
 	def_bool n

include/linux/shmem_fs.h +12 -0

index 103d1000a5a2e..92db49ebd5452 100644
@@ -12,6 +12,10 @@
 
 /* inode in-kernel data */
 
+#ifdef CONFIG_TMPFS_QUOTA
+#define SHMEM_MAXQUOTAS 2
+#endif
+
 struct shmem_inode_info {
 	spinlock_t		lock;
 	unsigned int		seals;		/* shmem seals */

@@ -163,4 +167,12 @@ extern int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd,
 #endif /* CONFIG_SHMEM */
 #endif /* CONFIG_USERFAULTFD */
 
+/*
+ * Used space is stored as unsigned 64-bit value in bytes but
+ * quota core supports only signed 64-bit values so use that
+ * as a limit
+ */
+#define SHMEM_QUOTA_MAX_SPC_LIMIT 0x7fffffffffffffffLL /* 2^63-1 */
+#define SHMEM_QUOTA_MAX_INO_LIMIT 0x7fffffffffffffffLL
+
 #endif

include/uapi/linux/quota.h +1 -0

index f17c9636a8593..52090105b8282 100644
@@ -77,6 +77,7 @@
 #define	QFMT_VFS_V0 2
 #define QFMT_OCFS2 3
 #define	QFMT_VFS_V1 4
+#define	QFMT_SHMEM 5
 
 /* Size of block in which space limits are passed through the quota
  * interface */

mm/Makefile +1 -1

index 8e105e5b3e293..977d2b66775a9 100644
@@ -52,7 +52,7 @@ obj-y			:= filemap.o mempool.o oom_kill.o fadvise.o \
 			   readahead.o swap.o truncate.o vmscan.o shmem.o \
 			   util.o mmzone.o vmstat.o backing-dev.o \
 			   mm_init.o percpu.o slab_common.o \
-			   compaction.o \
+			   compaction.o shmem_quota.o\
 			   interval_tree.o list_lru.o workingset.o \
 			   debug.o gup.o mmap_lock.o $(mmu-y)

mm/shmem_quota.c new file +318 -0

318 lines not on this page · raw message

Signature · 1 line

2.30.2

[email protected]

author

2023-04-26 10:20 UTC #5 lore raw

PATCH v4 5/6 shmem: quota support

In reply to [email protected]

From: Lukas Czerner <[email protected]>

Now the basic infra-structure is in place, enable quota support for tmpfs.

This offers user and group quotas to tmpfs (project quotas will be added later). Also, as other filesystems, the tmpfs quota is not supported within user namespaces yet, so idmapping is not translated.

Signed-off-by Lukas Czerner <[email protected]>

Signed-off-by Carlos Maiolino <[email protected]>

Reviewed-by Jan Kara <[email protected]>

---
 Documentation/filesystems/tmpfs.rst |  15 +++
 include/linux/shmem_fs.h            |   8 ++
 mm/shmem.c                          | 180 ++++++++++++++++++++++++++--
 3 files changed, 195 insertions(+), 8 deletions(-)

Documentation/filesystems/tmpfs.rst +15 -0

index 0408c245785e3..595e607274afb 100644
@@ -86,6 +86,21 @@ use up all the memory on the machine; but enhances the scalability of
 that instance in a system with many CPUs making intensive use of it.
 
 
+tmpfs also supports quota with the following mount options
+
+========  =============================================================
+quota     User and group quota accounting and enforcement is enabled on
+          the mount. Tmpfs is using hidden system quota files that are
+          initialized on mount.
+usrquota  User quota accounting and enforcement is enabled on the
+          mount.
+grpquota  Group quota accounting and enforcement is enabled on the
+          mount.
+========  =============================================================
+
+Note that tmpfs quotas do not support user namespaces so no uid/gid
+translation is done if quotas are enabled inside user namespaces.
+
 tmpfs has a mount option to set the NUMA memory allocation policy for
 all files in that instance (if CONFIG_NUMA is enabled) - which can be
 adjusted on the fly via 'mount -o remount ...'

include/linux/shmem_fs.h +8 -0

index 92db49ebd5452..b8e421e349868 100644
@@ -30,6 +30,9 @@ struct shmem_inode_info {
 	atomic_t		stop_eviction;	/* hold when working on inode */
 	struct timespec64	i_crtime;	/* file creation time */
 	unsigned int		fsflags;	/* flags for FS_IOC_[SG]ETFLAGS */
+#ifdef CONFIG_TMPFS_QUOTA
+	struct dquot		*i_dquot[MAXQUOTAS];
+#endif
 	struct inode		vfs_inode;
 };

@@ -175,4 +178,9 @@ extern int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd,
 #define SHMEM_QUOTA_MAX_SPC_LIMIT 0x7fffffffffffffffLL /* 2^63-1 */
 #define SHMEM_QUOTA_MAX_INO_LIMIT 0x7fffffffffffffffLL
 
+#ifdef CONFIG_TMPFS_QUOTA
+extern const struct dquot_operations shmem_quota_operations;
+extern struct quota_format_type shmem_quota_format;
+#endif /* CONFIG_TMPFS_QUOTA */
+
 #endif

mm/shmem.c +172 -8

335 lines not on this page · raw message

Signature · 1 line

2.30.2

[email protected]

author

2023-04-26 10:20 UTC #6 lore raw

PATCH v4 6/6 Add default quota limit mount options

In reply to [email protected]

From: Lukas Czerner <[email protected]>

Allow system administrator to set default global quota limits at tmpfs mount time.

Signed-off-by Lukas Czerner <[email protected]>

Signed-off-by Carlos Maiolino <[email protected]>

Reviewed-by Jan Kara <[email protected]>

---
 Documentation/filesystems/tmpfs.rst | 34 +++++++++++-----
 include/linux/shmem_fs.h            |  8 ++++
 mm/shmem.c                          | 61 +++++++++++++++++++++++++++++
 mm/shmem_quota.c                    | 34 +++++++++++++++-
 4 files changed, 127 insertions(+), 10 deletions(-)

Documentation/filesystems/tmpfs.rst +25 -9

index 595e607274afb..0c7bfba34baa6 100644
@@ -88,15 +88,31 @@ that instance in a system with many CPUs making intensive use of it.
 
 tmpfs also supports quota with the following mount options
 
-========  =============================================================
-quota     User and group quota accounting and enforcement is enabled on
-          the mount. Tmpfs is using hidden system quota files that are
-          initialized on mount.
-usrquota  User quota accounting and enforcement is enabled on the
-          mount.
-grpquota  Group quota accounting and enforcement is enabled on the
-          mount.
-========  =============================================================
+======================== =================================================
+quota                    User and group quota accounting and enforcement
+                         is enabled on the mount. Tmpfs is using hidden
+                         system quota files that are initialized on mount.
+usrquota                 User quota accounting and enforcement is enabled
+                         on the mount.
+grpquota                 Group quota accounting and enforcement is enabled
+                         on the mount.
+usrquota_block_hardlimit Set global user quota block hard limit.
+usrquota_inode_hardlimit Set global user quota inode hard limit.
+grpquota_block_hardlimit Set global group quota block hard limit.
+grpquota_inode_hardlimit Set global group quota inode hard limit.
+======================== =================================================
+
+None of the quota related mount options can be set or changed on remount.
+
+Quota limit parameters accept a suffix k, m or g for kilo, mega and giga
+and can't be changed on remount. Default global quota limits are taking
+effect for any and all user/group/project except root the first time the
+quota entry for user/group/project id is being accessed - typically the
+first time an inode with a particular id ownership is being created after
+the mount. In other words, instead of the limits being initialized to zero,
+they are initialized with the particular value provided with these mount
+options. The limits can be changed for any user/group id at any time as they
+normally can be.
 
 Note that tmpfs quotas do not support user namespaces so no uid/gid
 translation is done if quotas are enabled inside user namespaces.

include/linux/shmem_fs.h +8 -0

index b8e421e349868..8ca5e969f00fc 100644
@@ -41,6 +41,13 @@ struct shmem_inode_info {
 	(FS_IMMUTABLE_FL | FS_APPEND_FL | FS_NODUMP_FL | FS_NOATIME_FL)
 #define SHMEM_FL_INHERITED		(FS_NODUMP_FL | FS_NOATIME_FL)
 
+struct shmem_quota_limits {
+	qsize_t usrquota_bhardlimit; /* Default user quota block hard limit */
+	qsize_t usrquota_ihardlimit; /* Default user quota inode hard limit */
+	qsize_t grpquota_bhardlimit; /* Default group quota block hard limit */
+	qsize_t grpquota_ihardlimit; /* Default group quota inode hard limit */
+};
+
 struct shmem_sb_info {
 	unsigned long max_blocks;   /* How many blocks are allowed */
 	struct percpu_counter used_blocks;  /* How many are allocated */

@@ -58,6 +65,7 @@ struct shmem_sb_info {
 	spinlock_t shrinklist_lock;   /* Protects shrinklist */
 	struct list_head shrinklist;  /* List of shinkable inodes */
 	unsigned long shrinklist_len; /* Length of shrinklist */
+	struct shmem_quota_limits qlimits; /* Default quota limits */
 };
 
 static inline struct shmem_inode_info *SHMEM_I(struct inode *inode)

mm/shmem.c +61 -0

97 lines not on this page · raw message

mm/shmem_quota.c +33 -1

index c0b531e2ef688..e349c0901bced 100644
@@ -166,6 +166,7 @@ static int shmem_acquire_dquot(struct dquot *dquot)
 {
 	struct mem_dqinfo *info = sb_dqinfo(dquot->dq_sb, dquot->dq_id.type);
 	struct rb_node **n = &((struct rb_root *)info->dqi_priv)->rb_node;
+	struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;
 	struct rb_node *parent = NULL, *new_node = NULL;
 	struct quota_id *new_entry, *entry;
 	qid_t id = from_kqid(&init_user_ns, dquot->dq_id);

@@ -195,6 +196,14 @@ static int shmem_acquire_dquot(struct dquot *dquot)
 	}
 
 	new_entry->id = id;
+	if (dquot->dq_id.type == USRQUOTA) {
+		new_entry->bhardlimit = sbinfo->qlimits.usrquota_bhardlimit;
+		new_entry->ihardlimit = sbinfo->qlimits.usrquota_ihardlimit;
+	} else if (dquot->dq_id.type == GRPQUOTA) {
+		new_entry->bhardlimit = sbinfo->qlimits.grpquota_bhardlimit;
+		new_entry->ihardlimit = sbinfo->qlimits.grpquota_ihardlimit;
+	}
+
 	new_node = &new_entry->node;
 	rb_link_node(new_node, parent, n);
 	rb_insert_color(new_node, (struct rb_root *)info->dqi_priv);

@@ -224,6 +233,29 @@ static int shmem_acquire_dquot(struct dquot *dquot)
 	return ret;
 }
 
+static bool shmem_is_empty_dquot(struct dquot *dquot)
+{
+	struct shmem_sb_info *sbinfo = dquot->dq_sb->s_fs_info;
+	qsize_t bhardlimit;
+	qsize_t ihardlimit;
+
+	if (dquot->dq_id.type == USRQUOTA) {
+		bhardlimit = sbinfo->qlimits.usrquota_bhardlimit;
+		ihardlimit = sbinfo->qlimits.usrquota_ihardlimit;
+	} else if (dquot->dq_id.type == GRPQUOTA) {
+		bhardlimit = sbinfo->qlimits.grpquota_bhardlimit;
+		ihardlimit = sbinfo->qlimits.grpquota_ihardlimit;
+	}
+
+	if (test_bit(DQ_FAKE_B, &dquot->dq_flags) ||
+		(dquot->dq_dqb.dqb_curspace == 0 &&
+		 dquot->dq_dqb.dqb_curinodes == 0 &&
+		 dquot->dq_dqb.dqb_bhardlimit == bhardlimit &&
+		 dquot->dq_dqb.dqb_ihardlimit == ihardlimit))
+		return true;
+
+	return false;
+}
 /*
  * Store limits from dquot in the tree unless it's fake. If it is fake
  * remove the id from the tree since there is no useful information in

@@ -261,7 +293,7 @@ static int shmem_release_dquot(struct dquot *dquot)
 	return -ENOENT;
 
 found:
-	if (test_bit(DQ_FAKE_B, &dquot->dq_flags)) {
+	if (shmem_is_empty_dquot(dquot)) {
 		/* Remove entry from the tree */
 		rb_erase(&entry->node, info->dqi_priv);
 		kfree(entry);

Signature · 1 line

2.30.2

Carlos Maiolino

author

2023-05-15 08:54 UTC #7 lore raw

In reply to [email protected]

Quoted 8 lines · On Wed, Apr 26, 2023 at 12:20:02PM +0200, [email protected] wrote:

From: Carlos Maiolino <[email protected]>
Hello folks.
This is the final version of the quota support from tmpfs, with all the issues addressed, and now including RwB tags on all patches, and should be ready for merge. Details are within each patch, and the original cover-letter below.

Ping? Can somebody manage to pickup these patches?

Thanks!

Quoted 66 lines

Hi folks. this work has been done originally by Lukas, but he left the company, so I'm taking over his work from where he left it of. This series is virtually done, and he had updated it with comments from the last version, but, I'm initially posting it as a RFC because it's been a while since he posted the last version.
Most of what I did here was rebase his last work on top of current Linus's tree.
Honza, there is one patch from you in this series, which I believe you had it suggested to Lukas on a previous version.
The original cover-letter follows...
people have been asking for quota support in tmpfs many times in the past mostly to avoid one malicious user, or misbehaving user/program to consume all of the system memory. This has been partially solved with the size mount option, but some problems still prevail.
One of the problems is the fact that /dev/shm is still generally unprotected with this and another is administration overhead of managing multiple tmpfs mounts and lack of more fine grained control.
Quota support can solve all these problems in a somewhat standard way people are already familiar with from regular file systems. It can give us more fine grained control over how much memory user/groups can consume. Additionally it can also control number of inodes and with special quota mount options introduced with a second patch we can set global limits allowing us to replace the size mount option with quota entirely.
Currently the standard userspace quota tools (quota, xfs_quota) are only using quotactl ioctl which is expecting a block device. I patched quota [1] and xfs_quota [2] to use quotactl_fd in case we want to run the tools on mount point directory to work nicely with tmpfs.
The implementation was tested on patched version of xfstests [3].
[1] https://github.com/lczerner/quota/tree/quotactl_fd_support
[2] https://github.com/lczerner/xfsprogs/tree/quotactl_fd_support
[3] https://github.com/lczerner/xfstests/tree/tmpfs_quota_support
Jan Kara (1):
quota: Check presence of quota operation structures instead of
    ->quota_read and ->quota_write callbacks
Lukas Czerner (5):
shmem: make shmem_inode_acct_block() return error
shmem: make shmem_get_inode() return ERR_PTR instead of NULL
shmem: prepare shmem quota infrastructure
shmem: quota support
Add default quota limit mount options
 Documentation/filesystems/tmpfs.rst |  31 ++
 fs/Kconfig                          |  12 +
 fs/quota/dquot.c                    |   2 +-
 include/linux/shmem_fs.h            |  28 ++
 include/uapi/linux/quota.h          |   1 +
 mm/Makefile                         |   2 +-
 mm/shmem.c                          | 465 +++++++++++++++++++++-------
 mm/shmem_quota.c                    | 350 +++++++++++++++++++++
 8 files changed, 783 insertions(+), 108 deletions(-)
 create mode 100644 mm/shmem_quota.c
Signed-off-by Carlos Maiolino <[email protected]>
--
2.30.2

Signature · 1 line

Carlos Maiolino

Hugh Dickins

maintainer

2023-05-17 10:51 UTC #8 lore raw

Hi Carlos,

Quoted 11 lines · Carlos Maiolino

Quoted 8 lines · On Wed, Apr 26, 2023 at 12:20:02PM +0200, [email protected] wrote:
From: Carlos Maiolino <[email protected]>
Hello folks.
This is the final version of the quota support from tmpfs, with all the issues addressed, and now including RwB tags on all patches, and should be ready for merge. Details are within each patch, and the original cover-letter below.
Ping? Can somebody manage to pickup these patches?

Sorry, but it won't be me picking them up. Let's Cc Andrew Morton, through whose mm tree mm/shmem.c patches usually go. fs-wide updates often go through the fs tree, but this is not really in that category:
it's a specific addition of a feature new to tmpfs; plus I see that Andrew did have plenty of quota involvement back in the day. And let's Cc Christian Brauner, who was raising namespace questions in November.

tmpfs quotas: not a feature that I was ever hoping for, though I might have joked about it twenty years ago. Already we had the nr_blocks= or size= restriction; and the vm_enough_memory strict-non-overcommit restriction; and later the memcg charging restriction. Nowadays, with namespaces (and tmpfs FS_USERNS_MOUNT), I'd have imagined that the natural way to proceed would be to mount a size-limited tmpfs into the namespace of the user to be limited - but I'm no system designer, and likely just boasting my ignorance of namespaces again.

It does look as if Lukas and you have done a good job here:
it's a much lighter "invasion" than I was expecting (and the de-indentation changes looked reasonable to me too, though I didn't check through them). I was puzzled where the i_blocks accounting had vanished, but found it eventually in the dquot stubs.

So, I don't have an actual objection, and it counts for a lot that you have Jan Kara on board; but this is not work that I shall be able to support myself (I'll rarely even turn the CONFIG on). If Andrew thinks it should go in, then you will be needed to maintain it.

IIRC there's an outstanding issue, that namespaces are not properly supported here. Given Christian's work on idmapping, that will be unfortunate; but if it's just that you were waiting for the base series to go in, before correcting the namespace situation, that's understandable and should not delay further - but I hope that you know what more is needed, and that it won't add much more code.

Hugh

Quoted 72 lines

Thanks!
Quoted 66 lines
Hi folks. this work has been done originally by Lukas, but he left the company, so I'm taking over his work from where he left it of. This series is virtually done, and he had updated it with comments from the last version, but, I'm initially posting it as a RFC because it's been a while since he posted the last version.
Most of what I did here was rebase his last work on top of current Linus's tree.
Honza, there is one patch from you in this series, which I believe you had it suggested to Lukas on a previous version.
The original cover-letter follows...
people have been asking for quota support in tmpfs many times in the past mostly to avoid one malicious user, or misbehaving user/program to consume all of the system memory. This has been partially solved with the size mount option, but some problems still prevail.
One of the problems is the fact that /dev/shm is still generally unprotected with this and another is administration overhead of managing multiple tmpfs mounts and lack of more fine grained control.
Quota support can solve all these problems in a somewhat standard way people are already familiar with from regular file systems. It can give us more fine grained control over how much memory user/groups can consume. Additionally it can also control number of inodes and with special quota mount options introduced with a second patch we can set global limits allowing us to replace the size mount option with quota entirely.
Currently the standard userspace quota tools (quota, xfs_quota) are only using quotactl ioctl which is expecting a block device. I patched quota [1] and xfs_quota [2] to use quotactl_fd in case we want to run the tools on mount point directory to work nicely with tmpfs.
The implementation was tested on patched version of xfstests [3].
[1] https://github.com/lczerner/quota/tree/quotactl_fd_support
[2] https://github.com/lczerner/xfsprogs/tree/quotactl_fd_support
[3] https://github.com/lczerner/xfstests/tree/tmpfs_quota_support
Jan Kara (1):
quota: Check presence of quota operation structures instead of
    ->quota_read and ->quota_write callbacks
Lukas Czerner (5):
shmem: make shmem_inode_acct_block() return error
shmem: make shmem_get_inode() return ERR_PTR instead of NULL
shmem: prepare shmem quota infrastructure
shmem: quota support
Add default quota limit mount options
 Documentation/filesystems/tmpfs.rst |  31 ++
 fs/Kconfig                          |  12 +
 fs/quota/dquot.c                    |   2 +-
 include/linux/shmem_fs.h            |  28 ++
 include/uapi/linux/quota.h          |   1 +
 mm/Makefile                         |   2 +-
 mm/shmem.c                          | 465 +++++++++++++++++++++-------
 mm/shmem_quota.c                    | 350 +++++++++++++++++++++
 8 files changed, 783 insertions(+), 108 deletions(-)
 create mode 100644 mm/shmem_quota.c
Signed-off-by Carlos Maiolino <[email protected]>
--
2.30.2
Signature · 1 line
Carlos Maiolino

Carlos Maiolino

author

2023-05-17 12:22 UTC #9 lore raw

Quoted 13 lines · Hugh Dickins

Hi Carlos,
Quoted 10 lines · Carlos Maiolino
Quoted 8 lines · On Wed, Apr 26, 2023 at 12:20:02PM +0200, [email protected] wrote:
From: Carlos Maiolino <[email protected]>
Hello folks.
This is the final version of the quota support from tmpfs, with all the issues addressed, and now including RwB tags on all patches, and should be ready for merge. Details are within each patch, and the original cover-letter below.

Quoted 8 lines

Quoted 1 line
Ping? Can somebody manage to pickup these patches?
Sorry, but it won't be me picking them up. Let's Cc Andrew Morton, through whose mm tree mm/shmem.c patches usually go. fs-wide updates often go through the fs tree, but this is not really in that category:
it's a specific addition of a feature new to tmpfs; plus I see that Andrew did have plenty of quota involvement back in the day. And let's Cc Christian Brauner, who was raising namespace questions in November.

Thanks, I spoke with Christian on the previous thread. I believe we are all set in there by now.

Quoted 16 lines

tmpfs quotas: not a feature that I was ever hoping for, though I might have joked about it twenty years ago. Already we had the nr_blocks= or size= restriction; and the vm_enough_memory strict-non-overcommit restriction; and later the memcg charging restriction. Nowadays, with namespaces (and tmpfs FS_USERNS_MOUNT), I'd have imagined that the natural way to proceed would be to mount a size-limited tmpfs into the namespace of the user to be limited - but I'm no system designer, and likely just boasting my ignorance of namespaces again.
It does look as if Lukas and you have done a good job here:
it's a much lighter "invasion" than I was expecting (and the de-indentation changes looked reasonable to me too, though I didn't check through them). I was puzzled where the i_blocks accounting had vanished, but found it eventually in the dquot stubs.

Quoted 4 lines

So, I don't have an actual objection, and it counts for a lot that you have Jan Kara on board; but this is not work that I shall be able to support myself (I'll rarely even turn the CONFIG on). If Andrew thinks it should go in, then you will be needed to maintain it.

Sounds reasonable.

Quoted 7 lines

IIRC there's an outstanding issue, that namespaces are not properly supported here. Given Christian's work on idmapping, that will be unfortunate; but if it's just that you were waiting for the base series to go in, before correcting the namespace situation, that's understandable and should not delay further - but I hope that you know what more is needed, and that it won't add much more code.

Yup, I didn't want to add more features here by now to avoid further delays to have the basic series merged, both things I plan to implement here by now are userns support and prjquotas. Both AFAIK won't really need too much extra code, but will add more overhead on people to review, so having them as separated series and not part of the basic quota implementation, should ease the burden a bit.

Quoted 75 lines

Hugh
Quoted 72 lines
Thanks!
Quoted 62 lines
Hi folks. this work has been done originally by Lukas, but he left the company, so I'm taking over his work from where he left it of. This series is virtually done, and he had updated it with comments from the last version, but, I'm initially posting it as a RFC because it's been a while since he posted the last version.
Most of what I did here was rebase his last work on top of current Linus's tree.
Honza, there is one patch from you in this series, which I believe you had it suggested to Lukas on a previous version.
The original cover-letter follows...
people have been asking for quota support in tmpfs many times in the past mostly to avoid one malicious user, or misbehaving user/program to consume all of the system memory. This has been partially solved with the size mount option, but some problems still prevail.
One of the problems is the fact that /dev/shm is still generally unprotected with this and another is administration overhead of managing multiple tmpfs mounts and lack of more fine grained control.
Quota support can solve all these problems in a somewhat standard way people are already familiar with from regular file systems. It can give us more fine grained control over how much memory user/groups can consume. Additionally it can also control number of inodes and with special quota mount options introduced with a second patch we can set global limits allowing us to replace the size mount option with quota entirely.
Currently the standard userspace quota tools (quota, xfs_quota) are only using quotactl ioctl which is expecting a block device. I patched quota [1] and xfs_quota [2] to use quotactl_fd in case we want to run the tools on mount point directory to work nicely with tmpfs.
The implementation was tested on patched version of xfstests [3].
[1] https://github.com/lczerner/quota/tree/quotactl_fd_support
[2] https://github.com/lczerner/xfsprogs/tree/quotactl_fd_support
[3] https://github.com/lczerner/xfstests/tree/tmpfs_quota_support
Jan Kara (1):
quota: Check presence of quota operation structures instead of
    ->quota_read and ->quota_write callbacks
Lukas Czerner (5):
shmem: make shmem_inode_acct_block() return error
shmem: make shmem_get_inode() return ERR_PTR instead of NULL
shmem: prepare shmem quota infrastructure
shmem: quota support
Add default quota limit mount options
 Documentation/filesystems/tmpfs.rst |  31 ++
 fs/Kconfig                          |  12 +
 fs/quota/dquot.c                    |   2 +-
 include/linux/shmem_fs.h            |  28 ++
 include/uapi/linux/quota.h          |   1 +
 mm/Makefile                         |   2 +-
 mm/shmem.c                          | 465 +++++++++++++++++++++-------
 mm/shmem_quota.c                    | 350 +++++++++++++++++++++
 8 files changed, 783 insertions(+), 108 deletions(-)
 create mode 100644 mm/shmem_quota.c
Quoted 4 lines
Signed-off-by Carlos Maiolino <[email protected]>
--
2.30.2
--
Carlos Maiolino

Signature · 1 line

Carlos Maiolino