Skip to contentSkip to conversation

fs/fcntl: add fcntl F_GET_RSS

RFC PATCH In review Konstantin Khlebnikov 4 participants 8 messages Oct 28, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 fs/fcntl: add fcntl F_GET_RSS

This implements fcntl() for getting amount of resident memory in cache. Kernel already maintains counter for each inode, this patch just exposes it into userspace. Returned size is in kilobytes like values in procfs.

Alternatively this could be implemented via mapping file and collecting map of cached pages with mincore(). Which is much slower and O(n*log n).

Syscall fincore() never was implemented in Linux.
This fcntl() covers one of its use-cases with minimal footprint.

Unlike to mincore() this fcntl counts all pages, including allocated but not read yet (non-uptodate) and pages beyond end of file.

This employs same security model as mincore() and requires one of:
- file is opened for writing
- current user owns inode
- current user could open inode for writing

Usage:
resident_kb = fcntl(fd, F_GET_RSS);

Error codes:
-EINVAL - not supported
-EPERM - not writable / owner
-ENODATA - special inode without cache

Notes:
Range of pages could be evicted from cache using POSIX_FADV_DONTNEED. Populating with POSIX_FADV_WILLNEED is asynchronous and limited with disk read_ahead_kb and max_sectors_kb. It seems most effective way to read data into cache synchronously is a sendfile() into /dev/null.

---
 fs/fcntl.c                 |   30 ++++++++++++++++++++++++++++++
 include/uapi/linux/fcntl.h |    5 +++++
 2 files changed, 35 insertions(+)
fs/fcntl.c +30 -0
index 3d40771e8e7c..b241d3c925db 100644
@@ -25,6 +25,8 @@
 #include <linux/user_namespace.h>
 #include <linux/memfd.h>
 #include <linux/compat.h>
+#include <linux/dax.h>
+#include <linux/hugetlb.h>
 
 #include <linux/poll.h>
 #include <asm/siginfo.h>
@@ -319,6 +321,31 @@ static long fcntl_rw_hint(struct file *file, unsigned int cmd,
 	}
 }
 
+static long fcntl_get_rss(struct file *filp)
+{
+	struct address_space *mapping = filp->f_mapping;
+	unsigned long pages;
+
+	if (!mapping)
+		return -ENODATA;
+
+	/* The same limitations as for sys_mincore() */
+	if (!(filp->f_mode & FMODE_WRITE) &&
+	    !inode_owner_or_capable(mapping->host) &&
+	    inode_permission(mapping->host, MAY_WRITE))
+		return -EPERM;
+
+	if (dax_mapping(mapping))
+		pages = READ_ONCE(mapping->nrexceptional);
+	else
+		pages = READ_ONCE(mapping->nrpages);
+
+	if (is_file_hugepages(filp))
+		pages <<= huge_page_order(hstate_file(filp));
+
+	return pages << (PAGE_SHIFT - 10);	/* page -> kb */
+}
+
 static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,
 		struct file *filp)
 {
@@ -426,6 +453,9 @@ static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,
 	case F_SET_FILE_RW_HINT:
 		err = fcntl_rw_hint(filp, cmd, arg);
 		break;
+	case F_GET_RSS:
+		err = fcntl_get_rss(filp);
+		break;
 	default:
 		break;
 	}
include/uapi/linux/fcntl.h +5 -0
index 1d338357df8a..d467f1dbfc67 100644
@@ -54,6 +54,11 @@
 #define F_GET_FILE_RW_HINT	(F_LINUX_SPECIFIC_BASE + 13)
 #define F_SET_FILE_RW_HINT	(F_LINUX_SPECIFIC_BASE + 14)
 
+/*
+ * Get amount of resident memory in file cache in kilobytes.
+ */
+#define F_GET_RSS		(F_LINUX_SPECIFIC_BASE + 15)
+
 /*
  * Valid hint values for F_{GET,SET}_RW_HINT. 0 is "not set", or can be
  * used to clear any hints previously set.
#1 lore raw

fs/fcntl.c static long fcntl_rw_hint(struct file *file, unsigned int cmd, · line 341

Quoted 4 lines · Konstantin Khlebnikov
fs/fcntl.c +4 -0
+	if (dax_mapping(mapping))
+		pages = READ_ONCE(mapping->nrexceptional);
+	else
+		pages = READ_ONCE(mapping->nrpages);

I'm not sure this is the right calculation for DAX files. We haven't allocated any memory for DAX; we're just accessing storage directly. The entries in the page caache are just translation from file offset to physical address.

#2 lore raw
Quoted 11 lines · Matthew Wilcox

fs/fcntl.c static long fcntl_rw_hint(struct file *file, unsigned int cmd, · line 341

Quoted 4 lines · Konstantin Khlebnikov
fs/fcntl.c +4 -0
+	if (dax_mapping(mapping))
+		pages = READ_ONCE(mapping->nrexceptional);
+	else
+		pages = READ_ONCE(mapping->nrpages);

I'm not sure this is the right calculation for DAX files. We haven't allocated any memory for DAX; we're just accessing storage directly. The entries in the page caache are just translation from file offset to physical address.

Yep, makes sense. If RSS declared as memory usage then this chunk must do pages = READ_ONCE(mapping->nrpages) unconditionally and report 0 for DAX.

#3 lore raw
In reply to Konstantin Khlebnikov

* Konstantin Khlebnikov:

Quoted 3 lines

This implements fcntl() for getting amount of resident memory in cache. Kernel already maintains counter for each inode, this patch just exposes it into userspace. Returned size is in kilobytes like values in procfs.

I think this needs a 32-bit compat implementation which clamps the returned value to INT_MAX.

#4 lore raw
Quoted 9 lines · Florian Weimer

* Konstantin Khlebnikov:

Quoted 3 lines

This implements fcntl() for getting amount of resident memory in cache. Kernel already maintains counter for each inode, this patch just exposes it into userspace. Returned size is in kilobytes like values in procfs.

I think this needs a 32-bit compat implementation which clamps the returned value to INT_MAX.

32-bit machine couldn't hold more than 2TB cache in one file. Even radix tree wouldn't fit into low memory area.

#5 lore raw

* Konstantin Khlebnikov:

Quoted 7 lines · Florian Weimer
Quoted 6 lines

* Konstantin Khlebnikov:

Quoted 3 lines

This implements fcntl() for getting amount of resident memory in cache. Kernel already maintains counter for each inode, this patch just exposes it into userspace. Returned size is in kilobytes like values in procfs.

Quoted 6 lines
Quoted 3 lines

I think this needs a 32-bit compat implementation which clamps the returned value to INT_MAX.

32-bit machine couldn't hold more than 2TB cache in one file. Even radix tree wouldn't fit into low memory area.

I meant a 32-bit process running on a 64-bit kernel.

#6 lore raw
In reply to Konstantin Khlebnikov
Quoted 4 lines · Konstantin Khlebnikov

This implements fcntl() for getting amount of resident memory in cache. Kernel already maintains counter for each inode, this patch just exposes it into userspace. Returned size is in kilobytes like values in procfs.

This doesn't actually explain why anybody would want it, and what the usage scenario is.

Linus

#7 lore raw
Quoted 10 lines · Linus Torvalds
Quoted 4 lines · Konstantin Khlebnikov

This implements fcntl() for getting amount of resident memory in cache. Kernel already maintains counter for each inode, this patch just exposes it into userspace. Returned size is in kilobytes like values in procfs.

This doesn't actually explain why anybody would want it, and what the usage scenario is.

This really helps to plot memory usage distribution. Right now file cache have only total counters. Collecting statistics via mincore as implemented in page-types tool isn't efficient and very racy.

Usage scenario is the same as finding top memory usage among processes. But among files which are not always mapped anywhere.

For example if somebody writes\reads logs too intensive this file cache could bloat and push more important data out out memory.

Also little bit of introspection wouldn't hurt. Using this I've found unneeded pages beyond i_size.

Quoted 2 lines

Linus