Skip to contentSkip to conversation

pin files in memory after read

DISCUSSION Olaf Hering 3 participants 4 messages Jan 3, 2005 – Jan 4, 2005 VFS

lore
Share & tools
lore
#0 lore raw

Is there a way to always keep a file (once read from disk) in memory, no matter how much memory pressure exists?
There are always complains that updatedb and similar tools wipe out all caches. So I guess there is no such thing yet.

I simply want to avoid the spinup of my ibook harddisk when something has been 'forgotten' and must be loaded again (like opening a new screen window after a while).

The best I could do so far was a cramfs image. I copied it to tmpfs during early boot, then mount -o bind every cramfs file over the real binary on disk. Of course that will fail as soon as I want to update an affected package because the binary is busy (readonly). So there must be a better way to achieve this.

How can one tell the kernel to pin a file in memory once it was read? Maybe with an xattr or something?
Unfortunately I dont know about the block layer and other things involved, so I cant attach a patch that does what I want.

#1 lore raw
Quoted 15 lines · Olaf Hering

Is there a way to always keep a file (once read from disk) in memory, no matter how much memory pressure exists?
There are always complains that updatedb and similar tools wipe out all caches. So I guess there is no such thing yet.

I simply want to avoid the spinup of my ibook harddisk when something has been 'forgotten' and must be loaded again (like opening a new screen window after a while).

The best I could do so far was a cramfs image. I copied it to tmpfs during early boot, then mount -o bind every cramfs file over the real binary on disk. Of course that will fail as soon as I want to update an affected package because the binary is busy (readonly). So there must be a better way to achieve this.

Quoted 4 lines

How can one tell the kernel to pin a file in memory once it was read? Maybe with an xattr or something?
Unfortunately I dont know about the block layer and other things involved, so I cant attach a patch that does what I want.

you could write a small userspace daemon that mmaps the file and mlock's it....

#2 lore raw
Quoted 2 lines · Arjan van de Ven

you could write a small userspace daemon that mmaps the file and mlock's it....

Thanks.
It seems to work ok with this thing. I used this patch to find the files with an absolute path. Any idea how to get to the relative path like "./x" and print an absolute path for these files?

fs/open.c 2005-01-04 00:48:30.000000000 +0100 renamed from linux-2.6.10.orig/fs/open.c 2004-12-31 09:29:25.000000000 +0100 +11 -0
@@ -961,6 +961,17 @@ asmlinkage long sys_open(const char __us
 out:
                putname(tmp);
        }
+       if (0 && fd >= 0) {
+               if (filename[0] == '/' && filename[1] != '\0' && !(
+                                       !memcmp(filename,"/home/olaf/Mail",15) ||
+                                       !memcmp(filename,"/events",7) ||
+                                       !memcmp(filename,"/proc",5) ||
+                                       !memcmp(filename,"/sys",4) ||
+                                       !memcmp(filename,"/dev",4) ||
+                                       !memcmp(filename,"/var",4)
+                                       ))
+                       printk("OP%s %s\n",current->comm, filename);
+       }
        return fd;
 
 out_error:

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

#define file_list "/home/olaf/x"

size_t total;
void map(unsigned char *file, struct stat *sb)

{
	int fd;
	register unsigned char *p, c;
	if (stat(file, sb) < 0)
		return;
	fd = open(file, O_RDONLY);
	if (fd < 0)
		return;
	p = mmap(NULL, sb->st_size, PROT_READ, MAP_SHARED | MAP_LOCKED, fd, 0);
	if (p != MAP_FAILED && (total += sb->st_size))
		while (sb->st_size)
			c = p[sb->st_size--];
	close(fd);
	return;
}

int main(int argc, char *argv[])

{
	struct stat sb;
	int fd, ret, line_count;
	size_t len;
	off_t fs;
	void *flp;
	unsigned char *p1, *p2;
	fd = open(file_list, O_RDONLY);
	if (fd < 0) {
		perror(file_list);
		ret = fd;
		goto out;
	}
	ret = stat(file_list, &sb);
	if (ret < 0)
		goto out;
	flp = mmap(NULL, sb.st_size, PROT_READ, MAP_SHARED | MAP_LOCKED, fd, 0);
	if (flp == MAP_FAILED) {
		perror("mmap");
		goto out;
	}
	total += sb.st_size;
	p1 = flp;
	line_count = 0;
	fs = sb.st_size;
	while (fs > 0) {
		line_count++;
		printf("line %d ", line_count);
		len = 0;
		while (1) {
			//              printf("len %d\n", len);
			fs--;
			if (p1[len] != '\n') {
				len++;
				continue;
			}
			p2 = malloc(len);
			if (p2) {
				memcpy(p2, p1, len);
				p2[len] = '\0';
				printf("%u %s\n", len, p2);
				map(p2, &sb);
				free(p2);
				p1 = p1 + len + 1;
			}
			break;
		}
	}
	printf("sleeping ... %u\n", total);
	while (1)sleep(123456789);
      out:
	return ret;
}
#3 lore raw
Quoted 2 lines · * Olaf Hering ([email protected]) wrote:

with an absolute path. Any idea how to get to the relative path like "./x" and print an absolute path for these files?

d_path() will give that to you, once you've resolved the path to dentry and vfsmount pair.

thanks,
-chris

Signature · 1 line
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net