Skip to contentSkip to conversation

making an in-memory hashing table ["name" -> ino_t] with thousands of entries

DISCUSSION Luke Kenneth Casson Leighton 2 participants 3 messages Oct 1, 2004

lore
Share & tools
lore
#0 lore raw

dear kernel hackers,

i seek advice on how to do an in-kernel lookup table capable of storing thousands, potentially hundreds of thousands, of entries.

the reason for this is to move the fuse userspace inode lookup tables into the kernel.

fuse userspace servers have their own in-memory database of unique inode numbers which represent the file names, and there is a communication mechanism between userspace and kernelspace that transfers those inode numbers, amongst other things.

is there any _sane_ way to do this or should i leave the inode lookup table where it presently is - in userspace?

bearing in mind that for every file accessed via a fuse filesystem, a cache entry is created, and therefore the number of entries could potentially run into hundreds of thousands of entries.

l.

Signature · 7 lines
--
Truth, honesty and respect are rare commodities that all spring from
the same well: Love.  If you love yourself and everyone and everything
around you, funnily and coincidentally enough, life gets a lot better.
--
<a href="http://lkcl.net">      lkcl.net      </a> <br />
<a href="mailto:[email protected]"> [email protected] </a> <br />
#1 lore raw

hi :)

Quoted 4 lines · Luke Kenneth Casson Leighton

bearing in mind that for every file accessed via a fuse filesystem, a cache entry is created, and therefore the number of entries could potentially run into hundreds of thousands of entries.

but these can be cleaned if needed.

why do you have to move all inodes into the kernel when the kernel already cache those that are really needed?
(perhaps I'm misunderstandig fuses role here)

Signature · 1 line
Martin Waitz
#2 lore raw
Quoted 1 line · Martin Waitz

hi :)

hello martin: thanks for responding.

Quoted 11 lines · Luke Kenneth Casson Leighton
Quoted 4 lines

bearing in mind that for every file accessed via a fuse filesystem, a cache entry is created, and therefore the number of entries could potentially run into hundreds of thousands of entries.

but these can be cleaned if needed.

why do you have to move all inodes into the kernel when the kernel already cache those that are really needed?
(perhaps I'm misunderstandig fuses role here)

fuse is a userspace filesystem.

it can be absolutely anything.

however, the author of fuse, in order to simplify the userspace interface, manages the userspace-inode <-> userspace-fullpathnames in a library.

for the sake of argument, let's call this the fuse-inode-cache.

entries in the fuse-inode-cache must be persistent for as long as a filename exists on the userspace file system.

also, if an entry happens not to exist at the time a lookup is done, then an entry into the cache is created, associating that full path name with a newly created unique inode number (in the userspace fuse-inode-cache).

then, that information (the unique inode number) is communicated _back_ to the fuse module, along with the stat information (yes, a userspace lstat is done as well), and the kernel's inode structure is updated from that information.

anyway: for various reasons, inside the kernel, it must be possible to do a lookup from the full path name to the inode number.

smbfs suffers from exactly the same problem: inodes don't exist on the SMB protocol.

hm, i wonder if i should just rip stacks of code from the smbfs kernel module?

i note with interest the existence of a function smb_refresh_inode() which gets passed a dentry.

then i can do a getattr and go from there.

oh, man, this is hair-raising stuff.

l.

Signature · 7 lines
--
Truth, honesty and respect are rare commodities that all spring from
the same well: Love.  If you love yourself and everyone and everything
around you, funnily and coincidentally enough, life gets a lot better.
--
<a href="http://lkcl.net">      lkcl.net      </a> <br />
<a href="mailto:[email protected]"> [email protected] </a> <br />