Skip to contentSkip to conversation

eifs/fmfs (everything is filesystem/file mapping filesystem)

DISCUSSION mobi phil 2 participants 4 messages Nov 9, 2009 – Nov 10, 2009 2 messages not in the archive VFS

lore
Share & tools
lore

mobi phil

author
#0 lore raw

Hello mailing list members,

I am not sure about the best name, but I am sure it is sthg. very usefull. So I will call it mobifs, hope you do not mind :)
I am also not sure if there is sthg. like that out there, but based on my 15 years
of linux experience I never met sthg. like what I would like to talk about.

What I would like to have on my linux boxes is to access data the following way:

#>cat /dev/sda1@/partition1/etc/inittab@/defaultranlevel #usecase 0

#>cat /data/xmlfiles/apsettings.xml@xml/main/entry1/subentry/value #usecase 1 or
#>cat /data/xmlfiles/apsettings.xml@/main/entry1/subentry/value #note @ instead of @xml #usecase 2
or
#>cat /data/images/linux.img/@/partition1/etc/dhcpd/dhcpd.conf@group/next-server
#note twice the @ #usecase 3

I think you got the point. It is actually not a real filesystem, but an extension to the virtual filesystem.

How would it work? If the virtual filesystem subsystem of the kernel finds an @ it would redirect the call
to the mobifs main module. The mobifs will check if there is any string after @ till the next /. If yes,
it will look if there is any driver installed with that name. In usecase 1, it will look if there is a
driver to read xml files. If yes, it will call the driver and pass the rest of the string to it. The xml driver
will interpret the path request and output the necessary entry.

If there is nothing specified like in usecase0 (note annoying tmp. mount not needed!!!)mobifs will try to guess, and
if he finds a matching filetype (like the file command or mount does) it would pass the path to
the driver if there is any driver registered for that.

Drivers can be partition readers (do not know oficial technical name :)), existing filesystem drivers etc.
It should be possible to have kerenel drivers and userspace drivers. It seems rather logical
that for reading filesystems one would use kernelspace drivers, and for trivial file mapping operations
userspace program would fit better.

Usecas0 has twice @. This should mean double "redirection" or piping. Mobifs should help to feed the output
of one driver with the input of next one, etc. In case of usecase 0, the one driver would read the filesystem
on sda1, and feed the read of the driver that is able to read inittab files.

If sthg. like that would become to being, I would see its evolution as 2 main steps: first create
read only support, and then add write support. The same would happen with the drivers...

I was thinking about this the last few days, started to hack around the kernel, but before going
to the wrong direction, I wanted to ask more experienced people about what and what not to do!

for some more imagined usecases, see the PS.

rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com

P.S.

#>cat /databases/mysqldatabase/db.sql@mysql/tables/table1/2345/rowname #>cat /databases/mysqldatabase/db.sql@mysql/tables/table1/rowname/1232 #>cat /databases/mysqldatabase/db.sql@mysql/tables/table1/rowname

#>cat /xxx/xx/xx/xx/elfimage@elf/symbols

#>cat /sourcecode/cpp/thismodule/thatfile.h@cpp/methodist #>cat /sourcecode/cpp/thismodule/thatfile.h@cpp/includelist #>cat /sourcecode/cpp/thismodule/thatfile.cpp@cpp/methods/thatmethod

#>cat /sourcecode/cpp/thismodule/thatfile.h@cpp/methodist

#>tail -f /var/log/messages@grep/error #maybe this is less usefull

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#3 lore raw

Hello,

I was writing the following email to the linux-fsdevel, but as it might involve other parts of the kernel, I would share it on this ML as well. Your feedback is more than welcome.

---------- Forwarded message ----------
From: mobi phil <[email protected]>
Date: Mon, Nov 9, 2009 at 8:28 PM
Subject: eifs/fmfs (everything is filesystem/file mapping filesystem)
To: [email protected]

Hello mailing list members,

I am not sure about the best name, but I am sure it is sthg. very usefull. So I will call it mobifs, hope you do not mind :) I am also not sure if there is sthg. like that out there, but based on my 15 years of linux experience I never met sthg. like what I would like to talk about.

What I would like to have on my linux boxes is to access data the following way:

#>cat /dev/sda1@/partition1/etc/inittab@/defaultranlevel #usecase 0

#>cat /data/xmlfiles/apsettings.xml@xml/main/entry1/subentry/value #usecase 1 or
#>cat /data/xmlfiles/apsettings.xml@/main/entry1/subentry/value #note @ instead of  @xml #usecase 2
or
#>cat /data/images/linux.img/@/partition1/etc/dhcpd/dhcpd.conf@group/next-server #note twice the @  #usecase 3

I think you got the point. It is actually not a real filesystem, but an extension to the virtual filesystem.

How would it work? If the virtual filesystem subsystem of the kernel finds an @ it would redirect the call to the mobifs main module. The mobifs will check if there is any string after @ till the next /. If yes, it will look if there is any driver installed with that name. In usecase 1, it will look if there is a driver to read xml files. If yes, it will call the driver and pass the rest of the string to it. The xml driver will interpret the path request and output the necessary entry.

If there is nothing specified like in usecase0 (note annoying tmp. mount not needed!!!)mobifs will try to guess, and if he finds a matching filetype (like the file command or mount does) it would pass the path to  the driver if there is any driver registered for that.

Drivers can be partition readers (do not know oficial technical name :)), existing filesystem drivers etc. It should be possible to have kerenel drivers and userspace drivers. It seems rather logical that for reading filesystems one would use kernelspace drivers, and for trivial file mapping operations userspace program would fit better.

Usecas0 has twice @. This should mean double "redirection" or piping. Mobifs should help to feed the output of one driver with the input of next one, etc. In case of usecase 0, the one driver would read the filesystem on sda1, and feed the read of the driver that is able to read inittab files.

If sthg. like that would become to being, I would see its evolution as 2 main steps: first create read only support, and then add write support. The same would happen with the drivers...

I was thinking about this the last few days, started to hack around the kernel, but before going to the wrong direction, I wanted to ask more experienced people about what and what not to do!

for some more imagined usecases, see the PS.

rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com

P.S.

#>cat /databases/mysqldatabase/db.sql@mysql/tables/table1/2345/rowname #>cat /databases/mysqldatabase/db.sql@mysql/tables/table1/rowname/1232 #>cat /databases/mysqldatabase/db.sql@mysql/tables/table1/rowname

#>cat /xxx/xx/xx/xx/elfimage@elf/symbols

#>cat /sourcecode/cpp/thismodule/thatfile.h@cpp/methodist #>cat /sourcecode/cpp/thismodule/thatfile.h@cpp/includelist #>cat /sourcecode/cpp/thismodule/thatfile.cpp@cpp/methods/thatmethod

#>cat /sourcecode/cpp/thismodule/thatfile.h@cpp/methodist

#>tail -f /var/log/messages@grep/error #maybe this is less usefull

--
rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com

--
rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com

Signature · 5 lines
rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com
#4 lore raw
Quoted 10 lines

I am not sure about the best name, but I am sure it is sthg. very usefull. So I will call it mobifs, hope you do not mind :) I am also not sure if there is sthg. like that out there, but based on my 15 years of linux experience I never met sthg. like what I would like to talk about.

What I would like to have on my linux boxes is to access data the following way:

#>cat /dev/sda1@/partition1/etc/inittab@/defaultranlevel #usecase 0

google fuse

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
author
#5 lore raw
Quoted 1 line

google fuse

Sometimes manipulation works :) .. I did intentionally not mention fuse, so that we can focus on the topic when somebody will ask :) I know fuse, it is a treasure, am using fuse for db, wikipedia etc., etc. However the only annoying thing with fuse and kernel space drivers is that you have to mount before access. (And mount is not enough, like with loop device, you cannot mount straightforward partitions on a loop device) you have to tell the computer extra annoying instruction, that should be obvious, once setting up the framework. That is fuse is a partial solution for a powerful concept that I named "filesystem anywhere". I do not want to bother mounting fuse when I want to access a file that has a hierarchical structure, and the guy knowing to do the mapping from that internal structure to hierarchical filesystem is in place. Similar ideas were already discussed here I am sure (I was not on the kernel mailing lists that time), as some people (I would like to avoid names) were insisting on seeing data on a computer throug sthg. called unified path system if I am not mistaking.

by the way another example:

#>ls /mydata/ftpadresses/boot.kernel.org@http/gpxe_images #>ls /mygitprojects/linux-2.6/.git@git/tags
#>ls /mygitprojects/linux-2.6/.git@git/branches or with defaults
#>ls /mygitprojects/linux-2.6/.git@/tags
#>ls /mygitprojects/linux-2.6/.git@/branches

etc

Signature · 5 lines
rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com